2024-07-26T13:17:10.929 INFO:root:teuthology version: 0.0.1.dev379+g53ce146 2024-07-26T13:17:10.955 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-07-26T13:17:11.272 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-07-26_04:18:59-fs-wip-vshankar-testing-20240725.121331-debug-testing-default-smithi/7818864 branch: wip-vshankar-testing-20240725.121331-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} email: vshankar@redhat.com first_in_suite: false job_id: '7818864' kernel: client: sha1: distro flavor: default kdb: 1 sha1: 3e9255e907100f88049f320ca33d977e29155d12 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: vshankar-2024-07-26_04:18:59-fs-wip-vshankar-testing-20240725.121331-debug-testing-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20240725.121331-debug ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug cephsqlite: 20 mgr/snap_schedule/allow_m_granularity: true mgr/snap_schedule/dump_on_update: true mgr/snap_schedule/log_level: debug mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request mgr-modules: - snap_schedule sha1: e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240725.121331-debug sha1: e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 owner: scheduled_vshankar@teuthology priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 5532 sha1: e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 sleep_before_teardown: 0 subset: 1001/1024 suite: fs suite_branch: wip-vshankar-testing-20240725.121331-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi100.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAd43iYZVVxXI/hj5Mc1xAg9O5VOc1PAY1sjLeDIbM+gN+s2ECcT/KVUxAasUwADsYB1598s3PcY7O3LluU/jlc= smithi138.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO2ijrZL8gKZlJHEamiwY/D3ZbW8oU91YnbruYYZqSMAIW32zg+7i3kMJF8FntdASx6xCwdmtT6rDoeULLiWlwk= smithi167.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGnUCUxfb+tWOQtWviJUANX7OWEFnzGWvDYLzBY2w5hufagx2DKmg51LokSt2GKxkg1zK3eIuecUYatUTAk/85A= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 53ce1462e129f6eb4071986336534c740fdebd31 timestamp: 2024-07-26_04:18:59 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.4074525 2024-07-26T13:17:11.272 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528/qa; will attempt to use it 2024-07-26T13:17:11.273 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528/qa/tasks 2024-07-26T13:17:11.273 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-07-26T13:17:11.274 INFO:teuthology.task.internal:Checking packages... 2024-07-26T13:17:11.296 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528' 2024-07-26T13:17:11.296 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-07-26T13:17:11.296 INFO:teuthology.packaging:ref: None 2024-07-26T13:17:11.296 INFO:teuthology.packaging:tag: None 2024-07-26T13:17:11.296 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240725.121331-debug 2024-07-26T13:17:11.296 INFO:teuthology.packaging:sha1: e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 2024-07-26T13:17:11.296 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20240725.121331-debug 2024-07-26T13:17:11.447 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-5320.ge8f88bb1 2024-07-26T13:17:11.449 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-07-26T13:17:11.457 INFO:teuthology.task.internal:no buildpackages task found 2024-07-26T13:17:11.457 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-07-26T13:17:11.464 INFO:teuthology.task.internal:Saving configuration 2024-07-26T13:17:11.491 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-07-26T13:17:11.498 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-07-26T13:17:11.523 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-07-26_04:18:59-fs-wip-vshankar-testing-20240725.121331-debug-testing-default-smithi/7818864', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-07-26 13:12:11.112119', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAd43iYZVVxXI/hj5Mc1xAg9O5VOc1PAY1sjLeDIbM+gN+s2ECcT/KVUxAasUwADsYB1598s3PcY7O3LluU/jlc='} 2024-07-26T13:17:11.541 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi138.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-07-26_04:18:59-fs-wip-vshankar-testing-20240725.121331-debug-testing-default-smithi/7818864', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-07-26 13:12:11.113243', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO2ijrZL8gKZlJHEamiwY/D3ZbW8oU91YnbruYYZqSMAIW32zg+7i3kMJF8FntdASx6xCwdmtT6rDoeULLiWlwk='} 2024-07-26T13:17:11.564 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi167.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-07-26_04:18:59-fs-wip-vshankar-testing-20240725.121331-debug-testing-default-smithi/7818864', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-07-26 13:12:11.114137', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGnUCUxfb+tWOQtWviJUANX7OWEFnzGWvDYLzBY2w5hufagx2DKmg51LokSt2GKxkg1zK3eIuecUYatUTAk/85A='} 2024-07-26T13:17:11.564 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-07-26T13:17:11.570 INFO:teuthology.task.internal:roles: ubuntu@smithi100.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-07-26T13:17:11.570 INFO:teuthology.task.internal:roles: ubuntu@smithi138.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-07-26T13:17:11.570 INFO:teuthology.task.internal:roles: ubuntu@smithi167.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-07-26T13:17:11.570 INFO:teuthology.run_tasks:Running task console_log... 2024-07-26T13:17:11.642 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff620927ac0>, signals=[15]) 2024-07-26T13:17:11.643 INFO:teuthology.run_tasks:Running task internal.connect... 2024-07-26T13:17:11.648 INFO:teuthology.task.internal:Opening connections... 2024-07-26T13:17:11.648 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi100.front.sepia.ceph.com 2024-07-26T13:17:11.649 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-26T13:17:11.737 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi138.front.sepia.ceph.com 2024-07-26T13:17:11.738 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-26T13:17:11.820 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi167.front.sepia.ceph.com 2024-07-26T13:17:11.821 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-26T13:17:11.906 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-07-26T13:17:11.913 DEBUG:teuthology.orchestra.run.smithi100:> uname -m 2024-07-26T13:17:11.931 INFO:teuthology.orchestra.run.smithi100.stdout:x86_64 2024-07-26T13:17:11.931 DEBUG:teuthology.orchestra.run.smithi100:> cat /etc/os-release 2024-07-26T13:17:11.988 INFO:teuthology.orchestra.run.smithi100.stdout:NAME="CentOS Stream" 2024-07-26T13:17:11.988 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION="9" 2024-07-26T13:17:11.988 INFO:teuthology.orchestra.run.smithi100.stdout:ID="centos" 2024-07-26T13:17:11.988 INFO:teuthology.orchestra.run.smithi100.stdout:ID_LIKE="rhel fedora" 2024-07-26T13:17:11.988 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_ID="9" 2024-07-26T13:17:11.988 INFO:teuthology.orchestra.run.smithi100.stdout:PLATFORM_ID="platform:el9" 2024-07-26T13:17:11.988 INFO:teuthology.orchestra.run.smithi100.stdout:PRETTY_NAME="CentOS Stream 9" 2024-07-26T13:17:11.988 INFO:teuthology.orchestra.run.smithi100.stdout:ANSI_COLOR="0;31" 2024-07-26T13:17:11.989 INFO:teuthology.orchestra.run.smithi100.stdout:LOGO="fedora-logo-icon" 2024-07-26T13:17:11.989 INFO:teuthology.orchestra.run.smithi100.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-07-26T13:17:11.989 INFO:teuthology.orchestra.run.smithi100.stdout:HOME_URL="https://centos.org/" 2024-07-26T13:17:11.989 INFO:teuthology.orchestra.run.smithi100.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-07-26T13:17:11.989 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-07-26T13:17:11.989 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-07-26T13:17:11.990 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server 2024-07-26T13:17:12.020 DEBUG:teuthology.orchestra.run.smithi138:> uname -m 2024-07-26T13:17:12.038 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64 2024-07-26T13:17:12.038 DEBUG:teuthology.orchestra.run.smithi138:> cat /etc/os-release 2024-07-26T13:17:12.095 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="CentOS Stream" 2024-07-26T13:17:12.095 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="9" 2024-07-26T13:17:12.095 INFO:teuthology.orchestra.run.smithi138.stdout:ID="centos" 2024-07-26T13:17:12.096 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE="rhel fedora" 2024-07-26T13:17:12.096 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="9" 2024-07-26T13:17:12.096 INFO:teuthology.orchestra.run.smithi138.stdout:PLATFORM_ID="platform:el9" 2024-07-26T13:17:12.096 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="CentOS Stream 9" 2024-07-26T13:17:12.096 INFO:teuthology.orchestra.run.smithi138.stdout:ANSI_COLOR="0;31" 2024-07-26T13:17:12.096 INFO:teuthology.orchestra.run.smithi138.stdout:LOGO="fedora-logo-icon" 2024-07-26T13:17:12.096 INFO:teuthology.orchestra.run.smithi138.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-07-26T13:17:12.096 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://centos.org/" 2024-07-26T13:17:12.096 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-07-26T13:17:12.096 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-07-26T13:17:12.096 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-07-26T13:17:12.097 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2024-07-26T13:17:12.120 DEBUG:teuthology.orchestra.run.smithi167:> uname -m 2024-07-26T13:17:12.137 INFO:teuthology.orchestra.run.smithi167.stdout:x86_64 2024-07-26T13:17:12.137 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release 2024-07-26T13:17:12.194 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="CentOS Stream" 2024-07-26T13:17:12.194 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="9" 2024-07-26T13:17:12.194 INFO:teuthology.orchestra.run.smithi167.stdout:ID="centos" 2024-07-26T13:17:12.194 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="rhel fedora" 2024-07-26T13:17:12.194 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="9" 2024-07-26T13:17:12.194 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el9" 2024-07-26T13:17:12.194 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="CentOS Stream 9" 2024-07-26T13:17:12.194 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31" 2024-07-26T13:17:12.195 INFO:teuthology.orchestra.run.smithi167.stdout:LOGO="fedora-logo-icon" 2024-07-26T13:17:12.195 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-07-26T13:17:12.195 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://centos.org/" 2024-07-26T13:17:12.195 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-07-26T13:17:12.195 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-07-26T13:17:12.195 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-07-26T13:17:12.195 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server 2024-07-26T13:17:12.219 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-07-26T13:17:12.229 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-07-26T13:17:12.234 INFO:teuthology.task.internal:Checking for old test directory... 2024-07-26T13:17:12.234 DEBUG:teuthology.orchestra.run.smithi100:> test '!' -e /home/ubuntu/cephtest 2024-07-26T13:17:12.237 DEBUG:teuthology.orchestra.run.smithi138:> test '!' -e /home/ubuntu/cephtest 2024-07-26T13:17:12.239 DEBUG:teuthology.orchestra.run.smithi167:> test '!' -e /home/ubuntu/cephtest 2024-07-26T13:17:12.255 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-07-26T13:17:12.260 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-07-26T13:17:12.261 DEBUG:teuthology.orchestra.run.smithi100:> test -z $(ls -A /var/lib/ceph) 2024-07-26T13:17:12.293 DEBUG:teuthology.orchestra.run.smithi138:> test -z $(ls -A /var/lib/ceph) 2024-07-26T13:17:12.296 DEBUG:teuthology.orchestra.run.smithi167:> test -z $(ls -A /var/lib/ceph) 2024-07-26T13:17:12.433 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-07-26T13:17:12.505 INFO:teuthology.run_tasks:Running task kernel... 2024-07-26T13:17:12.521 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': 1, 'sha1': '3e9255e907100f88049f320ca33d977e29155d12'} 2024-07-26T13:17:12.521 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-07-26T13:17:12.521 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-07-26T13:17:12.522 DEBUG:teuthology.orchestra.run.smithi100:> test -f /run/.containerenv -o -f /.dockerenv 2024-07-26T13:17:12.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-26T13:17:12.538 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2024-07-26T13:17:12.596 INFO:teuthology.orchestra.run.smithi100.stdout:5.14.0-437.el9.x86_64 2024-07-26T13:17:12.597 INFO:teuthology.task.kernel:Running kernel on smithi100: 5.14.0-437.el9.x86_64 2024-07-26T13:17:12.597 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum install -y kernel 2024-07-26T13:17:17.075 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - BaseOS 2.9 MB/s | 8.2 MB 00:02 2024-07-26T13:17:18.818 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - AppStream 31 MB/s | 20 MB 00:00 2024-07-26T13:17:25.793 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - CRB 2.0 MB/s | 6.5 MB 00:03 2024-07-26T13:17:27.560 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 18 kB 00:00 2024-07-26T13:17:28.940 INFO:teuthology.orchestra.run.smithi100.stdout:Extra Packages for Enterprise Linux 18 MB/s | 22 MB 00:01 2024-07-26T13:17:34.445 INFO:teuthology.orchestra.run.smithi100.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-07-26T13:17:36.485 INFO:teuthology.orchestra.run.smithi100.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-07-26T13:17:36.485 INFO:teuthology.orchestra.run.smithi100.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-07-26T13:17:36.589 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-07-26T13:17:36.597 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-07-26T13:17:36.597 INFO:teuthology.orchestra.run.smithi100.stdout: Package Architecture Version Repository Size 2024-07-26T13:17:36.598 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-07-26T13:17:36.598 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-07-26T13:17:36.598 INFO:teuthology.orchestra.run.smithi100.stdout: kernel x86_64 5.14.0-480.el9 baseos 1.5 M 2024-07-26T13:17:36.598 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-07-26T13:17:36.598 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-core x86_64 5.14.0-480.el9 baseos 17 M 2024-07-26T13:17:36.598 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules x86_64 5.14.0-480.el9 baseos 36 M 2024-07-26T13:17:36.598 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules-core x86_64 5.14.0-480.el9 baseos 30 M 2024-07-26T13:17:36.598 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:17:36.598 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-07-26T13:17:36.598 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-07-26T13:17:36.598 INFO:teuthology.orchestra.run.smithi100.stdout:Install 4 Packages 2024-07-26T13:17:36.598 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:17:36.599 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 84 M 2024-07-26T13:17:36.599 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 126 M 2024-07-26T13:17:36.599 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-07-26T13:17:37.063 INFO:teuthology.orchestra.run.smithi100.stdout:(1/4): kernel-5.14.0-480.el9.x86_64.rpm 4.3 MB/s | 1.5 MB 00:00 2024-07-26T13:17:40.122 INFO:teuthology.orchestra.run.smithi100.stdout:(2/4): kernel-modules-core-5.14.0-480.el9.x86_6 9.8 MB/s | 30 MB 00:03 2024-07-26T13:17:40.289 INFO:teuthology.orchestra.run.smithi100.stdout:(3/4): kernel-core-5.14.0-480.el9.x86_64.rpm 4.8 MB/s | 17 MB 00:03 2024-07-26T13:17:42.006 INFO:teuthology.orchestra.run.smithi100.stdout:(4/4): kernel-modules-5.14.0-480.el9.x86_64.rpm 6.8 MB/s | 36 MB 00:05 2024-07-26T13:17:42.006 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-07-26T13:17:42.006 INFO:teuthology.orchestra.run.smithi100.stdout:Total 16 MB/s | 84 MB 00:05 2024-07-26T13:17:42.739 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-07-26T13:17:43.088 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-07-26T13:17:43.088 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-07-26T13:17:44.072 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-07-26T13:17:44.072 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-07-26T13:17:46.541 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-07-26T13:17:47.253 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-modules-core-5.14.0-480.el9.x86_64 1/4 2024-07-26T13:17:47.348 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-core-5.14.0-480.el9.x86_64 2/4 2024-07-26T13:17:50.017 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-core-5.14.0-480.el9.x86_64 2/4 2024-07-26T13:17:50.162 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-modules-5.14.0-480.el9.x86_64 3/4 2024-07-26T13:17:55.009 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-modules-5.14.0-480.el9.x86_64 3/4 2024-07-26T13:17:55.163 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-5.14.0-480.el9.x86_64 4/4 2024-07-26T13:17:59.975 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-modules-core-5.14.0-480.el9.x86_64 4/4 2024-07-26T13:18:29.579 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-core-5.14.0-480.el9.x86_64 4/4 2024-07-26T13:18:29.619 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-modules-5.14.0-480.el9.x86_64 4/4 2024-07-26T13:18:30.016 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-5.14.0-480.el9.x86_64 4/4 2024-07-26T13:18:30.016 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-5.14.0-480.el9.x86_64 1/4 2024-07-26T13:18:30.016 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-core-5.14.0-480.el9.x86_64 2/4 2024-07-26T13:18:30.016 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-modules-5.14.0-480.el9.x86_64 3/4 2024-07-26T13:18:30.885 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-modules-core-5.14.0-480.el9.x86_64 4/4 2024-07-26T13:18:30.886 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:18:30.886 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-07-26T13:18:30.886 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-5.14.0-480.el9.x86_64 2024-07-26T13:18:30.886 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-core-5.14.0-480.el9.x86_64 2024-07-26T13:18:30.886 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules-5.14.0-480.el9.x86_64 2024-07-26T13:18:30.886 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules-core-5.14.0-480.el9.x86_64 2024-07-26T13:18:30.886 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:18:30.886 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-07-26T13:18:31.284 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q kernel | sort -rV | head -n 1 2024-07-26T13:18:31.371 INFO:teuthology.orchestra.run.smithi100.stdout:kernel-5.14.0-480.el9.x86_64 2024-07-26T13:18:31.371 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-480.el9.x86_64 2024-07-26T13:18:31.372 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-480.el9.x86_64 2024-07-26T13:18:31.372 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-07-26T13:18:31.372 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-07-26T13:18:31.372 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-07-26T13:18:31.372 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi100.front.sepia.ceph.com, path=None, version=distro) 2024-07-26T13:18:31.372 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q kernel | sort -rV | head -n 1 2024-07-26T13:18:31.429 INFO:teuthology.orchestra.run.smithi100.stdout:kernel-5.14.0-480.el9.x86_64 2024-07-26T13:18:31.430 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-480.el9.x86_64 2024-07-26T13:18:31.430 DEBUG:teuthology.orchestra.run.smithi100:> sudo rpm -qi grub2-tools 2024-07-26T13:18:31.461 INFO:teuthology.orchestra.run.smithi100.stdout:Name : grub2-tools 2024-07-26T13:18:31.461 INFO:teuthology.orchestra.run.smithi100.stdout:Epoch : 1 2024-07-26T13:18:31.461 INFO:teuthology.orchestra.run.smithi100.stdout:Version : 2.06 2024-07-26T13:18:31.461 INFO:teuthology.orchestra.run.smithi100.stdout:Release : 68.el9 2024-07-26T13:18:31.461 INFO:teuthology.orchestra.run.smithi100.stdout:Architecture: x86_64 2024-07-26T13:18:31.461 INFO:teuthology.orchestra.run.smithi100.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-07-26T13:18:31.461 INFO:teuthology.orchestra.run.smithi100.stdout:Group : Unspecified 2024-07-26T13:18:31.461 INFO:teuthology.orchestra.run.smithi100.stdout:Size : 8264254 2024-07-26T13:18:31.461 INFO:teuthology.orchestra.run.smithi100.stdout:License : GPLv3+ 2024-07-26T13:18:31.461 INFO:teuthology.orchestra.run.smithi100.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-07-26T13:18:31.462 INFO:teuthology.orchestra.run.smithi100.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-07-26T13:18:31.462 INFO:teuthology.orchestra.run.smithi100.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-07-26T13:18:31.462 INFO:teuthology.orchestra.run.smithi100.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-07-26T13:18:31.462 INFO:teuthology.orchestra.run.smithi100.stdout:Packager : builder@centos.org 2024-07-26T13:18:31.462 INFO:teuthology.orchestra.run.smithi100.stdout:Vendor : CentOS 2024-07-26T13:18:31.462 INFO:teuthology.orchestra.run.smithi100.stdout:URL : http://www.gnu.org/software/grub/ 2024-07-26T13:18:31.462 INFO:teuthology.orchestra.run.smithi100.stdout:Summary : Support tools for GRUB. 2024-07-26T13:18:31.462 INFO:teuthology.orchestra.run.smithi100.stdout:Description : 2024-07-26T13:18:31.462 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:18:31.462 INFO:teuthology.orchestra.run.smithi100.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-07-26T13:18:31.462 INFO:teuthology.orchestra.run.smithi100.stdout:customizable bootloader with modular architecture. It supports a rich 2024-07-26T13:18:31.462 INFO:teuthology.orchestra.run.smithi100.stdout:variety of kernel formats, file systems, computer architectures and 2024-07-26T13:18:31.463 INFO:teuthology.orchestra.run.smithi100.stdout:hardware devices. 2024-07-26T13:18:31.463 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:18:31.463 INFO:teuthology.orchestra.run.smithi100.stdout:This subpackage provides tools for support of all platforms. 2024-07-26T13:18:31.463 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-07-26T13:18:31.463 INFO:teuthology.task.kernel:Updating grub on smithi100 to boot 5.14.0-480.el9.x86_64 2024-07-26T13:18:31.464 DEBUG:teuthology.orchestra.run.smithi100:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-07-26T13:18:32.040 INFO:teuthology.orchestra.run.smithi100.stderr:Generating grub configuration file ... 2024-07-26T13:18:33.368 INFO:teuthology.orchestra.run.smithi100.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-07-26T13:18:33.395 INFO:teuthology.orchestra.run.smithi100.stderr:done 2024-07-26T13:18:33.397 DEBUG:teuthology.orchestra.run.smithi100:> mktemp 2024-07-26T13:18:33.413 INFO:teuthology.orchestra.run.smithi100.stdout:/tmp/tmp.SI2ST6h4Zz 2024-07-26T13:18:33.413 DEBUG:teuthology.orchestra.run.smithi100:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.SI2ST6h4Zz 2024-07-26T13:18:33.478 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0666 /tmp/tmp.SI2ST6h4Zz 2024-07-26T13:18:33.617 DEBUG:teuthology.orchestra.remote:smithi100:/tmp/tmp.SI2ST6h4Zz is 6KB 2024-07-26T13:18:33.666 DEBUG:teuthology.orchestra.run.smithi100:> rm -fr /tmp/tmp.SI2ST6h4Zz 2024-07-26T13:18:33.680 DEBUG:teuthology.orchestra.run.smithi100:> sudo /bin/ls /boot/loader/entries || true 2024-07-26T13:18:33.751 INFO:teuthology.orchestra.run.smithi100.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-07-26T13:18:33.752 INFO:teuthology.orchestra.run.smithi100.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-07-26T13:18:33.752 INFO:teuthology.orchestra.run.smithi100.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-07-26T13:18:33.752 INFO:teuthology.orchestra.run.smithi100.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-480.el9.x86_64.conf 2024-07-26T13:18:33.753 DEBUG:teuthology.orchestra.run.smithi100:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-480.el9.x86_64 2024-07-26T13:18:33.910 DEBUG:teuthology.orchestra.run.smithi100:> sudo shutdown -r now 2024-07-26T13:19:03.914 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-07-26T13:19:03.914 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-07-26T13:19:03.915 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-26T13:19:22.334 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-07-26T13:19:31.335 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-07-26T13:19:31.336 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-26T13:19:34.398 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-07-26T13:19:46.399 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-07-26T13:19:46.400 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-26T13:20:18.246 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-07-26T13:20:18.672 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-07-26T13:20:18.672 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-480.el9.x86_64"... 2024-07-26T13:20:18.672 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2024-07-26T13:20:18.687 INFO:teuthology.orchestra.run.smithi100.stdout:5.14.0-480.el9.x86_64 2024-07-26T13:20:18.687 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-480.el9.x86_64 vs 5.14.0-480.el9.x86_64 2024-07-26T13:20:18.687 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-07-26T13:20:18.687 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-07-26T13:20:19.688 DEBUG:teuthology.parallel:result is None 2024-07-26T13:20:19.688 INFO:teuthology.run_tasks:Running task internal.base... 2024-07-26T13:20:19.696 INFO:teuthology.task.internal:Creating test directory... 2024-07-26T13:20:19.696 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-07-26T13:20:19.699 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-07-26T13:20:19.702 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-07-26T13:20:19.719 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-07-26T13:20:19.725 INFO:teuthology.run_tasks:Running task internal.archive... 2024-07-26T13:20:19.736 INFO:teuthology.task.internal:Creating archive directory... 2024-07-26T13:20:19.736 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-07-26T13:20:19.751 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-07-26T13:20:19.759 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-07-26T13:20:19.875 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-07-26T13:20:19.881 INFO:teuthology.task.internal:Enabling coredump saving... 2024-07-26T13:20:19.882 DEBUG:teuthology.orchestra.run.smithi138:> test -f /run/.containerenv -o -f /.dockerenv 2024-07-26T13:20:19.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-26T13:20:19.898 DEBUG:teuthology.orchestra.run.smithi167:> test -f /run/.containerenv -o -f /.dockerenv 2024-07-26T13:20:19.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-26T13:20:19.914 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-07-26T13:20:19.917 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-07-26T13:20:19.940 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-07-26T13:20:19.977 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-26T13:20:19.989 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-26T13:20:19.991 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-26T13:20:20.012 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-26T13:20:20.013 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-26T13:20:20.026 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-26T13:20:20.029 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-07-26T13:20:20.036 INFO:teuthology.task.internal:Configuring sudo... 2024-07-26T13:20:20.036 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-07-26T13:20:20.039 DEBUG:teuthology.orchestra.run.smithi138:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-07-26T13:20:20.057 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-07-26T13:20:20.095 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-07-26T13:20:20.104 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-07-26T13:20:20.104 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-07-26T13:20:20.106 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-07-26T13:20:20.125 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-07-26T13:20:20.152 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-26T13:20:20.197 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-26T13:20:20.236 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-26T13:20:20.317 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-26T13:20:20.353 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-07-26T13:20:20.354 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-07-26T13:20:20.415 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-26T13:20:20.459 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-26T13:20:20.511 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-26T13:20:20.595 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-26T13:20:20.635 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-07-26T13:20:20.635 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-07-26T13:20:20.700 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-26T13:20:20.745 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-26T13:20:20.800 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-26T13:20:20.883 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-26T13:20:20.925 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-07-26T13:20:20.925 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-07-26T13:20:20.990 DEBUG:teuthology.orchestra.run.smithi100:> sudo service rsyslog restart 2024-07-26T13:20:20.993 DEBUG:teuthology.orchestra.run.smithi138:> sudo service rsyslog restart 2024-07-26T13:20:20.995 DEBUG:teuthology.orchestra.run.smithi167:> sudo service rsyslog restart 2024-07-26T13:20:21.030 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-26T13:20:21.044 INFO:teuthology.orchestra.run.smithi100.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-26T13:20:21.063 INFO:teuthology.orchestra.run.smithi167.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-26T13:20:21.561 INFO:teuthology.run_tasks:Running task internal.timer... 2024-07-26T13:20:21.569 INFO:teuthology.task.internal:Starting timer... 2024-07-26T13:20:21.569 INFO:teuthology.run_tasks:Running task pcp... 2024-07-26T13:20:21.580 INFO:teuthology.run_tasks:Running task selinux... 2024-07-26T13:20:21.588 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-07-26T13:20:21.589 DEBUG:teuthology.orchestra.run.smithi100:> sudo service auditd rotate 2024-07-26T13:20:21.730 INFO:teuthology.orchestra.run.smithi100.stdout:Rotating logs: 2024-07-26T13:20:21.732 DEBUG:teuthology.orchestra.run.smithi138:> sudo service auditd rotate 2024-07-26T13:20:21.797 INFO:teuthology.orchestra.run.smithi138.stdout:Rotating logs: 2024-07-26T13:20:21.799 DEBUG:teuthology.orchestra.run.smithi167:> sudo service auditd rotate 2024-07-26T13:20:21.891 INFO:teuthology.orchestra.run.smithi167.stdout:Rotating logs: 2024-07-26T13:20:21.893 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-07-26T13:20:21.894 DEBUG:teuthology.orchestra.run.smithi100:> /usr/sbin/getenforce 2024-07-26T13:20:21.942 INFO:teuthology.orchestra.run.smithi100.stdout:Permissive 2024-07-26T13:20:21.942 DEBUG:teuthology.orchestra.run.smithi138:> /usr/sbin/getenforce 2024-07-26T13:20:21.968 INFO:teuthology.orchestra.run.smithi138.stdout:Permissive 2024-07-26T13:20:21.969 DEBUG:teuthology.orchestra.run.smithi167:> /usr/sbin/getenforce 2024-07-26T13:20:21.993 INFO:teuthology.orchestra.run.smithi167.stdout:Permissive 2024-07-26T13:20:21.993 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi100.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi138.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi167.front.sepia.ceph.com': 'permissive'} 2024-07-26T13:20:21.993 DEBUG:teuthology.orchestra.run.smithi100:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-26T13:20:22.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-26T13:20:22.034 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-26T13:20:22.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-26T13:20:22.063 DEBUG:teuthology.orchestra.run.smithi167:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-26T13:20:22.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-26T13:20:22.089 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-07-26T13:20:22.089 DEBUG:teuthology.orchestra.run.smithi100:> sudo /usr/sbin/setenforce permissive 2024-07-26T13:20:22.482 DEBUG:teuthology.orchestra.run.smithi138:> sudo /usr/sbin/setenforce permissive 2024-07-26T13:20:22.507 DEBUG:teuthology.orchestra.run.smithi167:> sudo /usr/sbin/setenforce permissive 2024-07-26T13:20:22.531 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-07-26T13:20:22.541 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-07-26T13:20:22.543 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-07-26T13:20:22.650 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-07-26T13:20:22.660 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-07-26T13:20:22.661 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi100.front.sepia.ceph.com,smithi138.front.sepia.ceph.com,smithi167.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-07-26T13:25:17.503 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi100.front.sepia.ceph.com'), Remote(name='ubuntu@smithi138.front.sepia.ceph.com'), Remote(name='ubuntu@smithi167.front.sepia.ceph.com')] 2024-07-26T13:25:17.504 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-07-26T13:25:17.505 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-26T13:25:17.585 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-07-26T13:25:17.634 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-07-26T13:25:17.635 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-07-26T13:25:17.635 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-26T13:25:17.711 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-07-26T13:25:17.785 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-07-26T13:25:17.785 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-07-26T13:25:17.786 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-26T13:25:17.876 DEBUG:teuthology.orchestra.run.smithi167:> true 2024-07-26T13:25:17.962 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-07-26T13:25:17.963 INFO:teuthology.run_tasks:Running task clock... 2024-07-26T13:25:17.974 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-07-26T13:25:17.974 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-07-26T13:25:17.975 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-26T13:25:17.977 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-07-26T13:25:17.977 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-26T13:25:17.979 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-07-26T13:25:17.979 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-26T13:25:18.004 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-07-26T13:25:18.005 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-07-26T13:25:18.018 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-07-26T13:25:18.019 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-07-26T13:25:18.032 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-07-26T13:25:18.040 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: ntpd: command not found 2024-07-26T13:25:18.044 INFO:teuthology.orchestra.run.smithi100.stderr:sudo: ntpd: command not found 2024-07-26T13:25:18.047 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-07-26T13:25:18.050 INFO:teuthology.orchestra.run.smithi138.stdout:506 Cannot talk to daemon 2024-07-26T13:25:18.055 INFO:teuthology.orchestra.run.smithi100.stdout:506 Cannot talk to daemon 2024-07-26T13:25:18.064 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-07-26T13:25:18.070 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-07-26T13:25:18.070 INFO:teuthology.orchestra.run.smithi167.stderr:sudo: ntpd: command not found 2024-07-26T13:25:18.077 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-07-26T13:25:18.081 INFO:teuthology.orchestra.run.smithi167.stdout:506 Cannot talk to daemon 2024-07-26T13:25:18.083 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-07-26T13:25:18.094 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-07-26T13:25:18.107 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-07-26T13:25:18.136 INFO:teuthology.orchestra.run.smithi100.stderr:bash: line 1: ntpq: command not found 2024-07-26T13:25:18.137 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-07-26T13:25:18.139 INFO:teuthology.orchestra.run.smithi100.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-26T13:25:18.139 INFO:teuthology.orchestra.run.smithi100.stdout:=============================================================================== 2024-07-26T13:25:18.139 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-26T13:25:18.140 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-26T13:25:18.140 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-26T13:25:18.140 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-26T13:25:18.141 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-26T13:25:18.141 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-07-26T13:25:18.141 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-26T13:25:18.142 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-26T13:25:18.142 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-26T13:25:18.142 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-26T13:25:18.160 INFO:teuthology.orchestra.run.smithi167.stderr:bash: line 1: ntpq: command not found 2024-07-26T13:25:18.163 INFO:teuthology.orchestra.run.smithi167.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-26T13:25:18.163 INFO:teuthology.orchestra.run.smithi167.stdout:=============================================================================== 2024-07-26T13:25:18.163 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-26T13:25:18.163 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-26T13:25:18.163 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-26T13:25:18.163 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-26T13:25:18.164 INFO:teuthology.run_tasks:Running task install... 2024-07-26T13:25:18.172 DEBUG:teuthology.task.install:project ceph 2024-07-26T13:25:18.173 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528'}} 2024-07-26T13:25:18.173 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528'} 2024-07-26T13:25:18.173 INFO:teuthology.task.install:Using flavor: default 2024-07-26T13:25:18.178 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-07-26T13:25:18.178 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-07-26T13:25:18.178 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528', 'tag': None, 'wait_for_package': False} 2024-07-26T13:25:18.179 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 2024-07-26T13:25:18.181 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528', 'tag': None, 'wait_for_package': False} 2024-07-26T13:25:18.181 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 2024-07-26T13:25:18.182 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528', 'tag': None, 'wait_for_package': False} 2024-07-26T13:25:18.182 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 2024-07-26T13:25:18.331 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240725.121331-debug/e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528/centos/9/flavors/default/ 2024-07-26T13:25:18.331 INFO:teuthology.task.install.rpm:Package version is 19.0.0-5320.ge8f88bb1 2024-07-26T13:25:18.332 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240725.121331-debug/e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528/centos/9/flavors/default/ 2024-07-26T13:25:18.333 INFO:teuthology.task.install.rpm:Package version is 19.0.0-5320.ge8f88bb1 2024-07-26T13:25:18.341 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240725.121331-debug/e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528/centos/9/flavors/default/ 2024-07-26T13:25:18.341 INFO:teuthology.task.install.rpm:Package version is 19.0.0-5320.ge8f88bb1 2024-07-26T13:25:18.443 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240725.121331-debug/e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240725.121331-debug/e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240725.121331-debug/e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-07-26T13:25:18.444 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-07-26T13:25:18.444 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-07-26T13:25:18.453 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240725.121331-debug/e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240725.121331-debug/e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240725.121331-debug/e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-07-26T13:25:18.453 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-07-26T13:25:18.453 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-07-26T13:25:18.459 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240725.121331-debug/e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240725.121331-debug/e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240725.121331-debug/e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-07-26T13:25:18.459 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-07-26T13:25:18.459 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-07-26T13:25:18.468 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-07-26T13:25:18.468 DEBUG:teuthology.orchestra.run.smithi167:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-07-26T13:25:18.480 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-07-26T13:25:18.480 DEBUG:teuthology.orchestra.run.smithi100:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-07-26T13:25:18.483 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-07-26T13:25:18.484 DEBUG:teuthology.orchestra.run.smithi138:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-07-26T13:25:18.533 DEBUG:teuthology.orchestra.run.smithi167:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-07-26T13:25:18.546 DEBUG:teuthology.orchestra.run.smithi100:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-07-26T13:25:18.548 DEBUG:teuthology.orchestra.run.smithi138:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-07-26T13:25:18.606 DEBUG:teuthology.orchestra.run.smithi167:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-07-26T13:25:18.620 DEBUG:teuthology.orchestra.run.smithi138:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-07-26T13:25:18.621 DEBUG:teuthology.orchestra.run.smithi100:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-07-26T13:25:18.669 INFO:teuthology.orchestra.run.smithi167.stdout:check_obsoletes = 1 2024-07-26T13:25:18.671 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum clean all 2024-07-26T13:25:18.682 INFO:teuthology.orchestra.run.smithi138.stdout:check_obsoletes = 1 2024-07-26T13:25:18.684 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum clean all 2024-07-26T13:25:18.685 INFO:teuthology.orchestra.run.smithi100.stdout:check_obsoletes = 1 2024-07-26T13:25:18.687 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum clean all 2024-07-26T13:25:18.894 INFO:teuthology.orchestra.run.smithi167.stdout:45 files removed 2024-07-26T13:25:18.909 INFO:teuthology.orchestra.run.smithi138.stdout:45 files removed 2024-07-26T13:25:18.917 INFO:teuthology.orchestra.run.smithi100.stdout:45 files removed 2024-07-26T13:25:18.921 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-07-26T13:25:18.936 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-07-26T13:25:18.944 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-07-26T13:25:19.505 INFO:teuthology.orchestra.run.smithi167.stdout:ceph packages for x86_64 260 kB/s | 83 kB 00:00 2024-07-26T13:25:19.530 INFO:teuthology.orchestra.run.smithi138.stdout:ceph packages for x86_64 249 kB/s | 83 kB 00:00 2024-07-26T13:25:19.532 INFO:teuthology.orchestra.run.smithi100.stdout:ceph packages for x86_64 268 kB/s | 83 kB 00:00 2024-07-26T13:25:19.839 INFO:teuthology.orchestra.run.smithi167.stdout:ceph noarch packages 82 kB/s | 24 kB 00:00 2024-07-26T13:25:19.872 INFO:teuthology.orchestra.run.smithi138.stdout:ceph noarch packages 80 kB/s | 24 kB 00:00 2024-07-26T13:25:19.908 INFO:teuthology.orchestra.run.smithi100.stdout:ceph noarch packages 74 kB/s | 24 kB 00:00 2024-07-26T13:25:20.140 INFO:teuthology.orchestra.run.smithi167.stdout:ceph source packages 7.0 kB/s | 1.9 kB 00:00 2024-07-26T13:25:20.173 INFO:teuthology.orchestra.run.smithi138.stdout:ceph source packages 7.0 kB/s | 1.9 kB 00:00 2024-07-26T13:25:20.241 INFO:teuthology.orchestra.run.smithi100.stdout:ceph source packages 6.4 kB/s | 1.9 kB 00:00 2024-07-26T13:25:20.782 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.2 MB 00:00 2024-07-26T13:25:21.931 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.2 MB 00:01 2024-07-26T13:25:22.483 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - AppStream 33 MB/s | 20 MB 00:00 2024-07-26T13:25:23.283 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - BaseOS 2.7 MB/s | 8.2 MB 00:02 2024-07-26T13:25:24.968 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - AppStream 33 MB/s | 20 MB 00:00 2024-07-26T13:25:28.700 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - CRB 2.6 MB/s | 6.5 MB 00:02 2024-07-26T13:25:30.146 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - CRB 4.5 MB/s | 6.5 MB 00:01 2024-07-26T13:25:30.151 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - Extras packages 77 kB/s | 18 kB 00:00 2024-07-26T13:25:30.582 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - AppStream 2.6 MB/s | 20 MB 00:07 2024-07-26T13:25:31.680 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 16 MB/s | 22 MB 00:01 2024-07-26T13:25:32.420 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - Extras packages 20 kB/s | 18 kB 00:00 2024-07-26T13:25:33.849 INFO:teuthology.orchestra.run.smithi100.stdout:Extra Packages for Enterprise Linux 17 MB/s | 22 MB 00:01 2024-07-26T13:25:35.358 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - CRB 6.1 MB/s | 6.5 MB 00:01 2024-07-26T13:25:36.959 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 18 kB 00:00 2024-07-26T13:25:37.078 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-07-26T13:25:38.254 INFO:teuthology.orchestra.run.smithi167.stdout:Extra Packages for Enterprise Linux 19 MB/s | 22 MB 00:01 2024-07-26T13:25:39.154 INFO:teuthology.orchestra.run.smithi100.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-07-26T13:25:39.341 INFO:teuthology.orchestra.run.smithi138.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-26T13:25:39.341 INFO:teuthology.orchestra.run.smithi138.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-26T13:25:39.352 INFO:teuthology.orchestra.run.smithi138.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-07-26T13:25:39.353 INFO:teuthology.orchestra.run.smithi138.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-07-26T13:25:39.354 INFO:teuthology.orchestra.run.smithi138.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-07-26T13:25:39.355 INFO:teuthology.orchestra.run.smithi138.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-07-26T13:25:39.356 INFO:teuthology.orchestra.run.smithi138.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-07-26T13:25:39.358 INFO:teuthology.orchestra.run.smithi138.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-07-26T13:25:39.359 INFO:teuthology.orchestra.run.smithi138.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-07-26T13:25:39.423 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-07-26T13:25:39.448 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================= 2024-07-26T13:25:39.448 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-07-26T13:25:39.448 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================= 2024-07-26T13:25:39.448 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-07-26T13:25:39.448 INFO:teuthology.orchestra.run.smithi138.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-07-26T13:25:39.448 INFO:teuthology.orchestra.run.smithi138.stdout: ceph x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 6.6 k 2024-07-26T13:25:39.448 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 5.5 M 2024-07-26T13:25:39.448 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 872 k 2024-07-26T13:25:39.448 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 148 k 2024-07-26T13:25:39.448 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 1.5 M 2024-07-26T13:25:39.448 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm noarch 2:19.0.0-5320.ge8f88bb1.el9 ceph-noarch 149 k 2024-07-26T13:25:39.449 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard noarch 2:19.0.0-5320.ge8f88bb1.el9 ceph-noarch 14 M 2024-07-26T13:25:39.449 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-5320.ge8f88bb1.el9 ceph-noarch 7.4 M 2024-07-26T13:25:39.449 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook noarch 2:19.0.0-5320.ge8f88bb1.el9 ceph-noarch 49 k 2024-07-26T13:25:39.449 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 10 M 2024-07-26T13:25:39.449 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 44 M 2024-07-26T13:25:39.449 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-volume noarch 2:19.0.0-5320.ge8f88bb1.el9 ceph-noarch 274 k 2024-07-26T13:25:39.449 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm noarch 2:19.0.0-5320.ge8f88bb1.el9 ceph-noarch 771 k 2024-07-26T13:25:39.449 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-mirror x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 225 k 2024-07-26T13:25:39.449 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-shell x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 27 k 2024-07-26T13:25:39.449 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-top noarch 2:19.0.0-5320.ge8f88bb1.el9 ceph-noarch 18 k 2024-07-26T13:25:39.449 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-07-26T13:25:39.449 INFO:teuthology.orchestra.run.smithi138.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-07-26T13:25:39.449 INFO:teuthology.orchestra.run.smithi138.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-07-26T13:25:39.450 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 32 k 2024-07-26T13:25:39.450 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2 x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 765 k 2024-07-26T13:25:39.450 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 127 k 2024-07-26T13:25:39.450 INFO:teuthology.orchestra.run.smithi138.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-07-26T13:25:39.450 INFO:teuthology.orchestra.run.smithi138.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-07-26T13:25:39.450 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-07-26T13:25:39.450 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-07-26T13:25:39.450 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-07-26T13:25:39.450 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-07-26T13:25:39.450 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 188 k 2024-07-26T13:25:39.450 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 372 k 2024-07-26T13:25:39.450 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 355 k 2024-07-26T13:25:39.450 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 116 k 2024-07-26T13:25:39.451 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 86 k 2024-07-26T13:25:39.451 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 3.1 M 2024-07-26T13:25:39.451 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 170 k 2024-07-26T13:25:39.451 INFO:teuthology.orchestra.run.smithi138.stdout: xfsprogs-devel x86_64 6.4.0-3.el9 appstream 96 k 2024-07-26T13:25:39.451 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrading: 2024-07-26T13:25:39.451 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager x86_64 1:1.48.2-2.el9 baseos 2.3 M 2024-07-26T13:25:39.451 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-libnm x86_64 1:1.48.2-2.el9 baseos 1.8 M 2024-07-26T13:25:39.451 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-team x86_64 1:1.48.2-2.el9 baseos 41 k 2024-07-26T13:25:39.451 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-tui x86_64 1:1.48.2-2.el9 baseos 247 k 2024-07-26T13:25:39.452 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-07-26T13:25:39.452 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-07-26T13:25:39.452 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-07-26T13:25:39.452 INFO:teuthology.orchestra.run.smithi138.stdout: librados2 x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 3.4 M 2024-07-26T13:25:39.452 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1 x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 3.1 M 2024-07-26T13:25:39.452 INFO:teuthology.orchestra.run.smithi138.stdout: openssl x86_64 1:3.2.2-2.el9 baseos 1.4 M 2024-07-26T13:25:39.452 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-libs x86_64 1:3.2.2-2.el9 baseos 2.4 M 2024-07-26T13:25:39.452 INFO:teuthology.orchestra.run.smithi138.stdout: xfsprogs x86_64 6.4.0-3.el9 baseos 1.1 M 2024-07-26T13:25:39.452 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-07-26T13:25:39.452 INFO:teuthology.orchestra.run.smithi138.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-07-26T13:25:39.452 INFO:teuthology.orchestra.run.smithi138.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-07-26T13:25:39.452 INFO:teuthology.orchestra.run.smithi138.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-07-26T13:25:39.452 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 20 M 2024-07-26T13:25:39.452 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards noarch 2:19.0.0-5320.ge8f88bb1.el9 ceph-noarch 34 k 2024-07-26T13:25:39.453 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 2.3 M 2024-07-26T13:25:39.453 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core noarch 2:19.0.0-5320.ge8f88bb1.el9 ceph-noarch 284 k 2024-07-26T13:25:39.453 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 4.9 M 2024-07-26T13:25:39.453 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 18 M 2024-07-26T13:25:39.453 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts noarch 2:19.0.0-5320.ge8f88bb1.el9 ceph-noarch 16 k 2024-07-26T13:25:39.453 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 25 k 2024-07-26T13:25:39.453 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-07-26T13:25:39.453 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-07-26T13:25:39.453 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-07-26T13:25:39.453 INFO:teuthology.orchestra.run.smithi138.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-07-26T13:25:39.453 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-07-26T13:25:39.453 INFO:teuthology.orchestra.run.smithi138.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-07-26T13:25:39.453 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-07-26T13:25:39.453 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-07-26T13:25:39.454 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 158 k 2024-07-26T13:25:39.454 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-07-26T13:25:39.454 INFO:teuthology.orchestra.run.smithi138.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-07-26T13:25:39.454 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-07-26T13:25:39.454 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-07-26T13:25:39.454 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-07-26T13:25:39.454 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1 x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 503 k 2024-07-26T13:25:39.454 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-07-26T13:25:39.454 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2 x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 5.3 M 2024-07-26T13:25:39.454 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-07-26T13:25:39.454 INFO:teuthology.orchestra.run.smithi138.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-07-26T13:25:39.454 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-07-26T13:25:39.454 INFO:teuthology.orchestra.run.smithi138.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-07-26T13:25:39.455 INFO:teuthology.orchestra.run.smithi138.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-07-26T13:25:39.455 INFO:teuthology.orchestra.run.smithi138.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-07-26T13:25:39.455 INFO:teuthology.orchestra.run.smithi138.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-07-26T13:25:39.455 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-07-26T13:25:39.455 INFO:teuthology.orchestra.run.smithi138.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-07-26T13:25:39.455 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-07-26T13:25:39.455 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-07-26T13:25:39.455 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-07-26T13:25:39.455 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-07-26T13:25:39.455 INFO:teuthology.orchestra.run.smithi138.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-07-26T13:25:39.455 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-07-26T13:25:39.455 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-07-26T13:25:39.455 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-07-26T13:25:39.456 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 45 k 2024-07-26T13:25:39.456 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 138 k 2024-07-26T13:25:39.456 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-07-26T13:25:39.456 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-07-26T13:25:39.456 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-07-26T13:25:39.456 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-07-26T13:25:39.456 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-07-26T13:25:39.456 INFO:teuthology.orchestra.run.smithi138.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-07-26T13:25:39.456 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-07-26T13:25:39.456 INFO:teuthology.orchestra.run.smithi138.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-07-26T13:25:39.456 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth noarch 1:2.32.0-1.el9 epel 216 k 2024-07-26T13:25:39.456 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-07-26T13:25:39.456 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-07-26T13:25:39.456 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-07-26T13:25:39.457 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-07-26T13:25:39.457 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-07-26T13:25:39.457 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-07-26T13:25:39.457 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-07-26T13:25:39.457 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-07-26T13:25:39.457 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-07-26T13:25:39.457 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-07-26T13:25:39.457 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-07-26T13:25:39.457 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-07-26T13:25:39.457 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-07-26T13:25:39.457 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-07-26T13:25:39.457 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-07-26T13:25:39.457 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-07-26T13:25:39.457 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-07-26T13:25:39.458 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-07-26T13:25:39.458 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-07-26T13:25:39.458 INFO:teuthology.orchestra.run.smithi138.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-07-26T13:25:39.458 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-07-26T13:25:39.458 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-07-26T13:25:39.458 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-07-26T13:25:39.458 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-07-26T13:25:39.458 INFO:teuthology.orchestra.run.smithi138.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-07-26T13:25:39.458 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-07-26T13:25:39.458 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-07-26T13:25:39.458 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-07-26T13:25:39.458 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-07-26T13:25:39.458 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-07-26T13:25:39.459 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-07-26T13:25:39.459 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-07-26T13:25:39.459 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-07-26T13:25:39.459 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-07-26T13:25:39.459 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-07-26T13:25:39.459 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-07-26T13:25:39.459 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-07-26T13:25:39.459 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-07-26T13:25:39.459 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-07-26T13:25:39.459 INFO:teuthology.orchestra.run.smithi138.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-07-26T13:25:39.459 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-07-26T13:25:39.459 INFO:teuthology.orchestra.run.smithi138.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-07-26T13:25:39.459 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-07-26T13:25:39.459 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-07-26T13:25:39.460 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-07-26T13:25:39.460 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-07-26T13:25:39.460 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-07-26T13:25:39.460 INFO:teuthology.orchestra.run.smithi138.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-07-26T13:25:39.460 INFO:teuthology.orchestra.run.smithi138.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-07-26T13:25:39.460 INFO:teuthology.orchestra.run.smithi138.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-07-26T13:25:39.460 INFO:teuthology.orchestra.run.smithi138.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-07-26T13:25:39.460 INFO:teuthology.orchestra.run.smithi138.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-07-26T13:25:39.460 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-07-26T13:25:39.460 INFO:teuthology.orchestra.run.smithi138.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-07-26T13:25:39.460 INFO:teuthology.orchestra.run.smithi138.stdout:Installing weak dependencies: 2024-07-26T13:25:39.460 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-07-26T13:25:39.460 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-07-26T13:25:39.461 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-07-26T13:25:39.461 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================= 2024-07-26T13:25:39.461 INFO:teuthology.orchestra.run.smithi138.stdout:Install 148 Packages 2024-07-26T13:25:39.461 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrade 12 Packages 2024-07-26T13:25:39.461 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-07-26T13:25:39.461 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 233 M 2024-07-26T13:25:39.462 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-07-26T13:25:40.079 INFO:teuthology.orchestra.run.smithi138.stdout:(1/160): ceph-19.0.0-5320.ge8f88bb1.el9.x86_64. 37 kB/s | 6.6 kB 00:00 2024-07-26T13:25:40.271 INFO:teuthology.orchestra.run.smithi138.stdout:(2/160): ceph-fuse-19.0.0-5320.ge8f88bb1.el9.x8 4.4 MB/s | 872 kB 00:00 2024-07-26T13:25:40.346 INFO:teuthology.orchestra.run.smithi138.stdout:(3/160): ceph-immutable-object-cache-19.0.0-532 1.9 MB/s | 148 kB 00:00 2024-07-26T13:25:40.438 INFO:teuthology.orchestra.run.smithi138.stdout:(4/160): ceph-base-19.0.0-5320.ge8f88bb1.el9.x8 10 MB/s | 5.5 MB 00:00 2024-07-26T13:25:40.546 INFO:teuthology.orchestra.run.smithi138.stdout:(5/160): ceph-mgr-19.0.0-5320.ge8f88bb1.el9.x86 14 MB/s | 1.5 MB 00:00 2024-07-26T13:25:40.639 INFO:teuthology.orchestra.run.smithi138.stdout:(6/160): ceph-mds-19.0.0-5320.ge8f88bb1.el9.x86 7.7 MB/s | 2.3 MB 00:00 2024-07-26T13:25:40.814 INFO:teuthology.orchestra.run.smithi138.stdout:(7/160): ceph-mon-19.0.0-5320.ge8f88bb1.el9.x86 18 MB/s | 4.9 MB 00:00 2024-07-26T13:25:41.172 INFO:teuthology.orchestra.run.smithi138.stdout:(8/160): ceph-common-19.0.0-5320.ge8f88bb1.el9. 16 MB/s | 20 MB 00:01 2024-07-26T13:25:41.223 INFO:teuthology.orchestra.run.smithi138.stdout:(9/160): ceph-selinux-19.0.0-5320.ge8f88bb1.el9 504 kB/s | 25 kB 00:00 2024-07-26T13:25:41.381 INFO:teuthology.orchestra.run.smithi138.stdout:(10/160): ceph-radosgw-19.0.0-5320.ge8f88bb1.el 18 MB/s | 10 MB 00:00 2024-07-26T13:25:41.448 INFO:teuthology.orchestra.run.smithi138.stdout:(11/160): cephfs-mirror-19.0.0-5320.ge8f88bb1.e 3.3 MB/s | 225 kB 00:00 2024-07-26T13:25:41.500 INFO:teuthology.orchestra.run.smithi100.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-26T13:25:41.501 INFO:teuthology.orchestra.run.smithi100.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-26T13:25:41.512 INFO:teuthology.orchestra.run.smithi100.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-07-26T13:25:41.513 INFO:teuthology.orchestra.run.smithi100.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-07-26T13:25:41.514 INFO:teuthology.orchestra.run.smithi100.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-07-26T13:25:41.515 INFO:teuthology.orchestra.run.smithi100.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-07-26T13:25:41.516 INFO:teuthology.orchestra.run.smithi100.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-07-26T13:25:41.519 INFO:teuthology.orchestra.run.smithi100.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-07-26T13:25:41.519 INFO:teuthology.orchestra.run.smithi100.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-07-26T13:25:41.524 INFO:teuthology.orchestra.run.smithi138.stdout:(12/160): cephfs-shell-19.0.0-5320.ge8f88bb1.el 361 kB/s | 27 kB 00:00 2024-07-26T13:25:41.585 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-07-26T13:25:41.599 INFO:teuthology.orchestra.run.smithi138.stdout:(13/160): libcephfs-devel-19.0.0-5320.ge8f88bb1 424 kB/s | 32 kB 00:00 2024-07-26T13:25:41.613 INFO:teuthology.orchestra.run.smithi100.stdout:======================================================================================= 2024-07-26T13:25:41.613 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2024-07-26T13:25:41.613 INFO:teuthology.orchestra.run.smithi100.stdout:======================================================================================= 2024-07-26T13:25:41.613 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-07-26T13:25:41.613 INFO:teuthology.orchestra.run.smithi100.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-07-26T13:25:41.613 INFO:teuthology.orchestra.run.smithi100.stdout: ceph x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 6.6 k 2024-07-26T13:25:41.613 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 5.5 M 2024-07-26T13:25:41.614 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-fuse x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 872 k 2024-07-26T13:25:41.614 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 148 k 2024-07-26T13:25:41.614 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 1.5 M 2024-07-26T13:25:41.614 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-cephadm noarch 2:19.0.0-5320.ge8f88bb1.el9 ceph-noarch 149 k 2024-07-26T13:25:41.614 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard noarch 2:19.0.0-5320.ge8f88bb1.el9 ceph-noarch 14 M 2024-07-26T13:25:41.614 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-5320.ge8f88bb1.el9 ceph-noarch 7.4 M 2024-07-26T13:25:41.614 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-rook noarch 2:19.0.0-5320.ge8f88bb1.el9 ceph-noarch 49 k 2024-07-26T13:25:41.614 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-radosgw x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 10 M 2024-07-26T13:25:41.614 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-test x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 44 M 2024-07-26T13:25:41.614 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-volume noarch 2:19.0.0-5320.ge8f88bb1.el9 ceph-noarch 274 k 2024-07-26T13:25:41.615 INFO:teuthology.orchestra.run.smithi100.stdout: cephadm noarch 2:19.0.0-5320.ge8f88bb1.el9 ceph-noarch 771 k 2024-07-26T13:25:41.615 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-mirror x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 225 k 2024-07-26T13:25:41.615 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-shell x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 27 k 2024-07-26T13:25:41.615 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-top noarch 2:19.0.0-5320.ge8f88bb1.el9 ceph-noarch 18 k 2024-07-26T13:25:41.615 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-07-26T13:25:41.615 INFO:teuthology.orchestra.run.smithi100.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-07-26T13:25:41.615 INFO:teuthology.orchestra.run.smithi100.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-07-26T13:25:41.615 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs-devel x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 32 k 2024-07-26T13:25:41.615 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs2 x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 765 k 2024-07-26T13:25:41.615 INFO:teuthology.orchestra.run.smithi100.stdout: librados-devel x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 127 k 2024-07-26T13:25:41.615 INFO:teuthology.orchestra.run.smithi100.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-07-26T13:25:41.615 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-07-26T13:25:41.615 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-07-26T13:25:41.616 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-07-26T13:25:41.616 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-07-26T13:25:41.616 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-07-26T13:25:41.616 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cephfs x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 188 k 2024-07-26T13:25:41.616 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rados x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 372 k 2024-07-26T13:25:41.616 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rbd x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 355 k 2024-07-26T13:25:41.616 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rgw x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 116 k 2024-07-26T13:25:41.616 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-fuse x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 86 k 2024-07-26T13:25:41.616 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-mirror x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 3.1 M 2024-07-26T13:25:41.616 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-nbd x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 170 k 2024-07-26T13:25:41.616 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-devel x86_64 6.4.0-3.el9 appstream 96 k 2024-07-26T13:25:41.616 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrading: 2024-07-26T13:25:41.616 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager x86_64 1:1.48.2-2.el9 baseos 2.3 M 2024-07-26T13:25:41.617 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-libnm x86_64 1:1.48.2-2.el9 baseos 1.8 M 2024-07-26T13:25:41.617 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-team x86_64 1:1.48.2-2.el9 baseos 41 k 2024-07-26T13:25:41.617 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-tui x86_64 1:1.48.2-2.el9 baseos 247 k 2024-07-26T13:25:41.617 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-07-26T13:25:41.617 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-07-26T13:25:41.617 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-07-26T13:25:41.617 INFO:teuthology.orchestra.run.smithi100.stdout: librados2 x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 3.4 M 2024-07-26T13:25:41.617 INFO:teuthology.orchestra.run.smithi100.stdout: librbd1 x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 3.1 M 2024-07-26T13:25:41.617 INFO:teuthology.orchestra.run.smithi100.stdout: openssl x86_64 1:3.2.2-2.el9 baseos 1.4 M 2024-07-26T13:25:41.617 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-libs x86_64 1:3.2.2-2.el9 baseos 2.4 M 2024-07-26T13:25:41.617 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs x86_64 6.4.0-3.el9 baseos 1.1 M 2024-07-26T13:25:41.617 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-07-26T13:25:41.617 INFO:teuthology.orchestra.run.smithi100.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-07-26T13:25:41.618 INFO:teuthology.orchestra.run.smithi100.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-07-26T13:25:41.618 INFO:teuthology.orchestra.run.smithi100.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-07-26T13:25:41.618 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-common x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 20 M 2024-07-26T13:25:41.618 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-grafana-dashboards noarch 2:19.0.0-5320.ge8f88bb1.el9 ceph-noarch 34 k 2024-07-26T13:25:41.618 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mds x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 2.3 M 2024-07-26T13:25:41.618 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core noarch 2:19.0.0-5320.ge8f88bb1.el9 ceph-noarch 284 k 2024-07-26T13:25:41.618 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mon x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 4.9 M 2024-07-26T13:25:41.618 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-osd x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 18 M 2024-07-26T13:25:41.618 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-prometheus-alerts noarch 2:19.0.0-5320.ge8f88bb1.el9 ceph-noarch 16 k 2024-07-26T13:25:41.618 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-selinux x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 25 k 2024-07-26T13:25:41.618 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-07-26T13:25:41.618 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-07-26T13:25:41.618 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-07-26T13:25:41.619 INFO:teuthology.orchestra.run.smithi100.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-07-26T13:25:41.619 INFO:teuthology.orchestra.run.smithi100.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-07-26T13:25:41.619 INFO:teuthology.orchestra.run.smithi100.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-07-26T13:25:41.619 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-07-26T13:25:41.619 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-07-26T13:25:41.619 INFO:teuthology.orchestra.run.smithi100.stdout: libcephsqlite x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 158 k 2024-07-26T13:25:41.619 INFO:teuthology.orchestra.run.smithi100.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-07-26T13:25:41.619 INFO:teuthology.orchestra.run.smithi100.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-07-26T13:25:41.619 INFO:teuthology.orchestra.run.smithi100.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-07-26T13:25:41.619 INFO:teuthology.orchestra.run.smithi100.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-07-26T13:25:41.619 INFO:teuthology.orchestra.run.smithi100.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-07-26T13:25:41.619 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1 x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 503 k 2024-07-26T13:25:41.619 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-07-26T13:25:41.620 INFO:teuthology.orchestra.run.smithi100.stdout: librgw2 x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 5.3 M 2024-07-26T13:25:41.620 INFO:teuthology.orchestra.run.smithi100.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-07-26T13:25:41.620 INFO:teuthology.orchestra.run.smithi100.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-07-26T13:25:41.620 INFO:teuthology.orchestra.run.smithi100.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-07-26T13:25:41.620 INFO:teuthology.orchestra.run.smithi100.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-07-26T13:25:41.620 INFO:teuthology.orchestra.run.smithi100.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-07-26T13:25:41.620 INFO:teuthology.orchestra.run.smithi100.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-07-26T13:25:41.620 INFO:teuthology.orchestra.run.smithi100.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-07-26T13:25:41.620 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-07-26T13:25:41.620 INFO:teuthology.orchestra.run.smithi100.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-07-26T13:25:41.620 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-07-26T13:25:41.620 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-07-26T13:25:41.620 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-07-26T13:25:41.621 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-07-26T13:25:41.621 INFO:teuthology.orchestra.run.smithi100.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-07-26T13:25:41.621 INFO:teuthology.orchestra.run.smithi100.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-07-26T13:25:41.621 INFO:teuthology.orchestra.run.smithi100.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-07-26T13:25:41.621 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-07-26T13:25:41.621 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 45 k 2024-07-26T13:25:41.621 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-common x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 138 k 2024-07-26T13:25:41.621 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-07-26T13:25:41.621 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-07-26T13:25:41.621 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-07-26T13:25:41.621 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-07-26T13:25:41.621 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-07-26T13:25:41.621 INFO:teuthology.orchestra.run.smithi100.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-07-26T13:25:41.622 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-07-26T13:25:41.622 INFO:teuthology.orchestra.run.smithi100.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-07-26T13:25:41.622 INFO:teuthology.orchestra.run.smithi100.stdout: python3-google-auth noarch 1:2.32.0-1.el9 epel 216 k 2024-07-26T13:25:41.622 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-07-26T13:25:41.622 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-07-26T13:25:41.622 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-07-26T13:25:41.622 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-07-26T13:25:41.622 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-07-26T13:25:41.622 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-07-26T13:25:41.622 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-07-26T13:25:41.622 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-07-26T13:25:41.622 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-07-26T13:25:41.622 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-07-26T13:25:41.622 INFO:teuthology.orchestra.run.smithi100.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-07-26T13:25:41.623 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-07-26T13:25:41.623 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-07-26T13:25:41.623 INFO:teuthology.orchestra.run.smithi100.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-07-26T13:25:41.623 INFO:teuthology.orchestra.run.smithi100.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-07-26T13:25:41.623 INFO:teuthology.orchestra.run.smithi100.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-07-26T13:25:41.623 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-07-26T13:25:41.623 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-07-26T13:25:41.623 INFO:teuthology.orchestra.run.smithi100.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-07-26T13:25:41.623 INFO:teuthology.orchestra.run.smithi100.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-07-26T13:25:41.623 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-07-26T13:25:41.623 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-07-26T13:25:41.623 INFO:teuthology.orchestra.run.smithi100.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-07-26T13:25:41.623 INFO:teuthology.orchestra.run.smithi100.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-07-26T13:25:41.623 INFO:teuthology.orchestra.run.smithi100.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-07-26T13:25:41.624 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-07-26T13:25:41.624 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-07-26T13:25:41.624 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-07-26T13:25:41.624 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-07-26T13:25:41.624 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-07-26T13:25:41.624 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-07-26T13:25:41.624 INFO:teuthology.orchestra.run.smithi100.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-07-26T13:25:41.624 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-07-26T13:25:41.624 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-07-26T13:25:41.624 INFO:teuthology.orchestra.run.smithi100.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-07-26T13:25:41.624 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-07-26T13:25:41.624 INFO:teuthology.orchestra.run.smithi100.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-07-26T13:25:41.624 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-07-26T13:25:41.625 INFO:teuthology.orchestra.run.smithi100.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-07-26T13:25:41.625 INFO:teuthology.orchestra.run.smithi100.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-07-26T13:25:41.625 INFO:teuthology.orchestra.run.smithi100.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-07-26T13:25:41.625 INFO:teuthology.orchestra.run.smithi100.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-07-26T13:25:41.625 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-07-26T13:25:41.625 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-07-26T13:25:41.625 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-07-26T13:25:41.625 INFO:teuthology.orchestra.run.smithi100.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-07-26T13:25:41.625 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-07-26T13:25:41.625 INFO:teuthology.orchestra.run.smithi100.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-07-26T13:25:41.625 INFO:teuthology.orchestra.run.smithi100.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-07-26T13:25:41.625 INFO:teuthology.orchestra.run.smithi100.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-07-26T13:25:41.625 INFO:teuthology.orchestra.run.smithi100.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-07-26T13:25:41.626 INFO:teuthology.orchestra.run.smithi100.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-07-26T13:25:41.626 INFO:teuthology.orchestra.run.smithi100.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-07-26T13:25:41.626 INFO:teuthology.orchestra.run.smithi100.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-07-26T13:25:41.626 INFO:teuthology.orchestra.run.smithi100.stdout:Installing weak dependencies: 2024-07-26T13:25:41.626 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-07-26T13:25:41.626 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:25:41.626 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-07-26T13:25:41.626 INFO:teuthology.orchestra.run.smithi100.stdout:======================================================================================= 2024-07-26T13:25:41.626 INFO:teuthology.orchestra.run.smithi100.stdout:Install 148 Packages 2024-07-26T13:25:41.626 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrade 12 Packages 2024-07-26T13:25:41.626 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:25:41.627 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 233 M 2024-07-26T13:25:41.627 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-07-26T13:25:41.699 INFO:teuthology.orchestra.run.smithi138.stdout:(14/160): libcephfs2-19.0.0-5320.ge8f88bb1.el9. 7.5 MB/s | 765 kB 00:00 2024-07-26T13:25:41.874 INFO:teuthology.orchestra.run.smithi138.stdout:(15/160): ceph-osd-19.0.0-5320.ge8f88bb1.el9.x8 14 MB/s | 18 MB 00:01 2024-07-26T13:25:41.900 INFO:teuthology.orchestra.run.smithi138.stdout:(16/160): libcephsqlite-19.0.0-5320.ge8f88bb1.e 790 kB/s | 158 kB 00:00 2024-07-26T13:25:41.925 INFO:teuthology.orchestra.run.smithi138.stdout:(17/160): librados-devel-19.0.0-5320.ge8f88bb1. 2.5 MB/s | 127 kB 00:00 2024-07-26T13:25:41.950 INFO:teuthology.orchestra.run.smithi138.stdout:(18/160): libradosstriper1-19.0.0-5320.ge8f88bb 9.8 MB/s | 503 kB 00:00 2024-07-26T13:25:42.001 INFO:teuthology.orchestra.run.smithi138.stdout:(19/160): python3-ceph-argparse-19.0.0-5320.ge8 902 kB/s | 45 kB 00:00 2024-07-26T13:25:42.076 INFO:teuthology.orchestra.run.smithi138.stdout:(20/160): python3-ceph-common-19.0.0-5320.ge8f8 1.8 MB/s | 138 kB 00:00 2024-07-26T13:25:42.159 INFO:teuthology.orchestra.run.smithi138.stdout:(21/160): librgw2-19.0.0-5320.ge8f88bb1.el9.x86 22 MB/s | 5.3 MB 00:00 2024-07-26T13:25:42.185 INFO:teuthology.orchestra.run.smithi138.stdout:(22/160): python3-cephfs-19.0.0-5320.ge8f88bb1. 1.7 MB/s | 188 kB 00:00 2024-07-26T13:25:42.210 INFO:teuthology.orchestra.run.smithi138.stdout:(23/160): python3-rados-19.0.0-5320.ge8f88bb1.e 7.2 MB/s | 372 kB 00:00 2024-07-26T13:25:42.244 INFO:teuthology.orchestra.run.smithi138.stdout:(24/160): python3-rbd-19.0.0-5320.ge8f88bb1.el9 5.9 MB/s | 355 kB 00:00 2024-07-26T13:25:42.269 INFO:teuthology.orchestra.run.smithi138.stdout:(25/160): python3-rgw-19.0.0-5320.ge8f88bb1.el9 1.9 MB/s | 116 kB 00:00 2024-07-26T13:25:42.303 INFO:teuthology.orchestra.run.smithi138.stdout:(26/160): rbd-fuse-19.0.0-5320.ge8f88bb1.el9.x8 1.4 MB/s | 86 kB 00:00 2024-07-26T13:25:42.353 INFO:teuthology.orchestra.run.smithi138.stdout:(27/160): rbd-nbd-19.0.0-5320.ge8f88bb1.el9.x86 3.3 MB/s | 170 kB 00:00 2024-07-26T13:25:42.429 INFO:teuthology.orchestra.run.smithi138.stdout:(28/160): rbd-mirror-19.0.0-5320.ge8f88bb1.el9. 19 MB/s | 3.1 MB 00:00 2024-07-26T13:25:42.454 INFO:teuthology.orchestra.run.smithi138.stdout:(29/160): ceph-grafana-dashboards-19.0.0-5320.g 343 kB/s | 34 kB 00:00 2024-07-26T13:25:42.479 INFO:teuthology.orchestra.run.smithi138.stdout:(30/160): ceph-mgr-cephadm-19.0.0-5320.ge8f88bb 2.9 MB/s | 149 kB 00:00 2024-07-26T13:25:42.697 INFO:teuthology.orchestra.run.smithi100.stdout:(1/160): ceph-19.0.0-5320.ge8f88bb1.el9.x86_64. 25 kB/s | 6.6 kB 00:00 2024-07-26T13:25:42.871 INFO:teuthology.orchestra.run.smithi138.stdout:(31/160): ceph-mgr-diskprediction-local-19.0.0- 19 MB/s | 7.4 MB 00:00 2024-07-26T13:25:42.955 INFO:teuthology.orchestra.run.smithi138.stdout:(32/160): ceph-mgr-modules-core-19.0.0-5320.ge8 3.3 MB/s | 284 kB 00:00 2024-07-26T13:25:42.972 INFO:teuthology.orchestra.run.smithi100.stdout:(2/160): ceph-fuse-19.0.0-5320.ge8f88bb1.el9.x8 3.1 MB/s | 872 kB 00:00 2024-07-26T13:25:43.022 INFO:teuthology.orchestra.run.smithi138.stdout:(33/160): ceph-mgr-rook-19.0.0-5320.ge8f88bb1.e 738 kB/s | 49 kB 00:00 2024-07-26T13:25:43.047 INFO:teuthology.orchestra.run.smithi100.stdout:(3/160): ceph-immutable-object-cache-19.0.0-532 1.9 MB/s | 148 kB 00:00 2024-07-26T13:25:43.197 INFO:teuthology.orchestra.run.smithi138.stdout:(34/160): ceph-mgr-dashboard-19.0.0-5320.ge8f88 19 MB/s | 14 MB 00:00 2024-07-26T13:25:43.214 INFO:teuthology.orchestra.run.smithi100.stdout:(4/160): ceph-mds-19.0.0-5320.ge8f88bb1.el9.x86 14 MB/s | 2.3 MB 00:00 2024-07-26T13:25:43.223 INFO:teuthology.orchestra.run.smithi138.stdout:(35/160): ceph-prometheus-alerts-19.0.0-5320.ge 82 kB/s | 16 kB 00:00 2024-07-26T13:25:43.248 INFO:teuthology.orchestra.run.smithi138.stdout:(36/160): ceph-volume-19.0.0-5320.ge8f88bb1.el9 5.4 MB/s | 274 kB 00:00 2024-07-26T13:25:43.298 INFO:teuthology.orchestra.run.smithi138.stdout:(37/160): cephadm-19.0.0-5320.ge8f88bb1.el9.noa 10 MB/s | 771 kB 00:00 2024-07-26T13:25:43.323 INFO:teuthology.orchestra.run.smithi100.stdout:(5/160): ceph-mgr-19.0.0-5320.ge8f88bb1.el9.x86 14 MB/s | 1.5 MB 00:00 2024-07-26T13:25:43.323 INFO:teuthology.orchestra.run.smithi138.stdout:(38/160): cephfs-top-19.0.0-5320.ge8f88bb1.el9. 241 kB/s | 18 kB 00:00 2024-07-26T13:25:43.569 INFO:teuthology.orchestra.run.smithi167.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-07-26T13:25:43.590 INFO:teuthology.orchestra.run.smithi100.stdout:(6/160): ceph-mon-19.0.0-5320.ge8f88bb1.el9.x86 18 MB/s | 4.9 MB 00:00 2024-07-26T13:25:43.665 INFO:teuthology.orchestra.run.smithi100.stdout:(7/160): ceph-base-19.0.0-5320.ge8f88bb1.el9.x8 4.4 MB/s | 5.5 MB 00:01 2024-07-26T13:25:43.824 INFO:teuthology.orchestra.run.smithi138.stdout:(39/160): ceph-test-19.0.0-5320.ge8f88bb1.el9.x 17 MB/s | 44 MB 00:02 2024-07-26T13:25:44.249 INFO:teuthology.orchestra.run.smithi138.stdout:(40/160): fuse-2.9.9-16.el9.x86_64.rpm 84 kB/s | 80 kB 00:00 2024-07-26T13:25:44.407 INFO:teuthology.orchestra.run.smithi100.stdout:(8/160): ceph-osd-19.0.0-5320.ge8f88bb1.el9.x86 22 MB/s | 18 MB 00:00 2024-07-26T13:25:44.457 INFO:teuthology.orchestra.run.smithi138.stdout:(41/160): libquadmath-11.4.1-3.el9.x86_64.rpm 336 kB/s | 195 kB 00:00 2024-07-26T13:25:44.508 INFO:teuthology.orchestra.run.smithi138.stdout:(42/160): python3-cffi-1.14.5-5.el9.x86_64.rpm 980 kB/s | 253 kB 00:00 2024-07-26T13:25:44.533 INFO:teuthology.orchestra.run.smithi138.stdout:(43/160): libgfortran-11.4.1-3.el9.x86_64.rpm 664 kB/s | 803 kB 00:01 2024-07-26T13:25:44.591 INFO:teuthology.orchestra.run.smithi100.stdout:(9/160): ceph-common-19.0.0-5320.ge8f88bb1.el9. 9.2 MB/s | 20 MB 00:02 2024-07-26T13:25:44.616 INFO:teuthology.orchestra.run.smithi100.stdout:(10/160): ceph-selinux-19.0.0-5320.ge8f88bb1.el 121 kB/s | 25 kB 00:00 2024-07-26T13:25:44.742 INFO:teuthology.orchestra.run.smithi100.stdout:(11/160): ceph-radosgw-19.0.0-5320.ge8f88bb1.el 9.7 MB/s | 10 MB 00:01 2024-07-26T13:25:44.750 INFO:teuthology.orchestra.run.smithi138.stdout:(44/160): python3-cryptography-36.0.1-4.el9.x86 4.2 MB/s | 1.2 MB 00:00 2024-07-26T13:25:44.767 INFO:teuthology.orchestra.run.smithi100.stdout:(12/160): cephfs-mirror-19.0.0-5320.ge8f88bb1.e 1.5 MB/s | 225 kB 00:00 2024-07-26T13:25:44.775 INFO:teuthology.orchestra.run.smithi138.stdout:(45/160): python3-ply-3.11-14.el9.noarch.rpm 399 kB/s | 106 kB 00:00 2024-07-26T13:25:44.792 INFO:teuthology.orchestra.run.smithi100.stdout:(13/160): cephfs-shell-19.0.0-5320.ge8f88bb1.el 541 kB/s | 27 kB 00:00 2024-07-26T13:25:44.800 INFO:teuthology.orchestra.run.smithi138.stdout:(46/160): python3-pycparser-2.20-6.el9.noarch.r 506 kB/s | 135 kB 00:00 2024-07-26T13:25:44.817 INFO:teuthology.orchestra.run.smithi100.stdout:(14/160): libcephfs-devel-19.0.0-5320.ge8f88bb1 632 kB/s | 32 kB 00:00 2024-07-26T13:25:44.843 INFO:teuthology.orchestra.run.smithi138.stdout:(47/160): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-07-26T13:25:44.868 INFO:teuthology.orchestra.run.smithi100.stdout:(15/160): libcephfs2-19.0.0-5320.ge8f88bb1.el9. 9.9 MB/s | 765 kB 00:00 2024-07-26T13:25:44.893 INFO:teuthology.orchestra.run.smithi100.stdout:(16/160): libcephsqlite-19.0.0-5320.ge8f88bb1.e 2.1 MB/s | 158 kB 00:00 2024-07-26T13:25:44.909 INFO:teuthology.orchestra.run.smithi138.stdout:(48/160): python3-urllib3-1.26.5-6.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-07-26T13:25:44.935 INFO:teuthology.orchestra.run.smithi100.stdout:(17/160): librados-devel-19.0.0-5320.ge8f88bb1. 1.9 MB/s | 127 kB 00:00 2024-07-26T13:25:44.968 INFO:teuthology.orchestra.run.smithi100.stdout:(18/160): libradosstriper1-19.0.0-5320.ge8f88bb 6.5 MB/s | 503 kB 00:00 2024-07-26T13:25:45.036 INFO:teuthology.orchestra.run.smithi100.stdout:(19/160): python3-ceph-argparse-19.0.0-5320.ge8 677 kB/s | 45 kB 00:00 2024-07-26T13:25:45.085 INFO:teuthology.orchestra.run.smithi138.stdout:(49/160): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-07-26T13:25:45.102 INFO:teuthology.orchestra.run.smithi100.stdout:(20/160): python3-ceph-common-19.0.0-5320.ge8f8 2.0 MB/s | 138 kB 00:00 2024-07-26T13:25:45.110 INFO:teuthology.orchestra.run.smithi138.stdout:(50/160): boost-program-options-1.75.0-8.el9.x8 534 kB/s | 107 kB 00:00 2024-07-26T13:25:45.144 INFO:teuthology.orchestra.run.smithi138.stdout:(51/160): elfutils-libelf-devel-0.191-4.el9.x86 371 kB/s | 22 kB 00:00 2024-07-26T13:25:45.177 INFO:teuthology.orchestra.run.smithi138.stdout:(52/160): bison-3.7.4-5.el9.x86_64.rpm 2.8 MB/s | 943 kB 00:00 2024-07-26T13:25:45.178 INFO:teuthology.orchestra.run.smithi100.stdout:(21/160): python3-cephfs-19.0.0-5320.ge8f88bb1. 2.5 MB/s | 188 kB 00:00 2024-07-26T13:25:45.211 INFO:teuthology.orchestra.run.smithi138.stdout:(53/160): flexiblas-3.0.4-8.el9.x86_64.rpm 487 kB/s | 33 kB 00:00 2024-07-26T13:25:45.245 INFO:teuthology.orchestra.run.smithi138.stdout:(54/160): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-07-26T13:25:45.270 INFO:teuthology.orchestra.run.smithi138.stdout:(55/160): flexiblas-openblas-openmp-3.0.4-8.el9 312 kB/s | 18 kB 00:00 2024-07-26T13:25:45.286 INFO:teuthology.orchestra.run.smithi100.stdout:(22/160): librgw2-19.0.0-5320.ge8f88bb1.el9.x86 15 MB/s | 5.3 MB 00:00 2024-07-26T13:25:45.304 INFO:teuthology.orchestra.run.smithi138.stdout:(56/160): libaio-devel-0.3.111-13.el9.x86_64.rp 189 kB/s | 11 kB 00:00 2024-07-26T13:25:45.320 INFO:teuthology.orchestra.run.smithi100.stdout:(23/160): python3-rados-19.0.0-5320.ge8f88bb1.e 2.6 MB/s | 372 kB 00:00 2024-07-26T13:25:45.345 INFO:teuthology.orchestra.run.smithi100.stdout:(24/160): python3-rbd-19.0.0-5320.ge8f88bb1.el9 5.9 MB/s | 355 kB 00:00 2024-07-26T13:25:45.354 INFO:teuthology.orchestra.run.smithi138.stdout:(57/160): flexiblas-netlib-3.0.4-8.el9.x86_64.r 17 MB/s | 3.0 MB 00:00 2024-07-26T13:25:45.371 INFO:teuthology.orchestra.run.smithi100.stdout:(25/160): python3-rgw-19.0.0-5320.ge8f88bb1.el9 2.3 MB/s | 116 kB 00:00 2024-07-26T13:25:45.371 INFO:teuthology.orchestra.run.smithi138.stdout:(58/160): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-07-26T13:25:45.396 INFO:teuthology.orchestra.run.smithi100.stdout:(26/160): rbd-fuse-19.0.0-5320.ge8f88bb1.el9.x8 1.7 MB/s | 86 kB 00:00 2024-07-26T13:25:45.396 INFO:teuthology.orchestra.run.smithi138.stdout:(59/160): librabbitmq-0.11.0-7.el9.x86_64.rpm 492 kB/s | 45 kB 00:00 2024-07-26T13:25:45.439 INFO:teuthology.orchestra.run.smithi138.stdout:(60/160): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 662 kB 00:00 2024-07-26T13:25:45.465 INFO:teuthology.orchestra.run.smithi138.stdout:(61/160): libuuid-devel-2.37.4-18.el9.x86_64.rp 380 kB/s | 25 kB 00:00 2024-07-26T13:25:45.472 INFO:teuthology.orchestra.run.smithi100.stdout:(27/160): rbd-nbd-19.0.0-5320.ge8f88bb1.el9.x86 2.2 MB/s | 170 kB 00:00 2024-07-26T13:25:45.489 INFO:teuthology.orchestra.run.smithi138.stdout:(62/160): libtool-2.4.6-46.el9.x86_64.rpm 6.2 MB/s | 581 kB 00:00 2024-07-26T13:25:45.514 INFO:teuthology.orchestra.run.smithi138.stdout:(63/160): libzstd-devel-1.5.1-2.el9.x86_64.rpm 628 kB/s | 47 kB 00:00 2024-07-26T13:25:45.530 INFO:teuthology.orchestra.run.smithi100.stdout:(28/160): rbd-mirror-19.0.0-5320.ge8f88bb1.el9. 19 MB/s | 3.1 MB 00:00 2024-07-26T13:25:45.539 INFO:teuthology.orchestra.run.smithi138.stdout:(64/160): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-07-26T13:25:45.555 INFO:teuthology.orchestra.run.smithi100.stdout:(29/160): ceph-grafana-dashboards-19.0.0-5320.g 412 kB/s | 34 kB 00:00 2024-07-26T13:25:45.565 INFO:teuthology.orchestra.run.smithi138.stdout:(65/160): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-07-26T13:25:45.589 INFO:teuthology.orchestra.run.smithi100.stdout:(30/160): ceph-mgr-cephadm-19.0.0-5320.ge8f88bb 2.5 MB/s | 149 kB 00:00 2024-07-26T13:25:45.590 INFO:teuthology.orchestra.run.smithi138.stdout:(66/160): openblas-0.3.21-2.el9.x86_64.rpm 466 kB/s | 35 kB 00:00 2024-07-26T13:25:45.674 INFO:teuthology.orchestra.run.smithi138.stdout:(67/160): postgresql-13.14-1.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-07-26T13:25:45.766 INFO:teuthology.orchestra.run.smithi138.stdout:(68/160): postgresql-contrib-13.14-1.el9.x86_64 9.7 MB/s | 911 kB 00:00 2024-07-26T13:25:45.815 INFO:teuthology.orchestra.run.smithi167.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-26T13:25:45.816 INFO:teuthology.orchestra.run.smithi167.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-26T13:25:45.827 INFO:teuthology.orchestra.run.smithi167.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-07-26T13:25:45.827 INFO:teuthology.orchestra.run.smithi167.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-07-26T13:25:45.828 INFO:teuthology.orchestra.run.smithi167.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-07-26T13:25:45.830 INFO:teuthology.orchestra.run.smithi167.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-07-26T13:25:45.831 INFO:teuthology.orchestra.run.smithi167.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-07-26T13:25:45.833 INFO:teuthology.orchestra.run.smithi167.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-07-26T13:25:45.834 INFO:teuthology.orchestra.run.smithi167.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-07-26T13:25:45.841 INFO:teuthology.orchestra.run.smithi138.stdout:(69/160): openblas-openmp-0.3.21-2.el9.x86_64.r 16 MB/s | 4.7 MB 00:00 2024-07-26T13:25:45.866 INFO:teuthology.orchestra.run.smithi138.stdout:(70/160): postgresql-private-libs-13.14-1.el9.x 1.3 MB/s | 133 kB 00:00 2024-07-26T13:25:45.897 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-07-26T13:25:45.907 INFO:teuthology.orchestra.run.smithi167.stdout:======================================================================================= 2024-07-26T13:25:45.907 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2024-07-26T13:25:45.908 INFO:teuthology.orchestra.run.smithi167.stdout:======================================================================================= 2024-07-26T13:25:45.908 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2024-07-26T13:25:45.908 INFO:teuthology.orchestra.run.smithi167.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-07-26T13:25:45.908 INFO:teuthology.orchestra.run.smithi167.stdout: ceph x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 6.6 k 2024-07-26T13:25:45.908 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 5.5 M 2024-07-26T13:25:45.908 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 872 k 2024-07-26T13:25:45.908 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 148 k 2024-07-26T13:25:45.908 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 1.5 M 2024-07-26T13:25:45.908 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm noarch 2:19.0.0-5320.ge8f88bb1.el9 ceph-noarch 149 k 2024-07-26T13:25:45.908 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard noarch 2:19.0.0-5320.ge8f88bb1.el9 ceph-noarch 14 M 2024-07-26T13:25:45.908 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-5320.ge8f88bb1.el9 ceph-noarch 7.4 M 2024-07-26T13:25:45.909 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook noarch 2:19.0.0-5320.ge8f88bb1.el9 ceph-noarch 49 k 2024-07-26T13:25:45.909 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 10 M 2024-07-26T13:25:45.909 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 44 M 2024-07-26T13:25:45.909 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume noarch 2:19.0.0-5320.ge8f88bb1.el9 ceph-noarch 274 k 2024-07-26T13:25:45.909 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm noarch 2:19.0.0-5320.ge8f88bb1.el9 ceph-noarch 771 k 2024-07-26T13:25:45.909 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-mirror x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 225 k 2024-07-26T13:25:45.909 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-shell x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 27 k 2024-07-26T13:25:45.909 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-top noarch 2:19.0.0-5320.ge8f88bb1.el9 ceph-noarch 18 k 2024-07-26T13:25:45.909 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-07-26T13:25:45.909 INFO:teuthology.orchestra.run.smithi167.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-07-26T13:25:45.910 INFO:teuthology.orchestra.run.smithi167.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-07-26T13:25:45.910 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 32 k 2024-07-26T13:25:45.910 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2 x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 765 k 2024-07-26T13:25:45.910 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 127 k 2024-07-26T13:25:45.910 INFO:teuthology.orchestra.run.smithi167.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-07-26T13:25:45.910 INFO:teuthology.orchestra.run.smithi167.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-07-26T13:25:45.910 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-07-26T13:25:45.910 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-07-26T13:25:45.910 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-07-26T13:25:45.910 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-07-26T13:25:45.911 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 188 k 2024-07-26T13:25:45.911 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 372 k 2024-07-26T13:25:45.911 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 355 k 2024-07-26T13:25:45.911 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 116 k 2024-07-26T13:25:45.911 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 86 k 2024-07-26T13:25:45.911 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 3.1 M 2024-07-26T13:25:45.911 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 170 k 2024-07-26T13:25:45.911 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs-devel x86_64 6.4.0-3.el9 appstream 96 k 2024-07-26T13:25:45.911 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrading: 2024-07-26T13:25:45.911 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager x86_64 1:1.48.2-2.el9 baseos 2.3 M 2024-07-26T13:25:45.912 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-libnm x86_64 1:1.48.2-2.el9 baseos 1.8 M 2024-07-26T13:25:45.912 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-team x86_64 1:1.48.2-2.el9 baseos 41 k 2024-07-26T13:25:45.912 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-tui x86_64 1:1.48.2-2.el9 baseos 247 k 2024-07-26T13:25:45.912 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-07-26T13:25:45.912 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-07-26T13:25:45.912 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-07-26T13:25:45.913 INFO:teuthology.orchestra.run.smithi167.stdout: librados2 x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 3.4 M 2024-07-26T13:25:45.914 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1 x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 3.1 M 2024-07-26T13:25:45.914 INFO:teuthology.orchestra.run.smithi167.stdout: openssl x86_64 1:3.2.2-2.el9 baseos 1.4 M 2024-07-26T13:25:45.914 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-libs x86_64 1:3.2.2-2.el9 baseos 2.4 M 2024-07-26T13:25:45.914 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs x86_64 6.4.0-3.el9 baseos 1.1 M 2024-07-26T13:25:45.914 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2024-07-26T13:25:45.914 INFO:teuthology.orchestra.run.smithi167.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-07-26T13:25:45.914 INFO:teuthology.orchestra.run.smithi167.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-07-26T13:25:45.914 INFO:teuthology.orchestra.run.smithi167.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-07-26T13:25:45.914 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 20 M 2024-07-26T13:25:45.914 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards noarch 2:19.0.0-5320.ge8f88bb1.el9 ceph-noarch 34 k 2024-07-26T13:25:45.915 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 2.3 M 2024-07-26T13:25:45.915 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core noarch 2:19.0.0-5320.ge8f88bb1.el9 ceph-noarch 284 k 2024-07-26T13:25:45.915 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 4.9 M 2024-07-26T13:25:45.915 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 18 M 2024-07-26T13:25:45.915 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts noarch 2:19.0.0-5320.ge8f88bb1.el9 ceph-noarch 16 k 2024-07-26T13:25:45.915 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 25 k 2024-07-26T13:25:45.915 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-07-26T13:25:45.916 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-07-26T13:25:45.916 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-07-26T13:25:45.916 INFO:teuthology.orchestra.run.smithi167.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-07-26T13:25:45.916 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-07-26T13:25:45.916 INFO:teuthology.orchestra.run.smithi167.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-07-26T13:25:45.916 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-07-26T13:25:45.916 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-07-26T13:25:45.916 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 158 k 2024-07-26T13:25:45.916 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-07-26T13:25:45.916 INFO:teuthology.orchestra.run.smithi167.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-07-26T13:25:45.917 INFO:teuthology.orchestra.run.smithi167.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-07-26T13:25:45.917 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-07-26T13:25:45.917 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-07-26T13:25:45.917 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1 x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 503 k 2024-07-26T13:25:45.917 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-07-26T13:25:45.917 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2 x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 5.3 M 2024-07-26T13:25:45.917 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-07-26T13:25:45.917 INFO:teuthology.orchestra.run.smithi167.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-07-26T13:25:45.917 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-07-26T13:25:45.917 INFO:teuthology.orchestra.run.smithi167.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-07-26T13:25:45.917 INFO:teuthology.orchestra.run.smithi167.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-07-26T13:25:45.917 INFO:teuthology.orchestra.run.smithi167.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-07-26T13:25:45.917 INFO:teuthology.orchestra.run.smithi167.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-07-26T13:25:45.918 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-07-26T13:25:45.918 INFO:teuthology.orchestra.run.smithi167.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-07-26T13:25:45.918 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-07-26T13:25:45.918 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-07-26T13:25:45.918 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-07-26T13:25:45.918 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-07-26T13:25:45.918 INFO:teuthology.orchestra.run.smithi167.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-07-26T13:25:45.918 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-07-26T13:25:45.918 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-07-26T13:25:45.918 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-07-26T13:25:45.918 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 45 k 2024-07-26T13:25:45.918 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common x86_64 2:19.0.0-5320.ge8f88bb1.el9 ceph 138 k 2024-07-26T13:25:45.918 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-07-26T13:25:45.919 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-07-26T13:25:45.919 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-07-26T13:25:45.919 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-07-26T13:25:45.919 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-07-26T13:25:45.919 INFO:teuthology.orchestra.run.smithi167.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-07-26T13:25:45.919 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-07-26T13:25:45.919 INFO:teuthology.orchestra.run.smithi167.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-07-26T13:25:45.919 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth noarch 1:2.32.0-1.el9 epel 216 k 2024-07-26T13:25:45.919 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-07-26T13:25:45.919 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-07-26T13:25:45.919 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-07-26T13:25:45.919 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-07-26T13:25:45.919 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-07-26T13:25:45.920 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-07-26T13:25:45.920 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-07-26T13:25:45.920 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-07-26T13:25:45.920 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-07-26T13:25:45.920 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-07-26T13:25:45.920 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-07-26T13:25:45.920 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-07-26T13:25:45.920 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-07-26T13:25:45.920 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-07-26T13:25:45.920 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-07-26T13:25:45.920 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-07-26T13:25:45.920 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-07-26T13:25:45.920 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-07-26T13:25:45.921 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-07-26T13:25:45.921 INFO:teuthology.orchestra.run.smithi167.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-07-26T13:25:45.921 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-07-26T13:25:45.921 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-07-26T13:25:45.921 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-07-26T13:25:45.921 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-07-26T13:25:45.921 INFO:teuthology.orchestra.run.smithi167.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-07-26T13:25:45.921 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-07-26T13:25:45.921 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-07-26T13:25:45.921 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-07-26T13:25:45.921 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-07-26T13:25:45.921 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-07-26T13:25:45.921 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-07-26T13:25:45.921 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-07-26T13:25:45.922 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-07-26T13:25:45.922 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-07-26T13:25:45.922 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-07-26T13:25:45.922 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-07-26T13:25:45.922 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-07-26T13:25:45.922 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-07-26T13:25:45.922 INFO:teuthology.orchestra.run.smithi167.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-07-26T13:25:45.922 INFO:teuthology.orchestra.run.smithi167.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-07-26T13:25:45.922 INFO:teuthology.orchestra.run.smithi167.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-07-26T13:25:45.922 INFO:teuthology.orchestra.run.smithi167.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-07-26T13:25:45.922 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-07-26T13:25:45.922 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-07-26T13:25:45.923 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-07-26T13:25:45.923 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-07-26T13:25:45.923 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-07-26T13:25:45.923 INFO:teuthology.orchestra.run.smithi167.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-07-26T13:25:45.923 INFO:teuthology.orchestra.run.smithi167.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-07-26T13:25:45.923 INFO:teuthology.orchestra.run.smithi167.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-07-26T13:25:45.924 INFO:teuthology.orchestra.run.smithi167.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-07-26T13:25:45.924 INFO:teuthology.orchestra.run.smithi167.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-07-26T13:25:45.924 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-07-26T13:25:45.924 INFO:teuthology.orchestra.run.smithi167.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-07-26T13:25:45.924 INFO:teuthology.orchestra.run.smithi167.stdout:Installing weak dependencies: 2024-07-26T13:25:45.924 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-07-26T13:25:45.924 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-07-26T13:25:45.924 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2024-07-26T13:25:45.924 INFO:teuthology.orchestra.run.smithi167.stdout:======================================================================================= 2024-07-26T13:25:45.924 INFO:teuthology.orchestra.run.smithi167.stdout:Install 148 Packages 2024-07-26T13:25:45.924 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrade 12 Packages 2024-07-26T13:25:45.925 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-07-26T13:25:45.925 INFO:teuthology.orchestra.run.smithi138.stdout:(71/160): openssl-devel-3.2.2-2.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-07-26T13:25:45.926 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 233 M 2024-07-26T13:25:45.926 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2024-07-26T13:25:45.958 INFO:teuthology.orchestra.run.smithi138.stdout:(72/160): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-07-26T13:25:45.984 INFO:teuthology.orchestra.run.smithi138.stdout:(73/160): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-07-26T13:25:46.042 INFO:teuthology.orchestra.run.smithi138.stdout:(74/160): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-07-26T13:25:46.114 INFO:teuthology.orchestra.run.smithi100.stdout:(31/160): ceph-mgr-diskprediction-local-19.0.0- 14 MB/s | 7.4 MB 00:00 2024-07-26T13:25:46.143 INFO:teuthology.orchestra.run.smithi138.stdout:(75/160): postgresql-server-13.14-1.el9.x86_64. 19 MB/s | 5.8 MB 00:00 2024-07-26T13:25:46.164 INFO:teuthology.orchestra.run.smithi100.stdout:(32/160): ceph-mgr-modules-core-19.0.0-5320.ge8 5.6 MB/s | 284 kB 00:00 2024-07-26T13:25:46.168 INFO:teuthology.orchestra.run.smithi138.stdout:(76/160): python3-devel-3.9.18-3.el9.x86_64.rpm 1.9 MB/s | 244 kB 00:00 2024-07-26T13:25:46.202 INFO:teuthology.orchestra.run.smithi138.stdout:(77/160): python3-jinja2-2.11.3-6.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-07-26T13:25:46.227 INFO:teuthology.orchestra.run.smithi138.stdout:(78/160): python3-jsonpatch-1.21-16.el9.noarch. 436 kB/s | 26 kB 00:00 2024-07-26T13:25:46.303 INFO:teuthology.orchestra.run.smithi138.stdout:(79/160): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-07-26T13:25:46.319 INFO:teuthology.orchestra.run.smithi138.stdout:(80/160): python3-jsonpointer-2.0-4.el9.noarch. 163 kB/s | 19 kB 00:00 2024-07-26T13:25:46.347 INFO:teuthology.orchestra.run.smithi138.stdout:(81/160): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-07-26T13:25:46.372 INFO:teuthology.orchestra.run.smithi138.stdout:(82/160): python3-markupsafe-1.1.1-12.el9.x86_6 500 kB/s | 35 kB 00:00 2024-07-26T13:25:46.405 INFO:teuthology.orchestra.run.smithi138.stdout:(83/160): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-07-26T13:25:46.431 INFO:teuthology.orchestra.run.smithi138.stdout:(84/160): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-07-26T13:25:46.464 INFO:teuthology.orchestra.run.smithi138.stdout:(85/160): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-07-26T13:25:46.489 INFO:teuthology.orchestra.run.smithi138.stdout:(86/160): python3-prettytable-0.7.2-27.el9.noar 720 kB/s | 42 kB 00:00 2024-07-26T13:25:46.573 INFO:teuthology.orchestra.run.smithi138.stdout:(87/160): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 5.1 MB 00:00 2024-07-26T13:25:46.598 INFO:teuthology.orchestra.run.smithi138.stdout:(88/160): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-07-26T13:25:46.607 INFO:teuthology.orchestra.run.smithi100.stdout:(33/160): ceph-test-19.0.0-5320.ge8f88bb1.el9.x 22 MB/s | 44 MB 00:02 2024-07-26T13:25:46.624 INFO:teuthology.orchestra.run.smithi138.stdout:(89/160): python3-protobuf-3.14.0-13.el9.noarch 1.7 MB/s | 269 kB 00:00 2024-07-26T13:25:46.632 INFO:teuthology.orchestra.run.smithi100.stdout:(34/160): ceph-mgr-rook-19.0.0-5320.ge8f88bb1.e 105 kB/s | 49 kB 00:00 2024-07-26T13:25:46.649 INFO:teuthology.orchestra.run.smithi138.stdout:(90/160): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-07-26T13:25:46.657 INFO:teuthology.orchestra.run.smithi100.stdout:(35/160): ceph-prometheus-alerts-19.0.0-5320.ge 329 kB/s | 16 kB 00:00 2024-07-26T13:25:46.674 INFO:teuthology.orchestra.run.smithi138.stdout:(91/160): python3-pytz-2021.1-5.el9.noarch.rpm 674 kB/s | 51 kB 00:00 2024-07-26T13:25:46.682 INFO:teuthology.orchestra.run.smithi100.stdout:(36/160): ceph-volume-19.0.0-5320.ge8f88bb1.el9 5.3 MB/s | 274 kB 00:00 2024-07-26T13:25:46.700 INFO:teuthology.orchestra.run.smithi138.stdout:(92/160): python3-requests-oauthlib-1.3.0-12.el 710 kB/s | 54 kB 00:00 2024-07-26T13:25:46.716 INFO:teuthology.orchestra.run.smithi100.stdout:(37/160): cephadm-19.0.0-5320.ge8f88bb1.el9.noa 13 MB/s | 771 kB 00:00 2024-07-26T13:25:46.725 INFO:teuthology.orchestra.run.smithi138.stdout:(93/160): python3-toml-0.10.2-6.el9.noarch.rpm 832 kB/s | 42 kB 00:00 2024-07-26T13:25:46.741 INFO:teuthology.orchestra.run.smithi100.stdout:(38/160): cephfs-top-19.0.0-5320.ge8f88bb1.el9. 311 kB/s | 18 kB 00:00 2024-07-26T13:25:46.758 INFO:teuthology.orchestra.run.smithi138.stdout:(94/160): python3-wcwidth-0.2.5-8.el9.noarch.rp 757 kB/s | 44 kB 00:00 2024-07-26T13:25:46.784 INFO:teuthology.orchestra.run.smithi138.stdout:(95/160): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-07-26T13:25:46.811 INFO:teuthology.orchestra.run.smithi167.stdout:(1/160): ceph-19.0.0-5320.ge8f88bb1.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-07-26T13:25:46.818 INFO:teuthology.orchestra.run.smithi138.stdout:(96/160): qatlib-service-24.02.0-1.el9.x86_64.r 612 kB/s | 36 kB 00:00 2024-07-26T13:25:46.843 INFO:teuthology.orchestra.run.smithi138.stdout:(97/160): qatzip-libs-1.2.0-1.el9.x86_64.rpm 810 kB/s | 47 kB 00:00 2024-07-26T13:25:46.885 INFO:teuthology.orchestra.run.smithi138.stdout:(98/160): socat-1.7.4.1-6.el9.x86_64.rpm 4.4 MB/s | 304 kB 00:00 2024-07-26T13:25:46.892 INFO:teuthology.orchestra.run.smithi100.stdout:(39/160): ceph-mgr-dashboard-19.0.0-5320.ge8f88 10 MB/s | 14 MB 00:01 2024-07-26T13:25:46.910 INFO:teuthology.orchestra.run.smithi138.stdout:(99/160): uuid-1.6.2-55.el9.x86_64.rpm 862 kB/s | 58 kB 00:00 2024-07-26T13:25:46.943 INFO:teuthology.orchestra.run.smithi138.stdout:(100/160): xfsprogs-devel-6.4.0-3.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-07-26T13:25:46.969 INFO:teuthology.orchestra.run.smithi138.stdout:(101/160): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-07-26T13:25:47.002 INFO:teuthology.orchestra.run.smithi138.stdout:(102/160): zlib-devel-1.2.11-41.el9.x86_64.rpm 764 kB/s | 45 kB 00:00 2024-07-26T13:25:47.042 INFO:teuthology.orchestra.run.smithi100.stdout:(40/160): fuse-2.9.9-16.el9.x86_64.rpm 245 kB/s | 80 kB 00:00 2024-07-26T13:25:47.045 INFO:teuthology.orchestra.run.smithi167.stdout:(2/160): ceph-fuse-19.0.0-5320.ge8f88bb1.el9.x8 3.7 MB/s | 872 kB 00:00 2024-07-26T13:25:47.109 INFO:teuthology.orchestra.run.smithi100.stdout:(41/160): libquadmath-11.4.1-3.el9.x86_64.rpm 901 kB/s | 195 kB 00:00 2024-07-26T13:25:47.120 INFO:teuthology.orchestra.run.smithi167.stdout:(3/160): ceph-base-19.0.0-5320.ge8f88bb1.el9.x8 11 MB/s | 5.5 MB 00:00 2024-07-26T13:25:47.142 INFO:teuthology.orchestra.run.smithi100.stdout:(42/160): libgfortran-11.4.1-3.el9.x86_64.rpm 2.0 MB/s | 803 kB 00:00 2024-07-26T13:25:47.154 INFO:teuthology.orchestra.run.smithi167.stdout:(4/160): ceph-immutable-object-cache-19.0.0-532 1.3 MB/s | 148 kB 00:00 2024-07-26T13:25:47.168 INFO:teuthology.orchestra.run.smithi100.stdout:(43/160): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.0 MB/s | 253 kB 00:00 2024-07-26T13:25:47.193 INFO:teuthology.orchestra.run.smithi100.stdout:(44/160): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-07-26T13:25:47.211 INFO:teuthology.orchestra.run.smithi138.stdout:(103/160): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-07-26T13:25:47.227 INFO:teuthology.orchestra.run.smithi100.stdout:(45/160): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-07-26T13:25:47.255 INFO:teuthology.orchestra.run.smithi167.stdout:(5/160): ceph-mds-19.0.0-5320.ge8f88bb1.el9.x86 17 MB/s | 2.3 MB 00:00 2024-07-26T13:25:47.277 INFO:teuthology.orchestra.run.smithi100.stdout:(46/160): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-07-26T13:25:47.295 INFO:teuthology.orchestra.run.smithi138.stdout:(104/160): abseil-cpp-20211102.0-3.el9.x86_64.r 6.5 MB/s | 548 kB 00:00 2024-07-26T13:25:47.302 INFO:teuthology.orchestra.run.smithi100.stdout:(47/160): python3-urllib3-1.26.5-6.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-07-26T13:25:47.313 INFO:teuthology.orchestra.run.smithi167.stdout:(6/160): ceph-mgr-19.0.0-5320.ge8f88bb1.el9.x86 9.6 MB/s | 1.5 MB 00:00 2024-07-26T13:25:47.344 INFO:teuthology.orchestra.run.smithi100.stdout:(48/160): python3-cryptography-36.0.1-4.el9.x86 5.2 MB/s | 1.2 MB 00:00 2024-07-26T13:25:47.362 INFO:teuthology.orchestra.run.smithi138.stdout:(105/160): gperftools-libs-2.9.1-2.el9.x86_64.r 7.3 MB/s | 309 kB 00:00 2024-07-26T13:25:47.387 INFO:teuthology.orchestra.run.smithi138.stdout:(106/160): grpc-data-1.46.7-10.el9.noarch.rpm 779 kB/s | 19 kB 00:00 2024-07-26T13:25:47.472 INFO:teuthology.orchestra.run.smithi167.stdout:(7/160): ceph-mon-19.0.0-5320.ge8f88bb1.el9.x86 23 MB/s | 4.9 MB 00:00 2024-07-26T13:25:47.562 INFO:teuthology.orchestra.run.smithi138.stdout:(107/160): libarrow-9.0.0-11.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-07-26T13:25:47.587 INFO:teuthology.orchestra.run.smithi138.stdout:(108/160): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.1 MB/s | 26 kB 00:00 2024-07-26T13:25:47.613 INFO:teuthology.orchestra.run.smithi138.stdout:(109/160): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-07-26T13:25:47.638 INFO:teuthology.orchestra.run.smithi138.stdout:(110/160): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-07-26T13:25:47.672 INFO:teuthology.orchestra.run.smithi138.stdout:(111/160): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-07-26T13:25:47.722 INFO:teuthology.orchestra.run.smithi138.stdout:(112/160): parquet-libs-9.0.0-11.el9.x86_64.rpm 16 MB/s | 839 kB 00:00 2024-07-26T13:25:47.748 INFO:teuthology.orchestra.run.smithi138.stdout:(113/160): lua-devel-5.4.4-4.el9.x86_64.rpm 29 kB/s | 22 kB 00:00 2024-07-26T13:25:47.806 INFO:teuthology.orchestra.run.smithi138.stdout:(114/160): python3-asyncssh-2.13.2-5.el9.noarch 6.4 MB/s | 548 kB 00:00 2024-07-26T13:25:47.831 INFO:teuthology.orchestra.run.smithi138.stdout:(115/160): python3-cachetools-4.2.4-1.el9.noarc 1.4 MB/s | 32 kB 00:00 2024-07-26T13:25:47.856 INFO:teuthology.orchestra.run.smithi138.stdout:(116/160): python3-bcrypt-3.2.2-1.el9.x86_64.rp 399 kB/s | 43 kB 00:00 2024-07-26T13:25:47.870 INFO:teuthology.orchestra.run.smithi100.stdout:(49/160): boost-program-options-1.75.0-8.el9.x8 203 kB/s | 107 kB 00:00 2024-07-26T13:25:47.881 INFO:teuthology.orchestra.run.smithi138.stdout:(117/160): python3-certifi-2023.05.07-4.el9.noa 282 kB/s | 14 kB 00:00 2024-07-26T13:25:47.915 INFO:teuthology.orchestra.run.smithi138.stdout:(118/160): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-07-26T13:25:47.939 INFO:teuthology.orchestra.run.smithi167.stdout:(8/160): ceph-radosgw-19.0.0-5320.ge8f88bb1.el9 22 MB/s | 10 MB 00:00 2024-07-26T13:25:47.940 INFO:teuthology.orchestra.run.smithi138.stdout:(119/160): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-07-26T13:25:47.975 INFO:teuthology.orchestra.run.smithi138.stdout:(120/160): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.0 MB/s | 303 kB 00:00 2024-07-26T13:25:48.000 INFO:teuthology.orchestra.run.smithi138.stdout:(121/160): python3-google-auth-2.32.0-1.el9.noa 3.6 MB/s | 216 kB 00:00 2024-07-26T13:25:48.020 INFO:teuthology.orchestra.run.smithi100.stdout:(50/160): automake-1.16.2-8.el9.noarch.rpm 928 kB/s | 689 kB 00:00 2024-07-26T13:25:48.033 INFO:teuthology.orchestra.run.smithi138.stdout:(122/160): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-07-26T13:25:48.045 INFO:teuthology.orchestra.run.smithi100.stdout:(51/160): elfutils-libelf-devel-0.191-4.el9.x86 124 kB/s | 22 kB 00:00 2024-07-26T13:25:48.050 INFO:teuthology.orchestra.run.smithi138.stdout:(123/160): python3-jaraco-8.2.1-3.el9.noarch.rp 639 kB/s | 11 kB 00:00 2024-07-26T13:25:48.071 INFO:teuthology.orchestra.run.smithi100.stdout:(52/160): bison-3.7.4-5.el9.x86_64.rpm 1.2 MB/s | 943 kB 00:00 2024-07-26T13:25:48.075 INFO:teuthology.orchestra.run.smithi138.stdout:(124/160): python3-jaraco-classes-3.2.1-5.el9.n 710 kB/s | 18 kB 00:00 2024-07-26T13:25:48.101 INFO:teuthology.orchestra.run.smithi138.stdout:(125/160): python3-jaraco-collections-3.0.0-8.e 928 kB/s | 23 kB 00:00 2024-07-26T13:25:48.126 INFO:teuthology.orchestra.run.smithi138.stdout:(126/160): python3-jaraco-functools-3.5.0-2.el9 778 kB/s | 19 kB 00:00 2024-07-26T13:25:48.151 INFO:teuthology.orchestra.run.smithi138.stdout:(127/160): python3-jaraco-text-3.2.0-6.el9.noar 783 kB/s | 20 kB 00:00 2024-07-26T13:25:48.155 INFO:teuthology.orchestra.run.smithi167.stdout:(9/160): ceph-common-19.0.0-5320.ge8f88bb1.el9. 13 MB/s | 20 MB 00:01 2024-07-26T13:25:48.162 INFO:teuthology.orchestra.run.smithi100.stdout:(53/160): flex-2.6.4-9.el9.x86_64.rpm 2.2 MB/s | 313 kB 00:00 2024-07-26T13:25:48.188 INFO:teuthology.orchestra.run.smithi100.stdout:(54/160): flexiblas-3.0.4-8.el9.x86_64.rpm 230 kB/s | 33 kB 00:00 2024-07-26T13:25:48.189 INFO:teuthology.orchestra.run.smithi167.stdout:(10/160): ceph-selinux-19.0.0-5320.ge8f88bb1.el 100 kB/s | 25 kB 00:00 2024-07-26T13:25:48.248 INFO:teuthology.orchestra.run.smithi167.stdout:(11/160): cephfs-mirror-19.0.0-5320.ge8f88bb1.e 3.8 MB/s | 225 kB 00:00 2024-07-26T13:25:48.268 INFO:teuthology.orchestra.run.smithi138.stdout:(128/160): protobuf-compiler-3.14.0-13.el9.x86_ 682 kB/s | 863 kB 00:01 2024-07-26T13:25:48.310 INFO:teuthology.orchestra.run.smithi138.stdout:(129/160): python3-grpcio-1.46.7-10.el9.x86_64. 6.1 MB/s | 2.0 MB 00:00 2024-07-26T13:25:48.315 INFO:teuthology.orchestra.run.smithi167.stdout:(12/160): cephfs-shell-19.0.0-5320.ge8f88bb1.el 407 kB/s | 27 kB 00:00 2024-07-26T13:25:48.369 INFO:teuthology.orchestra.run.smithi138.stdout:(130/160): python3-kubernetes-26.1.0-2.el9.noar 4.7 MB/s | 1.0 MB 00:00 2024-07-26T13:25:48.381 INFO:teuthology.orchestra.run.smithi167.stdout:(13/160): libcephfs-devel-19.0.0-5320.ge8f88bb1 477 kB/s | 32 kB 00:00 2024-07-26T13:25:48.394 INFO:teuthology.orchestra.run.smithi138.stdout:(131/160): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-07-26T13:25:48.396 INFO:teuthology.orchestra.run.smithi100.stdout:(55/160): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.2 MB/s | 3.0 MB 00:00 2024-07-26T13:25:48.419 INFO:teuthology.orchestra.run.smithi138.stdout:(132/160): python3-logutils-0.3.5-21.el9.noarch 306 kB/s | 46 kB 00:00 2024-07-26T13:25:48.421 INFO:teuthology.orchestra.run.smithi100.stdout:(56/160): flexiblas-openblas-openmp-3.0.4-8.el9 70 kB/s | 18 kB 00:00 2024-07-26T13:25:48.445 INFO:teuthology.orchestra.run.smithi138.stdout:(133/160): python3-natsort-7.1.1-5.el9.noarch.r 788 kB/s | 58 kB 00:00 2024-07-26T13:25:48.447 INFO:teuthology.orchestra.run.smithi100.stdout:(57/160): libaio-devel-0.3.111-13.el9.x86_64.rp 43 kB/s | 11 kB 00:00 2024-07-26T13:25:48.470 INFO:teuthology.orchestra.run.smithi138.stdout:(134/160): python3-portend-3.1.0-2.el9.noarch.r 326 kB/s | 16 kB 00:00 2024-07-26T13:25:48.482 INFO:teuthology.orchestra.run.smithi167.stdout:(14/160): libcephfs2-19.0.0-5320.ge8f88bb1.el9. 7.5 MB/s | 765 kB 00:00 2024-07-26T13:25:48.495 INFO:teuthology.orchestra.run.smithi138.stdout:(135/160): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-07-26T13:25:48.497 INFO:teuthology.orchestra.run.smithi100.stdout:(58/160): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-07-26T13:25:48.520 INFO:teuthology.orchestra.run.smithi138.stdout:(136/160): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-07-26T13:25:48.546 INFO:teuthology.orchestra.run.smithi138.stdout:(137/160): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-07-26T13:25:48.547 INFO:teuthology.orchestra.run.smithi100.stdout:(59/160): librabbitmq-0.11.0-7.el9.x86_64.rpm 452 kB/s | 45 kB 00:00 2024-07-26T13:25:48.571 INFO:teuthology.orchestra.run.smithi138.stdout:(138/160): python3-pecan-1.4.2-3.el9.noarch.rpm 1.5 MB/s | 272 kB 00:00 2024-07-26T13:25:48.581 INFO:teuthology.orchestra.run.smithi100.stdout:(60/160): librdkafka-1.6.1-102.el9.x86_64.rpm 4.8 MB/s | 662 kB 00:00 2024-07-26T13:25:48.596 INFO:teuthology.orchestra.run.smithi138.stdout:(139/160): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-07-26T13:25:48.615 INFO:teuthology.orchestra.run.smithi100.stdout:(61/160): libtool-2.4.6-46.el9.x86_64.rpm 4.8 MB/s | 581 kB 00:00 2024-07-26T13:25:48.622 INFO:teuthology.orchestra.run.smithi138.stdout:(140/160): python3-tempora-5.0.0-2.el9.noarch.r 708 kB/s | 36 kB 00:00 2024-07-26T13:25:48.640 INFO:teuthology.orchestra.run.smithi100.stdout:(62/160): libuuid-devel-2.37.4-18.el9.x86_64.rp 279 kB/s | 25 kB 00:00 2024-07-26T13:25:48.647 INFO:teuthology.orchestra.run.smithi138.stdout:(141/160): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-07-26T13:25:48.672 INFO:teuthology.orchestra.run.smithi138.stdout:(142/160): python3-typing-extensions-4.4.0-2.el 673 kB/s | 51 kB 00:00 2024-07-26T13:25:48.674 INFO:teuthology.orchestra.run.smithi100.stdout:(63/160): libzstd-devel-1.5.1-2.el9.x86_64.rpm 519 kB/s | 47 kB 00:00 2024-07-26T13:25:48.690 INFO:teuthology.orchestra.run.smithi167.stdout:(15/160): ceph-osd-19.0.0-5320.ge8f88bb1.el9.x8 13 MB/s | 18 MB 00:01 2024-07-26T13:25:48.698 INFO:teuthology.orchestra.run.smithi138.stdout:(143/160): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-07-26T13:25:48.716 INFO:teuthology.orchestra.run.smithi100.stdout:(64/160): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-07-26T13:25:48.716 INFO:teuthology.orchestra.run.smithi167.stdout:(16/160): libcephsqlite-19.0.0-5320.ge8f88bb1.e 677 kB/s | 158 kB 00:00 2024-07-26T13:25:48.723 INFO:teuthology.orchestra.run.smithi138.stdout:(144/160): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-07-26T13:25:48.741 INFO:teuthology.orchestra.run.smithi167.stdout:(17/160): librados-devel-19.0.0-5320.ge8f88bb1. 2.5 MB/s | 127 kB 00:00 2024-07-26T13:25:48.741 INFO:teuthology.orchestra.run.smithi100.stdout:(65/160): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-07-26T13:25:48.748 INFO:teuthology.orchestra.run.smithi138.stdout:(145/160): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-07-26T13:25:48.766 INFO:teuthology.orchestra.run.smithi100.stdout:(66/160): openblas-0.3.21-2.el9.x86_64.rpm 382 kB/s | 35 kB 00:00 2024-07-26T13:25:48.767 INFO:teuthology.orchestra.run.smithi167.stdout:(18/160): libradosstriper1-19.0.0-5320.ge8f88bb 9.8 MB/s | 503 kB 00:00 2024-07-26T13:25:48.773 INFO:teuthology.orchestra.run.smithi138.stdout:(146/160): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-07-26T13:25:48.799 INFO:teuthology.orchestra.run.smithi138.stdout:(147/160): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-07-26T13:25:48.816 INFO:teuthology.orchestra.run.smithi167.stdout:(19/160): python3-ceph-argparse-19.0.0-5320.ge8 902 kB/s | 45 kB 00:00 2024-07-26T13:25:48.875 INFO:teuthology.orchestra.run.smithi138.stdout:(148/160): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-07-26T13:25:48.909 INFO:teuthology.orchestra.run.smithi167.stdout:(20/160): python3-ceph-common-19.0.0-5320.ge8f8 1.5 MB/s | 138 kB 00:00 2024-07-26T13:25:48.958 INFO:teuthology.orchestra.run.smithi100.stdout:(67/160): openblas-openmp-0.3.21-2.el9.x86_64.r 20 MB/s | 4.7 MB 00:00 2024-07-26T13:25:49.000 INFO:teuthology.orchestra.run.smithi167.stdout:(21/160): librgw2-19.0.0-5320.ge8f88bb1.el9.x86 20 MB/s | 5.3 MB 00:00 2024-07-26T13:25:49.026 INFO:teuthology.orchestra.run.smithi167.stdout:(22/160): python3-cephfs-19.0.0-5320.ge8f88bb1. 1.6 MB/s | 188 kB 00:00 2024-07-26T13:25:49.060 INFO:teuthology.orchestra.run.smithi167.stdout:(23/160): python3-rados-19.0.0-5320.ge8f88bb1.e 6.2 MB/s | 372 kB 00:00 2024-07-26T13:25:49.083 INFO:teuthology.orchestra.run.smithi138.stdout:(149/160): NetworkManager-1.48.2-2.el9.x86_64.r 11 MB/s | 2.3 MB 00:00 2024-07-26T13:25:49.100 INFO:teuthology.orchestra.run.smithi100.stdout:(68/160): openssl-devel-3.2.2-2.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-07-26T13:25:49.101 INFO:teuthology.orchestra.run.smithi167.stdout:(24/160): python3-rbd-19.0.0-5320.ge8f88bb1.el9 4.6 MB/s | 355 kB 00:00 2024-07-26T13:25:49.118 INFO:teuthology.orchestra.run.smithi167.stdout:(25/160): python3-rgw-19.0.0-5320.ge8f88bb1.el9 1.9 MB/s | 116 kB 00:00 2024-07-26T13:25:49.142 INFO:teuthology.orchestra.run.smithi100.stdout:(69/160): postgresql-13.14-1.el9.x86_64.rpm 4.2 MB/s | 1.6 MB 00:00 2024-07-26T13:25:49.161 INFO:teuthology.orchestra.run.smithi167.stdout:(26/160): rbd-fuse-19.0.0-5320.ge8f88bb1.el9.x8 1.4 MB/s | 86 kB 00:00 2024-07-26T13:25:49.167 INFO:teuthology.orchestra.run.smithi100.stdout:(70/160): postgresql-contrib-13.14-1.el9.x86_64 4.3 MB/s | 911 kB 00:00 2024-07-26T13:25:49.183 INFO:teuthology.orchestra.run.smithi138.stdout:(150/160): librados2-19.0.0-5320.ge8f88bb1.el9. 8.2 MB/s | 3.4 MB 00:00 2024-07-26T13:25:49.209 INFO:teuthology.orchestra.run.smithi100.stdout:(71/160): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-07-26T13:25:49.233 INFO:teuthology.orchestra.run.smithi138.stdout:(151/160): NetworkManager-libnm-1.48.2-2.el9.x8 12 MB/s | 1.8 MB 00:00 2024-07-26T13:25:49.292 INFO:teuthology.orchestra.run.smithi138.stdout:(152/160): NetworkManager-team-1.48.2-2.el9.x86 381 kB/s | 41 kB 00:00 2024-07-26T13:25:49.342 INFO:teuthology.orchestra.run.smithi138.stdout:(153/160): librbd1-19.0.0-5320.ge8f88bb1.el9.x8 5.8 MB/s | 3.1 MB 00:00 2024-07-26T13:25:49.367 INFO:teuthology.orchestra.run.smithi138.stdout:(154/160): NetworkManager-tui-1.48.2-2.el9.x86_ 1.8 MB/s | 247 kB 00:00 2024-07-26T13:25:49.393 INFO:teuthology.orchestra.run.smithi138.stdout:(155/160): elfutils-debuginfod-client-0.191-4.e 365 kB/s | 37 kB 00:00 2024-07-26T13:25:49.402 INFO:teuthology.orchestra.run.smithi167.stdout:(27/160): rbd-mirror-19.0.0-5320.ge8f88bb1.el9. 11 MB/s | 3.1 MB 00:00 2024-07-26T13:25:49.426 INFO:teuthology.orchestra.run.smithi100.stdout:(72/160): postgresql-server-13.14-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-07-26T13:25:49.443 INFO:teuthology.orchestra.run.smithi138.stdout:(156/160): elfutils-libelf-0.191-4.el9.x86_64.r 2.0 MB/s | 207 kB 00:00 2024-07-26T13:25:49.444 INFO:teuthology.orchestra.run.smithi167.stdout:(28/160): ceph-grafana-dashboards-19.0.0-5320.g 824 kB/s | 34 kB 00:00 2024-07-26T13:25:49.460 INFO:teuthology.orchestra.run.smithi100.stdout:(73/160): protobuf-3.14.0-13.el9.x86_64.rpm 3.5 MB/s | 1.0 MB 00:00 2024-07-26T13:25:49.468 INFO:teuthology.orchestra.run.smithi138.stdout:(157/160): elfutils-libs-0.191-4.el9.x86_64.rpm 2.5 MB/s | 260 kB 00:00 2024-07-26T13:25:49.485 INFO:teuthology.orchestra.run.smithi100.stdout:(74/160): python3-attrs-20.3.0-7.el9.noarch.rpm 317 kB/s | 87 kB 00:00 2024-07-26T13:25:49.527 INFO:teuthology.orchestra.run.smithi138.stdout:(158/160): openssl-3.2.2-2.el9.x86_64.rpm 10 MB/s | 1.4 MB 00:00 2024-07-26T13:25:49.545 INFO:teuthology.orchestra.run.smithi167.stdout:(29/160): ceph-mgr-cephadm-19.0.0-5320.ge8f88bb 1.5 MB/s | 149 kB 00:00 2024-07-26T13:25:49.602 INFO:teuthology.orchestra.run.smithi138.stdout:(159/160): openssl-libs-3.2.2-2.el9.x86_64.rpm 15 MB/s | 2.4 MB 00:00 2024-07-26T13:25:49.627 INFO:teuthology.orchestra.run.smithi100.stdout:(75/160): python3-colorama-0.4.6-3.el9.noarch.r 432 kB/s | 61 kB 00:00 2024-07-26T13:25:49.635 INFO:teuthology.orchestra.run.smithi138.stdout:(160/160): xfsprogs-6.4.0-3.el9.x86_64.rpm 6.5 MB/s | 1.1 MB 00:00 2024-07-26T13:25:49.643 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-07-26T13:25:49.644 INFO:teuthology.orchestra.run.smithi138.stdout:Total 23 MB/s | 233 MB 00:10 2024-07-26T13:25:49.702 INFO:teuthology.orchestra.run.smithi100.stdout:(76/160): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 6.0 MB 00:00 2024-07-26T13:25:49.727 INFO:teuthology.orchestra.run.smithi100.stdout:(77/160): python3-devel-3.9.18-3.el9.x86_64.rpm 1.0 MB/s | 244 kB 00:00 2024-07-26T13:25:49.753 INFO:teuthology.orchestra.run.smithi100.stdout:(78/160): python3-jinja2-2.11.3-6.el9.noarch.rp 1.9 MB/s | 248 kB 00:00 2024-07-26T13:25:49.803 INFO:teuthology.orchestra.run.smithi100.stdout:(79/160): python3-jsonpatch-1.21-16.el9.noarch. 255 kB/s | 26 kB 00:00 2024-07-26T13:25:49.828 INFO:teuthology.orchestra.run.smithi100.stdout:(80/160): python3-jsonpointer-2.0-4.el9.noarch. 191 kB/s | 19 kB 00:00 2024-07-26T13:25:49.854 INFO:teuthology.orchestra.run.smithi100.stdout:(81/160): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-07-26T13:25:49.912 INFO:teuthology.orchestra.run.smithi100.stdout:(82/160): python3-markupsafe-1.1.1-12.el9.x86_6 320 kB/s | 35 kB 00:00 2024-07-26T13:25:49.970 INFO:teuthology.orchestra.run.smithi167.stdout:(30/160): ceph-test-19.0.0-5320.ge8f88bb1.el9.x 24 MB/s | 44 MB 00:01 2024-07-26T13:25:49.995 INFO:teuthology.orchestra.run.smithi167.stdout:(31/160): rbd-nbd-19.0.0-5320.ge8f88bb1.el9.x86 204 kB/s | 170 kB 00:00 2024-07-26T13:25:50.071 INFO:teuthology.orchestra.run.smithi167.stdout:(32/160): ceph-mgr-modules-core-19.0.0-5320.ge8 3.7 MB/s | 284 kB 00:00 2024-07-26T13:25:50.071 INFO:teuthology.orchestra.run.smithi100.stdout:(83/160): python3-numpy-1.20.1-5.el9.x86_64.rpm 21 MB/s | 5.1 MB 00:00 2024-07-26T13:25:50.096 INFO:teuthology.orchestra.run.smithi100.stdout:(84/160): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.0 MB/s | 250 kB 00:00 2024-07-26T13:25:50.121 INFO:teuthology.orchestra.run.smithi100.stdout:(85/160): python3-oauthlib-3.1.1-5.el9.noarch.r 1.0 MB/s | 222 kB 00:00 2024-07-26T13:25:50.162 INFO:teuthology.orchestra.run.smithi167.stdout:(33/160): ceph-mgr-rook-19.0.0-5320.ge8f88bb1.e 540 kB/s | 49 kB 00:00 2024-07-26T13:25:50.163 INFO:teuthology.orchestra.run.smithi100.stdout:(86/160): python3-packaging-20.9-5.el9.noarch.r 843 kB/s | 77 kB 00:00 2024-07-26T13:25:50.222 INFO:teuthology.orchestra.run.smithi100.stdout:(87/160): python3-prettytable-0.7.2-27.el9.noar 422 kB/s | 42 kB 00:00 2024-07-26T13:25:50.247 INFO:teuthology.orchestra.run.smithi100.stdout:(88/160): python3-protobuf-3.14.0-13.el9.noarch 2.1 MB/s | 269 kB 00:00 2024-07-26T13:25:50.263 INFO:teuthology.orchestra.run.smithi167.stdout:(34/160): ceph-mgr-diskprediction-local-19.0.0- 25 MB/s | 7.4 MB 00:00 2024-07-26T13:25:50.281 INFO:teuthology.orchestra.run.smithi100.stdout:(89/160): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-07-26T13:25:50.288 INFO:teuthology.orchestra.run.smithi167.stdout:(35/160): ceph-prometheus-alerts-19.0.0-5320.ge 131 kB/s | 16 kB 00:00 2024-07-26T13:25:50.322 INFO:teuthology.orchestra.run.smithi167.stdout:(36/160): ceph-volume-19.0.0-5320.ge8f88bb1.el9 4.6 MB/s | 274 kB 00:00 2024-07-26T13:25:50.323 INFO:teuthology.orchestra.run.smithi100.stdout:(90/160): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-07-26T13:25:50.348 INFO:teuthology.orchestra.run.smithi100.stdout:(91/160): python3-pytz-2021.1-5.el9.noarch.rpm 506 kB/s | 51 kB 00:00 2024-07-26T13:25:50.372 INFO:teuthology.orchestra.run.smithi167.stdout:(37/160): cephadm-19.0.0-5320.ge8f88bb1.el9.noa 9.0 MB/s | 771 kB 00:00 2024-07-26T13:25:50.382 INFO:teuthology.orchestra.run.smithi100.stdout:(92/160): python3-requests-oauthlib-1.3.0-12.el 531 kB/s | 54 kB 00:00 2024-07-26T13:25:50.397 INFO:teuthology.orchestra.run.smithi167.stdout:(38/160): cephfs-top-19.0.0-5320.ge8f88bb1.el9. 243 kB/s | 18 kB 00:00 2024-07-26T13:25:50.516 INFO:teuthology.orchestra.run.smithi100.stdout:(93/160): python3-toml-0.10.2-6.el9.noarch.rpm 249 kB/s | 42 kB 00:00 2024-07-26T13:25:50.539 INFO:teuthology.orchestra.run.smithi167.stdout:(39/160): ceph-mgr-dashboard-19.0.0-5320.ge8f88 14 MB/s | 14 MB 00:00 2024-07-26T13:25:50.540 INFO:teuthology.orchestra.run.smithi100.stdout:(94/160): python3-wcwidth-0.2.5-8.el9.noarch.rp 278 kB/s | 44 kB 00:00 2024-07-26T13:25:50.700 INFO:teuthology.orchestra.run.smithi100.stdout:(95/160): qatlib-service-24.02.0-1.el9.x86_64.r 229 kB/s | 36 kB 00:00 2024-07-26T13:25:50.825 INFO:teuthology.orchestra.run.smithi100.stdout:(96/160): qatlib-24.02.0-1.el9.x86_64.rpm 715 kB/s | 221 kB 00:00 2024-07-26T13:25:50.843 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-07-26T13:25:50.850 INFO:teuthology.orchestra.run.smithi100.stdout:(97/160): qatzip-libs-1.2.0-1.el9.x86_64.rpm 320 kB/s | 47 kB 00:00 2024-07-26T13:25:50.890 INFO:teuthology.orchestra.run.smithi167.stdout:(40/160): fuse-2.9.9-16.el9.x86_64.rpm 154 kB/s | 80 kB 00:00 2024-07-26T13:25:51.000 INFO:teuthology.orchestra.run.smithi100.stdout:(98/160): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-07-26T13:25:51.025 INFO:teuthology.orchestra.run.smithi100.stdout:(99/160): socat-1.7.4.1-6.el9.x86_64.rpm 1.5 MB/s | 304 kB 00:00 2024-07-26T13:25:51.040 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-07-26T13:25:51.040 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-07-26T13:25:51.048 INFO:teuthology.orchestra.run.smithi167.stdout:(41/160): libquadmath-11.4.1-3.el9.x86_64.rpm 384 kB/s | 195 kB 00:00 2024-07-26T13:25:51.051 INFO:teuthology.orchestra.run.smithi100.stdout:(100/160): uuid-1.6.2-55.el9.x86_64.rpm 289 kB/s | 58 kB 00:00 2024-07-26T13:25:51.101 INFO:teuthology.orchestra.run.smithi100.stdout:(101/160): xfsprogs-devel-6.4.0-3.el9.x86_64.rp 959 kB/s | 96 kB 00:00 2024-07-26T13:25:51.151 INFO:teuthology.orchestra.run.smithi100.stdout:(102/160): xmlstarlet-1.6.1-20.el9.x86_64.rpm 636 kB/s | 64 kB 00:00 2024-07-26T13:25:51.176 INFO:teuthology.orchestra.run.smithi100.stdout:(103/160): zlib-devel-1.2.11-41.el9.x86_64.rpm 357 kB/s | 45 kB 00:00 2024-07-26T13:25:51.194 INFO:teuthology.orchestra.run.smithi100.stdout:(104/160): lua-devel-5.4.4-4.el9.x86_64.rpm 242 kB/s | 22 kB 00:00 2024-07-26T13:25:51.240 INFO:teuthology.orchestra.run.smithi167.stdout:(42/160): python3-cffi-1.14.5-5.el9.x86_64.rpm 723 kB/s | 253 kB 00:00 2024-07-26T13:25:51.294 INFO:teuthology.orchestra.run.smithi100.stdout:(105/160): abseil-cpp-20211102.0-3.el9.x86_64.r 4.6 MB/s | 548 kB 00:00 2024-07-26T13:25:51.324 INFO:teuthology.orchestra.run.smithi167.stdout:(43/160): libgfortran-11.4.1-3.el9.x86_64.rpm 867 kB/s | 803 kB 00:00 2024-07-26T13:25:51.327 INFO:teuthology.orchestra.run.smithi100.stdout:(106/160): grpc-data-1.46.7-10.el9.noarch.rpm 591 kB/s | 19 kB 00:00 2024-07-26T13:25:51.349 INFO:teuthology.orchestra.run.smithi167.stdout:(44/160): python3-ply-3.11-14.el9.noarch.rpm 981 kB/s | 106 kB 00:00 2024-07-26T13:25:51.353 INFO:teuthology.orchestra.run.smithi100.stdout:(107/160): gperftools-libs-2.9.1-2.el9.x86_64.r 1.9 MB/s | 309 kB 00:00 2024-07-26T13:25:51.369 INFO:teuthology.orchestra.run.smithi100.stdout:(108/160): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.5 MB/s | 26 kB 00:00 2024-07-26T13:25:51.395 INFO:teuthology.orchestra.run.smithi100.stdout:(109/160): protobuf-compiler-3.14.0-13.el9.x86_ 3.5 MB/s | 863 kB 00:00 2024-07-26T13:25:51.420 INFO:teuthology.orchestra.run.smithi100.stdout:(110/160): liboath-2.6.7-2.el9.x86_64.rpm 963 kB/s | 49 kB 00:00 2024-07-26T13:25:51.433 INFO:teuthology.orchestra.run.smithi167.stdout:(45/160): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-07-26T13:25:51.458 INFO:teuthology.orchestra.run.smithi167.stdout:(46/160): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-07-26T13:25:51.487 INFO:teuthology.orchestra.run.smithi100.stdout:(111/160): luarocks-3.9.2-5.el9.noarch.rpm 3.6 MB/s | 151 kB 00:00 2024-07-26T13:25:51.512 INFO:teuthology.orchestra.run.smithi100.stdout:(112/160): libunwind-1.6.2-1.el9.x86_64.rpm 575 kB/s | 67 kB 00:00 2024-07-26T13:25:51.542 INFO:teuthology.orchestra.run.smithi167.stdout:(47/160): python3-urllib3-1.26.5-6.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-07-26T13:25:51.571 INFO:teuthology.orchestra.run.smithi100.stdout:(113/160): python3-asyncssh-2.13.2-5.el9.noarch 9.2 MB/s | 548 kB 00:00 2024-07-26T13:25:51.600 INFO:teuthology.orchestra.run.smithi167.stdout:(48/160): python3-cryptography-36.0.1-4.el9.x86 2.2 MB/s | 1.2 MB 00:00 2024-07-26T13:25:51.604 INFO:teuthology.orchestra.run.smithi100.stdout:(114/160): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-07-26T13:25:51.630 INFO:teuthology.orchestra.run.smithi100.stdout:(115/160): python3-bcrypt-3.2.2-1.el9.x86_64.rp 740 kB/s | 43 kB 00:00 2024-07-26T13:25:51.655 INFO:teuthology.orchestra.run.smithi100.stdout:(116/160): python3-cachetools-4.2.4-1.el9.noarc 637 kB/s | 32 kB 00:00 2024-07-26T13:25:51.681 INFO:teuthology.orchestra.run.smithi100.stdout:(117/160): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-07-26T13:25:51.706 INFO:teuthology.orchestra.run.smithi100.stdout:(118/160): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-07-26T13:25:51.790 INFO:teuthology.orchestra.run.smithi100.stdout:(119/160): libarrow-9.0.0-11.el9.x86_64.rpm 9.5 MB/s | 4.4 MB 00:00 2024-07-26T13:25:51.815 INFO:teuthology.orchestra.run.smithi100.stdout:(120/160): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-07-26T13:25:51.841 INFO:teuthology.orchestra.run.smithi100.stdout:(121/160): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.2 MB/s | 303 kB 00:00 2024-07-26T13:25:51.866 INFO:teuthology.orchestra.run.smithi100.stdout:(122/160): python3-google-auth-2.32.0-1.el9.noa 2.8 MB/s | 216 kB 00:00 2024-07-26T13:25:51.891 INFO:teuthology.orchestra.run.smithi100.stdout:(123/160): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-07-26T13:25:51.916 INFO:teuthology.orchestra.run.smithi100.stdout:(124/160): python3-jaraco-8.2.1-3.el9.noarch.rp 212 kB/s | 11 kB 00:00 2024-07-26T13:25:51.942 INFO:teuthology.orchestra.run.smithi100.stdout:(125/160): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-07-26T13:25:51.967 INFO:teuthology.orchestra.run.smithi100.stdout:(126/160): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-07-26T13:25:51.992 INFO:teuthology.orchestra.run.smithi100.stdout:(127/160): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-07-26T13:25:52.018 INFO:teuthology.orchestra.run.smithi100.stdout:(128/160): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-07-26T13:25:52.043 INFO:teuthology.orchestra.run.smithi100.stdout:(129/160): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-07-26T13:25:52.076 INFO:teuthology.orchestra.run.smithi100.stdout:(130/160): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-07-26T13:25:52.125 INFO:teuthology.orchestra.run.smithi167.stdout:(49/160): boost-program-options-1.75.0-8.el9.x8 204 kB/s | 107 kB 00:00 2024-07-26T13:25:52.127 INFO:teuthology.orchestra.run.smithi100.stdout:(131/160): python3-grpcio-1.46.7-10.el9.x86_64. 6.5 MB/s | 2.0 MB 00:00 2024-07-26T13:25:52.152 INFO:teuthology.orchestra.run.smithi100.stdout:(132/160): python3-natsort-7.1.1-5.el9.noarch.r 766 kB/s | 58 kB 00:00 2024-07-26T13:25:52.186 INFO:teuthology.orchestra.run.smithi100.stdout:(133/160): python3-kubernetes-26.1.0-2.el9.noar 5.3 MB/s | 1.0 MB 00:00 2024-07-26T13:25:52.211 INFO:teuthology.orchestra.run.smithi100.stdout:(134/160): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-07-26T13:25:52.217 INFO:teuthology.orchestra.run.smithi167.stdout:(50/160): automake-1.16.2-8.el9.noarch.rpm 908 kB/s | 689 kB 00:00 2024-07-26T13:25:52.234 INFO:teuthology.orchestra.run.smithi167.stdout:(51/160): elfutils-libelf-devel-0.191-4.el9.x86 200 kB/s | 22 kB 00:00 2024-07-26T13:25:52.236 INFO:teuthology.orchestra.run.smithi100.stdout:(135/160): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-07-26T13:25:52.261 INFO:teuthology.orchestra.run.smithi100.stdout:(136/160): python3-pyperclip-1.8.2-2.el9.noarch 564 kB/s | 28 kB 00:00 2024-07-26T13:25:52.287 INFO:teuthology.orchestra.run.smithi100.stdout:(137/160): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-07-26T13:25:52.312 INFO:teuthology.orchestra.run.smithi100.stdout:(138/160): python3-pyOpenSSL-21.0.0-1.el9.noarc 892 kB/s | 90 kB 00:00 2024-07-26T13:25:52.346 INFO:teuthology.orchestra.run.smithi100.stdout:(139/160): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-07-26T13:25:52.351 INFO:teuthology.orchestra.run.smithi167.stdout:(52/160): bison-3.7.4-5.el9.x86_64.rpm 1.2 MB/s | 943 kB 00:00 2024-07-26T13:25:52.371 INFO:teuthology.orchestra.run.smithi100.stdout:(140/160): python3-tempora-5.0.0-2.el9.noarch.r 611 kB/s | 36 kB 00:00 2024-07-26T13:25:52.376 INFO:teuthology.orchestra.run.smithi167.stdout:(53/160): flex-2.6.4-9.el9.x86_64.rpm 1.9 MB/s | 313 kB 00:00 2024-07-26T13:25:52.393 INFO:teuthology.orchestra.run.smithi167.stdout:(54/160): flexiblas-3.0.4-8.el9.x86_64.rpm 206 kB/s | 33 kB 00:00 2024-07-26T13:25:52.396 INFO:teuthology.orchestra.run.smithi100.stdout:(141/160): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-07-26T13:25:52.422 INFO:teuthology.orchestra.run.smithi100.stdout:(142/160): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-07-26T13:25:52.447 INFO:teuthology.orchestra.run.smithi100.stdout:(143/160): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-07-26T13:25:52.480 INFO:teuthology.orchestra.run.smithi100.stdout:(144/160): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-07-26T13:25:52.522 INFO:teuthology.orchestra.run.smithi100.stdout:(145/160): python3-zc-lockfile-2.0-10.el9.noarc 266 kB/s | 20 kB 00:00 2024-07-26T13:25:52.548 INFO:teuthology.orchestra.run.smithi100.stdout:(146/160): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-07-26T13:25:52.573 INFO:teuthology.orchestra.run.smithi100.stdout:(147/160): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-07-26T13:25:52.660 INFO:teuthology.orchestra.run.smithi167.stdout:(55/160): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.7 MB/s | 3.0 MB 00:00 2024-07-26T13:25:52.666 INFO:teuthology.orchestra.run.smithi100.stdout:(148/160): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-07-26T13:25:52.677 INFO:teuthology.orchestra.run.smithi167.stdout:(56/160): flexiblas-openblas-openmp-3.0.4-8.el9 64 kB/s | 18 kB 00:00 2024-07-26T13:25:52.694 INFO:teuthology.orchestra.run.smithi167.stdout:(57/160): libaio-devel-0.3.111-13.el9.x86_64.rp 37 kB/s | 11 kB 00:00 2024-07-26T13:25:52.778 INFO:teuthology.orchestra.run.smithi167.stdout:(58/160): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-07-26T13:25:52.794 INFO:teuthology.orchestra.run.smithi167.stdout:(59/160): librabbitmq-0.11.0-7.el9.x86_64.rpm 387 kB/s | 45 kB 00:00 2024-07-26T13:25:52.820 INFO:teuthology.orchestra.run.smithi167.stdout:(60/160): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 662 kB 00:00 2024-07-26T13:25:52.832 INFO:teuthology.orchestra.run.smithi100.stdout:(149/160): librados2-19.0.0-5320.ge8f88bb1.el9. 12 MB/s | 3.4 MB 00:00 2024-07-26T13:25:52.887 INFO:teuthology.orchestra.run.smithi167.stdout:(61/160): libtool-2.4.6-46.el9.x86_64.rpm 5.3 MB/s | 581 kB 00:00 2024-07-26T13:25:52.898 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-07-26T13:25:52.899 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-07-26T13:25:52.912 INFO:teuthology.orchestra.run.smithi167.stdout:(62/160): libuuid-devel-2.37.4-18.el9.x86_64.rp 276 kB/s | 25 kB 00:00 2024-07-26T13:25:52.929 INFO:teuthology.orchestra.run.smithi167.stdout:(63/160): libzstd-devel-1.5.1-2.el9.x86_64.rpm 436 kB/s | 47 kB 00:00 2024-07-26T13:25:52.974 INFO:teuthology.orchestra.run.smithi100.stdout:(150/160): librbd1-19.0.0-5320.ge8f88bb1.el9.x8 7.8 MB/s | 3.1 MB 00:00 2024-07-26T13:25:52.988 INFO:teuthology.orchestra.run.smithi167.stdout:(64/160): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-07-26T13:25:53.013 INFO:teuthology.orchestra.run.smithi167.stdout:(65/160): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-07-26T13:25:53.038 INFO:teuthology.orchestra.run.smithi167.stdout:(66/160): openblas-0.3.21-2.el9.x86_64.rpm 323 kB/s | 35 kB 00:00 2024-07-26T13:25:53.083 INFO:teuthology.orchestra.run.smithi100.stdout:(151/160): NetworkManager-team-1.48.2-2.el9.x86 376 kB/s | 41 kB 00:00 2024-07-26T13:25:53.200 INFO:teuthology.orchestra.run.smithi100.stdout:(152/160): NetworkManager-libnm-1.48.2-2.el9.x8 4.9 MB/s | 1.8 MB 00:00 2024-07-26T13:25:53.225 INFO:teuthology.orchestra.run.smithi100.stdout:(153/160): NetworkManager-tui-1.48.2-2.el9.x86_ 1.7 MB/s | 247 kB 00:00 2024-07-26T13:25:53.250 INFO:teuthology.orchestra.run.smithi100.stdout:(154/160): elfutils-debuginfod-client-0.191-4.e 730 kB/s | 37 kB 00:00 2024-07-26T13:25:53.255 INFO:teuthology.orchestra.run.smithi167.stdout:(67/160): openblas-openmp-0.3.21-2.el9.x86_64.r 18 MB/s | 4.7 MB 00:00 2024-07-26T13:25:53.309 INFO:teuthology.orchestra.run.smithi100.stdout:(155/160): elfutils-libelf-0.191-4.el9.x86_64.r 2.4 MB/s | 207 kB 00:00 2024-07-26T13:25:53.406 INFO:teuthology.orchestra.run.smithi167.stdout:(68/160): openssl-devel-3.2.2-2.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-07-26T13:25:53.426 INFO:teuthology.orchestra.run.smithi100.stdout:(156/160): elfutils-libs-0.191-4.el9.x86_64.rpm 1.4 MB/s | 260 kB 00:00 2024-07-26T13:25:53.447 INFO:teuthology.orchestra.run.smithi167.stdout:(69/160): postgresql-13.14-1.el9.x86_64.rpm 3.9 MB/s | 1.6 MB 00:00 2024-07-26T13:25:53.468 INFO:teuthology.orchestra.run.smithi100.stdout:(157/160): NetworkManager-1.48.2-2.el9.x86_64.r 2.9 MB/s | 2.3 MB 00:00 2024-07-26T13:25:53.481 INFO:teuthology.orchestra.run.smithi167.stdout:(70/160): postgresql-contrib-13.14-1.el9.x86_64 4.0 MB/s | 911 kB 00:00 2024-07-26T13:25:53.501 INFO:teuthology.orchestra.run.smithi100.stdout:(158/160): openssl-3.2.2-2.el9.x86_64.rpm 7.1 MB/s | 1.4 MB 00:00 2024-07-26T13:25:53.506 INFO:teuthology.orchestra.run.smithi167.stdout:(71/160): postgresql-private-libs-13.14-1.el9.x 1.3 MB/s | 133 kB 00:00 2024-07-26T13:25:53.740 INFO:teuthology.orchestra.run.smithi167.stdout:(72/160): postgresql-server-13.14-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-07-26T13:25:53.773 INFO:teuthology.orchestra.run.smithi167.stdout:(73/160): protobuf-3.14.0-13.el9.x86_64.rpm 3.5 MB/s | 1.0 MB 00:00 2024-07-26T13:25:53.799 INFO:teuthology.orchestra.run.smithi167.stdout:(74/160): python3-attrs-20.3.0-7.el9.noarch.rpm 299 kB/s | 87 kB 00:00 2024-07-26T13:25:53.932 INFO:teuthology.orchestra.run.smithi167.stdout:(75/160): python3-colorama-0.4.6-3.el9.noarch.r 460 kB/s | 61 kB 00:00 2024-07-26T13:25:54.016 INFO:teuthology.orchestra.run.smithi167.stdout:(76/160): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 6.0 MB 00:00 2024-07-26T13:25:54.026 INFO:teuthology.orchestra.run.smithi100.stdout:(159/160): xfsprogs-6.4.0-3.el9.x86_64.rpm 1.9 MB/s | 1.1 MB 00:00 2024-07-26T13:25:54.041 INFO:teuthology.orchestra.run.smithi167.stdout:(77/160): python3-devel-3.9.18-3.el9.x86_64.rpm 1.0 MB/s | 244 kB 00:00 2024-07-26T13:25:54.074 INFO:teuthology.orchestra.run.smithi167.stdout:(78/160): python3-jinja2-2.11.3-6.el9.noarch.rp 1.7 MB/s | 248 kB 00:00 2024-07-26T13:25:54.141 INFO:teuthology.orchestra.run.smithi167.stdout:(79/160): python3-jsonpatch-1.21-16.el9.noarch. 258 kB/s | 26 kB 00:00 2024-07-26T13:25:54.167 INFO:teuthology.orchestra.run.smithi167.stdout:(80/160): python3-jsonpointer-2.0-4.el9.noarch. 153 kB/s | 19 kB 00:00 2024-07-26T13:25:54.192 INFO:teuthology.orchestra.run.smithi167.stdout:(81/160): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-07-26T13:25:54.276 INFO:teuthology.orchestra.run.smithi167.stdout:(82/160): python3-markupsafe-1.1.1-12.el9.x86_6 259 kB/s | 35 kB 00:00 2024-07-26T13:25:54.426 INFO:teuthology.orchestra.run.smithi167.stdout:(83/160): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 5.1 MB 00:00 2024-07-26T13:25:54.451 INFO:teuthology.orchestra.run.smithi167.stdout:(84/160): python3-numpy-f2py-1.20.1-5.el9.x86_6 967 kB/s | 250 kB 00:00 2024-07-26T13:25:54.476 INFO:teuthology.orchestra.run.smithi167.stdout:(85/160): python3-oauthlib-3.1.1-5.el9.noarch.r 1.1 MB/s | 222 kB 00:00 2024-07-26T13:25:54.527 INFO:teuthology.orchestra.run.smithi167.stdout:(86/160): python3-packaging-20.9-5.el9.noarch.r 775 kB/s | 77 kB 00:00 2024-07-26T13:25:54.577 INFO:teuthology.orchestra.run.smithi167.stdout:(87/160): python3-prettytable-0.7.2-27.el9.noar 421 kB/s | 42 kB 00:00 2024-07-26T13:25:54.602 INFO:teuthology.orchestra.run.smithi167.stdout:(88/160): python3-protobuf-3.14.0-13.el9.noarch 2.1 MB/s | 269 kB 00:00 2024-07-26T13:25:54.628 INFO:teuthology.orchestra.run.smithi167.stdout:(89/160): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 159 kB 00:00 2024-07-26T13:25:54.643 INFO:teuthology.orchestra.run.smithi100.stdout:(160/160): openssl-libs-3.2.2-2.el9.x86_64.rpm 2.0 MB/s | 2.4 MB 00:01 2024-07-26T13:25:54.651 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-07-26T13:25:54.651 INFO:teuthology.orchestra.run.smithi100.stdout:Total 18 MB/s | 233 MB 00:13 2024-07-26T13:25:54.678 INFO:teuthology.orchestra.run.smithi167.stdout:(90/160): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-07-26T13:25:54.704 INFO:teuthology.orchestra.run.smithi167.stdout:(91/160): python3-pytz-2021.1-5.el9.noarch.rpm 507 kB/s | 51 kB 00:00 2024-07-26T13:25:54.729 INFO:teuthology.orchestra.run.smithi167.stdout:(92/160): python3-requests-oauthlib-1.3.0-12.el 532 kB/s | 54 kB 00:00 2024-07-26T13:25:54.863 INFO:teuthology.orchestra.run.smithi167.stdout:(93/160): python3-toml-0.10.2-6.el9.noarch.rpm 263 kB/s | 42 kB 00:00 2024-07-26T13:25:54.888 INFO:teuthology.orchestra.run.smithi167.stdout:(94/160): python3-wcwidth-0.2.5-8.el9.noarch.rp 278 kB/s | 44 kB 00:00 2024-07-26T13:25:55.063 INFO:teuthology.orchestra.run.smithi167.stdout:(95/160): qatlib-service-24.02.0-1.el9.x86_64.r 207 kB/s | 36 kB 00:00 2024-07-26T13:25:55.338 INFO:teuthology.orchestra.run.smithi167.stdout:(96/160): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-07-26T13:25:55.363 INFO:teuthology.orchestra.run.smithi167.stdout:(97/160): qatlib-24.02.0-1.el9.x86_64.rpm 441 kB/s | 221 kB 00:00 2024-07-26T13:25:55.389 INFO:teuthology.orchestra.run.smithi167.stdout:(98/160): qatzip-libs-1.2.0-1.el9.x86_64.rpm 147 kB/s | 47 kB 00:00 2024-07-26T13:25:55.439 INFO:teuthology.orchestra.run.smithi167.stdout:(99/160): socat-1.7.4.1-6.el9.x86_64.rpm 3.0 MB/s | 304 kB 00:00 2024-07-26T13:25:55.489 INFO:teuthology.orchestra.run.smithi167.stdout:(100/160): uuid-1.6.2-55.el9.x86_64.rpm 576 kB/s | 58 kB 00:00 2024-07-26T13:25:55.514 INFO:teuthology.orchestra.run.smithi167.stdout:(101/160): xfsprogs-devel-6.4.0-3.el9.x86_64.rp 762 kB/s | 96 kB 00:00 2024-07-26T13:25:55.540 INFO:teuthology.orchestra.run.smithi167.stdout:(102/160): xmlstarlet-1.6.1-20.el9.x86_64.rpm 633 kB/s | 64 kB 00:00 2024-07-26T13:25:55.590 INFO:teuthology.orchestra.run.smithi167.stdout:(103/160): zlib-devel-1.2.11-41.el9.x86_64.rpm 446 kB/s | 45 kB 00:00 2024-07-26T13:25:55.640 INFO:teuthology.orchestra.run.smithi167.stdout:(104/160): lua-devel-5.4.4-4.el9.x86_64.rpm 178 kB/s | 22 kB 00:00 2024-07-26T13:25:55.740 INFO:teuthology.orchestra.run.smithi167.stdout:(105/160): gperftools-libs-2.9.1-2.el9.x86_64.r 3.1 MB/s | 309 kB 00:00 2024-07-26T13:25:55.766 INFO:teuthology.orchestra.run.smithi167.stdout:(106/160): abseil-cpp-20211102.0-3.el9.x86_64.r 3.1 MB/s | 548 kB 00:00 2024-07-26T13:25:55.791 INFO:teuthology.orchestra.run.smithi167.stdout:(107/160): grpc-data-1.46.7-10.el9.noarch.rpm 388 kB/s | 19 kB 00:00 2024-07-26T13:25:55.825 INFO:teuthology.orchestra.run.smithi167.stdout:(108/160): protobuf-compiler-3.14.0-13.el9.x86_ 3.0 MB/s | 863 kB 00:00 2024-07-26T13:25:55.850 INFO:teuthology.orchestra.run.smithi167.stdout:(109/160): libarrow-doc-9.0.0-11.el9.noarch.rpm 443 kB/s | 26 kB 00:00 2024-07-26T13:25:55.899 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-07-26T13:25:55.900 INFO:teuthology.orchestra.run.smithi167.stdout:(110/160): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-07-26T13:25:55.934 INFO:teuthology.orchestra.run.smithi167.stdout:(111/160): liboath-2.6.7-2.el9.x86_64.rpm 446 kB/s | 49 kB 00:00 2024-07-26T13:25:55.959 INFO:teuthology.orchestra.run.smithi167.stdout:(112/160): luarocks-3.9.2-5.el9.noarch.rpm 2.6 MB/s | 151 kB 00:00 2024-07-26T13:25:56.043 INFO:teuthology.orchestra.run.smithi167.stdout:(113/160): python3-asyncssh-2.13.2-5.el9.noarch 6.4 MB/s | 548 kB 00:00 2024-07-26T13:25:56.068 INFO:teuthology.orchestra.run.smithi167.stdout:(114/160): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-07-26T13:25:56.102 INFO:teuthology.orchestra.run.smithi167.stdout:(115/160): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.9 MB/s | 839 kB 00:00 2024-07-26T13:25:56.103 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-07-26T13:25:56.103 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-07-26T13:25:56.127 INFO:teuthology.orchestra.run.smithi167.stdout:(116/160): python3-cachetools-4.2.4-1.el9.noarc 547 kB/s | 32 kB 00:00 2024-07-26T13:25:56.152 INFO:teuthology.orchestra.run.smithi167.stdout:(117/160): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-07-26T13:25:56.186 INFO:teuthology.orchestra.run.smithi167.stdout:(118/160): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-07-26T13:25:56.236 INFO:teuthology.orchestra.run.smithi167.stdout:(119/160): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-07-26T13:25:56.262 INFO:teuthology.orchestra.run.smithi167.stdout:(120/160): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-07-26T13:25:56.329 INFO:teuthology.orchestra.run.smithi167.stdout:(121/160): libarrow-9.0.0-11.el9.x86_64.rpm 7.8 MB/s | 4.4 MB 00:00 2024-07-26T13:25:56.354 INFO:teuthology.orchestra.run.smithi167.stdout:(122/160): python3-google-auth-2.32.0-1.el9.noa 1.8 MB/s | 216 kB 00:00 2024-07-26T13:25:56.379 INFO:teuthology.orchestra.run.smithi167.stdout:(123/160): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-07-26T13:25:56.404 INFO:teuthology.orchestra.run.smithi167.stdout:(124/160): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-07-26T13:25:56.422 INFO:teuthology.orchestra.run.smithi167.stdout:(125/160): python3-jaraco-classes-3.2.1-5.el9.n 421 kB/s | 18 kB 00:00 2024-07-26T13:25:56.438 INFO:teuthology.orchestra.run.smithi167.stdout:(126/160): python3-jaraco-collections-3.0.0-8.e 687 kB/s | 23 kB 00:00 2024-07-26T13:25:56.455 INFO:teuthology.orchestra.run.smithi167.stdout:(127/160): python3-jaraco-functools-3.5.0-2.el9 579 kB/s | 19 kB 00:00 2024-07-26T13:25:56.473 INFO:teuthology.orchestra.run.smithi167.stdout:(128/160): python3-jaraco-text-3.2.0-6.el9.noar 584 kB/s | 20 kB 00:00 2024-07-26T13:25:56.489 INFO:teuthology.orchestra.run.smithi167.stdout:(129/160): python3-logutils-0.3.5-21.el9.noarch 2.7 MB/s | 46 kB 00:00 2024-07-26T13:25:56.523 INFO:teuthology.orchestra.run.smithi167.stdout:(130/160): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-07-26T13:25:56.548 INFO:teuthology.orchestra.run.smithi167.stdout:(131/160): python3-natsort-7.1.1-5.el9.noarch.r 2.2 MB/s | 58 kB 00:00 2024-07-26T13:25:56.590 INFO:teuthology.orchestra.run.smithi167.stdout:(132/160): python3-grpcio-1.46.7-10.el9.x86_64. 6.2 MB/s | 2.0 MB 00:00 2024-07-26T13:25:56.624 INFO:teuthology.orchestra.run.smithi167.stdout:(133/160): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-07-26T13:25:56.649 INFO:teuthology.orchestra.run.smithi167.stdout:(134/160): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-07-26T13:25:56.674 INFO:teuthology.orchestra.run.smithi167.stdout:(135/160): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-07-26T13:25:56.699 INFO:teuthology.orchestra.run.smithi167.stdout:(136/160): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-07-26T13:25:56.725 INFO:teuthology.orchestra.run.smithi167.stdout:(137/160): python3-pyperclip-1.8.2-2.el9.noarch 373 kB/s | 28 kB 00:00 2024-07-26T13:25:56.750 INFO:teuthology.orchestra.run.smithi167.stdout:(138/160): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-07-26T13:25:56.776 INFO:teuthology.orchestra.run.smithi167.stdout:(139/160): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-07-26T13:25:56.801 INFO:teuthology.orchestra.run.smithi167.stdout:(140/160): python3-tempora-5.0.0-2.el9.noarch.r 709 kB/s | 36 kB 00:00 2024-07-26T13:25:56.826 INFO:teuthology.orchestra.run.smithi167.stdout:(141/160): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-07-26T13:25:56.851 INFO:teuthology.orchestra.run.smithi167.stdout:(142/160): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-07-26T13:25:56.868 INFO:teuthology.orchestra.run.smithi167.stdout:(143/160): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-07-26T13:25:56.894 INFO:teuthology.orchestra.run.smithi167.stdout:(144/160): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-07-26T13:25:56.919 INFO:teuthology.orchestra.run.smithi167.stdout:(145/160): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-07-26T13:25:56.944 INFO:teuthology.orchestra.run.smithi167.stdout:(146/160): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-07-26T13:25:56.970 INFO:teuthology.orchestra.run.smithi167.stdout:(147/160): python3-werkzeug-2.0.3-3.el9.1.noarc 3.5 MB/s | 427 kB 00:00 2024-07-26T13:25:57.045 INFO:teuthology.orchestra.run.smithi167.stdout:(148/160): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-07-26T13:25:57.212 INFO:teuthology.orchestra.run.smithi167.stdout:(149/160): librados2-19.0.0-5320.ge8f88bb1.el9. 13 MB/s | 3.4 MB 00:00 2024-07-26T13:25:57.729 INFO:teuthology.orchestra.run.smithi167.stdout:(150/160): librbd1-19.0.0-5320.ge8f88bb1.el9.x8 4.1 MB/s | 3.1 MB 00:00 2024-07-26T13:25:58.029 INFO:teuthology.orchestra.run.smithi167.stdout:(151/160): NetworkManager-1.48.2-2.el9.x86_64.r 2.3 MB/s | 2.3 MB 00:00 2024-07-26T13:25:58.054 INFO:teuthology.orchestra.run.smithi167.stdout:(152/160): NetworkManager-team-1.48.2-2.el9.x86 125 kB/s | 41 kB 00:00 2024-07-26T13:25:58.138 INFO:teuthology.orchestra.run.smithi167.stdout:(153/160): NetworkManager-tui-1.48.2-2.el9.x86_ 2.2 MB/s | 247 kB 00:00 2024-07-26T13:25:58.238 INFO:teuthology.orchestra.run.smithi167.stdout:(154/160): elfutils-debuginfod-client-0.191-4.e 200 kB/s | 37 kB 00:00 2024-07-26T13:25:58.263 INFO:teuthology.orchestra.run.smithi167.stdout:(155/160): elfutils-libelf-0.191-4.el9.x86_64.r 1.6 MB/s | 207 kB 00:00 2024-07-26T13:25:58.389 INFO:teuthology.orchestra.run.smithi167.stdout:(156/160): openssl-3.2.2-2.el9.x86_64.rpm 11 MB/s | 1.4 MB 00:00 2024-07-26T13:25:58.390 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-07-26T13:25:58.390 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-07-26T13:25:58.606 INFO:teuthology.orchestra.run.smithi167.stdout:(157/160): elfutils-libs-0.191-4.el9.x86_64.rpm 707 kB/s | 260 kB 00:00 2024-07-26T13:25:58.648 INFO:teuthology.orchestra.run.smithi167.stdout:(158/160): NetworkManager-libnm-1.48.2-2.el9.x8 1.3 MB/s | 1.8 MB 00:01 2024-07-26T13:25:58.981 INFO:teuthology.orchestra.run.smithi167.stdout:(159/160): xfsprogs-6.4.0-3.el9.x86_64.rpm 2.9 MB/s | 1.1 MB 00:00 2024-07-26T13:25:59.304 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-07-26T13:25:59.315 INFO:teuthology.orchestra.run.smithi167.stdout:(160/160): openssl-libs-3.2.2-2.el9.x86_64.rpm 2.6 MB/s | 2.4 MB 00:00 2024-07-26T13:25:59.321 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2024-07-26T13:25:59.321 INFO:teuthology.orchestra.run.smithi167.stdout:Total 17 MB/s | 233 MB 00:13 2024-07-26T13:25:59.381 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : openssl-libs-1:3.2.2-2.el9.x86_64 1/172 2024-07-26T13:25:59.428 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/172 2024-07-26T13:25:59.481 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/172 2024-07-26T13:25:59.607 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/172 2024-07-26T13:25:59.691 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/172 2024-07-26T13:25:59.735 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/172 2024-07-26T13:26:00.082 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/172 2024-07-26T13:26:00.208 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librados2-2:19.0.0-5320.ge8f88bb1.el9.x86_64 8/172 2024-07-26T13:26:00.333 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:19.0.0-5320.ge8f88bb1.el9.x86_64 8/172 2024-07-26T13:26:00.360 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs2-2:19.0.0-5320.ge8f88bb1.el9.x86_64 9/172 2024-07-26T13:26:00.454 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephfs2-2:19.0.0-5320.ge8f88bb1.el9.x86_64 9/172 2024-07-26T13:26:00.480 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rados-2:19.0.0-5320.ge8f88bb1.el9.x86_64 10/172 2024-07-26T13:26:00.513 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/172 2024-07-26T13:26:00.555 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2024-07-26T13:26:00.715 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/172 2024-07-26T13:26:00.749 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2024-07-26T13:26:00.749 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2024-07-26T13:26:00.779 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/172 2024-07-26T13:26:00.807 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephsqlite-2:19.0.0-5320.ge8f88bb1.el9.x86_64 14/172 2024-07-26T13:26:00.912 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephsqlite-2:19.0.0-5320.ge8f88bb1.el9.x86_64 14/172 2024-07-26T13:26:00.941 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libradosstriper1-2:19.0.0-5320.ge8f88bb1.el9.x86 15/172 2024-07-26T13:26:00.987 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libradosstriper1-2:19.0.0-5320.ge8f88bb1.el9.x86 15/172 2024-07-26T13:26:01.009 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-26T13:26:01.034 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-26T13:26:01.226 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-26T13:26:01.319 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-26T13:26:01.516 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : xfsprogs-6.4.0-3.el9.x86_64 18/172 2024-07-26T13:26:01.670 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-26T13:26:01.727 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-26T13:26:01.832 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-26T13:26:01.912 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-26T13:26:02.135 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-26T13:26:02.271 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : re2-1:20211101-3.el9.x86_64 21/172 2024-07-26T13:26:02.299 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 22/172 2024-07-26T13:26:02.334 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/172 2024-07-26T13:26:02.408 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/172 2024-07-26T13:26:02.452 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/172 2024-07-26T13:26:02.492 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-packaging-20.9-5.el9.noarch 26/172 2024-07-26T13:26:02.536 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/172 2024-07-26T13:26:02.570 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 28/172 2024-07-26T13:26:02.673 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2024-07-26T13:26:02.674 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2024-07-26T13:26:02.691 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lua-5.4.4-4.el9.x86_64 29/172 2024-07-26T13:26:02.874 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 30/172 2024-07-26T13:26:02.985 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 31/172 2024-07-26T13:26:03.016 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/172 2024-07-26T13:26:03.060 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 33/172 2024-07-26T13:26:03.136 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 34/172 2024-07-26T13:26:03.170 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-common-2:19.0.0-5320.ge8f88bb1.el9. 35/172 2024-07-26T13:26:03.216 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-argparse-2:19.0.0-5320.ge8f88bb1.el 36/172 2024-07-26T13:26:03.242 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cephfs-2:19.0.0-5320.ge8f88bb1.el9.x86_6 37/172 2024-07-26T13:26:03.313 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 38/172 2024-07-26T13:26:03.374 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : luarocks-3.9.2-5.el9.noarch 39/172 2024-07-26T13:26:03.448 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 40/172 2024-07-26T13:26:03.590 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 41/172 2024-07-26T13:26:03.643 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 42/172 2024-07-26T13:26:04.166 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rsa-4.9-2.el9.noarch 43/172 2024-07-26T13:26:04.247 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 44/172 2024-07-26T13:26:04.276 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 45/172 2024-07-26T13:26:04.312 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 46/172 2024-07-26T13:26:04.353 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 47/172 2024-07-26T13:26:04.391 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librados-devel-2:19.0.0-5320.ge8f88bb1.el9.x86_6 48/172 2024-07-26T13:26:04.433 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 49/172 2024-07-26T13:26:04.586 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 50/172 2024-07-26T13:26:04.643 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : openssl-1:3.2.2-2.el9.x86_64 51/172 2024-07-26T13:26:04.735 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 52/172 2024-07-26T13:26:04.798 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 53/172 2024-07-26T13:26:04.873 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/172 2024-07-26T13:26:04.904 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/172 2024-07-26T13:26:04.940 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/172 2024-07-26T13:26:04.987 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/172 2024-07-26T13:26:05.023 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/172 2024-07-26T13:26:05.066 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/172 2024-07-26T13:26:05.071 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-07-26T13:26:05.107 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/172 2024-07-26T13:26:05.185 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : openssl-libs-1:3.2.2-2.el9.x86_64 1/172 2024-07-26T13:26:05.228 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/172 2024-07-26T13:26:05.243 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/172 2024-07-26T13:26:05.283 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/172 2024-07-26T13:26:05.324 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/172 2024-07-26T13:26:05.349 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/172 2024-07-26T13:26:05.400 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/172 2024-07-26T13:26:05.410 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/172 2024-07-26T13:26:05.451 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 63/172 2024-07-26T13:26:05.550 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 64/172 2024-07-26T13:26:05.584 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-google-auth-1:2.32.0-1.el9.noarch 65/172 2024-07-26T13:26:05.624 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 66/172 2024-07-26T13:26:05.650 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 67/172 2024-07-26T13:26:05.675 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 68/172 2024-07-26T13:26:05.750 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 69/172 2024-07-26T13:26:05.780 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/172 2024-07-26T13:26:05.850 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : librados2-2:19.0.0-5320.ge8f88bb1.el9.x86_64 8/172 2024-07-26T13:26:05.876 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 70/172 2024-07-26T13:26:05.915 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 71/172 2024-07-26T13:26:05.943 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 72/172 2024-07-26T13:26:05.976 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librados2-2:19.0.0-5320.ge8f88bb1.el9.x86_64 8/172 2024-07-26T13:26:05.977 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 73/172 2024-07-26T13:26:06.001 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephfs2-2:19.0.0-5320.ge8f88bb1.el9.x86_64 9/172 2024-07-26T13:26:06.017 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : uuid-1.6.2-55.el9.x86_64 74/172 2024-07-26T13:26:06.048 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 75/172 2024-07-26T13:26:06.078 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 76/172 2024-07-26T13:26:06.097 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libcephfs2-2:19.0.0-5320.ge8f88bb1.el9.x86_64 9/172 2024-07-26T13:26:06.112 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 77/172 2024-07-26T13:26:06.123 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rados-2:19.0.0-5320.ge8f88bb1.el9.x86_64 10/172 2024-07-26T13:26:06.146 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 78/172 2024-07-26T13:26:06.167 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/172 2024-07-26T13:26:06.187 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 79/172 2024-07-26T13:26:06.222 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 80/172 2024-07-26T13:26:06.308 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 81/172 2024-07-26T13:26:06.352 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 82/172 2024-07-26T13:26:06.392 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 83/172 2024-07-26T13:26:06.440 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/172 2024-07-26T13:26:06.495 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 84/172 2024-07-26T13:26:06.504 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/172 2024-07-26T13:26:06.532 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephsqlite-2:19.0.0-5320.ge8f88bb1.el9.x86_64 14/172 2024-07-26T13:26:06.542 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-26T13:26:06.637 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libcephsqlite-2:19.0.0-5320.ge8f88bb1.el9.x86_64 14/172 2024-07-26T13:26:06.666 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libradosstriper1-2:19.0.0-5320.ge8f88bb1.el9.x86 15/172 2024-07-26T13:26:06.729 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libradosstriper1-2:19.0.0-5320.ge8f88bb1.el9.x86 15/172 2024-07-26T13:26:06.750 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-26T13:26:06.772 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-26T13:26:06.961 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-26T13:26:07.064 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-26T13:26:07.267 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/172 2024-07-26T13:26:07.315 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/172 2024-07-26T13:26:07.331 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : xfsprogs-6.4.0-3.el9.x86_64 18/172 2024-07-26T13:26:07.348 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/172 2024-07-26T13:26:07.448 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/172 2024-07-26T13:26:07.469 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-26T13:26:07.495 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 90/172 2024-07-26T13:26:07.529 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-26T13:26:07.539 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 91/172 2024-07-26T13:26:07.628 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 92/172 2024-07-26T13:26:07.634 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-26T13:26:07.668 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 93/172 2024-07-26T13:26:07.697 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-26T13:26:07.844 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-0.3.21-2.el9.x86_64 94/172 2024-07-26T13:26:07.874 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 95/172 2024-07-26T13:26:07.898 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-26T13:26:07.953 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 96/172 2024-07-26T13:26:08.002 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : re2-1:20211101-3.el9.x86_64 21/172 2024-07-26T13:26:08.043 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 22/172 2024-07-26T13:26:08.077 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/172 2024-07-26T13:26:08.142 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/172 2024-07-26T13:26:08.196 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/172 2024-07-26T13:26:08.229 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-packaging-20.9-5.el9.noarch 26/172 2024-07-26T13:26:08.289 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/172 2024-07-26T13:26:08.330 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 28/172 2024-07-26T13:26:08.357 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lua-5.4.4-4.el9.x86_64 29/172 2024-07-26T13:26:08.427 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 30/172 2024-07-26T13:26:08.482 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 31/172 2024-07-26T13:26:08.521 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/172 2024-07-26T13:26:08.565 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 33/172 2024-07-26T13:26:08.610 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 97/172 2024-07-26T13:26:08.642 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 34/172 2024-07-26T13:26:08.679 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ceph-common-2:19.0.0-5320.ge8f88bb1.el9. 35/172 2024-07-26T13:26:08.722 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 98/172 2024-07-26T13:26:08.731 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ceph-argparse-2:19.0.0-5320.ge8f88bb1.el 36/172 2024-07-26T13:26:08.756 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cephfs-2:19.0.0-5320.ge8f88bb1.el9.x86_6 37/172 2024-07-26T13:26:08.817 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 38/172 2024-07-26T13:26:08.863 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : luarocks-3.9.2-5.el9.noarch 39/172 2024-07-26T13:26:08.936 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 40/172 2024-07-26T13:26:09.071 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 41/172 2024-07-26T13:26:09.099 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2024-07-26T13:26:09.133 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 42/172 2024-07-26T13:26:09.194 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : openssl-libs-1:3.2.2-2.el9.x86_64 1/172 2024-07-26T13:26:09.232 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/172 2024-07-26T13:26:09.294 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/172 2024-07-26T13:26:09.328 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/172 2024-07-26T13:26:09.354 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/172 2024-07-26T13:26:09.396 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/172 2024-07-26T13:26:09.662 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rsa-4.9-2.el9.noarch 43/172 2024-07-26T13:26:09.735 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/172 2024-07-26T13:26:09.745 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 44/172 2024-07-26T13:26:09.775 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 45/172 2024-07-26T13:26:09.812 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 46/172 2024-07-26T13:26:09.851 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 47/172 2024-07-26T13:26:09.889 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librados-devel-2:19.0.0-5320.ge8f88bb1.el9.x86_6 48/172 2024-07-26T13:26:09.933 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 49/172 2024-07-26T13:26:09.949 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librados2-2:19.0.0-5320.ge8f88bb1.el9.x86_64 8/172 2024-07-26T13:26:09.953 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 99/172 2024-07-26T13:26:10.044 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 100/172 2024-07-26T13:26:10.072 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 101/172 2024-07-26T13:26:10.080 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-2:19.0.0-5320.ge8f88bb1.el9.x86_64 8/172 2024-07-26T13:26:10.087 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 50/172 2024-07-26T13:26:10.105 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 102/172 2024-07-26T13:26:10.106 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs2-2:19.0.0-5320.ge8f88bb1.el9.x86_64 9/172 2024-07-26T13:26:10.168 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : openssl-1:3.2.2-2.el9.x86_64 51/172 2024-07-26T13:26:10.202 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephfs2-2:19.0.0-5320.ge8f88bb1.el9.x86_64 9/172 2024-07-26T13:26:10.227 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rados-2:19.0.0-5320.ge8f88bb1.el9.x86_64 10/172 2024-07-26T13:26:10.259 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 52/172 2024-07-26T13:26:10.260 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/172 2024-07-26T13:26:10.323 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 53/172 2024-07-26T13:26:10.397 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/172 2024-07-26T13:26:10.427 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/172 2024-07-26T13:26:10.433 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 103/172 2024-07-26T13:26:10.443 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/172 2024-07-26T13:26:10.464 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/172 2024-07-26T13:26:10.476 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librbd1-2:19.0.0-5320.ge8f88bb1.el9.x86_64 104/172 2024-07-26T13:26:10.492 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/172 2024-07-26T13:26:10.511 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/172 2024-07-26T13:26:10.520 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephsqlite-2:19.0.0-5320.ge8f88bb1.el9.x86_64 14/172 2024-07-26T13:26:10.546 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/172 2024-07-26T13:26:10.571 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:19.0.0-5320.ge8f88bb1.el9.x86_64 104/172 2024-07-26T13:26:10.589 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/172 2024-07-26T13:26:10.598 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rbd-2:19.0.0-5320.ge8f88bb1.el9.x86_64 105/172 2024-07-26T13:26:10.617 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephsqlite-2:19.0.0-5320.ge8f88bb1.el9.x86_64 14/172 2024-07-26T13:26:10.631 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/172 2024-07-26T13:26:10.634 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 106/172 2024-07-26T13:26:10.646 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libradosstriper1-2:19.0.0-5320.ge8f88bb1.el9.x86 15/172 2024-07-26T13:26:10.708 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libradosstriper1-2:19.0.0-5320.ge8f88bb1.el9.x86 15/172 2024-07-26T13:26:10.730 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-26T13:26:10.749 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-26T13:26:10.766 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/172 2024-07-26T13:26:10.808 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/172 2024-07-26T13:26:10.849 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 63/172 2024-07-26T13:26:10.938 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-26T13:26:10.974 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 64/172 2024-07-26T13:26:11.009 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-google-auth-1:2.32.0-1.el9.noarch 65/172 2024-07-26T13:26:11.029 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-26T13:26:11.050 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 66/172 2024-07-26T13:26:11.082 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 67/172 2024-07-26T13:26:11.106 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 68/172 2024-07-26T13:26:11.133 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 107/172 2024-07-26T13:26:11.163 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librgw2-2:19.0.0-5320.ge8f88bb1.el9.x86_64 108/172 2024-07-26T13:26:11.227 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 69/172 2024-07-26T13:26:11.228 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : xfsprogs-6.4.0-3.el9.x86_64 18/172 2024-07-26T13:26:11.235 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librgw2-2:19.0.0-5320.ge8f88bb1.el9.x86_64 108/172 2024-07-26T13:26:11.264 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rgw-2:19.0.0-5320.ge8f88bb1.el9.x86_64 109/172 2024-07-26T13:26:11.389 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-26T13:26:11.466 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-26T13:26:11.499 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 70/172 2024-07-26T13:26:11.539 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 71/172 2024-07-26T13:26:11.563 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-26T13:26:11.567 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 72/172 2024-07-26T13:26:11.601 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 73/172 2024-07-26T13:26:11.640 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : uuid-1.6.2-55.el9.x86_64 74/172 2024-07-26T13:26:11.658 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-26T13:26:11.688 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 75/172 2024-07-26T13:26:11.719 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 76/172 2024-07-26T13:26:11.753 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 77/172 2024-07-26T13:26:11.786 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 78/172 2024-07-26T13:26:11.828 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 79/172 2024-07-26T13:26:11.867 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-26T13:26:11.870 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 80/172 2024-07-26T13:26:11.956 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 81/172 2024-07-26T13:26:11.975 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : re2-1:20211101-3.el9.x86_64 21/172 2024-07-26T13:26:12.004 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 22/172 2024-07-26T13:26:12.006 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 82/172 2024-07-26T13:26:12.039 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/172 2024-07-26T13:26:12.041 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 83/172 2024-07-26T13:26:12.122 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/172 2024-07-26T13:26:12.142 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 84/172 2024-07-26T13:26:12.183 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/172 2024-07-26T13:26:12.199 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-26T13:26:12.223 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-packaging-20.9-5.el9.noarch 26/172 2024-07-26T13:26:12.266 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/172 2024-07-26T13:26:12.299 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 28/172 2024-07-26T13:26:12.327 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lua-5.4.4-4.el9.x86_64 29/172 2024-07-26T13:26:12.396 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 30/172 2024-07-26T13:26:12.449 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 31/172 2024-07-26T13:26:12.480 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/172 2024-07-26T13:26:12.515 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 33/172 2024-07-26T13:26:12.591 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 34/172 2024-07-26T13:26:12.626 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-common-2:19.0.0-5320.ge8f88bb1.el9. 35/172 2024-07-26T13:26:12.671 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-argparse-2:19.0.0-5320.ge8f88bb1.el 36/172 2024-07-26T13:26:12.698 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cephfs-2:19.0.0-5320.ge8f88bb1.el9.x86_6 37/172 2024-07-26T13:26:12.770 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 38/172 2024-07-26T13:26:12.813 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : luarocks-3.9.2-5.el9.noarch 39/172 2024-07-26T13:26:12.889 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 40/172 2024-07-26T13:26:12.906 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/172 2024-07-26T13:26:12.955 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/172 2024-07-26T13:26:12.989 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/172 2024-07-26T13:26:13.022 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 41/172 2024-07-26T13:26:13.082 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 42/172 2024-07-26T13:26:13.103 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/172 2024-07-26T13:26:13.161 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 90/172 2024-07-26T13:26:13.206 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 91/172 2024-07-26T13:26:13.292 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 92/172 2024-07-26T13:26:13.334 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 93/172 2024-07-26T13:26:13.509 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openblas-0.3.21-2.el9.x86_64 94/172 2024-07-26T13:26:13.539 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 95/172 2024-07-26T13:26:13.542 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:19.0.0-5320.ge8f88bb1.el9.x86_64 110/172 2024-07-26T13:26:13.577 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-common-2:19.0.0-5320.ge8f88bb1.el9.x86_64 110/172 2024-07-26T13:26:13.598 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rsa-4.9-2.el9.noarch 43/172 2024-07-26T13:26:13.618 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 96/172 2024-07-26T13:26:13.677 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 44/172 2024-07-26T13:26:13.707 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 45/172 2024-07-26T13:26:13.743 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 46/172 2024-07-26T13:26:13.792 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 47/172 2024-07-26T13:26:13.830 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librados-devel-2:19.0.0-5320.ge8f88bb1.el9.x86_6 48/172 2024-07-26T13:26:13.872 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 49/172 2024-07-26T13:26:14.024 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 50/172 2024-07-26T13:26:14.083 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : openssl-1:3.2.2-2.el9.x86_64 51/172 2024-07-26T13:26:14.143 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:19.0.0-5320.ge8f88bb1.el9.x86_64 110/172 2024-07-26T13:26:14.175 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 52/172 2024-07-26T13:26:14.180 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-base-2:19.0.0-5320.ge8f88bb1.el9.x86_64 111/172 2024-07-26T13:26:14.240 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 53/172 2024-07-26T13:26:14.260 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 97/172 2024-07-26T13:26:14.272 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-base-2:19.0.0-5320.ge8f88bb1.el9.x86_64 111/172 2024-07-26T13:26:14.272 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-07-26T13:26:14.273 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-07-26T13:26:14.273 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-07-26T13:26:14.298 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-selinux-2:19.0.0-5320.ge8f88bb1.el9.x86_64 112/172 2024-07-26T13:26:14.312 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/172 2024-07-26T13:26:14.343 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/172 2024-07-26T13:26:14.369 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 98/172 2024-07-26T13:26:14.380 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/172 2024-07-26T13:26:14.427 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/172 2024-07-26T13:26:14.463 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/172 2024-07-26T13:26:14.505 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/172 2024-07-26T13:26:14.548 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/172 2024-07-26T13:26:14.686 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/172 2024-07-26T13:26:14.725 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/172 2024-07-26T13:26:14.766 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 63/172 2024-07-26T13:26:14.865 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 64/172 2024-07-26T13:26:14.899 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-google-auth-1:2.32.0-1.el9.noarch 65/172 2024-07-26T13:26:14.939 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 66/172 2024-07-26T13:26:14.973 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 67/172 2024-07-26T13:26:14.999 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 68/172 2024-07-26T13:26:15.090 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 69/172 2024-07-26T13:26:15.216 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 70/172 2024-07-26T13:26:15.431 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 71/172 2024-07-26T13:26:15.467 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 72/172 2024-07-26T13:26:15.502 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 73/172 2024-07-26T13:26:15.542 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : uuid-1.6.2-55.el9.x86_64 74/172 2024-07-26T13:26:15.581 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 75/172 2024-07-26T13:26:15.582 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 99/172 2024-07-26T13:26:15.611 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 76/172 2024-07-26T13:26:15.645 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 77/172 2024-07-26T13:26:15.679 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 78/172 2024-07-26T13:26:15.685 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 100/172 2024-07-26T13:26:15.712 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 101/172 2024-07-26T13:26:15.721 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 79/172 2024-07-26T13:26:15.746 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 102/172 2024-07-26T13:26:15.755 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 80/172 2024-07-26T13:26:15.845 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 81/172 2024-07-26T13:26:15.893 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 82/172 2024-07-26T13:26:15.932 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 83/172 2024-07-26T13:26:16.038 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 84/172 2024-07-26T13:26:16.077 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 103/172 2024-07-26T13:26:16.100 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-26T13:26:16.134 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : librbd1-2:19.0.0-5320.ge8f88bb1.el9.x86_64 104/172 2024-07-26T13:26:16.229 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librbd1-2:19.0.0-5320.ge8f88bb1.el9.x86_64 104/172 2024-07-26T13:26:16.255 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rbd-2:19.0.0-5320.ge8f88bb1.el9.x86_64 105/172 2024-07-26T13:26:16.292 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 106/172 2024-07-26T13:26:16.790 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 107/172 2024-07-26T13:26:16.812 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librgw2-2:19.0.0-5320.ge8f88bb1.el9.x86_64 108/172 2024-07-26T13:26:16.822 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/172 2024-07-26T13:26:16.872 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/172 2024-07-26T13:26:16.884 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librgw2-2:19.0.0-5320.ge8f88bb1.el9.x86_64 108/172 2024-07-26T13:26:16.906 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/172 2024-07-26T13:26:16.913 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rgw-2:19.0.0-5320.ge8f88bb1.el9.x86_64 109/172 2024-07-26T13:26:17.006 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/172 2024-07-26T13:26:17.061 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 90/172 2024-07-26T13:26:17.106 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 91/172 2024-07-26T13:26:17.198 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 92/172 2024-07-26T13:26:17.252 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 93/172 2024-07-26T13:26:17.429 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-0.3.21-2.el9.x86_64 94/172 2024-07-26T13:26:17.456 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 95/172 2024-07-26T13:26:17.534 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 96/172 2024-07-26T13:26:18.180 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 97/172 2024-07-26T13:26:18.305 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 98/172 2024-07-26T13:26:19.172 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-common-2:19.0.0-5320.ge8f88bb1.el9.x86_64 110/172 2024-07-26T13:26:19.210 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-common-2:19.0.0-5320.ge8f88bb1.el9.x86_64 110/172 2024-07-26T13:26:19.540 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 99/172 2024-07-26T13:26:19.627 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 100/172 2024-07-26T13:26:19.655 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 101/172 2024-07-26T13:26:19.688 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 102/172 2024-07-26T13:26:19.780 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-common-2:19.0.0-5320.ge8f88bb1.el9.x86_64 110/172 2024-07-26T13:26:19.822 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-base-2:19.0.0-5320.ge8f88bb1.el9.x86_64 111/172 2024-07-26T13:26:19.894 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-base-2:19.0.0-5320.ge8f88bb1.el9.x86_64 111/172 2024-07-26T13:26:19.895 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-07-26T13:26:19.895 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-07-26T13:26:19.895 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:26:19.931 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-selinux-2:19.0.0-5320.ge8f88bb1.el9.x86_64 112/172 2024-07-26T13:26:20.014 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 103/172 2024-07-26T13:26:20.059 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librbd1-2:19.0.0-5320.ge8f88bb1.el9.x86_64 104/172 2024-07-26T13:26:20.154 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-2:19.0.0-5320.ge8f88bb1.el9.x86_64 104/172 2024-07-26T13:26:20.181 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rbd-2:19.0.0-5320.ge8f88bb1.el9.x86_64 105/172 2024-07-26T13:26:20.217 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 106/172 2024-07-26T13:26:20.716 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 107/172 2024-07-26T13:26:20.738 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librgw2-2:19.0.0-5320.ge8f88bb1.el9.x86_64 108/172 2024-07-26T13:26:20.810 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librgw2-2:19.0.0-5320.ge8f88bb1.el9.x86_64 108/172 2024-07-26T13:26:20.839 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rgw-2:19.0.0-5320.ge8f88bb1.el9.x86_64 109/172 2024-07-26T13:26:23.123 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:19.0.0-5320.ge8f88bb1.el9.x86_64 110/172 2024-07-26T13:26:23.163 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-common-2:19.0.0-5320.ge8f88bb1.el9.x86_64 110/172 2024-07-26T13:26:23.727 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:19.0.0-5320.ge8f88bb1.el9.x86_64 110/172 2024-07-26T13:26:23.770 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-base-2:19.0.0-5320.ge8f88bb1.el9.x86_64 111/172 2024-07-26T13:26:23.862 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-base-2:19.0.0-5320.ge8f88bb1.el9.x86_64 111/172 2024-07-26T13:26:23.862 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-07-26T13:26:23.862 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-07-26T13:26:23.863 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-07-26T13:26:23.897 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-selinux-2:19.0.0-5320.ge8f88bb1.el9.x86_64 112/172 2024-07-26T13:26:26.314 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-selinux-2:19.0.0-5320.ge8f88bb1.el9.x86_64 112/172 2024-07-26T13:26:26.314 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /sys 2024-07-26T13:26:26.314 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /proc 2024-07-26T13:26:26.314 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /mnt 2024-07-26T13:26:26.314 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /var/tmp 2024-07-26T13:26:26.314 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /home 2024-07-26T13:26:26.314 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /root 2024-07-26T13:26:26.315 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /tmp 2024-07-26T13:26:26.315 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-07-26T13:26:26.494 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-volume-2:19.0.0-5320.ge8f88bb1.el9.noarch 113/172 2024-07-26T13:26:26.510 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-volume-2:19.0.0-5320.ge8f88bb1.el9.noarch 113/172 2024-07-26T13:26:26.510 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-26T13:26:26.510 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-07-26T13:26:26.510 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-07-26T13:26:28.106 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-osd-2:19.0.0-5320.ge8f88bb1.el9.x86_64 114/172 2024-07-26T13:26:28.133 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-osd-2:19.0.0-5320.ge8f88bb1.el9.x86_64 114/172 2024-07-26T13:26:28.133 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-26T13:26:28.133 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-07-26T13:26:28.133 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-26T13:26:28.133 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-26T13:26:28.133 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-07-26T13:26:28.357 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mds-2:19.0.0-5320.ge8f88bb1.el9.x86_64 115/172 2024-07-26T13:26:28.380 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mds-2:19.0.0-5320.ge8f88bb1.el9.x86_64 115/172 2024-07-26T13:26:28.380 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-26T13:26:28.380 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-07-26T13:26:28.380 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-26T13:26:28.380 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-26T13:26:28.380 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-07-26T13:26:28.843 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mon-2:19.0.0-5320.ge8f88bb1.el9.x86_64 116/172 2024-07-26T13:26:28.864 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mon-2:19.0.0-5320.ge8f88bb1.el9.x86_64 116/172 2024-07-26T13:26:28.864 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-26T13:26:28.864 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-07-26T13:26:28.865 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-26T13:26:28.865 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-26T13:26:28.865 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-07-26T13:26:28.963 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : automake-1.16.2-8.el9.noarch 117/172 2024-07-26T13:26:29.021 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/172 2024-07-26T13:26:29.076 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/172 2024-07-26T13:26:29.262 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/172 2024-07-26T13:26:29.310 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 121/172 2024-07-26T13:26:29.388 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/172 2024-07-26T13:26:29.476 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 123/172 2024-07-26T13:26:29.616 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/172 2024-07-26T13:26:29.661 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/172 2024-07-26T13:26:29.693 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/172 2024-07-26T13:26:29.721 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : fuse-2.9.9-16.el9.x86_64 127/172 2024-07-26T13:26:30.150 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephadm-2:19.0.0-5320.ge8f88bb1.el9.noarch 128/172 2024-07-26T13:26:30.179 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephadm-2:19.0.0-5320.ge8f88bb1.el9.noarch 128/172 2024-07-26T13:26:30.211 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-5320.ge8f88bb1.e 129/172 2024-07-26T13:26:30.301 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-5320.ge8f88bb1. 130/172 2024-07-26T13:26:30.335 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-5320.ge8f88bb1.el9.noa 131/172 2024-07-26T13:26:31.794 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-selinux-2:19.0.0-5320.ge8f88bb1.el9.x86_64 112/172 2024-07-26T13:26:31.795 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /sys 2024-07-26T13:26:31.795 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /proc 2024-07-26T13:26:31.795 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /mnt 2024-07-26T13:26:31.795 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /var/tmp 2024-07-26T13:26:31.795 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /home 2024-07-26T13:26:31.795 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /root 2024-07-26T13:26:31.796 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /tmp 2024-07-26T13:26:31.796 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:26:31.971 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-volume-2:19.0.0-5320.ge8f88bb1.el9.noarch 113/172 2024-07-26T13:26:31.987 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-volume-2:19.0.0-5320.ge8f88bb1.el9.noarch 113/172 2024-07-26T13:26:31.987 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-26T13:26:31.987 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-07-26T13:26:31.987 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:26:33.508 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-5320.ge8f88bb1.el9.noa 131/172 2024-07-26T13:26:33.569 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-5320.ge8f88bb1.el9.n 132/172 2024-07-26T13:26:33.574 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-osd-2:19.0.0-5320.ge8f88bb1.el9.x86_64 114/172 2024-07-26T13:26:33.600 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-osd-2:19.0.0-5320.ge8f88bb1.el9.x86_64 114/172 2024-07-26T13:26:33.601 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-26T13:26:33.601 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-07-26T13:26:33.601 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-26T13:26:33.601 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-26T13:26:33.601 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:26:33.833 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mds-2:19.0.0-5320.ge8f88bb1.el9.x86_64 115/172 2024-07-26T13:26:33.856 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mds-2:19.0.0-5320.ge8f88bb1.el9.x86_64 115/172 2024-07-26T13:26:33.856 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-26T13:26:33.856 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-07-26T13:26:33.856 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-26T13:26:33.856 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-26T13:26:33.856 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:26:34.317 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mon-2:19.0.0-5320.ge8f88bb1.el9.x86_64 116/172 2024-07-26T13:26:34.340 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mon-2:19.0.0-5320.ge8f88bb1.el9.x86_64 116/172 2024-07-26T13:26:34.340 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-26T13:26:34.340 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-07-26T13:26:34.340 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-26T13:26:34.341 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-26T13:26:34.341 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:26:34.438 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : automake-1.16.2-8.el9.noarch 117/172 2024-07-26T13:26:34.487 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/172 2024-07-26T13:26:34.532 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-5320.ge8f88bb1.el9.n 132/172 2024-07-26T13:26:34.543 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/172 2024-07-26T13:26:34.569 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-5320.ge8f 133/172 2024-07-26T13:26:34.682 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-5320.ge8f 133/172 2024-07-26T13:26:34.744 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/172 2024-07-26T13:26:34.792 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 121/172 2024-07-26T13:26:34.845 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-5320.ge8f88bb1.el 134/172 2024-07-26T13:26:34.862 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/172 2024-07-26T13:26:34.872 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-2:19.0.0-5320.ge8f88bb1.el9.x86_64 135/172 2024-07-26T13:26:34.894 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-2:19.0.0-5320.ge8f88bb1.el9.x86_64 135/172 2024-07-26T13:26:34.895 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-26T13:26:34.895 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-07-26T13:26:34.895 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-26T13:26:34.895 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-26T13:26:34.895 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-07-26T13:26:34.939 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-rook-2:19.0.0-5320.ge8f88bb1.el9.noarch 136/172 2024-07-26T13:26:34.950 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-5320.ge8f88bb1.el9.noarch 136/172 2024-07-26T13:26:34.958 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 123/172 2024-07-26T13:26:35.050 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-2:19.0.0-5320.ge8f88bb1.el9.x86_64 137/172 2024-07-26T13:26:35.097 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/172 2024-07-26T13:26:35.105 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-fuse-2:19.0.0-5320.ge8f88bb1.el9.x86_64 138/172 2024-07-26T13:26:35.134 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/172 2024-07-26T13:26:35.166 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/172 2024-07-26T13:26:35.202 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : fuse-2.9.9-16.el9.x86_64 127/172 2024-07-26T13:26:35.399 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-selinux-2:19.0.0-5320.ge8f88bb1.el9.x86_64 112/172 2024-07-26T13:26:35.399 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /sys 2024-07-26T13:26:35.399 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /proc 2024-07-26T13:26:35.399 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /mnt 2024-07-26T13:26:35.399 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /var/tmp 2024-07-26T13:26:35.400 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /home 2024-07-26T13:26:35.400 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /root 2024-07-26T13:26:35.400 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /tmp 2024-07-26T13:26:35.400 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-07-26T13:26:35.582 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-volume-2:19.0.0-5320.ge8f88bb1.el9.noarch 113/172 2024-07-26T13:26:35.597 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-volume-2:19.0.0-5320.ge8f88bb1.el9.noarch 113/172 2024-07-26T13:26:35.597 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-26T13:26:35.597 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-07-26T13:26:35.597 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-07-26T13:26:35.665 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: cephadm-2:19.0.0-5320.ge8f88bb1.el9.noarch 128/172 2024-07-26T13:26:35.693 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephadm-2:19.0.0-5320.ge8f88bb1.el9.noarch 128/172 2024-07-26T13:26:35.725 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-5320.ge8f88bb1.e 129/172 2024-07-26T13:26:35.790 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-5320.ge8f88bb1. 130/172 2024-07-26T13:26:35.816 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-5320.ge8f88bb1.el9.noa 131/172 2024-07-26T13:26:36.065 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libtool-2.4.6-46.el9.x86_64 139/172 2024-07-26T13:26:36.092 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-radosgw-2:19.0.0-5320.ge8f88bb1.el9.x86_64 140/172 2024-07-26T13:26:36.115 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-5320.ge8f88bb1.el9.x86_64 140/172 2024-07-26T13:26:36.115 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-26T13:26:36.115 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-07-26T13:26:36.115 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-26T13:26:36.115 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-26T13:26:36.116 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-07-26T13:26:36.160 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-5320.ge8f88 141/172 2024-07-26T13:26:36.181 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-5320.ge8f88 141/172 2024-07-26T13:26:36.181 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-26T13:26:36.182 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-07-26T13:26:36.182 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-07-26T13:26:36.227 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-mirror-2:19.0.0-5320.ge8f88bb1.el9.x86_64 142/172 2024-07-26T13:26:36.249 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-5320.ge8f88bb1.el9.x86_64 142/172 2024-07-26T13:26:36.249 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-26T13:26:36.249 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-07-26T13:26:36.249 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-26T13:26:36.249 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-26T13:26:36.249 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-07-26T13:26:36.569 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-mirror-2:19.0.0-5320.ge8f88bb1.el9.x86_64 143/172 2024-07-26T13:26:36.591 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-mirror-2:19.0.0-5320.ge8f88bb1.el9.x86_64 143/172 2024-07-26T13:26:36.592 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-26T13:26:36.592 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-07-26T13:26:36.592 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-26T13:26:36.592 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-26T13:26:36.592 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-07-26T13:26:37.177 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-osd-2:19.0.0-5320.ge8f88bb1.el9.x86_64 114/172 2024-07-26T13:26:37.203 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-osd-2:19.0.0-5320.ge8f88bb1.el9.x86_64 114/172 2024-07-26T13:26:37.203 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-26T13:26:37.203 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-07-26T13:26:37.203 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-26T13:26:37.203 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-26T13:26:37.203 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-07-26T13:26:37.428 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mds-2:19.0.0-5320.ge8f88bb1.el9.x86_64 115/172 2024-07-26T13:26:37.450 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mds-2:19.0.0-5320.ge8f88bb1.el9.x86_64 115/172 2024-07-26T13:26:37.450 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-26T13:26:37.451 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-07-26T13:26:37.451 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-26T13:26:37.451 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-26T13:26:37.451 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-07-26T13:26:37.912 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mon-2:19.0.0-5320.ge8f88bb1.el9.x86_64 116/172 2024-07-26T13:26:37.934 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mon-2:19.0.0-5320.ge8f88bb1.el9.x86_64 116/172 2024-07-26T13:26:37.935 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-26T13:26:37.935 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-07-26T13:26:37.935 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-26T13:26:37.935 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-26T13:26:37.935 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-07-26T13:26:38.051 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : automake-1.16.2-8.el9.noarch 117/172 2024-07-26T13:26:38.107 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/172 2024-07-26T13:26:38.171 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/172 2024-07-26T13:26:38.356 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/172 2024-07-26T13:26:38.404 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 121/172 2024-07-26T13:26:38.474 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/172 2024-07-26T13:26:38.571 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 123/172 2024-07-26T13:26:38.710 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/172 2024-07-26T13:26:38.755 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/172 2024-07-26T13:26:38.787 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/172 2024-07-26T13:26:38.815 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : fuse-2.9.9-16.el9.x86_64 127/172 2024-07-26T13:26:38.939 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-5320.ge8f88bb1.el9.noa 131/172 2024-07-26T13:26:38.996 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-5320.ge8f88bb1.el9.n 132/172 2024-07-26T13:26:39.276 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephadm-2:19.0.0-5320.ge8f88bb1.el9.noarch 128/172 2024-07-26T13:26:39.305 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephadm-2:19.0.0-5320.ge8f88bb1.el9.noarch 128/172 2024-07-26T13:26:39.346 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-5320.ge8f88bb1.e 129/172 2024-07-26T13:26:39.419 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-5320.ge8f88bb1. 130/172 2024-07-26T13:26:39.454 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-5320.ge8f88bb1.el9.noa 131/172 2024-07-26T13:26:39.954 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-5320.ge8f88bb1.el9.n 132/172 2024-07-26T13:26:39.989 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-5320.ge8f 133/172 2024-07-26T13:26:40.101 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-5320.ge8f 133/172 2024-07-26T13:26:40.273 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-5320.ge8f88bb1.el 134/172 2024-07-26T13:26:40.299 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-2:19.0.0-5320.ge8f88bb1.el9.x86_64 135/172 2024-07-26T13:26:40.323 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-2:19.0.0-5320.ge8f88bb1.el9.x86_64 135/172 2024-07-26T13:26:40.323 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-26T13:26:40.323 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-07-26T13:26:40.323 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-26T13:26:40.324 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-26T13:26:40.324 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:26:40.375 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-rook-2:19.0.0-5320.ge8f88bb1.el9.noarch 136/172 2024-07-26T13:26:40.386 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-5320.ge8f88bb1.el9.noarch 136/172 2024-07-26T13:26:40.486 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-2:19.0.0-5320.ge8f88bb1.el9.x86_64 137/172 2024-07-26T13:26:40.545 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-fuse-2:19.0.0-5320.ge8f88bb1.el9.x86_64 138/172 2024-07-26T13:26:40.983 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-test-2:19.0.0-5320.ge8f88bb1.el9.x86_64 144/172 2024-07-26T13:26:41.024 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-fuse-2:19.0.0-5320.ge8f88bb1.el9.x86_64 145/172 2024-07-26T13:26:41.065 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-nbd-2:19.0.0-5320.ge8f88bb1.el9.x86_64 146/172 2024-07-26T13:26:41.097 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xfsprogs-devel-6.4.0-3.el9.x86_64 147/172 2024-07-26T13:26:41.125 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 148/172 2024-07-26T13:26:41.311 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-shell-2:19.0.0-5320.ge8f88bb1.el9.x86_64 149/172 2024-07-26T13:26:41.355 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 150/172 2024-07-26T13:26:41.387 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs-devel-2:19.0.0-5320.ge8f88bb1.el9.x86_ 151/172 2024-07-26T13:26:41.420 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 152/172 2024-07-26T13:26:41.449 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : NetworkManager-team-1:1.48.2-2.el9.x86_64 153/172 2024-07-26T13:26:41.488 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : NetworkManager-tui-1:1.48.2-2.el9.x86_64 154/172 2024-07-26T13:26:41.509 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libtool-2.4.6-46.el9.x86_64 139/172 2024-07-26T13:26:41.544 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-radosgw-2:19.0.0-5320.ge8f88bb1.el9.x86_64 140/172 2024-07-26T13:26:41.568 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-5320.ge8f88bb1.el9.x86_64 140/172 2024-07-26T13:26:41.568 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-26T13:26:41.568 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-07-26T13:26:41.568 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-26T13:26:41.568 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-26T13:26:41.569 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:26:41.612 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-5320.ge8f88 141/172 2024-07-26T13:26:41.635 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-5320.ge8f88 141/172 2024-07-26T13:26:41.635 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-26T13:26:41.635 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-07-26T13:26:41.635 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:26:41.687 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-mirror-2:19.0.0-5320.ge8f88bb1.el9.x86_64 142/172 2024-07-26T13:26:41.711 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-5320.ge8f88bb1.el9.x86_64 142/172 2024-07-26T13:26:41.711 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-26T13:26:41.711 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-07-26T13:26:41.711 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-26T13:26:41.711 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-26T13:26:41.711 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:26:42.038 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-mirror-2:19.0.0-5320.ge8f88bb1.el9.x86_64 143/172 2024-07-26T13:26:42.058 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-26T13:26:42.061 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: rbd-mirror-2:19.0.0-5320.ge8f88bb1.el9.x86_64 143/172 2024-07-26T13:26:42.061 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-26T13:26:42.062 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-07-26T13:26:42.062 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-26T13:26:42.062 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-26T13:26:42.063 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:26:42.098 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-26T13:26:42.119 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-26T13:26:42.600 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-5320.ge8f88bb1.el9.noa 131/172 2024-07-26T13:26:42.669 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-5320.ge8f88bb1.el9.n 132/172 2024-07-26T13:26:42.953 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-top-2:19.0.0-5320.ge8f88bb1.el9.noarch 156/172 2024-07-26T13:26:43.058 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 157/172 2024-07-26T13:26:43.107 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 158/172 2024-07-26T13:26:43.190 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flex-2.6.4-9.el9.x86_64 159/172 2024-07-26T13:26:43.219 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : bison-3.7.4-5.el9.x86_64 160/172 2024-07-26T13:26:43.245 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/172 2024-07-26T13:26:43.270 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-26T13:26:43.297 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 163/172 2024-07-26T13:26:43.297 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-26T13:26:43.373 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-26T13:26:43.406 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 165/172 2024-07-26T13:26:43.407 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-26T13:26:43.491 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-26T13:26:43.516 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 167/172 2024-07-26T13:26:43.521 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-26T13:26:43.521 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-26T13:26:43.629 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-5320.ge8f88bb1.el9.n 132/172 2024-07-26T13:26:43.661 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-5320.ge8f 133/172 2024-07-26T13:26:43.693 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-26T13:26:43.693 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-26T13:26:43.773 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-5320.ge8f 133/172 2024-07-26T13:26:43.779 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-26T13:26:43.812 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-26T13:26:43.838 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 171/172 2024-07-26T13:26:43.902 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-26T13:26:43.937 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-5320.ge8f88bb1.el 134/172 2024-07-26T13:26:43.963 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-2:19.0.0-5320.ge8f88bb1.el9.x86_64 135/172 2024-07-26T13:26:43.987 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-2:19.0.0-5320.ge8f88bb1.el9.x86_64 135/172 2024-07-26T13:26:43.987 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-26T13:26:43.987 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-07-26T13:26:43.987 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-26T13:26:43.988 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-26T13:26:43.988 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-07-26T13:26:44.031 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-rook-2:19.0.0-5320.ge8f88bb1.el9.noarch 136/172 2024-07-26T13:26:44.042 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-5320.ge8f88bb1.el9.noarch 136/172 2024-07-26T13:26:44.141 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-2:19.0.0-5320.ge8f88bb1.el9.x86_64 137/172 2024-07-26T13:26:44.188 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-fuse-2:19.0.0-5320.ge8f88bb1.el9.x86_64 138/172 2024-07-26T13:26:45.154 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libtool-2.4.6-46.el9.x86_64 139/172 2024-07-26T13:26:45.192 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-radosgw-2:19.0.0-5320.ge8f88bb1.el9.x86_64 140/172 2024-07-26T13:26:45.215 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-5320.ge8f88bb1.el9.x86_64 140/172 2024-07-26T13:26:45.215 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-26T13:26:45.215 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-07-26T13:26:45.215 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-26T13:26:45.215 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-26T13:26:45.215 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-07-26T13:26:45.259 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-5320.ge8f88 141/172 2024-07-26T13:26:45.281 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-5320.ge8f88 141/172 2024-07-26T13:26:45.281 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-26T13:26:45.281 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-07-26T13:26:45.281 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-07-26T13:26:45.326 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-mirror-2:19.0.0-5320.ge8f88bb1.el9.x86_64 142/172 2024-07-26T13:26:45.349 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-5320.ge8f88bb1.el9.x86_64 142/172 2024-07-26T13:26:45.349 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-26T13:26:45.349 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-07-26T13:26:45.349 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-26T13:26:45.349 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-26T13:26:45.349 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-07-26T13:26:45.669 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-mirror-2:19.0.0-5320.ge8f88bb1.el9.x86_64 143/172 2024-07-26T13:26:45.692 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: rbd-mirror-2:19.0.0-5320.ge8f88bb1.el9.x86_64 143/172 2024-07-26T13:26:45.692 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-26T13:26:45.692 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-07-26T13:26:45.692 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-26T13:26:45.692 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-26T13:26:45.692 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-07-26T13:26:46.442 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-test-2:19.0.0-5320.ge8f88bb1.el9.x86_64 144/172 2024-07-26T13:26:46.483 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-fuse-2:19.0.0-5320.ge8f88bb1.el9.x86_64 145/172 2024-07-26T13:26:46.523 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-nbd-2:19.0.0-5320.ge8f88bb1.el9.x86_64 146/172 2024-07-26T13:26:46.564 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : xfsprogs-devel-6.4.0-3.el9.x86_64 147/172 2024-07-26T13:26:46.600 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 148/172 2024-07-26T13:26:46.794 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-shell-2:19.0.0-5320.ge8f88bb1.el9.x86_64 149/172 2024-07-26T13:26:46.846 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 150/172 2024-07-26T13:26:46.879 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephfs-devel-2:19.0.0-5320.ge8f88bb1.el9.x86_ 151/172 2024-07-26T13:26:46.919 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 152/172 2024-07-26T13:26:46.948 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-team-1:1.48.2-2.el9.x86_64 153/172 2024-07-26T13:26:47.003 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-tui-1:1.48.2-2.el9.x86_64 154/172 2024-07-26T13:26:47.624 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-26T13:26:47.659 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-26T13:26:47.681 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-26T13:26:48.526 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-top-2:19.0.0-5320.ge8f88bb1.el9.noarch 156/172 2024-07-26T13:26:48.614 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 157/172 2024-07-26T13:26:48.655 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 158/172 2024-07-26T13:26:48.745 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flex-2.6.4-9.el9.x86_64 159/172 2024-07-26T13:26:48.776 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : bison-3.7.4-5.el9.x86_64 160/172 2024-07-26T13:26:48.802 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/172 2024-07-26T13:26:48.826 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-26T13:26:48.854 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 163/172 2024-07-26T13:26:48.854 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-26T13:26:48.938 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-26T13:26:48.979 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 165/172 2024-07-26T13:26:48.979 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-26T13:26:49.063 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-26T13:26:49.088 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 167/172 2024-07-26T13:26:49.094 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-26T13:26:49.094 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-26T13:26:49.290 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-26T13:26:49.290 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-26T13:26:49.358 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-26T13:26:49.382 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-26T13:26:49.417 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 171/172 2024-07-26T13:26:49.491 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-26T13:26:50.052 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-test-2:19.0.0-5320.ge8f88bb1.el9.x86_64 144/172 2024-07-26T13:26:50.090 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-fuse-2:19.0.0-5320.ge8f88bb1.el9.x86_64 145/172 2024-07-26T13:26:50.139 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-nbd-2:19.0.0-5320.ge8f88bb1.el9.x86_64 146/172 2024-07-26T13:26:50.179 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : xfsprogs-devel-6.4.0-3.el9.x86_64 147/172 2024-07-26T13:26:50.215 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 148/172 2024-07-26T13:26:50.395 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-shell-2:19.0.0-5320.ge8f88bb1.el9.x86_64 149/172 2024-07-26T13:26:50.438 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 150/172 2024-07-26T13:26:50.470 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs-devel-2:19.0.0-5320.ge8f88bb1.el9.x86_ 151/172 2024-07-26T13:26:50.511 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 152/172 2024-07-26T13:26:50.534 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-26T13:26:50.534 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-2:19.0.0-5320.ge8f88bb1.el9.x86_64 1/172 2024-07-26T13:26:50.534 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-base-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2/172 2024-07-26T13:26:50.534 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-common-2:19.0.0-5320.ge8f88bb1.el9.x86_64 3/172 2024-07-26T13:26:50.534 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-fuse-2:19.0.0-5320.ge8f88bb1.el9.x86_64 4/172 2024-07-26T13:26:50.534 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-5320.ge8f88 5/172 2024-07-26T13:26:50.534 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mds-2:19.0.0-5320.ge8f88bb1.el9.x86_64 6/172 2024-07-26T13:26:50.535 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-2:19.0.0-5320.ge8f88bb1.el9.x86_64 7/172 2024-07-26T13:26:50.535 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mon-2:19.0.0-5320.ge8f88bb1.el9.x86_64 8/172 2024-07-26T13:26:50.535 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-osd-2:19.0.0-5320.ge8f88bb1.el9.x86_64 9/172 2024-07-26T13:26:50.535 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-radosgw-2:19.0.0-5320.ge8f88bb1.el9.x86_64 10/172 2024-07-26T13:26:50.535 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-selinux-2:19.0.0-5320.ge8f88bb1.el9.x86_64 11/172 2024-07-26T13:26:50.535 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-test-2:19.0.0-5320.ge8f88bb1.el9.x86_64 12/172 2024-07-26T13:26:50.535 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-mirror-2:19.0.0-5320.ge8f88bb1.el9.x86_64 13/172 2024-07-26T13:26:50.535 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-shell-2:19.0.0-5320.ge8f88bb1.el9.x86_64 14/172 2024-07-26T13:26:50.535 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs-devel-2:19.0.0-5320.ge8f88bb1.el9.x86_ 15/172 2024-07-26T13:26:50.535 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs2-2:19.0.0-5320.ge8f88bb1.el9.x86_64 16/172 2024-07-26T13:26:50.535 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephsqlite-2:19.0.0-5320.ge8f88bb1.el9.x86_64 17/172 2024-07-26T13:26:50.535 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados-devel-2:19.0.0-5320.ge8f88bb1.el9.x86_6 18/172 2024-07-26T13:26:50.535 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libradosstriper1-2:19.0.0-5320.ge8f88bb1.el9.x86 19/172 2024-07-26T13:26:50.536 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librgw2-2:19.0.0-5320.ge8f88bb1.el9.x86_64 20/172 2024-07-26T13:26:50.536 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-argparse-2:19.0.0-5320.ge8f88bb1.el 21/172 2024-07-26T13:26:50.536 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-common-2:19.0.0-5320.ge8f88bb1.el9. 22/172 2024-07-26T13:26:50.536 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cephfs-2:19.0.0-5320.ge8f88bb1.el9.x86_6 23/172 2024-07-26T13:26:50.536 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rados-2:19.0.0-5320.ge8f88bb1.el9.x86_64 24/172 2024-07-26T13:26:50.536 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rbd-2:19.0.0-5320.ge8f88bb1.el9.x86_64 25/172 2024-07-26T13:26:50.536 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rgw-2:19.0.0-5320.ge8f88bb1.el9.x86_64 26/172 2024-07-26T13:26:50.536 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-fuse-2:19.0.0-5320.ge8f88bb1.el9.x86_64 27/172 2024-07-26T13:26:50.536 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-mirror-2:19.0.0-5320.ge8f88bb1.el9.x86_64 28/172 2024-07-26T13:26:50.536 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-nbd-2:19.0.0-5320.ge8f88bb1.el9.x86_64 29/172 2024-07-26T13:26:50.536 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-5320.ge8f88bb1. 30/172 2024-07-26T13:26:50.536 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-5320.ge8f88bb1.el9.noa 31/172 2024-07-26T13:26:50.536 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-5320.ge8f88bb1.el9.n 32/172 2024-07-26T13:26:50.536 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-5320.ge8f 33/172 2024-07-26T13:26:50.536 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-5320.ge8f88bb1.el 34/172 2024-07-26T13:26:50.537 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-rook-2:19.0.0-5320.ge8f88bb1.el9.noarch 35/172 2024-07-26T13:26:50.537 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-5320.ge8f88bb1.e 36/172 2024-07-26T13:26:50.537 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-volume-2:19.0.0-5320.ge8f88bb1.el9.noarch 37/172 2024-07-26T13:26:50.537 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephadm-2:19.0.0-5320.ge8f88bb1.el9.noarch 38/172 2024-07-26T13:26:50.537 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-top-2:19.0.0-5320.ge8f88bb1.el9.noarch 39/172 2024-07-26T13:26:50.537 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/172 2024-07-26T13:26:50.537 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/172 2024-07-26T13:26:50.537 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/172 2024-07-26T13:26:50.537 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/172 2024-07-26T13:26:50.537 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/172 2024-07-26T13:26:50.537 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/172 2024-07-26T13:26:50.537 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/172 2024-07-26T13:26:50.537 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/172 2024-07-26T13:26:50.538 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/172 2024-07-26T13:26:50.538 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/172 2024-07-26T13:26:50.538 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/172 2024-07-26T13:26:50.538 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/172 2024-07-26T13:26:50.538 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/172 2024-07-26T13:26:50.538 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/172 2024-07-26T13:26:50.538 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/172 2024-07-26T13:26:50.538 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/172 2024-07-26T13:26:50.538 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/172 2024-07-26T13:26:50.538 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/172 2024-07-26T13:26:50.538 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/172 2024-07-26T13:26:50.538 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/172 2024-07-26T13:26:50.538 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/172 2024-07-26T13:26:50.538 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/172 2024-07-26T13:26:50.539 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/172 2024-07-26T13:26:50.539 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/172 2024-07-26T13:26:50.539 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/172 2024-07-26T13:26:50.539 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/172 2024-07-26T13:26:50.539 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/172 2024-07-26T13:26:50.539 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/172 2024-07-26T13:26:50.539 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/172 2024-07-26T13:26:50.539 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/172 2024-07-26T13:26:50.539 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/172 2024-07-26T13:26:50.539 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/172 2024-07-26T13:26:50.539 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/172 2024-07-26T13:26:50.539 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/172 2024-07-26T13:26:50.539 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/172 2024-07-26T13:26:50.539 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/172 2024-07-26T13:26:50.540 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/172 2024-07-26T13:26:50.540 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/172 2024-07-26T13:26:50.540 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/172 2024-07-26T13:26:50.540 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/172 2024-07-26T13:26:50.540 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/172 2024-07-26T13:26:50.540 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/172 2024-07-26T13:26:50.540 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/172 2024-07-26T13:26:50.540 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/172 2024-07-26T13:26:50.540 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/172 2024-07-26T13:26:50.540 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-26T13:26:50.540 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/172 2024-07-26T13:26:50.540 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/172 2024-07-26T13:26:50.540 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/172 2024-07-26T13:26:50.540 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/172 2024-07-26T13:26:50.541 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/172 2024-07-26T13:26:50.541 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/172 2024-07-26T13:26:50.541 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/172 2024-07-26T13:26:50.541 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/172 2024-07-26T13:26:50.541 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/172 2024-07-26T13:26:50.541 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/172 2024-07-26T13:26:50.541 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/172 2024-07-26T13:26:50.541 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/172 2024-07-26T13:26:50.541 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/172 2024-07-26T13:26:50.541 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/172 2024-07-26T13:26:50.541 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/172 2024-07-26T13:26:50.541 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xfsprogs-devel-6.4.0-3.el9.x86_64 101/172 2024-07-26T13:26:50.541 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/172 2024-07-26T13:26:50.541 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/172 2024-07-26T13:26:50.541 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/172 2024-07-26T13:26:50.542 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/172 2024-07-26T13:26:50.542 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/172 2024-07-26T13:26:50.542 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/172 2024-07-26T13:26:50.542 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/172 2024-07-26T13:26:50.542 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/172 2024-07-26T13:26:50.542 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/172 2024-07-26T13:26:50.542 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/172 2024-07-26T13:26:50.542 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/172 2024-07-26T13:26:50.542 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/172 2024-07-26T13:26:50.542 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/172 2024-07-26T13:26:50.542 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/172 2024-07-26T13:26:50.542 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/172 2024-07-26T13:26:50.543 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/172 2024-07-26T13:26:50.543 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/172 2024-07-26T13:26:50.543 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/172 2024-07-26T13:26:50.543 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/172 2024-07-26T13:26:50.543 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/172 2024-07-26T13:26:50.543 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-google-auth-1:2.32.0-1.el9.noarch 122/172 2024-07-26T13:26:50.543 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/172 2024-07-26T13:26:50.543 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/172 2024-07-26T13:26:50.543 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/172 2024-07-26T13:26:50.543 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/172 2024-07-26T13:26:50.543 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/172 2024-07-26T13:26:50.543 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/172 2024-07-26T13:26:50.543 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/172 2024-07-26T13:26:50.543 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/172 2024-07-26T13:26:50.544 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/172 2024-07-26T13:26:50.544 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/172 2024-07-26T13:26:50.544 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/172 2024-07-26T13:26:50.544 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/172 2024-07-26T13:26:50.544 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/172 2024-07-26T13:26:50.544 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/172 2024-07-26T13:26:50.544 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/172 2024-07-26T13:26:50.544 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/172 2024-07-26T13:26:50.544 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/172 2024-07-26T13:26:50.544 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/172 2024-07-26T13:26:50.544 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/172 2024-07-26T13:26:50.544 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/172 2024-07-26T13:26:50.544 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/172 2024-07-26T13:26:50.544 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/172 2024-07-26T13:26:50.544 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/172 2024-07-26T13:26:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/172 2024-07-26T13:26:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/172 2024-07-26T13:26:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/172 2024-07-26T13:26:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:19.0.0-5320.ge8f88bb1.el9.x86_64 149/172 2024-07-26T13:26:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/172 2024-07-26T13:26:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:19.0.0-5320.ge8f88bb1.el9.x86_64 151/172 2024-07-26T13:26:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/172 2024-07-26T13:26:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-1:1.48.2-2.el9.x86_64 153/172 2024-07-26T13:26:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/172 2024-07-26T13:26:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 155/172 2024-07-26T13:26:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/172 2024-07-26T13:26:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-team-1:1.48.2-2.el9.x86_64 157/172 2024-07-26T13:26:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/172 2024-07-26T13:26:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-tui-1:1.48.2-2.el9.x86_64 159/172 2024-07-26T13:26:50.546 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/172 2024-07-26T13:26:50.546 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/172 2024-07-26T13:26:50.546 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-26T13:26:50.546 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/172 2024-07-26T13:26:50.546 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/172 2024-07-26T13:26:50.546 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/172 2024-07-26T13:26:50.546 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/172 2024-07-26T13:26:50.546 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-1:3.2.2-2.el9.x86_64 167/172 2024-07-26T13:26:50.546 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/172 2024-07-26T13:26:50.546 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-libs-1:3.2.2-2.el9.x86_64 169/172 2024-07-26T13:26:50.546 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-26T13:26:50.546 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xfsprogs-6.4.0-3.el9.x86_64 171/172 2024-07-26T13:26:50.553 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : NetworkManager-team-1:1.48.2-2.el9.x86_64 153/172 2024-07-26T13:26:50.600 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : NetworkManager-tui-1:1.48.2-2.el9.x86_64 154/172 2024-07-26T13:26:51.247 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-26T13:26:51.300 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-26T13:26:51.320 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-26T13:26:52.158 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-top-2:19.0.0-5320.ge8f88bb1.el9.noarch 156/172 2024-07-26T13:26:52.249 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 157/172 2024-07-26T13:26:52.290 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 158/172 2024-07-26T13:26:52.371 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flex-2.6.4-9.el9.x86_64 159/172 2024-07-26T13:26:52.411 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : bison-3.7.4-5.el9.x86_64 160/172 2024-07-26T13:26:52.437 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/172 2024-07-26T13:26:52.462 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-26T13:26:52.489 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 163/172 2024-07-26T13:26:52.489 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-26T13:26:52.573 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-26T13:26:52.623 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 165/172 2024-07-26T13:26:52.623 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-26T13:26:52.708 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-26T13:26:52.733 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 167/172 2024-07-26T13:26:52.738 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-26T13:26:52.738 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-26T13:26:52.919 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-26T13:26:52.919 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-26T13:26:53.003 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-26T13:26:53.028 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-26T13:26:53.062 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 171/172 2024-07-26T13:26:53.134 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-26T13:26:53.263 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-26T13:26:53.263 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-07-26T13:26:53.263 INFO:teuthology.orchestra.run.smithi138.stdout:Upgraded: 2024-07-26T13:26:53.263 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-1:1.48.2-2.el9.x86_64 2024-07-26T13:26:53.263 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 2024-07-26T13:26:53.263 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-team-1:1.48.2-2.el9.x86_64 2024-07-26T13:26:53.264 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-tui-1:1.48.2-2.el9.x86_64 2024-07-26T13:26:53.264 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-07-26T13:26:53.264 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-07-26T13:26:53.264 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-07-26T13:26:53.264 INFO:teuthology.orchestra.run.smithi138.stdout: librados2-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:53.264 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:53.264 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-1:3.2.2-2.el9.x86_64 2024-07-26T13:26:53.264 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-libs-1:3.2.2-2.el9.x86_64 2024-07-26T13:26:53.264 INFO:teuthology.orchestra.run.smithi138.stdout: xfsprogs-6.4.0-3.el9.x86_64 2024-07-26T13:26:53.264 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-07-26T13:26:53.264 INFO:teuthology.orchestra.run.smithi138.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-07-26T13:26:53.264 INFO:teuthology.orchestra.run.smithi138.stdout: automake-1.16.2-8.el9.noarch 2024-07-26T13:26:53.264 INFO:teuthology.orchestra.run.smithi138.stdout: bison-3.7.4-5.el9.x86_64 2024-07-26T13:26:53.264 INFO:teuthology.orchestra.run.smithi138.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-07-26T13:26:53.265 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:53.265 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:53.265 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:53.265 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:53.265 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards-2:19.0.0-5320.ge8f88bb1.el9.noarch 2024-07-26T13:26:53.265 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:53.265 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:53.265 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:53.265 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm-2:19.0.0-5320.ge8f88bb1.el9.noarch 2024-07-26T13:26:53.265 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard-2:19.0.0-5320.ge8f88bb1.el9.noarch 2024-07-26T13:26:53.265 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local-2:19.0.0-5320.ge8f88bb1.el9.noarch 2024-07-26T13:26:53.265 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core-2:19.0.0-5320.ge8f88bb1.el9.noarch 2024-07-26T13:26:53.265 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook-2:19.0.0-5320.ge8f88bb1.el9.noarch 2024-07-26T13:26:53.265 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:53.266 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:53.266 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts-2:19.0.0-5320.ge8f88bb1.el9.noarch 2024-07-26T13:26:53.266 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:53.266 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:53.266 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:53.266 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-volume-2:19.0.0-5320.ge8f88bb1.el9.noarch 2024-07-26T13:26:53.266 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm-2:19.0.0-5320.ge8f88bb1.el9.noarch 2024-07-26T13:26:53.266 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-mirror-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:53.266 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-shell-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:53.266 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-top-2:19.0.0-5320.ge8f88bb1.el9.noarch 2024-07-26T13:26:53.266 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-07-26T13:26:53.266 INFO:teuthology.orchestra.run.smithi138.stdout: flex-2.6.4-9.el9.x86_64 2024-07-26T13:26:53.266 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-07-26T13:26:53.267 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-07-26T13:26:53.267 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-07-26T13:26:53.267 INFO:teuthology.orchestra.run.smithi138.stdout: fuse-2.9.9-16.el9.x86_64 2024-07-26T13:26:53.267 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-07-26T13:26:53.267 INFO:teuthology.orchestra.run.smithi138.stdout: grpc-data-1.46.7-10.el9.noarch 2024-07-26T13:26:53.267 INFO:teuthology.orchestra.run.smithi138.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-07-26T13:26:53.267 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-9.0.0-11.el9.x86_64 2024-07-26T13:26:53.267 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-07-26T13:26:53.267 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:53.267 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:53.267 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:53.268 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-07-26T13:26:53.268 INFO:teuthology.orchestra.run.smithi138.stdout: liboath-2.6.7-2.el9.x86_64 2024-07-26T13:26:53.268 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-07-26T13:26:53.268 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-07-26T13:26:53.268 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-07-26T13:26:53.268 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:53.268 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:53.268 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-07-26T13:26:53.268 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:53.268 INFO:teuthology.orchestra.run.smithi138.stdout: libtool-2.4.6-46.el9.x86_64 2024-07-26T13:26:53.268 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind-1.6.2-1.el9.x86_64 2024-07-26T13:26:53.268 INFO:teuthology.orchestra.run.smithi138.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-07-26T13:26:53.268 INFO:teuthology.orchestra.run.smithi138.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-07-26T13:26:53.269 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-07-26T13:26:53.269 INFO:teuthology.orchestra.run.smithi138.stdout: lua-5.4.4-4.el9.x86_64 2024-07-26T13:26:53.269 INFO:teuthology.orchestra.run.smithi138.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-07-26T13:26:53.269 INFO:teuthology.orchestra.run.smithi138.stdout: luarocks-3.9.2-5.el9.noarch 2024-07-26T13:26:53.269 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-0.3.21-2.el9.x86_64 2024-07-26T13:26:53.269 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-07-26T13:26:53.269 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-07-26T13:26:53.269 INFO:teuthology.orchestra.run.smithi138.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-07-26T13:26:53.270 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-13.14-1.el9.x86_64 2024-07-26T13:26:53.270 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-07-26T13:26:53.270 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-07-26T13:26:53.270 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-server-13.14-1.el9.x86_64 2024-07-26T13:26:53.270 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-3.14.0-13.el9.x86_64 2024-07-26T13:26:53.270 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-07-26T13:26:53.270 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-07-26T13:26:53.270 INFO:teuthology.orchestra.run.smithi138.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-07-26T13:26:53.270 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel-2.9.1-2.el9.noarch 2024-07-26T13:26:53.270 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-07-26T13:26:53.270 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-07-26T13:26:53.270 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:53.270 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:53.270 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:53.271 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-07-26T13:26:53.271 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-07-26T13:26:53.271 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-07-26T13:26:53.271 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-07-26T13:26:53.271 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-07-26T13:26:53.271 INFO:teuthology.orchestra.run.smithi138.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-07-26T13:26:53.271 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-07-26T13:26:53.271 INFO:teuthology.orchestra.run.smithi138.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-07-26T13:26:53.271 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth-1:2.32.0-1.el9.noarch 2024-07-26T13:26:53.271 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-07-26T13:26:53.271 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-07-26T13:26:53.271 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-07-26T13:26:53.271 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-07-26T13:26:53.271 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-07-26T13:26:53.272 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-07-26T13:26:53.272 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-07-26T13:26:53.272 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-07-26T13:26:53.272 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-07-26T13:26:53.272 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-07-26T13:26:53.272 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-07-26T13:26:53.272 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-07-26T13:26:53.272 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako-1.1.4-6.el9.noarch 2024-07-26T13:26:53.272 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-07-26T13:26:53.272 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-07-26T13:26:53.272 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-07-26T13:26:53.272 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-07-26T13:26:53.272 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-07-26T13:26:53.272 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-07-26T13:26:53.272 INFO:teuthology.orchestra.run.smithi138.stdout: python3-packaging-20.9-5.el9.noarch 2024-07-26T13:26:53.273 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-07-26T13:26:53.273 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply-3.11-14.el9.noarch 2024-07-26T13:26:53.273 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend-3.1.0-2.el9.noarch 2024-07-26T13:26:53.273 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-07-26T13:26:53.273 INFO:teuthology.orchestra.run.smithi138.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-07-26T13:26:53.273 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-07-26T13:26:53.273 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-07-26T13:26:53.273 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-07-26T13:26:53.273 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser-2.20-6.el9.noarch 2024-07-26T13:26:53.273 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-07-26T13:26:53.273 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz-2021.1-5.el9.noarch 2024-07-26T13:26:53.273 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:53.273 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:53.273 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-07-26T13:26:53.273 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-2.25.1-8.el9.noarch 2024-07-26T13:26:53.274 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-07-26T13:26:53.274 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:53.274 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes-2.5.1-5.el9.noarch 2024-07-26T13:26:53.274 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa-4.9-2.el9.noarch 2024-07-26T13:26:53.274 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-07-26T13:26:53.274 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-07-26T13:26:53.274 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml-0.10.2-6.el9.noarch 2024-07-26T13:26:53.274 INFO:teuthology.orchestra.run.smithi138.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-07-26T13:26:53.274 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-07-26T13:26:53.274 INFO:teuthology.orchestra.run.smithi138.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-07-26T13:26:53.274 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob-1.8.7-6.el9.noarch 2024-07-26T13:26:53.274 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-07-26T13:26:53.274 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-07-26T13:26:53.274 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-07-26T13:26:53.275 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-24.02.0-1.el9.x86_64 2024-07-26T13:26:53.275 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-07-26T13:26:53.275 INFO:teuthology.orchestra.run.smithi138.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-07-26T13:26:53.275 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:53.275 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:53.275 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:53.275 INFO:teuthology.orchestra.run.smithi138.stdout: re2-1:20211101-3.el9.x86_64 2024-07-26T13:26:53.275 INFO:teuthology.orchestra.run.smithi138.stdout: socat-1.7.4.1-6.el9.x86_64 2024-07-26T13:26:53.275 INFO:teuthology.orchestra.run.smithi138.stdout: thrift-0.15.0-2.el9.x86_64 2024-07-26T13:26:53.275 INFO:teuthology.orchestra.run.smithi138.stdout: uuid-1.6.2-55.el9.x86_64 2024-07-26T13:26:53.275 INFO:teuthology.orchestra.run.smithi138.stdout: xfsprogs-devel-6.4.0-3.el9.x86_64 2024-07-26T13:26:53.275 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-07-26T13:26:53.276 INFO:teuthology.orchestra.run.smithi138.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-07-26T13:26:53.276 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-07-26T13:26:53.276 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-07-26T13:26:53.719 DEBUG:teuthology.parallel:result is None 2024-07-26T13:26:55.999 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-26T13:26:55.999 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-2:19.0.0-5320.ge8f88bb1.el9.x86_64 1/172 2024-07-26T13:26:55.999 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-base-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2/172 2024-07-26T13:26:55.999 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-common-2:19.0.0-5320.ge8f88bb1.el9.x86_64 3/172 2024-07-26T13:26:55.999 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-fuse-2:19.0.0-5320.ge8f88bb1.el9.x86_64 4/172 2024-07-26T13:26:55.999 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-5320.ge8f88 5/172 2024-07-26T13:26:56.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mds-2:19.0.0-5320.ge8f88bb1.el9.x86_64 6/172 2024-07-26T13:26:56.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-2:19.0.0-5320.ge8f88bb1.el9.x86_64 7/172 2024-07-26T13:26:56.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mon-2:19.0.0-5320.ge8f88bb1.el9.x86_64 8/172 2024-07-26T13:26:56.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-osd-2:19.0.0-5320.ge8f88bb1.el9.x86_64 9/172 2024-07-26T13:26:56.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-radosgw-2:19.0.0-5320.ge8f88bb1.el9.x86_64 10/172 2024-07-26T13:26:56.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-selinux-2:19.0.0-5320.ge8f88bb1.el9.x86_64 11/172 2024-07-26T13:26:56.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-test-2:19.0.0-5320.ge8f88bb1.el9.x86_64 12/172 2024-07-26T13:26:56.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-mirror-2:19.0.0-5320.ge8f88bb1.el9.x86_64 13/172 2024-07-26T13:26:56.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-shell-2:19.0.0-5320.ge8f88bb1.el9.x86_64 14/172 2024-07-26T13:26:56.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephfs-devel-2:19.0.0-5320.ge8f88bb1.el9.x86_ 15/172 2024-07-26T13:26:56.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephfs2-2:19.0.0-5320.ge8f88bb1.el9.x86_64 16/172 2024-07-26T13:26:56.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephsqlite-2:19.0.0-5320.ge8f88bb1.el9.x86_64 17/172 2024-07-26T13:26:56.000 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados-devel-2:19.0.0-5320.ge8f88bb1.el9.x86_6 18/172 2024-07-26T13:26:56.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libradosstriper1-2:19.0.0-5320.ge8f88bb1.el9.x86 19/172 2024-07-26T13:26:56.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librgw2-2:19.0.0-5320.ge8f88bb1.el9.x86_64 20/172 2024-07-26T13:26:56.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ceph-argparse-2:19.0.0-5320.ge8f88bb1.el 21/172 2024-07-26T13:26:56.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ceph-common-2:19.0.0-5320.ge8f88bb1.el9. 22/172 2024-07-26T13:26:56.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cephfs-2:19.0.0-5320.ge8f88bb1.el9.x86_6 23/172 2024-07-26T13:26:56.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rados-2:19.0.0-5320.ge8f88bb1.el9.x86_64 24/172 2024-07-26T13:26:56.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rbd-2:19.0.0-5320.ge8f88bb1.el9.x86_64 25/172 2024-07-26T13:26:56.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rgw-2:19.0.0-5320.ge8f88bb1.el9.x86_64 26/172 2024-07-26T13:26:56.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-fuse-2:19.0.0-5320.ge8f88bb1.el9.x86_64 27/172 2024-07-26T13:26:56.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-mirror-2:19.0.0-5320.ge8f88bb1.el9.x86_64 28/172 2024-07-26T13:26:56.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-nbd-2:19.0.0-5320.ge8f88bb1.el9.x86_64 29/172 2024-07-26T13:26:56.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-5320.ge8f88bb1. 30/172 2024-07-26T13:26:56.001 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-5320.ge8f88bb1.el9.noa 31/172 2024-07-26T13:26:56.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-5320.ge8f88bb1.el9.n 32/172 2024-07-26T13:26:56.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-5320.ge8f 33/172 2024-07-26T13:26:56.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-5320.ge8f88bb1.el 34/172 2024-07-26T13:26:56.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-rook-2:19.0.0-5320.ge8f88bb1.el9.noarch 35/172 2024-07-26T13:26:56.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-5320.ge8f88bb1.e 36/172 2024-07-26T13:26:56.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-volume-2:19.0.0-5320.ge8f88bb1.el9.noarch 37/172 2024-07-26T13:26:56.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephadm-2:19.0.0-5320.ge8f88bb1.el9.noarch 38/172 2024-07-26T13:26:56.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-top-2:19.0.0-5320.ge8f88bb1.el9.noarch 39/172 2024-07-26T13:26:56.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/172 2024-07-26T13:26:56.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/172 2024-07-26T13:26:56.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/172 2024-07-26T13:26:56.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/172 2024-07-26T13:26:56.002 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/172 2024-07-26T13:26:56.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/172 2024-07-26T13:26:56.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/172 2024-07-26T13:26:56.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/172 2024-07-26T13:26:56.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/172 2024-07-26T13:26:56.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/172 2024-07-26T13:26:56.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/172 2024-07-26T13:26:56.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/172 2024-07-26T13:26:56.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/172 2024-07-26T13:26:56.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/172 2024-07-26T13:26:56.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/172 2024-07-26T13:26:56.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/172 2024-07-26T13:26:56.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/172 2024-07-26T13:26:56.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/172 2024-07-26T13:26:56.003 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/172 2024-07-26T13:26:56.004 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/172 2024-07-26T13:26:56.004 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/172 2024-07-26T13:26:56.004 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/172 2024-07-26T13:26:56.004 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/172 2024-07-26T13:26:56.004 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/172 2024-07-26T13:26:56.004 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/172 2024-07-26T13:26:56.004 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/172 2024-07-26T13:26:56.004 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/172 2024-07-26T13:26:56.004 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/172 2024-07-26T13:26:56.004 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/172 2024-07-26T13:26:56.004 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/172 2024-07-26T13:26:56.004 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/172 2024-07-26T13:26:56.004 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/172 2024-07-26T13:26:56.005 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/172 2024-07-26T13:26:56.005 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/172 2024-07-26T13:26:56.005 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/172 2024-07-26T13:26:56.005 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/172 2024-07-26T13:26:56.005 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/172 2024-07-26T13:26:56.005 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/172 2024-07-26T13:26:56.005 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/172 2024-07-26T13:26:56.005 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/172 2024-07-26T13:26:56.005 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/172 2024-07-26T13:26:56.005 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/172 2024-07-26T13:26:56.005 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/172 2024-07-26T13:26:56.005 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/172 2024-07-26T13:26:56.005 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/172 2024-07-26T13:26:56.005 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-26T13:26:56.006 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/172 2024-07-26T13:26:56.006 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/172 2024-07-26T13:26:56.006 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/172 2024-07-26T13:26:56.006 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/172 2024-07-26T13:26:56.006 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/172 2024-07-26T13:26:56.006 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/172 2024-07-26T13:26:56.006 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/172 2024-07-26T13:26:56.006 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/172 2024-07-26T13:26:56.006 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/172 2024-07-26T13:26:56.006 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/172 2024-07-26T13:26:56.006 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/172 2024-07-26T13:26:56.006 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/172 2024-07-26T13:26:56.006 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/172 2024-07-26T13:26:56.007 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/172 2024-07-26T13:26:56.007 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/172 2024-07-26T13:26:56.007 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsprogs-devel-6.4.0-3.el9.x86_64 101/172 2024-07-26T13:26:56.007 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/172 2024-07-26T13:26:56.007 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/172 2024-07-26T13:26:56.007 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/172 2024-07-26T13:26:56.007 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/172 2024-07-26T13:26:56.007 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/172 2024-07-26T13:26:56.007 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/172 2024-07-26T13:26:56.007 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/172 2024-07-26T13:26:56.007 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/172 2024-07-26T13:26:56.007 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/172 2024-07-26T13:26:56.007 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/172 2024-07-26T13:26:56.008 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/172 2024-07-26T13:26:56.008 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/172 2024-07-26T13:26:56.008 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/172 2024-07-26T13:26:56.008 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/172 2024-07-26T13:26:56.008 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/172 2024-07-26T13:26:56.010 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/172 2024-07-26T13:26:56.010 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/172 2024-07-26T13:26:56.010 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/172 2024-07-26T13:26:56.010 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/172 2024-07-26T13:26:56.010 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/172 2024-07-26T13:26:56.010 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-google-auth-1:2.32.0-1.el9.noarch 122/172 2024-07-26T13:26:56.010 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/172 2024-07-26T13:26:56.010 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/172 2024-07-26T13:26:56.010 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/172 2024-07-26T13:26:56.010 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/172 2024-07-26T13:26:56.011 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/172 2024-07-26T13:26:56.011 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/172 2024-07-26T13:26:56.011 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/172 2024-07-26T13:26:56.011 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/172 2024-07-26T13:26:56.011 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/172 2024-07-26T13:26:56.011 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/172 2024-07-26T13:26:56.011 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/172 2024-07-26T13:26:56.011 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/172 2024-07-26T13:26:56.011 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/172 2024-07-26T13:26:56.011 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/172 2024-07-26T13:26:56.011 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/172 2024-07-26T13:26:56.011 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/172 2024-07-26T13:26:56.011 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/172 2024-07-26T13:26:56.011 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/172 2024-07-26T13:26:56.012 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/172 2024-07-26T13:26:56.012 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/172 2024-07-26T13:26:56.012 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/172 2024-07-26T13:26:56.012 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/172 2024-07-26T13:26:56.012 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/172 2024-07-26T13:26:56.012 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/172 2024-07-26T13:26:56.012 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/172 2024-07-26T13:26:56.012 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/172 2024-07-26T13:26:56.012 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados2-2:19.0.0-5320.ge8f88bb1.el9.x86_64 149/172 2024-07-26T13:26:56.012 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/172 2024-07-26T13:26:56.012 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librbd1-2:19.0.0-5320.ge8f88bb1.el9.x86_64 151/172 2024-07-26T13:26:56.012 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/172 2024-07-26T13:26:56.012 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-1:1.48.2-2.el9.x86_64 153/172 2024-07-26T13:26:56.012 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/172 2024-07-26T13:26:56.013 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 155/172 2024-07-26T13:26:56.013 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/172 2024-07-26T13:26:56.013 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-team-1:1.48.2-2.el9.x86_64 157/172 2024-07-26T13:26:56.013 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/172 2024-07-26T13:26:56.013 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-tui-1:1.48.2-2.el9.x86_64 159/172 2024-07-26T13:26:56.013 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/172 2024-07-26T13:26:56.013 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/172 2024-07-26T13:26:56.013 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-26T13:26:56.013 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/172 2024-07-26T13:26:56.013 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/172 2024-07-26T13:26:56.013 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/172 2024-07-26T13:26:56.013 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/172 2024-07-26T13:26:56.013 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-1:3.2.2-2.el9.x86_64 167/172 2024-07-26T13:26:56.013 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/172 2024-07-26T13:26:56.014 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-libs-1:3.2.2-2.el9.x86_64 169/172 2024-07-26T13:26:56.014 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-26T13:26:56.014 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsprogs-6.4.0-3.el9.x86_64 171/172 2024-07-26T13:26:58.885 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-26T13:26:58.885 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:26:58.885 INFO:teuthology.orchestra.run.smithi100.stdout:Upgraded: 2024-07-26T13:26:58.885 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-1:1.48.2-2.el9.x86_64 2024-07-26T13:26:58.885 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 2024-07-26T13:26:58.885 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-team-1:1.48.2-2.el9.x86_64 2024-07-26T13:26:58.886 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-tui-1:1.48.2-2.el9.x86_64 2024-07-26T13:26:58.886 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-07-26T13:26:58.886 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-07-26T13:26:58.886 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-07-26T13:26:58.886 INFO:teuthology.orchestra.run.smithi100.stdout: librados2-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:58.886 INFO:teuthology.orchestra.run.smithi100.stdout: librbd1-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:58.886 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-1:3.2.2-2.el9.x86_64 2024-07-26T13:26:58.886 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-libs-1:3.2.2-2.el9.x86_64 2024-07-26T13:26:58.886 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-6.4.0-3.el9.x86_64 2024-07-26T13:26:58.886 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-07-26T13:26:58.886 INFO:teuthology.orchestra.run.smithi100.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-07-26T13:26:58.886 INFO:teuthology.orchestra.run.smithi100.stdout: automake-1.16.2-8.el9.noarch 2024-07-26T13:26:58.886 INFO:teuthology.orchestra.run.smithi100.stdout: bison-3.7.4-5.el9.x86_64 2024-07-26T13:26:58.887 INFO:teuthology.orchestra.run.smithi100.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-07-26T13:26:58.887 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:58.887 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:58.887 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-common-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:58.887 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-fuse-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:58.887 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-grafana-dashboards-2:19.0.0-5320.ge8f88bb1.el9.noarch 2024-07-26T13:26:58.887 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-immutable-object-cache-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:58.887 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mds-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:58.887 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:58.887 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-cephadm-2:19.0.0-5320.ge8f88bb1.el9.noarch 2024-07-26T13:26:58.887 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard-2:19.0.0-5320.ge8f88bb1.el9.noarch 2024-07-26T13:26:58.887 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-diskprediction-local-2:19.0.0-5320.ge8f88bb1.el9.noarch 2024-07-26T13:26:58.888 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core-2:19.0.0-5320.ge8f88bb1.el9.noarch 2024-07-26T13:26:58.888 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-rook-2:19.0.0-5320.ge8f88bb1.el9.noarch 2024-07-26T13:26:58.888 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mon-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:58.888 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-osd-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:58.888 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-prometheus-alerts-2:19.0.0-5320.ge8f88bb1.el9.noarch 2024-07-26T13:26:58.888 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-radosgw-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:58.888 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-selinux-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:58.888 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-test-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:58.888 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-volume-2:19.0.0-5320.ge8f88bb1.el9.noarch 2024-07-26T13:26:58.888 INFO:teuthology.orchestra.run.smithi100.stdout: cephadm-2:19.0.0-5320.ge8f88bb1.el9.noarch 2024-07-26T13:26:58.889 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-mirror-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:58.889 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-shell-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:58.889 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-top-2:19.0.0-5320.ge8f88bb1.el9.noarch 2024-07-26T13:26:58.889 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-07-26T13:26:58.889 INFO:teuthology.orchestra.run.smithi100.stdout: flex-2.6.4-9.el9.x86_64 2024-07-26T13:26:58.889 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-07-26T13:26:58.889 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-07-26T13:26:58.889 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-07-26T13:26:58.889 INFO:teuthology.orchestra.run.smithi100.stdout: fuse-2.9.9-16.el9.x86_64 2024-07-26T13:26:58.889 INFO:teuthology.orchestra.run.smithi100.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-07-26T13:26:58.889 INFO:teuthology.orchestra.run.smithi100.stdout: grpc-data-1.46.7-10.el9.noarch 2024-07-26T13:26:58.890 INFO:teuthology.orchestra.run.smithi100.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-07-26T13:26:58.890 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-9.0.0-11.el9.x86_64 2024-07-26T13:26:58.890 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-07-26T13:26:58.890 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs-devel-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:58.890 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs2-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:58.890 INFO:teuthology.orchestra.run.smithi100.stdout: libcephsqlite-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:58.890 INFO:teuthology.orchestra.run.smithi100.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-07-26T13:26:58.890 INFO:teuthology.orchestra.run.smithi100.stdout: liboath-2.6.7-2.el9.x86_64 2024-07-26T13:26:58.890 INFO:teuthology.orchestra.run.smithi100.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-07-26T13:26:58.890 INFO:teuthology.orchestra.run.smithi100.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-07-26T13:26:58.890 INFO:teuthology.orchestra.run.smithi100.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-07-26T13:26:58.891 INFO:teuthology.orchestra.run.smithi100.stdout: librados-devel-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:58.891 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:58.891 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-07-26T13:26:58.891 INFO:teuthology.orchestra.run.smithi100.stdout: librgw2-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:58.891 INFO:teuthology.orchestra.run.smithi100.stdout: libtool-2.4.6-46.el9.x86_64 2024-07-26T13:26:58.891 INFO:teuthology.orchestra.run.smithi100.stdout: libunwind-1.6.2-1.el9.x86_64 2024-07-26T13:26:58.891 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-07-26T13:26:58.891 INFO:teuthology.orchestra.run.smithi100.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-07-26T13:26:58.891 INFO:teuthology.orchestra.run.smithi100.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-07-26T13:26:58.891 INFO:teuthology.orchestra.run.smithi100.stdout: lua-5.4.4-4.el9.x86_64 2024-07-26T13:26:58.891 INFO:teuthology.orchestra.run.smithi100.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-07-26T13:26:58.891 INFO:teuthology.orchestra.run.smithi100.stdout: luarocks-3.9.2-5.el9.noarch 2024-07-26T13:26:58.891 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-0.3.21-2.el9.x86_64 2024-07-26T13:26:58.891 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-07-26T13:26:58.892 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-07-26T13:26:58.892 INFO:teuthology.orchestra.run.smithi100.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-07-26T13:26:58.892 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-13.14-1.el9.x86_64 2024-07-26T13:26:58.892 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-07-26T13:26:58.892 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-07-26T13:26:58.892 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-server-13.14-1.el9.x86_64 2024-07-26T13:26:58.892 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-3.14.0-13.el9.x86_64 2024-07-26T13:26:58.892 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-07-26T13:26:58.892 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-07-26T13:26:58.892 INFO:teuthology.orchestra.run.smithi100.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-07-26T13:26:58.892 INFO:teuthology.orchestra.run.smithi100.stdout: python3-babel-2.9.1-2.el9.noarch 2024-07-26T13:26:58.892 INFO:teuthology.orchestra.run.smithi100.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-07-26T13:26:58.892 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-07-26T13:26:58.893 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:58.893 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-common-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:58.893 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cephfs-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:58.893 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-07-26T13:26:58.893 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-07-26T13:26:58.893 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-07-26T13:26:58.893 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-07-26T13:26:58.893 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-07-26T13:26:58.893 INFO:teuthology.orchestra.run.smithi100.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-07-26T13:26:58.893 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-07-26T13:26:58.893 INFO:teuthology.orchestra.run.smithi100.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-07-26T13:26:58.893 INFO:teuthology.orchestra.run.smithi100.stdout: python3-google-auth-1:2.32.0-1.el9.noarch 2024-07-26T13:26:58.893 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-07-26T13:26:58.893 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-07-26T13:26:58.894 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-07-26T13:26:58.894 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-07-26T13:26:58.894 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-07-26T13:26:58.894 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-07-26T13:26:58.894 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-07-26T13:26:58.894 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-07-26T13:26:58.894 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-07-26T13:26:58.894 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-07-26T13:26:58.894 INFO:teuthology.orchestra.run.smithi100.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-07-26T13:26:58.894 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-07-26T13:26:58.894 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako-1.1.4-6.el9.noarch 2024-07-26T13:26:58.894 INFO:teuthology.orchestra.run.smithi100.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-07-26T13:26:58.894 INFO:teuthology.orchestra.run.smithi100.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-07-26T13:26:58.894 INFO:teuthology.orchestra.run.smithi100.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-07-26T13:26:58.895 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-07-26T13:26:58.895 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-07-26T13:26:58.895 INFO:teuthology.orchestra.run.smithi100.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-07-26T13:26:58.895 INFO:teuthology.orchestra.run.smithi100.stdout: python3-packaging-20.9-5.el9.noarch 2024-07-26T13:26:58.895 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-07-26T13:26:58.895 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ply-3.11-14.el9.noarch 2024-07-26T13:26:58.895 INFO:teuthology.orchestra.run.smithi100.stdout: python3-portend-3.1.0-2.el9.noarch 2024-07-26T13:26:58.895 INFO:teuthology.orchestra.run.smithi100.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-07-26T13:26:58.895 INFO:teuthology.orchestra.run.smithi100.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-07-26T13:26:58.895 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-07-26T13:26:58.895 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-07-26T13:26:58.895 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-07-26T13:26:58.895 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pycparser-2.20-6.el9.noarch 2024-07-26T13:26:58.895 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-07-26T13:26:58.896 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytz-2021.1-5.el9.noarch 2024-07-26T13:26:58.896 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rados-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:58.896 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rbd-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:58.896 INFO:teuthology.orchestra.run.smithi100.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-07-26T13:26:58.896 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-2.25.1-8.el9.noarch 2024-07-26T13:26:58.896 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-07-26T13:26:58.896 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rgw-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:58.896 INFO:teuthology.orchestra.run.smithi100.stdout: python3-routes-2.5.1-5.el9.noarch 2024-07-26T13:26:58.896 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rsa-4.9-2.el9.noarch 2024-07-26T13:26:58.896 INFO:teuthology.orchestra.run.smithi100.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-07-26T13:26:58.896 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-07-26T13:26:58.896 INFO:teuthology.orchestra.run.smithi100.stdout: python3-toml-0.10.2-6.el9.noarch 2024-07-26T13:26:58.896 INFO:teuthology.orchestra.run.smithi100.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-07-26T13:26:58.896 INFO:teuthology.orchestra.run.smithi100.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-07-26T13:26:58.897 INFO:teuthology.orchestra.run.smithi100.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-07-26T13:26:58.897 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob-1.8.7-6.el9.noarch 2024-07-26T13:26:58.897 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-07-26T13:26:58.897 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-07-26T13:26:58.897 INFO:teuthology.orchestra.run.smithi100.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-07-26T13:26:58.897 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-24.02.0-1.el9.x86_64 2024-07-26T13:26:58.897 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-07-26T13:26:58.897 INFO:teuthology.orchestra.run.smithi100.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-07-26T13:26:58.897 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-fuse-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:58.897 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-mirror-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:58.897 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-nbd-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:26:58.897 INFO:teuthology.orchestra.run.smithi100.stdout: re2-1:20211101-3.el9.x86_64 2024-07-26T13:26:58.897 INFO:teuthology.orchestra.run.smithi100.stdout: socat-1.7.4.1-6.el9.x86_64 2024-07-26T13:26:58.897 INFO:teuthology.orchestra.run.smithi100.stdout: thrift-0.15.0-2.el9.x86_64 2024-07-26T13:26:58.898 INFO:teuthology.orchestra.run.smithi100.stdout: uuid-1.6.2-55.el9.x86_64 2024-07-26T13:26:58.898 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-devel-6.4.0-3.el9.x86_64 2024-07-26T13:26:58.898 INFO:teuthology.orchestra.run.smithi100.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-07-26T13:26:58.898 INFO:teuthology.orchestra.run.smithi100.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-07-26T13:26:58.898 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:26:58.898 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-07-26T13:26:59.317 DEBUG:teuthology.parallel:result is None 2024-07-26T13:26:59.637 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-26T13:26:59.638 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-2:19.0.0-5320.ge8f88bb1.el9.x86_64 1/172 2024-07-26T13:26:59.638 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-base-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2/172 2024-07-26T13:26:59.638 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-common-2:19.0.0-5320.ge8f88bb1.el9.x86_64 3/172 2024-07-26T13:26:59.638 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-fuse-2:19.0.0-5320.ge8f88bb1.el9.x86_64 4/172 2024-07-26T13:26:59.638 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-5320.ge8f88 5/172 2024-07-26T13:26:59.638 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mds-2:19.0.0-5320.ge8f88bb1.el9.x86_64 6/172 2024-07-26T13:26:59.638 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-2:19.0.0-5320.ge8f88bb1.el9.x86_64 7/172 2024-07-26T13:26:59.638 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mon-2:19.0.0-5320.ge8f88bb1.el9.x86_64 8/172 2024-07-26T13:26:59.638 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-osd-2:19.0.0-5320.ge8f88bb1.el9.x86_64 9/172 2024-07-26T13:26:59.638 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-radosgw-2:19.0.0-5320.ge8f88bb1.el9.x86_64 10/172 2024-07-26T13:26:59.638 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-selinux-2:19.0.0-5320.ge8f88bb1.el9.x86_64 11/172 2024-07-26T13:26:59.638 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-test-2:19.0.0-5320.ge8f88bb1.el9.x86_64 12/172 2024-07-26T13:26:59.639 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-mirror-2:19.0.0-5320.ge8f88bb1.el9.x86_64 13/172 2024-07-26T13:26:59.639 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-shell-2:19.0.0-5320.ge8f88bb1.el9.x86_64 14/172 2024-07-26T13:26:59.639 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs-devel-2:19.0.0-5320.ge8f88bb1.el9.x86_ 15/172 2024-07-26T13:26:59.639 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs2-2:19.0.0-5320.ge8f88bb1.el9.x86_64 16/172 2024-07-26T13:26:59.639 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephsqlite-2:19.0.0-5320.ge8f88bb1.el9.x86_64 17/172 2024-07-26T13:26:59.639 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados-devel-2:19.0.0-5320.ge8f88bb1.el9.x86_6 18/172 2024-07-26T13:26:59.639 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libradosstriper1-2:19.0.0-5320.ge8f88bb1.el9.x86 19/172 2024-07-26T13:26:59.639 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librgw2-2:19.0.0-5320.ge8f88bb1.el9.x86_64 20/172 2024-07-26T13:26:59.639 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-argparse-2:19.0.0-5320.ge8f88bb1.el 21/172 2024-07-26T13:26:59.639 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-common-2:19.0.0-5320.ge8f88bb1.el9. 22/172 2024-07-26T13:26:59.639 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cephfs-2:19.0.0-5320.ge8f88bb1.el9.x86_6 23/172 2024-07-26T13:26:59.639 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rados-2:19.0.0-5320.ge8f88bb1.el9.x86_64 24/172 2024-07-26T13:26:59.639 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rbd-2:19.0.0-5320.ge8f88bb1.el9.x86_64 25/172 2024-07-26T13:26:59.640 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rgw-2:19.0.0-5320.ge8f88bb1.el9.x86_64 26/172 2024-07-26T13:26:59.640 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-fuse-2:19.0.0-5320.ge8f88bb1.el9.x86_64 27/172 2024-07-26T13:26:59.640 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-mirror-2:19.0.0-5320.ge8f88bb1.el9.x86_64 28/172 2024-07-26T13:26:59.640 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-nbd-2:19.0.0-5320.ge8f88bb1.el9.x86_64 29/172 2024-07-26T13:26:59.640 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-5320.ge8f88bb1. 30/172 2024-07-26T13:26:59.652 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-5320.ge8f88bb1.el9.noa 31/172 2024-07-26T13:26:59.653 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-5320.ge8f88bb1.el9.n 32/172 2024-07-26T13:26:59.653 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-5320.ge8f 33/172 2024-07-26T13:26:59.653 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-5320.ge8f88bb1.el 34/172 2024-07-26T13:26:59.653 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-rook-2:19.0.0-5320.ge8f88bb1.el9.noarch 35/172 2024-07-26T13:26:59.653 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-5320.ge8f88bb1.e 36/172 2024-07-26T13:26:59.653 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-volume-2:19.0.0-5320.ge8f88bb1.el9.noarch 37/172 2024-07-26T13:26:59.653 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephadm-2:19.0.0-5320.ge8f88bb1.el9.noarch 38/172 2024-07-26T13:26:59.653 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-top-2:19.0.0-5320.ge8f88bb1.el9.noarch 39/172 2024-07-26T13:26:59.653 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/172 2024-07-26T13:26:59.653 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/172 2024-07-26T13:26:59.653 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/172 2024-07-26T13:26:59.653 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/172 2024-07-26T13:26:59.654 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/172 2024-07-26T13:26:59.654 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/172 2024-07-26T13:26:59.654 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/172 2024-07-26T13:26:59.654 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/172 2024-07-26T13:26:59.654 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/172 2024-07-26T13:26:59.654 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/172 2024-07-26T13:26:59.654 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/172 2024-07-26T13:26:59.654 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/172 2024-07-26T13:26:59.654 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/172 2024-07-26T13:26:59.654 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/172 2024-07-26T13:26:59.654 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/172 2024-07-26T13:26:59.655 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/172 2024-07-26T13:26:59.655 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/172 2024-07-26T13:26:59.655 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/172 2024-07-26T13:26:59.655 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/172 2024-07-26T13:26:59.655 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/172 2024-07-26T13:26:59.655 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/172 2024-07-26T13:26:59.655 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/172 2024-07-26T13:26:59.655 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/172 2024-07-26T13:26:59.655 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/172 2024-07-26T13:26:59.655 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/172 2024-07-26T13:26:59.655 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/172 2024-07-26T13:26:59.655 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/172 2024-07-26T13:26:59.655 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/172 2024-07-26T13:26:59.656 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/172 2024-07-26T13:26:59.656 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/172 2024-07-26T13:26:59.656 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/172 2024-07-26T13:26:59.656 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/172 2024-07-26T13:26:59.656 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/172 2024-07-26T13:26:59.656 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/172 2024-07-26T13:26:59.656 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/172 2024-07-26T13:26:59.656 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/172 2024-07-26T13:26:59.656 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/172 2024-07-26T13:26:59.656 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/172 2024-07-26T13:26:59.656 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/172 2024-07-26T13:26:59.656 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/172 2024-07-26T13:26:59.656 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/172 2024-07-26T13:26:59.656 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/172 2024-07-26T13:26:59.657 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/172 2024-07-26T13:26:59.657 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/172 2024-07-26T13:26:59.657 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/172 2024-07-26T13:26:59.657 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-26T13:26:59.657 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/172 2024-07-26T13:26:59.657 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/172 2024-07-26T13:26:59.657 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/172 2024-07-26T13:26:59.657 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/172 2024-07-26T13:26:59.657 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/172 2024-07-26T13:26:59.657 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/172 2024-07-26T13:26:59.657 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/172 2024-07-26T13:26:59.657 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/172 2024-07-26T13:26:59.657 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/172 2024-07-26T13:26:59.658 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/172 2024-07-26T13:26:59.658 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/172 2024-07-26T13:26:59.658 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/172 2024-07-26T13:26:59.658 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/172 2024-07-26T13:26:59.658 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/172 2024-07-26T13:26:59.658 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/172 2024-07-26T13:26:59.658 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xfsprogs-devel-6.4.0-3.el9.x86_64 101/172 2024-07-26T13:26:59.658 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/172 2024-07-26T13:26:59.658 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/172 2024-07-26T13:26:59.658 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/172 2024-07-26T13:26:59.658 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/172 2024-07-26T13:26:59.658 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/172 2024-07-26T13:26:59.659 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/172 2024-07-26T13:26:59.659 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/172 2024-07-26T13:26:59.659 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/172 2024-07-26T13:26:59.659 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/172 2024-07-26T13:26:59.659 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/172 2024-07-26T13:26:59.659 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/172 2024-07-26T13:26:59.659 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/172 2024-07-26T13:26:59.659 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/172 2024-07-26T13:26:59.659 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/172 2024-07-26T13:26:59.659 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/172 2024-07-26T13:26:59.659 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/172 2024-07-26T13:26:59.659 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/172 2024-07-26T13:26:59.659 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/172 2024-07-26T13:26:59.660 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/172 2024-07-26T13:26:59.660 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/172 2024-07-26T13:26:59.660 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-google-auth-1:2.32.0-1.el9.noarch 122/172 2024-07-26T13:26:59.660 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/172 2024-07-26T13:26:59.660 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/172 2024-07-26T13:26:59.660 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/172 2024-07-26T13:26:59.660 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/172 2024-07-26T13:26:59.660 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/172 2024-07-26T13:26:59.660 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/172 2024-07-26T13:26:59.660 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/172 2024-07-26T13:26:59.660 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/172 2024-07-26T13:26:59.660 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/172 2024-07-26T13:26:59.660 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/172 2024-07-26T13:26:59.661 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/172 2024-07-26T13:26:59.661 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/172 2024-07-26T13:26:59.661 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/172 2024-07-26T13:26:59.661 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/172 2024-07-26T13:26:59.661 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/172 2024-07-26T13:26:59.661 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/172 2024-07-26T13:26:59.661 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/172 2024-07-26T13:26:59.661 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/172 2024-07-26T13:26:59.661 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/172 2024-07-26T13:26:59.661 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/172 2024-07-26T13:26:59.661 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/172 2024-07-26T13:26:59.661 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/172 2024-07-26T13:26:59.661 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/172 2024-07-26T13:26:59.662 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/172 2024-07-26T13:26:59.662 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/172 2024-07-26T13:26:59.662 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/172 2024-07-26T13:26:59.662 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-2:19.0.0-5320.ge8f88bb1.el9.x86_64 149/172 2024-07-26T13:26:59.662 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/172 2024-07-26T13:26:59.662 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-2:19.0.0-5320.ge8f88bb1.el9.x86_64 151/172 2024-07-26T13:26:59.662 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/172 2024-07-26T13:26:59.662 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-1:1.48.2-2.el9.x86_64 153/172 2024-07-26T13:26:59.662 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/172 2024-07-26T13:26:59.662 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 155/172 2024-07-26T13:26:59.662 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/172 2024-07-26T13:26:59.662 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-team-1:1.48.2-2.el9.x86_64 157/172 2024-07-26T13:26:59.662 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/172 2024-07-26T13:26:59.662 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-tui-1:1.48.2-2.el9.x86_64 159/172 2024-07-26T13:26:59.663 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/172 2024-07-26T13:26:59.663 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/172 2024-07-26T13:26:59.663 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-26T13:26:59.663 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/172 2024-07-26T13:26:59.663 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/172 2024-07-26T13:26:59.663 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/172 2024-07-26T13:26:59.663 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/172 2024-07-26T13:26:59.663 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-1:3.2.2-2.el9.x86_64 167/172 2024-07-26T13:26:59.663 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/172 2024-07-26T13:26:59.663 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-libs-1:3.2.2-2.el9.x86_64 169/172 2024-07-26T13:26:59.663 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-26T13:26:59.663 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xfsprogs-6.4.0-3.el9.x86_64 171/172 2024-07-26T13:27:02.454 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-26T13:27:02.454 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-07-26T13:27:02.455 INFO:teuthology.orchestra.run.smithi167.stdout:Upgraded: 2024-07-26T13:27:02.455 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-1:1.48.2-2.el9.x86_64 2024-07-26T13:27:02.455 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 2024-07-26T13:27:02.455 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-team-1:1.48.2-2.el9.x86_64 2024-07-26T13:27:02.455 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-tui-1:1.48.2-2.el9.x86_64 2024-07-26T13:27:02.455 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-07-26T13:27:02.455 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-07-26T13:27:02.455 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-07-26T13:27:02.455 INFO:teuthology.orchestra.run.smithi167.stdout: librados2-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:27:02.455 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:27:02.455 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-1:3.2.2-2.el9.x86_64 2024-07-26T13:27:02.455 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-libs-1:3.2.2-2.el9.x86_64 2024-07-26T13:27:02.456 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs-6.4.0-3.el9.x86_64 2024-07-26T13:27:02.456 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2024-07-26T13:27:02.456 INFO:teuthology.orchestra.run.smithi167.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-07-26T13:27:02.456 INFO:teuthology.orchestra.run.smithi167.stdout: automake-1.16.2-8.el9.noarch 2024-07-26T13:27:02.456 INFO:teuthology.orchestra.run.smithi167.stdout: bison-3.7.4-5.el9.x86_64 2024-07-26T13:27:02.456 INFO:teuthology.orchestra.run.smithi167.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-07-26T13:27:02.456 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:27:02.456 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:27:02.456 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:27:02.456 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:27:02.456 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards-2:19.0.0-5320.ge8f88bb1.el9.noarch 2024-07-26T13:27:02.456 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:27:02.456 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:27:02.457 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:27:02.457 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm-2:19.0.0-5320.ge8f88bb1.el9.noarch 2024-07-26T13:27:02.457 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard-2:19.0.0-5320.ge8f88bb1.el9.noarch 2024-07-26T13:27:02.457 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local-2:19.0.0-5320.ge8f88bb1.el9.noarch 2024-07-26T13:27:02.457 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core-2:19.0.0-5320.ge8f88bb1.el9.noarch 2024-07-26T13:27:02.457 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook-2:19.0.0-5320.ge8f88bb1.el9.noarch 2024-07-26T13:27:02.457 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:27:02.457 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:27:02.457 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts-2:19.0.0-5320.ge8f88bb1.el9.noarch 2024-07-26T13:27:02.457 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:27:02.457 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:27:02.457 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:27:02.458 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume-2:19.0.0-5320.ge8f88bb1.el9.noarch 2024-07-26T13:27:02.458 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm-2:19.0.0-5320.ge8f88bb1.el9.noarch 2024-07-26T13:27:02.458 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-mirror-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:27:02.458 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-shell-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:27:02.458 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-top-2:19.0.0-5320.ge8f88bb1.el9.noarch 2024-07-26T13:27:02.458 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-07-26T13:27:02.458 INFO:teuthology.orchestra.run.smithi167.stdout: flex-2.6.4-9.el9.x86_64 2024-07-26T13:27:02.458 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-07-26T13:27:02.458 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-07-26T13:27:02.458 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-07-26T13:27:02.458 INFO:teuthology.orchestra.run.smithi167.stdout: fuse-2.9.9-16.el9.x86_64 2024-07-26T13:27:02.458 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-07-26T13:27:02.458 INFO:teuthology.orchestra.run.smithi167.stdout: grpc-data-1.46.7-10.el9.noarch 2024-07-26T13:27:02.459 INFO:teuthology.orchestra.run.smithi167.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-07-26T13:27:02.459 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-9.0.0-11.el9.x86_64 2024-07-26T13:27:02.459 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-07-26T13:27:02.459 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:27:02.459 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:27:02.459 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:27:02.459 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-07-26T13:27:02.459 INFO:teuthology.orchestra.run.smithi167.stdout: liboath-2.6.7-2.el9.x86_64 2024-07-26T13:27:02.459 INFO:teuthology.orchestra.run.smithi167.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-07-26T13:27:02.459 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-07-26T13:27:02.459 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-07-26T13:27:02.459 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:27:02.459 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:27:02.460 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-07-26T13:27:02.460 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:27:02.460 INFO:teuthology.orchestra.run.smithi167.stdout: libtool-2.4.6-46.el9.x86_64 2024-07-26T13:27:02.460 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind-1.6.2-1.el9.x86_64 2024-07-26T13:27:02.460 INFO:teuthology.orchestra.run.smithi167.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-07-26T13:27:02.460 INFO:teuthology.orchestra.run.smithi167.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-07-26T13:27:02.460 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-07-26T13:27:02.460 INFO:teuthology.orchestra.run.smithi167.stdout: lua-5.4.4-4.el9.x86_64 2024-07-26T13:27:02.460 INFO:teuthology.orchestra.run.smithi167.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-07-26T13:27:02.460 INFO:teuthology.orchestra.run.smithi167.stdout: luarocks-3.9.2-5.el9.noarch 2024-07-26T13:27:02.460 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-0.3.21-2.el9.x86_64 2024-07-26T13:27:02.460 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-07-26T13:27:02.460 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-07-26T13:27:02.461 INFO:teuthology.orchestra.run.smithi167.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-07-26T13:27:02.461 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-13.14-1.el9.x86_64 2024-07-26T13:27:02.461 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-07-26T13:27:02.461 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-07-26T13:27:02.461 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-server-13.14-1.el9.x86_64 2024-07-26T13:27:02.461 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-3.14.0-13.el9.x86_64 2024-07-26T13:27:02.461 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-07-26T13:27:02.461 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-07-26T13:27:02.461 INFO:teuthology.orchestra.run.smithi167.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-07-26T13:27:02.461 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel-2.9.1-2.el9.noarch 2024-07-26T13:27:02.462 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-07-26T13:27:02.462 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-07-26T13:27:02.462 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:27:02.462 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:27:02.462 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:27:02.462 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-07-26T13:27:02.462 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-07-26T13:27:02.462 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-07-26T13:27:02.462 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-07-26T13:27:02.462 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-07-26T13:27:02.462 INFO:teuthology.orchestra.run.smithi167.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-07-26T13:27:02.462 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-07-26T13:27:02.463 INFO:teuthology.orchestra.run.smithi167.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-07-26T13:27:02.463 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth-1:2.32.0-1.el9.noarch 2024-07-26T13:27:02.463 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-07-26T13:27:02.463 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-07-26T13:27:02.463 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-07-26T13:27:02.463 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-07-26T13:27:02.463 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-07-26T13:27:02.463 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-07-26T13:27:02.463 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-07-26T13:27:02.463 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-07-26T13:27:02.463 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-07-26T13:27:02.463 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-07-26T13:27:02.463 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-07-26T13:27:02.464 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-07-26T13:27:02.464 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako-1.1.4-6.el9.noarch 2024-07-26T13:27:02.464 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-07-26T13:27:02.464 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-07-26T13:27:02.464 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-07-26T13:27:02.464 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-07-26T13:27:02.464 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-07-26T13:27:02.464 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-07-26T13:27:02.464 INFO:teuthology.orchestra.run.smithi167.stdout: python3-packaging-20.9-5.el9.noarch 2024-07-26T13:27:02.464 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-07-26T13:27:02.464 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ply-3.11-14.el9.noarch 2024-07-26T13:27:02.464 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend-3.1.0-2.el9.noarch 2024-07-26T13:27:02.464 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-07-26T13:27:02.465 INFO:teuthology.orchestra.run.smithi167.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-07-26T13:27:02.465 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-07-26T13:27:02.465 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-07-26T13:27:02.465 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-07-26T13:27:02.465 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pycparser-2.20-6.el9.noarch 2024-07-26T13:27:02.465 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-07-26T13:27:02.465 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz-2021.1-5.el9.noarch 2024-07-26T13:27:02.465 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:27:02.465 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:27:02.465 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-07-26T13:27:02.465 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-2.25.1-8.el9.noarch 2024-07-26T13:27:02.465 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-07-26T13:27:02.465 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:27:02.466 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes-2.5.1-5.el9.noarch 2024-07-26T13:27:02.466 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa-4.9-2.el9.noarch 2024-07-26T13:27:02.466 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-07-26T13:27:02.466 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-07-26T13:27:02.466 INFO:teuthology.orchestra.run.smithi167.stdout: python3-toml-0.10.2-6.el9.noarch 2024-07-26T13:27:02.466 INFO:teuthology.orchestra.run.smithi167.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-07-26T13:27:02.466 INFO:teuthology.orchestra.run.smithi167.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-07-26T13:27:02.466 INFO:teuthology.orchestra.run.smithi167.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-07-26T13:27:02.466 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob-1.8.7-6.el9.noarch 2024-07-26T13:27:02.466 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-07-26T13:27:02.466 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-07-26T13:27:02.466 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-07-26T13:27:02.466 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-24.02.0-1.el9.x86_64 2024-07-26T13:27:02.466 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-07-26T13:27:02.467 INFO:teuthology.orchestra.run.smithi167.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-07-26T13:27:02.467 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:27:02.467 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:27:02.467 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd-2:19.0.0-5320.ge8f88bb1.el9.x86_64 2024-07-26T13:27:02.467 INFO:teuthology.orchestra.run.smithi167.stdout: re2-1:20211101-3.el9.x86_64 2024-07-26T13:27:02.467 INFO:teuthology.orchestra.run.smithi167.stdout: socat-1.7.4.1-6.el9.x86_64 2024-07-26T13:27:02.467 INFO:teuthology.orchestra.run.smithi167.stdout: thrift-0.15.0-2.el9.x86_64 2024-07-26T13:27:02.467 INFO:teuthology.orchestra.run.smithi167.stdout: uuid-1.6.2-55.el9.x86_64 2024-07-26T13:27:02.467 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs-devel-6.4.0-3.el9.x86_64 2024-07-26T13:27:02.467 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-07-26T13:27:02.468 INFO:teuthology.orchestra.run.smithi167.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-07-26T13:27:02.468 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-07-26T13:27:02.468 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-07-26T13:27:02.875 DEBUG:teuthology.parallel:result is None 2024-07-26T13:27:02.875 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 2024-07-26T13:27:03.023 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-07-26T13:27:03.050 INFO:teuthology.orchestra.run.smithi100.stdout:19.0.0-5320.ge8f88bb1.el9 2024-07-26T13:27:03.050 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-5320.ge8f88bb1.el9 2024-07-26T13:27:03.051 INFO:teuthology.task.install:The correct ceph version 19.0.0-5320.ge8f88bb1 is installed. 2024-07-26T13:27:03.053 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 2024-07-26T13:27:03.198 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-07-26T13:27:03.225 INFO:teuthology.orchestra.run.smithi138.stdout:19.0.0-5320.ge8f88bb1.el9 2024-07-26T13:27:03.225 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-5320.ge8f88bb1.el9 2024-07-26T13:27:03.225 INFO:teuthology.task.install:The correct ceph version 19.0.0-5320.ge8f88bb1 is installed. 2024-07-26T13:27:03.227 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 2024-07-26T13:27:03.373 DEBUG:teuthology.orchestra.run.smithi167:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-07-26T13:27:03.396 INFO:teuthology.orchestra.run.smithi167.stdout:19.0.0-5320.ge8f88bb1.el9 2024-07-26T13:27:03.396 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-5320.ge8f88bb1.el9 2024-07-26T13:27:03.396 INFO:teuthology.task.install:The correct ceph version 19.0.0-5320.ge8f88bb1 is installed. 2024-07-26T13:27:03.398 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-07-26T13:27:03.398 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-07-26T13:27:03.398 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-07-26T13:27:03.430 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-07-26T13:27:03.431 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-07-26T13:27:03.462 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-07-26T13:27:03.462 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-07-26T13:27:03.489 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-07-26T13:27:03.489 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-07-26T13:27:03.489 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/daemon-helper 2024-07-26T13:27:03.518 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-07-26T13:27:03.585 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-07-26T13:27:03.586 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/daemon-helper 2024-07-26T13:27:03.613 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-07-26T13:27:03.679 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-07-26T13:27:03.679 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/daemon-helper 2024-07-26T13:27:03.704 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-07-26T13:27:03.768 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-07-26T13:27:03.769 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-07-26T13:27:03.769 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/adjust-ulimits 2024-07-26T13:27:03.796 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-07-26T13:27:03.863 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-07-26T13:27:03.863 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/adjust-ulimits 2024-07-26T13:27:03.890 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-07-26T13:27:03.956 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-07-26T13:27:03.957 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/adjust-ulimits 2024-07-26T13:27:03.981 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-07-26T13:27:04.046 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-07-26T13:27:04.046 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-07-26T13:27:04.046 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/stdin-killer 2024-07-26T13:27:04.072 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-07-26T13:27:04.138 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-07-26T13:27:04.138 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/stdin-killer 2024-07-26T13:27:04.166 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-07-26T13:27:04.233 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-07-26T13:27:04.233 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/stdin-killer 2024-07-26T13:27:04.259 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-07-26T13:27:04.324 INFO:teuthology.run_tasks:Running task cephadm... 2024-07-26T13:27:04.426 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug cephsqlite': 20, 'mgr/snap_schedule/allow_m_granularity': True, 'mgr/snap_schedule/dump_on_update': True, 'mgr/snap_schedule/log_level': 'debug'}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'mgr-modules': ['snap_schedule'], 'sha1': 'e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528', 'subvols': {'create': 2}} 2024-07-26T13:27:04.426 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 2024-07-26T13:27:04.426 INFO:tasks.cephadm:Cluster fsid is bf83e2e6-4b52-11ef-bcb7-c7b262605968 2024-07-26T13:27:04.427 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-07-26T13:27:04.427 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.100', 'mon.b': '172.21.15.138', 'mon.c': '172.21.15.167'} 2024-07-26T13:27:04.427 INFO:tasks.cephadm:First mon is mon.a on smithi100 2024-07-26T13:27:04.427 INFO:tasks.cephadm:First mgr is x 2024-07-26T13:27:04.427 INFO:tasks.cephadm:Normalizing hostnames... 2024-07-26T13:27:04.427 DEBUG:teuthology.orchestra.run.smithi100:> sudo hostname $(hostname -s) 2024-07-26T13:27:04.453 DEBUG:teuthology.orchestra.run.smithi138:> sudo hostname $(hostname -s) 2024-07-26T13:27:04.481 DEBUG:teuthology.orchestra.run.smithi167:> sudo hostname $(hostname -s) 2024-07-26T13:27:04.506 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-07-26T13:27:04.507 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 2024-07-26T13:27:04.649 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/81524/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-5320-ge8f88bb1', 'node_name': '172.21.4.69+confusa07', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-5320.ge8f88bb1'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240725.121331-debug/e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-07-25 16:18:49.953482', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20240725.121331-debug', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20240725.121331-debug/e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528/centos/9/flavors/default/', 'archs': ['arm64', 'x86_64', 'source'], 'distro': 'centos'}, {'status': 'ready', 'sha1': 'e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/81524/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-5320-ge8f88bb1', 'node_name': '172.21.2.8+braggi08', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-5320.ge8f88bb1'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240725.121331-debug/e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-07-25 15:10:42.298781', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20240725.121331-debug', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20240725.121331-debug/e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-07-26T13:27:04.794 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-vshankar-testing-20240725.121331-debug, sha1 e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 2024-07-26T13:27:04.796 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240725.121331-debug/e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528/centos/9/x86_64/flavors/default/cephadm 2024-07-26T13:27:04.796 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240725.121331-debug/e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528/centos/9/x86_64/flavors/default/cephadm 2024-07-26T13:27:04.796 DEBUG:teuthology.orchestra.run.smithi100:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240725.121331-debug/e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-07-26T13:27:05.168 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r--. 1 ubuntu ubuntu 789811 Jul 26 13:27 /home/ubuntu/cephtest/cephadm 2024-07-26T13:27:05.168 DEBUG:teuthology.orchestra.run.smithi138:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240725.121331-debug/e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-07-26T13:27:05.553 INFO:teuthology.orchestra.run.smithi138.stdout:-rw-r--r--. 1 ubuntu ubuntu 789811 Jul 26 13:27 /home/ubuntu/cephtest/cephadm 2024-07-26T13:27:05.554 DEBUG:teuthology.orchestra.run.smithi167:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240725.121331-debug/e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-07-26T13:27:05.925 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-r--r--. 1 ubuntu ubuntu 789811 Jul 26 13:27 /home/ubuntu/cephtest/cephadm 2024-07-26T13:27:05.925 DEBUG:teuthology.orchestra.run.smithi100:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-07-26T13:27:05.945 DEBUG:teuthology.orchestra.run.smithi138:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-07-26T13:27:05.964 DEBUG:teuthology.orchestra.run.smithi167:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-07-26T13:27:05.997 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 on all hosts... 2024-07-26T13:27:05.998 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 pull 2024-07-26T13:27:06.000 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 pull 2024-07-26T13:27:06.006 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 pull 2024-07-26T13:27:06.896 INFO:teuthology.orchestra.run.smithi167.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528... 2024-07-26T13:27:06.897 INFO:teuthology.orchestra.run.smithi100.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528... 2024-07-26T13:27:06.903 INFO:teuthology.orchestra.run.smithi138.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528... 2024-07-26T13:27:43.001 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-07-26T13:27:43.001 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_version": "ceph version 19.0.0-5320-ge8f88bb1 (e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528) squid (dev)", 2024-07-26T13:27:43.001 INFO:teuthology.orchestra.run.smithi167.stdout: "image_id": "ae51f432d4fa82622bc253ca9e2da7666c617e096e99d43d0a0e4fdaca8865f8", 2024-07-26T13:27:43.001 INFO:teuthology.orchestra.run.smithi167.stdout: "repo_digests": [ 2024-07-26T13:27:43.001 INFO:teuthology.orchestra.run.smithi167.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2" 2024-07-26T13:27:43.002 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-07-26T13:27:43.002 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-07-26T13:28:01.657 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-07-26T13:28:01.657 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_version": "ceph version 19.0.0-5320-ge8f88bb1 (e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528) squid (dev)", 2024-07-26T13:28:01.657 INFO:teuthology.orchestra.run.smithi138.stdout: "image_id": "ae51f432d4fa82622bc253ca9e2da7666c617e096e99d43d0a0e4fdaca8865f8", 2024-07-26T13:28:01.657 INFO:teuthology.orchestra.run.smithi138.stdout: "repo_digests": [ 2024-07-26T13:28:01.657 INFO:teuthology.orchestra.run.smithi138.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2" 2024-07-26T13:28:01.658 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-07-26T13:28:01.658 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-07-26T13:28:01.959 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:28:01.959 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_version": "ceph version 19.0.0-5320-ge8f88bb1 (e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528) squid (dev)", 2024-07-26T13:28:01.959 INFO:teuthology.orchestra.run.smithi100.stdout: "image_id": "ae51f432d4fa82622bc253ca9e2da7666c617e096e99d43d0a0e4fdaca8865f8", 2024-07-26T13:28:01.959 INFO:teuthology.orchestra.run.smithi100.stdout: "repo_digests": [ 2024-07-26T13:28:01.959 INFO:teuthology.orchestra.run.smithi100.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2" 2024-07-26T13:28:01.959 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-07-26T13:28:01.959 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:28:01.987 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /etc/ceph 2024-07-26T13:28:02.050 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /etc/ceph 2024-07-26T13:28:02.079 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /etc/ceph 2024-07-26T13:28:02.114 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 777 /etc/ceph 2024-07-26T13:28:02.139 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 777 /etc/ceph 2024-07-26T13:28:02.164 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 777 /etc/ceph 2024-07-26T13:28:02.192 INFO:tasks.cephadm:Writing seed config... 2024-07-26T13:28:02.193 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-07-26T13:28:02.194 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-07-26T13:28:02.194 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-07-26T13:28:02.194 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-07-26T13:28:02.194 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-07-26T13:28:02.194 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-07-26T13:28:02.194 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-07-26T13:28:02.194 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-07-26T13:28:02.194 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-07-26T13:28:02.194 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-07-26T13:28:02.194 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-07-26T13:28:02.194 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-07-26T13:28:02.194 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-07-26T13:28:02.194 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-07-26T13:28:02.194 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-07-26T13:28:02.195 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-07-26T13:28:02.195 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-07-26T13:28:02.195 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-07-26T13:28:02.195 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-07-26T13:28:02.195 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-07-26T13:28:02.197 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-07-26T13:28:02.197 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-07-26T13:28:02.198 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-07-26T13:28:02.198 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-07-26T13:28:02.198 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-07-26T13:28:02.198 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-07-26T13:28:02.214 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = bf83e2e6-4b52-11ef-bcb7-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-07-26T13:28:02.214 DEBUG:teuthology.orchestra.run.smithi100:mon.a> sudo journalctl -f -n 0 -u ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mon.a.service 2024-07-26T13:28:02.257 DEBUG:teuthology.orchestra.run.smithi100:mgr.x> sudo journalctl -f -n 0 -u ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mgr.x.service 2024-07-26T13:28:02.299 INFO:tasks.cephadm:Bootstrapping... 2024-07-26T13:28:02.299 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 -v bootstrap --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.100 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-07-26T13:28:02.502 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-07-26T13:28:02.502 INFO:teuthology.orchestra.run.smithi100.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528', '-v', 'bootstrap', '--fsid', 'bf83e2e6-4b52-11ef-bcb7-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.100', '--skip-admin-label'] 2024-07-26T13:28:02.503 INFO:teuthology.orchestra.run.smithi100.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-07-26T13:28:02.503 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying podman|docker is present... 2024-07-26T13:28:02.528 INFO:teuthology.orchestra.run.smithi100.stdout:/bin/podman: stdout 5.1.2 2024-07-26T13:28:02.528 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying lvm2 is present... 2024-07-26T13:28:02.528 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying time synchronization is in place... 2024-07-26T13:28:02.536 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-07-26T13:28:02.536 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-07-26T13:28:02.542 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-07-26T13:28:02.542 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-07-26T13:28:02.549 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout enabled 2024-07-26T13:28:02.555 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout active 2024-07-26T13:28:02.555 INFO:teuthology.orchestra.run.smithi100.stdout:Unit chronyd.service is enabled and running 2024-07-26T13:28:02.556 INFO:teuthology.orchestra.run.smithi100.stdout:Repeating the final host check... 2024-07-26T13:28:02.579 INFO:teuthology.orchestra.run.smithi100.stdout:/bin/podman: stdout 5.1.2 2024-07-26T13:28:02.579 INFO:teuthology.orchestra.run.smithi100.stdout:podman (/bin/podman) version 5.1.2 is present 2024-07-26T13:28:02.579 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl is present 2024-07-26T13:28:02.579 INFO:teuthology.orchestra.run.smithi100.stdout:lvcreate is present 2024-07-26T13:28:02.585 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-07-26T13:28:02.586 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-07-26T13:28:02.592 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-07-26T13:28:02.592 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-07-26T13:28:02.598 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout enabled 2024-07-26T13:28:02.607 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout active 2024-07-26T13:28:02.607 INFO:teuthology.orchestra.run.smithi100.stdout:Unit chronyd.service is enabled and running 2024-07-26T13:28:02.607 INFO:teuthology.orchestra.run.smithi100.stdout:Host looks OK 2024-07-26T13:28:02.607 INFO:teuthology.orchestra.run.smithi100.stdout:Cluster fsid: bf83e2e6-4b52-11ef-bcb7-c7b262605968 2024-07-26T13:28:02.608 INFO:teuthology.orchestra.run.smithi100.stdout:Acquiring lock 139917073204032 on /run/cephadm/bf83e2e6-4b52-11ef-bcb7-c7b262605968.lock 2024-07-26T13:28:02.608 INFO:teuthology.orchestra.run.smithi100.stdout:Lock 139917073204032 acquired on /run/cephadm/bf83e2e6-4b52-11ef-bcb7-c7b262605968.lock 2024-07-26T13:28:02.608 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying IP 172.21.15.100 port 3300 ... 2024-07-26T13:28:02.608 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying IP 172.21.15.100 port 6789 ... 2024-07-26T13:28:02.608 INFO:teuthology.orchestra.run.smithi100.stdout:Base mon IP(s) is [172.21.15.100:3300, 172.21.15.100:6789], mon addrv is [v2:172.21.15.100:3300,v1:172.21.15.100:6789] 2024-07-26T13:28:02.609 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.100 metric 100 2024-07-26T13:28:02.609 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.100 metric 100 2024-07-26T13:28:02.611 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-07-26T13:28:02.611 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-07-26T13:28:02.611 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1792sec hoplimit 64 pref medium 2024-07-26T13:28:02.614 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-07-26T13:28:02.614 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-07-26T13:28:02.614 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-07-26T13:28:02.614 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-07-26T13:28:02.614 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d733/64 scope link 2024-07-26T13:28:02.614 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-07-26T13:28:02.615 INFO:teuthology.orchestra.run.smithi100.stdout:Mon IP `172.21.15.100` is in CIDR network `172.21.0.0/20` 2024-07-26T13:28:02.615 INFO:teuthology.orchestra.run.smithi100.stdout:Mon IP `172.21.15.100` is in CIDR network `172.21.0.0/20` 2024-07-26T13:28:02.615 INFO:teuthology.orchestra.run.smithi100.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-07-26T13:28:02.615 INFO:teuthology.orchestra.run.smithi100.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-07-26T13:28:02.616 INFO:teuthology.orchestra.run.smithi100.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528... 2024-07-26T13:28:04.692 INFO:teuthology.orchestra.run.smithi100.stdout:/bin/podman: stdout ae51f432d4fa82622bc253ca9e2da7666c617e096e99d43d0a0e4fdaca8865f8 2024-07-26T13:28:04.692 INFO:teuthology.orchestra.run.smithi100.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528... 2024-07-26T13:28:04.692 INFO:teuthology.orchestra.run.smithi100.stdout:/bin/podman: stderr Getting image source signatures 2024-07-26T13:28:04.692 INFO:teuthology.orchestra.run.smithi100.stdout:/bin/podman: stderr Copying blob sha256:27b44fafaf0d8ddf6712bf077b0e069fd1632e81c2016bcdf9194ad64e762303 2024-07-26T13:28:04.692 INFO:teuthology.orchestra.run.smithi100.stdout:/bin/podman: stderr Copying blob sha256:43735fbce9214f061a6894341d7822563f5aafe16fc7532189f4415cff6d2d91 2024-07-26T13:28:04.692 INFO:teuthology.orchestra.run.smithi100.stdout:/bin/podman: stderr Copying config sha256:ae51f432d4fa82622bc253ca9e2da7666c617e096e99d43d0a0e4fdaca8865f8 2024-07-26T13:28:04.693 INFO:teuthology.orchestra.run.smithi100.stdout:/bin/podman: stderr Writing manifest to image destination 2024-07-26T13:28:13.904 INFO:teuthology.orchestra.run.smithi100.stdout:ceph: stdout ceph version 19.0.0-5320-ge8f88bb1 (e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528) squid (dev) 2024-07-26T13:28:13.905 INFO:teuthology.orchestra.run.smithi100.stdout:Ceph version: ceph version 19.0.0-5320-ge8f88bb1 (e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528) squid (dev) 2024-07-26T13:28:13.905 INFO:teuthology.orchestra.run.smithi100.stdout:Extracting ceph user uid/gid from container image... 2024-07-26T13:28:16.665 INFO:teuthology.orchestra.run.smithi100.stdout:stat: stdout 167 167 2024-07-26T13:28:16.665 INFO:teuthology.orchestra.run.smithi100.stdout:Creating initial keys... 2024-07-26T13:28:17.578 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-authtool: stdout AQBwpKNmizCaOhAASsengPkXtsNST/YGjBylHw== 2024-07-26T13:28:18.418 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-authtool: stdout AQBxpKNmMl2RNBAAptxb+lPBplHYJJRpBWs1DQ== 2024-07-26T13:28:19.490 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-authtool: stdout AQBypKNmg6KCLhAAbgBkSTE5N2y/LabFO33i1A== 2024-07-26T13:28:19.491 INFO:teuthology.orchestra.run.smithi100.stdout:Creating initial monmap... 2024-07-26T13:28:20.438 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-07-26T13:28:20.439 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-07-26T13:28:20.439 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to bf83e2e6-4b52-11ef-bcb7-c7b262605968 2024-07-26T13:28:20.439 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-07-26T13:28:20.439 INFO:teuthology.orchestra.run.smithi100.stdout:monmaptool for a [v2:172.21.15.100:3300,v1:172.21.15.100:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-07-26T13:28:20.439 INFO:teuthology.orchestra.run.smithi100.stdout:setting min_mon_release = quincy 2024-07-26T13:28:20.439 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/monmaptool: set fsid to bf83e2e6-4b52-11ef-bcb7-c7b262605968 2024-07-26T13:28:20.439 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-07-26T13:28:20.439 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:28:20.439 INFO:teuthology.orchestra.run.smithi100.stdout:Creating mon... 2024-07-26T13:28:22.149 INFO:teuthology.orchestra.run.smithi100.stdout:create mon.a on 2024-07-26T13:28:22.354 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-07-26T13:28:22.515 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-07-26T13:28:22.704 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968.target → /etc/systemd/system/ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968.target. 2024-07-26T13:28:22.704 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968.target → /etc/systemd/system/ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968.target. 2024-07-26T13:28:22.962 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mon.a 2024-07-26T13:28:22.962 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Failed to reset failed state of unit ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mon.a.service: Unit ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mon.a.service not loaded. 2024-07-26T13:28:23.128 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968.target.wants/ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mon.a.service → /etc/systemd/system/ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@.service. 2024-07-26T13:28:23.684 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-07-26T13:28:23.684 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout disabled 2024-07-26T13:28:23.691 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-07-26T13:28:23.691 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-07-26T13:28:23.692 INFO:teuthology.orchestra.run.smithi100.stdout:firewalld.service is not enabled 2024-07-26T13:28:23.692 INFO:teuthology.orchestra.run.smithi100.stdout:Not possible to enable service . firewalld.service is not available 2024-07-26T13:28:23.692 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for mon to start... 2024-07-26T13:28:23.692 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for mon... 2024-07-26T13:28:24.735 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:24 smithi100 ceph-mon[32234]: from='client.? 172.21.15.100:0/2465759265' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-07-26T13:28:24.738 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout cluster: 2024-07-26T13:28:24.738 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout id: bf83e2e6-4b52-11ef-bcb7-c7b262605968 2024-07-26T13:28:24.738 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-07-26T13:28:24.739 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-07-26T13:28:24.739 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout services: 2024-07-26T13:28:24.739 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.544504s) 2024-07-26T13:28:24.739 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-07-26T13:28:24.739 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-07-26T13:28:24.739 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-07-26T13:28:24.739 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout data: 2024-07-26T13:28:24.739 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-07-26T13:28:24.739 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-07-26T13:28:24.739 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-07-26T13:28:24.739 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout pgs: 2024-07-26T13:28:24.739 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-07-26T13:28:24.739 INFO:teuthology.orchestra.run.smithi100.stdout:mon is available 2024-07-26T13:28:24.740 INFO:teuthology.orchestra.run.smithi100.stdout:Assimilating anything we can from ceph.conf... 2024-07-26T13:28:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:25 smithi100 ceph-mon[32234]: from='client.? 172.21.15.100:0/3486548070' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-07-26T13:28:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:25 smithi100 ceph-mon[32234]: from='client.? 172.21.15.100:0/3486548070' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-07-26T13:28:25.751 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-07-26T13:28:25.754 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout [global] 2024-07-26T13:28:25.755 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout fsid = bf83e2e6-4b52-11ef-bcb7-c7b262605968 2024-07-26T13:28:25.755 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-07-26T13:28:25.755 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.100:3300,v1:172.21.15.100:6789] 2024-07-26T13:28:25.755 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-07-26T13:28:25.755 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-07-26T13:28:25.755 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-07-26T13:28:25.755 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-07-26T13:28:25.755 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-07-26T13:28:25.755 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout [mgr] 2024-07-26T13:28:25.755 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-07-26T13:28:25.755 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-07-26T13:28:25.756 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout [osd] 2024-07-26T13:28:25.756 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-07-26T13:28:25.756 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-07-26T13:28:25.756 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-07-26T13:28:25.756 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-07-26T13:28:25.756 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-07-26T13:28:25.756 INFO:teuthology.orchestra.run.smithi100.stdout:Generating new minimal ceph.conf... 2024-07-26T13:28:26.750 INFO:teuthology.orchestra.run.smithi100.stdout:Restarting the monitor... 2024-07-26T13:28:26.765 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:26 smithi100 ceph-mon[32234]: from='client.? 172.21.15.100:0/617855868' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:28:27.017 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:26 smithi100 systemd[1]: Stopping Ceph mon.a for bf83e2e6-4b52-11ef-bcb7-c7b262605968... 2024-07-26T13:28:27.476 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:27 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mon-a[32230]: 2024-07-26T13:28:27.016+0000 7fb579914640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-07-26T13:28:27.476 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:27 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mon-a[32230]: 2024-07-26T13:28:27.016+0000 7fb579914640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-07-26T13:28:27.476 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:27 smithi100 podman[32477]: 2024-07-26 13:28:27.078080912 +0000 UTC m=+0.167841311 container died 002c67977c4af72b320377b6224b73f82031e835fbef3e75a88ff25d9305d402 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mon-a, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux ) 2024-07-26T13:28:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:27 smithi100 podman[32477]: 2024-07-26 13:28:27.558103079 +0000 UTC m=+0.647863465 container cleanup 002c67977c4af72b320377b6224b73f82031e835fbef3e75a88ff25d9305d402 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0) 2024-07-26T13:28:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:27 smithi100 bash[32477]: ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mon-a 2024-07-26T13:28:28.110 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:27 smithi100 podman[32488]: 2024-07-26 13:28:27.758878357 +0000 UTC m=+0.680252847 container remove 002c67977c4af72b320377b6224b73f82031e835fbef3e75a88ff25d9305d402 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-07-26T13:28:28.110 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:27 smithi100 systemd[1]: ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mon.a.service: Deactivated successfully. 2024-07-26T13:28:28.110 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:27 smithi100 systemd[1]: Stopped Ceph mon.a for bf83e2e6-4b52-11ef-bcb7-c7b262605968. 2024-07-26T13:28:28.110 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:27 smithi100 systemd[1]: Starting Ceph mon.a for bf83e2e6-4b52-11ef-bcb7-c7b262605968... 2024-07-26T13:28:28.384 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 podman[32581]: 2024-07-26 13:28:28.012394743 +0000 UTC m=+0.020391649 image pull ae51f432d4fa82622bc253ca9e2da7666c617e096e99d43d0a0e4fdaca8865f8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 2024-07-26T13:28:28.384 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 podman[32581]: 2024-07-26 13:28:28.134670355 +0000 UTC m=+0.142667260 container create 304ecaf8e6425e7342311b96a955e4dbbf347888a5eda55a70d73b8d96d4d639 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-07-26T13:28:28.384 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 podman[32581]: 2024-07-26 13:28:28.348952106 +0000 UTC m=+0.356949024 container init 304ecaf8e6425e7342311b96a955e4dbbf347888a5eda55a70d73b8d96d4d639 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mon-a, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-07-26T13:28:28.384 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 podman[32581]: 2024-07-26 13:28:28.352059451 +0000 UTC m=+0.360056359 container start 304ecaf8e6425e7342311b96a955e4dbbf347888a5eda55a70d73b8d96d4d639 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0) 2024-07-26T13:28:28.425 INFO:teuthology.orchestra.run.smithi100.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-07-26T13:28:28.637 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: set uid:gid to 167:167 (ceph:ceph) 2024-07-26T13:28:28.637 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: ceph version 19.0.0-5320-ge8f88bb1 (e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528) squid (dev), process ceph-mon, pid 2 2024-07-26T13:28:28.638 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: pidfile_write: ignore empty --pid-file 2024-07-26T13:28:28.638 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: load: jerasure load: lrc 2024-07-26T13:28:28.638 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: RocksDB version: 7.9.2 2024-07-26T13:28:28.638 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Git sha 0 2024-07-26T13:28:28.638 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Compile date 2024-07-25 13:49:38 2024-07-26T13:28:28.638 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: DB SUMMARY 2024-07-26T13:28:28.638 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: DB Session ID: A4XLBV9XJ4HCRP0SX2VF 2024-07-26T13:28:28.638 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: CURRENT file: CURRENT 2024-07-26T13:28:28.638 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: IDENTITY file: IDENTITY 2024-07-26T13:28:28.638 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-07-26T13:28:28.638 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-07-26T13:28:28.639 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 94445 ; 2024-07-26T13:28:28.639 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.error_if_exists: 0 2024-07-26T13:28:28.639 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.create_if_missing: 0 2024-07-26T13:28:28.639 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.paranoid_checks: 1 2024-07-26T13:28:28.639 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.flush_verify_memtable_count: 1 2024-07-26T13:28:28.639 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-07-26T13:28:28.639 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-07-26T13:28:28.639 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.env: 0x55c4982ab280 2024-07-26T13:28:28.639 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.fs: PosixFileSystem 2024-07-26T13:28:28.639 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.info_log: 0x55c49d7a9080 2024-07-26T13:28:28.639 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.max_file_opening_threads: 16 2024-07-26T13:28:28.639 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.statistics: (nil) 2024-07-26T13:28:28.640 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.use_fsync: 0 2024-07-26T13:28:28.640 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.max_log_file_size: 0 2024-07-26T13:28:28.640 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-07-26T13:28:28.640 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.log_file_time_to_roll: 0 2024-07-26T13:28:28.640 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.keep_log_file_num: 1000 2024-07-26T13:28:28.640 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.recycle_log_file_num: 0 2024-07-26T13:28:28.640 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.allow_fallocate: 1 2024-07-26T13:28:28.640 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.allow_mmap_reads: 0 2024-07-26T13:28:28.640 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.allow_mmap_writes: 0 2024-07-26T13:28:28.640 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.use_direct_reads: 0 2024-07-26T13:28:28.640 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-07-26T13:28:28.640 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.create_missing_column_families: 0 2024-07-26T13:28:28.641 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.db_log_dir: 2024-07-26T13:28:28.641 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.wal_dir: 2024-07-26T13:28:28.641 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.table_cache_numshardbits: 6 2024-07-26T13:28:28.641 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.WAL_ttl_seconds: 0 2024-07-26T13:28:28.641 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.WAL_size_limit_MB: 0 2024-07-26T13:28:28.641 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-07-26T13:28:28.641 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-07-26T13:28:28.641 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.is_fd_close_on_exec: 1 2024-07-26T13:28:28.641 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.advise_random_on_open: 1 2024-07-26T13:28:28.641 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.db_write_buffer_size: 0 2024-07-26T13:28:28.641 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.write_buffer_manager: 0x55c49d7ad5e0 2024-07-26T13:28:28.641 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-07-26T13:28:28.641 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-07-26T13:28:28.642 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.use_adaptive_mutex: 0 2024-07-26T13:28:28.642 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.rate_limiter: (nil) 2024-07-26T13:28:28.642 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-07-26T13:28:28.642 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.wal_recovery_mode: 2 2024-07-26T13:28:28.642 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.enable_thread_tracking: 0 2024-07-26T13:28:28.642 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.enable_pipelined_write: 0 2024-07-26T13:28:28.642 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.unordered_write: 0 2024-07-26T13:28:28.642 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-07-26T13:28:28.642 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-07-26T13:28:28.642 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-07-26T13:28:28.642 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-07-26T13:28:28.642 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.row_cache: None 2024-07-26T13:28:28.643 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.wal_filter: None 2024-07-26T13:28:28.643 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-07-26T13:28:28.643 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.allow_ingest_behind: 0 2024-07-26T13:28:28.643 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.two_write_queues: 0 2024-07-26T13:28:28.643 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.manual_wal_flush: 0 2024-07-26T13:28:28.643 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.wal_compression: 0 2024-07-26T13:28:28.643 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.atomic_flush: 0 2024-07-26T13:28:28.643 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-07-26T13:28:28.643 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.persist_stats_to_disk: 0 2024-07-26T13:28:28.643 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.write_dbid_to_manifest: 0 2024-07-26T13:28:28.643 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.log_readahead_size: 0 2024-07-26T13:28:28.643 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-07-26T13:28:28.643 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.best_efforts_recovery: 0 2024-07-26T13:28:28.644 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-07-26T13:28:28.644 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-07-26T13:28:28.644 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.allow_data_in_errors: 0 2024-07-26T13:28:28.644 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.db_host_id: __hostname__ 2024-07-26T13:28:28.644 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.enforce_single_del_contracts: true 2024-07-26T13:28:28.645 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.max_background_jobs: 2 2024-07-26T13:28:28.645 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.max_background_compactions: -1 2024-07-26T13:28:28.645 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.max_subcompactions: 1 2024-07-26T13:28:28.646 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-07-26T13:28:28.646 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-07-26T13:28:28.646 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.delayed_write_rate : 16777216 2024-07-26T13:28:28.646 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.max_total_wal_size: 0 2024-07-26T13:28:28.646 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-07-26T13:28:28.646 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.stats_dump_period_sec: 600 2024-07-26T13:28:28.646 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.stats_persist_period_sec: 600 2024-07-26T13:28:28.646 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-07-26T13:28:28.646 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.max_open_files: -1 2024-07-26T13:28:28.646 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.bytes_per_sync: 0 2024-07-26T13:28:28.646 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.wal_bytes_per_sync: 0 2024-07-26T13:28:28.646 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.strict_bytes_per_sync: 0 2024-07-26T13:28:28.647 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.compaction_readahead_size: 0 2024-07-26T13:28:28.647 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.max_background_flushes: -1 2024-07-26T13:28:28.647 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Compression algorithms supported: 2024-07-26T13:28:28.647 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: kZSTD supported: 0 2024-07-26T13:28:28.647 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: kXpressCompression supported: 0 2024-07-26T13:28:28.647 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: kBZip2Compression supported: 0 2024-07-26T13:28:28.647 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-07-26T13:28:28.647 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: kLZ4Compression supported: 1 2024-07-26T13:28:28.647 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: kZlibCompression supported: 1 2024-07-26T13:28:28.647 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: kLZ4HCCompression supported: 1 2024-07-26T13:28:28.647 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: kSnappyCompression supported: 1 2024-07-26T13:28:28.647 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Fast CRC32 supported: Supported on x86 2024-07-26T13:28:28.647 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: DMutex implementation: pthread_mutex_t 2024-07-26T13:28:28.647 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-07-26T13:28:28.648 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-07-26T13:28:28.648 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-07-26T13:28:28.648 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.merge_operator: 2024-07-26T13:28:28.648 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.compaction_filter: None 2024-07-26T13:28:28.648 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.compaction_filter_factory: None 2024-07-26T13:28:28.648 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.sst_partitioner_factory: None 2024-07-26T13:28:28.648 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.memtable_factory: SkipListFactory 2024-07-26T13:28:28.648 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.table_factory: BlockBasedTable 2024-07-26T13:28:28.648 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c49d7a8c80) 2024-07-26T13:28:28.648 INFO:journalctl@ceph.mon.a.smithi100.stdout: cache_index_and_filter_blocks: 1 2024-07-26T13:28:28.648 INFO:journalctl@ceph.mon.a.smithi100.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-07-26T13:28:28.648 INFO:journalctl@ceph.mon.a.smithi100.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-07-26T13:28:28.648 INFO:journalctl@ceph.mon.a.smithi100.stdout: pin_top_level_index_and_filter: 1 2024-07-26T13:28:28.648 INFO:journalctl@ceph.mon.a.smithi100.stdout: index_type: 0 2024-07-26T13:28:28.649 INFO:journalctl@ceph.mon.a.smithi100.stdout: data_block_index_type: 0 2024-07-26T13:28:28.649 INFO:journalctl@ceph.mon.a.smithi100.stdout: index_shortening: 1 2024-07-26T13:28:28.649 INFO:journalctl@ceph.mon.a.smithi100.stdout: data_block_hash_table_util_ratio: 0.750000 2024-07-26T13:28:28.649 INFO:journalctl@ceph.mon.a.smithi100.stdout: checksum: 4 2024-07-26T13:28:28.649 INFO:journalctl@ceph.mon.a.smithi100.stdout: no_block_cache: 0 2024-07-26T13:28:28.649 INFO:journalctl@ceph.mon.a.smithi100.stdout: block_cache: 0x55c49976d610 2024-07-26T13:28:28.649 INFO:journalctl@ceph.mon.a.smithi100.stdout: block_cache_name: BinnedLRUCache 2024-07-26T13:28:28.649 INFO:journalctl@ceph.mon.a.smithi100.stdout: block_cache_options: 2024-07-26T13:28:28.649 INFO:journalctl@ceph.mon.a.smithi100.stdout: capacity : 536870912 2024-07-26T13:28:28.649 INFO:journalctl@ceph.mon.a.smithi100.stdout: num_shard_bits : 4 2024-07-26T13:28:28.649 INFO:journalctl@ceph.mon.a.smithi100.stdout: strict_capacity_limit : 0 2024-07-26T13:28:28.649 INFO:journalctl@ceph.mon.a.smithi100.stdout: high_pri_pool_ratio: 0.000 2024-07-26T13:28:28.649 INFO:journalctl@ceph.mon.a.smithi100.stdout: block_cache_compressed: (nil) 2024-07-26T13:28:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout: persistent_cache: (nil) 2024-07-26T13:28:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout: block_size: 4096 2024-07-26T13:28:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout: block_size_deviation: 10 2024-07-26T13:28:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout: block_restart_interval: 16 2024-07-26T13:28:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout: index_block_restart_interval: 1 2024-07-26T13:28:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout: metadata_block_size: 4096 2024-07-26T13:28:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout: partition_filters: 0 2024-07-26T13:28:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout: use_delta_encoding: 1 2024-07-26T13:28:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout: filter_policy: bloomfilter 2024-07-26T13:28:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout: whole_key_filtering: 1 2024-07-26T13:28:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout: verify_compression: 0 2024-07-26T13:28:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout: read_amp_bytes_per_bit: 0 2024-07-26T13:28:28.651 INFO:journalctl@ceph.mon.a.smithi100.stdout: format_version: 5 2024-07-26T13:28:28.651 INFO:journalctl@ceph.mon.a.smithi100.stdout: enable_index_compression: 1 2024-07-26T13:28:28.651 INFO:journalctl@ceph.mon.a.smithi100.stdout: block_align: 0 2024-07-26T13:28:28.651 INFO:journalctl@ceph.mon.a.smithi100.stdout: max_auto_readahead_size: 262144 2024-07-26T13:28:28.651 INFO:journalctl@ceph.mon.a.smithi100.stdout: prepopulate_block_cache: 0 2024-07-26T13:28:28.651 INFO:journalctl@ceph.mon.a.smithi100.stdout: initial_auto_readahead_size: 8192 2024-07-26T13:28:28.651 INFO:journalctl@ceph.mon.a.smithi100.stdout: num_file_reads_for_auto_readahead: 2 2024-07-26T13:28:28.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.write_buffer_size: 33554432 2024-07-26T13:28:28.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.max_write_buffer_number: 2 2024-07-26T13:28:28.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.compression: NoCompression 2024-07-26T13:28:28.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.bottommost_compression: Disabled 2024-07-26T13:28:28.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.prefix_extractor: nullptr 2024-07-26T13:28:28.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-07-26T13:28:28.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.num_levels: 7 2024-07-26T13:28:28.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-07-26T13:28:28.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-07-26T13:28:28.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-07-26T13:28:28.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-07-26T13:28:28.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-07-26T13:28:28.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-07-26T13:28:28.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-07-26T13:28:28.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-07-26T13:28:28.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-07-26T13:28:28.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-07-26T13:28:28.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-07-26T13:28:28.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-07-26T13:28:28.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.compression_opts.window_bits: -14 2024-07-26T13:28:28.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.compression_opts.level: 32767 2024-07-26T13:28:28.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.compression_opts.strategy: 0 2024-07-26T13:28:28.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-07-26T13:28:28.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-07-26T13:28:28.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-07-26T13:28:28.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-07-26T13:28:28.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.compression_opts.enabled: false 2024-07-26T13:28:28.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-07-26T13:28:28.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-07-26T13:28:28.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-07-26T13:28:28.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-07-26T13:28:28.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.target_file_size_base: 67108864 2024-07-26T13:28:28.654 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.target_file_size_multiplier: 1 2024-07-26T13:28:28.654 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-07-26T13:28:28.654 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-07-26T13:28:28.654 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-07-26T13:28:28.654 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-07-26T13:28:28.654 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-07-26T13:28:28.654 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-07-26T13:28:28.654 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-07-26T13:28:28.654 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-07-26T13:28:28.654 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-07-26T13:28:28.654 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-07-26T13:28:28.654 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-07-26T13:28:28.654 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-07-26T13:28:28.655 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-07-26T13:28:28.655 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.arena_block_size: 1048576 2024-07-26T13:28:28.655 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-07-26T13:28:28.655 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-07-26T13:28:28.655 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.disable_auto_compactions: 0 2024-07-26T13:28:28.655 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-07-26T13:28:28.655 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-07-26T13:28:28.655 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-07-26T13:28:28.655 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-07-26T13:28:28.655 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-07-26T13:28:28.655 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-07-26T13:28:28.655 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-07-26T13:28:28.655 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-07-26T13:28:28.656 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-07-26T13:28:28.656 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-07-26T13:28:28.656 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-07-26T13:28:28.656 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.inplace_update_support: 0 2024-07-26T13:28:28.656 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.inplace_update_num_locks: 10000 2024-07-26T13:28:28.656 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-07-26T13:28:28.656 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-07-26T13:28:28.656 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.memtable_huge_page_size: 0 2024-07-26T13:28:28.656 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.bloom_locality: 0 2024-07-26T13:28:28.656 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.max_successive_merges: 0 2024-07-26T13:28:28.656 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.optimize_filters_for_hits: 0 2024-07-26T13:28:28.656 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.paranoid_file_checks: 0 2024-07-26T13:28:28.656 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.force_consistency_checks: 1 2024-07-26T13:28:28.656 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.report_bg_io_stats: 0 2024-07-26T13:28:28.657 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.ttl: 2592000 2024-07-26T13:28:28.657 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.periodic_compaction_seconds: 0 2024-07-26T13:28:28.657 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-07-26T13:28:28.657 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-07-26T13:28:28.657 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.enable_blob_files: false 2024-07-26T13:28:28.657 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.min_blob_size: 0 2024-07-26T13:28:28.657 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.blob_file_size: 268435456 2024-07-26T13:28:28.657 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.blob_compression_type: NoCompression 2024-07-26T13:28:28.657 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.enable_blob_garbage_collection: false 2024-07-26T13:28:28.657 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-07-26T13:28:28.657 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-07-26T13:28:28.657 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-07-26T13:28:28.657 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.blob_file_starting_level: 0 2024-07-26T13:28:28.657 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-07-26T13:28:28.657 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-07-26T13:28:28.658 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-07-26T13:28:28.658 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9ce41d3d-4f57-4ff9-b3d1-eff25fe6b8a6 2024-07-26T13:28:28.658 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: EVENT_LOG_v1 {"time_micros": 1722000508389145, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-07-26T13:28:28.658 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-07-26T13:28:28.658 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: EVENT_LOG_v1 {"time_micros": 1722000508390233, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 90675, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 280, "table_properties": {"data_size": 88716, "index_size": 279, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11486, "raw_average_key_size": 48, "raw_value_size": 82241, "raw_average_value_size": 344, "num_data_blocks": 12, "num_entries": 239, "num_filter_entries": 239, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1722000508, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9ce41d3d-4f57-4ff9-b3d1-eff25fe6b8a6", "db_session_id": "A4XLBV9XJ4HCRP0SX2VF", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-07-26T13:28:28.658 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: EVENT_LOG_v1 {"time_micros": 1722000508390333, "job": 1, "event": "recovery_finished"} 2024-07-26T13:28:28.658 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-07-26T13:28:28.658 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-07-26T13:28:28.658 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c49d7caa80 2024-07-26T13:28:28.658 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: DB pointer 0x55c49e56a000 2024-07-26T13:28:28.658 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-07-26T13:28:28.658 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-07-26T13:28:28.659 INFO:journalctl@ceph.mon.a.smithi100.stdout: ** DB Stats ** 2024-07-26T13:28:28.659 INFO:journalctl@ceph.mon.a.smithi100.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-26T13:28:28.659 INFO:journalctl@ceph.mon.a.smithi100.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-07-26T13:28:28.659 INFO:journalctl@ceph.mon.a.smithi100.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-07-26T13:28:28.659 INFO:journalctl@ceph.mon.a.smithi100.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-07-26T13:28:28.659 INFO:journalctl@ceph.mon.a.smithi100.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-07-26T13:28:28.659 INFO:journalctl@ceph.mon.a.smithi100.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-07-26T13:28:28.659 INFO:journalctl@ceph.mon.a.smithi100.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-07-26T13:28:28.659 INFO:journalctl@ceph.mon.a.smithi100.stdout: 2024-07-26T13:28:28.659 INFO:journalctl@ceph.mon.a.smithi100.stdout: ** Compaction Stats [default] ** 2024-07-26T13:28:28.659 INFO:journalctl@ceph.mon.a.smithi100.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-07-26T13:28:28.659 INFO:journalctl@ceph.mon.a.smithi100.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-07-26T13:28:28.659 INFO:journalctl@ceph.mon.a.smithi100.stdout: L0 2/0 90.41 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 127.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-07-26T13:28:28.660 INFO:journalctl@ceph.mon.a.smithi100.stdout: Sum 2/0 90.41 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 127.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-07-26T13:28:28.660 INFO:journalctl@ceph.mon.a.smithi100.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 127.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-07-26T13:28:28.660 INFO:journalctl@ceph.mon.a.smithi100.stdout: 2024-07-26T13:28:28.660 INFO:journalctl@ceph.mon.a.smithi100.stdout: ** Compaction Stats [default] ** 2024-07-26T13:28:28.660 INFO:journalctl@ceph.mon.a.smithi100.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-07-26T13:28:28.660 INFO:journalctl@ceph.mon.a.smithi100.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-07-26T13:28:28.660 INFO:journalctl@ceph.mon.a.smithi100.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 127.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-07-26T13:28:28.660 INFO:journalctl@ceph.mon.a.smithi100.stdout: 2024-07-26T13:28:28.660 INFO:journalctl@ceph.mon.a.smithi100.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-07-26T13:28:28.660 INFO:journalctl@ceph.mon.a.smithi100.stdout: 2024-07-26T13:28:28.660 INFO:journalctl@ceph.mon.a.smithi100.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-26T13:28:28.660 INFO:journalctl@ceph.mon.a.smithi100.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-07-26T13:28:28.660 INFO:journalctl@ceph.mon.a.smithi100.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-07-26T13:28:28.660 INFO:journalctl@ceph.mon.a.smithi100.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-07-26T13:28:28.661 INFO:journalctl@ceph.mon.a.smithi100.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-07-26T13:28:28.661 INFO:journalctl@ceph.mon.a.smithi100.stdout: AddFile(Keys): cumulative 0, interval 0 2024-07-26T13:28:28.661 INFO:journalctl@ceph.mon.a.smithi100.stdout: Cumulative compaction: 0.00 GB write, 26.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-26T13:28:28.661 INFO:journalctl@ceph.mon.a.smithi100.stdout: Interval compaction: 0.00 GB write, 26.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-26T13:28:28.661 INFO:journalctl@ceph.mon.a.smithi100.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-07-26T13:28:28.661 INFO:journalctl@ceph.mon.a.smithi100.stdout: Block cache BinnedLRUCache@0x55c49976d610#2 capacity: 512.00 MB usage: 1.31 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-07-26T13:28:28.661 INFO:journalctl@ceph.mon.a.smithi100.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.84 KB,0.000160933%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-07-26T13:28:28.661 INFO:journalctl@ceph.mon.a.smithi100.stdout: 2024-07-26T13:28:28.661 INFO:journalctl@ceph.mon.a.smithi100.stdout: ** File Read Latency Histogram By Level [default] ** 2024-07-26T13:28:28.661 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: starting mon.a rank 0 at public addrs [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] at bind addrs [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 2024-07-26T13:28:28.661 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: mon.a@-1(???) e1 preinit fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 2024-07-26T13:28:28.661 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: mon.a@-1(???).mds e1 new map 2024-07-26T13:28:28.661 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: mon.a@-1(???).mds e1 print_map 2024-07-26T13:28:28.662 INFO:journalctl@ceph.mon.a.smithi100.stdout: e1 2024-07-26T13:28:28.662 INFO:journalctl@ceph.mon.a.smithi100.stdout: btime 2024-07-26T13:28:23:639568+0000 2024-07-26T13:28:28.662 INFO:journalctl@ceph.mon.a.smithi100.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-07-26T13:28:28.662 INFO:journalctl@ceph.mon.a.smithi100.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-07-26T13:28:28.662 INFO:journalctl@ceph.mon.a.smithi100.stdout: legacy client fscid: -1 2024-07-26T13:28:28.662 INFO:journalctl@ceph.mon.a.smithi100.stdout: 2024-07-26T13:28:28.662 INFO:journalctl@ceph.mon.a.smithi100.stdout: No filesystems configured 2024-07-26T13:28:28.662 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-07-26T13:28:28.662 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-07-26T13:28:28.662 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-07-26T13:28:28.662 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-07-26T13:28:28.662 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-07-26T13:28:28.662 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 bash[32581]: 304ecaf8e6425e7342311b96a955e4dbbf347888a5eda55a70d73b8d96d4d639 2024-07-26T13:28:28.662 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 systemd[1]: Started Ceph mon.a for bf83e2e6-4b52-11ef-bcb7-c7b262605968. 2024-07-26T13:28:28.663 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: mon.a is new leader, mons a in quorum (ranks 0) 2024-07-26T13:28:28.663 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: monmap epoch 1 2024-07-26T13:28:28.663 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 2024-07-26T13:28:28.663 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: last_changed 2024-07-26T13:28:19.859615+0000 2024-07-26T13:28:28.663 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: created 2024-07-26T13:28:19.859615+0000 2024-07-26T13:28:28.663 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: min_mon_release 19 (squid) 2024-07-26T13:28:28.663 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: election_strategy: 1 2024-07-26T13:28:28.663 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: 0: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.a 2024-07-26T13:28:28.663 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: fsmap 2024-07-26T13:28:28.663 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: osdmap e1: 0 total, 0 up, 0 in 2024-07-26T13:28:28.663 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:28 smithi100 ceph-mon[32599]: mgrmap e1: no daemons active 2024-07-26T13:28:29.481 INFO:teuthology.orchestra.run.smithi100.stdout:Wrote config to /etc/ceph/ceph.conf 2024-07-26T13:28:29.502 INFO:teuthology.orchestra.run.smithi100.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-07-26T13:28:29.503 INFO:teuthology.orchestra.run.smithi100.stdout:Creating mgr... 2024-07-26T13:28:29.503 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying port 0.0.0.0:9283 ... 2024-07-26T13:28:29.503 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying port 0.0.0.0:8765 ... 2024-07-26T13:28:29.754 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mgr.x 2024-07-26T13:28:29.755 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Failed to reset failed state of unit ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mgr.x.service: Unit ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mgr.x.service not loaded. 2024-07-26T13:28:29.901 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968.target.wants/ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@.service. 2024-07-26T13:28:30.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:29 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2799840324' entity='client.admin' 2024-07-26T13:28:30.245 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:30 smithi100 podman[32814]: 2024-07-26 13:28:30.137247499 +0000 UTC m=+0.115389196 container create 448c8fb66ee41503d0bbef53610ba13f285e05eae121e9b63bf1d0e058cce960 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240724, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-26T13:28:30.489 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-07-26T13:28:30.490 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout disabled 2024-07-26T13:28:30.498 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-07-26T13:28:30.498 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-07-26T13:28:30.498 INFO:teuthology.orchestra.run.smithi100.stdout:firewalld.service is not enabled 2024-07-26T13:28:30.498 INFO:teuthology.orchestra.run.smithi100.stdout:Not possible to enable service . firewalld.service is not available 2024-07-26T13:28:30.509 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-07-26T13:28:30.509 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout disabled 2024-07-26T13:28:30.518 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-07-26T13:28:30.518 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-07-26T13:28:30.518 INFO:teuthology.orchestra.run.smithi100.stdout:firewalld.service is not enabled 2024-07-26T13:28:30.518 INFO:teuthology.orchestra.run.smithi100.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-07-26T13:28:30.518 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for mgr to start... 2024-07-26T13:28:30.518 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for mgr... 2024-07-26T13:28:30.643 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:30 smithi100 podman[32814]: 2024-07-26 13:28:30.381377714 +0000 UTC m=+0.359519404 container init 448c8fb66ee41503d0bbef53610ba13f285e05eae121e9b63bf1d0e058cce960 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-26T13:28:30.643 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:30 smithi100 podman[32814]: 2024-07-26 13:28:30.384106339 +0000 UTC m=+0.362248030 container start 448c8fb66ee41503d0bbef53610ba13f285e05eae121e9b63bf1d0e058cce960 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-07-26T13:28:30.643 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:30 smithi100 bash[32814]: 448c8fb66ee41503d0bbef53610ba13f285e05eae121e9b63bf1d0e058cce960 2024-07-26T13:28:30.643 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:30 smithi100 systemd[1]: Started Ceph mgr.x for bf83e2e6-4b52-11ef-bcb7-c7b262605968. 2024-07-26T13:28:31.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:31 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/478684269' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-07-26T13:28:31.616 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-07-26T13:28:31.621 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-07-26T13:28:31.622 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsid": "bf83e2e6-4b52-11ef-bcb7-c7b262605968", 2024-07-26T13:28:31.622 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "health": { 2024-07-26T13:28:31.622 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-07-26T13:28:31.622 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-07-26T13:28:31.622 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-07-26T13:28:31.622 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-07-26T13:28:31.622 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-07-26T13:28:31.622 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-07-26T13:28:31.622 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 0 2024-07-26T13:28:31.622 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-07-26T13:28:31.622 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-07-26T13:28:31.623 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "a" 2024-07-26T13:28:31.623 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-07-26T13:28:31.623 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-07-26T13:28:31.623 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "monmap": { 2024-07-26T13:28:31.623 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-26T13:28:31.623 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-07-26T13:28:31.623 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-07-26T13:28:31.623 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-07-26T13:28:31.623 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-07-26T13:28:31.623 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-26T13:28:31.623 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-07-26T13:28:31.623 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-07-26T13:28:31.623 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-07-26T13:28:31.623 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-07-26T13:28:31.623 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-07-26T13:28:31.624 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-07-26T13:28:31.628 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-07-26T13:28:31.628 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-07-26T13:28:31.628 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-07-26T13:28:31.628 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-07-26T13:28:31.628 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-07-26T13:28:31.628 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-07-26T13:28:31.628 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-07-26T13:28:31.628 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-07-26T13:28:31.628 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-07-26T13:28:31.628 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-07-26T13:28:31.628 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-07-26T13:28:31.628 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-07-26T13:28:31.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-26T13:28:31.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "btime": "2024-07-26T13:28:23:639568+0000", 2024-07-26T13:28:31.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-07-26T13:28:31.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-07-26T13:28:31.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-07-26T13:28:31.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-07-26T13:28:31.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "available": false, 2024-07-26T13:28:31.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-07-26T13:28:31.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modules": [ 2024-07-26T13:28:31.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "iostat", 2024-07-26T13:28:31.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "nfs", 2024-07-26T13:28:31.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "restful" 2024-07-26T13:28:31.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-07-26T13:28:31.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-26T13:28:31.630 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-07-26T13:28:31.630 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-07-26T13:28:31.630 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-26T13:28:31.630 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modified": "2024-07-26T13:28:23.640382+0000", 2024-07-26T13:28:31.630 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-26T13:28:31.630 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-07-26T13:28:31.630 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-07-26T13:28:31.630 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-07-26T13:28:31.630 INFO:teuthology.orchestra.run.smithi100.stdout:mgr not available, waiting (1/15)... 2024-07-26T13:28:34.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:34 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/855336248' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-07-26T13:28:34.641 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-07-26T13:28:34.641 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-07-26T13:28:34.641 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsid": "bf83e2e6-4b52-11ef-bcb7-c7b262605968", 2024-07-26T13:28:34.641 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "health": { 2024-07-26T13:28:34.641 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-07-26T13:28:34.641 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-07-26T13:28:34.641 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-07-26T13:28:34.641 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-07-26T13:28:34.641 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-07-26T13:28:34.641 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-07-26T13:28:34.642 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 0 2024-07-26T13:28:34.642 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-07-26T13:28:34.642 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-07-26T13:28:34.642 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "a" 2024-07-26T13:28:34.642 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-07-26T13:28:34.642 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-07-26T13:28:34.642 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "monmap": { 2024-07-26T13:28:34.642 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-26T13:28:34.642 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-07-26T13:28:34.642 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-07-26T13:28:34.642 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-07-26T13:28:34.642 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-07-26T13:28:34.642 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-26T13:28:34.643 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-07-26T13:28:34.643 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-07-26T13:28:34.643 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-07-26T13:28:34.643 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-07-26T13:28:34.646 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-07-26T13:28:34.646 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-07-26T13:28:34.646 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-07-26T13:28:34.646 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-07-26T13:28:34.647 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-07-26T13:28:34.647 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-07-26T13:28:34.647 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-07-26T13:28:34.647 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-07-26T13:28:34.647 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-07-26T13:28:34.647 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-07-26T13:28:34.647 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-07-26T13:28:34.647 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-07-26T13:28:34.647 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-07-26T13:28:34.647 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-07-26T13:28:34.647 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-26T13:28:34.647 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "btime": "2024-07-26T13:28:23:639568+0000", 2024-07-26T13:28:34.647 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-07-26T13:28:34.647 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-07-26T13:28:34.648 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-07-26T13:28:34.648 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-07-26T13:28:34.648 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "available": false, 2024-07-26T13:28:34.648 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-07-26T13:28:34.648 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modules": [ 2024-07-26T13:28:34.648 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "iostat", 2024-07-26T13:28:34.648 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "nfs", 2024-07-26T13:28:34.648 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "restful" 2024-07-26T13:28:34.648 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-07-26T13:28:34.648 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-26T13:28:34.648 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-07-26T13:28:34.648 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-07-26T13:28:34.648 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-26T13:28:34.648 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modified": "2024-07-26T13:28:23.640382+0000", 2024-07-26T13:28:34.649 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-26T13:28:34.649 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-07-26T13:28:34.649 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-07-26T13:28:34.649 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-07-26T13:28:34.649 INFO:teuthology.orchestra.run.smithi100.stdout:mgr not available, waiting (2/15)... 2024-07-26T13:28:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:35 smithi100 ceph-mon[32599]: Activating manager daemon x 2024-07-26T13:28:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:35 smithi100 ceph-mon[32599]: mgrmap e2: x(active, starting, since 0.00357794s) 2024-07-26T13:28:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:35 smithi100 ceph-mon[32599]: from='mgr.14100 172.21.15.100:0/3509102976' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-26T13:28:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:35 smithi100 ceph-mon[32599]: from='mgr.14100 172.21.15.100:0/3509102976' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-26T13:28:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:35 smithi100 ceph-mon[32599]: from='mgr.14100 172.21.15.100:0/3509102976' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-26T13:28:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:35 smithi100 ceph-mon[32599]: from='mgr.14100 172.21.15.100:0/3509102976' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-26T13:28:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:35 smithi100 ceph-mon[32599]: from='mgr.14100 172.21.15.100:0/3509102976' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-07-26T13:28:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:35 smithi100 ceph-mon[32599]: Manager daemon x is now available 2024-07-26T13:28:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:35 smithi100 ceph-mon[32599]: from='mgr.14100 172.21.15.100:0/3509102976' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-07-26T13:28:35.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:35 smithi100 ceph-mon[32599]: from='mgr.14100 172.21.15.100:0/3509102976' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-07-26T13:28:35.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:35 smithi100 ceph-mon[32599]: from='mgr.14100 172.21.15.100:0/3509102976' entity='mgr.x' 2024-07-26T13:28:35.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:35 smithi100 ceph-mon[32599]: from='mgr.14100 172.21.15.100:0/3509102976' entity='mgr.x' 2024-07-26T13:28:35.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:35 smithi100 ceph-mon[32599]: from='mgr.14100 172.21.15.100:0/3509102976' entity='mgr.x' 2024-07-26T13:28:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:37 smithi100 ceph-mon[32599]: mgrmap e3: x(active, since 1.00748s) 2024-07-26T13:28:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:37 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/692392055' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-07-26T13:28:38.055 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-07-26T13:28:38.055 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-07-26T13:28:38.055 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsid": "bf83e2e6-4b52-11ef-bcb7-c7b262605968", 2024-07-26T13:28:38.056 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "health": { 2024-07-26T13:28:38.056 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-07-26T13:28:38.056 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-07-26T13:28:38.056 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-07-26T13:28:38.056 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-07-26T13:28:38.056 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-07-26T13:28:38.056 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-07-26T13:28:38.056 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 0 2024-07-26T13:28:38.056 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-07-26T13:28:38.056 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-07-26T13:28:38.056 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "a" 2024-07-26T13:28:38.056 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-07-26T13:28:38.056 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-07-26T13:28:38.057 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "monmap": { 2024-07-26T13:28:38.057 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-26T13:28:38.057 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-07-26T13:28:38.057 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-07-26T13:28:38.057 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-07-26T13:28:38.057 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-07-26T13:28:38.057 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-26T13:28:38.057 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-07-26T13:28:38.057 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-07-26T13:28:38.057 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-07-26T13:28:38.057 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-07-26T13:28:38.061 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-07-26T13:28:38.061 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-07-26T13:28:38.061 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-07-26T13:28:38.061 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-07-26T13:28:38.061 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-07-26T13:28:38.061 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-07-26T13:28:38.061 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-07-26T13:28:38.061 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-07-26T13:28:38.061 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-07-26T13:28:38.061 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-07-26T13:28:38.061 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-07-26T13:28:38.062 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-07-26T13:28:38.062 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-07-26T13:28:38.062 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-07-26T13:28:38.062 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-26T13:28:38.062 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "btime": "2024-07-26T13:28:23:639568+0000", 2024-07-26T13:28:38.062 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-07-26T13:28:38.062 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-07-26T13:28:38.062 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-07-26T13:28:38.062 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-07-26T13:28:38.062 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "available": true, 2024-07-26T13:28:38.062 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-07-26T13:28:38.062 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modules": [ 2024-07-26T13:28:38.062 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "iostat", 2024-07-26T13:28:38.063 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "nfs", 2024-07-26T13:28:38.063 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "restful" 2024-07-26T13:28:38.063 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-07-26T13:28:38.063 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-26T13:28:38.063 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-07-26T13:28:38.063 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-07-26T13:28:38.063 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-26T13:28:38.063 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modified": "2024-07-26T13:28:23.640382+0000", 2024-07-26T13:28:38.063 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-26T13:28:38.063 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-07-26T13:28:38.063 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-07-26T13:28:38.063 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-07-26T13:28:38.063 INFO:teuthology.orchestra.run.smithi100.stdout:mgr is available 2024-07-26T13:28:38.664 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:38 smithi100 ceph-mon[32599]: mgrmap e4: x(active, since 2s) 2024-07-26T13:28:39.186 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-07-26T13:28:39.186 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout [global] 2024-07-26T13:28:39.186 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout fsid = bf83e2e6-4b52-11ef-bcb7-c7b262605968 2024-07-26T13:28:39.186 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-07-26T13:28:39.186 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.100:3300,v1:172.21.15.100:6789] 2024-07-26T13:28:39.186 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-07-26T13:28:39.186 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-07-26T13:28:39.186 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-07-26T13:28:39.186 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-07-26T13:28:39.186 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-07-26T13:28:39.186 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout [mgr] 2024-07-26T13:28:39.187 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-07-26T13:28:39.187 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-07-26T13:28:39.187 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout [osd] 2024-07-26T13:28:39.187 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-07-26T13:28:39.187 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-07-26T13:28:39.187 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-07-26T13:28:39.187 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-07-26T13:28:39.187 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-07-26T13:28:39.187 INFO:teuthology.orchestra.run.smithi100.stdout:Enabling cephadm module... 2024-07-26T13:28:39.526 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:39 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/69863483' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-07-26T13:28:40.679 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:40 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2206739627' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-07-26T13:28:40.679 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:40 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: ignoring --setuser ceph since I am not root 2024-07-26T13:28:40.679 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:40 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: ignoring --setgroup ceph since I am not root 2024-07-26T13:28:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:41 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2206739627' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-07-26T13:28:41.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:41 smithi100 ceph-mon[32599]: mgrmap e5: x(active, since 5s) 2024-07-26T13:28:42.173 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-07-26T13:28:42.173 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-07-26T13:28:42.173 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "available": true, 2024-07-26T13:28:42.173 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-07-26T13:28:42.173 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-07-26T13:28:42.173 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-07-26T13:28:42.174 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for the mgr to restart... 2024-07-26T13:28:42.174 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for mgr epoch 5... 2024-07-26T13:28:42.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:42 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/3989579688' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-07-26T13:28:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:45 smithi100 ceph-mon[32599]: Active manager daemon x restarted 2024-07-26T13:28:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:45 smithi100 ceph-mon[32599]: Activating manager daemon x 2024-07-26T13:28:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:45 smithi100 ceph-mon[32599]: osdmap e2: 0 total, 0 up, 0 in 2024-07-26T13:28:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:45 smithi100 ceph-mon[32599]: mgrmap e6: x(active, starting, since 0.00363915s) 2024-07-26T13:28:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:45 smithi100 ceph-mon[32599]: from='mgr.14118 172.21.15.100:0/3174977677' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-26T13:28:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:45 smithi100 ceph-mon[32599]: from='mgr.14118 172.21.15.100:0/3174977677' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-07-26T13:28:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:45 smithi100 ceph-mon[32599]: from='mgr.14118 172.21.15.100:0/3174977677' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-26T13:28:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:45 smithi100 ceph-mon[32599]: from='mgr.14118 172.21.15.100:0/3174977677' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-26T13:28:45.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:45 smithi100 ceph-mon[32599]: from='mgr.14118 172.21.15.100:0/3174977677' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-26T13:28:45.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:45 smithi100 ceph-mon[32599]: Manager daemon x is now available 2024-07-26T13:28:45.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:45 smithi100 ceph-mon[32599]: from='mgr.14118 172.21.15.100:0/3174977677' entity='mgr.x' 2024-07-26T13:28:45.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:45 smithi100 ceph-mon[32599]: from='mgr.14118 172.21.15.100:0/3174977677' entity='mgr.x' 2024-07-26T13:28:45.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:45 smithi100 ceph-mon[32599]: from='mgr.14118 172.21.15.100:0/3174977677' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:28:45.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:45 smithi100 ceph-mon[32599]: from='mgr.14118 172.21.15.100:0/3174977677' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:28:45.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:45 smithi100 ceph-mon[32599]: from='mgr.14118 172.21.15.100:0/3174977677' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-07-26T13:28:45.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:45 smithi100 ceph-mon[32599]: from='mgr.14118 172.21.15.100:0/3174977677' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-07-26T13:28:46.641 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:46 smithi100 ceph-mon[32599]: Found migration_current of "None". Setting to last migration. 2024-07-26T13:28:46.642 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:46 smithi100 ceph-mon[32599]: from='mgr.14118 172.21.15.100:0/3174977677' entity='mgr.x' 2024-07-26T13:28:46.642 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:46 smithi100 ceph-mon[32599]: from='mgr.14118 172.21.15.100:0/3174977677' entity='mgr.x' 2024-07-26T13:28:46.642 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:46 smithi100 ceph-mon[32599]: mgrmap e7: x(active, since 1.0065s) 2024-07-26T13:28:46.861 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-07-26T13:28:46.862 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-07-26T13:28:46.862 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "initialized": true 2024-07-26T13:28:46.862 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-07-26T13:28:46.862 INFO:teuthology.orchestra.run.smithi100.stdout:mgr epoch 5 is available 2024-07-26T13:28:46.862 INFO:teuthology.orchestra.run.smithi100.stdout:Setting orchestrator backend to cephadm... 2024-07-26T13:28:47.244 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:47 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: 2024-07-26T13:28:47.156+0000 7fa40e7aa640 -1 log_channel(cephadm) log [ERR] : [26/Jul/2024:13:28:47] ENGINE Error in HTTPServer.serve 2024-07-26T13:28:47.244 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:47 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: Traceback (most recent call last): 2024-07-26T13:28:47.244 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:47 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-26T13:28:47.244 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:47 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: self._connections.run(self.expiration_interval) 2024-07-26T13:28:47.245 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:47 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-26T13:28:47.245 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:47 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: self._run(expiration_interval) 2024-07-26T13:28:47.245 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:47 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-26T13:28:47.245 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:47 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: new_conn = self._from_server_socket(self.server.socket) 2024-07-26T13:28:47.245 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:47 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-26T13:28:47.245 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:47 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-26T13:28:47.245 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:47 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-26T13:28:47.245 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:47 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: s = self.context.wrap_socket( 2024-07-26T13:28:47.245 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:47 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-26T13:28:47.245 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:47 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: return self.sslsocket_class._create( 2024-07-26T13:28:47.245 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:47 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-26T13:28:47.245 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:47 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: self.do_handshake() 2024-07-26T13:28:47.245 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:47 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-26T13:28:47.246 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:47 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: self._sslobj.do_handshake() 2024-07-26T13:28:47.246 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:47 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-26T13:28:47.246 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:47 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: 2024-07-26T13:28:47.739 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:47 smithi100 ceph-mon[32599]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-07-26T13:28:47.739 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:47 smithi100 ceph-mon[32599]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-07-26T13:28:47.739 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:47 smithi100 ceph-mon[32599]: from='mgr.14118 172.21.15.100:0/3174977677' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:28:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:48 smithi100 ceph-mon[32599]: [26/Jul/2024:13:28:46] ENGINE Bus STARTING 2024-07-26T13:28:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:48 smithi100 ceph-mon[32599]: [26/Jul/2024:13:28:47] ENGINE Serving on http://172.21.15.100:8765 2024-07-26T13:28:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:48 smithi100 ceph-mon[32599]: [26/Jul/2024:13:28:47] ENGINE Serving on https://172.21.15.100:7150 2024-07-26T13:28:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:48 smithi100 ceph-mon[32599]: [26/Jul/2024:13:28:47] ENGINE Bus STARTED 2024-07-26T13:28:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:48 smithi100 ceph-mon[32599]: [26/Jul/2024:13:28:47] ENGINE Error in HTTPServer.serve 2024-07-26T13:28:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout: Traceback (most recent call last): 2024-07-26T13:28:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-26T13:28:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout: self._connections.run(self.expiration_interval) 2024-07-26T13:28:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-26T13:28:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout: self._run(expiration_interval) 2024-07-26T13:28:48.745 INFO:journalctl@ceph.mon.a.smithi100.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-26T13:28:48.745 INFO:journalctl@ceph.mon.a.smithi100.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-26T13:28:48.745 INFO:journalctl@ceph.mon.a.smithi100.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-26T13:28:48.745 INFO:journalctl@ceph.mon.a.smithi100.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-26T13:28:48.745 INFO:journalctl@ceph.mon.a.smithi100.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-26T13:28:48.745 INFO:journalctl@ceph.mon.a.smithi100.stdout: s = self.context.wrap_socket( 2024-07-26T13:28:48.745 INFO:journalctl@ceph.mon.a.smithi100.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-26T13:28:48.745 INFO:journalctl@ceph.mon.a.smithi100.stdout: return self.sslsocket_class._create( 2024-07-26T13:28:48.745 INFO:journalctl@ceph.mon.a.smithi100.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-26T13:28:48.745 INFO:journalctl@ceph.mon.a.smithi100.stdout: self.do_handshake() 2024-07-26T13:28:48.745 INFO:journalctl@ceph.mon.a.smithi100.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-26T13:28:48.745 INFO:journalctl@ceph.mon.a.smithi100.stdout: self._sslobj.do_handshake() 2024-07-26T13:28:48.745 INFO:journalctl@ceph.mon.a.smithi100.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-26T13:28:48.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:48 smithi100 ceph-mon[32599]: mgrmap e8: x(active, since 2s) 2024-07-26T13:28:48.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:48 smithi100 ceph-mon[32599]: from='mgr.14118 172.21.15.100:0/3174977677' entity='mgr.x' 2024-07-26T13:28:48.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:48 smithi100 ceph-mon[32599]: from='mgr.14118 172.21.15.100:0/3174977677' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:28:49.140 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout value unchanged 2024-07-26T13:28:49.140 INFO:teuthology.orchestra.run.smithi100.stdout:Generating ssh key... 2024-07-26T13:28:49.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:49 smithi100 ceph-mon[32599]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:28:50.110 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:49 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: Generating public/private rsa key pair. 2024-07-26T13:28:50.110 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:49 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: Your identification has been saved in /tmp/tmp1navw5ns/key 2024-07-26T13:28:50.110 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:49 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: Your public key has been saved in /tmp/tmp1navw5ns/key.pub 2024-07-26T13:28:50.110 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:49 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: The key fingerprint is: 2024-07-26T13:28:50.111 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:49 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: SHA256:w4Z6od6o+odvrpA06t8hURYsDkG+ueeelnRKgANicao ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968 2024-07-26T13:28:50.111 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:49 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: The key's randomart image is: 2024-07-26T13:28:50.111 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:49 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: +---[RSA 3072]----+ 2024-07-26T13:28:50.111 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:49 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: |.+o... | 2024-07-26T13:28:50.111 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:49 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: |+oo. .. | 2024-07-26T13:28:50.111 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:49 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: |=oo .o | 2024-07-26T13:28:50.111 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:49 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: |+.o.o o | 2024-07-26T13:28:50.111 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:49 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: |E*.. o S | 2024-07-26T13:28:50.111 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:49 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: |o.oo.+ o . | 2024-07-26T13:28:50.111 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:49 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: |+.o+*.. | 2024-07-26T13:28:50.111 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:49 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: |..+=*+. | 2024-07-26T13:28:50.111 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:49 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: |.+B%=.. | 2024-07-26T13:28:50.111 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:28:49 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: +----[SHA256]-----+ 2024-07-26T13:28:50.671 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:50 smithi100 ceph-mon[32599]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:28:50.672 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:50 smithi100 ceph-mon[32599]: from='mgr.14118 172.21.15.100:0/3174977677' entity='mgr.x' 2024-07-26T13:28:50.672 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:50 smithi100 ceph-mon[32599]: from='mgr.14118 172.21.15.100:0/3174977677' entity='mgr.x' 2024-07-26T13:28:51.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:51 smithi100 ceph-mon[32599]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:28:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:51 smithi100 ceph-mon[32599]: Generating ssh key... 2024-07-26T13:28:51.528 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhKFMuiMnp/UOQEroA9RXkPiyF4YMuDvTct8wSMWUOLBqdowwu4l/v60557QMLCKm7RfawuwmCWGEnafoGKdWgAtgrKOfQtCykXOQ64rlhvT7PaWQZegy+IJs6QhbpeZRxYV30m06HSnDORxbzxo1fPikDDJjmgvlZtr43vWBiRlrfccHxZdK+D3cbMsz2YzfifGAQqSQVn+URDTs/xcB0db7XMZ7nXs5rdD1nWrD+Z9X/TKqsRyvmy5L8D1yg3pzJFdDU69whfhNK6NolzEYYHisj04B8KXmgAtAqG35VoCI04gDOU5DjoRlE6ebnkXzfjrwR9kpcEwriRL18wczZjf/EDTzpKPn0i9LyWTuBfG1IUE6djrxg9PcIaJaNs+0a/9zx4IdRbZeI2diqL0+WKUFLfCKdm2TvdLvU85j+suTVffNlAQs9GEiFHXDtrr7tcg/07UsLvy5bivMIPzgYas4YRJKRhljVuLZZjV8kQQX6CqqRgIKlzt0Qu/1r1M8= ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968 2024-07-26T13:28:51.528 INFO:teuthology.orchestra.run.smithi100.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-07-26T13:28:51.528 INFO:teuthology.orchestra.run.smithi100.stdout:Adding key to root@localhost authorized_keys... 2024-07-26T13:28:51.529 INFO:teuthology.orchestra.run.smithi100.stdout:Adding host smithi100... 2024-07-26T13:28:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:52 smithi100 ceph-mon[32599]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:28:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:53 smithi100 ceph-mon[32599]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi100", "addr": "172.21.15.100", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:28:54.699 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:54 smithi100 ceph-mon[32599]: Deploying cephadm binary to smithi100 2024-07-26T13:28:55.802 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout Added host 'smithi100' with addr '172.21.15.100' 2024-07-26T13:28:55.803 INFO:teuthology.orchestra.run.smithi100.stdout:Deploying unmanaged mon service... 2024-07-26T13:28:56.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:56 smithi100 ceph-mon[32599]: from='mgr.14118 172.21.15.100:0/3174977677' entity='mgr.x' 2024-07-26T13:28:56.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:56 smithi100 ceph-mon[32599]: Added host smithi100 2024-07-26T13:28:56.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:56 smithi100 ceph-mon[32599]: from='mgr.14118 172.21.15.100:0/3174977677' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:28:57.011 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-07-26T13:28:57.011 INFO:teuthology.orchestra.run.smithi100.stdout:Deploying unmanaged mgr service... 2024-07-26T13:28:57.551 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:57 smithi100 ceph-mon[32599]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:28:57.551 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:57 smithi100 ceph-mon[32599]: Saving service mon spec with placement count:5 2024-07-26T13:28:57.551 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:57 smithi100 ceph-mon[32599]: from='mgr.14118 172.21.15.100:0/3174977677' entity='mgr.x' 2024-07-26T13:28:58.103 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-07-26T13:28:58.637 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:58 smithi100 ceph-mon[32599]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:28:58.637 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:58 smithi100 ceph-mon[32599]: Saving service mgr spec with placement count:2 2024-07-26T13:28:58.637 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:58 smithi100 ceph-mon[32599]: from='mgr.14118 172.21.15.100:0/3174977677' entity='mgr.x' 2024-07-26T13:28:59.960 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:59 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2343104724' entity='client.admin' 2024-07-26T13:28:59.960 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:28:59 smithi100 ceph-mon[32599]: from='mgr.14118 172.21.15.100:0/3174977677' entity='mgr.x' 2024-07-26T13:29:01.236 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:00 smithi100 ceph-mon[32599]: from='mgr.14118 172.21.15.100:0/3174977677' entity='mgr.x' 2024-07-26T13:29:01.236 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:00 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/882991142' entity='client.admin' 2024-07-26T13:29:01.238 INFO:teuthology.orchestra.run.smithi100.stdout:Enabling the dashboard module... 2024-07-26T13:29:03.244 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:29:03 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: ignoring --setuser ceph since I am not root 2024-07-26T13:29:03.244 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:29:03 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: ignoring --setgroup ceph since I am not root 2024-07-26T13:29:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:02 smithi100 ceph-mon[32599]: from='mgr.14118 172.21.15.100:0/3174977677' entity='mgr.x' 2024-07-26T13:29:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:02 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/4113217258' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-07-26T13:29:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:02 smithi100 ceph-mon[32599]: from='mgr.14118 172.21.15.100:0/3174977677' entity='mgr.x' 2024-07-26T13:29:04.226 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:03 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/4113217258' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-07-26T13:29:04.226 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:03 smithi100 ceph-mon[32599]: mgrmap e9: x(active, since 17s) 2024-07-26T13:29:05.365 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:05 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/1911868741' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-07-26T13:29:05.611 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-07-26T13:29:05.611 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-07-26T13:29:05.611 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "available": true, 2024-07-26T13:29:05.611 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-07-26T13:29:05.611 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-07-26T13:29:05.611 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-07-26T13:29:05.611 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for the mgr to restart... 2024-07-26T13:29:05.611 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for mgr epoch 9... 2024-07-26T13:29:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:08 smithi100 ceph-mon[32599]: Active manager daemon x restarted 2024-07-26T13:29:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:08 smithi100 ceph-mon[32599]: Activating manager daemon x 2024-07-26T13:29:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:08 smithi100 ceph-mon[32599]: osdmap e3: 0 total, 0 up, 0 in 2024-07-26T13:29:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:08 smithi100 ceph-mon[32599]: mgrmap e10: x(active, starting, since 0.00433713s) 2024-07-26T13:29:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:08 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-26T13:29:08.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:08 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-07-26T13:29:08.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:08 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-26T13:29:08.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:08 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-26T13:29:08.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:08 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-26T13:29:08.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:08 smithi100 ceph-mon[32599]: Manager daemon x is now available 2024-07-26T13:29:08.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:08 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:29:08.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:08 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-07-26T13:29:08.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:08 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-07-26T13:29:09.331 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:29:09 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: 2024-07-26T13:29:09.073+0000 7f4c3166d640 -1 log_channel(cephadm) log [ERR] : [26/Jul/2024:13:29:09] ENGINE Error in HTTPServer.serve 2024-07-26T13:29:09.331 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:29:09 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: Traceback (most recent call last): 2024-07-26T13:29:09.331 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:29:09 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-26T13:29:09.331 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:29:09 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: self._connections.run(self.expiration_interval) 2024-07-26T13:29:09.331 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:29:09 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-26T13:29:09.331 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:29:09 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: self._run(expiration_interval) 2024-07-26T13:29:09.331 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:29:09 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-26T13:29:09.331 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:29:09 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: new_conn = self._from_server_socket(self.server.socket) 2024-07-26T13:29:09.331 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:29:09 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-26T13:29:09.331 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:29:09 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-26T13:29:09.331 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:29:09 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-26T13:29:09.332 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:29:09 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: s = self.context.wrap_socket( 2024-07-26T13:29:09.332 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:29:09 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-26T13:29:09.332 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:29:09 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: return self.sslsocket_class._create( 2024-07-26T13:29:09.332 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:29:09 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-26T13:29:09.332 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:29:09 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: self.do_handshake() 2024-07-26T13:29:09.332 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:29:09 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-26T13:29:09.332 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:29:09 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: self._sslobj.do_handshake() 2024-07-26T13:29:09.332 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:29:09 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-26T13:29:09.332 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:29:09 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: 2024-07-26T13:29:09.786 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-07-26T13:29:09.787 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-07-26T13:29:09.787 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "initialized": true 2024-07-26T13:29:09.787 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-07-26T13:29:09.787 INFO:teuthology.orchestra.run.smithi100.stdout:mgr epoch 9 is available 2024-07-26T13:29:09.787 INFO:teuthology.orchestra.run.smithi100.stdout:Generating a dashboard self-signed certificate... 2024-07-26T13:29:10.096 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:09 smithi100 ceph-mon[32599]: [26/Jul/2024:13:29:08] ENGINE Bus STARTING 2024-07-26T13:29:10.096 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:09 smithi100 ceph-mon[32599]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-07-26T13:29:10.096 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:09 smithi100 ceph-mon[32599]: mgrmap e11: x(active, since 1.00606s) 2024-07-26T13:29:10.096 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:09 smithi100 ceph-mon[32599]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-07-26T13:29:10.097 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:09 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:10.097 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:09 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:11.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:10 smithi100 ceph-mon[32599]: [26/Jul/2024:13:29:09] ENGINE Serving on https://172.21.15.100:7150 2024-07-26T13:29:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:10 smithi100 ceph-mon[32599]: [26/Jul/2024:13:29:09] ENGINE Error in HTTPServer.serve 2024-07-26T13:29:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout: Traceback (most recent call last): 2024-07-26T13:29:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-26T13:29:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout: self._connections.run(self.expiration_interval) 2024-07-26T13:29:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-26T13:29:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout: self._run(expiration_interval) 2024-07-26T13:29:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-26T13:29:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-26T13:29:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-26T13:29:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-26T13:29:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-26T13:29:11.245 INFO:journalctl@ceph.mon.a.smithi100.stdout: s = self.context.wrap_socket( 2024-07-26T13:29:11.245 INFO:journalctl@ceph.mon.a.smithi100.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-26T13:29:11.245 INFO:journalctl@ceph.mon.a.smithi100.stdout: return self.sslsocket_class._create( 2024-07-26T13:29:11.245 INFO:journalctl@ceph.mon.a.smithi100.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-26T13:29:11.245 INFO:journalctl@ceph.mon.a.smithi100.stdout: self.do_handshake() 2024-07-26T13:29:11.245 INFO:journalctl@ceph.mon.a.smithi100.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-26T13:29:11.245 INFO:journalctl@ceph.mon.a.smithi100.stdout: self._sslobj.do_handshake() 2024-07-26T13:29:11.245 INFO:journalctl@ceph.mon.a.smithi100.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-26T13:29:11.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:10 smithi100 ceph-mon[32599]: [26/Jul/2024:13:29:09] ENGINE Serving on http://172.21.15.100:8765 2024-07-26T13:29:11.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:10 smithi100 ceph-mon[32599]: [26/Jul/2024:13:29:09] ENGINE Bus STARTED 2024-07-26T13:29:11.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:10 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:11.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:10 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:11.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:10 smithi100 ceph-mon[32599]: mgrmap e12: x(active, since 2s) 2024-07-26T13:29:11.565 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-07-26T13:29:11.565 INFO:teuthology.orchestra.run.smithi100.stdout:Creating initial admin user... 2024-07-26T13:29:12.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:11 smithi100 ceph-mon[32599]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:29:12.988 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$76bx8khO7hdowCvRc9xWK.6kP3/m12hYj7XIGHQFEUB2fSe.k6Fmi", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1722000552, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-07-26T13:29:12.988 INFO:teuthology.orchestra.run.smithi100.stdout:Fetching dashboard port number... 2024-07-26T13:29:13.647 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:13 smithi100 ceph-mon[32599]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:29:13.647 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:13 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:14.161 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 8443 2024-07-26T13:29:14.170 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-07-26T13:29:14.170 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout disabled 2024-07-26T13:29:14.177 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-07-26T13:29:14.177 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-07-26T13:29:14.178 INFO:teuthology.orchestra.run.smithi100.stdout:firewalld.service is not enabled 2024-07-26T13:29:14.178 INFO:teuthology.orchestra.run.smithi100.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-07-26T13:29:14.179 INFO:teuthology.orchestra.run.smithi100.stdout:Ceph Dashboard is now available at: 2024-07-26T13:29:14.179 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:29:14.179 INFO:teuthology.orchestra.run.smithi100.stdout: URL: https://smithi100.front.sepia.ceph.com:8443/ 2024-07-26T13:29:14.179 INFO:teuthology.orchestra.run.smithi100.stdout: User: admin 2024-07-26T13:29:14.180 INFO:teuthology.orchestra.run.smithi100.stdout: Password: s2nphagh67 2024-07-26T13:29:14.180 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:29:14.180 INFO:teuthology.orchestra.run.smithi100.stdout:Saving cluster configuration to /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/config directory 2024-07-26T13:29:14.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:14 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/1654387282' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-07-26T13:29:15.443 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-07-26T13:29:15.443 INFO:teuthology.orchestra.run.smithi100.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-07-26T13:29:15.444 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:29:15.444 INFO:teuthology.orchestra.run.smithi100.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-07-26T13:29:15.444 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:29:15.444 INFO:teuthology.orchestra.run.smithi100.stdout:Or, if you are only running a single cluster on this host: 2024-07-26T13:29:15.444 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:29:15.444 INFO:teuthology.orchestra.run.smithi100.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-07-26T13:29:15.444 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:29:15.444 INFO:teuthology.orchestra.run.smithi100.stdout:Please consider enabling telemetry to help improve Ceph: 2024-07-26T13:29:15.444 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:29:15.444 INFO:teuthology.orchestra.run.smithi100.stdout: ceph telemetry on 2024-07-26T13:29:15.444 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:29:15.444 INFO:teuthology.orchestra.run.smithi100.stdout:For more information see: 2024-07-26T13:29:15.445 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:29:15.445 INFO:teuthology.orchestra.run.smithi100.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-07-26T13:29:15.445 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:29:15.445 INFO:teuthology.orchestra.run.smithi100.stdout:Bootstrap complete. 2024-07-26T13:29:15.482 INFO:tasks.cephadm:Fetching config... 2024-07-26T13:29:15.483 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-07-26T13:29:15.483 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-07-26T13:29:15.498 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-07-26T13:29:15.498 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-07-26T13:29:15.499 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-07-26T13:29:15.555 INFO:tasks.cephadm:Fetching mon keyring... 2024-07-26T13:29:15.555 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-07-26T13:29:15.555 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/keyring of=/dev/stdout 2024-07-26T13:29:15.622 INFO:tasks.cephadm:Fetching pub ssh key... 2024-07-26T13:29:15.622 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-07-26T13:29:15.622 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-07-26T13:29:15.676 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-07-26T13:29:15.677 DEBUG:teuthology.orchestra.run.smithi100:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhKFMuiMnp/UOQEroA9RXkPiyF4YMuDvTct8wSMWUOLBqdowwu4l/v60557QMLCKm7RfawuwmCWGEnafoGKdWgAtgrKOfQtCykXOQ64rlhvT7PaWQZegy+IJs6QhbpeZRxYV30m06HSnDORxbzxo1fPikDDJjmgvlZtr43vWBiRlrfccHxZdK+D3cbMsz2YzfifGAQqSQVn+URDTs/xcB0db7XMZ7nXs5rdD1nWrD+Z9X/TKqsRyvmy5L8D1yg3pzJFdDU69whfhNK6NolzEYYHisj04B8KXmgAtAqG35VoCI04gDOU5DjoRlE6ebnkXzfjrwR9kpcEwriRL18wczZjf/EDTzpKPn0i9LyWTuBfG1IUE6djrxg9PcIaJaNs+0a/9zx4IdRbZeI2diqL0+WKUFLfCKdm2TvdLvU85j+suTVffNlAQs9GEiFHXDtrr7tcg/07UsLvy5bivMIPzgYas4YRJKRhljVuLZZjV8kQQX6CqqRgIKlzt0Qu/1r1M8= ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-07-26T13:29:15.738 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:15 smithi100 ceph-mon[32599]: mgrmap e13: x(active, since 6s) 2024-07-26T13:29:15.739 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:15 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/1824610660' entity='client.admin' 2024-07-26T13:29:15.751 INFO:teuthology.orchestra.run.smithi100.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhKFMuiMnp/UOQEroA9RXkPiyF4YMuDvTct8wSMWUOLBqdowwu4l/v60557QMLCKm7RfawuwmCWGEnafoGKdWgAtgrKOfQtCykXOQ64rlhvT7PaWQZegy+IJs6QhbpeZRxYV30m06HSnDORxbzxo1fPikDDJjmgvlZtr43vWBiRlrfccHxZdK+D3cbMsz2YzfifGAQqSQVn+URDTs/xcB0db7XMZ7nXs5rdD1nWrD+Z9X/TKqsRyvmy5L8D1yg3pzJFdDU69whfhNK6NolzEYYHisj04B8KXmgAtAqG35VoCI04gDOU5DjoRlE6ebnkXzfjrwR9kpcEwriRL18wczZjf/EDTzpKPn0i9LyWTuBfG1IUE6djrxg9PcIaJaNs+0a/9zx4IdRbZeI2diqL0+WKUFLfCKdm2TvdLvU85j+suTVffNlAQs9GEiFHXDtrr7tcg/07UsLvy5bivMIPzgYas4YRJKRhljVuLZZjV8kQQX6CqqRgIKlzt0Qu/1r1M8= ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968 2024-07-26T13:29:15.762 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhKFMuiMnp/UOQEroA9RXkPiyF4YMuDvTct8wSMWUOLBqdowwu4l/v60557QMLCKm7RfawuwmCWGEnafoGKdWgAtgrKOfQtCykXOQ64rlhvT7PaWQZegy+IJs6QhbpeZRxYV30m06HSnDORxbzxo1fPikDDJjmgvlZtr43vWBiRlrfccHxZdK+D3cbMsz2YzfifGAQqSQVn+URDTs/xcB0db7XMZ7nXs5rdD1nWrD+Z9X/TKqsRyvmy5L8D1yg3pzJFdDU69whfhNK6NolzEYYHisj04B8KXmgAtAqG35VoCI04gDOU5DjoRlE6ebnkXzfjrwR9kpcEwriRL18wczZjf/EDTzpKPn0i9LyWTuBfG1IUE6djrxg9PcIaJaNs+0a/9zx4IdRbZeI2diqL0+WKUFLfCKdm2TvdLvU85j+suTVffNlAQs9GEiFHXDtrr7tcg/07UsLvy5bivMIPzgYas4YRJKRhljVuLZZjV8kQQX6CqqRgIKlzt0Qu/1r1M8= ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-07-26T13:29:15.801 INFO:teuthology.orchestra.run.smithi138.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhKFMuiMnp/UOQEroA9RXkPiyF4YMuDvTct8wSMWUOLBqdowwu4l/v60557QMLCKm7RfawuwmCWGEnafoGKdWgAtgrKOfQtCykXOQ64rlhvT7PaWQZegy+IJs6QhbpeZRxYV30m06HSnDORxbzxo1fPikDDJjmgvlZtr43vWBiRlrfccHxZdK+D3cbMsz2YzfifGAQqSQVn+URDTs/xcB0db7XMZ7nXs5rdD1nWrD+Z9X/TKqsRyvmy5L8D1yg3pzJFdDU69whfhNK6NolzEYYHisj04B8KXmgAtAqG35VoCI04gDOU5DjoRlE6ebnkXzfjrwR9kpcEwriRL18wczZjf/EDTzpKPn0i9LyWTuBfG1IUE6djrxg9PcIaJaNs+0a/9zx4IdRbZeI2diqL0+WKUFLfCKdm2TvdLvU85j+suTVffNlAQs9GEiFHXDtrr7tcg/07UsLvy5bivMIPzgYas4YRJKRhljVuLZZjV8kQQX6CqqRgIKlzt0Qu/1r1M8= ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968 2024-07-26T13:29:15.813 DEBUG:teuthology.orchestra.run.smithi167:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhKFMuiMnp/UOQEroA9RXkPiyF4YMuDvTct8wSMWUOLBqdowwu4l/v60557QMLCKm7RfawuwmCWGEnafoGKdWgAtgrKOfQtCykXOQ64rlhvT7PaWQZegy+IJs6QhbpeZRxYV30m06HSnDORxbzxo1fPikDDJjmgvlZtr43vWBiRlrfccHxZdK+D3cbMsz2YzfifGAQqSQVn+URDTs/xcB0db7XMZ7nXs5rdD1nWrD+Z9X/TKqsRyvmy5L8D1yg3pzJFdDU69whfhNK6NolzEYYHisj04B8KXmgAtAqG35VoCI04gDOU5DjoRlE6ebnkXzfjrwR9kpcEwriRL18wczZjf/EDTzpKPn0i9LyWTuBfG1IUE6djrxg9PcIaJaNs+0a/9zx4IdRbZeI2diqL0+WKUFLfCKdm2TvdLvU85j+suTVffNlAQs9GEiFHXDtrr7tcg/07UsLvy5bivMIPzgYas4YRJKRhljVuLZZjV8kQQX6CqqRgIKlzt0Qu/1r1M8= ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-07-26T13:29:15.853 INFO:teuthology.orchestra.run.smithi167.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhKFMuiMnp/UOQEroA9RXkPiyF4YMuDvTct8wSMWUOLBqdowwu4l/v60557QMLCKm7RfawuwmCWGEnafoGKdWgAtgrKOfQtCykXOQ64rlhvT7PaWQZegy+IJs6QhbpeZRxYV30m06HSnDORxbzxo1fPikDDJjmgvlZtr43vWBiRlrfccHxZdK+D3cbMsz2YzfifGAQqSQVn+URDTs/xcB0db7XMZ7nXs5rdD1nWrD+Z9X/TKqsRyvmy5L8D1yg3pzJFdDU69whfhNK6NolzEYYHisj04B8KXmgAtAqG35VoCI04gDOU5DjoRlE6ebnkXzfjrwR9kpcEwriRL18wczZjf/EDTzpKPn0i9LyWTuBfG1IUE6djrxg9PcIaJaNs+0a/9zx4IdRbZeI2diqL0+WKUFLfCKdm2TvdLvU85j+suTVffNlAQs9GEiFHXDtrr7tcg/07UsLvy5bivMIPzgYas4YRJKRhljVuLZZjV8kQQX6CqqRgIKlzt0Qu/1r1M8= ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968 2024-07-26T13:29:15.865 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-07-26T13:29:16.125 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:29:19.918 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-07-26T13:29:19.918 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-07-26T13:29:20.139 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:19 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2213853225' entity='client.admin' 2024-07-26T13:29:20.139 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:19 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:20.139 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:19 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:20.139 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:19 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi100", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:29:20.139 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:19 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:20.140 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:19 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:29:20.140 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:19 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:20.167 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:29:22.597 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi138 2024-07-26T13:29:22.597 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-07-26T13:29:22.597 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/etc/ceph/ceph.conf 2024-07-26T13:29:22.616 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-07-26T13:29:22.616 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-07-26T13:29:22.677 INFO:tasks.cephadm:Adding host smithi138 to orchestrator... 2024-07-26T13:29:22.677 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph orch host add smithi138 2024-07-26T13:29:22.932 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:29:23.198 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:22 smithi100 ceph-mon[32599]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:29:23.198 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:22 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:23.198 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:22 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:29:23.198 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:22 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:29:23.198 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:22 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:29:23.198 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:22 smithi100 ceph-mon[32599]: Updating smithi100:/etc/ceph/ceph.conf 2024-07-26T13:29:23.198 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:22 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:23.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:22 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:23.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:22 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:23.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:23 smithi100 ceph-mon[32599]: Updating smithi100:/var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/config/ceph.conf 2024-07-26T13:29:23.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:23 smithi100 ceph-mon[32599]: Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-07-26T13:29:23.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:23 smithi100 ceph-mon[32599]: Updating smithi100:/var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/config/ceph.client.admin.keyring 2024-07-26T13:29:25.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:25 smithi100 ceph-mon[32599]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi138", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:29:26.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:26 smithi100 ceph-mon[32599]: Deploying cephadm binary to smithi138 2024-07-26T13:29:27.795 INFO:teuthology.orchestra.run.smithi100.stdout:Added host 'smithi138' with addr '172.21.15.138' 2024-07-26T13:29:28.421 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph orch host ls --format=json 2024-07-26T13:29:28.666 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:29:29.098 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:28 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:29.098 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:28 smithi100 ceph-mon[32599]: Added host smithi138 2024-07-26T13:29:29.098 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:28 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:29:29.098 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:28 smithi100 ceph-mon[32599]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:29:29.098 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:28 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:29.098 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:28 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:30.241 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:29:30.241 INFO:teuthology.orchestra.run.smithi100.stdout:[{"addr": "172.21.15.100", "hostname": "smithi100", "labels": [], "status": ""}, {"addr": "172.21.15.138", "hostname": "smithi138", "labels": [], "status": ""}] 2024-07-26T13:29:30.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:30 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:30.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:30 smithi100 ceph-mon[32599]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:29:30.767 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi167 2024-07-26T13:29:30.767 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-07-26T13:29:30.768 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/etc/ceph/ceph.conf 2024-07-26T13:29:30.786 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-07-26T13:29:30.786 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-07-26T13:29:30.844 INFO:tasks.cephadm:Adding host smithi167 to orchestrator... 2024-07-26T13:29:30.844 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph orch host add smithi167 2024-07-26T13:29:31.091 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:29:31.410 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:31 smithi100 ceph-mon[32599]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-26T13:29:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:32 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:32 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:32 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:32 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:32 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi138", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:29:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:32 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:29:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:32 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:29:33.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:32 smithi100 ceph-mon[32599]: Updating smithi138:/etc/ceph/ceph.conf 2024-07-26T13:29:33.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:32 smithi100 ceph-mon[32599]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:29:33.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:32 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:33.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:32 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:33.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:32 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:34.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:33 smithi100 ceph-mon[32599]: Updating smithi138:/var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/config/ceph.conf 2024-07-26T13:29:34.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:33 smithi100 ceph-mon[32599]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-07-26T13:29:34.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:33 smithi100 ceph-mon[32599]: Updating smithi138:/var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/config/ceph.client.admin.keyring 2024-07-26T13:29:34.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:33 smithi100 ceph-mon[32599]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi167", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:29:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:34 smithi100 ceph-mon[32599]: Deploying cephadm binary to smithi167 2024-07-26T13:29:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:34 smithi100 ceph-mon[32599]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:29:36.123 INFO:teuthology.orchestra.run.smithi100.stdout:Added host 'smithi167' with addr '172.21.15.167' 2024-07-26T13:29:36.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:36 smithi100 ceph-mon[32599]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:29:36.780 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph orch host ls --format=json 2024-07-26T13:29:37.027 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:29:37.422 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:37 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:37.422 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:37 smithi100 ceph-mon[32599]: Added host smithi167 2024-07-26T13:29:37.422 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:37 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:29:37.422 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:37 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:37.422 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:37 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:38.738 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:29:38.738 INFO:teuthology.orchestra.run.smithi100.stdout:[{"addr": "172.21.15.100", "hostname": "smithi100", "labels": [], "status": ""}, {"addr": "172.21.15.138", "hostname": "smithi138", "labels": [], "status": ""}, {"addr": "172.21.15.167", "hostname": "smithi167", "labels": [], "status": ""}] 2024-07-26T13:29:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:38 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:38 smithi100 ceph-mon[32599]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:29:39.260 INFO:tasks.cephadm:Setting crush tunables to default 2024-07-26T13:29:39.260 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph osd crush tunables default 2024-07-26T13:29:39.505 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:29:39.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:39 smithi100 ceph-mon[32599]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-26T13:29:40.874 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:40 smithi100 ceph-mon[32599]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:29:40.874 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:40 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:40.874 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:40 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:40.874 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:40 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:40.875 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:40 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:40.875 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:40 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi167", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:29:40.875 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:40 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:29:40.875 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:40 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:29:41.779 INFO:teuthology.orchestra.run.smithi100.stderr:adjusted tunables profile to default 2024-07-26T13:29:42.029 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:41 smithi100 ceph-mon[32599]: Updating smithi167:/etc/ceph/ceph.conf 2024-07-26T13:29:42.029 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:41 smithi100 ceph-mon[32599]: Updating smithi167:/var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/config/ceph.conf 2024-07-26T13:29:42.029 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:41 smithi100 ceph-mon[32599]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2024-07-26T13:29:42.030 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:41 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/757579215' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-07-26T13:29:42.030 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:41 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:42.030 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:41 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:42.030 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:41 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:42.356 INFO:tasks.cephadm:Adding mon.a on smithi100 2024-07-26T13:29:42.356 INFO:tasks.cephadm:Adding mon.b on smithi138 2024-07-26T13:29:42.356 INFO:tasks.cephadm:Adding mon.c on smithi167 2024-07-26T13:29:42.357 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph orch apply mon '3;smithi100:172.21.15.100=a;smithi138:172.21.15.138=b;smithi167:172.21.15.167=c' 2024-07-26T13:29:42.606 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/config/ceph.conf 2024-07-26T13:29:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:42 smithi100 ceph-mon[32599]: Updating smithi167:/var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/config/ceph.client.admin.keyring 2024-07-26T13:29:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:42 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/757579215' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-07-26T13:29:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:42 smithi100 ceph-mon[32599]: osdmap e4: 0 total, 0 up, 0 in 2024-07-26T13:29:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:42 smithi100 ceph-mon[32599]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:29:44.299 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled mon update... 2024-07-26T13:29:44.480 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:44 smithi100 ceph-mon[32599]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:29:44.946 DEBUG:teuthology.orchestra.run.smithi138:mon.b> sudo journalctl -f -n 0 -u ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mon.b.service 2024-07-26T13:29:44.949 DEBUG:teuthology.orchestra.run.smithi167:mon.c> sudo journalctl -f -n 0 -u ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mon.c.service 2024-07-26T13:29:44.951 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-07-26T13:29:44.951 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph mon dump -f json 2024-07-26T13:29:45.551 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/config/ceph.conf 2024-07-26T13:29:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:45 smithi100 ceph-mon[32599]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi100:172.21.15.100=a;smithi138:172.21.15.138=b;smithi167:172.21.15.167=c", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:29:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:45 smithi100 ceph-mon[32599]: Saving service mon spec with placement smithi100:172.21.15.100=a;smithi138:172.21.15.138=b;smithi167:172.21.15.167=c;count:3 2024-07-26T13:29:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:45 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:45 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:29:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:45 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:29:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:45 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:29:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:45 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:45 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-26T13:29:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:45 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:29:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:45 smithi100 ceph-mon[32599]: Deploying daemon mon.c on smithi167 2024-07-26T13:29:46.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:46 smithi100 ceph-mon[32599]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:29:48.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:48 smithi100 ceph-mon[32599]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:29:48.764 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-07-26T13:29:48.765 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":1,"fsid":"bf83e2e6-4b52-11ef-bcb7-c7b262605968","modified":"2024-07-26T13:28:19.859615Z","created":"2024-07-26T13:28:19.859615Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:3300","nonce":0},{"type":"v1","addr":"172.21.15.100:6789","nonce":0}]},"addr":"172.21.15.100:6789/0","public_addr":"172.21.15.100:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-07-26T13:29:48.765 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 1 2024-07-26T13:29:49.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:49 smithi100 ceph-mon[32599]: from='client.? 172.21.15.167:0/1233631301' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-26T13:29:49.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:29:49 smithi167 systemd[1]: Started Ceph mon.c for bf83e2e6-4b52-11ef-bcb7-c7b262605968. 2024-07-26T13:29:50.349 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-07-26T13:29:50.350 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph mon dump -f json 2024-07-26T13:29:50.601 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.c/config 2024-07-26T13:29:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:54 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-26T13:29:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:54 smithi100 ceph-mon[32599]: mon.a calling monitor election 2024-07-26T13:29:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:54 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-26T13:29:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:54 smithi100 ceph-mon[32599]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:29:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:54 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-26T13:29:54.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:54 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-26T13:29:54.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:54 smithi100 ceph-mon[32599]: mon.c calling monitor election 2024-07-26T13:29:54.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:54 smithi100 ceph-mon[32599]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:29:54.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:54 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-26T13:29:54.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:54 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-26T13:29:54.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:54 smithi100 ceph-mon[32599]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:29:54.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:54 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-26T13:29:54.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:54 smithi100 ceph-mon[32599]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-07-26T13:29:54.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:54 smithi100 ceph-mon[32599]: monmap epoch 2 2024-07-26T13:29:54.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:54 smithi100 ceph-mon[32599]: fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 2024-07-26T13:29:54.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:54 smithi100 ceph-mon[32599]: last_changed 2024-07-26T13:29:49.565432+0000 2024-07-26T13:29:54.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:54 smithi100 ceph-mon[32599]: created 2024-07-26T13:28:19.859615+0000 2024-07-26T13:29:54.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:54 smithi100 ceph-mon[32599]: min_mon_release 19 (squid) 2024-07-26T13:29:54.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:54 smithi100 ceph-mon[32599]: election_strategy: 1 2024-07-26T13:29:54.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:54 smithi100 ceph-mon[32599]: 0: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.a 2024-07-26T13:29:54.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:54 smithi100 ceph-mon[32599]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-07-26T13:29:54.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:54 smithi100 ceph-mon[32599]: fsmap 2024-07-26T13:29:54.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:54 smithi100 ceph-mon[32599]: osdmap e4: 0 total, 0 up, 0 in 2024-07-26T13:29:54.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:54 smithi100 ceph-mon[32599]: mgrmap e13: x(active, since 46s) 2024-07-26T13:29:54.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:54 smithi100 ceph-mon[32599]: overall HEALTH_OK 2024-07-26T13:29:54.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:54 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:54.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:54 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:54.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:54 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:54.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:54 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-26T13:29:54.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:54 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:29:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:29:54 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-26T13:29:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:29:54 smithi167 ceph-mon[37140]: mon.a calling monitor election 2024-07-26T13:29:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:29:54 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-26T13:29:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:29:54 smithi167 ceph-mon[37140]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:29:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:29:54 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-26T13:29:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:29:54 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-26T13:29:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:29:54 smithi167 ceph-mon[37140]: mon.c calling monitor election 2024-07-26T13:29:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:29:54 smithi167 ceph-mon[37140]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:29:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:29:54 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-26T13:29:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:29:54 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-26T13:29:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:29:54 smithi167 ceph-mon[37140]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:29:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:29:54 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-26T13:29:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:29:54 smithi167 ceph-mon[37140]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-07-26T13:29:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:29:54 smithi167 ceph-mon[37140]: monmap epoch 2 2024-07-26T13:29:55.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:29:54 smithi167 ceph-mon[37140]: fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 2024-07-26T13:29:55.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:29:54 smithi167 ceph-mon[37140]: last_changed 2024-07-26T13:29:49.565432+0000 2024-07-26T13:29:55.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:29:54 smithi167 ceph-mon[37140]: created 2024-07-26T13:28:19.859615+0000 2024-07-26T13:29:55.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:29:54 smithi167 ceph-mon[37140]: min_mon_release 19 (squid) 2024-07-26T13:29:55.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:29:54 smithi167 ceph-mon[37140]: election_strategy: 1 2024-07-26T13:29:55.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:29:54 smithi167 ceph-mon[37140]: 0: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.a 2024-07-26T13:29:55.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:29:54 smithi167 ceph-mon[37140]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-07-26T13:29:55.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:29:54 smithi167 ceph-mon[37140]: fsmap 2024-07-26T13:29:55.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:29:54 smithi167 ceph-mon[37140]: osdmap e4: 0 total, 0 up, 0 in 2024-07-26T13:29:55.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:29:54 smithi167 ceph-mon[37140]: mgrmap e13: x(active, since 46s) 2024-07-26T13:29:55.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:29:54 smithi167 ceph-mon[37140]: overall HEALTH_OK 2024-07-26T13:29:55.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:29:54 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:55.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:29:54 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:55.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:29:54 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:55.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:29:54 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-26T13:29:55.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:29:54 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:29:55.274 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-07-26T13:29:55.274 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":2,"fsid":"bf83e2e6-4b52-11ef-bcb7-c7b262605968","modified":"2024-07-26T13:29:49.565432Z","created":"2024-07-26T13:28:19.859615Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:3300","nonce":0},{"type":"v1","addr":"172.21.15.100:6789","nonce":0}]},"addr":"172.21.15.100:6789/0","public_addr":"172.21.15.100:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:3300","nonce":0},{"type":"v1","addr":"172.21.15.167:6789","nonce":0}]},"addr":"172.21.15.167:6789/0","public_addr":"172.21.15.167:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-07-26T13:29:55.274 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 2 2024-07-26T13:29:55.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:29:55 smithi167 ceph-mon[37140]: Deploying daemon mon.b on smithi138 2024-07-26T13:29:55.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:29:55 smithi167 ceph-mon[37140]: from='client.? 172.21.15.167:0/2142453049' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-26T13:29:55.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:29:55 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-26T13:29:55.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:55 smithi100 ceph-mon[32599]: Deploying daemon mon.b on smithi138 2024-07-26T13:29:55.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:55 smithi100 ceph-mon[32599]: from='client.? 172.21.15.167:0/2142453049' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-26T13:29:55.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:55 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-26T13:29:56.855 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-07-26T13:29:56.855 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph mon dump -f json 2024-07-26T13:29:56.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:56 smithi100 ceph-mon[32599]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:29:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:29:56 smithi167 ceph-mon[37140]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:29:57.109 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.c/config 2024-07-26T13:29:58.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:29:58 smithi167 ceph-mon[37140]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:29:58.458 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-07-26T13:29:58.458 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-07-26T13:29:58.459 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-07-26T13:29:58.459 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-07-26T13:29:58.459 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-07-26T13:29:58.459 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-07-26T13:29:58.459 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-07-26T13:29:58.459 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-07-26T13:29:58.459 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-07-26T13:29:58.459 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-07-26T13:29:58.459 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: Deploying cephadm binary to smithi167 2024-07-26T13:29:58.459 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:29:58.459 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:29:58.459 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:58.459 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: Added host smithi167 2024-07-26T13:29:58.460 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:29:58.460 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:58.460 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:58.460 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:58.460 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:29:58.460 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-26T13:29:58.460 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:29:58.460 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:58.460 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:58.460 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:58.460 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:58.460 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi167", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:29:58.460 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:29:58.460 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:29:58.461 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: Updating smithi167:/etc/ceph/ceph.conf 2024-07-26T13:29:58.461 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: Updating smithi167:/var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/config/ceph.conf 2024-07-26T13:29:58.461 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2024-07-26T13:29:58.461 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/757579215' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-07-26T13:29:58.461 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:58.461 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:58.461 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:58.461 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: Updating smithi167:/var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/config/ceph.client.admin.keyring 2024-07-26T13:29:58.461 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/757579215' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-07-26T13:29:58.461 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: osdmap e4: 0 total, 0 up, 0 in 2024-07-26T13:29:58.461 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:29:58.461 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:29:58.461 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi100:172.21.15.100=a;smithi138:172.21.15.138=b;smithi167:172.21.15.167=c", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:29:58.462 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: Saving service mon spec with placement smithi100:172.21.15.100=a;smithi138:172.21.15.138=b;smithi167:172.21.15.167=c;count:3 2024-07-26T13:29:58.462 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:58.462 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:29:58.462 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:29:58.462 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:29:58.462 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:58.462 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-26T13:29:58.462 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:29:58.462 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: Deploying daemon mon.c on smithi167 2024-07-26T13:29:58.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:29:58 smithi100 ceph-mon[32599]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:29:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:29:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:29:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='client.? 172.21.15.167:0/1233631301' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-26T13:29:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-26T13:29:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: mon.a calling monitor election 2024-07-26T13:29:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-26T13:29:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:29:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-26T13:29:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-26T13:29:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: mon.c calling monitor election 2024-07-26T13:29:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:29:58.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-26T13:29:58.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-26T13:29:58.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:29:58.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-26T13:29:58.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-07-26T13:29:58.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: monmap epoch 2 2024-07-26T13:29:58.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 2024-07-26T13:29:58.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: last_changed 2024-07-26T13:29:49.565432+0000 2024-07-26T13:29:58.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: created 2024-07-26T13:28:19.859615+0000 2024-07-26T13:29:58.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: min_mon_release 19 (squid) 2024-07-26T13:29:58.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: election_strategy: 1 2024-07-26T13:29:58.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: 0: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.a 2024-07-26T13:29:58.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-07-26T13:29:58.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: fsmap 2024-07-26T13:29:58.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: osdmap e4: 0 total, 0 up, 0 in 2024-07-26T13:29:58.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: mgrmap e13: x(active, since 46s) 2024-07-26T13:29:58.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: overall HEALTH_OK 2024-07-26T13:29:58.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:58.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:58.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:29:58.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-26T13:29:58.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:29:58.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: Deploying daemon mon.b on smithi138 2024-07-26T13:29:58.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='client.? 172.21.15.167:0/2142453049' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-26T13:29:58.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-26T13:29:58.944 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:29:58.944 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:29:58.944 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 ceph-mon[36954]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-07-26T13:29:58.944 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 bash[36935]: 258042bba51c1021d3edde0632c3e548ce465630d2b02b4de0b4a30ef866df67 2024-07-26T13:29:58.944 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:29:58 smithi138 systemd[1]: Started Ceph mon.b for bf83e2e6-4b52-11ef-bcb7-c7b262605968. 2024-07-26T13:30:03.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:03 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-26T13:30:03.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:03 smithi100 ceph-mon[32599]: mon.a calling monitor election 2024-07-26T13:30:03.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:03 smithi100 ceph-mon[32599]: mon.c calling monitor election 2024-07-26T13:30:03.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:03 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-26T13:30:03.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:03 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-26T13:30:03.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:03 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-26T13:30:03.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:03 smithi100 ceph-mon[32599]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:03.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:03 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-26T13:30:03.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:03 smithi100 ceph-mon[32599]: mon.b calling monitor election 2024-07-26T13:30:03.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:03 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-26T13:30:03.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:03 smithi100 ceph-mon[32599]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:03.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:03 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-26T13:30:03.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:03 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-26T13:30:03.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:03 smithi100 ceph-mon[32599]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-07-26T13:30:03.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:03 smithi100 ceph-mon[32599]: monmap epoch 3 2024-07-26T13:30:03.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:03 smithi100 ceph-mon[32599]: fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 2024-07-26T13:30:03.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:03 smithi100 ceph-mon[32599]: last_changed 2024-07-26T13:29:58.463268+0000 2024-07-26T13:30:03.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:03 smithi100 ceph-mon[32599]: created 2024-07-26T13:28:19.859615+0000 2024-07-26T13:30:03.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:03 smithi100 ceph-mon[32599]: min_mon_release 19 (squid) 2024-07-26T13:30:03.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:03 smithi100 ceph-mon[32599]: election_strategy: 1 2024-07-26T13:30:03.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:03 smithi100 ceph-mon[32599]: 0: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.a 2024-07-26T13:30:03.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:03 smithi100 ceph-mon[32599]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-07-26T13:30:03.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:03 smithi100 ceph-mon[32599]: 2: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.b 2024-07-26T13:30:03.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:03 smithi100 ceph-mon[32599]: fsmap 2024-07-26T13:30:03.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:03 smithi100 ceph-mon[32599]: osdmap e4: 0 total, 0 up, 0 in 2024-07-26T13:30:03.905 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:03 smithi100 ceph-mon[32599]: mgrmap e13: x(active, since 55s) 2024-07-26T13:30:03.905 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:03 smithi100 ceph-mon[32599]: overall HEALTH_OK 2024-07-26T13:30:03.905 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:03 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:03.905 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:03 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:03.905 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:03 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:03.905 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:03 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:03.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:03 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-26T13:30:03.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:03 smithi167 ceph-mon[37140]: mon.a calling monitor election 2024-07-26T13:30:03.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:03 smithi167 ceph-mon[37140]: mon.c calling monitor election 2024-07-26T13:30:03.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:03 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-26T13:30:03.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:03 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-26T13:30:03.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:03 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-26T13:30:03.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:03 smithi167 ceph-mon[37140]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:03.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:03 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-26T13:30:03.930 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:03 smithi167 ceph-mon[37140]: mon.b calling monitor election 2024-07-26T13:30:03.930 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:03 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-26T13:30:03.930 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:03 smithi167 ceph-mon[37140]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:03.930 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:03 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-26T13:30:03.930 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:03 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-26T13:30:03.930 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:03 smithi167 ceph-mon[37140]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-07-26T13:30:03.931 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:03 smithi167 ceph-mon[37140]: monmap epoch 3 2024-07-26T13:30:03.931 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:03 smithi167 ceph-mon[37140]: fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 2024-07-26T13:30:03.931 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:03 smithi167 ceph-mon[37140]: last_changed 2024-07-26T13:29:58.463268+0000 2024-07-26T13:30:03.931 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:03 smithi167 ceph-mon[37140]: created 2024-07-26T13:28:19.859615+0000 2024-07-26T13:30:03.931 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:03 smithi167 ceph-mon[37140]: min_mon_release 19 (squid) 2024-07-26T13:30:03.931 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:03 smithi167 ceph-mon[37140]: election_strategy: 1 2024-07-26T13:30:03.931 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:03 smithi167 ceph-mon[37140]: 0: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.a 2024-07-26T13:30:03.931 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:03 smithi167 ceph-mon[37140]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-07-26T13:30:03.932 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:03 smithi167 ceph-mon[37140]: 2: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.b 2024-07-26T13:30:03.932 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:03 smithi167 ceph-mon[37140]: fsmap 2024-07-26T13:30:03.932 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:03 smithi167 ceph-mon[37140]: osdmap e4: 0 total, 0 up, 0 in 2024-07-26T13:30:03.932 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:03 smithi167 ceph-mon[37140]: mgrmap e13: x(active, since 55s) 2024-07-26T13:30:03.932 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:03 smithi167 ceph-mon[37140]: overall HEALTH_OK 2024-07-26T13:30:03.932 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:03 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:03.932 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:03 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:03.933 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:03 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:03.933 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:03 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:03.933 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:03 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-26T13:30:03.933 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:03 smithi138 ceph-mon[36954]: mon.a calling monitor election 2024-07-26T13:30:03.934 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:03 smithi138 ceph-mon[36954]: mon.c calling monitor election 2024-07-26T13:30:03.934 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:03 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-26T13:30:03.934 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:03 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-26T13:30:03.934 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:03 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-26T13:30:03.934 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:03 smithi138 ceph-mon[36954]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:03.934 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:03 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-26T13:30:03.935 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:03 smithi138 ceph-mon[36954]: mon.b calling monitor election 2024-07-26T13:30:03.935 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:03 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-26T13:30:03.935 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:03 smithi138 ceph-mon[36954]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:03.935 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:03 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-26T13:30:03.935 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:03 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-26T13:30:03.935 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:03 smithi138 ceph-mon[36954]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-07-26T13:30:03.935 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:03 smithi138 ceph-mon[36954]: monmap epoch 3 2024-07-26T13:30:03.935 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:03 smithi138 ceph-mon[36954]: fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 2024-07-26T13:30:03.936 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:03 smithi138 ceph-mon[36954]: last_changed 2024-07-26T13:29:58.463268+0000 2024-07-26T13:30:03.936 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:03 smithi138 ceph-mon[36954]: created 2024-07-26T13:28:19.859615+0000 2024-07-26T13:30:03.936 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:03 smithi138 ceph-mon[36954]: min_mon_release 19 (squid) 2024-07-26T13:30:03.936 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:03 smithi138 ceph-mon[36954]: election_strategy: 1 2024-07-26T13:30:03.936 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:03 smithi138 ceph-mon[36954]: 0: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.a 2024-07-26T13:30:03.936 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:03 smithi138 ceph-mon[36954]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-07-26T13:30:03.937 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:03 smithi138 ceph-mon[36954]: 2: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.b 2024-07-26T13:30:03.937 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:03 smithi138 ceph-mon[36954]: fsmap 2024-07-26T13:30:03.937 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:03 smithi138 ceph-mon[36954]: osdmap e4: 0 total, 0 up, 0 in 2024-07-26T13:30:03.937 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:03 smithi138 ceph-mon[36954]: mgrmap e13: x(active, since 55s) 2024-07-26T13:30:03.937 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:03 smithi138 ceph-mon[36954]: overall HEALTH_OK 2024-07-26T13:30:03.937 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:03 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:03.938 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:03 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:03.938 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:03 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:03.938 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:03 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:04.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:04 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:30:04.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:04 smithi138 ceph-mon[36954]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:04.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:04 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-26T13:30:04.790 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-07-26T13:30:04.790 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":3,"fsid":"bf83e2e6-4b52-11ef-bcb7-c7b262605968","modified":"2024-07-26T13:29:58.463268Z","created":"2024-07-26T13:28:19.859615Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:3300","nonce":0},{"type":"v1","addr":"172.21.15.100:6789","nonce":0}]},"addr":"172.21.15.100:6789/0","public_addr":"172.21.15.100:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:3300","nonce":0},{"type":"v1","addr":"172.21.15.167:6789","nonce":0}]},"addr":"172.21.15.167:6789/0","public_addr":"172.21.15.167:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:3300","nonce":0},{"type":"v1","addr":"172.21.15.138:6789","nonce":0}]},"addr":"172.21.15.138:6789/0","public_addr":"172.21.15.138:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-07-26T13:30:04.791 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 3 2024-07-26T13:30:04.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:04 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:30:04.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:04 smithi167 ceph-mon[37140]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:04.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:04 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-26T13:30:04.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:04 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:30:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:04 smithi100 ceph-mon[32599]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:04 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-26T13:30:05.444 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-07-26T13:30:05.444 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph config generate-minimal-conf 2024-07-26T13:30:05.723 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:05 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.724 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:05 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.724 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:05 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.724 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:05 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.724 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:05 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:05.724 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:05 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:30:05.724 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:05 smithi100 ceph-mon[32599]: Updating smithi100:/etc/ceph/ceph.conf 2024-07-26T13:30:05.724 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:05 smithi100 ceph-mon[32599]: Updating smithi138:/etc/ceph/ceph.conf 2024-07-26T13:30:05.724 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:05 smithi100 ceph-mon[32599]: Updating smithi167:/etc/ceph/ceph.conf 2024-07-26T13:30:05.724 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:05 smithi100 ceph-mon[32599]: from='client.? 172.21.15.167:0/1290178342' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-26T13:30:05.724 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:05 smithi100 ceph-mon[32599]: Updating smithi138:/var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/config/ceph.conf 2024-07-26T13:30:05.724 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:05 smithi100 ceph-mon[32599]: Updating smithi100:/var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/config/ceph.conf 2024-07-26T13:30:05.724 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:05 smithi100 ceph-mon[32599]: Updating smithi167:/var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/config/ceph.conf 2024-07-26T13:30:05.724 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:05 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.725 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:05 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.725 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:05 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.725 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:05 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.725 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:05 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.725 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:05 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.725 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:05 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.725 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:05 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.725 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:05 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.725 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:05 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.725 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:05 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.725 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:05 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-26T13:30:05.725 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:05 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-26T13:30:05.725 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:05 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:05.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:05 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:05 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:05 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:05 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:05 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:05.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:05 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:30:05.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:05 smithi167 ceph-mon[37140]: Updating smithi100:/etc/ceph/ceph.conf 2024-07-26T13:30:05.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:05 smithi167 ceph-mon[37140]: Updating smithi138:/etc/ceph/ceph.conf 2024-07-26T13:30:05.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:05 smithi167 ceph-mon[37140]: Updating smithi167:/etc/ceph/ceph.conf 2024-07-26T13:30:05.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:05 smithi167 ceph-mon[37140]: from='client.? 172.21.15.167:0/1290178342' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-26T13:30:05.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:05 smithi167 ceph-mon[37140]: Updating smithi138:/var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/config/ceph.conf 2024-07-26T13:30:05.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:05 smithi167 ceph-mon[37140]: Updating smithi100:/var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/config/ceph.conf 2024-07-26T13:30:05.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:05 smithi167 ceph-mon[37140]: Updating smithi167:/var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/config/ceph.conf 2024-07-26T13:30:05.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:05 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:05 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:05 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:05 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:05 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:05 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:05 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:05 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:05 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:05 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:05 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:05 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-26T13:30:05.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:05 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-26T13:30:05.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:05 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:05 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:05 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:05 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:05 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:05 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:05.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:05 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:30:05.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:05 smithi138 ceph-mon[36954]: Updating smithi100:/etc/ceph/ceph.conf 2024-07-26T13:30:05.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:05 smithi138 ceph-mon[36954]: Updating smithi138:/etc/ceph/ceph.conf 2024-07-26T13:30:05.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:05 smithi138 ceph-mon[36954]: Updating smithi167:/etc/ceph/ceph.conf 2024-07-26T13:30:05.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:05 smithi138 ceph-mon[36954]: from='client.? 172.21.15.167:0/1290178342' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-26T13:30:05.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:05 smithi138 ceph-mon[36954]: Updating smithi138:/var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/config/ceph.conf 2024-07-26T13:30:05.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:05 smithi138 ceph-mon[36954]: Updating smithi100:/var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/config/ceph.conf 2024-07-26T13:30:05.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:05 smithi138 ceph-mon[36954]: Updating smithi167:/var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/config/ceph.conf 2024-07-26T13:30:05.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:05 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:05 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:05 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:05 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:05 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:05 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:05 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:05 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:05 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:05 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:05 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:05.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:05 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-26T13:30:05.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:05 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-26T13:30:05.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:05 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:06.477 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:30:06.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:06 smithi100 ceph-mon[32599]: Reconfiguring mon.a (unknown last config time)... 2024-07-26T13:30:06.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:06 smithi100 ceph-mon[32599]: Reconfiguring daemon mon.a on smithi100 2024-07-26T13:30:06.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:06 smithi100 ceph-mon[32599]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:06.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:06 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:06.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:06 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:06.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:06 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-26T13:30:06.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:06 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-26T13:30:06.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:06 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:06.878 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:06 smithi138 ceph-mon[36954]: Reconfiguring mon.a (unknown last config time)... 2024-07-26T13:30:06.878 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:06 smithi138 ceph-mon[36954]: Reconfiguring daemon mon.a on smithi100 2024-07-26T13:30:06.878 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:06 smithi138 ceph-mon[36954]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:06.878 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:06 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:06.878 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:06 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:06.878 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:06 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-26T13:30:06.879 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:06 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-26T13:30:06.879 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:06 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:06 smithi167 ceph-mon[37140]: Reconfiguring mon.a (unknown last config time)... 2024-07-26T13:30:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:06 smithi167 ceph-mon[37140]: Reconfiguring daemon mon.a on smithi100 2024-07-26T13:30:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:06 smithi167 ceph-mon[37140]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:06 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:06 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:06 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-26T13:30:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:06 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-26T13:30:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:06 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:07.782 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:07 smithi100 ceph-mon[32599]: Reconfiguring mon.b (monmap changed)... 2024-07-26T13:30:07.782 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:07 smithi100 ceph-mon[32599]: Reconfiguring daemon mon.b on smithi138 2024-07-26T13:30:07.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:07 smithi138 ceph-mon[36954]: Reconfiguring mon.b (monmap changed)... 2024-07-26T13:30:07.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:07 smithi138 ceph-mon[36954]: Reconfiguring daemon mon.b on smithi138 2024-07-26T13:30:08.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:07 smithi167 ceph-mon[37140]: Reconfiguring mon.b (monmap changed)... 2024-07-26T13:30:08.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:07 smithi167 ceph-mon[37140]: Reconfiguring daemon mon.b on smithi138 2024-07-26T13:30:08.196 INFO:teuthology.orchestra.run.smithi100.stdout:# minimal ceph.conf for bf83e2e6-4b52-11ef-bcb7-c7b262605968 2024-07-26T13:30:08.196 INFO:teuthology.orchestra.run.smithi100.stdout:[global] 2024-07-26T13:30:08.196 INFO:teuthology.orchestra.run.smithi100.stdout: fsid = bf83e2e6-4b52-11ef-bcb7-c7b262605968 2024-07-26T13:30:08.196 INFO:teuthology.orchestra.run.smithi100.stdout: mon_host = [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] 2024-07-26T13:30:08.971 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:08 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:08.971 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:08 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:08.971 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:08 smithi100 ceph-mon[32599]: Reconfiguring mon.c (monmap changed)... 2024-07-26T13:30:08.971 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:08 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-26T13:30:08.972 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:08 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-26T13:30:08.972 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:08 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:08.972 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:08 smithi100 ceph-mon[32599]: Reconfiguring daemon mon.c on smithi167 2024-07-26T13:30:08.972 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:08 smithi100 ceph-mon[32599]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:08.972 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:08 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:08.972 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:08 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2961905728' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:09.004 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-07-26T13:30:09.005 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-07-26T13:30:09.005 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.conf 2024-07-26T13:30:09.037 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-07-26T13:30:09.037 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-07-26T13:30:09.109 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-07-26T13:30:09.109 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.conf 2024-07-26T13:30:09.138 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:08 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:09.138 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:08 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:09.138 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:08 smithi138 ceph-mon[36954]: Reconfiguring mon.c (monmap changed)... 2024-07-26T13:30:09.138 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:08 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-26T13:30:09.138 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:08 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-26T13:30:09.138 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:08 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:09.138 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:08 smithi138 ceph-mon[36954]: Reconfiguring daemon mon.c on smithi167 2024-07-26T13:30:09.138 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:08 smithi138 ceph-mon[36954]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:09.138 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:08 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:09.139 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:08 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/2961905728' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:09.141 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-07-26T13:30:09.141 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-07-26T13:30:09.172 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:08 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:09.172 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:08 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:09.172 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:08 smithi167 ceph-mon[37140]: Reconfiguring mon.c (monmap changed)... 2024-07-26T13:30:09.173 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:08 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-26T13:30:09.173 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:08 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-26T13:30:09.173 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:08 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:09.173 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:08 smithi167 ceph-mon[37140]: Reconfiguring daemon mon.c on smithi167 2024-07-26T13:30:09.173 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:08 smithi167 ceph-mon[37140]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:09.173 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:08 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:09.173 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:08 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/2961905728' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:09.210 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-07-26T13:30:09.210 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.conf 2024-07-26T13:30:09.238 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-07-26T13:30:09.238 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-07-26T13:30:09.302 INFO:tasks.cephadm:enabling module snap_schedule 2024-07-26T13:30:09.302 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- sudo ceph --cluster ceph mgr module enable snap_schedule 2024-07-26T13:30:09.546 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:30:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:10 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:10 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:10.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:10 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:30:10.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:10 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:10.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:10 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:30:10.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:10 smithi100 ceph-mon[32599]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:10.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:10 smithi100 ceph-mon[32599]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:10.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:10 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:10.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:10 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:10.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:10 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:30:10.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:10 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:10.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:10 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:30:10.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:10 smithi167 ceph-mon[37140]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:10.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:10 smithi167 ceph-mon[37140]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:10 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:10 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:10 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:30:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:10 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:10 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:30:10.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:10 smithi138 ceph-mon[36954]: from='mgr.14150 172.21.15.100:0/1129746696' entity='mgr.x' 2024-07-26T13:30:10.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:10 smithi138 ceph-mon[36954]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:11.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:11 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/1589965177' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-07-26T13:30:11.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:11 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/1589965177' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-07-26T13:30:11.744 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:30:11 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: ignoring --setuser ceph since I am not root 2024-07-26T13:30:11.744 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:30:11 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: ignoring --setgroup ceph since I am not root 2024-07-26T13:30:11.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:11 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/1589965177' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-07-26T13:30:11.971 INFO:tasks.cephadm:Adding mgr.x on smithi100 2024-07-26T13:30:11.971 INFO:tasks.cephadm:Adding mgr.y on smithi138 2024-07-26T13:30:11.971 INFO:tasks.cephadm:Adding mgr.z on smithi167 2024-07-26T13:30:11.971 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph orch apply mgr '3;smithi100=x;smithi138=y;smithi167=z' 2024-07-26T13:30:12.227 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.c/config 2024-07-26T13:30:12.650 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:12 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/1589965177' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-07-26T13:30:12.650 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:12 smithi167 ceph-mon[37140]: mgrmap e14: x(active, since 63s) 2024-07-26T13:30:12.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:12 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/1589965177' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-07-26T13:30:12.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:12 smithi138 ceph-mon[36954]: mgrmap e14: x(active, since 63s) 2024-07-26T13:30:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:12 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/1589965177' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-07-26T13:30:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:12 smithi100 ceph-mon[32599]: mgrmap e14: x(active, since 63s) 2024-07-26T13:30:16.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:16 smithi167 ceph-mon[37140]: Active manager daemon x restarted 2024-07-26T13:30:16.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:16 smithi167 ceph-mon[37140]: Activating manager daemon x 2024-07-26T13:30:16.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:16 smithi167 ceph-mon[37140]: osdmap e5: 0 total, 0 up, 0 in 2024-07-26T13:30:16.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:16 smithi167 ceph-mon[37140]: mgrmap e15: x(active, starting, since 0.00472888s) 2024-07-26T13:30:16.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-26T13:30:16.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-26T13:30:16.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-26T13:30:16.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-07-26T13:30:16.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-26T13:30:16.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-26T13:30:16.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-26T13:30:16.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:16 smithi167 ceph-mon[37140]: Manager daemon x is now available 2024-07-26T13:30:16.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:30:16.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-07-26T13:30:16.680 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:16 smithi138 ceph-mon[36954]: Active manager daemon x restarted 2024-07-26T13:30:16.680 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:16 smithi138 ceph-mon[36954]: Activating manager daemon x 2024-07-26T13:30:16.680 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:16 smithi138 ceph-mon[36954]: osdmap e5: 0 total, 0 up, 0 in 2024-07-26T13:30:16.680 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:16 smithi138 ceph-mon[36954]: mgrmap e15: x(active, starting, since 0.00472888s) 2024-07-26T13:30:16.680 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-26T13:30:16.681 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-26T13:30:16.681 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-26T13:30:16.681 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-07-26T13:30:16.681 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-26T13:30:16.681 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-26T13:30:16.681 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-26T13:30:16.681 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:16 smithi138 ceph-mon[36954]: Manager daemon x is now available 2024-07-26T13:30:16.681 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:30:16.681 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-07-26T13:30:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:16 smithi100 ceph-mon[32599]: Active manager daemon x restarted 2024-07-26T13:30:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:16 smithi100 ceph-mon[32599]: Activating manager daemon x 2024-07-26T13:30:16.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:16 smithi100 ceph-mon[32599]: osdmap e5: 0 total, 0 up, 0 in 2024-07-26T13:30:16.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:16 smithi100 ceph-mon[32599]: mgrmap e15: x(active, starting, since 0.00472888s) 2024-07-26T13:30:16.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-26T13:30:16.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-26T13:30:16.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-26T13:30:16.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-07-26T13:30:16.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-26T13:30:16.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-26T13:30:16.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-26T13:30:16.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:16 smithi100 ceph-mon[32599]: Manager daemon x is now available 2024-07-26T13:30:16.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:30:16.747 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-07-26T13:30:17.271 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled mgr update... 2024-07-26T13:30:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-07-26T13:30:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:17 smithi100 ceph-mon[32599]: mgrmap e16: x(active, since 1.01014s) 2024-07-26T13:30:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:17.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-07-26T13:30:17.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:17 smithi167 ceph-mon[37140]: mgrmap e16: x(active, since 1.01014s) 2024-07-26T13:30:17.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-07-26T13:30:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:17 smithi138 ceph-mon[36954]: mgrmap e16: x(active, since 1.01014s) 2024-07-26T13:30:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:17.822 DEBUG:teuthology.orchestra.run.smithi138:mgr.y> sudo journalctl -f -n 0 -u ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mgr.y.service 2024-07-26T13:30:17.825 DEBUG:teuthology.orchestra.run.smithi167:mgr.z> sudo journalctl -f -n 0 -u ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mgr.z.service 2024-07-26T13:30:17.827 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-07-26T13:30:17.827 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph config set client client_mount_timeout 600 2024-07-26T13:30:17.829 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-07-26T13:30:17.830 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph config set client debug_client 20 2024-07-26T13:30:17.872 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-07-26T13:30:17.873 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph config set client debug_ms 1 2024-07-26T13:30:17.916 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-07-26T13:30:17.917 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph config set client rados_mon_op_timeout 900 2024-07-26T13:30:17.959 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-07-26T13:30:17.960 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph config set client rados_osd_op_timeout 900 2024-07-26T13:30:18.003 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-07-26T13:30:18.003 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph config set mds debug_mds 20 2024-07-26T13:30:18.045 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-07-26T13:30:18.046 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph config set mds debug_mds_balancer 20 2024-07-26T13:30:18.079 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:30:17 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: 2024-07-26T13:30:17.811+0000 7f9e4aeeb640 -1 log_channel(cephadm) log [ERR] : [26/Jul/2024:13:30:17] ENGINE Error in HTTPServer.serve 2024-07-26T13:30:18.079 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:30:17 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: Traceback (most recent call last): 2024-07-26T13:30:18.079 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:30:17 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-26T13:30:18.079 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:30:17 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: self._connections.run(self.expiration_interval) 2024-07-26T13:30:18.079 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:30:17 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-26T13:30:18.079 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:30:17 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: self._run(expiration_interval) 2024-07-26T13:30:18.079 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:30:17 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-26T13:30:18.079 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:30:17 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: new_conn = self._from_server_socket(self.server.socket) 2024-07-26T13:30:18.080 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:30:17 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-26T13:30:18.080 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:30:17 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-26T13:30:18.080 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:30:17 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-26T13:30:18.080 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:30:17 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: s = self.context.wrap_socket( 2024-07-26T13:30:18.080 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:30:17 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-26T13:30:18.080 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:30:17 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: return self.sslsocket_class._create( 2024-07-26T13:30:18.080 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:30:17 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-26T13:30:18.080 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:30:17 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: self.do_handshake() 2024-07-26T13:30:18.080 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:30:17 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-26T13:30:18.080 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:30:17 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: self._sslobj.do_handshake() 2024-07-26T13:30:18.080 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:30:17 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-26T13:30:18.080 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:30:17 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: 2024-07-26T13:30:18.081 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-07-26T13:30:18.081 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph config set mds debug_ms 2 2024-07-26T13:30:18.125 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2024-07-26T13:30:18.125 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph config set mds mds_bal_fragment_size_max 10000 2024-07-26T13:30:18.170 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2024-07-26T13:30:18.170 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph config set mds mds_bal_merge_size 5 2024-07-26T13:30:18.210 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:30:18.214 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2024-07-26T13:30:18.214 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph config set mds mds_bal_split_bits 3 2024-07-26T13:30:18.265 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2024-07-26T13:30:18.265 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph config set mds mds_bal_split_size 100 2024-07-26T13:30:18.311 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-07-26T13:30:18.311 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph config set mds mds_debug_frag True 2024-07-26T13:30:18.328 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:18 smithi100 ceph-mon[32599]: Saving service mgr spec with placement smithi100=x;smithi138=y;smithi167=z;count:3 2024-07-26T13:30:18.328 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:18 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:18.328 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:18 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:18.328 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:18 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:18.328 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:18 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:18.328 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:18 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:18.329 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:18 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:18.334 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-07-26T13:30:18.335 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph config set mds mds_debug_scatterstat True 2024-07-26T13:30:18.344 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:30:18.361 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-07-26T13:30:18.361 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph config set mds mds_op_complaint_time 180 2024-07-26T13:30:18.362 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:30:18.420 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-07-26T13:30:18.420 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph config set mds mds_verify_scatter True 2024-07-26T13:30:18.464 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-07-26T13:30:18.464 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph config set mds osd_op_complaint_time 180 2024-07-26T13:30:18.517 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-07-26T13:30:18.517 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph config set mds rados_mon_op_timeout 900 2024-07-26T13:30:18.570 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-07-26T13:30:18.570 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph config set mds rados_osd_op_timeout 900 2024-07-26T13:30:18.622 INFO:tasks.cephadm:setting config [mgr] debug_cephsqlite = 20 2024-07-26T13:30:18.622 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph config set mgr debug_cephsqlite 20 2024-07-26T13:30:18.672 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/allow_m_granularity = True 2024-07-26T13:30:18.672 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-07-26T13:30:18.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:18 smithi138 ceph-mon[36954]: Saving service mgr spec with placement smithi100=x;smithi138=y;smithi167=z;count:3 2024-07-26T13:30:18.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:18 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:18.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:18 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:18.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:18 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:18.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:18 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:18.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:18 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:18.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:18 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:18.745 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/dump_on_update = True 2024-07-26T13:30:18.745 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-07-26T13:30:18.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:18 smithi167 ceph-mon[37140]: Saving service mgr spec with placement smithi100=x;smithi138=y;smithi167=z;count:3 2024-07-26T13:30:18.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:18 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:18.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:18 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:18.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:18 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:18.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:18 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:18 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:18 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:18.791 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/log_level = debug 2024-07-26T13:30:18.792 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph config set mgr mgr/snap_schedule/log_level debug 2024-07-26T13:30:18.861 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-07-26T13:30:18.861 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph config set mon mon_op_complaint_time 120 2024-07-26T13:30:18.910 INFO:tasks.cephadm:setting config [mon] mon_warn_on_pool_no_app = False 2024-07-26T13:30:18.910 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph config set mon mon_warn_on_pool_no_app False 2024-07-26T13:30:18.964 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-07-26T13:30:18.964 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph config set osd osd_op_complaint_time 180 2024-07-26T13:30:19.012 DEBUG:tasks.cephadm:set 26 configs 2024-07-26T13:30:19.012 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'bf83e2e6-4b52-11ef-bcb7-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi100') 2024-07-26T13:30:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:19 smithi138 ceph-mon[36954]: [26/Jul/2024:13:30:17] ENGINE Bus STARTING 2024-07-26T13:30:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:19 smithi138 ceph-mon[36954]: [26/Jul/2024:13:30:17] ENGINE Serving on http://172.21.15.100:8765 2024-07-26T13:30:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:19 smithi138 ceph-mon[36954]: [26/Jul/2024:13:30:17] ENGINE Serving on https://172.21.15.100:7150 2024-07-26T13:30:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:19 smithi138 ceph-mon[36954]: [26/Jul/2024:13:30:17] ENGINE Bus STARTED 2024-07-26T13:30:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:19 smithi138 ceph-mon[36954]: [26/Jul/2024:13:30:17] ENGINE Error in HTTPServer.serve 2024-07-26T13:30:19.691 INFO:journalctl@ceph.mon.b.smithi138.stdout: Traceback (most recent call last): 2024-07-26T13:30:19.691 INFO:journalctl@ceph.mon.b.smithi138.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-26T13:30:19.691 INFO:journalctl@ceph.mon.b.smithi138.stdout: self._connections.run(self.expiration_interval) 2024-07-26T13:30:19.691 INFO:journalctl@ceph.mon.b.smithi138.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-26T13:30:19.691 INFO:journalctl@ceph.mon.b.smithi138.stdout: self._run(expiration_interval) 2024-07-26T13:30:19.691 INFO:journalctl@ceph.mon.b.smithi138.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-26T13:30:19.691 INFO:journalctl@ceph.mon.b.smithi138.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-26T13:30:19.691 INFO:journalctl@ceph.mon.b.smithi138.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-26T13:30:19.691 INFO:journalctl@ceph.mon.b.smithi138.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-26T13:30:19.691 INFO:journalctl@ceph.mon.b.smithi138.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-26T13:30:19.691 INFO:journalctl@ceph.mon.b.smithi138.stdout: s = self.context.wrap_socket( 2024-07-26T13:30:19.691 INFO:journalctl@ceph.mon.b.smithi138.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-26T13:30:19.692 INFO:journalctl@ceph.mon.b.smithi138.stdout: return self.sslsocket_class._create( 2024-07-26T13:30:19.692 INFO:journalctl@ceph.mon.b.smithi138.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-26T13:30:19.692 INFO:journalctl@ceph.mon.b.smithi138.stdout: self.do_handshake() 2024-07-26T13:30:19.692 INFO:journalctl@ceph.mon.b.smithi138.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-26T13:30:19.692 INFO:journalctl@ceph.mon.b.smithi138.stdout: self._sslobj.do_handshake() 2024-07-26T13:30:19.692 INFO:journalctl@ceph.mon.b.smithi138.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-26T13:30:19.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:19.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:19.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi138", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:30:19.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:19 smithi138 ceph-mon[36954]: mgrmap e17: x(active, since 2s) 2024-07-26T13:30:19.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:19.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:19.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi167", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:30:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:19 smithi100 ceph-mon[32599]: [26/Jul/2024:13:30:17] ENGINE Bus STARTING 2024-07-26T13:30:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:19 smithi100 ceph-mon[32599]: [26/Jul/2024:13:30:17] ENGINE Serving on http://172.21.15.100:8765 2024-07-26T13:30:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:19 smithi100 ceph-mon[32599]: [26/Jul/2024:13:30:17] ENGINE Serving on https://172.21.15.100:7150 2024-07-26T13:30:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:19 smithi100 ceph-mon[32599]: [26/Jul/2024:13:30:17] ENGINE Bus STARTED 2024-07-26T13:30:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:19 smithi100 ceph-mon[32599]: [26/Jul/2024:13:30:17] ENGINE Error in HTTPServer.serve 2024-07-26T13:30:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout: Traceback (most recent call last): 2024-07-26T13:30:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-26T13:30:19.745 INFO:journalctl@ceph.mon.a.smithi100.stdout: self._connections.run(self.expiration_interval) 2024-07-26T13:30:19.745 INFO:journalctl@ceph.mon.a.smithi100.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-26T13:30:19.745 INFO:journalctl@ceph.mon.a.smithi100.stdout: self._run(expiration_interval) 2024-07-26T13:30:19.745 INFO:journalctl@ceph.mon.a.smithi100.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-26T13:30:19.745 INFO:journalctl@ceph.mon.a.smithi100.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-26T13:30:19.745 INFO:journalctl@ceph.mon.a.smithi100.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-26T13:30:19.745 INFO:journalctl@ceph.mon.a.smithi100.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-26T13:30:19.745 INFO:journalctl@ceph.mon.a.smithi100.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-26T13:30:19.745 INFO:journalctl@ceph.mon.a.smithi100.stdout: s = self.context.wrap_socket( 2024-07-26T13:30:19.745 INFO:journalctl@ceph.mon.a.smithi100.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-26T13:30:19.745 INFO:journalctl@ceph.mon.a.smithi100.stdout: return self.sslsocket_class._create( 2024-07-26T13:30:19.745 INFO:journalctl@ceph.mon.a.smithi100.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-26T13:30:19.746 INFO:journalctl@ceph.mon.a.smithi100.stdout: self.do_handshake() 2024-07-26T13:30:19.746 INFO:journalctl@ceph.mon.a.smithi100.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-26T13:30:19.746 INFO:journalctl@ceph.mon.a.smithi100.stdout: self._sslobj.do_handshake() 2024-07-26T13:30:19.746 INFO:journalctl@ceph.mon.a.smithi100.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-26T13:30:19.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:19.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:19.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi138", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:30:19.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:19 smithi100 ceph-mon[32599]: mgrmap e17: x(active, since 2s) 2024-07-26T13:30:19.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:19.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:19.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi167", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:30:19.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:19 smithi167 ceph-mon[37140]: [26/Jul/2024:13:30:17] ENGINE Bus STARTING 2024-07-26T13:30:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:19 smithi167 ceph-mon[37140]: [26/Jul/2024:13:30:17] ENGINE Serving on http://172.21.15.100:8765 2024-07-26T13:30:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:19 smithi167 ceph-mon[37140]: [26/Jul/2024:13:30:17] ENGINE Serving on https://172.21.15.100:7150 2024-07-26T13:30:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:19 smithi167 ceph-mon[37140]: [26/Jul/2024:13:30:17] ENGINE Bus STARTED 2024-07-26T13:30:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:19 smithi167 ceph-mon[37140]: [26/Jul/2024:13:30:17] ENGINE Error in HTTPServer.serve 2024-07-26T13:30:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout: Traceback (most recent call last): 2024-07-26T13:30:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-26T13:30:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout: self._connections.run(self.expiration_interval) 2024-07-26T13:30:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-26T13:30:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout: self._run(expiration_interval) 2024-07-26T13:30:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-26T13:30:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-26T13:30:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-26T13:30:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-26T13:30:19.774 INFO:journalctl@ceph.mon.c.smithi167.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-26T13:30:19.774 INFO:journalctl@ceph.mon.c.smithi167.stdout: s = self.context.wrap_socket( 2024-07-26T13:30:19.774 INFO:journalctl@ceph.mon.c.smithi167.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-26T13:30:19.774 INFO:journalctl@ceph.mon.c.smithi167.stdout: return self.sslsocket_class._create( 2024-07-26T13:30:19.774 INFO:journalctl@ceph.mon.c.smithi167.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-26T13:30:19.774 INFO:journalctl@ceph.mon.c.smithi167.stdout: self.do_handshake() 2024-07-26T13:30:19.774 INFO:journalctl@ceph.mon.c.smithi167.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-26T13:30:19.774 INFO:journalctl@ceph.mon.c.smithi167.stdout: self._sslobj.do_handshake() 2024-07-26T13:30:19.774 INFO:journalctl@ceph.mon.c.smithi167.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-26T13:30:19.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:19.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:19.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi138", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:30:19.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:19 smithi167 ceph-mon[37140]: mgrmap e17: x(active, since 2s) 2024-07-26T13:30:19.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:19.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:19.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi167", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:30:22.663 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:22.663 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:22.664 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi100", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:30:22.664 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:22.664 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:30:22.664 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:22 smithi100 ceph-mon[32599]: Updating smithi100:/etc/ceph/ceph.conf 2024-07-26T13:30:22.664 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:22 smithi100 ceph-mon[32599]: Updating smithi138:/etc/ceph/ceph.conf 2024-07-26T13:30:22.664 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:22 smithi100 ceph-mon[32599]: Updating smithi167:/etc/ceph/ceph.conf 2024-07-26T13:30:22.664 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:22 smithi100 ceph-mon[32599]: Updating smithi100:/var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/config/ceph.conf 2024-07-26T13:30:22.664 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:22 smithi100 ceph-mon[32599]: Updating smithi138:/var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/config/ceph.conf 2024-07-26T13:30:22.664 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:22 smithi100 ceph-mon[32599]: Updating smithi167:/var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/config/ceph.conf 2024-07-26T13:30:22.664 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:22 smithi100 ceph-mon[32599]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-07-26T13:30:22.664 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:22 smithi100 ceph-mon[32599]: Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-07-26T13:30:22.664 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:22 smithi100 ceph-mon[32599]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2024-07-26T13:30:22.665 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:22 smithi100 ceph-mon[32599]: Updating smithi138:/var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/config/ceph.client.admin.keyring 2024-07-26T13:30:22.665 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:22 smithi100 ceph-mon[32599]: Updating smithi167:/var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/config/ceph.client.admin.keyring 2024-07-26T13:30:22.665 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:22 smithi100 ceph-mon[32599]: from='client.? ' entity='client.admin' 2024-07-26T13:30:22.665 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:22 smithi100 ceph-mon[32599]: Updating smithi100:/var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/config/ceph.client.admin.keyring 2024-07-26T13:30:22.665 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:22 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/4064531376' entity='client.admin' 2024-07-26T13:30:22.665 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:22.665 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:22.665 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:22.665 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi100", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:30:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:30:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:22 smithi138 ceph-mon[36954]: Updating smithi100:/etc/ceph/ceph.conf 2024-07-26T13:30:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:22 smithi138 ceph-mon[36954]: Updating smithi138:/etc/ceph/ceph.conf 2024-07-26T13:30:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:22 smithi138 ceph-mon[36954]: Updating smithi167:/etc/ceph/ceph.conf 2024-07-26T13:30:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:22 smithi138 ceph-mon[36954]: Updating smithi100:/var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/config/ceph.conf 2024-07-26T13:30:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:22 smithi138 ceph-mon[36954]: Updating smithi138:/var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/config/ceph.conf 2024-07-26T13:30:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:22 smithi138 ceph-mon[36954]: Updating smithi167:/var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/config/ceph.conf 2024-07-26T13:30:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:22 smithi138 ceph-mon[36954]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-07-26T13:30:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:22 smithi138 ceph-mon[36954]: Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-07-26T13:30:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:22 smithi138 ceph-mon[36954]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2024-07-26T13:30:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:22 smithi138 ceph-mon[36954]: Updating smithi138:/var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/config/ceph.client.admin.keyring 2024-07-26T13:30:22.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:22 smithi138 ceph-mon[36954]: Updating smithi167:/var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/config/ceph.client.admin.keyring 2024-07-26T13:30:22.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:22 smithi138 ceph-mon[36954]: from='client.? ' entity='client.admin' 2024-07-26T13:30:22.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:22 smithi138 ceph-mon[36954]: Updating smithi100:/var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/config/ceph.client.admin.keyring 2024-07-26T13:30:22.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:22 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/4064531376' entity='client.admin' 2024-07-26T13:30:22.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:22.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:22.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:22.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:22.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:22.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:22.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi100", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:30:22.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:22.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:30:22.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:22 smithi167 ceph-mon[37140]: Updating smithi100:/etc/ceph/ceph.conf 2024-07-26T13:30:22.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:22 smithi167 ceph-mon[37140]: Updating smithi138:/etc/ceph/ceph.conf 2024-07-26T13:30:22.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:22 smithi167 ceph-mon[37140]: Updating smithi167:/etc/ceph/ceph.conf 2024-07-26T13:30:22.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:22 smithi167 ceph-mon[37140]: Updating smithi100:/var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/config/ceph.conf 2024-07-26T13:30:22.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:22 smithi167 ceph-mon[37140]: Updating smithi138:/var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/config/ceph.conf 2024-07-26T13:30:22.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:22 smithi167 ceph-mon[37140]: Updating smithi167:/var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/config/ceph.conf 2024-07-26T13:30:22.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:22 smithi167 ceph-mon[37140]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-07-26T13:30:22.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:22 smithi167 ceph-mon[37140]: Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-07-26T13:30:22.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:22 smithi167 ceph-mon[37140]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2024-07-26T13:30:22.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:22 smithi167 ceph-mon[37140]: Updating smithi138:/var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/config/ceph.client.admin.keyring 2024-07-26T13:30:22.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:22 smithi167 ceph-mon[37140]: Updating smithi167:/var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/config/ceph.client.admin.keyring 2024-07-26T13:30:22.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:22 smithi167 ceph-mon[37140]: from='client.? ' entity='client.admin' 2024-07-26T13:30:22.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:22 smithi167 ceph-mon[37140]: Updating smithi100:/var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/config/ceph.client.admin.keyring 2024-07-26T13:30:22.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:22 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/4064531376' entity='client.admin' 2024-07-26T13:30:22.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:22.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:22.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:22.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:23.165 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:30:23.172 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:30:23.173 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:30:23.173 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:30:23.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:23.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:23.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:23.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-26T13:30:23.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-26T13:30:23.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-26T13:30:23.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:23.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:23 smithi167 ceph-mon[37140]: Deploying daemon mgr.z on smithi167 2024-07-26T13:30:23.860 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:23.860 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:23.860 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:23.860 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-26T13:30:23.860 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-26T13:30:23.861 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-26T13:30:23.861 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:23.861 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:23 smithi100 ceph-mon[32599]: Deploying daemon mgr.z on smithi167 2024-07-26T13:30:23.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:23.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:23.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:23.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-26T13:30:23.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-26T13:30:23.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-26T13:30:23.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:23.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:23 smithi138 ceph-mon[36954]: Deploying daemon mgr.z on smithi167 2024-07-26T13:30:24.523 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jul 26 13:30:24 smithi167 systemd[1]: Starting Ceph mgr.z for bf83e2e6-4b52-11ef-bcb7-c7b262605968... 2024-07-26T13:30:24.816 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jul 26 13:30:24 smithi167 podman[39959]: 2024-07-26 13:30:24.443586481 +0000 UTC m=+0.017320003 image pull ae51f432d4fa82622bc253ca9e2da7666c617e096e99d43d0a0e4fdaca8865f8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2 2024-07-26T13:30:24.816 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jul 26 13:30:24 smithi167 podman[39959]: 2024-07-26 13:30:24.555504952 +0000 UTC m=+0.129238462 container create 3286874f14a61ab5b47610f6d847d8880d5665bb6d731cf66dbf735d30b23dda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-07-26T13:30:24.816 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jul 26 13:30:24 smithi167 podman[39959]: 2024-07-26 13:30:24.816041338 +0000 UTC m=+0.389774859 container init 3286874f14a61ab5b47610f6d847d8880d5665bb6d731cf66dbf735d30b23dda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-26T13:30:25.272 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jul 26 13:30:24 smithi167 podman[39959]: 2024-07-26 13:30:24.819415065 +0000 UTC m=+0.393148574 container start 3286874f14a61ab5b47610f6d847d8880d5665bb6d731cf66dbf735d30b23dda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-z, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-26T13:30:25.273 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jul 26 13:30:24 smithi167 ceph-mgr[39978]: -- 172.21.15.167:0/1673915806 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d516a030e0 con 0x55d51b7a2480 2024-07-26T13:30:25.273 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jul 26 13:30:24 smithi167 bash[39959]: 3286874f14a61ab5b47610f6d847d8880d5665bb6d731cf66dbf735d30b23dda 2024-07-26T13:30:25.273 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Jul 26 13:30:24 smithi167 systemd[1]: Started Ceph mgr.z for bf83e2e6-4b52-11ef-bcb7-c7b262605968. 2024-07-26T13:30:25.315 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:30:25.322 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:30:25.323 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:30:25.323 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:30:26.235 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'bf83e2e6-4b52-11ef-bcb7-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi100') 2024-07-26T13:30:26.235 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'bf83e2e6-4b52-11ef-bcb7-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi100') 2024-07-26T13:30:26.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:26.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:26.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:26.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-26T13:30:26.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-26T13:30:26.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-26T13:30:26.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:26.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:26 smithi167 ceph-mon[37140]: Deploying daemon mgr.y on smithi138 2024-07-26T13:30:26.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:26.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:26.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:26.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-26T13:30:26.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-26T13:30:26.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-26T13:30:26.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:26.315 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:26 smithi100 ceph-mon[32599]: Deploying daemon mgr.y on smithi138 2024-07-26T13:30:26.334 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:26.334 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:26.334 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:26.334 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-26T13:30:26.334 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-26T13:30:26.334 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-26T13:30:26.334 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:26.334 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:26 smithi138 ceph-mon[36954]: Deploying daemon mgr.y on smithi138 2024-07-26T13:30:26.940 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Jul 26 13:30:26 smithi138 podman[39312]: 2024-07-26 13:30:26.886271821 +0000 UTC m=+0.119429013 container create 0052381d13c6e42417d3bd279436b08331f11e17eef6ee3146b62c2114c010a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-y, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb) 2024-07-26T13:30:27.361 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:27 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/4165065512' entity='client.admin' 2024-07-26T13:30:27.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:27 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/4165065512' entity='client.admin' 2024-07-26T13:30:27.440 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Jul 26 13:30:27 smithi138 podman[39312]: 2024-07-26 13:30:27.097269816 +0000 UTC m=+0.330426952 container init 0052381d13c6e42417d3bd279436b08331f11e17eef6ee3146b62c2114c010a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-07-26T13:30:27.440 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Jul 26 13:30:27 smithi138 podman[39312]: 2024-07-26 13:30:27.100341365 +0000 UTC m=+0.333498485 container start 0052381d13c6e42417d3bd279436b08331f11e17eef6ee3146b62c2114c010a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.36.0) 2024-07-26T13:30:27.440 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Jul 26 13:30:27 smithi138 ceph-mgr[39331]: -- 172.21.15.138:0/2425638883 <== mon.2 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5579105150e0 con 0x557910533a80 2024-07-26T13:30:27.440 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Jul 26 13:30:27 smithi138 bash[39312]: 0052381d13c6e42417d3bd279436b08331f11e17eef6ee3146b62c2114c010a5 2024-07-26T13:30:27.441 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Jul 26 13:30:27 smithi138 systemd[1]: Started Ceph mgr.y for bf83e2e6-4b52-11ef-bcb7-c7b262605968. 2024-07-26T13:30:27.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:27 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/4165065512' entity='client.admin' 2024-07-26T13:30:27.768 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:30:28.522 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:28.522 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:28.523 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:28.523 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:28.523 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:30:28.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:28.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:28.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:28.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:28.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:30:28.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:28.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:28.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:28.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:28.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:30:28.698 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:30:28.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:28.736+0000 7fb8b4e2c640 1 -- 172.21.15.100:0/2540581162 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb89803edb0 msgr2=0x7fb8980412c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:28.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:28.736+0000 7fb8b4e2c640 1 --2- 172.21.15.100:0/2540581162 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb89803edb0 0x7fb8980412c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0009fc0 tx=0x7fb8a0005f90 comp rx=0 tx=0).stop 2024-07-26T13:30:28.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:28.736+0000 7fb8b4e2c640 1 -- 172.21.15.100:0/2540581162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b0107dd0 msgr2=0x7fb8b01990d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:28.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:28.736+0000 7fb8b4e2c640 1 --2- 172.21.15.100:0/2540581162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b0107dd0 0x7fb8b01990d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4032af0 tx=0x7fb8a4032b20 comp rx=0 tx=0).stop 2024-07-26T13:30:28.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:28.736+0000 7fb8b4e2c640 1 -- 172.21.15.100:0/2540581162 shutdown_connections 2024-07-26T13:30:28.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:28.736+0000 7fb8b4e2c640 1 --2- 172.21.15.100:0/2540581162 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb89803edb0 0x7fb8980412c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:28.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:28.736+0000 7fb8b4e2c640 1 --2- 172.21.15.100:0/2540581162 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8b0108720 0x7fb8b019e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:28.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:28.736+0000 7fb8b4e2c640 1 --2- 172.21.15.100:0/2540581162 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb8b0107dd0 0x7fb8b01990d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:28.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:28.736+0000 7fb8b4e2c640 1 --2- 172.21.15.100:0/2540581162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8b0102490 0x7fb8b0198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:28.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:28.736+0000 7fb8b4e2c640 1 -- 172.21.15.100:0/2540581162 >> 172.21.15.100:0/2540581162 conn(0x7fb8b0079160 msgr2=0x7fb8b0103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:30:28.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:28.736+0000 7fb8b4e2c640 1 -- 172.21.15.100:0/2540581162 shutdown_connections 2024-07-26T13:30:28.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:28.736+0000 7fb8b4e2c640 1 -- 172.21.15.100:0/2540581162 wait complete. 2024-07-26T13:30:29.409 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:29.410 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:29.410 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:29.410 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:29.410 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:29.410 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:30:29.410 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:29.410 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-26T13:30:29.410 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-26T13:30:29.410 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:29.410 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:29 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2540581162' entity='client.admin' 2024-07-26T13:30:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:30:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-26T13:30:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-26T13:30:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:29 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/2540581162' entity='client.admin' 2024-07-26T13:30:29.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:29.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:29.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:30:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-26T13:30:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-26T13:30:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:29 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/2540581162' entity='client.admin' 2024-07-26T13:30:30.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:30 smithi100 ceph-mon[32599]: Reconfiguring mgr.x (unknown last config time)... 2024-07-26T13:30:30.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:30 smithi100 ceph-mon[32599]: Reconfiguring daemon mgr.x on smithi100 2024-07-26T13:30:30.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:30 smithi100 ceph-mon[32599]: Standby manager daemon z started 2024-07-26T13:30:30.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:30 smithi100 ceph-mon[32599]: from='mgr.? 172.21.15.167:0/2880879870' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-07-26T13:30:30.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:30 smithi100 ceph-mon[32599]: from='mgr.? 172.21.15.167:0/2880879870' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-26T13:30:30.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:30 smithi100 ceph-mon[32599]: from='mgr.? 172.21.15.167:0/2880879870' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-07-26T13:30:30.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:30 smithi100 ceph-mon[32599]: from='mgr.? 172.21.15.167:0/2880879870' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-26T13:30:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:30 smithi138 ceph-mon[36954]: Reconfiguring mgr.x (unknown last config time)... 2024-07-26T13:30:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:30 smithi138 ceph-mon[36954]: Reconfiguring daemon mgr.x on smithi100 2024-07-26T13:30:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:30 smithi138 ceph-mon[36954]: Standby manager daemon z started 2024-07-26T13:30:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:30 smithi138 ceph-mon[36954]: from='mgr.? 172.21.15.167:0/2880879870' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-07-26T13:30:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:30 smithi138 ceph-mon[36954]: from='mgr.? 172.21.15.167:0/2880879870' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-26T13:30:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:30 smithi138 ceph-mon[36954]: from='mgr.? 172.21.15.167:0/2880879870' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-07-26T13:30:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:30 smithi138 ceph-mon[36954]: from='mgr.? 172.21.15.167:0/2880879870' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-26T13:30:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:30 smithi167 ceph-mon[37140]: Reconfiguring mgr.x (unknown last config time)... 2024-07-26T13:30:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:30 smithi167 ceph-mon[37140]: Reconfiguring daemon mgr.x on smithi100 2024-07-26T13:30:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:30 smithi167 ceph-mon[37140]: Standby manager daemon z started 2024-07-26T13:30:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:30 smithi167 ceph-mon[37140]: from='mgr.? 172.21.15.167:0/2880879870' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-07-26T13:30:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:30 smithi167 ceph-mon[37140]: from='mgr.? 172.21.15.167:0/2880879870' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-26T13:30:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:30 smithi167 ceph-mon[37140]: from='mgr.? 172.21.15.167:0/2880879870' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-07-26T13:30:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:30 smithi167 ceph-mon[37140]: from='mgr.? 172.21.15.167:0/2880879870' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-26T13:30:31.598 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:31 smithi100 ceph-mon[32599]: mgrmap e18: x(active, since 14s), standbys: z 2024-07-26T13:30:31.598 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-07-26T13:30:31.598 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:31 smithi138 ceph-mon[36954]: mgrmap e18: x(active, since 14s), standbys: z 2024-07-26T13:30:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-07-26T13:30:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:31.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:31 smithi167 ceph-mon[37140]: mgrmap e18: x(active, since 14s), standbys: z 2024-07-26T13:30:31.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-07-26T13:30:31.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:31.803 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:30:31.803 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:30:31.803 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:30:31.804 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:30:32.557 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:32 smithi100 ceph-mon[32599]: Standby manager daemon y started 2024-07-26T13:30:32.557 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:32 smithi100 ceph-mon[32599]: from='mgr.? 172.21.15.138:0/395960719' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-07-26T13:30:32.558 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:32 smithi100 ceph-mon[32599]: from='mgr.? 172.21.15.138:0/395960719' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-26T13:30:32.558 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:32 smithi100 ceph-mon[32599]: from='mgr.? 172.21.15.138:0/395960719' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-07-26T13:30:32.558 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:32 smithi100 ceph-mon[32599]: from='mgr.? 172.21.15.138:0/395960719' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-26T13:30:32.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:32 smithi138 ceph-mon[36954]: Standby manager daemon y started 2024-07-26T13:30:32.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:32 smithi138 ceph-mon[36954]: from='mgr.? 172.21.15.138:0/395960719' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-07-26T13:30:32.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:32 smithi138 ceph-mon[36954]: from='mgr.? 172.21.15.138:0/395960719' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-26T13:30:32.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:32 smithi138 ceph-mon[36954]: from='mgr.? 172.21.15.138:0/395960719' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-07-26T13:30:32.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:32 smithi138 ceph-mon[36954]: from='mgr.? 172.21.15.138:0/395960719' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-26T13:30:32.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:32 smithi167 ceph-mon[37140]: Standby manager daemon y started 2024-07-26T13:30:32.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:32 smithi167 ceph-mon[37140]: from='mgr.? 172.21.15.138:0/395960719' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-07-26T13:30:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:32 smithi167 ceph-mon[37140]: from='mgr.? 172.21.15.138:0/395960719' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-26T13:30:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:32 smithi167 ceph-mon[37140]: from='mgr.? 172.21.15.138:0/395960719' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-07-26T13:30:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:32 smithi167 ceph-mon[37140]: from='mgr.? 172.21.15.138:0/395960719' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-26T13:30:33.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:33 smithi100 ceph-mon[32599]: mgrmap e19: x(active, since 16s), standbys: y, z 2024-07-26T13:30:33.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-07-26T13:30:33.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:33.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:33.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:30:33.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:33.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:30:33.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:33 smithi138 ceph-mon[36954]: mgrmap e19: x(active, since 16s), standbys: y, z 2024-07-26T13:30:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-07-26T13:30:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:30:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:30:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:33.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:33 smithi167 ceph-mon[37140]: mgrmap e19: x(active, since 16s), standbys: y, z 2024-07-26T13:30:33.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-07-26T13:30:33.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:33.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:33.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:30:33.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:33.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:30:33.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:35 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2279447976' entity='client.admin' 2024-07-26T13:30:35.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:35 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/2279447976' entity='client.admin' 2024-07-26T13:30:35.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:35 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/2279447976' entity='client.admin' 2024-07-26T13:30:36.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.103+0000 7fd2b44a8640 1 -- 172.21.15.100:0/1753452974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2ac104be0 msgr2=0x7fd2ac104ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:36.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.103+0000 7fd2b44a8640 1 --2- 172.21.15.100:0/1753452974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2ac104be0 0x7fd2ac104ff0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd29c009900 tx=0x7fd29c02a410 comp rx=0 tx=0).stop 2024-07-26T13:30:36.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.103+0000 7fd2b44a8640 1 -- 172.21.15.100:0/1753452974 shutdown_connections 2024-07-26T13:30:36.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.103+0000 7fd2b44a8640 1 --2- 172.21.15.100:0/1753452974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2ac105530 0x7fd2ac10efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:36.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.103+0000 7fd2b44a8640 1 --2- 172.21.15.100:0/1753452974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2ac104be0 0x7fd2ac104ff0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:36.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.103+0000 7fd2b44a8640 1 --2- 172.21.15.100:0/1753452974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2ac102bd0 0x7fd2ac102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:36.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.103+0000 7fd2b44a8640 1 -- 172.21.15.100:0/1753452974 >> 172.21.15.100:0/1753452974 conn(0x7fd2ac079860 msgr2=0x7fd2ac079c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:30:36.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.103+0000 7fd2b44a8640 1 -- 172.21.15.100:0/1753452974 shutdown_connections 2024-07-26T13:30:36.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.104+0000 7fd2b44a8640 1 -- 172.21.15.100:0/1753452974 wait complete. 2024-07-26T13:30:36.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.104+0000 7fd2b44a8640 1 Processor -- start 2024-07-26T13:30:36.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.105+0000 7fd2b44a8640 1 -- start start 2024-07-26T13:30:36.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.105+0000 7fd2b44a8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2ac102bd0 0x7fd2ac198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:36.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.105+0000 7fd2b44a8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2ac104be0 0x7fd2ac199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:36.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.105+0000 7fd2b44a8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2ac105530 0x7fd2ac19e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:36.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.105+0000 7fd2b44a8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ac112160 con 0x7fd2ac104be0 2024-07-26T13:30:36.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.105+0000 7fd2b44a8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ac111fe0 con 0x7fd2ac102bd0 2024-07-26T13:30:36.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.105+0000 7fd2b44a8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ac1122e0 con 0x7fd2ac105530 2024-07-26T13:30:36.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.105+0000 7fd2b1a43640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2ac104be0 0x7fd2ac199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:36.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.105+0000 7fd2b1a43640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2ac104be0 0x7fd2ac199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53570/0 (socket says 172.21.15.100:53570) 2024-07-26T13:30:36.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.105+0000 7fd2b1a43640 1 -- 172.21.15.100:0/2279447976 learned_addr learned my addr 172.21.15.100:0/2279447976 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:30:36.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.105+0000 7fd2b2244640 1 --2- 172.21.15.100:0/2279447976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2ac102bd0 0x7fd2ac198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:36.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.105+0000 7fd2b2a45640 1 --2- 172.21.15.100:0/2279447976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2ac105530 0x7fd2ac19e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:36.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.105+0000 7fd2b1a43640 1 -- 172.21.15.100:0/2279447976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2ac105530 msgr2=0x7fd2ac19e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:36.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.105+0000 7fd2b1a43640 1 --2- 172.21.15.100:0/2279447976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2ac105530 0x7fd2ac19e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:36.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.105+0000 7fd2b1a43640 1 -- 172.21.15.100:0/2279447976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2ac102bd0 msgr2=0x7fd2ac198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:36.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.105+0000 7fd2b1a43640 1 --2- 172.21.15.100:0/2279447976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2ac102bd0 0x7fd2ac198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:36.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.105+0000 7fd2b1a43640 1 -- 172.21.15.100:0/2279447976 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2ac19ec00 con 0x7fd2ac104be0 2024-07-26T13:30:36.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.105+0000 7fd2b2a45640 1 --2- 172.21.15.100:0/2279447976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2ac105530 0x7fd2ac19e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:30:36.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.105+0000 7fd2b1a43640 1 --2- 172.21.15.100:0/2279447976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2ac104be0 0x7fd2ac199230 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd29c005940 tx=0x7fd29c032940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:30:36.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.106+0000 7fd2a37fe640 1 -- 172.21.15.100:0/2279447976 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c03f030 con 0x7fd2ac104be0 2024-07-26T13:30:36.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.106+0000 7fd2b44a8640 1 -- 172.21.15.100:0/2279447976 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2ac19ee90 con 0x7fd2ac104be0 2024-07-26T13:30:36.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.106+0000 7fd2a37fe640 1 -- 172.21.15.100:0/2279447976 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7fd29c032d40 con 0x7fd2ac104be0 2024-07-26T13:30:36.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.106+0000 7fd2a37fe640 1 -- 172.21.15.100:0/2279447976 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c035070 con 0x7fd2ac104be0 2024-07-26T13:30:36.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.106+0000 7fd2b44a8640 1 -- 172.21.15.100:0/2279447976 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2ac1a5650 con 0x7fd2ac104be0 2024-07-26T13:30:36.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.107+0000 7fd2b44a8640 1 -- 172.21.15.100:0/2279447976 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2ac102fe0 con 0x7fd2ac104be0 2024-07-26T13:30:36.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.108+0000 7fd2a37fe640 1 -- 172.21.15.100:0/2279447976 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7fd29c042030 con 0x7fd2ac104be0 2024-07-26T13:30:36.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.109+0000 7fd2a37fe640 1 --2- 172.21.15.100:0/2279447976 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd2980b3b90 0x7fd2980b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:36.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.109+0000 7fd2b2244640 1 --2- 172.21.15.100:0/2279447976 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd2980b3b90 0x7fd2980b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:36.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.109+0000 7fd2a37fe640 1 -- 172.21.15.100:0/2279447976 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fd29c102760 con 0x7fd2ac104be0 2024-07-26T13:30:36.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.109+0000 7fd2b2244640 1 --2- 172.21.15.100:0/2279447976 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd2980b3b90 0x7fd2980b60a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac06b5a0 tx=0x7fd2a8005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:30:36.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.112+0000 7fd2a37fe640 1 -- 172.21.15.100:0/2279447976 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fd29c0cb9b0 con 0x7fd2ac104be0 2024-07-26T13:30:36.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.299+0000 7fd2b44a8640 1 -- 172.21.15.100:0/2279447976 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7fd2ac10a5c0 con 0x7fd2ac104be0 2024-07-26T13:30:36.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.303+0000 7fd2a37fe640 1 -- 172.21.15.100:0/2279447976 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v14)=0 v14) ==== 117+0+0 (secure 0 0 0) 0x7fd29c0cb2b0 con 0x7fd2ac104be0 2024-07-26T13:30:36.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.306+0000 7fd2b44a8640 1 -- 172.21.15.100:0/2279447976 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd2980b3b90 msgr2=0x7fd2980b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:36.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.306+0000 7fd2b44a8640 1 --2- 172.21.15.100:0/2279447976 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd2980b3b90 0x7fd2980b60a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac06b5a0 tx=0x7fd2a8005f90 comp rx=0 tx=0).stop 2024-07-26T13:30:36.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.306+0000 7fd2b44a8640 1 -- 172.21.15.100:0/2279447976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2ac104be0 msgr2=0x7fd2ac199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:36.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.306+0000 7fd2b44a8640 1 --2- 172.21.15.100:0/2279447976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2ac104be0 0x7fd2ac199230 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd29c005940 tx=0x7fd29c032940 comp rx=0 tx=0).stop 2024-07-26T13:30:36.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.307+0000 7fd2b44a8640 1 -- 172.21.15.100:0/2279447976 shutdown_connections 2024-07-26T13:30:36.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.307+0000 7fd2b44a8640 1 --2- 172.21.15.100:0/2279447976 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd2980b3b90 0x7fd2980b60a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:36.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.307+0000 7fd2b44a8640 1 --2- 172.21.15.100:0/2279447976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2ac105530 0x7fd2ac19e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:36.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.307+0000 7fd2b44a8640 1 --2- 172.21.15.100:0/2279447976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2ac104be0 0x7fd2ac199230 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:36.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.307+0000 7fd2b44a8640 1 --2- 172.21.15.100:0/2279447976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2ac102bd0 0x7fd2ac198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:36.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.307+0000 7fd2b44a8640 1 -- 172.21.15.100:0/2279447976 >> 172.21.15.100:0/2279447976 conn(0x7fd2ac079860 msgr2=0x7fd2ac105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:30:36.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.307+0000 7fd2b44a8640 1 -- 172.21.15.100:0/2279447976 shutdown_connections 2024-07-26T13:30:36.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:34.307+0000 7fd2b44a8640 1 -- 172.21.15.100:0/2279447976 wait complete. 2024-07-26T13:30:36.440 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:30:36.733 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'bf83e2e6-4b52-11ef-bcb7-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi100') 2024-07-26T13:30:37.583 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:37 smithi100 ceph-mon[32599]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:37.604 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:37 smithi167 ceph-mon[37140]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:37.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.604+0000 7fd7a42b0640 1 -- 172.21.15.100:0/1876315286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd79c108760 msgr2=0x7fd79c10ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:37.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.604+0000 7fd7a42b0640 1 --2- 172.21.15.100:0/1876315286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd79c108760 0x7fd79c10ef90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd794009ca0 tx=0x7fd79402a8c0 comp rx=0 tx=0).stop 2024-07-26T13:30:37.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.605+0000 7fd7a42b0640 1 -- 172.21.15.100:0/1876315286 shutdown_connections 2024-07-26T13:30:37.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.605+0000 7fd7a42b0640 1 --2- 172.21.15.100:0/1876315286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd79c108760 0x7fd79c10ef90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:37.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.605+0000 7fd7a42b0640 1 --2- 172.21.15.100:0/1876315286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd79c107e10 0x7fd79c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:37.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.605+0000 7fd7a42b0640 1 --2- 172.21.15.100:0/1876315286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd79c1024d0 0x7fd79c1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:37.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.605+0000 7fd7a42b0640 1 -- 172.21.15.100:0/1876315286 >> 172.21.15.100:0/1876315286 conn(0x7fd79c079180 msgr2=0x7fd79c079590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:30:37.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.605+0000 7fd7a42b0640 1 -- 172.21.15.100:0/1876315286 shutdown_connections 2024-07-26T13:30:37.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.605+0000 7fd7a42b0640 1 -- 172.21.15.100:0/1876315286 wait complete. 2024-07-26T13:30:37.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.605+0000 7fd7a42b0640 1 Processor -- start 2024-07-26T13:30:37.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.606+0000 7fd7a42b0640 1 -- start start 2024-07-26T13:30:37.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.606+0000 7fd7a42b0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd79c1024d0 0x7fd79c19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:37.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.606+0000 7fd7a42b0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd79c107e10 0x7fd79c19b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:37.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.606+0000 7fd7a42b0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd79c108760 0x7fd79c19b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:37.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.606+0000 7fd7a42b0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c112160 con 0x7fd79c1024d0 2024-07-26T13:30:37.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.606+0000 7fd7a42b0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c111fe0 con 0x7fd79c108760 2024-07-26T13:30:37.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.606+0000 7fd7a42b0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c1122e0 con 0x7fd79c107e10 2024-07-26T13:30:37.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.606+0000 7fd7a204c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd79c1024d0 0x7fd79c19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:37.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.606+0000 7fd7a204c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd79c1024d0 0x7fd79c19aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56290/0 (socket says 172.21.15.100:56290) 2024-07-26T13:30:37.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.606+0000 7fd7a204c640 1 -- 172.21.15.100:0/4140903951 learned_addr learned my addr 172.21.15.100:0/4140903951 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:30:37.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.606+0000 7fd7a184b640 1 --2- 172.21.15.100:0/4140903951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd79c107e10 0x7fd79c19b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:37.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.606+0000 7fd7a284d640 1 --2- 172.21.15.100:0/4140903951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd79c108760 0x7fd79c19b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:37.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.606+0000 7fd7a204c640 1 -- 172.21.15.100:0/4140903951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd79c107e10 msgr2=0x7fd79c19b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:37.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.606+0000 7fd7a204c640 1 --2- 172.21.15.100:0/4140903951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd79c107e10 0x7fd79c19b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:37.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.606+0000 7fd7a204c640 1 -- 172.21.15.100:0/4140903951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd79c108760 msgr2=0x7fd79c19b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:37.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.606+0000 7fd7a204c640 1 --2- 172.21.15.100:0/4140903951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd79c108760 0x7fd79c19b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:37.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.606+0000 7fd7a204c640 1 -- 172.21.15.100:0/4140903951 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd79c1a5d20 con 0x7fd79c1024d0 2024-07-26T13:30:37.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.606+0000 7fd7a284d640 1 --2- 172.21.15.100:0/4140903951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd79c108760 0x7fd79c19b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:30:37.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.607+0000 7fd7a204c640 1 --2- 172.21.15.100:0/4140903951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd79c1024d0 0x7fd79c19aaf0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd79800e9f0 tx=0x7fd79800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:30:37.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.607+0000 7fd7937fe640 1 -- 172.21.15.100:0/4140903951 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79800ccc0 con 0x7fd79c1024d0 2024-07-26T13:30:37.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.607+0000 7fd7937fe640 1 -- 172.21.15.100:0/4140903951 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7fd79800ce60 con 0x7fd79c1024d0 2024-07-26T13:30:37.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.607+0000 7fd7a42b0640 1 -- 172.21.15.100:0/4140903951 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd79c1a5fb0 con 0x7fd79c1024d0 2024-07-26T13:30:37.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.607+0000 7fd7937fe640 1 -- 172.21.15.100:0/4140903951 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd798010430 con 0x7fd79c1024d0 2024-07-26T13:30:37.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.607+0000 7fd7a42b0640 1 -- 172.21.15.100:0/4140903951 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd79c1a6470 con 0x7fd79c1024d0 2024-07-26T13:30:37.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.608+0000 7fd7a42b0640 1 -- 172.21.15.100:0/4140903951 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd79c1028e0 con 0x7fd79c1024d0 2024-07-26T13:30:37.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.609+0000 7fd7937fe640 1 -- 172.21.15.100:0/4140903951 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7fd798004eb0 con 0x7fd79c1024d0 2024-07-26T13:30:37.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.611+0000 7fd7937fe640 1 --2- 172.21.15.100:0/4140903951 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd7880b3b90 0x7fd7880b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:37.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.611+0000 7fd7a184b640 1 --2- 172.21.15.100:0/4140903951 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd7880b3b90 0x7fd7880b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:37.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.611+0000 7fd7937fe640 1 -- 172.21.15.100:0/4140903951 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fd7980e26f0 con 0x7fd79c1024d0 2024-07-26T13:30:37.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.611+0000 7fd7a184b640 1 --2- 172.21.15.100:0/4140903951 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd7880b3b90 0x7fd7880b60a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd79c079990 tx=0x7fd78c00a2d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:30:37.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.614+0000 7fd7937fe640 1 -- 172.21.15.100:0/4140903951 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fd798055600 con 0x7fd79c1024d0 2024-07-26T13:30:37.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:37 smithi138 ceph-mon[36954]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:37.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.793+0000 7fd7a42b0640 1 -- 172.21.15.100:0/4140903951 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7fd79c10a550 con 0x7fd79c1024d0 2024-07-26T13:30:37.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.797+0000 7fd7937fe640 1 -- 172.21.15.100:0/4140903951 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v15)=0 v15) ==== 108+0+0 (secure 0 0 0) 0x7fd7980ab8a0 con 0x7fd79c1024d0 2024-07-26T13:30:37.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.800+0000 7fd7a42b0640 1 -- 172.21.15.100:0/4140903951 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd7880b3b90 msgr2=0x7fd7880b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:37.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.800+0000 7fd7a42b0640 1 --2- 172.21.15.100:0/4140903951 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd7880b3b90 0x7fd7880b60a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd79c079990 tx=0x7fd78c00a2d0 comp rx=0 tx=0).stop 2024-07-26T13:30:37.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.800+0000 7fd7a42b0640 1 -- 172.21.15.100:0/4140903951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd79c1024d0 msgr2=0x7fd79c19aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:37.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.800+0000 7fd7a42b0640 1 --2- 172.21.15.100:0/4140903951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd79c1024d0 0x7fd79c19aaf0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd79800e9f0 tx=0x7fd79800edf0 comp rx=0 tx=0).stop 2024-07-26T13:30:37.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.800+0000 7fd7a42b0640 1 -- 172.21.15.100:0/4140903951 shutdown_connections 2024-07-26T13:30:37.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.800+0000 7fd7a42b0640 1 --2- 172.21.15.100:0/4140903951 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd7880b3b90 0x7fd7880b60a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:37.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.800+0000 7fd7a42b0640 1 --2- 172.21.15.100:0/4140903951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd79c108760 0x7fd79c19b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:37.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.800+0000 7fd7a42b0640 1 --2- 172.21.15.100:0/4140903951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd79c107e10 0x7fd79c19b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:37.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.800+0000 7fd7a42b0640 1 --2- 172.21.15.100:0/4140903951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd79c1024d0 0x7fd79c19aaf0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:37.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.800+0000 7fd7a42b0640 1 -- 172.21.15.100:0/4140903951 >> 172.21.15.100:0/4140903951 conn(0x7fd79c079180 msgr2=0x7fd79c105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:30:37.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.800+0000 7fd7a42b0640 1 -- 172.21.15.100:0/4140903951 shutdown_connections 2024-07-26T13:30:37.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:37.800+0000 7fd7a42b0640 1 -- 172.21.15.100:0/4140903951 wait complete. 2024-07-26T13:30:38.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.804+0000 7f381bf28640 1 -- 172.21.15.100:0/4228083018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3814102490 msgr2=0x7f38141028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:38.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.804+0000 7f381bf28640 1 --2- 172.21.15.100:0/4228083018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3814102490 0x7f38141028a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3810009900 tx=0x7f381002a410 comp rx=0 tx=0).stop 2024-07-26T13:30:38.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.804+0000 7f381bf28640 1 -- 172.21.15.100:0/4228083018 shutdown_connections 2024-07-26T13:30:38.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.804+0000 7f381bf28640 1 --2- 172.21.15.100:0/4228083018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3814108720 0x7f381410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:38.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.804+0000 7f381bf28640 1 --2- 172.21.15.100:0/4228083018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3814107dd0 0x7f38141081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:38.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.804+0000 7f381bf28640 1 --2- 172.21.15.100:0/4228083018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3814102490 0x7f38141028a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:38.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.804+0000 7f381bf28640 1 -- 172.21.15.100:0/4228083018 >> 172.21.15.100:0/4228083018 conn(0x7f3814079160 msgr2=0x7f3814079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:30:38.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.804+0000 7f381bf28640 1 -- 172.21.15.100:0/4228083018 shutdown_connections 2024-07-26T13:30:38.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.804+0000 7f381bf28640 1 -- 172.21.15.100:0/4228083018 wait complete. 2024-07-26T13:30:38.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.805+0000 7f381bf28640 1 Processor -- start 2024-07-26T13:30:38.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.805+0000 7f381bf28640 1 -- start start 2024-07-26T13:30:38.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.805+0000 7f381bf28640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3814102490 0x7f3814199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:38.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.805+0000 7f381bf28640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3814107dd0 0x7f381419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:38.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.805+0000 7f381bf28640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3814108720 0x7f381419f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:38.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.805+0000 7f381bf28640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f38141132a0 con 0x7f3814102490 2024-07-26T13:30:38.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.805+0000 7f381bf28640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3814113120 con 0x7f3814107dd0 2024-07-26T13:30:38.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.805+0000 7f381bf28640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3814113420 con 0x7f3814108720 2024-07-26T13:30:38.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.805+0000 7f3819cc4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3814102490 0x7f3814199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:38.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.805+0000 7f3819cc4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3814102490 0x7f3814199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56314/0 (socket says 172.21.15.100:56314) 2024-07-26T13:30:38.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.805+0000 7f3819cc4640 1 -- 172.21.15.100:0/3686408600 learned_addr learned my addr 172.21.15.100:0/3686408600 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:30:38.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.805+0000 7f381a4c5640 1 --2- 172.21.15.100:0/3686408600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3814108720 0x7f381419f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:38.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.805+0000 7f38194c3640 1 --2- 172.21.15.100:0/3686408600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3814107dd0 0x7f381419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:38.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.805+0000 7f3819cc4640 1 -- 172.21.15.100:0/3686408600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3814108720 msgr2=0x7f381419f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:38.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.805+0000 7f3819cc4640 1 --2- 172.21.15.100:0/3686408600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3814108720 0x7f381419f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:38.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.805+0000 7f3819cc4640 1 -- 172.21.15.100:0/3686408600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3814107dd0 msgr2=0x7f381419a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:38.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.805+0000 7f3819cc4640 1 --2- 172.21.15.100:0/3686408600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3814107dd0 0x7f381419a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:38.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.805+0000 7f3819cc4640 1 -- 172.21.15.100:0/3686408600 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f381419fc80 con 0x7f3814102490 2024-07-26T13:30:38.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.805+0000 7f38194c3640 1 --2- 172.21.15.100:0/3686408600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3814107dd0 0x7f381419a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:30:38.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.806+0000 7f3819cc4640 1 --2- 172.21.15.100:0/3686408600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3814102490 0x7f3814199d10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3810005ab0 tx=0x7f3810003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:30:38.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.806+0000 7f380affd640 1 -- 172.21.15.100:0/3686408600 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381003f070 con 0x7f3814102490 2024-07-26T13:30:38.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.806+0000 7f380affd640 1 -- 172.21.15.100:0/3686408600 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f3810031b20 con 0x7f3814102490 2024-07-26T13:30:38.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.806+0000 7f381bf28640 1 -- 172.21.15.100:0/3686408600 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f381419ff10 con 0x7f3814102490 2024-07-26T13:30:38.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.806+0000 7f380affd640 1 -- 172.21.15.100:0/3686408600 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3810035030 con 0x7f3814102490 2024-07-26T13:30:38.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.806+0000 7f381bf28640 1 -- 172.21.15.100:0/3686408600 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38141a66d0 con 0x7f3814102490 2024-07-26T13:30:38.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.807+0000 7f381bf28640 1 -- 172.21.15.100:0/3686408600 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38141028a0 con 0x7f3814102490 2024-07-26T13:30:38.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.808+0000 7f380affd640 1 -- 172.21.15.100:0/3686408600 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7f3810042070 con 0x7f3814102490 2024-07-26T13:30:38.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.809+0000 7f380affd640 1 --2- 172.21.15.100:0/3686408600 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f38000b3c10 0x7f38000b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:38.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.809+0000 7f38194c3640 1 --2- 172.21.15.100:0/3686408600 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f38000b3c10 0x7f38000b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:38.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.809+0000 7f380affd640 1 -- 172.21.15.100:0/3686408600 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f3810102b50 con 0x7f3814102490 2024-07-26T13:30:38.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.810+0000 7f38194c3640 1 --2- 172.21.15.100:0/3686408600 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f38000b3c10 0x7f38000b6120 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f38141007c0 tx=0x7f3804005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:30:38.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.813+0000 7f380affd640 1 -- 172.21.15.100:0/3686408600 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f38100cbd00 con 0x7f3814102490 2024-07-26T13:30:38.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.993+0000 7f381bf28640 1 -- 172.21.15.100:0/3686408600 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f381410a4f0 con 0x7f3814102490 2024-07-26T13:30:38.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.998+0000 7f380affd640 1 -- 172.21.15.100:0/3686408600 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v16)=0 v16) ==== 123+0+0 (secure 0 0 0) 0x7f38100cb600 con 0x7f3814102490 2024-07-26T13:30:39.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:38.999+0000 7f380affd640 1 -- 172.21.15.100:0/3686408600 <== mon.0 v2:172.21.15.100:3300/0 8 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3810031cd0 con 0x7f3814102490 2024-07-26T13:30:39.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.000+0000 7f381bf28640 1 -- 172.21.15.100:0/3686408600 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f38000b3c10 msgr2=0x7f38000b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:39.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.000+0000 7f381bf28640 1 --2- 172.21.15.100:0/3686408600 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f38000b3c10 0x7f38000b6120 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f38141007c0 tx=0x7f3804005f90 comp rx=0 tx=0).stop 2024-07-26T13:30:39.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.000+0000 7f381bf28640 1 -- 172.21.15.100:0/3686408600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3814102490 msgr2=0x7f3814199d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:39.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.000+0000 7f381bf28640 1 --2- 172.21.15.100:0/3686408600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3814102490 0x7f3814199d10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3810005ab0 tx=0x7f3810003c10 comp rx=0 tx=0).stop 2024-07-26T13:30:39.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.000+0000 7f381bf28640 1 -- 172.21.15.100:0/3686408600 shutdown_connections 2024-07-26T13:30:39.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.000+0000 7f381bf28640 1 --2- 172.21.15.100:0/3686408600 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f38000b3c10 0x7f38000b6120 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:39.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.000+0000 7f381bf28640 1 --2- 172.21.15.100:0/3686408600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3814108720 0x7f381419f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:39.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.000+0000 7f381bf28640 1 --2- 172.21.15.100:0/3686408600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3814107dd0 0x7f381419a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:39.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.000+0000 7f381bf28640 1 --2- 172.21.15.100:0/3686408600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3814102490 0x7f3814199d10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:39.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.000+0000 7f381bf28640 1 -- 172.21.15.100:0/3686408600 >> 172.21.15.100:0/3686408600 conn(0x7f3814079160 msgr2=0x7f3814100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:30:39.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.000+0000 7f381bf28640 1 -- 172.21.15.100:0/3686408600 shutdown_connections 2024-07-26T13:30:39.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.000+0000 7f381bf28640 1 -- 172.21.15.100:0/3686408600 wait complete. 2024-07-26T13:30:39.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:38 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/4140903951' entity='client.admin' 2024-07-26T13:30:39.044 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:38 smithi100 ceph-mon[32599]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:38 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/4140903951' entity='client.admin' 2024-07-26T13:30:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:38 smithi138 ceph-mon[36954]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:39.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:38 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/4140903951' entity='client.admin' 2024-07-26T13:30:39.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:38 smithi167 ceph-mon[37140]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:39.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.714+0000 7fb83d24c640 1 -- 172.21.15.100:0/2718477394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb838107e00 msgr2=0x7fb838108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:39.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.714+0000 7fb83d24c640 1 --2- 172.21.15.100:0/2718477394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb838107e00 0x7fb838108210 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb82c009900 tx=0x7fb82c02a410 comp rx=0 tx=0).stop 2024-07-26T13:30:39.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.714+0000 7fb83d24c640 1 -- 172.21.15.100:0/2718477394 shutdown_connections 2024-07-26T13:30:39.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.714+0000 7fb83d24c640 1 --2- 172.21.15.100:0/2718477394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb838108750 0x7fb83810ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:39.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.714+0000 7fb83d24c640 1 --2- 172.21.15.100:0/2718477394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb838107e00 0x7fb838108210 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:39.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.714+0000 7fb83d24c640 1 --2- 172.21.15.100:0/2718477394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8381024c0 0x7fb8381028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:39.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.714+0000 7fb83d24c640 1 -- 172.21.15.100:0/2718477394 >> 172.21.15.100:0/2718477394 conn(0x7fb838079180 msgr2=0x7fb838079590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:30:39.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.714+0000 7fb83d24c640 1 -- 172.21.15.100:0/2718477394 shutdown_connections 2024-07-26T13:30:39.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.714+0000 7fb83d24c640 1 -- 172.21.15.100:0/2718477394 wait complete. 2024-07-26T13:30:39.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.715+0000 7fb83d24c640 1 Processor -- start 2024-07-26T13:30:39.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.715+0000 7fb83d24c640 1 -- start start 2024-07-26T13:30:39.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.715+0000 7fb83d24c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8381024c0 0x7fb83819abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:39.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.715+0000 7fb83d24c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb838107e00 0x7fb83819b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:39.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.715+0000 7fb83d24c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb838108750 0x7fb83819b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:39.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.715+0000 7fb83d24c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb838112240 con 0x7fb838107e00 2024-07-26T13:30:39.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.715+0000 7fb83d24c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb8381120c0 con 0x7fb8381024c0 2024-07-26T13:30:39.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.715+0000 7fb83d24c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb8381123c0 con 0x7fb838108750 2024-07-26T13:30:39.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.716+0000 7fb83659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb838107e00 0x7fb83819b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:39.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.716+0000 7fb83659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb838107e00 0x7fb83819b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56326/0 (socket says 172.21.15.100:56326) 2024-07-26T13:30:39.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.716+0000 7fb83659c640 1 -- 172.21.15.100:0/3813085371 learned_addr learned my addr 172.21.15.100:0/3813085371 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:30:39.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.716+0000 7fb83759e640 1 --2- 172.21.15.100:0/3813085371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb838108750 0x7fb83819b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:39.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.716+0000 7fb836d9d640 1 --2- 172.21.15.100:0/3813085371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8381024c0 0x7fb83819abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:39.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.716+0000 7fb83659c640 1 -- 172.21.15.100:0/3813085371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb838108750 msgr2=0x7fb83819b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:39.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.716+0000 7fb83659c640 1 --2- 172.21.15.100:0/3813085371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb838108750 0x7fb83819b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:39.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.716+0000 7fb83659c640 1 -- 172.21.15.100:0/3813085371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8381024c0 msgr2=0x7fb83819abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:39.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.716+0000 7fb83659c640 1 --2- 172.21.15.100:0/3813085371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8381024c0 0x7fb83819abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:39.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.716+0000 7fb83659c640 1 -- 172.21.15.100:0/3813085371 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8381a5e80 con 0x7fb838107e00 2024-07-26T13:30:39.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.716+0000 7fb83659c640 1 --2- 172.21.15.100:0/3813085371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb838107e00 0x7fb83819b130 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb82c005940 tx=0x7fb82c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:30:39.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.716+0000 7fb81ffff640 1 -- 172.21.15.100:0/3813085371 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c03f030 con 0x7fb838107e00 2024-07-26T13:30:39.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.716+0000 7fb81ffff640 1 -- 172.21.15.100:0/3813085371 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb82c031800 con 0x7fb838107e00 2024-07-26T13:30:39.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.716+0000 7fb83d24c640 1 -- 172.21.15.100:0/3813085371 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8381a6110 con 0x7fb838107e00 2024-07-26T13:30:39.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.716+0000 7fb81ffff640 1 -- 172.21.15.100:0/3813085371 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c035070 con 0x7fb838107e00 2024-07-26T13:30:39.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.716+0000 7fb83d24c640 1 -- 172.21.15.100:0/3813085371 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8381a6640 con 0x7fb838107e00 2024-07-26T13:30:39.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.717+0000 7fb83d24c640 1 -- 172.21.15.100:0/3813085371 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb838103740 con 0x7fb838107e00 2024-07-26T13:30:39.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.720+0000 7fb81ffff640 1 -- 172.21.15.100:0/3813085371 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7fb82c042030 con 0x7fb838107e00 2024-07-26T13:30:39.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.720+0000 7fb81ffff640 1 --2- 172.21.15.100:0/3813085371 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb8200b3c40 0x7fb8200b6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:39.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.720+0000 7fb836d9d640 1 --2- 172.21.15.100:0/3813085371 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb8200b3c40 0x7fb8200b6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:39.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.721+0000 7fb81ffff640 1 -- 172.21.15.100:0/3813085371 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fb82c1027f0 con 0x7fb838107e00 2024-07-26T13:30:39.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.721+0000 7fb836d9d640 1 --2- 172.21.15.100:0/3813085371 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb8200b3c40 0x7fb8200b6150 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb83806c110 tx=0x7fb82800b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:30:39.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.724+0000 7fb81ffff640 1 -- 172.21.15.100:0/3813085371 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fb82c0cb3f0 con 0x7fb838107e00 2024-07-26T13:30:39.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.902+0000 7fb83d24c640 1 -- 172.21.15.100:0/3813085371 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7fb8381028d0 con 0x7fb838107e00 2024-07-26T13:30:39.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.906+0000 7fb81ffff640 1 -- 172.21.15.100:0/3813085371 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v17)=0 v17) ==== 121+0+0 (secure 0 0 0) 0x7fb82c0cf940 con 0x7fb838107e00 2024-07-26T13:30:39.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.908+0000 7fb83d24c640 1 -- 172.21.15.100:0/3813085371 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb8200b3c40 msgr2=0x7fb8200b6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:39.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.908+0000 7fb83d24c640 1 --2- 172.21.15.100:0/3813085371 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb8200b3c40 0x7fb8200b6150 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb83806c110 tx=0x7fb82800b390 comp rx=0 tx=0).stop 2024-07-26T13:30:39.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.908+0000 7fb83d24c640 1 -- 172.21.15.100:0/3813085371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb838107e00 msgr2=0x7fb83819b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:39.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.908+0000 7fb83d24c640 1 --2- 172.21.15.100:0/3813085371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb838107e00 0x7fb83819b130 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb82c005940 tx=0x7fb82c0042f0 comp rx=0 tx=0).stop 2024-07-26T13:30:39.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.908+0000 7fb83d24c640 1 -- 172.21.15.100:0/3813085371 shutdown_connections 2024-07-26T13:30:39.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.908+0000 7fb83d24c640 1 --2- 172.21.15.100:0/3813085371 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb8200b3c40 0x7fb8200b6150 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:39.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.908+0000 7fb83d24c640 1 --2- 172.21.15.100:0/3813085371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb838108750 0x7fb83819b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:39.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.908+0000 7fb83d24c640 1 --2- 172.21.15.100:0/3813085371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb838107e00 0x7fb83819b130 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:39.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.908+0000 7fb83d24c640 1 --2- 172.21.15.100:0/3813085371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8381024c0 0x7fb83819abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:39.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.908+0000 7fb83d24c640 1 -- 172.21.15.100:0/3813085371 >> 172.21.15.100:0/3813085371 conn(0x7fb838079180 msgr2=0x7fb838103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:30:39.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.909+0000 7fb83d24c640 1 -- 172.21.15.100:0/3813085371 shutdown_connections 2024-07-26T13:30:39.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:39.909+0000 7fb83d24c640 1 -- 172.21.15.100:0/3813085371 wait complete. 2024-07-26T13:30:40.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:39 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/3686408600' entity='client.admin' 2024-07-26T13:30:40.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:39 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/3813085371' entity='client.admin' 2024-07-26T13:30:40.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:39 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/3686408600' entity='client.admin' 2024-07-26T13:30:40.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:39 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/3813085371' entity='client.admin' 2024-07-26T13:30:40.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:39 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/3686408600' entity='client.admin' 2024-07-26T13:30:40.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:39 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/3813085371' entity='client.admin' 2024-07-26T13:30:40.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.496+0000 7fb74ffeb640 1 -- 172.21.15.100:0/2442011913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb748108720 msgr2=0x7fb74810ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:40.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.496+0000 7fb74ffeb640 1 --2- 172.21.15.100:0/2442011913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb748108720 0x7fb74810ef50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb740009900 tx=0x7fb74002a410 comp rx=0 tx=0).stop 2024-07-26T13:30:40.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.496+0000 7fb74ffeb640 1 -- 172.21.15.100:0/2442011913 shutdown_connections 2024-07-26T13:30:40.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.496+0000 7fb74ffeb640 1 --2- 172.21.15.100:0/2442011913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb748108720 0x7fb74810ef50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:40.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.496+0000 7fb74ffeb640 1 --2- 172.21.15.100:0/2442011913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb748107dd0 0x7fb7481081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:40.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.496+0000 7fb74ffeb640 1 --2- 172.21.15.100:0/2442011913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb748102490 0x7fb7481028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:40.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.496+0000 7fb74ffeb640 1 -- 172.21.15.100:0/2442011913 >> 172.21.15.100:0/2442011913 conn(0x7fb748079160 msgr2=0x7fb748079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:30:40.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.496+0000 7fb74ffeb640 1 -- 172.21.15.100:0/2442011913 shutdown_connections 2024-07-26T13:30:40.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.496+0000 7fb74ffeb640 1 -- 172.21.15.100:0/2442011913 wait complete. 2024-07-26T13:30:40.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.497+0000 7fb74ffeb640 1 Processor -- start 2024-07-26T13:30:40.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.497+0000 7fb74ffeb640 1 -- start start 2024-07-26T13:30:40.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.497+0000 7fb74ffeb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb748102490 0x7fb748199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:40.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.497+0000 7fb74ffeb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb748107dd0 0x7fb74819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:40.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.497+0000 7fb74ffeb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb748108720 0x7fb74819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:40.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.497+0000 7fb74ffeb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb748112260 con 0x7fb748108720 2024-07-26T13:30:40.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.497+0000 7fb74ffeb640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb7481120e0 con 0x7fb748107dd0 2024-07-26T13:30:40.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.497+0000 7fb74ffeb640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb7481123e0 con 0x7fb748102490 2024-07-26T13:30:40.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.497+0000 7fb74e588640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb748108720 0x7fb74819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:40.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.497+0000 7fb74e588640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb748108720 0x7fb74819f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56344/0 (socket says 172.21.15.100:56344) 2024-07-26T13:30:40.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.497+0000 7fb74e588640 1 -- 172.21.15.100:0/3801950642 learned_addr learned my addr 172.21.15.100:0/3801950642 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:30:40.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.497+0000 7fb74dd87640 1 --2- 172.21.15.100:0/3801950642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb748102490 0x7fb748199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:40.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.497+0000 7fb74d586640 1 --2- 172.21.15.100:0/3801950642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb748107dd0 0x7fb74819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:40.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.497+0000 7fb74e588640 1 -- 172.21.15.100:0/3801950642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb748102490 msgr2=0x7fb748199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:40.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.497+0000 7fb74e588640 1 --2- 172.21.15.100:0/3801950642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb748102490 0x7fb748199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:40.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.497+0000 7fb74e588640 1 -- 172.21.15.100:0/3801950642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb748107dd0 msgr2=0x7fb74819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:40.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.497+0000 7fb74e588640 1 --2- 172.21.15.100:0/3801950642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb748107dd0 0x7fb74819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:40.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.498+0000 7fb74e588640 1 -- 172.21.15.100:0/3801950642 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb74819fe30 con 0x7fb748108720 2024-07-26T13:30:40.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.498+0000 7fb74dd87640 1 --2- 172.21.15.100:0/3801950642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb748102490 0x7fb748199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:30:40.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.498+0000 7fb74d586640 1 --2- 172.21.15.100:0/3801950642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb748107dd0 0x7fb74819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:30:40.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.498+0000 7fb74e588640 1 --2- 172.21.15.100:0/3801950642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb748108720 0x7fb74819f700 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb740005ab0 tx=0x7fb740004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:30:40.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.498+0000 7fb746ffd640 1 -- 172.21.15.100:0/3801950642 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74003f030 con 0x7fb748108720 2024-07-26T13:30:40.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.498+0000 7fb746ffd640 1 -- 172.21.15.100:0/3801950642 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb740003a80 con 0x7fb748108720 2024-07-26T13:30:40.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.498+0000 7fb746ffd640 1 -- 172.21.15.100:0/3801950642 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb740035070 con 0x7fb748108720 2024-07-26T13:30:40.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.498+0000 7fb74ffeb640 1 -- 172.21.15.100:0/3801950642 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7481a00c0 con 0x7fb748108720 2024-07-26T13:30:40.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.498+0000 7fb74ffeb640 1 -- 172.21.15.100:0/3801950642 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7481a6880 con 0x7fb748108720 2024-07-26T13:30:40.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.499+0000 7fb74ffeb640 1 -- 172.21.15.100:0/3801950642 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb748103710 con 0x7fb748108720 2024-07-26T13:30:40.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.500+0000 7fb746ffd640 1 -- 172.21.15.100:0/3801950642 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7fb740042030 con 0x7fb748108720 2024-07-26T13:30:40.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.501+0000 7fb746ffd640 1 --2- 172.21.15.100:0/3801950642 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb7300b3b90 0x7fb7300b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:40.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.501+0000 7fb74dd87640 1 --2- 172.21.15.100:0/3801950642 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb7300b3b90 0x7fb7300b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:40.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.501+0000 7fb746ffd640 1 -- 172.21.15.100:0/3801950642 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fb740102990 con 0x7fb748108720 2024-07-26T13:30:40.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.502+0000 7fb74dd87640 1 --2- 172.21.15.100:0/3801950642 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb7300b3b90 0x7fb7300b60a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb738005fd0 tx=0x7fb738009520 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:30:40.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.504+0000 7fb746ffd640 1 -- 172.21.15.100:0/3801950642 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fb7400cbbe0 con 0x7fb748108720 2024-07-26T13:30:40.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.685+0000 7fb74ffeb640 1 -- 172.21.15.100:0/3801950642 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fb7481028a0 con 0x7fb748108720 2024-07-26T13:30:40.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.689+0000 7fb746ffd640 1 -- 172.21.15.100:0/3801950642 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v18)=0 v18) ==== 121+0+0 (secure 0 0 0) 0x7fb7481028a0 con 0x7fb748108720 2024-07-26T13:30:40.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.693+0000 7fb74ffeb640 1 -- 172.21.15.100:0/3801950642 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb7300b3b90 msgr2=0x7fb7300b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:40.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.693+0000 7fb74ffeb640 1 --2- 172.21.15.100:0/3801950642 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb7300b3b90 0x7fb7300b60a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb738005fd0 tx=0x7fb738009520 comp rx=0 tx=0).stop 2024-07-26T13:30:40.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.693+0000 7fb74ffeb640 1 -- 172.21.15.100:0/3801950642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb748108720 msgr2=0x7fb74819f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:40.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.693+0000 7fb74ffeb640 1 --2- 172.21.15.100:0/3801950642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb748108720 0x7fb74819f700 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb740005ab0 tx=0x7fb740004270 comp rx=0 tx=0).stop 2024-07-26T13:30:40.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.693+0000 7fb74ffeb640 1 -- 172.21.15.100:0/3801950642 shutdown_connections 2024-07-26T13:30:40.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.693+0000 7fb74ffeb640 1 --2- 172.21.15.100:0/3801950642 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb7300b3b90 0x7fb7300b60a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:40.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.693+0000 7fb74ffeb640 1 --2- 172.21.15.100:0/3801950642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb748108720 0x7fb74819f700 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:40.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.693+0000 7fb74ffeb640 1 --2- 172.21.15.100:0/3801950642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb748107dd0 0x7fb74819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:40.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.693+0000 7fb74ffeb640 1 --2- 172.21.15.100:0/3801950642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb748102490 0x7fb748199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:40.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.693+0000 7fb74ffeb640 1 -- 172.21.15.100:0/3801950642 >> 172.21.15.100:0/3801950642 conn(0x7fb748079160 msgr2=0x7fb7480ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:30:40.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.693+0000 7fb74ffeb640 1 -- 172.21.15.100:0/3801950642 shutdown_connections 2024-07-26T13:30:40.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.693+0000 7fb74ffeb640 1 -- 172.21.15.100:0/3801950642 wait complete. 2024-07-26T13:30:40.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.966+0000 7f7ead3de640 1 -- 172.21.15.100:0/2715398377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea8107dd0 msgr2=0x7f7ea81081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:40.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.966+0000 7f7ead3de640 1 --2- 172.21.15.100:0/2715398377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea8107dd0 0x7f7ea81081e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7e90009900 tx=0x7f7e9002a410 comp rx=0 tx=0).stop 2024-07-26T13:30:40.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.966+0000 7f7ead3de640 1 -- 172.21.15.100:0/2715398377 shutdown_connections 2024-07-26T13:30:40.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.966+0000 7f7ead3de640 1 --2- 172.21.15.100:0/2715398377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ea8108720 0x7f7ea810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:40.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.966+0000 7f7ead3de640 1 --2- 172.21.15.100:0/2715398377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea8107dd0 0x7f7ea81081e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:40.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.966+0000 7f7ead3de640 1 --2- 172.21.15.100:0/2715398377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ea8102490 0x7f7ea81028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:40.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.966+0000 7f7ead3de640 1 -- 172.21.15.100:0/2715398377 >> 172.21.15.100:0/2715398377 conn(0x7f7ea8079160 msgr2=0x7f7ea8079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:30:40.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.966+0000 7f7ead3de640 1 -- 172.21.15.100:0/2715398377 shutdown_connections 2024-07-26T13:30:40.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.966+0000 7f7ead3de640 1 -- 172.21.15.100:0/2715398377 wait complete. 2024-07-26T13:30:40.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.967+0000 7f7ead3de640 1 Processor -- start 2024-07-26T13:30:40.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.967+0000 7f7ead3de640 1 -- start start 2024-07-26T13:30:40.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.967+0000 7f7ead3de640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ea8102490 0x7f7ea8199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:40.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.967+0000 7f7ead3de640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea8107dd0 0x7f7ea819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:40.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.967+0000 7f7ead3de640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ea8108720 0x7f7ea819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:40.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.967+0000 7f7ead3de640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea8113390 con 0x7f7ea8107dd0 2024-07-26T13:30:40.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.967+0000 7f7ead3de640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea8113210 con 0x7f7ea8102490 2024-07-26T13:30:40.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.967+0000 7f7ead3de640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea8113510 con 0x7f7ea8108720 2024-07-26T13:30:40.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.967+0000 7f7ea67fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea8107dd0 0x7f7ea819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:40.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.967+0000 7f7ea67fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea8107dd0 0x7f7ea819a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56372/0 (socket says 172.21.15.100:56372) 2024-07-26T13:30:40.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.967+0000 7f7ea67fc640 1 -- 172.21.15.100:0/3116578912 learned_addr learned my addr 172.21.15.100:0/3116578912 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:30:40.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.968+0000 7f7ea6ffd640 1 --2- 172.21.15.100:0/3116578912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ea8102490 0x7f7ea8199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:40.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.968+0000 7f7ea77fe640 1 --2- 172.21.15.100:0/3116578912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ea8108720 0x7f7ea819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:40.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.968+0000 7f7ea67fc640 1 -- 172.21.15.100:0/3116578912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ea8108720 msgr2=0x7f7ea819f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:40.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.968+0000 7f7ea67fc640 1 --2- 172.21.15.100:0/3116578912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ea8108720 0x7f7ea819f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:40.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.968+0000 7f7ea67fc640 1 -- 172.21.15.100:0/3116578912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ea8102490 msgr2=0x7f7ea8199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:40.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.968+0000 7f7ea67fc640 1 --2- 172.21.15.100:0/3116578912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ea8102490 0x7f7ea8199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:40.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.968+0000 7f7ea67fc640 1 -- 172.21.15.100:0/3116578912 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea819fdf0 con 0x7f7ea8107dd0 2024-07-26T13:30:40.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.968+0000 7f7ea77fe640 1 --2- 172.21.15.100:0/3116578912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ea8108720 0x7f7ea819f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:30:40.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.968+0000 7f7ea67fc640 1 --2- 172.21.15.100:0/3116578912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea8107dd0 0x7f7ea819a420 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7e90005940 tx=0x7f7e900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:30:40.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.968+0000 7f7e9ffff640 1 -- 172.21.15.100:0/3116578912 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9003f030 con 0x7f7ea8107dd0 2024-07-26T13:30:40.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.968+0000 7f7e9ffff640 1 -- 172.21.15.100:0/3116578912 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7e90031800 con 0x7f7ea8107dd0 2024-07-26T13:30:40.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.968+0000 7f7ead3de640 1 -- 172.21.15.100:0/3116578912 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ea81a0080 con 0x7f7ea8107dd0 2024-07-26T13:30:40.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.968+0000 7f7e9ffff640 1 -- 172.21.15.100:0/3116578912 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e90035070 con 0x7f7ea8107dd0 2024-07-26T13:30:40.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.968+0000 7f7ead3de640 1 -- 172.21.15.100:0/3116578912 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ea81a6840 con 0x7f7ea8107dd0 2024-07-26T13:30:40.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.969+0000 7f7ead3de640 1 -- 172.21.15.100:0/3116578912 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ea81028a0 con 0x7f7ea8107dd0 2024-07-26T13:30:40.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.971+0000 7f7e9ffff640 1 -- 172.21.15.100:0/3116578912 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7f7e90042030 con 0x7f7ea8107dd0 2024-07-26T13:30:40.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.971+0000 7f7e9ffff640 1 --2- 172.21.15.100:0/3116578912 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f7e940b3c10 0x7f7e940b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:40.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.972+0000 7f7ea6ffd640 1 --2- 172.21.15.100:0/3116578912 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f7e940b3c10 0x7f7e940b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:40.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.972+0000 7f7e9ffff640 1 -- 172.21.15.100:0/3116578912 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f7e90102910 con 0x7f7ea8107dd0 2024-07-26T13:30:40.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.972+0000 7f7ea6ffd640 1 --2- 172.21.15.100:0/3116578912 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f7e940b3c10 0x7f7e940b6120 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7e98005fd0 tx=0x7f7e9800b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:30:40.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:40.975+0000 7f7e9ffff640 1 -- 172.21.15.100:0/3116578912 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f7e900cbb60 con 0x7f7ea8107dd0 2024-07-26T13:30:41.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:41.153+0000 7f7ead3de640 1 -- 172.21.15.100:0/3116578912 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f7ea819afb0 con 0x7f7ea8107dd0 2024-07-26T13:30:41.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:41.158+0000 7f7e9ffff640 1 -- 172.21.15.100:0/3116578912 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v19)=0 v19) ==== 115+0+0 (secure 0 0 0) 0x7f7e900cb460 con 0x7f7ea8107dd0 2024-07-26T13:30:41.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:41.160+0000 7f7ead3de640 1 -- 172.21.15.100:0/3116578912 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f7e940b3c10 msgr2=0x7f7e940b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:41.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:41.160+0000 7f7ead3de640 1 --2- 172.21.15.100:0/3116578912 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f7e940b3c10 0x7f7e940b6120 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7e98005fd0 tx=0x7f7e9800b390 comp rx=0 tx=0).stop 2024-07-26T13:30:41.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:41.160+0000 7f7ead3de640 1 -- 172.21.15.100:0/3116578912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea8107dd0 msgr2=0x7f7ea819a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:41.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:41.160+0000 7f7ead3de640 1 --2- 172.21.15.100:0/3116578912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea8107dd0 0x7f7ea819a420 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7e90005940 tx=0x7f7e900042f0 comp rx=0 tx=0).stop 2024-07-26T13:30:41.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:41.160+0000 7f7ead3de640 1 -- 172.21.15.100:0/3116578912 shutdown_connections 2024-07-26T13:30:41.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:41.160+0000 7f7ead3de640 1 --2- 172.21.15.100:0/3116578912 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f7e940b3c10 0x7f7e940b6120 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:41.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:41.160+0000 7f7ead3de640 1 --2- 172.21.15.100:0/3116578912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ea8108720 0x7f7ea819f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:41.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:41.160+0000 7f7ead3de640 1 --2- 172.21.15.100:0/3116578912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ea8107dd0 0x7f7ea819a420 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:41.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:41.160+0000 7f7ead3de640 1 --2- 172.21.15.100:0/3116578912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ea8102490 0x7f7ea8199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:41.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:41.160+0000 7f7ead3de640 1 -- 172.21.15.100:0/3116578912 >> 172.21.15.100:0/3116578912 conn(0x7f7ea8079160 msgr2=0x7f7ea8103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:30:41.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:41.160+0000 7f7ead3de640 1 -- 172.21.15.100:0/3116578912 shutdown_connections 2024-07-26T13:30:41.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:41.160+0000 7f7ead3de640 1 -- 172.21.15.100:0/3116578912 wait complete. 2024-07-26T13:30:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:41 smithi100 ceph-mon[32599]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:41 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/3801950642' entity='client.admin' 2024-07-26T13:30:41.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:41 smithi167 ceph-mon[37140]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:41.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:41 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/3801950642' entity='client.admin' 2024-07-26T13:30:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:41 smithi138 ceph-mon[36954]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:41 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/3801950642' entity='client.admin' 2024-07-26T13:30:42.235 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:42 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/3116578912' entity='client.admin' 2024-07-26T13:30:42.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.234+0000 7fe1726f8640 1 -- 172.21.15.100:0/3290182777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe16c101cd0 msgr2=0x7fe16c1020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:42.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.234+0000 7fe1726f8640 1 --2- 172.21.15.100:0/3290182777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe16c101cd0 0x7fe16c1020e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe15c00aa10 tx=0x7fe15c02b720 comp rx=0 tx=0).stop 2024-07-26T13:30:42.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.234+0000 7fe1726f8640 1 -- 172.21.15.100:0/3290182777 shutdown_connections 2024-07-26T13:30:42.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.234+0000 7fe1726f8640 1 --2- 172.21.15.100:0/3290182777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe16c107f60 0x7fe16c10e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:42.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.234+0000 7fe1726f8640 1 --2- 172.21.15.100:0/3290182777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe16c107610 0x7fe16c107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:42.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.234+0000 7fe1726f8640 1 --2- 172.21.15.100:0/3290182777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe16c101cd0 0x7fe16c1020e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:42.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.234+0000 7fe1726f8640 1 -- 172.21.15.100:0/3290182777 >> 172.21.15.100:0/3290182777 conn(0x7fe16c0798f0 msgr2=0x7fe16c079d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:30:42.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.234+0000 7fe1726f8640 1 -- 172.21.15.100:0/3290182777 shutdown_connections 2024-07-26T13:30:42.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.234+0000 7fe1726f8640 1 -- 172.21.15.100:0/3290182777 wait complete. 2024-07-26T13:30:42.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.235+0000 7fe1726f8640 1 Processor -- start 2024-07-26T13:30:42.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.235+0000 7fe1726f8640 1 -- start start 2024-07-26T13:30:42.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.236+0000 7fe1726f8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe16c101cd0 0x7fe16c197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:42.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.236+0000 7fe1726f8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe16c107610 0x7fe16c198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:42.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.236+0000 7fe1726f8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe16c107f60 0x7fe16c1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:42.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.236+0000 7fe1726f8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c1112d0 con 0x7fe16c101cd0 2024-07-26T13:30:42.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.236+0000 7fe1726f8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c111150 con 0x7fe16c107f60 2024-07-26T13:30:42.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.236+0000 7fe1726f8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c111450 con 0x7fe16c107610 2024-07-26T13:30:42.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.236+0000 7fe16bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe16c101cd0 0x7fe16c197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:42.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.236+0000 7fe16bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe16c101cd0 0x7fe16c197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56386/0 (socket says 172.21.15.100:56386) 2024-07-26T13:30:42.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.236+0000 7fe16bfff640 1 -- 172.21.15.100:0/61886599 learned_addr learned my addr 172.21.15.100:0/61886599 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:30:42.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.236+0000 7fe170c95640 1 --2- 172.21.15.100:0/61886599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe16c107f60 0x7fe16c1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:42.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.236+0000 7fe16b7fe640 1 --2- 172.21.15.100:0/61886599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe16c107610 0x7fe16c198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:42.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.236+0000 7fe16bfff640 1 -- 172.21.15.100:0/61886599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe16c107610 msgr2=0x7fe16c198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:42.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.236+0000 7fe16bfff640 1 --2- 172.21.15.100:0/61886599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe16c107610 0x7fe16c198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:42.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.236+0000 7fe16bfff640 1 -- 172.21.15.100:0/61886599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe16c107f60 msgr2=0x7fe16c1985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:42.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.236+0000 7fe16bfff640 1 --2- 172.21.15.100:0/61886599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe16c107f60 0x7fe16c1985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:42.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.236+0000 7fe16bfff640 1 -- 172.21.15.100:0/61886599 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe16c1a8140 con 0x7fe16c101cd0 2024-07-26T13:30:42.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.236+0000 7fe16b7fe640 1 --2- 172.21.15.100:0/61886599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe16c107610 0x7fe16c198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:30:42.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.236+0000 7fe16bfff640 1 --2- 172.21.15.100:0/61886599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe16c101cd0 0x7fe16c197b50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe15c006010 tx=0x7fe15c003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:30:42.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.237+0000 7fe1697fa640 1 -- 172.21.15.100:0/61886599 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c03f070 con 0x7fe16c101cd0 2024-07-26T13:30:42.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.237+0000 7fe1726f8640 1 -- 172.21.15.100:0/61886599 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe16c1a83d0 con 0x7fe16c101cd0 2024-07-26T13:30:42.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.237+0000 7fe1697fa640 1 -- 172.21.15.100:0/61886599 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe15c030db0 con 0x7fe16c101cd0 2024-07-26T13:30:42.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.237+0000 7fe1697fa640 1 -- 172.21.15.100:0/61886599 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c036040 con 0x7fe16c101cd0 2024-07-26T13:30:42.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.237+0000 7fe1726f8640 1 -- 172.21.15.100:0/61886599 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe16c1a88b0 con 0x7fe16c101cd0 2024-07-26T13:30:42.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.238+0000 7fe1726f8640 1 -- 172.21.15.100:0/61886599 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe16c1020e0 con 0x7fe16c101cd0 2024-07-26T13:30:42.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.239+0000 7fe1697fa640 1 -- 172.21.15.100:0/61886599 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7fe15c042070 con 0x7fe16c101cd0 2024-07-26T13:30:42.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.240+0000 7fe1697fa640 1 --2- 172.21.15.100:0/61886599 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe1540b3b90 0x7fe1540b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:42.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.240+0000 7fe16b7fe640 1 --2- 172.21.15.100:0/61886599 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe1540b3b90 0x7fe1540b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:42.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.240+0000 7fe1697fa640 1 -- 172.21.15.100:0/61886599 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fe15c102cb0 con 0x7fe16c101cd0 2024-07-26T13:30:42.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.240+0000 7fe16b7fe640 1 --2- 172.21.15.100:0/61886599 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe1540b3b90 0x7fe1540b60a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe16c1072a0 tx=0x7fe160005ec0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:30:42.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.243+0000 7fe1697fa640 1 -- 172.21.15.100:0/61886599 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fe15c0cbf00 con 0x7fe16c101cd0 2024-07-26T13:30:42.275 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:30:42.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.427+0000 7fe1726f8640 1 -- 172.21.15.100:0/61886599 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7fe16c198e50 con 0x7fe16c101cd0 2024-07-26T13:30:42.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.432+0000 7fe1697fa640 1 -- 172.21.15.100:0/61886599 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v20)=0 v20) ==== 120+0+0 (secure 0 0 0) 0x7fe15c0d41d0 con 0x7fe16c101cd0 2024-07-26T13:30:42.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.434+0000 7fe1726f8640 1 -- 172.21.15.100:0/61886599 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe1540b3b90 msgr2=0x7fe1540b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:42.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.434+0000 7fe1726f8640 1 --2- 172.21.15.100:0/61886599 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe1540b3b90 0x7fe1540b60a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe16c1072a0 tx=0x7fe160005ec0 comp rx=0 tx=0).stop 2024-07-26T13:30:42.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.434+0000 7fe1726f8640 1 -- 172.21.15.100:0/61886599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe16c101cd0 msgr2=0x7fe16c197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:42.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.434+0000 7fe1726f8640 1 --2- 172.21.15.100:0/61886599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe16c101cd0 0x7fe16c197b50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe15c006010 tx=0x7fe15c003d50 comp rx=0 tx=0).stop 2024-07-26T13:30:42.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.434+0000 7fe1726f8640 1 -- 172.21.15.100:0/61886599 shutdown_connections 2024-07-26T13:30:42.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.434+0000 7fe1726f8640 1 --2- 172.21.15.100:0/61886599 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe1540b3b90 0x7fe1540b60a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:42.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.434+0000 7fe1726f8640 1 --2- 172.21.15.100:0/61886599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe16c107f60 0x7fe16c1985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:42.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.434+0000 7fe1726f8640 1 --2- 172.21.15.100:0/61886599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe16c107610 0x7fe16c198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:42.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.434+0000 7fe1726f8640 1 --2- 172.21.15.100:0/61886599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe16c101cd0 0x7fe16c197b50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:42.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.434+0000 7fe1726f8640 1 -- 172.21.15.100:0/61886599 >> 172.21.15.100:0/61886599 conn(0x7fe16c0798f0 msgr2=0x7fe16c1057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:30:42.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.435+0000 7fe1726f8640 1 -- 172.21.15.100:0/61886599 shutdown_connections 2024-07-26T13:30:42.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.435+0000 7fe1726f8640 1 -- 172.21.15.100:0/61886599 wait complete. 2024-07-26T13:30:42.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:42 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/3116578912' entity='client.admin' 2024-07-26T13:30:42.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:42 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/3116578912' entity='client.admin' 2024-07-26T13:30:42.957 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:30:42.958 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:30:42.959 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:30:43.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:43 smithi138 ceph-mon[36954]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:43.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:43 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/61886599' entity='client.admin' 2024-07-26T13:30:43.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:43 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/2684802549' entity='client.admin' 2024-07-26T13:30:43.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:43 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:30:43.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:43 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:43.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:43 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:30:43.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:43 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:43 smithi100 ceph-mon[32599]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:43 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/61886599' entity='client.admin' 2024-07-26T13:30:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:43 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2684802549' entity='client.admin' 2024-07-26T13:30:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:43 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:30:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:43 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:43 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:30:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:43 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:43.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:43 smithi167 ceph-mon[37140]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:43.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:43 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/61886599' entity='client.admin' 2024-07-26T13:30:43.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:43 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/2684802549' entity='client.admin' 2024-07-26T13:30:43.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:43 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:30:43.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:43 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:43.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:43 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:30:43.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:43 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:44.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.674+0000 7fb71d6f6640 1 -- 172.21.15.100:0/1016825687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb718108720 msgr2=0x7fb71810ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:44.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.674+0000 7fb71d6f6640 1 --2- 172.21.15.100:0/1016825687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb718108720 0x7fb71810ef50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb710009ca0 tx=0x7fb71002a8c0 comp rx=0 tx=0).stop 2024-07-26T13:30:44.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.674+0000 7fb71d6f6640 1 -- 172.21.15.100:0/1016825687 shutdown_connections 2024-07-26T13:30:44.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.674+0000 7fb71d6f6640 1 --2- 172.21.15.100:0/1016825687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb718108720 0x7fb71810ef50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:44.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.674+0000 7fb71d6f6640 1 --2- 172.21.15.100:0/1016825687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb718107dd0 0x7fb7181081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:44.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.674+0000 7fb71d6f6640 1 --2- 172.21.15.100:0/1016825687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb718102490 0x7fb7181028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:44.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.674+0000 7fb71d6f6640 1 -- 172.21.15.100:0/1016825687 >> 172.21.15.100:0/1016825687 conn(0x7fb718079160 msgr2=0x7fb718079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:30:44.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.674+0000 7fb71d6f6640 1 -- 172.21.15.100:0/1016825687 shutdown_connections 2024-07-26T13:30:44.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.674+0000 7fb71d6f6640 1 -- 172.21.15.100:0/1016825687 wait complete. 2024-07-26T13:30:44.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.675+0000 7fb71d6f6640 1 Processor -- start 2024-07-26T13:30:44.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.675+0000 7fb71d6f6640 1 -- start start 2024-07-26T13:30:44.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.675+0000 7fb71d6f6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb718102490 0x7fb718198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:44.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.675+0000 7fb71d6f6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb718107dd0 0x7fb7181992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:44.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.675+0000 7fb71d6f6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb718108720 0x7fb71819e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:44.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.675+0000 7fb71d6f6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb7181122b0 con 0x7fb718108720 2024-07-26T13:30:44.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.675+0000 7fb71d6f6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb718112130 con 0x7fb718102490 2024-07-26T13:30:44.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.675+0000 7fb71d6f6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb718112430 con 0x7fb718107dd0 2024-07-26T13:30:44.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.675+0000 7fb7177fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb718108720 0x7fb71819e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:44.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.675+0000 7fb7177fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb718108720 0x7fb71819e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56416/0 (socket says 172.21.15.100:56416) 2024-07-26T13:30:44.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.675+0000 7fb7177fe640 1 -- 172.21.15.100:0/1234813929 learned_addr learned my addr 172.21.15.100:0/1234813929 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:30:44.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.675+0000 7fb7167fc640 1 --2- 172.21.15.100:0/1234813929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb718107dd0 0x7fb7181992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:44.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.676+0000 7fb716ffd640 1 --2- 172.21.15.100:0/1234813929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb718102490 0x7fb718198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:44.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.676+0000 7fb7177fe640 1 -- 172.21.15.100:0/1234813929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb718107dd0 msgr2=0x7fb7181992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:44.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.676+0000 7fb7177fe640 1 --2- 172.21.15.100:0/1234813929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb718107dd0 0x7fb7181992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:44.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.676+0000 7fb7177fe640 1 -- 172.21.15.100:0/1234813929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb718102490 msgr2=0x7fb718198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:44.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.676+0000 7fb7177fe640 1 --2- 172.21.15.100:0/1234813929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb718102490 0x7fb718198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:44.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.676+0000 7fb7177fe640 1 -- 172.21.15.100:0/1234813929 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb71819ec90 con 0x7fb718108720 2024-07-26T13:30:44.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.676+0000 7fb716ffd640 1 --2- 172.21.15.100:0/1234813929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb718102490 0x7fb718198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:30:44.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.676+0000 7fb7177fe640 1 --2- 172.21.15.100:0/1234813929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb718108720 0x7fb71819e560 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb710005e30 tx=0x7fb7100305c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:30:44.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.676+0000 7fb6fffff640 1 -- 172.21.15.100:0/1234813929 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb710049070 con 0x7fb718108720 2024-07-26T13:30:44.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.676+0000 7fb71d6f6640 1 -- 172.21.15.100:0/1234813929 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb71819ef20 con 0x7fb718108720 2024-07-26T13:30:44.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.676+0000 7fb6fffff640 1 -- 172.21.15.100:0/1234813929 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb710030bc0 con 0x7fb718108720 2024-07-26T13:30:44.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.676+0000 7fb6fffff640 1 -- 172.21.15.100:0/1234813929 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb710004410 con 0x7fb718108720 2024-07-26T13:30:44.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.676+0000 7fb71d6f6640 1 -- 172.21.15.100:0/1234813929 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb71806a890 con 0x7fb718108720 2024-07-26T13:30:44.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.677+0000 7fb71d6f6640 1 -- 172.21.15.100:0/1234813929 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7181028a0 con 0x7fb718108720 2024-07-26T13:30:44.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.679+0000 7fb6fffff640 1 -- 172.21.15.100:0/1234813929 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7fb71004c070 con 0x7fb718108720 2024-07-26T13:30:44.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.679+0000 7fb6fffff640 1 --2- 172.21.15.100:0/1234813929 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb7000b3b90 0x7fb7000b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:44.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.679+0000 7fb716ffd640 1 --2- 172.21.15.100:0/1234813929 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb7000b3b90 0x7fb7000b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:44.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.679+0000 7fb6fffff640 1 -- 172.21.15.100:0/1234813929 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fb710104870 con 0x7fb718108720 2024-07-26T13:30:44.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.680+0000 7fb716ffd640 1 --2- 172.21.15.100:0/1234813929 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb7000b3b90 0x7fb7000b60a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb7080058c0 tx=0x7fb708005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:30:44.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.684+0000 7fb6fffff640 1 -- 172.21.15.100:0/1234813929 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fb710082600 con 0x7fb718108720 2024-07-26T13:30:44.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:44 smithi100 ceph-mon[32599]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:44.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:44 smithi167 ceph-mon[37140]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:44.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.861+0000 7fb71d6f6640 1 -- 172.21.15.100:0/1234813929 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7fb71806b2b0 con 0x7fb718108720 2024-07-26T13:30:44.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.865+0000 7fb6fffff640 1 -- 172.21.15.100:0/1234813929 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v22)=0 v22) ==== 123+0+0 (secure 0 0 0) 0x7fb7100cdac0 con 0x7fb718108720 2024-07-26T13:30:44.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.866+0000 7fb6fffff640 1 -- 172.21.15.100:0/1234813929 <== mon.0 v2:172.21.15.100:3300/0 8 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb71010a080 con 0x7fb718108720 2024-07-26T13:30:44.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.868+0000 7fb71d6f6640 1 -- 172.21.15.100:0/1234813929 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb7000b3b90 msgr2=0x7fb7000b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:44.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.868+0000 7fb71d6f6640 1 --2- 172.21.15.100:0/1234813929 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb7000b3b90 0x7fb7000b60a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb7080058c0 tx=0x7fb708005850 comp rx=0 tx=0).stop 2024-07-26T13:30:44.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.868+0000 7fb71d6f6640 1 -- 172.21.15.100:0/1234813929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb718108720 msgr2=0x7fb71819e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:44.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.868+0000 7fb71d6f6640 1 --2- 172.21.15.100:0/1234813929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb718108720 0x7fb71819e560 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb710005e30 tx=0x7fb7100305c0 comp rx=0 tx=0).stop 2024-07-26T13:30:44.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.868+0000 7fb71d6f6640 1 -- 172.21.15.100:0/1234813929 shutdown_connections 2024-07-26T13:30:44.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.868+0000 7fb71d6f6640 1 --2- 172.21.15.100:0/1234813929 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb7000b3b90 0x7fb7000b60a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:44.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.868+0000 7fb71d6f6640 1 --2- 172.21.15.100:0/1234813929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb718108720 0x7fb71819e560 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:44.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.868+0000 7fb71d6f6640 1 --2- 172.21.15.100:0/1234813929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb718107dd0 0x7fb7181992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:44.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.868+0000 7fb71d6f6640 1 --2- 172.21.15.100:0/1234813929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb718102490 0x7fb718198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:44.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.868+0000 7fb71d6f6640 1 -- 172.21.15.100:0/1234813929 >> 172.21.15.100:0/1234813929 conn(0x7fb718079160 msgr2=0x7fb718104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:30:44.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.868+0000 7fb71d6f6640 1 -- 172.21.15.100:0/1234813929 shutdown_connections 2024-07-26T13:30:44.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:44.868+0000 7fb71d6f6640 1 -- 172.21.15.100:0/1234813929 wait complete. 2024-07-26T13:30:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:44 smithi138 ceph-mon[36954]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:46.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.473+0000 7f50b3264640 1 -- 172.21.15.100:0/1414105386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50ac109820 msgr2=0x7f50ac110050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:46.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.473+0000 7f50b3264640 1 --2- 172.21.15.100:0/1414105386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50ac109820 0x7f50ac110050 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f50a4009ca0 tx=0x7f50a402a8c0 comp rx=0 tx=0).stop 2024-07-26T13:30:46.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.473+0000 7f50b3264640 1 -- 172.21.15.100:0/1414105386 shutdown_connections 2024-07-26T13:30:46.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.473+0000 7f50b3264640 1 --2- 172.21.15.100:0/1414105386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50ac109820 0x7f50ac110050 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:46.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.473+0000 7f50b3264640 1 --2- 172.21.15.100:0/1414105386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50ac108ed0 0x7f50ac1092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:46.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.473+0000 7f50b3264640 1 --2- 172.21.15.100:0/1414105386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50ac103590 0x7f50ac1039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:46.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.473+0000 7f50b3264640 1 -- 172.21.15.100:0/1414105386 >> 172.21.15.100:0/1414105386 conn(0x7f50ac07b280 msgr2=0x7f50ac07b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:30:46.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.473+0000 7f50b3264640 1 -- 172.21.15.100:0/1414105386 shutdown_connections 2024-07-26T13:30:46.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.473+0000 7f50b3264640 1 -- 172.21.15.100:0/1414105386 wait complete. 2024-07-26T13:30:46.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.473+0000 7f50b3264640 1 Processor -- start 2024-07-26T13:30:46.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.473+0000 7f50b3264640 1 -- start start 2024-07-26T13:30:46.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.474+0000 7f50b3264640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50ac103590 0x7f50ac075140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:46.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.474+0000 7f50b3264640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50ac108ed0 0x7f50ac075680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:46.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.474+0000 7f50b3264640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50ac109820 0x7f50ac075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:46.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.474+0000 7f50b3264640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac1133b0 con 0x7f50ac109820 2024-07-26T13:30:46.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.474+0000 7f50b3264640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac113230 con 0x7f50ac103590 2024-07-26T13:30:46.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.474+0000 7f50b3264640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac113530 con 0x7f50ac108ed0 2024-07-26T13:30:46.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.474+0000 7f50b1801640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50ac109820 0x7f50ac075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:46.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.474+0000 7f50b1801640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50ac109820 0x7f50ac075bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56404/0 (socket says 172.21.15.100:56404) 2024-07-26T13:30:46.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.474+0000 7f50b1801640 1 -- 172.21.15.100:0/2684802549 learned_addr learned my addr 172.21.15.100:0/2684802549 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:30:46.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.474+0000 7f50b1000640 1 --2- 172.21.15.100:0/2684802549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50ac103590 0x7f50ac075140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:46.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.474+0000 7f50abfff640 1 --2- 172.21.15.100:0/2684802549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50ac108ed0 0x7f50ac075680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:46.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.474+0000 7f50b1801640 1 -- 172.21.15.100:0/2684802549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50ac108ed0 msgr2=0x7f50ac075680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:46.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.474+0000 7f50b1801640 1 --2- 172.21.15.100:0/2684802549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50ac108ed0 0x7f50ac075680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:46.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.474+0000 7f50b1801640 1 -- 172.21.15.100:0/2684802549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50ac103590 msgr2=0x7f50ac075140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:46.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.474+0000 7f50b1801640 1 --2- 172.21.15.100:0/2684802549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50ac103590 0x7f50ac075140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:46.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.474+0000 7f50b1801640 1 -- 172.21.15.100:0/2684802549 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50ac1a80a0 con 0x7f50ac109820 2024-07-26T13:30:46.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.474+0000 7f50abfff640 1 --2- 172.21.15.100:0/2684802549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50ac108ed0 0x7f50ac075680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:30:46.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.474+0000 7f50b1000640 1 --2- 172.21.15.100:0/2684802549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50ac103590 0x7f50ac075140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:30:46.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.474+0000 7f50b1801640 1 --2- 172.21.15.100:0/2684802549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50ac109820 0x7f50ac075bc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f50a4006010 tx=0x7f50a40093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:30:46.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.474+0000 7f50a9ffb640 1 -- 172.21.15.100:0/2684802549 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a4003b70 con 0x7f50ac109820 2024-07-26T13:30:46.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.475+0000 7f50b3264640 1 -- 172.21.15.100:0/2684802549 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50ac1a8330 con 0x7f50ac109820 2024-07-26T13:30:46.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.475+0000 7f50a9ffb640 1 -- 172.21.15.100:0/2684802549 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f50a4003d10 con 0x7f50ac109820 2024-07-26T13:30:46.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.475+0000 7f50a9ffb640 1 -- 172.21.15.100:0/2684802549 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a4004600 con 0x7f50ac109820 2024-07-26T13:30:46.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.475+0000 7f50b3264640 1 -- 172.21.15.100:0/2684802549 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50ac1a88c0 con 0x7f50ac109820 2024-07-26T13:30:46.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.476+0000 7f50b3264640 1 -- 172.21.15.100:0/2684802549 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50ac1039a0 con 0x7f50ac109820 2024-07-26T13:30:46.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.477+0000 7f50a9ffb640 1 -- 172.21.15.100:0/2684802549 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7f50a4004b40 con 0x7f50ac109820 2024-07-26T13:30:46.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.478+0000 7f50a9ffb640 1 --2- 172.21.15.100:0/2684802549 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f509c0b3b90 0x7f509c0b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:46.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.478+0000 7f50b1000640 1 --2- 172.21.15.100:0/2684802549 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f509c0b3b90 0x7f509c0b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:46.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.478+0000 7f50a9ffb640 1 -- 172.21.15.100:0/2684802549 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f50a4103390 con 0x7f50ac109820 2024-07-26T13:30:46.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.479+0000 7f50b1000640 1 --2- 172.21.15.100:0/2684802549 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f509c0b3b90 0x7f509c0b60a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f50ac076b90 tx=0x7f50a0005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:30:46.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.481+0000 7f50a9ffb640 1 -- 172.21.15.100:0/2684802549 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f50a40cc5e0 con 0x7f50ac109820 2024-07-26T13:30:46.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.661+0000 7f50b3264640 1 -- 172.21.15.100:0/2684802549 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f50ac076440 con 0x7f50ac109820 2024-07-26T13:30:46.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.665+0000 7f50a9ffb640 1 -- 172.21.15.100:0/2684802549 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v21)=0 v21) ==== 138+0+0 (secure 0 0 0) 0x7f50ac076440 con 0x7f50ac109820 2024-07-26T13:30:46.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.667+0000 7f50b3264640 1 -- 172.21.15.100:0/2684802549 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f509c0b3b90 msgr2=0x7f509c0b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:46.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.667+0000 7f50b3264640 1 --2- 172.21.15.100:0/2684802549 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f509c0b3b90 0x7f509c0b60a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f50ac076b90 tx=0x7f50a0005f90 comp rx=0 tx=0).stop 2024-07-26T13:30:46.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.667+0000 7f50b3264640 1 -- 172.21.15.100:0/2684802549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50ac109820 msgr2=0x7f50ac075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:46.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.667+0000 7f50b3264640 1 --2- 172.21.15.100:0/2684802549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50ac109820 0x7f50ac075bc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f50a4006010 tx=0x7f50a40093d0 comp rx=0 tx=0).stop 2024-07-26T13:30:46.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.668+0000 7f50b3264640 1 -- 172.21.15.100:0/2684802549 shutdown_connections 2024-07-26T13:30:46.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.668+0000 7f50b3264640 1 --2- 172.21.15.100:0/2684802549 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f509c0b3b90 0x7f509c0b60a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:46.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.668+0000 7f50b3264640 1 --2- 172.21.15.100:0/2684802549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50ac109820 0x7f50ac075bc0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:46.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.668+0000 7f50b3264640 1 --2- 172.21.15.100:0/2684802549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50ac108ed0 0x7f50ac075680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:46.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.668+0000 7f50b3264640 1 --2- 172.21.15.100:0/2684802549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50ac103590 0x7f50ac075140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:46.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.668+0000 7f50b3264640 1 -- 172.21.15.100:0/2684802549 >> 172.21.15.100:0/2684802549 conn(0x7f50ac07b280 msgr2=0x7f50ac106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:30:46.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.668+0000 7f50b3264640 1 -- 172.21.15.100:0/2684802549 shutdown_connections 2024-07-26T13:30:46.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:42.668+0000 7f50b3264640 1 -- 172.21.15.100:0/2684802549 wait complete. 2024-07-26T13:30:46.112 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:45 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/1234813929' entity='client.admin' 2024-07-26T13:30:46.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:45 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/1234813929' entity='client.admin' 2024-07-26T13:30:46.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:45 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/1234813929' entity='client.admin' 2024-07-26T13:30:46.531 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:30:46.711 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:30:46.711 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:30:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:46 smithi138 ceph-mon[36954]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:47.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:46 smithi100 ceph-mon[32599]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:47.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:46 smithi167 ceph-mon[37140]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:47.393 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'bf83e2e6-4b52-11ef-bcb7-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi100') 2024-07-26T13:30:47.393 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'bf83e2e6-4b52-11ef-bcb7-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi100') 2024-07-26T13:30:47.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.912+0000 7f711b89f640 1 -- 172.21.15.100:0/1868576300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7114108720 msgr2=0x7f711410ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:47.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.912+0000 7f711b89f640 1 --2- 172.21.15.100:0/1868576300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7114108720 0x7f711410ef50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f710c009ca0 tx=0x7f710c02a8c0 comp rx=0 tx=0).stop 2024-07-26T13:30:47.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.912+0000 7f711b89f640 1 -- 172.21.15.100:0/1868576300 shutdown_connections 2024-07-26T13:30:47.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.912+0000 7f711b89f640 1 --2- 172.21.15.100:0/1868576300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7114108720 0x7f711410ef50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:47.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.912+0000 7f711b89f640 1 --2- 172.21.15.100:0/1868576300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7114107dd0 0x7f71141081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:47.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.912+0000 7f711b89f640 1 --2- 172.21.15.100:0/1868576300 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7114102490 0x7f71141028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:47.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.912+0000 7f711b89f640 1 -- 172.21.15.100:0/1868576300 >> 172.21.15.100:0/1868576300 conn(0x7f7114079160 msgr2=0x7f7114079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:30:47.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.912+0000 7f711b89f640 1 -- 172.21.15.100:0/1868576300 shutdown_connections 2024-07-26T13:30:47.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.912+0000 7f711b89f640 1 -- 172.21.15.100:0/1868576300 wait complete. 2024-07-26T13:30:47.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.913+0000 7f711b89f640 1 Processor -- start 2024-07-26T13:30:47.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.913+0000 7f711b89f640 1 -- start start 2024-07-26T13:30:47.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.914+0000 7f711b89f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7114102490 0x7f711419ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:47.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.914+0000 7f711b89f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7114107dd0 0x7f711419b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:47.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.914+0000 7f711b89f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7114108720 0x7f711419b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:47.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.914+0000 7f711b89f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f71141123f0 con 0x7f7114107dd0 2024-07-26T13:30:47.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.914+0000 7f711b89f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7114112270 con 0x7f7114108720 2024-07-26T13:30:47.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.914+0000 7f711b89f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7114112570 con 0x7f7114102490 2024-07-26T13:30:47.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.914+0000 7f7118e3a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7114107dd0 0x7f711419b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:47.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.914+0000 7f7118e3a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7114107dd0 0x7f711419b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47100/0 (socket says 172.21.15.100:47100) 2024-07-26T13:30:47.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.914+0000 7f7118e3a640 1 -- 172.21.15.100:0/3642760252 learned_addr learned my addr 172.21.15.100:0/3642760252 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:30:47.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.914+0000 7f7119e3c640 1 --2- 172.21.15.100:0/3642760252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7114108720 0x7f711419b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:47.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.914+0000 7f711963b640 1 --2- 172.21.15.100:0/3642760252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7114102490 0x7f711419ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:47.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.914+0000 7f7118e3a640 1 -- 172.21.15.100:0/3642760252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7114102490 msgr2=0x7f711419ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:47.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.914+0000 7f7118e3a640 1 --2- 172.21.15.100:0/3642760252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7114102490 0x7f711419ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:47.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.914+0000 7f7118e3a640 1 -- 172.21.15.100:0/3642760252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7114108720 msgr2=0x7f711419b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:47.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.914+0000 7f7118e3a640 1 --2- 172.21.15.100:0/3642760252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7114108720 0x7f711419b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:47.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.914+0000 7f7118e3a640 1 -- 172.21.15.100:0/3642760252 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71141a5fb0 con 0x7f7114107dd0 2024-07-26T13:30:47.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.914+0000 7f7119e3c640 1 --2- 172.21.15.100:0/3642760252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7114108720 0x7f711419b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:30:47.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.914+0000 7f7118e3a640 1 --2- 172.21.15.100:0/3642760252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7114107dd0 0x7f711419b2c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f710800c970 tx=0x7f710800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:30:47.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.914+0000 7f71127fc640 1 -- 172.21.15.100:0/3642760252 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7108007bb0 con 0x7f7114107dd0 2024-07-26T13:30:47.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.914+0000 7f71127fc640 1 -- 172.21.15.100:0/3642760252 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7108007d50 con 0x7f7114107dd0 2024-07-26T13:30:47.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.914+0000 7f71127fc640 1 -- 172.21.15.100:0/3642760252 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7108018730 con 0x7f7114107dd0 2024-07-26T13:30:47.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.914+0000 7f711b89f640 1 -- 172.21.15.100:0/3642760252 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71141a62a0 con 0x7f7114107dd0 2024-07-26T13:30:47.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.915+0000 7f711b89f640 1 -- 172.21.15.100:0/3642760252 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71141a67e0 con 0x7f7114107dd0 2024-07-26T13:30:47.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.915+0000 7f711b89f640 1 -- 172.21.15.100:0/3642760252 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71141028a0 con 0x7f7114107dd0 2024-07-26T13:30:47.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.917+0000 7f71127fc640 1 -- 172.21.15.100:0/3642760252 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7f710800f990 con 0x7f7114107dd0 2024-07-26T13:30:47.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.918+0000 7f71127fc640 1 --2- 172.21.15.100:0/3642760252 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f70fc0b3c10 0x7f70fc0b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:47.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.918+0000 7f711963b640 1 --2- 172.21.15.100:0/3642760252 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f70fc0b3c10 0x7f70fc0b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:47.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.918+0000 7f71127fc640 1 -- 172.21.15.100:0/3642760252 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f71080e23d0 con 0x7f7114107dd0 2024-07-26T13:30:47.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.918+0000 7f711963b640 1 --2- 172.21.15.100:0/3642760252 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f70fc0b3c10 0x7f70fc0b6120 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f711406b410 tx=0x7f7104005cc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:30:47.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:47.921+0000 7f71127fc640 1 -- 172.21.15.100:0/3642760252 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f7108014030 con 0x7f7114107dd0 2024-07-26T13:30:48.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.100+0000 7f711b89f640 1 -- 172.21.15.100:0/3642760252 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f711406acc0 con 0x7f7114107dd0 2024-07-26T13:30:48.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.105+0000 7f71127fc640 1 -- 172.21.15.100:0/3642760252 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v23)=0 v23) ==== 106+0+0 (secure 0 0 0) 0x7f71080ab6a0 con 0x7f7114107dd0 2024-07-26T13:30:48.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.107+0000 7f5a9c6ee640 1 -- 172.21.15.100:0/4089514102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a94108720 msgr2=0x7f5a9410ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:48.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.107+0000 7f5a9c6ee640 1 --2- 172.21.15.100:0/4089514102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a94108720 0x7f5a9410ef50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c009ca0 tx=0x7f5a8c02a8c0 comp rx=0 tx=0).stop 2024-07-26T13:30:48.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.107+0000 7f711b89f640 1 -- 172.21.15.100:0/3642760252 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f70fc0b3c10 msgr2=0x7f70fc0b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:48.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.107+0000 7f711b89f640 1 --2- 172.21.15.100:0/3642760252 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f70fc0b3c10 0x7f70fc0b6120 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f711406b410 tx=0x7f7104005cc0 comp rx=0 tx=0).stop 2024-07-26T13:30:48.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.107+0000 7f711b89f640 1 -- 172.21.15.100:0/3642760252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7114107dd0 msgr2=0x7f711419b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:48.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.107+0000 7f711b89f640 1 --2- 172.21.15.100:0/3642760252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7114107dd0 0x7f711419b2c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f710800c970 tx=0x7f710800ce30 comp rx=0 tx=0).stop 2024-07-26T13:30:48.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.107+0000 7f711b89f640 1 -- 172.21.15.100:0/3642760252 shutdown_connections 2024-07-26T13:30:48.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.107+0000 7f711b89f640 1 --2- 172.21.15.100:0/3642760252 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f70fc0b3c10 0x7f70fc0b6120 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:48.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.107+0000 7f711b89f640 1 --2- 172.21.15.100:0/3642760252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7114108720 0x7f711419b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:48.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.107+0000 7f711b89f640 1 --2- 172.21.15.100:0/3642760252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7114107dd0 0x7f711419b2c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:48.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.107+0000 7f711b89f640 1 --2- 172.21.15.100:0/3642760252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7114102490 0x7f711419ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:48.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.108+0000 7f711b89f640 1 -- 172.21.15.100:0/3642760252 >> 172.21.15.100:0/3642760252 conn(0x7f7114079160 msgr2=0x7f7114104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:30:48.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.108+0000 7f711b89f640 1 -- 172.21.15.100:0/3642760252 shutdown_connections 2024-07-26T13:30:48.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.108+0000 7f711b89f640 1 -- 172.21.15.100:0/3642760252 wait complete. 2024-07-26T13:30:48.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.108+0000 7f5a9c6ee640 1 -- 172.21.15.100:0/4089514102 shutdown_connections 2024-07-26T13:30:48.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.108+0000 7f5a9c6ee640 1 --2- 172.21.15.100:0/4089514102 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a94108720 0x7f5a9410ef50 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:48.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.108+0000 7f5a9c6ee640 1 --2- 172.21.15.100:0/4089514102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a94107dd0 0x7f5a941081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:48.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.108+0000 7f5a9c6ee640 1 --2- 172.21.15.100:0/4089514102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a94102490 0x7f5a941028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:48.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.108+0000 7f5a9c6ee640 1 -- 172.21.15.100:0/4089514102 >> 172.21.15.100:0/4089514102 conn(0x7f5a94079160 msgr2=0x7f5a94079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:30:48.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.108+0000 7f5a9c6ee640 1 -- 172.21.15.100:0/4089514102 shutdown_connections 2024-07-26T13:30:48.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.108+0000 7f5a9c6ee640 1 -- 172.21.15.100:0/4089514102 wait complete. 2024-07-26T13:30:48.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.108+0000 7f5a9c6ee640 1 Processor -- start 2024-07-26T13:30:48.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.109+0000 7f5a9c6ee640 1 -- start start 2024-07-26T13:30:48.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.109+0000 7f5a9c6ee640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a94102490 0x7f5a9419acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:48.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.109+0000 7f5a9c6ee640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a94107dd0 0x7f5a9419b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:48.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.109+0000 7f5a9c6ee640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a94108720 0x7f5a9419b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:48.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.109+0000 7f5a9c6ee640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5a941123a0 con 0x7f5a94102490 2024-07-26T13:30:48.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.109+0000 7f5a9c6ee640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5a94112220 con 0x7f5a94107dd0 2024-07-26T13:30:48.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.109+0000 7f5a9c6ee640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5a94112520 con 0x7f5a94108720 2024-07-26T13:30:48.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.109+0000 7f5a9a48a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a94102490 0x7f5a9419acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:48.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.109+0000 7f5a9a48a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a94102490 0x7f5a9419acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47112/0 (socket says 172.21.15.100:47112) 2024-07-26T13:30:48.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.109+0000 7f5a9a48a640 1 -- 172.21.15.100:0/3378328555 learned_addr learned my addr 172.21.15.100:0/3378328555 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:30:48.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.109+0000 7f5a99c89640 1 --2- 172.21.15.100:0/3378328555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a94107dd0 0x7f5a9419b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:48.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.109+0000 7f5a9ac8b640 1 --2- 172.21.15.100:0/3378328555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a94108720 0x7f5a9419b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:48.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.109+0000 7f5a9a48a640 1 -- 172.21.15.100:0/3378328555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a94108720 msgr2=0x7f5a9419b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:48.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.109+0000 7f5a9a48a640 1 --2- 172.21.15.100:0/3378328555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a94108720 0x7f5a9419b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:48.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.109+0000 7f5a9a48a640 1 -- 172.21.15.100:0/3378328555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a94107dd0 msgr2=0x7f5a9419b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:48.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.109+0000 7f5a9a48a640 1 --2- 172.21.15.100:0/3378328555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a94107dd0 0x7f5a9419b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:48.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.109+0000 7f5a9a48a640 1 -- 172.21.15.100:0/3378328555 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a941a5fb0 con 0x7f5a94102490 2024-07-26T13:30:48.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.109+0000 7f5a9ac8b640 1 --2- 172.21.15.100:0/3378328555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a94108720 0x7f5a9419b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:30:48.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.110+0000 7f5a9a48a640 1 --2- 172.21.15.100:0/3378328555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a94102490 0x7f5a9419acf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5a9000e9f0 tx=0x7f5a9000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:30:48.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.110+0000 7f5a8b7fe640 1 -- 172.21.15.100:0/3378328555 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9000cd80 con 0x7f5a94102490 2024-07-26T13:30:48.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.110+0000 7f5a8b7fe640 1 -- 172.21.15.100:0/3378328555 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a900108e0 con 0x7f5a94102490 2024-07-26T13:30:48.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.110+0000 7f5a9c6ee640 1 -- 172.21.15.100:0/3378328555 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a941a62a0 con 0x7f5a94102490 2024-07-26T13:30:48.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.110+0000 7f5a8b7fe640 1 -- 172.21.15.100:0/3378328555 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a90018980 con 0x7f5a94102490 2024-07-26T13:30:48.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.110+0000 7f5a9c6ee640 1 -- 172.21.15.100:0/3378328555 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a941a67e0 con 0x7f5a94102490 2024-07-26T13:30:48.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.111+0000 7f5a9c6ee640 1 -- 172.21.15.100:0/3378328555 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a941028a0 con 0x7f5a94102490 2024-07-26T13:30:48.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.113+0000 7f5a8b7fe640 1 -- 172.21.15.100:0/3378328555 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7f5a90010430 con 0x7f5a94102490 2024-07-26T13:30:48.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.113+0000 7f5a8b7fe640 1 --2- 172.21.15.100:0/3378328555 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5a800b3b90 0x7f5a800b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:48.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.114+0000 7f5a8b7fe640 1 -- 172.21.15.100:0/3378328555 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f5a900e2ac0 con 0x7f5a94102490 2024-07-26T13:30:48.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.114+0000 7f5a99c89640 1 --2- 172.21.15.100:0/3378328555 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5a800b3b90 0x7f5a800b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:48.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.114+0000 7f5a99c89640 1 --2- 172.21.15.100:0/3378328555 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5a800b3b90 0x7f5a800b60a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5a94079760 tx=0x7f5a84005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:30:48.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.117+0000 7f5a8b7fe640 1 -- 172.21.15.100:0/3378328555 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f5a900abd10 con 0x7f5a94102490 2024-07-26T13:30:48.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.299+0000 7f5a9c6ee640 1 -- 172.21.15.100:0/3378328555 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f5a9402e0c0 con 0x7f5a94102490 2024-07-26T13:30:48.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.306+0000 7f5a8b7fe640 1 -- 172.21.15.100:0/3378328555 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v24)=0 v24) ==== 122+0+0 (secure 0 0 0) 0x7f5a900ab610 con 0x7f5a94102490 2024-07-26T13:30:48.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.308+0000 7f5a9c6ee640 1 -- 172.21.15.100:0/3378328555 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5a800b3b90 msgr2=0x7f5a800b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:48.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.308+0000 7f5a9c6ee640 1 --2- 172.21.15.100:0/3378328555 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5a800b3b90 0x7f5a800b60a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5a94079760 tx=0x7f5a84005f90 comp rx=0 tx=0).stop 2024-07-26T13:30:48.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.308+0000 7f5a9c6ee640 1 -- 172.21.15.100:0/3378328555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a94102490 msgr2=0x7f5a9419acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:48.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.308+0000 7f5a9c6ee640 1 --2- 172.21.15.100:0/3378328555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a94102490 0x7f5a9419acf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5a9000e9f0 tx=0x7f5a9000eeb0 comp rx=0 tx=0).stop 2024-07-26T13:30:48.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.308+0000 7f5a9c6ee640 1 -- 172.21.15.100:0/3378328555 shutdown_connections 2024-07-26T13:30:48.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.308+0000 7f5a9c6ee640 1 --2- 172.21.15.100:0/3378328555 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5a800b3b90 0x7f5a800b60a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:48.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.308+0000 7f5a9c6ee640 1 --2- 172.21.15.100:0/3378328555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a94108720 0x7f5a9419b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:48.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.308+0000 7f5a9c6ee640 1 --2- 172.21.15.100:0/3378328555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a94107dd0 0x7f5a9419b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:48.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.308+0000 7f5a9c6ee640 1 --2- 172.21.15.100:0/3378328555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a94102490 0x7f5a9419acf0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:48.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.308+0000 7f5a9c6ee640 1 -- 172.21.15.100:0/3378328555 >> 172.21.15.100:0/3378328555 conn(0x7f5a94079160 msgr2=0x7f5a94104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:30:48.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.308+0000 7f5a9c6ee640 1 -- 172.21.15.100:0/3378328555 shutdown_connections 2024-07-26T13:30:48.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:48.308+0000 7f5a9c6ee640 1 -- 172.21.15.100:0/3378328555 wait complete. 2024-07-26T13:30:48.935 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'bf83e2e6-4b52-11ef-bcb7-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi100') 2024-07-26T13:30:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:49 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/3642760252' entity='client.admin' 2024-07-26T13:30:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:49 smithi100 ceph-mon[32599]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:49 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/3378328555' entity='client.admin' 2024-07-26T13:30:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:49 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/3642760252' entity='client.admin' 2024-07-26T13:30:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:49 smithi138 ceph-mon[36954]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:49 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/3378328555' entity='client.admin' 2024-07-26T13:30:49.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:49 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/3642760252' entity='client.admin' 2024-07-26T13:30:49.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:49 smithi167 ceph-mon[37140]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:49.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:49 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/3378328555' entity='client.admin' 2024-07-26T13:30:50.084 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'bf83e2e6-4b52-11ef-bcb7-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi100') 2024-07-26T13:30:51.515 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:51 smithi100 ceph-mon[32599]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:51.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:51 smithi138 ceph-mon[36954]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:51.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:51 smithi167 ceph-mon[37140]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:53.488 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:53 smithi100 ceph-mon[32599]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:53.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:53 smithi138 ceph-mon[36954]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:53.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:53 smithi167 ceph-mon[37140]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:54.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.888+0000 7f456146a640 1 -- 172.21.15.100:0/2975117426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f455c107dd0 msgr2=0x7f455c1081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:54.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.888+0000 7f456146a640 1 --2- 172.21.15.100:0/2975117426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f455c107dd0 0x7f455c1081e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4544009900 tx=0x7f454402a410 comp rx=0 tx=0).stop 2024-07-26T13:30:54.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.888+0000 7f456146a640 1 -- 172.21.15.100:0/2975117426 shutdown_connections 2024-07-26T13:30:54.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.888+0000 7f456146a640 1 --2- 172.21.15.100:0/2975117426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f455c108720 0x7f455c10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:54.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.888+0000 7f456146a640 1 --2- 172.21.15.100:0/2975117426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f455c107dd0 0x7f455c1081e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:54.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.888+0000 7f456146a640 1 --2- 172.21.15.100:0/2975117426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f455c102490 0x7f455c1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:54.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.888+0000 7f456146a640 1 -- 172.21.15.100:0/2975117426 >> 172.21.15.100:0/2975117426 conn(0x7f455c079160 msgr2=0x7f455c079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:30:54.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.888+0000 7f456146a640 1 -- 172.21.15.100:0/2975117426 shutdown_connections 2024-07-26T13:30:54.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.888+0000 7f456146a640 1 -- 172.21.15.100:0/2975117426 wait complete. 2024-07-26T13:30:54.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.889+0000 7f456146a640 1 Processor -- start 2024-07-26T13:30:54.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.889+0000 7f456146a640 1 -- start start 2024-07-26T13:30:54.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.889+0000 7f456146a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f455c102490 0x7f455c19c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:54.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.889+0000 7f456146a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f455c107dd0 0x7f455c19c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:54.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.890+0000 7f456146a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f455c108720 0x7f455c1a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:54.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.890+0000 7f456146a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f455c1123d0 con 0x7f455c102490 2024-07-26T13:30:54.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.890+0000 7f456146a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f455c112250 con 0x7f455c108720 2024-07-26T13:30:54.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.890+0000 7f456146a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f455c112550 con 0x7f455c107dd0 2024-07-26T13:30:54.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.890+0000 7f455affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f455c102490 0x7f455c19c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:54.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.890+0000 7f455affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f455c102490 0x7f455c19c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47120/0 (socket says 172.21.15.100:47120) 2024-07-26T13:30:54.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.890+0000 7f455affd640 1 -- 172.21.15.100:0/4094624454 learned_addr learned my addr 172.21.15.100:0/4094624454 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:30:54.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.890+0000 7f455b7fe640 1 --2- 172.21.15.100:0/4094624454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f455c108720 0x7f455c1a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:54.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.890+0000 7f455a7fc640 1 --2- 172.21.15.100:0/4094624454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f455c107dd0 0x7f455c19c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:54.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.890+0000 7f455affd640 1 -- 172.21.15.100:0/4094624454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f455c107dd0 msgr2=0x7f455c19c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:54.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.890+0000 7f455affd640 1 --2- 172.21.15.100:0/4094624454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f455c107dd0 0x7f455c19c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:54.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.890+0000 7f455affd640 1 -- 172.21.15.100:0/4094624454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f455c108720 msgr2=0x7f455c1a19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:54.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.890+0000 7f455affd640 1 --2- 172.21.15.100:0/4094624454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f455c108720 0x7f455c1a19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:54.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.890+0000 7f455affd640 1 -- 172.21.15.100:0/4094624454 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f455c1a2120 con 0x7f455c102490 2024-07-26T13:30:54.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.890+0000 7f455b7fe640 1 --2- 172.21.15.100:0/4094624454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f455c108720 0x7f455c1a19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:30:54.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.890+0000 7f455affd640 1 --2- 172.21.15.100:0/4094624454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f455c102490 0x7f455c19c1b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f454c00ba50 tx=0x7f454c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:30:54.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.890+0000 7f4553fff640 1 -- 172.21.15.100:0/4094624454 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c019070 con 0x7f455c102490 2024-07-26T13:30:54.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.890+0000 7f456146a640 1 -- 172.21.15.100:0/4094624454 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f455c1a2410 con 0x7f455c102490 2024-07-26T13:30:54.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.890+0000 7f4553fff640 1 -- 172.21.15.100:0/4094624454 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f454c00cbe0 con 0x7f455c102490 2024-07-26T13:30:54.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.890+0000 7f4553fff640 1 -- 172.21.15.100:0/4094624454 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c0075d0 con 0x7f455c102490 2024-07-26T13:30:54.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.891+0000 7f456146a640 1 -- 172.21.15.100:0/4094624454 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f455c1a8b70 con 0x7f455c102490 2024-07-26T13:30:54.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.892+0000 7f456146a640 1 -- 172.21.15.100:0/4094624454 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f455c1028a0 con 0x7f455c102490 2024-07-26T13:30:54.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.893+0000 7f4553fff640 1 -- 172.21.15.100:0/4094624454 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7f454c01c070 con 0x7f455c102490 2024-07-26T13:30:54.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.894+0000 7f4553fff640 1 --2- 172.21.15.100:0/4094624454 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f45480b3b90 0x7f45480b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:54.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.894+0000 7f455a7fc640 1 --2- 172.21.15.100:0/4094624454 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f45480b3b90 0x7f45480b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:54.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.894+0000 7f4553fff640 1 -- 172.21.15.100:0/4094624454 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f454c0e1bb0 con 0x7f455c102490 2024-07-26T13:30:54.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.894+0000 7f455a7fc640 1 --2- 172.21.15.100:0/4094624454 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f45480b3b90 0x7f45480b60a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4544000c00 tx=0x7f4544031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:30:54.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:54.897+0000 7f4553fff640 1 -- 172.21.15.100:0/4094624454 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f454c0aae00 con 0x7f455c102490 2024-07-26T13:30:55.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:55.090+0000 7f456146a640 1 -- 172.21.15.100:0/4094624454 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f455c19d2e0 con 0x7f455c102490 2024-07-26T13:30:55.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:55.095+0000 7f4553fff640 1 -- 172.21.15.100:0/4094624454 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v25)=0 v25) ==== 129+0+0 (secure 0 0 0) 0x7f455c19d2e0 con 0x7f455c102490 2024-07-26T13:30:55.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:55.097+0000 7f456146a640 1 -- 172.21.15.100:0/4094624454 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f45480b3b90 msgr2=0x7f45480b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:55.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:55.097+0000 7f456146a640 1 --2- 172.21.15.100:0/4094624454 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f45480b3b90 0x7f45480b60a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4544000c00 tx=0x7f4544031400 comp rx=0 tx=0).stop 2024-07-26T13:30:55.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:55.097+0000 7f456146a640 1 -- 172.21.15.100:0/4094624454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f455c102490 msgr2=0x7f455c19c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:55.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:55.097+0000 7f456146a640 1 --2- 172.21.15.100:0/4094624454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f455c102490 0x7f455c19c1b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f454c00ba50 tx=0x7f454c00bf10 comp rx=0 tx=0).stop 2024-07-26T13:30:55.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:55.097+0000 7f456146a640 1 -- 172.21.15.100:0/4094624454 shutdown_connections 2024-07-26T13:30:55.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:55.097+0000 7f456146a640 1 --2- 172.21.15.100:0/4094624454 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f45480b3b90 0x7f45480b60a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:55.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:55.097+0000 7f456146a640 1 --2- 172.21.15.100:0/4094624454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f455c108720 0x7f455c1a19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:55.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:55.097+0000 7f456146a640 1 --2- 172.21.15.100:0/4094624454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f455c107dd0 0x7f455c19c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:55.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:55.097+0000 7f456146a640 1 --2- 172.21.15.100:0/4094624454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f455c102490 0x7f455c19c1b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:55.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:55.097+0000 7f456146a640 1 -- 172.21.15.100:0/4094624454 >> 172.21.15.100:0/4094624454 conn(0x7f455c079160 msgr2=0x7f455c104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:30:55.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:55.097+0000 7f456146a640 1 -- 172.21.15.100:0/4094624454 shutdown_connections 2024-07-26T13:30:55.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:55.097+0000 7f456146a640 1 -- 172.21.15.100:0/4094624454 wait complete. 2024-07-26T13:30:55.311 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:30:55.424 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:55 smithi100 ceph-mon[32599]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:55.424 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:55 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/4094624454' entity='client.admin' 2024-07-26T13:30:55.424 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:30:55.424 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:55.424 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:30:55.424 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:55.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:55 smithi138 ceph-mon[36954]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:55.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:55 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/4094624454' entity='client.admin' 2024-07-26T13:30:55.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:30:55.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:55.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:30:55.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:55.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:55 smithi167 ceph-mon[37140]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:55.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:55 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/4094624454' entity='client.admin' 2024-07-26T13:30:55.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:30:55.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:30:55.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:30:55.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:30:56.049 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'bf83e2e6-4b52-11ef-bcb7-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi100') 2024-07-26T13:30:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:57 smithi100 ceph-mon[32599]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:57.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:57 smithi138 ceph-mon[36954]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:57.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:57 smithi167 ceph-mon[37140]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:58.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.143+0000 7f28e6599640 1 -- 172.21.15.100:0/187281116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28e01019f0 msgr2=0x7f28e0101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:58.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.143+0000 7f28e6599640 1 --2- 172.21.15.100:0/187281116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28e01019f0 0x7f28e0101e00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f28d0009900 tx=0x7f28d002a410 comp rx=0 tx=0).stop 2024-07-26T13:30:58.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.143+0000 7f28e6599640 1 -- 172.21.15.100:0/187281116 shutdown_connections 2024-07-26T13:30:58.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.143+0000 7f28e6599640 1 --2- 172.21.15.100:0/187281116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28e0109880 0x7f28e01100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:58.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.143+0000 7f28e6599640 1 --2- 172.21.15.100:0/187281116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28e0108f30 0x7f28e0109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:58.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.143+0000 7f28e6599640 1 --2- 172.21.15.100:0/187281116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28e01019f0 0x7f28e0101e00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:58.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.143+0000 7f28e6599640 1 -- 172.21.15.100:0/187281116 >> 172.21.15.100:0/187281116 conn(0x7f28e00799c0 msgr2=0x7f28e0079dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:30:58.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.143+0000 7f28e6599640 1 -- 172.21.15.100:0/187281116 shutdown_connections 2024-07-26T13:30:58.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.143+0000 7f28e6599640 1 -- 172.21.15.100:0/187281116 wait complete. 2024-07-26T13:30:58.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.144+0000 7f28e6599640 1 Processor -- start 2024-07-26T13:30:58.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.144+0000 7f28e6599640 1 -- start start 2024-07-26T13:30:58.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.144+0000 7f28e6599640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28e01019f0 0x7f28e0199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:58.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.145+0000 7f28e6599640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28e0108f30 0x7f28e019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:58.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.145+0000 7f28e6599640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28e0109880 0x7f28e019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:58.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.145+0000 7f28e6599640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f28e01135a0 con 0x7f28e0109880 2024-07-26T13:30:58.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.145+0000 7f28e6599640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f28e0113420 con 0x7f28e01019f0 2024-07-26T13:30:58.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.145+0000 7f28e6599640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f28e0113720 con 0x7f28e0108f30 2024-07-26T13:30:58.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.145+0000 7f28e4b36640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28e0109880 0x7f28e019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:58.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.145+0000 7f28e4b36640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28e0109880 0x7f28e019f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39680/0 (socket says 172.21.15.100:39680) 2024-07-26T13:30:58.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.145+0000 7f28e4b36640 1 -- 172.21.15.100:0/3189044893 learned_addr learned my addr 172.21.15.100:0/3189044893 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:30:58.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.145+0000 7f28dffff640 1 --2- 172.21.15.100:0/3189044893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28e01019f0 0x7f28e0199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:58.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.145+0000 7f28df7fe640 1 --2- 172.21.15.100:0/3189044893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28e0108f30 0x7f28e019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:58.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.145+0000 7f28e4b36640 1 -- 172.21.15.100:0/3189044893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28e0108f30 msgr2=0x7f28e019a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:58.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.145+0000 7f28e4b36640 1 --2- 172.21.15.100:0/3189044893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28e0108f30 0x7f28e019a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:58.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.145+0000 7f28e4b36640 1 -- 172.21.15.100:0/3189044893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28e01019f0 msgr2=0x7f28e0199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:58.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.145+0000 7f28e4b36640 1 --2- 172.21.15.100:0/3189044893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28e01019f0 0x7f28e0199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:58.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.145+0000 7f28e4b36640 1 -- 172.21.15.100:0/3189044893 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28e019fe00 con 0x7f28e0109880 2024-07-26T13:30:58.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.145+0000 7f28df7fe640 1 --2- 172.21.15.100:0/3189044893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28e0108f30 0x7f28e019a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:30:58.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.145+0000 7f28e4b36640 1 --2- 172.21.15.100:0/3189044893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28e0109880 0x7f28e019f790 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f28d800b970 tx=0x7f28d800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:30:58.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.145+0000 7f28dd7fa640 1 -- 172.21.15.100:0/3189044893 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d8019070 con 0x7f28e0109880 2024-07-26T13:30:58.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.145+0000 7f28dd7fa640 1 -- 172.21.15.100:0/3189044893 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28d8014440 con 0x7f28e0109880 2024-07-26T13:30:58.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.145+0000 7f28e6599640 1 -- 172.21.15.100:0/3189044893 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28e01a00f0 con 0x7f28e0109880 2024-07-26T13:30:58.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.145+0000 7f28dd7fa640 1 -- 172.21.15.100:0/3189044893 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d80075d0 con 0x7f28e0109880 2024-07-26T13:30:58.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.145+0000 7f28e6599640 1 -- 172.21.15.100:0/3189044893 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28e01a6910 con 0x7f28e0109880 2024-07-26T13:30:58.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.146+0000 7f28e6599640 1 -- 172.21.15.100:0/3189044893 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28e0101e00 con 0x7f28e0109880 2024-07-26T13:30:58.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.148+0000 7f28dd7fa640 1 -- 172.21.15.100:0/3189044893 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7f28d801c070 con 0x7f28e0109880 2024-07-26T13:30:58.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.149+0000 7f28dd7fa640 1 --2- 172.21.15.100:0/3189044893 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f28c80b3bc0 0x7f28c80b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:30:58.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.149+0000 7f28dffff640 1 --2- 172.21.15.100:0/3189044893 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f28c80b3bc0 0x7f28c80b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:30:58.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.149+0000 7f28dd7fa640 1 -- 172.21.15.100:0/3189044893 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f28d80e1b00 con 0x7f28e0109880 2024-07-26T13:30:58.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.149+0000 7f28dffff640 1 --2- 172.21.15.100:0/3189044893 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f28c80b3bc0 0x7f28c80b60d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f28d0031440 tx=0x7f28d0005d50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:30:58.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.152+0000 7f28dd7fa640 1 -- 172.21.15.100:0/3189044893 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f28d80aad50 con 0x7f28e0109880 2024-07-26T13:30:58.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.332+0000 7f28e6599640 1 -- 172.21.15.100:0/3189044893 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7f28e019b080 con 0x7f28e0109880 2024-07-26T13:30:58.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.336+0000 7f28dd7fa640 1 -- 172.21.15.100:0/3189044893 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v26)=0 v26) ==== 127+0+0 (secure 0 0 0) 0x7f28e019b080 con 0x7f28e0109880 2024-07-26T13:30:58.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.339+0000 7f28e6599640 1 -- 172.21.15.100:0/3189044893 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f28c80b3bc0 msgr2=0x7f28c80b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:58.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.339+0000 7f28e6599640 1 --2- 172.21.15.100:0/3189044893 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f28c80b3bc0 0x7f28c80b60d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f28d0031440 tx=0x7f28d0005d50 comp rx=0 tx=0).stop 2024-07-26T13:30:58.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.339+0000 7f28e6599640 1 -- 172.21.15.100:0/3189044893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28e0109880 msgr2=0x7f28e019f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:30:58.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.339+0000 7f28e6599640 1 --2- 172.21.15.100:0/3189044893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28e0109880 0x7f28e019f790 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f28d800b970 tx=0x7f28d800be30 comp rx=0 tx=0).stop 2024-07-26T13:30:58.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.339+0000 7f28e6599640 1 -- 172.21.15.100:0/3189044893 shutdown_connections 2024-07-26T13:30:58.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.339+0000 7f28e6599640 1 --2- 172.21.15.100:0/3189044893 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f28c80b3bc0 0x7f28c80b60d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:58.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.339+0000 7f28e6599640 1 --2- 172.21.15.100:0/3189044893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28e0109880 0x7f28e019f790 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:58.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.339+0000 7f28e6599640 1 --2- 172.21.15.100:0/3189044893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28e0108f30 0x7f28e019a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:58.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.339+0000 7f28e6599640 1 --2- 172.21.15.100:0/3189044893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28e01019f0 0x7f28e0199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:30:58.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.339+0000 7f28e6599640 1 -- 172.21.15.100:0/3189044893 >> 172.21.15.100:0/3189044893 conn(0x7f28e00799c0 msgr2=0x7f28e0106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:30:58.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.339+0000 7f28e6599640 1 -- 172.21.15.100:0/3189044893 shutdown_connections 2024-07-26T13:30:58.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:30:58.339+0000 7f28e6599640 1 -- 172.21.15.100:0/3189044893 wait complete. 2024-07-26T13:30:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:59 smithi100 ceph-mon[32599]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:30:59 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/3189044893' entity='client.admin' 2024-07-26T13:30:59.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:59 smithi138 ceph-mon[36954]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:59.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:30:59 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/3189044893' entity='client.admin' 2024-07-26T13:30:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:59 smithi167 ceph-mon[37140]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:30:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:30:59 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/3189044893' entity='client.admin' 2024-07-26T13:31:00.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.179+0000 7ff30073e640 1 -- 172.21.15.100:0/121409121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2f8107e50 msgr2=0x7ff2f8108260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:00.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.179+0000 7ff30073e640 1 --2- 172.21.15.100:0/121409121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2f8107e50 0x7ff2f8108260 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8009900 tx=0x7ff2e802a410 comp rx=0 tx=0).stop 2024-07-26T13:31:00.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.179+0000 7ff30073e640 1 -- 172.21.15.100:0/121409121 shutdown_connections 2024-07-26T13:31:00.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.179+0000 7ff30073e640 1 --2- 172.21.15.100:0/121409121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2f81087a0 0x7ff2f810efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:00.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.179+0000 7ff30073e640 1 --2- 172.21.15.100:0/121409121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2f8107e50 0x7ff2f8108260 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:00.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.179+0000 7ff30073e640 1 --2- 172.21.15.100:0/121409121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2f8102510 0x7ff2f8102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:00.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.179+0000 7ff30073e640 1 -- 172.21.15.100:0/121409121 >> 172.21.15.100:0/121409121 conn(0x7ff2f80791a0 msgr2=0x7ff2f80795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:31:00.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.179+0000 7ff30073e640 1 -- 172.21.15.100:0/121409121 shutdown_connections 2024-07-26T13:31:00.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.179+0000 7ff30073e640 1 -- 172.21.15.100:0/121409121 wait complete. 2024-07-26T13:31:00.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.180+0000 7ff30073e640 1 Processor -- start 2024-07-26T13:31:00.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.180+0000 7ff30073e640 1 -- start start 2024-07-26T13:31:00.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.180+0000 7ff30073e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2f8102510 0x7ff2f810fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:00.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.180+0000 7ff30073e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2f8107e50 0x7ff2f810fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:00.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.180+0000 7ff30073e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2f81087a0 0x7ff2f8110530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:00.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.180+0000 7ff30073e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f8113580 con 0x7ff2f81087a0 2024-07-26T13:31:00.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.180+0000 7ff30073e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f8113400 con 0x7ff2f8107e50 2024-07-26T13:31:00.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.180+0000 7ff30073e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f8113700 con 0x7ff2f8102510 2024-07-26T13:31:00.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.180+0000 7ff2fecdb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2f81087a0 0x7ff2f8110530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:00.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.180+0000 7ff2fecdb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2f81087a0 0x7ff2f8110530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39694/0 (socket says 172.21.15.100:39694) 2024-07-26T13:31:00.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.180+0000 7ff2fecdb640 1 -- 172.21.15.100:0/504691842 learned_addr learned my addr 172.21.15.100:0/504691842 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:31:00.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.181+0000 7ff2fe4da640 1 --2- 172.21.15.100:0/504691842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2f8102510 0x7ff2f810fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:00.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.181+0000 7ff2fdcd9640 1 --2- 172.21.15.100:0/504691842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2f8107e50 0x7ff2f810fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:00.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.181+0000 7ff2fecdb640 1 -- 172.21.15.100:0/504691842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2f8102510 msgr2=0x7ff2f810fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:00.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.181+0000 7ff2fecdb640 1 --2- 172.21.15.100:0/504691842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2f8102510 0x7ff2f810fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:00.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.181+0000 7ff2fecdb640 1 -- 172.21.15.100:0/504691842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2f8107e50 msgr2=0x7ff2f810fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:00.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.181+0000 7ff2fecdb640 1 --2- 172.21.15.100:0/504691842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2f8107e50 0x7ff2f810fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:00.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.181+0000 7ff2fecdb640 1 -- 172.21.15.100:0/504691842 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f81a8310 con 0x7ff2f81087a0 2024-07-26T13:31:00.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.181+0000 7ff2fdcd9640 1 --2- 172.21.15.100:0/504691842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2f8107e50 0x7ff2f810fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:31:00.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.181+0000 7ff2fecdb640 1 --2- 172.21.15.100:0/504691842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2f81087a0 0x7ff2f8110530 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000cad0 tx=0x7ff2f000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:31:00.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.181+0000 7ff2ef7fe640 1 -- 172.21.15.100:0/504691842 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f000ad60 con 0x7ff2f81087a0 2024-07-26T13:31:00.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.181+0000 7ff2ef7fe640 1 -- 172.21.15.100:0/504691842 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2f0005020 con 0x7ff2f81087a0 2024-07-26T13:31:00.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.181+0000 7ff30073e640 1 -- 172.21.15.100:0/504691842 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2f81a8600 con 0x7ff2f81087a0 2024-07-26T13:31:00.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.181+0000 7ff2ef7fe640 1 -- 172.21.15.100:0/504691842 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f000f6b0 con 0x7ff2f81087a0 2024-07-26T13:31:00.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.181+0000 7ff30073e640 1 -- 172.21.15.100:0/504691842 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2f81a8af0 con 0x7ff2f81087a0 2024-07-26T13:31:00.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.182+0000 7ff30073e640 1 -- 172.21.15.100:0/504691842 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2f8103790 con 0x7ff2f81087a0 2024-07-26T13:31:00.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.184+0000 7ff2ef7fe640 1 -- 172.21.15.100:0/504691842 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7ff2f00051d0 con 0x7ff2f81087a0 2024-07-26T13:31:00.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.184+0000 7ff2ef7fe640 1 --2- 172.21.15.100:0/504691842 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff2e40b3b90 0x7ff2e40b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:00.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.184+0000 7ff2fe4da640 1 --2- 172.21.15.100:0/504691842 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff2e40b3b90 0x7ff2e40b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:00.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.184+0000 7ff2ef7fe640 1 -- 172.21.15.100:0/504691842 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7ff2f00e2040 con 0x7ff2f81087a0 2024-07-26T13:31:00.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.185+0000 7ff2fe4da640 1 --2- 172.21.15.100:0/504691842 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff2e40b3b90 0x7ff2e40b60a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8111500 tx=0x7ff2f4009110 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:31:00.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.189+0000 7ff2ef7fe640 1 -- 172.21.15.100:0/504691842 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7ff2f0014030 con 0x7ff2f81087a0 2024-07-26T13:31:00.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.370+0000 7ff30073e640 1 -- 172.21.15.100:0/504691842 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7ff2f802e0c0 con 0x7ff2f81087a0 2024-07-26T13:31:00.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.374+0000 7ff2ef7fe640 1 -- 172.21.15.100:0/504691842 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v27)=0 v27) ==== 118+0+0 (secure 0 0 0) 0x7ff2f00ab290 con 0x7ff2f81087a0 2024-07-26T13:31:00.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.376+0000 7ff30073e640 1 -- 172.21.15.100:0/504691842 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff2e40b3b90 msgr2=0x7ff2e40b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:00.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.376+0000 7ff30073e640 1 --2- 172.21.15.100:0/504691842 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff2e40b3b90 0x7ff2e40b60a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8111500 tx=0x7ff2f4009110 comp rx=0 tx=0).stop 2024-07-26T13:31:00.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.376+0000 7ff30073e640 1 -- 172.21.15.100:0/504691842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2f81087a0 msgr2=0x7ff2f8110530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:00.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.376+0000 7ff30073e640 1 --2- 172.21.15.100:0/504691842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2f81087a0 0x7ff2f8110530 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000cad0 tx=0x7ff2f000ced0 comp rx=0 tx=0).stop 2024-07-26T13:31:00.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.377+0000 7ff30073e640 1 -- 172.21.15.100:0/504691842 shutdown_connections 2024-07-26T13:31:00.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.377+0000 7ff30073e640 1 --2- 172.21.15.100:0/504691842 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff2e40b3b90 0x7ff2e40b60a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:00.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.377+0000 7ff30073e640 1 --2- 172.21.15.100:0/504691842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2f81087a0 0x7ff2f8110530 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:00.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.377+0000 7ff30073e640 1 --2- 172.21.15.100:0/504691842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2f8107e50 0x7ff2f810fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:00.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.377+0000 7ff30073e640 1 --2- 172.21.15.100:0/504691842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2f8102510 0x7ff2f810fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:00.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.377+0000 7ff30073e640 1 -- 172.21.15.100:0/504691842 >> 172.21.15.100:0/504691842 conn(0x7ff2f80791a0 msgr2=0x7ff2f8105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:31:00.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.377+0000 7ff30073e640 1 -- 172.21.15.100:0/504691842 shutdown_connections 2024-07-26T13:31:00.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:00.377+0000 7ff30073e640 1 -- 172.21.15.100:0/504691842 wait complete. 2024-07-26T13:31:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:01 smithi100 ceph-mon[32599]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:01 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/504691842' entity='client.admin' 2024-07-26T13:31:01.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:01 smithi138 ceph-mon[36954]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:01.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:01 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/504691842' entity='client.admin' 2024-07-26T13:31:01.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:01 smithi167 ceph-mon[37140]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:01.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:01 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/504691842' entity='client.admin' 2024-07-26T13:31:02.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.109+0000 7fcbf72bf640 1 -- 172.21.15.100:0/3581552202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbf0102490 msgr2=0x7fcbf01028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:02.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.109+0000 7fcbf72bf640 1 --2- 172.21.15.100:0/3581552202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbf0102490 0x7fcbf01028a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0009ca0 tx=0x7fcbe002a8c0 comp rx=0 tx=0).stop 2024-07-26T13:31:02.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.109+0000 7fcbf72bf640 1 -- 172.21.15.100:0/3581552202 shutdown_connections 2024-07-26T13:31:02.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.109+0000 7fcbf72bf640 1 --2- 172.21.15.100:0/3581552202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbf0108720 0x7fcbf010ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:02.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.109+0000 7fcbf72bf640 1 --2- 172.21.15.100:0/3581552202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbf0107dd0 0x7fcbf01081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:02.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.109+0000 7fcbf72bf640 1 --2- 172.21.15.100:0/3581552202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbf0102490 0x7fcbf01028a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:02.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.109+0000 7fcbf72bf640 1 -- 172.21.15.100:0/3581552202 >> 172.21.15.100:0/3581552202 conn(0x7fcbf0079190 msgr2=0x7fcbf00795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:31:02.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.109+0000 7fcbf72bf640 1 -- 172.21.15.100:0/3581552202 shutdown_connections 2024-07-26T13:31:02.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.109+0000 7fcbf72bf640 1 -- 172.21.15.100:0/3581552202 wait complete. 2024-07-26T13:31:02.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.110+0000 7fcbf72bf640 1 Processor -- start 2024-07-26T13:31:02.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.110+0000 7fcbf72bf640 1 -- start start 2024-07-26T13:31:02.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.110+0000 7fcbf72bf640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbf0102490 0x7fcbf019ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:02.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.110+0000 7fcbf72bf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbf0107dd0 0x7fcbf019b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:02.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.110+0000 7fcbf72bf640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbf0108720 0x7fcbf019b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:02.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.110+0000 7fcbf72bf640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf0112420 con 0x7fcbf0107dd0 2024-07-26T13:31:02.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.110+0000 7fcbf72bf640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf01122a0 con 0x7fcbf0102490 2024-07-26T13:31:02.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.110+0000 7fcbf72bf640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf01125a0 con 0x7fcbf0108720 2024-07-26T13:31:02.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.110+0000 7fcbf485a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbf0107dd0 0x7fcbf019b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:02.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.110+0000 7fcbf485a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbf0107dd0 0x7fcbf019b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39712/0 (socket says 172.21.15.100:39712) 2024-07-26T13:31:02.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.110+0000 7fcbf485a640 1 -- 172.21.15.100:0/1240474165 learned_addr learned my addr 172.21.15.100:0/1240474165 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:31:02.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.110+0000 7fcbf505b640 1 --2- 172.21.15.100:0/1240474165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbf0102490 0x7fcbf019ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:02.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.111+0000 7fcbf585c640 1 --2- 172.21.15.100:0/1240474165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbf0108720 0x7fcbf019b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:02.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.111+0000 7fcbf485a640 1 -- 172.21.15.100:0/1240474165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbf0108720 msgr2=0x7fcbf019b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:02.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.111+0000 7fcbf485a640 1 --2- 172.21.15.100:0/1240474165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbf0108720 0x7fcbf019b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:02.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.111+0000 7fcbf485a640 1 -- 172.21.15.100:0/1240474165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbf0102490 msgr2=0x7fcbf019ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:02.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.111+0000 7fcbf485a640 1 --2- 172.21.15.100:0/1240474165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbf0102490 0x7fcbf019ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:02.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.111+0000 7fcbf485a640 1 -- 172.21.15.100:0/1240474165 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbf01a5fd0 con 0x7fcbf0107dd0 2024-07-26T13:31:02.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.111+0000 7fcbf485a640 1 --2- 172.21.15.100:0/1240474165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbf0107dd0 0x7fcbf019b250 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcbe400d740 tx=0x7fcbe400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:31:02.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.111+0000 7fcbee7fc640 1 -- 172.21.15.100:0/1240474165 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe4017070 con 0x7fcbf0107dd0 2024-07-26T13:31:02.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.111+0000 7fcbee7fc640 1 -- 172.21.15.100:0/1240474165 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbe400de50 con 0x7fcbf0107dd0 2024-07-26T13:31:02.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.111+0000 7fcbee7fc640 1 -- 172.21.15.100:0/1240474165 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe400f620 con 0x7fcbf0107dd0 2024-07-26T13:31:02.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.111+0000 7fcbf72bf640 1 -- 172.21.15.100:0/1240474165 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbf01a62c0 con 0x7fcbf0107dd0 2024-07-26T13:31:02.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.111+0000 7fcbf72bf640 1 -- 172.21.15.100:0/1240474165 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbf01a67b0 con 0x7fcbf0107dd0 2024-07-26T13:31:02.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.112+0000 7fcbf72bf640 1 -- 172.21.15.100:0/1240474165 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbf0103710 con 0x7fcbf0107dd0 2024-07-26T13:31:02.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.114+0000 7fcbee7fc640 1 -- 172.21.15.100:0/1240474165 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7fcbe401a070 con 0x7fcbf0107dd0 2024-07-26T13:31:02.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.114+0000 7fcbee7fc640 1 --2- 172.21.15.100:0/1240474165 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fcbd80b3c10 0x7fcbd80b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:02.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.114+0000 7fcbf505b640 1 --2- 172.21.15.100:0/1240474165 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fcbd80b3c10 0x7fcbd80b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:02.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.114+0000 7fcbee7fc640 1 -- 172.21.15.100:0/1240474165 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fcbe40e1990 con 0x7fcbf0107dd0 2024-07-26T13:31:02.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.115+0000 7fcbf505b640 1 --2- 172.21.15.100:0/1240474165 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fcbd80b3c10 0x7fcbd80b6120 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcbf006c150 tx=0x7fcbe0005fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:31:02.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.117+0000 7fcbee7fc640 1 -- 172.21.15.100:0/1240474165 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fcbe40aac60 con 0x7fcbf0107dd0 2024-07-26T13:31:02.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.300+0000 7fcbf72bf640 1 -- 172.21.15.100:0/1240474165 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fcbf01028a0 con 0x7fcbf0107dd0 2024-07-26T13:31:02.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.304+0000 7fcbee7fc640 1 -- 172.21.15.100:0/1240474165 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v28)=0 v28) ==== 133+0+0 (secure 0 0 0) 0x7fcbe40aa560 con 0x7fcbf0107dd0 2024-07-26T13:31:02.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.307+0000 7fcbf72bf640 1 -- 172.21.15.100:0/1240474165 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fcbd80b3c10 msgr2=0x7fcbd80b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:02.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.307+0000 7fcbf72bf640 1 --2- 172.21.15.100:0/1240474165 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fcbd80b3c10 0x7fcbd80b6120 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcbf006c150 tx=0x7fcbe0005fd0 comp rx=0 tx=0).stop 2024-07-26T13:31:02.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.307+0000 7fcbf72bf640 1 -- 172.21.15.100:0/1240474165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbf0107dd0 msgr2=0x7fcbf019b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:02.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.307+0000 7fcbf72bf640 1 --2- 172.21.15.100:0/1240474165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbf0107dd0 0x7fcbf019b250 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcbe400d740 tx=0x7fcbe400dc00 comp rx=0 tx=0).stop 2024-07-26T13:31:02.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.307+0000 7fcbf72bf640 1 -- 172.21.15.100:0/1240474165 shutdown_connections 2024-07-26T13:31:02.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.307+0000 7fcbf72bf640 1 --2- 172.21.15.100:0/1240474165 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fcbd80b3c10 0x7fcbd80b6120 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:02.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.307+0000 7fcbf72bf640 1 --2- 172.21.15.100:0/1240474165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbf0108720 0x7fcbf019b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:02.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.307+0000 7fcbf72bf640 1 --2- 172.21.15.100:0/1240474165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcbf0107dd0 0x7fcbf019b250 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:02.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.307+0000 7fcbf72bf640 1 --2- 172.21.15.100:0/1240474165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbf0102490 0x7fcbf019ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:02.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.307+0000 7fcbf72bf640 1 -- 172.21.15.100:0/1240474165 >> 172.21.15.100:0/1240474165 conn(0x7fcbf0079190 msgr2=0x7fcbf0105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:31:02.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.307+0000 7fcbf72bf640 1 -- 172.21.15.100:0/1240474165 shutdown_connections 2024-07-26T13:31:02.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:02.308+0000 7fcbf72bf640 1 -- 172.21.15.100:0/1240474165 wait complete. 2024-07-26T13:31:03.329 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'bf83e2e6-4b52-11ef-bcb7-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi100') 2024-07-26T13:31:03.510 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:03 smithi100 ceph-mon[32599]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:03.510 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:03 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/1240474165' entity='client.admin' 2024-07-26T13:31:03.510 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:31:03.510 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:31:03.511 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:31:03.511 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:03.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.535+0000 7f2538f49640 1 -- 172.21.15.100:0/1427518614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2534108720 msgr2=0x7f253410ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:03.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.535+0000 7f2538f49640 1 --2- 172.21.15.100:0/1427518614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2534108720 0x7f253410ef50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f252c009ca0 tx=0x7f252c02a8c0 comp rx=0 tx=0).stop 2024-07-26T13:31:03.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.535+0000 7f2538f49640 1 -- 172.21.15.100:0/1427518614 shutdown_connections 2024-07-26T13:31:03.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.535+0000 7f2538f49640 1 --2- 172.21.15.100:0/1427518614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2534108720 0x7f253410ef50 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:03.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.535+0000 7f2538f49640 1 --2- 172.21.15.100:0/1427518614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2534107dd0 0x7f25341081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:03.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.535+0000 7f2538f49640 1 --2- 172.21.15.100:0/1427518614 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2534102490 0x7f25341028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:03.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.535+0000 7f2538f49640 1 -- 172.21.15.100:0/1427518614 >> 172.21.15.100:0/1427518614 conn(0x7f2534079160 msgr2=0x7f2534079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:31:03.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.535+0000 7f2538f49640 1 -- 172.21.15.100:0/1427518614 shutdown_connections 2024-07-26T13:31:03.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.536+0000 7f2538f49640 1 -- 172.21.15.100:0/1427518614 wait complete. 2024-07-26T13:31:03.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.536+0000 7f2538f49640 1 Processor -- start 2024-07-26T13:31:03.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.536+0000 7f2538f49640 1 -- start start 2024-07-26T13:31:03.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.537+0000 7f2538f49640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2534102490 0x7f2534199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:03.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.537+0000 7f2538f49640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2534107dd0 0x7f253419a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:03.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.537+0000 7f2538f49640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2534108720 0x7f253419f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:03.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.537+0000 7f2538f49640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f25341134b0 con 0x7f2534108720 2024-07-26T13:31:03.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.537+0000 7f2538f49640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2534113330 con 0x7f2534107dd0 2024-07-26T13:31:03.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.537+0000 7f2538f49640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2534113630 con 0x7f2534102490 2024-07-26T13:31:03.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.537+0000 7f2532d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2534108720 0x7f253419f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:03.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.537+0000 7f2532d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2534108720 0x7f253419f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39724/0 (socket says 172.21.15.100:39724) 2024-07-26T13:31:03.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.537+0000 7f2532d9d640 1 -- 172.21.15.100:0/4207390182 learned_addr learned my addr 172.21.15.100:0/4207390182 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:31:03.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.537+0000 7f253259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2534102490 0x7f2534199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:03.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.537+0000 7f2531d9b640 1 --2- 172.21.15.100:0/4207390182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2534107dd0 0x7f253419a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:03.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.537+0000 7f2532d9d640 1 -- 172.21.15.100:0/4207390182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2534102490 msgr2=0x7f2534199e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:03.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.537+0000 7f2532d9d640 1 --2- 172.21.15.100:0/4207390182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2534102490 0x7f2534199e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:03.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.537+0000 7f2532d9d640 1 -- 172.21.15.100:0/4207390182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2534107dd0 msgr2=0x7f253419a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:03.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.537+0000 7f2532d9d640 1 --2- 172.21.15.100:0/4207390182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2534107dd0 0x7f253419a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:03.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.537+0000 7f2532d9d640 1 -- 172.21.15.100:0/4207390182 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f253419fd70 con 0x7f2534108720 2024-07-26T13:31:03.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.537+0000 7f2532d9d640 1 --2- 172.21.15.100:0/4207390182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2534108720 0x7f253419f640 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f252c005cc0 tx=0x7f252c02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:31:03.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.537+0000 7f25237fe640 1 -- 172.21.15.100:0/4207390182 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c003bf0 con 0x7f2534108720 2024-07-26T13:31:03.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.538+0000 7f25237fe640 1 -- 172.21.15.100:0/4207390182 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f252c003d90 con 0x7f2534108720 2024-07-26T13:31:03.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.538+0000 7f2538f49640 1 -- 172.21.15.100:0/4207390182 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25341a0000 con 0x7f2534108720 2024-07-26T13:31:03.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.538+0000 7f25237fe640 1 -- 172.21.15.100:0/4207390182 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c02fbc0 con 0x7f2534108720 2024-07-26T13:31:03.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.538+0000 7f2538f49640 1 -- 172.21.15.100:0/4207390182 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25341a67c0 con 0x7f2534108720 2024-07-26T13:31:03.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.539+0000 7f2538f49640 1 -- 172.21.15.100:0/4207390182 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25341028a0 con 0x7f2534108720 2024-07-26T13:31:03.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.540+0000 7f25237fe640 1 -- 172.21.15.100:0/4207390182 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7f252c02fd60 con 0x7f2534108720 2024-07-26T13:31:03.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.541+0000 7f25237fe640 1 --2- 172.21.15.100:0/4207390182 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f251c0b3c10 0x7f251c0b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:03.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.541+0000 7f253259c640 1 --2- 172.21.15.100:0/4207390182 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f251c0b3c10 0x7f251c0b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:03.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.541+0000 7f25237fe640 1 -- 172.21.15.100:0/4207390182 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f252c103790 con 0x7f2534108720 2024-07-26T13:31:03.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.541+0000 7f253259c640 1 --2- 172.21.15.100:0/4207390182 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f251c0b3c10 0x7f251c0b6120 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2524005fd0 tx=0x7f252400b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:31:03.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.544+0000 7f25237fe640 1 -- 172.21.15.100:0/4207390182 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f252c0cc9e0 con 0x7f2534108720 2024-07-26T13:31:03.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:03 smithi138 ceph-mon[36954]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:03.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:03 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/1240474165' entity='client.admin' 2024-07-26T13:31:03.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:31:03.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:31:03.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:31:03.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:03.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.721+0000 7f2538f49640 1 -- 172.21.15.100:0/4207390182 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f253419af30 con 0x7f2534108720 2024-07-26T13:31:03.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.725+0000 7f25237fe640 1 -- 172.21.15.100:0/4207390182 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v29)=0 v29) ==== 121+0+0 (secure 0 0 0) 0x7f253419af30 con 0x7f2534108720 2024-07-26T13:31:03.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.727+0000 7f2538f49640 1 -- 172.21.15.100:0/4207390182 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f251c0b3c10 msgr2=0x7f251c0b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:03.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.727+0000 7f2538f49640 1 --2- 172.21.15.100:0/4207390182 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f251c0b3c10 0x7f251c0b6120 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2524005fd0 tx=0x7f252400b390 comp rx=0 tx=0).stop 2024-07-26T13:31:03.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.727+0000 7f2538f49640 1 -- 172.21.15.100:0/4207390182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2534108720 msgr2=0x7f253419f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:03.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.727+0000 7f2538f49640 1 --2- 172.21.15.100:0/4207390182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2534108720 0x7f253419f640 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f252c005cc0 tx=0x7f252c02f380 comp rx=0 tx=0).stop 2024-07-26T13:31:03.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.728+0000 7f2538f49640 1 -- 172.21.15.100:0/4207390182 shutdown_connections 2024-07-26T13:31:03.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.728+0000 7f2538f49640 1 --2- 172.21.15.100:0/4207390182 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f251c0b3c10 0x7f251c0b6120 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:03.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.728+0000 7f2538f49640 1 --2- 172.21.15.100:0/4207390182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2534108720 0x7f253419f640 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:03.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.728+0000 7f2538f49640 1 --2- 172.21.15.100:0/4207390182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2534107dd0 0x7f253419a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:03.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.728+0000 7f2538f49640 1 --2- 172.21.15.100:0/4207390182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2534102490 0x7f2534199e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:03.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.728+0000 7f2538f49640 1 -- 172.21.15.100:0/4207390182 >> 172.21.15.100:0/4207390182 conn(0x7f2534079160 msgr2=0x7f2534104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:31:03.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.728+0000 7f2538f49640 1 -- 172.21.15.100:0/4207390182 shutdown_connections 2024-07-26T13:31:03.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.728+0000 7f2538f49640 1 -- 172.21.15.100:0/4207390182 wait complete. 2024-07-26T13:31:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:03 smithi167 ceph-mon[37140]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:03 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/1240474165' entity='client.admin' 2024-07-26T13:31:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:31:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:31:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:31:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:04.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.777+0000 7f5afddc2640 1 -- 172.21.15.100:0/530981804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5af8075160 msgr2=0x7f5af8075570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:04.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.777+0000 7f5afddc2640 1 --2- 172.21.15.100:0/530981804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5af8075160 0x7f5af8075570 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8009900 tx=0x7f5ae802a410 comp rx=0 tx=0).stop 2024-07-26T13:31:04.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.777+0000 7f5afddc2640 1 -- 172.21.15.100:0/530981804 shutdown_connections 2024-07-26T13:31:04.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.777+0000 7f5afddc2640 1 --2- 172.21.15.100:0/530981804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5af810fc40 0x7f5af8112060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:04.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.777+0000 7f5afddc2640 1 --2- 172.21.15.100:0/530981804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5af8075ab0 0x7f5af810f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:04.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.777+0000 7f5afddc2640 1 --2- 172.21.15.100:0/530981804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5af8075160 0x7f5af8075570 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:04.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.777+0000 7f5afddc2640 1 -- 172.21.15.100:0/530981804 >> 172.21.15.100:0/530981804 conn(0x7f5af807b300 msgr2=0x7f5af807b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:31:04.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.777+0000 7f5afddc2640 1 -- 172.21.15.100:0/530981804 shutdown_connections 2024-07-26T13:31:04.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.777+0000 7f5afddc2640 1 -- 172.21.15.100:0/530981804 wait complete. 2024-07-26T13:31:04.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.778+0000 7f5afddc2640 1 Processor -- start 2024-07-26T13:31:04.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.778+0000 7f5afddc2640 1 -- start start 2024-07-26T13:31:04.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.778+0000 7f5afddc2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5af8075160 0x7f5af819c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:04.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.778+0000 7f5afddc2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5af8075ab0 0x7f5af819c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:04.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.778+0000 7f5afddc2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5af810fc40 0x7f5af81a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:04.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.778+0000 7f5afddc2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5af8115760 con 0x7f5af8075160 2024-07-26T13:31:04.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.778+0000 7f5afddc2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5af81155e0 con 0x7f5af8075ab0 2024-07-26T13:31:04.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.778+0000 7f5afddc2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5af81158e0 con 0x7f5af810fc40 2024-07-26T13:31:04.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.778+0000 7f5af77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5af8075160 0x7f5af819c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:04.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.778+0000 7f5af77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5af8075160 0x7f5af819c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39762/0 (socket says 172.21.15.100:39762) 2024-07-26T13:31:04.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.778+0000 7f5af77fe640 1 -- 172.21.15.100:0/672362902 learned_addr learned my addr 172.21.15.100:0/672362902 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:31:04.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.778+0000 7f5af6ffd640 1 --2- 172.21.15.100:0/672362902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5af8075ab0 0x7f5af819c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:04.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.778+0000 7f5af7fff640 1 --2- 172.21.15.100:0/672362902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5af810fc40 0x7f5af81a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:04.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.779+0000 7f5af77fe640 1 -- 172.21.15.100:0/672362902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5af810fc40 msgr2=0x7f5af81a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:04.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.779+0000 7f5af77fe640 1 --2- 172.21.15.100:0/672362902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5af810fc40 0x7f5af81a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:04.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.779+0000 7f5af77fe640 1 -- 172.21.15.100:0/672362902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5af8075ab0 msgr2=0x7f5af819c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:04.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.779+0000 7f5af77fe640 1 --2- 172.21.15.100:0/672362902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5af8075ab0 0x7f5af819c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:04.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.779+0000 7f5af77fe640 1 -- 172.21.15.100:0/672362902 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af81a2100 con 0x7f5af8075160 2024-07-26T13:31:04.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.779+0000 7f5af7fff640 1 --2- 172.21.15.100:0/672362902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5af810fc40 0x7f5af81a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:31:04.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.779+0000 7f5af77fe640 1 --2- 172.21.15.100:0/672362902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5af8075160 0x7f5af819c190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8005ab0 tx=0x7f5ae8003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:31:04.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.779+0000 7f5af4ff9640 1 -- 172.21.15.100:0/672362902 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae803f070 con 0x7f5af8075160 2024-07-26T13:31:04.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.779+0000 7f5afddc2640 1 -- 172.21.15.100:0/672362902 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5af81a2390 con 0x7f5af8075160 2024-07-26T13:31:04.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.779+0000 7f5af4ff9640 1 -- 172.21.15.100:0/672362902 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ae8031d60 con 0x7f5af8075160 2024-07-26T13:31:04.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.779+0000 7f5af4ff9640 1 -- 172.21.15.100:0/672362902 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8035030 con 0x7f5af8075160 2024-07-26T13:31:04.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.779+0000 7f5afddc2640 1 -- 172.21.15.100:0/672362902 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5af81a8b50 con 0x7f5af8075160 2024-07-26T13:31:04.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.780+0000 7f5afddc2640 1 -- 172.21.15.100:0/672362902 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5af8075570 con 0x7f5af8075160 2024-07-26T13:31:04.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.782+0000 7f5af4ff9640 1 -- 172.21.15.100:0/672362902 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7f5ae8042070 con 0x7f5af8075160 2024-07-26T13:31:04.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.782+0000 7f5af4ff9640 1 --2- 172.21.15.100:0/672362902 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5ae40b3b90 0x7f5ae40b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:04.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.783+0000 7f5af6ffd640 1 --2- 172.21.15.100:0/672362902 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5ae40b3b90 0x7f5ae40b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:04.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.783+0000 7f5af4ff9640 1 -- 172.21.15.100:0/672362902 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f5ae8102ec0 con 0x7f5af8075160 2024-07-26T13:31:04.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.783+0000 7f5af6ffd640 1 --2- 172.21.15.100:0/672362902 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5ae40b3b90 0x7f5ae40b60a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5af8111790 tx=0x7f5ae000a250 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:31:04.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.786+0000 7f5af4ff9640 1 -- 172.21.15.100:0/672362902 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f5ae80cc110 con 0x7f5af8075160 2024-07-26T13:31:04.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.956+0000 7f1dcb735640 1 -- 172.21.15.100:0/3564050915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1dc4102060 msgr2=0x7f1dc4102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:04.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.956+0000 7f1dcb735640 1 --2- 172.21.15.100:0/3564050915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1dc4102060 0x7f1dc4102470 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009900 tx=0x7f1dc002a480 comp rx=0 tx=0).stop 2024-07-26T13:31:04.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.956+0000 7f1dcb735640 1 -- 172.21.15.100:0/3564050915 shutdown_connections 2024-07-26T13:31:04.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.956+0000 7f1dcb735640 1 --2- 172.21.15.100:0/3564050915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dc410fc10 0x7f1dc4112030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:04.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.956+0000 7f1dcb735640 1 --2- 172.21.15.100:0/3564050915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dc41029b0 0x7f1dc410f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:04.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.956+0000 7f1dcb735640 1 --2- 172.21.15.100:0/3564050915 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1dc4102060 0x7f1dc4102470 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:04.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.956+0000 7f1dcb735640 1 -- 172.21.15.100:0/3564050915 >> 172.21.15.100:0/3564050915 conn(0x7f1dc4079160 msgr2=0x7f1dc4079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:31:04.958 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:04 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/4207390182' entity='client.admin' 2024-07-26T13:31:04.958 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:04 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/876345700' entity='client.admin' 2024-07-26T13:31:04.958 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:04 smithi100 ceph-mon[32599]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:04.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.957+0000 7f1dcb735640 1 -- 172.21.15.100:0/3564050915 shutdown_connections 2024-07-26T13:31:04.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.957+0000 7f1dcb735640 1 -- 172.21.15.100:0/3564050915 wait complete. 2024-07-26T13:31:04.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.958+0000 7f1dcb735640 1 Processor -- start 2024-07-26T13:31:04.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.958+0000 7f1dcb735640 1 -- start start 2024-07-26T13:31:04.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.958+0000 7f1dcb735640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1dc4102060 0x7f1dc419c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:04.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.958+0000 7f1dcb735640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dc41029b0 0x7f1dc419c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:04.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.958+0000 7f1dcb735640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dc410fc10 0x7f1dc41a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:04.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.958+0000 7f1dcb735640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc4115690 con 0x7f1dc4102060 2024-07-26T13:31:04.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.958+0000 7f1dcb735640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc4115510 con 0x7f1dc410fc10 2024-07-26T13:31:04.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.958+0000 7f1dcb735640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc4115810 con 0x7f1dc41029b0 2024-07-26T13:31:04.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.958+0000 7f1dc94d1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1dc4102060 0x7f1dc419c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:04.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.958+0000 7f1dc94d1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1dc4102060 0x7f1dc419c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39792/0 (socket says 172.21.15.100:39792) 2024-07-26T13:31:04.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.958+0000 7f1dc94d1640 1 -- 172.21.15.100:0/190495084 learned_addr learned my addr 172.21.15.100:0/190495084 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:31:04.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.958+0000 7f1dc8cd0640 1 --2- 172.21.15.100:0/190495084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dc41029b0 0x7f1dc419c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:04.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.958+0000 7f1dc9cd2640 1 --2- 172.21.15.100:0/190495084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dc410fc10 0x7f1dc41a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:04.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.958+0000 7f1dc94d1640 1 -- 172.21.15.100:0/190495084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dc41029b0 msgr2=0x7f1dc419c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:04.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.958+0000 7f1dc94d1640 1 --2- 172.21.15.100:0/190495084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dc41029b0 0x7f1dc419c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:04.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.958+0000 7f1dc94d1640 1 -- 172.21.15.100:0/190495084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dc410fc10 msgr2=0x7f1dc41a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:04.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.958+0000 7f1dc94d1640 1 --2- 172.21.15.100:0/190495084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dc410fc10 0x7f1dc41a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:04.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.959+0000 7f1dc94d1640 1 -- 172.21.15.100:0/190495084 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc41a1f50 con 0x7f1dc4102060 2024-07-26T13:31:04.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.959+0000 7f1dc9cd2640 1 --2- 172.21.15.100:0/190495084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dc410fc10 0x7f1dc41a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:31:04.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.959+0000 7f1dc94d1640 1 --2- 172.21.15.100:0/190495084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1dc4102060 0x7f1dc419c0a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0005ab0 tx=0x7f1dc0003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:31:04.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.959+0000 7f1dba7fc640 1 -- 172.21.15.100:0/190495084 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc003f070 con 0x7f1dc4102060 2024-07-26T13:31:04.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.959+0000 7f1dba7fc640 1 -- 172.21.15.100:0/190495084 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dc0031d60 con 0x7f1dc4102060 2024-07-26T13:31:04.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.959+0000 7f1dba7fc640 1 -- 172.21.15.100:0/190495084 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0035030 con 0x7f1dc4102060 2024-07-26T13:31:04.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.959+0000 7f1dcb735640 1 -- 172.21.15.100:0/190495084 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dc41a21e0 con 0x7f1dc4102060 2024-07-26T13:31:04.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.959+0000 7f1dcb735640 1 -- 172.21.15.100:0/190495084 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dc41a2670 con 0x7f1dc4102060 2024-07-26T13:31:04.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.960+0000 7f1dcb735640 1 -- 172.21.15.100:0/190495084 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dc4102470 con 0x7f1dc4102060 2024-07-26T13:31:04.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.962+0000 7f1dba7fc640 1 -- 172.21.15.100:0/190495084 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7f1dc0042070 con 0x7f1dc4102060 2024-07-26T13:31:04.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.963+0000 7f1dba7fc640 1 --2- 172.21.15.100:0/190495084 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1db00b3bc0 0x7f1db00b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:04.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.963+0000 7f1dc8cd0640 1 --2- 172.21.15.100:0/190495084 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1db00b3bc0 0x7f1db00b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:04.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.963+0000 7f1dba7fc640 1 -- 172.21.15.100:0/190495084 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f1dc0102b00 con 0x7f1dc4102060 2024-07-26T13:31:04.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.963+0000 7f1dc8cd0640 1 --2- 172.21.15.100:0/190495084 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1db00b3bc0 0x7f1db00b60d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4103950 tx=0x7f1db400b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:31:04.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.970+0000 7f1dba7fc640 1 -- 172.21.15.100:0/190495084 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f1dc00cbd50 con 0x7f1dc4102060 2024-07-26T13:31:04.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.978+0000 7f5afddc2640 1 -- 172.21.15.100:0/672362902 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=debug_cephsqlite}] v 0) -- 0x7f5af81a8db0 con 0x7f5af8075160 2024-07-26T13:31:04.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.984+0000 7f5af4ff9640 1 -- 172.21.15.100:0/672362902 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_cephsqlite}]=0 v31)=0 v31) ==== 115+0+0 (secure 0 0 0) 0x7f5ae80cba10 con 0x7f5af8075160 2024-07-26T13:31:04.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.986+0000 7f5afddc2640 1 -- 172.21.15.100:0/672362902 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5ae40b3b90 msgr2=0x7f5ae40b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:04.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.986+0000 7f5afddc2640 1 --2- 172.21.15.100:0/672362902 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5ae40b3b90 0x7f5ae40b60a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5af8111790 tx=0x7f5ae000a250 comp rx=0 tx=0).stop 2024-07-26T13:31:04.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.986+0000 7f5afddc2640 1 -- 172.21.15.100:0/672362902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5af8075160 msgr2=0x7f5af819c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:04.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.986+0000 7f5afddc2640 1 --2- 172.21.15.100:0/672362902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5af8075160 0x7f5af819c190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8005ab0 tx=0x7f5ae8003cd0 comp rx=0 tx=0).stop 2024-07-26T13:31:04.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.986+0000 7f5afddc2640 1 -- 172.21.15.100:0/672362902 shutdown_connections 2024-07-26T13:31:04.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.986+0000 7f5afddc2640 1 --2- 172.21.15.100:0/672362902 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5ae40b3b90 0x7f5ae40b60a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:04.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.986+0000 7f5afddc2640 1 --2- 172.21.15.100:0/672362902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5af810fc40 0x7f5af81a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:04.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.986+0000 7f5afddc2640 1 --2- 172.21.15.100:0/672362902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5af8075ab0 0x7f5af819c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:04.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.986+0000 7f5afddc2640 1 --2- 172.21.15.100:0/672362902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5af8075160 0x7f5af819c190 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:04.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.986+0000 7f5afddc2640 1 -- 172.21.15.100:0/672362902 >> 172.21.15.100:0/672362902 conn(0x7f5af807b300 msgr2=0x7f5af8109770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:31:04.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.987+0000 7f5afddc2640 1 -- 172.21.15.100:0/672362902 shutdown_connections 2024-07-26T13:31:04.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:04.987+0000 7f5afddc2640 1 -- 172.21.15.100:0/672362902 wait complete. 2024-07-26T13:31:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:04 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/4207390182' entity='client.admin' 2024-07-26T13:31:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:04 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/876345700' entity='client.admin' 2024-07-26T13:31:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:04 smithi167 ceph-mon[37140]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:05.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:05.163+0000 7f1dcb735640 1 -- 172.21.15.100:0/190495084 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f1dc41a8b50 con 0x7f1dc4102060 2024-07-26T13:31:05.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:05.167+0000 7f1dba7fc640 1 -- 172.21.15.100:0/190495084 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v32)=0 v32) ==== 120+0+0 (secure 0 0 0) 0x7f1dc00cb650 con 0x7f1dc4102060 2024-07-26T13:31:05.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:05.170+0000 7f1dcb735640 1 -- 172.21.15.100:0/190495084 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1db00b3bc0 msgr2=0x7f1db00b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:05.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:05.170+0000 7f1dcb735640 1 --2- 172.21.15.100:0/190495084 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1db00b3bc0 0x7f1db00b60d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4103950 tx=0x7f1db400b410 comp rx=0 tx=0).stop 2024-07-26T13:31:05.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:05.170+0000 7f1dcb735640 1 -- 172.21.15.100:0/190495084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1dc4102060 msgr2=0x7f1dc419c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:05.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:05.170+0000 7f1dcb735640 1 --2- 172.21.15.100:0/190495084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1dc4102060 0x7f1dc419c0a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0005ab0 tx=0x7f1dc0003cd0 comp rx=0 tx=0).stop 2024-07-26T13:31:05.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:05.170+0000 7f1dcb735640 1 -- 172.21.15.100:0/190495084 shutdown_connections 2024-07-26T13:31:05.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:05.170+0000 7f1dcb735640 1 --2- 172.21.15.100:0/190495084 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1db00b3bc0 0x7f1db00b60d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:05.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:05.170+0000 7f1dcb735640 1 --2- 172.21.15.100:0/190495084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dc410fc10 0x7f1dc41a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:05.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:05.170+0000 7f1dcb735640 1 --2- 172.21.15.100:0/190495084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dc41029b0 0x7f1dc419c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:05.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:05.170+0000 7f1dcb735640 1 --2- 172.21.15.100:0/190495084 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1dc4102060 0x7f1dc419c0a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:05.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:05.170+0000 7f1dcb735640 1 -- 172.21.15.100:0/190495084 >> 172.21.15.100:0/190495084 conn(0x7f1dc4079160 msgr2=0x7f1dc41097b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:31:05.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:05.170+0000 7f1dcb735640 1 -- 172.21.15.100:0/190495084 shutdown_connections 2024-07-26T13:31:05.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:05.170+0000 7f1dcb735640 1 -- 172.21.15.100:0/190495084 wait complete. 2024-07-26T13:31:05.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:04 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/4207390182' entity='client.admin' 2024-07-26T13:31:05.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:04 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/876345700' entity='client.admin' 2024-07-26T13:31:05.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:04 smithi138 ceph-mon[36954]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:06.234 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:05 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/672362902' entity='client.admin' 2024-07-26T13:31:06.234 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:05 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:31:06.234 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:05 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:31:06.234 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:05 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:31:06.234 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:05 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:06.234 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:05 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/190495084' entity='client.admin' 2024-07-26T13:31:06.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:05 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/672362902' entity='client.admin' 2024-07-26T13:31:06.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:05 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:31:06.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:05 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:31:06.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:05 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:31:06.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:05 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:06.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:05 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/190495084' entity='client.admin' 2024-07-26T13:31:06.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:05 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/672362902' entity='client.admin' 2024-07-26T13:31:06.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:05 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:31:06.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:05 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:31:06.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:05 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:31:06.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:05 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:06.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:05 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/190495084' entity='client.admin' 2024-07-26T13:31:06.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.772+0000 7f27e919d640 1 -- 172.21.15.100:0/1460627897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27e4105520 msgr2=0x7f27e410ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:06.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.772+0000 7f27e919d640 1 --2- 172.21.15.100:0/1460627897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27e4105520 0x7f27e410ef70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f27dc009ca0 tx=0x7f27dc02a8c0 comp rx=0 tx=0).stop 2024-07-26T13:31:06.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.772+0000 7f27e919d640 1 -- 172.21.15.100:0/1460627897 shutdown_connections 2024-07-26T13:31:06.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.772+0000 7f27e919d640 1 --2- 172.21.15.100:0/1460627897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27e4105520 0x7f27e410ef70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:06.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.772+0000 7f27e919d640 1 --2- 172.21.15.100:0/1460627897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27e4104bd0 0x7f27e4104fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:06.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.772+0000 7f27e919d640 1 --2- 172.21.15.100:0/1460627897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27e4102ae0 0x7f27e4102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:06.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.772+0000 7f27e919d640 1 -- 172.21.15.100:0/1460627897 >> 172.21.15.100:0/1460627897 conn(0x7f27e40776d0 msgr2=0x7f27e4077b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:31:06.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.772+0000 7f27e919d640 1 -- 172.21.15.100:0/1460627897 shutdown_connections 2024-07-26T13:31:06.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.772+0000 7f27e919d640 1 -- 172.21.15.100:0/1460627897 wait complete. 2024-07-26T13:31:06.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.773+0000 7f27e919d640 1 Processor -- start 2024-07-26T13:31:06.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.773+0000 7f27e919d640 1 -- start start 2024-07-26T13:31:06.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.773+0000 7f27e919d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27e4102ae0 0x7f27e419acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:06.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.773+0000 7f27e919d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27e4104bd0 0x7f27e419b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:06.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.773+0000 7f27e919d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27e4105520 0x7f27e419b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:06.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.774+0000 7f27e919d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f27e41123e0 con 0x7f27e4105520 2024-07-26T13:31:06.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.774+0000 7f27e919d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f27e4112260 con 0x7f27e4104bd0 2024-07-26T13:31:06.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.774+0000 7f27e919d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f27e4112560 con 0x7f27e4102ae0 2024-07-26T13:31:06.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.774+0000 7f27e359e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27e4105520 0x7f27e419b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:06.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.774+0000 7f27e359e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27e4105520 0x7f27e419b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39750/0 (socket says 172.21.15.100:39750) 2024-07-26T13:31:06.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.774+0000 7f27e359e640 1 -- 172.21.15.100:0/876345700 learned_addr learned my addr 172.21.15.100:0/876345700 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:31:06.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.774+0000 7f27e2d9d640 1 --2- 172.21.15.100:0/876345700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27e4102ae0 0x7f27e419acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:06.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.774+0000 7f27e259c640 1 --2- 172.21.15.100:0/876345700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27e4104bd0 0x7f27e419b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:06.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.774+0000 7f27e359e640 1 -- 172.21.15.100:0/876345700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27e4102ae0 msgr2=0x7f27e419acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:06.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.774+0000 7f27e359e640 1 --2- 172.21.15.100:0/876345700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27e4102ae0 0x7f27e419acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:06.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.774+0000 7f27e359e640 1 -- 172.21.15.100:0/876345700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27e4104bd0 msgr2=0x7f27e419b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:06.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.774+0000 7f27e359e640 1 --2- 172.21.15.100:0/876345700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27e4104bd0 0x7f27e419b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:06.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.774+0000 7f27e359e640 1 -- 172.21.15.100:0/876345700 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e41a5f60 con 0x7f27e4105520 2024-07-26T13:31:06.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.774+0000 7f27e359e640 1 --2- 172.21.15.100:0/876345700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27e4105520 0x7f27e419b750 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f27dc005cc0 tx=0x7f27dc02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:31:06.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.774+0000 7f27cbfff640 1 -- 172.21.15.100:0/876345700 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc003bf0 con 0x7f27e4105520 2024-07-26T13:31:06.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.774+0000 7f27e919d640 1 -- 172.21.15.100:0/876345700 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27e41a61f0 con 0x7f27e4105520 2024-07-26T13:31:06.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.774+0000 7f27cbfff640 1 -- 172.21.15.100:0/876345700 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27dc003d90 con 0x7f27e4105520 2024-07-26T13:31:06.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.774+0000 7f27cbfff640 1 -- 172.21.15.100:0/876345700 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc02fbc0 con 0x7f27e4105520 2024-07-26T13:31:06.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.775+0000 7f27e919d640 1 -- 172.21.15.100:0/876345700 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27e41a6730 con 0x7f27e4105520 2024-07-26T13:31:06.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.775+0000 7f27e919d640 1 -- 172.21.15.100:0/876345700 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27e4102ef0 con 0x7f27e4105520 2024-07-26T13:31:06.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.777+0000 7f27cbfff640 1 -- 172.21.15.100:0/876345700 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7f27dc02fd60 con 0x7f27e4105520 2024-07-26T13:31:06.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.778+0000 7f27cbfff640 1 --2- 172.21.15.100:0/876345700 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f27cc0b3bc0 0x7f27cc0b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:06.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.778+0000 7f27e2d9d640 1 --2- 172.21.15.100:0/876345700 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f27cc0b3bc0 0x7f27cc0b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:06.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.778+0000 7f27cbfff640 1 -- 172.21.15.100:0/876345700 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f27dc103700 con 0x7f27e4105520 2024-07-26T13:31:06.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.778+0000 7f27e2d9d640 1 --2- 172.21.15.100:0/876345700 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f27cc0b3bc0 0x7f27cc0b60d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f27d40058c0 tx=0x7f27d400a2d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:31:06.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.781+0000 7f27cbfff640 1 -- 172.21.15.100:0/876345700 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f27dc0cc950 con 0x7f27e4105520 2024-07-26T13:31:06.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.962+0000 7f27e919d640 1 -- 172.21.15.100:0/876345700 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7f27e402e0c0 con 0x7f27e4105520 2024-07-26T13:31:06.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.966+0000 7f27cbfff640 1 -- 172.21.15.100:0/876345700 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v30)=0 v30) ==== 125+0+0 (secure 0 0 0) 0x7f27dc0cc250 con 0x7f27e4105520 2024-07-26T13:31:06.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.969+0000 7f27e919d640 1 -- 172.21.15.100:0/876345700 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f27cc0b3bc0 msgr2=0x7f27cc0b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:06.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.969+0000 7f27e919d640 1 --2- 172.21.15.100:0/876345700 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f27cc0b3bc0 0x7f27cc0b60d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f27d40058c0 tx=0x7f27d400a2d0 comp rx=0 tx=0).stop 2024-07-26T13:31:06.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.969+0000 7f27e919d640 1 -- 172.21.15.100:0/876345700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27e4105520 msgr2=0x7f27e419b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:06.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.969+0000 7f27e919d640 1 --2- 172.21.15.100:0/876345700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27e4105520 0x7f27e419b750 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f27dc005cc0 tx=0x7f27dc02f380 comp rx=0 tx=0).stop 2024-07-26T13:31:06.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.969+0000 7f27e919d640 1 -- 172.21.15.100:0/876345700 shutdown_connections 2024-07-26T13:31:06.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.969+0000 7f27e919d640 1 --2- 172.21.15.100:0/876345700 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f27cc0b3bc0 0x7f27cc0b60d0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:06.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.969+0000 7f27e919d640 1 --2- 172.21.15.100:0/876345700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27e4105520 0x7f27e419b750 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:06.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.969+0000 7f27e919d640 1 --2- 172.21.15.100:0/876345700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27e4104bd0 0x7f27e419b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:06.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.969+0000 7f27e919d640 1 --2- 172.21.15.100:0/876345700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27e4102ae0 0x7f27e419acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:06.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.969+0000 7f27e919d640 1 -- 172.21.15.100:0/876345700 >> 172.21.15.100:0/876345700 conn(0x7f27e40776d0 msgr2=0x7f27e4105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:31:06.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.969+0000 7f27e919d640 1 -- 172.21.15.100:0/876345700 shutdown_connections 2024-07-26T13:31:06.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:03.969+0000 7f27e919d640 1 -- 172.21.15.100:0/876345700 wait complete. 2024-07-26T13:31:07.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.092+0000 7f1b55d38640 1 -- 172.21.15.100:0/3928435743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b501098e0 msgr2=0x7f1b50110110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:07.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.092+0000 7f1b55d38640 1 --2- 172.21.15.100:0/3928435743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b501098e0 0x7f1b50110110 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1b48009ca0 tx=0x7f1b4802a8c0 comp rx=0 tx=0).stop 2024-07-26T13:31:07.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.092+0000 7f1b55d38640 1 -- 172.21.15.100:0/3928435743 shutdown_connections 2024-07-26T13:31:07.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.092+0000 7f1b55d38640 1 --2- 172.21.15.100:0/3928435743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b501098e0 0x7f1b50110110 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:07.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.092+0000 7f1b55d38640 1 --2- 172.21.15.100:0/3928435743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b50108f90 0x7f1b501093a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:07.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.092+0000 7f1b55d38640 1 --2- 172.21.15.100:0/3928435743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b50101ab0 0x7f1b50101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:07.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.092+0000 7f1b55d38640 1 -- 172.21.15.100:0/3928435743 >> 172.21.15.100:0/3928435743 conn(0x7f1b500798f0 msgr2=0x7f1b50079d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:31:07.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.092+0000 7f1b55d38640 1 -- 172.21.15.100:0/3928435743 shutdown_connections 2024-07-26T13:31:07.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.092+0000 7f1b55d38640 1 -- 172.21.15.100:0/3928435743 wait complete. 2024-07-26T13:31:07.094 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:06 smithi100 ceph-mon[32599]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:07.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.092+0000 7f1b55d38640 1 Processor -- start 2024-07-26T13:31:07.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.093+0000 7f1b55d38640 1 -- start start 2024-07-26T13:31:07.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.093+0000 7f1b55d38640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b50101ab0 0x7f1b50199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:07.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.093+0000 7f1b55d38640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b50108f90 0x7f1b5019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:07.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.093+0000 7f1b55d38640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b501098e0 0x7f1b5019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:07.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.093+0000 7f1b55d38640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1b501135e0 con 0x7f1b50101ab0 2024-07-26T13:31:07.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.093+0000 7f1b55d38640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1b50113460 con 0x7f1b50108f90 2024-07-26T13:31:07.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.093+0000 7f1b55d38640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1b50113760 con 0x7f1b501098e0 2024-07-26T13:31:07.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.093+0000 7f1b4f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b50101ab0 0x7f1b50199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:07.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.093+0000 7f1b4f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b50101ab0 0x7f1b50199ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51746/0 (socket says 172.21.15.100:51746) 2024-07-26T13:31:07.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.093+0000 7f1b4f7fe640 1 -- 172.21.15.100:0/732732 learned_addr learned my addr 172.21.15.100:0/732732 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:31:07.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.093+0000 7f1b4effd640 1 --2- 172.21.15.100:0/732732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b50108f90 0x7f1b5019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:07.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.093+0000 7f1b4ffff640 1 --2- 172.21.15.100:0/732732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b501098e0 0x7f1b5019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:07.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.093+0000 7f1b4f7fe640 1 -- 172.21.15.100:0/732732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b501098e0 msgr2=0x7f1b5019f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:07.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.093+0000 7f1b4f7fe640 1 --2- 172.21.15.100:0/732732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b501098e0 0x7f1b5019f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:07.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.093+0000 7f1b4f7fe640 1 -- 172.21.15.100:0/732732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b50108f90 msgr2=0x7f1b5019a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:07.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.093+0000 7f1b4f7fe640 1 --2- 172.21.15.100:0/732732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b50108f90 0x7f1b5019a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:07.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.093+0000 7f1b4f7fe640 1 -- 172.21.15.100:0/732732 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b5019fe40 con 0x7f1b50101ab0 2024-07-26T13:31:07.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.094+0000 7f1b4f7fe640 1 --2- 172.21.15.100:0/732732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b50101ab0 0x7f1b50199ed0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1b4000c890 tx=0x7f1b4000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:31:07.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.094+0000 7f1b4cff9640 1 -- 172.21.15.100:0/732732 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b40004220 con 0x7f1b50101ab0 2024-07-26T13:31:07.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.094+0000 7f1b4cff9640 1 -- 172.21.15.100:0/732732 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b400043c0 con 0x7f1b50101ab0 2024-07-26T13:31:07.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.094+0000 7f1b55d38640 1 -- 172.21.15.100:0/732732 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b501a0130 con 0x7f1b50101ab0 2024-07-26T13:31:07.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.094+0000 7f1b4cff9640 1 -- 172.21.15.100:0/732732 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b40004a00 con 0x7f1b50101ab0 2024-07-26T13:31:07.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.094+0000 7f1b55d38640 1 -- 172.21.15.100:0/732732 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b501a6890 con 0x7f1b50101ab0 2024-07-26T13:31:07.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.095+0000 7f1b55d38640 1 -- 172.21.15.100:0/732732 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b50101ec0 con 0x7f1b50101ab0 2024-07-26T13:31:07.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.097+0000 7f1b4cff9640 1 -- 172.21.15.100:0/732732 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7f1b400079b0 con 0x7f1b50101ab0 2024-07-26T13:31:07.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.098+0000 7f1b4cff9640 1 --2- 172.21.15.100:0/732732 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1b380b3b90 0x7f1b380b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:07.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.098+0000 7f1b4effd640 1 --2- 172.21.15.100:0/732732 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1b380b3b90 0x7f1b380b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:07.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.098+0000 7f1b4cff9640 1 -- 172.21.15.100:0/732732 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f1b400e21a0 con 0x7f1b50101ab0 2024-07-26T13:31:07.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.098+0000 7f1b4effd640 1 --2- 172.21.15.100:0/732732 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1b380b3b90 0x7f1b380b60a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1b50108b40 tx=0x7f1b4400b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:31:07.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.100+0000 7f1b4cff9640 1 -- 172.21.15.100:0/732732 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f1b40014030 con 0x7f1b50101ab0 2024-07-26T13:31:07.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:06 smithi138 ceph-mon[36954]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:07.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:06 smithi167 ceph-mon[37140]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:07.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.280+0000 7f1b55d38640 1 -- 172.21.15.100:0/732732 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f1b5019b000 con 0x7f1b50101ab0 2024-07-26T13:31:07.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.284+0000 7f1b4cff9640 1 -- 172.21.15.100:0/732732 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v33)=0 v33) ==== 119+0+0 (secure 0 0 0) 0x7f1b400ab3f0 con 0x7f1b50101ab0 2024-07-26T13:31:07.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.287+0000 7f1b55d38640 1 -- 172.21.15.100:0/732732 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1b380b3b90 msgr2=0x7f1b380b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:07.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.287+0000 7f1b55d38640 1 --2- 172.21.15.100:0/732732 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1b380b3b90 0x7f1b380b60a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1b50108b40 tx=0x7f1b4400b390 comp rx=0 tx=0).stop 2024-07-26T13:31:07.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.287+0000 7f1b55d38640 1 -- 172.21.15.100:0/732732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b50101ab0 msgr2=0x7f1b50199ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:07.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.287+0000 7f1b55d38640 1 --2- 172.21.15.100:0/732732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b50101ab0 0x7f1b50199ed0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1b4000c890 tx=0x7f1b4000cd50 comp rx=0 tx=0).stop 2024-07-26T13:31:07.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.287+0000 7f1b55d38640 1 -- 172.21.15.100:0/732732 shutdown_connections 2024-07-26T13:31:07.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.287+0000 7f1b55d38640 1 --2- 172.21.15.100:0/732732 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1b380b3b90 0x7f1b380b60a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:07.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.287+0000 7f1b55d38640 1 --2- 172.21.15.100:0/732732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b501098e0 0x7f1b5019f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:07.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.287+0000 7f1b55d38640 1 --2- 172.21.15.100:0/732732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b50108f90 0x7f1b5019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:07.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.287+0000 7f1b55d38640 1 --2- 172.21.15.100:0/732732 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b50101ab0 0x7f1b50199ed0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:07.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.287+0000 7f1b55d38640 1 -- 172.21.15.100:0/732732 >> 172.21.15.100:0/732732 conn(0x7f1b500798f0 msgr2=0x7f1b50106f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:31:07.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.287+0000 7f1b55d38640 1 -- 172.21.15.100:0/732732 shutdown_connections 2024-07-26T13:31:07.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.287+0000 7f1b55d38640 1 -- 172.21.15.100:0/732732 wait complete. 2024-07-26T13:31:08.356 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:08 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/732732' entity='client.admin' 2024-07-26T13:31:08.356 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:08 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/3666978576' entity='client.admin' 2024-07-26T13:31:08.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.556+0000 7f89c9e38640 1 -- 172.21.15.100:0/44261672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89c410fb90 msgr2=0x7f89c4111fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:08.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.556+0000 7f89c9e38640 1 --2- 172.21.15.100:0/44261672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89c410fb90 0x7f89c4111fb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f89bc009900 tx=0x7f89bc02a410 comp rx=0 tx=0).stop 2024-07-26T13:31:08.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.556+0000 7f89c9e38640 1 -- 172.21.15.100:0/44261672 shutdown_connections 2024-07-26T13:31:08.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.556+0000 7f89c9e38640 1 --2- 172.21.15.100:0/44261672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89c410fb90 0x7f89c4111fb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:08.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.556+0000 7f89c9e38640 1 --2- 172.21.15.100:0/44261672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89c4075ab0 0x7f89c410f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:08.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.556+0000 7f89c9e38640 1 --2- 172.21.15.100:0/44261672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89c4075160 0x7f89c4075570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:08.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.556+0000 7f89c9e38640 1 -- 172.21.15.100:0/44261672 >> 172.21.15.100:0/44261672 conn(0x7f89c40ff040 msgr2=0x7f89c4101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:31:08.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.556+0000 7f89c9e38640 1 -- 172.21.15.100:0/44261672 shutdown_connections 2024-07-26T13:31:08.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.556+0000 7f89c9e38640 1 -- 172.21.15.100:0/44261672 wait complete. 2024-07-26T13:31:08.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.556+0000 7f89c9e38640 1 Processor -- start 2024-07-26T13:31:08.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.557+0000 7f89c9e38640 1 -- start start 2024-07-26T13:31:08.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.557+0000 7f89c9e38640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89c4075160 0x7f89c419c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:08.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.557+0000 7f89c9e38640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89c4075ab0 0x7f89c419c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:08.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.557+0000 7f89c9e38640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89c410fb90 0x7f89c41a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:08.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.557+0000 7f89c9e38640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f89c4115730 con 0x7f89c4075ab0 2024-07-26T13:31:08.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.557+0000 7f89c9e38640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f89c41155b0 con 0x7f89c410fb90 2024-07-26T13:31:08.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.557+0000 7f89c9e38640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f89c41158b0 con 0x7f89c4075160 2024-07-26T13:31:08.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.557+0000 7f89c2ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89c4075ab0 0x7f89c419c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:08.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.557+0000 7f89c2ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89c4075ab0 0x7f89c419c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51794/0 (socket says 172.21.15.100:51794) 2024-07-26T13:31:08.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.557+0000 7f89c2ffd640 1 -- 172.21.15.100:0/760158468 learned_addr learned my addr 172.21.15.100:0/760158468 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:31:08.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.557+0000 7f89c3fff640 1 --2- 172.21.15.100:0/760158468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89c410fb90 0x7f89c41a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:08.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.557+0000 7f89c37fe640 1 --2- 172.21.15.100:0/760158468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89c4075160 0x7f89c419c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:08.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.557+0000 7f89c2ffd640 1 -- 172.21.15.100:0/760158468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89c4075160 msgr2=0x7f89c419c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:08.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.557+0000 7f89c2ffd640 1 --2- 172.21.15.100:0/760158468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89c4075160 0x7f89c419c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:08.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.557+0000 7f89c2ffd640 1 -- 172.21.15.100:0/760158468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89c410fb90 msgr2=0x7f89c41a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:08.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.557+0000 7f89c2ffd640 1 --2- 172.21.15.100:0/760158468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89c410fb90 0x7f89c41a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:08.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.557+0000 7f89c2ffd640 1 -- 172.21.15.100:0/760158468 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c41a2050 con 0x7f89c4075ab0 2024-07-26T13:31:08.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.558+0000 7f89c2ffd640 1 --2- 172.21.15.100:0/760158468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89c4075ab0 0x7f89c419c680 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f89b8006fd0 tx=0x7f89b800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:31:08.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.558+0000 7f89c0ff9640 1 -- 172.21.15.100:0/760158468 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b8011830 con 0x7f89c4075ab0 2024-07-26T13:31:08.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.558+0000 7f89c0ff9640 1 -- 172.21.15.100:0/760158468 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89b8016070 con 0x7f89c4075ab0 2024-07-26T13:31:08.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.558+0000 7f89c0ff9640 1 -- 172.21.15.100:0/760158468 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b8010e00 con 0x7f89c4075ab0 2024-07-26T13:31:08.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.558+0000 7f89c9e38640 1 -- 172.21.15.100:0/760158468 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89c41a2340 con 0x7f89c4075ab0 2024-07-26T13:31:08.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.558+0000 7f89c9e38640 1 -- 172.21.15.100:0/760158468 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89c41a8aa0 con 0x7f89c4075ab0 2024-07-26T13:31:08.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.559+0000 7f89c9e38640 1 -- 172.21.15.100:0/760158468 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89c4075570 con 0x7f89c4075ab0 2024-07-26T13:31:08.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.560+0000 7f89c0ff9640 1 -- 172.21.15.100:0/760158468 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7f89b801b460 con 0x7f89c4075ab0 2024-07-26T13:31:08.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.561+0000 7f89c0ff9640 1 --2- 172.21.15.100:0/760158468 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f89ac0b3b90 0x7f89ac0b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:08.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.561+0000 7f89c37fe640 1 --2- 172.21.15.100:0/760158468 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f89ac0b3b90 0x7f89ac0b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:08.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.561+0000 7f89c0ff9640 1 -- 172.21.15.100:0/760158468 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f89b80e1d50 con 0x7f89c4075ab0 2024-07-26T13:31:08.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.561+0000 7f89c37fe640 1 --2- 172.21.15.100:0/760158468 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f89ac0b3b90 0x7f89ac0b60a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f89b4005fd0 tx=0x7f89b4009870 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:31:08.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.564+0000 7f89c0ff9640 1 -- 172.21.15.100:0/760158468 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f89b80aaf80 con 0x7f89c4075ab0 2024-07-26T13:31:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:08 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/732732' entity='client.admin' 2024-07-26T13:31:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:08 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/3666978576' entity='client.admin' 2024-07-26T13:31:08.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.774+0000 7f7552dc3640 1 -- 172.21.15.100:0/492734747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f754c108ed0 msgr2=0x7f754c1092e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:08.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.774+0000 7f7552dc3640 1 --2- 172.21.15.100:0/492734747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f754c108ed0 0x7f754c1092e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7538009900 tx=0x7f753802a410 comp rx=0 tx=0).stop 2024-07-26T13:31:08.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.774+0000 7f7552dc3640 1 -- 172.21.15.100:0/492734747 shutdown_connections 2024-07-26T13:31:08.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.774+0000 7f7552dc3640 1 --2- 172.21.15.100:0/492734747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f754c109820 0x7f754c110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:08.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.774+0000 7f7552dc3640 1 --2- 172.21.15.100:0/492734747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f754c108ed0 0x7f754c1092e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:08.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.774+0000 7f7552dc3640 1 --2- 172.21.15.100:0/492734747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f754c103590 0x7f754c1039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:08.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.774+0000 7f7552dc3640 1 -- 172.21.15.100:0/492734747 >> 172.21.15.100:0/492734747 conn(0x7f754c07b280 msgr2=0x7f754c07b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:31:08.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.774+0000 7f7552dc3640 1 -- 172.21.15.100:0/492734747 shutdown_connections 2024-07-26T13:31:08.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.774+0000 7f7552dc3640 1 -- 172.21.15.100:0/492734747 wait complete. 2024-07-26T13:31:08.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.775+0000 7f7552dc3640 1 Processor -- start 2024-07-26T13:31:08.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.775+0000 7f7552dc3640 1 -- start start 2024-07-26T13:31:08.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.775+0000 7f7552dc3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f754c103590 0x7f754c199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:08.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.775+0000 7f7552dc3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f754c108ed0 0x7f754c19a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:08.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.775+0000 7f7552dc3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f754c109820 0x7f754c19f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:08.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.775+0000 7f7552dc3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f754c1134e0 con 0x7f754c103590 2024-07-26T13:31:08.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.775+0000 7f7552dc3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f754c113360 con 0x7f754c108ed0 2024-07-26T13:31:08.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.775+0000 7f7552dc3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f754c113660 con 0x7f754c109820 2024-07-26T13:31:08.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.775+0000 7f7550b5f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f754c103590 0x7f754c199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:08.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.775+0000 7f7550b5f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f754c103590 0x7f754c199f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51772/0 (socket says 172.21.15.100:51772) 2024-07-26T13:31:08.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.775+0000 7f7550b5f640 1 -- 172.21.15.100:0/3666978576 learned_addr learned my addr 172.21.15.100:0/3666978576 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:31:08.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.775+0000 7f7551360640 1 --2- 172.21.15.100:0/3666978576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f754c109820 0x7f754c19f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:08.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.776+0000 7f754bfff640 1 --2- 172.21.15.100:0/3666978576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f754c108ed0 0x7f754c19a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:08.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.776+0000 7f7550b5f640 1 -- 172.21.15.100:0/3666978576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f754c109820 msgr2=0x7f754c19f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:08.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.776+0000 7f7550b5f640 1 --2- 172.21.15.100:0/3666978576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f754c109820 0x7f754c19f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:08.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.776+0000 7f7550b5f640 1 -- 172.21.15.100:0/3666978576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f754c108ed0 msgr2=0x7f754c19a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:08.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.776+0000 7f7550b5f640 1 --2- 172.21.15.100:0/3666978576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f754c108ed0 0x7f754c19a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:08.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.776+0000 7f7550b5f640 1 -- 172.21.15.100:0/3666978576 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f754c19fee0 con 0x7f754c103590 2024-07-26T13:31:08.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.776+0000 7f7550b5f640 1 --2- 172.21.15.100:0/3666978576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f754c103590 0x7f754c199f70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f754000d950 tx=0x7f754000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:31:08.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.776+0000 7f7549ffb640 1 -- 172.21.15.100:0/3666978576 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7540017070 con 0x7f754c103590 2024-07-26T13:31:08.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.776+0000 7f7549ffb640 1 -- 172.21.15.100:0/3666978576 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7540009c30 con 0x7f754c103590 2024-07-26T13:31:08.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.776+0000 7f7552dc3640 1 -- 172.21.15.100:0/3666978576 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f754c1a01d0 con 0x7f754c103590 2024-07-26T13:31:08.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.776+0000 7f7549ffb640 1 -- 172.21.15.100:0/3666978576 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754000f620 con 0x7f754c103590 2024-07-26T13:31:08.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.776+0000 7f7552dc3640 1 -- 172.21.15.100:0/3666978576 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f754c1a6930 con 0x7f754c103590 2024-07-26T13:31:08.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.777+0000 7f7552dc3640 1 -- 172.21.15.100:0/3666978576 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f754c1039a0 con 0x7f754c103590 2024-07-26T13:31:08.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.779+0000 7f7549ffb640 1 -- 172.21.15.100:0/3666978576 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7f7540010460 con 0x7f754c103590 2024-07-26T13:31:08.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.779+0000 7f7549ffb640 1 --2- 172.21.15.100:0/3666978576 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f753c0b3b90 0x7f753c0b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:08.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.779+0000 7f754bfff640 1 --2- 172.21.15.100:0/3666978576 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f753c0b3b90 0x7f753c0b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:08.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.779+0000 7f7549ffb640 1 -- 172.21.15.100:0/3666978576 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f75400e1c50 con 0x7f754c103590 2024-07-26T13:31:08.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.780+0000 7f754bfff640 1 --2- 172.21.15.100:0/3666978576 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f753c0b3b90 0x7f753c0b60a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f753802aa00 tx=0x7f7538031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:31:08.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.784+0000 7f7549ffb640 1 -- 172.21.15.100:0/3666978576 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f75400aaea0 con 0x7f754c103590 2024-07-26T13:31:08.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.960+0000 7f7552dc3640 1 -- 172.21.15.100:0/3666978576 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7f754c19b0a0 con 0x7f754c103590 2024-07-26T13:31:08.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.966+0000 7f7549ffb640 1 -- 172.21.15.100:0/3666978576 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v34)=0 v34) ==== 116+0+0 (secure 0 0 0) 0x7f7540009de0 con 0x7f754c103590 2024-07-26T13:31:08.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.968+0000 7f7552dc3640 1 -- 172.21.15.100:0/3666978576 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f753c0b3b90 msgr2=0x7f753c0b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:08.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.968+0000 7f7552dc3640 1 --2- 172.21.15.100:0/3666978576 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f753c0b3b90 0x7f753c0b60a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f753802aa00 tx=0x7f7538031400 comp rx=0 tx=0).stop 2024-07-26T13:31:08.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.968+0000 7f7552dc3640 1 -- 172.21.15.100:0/3666978576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f754c103590 msgr2=0x7f754c199f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:08.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.968+0000 7f7552dc3640 1 --2- 172.21.15.100:0/3666978576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f754c103590 0x7f754c199f70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f754000d950 tx=0x7f754000de10 comp rx=0 tx=0).stop 2024-07-26T13:31:08.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.969+0000 7f7552dc3640 1 -- 172.21.15.100:0/3666978576 shutdown_connections 2024-07-26T13:31:08.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.969+0000 7f7552dc3640 1 --2- 172.21.15.100:0/3666978576 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f753c0b3b90 0x7f753c0b60a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:08.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.969+0000 7f7552dc3640 1 --2- 172.21.15.100:0/3666978576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f754c109820 0x7f754c19f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:08.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.969+0000 7f7552dc3640 1 --2- 172.21.15.100:0/3666978576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f754c108ed0 0x7f754c19a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:08.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.969+0000 7f7552dc3640 1 --2- 172.21.15.100:0/3666978576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f754c103590 0x7f754c199f70 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:08.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.969+0000 7f7552dc3640 1 -- 172.21.15.100:0/3666978576 >> 172.21.15.100:0/3666978576 conn(0x7f754c07b280 msgr2=0x7f754c107010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:31:08.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.969+0000 7f7552dc3640 1 -- 172.21.15.100:0/3666978576 shutdown_connections 2024-07-26T13:31:08.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:07.969+0000 7f7552dc3640 1 -- 172.21.15.100:0/3666978576 wait complete. 2024-07-26T13:31:08.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.739+0000 7f89c9e38640 1 -- 172.21.15.100:0/760158468 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f89c419d210 con 0x7f89c4075ab0 2024-07-26T13:31:08.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.745+0000 7f89c0ff9640 1 -- 172.21.15.100:0/760158468 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v35)=0 v35) ==== 121+0+0 (secure 0 0 0) 0x7f89c419d210 con 0x7f89c4075ab0 2024-07-26T13:31:08.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.747+0000 7f89c9e38640 1 -- 172.21.15.100:0/760158468 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f89ac0b3b90 msgr2=0x7f89ac0b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:08.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.747+0000 7f89c9e38640 1 --2- 172.21.15.100:0/760158468 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f89ac0b3b90 0x7f89ac0b60a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f89b4005fd0 tx=0x7f89b4009870 comp rx=0 tx=0).stop 2024-07-26T13:31:08.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.747+0000 7f89c9e38640 1 -- 172.21.15.100:0/760158468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89c4075ab0 msgr2=0x7f89c419c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:08.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.747+0000 7f89c9e38640 1 --2- 172.21.15.100:0/760158468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89c4075ab0 0x7f89c419c680 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f89b8006fd0 tx=0x7f89b800b8e0 comp rx=0 tx=0).stop 2024-07-26T13:31:08.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.747+0000 7f89c9e38640 1 -- 172.21.15.100:0/760158468 shutdown_connections 2024-07-26T13:31:08.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.747+0000 7f89c9e38640 1 --2- 172.21.15.100:0/760158468 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f89ac0b3b90 0x7f89ac0b60a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:08.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.747+0000 7f89c9e38640 1 --2- 172.21.15.100:0/760158468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89c410fb90 0x7f89c41a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:08.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.747+0000 7f89c9e38640 1 --2- 172.21.15.100:0/760158468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89c4075ab0 0x7f89c419c680 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:08.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.747+0000 7f89c9e38640 1 --2- 172.21.15.100:0/760158468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89c4075160 0x7f89c419c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:08.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.747+0000 7f89c9e38640 1 -- 172.21.15.100:0/760158468 >> 172.21.15.100:0/760158468 conn(0x7f89c40ff040 msgr2=0x7f89c41096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:31:08.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.748+0000 7f89c9e38640 1 -- 172.21.15.100:0/760158468 shutdown_connections 2024-07-26T13:31:08.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:08.748+0000 7f89c9e38640 1 -- 172.21.15.100:0/760158468 wait complete. 2024-07-26T13:31:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:08 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/732732' entity='client.admin' 2024-07-26T13:31:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:08 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/3666978576' entity='client.admin' 2024-07-26T13:31:09.407 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:09 smithi100 ceph-mon[32599]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:09.407 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:09 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/760158468' entity='client.admin' 2024-07-26T13:31:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:09 smithi138 ceph-mon[36954]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:09 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/760158468' entity='client.admin' 2024-07-26T13:31:09.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:09 smithi167 ceph-mon[37140]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:09.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:09 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/760158468' entity='client.admin' 2024-07-26T13:31:09.862 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'bf83e2e6-4b52-11ef-bcb7-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi100') 2024-07-26T13:31:09.862 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'bf83e2e6-4b52-11ef-bcb7-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi100') 2024-07-26T13:31:09.863 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'bf83e2e6-4b52-11ef-bcb7-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi100') 2024-07-26T13:31:09.863 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'bf83e2e6-4b52-11ef-bcb7-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi100') 2024-07-26T13:31:09.863 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'bf83e2e6-4b52-11ef-bcb7-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi100') 2024-07-26T13:31:09.863 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'bf83e2e6-4b52-11ef-bcb7-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi100') 2024-07-26T13:31:09.864 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'bf83e2e6-4b52-11ef-bcb7-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi100') 2024-07-26T13:31:09.864 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'bf83e2e6-4b52-11ef-bcb7-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi100') 2024-07-26T13:31:09.864 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'bf83e2e6-4b52-11ef-bcb7-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi100') 2024-07-26T13:31:09.864 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'bf83e2e6-4b52-11ef-bcb7-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'debug_cephsqlite', '20'], check_status=True, hostname='smithi100') 2024-07-26T13:31:09.864 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'bf83e2e6-4b52-11ef-bcb7-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/allow_m_granularity', 'True'], check_status=True, hostname='smithi100') 2024-07-26T13:31:09.865 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'bf83e2e6-4b52-11ef-bcb7-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/dump_on_update', 'True'], check_status=True, hostname='smithi100') 2024-07-26T13:31:09.865 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'bf83e2e6-4b52-11ef-bcb7-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/log_level', 'debug'], check_status=True, hostname='smithi100') 2024-07-26T13:31:09.865 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'bf83e2e6-4b52-11ef-bcb7-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi100') 2024-07-26T13:31:10.272 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'bf83e2e6-4b52-11ef-bcb7-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi100') 2024-07-26T13:31:10.272 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'bf83e2e6-4b52-11ef-bcb7-c7b262605968', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi100') 2024-07-26T13:31:10.273 INFO:tasks.cephadm:Deploying OSDs... 2024-07-26T13:31:10.273 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-07-26T13:31:10.273 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/scratch_devs of=/dev/stdout 2024-07-26T13:31:10.290 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-07-26T13:31:10.291 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_1 2024-07-26T13:31:10.346 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-07-26T13:31:10.346 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-26T13:31:10.346 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-07-26T13:31:10.347 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-26T13:31:10.347 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-07-26T13:31:10.347 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-07-26 13:29:18.640993678 +0000 2024-07-26T13:31:10.347 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-07-26 13:24:06.635271364 +0000 2024-07-26T13:31:10.347 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-07-26 13:24:06.635271364 +0000 2024-07-26T13:31:10.347 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-07-26 13:24:06.635271364 +0000 2024-07-26T13:31:10.347 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-07-26T13:31:10.411 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-07-26T13:31:10.411 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-07-26T13:31:10.411 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000212546 s, 2.4 MB/s 2024-07-26T13:31:10.412 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-07-26T13:31:10.469 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_2 2024-07-26T13:31:10.525 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-07-26T13:31:10.526 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-26T13:31:10.526 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-07-26T13:31:10.526 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-26T13:31:10.526 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-07-26T13:31:10.526 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-07-26 13:29:18.666993019 +0000 2024-07-26T13:31:10.526 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-07-26 13:24:06.966262606 +0000 2024-07-26T13:31:10.526 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-07-26 13:24:06.966262606 +0000 2024-07-26T13:31:10.526 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-07-26 13:24:06.966262606 +0000 2024-07-26T13:31:10.526 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-07-26T13:31:10.588 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-07-26T13:31:10.588 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-07-26T13:31:10.588 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000177444 s, 2.9 MB/s 2024-07-26T13:31:10.590 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-07-26T13:31:10.646 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_3 2024-07-26T13:31:10.700 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-07-26T13:31:10.700 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-26T13:31:10.700 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-07-26T13:31:10.700 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-26T13:31:10.701 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-07-26T13:31:10.701 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-07-26 13:29:18.690992411 +0000 2024-07-26T13:31:10.701 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-07-26 13:24:07.298253821 +0000 2024-07-26T13:31:10.701 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-07-26 13:24:07.298253821 +0000 2024-07-26T13:31:10.701 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-07-26 13:24:07.298253821 +0000 2024-07-26T13:31:10.701 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-07-26T13:31:10.763 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-07-26T13:31:10.763 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-07-26T13:31:10.763 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000247045 s, 2.1 MB/s 2024-07-26T13:31:10.765 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-07-26T13:31:10.820 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_4 2024-07-26T13:31:10.876 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-07-26T13:31:10.876 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-26T13:31:10.876 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-07-26T13:31:10.876 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-26T13:31:10.876 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-07-26T13:31:10.876 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-07-26 13:29:18.715991777 +0000 2024-07-26T13:31:10.876 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-07-26 13:24:07.636244877 +0000 2024-07-26T13:31:10.876 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-07-26 13:24:07.636244877 +0000 2024-07-26T13:31:10.876 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-07-26 13:24:07.636244877 +0000 2024-07-26T13:31:10.877 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-07-26T13:31:10.940 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-07-26T13:31:10.940 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-07-26T13:31:10.940 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000180483 s, 2.8 MB/s 2024-07-26T13:31:10.942 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-07-26T13:31:10.997 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-07-26T13:31:10.998 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/scratch_devs of=/dev/stdout 2024-07-26T13:31:11.016 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-07-26T13:31:11.017 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_1 2024-07-26T13:31:11.075 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-07-26T13:31:11.075 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-26T13:31:11.075 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-07-26T13:31:11.075 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-26T13:31:11.075 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-07-26T13:31:11.075 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-07-26 13:30:26.610230307 +0000 2024-07-26T13:31:11.075 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-07-26 13:24:24.691837814 +0000 2024-07-26T13:31:11.075 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-07-26 13:24:24.691837814 +0000 2024-07-26T13:31:11.076 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-07-26 13:24:24.691837814 +0000 2024-07-26T13:31:11.076 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-07-26T13:31:11.144 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-07-26T13:31:11.144 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-07-26T13:31:11.144 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000307573 s, 1.7 MB/s 2024-07-26T13:31:11.146 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-07-26T13:31:11.205 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_2 2024-07-26T13:31:11.262 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-07-26T13:31:11.262 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-26T13:31:11.262 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-07-26T13:31:11.262 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-26T13:31:11.263 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-07-26T13:31:11.263 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-07-26 13:30:26.611230286 +0000 2024-07-26T13:31:11.263 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-07-26 13:24:25.038830412 +0000 2024-07-26T13:31:11.263 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-07-26 13:24:25.038830412 +0000 2024-07-26T13:31:11.263 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-07-26 13:24:25.038830412 +0000 2024-07-26T13:31:11.263 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-07-26T13:31:11.329 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-07-26T13:31:11.330 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-07-26T13:31:11.330 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000232633 s, 2.2 MB/s 2024-07-26T13:31:11.331 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-07-26T13:31:11.389 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_3 2024-07-26T13:31:11.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:11 smithi138 ceph-mon[36954]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:11.447 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-07-26T13:31:11.448 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-26T13:31:11.448 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-07-26T13:31:11.448 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-26T13:31:11.448 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-07-26T13:31:11.448 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-07-26 13:30:26.611230286 +0000 2024-07-26T13:31:11.448 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-07-26 13:24:25.367823395 +0000 2024-07-26T13:31:11.448 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-07-26 13:24:25.367823395 +0000 2024-07-26T13:31:11.448 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-07-26 13:24:25.367823395 +0000 2024-07-26T13:31:11.448 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-07-26T13:31:11.514 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-07-26T13:31:11.514 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-07-26T13:31:11.515 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.00021764 s, 2.4 MB/s 2024-07-26T13:31:11.516 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-07-26T13:31:11.574 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_4 2024-07-26T13:31:11.632 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-07-26T13:31:11.632 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-26T13:31:11.632 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-07-26T13:31:11.632 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-26T13:31:11.633 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-07-26T13:31:11.633 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-07-26 13:30:26.612230266 +0000 2024-07-26T13:31:11.633 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-07-26 13:24:25.703816228 +0000 2024-07-26T13:31:11.633 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-07-26 13:24:25.703816228 +0000 2024-07-26T13:31:11.633 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-07-26 13:24:25.703816228 +0000 2024-07-26T13:31:11.633 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-07-26T13:31:11.698 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-07-26T13:31:11.698 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-07-26T13:31:11.698 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000217091 s, 2.4 MB/s 2024-07-26T13:31:11.699 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-07-26T13:31:11.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:11 smithi100 ceph-mon[32599]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:11.758 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-07-26T13:31:11.758 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/scratch_devs of=/dev/stdout 2024-07-26T13:31:11.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:11 smithi167 ceph-mon[37140]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:11.777 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-07-26T13:31:11.777 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_1 2024-07-26T13:31:11.835 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-07-26T13:31:11.835 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-26T13:31:11.835 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-07-26T13:31:11.835 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-26T13:31:11.835 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-07-26T13:31:11.836 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-07-26 13:30:24.266696604 +0000 2024-07-26T13:31:11.836 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-07-26 13:24:23.119702814 +0000 2024-07-26T13:31:11.836 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-07-26 13:24:23.119702814 +0000 2024-07-26T13:31:11.836 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-07-26 13:24:23.119702814 +0000 2024-07-26T13:31:11.836 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-07-26T13:31:11.904 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-07-26T13:31:11.904 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-07-26T13:31:11.904 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000256998 s, 2.0 MB/s 2024-07-26T13:31:11.905 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-07-26T13:31:11.963 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_2 2024-07-26T13:31:12.019 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-07-26T13:31:12.019 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-26T13:31:12.019 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-07-26T13:31:12.020 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-26T13:31:12.020 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-07-26T13:31:12.020 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-07-26 13:30:24.266696604 +0000 2024-07-26T13:31:12.020 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-07-26 13:24:23.453696859 +0000 2024-07-26T13:31:12.020 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-07-26 13:24:23.453696859 +0000 2024-07-26T13:31:12.020 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-07-26 13:24:23.453696859 +0000 2024-07-26T13:31:12.020 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-07-26T13:31:12.084 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-07-26T13:31:12.084 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-07-26T13:31:12.084 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000219614 s, 2.3 MB/s 2024-07-26T13:31:12.085 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-07-26T13:31:12.142 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_3 2024-07-26T13:31:12.198 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-07-26T13:31:12.198 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-26T13:31:12.199 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-07-26T13:31:12.199 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-26T13:31:12.199 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-07-26T13:31:12.199 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-07-26 13:30:24.267696587 +0000 2024-07-26T13:31:12.199 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-07-26 13:24:23.792690816 +0000 2024-07-26T13:31:12.199 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-07-26 13:24:23.792690816 +0000 2024-07-26T13:31:12.199 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-07-26 13:24:23.792690816 +0000 2024-07-26T13:31:12.199 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-07-26T13:31:12.262 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-07-26T13:31:12.262 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-07-26T13:31:12.262 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000185502 s, 2.8 MB/s 2024-07-26T13:31:12.264 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-07-26T13:31:12.321 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_4 2024-07-26T13:31:12.377 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-07-26T13:31:12.377 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-26T13:31:12.377 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-07-26T13:31:12.377 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-26T13:31:12.377 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-07-26T13:31:12.377 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-07-26 13:30:24.267696587 +0000 2024-07-26T13:31:12.377 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-07-26 13:24:24.117685022 +0000 2024-07-26T13:31:12.377 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-07-26 13:24:24.117685022 +0000 2024-07-26T13:31:12.378 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-07-26 13:24:24.117685022 +0000 2024-07-26T13:31:12.378 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-07-26T13:31:12.440 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-07-26T13:31:12.440 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-07-26T13:31:12.441 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.00021385 s, 2.4 MB/s 2024-07-26T13:31:12.442 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-07-26T13:31:12.498 INFO:tasks.cephadm:Deploying osd.0 on smithi100 with /dev/vg_nvme/lv_4... 2024-07-26T13:31:12.499 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-07-26T13:31:12.753 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:31:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:13 smithi100 ceph-mon[32599]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:13 smithi138 ceph-mon[36954]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:13 smithi167 ceph-mon[37140]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:14.399 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-07-26T13:31:14.495 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-07-26T13:31:14.509 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-07-26T13:31:14.509 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-07-26T13:31:14.509 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114195 s, 918 MB/s 2024-07-26T13:31:14.509 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-07-26T13:31:15.027 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph orch daemon add osd smithi100:vg_nvme/lv_4 2024-07-26T13:31:15.275 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:31:15.576 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:15 smithi100 ceph-mon[32599]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:15 smithi138 ceph-mon[36954]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:15 smithi167 ceph-mon[37140]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:16.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.681+0000 7fb0564f0640 1 -- 172.21.15.100:0/4144770577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb050109840 msgr2=0x7fb050110070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:16.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.681+0000 7fb0564f0640 1 --2- 172.21.15.100:0/4144770577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb050109840 0x7fb050110070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb048009ca0 tx=0x7fb04802a8e0 comp rx=0 tx=0).stop 2024-07-26T13:31:16.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.681+0000 7fb0564f0640 1 -- 172.21.15.100:0/4144770577 shutdown_connections 2024-07-26T13:31:16.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.681+0000 7fb0564f0640 1 --2- 172.21.15.100:0/4144770577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb050109840 0x7fb050110070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:16.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.681+0000 7fb0564f0640 1 --2- 172.21.15.100:0/4144770577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb050108ef0 0x7fb050109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:16.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.681+0000 7fb0564f0640 1 --2- 172.21.15.100:0/4144770577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb0501035b0 0x7fb0501039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:16.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.681+0000 7fb0564f0640 1 -- 172.21.15.100:0/4144770577 >> 172.21.15.100:0/4144770577 conn(0x7fb0500ff000 msgr2=0x7fb050101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:31:16.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.681+0000 7fb0564f0640 1 -- 172.21.15.100:0/4144770577 shutdown_connections 2024-07-26T13:31:16.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.681+0000 7fb0564f0640 1 -- 172.21.15.100:0/4144770577 wait complete. 2024-07-26T13:31:16.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.682+0000 7fb0564f0640 1 Processor -- start 2024-07-26T13:31:16.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.682+0000 7fb0564f0640 1 -- start start 2024-07-26T13:31:16.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.682+0000 7fb0564f0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0501035b0 0x7fb050199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:16.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.682+0000 7fb0564f0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb050108ef0 0x7fb05019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:16.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.682+0000 7fb0564f0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb050109840 0x7fb05019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:16.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.682+0000 7fb0564f0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb050113540 con 0x7fb050108ef0 2024-07-26T13:31:16.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.682+0000 7fb0564f0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb0501133c0 con 0x7fb0501035b0 2024-07-26T13:31:16.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.682+0000 7fb0564f0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb0501136c0 con 0x7fb050109840 2024-07-26T13:31:16.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.682+0000 7fb047fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb050108ef0 0x7fb05019a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:16.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.682+0000 7fb047fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb050108ef0 0x7fb05019a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39560/0 (socket says 172.21.15.100:39560) 2024-07-26T13:31:16.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.682+0000 7fb047fff640 1 -- 172.21.15.100:0/4277278076 learned_addr learned my addr 172.21.15.100:0/4277278076 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:31:16.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.682+0000 7fb054a8d640 1 --2- 172.21.15.100:0/4277278076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb050109840 0x7fb05019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:16.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.682+0000 7fb04ffff640 1 --2- 172.21.15.100:0/4277278076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0501035b0 0x7fb050199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:16.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.682+0000 7fb047fff640 1 -- 172.21.15.100:0/4277278076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb050109840 msgr2=0x7fb05019f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:16.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.682+0000 7fb047fff640 1 --2- 172.21.15.100:0/4277278076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb050109840 0x7fb05019f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:16.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.682+0000 7fb047fff640 1 -- 172.21.15.100:0/4277278076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0501035b0 msgr2=0x7fb050199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:16.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.682+0000 7fb047fff640 1 --2- 172.21.15.100:0/4277278076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0501035b0 0x7fb050199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:16.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.682+0000 7fb047fff640 1 -- 172.21.15.100:0/4277278076 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb05019fd60 con 0x7fb050108ef0 2024-07-26T13:31:16.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.682+0000 7fb054a8d640 1 --2- 172.21.15.100:0/4277278076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb050109840 0x7fb05019f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:31:16.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.682+0000 7fb04ffff640 1 --2- 172.21.15.100:0/4277278076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0501035b0 0x7fb050199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:31:16.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.683+0000 7fb047fff640 1 --2- 172.21.15.100:0/4277278076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb050108ef0 0x7fb05019a450 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb038006fd0 tx=0x7fb03800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:31:16.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.683+0000 7fb04dffb640 1 -- 172.21.15.100:0/4277278076 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb038011830 con 0x7fb050108ef0 2024-07-26T13:31:16.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.683+0000 7fb04dffb640 1 -- 172.21.15.100:0/4277278076 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb038016070 con 0x7fb050108ef0 2024-07-26T13:31:16.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.683+0000 7fb0564f0640 1 -- 172.21.15.100:0/4277278076 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0501a0050 con 0x7fb050108ef0 2024-07-26T13:31:16.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.683+0000 7fb04dffb640 1 -- 172.21.15.100:0/4277278076 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb038010e00 con 0x7fb050108ef0 2024-07-26T13:31:16.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.683+0000 7fb0564f0640 1 -- 172.21.15.100:0/4277278076 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0501a6870 con 0x7fb050108ef0 2024-07-26T13:31:16.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.684+0000 7fb0564f0640 1 -- 172.21.15.100:0/4277278076 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0501039c0 con 0x7fb050108ef0 2024-07-26T13:31:16.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.685+0000 7fb04dffb640 1 -- 172.21.15.100:0/4277278076 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7fb03801a3c0 con 0x7fb050108ef0 2024-07-26T13:31:16.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.686+0000 7fb04dffb640 1 --2- 172.21.15.100:0/4277278076 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb03c0b3c40 0x7fb03c0b6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:16.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.686+0000 7fb04ffff640 1 --2- 172.21.15.100:0/4277278076 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb03c0b3c40 0x7fb03c0b6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:16.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.686+0000 7fb04dffb640 1 -- 172.21.15.100:0/4277278076 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fb0380e1cc0 con 0x7fb050108ef0 2024-07-26T13:31:16.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.686+0000 7fb04ffff640 1 --2- 172.21.15.100:0/4277278076 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb03c0b3c40 0x7fb03c0b6150 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb04000abe0 tx=0x7fb04000a5c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:31:16.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.689+0000 7fb04dffb640 1 -- 172.21.15.100:0/4277278076 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fb0380aaf10 con 0x7fb050108ef0 2024-07-26T13:31:16.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:16.870+0000 7fb0564f0640 1 -- 172.21.15.100:0/4277278076 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fb050063230 con 0x7fb03c0b3c40 2024-07-26T13:31:17.569 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:17 smithi100 ceph-mon[32599]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:17.569 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-26T13:31:17.569 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-26T13:31:17.569 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:31:17.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:17 smithi167 ceph-mon[37140]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:17.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-26T13:31:17.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-26T13:31:17.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:31:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:17 smithi138 ceph-mon[36954]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-26T13:31:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-26T13:31:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:31:18.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:18 smithi100 ceph-mon[32599]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:31:18.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:18 smithi138 ceph-mon[36954]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:31:18.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:18 smithi167 ceph-mon[37140]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:31:19.617 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:19 smithi100 ceph-mon[32599]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:19.618 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:19 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2610682455' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d714036d-9dda-4cb5-af16-29f83a09dd70"}]: dispatch 2024-07-26T13:31:19.618 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:19 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2610682455' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d714036d-9dda-4cb5-af16-29f83a09dd70"}]': finished 2024-07-26T13:31:19.618 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:19 smithi100 ceph-mon[32599]: osdmap e6: 1 total, 0 up, 1 in 2024-07-26T13:31:19.618 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-26T13:31:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:19 smithi138 ceph-mon[36954]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:19 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/2610682455' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d714036d-9dda-4cb5-af16-29f83a09dd70"}]: dispatch 2024-07-26T13:31:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:19 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/2610682455' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d714036d-9dda-4cb5-af16-29f83a09dd70"}]': finished 2024-07-26T13:31:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:19 smithi138 ceph-mon[36954]: osdmap e6: 1 total, 0 up, 1 in 2024-07-26T13:31:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-26T13:31:19.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:19 smithi167 ceph-mon[37140]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:19.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:19 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/2610682455' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d714036d-9dda-4cb5-af16-29f83a09dd70"}]: dispatch 2024-07-26T13:31:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:19 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/2610682455' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d714036d-9dda-4cb5-af16-29f83a09dd70"}]': finished 2024-07-26T13:31:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:19 smithi167 ceph-mon[37140]: osdmap e6: 1 total, 0 up, 1 in 2024-07-26T13:31:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-26T13:31:20.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:20 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/3931147642' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-26T13:31:20.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:20 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/3931147642' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-26T13:31:20.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:20 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/3931147642' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-26T13:31:21.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:21 smithi138 ceph-mon[36954]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:21.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:21 smithi100 ceph-mon[32599]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:21.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:21 smithi167 ceph-mon[37140]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:23.675 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:23 smithi100 ceph-mon[32599]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:23.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:23 smithi138 ceph-mon[36954]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:23.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:23 smithi167 ceph-mon[37140]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:25 smithi100 ceph-mon[32599]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:25 smithi138 ceph-mon[36954]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:25.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:25 smithi167 ceph-mon[37140]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:26.554 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-07-26T13:31:26.554 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:31:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-07-26T13:31:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:31:26.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-07-26T13:31:26.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:31:27.473 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:27 smithi100 ceph-mon[32599]: Deploying daemon osd.0 on smithi100 2024-07-26T13:31:27.473 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:27 smithi100 ceph-mon[32599]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:27 smithi138 ceph-mon[36954]: Deploying daemon osd.0 on smithi100 2024-07-26T13:31:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:27 smithi138 ceph-mon[36954]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:27 smithi167 ceph-mon[37140]: Deploying daemon osd.0 on smithi100 2024-07-26T13:31:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:27 smithi167 ceph-mon[37140]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:29.627 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:29 smithi100 ceph-mon[32599]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:29 smithi138 ceph-mon[36954]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:29.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:29 smithi167 ceph-mon[37140]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:31.519 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:31 smithi100 ceph-mon[32599]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:31.519 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:31:31.519 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:31.519 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:31 smithi138 ceph-mon[36954]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:31:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:31.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:31 smithi167 ceph-mon[37140]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:31.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:31:31.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:31.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:33 smithi138 ceph-mon[36954]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:33 smithi100 ceph-mon[32599]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:33.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:33 smithi167 ceph-mon[37140]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:33.825 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 0 on host 'smithi100' 2024-07-26T13:31:33.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:33.823+0000 7fb04dffb640 1 -- 172.21.15.100:0/4277278076 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb050063230 con 0x7fb03c0b3c40 2024-07-26T13:31:33.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:33.826+0000 7fb0564f0640 1 -- 172.21.15.100:0/4277278076 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb03c0b3c40 msgr2=0x7fb03c0b6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:33.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:33.826+0000 7fb0564f0640 1 --2- 172.21.15.100:0/4277278076 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb03c0b3c40 0x7fb03c0b6150 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb04000abe0 tx=0x7fb04000a5c0 comp rx=0 tx=0).stop 2024-07-26T13:31:33.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:33.826+0000 7fb0564f0640 1 -- 172.21.15.100:0/4277278076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb050108ef0 msgr2=0x7fb05019a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:33.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:33.826+0000 7fb0564f0640 1 --2- 172.21.15.100:0/4277278076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb050108ef0 0x7fb05019a450 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb038006fd0 tx=0x7fb03800b8e0 comp rx=0 tx=0).stop 2024-07-26T13:31:33.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:33.826+0000 7fb0564f0640 1 -- 172.21.15.100:0/4277278076 shutdown_connections 2024-07-26T13:31:33.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:33.826+0000 7fb0564f0640 1 --2- 172.21.15.100:0/4277278076 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb03c0b3c40 0x7fb03c0b6150 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:33.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:33.826+0000 7fb0564f0640 1 --2- 172.21.15.100:0/4277278076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb050109840 0x7fb05019f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:33.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:33.826+0000 7fb0564f0640 1 --2- 172.21.15.100:0/4277278076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb050108ef0 0x7fb05019a450 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:33.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:33.826+0000 7fb0564f0640 1 --2- 172.21.15.100:0/4277278076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0501035b0 0x7fb050199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:33.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:33.826+0000 7fb0564f0640 1 -- 172.21.15.100:0/4277278076 >> 172.21.15.100:0/4277278076 conn(0x7fb0500ff000 msgr2=0x7fb050101420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:31:33.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:33.826+0000 7fb0564f0640 1 -- 172.21.15.100:0/4277278076 shutdown_connections 2024-07-26T13:31:33.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:31:33.826+0000 7fb0564f0640 1 -- 172.21.15.100:0/4277278076 wait complete. 2024-07-26T13:31:34.405 DEBUG:teuthology.orchestra.run.smithi100:osd.0> sudo journalctl -f -n 0 -u ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@osd.0.service 2024-07-26T13:31:34.408 INFO:tasks.cephadm:Deploying osd.1 on smithi138 with /dev/vg_nvme/lv_4... 2024-07-26T13:31:34.408 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-07-26T13:31:34.663 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.b/config 2024-07-26T13:31:35.089 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:35.089 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:35.089 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:35.089 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:35.089 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:34 smithi100 ceph-mon[32599]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:35.089 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:34 smithi100 ceph-mon[32599]: from='osd.0 [v2:172.21.15.100:6802/1480401850,v1:172.21.15.100:6803/1480401850]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-07-26T13:31:35.090 INFO:journalctl@ceph.osd.0.smithi100.stdout:Jul 26 13:31:34 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-osd-0[51973]: 2024-07-26T13:31:34.773+0000 7fe858e0b640 -1 osd.0 0 log_to_monitors true 2024-07-26T13:31:35.122 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:35.122 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:35.122 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:35.122 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:35.123 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:34 smithi138 ceph-mon[36954]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:35.123 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:34 smithi138 ceph-mon[36954]: from='osd.0 [v2:172.21.15.100:6802/1480401850,v1:172.21.15.100:6803/1480401850]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-07-26T13:31:35.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:35.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:34 smithi167 ceph-mon[37140]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:34 smithi167 ceph-mon[37140]: from='osd.0 [v2:172.21.15.100:6802/1480401850,v1:172.21.15.100:6803/1480401850]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-07-26T13:31:35.994 INFO:journalctl@ceph.osd.0.smithi100.stdout:Jul 26 13:31:35 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-osd-0[51973]: 2024-07-26T13:31:35.868+0000 7fe854d8d640 -1 osd.0 0 waiting for initial osdmap 2024-07-26T13:31:35.994 INFO:journalctl@ceph.osd.0.smithi100.stdout:Jul 26 13:31:35 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-osd-0[51973]: 2024-07-26T13:31:35.871+0000 7fe850b8b640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-26T13:31:35.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:35 smithi100 ceph-mon[32599]: from='osd.0 [v2:172.21.15.100:6802/1480401850,v1:172.21.15.100:6803/1480401850]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-07-26T13:31:35.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:35 smithi100 ceph-mon[32599]: osdmap e7: 1 total, 0 up, 1 in 2024-07-26T13:31:35.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:35 smithi100 ceph-mon[32599]: from='osd.0 [v2:172.21.15.100:6802/1480401850,v1:172.21.15.100:6803/1480401850]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-07-26T13:31:35.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-26T13:31:36.002 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:35 smithi138 ceph-mon[36954]: from='osd.0 [v2:172.21.15.100:6802/1480401850,v1:172.21.15.100:6803/1480401850]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-07-26T13:31:36.002 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:35 smithi138 ceph-mon[36954]: osdmap e7: 1 total, 0 up, 1 in 2024-07-26T13:31:36.002 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:35 smithi138 ceph-mon[36954]: from='osd.0 [v2:172.21.15.100:6802/1480401850,v1:172.21.15.100:6803/1480401850]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-07-26T13:31:36.002 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-26T13:31:36.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:35 smithi167 ceph-mon[37140]: from='osd.0 [v2:172.21.15.100:6802/1480401850,v1:172.21.15.100:6803/1480401850]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-07-26T13:31:36.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:35 smithi167 ceph-mon[37140]: osdmap e7: 1 total, 0 up, 1 in 2024-07-26T13:31:36.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:35 smithi167 ceph-mon[37140]: from='osd.0 [v2:172.21.15.100:6802/1480401850,v1:172.21.15.100:6803/1480401850]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-07-26T13:31:36.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-26T13:31:36.369 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-07-26T13:31:36.473 INFO:teuthology.orchestra.run.smithi138.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-07-26T13:31:36.486 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10+0 records in 2024-07-26T13:31:36.486 INFO:teuthology.orchestra.run.smithi138.stderr:10+0 records out 2024-07-26T13:31:36.486 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111455 s, 941 MB/s 2024-07-26T13:31:36.486 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping successful for: 2024-07-26T13:31:37.082 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:36 smithi138 ceph-mon[36954]: from='osd.0 [v2:172.21.15.100:6802/1480401850,v1:172.21.15.100:6803/1480401850]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-07-26T13:31:37.082 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:36 smithi138 ceph-mon[36954]: osdmap e8: 1 total, 0 up, 1 in 2024-07-26T13:31:37.083 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-26T13:31:37.083 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-26T13:31:37.083 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:36 smithi138 ceph-mon[36954]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:37.109 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_4 2024-07-26T13:31:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:36 smithi100 ceph-mon[32599]: from='osd.0 [v2:172.21.15.100:6802/1480401850,v1:172.21.15.100:6803/1480401850]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-07-26T13:31:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:36 smithi100 ceph-mon[32599]: osdmap e8: 1 total, 0 up, 1 in 2024-07-26T13:31:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-26T13:31:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-26T13:31:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:36 smithi100 ceph-mon[32599]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:37.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:36 smithi167 ceph-mon[37140]: from='osd.0 [v2:172.21.15.100:6802/1480401850,v1:172.21.15.100:6803/1480401850]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-07-26T13:31:37.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:36 smithi167 ceph-mon[37140]: osdmap e8: 1 total, 0 up, 1 in 2024-07-26T13:31:37.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-26T13:31:37.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-26T13:31:37.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:36 smithi167 ceph-mon[37140]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-26T13:31:37.366 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.b/config 2024-07-26T13:31:38.163 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:37 smithi138 ceph-mon[36954]: purged_snaps scrub starts 2024-07-26T13:31:38.163 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:37 smithi138 ceph-mon[36954]: purged_snaps scrub ok 2024-07-26T13:31:38.163 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:37 smithi138 ceph-mon[36954]: osd.0 [v2:172.21.15.100:6802/1480401850,v1:172.21.15.100:6803/1480401850] boot 2024-07-26T13:31:38.163 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:37 smithi138 ceph-mon[36954]: osdmap e9: 1 total, 1 up, 1 in 2024-07-26T13:31:38.163 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-26T13:31:38.163 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:38.164 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:38.164 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:31:38.164 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:38.164 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:31:38.164 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:31:38.164 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:38.164 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:31:38.164 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:31:38.164 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:31:38.164 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:38.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:37 smithi100 ceph-mon[32599]: purged_snaps scrub starts 2024-07-26T13:31:38.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:37 smithi100 ceph-mon[32599]: purged_snaps scrub ok 2024-07-26T13:31:38.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:37 smithi100 ceph-mon[32599]: osd.0 [v2:172.21.15.100:6802/1480401850,v1:172.21.15.100:6803/1480401850] boot 2024-07-26T13:31:38.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:37 smithi100 ceph-mon[32599]: osdmap e9: 1 total, 1 up, 1 in 2024-07-26T13:31:38.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-26T13:31:38.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:38.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:38.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:31:38.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:38.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:31:38.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:31:38.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:38.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:31:38.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:31:38.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:31:38.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:38.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:37 smithi167 ceph-mon[37140]: purged_snaps scrub starts 2024-07-26T13:31:38.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:37 smithi167 ceph-mon[37140]: purged_snaps scrub ok 2024-07-26T13:31:38.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:37 smithi167 ceph-mon[37140]: osd.0 [v2:172.21.15.100:6802/1480401850,v1:172.21.15.100:6803/1480401850] boot 2024-07-26T13:31:38.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:37 smithi167 ceph-mon[37140]: osdmap e9: 1 total, 1 up, 1 in 2024-07-26T13:31:38.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-26T13:31:38.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:38.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:38.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:31:38.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:38.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:31:38.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:31:38.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:38.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:31:38.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:31:38.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:31:38.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:38.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.933+0000 7ff61d4d7640 1 -- 172.21.15.138:0/3438006492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff618102ca0 msgr2=0x7ff6181030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:38.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.933+0000 7ff61d4d7640 1 --2- 172.21.15.138:0/3438006492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff618102ca0 0x7ff6181030b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff608009900 tx=0x7ff60802a410 comp rx=0 tx=0).stop 2024-07-26T13:31:38.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.933+0000 7ff61d4d7640 1 -- 172.21.15.138:0/3438006492 shutdown_connections 2024-07-26T13:31:38.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.933+0000 7ff61d4d7640 1 --2- 172.21.15.138:0/3438006492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6181055c0 0x7ff61810f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:38.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.933+0000 7ff61d4d7640 1 --2- 172.21.15.138:0/3438006492 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff618104c70 0x7ff618105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:38.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.933+0000 7ff61d4d7640 1 --2- 172.21.15.138:0/3438006492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff618102ca0 0x7ff6181030b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:38.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.933+0000 7ff61d4d7640 1 -- 172.21.15.138:0/3438006492 >> 172.21.15.138:0/3438006492 conn(0x7ff6180798f0 msgr2=0x7ff618079d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:31:38.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.933+0000 7ff61d4d7640 1 -- 172.21.15.138:0/3438006492 shutdown_connections 2024-07-26T13:31:38.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.933+0000 7ff61d4d7640 1 -- 172.21.15.138:0/3438006492 wait complete. 2024-07-26T13:31:38.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.933+0000 7ff61d4d7640 1 Processor -- start 2024-07-26T13:31:38.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.933+0000 7ff61d4d7640 1 -- start start 2024-07-26T13:31:38.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.934+0000 7ff61d4d7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff618102ca0 0x7ff618198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:38.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.934+0000 7ff61d4d7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff618104c70 0x7ff618199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:38.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.934+0000 7ff61d4d7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6181055c0 0x7ff6181995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:38.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.934+0000 7ff61d4d7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff6181124d0 con 0x7ff618102ca0 2024-07-26T13:31:38.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.934+0000 7ff61d4d7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff618112350 con 0x7ff618104c70 2024-07-26T13:31:38.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.934+0000 7ff61d4d7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff618112650 con 0x7ff6181055c0 2024-07-26T13:31:38.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.934+0000 7ff60ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff618104c70 0x7ff618199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:38.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.934+0000 7ff60ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff618104c70 0x7ff618199090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50414/0 (socket says 172.21.15.138:50414) 2024-07-26T13:31:38.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.934+0000 7ff60ffff640 1 -- 172.21.15.138:0/2209786571 learned_addr learned my addr 172.21.15.138:0/2209786571 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-07-26T13:31:38.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.934+0000 7ff616ffd640 1 --2- 172.21.15.138:0/2209786571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff618102ca0 0x7ff618198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:38.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.934+0000 7ff6177fe640 1 --2- 172.21.15.138:0/2209786571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6181055c0 0x7ff6181995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:38.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.934+0000 7ff60ffff640 1 -- 172.21.15.138:0/2209786571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6181055c0 msgr2=0x7ff6181995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:38.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.934+0000 7ff60ffff640 1 --2- 172.21.15.138:0/2209786571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6181055c0 0x7ff6181995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:38.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.934+0000 7ff60ffff640 1 -- 172.21.15.138:0/2209786571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff618102ca0 msgr2=0x7ff618198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:38.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.934+0000 7ff60ffff640 1 --2- 172.21.15.138:0/2209786571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff618102ca0 0x7ff618198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:38.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.934+0000 7ff60ffff640 1 -- 172.21.15.138:0/2209786571 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6181a8200 con 0x7ff618104c70 2024-07-26T13:31:38.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.934+0000 7ff6177fe640 1 --2- 172.21.15.138:0/2209786571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6181055c0 0x7ff6181995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:31:38.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.934+0000 7ff616ffd640 1 --2- 172.21.15.138:0/2209786571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff618102ca0 0x7ff618198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:31:38.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.934+0000 7ff60ffff640 1 --2- 172.21.15.138:0/2209786571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff618104c70 0x7ff618199090 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff60000c970 tx=0x7ff60000ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:31:38.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.934+0000 7ff614ff9640 1 -- 172.21.15.138:0/2209786571 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff600007bb0 con 0x7ff618104c70 2024-07-26T13:31:38.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.934+0000 7ff614ff9640 1 -- 172.21.15.138:0/2209786571 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff600007d50 con 0x7ff618104c70 2024-07-26T13:31:38.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.934+0000 7ff61d4d7640 1 -- 172.21.15.138:0/2209786571 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6181a84f0 con 0x7ff618104c70 2024-07-26T13:31:38.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.934+0000 7ff614ff9640 1 -- 172.21.15.138:0/2209786571 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60000f450 con 0x7ff618104c70 2024-07-26T13:31:38.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.935+0000 7ff61d4d7640 1 -- 172.21.15.138:0/2209786571 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6181a8a30 con 0x7ff618104c70 2024-07-26T13:31:38.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.936+0000 7ff61d4d7640 1 -- 172.21.15.138:0/2209786571 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff618101d10 con 0x7ff618104c70 2024-07-26T13:31:38.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.937+0000 7ff614ff9640 1 -- 172.21.15.138:0/2209786571 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7ff600004f20 con 0x7ff618104c70 2024-07-26T13:31:38.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.938+0000 7ff614ff9640 1 --2- 172.21.15.138:0/2209786571 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff6040b3b90 0x7ff6040b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:31:38.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.938+0000 7ff614ff9640 1 -- 172.21.15.138:0/2209786571 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1982+0+0 (secure 0 0 0) 0x7ff6000e23a0 con 0x7ff618104c70 2024-07-26T13:31:38.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.938+0000 7ff616ffd640 1 --2- 172.21.15.138:0/2209786571 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff6040b3b90 0x7ff6040b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:31:38.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.939+0000 7ff616ffd640 1 --2- 172.21.15.138:0/2209786571 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff6040b3b90 0x7ff6040b60a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff608000c00 tx=0x7ff608031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:31:38.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:38.942+0000 7ff614ff9640 1 -- 172.21.15.138:0/2209786571 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7ff600014030 con 0x7ff618104c70 2024-07-26T13:31:39.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:39.124+0000 7ff61d4d7640 1 -- 172.21.15.138:0/2209786571 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ff618063230 con 0x7ff6040b3b90 2024-07-26T13:31:39.126 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:38 smithi138 ceph-mon[36954]: Detected new or changed devices on smithi100 2024-07-26T13:31:39.126 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:38 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi100 to 17087M 2024-07-26T13:31:39.126 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:38 smithi138 ceph-mon[36954]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-26T13:31:39.126 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:38 smithi138 ceph-mon[36954]: osdmap e10: 1 total, 1 up, 1 in 2024-07-26T13:31:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:38 smithi100 ceph-mon[32599]: Detected new or changed devices on smithi100 2024-07-26T13:31:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:38 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi100 to 17087M 2024-07-26T13:31:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:38 smithi100 ceph-mon[32599]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-26T13:31:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:38 smithi100 ceph-mon[32599]: osdmap e10: 1 total, 1 up, 1 in 2024-07-26T13:31:39.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:38 smithi167 ceph-mon[37140]: Detected new or changed devices on smithi100 2024-07-26T13:31:39.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:38 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi100 to 17087M 2024-07-26T13:31:39.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:38 smithi167 ceph-mon[37140]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-26T13:31:39.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:38 smithi167 ceph-mon[37140]: osdmap e10: 1 total, 1 up, 1 in 2024-07-26T13:31:40.189 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:39 smithi138 ceph-mon[36954]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:31:40.189 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:39 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-26T13:31:40.189 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:39 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-26T13:31:40.189 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:39 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:31:40.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:39 smithi100 ceph-mon[32599]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:31:40.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:39 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-26T13:31:40.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:39 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-26T13:31:40.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:39 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:31:40.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:39 smithi167 ceph-mon[37140]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:31:40.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:39 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-26T13:31:40.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:39 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-26T13:31:40.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:39 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:31:41.078 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:40 smithi138 ceph-mon[36954]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-26T13:31:41.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:40 smithi100 ceph-mon[32599]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-26T13:31:41.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:40 smithi167 ceph-mon[37140]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-26T13:31:41.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:41 smithi138 ceph-mon[36954]: from='client.? 172.21.15.138:0/3660917352' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3dd0eaa1-618a-495a-9819-e01c0f869ae8"}]: dispatch 2024-07-26T13:31:41.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:41 smithi138 ceph-mon[36954]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3dd0eaa1-618a-495a-9819-e01c0f869ae8"}]: dispatch 2024-07-26T13:31:41.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:41 smithi138 ceph-mon[36954]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3dd0eaa1-618a-495a-9819-e01c0f869ae8"}]': finished 2024-07-26T13:31:41.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:41 smithi138 ceph-mon[36954]: osdmap e11: 2 total, 1 up, 2 in 2024-07-26T13:31:41.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-26T13:31:42.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:41 smithi100 ceph-mon[32599]: from='client.? 172.21.15.138:0/3660917352' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3dd0eaa1-618a-495a-9819-e01c0f869ae8"}]: dispatch 2024-07-26T13:31:42.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:41 smithi100 ceph-mon[32599]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3dd0eaa1-618a-495a-9819-e01c0f869ae8"}]: dispatch 2024-07-26T13:31:42.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:41 smithi100 ceph-mon[32599]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3dd0eaa1-618a-495a-9819-e01c0f869ae8"}]': finished 2024-07-26T13:31:42.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:41 smithi100 ceph-mon[32599]: osdmap e11: 2 total, 1 up, 2 in 2024-07-26T13:31:42.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-26T13:31:42.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:41 smithi167 ceph-mon[37140]: from='client.? 172.21.15.138:0/3660917352' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3dd0eaa1-618a-495a-9819-e01c0f869ae8"}]: dispatch 2024-07-26T13:31:42.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:41 smithi167 ceph-mon[37140]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3dd0eaa1-618a-495a-9819-e01c0f869ae8"}]: dispatch 2024-07-26T13:31:42.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:41 smithi167 ceph-mon[37140]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3dd0eaa1-618a-495a-9819-e01c0f869ae8"}]': finished 2024-07-26T13:31:42.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:41 smithi167 ceph-mon[37140]: osdmap e11: 2 total, 1 up, 2 in 2024-07-26T13:31:42.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-26T13:31:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:42 smithi138 ceph-mon[36954]: from='client.? 172.21.15.138:0/3488274851' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-26T13:31:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:42 smithi138 ceph-mon[36954]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-26T13:31:43.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:42 smithi100 ceph-mon[32599]: from='client.? 172.21.15.138:0/3488274851' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-26T13:31:43.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:42 smithi100 ceph-mon[32599]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-26T13:31:43.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:42 smithi167 ceph-mon[37140]: from='client.? 172.21.15.138:0/3488274851' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-26T13:31:43.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:42 smithi167 ceph-mon[37140]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-26T13:31:45.605 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:45 smithi138 ceph-mon[36954]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-26T13:31:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:45 smithi100 ceph-mon[32599]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-26T13:31:45.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:45 smithi167 ceph-mon[37140]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-26T13:31:47.561 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:47 smithi138 ceph-mon[36954]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-26T13:31:47.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:47 smithi100 ceph-mon[32599]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-26T13:31:47.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:47 smithi167 ceph-mon[37140]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-26T13:31:48.420 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:48 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-07-26T13:31:48.420 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:48 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:31:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:48 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-07-26T13:31:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:48 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:31:48.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:48 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-07-26T13:31:48.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:48 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:31:49.596 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:49 smithi138 ceph-mon[36954]: Deploying daemon osd.1 on smithi138 2024-07-26T13:31:49.596 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:49 smithi138 ceph-mon[36954]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-26T13:31:49.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:49 smithi100 ceph-mon[32599]: Deploying daemon osd.1 on smithi138 2024-07-26T13:31:49.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:49 smithi100 ceph-mon[32599]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-26T13:31:49.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:49 smithi167 ceph-mon[37140]: Deploying daemon osd.1 on smithi138 2024-07-26T13:31:49.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:49 smithi167 ceph-mon[37140]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-26T13:31:51.599 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:51 smithi138 ceph-mon[36954]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-26T13:31:51.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:51 smithi100 ceph-mon[32599]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-26T13:31:51.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:51 smithi167 ceph-mon[37140]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-26T13:31:53.683 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:53 smithi138 ceph-mon[36954]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-26T13:31:53.684 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:53 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:31:53.684 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:53 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:53.684 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:53 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:53 smithi100 ceph-mon[32599]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-26T13:31:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:53 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:31:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:53 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:53 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:53.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:53 smithi167 ceph-mon[37140]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-26T13:31:53.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:53 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:31:53.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:53 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:53.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:53 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:55.426 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:55 smithi138 ceph-mon[36954]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-26T13:31:55.426 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:55.426 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:55.426 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:31:55.426 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:31:55.426 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:55 smithi100 ceph-mon[32599]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-26T13:31:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:31:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:31:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:55.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:55 smithi167 ceph-mon[37140]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-26T13:31:55.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:55.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:55.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:31:55.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:31:55.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:56.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:56.463+0000 7ff614ff9640 1 -- 172.21.15.138:0/2209786571 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff618063230 con 0x7ff6040b3b90 2024-07-26T13:31:56.465 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 1 on host 'smithi138' 2024-07-26T13:31:56.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:56.466+0000 7ff61d4d7640 1 -- 172.21.15.138:0/2209786571 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff6040b3b90 msgr2=0x7ff6040b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:56.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:56.466+0000 7ff61d4d7640 1 --2- 172.21.15.138:0/2209786571 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff6040b3b90 0x7ff6040b60a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff608000c00 tx=0x7ff608031400 comp rx=0 tx=0).stop 2024-07-26T13:31:56.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:56.466+0000 7ff61d4d7640 1 -- 172.21.15.138:0/2209786571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff618104c70 msgr2=0x7ff618199090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:31:56.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:56.466+0000 7ff61d4d7640 1 --2- 172.21.15.138:0/2209786571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff618104c70 0x7ff618199090 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff60000c970 tx=0x7ff60000ce30 comp rx=0 tx=0).stop 2024-07-26T13:31:56.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:56.466+0000 7ff61d4d7640 1 -- 172.21.15.138:0/2209786571 shutdown_connections 2024-07-26T13:31:56.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:56.466+0000 7ff61d4d7640 1 --2- 172.21.15.138:0/2209786571 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff6040b3b90 0x7ff6040b60a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:56.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:56.466+0000 7ff61d4d7640 1 --2- 172.21.15.138:0/2209786571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6181055c0 0x7ff6181995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:56.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:56.466+0000 7ff61d4d7640 1 --2- 172.21.15.138:0/2209786571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff618104c70 0x7ff618199090 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:56.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:56.466+0000 7ff61d4d7640 1 --2- 172.21.15.138:0/2209786571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff618102ca0 0x7ff618198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:31:56.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:56.466+0000 7ff61d4d7640 1 -- 172.21.15.138:0/2209786571 >> 172.21.15.138:0/2209786571 conn(0x7ff6180798f0 msgr2=0x7ff618106db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:31:56.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:56.466+0000 7ff61d4d7640 1 -- 172.21.15.138:0/2209786571 shutdown_connections 2024-07-26T13:31:56.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:31:56.466+0000 7ff61d4d7640 1 -- 172.21.15.138:0/2209786571 wait complete. 2024-07-26T13:31:57.127 DEBUG:teuthology.orchestra.run.smithi138:osd.1> sudo journalctl -f -n 0 -u ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@osd.1.service 2024-07-26T13:31:57.129 INFO:tasks.cephadm:Deploying osd.2 on smithi167 with /dev/vg_nvme/lv_4... 2024-07-26T13:31:57.129 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-07-26T13:31:57.387 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.c/config 2024-07-26T13:31:57.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:57 smithi167 ceph-mon[37140]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-26T13:31:57.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:31:57.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:57.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:57.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:57 smithi167 ceph-mon[37140]: from='osd.1 [v2:172.21.15.138:6800/3090877671,v1:172.21.15.138:6801/3090877671]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-26T13:31:57.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:57 smithi167 ceph-mon[37140]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-26T13:31:57.678 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:57 smithi138 ceph-mon[36954]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-26T13:31:57.678 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:31:57.678 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:57.678 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:57.678 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:57 smithi138 ceph-mon[36954]: from='osd.1 [v2:172.21.15.138:6800/3090877671,v1:172.21.15.138:6801/3090877671]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-26T13:31:57.678 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:57 smithi138 ceph-mon[36954]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-26T13:31:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:57 smithi100 ceph-mon[32599]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-26T13:31:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:31:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:31:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:57 smithi100 ceph-mon[32599]: from='osd.1 [v2:172.21.15.138:6800/3090877671,v1:172.21.15.138:6801/3090877671]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-26T13:31:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:57 smithi100 ceph-mon[32599]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-26T13:31:58.618 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:58 smithi138 ceph-mon[36954]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-07-26T13:31:58.619 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:58 smithi138 ceph-mon[36954]: osdmap e12: 2 total, 1 up, 2 in 2024-07-26T13:31:58.619 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-26T13:31:58.619 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:58 smithi138 ceph-mon[36954]: from='osd.1 [v2:172.21.15.138:6800/3090877671,v1:172.21.15.138:6801/3090877671]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-07-26T13:31:58.619 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:58 smithi138 ceph-mon[36954]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-07-26T13:31:58.619 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:58 smithi138 ceph-mon[36954]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-26T13:31:58.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:58 smithi167 ceph-mon[37140]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-07-26T13:31:58.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:58 smithi167 ceph-mon[37140]: osdmap e12: 2 total, 1 up, 2 in 2024-07-26T13:31:58.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-26T13:31:58.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:58 smithi167 ceph-mon[37140]: from='osd.1 [v2:172.21.15.138:6800/3090877671,v1:172.21.15.138:6801/3090877671]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-07-26T13:31:58.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:58 smithi167 ceph-mon[37140]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-07-26T13:31:58.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:58 smithi167 ceph-mon[37140]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-26T13:31:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:58 smithi100 ceph-mon[32599]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-07-26T13:31:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:58 smithi100 ceph-mon[32599]: osdmap e12: 2 total, 1 up, 2 in 2024-07-26T13:31:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-26T13:31:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:58 smithi100 ceph-mon[32599]: from='osd.1 [v2:172.21.15.138:6800/3090877671,v1:172.21.15.138:6801/3090877671]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-07-26T13:31:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:58 smithi100 ceph-mon[32599]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-07-26T13:31:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:58 smithi100 ceph-mon[32599]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-26T13:31:59.085 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-07-26T13:31:59.166 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-07-26T13:31:59.181 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2024-07-26T13:31:59.181 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2024-07-26T13:31:59.181 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130941 s, 801 MB/s 2024-07-26T13:31:59.181 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2024-07-26T13:31:59.368 INFO:journalctl@ceph.osd.1.smithi138.stdout:Jul 26 13:31:58 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-osd-1[42842]: 2024-07-26T13:31:58.945+0000 7f138a88d640 -1 osd.1 0 waiting for initial osdmap 2024-07-26T13:31:59.368 INFO:journalctl@ceph.osd.1.smithi138.stdout:Jul 26 13:31:58 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-osd-1[42842]: 2024-07-26T13:31:58.949+0000 7f1382683640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-26T13:31:59.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:59 smithi138 ceph-mon[36954]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-07-26T13:31:59.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:59 smithi138 ceph-mon[36954]: osdmap e13: 2 total, 1 up, 2 in 2024-07-26T13:31:59.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-26T13:31:59.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-26T13:31:59.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:31:59 smithi138 ceph-mon[36954]: from='osd.1 ' entity='osd.1' 2024-07-26T13:31:59.743 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:59 smithi167 ceph-mon[37140]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-07-26T13:31:59.743 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:59 smithi167 ceph-mon[37140]: osdmap e13: 2 total, 1 up, 2 in 2024-07-26T13:31:59.743 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-26T13:31:59.743 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-26T13:31:59.743 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:31:59 smithi167 ceph-mon[37140]: from='osd.1 ' entity='osd.1' 2024-07-26T13:31:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:59 smithi100 ceph-mon[32599]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-07-26T13:31:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:59 smithi100 ceph-mon[32599]: osdmap e13: 2 total, 1 up, 2 in 2024-07-26T13:31:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-26T13:31:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-26T13:31:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:31:59 smithi100 ceph-mon[32599]: from='osd.1 ' entity='osd.1' 2024-07-26T13:31:59.772 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_4 2024-07-26T13:32:00.026 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.c/config 2024-07-26T13:32:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:00 smithi100 ceph-mon[32599]: purged_snaps scrub starts 2024-07-26T13:32:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:00 smithi100 ceph-mon[32599]: purged_snaps scrub ok 2024-07-26T13:32:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-26T13:32:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:00 smithi100 ceph-mon[32599]: osd.1 [v2:172.21.15.138:6800/3090877671,v1:172.21.15.138:6801/3090877671] boot 2024-07-26T13:32:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:00 smithi100 ceph-mon[32599]: osdmap e14: 2 total, 2 up, 2 in 2024-07-26T13:32:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-26T13:32:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:00 smithi100 ceph-mon[32599]: Detected new or changed devices on smithi138 2024-07-26T13:32:00.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:00.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:00.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:32:00.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:00 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi138 to 17088M 2024-07-26T13:32:00.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:00.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:32:00.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:32:00.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:00.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:00 smithi100 ceph-mon[32599]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-26T13:32:00.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:00 smithi167 ceph-mon[37140]: purged_snaps scrub starts 2024-07-26T13:32:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:00 smithi167 ceph-mon[37140]: purged_snaps scrub ok 2024-07-26T13:32:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-26T13:32:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:00 smithi167 ceph-mon[37140]: osd.1 [v2:172.21.15.138:6800/3090877671,v1:172.21.15.138:6801/3090877671] boot 2024-07-26T13:32:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:00 smithi167 ceph-mon[37140]: osdmap e14: 2 total, 2 up, 2 in 2024-07-26T13:32:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-26T13:32:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:00 smithi167 ceph-mon[37140]: Detected new or changed devices on smithi138 2024-07-26T13:32:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:32:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:00 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi138 to 17088M 2024-07-26T13:32:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:32:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:32:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:00 smithi167 ceph-mon[37140]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-26T13:32:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:00 smithi138 ceph-mon[36954]: purged_snaps scrub starts 2024-07-26T13:32:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:00 smithi138 ceph-mon[36954]: purged_snaps scrub ok 2024-07-26T13:32:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-26T13:32:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:00 smithi138 ceph-mon[36954]: osd.1 [v2:172.21.15.138:6800/3090877671,v1:172.21.15.138:6801/3090877671] boot 2024-07-26T13:32:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:00 smithi138 ceph-mon[36954]: osdmap e14: 2 total, 2 up, 2 in 2024-07-26T13:32:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-26T13:32:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:00 smithi138 ceph-mon[36954]: Detected new or changed devices on smithi138 2024-07-26T13:32:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:32:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:00 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi138 to 17088M 2024-07-26T13:32:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:32:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:32:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:00 smithi138 ceph-mon[36954]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-07-26T13:32:01.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.585+0000 7f3a9907b640 1 -- 172.21.15.167:0/627570912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a94108750 msgr2=0x7f3a9410ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:32:01.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.585+0000 7f3a9907b640 1 --2- 172.21.15.167:0/627570912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a94108750 0x7f3a9410ef80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c009ca0 tx=0x7f3a8c02a8c0 comp rx=0 tx=0).stop 2024-07-26T13:32:01.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.585+0000 7f3a9907b640 1 -- 172.21.15.167:0/627570912 shutdown_connections 2024-07-26T13:32:01.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.585+0000 7f3a9907b640 1 --2- 172.21.15.167:0/627570912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a94108750 0x7f3a9410ef80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:32:01.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.585+0000 7f3a9907b640 1 --2- 172.21.15.167:0/627570912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a94107e00 0x7f3a94108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:32:01.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.585+0000 7f3a9907b640 1 --2- 172.21.15.167:0/627570912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a941024c0 0x7f3a941028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:32:01.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.585+0000 7f3a9907b640 1 -- 172.21.15.167:0/627570912 >> 172.21.15.167:0/627570912 conn(0x7f3a94079180 msgr2=0x7f3a94079590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:32:01.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.585+0000 7f3a9907b640 1 -- 172.21.15.167:0/627570912 shutdown_connections 2024-07-26T13:32:01.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.585+0000 7f3a9907b640 1 -- 172.21.15.167:0/627570912 wait complete. 2024-07-26T13:32:01.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.586+0000 7f3a9907b640 1 Processor -- start 2024-07-26T13:32:01.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.586+0000 7f3a9907b640 1 -- start start 2024-07-26T13:32:01.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.586+0000 7f3a9907b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a941024c0 0x7f3a9419acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:32:01.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.586+0000 7f3a9907b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a94107e00 0x7f3a9419b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:32:01.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.586+0000 7f3a9907b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a94108750 0x7f3a9419b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:32:01.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.586+0000 7f3a9907b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3a941123d0 con 0x7f3a94107e00 2024-07-26T13:32:01.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.586+0000 7f3a9907b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3a94112250 con 0x7f3a94108750 2024-07-26T13:32:01.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.586+0000 7f3a9907b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3a94112550 con 0x7f3a941024c0 2024-07-26T13:32:01.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.586+0000 7f3a92d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a941024c0 0x7f3a9419acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:32:01.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.586+0000 7f3a92d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a941024c0 0x7f3a9419acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58114/0 (socket says 172.21.15.167:58114) 2024-07-26T13:32:01.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.586+0000 7f3a92d9d640 1 -- 172.21.15.167:0/3400575433 learned_addr learned my addr 172.21.15.167:0/3400575433 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-07-26T13:32:01.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.586+0000 7f3a8bfff640 1 --2- 172.21.15.167:0/3400575433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a94107e00 0x7f3a9419b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:32:01.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.586+0000 7f3a92d9d640 1 -- 172.21.15.167:0/3400575433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a94108750 msgr2=0x7f3a9419b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:32:01.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.586+0000 7f3a9359e640 1 --2- 172.21.15.167:0/3400575433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a94108750 0x7f3a9419b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:32:01.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.586+0000 7f3a92d9d640 1 --2- 172.21.15.167:0/3400575433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a94108750 0x7f3a9419b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:32:01.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.586+0000 7f3a92d9d640 1 -- 172.21.15.167:0/3400575433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a94107e00 msgr2=0x7f3a9419b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:32:01.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.586+0000 7f3a92d9d640 1 --2- 172.21.15.167:0/3400575433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a94107e00 0x7f3a9419b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:32:01.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.586+0000 7f3a92d9d640 1 -- 172.21.15.167:0/3400575433 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a941a5f80 con 0x7f3a941024c0 2024-07-26T13:32:01.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.586+0000 7f3a9359e640 1 --2- 172.21.15.167:0/3400575433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a94108750 0x7f3a9419b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:32:01.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.586+0000 7f3a8bfff640 1 --2- 172.21.15.167:0/3400575433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a94107e00 0x7f3a9419b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:32:01.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.587+0000 7f3a92d9d640 1 --2- 172.21.15.167:0/3400575433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a941024c0 0x7f3a9419acc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3a8400d950 tx=0x7f3a8400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:32:01.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.587+0000 7f3a90d99640 1 -- 172.21.15.167:0/3400575433 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a84017070 con 0x7f3a941024c0 2024-07-26T13:32:01.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.587+0000 7f3a90d99640 1 -- 172.21.15.167:0/3400575433 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a84009c30 con 0x7f3a941024c0 2024-07-26T13:32:01.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.587+0000 7f3a9907b640 1 -- 172.21.15.167:0/3400575433 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a941a6270 con 0x7f3a941024c0 2024-07-26T13:32:01.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.587+0000 7f3a90d99640 1 -- 172.21.15.167:0/3400575433 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8400f620 con 0x7f3a941024c0 2024-07-26T13:32:01.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.587+0000 7f3a9907b640 1 -- 172.21.15.167:0/3400575433 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a941a67b0 con 0x7f3a941024c0 2024-07-26T13:32:01.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.588+0000 7f3a9907b640 1 -- 172.21.15.167:0/3400575433 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a94103740 con 0x7f3a941024c0 2024-07-26T13:32:01.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.589+0000 7f3a90d99640 1 -- 172.21.15.167:0/3400575433 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7f3a84010460 con 0x7f3a941024c0 2024-07-26T13:32:01.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.590+0000 7f3a90d99640 1 --2- 172.21.15.167:0/3400575433 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3a800b3b90 0x7f3a800b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:32:01.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.590+0000 7f3a90d99640 1 -- 172.21.15.167:0/3400575433 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2524+0+0 (secure 0 0 0) 0x7f3a840e2170 con 0x7f3a941024c0 2024-07-26T13:32:01.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.590+0000 7f3a8bfff640 1 --2- 172.21.15.167:0/3400575433 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3a800b3b90 0x7f3a800b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:32:01.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.591+0000 7f3a8bfff640 1 --2- 172.21.15.167:0/3400575433 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3a800b3b90 0x7f3a800b60a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c005fd0 tx=0x7f3a7c00b5d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:32:01.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.593+0000 7f3a90d99640 1 -- 172.21.15.167:0/3400575433 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f3a840aaf40 con 0x7f3a941024c0 2024-07-26T13:32:01.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:01.776+0000 7f3a9907b640 1 -- 172.21.15.167:0/3400575433 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f3a94063230 con 0x7f3a800b3b90 2024-07-26T13:32:02.347 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:02 smithi167 ceph-mon[37140]: osdmap e15: 2 total, 2 up, 2 in 2024-07-26T13:32:02.347 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-26T13:32:02.347 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-26T13:32:02.347 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:32:02.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:02 smithi138 ceph-mon[36954]: osdmap e15: 2 total, 2 up, 2 in 2024-07-26T13:32:02.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-26T13:32:02.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-26T13:32:02.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:32:02.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:02 smithi100 ceph-mon[32599]: osdmap e15: 2 total, 2 up, 2 in 2024-07-26T13:32:02.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-26T13:32:02.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-26T13:32:02.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:32:03.387 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:03 smithi167 ceph-mon[37140]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:32:03.387 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:03 smithi167 ceph-mon[37140]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-26T13:32:03.395 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:03 smithi100 ceph-mon[32599]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:32:03.395 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:03 smithi100 ceph-mon[32599]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-26T13:32:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:03 smithi138 ceph-mon[36954]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:32:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:03 smithi138 ceph-mon[36954]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-26T13:32:04.435 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:04 smithi167 ceph-mon[37140]: from='client.? 172.21.15.167:0/2567611830' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5c710a78-17ae-41a8-ac16-a678f5d35672"}]: dispatch 2024-07-26T13:32:04.435 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:04 smithi167 ceph-mon[37140]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5c710a78-17ae-41a8-ac16-a678f5d35672"}]: dispatch 2024-07-26T13:32:04.435 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:04 smithi167 ceph-mon[37140]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5c710a78-17ae-41a8-ac16-a678f5d35672"}]': finished 2024-07-26T13:32:04.435 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:04 smithi167 ceph-mon[37140]: osdmap e16: 3 total, 2 up, 3 in 2024-07-26T13:32:04.435 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-26T13:32:04.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:04 smithi138 ceph-mon[36954]: from='client.? 172.21.15.167:0/2567611830' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5c710a78-17ae-41a8-ac16-a678f5d35672"}]: dispatch 2024-07-26T13:32:04.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:04 smithi138 ceph-mon[36954]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5c710a78-17ae-41a8-ac16-a678f5d35672"}]: dispatch 2024-07-26T13:32:04.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:04 smithi138 ceph-mon[36954]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5c710a78-17ae-41a8-ac16-a678f5d35672"}]': finished 2024-07-26T13:32:04.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:04 smithi138 ceph-mon[36954]: osdmap e16: 3 total, 2 up, 3 in 2024-07-26T13:32:04.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-26T13:32:04.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:04 smithi100 ceph-mon[32599]: from='client.? 172.21.15.167:0/2567611830' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5c710a78-17ae-41a8-ac16-a678f5d35672"}]: dispatch 2024-07-26T13:32:04.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:04 smithi100 ceph-mon[32599]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5c710a78-17ae-41a8-ac16-a678f5d35672"}]: dispatch 2024-07-26T13:32:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:04 smithi100 ceph-mon[32599]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5c710a78-17ae-41a8-ac16-a678f5d35672"}]': finished 2024-07-26T13:32:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:04 smithi100 ceph-mon[32599]: osdmap e16: 3 total, 2 up, 3 in 2024-07-26T13:32:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-26T13:32:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:05 smithi138 ceph-mon[36954]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-26T13:32:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:05 smithi138 ceph-mon[36954]: from='client.? 172.21.15.167:0/3196173643' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-26T13:32:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:05 smithi100 ceph-mon[32599]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-26T13:32:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:05 smithi100 ceph-mon[32599]: from='client.? 172.21.15.167:0/3196173643' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-26T13:32:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:05 smithi167 ceph-mon[37140]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-26T13:32:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:05 smithi167 ceph-mon[37140]: from='client.? 172.21.15.167:0/3196173643' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-26T13:32:07.625 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:07 smithi167 ceph-mon[37140]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-26T13:32:07.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:07 smithi138 ceph-mon[36954]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-26T13:32:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:07 smithi100 ceph-mon[32599]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-26T13:32:09.606 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:09 smithi167 ceph-mon[37140]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-26T13:32:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:09 smithi138 ceph-mon[36954]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-26T13:32:09.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:09 smithi100 ceph-mon[32599]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-26T13:32:11.465 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:11 smithi167 ceph-mon[37140]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-26T13:32:11.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-07-26T13:32:11.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:32:11.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:11 smithi138 ceph-mon[36954]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-26T13:32:11.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-07-26T13:32:11.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:32:11.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:11 smithi100 ceph-mon[32599]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-26T13:32:11.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-07-26T13:32:11.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:32:12.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:12 smithi167 ceph-mon[37140]: Deploying daemon osd.2 on smithi167 2024-07-26T13:32:12.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:12 smithi138 ceph-mon[36954]: Deploying daemon osd.2 on smithi167 2024-07-26T13:32:12.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:12 smithi100 ceph-mon[32599]: Deploying daemon osd.2 on smithi167 2024-07-26T13:32:13.460 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:13 smithi167 ceph-mon[37140]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-26T13:32:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:13 smithi138 ceph-mon[36954]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-26T13:32:13.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:13 smithi100 ceph-mon[32599]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-26T13:32:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:15 smithi138 ceph-mon[36954]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-26T13:32:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:15 smithi100 ceph-mon[32599]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-26T13:32:15.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:15 smithi167 ceph-mon[37140]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-26T13:32:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:32:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:32:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:16.761 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:32:16.761 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:16.761 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:17.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:17 smithi167 ceph-mon[37140]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-26T13:32:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:17 smithi138 ceph-mon[36954]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-26T13:32:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:17 smithi100 ceph-mon[32599]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-26T13:32:18.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:18.605+0000 7f3a90d99640 1 -- 172.21.15.167:0/3400575433 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3a94063230 con 0x7f3a800b3b90 2024-07-26T13:32:18.607 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 2 on host 'smithi167' 2024-07-26T13:32:18.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:18.607+0000 7f3a9907b640 1 -- 172.21.15.167:0/3400575433 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3a800b3b90 msgr2=0x7f3a800b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:32:18.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:18.607+0000 7f3a9907b640 1 --2- 172.21.15.167:0/3400575433 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3a800b3b90 0x7f3a800b60a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c005fd0 tx=0x7f3a7c00b5d0 comp rx=0 tx=0).stop 2024-07-26T13:32:18.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:18.607+0000 7f3a9907b640 1 -- 172.21.15.167:0/3400575433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a941024c0 msgr2=0x7f3a9419acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:32:18.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:18.607+0000 7f3a9907b640 1 --2- 172.21.15.167:0/3400575433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a941024c0 0x7f3a9419acc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3a8400d950 tx=0x7f3a8400de10 comp rx=0 tx=0).stop 2024-07-26T13:32:18.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:18.607+0000 7f3a9907b640 1 -- 172.21.15.167:0/3400575433 shutdown_connections 2024-07-26T13:32:18.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:18.607+0000 7f3a9907b640 1 --2- 172.21.15.167:0/3400575433 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3a800b3b90 0x7f3a800b60a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:32:18.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:18.607+0000 7f3a9907b640 1 --2- 172.21.15.167:0/3400575433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a94108750 0x7f3a9419b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:32:18.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:18.607+0000 7f3a9907b640 1 --2- 172.21.15.167:0/3400575433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a94107e00 0x7f3a9419b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:32:18.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:18.607+0000 7f3a9907b640 1 --2- 172.21.15.167:0/3400575433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a941024c0 0x7f3a9419acc0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:32:18.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:18.607+0000 7f3a9907b640 1 -- 172.21.15.167:0/3400575433 >> 172.21.15.167:0/3400575433 conn(0x7f3a94079180 msgr2=0x7f3a94104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:32:18.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:18.607+0000 7f3a9907b640 1 -- 172.21.15.167:0/3400575433 shutdown_connections 2024-07-26T13:32:18.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:32:18.607+0000 7f3a9907b640 1 -- 172.21.15.167:0/3400575433 wait complete. 2024-07-26T13:32:19.142 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:18 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:19.142 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:18 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:19.142 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:18 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:32:19.142 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:18 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:32:19.142 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:18 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:19.142 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:18 smithi167 ceph-mon[37140]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-26T13:32:19.143 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:18 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:32:19.143 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:18 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:19.143 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:18 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:19.170 DEBUG:teuthology.orchestra.run.smithi167:osd.2> sudo journalctl -f -n 0 -u ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@osd.2.service 2024-07-26T13:32:19.172 INFO:tasks.cephadm:Deploying osd.3 on smithi100 with /dev/vg_nvme/lv_3... 2024-07-26T13:32:19.172 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-07-26T13:32:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:18 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:18 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:18 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:32:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:18 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:32:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:18 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:18 smithi138 ceph-mon[36954]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-26T13:32:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:18 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:32:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:18 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:18 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:19.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:18 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:19.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:18 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:19.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:18 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:32:19.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:18 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:32:19.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:18 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:19.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:18 smithi100 ceph-mon[32599]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-26T13:32:19.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:18 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:32:19.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:18 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:19.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:18 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:19.435 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:32:19.678 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jul 26 13:32:19 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-osd-2[43566]: 2024-07-26T13:32:19.490+0000 7f837f8ec640 -1 osd.2 0 log_to_monitors true 2024-07-26T13:32:20.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:19 smithi167 ceph-mon[37140]: from='osd.2 [v2:172.21.15.167:6800/3583424790,v1:172.21.15.167:6801/3583424790]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-26T13:32:20.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:19 smithi167 ceph-mon[37140]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-26T13:32:20.087 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:19 smithi100 ceph-mon[32599]: from='osd.2 [v2:172.21.15.167:6800/3583424790,v1:172.21.15.167:6801/3583424790]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-26T13:32:20.087 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:19 smithi100 ceph-mon[32599]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-26T13:32:20.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:19 smithi138 ceph-mon[36954]: from='osd.2 [v2:172.21.15.167:6800/3583424790,v1:172.21.15.167:6801/3583424790]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-26T13:32:20.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:19 smithi138 ceph-mon[36954]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-26T13:32:20.975 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-07-26T13:32:20.975 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:20 smithi100 ceph-mon[32599]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-07-26T13:32:20.975 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:20 smithi100 ceph-mon[32599]: osdmap e17: 3 total, 2 up, 3 in 2024-07-26T13:32:20.975 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:20 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-26T13:32:20.975 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:20 smithi100 ceph-mon[32599]: from='osd.2 [v2:172.21.15.167:6800/3583424790,v1:172.21.15.167:6801/3583424790]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-07-26T13:32:20.975 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:20 smithi100 ceph-mon[32599]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-07-26T13:32:20.975 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:20 smithi100 ceph-mon[32599]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-26T13:32:21.054 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-07-26T13:32:21.068 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-07-26T13:32:21.068 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-07-26T13:32:21.068 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114731 s, 914 MB/s 2024-07-26T13:32:21.068 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-07-26T13:32:21.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:20 smithi138 ceph-mon[36954]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-07-26T13:32:21.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:20 smithi138 ceph-mon[36954]: osdmap e17: 3 total, 2 up, 3 in 2024-07-26T13:32:21.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:20 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-26T13:32:21.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:20 smithi138 ceph-mon[36954]: from='osd.2 [v2:172.21.15.167:6800/3583424790,v1:172.21.15.167:6801/3583424790]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-07-26T13:32:21.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:20 smithi138 ceph-mon[36954]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-07-26T13:32:21.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:20 smithi138 ceph-mon[36954]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-26T13:32:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:20 smithi167 ceph-mon[37140]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-07-26T13:32:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:20 smithi167 ceph-mon[37140]: osdmap e17: 3 total, 2 up, 3 in 2024-07-26T13:32:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:20 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-26T13:32:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:20 smithi167 ceph-mon[37140]: from='osd.2 [v2:172.21.15.167:6800/3583424790,v1:172.21.15.167:6801/3583424790]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-07-26T13:32:21.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:20 smithi167 ceph-mon[37140]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-07-26T13:32:21.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:20 smithi167 ceph-mon[37140]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-26T13:32:21.680 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jul 26 13:32:21 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-osd-2[43566]: 2024-07-26T13:32:21.278+0000 7f837b86e640 -1 osd.2 0 waiting for initial osdmap 2024-07-26T13:32:21.681 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jul 26 13:32:21 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-osd-2[43566]: 2024-07-26T13:32:21.283+0000 7f8373664640 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-26T13:32:21.698 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph orch daemon add osd smithi100:vg_nvme/lv_3 2024-07-26T13:32:21.981 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:21 smithi167 ceph-mon[37140]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-07-26T13:32:21.982 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:21 smithi167 ceph-mon[37140]: osdmap e18: 3 total, 2 up, 3 in 2024-07-26T13:32:21.982 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:21 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-26T13:32:21.982 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:21 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-26T13:32:21.982 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:21 smithi167 ceph-mon[37140]: from='osd.2 ' entity='osd.2' 2024-07-26T13:32:21.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:21 smithi100 ceph-mon[32599]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-07-26T13:32:21.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:21 smithi100 ceph-mon[32599]: osdmap e18: 3 total, 2 up, 3 in 2024-07-26T13:32:21.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:21 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-26T13:32:21.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:21 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-26T13:32:21.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:21 smithi100 ceph-mon[32599]: from='osd.2 ' entity='osd.2' 2024-07-26T13:32:21.998 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:32:22.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:21 smithi138 ceph-mon[36954]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-07-26T13:32:22.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:21 smithi138 ceph-mon[36954]: osdmap e18: 3 total, 2 up, 3 in 2024-07-26T13:32:22.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:21 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-26T13:32:22.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:21 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-26T13:32:22.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:21 smithi138 ceph-mon[36954]: from='osd.2 ' entity='osd.2' 2024-07-26T13:32:23.126 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:22 smithi100 ceph-mon[32599]: purged_snaps scrub starts 2024-07-26T13:32:23.127 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:22 smithi100 ceph-mon[32599]: purged_snaps scrub ok 2024-07-26T13:32:23.127 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-26T13:32:23.127 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:22 smithi100 ceph-mon[32599]: Detected new or changed devices on smithi167 2024-07-26T13:32:23.127 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:23.127 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:23.127 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:32:23.127 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:22 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi167 to 17088M 2024-07-26T13:32:23.127 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:23.127 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:32:23.127 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:32:23.127 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:23.127 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:22 smithi100 ceph-mon[32599]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-26T13:32:23.128 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:22 smithi100 ceph-mon[32599]: osd.2 [v2:172.21.15.167:6800/3583424790,v1:172.21.15.167:6801/3583424790] boot 2024-07-26T13:32:23.128 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:22 smithi100 ceph-mon[32599]: osdmap e19: 3 total, 3 up, 3 in 2024-07-26T13:32:23.128 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-26T13:32:23.128 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-07-26T13:32:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:22 smithi138 ceph-mon[36954]: purged_snaps scrub starts 2024-07-26T13:32:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:22 smithi138 ceph-mon[36954]: purged_snaps scrub ok 2024-07-26T13:32:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-26T13:32:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:22 smithi138 ceph-mon[36954]: Detected new or changed devices on smithi167 2024-07-26T13:32:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:23.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:23.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:32:23.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:22 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi167 to 17088M 2024-07-26T13:32:23.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:23.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:32:23.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:32:23.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:23.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:22 smithi138 ceph-mon[36954]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-26T13:32:23.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:22 smithi138 ceph-mon[36954]: osd.2 [v2:172.21.15.167:6800/3583424790,v1:172.21.15.167:6801/3583424790] boot 2024-07-26T13:32:23.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:22 smithi138 ceph-mon[36954]: osdmap e19: 3 total, 3 up, 3 in 2024-07-26T13:32:23.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-26T13:32:23.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-07-26T13:32:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:22 smithi167 ceph-mon[37140]: purged_snaps scrub starts 2024-07-26T13:32:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:22 smithi167 ceph-mon[37140]: purged_snaps scrub ok 2024-07-26T13:32:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-26T13:32:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:22 smithi167 ceph-mon[37140]: Detected new or changed devices on smithi167 2024-07-26T13:32:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:32:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:22 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi167 to 17088M 2024-07-26T13:32:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:32:23.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:32:23.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:23.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:22 smithi167 ceph-mon[37140]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-07-26T13:32:23.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:22 smithi167 ceph-mon[37140]: osd.2 [v2:172.21.15.167:6800/3583424790,v1:172.21.15.167:6801/3583424790] boot 2024-07-26T13:32:23.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:22 smithi167 ceph-mon[37140]: osdmap e19: 3 total, 3 up, 3 in 2024-07-26T13:32:23.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-26T13:32:23.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-07-26T13:32:23.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.612+0000 7f6b10496640 1 -- 172.21.15.100:0/391998087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b08104c50 msgr2=0x7f6b08105060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:32:23.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.612+0000 7f6b10496640 1 --2- 172.21.15.100:0/391998087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b08104c50 0x7f6b08105060 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6b04009900 tx=0x7f6b0402a410 comp rx=0 tx=0).stop 2024-07-26T13:32:23.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.612+0000 7f6b10496640 1 -- 172.21.15.100:0/391998087 shutdown_connections 2024-07-26T13:32:23.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.612+0000 7f6b10496640 1 --2- 172.21.15.100:0/391998087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b081055a0 0x7f6b0810f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:32:23.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.612+0000 7f6b10496640 1 --2- 172.21.15.100:0/391998087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b08104c50 0x7f6b08105060 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:32:23.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.612+0000 7f6b10496640 1 --2- 172.21.15.100:0/391998087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b08102c40 0x7f6b08103050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:32:23.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.612+0000 7f6b10496640 1 -- 172.21.15.100:0/391998087 >> 172.21.15.100:0/391998087 conn(0x7f6b080798f0 msgr2=0x7f6b08079d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:32:23.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.613+0000 7f6b10496640 1 -- 172.21.15.100:0/391998087 shutdown_connections 2024-07-26T13:32:23.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.613+0000 7f6b10496640 1 -- 172.21.15.100:0/391998087 wait complete. 2024-07-26T13:32:23.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.613+0000 7f6b10496640 1 Processor -- start 2024-07-26T13:32:23.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.613+0000 7f6b10496640 1 -- start start 2024-07-26T13:32:23.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.613+0000 7f6b10496640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b08102c40 0x7f6b0819a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:32:23.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.613+0000 7f6b10496640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b08104c50 0x7f6b0819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:32:23.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.613+0000 7f6b10496640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b081055a0 0x7f6b0819f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:32:23.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.614+0000 7f6b10496640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6b08113640 con 0x7f6b081055a0 2024-07-26T13:32:23.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.614+0000 7f6b10496640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6b081134c0 con 0x7f6b08102c40 2024-07-26T13:32:23.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.614+0000 7f6b10496640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6b081137c0 con 0x7f6b08104c50 2024-07-26T13:32:23.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.614+0000 7f6b0ea33640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b081055a0 0x7f6b0819f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:32:23.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.614+0000 7f6b0ea33640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b081055a0 0x7f6b0819f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35090/0 (socket says 172.21.15.100:35090) 2024-07-26T13:32:23.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.614+0000 7f6b0ea33640 1 -- 172.21.15.100:0/582866681 learned_addr learned my addr 172.21.15.100:0/582866681 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:32:23.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.614+0000 7f6b0da31640 1 --2- 172.21.15.100:0/582866681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b08104c50 0x7f6b0819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:32:23.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.614+0000 7f6b0e232640 1 --2- 172.21.15.100:0/582866681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b08102c40 0x7f6b0819a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:32:23.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.614+0000 7f6b0ea33640 1 -- 172.21.15.100:0/582866681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b08104c50 msgr2=0x7f6b0819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:32:23.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.614+0000 7f6b0ea33640 1 --2- 172.21.15.100:0/582866681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b08104c50 0x7f6b0819a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:32:23.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.614+0000 7f6b0ea33640 1 -- 172.21.15.100:0/582866681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b08102c40 msgr2=0x7f6b0819a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:32:23.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.614+0000 7f6b0ea33640 1 --2- 172.21.15.100:0/582866681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b08102c40 0x7f6b0819a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:32:23.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.614+0000 7f6b0ea33640 1 -- 172.21.15.100:0/582866681 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b081a0000 con 0x7f6b081055a0 2024-07-26T13:32:23.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.614+0000 7f6b0da31640 1 --2- 172.21.15.100:0/582866681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b08104c50 0x7f6b0819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:32:23.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.614+0000 7f6b0e232640 1 --2- 172.21.15.100:0/582866681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b08102c40 0x7f6b0819a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:32:23.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.614+0000 7f6b0ea33640 1 --2- 172.21.15.100:0/582866681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b081055a0 0x7f6b0819f8d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6b0000b970 tx=0x7f6b0000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:32:23.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.614+0000 7f6afb7fe640 1 -- 172.21.15.100:0/582866681 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b00019070 con 0x7f6b081055a0 2024-07-26T13:32:23.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.614+0000 7f6afb7fe640 1 -- 172.21.15.100:0/582866681 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b00014440 con 0x7f6b081055a0 2024-07-26T13:32:23.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.614+0000 7f6b10496640 1 -- 172.21.15.100:0/582866681 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b081a02f0 con 0x7f6b081055a0 2024-07-26T13:32:23.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.614+0000 7f6afb7fe640 1 -- 172.21.15.100:0/582866681 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b000075d0 con 0x7f6b081055a0 2024-07-26T13:32:23.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.614+0000 7f6b10496640 1 -- 172.21.15.100:0/582866681 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b081a6a50 con 0x7f6b081055a0 2024-07-26T13:32:23.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.616+0000 7f6b10496640 1 -- 172.21.15.100:0/582866681 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b08103050 con 0x7f6b081055a0 2024-07-26T13:32:23.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.617+0000 7f6afb7fe640 1 -- 172.21.15.100:0/582866681 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7f6b0001c070 con 0x7f6b081055a0 2024-07-26T13:32:23.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.618+0000 7f6afb7fe640 1 --2- 172.21.15.100:0/582866681 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6af40b3b90 0x7f6af40b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:32:23.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.618+0000 7f6b0e232640 1 --2- 172.21.15.100:0/582866681 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6af40b3b90 0x7f6af40b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:32:23.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.618+0000 7f6afb7fe640 1 -- 172.21.15.100:0/582866681 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3441+0+0 (secure 0 0 0) 0x7f6b000e2330 con 0x7f6b081055a0 2024-07-26T13:32:23.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.618+0000 7f6b0e232640 1 --2- 172.21.15.100:0/582866681 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6af40b3b90 0x7f6af40b60a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6afc009930 tx=0x7f6afc008040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:32:23.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.621+0000 7f6afb7fe640 1 -- 172.21.15.100:0/582866681 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f6b000aad60 con 0x7f6b081055a0 2024-07-26T13:32:23.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:23.801+0000 7f6b10496640 1 -- 172.21.15.100:0/582866681 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f6b08063230 con 0x7f6af40b3b90 2024-07-26T13:32:24.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-07-26T13:32:24.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:24 smithi100 ceph-mon[32599]: osdmap e20: 3 total, 3 up, 3 in 2024-07-26T13:32:24.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-07-26T13:32:24.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-26T13:32:24.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-26T13:32:24.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:32:24.495 INFO:journalctl@ceph.osd.0.smithi100.stdout:Jul 26 13:32:24 smithi100 sudo[56572]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-26T13:32:24.495 INFO:journalctl@ceph.osd.0.smithi100.stdout:Jul 26 13:32:24 smithi100 sudo[56572]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-26T13:32:24.495 INFO:journalctl@ceph.osd.0.smithi100.stdout:Jul 26 13:32:24 smithi100 sudo[56572]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-26T13:32:24.495 INFO:journalctl@ceph.osd.0.smithi100.stdout:Jul 26 13:32:24 smithi100 sudo[56572]: pam_unix(sudo:session): session closed for user root 2024-07-26T13:32:24.495 INFO:journalctl@ceph.osd.0.smithi100.stdout:Jul 26 13:32:24 smithi100 sudo[56576]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-26T13:32:24.495 INFO:journalctl@ceph.osd.0.smithi100.stdout:Jul 26 13:32:24 smithi100 sudo[56576]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-26T13:32:24.495 INFO:journalctl@ceph.osd.0.smithi100.stdout:Jul 26 13:32:24 smithi100 sudo[56576]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-26T13:32:24.495 INFO:journalctl@ceph.osd.0.smithi100.stdout:Jul 26 13:32:24 smithi100 sudo[56576]: pam_unix(sudo:session): session closed for user root 2024-07-26T13:32:24.609 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-07-26T13:32:24.609 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:24 smithi167 ceph-mon[37140]: osdmap e20: 3 total, 3 up, 3 in 2024-07-26T13:32:24.609 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-07-26T13:32:24.610 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-26T13:32:24.610 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-26T13:32:24.610 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:32:24.610 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jul 26 13:32:24 smithi167 sudo[47628]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-26T13:32:24.610 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jul 26 13:32:24 smithi167 sudo[47628]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-26T13:32:24.610 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jul 26 13:32:24 smithi167 sudo[47628]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-26T13:32:24.611 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jul 26 13:32:24 smithi167 sudo[47628]: pam_unix(sudo:session): session closed for user root 2024-07-26T13:32:24.611 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jul 26 13:32:24 smithi167 sudo[47631]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-26T13:32:24.611 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jul 26 13:32:24 smithi167 sudo[47631]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-26T13:32:24.611 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jul 26 13:32:24 smithi167 sudo[47631]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-26T13:32:24.611 INFO:journalctl@ceph.osd.2.smithi167.stdout:Jul 26 13:32:24 smithi167 sudo[47631]: pam_unix(sudo:session): session closed for user root 2024-07-26T13:32:24.649 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-07-26T13:32:24.649 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:24 smithi138 ceph-mon[36954]: osdmap e20: 3 total, 3 up, 3 in 2024-07-26T13:32:24.650 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-07-26T13:32:24.650 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-26T13:32:24.650 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-26T13:32:24.650 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:32:24.650 INFO:journalctl@ceph.osd.1.smithi138.stdout:Jul 26 13:32:24 smithi138 sudo[46840]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-26T13:32:24.650 INFO:journalctl@ceph.osd.1.smithi138.stdout:Jul 26 13:32:24 smithi138 sudo[46840]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-26T13:32:24.650 INFO:journalctl@ceph.osd.1.smithi138.stdout:Jul 26 13:32:24 smithi138 sudo[46840]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-26T13:32:24.650 INFO:journalctl@ceph.osd.1.smithi138.stdout:Jul 26 13:32:24 smithi138 sudo[46840]: pam_unix(sudo:session): session closed for user root 2024-07-26T13:32:24.650 INFO:journalctl@ceph.osd.1.smithi138.stdout:Jul 26 13:32:24 smithi138 sudo[46843]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-26T13:32:24.650 INFO:journalctl@ceph.osd.1.smithi138.stdout:Jul 26 13:32:24 smithi138 sudo[46843]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-26T13:32:24.651 INFO:journalctl@ceph.osd.1.smithi138.stdout:Jul 26 13:32:24 smithi138 sudo[46843]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-26T13:32:24.651 INFO:journalctl@ceph.osd.1.smithi138.stdout:Jul 26 13:32:24 smithi138 sudo[46843]: pam_unix(sudo:session): session closed for user root 2024-07-26T13:32:24.845 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:24 smithi100 sudo[56594]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-26T13:32:24.846 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:24 smithi100 sudo[56594]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-26T13:32:24.846 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:24 smithi100 sudo[56594]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-26T13:32:24.846 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:24 smithi100 sudo[56594]: pam_unix(sudo:session): session closed for user root 2024-07-26T13:32:24.846 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:24 smithi100 sudo[56597]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-26T13:32:24.846 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:24 smithi100 sudo[56597]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-26T13:32:24.846 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:24 smithi100 sudo[56597]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-26T13:32:24.846 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:24 smithi100 sudo[56597]: pam_unix(sudo:session): session closed for user root 2024-07-26T13:32:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:24 smithi138 sudo[46846]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-26T13:32:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:24 smithi138 sudo[46846]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-26T13:32:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:24 smithi138 sudo[46846]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-26T13:32:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:24 smithi138 sudo[46846]: pam_unix(sudo:session): session closed for user root 2024-07-26T13:32:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:24 smithi138 sudo[46849]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-26T13:32:24.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:24 smithi138 sudo[46849]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-26T13:32:24.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:24 smithi138 sudo[46849]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-26T13:32:24.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:24 smithi138 sudo[46849]: pam_unix(sudo:session): session closed for user root 2024-07-26T13:32:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:24 smithi167 sudo[47634]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-26T13:32:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:24 smithi167 sudo[47634]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-26T13:32:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:24 smithi167 sudo[47634]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-26T13:32:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:24 smithi167 sudo[47634]: pam_unix(sudo:session): session closed for user root 2024-07-26T13:32:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:24 smithi167 sudo[47637]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-26T13:32:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:24 smithi167 sudo[47637]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-26T13:32:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:24 smithi167 sudo[47637]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-26T13:32:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:24 smithi167 sudo[47637]: pam_unix(sudo:session): session closed for user root 2024-07-26T13:32:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:25 smithi100 ceph-mon[32599]: from='client.14478 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:32:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:25 smithi100 ceph-mon[32599]: pgmap v72: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-26T13:32:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:25 smithi100 ceph-mon[32599]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-07-26T13:32:25.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-07-26T13:32:25.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:25 smithi100 ceph-mon[32599]: osdmap e21: 3 total, 3 up, 3 in 2024-07-26T13:32:25.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:25.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:25.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:25.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:25 smithi100 ceph-mon[32599]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-26T13:32:25.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:25 smithi100 ceph-mon[32599]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-26T13:32:25.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-26T13:32:25.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-26T13:32:25.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-26T13:32:25.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:25 smithi100 ceph-mon[32599]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-26T13:32:25.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-26T13:32:25.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-26T13:32:25.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-26T13:32:25.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:25 smithi100 ceph-mon[32599]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-26T13:32:25.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:25 smithi100 ceph-mon[32599]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-26T13:32:25.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-26T13:32:25.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-26T13:32:25.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-26T13:32:25.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:25 smithi100 ceph-mon[32599]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-26T13:32:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:25 smithi138 ceph-mon[36954]: from='client.14478 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:32:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:25 smithi138 ceph-mon[36954]: pgmap v72: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-26T13:32:25.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:25 smithi138 ceph-mon[36954]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-07-26T13:32:25.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-07-26T13:32:25.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:25 smithi138 ceph-mon[36954]: osdmap e21: 3 total, 3 up, 3 in 2024-07-26T13:32:25.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:25.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:25.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:25.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:25 smithi138 ceph-mon[36954]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-26T13:32:25.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:25 smithi138 ceph-mon[36954]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-26T13:32:25.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-26T13:32:25.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-26T13:32:25.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-26T13:32:25.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:25 smithi138 ceph-mon[36954]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-26T13:32:25.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-26T13:32:25.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-26T13:32:25.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-26T13:32:25.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:25 smithi138 ceph-mon[36954]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-26T13:32:25.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:25 smithi138 ceph-mon[36954]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-26T13:32:25.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-26T13:32:25.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-26T13:32:25.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-26T13:32:25.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:25 smithi138 ceph-mon[36954]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-26T13:32:25.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:25 smithi167 ceph-mon[37140]: from='client.14478 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:32:25.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:25 smithi167 ceph-mon[37140]: pgmap v72: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-26T13:32:25.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:25 smithi167 ceph-mon[37140]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-07-26T13:32:25.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-07-26T13:32:25.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:25 smithi167 ceph-mon[37140]: osdmap e21: 3 total, 3 up, 3 in 2024-07-26T13:32:25.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:25.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:25.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:25.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:25 smithi167 ceph-mon[37140]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-26T13:32:25.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:25 smithi167 ceph-mon[37140]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-26T13:32:25.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-26T13:32:25.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-26T13:32:25.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-26T13:32:25.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:25 smithi167 ceph-mon[37140]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-26T13:32:25.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-26T13:32:25.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-26T13:32:25.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-26T13:32:25.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:25 smithi167 ceph-mon[37140]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-26T13:32:25.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:25 smithi167 ceph-mon[37140]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-26T13:32:25.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-26T13:32:25.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-26T13:32:25.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-26T13:32:25.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:25 smithi167 ceph-mon[37140]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-26T13:32:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:26 smithi138 ceph-mon[36954]: osdmap e22: 3 total, 3 up, 3 in 2024-07-26T13:32:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:26 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/3467977862' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be845495-8c31-4834-a45c-bdc576711962"}]: dispatch 2024-07-26T13:32:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:26 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/3467977862' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "be845495-8c31-4834-a45c-bdc576711962"}]': finished 2024-07-26T13:32:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:26 smithi138 ceph-mon[36954]: osdmap e23: 4 total, 3 up, 4 in 2024-07-26T13:32:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-26T13:32:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:26 smithi100 ceph-mon[32599]: osdmap e22: 3 total, 3 up, 3 in 2024-07-26T13:32:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:26 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/3467977862' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be845495-8c31-4834-a45c-bdc576711962"}]: dispatch 2024-07-26T13:32:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:26 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/3467977862' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "be845495-8c31-4834-a45c-bdc576711962"}]': finished 2024-07-26T13:32:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:26 smithi100 ceph-mon[32599]: osdmap e23: 4 total, 3 up, 4 in 2024-07-26T13:32:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-26T13:32:26.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:26 smithi167 ceph-mon[37140]: osdmap e22: 3 total, 3 up, 3 in 2024-07-26T13:32:26.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:26 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/3467977862' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "be845495-8c31-4834-a45c-bdc576711962"}]: dispatch 2024-07-26T13:32:26.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:26 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/3467977862' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "be845495-8c31-4834-a45c-bdc576711962"}]': finished 2024-07-26T13:32:26.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:26 smithi167 ceph-mon[37140]: osdmap e23: 4 total, 3 up, 4 in 2024-07-26T13:32:26.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-26T13:32:27.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:27.167+0000 7f6afb7fe640 1 -- 172.21.15.100:0/582866681 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f6b000a68f0 con 0x7f6b081055a0 2024-07-26T13:32:27.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:27 smithi100 ceph-mon[32599]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-26T13:32:27.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:27 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/745766867' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-26T13:32:27.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:27 smithi100 ceph-mon[32599]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-07-26T13:32:27.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:27 smithi100 ceph-mon[32599]: Cluster is now healthy 2024-07-26T13:32:27.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:27 smithi100 ceph-mon[32599]: mgrmap e20: x(active, since 2m), standbys: y, z 2024-07-26T13:32:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:27 smithi138 ceph-mon[36954]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-26T13:32:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:27 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/745766867' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-26T13:32:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:27 smithi138 ceph-mon[36954]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-07-26T13:32:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:27 smithi138 ceph-mon[36954]: Cluster is now healthy 2024-07-26T13:32:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:27 smithi138 ceph-mon[36954]: mgrmap e20: x(active, since 2m), standbys: y, z 2024-07-26T13:32:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:27 smithi167 ceph-mon[37140]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-26T13:32:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:27 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/745766867' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-26T13:32:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:27 smithi167 ceph-mon[37140]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-07-26T13:32:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:27 smithi167 ceph-mon[37140]: Cluster is now healthy 2024-07-26T13:32:27.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:27 smithi167 ceph-mon[37140]: mgrmap e20: x(active, since 2m), standbys: y, z 2024-07-26T13:32:29.672 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:29 smithi100 ceph-mon[32599]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-26T13:32:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:29 smithi138 ceph-mon[36954]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-26T13:32:29.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:29 smithi167 ceph-mon[37140]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-26T13:32:31.678 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:31 smithi100 ceph-mon[32599]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-26T13:32:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:31 smithi138 ceph-mon[36954]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-26T13:32:31.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:31 smithi167 ceph-mon[37140]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-26T13:32:33.500 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:33 smithi100 ceph-mon[32599]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-26T13:32:33.500 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-07-26T13:32:33.500 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:32:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:33 smithi138 ceph-mon[36954]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-26T13:32:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-07-26T13:32:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:32:33.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:33 smithi167 ceph-mon[37140]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-26T13:32:33.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-07-26T13:32:33.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:32:34.594 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:34 smithi100 ceph-mon[32599]: Deploying daemon osd.3 on smithi100 2024-07-26T13:32:34.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:34 smithi138 ceph-mon[36954]: Deploying daemon osd.3 on smithi100 2024-07-26T13:32:34.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:34 smithi167 ceph-mon[37140]: Deploying daemon osd.3 on smithi100 2024-07-26T13:32:35.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:35 smithi138 ceph-mon[36954]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-26T13:32:35.740 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:35 smithi100 ceph-mon[32599]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-26T13:32:35.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:35 smithi167 ceph-mon[37140]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-26T13:32:37.486 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:37 smithi100 ceph-mon[32599]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-26T13:32:37.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:37 smithi138 ceph-mon[36954]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-26T13:32:37.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:37 smithi167 ceph-mon[37140]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-26T13:32:38.610 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:32:38.610 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:38.610 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:38.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:32:38.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:38.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:38.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:32:38.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:38.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:39.609 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:39 smithi100 ceph-mon[32599]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-26T13:32:39.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:39 smithi138 ceph-mon[36954]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-26T13:32:39.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:39 smithi167 ceph-mon[37140]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-26T13:32:41.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:41.198+0000 7f6afb7fe640 1 -- 172.21.15.100:0/582866681 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6b08063230 con 0x7f6af40b3b90 2024-07-26T13:32:41.200 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 3 on host 'smithi100' 2024-07-26T13:32:41.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:41.201+0000 7f6b10496640 1 -- 172.21.15.100:0/582866681 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6af40b3b90 msgr2=0x7f6af40b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:32:41.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:41.201+0000 7f6b10496640 1 --2- 172.21.15.100:0/582866681 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6af40b3b90 0x7f6af40b60a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6afc009930 tx=0x7f6afc008040 comp rx=0 tx=0).stop 2024-07-26T13:32:41.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:41.201+0000 7f6b10496640 1 -- 172.21.15.100:0/582866681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b081055a0 msgr2=0x7f6b0819f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:32:41.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:41.201+0000 7f6b10496640 1 --2- 172.21.15.100:0/582866681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b081055a0 0x7f6b0819f8d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6b0000b970 tx=0x7f6b0000be30 comp rx=0 tx=0).stop 2024-07-26T13:32:41.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:41.201+0000 7f6b10496640 1 -- 172.21.15.100:0/582866681 shutdown_connections 2024-07-26T13:32:41.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:41.201+0000 7f6b10496640 1 --2- 172.21.15.100:0/582866681 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6af40b3b90 0x7f6af40b60a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:32:41.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:41.201+0000 7f6b10496640 1 --2- 172.21.15.100:0/582866681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b081055a0 0x7f6b0819f8d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:32:41.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:41.201+0000 7f6b10496640 1 --2- 172.21.15.100:0/582866681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b08104c50 0x7f6b0819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:32:41.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:41.201+0000 7f6b10496640 1 --2- 172.21.15.100:0/582866681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b08102c40 0x7f6b0819a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:32:41.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:41.201+0000 7f6b10496640 1 -- 172.21.15.100:0/582866681 >> 172.21.15.100:0/582866681 conn(0x7f6b080798f0 msgr2=0x7f6b08105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:32:41.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:41.201+0000 7f6b10496640 1 -- 172.21.15.100:0/582866681 shutdown_connections 2024-07-26T13:32:41.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:32:41.201+0000 7f6b10496640 1 -- 172.21.15.100:0/582866681 wait complete. 2024-07-26T13:32:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:41 smithi100 ceph-mon[32599]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-26T13:32:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:41.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:41 smithi138 ceph-mon[36954]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-26T13:32:41.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:41.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:41.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:41 smithi167 ceph-mon[37140]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-26T13:32:41.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:41.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:41.813 DEBUG:teuthology.orchestra.run.smithi100:osd.3> sudo journalctl -f -n 0 -u ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@osd.3.service 2024-07-26T13:32:41.815 INFO:tasks.cephadm:Deploying osd.4 on smithi138 with /dev/vg_nvme/lv_3... 2024-07-26T13:32:41.815 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-07-26T13:32:42.076 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.b/config 2024-07-26T13:32:42.243 INFO:journalctl@ceph.osd.3.smithi100.stdout:Jul 26 13:32:41 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-osd-3[58682]: 2024-07-26T13:32:41.941+0000 7f515de19640 -1 osd.3 0 log_to_monitors true 2024-07-26T13:32:42.905 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:42 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:42.906 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:42 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:42.906 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:42 smithi100 ceph-mon[32599]: from='osd.3 [v2:172.21.15.100:6810/3626762419,v1:172.21.15.100:6811/3626762419]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-07-26T13:32:42.906 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:42 smithi100 ceph-mon[32599]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-26T13:32:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:42 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:43.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:42 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:43.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:42 smithi138 ceph-mon[36954]: from='osd.3 [v2:172.21.15.100:6810/3626762419,v1:172.21.15.100:6811/3626762419]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-07-26T13:32:43.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:42 smithi138 ceph-mon[36954]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-26T13:32:43.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:42 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:43.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:42 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:43.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:42 smithi167 ceph-mon[37140]: from='osd.3 [v2:172.21.15.100:6810/3626762419,v1:172.21.15.100:6811/3626762419]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-07-26T13:32:43.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:42 smithi167 ceph-mon[37140]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-26T13:32:43.868 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-07-26T13:32:43.959 INFO:teuthology.orchestra.run.smithi138.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-07-26T13:32:43.973 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10+0 records in 2024-07-26T13:32:43.974 INFO:teuthology.orchestra.run.smithi138.stderr:10+0 records out 2024-07-26T13:32:43.974 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125051 s, 839 MB/s 2024-07-26T13:32:43.974 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping successful for: 2024-07-26T13:32:43.979 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:43 smithi100 ceph-mon[32599]: from='osd.3 [v2:172.21.15.100:6810/3626762419,v1:172.21.15.100:6811/3626762419]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-07-26T13:32:43.979 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:43 smithi100 ceph-mon[32599]: osdmap e24: 4 total, 3 up, 4 in 2024-07-26T13:32:43.979 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:43 smithi100 ceph-mon[32599]: from='osd.3 [v2:172.21.15.100:6810/3626762419,v1:172.21.15.100:6811/3626762419]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-07-26T13:32:43.979 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:43 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-26T13:32:44.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:43 smithi138 ceph-mon[36954]: from='osd.3 [v2:172.21.15.100:6810/3626762419,v1:172.21.15.100:6811/3626762419]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-07-26T13:32:44.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:43 smithi138 ceph-mon[36954]: osdmap e24: 4 total, 3 up, 4 in 2024-07-26T13:32:44.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:43 smithi138 ceph-mon[36954]: from='osd.3 [v2:172.21.15.100:6810/3626762419,v1:172.21.15.100:6811/3626762419]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-07-26T13:32:44.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:43 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-26T13:32:44.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:43 smithi167 ceph-mon[37140]: from='osd.3 [v2:172.21.15.100:6810/3626762419,v1:172.21.15.100:6811/3626762419]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-07-26T13:32:44.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:43 smithi167 ceph-mon[37140]: osdmap e24: 4 total, 3 up, 4 in 2024-07-26T13:32:44.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:43 smithi167 ceph-mon[37140]: from='osd.3 [v2:172.21.15.100:6810/3626762419,v1:172.21.15.100:6811/3626762419]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-07-26T13:32:44.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:43 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-26T13:32:44.638 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_3 2024-07-26T13:32:44.721 INFO:journalctl@ceph.osd.3.smithi100.stdout:Jul 26 13:32:44 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-osd-3[58682]: 2024-07-26T13:32:44.346+0000 7f5159d9b640 -1 osd.3 0 waiting for initial osdmap 2024-07-26T13:32:44.721 INFO:journalctl@ceph.osd.3.smithi100.stdout:Jul 26 13:32:44 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-osd-3[58682]: 2024-07-26T13:32:44.355+0000 7f5151b91640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-26T13:32:44.893 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.b/config 2024-07-26T13:32:44.992 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:44 smithi100 ceph-mon[32599]: from='osd.3 [v2:172.21.15.100:6810/3626762419,v1:172.21.15.100:6811/3626762419]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-07-26T13:32:44.992 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:44 smithi100 ceph-mon[32599]: osdmap e25: 4 total, 3 up, 4 in 2024-07-26T13:32:44.992 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-26T13:32:44.992 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-26T13:32:44.992 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:44 smithi100 ceph-mon[32599]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-26T13:32:44.992 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:44 smithi100 ceph-mon[32599]: from='osd.3 [v2:172.21.15.100:6810/3626762419,v1:172.21.15.100:6811/3626762419]' entity='osd.3' 2024-07-26T13:32:44.992 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:44 smithi100 ceph-mon[32599]: osd.3 [v2:172.21.15.100:6810/3626762419,v1:172.21.15.100:6811/3626762419] boot 2024-07-26T13:32:44.992 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:44 smithi100 ceph-mon[32599]: osdmap e26: 4 total, 4 up, 4 in 2024-07-26T13:32:44.992 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-26T13:32:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:44 smithi138 ceph-mon[36954]: from='osd.3 [v2:172.21.15.100:6810/3626762419,v1:172.21.15.100:6811/3626762419]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-07-26T13:32:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:44 smithi138 ceph-mon[36954]: osdmap e25: 4 total, 3 up, 4 in 2024-07-26T13:32:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-26T13:32:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-26T13:32:45.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:44 smithi138 ceph-mon[36954]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-26T13:32:45.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:44 smithi138 ceph-mon[36954]: from='osd.3 [v2:172.21.15.100:6810/3626762419,v1:172.21.15.100:6811/3626762419]' entity='osd.3' 2024-07-26T13:32:45.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:44 smithi138 ceph-mon[36954]: osd.3 [v2:172.21.15.100:6810/3626762419,v1:172.21.15.100:6811/3626762419] boot 2024-07-26T13:32:45.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:44 smithi138 ceph-mon[36954]: osdmap e26: 4 total, 4 up, 4 in 2024-07-26T13:32:45.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-26T13:32:45.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:44 smithi167 ceph-mon[37140]: from='osd.3 [v2:172.21.15.100:6810/3626762419,v1:172.21.15.100:6811/3626762419]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-07-26T13:32:45.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:44 smithi167 ceph-mon[37140]: osdmap e25: 4 total, 3 up, 4 in 2024-07-26T13:32:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-26T13:32:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-26T13:32:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:44 smithi167 ceph-mon[37140]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-07-26T13:32:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:44 smithi167 ceph-mon[37140]: from='osd.3 [v2:172.21.15.100:6810/3626762419,v1:172.21.15.100:6811/3626762419]' entity='osd.3' 2024-07-26T13:32:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:44 smithi167 ceph-mon[37140]: osd.3 [v2:172.21.15.100:6810/3626762419,v1:172.21.15.100:6811/3626762419] boot 2024-07-26T13:32:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:44 smithi167 ceph-mon[37140]: osdmap e26: 4 total, 4 up, 4 in 2024-07-26T13:32:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-26T13:32:46.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:45 smithi138 ceph-mon[36954]: purged_snaps scrub starts 2024-07-26T13:32:46.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:45 smithi138 ceph-mon[36954]: purged_snaps scrub ok 2024-07-26T13:32:46.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:45 smithi138 ceph-mon[36954]: Detected new or changed devices on smithi100 2024-07-26T13:32:46.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:45 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:46.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:45 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:46.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:45 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:32:46.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:45 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:32:46.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:45 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi100 to 8543M 2024-07-26T13:32:46.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:45 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:46.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:45 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:32:46.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:45 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:32:46.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:45 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:46.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:45 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:32:46.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:45 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:32:46.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:45 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:32:46.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:45 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:46.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:45 smithi138 ceph-mon[36954]: osdmap e27: 4 total, 4 up, 4 in 2024-07-26T13:32:46.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:45 smithi100 ceph-mon[32599]: purged_snaps scrub starts 2024-07-26T13:32:46.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:45 smithi100 ceph-mon[32599]: purged_snaps scrub ok 2024-07-26T13:32:46.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:45 smithi100 ceph-mon[32599]: Detected new or changed devices on smithi100 2024-07-26T13:32:46.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:45 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:46.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:45 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:46.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:45 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:32:46.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:45 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:32:46.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:45 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi100 to 8543M 2024-07-26T13:32:46.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:45 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:46.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:45 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:32:46.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:45 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:32:46.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:45 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:46.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:45 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:32:46.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:45 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:32:46.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:45 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:32:46.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:45 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:46.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:45 smithi100 ceph-mon[32599]: osdmap e27: 4 total, 4 up, 4 in 2024-07-26T13:32:46.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:45 smithi167 ceph-mon[37140]: purged_snaps scrub starts 2024-07-26T13:32:46.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:45 smithi167 ceph-mon[37140]: purged_snaps scrub ok 2024-07-26T13:32:46.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:45 smithi167 ceph-mon[37140]: Detected new or changed devices on smithi100 2024-07-26T13:32:46.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:45 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:46.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:45 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:46.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:45 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:32:46.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:45 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:32:46.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:45 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi100 to 8543M 2024-07-26T13:32:46.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:45 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:46.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:45 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:32:46.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:45 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:32:46.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:45 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:46.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:45 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:32:46.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:45 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:32:46.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:45 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:32:46.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:45 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:32:46.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:45 smithi167 ceph-mon[37140]: osdmap e27: 4 total, 4 up, 4 in 2024-07-26T13:32:46.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.434+0000 7fdbd2f9e640 1 -- 172.21.15.138:0/4209344408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbcc109650 msgr2=0x7fdbcc10fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:32:46.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.434+0000 7fdbd2f9e640 1 --2- 172.21.15.138:0/4209344408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbcc109650 0x7fdbcc10fe80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4009900 tx=0x7fdbc402a410 comp rx=0 tx=0).stop 2024-07-26T13:32:46.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.434+0000 7fdbd2f9e640 1 -- 172.21.15.138:0/4209344408 shutdown_connections 2024-07-26T13:32:46.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.434+0000 7fdbd2f9e640 1 --2- 172.21.15.138:0/4209344408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbcc109650 0x7fdbcc10fe80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:32:46.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.434+0000 7fdbd2f9e640 1 --2- 172.21.15.138:0/4209344408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbcc108d00 0x7fdbcc109110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:32:46.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.434+0000 7fdbd2f9e640 1 --2- 172.21.15.138:0/4209344408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbcc1033c0 0x7fdbcc1037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:32:46.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.434+0000 7fdbd2f9e640 1 -- 172.21.15.138:0/4209344408 >> 172.21.15.138:0/4209344408 conn(0x7fdbcc0fee10 msgr2=0x7fdbcc101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:32:46.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.434+0000 7fdbd2f9e640 1 -- 172.21.15.138:0/4209344408 shutdown_connections 2024-07-26T13:32:46.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.434+0000 7fdbd2f9e640 1 -- 172.21.15.138:0/4209344408 wait complete. 2024-07-26T13:32:46.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.435+0000 7fdbd2f9e640 1 Processor -- start 2024-07-26T13:32:46.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.435+0000 7fdbd2f9e640 1 -- start start 2024-07-26T13:32:46.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.435+0000 7fdbd2f9e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbcc1033c0 0x7fdbcc199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:32:46.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.435+0000 7fdbd2f9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbcc108d00 0x7fdbcc19a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:32:46.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.435+0000 7fdbd2f9e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbcc109650 0x7fdbcc19f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:32:46.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.435+0000 7fdbd2f9e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdbcc113320 con 0x7fdbcc108d00 2024-07-26T13:32:46.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.435+0000 7fdbd2f9e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdbcc1131a0 con 0x7fdbcc109650 2024-07-26T13:32:46.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.435+0000 7fdbd2f9e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdbcc1134a0 con 0x7fdbcc1033c0 2024-07-26T13:32:46.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.435+0000 7fdbd279d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbcc109650 0x7fdbcc19f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:32:46.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.435+0000 7fdbd279d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbcc109650 0x7fdbcc19f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42190/0 (socket says 172.21.15.138:42190) 2024-07-26T13:32:46.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.435+0000 7fdbd279d640 1 -- 172.21.15.138:0/698024220 learned_addr learned my addr 172.21.15.138:0/698024220 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-07-26T13:32:46.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.435+0000 7fdbd1f9c640 1 --2- 172.21.15.138:0/698024220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbcc1033c0 0x7fdbcc199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:32:46.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.435+0000 7fdbd179b640 1 --2- 172.21.15.138:0/698024220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbcc108d00 0x7fdbcc19a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:32:46.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.435+0000 7fdbd279d640 1 -- 172.21.15.138:0/698024220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbcc1033c0 msgr2=0x7fdbcc199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:32:46.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.435+0000 7fdbd279d640 1 --2- 172.21.15.138:0/698024220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbcc1033c0 0x7fdbcc199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:32:46.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.435+0000 7fdbd279d640 1 -- 172.21.15.138:0/698024220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbcc108d00 msgr2=0x7fdbcc19a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:32:46.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.435+0000 7fdbd279d640 1 --2- 172.21.15.138:0/698024220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbcc108d00 0x7fdbcc19a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:32:46.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.435+0000 7fdbd279d640 1 -- 172.21.15.138:0/698024220 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbcc19fc20 con 0x7fdbcc109650 2024-07-26T13:32:46.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.435+0000 7fdbd1f9c640 1 --2- 172.21.15.138:0/698024220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbcc1033c0 0x7fdbcc199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:32:46.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.436+0000 7fdbd279d640 1 --2- 172.21.15.138:0/698024220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbcc109650 0x7fdbcc19f4f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4004770 tx=0x7fdbc4004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:32:46.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.436+0000 7fdbc2ffd640 1 -- 172.21.15.138:0/698024220 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc403f030 con 0x7fdbcc109650 2024-07-26T13:32:46.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.436+0000 7fdbc2ffd640 1 -- 172.21.15.138:0/698024220 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbc4003aa0 con 0x7fdbcc109650 2024-07-26T13:32:46.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.436+0000 7fdbc2ffd640 1 -- 172.21.15.138:0/698024220 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4035070 con 0x7fdbcc109650 2024-07-26T13:32:46.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.436+0000 7fdbd2f9e640 1 -- 172.21.15.138:0/698024220 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbcc19feb0 con 0x7fdbcc109650 2024-07-26T13:32:46.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.436+0000 7fdbd2f9e640 1 -- 172.21.15.138:0/698024220 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbcc1a6670 con 0x7fdbcc109650 2024-07-26T13:32:46.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.437+0000 7fdbd2f9e640 1 -- 172.21.15.138:0/698024220 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbcc104640 con 0x7fdbcc109650 2024-07-26T13:32:46.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.438+0000 7fdbc2ffd640 1 -- 172.21.15.138:0/698024220 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7fdbc4042030 con 0x7fdbcc109650 2024-07-26T13:32:46.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.439+0000 7fdbc2ffd640 1 --2- 172.21.15.138:0/698024220 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fdbb80b3c20 0x7fdbb80b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:32:46.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.439+0000 7fdbc2ffd640 1 -- 172.21.15.138:0/698024220 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3884+0+0 (secure 0 0 0) 0x7fdbc4103600 con 0x7fdbcc109650 2024-07-26T13:32:46.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.439+0000 7fdbd1f9c640 1 --2- 172.21.15.138:0/698024220 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fdbb80b3c20 0x7fdbb80b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:32:46.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.440+0000 7fdbd1f9c640 1 --2- 172.21.15.138:0/698024220 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fdbb80b3c20 0x7fdbb80b6130 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8005fd0 tx=0x7fdbc8009600 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:32:46.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.443+0000 7fdbc2ffd640 1 -- 172.21.15.138:0/698024220 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fdbc40cbd00 con 0x7fdbcc109650 2024-07-26T13:32:46.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:32:46.624+0000 7fdbd2f9e640 1 -- 172.21.15.138:0/698024220 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fdbcc063230 con 0x7fdbb80b3c20 2024-07-26T13:32:47.183 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:46 smithi138 ceph-mon[36954]: pgmap v90: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-07-26T13:32:47.183 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-26T13:32:47.183 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-26T13:32:47.184 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:32:47.184 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:46 smithi138 ceph-mon[36954]: osdmap e28: 4 total, 4 up, 4 in 2024-07-26T13:32:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:46 smithi100 ceph-mon[32599]: pgmap v90: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-07-26T13:32:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-26T13:32:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-26T13:32:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:32:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:46 smithi100 ceph-mon[32599]: osdmap e28: 4 total, 4 up, 4 in 2024-07-26T13:32:47.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:46 smithi167 ceph-mon[37140]: pgmap v90: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-07-26T13:32:47.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-26T13:32:47.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-26T13:32:47.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:32:47.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:46 smithi167 ceph-mon[37140]: osdmap e28: 4 total, 4 up, 4 in 2024-07-26T13:32:48.066 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:47 smithi138 ceph-mon[36954]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:32:48.188 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:47 smithi167 ceph-mon[37140]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:32:48.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:47 smithi100 ceph-mon[32599]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:32:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:48 smithi138 ceph-mon[36954]: pgmap v92: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-07-26T13:32:49.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:48 smithi100 ceph-mon[32599]: pgmap v92: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-07-26T13:32:49.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:48 smithi167 ceph-mon[37140]: pgmap v92: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-07-26T13:32:50.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:49 smithi100 ceph-mon[32599]: from='client.? 172.21.15.138:0/4054113687' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "520554cc-e846-46f9-af77-05a1de65a452"}]: dispatch 2024-07-26T13:32:50.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:49 smithi100 ceph-mon[32599]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "520554cc-e846-46f9-af77-05a1de65a452"}]: dispatch 2024-07-26T13:32:50.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:49 smithi100 ceph-mon[32599]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "520554cc-e846-46f9-af77-05a1de65a452"}]': finished 2024-07-26T13:32:50.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:49 smithi100 ceph-mon[32599]: osdmap e29: 5 total, 4 up, 5 in 2024-07-26T13:32:50.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:49 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-26T13:32:50.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:49 smithi100 ceph-mon[32599]: from='client.? 172.21.15.138:0/433531297' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-26T13:32:50.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:49 smithi167 ceph-mon[37140]: from='client.? 172.21.15.138:0/4054113687' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "520554cc-e846-46f9-af77-05a1de65a452"}]: dispatch 2024-07-26T13:32:50.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:49 smithi167 ceph-mon[37140]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "520554cc-e846-46f9-af77-05a1de65a452"}]: dispatch 2024-07-26T13:32:50.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:49 smithi167 ceph-mon[37140]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "520554cc-e846-46f9-af77-05a1de65a452"}]': finished 2024-07-26T13:32:50.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:49 smithi167 ceph-mon[37140]: osdmap e29: 5 total, 4 up, 5 in 2024-07-26T13:32:50.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:49 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-26T13:32:50.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:49 smithi167 ceph-mon[37140]: from='client.? 172.21.15.138:0/433531297' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-26T13:32:50.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:49 smithi138 ceph-mon[36954]: from='client.? 172.21.15.138:0/4054113687' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "520554cc-e846-46f9-af77-05a1de65a452"}]: dispatch 2024-07-26T13:32:50.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:49 smithi138 ceph-mon[36954]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "520554cc-e846-46f9-af77-05a1de65a452"}]: dispatch 2024-07-26T13:32:50.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:49 smithi138 ceph-mon[36954]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "520554cc-e846-46f9-af77-05a1de65a452"}]': finished 2024-07-26T13:32:50.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:49 smithi138 ceph-mon[36954]: osdmap e29: 5 total, 4 up, 5 in 2024-07-26T13:32:50.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:49 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-26T13:32:50.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:49 smithi138 ceph-mon[36954]: from='client.? 172.21.15.138:0/433531297' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-26T13:32:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:50 smithi100 ceph-mon[32599]: pgmap v94: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-07-26T13:32:51.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:50 smithi167 ceph-mon[37140]: pgmap v94: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-07-26T13:32:51.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:50 smithi138 ceph-mon[36954]: pgmap v94: 1 pgs: 1 remapped+peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-07-26T13:32:53.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:53 smithi138 ceph-mon[36954]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-07-26T13:32:53.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:53 smithi100 ceph-mon[32599]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-07-26T13:32:53.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:53 smithi167 ceph-mon[37140]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-07-26T13:32:55.412 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:55 smithi138 ceph-mon[36954]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-26T13:32:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:55 smithi100 ceph-mon[32599]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-26T13:32:55.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:55 smithi167 ceph-mon[37140]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-26T13:32:56.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-07-26T13:32:56.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:32:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-07-26T13:32:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:32:56.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-07-26T13:32:56.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:32:57.631 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:57 smithi138 ceph-mon[36954]: Deploying daemon osd.4 on smithi138 2024-07-26T13:32:57.631 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:57 smithi138 ceph-mon[36954]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-07-26T13:32:57.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:57 smithi100 ceph-mon[32599]: Deploying daemon osd.4 on smithi138 2024-07-26T13:32:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:57 smithi100 ceph-mon[32599]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-07-26T13:32:57.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:57 smithi167 ceph-mon[37140]: Deploying daemon osd.4 on smithi138 2024-07-26T13:32:57.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:57 smithi167 ceph-mon[37140]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-07-26T13:32:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:32:59 smithi138 ceph-mon[36954]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-26T13:32:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:32:59 smithi100 ceph-mon[32599]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-26T13:32:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:32:59 smithi167 ceph-mon[37140]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-26T13:33:01.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:01 smithi138 ceph-mon[36954]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-07-26T13:33:01.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:01 smithi100 ceph-mon[32599]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-07-26T13:33:01.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:01 smithi167 ceph-mon[37140]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-07-26T13:33:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:33:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:02.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:33:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:02.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:33:02.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:02.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:03.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:03 smithi100 ceph-mon[32599]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-07-26T13:33:03.757 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:03 smithi138 ceph-mon[36954]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-07-26T13:33:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:03 smithi167 ceph-mon[37140]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-07-26T13:33:04.481 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 4 on host 'smithi138' 2024-07-26T13:33:04.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:04.479+0000 7fdbc2ffd640 1 -- 172.21.15.138:0/698024220 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdbcc063230 con 0x7fdbb80b3c20 2024-07-26T13:33:04.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:04.482+0000 7fdbd2f9e640 1 -- 172.21.15.138:0/698024220 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fdbb80b3c20 msgr2=0x7fdbb80b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:33:04.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:04.482+0000 7fdbd2f9e640 1 --2- 172.21.15.138:0/698024220 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fdbb80b3c20 0x7fdbb80b6130 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8005fd0 tx=0x7fdbc8009600 comp rx=0 tx=0).stop 2024-07-26T13:33:04.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:04.482+0000 7fdbd2f9e640 1 -- 172.21.15.138:0/698024220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbcc109650 msgr2=0x7fdbcc19f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:33:04.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:04.482+0000 7fdbd2f9e640 1 --2- 172.21.15.138:0/698024220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbcc109650 0x7fdbcc19f4f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4004770 tx=0x7fdbc4004270 comp rx=0 tx=0).stop 2024-07-26T13:33:04.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:04.482+0000 7fdbd2f9e640 1 -- 172.21.15.138:0/698024220 shutdown_connections 2024-07-26T13:33:04.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:04.482+0000 7fdbd2f9e640 1 --2- 172.21.15.138:0/698024220 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fdbb80b3c20 0x7fdbb80b6130 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:33:04.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:04.482+0000 7fdbd2f9e640 1 --2- 172.21.15.138:0/698024220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbcc109650 0x7fdbcc19f4f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:33:04.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:04.482+0000 7fdbd2f9e640 1 --2- 172.21.15.138:0/698024220 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbcc108d00 0x7fdbcc19a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:33:04.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:04.482+0000 7fdbd2f9e640 1 --2- 172.21.15.138:0/698024220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbcc1033c0 0x7fdbcc199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:33:04.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:04.482+0000 7fdbd2f9e640 1 -- 172.21.15.138:0/698024220 >> 172.21.15.138:0/698024220 conn(0x7fdbcc0fee10 msgr2=0x7fdbcc101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:33:04.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:04.482+0000 7fdbd2f9e640 1 -- 172.21.15.138:0/698024220 shutdown_connections 2024-07-26T13:33:04.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:04.482+0000 7fdbd2f9e640 1 -- 172.21.15.138:0/698024220 wait complete. 2024-07-26T13:33:05.141 DEBUG:teuthology.orchestra.run.smithi138:osd.4> sudo journalctl -f -n 0 -u ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@osd.4.service 2024-07-26T13:33:05.144 INFO:tasks.cephadm:Deploying osd.5 on smithi167 with /dev/vg_nvme/lv_3... 2024-07-26T13:33:05.144 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-07-26T13:33:05.406 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.c/config 2024-07-26T13:33:05.440 INFO:journalctl@ceph.osd.4.smithi138.stdout:Jul 26 13:33:05 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-osd-4[49673]: 2024-07-26T13:33:05.330+0000 7ff13be0a640 -1 osd.4 0 log_to_monitors true 2024-07-26T13:33:05.741 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:05 smithi167 ceph-mon[37140]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-07-26T13:33:05.741 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:05 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:05.741 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:05 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:05.741 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:05 smithi167 ceph-mon[37140]: from='osd.4 [v2:172.21.15.138:6808/3757334356,v1:172.21.15.138:6809/3757334356]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-26T13:33:05.741 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:05 smithi167 ceph-mon[37140]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-26T13:33:05.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:05 smithi100 ceph-mon[32599]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-07-26T13:33:05.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:05 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:05.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:05 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:05.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:05 smithi100 ceph-mon[32599]: from='osd.4 [v2:172.21.15.138:6808/3757334356,v1:172.21.15.138:6809/3757334356]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-26T13:33:05.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:05 smithi100 ceph-mon[32599]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-26T13:33:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:05 smithi138 ceph-mon[36954]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-07-26T13:33:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:05 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:05 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:05 smithi138 ceph-mon[36954]: from='osd.4 [v2:172.21.15.138:6808/3757334356,v1:172.21.15.138:6809/3757334356]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-26T13:33:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:05 smithi138 ceph-mon[36954]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-26T13:33:06.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:06 smithi138 ceph-mon[36954]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-07-26T13:33:06.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:06 smithi138 ceph-mon[36954]: osdmap e30: 5 total, 4 up, 5 in 2024-07-26T13:33:06.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-26T13:33:06.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:06 smithi138 ceph-mon[36954]: from='osd.4 [v2:172.21.15.138:6808/3757334356,v1:172.21.15.138:6809/3757334356]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-07-26T13:33:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:06 smithi138 ceph-mon[36954]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-07-26T13:33:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:06 smithi138 ceph-mon[36954]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-07-26T13:33:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:06 smithi100 ceph-mon[32599]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-07-26T13:33:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:06 smithi100 ceph-mon[32599]: osdmap e30: 5 total, 4 up, 5 in 2024-07-26T13:33:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-26T13:33:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:06 smithi100 ceph-mon[32599]: from='osd.4 [v2:172.21.15.138:6808/3757334356,v1:172.21.15.138:6809/3757334356]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-07-26T13:33:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:06 smithi100 ceph-mon[32599]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-07-26T13:33:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:06 smithi100 ceph-mon[32599]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-07-26T13:33:06.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:06 smithi167 ceph-mon[37140]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-07-26T13:33:06.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:06 smithi167 ceph-mon[37140]: osdmap e30: 5 total, 4 up, 5 in 2024-07-26T13:33:06.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-26T13:33:06.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:06 smithi167 ceph-mon[37140]: from='osd.4 [v2:172.21.15.138:6808/3757334356,v1:172.21.15.138:6809/3757334356]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-07-26T13:33:06.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:06 smithi167 ceph-mon[37140]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-07-26T13:33:06.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:06.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:06.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:06 smithi167 ceph-mon[37140]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-07-26T13:33:07.160 INFO:journalctl@ceph.osd.4.smithi138.stdout:Jul 26 13:33:06 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-osd-4[49673]: 2024-07-26T13:33:06.931+0000 7ff137d8c640 -1 osd.4 0 waiting for initial osdmap 2024-07-26T13:33:07.161 INFO:journalctl@ceph.osd.4.smithi138.stdout:Jul 26 13:33:06 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-osd-4[49673]: 2024-07-26T13:33:06.937+0000 7ff12fb82640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-26T13:33:07.282 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-07-26T13:33:07.378 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-07-26T13:33:07.391 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2024-07-26T13:33:07.391 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2024-07-26T13:33:07.391 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114465 s, 916 MB/s 2024-07-26T13:33:07.391 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2024-07-26T13:33:07.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:07 smithi138 ceph-mon[36954]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-07-26T13:33:07.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:07 smithi138 ceph-mon[36954]: osdmap e31: 5 total, 4 up, 5 in 2024-07-26T13:33:07.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-26T13:33:07.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-26T13:33:07.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:07 smithi138 ceph-mon[36954]: from='osd.4 ' entity='osd.4' 2024-07-26T13:33:07.710 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:07 smithi167 ceph-mon[37140]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-07-26T13:33:07.710 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:07 smithi167 ceph-mon[37140]: osdmap e31: 5 total, 4 up, 5 in 2024-07-26T13:33:07.711 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-26T13:33:07.711 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-26T13:33:07.711 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:07 smithi167 ceph-mon[37140]: from='osd.4 ' entity='osd.4' 2024-07-26T13:33:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:07 smithi100 ceph-mon[32599]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-07-26T13:33:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:07 smithi100 ceph-mon[32599]: osdmap e31: 5 total, 4 up, 5 in 2024-07-26T13:33:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-26T13:33:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-26T13:33:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:07 smithi100 ceph-mon[32599]: from='osd.4 ' entity='osd.4' 2024-07-26T13:33:07.998 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_3 2024-07-26T13:33:08.249 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.c/config 2024-07-26T13:33:08.614 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:08 smithi167 ceph-mon[37140]: purged_snaps scrub starts 2024-07-26T13:33:08.614 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:08 smithi167 ceph-mon[37140]: purged_snaps scrub ok 2024-07-26T13:33:08.614 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-26T13:33:08.614 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:08 smithi167 ceph-mon[37140]: osd.4 [v2:172.21.15.138:6808/3757334356,v1:172.21.15.138:6809/3757334356] boot 2024-07-26T13:33:08.614 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:08 smithi167 ceph-mon[37140]: osdmap e32: 5 total, 5 up, 5 in 2024-07-26T13:33:08.614 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-26T13:33:08.614 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:08 smithi167 ceph-mon[37140]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-07-26T13:33:08.669 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:08 smithi138 ceph-mon[36954]: purged_snaps scrub starts 2024-07-26T13:33:08.669 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:08 smithi138 ceph-mon[36954]: purged_snaps scrub ok 2024-07-26T13:33:08.669 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-26T13:33:08.669 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:08 smithi138 ceph-mon[36954]: osd.4 [v2:172.21.15.138:6808/3757334356,v1:172.21.15.138:6809/3757334356] boot 2024-07-26T13:33:08.669 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:08 smithi138 ceph-mon[36954]: osdmap e32: 5 total, 5 up, 5 in 2024-07-26T13:33:08.669 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-26T13:33:08.669 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:08 smithi138 ceph-mon[36954]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-07-26T13:33:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:08 smithi100 ceph-mon[32599]: purged_snaps scrub starts 2024-07-26T13:33:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:08 smithi100 ceph-mon[32599]: purged_snaps scrub ok 2024-07-26T13:33:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-26T13:33:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:08 smithi100 ceph-mon[32599]: osd.4 [v2:172.21.15.138:6808/3757334356,v1:172.21.15.138:6809/3757334356] boot 2024-07-26T13:33:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:08 smithi100 ceph-mon[32599]: osdmap e32: 5 total, 5 up, 5 in 2024-07-26T13:33:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-26T13:33:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:08 smithi100 ceph-mon[32599]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-07-26T13:33:09.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.802+0000 7f09ed59f640 1 -- 172.21.15.167:0/1160642426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09e8108560 msgr2=0x7f09e810ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:33:09.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.802+0000 7f09ed59f640 1 --2- 172.21.15.167:0/1160642426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09e8108560 0x7f09e810ed90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f09e0009ca0 tx=0x7f09e002a8c0 comp rx=0 tx=0).stop 2024-07-26T13:33:09.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.802+0000 7f09ed59f640 1 -- 172.21.15.167:0/1160642426 shutdown_connections 2024-07-26T13:33:09.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.802+0000 7f09ed59f640 1 --2- 172.21.15.167:0/1160642426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09e8108560 0x7f09e810ed90 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:33:09.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.802+0000 7f09ed59f640 1 --2- 172.21.15.167:0/1160642426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09e8107c10 0x7f09e8108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:33:09.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.802+0000 7f09ed59f640 1 --2- 172.21.15.167:0/1160642426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09e81022d0 0x7f09e81026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:33:09.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.802+0000 7f09ed59f640 1 -- 172.21.15.167:0/1160642426 >> 172.21.15.167:0/1160642426 conn(0x7f09e80fdd60 msgr2=0x7f09e81001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:33:09.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.802+0000 7f09ed59f640 1 -- 172.21.15.167:0/1160642426 shutdown_connections 2024-07-26T13:33:09.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.802+0000 7f09ed59f640 1 -- 172.21.15.167:0/1160642426 wait complete. 2024-07-26T13:33:09.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.803+0000 7f09ed59f640 1 Processor -- start 2024-07-26T13:33:09.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.803+0000 7f09ed59f640 1 -- start start 2024-07-26T13:33:09.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.803+0000 7f09ed59f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09e81022d0 0x7f09e819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:33:09.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.803+0000 7f09ed59f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09e8107c10 0x7f09e819b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:33:09.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.803+0000 7f09ed59f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09e8108560 0x7f09e819b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:33:09.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.803+0000 7f09ed59f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f09e8112210 con 0x7f09e81022d0 2024-07-26T13:33:09.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.803+0000 7f09ed59f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f09e8112090 con 0x7f09e8108560 2024-07-26T13:33:09.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.803+0000 7f09ed59f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f09e8112390 con 0x7f09e8107c10 2024-07-26T13:33:09.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.803+0000 7f09e77fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09e8107c10 0x7f09e819b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:33:09.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.803+0000 7f09e77fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09e8107c10 0x7f09e819b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35952/0 (socket says 172.21.15.167:35952) 2024-07-26T13:33:09.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.803+0000 7f09e77fe640 1 -- 172.21.15.167:0/3066790959 learned_addr learned my addr 172.21.15.167:0/3066790959 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-07-26T13:33:09.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.803+0000 7f09e7fff640 1 --2- 172.21.15.167:0/3066790959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09e81022d0 0x7f09e819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:33:09.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.804+0000 7f09ecd9e640 1 --2- 172.21.15.167:0/3066790959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09e8108560 0x7f09e819b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:33:09.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.804+0000 7f09e77fe640 1 -- 172.21.15.167:0/3066790959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09e8108560 msgr2=0x7f09e819b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:33:09.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.804+0000 7f09e77fe640 1 --2- 172.21.15.167:0/3066790959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09e8108560 0x7f09e819b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:33:09.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.804+0000 7f09e77fe640 1 -- 172.21.15.167:0/3066790959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09e81022d0 msgr2=0x7f09e819aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:33:09.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.804+0000 7f09e77fe640 1 --2- 172.21.15.167:0/3066790959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09e81022d0 0x7f09e819aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:33:09.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.804+0000 7f09e77fe640 1 -- 172.21.15.167:0/3066790959 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e81a5d80 con 0x7f09e8107c10 2024-07-26T13:33:09.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.804+0000 7f09e77fe640 1 --2- 172.21.15.167:0/3066790959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09e8107c10 0x7f09e819b000 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00d740 tx=0x7f09dc00dc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:33:09.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.804+0000 7f09e57fa640 1 -- 172.21.15.167:0/3066790959 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc017070 con 0x7f09e8107c10 2024-07-26T13:33:09.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.804+0000 7f09ed59f640 1 -- 172.21.15.167:0/3066790959 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e81a6070 con 0x7f09e8107c10 2024-07-26T13:33:09.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.804+0000 7f09e57fa640 1 -- 172.21.15.167:0/3066790959 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09dc00de50 con 0x7f09e8107c10 2024-07-26T13:33:09.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.804+0000 7f09e57fa640 1 -- 172.21.15.167:0/3066790959 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc00f620 con 0x7f09e8107c10 2024-07-26T13:33:09.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.804+0000 7f09ed59f640 1 -- 172.21.15.167:0/3066790959 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e81a65b0 con 0x7f09e8107c10 2024-07-26T13:33:09.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.805+0000 7f09ed59f640 1 -- 172.21.15.167:0/3066790959 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09e81026e0 con 0x7f09e8107c10 2024-07-26T13:33:09.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.807+0000 7f09e57fa640 1 -- 172.21.15.167:0/3066790959 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f09dc01a070 con 0x7f09e8107c10 2024-07-26T13:33:09.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.807+0000 7f09e57fa640 1 --2- 172.21.15.167:0/3066790959 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f09d00b3ca0 0x7f09d00b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:33:09.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.808+0000 7f09e57fa640 1 -- 172.21.15.167:0/3066790959 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4316+0+0 (secure 0 0 0) 0x7f09dc0e26e0 con 0x7f09e8107c10 2024-07-26T13:33:09.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.808+0000 7f09e7fff640 1 --2- 172.21.15.167:0/3066790959 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f09d00b3ca0 0x7f09d00b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:33:09.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.808+0000 7f09e7fff640 1 --2- 172.21.15.167:0/3066790959 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f09d00b3ca0 0x7f09d00b61b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f09e806bf50 tx=0x7f09d800b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:33:09.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.811+0000 7f09e57fa640 1 -- 172.21.15.167:0/3066790959 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f09dc0aac30 con 0x7f09e8107c10 2024-07-26T13:33:09.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:09 smithi100 ceph-mon[32599]: Detected new or changed devices on smithi138 2024-07-26T13:33:09.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:09.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:09.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:33:09.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:33:09.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:09 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi138 to 8544M 2024-07-26T13:33:09.997 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:09.997 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:33:09.998 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:33:09.998 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:09.998 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:33:09.998 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:33:09.998 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:33:09.998 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:09.998 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:09 smithi100 ceph-mon[32599]: osdmap e33: 5 total, 5 up, 5 in 2024-07-26T13:33:09.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:09.993+0000 7f09ed59f640 1 -- 172.21.15.167:0/3066790959 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f09e8063230 con 0x7f09d00b3ca0 2024-07-26T13:33:09.999 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:09 smithi167 ceph-mon[37140]: Detected new or changed devices on smithi138 2024-07-26T13:33:09.999 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:09.999 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:09.999 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:33:09.999 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:33:09.999 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:09 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi138 to 8544M 2024-07-26T13:33:09.999 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:09.999 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:33:09.999 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:33:09.999 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:09.999 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:33:10.000 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:33:10.000 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:33:10.000 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:10.000 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:09 smithi167 ceph-mon[37140]: osdmap e33: 5 total, 5 up, 5 in 2024-07-26T13:33:10.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:09 smithi138 ceph-mon[36954]: Detected new or changed devices on smithi138 2024-07-26T13:33:10.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:10.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:10.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:33:10.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:33:10.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:09 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi138 to 8544M 2024-07-26T13:33:10.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:10.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:33:10.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:33:10.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:10.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:33:10.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:33:10.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:33:10.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:10.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:09 smithi138 ceph-mon[36954]: osdmap e33: 5 total, 5 up, 5 in 2024-07-26T13:33:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:10 smithi100 ceph-mon[32599]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:33:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:10 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-26T13:33:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:10 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-26T13:33:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:10 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:33:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:10 smithi100 ceph-mon[32599]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-07-26T13:33:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:10 smithi167 ceph-mon[37140]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:33:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:10 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-26T13:33:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:10 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-26T13:33:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:10 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:33:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:10 smithi167 ceph-mon[37140]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-07-26T13:33:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:10 smithi138 ceph-mon[36954]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:33:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:10 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-26T13:33:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:10 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-26T13:33:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:10 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:33:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:10 smithi138 ceph-mon[36954]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-07-26T13:33:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:13 smithi138 ceph-mon[36954]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-07-26T13:33:13.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:13 smithi138 ceph-mon[36954]: from='client.? 172.21.15.167:0/1958288458' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68dc8ad6-31d2-45b5-9aa7-62c4218d46e2"}]: dispatch 2024-07-26T13:33:13.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:13 smithi138 ceph-mon[36954]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68dc8ad6-31d2-45b5-9aa7-62c4218d46e2"}]: dispatch 2024-07-26T13:33:13.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:13 smithi138 ceph-mon[36954]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68dc8ad6-31d2-45b5-9aa7-62c4218d46e2"}]': finished 2024-07-26T13:33:13.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:13 smithi138 ceph-mon[36954]: osdmap e34: 6 total, 5 up, 6 in 2024-07-26T13:33:13.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-26T13:33:13.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:13 smithi138 ceph-mon[36954]: from='client.? 172.21.15.167:0/34529176' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-26T13:33:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:13 smithi100 ceph-mon[32599]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-07-26T13:33:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:13 smithi100 ceph-mon[32599]: from='client.? 172.21.15.167:0/1958288458' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68dc8ad6-31d2-45b5-9aa7-62c4218d46e2"}]: dispatch 2024-07-26T13:33:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:13 smithi100 ceph-mon[32599]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68dc8ad6-31d2-45b5-9aa7-62c4218d46e2"}]: dispatch 2024-07-26T13:33:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:13 smithi100 ceph-mon[32599]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68dc8ad6-31d2-45b5-9aa7-62c4218d46e2"}]': finished 2024-07-26T13:33:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:13 smithi100 ceph-mon[32599]: osdmap e34: 6 total, 5 up, 6 in 2024-07-26T13:33:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-26T13:33:13.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:13 smithi100 ceph-mon[32599]: from='client.? 172.21.15.167:0/34529176' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-26T13:33:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:13 smithi167 ceph-mon[37140]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-07-26T13:33:13.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:13 smithi167 ceph-mon[37140]: from='client.? 172.21.15.167:0/1958288458' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68dc8ad6-31d2-45b5-9aa7-62c4218d46e2"}]: dispatch 2024-07-26T13:33:13.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:13 smithi167 ceph-mon[37140]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68dc8ad6-31d2-45b5-9aa7-62c4218d46e2"}]: dispatch 2024-07-26T13:33:13.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:13 smithi167 ceph-mon[37140]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68dc8ad6-31d2-45b5-9aa7-62c4218d46e2"}]': finished 2024-07-26T13:33:13.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:13 smithi167 ceph-mon[37140]: osdmap e34: 6 total, 5 up, 6 in 2024-07-26T13:33:13.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-26T13:33:13.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:13 smithi167 ceph-mon[37140]: from='client.? 172.21.15.167:0/34529176' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-26T13:33:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:15 smithi138 ceph-mon[36954]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-07-26T13:33:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:15 smithi100 ceph-mon[32599]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-07-26T13:33:15.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:15 smithi167 ceph-mon[37140]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-07-26T13:33:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:17 smithi138 ceph-mon[36954]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-07-26T13:33:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:17 smithi100 ceph-mon[32599]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-07-26T13:33:17.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:17 smithi167 ceph-mon[37140]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-07-26T13:33:19.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:19 smithi167 ceph-mon[37140]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-07-26T13:33:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-07-26T13:33:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:33:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:19 smithi138 ceph-mon[36954]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-07-26T13:33:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-07-26T13:33:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:33:19.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:19 smithi100 ceph-mon[32599]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-07-26T13:33:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-07-26T13:33:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:33:20.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:20 smithi138 ceph-mon[36954]: Deploying daemon osd.5 on smithi167 2024-07-26T13:33:20.693 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:20 smithi167 ceph-mon[37140]: Deploying daemon osd.5 on smithi167 2024-07-26T13:33:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:20 smithi100 ceph-mon[32599]: Deploying daemon osd.5 on smithi167 2024-07-26T13:33:21.516 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:21 smithi167 ceph-mon[37140]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-07-26T13:33:21.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:21 smithi138 ceph-mon[36954]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-07-26T13:33:21.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:21 smithi100 ceph-mon[32599]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-07-26T13:33:23.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:23 smithi167 ceph-mon[37140]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-07-26T13:33:23.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:23 smithi138 ceph-mon[36954]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-07-26T13:33:23.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:23 smithi100 ceph-mon[32599]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-07-26T13:33:25.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:25 smithi167 ceph-mon[37140]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-07-26T13:33:25.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:33:25.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:25.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:25 smithi138 ceph-mon[36954]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-07-26T13:33:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:33:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:25 smithi100 ceph-mon[32599]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-07-26T13:33:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:33:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:27 smithi138 ceph-mon[36954]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-07-26T13:33:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:27.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:33:27.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:33:27.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:27.707 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:27 smithi167 ceph-mon[37140]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-07-26T13:33:27.707 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:27.708 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:27.708 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:33:27.708 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:33:27.708 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:27 smithi100 ceph-mon[32599]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-07-26T13:33:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:33:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:33:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:28.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:28.049+0000 7f09e57fa640 1 -- 172.21.15.167:0/3066790959 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f09e8063230 con 0x7f09d00b3ca0 2024-07-26T13:33:28.050 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 5 on host 'smithi167' 2024-07-26T13:33:28.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:28.051+0000 7f09ed59f640 1 -- 172.21.15.167:0/3066790959 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f09d00b3ca0 msgr2=0x7f09d00b61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:33:28.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:28.051+0000 7f09ed59f640 1 --2- 172.21.15.167:0/3066790959 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f09d00b3ca0 0x7f09d00b61b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f09e806bf50 tx=0x7f09d800b410 comp rx=0 tx=0).stop 2024-07-26T13:33:28.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:28.051+0000 7f09ed59f640 1 -- 172.21.15.167:0/3066790959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09e8107c10 msgr2=0x7f09e819b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:33:28.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:28.051+0000 7f09ed59f640 1 --2- 172.21.15.167:0/3066790959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09e8107c10 0x7f09e819b000 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00d740 tx=0x7f09dc00dc00 comp rx=0 tx=0).stop 2024-07-26T13:33:28.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:28.051+0000 7f09ed59f640 1 -- 172.21.15.167:0/3066790959 shutdown_connections 2024-07-26T13:33:28.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:28.051+0000 7f09ed59f640 1 --2- 172.21.15.167:0/3066790959 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f09d00b3ca0 0x7f09d00b61b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:33:28.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:28.051+0000 7f09ed59f640 1 --2- 172.21.15.167:0/3066790959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09e8108560 0x7f09e819b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:33:28.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:28.051+0000 7f09ed59f640 1 --2- 172.21.15.167:0/3066790959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09e8107c10 0x7f09e819b000 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:33:28.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:28.051+0000 7f09ed59f640 1 --2- 172.21.15.167:0/3066790959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09e81022d0 0x7f09e819aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:33:28.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:28.052+0000 7f09ed59f640 1 -- 172.21.15.167:0/3066790959 >> 172.21.15.167:0/3066790959 conn(0x7f09e80fdd60 msgr2=0x7f09e8107270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:33:28.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:28.052+0000 7f09ed59f640 1 -- 172.21.15.167:0/3066790959 shutdown_connections 2024-07-26T13:33:28.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:33:28.052+0000 7f09ed59f640 1 -- 172.21.15.167:0/3066790959 wait complete. 2024-07-26T13:33:28.662 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:33:28.662 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:28.662 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:28.662 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:28 smithi167 ceph-mon[37140]: from='osd.5 [v2:172.21.15.167:6808/3239482836,v1:172.21.15.167:6809/3239482836]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-26T13:33:28.662 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:28 smithi167 ceph-mon[37140]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-26T13:33:28.689 DEBUG:teuthology.orchestra.run.smithi167:osd.5> sudo journalctl -f -n 0 -u ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@osd.5.service 2024-07-26T13:33:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:33:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:28 smithi138 ceph-mon[36954]: from='osd.5 [v2:172.21.15.167:6808/3239482836,v1:172.21.15.167:6809/3239482836]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-26T13:33:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:28 smithi138 ceph-mon[36954]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-26T13:33:28.693 INFO:tasks.cephadm:Deploying osd.6 on smithi100 with /dev/vg_nvme/lv_2... 2024-07-26T13:33:28.693 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-07-26T13:33:28.722 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:33:28.722 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:28.722 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:28.722 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:28 smithi100 ceph-mon[32599]: from='osd.5 [v2:172.21.15.167:6808/3239482836,v1:172.21.15.167:6809/3239482836]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-26T13:33:28.722 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:28 smithi100 ceph-mon[32599]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-26T13:33:28.956 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:33:29.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:29 smithi167 ceph-mon[37140]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-07-26T13:33:29.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:29 smithi167 ceph-mon[37140]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-07-26T13:33:29.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:29 smithi167 ceph-mon[37140]: from='osd.5 [v2:172.21.15.167:6808/3239482836,v1:172.21.15.167:6809/3239482836]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-07-26T13:33:29.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:29 smithi167 ceph-mon[37140]: osdmap e35: 6 total, 5 up, 6 in 2024-07-26T13:33:29.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-26T13:33:29.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:29 smithi167 ceph-mon[37140]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-07-26T13:33:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:29 smithi138 ceph-mon[36954]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-07-26T13:33:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:29 smithi138 ceph-mon[36954]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-07-26T13:33:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:29 smithi138 ceph-mon[36954]: from='osd.5 [v2:172.21.15.167:6808/3239482836,v1:172.21.15.167:6809/3239482836]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-07-26T13:33:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:29 smithi138 ceph-mon[36954]: osdmap e35: 6 total, 5 up, 6 in 2024-07-26T13:33:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-26T13:33:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:29 smithi138 ceph-mon[36954]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-07-26T13:33:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:29 smithi100 ceph-mon[32599]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-07-26T13:33:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:29 smithi100 ceph-mon[32599]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-07-26T13:33:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:29 smithi100 ceph-mon[32599]: from='osd.5 [v2:172.21.15.167:6808/3239482836,v1:172.21.15.167:6809/3239482836]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-07-26T13:33:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:29 smithi100 ceph-mon[32599]: osdmap e35: 6 total, 5 up, 6 in 2024-07-26T13:33:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-26T13:33:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:29 smithi100 ceph-mon[32599]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-07-26T13:33:30.809 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-07-26T13:33:30.890 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jul 26 13:33:30 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-osd-5[50499]: 2024-07-26T13:33:30.542+0000 7fd055a5a640 -1 osd.5 0 waiting for initial osdmap 2024-07-26T13:33:30.891 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jul 26 13:33:30 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-osd-5[50499]: 2024-07-26T13:33:30.551+0000 7fd04d850640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-26T13:33:30.893 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-07-26T13:33:30.906 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-07-26T13:33:30.906 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-07-26T13:33:30.906 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114032 s, 920 MB/s 2024-07-26T13:33:30.906 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-07-26T13:33:31.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:31 smithi167 ceph-mon[37140]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-07-26T13:33:31.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:31 smithi167 ceph-mon[37140]: osdmap e36: 6 total, 5 up, 6 in 2024-07-26T13:33:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-26T13:33:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-26T13:33:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:31 smithi167 ceph-mon[37140]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-07-26T13:33:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:31 smithi167 ceph-mon[37140]: from='osd.5 ' entity='osd.5' 2024-07-26T13:33:31.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:31 smithi138 ceph-mon[36954]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-07-26T13:33:31.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:31 smithi138 ceph-mon[36954]: osdmap e36: 6 total, 5 up, 6 in 2024-07-26T13:33:31.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-26T13:33:31.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-26T13:33:31.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:31 smithi138 ceph-mon[36954]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-07-26T13:33:31.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:31 smithi138 ceph-mon[36954]: from='osd.5 ' entity='osd.5' 2024-07-26T13:33:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:31 smithi100 ceph-mon[32599]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-07-26T13:33:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:31 smithi100 ceph-mon[32599]: osdmap e36: 6 total, 5 up, 6 in 2024-07-26T13:33:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-26T13:33:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-26T13:33:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:31 smithi100 ceph-mon[32599]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-07-26T13:33:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:31 smithi100 ceph-mon[32599]: from='osd.5 ' entity='osd.5' 2024-07-26T13:33:31.691 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph orch daemon add osd smithi100:vg_nvme/lv_2 2024-07-26T13:33:31.989 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:33:32.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:32 smithi100 ceph-mon[32599]: purged_snaps scrub starts 2024-07-26T13:33:32.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:32 smithi100 ceph-mon[32599]: purged_snaps scrub ok 2024-07-26T13:33:32.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:32 smithi100 ceph-mon[32599]: osd.5 [v2:172.21.15.167:6808/3239482836,v1:172.21.15.167:6809/3239482836] boot 2024-07-26T13:33:32.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:32 smithi100 ceph-mon[32599]: osdmap e37: 6 total, 6 up, 6 in 2024-07-26T13:33:32.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-26T13:33:32.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:32.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:32.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:33:32.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:33:32.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:32.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:33:32.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:33:32.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:32.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:32 smithi138 ceph-mon[36954]: purged_snaps scrub starts 2024-07-26T13:33:32.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:32 smithi138 ceph-mon[36954]: purged_snaps scrub ok 2024-07-26T13:33:32.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:32 smithi138 ceph-mon[36954]: osd.5 [v2:172.21.15.167:6808/3239482836,v1:172.21.15.167:6809/3239482836] boot 2024-07-26T13:33:32.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:32 smithi138 ceph-mon[36954]: osdmap e37: 6 total, 6 up, 6 in 2024-07-26T13:33:32.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-26T13:33:32.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:32.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:32.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:33:32.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:33:32.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:32.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:33:32.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:33:32.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:32.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:32 smithi167 ceph-mon[37140]: purged_snaps scrub starts 2024-07-26T13:33:32.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:32 smithi167 ceph-mon[37140]: purged_snaps scrub ok 2024-07-26T13:33:32.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:32 smithi167 ceph-mon[37140]: osd.5 [v2:172.21.15.167:6808/3239482836,v1:172.21.15.167:6809/3239482836] boot 2024-07-26T13:33:32.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:32 smithi167 ceph-mon[37140]: osdmap e37: 6 total, 6 up, 6 in 2024-07-26T13:33:32.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-26T13:33:32.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:32.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:32.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:33:32.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:33:32.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:32.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:33:32.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:33:32.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:33.158 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:33 smithi100 ceph-mon[32599]: Detected new or changed devices on smithi167 2024-07-26T13:33:33.158 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:33 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi167 to 8544M 2024-07-26T13:33:33.158 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:33 smithi100 ceph-mon[32599]: osdmap e38: 6 total, 6 up, 6 in 2024-07-26T13:33:33.158 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:33 smithi100 ceph-mon[32599]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-07-26T13:33:33.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:33 smithi138 ceph-mon[36954]: Detected new or changed devices on smithi167 2024-07-26T13:33:33.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:33 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi167 to 8544M 2024-07-26T13:33:33.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:33 smithi138 ceph-mon[36954]: osdmap e38: 6 total, 6 up, 6 in 2024-07-26T13:33:33.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:33 smithi138 ceph-mon[36954]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-07-26T13:33:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:33 smithi167 ceph-mon[37140]: Detected new or changed devices on smithi167 2024-07-26T13:33:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:33 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi167 to 8544M 2024-07-26T13:33:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:33 smithi167 ceph-mon[37140]: osdmap e38: 6 total, 6 up, 6 in 2024-07-26T13:33:33.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:33 smithi167 ceph-mon[37140]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-07-26T13:33:33.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.670+0000 7fac99757640 1 -- 172.21.15.100:0/1661769347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac94107dd0 msgr2=0x7fac941081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:33:33.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.670+0000 7fac99757640 1 --2- 172.21.15.100:0/1661769347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac94107dd0 0x7fac941081e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fac7c009900 tx=0x7fac7c02a410 comp rx=0 tx=0).stop 2024-07-26T13:33:33.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.670+0000 7fac99757640 1 -- 172.21.15.100:0/1661769347 shutdown_connections 2024-07-26T13:33:33.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.670+0000 7fac99757640 1 --2- 172.21.15.100:0/1661769347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac94108720 0x7fac9410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:33:33.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.670+0000 7fac99757640 1 --2- 172.21.15.100:0/1661769347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac94107dd0 0x7fac941081e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:33:33.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.670+0000 7fac99757640 1 --2- 172.21.15.100:0/1661769347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac94102490 0x7fac941028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:33:33.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.670+0000 7fac99757640 1 -- 172.21.15.100:0/1661769347 >> 172.21.15.100:0/1661769347 conn(0x7fac94079160 msgr2=0x7fac94079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:33:33.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.670+0000 7fac99757640 1 -- 172.21.15.100:0/1661769347 shutdown_connections 2024-07-26T13:33:33.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.670+0000 7fac99757640 1 -- 172.21.15.100:0/1661769347 wait complete. 2024-07-26T13:33:33.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.671+0000 7fac99757640 1 Processor -- start 2024-07-26T13:33:33.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.671+0000 7fac99757640 1 -- start start 2024-07-26T13:33:33.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.671+0000 7fac99757640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac94102490 0x7fac9419ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:33:33.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.671+0000 7fac99757640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac94107dd0 0x7fac9419b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:33:33.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.671+0000 7fac99757640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac94108720 0x7fac9419b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:33:33.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.671+0000 7fac99757640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fac941123f0 con 0x7fac94107dd0 2024-07-26T13:33:33.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.671+0000 7fac99757640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fac94112270 con 0x7fac94102490 2024-07-26T13:33:33.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.671+0000 7fac99757640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fac94112570 con 0x7fac94108720 2024-07-26T13:33:33.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.671+0000 7fac927fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac94107dd0 0x7fac9419b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:33:33.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.671+0000 7fac927fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac94107dd0 0x7fac9419b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56928/0 (socket says 172.21.15.100:56928) 2024-07-26T13:33:33.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.671+0000 7fac927fc640 1 -- 172.21.15.100:0/1164164701 learned_addr learned my addr 172.21.15.100:0/1164164701 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:33:33.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.672+0000 7fac937fe640 1 --2- 172.21.15.100:0/1164164701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac94108720 0x7fac9419b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:33:33.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.672+0000 7fac92ffd640 1 --2- 172.21.15.100:0/1164164701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac94102490 0x7fac9419ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:33:33.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.672+0000 7fac927fc640 1 -- 172.21.15.100:0/1164164701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac94108720 msgr2=0x7fac9419b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:33:33.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.672+0000 7fac927fc640 1 --2- 172.21.15.100:0/1164164701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac94108720 0x7fac9419b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:33:33.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.672+0000 7fac927fc640 1 -- 172.21.15.100:0/1164164701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac94102490 msgr2=0x7fac9419ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:33:33.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.672+0000 7fac927fc640 1 --2- 172.21.15.100:0/1164164701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac94102490 0x7fac9419ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:33:33.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.672+0000 7fac927fc640 1 -- 172.21.15.100:0/1164164701 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac941a5f90 con 0x7fac94107dd0 2024-07-26T13:33:33.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.672+0000 7fac92ffd640 1 --2- 172.21.15.100:0/1164164701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac94102490 0x7fac9419ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:33:33.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.672+0000 7fac937fe640 1 --2- 172.21.15.100:0/1164164701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac94108720 0x7fac9419b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:33:33.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.672+0000 7fac927fc640 1 --2- 172.21.15.100:0/1164164701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac94107dd0 0x7fac9419b240 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fac7c032fd0 tx=0x7fac7c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:33:33.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.672+0000 7fac8bfff640 1 -- 172.21.15.100:0/1164164701 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c03f030 con 0x7fac94107dd0 2024-07-26T13:33:33.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.672+0000 7fac8bfff640 1 -- 172.21.15.100:0/1164164701 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac7c004490 con 0x7fac94107dd0 2024-07-26T13:33:33.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.672+0000 7fac99757640 1 -- 172.21.15.100:0/1164164701 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac941a6220 con 0x7fac94107dd0 2024-07-26T13:33:33.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.672+0000 7fac8bfff640 1 -- 172.21.15.100:0/1164164701 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c035070 con 0x7fac94107dd0 2024-07-26T13:33:33.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.672+0000 7fac99757640 1 -- 172.21.15.100:0/1164164701 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac941a67d0 con 0x7fac94107dd0 2024-07-26T13:33:33.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.673+0000 7fac99757640 1 -- 172.21.15.100:0/1164164701 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac941028a0 con 0x7fac94107dd0 2024-07-26T13:33:33.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.675+0000 7fac8bfff640 1 -- 172.21.15.100:0/1164164701 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7fac7c042030 con 0x7fac94107dd0 2024-07-26T13:33:33.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.676+0000 7fac8bfff640 1 --2- 172.21.15.100:0/1164164701 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fac800b3cd0 0x7fac800b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:33:33.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.676+0000 7fac92ffd640 1 --2- 172.21.15.100:0/1164164701 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fac800b3cd0 0x7fac800b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:33:33.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.676+0000 7fac8bfff640 1 -- 172.21.15.100:0/1164164701 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4748+0+0 (secure 0 0 0) 0x7fac7c103780 con 0x7fac94107dd0 2024-07-26T13:33:33.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.676+0000 7fac92ffd640 1 --2- 172.21.15.100:0/1164164701 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fac800b3cd0 0x7fac800b61e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fac9406b3f0 tx=0x7fac84009400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:33:33.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.679+0000 7fac8bfff640 1 -- 172.21.15.100:0/1164164701 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fac7c0cbbc0 con 0x7fac94107dd0 2024-07-26T13:33:33.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:33.861+0000 7fac99757640 1 -- 172.21.15.100:0/1164164701 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fac94063230 con 0x7fac800b3cd0 2024-07-26T13:33:34.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:34 smithi100 ceph-mon[32599]: osdmap e39: 6 total, 6 up, 6 in 2024-07-26T13:33:34.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-26T13:33:34.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-26T13:33:34.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:33:34.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:34 smithi138 ceph-mon[36954]: osdmap e39: 6 total, 6 up, 6 in 2024-07-26T13:33:34.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-26T13:33:34.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-26T13:33:34.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:33:34.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:34 smithi167 ceph-mon[37140]: osdmap e39: 6 total, 6 up, 6 in 2024-07-26T13:33:34.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-26T13:33:34.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-26T13:33:34.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:33:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:35 smithi100 ceph-mon[32599]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:33:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:35 smithi100 ceph-mon[32599]: pgmap v126: 1 pgs: 1 peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-07-26T13:33:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:35 smithi138 ceph-mon[36954]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:33:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:35 smithi138 ceph-mon[36954]: pgmap v126: 1 pgs: 1 peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-07-26T13:33:35.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:35 smithi167 ceph-mon[37140]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:33:35.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:35 smithi167 ceph-mon[37140]: pgmap v126: 1 pgs: 1 peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-07-26T13:33:37.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:37 smithi138 ceph-mon[36954]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-26T13:33:37.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:37 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/673392386' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "420e4471-5626-41cc-a8e1-69c801196b2c"}]: dispatch 2024-07-26T13:33:37.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:37 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/673392386' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "420e4471-5626-41cc-a8e1-69c801196b2c"}]': finished 2024-07-26T13:33:37.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:37 smithi138 ceph-mon[36954]: osdmap e40: 7 total, 6 up, 7 in 2024-07-26T13:33:37.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-26T13:33:37.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:37 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/3000680220' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-26T13:33:37.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:37 smithi100 ceph-mon[32599]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-26T13:33:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:37 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/673392386' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "420e4471-5626-41cc-a8e1-69c801196b2c"}]: dispatch 2024-07-26T13:33:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:37 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/673392386' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "420e4471-5626-41cc-a8e1-69c801196b2c"}]': finished 2024-07-26T13:33:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:37 smithi100 ceph-mon[32599]: osdmap e40: 7 total, 6 up, 7 in 2024-07-26T13:33:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-26T13:33:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:37 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/3000680220' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-26T13:33:37.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:37 smithi167 ceph-mon[37140]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-26T13:33:37.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:37 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/673392386' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "420e4471-5626-41cc-a8e1-69c801196b2c"}]: dispatch 2024-07-26T13:33:37.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:37 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/673392386' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "420e4471-5626-41cc-a8e1-69c801196b2c"}]': finished 2024-07-26T13:33:37.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:37 smithi167 ceph-mon[37140]: osdmap e40: 7 total, 6 up, 7 in 2024-07-26T13:33:37.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-26T13:33:37.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:37 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/3000680220' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-26T13:33:39.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:39 smithi138 ceph-mon[36954]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-07-26T13:33:39.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:39 smithi100 ceph-mon[32599]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-07-26T13:33:39.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:39 smithi167 ceph-mon[37140]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-07-26T13:33:41.509 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:41 smithi100 ceph-mon[32599]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-26T13:33:41.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:41 smithi138 ceph-mon[36954]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-26T13:33:41.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:41 smithi167 ceph-mon[37140]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-26T13:33:43.635 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:43 smithi100 ceph-mon[32599]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-07-26T13:33:43.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:43 smithi138 ceph-mon[36954]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-07-26T13:33:43.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:43 smithi167 ceph-mon[37140]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-07-26T13:33:44.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-07-26T13:33:44.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:33:44.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-07-26T13:33:44.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:33:44.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-07-26T13:33:44.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:33:45.635 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:45 smithi100 ceph-mon[32599]: Deploying daemon osd.6 on smithi100 2024-07-26T13:33:45.635 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:45 smithi100 ceph-mon[32599]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-26T13:33:45.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:45 smithi138 ceph-mon[36954]: Deploying daemon osd.6 on smithi100 2024-07-26T13:33:45.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:45 smithi138 ceph-mon[36954]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-26T13:33:45.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:45 smithi167 ceph-mon[37140]: Deploying daemon osd.6 on smithi100 2024-07-26T13:33:45.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:45 smithi167 ceph-mon[37140]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-26T13:33:47.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:47 smithi138 ceph-mon[36954]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-07-26T13:33:47.734 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:47 smithi100 ceph-mon[32599]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-07-26T13:33:47.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:47 smithi167 ceph-mon[37140]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-07-26T13:33:49.658 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:49 smithi100 ceph-mon[32599]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-07-26T13:33:49.658 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:49 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:33:49.658 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:49 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:49.659 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:49 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:49.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:49 smithi138 ceph-mon[36954]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-07-26T13:33:49.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:49 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:33:49.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:49 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:49.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:49 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:49.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:49 smithi167 ceph-mon[37140]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-07-26T13:33:49.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:49 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:33:49.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:49 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:49.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:49 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:51.612 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:51 smithi100 ceph-mon[32599]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-07-26T13:33:51.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:51 smithi138 ceph-mon[36954]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-07-26T13:33:51.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:51 smithi167 ceph-mon[37140]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-07-26T13:33:52.672 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 6 on host 'smithi100' 2024-07-26T13:33:52.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:52.671+0000 7fac8bfff640 1 -- 172.21.15.100:0/1164164701 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fac94063230 con 0x7fac800b3cd0 2024-07-26T13:33:52.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:52.673+0000 7fac99757640 1 -- 172.21.15.100:0/1164164701 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fac800b3cd0 msgr2=0x7fac800b61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:33:52.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:52.673+0000 7fac99757640 1 --2- 172.21.15.100:0/1164164701 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fac800b3cd0 0x7fac800b61e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fac9406b3f0 tx=0x7fac84009400 comp rx=0 tx=0).stop 2024-07-26T13:33:52.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:52.673+0000 7fac99757640 1 -- 172.21.15.100:0/1164164701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac94107dd0 msgr2=0x7fac9419b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:33:52.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:52.673+0000 7fac99757640 1 --2- 172.21.15.100:0/1164164701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac94107dd0 0x7fac9419b240 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fac7c032fd0 tx=0x7fac7c004270 comp rx=0 tx=0).stop 2024-07-26T13:33:52.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:52.673+0000 7fac99757640 1 -- 172.21.15.100:0/1164164701 shutdown_connections 2024-07-26T13:33:52.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:52.673+0000 7fac99757640 1 --2- 172.21.15.100:0/1164164701 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fac800b3cd0 0x7fac800b61e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:33:52.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:52.673+0000 7fac99757640 1 --2- 172.21.15.100:0/1164164701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac94108720 0x7fac9419b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:33:52.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:52.673+0000 7fac99757640 1 --2- 172.21.15.100:0/1164164701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac94107dd0 0x7fac9419b240 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:33:52.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:52.673+0000 7fac99757640 1 --2- 172.21.15.100:0/1164164701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac94102490 0x7fac9419ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:33:52.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:52.673+0000 7fac99757640 1 -- 172.21.15.100:0/1164164701 >> 172.21.15.100:0/1164164701 conn(0x7fac94079160 msgr2=0x7fac94103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:33:52.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:52.673+0000 7fac99757640 1 -- 172.21.15.100:0/1164164701 shutdown_connections 2024-07-26T13:33:52.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:33:52.673+0000 7fac99757640 1 -- 172.21.15.100:0/1164164701 wait complete. 2024-07-26T13:33:52.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:52.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:52.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:52 smithi100 ceph-mon[32599]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-07-26T13:33:52.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:52.953 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:52 smithi138 ceph-mon[36954]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-07-26T13:33:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:53.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:53.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:53.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:53.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:52 smithi167 ceph-mon[37140]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-07-26T13:33:53.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:53.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:53.287 DEBUG:teuthology.orchestra.run.smithi100:osd.6> sudo journalctl -f -n 0 -u ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@osd.6.service 2024-07-26T13:33:53.289 INFO:tasks.cephadm:Deploying osd.7 on smithi138 with /dev/vg_nvme/lv_2... 2024-07-26T13:33:53.289 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-07-26T13:33:53.548 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.b/config 2024-07-26T13:33:53.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:53 smithi138 ceph-mon[36954]: from='osd.6 [v2:172.21.15.100:6818/3218415484,v1:172.21.15.100:6819/3218415484]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-07-26T13:33:53.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:53 smithi100 ceph-mon[32599]: from='osd.6 [v2:172.21.15.100:6818/3218415484,v1:172.21.15.100:6819/3218415484]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-07-26T13:33:54.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:53 smithi167 ceph-mon[37140]: from='osd.6 [v2:172.21.15.100:6818/3218415484,v1:172.21.15.100:6819/3218415484]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-07-26T13:33:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:54 smithi100 ceph-mon[32599]: from='osd.6 [v2:172.21.15.100:6818/3218415484,v1:172.21.15.100:6819/3218415484]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-07-26T13:33:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:54 smithi100 ceph-mon[32599]: osdmap e41: 7 total, 6 up, 7 in 2024-07-26T13:33:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:54 smithi100 ceph-mon[32599]: from='osd.6 [v2:172.21.15.100:6818/3218415484,v1:172.21.15.100:6819/3218415484]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-07-26T13:33:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-26T13:33:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:54 smithi100 ceph-mon[32599]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-07-26T13:33:54.995 INFO:journalctl@ceph.osd.6.smithi100.stdout:Jul 26 13:33:54 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-osd-6[65594]: 2024-07-26T13:33:54.821+0000 7fd7297ef640 -1 osd.6 0 waiting for initial osdmap 2024-07-26T13:33:54.995 INFO:journalctl@ceph.osd.6.smithi100.stdout:Jul 26 13:33:54 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-osd-6[65594]: 2024-07-26T13:33:54.831+0000 7fd7255ed640 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-26T13:33:55.054 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:54 smithi138 ceph-mon[36954]: from='osd.6 [v2:172.21.15.100:6818/3218415484,v1:172.21.15.100:6819/3218415484]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-07-26T13:33:55.054 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:54 smithi138 ceph-mon[36954]: osdmap e41: 7 total, 6 up, 7 in 2024-07-26T13:33:55.055 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:54 smithi138 ceph-mon[36954]: from='osd.6 [v2:172.21.15.100:6818/3218415484,v1:172.21.15.100:6819/3218415484]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-07-26T13:33:55.055 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-26T13:33:55.055 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:54 smithi138 ceph-mon[36954]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-07-26T13:33:55.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:54 smithi167 ceph-mon[37140]: from='osd.6 [v2:172.21.15.100:6818/3218415484,v1:172.21.15.100:6819/3218415484]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-07-26T13:33:55.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:54 smithi167 ceph-mon[37140]: osdmap e41: 7 total, 6 up, 7 in 2024-07-26T13:33:55.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:54 smithi167 ceph-mon[37140]: from='osd.6 [v2:172.21.15.100:6818/3218415484,v1:172.21.15.100:6819/3218415484]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-07-26T13:33:55.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-26T13:33:55.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:54 smithi167 ceph-mon[37140]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-07-26T13:33:55.446 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-07-26T13:33:55.536 INFO:teuthology.orchestra.run.smithi138.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-07-26T13:33:55.551 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10+0 records in 2024-07-26T13:33:55.551 INFO:teuthology.orchestra.run.smithi138.stderr:10+0 records out 2024-07-26T13:33:55.551 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126099 s, 832 MB/s 2024-07-26T13:33:55.551 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping successful for: 2024-07-26T13:33:55.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:55 smithi100 ceph-mon[32599]: from='osd.6 [v2:172.21.15.100:6818/3218415484,v1:172.21.15.100:6819/3218415484]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-07-26T13:33:55.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:55 smithi100 ceph-mon[32599]: osdmap e42: 7 total, 6 up, 7 in 2024-07-26T13:33:55.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-26T13:33:55.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-26T13:33:56.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:55 smithi138 ceph-mon[36954]: from='osd.6 [v2:172.21.15.100:6818/3218415484,v1:172.21.15.100:6819/3218415484]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-07-26T13:33:56.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:55 smithi138 ceph-mon[36954]: osdmap e42: 7 total, 6 up, 7 in 2024-07-26T13:33:56.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-26T13:33:56.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-26T13:33:56.237 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_2 2024-07-26T13:33:56.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:55 smithi167 ceph-mon[37140]: from='osd.6 [v2:172.21.15.100:6818/3218415484,v1:172.21.15.100:6819/3218415484]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-07-26T13:33:56.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:55 smithi167 ceph-mon[37140]: osdmap e42: 7 total, 6 up, 7 in 2024-07-26T13:33:56.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-26T13:33:56.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-26T13:33:56.492 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.b/config 2024-07-26T13:33:56.884 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:56 smithi138 ceph-mon[36954]: purged_snaps scrub starts 2024-07-26T13:33:56.885 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:56 smithi138 ceph-mon[36954]: purged_snaps scrub ok 2024-07-26T13:33:56.885 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-26T13:33:56.885 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:56 smithi138 ceph-mon[36954]: osd.6 [v2:172.21.15.100:6818/3218415484,v1:172.21.15.100:6819/3218415484] boot 2024-07-26T13:33:56.885 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:56 smithi138 ceph-mon[36954]: osdmap e43: 7 total, 7 up, 7 in 2024-07-26T13:33:56.885 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-26T13:33:56.885 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:56 smithi138 ceph-mon[36954]: Detected new or changed devices on smithi100 2024-07-26T13:33:56.885 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:56.885 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:56.885 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:33:56.885 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:33:56.886 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:33:56.886 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:56 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi100 to 5695M 2024-07-26T13:33:56.886 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:56.886 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:33:56.886 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:33:56.886 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:56.886 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:56 smithi138 ceph-mon[36954]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-07-26T13:33:56.886 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:33:56.886 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:33:56.886 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:33:56.886 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:56 smithi100 ceph-mon[32599]: purged_snaps scrub starts 2024-07-26T13:33:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:56 smithi100 ceph-mon[32599]: purged_snaps scrub ok 2024-07-26T13:33:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-26T13:33:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:56 smithi100 ceph-mon[32599]: osd.6 [v2:172.21.15.100:6818/3218415484,v1:172.21.15.100:6819/3218415484] boot 2024-07-26T13:33:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:56 smithi100 ceph-mon[32599]: osdmap e43: 7 total, 7 up, 7 in 2024-07-26T13:33:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-26T13:33:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:56 smithi100 ceph-mon[32599]: Detected new or changed devices on smithi100 2024-07-26T13:33:57.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:57.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:57.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:33:57.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:33:57.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:33:57.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:56 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi100 to 5695M 2024-07-26T13:33:57.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:57.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:33:57.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:33:57.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:57.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:56 smithi100 ceph-mon[32599]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-07-26T13:33:57.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:33:57.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:33:57.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:33:57.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:57.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:56 smithi167 ceph-mon[37140]: purged_snaps scrub starts 2024-07-26T13:33:57.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:56 smithi167 ceph-mon[37140]: purged_snaps scrub ok 2024-07-26T13:33:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-26T13:33:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:56 smithi167 ceph-mon[37140]: osd.6 [v2:172.21.15.100:6818/3218415484,v1:172.21.15.100:6819/3218415484] boot 2024-07-26T13:33:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:56 smithi167 ceph-mon[37140]: osdmap e43: 7 total, 7 up, 7 in 2024-07-26T13:33:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-26T13:33:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:56 smithi167 ceph-mon[37140]: Detected new or changed devices on smithi100 2024-07-26T13:33:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:33:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:33:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:33:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:56 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi100 to 5695M 2024-07-26T13:33:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:33:57.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:33:57.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:57.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:56 smithi167 ceph-mon[37140]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-07-26T13:33:57.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:33:57.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:33:57.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:33:57.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:33:57.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:57 smithi138 ceph-mon[36954]: osdmap e44: 7 total, 7 up, 7 in 2024-07-26T13:33:58.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.191+0000 7f0d65bfb640 1 -- 172.21.15.138:0/861453557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d60102490 msgr2=0x7f0d601028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:33:58.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.191+0000 7f0d65bfb640 1 --2- 172.21.15.138:0/861453557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d60102490 0x7f0d601028a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009900 tx=0x7f0d5002a410 comp rx=0 tx=0).stop 2024-07-26T13:33:58.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.191+0000 7f0d65bfb640 1 -- 172.21.15.138:0/861453557 shutdown_connections 2024-07-26T13:33:58.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.191+0000 7f0d65bfb640 1 --2- 172.21.15.138:0/861453557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d60108720 0x7f0d6010ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:33:58.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.191+0000 7f0d65bfb640 1 --2- 172.21.15.138:0/861453557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d60107dd0 0x7f0d601081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:33:58.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.191+0000 7f0d65bfb640 1 --2- 172.21.15.138:0/861453557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d60102490 0x7f0d601028a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:33:58.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.191+0000 7f0d65bfb640 1 -- 172.21.15.138:0/861453557 >> 172.21.15.138:0/861453557 conn(0x7f0d60079160 msgr2=0x7f0d60079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:33:58.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.191+0000 7f0d65bfb640 1 -- 172.21.15.138:0/861453557 shutdown_connections 2024-07-26T13:33:58.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.191+0000 7f0d65bfb640 1 -- 172.21.15.138:0/861453557 wait complete. 2024-07-26T13:33:58.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.192+0000 7f0d65bfb640 1 Processor -- start 2024-07-26T13:33:58.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.192+0000 7f0d65bfb640 1 -- start start 2024-07-26T13:33:58.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.192+0000 7f0d65bfb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d60102490 0x7f0d60198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:33:58.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.193+0000 7f0d65bfb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d60107dd0 0x7f0d60199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:33:58.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.193+0000 7f0d65bfb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d60108720 0x7f0d6019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:33:58.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.193+0000 7f0d65bfb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0d601123f0 con 0x7f0d60102490 2024-07-26T13:33:58.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.193+0000 7f0d65bfb640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0d60112270 con 0x7f0d60108720 2024-07-26T13:33:58.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.193+0000 7f0d65bfb640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0d60112570 con 0x7f0d60107dd0 2024-07-26T13:33:58.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.193+0000 7f0d5ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d60108720 0x7f0d6019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:33:58.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.193+0000 7f0d5ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d60108720 0x7f0d6019e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34016/0 (socket says 172.21.15.138:34016) 2024-07-26T13:33:58.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.193+0000 7f0d5ffff640 1 -- 172.21.15.138:0/1487335411 learned_addr learned my addr 172.21.15.138:0/1487335411 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-07-26T13:33:58.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.193+0000 7f0d5f7fe640 1 --2- 172.21.15.138:0/1487335411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d60102490 0x7f0d60198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:33:58.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.193+0000 7f0d56ffd640 1 --2- 172.21.15.138:0/1487335411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d60107dd0 0x7f0d60199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:33:58.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.193+0000 7f0d5ffff640 1 -- 172.21.15.138:0/1487335411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d60107dd0 msgr2=0x7f0d60199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:33:58.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.193+0000 7f0d5ffff640 1 --2- 172.21.15.138:0/1487335411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d60107dd0 0x7f0d60199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:33:58.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.193+0000 7f0d5ffff640 1 -- 172.21.15.138:0/1487335411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d60102490 msgr2=0x7f0d60198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:33:58.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.193+0000 7f0d5ffff640 1 --2- 172.21.15.138:0/1487335411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d60102490 0x7f0d60198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:33:58.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.193+0000 7f0d5ffff640 1 -- 172.21.15.138:0/1487335411 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d6019edd0 con 0x7f0d60108720 2024-07-26T13:33:58.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.193+0000 7f0d5f7fe640 1 --2- 172.21.15.138:0/1487335411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d60102490 0x7f0d60198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:33:58.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.193+0000 7f0d56ffd640 1 --2- 172.21.15.138:0/1487335411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d60107dd0 0x7f0d60199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:33:58.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.193+0000 7f0d5ffff640 1 --2- 172.21.15.138:0/1487335411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d60108720 0x7f0d6019e6a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0d5800cad0 tx=0x7f0d5800cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:33:58.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.193+0000 7f0d5d7fa640 1 -- 172.21.15.138:0/1487335411 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5800ae20 con 0x7f0d60108720 2024-07-26T13:33:58.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.193+0000 7f0d5d7fa640 1 -- 172.21.15.138:0/1487335411 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d58005020 con 0x7f0d60108720 2024-07-26T13:33:58.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.193+0000 7f0d65bfb640 1 -- 172.21.15.138:0/1487335411 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d6019f0c0 con 0x7f0d60108720 2024-07-26T13:33:58.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.193+0000 7f0d5d7fa640 1 -- 172.21.15.138:0/1487335411 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5800f6b0 con 0x7f0d60108720 2024-07-26T13:33:58.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.193+0000 7f0d65bfb640 1 -- 172.21.15.138:0/1487335411 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d601a5820 con 0x7f0d60108720 2024-07-26T13:33:58.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.195+0000 7f0d65bfb640 1 -- 172.21.15.138:0/1487335411 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d601028a0 con 0x7f0d60108720 2024-07-26T13:33:58.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.196+0000 7f0d5d7fa640 1 -- 172.21.15.138:0/1487335411 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f0d580051d0 con 0x7f0d60108720 2024-07-26T13:33:58.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.197+0000 7f0d5d7fa640 1 --2- 172.21.15.138:0/1487335411 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f0d4c0b3c20 0x7f0d4c0b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:33:58.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.197+0000 7f0d5d7fa640 1 -- 172.21.15.138:0/1487335411 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5180+0+0 (secure 0 0 0) 0x7f0d580e34e0 con 0x7f0d60108720 2024-07-26T13:33:58.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.197+0000 7f0d5f7fe640 1 --2- 172.21.15.138:0/1487335411 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f0d4c0b3c20 0x7f0d4c0b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:33:58.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.197+0000 7f0d5f7fe640 1 --2- 172.21.15.138:0/1487335411 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f0d4c0b3c20 0x7f0d4c0b6130 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0d500096c0 tx=0x7f0d50031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:33:58.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.200+0000 7f0d5d7fa640 1 -- 172.21.15.138:0/1487335411 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f0d58014030 con 0x7f0d60108720 2024-07-26T13:33:58.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:57 smithi100 ceph-mon[32599]: osdmap e44: 7 total, 7 up, 7 in 2024-07-26T13:33:58.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:57 smithi167 ceph-mon[37140]: osdmap e44: 7 total, 7 up, 7 in 2024-07-26T13:33:58.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:33:58.381+0000 7f0d65bfb640 1 -- 172.21.15.138:0/1487335411 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f0d60063230 con 0x7f0d4c0b3c20 2024-07-26T13:33:59.054 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:58 smithi138 ceph-mon[36954]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-07-26T13:33:59.054 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:58 smithi138 ceph-mon[36954]: from='client.24404 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:33:59.054 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-26T13:33:59.054 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-26T13:33:59.054 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:33:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:33:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:58 smithi100 ceph-mon[32599]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-07-26T13:33:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:58 smithi100 ceph-mon[32599]: from='client.24404 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:33:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-26T13:33:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-26T13:33:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:33:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:33:59.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:58 smithi167 ceph-mon[37140]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-07-26T13:33:59.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:58 smithi167 ceph-mon[37140]: from='client.24404 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:33:59.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-26T13:33:59.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-26T13:33:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:33:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:34:01.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:01 smithi138 ceph-mon[36954]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-07-26T13:34:01.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:01 smithi138 ceph-mon[36954]: from='client.? 172.21.15.138:0/853050006' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a27b097e-207d-44c3-af63-544f228b83ec"}]: dispatch 2024-07-26T13:34:01.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:01 smithi138 ceph-mon[36954]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a27b097e-207d-44c3-af63-544f228b83ec"}]: dispatch 2024-07-26T13:34:01.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:01 smithi138 ceph-mon[36954]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a27b097e-207d-44c3-af63-544f228b83ec"}]': finished 2024-07-26T13:34:01.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:01 smithi138 ceph-mon[36954]: osdmap e45: 8 total, 7 up, 8 in 2024-07-26T13:34:01.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-26T13:34:01.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:01 smithi100 ceph-mon[32599]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-07-26T13:34:01.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:01 smithi100 ceph-mon[32599]: from='client.? 172.21.15.138:0/853050006' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a27b097e-207d-44c3-af63-544f228b83ec"}]: dispatch 2024-07-26T13:34:01.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:01 smithi100 ceph-mon[32599]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a27b097e-207d-44c3-af63-544f228b83ec"}]: dispatch 2024-07-26T13:34:01.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:01 smithi100 ceph-mon[32599]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a27b097e-207d-44c3-af63-544f228b83ec"}]': finished 2024-07-26T13:34:01.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:01 smithi100 ceph-mon[32599]: osdmap e45: 8 total, 7 up, 8 in 2024-07-26T13:34:01.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-26T13:34:01.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:01 smithi167 ceph-mon[37140]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-07-26T13:34:01.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:01 smithi167 ceph-mon[37140]: from='client.? 172.21.15.138:0/853050006' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a27b097e-207d-44c3-af63-544f228b83ec"}]: dispatch 2024-07-26T13:34:01.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:01 smithi167 ceph-mon[37140]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a27b097e-207d-44c3-af63-544f228b83ec"}]: dispatch 2024-07-26T13:34:01.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:01 smithi167 ceph-mon[37140]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a27b097e-207d-44c3-af63-544f228b83ec"}]': finished 2024-07-26T13:34:01.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:01 smithi167 ceph-mon[37140]: osdmap e45: 8 total, 7 up, 8 in 2024-07-26T13:34:01.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-26T13:34:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:02 smithi138 ceph-mon[36954]: from='client.? 172.21.15.138:0/3564938039' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-26T13:34:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:02 smithi100 ceph-mon[32599]: from='client.? 172.21.15.138:0/3564938039' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-26T13:34:02.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:02 smithi167 ceph-mon[37140]: from='client.? 172.21.15.138:0/3564938039' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-26T13:34:03.645 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:03 smithi100 ceph-mon[32599]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-07-26T13:34:03.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:03 smithi138 ceph-mon[36954]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-07-26T13:34:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:03 smithi167 ceph-mon[37140]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-07-26T13:34:05.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:05 smithi138 ceph-mon[36954]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-07-26T13:34:05.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:05 smithi100 ceph-mon[32599]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-07-26T13:34:05.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:05 smithi167 ceph-mon[37140]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-07-26T13:34:07.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:07 smithi138 ceph-mon[36954]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-07-26T13:34:07.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:07 smithi100 ceph-mon[32599]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-07-26T13:34:07.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:07 smithi167 ceph-mon[37140]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-07-26T13:34:08.422 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-07-26T13:34:08.422 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:34:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-07-26T13:34:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:34:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-07-26T13:34:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:34:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:09 smithi138 ceph-mon[36954]: Deploying daemon osd.7 on smithi138 2024-07-26T13:34:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:09 smithi138 ceph-mon[36954]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-07-26T13:34:09.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:09 smithi100 ceph-mon[32599]: Deploying daemon osd.7 on smithi138 2024-07-26T13:34:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:09 smithi100 ceph-mon[32599]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-07-26T13:34:09.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:09 smithi167 ceph-mon[37140]: Deploying daemon osd.7 on smithi138 2024-07-26T13:34:09.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:09 smithi167 ceph-mon[37140]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-07-26T13:34:11.550 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:11 smithi138 ceph-mon[36954]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-07-26T13:34:11.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:11 smithi100 ceph-mon[32599]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-07-26T13:34:11.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:11 smithi167 ceph-mon[37140]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-07-26T13:34:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:13 smithi138 ceph-mon[36954]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-07-26T13:34:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:34:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:13 smithi100 ceph-mon[32599]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-07-26T13:34:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:34:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:13 smithi167 ceph-mon[37140]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-07-26T13:34:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:34:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:15 smithi138 ceph-mon[36954]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-07-26T13:34:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:15 smithi100 ceph-mon[32599]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-07-26T13:34:15.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:15 smithi167 ceph-mon[37140]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-07-26T13:34:16.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:34:16.943+0000 7f0d5d7fa640 1 -- 172.21.15.138:0/1487335411 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0d60063230 con 0x7f0d4c0b3c20 2024-07-26T13:34:16.945 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 7 on host 'smithi138' 2024-07-26T13:34:16.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:34:16.946+0000 7f0d65bfb640 1 -- 172.21.15.138:0/1487335411 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f0d4c0b3c20 msgr2=0x7f0d4c0b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:34:16.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:34:16.946+0000 7f0d65bfb640 1 --2- 172.21.15.138:0/1487335411 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f0d4c0b3c20 0x7f0d4c0b6130 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0d500096c0 tx=0x7f0d50031400 comp rx=0 tx=0).stop 2024-07-26T13:34:16.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:34:16.946+0000 7f0d65bfb640 1 -- 172.21.15.138:0/1487335411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d60108720 msgr2=0x7f0d6019e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:34:16.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:34:16.946+0000 7f0d65bfb640 1 --2- 172.21.15.138:0/1487335411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d60108720 0x7f0d6019e6a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0d5800cad0 tx=0x7f0d5800cf90 comp rx=0 tx=0).stop 2024-07-26T13:34:16.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:34:16.946+0000 7f0d65bfb640 1 -- 172.21.15.138:0/1487335411 shutdown_connections 2024-07-26T13:34:16.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:34:16.946+0000 7f0d65bfb640 1 --2- 172.21.15.138:0/1487335411 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f0d4c0b3c20 0x7f0d4c0b6130 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:34:16.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:34:16.946+0000 7f0d65bfb640 1 --2- 172.21.15.138:0/1487335411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d60108720 0x7f0d6019e6a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:34:16.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:34:16.946+0000 7f0d65bfb640 1 --2- 172.21.15.138:0/1487335411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d60107dd0 0x7f0d60199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:34:16.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:34:16.946+0000 7f0d65bfb640 1 --2- 172.21.15.138:0/1487335411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d60102490 0x7f0d60198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:34:16.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:34:16.946+0000 7f0d65bfb640 1 -- 172.21.15.138:0/1487335411 >> 172.21.15.138:0/1487335411 conn(0x7f0d60079160 msgr2=0x7f0d60100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:34:16.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:34:16.946+0000 7f0d65bfb640 1 -- 172.21.15.138:0/1487335411 shutdown_connections 2024-07-26T13:34:16.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:34:16.946+0000 7f0d65bfb640 1 -- 172.21.15.138:0/1487335411 wait complete. 2024-07-26T13:34:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:34:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:34:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:17 smithi138 ceph-mon[36954]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-07-26T13:34:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:34:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:34:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:34:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:17 smithi100 ceph-mon[32599]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-07-26T13:34:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:34:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:17.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:17.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:17.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:34:17.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:34:17.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:17.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:17 smithi167 ceph-mon[37140]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-07-26T13:34:17.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:34:17.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:17.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:17.643 DEBUG:teuthology.orchestra.run.smithi138:osd.7> sudo journalctl -f -n 0 -u ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@osd.7.service 2024-07-26T13:34:17.645 INFO:tasks.cephadm:Deploying osd.8 on smithi167 with /dev/vg_nvme/lv_2... 2024-07-26T13:34:17.646 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-07-26T13:34:17.906 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.c/config 2024-07-26T13:34:18.353 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:18 smithi167 ceph-mon[37140]: from='osd.7 [v2:172.21.15.138:6816/45541749,v1:172.21.15.138:6817/45541749]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-07-26T13:34:18.353 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:18 smithi167 ceph-mon[37140]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-07-26T13:34:18.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:18 smithi138 ceph-mon[36954]: from='osd.7 [v2:172.21.15.138:6816/45541749,v1:172.21.15.138:6817/45541749]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-07-26T13:34:18.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:18 smithi138 ceph-mon[36954]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-07-26T13:34:18.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:18 smithi100 ceph-mon[32599]: from='osd.7 [v2:172.21.15.138:6816/45541749,v1:172.21.15.138:6817/45541749]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-07-26T13:34:18.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:18 smithi100 ceph-mon[32599]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-07-26T13:34:19.263 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:19 smithi167 ceph-mon[37140]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-07-26T13:34:19.263 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:19 smithi167 ceph-mon[37140]: osdmap e46: 8 total, 7 up, 8 in 2024-07-26T13:34:19.263 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-26T13:34:19.263 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:19 smithi167 ceph-mon[37140]: from='osd.7 [v2:172.21.15.138:6816/45541749,v1:172.21.15.138:6817/45541749]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-07-26T13:34:19.263 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:19 smithi167 ceph-mon[37140]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-07-26T13:34:19.263 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:19 smithi167 ceph-mon[37140]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-07-26T13:34:19.268 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:19 smithi138 ceph-mon[36954]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-07-26T13:34:19.268 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:19 smithi138 ceph-mon[36954]: osdmap e46: 8 total, 7 up, 8 in 2024-07-26T13:34:19.268 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-26T13:34:19.268 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:19 smithi138 ceph-mon[36954]: from='osd.7 [v2:172.21.15.138:6816/45541749,v1:172.21.15.138:6817/45541749]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-07-26T13:34:19.268 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:19 smithi138 ceph-mon[36954]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-07-26T13:34:19.268 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:19 smithi138 ceph-mon[36954]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-07-26T13:34:19.268 INFO:journalctl@ceph.osd.7.smithi138.stdout:Jul 26 13:34:19 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-osd-7[56166]: 2024-07-26T13:34:19.160+0000 7faccb5b6640 -1 osd.7 0 waiting for initial osdmap 2024-07-26T13:34:19.268 INFO:journalctl@ceph.osd.7.smithi138.stdout:Jul 26 13:34:19 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-osd-7[56166]: 2024-07-26T13:34:19.169+0000 7facc73b4640 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-26T13:34:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:19 smithi100 ceph-mon[32599]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-07-26T13:34:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:19 smithi100 ceph-mon[32599]: osdmap e46: 8 total, 7 up, 8 in 2024-07-26T13:34:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-26T13:34:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:19 smithi100 ceph-mon[32599]: from='osd.7 [v2:172.21.15.138:6816/45541749,v1:172.21.15.138:6817/45541749]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-07-26T13:34:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:19 smithi100 ceph-mon[32599]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-07-26T13:34:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:19 smithi100 ceph-mon[32599]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-07-26T13:34:19.632 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-07-26T13:34:19.726 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-07-26T13:34:19.739 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2024-07-26T13:34:19.739 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2024-07-26T13:34:19.739 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114143 s, 919 MB/s 2024-07-26T13:34:19.739 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2024-07-26T13:34:20.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:20 smithi167 ceph-mon[37140]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-07-26T13:34:20.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:20 smithi167 ceph-mon[37140]: osdmap e47: 8 total, 7 up, 8 in 2024-07-26T13:34:20.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:20 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-26T13:34:20.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:20 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-26T13:34:20.327 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_2 2024-07-26T13:34:20.372 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:20 smithi138 ceph-mon[36954]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-07-26T13:34:20.372 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:20 smithi138 ceph-mon[36954]: osdmap e47: 8 total, 7 up, 8 in 2024-07-26T13:34:20.372 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:20 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-26T13:34:20.372 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:20 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-26T13:34:20.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:20 smithi100 ceph-mon[32599]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-07-26T13:34:20.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:20 smithi100 ceph-mon[32599]: osdmap e47: 8 total, 7 up, 8 in 2024-07-26T13:34:20.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:20 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-26T13:34:20.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:20 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-26T13:34:20.585 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.c/config 2024-07-26T13:34:21.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:21 smithi167 ceph-mon[37140]: purged_snaps scrub starts 2024-07-26T13:34:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:21 smithi167 ceph-mon[37140]: purged_snaps scrub ok 2024-07-26T13:34:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:21 smithi167 ceph-mon[37140]: osd.7 [v2:172.21.15.138:6816/45541749,v1:172.21.15.138:6817/45541749] boot 2024-07-26T13:34:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:21 smithi167 ceph-mon[37140]: osdmap e48: 8 total, 8 up, 8 in 2024-07-26T13:34:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:21 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-26T13:34:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:21 smithi167 ceph-mon[37140]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-07-26T13:34:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:21 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:21 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:21 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:34:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:21 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:34:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:21 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:34:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:21 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:21 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:34:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:21 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:34:21.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:21 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:21.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:21 smithi138 ceph-mon[36954]: purged_snaps scrub starts 2024-07-26T13:34:21.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:21 smithi138 ceph-mon[36954]: purged_snaps scrub ok 2024-07-26T13:34:21.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:21 smithi138 ceph-mon[36954]: osd.7 [v2:172.21.15.138:6816/45541749,v1:172.21.15.138:6817/45541749] boot 2024-07-26T13:34:21.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:21 smithi138 ceph-mon[36954]: osdmap e48: 8 total, 8 up, 8 in 2024-07-26T13:34:21.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:21 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-26T13:34:21.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:21 smithi138 ceph-mon[36954]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-07-26T13:34:21.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:21 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:21.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:21 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:21.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:21 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:34:21.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:21 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:34:21.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:21 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:34:21.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:21 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:21.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:21 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:34:21.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:21 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:34:21.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:21 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:21 smithi100 ceph-mon[32599]: purged_snaps scrub starts 2024-07-26T13:34:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:21 smithi100 ceph-mon[32599]: purged_snaps scrub ok 2024-07-26T13:34:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:21 smithi100 ceph-mon[32599]: osd.7 [v2:172.21.15.138:6816/45541749,v1:172.21.15.138:6817/45541749] boot 2024-07-26T13:34:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:21 smithi100 ceph-mon[32599]: osdmap e48: 8 total, 8 up, 8 in 2024-07-26T13:34:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:21 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-26T13:34:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:21 smithi100 ceph-mon[32599]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-07-26T13:34:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:21 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:21 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:21 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:34:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:21 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:34:21.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:21 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:34:21.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:21 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:21.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:21 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:34:21.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:21 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:34:21.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:21 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:22.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.188+0000 7fa203c0c640 1 -- 172.21.15.167:0/3610909592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1fc108ee0 msgr2=0x7fa1fc1092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:34:22.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.188+0000 7fa203c0c640 1 --2- 172.21.15.167:0/3610909592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1fc108ee0 0x7fa1fc1092f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009900 tx=0x7fa1ec02a410 comp rx=0 tx=0).stop 2024-07-26T13:34:22.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.188+0000 7fa203c0c640 1 -- 172.21.15.167:0/3610909592 shutdown_connections 2024-07-26T13:34:22.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.188+0000 7fa203c0c640 1 --2- 172.21.15.167:0/3610909592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1fc109830 0x7fa1fc110060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:34:22.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.188+0000 7fa203c0c640 1 --2- 172.21.15.167:0/3610909592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1fc108ee0 0x7fa1fc1092f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:34:22.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.188+0000 7fa203c0c640 1 --2- 172.21.15.167:0/3610909592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1fc1035f0 0x7fa1fc103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:34:22.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.188+0000 7fa203c0c640 1 -- 172.21.15.167:0/3610909592 >> 172.21.15.167:0/3610909592 conn(0x7fa1fc07b340 msgr2=0x7fa1fc07b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:34:22.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.188+0000 7fa203c0c640 1 -- 172.21.15.167:0/3610909592 shutdown_connections 2024-07-26T13:34:22.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.188+0000 7fa203c0c640 1 -- 172.21.15.167:0/3610909592 wait complete. 2024-07-26T13:34:22.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.189+0000 7fa203c0c640 1 Processor -- start 2024-07-26T13:34:22.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.189+0000 7fa203c0c640 1 -- start start 2024-07-26T13:34:22.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.189+0000 7fa203c0c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1fc1035f0 0x7fa1fc199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:34:22.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.189+0000 7fa203c0c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1fc108ee0 0x7fa1fc19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:34:22.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.189+0000 7fa203c0c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1fc109830 0x7fa1fc19f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:34:22.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.189+0000 7fa203c0c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc113510 con 0x7fa1fc108ee0 2024-07-26T13:34:22.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.189+0000 7fa203c0c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc113390 con 0x7fa1fc1035f0 2024-07-26T13:34:22.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.189+0000 7fa203c0c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc113690 con 0x7fa1fc109830 2024-07-26T13:34:22.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.190+0000 7fa2021a9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1fc109830 0x7fa1fc19f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:34:22.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.190+0000 7fa2021a9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1fc109830 0x7fa1fc19f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:38472/0 (socket says 172.21.15.167:38472) 2024-07-26T13:34:22.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.190+0000 7fa2021a9640 1 -- 172.21.15.167:0/743493509 learned_addr learned my addr 172.21.15.167:0/743493509 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-07-26T13:34:22.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.190+0000 7fa2019a8640 1 --2- 172.21.15.167:0/743493509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1fc1035f0 0x7fa1fc199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:34:22.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.190+0000 7fa2011a7640 1 --2- 172.21.15.167:0/743493509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1fc108ee0 0x7fa1fc19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:34:22.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.190+0000 7fa2021a9640 1 -- 172.21.15.167:0/743493509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1fc1035f0 msgr2=0x7fa1fc199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:34:22.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.190+0000 7fa2021a9640 1 --2- 172.21.15.167:0/743493509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1fc1035f0 0x7fa1fc199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:34:22.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.190+0000 7fa2021a9640 1 -- 172.21.15.167:0/743493509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1fc108ee0 msgr2=0x7fa1fc19a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:34:22.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.190+0000 7fa2021a9640 1 --2- 172.21.15.167:0/743493509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1fc108ee0 0x7fa1fc19a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:34:22.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.190+0000 7fa2021a9640 1 -- 172.21.15.167:0/743493509 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1fc19fe10 con 0x7fa1fc109830 2024-07-26T13:34:22.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.190+0000 7fa2011a7640 1 --2- 172.21.15.167:0/743493509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1fc108ee0 0x7fa1fc19a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:34:22.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.190+0000 7fa2021a9640 1 --2- 172.21.15.167:0/743493509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1fc109830 0x7fa1fc19f6e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa1f400eb10 tx=0x7fa1f400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:34:22.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.190+0000 7fa1f2ffd640 1 -- 172.21.15.167:0/743493509 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4004b20 con 0x7fa1fc109830 2024-07-26T13:34:22.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.190+0000 7fa203c0c640 1 -- 172.21.15.167:0/743493509 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1fc1a0100 con 0x7fa1fc109830 2024-07-26T13:34:22.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.190+0000 7fa1f2ffd640 1 -- 172.21.15.167:0/743493509 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1f4004cc0 con 0x7fa1fc109830 2024-07-26T13:34:22.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.190+0000 7fa1f2ffd640 1 -- 172.21.15.167:0/743493509 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4010670 con 0x7fa1fc109830 2024-07-26T13:34:22.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.190+0000 7fa203c0c640 1 -- 172.21.15.167:0/743493509 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1fc1a6860 con 0x7fa1fc109830 2024-07-26T13:34:22.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.191+0000 7fa203c0c640 1 -- 172.21.15.167:0/743493509 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1fc103a00 con 0x7fa1fc109830 2024-07-26T13:34:22.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.193+0000 7fa1f2ffd640 1 -- 172.21.15.167:0/743493509 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7fa1f4005020 con 0x7fa1fc109830 2024-07-26T13:34:22.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.194+0000 7fa1f2ffd640 1 --2- 172.21.15.167:0/743493509 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fa1e80b3c20 0x7fa1e80b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:34:22.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.194+0000 7fa1f2ffd640 1 -- 172.21.15.167:0/743493509 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5612+0+0 (secure 0 0 0) 0x7fa1f40e3730 con 0x7fa1fc109830 2024-07-26T13:34:22.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.194+0000 7fa2019a8640 1 --2- 172.21.15.167:0/743493509 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fa1e80b3c20 0x7fa1e80b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:34:22.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.194+0000 7fa2019a8640 1 --2- 172.21.15.167:0/743493509 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fa1e80b3c20 0x7fa1e80b6130 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8005d00 tx=0x7fa1f800b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:34:22.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.198+0000 7fa1f2ffd640 1 -- 172.21.15.167:0/743493509 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fa1f40ab810 con 0x7fa1fc109830 2024-07-26T13:34:22.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:22 smithi167 ceph-mon[37140]: Detected new or changed devices on smithi138 2024-07-26T13:34:22.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:22 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi138 to 5696M 2024-07-26T13:34:22.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:22 smithi167 ceph-mon[37140]: osdmap e49: 8 total, 8 up, 8 in 2024-07-26T13:34:22.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:22.382+0000 7fa203c0c640 1 -- 172.21.15.167:0/743493509 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fa1fc063230 con 0x7fa1e80b3c20 2024-07-26T13:34:22.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:22 smithi138 ceph-mon[36954]: Detected new or changed devices on smithi138 2024-07-26T13:34:22.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:22 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi138 to 5696M 2024-07-26T13:34:22.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:22 smithi138 ceph-mon[36954]: osdmap e49: 8 total, 8 up, 8 in 2024-07-26T13:34:22.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:22 smithi100 ceph-mon[32599]: Detected new or changed devices on smithi138 2024-07-26T13:34:22.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:22 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi138 to 5696M 2024-07-26T13:34:22.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:22 smithi100 ceph-mon[32599]: osdmap e49: 8 total, 8 up, 8 in 2024-07-26T13:34:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:23 smithi138 ceph-mon[36954]: osdmap e50: 8 total, 8 up, 8 in 2024-07-26T13:34:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:23 smithi138 ceph-mon[36954]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-07-26T13:34:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:23 smithi138 ceph-mon[36954]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:34:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-26T13:34:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-26T13:34:23.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:34:23.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:23 smithi167 ceph-mon[37140]: osdmap e50: 8 total, 8 up, 8 in 2024-07-26T13:34:23.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:23 smithi167 ceph-mon[37140]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-07-26T13:34:23.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:23 smithi167 ceph-mon[37140]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:34:23.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-26T13:34:23.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-26T13:34:23.483 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:34:23.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:23 smithi100 ceph-mon[32599]: osdmap e50: 8 total, 8 up, 8 in 2024-07-26T13:34:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:23 smithi100 ceph-mon[32599]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-07-26T13:34:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:23 smithi100 ceph-mon[32599]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:34:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-26T13:34:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-26T13:34:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:34:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:25 smithi138 ceph-mon[36954]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-07-26T13:34:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:25 smithi138 ceph-mon[36954]: from='client.? 172.21.15.167:0/1649254098' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e865858-92cc-48a0-a464-5767d19e8ac0"}]: dispatch 2024-07-26T13:34:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:25 smithi138 ceph-mon[36954]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e865858-92cc-48a0-a464-5767d19e8ac0"}]: dispatch 2024-07-26T13:34:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:25 smithi138 ceph-mon[36954]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e865858-92cc-48a0-a464-5767d19e8ac0"}]': finished 2024-07-26T13:34:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:25 smithi138 ceph-mon[36954]: osdmap e51: 9 total, 8 up, 9 in 2024-07-26T13:34:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-26T13:34:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:25 smithi100 ceph-mon[32599]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-07-26T13:34:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:25 smithi100 ceph-mon[32599]: from='client.? 172.21.15.167:0/1649254098' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e865858-92cc-48a0-a464-5767d19e8ac0"}]: dispatch 2024-07-26T13:34:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:25 smithi100 ceph-mon[32599]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e865858-92cc-48a0-a464-5767d19e8ac0"}]: dispatch 2024-07-26T13:34:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:25 smithi100 ceph-mon[32599]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e865858-92cc-48a0-a464-5767d19e8ac0"}]': finished 2024-07-26T13:34:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:25 smithi100 ceph-mon[32599]: osdmap e51: 9 total, 8 up, 9 in 2024-07-26T13:34:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-26T13:34:25.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:25 smithi167 ceph-mon[37140]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-07-26T13:34:25.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:25 smithi167 ceph-mon[37140]: from='client.? 172.21.15.167:0/1649254098' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e865858-92cc-48a0-a464-5767d19e8ac0"}]: dispatch 2024-07-26T13:34:25.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:25 smithi167 ceph-mon[37140]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e865858-92cc-48a0-a464-5767d19e8ac0"}]: dispatch 2024-07-26T13:34:25.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:25 smithi167 ceph-mon[37140]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e865858-92cc-48a0-a464-5767d19e8ac0"}]': finished 2024-07-26T13:34:25.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:25 smithi167 ceph-mon[37140]: osdmap e51: 9 total, 8 up, 9 in 2024-07-26T13:34:25.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-26T13:34:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:26 smithi138 ceph-mon[36954]: from='client.? 172.21.15.167:0/3121148058' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-26T13:34:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:26 smithi100 ceph-mon[32599]: from='client.? 172.21.15.167:0/3121148058' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-26T13:34:26.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:26 smithi167 ceph-mon[37140]: from='client.? 172.21.15.167:0/3121148058' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-26T13:34:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:27 smithi138 ceph-mon[36954]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-26T13:34:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:27 smithi100 ceph-mon[32599]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-26T13:34:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:27 smithi167 ceph-mon[37140]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-26T13:34:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:29 smithi138 ceph-mon[36954]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-07-26T13:34:29.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:29 smithi167 ceph-mon[37140]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-07-26T13:34:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:29 smithi100 ceph-mon[32599]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-07-26T13:34:31.613 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:31 smithi167 ceph-mon[37140]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-07-26T13:34:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:31 smithi138 ceph-mon[36954]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-07-26T13:34:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:31 smithi100 ceph-mon[32599]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-07-26T13:34:32.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-07-26T13:34:32.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:34:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-07-26T13:34:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:34:32.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-07-26T13:34:32.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:34:33.591 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:33 smithi167 ceph-mon[37140]: Deploying daemon osd.8 on smithi167 2024-07-26T13:34:33.591 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:33 smithi167 ceph-mon[37140]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-26T13:34:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:33 smithi138 ceph-mon[36954]: Deploying daemon osd.8 on smithi167 2024-07-26T13:34:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:33 smithi138 ceph-mon[36954]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-26T13:34:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:33 smithi100 ceph-mon[32599]: Deploying daemon osd.8 on smithi167 2024-07-26T13:34:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:33 smithi100 ceph-mon[32599]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-26T13:34:35.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:35 smithi167 ceph-mon[37140]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-26T13:34:35.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:35 smithi138 ceph-mon[36954]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-26T13:34:35.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:35 smithi100 ceph-mon[32599]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-26T13:34:37.592 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:37 smithi167 ceph-mon[37140]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-07-26T13:34:37.592 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:34:37.593 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:37.593 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:37.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:37 smithi138 ceph-mon[36954]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-07-26T13:34:37.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:34:37.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:37.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:37 smithi100 ceph-mon[32599]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-07-26T13:34:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:34:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:39.658 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:39 smithi167 ceph-mon[37140]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-07-26T13:34:39.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:39 smithi138 ceph-mon[36954]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-07-26T13:34:39.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:39 smithi100 ceph-mon[32599]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-07-26T13:34:40.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:40.463+0000 7fa1f2ffd640 1 -- 172.21.15.167:0/743493509 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa1fc063230 con 0x7fa1e80b3c20 2024-07-26T13:34:40.464 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 8 on host 'smithi167' 2024-07-26T13:34:40.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:40.465+0000 7fa203c0c640 1 -- 172.21.15.167:0/743493509 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fa1e80b3c20 msgr2=0x7fa1e80b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:34:40.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:40.465+0000 7fa203c0c640 1 --2- 172.21.15.167:0/743493509 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fa1e80b3c20 0x7fa1e80b6130 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8005d00 tx=0x7fa1f800b410 comp rx=0 tx=0).stop 2024-07-26T13:34:40.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:40.465+0000 7fa203c0c640 1 -- 172.21.15.167:0/743493509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1fc109830 msgr2=0x7fa1fc19f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:34:40.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:40.465+0000 7fa203c0c640 1 --2- 172.21.15.167:0/743493509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1fc109830 0x7fa1fc19f6e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa1f400eb10 tx=0x7fa1f400c2d0 comp rx=0 tx=0).stop 2024-07-26T13:34:40.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:40.465+0000 7fa203c0c640 1 -- 172.21.15.167:0/743493509 shutdown_connections 2024-07-26T13:34:40.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:40.465+0000 7fa203c0c640 1 --2- 172.21.15.167:0/743493509 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fa1e80b3c20 0x7fa1e80b6130 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:34:40.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:40.465+0000 7fa203c0c640 1 --2- 172.21.15.167:0/743493509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1fc109830 0x7fa1fc19f6e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:34:40.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:40.465+0000 7fa203c0c640 1 --2- 172.21.15.167:0/743493509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1fc108ee0 0x7fa1fc19a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:34:40.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:40.465+0000 7fa203c0c640 1 --2- 172.21.15.167:0/743493509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1fc1035f0 0x7fa1fc199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:34:40.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:40.465+0000 7fa203c0c640 1 -- 172.21.15.167:0/743493509 >> 172.21.15.167:0/743493509 conn(0x7fa1fc07b340 msgr2=0x7fa1fc106f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:34:40.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:40.466+0000 7fa203c0c640 1 -- 172.21.15.167:0/743493509 shutdown_connections 2024-07-26T13:34:40.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:34:40.466+0000 7fa203c0c640 1 -- 172.21.15.167:0/743493509 wait complete. 2024-07-26T13:34:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:34:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:34:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:34:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:40.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:40.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:40.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:34:40.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:34:40.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:40.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:34:40.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:40.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:34:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:34:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:34:40.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:40.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:41.061 DEBUG:teuthology.orchestra.run.smithi167:osd.8> sudo journalctl -f -n 0 -u ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@osd.8.service 2024-07-26T13:34:41.064 INFO:tasks.cephadm:Deploying osd.9 on smithi100 with /dev/vg_nvme/lv_1... 2024-07-26T13:34:41.064 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-07-26T13:34:41.330 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:34:41.604 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:41 smithi167 ceph-mon[37140]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-07-26T13:34:41.604 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:41 smithi167 ceph-mon[37140]: from='osd.8 [v2:172.21.15.167:6816/4031159803,v1:172.21.15.167:6817/4031159803]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-07-26T13:34:41.604 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:41 smithi167 ceph-mon[37140]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-07-26T13:34:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:41 smithi100 ceph-mon[32599]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-07-26T13:34:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:41 smithi100 ceph-mon[32599]: from='osd.8 [v2:172.21.15.167:6816/4031159803,v1:172.21.15.167:6817/4031159803]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-07-26T13:34:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:41 smithi100 ceph-mon[32599]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-07-26T13:34:41.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:41 smithi138 ceph-mon[36954]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-07-26T13:34:41.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:41 smithi138 ceph-mon[36954]: from='osd.8 [v2:172.21.15.167:6816/4031159803,v1:172.21.15.167:6817/4031159803]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-07-26T13:34:41.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:41 smithi138 ceph-mon[36954]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-07-26T13:34:42.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:42 smithi100 ceph-mon[32599]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-07-26T13:34:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:42 smithi100 ceph-mon[32599]: osdmap e52: 9 total, 8 up, 9 in 2024-07-26T13:34:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:42 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-26T13:34:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:42 smithi100 ceph-mon[32599]: from='osd.8 [v2:172.21.15.167:6816/4031159803,v1:172.21.15.167:6817/4031159803]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-07-26T13:34:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:42 smithi100 ceph-mon[32599]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-07-26T13:34:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:42 smithi100 ceph-mon[32599]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-07-26T13:34:42.755 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:42 smithi167 ceph-mon[37140]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-07-26T13:34:42.755 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:42 smithi167 ceph-mon[37140]: osdmap e52: 9 total, 8 up, 9 in 2024-07-26T13:34:42.755 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:42 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-26T13:34:42.755 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:42 smithi167 ceph-mon[37140]: from='osd.8 [v2:172.21.15.167:6816/4031159803,v1:172.21.15.167:6817/4031159803]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-07-26T13:34:42.756 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:42 smithi167 ceph-mon[37140]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-07-26T13:34:42.756 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:42 smithi167 ceph-mon[37140]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-07-26T13:34:42.756 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jul 26 13:34:42 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-osd-8[57022]: 2024-07-26T13:34:42.503+0000 7f7df05d1640 -1 osd.8 0 waiting for initial osdmap 2024-07-26T13:34:42.756 INFO:journalctl@ceph.osd.8.smithi167.stdout:Jul 26 13:34:42 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-osd-8[57022]: 2024-07-26T13:34:42.513+0000 7f7dec3cf640 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-26T13:34:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:42 smithi138 ceph-mon[36954]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-07-26T13:34:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:42 smithi138 ceph-mon[36954]: osdmap e52: 9 total, 8 up, 9 in 2024-07-26T13:34:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:42 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-26T13:34:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:42 smithi138 ceph-mon[36954]: from='osd.8 [v2:172.21.15.167:6816/4031159803,v1:172.21.15.167:6817/4031159803]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-07-26T13:34:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:42 smithi138 ceph-mon[36954]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-07-26T13:34:42.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:42 smithi138 ceph-mon[36954]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-07-26T13:34:43.176 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-07-26T13:34:43.260 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-07-26T13:34:43.273 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-07-26T13:34:43.273 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-07-26T13:34:43.274 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115143 s, 911 MB/s 2024-07-26T13:34:43.274 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-07-26T13:34:43.758 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:43 smithi167 ceph-mon[37140]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-07-26T13:34:43.758 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:43 smithi167 ceph-mon[37140]: osdmap e53: 9 total, 8 up, 9 in 2024-07-26T13:34:43.758 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:43 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-26T13:34:43.758 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:43 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-26T13:34:43.759 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:43 smithi167 ceph-mon[37140]: osd.8 [v2:172.21.15.167:6816/4031159803,v1:172.21.15.167:6817/4031159803] boot 2024-07-26T13:34:43.759 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:43 smithi167 ceph-mon[37140]: osdmap e54: 9 total, 9 up, 9 in 2024-07-26T13:34:43.759 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:43 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-26T13:34:43.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:43 smithi138 ceph-mon[36954]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-07-26T13:34:43.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:43 smithi138 ceph-mon[36954]: osdmap e53: 9 total, 8 up, 9 in 2024-07-26T13:34:43.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:43 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-26T13:34:43.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:43 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-26T13:34:43.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:43 smithi138 ceph-mon[36954]: osd.8 [v2:172.21.15.167:6816/4031159803,v1:172.21.15.167:6817/4031159803] boot 2024-07-26T13:34:43.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:43 smithi138 ceph-mon[36954]: osdmap e54: 9 total, 9 up, 9 in 2024-07-26T13:34:43.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:43 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-26T13:34:43.959 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:43 smithi100 ceph-mon[32599]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-07-26T13:34:43.960 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:43 smithi100 ceph-mon[32599]: osdmap e53: 9 total, 8 up, 9 in 2024-07-26T13:34:43.960 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:43 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-26T13:34:43.960 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:43 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-26T13:34:43.960 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:43 smithi100 ceph-mon[32599]: osd.8 [v2:172.21.15.167:6816/4031159803,v1:172.21.15.167:6817/4031159803] boot 2024-07-26T13:34:43.960 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:43 smithi100 ceph-mon[32599]: osdmap e54: 9 total, 9 up, 9 in 2024-07-26T13:34:43.960 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:43 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-26T13:34:43.988 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph orch daemon add osd smithi100:vg_nvme/lv_1 2024-07-26T13:34:44.261 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:34:44.722 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:44 smithi100 ceph-mon[32599]: purged_snaps scrub starts 2024-07-26T13:34:44.722 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:44 smithi100 ceph-mon[32599]: purged_snaps scrub ok 2024-07-26T13:34:44.722 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:44 smithi100 ceph-mon[32599]: Detected new or changed devices on smithi167 2024-07-26T13:34:44.722 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:44.722 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:44.722 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:34:44.722 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:34:44.722 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:34:44.722 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:44 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi167 to 5696M 2024-07-26T13:34:44.722 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:44.723 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:34:44.723 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:34:44.723 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:44.723 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:44 smithi100 ceph-mon[32599]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-07-26T13:34:44.723 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:44 smithi100 ceph-mon[32599]: osdmap e55: 9 total, 9 up, 9 in 2024-07-26T13:34:44.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:44 smithi167 ceph-mon[37140]: purged_snaps scrub starts 2024-07-26T13:34:44.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:44 smithi167 ceph-mon[37140]: purged_snaps scrub ok 2024-07-26T13:34:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:44 smithi167 ceph-mon[37140]: Detected new or changed devices on smithi167 2024-07-26T13:34:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:34:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:34:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:34:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:44 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi167 to 5696M 2024-07-26T13:34:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:34:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:34:44.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:44.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:44 smithi167 ceph-mon[37140]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-07-26T13:34:44.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:44 smithi167 ceph-mon[37140]: osdmap e55: 9 total, 9 up, 9 in 2024-07-26T13:34:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:44 smithi138 ceph-mon[36954]: purged_snaps scrub starts 2024-07-26T13:34:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:44 smithi138 ceph-mon[36954]: purged_snaps scrub ok 2024-07-26T13:34:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:44 smithi138 ceph-mon[36954]: Detected new or changed devices on smithi167 2024-07-26T13:34:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:44.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:44.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:34:44.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:34:44.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:34:44.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:44 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi167 to 5696M 2024-07-26T13:34:44.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:44.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:34:44.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:34:44.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:34:44.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:44 smithi138 ceph-mon[36954]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-07-26T13:34:44.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:44 smithi138 ceph-mon[36954]: osdmap e55: 9 total, 9 up, 9 in 2024-07-26T13:34:45.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.870+0000 7fee03f2b640 1 -- 172.21.15.100:0/2890448644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fedfc10fc30 msgr2=0x7fedfc112050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:34:45.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.870+0000 7fee03f2b640 1 --2- 172.21.15.100:0/2890448644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fedfc10fc30 0x7fedfc112050 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fedf4009ca0 tx=0x7fedf402a950 comp rx=0 tx=0).stop 2024-07-26T13:34:45.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.870+0000 7fee03f2b640 1 -- 172.21.15.100:0/2890448644 shutdown_connections 2024-07-26T13:34:45.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.870+0000 7fee03f2b640 1 --2- 172.21.15.100:0/2890448644 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fedfc10fc30 0x7fedfc112050 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:34:45.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.870+0000 7fee03f2b640 1 --2- 172.21.15.100:0/2890448644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedfc1029d0 0x7fedfc10f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:34:45.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.870+0000 7fee03f2b640 1 --2- 172.21.15.100:0/2890448644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedfc102080 0x7fedfc102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:34:45.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.870+0000 7fee03f2b640 1 -- 172.21.15.100:0/2890448644 >> 172.21.15.100:0/2890448644 conn(0x7fedfc079190 msgr2=0x7fedfc0795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:34:45.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.870+0000 7fee03f2b640 1 -- 172.21.15.100:0/2890448644 shutdown_connections 2024-07-26T13:34:45.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.870+0000 7fee03f2b640 1 -- 172.21.15.100:0/2890448644 wait complete. 2024-07-26T13:34:45.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.871+0000 7fee03f2b640 1 Processor -- start 2024-07-26T13:34:45.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.871+0000 7fee03f2b640 1 -- start start 2024-07-26T13:34:45.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.871+0000 7fee03f2b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedfc102080 0x7fedfc19c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:34:45.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.871+0000 7fee03f2b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedfc1029d0 0x7fedfc19c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:34:45.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.871+0000 7fee03f2b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fedfc10fc30 0x7fedfc1a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:34:45.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.871+0000 7fee03f2b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc1156a0 con 0x7fedfc10fc30 2024-07-26T13:34:45.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.871+0000 7fee03f2b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc115520 con 0x7fedfc102080 2024-07-26T13:34:45.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.871+0000 7fee03f2b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc115820 con 0x7fedfc1029d0 2024-07-26T13:34:45.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.871+0000 7fee024c8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fedfc10fc30 0x7fedfc1a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:34:45.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.871+0000 7fee024c8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fedfc10fc30 0x7fedfc1a1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46900/0 (socket says 172.21.15.100:46900) 2024-07-26T13:34:45.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.871+0000 7fee024c8640 1 -- 172.21.15.100:0/1344664830 learned_addr learned my addr 172.21.15.100:0/1344664830 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:34:45.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.871+0000 7fee01cc7640 1 --2- 172.21.15.100:0/1344664830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedfc102080 0x7fedfc19c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:34:45.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.871+0000 7fee014c6640 1 --2- 172.21.15.100:0/1344664830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedfc1029d0 0x7fedfc19c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:34:45.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.871+0000 7fee024c8640 1 -- 172.21.15.100:0/1344664830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedfc1029d0 msgr2=0x7fedfc19c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:34:45.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.871+0000 7fee024c8640 1 --2- 172.21.15.100:0/1344664830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedfc1029d0 0x7fedfc19c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:34:45.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.871+0000 7fee024c8640 1 -- 172.21.15.100:0/1344664830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedfc102080 msgr2=0x7fedfc19c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:34:45.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.871+0000 7fee024c8640 1 --2- 172.21.15.100:0/1344664830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedfc102080 0x7fedfc19c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:34:45.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.871+0000 7fee024c8640 1 -- 172.21.15.100:0/1344664830 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedfc1a1fc0 con 0x7fedfc10fc30 2024-07-26T13:34:45.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.871+0000 7fee014c6640 1 --2- 172.21.15.100:0/1344664830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedfc1029d0 0x7fedfc19c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:34:45.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.872+0000 7fee024c8640 1 --2- 172.21.15.100:0/1344664830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fedfc10fc30 0x7fedfc1a1890 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fedf402ae70 tx=0x7fedf402f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:34:45.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.872+0000 7fedf2ffd640 1 -- 172.21.15.100:0/1344664830 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf4034040 con 0x7fedfc10fc30 2024-07-26T13:34:45.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.872+0000 7fee03f2b640 1 -- 172.21.15.100:0/1344664830 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedfc1a2250 con 0x7fedfc10fc30 2024-07-26T13:34:45.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.872+0000 7fedf2ffd640 1 -- 172.21.15.100:0/1344664830 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedf40308f0 con 0x7fedfc10fc30 2024-07-26T13:34:45.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.872+0000 7fedf2ffd640 1 -- 172.21.15.100:0/1344664830 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf40047f0 con 0x7fedfc10fc30 2024-07-26T13:34:45.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.872+0000 7fee03f2b640 1 -- 172.21.15.100:0/1344664830 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedfc1a8a10 con 0x7fedfc10fc30 2024-07-26T13:34:45.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.873+0000 7fee03f2b640 1 -- 172.21.15.100:0/1344664830 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedfc06bc20 con 0x7fedfc10fc30 2024-07-26T13:34:45.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.876+0000 7fedf2ffd640 1 -- 172.21.15.100:0/1344664830 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7fedf4037020 con 0x7fedfc10fc30 2024-07-26T13:34:45.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.876+0000 7fedf2ffd640 1 --2- 172.21.15.100:0/1344664830 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fede80b3c20 0x7fede80b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:34:45.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.876+0000 7fee01cc7640 1 --2- 172.21.15.100:0/1344664830 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fede80b3c20 0x7fede80b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:34:45.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.876+0000 7fedf2ffd640 1 -- 172.21.15.100:0/1344664830 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6044+0+0 (secure 0 0 0) 0x7fedf41044d0 con 0x7fedfc10fc30 2024-07-26T13:34:45.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.877+0000 7fee01cc7640 1 --2- 172.21.15.100:0/1344664830 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fede80b3c20 0x7fede80b6130 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fedf800bbe0 tx=0x7fedf800b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:34:45.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:45.879+0000 7fedf2ffd640 1 -- 172.21.15.100:0/1344664830 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fedf40cc490 con 0x7fedfc10fc30 2024-07-26T13:34:46.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:34:46.063+0000 7fee03f2b640 1 -- 172.21.15.100:0/1344664830 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fedfc108db0 con 0x7fede80b3c20 2024-07-26T13:34:46.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:46 smithi100 ceph-mon[32599]: osdmap e56: 9 total, 9 up, 9 in 2024-07-26T13:34:46.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-26T13:34:46.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-26T13:34:46.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:34:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:46 smithi167 ceph-mon[37140]: osdmap e56: 9 total, 9 up, 9 in 2024-07-26T13:34:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-26T13:34:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-26T13:34:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:34:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:46 smithi138 ceph-mon[36954]: osdmap e56: 9 total, 9 up, 9 in 2024-07-26T13:34:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-26T13:34:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-26T13:34:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:34:47.734 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:47 smithi100 ceph-mon[32599]: from='client.14625 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:34:47.734 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:47 smithi100 ceph-mon[32599]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-07-26T13:34:47.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:47 smithi167 ceph-mon[37140]: from='client.14625 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:34:47.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:47 smithi167 ceph-mon[37140]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-07-26T13:34:47.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:47 smithi138 ceph-mon[36954]: from='client.14625 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:34:47.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:47 smithi138 ceph-mon[36954]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-07-26T13:34:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:48 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/908836772' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b05b14a-1195-42f5-ac01-d5634cb9c888"}]: dispatch 2024-07-26T13:34:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:48 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/908836772' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8b05b14a-1195-42f5-ac01-d5634cb9c888"}]': finished 2024-07-26T13:34:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:48 smithi100 ceph-mon[32599]: osdmap e57: 10 total, 9 up, 10 in 2024-07-26T13:34:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:48 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-26T13:34:48.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:48 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/908836772' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b05b14a-1195-42f5-ac01-d5634cb9c888"}]: dispatch 2024-07-26T13:34:48.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:48 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/908836772' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8b05b14a-1195-42f5-ac01-d5634cb9c888"}]': finished 2024-07-26T13:34:48.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:48 smithi167 ceph-mon[37140]: osdmap e57: 10 total, 9 up, 10 in 2024-07-26T13:34:48.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:48 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-26T13:34:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:48 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/908836772' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b05b14a-1195-42f5-ac01-d5634cb9c888"}]: dispatch 2024-07-26T13:34:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:48 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/908836772' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8b05b14a-1195-42f5-ac01-d5634cb9c888"}]': finished 2024-07-26T13:34:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:48 smithi138 ceph-mon[36954]: osdmap e57: 10 total, 9 up, 10 in 2024-07-26T13:34:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:48 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-26T13:34:49.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:49 smithi100 ceph-mon[32599]: pgmap v180: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-07-26T13:34:49.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:49 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/73096384' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-26T13:34:49.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:49 smithi167 ceph-mon[37140]: pgmap v180: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-07-26T13:34:49.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:49 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/73096384' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-26T13:34:49.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:49 smithi138 ceph-mon[36954]: pgmap v180: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-07-26T13:34:49.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:49 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/73096384' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-26T13:34:50.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:50 smithi167 ceph-mon[37140]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-26T13:34:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:50 smithi138 ceph-mon[36954]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-26T13:34:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:50 smithi100 ceph-mon[32599]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-26T13:34:53.469 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:53 smithi100 ceph-mon[32599]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-07-26T13:34:53.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:53 smithi138 ceph-mon[36954]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-07-26T13:34:53.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:53 smithi167 ceph-mon[37140]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-07-26T13:34:55.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:55 smithi138 ceph-mon[36954]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-07-26T13:34:55.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:55 smithi100 ceph-mon[32599]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-07-26T13:34:55.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:55 smithi167 ceph-mon[37140]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-07-26T13:34:56.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-07-26T13:34:56.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:34:56.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:56 smithi100 ceph-mon[32599]: Deploying daemon osd.9 on smithi100 2024-07-26T13:34:56.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-07-26T13:34:56.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:34:56.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:56 smithi138 ceph-mon[36954]: Deploying daemon osd.9 on smithi100 2024-07-26T13:34:56.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-07-26T13:34:56.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:34:56.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:56 smithi167 ceph-mon[37140]: Deploying daemon osd.9 on smithi100 2024-07-26T13:34:57.476 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:57 smithi100 ceph-mon[32599]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-26T13:34:57.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:57 smithi138 ceph-mon[36954]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-26T13:34:57.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:57 smithi167 ceph-mon[37140]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-26T13:34:59.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:34:59 smithi138 ceph-mon[36954]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-26T13:34:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:34:59 smithi100 ceph-mon[32599]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-26T13:34:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:34:59 smithi167 ceph-mon[37140]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-26T13:35:01.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:01 smithi100 ceph-mon[32599]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-07-26T13:35:01.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:35:01.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:01.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:01.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:01 smithi138 ceph-mon[36954]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-07-26T13:35:01.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:35:01.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:01.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:01.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:01 smithi167 ceph-mon[37140]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-07-26T13:35:01.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:35:01.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:01.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:03.529 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:03 smithi100 ceph-mon[32599]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-07-26T13:35:03.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:03 smithi138 ceph-mon[36954]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-07-26T13:35:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:03 smithi167 ceph-mon[37140]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-07-26T13:35:04.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:04.454+0000 7fedf2ffd640 1 -- 172.21.15.100:0/1344664830 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fedfc108db0 con 0x7fede80b3c20 2024-07-26T13:35:04.456 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 9 on host 'smithi100' 2024-07-26T13:35:04.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:04.458+0000 7fee03f2b640 1 -- 172.21.15.100:0/1344664830 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fede80b3c20 msgr2=0x7fede80b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:35:04.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:04.458+0000 7fee03f2b640 1 --2- 172.21.15.100:0/1344664830 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fede80b3c20 0x7fede80b6130 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fedf800bbe0 tx=0x7fedf800b390 comp rx=0 tx=0).stop 2024-07-26T13:35:04.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:04.458+0000 7fee03f2b640 1 -- 172.21.15.100:0/1344664830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fedfc10fc30 msgr2=0x7fedfc1a1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:35:04.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:04.458+0000 7fee03f2b640 1 --2- 172.21.15.100:0/1344664830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fedfc10fc30 0x7fedfc1a1890 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fedf402ae70 tx=0x7fedf402f5d0 comp rx=0 tx=0).stop 2024-07-26T13:35:04.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:04.458+0000 7fee03f2b640 1 -- 172.21.15.100:0/1344664830 shutdown_connections 2024-07-26T13:35:04.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:04.458+0000 7fee03f2b640 1 --2- 172.21.15.100:0/1344664830 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fede80b3c20 0x7fede80b6130 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:35:04.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:04.458+0000 7fee03f2b640 1 --2- 172.21.15.100:0/1344664830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fedfc10fc30 0x7fedfc1a1890 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:35:04.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:04.458+0000 7fee03f2b640 1 --2- 172.21.15.100:0/1344664830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedfc1029d0 0x7fedfc19c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:35:04.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:04.458+0000 7fee03f2b640 1 --2- 172.21.15.100:0/1344664830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedfc102080 0x7fedfc19c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:35:04.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:04.458+0000 7fee03f2b640 1 -- 172.21.15.100:0/1344664830 >> 172.21.15.100:0/1344664830 conn(0x7fedfc079190 msgr2=0x7fedfc1086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:35:04.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:04.458+0000 7fee03f2b640 1 -- 172.21.15.100:0/1344664830 shutdown_connections 2024-07-26T13:35:04.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:04.458+0000 7fee03f2b640 1 -- 172.21.15.100:0/1344664830 wait complete. 2024-07-26T13:35:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:35:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:35:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:04.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:04 smithi100 ceph-mon[32599]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-07-26T13:35:04.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:35:04.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:04.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:35:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:35:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:04 smithi167 ceph-mon[37140]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-07-26T13:35:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:35:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:05.072 DEBUG:teuthology.orchestra.run.smithi100:osd.9> sudo journalctl -f -n 0 -u ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@osd.9.service 2024-07-26T13:35:05.074 INFO:tasks.cephadm:Deploying osd.10 on smithi138 with /dev/vg_nvme/lv_1... 2024-07-26T13:35:05.074 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-07-26T13:35:05.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:05.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:05.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:35:05.100 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:35:05.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:05.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:04 smithi138 ceph-mon[36954]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-07-26T13:35:05.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:35:05.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:05.101 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:05.334 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.b/config 2024-07-26T13:35:05.733 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:05 smithi138 ceph-mon[36954]: from='osd.9 [v2:172.21.15.100:6826/1840140273,v1:172.21.15.100:6827/1840140273]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-07-26T13:35:05.882 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:05 smithi100 ceph-mon[32599]: from='osd.9 [v2:172.21.15.100:6826/1840140273,v1:172.21.15.100:6827/1840140273]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-07-26T13:35:06.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:05 smithi167 ceph-mon[37140]: from='osd.9 [v2:172.21.15.100:6826/1840140273,v1:172.21.15.100:6827/1840140273]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-07-26T13:35:06.822 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:06 smithi138 ceph-mon[36954]: from='osd.9 [v2:172.21.15.100:6826/1840140273,v1:172.21.15.100:6827/1840140273]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-07-26T13:35:06.822 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:06 smithi138 ceph-mon[36954]: osdmap e58: 10 total, 9 up, 10 in 2024-07-26T13:35:06.822 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:06 smithi138 ceph-mon[36954]: from='osd.9 [v2:172.21.15.100:6826/1840140273,v1:172.21.15.100:6827/1840140273]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-07-26T13:35:06.823 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-26T13:35:06.823 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:06 smithi138 ceph-mon[36954]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-07-26T13:35:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:06 smithi100 ceph-mon[32599]: from='osd.9 [v2:172.21.15.100:6826/1840140273,v1:172.21.15.100:6827/1840140273]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-07-26T13:35:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:06 smithi100 ceph-mon[32599]: osdmap e58: 10 total, 9 up, 10 in 2024-07-26T13:35:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:06 smithi100 ceph-mon[32599]: from='osd.9 [v2:172.21.15.100:6826/1840140273,v1:172.21.15.100:6827/1840140273]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-07-26T13:35:06.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-26T13:35:06.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:06 smithi100 ceph-mon[32599]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-07-26T13:35:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:06 smithi167 ceph-mon[37140]: from='osd.9 [v2:172.21.15.100:6826/1840140273,v1:172.21.15.100:6827/1840140273]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-07-26T13:35:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:06 smithi167 ceph-mon[37140]: osdmap e58: 10 total, 9 up, 10 in 2024-07-26T13:35:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:06 smithi167 ceph-mon[37140]: from='osd.9 [v2:172.21.15.100:6826/1840140273,v1:172.21.15.100:6827/1840140273]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-07-26T13:35:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-26T13:35:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:06 smithi167 ceph-mon[37140]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-07-26T13:35:07.333 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-07-26T13:35:07.423 INFO:teuthology.orchestra.run.smithi138.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-07-26T13:35:07.437 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10+0 records in 2024-07-26T13:35:07.437 INFO:teuthology.orchestra.run.smithi138.stderr:10+0 records out 2024-07-26T13:35:07.437 INFO:teuthology.orchestra.run.smithi138.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113411 s, 925 MB/s 2024-07-26T13:35:07.437 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping successful for: 2024-07-26T13:35:07.494 INFO:journalctl@ceph.osd.9.smithi100.stdout:Jul 26 13:35:07 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-osd-9[72236]: 2024-07-26T13:35:07.214+0000 7feee30dd640 -1 osd.9 0 waiting for initial osdmap 2024-07-26T13:35:07.494 INFO:journalctl@ceph.osd.9.smithi100.stdout:Jul 26 13:35:07 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-osd-9[72236]: 2024-07-26T13:35:07.226+0000 7feedaed3640 -1 osd.9 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-26T13:35:07.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:07 smithi100 ceph-mon[32599]: from='osd.9 [v2:172.21.15.100:6826/1840140273,v1:172.21.15.100:6827/1840140273]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-07-26T13:35:07.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:07 smithi100 ceph-mon[32599]: osdmap e59: 10 total, 9 up, 10 in 2024-07-26T13:35:07.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-26T13:35:07.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-26T13:35:07.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:07 smithi100 ceph-mon[32599]: from='osd.9 [v2:172.21.15.100:6826/1840140273,v1:172.21.15.100:6827/1840140273]' entity='osd.9' 2024-07-26T13:35:08.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:07 smithi167 ceph-mon[37140]: from='osd.9 [v2:172.21.15.100:6826/1840140273,v1:172.21.15.100:6827/1840140273]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-07-26T13:35:08.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:07 smithi167 ceph-mon[37140]: osdmap e59: 10 total, 9 up, 10 in 2024-07-26T13:35:08.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-26T13:35:08.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-26T13:35:08.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:07 smithi167 ceph-mon[37140]: from='osd.9 [v2:172.21.15.100:6826/1840140273,v1:172.21.15.100:6827/1840140273]' entity='osd.9' 2024-07-26T13:35:08.165 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:07 smithi138 ceph-mon[36954]: from='osd.9 [v2:172.21.15.100:6826/1840140273,v1:172.21.15.100:6827/1840140273]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-07-26T13:35:08.165 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:07 smithi138 ceph-mon[36954]: osdmap e59: 10 total, 9 up, 10 in 2024-07-26T13:35:08.165 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-26T13:35:08.165 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-26T13:35:08.165 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:07 smithi138 ceph-mon[36954]: from='osd.9 [v2:172.21.15.100:6826/1840140273,v1:172.21.15.100:6827/1840140273]' entity='osd.9' 2024-07-26T13:35:08.194 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_1 2024-07-26T13:35:08.494 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.b/config 2024-07-26T13:35:08.885 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:08 smithi138 ceph-mon[36954]: purged_snaps scrub starts 2024-07-26T13:35:08.885 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:08 smithi138 ceph-mon[36954]: purged_snaps scrub ok 2024-07-26T13:35:08.886 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-26T13:35:08.886 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:08 smithi138 ceph-mon[36954]: Detected new or changed devices on smithi100 2024-07-26T13:35:08.886 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:08.886 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:08.886 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:35:08.886 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:35:08.886 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:35:08.886 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:35:08.886 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:08 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi100 to 4271M 2024-07-26T13:35:08.886 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:08.886 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:35:08.886 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:35:08.887 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:08.887 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:08 smithi138 ceph-mon[36954]: osd.9 [v2:172.21.15.100:6826/1840140273,v1:172.21.15.100:6827/1840140273] boot 2024-07-26T13:35:08.887 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:08 smithi138 ceph-mon[36954]: osdmap e60: 10 total, 10 up, 10 in 2024-07-26T13:35:08.887 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-26T13:35:08.887 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:08 smithi138 ceph-mon[36954]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-07-26T13:35:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:08 smithi100 ceph-mon[32599]: purged_snaps scrub starts 2024-07-26T13:35:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:08 smithi100 ceph-mon[32599]: purged_snaps scrub ok 2024-07-26T13:35:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-26T13:35:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:08 smithi100 ceph-mon[32599]: Detected new or changed devices on smithi100 2024-07-26T13:35:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:35:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:35:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:35:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:35:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:08 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi100 to 4271M 2024-07-26T13:35:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:35:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:35:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:08 smithi100 ceph-mon[32599]: osd.9 [v2:172.21.15.100:6826/1840140273,v1:172.21.15.100:6827/1840140273] boot 2024-07-26T13:35:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:08 smithi100 ceph-mon[32599]: osdmap e60: 10 total, 10 up, 10 in 2024-07-26T13:35:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-26T13:35:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:08 smithi100 ceph-mon[32599]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-07-26T13:35:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:08 smithi167 ceph-mon[37140]: purged_snaps scrub starts 2024-07-26T13:35:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:08 smithi167 ceph-mon[37140]: purged_snaps scrub ok 2024-07-26T13:35:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-26T13:35:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:08 smithi167 ceph-mon[37140]: Detected new or changed devices on smithi100 2024-07-26T13:35:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:35:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:35:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:35:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:35:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:08 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi100 to 4271M 2024-07-26T13:35:09.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:09.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:35:09.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:35:09.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:09.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:08 smithi167 ceph-mon[37140]: osd.9 [v2:172.21.15.100:6826/1840140273,v1:172.21.15.100:6827/1840140273] boot 2024-07-26T13:35:09.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:08 smithi167 ceph-mon[37140]: osdmap e60: 10 total, 10 up, 10 in 2024-07-26T13:35:09.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-26T13:35:09.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:08 smithi167 ceph-mon[37140]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-07-26T13:35:10.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.220+0000 7fd8c4ad8640 1 -- 172.21.15.138:0/2205032630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8c0108790 msgr2=0x7fd8c010efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:35:10.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.220+0000 7fd8c4ad8640 1 --2- 172.21.15.138:0/2205032630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8c0108790 0x7fd8c010efc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8009900 tx=0x7fd8b802a410 comp rx=0 tx=0).stop 2024-07-26T13:35:10.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.220+0000 7fd8c4ad8640 1 -- 172.21.15.138:0/2205032630 shutdown_connections 2024-07-26T13:35:10.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.220+0000 7fd8c4ad8640 1 --2- 172.21.15.138:0/2205032630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8c0108790 0x7fd8c010efc0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:35:10.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.220+0000 7fd8c4ad8640 1 --2- 172.21.15.138:0/2205032630 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c0107e40 0x7fd8c0108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:35:10.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.220+0000 7fd8c4ad8640 1 --2- 172.21.15.138:0/2205032630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8c0102550 0x7fd8c0102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:35:10.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.220+0000 7fd8c4ad8640 1 -- 172.21.15.138:0/2205032630 >> 172.21.15.138:0/2205032630 conn(0x7fd8c0079130 msgr2=0x7fd8c0079540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:35:10.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.220+0000 7fd8c4ad8640 1 -- 172.21.15.138:0/2205032630 shutdown_connections 2024-07-26T13:35:10.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.220+0000 7fd8c4ad8640 1 -- 172.21.15.138:0/2205032630 wait complete. 2024-07-26T13:35:10.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.221+0000 7fd8c4ad8640 1 Processor -- start 2024-07-26T13:35:10.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.221+0000 7fd8c4ad8640 1 -- start start 2024-07-26T13:35:10.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.221+0000 7fd8c4ad8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c0102550 0x7fd8c0199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:35:10.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.221+0000 7fd8c4ad8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8c0107e40 0x7fd8c019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:35:10.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.221+0000 7fd8c4ad8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8c0108790 0x7fd8c019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:35:10.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.221+0000 7fd8c4ad8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c0113570 con 0x7fd8c0102550 2024-07-26T13:35:10.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.221+0000 7fd8c4ad8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c01133f0 con 0x7fd8c0108790 2024-07-26T13:35:10.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.221+0000 7fd8c4ad8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c01136f0 con 0x7fd8c0107e40 2024-07-26T13:35:10.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.222+0000 7fd8bed9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8c0108790 0x7fd8c019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:35:10.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.222+0000 7fd8bed9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8c0108790 0x7fd8c019f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54832/0 (socket says 172.21.15.138:54832) 2024-07-26T13:35:10.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.222+0000 7fd8bed9d640 1 -- 172.21.15.138:0/613446469 learned_addr learned my addr 172.21.15.138:0/613446469 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-07-26T13:35:10.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.222+0000 7fd8be59c640 1 --2- 172.21.15.138:0/613446469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c0102550 0x7fd8c0199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:35:10.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.222+0000 7fd8bdd9b640 1 --2- 172.21.15.138:0/613446469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8c0107e40 0x7fd8c019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:35:10.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.222+0000 7fd8bed9d640 1 -- 172.21.15.138:0/613446469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8c0107e40 msgr2=0x7fd8c019a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:35:10.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.222+0000 7fd8bed9d640 1 --2- 172.21.15.138:0/613446469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8c0107e40 0x7fd8c019a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:35:10.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.222+0000 7fd8bed9d640 1 -- 172.21.15.138:0/613446469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c0102550 msgr2=0x7fd8c0199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:35:10.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.222+0000 7fd8bed9d640 1 --2- 172.21.15.138:0/613446469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c0102550 0x7fd8c0199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:35:10.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.222+0000 7fd8bed9d640 1 -- 172.21.15.138:0/613446469 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c019fe70 con 0x7fd8c0108790 2024-07-26T13:35:10.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.222+0000 7fd8be59c640 1 --2- 172.21.15.138:0/613446469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c0102550 0x7fd8c0199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:35:10.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.222+0000 7fd8bed9d640 1 --2- 172.21.15.138:0/613446469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8c0108790 0x7fd8c019f740 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8004770 tx=0x7fd8b8004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:35:10.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.222+0000 7fd8af7fe640 1 -- 172.21.15.138:0/613446469 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b803f030 con 0x7fd8c0108790 2024-07-26T13:35:10.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.222+0000 7fd8c4ad8640 1 -- 172.21.15.138:0/613446469 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8c01a0100 con 0x7fd8c0108790 2024-07-26T13:35:10.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.222+0000 7fd8af7fe640 1 -- 172.21.15.138:0/613446469 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8b8003aa0 con 0x7fd8c0108790 2024-07-26T13:35:10.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.222+0000 7fd8af7fe640 1 -- 172.21.15.138:0/613446469 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b8035070 con 0x7fd8c0108790 2024-07-26T13:35:10.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.222+0000 7fd8c4ad8640 1 -- 172.21.15.138:0/613446469 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8c01a68c0 con 0x7fd8c0108790 2024-07-26T13:35:10.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.223+0000 7fd8c4ad8640 1 -- 172.21.15.138:0/613446469 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8c0103780 con 0x7fd8c0108790 2024-07-26T13:35:10.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.225+0000 7fd8af7fe640 1 -- 172.21.15.138:0/613446469 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7fd8b8042030 con 0x7fd8c0108790 2024-07-26T13:35:10.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.226+0000 7fd8af7fe640 1 --2- 172.21.15.138:0/613446469 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd8a80b3c20 0x7fd8a80b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:35:10.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.226+0000 7fd8af7fe640 1 -- 172.21.15.138:0/613446469 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6476+0+0 (secure 0 0 0) 0x7fd8b8103f80 con 0x7fd8c0108790 2024-07-26T13:35:10.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.226+0000 7fd8be59c640 1 --2- 172.21.15.138:0/613446469 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd8a80b3c20 0x7fd8a80b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:35:10.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.227+0000 7fd8be59c640 1 --2- 172.21.15.138:0/613446469 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd8a80b3c20 0x7fd8a80b6130 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0005fd0 tx=0x7fd8b00093e0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:35:10.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.229+0000 7fd8af7fe640 1 -- 172.21.15.138:0/613446469 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fd8b80cbd00 con 0x7fd8c0108790 2024-07-26T13:35:10.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:10.411+0000 7fd8c4ad8640 1 -- 172.21.15.138:0/613446469 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fd8c0063230 con 0x7fd8a80b3c20 2024-07-26T13:35:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:10 smithi100 ceph-mon[32599]: osdmap e61: 10 total, 10 up, 10 in 2024-07-26T13:35:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:10 smithi100 ceph-mon[32599]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-07-26T13:35:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:10 smithi100 ceph-mon[32599]: from='client.24476 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:35:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:10 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-26T13:35:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:10 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-26T13:35:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:10 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:35:11.005 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:10 smithi138 ceph-mon[36954]: osdmap e61: 10 total, 10 up, 10 in 2024-07-26T13:35:11.005 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:10 smithi138 ceph-mon[36954]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-07-26T13:35:11.005 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:10 smithi138 ceph-mon[36954]: from='client.24476 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:35:11.005 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:10 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-26T13:35:11.005 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:10 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-26T13:35:11.005 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:10 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:35:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:10 smithi167 ceph-mon[37140]: osdmap e61: 10 total, 10 up, 10 in 2024-07-26T13:35:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:10 smithi167 ceph-mon[37140]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-07-26T13:35:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:10 smithi167 ceph-mon[37140]: from='client.24476 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:35:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:10 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-26T13:35:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:10 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-26T13:35:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:10 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:35:13.539 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:13 smithi138 ceph-mon[36954]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-07-26T13:35:13.539 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:13 smithi138 ceph-mon[36954]: from='client.? 172.21.15.138:0/2635171150' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab77ebe7-5254-4dc3-bc54-625698e84afc"}]: dispatch 2024-07-26T13:35:13.540 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:13 smithi138 ceph-mon[36954]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab77ebe7-5254-4dc3-bc54-625698e84afc"}]: dispatch 2024-07-26T13:35:13.540 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:13 smithi138 ceph-mon[36954]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab77ebe7-5254-4dc3-bc54-625698e84afc"}]': finished 2024-07-26T13:35:13.540 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:13 smithi138 ceph-mon[36954]: osdmap e62: 11 total, 10 up, 11 in 2024-07-26T13:35:13.540 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-26T13:35:13.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:13 smithi100 ceph-mon[32599]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-07-26T13:35:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:13 smithi100 ceph-mon[32599]: from='client.? 172.21.15.138:0/2635171150' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab77ebe7-5254-4dc3-bc54-625698e84afc"}]: dispatch 2024-07-26T13:35:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:13 smithi100 ceph-mon[32599]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab77ebe7-5254-4dc3-bc54-625698e84afc"}]: dispatch 2024-07-26T13:35:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:13 smithi100 ceph-mon[32599]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab77ebe7-5254-4dc3-bc54-625698e84afc"}]': finished 2024-07-26T13:35:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:13 smithi100 ceph-mon[32599]: osdmap e62: 11 total, 10 up, 11 in 2024-07-26T13:35:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-26T13:35:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:13 smithi167 ceph-mon[37140]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-07-26T13:35:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:13 smithi167 ceph-mon[37140]: from='client.? 172.21.15.138:0/2635171150' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab77ebe7-5254-4dc3-bc54-625698e84afc"}]: dispatch 2024-07-26T13:35:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:13 smithi167 ceph-mon[37140]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab77ebe7-5254-4dc3-bc54-625698e84afc"}]: dispatch 2024-07-26T13:35:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:13 smithi167 ceph-mon[37140]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab77ebe7-5254-4dc3-bc54-625698e84afc"}]': finished 2024-07-26T13:35:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:13 smithi167 ceph-mon[37140]: osdmap e62: 11 total, 10 up, 11 in 2024-07-26T13:35:13.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-26T13:35:14.672 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:14 smithi100 ceph-mon[32599]: from='client.? 172.21.15.138:0/3231287626' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-26T13:35:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:14 smithi138 ceph-mon[36954]: from='client.? 172.21.15.138:0/3231287626' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-26T13:35:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:14 smithi167 ceph-mon[37140]: from='client.? 172.21.15.138:0/3231287626' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-26T13:35:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:15 smithi138 ceph-mon[36954]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-07-26T13:35:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:15 smithi100 ceph-mon[32599]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-07-26T13:35:15.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:15 smithi167 ceph-mon[37140]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-07-26T13:35:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:17 smithi138 ceph-mon[36954]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-07-26T13:35:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:17 smithi100 ceph-mon[32599]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-07-26T13:35:17.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:17 smithi167 ceph-mon[37140]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-07-26T13:35:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:19 smithi138 ceph-mon[36954]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-07-26T13:35:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:19 smithi100 ceph-mon[32599]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-07-26T13:35:19.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:19 smithi167 ceph-mon[37140]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-07-26T13:35:20.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:20 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-07-26T13:35:20.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:20 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:35:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:20 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-07-26T13:35:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:20 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:35:20.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:20 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-07-26T13:35:20.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:20 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:35:21.598 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:21 smithi138 ceph-mon[36954]: Deploying daemon osd.10 on smithi138 2024-07-26T13:35:21.599 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:21 smithi138 ceph-mon[36954]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-07-26T13:35:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:21 smithi100 ceph-mon[32599]: Deploying daemon osd.10 on smithi138 2024-07-26T13:35:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:21 smithi100 ceph-mon[32599]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-07-26T13:35:21.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:21 smithi167 ceph-mon[37140]: Deploying daemon osd.10 on smithi138 2024-07-26T13:35:21.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:21 smithi167 ceph-mon[37140]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-07-26T13:35:23.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:23 smithi138 ceph-mon[36954]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-07-26T13:35:23.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:23 smithi100 ceph-mon[32599]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-07-26T13:35:23.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:23 smithi167 ceph-mon[37140]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-07-26T13:35:25.680 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:25 smithi138 ceph-mon[36954]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-07-26T13:35:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:25 smithi100 ceph-mon[32599]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-07-26T13:35:25.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:25 smithi167 ceph-mon[37140]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-07-26T13:35:26.567 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:35:26.567 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:26.567 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:26.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:35:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:26.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:35:26.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:26.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:27.683 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:27 smithi138 ceph-mon[36954]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-07-26T13:35:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:27 smithi100 ceph-mon[32599]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-07-26T13:35:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:27 smithi167 ceph-mon[37140]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-07-26T13:35:29.478 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 10 on host 'smithi138' 2024-07-26T13:35:29.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:29.476+0000 7fd8af7fe640 1 -- 172.21.15.138:0/613446469 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fd8c0063230 con 0x7fd8a80b3c20 2024-07-26T13:35:29.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:29.478+0000 7fd8c4ad8640 1 -- 172.21.15.138:0/613446469 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd8a80b3c20 msgr2=0x7fd8a80b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:35:29.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:29.478+0000 7fd8c4ad8640 1 --2- 172.21.15.138:0/613446469 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd8a80b3c20 0x7fd8a80b6130 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0005fd0 tx=0x7fd8b00093e0 comp rx=0 tx=0).stop 2024-07-26T13:35:29.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:29.478+0000 7fd8c4ad8640 1 -- 172.21.15.138:0/613446469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8c0108790 msgr2=0x7fd8c019f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:35:29.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:29.478+0000 7fd8c4ad8640 1 --2- 172.21.15.138:0/613446469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8c0108790 0x7fd8c019f740 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8004770 tx=0x7fd8b8004270 comp rx=0 tx=0).stop 2024-07-26T13:35:29.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:29.478+0000 7fd8c4ad8640 1 -- 172.21.15.138:0/613446469 shutdown_connections 2024-07-26T13:35:29.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:29.478+0000 7fd8c4ad8640 1 --2- 172.21.15.138:0/613446469 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd8a80b3c20 0x7fd8a80b6130 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:35:29.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:29.478+0000 7fd8c4ad8640 1 --2- 172.21.15.138:0/613446469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8c0108790 0x7fd8c019f740 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:35:29.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:29.478+0000 7fd8c4ad8640 1 --2- 172.21.15.138:0/613446469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8c0107e40 0x7fd8c019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:35:29.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:29.478+0000 7fd8c4ad8640 1 --2- 172.21.15.138:0/613446469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd8c0102550 0x7fd8c0199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:35:29.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:29.478+0000 7fd8c4ad8640 1 -- 172.21.15.138:0/613446469 >> 172.21.15.138:0/613446469 conn(0x7fd8c0079130 msgr2=0x7fd8c0105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:35:29.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:29.478+0000 7fd8c4ad8640 1 -- 172.21.15.138:0/613446469 shutdown_connections 2024-07-26T13:35:29.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-07-26T13:35:29.478+0000 7fd8c4ad8640 1 -- 172.21.15.138:0/613446469 wait complete. 2024-07-26T13:35:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:29 smithi138 ceph-mon[36954]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-07-26T13:35:29.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:29 smithi100 ceph-mon[32599]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-07-26T13:35:29.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:29 smithi167 ceph-mon[37140]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-07-26T13:35:30.161 DEBUG:teuthology.orchestra.run.smithi138:osd.10> sudo journalctl -f -n 0 -u ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@osd.10.service 2024-07-26T13:35:30.163 INFO:tasks.cephadm:Deploying osd.11 on smithi167 with /dev/vg_nvme/lv_1... 2024-07-26T13:35:30.163 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-07-26T13:35:30.419 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.c/config 2024-07-26T13:35:30.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:30 smithi100 ceph-mon[32599]: from='osd.10 [v2:172.21.15.138:6824/703382899,v1:172.21.15.138:6825/703382899]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-07-26T13:35:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:30 smithi100 ceph-mon[32599]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-07-26T13:35:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:30 smithi167 ceph-mon[37140]: from='osd.10 [v2:172.21.15.138:6824/703382899,v1:172.21.15.138:6825/703382899]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-07-26T13:35:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:30 smithi167 ceph-mon[37140]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-07-26T13:35:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:30.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:30.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:30.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:30 smithi138 ceph-mon[36954]: from='osd.10 [v2:172.21.15.138:6824/703382899,v1:172.21.15.138:6825/703382899]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-07-26T13:35:30.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:30 smithi138 ceph-mon[36954]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-07-26T13:35:30.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:30.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:31 smithi100 ceph-mon[32599]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-07-26T13:35:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:31 smithi100 ceph-mon[32599]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-07-26T13:35:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:31 smithi100 ceph-mon[32599]: from='osd.10 [v2:172.21.15.138:6824/703382899,v1:172.21.15.138:6825/703382899]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-07-26T13:35:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:31 smithi100 ceph-mon[32599]: osdmap e63: 11 total, 10 up, 11 in 2024-07-26T13:35:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-26T13:35:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:31 smithi100 ceph-mon[32599]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-07-26T13:35:31.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:31 smithi167 ceph-mon[37140]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-07-26T13:35:31.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:31 smithi167 ceph-mon[37140]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-07-26T13:35:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:31 smithi167 ceph-mon[37140]: from='osd.10 [v2:172.21.15.138:6824/703382899,v1:172.21.15.138:6825/703382899]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-07-26T13:35:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:31 smithi167 ceph-mon[37140]: osdmap e63: 11 total, 10 up, 11 in 2024-07-26T13:35:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-26T13:35:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:31 smithi167 ceph-mon[37140]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-07-26T13:35:31.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:31 smithi138 ceph-mon[36954]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-07-26T13:35:31.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:31 smithi138 ceph-mon[36954]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-07-26T13:35:31.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:31 smithi138 ceph-mon[36954]: from='osd.10 [v2:172.21.15.138:6824/703382899,v1:172.21.15.138:6825/703382899]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-07-26T13:35:31.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:31 smithi138 ceph-mon[36954]: osdmap e63: 11 total, 10 up, 11 in 2024-07-26T13:35:31.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-26T13:35:31.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:31 smithi138 ceph-mon[36954]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-07-26T13:35:32.177 INFO:journalctl@ceph.osd.10.smithi138.stdout:Jul 26 13:35:31 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-osd-10[63167]: 2024-07-26T13:35:31.909+0000 7f6243ef4640 -1 osd.10 0 waiting for initial osdmap 2024-07-26T13:35:32.177 INFO:journalctl@ceph.osd.10.smithi138.stdout:Jul 26 13:35:31 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-osd-10[63167]: 2024-07-26T13:35:31.927+0000 7f623b4d5640 -1 osd.10 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-26T13:35:32.291 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-07-26T13:35:32.378 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-07-26T13:35:32.392 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2024-07-26T13:35:32.392 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2024-07-26T13:35:32.392 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011354 s, 924 MB/s 2024-07-26T13:35:32.392 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2024-07-26T13:35:32.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:32 smithi167 ceph-mon[37140]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-07-26T13:35:32.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:32 smithi167 ceph-mon[37140]: osdmap e64: 11 total, 10 up, 11 in 2024-07-26T13:35:32.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-26T13:35:32.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-26T13:35:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:32 smithi167 ceph-mon[37140]: from='osd.10 ' entity='osd.10' 2024-07-26T13:35:32.886 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:32 smithi138 ceph-mon[36954]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-07-26T13:35:32.887 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:32 smithi138 ceph-mon[36954]: osdmap e64: 11 total, 10 up, 11 in 2024-07-26T13:35:32.887 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-26T13:35:32.887 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-26T13:35:32.887 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:32 smithi138 ceph-mon[36954]: from='osd.10 ' entity='osd.10' 2024-07-26T13:35:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:32 smithi100 ceph-mon[32599]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-07-26T13:35:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:32 smithi100 ceph-mon[32599]: osdmap e64: 11 total, 10 up, 11 in 2024-07-26T13:35:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-26T13:35:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-26T13:35:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:32 smithi100 ceph-mon[32599]: from='osd.10 ' entity='osd.10' 2024-07-26T13:35:33.100 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_1 2024-07-26T13:35:33.358 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.c/config 2024-07-26T13:35:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:33 smithi138 ceph-mon[36954]: purged_snaps scrub starts 2024-07-26T13:35:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:33 smithi138 ceph-mon[36954]: purged_snaps scrub ok 2024-07-26T13:35:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:33 smithi138 ceph-mon[36954]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-07-26T13:35:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:33 smithi138 ceph-mon[36954]: osd.10 [v2:172.21.15.138:6824/703382899,v1:172.21.15.138:6825/703382899] boot 2024-07-26T13:35:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:33 smithi138 ceph-mon[36954]: osdmap e65: 11 total, 11 up, 11 in 2024-07-26T13:35:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-26T13:35:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:33 smithi138 ceph-mon[36954]: osdmap e66: 11 total, 11 up, 11 in 2024-07-26T13:35:33.725 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:33 smithi167 ceph-mon[37140]: purged_snaps scrub starts 2024-07-26T13:35:33.725 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:33 smithi167 ceph-mon[37140]: purged_snaps scrub ok 2024-07-26T13:35:33.725 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:33 smithi167 ceph-mon[37140]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-07-26T13:35:33.725 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:33 smithi167 ceph-mon[37140]: osd.10 [v2:172.21.15.138:6824/703382899,v1:172.21.15.138:6825/703382899] boot 2024-07-26T13:35:33.725 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:33 smithi167 ceph-mon[37140]: osdmap e65: 11 total, 11 up, 11 in 2024-07-26T13:35:33.725 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-26T13:35:33.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:33 smithi167 ceph-mon[37140]: osdmap e66: 11 total, 11 up, 11 in 2024-07-26T13:35:33.895 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:33 smithi100 ceph-mon[32599]: purged_snaps scrub starts 2024-07-26T13:35:33.895 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:33 smithi100 ceph-mon[32599]: purged_snaps scrub ok 2024-07-26T13:35:33.895 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:33 smithi100 ceph-mon[32599]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-07-26T13:35:33.895 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:33 smithi100 ceph-mon[32599]: osd.10 [v2:172.21.15.138:6824/703382899,v1:172.21.15.138:6825/703382899] boot 2024-07-26T13:35:33.895 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:33 smithi100 ceph-mon[32599]: osdmap e65: 11 total, 11 up, 11 in 2024-07-26T13:35:33.895 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-26T13:35:33.896 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:33 smithi100 ceph-mon[32599]: osdmap e66: 11 total, 11 up, 11 in 2024-07-26T13:35:34.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.875+0000 7fec2c6b1640 1 -- 172.21.15.167:0/2774927712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec24107dd0 msgr2=0x7fec241081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:35:34.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.875+0000 7fec2c6b1640 1 --2- 172.21.15.167:0/2774927712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec24107dd0 0x7fec241081e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fec14009900 tx=0x7fec1402a410 comp rx=0 tx=0).stop 2024-07-26T13:35:34.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.875+0000 7fec2c6b1640 1 -- 172.21.15.167:0/2774927712 shutdown_connections 2024-07-26T13:35:34.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.875+0000 7fec2c6b1640 1 --2- 172.21.15.167:0/2774927712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec24108720 0x7fec2410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:35:34.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.875+0000 7fec2c6b1640 1 --2- 172.21.15.167:0/2774927712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec24107dd0 0x7fec241081e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:35:34.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.875+0000 7fec2c6b1640 1 --2- 172.21.15.167:0/2774927712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec24102490 0x7fec241028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:35:34.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.875+0000 7fec2c6b1640 1 -- 172.21.15.167:0/2774927712 >> 172.21.15.167:0/2774927712 conn(0x7fec24079160 msgr2=0x7fec24079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:35:34.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.875+0000 7fec2c6b1640 1 -- 172.21.15.167:0/2774927712 shutdown_connections 2024-07-26T13:35:34.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.875+0000 7fec2c6b1640 1 -- 172.21.15.167:0/2774927712 wait complete. 2024-07-26T13:35:34.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.876+0000 7fec2c6b1640 1 Processor -- start 2024-07-26T13:35:34.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.876+0000 7fec2c6b1640 1 -- start start 2024-07-26T13:35:34.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.876+0000 7fec2c6b1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec24102490 0x7fec24198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:35:34.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.876+0000 7fec2c6b1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec24107dd0 0x7fec241992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:35:34.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.876+0000 7fec2c6b1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec24108720 0x7fec2419e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:35:34.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.876+0000 7fec2c6b1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fec24112370 con 0x7fec24108720 2024-07-26T13:35:34.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.876+0000 7fec2c6b1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fec241121f0 con 0x7fec24102490 2024-07-26T13:35:34.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.876+0000 7fec2c6b1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fec241124f0 con 0x7fec24107dd0 2024-07-26T13:35:34.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.876+0000 7fec29c4c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec24107dd0 0x7fec241992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:35:34.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.876+0000 7fec29c4c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec24107dd0 0x7fec241992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:60696/0 (socket says 172.21.15.167:60696) 2024-07-26T13:35:34.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.876+0000 7fec29c4c640 1 -- 172.21.15.167:0/1298044601 learned_addr learned my addr 172.21.15.167:0/1298044601 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-07-26T13:35:34.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.876+0000 7fec2a44d640 1 --2- 172.21.15.167:0/1298044601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec24102490 0x7fec24198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:35:34.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.876+0000 7fec2ac4e640 1 --2- 172.21.15.167:0/1298044601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec24108720 0x7fec2419e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:35:34.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.877+0000 7fec29c4c640 1 -- 172.21.15.167:0/1298044601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec24102490 msgr2=0x7fec24198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:35:34.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.877+0000 7fec29c4c640 1 --2- 172.21.15.167:0/1298044601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec24102490 0x7fec24198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:35:34.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.877+0000 7fec29c4c640 1 -- 172.21.15.167:0/1298044601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec24108720 msgr2=0x7fec2419e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:35:34.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.877+0000 7fec29c4c640 1 --2- 172.21.15.167:0/1298044601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec24108720 0x7fec2419e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:35:34.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.877+0000 7fec29c4c640 1 -- 172.21.15.167:0/1298044601 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec2419ecb0 con 0x7fec24107dd0 2024-07-26T13:35:34.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.877+0000 7fec29c4c640 1 --2- 172.21.15.167:0/1298044601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec24107dd0 0x7fec241992e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fec14006010 tx=0x7fec14004060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:35:34.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.877+0000 7fec1b7fe640 1 -- 172.21.15.167:0/1298044601 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1403f030 con 0x7fec24107dd0 2024-07-26T13:35:34.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.877+0000 7fec1b7fe640 1 -- 172.21.15.167:0/1298044601 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec14003aa0 con 0x7fec24107dd0 2024-07-26T13:35:34.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.877+0000 7fec1b7fe640 1 -- 172.21.15.167:0/1298044601 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec14035070 con 0x7fec24107dd0 2024-07-26T13:35:34.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.877+0000 7fec2c6b1640 1 -- 172.21.15.167:0/1298044601 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec2419ef40 con 0x7fec24107dd0 2024-07-26T13:35:34.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.877+0000 7fec2c6b1640 1 -- 172.21.15.167:0/1298044601 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec241a5700 con 0x7fec24107dd0 2024-07-26T13:35:34.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.878+0000 7fec2c6b1640 1 -- 172.21.15.167:0/1298044601 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec241028a0 con 0x7fec24107dd0 2024-07-26T13:35:34.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.881+0000 7fec1b7fe640 1 -- 172.21.15.167:0/1298044601 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7fec14042030 con 0x7fec24107dd0 2024-07-26T13:35:34.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.881+0000 7fec1b7fe640 1 --2- 172.21.15.167:0/1298044601 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fec100b3ca0 0x7fec100b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:35:34.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.881+0000 7fec2a44d640 1 --2- 172.21.15.167:0/1298044601 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fec100b3ca0 0x7fec100b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:35:34.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.881+0000 7fec1b7fe640 1 -- 172.21.15.167:0/1298044601 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6909+0+0 (secure 0 0 0) 0x7fec14103e40 con 0x7fec24107dd0 2024-07-26T13:35:34.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.882+0000 7fec2a44d640 1 --2- 172.21.15.167:0/1298044601 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fec100b3ca0 0x7fec100b61b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fec20009bd0 tx=0x7fec20005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:35:34.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:34.884+0000 7fec1b7fe640 1 -- 172.21.15.167:0/1298044601 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fec140cb3c0 con 0x7fec24107dd0 2024-07-26T13:35:35.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:35.067+0000 7fec2c6b1640 1 -- 172.21.15.167:0/1298044601 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fec2410a540 con 0x7fec100b3ca0 2024-07-26T13:35:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:34 smithi100 ceph-mon[32599]: Detected new or changed devices on smithi138 2024-07-26T13:35:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:35:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:35:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:35:35.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:35:35.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:34 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi138 to 4272M 2024-07-26T13:35:35.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:35.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:35:35.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:35:35.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:35.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:35:35.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:35:35.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:35:35.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:35.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:34 smithi100 ceph-mon[32599]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-07-26T13:35:35.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:34 smithi100 ceph-mon[32599]: osdmap e67: 11 total, 11 up, 11 in 2024-07-26T13:35:35.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:34 smithi167 ceph-mon[37140]: Detected new or changed devices on smithi138 2024-07-26T13:35:35.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:35.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:35:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:35:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:35:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:35:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:34 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi138 to 4272M 2024-07-26T13:35:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:35:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:35:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:35:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:35:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:35:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:35.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:34 smithi167 ceph-mon[37140]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-07-26T13:35:35.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:34 smithi167 ceph-mon[37140]: osdmap e67: 11 total, 11 up, 11 in 2024-07-26T13:35:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:34 smithi138 ceph-mon[36954]: Detected new or changed devices on smithi138 2024-07-26T13:35:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:35:35.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:35:35.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:35:35.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:35:35.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:34 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi138 to 4272M 2024-07-26T13:35:35.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:35.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:35:35.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:35:35.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:35.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:35:35.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:35:35.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:35:35.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:35.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:34 smithi138 ceph-mon[36954]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-07-26T13:35:35.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:34 smithi138 ceph-mon[36954]: osdmap e67: 11 total, 11 up, 11 in 2024-07-26T13:35:36.164 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:35 smithi167 ceph-mon[37140]: from='client.24499 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:35:36.165 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-26T13:35:36.165 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-26T13:35:36.165 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:35:36.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:35 smithi100 ceph-mon[32599]: from='client.24499 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:35:36.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-26T13:35:36.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-26T13:35:36.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:35:36.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:35 smithi138 ceph-mon[36954]: from='client.24499 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:35:36.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-26T13:35:36.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-26T13:35:36.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:35:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:36 smithi167 ceph-mon[37140]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-07-26T13:35:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:36 smithi138 ceph-mon[36954]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-07-26T13:35:37.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:36 smithi100 ceph-mon[32599]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-07-26T13:35:38.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:37 smithi100 ceph-mon[32599]: from='client.? 172.21.15.167:0/1305660883' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "229cf91e-e9ef-48f6-a677-c5c5705b315a"}]: dispatch 2024-07-26T13:35:38.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:37 smithi100 ceph-mon[32599]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "229cf91e-e9ef-48f6-a677-c5c5705b315a"}]: dispatch 2024-07-26T13:35:38.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:37 smithi100 ceph-mon[32599]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "229cf91e-e9ef-48f6-a677-c5c5705b315a"}]': finished 2024-07-26T13:35:38.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:37 smithi100 ceph-mon[32599]: osdmap e68: 12 total, 11 up, 12 in 2024-07-26T13:35:38.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-26T13:35:38.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:37 smithi167 ceph-mon[37140]: from='client.? 172.21.15.167:0/1305660883' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "229cf91e-e9ef-48f6-a677-c5c5705b315a"}]: dispatch 2024-07-26T13:35:38.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:37 smithi167 ceph-mon[37140]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "229cf91e-e9ef-48f6-a677-c5c5705b315a"}]: dispatch 2024-07-26T13:35:38.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:37 smithi167 ceph-mon[37140]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "229cf91e-e9ef-48f6-a677-c5c5705b315a"}]': finished 2024-07-26T13:35:38.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:37 smithi167 ceph-mon[37140]: osdmap e68: 12 total, 11 up, 12 in 2024-07-26T13:35:38.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-26T13:35:38.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:37 smithi138 ceph-mon[36954]: from='client.? 172.21.15.167:0/1305660883' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "229cf91e-e9ef-48f6-a677-c5c5705b315a"}]: dispatch 2024-07-26T13:35:38.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:37 smithi138 ceph-mon[36954]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "229cf91e-e9ef-48f6-a677-c5c5705b315a"}]: dispatch 2024-07-26T13:35:38.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:37 smithi138 ceph-mon[36954]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "229cf91e-e9ef-48f6-a677-c5c5705b315a"}]': finished 2024-07-26T13:35:38.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:37 smithi138 ceph-mon[36954]: osdmap e68: 12 total, 11 up, 12 in 2024-07-26T13:35:38.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-26T13:35:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:38 smithi100 ceph-mon[32599]: from='client.? 172.21.15.167:0/2087998314' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-26T13:35:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:38 smithi100 ceph-mon[32599]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-07-26T13:35:39.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:38 smithi167 ceph-mon[37140]: from='client.? 172.21.15.167:0/2087998314' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-26T13:35:39.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:38 smithi167 ceph-mon[37140]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-07-26T13:35:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:38 smithi138 ceph-mon[36954]: from='client.? 172.21.15.167:0/2087998314' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-26T13:35:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:38 smithi138 ceph-mon[36954]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-07-26T13:35:41.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:41 smithi138 ceph-mon[36954]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-07-26T13:35:41.709 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:41 smithi167 ceph-mon[37140]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-07-26T13:35:41.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:41 smithi100 ceph-mon[32599]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-07-26T13:35:43.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:43 smithi138 ceph-mon[36954]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-26T13:35:43.743 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:43 smithi167 ceph-mon[37140]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-26T13:35:43.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:43 smithi100 ceph-mon[32599]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-26T13:35:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:45 smithi100 ceph-mon[32599]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-26T13:35:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:45 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-07-26T13:35:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:45 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:35:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:45 smithi100 ceph-mon[32599]: Deploying daemon osd.11 on smithi167 2024-07-26T13:35:45.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:45 smithi167 ceph-mon[37140]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-26T13:35:45.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:45 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-07-26T13:35:45.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:45 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:35:45.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:45 smithi167 ceph-mon[37140]: Deploying daemon osd.11 on smithi167 2024-07-26T13:35:45.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:45 smithi138 ceph-mon[36954]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-26T13:35:45.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:45 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-07-26T13:35:45.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:45 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:35:45.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:45 smithi138 ceph-mon[36954]: Deploying daemon osd.11 on smithi167 2024-07-26T13:35:47.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:47 smithi167 ceph-mon[37140]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-07-26T13:35:47.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:47 smithi138 ceph-mon[36954]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-07-26T13:35:47.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:47 smithi100 ceph-mon[32599]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-07-26T13:35:49.514 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:49 smithi167 ceph-mon[37140]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-07-26T13:35:49.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:49 smithi138 ceph-mon[36954]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-07-26T13:35:49.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:49 smithi100 ceph-mon[32599]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-07-26T13:35:50.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:35:50.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:50.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:50.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:35:50.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:50.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:50.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:35:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:51.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:51 smithi167 ceph-mon[37140]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-07-26T13:35:51.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:51 smithi138 ceph-mon[36954]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-07-26T13:35:51.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:51 smithi100 ceph-mon[32599]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-07-26T13:35:53.467 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 11 on host 'smithi167' 2024-07-26T13:35:53.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:53.465+0000 7fec1b7fe640 1 -- 172.21.15.167:0/1298044601 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fec2410a540 con 0x7fec100b3ca0 2024-07-26T13:35:53.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:53.467+0000 7fec2c6b1640 1 -- 172.21.15.167:0/1298044601 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fec100b3ca0 msgr2=0x7fec100b61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:35:53.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:53.467+0000 7fec2c6b1640 1 --2- 172.21.15.167:0/1298044601 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fec100b3ca0 0x7fec100b61b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fec20009bd0 tx=0x7fec20005f90 comp rx=0 tx=0).stop 2024-07-26T13:35:53.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:53.467+0000 7fec2c6b1640 1 -- 172.21.15.167:0/1298044601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec24107dd0 msgr2=0x7fec241992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:35:53.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:53.467+0000 7fec2c6b1640 1 --2- 172.21.15.167:0/1298044601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec24107dd0 0x7fec241992e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fec14006010 tx=0x7fec14004060 comp rx=0 tx=0).stop 2024-07-26T13:35:53.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:53.467+0000 7fec2c6b1640 1 -- 172.21.15.167:0/1298044601 shutdown_connections 2024-07-26T13:35:53.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:53.467+0000 7fec2c6b1640 1 --2- 172.21.15.167:0/1298044601 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fec100b3ca0 0x7fec100b61b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:35:53.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:53.467+0000 7fec2c6b1640 1 --2- 172.21.15.167:0/1298044601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec24108720 0x7fec2419e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:35:53.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:53.467+0000 7fec2c6b1640 1 --2- 172.21.15.167:0/1298044601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec24107dd0 0x7fec241992e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:35:53.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:53.467+0000 7fec2c6b1640 1 --2- 172.21.15.167:0/1298044601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec24102490 0x7fec24198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:35:53.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:53.467+0000 7fec2c6b1640 1 -- 172.21.15.167:0/1298044601 >> 172.21.15.167:0/1298044601 conn(0x7fec24079160 msgr2=0x7fec24104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:35:53.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:53.467+0000 7fec2c6b1640 1 -- 172.21.15.167:0/1298044601 shutdown_connections 2024-07-26T13:35:53.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:35:53.468+0000 7fec2c6b1640 1 -- 172.21.15.167:0/1298044601 wait complete. 2024-07-26T13:35:53.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:53 smithi138 ceph-mon[36954]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-07-26T13:35:53.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:53 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:53.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:53 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:53.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:53 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:35:53.694 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:53 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:35:53.694 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:53 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:53.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:53 smithi167 ceph-mon[37140]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-07-26T13:35:53.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:53 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:53.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:53 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:53.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:53 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:35:53.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:53 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:35:53.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:53 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:53.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:53 smithi100 ceph-mon[32599]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-07-26T13:35:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:53 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:53 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:53 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:35:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:53 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:35:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:53 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:54.145 DEBUG:teuthology.orchestra.run.smithi167:osd.11> sudo journalctl -f -n 0 -u ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@osd.11.service 2024-07-26T13:35:54.148 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-07-26T13:35:54.148 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph osd stat -f json 2024-07-26T13:35:54.417 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:35:54.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:35:54.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:54.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:54.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:54 smithi100 ceph-mon[32599]: from='osd.11 [v2:172.21.15.167:6824/1032343068,v1:172.21.15.167:6825/1032343068]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-07-26T13:35:54.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:54 smithi100 ceph-mon[32599]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-07-26T13:35:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:35:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:54 smithi138 ceph-mon[36954]: from='osd.11 [v2:172.21.15.167:6824/1032343068,v1:172.21.15.167:6825/1032343068]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-07-26T13:35:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:54 smithi138 ceph-mon[36954]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-07-26T13:35:54.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:35:54.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:54.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:54.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:54 smithi167 ceph-mon[37140]: from='osd.11 [v2:172.21.15.167:6824/1032343068,v1:172.21.15.167:6825/1032343068]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-07-26T13:35:54.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:54 smithi167 ceph-mon[37140]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-07-26T13:35:55.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:55 smithi167 ceph-mon[37140]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-07-26T13:35:55.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:55 smithi167 ceph-mon[37140]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-07-26T13:35:55.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:55 smithi167 ceph-mon[37140]: osdmap e69: 12 total, 11 up, 12 in 2024-07-26T13:35:55.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-26T13:35:55.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:55 smithi167 ceph-mon[37140]: from='osd.11 [v2:172.21.15.167:6824/1032343068,v1:172.21.15.167:6825/1032343068]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-07-26T13:35:55.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:55 smithi167 ceph-mon[37140]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-07-26T13:35:55.575 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jul 26 13:35:55 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-osd-11[63909]: 2024-07-26T13:35:55.477+0000 7fee5fd8d640 -1 osd.11 0 waiting for initial osdmap 2024-07-26T13:35:55.576 INFO:journalctl@ceph.osd.11.smithi167.stdout:Jul 26 13:35:55 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-osd-11[63909]: 2024-07-26T13:35:55.490+0000 7fee5bb8b640 -1 osd.11 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-26T13:35:55.741 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:55 smithi100 ceph-mon[32599]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-07-26T13:35:55.741 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:55 smithi100 ceph-mon[32599]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-07-26T13:35:55.741 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:55 smithi100 ceph-mon[32599]: osdmap e69: 12 total, 11 up, 12 in 2024-07-26T13:35:55.741 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-26T13:35:55.741 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:55 smithi100 ceph-mon[32599]: from='osd.11 [v2:172.21.15.167:6824/1032343068,v1:172.21.15.167:6825/1032343068]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-07-26T13:35:55.741 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:55 smithi100 ceph-mon[32599]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-07-26T13:35:55.867 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:55 smithi138 ceph-mon[36954]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-07-26T13:35:55.867 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:55 smithi138 ceph-mon[36954]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-07-26T13:35:55.867 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:55 smithi138 ceph-mon[36954]: osdmap e69: 12 total, 11 up, 12 in 2024-07-26T13:35:55.867 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-26T13:35:55.867 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:55 smithi138 ceph-mon[36954]: from='osd.11 [v2:172.21.15.167:6824/1032343068,v1:172.21.15.167:6825/1032343068]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-07-26T13:35:55.867 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:55 smithi138 ceph-mon[36954]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-07-26T13:35:56.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.090+0000 7f397e4c5640 1 -- 172.21.15.100:0/430019728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3978108720 msgr2=0x7f397810ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:35:56.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.090+0000 7f397e4c5640 1 --2- 172.21.15.100:0/430019728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3978108720 0x7f397810ef50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3970009ca0 tx=0x7f397002a8c0 comp rx=0 tx=0).stop 2024-07-26T13:35:56.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.090+0000 7f397e4c5640 1 -- 172.21.15.100:0/430019728 shutdown_connections 2024-07-26T13:35:56.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.090+0000 7f397e4c5640 1 --2- 172.21.15.100:0/430019728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3978108720 0x7f397810ef50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:35:56.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.090+0000 7f397e4c5640 1 --2- 172.21.15.100:0/430019728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3978107dd0 0x7f39781081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:35:56.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.090+0000 7f397e4c5640 1 --2- 172.21.15.100:0/430019728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3978102490 0x7f39781028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:35:56.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.090+0000 7f397e4c5640 1 -- 172.21.15.100:0/430019728 >> 172.21.15.100:0/430019728 conn(0x7f3978079160 msgr2=0x7f3978079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:35:56.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.090+0000 7f397e4c5640 1 -- 172.21.15.100:0/430019728 shutdown_connections 2024-07-26T13:35:56.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.090+0000 7f397e4c5640 1 -- 172.21.15.100:0/430019728 wait complete. 2024-07-26T13:35:56.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.091+0000 7f397e4c5640 1 Processor -- start 2024-07-26T13:35:56.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.091+0000 7f397e4c5640 1 -- start start 2024-07-26T13:35:56.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.091+0000 7f397e4c5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3978102490 0x7f3978198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:35:56.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.091+0000 7f397e4c5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3978107dd0 0x7f3978199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:35:56.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.092+0000 7f397e4c5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3978108720 0x7f397819e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:35:56.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.092+0000 7f397e4c5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f39781123a0 con 0x7f3978108720 2024-07-26T13:35:56.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.092+0000 7f397e4c5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3978112220 con 0x7f3978102490 2024-07-26T13:35:56.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.092+0000 7f397e4c5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3978112520 con 0x7f3978107dd0 2024-07-26T13:35:56.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.092+0000 7f397ca62640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3978108720 0x7f397819e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:35:56.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.092+0000 7f397ca62640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3978108720 0x7f397819e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58876/0 (socket says 172.21.15.100:58876) 2024-07-26T13:35:56.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.092+0000 7f397ca62640 1 -- 172.21.15.100:0/746538211 learned_addr learned my addr 172.21.15.100:0/746538211 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:35:56.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.092+0000 7f3977fff640 1 --2- 172.21.15.100:0/746538211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3978102490 0x7f3978198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:35:56.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.092+0000 7f396ffff640 1 --2- 172.21.15.100:0/746538211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3978107dd0 0x7f3978199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:35:56.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.092+0000 7f397ca62640 1 -- 172.21.15.100:0/746538211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3978107dd0 msgr2=0x7f3978199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:35:56.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.092+0000 7f397ca62640 1 --2- 172.21.15.100:0/746538211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3978107dd0 0x7f3978199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:35:56.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.092+0000 7f397ca62640 1 -- 172.21.15.100:0/746538211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3978102490 msgr2=0x7f3978198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:35:56.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.092+0000 7f397ca62640 1 --2- 172.21.15.100:0/746538211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3978102490 0x7f3978198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:35:56.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.092+0000 7f397ca62640 1 -- 172.21.15.100:0/746538211 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f397819ed60 con 0x7f3978108720 2024-07-26T13:35:56.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.092+0000 7f396ffff640 1 --2- 172.21.15.100:0/746538211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3978107dd0 0x7f3978199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:35:56.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.092+0000 7f397ca62640 1 --2- 172.21.15.100:0/746538211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3978108720 0x7f397819e630 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f39700098d0 tx=0x7f39700317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:35:56.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.092+0000 7f3975ffb640 1 -- 172.21.15.100:0/746538211 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3970003b70 con 0x7f3978108720 2024-07-26T13:35:56.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.092+0000 7f3975ffb640 1 -- 172.21.15.100:0/746538211 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3970003d10 con 0x7f3978108720 2024-07-26T13:35:56.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.093+0000 7f397e4c5640 1 -- 172.21.15.100:0/746538211 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f397819eff0 con 0x7f3978108720 2024-07-26T13:35:56.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.093+0000 7f3975ffb640 1 -- 172.21.15.100:0/746538211 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397002fa20 con 0x7f3978108720 2024-07-26T13:35:56.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.093+0000 7f397e4c5640 1 -- 172.21.15.100:0/746538211 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39781a57b0 con 0x7f3978108720 2024-07-26T13:35:56.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.094+0000 7f397e4c5640 1 -- 172.21.15.100:0/746538211 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39781028a0 con 0x7f3978108720 2024-07-26T13:35:56.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.096+0000 7f3975ffb640 1 -- 172.21.15.100:0/746538211 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f397002fbe0 con 0x7f3978108720 2024-07-26T13:35:56.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.097+0000 7f3975ffb640 1 --2- 172.21.15.100:0/746538211 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f39640b3ca0 0x7f39640b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:35:56.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.097+0000 7f3977fff640 1 --2- 172.21.15.100:0/746538211 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f39640b3ca0 0x7f39640b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:35:56.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.097+0000 7f3975ffb640 1 -- 172.21.15.100:0/746538211 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7062+0+0 (secure 0 0 0) 0x7f3970104dc0 con 0x7f3978108720 2024-07-26T13:35:56.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.097+0000 7f3977fff640 1 --2- 172.21.15.100:0/746538211 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f39640b3ca0 0x7f39640b61b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f39680058c0 tx=0x7f396800a2d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:35:56.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.100+0000 7f3975ffb640 1 -- 172.21.15.100:0/746538211 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f39700cc850 con 0x7f3978108720 2024-07-26T13:35:56.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.274+0000 7f397e4c5640 1 -- 172.21.15.100:0/746538211 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f3978199f20 con 0x7f3978108720 2024-07-26T13:35:56.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.275+0000 7f3975ffb640 1 -- 172.21.15.100:0/746538211 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7f3978199f20 con 0x7f3978108720 2024-07-26T13:35:56.276 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:35:56.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.277+0000 7f397e4c5640 1 -- 172.21.15.100:0/746538211 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f39640b3ca0 msgr2=0x7f39640b61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:35:56.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.277+0000 7f397e4c5640 1 --2- 172.21.15.100:0/746538211 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f39640b3ca0 0x7f39640b61b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f39680058c0 tx=0x7f396800a2d0 comp rx=0 tx=0).stop 2024-07-26T13:35:56.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.277+0000 7f397e4c5640 1 -- 172.21.15.100:0/746538211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3978108720 msgr2=0x7f397819e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:35:56.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.277+0000 7f397e4c5640 1 --2- 172.21.15.100:0/746538211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3978108720 0x7f397819e630 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f39700098d0 tx=0x7f39700317b0 comp rx=0 tx=0).stop 2024-07-26T13:35:56.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.277+0000 7f397e4c5640 1 -- 172.21.15.100:0/746538211 shutdown_connections 2024-07-26T13:35:56.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.277+0000 7f397e4c5640 1 --2- 172.21.15.100:0/746538211 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f39640b3ca0 0x7f39640b61b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:35:56.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.277+0000 7f397e4c5640 1 --2- 172.21.15.100:0/746538211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3978108720 0x7f397819e630 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:35:56.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.277+0000 7f397e4c5640 1 --2- 172.21.15.100:0/746538211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3978107dd0 0x7f3978199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:35:56.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.277+0000 7f397e4c5640 1 --2- 172.21.15.100:0/746538211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3978102490 0x7f3978198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:35:56.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.277+0000 7f397e4c5640 1 -- 172.21.15.100:0/746538211 >> 172.21.15.100:0/746538211 conn(0x7f3978079160 msgr2=0x7f3978104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:35:56.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.277+0000 7f397e4c5640 1 -- 172.21.15.100:0/746538211 shutdown_connections 2024-07-26T13:35:56.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:35:56.277+0000 7f397e4c5640 1 -- 172.21.15.100:0/746538211 wait complete. 2024-07-26T13:35:56.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:56 smithi167 ceph-mon[37140]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-07-26T13:35:56.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:56 smithi167 ceph-mon[37140]: osdmap e70: 12 total, 11 up, 12 in 2024-07-26T13:35:56.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-26T13:35:56.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-26T13:35:56.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:56 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/746538211' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-26T13:35:56.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:56 smithi100 ceph-mon[32599]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-07-26T13:35:56.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:56 smithi100 ceph-mon[32599]: osdmap e70: 12 total, 11 up, 12 in 2024-07-26T13:35:56.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-26T13:35:56.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-26T13:35:56.571 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:56 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/746538211' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-26T13:35:56.795 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:56 smithi138 ceph-mon[36954]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-07-26T13:35:56.795 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:56 smithi138 ceph-mon[36954]: osdmap e70: 12 total, 11 up, 12 in 2024-07-26T13:35:56.795 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-26T13:35:56.795 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-26T13:35:56.795 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:56 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/746538211' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-26T13:35:57.071 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":70,"num_osds":12,"num_up_osds":11,"osd_up_since":1722000932,"num_in_osds":12,"osd_in_since":1722000937,"num_remapped_pgs":0} 2024-07-26T13:35:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:57 smithi100 ceph-mon[32599]: purged_snaps scrub starts 2024-07-26T13:35:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:57 smithi100 ceph-mon[32599]: purged_snaps scrub ok 2024-07-26T13:35:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:57 smithi100 ceph-mon[32599]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-07-26T13:35:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:57 smithi100 ceph-mon[32599]: osd.11 [v2:172.21.15.167:6824/1032343068,v1:172.21.15.167:6825/1032343068] boot 2024-07-26T13:35:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:57 smithi100 ceph-mon[32599]: osdmap e71: 12 total, 12 up, 12 in 2024-07-26T13:35:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-26T13:35:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:35:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:35:57.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:35:57.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:35:57.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:57.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:35:57.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:35:57.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:57.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:57 smithi167 ceph-mon[37140]: purged_snaps scrub starts 2024-07-26T13:35:57.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:57 smithi167 ceph-mon[37140]: purged_snaps scrub ok 2024-07-26T13:35:57.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:57 smithi167 ceph-mon[37140]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-07-26T13:35:57.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:57 smithi167 ceph-mon[37140]: osd.11 [v2:172.21.15.167:6824/1032343068,v1:172.21.15.167:6825/1032343068] boot 2024-07-26T13:35:57.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:57 smithi167 ceph-mon[37140]: osdmap e71: 12 total, 12 up, 12 in 2024-07-26T13:35:57.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-26T13:35:57.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:57.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:57.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:35:57.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:35:57.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:35:57.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:35:57.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:57.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:35:57.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:35:57.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:57.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:57 smithi138 ceph-mon[36954]: purged_snaps scrub starts 2024-07-26T13:35:57.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:57 smithi138 ceph-mon[36954]: purged_snaps scrub ok 2024-07-26T13:35:57.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:57 smithi138 ceph-mon[36954]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail 2024-07-26T13:35:57.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:57 smithi138 ceph-mon[36954]: osd.11 [v2:172.21.15.167:6824/1032343068,v1:172.21.15.167:6825/1032343068] boot 2024-07-26T13:35:57.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:57 smithi138 ceph-mon[36954]: osdmap e71: 12 total, 12 up, 12 in 2024-07-26T13:35:57.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-26T13:35:57.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:57.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:57.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:35:57.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:35:57.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:35:57.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:35:57.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:57.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:35:57.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:35:57.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:35:58.072 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph osd stat -f json 2024-07-26T13:35:58.330 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:35:58.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:58 smithi100 ceph-mon[32599]: Detected new or changed devices on smithi167 2024-07-26T13:35:58.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:58 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi167 to 4272M 2024-07-26T13:35:58.703 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:58 smithi100 ceph-mon[32599]: osdmap e72: 12 total, 12 up, 12 in 2024-07-26T13:35:58.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:58 smithi167 ceph-mon[37140]: Detected new or changed devices on smithi167 2024-07-26T13:35:58.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:58 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi167 to 4272M 2024-07-26T13:35:58.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:58 smithi167 ceph-mon[37140]: osdmap e72: 12 total, 12 up, 12 in 2024-07-26T13:35:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:58 smithi138 ceph-mon[36954]: Detected new or changed devices on smithi167 2024-07-26T13:35:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:58 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi167 to 4272M 2024-07-26T13:35:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:58 smithi138 ceph-mon[36954]: osdmap e72: 12 total, 12 up, 12 in 2024-07-26T13:35:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:35:59 smithi100 ceph-mon[32599]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:35:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:35:59 smithi167 ceph-mon[37140]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:35:59.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:35:59 smithi138 ceph-mon[36954]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:00.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.150+0000 7f4278e23640 1 -- 172.21.15.100:0/3851620399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42741087e0 msgr2=0x7f427410f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:00.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.150+0000 7f4278e23640 1 --2- 172.21.15.100:0/3851620399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42741087e0 0x7f427410f010 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f426c009ca0 tx=0x7f426c02a8c0 comp rx=0 tx=0).stop 2024-07-26T13:36:00.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.150+0000 7f4278e23640 1 -- 172.21.15.100:0/3851620399 shutdown_connections 2024-07-26T13:36:00.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.150+0000 7f4278e23640 1 --2- 172.21.15.100:0/3851620399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42741087e0 0x7f427410f010 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:00.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.150+0000 7f4278e23640 1 --2- 172.21.15.100:0/3851620399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4274107e90 0x7f42741082a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:00.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.150+0000 7f4278e23640 1 --2- 172.21.15.100:0/3851620399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4274102c20 0x7f4274103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:00.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.150+0000 7f4278e23640 1 -- 172.21.15.100:0/3851620399 >> 172.21.15.100:0/3851620399 conn(0x7f42740798f0 msgr2=0x7f4274079d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:36:00.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.150+0000 7f4278e23640 1 -- 172.21.15.100:0/3851620399 shutdown_connections 2024-07-26T13:36:00.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.150+0000 7f4278e23640 1 -- 172.21.15.100:0/3851620399 wait complete. 2024-07-26T13:36:00.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.151+0000 7f4278e23640 1 Processor -- start 2024-07-26T13:36:00.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.151+0000 7f4278e23640 1 -- start start 2024-07-26T13:36:00.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.151+0000 7f4278e23640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4274102c20 0x7f42740fe920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:00.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.151+0000 7f4278e23640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4274107e90 0x7f42740fee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:00.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.151+0000 7f4278e23640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42741087e0 0x7f42740ff3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:00.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.151+0000 7f4278e23640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f42741124b0 con 0x7f42741087e0 2024-07-26T13:36:00.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.151+0000 7f4278e23640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4274112330 con 0x7f4274107e90 2024-07-26T13:36:00.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.151+0000 7f4278e23640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4274112630 con 0x7f4274102c20 2024-07-26T13:36:00.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.151+0000 7f4272d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42741087e0 0x7f42740ff3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:00.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.151+0000 7f427259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4274102c20 0x7f42740fe920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:00.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.151+0000 7f4272d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42741087e0 0x7f42740ff3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58900/0 (socket says 172.21.15.100:58900) 2024-07-26T13:36:00.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.151+0000 7f4272d9d640 1 -- 172.21.15.100:0/3300545250 learned_addr learned my addr 172.21.15.100:0/3300545250 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:36:00.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.152+0000 7f426bfff640 1 --2- 172.21.15.100:0/3300545250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4274107e90 0x7f42740fee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:00.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.152+0000 7f427259c640 1 -- 172.21.15.100:0/3300545250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4274107e90 msgr2=0x7f42740fee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:00.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.152+0000 7f427259c640 1 --2- 172.21.15.100:0/3300545250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4274107e90 0x7f42740fee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:00.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.152+0000 7f427259c640 1 -- 172.21.15.100:0/3300545250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42741087e0 msgr2=0x7f42740ff3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:00.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.152+0000 7f427259c640 1 --2- 172.21.15.100:0/3300545250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42741087e0 0x7f42740ff3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:00.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.152+0000 7f427259c640 1 -- 172.21.15.100:0/3300545250 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42741a8250 con 0x7f4274102c20 2024-07-26T13:36:00.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.152+0000 7f4272d9d640 1 --2- 172.21.15.100:0/3300545250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42741087e0 0x7f42740ff3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:36:00.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.152+0000 7f427259c640 1 --2- 172.21.15.100:0/3300545250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4274102c20 0x7f42740fe920 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4264005fd0 tx=0x7f426400dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:36:00.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.152+0000 7f426b7fe640 1 -- 172.21.15.100:0/3300545250 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4264009c30 con 0x7f4274102c20 2024-07-26T13:36:00.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.152+0000 7f426b7fe640 1 -- 172.21.15.100:0/3300545250 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4264009dd0 con 0x7f4274102c20 2024-07-26T13:36:00.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.152+0000 7f426b7fe640 1 -- 172.21.15.100:0/3300545250 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4264003e20 con 0x7f4274102c20 2024-07-26T13:36:00.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.152+0000 7f4278e23640 1 -- 172.21.15.100:0/3300545250 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42741a8540 con 0x7f4274102c20 2024-07-26T13:36:00.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.152+0000 7f4278e23640 1 -- 172.21.15.100:0/3300545250 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42741a8a80 con 0x7f4274102c20 2024-07-26T13:36:00.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.153+0000 7f4278e23640 1 -- 172.21.15.100:0/3300545250 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4274103030 con 0x7f4274102c20 2024-07-26T13:36:00.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.155+0000 7f426b7fe640 1 -- 172.21.15.100:0/3300545250 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f4264003fe0 con 0x7f4274102c20 2024-07-26T13:36:00.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.156+0000 7f426b7fe640 1 --2- 172.21.15.100:0/3300545250 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f42600b3c20 0x7f42600b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:00.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.156+0000 7f426bfff640 1 --2- 172.21.15.100:0/3300545250 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f42600b3c20 0x7f42600b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:00.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.156+0000 7f426b7fe640 1 -- 172.21.15.100:0/3300545250 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f42640e3320 con 0x7f4274102c20 2024-07-26T13:36:00.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.156+0000 7f426bfff640 1 --2- 172.21.15.100:0/3300545250 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f42600b3c20 0x7f42600b6130 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f427407a170 tx=0x7f425c005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:36:00.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.159+0000 7f426b7fe640 1 -- 172.21.15.100:0/3300545250 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f4264017020 con 0x7f4274102c20 2024-07-26T13:36:00.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.333+0000 7f4278e23640 1 -- 172.21.15.100:0/3300545250 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f42740ffc20 con 0x7f4274102c20 2024-07-26T13:36:00.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.333+0000 7f426b7fe640 1 -- 172.21.15.100:0/3300545250 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7f42640aa6f0 con 0x7f4274102c20 2024-07-26T13:36:00.335 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:36:00.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.336+0000 7f4278e23640 1 -- 172.21.15.100:0/3300545250 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f42600b3c20 msgr2=0x7f42600b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:00.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.336+0000 7f4278e23640 1 --2- 172.21.15.100:0/3300545250 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f42600b3c20 0x7f42600b6130 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f427407a170 tx=0x7f425c005f90 comp rx=0 tx=0).stop 2024-07-26T13:36:00.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.336+0000 7f4278e23640 1 -- 172.21.15.100:0/3300545250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4274102c20 msgr2=0x7f42740fe920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:00.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.336+0000 7f4278e23640 1 --2- 172.21.15.100:0/3300545250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4274102c20 0x7f42740fe920 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4264005fd0 tx=0x7f426400dde0 comp rx=0 tx=0).stop 2024-07-26T13:36:00.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.336+0000 7f4278e23640 1 -- 172.21.15.100:0/3300545250 shutdown_connections 2024-07-26T13:36:00.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.336+0000 7f4278e23640 1 --2- 172.21.15.100:0/3300545250 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f42600b3c20 0x7f42600b6130 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:00.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.336+0000 7f4278e23640 1 --2- 172.21.15.100:0/3300545250 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42741087e0 0x7f42740ff3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:00.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.336+0000 7f4278e23640 1 --2- 172.21.15.100:0/3300545250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4274107e90 0x7f42740fee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:00.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.336+0000 7f4278e23640 1 --2- 172.21.15.100:0/3300545250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4274102c20 0x7f42740fe920 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:00.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.336+0000 7f4278e23640 1 -- 172.21.15.100:0/3300545250 >> 172.21.15.100:0/3300545250 conn(0x7f42740798f0 msgr2=0x7f4274100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:36:00.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.336+0000 7f4278e23640 1 -- 172.21.15.100:0/3300545250 shutdown_connections 2024-07-26T13:36:00.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:00.336+0000 7f4278e23640 1 -- 172.21.15.100:0/3300545250 wait complete. 2024-07-26T13:36:00.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:00 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/3300545250' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-26T13:36:00.924 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:00 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/3300545250' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-26T13:36:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:00 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/3300545250' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-26T13:36:00.951 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":72,"num_osds":12,"num_up_osds":12,"osd_up_since":1722000956,"num_in_osds":12,"osd_in_since":1722000937,"num_remapped_pgs":0} 2024-07-26T13:36:00.952 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph osd dump --format=json 2024-07-26T13:36:01.202 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:36:01.586 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:01 smithi100 ceph-mon[32599]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:01.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:01 smithi167 ceph-mon[37140]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:01.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:01 smithi138 ceph-mon[36954]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:02 smithi100 ceph-mon[32599]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:02.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:02 smithi167 ceph-mon[37140]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:02 smithi138 ceph-mon[36954]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:03.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.008+0000 7f7261b77640 1 -- 172.21.15.100:0/3712219301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f725c103590 msgr2=0x7f725c1039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:03.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.008+0000 7f7261b77640 1 --2- 172.21.15.100:0/3712219301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f725c103590 0x7f725c1039a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f724c009900 tx=0x7f724c02a410 comp rx=0 tx=0).stop 2024-07-26T13:36:03.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.008+0000 7f7261b77640 1 -- 172.21.15.100:0/3712219301 shutdown_connections 2024-07-26T13:36:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.008+0000 7f7261b77640 1 --2- 172.21.15.100:0/3712219301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f725c109820 0x7f725c110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.008+0000 7f7261b77640 1 --2- 172.21.15.100:0/3712219301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f725c108ed0 0x7f725c1092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.008+0000 7f7261b77640 1 --2- 172.21.15.100:0/3712219301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f725c103590 0x7f725c1039a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.009+0000 7f7261b77640 1 -- 172.21.15.100:0/3712219301 >> 172.21.15.100:0/3712219301 conn(0x7f725c07b280 msgr2=0x7f725c07b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:36:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.009+0000 7f7261b77640 1 -- 172.21.15.100:0/3712219301 shutdown_connections 2024-07-26T13:36:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.009+0000 7f7261b77640 1 -- 172.21.15.100:0/3712219301 wait complete. 2024-07-26T13:36:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.009+0000 7f7261b77640 1 Processor -- start 2024-07-26T13:36:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.009+0000 7f7261b77640 1 -- start start 2024-07-26T13:36:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.009+0000 7f7261b77640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f725c103590 0x7f725c075140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.009+0000 7f7261b77640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f725c108ed0 0x7f725c075680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.009+0000 7f7261b77640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f725c109820 0x7f725c075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.009+0000 7f7261b77640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f725c113540 con 0x7f725c109820 2024-07-26T13:36:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.009+0000 7f7261b77640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f725c1133c0 con 0x7f725c108ed0 2024-07-26T13:36:03.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.009+0000 7f7261b77640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f725c1136c0 con 0x7f725c103590 2024-07-26T13:36:03.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.010+0000 7f725bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f725c109820 0x7f725c075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:03.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.010+0000 7f725bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f725c109820 0x7f725c075bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58912/0 (socket says 172.21.15.100:58912) 2024-07-26T13:36:03.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.010+0000 7f725bfff640 1 -- 172.21.15.100:0/2724506119 learned_addr learned my addr 172.21.15.100:0/2724506119 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:36:03.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.010+0000 7f725b7fe640 1 --2- 172.21.15.100:0/2724506119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f725c103590 0x7f725c075140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:03.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.010+0000 7f725bfff640 1 -- 172.21.15.100:0/2724506119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f725c103590 msgr2=0x7f725c075140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:03.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.010+0000 7f7253fff640 1 --2- 172.21.15.100:0/2724506119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f725c108ed0 0x7f725c075680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:03.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.010+0000 7f725bfff640 1 --2- 172.21.15.100:0/2724506119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f725c103590 0x7f725c075140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:03.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.010+0000 7f725bfff640 1 -- 172.21.15.100:0/2724506119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f725c108ed0 msgr2=0x7f725c075680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:03.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.010+0000 7f725bfff640 1 --2- 172.21.15.100:0/2724506119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f725c108ed0 0x7f725c075680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:03.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.010+0000 7f725bfff640 1 -- 172.21.15.100:0/2724506119 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f725c1a81d0 con 0x7f725c109820 2024-07-26T13:36:03.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.010+0000 7f725bfff640 1 --2- 172.21.15.100:0/2724506119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f725c109820 0x7f725c075bc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f725400b970 tx=0x7f725400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:36:03.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.010+0000 7f72597fa640 1 -- 172.21.15.100:0/2724506119 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7254019070 con 0x7f725c109820 2024-07-26T13:36:03.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.010+0000 7f72597fa640 1 -- 172.21.15.100:0/2724506119 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7254014440 con 0x7f725c109820 2024-07-26T13:36:03.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.010+0000 7f7261b77640 1 -- 172.21.15.100:0/2724506119 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f725c1a84c0 con 0x7f725c109820 2024-07-26T13:36:03.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.010+0000 7f72597fa640 1 -- 172.21.15.100:0/2724506119 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72540075d0 con 0x7f725c109820 2024-07-26T13:36:03.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.010+0000 7f7261b77640 1 -- 172.21.15.100:0/2724506119 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f725c1a8a30 con 0x7f725c109820 2024-07-26T13:36:03.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.011+0000 7f7261b77640 1 -- 172.21.15.100:0/2724506119 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f725c1039a0 con 0x7f725c109820 2024-07-26T13:36:03.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.013+0000 7f72597fa640 1 -- 172.21.15.100:0/2724506119 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f725401c070 con 0x7f725c109820 2024-07-26T13:36:03.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.013+0000 7f72597fa640 1 --2- 172.21.15.100:0/2724506119 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f72480b3c20 0x7f72480b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:03.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.014+0000 7f725b7fe640 1 --2- 172.21.15.100:0/2724506119 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f72480b3c20 0x7f72480b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:03.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.014+0000 7f72597fa640 1 -- 172.21.15.100:0/2724506119 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f72540e33b0 con 0x7f725c109820 2024-07-26T13:36:03.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.014+0000 7f725b7fe640 1 --2- 172.21.15.100:0/2724506119 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f72480b3c20 0x7f72480b6130 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f724c000c00 tx=0x7f724c031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:36:03.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.016+0000 7f72597fa640 1 -- 172.21.15.100:0/2724506119 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f72540aadd0 con 0x7f725c109820 2024-07-26T13:36:03.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.191+0000 7f7261b77640 1 -- 172.21.15.100:0/2724506119 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f725c076440 con 0x7f725c109820 2024-07-26T13:36:03.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.192+0000 7f72597fa640 1 -- 172.21.15.100:0/2724506119 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+18832 (secure 0 0 0) 0x7f72540aa6d0 con 0x7f725c109820 2024-07-26T13:36:03.194 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:36:03.194 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":72,"fsid":"bf83e2e6-4b52-11ef-bcb7-c7b262605968","created":"2024-07-26T13:28:23.639785+0000","modified":"2024-07-26T13:35:57.482051+0000","last_up_change":"2024-07-26T13:35:56.475554+0000","last_in_change":"2024-07-26T13:35:37.472564+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-26T13:32:22.329969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d714036d-9dda-4cb5-af16-29f83a09dd70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6803","nonce":1480401850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6805","nonce":1480401850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6809","nonce":1480401850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6807","nonce":1480401850}]},"public_addr":"172.21.15.100:6803/1480401850","cluster_addr":"172.21.15.100:6805/1480401850","heartbeat_back_addr":"172.21.15.100:6809/1480401850","heartbeat_front_addr":"172.21.15.100:6807/1480401850","state":["exists","up"]},{"osd":1,"uuid":"3dd0eaa1-618a-495a-9819-e01c0f869ae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6801","nonce":3090877671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6803","nonce":3090877671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6807","nonce":3090877671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6805","nonce":3090877671}]},"public_addr":"172.21.15.138:6801/3090877671","cluster_addr":"172.21.15.138:6803/3090877671","heartbeat_back_addr":"172.21.15.138:6807/3090877671","heartbeat_front_addr":"172.21.15.138:6805/3090877671","state":["exists","up"]},{"osd":2,"uuid":"5c710a78-17ae-41a8-ac16-a678f5d35672","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6801","nonce":3583424790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6803","nonce":3583424790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6807","nonce":3583424790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6805","nonce":3583424790}]},"public_addr":"172.21.15.167:6801/3583424790","cluster_addr":"172.21.15.167:6803/3583424790","heartbeat_back_addr":"172.21.15.167:6807/3583424790","heartbeat_front_addr":"172.21.15.167:6805/3583424790","state":["exists","up"]},{"osd":3,"uuid":"be845495-8c31-4834-a45c-bdc576711962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6811","nonce":3626762419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6813","nonce":3626762419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6817","nonce":3626762419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6815","nonce":3626762419}]},"public_addr":"172.21.15.100:6811/3626762419","cluster_addr":"172.21.15.100:6813/3626762419","heartbeat_back_addr":"172.21.15.100:6817/3626762419","heartbeat_front_addr":"172.21.15.100:6815/3626762419","state":["exists","up"]},{"osd":4,"uuid":"520554cc-e846-46f9-af77-05a1de65a452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6809","nonce":3757334356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6811","nonce":3757334356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6815","nonce":3757334356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6813","nonce":3757334356}]},"public_addr":"172.21.15.138:6809/3757334356","cluster_addr":"172.21.15.138:6811/3757334356","heartbeat_back_addr":"172.21.15.138:6815/3757334356","heartbeat_front_addr":"172.21.15.138:6813/3757334356","state":["exists","up"]},{"osd":5,"uuid":"68dc8ad6-31d2-45b5-9aa7-62c4218d46e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6809","nonce":3239482836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6811","nonce":3239482836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6815","nonce":3239482836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6813","nonce":3239482836}]},"public_addr":"172.21.15.167:6809/3239482836","cluster_addr":"172.21.15.167:6811/3239482836","heartbeat_back_addr":"172.21.15.167:6815/3239482836","heartbeat_front_addr":"172.21.15.167:6813/3239482836","state":["exists","up"]},{"osd":6,"uuid":"420e4471-5626-41cc-a8e1-69c801196b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6819","nonce":3218415484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6821","nonce":3218415484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6825","nonce":3218415484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6823","nonce":3218415484}]},"public_addr":"172.21.15.100:6819/3218415484","cluster_addr":"172.21.15.100:6821/3218415484","heartbeat_back_addr":"172.21.15.100:6825/3218415484","heartbeat_front_addr":"172.21.15.100:6823/3218415484","state":["exists","up"]},{"osd":7,"uuid":"a27b097e-207d-44c3-af63-544f228b83ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6817","nonce":45541749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6819","nonce":45541749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6823","nonce":45541749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6821","nonce":45541749}]},"public_addr":"172.21.15.138:6817/45541749","cluster_addr":"172.21.15.138:6819/45541749","heartbeat_back_addr":"172.21.15.138:6823/45541749","heartbeat_front_addr":"172.21.15.138:6821/45541749","state":["exists","up"]},{"osd":8,"uuid":"3e865858-92cc-48a0-a464-5767d19e8ac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6817","nonce":4031159803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6819","nonce":4031159803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6823","nonce":4031159803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6821","nonce":4031159803}]},"public_addr":"172.21.15.167:6817/4031159803","cluster_addr":"172.21.15.167:6819/4031159803","heartbeat_back_addr":"172.21.15.167:6823/4031159803","heartbeat_front_addr":"172.21.15.167:6821/4031159803","state":["exists","up"]},{"osd":9,"uuid":"8b05b14a-1195-42f5-ac01-d5634cb9c888","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6827","nonce":1840140273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6829","nonce":1840140273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6833","nonce":1840140273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6831","nonce":1840140273}]},"public_addr":"172.21.15.100:6827/1840140273","cluster_addr":"172.21.15.100:6829/1840140273","heartbeat_back_addr":"172.21.15.100:6833/1840140273","heartbeat_front_addr":"172.21.15.100:6831/1840140273","state":["exists","up"]},{"osd":10,"uuid":"ab77ebe7-5254-4dc3-bc54-625698e84afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6825","nonce":703382899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6827","nonce":703382899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6831","nonce":703382899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6829","nonce":703382899}]},"public_addr":"172.21.15.138:6825/703382899","cluster_addr":"172.21.15.138:6827/703382899","heartbeat_back_addr":"172.21.15.138:6831/703382899","heartbeat_front_addr":"172.21.15.138:6829/703382899","state":["exists","up"]},{"osd":11,"uuid":"229cf91e-e9ef-48f6-a677-c5c5705b315a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6825","nonce":1032343068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6827","nonce":1032343068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6831","nonce":1032343068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6829","nonce":1032343068}]},"public_addr":"172.21.15.167:6825/1032343068","cluster_addr":"172.21.15.167:6827/1032343068","heartbeat_back_addr":"172.21.15.167:6831/1032343068","heartbeat_front_addr":"172.21.15.167:6829/1032343068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:31:35.800702+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:31:58.087922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:32:20.463183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:32:42.920598+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:33:06.332655+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:33:29.381407+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:33:53.975963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:34:18.226813+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:34:41.845770+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:35:05.722275+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:35:30.862704+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:35:54.738697+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:0/1874993112":"2024-07-27T13:30:16.253096+0000","172.21.15.100:0/2364620305":"2024-07-27T13:28:45.340334+0000","172.21.15.100:6800/3096076075":"2024-07-27T13:29:07.979178+0000","172.21.15.100:0/1894966501":"2024-07-27T13:30:16.253096+0000","172.21.15.100:0/3411776672":"2024-07-27T13:28:45.340334+0000","172.21.15.100:0/2853423120":"2024-07-27T13:29:07.979178+0000","172.21.15.100:6800/3025873570":"2024-07-27T13:28:45.340334+0000","172.21.15.100:0/4887181":"2024-07-27T13:29:07.979178+0000","172.21.15.100:6801/908675140":"2024-07-27T13:30:16.253096+0000","172.21.15.100:6801/3025873570":"2024-07-27T13:28:45.340334+0000","172.21.15.100:6800/908675140":"2024-07-27T13:30:16.253096+0000","172.21.15.100:6801/3096076075":"2024-07-27T13:29:07.979178+0000","172.21.15.100:0/2302632052":"2024-07-27T13:29:07.979178+0000","172.21.15.100:0/3736098915":"2024-07-27T13:28:45.340334+0000","172.21.15.100:0/3243030560":"2024-07-27T13:30:16.253096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-26T13:36:03.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.194+0000 7f7261b77640 1 -- 172.21.15.100:0/2724506119 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f72480b3c20 msgr2=0x7f72480b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:03.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.194+0000 7f7261b77640 1 --2- 172.21.15.100:0/2724506119 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f72480b3c20 0x7f72480b6130 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f724c000c00 tx=0x7f724c031400 comp rx=0 tx=0).stop 2024-07-26T13:36:03.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.194+0000 7f7261b77640 1 -- 172.21.15.100:0/2724506119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f725c109820 msgr2=0x7f725c075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:03.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.194+0000 7f7261b77640 1 --2- 172.21.15.100:0/2724506119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f725c109820 0x7f725c075bc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f725400b970 tx=0x7f725400be30 comp rx=0 tx=0).stop 2024-07-26T13:36:03.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.194+0000 7f7261b77640 1 -- 172.21.15.100:0/2724506119 shutdown_connections 2024-07-26T13:36:03.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.194+0000 7f7261b77640 1 --2- 172.21.15.100:0/2724506119 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f72480b3c20 0x7f72480b6130 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:03.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.194+0000 7f7261b77640 1 --2- 172.21.15.100:0/2724506119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f725c109820 0x7f725c075bc0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:03.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.194+0000 7f7261b77640 1 --2- 172.21.15.100:0/2724506119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f725c108ed0 0x7f725c075680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:03.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.194+0000 7f7261b77640 1 --2- 172.21.15.100:0/2724506119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f725c103590 0x7f725c075140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:03.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.194+0000 7f7261b77640 1 -- 172.21.15.100:0/2724506119 >> 172.21.15.100:0/2724506119 conn(0x7f725c07b280 msgr2=0x7f725c106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:36:03.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.195+0000 7f7261b77640 1 -- 172.21.15.100:0/2724506119 shutdown_connections 2024-07-26T13:36:03.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:03.195+0000 7f7261b77640 1 -- 172.21.15.100:0/2724506119 wait complete. 2024-07-26T13:36:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:03 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/2724506119' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-26T13:36:03.892 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:03 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2724506119' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-26T13:36:03.921 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-26T13:32:22.329969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-26T13:36:03.922 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-07-26T13:36:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:03 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/2724506119' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-26T13:36:04.176 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:36:04.553 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:04 smithi100 ceph-mon[32599]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:04.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:04 smithi167 ceph-mon[37140]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:04 smithi138 ceph-mon[36954]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:05.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.903+0000 7fb18d29f640 1 -- 172.21.15.100:0/3046648386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb188108720 msgr2=0x7fb18810ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:05.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.903+0000 7fb18d29f640 1 --2- 172.21.15.100:0/3046648386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb188108720 0x7fb18810ef50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb180009ca0 tx=0x7fb18002a8c0 comp rx=0 tx=0).stop 2024-07-26T13:36:05.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.903+0000 7fb18d29f640 1 -- 172.21.15.100:0/3046648386 shutdown_connections 2024-07-26T13:36:05.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.903+0000 7fb18d29f640 1 --2- 172.21.15.100:0/3046648386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb188108720 0x7fb18810ef50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:05.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.903+0000 7fb18d29f640 1 --2- 172.21.15.100:0/3046648386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb188107dd0 0x7fb1881081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:05.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.903+0000 7fb18d29f640 1 --2- 172.21.15.100:0/3046648386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb188102490 0x7fb1881028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:05.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.903+0000 7fb18d29f640 1 -- 172.21.15.100:0/3046648386 >> 172.21.15.100:0/3046648386 conn(0x7fb188079160 msgr2=0x7fb188079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:36:05.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.903+0000 7fb18d29f640 1 -- 172.21.15.100:0/3046648386 shutdown_connections 2024-07-26T13:36:05.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.903+0000 7fb18d29f640 1 -- 172.21.15.100:0/3046648386 wait complete. 2024-07-26T13:36:05.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.904+0000 7fb18d29f640 1 Processor -- start 2024-07-26T13:36:05.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.904+0000 7fb18d29f640 1 -- start start 2024-07-26T13:36:05.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.904+0000 7fb18d29f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb188102490 0x7fb188199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:05.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.904+0000 7fb18d29f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb188107dd0 0x7fb18819a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:05.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.904+0000 7fb18d29f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb188108720 0x7fb18819f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:05.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.905+0000 7fb18d29f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb1881134b0 con 0x7fb188107dd0 2024-07-26T13:36:05.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.905+0000 7fb18d29f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb188113330 con 0x7fb188108720 2024-07-26T13:36:05.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.905+0000 7fb18d29f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb188113630 con 0x7fb188102490 2024-07-26T13:36:05.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.905+0000 7fb17ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb188107dd0 0x7fb18819a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:05.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.905+0000 7fb17ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb188107dd0 0x7fb18819a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46318/0 (socket says 172.21.15.100:46318) 2024-07-26T13:36:05.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.905+0000 7fb17ffff640 1 -- 172.21.15.100:0/539542589 learned_addr learned my addr 172.21.15.100:0/539542589 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:36:05.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.905+0000 7fb186ffd640 1 --2- 172.21.15.100:0/539542589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb188102490 0x7fb188199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:05.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.905+0000 7fb17ffff640 1 -- 172.21.15.100:0/539542589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb188102490 msgr2=0x7fb188199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:05.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.905+0000 7fb1877fe640 1 --2- 172.21.15.100:0/539542589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb188108720 0x7fb18819f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:05.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.905+0000 7fb17ffff640 1 --2- 172.21.15.100:0/539542589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb188102490 0x7fb188199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:05.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.905+0000 7fb17ffff640 1 -- 172.21.15.100:0/539542589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb188108720 msgr2=0x7fb18819f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:05.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.905+0000 7fb17ffff640 1 --2- 172.21.15.100:0/539542589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb188108720 0x7fb18819f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:05.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.905+0000 7fb17ffff640 1 -- 172.21.15.100:0/539542589 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb18819fcf0 con 0x7fb188107dd0 2024-07-26T13:36:05.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.905+0000 7fb186ffd640 1 --2- 172.21.15.100:0/539542589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb188102490 0x7fb188199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:36:05.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.905+0000 7fb17ffff640 1 --2- 172.21.15.100:0/539542589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb188107dd0 0x7fb18819a3e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb17000c970 tx=0x7fb17000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:36:05.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.905+0000 7fb184ff9640 1 -- 172.21.15.100:0/539542589 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb170014070 con 0x7fb188107dd0 2024-07-26T13:36:05.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.905+0000 7fb184ff9640 1 -- 172.21.15.100:0/539542589 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb170004440 con 0x7fb188107dd0 2024-07-26T13:36:05.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.905+0000 7fb18d29f640 1 -- 172.21.15.100:0/539542589 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb18819ffe0 con 0x7fb188107dd0 2024-07-26T13:36:05.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.905+0000 7fb184ff9640 1 -- 172.21.15.100:0/539542589 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb170003e50 con 0x7fb188107dd0 2024-07-26T13:36:05.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.906+0000 7fb18d29f640 1 -- 172.21.15.100:0/539542589 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb18810f8c0 con 0x7fb188107dd0 2024-07-26T13:36:05.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.907+0000 7fb18d29f640 1 -- 172.21.15.100:0/539542589 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb188103710 con 0x7fb188107dd0 2024-07-26T13:36:05.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.908+0000 7fb184ff9640 1 -- 172.21.15.100:0/539542589 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7fb170017070 con 0x7fb188107dd0 2024-07-26T13:36:05.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.909+0000 7fb184ff9640 1 --2- 172.21.15.100:0/539542589 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb1740b3df0 0x7fb1740b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:05.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.909+0000 7fb186ffd640 1 --2- 172.21.15.100:0/539542589 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb1740b3df0 0x7fb1740b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:05.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.909+0000 7fb184ff9640 1 -- 172.21.15.100:0/539542589 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7fb1700e3c20 con 0x7fb188107dd0 2024-07-26T13:36:05.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.910+0000 7fb186ffd640 1 --2- 172.21.15.100:0/539542589 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb1740b3df0 0x7fb1740b6300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb1780058c0 tx=0x7fb178005d30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:36:05.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:05.912+0000 7fb184ff9640 1 -- 172.21.15.100:0/539542589 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fb17001e030 con 0x7fb188107dd0 2024-07-26T13:36:06.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:06.086+0000 7fb18d29f640 1 -- 172.21.15.100:0/539542589 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb18810a4f0 con 0x7fb188107dd0 2024-07-26T13:36:06.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:06.087+0000 7fb184ff9640 1 -- 172.21.15.100:0/539542589 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7fb1700035f0 con 0x7fb188107dd0 2024-07-26T13:36:06.088 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-07-26T13:36:06.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:06.089+0000 7fb18d29f640 1 -- 172.21.15.100:0/539542589 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb1740b3df0 msgr2=0x7fb1740b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:06.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:06.089+0000 7fb18d29f640 1 --2- 172.21.15.100:0/539542589 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb1740b3df0 0x7fb1740b6300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb1780058c0 tx=0x7fb178005d30 comp rx=0 tx=0).stop 2024-07-26T13:36:06.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:06.089+0000 7fb18d29f640 1 -- 172.21.15.100:0/539542589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb188107dd0 msgr2=0x7fb18819a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:06.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:06.089+0000 7fb18d29f640 1 --2- 172.21.15.100:0/539542589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb188107dd0 0x7fb18819a3e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb17000c970 tx=0x7fb17000ce30 comp rx=0 tx=0).stop 2024-07-26T13:36:06.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:06.089+0000 7fb18d29f640 1 -- 172.21.15.100:0/539542589 shutdown_connections 2024-07-26T13:36:06.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:06.089+0000 7fb18d29f640 1 --2- 172.21.15.100:0/539542589 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb1740b3df0 0x7fb1740b6300 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:06.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:06.089+0000 7fb18d29f640 1 --2- 172.21.15.100:0/539542589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb188108720 0x7fb18819f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:06.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:06.089+0000 7fb18d29f640 1 --2- 172.21.15.100:0/539542589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb188107dd0 0x7fb18819a3e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:06.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:06.089+0000 7fb18d29f640 1 --2- 172.21.15.100:0/539542589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb188102490 0x7fb188199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:06.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:06.089+0000 7fb18d29f640 1 -- 172.21.15.100:0/539542589 >> 172.21.15.100:0/539542589 conn(0x7fb188079160 msgr2=0x7fb188103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:36:06.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:06.089+0000 7fb18d29f640 1 -- 172.21.15.100:0/539542589 shutdown_connections 2024-07-26T13:36:06.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:06.089+0000 7fb18d29f640 1 -- 172.21.15.100:0/539542589 wait complete. 2024-07-26T13:36:06.391 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:06 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/539542589' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-26T13:36:06.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:06 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/539542589' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-26T13:36:06.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:06 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/539542589' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-26T13:36:06.812 INFO:tasks.cephadm:Adding mds.a on smithi100 2024-07-26T13:36:06.812 INFO:tasks.cephadm:Adding mds.d on smithi100 2024-07-26T13:36:06.812 INFO:tasks.cephadm:Adding mds.g on smithi100 2024-07-26T13:36:06.812 INFO:tasks.cephadm:Adding mds.j on smithi100 2024-07-26T13:36:06.812 INFO:tasks.cephadm:Adding mds.b on smithi138 2024-07-26T13:36:06.812 INFO:tasks.cephadm:Adding mds.e on smithi138 2024-07-26T13:36:06.812 INFO:tasks.cephadm:Adding mds.h on smithi138 2024-07-26T13:36:06.812 INFO:tasks.cephadm:Adding mds.k on smithi138 2024-07-26T13:36:06.812 INFO:tasks.cephadm:Adding mds.c on smithi167 2024-07-26T13:36:06.812 INFO:tasks.cephadm:Adding mds.f on smithi167 2024-07-26T13:36:06.812 INFO:tasks.cephadm:Adding mds.i on smithi167 2024-07-26T13:36:06.813 INFO:tasks.cephadm:Adding mds.l on smithi167 2024-07-26T13:36:06.813 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph orch apply mds all '12;smithi100=a;smithi100=d;smithi100=g;smithi100=j;smithi138=b;smithi138=e;smithi138=h;smithi138=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l' 2024-07-26T13:36:07.077 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.c/config 2024-07-26T13:36:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:07 smithi138 ceph-mon[36954]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:07.457 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:07 smithi167 ceph-mon[37140]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:07.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:07 smithi100 ceph-mon[32599]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:08.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.699+0000 7f9808729640 1 -- 172.21.15.167:0/3172155578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9800108ed0 msgr2=0x7f98001092e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:08.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.699+0000 7f9808729640 1 --2- 172.21.15.167:0/3172155578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9800108ed0 0x7f98001092e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f97f0009900 tx=0x7f97f002a410 comp rx=0 tx=0).stop 2024-07-26T13:36:08.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.699+0000 7f9808729640 1 -- 172.21.15.167:0/3172155578 shutdown_connections 2024-07-26T13:36:08.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.699+0000 7f9808729640 1 --2- 172.21.15.167:0/3172155578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9800109820 0x7f9800110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:08.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.699+0000 7f9808729640 1 --2- 172.21.15.167:0/3172155578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9800108ed0 0x7f98001092e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:08.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.699+0000 7f9808729640 1 --2- 172.21.15.167:0/3172155578 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9800103590 0x7f98001039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:08.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.699+0000 7f9808729640 1 -- 172.21.15.167:0/3172155578 >> 172.21.15.167:0/3172155578 conn(0x7f980007b280 msgr2=0x7f980007b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:36:08.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.699+0000 7f9808729640 1 -- 172.21.15.167:0/3172155578 shutdown_connections 2024-07-26T13:36:08.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.699+0000 7f9808729640 1 -- 172.21.15.167:0/3172155578 wait complete. 2024-07-26T13:36:08.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.700+0000 7f9808729640 1 Processor -- start 2024-07-26T13:36:08.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.700+0000 7f9808729640 1 -- start start 2024-07-26T13:36:08.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.700+0000 7f9808729640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9800103590 0x7f9800199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:08.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.700+0000 7f9808729640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9800108ed0 0x7f980019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:08.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.700+0000 7f9808729640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9800109820 0x7f980019f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:08.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.700+0000 7f9808729640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9800113500 con 0x7f9800108ed0 2024-07-26T13:36:08.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.700+0000 7f9808729640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9800113380 con 0x7f9800103590 2024-07-26T13:36:08.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.700+0000 7f9808729640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9800113680 con 0x7f9800109820 2024-07-26T13:36:08.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.700+0000 7f9806cc6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9800109820 0x7f980019f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:08.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.700+0000 7f9806cc6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9800109820 0x7f980019f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54014/0 (socket says 172.21.15.167:54014) 2024-07-26T13:36:08.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.700+0000 7f9806cc6640 1 -- 172.21.15.167:0/1026017653 learned_addr learned my addr 172.21.15.167:0/1026017653 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-07-26T13:36:08.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.700+0000 7f9805cc4640 1 --2- 172.21.15.167:0/1026017653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9800108ed0 0x7f980019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:08.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.700+0000 7f98064c5640 1 --2- 172.21.15.167:0/1026017653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9800103590 0x7f9800199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:08.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.700+0000 7f9806cc6640 1 -- 172.21.15.167:0/1026017653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9800103590 msgr2=0x7f9800199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:08.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.700+0000 7f9806cc6640 1 --2- 172.21.15.167:0/1026017653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9800103590 0x7f9800199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:08.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.700+0000 7f9806cc6640 1 -- 172.21.15.167:0/1026017653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9800108ed0 msgr2=0x7f980019a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:08.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.700+0000 7f9806cc6640 1 --2- 172.21.15.167:0/1026017653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9800108ed0 0x7f980019a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:08.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.700+0000 7f9806cc6640 1 -- 172.21.15.167:0/1026017653 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f980019fda0 con 0x7f9800109820 2024-07-26T13:36:08.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.700+0000 7f9805cc4640 1 --2- 172.21.15.167:0/1026017653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9800108ed0 0x7f980019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:36:08.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.700+0000 7f98064c5640 1 --2- 172.21.15.167:0/1026017653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9800103590 0x7f9800199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:36:08.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.701+0000 7f9806cc6640 1 --2- 172.21.15.167:0/1026017653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9800109820 0x7f980019f730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f97f800b970 tx=0x7f97f800be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:36:08.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.701+0000 7f97f77fe640 1 -- 172.21.15.167:0/1026017653 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f8019070 con 0x7f9800109820 2024-07-26T13:36:08.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.701+0000 7f9808729640 1 -- 172.21.15.167:0/1026017653 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98001a0090 con 0x7f9800109820 2024-07-26T13:36:08.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.701+0000 7f97f77fe640 1 -- 172.21.15.167:0/1026017653 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97f8014440 con 0x7f9800109820 2024-07-26T13:36:08.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.701+0000 7f97f77fe640 1 -- 172.21.15.167:0/1026017653 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f80075d0 con 0x7f9800109820 2024-07-26T13:36:08.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.701+0000 7f9808729640 1 -- 172.21.15.167:0/1026017653 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98001a68b0 con 0x7f9800109820 2024-07-26T13:36:08.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.702+0000 7f9808729640 1 -- 172.21.15.167:0/1026017653 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98001039a0 con 0x7f9800109820 2024-07-26T13:36:08.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.703+0000 7f97f77fe640 1 -- 172.21.15.167:0/1026017653 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f97f801c070 con 0x7f9800109820 2024-07-26T13:36:08.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.704+0000 7f97f77fe640 1 --2- 172.21.15.167:0/1026017653 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f97ec0b3c50 0x7f97ec0b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:08.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.704+0000 7f97f77fe640 1 -- 172.21.15.167:0/1026017653 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f97f80e3340 con 0x7f9800109820 2024-07-26T13:36:08.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.704+0000 7f98064c5640 1 --2- 172.21.15.167:0/1026017653 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f97ec0b3c50 0x7f97ec0b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:08.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.705+0000 7f98064c5640 1 --2- 172.21.15.167:0/1026017653 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f97ec0b3c50 0x7f97ec0b6160 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f97fc00baa0 tx=0x7f97fc005d90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:36:08.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.707+0000 7f97f77fe640 1 -- 172.21.15.167:0/1026017653 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f97f80aad60 con 0x7f9800109820 2024-07-26T13:36:08.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.892+0000 7f9808729640 1 -- 172.21.15.167:0/1026017653 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi100=a;smithi100=d;smithi100=g;smithi100=j;smithi138=b;smithi138=e;smithi138=h;smithi138=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}) -- 0x7f9800063230 con 0x7f97ec0b3c50 2024-07-26T13:36:08.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.900+0000 7f97f77fe640 1 -- 172.21.15.167:0/1026017653 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f9800063230 con 0x7f97ec0b3c50 2024-07-26T13:36:08.902 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled mds.all update... 2024-07-26T13:36:08.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.903+0000 7f9808729640 1 -- 172.21.15.167:0/1026017653 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f97ec0b3c50 msgr2=0x7f97ec0b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:08.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.903+0000 7f9808729640 1 --2- 172.21.15.167:0/1026017653 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f97ec0b3c50 0x7f97ec0b6160 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f97fc00baa0 tx=0x7f97fc005d90 comp rx=0 tx=0).stop 2024-07-26T13:36:08.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.903+0000 7f9808729640 1 -- 172.21.15.167:0/1026017653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9800109820 msgr2=0x7f980019f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:08.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.903+0000 7f9808729640 1 --2- 172.21.15.167:0/1026017653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9800109820 0x7f980019f730 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f97f800b970 tx=0x7f97f800be30 comp rx=0 tx=0).stop 2024-07-26T13:36:08.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.903+0000 7f9808729640 1 -- 172.21.15.167:0/1026017653 shutdown_connections 2024-07-26T13:36:08.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.903+0000 7f9808729640 1 --2- 172.21.15.167:0/1026017653 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f97ec0b3c50 0x7f97ec0b6160 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:08.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.903+0000 7f9808729640 1 --2- 172.21.15.167:0/1026017653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9800109820 0x7f980019f730 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:08.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.903+0000 7f9808729640 1 --2- 172.21.15.167:0/1026017653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9800108ed0 0x7f980019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:08.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.903+0000 7f9808729640 1 --2- 172.21.15.167:0/1026017653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9800103590 0x7f9800199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:08.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.903+0000 7f9808729640 1 -- 172.21.15.167:0/1026017653 >> 172.21.15.167:0/1026017653 conn(0x7f980007b280 msgr2=0x7f9800106f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:36:08.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.903+0000 7f9808729640 1 -- 172.21.15.167:0/1026017653 shutdown_connections 2024-07-26T13:36:08.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-07-26T13:36:08.903+0000 7f9808729640 1 -- 172.21.15.167:0/1026017653 wait complete. 2024-07-26T13:36:09.535 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:09 smithi167 ceph-mon[37140]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:09.535 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:09.535 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:36:09.535 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:09.535 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:36:09.535 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:09.535 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-26T13:36:09.535 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-26T13:36:09.535 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:09.657 DEBUG:teuthology.orchestra.run.smithi100:mds.a> sudo journalctl -f -n 0 -u ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.a.service 2024-07-26T13:36:09.660 DEBUG:teuthology.orchestra.run.smithi100:mds.d> sudo journalctl -f -n 0 -u ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.d.service 2024-07-26T13:36:09.681 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:09 smithi100 ceph-mon[32599]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:09.681 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:09.681 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:36:09.681 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:09.681 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:36:09.681 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:09.681 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-26T13:36:09.681 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-26T13:36:09.681 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:09.682 DEBUG:teuthology.orchestra.run.smithi100:mds.g> sudo journalctl -f -n 0 -u ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.g.service 2024-07-26T13:36:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:09 smithi138 ceph-mon[36954]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:36:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:36:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-26T13:36:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-26T13:36:09.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:09.726 DEBUG:teuthology.orchestra.run.smithi100:mds.j> sudo journalctl -f -n 0 -u ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.j.service 2024-07-26T13:36:09.769 DEBUG:teuthology.orchestra.run.smithi138:mds.b> sudo journalctl -f -n 0 -u ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.b.service 2024-07-26T13:36:09.772 DEBUG:teuthology.orchestra.run.smithi138:mds.e> sudo journalctl -f -n 0 -u ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.e.service 2024-07-26T13:36:09.815 DEBUG:teuthology.orchestra.run.smithi138:mds.h> sudo journalctl -f -n 0 -u ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.h.service 2024-07-26T13:36:09.858 DEBUG:teuthology.orchestra.run.smithi138:mds.k> sudo journalctl -f -n 0 -u ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.k.service 2024-07-26T13:36:09.901 DEBUG:teuthology.orchestra.run.smithi167:mds.c> sudo journalctl -f -n 0 -u ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.c.service 2024-07-26T13:36:09.904 DEBUG:teuthology.orchestra.run.smithi167:mds.f> sudo journalctl -f -n 0 -u ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.f.service 2024-07-26T13:36:09.946 DEBUG:teuthology.orchestra.run.smithi167:mds.i> sudo journalctl -f -n 0 -u ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.i.service 2024-07-26T13:36:09.990 DEBUG:teuthology.orchestra.run.smithi167:mds.l> sudo journalctl -f -n 0 -u ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.l.service 2024-07-26T13:36:10.033 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-07-26T13:36:10.033 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:36:10.283 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:36:10.658 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:10 smithi100 ceph-mon[32599]: from='client.24541 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi100=a;smithi100=d;smithi100=g;smithi100=j;smithi138=b;smithi138=e;smithi138=h;smithi138=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:36:10.658 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:10 smithi100 ceph-mon[32599]: Saving service mds.all spec with placement smithi100=a;smithi100=d;smithi100=g;smithi100=j;smithi138=b;smithi138=e;smithi138=h;smithi138=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2024-07-26T13:36:10.658 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:10 smithi100 ceph-mon[32599]: Deploying daemon mds.l on smithi167 2024-07-26T13:36:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:10 smithi138 ceph-mon[36954]: from='client.24541 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi100=a;smithi100=d;smithi100=g;smithi100=j;smithi138=b;smithi138=e;smithi138=h;smithi138=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:36:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:10 smithi138 ceph-mon[36954]: Saving service mds.all spec with placement smithi100=a;smithi100=d;smithi100=g;smithi100=j;smithi138=b;smithi138=e;smithi138=h;smithi138=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2024-07-26T13:36:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:10 smithi138 ceph-mon[36954]: Deploying daemon mds.l on smithi167 2024-07-26T13:36:10.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:10 smithi167 ceph-mon[37140]: from='client.24541 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi100=a;smithi100=d;smithi100=g;smithi100=j;smithi138=b;smithi138=e;smithi138=h;smithi138=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:36:10.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:10 smithi167 ceph-mon[37140]: Saving service mds.all spec with placement smithi100=a;smithi100=d;smithi100=g;smithi100=j;smithi138=b;smithi138=e;smithi138=h;smithi138=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2024-07-26T13:36:10.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:10 smithi167 ceph-mon[37140]: Deploying daemon mds.l on smithi167 2024-07-26T13:36:11.407 INFO:journalctl@ceph.mds.l.smithi167.stdout:Jul 26 13:36:11 smithi167 systemd[1]: Starting Ceph mds.l for bf83e2e6-4b52-11ef-bcb7-c7b262605968... 2024-07-26T13:36:11.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:11 smithi100 ceph-mon[32599]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:11.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:11 smithi138 ceph-mon[36954]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:11.723 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:11 smithi167 ceph-mon[37140]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:11.723 INFO:journalctl@ceph.mds.l.smithi167.stdout:Jul 26 13:36:11 smithi167 podman[68410]: 2024-07-26 13:36:11.391853248 +0000 UTC m=+0.017417937 image pull ae51f432d4fa82622bc253ca9e2da7666c617e096e99d43d0a0e4fdaca8865f8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2 2024-07-26T13:36:11.723 INFO:journalctl@ceph.mds.l.smithi167.stdout:Jul 26 13:36:11 smithi167 podman[68410]: 2024-07-26 13:36:11.494650946 +0000 UTC m=+0.120215622 container create dad30bd935ace34bc1e311cf761cb9e2a2a733feaebea9948fa5fa6d79648347 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-07-26T13:36:11.723 INFO:journalctl@ceph.mds.l.smithi167.stdout:Jul 26 13:36:11 smithi167 podman[68410]: 2024-07-26 13:36:11.722993458 +0000 UTC m=+0.348558135 container init dad30bd935ace34bc1e311cf761cb9e2a2a733feaebea9948fa5fa6d79648347 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-l, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, ceph=True, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-07-26T13:36:12.022 INFO:journalctl@ceph.mds.l.smithi167.stdout:Jul 26 13:36:11 smithi167 podman[68410]: 2024-07-26 13:36:11.72590537 +0000 UTC m=+0.351470047 container start dad30bd935ace34bc1e311cf761cb9e2a2a733feaebea9948fa5fa6d79648347 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-l, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240724, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-07-26T13:36:12.022 INFO:journalctl@ceph.mds.l.smithi167.stdout:Jul 26 13:36:11 smithi167 ceph-mds[68430]: -- 172.21.15.167:0/1554150023 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5632b16a5860 con 0x5632b5847600 2024-07-26T13:36:12.022 INFO:journalctl@ceph.mds.l.smithi167.stdout:Jul 26 13:36:11 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-l[68426]: starting mds.l at 2024-07-26T13:36:12.023 INFO:journalctl@ceph.mds.l.smithi167.stdout:Jul 26 13:36:11 smithi167 bash[68410]: dad30bd935ace34bc1e311cf761cb9e2a2a733feaebea9948fa5fa6d79648347 2024-07-26T13:36:12.023 INFO:journalctl@ceph.mds.l.smithi167.stdout:Jul 26 13:36:11 smithi167 systemd[1]: Started Ceph mds.l for bf83e2e6-4b52-11ef-bcb7-c7b262605968. 2024-07-26T13:36:12.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.061+0000 7f90f2a07640 1 -- 172.21.15.100:0/83358347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90ec103590 msgr2=0x7f90ec1039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:12.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.061+0000 7f90f2a07640 1 --2- 172.21.15.100:0/83358347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90ec103590 0x7f90ec1039a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f90dc009900 tx=0x7f90dc02a410 comp rx=0 tx=0).stop 2024-07-26T13:36:12.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.061+0000 7f90f2a07640 1 -- 172.21.15.100:0/83358347 shutdown_connections 2024-07-26T13:36:12.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.061+0000 7f90f2a07640 1 --2- 172.21.15.100:0/83358347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90ec109820 0x7f90ec110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:12.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.061+0000 7f90f2a07640 1 --2- 172.21.15.100:0/83358347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90ec108ed0 0x7f90ec1092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:12.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.061+0000 7f90f2a07640 1 --2- 172.21.15.100:0/83358347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90ec103590 0x7f90ec1039a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:12.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.061+0000 7f90f2a07640 1 -- 172.21.15.100:0/83358347 >> 172.21.15.100:0/83358347 conn(0x7f90ec07b280 msgr2=0x7f90ec07b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:36:12.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.061+0000 7f90f2a07640 1 -- 172.21.15.100:0/83358347 shutdown_connections 2024-07-26T13:36:12.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.061+0000 7f90f2a07640 1 -- 172.21.15.100:0/83358347 wait complete. 2024-07-26T13:36:12.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.062+0000 7f90f2a07640 1 Processor -- start 2024-07-26T13:36:12.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.062+0000 7f90f2a07640 1 -- start start 2024-07-26T13:36:12.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.063+0000 7f90f2a07640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90ec103590 0x7f90ec199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:12.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.063+0000 7f90f2a07640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90ec108ed0 0x7f90ec19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:12.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.063+0000 7f90f2a07640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90ec109820 0x7f90ec19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:12.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.063+0000 7f90f2a07640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f90ec1134f0 con 0x7f90ec103590 2024-07-26T13:36:12.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.063+0000 7f90f2a07640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f90ec113370 con 0x7f90ec109820 2024-07-26T13:36:12.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.063+0000 7f90f2a07640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f90ec113670 con 0x7f90ec108ed0 2024-07-26T13:36:12.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.063+0000 7f90ebfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90ec103590 0x7f90ec199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:12.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.063+0000 7f90ebfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90ec103590 0x7f90ec199ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46334/0 (socket says 172.21.15.100:46334) 2024-07-26T13:36:12.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.063+0000 7f90ebfff640 1 -- 172.21.15.100:0/3680073469 learned_addr learned my addr 172.21.15.100:0/3680073469 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:36:12.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.063+0000 7f90eb7fe640 1 --2- 172.21.15.100:0/3680073469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90ec108ed0 0x7f90ec19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:12.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.063+0000 7f90eb7fe640 1 -- 172.21.15.100:0/3680073469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90ec109820 msgr2=0x7f90ec19f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:12.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.063+0000 7f90eb7fe640 1 --2- 172.21.15.100:0/3680073469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90ec109820 0x7f90ec19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:12.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.063+0000 7f90eb7fe640 1 -- 172.21.15.100:0/3680073469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90ec103590 msgr2=0x7f90ec199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:12.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.063+0000 7f90eb7fe640 1 --2- 172.21.15.100:0/3680073469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90ec103590 0x7f90ec199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:12.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.063+0000 7f90eb7fe640 1 -- 172.21.15.100:0/3680073469 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90ec19fe50 con 0x7f90ec108ed0 2024-07-26T13:36:12.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.063+0000 7f90ebfff640 1 --2- 172.21.15.100:0/3680073469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90ec103590 0x7f90ec199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:36:12.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.064+0000 7f90eb7fe640 1 --2- 172.21.15.100:0/3680073469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90ec108ed0 0x7f90ec19a480 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f90ec06c420 tx=0x7f90e000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:36:12.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.064+0000 7f90e97fa640 1 -- 172.21.15.100:0/3680073469 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e000cc40 con 0x7f90ec108ed0 2024-07-26T13:36:12.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.064+0000 7f90f2a07640 1 -- 172.21.15.100:0/3680073469 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90ec1a0140 con 0x7f90ec108ed0 2024-07-26T13:36:12.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.064+0000 7f90e97fa640 1 -- 172.21.15.100:0/3680073469 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90e000cde0 con 0x7f90ec108ed0 2024-07-26T13:36:12.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.064+0000 7f90e97fa640 1 -- 172.21.15.100:0/3680073469 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e0004a00 con 0x7f90ec108ed0 2024-07-26T13:36:12.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.064+0000 7f90f2a07640 1 -- 172.21.15.100:0/3680073469 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90ec075020 con 0x7f90ec108ed0 2024-07-26T13:36:12.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.065+0000 7f90f2a07640 1 -- 172.21.15.100:0/3680073469 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90ec1039a0 con 0x7f90ec108ed0 2024-07-26T13:36:12.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.066+0000 7f90e97fa640 1 -- 172.21.15.100:0/3680073469 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f90e0004ba0 con 0x7f90ec108ed0 2024-07-26T13:36:12.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.067+0000 7f90e97fa640 1 --2- 172.21.15.100:0/3680073469 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f90d40b3c20 0x7f90d40b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:12.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.067+0000 7f90ebfff640 1 --2- 172.21.15.100:0/3680073469 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f90d40b3c20 0x7f90d40b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:12.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.067+0000 7f90e97fa640 1 -- 172.21.15.100:0/3680073469 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f90e00e39c0 con 0x7f90ec108ed0 2024-07-26T13:36:12.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.068+0000 7f90ebfff640 1 --2- 172.21.15.100:0/3680073469 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f90d40b3c20 0x7f90d40b6130 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f90dc031440 tx=0x7f90dc0058d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:36:12.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.070+0000 7f90e97fa640 1 -- 172.21.15.100:0/3680073469 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f90e00ab380 con 0x7f90ec108ed0 2024-07-26T13:36:12.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.278+0000 7f90f2a07640 1 -- 172.21.15.100:0/3680073469 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90ec19b010 con 0x7f90ec108ed0 2024-07-26T13:36:12.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.279+0000 7f90e97fa640 1 -- 172.21.15.100:0/3680073469 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1435 (secure 0 0 0) 0x7f90ec19b010 con 0x7f90ec108ed0 2024-07-26T13:36:12.280 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:36:12.280 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":2,"btime":"2024-07-26T13:36:11:763593+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-07-26T13:36:12.280 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 2 2024-07-26T13:36:12.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.281+0000 7f90f2a07640 1 -- 172.21.15.100:0/3680073469 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f90d40b3c20 msgr2=0x7f90d40b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:12.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.281+0000 7f90f2a07640 1 --2- 172.21.15.100:0/3680073469 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f90d40b3c20 0x7f90d40b6130 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f90dc031440 tx=0x7f90dc0058d0 comp rx=0 tx=0).stop 2024-07-26T13:36:12.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.281+0000 7f90f2a07640 1 -- 172.21.15.100:0/3680073469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90ec108ed0 msgr2=0x7f90ec19a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:12.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.281+0000 7f90f2a07640 1 --2- 172.21.15.100:0/3680073469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90ec108ed0 0x7f90ec19a480 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f90ec06c420 tx=0x7f90e000ee50 comp rx=0 tx=0).stop 2024-07-26T13:36:12.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.281+0000 7f90f2a07640 1 -- 172.21.15.100:0/3680073469 shutdown_connections 2024-07-26T13:36:12.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.281+0000 7f90f2a07640 1 --2- 172.21.15.100:0/3680073469 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f90d40b3c20 0x7f90d40b6130 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:12.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.281+0000 7f90f2a07640 1 --2- 172.21.15.100:0/3680073469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90ec109820 0x7f90ec19f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:12.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.281+0000 7f90f2a07640 1 --2- 172.21.15.100:0/3680073469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90ec108ed0 0x7f90ec19a480 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:12.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.281+0000 7f90f2a07640 1 --2- 172.21.15.100:0/3680073469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90ec103590 0x7f90ec199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:12.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.281+0000 7f90f2a07640 1 -- 172.21.15.100:0/3680073469 >> 172.21.15.100:0/3680073469 conn(0x7f90ec07b280 msgr2=0x7f90ec106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:36:12.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.281+0000 7f90f2a07640 1 -- 172.21.15.100:0/3680073469 shutdown_connections 2024-07-26T13:36:12.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:12.281+0000 7f90f2a07640 1 -- 172.21.15.100:0/3680073469 wait complete. 2024-07-26T13:36:12.935 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:12 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.167:6832/174302302,v1:172.21.15.167:6833/174302302] up:boot 2024-07-26T13:36:12.935 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:12 smithi100 ceph-mon[32599]: fsmap 1 up:standby 2024-07-26T13:36:12.935 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:12 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-26T13:36:12.935 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:12 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:12.935 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:12 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:12.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:12 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:12.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:12 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-26T13:36:12.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:12 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-26T13:36:12.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:12 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:12.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:12 smithi100 ceph-mon[32599]: Deploying daemon mds.k on smithi138 2024-07-26T13:36:12.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:12 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/3680073469' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:36:12.936 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:12 smithi100 ceph-mon[32599]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:12 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.167:6832/174302302,v1:172.21.15.167:6833/174302302] up:boot 2024-07-26T13:36:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:12 smithi138 ceph-mon[36954]: fsmap 1 up:standby 2024-07-26T13:36:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:12 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-26T13:36:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:12 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:12 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:12 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:12 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-26T13:36:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:12 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-26T13:36:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:12 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:12 smithi138 ceph-mon[36954]: Deploying daemon mds.k on smithi138 2024-07-26T13:36:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:12 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/3680073469' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:36:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:12 smithi138 ceph-mon[36954]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:12 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.167:6832/174302302,v1:172.21.15.167:6833/174302302] up:boot 2024-07-26T13:36:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:12 smithi167 ceph-mon[37140]: fsmap 1 up:standby 2024-07-26T13:36:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:12 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-26T13:36:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:12 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:12 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:12 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:12 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-26T13:36:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:12 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-26T13:36:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:12 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:12 smithi167 ceph-mon[37140]: Deploying daemon mds.k on smithi138 2024-07-26T13:36:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:12 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/3680073469' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:36:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:12 smithi167 ceph-mon[37140]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:13.940 INFO:journalctl@ceph.mds.k.smithi138.stdout:Jul 26 13:36:13 smithi138 systemd[1]: Starting Ceph mds.k for bf83e2e6-4b52-11ef-bcb7-c7b262605968... 2024-07-26T13:36:14.303 INFO:journalctl@ceph.mds.k.smithi138.stdout:Jul 26 13:36:14 smithi138 podman[67556]: 2024-07-26 13:36:13.929726589 +0000 UTC m=+0.017229895 image pull ae51f432d4fa82622bc253ca9e2da7666c617e096e99d43d0a0e4fdaca8865f8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2 2024-07-26T13:36:14.303 INFO:journalctl@ceph.mds.k.smithi138.stdout:Jul 26 13:36:14 smithi138 podman[67556]: 2024-07-26 13:36:14.042260519 +0000 UTC m=+0.129763854 container create e713efffd2cf13569c98f610a8f3982a2d55f538afd0862df38f90ba95aab678 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-k, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-07-26T13:36:14.304 INFO:journalctl@ceph.mds.k.smithi138.stdout:Jul 26 13:36:14 smithi138 podman[67556]: 2024-07-26 13:36:14.303342926 +0000 UTC m=+0.390846219 container init e713efffd2cf13569c98f610a8f3982a2d55f538afd0862df38f90ba95aab678 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-k, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-07-26T13:36:14.690 INFO:journalctl@ceph.mds.k.smithi138.stdout:Jul 26 13:36:14 smithi138 podman[67556]: 2024-07-26 13:36:14.306089689 +0000 UTC m=+0.393592983 container start e713efffd2cf13569c98f610a8f3982a2d55f538afd0862df38f90ba95aab678 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-k, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-26T13:36:14.690 INFO:journalctl@ceph.mds.k.smithi138.stdout:Jul 26 13:36:14 smithi138 ceph-mds[67575]: -- 172.21.15.138:0/70054222 <== mon.2 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55acee547860 con 0x55acf26e9a80 2024-07-26T13:36:14.691 INFO:journalctl@ceph.mds.k.smithi138.stdout:Jul 26 13:36:14 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-k[67571]: starting mds.k at 2024-07-26T13:36:14.691 INFO:journalctl@ceph.mds.k.smithi138.stdout:Jul 26 13:36:14 smithi138 bash[67556]: e713efffd2cf13569c98f610a8f3982a2d55f538afd0862df38f90ba95aab678 2024-07-26T13:36:14.691 INFO:journalctl@ceph.mds.k.smithi138.stdout:Jul 26 13:36:14 smithi138 systemd[1]: Started Ceph mds.k for bf83e2e6-4b52-11ef-bcb7-c7b262605968. 2024-07-26T13:36:14.964 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:36:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:15 smithi100 ceph-mon[32599]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:15 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:15 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:15 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:15 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-26T13:36:15.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:15 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-26T13:36:15.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:15 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:15.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:15 smithi100 ceph-mon[32599]: Deploying daemon mds.j on smithi100 2024-07-26T13:36:15.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:15 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.138:6832/2139594730,v1:172.21.15.138:6833/2139594730] up:boot 2024-07-26T13:36:15.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:15 smithi100 ceph-mon[32599]: fsmap 2 up:standby 2024-07-26T13:36:15.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:15 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-07-26T13:36:15.716 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:15 smithi138 ceph-mon[36954]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:15.716 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:15 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:15.716 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:15 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:15.716 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:15 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:15.716 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:15 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-26T13:36:15.717 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:15 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-26T13:36:15.717 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:15 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:15.717 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:15 smithi138 ceph-mon[36954]: Deploying daemon mds.j on smithi100 2024-07-26T13:36:15.717 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:15 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.138:6832/2139594730,v1:172.21.15.138:6833/2139594730] up:boot 2024-07-26T13:36:15.717 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:15 smithi138 ceph-mon[36954]: fsmap 2 up:standby 2024-07-26T13:36:15.717 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:15 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-07-26T13:36:15.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:15 smithi167 ceph-mon[37140]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:15 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:15 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:15 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:15 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-26T13:36:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:15 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-26T13:36:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:15 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:15 smithi167 ceph-mon[37140]: Deploying daemon mds.j on smithi100 2024-07-26T13:36:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:15 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.138:6832/2139594730,v1:172.21.15.138:6833/2139594730] up:boot 2024-07-26T13:36:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:15 smithi167 ceph-mon[37140]: fsmap 2 up:standby 2024-07-26T13:36:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:15 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-07-26T13:36:16.166 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:36:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:17 smithi100 ceph-mon[32599]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:17.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:17 smithi167 ceph-mon[37140]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:17.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:17 smithi138 ceph-mon[36954]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:18.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.988+0000 7f7b4cdc2640 1 -- 172.21.15.100:0/1091816596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b48102490 msgr2=0x7f7b481028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:18.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.988+0000 7f7b4cdc2640 1 --2- 172.21.15.100:0/1091816596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b48102490 0x7f7b481028a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7b38009900 tx=0x7f7b3802a410 comp rx=0 tx=0).stop 2024-07-26T13:36:18.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.988+0000 7f7b4cdc2640 1 -- 172.21.15.100:0/1091816596 shutdown_connections 2024-07-26T13:36:18.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.988+0000 7f7b4cdc2640 1 --2- 172.21.15.100:0/1091816596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b48108720 0x7f7b4810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:18.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.988+0000 7f7b4cdc2640 1 --2- 172.21.15.100:0/1091816596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b48107dd0 0x7f7b481081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:18.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.988+0000 7f7b4cdc2640 1 --2- 172.21.15.100:0/1091816596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b48102490 0x7f7b481028a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:18.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.988+0000 7f7b4cdc2640 1 -- 172.21.15.100:0/1091816596 >> 172.21.15.100:0/1091816596 conn(0x7f7b48079160 msgr2=0x7f7b48079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:36:18.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.988+0000 7f7b4cdc2640 1 -- 172.21.15.100:0/1091816596 shutdown_connections 2024-07-26T13:36:18.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.988+0000 7f7b4cdc2640 1 -- 172.21.15.100:0/1091816596 wait complete. 2024-07-26T13:36:18.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.989+0000 7f7b4cdc2640 1 Processor -- start 2024-07-26T13:36:18.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.989+0000 7f7b4cdc2640 1 -- start start 2024-07-26T13:36:18.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.990+0000 7f7b4cdc2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b48102490 0x7f7b4819ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:18.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.990+0000 7f7b4cdc2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b48107dd0 0x7f7b4819b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:18.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.990+0000 7f7b4cdc2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b48108720 0x7f7b4819b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:18.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.990+0000 7f7b4cdc2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7b48112400 con 0x7f7b48102490 2024-07-26T13:36:18.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.990+0000 7f7b4cdc2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7b48112280 con 0x7f7b48108720 2024-07-26T13:36:18.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.990+0000 7f7b4cdc2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7b48112580 con 0x7f7b48107dd0 2024-07-26T13:36:18.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.990+0000 7f7b4659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b48102490 0x7f7b4819ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:18.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.990+0000 7f7b4659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b48102490 0x7f7b4819ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47396/0 (socket says 172.21.15.100:47396) 2024-07-26T13:36:18.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.990+0000 7f7b4659c640 1 -- 172.21.15.100:0/2716207098 learned_addr learned my addr 172.21.15.100:0/2716207098 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:36:18.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.990+0000 7f7b3dd9b640 1 --2- 172.21.15.100:0/2716207098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b48107dd0 0x7f7b4819b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:18.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.990+0000 7f7b46d9d640 1 --2- 172.21.15.100:0/2716207098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b48108720 0x7f7b4819b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:18.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.990+0000 7f7b4659c640 1 -- 172.21.15.100:0/2716207098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b48107dd0 msgr2=0x7f7b4819b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:18.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.990+0000 7f7b4659c640 1 --2- 172.21.15.100:0/2716207098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b48107dd0 0x7f7b4819b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:18.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.990+0000 7f7b4659c640 1 -- 172.21.15.100:0/2716207098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b48108720 msgr2=0x7f7b4819b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:18.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.990+0000 7f7b4659c640 1 --2- 172.21.15.100:0/2716207098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b48108720 0x7f7b4819b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:18.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.990+0000 7f7b4659c640 1 -- 172.21.15.100:0/2716207098 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b481a5fb0 con 0x7f7b48102490 2024-07-26T13:36:18.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.990+0000 7f7b3dd9b640 1 --2- 172.21.15.100:0/2716207098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b48107dd0 0x7f7b4819b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:36:18.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.990+0000 7f7b4659c640 1 --2- 172.21.15.100:0/2716207098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b48102490 0x7f7b4819ad30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7b38006010 tx=0x7f7b380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:36:18.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.990+0000 7f7b3ffff640 1 -- 172.21.15.100:0/2716207098 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3803f030 con 0x7f7b48102490 2024-07-26T13:36:18.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.990+0000 7f7b4cdc2640 1 -- 172.21.15.100:0/2716207098 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b481a6240 con 0x7f7b48102490 2024-07-26T13:36:18.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.990+0000 7f7b3ffff640 1 -- 172.21.15.100:0/2716207098 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b38003800 con 0x7f7b48102490 2024-07-26T13:36:18.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.990+0000 7f7b3ffff640 1 -- 172.21.15.100:0/2716207098 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b38035070 con 0x7f7b48102490 2024-07-26T13:36:18.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.991+0000 7f7b4cdc2640 1 -- 172.21.15.100:0/2716207098 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b481a67a0 con 0x7f7b48102490 2024-07-26T13:36:18.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.991+0000 7f7b4cdc2640 1 -- 172.21.15.100:0/2716207098 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b481028a0 con 0x7f7b48102490 2024-07-26T13:36:18.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.993+0000 7f7b3ffff640 1 -- 172.21.15.100:0/2716207098 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f7b38003cb0 con 0x7f7b48102490 2024-07-26T13:36:18.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.994+0000 7f7b3ffff640 1 --2- 172.21.15.100:0/2716207098 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f7b340b3c20 0x7f7b340b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:18.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.994+0000 7f7b3dd9b640 1 --2- 172.21.15.100:0/2716207098 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f7b340b3c20 0x7f7b340b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:18.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.994+0000 7f7b3ffff640 1 -- 172.21.15.100:0/2716207098 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f7b38104520 con 0x7f7b48102490 2024-07-26T13:36:18.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.994+0000 7f7b3dd9b640 1 --2- 172.21.15.100:0/2716207098 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f7b340b3c20 0x7f7b340b6130 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7b48100680 tx=0x7f7b3000a5c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:36:18.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:17.997+0000 7f7b3ffff640 1 -- 172.21.15.100:0/2716207098 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f7b380cbf40 con 0x7f7b48102490 2024-07-26T13:36:18.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:18.215+0000 7f7b4cdc2640 1 -- 172.21.15.100:0/2716207098 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b4802e0c0 con 0x7f7b48102490 2024-07-26T13:36:18.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:18.215+0000 7f7b3ffff640 1 -- 172.21.15.100:0/2716207098 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2263 (secure 0 0 0) 0x7f7b380cb840 con 0x7f7b48102490 2024-07-26T13:36:18.216 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:36:18.217 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":3,"btime":"2024-07-26T13:36:14:494158+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-07-26T13:36:18.217 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 3 2024-07-26T13:36:18.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:18.217+0000 7f7b4cdc2640 1 -- 172.21.15.100:0/2716207098 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f7b340b3c20 msgr2=0x7f7b340b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:18.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:18.217+0000 7f7b4cdc2640 1 --2- 172.21.15.100:0/2716207098 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f7b340b3c20 0x7f7b340b6130 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7b48100680 tx=0x7f7b3000a5c0 comp rx=0 tx=0).stop 2024-07-26T13:36:18.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:18.217+0000 7f7b4cdc2640 1 -- 172.21.15.100:0/2716207098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b48102490 msgr2=0x7f7b4819ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:18.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:18.217+0000 7f7b4cdc2640 1 --2- 172.21.15.100:0/2716207098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b48102490 0x7f7b4819ad30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7b38006010 tx=0x7f7b380042f0 comp rx=0 tx=0).stop 2024-07-26T13:36:18.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:18.217+0000 7f7b4cdc2640 1 -- 172.21.15.100:0/2716207098 shutdown_connections 2024-07-26T13:36:18.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:18.217+0000 7f7b4cdc2640 1 --2- 172.21.15.100:0/2716207098 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f7b340b3c20 0x7f7b340b6130 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:18.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:18.217+0000 7f7b4cdc2640 1 --2- 172.21.15.100:0/2716207098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b48108720 0x7f7b4819b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:18.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:18.217+0000 7f7b4cdc2640 1 --2- 172.21.15.100:0/2716207098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b48107dd0 0x7f7b4819b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:18.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:18.217+0000 7f7b4cdc2640 1 --2- 172.21.15.100:0/2716207098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7b48102490 0x7f7b4819ad30 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:18.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:18.217+0000 7f7b4cdc2640 1 -- 172.21.15.100:0/2716207098 >> 172.21.15.100:0/2716207098 conn(0x7f7b48079160 msgr2=0x7f7b48100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:36:18.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:18.217+0000 7f7b4cdc2640 1 -- 172.21.15.100:0/2716207098 shutdown_connections 2024-07-26T13:36:18.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:18.217+0000 7f7b4cdc2640 1 -- 172.21.15.100:0/2716207098 wait complete. 2024-07-26T13:36:18.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:18 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2716207098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:36:18.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:18 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/2716207098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:36:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:18 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/2716207098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:36:19.151 INFO:journalctl@ceph.mds.j.smithi100.stdout:Jul 26 13:36:19 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-j[78036]: starting mds.j at 2024-07-26T13:36:19.151 INFO:journalctl@ceph.mds.j.smithi100.stdout:Jul 26 13:36:19 smithi100 bash[78009]: 086b81ab71160f07399ef06af09adcb59cd4c79ffbf5619e0e7c9e194cb1a651 2024-07-26T13:36:19.464 INFO:journalctl@ceph.mds.j.smithi100.stdout:Jul 26 13:36:19 smithi100 systemd[1]: Started Ceph mds.j for bf83e2e6-4b52-11ef-bcb7-c7b262605968. 2024-07-26T13:36:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:19 smithi100 ceph-mon[32599]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:19.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-26T13:36:19.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-26T13:36:19.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:19.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:19 smithi167 ceph-mon[37140]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:19.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:19.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-26T13:36:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-26T13:36:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:19 smithi138 ceph-mon[36954]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-26T13:36:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-26T13:36:19.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:20 smithi100 ceph-mon[32599]: Deploying daemon mds.d on smithi100 2024-07-26T13:36:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:20 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.100:6834/2380716258,v1:172.21.15.100:6835/2380716258] up:boot 2024-07-26T13:36:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:20 smithi100 ceph-mon[32599]: fsmap 3 up:standby 2024-07-26T13:36:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:20 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-07-26T13:36:20.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:20 smithi167 ceph-mon[37140]: Deploying daemon mds.d on smithi100 2024-07-26T13:36:20.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:20 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.100:6834/2380716258,v1:172.21.15.100:6835/2380716258] up:boot 2024-07-26T13:36:20.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:20 smithi167 ceph-mon[37140]: fsmap 3 up:standby 2024-07-26T13:36:20.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:20 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-07-26T13:36:20.928 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:36:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:20 smithi138 ceph-mon[36954]: Deploying daemon mds.d on smithi100 2024-07-26T13:36:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:20 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.100:6834/2380716258,v1:172.21.15.100:6835/2380716258] up:boot 2024-07-26T13:36:20.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:20 smithi138 ceph-mon[36954]: fsmap 3 up:standby 2024-07-26T13:36:20.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:20 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-07-26T13:36:21.244 INFO:journalctl@ceph.mds.d.smithi100.stdout:Jul 26 13:36:21 smithi100 systemd[1]: Starting Ceph mds.d for bf83e2e6-4b52-11ef-bcb7-c7b262605968... 2024-07-26T13:36:21.583 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:21 smithi100 ceph-mon[32599]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:21.584 INFO:journalctl@ceph.mds.d.smithi100.stdout:Jul 26 13:36:21 smithi100 podman[78352]: 2024-07-26 13:36:21.202317836 +0000 UTC m=+0.020266084 image pull ae51f432d4fa82622bc253ca9e2da7666c617e096e99d43d0a0e4fdaca8865f8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2 2024-07-26T13:36:21.584 INFO:journalctl@ceph.mds.d.smithi100.stdout:Jul 26 13:36:21 smithi100 podman[78352]: 2024-07-26 13:36:21.310838137 +0000 UTC m=+0.128786429 container create 2af282dd51697cede212cd730c892e5e8f6d83af9fd7e5e8a26b10d991af8ae9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-d, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, org.label-schema.build-date=20240724, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-26T13:36:21.584 INFO:journalctl@ceph.mds.d.smithi100.stdout:Jul 26 13:36:21 smithi100 podman[78352]: 2024-07-26 13:36:21.547196758 +0000 UTC m=+0.365145025 container init 2af282dd51697cede212cd730c892e5e8f6d83af9fd7e5e8a26b10d991af8ae9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-d, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-07-26T13:36:21.584 INFO:journalctl@ceph.mds.d.smithi100.stdout:Jul 26 13:36:21 smithi100 podman[78352]: 2024-07-26 13:36:21.550179841 +0000 UTC m=+0.368128091 container start 2af282dd51697cede212cd730c892e5e8f6d83af9fd7e5e8a26b10d991af8ae9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-26T13:36:21.584 INFO:journalctl@ceph.mds.d.smithi100.stdout:Jul 26 13:36:21 smithi100 ceph-mds[78398]: -- 172.21.15.100:0/1776810020 <== mon.0 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55687f72fc20 con 0x5568838d1a80 2024-07-26T13:36:21.647 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:36:21.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:21 smithi167 ceph-mon[37140]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:21.890 INFO:journalctl@ceph.mds.d.smithi100.stdout:Jul 26 13:36:21 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-d[78380]: starting mds.d at 2024-07-26T13:36:21.890 INFO:journalctl@ceph.mds.d.smithi100.stdout:Jul 26 13:36:21 smithi100 bash[78352]: 2af282dd51697cede212cd730c892e5e8f6d83af9fd7e5e8a26b10d991af8ae9 2024-07-26T13:36:21.890 INFO:journalctl@ceph.mds.d.smithi100.stdout:Jul 26 13:36:21 smithi100 systemd[1]: Started Ceph mds.d for bf83e2e6-4b52-11ef-bcb7-c7b262605968. 2024-07-26T13:36:21.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:21 smithi138 ceph-mon[36954]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-26T13:36:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-26T13:36:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:22 smithi138 ceph-mon[36954]: Deploying daemon mds.e on smithi138 2024-07-26T13:36:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:22 smithi138 ceph-mon[36954]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:22 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.100:6836/3079467433,v1:172.21.15.100:6837/3079467433] up:boot 2024-07-26T13:36:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:22 smithi138 ceph-mon[36954]: fsmap 4 up:standby 2024-07-26T13:36:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-07-26T13:36:22.982 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:22.982 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:22.982 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:22.982 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-26T13:36:22.982 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-26T13:36:22.982 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:22.982 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:22 smithi100 ceph-mon[32599]: Deploying daemon mds.e on smithi138 2024-07-26T13:36:22.983 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:22 smithi100 ceph-mon[32599]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:22.983 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:22 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.100:6836/3079467433,v1:172.21.15.100:6837/3079467433] up:boot 2024-07-26T13:36:22.983 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:22 smithi100 ceph-mon[32599]: fsmap 4 up:standby 2024-07-26T13:36:22.983 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-07-26T13:36:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-26T13:36:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-26T13:36:23.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:23.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:22 smithi167 ceph-mon[37140]: Deploying daemon mds.e on smithi138 2024-07-26T13:36:23.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:22 smithi167 ceph-mon[37140]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:23.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:22 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.100:6836/3079467433,v1:172.21.15.100:6837/3079467433] up:boot 2024-07-26T13:36:23.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:22 smithi167 ceph-mon[37140]: fsmap 4 up:standby 2024-07-26T13:36:23.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-07-26T13:36:23.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.380+0000 7f037a09f640 1 -- 172.21.15.100:0/290469039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0374102540 msgr2=0x7f0374102950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:23.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.380+0000 7f037a09f640 1 --2- 172.21.15.100:0/290469039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0374102540 0x7f0374102950 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f035c009900 tx=0x7f035c02a410 comp rx=0 tx=0).stop 2024-07-26T13:36:23.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.380+0000 7f037a09f640 1 -- 172.21.15.100:0/290469039 shutdown_connections 2024-07-26T13:36:23.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.380+0000 7f037a09f640 1 --2- 172.21.15.100:0/290469039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03741087d0 0x7f037410f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:23.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.380+0000 7f037a09f640 1 --2- 172.21.15.100:0/290469039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0374107e80 0x7f0374108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:23.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.380+0000 7f037a09f640 1 --2- 172.21.15.100:0/290469039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0374102540 0x7f0374102950 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:23.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.380+0000 7f037a09f640 1 -- 172.21.15.100:0/290469039 >> 172.21.15.100:0/290469039 conn(0x7f0374079100 msgr2=0x7f0374079510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:36:23.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.380+0000 7f037a09f640 1 -- 172.21.15.100:0/290469039 shutdown_connections 2024-07-26T13:36:23.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.380+0000 7f037a09f640 1 -- 172.21.15.100:0/290469039 wait complete. 2024-07-26T13:36:23.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.381+0000 7f037a09f640 1 Processor -- start 2024-07-26T13:36:23.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.381+0000 7f037a09f640 1 -- start start 2024-07-26T13:36:23.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.381+0000 7f037a09f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0374102540 0x7f03740ff0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:23.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.381+0000 7f037a09f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0374107e80 0x7f03740ff600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:23.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.381+0000 7f037a09f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03741087d0 0x7f03740ffb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:23.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.382+0000 7f037a09f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f03741124f0 con 0x7f0374107e80 2024-07-26T13:36:23.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.382+0000 7f037a09f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0374112370 con 0x7f03741087d0 2024-07-26T13:36:23.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.382+0000 7f037a09f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0374112670 con 0x7f0374102540 2024-07-26T13:36:23.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.382+0000 7f0372ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0374107e80 0x7f03740ff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:23.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.382+0000 7f0372ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0374107e80 0x7f03740ff600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47468/0 (socket says 172.21.15.100:47468) 2024-07-26T13:36:23.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.382+0000 7f0372ffd640 1 -- 172.21.15.100:0/3295229283 learned_addr learned my addr 172.21.15.100:0/3295229283 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:36:23.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.382+0000 7f03737fe640 1 --2- 172.21.15.100:0/3295229283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0374102540 0x7f03740ff0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:23.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.382+0000 7f0373fff640 1 --2- 172.21.15.100:0/3295229283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03741087d0 0x7f03740ffb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:23.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.382+0000 7f0372ffd640 1 -- 172.21.15.100:0/3295229283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0374102540 msgr2=0x7f03740ff0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:23.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.382+0000 7f0372ffd640 1 --2- 172.21.15.100:0/3295229283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0374102540 0x7f03740ff0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:23.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.382+0000 7f0372ffd640 1 -- 172.21.15.100:0/3295229283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03741087d0 msgr2=0x7f03740ffb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:23.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.382+0000 7f0372ffd640 1 --2- 172.21.15.100:0/3295229283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03741087d0 0x7f03740ffb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:23.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.382+0000 7f0372ffd640 1 -- 172.21.15.100:0/3295229283 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03741a8310 con 0x7f0374107e80 2024-07-26T13:36:23.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.382+0000 7f0372ffd640 1 --2- 172.21.15.100:0/3295229283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0374107e80 0x7f03740ff600 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0364006fd0 tx=0x7f036400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:36:23.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.382+0000 7f0370ff9640 1 -- 172.21.15.100:0/3295229283 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0364011830 con 0x7f0374107e80 2024-07-26T13:36:23.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.382+0000 7f037a09f640 1 -- 172.21.15.100:0/3295229283 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03741a8600 con 0x7f0374107e80 2024-07-26T13:36:23.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.382+0000 7f0370ff9640 1 -- 172.21.15.100:0/3295229283 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0364016070 con 0x7f0374107e80 2024-07-26T13:36:23.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.382+0000 7f0370ff9640 1 -- 172.21.15.100:0/3295229283 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0364010e00 con 0x7f0374107e80 2024-07-26T13:36:23.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.382+0000 7f037a09f640 1 -- 172.21.15.100:0/3295229283 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03741a8b90 con 0x7f0374107e80 2024-07-26T13:36:23.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.383+0000 7f037a09f640 1 -- 172.21.15.100:0/3295229283 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0374102950 con 0x7f0374107e80 2024-07-26T13:36:23.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.385+0000 7f0370ff9640 1 -- 172.21.15.100:0/3295229283 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f036401b460 con 0x7f0374107e80 2024-07-26T13:36:23.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.385+0000 7f0370ff9640 1 --2- 172.21.15.100:0/3295229283 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f03600b3c20 0x7f03600b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:23.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.386+0000 7f03737fe640 1 --2- 172.21.15.100:0/3295229283 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f03600b3c20 0x7f03600b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:23.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.386+0000 7f0370ff9640 1 -- 172.21.15.100:0/3295229283 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f03640e34c0 con 0x7f0374107e80 2024-07-26T13:36:23.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.386+0000 7f03737fe640 1 --2- 172.21.15.100:0/3295229283 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f03600b3c20 0x7f03600b6130 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f035c000c00 tx=0x7f035c031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:36:23.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.389+0000 7f0370ff9640 1 -- 172.21.15.100:0/3295229283 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f03640aaee0 con 0x7f0374107e80 2024-07-26T13:36:23.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.597+0000 7f037a09f640 1 -- 172.21.15.100:0/3295229283 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f037410b5c0 con 0x7f0374107e80 2024-07-26T13:36:23.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.598+0000 7f0370ff9640 1 -- 172.21.15.100:0/3295229283 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3919 (secure 0 0 0) 0x7f03640b31b0 con 0x7f0374107e80 2024-07-26T13:36:23.599 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:36:23.599 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":5,"btime":"2024-07-26T13:36:22:490458+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-07-26T13:36:23.599 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 5 2024-07-26T13:36:23.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.600+0000 7f037a09f640 1 -- 172.21.15.100:0/3295229283 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f03600b3c20 msgr2=0x7f03600b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:23.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.600+0000 7f037a09f640 1 --2- 172.21.15.100:0/3295229283 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f03600b3c20 0x7f03600b6130 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f035c000c00 tx=0x7f035c031400 comp rx=0 tx=0).stop 2024-07-26T13:36:23.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.600+0000 7f037a09f640 1 -- 172.21.15.100:0/3295229283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0374107e80 msgr2=0x7f03740ff600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:23.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.600+0000 7f037a09f640 1 --2- 172.21.15.100:0/3295229283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0374107e80 0x7f03740ff600 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0364006fd0 tx=0x7f036400b8e0 comp rx=0 tx=0).stop 2024-07-26T13:36:23.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.600+0000 7f037a09f640 1 -- 172.21.15.100:0/3295229283 shutdown_connections 2024-07-26T13:36:23.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.600+0000 7f037a09f640 1 --2- 172.21.15.100:0/3295229283 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f03600b3c20 0x7f03600b6130 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:23.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.600+0000 7f037a09f640 1 --2- 172.21.15.100:0/3295229283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03741087d0 0x7f03740ffb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:23.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.600+0000 7f037a09f640 1 --2- 172.21.15.100:0/3295229283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0374107e80 0x7f03740ff600 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:23.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.600+0000 7f037a09f640 1 --2- 172.21.15.100:0/3295229283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0374102540 0x7f03740ff0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:23.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.600+0000 7f037a09f640 1 -- 172.21.15.100:0/3295229283 >> 172.21.15.100:0/3295229283 conn(0x7f0374079100 msgr2=0x7f03741015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:36:23.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.600+0000 7f037a09f640 1 -- 172.21.15.100:0/3295229283 shutdown_connections 2024-07-26T13:36:23.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:23.600+0000 7f037a09f640 1 -- 172.21.15.100:0/3295229283 wait complete. 2024-07-26T13:36:23.856 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:23 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/3295229283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:36:23.913 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:23 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/3295229283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:36:24.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:23 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/3295229283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:36:24.440 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:36:24 smithi138 systemd[1]: Started Ceph mds.e for bf83e2e6-4b52-11ef-bcb7-c7b262605968. 2024-07-26T13:36:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-26T13:36:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-26T13:36:25.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:25.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:25 smithi138 ceph-mon[36954]: Deploying daemon mds.f on smithi167 2024-07-26T13:36:25.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:25 smithi138 ceph-mon[36954]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:25.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:25 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] up:boot 2024-07-26T13:36:25.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:25 smithi138 ceph-mon[36954]: fsmap 5 up:standby 2024-07-26T13:36:25.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-26T13:36:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-26T13:36:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-26T13:36:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:25.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:25 smithi100 ceph-mon[32599]: Deploying daemon mds.f on smithi167 2024-07-26T13:36:25.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:25 smithi100 ceph-mon[32599]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:25.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:25 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] up:boot 2024-07-26T13:36:25.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:25 smithi100 ceph-mon[32599]: fsmap 5 up:standby 2024-07-26T13:36:25.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-26T13:36:25.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:25.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:25.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:25.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-26T13:36:25.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-26T13:36:25.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:25.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:25 smithi167 ceph-mon[37140]: Deploying daemon mds.f on smithi167 2024-07-26T13:36:25.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:25 smithi167 ceph-mon[37140]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:25.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:25 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] up:boot 2024-07-26T13:36:25.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:25 smithi167 ceph-mon[37140]: fsmap 5 up:standby 2024-07-26T13:36:25.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-26T13:36:26.217 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:36:26.461 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jul 26 13:36:26 smithi167 systemd[1]: Starting Ceph mds.f for bf83e2e6-4b52-11ef-bcb7-c7b262605968... 2024-07-26T13:36:26.485 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:36:26.713 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jul 26 13:36:26 smithi167 podman[68716]: 2024-07-26 13:36:26.363290924 +0000 UTC m=+0.017746027 image pull ae51f432d4fa82622bc253ca9e2da7666c617e096e99d43d0a0e4fdaca8865f8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2 2024-07-26T13:36:26.714 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jul 26 13:36:26 smithi167 podman[68716]: 2024-07-26 13:36:26.476538547 +0000 UTC m=+0.130993646 container create 3b039426e42c626f2af54a20c76f4f98d119ba4b822b907e74cac2b9d859b6f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-f, org.label-schema.build-date=20240724, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-07-26T13:36:26.714 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jul 26 13:36:26 smithi167 podman[68716]: 2024-07-26 13:36:26.713404333 +0000 UTC m=+0.367859429 container init 3b039426e42c626f2af54a20c76f4f98d119ba4b822b907e74cac2b9d859b6f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-f, org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-07-26T13:36:27.022 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jul 26 13:36:26 smithi167 podman[68716]: 2024-07-26 13:36:26.715939372 +0000 UTC m=+0.370394468 container start 3b039426e42c626f2af54a20c76f4f98d119ba4b822b907e74cac2b9d859b6f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb) 2024-07-26T13:36:27.022 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jul 26 13:36:26 smithi167 ceph-mds[68736]: -- 172.21.15.167:0/130026178 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5608f1087860 con 0x5608f5229180 2024-07-26T13:36:27.022 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jul 26 13:36:26 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-f[68732]: starting mds.f at 2024-07-26T13:36:27.023 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jul 26 13:36:26 smithi167 bash[68716]: 3b039426e42c626f2af54a20c76f4f98d119ba4b822b907e74cac2b9d859b6f0 2024-07-26T13:36:27.023 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jul 26 13:36:26 smithi167 systemd[1]: Started Ceph mds.f for bf83e2e6-4b52-11ef-bcb7-c7b262605968. 2024-07-26T13:36:27.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:27 smithi167 ceph-mon[37140]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:27.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:27.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:27.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:27.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-26T13:36:27.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-26T13:36:27.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:27 smithi138 ceph-mon[36954]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-26T13:36:27.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-26T13:36:27.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:27 smithi100 ceph-mon[32599]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:27.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:27.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:27.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-26T13:36:27.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-26T13:36:27.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:28.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.290+0000 7f03ec339640 1 -- 172.21.15.100:0/992101021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03e4102490 msgr2=0x7f03e41028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:28.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.290+0000 7f03ec339640 1 --2- 172.21.15.100:0/992101021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03e4102490 0x7f03e41028a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f03d4009ca0 tx=0x7f03d402a8c0 comp rx=0 tx=0).stop 2024-07-26T13:36:28.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.290+0000 7f03ec339640 1 -- 172.21.15.100:0/992101021 shutdown_connections 2024-07-26T13:36:28.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.290+0000 7f03ec339640 1 --2- 172.21.15.100:0/992101021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03e4108720 0x7f03e410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:28.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.290+0000 7f03ec339640 1 --2- 172.21.15.100:0/992101021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03e4107dd0 0x7f03e41081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:28.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.290+0000 7f03ec339640 1 --2- 172.21.15.100:0/992101021 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03e4102490 0x7f03e41028a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:28.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.290+0000 7f03ec339640 1 -- 172.21.15.100:0/992101021 >> 172.21.15.100:0/992101021 conn(0x7f03e4079160 msgr2=0x7f03e4079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:36:28.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.290+0000 7f03ec339640 1 -- 172.21.15.100:0/992101021 shutdown_connections 2024-07-26T13:36:28.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.290+0000 7f03ec339640 1 -- 172.21.15.100:0/992101021 wait complete. 2024-07-26T13:36:28.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.291+0000 7f03ec339640 1 Processor -- start 2024-07-26T13:36:28.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.291+0000 7f03ec339640 1 -- start start 2024-07-26T13:36:28.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.291+0000 7f03ec339640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03e4102490 0x7f03e4198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:28.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.291+0000 7f03ec339640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03e4107dd0 0x7f03e41993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:28.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.291+0000 7f03ec339640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03e4108720 0x7f03e419e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:28.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.291+0000 7f03ec339640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f03e4112380 con 0x7f03e4102490 2024-07-26T13:36:28.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.291+0000 7f03ec339640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f03e4112200 con 0x7f03e4108720 2024-07-26T13:36:28.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.291+0000 7f03ec339640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f03e4112500 con 0x7f03e4107dd0 2024-07-26T13:36:28.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.291+0000 7f03ea0d5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03e4102490 0x7f03e4198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:28.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.291+0000 7f03ea0d5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03e4102490 0x7f03e4198e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52064/0 (socket says 172.21.15.100:52064) 2024-07-26T13:36:28.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.291+0000 7f03ea0d5640 1 -- 172.21.15.100:0/662495873 learned_addr learned my addr 172.21.15.100:0/662495873 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:36:28.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.292+0000 7f03e98d4640 1 --2- 172.21.15.100:0/662495873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03e4107dd0 0x7f03e41993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:28.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.292+0000 7f03ea0d5640 1 -- 172.21.15.100:0/662495873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03e4107dd0 msgr2=0x7f03e41993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:28.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.292+0000 7f03ea0d5640 1 --2- 172.21.15.100:0/662495873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03e4107dd0 0x7f03e41993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:28.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.292+0000 7f03ea0d5640 1 -- 172.21.15.100:0/662495873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03e4108720 msgr2=0x7f03e419e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:28.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.292+0000 7f03ea8d6640 1 --2- 172.21.15.100:0/662495873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03e4108720 0x7f03e419e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:28.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.292+0000 7f03ea0d5640 1 --2- 172.21.15.100:0/662495873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03e4108720 0x7f03e419e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:28.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.292+0000 7f03ea0d5640 1 -- 172.21.15.100:0/662495873 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e419ece0 con 0x7f03e4102490 2024-07-26T13:36:28.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.292+0000 7f03ea0d5640 1 --2- 172.21.15.100:0/662495873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03e4102490 0x7f03e4198e30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f03d4006010 tx=0x7f03d40319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:36:28.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.292+0000 7f03e37fe640 1 -- 172.21.15.100:0/662495873 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d4003bf0 con 0x7f03e4102490 2024-07-26T13:36:28.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.292+0000 7f03e37fe640 1 -- 172.21.15.100:0/662495873 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03d4003d90 con 0x7f03e4102490 2024-07-26T13:36:28.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.292+0000 7f03ec339640 1 -- 172.21.15.100:0/662495873 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03e419ef70 con 0x7f03e4102490 2024-07-26T13:36:28.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.292+0000 7f03e37fe640 1 -- 172.21.15.100:0/662495873 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d402fc20 con 0x7f03e4102490 2024-07-26T13:36:28.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.292+0000 7f03ec339640 1 -- 172.21.15.100:0/662495873 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03e41a57f0 con 0x7f03e4102490 2024-07-26T13:36:28.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.293+0000 7f03ec339640 1 -- 172.21.15.100:0/662495873 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03e41028a0 con 0x7f03e4102490 2024-07-26T13:36:28.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.295+0000 7f03e37fe640 1 -- 172.21.15.100:0/662495873 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f03d40046a0 con 0x7f03e4102490 2024-07-26T13:36:28.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.295+0000 7f03e37fe640 1 --2- 172.21.15.100:0/662495873 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f03cc0b3cd0 0x7f03cc0b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:28.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.296+0000 7f03e98d4640 1 --2- 172.21.15.100:0/662495873 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f03cc0b3cd0 0x7f03cc0b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:28.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.296+0000 7f03e37fe640 1 -- 172.21.15.100:0/662495873 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f03d4104fe0 con 0x7f03e4102490 2024-07-26T13:36:28.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.296+0000 7f03e98d4640 1 --2- 172.21.15.100:0/662495873 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f03cc0b3cd0 0x7f03cc0b61e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f03d8005fd0 tx=0x7f03d8005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:36:28.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.299+0000 7f03e37fe640 1 -- 172.21.15.100:0/662495873 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f03d40cca00 con 0x7f03e4102490 2024-07-26T13:36:28.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.517+0000 7f03ec339640 1 -- 172.21.15.100:0/662495873 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03e4199f60 con 0x7f03e4102490 2024-07-26T13:36:28.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.517+0000 7f03e37fe640 1 -- 172.21.15.100:0/662495873 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5572 (secure 0 0 0) 0x7f03d40cc300 con 0x7f03e4102490 2024-07-26T13:36:28.518 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:36:28.519 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":7,"btime":"2024-07-26T13:36:27:402296+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24557,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/3290105300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3290105300},{"type":"v1","addr":"172.21.15.138:6835","nonce":3290105300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24586,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-07-26T13:36:28.519 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 7 2024-07-26T13:36:28.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.519+0000 7f03ec339640 1 -- 172.21.15.100:0/662495873 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f03cc0b3cd0 msgr2=0x7f03cc0b61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:28.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.519+0000 7f03ec339640 1 --2- 172.21.15.100:0/662495873 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f03cc0b3cd0 0x7f03cc0b61e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f03d8005fd0 tx=0x7f03d8005850 comp rx=0 tx=0).stop 2024-07-26T13:36:28.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.519+0000 7f03ec339640 1 -- 172.21.15.100:0/662495873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03e4102490 msgr2=0x7f03e4198e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:28.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.519+0000 7f03ec339640 1 --2- 172.21.15.100:0/662495873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03e4102490 0x7f03e4198e30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f03d4006010 tx=0x7f03d40319f0 comp rx=0 tx=0).stop 2024-07-26T13:36:28.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.519+0000 7f03ec339640 1 -- 172.21.15.100:0/662495873 shutdown_connections 2024-07-26T13:36:28.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.519+0000 7f03ec339640 1 --2- 172.21.15.100:0/662495873 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f03cc0b3cd0 0x7f03cc0b61e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:28.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.519+0000 7f03ec339640 1 --2- 172.21.15.100:0/662495873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03e4108720 0x7f03e419e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:28.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.519+0000 7f03ec339640 1 --2- 172.21.15.100:0/662495873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03e4107dd0 0x7f03e41993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:28.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.519+0000 7f03ec339640 1 --2- 172.21.15.100:0/662495873 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03e4102490 0x7f03e4198e30 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:28.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.519+0000 7f03ec339640 1 -- 172.21.15.100:0/662495873 >> 172.21.15.100:0/662495873 conn(0x7f03e4079160 msgr2=0x7f03e4100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:36:28.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.520+0000 7f03ec339640 1 -- 172.21.15.100:0/662495873 shutdown_connections 2024-07-26T13:36:28.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:28.520+0000 7f03ec339640 1 -- 172.21.15.100:0/662495873 wait complete. 2024-07-26T13:36:28.597 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:28 smithi100 ceph-mon[32599]: Deploying daemon mds.c on smithi167 2024-07-26T13:36:28.597 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:28 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.167:6834/641259145,v1:172.21.15.167:6835/641259145] up:boot 2024-07-26T13:36:28.597 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:28 smithi100 ceph-mon[32599]: fsmap 6 up:standby 2024-07-26T13:36:28.598 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-26T13:36:28.619 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:28 smithi167 ceph-mon[37140]: Deploying daemon mds.c on smithi167 2024-07-26T13:36:28.619 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:28 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.167:6834/641259145,v1:172.21.15.167:6835/641259145] up:boot 2024-07-26T13:36:28.620 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:28 smithi167 ceph-mon[37140]: fsmap 6 up:standby 2024-07-26T13:36:28.620 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-26T13:36:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:28 smithi138 ceph-mon[36954]: Deploying daemon mds.c on smithi167 2024-07-26T13:36:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:28 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.167:6834/641259145,v1:172.21.15.167:6835/641259145] up:boot 2024-07-26T13:36:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:28 smithi138 ceph-mon[36954]: fsmap 6 up:standby 2024-07-26T13:36:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-26T13:36:29.019 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:36:28 smithi167 systemd[1]: Starting Ceph mds.c for bf83e2e6-4b52-11ef-bcb7-c7b262605968... 2024-07-26T13:36:29.019 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:36:29 smithi167 podman[69015]: 2024-07-26 13:36:28.92048083 +0000 UTC m=+0.017938648 image pull ae51f432d4fa82622bc253ca9e2da7666c617e096e99d43d0a0e4fdaca8865f8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2 2024-07-26T13:36:29.270 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:36:29 smithi167 podman[69015]: 2024-07-26 13:36:29.025810132 +0000 UTC m=+0.123267978 container create 6876119001b6632c2578cacbbb590f29bc2c03a736c801fe755d40bdbf818d0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-26T13:36:29.271 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:36:29 smithi167 podman[69015]: 2024-07-26 13:36:29.27046768 +0000 UTC m=+0.367925484 container init 6876119001b6632c2578cacbbb590f29bc2c03a736c801fe755d40bdbf818d0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-07-26T13:36:29.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:29 smithi167 ceph-mon[37140]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:29.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:29 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/662495873' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:36:29.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:29.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:29.524 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:36:29 smithi167 podman[69015]: 2024-07-26 13:36:29.273299884 +0000 UTC m=+0.370757713 container start 6876119001b6632c2578cacbbb590f29bc2c03a736c801fe755d40bdbf818d0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, io.buildah.version=1.36.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-26T13:36:29.524 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:36:29 smithi167 ceph-mds[69035]: -- 172.21.15.167:0/3365111571 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d78adb1c20 con 0x55d78ef53a80 2024-07-26T13:36:29.524 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:36:29 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: starting mds.c at 2024-07-26T13:36:29.524 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:36:29 smithi167 bash[69015]: 6876119001b6632c2578cacbbb590f29bc2c03a736c801fe755d40bdbf818d0e 2024-07-26T13:36:29.525 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:36:29 smithi167 systemd[1]: Started Ceph mds.c for bf83e2e6-4b52-11ef-bcb7-c7b262605968. 2024-07-26T13:36:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:29 smithi138 ceph-mon[36954]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:29 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/662495873' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:36:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:29 smithi100 ceph-mon[32599]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:29 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/662495873' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:36:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:30.430 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:30.430 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-26T13:36:30.431 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:30 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] up:boot 2024-07-26T13:36:30.431 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:30 smithi138 ceph-mon[36954]: fsmap 7 up:standby 2024-07-26T13:36:30.431 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-26T13:36:30.431 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-26T13:36:30.431 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:30.431 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:30 smithi138 ceph-mon[36954]: Deploying daemon mds.h on smithi138 2024-07-26T13:36:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-26T13:36:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:30 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] up:boot 2024-07-26T13:36:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:30 smithi100 ceph-mon[32599]: fsmap 7 up:standby 2024-07-26T13:36:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-26T13:36:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-26T13:36:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:30 smithi100 ceph-mon[32599]: Deploying daemon mds.h on smithi138 2024-07-26T13:36:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-26T13:36:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:30 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] up:boot 2024-07-26T13:36:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:30 smithi167 ceph-mon[37140]: fsmap 7 up:standby 2024-07-26T13:36:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-26T13:36:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-26T13:36:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:30 smithi167 ceph-mon[37140]: Deploying daemon mds.h on smithi138 2024-07-26T13:36:31.180 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:36:31.451 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:36:31.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:31 smithi100 ceph-mon[32599]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:31.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:31 smithi138 ceph-mon[36954]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:31.691 INFO:journalctl@ceph.mds.h.smithi138.stdout:Jul 26 13:36:31 smithi138 systemd[1]: Starting Ceph mds.h for bf83e2e6-4b52-11ef-bcb7-c7b262605968... 2024-07-26T13:36:31.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:31 smithi167 ceph-mon[37140]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:32.039 INFO:journalctl@ceph.mds.h.smithi138.stdout:Jul 26 13:36:31 smithi138 podman[68180]: 2024-07-26 13:36:31.603470164 +0000 UTC m=+0.017232755 image pull ae51f432d4fa82622bc253ca9e2da7666c617e096e99d43d0a0e4fdaca8865f8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2 2024-07-26T13:36:32.040 INFO:journalctl@ceph.mds.h.smithi138.stdout:Jul 26 13:36:31 smithi138 podman[68180]: 2024-07-26 13:36:31.719678235 +0000 UTC m=+0.133440828 container create 10a4495ac07b03af90f95cdb897b58ebd116801e88829ac21d4bb6996d457de4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-h, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240724, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-26T13:36:32.040 INFO:journalctl@ceph.mds.h.smithi138.stdout:Jul 26 13:36:32 smithi138 podman[68180]: 2024-07-26 13:36:32.039227151 +0000 UTC m=+0.452989741 container init 10a4495ac07b03af90f95cdb897b58ebd116801e88829ac21d4bb6996d457de4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-07-26T13:36:32.440 INFO:journalctl@ceph.mds.h.smithi138.stdout:Jul 26 13:36:32 smithi138 podman[68180]: 2024-07-26 13:36:32.04209568 +0000 UTC m=+0.455858270 container start 10a4495ac07b03af90f95cdb897b58ebd116801e88829ac21d4bb6996d457de4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-26T13:36:32.440 INFO:journalctl@ceph.mds.h.smithi138.stdout:Jul 26 13:36:32 smithi138 ceph-mds[68199]: -- 172.21.15.138:0/79094987 <== mon.2 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5632e4c8fc20 con 0x5632e8e31180 2024-07-26T13:36:32.440 INFO:journalctl@ceph.mds.h.smithi138.stdout:Jul 26 13:36:32 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-h[68195]: starting mds.h at 2024-07-26T13:36:32.441 INFO:journalctl@ceph.mds.h.smithi138.stdout:Jul 26 13:36:32 smithi138 bash[68180]: 10a4495ac07b03af90f95cdb897b58ebd116801e88829ac21d4bb6996d457de4 2024-07-26T13:36:32.441 INFO:journalctl@ceph.mds.h.smithi138.stdout:Jul 26 13:36:32 smithi138 systemd[1]: Started Ceph mds.h for bf83e2e6-4b52-11ef-bcb7-c7b262605968. 2024-07-26T13:36:33.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.156+0000 7f58c7492640 1 -- 172.21.15.100:0/2907478664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58c01087a0 msgr2=0x7f58c010efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:33.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.156+0000 7f58c7492640 1 --2- 172.21.15.100:0/2907478664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58c01087a0 0x7f58c010efd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f58b8009ca0 tx=0x7f58b802a8e0 comp rx=0 tx=0).stop 2024-07-26T13:36:33.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.156+0000 7f58c7492640 1 -- 172.21.15.100:0/2907478664 shutdown_connections 2024-07-26T13:36:33.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.156+0000 7f58c7492640 1 --2- 172.21.15.100:0/2907478664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58c01087a0 0x7f58c010efd0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:33.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.156+0000 7f58c7492640 1 --2- 172.21.15.100:0/2907478664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58c0107e50 0x7f58c0108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:33.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.156+0000 7f58c7492640 1 --2- 172.21.15.100:0/2907478664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58c0102510 0x7f58c0102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:33.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.156+0000 7f58c7492640 1 -- 172.21.15.100:0/2907478664 >> 172.21.15.100:0/2907478664 conn(0x7f58c00791a0 msgr2=0x7f58c00795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:36:33.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.156+0000 7f58c7492640 1 -- 172.21.15.100:0/2907478664 shutdown_connections 2024-07-26T13:36:33.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.156+0000 7f58c7492640 1 -- 172.21.15.100:0/2907478664 wait complete. 2024-07-26T13:36:33.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.157+0000 7f58c7492640 1 Processor -- start 2024-07-26T13:36:33.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.157+0000 7f58c7492640 1 -- start start 2024-07-26T13:36:33.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.157+0000 7f58c7492640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58c0102510 0x7f58c0199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:33.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.157+0000 7f58c7492640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58c0107e50 0x7f58c019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:33.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.157+0000 7f58c7492640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58c01087a0 0x7f58c019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:33.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.157+0000 7f58c7492640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f58c0113620 con 0x7f58c01087a0 2024-07-26T13:36:33.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.157+0000 7f58c7492640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f58c01134a0 con 0x7f58c0102510 2024-07-26T13:36:33.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.157+0000 7f58c7492640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f58c01137a0 con 0x7f58c0107e50 2024-07-26T13:36:33.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.157+0000 7f58c5a2f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58c01087a0 0x7f58c019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:33.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.157+0000 7f58c5a2f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58c01087a0 0x7f58c019f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52092/0 (socket says 172.21.15.100:52092) 2024-07-26T13:36:33.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.157+0000 7f58c5a2f640 1 -- 172.21.15.100:0/2428485184 learned_addr learned my addr 172.21.15.100:0/2428485184 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:36:33.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.158+0000 7f58c4a2d640 1 --2- 172.21.15.100:0/2428485184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58c0107e50 0x7f58c019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:33.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.158+0000 7f58c522e640 1 --2- 172.21.15.100:0/2428485184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58c0102510 0x7f58c0199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:33.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.158+0000 7f58c5a2f640 1 -- 172.21.15.100:0/2428485184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58c0107e50 msgr2=0x7f58c019a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:33.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.158+0000 7f58c5a2f640 1 --2- 172.21.15.100:0/2428485184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58c0107e50 0x7f58c019a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:33.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.158+0000 7f58c5a2f640 1 -- 172.21.15.100:0/2428485184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58c0102510 msgr2=0x7f58c0199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:33.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.158+0000 7f58c5a2f640 1 --2- 172.21.15.100:0/2428485184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58c0102510 0x7f58c0199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:33.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.158+0000 7f58c5a2f640 1 -- 172.21.15.100:0/2428485184 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c019ff20 con 0x7f58c01087a0 2024-07-26T13:36:33.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.158+0000 7f58c522e640 1 --2- 172.21.15.100:0/2428485184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58c0102510 0x7f58c0199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:36:33.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.158+0000 7f58c5a2f640 1 --2- 172.21.15.100:0/2428485184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58c01087a0 0x7f58c019f7f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f58b8005cc0 tx=0x7f58b8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:36:33.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.158+0000 7f58b27fc640 1 -- 172.21.15.100:0/2428485184 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b8004040 con 0x7f58c01087a0 2024-07-26T13:36:33.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.158+0000 7f58c7492640 1 -- 172.21.15.100:0/2428485184 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58c01a01b0 con 0x7f58c01087a0 2024-07-26T13:36:33.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.158+0000 7f58b27fc640 1 -- 172.21.15.100:0/2428485184 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58b8004690 con 0x7f58c01087a0 2024-07-26T13:36:33.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.158+0000 7f58b27fc640 1 -- 172.21.15.100:0/2428485184 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b8030e70 con 0x7f58c01087a0 2024-07-26T13:36:33.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.158+0000 7f58c7492640 1 -- 172.21.15.100:0/2428485184 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58c01a6970 con 0x7f58c01087a0 2024-07-26T13:36:33.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.159+0000 7f58c7492640 1 -- 172.21.15.100:0/2428485184 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58c0103790 con 0x7f58c01087a0 2024-07-26T13:36:33.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.161+0000 7f58b27fc640 1 -- 172.21.15.100:0/2428485184 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f58b80305e0 con 0x7f58c01087a0 2024-07-26T13:36:33.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.162+0000 7f58b27fc640 1 --2- 172.21.15.100:0/2428485184 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f58ac0b3c20 0x7f58ac0b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:33.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.162+0000 7f58c522e640 1 --2- 172.21.15.100:0/2428485184 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f58ac0b3c20 0x7f58ac0b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:33.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.162+0000 7f58b27fc640 1 -- 172.21.15.100:0/2428485184 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f58b8104ff0 con 0x7f58c01087a0 2024-07-26T13:36:33.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.163+0000 7f58c522e640 1 --2- 172.21.15.100:0/2428485184 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f58ac0b3c20 0x7f58ac0b6130 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f58b4006fd0 tx=0x7f58b4008040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:36:33.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.165+0000 7f58b27fc640 1 -- 172.21.15.100:0/2428485184 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f58b80cca10 con 0x7f58c01087a0 2024-07-26T13:36:33.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-26T13:36:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-26T13:36:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:33 smithi167 ceph-mon[37140]: Deploying daemon mds.i on smithi167 2024-07-26T13:36:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:33 smithi167 ceph-mon[37140]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:33 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.138:6836/3393225483,v1:172.21.15.138:6837/3393225483] up:boot 2024-07-26T13:36:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:33 smithi167 ceph-mon[37140]: fsmap 8 up:standby 2024-07-26T13:36:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-26T13:36:33.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.377+0000 7f58c7492640 1 -- 172.21.15.100:0/2428485184 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58c0102920 con 0x7f58c01087a0 2024-07-26T13:36:33.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.378+0000 7f58b27fc640 1 -- 172.21.15.100:0/2428485184 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7228 (secure 0 0 0) 0x7f58c0102920 con 0x7f58c01087a0 2024-07-26T13:36:33.380 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:36:33.380 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":9,"btime":"2024-07-26T13:36:32:435866+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24557,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/3290105300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3290105300},{"type":"v1","addr":"172.21.15.138:6835","nonce":3290105300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24575,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3393225483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3393225483},{"type":"v1","addr":"172.21.15.138:6837","nonce":3393225483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24586,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24595,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3828634468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3828634468},{"type":"v1","addr":"172.21.15.167:6837","nonce":3828634468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-07-26T13:36:33.380 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 9 2024-07-26T13:36:33.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.380+0000 7f58c7492640 1 -- 172.21.15.100:0/2428485184 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f58ac0b3c20 msgr2=0x7f58ac0b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:33.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.380+0000 7f58c7492640 1 --2- 172.21.15.100:0/2428485184 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f58ac0b3c20 0x7f58ac0b6130 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f58b4006fd0 tx=0x7f58b4008040 comp rx=0 tx=0).stop 2024-07-26T13:36:33.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.380+0000 7f58c7492640 1 -- 172.21.15.100:0/2428485184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58c01087a0 msgr2=0x7f58c019f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:33.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.380+0000 7f58c7492640 1 --2- 172.21.15.100:0/2428485184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58c01087a0 0x7f58c019f7f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f58b8005cc0 tx=0x7f58b8003ba0 comp rx=0 tx=0).stop 2024-07-26T13:36:33.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.380+0000 7f58c7492640 1 -- 172.21.15.100:0/2428485184 shutdown_connections 2024-07-26T13:36:33.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.380+0000 7f58c7492640 1 --2- 172.21.15.100:0/2428485184 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f58ac0b3c20 0x7f58ac0b6130 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:33.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.380+0000 7f58c7492640 1 --2- 172.21.15.100:0/2428485184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f58c01087a0 0x7f58c019f7f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:33.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.380+0000 7f58c7492640 1 --2- 172.21.15.100:0/2428485184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f58c0107e50 0x7f58c019a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:33.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.380+0000 7f58c7492640 1 --2- 172.21.15.100:0/2428485184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58c0102510 0x7f58c0199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:33.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.380+0000 7f58c7492640 1 -- 172.21.15.100:0/2428485184 >> 172.21.15.100:0/2428485184 conn(0x7f58c00791a0 msgr2=0x7f58c0105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:36:33.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.380+0000 7f58c7492640 1 -- 172.21.15.100:0/2428485184 shutdown_connections 2024-07-26T13:36:33.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:33.380+0000 7f58c7492640 1 -- 172.21.15.100:0/2428485184 wait complete. 2024-07-26T13:36:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-26T13:36:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-26T13:36:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:33 smithi100 ceph-mon[32599]: Deploying daemon mds.i on smithi167 2024-07-26T13:36:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:33 smithi100 ceph-mon[32599]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:33.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:33 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.138:6836/3393225483,v1:172.21.15.138:6837/3393225483] up:boot 2024-07-26T13:36:33.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:33 smithi100 ceph-mon[32599]: fsmap 8 up:standby 2024-07-26T13:36:33.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-26T13:36:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-26T13:36:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-26T13:36:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:33 smithi138 ceph-mon[36954]: Deploying daemon mds.i on smithi167 2024-07-26T13:36:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:33 smithi138 ceph-mon[36954]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:33 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.138:6836/3393225483,v1:172.21.15.138:6837/3393225483] up:boot 2024-07-26T13:36:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:33 smithi138 ceph-mon[36954]: fsmap 8 up:standby 2024-07-26T13:36:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-26T13:36:34.225 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:36:34 smithi167 systemd[1]: Starting Ceph mds.i for bf83e2e6-4b52-11ef-bcb7-c7b262605968... 2024-07-26T13:36:34.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:34 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2428485184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:36:34.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:34 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/2428485184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:36:34.523 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:36:34 smithi167 podman[69318]: 2024-07-26 13:36:34.28421976 +0000 UTC m=+0.017213818 image pull ae51f432d4fa82622bc253ca9e2da7666c617e096e99d43d0a0e4fdaca8865f8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2 2024-07-26T13:36:34.523 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:36:34 smithi167 podman[69318]: 2024-07-26 13:36:34.407324155 +0000 UTC m=+0.140318204 container create dd8ee84abc2d1afc61ce4713e9267a89e11913afbf6df3bcd49ffb48163c4f1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True) 2024-07-26T13:36:34.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:34 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/2428485184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:36:35.023 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:36:34 smithi167 podman[69318]: 2024-07-26 13:36:34.660837896 +0000 UTC m=+0.393831960 container init dd8ee84abc2d1afc61ce4713e9267a89e11913afbf6df3bcd49ffb48163c4f1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-26T13:36:35.023 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:36:34 smithi167 podman[69318]: 2024-07-26 13:36:34.663425284 +0000 UTC m=+0.396419327 container start dd8ee84abc2d1afc61ce4713e9267a89e11913afbf6df3bcd49ffb48163c4f1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-07-26T13:36:35.023 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:36:34 smithi167 ceph-mds[69336]: -- 172.21.15.167:0/2261217295 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558928151c20 con 0x55892c2f3a80 2024-07-26T13:36:35.023 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:36:34 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: starting mds.i at 2024-07-26T13:36:35.023 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:36:34 smithi167 bash[69318]: dd8ee84abc2d1afc61ce4713e9267a89e11913afbf6df3bcd49ffb48163c4f1f 2024-07-26T13:36:35.023 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:36:34 smithi167 systemd[1]: Started Ceph mds.i for bf83e2e6-4b52-11ef-bcb7-c7b262605968. 2024-07-26T13:36:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:35 smithi100 ceph-mon[32599]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-26T13:36:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-26T13:36:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:35.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:35 smithi167 ceph-mon[37140]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:35.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:35.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:35.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:35.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-26T13:36:35.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-26T13:36:35.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:35.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:35 smithi138 ceph-mon[36954]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:35.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:35.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:35.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:35.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-26T13:36:35.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-26T13:36:35.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:36.166 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:36:36.478 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:36 smithi100 ceph-mon[32599]: Deploying daemon mds.g on smithi100 2024-07-26T13:36:36.479 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:36 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] up:boot 2024-07-26T13:36:36.479 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:36 smithi100 ceph-mon[32599]: fsmap 9 up:standby 2024-07-26T13:36:36.479 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-26T13:36:36.484 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:36:36.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:36 smithi167 ceph-mon[37140]: Deploying daemon mds.g on smithi100 2024-07-26T13:36:36.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:36 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] up:boot 2024-07-26T13:36:36.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:36 smithi167 ceph-mon[37140]: fsmap 9 up:standby 2024-07-26T13:36:36.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-26T13:36:36.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:36 smithi138 ceph-mon[36954]: Deploying daemon mds.g on smithi100 2024-07-26T13:36:36.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:36 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] up:boot 2024-07-26T13:36:36.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:36 smithi138 ceph-mon[36954]: fsmap 9 up:standby 2024-07-26T13:36:36.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-26T13:36:37.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:37 smithi138 ceph-mon[36954]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:37 smithi100 ceph-mon[32599]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:37.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:37 smithi167 ceph-mon[37140]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:38.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.246+0000 7efd584e6640 1 -- 172.21.15.100:0/4017923204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd50107dd0 msgr2=0x7efd501081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:38.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.246+0000 7efd584e6640 1 --2- 172.21.15.100:0/4017923204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd50107dd0 0x7efd501081e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efd40009900 tx=0x7efd4002a410 comp rx=0 tx=0).stop 2024-07-26T13:36:38.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.246+0000 7efd584e6640 1 -- 172.21.15.100:0/4017923204 shutdown_connections 2024-07-26T13:36:38.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.246+0000 7efd584e6640 1 --2- 172.21.15.100:0/4017923204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd50108720 0x7efd5010ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:38.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.246+0000 7efd584e6640 1 --2- 172.21.15.100:0/4017923204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd50107dd0 0x7efd501081e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:38.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.246+0000 7efd584e6640 1 --2- 172.21.15.100:0/4017923204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd50102490 0x7efd501028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:38.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.246+0000 7efd584e6640 1 -- 172.21.15.100:0/4017923204 >> 172.21.15.100:0/4017923204 conn(0x7efd50079160 msgr2=0x7efd50079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:36:38.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.246+0000 7efd584e6640 1 -- 172.21.15.100:0/4017923204 shutdown_connections 2024-07-26T13:36:38.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.246+0000 7efd584e6640 1 -- 172.21.15.100:0/4017923204 wait complete. 2024-07-26T13:36:38.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.247+0000 7efd584e6640 1 Processor -- start 2024-07-26T13:36:38.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.247+0000 7efd584e6640 1 -- start start 2024-07-26T13:36:38.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.247+0000 7efd584e6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd50102490 0x7efd50198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:38.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.247+0000 7efd584e6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd50107dd0 0x7efd50199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:38.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.247+0000 7efd584e6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd50108720 0x7efd5019e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:38.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.247+0000 7efd584e6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efd50112370 con 0x7efd50102490 2024-07-26T13:36:38.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.247+0000 7efd584e6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efd501121f0 con 0x7efd50108720 2024-07-26T13:36:38.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.247+0000 7efd584e6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efd501124f0 con 0x7efd50107dd0 2024-07-26T13:36:38.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.247+0000 7efd56282640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd50102490 0x7efd50198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:38.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.247+0000 7efd56282640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd50102490 0x7efd50198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45172/0 (socket says 172.21.15.100:45172) 2024-07-26T13:36:38.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.247+0000 7efd56282640 1 -- 172.21.15.100:0/2246388913 learned_addr learned my addr 172.21.15.100:0/2246388913 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:36:38.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.247+0000 7efd56a83640 1 --2- 172.21.15.100:0/2246388913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd50108720 0x7efd5019e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:38.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.247+0000 7efd55a81640 1 --2- 172.21.15.100:0/2246388913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd50107dd0 0x7efd50199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:38.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.248+0000 7efd56282640 1 -- 172.21.15.100:0/2246388913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd50107dd0 msgr2=0x7efd50199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:38.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.248+0000 7efd56282640 1 --2- 172.21.15.100:0/2246388913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd50107dd0 0x7efd50199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:38.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.248+0000 7efd56282640 1 -- 172.21.15.100:0/2246388913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd50108720 msgr2=0x7efd5019e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:38.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.248+0000 7efd56282640 1 --2- 172.21.15.100:0/2246388913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd50108720 0x7efd5019e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:38.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.248+0000 7efd56282640 1 -- 172.21.15.100:0/2246388913 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd5019ed10 con 0x7efd50102490 2024-07-26T13:36:38.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.248+0000 7efd56282640 1 --2- 172.21.15.100:0/2246388913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd50102490 0x7efd50198da0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7efd4c00d770 tx=0x7efd4c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:36:38.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.248+0000 7efd477fe640 1 -- 172.21.15.100:0/2246388913 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c017070 con 0x7efd50102490 2024-07-26T13:36:38.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.248+0000 7efd477fe640 1 -- 172.21.15.100:0/2246388913 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd4c0054d0 con 0x7efd50102490 2024-07-26T13:36:38.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.248+0000 7efd477fe640 1 -- 172.21.15.100:0/2246388913 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c004c30 con 0x7efd50102490 2024-07-26T13:36:38.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.248+0000 7efd584e6640 1 -- 172.21.15.100:0/2246388913 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd5019f000 con 0x7efd50102490 2024-07-26T13:36:38.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.248+0000 7efd584e6640 1 -- 172.21.15.100:0/2246388913 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd5006a890 con 0x7efd50102490 2024-07-26T13:36:38.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.249+0000 7efd584e6640 1 -- 172.21.15.100:0/2246388913 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd501028a0 con 0x7efd50102490 2024-07-26T13:36:38.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.252+0000 7efd477fe640 1 -- 172.21.15.100:0/2246388913 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7efd4c004dd0 con 0x7efd50102490 2024-07-26T13:36:38.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.252+0000 7efd477fe640 1 --2- 172.21.15.100:0/2246388913 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7efd3c0b3ca0 0x7efd3c0b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:38.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.253+0000 7efd55a81640 1 --2- 172.21.15.100:0/2246388913 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7efd3c0b3ca0 0x7efd3c0b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:38.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.253+0000 7efd477fe640 1 -- 172.21.15.100:0/2246388913 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7efd4c0e3700 con 0x7efd50102490 2024-07-26T13:36:38.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.253+0000 7efd55a81640 1 --2- 172.21.15.100:0/2246388913 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7efd3c0b3ca0 0x7efd3c0b61b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efd40000c00 tx=0x7efd40031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:36:38.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.255+0000 7efd477fe640 1 -- 172.21.15.100:0/2246388913 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7efd4c0ab1a0 con 0x7efd50102490 2024-07-26T13:36:38.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.467+0000 7efd584e6640 1 -- 172.21.15.100:0/2246388913 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd5006b2e0 con 0x7efd50102490 2024-07-26T13:36:38.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.468+0000 7efd477fe640 1 -- 172.21.15.100:0/2246388913 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8055 (secure 0 0 0) 0x7efd4c0aaaa0 con 0x7efd50102490 2024-07-26T13:36:38.469 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:36:38.469 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":10,"btime":"2024-07-26T13:36:35:225064+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24557,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/3290105300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3290105300},{"type":"v1","addr":"172.21.15.138:6835","nonce":3290105300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24575,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3393225483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3393225483},{"type":"v1","addr":"172.21.15.138:6837","nonce":3393225483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24586,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24595,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3828634468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3828634468},{"type":"v1","addr":"172.21.15.167:6837","nonce":3828634468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24607,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/398150514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":398150514},{"type":"v1","addr":"172.21.15.167:6839","nonce":398150514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-07-26T13:36:38.469 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 10 2024-07-26T13:36:38.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.470+0000 7efd584e6640 1 -- 172.21.15.100:0/2246388913 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7efd3c0b3ca0 msgr2=0x7efd3c0b61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:38.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.470+0000 7efd584e6640 1 --2- 172.21.15.100:0/2246388913 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7efd3c0b3ca0 0x7efd3c0b61b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efd40000c00 tx=0x7efd40031400 comp rx=0 tx=0).stop 2024-07-26T13:36:38.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.470+0000 7efd584e6640 1 -- 172.21.15.100:0/2246388913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd50102490 msgr2=0x7efd50198da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:38.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.470+0000 7efd584e6640 1 --2- 172.21.15.100:0/2246388913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd50102490 0x7efd50198da0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7efd4c00d770 tx=0x7efd4c00dc30 comp rx=0 tx=0).stop 2024-07-26T13:36:38.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.470+0000 7efd584e6640 1 -- 172.21.15.100:0/2246388913 shutdown_connections 2024-07-26T13:36:38.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.470+0000 7efd584e6640 1 --2- 172.21.15.100:0/2246388913 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7efd3c0b3ca0 0x7efd3c0b61b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:38.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.470+0000 7efd584e6640 1 --2- 172.21.15.100:0/2246388913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd50108720 0x7efd5019e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:38.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.470+0000 7efd584e6640 1 --2- 172.21.15.100:0/2246388913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd50107dd0 0x7efd50199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:38.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.470+0000 7efd584e6640 1 --2- 172.21.15.100:0/2246388913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efd50102490 0x7efd50198da0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:38.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.470+0000 7efd584e6640 1 -- 172.21.15.100:0/2246388913 >> 172.21.15.100:0/2246388913 conn(0x7efd50079160 msgr2=0x7efd50103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:36:38.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.470+0000 7efd584e6640 1 -- 172.21.15.100:0/2246388913 shutdown_connections 2024-07-26T13:36:38.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:38.470+0000 7efd584e6640 1 -- 172.21.15.100:0/2246388913 wait complete. 2024-07-26T13:36:38.597 INFO:journalctl@ceph.mds.g.smithi100.stdout:Jul 26 13:36:38 smithi100 podman[79236]: 2024-07-26 13:36:38.596297146 +0000 UTC m=+0.548682666 container init 3e857d35b9497b351b8a8647f0c2c6fda43a8f775528094a1582e6a1c419cc97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-g, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-26T13:36:38.994 INFO:journalctl@ceph.mds.g.smithi100.stdout:Jul 26 13:36:38 smithi100 podman[79236]: 2024-07-26 13:36:38.599404394 +0000 UTC m=+0.551789912 container start 3e857d35b9497b351b8a8647f0c2c6fda43a8f775528094a1582e6a1c419cc97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-g, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-26T13:36:38.994 INFO:journalctl@ceph.mds.g.smithi100.stdout:Jul 26 13:36:38 smithi100 ceph-mds[79289]: -- 172.21.15.100:0/635370552 <== mon.0 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561e7c7f1860 con 0x561e80993a80 2024-07-26T13:36:38.994 INFO:journalctl@ceph.mds.g.smithi100.stdout:Jul 26 13:36:38 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-g[79272]: starting mds.g at 2024-07-26T13:36:38.994 INFO:journalctl@ceph.mds.g.smithi100.stdout:Jul 26 13:36:38 smithi100 bash[79236]: 3e857d35b9497b351b8a8647f0c2c6fda43a8f775528094a1582e6a1c419cc97 2024-07-26T13:36:38.994 INFO:journalctl@ceph.mds.g.smithi100.stdout:Jul 26 13:36:38 smithi100 systemd[1]: Started Ceph mds.g for bf83e2e6-4b52-11ef-bcb7-c7b262605968. 2024-07-26T13:36:39.622 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:39 smithi100 ceph-mon[32599]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:39.622 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:39 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2246388913' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:36:39.622 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:39 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:39.622 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:39 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:39.622 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:39 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:39.622 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:39 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-26T13:36:39.622 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:39 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-26T13:36:39.622 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:39 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:39.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:39 smithi138 ceph-mon[36954]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:39.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:39 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/2246388913' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:36:39.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:39 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:39.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:39 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:39.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:39 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:39.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:39 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-26T13:36:39.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:39 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-26T13:36:39.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:39 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:39.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:39 smithi167 ceph-mon[37140]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:39.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:39 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/2246388913' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:36:39.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:39 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:39.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:39 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:39.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:39 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:39.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:39 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-26T13:36:39.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:39 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-26T13:36:39.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:39 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:40.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:40 smithi138 ceph-mon[36954]: Deploying daemon mds.a on smithi100 2024-07-26T13:36:40.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:40 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.100:6838/313501409,v1:172.21.15.100:6839/313501409] up:boot 2024-07-26T13:36:40.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:40 smithi138 ceph-mon[36954]: fsmap 10 up:standby 2024-07-26T13:36:40.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-07-26T13:36:40.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:40 smithi100 ceph-mon[32599]: Deploying daemon mds.a on smithi100 2024-07-26T13:36:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:40 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.100:6838/313501409,v1:172.21.15.100:6839/313501409] up:boot 2024-07-26T13:36:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:40 smithi100 ceph-mon[32599]: fsmap 10 up:standby 2024-07-26T13:36:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-07-26T13:36:40.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:40 smithi167 ceph-mon[37140]: Deploying daemon mds.a on smithi100 2024-07-26T13:36:40.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:40 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.100:6838/313501409,v1:172.21.15.100:6839/313501409] up:boot 2024-07-26T13:36:40.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:40 smithi167 ceph-mon[37140]: fsmap 10 up:standby 2024-07-26T13:36:40.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-07-26T13:36:41.165 INFO:journalctl@ceph.mds.a.smithi100.stdout:Jul 26 13:36:40 smithi100 systemd[1]: Starting Ceph mds.a for bf83e2e6-4b52-11ef-bcb7-c7b262605968... 2024-07-26T13:36:41.354 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:36:41.415 INFO:journalctl@ceph.mds.a.smithi100.stdout:Jul 26 13:36:41 smithi100 podman[79574]: 2024-07-26 13:36:41.067163147 +0000 UTC m=+0.017303903 image pull ae51f432d4fa82622bc253ca9e2da7666c617e096e99d43d0a0e4fdaca8865f8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2 2024-07-26T13:36:41.416 INFO:journalctl@ceph.mds.a.smithi100.stdout:Jul 26 13:36:41 smithi100 podman[79574]: 2024-07-26 13:36:41.170844014 +0000 UTC m=+0.120984761 container create e5620637ae0a8280b4e2e08a31c3193d304082a4127ecd7e93704c533bd1bc66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-a, io.buildah.version=1.36.0, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-07-26T13:36:41.416 INFO:journalctl@ceph.mds.a.smithi100.stdout:Jul 26 13:36:41 smithi100 podman[79574]: 2024-07-26 13:36:41.415344708 +0000 UTC m=+0.365485455 container init e5620637ae0a8280b4e2e08a31c3193d304082a4127ecd7e93704c533bd1bc66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-a, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-07-26T13:36:41.627 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:36:41.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:41 smithi138 ceph-mon[36954]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:41 smithi100 ceph-mon[32599]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:41.744 INFO:journalctl@ceph.mds.a.smithi100.stdout:Jul 26 13:36:41 smithi100 podman[79574]: 2024-07-26 13:36:41.418780785 +0000 UTC m=+0.368921531 container start e5620637ae0a8280b4e2e08a31c3193d304082a4127ecd7e93704c533bd1bc66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb) 2024-07-26T13:36:41.744 INFO:journalctl@ceph.mds.a.smithi100.stdout:Jul 26 13:36:41 smithi100 ceph-mds[79611]: -- 172.21.15.100:0/3060057678 <== mon.0 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aaa6ddf860 con 0x55aaaaf81a80 2024-07-26T13:36:41.744 INFO:journalctl@ceph.mds.a.smithi100.stdout:Jul 26 13:36:41 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-a[79589]: starting mds.a at 2024-07-26T13:36:41.744 INFO:journalctl@ceph.mds.a.smithi100.stdout:Jul 26 13:36:41 smithi100 bash[79574]: e5620637ae0a8280b4e2e08a31c3193d304082a4127ecd7e93704c533bd1bc66 2024-07-26T13:36:41.744 INFO:journalctl@ceph.mds.a.smithi100.stdout:Jul 26 13:36:41 smithi100 systemd[1]: Started Ceph mds.a for bf83e2e6-4b52-11ef-bcb7-c7b262605968. 2024-07-26T13:36:41.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:41 smithi167 ceph-mon[37140]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:42.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:42 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:42.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:42 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:42.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:42 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:42.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:42 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-26T13:36:42.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:42 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-26T13:36:42.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:42 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:42.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:42 smithi138 ceph-mon[36954]: Deploying daemon mds.b on smithi138 2024-07-26T13:36:42.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:42 smithi138 ceph-mon[36954]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:42.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:42 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.100:6840/1402975506,v1:172.21.15.100:6841/1402975506] up:boot 2024-07-26T13:36:42.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:42 smithi138 ceph-mon[36954]: fsmap 11 up:standby 2024-07-26T13:36:42.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:42 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-07-26T13:36:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:42 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:42 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:42 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:42 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-26T13:36:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:42 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-26T13:36:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:42 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:42 smithi100 ceph-mon[32599]: Deploying daemon mds.b on smithi138 2024-07-26T13:36:42.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:42 smithi100 ceph-mon[32599]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:42.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:42 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.100:6840/1402975506,v1:172.21.15.100:6841/1402975506] up:boot 2024-07-26T13:36:42.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:42 smithi100 ceph-mon[32599]: fsmap 11 up:standby 2024-07-26T13:36:42.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:42 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-07-26T13:36:43.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:42 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:43.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:42 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:43.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:42 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:43.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:42 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-26T13:36:43.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:42 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-26T13:36:43.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:42 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:43.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:42 smithi167 ceph-mon[37140]: Deploying daemon mds.b on smithi138 2024-07-26T13:36:43.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:42 smithi167 ceph-mon[37140]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:43.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:42 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.100:6840/1402975506,v1:172.21.15.100:6841/1402975506] up:boot 2024-07-26T13:36:43.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:42 smithi167 ceph-mon[37140]: fsmap 11 up:standby 2024-07-26T13:36:43.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:42 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-07-26T13:36:43.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.292+0000 7fac4d1ef640 1 -- 172.21.15.100:0/1057116502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac48103590 msgr2=0x7fac481039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:43.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.292+0000 7fac4d1ef640 1 --2- 172.21.15.100:0/1057116502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac48103590 0x7fac481039a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fac38009900 tx=0x7fac3802a410 comp rx=0 tx=0).stop 2024-07-26T13:36:43.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.292+0000 7fac4d1ef640 1 -- 172.21.15.100:0/1057116502 shutdown_connections 2024-07-26T13:36:43.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.292+0000 7fac4d1ef640 1 --2- 172.21.15.100:0/1057116502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac48109820 0x7fac48110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:43.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.292+0000 7fac4d1ef640 1 --2- 172.21.15.100:0/1057116502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac48108ed0 0x7fac481092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:43.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.292+0000 7fac4d1ef640 1 --2- 172.21.15.100:0/1057116502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac48103590 0x7fac481039a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:43.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.292+0000 7fac4d1ef640 1 -- 172.21.15.100:0/1057116502 >> 172.21.15.100:0/1057116502 conn(0x7fac4807b280 msgr2=0x7fac4807b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:36:43.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.292+0000 7fac4d1ef640 1 -- 172.21.15.100:0/1057116502 shutdown_connections 2024-07-26T13:36:43.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.292+0000 7fac4d1ef640 1 -- 172.21.15.100:0/1057116502 wait complete. 2024-07-26T13:36:43.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.293+0000 7fac4d1ef640 1 Processor -- start 2024-07-26T13:36:43.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.293+0000 7fac4d1ef640 1 -- start start 2024-07-26T13:36:43.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.294+0000 7fac4d1ef640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac48103590 0x7fac48199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:43.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.294+0000 7fac4d1ef640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac48108ed0 0x7fac4819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:43.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.294+0000 7fac4d1ef640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac48109820 0x7fac4819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:43.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.294+0000 7fac4d1ef640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fac481134f0 con 0x7fac48103590 2024-07-26T13:36:43.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.294+0000 7fac4d1ef640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fac48113370 con 0x7fac48109820 2024-07-26T13:36:43.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.294+0000 7fac4d1ef640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fac48113670 con 0x7fac48108ed0 2024-07-26T13:36:43.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.294+0000 7fac46d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac48103590 0x7fac48199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:43.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.294+0000 7fac46d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac48103590 0x7fac48199ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45248/0 (socket says 172.21.15.100:45248) 2024-07-26T13:36:43.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.294+0000 7fac46d9d640 1 -- 172.21.15.100:0/1626325136 learned_addr learned my addr 172.21.15.100:0/1626325136 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:36:43.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.294+0000 7fac4759e640 1 --2- 172.21.15.100:0/1626325136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac48109820 0x7fac4819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:43.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.294+0000 7fac4659c640 1 --2- 172.21.15.100:0/1626325136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac48108ed0 0x7fac4819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:43.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.294+0000 7fac46d9d640 1 -- 172.21.15.100:0/1626325136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac48108ed0 msgr2=0x7fac4819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:43.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.294+0000 7fac46d9d640 1 --2- 172.21.15.100:0/1626325136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac48108ed0 0x7fac4819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:43.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.294+0000 7fac46d9d640 1 -- 172.21.15.100:0/1626325136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac48109820 msgr2=0x7fac4819f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:43.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.294+0000 7fac46d9d640 1 --2- 172.21.15.100:0/1626325136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac48109820 0x7fac4819f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:43.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.294+0000 7fac46d9d640 1 -- 172.21.15.100:0/1626325136 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac4819fe50 con 0x7fac48103590 2024-07-26T13:36:43.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.294+0000 7fac46d9d640 1 --2- 172.21.15.100:0/1626325136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac48103590 0x7fac48199ee0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fac38005940 tx=0x7fac38004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:36:43.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.294+0000 7fac2ffff640 1 -- 172.21.15.100:0/1626325136 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3803f030 con 0x7fac48103590 2024-07-26T13:36:43.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.294+0000 7fac4d1ef640 1 -- 172.21.15.100:0/1626325136 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac481a00e0 con 0x7fac48103590 2024-07-26T13:36:43.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.295+0000 7fac2ffff640 1 -- 172.21.15.100:0/1626325136 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac38003aa0 con 0x7fac48103590 2024-07-26T13:36:43.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.295+0000 7fac2ffff640 1 -- 172.21.15.100:0/1626325136 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac38035070 con 0x7fac48103590 2024-07-26T13:36:43.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.295+0000 7fac4d1ef640 1 -- 172.21.15.100:0/1626325136 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac481a68a0 con 0x7fac48103590 2024-07-26T13:36:43.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.295+0000 7fac4d1ef640 1 -- 172.21.15.100:0/1626325136 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac481039a0 con 0x7fac48103590 2024-07-26T13:36:43.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.297+0000 7fac2ffff640 1 -- 172.21.15.100:0/1626325136 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7fac38042030 con 0x7fac48103590 2024-07-26T13:36:43.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.298+0000 7fac2ffff640 1 --2- 172.21.15.100:0/1626325136 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fac300b3c20 0x7fac300b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:43.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.298+0000 7fac4659c640 1 --2- 172.21.15.100:0/1626325136 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fac300b3c20 0x7fac300b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:43.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.298+0000 7fac2ffff640 1 -- 172.21.15.100:0/1626325136 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7fac38104150 con 0x7fac48103590 2024-07-26T13:36:43.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.298+0000 7fac4659c640 1 --2- 172.21.15.100:0/1626325136 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fac300b3c20 0x7fac300b6130 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fac48105f70 tx=0x7fac3c005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:36:43.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.301+0000 7fac2ffff640 1 -- 172.21.15.100:0/1626325136 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fac380cb520 con 0x7fac48103590 2024-07-26T13:36:43.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.537+0000 7fac4d1ef640 1 -- 172.21.15.100:0/1626325136 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac4810b5d0 con 0x7fac48103590 2024-07-26T13:36:43.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.538+0000 7fac2ffff640 1 -- 172.21.15.100:0/1626325136 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9710 (secure 0 0 0) 0x7fac380cfa70 con 0x7fac48103590 2024-07-26T13:36:43.539 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:36:43.539 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":12,"btime":"2024-07-26T13:36:42:421145+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14790,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/313501409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":313501409},{"type":"v1","addr":"172.21.15.100:6839","nonce":313501409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/1402975506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":1402975506},{"type":"v1","addr":"172.21.15.100:6841","nonce":1402975506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24557,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/3290105300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3290105300},{"type":"v1","addr":"172.21.15.138:6835","nonce":3290105300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24575,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3393225483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3393225483},{"type":"v1","addr":"172.21.15.138:6837","nonce":3393225483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24586,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24595,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3828634468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3828634468},{"type":"v1","addr":"172.21.15.167:6837","nonce":3828634468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24607,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/398150514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":398150514},{"type":"v1","addr":"172.21.15.167:6839","nonce":398150514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-07-26T13:36:43.539 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 12 2024-07-26T13:36:43.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.540+0000 7fac4d1ef640 1 -- 172.21.15.100:0/1626325136 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fac300b3c20 msgr2=0x7fac300b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:43.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.540+0000 7fac4d1ef640 1 --2- 172.21.15.100:0/1626325136 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fac300b3c20 0x7fac300b6130 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fac48105f70 tx=0x7fac3c005f90 comp rx=0 tx=0).stop 2024-07-26T13:36:43.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.540+0000 7fac4d1ef640 1 -- 172.21.15.100:0/1626325136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac48103590 msgr2=0x7fac48199ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:43.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.540+0000 7fac4d1ef640 1 --2- 172.21.15.100:0/1626325136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac48103590 0x7fac48199ee0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fac38005940 tx=0x7fac38004270 comp rx=0 tx=0).stop 2024-07-26T13:36:43.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.540+0000 7fac4d1ef640 1 -- 172.21.15.100:0/1626325136 shutdown_connections 2024-07-26T13:36:43.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.540+0000 7fac4d1ef640 1 --2- 172.21.15.100:0/1626325136 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fac300b3c20 0x7fac300b6130 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:43.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.540+0000 7fac4d1ef640 1 --2- 172.21.15.100:0/1626325136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac48109820 0x7fac4819f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:43.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.540+0000 7fac4d1ef640 1 --2- 172.21.15.100:0/1626325136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac48108ed0 0x7fac4819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:43.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.540+0000 7fac4d1ef640 1 --2- 172.21.15.100:0/1626325136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac48103590 0x7fac48199ee0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:43.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.540+0000 7fac4d1ef640 1 -- 172.21.15.100:0/1626325136 >> 172.21.15.100:0/1626325136 conn(0x7fac4807b280 msgr2=0x7fac48106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:36:43.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.540+0000 7fac4d1ef640 1 -- 172.21.15.100:0/1626325136 shutdown_connections 2024-07-26T13:36:43.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:43.540+0000 7fac4d1ef640 1 -- 172.21.15.100:0/1626325136 wait complete. 2024-07-26T13:36:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:43 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/1626325136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:36:43.899 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:43 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/1626325136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:36:43.899 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:36:43 smithi138 systemd[1]: Starting Ceph mds.b for bf83e2e6-4b52-11ef-bcb7-c7b262605968... 2024-07-26T13:36:44.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:43 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/1626325136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:36:44.164 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:36:43 smithi138 podman[68485]: 2024-07-26 13:36:43.801087088 +0000 UTC m=+0.017020709 image pull ae51f432d4fa82622bc253ca9e2da7666c617e096e99d43d0a0e4fdaca8865f8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2 2024-07-26T13:36:44.164 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:36:43 smithi138 podman[68485]: 2024-07-26 13:36:43.918682003 +0000 UTC m=+0.134615615 container create e336832bdc01fd5a1dea2d988067e30ce239fbac9099bf3fbc87406a15235104 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-07-26T13:36:44.164 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:36:44 smithi138 podman[68485]: 2024-07-26 13:36:44.163527803 +0000 UTC m=+0.379461410 container init e336832bdc01fd5a1dea2d988067e30ce239fbac9099bf3fbc87406a15235104 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240724, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-07-26T13:36:44.440 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:36:44 smithi138 podman[68485]: 2024-07-26 13:36:44.166393891 +0000 UTC m=+0.382327507 container start e336832bdc01fd5a1dea2d988067e30ce239fbac9099bf3fbc87406a15235104 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240724) 2024-07-26T13:36:44.440 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:36:44 smithi138 ceph-mds[68504]: -- 172.21.15.138:0/461571015 <== mon.2 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bb94785860 con 0x55bb98927180 2024-07-26T13:36:44.440 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:36:44 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: starting mds.b at 2024-07-26T13:36:44.440 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:36:44 smithi138 bash[68485]: e336832bdc01fd5a1dea2d988067e30ce239fbac9099bf3fbc87406a15235104 2024-07-26T13:36:44.440 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:36:44 smithi138 systemd[1]: Started Ceph mds.b for bf83e2e6-4b52-11ef-bcb7-c7b262605968. 2024-07-26T13:36:45.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:45 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:45.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:45 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:45.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:45 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:45.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:45 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:45.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:45 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:45.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:45 smithi138 ceph-mon[36954]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:45.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:45 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:36:45.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:45 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] up:boot 2024-07-26T13:36:45.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:45 smithi138 ceph-mon[36954]: fsmap 12 up:standby 2024-07-26T13:36:45.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:45 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-26T13:36:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:45 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:45 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:45 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:45 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:45 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:45 smithi100 ceph-mon[32599]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:45 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:36:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:45 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] up:boot 2024-07-26T13:36:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:45 smithi100 ceph-mon[32599]: fsmap 12 up:standby 2024-07-26T13:36:45.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:45 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-26T13:36:45.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:45 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:45.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:45 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:45.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:45 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:45.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:45 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:45.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:45 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:45.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:45 smithi167 ceph-mon[37140]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:45.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:45 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:36:45.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:45 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] up:boot 2024-07-26T13:36:45.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:45 smithi167 ceph-mon[37140]: fsmap 12 up:standby 2024-07-26T13:36:45.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:45 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-26T13:36:46.217 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:36:46.490 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:36:47.016 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:47.016 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:47.016 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:47.016 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:47.016 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:47.016 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:47.016 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:47.017 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:36:47.017 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:47.017 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:36:47.017 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:47.017 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:46 smithi138 ceph-mon[36954]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:36:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:36:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:46 smithi167 ceph-mon[37140]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:36:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:47.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:36:47.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:47.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:46 smithi100 ceph-mon[32599]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:48.663 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:48 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:48.663 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:48 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:48.663 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:48 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:48.663 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:48 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:48.663 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:48 smithi100 ceph-mon[32599]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:48.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.902+0000 7f6cd8db2640 1 -- 172.21.15.100:0/2116475283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cd4107df0 msgr2=0x7f6cd4108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:48.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.902+0000 7f6cd8db2640 1 --2- 172.21.15.100:0/2116475283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cd4107df0 0x7f6cd4108200 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8009900 tx=0x7f6cc802a410 comp rx=0 tx=0).stop 2024-07-26T13:36:48.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.902+0000 7f6cd8db2640 1 -- 172.21.15.100:0/2116475283 shutdown_connections 2024-07-26T13:36:48.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.902+0000 7f6cd8db2640 1 --2- 172.21.15.100:0/2116475283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cd4108740 0x7f6cd410ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:48.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.902+0000 7f6cd8db2640 1 --2- 172.21.15.100:0/2116475283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cd4107df0 0x7f6cd4108200 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:48.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.902+0000 7f6cd8db2640 1 --2- 172.21.15.100:0/2116475283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cd41024b0 0x7f6cd41028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:48.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.902+0000 7f6cd8db2640 1 -- 172.21.15.100:0/2116475283 >> 172.21.15.100:0/2116475283 conn(0x7f6cd4079170 msgr2=0x7f6cd4079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:36:48.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.902+0000 7f6cd8db2640 1 -- 172.21.15.100:0/2116475283 shutdown_connections 2024-07-26T13:36:48.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.902+0000 7f6cd8db2640 1 -- 172.21.15.100:0/2116475283 wait complete. 2024-07-26T13:36:48.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.903+0000 7f6cd8db2640 1 Processor -- start 2024-07-26T13:36:48.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.903+0000 7f6cd8db2640 1 -- start start 2024-07-26T13:36:48.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.904+0000 7f6cd8db2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cd41024b0 0x7f6cd419aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:48.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.904+0000 7f6cd8db2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cd4107df0 0x7f6cd419b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:48.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.904+0000 7f6cd8db2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cd4108740 0x7f6cd419b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:48.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.904+0000 7f6cd8db2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd4112390 con 0x7f6cd4108740 2024-07-26T13:36:48.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.904+0000 7f6cd8db2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd4112210 con 0x7f6cd41024b0 2024-07-26T13:36:48.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.904+0000 7f6cd8db2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd4112510 con 0x7f6cd4107df0 2024-07-26T13:36:48.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.904+0000 7f6cd2d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cd4108740 0x7f6cd419b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:48.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.904+0000 7f6cd2d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cd4108740 0x7f6cd419b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41208/0 (socket says 172.21.15.100:41208) 2024-07-26T13:36:48.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.904+0000 7f6cd2d9d640 1 -- 172.21.15.100:0/3329812176 learned_addr learned my addr 172.21.15.100:0/3329812176 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:36:48.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.904+0000 7f6cd1d9b640 1 --2- 172.21.15.100:0/3329812176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cd4107df0 0x7f6cd419b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:48.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.904+0000 7f6cd259c640 1 --2- 172.21.15.100:0/3329812176 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cd41024b0 0x7f6cd419aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:48.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.904+0000 7f6cd2d9d640 1 -- 172.21.15.100:0/3329812176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cd4107df0 msgr2=0x7f6cd419b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:48.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.904+0000 7f6cd2d9d640 1 --2- 172.21.15.100:0/3329812176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cd4107df0 0x7f6cd419b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:48.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.904+0000 7f6cd2d9d640 1 -- 172.21.15.100:0/3329812176 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cd41024b0 msgr2=0x7f6cd419aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:48.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.904+0000 7f6cd2d9d640 1 --2- 172.21.15.100:0/3329812176 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cd41024b0 0x7f6cd419aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:48.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.904+0000 7f6cd2d9d640 1 -- 172.21.15.100:0/3329812176 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd41a6130 con 0x7f6cd4108740 2024-07-26T13:36:48.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.904+0000 7f6cd259c640 1 --2- 172.21.15.100:0/3329812176 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cd41024b0 0x7f6cd419aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:36:48.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.904+0000 7f6cd2d9d640 1 --2- 172.21.15.100:0/3329812176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cd4108740 0x7f6cd419b920 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc00eb10 tx=0x7f6ccc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:36:48.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.904+0000 7f6cc37fe640 1 -- 172.21.15.100:0/3329812176 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc004b20 con 0x7f6cd4108740 2024-07-26T13:36:48.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.904+0000 7f6cd8db2640 1 -- 172.21.15.100:0/3329812176 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cd41a6420 con 0x7f6cd4108740 2024-07-26T13:36:48.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.904+0000 7f6cc37fe640 1 -- 172.21.15.100:0/3329812176 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ccc004cc0 con 0x7f6cd4108740 2024-07-26T13:36:48.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.904+0000 7f6cc37fe640 1 -- 172.21.15.100:0/3329812176 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc010690 con 0x7f6cd4108740 2024-07-26T13:36:48.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.905+0000 7f6cd8db2640 1 -- 172.21.15.100:0/3329812176 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cd41a6980 con 0x7f6cd4108740 2024-07-26T13:36:48.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.906+0000 7f6cd8db2640 1 -- 172.21.15.100:0/3329812176 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cd4103730 con 0x7f6cd4108740 2024-07-26T13:36:48.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.907+0000 7f6cc37fe640 1 -- 172.21.15.100:0/3329812176 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f6ccc005000 con 0x7f6cd4108740 2024-07-26T13:36:48.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.908+0000 7f6cc37fe640 1 --2- 172.21.15.100:0/3329812176 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6cbc0b3c50 0x7f6cbc0b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:48.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.908+0000 7f6cd259c640 1 --2- 172.21.15.100:0/3329812176 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6cbc0b3c50 0x7f6cbc0b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:48.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.908+0000 7f6cc37fe640 1 -- 172.21.15.100:0/3329812176 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f6ccc0e3fd0 con 0x7f6cd4108740 2024-07-26T13:36:48.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.908+0000 7f6cd259c640 1 --2- 172.21.15.100:0/3329812176 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6cbc0b3c50 0x7f6cbc0b6160 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6cd406c0d0 tx=0x7f6cc4005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:36:48.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:48.911+0000 7f6cc37fe640 1 -- 172.21.15.100:0/3329812176 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f6ccc055600 con 0x7f6cd4108740 2024-07-26T13:36:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:48 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:48 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:48 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:48 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:48.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:48 smithi138 ceph-mon[36954]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:48 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:48 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:48 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:48 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:48 smithi167 ceph-mon[37140]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:49.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:49.146+0000 7f6cd8db2640 1 -- 172.21.15.100:0/3329812176 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6cd41028c0 con 0x7f6cd4108740 2024-07-26T13:36:49.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:49.147+0000 7f6cc37fe640 1 -- 172.21.15.100:0/3329812176 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10539 (secure 0 0 0) 0x7f6cd41028c0 con 0x7f6cd4108740 2024-07-26T13:36:49.148 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:36:49.148 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":13,"btime":"2024-07-26T13:36:44:601534+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14790,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/313501409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":313501409},{"type":"v1","addr":"172.21.15.100:6839","nonce":313501409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/1402975506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":1402975506},{"type":"v1","addr":"172.21.15.100:6841","nonce":1402975506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24557,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/3290105300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3290105300},{"type":"v1","addr":"172.21.15.138:6835","nonce":3290105300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24575,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3393225483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3393225483},{"type":"v1","addr":"172.21.15.138:6837","nonce":3393225483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24586,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24595,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3828634468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3828634468},{"type":"v1","addr":"172.21.15.167:6837","nonce":3828634468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24605,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6839/1580925471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1580925471},{"type":"v1","addr":"172.21.15.138:6839","nonce":1580925471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24607,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/398150514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":398150514},{"type":"v1","addr":"172.21.15.167:6839","nonce":398150514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-07-26T13:36:49.148 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 13 2024-07-26T13:36:49.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:49.149+0000 7f6cd8db2640 1 -- 172.21.15.100:0/3329812176 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6cbc0b3c50 msgr2=0x7f6cbc0b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:49.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:49.149+0000 7f6cd8db2640 1 --2- 172.21.15.100:0/3329812176 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6cbc0b3c50 0x7f6cbc0b6160 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6cd406c0d0 tx=0x7f6cc4005f90 comp rx=0 tx=0).stop 2024-07-26T13:36:49.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:49.149+0000 7f6cd8db2640 1 -- 172.21.15.100:0/3329812176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cd4108740 msgr2=0x7f6cd419b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:49.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:49.149+0000 7f6cd8db2640 1 --2- 172.21.15.100:0/3329812176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cd4108740 0x7f6cd419b920 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc00eb10 tx=0x7f6ccc00c2d0 comp rx=0 tx=0).stop 2024-07-26T13:36:49.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:49.149+0000 7f6cd8db2640 1 -- 172.21.15.100:0/3329812176 shutdown_connections 2024-07-26T13:36:49.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:49.149+0000 7f6cd8db2640 1 --2- 172.21.15.100:0/3329812176 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6cbc0b3c50 0x7f6cbc0b6160 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:49.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:49.149+0000 7f6cd8db2640 1 --2- 172.21.15.100:0/3329812176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cd4108740 0x7f6cd419b920 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:49.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:49.149+0000 7f6cd8db2640 1 --2- 172.21.15.100:0/3329812176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cd4107df0 0x7f6cd419b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:49.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:49.149+0000 7f6cd8db2640 1 --2- 172.21.15.100:0/3329812176 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cd41024b0 0x7f6cd419aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:49.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:49.149+0000 7f6cd8db2640 1 -- 172.21.15.100:0/3329812176 >> 172.21.15.100:0/3329812176 conn(0x7f6cd4079170 msgr2=0x7f6cd4104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:36:49.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:49.149+0000 7f6cd8db2640 1 -- 172.21.15.100:0/3329812176 shutdown_connections 2024-07-26T13:36:49.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:49.149+0000 7f6cd8db2640 1 -- 172.21.15.100:0/3329812176 wait complete. 2024-07-26T13:36:49.803 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:49 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/3329812176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:36:49.803 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:49 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:49.803 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:49 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:49.803 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:49 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:49.803 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:49 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:36:49.803 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:49 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:49.831 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-07-26T13:36:49.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:49 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/3329812176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:36:49.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:49 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:49.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:49 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:49.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:49 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:49.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:49 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:36:49.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:49 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:50.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:49 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/3329812176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:36:50.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:49 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:50.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:49 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:50.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:49 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:36:50.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:49 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:36:50.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:49 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:36:50.085 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:36:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:50 smithi138 ceph-mon[36954]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:50.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:50 smithi100 ceph-mon[32599]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:50 smithi167 ceph-mon[37140]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:51.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.687+0000 7f3babff1640 1 -- 172.21.15.100:0/3960344076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ba4102490 msgr2=0x7f3ba41028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:51.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.687+0000 7f3babff1640 1 --2- 172.21.15.100:0/3960344076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ba4102490 0x7f3ba41028a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c009900 tx=0x7f3b9c02a410 comp rx=0 tx=0).stop 2024-07-26T13:36:51.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.687+0000 7f3babff1640 1 -- 172.21.15.100:0/3960344076 shutdown_connections 2024-07-26T13:36:51.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.687+0000 7f3babff1640 1 --2- 172.21.15.100:0/3960344076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ba4108720 0x7f3ba410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:51.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.687+0000 7f3babff1640 1 --2- 172.21.15.100:0/3960344076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ba4107dd0 0x7f3ba41081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:51.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.687+0000 7f3babff1640 1 --2- 172.21.15.100:0/3960344076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ba4102490 0x7f3ba41028a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:51.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.687+0000 7f3babff1640 1 -- 172.21.15.100:0/3960344076 >> 172.21.15.100:0/3960344076 conn(0x7f3ba4079160 msgr2=0x7f3ba4079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:36:51.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.687+0000 7f3babff1640 1 -- 172.21.15.100:0/3960344076 shutdown_connections 2024-07-26T13:36:51.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.687+0000 7f3babff1640 1 -- 172.21.15.100:0/3960344076 wait complete. 2024-07-26T13:36:51.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.688+0000 7f3babff1640 1 Processor -- start 2024-07-26T13:36:51.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.688+0000 7f3babff1640 1 -- start start 2024-07-26T13:36:51.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.688+0000 7f3babff1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ba4102490 0x7f3ba419acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:51.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.688+0000 7f3babff1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ba4107dd0 0x7f3ba419b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:51.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.688+0000 7f3babff1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ba4108720 0x7f3ba419b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:51.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.688+0000 7f3babff1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba4112380 con 0x7f3ba4107dd0 2024-07-26T13:36:51.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.688+0000 7f3babff1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba4112200 con 0x7f3ba4108720 2024-07-26T13:36:51.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.688+0000 7f3babff1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba4112500 con 0x7f3ba4102490 2024-07-26T13:36:51.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.688+0000 7f3ba958c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ba4107dd0 0x7f3ba419b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:51.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.688+0000 7f3ba958c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ba4107dd0 0x7f3ba419b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41226/0 (socket says 172.21.15.100:41226) 2024-07-26T13:36:51.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.688+0000 7f3ba958c640 1 -- 172.21.15.100:0/2609388757 learned_addr learned my addr 172.21.15.100:0/2609388757 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:36:51.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.688+0000 7f3baa58e640 1 --2- 172.21.15.100:0/2609388757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ba4108720 0x7f3ba419b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:51.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.688+0000 7f3ba9d8d640 1 --2- 172.21.15.100:0/2609388757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ba4102490 0x7f3ba419acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:51.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.688+0000 7f3ba958c640 1 -- 172.21.15.100:0/2609388757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ba4102490 msgr2=0x7f3ba419acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:51.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.688+0000 7f3ba958c640 1 --2- 172.21.15.100:0/2609388757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ba4102490 0x7f3ba419acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:51.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.688+0000 7f3ba958c640 1 -- 172.21.15.100:0/2609388757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ba4108720 msgr2=0x7f3ba419b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:51.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.688+0000 7f3ba958c640 1 --2- 172.21.15.100:0/2609388757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ba4108720 0x7f3ba419b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:51.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.688+0000 7f3ba958c640 1 -- 172.21.15.100:0/2609388757 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ba41a6000 con 0x7f3ba4107dd0 2024-07-26T13:36:51.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.689+0000 7f3ba958c640 1 --2- 172.21.15.100:0/2609388757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ba4107dd0 0x7f3ba419b230 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3b9800ea80 tx=0x7f3b9800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:36:51.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.689+0000 7f3ba2ffd640 1 -- 172.21.15.100:0/2609388757 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9800ce40 con 0x7f3ba4107dd0 2024-07-26T13:36:51.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.689+0000 7f3ba2ffd640 1 -- 172.21.15.100:0/2609388757 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b98011920 con 0x7f3ba4107dd0 2024-07-26T13:36:51.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.689+0000 7f3babff1640 1 -- 172.21.15.100:0/2609388757 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ba41a62f0 con 0x7f3ba4107dd0 2024-07-26T13:36:51.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.689+0000 7f3ba2ffd640 1 -- 172.21.15.100:0/2609388757 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9801aa20 con 0x7f3ba4107dd0 2024-07-26T13:36:51.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.689+0000 7f3babff1640 1 -- 172.21.15.100:0/2609388757 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ba41a6830 con 0x7f3ba4107dd0 2024-07-26T13:36:51.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.690+0000 7f3babff1640 1 -- 172.21.15.100:0/2609388757 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ba41028a0 con 0x7f3ba4107dd0 2024-07-26T13:36:51.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.691+0000 7f3ba2ffd640 1 -- 172.21.15.100:0/2609388757 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f3b9801abc0 con 0x7f3ba4107dd0 2024-07-26T13:36:51.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.692+0000 7f3ba2ffd640 1 --2- 172.21.15.100:0/2609388757 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3b8c0b3c20 0x7f3b8c0b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:51.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.692+0000 7f3ba9d8d640 1 --2- 172.21.15.100:0/2609388757 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3b8c0b3c20 0x7f3b8c0b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:51.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.692+0000 7f3ba2ffd640 1 -- 172.21.15.100:0/2609388757 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f3b980e40a0 con 0x7f3ba4107dd0 2024-07-26T13:36:51.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.693+0000 7f3ba9d8d640 1 --2- 172.21.15.100:0/2609388757 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3b8c0b3c20 0x7f3b8c0b6130 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c000c00 tx=0x7f3b9c031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:36:51.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.695+0000 7f3ba2ffd640 1 -- 172.21.15.100:0/2609388757 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f3b98057600 con 0x7f3ba4107dd0 2024-07-26T13:36:51.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.870+0000 7f3babff1640 1 -- 172.21.15.100:0/2609388757 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f3ba406ad00 con 0x7f3ba4107dd0 2024-07-26T13:36:51.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.871+0000 7f3ba2ffd640 1 -- 172.21.15.100:0/2609388757 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v72) ==== 84+0+65 (secure 0 0 0) 0x7f3ba406ad00 con 0x7f3ba4107dd0 2024-07-26T13:36:51.872 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:36:51.872 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-07-26T13:36:51.872 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-07-26T13:36:51.872 INFO:teuthology.orchestra.run.smithi100.stdout: "poolnum": 1, 2024-07-26T13:36:51.873 INFO:teuthology.orchestra.run.smithi100.stdout: "poolname": ".mgr" 2024-07-26T13:36:51.873 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:36:51.873 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-07-26T13:36:51.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.873+0000 7f3babff1640 1 -- 172.21.15.100:0/2609388757 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3b8c0b3c20 msgr2=0x7f3b8c0b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:51.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.873+0000 7f3babff1640 1 --2- 172.21.15.100:0/2609388757 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3b8c0b3c20 0x7f3b8c0b6130 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c000c00 tx=0x7f3b9c031400 comp rx=0 tx=0).stop 2024-07-26T13:36:51.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.873+0000 7f3babff1640 1 -- 172.21.15.100:0/2609388757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ba4107dd0 msgr2=0x7f3ba419b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:51.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.873+0000 7f3babff1640 1 --2- 172.21.15.100:0/2609388757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ba4107dd0 0x7f3ba419b230 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3b9800ea80 tx=0x7f3b9800ef40 comp rx=0 tx=0).stop 2024-07-26T13:36:51.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.873+0000 7f3babff1640 1 -- 172.21.15.100:0/2609388757 shutdown_connections 2024-07-26T13:36:51.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.873+0000 7f3babff1640 1 --2- 172.21.15.100:0/2609388757 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3b8c0b3c20 0x7f3b8c0b6130 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:51.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.873+0000 7f3babff1640 1 --2- 172.21.15.100:0/2609388757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ba4108720 0x7f3ba419b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:51.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.873+0000 7f3babff1640 1 --2- 172.21.15.100:0/2609388757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ba4107dd0 0x7f3ba419b230 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:51.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.873+0000 7f3babff1640 1 --2- 172.21.15.100:0/2609388757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ba4102490 0x7f3ba419acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:51.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.873+0000 7f3babff1640 1 -- 172.21.15.100:0/2609388757 >> 172.21.15.100:0/2609388757 conn(0x7f3ba4079160 msgr2=0x7f3ba4100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:36:51.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.873+0000 7f3babff1640 1 -- 172.21.15.100:0/2609388757 shutdown_connections 2024-07-26T13:36:51.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:51.873+0000 7f3babff1640 1 -- 172.21.15.100:0/2609388757 wait complete. 2024-07-26T13:36:52.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:51 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/2609388757' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-07-26T13:36:52.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:51 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2609388757' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-07-26T13:36:52.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:51 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/2609388757' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-07-26T13:36:52.552 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-07-26T13:36:52.552 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-07-26T13:36:52.807 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:36:53.183 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:52 smithi100 ceph-mon[32599]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:52 smithi138 ceph-mon[36954]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:53.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:52 smithi167 ceph-mon[37140]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:54.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.402+0000 7f972a1a9640 1 -- 172.21.15.100:0/1291381028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9724107df0 msgr2=0x7f9724108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:54.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.402+0000 7f972a1a9640 1 --2- 172.21.15.100:0/1291381028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9724107df0 0x7f9724108200 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9718009900 tx=0x7f971802a410 comp rx=0 tx=0).stop 2024-07-26T13:36:54.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.402+0000 7f972a1a9640 1 -- 172.21.15.100:0/1291381028 shutdown_connections 2024-07-26T13:36:54.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.402+0000 7f972a1a9640 1 --2- 172.21.15.100:0/1291381028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9724108740 0x7f972410ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:54.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.402+0000 7f972a1a9640 1 --2- 172.21.15.100:0/1291381028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9724107df0 0x7f9724108200 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:54.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.402+0000 7f972a1a9640 1 --2- 172.21.15.100:0/1291381028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97241024b0 0x7f97241028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:54.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.402+0000 7f972a1a9640 1 -- 172.21.15.100:0/1291381028 >> 172.21.15.100:0/1291381028 conn(0x7f9724079170 msgr2=0x7f9724079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:36:54.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.402+0000 7f972a1a9640 1 -- 172.21.15.100:0/1291381028 shutdown_connections 2024-07-26T13:36:54.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.402+0000 7f972a1a9640 1 -- 172.21.15.100:0/1291381028 wait complete. 2024-07-26T13:36:54.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.403+0000 7f972a1a9640 1 Processor -- start 2024-07-26T13:36:54.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.403+0000 7f972a1a9640 1 -- start start 2024-07-26T13:36:54.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.403+0000 7f972a1a9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97241024b0 0x7f9724199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:54.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.403+0000 7f972a1a9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9724107df0 0x7f972419a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:54.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.403+0000 7f972a1a9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9724108740 0x7f972419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:54.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.403+0000 7f972a1a9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9724113510 con 0x7f97241024b0 2024-07-26T13:36:54.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.403+0000 7f972a1a9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9724113390 con 0x7f9724108740 2024-07-26T13:36:54.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.403+0000 7f972a1a9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9724113690 con 0x7f9724107df0 2024-07-26T13:36:54.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.403+0000 7f97237fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97241024b0 0x7f9724199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:54.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.403+0000 7f97237fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97241024b0 0x7f9724199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41250/0 (socket says 172.21.15.100:41250) 2024-07-26T13:36:54.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.403+0000 7f97237fe640 1 -- 172.21.15.100:0/3160402851 learned_addr learned my addr 172.21.15.100:0/3160402851 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:36:54.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.403+0000 7f9722ffd640 1 --2- 172.21.15.100:0/3160402851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9724107df0 0x7f972419a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:54.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.403+0000 7f9723fff640 1 --2- 172.21.15.100:0/3160402851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9724108740 0x7f972419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:54.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.403+0000 7f97237fe640 1 -- 172.21.15.100:0/3160402851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9724107df0 msgr2=0x7f972419a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:54.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.403+0000 7f97237fe640 1 --2- 172.21.15.100:0/3160402851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9724107df0 0x7f972419a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:54.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.403+0000 7f97237fe640 1 -- 172.21.15.100:0/3160402851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9724108740 msgr2=0x7f972419f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:54.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.403+0000 7f97237fe640 1 --2- 172.21.15.100:0/3160402851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9724108740 0x7f972419f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:54.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.403+0000 7f97237fe640 1 -- 172.21.15.100:0/3160402851 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f972419ff30 con 0x7f97241024b0 2024-07-26T13:36:54.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.404+0000 7f97237fe640 1 --2- 172.21.15.100:0/3160402851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97241024b0 0x7f9724199fc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f971400d770 tx=0x7f971400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:36:54.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.404+0000 7f9720ff9640 1 -- 172.21.15.100:0/3160402851 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9714017070 con 0x7f97241024b0 2024-07-26T13:36:54.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.404+0000 7f972a1a9640 1 -- 172.21.15.100:0/3160402851 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97241a0220 con 0x7f97241024b0 2024-07-26T13:36:54.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.404+0000 7f9720ff9640 1 -- 172.21.15.100:0/3160402851 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97140054d0 con 0x7f97241024b0 2024-07-26T13:36:54.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.404+0000 7f972a1a9640 1 -- 172.21.15.100:0/3160402851 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97241a6980 con 0x7f97241024b0 2024-07-26T13:36:54.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.404+0000 7f9720ff9640 1 -- 172.21.15.100:0/3160402851 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9714004c30 con 0x7f97241024b0 2024-07-26T13:36:54.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.405+0000 7f972a1a9640 1 -- 172.21.15.100:0/3160402851 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9724103730 con 0x7f97241024b0 2024-07-26T13:36:54.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.406+0000 7f9720ff9640 1 -- 172.21.15.100:0/3160402851 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f9714004dd0 con 0x7f97241024b0 2024-07-26T13:36:54.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.407+0000 7f9720ff9640 1 --2- 172.21.15.100:0/3160402851 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f970c0b3c20 0x7f970c0b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:54.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.407+0000 7f9722ffd640 1 --2- 172.21.15.100:0/3160402851 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f970c0b3c20 0x7f970c0b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:54.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.407+0000 7f9720ff9640 1 -- 172.21.15.100:0/3160402851 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f97140e3730 con 0x7f97241024b0 2024-07-26T13:36:54.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.408+0000 7f9722ffd640 1 --2- 172.21.15.100:0/3160402851 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f970c0b3c20 0x7f970c0b6130 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9718000c00 tx=0x7f9718031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:36:54.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.410+0000 7f9720ff9640 1 -- 172.21.15.100:0/3160402851 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f97140ab150 con 0x7f97241024b0 2024-07-26T13:36:54.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:54.585+0000 7f972a1a9640 1 -- 172.21.15.100:0/3160402851 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f97241028c0 con 0x7f97241024b0 2024-07-26T13:36:55.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:55.414+0000 7f9720ff9640 1 -- 172.21.15.100:0/3160402851 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v73) ==== 138+0+0 (secure 0 0 0) 0x7f97241028c0 con 0x7f97241024b0 2024-07-26T13:36:55.415 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' created 2024-07-26T13:36:55.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:55.417+0000 7f972a1a9640 1 -- 172.21.15.100:0/3160402851 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f970c0b3c20 msgr2=0x7f970c0b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:55.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:55.417+0000 7f972a1a9640 1 --2- 172.21.15.100:0/3160402851 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f970c0b3c20 0x7f970c0b6130 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9718000c00 tx=0x7f9718031400 comp rx=0 tx=0).stop 2024-07-26T13:36:55.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:55.417+0000 7f972a1a9640 1 -- 172.21.15.100:0/3160402851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97241024b0 msgr2=0x7f9724199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:55.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:55.417+0000 7f972a1a9640 1 --2- 172.21.15.100:0/3160402851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97241024b0 0x7f9724199fc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f971400d770 tx=0x7f971400dc30 comp rx=0 tx=0).stop 2024-07-26T13:36:55.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:55.417+0000 7f972a1a9640 1 -- 172.21.15.100:0/3160402851 shutdown_connections 2024-07-26T13:36:55.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:55.417+0000 7f972a1a9640 1 --2- 172.21.15.100:0/3160402851 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f970c0b3c20 0x7f970c0b6130 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:55.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:55.417+0000 7f972a1a9640 1 --2- 172.21.15.100:0/3160402851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9724108740 0x7f972419f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:55.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:55.417+0000 7f972a1a9640 1 --2- 172.21.15.100:0/3160402851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9724107df0 0x7f972419a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:55.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:55.417+0000 7f972a1a9640 1 --2- 172.21.15.100:0/3160402851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97241024b0 0x7f9724199fc0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:55.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:55.417+0000 7f972a1a9640 1 -- 172.21.15.100:0/3160402851 >> 172.21.15.100:0/3160402851 conn(0x7f9724079170 msgr2=0x7f9724103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:36:55.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:55.417+0000 7f972a1a9640 1 -- 172.21.15.100:0/3160402851 shutdown_connections 2024-07-26T13:36:55.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:55.418+0000 7f972a1a9640 1 -- 172.21.15.100:0/3160402851 wait complete. 2024-07-26T13:36:55.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:55 smithi138 ceph-mon[36954]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:55.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:55 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/3160402851' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-07-26T13:36:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:55 smithi100 ceph-mon[32599]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:55 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/3160402851' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-07-26T13:36:55.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:55 smithi167 ceph-mon[37140]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:55.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:55 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/3160402851' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-07-26T13:36:56.081 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-07-26T13:36:56.337 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:36:56.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:56 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/3160402851' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-07-26T13:36:56.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:56 smithi138 ceph-mon[36954]: osdmap e73: 12 total, 12 up, 12 in 2024-07-26T13:36:56.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:56 smithi138 ceph-mon[36954]: osdmap e74: 12 total, 12 up, 12 in 2024-07-26T13:36:56.708 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:56 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/3160402851' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-07-26T13:36:56.708 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:56 smithi100 ceph-mon[32599]: osdmap e73: 12 total, 12 up, 12 in 2024-07-26T13:36:56.708 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:56 smithi100 ceph-mon[32599]: osdmap e74: 12 total, 12 up, 12 in 2024-07-26T13:36:56.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:56 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/3160402851' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-07-26T13:36:56.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:56 smithi167 ceph-mon[37140]: osdmap e73: 12 total, 12 up, 12 in 2024-07-26T13:36:56.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:56 smithi167 ceph-mon[37140]: osdmap e74: 12 total, 12 up, 12 in 2024-07-26T13:36:57.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:57 smithi100 ceph-mon[32599]: pgmap v261: 33 pgs: 1 creating+peering, 31 unknown, 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:57.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:57 smithi100 ceph-mon[32599]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-07-26T13:36:57.455 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:57 smithi100 ceph-mon[32599]: osdmap e75: 12 total, 12 up, 12 in 2024-07-26T13:36:57.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:57 smithi138 ceph-mon[36954]: pgmap v261: 33 pgs: 1 creating+peering, 31 unknown, 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:57.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:57 smithi138 ceph-mon[36954]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-07-26T13:36:57.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:57 smithi138 ceph-mon[36954]: osdmap e75: 12 total, 12 up, 12 in 2024-07-26T13:36:57.764 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:57 smithi167 ceph-mon[37140]: pgmap v261: 33 pgs: 1 creating+peering, 31 unknown, 1 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:57.765 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:57 smithi167 ceph-mon[37140]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-07-26T13:36:57.765 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:57 smithi167 ceph-mon[37140]: osdmap e75: 12 total, 12 up, 12 in 2024-07-26T13:36:57.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.963+0000 7f8e81395640 1 -- 172.21.15.100:0/1425909947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e7c102490 msgr2=0x7f8e7c1028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:57.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.963+0000 7f8e81395640 1 --2- 172.21.15.100:0/1425909947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e7c102490 0x7f8e7c1028a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c009900 tx=0x7f8e6c02a410 comp rx=0 tx=0).stop 2024-07-26T13:36:57.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.963+0000 7f8e81395640 1 -- 172.21.15.100:0/1425909947 shutdown_connections 2024-07-26T13:36:57.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.963+0000 7f8e81395640 1 --2- 172.21.15.100:0/1425909947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e7c108720 0x7f8e7c10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:57.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.963+0000 7f8e81395640 1 --2- 172.21.15.100:0/1425909947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e7c107dd0 0x7f8e7c1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:57.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.963+0000 7f8e81395640 1 --2- 172.21.15.100:0/1425909947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e7c102490 0x7f8e7c1028a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:57.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.963+0000 7f8e81395640 1 -- 172.21.15.100:0/1425909947 >> 172.21.15.100:0/1425909947 conn(0x7f8e7c079160 msgr2=0x7f8e7c079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:36:57.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.963+0000 7f8e81395640 1 -- 172.21.15.100:0/1425909947 shutdown_connections 2024-07-26T13:36:57.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.963+0000 7f8e81395640 1 -- 172.21.15.100:0/1425909947 wait complete. 2024-07-26T13:36:57.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.964+0000 7f8e81395640 1 Processor -- start 2024-07-26T13:36:57.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.964+0000 7f8e81395640 1 -- start start 2024-07-26T13:36:57.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.964+0000 7f8e81395640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e7c102490 0x7f8e7c19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:57.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.964+0000 7f8e81395640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e7c107dd0 0x7f8e7c19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:57.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.964+0000 7f8e81395640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e7c108720 0x7f8e7c19b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:57.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.964+0000 7f8e81395640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7c112400 con 0x7f8e7c102490 2024-07-26T13:36:57.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.964+0000 7f8e81395640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7c112280 con 0x7f8e7c107dd0 2024-07-26T13:36:57.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.964+0000 7f8e81395640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7c112580 con 0x7f8e7c108720 2024-07-26T13:36:57.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.964+0000 7f8e7affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e7c102490 0x7f8e7c19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:57.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.964+0000 7f8e7affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e7c102490 0x7f8e7c19adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42286/0 (socket says 172.21.15.100:42286) 2024-07-26T13:36:57.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.964+0000 7f8e7affd640 1 -- 172.21.15.100:0/3510707346 learned_addr learned my addr 172.21.15.100:0/3510707346 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:36:57.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.965+0000 7f8e7a7fc640 1 --2- 172.21.15.100:0/3510707346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e7c107dd0 0x7f8e7c19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:57.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.965+0000 7f8e7b7fe640 1 --2- 172.21.15.100:0/3510707346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e7c108720 0x7f8e7c19b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:57.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.965+0000 7f8e7affd640 1 -- 172.21.15.100:0/3510707346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e7c108720 msgr2=0x7f8e7c19b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:57.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.965+0000 7f8e7affd640 1 --2- 172.21.15.100:0/3510707346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e7c108720 0x7f8e7c19b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:57.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.965+0000 7f8e7affd640 1 -- 172.21.15.100:0/3510707346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e7c107dd0 msgr2=0x7f8e7c19b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:57.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.965+0000 7f8e7affd640 1 --2- 172.21.15.100:0/3510707346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e7c107dd0 0x7f8e7c19b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:57.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.965+0000 7f8e7affd640 1 -- 172.21.15.100:0/3510707346 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e7c1a6080 con 0x7f8e7c102490 2024-07-26T13:36:57.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.965+0000 7f8e7affd640 1 --2- 172.21.15.100:0/3510707346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e7c102490 0x7f8e7c19adf0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c006010 tx=0x7f8e6c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:36:57.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.965+0000 7f8e63fff640 1 -- 172.21.15.100:0/3510707346 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c03f030 con 0x7f8e7c102490 2024-07-26T13:36:57.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.965+0000 7f8e81395640 1 -- 172.21.15.100:0/3510707346 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e7c1a6310 con 0x7f8e7c102490 2024-07-26T13:36:57.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.965+0000 7f8e63fff640 1 -- 172.21.15.100:0/3510707346 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e6c003800 con 0x7f8e7c102490 2024-07-26T13:36:57.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.965+0000 7f8e63fff640 1 -- 172.21.15.100:0/3510707346 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c035070 con 0x7f8e7c102490 2024-07-26T13:36:57.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.965+0000 7f8e81395640 1 -- 172.21.15.100:0/3510707346 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e7c1a67f0 con 0x7f8e7c102490 2024-07-26T13:36:57.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.966+0000 7f8e81395640 1 -- 172.21.15.100:0/3510707346 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e7c1028a0 con 0x7f8e7c102490 2024-07-26T13:36:57.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.968+0000 7f8e63fff640 1 -- 172.21.15.100:0/3510707346 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f8e6c003cb0 con 0x7f8e7c102490 2024-07-26T13:36:57.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.969+0000 7f8e63fff640 1 --2- 172.21.15.100:0/3510707346 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f8e640b3c50 0x7f8e640b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:36:57.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.969+0000 7f8e7a7fc640 1 --2- 172.21.15.100:0/3510707346 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f8e640b3c50 0x7f8e640b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:36:57.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.969+0000 7f8e63fff640 1 -- 172.21.15.100:0/3510707346 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7712+0+0 (secure 0 0 0) 0x7f8e6c1045f0 con 0x7f8e7c102490 2024-07-26T13:36:57.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.969+0000 7f8e7a7fc640 1 --2- 172.21.15.100:0/3510707346 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f8e640b3c50 0x7f8e640b6160 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c100680 tx=0x7f8e7000a2d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:36:57.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:57.972+0000 7f8e63fff640 1 -- 172.21.15.100:0/3510707346 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f8e6c0cbe20 con 0x7f8e7c102490 2024-07-26T13:36:58.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:58.144+0000 7f8e81395640 1 -- 172.21.15.100:0/3510707346 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f8e7c06aca0 con 0x7f8e7c102490 2024-07-26T13:36:58.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:58.438+0000 7f8e63fff640 1 -- 172.21.15.100:0/3510707346 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v76) ==== 170+0+0 (secure 0 0 0) 0x7f8e6c0cb720 con 0x7f8e7c102490 2024-07-26T13:36:58.440 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' created 2024-07-26T13:36:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:58.441+0000 7f8e81395640 1 -- 172.21.15.100:0/3510707346 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f8e640b3c50 msgr2=0x7f8e640b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:58.441+0000 7f8e81395640 1 --2- 172.21.15.100:0/3510707346 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f8e640b3c50 0x7f8e640b6160 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c100680 tx=0x7f8e7000a2d0 comp rx=0 tx=0).stop 2024-07-26T13:36:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:58.441+0000 7f8e81395640 1 -- 172.21.15.100:0/3510707346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e7c102490 msgr2=0x7f8e7c19adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:36:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:58.441+0000 7f8e81395640 1 --2- 172.21.15.100:0/3510707346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e7c102490 0x7f8e7c19adf0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c006010 tx=0x7f8e6c0042f0 comp rx=0 tx=0).stop 2024-07-26T13:36:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:58.442+0000 7f8e81395640 1 -- 172.21.15.100:0/3510707346 shutdown_connections 2024-07-26T13:36:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:58.442+0000 7f8e81395640 1 --2- 172.21.15.100:0/3510707346 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f8e640b3c50 0x7f8e640b6160 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:58.442+0000 7f8e81395640 1 --2- 172.21.15.100:0/3510707346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e7c108720 0x7f8e7c19b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:58.442+0000 7f8e81395640 1 --2- 172.21.15.100:0/3510707346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e7c107dd0 0x7f8e7c19b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:58.442+0000 7f8e81395640 1 --2- 172.21.15.100:0/3510707346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e7c102490 0x7f8e7c19adf0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:36:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:58.442+0000 7f8e81395640 1 -- 172.21.15.100:0/3510707346 >> 172.21.15.100:0/3510707346 conn(0x7f8e7c079160 msgr2=0x7f8e7c100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:36:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:58.442+0000 7f8e81395640 1 -- 172.21.15.100:0/3510707346 shutdown_connections 2024-07-26T13:36:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:36:58.442+0000 7f8e81395640 1 -- 172.21.15.100:0/3510707346 wait complete. 2024-07-26T13:36:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:58 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/3510707346' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-07-26T13:36:58.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:58 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/3510707346' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-07-26T13:36:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:58 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/3510707346' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-07-26T13:36:59.177 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-07-26T13:36:59.441 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:36:59.461 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:59 smithi100 ceph-mon[32599]: pgmap v264: 33 pgs: 1 creating+peering, 20 unknown, 12 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:59.461 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:59 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/3510707346' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-07-26T13:36:59.461 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:36:59 smithi100 ceph-mon[32599]: osdmap e76: 12 total, 12 up, 12 in 2024-07-26T13:36:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:59 smithi167 ceph-mon[37140]: pgmap v264: 33 pgs: 1 creating+peering, 20 unknown, 12 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:59 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/3510707346' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-07-26T13:36:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:36:59 smithi167 ceph-mon[37140]: osdmap e76: 12 total, 12 up, 12 in 2024-07-26T13:36:59.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:59 smithi138 ceph-mon[36954]: pgmap v264: 33 pgs: 1 creating+peering, 20 unknown, 12 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:36:59.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:59 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/3510707346' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-07-26T13:36:59.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:36:59 smithi138 ceph-mon[36954]: osdmap e76: 12 total, 12 up, 12 in 2024-07-26T13:37:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:00 smithi100 ceph-mon[32599]: osdmap e77: 12 total, 12 up, 12 in 2024-07-26T13:37:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:00 smithi100 ceph-mon[32599]: osdmap e78: 12 total, 12 up, 12 in 2024-07-26T13:37:00.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:00 smithi167 ceph-mon[37140]: osdmap e77: 12 total, 12 up, 12 in 2024-07-26T13:37:00.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:00 smithi167 ceph-mon[37140]: osdmap e78: 12 total, 12 up, 12 in 2024-07-26T13:37:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:00 smithi138 ceph-mon[36954]: osdmap e77: 12 total, 12 up, 12 in 2024-07-26T13:37:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:00 smithi138 ceph-mon[36954]: osdmap e78: 12 total, 12 up, 12 in 2024-07-26T13:37:01.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.062+0000 7fd80035f640 1 -- 172.21.15.100:0/360332793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7f8108730 msgr2=0x7fd7f810ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:01.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.062+0000 7fd80035f640 1 --2- 172.21.15.100:0/360332793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7f8108730 0x7fd7f810ef60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0009ca0 tx=0x7fd7f002a8c0 comp rx=0 tx=0).stop 2024-07-26T13:37:01.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.062+0000 7fd80035f640 1 -- 172.21.15.100:0/360332793 shutdown_connections 2024-07-26T13:37:01.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.062+0000 7fd80035f640 1 --2- 172.21.15.100:0/360332793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7f8108730 0x7fd7f810ef60 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:01.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.062+0000 7fd80035f640 1 --2- 172.21.15.100:0/360332793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7f8107de0 0x7fd7f81081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:01.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.062+0000 7fd80035f640 1 --2- 172.21.15.100:0/360332793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7f81024a0 0x7fd7f81028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:01.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.062+0000 7fd80035f640 1 -- 172.21.15.100:0/360332793 >> 172.21.15.100:0/360332793 conn(0x7fd7f8079140 msgr2=0x7fd7f8079550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:01.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.062+0000 7fd80035f640 1 -- 172.21.15.100:0/360332793 shutdown_connections 2024-07-26T13:37:01.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.062+0000 7fd80035f640 1 -- 172.21.15.100:0/360332793 wait complete. 2024-07-26T13:37:01.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.062+0000 7fd80035f640 1 Processor -- start 2024-07-26T13:37:01.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.062+0000 7fd80035f640 1 -- start start 2024-07-26T13:37:01.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.063+0000 7fd80035f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7f81024a0 0x7fd7f8199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:01.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.063+0000 7fd80035f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7f8107de0 0x7fd7f819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:01.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.063+0000 7fd80035f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7f8108730 0x7fd7f819f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:01.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.063+0000 7fd80035f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f8113510 con 0x7fd7f8107de0 2024-07-26T13:37:01.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.063+0000 7fd80035f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f8113390 con 0x7fd7f8108730 2024-07-26T13:37:01.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.063+0000 7fd80035f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f8113690 con 0x7fd7f81024a0 2024-07-26T13:37:01.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.063+0000 7fd7fd8fa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7f8107de0 0x7fd7f819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:01.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.063+0000 7fd7fd8fa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7f8107de0 0x7fd7f819a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42310/0 (socket says 172.21.15.100:42310) 2024-07-26T13:37:01.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.063+0000 7fd7fd8fa640 1 -- 172.21.15.100:0/2894697701 learned_addr learned my addr 172.21.15.100:0/2894697701 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:37:01.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.063+0000 7fd7fe0fb640 1 --2- 172.21.15.100:0/2894697701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7f81024a0 0x7fd7f8199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:01.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.063+0000 7fd7fe8fc640 1 --2- 172.21.15.100:0/2894697701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7f8108730 0x7fd7f819f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:01.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.063+0000 7fd7fd8fa640 1 -- 172.21.15.100:0/2894697701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7f81024a0 msgr2=0x7fd7f8199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:01.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.063+0000 7fd7fd8fa640 1 --2- 172.21.15.100:0/2894697701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7f81024a0 0x7fd7f8199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:01.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.063+0000 7fd7fd8fa640 1 -- 172.21.15.100:0/2894697701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7f8108730 msgr2=0x7fd7f819f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:01.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.063+0000 7fd7fd8fa640 1 --2- 172.21.15.100:0/2894697701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7f8108730 0x7fd7f819f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:01.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.063+0000 7fd7fd8fa640 1 -- 172.21.15.100:0/2894697701 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f819fe90 con 0x7fd7f8107de0 2024-07-26T13:37:01.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.063+0000 7fd7fd8fa640 1 --2- 172.21.15.100:0/2894697701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7f8107de0 0x7fd7f819a4c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd7e800d740 tx=0x7fd7e800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:01.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.063+0000 7fd7ef7fe640 1 -- 172.21.15.100:0/2894697701 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8017070 con 0x7fd7f8107de0 2024-07-26T13:37:01.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.063+0000 7fd80035f640 1 -- 172.21.15.100:0/2894697701 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7f81a0180 con 0x7fd7f8107de0 2024-07-26T13:37:01.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.063+0000 7fd7ef7fe640 1 -- 172.21.15.100:0/2894697701 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7e800de50 con 0x7fd7f8107de0 2024-07-26T13:37:01.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.064+0000 7fd7ef7fe640 1 -- 172.21.15.100:0/2894697701 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e800f620 con 0x7fd7f8107de0 2024-07-26T13:37:01.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.064+0000 7fd80035f640 1 -- 172.21.15.100:0/2894697701 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7f81a68e0 con 0x7fd7f8107de0 2024-07-26T13:37:01.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.064+0000 7fd80035f640 1 -- 172.21.15.100:0/2894697701 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7f8103720 con 0x7fd7f8107de0 2024-07-26T13:37:01.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.066+0000 7fd7ef7fe640 1 -- 172.21.15.100:0/2894697701 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7fd7e801a070 con 0x7fd7f8107de0 2024-07-26T13:37:01.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.067+0000 7fd7ef7fe640 1 --2- 172.21.15.100:0/2894697701 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd7e40b3ca0 0x7fd7e40b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:01.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.067+0000 7fd7fe0fb640 1 --2- 172.21.15.100:0/2894697701 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd7e40b3ca0 0x7fd7e40b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:01.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.067+0000 7fd7ef7fe640 1 -- 172.21.15.100:0/2894697701 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8094+0+0 (secure 0 0 0) 0x7fd7e80e3500 con 0x7fd7f8107de0 2024-07-26T13:37:01.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.067+0000 7fd7fe0fb640 1 --2- 172.21.15.100:0/2894697701 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd7e40b3ca0 0x7fd7e40b61b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4005fd0 tx=0x7fd7f4005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:01.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.070+0000 7fd7ef7fe640 1 -- 172.21.15.100:0/2894697701 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fd7e80aacb0 con 0x7fd7f8107de0 2024-07-26T13:37:01.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.280+0000 7fd80035f640 1 -- 172.21.15.100:0/2894697701 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd7f81028b0 con 0x7fd7f8107de0 2024-07-26T13:37:01.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.466+0000 7fd7ef7fe640 1 -- 172.21.15.100:0/2894697701 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fd7f81028b0 con 0x7fd7f8107de0 2024-07-26T13:37:01.467 INFO:teuthology.orchestra.run.smithi100.stderr:new fs with metadata pool 2 and data pool 3 2024-07-26T13:37:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.469+0000 7fd80035f640 1 -- 172.21.15.100:0/2894697701 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd7e40b3ca0 msgr2=0x7fd7e40b61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.469+0000 7fd80035f640 1 --2- 172.21.15.100:0/2894697701 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd7e40b3ca0 0x7fd7e40b61b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4005fd0 tx=0x7fd7f4005850 comp rx=0 tx=0).stop 2024-07-26T13:37:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.469+0000 7fd80035f640 1 -- 172.21.15.100:0/2894697701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7f8107de0 msgr2=0x7fd7f819a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.469+0000 7fd80035f640 1 --2- 172.21.15.100:0/2894697701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7f8107de0 0x7fd7f819a4c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd7e800d740 tx=0x7fd7e800dc00 comp rx=0 tx=0).stop 2024-07-26T13:37:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.469+0000 7fd80035f640 1 -- 172.21.15.100:0/2894697701 shutdown_connections 2024-07-26T13:37:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.469+0000 7fd80035f640 1 --2- 172.21.15.100:0/2894697701 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd7e40b3ca0 0x7fd7e40b61b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.469+0000 7fd80035f640 1 --2- 172.21.15.100:0/2894697701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7f8108730 0x7fd7f819f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:01.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.469+0000 7fd80035f640 1 --2- 172.21.15.100:0/2894697701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7f8107de0 0x7fd7f819a4c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:01.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.469+0000 7fd80035f640 1 --2- 172.21.15.100:0/2894697701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7f81024a0 0x7fd7f8199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:01.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.469+0000 7fd80035f640 1 -- 172.21.15.100:0/2894697701 >> 172.21.15.100:0/2894697701 conn(0x7fd7f8079140 msgr2=0x7fd7f8103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:01.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.469+0000 7fd80035f640 1 -- 172.21.15.100:0/2894697701 shutdown_connections 2024-07-26T13:37:01.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:01.469+0000 7fd80035f640 1 -- 172.21.15.100:0/2894697701 wait complete. 2024-07-26T13:37:01.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:01 smithi100 ceph-mon[32599]: pgmap v267: 97 pgs: 13 creating+peering, 36 unknown, 48 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:37:01.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:01 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2894697701' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-07-26T13:37:01.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:01 smithi100 ceph-mon[32599]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-07-26T13:37:01.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:01 smithi167 ceph-mon[37140]: pgmap v267: 97 pgs: 13 creating+peering, 36 unknown, 48 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:37:01.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:01 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/2894697701' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-07-26T13:37:01.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:01 smithi167 ceph-mon[37140]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-07-26T13:37:01.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:01 smithi138 ceph-mon[36954]: pgmap v267: 97 pgs: 13 creating+peering, 36 unknown, 48 active+clean; 577 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-26T13:37:01.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:01 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/2894697701' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-07-26T13:37:01.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:01 smithi138 ceph-mon[36954]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-07-26T13:37:02.228 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph osd dump --format=json 2024-07-26T13:37:02.485 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:37:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:02 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2894697701' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-07-26T13:37:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:02 smithi100 ceph-mon[32599]: osdmap e79: 12 total, 12 up, 12 in 2024-07-26T13:37:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:02 smithi100 ceph-mon[32599]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-07-26T13:37:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:02 smithi100 ceph-mon[32599]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-07-26T13:37:02.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:02 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/2894697701' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-07-26T13:37:02.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:02 smithi167 ceph-mon[37140]: osdmap e79: 12 total, 12 up, 12 in 2024-07-26T13:37:02.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:02 smithi167 ceph-mon[37140]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-07-26T13:37:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:02 smithi167 ceph-mon[37140]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-07-26T13:37:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:02 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/2894697701' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-07-26T13:37:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:02 smithi138 ceph-mon[36954]: osdmap e79: 12 total, 12 up, 12 in 2024-07-26T13:37:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:02 smithi138 ceph-mon[36954]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-07-26T13:37:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:02 smithi138 ceph-mon[36954]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-07-26T13:37:03.608 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:03 smithi100 ceph-mon[32599]: pgmap v270: 97 pgs: 12 creating+peering, 19 unknown, 66 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-07-26T13:37:03.608 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:03 smithi100 ceph-mon[32599]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-07-26T13:37:03.608 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:03 smithi100 ceph-mon[32599]: Cluster is now healthy 2024-07-26T13:37:03.608 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:03 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] up:active 2024-07-26T13:37:03.608 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:03 smithi100 ceph-mon[32599]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-07-26T13:37:03.608 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:03 smithi100 ceph-mon[32599]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-07-26T13:37:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:03 smithi167 ceph-mon[37140]: pgmap v270: 97 pgs: 12 creating+peering, 19 unknown, 66 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-07-26T13:37:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:03 smithi167 ceph-mon[37140]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-07-26T13:37:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:03 smithi167 ceph-mon[37140]: Cluster is now healthy 2024-07-26T13:37:03.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:03 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] up:active 2024-07-26T13:37:03.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:03 smithi167 ceph-mon[37140]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-07-26T13:37:03.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:03 smithi167 ceph-mon[37140]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-07-26T13:37:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:03 smithi138 ceph-mon[36954]: pgmap v270: 97 pgs: 12 creating+peering, 19 unknown, 66 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-07-26T13:37:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:03 smithi138 ceph-mon[36954]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-07-26T13:37:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:03 smithi138 ceph-mon[36954]: Cluster is now healthy 2024-07-26T13:37:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:03 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] up:active 2024-07-26T13:37:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:03 smithi138 ceph-mon[36954]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-07-26T13:37:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:03 smithi138 ceph-mon[36954]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-07-26T13:37:04.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.142+0000 7f594a8ea640 1 -- 172.21.15.100:0/4017169066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5944103590 msgr2=0x7f59441039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:04.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.142+0000 7f594a8ea640 1 --2- 172.21.15.100:0/4017169066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5944103590 0x7f59441039a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5934009900 tx=0x7f593402a410 comp rx=0 tx=0).stop 2024-07-26T13:37:04.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.142+0000 7f594a8ea640 1 -- 172.21.15.100:0/4017169066 shutdown_connections 2024-07-26T13:37:04.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.142+0000 7f594a8ea640 1 --2- 172.21.15.100:0/4017169066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5944109820 0x7f5944110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:04.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.142+0000 7f594a8ea640 1 --2- 172.21.15.100:0/4017169066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5944108ed0 0x7f59441092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:04.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.142+0000 7f594a8ea640 1 --2- 172.21.15.100:0/4017169066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5944103590 0x7f59441039a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:04.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.142+0000 7f594a8ea640 1 -- 172.21.15.100:0/4017169066 >> 172.21.15.100:0/4017169066 conn(0x7f594407b280 msgr2=0x7f594407b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:04.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.142+0000 7f594a8ea640 1 -- 172.21.15.100:0/4017169066 shutdown_connections 2024-07-26T13:37:04.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.142+0000 7f594a8ea640 1 -- 172.21.15.100:0/4017169066 wait complete. 2024-07-26T13:37:04.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.142+0000 7f594a8ea640 1 Processor -- start 2024-07-26T13:37:04.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.143+0000 7f594a8ea640 1 -- start start 2024-07-26T13:37:04.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.143+0000 7f594a8ea640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5944103590 0x7f5944199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:04.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.143+0000 7f594a8ea640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5944108ed0 0x7f594419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:04.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.143+0000 7f594a8ea640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5944109820 0x7f594419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:04.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.143+0000 7f594a8ea640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f59441134f0 con 0x7f5944108ed0 2024-07-26T13:37:04.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.143+0000 7f594a8ea640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5944113370 con 0x7f5944109820 2024-07-26T13:37:04.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.143+0000 7f594a8ea640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5944113670 con 0x7f5944103590 2024-07-26T13:37:04.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.143+0000 7f59437fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5944108ed0 0x7f594419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:04.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.143+0000 7f59437fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5944108ed0 0x7f594419a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42326/0 (socket says 172.21.15.100:42326) 2024-07-26T13:37:04.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.143+0000 7f59437fe640 1 -- 172.21.15.100:0/3015989199 learned_addr learned my addr 172.21.15.100:0/3015989199 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:37:04.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.143+0000 7f5948e87640 1 --2- 172.21.15.100:0/3015989199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5944109820 0x7f594419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:04.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.143+0000 7f5943fff640 1 --2- 172.21.15.100:0/3015989199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5944103590 0x7f5944199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:04.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.143+0000 7f59437fe640 1 -- 172.21.15.100:0/3015989199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5944103590 msgr2=0x7f5944199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:04.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.143+0000 7f59437fe640 1 --2- 172.21.15.100:0/3015989199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5944103590 0x7f5944199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:04.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.143+0000 7f59437fe640 1 -- 172.21.15.100:0/3015989199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5944109820 msgr2=0x7f594419f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:04.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.143+0000 7f59437fe640 1 --2- 172.21.15.100:0/3015989199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5944109820 0x7f594419f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:04.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.143+0000 7f59437fe640 1 -- 172.21.15.100:0/3015989199 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f594419fe50 con 0x7f5944108ed0 2024-07-26T13:37:04.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.144+0000 7f5943fff640 1 --2- 172.21.15.100:0/3015989199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5944103590 0x7f5944199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:37:04.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.144+0000 7f59437fe640 1 --2- 172.21.15.100:0/3015989199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5944108ed0 0x7f594419a540 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f593800d750 tx=0x7f593800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:04.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.144+0000 7f59417fa640 1 -- 172.21.15.100:0/3015989199 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5938017070 con 0x7f5944108ed0 2024-07-26T13:37:04.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.144+0000 7f594a8ea640 1 -- 172.21.15.100:0/3015989199 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59441a0140 con 0x7f5944108ed0 2024-07-26T13:37:04.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.144+0000 7f59417fa640 1 -- 172.21.15.100:0/3015989199 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59380054d0 con 0x7f5944108ed0 2024-07-26T13:37:04.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.144+0000 7f59417fa640 1 -- 172.21.15.100:0/3015989199 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5938004c30 con 0x7f5944108ed0 2024-07-26T13:37:04.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.144+0000 7f594a8ea640 1 -- 172.21.15.100:0/3015989199 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59441a6960 con 0x7f5944108ed0 2024-07-26T13:37:04.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.145+0000 7f594a8ea640 1 -- 172.21.15.100:0/3015989199 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59441039a0 con 0x7f5944108ed0 2024-07-26T13:37:04.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.147+0000 7f59417fa640 1 -- 172.21.15.100:0/3015989199 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f5938004dd0 con 0x7f5944108ed0 2024-07-26T13:37:04.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.147+0000 7f59417fa640 1 --2- 172.21.15.100:0/3015989199 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f592c0b8060 0x7f592c0ba570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:04.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.147+0000 7f5943fff640 1 --2- 172.21.15.100:0/3015989199 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f592c0b8060 0x7f592c0ba570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:04.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.148+0000 7f59417fa640 1 -- 172.21.15.100:0/3015989199 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8194+0+0 (secure 0 0 0) 0x7f59380e3a00 con 0x7f5944108ed0 2024-07-26T13:37:04.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.148+0000 7f5943fff640 1 --2- 172.21.15.100:0/3015989199 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f592c0b8060 0x7f592c0ba570 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5934031440 tx=0x7f5934009680 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:04.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.150+0000 7f59417fa640 1 -- 172.21.15.100:0/3015989199 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f59380ab070 con 0x7f5944108ed0 2024-07-26T13:37:04.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.325+0000 7f594a8ea640 1 -- 172.21.15.100:0/3015989199 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f594410b5a0 con 0x7f5944108ed0 2024-07-26T13:37:04.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.326+0000 7f59417fa640 1 -- 172.21.15.100:0/3015989199 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22595 (secure 0 0 0) 0x7f59380aa970 con 0x7f5944108ed0 2024-07-26T13:37:04.328 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:37:04.328 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":79,"fsid":"bf83e2e6-4b52-11ef-bcb7-c7b262605968","created":"2024-07-26T13:28:23.639785+0000","modified":"2024-07-26T13:37:01.281880+0000","last_up_change":"2024-07-26T13:35:56.475554+0000","last_in_change":"2024-07-26T13:35:37.472564+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-26T13:32:22.329969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-26T13:36:54.590525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-26T13:36:58.149545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d714036d-9dda-4cb5-af16-29f83a09dd70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6803","nonce":1480401850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6805","nonce":1480401850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6809","nonce":1480401850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6807","nonce":1480401850}]},"public_addr":"172.21.15.100:6803/1480401850","cluster_addr":"172.21.15.100:6805/1480401850","heartbeat_back_addr":"172.21.15.100:6809/1480401850","heartbeat_front_addr":"172.21.15.100:6807/1480401850","state":["exists","up"]},{"osd":1,"uuid":"3dd0eaa1-618a-495a-9819-e01c0f869ae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6801","nonce":3090877671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6803","nonce":3090877671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6807","nonce":3090877671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6805","nonce":3090877671}]},"public_addr":"172.21.15.138:6801/3090877671","cluster_addr":"172.21.15.138:6803/3090877671","heartbeat_back_addr":"172.21.15.138:6807/3090877671","heartbeat_front_addr":"172.21.15.138:6805/3090877671","state":["exists","up"]},{"osd":2,"uuid":"5c710a78-17ae-41a8-ac16-a678f5d35672","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6801","nonce":3583424790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6803","nonce":3583424790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6807","nonce":3583424790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6805","nonce":3583424790}]},"public_addr":"172.21.15.167:6801/3583424790","cluster_addr":"172.21.15.167:6803/3583424790","heartbeat_back_addr":"172.21.15.167:6807/3583424790","heartbeat_front_addr":"172.21.15.167:6805/3583424790","state":["exists","up"]},{"osd":3,"uuid":"be845495-8c31-4834-a45c-bdc576711962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6811","nonce":3626762419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6813","nonce":3626762419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6817","nonce":3626762419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6815","nonce":3626762419}]},"public_addr":"172.21.15.100:6811/3626762419","cluster_addr":"172.21.15.100:6813/3626762419","heartbeat_back_addr":"172.21.15.100:6817/3626762419","heartbeat_front_addr":"172.21.15.100:6815/3626762419","state":["exists","up"]},{"osd":4,"uuid":"520554cc-e846-46f9-af77-05a1de65a452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6809","nonce":3757334356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6811","nonce":3757334356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6815","nonce":3757334356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6813","nonce":3757334356}]},"public_addr":"172.21.15.138:6809/3757334356","cluster_addr":"172.21.15.138:6811/3757334356","heartbeat_back_addr":"172.21.15.138:6815/3757334356","heartbeat_front_addr":"172.21.15.138:6813/3757334356","state":["exists","up"]},{"osd":5,"uuid":"68dc8ad6-31d2-45b5-9aa7-62c4218d46e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6809","nonce":3239482836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6811","nonce":3239482836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6815","nonce":3239482836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6813","nonce":3239482836}]},"public_addr":"172.21.15.167:6809/3239482836","cluster_addr":"172.21.15.167:6811/3239482836","heartbeat_back_addr":"172.21.15.167:6815/3239482836","heartbeat_front_addr":"172.21.15.167:6813/3239482836","state":["exists","up"]},{"osd":6,"uuid":"420e4471-5626-41cc-a8e1-69c801196b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6819","nonce":3218415484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6821","nonce":3218415484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6825","nonce":3218415484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6823","nonce":3218415484}]},"public_addr":"172.21.15.100:6819/3218415484","cluster_addr":"172.21.15.100:6821/3218415484","heartbeat_back_addr":"172.21.15.100:6825/3218415484","heartbeat_front_addr":"172.21.15.100:6823/3218415484","state":["exists","up"]},{"osd":7,"uuid":"a27b097e-207d-44c3-af63-544f228b83ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6817","nonce":45541749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6819","nonce":45541749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6823","nonce":45541749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6821","nonce":45541749}]},"public_addr":"172.21.15.138:6817/45541749","cluster_addr":"172.21.15.138:6819/45541749","heartbeat_back_addr":"172.21.15.138:6823/45541749","heartbeat_front_addr":"172.21.15.138:6821/45541749","state":["exists","up"]},{"osd":8,"uuid":"3e865858-92cc-48a0-a464-5767d19e8ac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6817","nonce":4031159803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6819","nonce":4031159803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6823","nonce":4031159803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6821","nonce":4031159803}]},"public_addr":"172.21.15.167:6817/4031159803","cluster_addr":"172.21.15.167:6819/4031159803","heartbeat_back_addr":"172.21.15.167:6823/4031159803","heartbeat_front_addr":"172.21.15.167:6821/4031159803","state":["exists","up"]},{"osd":9,"uuid":"8b05b14a-1195-42f5-ac01-d5634cb9c888","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6827","nonce":1840140273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6829","nonce":1840140273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6833","nonce":1840140273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6831","nonce":1840140273}]},"public_addr":"172.21.15.100:6827/1840140273","cluster_addr":"172.21.15.100:6829/1840140273","heartbeat_back_addr":"172.21.15.100:6833/1840140273","heartbeat_front_addr":"172.21.15.100:6831/1840140273","state":["exists","up"]},{"osd":10,"uuid":"ab77ebe7-5254-4dc3-bc54-625698e84afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6825","nonce":703382899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6827","nonce":703382899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6831","nonce":703382899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6829","nonce":703382899}]},"public_addr":"172.21.15.138:6825/703382899","cluster_addr":"172.21.15.138:6827/703382899","heartbeat_back_addr":"172.21.15.138:6831/703382899","heartbeat_front_addr":"172.21.15.138:6829/703382899","state":["exists","up"]},{"osd":11,"uuid":"229cf91e-e9ef-48f6-a677-c5c5705b315a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6825","nonce":1032343068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6827","nonce":1032343068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6831","nonce":1032343068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6829","nonce":1032343068}]},"public_addr":"172.21.15.167:6825/1032343068","cluster_addr":"172.21.15.167:6827/1032343068","heartbeat_back_addr":"172.21.15.167:6831/1032343068","heartbeat_front_addr":"172.21.15.167:6829/1032343068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:31:35.800702+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:31:58.087922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:32:20.463183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:32:42.920598+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:33:06.332655+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:33:29.381407+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:33:53.975963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:34:18.226813+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:34:41.845770+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:35:05.722275+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:35:30.862704+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:35:54.738697+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:0/1874993112":"2024-07-27T13:30:16.253096+0000","172.21.15.100:0/2364620305":"2024-07-27T13:28:45.340334+0000","172.21.15.100:6800/3096076075":"2024-07-27T13:29:07.979178+0000","172.21.15.100:0/1894966501":"2024-07-27T13:30:16.253096+0000","172.21.15.100:0/3411776672":"2024-07-27T13:28:45.340334+0000","172.21.15.100:0/2853423120":"2024-07-27T13:29:07.979178+0000","172.21.15.100:6800/3025873570":"2024-07-27T13:28:45.340334+0000","172.21.15.100:0/4887181":"2024-07-27T13:29:07.979178+0000","172.21.15.100:6801/908675140":"2024-07-27T13:30:16.253096+0000","172.21.15.100:6801/3025873570":"2024-07-27T13:28:45.340334+0000","172.21.15.100:6800/908675140":"2024-07-27T13:30:16.253096+0000","172.21.15.100:6801/3096076075":"2024-07-27T13:29:07.979178+0000","172.21.15.100:0/2302632052":"2024-07-27T13:29:07.979178+0000","172.21.15.100:0/3736098915":"2024-07-27T13:28:45.340334+0000","172.21.15.100:0/3243030560":"2024-07-27T13:30:16.253096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-26T13:37:04.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.329+0000 7f594a8ea640 1 -- 172.21.15.100:0/3015989199 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f592c0b8060 msgr2=0x7f592c0ba570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:04.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.329+0000 7f594a8ea640 1 --2- 172.21.15.100:0/3015989199 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f592c0b8060 0x7f592c0ba570 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5934031440 tx=0x7f5934009680 comp rx=0 tx=0).stop 2024-07-26T13:37:04.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.329+0000 7f594a8ea640 1 -- 172.21.15.100:0/3015989199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5944108ed0 msgr2=0x7f594419a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:04.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.329+0000 7f594a8ea640 1 --2- 172.21.15.100:0/3015989199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5944108ed0 0x7f594419a540 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f593800d750 tx=0x7f593800dc10 comp rx=0 tx=0).stop 2024-07-26T13:37:04.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.329+0000 7f594a8ea640 1 -- 172.21.15.100:0/3015989199 shutdown_connections 2024-07-26T13:37:04.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.329+0000 7f594a8ea640 1 --2- 172.21.15.100:0/3015989199 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f592c0b8060 0x7f592c0ba570 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:04.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.329+0000 7f594a8ea640 1 --2- 172.21.15.100:0/3015989199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5944109820 0x7f594419f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:04.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.329+0000 7f594a8ea640 1 --2- 172.21.15.100:0/3015989199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5944108ed0 0x7f594419a540 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:04.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.329+0000 7f594a8ea640 1 --2- 172.21.15.100:0/3015989199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5944103590 0x7f5944199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:04.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.329+0000 7f594a8ea640 1 -- 172.21.15.100:0/3015989199 >> 172.21.15.100:0/3015989199 conn(0x7f594407b280 msgr2=0x7f5944107070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:04.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.329+0000 7f594a8ea640 1 -- 172.21.15.100:0/3015989199 shutdown_connections 2024-07-26T13:37:04.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:04.329+0000 7f594a8ea640 1 -- 172.21.15.100:0/3015989199 wait complete. 2024-07-26T13:37:04.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:04 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/3015989199' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-26T13:37:04.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:04 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/3015989199' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-26T13:37:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:04 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/3015989199' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-26T13:37:05.028 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph osd dump --format=json 2024-07-26T13:37:05.283 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:37:05.492 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:37:05 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 2024-07-26T13:37:05.237+0000 7f740ace1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-26T13:37:05.712 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:05 smithi100 ceph-mon[32599]: pgmap v271: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 7 op/s 2024-07-26T13:37:05.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:05 smithi167 ceph-mon[37140]: pgmap v271: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 7 op/s 2024-07-26T13:37:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:05 smithi138 ceph-mon[36954]: pgmap v271: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 7 op/s 2024-07-26T13:37:06.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.975+0000 7fc470dec640 1 -- 172.21.15.100:0/691844862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc46c1035e0 msgr2=0x7fc46c1039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:06.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.975+0000 7fc470dec640 1 --2- 172.21.15.100:0/691844862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc46c1035e0 0x7fc46c1039f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc45c009900 tx=0x7fc45c02a410 comp rx=0 tx=0).stop 2024-07-26T13:37:06.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.975+0000 7fc470dec640 1 -- 172.21.15.100:0/691844862 shutdown_connections 2024-07-26T13:37:06.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.975+0000 7fc470dec640 1 --2- 172.21.15.100:0/691844862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc46c109870 0x7fc46c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:06.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.975+0000 7fc470dec640 1 --2- 172.21.15.100:0/691844862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc46c108f20 0x7fc46c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:06.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.975+0000 7fc470dec640 1 --2- 172.21.15.100:0/691844862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc46c1035e0 0x7fc46c1039f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:06.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.975+0000 7fc470dec640 1 -- 172.21.15.100:0/691844862 >> 172.21.15.100:0/691844862 conn(0x7fc46c0ff030 msgr2=0x7fc46c101480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:06.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.975+0000 7fc470dec640 1 -- 172.21.15.100:0/691844862 shutdown_connections 2024-07-26T13:37:06.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.975+0000 7fc470dec640 1 -- 172.21.15.100:0/691844862 wait complete. 2024-07-26T13:37:06.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.976+0000 7fc470dec640 1 Processor -- start 2024-07-26T13:37:06.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.976+0000 7fc470dec640 1 -- start start 2024-07-26T13:37:06.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.976+0000 7fc470dec640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc46c1035e0 0x7fc46c075140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:06.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.976+0000 7fc470dec640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc46c108f20 0x7fc46c075680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:06.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.976+0000 7fc470dec640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc46c109870 0x7fc46c075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:06.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.976+0000 7fc470dec640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc46c113540 con 0x7fc46c108f20 2024-07-26T13:37:06.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.976+0000 7fc470dec640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc46c1133c0 con 0x7fc46c1035e0 2024-07-26T13:37:06.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.976+0000 7fc470dec640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc46c1136c0 con 0x7fc46c109870 2024-07-26T13:37:06.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.976+0000 7fc469d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc46c108f20 0x7fc46c075680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:06.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.976+0000 7fc469d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc46c108f20 0x7fc46c075680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51428/0 (socket says 172.21.15.100:51428) 2024-07-26T13:37:06.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.976+0000 7fc469d9b640 1 -- 172.21.15.100:0/653769359 learned_addr learned my addr 172.21.15.100:0/653769359 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:37:06.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.976+0000 7fc46a59c640 1 --2- 172.21.15.100:0/653769359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc46c1035e0 0x7fc46c075140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:06.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.977+0000 7fc46ad9d640 1 --2- 172.21.15.100:0/653769359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc46c109870 0x7fc46c075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:06.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.977+0000 7fc469d9b640 1 -- 172.21.15.100:0/653769359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc46c109870 msgr2=0x7fc46c075bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:06.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.977+0000 7fc469d9b640 1 --2- 172.21.15.100:0/653769359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc46c109870 0x7fc46c075bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:06.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.977+0000 7fc469d9b640 1 -- 172.21.15.100:0/653769359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc46c1035e0 msgr2=0x7fc46c075140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:06.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.977+0000 7fc469d9b640 1 --2- 172.21.15.100:0/653769359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc46c1035e0 0x7fc46c075140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:06.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.977+0000 7fc469d9b640 1 -- 172.21.15.100:0/653769359 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc46c1a8140 con 0x7fc46c108f20 2024-07-26T13:37:06.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.977+0000 7fc469d9b640 1 --2- 172.21.15.100:0/653769359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc46c108f20 0x7fc46c075680 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc46000c8c0 tx=0x7fc46000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:06.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.977+0000 7fc45b7fe640 1 -- 172.21.15.100:0/653769359 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc460007bb0 con 0x7fc46c108f20 2024-07-26T13:37:06.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.977+0000 7fc470dec640 1 -- 172.21.15.100:0/653769359 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc46c1a8430 con 0x7fc46c108f20 2024-07-26T13:37:06.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.977+0000 7fc45b7fe640 1 -- 172.21.15.100:0/653769359 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc460007d50 con 0x7fc46c108f20 2024-07-26T13:37:06.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.977+0000 7fc45b7fe640 1 -- 172.21.15.100:0/653769359 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4600187c0 con 0x7fc46c108f20 2024-07-26T13:37:06.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.977+0000 7fc470dec640 1 -- 172.21.15.100:0/653769359 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc46c1a8970 con 0x7fc46c108f20 2024-07-26T13:37:06.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.978+0000 7fc470dec640 1 -- 172.21.15.100:0/653769359 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc46c1039f0 con 0x7fc46c108f20 2024-07-26T13:37:06.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.980+0000 7fc45b7fe640 1 -- 172.21.15.100:0/653769359 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7fc46000fa20 con 0x7fc46c108f20 2024-07-26T13:37:06.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.980+0000 7fc45b7fe640 1 --2- 172.21.15.100:0/653769359 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc4540b3c20 0x7fc4540b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:06.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.980+0000 7fc46a59c640 1 --2- 172.21.15.100:0/653769359 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc4540b3c20 0x7fc4540b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:06.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.980+0000 7fc45b7fe640 1 -- 172.21.15.100:0/653769359 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8194+0+0 (secure 0 0 0) 0x7fc4600e4000 con 0x7fc46c108f20 2024-07-26T13:37:06.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.981+0000 7fc46a59c640 1 --2- 172.21.15.100:0/653769359 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc4540b3c20 0x7fc4540b6130 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc45c000c00 tx=0x7fc45c031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:06.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:06.983+0000 7fc45b7fe640 1 -- 172.21.15.100:0/653769359 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fc460014030 con 0x7fc46c108f20 2024-07-26T13:37:07.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:07.158+0000 7fc470dec640 1 -- 172.21.15.100:0/653769359 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc46c02e0c0 con 0x7fc46c108f20 2024-07-26T13:37:07.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:07.159+0000 7fc45b7fe640 1 -- 172.21.15.100:0/653769359 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22595 (secure 0 0 0) 0x7fc4600ab700 con 0x7fc46c108f20 2024-07-26T13:37:07.160 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:37:07.160 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":79,"fsid":"bf83e2e6-4b52-11ef-bcb7-c7b262605968","created":"2024-07-26T13:28:23.639785+0000","modified":"2024-07-26T13:37:01.281880+0000","last_up_change":"2024-07-26T13:35:56.475554+0000","last_in_change":"2024-07-26T13:35:37.472564+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-26T13:32:22.329969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-26T13:36:54.590525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-26T13:36:58.149545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d714036d-9dda-4cb5-af16-29f83a09dd70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6803","nonce":1480401850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6805","nonce":1480401850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6809","nonce":1480401850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6807","nonce":1480401850}]},"public_addr":"172.21.15.100:6803/1480401850","cluster_addr":"172.21.15.100:6805/1480401850","heartbeat_back_addr":"172.21.15.100:6809/1480401850","heartbeat_front_addr":"172.21.15.100:6807/1480401850","state":["exists","up"]},{"osd":1,"uuid":"3dd0eaa1-618a-495a-9819-e01c0f869ae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6801","nonce":3090877671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6803","nonce":3090877671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6807","nonce":3090877671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6805","nonce":3090877671}]},"public_addr":"172.21.15.138:6801/3090877671","cluster_addr":"172.21.15.138:6803/3090877671","heartbeat_back_addr":"172.21.15.138:6807/3090877671","heartbeat_front_addr":"172.21.15.138:6805/3090877671","state":["exists","up"]},{"osd":2,"uuid":"5c710a78-17ae-41a8-ac16-a678f5d35672","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6801","nonce":3583424790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6803","nonce":3583424790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6807","nonce":3583424790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6805","nonce":3583424790}]},"public_addr":"172.21.15.167:6801/3583424790","cluster_addr":"172.21.15.167:6803/3583424790","heartbeat_back_addr":"172.21.15.167:6807/3583424790","heartbeat_front_addr":"172.21.15.167:6805/3583424790","state":["exists","up"]},{"osd":3,"uuid":"be845495-8c31-4834-a45c-bdc576711962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6811","nonce":3626762419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6813","nonce":3626762419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6817","nonce":3626762419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6815","nonce":3626762419}]},"public_addr":"172.21.15.100:6811/3626762419","cluster_addr":"172.21.15.100:6813/3626762419","heartbeat_back_addr":"172.21.15.100:6817/3626762419","heartbeat_front_addr":"172.21.15.100:6815/3626762419","state":["exists","up"]},{"osd":4,"uuid":"520554cc-e846-46f9-af77-05a1de65a452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6809","nonce":3757334356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6811","nonce":3757334356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6815","nonce":3757334356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6813","nonce":3757334356}]},"public_addr":"172.21.15.138:6809/3757334356","cluster_addr":"172.21.15.138:6811/3757334356","heartbeat_back_addr":"172.21.15.138:6815/3757334356","heartbeat_front_addr":"172.21.15.138:6813/3757334356","state":["exists","up"]},{"osd":5,"uuid":"68dc8ad6-31d2-45b5-9aa7-62c4218d46e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6809","nonce":3239482836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6811","nonce":3239482836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6815","nonce":3239482836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6813","nonce":3239482836}]},"public_addr":"172.21.15.167:6809/3239482836","cluster_addr":"172.21.15.167:6811/3239482836","heartbeat_back_addr":"172.21.15.167:6815/3239482836","heartbeat_front_addr":"172.21.15.167:6813/3239482836","state":["exists","up"]},{"osd":6,"uuid":"420e4471-5626-41cc-a8e1-69c801196b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6819","nonce":3218415484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6821","nonce":3218415484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6825","nonce":3218415484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6823","nonce":3218415484}]},"public_addr":"172.21.15.100:6819/3218415484","cluster_addr":"172.21.15.100:6821/3218415484","heartbeat_back_addr":"172.21.15.100:6825/3218415484","heartbeat_front_addr":"172.21.15.100:6823/3218415484","state":["exists","up"]},{"osd":7,"uuid":"a27b097e-207d-44c3-af63-544f228b83ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6817","nonce":45541749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6819","nonce":45541749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6823","nonce":45541749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6821","nonce":45541749}]},"public_addr":"172.21.15.138:6817/45541749","cluster_addr":"172.21.15.138:6819/45541749","heartbeat_back_addr":"172.21.15.138:6823/45541749","heartbeat_front_addr":"172.21.15.138:6821/45541749","state":["exists","up"]},{"osd":8,"uuid":"3e865858-92cc-48a0-a464-5767d19e8ac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6817","nonce":4031159803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6819","nonce":4031159803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6823","nonce":4031159803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6821","nonce":4031159803}]},"public_addr":"172.21.15.167:6817/4031159803","cluster_addr":"172.21.15.167:6819/4031159803","heartbeat_back_addr":"172.21.15.167:6823/4031159803","heartbeat_front_addr":"172.21.15.167:6821/4031159803","state":["exists","up"]},{"osd":9,"uuid":"8b05b14a-1195-42f5-ac01-d5634cb9c888","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6827","nonce":1840140273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6829","nonce":1840140273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6833","nonce":1840140273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6831","nonce":1840140273}]},"public_addr":"172.21.15.100:6827/1840140273","cluster_addr":"172.21.15.100:6829/1840140273","heartbeat_back_addr":"172.21.15.100:6833/1840140273","heartbeat_front_addr":"172.21.15.100:6831/1840140273","state":["exists","up"]},{"osd":10,"uuid":"ab77ebe7-5254-4dc3-bc54-625698e84afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6825","nonce":703382899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6827","nonce":703382899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6831","nonce":703382899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6829","nonce":703382899}]},"public_addr":"172.21.15.138:6825/703382899","cluster_addr":"172.21.15.138:6827/703382899","heartbeat_back_addr":"172.21.15.138:6831/703382899","heartbeat_front_addr":"172.21.15.138:6829/703382899","state":["exists","up"]},{"osd":11,"uuid":"229cf91e-e9ef-48f6-a677-c5c5705b315a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6825","nonce":1032343068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6827","nonce":1032343068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6831","nonce":1032343068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6829","nonce":1032343068}]},"public_addr":"172.21.15.167:6825/1032343068","cluster_addr":"172.21.15.167:6827/1032343068","heartbeat_back_addr":"172.21.15.167:6831/1032343068","heartbeat_front_addr":"172.21.15.167:6829/1032343068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:31:35.800702+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:31:58.087922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:32:20.463183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:32:42.920598+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:33:06.332655+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:33:29.381407+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:33:53.975963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:34:18.226813+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:34:41.845770+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:35:05.722275+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:35:30.862704+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:35:54.738697+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:0/1874993112":"2024-07-27T13:30:16.253096+0000","172.21.15.100:0/2364620305":"2024-07-27T13:28:45.340334+0000","172.21.15.100:6800/3096076075":"2024-07-27T13:29:07.979178+0000","172.21.15.100:0/1894966501":"2024-07-27T13:30:16.253096+0000","172.21.15.100:0/3411776672":"2024-07-27T13:28:45.340334+0000","172.21.15.100:0/2853423120":"2024-07-27T13:29:07.979178+0000","172.21.15.100:6800/3025873570":"2024-07-27T13:28:45.340334+0000","172.21.15.100:0/4887181":"2024-07-27T13:29:07.979178+0000","172.21.15.100:6801/908675140":"2024-07-27T13:30:16.253096+0000","172.21.15.100:6801/3025873570":"2024-07-27T13:28:45.340334+0000","172.21.15.100:6800/908675140":"2024-07-27T13:30:16.253096+0000","172.21.15.100:6801/3096076075":"2024-07-27T13:29:07.979178+0000","172.21.15.100:0/2302632052":"2024-07-27T13:29:07.979178+0000","172.21.15.100:0/3736098915":"2024-07-27T13:28:45.340334+0000","172.21.15.100:0/3243030560":"2024-07-27T13:30:16.253096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-26T13:37:07.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:07.161+0000 7fc470dec640 1 -- 172.21.15.100:0/653769359 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc4540b3c20 msgr2=0x7fc4540b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:07.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:07.161+0000 7fc470dec640 1 --2- 172.21.15.100:0/653769359 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc4540b3c20 0x7fc4540b6130 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc45c000c00 tx=0x7fc45c031400 comp rx=0 tx=0).stop 2024-07-26T13:37:07.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:07.161+0000 7fc470dec640 1 -- 172.21.15.100:0/653769359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc46c108f20 msgr2=0x7fc46c075680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:07.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:07.161+0000 7fc470dec640 1 --2- 172.21.15.100:0/653769359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc46c108f20 0x7fc46c075680 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc46000c8c0 tx=0x7fc46000cd80 comp rx=0 tx=0).stop 2024-07-26T13:37:07.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:07.161+0000 7fc470dec640 1 -- 172.21.15.100:0/653769359 shutdown_connections 2024-07-26T13:37:07.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:07.161+0000 7fc470dec640 1 --2- 172.21.15.100:0/653769359 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc4540b3c20 0x7fc4540b6130 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:07.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:07.161+0000 7fc470dec640 1 --2- 172.21.15.100:0/653769359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc46c109870 0x7fc46c075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:07.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:07.161+0000 7fc470dec640 1 --2- 172.21.15.100:0/653769359 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc46c108f20 0x7fc46c075680 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:07.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:07.161+0000 7fc470dec640 1 --2- 172.21.15.100:0/653769359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc46c1035e0 0x7fc46c075140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:07.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:07.161+0000 7fc470dec640 1 -- 172.21.15.100:0/653769359 >> 172.21.15.100:0/653769359 conn(0x7fc46c0ff030 msgr2=0x7fc46c101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:07.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:07.161+0000 7fc470dec640 1 -- 172.21.15.100:0/653769359 shutdown_connections 2024-07-26T13:37:07.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:07.161+0000 7fc470dec640 1 -- 172.21.15.100:0/653769359 wait complete. 2024-07-26T13:37:07.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:07 smithi100 ceph-mon[32599]: pgmap v272: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-07-26T13:37:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:07 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/653769359' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-26T13:37:07.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:07 smithi167 ceph-mon[37140]: pgmap v272: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-07-26T13:37:07.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:07 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/653769359' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-26T13:37:07.846 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-07-26T13:37:07.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:07 smithi138 ceph-mon[36954]: pgmap v272: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-07-26T13:37:07.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:07 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/653769359' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-26T13:37:08.101 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:37:09.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:09 smithi167 ceph-mon[37140]: pgmap v273: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-07-26T13:37:09.807 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:09 smithi100 ceph-mon[32599]: pgmap v273: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-07-26T13:37:09.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.805+0000 7f77a3893640 1 -- 172.21.15.100:0/3344020033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f779c101a30 msgr2=0x7f779c101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:09.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.805+0000 7f77a3893640 1 --2- 172.21.15.100:0/3344020033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f779c101a30 0x7f779c101e40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7790009900 tx=0x7f779002a410 comp rx=0 tx=0).stop 2024-07-26T13:37:09.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.805+0000 7f77a3893640 1 -- 172.21.15.100:0/3344020033 shutdown_connections 2024-07-26T13:37:09.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.805+0000 7f77a3893640 1 --2- 172.21.15.100:0/3344020033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f779c106660 0x7f779c1100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:09.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.805+0000 7f77a3893640 1 --2- 172.21.15.100:0/3344020033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f779c105d10 0x7f779c106120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:09.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.805+0000 7f77a3893640 1 --2- 172.21.15.100:0/3344020033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f779c101a30 0x7f779c101e40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:09.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.805+0000 7f77a3893640 1 -- 172.21.15.100:0/3344020033 >> 172.21.15.100:0/3344020033 conn(0x7f779c079870 msgr2=0x7f779c079c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:09.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.805+0000 7f77a3893640 1 -- 172.21.15.100:0/3344020033 shutdown_connections 2024-07-26T13:37:09.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.805+0000 7f77a3893640 1 -- 172.21.15.100:0/3344020033 wait complete. 2024-07-26T13:37:09.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.806+0000 7f77a3893640 1 Processor -- start 2024-07-26T13:37:09.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.806+0000 7f77a3893640 1 -- start start 2024-07-26T13:37:09.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.806+0000 7f77a3893640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f779c101a30 0x7f779c199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:09.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.806+0000 7f77a3893640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f779c105d10 0x7f779c19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:09.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.806+0000 7f77a3893640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f779c106660 0x7f779c19f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:09.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.806+0000 7f77a3893640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f779c113570 con 0x7f779c101a30 2024-07-26T13:37:09.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.806+0000 7f77a3893640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f779c1133f0 con 0x7f779c105d10 2024-07-26T13:37:09.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.806+0000 7f77a3893640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f779c1136f0 con 0x7f779c106660 2024-07-26T13:37:09.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.806+0000 7f77a162f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f779c101a30 0x7f779c199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:09.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.807+0000 7f77a162f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f779c101a30 0x7f779c199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51456/0 (socket says 172.21.15.100:51456) 2024-07-26T13:37:09.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.807+0000 7f77a162f640 1 -- 172.21.15.100:0/2469503338 learned_addr learned my addr 172.21.15.100:0/2469503338 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:37:09.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.807+0000 7f77a0e2e640 1 --2- 172.21.15.100:0/2469503338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f779c105d10 0x7f779c19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:09.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.807+0000 7f77a1e30640 1 --2- 172.21.15.100:0/2469503338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f779c106660 0x7f779c19f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:09.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.807+0000 7f77a162f640 1 -- 172.21.15.100:0/2469503338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f779c106660 msgr2=0x7f779c19f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:09.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.807+0000 7f77a162f640 1 --2- 172.21.15.100:0/2469503338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f779c106660 0x7f779c19f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:09.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.807+0000 7f77a162f640 1 -- 172.21.15.100:0/2469503338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f779c105d10 msgr2=0x7f779c19a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:09.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.807+0000 7f77a162f640 1 --2- 172.21.15.100:0/2469503338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f779c105d10 0x7f779c19a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:09.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.807+0000 7f77a162f640 1 -- 172.21.15.100:0/2469503338 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f779c19fe50 con 0x7f779c101a30 2024-07-26T13:37:09.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.807+0000 7f77a162f640 1 --2- 172.21.15.100:0/2469503338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f779c101a30 0x7f779c199fa0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7790006010 tx=0x7f77900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:09.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.807+0000 7f778e7fc640 1 -- 172.21.15.100:0/2469503338 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779003f030 con 0x7f779c101a30 2024-07-26T13:37:09.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.807+0000 7f778e7fc640 1 -- 172.21.15.100:0/2469503338 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7790003800 con 0x7f779c101a30 2024-07-26T13:37:09.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.807+0000 7f77a3893640 1 -- 172.21.15.100:0/2469503338 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f779c1a00e0 con 0x7f779c101a30 2024-07-26T13:37:09.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.807+0000 7f778e7fc640 1 -- 172.21.15.100:0/2469503338 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7790035070 con 0x7f779c101a30 2024-07-26T13:37:09.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.807+0000 7f77a3893640 1 -- 172.21.15.100:0/2469503338 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f779c1a05c0 con 0x7f779c101a30 2024-07-26T13:37:09.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.808+0000 7f77a3893640 1 -- 172.21.15.100:0/2469503338 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f779c101e40 con 0x7f779c101a30 2024-07-26T13:37:09.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.810+0000 7f778e7fc640 1 -- 172.21.15.100:0/2469503338 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f7790003cb0 con 0x7f779c101a30 2024-07-26T13:37:09.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.810+0000 7f778e7fc640 1 --2- 172.21.15.100:0/2469503338 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f77880b3c50 0x7f77880b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:09.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.811+0000 7f77a0e2e640 1 --2- 172.21.15.100:0/2469503338 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f77880b3c50 0x7f77880b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:09.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.811+0000 7f778e7fc640 1 -- 172.21.15.100:0/2469503338 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8194+0+0 (secure 0 0 0) 0x7f77901046f0 con 0x7f779c101a30 2024-07-26T13:37:09.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.811+0000 7f77a0e2e640 1 --2- 172.21.15.100:0/2469503338 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f77880b3c50 0x7f77880b6160 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f779c06ae70 tx=0x7f779800a2d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:09.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:09.814+0000 7f778e7fc640 1 -- 172.21.15.100:0/2469503338 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f77900cbd60 con 0x7f779c101a30 2024-07-26T13:37:09.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:09 smithi138 ceph-mon[36954]: pgmap v273: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-07-26T13:37:10.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:10.024+0000 7f77a3893640 1 -- 172.21.15.100:0/2469503338 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f779c19b0d0 con 0x7f779c101a30 2024-07-26T13:37:10.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:10.501+0000 7f778e7fc640 1 -- 172.21.15.100:0/2469503338 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f77900cb660 con 0x7f779c101a30 2024-07-26T13:37:10.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:10.504+0000 7f77a3893640 1 -- 172.21.15.100:0/2469503338 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f77880b3c50 msgr2=0x7f77880b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:10.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:10.504+0000 7f77a3893640 1 --2- 172.21.15.100:0/2469503338 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f77880b3c50 0x7f77880b6160 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f779c06ae70 tx=0x7f779800a2d0 comp rx=0 tx=0).stop 2024-07-26T13:37:10.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:10.504+0000 7f77a3893640 1 -- 172.21.15.100:0/2469503338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f779c101a30 msgr2=0x7f779c199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:10.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:10.504+0000 7f77a3893640 1 --2- 172.21.15.100:0/2469503338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f779c101a30 0x7f779c199fa0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7790006010 tx=0x7f77900042f0 comp rx=0 tx=0).stop 2024-07-26T13:37:10.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:10.504+0000 7f77a3893640 1 -- 172.21.15.100:0/2469503338 shutdown_connections 2024-07-26T13:37:10.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:10.504+0000 7f77a3893640 1 --2- 172.21.15.100:0/2469503338 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f77880b3c50 0x7f77880b6160 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:10.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:10.504+0000 7f77a3893640 1 --2- 172.21.15.100:0/2469503338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f779c106660 0x7f779c19f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:10.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:10.504+0000 7f77a3893640 1 --2- 172.21.15.100:0/2469503338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f779c105d10 0x7f779c19a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:10.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:10.504+0000 7f77a3893640 1 --2- 172.21.15.100:0/2469503338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f779c101a30 0x7f779c199fa0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:10.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:10.504+0000 7f77a3893640 1 -- 172.21.15.100:0/2469503338 >> 172.21.15.100:0/2469503338 conn(0x7f779c079870 msgr2=0x7f779c107a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:10.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:10.504+0000 7f77a3893640 1 -- 172.21.15.100:0/2469503338 shutdown_connections 2024-07-26T13:37:10.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:10.504+0000 7f77a3893640 1 -- 172.21.15.100:0/2469503338 wait complete. 2024-07-26T13:37:10.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:10 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/2469503338' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-07-26T13:37:10.885 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:10 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2469503338' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-07-26T13:37:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:10 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/2469503338' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-07-26T13:37:11.206 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-07-26T13:37:11.206 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-07-26T13:37:11.460 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:37:11.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:11 smithi167 ceph-mon[37140]: pgmap v274: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-07-26T13:37:11.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:11 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/2469503338' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-07-26T13:37:11.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:11 smithi167 ceph-mon[37140]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-07-26T13:37:11.848 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:11 smithi100 ceph-mon[32599]: pgmap v274: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-07-26T13:37:11.848 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:11 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2469503338' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-07-26T13:37:11.848 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:11 smithi100 ceph-mon[32599]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-07-26T13:37:11.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:11 smithi138 ceph-mon[36954]: pgmap v274: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-07-26T13:37:11.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:11 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/2469503338' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-07-26T13:37:11.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:11 smithi138 ceph-mon[36954]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-07-26T13:37:13.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.223+0000 7fb44bec2640 1 -- 172.21.15.100:0/3278794710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb444103660 msgr2=0x7fb444103a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:13.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.223+0000 7fb44bec2640 1 --2- 172.21.15.100:0/3278794710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb444103660 0x7fb444103a70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb434009ca0 tx=0x7fb43402a8c0 comp rx=0 tx=0).stop 2024-07-26T13:37:13.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.223+0000 7fb44bec2640 1 -- 172.21.15.100:0/3278794710 shutdown_connections 2024-07-26T13:37:13.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.223+0000 7fb44bec2640 1 --2- 172.21.15.100:0/3278794710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4441098f0 0x7fb444110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:13.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.223+0000 7fb44bec2640 1 --2- 172.21.15.100:0/3278794710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb444108fa0 0x7fb4441093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:13.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.223+0000 7fb44bec2640 1 --2- 172.21.15.100:0/3278794710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb444103660 0x7fb444103a70 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:13.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.223+0000 7fb44bec2640 1 -- 172.21.15.100:0/3278794710 >> 172.21.15.100:0/3278794710 conn(0x7fb44407b300 msgr2=0x7fb44407b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:13.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.223+0000 7fb44bec2640 1 -- 172.21.15.100:0/3278794710 shutdown_connections 2024-07-26T13:37:13.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.223+0000 7fb44bec2640 1 -- 172.21.15.100:0/3278794710 wait complete. 2024-07-26T13:37:13.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.224+0000 7fb44bec2640 1 Processor -- start 2024-07-26T13:37:13.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.224+0000 7fb44bec2640 1 -- start start 2024-07-26T13:37:13.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.224+0000 7fb44bec2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb444103660 0x7fb444199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:13.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.224+0000 7fb44bec2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb444108fa0 0x7fb44419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:13.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.224+0000 7fb44bec2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4441098f0 0x7fb44419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:13.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.224+0000 7fb44bec2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb444113520 con 0x7fb444108fa0 2024-07-26T13:37:13.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.224+0000 7fb44bec2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb4441133a0 con 0x7fb4441098f0 2024-07-26T13:37:13.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.224+0000 7fb44bec2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb4441136a0 con 0x7fb444103660 2024-07-26T13:37:13.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.224+0000 7fb44945d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb444108fa0 0x7fb44419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:13.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.224+0000 7fb44a45f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4441098f0 0x7fb44419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:13.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.224+0000 7fb44945d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb444108fa0 0x7fb44419a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51486/0 (socket says 172.21.15.100:51486) 2024-07-26T13:37:13.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.224+0000 7fb44945d640 1 -- 172.21.15.100:0/4186008262 learned_addr learned my addr 172.21.15.100:0/4186008262 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:37:13.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.224+0000 7fb449c5e640 1 --2- 172.21.15.100:0/4186008262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb444103660 0x7fb444199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:13.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.225+0000 7fb44945d640 1 -- 172.21.15.100:0/4186008262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb444103660 msgr2=0x7fb444199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:13.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.225+0000 7fb44945d640 1 --2- 172.21.15.100:0/4186008262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb444103660 0x7fb444199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:13.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.225+0000 7fb44945d640 1 -- 172.21.15.100:0/4186008262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4441098f0 msgr2=0x7fb44419f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:13.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.225+0000 7fb44945d640 1 --2- 172.21.15.100:0/4186008262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4441098f0 0x7fb44419f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:13.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.225+0000 7fb44945d640 1 -- 172.21.15.100:0/4186008262 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb44419fde0 con 0x7fb444108fa0 2024-07-26T13:37:13.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.225+0000 7fb44945d640 1 --2- 172.21.15.100:0/4186008262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb444108fa0 0x7fb44419a4d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb43800c970 tx=0x7fb43800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:13.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.225+0000 7fb442ffd640 1 -- 172.21.15.100:0/4186008262 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb438004220 con 0x7fb444108fa0 2024-07-26T13:37:13.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.225+0000 7fb442ffd640 1 -- 172.21.15.100:0/4186008262 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4380043c0 con 0x7fb444108fa0 2024-07-26T13:37:13.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.225+0000 7fb44bec2640 1 -- 172.21.15.100:0/4186008262 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4441a00d0 con 0x7fb444108fa0 2024-07-26T13:37:13.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.225+0000 7fb442ffd640 1 -- 172.21.15.100:0/4186008262 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb438004a00 con 0x7fb444108fa0 2024-07-26T13:37:13.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.225+0000 7fb44bec2640 1 -- 172.21.15.100:0/4186008262 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4441a68f0 con 0x7fb444108fa0 2024-07-26T13:37:13.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.226+0000 7fb44bec2640 1 -- 172.21.15.100:0/4186008262 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb444103a70 con 0x7fb444108fa0 2024-07-26T13:37:13.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.229+0000 7fb442ffd640 1 -- 172.21.15.100:0/4186008262 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7fb4380079b0 con 0x7fb444108fa0 2024-07-26T13:37:13.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.229+0000 7fb442ffd640 1 --2- 172.21.15.100:0/4186008262 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb42c0b3cd0 0x7fb42c0b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:13.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.229+0000 7fb449c5e640 1 --2- 172.21.15.100:0/4186008262 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb42c0b3cd0 0x7fb42c0b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:13.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.229+0000 7fb442ffd640 1 -- 172.21.15.100:0/4186008262 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8194+0+0 (secure 0 0 0) 0x7fb4380e3cf0 con 0x7fb444108fa0 2024-07-26T13:37:13.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.229+0000 7fb449c5e640 1 --2- 172.21.15.100:0/4186008262 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb42c0b3cd0 0x7fb42c0b61e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb434009a60 tx=0x7fb4340093e0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:13.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.232+0000 7fb442ffd640 1 -- 172.21.15.100:0/4186008262 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fb438014030 con 0x7fb444108fa0 2024-07-26T13:37:13.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.439+0000 7fb44bec2640 1 -- 172.21.15.100:0/4186008262 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fb44410b670 con 0x7fb444108fa0 2024-07-26T13:37:13.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.515+0000 7fb442ffd640 1 -- 172.21.15.100:0/4186008262 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7fb4380aacb0 con 0x7fb444108fa0 2024-07-26T13:37:13.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.517+0000 7fb44bec2640 1 -- 172.21.15.100:0/4186008262 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb42c0b3cd0 msgr2=0x7fb42c0b61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:13.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.517+0000 7fb44bec2640 1 --2- 172.21.15.100:0/4186008262 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb42c0b3cd0 0x7fb42c0b61e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb434009a60 tx=0x7fb4340093e0 comp rx=0 tx=0).stop 2024-07-26T13:37:13.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.517+0000 7fb44bec2640 1 -- 172.21.15.100:0/4186008262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb444108fa0 msgr2=0x7fb44419a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:13.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.517+0000 7fb44bec2640 1 --2- 172.21.15.100:0/4186008262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb444108fa0 0x7fb44419a4d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb43800c970 tx=0x7fb43800ce30 comp rx=0 tx=0).stop 2024-07-26T13:37:13.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.517+0000 7fb44bec2640 1 -- 172.21.15.100:0/4186008262 shutdown_connections 2024-07-26T13:37:13.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.517+0000 7fb44bec2640 1 --2- 172.21.15.100:0/4186008262 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb42c0b3cd0 0x7fb42c0b61e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:13.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.517+0000 7fb44bec2640 1 --2- 172.21.15.100:0/4186008262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4441098f0 0x7fb44419f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:13.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.517+0000 7fb44bec2640 1 --2- 172.21.15.100:0/4186008262 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb444108fa0 0x7fb44419a4d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:13.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.517+0000 7fb44bec2640 1 --2- 172.21.15.100:0/4186008262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb444103660 0x7fb444199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:13.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.517+0000 7fb44bec2640 1 -- 172.21.15.100:0/4186008262 >> 172.21.15.100:0/4186008262 conn(0x7fb44407b300 msgr2=0x7fb4441070a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:13.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.517+0000 7fb44bec2640 1 -- 172.21.15.100:0/4186008262 shutdown_connections 2024-07-26T13:37:13.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:13.517+0000 7fb44bec2640 1 -- 172.21.15.100:0/4186008262 wait complete. 2024-07-26T13:37:13.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:13 smithi100 ceph-mon[32599]: pgmap v275: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-07-26T13:37:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:13 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/4186008262' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-07-26T13:37:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:13 smithi167 ceph-mon[37140]: pgmap v275: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-07-26T13:37:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:13 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/4186008262' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-07-26T13:37:13.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:13 smithi138 ceph-mon[36954]: pgmap v275: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-07-26T13:37:13.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:13 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/4186008262' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-07-26T13:37:14.167 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-07-26T13:37:14.428 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:37:14.722 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:14 smithi100 ceph-mon[32599]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-07-26T13:37:14.722 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:14 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/4186008262' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-07-26T13:37:14.722 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:14 smithi100 ceph-mon[32599]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-07-26T13:37:14.722 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:14 smithi100 ceph-mon[32599]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-07-26T13:37:14.722 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:14 smithi100 ceph-mon[32599]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-07-26T13:37:14.722 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:14 smithi100 ceph-mon[32599]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-07-26T13:37:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:14 smithi167 ceph-mon[37140]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-07-26T13:37:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:14 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/4186008262' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-07-26T13:37:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:14 smithi167 ceph-mon[37140]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-07-26T13:37:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:14 smithi167 ceph-mon[37140]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-07-26T13:37:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:14 smithi167 ceph-mon[37140]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-07-26T13:37:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:14 smithi167 ceph-mon[37140]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-07-26T13:37:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:14 smithi138 ceph-mon[36954]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-07-26T13:37:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:14 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/4186008262' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-07-26T13:37:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:14 smithi138 ceph-mon[36954]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-07-26T13:37:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:14 smithi138 ceph-mon[36954]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-07-26T13:37:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:14 smithi138 ceph-mon[36954]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-07-26T13:37:14.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:14 smithi138 ceph-mon[36954]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-07-26T13:37:15.512 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:37:15 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 2024-07-26T13:37:15.238+0000 7f740ace1640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-07-26T13:37:15.632 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:15 smithi100 ceph-mon[32599]: pgmap v276: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-07-26T13:37:15.632 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:15 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] up:active 2024-07-26T13:37:15.632 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:15 smithi100 ceph-mon[32599]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-07-26T13:37:15.632 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:15 smithi100 ceph-mon[32599]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-07-26T13:37:15.632 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:15 smithi100 ceph-mon[32599]: Cluster is now healthy 2024-07-26T13:37:15.632 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:15 smithi100 ceph-mon[32599]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-07-26T13:37:15.632 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:15 smithi100 ceph-mon[32599]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-07-26T13:37:15.632 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:15 smithi100 ceph-mon[32599]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-07-26T13:37:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:15 smithi167 ceph-mon[37140]: pgmap v276: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-07-26T13:37:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:15 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] up:active 2024-07-26T13:37:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:15 smithi167 ceph-mon[37140]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-07-26T13:37:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:15 smithi167 ceph-mon[37140]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-07-26T13:37:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:15 smithi167 ceph-mon[37140]: Cluster is now healthy 2024-07-26T13:37:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:15 smithi167 ceph-mon[37140]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-07-26T13:37:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:15 smithi167 ceph-mon[37140]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-07-26T13:37:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:15 smithi167 ceph-mon[37140]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-07-26T13:37:15.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:15 smithi138 ceph-mon[36954]: pgmap v276: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-07-26T13:37:15.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:15 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] up:active 2024-07-26T13:37:15.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:15 smithi138 ceph-mon[36954]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-07-26T13:37:15.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:15 smithi138 ceph-mon[36954]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-07-26T13:37:15.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:15 smithi138 ceph-mon[36954]: Cluster is now healthy 2024-07-26T13:37:15.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:15 smithi138 ceph-mon[36954]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-07-26T13:37:15.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:15 smithi138 ceph-mon[36954]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-07-26T13:37:15.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:15 smithi138 ceph-mon[36954]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-07-26T13:37:16.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.144+0000 7f732e015640 1 -- 172.21.15.100:0/1426113297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73281024c0 msgr2=0x7f73281028d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:16.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.145+0000 7f732e015640 1 --2- 172.21.15.100:0/1426113297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73281024c0 0x7f73281028d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7318009900 tx=0x7f731802a410 comp rx=0 tx=0).stop 2024-07-26T13:37:16.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.145+0000 7f732e015640 1 -- 172.21.15.100:0/1426113297 shutdown_connections 2024-07-26T13:37:16.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.145+0000 7f732e015640 1 --2- 172.21.15.100:0/1426113297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7328108750 0x7f732810ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:16.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.145+0000 7f732e015640 1 --2- 172.21.15.100:0/1426113297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7328107e00 0x7f7328108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:16.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.145+0000 7f732e015640 1 --2- 172.21.15.100:0/1426113297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f73281024c0 0x7f73281028d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:16.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.145+0000 7f732e015640 1 -- 172.21.15.100:0/1426113297 >> 172.21.15.100:0/1426113297 conn(0x7f7328079180 msgr2=0x7f7328079590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:16.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.145+0000 7f732e015640 1 -- 172.21.15.100:0/1426113297 shutdown_connections 2024-07-26T13:37:16.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.145+0000 7f732e015640 1 -- 172.21.15.100:0/1426113297 wait complete. 2024-07-26T13:37:16.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.145+0000 7f732e015640 1 Processor -- start 2024-07-26T13:37:16.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.146+0000 7f732e015640 1 -- start start 2024-07-26T13:37:16.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.146+0000 7f732e015640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73281024c0 0x7f7328198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:16.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.146+0000 7f732e015640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7328107e00 0x7f7328199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:16.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.146+0000 7f732e015640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7328108750 0x7f732819e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:16.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.146+0000 7f732e015640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7328112420 con 0x7f7328108750 2024-07-26T13:37:16.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.146+0000 7f732e015640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f73281122a0 con 0x7f7328107e00 2024-07-26T13:37:16.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.146+0000 7f732e015640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f73281125a0 con 0x7f73281024c0 2024-07-26T13:37:16.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.146+0000 7f7327fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7328108750 0x7f732819e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:16.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.146+0000 7f7327fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7328108750 0x7f732819e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33334/0 (socket says 172.21.15.100:33334) 2024-07-26T13:37:16.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.146+0000 7f7327fff640 1 -- 172.21.15.100:0/3672740126 learned_addr learned my addr 172.21.15.100:0/3672740126 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:37:16.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.146+0000 7f73277fe640 1 --2- 172.21.15.100:0/3672740126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73281024c0 0x7f7328198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:16.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.146+0000 7f7326ffd640 1 --2- 172.21.15.100:0/3672740126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7328107e00 0x7f7328199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:16.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.146+0000 7f7327fff640 1 -- 172.21.15.100:0/3672740126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73281024c0 msgr2=0x7f7328198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:16.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.146+0000 7f7327fff640 1 --2- 172.21.15.100:0/3672740126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73281024c0 0x7f7328198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:16.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.146+0000 7f7327fff640 1 -- 172.21.15.100:0/3672740126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7328107e00 msgr2=0x7f7328199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:16.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.146+0000 7f7327fff640 1 --2- 172.21.15.100:0/3672740126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7328107e00 0x7f7328199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:16.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.146+0000 7f7327fff640 1 -- 172.21.15.100:0/3672740126 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732819ed60 con 0x7f7328108750 2024-07-26T13:37:16.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.146+0000 7f7326ffd640 1 --2- 172.21.15.100:0/3672740126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7328107e00 0x7f7328199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:37:16.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.147+0000 7f7327fff640 1 --2- 172.21.15.100:0/3672740126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7328108750 0x7f732819e6f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f732000af10 tx=0x7f732000cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:16.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.147+0000 7f7324ff9640 1 -- 172.21.15.100:0/3672740126 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732001c070 con 0x7f7328108750 2024-07-26T13:37:16.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.147+0000 7f732e015640 1 -- 172.21.15.100:0/3672740126 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f732819f050 con 0x7f7328108750 2024-07-26T13:37:16.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.147+0000 7f7324ff9640 1 -- 172.21.15.100:0/3672740126 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7320007bb0 con 0x7f7328108750 2024-07-26T13:37:16.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.147+0000 7f7324ff9640 1 -- 172.21.15.100:0/3672740126 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73200176e0 con 0x7f7328108750 2024-07-26T13:37:16.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.147+0000 7f732e015640 1 -- 172.21.15.100:0/3672740126 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f732806b5b0 con 0x7f7328108750 2024-07-26T13:37:16.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.148+0000 7f732e015640 1 -- 172.21.15.100:0/3672740126 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7328103740 con 0x7f7328108750 2024-07-26T13:37:16.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.150+0000 7f7324ff9640 1 -- 172.21.15.100:0/3672740126 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f732001f070 con 0x7f7328108750 2024-07-26T13:37:16.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.151+0000 7f7324ff9640 1 --2- 172.21.15.100:0/3672740126 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f73100b3c50 0x7f73100b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:16.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.151+0000 7f73277fe640 1 --2- 172.21.15.100:0/3672740126 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f73100b3c50 0x7f73100b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:16.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.151+0000 7f7324ff9640 1 -- 172.21.15.100:0/3672740126 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8194+0+0 (secure 0 0 0) 0x7f73200e5970 con 0x7f7328108750 2024-07-26T13:37:16.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.151+0000 7f73277fe640 1 --2- 172.21.15.100:0/3672740126 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f73100b3c50 0x7f73100b6160 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7318031440 tx=0x7f7318005d50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:16.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.154+0000 7f7324ff9640 1 -- 172.21.15.100:0/3672740126 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f7320026080 con 0x7f7328108750 2024-07-26T13:37:16.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.376+0000 7f732e015640 1 -- 172.21.15.100:0/3672740126 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f73281028d0 con 0x7f7328108750 2024-07-26T13:37:16.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.538+0000 7f7324ff9640 1 -- 172.21.15.100:0/3672740126 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f73281028d0 con 0x7f7328108750 2024-07-26T13:37:16.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.541+0000 7f732e015640 1 -- 172.21.15.100:0/3672740126 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f73100b3c50 msgr2=0x7f73100b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:16.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.541+0000 7f732e015640 1 --2- 172.21.15.100:0/3672740126 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f73100b3c50 0x7f73100b6160 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7318031440 tx=0x7f7318005d50 comp rx=0 tx=0).stop 2024-07-26T13:37:16.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.541+0000 7f732e015640 1 -- 172.21.15.100:0/3672740126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7328108750 msgr2=0x7f732819e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:16.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.541+0000 7f732e015640 1 --2- 172.21.15.100:0/3672740126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7328108750 0x7f732819e6f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f732000af10 tx=0x7f732000cd40 comp rx=0 tx=0).stop 2024-07-26T13:37:16.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.542+0000 7f732e015640 1 -- 172.21.15.100:0/3672740126 shutdown_connections 2024-07-26T13:37:16.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.542+0000 7f732e015640 1 --2- 172.21.15.100:0/3672740126 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f73100b3c50 0x7f73100b6160 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:16.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.542+0000 7f732e015640 1 --2- 172.21.15.100:0/3672740126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7328108750 0x7f732819e6f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:16.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.542+0000 7f732e015640 1 --2- 172.21.15.100:0/3672740126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7328107e00 0x7f7328199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:16.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.542+0000 7f732e015640 1 --2- 172.21.15.100:0/3672740126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73281024c0 0x7f7328198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:16.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.542+0000 7f732e015640 1 -- 172.21.15.100:0/3672740126 >> 172.21.15.100:0/3672740126 conn(0x7f7328079180 msgr2=0x7f7328100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:16.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.542+0000 7f732e015640 1 -- 172.21.15.100:0/3672740126 shutdown_connections 2024-07-26T13:37:16.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:16.542+0000 7f732e015640 1 -- 172.21.15.100:0/3672740126 wait complete. 2024-07-26T13:37:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:16 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] up:active 2024-07-26T13:37:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:16 smithi100 ceph-mon[32599]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-07-26T13:37:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-07-26T13:37:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-07-26T13:37:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]: dispatch 2024-07-26T13:37:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-07-26T13:37:16.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-07-26T13:37:16.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-07-26T13:37:16.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-07-26T13:37:16.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-07-26T13:37:16.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-07-26T13:37:16.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-07-26T13:37:16.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:16 smithi100 ceph-mon[32599]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 1 op/s 2024-07-26T13:37:16.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:16 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/3672740126' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-07-26T13:37:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:16 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] up:active 2024-07-26T13:37:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:16 smithi138 ceph-mon[36954]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-07-26T13:37:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-07-26T13:37:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-07-26T13:37:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]: dispatch 2024-07-26T13:37:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-07-26T13:37:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-07-26T13:37:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-07-26T13:37:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-07-26T13:37:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-07-26T13:37:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-07-26T13:37:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-07-26T13:37:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:16 smithi138 ceph-mon[36954]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 1 op/s 2024-07-26T13:37:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:16 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/3672740126' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-07-26T13:37:17.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:16 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] up:active 2024-07-26T13:37:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:16 smithi167 ceph-mon[37140]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-07-26T13:37:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-07-26T13:37:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-07-26T13:37:17.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]: dispatch 2024-07-26T13:37:17.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-07-26T13:37:17.027 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-07-26T13:37:17.027 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-07-26T13:37:17.027 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-07-26T13:37:17.027 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-07-26T13:37:17.027 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-07-26T13:37:17.027 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-07-26T13:37:17.027 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:16 smithi167 ceph-mon[37140]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 1 op/s 2024-07-26T13:37:17.027 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:16 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/3672740126' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-07-26T13:37:17.167 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-07-26T13:37:17.421 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:37:17.828 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:17 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/3672740126' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-07-26T13:37:17.828 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-07-26T13:37:17.828 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-07-26T13:37:17.828 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2024-07-26T13:37:17.828 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-07-26T13:37:17.828 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-07-26T13:37:17.828 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-07-26T13:37:17.829 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-07-26T13:37:17.829 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-07-26T13:37:17.829 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-07-26T13:37:17.829 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-07-26T13:37:17.829 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:17 smithi100 ceph-mon[32599]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-07-26T13:37:17.829 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:17 smithi100 ceph-mon[32599]: osdmap e80: 12 total, 12 up, 12 in 2024-07-26T13:37:17.829 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:17 smithi100 ceph-mon[32599]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:37:17.924 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:17 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/3672740126' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-07-26T13:37:17.924 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-07-26T13:37:17.924 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-07-26T13:37:17.924 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2024-07-26T13:37:17.924 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-07-26T13:37:17.924 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-07-26T13:37:17.924 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-07-26T13:37:17.924 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-07-26T13:37:17.925 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-07-26T13:37:17.925 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-07-26T13:37:17.925 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-07-26T13:37:17.925 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:17 smithi167 ceph-mon[37140]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-07-26T13:37:17.925 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:17 smithi167 ceph-mon[37140]: osdmap e80: 12 total, 12 up, 12 in 2024-07-26T13:37:17.925 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:17 smithi167 ceph-mon[37140]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:37:17.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:17 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/3672740126' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-07-26T13:37:17.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-07-26T13:37:17.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-07-26T13:37:17.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2024-07-26T13:37:17.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-07-26T13:37:17.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-07-26T13:37:17.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-07-26T13:37:17.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-07-26T13:37:17.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-07-26T13:37:17.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-07-26T13:37:17.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-07-26T13:37:17.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:17 smithi138 ceph-mon[36954]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-07-26T13:37:17.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:17 smithi138 ceph-mon[36954]: osdmap e80: 12 total, 12 up, 12 in 2024-07-26T13:37:17.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:17 smithi138 ceph-mon[36954]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:37:18.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:18 smithi100 ceph-mon[32599]: osdmap e81: 12 total, 12 up, 12 in 2024-07-26T13:37:18.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:18 smithi100 ceph-mon[32599]: pgmap v280: 97 pgs: 4 peering, 93 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 895 B/s wr, 5 op/s 2024-07-26T13:37:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:18 smithi138 ceph-mon[36954]: osdmap e81: 12 total, 12 up, 12 in 2024-07-26T13:37:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:18 smithi138 ceph-mon[36954]: pgmap v280: 97 pgs: 4 peering, 93 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 895 B/s wr, 5 op/s 2024-07-26T13:37:19.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:18 smithi167 ceph-mon[37140]: osdmap e81: 12 total, 12 up, 12 in 2024-07-26T13:37:19.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:18 smithi167 ceph-mon[37140]: pgmap v280: 97 pgs: 4 peering, 93 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 895 B/s wr, 5 op/s 2024-07-26T13:37:19.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.053+0000 7fb95d865640 1 -- 172.21.15.100:0/2211030934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb958107dd0 msgr2=0x7fb9581081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:19.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.053+0000 7fb95d865640 1 --2- 172.21.15.100:0/2211030934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb958107dd0 0x7fb9581081e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb940009900 tx=0x7fb94002a410 comp rx=0 tx=0).stop 2024-07-26T13:37:19.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.053+0000 7fb95d865640 1 -- 172.21.15.100:0/2211030934 shutdown_connections 2024-07-26T13:37:19.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.053+0000 7fb95d865640 1 --2- 172.21.15.100:0/2211030934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb958108720 0x7fb95810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:19.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.053+0000 7fb95d865640 1 --2- 172.21.15.100:0/2211030934 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb958107dd0 0x7fb9581081e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:19.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.053+0000 7fb95d865640 1 --2- 172.21.15.100:0/2211030934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb958102490 0x7fb9581028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:19.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.053+0000 7fb95d865640 1 -- 172.21.15.100:0/2211030934 >> 172.21.15.100:0/2211030934 conn(0x7fb958079160 msgr2=0x7fb958079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:19.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.053+0000 7fb95d865640 1 -- 172.21.15.100:0/2211030934 shutdown_connections 2024-07-26T13:37:19.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.053+0000 7fb95d865640 1 -- 172.21.15.100:0/2211030934 wait complete. 2024-07-26T13:37:19.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.054+0000 7fb95d865640 1 Processor -- start 2024-07-26T13:37:19.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.054+0000 7fb95d865640 1 -- start start 2024-07-26T13:37:19.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.055+0000 7fb95d865640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb958102490 0x7fb958198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:19.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.055+0000 7fb95d865640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb958107dd0 0x7fb958199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:19.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.055+0000 7fb95d865640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb958108720 0x7fb95819e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:19.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.055+0000 7fb95d865640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb9581123c0 con 0x7fb958102490 2024-07-26T13:37:19.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.055+0000 7fb95d865640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb958112240 con 0x7fb958108720 2024-07-26T13:37:19.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.055+0000 7fb95d865640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb958112540 con 0x7fb958107dd0 2024-07-26T13:37:19.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.055+0000 7fb956ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb958102490 0x7fb958198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:19.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.055+0000 7fb956ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb958102490 0x7fb958198d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33346/0 (socket says 172.21.15.100:33346) 2024-07-26T13:37:19.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.055+0000 7fb956ffd640 1 -- 172.21.15.100:0/174562369 learned_addr learned my addr 172.21.15.100:0/174562369 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:37:19.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.055+0000 7fb94e7fc640 1 --2- 172.21.15.100:0/174562369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb958107dd0 0x7fb958199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:19.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.055+0000 7fb9577fe640 1 --2- 172.21.15.100:0/174562369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb958108720 0x7fb95819e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:19.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.055+0000 7fb956ffd640 1 -- 172.21.15.100:0/174562369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb958107dd0 msgr2=0x7fb958199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:19.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.055+0000 7fb956ffd640 1 --2- 172.21.15.100:0/174562369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb958107dd0 0x7fb958199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:19.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.055+0000 7fb956ffd640 1 -- 172.21.15.100:0/174562369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb958108720 msgr2=0x7fb95819e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:19.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.055+0000 7fb956ffd640 1 --2- 172.21.15.100:0/174562369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb958108720 0x7fb95819e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:19.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.055+0000 7fb956ffd640 1 -- 172.21.15.100:0/174562369 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb95819ed00 con 0x7fb958102490 2024-07-26T13:37:19.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.055+0000 7fb956ffd640 1 --2- 172.21.15.100:0/174562369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb958102490 0x7fb958198d90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb94800d770 tx=0x7fb94800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:19.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.055+0000 7fb95c863640 1 -- 172.21.15.100:0/174562369 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb948017070 con 0x7fb958102490 2024-07-26T13:37:19.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.055+0000 7fb95d865640 1 -- 172.21.15.100:0/174562369 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb95819eff0 con 0x7fb958102490 2024-07-26T13:37:19.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.055+0000 7fb95c863640 1 -- 172.21.15.100:0/174562369 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9480054d0 con 0x7fb958102490 2024-07-26T13:37:19.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.055+0000 7fb95c863640 1 -- 172.21.15.100:0/174562369 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb948004c30 con 0x7fb958102490 2024-07-26T13:37:19.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.055+0000 7fb95d865640 1 -- 172.21.15.100:0/174562369 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9581a5750 con 0x7fb958102490 2024-07-26T13:37:19.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.056+0000 7fb95d865640 1 -- 172.21.15.100:0/174562369 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb958103710 con 0x7fb958102490 2024-07-26T13:37:19.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.058+0000 7fb95c863640 1 -- 172.21.15.100:0/174562369 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7fb948004dd0 con 0x7fb958102490 2024-07-26T13:37:19.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.058+0000 7fb95c863640 1 --2- 172.21.15.100:0/174562369 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb9440b3c20 0x7fb9440b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:19.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.059+0000 7fb94e7fc640 1 --2- 172.21.15.100:0/174562369 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb9440b3c20 0x7fb9440b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:19.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.059+0000 7fb95c863640 1 -- 172.21.15.100:0/174562369 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fb9480e3cb0 con 0x7fb958102490 2024-07-26T13:37:19.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.059+0000 7fb94e7fc640 1 --2- 172.21.15.100:0/174562369 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb9440b3c20 0x7fb9440b6130 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb940006010 tx=0x7fb940031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:19.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.062+0000 7fb95c863640 1 -- 172.21.15.100:0/174562369 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fb9480ab200 con 0x7fb958102490 2024-07-26T13:37:19.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.270+0000 7fb95d865640 1 -- 172.21.15.100:0/174562369 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fb95810a540 con 0x7fb958102490 2024-07-26T13:37:19.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.563+0000 7fb95c863640 1 -- 172.21.15.100:0/174562369 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7fb9480aab00 con 0x7fb958102490 2024-07-26T13:37:19.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.566+0000 7fb95d865640 1 -- 172.21.15.100:0/174562369 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb9440b3c20 msgr2=0x7fb9440b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:19.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.566+0000 7fb95d865640 1 --2- 172.21.15.100:0/174562369 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb9440b3c20 0x7fb9440b6130 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb940006010 tx=0x7fb940031400 comp rx=0 tx=0).stop 2024-07-26T13:37:19.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.566+0000 7fb95d865640 1 -- 172.21.15.100:0/174562369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb958102490 msgr2=0x7fb958198d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:19.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.566+0000 7fb95d865640 1 --2- 172.21.15.100:0/174562369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb958102490 0x7fb958198d90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb94800d770 tx=0x7fb94800dc30 comp rx=0 tx=0).stop 2024-07-26T13:37:19.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.566+0000 7fb95d865640 1 -- 172.21.15.100:0/174562369 shutdown_connections 2024-07-26T13:37:19.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.566+0000 7fb95d865640 1 --2- 172.21.15.100:0/174562369 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb9440b3c20 0x7fb9440b6130 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:19.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.566+0000 7fb95d865640 1 --2- 172.21.15.100:0/174562369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb958108720 0x7fb95819e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:19.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.566+0000 7fb95d865640 1 --2- 172.21.15.100:0/174562369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb958107dd0 0x7fb958199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:19.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.566+0000 7fb95d865640 1 --2- 172.21.15.100:0/174562369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb958102490 0x7fb958198d90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:19.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.566+0000 7fb95d865640 1 -- 172.21.15.100:0/174562369 >> 172.21.15.100:0/174562369 conn(0x7fb958079160 msgr2=0x7fb958104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:19.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.566+0000 7fb95d865640 1 -- 172.21.15.100:0/174562369 shutdown_connections 2024-07-26T13:37:19.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:19.566+0000 7fb95d865640 1 -- 172.21.15.100:0/174562369 wait complete. 2024-07-26T13:37:19.908 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:19 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/174562369' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-07-26T13:37:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:19 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/174562369' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-07-26T13:37:20.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:19 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/174562369' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-07-26T13:37:20.347 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-07-26T13:37:20.347 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-07-26T13:37:20.606 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:37:20.871 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:20 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/174562369' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-07-26T13:37:20.871 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:20 smithi100 ceph-mon[32599]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:37:20.871 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:20 smithi100 ceph-mon[32599]: pgmap v281: 97 pgs: 4 peering, 93 active+clean; 582 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 3.0 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-07-26T13:37:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:20 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/174562369' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-07-26T13:37:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:20 smithi138 ceph-mon[36954]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:37:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:20 smithi138 ceph-mon[36954]: pgmap v281: 97 pgs: 4 peering, 93 active+clean; 582 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 3.0 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-07-26T13:37:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:20 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/174562369' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-07-26T13:37:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:20 smithi167 ceph-mon[37140]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:37:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:20 smithi167 ceph-mon[37140]: pgmap v281: 97 pgs: 4 peering, 93 active+clean; 582 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 3.0 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-07-26T13:37:22.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.356+0000 7fee58156640 1 -- 172.21.15.100:0/3150031406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee50103590 msgr2=0x7fee501039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:22.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.356+0000 7fee58156640 1 --2- 172.21.15.100:0/3150031406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee50103590 0x7fee501039a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009ca0 tx=0x7fee4c02a8e0 comp rx=0 tx=0).stop 2024-07-26T13:37:22.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.356+0000 7fee58156640 1 -- 172.21.15.100:0/3150031406 shutdown_connections 2024-07-26T13:37:22.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.356+0000 7fee58156640 1 --2- 172.21.15.100:0/3150031406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee50109820 0x7fee50110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:22.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.356+0000 7fee58156640 1 --2- 172.21.15.100:0/3150031406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee50108ed0 0x7fee501092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:22.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.356+0000 7fee58156640 1 --2- 172.21.15.100:0/3150031406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee50103590 0x7fee501039a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:22.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.356+0000 7fee58156640 1 -- 172.21.15.100:0/3150031406 >> 172.21.15.100:0/3150031406 conn(0x7fee5007b280 msgr2=0x7fee5007b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:22.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.356+0000 7fee58156640 1 -- 172.21.15.100:0/3150031406 shutdown_connections 2024-07-26T13:37:22.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.356+0000 7fee58156640 1 -- 172.21.15.100:0/3150031406 wait complete. 2024-07-26T13:37:22.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.357+0000 7fee58156640 1 Processor -- start 2024-07-26T13:37:22.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.357+0000 7fee58156640 1 -- start start 2024-07-26T13:37:22.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.357+0000 7fee58156640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee50103590 0x7fee50199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:22.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.357+0000 7fee58156640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee50108ed0 0x7fee5019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:22.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.357+0000 7fee58156640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee50109820 0x7fee5019f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:22.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.357+0000 7fee58156640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fee501135a0 con 0x7fee50108ed0 2024-07-26T13:37:22.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.357+0000 7fee58156640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fee50113420 con 0x7fee50103590 2024-07-26T13:37:22.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.357+0000 7fee58156640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fee50113720 con 0x7fee50109820 2024-07-26T13:37:22.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.357+0000 7fee556f1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee50108ed0 0x7fee5019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:22.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.357+0000 7fee556f1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee50108ed0 0x7fee5019a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33364/0 (socket says 172.21.15.100:33364) 2024-07-26T13:37:22.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.357+0000 7fee556f1640 1 -- 172.21.15.100:0/4030030123 learned_addr learned my addr 172.21.15.100:0/4030030123 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:37:22.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.357+0000 7fee55ef2640 1 --2- 172.21.15.100:0/4030030123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee50103590 0x7fee50199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:22.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.357+0000 7fee566f3640 1 --2- 172.21.15.100:0/4030030123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee50109820 0x7fee5019f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:22.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.357+0000 7fee556f1640 1 -- 172.21.15.100:0/4030030123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee50109820 msgr2=0x7fee5019f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:22.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.357+0000 7fee556f1640 1 --2- 172.21.15.100:0/4030030123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee50109820 0x7fee5019f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:22.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.357+0000 7fee556f1640 1 -- 172.21.15.100:0/4030030123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee50103590 msgr2=0x7fee50199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:22.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.357+0000 7fee556f1640 1 --2- 172.21.15.100:0/4030030123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee50103590 0x7fee50199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:22.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.357+0000 7fee556f1640 1 -- 172.21.15.100:0/4030030123 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee5019ff60 con 0x7fee50108ed0 2024-07-26T13:37:22.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.358+0000 7fee556f1640 1 --2- 172.21.15.100:0/4030030123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee50108ed0 0x7fee5019a590 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fee40006fd0 tx=0x7fee4000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:22.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.358+0000 7fee46ffd640 1 -- 172.21.15.100:0/4030030123 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee40011830 con 0x7fee50108ed0 2024-07-26T13:37:22.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.358+0000 7fee46ffd640 1 -- 172.21.15.100:0/4030030123 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee40016070 con 0x7fee50108ed0 2024-07-26T13:37:22.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.358+0000 7fee58156640 1 -- 172.21.15.100:0/4030030123 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee501a0250 con 0x7fee50108ed0 2024-07-26T13:37:22.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.358+0000 7fee46ffd640 1 -- 172.21.15.100:0/4030030123 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee40010e00 con 0x7fee50108ed0 2024-07-26T13:37:22.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.358+0000 7fee58156640 1 -- 172.21.15.100:0/4030030123 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee501a69b0 con 0x7fee50108ed0 2024-07-26T13:37:22.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.359+0000 7fee58156640 1 -- 172.21.15.100:0/4030030123 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee50103a20 con 0x7fee50108ed0 2024-07-26T13:37:22.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.360+0000 7fee46ffd640 1 -- 172.21.15.100:0/4030030123 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7fee4001b460 con 0x7fee50108ed0 2024-07-26T13:37:22.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.361+0000 7fee46ffd640 1 --2- 172.21.15.100:0/4030030123 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fee3c0b3c20 0x7fee3c0b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:22.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.361+0000 7fee55ef2640 1 --2- 172.21.15.100:0/4030030123 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fee3c0b3c20 0x7fee3c0b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:22.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.361+0000 7fee46ffd640 1 -- 172.21.15.100:0/4030030123 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fee400e3a70 con 0x7fee50108ed0 2024-07-26T13:37:22.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.361+0000 7fee55ef2640 1 --2- 172.21.15.100:0/4030030123 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fee3c0b3c20 0x7fee3c0b6130 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009a60 tx=0x7fee4c0093e0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:22.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.364+0000 7fee46ffd640 1 -- 172.21.15.100:0/4030030123 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fee400aafc0 con 0x7fee50108ed0 2024-07-26T13:37:22.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.565+0000 7fee58156640 1 -- 172.21.15.100:0/4030030123 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fee50063230 con 0x7fee3c0b3c20 2024-07-26T13:37:22.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.594+0000 7fee46ffd640 1 -- 172.21.15.100:0/4030030123 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fee50063230 con 0x7fee3c0b3c20 2024-07-26T13:37:22.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.596+0000 7fee58156640 1 -- 172.21.15.100:0/4030030123 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fee3c0b3c20 msgr2=0x7fee3c0b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:22.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.596+0000 7fee58156640 1 --2- 172.21.15.100:0/4030030123 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fee3c0b3c20 0x7fee3c0b6130 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009a60 tx=0x7fee4c0093e0 comp rx=0 tx=0).stop 2024-07-26T13:37:22.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.596+0000 7fee58156640 1 -- 172.21.15.100:0/4030030123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee50108ed0 msgr2=0x7fee5019a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:22.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.596+0000 7fee58156640 1 --2- 172.21.15.100:0/4030030123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee50108ed0 0x7fee5019a590 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fee40006fd0 tx=0x7fee4000b8e0 comp rx=0 tx=0).stop 2024-07-26T13:37:22.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.596+0000 7fee58156640 1 -- 172.21.15.100:0/4030030123 shutdown_connections 2024-07-26T13:37:22.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.596+0000 7fee58156640 1 --2- 172.21.15.100:0/4030030123 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fee3c0b3c20 0x7fee3c0b6130 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:22.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.596+0000 7fee58156640 1 --2- 172.21.15.100:0/4030030123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee50109820 0x7fee5019f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:22.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.596+0000 7fee58156640 1 --2- 172.21.15.100:0/4030030123 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee50108ed0 0x7fee5019a590 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:22.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.596+0000 7fee58156640 1 --2- 172.21.15.100:0/4030030123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee50103590 0x7fee50199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:22.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.596+0000 7fee58156640 1 -- 172.21.15.100:0/4030030123 >> 172.21.15.100:0/4030030123 conn(0x7fee5007b280 msgr2=0x7fee50107010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:22.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.596+0000 7fee58156640 1 -- 172.21.15.100:0/4030030123 shutdown_connections 2024-07-26T13:37:22.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:22.596+0000 7fee58156640 1 -- 172.21.15.100:0/4030030123 wait complete. 2024-07-26T13:37:22.994 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:37:22 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: 2024-07-26T13:37:22.571+0000 7f9e580bf640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-26T13:37:22.994 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:37:22 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: 2024-07-26T13:37:22.571+0000 7f9e580bf640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-26T13:37:22.994 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:37:22 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: 2024-07-26T13:37:22.571+0000 7f9e580bf640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-26T13:37:22.994 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:37:22 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: 2024-07-26T13:37:22.571+0000 7f9e580bf640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-26T13:37:22.994 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:37:22 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: 2024-07-26T13:37:22.571+0000 7f9e580bf640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-26T13:37:23.266 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-07-26T13:37:23.522 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:37:23.644 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:23 smithi100 ceph-mon[32599]: pgmap v282: 97 pgs: 2 peering, 95 active+clean; 582 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 3.0 KiB/s wr, 14 op/s; 77 B/s, 0 objects/s recovering 2024-07-26T13:37:23.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:23 smithi138 ceph-mon[36954]: pgmap v282: 97 pgs: 2 peering, 95 active+clean; 582 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 3.0 KiB/s wr, 14 op/s; 77 B/s, 0 objects/s recovering 2024-07-26T13:37:23.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:23 smithi167 ceph-mon[37140]: pgmap v282: 97 pgs: 2 peering, 95 active+clean; 582 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 3.0 KiB/s wr, 14 op/s; 77 B/s, 0 objects/s recovering 2024-07-26T13:37:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:24 smithi138 ceph-mon[36954]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:37:24.721 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:24 smithi100 ceph-mon[32599]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:37:24.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:24 smithi167 ceph-mon[37140]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:37:25.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.194+0000 7f538eda4640 1 -- 172.21.15.100:0/2798909700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53881087a0 msgr2=0x7f538810efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:25.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.194+0000 7f538eda4640 1 --2- 172.21.15.100:0/2798909700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53881087a0 0x7f538810efd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5380009ca0 tx=0x7f538002a8c0 comp rx=0 tx=0).stop 2024-07-26T13:37:25.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.194+0000 7f538eda4640 1 -- 172.21.15.100:0/2798909700 shutdown_connections 2024-07-26T13:37:25.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.194+0000 7f538eda4640 1 --2- 172.21.15.100:0/2798909700 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53881087a0 0x7f538810efd0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:25.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.194+0000 7f538eda4640 1 --2- 172.21.15.100:0/2798909700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5388107e50 0x7f5388108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:25.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.194+0000 7f538eda4640 1 --2- 172.21.15.100:0/2798909700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5388102510 0x7f5388102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:25.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.194+0000 7f538eda4640 1 -- 172.21.15.100:0/2798909700 >> 172.21.15.100:0/2798909700 conn(0x7f53880791a0 msgr2=0x7f53880795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:25.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.194+0000 7f538eda4640 1 -- 172.21.15.100:0/2798909700 shutdown_connections 2024-07-26T13:37:25.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.194+0000 7f538eda4640 1 -- 172.21.15.100:0/2798909700 wait complete. 2024-07-26T13:37:25.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.195+0000 7f538eda4640 1 Processor -- start 2024-07-26T13:37:25.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.195+0000 7f538eda4640 1 -- start start 2024-07-26T13:37:25.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.195+0000 7f538eda4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5388102510 0x7f5388198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:25.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.195+0000 7f538eda4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5388107e50 0x7f5388199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:25.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.195+0000 7f538eda4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53881087a0 0x7f538819e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:25.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.195+0000 7f538eda4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5388112470 con 0x7f5388102510 2024-07-26T13:37:25.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.195+0000 7f538eda4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f53881122f0 con 0x7f5388107e50 2024-07-26T13:37:25.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.195+0000 7f538eda4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f53881125f0 con 0x7f53881087a0 2024-07-26T13:37:25.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.195+0000 7f538cb40640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5388102510 0x7f5388198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:25.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.195+0000 7f538cb40640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5388102510 0x7f5388198de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51686/0 (socket says 172.21.15.100:51686) 2024-07-26T13:37:25.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.195+0000 7f538cb40640 1 -- 172.21.15.100:0/2902636079 learned_addr learned my addr 172.21.15.100:0/2902636079 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:37:25.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.195+0000 7f5387fff640 1 --2- 172.21.15.100:0/2902636079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5388107e50 0x7f5388199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:25.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.195+0000 7f538d341640 1 --2- 172.21.15.100:0/2902636079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53881087a0 0x7f538819e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:25.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.195+0000 7f538cb40640 1 -- 172.21.15.100:0/2902636079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53881087a0 msgr2=0x7f538819e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:25.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.195+0000 7f538cb40640 1 --2- 172.21.15.100:0/2902636079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53881087a0 0x7f538819e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:25.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.195+0000 7f538cb40640 1 -- 172.21.15.100:0/2902636079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5388107e50 msgr2=0x7f5388199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:25.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.195+0000 7f538cb40640 1 --2- 172.21.15.100:0/2902636079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5388107e50 0x7f5388199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:25.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.195+0000 7f538cb40640 1 -- 172.21.15.100:0/2902636079 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f538819ed50 con 0x7f5388102510 2024-07-26T13:37:25.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.196+0000 7f538cb40640 1 --2- 172.21.15.100:0/2902636079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5388102510 0x7f5388198de0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f537800c970 tx=0x7f537800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:25.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.196+0000 7f5385ffb640 1 -- 172.21.15.100:0/2902636079 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5378007bb0 con 0x7f5388102510 2024-07-26T13:37:25.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.196+0000 7f5385ffb640 1 -- 172.21.15.100:0/2902636079 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5378007d50 con 0x7f5388102510 2024-07-26T13:37:25.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.196+0000 7f538eda4640 1 -- 172.21.15.100:0/2902636079 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f538819f040 con 0x7f5388102510 2024-07-26T13:37:25.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.196+0000 7f5385ffb640 1 -- 172.21.15.100:0/2902636079 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537800f450 con 0x7f5388102510 2024-07-26T13:37:25.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.196+0000 7f538eda4640 1 -- 172.21.15.100:0/2902636079 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53881a57a0 con 0x7f5388102510 2024-07-26T13:37:25.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.197+0000 7f538eda4640 1 -- 172.21.15.100:0/2902636079 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5388103790 con 0x7f5388102510 2024-07-26T13:37:25.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.198+0000 7f5385ffb640 1 -- 172.21.15.100:0/2902636079 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f5378004f60 con 0x7f5388102510 2024-07-26T13:37:25.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.199+0000 7f5385ffb640 1 --2- 172.21.15.100:0/2902636079 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f53700b3d30 0x7f53700b6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:25.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.199+0000 7f5387fff640 1 --2- 172.21.15.100:0/2902636079 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f53700b3d30 0x7f53700b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:25.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.199+0000 7f5385ffb640 1 -- 172.21.15.100:0/2902636079 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f53780e3b20 con 0x7f5388102510 2024-07-26T13:37:25.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.200+0000 7f5387fff640 1 --2- 172.21.15.100:0/2902636079 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f53700b3d30 0x7f53700b6240 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5388105aa0 tx=0x7f537c00b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:25.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.203+0000 7f5385ffb640 1 -- 172.21.15.100:0/2902636079 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f5378014030 con 0x7f5388102510 2024-07-26T13:37:25.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.403+0000 7f538eda4640 1 -- 172.21.15.100:0/2902636079 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f5388063230 con 0x7f53700b3d30 2024-07-26T13:37:25.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.429+0000 7f5385ffb640 1 -- 172.21.15.100:0/2902636079 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5388063230 con 0x7f53700b3d30 2024-07-26T13:37:25.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.431+0000 7f538eda4640 1 -- 172.21.15.100:0/2902636079 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f53700b3d30 msgr2=0x7f53700b6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:25.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.431+0000 7f538eda4640 1 --2- 172.21.15.100:0/2902636079 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f53700b3d30 0x7f53700b6240 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5388105aa0 tx=0x7f537c00b390 comp rx=0 tx=0).stop 2024-07-26T13:37:25.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.431+0000 7f538eda4640 1 -- 172.21.15.100:0/2902636079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5388102510 msgr2=0x7f5388198de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:25.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.431+0000 7f538eda4640 1 --2- 172.21.15.100:0/2902636079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5388102510 0x7f5388198de0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f537800c970 tx=0x7f537800ce30 comp rx=0 tx=0).stop 2024-07-26T13:37:25.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.431+0000 7f538eda4640 1 -- 172.21.15.100:0/2902636079 shutdown_connections 2024-07-26T13:37:25.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.431+0000 7f538eda4640 1 --2- 172.21.15.100:0/2902636079 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f53700b3d30 0x7f53700b6240 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:25.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.431+0000 7f538eda4640 1 --2- 172.21.15.100:0/2902636079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53881087a0 0x7f538819e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:25.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.431+0000 7f538eda4640 1 --2- 172.21.15.100:0/2902636079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5388107e50 0x7f5388199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:25.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.431+0000 7f538eda4640 1 --2- 172.21.15.100:0/2902636079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5388102510 0x7f5388198de0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:25.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.431+0000 7f538eda4640 1 -- 172.21.15.100:0/2902636079 >> 172.21.15.100:0/2902636079 conn(0x7f53880791a0 msgr2=0x7f5388103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:25.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.431+0000 7f538eda4640 1 -- 172.21.15.100:0/2902636079 shutdown_connections 2024-07-26T13:37:25.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:25.431+0000 7f538eda4640 1 -- 172.21.15.100:0/2902636079 wait complete. 2024-07-26T13:37:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:25 smithi138 ceph-mon[36954]: pgmap v283: 97 pgs: 97 active+clean; 588 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.2 KiB/s wr, 16 op/s; 80 B/s, 0 objects/s recovering 2024-07-26T13:37:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:25 smithi138 ceph-mon[36954]: mgrmap e21: x(active, since 7m), standbys: y, z 2024-07-26T13:37:25.729 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:25 smithi100 ceph-mon[32599]: pgmap v283: 97 pgs: 97 active+clean; 588 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.2 KiB/s wr, 16 op/s; 80 B/s, 0 objects/s recovering 2024-07-26T13:37:25.729 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:25 smithi100 ceph-mon[32599]: mgrmap e21: x(active, since 7m), standbys: y, z 2024-07-26T13:37:25.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:25 smithi167 ceph-mon[37140]: pgmap v283: 97 pgs: 97 active+clean; 588 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.2 KiB/s wr, 16 op/s; 80 B/s, 0 objects/s recovering 2024-07-26T13:37:25.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:25 smithi167 ceph-mon[37140]: mgrmap e21: x(active, since 7m), standbys: y, z 2024-07-26T13:37:26.024 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-07-26T13:37:26.279 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:37:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:26 smithi138 ceph-mon[36954]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:37:26.725 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:26 smithi100 ceph-mon[32599]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:37:26.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:26 smithi167 ceph-mon[37140]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:37:27.646 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:27 smithi100 ceph-mon[32599]: pgmap v284: 97 pgs: 97 active+clean; 588 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 3.5 KiB/s wr, 13 op/s; 65 B/s, 0 objects/s recovering 2024-07-26T13:37:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:27 smithi138 ceph-mon[36954]: pgmap v284: 97 pgs: 97 active+clean; 588 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 3.5 KiB/s wr, 13 op/s; 65 B/s, 0 objects/s recovering 2024-07-26T13:37:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:27 smithi167 ceph-mon[37140]: pgmap v284: 97 pgs: 97 active+clean; 588 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 3.5 KiB/s wr, 13 op/s; 65 B/s, 0 objects/s recovering 2024-07-26T13:37:27.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.895+0000 7f1a8af05640 1 -- 172.21.15.100:0/4151780720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a84108fa0 msgr2=0x7f1a841093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:27.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.895+0000 7f1a8af05640 1 --2- 172.21.15.100:0/4151780720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a84108fa0 0x7f1a841093b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1a70009900 tx=0x7f1a7002a410 comp rx=0 tx=0).stop 2024-07-26T13:37:27.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.895+0000 7f1a8af05640 1 -- 172.21.15.100:0/4151780720 shutdown_connections 2024-07-26T13:37:27.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.895+0000 7f1a8af05640 1 --2- 172.21.15.100:0/4151780720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a841098f0 0x7f1a84110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:27.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.895+0000 7f1a8af05640 1 --2- 172.21.15.100:0/4151780720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a84108fa0 0x7f1a841093b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:27.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.895+0000 7f1a8af05640 1 --2- 172.21.15.100:0/4151780720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a84103660 0x7f1a84103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:27.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.895+0000 7f1a8af05640 1 -- 172.21.15.100:0/4151780720 >> 172.21.15.100:0/4151780720 conn(0x7f1a8407b300 msgr2=0x7f1a8407b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:27.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.895+0000 7f1a8af05640 1 -- 172.21.15.100:0/4151780720 shutdown_connections 2024-07-26T13:37:27.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.895+0000 7f1a8af05640 1 -- 172.21.15.100:0/4151780720 wait complete. 2024-07-26T13:37:27.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.896+0000 7f1a8af05640 1 Processor -- start 2024-07-26T13:37:27.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.896+0000 7f1a8af05640 1 -- start start 2024-07-26T13:37:27.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.896+0000 7f1a8af05640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a84103660 0x7f1a84199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:27.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.896+0000 7f1a8af05640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a84108fa0 0x7f1a8419a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:27.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.896+0000 7f1a8af05640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a841098f0 0x7f1a8419f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:27.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.896+0000 7f1a8af05640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1a84113530 con 0x7f1a841098f0 2024-07-26T13:37:27.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.896+0000 7f1a8af05640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1a841133b0 con 0x7f1a84103660 2024-07-26T13:37:27.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.896+0000 7f1a8af05640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1a841136b0 con 0x7f1a84108fa0 2024-07-26T13:37:27.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.896+0000 7f1a894a2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a841098f0 0x7f1a8419f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:27.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.896+0000 7f1a894a2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a841098f0 0x7f1a8419f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51702/0 (socket says 172.21.15.100:51702) 2024-07-26T13:37:27.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.896+0000 7f1a894a2640 1 -- 172.21.15.100:0/2012632739 learned_addr learned my addr 172.21.15.100:0/2012632739 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:37:27.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.896+0000 7f1a88ca1640 1 --2- 172.21.15.100:0/2012632739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a84103660 0x7f1a84199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:27.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.897+0000 7f1a83fff640 1 --2- 172.21.15.100:0/2012632739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a84108fa0 0x7f1a8419a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:27.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.897+0000 7f1a894a2640 1 -- 172.21.15.100:0/2012632739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a84108fa0 msgr2=0x7f1a8419a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:27.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.897+0000 7f1a894a2640 1 --2- 172.21.15.100:0/2012632739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a84108fa0 0x7f1a8419a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:27.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.897+0000 7f1a894a2640 1 -- 172.21.15.100:0/2012632739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a84103660 msgr2=0x7f1a84199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:27.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.897+0000 7f1a894a2640 1 --2- 172.21.15.100:0/2012632739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a84103660 0x7f1a84199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:27.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.897+0000 7f1a894a2640 1 -- 172.21.15.100:0/2012632739 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a8419fed0 con 0x7f1a841098f0 2024-07-26T13:37:27.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.897+0000 7f1a894a2640 1 --2- 172.21.15.100:0/2012632739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a841098f0 0x7f1a8419f7a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c00cad0 tx=0x7f1a7c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:27.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.897+0000 7f1a81ffb640 1 -- 172.21.15.100:0/2012632739 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c00ad60 con 0x7f1a841098f0 2024-07-26T13:37:27.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.897+0000 7f1a81ffb640 1 -- 172.21.15.100:0/2012632739 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a7c005020 con 0x7f1a841098f0 2024-07-26T13:37:27.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.897+0000 7f1a8af05640 1 -- 172.21.15.100:0/2012632739 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a841a01c0 con 0x7f1a841098f0 2024-07-26T13:37:27.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.897+0000 7f1a81ffb640 1 -- 172.21.15.100:0/2012632739 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c00f6b0 con 0x7f1a841098f0 2024-07-26T13:37:27.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.897+0000 7f1a8af05640 1 -- 172.21.15.100:0/2012632739 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a841a6920 con 0x7f1a841098f0 2024-07-26T13:37:27.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.898+0000 7f1a8af05640 1 -- 172.21.15.100:0/2012632739 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a84103a70 con 0x7f1a841098f0 2024-07-26T13:37:27.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.900+0000 7f1a81ffb640 1 -- 172.21.15.100:0/2012632739 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f1a7c0051d0 con 0x7f1a841098f0 2024-07-26T13:37:27.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.901+0000 7f1a81ffb640 1 --2- 172.21.15.100:0/2012632739 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1a740b3cb0 0x7f1a740b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:27.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.901+0000 7f1a88ca1640 1 --2- 172.21.15.100:0/2012632739 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1a740b3cb0 0x7f1a740b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:27.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.901+0000 7f1a81ffb640 1 -- 172.21.15.100:0/2012632739 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f1a7c0e4300 con 0x7f1a841098f0 2024-07-26T13:37:27.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.901+0000 7f1a88ca1640 1 --2- 172.21.15.100:0/2012632739 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1a740b3cb0 0x7f1a740b61c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1a78009990 tx=0x7f1a78009110 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:27.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:27.904+0000 7f1a81ffb640 1 -- 172.21.15.100:0/2012632739 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f1a7c014030 con 0x7f1a841098f0 2024-07-26T13:37:28.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:28.104+0000 7f1a8af05640 1 -- 172.21.15.100:0/2012632739 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f1a84063230 con 0x7f1a740b3cb0 2024-07-26T13:37:28.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:28.109+0000 7f1a81ffb640 1 -- 172.21.15.100:0/2012632739 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f1a84063230 con 0x7f1a740b3cb0 2024-07-26T13:37:28.110 INFO:teuthology.orchestra.run.smithi100.stdout:/volumes/qa/sv_0/7caa30ca-301e-4a1c-8277-7c31b4d8c429 2024-07-26T13:37:28.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:28.111+0000 7f1a8af05640 1 -- 172.21.15.100:0/2012632739 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1a740b3cb0 msgr2=0x7f1a740b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:28.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:28.111+0000 7f1a8af05640 1 --2- 172.21.15.100:0/2012632739 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1a740b3cb0 0x7f1a740b61c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1a78009990 tx=0x7f1a78009110 comp rx=0 tx=0).stop 2024-07-26T13:37:28.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:28.111+0000 7f1a8af05640 1 -- 172.21.15.100:0/2012632739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a841098f0 msgr2=0x7f1a8419f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:28.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:28.111+0000 7f1a8af05640 1 --2- 172.21.15.100:0/2012632739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a841098f0 0x7f1a8419f7a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c00cad0 tx=0x7f1a7c00ced0 comp rx=0 tx=0).stop 2024-07-26T13:37:28.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:28.111+0000 7f1a8af05640 1 -- 172.21.15.100:0/2012632739 shutdown_connections 2024-07-26T13:37:28.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:28.111+0000 7f1a8af05640 1 --2- 172.21.15.100:0/2012632739 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1a740b3cb0 0x7f1a740b61c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:28.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:28.111+0000 7f1a8af05640 1 --2- 172.21.15.100:0/2012632739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a841098f0 0x7f1a8419f7a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:28.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:28.111+0000 7f1a8af05640 1 --2- 172.21.15.100:0/2012632739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a84108fa0 0x7f1a8419a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:28.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:28.111+0000 7f1a8af05640 1 --2- 172.21.15.100:0/2012632739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a84103660 0x7f1a84199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:28.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:28.111+0000 7f1a8af05640 1 -- 172.21.15.100:0/2012632739 >> 172.21.15.100:0/2012632739 conn(0x7f1a8407b300 msgr2=0x7f1a84107090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:28.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:28.112+0000 7f1a8af05640 1 -- 172.21.15.100:0/2012632739 shutdown_connections 2024-07-26T13:37:28.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:28.112+0000 7f1a8af05640 1 -- 172.21.15.100:0/2012632739 wait complete. 2024-07-26T13:37:28.736 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-07-26T13:37:28.991 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:37:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:29 smithi138 ceph-mon[36954]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:37:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:29 smithi138 ceph-mon[36954]: pgmap v285: 97 pgs: 97 active+clean; 589 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.8 KiB/s wr, 11 op/s; 59 B/s, 0 objects/s recovering 2024-07-26T13:37:29.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:29 smithi100 ceph-mon[32599]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:37:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:29 smithi100 ceph-mon[32599]: pgmap v285: 97 pgs: 97 active+clean; 589 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.8 KiB/s wr, 11 op/s; 59 B/s, 0 objects/s recovering 2024-07-26T13:37:29.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:29 smithi167 ceph-mon[37140]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:37:29.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:29 smithi167 ceph-mon[37140]: pgmap v285: 97 pgs: 97 active+clean; 589 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.8 KiB/s wr, 11 op/s; 59 B/s, 0 objects/s recovering 2024-07-26T13:37:30.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.681+0000 7fc2f4894640 1 -- 172.21.15.100:0/766120494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2f0108720 msgr2=0x7fc2f010ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:30.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.681+0000 7fc2f4894640 1 --2- 172.21.15.100:0/766120494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2f0108720 0x7fc2f010ef50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8009ca0 tx=0x7fc2e802a8c0 comp rx=0 tx=0).stop 2024-07-26T13:37:30.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.681+0000 7fc2f4894640 1 -- 172.21.15.100:0/766120494 shutdown_connections 2024-07-26T13:37:30.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.681+0000 7fc2f4894640 1 --2- 172.21.15.100:0/766120494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2f0108720 0x7fc2f010ef50 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:30.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.681+0000 7fc2f4894640 1 --2- 172.21.15.100:0/766120494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2f0107dd0 0x7fc2f01081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:30.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.681+0000 7fc2f4894640 1 --2- 172.21.15.100:0/766120494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2f0102490 0x7fc2f01028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:30.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.681+0000 7fc2f4894640 1 -- 172.21.15.100:0/766120494 >> 172.21.15.100:0/766120494 conn(0x7fc2f0079190 msgr2=0x7fc2f00795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:30.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.681+0000 7fc2f4894640 1 -- 172.21.15.100:0/766120494 shutdown_connections 2024-07-26T13:37:30.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.681+0000 7fc2f4894640 1 -- 172.21.15.100:0/766120494 wait complete. 2024-07-26T13:37:30.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.682+0000 7fc2f4894640 1 Processor -- start 2024-07-26T13:37:30.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.682+0000 7fc2f4894640 1 -- start start 2024-07-26T13:37:30.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.683+0000 7fc2f4894640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2f0102490 0x7fc2f0199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:30.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.683+0000 7fc2f4894640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2f0107dd0 0x7fc2f019a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:30.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.683+0000 7fc2f4894640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2f0108720 0x7fc2f019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:30.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.683+0000 7fc2f4894640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f0113500 con 0x7fc2f0107dd0 2024-07-26T13:37:30.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.683+0000 7fc2f4894640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f0113380 con 0x7fc2f0108720 2024-07-26T13:37:30.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.683+0000 7fc2f4894640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f0113680 con 0x7fc2f0102490 2024-07-26T13:37:30.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.683+0000 7fc2edd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2f0107dd0 0x7fc2f019a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:30.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.683+0000 7fc2edd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2f0107dd0 0x7fc2f019a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51710/0 (socket says 172.21.15.100:51710) 2024-07-26T13:37:30.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.683+0000 7fc2edd9b640 1 -- 172.21.15.100:0/3049106309 learned_addr learned my addr 172.21.15.100:0/3049106309 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:37:30.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.683+0000 7fc2ee59c640 1 --2- 172.21.15.100:0/3049106309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2f0102490 0x7fc2f0199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:30.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.683+0000 7fc2edd9b640 1 -- 172.21.15.100:0/3049106309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2f0102490 msgr2=0x7fc2f0199e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:30.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.683+0000 7fc2edd9b640 1 --2- 172.21.15.100:0/3049106309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2f0102490 0x7fc2f0199e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:30.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.683+0000 7fc2edd9b640 1 -- 172.21.15.100:0/3049106309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2f0108720 msgr2=0x7fc2f019f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:30.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.683+0000 7fc2edd9b640 1 --2- 172.21.15.100:0/3049106309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2f0108720 0x7fc2f019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:30.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.683+0000 7fc2edd9b640 1 -- 172.21.15.100:0/3049106309 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f019fd40 con 0x7fc2f0107dd0 2024-07-26T13:37:30.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.683+0000 7fc2ee59c640 1 --2- 172.21.15.100:0/3049106309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2f0102490 0x7fc2f0199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:37:30.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.683+0000 7fc2edd9b640 1 --2- 172.21.15.100:0/3049106309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2f0107dd0 0x7fc2f019a430 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc2e400c8c0 tx=0x7fc2e400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:30.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.683+0000 7fc2df7fe640 1 -- 172.21.15.100:0/3049106309 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e4007bb0 con 0x7fc2f0107dd0 2024-07-26T13:37:30.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.683+0000 7fc2df7fe640 1 -- 172.21.15.100:0/3049106309 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2e4007d50 con 0x7fc2f0107dd0 2024-07-26T13:37:30.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.683+0000 7fc2df7fe640 1 -- 172.21.15.100:0/3049106309 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e40187c0 con 0x7fc2f0107dd0 2024-07-26T13:37:30.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.683+0000 7fc2f4894640 1 -- 172.21.15.100:0/3049106309 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2f01a0030 con 0x7fc2f0107dd0 2024-07-26T13:37:30.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.683+0000 7fc2f4894640 1 -- 172.21.15.100:0/3049106309 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2f01a6850 con 0x7fc2f0107dd0 2024-07-26T13:37:30.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.684+0000 7fc2f4894640 1 -- 172.21.15.100:0/3049106309 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2f0103710 con 0x7fc2f0107dd0 2024-07-26T13:37:30.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.686+0000 7fc2df7fe640 1 -- 172.21.15.100:0/3049106309 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7fc2e400fa20 con 0x7fc2f0107dd0 2024-07-26T13:37:30.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.687+0000 7fc2df7fe640 1 --2- 172.21.15.100:0/3049106309 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc2d80b3ce0 0x7fc2d80b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:30.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.687+0000 7fc2ee59c640 1 --2- 172.21.15.100:0/3049106309 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc2d80b3ce0 0x7fc2d80b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:30.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.687+0000 7fc2df7fe640 1 -- 172.21.15.100:0/3049106309 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fc2e40e4280 con 0x7fc2f0107dd0 2024-07-26T13:37:30.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.687+0000 7fc2ee59c640 1 --2- 172.21.15.100:0/3049106309 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc2d80b3ce0 0x7fc2d80b61f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc2e000baa0 tx=0x7fc2e0005d90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:30.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.690+0000 7fc2df7fe640 1 -- 172.21.15.100:0/3049106309 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fc2e4014030 con 0x7fc2f0107dd0 2024-07-26T13:37:30.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.890+0000 7fc2f4894640 1 -- 172.21.15.100:0/3049106309 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc2f0063230 con 0x7fc2d80b3ce0 2024-07-26T13:37:30.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.920+0000 7fc2df7fe640 1 -- 172.21.15.100:0/3049106309 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc2f0063230 con 0x7fc2d80b3ce0 2024-07-26T13:37:30.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.922+0000 7fc2f4894640 1 -- 172.21.15.100:0/3049106309 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc2d80b3ce0 msgr2=0x7fc2d80b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:30.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.922+0000 7fc2f4894640 1 --2- 172.21.15.100:0/3049106309 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc2d80b3ce0 0x7fc2d80b61f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc2e000baa0 tx=0x7fc2e0005d90 comp rx=0 tx=0).stop 2024-07-26T13:37:30.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.922+0000 7fc2f4894640 1 -- 172.21.15.100:0/3049106309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2f0107dd0 msgr2=0x7fc2f019a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:30.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.922+0000 7fc2f4894640 1 --2- 172.21.15.100:0/3049106309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2f0107dd0 0x7fc2f019a430 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc2e400c8c0 tx=0x7fc2e400cd80 comp rx=0 tx=0).stop 2024-07-26T13:37:30.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.922+0000 7fc2f4894640 1 -- 172.21.15.100:0/3049106309 shutdown_connections 2024-07-26T13:37:30.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.922+0000 7fc2f4894640 1 --2- 172.21.15.100:0/3049106309 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc2d80b3ce0 0x7fc2d80b61f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:30.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.922+0000 7fc2f4894640 1 --2- 172.21.15.100:0/3049106309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2f0108720 0x7fc2f019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:30.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.922+0000 7fc2f4894640 1 --2- 172.21.15.100:0/3049106309 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2f0107dd0 0x7fc2f019a430 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:30.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.922+0000 7fc2f4894640 1 --2- 172.21.15.100:0/3049106309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc2f0102490 0x7fc2f0199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:30.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.922+0000 7fc2f4894640 1 -- 172.21.15.100:0/3049106309 >> 172.21.15.100:0/3049106309 conn(0x7fc2f0079190 msgr2=0x7fc2f0105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:30.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.922+0000 7fc2f4894640 1 -- 172.21.15.100:0/3049106309 shutdown_connections 2024-07-26T13:37:30.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:30.922+0000 7fc2f4894640 1 -- 172.21.15.100:0/3049106309 wait complete. 2024-07-26T13:37:31.617 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:31 smithi100 ceph-mon[32599]: pgmap v286: 97 pgs: 97 active+clean; 603 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.7 KiB/s wr, 11 op/s; 53 B/s, 0 objects/s recovering 2024-07-26T13:37:31.644 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-07-26T13:37:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:31 smithi138 ceph-mon[36954]: pgmap v286: 97 pgs: 97 active+clean; 603 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.7 KiB/s wr, 11 op/s; 53 B/s, 0 objects/s recovering 2024-07-26T13:37:31.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:31 smithi167 ceph-mon[37140]: pgmap v286: 97 pgs: 97 active+clean; 603 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.7 KiB/s wr, 11 op/s; 53 B/s, 0 objects/s recovering 2024-07-26T13:37:31.905 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:37:32.716 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:32 smithi100 ceph-mon[32599]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:37:32.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:32 smithi167 ceph-mon[37140]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:37:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:32 smithi138 ceph-mon[36954]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:37:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:33 smithi100 ceph-mon[32599]: pgmap v287: 97 pgs: 97 active+clean; 603 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.3 KiB/s wr, 7 op/s; 53 B/s, 0 objects/s recovering 2024-07-26T13:37:33.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:33 smithi167 ceph-mon[37140]: pgmap v287: 97 pgs: 97 active+clean; 603 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.3 KiB/s wr, 7 op/s; 53 B/s, 0 objects/s recovering 2024-07-26T13:37:33.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.758+0000 7f8872f13640 1 -- 172.21.15.100:0/2404038949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f886c108730 msgr2=0x7f886c10ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:33.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.758+0000 7f8872f13640 1 --2- 172.21.15.100:0/2404038949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f886c108730 0x7f886c10ef60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8864009ca0 tx=0x7f886402a8c0 comp rx=0 tx=0).stop 2024-07-26T13:37:33.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.758+0000 7f8872f13640 1 -- 172.21.15.100:0/2404038949 shutdown_connections 2024-07-26T13:37:33.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.758+0000 7f8872f13640 1 --2- 172.21.15.100:0/2404038949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f886c108730 0x7f886c10ef60 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:33.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.758+0000 7f8872f13640 1 --2- 172.21.15.100:0/2404038949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f886c107de0 0x7f886c1081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:33.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.758+0000 7f8872f13640 1 --2- 172.21.15.100:0/2404038949 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f886c1024a0 0x7f886c1028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:33.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.758+0000 7f8872f13640 1 -- 172.21.15.100:0/2404038949 >> 172.21.15.100:0/2404038949 conn(0x7f886c079140 msgr2=0x7f886c079550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:33.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.758+0000 7f8872f13640 1 -- 172.21.15.100:0/2404038949 shutdown_connections 2024-07-26T13:37:33.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.758+0000 7f8872f13640 1 -- 172.21.15.100:0/2404038949 wait complete. 2024-07-26T13:37:33.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.759+0000 7f8872f13640 1 Processor -- start 2024-07-26T13:37:33.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.759+0000 7f8872f13640 1 -- start start 2024-07-26T13:37:33.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.760+0000 7f8872f13640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f886c1024a0 0x7f886c19ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:33.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.760+0000 7f8872f13640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f886c107de0 0x7f886c19b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:33.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.760+0000 7f8872f13640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f886c108730 0x7f886c19b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:33.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.760+0000 7f8872f13640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f886c1123e0 con 0x7f886c1024a0 2024-07-26T13:37:33.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.760+0000 7f8872f13640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f886c112260 con 0x7f886c108730 2024-07-26T13:37:33.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.760+0000 7f8872f13640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f886c112560 con 0x7f886c107de0 2024-07-26T13:37:33.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.760+0000 7f8870caf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f886c1024a0 0x7f886c19ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:33.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.760+0000 7f8870caf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f886c1024a0 0x7f886c19ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51720/0 (socket says 172.21.15.100:51720) 2024-07-26T13:37:33.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.760+0000 7f8870caf640 1 -- 172.21.15.100:0/1070469515 learned_addr learned my addr 172.21.15.100:0/1070469515 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:37:33.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.760+0000 7f886bfff640 1 --2- 172.21.15.100:0/1070469515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f886c107de0 0x7f886c19b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:33.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.760+0000 7f8870caf640 1 -- 172.21.15.100:0/1070469515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f886c107de0 msgr2=0x7f886c19b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:33.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.760+0000 7f8870caf640 1 --2- 172.21.15.100:0/1070469515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f886c107de0 0x7f886c19b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:33.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.760+0000 7f8870caf640 1 -- 172.21.15.100:0/1070469515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f886c108730 msgr2=0x7f886c19b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:33.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.760+0000 7f8870caf640 1 --2- 172.21.15.100:0/1070469515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f886c108730 0x7f886c19b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:33.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.760+0000 7f8870caf640 1 -- 172.21.15.100:0/1070469515 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f886c1a5f40 con 0x7f886c1024a0 2024-07-26T13:37:33.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.760+0000 7f8870caf640 1 --2- 172.21.15.100:0/1070469515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f886c1024a0 0x7f886c19ac30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f885c00d950 tx=0x7f885c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:33.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.760+0000 7f8869ffb640 1 -- 172.21.15.100:0/1070469515 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885c017070 con 0x7f886c1024a0 2024-07-26T13:37:33.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.760+0000 7f8869ffb640 1 -- 172.21.15.100:0/1070469515 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f885c009c30 con 0x7f886c1024a0 2024-07-26T13:37:33.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.760+0000 7f8872f13640 1 -- 172.21.15.100:0/1070469515 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f886c1a6230 con 0x7f886c1024a0 2024-07-26T13:37:33.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.760+0000 7f8869ffb640 1 -- 172.21.15.100:0/1070469515 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885c00f620 con 0x7f886c1024a0 2024-07-26T13:37:33.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.761+0000 7f8872f13640 1 -- 172.21.15.100:0/1070469515 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f886c1a6770 con 0x7f886c1024a0 2024-07-26T13:37:33.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.761+0000 7f8872f13640 1 -- 172.21.15.100:0/1070469515 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f886c103720 con 0x7f886c1024a0 2024-07-26T13:37:33.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.763+0000 7f8869ffb640 1 -- 172.21.15.100:0/1070469515 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f885c010460 con 0x7f886c1024a0 2024-07-26T13:37:33.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.764+0000 7f8869ffb640 1 --2- 172.21.15.100:0/1070469515 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f88540b3cb0 0x7f88540b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:33.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.764+0000 7f886bfff640 1 --2- 172.21.15.100:0/1070469515 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f88540b3cb0 0x7f88540b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:33.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.764+0000 7f8869ffb640 1 -- 172.21.15.100:0/1070469515 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f885c0e3b20 con 0x7f886c1024a0 2024-07-26T13:37:33.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.764+0000 7f886bfff640 1 --2- 172.21.15.100:0/1070469515 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f88540b3cb0 0x7f88540b61c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f886c0797b0 tx=0x7f886000b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:33.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.768+0000 7f8869ffb640 1 -- 172.21.15.100:0/1070469515 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f885c0aaf10 con 0x7f886c1024a0 2024-07-26T13:37:33.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:33 smithi138 ceph-mon[36954]: pgmap v287: 97 pgs: 97 active+clean; 603 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.3 KiB/s wr, 7 op/s; 53 B/s, 0 objects/s recovering 2024-07-26T13:37:33.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.967+0000 7f8872f13640 1 -- 172.21.15.100:0/1070469515 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f886c063230 con 0x7f88540b3cb0 2024-07-26T13:37:33.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.972+0000 7f8869ffb640 1 -- 172.21.15.100:0/1070469515 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f886c063230 con 0x7f88540b3cb0 2024-07-26T13:37:33.973 INFO:teuthology.orchestra.run.smithi100.stdout:/volumes/qa/sv_1/425a8b37-4935-4bf6-a775-12ed75a40837 2024-07-26T13:37:33.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.974+0000 7f8872f13640 1 -- 172.21.15.100:0/1070469515 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f88540b3cb0 msgr2=0x7f88540b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:33.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.974+0000 7f8872f13640 1 --2- 172.21.15.100:0/1070469515 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f88540b3cb0 0x7f88540b61c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f886c0797b0 tx=0x7f886000b390 comp rx=0 tx=0).stop 2024-07-26T13:37:33.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.974+0000 7f8872f13640 1 -- 172.21.15.100:0/1070469515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f886c1024a0 msgr2=0x7f886c19ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:33.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.974+0000 7f8872f13640 1 --2- 172.21.15.100:0/1070469515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f886c1024a0 0x7f886c19ac30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f885c00d950 tx=0x7f885c00de10 comp rx=0 tx=0).stop 2024-07-26T13:37:33.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.974+0000 7f8872f13640 1 -- 172.21.15.100:0/1070469515 shutdown_connections 2024-07-26T13:37:33.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.974+0000 7f8872f13640 1 --2- 172.21.15.100:0/1070469515 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f88540b3cb0 0x7f88540b61c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:33.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.974+0000 7f8872f13640 1 --2- 172.21.15.100:0/1070469515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f886c108730 0x7f886c19b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:33.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.974+0000 7f8872f13640 1 --2- 172.21.15.100:0/1070469515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f886c107de0 0x7f886c19b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:33.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.974+0000 7f8872f13640 1 --2- 172.21.15.100:0/1070469515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f886c1024a0 0x7f886c19ac30 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:33.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.974+0000 7f8872f13640 1 -- 172.21.15.100:0/1070469515 >> 172.21.15.100:0/1070469515 conn(0x7f886c079140 msgr2=0x7f886c105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:33.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.974+0000 7f8872f13640 1 -- 172.21.15.100:0/1070469515 shutdown_connections 2024-07-26T13:37:33.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:33.974+0000 7f8872f13640 1 -- 172.21.15.100:0/1070469515 wait complete. 2024-07-26T13:37:34.717 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:37:34.972 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:37:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:35 smithi100 ceph-mon[32599]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:37:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:35 smithi100 ceph-mon[32599]: pgmap v288: 97 pgs: 97 active+clean; 622 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 1 B/s, 0 objects/s recovering 2024-07-26T13:37:35.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:35 smithi167 ceph-mon[37140]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:37:35.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:35 smithi167 ceph-mon[37140]: pgmap v288: 97 pgs: 97 active+clean; 622 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 1 B/s, 0 objects/s recovering 2024-07-26T13:37:35.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:35 smithi138 ceph-mon[36954]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:37:35.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:35 smithi138 ceph-mon[36954]: pgmap v288: 97 pgs: 97 active+clean; 622 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 1 B/s, 0 objects/s recovering 2024-07-26T13:37:36.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.534+0000 7fe1f4936640 1 -- 172.21.15.100:0/942869402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1f0075ab0 msgr2=0x7fe1f010f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:36.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.534+0000 7fe1f4936640 1 --2- 172.21.15.100:0/942869402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1f0075ab0 0x7fe1f010f630 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4009900 tx=0x7fe1e402a410 comp rx=0 tx=0).stop 2024-07-26T13:37:36.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.534+0000 7fe1f4936640 1 -- 172.21.15.100:0/942869402 shutdown_connections 2024-07-26T13:37:36.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.534+0000 7fe1f4936640 1 --2- 172.21.15.100:0/942869402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1f010fb70 0x7fe1f0111f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:36.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.534+0000 7fe1f4936640 1 --2- 172.21.15.100:0/942869402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1f0075ab0 0x7fe1f010f630 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:36.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.534+0000 7fe1f4936640 1 --2- 172.21.15.100:0/942869402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1f0075160 0x7fe1f0075570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:36.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.534+0000 7fe1f4936640 1 -- 172.21.15.100:0/942869402 >> 172.21.15.100:0/942869402 conn(0x7fe1f007b280 msgr2=0x7fe1f007b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:36.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.534+0000 7fe1f4936640 1 -- 172.21.15.100:0/942869402 shutdown_connections 2024-07-26T13:37:36.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.534+0000 7fe1f4936640 1 -- 172.21.15.100:0/942869402 wait complete. 2024-07-26T13:37:36.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.535+0000 7fe1f4936640 1 Processor -- start 2024-07-26T13:37:36.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.535+0000 7fe1f4936640 1 -- start start 2024-07-26T13:37:36.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.536+0000 7fe1f4936640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1f0075160 0x7fe1f019c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:36.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.536+0000 7fe1f4936640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1f0075ab0 0x7fe1f019c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:36.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.536+0000 7fe1f4936640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1f010fb70 0x7fe1f01a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:36.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.536+0000 7fe1f4936640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f0115690 con 0x7fe1f0075160 2024-07-26T13:37:36.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.536+0000 7fe1f4936640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f0115510 con 0x7fe1f010fb70 2024-07-26T13:37:36.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.536+0000 7fe1f4936640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f0115810 con 0x7fe1f0075ab0 2024-07-26T13:37:36.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.536+0000 7fe1ee59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1f0075160 0x7fe1f019c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:36.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.536+0000 7fe1ee59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1f0075160 0x7fe1f019c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49978/0 (socket says 172.21.15.100:49978) 2024-07-26T13:37:36.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.536+0000 7fe1ee59c640 1 -- 172.21.15.100:0/1567652508 learned_addr learned my addr 172.21.15.100:0/1567652508 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:37:36.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.536+0000 7fe1eed9d640 1 --2- 172.21.15.100:0/1567652508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1f010fb70 0x7fe1f01a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:36.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.536+0000 7fe1edd9b640 1 --2- 172.21.15.100:0/1567652508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1f0075ab0 0x7fe1f019c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:36.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.536+0000 7fe1ee59c640 1 -- 172.21.15.100:0/1567652508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1f0075ab0 msgr2=0x7fe1f019c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:36.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.536+0000 7fe1ee59c640 1 --2- 172.21.15.100:0/1567652508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1f0075ab0 0x7fe1f019c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:36.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.536+0000 7fe1ee59c640 1 -- 172.21.15.100:0/1567652508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1f010fb70 msgr2=0x7fe1f01a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:36.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.536+0000 7fe1ee59c640 1 --2- 172.21.15.100:0/1567652508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1f010fb70 0x7fe1f01a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:36.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.536+0000 7fe1ee59c640 1 -- 172.21.15.100:0/1567652508 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f01a2030 con 0x7fe1f0075160 2024-07-26T13:37:36.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.536+0000 7fe1eed9d640 1 --2- 172.21.15.100:0/1567652508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1f010fb70 0x7fe1f01a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:37:36.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.536+0000 7fe1edd9b640 1 --2- 172.21.15.100:0/1567652508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1f0075ab0 0x7fe1f019c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:37:36.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.536+0000 7fe1ee59c640 1 --2- 172.21.15.100:0/1567652508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1f0075160 0x7fe1f019c0c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe1e000c970 tx=0x7fe1e000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:36.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.537+0000 7fe1df7fe640 1 -- 172.21.15.100:0/1567652508 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0007bb0 con 0x7fe1f0075160 2024-07-26T13:37:36.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.537+0000 7fe1f4936640 1 -- 172.21.15.100:0/1567652508 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1f01a2320 con 0x7fe1f0075160 2024-07-26T13:37:36.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.537+0000 7fe1df7fe640 1 -- 172.21.15.100:0/1567652508 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1e0007d50 con 0x7fe1f0075160 2024-07-26T13:37:36.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.537+0000 7fe1df7fe640 1 -- 172.21.15.100:0/1567652508 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e000f450 con 0x7fe1f0075160 2024-07-26T13:37:36.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.537+0000 7fe1f4936640 1 -- 172.21.15.100:0/1567652508 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1f01a8a80 con 0x7fe1f0075160 2024-07-26T13:37:36.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.538+0000 7fe1f4936640 1 -- 172.21.15.100:0/1567652508 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1f0076670 con 0x7fe1f0075160 2024-07-26T13:37:36.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.539+0000 7fe1df7fe640 1 -- 172.21.15.100:0/1567652508 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7fe1e0004f20 con 0x7fe1f0075160 2024-07-26T13:37:36.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.540+0000 7fe1df7fe640 1 --2- 172.21.15.100:0/1567652508 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe1d80b3cb0 0x7fe1d80b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:36.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.540+0000 7fe1edd9b640 1 --2- 172.21.15.100:0/1567652508 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe1d80b3cb0 0x7fe1d80b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:36.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.540+0000 7fe1df7fe640 1 -- 172.21.15.100:0/1567652508 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fe1e00e4100 con 0x7fe1f0075160 2024-07-26T13:37:36.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.541+0000 7fe1edd9b640 1 --2- 172.21.15.100:0/1567652508 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe1d80b3cb0 0x7fe1d80b61c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4009230 tx=0x7fe1e4000bc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:36.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.543+0000 7fe1df7fe640 1 -- 172.21.15.100:0/1567652508 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fe1e0014030 con 0x7fe1f0075160 2024-07-26T13:37:36.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.753+0000 7fe1f4936640 1 -- 172.21.15.100:0/1567652508 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1f0075570 con 0x7fe1f0075160 2024-07-26T13:37:36.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.754+0000 7fe1df7fe640 1 -- 172.21.15.100:0/1567652508 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11927 (secure 0 0 0) 0x7fe1e00ab4d0 con 0x7fe1f0075160 2024-07-26T13:37:36.755 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:37:36.755 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":25,"btime":"2024-07-26T13:37:19:558299+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14790,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/313501409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":313501409},{"type":"v1","addr":"172.21.15.100:6839","nonce":313501409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/1402975506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":1402975506},{"type":"v1","addr":"172.21.15.100:6841","nonce":1402975506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-26T13:37:01.281843+0000","modified":"2024-07-26T13:37:19.272258+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24607,"mds_1":24605,"mds_2":24595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24557":{"gid":24557,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/3290105300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3290105300},{"type":"v1","addr":"172.21.15.138:6835","nonce":3290105300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/3393225483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3393225483},{"type":"v1","addr":"172.21.15.138:6837","nonce":3393225483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24595":{"gid":24595,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6837/3828634468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3828634468},{"type":"v1","addr":"172.21.15.167:6837","nonce":3828634468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.138:6839/1580925471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1580925471},{"type":"v1","addr":"172.21.15.138:6839","nonce":1580925471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/398150514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":398150514},{"type":"v1","addr":"172.21.15.167:6839","nonce":398150514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24607,"qdb_cluster":[24607,24605,24595]},"id":1}]} 2024-07-26T13:37:36.755 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 25 2024-07-26T13:37:36.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.756+0000 7fe1f4936640 1 -- 172.21.15.100:0/1567652508 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe1d80b3cb0 msgr2=0x7fe1d80b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:36.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.756+0000 7fe1f4936640 1 --2- 172.21.15.100:0/1567652508 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe1d80b3cb0 0x7fe1d80b61c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4009230 tx=0x7fe1e4000bc0 comp rx=0 tx=0).stop 2024-07-26T13:37:36.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.756+0000 7fe1f4936640 1 -- 172.21.15.100:0/1567652508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1f0075160 msgr2=0x7fe1f019c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:36.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.756+0000 7fe1f4936640 1 --2- 172.21.15.100:0/1567652508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1f0075160 0x7fe1f019c0c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe1e000c970 tx=0x7fe1e000ce30 comp rx=0 tx=0).stop 2024-07-26T13:37:36.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.756+0000 7fe1f4936640 1 -- 172.21.15.100:0/1567652508 shutdown_connections 2024-07-26T13:37:36.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.756+0000 7fe1f4936640 1 --2- 172.21.15.100:0/1567652508 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe1d80b3cb0 0x7fe1d80b61c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:36.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.756+0000 7fe1f4936640 1 --2- 172.21.15.100:0/1567652508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1f010fb70 0x7fe1f01a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:36.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.756+0000 7fe1f4936640 1 --2- 172.21.15.100:0/1567652508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1f0075ab0 0x7fe1f019c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:36.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.756+0000 7fe1f4936640 1 --2- 172.21.15.100:0/1567652508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe1f0075160 0x7fe1f019c0c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:36.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.756+0000 7fe1f4936640 1 -- 172.21.15.100:0/1567652508 >> 172.21.15.100:0/1567652508 conn(0x7fe1f007b280 msgr2=0x7fe1f01013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:36.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.756+0000 7fe1f4936640 1 -- 172.21.15.100:0/1567652508 shutdown_connections 2024-07-26T13:37:36.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:36.756+0000 7fe1f4936640 1 -- 172.21.15.100:0/1567652508 wait complete. 2024-07-26T13:37:37.464 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph osd dump --format=json 2024-07-26T13:37:37.720 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:37:37.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:37 smithi100 ceph-mon[32599]: pgmap v289: 97 pgs: 97 active+clean; 622 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2024-07-26T13:37:37.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:37 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/1567652508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:37:37.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:37 smithi167 ceph-mon[37140]: pgmap v289: 97 pgs: 97 active+clean; 622 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2024-07-26T13:37:37.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:37 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/1567652508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:37:37.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:37 smithi138 ceph-mon[36954]: pgmap v289: 97 pgs: 97 active+clean; 622 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2024-07-26T13:37:37.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:37 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/1567652508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:37:39.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.537+0000 7f35cf6f1640 1 -- 172.21.15.100:0/3609258715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35c8108720 msgr2=0x7f35c810ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:39.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.537+0000 7f35cf6f1640 1 --2- 172.21.15.100:0/3609258715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35c8108720 0x7f35c810ef50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f35c0009ca0 tx=0x7f35c002a8c0 comp rx=0 tx=0).stop 2024-07-26T13:37:39.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.537+0000 7f35cf6f1640 1 -- 172.21.15.100:0/3609258715 shutdown_connections 2024-07-26T13:37:39.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.537+0000 7f35cf6f1640 1 --2- 172.21.15.100:0/3609258715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35c8108720 0x7f35c810ef50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:39.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.537+0000 7f35cf6f1640 1 --2- 172.21.15.100:0/3609258715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35c8107dd0 0x7f35c81081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:39.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.537+0000 7f35cf6f1640 1 --2- 172.21.15.100:0/3609258715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35c8102490 0x7f35c81028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:39.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.537+0000 7f35cf6f1640 1 -- 172.21.15.100:0/3609258715 >> 172.21.15.100:0/3609258715 conn(0x7f35c8079160 msgr2=0x7f35c8079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:39.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.537+0000 7f35cf6f1640 1 -- 172.21.15.100:0/3609258715 shutdown_connections 2024-07-26T13:37:39.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.537+0000 7f35cf6f1640 1 -- 172.21.15.100:0/3609258715 wait complete. 2024-07-26T13:37:39.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.537+0000 7f35cf6f1640 1 Processor -- start 2024-07-26T13:37:39.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.537+0000 7f35cf6f1640 1 -- start start 2024-07-26T13:37:39.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.538+0000 7f35cf6f1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35c8102490 0x7f35c80ff0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:39.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.538+0000 7f35cf6f1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35c8107dd0 0x7f35c80ff5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:39.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.538+0000 7f35cf6f1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35c8108720 0x7f35c80ffb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:39.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.538+0000 7f35cf6f1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f35c81123a0 con 0x7f35c8108720 2024-07-26T13:37:39.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.538+0000 7f35cf6f1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f35c8112220 con 0x7f35c8107dd0 2024-07-26T13:37:39.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.538+0000 7f35cf6f1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f35c8112520 con 0x7f35c8102490 2024-07-26T13:37:39.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.538+0000 7f35cdc8e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35c8108720 0x7f35c80ffb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:39.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.538+0000 7f35cdc8e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35c8108720 0x7f35c80ffb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50000/0 (socket says 172.21.15.100:50000) 2024-07-26T13:37:39.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.538+0000 7f35cdc8e640 1 -- 172.21.15.100:0/1501466179 learned_addr learned my addr 172.21.15.100:0/1501466179 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:37:39.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.538+0000 7f35cd48d640 1 --2- 172.21.15.100:0/1501466179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35c8102490 0x7f35c80ff0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:39.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.538+0000 7f35ccc8c640 1 --2- 172.21.15.100:0/1501466179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35c8107dd0 0x7f35c80ff5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:39.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.538+0000 7f35cdc8e640 1 -- 172.21.15.100:0/1501466179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35c8102490 msgr2=0x7f35c80ff0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:39.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.538+0000 7f35cdc8e640 1 --2- 172.21.15.100:0/1501466179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35c8102490 0x7f35c80ff0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:39.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.538+0000 7f35cdc8e640 1 -- 172.21.15.100:0/1501466179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35c8107dd0 msgr2=0x7f35c80ff5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:39.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.538+0000 7f35cdc8e640 1 --2- 172.21.15.100:0/1501466179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35c8107dd0 0x7f35c80ff5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:39.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.538+0000 7f35cdc8e640 1 -- 172.21.15.100:0/1501466179 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35c81a8120 con 0x7f35c8108720 2024-07-26T13:37:39.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.538+0000 7f35cdc8e640 1 --2- 172.21.15.100:0/1501466179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35c8108720 0x7f35c80ffb20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f35c0005cc0 tx=0x7f35c002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:39.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.539+0000 7f35be7fc640 1 -- 172.21.15.100:0/1501466179 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c0003bf0 con 0x7f35c8108720 2024-07-26T13:37:39.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.539+0000 7f35be7fc640 1 -- 172.21.15.100:0/1501466179 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35c0003d90 con 0x7f35c8108720 2024-07-26T13:37:39.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.539+0000 7f35cf6f1640 1 -- 172.21.15.100:0/1501466179 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35c81a83b0 con 0x7f35c8108720 2024-07-26T13:37:39.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.539+0000 7f35be7fc640 1 -- 172.21.15.100:0/1501466179 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c002fbc0 con 0x7f35c8108720 2024-07-26T13:37:39.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.539+0000 7f35cf6f1640 1 -- 172.21.15.100:0/1501466179 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35c81a88f0 con 0x7f35c8108720 2024-07-26T13:37:39.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.540+0000 7f35b3fff640 1 -- 172.21.15.100:0/1501466179 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35a8005290 con 0x7f35c8108720 2024-07-26T13:37:39.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.541+0000 7f35be7fc640 1 -- 172.21.15.100:0/1501466179 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f35c002fd60 con 0x7f35c8108720 2024-07-26T13:37:39.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.543+0000 7f35be7fc640 1 --2- 172.21.15.100:0/1501466179 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f35b40b3cb0 0x7f35b40b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:39.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.543+0000 7f35cd48d640 1 --2- 172.21.15.100:0/1501466179 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f35b40b3cb0 0x7f35b40b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:39.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.543+0000 7f35be7fc640 1 -- 172.21.15.100:0/1501466179 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f35c0105410 con 0x7f35c8108720 2024-07-26T13:37:39.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.543+0000 7f35cd48d640 1 --2- 172.21.15.100:0/1501466179 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f35b40b3cb0 0x7f35b40b61c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f35c8100af0 tx=0x7f35c400b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:39.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.547+0000 7f35be7fc640 1 -- 172.21.15.100:0/1501466179 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f35c00cc910 con 0x7f35c8108720 2024-07-26T13:37:39.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.718+0000 7f35b3fff640 1 -- 172.21.15.100:0/1501466179 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f35a8005850 con 0x7f35c8108720 2024-07-26T13:37:39.720 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:39 smithi100 ceph-mon[32599]: pgmap v290: 97 pgs: 97 active+clean; 622 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2024-07-26T13:37:39.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.720+0000 7f35be7fc640 1 -- 172.21.15.100:0/1501466179 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+23121 (secure 0 0 0) 0x7f35c00cc210 con 0x7f35c8108720 2024-07-26T13:37:39.721 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:37:39.721 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":81,"fsid":"bf83e2e6-4b52-11ef-bcb7-c7b262605968","created":"2024-07-26T13:28:23.639785+0000","modified":"2024-07-26T13:37:17.539521+0000","last_up_change":"2024-07-26T13:35:56.475554+0000","last_in_change":"2024-07-26T13:35:37.472564+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-26T13:32:22.329969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-26T13:36:54.590525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-26T13:36:58.149545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d714036d-9dda-4cb5-af16-29f83a09dd70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6803","nonce":1480401850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6805","nonce":1480401850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6809","nonce":1480401850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6807","nonce":1480401850}]},"public_addr":"172.21.15.100:6803/1480401850","cluster_addr":"172.21.15.100:6805/1480401850","heartbeat_back_addr":"172.21.15.100:6809/1480401850","heartbeat_front_addr":"172.21.15.100:6807/1480401850","state":["exists","up"]},{"osd":1,"uuid":"3dd0eaa1-618a-495a-9819-e01c0f869ae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6801","nonce":3090877671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6803","nonce":3090877671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6807","nonce":3090877671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6805","nonce":3090877671}]},"public_addr":"172.21.15.138:6801/3090877671","cluster_addr":"172.21.15.138:6803/3090877671","heartbeat_back_addr":"172.21.15.138:6807/3090877671","heartbeat_front_addr":"172.21.15.138:6805/3090877671","state":["exists","up"]},{"osd":2,"uuid":"5c710a78-17ae-41a8-ac16-a678f5d35672","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6801","nonce":3583424790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6803","nonce":3583424790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6807","nonce":3583424790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6805","nonce":3583424790}]},"public_addr":"172.21.15.167:6801/3583424790","cluster_addr":"172.21.15.167:6803/3583424790","heartbeat_back_addr":"172.21.15.167:6807/3583424790","heartbeat_front_addr":"172.21.15.167:6805/3583424790","state":["exists","up"]},{"osd":3,"uuid":"be845495-8c31-4834-a45c-bdc576711962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6811","nonce":3626762419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6813","nonce":3626762419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6817","nonce":3626762419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6815","nonce":3626762419}]},"public_addr":"172.21.15.100:6811/3626762419","cluster_addr":"172.21.15.100:6813/3626762419","heartbeat_back_addr":"172.21.15.100:6817/3626762419","heartbeat_front_addr":"172.21.15.100:6815/3626762419","state":["exists","up"]},{"osd":4,"uuid":"520554cc-e846-46f9-af77-05a1de65a452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6809","nonce":3757334356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6811","nonce":3757334356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6815","nonce":3757334356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6813","nonce":3757334356}]},"public_addr":"172.21.15.138:6809/3757334356","cluster_addr":"172.21.15.138:6811/3757334356","heartbeat_back_addr":"172.21.15.138:6815/3757334356","heartbeat_front_addr":"172.21.15.138:6813/3757334356","state":["exists","up"]},{"osd":5,"uuid":"68dc8ad6-31d2-45b5-9aa7-62c4218d46e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6809","nonce":3239482836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6811","nonce":3239482836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6815","nonce":3239482836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6813","nonce":3239482836}]},"public_addr":"172.21.15.167:6809/3239482836","cluster_addr":"172.21.15.167:6811/3239482836","heartbeat_back_addr":"172.21.15.167:6815/3239482836","heartbeat_front_addr":"172.21.15.167:6813/3239482836","state":["exists","up"]},{"osd":6,"uuid":"420e4471-5626-41cc-a8e1-69c801196b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6819","nonce":3218415484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6821","nonce":3218415484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6825","nonce":3218415484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6823","nonce":3218415484}]},"public_addr":"172.21.15.100:6819/3218415484","cluster_addr":"172.21.15.100:6821/3218415484","heartbeat_back_addr":"172.21.15.100:6825/3218415484","heartbeat_front_addr":"172.21.15.100:6823/3218415484","state":["exists","up"]},{"osd":7,"uuid":"a27b097e-207d-44c3-af63-544f228b83ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6817","nonce":45541749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6819","nonce":45541749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6823","nonce":45541749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6821","nonce":45541749}]},"public_addr":"172.21.15.138:6817/45541749","cluster_addr":"172.21.15.138:6819/45541749","heartbeat_back_addr":"172.21.15.138:6823/45541749","heartbeat_front_addr":"172.21.15.138:6821/45541749","state":["exists","up"]},{"osd":8,"uuid":"3e865858-92cc-48a0-a464-5767d19e8ac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6817","nonce":4031159803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6819","nonce":4031159803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6823","nonce":4031159803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6821","nonce":4031159803}]},"public_addr":"172.21.15.167:6817/4031159803","cluster_addr":"172.21.15.167:6819/4031159803","heartbeat_back_addr":"172.21.15.167:6823/4031159803","heartbeat_front_addr":"172.21.15.167:6821/4031159803","state":["exists","up"]},{"osd":9,"uuid":"8b05b14a-1195-42f5-ac01-d5634cb9c888","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6827","nonce":1840140273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6829","nonce":1840140273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6833","nonce":1840140273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6831","nonce":1840140273}]},"public_addr":"172.21.15.100:6827/1840140273","cluster_addr":"172.21.15.100:6829/1840140273","heartbeat_back_addr":"172.21.15.100:6833/1840140273","heartbeat_front_addr":"172.21.15.100:6831/1840140273","state":["exists","up"]},{"osd":10,"uuid":"ab77ebe7-5254-4dc3-bc54-625698e84afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6825","nonce":703382899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6827","nonce":703382899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6831","nonce":703382899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6829","nonce":703382899}]},"public_addr":"172.21.15.138:6825/703382899","cluster_addr":"172.21.15.138:6827/703382899","heartbeat_back_addr":"172.21.15.138:6831/703382899","heartbeat_front_addr":"172.21.15.138:6829/703382899","state":["exists","up"]},{"osd":11,"uuid":"229cf91e-e9ef-48f6-a677-c5c5705b315a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6825","nonce":1032343068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6827","nonce":1032343068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6831","nonce":1032343068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6829","nonce":1032343068}]},"public_addr":"172.21.15.167:6825/1032343068","cluster_addr":"172.21.15.167:6827/1032343068","heartbeat_back_addr":"172.21.15.167:6831/1032343068","heartbeat_front_addr":"172.21.15.167:6829/1032343068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:31:35.800702+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:31:58.087922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:32:20.463183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:32:42.920598+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:33:06.332655+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:33:29.381407+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:33:53.975963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:34:18.226813+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:34:41.845770+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:35:05.722275+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:35:30.862704+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:35:54.738697+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:0/1874993112":"2024-07-27T13:30:16.253096+0000","172.21.15.100:0/2364620305":"2024-07-27T13:28:45.340334+0000","172.21.15.100:6800/3096076075":"2024-07-27T13:29:07.979178+0000","172.21.15.100:0/1894966501":"2024-07-27T13:30:16.253096+0000","172.21.15.100:0/3411776672":"2024-07-27T13:28:45.340334+0000","172.21.15.100:0/2853423120":"2024-07-27T13:29:07.979178+0000","172.21.15.100:6800/3025873570":"2024-07-27T13:28:45.340334+0000","172.21.15.100:0/4887181":"2024-07-27T13:29:07.979178+0000","172.21.15.100:6801/908675140":"2024-07-27T13:30:16.253096+0000","172.21.15.100:6801/3025873570":"2024-07-27T13:28:45.340334+0000","172.21.15.100:6800/908675140":"2024-07-27T13:30:16.253096+0000","172.21.15.100:6801/3096076075":"2024-07-27T13:29:07.979178+0000","172.21.15.100:0/2302632052":"2024-07-27T13:29:07.979178+0000","172.21.15.100:0/3736098915":"2024-07-27T13:28:45.340334+0000","172.21.15.100:0/3243030560":"2024-07-27T13:30:16.253096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-26T13:37:39.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.723+0000 7f35b3fff640 1 -- 172.21.15.100:0/1501466179 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f35b40b3cb0 msgr2=0x7f35b40b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:39.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.723+0000 7f35b3fff640 1 --2- 172.21.15.100:0/1501466179 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f35b40b3cb0 0x7f35b40b61c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f35c8100af0 tx=0x7f35c400b390 comp rx=0 tx=0).stop 2024-07-26T13:37:39.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.723+0000 7f35b3fff640 1 -- 172.21.15.100:0/1501466179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35c8108720 msgr2=0x7f35c80ffb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:39.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.723+0000 7f35b3fff640 1 --2- 172.21.15.100:0/1501466179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35c8108720 0x7f35c80ffb20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f35c0005cc0 tx=0x7f35c002f380 comp rx=0 tx=0).stop 2024-07-26T13:37:39.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.723+0000 7f35b3fff640 1 -- 172.21.15.100:0/1501466179 shutdown_connections 2024-07-26T13:37:39.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.723+0000 7f35b3fff640 1 --2- 172.21.15.100:0/1501466179 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f35b40b3cb0 0x7f35b40b61c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:39.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.723+0000 7f35b3fff640 1 --2- 172.21.15.100:0/1501466179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35c8108720 0x7f35c80ffb20 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:39.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.723+0000 7f35b3fff640 1 --2- 172.21.15.100:0/1501466179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35c8107dd0 0x7f35c80ff5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:39.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.723+0000 7f35b3fff640 1 --2- 172.21.15.100:0/1501466179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35c8102490 0x7f35c80ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:39.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.723+0000 7f35b3fff640 1 -- 172.21.15.100:0/1501466179 >> 172.21.15.100:0/1501466179 conn(0x7f35c8079160 msgr2=0x7f35c806ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:39.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.723+0000 7f35b3fff640 1 -- 172.21.15.100:0/1501466179 shutdown_connections 2024-07-26T13:37:39.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:39.723+0000 7f35b3fff640 1 -- 172.21.15.100:0/1501466179 wait complete. 2024-07-26T13:37:39.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:39 smithi167 ceph-mon[37140]: pgmap v290: 97 pgs: 97 active+clean; 622 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2024-07-26T13:37:39.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:39 smithi138 ceph-mon[36954]: pgmap v290: 97 pgs: 97 active+clean; 622 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2024-07-26T13:37:40.467 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-07-26T13:37:40.467 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph pg dump --format=json 2024-07-26T13:37:40.722 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:37:40.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:40 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/1501466179' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-26T13:37:40.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:40 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/1501466179' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-26T13:37:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:40 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/1501466179' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-26T13:37:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:41 smithi100 ceph-mon[32599]: pgmap v291: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-07-26T13:37:41.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:41 smithi167 ceph-mon[37140]: pgmap v291: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-07-26T13:37:41.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:41 smithi138 ceph-mon[36954]: pgmap v291: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-07-26T13:37:42.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.482+0000 7f72170df640 1 -- 172.21.15.100:0/3056990993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f721010d8a0 msgr2=0x7f721010fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:42.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.482+0000 7f72170df640 1 --2- 172.21.15.100:0/3056990993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f721010d8a0 0x7f721010fd40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f71f8009900 tx=0x7f71f802a410 comp rx=0 tx=0).stop 2024-07-26T13:37:42.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.482+0000 7f72170df640 1 -- 172.21.15.100:0/3056990993 shutdown_connections 2024-07-26T13:37:42.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.482+0000 7f72170df640 1 --2- 172.21.15.100:0/3056990993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f721010d8a0 0x7f721010fd40 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:42.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.482+0000 7f72170df640 1 --2- 172.21.15.100:0/3056990993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7210104d20 0x7f721010d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:42.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.482+0000 7f72170df640 1 --2- 172.21.15.100:0/3056990993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72101042f0 0x7f7210104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:42.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.482+0000 7f72170df640 1 -- 172.21.15.100:0/3056990993 >> 172.21.15.100:0/3056990993 conn(0x7f7210079170 msgr2=0x7f7210079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:42.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.482+0000 7f72170df640 1 -- 172.21.15.100:0/3056990993 shutdown_connections 2024-07-26T13:37:42.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.482+0000 7f72170df640 1 -- 172.21.15.100:0/3056990993 wait complete. 2024-07-26T13:37:42.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.483+0000 7f72170df640 1 Processor -- start 2024-07-26T13:37:42.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.483+0000 7f72170df640 1 -- start start 2024-07-26T13:37:42.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.483+0000 7f72170df640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72101042f0 0x7f721019c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:42.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.483+0000 7f72170df640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7210104d20 0x7f721019c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:42.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.483+0000 7f72170df640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f721010d8a0 0x7f72101a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:42.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.483+0000 7f72170df640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7210113510 con 0x7f7210104d20 2024-07-26T13:37:42.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.483+0000 7f72170df640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7210113390 con 0x7f721010d8a0 2024-07-26T13:37:42.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.483+0000 7f72170df640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7210113690 con 0x7f72101042f0 2024-07-26T13:37:42.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.483+0000 7f720ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7210104d20 0x7f721019c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:42.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.483+0000 7f720ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7210104d20 0x7f721019c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50010/0 (socket says 172.21.15.100:50010) 2024-07-26T13:37:42.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.483+0000 7f720ffff640 1 -- 172.21.15.100:0/1410953337 learned_addr learned my addr 172.21.15.100:0/1410953337 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:37:42.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.483+0000 7f7214e7b640 1 --2- 172.21.15.100:0/1410953337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72101042f0 0x7f721019c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:42.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.483+0000 7f721567c640 1 --2- 172.21.15.100:0/1410953337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f721010d8a0 0x7f72101a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:42.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.483+0000 7f720ffff640 1 -- 172.21.15.100:0/1410953337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72101042f0 msgr2=0x7f721019c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:42.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.483+0000 7f720ffff640 1 --2- 172.21.15.100:0/1410953337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72101042f0 0x7f721019c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:42.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.483+0000 7f720ffff640 1 -- 172.21.15.100:0/1410953337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f721010d8a0 msgr2=0x7f72101a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:42.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.483+0000 7f720ffff640 1 --2- 172.21.15.100:0/1410953337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f721010d8a0 0x7f72101a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:42.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.483+0000 7f720ffff640 1 -- 172.21.15.100:0/1410953337 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72101a20d0 con 0x7f7210104d20 2024-07-26T13:37:42.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.483+0000 7f721567c640 1 --2- 172.21.15.100:0/1410953337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f721010d8a0 0x7f72101a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:37:42.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.483+0000 7f7214e7b640 1 --2- 172.21.15.100:0/1410953337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72101042f0 0x7f721019c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:37:42.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.484+0000 7f720ffff640 1 --2- 172.21.15.100:0/1410953337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7210104d20 0x7f721019c700 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f720000b9c0 tx=0x7f720000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:42.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.484+0000 7f720dffb640 1 -- 172.21.15.100:0/1410953337 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7200019070 con 0x7f7210104d20 2024-07-26T13:37:42.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.484+0000 7f720dffb640 1 -- 172.21.15.100:0/1410953337 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f720000cba0 con 0x7f7210104d20 2024-07-26T13:37:42.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.484+0000 7f72170df640 1 -- 172.21.15.100:0/1410953337 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72101a23c0 con 0x7f7210104d20 2024-07-26T13:37:42.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.484+0000 7f720dffb640 1 -- 172.21.15.100:0/1410953337 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72000125a0 con 0x7f7210104d20 2024-07-26T13:37:42.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.484+0000 7f72170df640 1 -- 172.21.15.100:0/1410953337 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72101a8b20 con 0x7f7210104d20 2024-07-26T13:37:42.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.485+0000 7f72170df640 1 -- 172.21.15.100:0/1410953337 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f721006b890 con 0x7f7210104d20 2024-07-26T13:37:42.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.486+0000 7f720dffb640 1 -- 172.21.15.100:0/1410953337 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f720001c070 con 0x7f7210104d20 2024-07-26T13:37:42.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.487+0000 7f720dffb640 1 --2- 172.21.15.100:0/1410953337 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f71fc0b3cb0 0x7f71fc0b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:42.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.488+0000 7f7214e7b640 1 --2- 172.21.15.100:0/1410953337 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f71fc0b3cb0 0x7f71fc0b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:42.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.488+0000 7f720dffb640 1 -- 172.21.15.100:0/1410953337 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f72000e3a20 con 0x7f7210104d20 2024-07-26T13:37:42.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.488+0000 7f7214e7b640 1 --2- 172.21.15.100:0/1410953337 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f71fc0b3cb0 0x7f71fc0b61c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f721006bf90 tx=0x7f7208005e70 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:42.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.490+0000 7f720dffb640 1 -- 172.21.15.100:0/1410953337 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f72000aaf20 con 0x7f7210104d20 2024-07-26T13:37:42.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.660+0000 7f72170df640 1 -- 172.21.15.100:0/1410953337 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f72101007e0 con 0x7f71fc0b3cb0 2024-07-26T13:37:42.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.664+0000 7f720dffb640 1 -- 172.21.15.100:0/1410953337 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258943 (secure 0 0 0) 0x7f72101007e0 con 0x7f71fc0b3cb0 2024-07-26T13:37:42.668 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:37:42.669 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-07-26T13:37:42.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.668+0000 7f72170df640 1 -- 172.21.15.100:0/1410953337 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f71fc0b3cb0 msgr2=0x7f71fc0b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:42.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.668+0000 7f72170df640 1 --2- 172.21.15.100:0/1410953337 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f71fc0b3cb0 0x7f71fc0b61c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f721006bf90 tx=0x7f7208005e70 comp rx=0 tx=0).stop 2024-07-26T13:37:42.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.668+0000 7f72170df640 1 -- 172.21.15.100:0/1410953337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7210104d20 msgr2=0x7f721019c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:42.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.668+0000 7f72170df640 1 --2- 172.21.15.100:0/1410953337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7210104d20 0x7f721019c700 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f720000b9c0 tx=0x7f720000be80 comp rx=0 tx=0).stop 2024-07-26T13:37:42.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.668+0000 7f72170df640 1 -- 172.21.15.100:0/1410953337 shutdown_connections 2024-07-26T13:37:42.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.668+0000 7f72170df640 1 --2- 172.21.15.100:0/1410953337 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f71fc0b3cb0 0x7f71fc0b61c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:42.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.668+0000 7f72170df640 1 --2- 172.21.15.100:0/1410953337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f721010d8a0 0x7f72101a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:42.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.668+0000 7f72170df640 1 --2- 172.21.15.100:0/1410953337 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7210104d20 0x7f721019c700 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:42.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.668+0000 7f72170df640 1 --2- 172.21.15.100:0/1410953337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72101042f0 0x7f721019c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:42.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.668+0000 7f72170df640 1 -- 172.21.15.100:0/1410953337 >> 172.21.15.100:0/1410953337 conn(0x7f7210079170 msgr2=0x7f72101000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:42.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.668+0000 7f72170df640 1 -- 172.21.15.100:0/1410953337 shutdown_connections 2024-07-26T13:37:42.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:42.668+0000 7f72170df640 1 -- 172.21.15.100:0/1410953337 wait complete. 2024-07-26T13:37:43.509 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-07-26T13:37:42.369096+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651934,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":244,"num_read_kb":215,"num_write":285,"num_write_kb":2257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":2952484,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368324,"kb_used_data":10836,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573500,"statfs":{"total":1151940427776,"available":1151563264000,"internally_reserved":0,"allocated":11096064,"data_stored":7269966,"data_compressed":177282,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":34727,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":73,"num_read_kb":70,"num_write":22,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003470"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.547178+0000","last_change":"2024-07-26T13:36:59.462289+0000","last_active":"2024-07-26T13:37:17.547178+0000","last_peered":"2024-07-26T13:37:17.547178+0000","last_clean":"2024-07-26T13:37:17.547178+0000","last_became_active":"2024-07-26T13:36:59.460311+0000","last_became_peered":"2024-07-26T13:36:59.460311+0000","last_unstale":"2024-07-26T13:37:17.547178+0000","last_undegraded":"2024-07-26T13:37:17.547178+0000","last_fullsized":"2024-07-26T13:37:17.547178+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:04:59.123716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.545919+0000","last_change":"2024-07-26T13:36:59.463218+0000","last_active":"2024-07-26T13:37:17.545919+0000","last_peered":"2024-07-26T13:37:17.545919+0000","last_clean":"2024-07-26T13:37:17.545919+0000","last_became_active":"2024-07-26T13:36:59.462889+0000","last_became_peered":"2024-07-26T13:36:59.462889+0000","last_unstale":"2024-07-26T13:37:17.545919+0000","last_undegraded":"2024-07-26T13:37:17.545919+0000","last_fullsized":"2024-07-26T13:37:17.545919+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:30:46.788669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.545077+0000","last_change":"2024-07-26T13:36:59.456113+0000","last_active":"2024-07-26T13:37:17.545077+0000","last_peered":"2024-07-26T13:37:17.545077+0000","last_clean":"2024-07-26T13:37:17.545077+0000","last_became_active":"2024-07-26T13:36:59.455622+0000","last_became_peered":"2024-07-26T13:36:59.455622+0000","last_unstale":"2024-07-26T13:37:17.545077+0000","last_undegraded":"2024-07-26T13:37:17.545077+0000","last_fullsized":"2024-07-26T13:37:17.545077+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:36:26.474799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.556669+0000","last_change":"2024-07-26T13:37:17.555947+0000","last_active":"2024-07-26T13:37:17.556669+0000","last_peered":"2024-07-26T13:37:17.556669+0000","last_clean":"2024-07-26T13:37:17.556669+0000","last_became_active":"2024-07-26T13:37:17.555607+0000","last_became_peered":"2024-07-26T13:37:17.555607+0000","last_unstale":"2024-07-26T13:37:17.556669+0000","last_undegraded":"2024-07-26T13:37:17.556669+0000","last_fullsized":"2024-07-26T13:37:17.556669+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:19:59.603225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551495+0000","last_change":"2024-07-26T13:36:59.474896+0000","last_active":"2024-07-26T13:37:17.551495+0000","last_peered":"2024-07-26T13:37:17.551495+0000","last_clean":"2024-07-26T13:37:17.551495+0000","last_became_active":"2024-07-26T13:36:59.474516+0000","last_became_peered":"2024-07-26T13:36:59.474516+0000","last_unstale":"2024-07-26T13:37:17.551495+0000","last_undegraded":"2024-07-26T13:37:17.551495+0000","last_fullsized":"2024-07-26T13:37:17.551495+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:38:18.994280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.556329+0000","last_change":"2024-07-26T13:37:17.556191+0000","last_active":"2024-07-26T13:37:17.556329+0000","last_peered":"2024-07-26T13:37:17.556329+0000","last_clean":"2024-07-26T13:37:17.556329+0000","last_became_active":"2024-07-26T13:37:17.555715+0000","last_became_peered":"2024-07-26T13:37:17.555715+0000","last_unstale":"2024-07-26T13:37:17.556329+0000","last_undegraded":"2024-07-26T13:37:17.556329+0000","last_fullsized":"2024-07-26T13:37:17.556329+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:03:18.352049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550568+0000","last_change":"2024-07-26T13:36:59.469515+0000","last_active":"2024-07-26T13:37:17.550568+0000","last_peered":"2024-07-26T13:37:17.550568+0000","last_clean":"2024-07-26T13:37:17.550568+0000","last_became_active":"2024-07-26T13:36:59.469106+0000","last_became_peered":"2024-07-26T13:36:59.469106+0000","last_unstale":"2024-07-26T13:37:17.550568+0000","last_undegraded":"2024-07-26T13:37:17.550568+0000","last_fullsized":"2024-07-26T13:37:17.550568+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:10:05.455329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.549794+0000","last_change":"2024-07-26T13:37:17.549365+0000","last_active":"2024-07-26T13:37:17.549794+0000","last_peered":"2024-07-26T13:37:17.549794+0000","last_clean":"2024-07-26T13:37:17.549794+0000","last_became_active":"2024-07-26T13:37:17.549036+0000","last_became_peered":"2024-07-26T13:37:17.549036+0000","last_unstale":"2024-07-26T13:37:17.549794+0000","last_undegraded":"2024-07-26T13:37:17.549794+0000","last_fullsized":"2024-07-26T13:37:17.549794+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:27:43.930981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.544175+0000","last_change":"2024-07-26T13:36:59.459306+0000","last_active":"2024-07-26T13:37:17.544175+0000","last_peered":"2024-07-26T13:37:17.544175+0000","last_clean":"2024-07-26T13:37:17.544175+0000","last_became_active":"2024-07-26T13:36:59.458714+0000","last_became_peered":"2024-07-26T13:36:59.458714+0000","last_unstale":"2024-07-26T13:37:17.544175+0000","last_undegraded":"2024-07-26T13:37:17.544175+0000","last_fullsized":"2024-07-26T13:37:17.544175+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:34:18.512454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.547670+0000","last_change":"2024-07-26T13:36:59.466556+0000","last_active":"2024-07-26T13:37:17.547670+0000","last_peered":"2024-07-26T13:37:17.547670+0000","last_clean":"2024-07-26T13:37:17.547670+0000","last_became_active":"2024-07-26T13:36:59.466147+0000","last_became_peered":"2024-07-26T13:36:59.466147+0000","last_unstale":"2024-07-26T13:37:17.547670+0000","last_undegraded":"2024-07-26T13:37:17.547670+0000","last_fullsized":"2024-07-26T13:37:17.547670+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:43:11.359314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551125+0000","last_change":"2024-07-26T13:36:59.473685+0000","last_active":"2024-07-26T13:37:17.551125+0000","last_peered":"2024-07-26T13:37:17.551125+0000","last_clean":"2024-07-26T13:37:17.551125+0000","last_became_active":"2024-07-26T13:36:59.473314+0000","last_became_peered":"2024-07-26T13:36:59.473314+0000","last_unstale":"2024-07-26T13:37:17.551125+0000","last_undegraded":"2024-07-26T13:37:17.551125+0000","last_fullsized":"2024-07-26T13:37:17.551125+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:41:51.612348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.561132+0000","last_change":"2024-07-26T13:36:59.471586+0000","last_active":"2024-07-26T13:37:17.561132+0000","last_peered":"2024-07-26T13:37:17.561132+0000","last_clean":"2024-07-26T13:37:17.561132+0000","last_became_active":"2024-07-26T13:36:59.470995+0000","last_became_peered":"2024-07-26T13:36:59.470995+0000","last_unstale":"2024-07-26T13:37:17.561132+0000","last_undegraded":"2024-07-26T13:37:17.561132+0000","last_fullsized":"2024-07-26T13:37:17.561132+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:54:07.957937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.565391+0000","last_change":"2024-07-26T13:36:59.460018+0000","last_active":"2024-07-26T13:37:17.565391+0000","last_peered":"2024-07-26T13:37:17.565391+0000","last_clean":"2024-07-26T13:37:17.565391+0000","last_became_active":"2024-07-26T13:36:59.459671+0000","last_became_peered":"2024-07-26T13:36:59.459671+0000","last_unstale":"2024-07-26T13:37:17.565391+0000","last_undegraded":"2024-07-26T13:37:17.565391+0000","last_fullsized":"2024-07-26T13:37:17.565391+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:19:33.923360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.544927+0000","last_change":"2024-07-26T13:36:59.470271+0000","last_active":"2024-07-26T13:37:17.544927+0000","last_peered":"2024-07-26T13:37:17.544927+0000","last_clean":"2024-07-26T13:37:17.544927+0000","last_became_active":"2024-07-26T13:36:59.469941+0000","last_became_peered":"2024-07-26T13:36:59.469941+0000","last_unstale":"2024-07-26T13:37:17.544927+0000","last_undegraded":"2024-07-26T13:37:17.544927+0000","last_fullsized":"2024-07-26T13:37:17.544927+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:38:39.329366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.544960+0000","last_change":"2024-07-26T13:36:59.460133+0000","last_active":"2024-07-26T13:37:17.544960+0000","last_peered":"2024-07-26T13:37:17.544960+0000","last_clean":"2024-07-26T13:37:17.544960+0000","last_became_active":"2024-07-26T13:36:59.459794+0000","last_became_peered":"2024-07-26T13:36:59.459794+0000","last_unstale":"2024-07-26T13:37:17.544960+0000","last_undegraded":"2024-07-26T13:37:17.544960+0000","last_fullsized":"2024-07-26T13:37:17.544960+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:15:36.024347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.543706+0000","last_change":"2024-07-26T13:36:59.462390+0000","last_active":"2024-07-26T13:37:17.543706+0000","last_peered":"2024-07-26T13:37:17.543706+0000","last_clean":"2024-07-26T13:37:17.543706+0000","last_became_active":"2024-07-26T13:36:59.461968+0000","last_became_peered":"2024-07-26T13:36:59.461968+0000","last_unstale":"2024-07-26T13:37:17.543706+0000","last_undegraded":"2024-07-26T13:37:17.543706+0000","last_fullsized":"2024-07-26T13:37:17.543706+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:49:47.142472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.547331+0000","last_change":"2024-07-26T13:36:59.460206+0000","last_active":"2024-07-26T13:37:17.547331+0000","last_peered":"2024-07-26T13:37:17.547331+0000","last_clean":"2024-07-26T13:37:17.547331+0000","last_became_active":"2024-07-26T13:36:59.459811+0000","last_became_peered":"2024-07-26T13:36:59.459811+0000","last_unstale":"2024-07-26T13:37:17.547331+0000","last_undegraded":"2024-07-26T13:37:17.547331+0000","last_fullsized":"2024-07-26T13:37:17.547331+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:46:57.112314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.545663+0000","last_change":"2024-07-26T13:36:59.460197+0000","last_active":"2024-07-26T13:37:17.545663+0000","last_peered":"2024-07-26T13:37:17.545663+0000","last_clean":"2024-07-26T13:37:17.545663+0000","last_became_active":"2024-07-26T13:36:59.459686+0000","last_became_peered":"2024-07-26T13:36:59.459686+0000","last_unstale":"2024-07-26T13:37:17.545663+0000","last_undegraded":"2024-07-26T13:37:17.545663+0000","last_fullsized":"2024-07-26T13:37:17.545663+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T13:39:22.407376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550550+0000","last_change":"2024-07-26T13:36:59.462545+0000","last_active":"2024-07-26T13:37:17.550550+0000","last_peered":"2024-07-26T13:37:17.550550+0000","last_clean":"2024-07-26T13:37:17.550550+0000","last_became_active":"2024-07-26T13:36:59.462115+0000","last_became_peered":"2024-07-26T13:36:59.462115+0000","last_unstale":"2024-07-26T13:37:17.550550+0000","last_undegraded":"2024-07-26T13:37:17.550550+0000","last_fullsized":"2024-07-26T13:37:17.550550+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T21:44:13.715076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.547646+0000","last_change":"2024-07-26T13:36:59.465317+0000","last_active":"2024-07-26T13:37:17.547646+0000","last_peered":"2024-07-26T13:37:17.547646+0000","last_clean":"2024-07-26T13:37:17.547646+0000","last_became_active":"2024-07-26T13:36:59.464943+0000","last_became_peered":"2024-07-26T13:36:59.464943+0000","last_unstale":"2024-07-26T13:37:17.547646+0000","last_undegraded":"2024-07-26T13:37:17.547646+0000","last_fullsized":"2024-07-26T13:37:17.547646+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T13:53:54.160584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.565505+0000","last_change":"2024-07-26T13:36:59.462884+0000","last_active":"2024-07-26T13:37:17.565505+0000","last_peered":"2024-07-26T13:37:17.565505+0000","last_clean":"2024-07-26T13:37:17.565505+0000","last_became_active":"2024-07-26T13:36:59.462564+0000","last_became_peered":"2024-07-26T13:36:59.462564+0000","last_unstale":"2024-07-26T13:37:17.565505+0000","last_undegraded":"2024-07-26T13:37:17.565505+0000","last_fullsized":"2024-07-26T13:37:17.565505+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:50:19.549148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.543901+0000","last_change":"2024-07-26T13:36:59.468716+0000","last_active":"2024-07-26T13:37:17.543901+0000","last_peered":"2024-07-26T13:37:17.543901+0000","last_clean":"2024-07-26T13:37:17.543901+0000","last_became_active":"2024-07-26T13:36:59.468048+0000","last_became_peered":"2024-07-26T13:36:59.468048+0000","last_unstale":"2024-07-26T13:37:17.543901+0000","last_undegraded":"2024-07-26T13:37:17.543901+0000","last_fullsized":"2024-07-26T13:37:17.543901+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:51:29.510047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.542939+0000","last_change":"2024-07-26T13:36:59.466565+0000","last_active":"2024-07-26T13:37:17.542939+0000","last_peered":"2024-07-26T13:37:17.542939+0000","last_clean":"2024-07-26T13:37:17.542939+0000","last_became_active":"2024-07-26T13:36:59.466235+0000","last_became_peered":"2024-07-26T13:36:59.466235+0000","last_unstale":"2024-07-26T13:37:17.542939+0000","last_undegraded":"2024-07-26T13:37:17.542939+0000","last_fullsized":"2024-07-26T13:37:17.542939+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:28:44.326821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.546853+0000","last_change":"2024-07-26T13:36:59.458753+0000","last_active":"2024-07-26T13:37:17.546853+0000","last_peered":"2024-07-26T13:37:17.546853+0000","last_clean":"2024-07-26T13:37:17.546853+0000","last_became_active":"2024-07-26T13:36:59.458333+0000","last_became_peered":"2024-07-26T13:36:59.458333+0000","last_unstale":"2024-07-26T13:37:17.546853+0000","last_undegraded":"2024-07-26T13:37:17.546853+0000","last_fullsized":"2024-07-26T13:37:17.546853+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:33:54.806212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.543527+0000","last_change":"2024-07-26T13:36:59.468657+0000","last_active":"2024-07-26T13:37:17.543527+0000","last_peered":"2024-07-26T13:37:17.543527+0000","last_clean":"2024-07-26T13:37:17.543527+0000","last_became_active":"2024-07-26T13:36:59.468121+0000","last_became_peered":"2024-07-26T13:36:59.468121+0000","last_unstale":"2024-07-26T13:37:17.543527+0000","last_undegraded":"2024-07-26T13:37:17.543527+0000","last_fullsized":"2024-07-26T13:37:17.543527+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:23:53.959836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550962+0000","last_change":"2024-07-26T13:36:59.469067+0000","last_active":"2024-07-26T13:37:17.550962+0000","last_peered":"2024-07-26T13:37:17.550962+0000","last_clean":"2024-07-26T13:37:17.550962+0000","last_became_active":"2024-07-26T13:36:59.468706+0000","last_became_peered":"2024-07-26T13:36:59.468706+0000","last_unstale":"2024-07-26T13:37:17.550962+0000","last_undegraded":"2024-07-26T13:37:17.550962+0000","last_fullsized":"2024-07-26T13:37:17.550962+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:21:35.974618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.548550+0000","last_change":"2024-07-26T13:36:59.469807+0000","last_active":"2024-07-26T13:37:17.548550+0000","last_peered":"2024-07-26T13:37:17.548550+0000","last_clean":"2024-07-26T13:37:17.548550+0000","last_became_active":"2024-07-26T13:36:59.469020+0000","last_became_peered":"2024-07-26T13:36:59.469020+0000","last_unstale":"2024-07-26T13:37:17.548550+0000","last_undegraded":"2024-07-26T13:37:17.548550+0000","last_fullsized":"2024-07-26T13:37:17.548550+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:29:57.931598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.565302+0000","last_change":"2024-07-26T13:36:59.465236+0000","last_active":"2024-07-26T13:37:17.565302+0000","last_peered":"2024-07-26T13:37:17.565302+0000","last_clean":"2024-07-26T13:37:17.565302+0000","last_became_active":"2024-07-26T13:36:59.464415+0000","last_became_peered":"2024-07-26T13:36:59.464415+0000","last_unstale":"2024-07-26T13:37:17.565302+0000","last_undegraded":"2024-07-26T13:37:17.565302+0000","last_fullsized":"2024-07-26T13:37:17.565302+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:46:20.373688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.545250+0000","last_change":"2024-07-26T13:36:59.466113+0000","last_active":"2024-07-26T13:37:17.545250+0000","last_peered":"2024-07-26T13:37:17.545250+0000","last_clean":"2024-07-26T13:37:17.545250+0000","last_became_active":"2024-07-26T13:36:59.465743+0000","last_became_peered":"2024-07-26T13:36:59.465743+0000","last_unstale":"2024-07-26T13:37:17.545250+0000","last_undegraded":"2024-07-26T13:37:17.545250+0000","last_fullsized":"2024-07-26T13:37:17.545250+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:14:41.839204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551356+0000","last_change":"2024-07-26T13:36:59.460672+0000","last_active":"2024-07-26T13:37:17.551356+0000","last_peered":"2024-07-26T13:37:17.551356+0000","last_clean":"2024-07-26T13:37:17.551356+0000","last_became_active":"2024-07-26T13:36:59.460247+0000","last_became_peered":"2024-07-26T13:36:59.460247+0000","last_unstale":"2024-07-26T13:37:17.551356+0000","last_undegraded":"2024-07-26T13:37:17.551356+0000","last_fullsized":"2024-07-26T13:37:17.551356+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:37:51.109933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551707+0000","last_change":"2024-07-26T13:36:59.464784+0000","last_active":"2024-07-26T13:37:17.551707+0000","last_peered":"2024-07-26T13:37:17.551707+0000","last_clean":"2024-07-26T13:37:17.551707+0000","last_became_active":"2024-07-26T13:36:59.464261+0000","last_became_peered":"2024-07-26T13:36:59.464261+0000","last_unstale":"2024-07-26T13:37:17.551707+0000","last_undegraded":"2024-07-26T13:37:17.551707+0000","last_fullsized":"2024-07-26T13:37:17.551707+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:43:10.134037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.546345+0000","last_change":"2024-07-26T13:36:59.460421+0000","last_active":"2024-07-26T13:37:17.546345+0000","last_peered":"2024-07-26T13:37:17.546345+0000","last_clean":"2024-07-26T13:37:17.546345+0000","last_became_active":"2024-07-26T13:36:59.460010+0000","last_became_peered":"2024-07-26T13:36:59.460010+0000","last_unstale":"2024-07-26T13:37:17.546345+0000","last_undegraded":"2024-07-26T13:37:17.546345+0000","last_fullsized":"2024-07-26T13:37:17.546345+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:16:21.651673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.553536+0000","last_change":"2024-07-26T13:36:59.468826+0000","last_active":"2024-07-26T13:37:17.553536+0000","last_peered":"2024-07-26T13:37:17.553536+0000","last_clean":"2024-07-26T13:37:17.553536+0000","last_became_active":"2024-07-26T13:36:59.468406+0000","last_became_peered":"2024-07-26T13:36:59.468406+0000","last_unstale":"2024-07-26T13:37:17.553536+0000","last_undegraded":"2024-07-26T13:37:17.553536+0000","last_fullsized":"2024-07-26T13:37:17.553536+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:22:10.590545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":74,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:40.598532+0000","last_change":"2024-07-26T13:37:17.559014+0000","last_active":"2024-07-26T13:37:40.598532+0000","last_peered":"2024-07-26T13:37:40.598532+0000","last_clean":"2024-07-26T13:37:40.598532+0000","last_became_active":"2024-07-26T13:37:17.552325+0000","last_became_peered":"2024-07-26T13:37:17.552325+0000","last_unstale":"2024-07-26T13:37:40.598532+0000","last_undegraded":"2024-07-26T13:37:40.598532+0000","last_fullsized":"2024-07-26T13:37:40.598532+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:26:26.790337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.563761+0000","last_change":"2024-07-26T13:37:17.562871+0000","last_active":"2024-07-26T13:37:17.563761+0000","last_peered":"2024-07-26T13:37:17.563761+0000","last_clean":"2024-07-26T13:37:17.563761+0000","last_became_active":"2024-07-26T13:37:17.562363+0000","last_became_peered":"2024-07-26T13:37:17.562363+0000","last_unstale":"2024-07-26T13:37:17.563761+0000","last_undegraded":"2024-07-26T13:37:17.563761+0000","last_fullsized":"2024-07-26T13:37:17.563761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:25:36.172091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"79'3","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.547720+0000","last_change":"2024-07-26T13:36:56.434778+0000","last_active":"2024-07-26T13:37:17.547720+0000","last_peered":"2024-07-26T13:37:17.547720+0000","last_clean":"2024-07-26T13:37:17.547720+0000","last_became_active":"2024-07-26T13:36:56.434288+0000","last_became_peered":"2024-07-26T13:36:56.434288+0000","last_unstale":"2024-07-26T13:37:17.547720+0000","last_undegraded":"2024-07-26T13:37:17.547720+0000","last_fullsized":"2024-07-26T13:37:17.547720+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:24:22.990358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.565086+0000","last_change":"2024-07-26T13:36:56.424665+0000","last_active":"2024-07-26T13:37:17.565086+0000","last_peered":"2024-07-26T13:37:17.565086+0000","last_clean":"2024-07-26T13:37:17.565086+0000","last_became_active":"2024-07-26T13:36:56.424290+0000","last_became_peered":"2024-07-26T13:36:56.424290+0000","last_unstale":"2024-07-26T13:37:17.565086+0000","last_undegraded":"2024-07-26T13:37:17.565086+0000","last_fullsized":"2024-07-26T13:37:17.565086+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:49:59.520194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.542987+0000","last_change":"2024-07-26T13:36:59.454609+0000","last_active":"2024-07-26T13:37:17.542987+0000","last_peered":"2024-07-26T13:37:17.542987+0000","last_clean":"2024-07-26T13:37:17.542987+0000","last_became_active":"2024-07-26T13:36:59.454228+0000","last_became_peered":"2024-07-26T13:36:59.454228+0000","last_unstale":"2024-07-26T13:37:17.542987+0000","last_undegraded":"2024-07-26T13:37:17.542987+0000","last_fullsized":"2024-07-26T13:37:17.542987+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:02:06.244433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.547076+0000","last_change":"2024-07-26T13:36:56.430268+0000","last_active":"2024-07-26T13:37:17.547076+0000","last_peered":"2024-07-26T13:37:17.547076+0000","last_clean":"2024-07-26T13:37:17.547076+0000","last_became_active":"2024-07-26T13:36:56.428583+0000","last_became_peered":"2024-07-26T13:36:56.428583+0000","last_unstale":"2024-07-26T13:37:17.547076+0000","last_undegraded":"2024-07-26T13:37:17.547076+0000","last_fullsized":"2024-07-26T13:37:17.547076+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:07:05.864886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.544501+0000","last_change":"2024-07-26T13:36:59.456188+0000","last_active":"2024-07-26T13:37:17.544501+0000","last_peered":"2024-07-26T13:37:17.544501+0000","last_clean":"2024-07-26T13:37:17.544501+0000","last_became_active":"2024-07-26T13:36:59.455842+0000","last_became_peered":"2024-07-26T13:36:59.455842+0000","last_unstale":"2024-07-26T13:37:17.544501+0000","last_undegraded":"2024-07-26T13:37:17.544501+0000","last_fullsized":"2024-07-26T13:37:17.544501+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:00:19.162239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'3","reported_seq":50,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:36.592353+0000","last_change":"2024-07-26T13:36:56.432818+0000","last_active":"2024-07-26T13:37:36.592353+0000","last_peered":"2024-07-26T13:37:36.592353+0000","last_clean":"2024-07-26T13:37:36.592353+0000","last_became_active":"2024-07-26T13:36:56.432419+0000","last_became_peered":"2024-07-26T13:36:56.432419+0000","last_unstale":"2024-07-26T13:37:36.592353+0000","last_undegraded":"2024-07-26T13:37:36.592353+0000","last_fullsized":"2024-07-26T13:37:36.592353+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:55:51.972470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.544086+0000","last_change":"2024-07-26T13:36:59.460033+0000","last_active":"2024-07-26T13:37:17.544086+0000","last_peered":"2024-07-26T13:37:17.544086+0000","last_clean":"2024-07-26T13:37:17.544086+0000","last_became_active":"2024-07-26T13:36:59.459562+0000","last_became_peered":"2024-07-26T13:36:59.459562+0000","last_unstale":"2024-07-26T13:37:17.544086+0000","last_undegraded":"2024-07-26T13:37:17.544086+0000","last_fullsized":"2024-07-26T13:37:17.544086+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:58:14.155973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551314+0000","last_change":"2024-07-26T13:36:56.432563+0000","last_active":"2024-07-26T13:37:17.551314+0000","last_peered":"2024-07-26T13:37:17.551314+0000","last_clean":"2024-07-26T13:37:17.551314+0000","last_became_active":"2024-07-26T13:36:56.431134+0000","last_became_peered":"2024-07-26T13:36:56.431134+0000","last_unstale":"2024-07-26T13:37:17.551314+0000","last_undegraded":"2024-07-26T13:37:17.551314+0000","last_fullsized":"2024-07-26T13:37:17.551314+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:42:00.653345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.545571+0000","last_change":"2024-07-26T13:36:59.469782+0000","last_active":"2024-07-26T13:37:17.545571+0000","last_peered":"2024-07-26T13:37:17.545571+0000","last_clean":"2024-07-26T13:37:17.545571+0000","last_became_active":"2024-07-26T13:36:59.469412+0000","last_became_peered":"2024-07-26T13:36:59.469412+0000","last_unstale":"2024-07-26T13:37:17.545571+0000","last_undegraded":"2024-07-26T13:37:17.545571+0000","last_fullsized":"2024-07-26T13:37:17.545571+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:28:07.741486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.548023+0000","last_change":"2024-07-26T13:36:56.488981+0000","last_active":"2024-07-26T13:37:17.548023+0000","last_peered":"2024-07-26T13:37:17.548023+0000","last_clean":"2024-07-26T13:37:17.548023+0000","last_became_active":"2024-07-26T13:36:56.488480+0000","last_became_peered":"2024-07-26T13:36:56.488480+0000","last_unstale":"2024-07-26T13:37:17.548023+0000","last_undegraded":"2024-07-26T13:37:17.548023+0000","last_fullsized":"2024-07-26T13:37:17.548023+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:41:10.041007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.543895+0000","last_change":"2024-07-26T13:36:59.473183+0000","last_active":"2024-07-26T13:37:17.543895+0000","last_peered":"2024-07-26T13:37:17.543895+0000","last_clean":"2024-07-26T13:37:17.543895+0000","last_became_active":"2024-07-26T13:36:59.472814+0000","last_became_peered":"2024-07-26T13:36:59.472814+0000","last_unstale":"2024-07-26T13:37:17.543895+0000","last_undegraded":"2024-07-26T13:37:17.543895+0000","last_fullsized":"2024-07-26T13:37:17.543895+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:57:03.396384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'26","reported_seq":101,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:37.602580+0000","last_change":"2024-07-26T13:36:56.435034+0000","last_active":"2024-07-26T13:37:37.602580+0000","last_peered":"2024-07-26T13:37:37.602580+0000","last_clean":"2024-07-26T13:37:37.602580+0000","last_became_active":"2024-07-26T13:36:56.434611+0000","last_became_peered":"2024-07-26T13:36:56.434611+0000","last_unstale":"2024-07-26T13:37:37.602580+0000","last_undegraded":"2024-07-26T13:37:37.602580+0000","last_fullsized":"2024-07-26T13:37:37.602580+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:18:27.272077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56940,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":49,"num_read_kb":79,"num_write":27,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.553386+0000","last_change":"2024-07-26T13:36:59.465831+0000","last_active":"2024-07-26T13:37:17.553386+0000","last_peered":"2024-07-26T13:37:17.553386+0000","last_clean":"2024-07-26T13:37:17.553386+0000","last_became_active":"2024-07-26T13:36:59.465404+0000","last_became_peered":"2024-07-26T13:36:59.465404+0000","last_unstale":"2024-07-26T13:37:17.553386+0000","last_undegraded":"2024-07-26T13:37:17.553386+0000","last_fullsized":"2024-07-26T13:37:17.553386+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:42:53.795926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'6","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.552433+0000","last_change":"2024-07-26T13:36:56.435170+0000","last_active":"2024-07-26T13:37:17.552433+0000","last_peered":"2024-07-26T13:37:17.552433+0000","last_clean":"2024-07-26T13:37:17.552433+0000","last_became_active":"2024-07-26T13:36:56.434715+0000","last_became_peered":"2024-07-26T13:36:56.434715+0000","last_unstale":"2024-07-26T13:37:17.552433+0000","last_undegraded":"2024-07-26T13:37:17.552433+0000","last_fullsized":"2024-07-26T13:37:17.552433+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:05:09.712510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.565171+0000","last_change":"2024-07-26T13:36:59.461926+0000","last_active":"2024-07-26T13:37:17.565171+0000","last_peered":"2024-07-26T13:37:17.565171+0000","last_clean":"2024-07-26T13:37:17.565171+0000","last_became_active":"2024-07-26T13:36:59.461512+0000","last_became_peered":"2024-07-26T13:36:59.461512+0000","last_unstale":"2024-07-26T13:37:17.565171+0000","last_undegraded":"2024-07-26T13:37:17.565171+0000","last_fullsized":"2024-07-26T13:37:17.565171+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:34:24.405663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.546843+0000","last_change":"2024-07-26T13:36:56.430062+0000","last_active":"2024-07-26T13:37:17.546843+0000","last_peered":"2024-07-26T13:37:17.546843+0000","last_clean":"2024-07-26T13:37:17.546843+0000","last_became_active":"2024-07-26T13:36:56.429654+0000","last_became_peered":"2024-07-26T13:36:56.429654+0000","last_unstale":"2024-07-26T13:37:17.546843+0000","last_undegraded":"2024-07-26T13:37:17.546843+0000","last_fullsized":"2024-07-26T13:37:17.546843+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:21:09.514359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550829+0000","last_change":"2024-07-26T13:36:59.474459+0000","last_active":"2024-07-26T13:37:17.550829+0000","last_peered":"2024-07-26T13:37:17.550829+0000","last_clean":"2024-07-26T13:37:17.550829+0000","last_became_active":"2024-07-26T13:36:59.474061+0000","last_became_peered":"2024-07-26T13:36:59.474061+0000","last_unstale":"2024-07-26T13:37:17.550829+0000","last_undegraded":"2024-07-26T13:37:17.550829+0000","last_fullsized":"2024-07-26T13:37:17.550829+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:37:21.691491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'4","reported_seq":55,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:36.592699+0000","last_change":"2024-07-26T13:36:56.432915+0000","last_active":"2024-07-26T13:37:36.592699+0000","last_peered":"2024-07-26T13:37:36.592699+0000","last_clean":"2024-07-26T13:37:36.592699+0000","last_became_active":"2024-07-26T13:36:56.432499+0000","last_became_peered":"2024-07-26T13:36:56.432499+0000","last_unstale":"2024-07-26T13:37:36.592699+0000","last_undegraded":"2024-07-26T13:37:36.592699+0000","last_fullsized":"2024-07-26T13:37:36.592699+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:02:14.306950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550560+0000","last_change":"2024-07-26T13:36:59.468945+0000","last_active":"2024-07-26T13:37:17.550560+0000","last_peered":"2024-07-26T13:37:17.550560+0000","last_clean":"2024-07-26T13:37:17.550560+0000","last_became_active":"2024-07-26T13:36:59.467970+0000","last_became_peered":"2024-07-26T13:36:59.467970+0000","last_unstale":"2024-07-26T13:37:17.550560+0000","last_undegraded":"2024-07-26T13:37:17.550560+0000","last_fullsized":"2024-07-26T13:37:17.550560+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T13:59:19.514945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'15","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551317+0000","last_change":"2024-07-26T13:36:56.487014+0000","last_active":"2024-07-26T13:37:17.551317+0000","last_peered":"2024-07-26T13:37:17.551317+0000","last_clean":"2024-07-26T13:37:17.551317+0000","last_became_active":"2024-07-26T13:36:56.486470+0000","last_became_peered":"2024-07-26T13:36:56.486470+0000","last_unstale":"2024-07-26T13:37:17.551317+0000","last_undegraded":"2024-07-26T13:37:17.551317+0000","last_fullsized":"2024-07-26T13:37:17.551317+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:07:02.205644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.546513+0000","last_change":"2024-07-26T13:36:59.462465+0000","last_active":"2024-07-26T13:37:17.546513+0000","last_peered":"2024-07-26T13:37:17.546513+0000","last_clean":"2024-07-26T13:37:17.546513+0000","last_became_active":"2024-07-26T13:36:59.462024+0000","last_became_peered":"2024-07-26T13:36:59.462024+0000","last_unstale":"2024-07-26T13:37:17.546513+0000","last_undegraded":"2024-07-26T13:37:17.546513+0000","last_fullsized":"2024-07-26T13:37:17.546513+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:34:14.317762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551923+0000","last_change":"2024-07-26T13:36:56.435007+0000","last_active":"2024-07-26T13:37:17.551923+0000","last_peered":"2024-07-26T13:37:17.551923+0000","last_clean":"2024-07-26T13:37:17.551923+0000","last_became_active":"2024-07-26T13:36:56.434444+0000","last_became_peered":"2024-07-26T13:36:56.434444+0000","last_unstale":"2024-07-26T13:37:17.551923+0000","last_undegraded":"2024-07-26T13:37:17.551923+0000","last_fullsized":"2024-07-26T13:37:17.551923+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:27:51.041349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.546679+0000","last_change":"2024-07-26T13:36:59.464164+0000","last_active":"2024-07-26T13:37:17.546679+0000","last_peered":"2024-07-26T13:37:17.546679+0000","last_clean":"2024-07-26T13:37:17.546679+0000","last_became_active":"2024-07-26T13:36:59.463782+0000","last_became_peered":"2024-07-26T13:36:59.463782+0000","last_unstale":"2024-07-26T13:37:17.546679+0000","last_undegraded":"2024-07-26T13:37:17.546679+0000","last_fullsized":"2024-07-26T13:37:17.546679+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:48:06.481731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.565453+0000","last_change":"2024-07-26T13:36:56.429653+0000","last_active":"2024-07-26T13:37:17.565453+0000","last_peered":"2024-07-26T13:37:17.565453+0000","last_clean":"2024-07-26T13:37:17.565453+0000","last_became_active":"2024-07-26T13:36:56.429060+0000","last_became_peered":"2024-07-26T13:36:56.429060+0000","last_unstale":"2024-07-26T13:37:17.565453+0000","last_undegraded":"2024-07-26T13:37:17.565453+0000","last_fullsized":"2024-07-26T13:37:17.565453+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:41:48.261709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551343+0000","last_change":"2024-07-26T13:36:59.471560+0000","last_active":"2024-07-26T13:37:17.551343+0000","last_peered":"2024-07-26T13:37:17.551343+0000","last_clean":"2024-07-26T13:37:17.551343+0000","last_became_active":"2024-07-26T13:36:59.471151+0000","last_became_peered":"2024-07-26T13:36:59.471151+0000","last_unstale":"2024-07-26T13:37:17.551343+0000","last_undegraded":"2024-07-26T13:37:17.551343+0000","last_fullsized":"2024-07-26T13:37:17.551343+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:49:20.513308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.548108+0000","last_change":"2024-07-26T13:36:56.430010+0000","last_active":"2024-07-26T13:37:17.548108+0000","last_peered":"2024-07-26T13:37:17.548108+0000","last_clean":"2024-07-26T13:37:17.548108+0000","last_became_active":"2024-07-26T13:36:56.429120+0000","last_became_peered":"2024-07-26T13:36:56.429120+0000","last_unstale":"2024-07-26T13:37:17.548108+0000","last_undegraded":"2024-07-26T13:37:17.548108+0000","last_fullsized":"2024-07-26T13:37:17.548108+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:23:43.321975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551149+0000","last_change":"2024-07-26T13:36:59.467788+0000","last_active":"2024-07-26T13:37:17.551149+0000","last_peered":"2024-07-26T13:37:17.551149+0000","last_clean":"2024-07-26T13:37:17.551149+0000","last_became_active":"2024-07-26T13:36:59.467399+0000","last_became_peered":"2024-07-26T13:36:59.467399+0000","last_unstale":"2024-07-26T13:37:17.551149+0000","last_undegraded":"2024-07-26T13:37:17.551149+0000","last_fullsized":"2024-07-26T13:37:17.551149+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:48:51.442999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'3","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.553251+0000","last_change":"2024-07-26T13:36:56.588633+0000","last_active":"2024-07-26T13:37:17.553251+0000","last_peered":"2024-07-26T13:37:17.553251+0000","last_clean":"2024-07-26T13:37:17.553251+0000","last_became_active":"2024-07-26T13:36:56.588113+0000","last_became_peered":"2024-07-26T13:36:56.588113+0000","last_unstale":"2024-07-26T13:37:17.553251+0000","last_undegraded":"2024-07-26T13:37:17.553251+0000","last_fullsized":"2024-07-26T13:37:17.553251+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:03:55.075052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551116+0000","last_change":"2024-07-26T13:36:59.472854+0000","last_active":"2024-07-26T13:37:17.551116+0000","last_peered":"2024-07-26T13:37:17.551116+0000","last_clean":"2024-07-26T13:37:17.551116+0000","last_became_active":"2024-07-26T13:36:59.472466+0000","last_became_peered":"2024-07-26T13:36:59.472466+0000","last_unstale":"2024-07-26T13:37:17.551116+0000","last_undegraded":"2024-07-26T13:37:17.551116+0000","last_fullsized":"2024-07-26T13:37:17.551116+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:50:57.454149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":102,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550777+0000","last_change":"2024-07-26T13:35:34.839121+0000","last_active":"2024-07-26T13:37:17.550777+0000","last_peered":"2024-07-26T13:37:17.550777+0000","last_clean":"2024-07-26T13:37:17.550777+0000","last_became_active":"2024-07-26T13:35:34.833465+0000","last_became_peered":"2024-07-26T13:35:34.833465+0000","last_unstale":"2024-07-26T13:37:17.550777+0000","last_undegraded":"2024-07-26T13:37:17.550777+0000","last_fullsized":"2024-07-26T13:37:17.550777+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:32:23.285428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:32:23.285428+0000","last_clean_scrub_stamp":"2024-07-26T13:32:23.285428+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:29:31.650487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'3","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.561400+0000","last_change":"2024-07-26T13:36:56.428648+0000","last_active":"2024-07-26T13:37:17.561400+0000","last_peered":"2024-07-26T13:37:17.561400+0000","last_clean":"2024-07-26T13:37:17.561400+0000","last_became_active":"2024-07-26T13:36:56.427665+0000","last_became_peered":"2024-07-26T13:36:56.427665+0000","last_unstale":"2024-07-26T13:37:17.561400+0000","last_undegraded":"2024-07-26T13:37:17.561400+0000","last_fullsized":"2024-07-26T13:37:17.561400+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:41:39.371687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.543335+0000","last_change":"2024-07-26T13:36:59.460625+0000","last_active":"2024-07-26T13:37:17.543335+0000","last_peered":"2024-07-26T13:37:17.543335+0000","last_clean":"2024-07-26T13:37:17.543335+0000","last_became_active":"2024-07-26T13:36:59.458658+0000","last_became_peered":"2024-07-26T13:36:59.458658+0000","last_unstale":"2024-07-26T13:37:17.543335+0000","last_undegraded":"2024-07-26T13:37:17.543335+0000","last_fullsized":"2024-07-26T13:37:17.543335+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:46:42.812402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550730+0000","last_change":"2024-07-26T13:36:56.432617+0000","last_active":"2024-07-26T13:37:17.550730+0000","last_peered":"2024-07-26T13:37:17.550730+0000","last_clean":"2024-07-26T13:37:17.550730+0000","last_became_active":"2024-07-26T13:36:56.431519+0000","last_became_peered":"2024-07-26T13:36:56.431519+0000","last_unstale":"2024-07-26T13:37:17.550730+0000","last_undegraded":"2024-07-26T13:37:17.550730+0000","last_fullsized":"2024-07-26T13:37:17.550730+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:32:21.120076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.565072+0000","last_change":"2024-07-26T13:36:59.464084+0000","last_active":"2024-07-26T13:37:17.565072+0000","last_peered":"2024-07-26T13:37:17.565072+0000","last_clean":"2024-07-26T13:37:17.565072+0000","last_became_active":"2024-07-26T13:36:59.463755+0000","last_became_peered":"2024-07-26T13:36:59.463755+0000","last_unstale":"2024-07-26T13:37:17.565072+0000","last_undegraded":"2024-07-26T13:37:17.565072+0000","last_fullsized":"2024-07-26T13:37:17.565072+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:53:20.034188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'4","reported_seq":57,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:40.597751+0000","last_change":"2024-07-26T13:36:56.486071+0000","last_active":"2024-07-26T13:37:40.597751+0000","last_peered":"2024-07-26T13:37:40.597751+0000","last_clean":"2024-07-26T13:37:40.597751+0000","last_became_active":"2024-07-26T13:36:56.485672+0000","last_became_peered":"2024-07-26T13:36:56.485672+0000","last_unstale":"2024-07-26T13:37:40.597751+0000","last_undegraded":"2024-07-26T13:37:40.597751+0000","last_fullsized":"2024-07-26T13:37:40.597751+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T21:29:45.666651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.566653+0000","last_change":"2024-07-26T13:37:17.566555+0000","last_active":"2024-07-26T13:37:17.566653+0000","last_peered":"2024-07-26T13:37:17.566653+0000","last_clean":"2024-07-26T13:37:17.566653+0000","last_became_active":"2024-07-26T13:37:17.566040+0000","last_became_peered":"2024-07-26T13:37:17.566040+0000","last_unstale":"2024-07-26T13:37:17.566653+0000","last_undegraded":"2024-07-26T13:37:17.566653+0000","last_fullsized":"2024-07-26T13:37:17.566653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:48:33.078890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'1","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550922+0000","last_change":"2024-07-26T13:36:56.431811+0000","last_active":"2024-07-26T13:37:17.550922+0000","last_peered":"2024-07-26T13:37:17.550922+0000","last_clean":"2024-07-26T13:37:17.550922+0000","last_became_active":"2024-07-26T13:36:56.431223+0000","last_became_peered":"2024-07-26T13:36:56.431223+0000","last_unstale":"2024-07-26T13:37:17.550922+0000","last_undegraded":"2024-07-26T13:37:17.550922+0000","last_fullsized":"2024-07-26T13:37:17.550922+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:08:00.334515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.547257+0000","last_change":"2024-07-26T13:36:59.463972+0000","last_active":"2024-07-26T13:37:17.547257+0000","last_peered":"2024-07-26T13:37:17.547257+0000","last_clean":"2024-07-26T13:37:17.547257+0000","last_became_active":"2024-07-26T13:36:59.463496+0000","last_became_peered":"2024-07-26T13:36:59.463496+0000","last_unstale":"2024-07-26T13:37:17.547257+0000","last_undegraded":"2024-07-26T13:37:17.547257+0000","last_fullsized":"2024-07-26T13:37:17.547257+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:02:06.215372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.545709+0000","last_change":"2024-07-26T13:36:56.486681+0000","last_active":"2024-07-26T13:37:17.545709+0000","last_peered":"2024-07-26T13:37:17.545709+0000","last_clean":"2024-07-26T13:37:17.545709+0000","last_became_active":"2024-07-26T13:36:56.486328+0000","last_became_peered":"2024-07-26T13:36:56.486328+0000","last_unstale":"2024-07-26T13:37:17.545709+0000","last_undegraded":"2024-07-26T13:37:17.545709+0000","last_fullsized":"2024-07-26T13:37:17.545709+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:10:36.557707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.547494+0000","last_change":"2024-07-26T13:36:59.451415+0000","last_active":"2024-07-26T13:37:17.547494+0000","last_peered":"2024-07-26T13:37:17.547494+0000","last_clean":"2024-07-26T13:37:17.547494+0000","last_became_active":"2024-07-26T13:36:59.450992+0000","last_became_peered":"2024-07-26T13:36:59.450992+0000","last_unstale":"2024-07-26T13:37:17.547494+0000","last_undegraded":"2024-07-26T13:37:17.547494+0000","last_fullsized":"2024-07-26T13:37:17.547494+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:29:11.303916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.555734+0000","last_change":"2024-07-26T13:37:17.555643+0000","last_active":"2024-07-26T13:37:17.555734+0000","last_peered":"2024-07-26T13:37:17.555734+0000","last_clean":"2024-07-26T13:37:17.555734+0000","last_became_active":"2024-07-26T13:37:17.549756+0000","last_became_peered":"2024-07-26T13:37:17.549756+0000","last_unstale":"2024-07-26T13:37:17.555734+0000","last_undegraded":"2024-07-26T13:37:17.555734+0000","last_fullsized":"2024-07-26T13:37:17.555734+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:20:16.911286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.565386+0000","last_change":"2024-07-26T13:36:59.465538+0000","last_active":"2024-07-26T13:37:17.565386+0000","last_peered":"2024-07-26T13:37:17.565386+0000","last_clean":"2024-07-26T13:37:17.565386+0000","last_became_active":"2024-07-26T13:36:59.464886+0000","last_became_peered":"2024-07-26T13:36:59.464886+0000","last_unstale":"2024-07-26T13:37:17.565386+0000","last_undegraded":"2024-07-26T13:37:17.565386+0000","last_fullsized":"2024-07-26T13:37:17.565386+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:19:12.273116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'10","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:35.479411+0000","last_change":"2024-07-26T13:36:56.428355+0000","last_active":"2024-07-26T13:37:35.479411+0000","last_peered":"2024-07-26T13:37:35.479411+0000","last_clean":"2024-07-26T13:37:35.479411+0000","last_became_active":"2024-07-26T13:36:56.427213+0000","last_became_peered":"2024-07-26T13:36:56.427213+0000","last_unstale":"2024-07-26T13:37:35.479411+0000","last_undegraded":"2024-07-26T13:37:35.479411+0000","last_fullsized":"2024-07-26T13:37:35.479411+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:48:09.730067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.543752+0000","last_change":"2024-07-26T13:36:59.459378+0000","last_active":"2024-07-26T13:37:17.543752+0000","last_peered":"2024-07-26T13:37:17.543752+0000","last_clean":"2024-07-26T13:37:17.543752+0000","last_became_active":"2024-07-26T13:36:59.458799+0000","last_became_peered":"2024-07-26T13:36:59.458799+0000","last_unstale":"2024-07-26T13:37:17.543752+0000","last_undegraded":"2024-07-26T13:37:17.543752+0000","last_fullsized":"2024-07-26T13:37:17.543752+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:45:19.587800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.561367+0000","last_change":"2024-07-26T13:36:56.588819+0000","last_active":"2024-07-26T13:37:17.561367+0000","last_peered":"2024-07-26T13:37:17.561367+0000","last_clean":"2024-07-26T13:37:17.561367+0000","last_became_active":"2024-07-26T13:36:56.588430+0000","last_became_peered":"2024-07-26T13:36:56.588430+0000","last_unstale":"2024-07-26T13:37:17.561367+0000","last_undegraded":"2024-07-26T13:37:17.561367+0000","last_fullsized":"2024-07-26T13:37:17.561367+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:07:21.387618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.562120+0000","last_change":"2024-07-26T13:37:17.562040+0000","last_active":"2024-07-26T13:37:17.562120+0000","last_peered":"2024-07-26T13:37:17.562120+0000","last_clean":"2024-07-26T13:37:17.562120+0000","last_became_active":"2024-07-26T13:37:17.561542+0000","last_became_peered":"2024-07-26T13:37:17.561542+0000","last_unstale":"2024-07-26T13:37:17.562120+0000","last_undegraded":"2024-07-26T13:37:17.562120+0000","last_fullsized":"2024-07-26T13:37:17.562120+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:21:08.161954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.544058+0000","last_change":"2024-07-26T13:36:56.489106+0000","last_active":"2024-07-26T13:37:17.544058+0000","last_peered":"2024-07-26T13:37:17.544058+0000","last_clean":"2024-07-26T13:37:17.544058+0000","last_became_active":"2024-07-26T13:36:56.488706+0000","last_became_peered":"2024-07-26T13:36:56.488706+0000","last_unstale":"2024-07-26T13:37:17.544058+0000","last_undegraded":"2024-07-26T13:37:17.544058+0000","last_fullsized":"2024-07-26T13:37:17.544058+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:00:59.204434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551358+0000","last_change":"2024-07-26T13:36:59.470412+0000","last_active":"2024-07-26T13:37:17.551358+0000","last_peered":"2024-07-26T13:37:17.551358+0000","last_clean":"2024-07-26T13:37:17.551358+0000","last_became_active":"2024-07-26T13:36:59.470010+0000","last_became_peered":"2024-07-26T13:36:59.470010+0000","last_unstale":"2024-07-26T13:37:17.551358+0000","last_undegraded":"2024-07-26T13:37:17.551358+0000","last_fullsized":"2024-07-26T13:37:17.551358+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:50:57.987600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.543473+0000","last_change":"2024-07-26T13:36:56.428671+0000","last_active":"2024-07-26T13:37:17.543473+0000","last_peered":"2024-07-26T13:37:17.543473+0000","last_clean":"2024-07-26T13:37:17.543473+0000","last_became_active":"2024-07-26T13:36:56.428301+0000","last_became_peered":"2024-07-26T13:36:56.428301+0000","last_unstale":"2024-07-26T13:37:17.543473+0000","last_undegraded":"2024-07-26T13:37:17.543473+0000","last_fullsized":"2024-07-26T13:37:17.543473+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:59:19.593664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.546377+0000","last_change":"2024-07-26T13:36:59.458627+0000","last_active":"2024-07-26T13:37:17.546377+0000","last_peered":"2024-07-26T13:37:17.546377+0000","last_clean":"2024-07-26T13:37:17.546377+0000","last_became_active":"2024-07-26T13:36:59.458199+0000","last_became_peered":"2024-07-26T13:36:59.458199+0000","last_unstale":"2024-07-26T13:37:17.546377+0000","last_undegraded":"2024-07-26T13:37:17.546377+0000","last_fullsized":"2024-07-26T13:37:17.546377+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:54:41.833223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'1","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551478+0000","last_change":"2024-07-26T13:36:56.431548+0000","last_active":"2024-07-26T13:37:17.551478+0000","last_peered":"2024-07-26T13:37:17.551478+0000","last_clean":"2024-07-26T13:37:17.551478+0000","last_became_active":"2024-07-26T13:36:56.430989+0000","last_became_peered":"2024-07-26T13:36:56.430989+0000","last_unstale":"2024-07-26T13:37:17.551478+0000","last_undegraded":"2024-07-26T13:37:17.551478+0000","last_fullsized":"2024-07-26T13:37:17.551478+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:46:51.655324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.543108+0000","last_change":"2024-07-26T13:36:59.458290+0000","last_active":"2024-07-26T13:37:17.543108+0000","last_peered":"2024-07-26T13:37:17.543108+0000","last_clean":"2024-07-26T13:37:17.543108+0000","last_became_active":"2024-07-26T13:36:59.457942+0000","last_became_peered":"2024-07-26T13:36:59.457942+0000","last_unstale":"2024-07-26T13:37:17.543108+0000","last_undegraded":"2024-07-26T13:37:17.543108+0000","last_fullsized":"2024-07-26T13:37:17.543108+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:51:58.797441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.567687+0000","last_change":"2024-07-26T13:37:17.567624+0000","last_active":"2024-07-26T13:37:17.567687+0000","last_peered":"2024-07-26T13:37:17.567687+0000","last_clean":"2024-07-26T13:37:17.567687+0000","last_became_active":"2024-07-26T13:37:17.562455+0000","last_became_peered":"2024-07-26T13:37:17.562455+0000","last_unstale":"2024-07-26T13:37:17.567687+0000","last_undegraded":"2024-07-26T13:37:17.567687+0000","last_fullsized":"2024-07-26T13:37:17.567687+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:50:36.418750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550738+0000","last_change":"2024-07-26T13:36:59.474034+0000","last_active":"2024-07-26T13:37:17.550738+0000","last_peered":"2024-07-26T13:37:17.550738+0000","last_clean":"2024-07-26T13:37:17.550738+0000","last_became_active":"2024-07-26T13:36:59.472358+0000","last_became_peered":"2024-07-26T13:36:59.472358+0000","last_unstale":"2024-07-26T13:37:17.550738+0000","last_undegraded":"2024-07-26T13:37:17.550738+0000","last_fullsized":"2024-07-26T13:37:17.550738+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:36:44.565413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.547175+0000","last_change":"2024-07-26T13:36:59.465546+0000","last_active":"2024-07-26T13:37:17.547175+0000","last_peered":"2024-07-26T13:37:17.547175+0000","last_clean":"2024-07-26T13:37:17.547175+0000","last_became_active":"2024-07-26T13:36:59.465137+0000","last_became_peered":"2024-07-26T13:36:59.465137+0000","last_unstale":"2024-07-26T13:37:17.547175+0000","last_undegraded":"2024-07-26T13:37:17.547175+0000","last_fullsized":"2024-07-26T13:37:17.547175+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:35:13.591354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.562100+0000","last_change":"2024-07-26T13:37:17.562012+0000","last_active":"2024-07-26T13:37:17.562100+0000","last_peered":"2024-07-26T13:37:17.562100+0000","last_clean":"2024-07-26T13:37:17.562100+0000","last_became_active":"2024-07-26T13:37:17.550218+0000","last_became_peered":"2024-07-26T13:37:17.550218+0000","last_unstale":"2024-07-26T13:37:17.562100+0000","last_undegraded":"2024-07-26T13:37:17.562100+0000","last_fullsized":"2024-07-26T13:37:17.562100+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:46:16.559533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.543841+0000","last_change":"2024-07-26T13:36:59.456896+0000","last_active":"2024-07-26T13:37:17.543841+0000","last_peered":"2024-07-26T13:37:17.543841+0000","last_clean":"2024-07-26T13:37:17.543841+0000","last_became_active":"2024-07-26T13:36:59.456530+0000","last_became_peered":"2024-07-26T13:36:59.456530+0000","last_unstale":"2024-07-26T13:37:17.543841+0000","last_undegraded":"2024-07-26T13:37:17.543841+0000","last_fullsized":"2024-07-26T13:37:17.543841+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:59:56.512501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'4","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.553522+0000","last_change":"2024-07-26T13:36:56.433119+0000","last_active":"2024-07-26T13:37:17.553522+0000","last_peered":"2024-07-26T13:37:17.553522+0000","last_clean":"2024-07-26T13:37:17.553522+0000","last_became_active":"2024-07-26T13:36:56.432765+0000","last_became_peered":"2024-07-26T13:36:56.432765+0000","last_unstale":"2024-07-26T13:37:17.553522+0000","last_undegraded":"2024-07-26T13:37:17.553522+0000","last_fullsized":"2024-07-26T13:37:17.553522+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:31:29.197106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.545548+0000","last_change":"2024-07-26T13:36:59.462125+0000","last_active":"2024-07-26T13:37:17.545548+0000","last_peered":"2024-07-26T13:37:17.545548+0000","last_clean":"2024-07-26T13:37:17.545548+0000","last_became_active":"2024-07-26T13:36:59.461796+0000","last_became_peered":"2024-07-26T13:36:59.461796+0000","last_unstale":"2024-07-26T13:37:17.545548+0000","last_undegraded":"2024-07-26T13:37:17.545548+0000","last_fullsized":"2024-07-26T13:37:17.545548+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:44:10.329046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.543605+0000","last_change":"2024-07-26T13:36:56.487251+0000","last_active":"2024-07-26T13:37:17.543605+0000","last_peered":"2024-07-26T13:37:17.543605+0000","last_clean":"2024-07-26T13:37:17.543605+0000","last_became_active":"2024-07-26T13:36:56.486918+0000","last_became_peered":"2024-07-26T13:36:56.486918+0000","last_unstale":"2024-07-26T13:37:17.543605+0000","last_undegraded":"2024-07-26T13:37:17.543605+0000","last_fullsized":"2024-07-26T13:37:17.543605+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:41:31.248895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.543268+0000","last_change":"2024-07-26T13:36:59.470810+0000","last_active":"2024-07-26T13:37:17.543268+0000","last_peered":"2024-07-26T13:37:17.543268+0000","last_clean":"2024-07-26T13:37:17.543268+0000","last_became_active":"2024-07-26T13:36:59.467371+0000","last_became_peered":"2024-07-26T13:36:59.467371+0000","last_unstale":"2024-07-26T13:37:17.543268+0000","last_undegraded":"2024-07-26T13:37:17.543268+0000","last_fullsized":"2024-07-26T13:37:17.543268+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:23:48.572933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'9","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.545993+0000","last_change":"2024-07-26T13:36:56.433598+0000","last_active":"2024-07-26T13:37:17.545993+0000","last_peered":"2024-07-26T13:37:17.545993+0000","last_clean":"2024-07-26T13:37:17.545993+0000","last_became_active":"2024-07-26T13:36:56.433325+0000","last_became_peered":"2024-07-26T13:36:56.433325+0000","last_unstale":"2024-07-26T13:37:17.545993+0000","last_undegraded":"2024-07-26T13:37:17.545993+0000","last_fullsized":"2024-07-26T13:37:17.545993+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:17:33.651765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61332,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":163,"num_read_kb":146,"num_write":160,"num_write_kb":105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":644,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":189391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247480,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678039,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":443451,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874268,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":1092241,"data_compressed":41637,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037793,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":897024,"data_stored":443665,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234021,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":443563,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430250,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":1035906,"data_compressed":41637,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593776,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":443054,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790005,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":1035199,"data_compressed":41637,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953529,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":888832,"data_stored":443882,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149757,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":443365,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378690,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":500515,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542215,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":950272,"data_stored":500800,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705739,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":905216,"data_stored":444325,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34195,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34195,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":57668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-26T13:37:43.513 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph pg dump --format=json 2024-07-26T13:37:43.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:43 smithi100 ceph-mon[32599]: pgmap v292: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2024-07-26T13:37:43.767 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:37:43.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:43 smithi167 ceph-mon[37140]: pgmap v292: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2024-07-26T13:37:43.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:43 smithi138 ceph-mon[36954]: pgmap v292: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2024-07-26T13:37:44.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:44 smithi100 ceph-mon[32599]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-26T13:37:44.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:44 smithi167 ceph-mon[37140]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-26T13:37:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:44 smithi138 ceph-mon[36954]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-26T13:37:45.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.541+0000 7f418b103640 1 -- 172.21.15.100:0/38798038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4184102490 msgr2=0x7f41841028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:45.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.541+0000 7f418b103640 1 --2- 172.21.15.100:0/38798038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4184102490 0x7f41841028a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4178009900 tx=0x7f417802a410 comp rx=0 tx=0).stop 2024-07-26T13:37:45.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.541+0000 7f418b103640 1 -- 172.21.15.100:0/38798038 shutdown_connections 2024-07-26T13:37:45.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.541+0000 7f418b103640 1 --2- 172.21.15.100:0/38798038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4184108720 0x7f418410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:45.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.541+0000 7f418b103640 1 --2- 172.21.15.100:0/38798038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4184107dd0 0x7f41841081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:45.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.541+0000 7f418b103640 1 --2- 172.21.15.100:0/38798038 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4184102490 0x7f41841028a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:45.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.541+0000 7f418b103640 1 -- 172.21.15.100:0/38798038 >> 172.21.15.100:0/38798038 conn(0x7f4184079160 msgr2=0x7f4184079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:45.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.542+0000 7f418b103640 1 -- 172.21.15.100:0/38798038 shutdown_connections 2024-07-26T13:37:45.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.542+0000 7f418b103640 1 -- 172.21.15.100:0/38798038 wait complete. 2024-07-26T13:37:45.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.542+0000 7f418b103640 1 Processor -- start 2024-07-26T13:37:45.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.542+0000 7f418b103640 1 -- start start 2024-07-26T13:37:45.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.543+0000 7f418b103640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4184102490 0x7f4184198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:45.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.543+0000 7f418b103640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4184107dd0 0x7f4184199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:45.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.543+0000 7f418b103640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4184108720 0x7f418419e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:45.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.543+0000 7f418b103640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f41841123a0 con 0x7f4184108720 2024-07-26T13:37:45.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.543+0000 7f418b103640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4184112220 con 0x7f4184102490 2024-07-26T13:37:45.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.543+0000 7f418b103640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4184112520 con 0x7f4184107dd0 2024-07-26T13:37:45.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.543+0000 7f41896a0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4184108720 0x7f418419e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:45.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.543+0000 7f41896a0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4184108720 0x7f418419e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55028/0 (socket says 172.21.15.100:55028) 2024-07-26T13:37:45.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.543+0000 7f41896a0640 1 -- 172.21.15.100:0/387955721 learned_addr learned my addr 172.21.15.100:0/387955721 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:37:45.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.543+0000 7f4188e9f640 1 --2- 172.21.15.100:0/387955721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4184102490 0x7f4184198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:45.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.543+0000 7f4183fff640 1 --2- 172.21.15.100:0/387955721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4184107dd0 0x7f4184199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:45.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.543+0000 7f41896a0640 1 -- 172.21.15.100:0/387955721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4184107dd0 msgr2=0x7f4184199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:45.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.543+0000 7f41896a0640 1 --2- 172.21.15.100:0/387955721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4184107dd0 0x7f4184199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:45.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.543+0000 7f41896a0640 1 -- 172.21.15.100:0/387955721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4184102490 msgr2=0x7f4184198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:45.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.543+0000 7f41896a0640 1 --2- 172.21.15.100:0/387955721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4184102490 0x7f4184198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:45.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.543+0000 7f41896a0640 1 -- 172.21.15.100:0/387955721 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f418419ed20 con 0x7f4184108720 2024-07-26T13:37:45.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.543+0000 7f4188e9f640 1 --2- 172.21.15.100:0/387955721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4184102490 0x7f4184198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:37:45.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.543+0000 7f41896a0640 1 --2- 172.21.15.100:0/387955721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4184108720 0x7f418419e5f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f417c00eaa0 tx=0x7f417c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:45.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.544+0000 7f4181ffb640 1 -- 172.21.15.100:0/387955721 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417c00cdb0 con 0x7f4184108720 2024-07-26T13:37:45.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.544+0000 7f418b103640 1 -- 172.21.15.100:0/387955721 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f418419f010 con 0x7f4184108720 2024-07-26T13:37:45.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.544+0000 7f4181ffb640 1 -- 172.21.15.100:0/387955721 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f417c005020 con 0x7f4184108720 2024-07-26T13:37:45.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.544+0000 7f4181ffb640 1 -- 172.21.15.100:0/387955721 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417c0116d0 con 0x7f4184108720 2024-07-26T13:37:45.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.544+0000 7f418b103640 1 -- 172.21.15.100:0/387955721 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41841a5770 con 0x7f4184108720 2024-07-26T13:37:45.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.545+0000 7f418b103640 1 -- 172.21.15.100:0/387955721 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4184103710 con 0x7f4184108720 2024-07-26T13:37:45.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.546+0000 7f4181ffb640 1 -- 172.21.15.100:0/387955721 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f417c0054a0 con 0x7f4184108720 2024-07-26T13:37:45.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.547+0000 7f4181ffb640 1 --2- 172.21.15.100:0/387955721 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f41740b3cb0 0x7f41740b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:45.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.547+0000 7f4188e9f640 1 --2- 172.21.15.100:0/387955721 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f41740b3cb0 0x7f41740b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:45.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.547+0000 7f4181ffb640 1 -- 172.21.15.100:0/387955721 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f417c0e4580 con 0x7f4184108720 2024-07-26T13:37:45.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.547+0000 7f4188e9f640 1 --2- 172.21.15.100:0/387955721 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f41740b3cb0 0x7f41740b61c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4178031440 tx=0x7f4178009680 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:45.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.550+0000 7f4181ffb640 1 -- 172.21.15.100:0/387955721 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f417c016030 con 0x7f4184108720 2024-07-26T13:37:45.565 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:45 smithi100 ceph-mon[32599]: pgmap v293: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.1 KiB/s wr, 9 op/s 2024-07-26T13:37:45.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.721+0000 7f418b103640 1 -- 172.21.15.100:0/387955721 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4184063230 con 0x7f41740b3cb0 2024-07-26T13:37:45.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.726+0000 7f4181ffb640 1 -- 172.21.15.100:0/387955721 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258944 (secure 0 0 0) 0x7f4184063230 con 0x7f41740b3cb0 2024-07-26T13:37:45.729 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:37:45.730 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-07-26T13:37:45.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.730+0000 7f418b103640 1 -- 172.21.15.100:0/387955721 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f41740b3cb0 msgr2=0x7f41740b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:45.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.730+0000 7f418b103640 1 --2- 172.21.15.100:0/387955721 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f41740b3cb0 0x7f41740b61c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4178031440 tx=0x7f4178009680 comp rx=0 tx=0).stop 2024-07-26T13:37:45.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.730+0000 7f418b103640 1 -- 172.21.15.100:0/387955721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4184108720 msgr2=0x7f418419e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:45.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.730+0000 7f418b103640 1 --2- 172.21.15.100:0/387955721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4184108720 0x7f418419e5f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f417c00eaa0 tx=0x7f417c00ef60 comp rx=0 tx=0).stop 2024-07-26T13:37:45.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.730+0000 7f418b103640 1 -- 172.21.15.100:0/387955721 shutdown_connections 2024-07-26T13:37:45.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.730+0000 7f418b103640 1 --2- 172.21.15.100:0/387955721 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f41740b3cb0 0x7f41740b61c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:45.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.730+0000 7f418b103640 1 --2- 172.21.15.100:0/387955721 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4184108720 0x7f418419e5f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:45.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.730+0000 7f418b103640 1 --2- 172.21.15.100:0/387955721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4184107dd0 0x7f4184199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:45.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.730+0000 7f418b103640 1 --2- 172.21.15.100:0/387955721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4184102490 0x7f4184198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:45.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.730+0000 7f418b103640 1 -- 172.21.15.100:0/387955721 >> 172.21.15.100:0/387955721 conn(0x7f4184079160 msgr2=0x7f4184100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:45.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.730+0000 7f418b103640 1 -- 172.21.15.100:0/387955721 shutdown_connections 2024-07-26T13:37:45.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:45.730+0000 7f418b103640 1 -- 172.21.15.100:0/387955721 wait complete. 2024-07-26T13:37:45.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:45 smithi167 ceph-mon[37140]: pgmap v293: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.1 KiB/s wr, 9 op/s 2024-07-26T13:37:45.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:45 smithi138 ceph-mon[36954]: pgmap v293: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.1 KiB/s wr, 9 op/s 2024-07-26T13:37:46.345 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-07-26T13:37:44.369614+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657329,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":265,"num_read_kb":231,"num_write":290,"num_write_kb":2265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":2952484,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":234,"ondisk_log_size":234,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368340,"kb_used_data":10852,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573484,"statfs":{"total":1151940427776,"available":1151563247616,"internally_reserved":0,"allocated":11112448,"data_stored":7280756,"data_compressed":177282,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":40122,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":84,"num_read_kb":81,"num_write":27,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003458"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.547178+0000","last_change":"2024-07-26T13:36:59.462289+0000","last_active":"2024-07-26T13:37:17.547178+0000","last_peered":"2024-07-26T13:37:17.547178+0000","last_clean":"2024-07-26T13:37:17.547178+0000","last_became_active":"2024-07-26T13:36:59.460311+0000","last_became_peered":"2024-07-26T13:36:59.460311+0000","last_unstale":"2024-07-26T13:37:17.547178+0000","last_undegraded":"2024-07-26T13:37:17.547178+0000","last_fullsized":"2024-07-26T13:37:17.547178+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:04:59.123716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.545919+0000","last_change":"2024-07-26T13:36:59.463218+0000","last_active":"2024-07-26T13:37:17.545919+0000","last_peered":"2024-07-26T13:37:17.545919+0000","last_clean":"2024-07-26T13:37:17.545919+0000","last_became_active":"2024-07-26T13:36:59.462889+0000","last_became_peered":"2024-07-26T13:36:59.462889+0000","last_unstale":"2024-07-26T13:37:17.545919+0000","last_undegraded":"2024-07-26T13:37:17.545919+0000","last_fullsized":"2024-07-26T13:37:17.545919+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:30:46.788669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.545077+0000","last_change":"2024-07-26T13:36:59.456113+0000","last_active":"2024-07-26T13:37:17.545077+0000","last_peered":"2024-07-26T13:37:17.545077+0000","last_clean":"2024-07-26T13:37:17.545077+0000","last_became_active":"2024-07-26T13:36:59.455622+0000","last_became_peered":"2024-07-26T13:36:59.455622+0000","last_unstale":"2024-07-26T13:37:17.545077+0000","last_undegraded":"2024-07-26T13:37:17.545077+0000","last_fullsized":"2024-07-26T13:37:17.545077+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:36:26.474799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.556669+0000","last_change":"2024-07-26T13:37:17.555947+0000","last_active":"2024-07-26T13:37:17.556669+0000","last_peered":"2024-07-26T13:37:17.556669+0000","last_clean":"2024-07-26T13:37:17.556669+0000","last_became_active":"2024-07-26T13:37:17.555607+0000","last_became_peered":"2024-07-26T13:37:17.555607+0000","last_unstale":"2024-07-26T13:37:17.556669+0000","last_undegraded":"2024-07-26T13:37:17.556669+0000","last_fullsized":"2024-07-26T13:37:17.556669+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:19:59.603225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551495+0000","last_change":"2024-07-26T13:36:59.474896+0000","last_active":"2024-07-26T13:37:17.551495+0000","last_peered":"2024-07-26T13:37:17.551495+0000","last_clean":"2024-07-26T13:37:17.551495+0000","last_became_active":"2024-07-26T13:36:59.474516+0000","last_became_peered":"2024-07-26T13:36:59.474516+0000","last_unstale":"2024-07-26T13:37:17.551495+0000","last_undegraded":"2024-07-26T13:37:17.551495+0000","last_fullsized":"2024-07-26T13:37:17.551495+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:38:18.994280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.556329+0000","last_change":"2024-07-26T13:37:17.556191+0000","last_active":"2024-07-26T13:37:17.556329+0000","last_peered":"2024-07-26T13:37:17.556329+0000","last_clean":"2024-07-26T13:37:17.556329+0000","last_became_active":"2024-07-26T13:37:17.555715+0000","last_became_peered":"2024-07-26T13:37:17.555715+0000","last_unstale":"2024-07-26T13:37:17.556329+0000","last_undegraded":"2024-07-26T13:37:17.556329+0000","last_fullsized":"2024-07-26T13:37:17.556329+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:03:18.352049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550568+0000","last_change":"2024-07-26T13:36:59.469515+0000","last_active":"2024-07-26T13:37:17.550568+0000","last_peered":"2024-07-26T13:37:17.550568+0000","last_clean":"2024-07-26T13:37:17.550568+0000","last_became_active":"2024-07-26T13:36:59.469106+0000","last_became_peered":"2024-07-26T13:36:59.469106+0000","last_unstale":"2024-07-26T13:37:17.550568+0000","last_undegraded":"2024-07-26T13:37:17.550568+0000","last_fullsized":"2024-07-26T13:37:17.550568+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:10:05.455329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.549794+0000","last_change":"2024-07-26T13:37:17.549365+0000","last_active":"2024-07-26T13:37:17.549794+0000","last_peered":"2024-07-26T13:37:17.549794+0000","last_clean":"2024-07-26T13:37:17.549794+0000","last_became_active":"2024-07-26T13:37:17.549036+0000","last_became_peered":"2024-07-26T13:37:17.549036+0000","last_unstale":"2024-07-26T13:37:17.549794+0000","last_undegraded":"2024-07-26T13:37:17.549794+0000","last_fullsized":"2024-07-26T13:37:17.549794+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:27:43.930981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.544175+0000","last_change":"2024-07-26T13:36:59.459306+0000","last_active":"2024-07-26T13:37:17.544175+0000","last_peered":"2024-07-26T13:37:17.544175+0000","last_clean":"2024-07-26T13:37:17.544175+0000","last_became_active":"2024-07-26T13:36:59.458714+0000","last_became_peered":"2024-07-26T13:36:59.458714+0000","last_unstale":"2024-07-26T13:37:17.544175+0000","last_undegraded":"2024-07-26T13:37:17.544175+0000","last_fullsized":"2024-07-26T13:37:17.544175+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:34:18.512454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.547670+0000","last_change":"2024-07-26T13:36:59.466556+0000","last_active":"2024-07-26T13:37:17.547670+0000","last_peered":"2024-07-26T13:37:17.547670+0000","last_clean":"2024-07-26T13:37:17.547670+0000","last_became_active":"2024-07-26T13:36:59.466147+0000","last_became_peered":"2024-07-26T13:36:59.466147+0000","last_unstale":"2024-07-26T13:37:17.547670+0000","last_undegraded":"2024-07-26T13:37:17.547670+0000","last_fullsized":"2024-07-26T13:37:17.547670+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:43:11.359314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551125+0000","last_change":"2024-07-26T13:36:59.473685+0000","last_active":"2024-07-26T13:37:17.551125+0000","last_peered":"2024-07-26T13:37:17.551125+0000","last_clean":"2024-07-26T13:37:17.551125+0000","last_became_active":"2024-07-26T13:36:59.473314+0000","last_became_peered":"2024-07-26T13:36:59.473314+0000","last_unstale":"2024-07-26T13:37:17.551125+0000","last_undegraded":"2024-07-26T13:37:17.551125+0000","last_fullsized":"2024-07-26T13:37:17.551125+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:41:51.612348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.561132+0000","last_change":"2024-07-26T13:36:59.471586+0000","last_active":"2024-07-26T13:37:17.561132+0000","last_peered":"2024-07-26T13:37:17.561132+0000","last_clean":"2024-07-26T13:37:17.561132+0000","last_became_active":"2024-07-26T13:36:59.470995+0000","last_became_peered":"2024-07-26T13:36:59.470995+0000","last_unstale":"2024-07-26T13:37:17.561132+0000","last_undegraded":"2024-07-26T13:37:17.561132+0000","last_fullsized":"2024-07-26T13:37:17.561132+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:54:07.957937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.565391+0000","last_change":"2024-07-26T13:36:59.460018+0000","last_active":"2024-07-26T13:37:17.565391+0000","last_peered":"2024-07-26T13:37:17.565391+0000","last_clean":"2024-07-26T13:37:17.565391+0000","last_became_active":"2024-07-26T13:36:59.459671+0000","last_became_peered":"2024-07-26T13:36:59.459671+0000","last_unstale":"2024-07-26T13:37:17.565391+0000","last_undegraded":"2024-07-26T13:37:17.565391+0000","last_fullsized":"2024-07-26T13:37:17.565391+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:19:33.923360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.544927+0000","last_change":"2024-07-26T13:36:59.470271+0000","last_active":"2024-07-26T13:37:17.544927+0000","last_peered":"2024-07-26T13:37:17.544927+0000","last_clean":"2024-07-26T13:37:17.544927+0000","last_became_active":"2024-07-26T13:36:59.469941+0000","last_became_peered":"2024-07-26T13:36:59.469941+0000","last_unstale":"2024-07-26T13:37:17.544927+0000","last_undegraded":"2024-07-26T13:37:17.544927+0000","last_fullsized":"2024-07-26T13:37:17.544927+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:38:39.329366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.544960+0000","last_change":"2024-07-26T13:36:59.460133+0000","last_active":"2024-07-26T13:37:17.544960+0000","last_peered":"2024-07-26T13:37:17.544960+0000","last_clean":"2024-07-26T13:37:17.544960+0000","last_became_active":"2024-07-26T13:36:59.459794+0000","last_became_peered":"2024-07-26T13:36:59.459794+0000","last_unstale":"2024-07-26T13:37:17.544960+0000","last_undegraded":"2024-07-26T13:37:17.544960+0000","last_fullsized":"2024-07-26T13:37:17.544960+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:15:36.024347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.543706+0000","last_change":"2024-07-26T13:36:59.462390+0000","last_active":"2024-07-26T13:37:17.543706+0000","last_peered":"2024-07-26T13:37:17.543706+0000","last_clean":"2024-07-26T13:37:17.543706+0000","last_became_active":"2024-07-26T13:36:59.461968+0000","last_became_peered":"2024-07-26T13:36:59.461968+0000","last_unstale":"2024-07-26T13:37:17.543706+0000","last_undegraded":"2024-07-26T13:37:17.543706+0000","last_fullsized":"2024-07-26T13:37:17.543706+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:49:47.142472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.547331+0000","last_change":"2024-07-26T13:36:59.460206+0000","last_active":"2024-07-26T13:37:17.547331+0000","last_peered":"2024-07-26T13:37:17.547331+0000","last_clean":"2024-07-26T13:37:17.547331+0000","last_became_active":"2024-07-26T13:36:59.459811+0000","last_became_peered":"2024-07-26T13:36:59.459811+0000","last_unstale":"2024-07-26T13:37:17.547331+0000","last_undegraded":"2024-07-26T13:37:17.547331+0000","last_fullsized":"2024-07-26T13:37:17.547331+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:46:57.112314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.545663+0000","last_change":"2024-07-26T13:36:59.460197+0000","last_active":"2024-07-26T13:37:17.545663+0000","last_peered":"2024-07-26T13:37:17.545663+0000","last_clean":"2024-07-26T13:37:17.545663+0000","last_became_active":"2024-07-26T13:36:59.459686+0000","last_became_peered":"2024-07-26T13:36:59.459686+0000","last_unstale":"2024-07-26T13:37:17.545663+0000","last_undegraded":"2024-07-26T13:37:17.545663+0000","last_fullsized":"2024-07-26T13:37:17.545663+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T13:39:22.407376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550550+0000","last_change":"2024-07-26T13:36:59.462545+0000","last_active":"2024-07-26T13:37:17.550550+0000","last_peered":"2024-07-26T13:37:17.550550+0000","last_clean":"2024-07-26T13:37:17.550550+0000","last_became_active":"2024-07-26T13:36:59.462115+0000","last_became_peered":"2024-07-26T13:36:59.462115+0000","last_unstale":"2024-07-26T13:37:17.550550+0000","last_undegraded":"2024-07-26T13:37:17.550550+0000","last_fullsized":"2024-07-26T13:37:17.550550+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T21:44:13.715076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.547646+0000","last_change":"2024-07-26T13:36:59.465317+0000","last_active":"2024-07-26T13:37:17.547646+0000","last_peered":"2024-07-26T13:37:17.547646+0000","last_clean":"2024-07-26T13:37:17.547646+0000","last_became_active":"2024-07-26T13:36:59.464943+0000","last_became_peered":"2024-07-26T13:36:59.464943+0000","last_unstale":"2024-07-26T13:37:17.547646+0000","last_undegraded":"2024-07-26T13:37:17.547646+0000","last_fullsized":"2024-07-26T13:37:17.547646+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T13:53:54.160584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.565505+0000","last_change":"2024-07-26T13:36:59.462884+0000","last_active":"2024-07-26T13:37:17.565505+0000","last_peered":"2024-07-26T13:37:17.565505+0000","last_clean":"2024-07-26T13:37:17.565505+0000","last_became_active":"2024-07-26T13:36:59.462564+0000","last_became_peered":"2024-07-26T13:36:59.462564+0000","last_unstale":"2024-07-26T13:37:17.565505+0000","last_undegraded":"2024-07-26T13:37:17.565505+0000","last_fullsized":"2024-07-26T13:37:17.565505+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:50:19.549148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.543901+0000","last_change":"2024-07-26T13:36:59.468716+0000","last_active":"2024-07-26T13:37:17.543901+0000","last_peered":"2024-07-26T13:37:17.543901+0000","last_clean":"2024-07-26T13:37:17.543901+0000","last_became_active":"2024-07-26T13:36:59.468048+0000","last_became_peered":"2024-07-26T13:36:59.468048+0000","last_unstale":"2024-07-26T13:37:17.543901+0000","last_undegraded":"2024-07-26T13:37:17.543901+0000","last_fullsized":"2024-07-26T13:37:17.543901+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:51:29.510047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.542939+0000","last_change":"2024-07-26T13:36:59.466565+0000","last_active":"2024-07-26T13:37:17.542939+0000","last_peered":"2024-07-26T13:37:17.542939+0000","last_clean":"2024-07-26T13:37:17.542939+0000","last_became_active":"2024-07-26T13:36:59.466235+0000","last_became_peered":"2024-07-26T13:36:59.466235+0000","last_unstale":"2024-07-26T13:37:17.542939+0000","last_undegraded":"2024-07-26T13:37:17.542939+0000","last_fullsized":"2024-07-26T13:37:17.542939+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:28:44.326821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.546853+0000","last_change":"2024-07-26T13:36:59.458753+0000","last_active":"2024-07-26T13:37:17.546853+0000","last_peered":"2024-07-26T13:37:17.546853+0000","last_clean":"2024-07-26T13:37:17.546853+0000","last_became_active":"2024-07-26T13:36:59.458333+0000","last_became_peered":"2024-07-26T13:36:59.458333+0000","last_unstale":"2024-07-26T13:37:17.546853+0000","last_undegraded":"2024-07-26T13:37:17.546853+0000","last_fullsized":"2024-07-26T13:37:17.546853+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:33:54.806212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.543527+0000","last_change":"2024-07-26T13:36:59.468657+0000","last_active":"2024-07-26T13:37:17.543527+0000","last_peered":"2024-07-26T13:37:17.543527+0000","last_clean":"2024-07-26T13:37:17.543527+0000","last_became_active":"2024-07-26T13:36:59.468121+0000","last_became_peered":"2024-07-26T13:36:59.468121+0000","last_unstale":"2024-07-26T13:37:17.543527+0000","last_undegraded":"2024-07-26T13:37:17.543527+0000","last_fullsized":"2024-07-26T13:37:17.543527+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:23:53.959836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550962+0000","last_change":"2024-07-26T13:36:59.469067+0000","last_active":"2024-07-26T13:37:17.550962+0000","last_peered":"2024-07-26T13:37:17.550962+0000","last_clean":"2024-07-26T13:37:17.550962+0000","last_became_active":"2024-07-26T13:36:59.468706+0000","last_became_peered":"2024-07-26T13:36:59.468706+0000","last_unstale":"2024-07-26T13:37:17.550962+0000","last_undegraded":"2024-07-26T13:37:17.550962+0000","last_fullsized":"2024-07-26T13:37:17.550962+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:21:35.974618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.548550+0000","last_change":"2024-07-26T13:36:59.469807+0000","last_active":"2024-07-26T13:37:17.548550+0000","last_peered":"2024-07-26T13:37:17.548550+0000","last_clean":"2024-07-26T13:37:17.548550+0000","last_became_active":"2024-07-26T13:36:59.469020+0000","last_became_peered":"2024-07-26T13:36:59.469020+0000","last_unstale":"2024-07-26T13:37:17.548550+0000","last_undegraded":"2024-07-26T13:37:17.548550+0000","last_fullsized":"2024-07-26T13:37:17.548550+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:29:57.931598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.565302+0000","last_change":"2024-07-26T13:36:59.465236+0000","last_active":"2024-07-26T13:37:17.565302+0000","last_peered":"2024-07-26T13:37:17.565302+0000","last_clean":"2024-07-26T13:37:17.565302+0000","last_became_active":"2024-07-26T13:36:59.464415+0000","last_became_peered":"2024-07-26T13:36:59.464415+0000","last_unstale":"2024-07-26T13:37:17.565302+0000","last_undegraded":"2024-07-26T13:37:17.565302+0000","last_fullsized":"2024-07-26T13:37:17.565302+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:46:20.373688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.545250+0000","last_change":"2024-07-26T13:36:59.466113+0000","last_active":"2024-07-26T13:37:17.545250+0000","last_peered":"2024-07-26T13:37:17.545250+0000","last_clean":"2024-07-26T13:37:17.545250+0000","last_became_active":"2024-07-26T13:36:59.465743+0000","last_became_peered":"2024-07-26T13:36:59.465743+0000","last_unstale":"2024-07-26T13:37:17.545250+0000","last_undegraded":"2024-07-26T13:37:17.545250+0000","last_fullsized":"2024-07-26T13:37:17.545250+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:14:41.839204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551356+0000","last_change":"2024-07-26T13:36:59.460672+0000","last_active":"2024-07-26T13:37:17.551356+0000","last_peered":"2024-07-26T13:37:17.551356+0000","last_clean":"2024-07-26T13:37:17.551356+0000","last_became_active":"2024-07-26T13:36:59.460247+0000","last_became_peered":"2024-07-26T13:36:59.460247+0000","last_unstale":"2024-07-26T13:37:17.551356+0000","last_undegraded":"2024-07-26T13:37:17.551356+0000","last_fullsized":"2024-07-26T13:37:17.551356+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:37:51.109933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551707+0000","last_change":"2024-07-26T13:36:59.464784+0000","last_active":"2024-07-26T13:37:17.551707+0000","last_peered":"2024-07-26T13:37:17.551707+0000","last_clean":"2024-07-26T13:37:17.551707+0000","last_became_active":"2024-07-26T13:36:59.464261+0000","last_became_peered":"2024-07-26T13:36:59.464261+0000","last_unstale":"2024-07-26T13:37:17.551707+0000","last_undegraded":"2024-07-26T13:37:17.551707+0000","last_fullsized":"2024-07-26T13:37:17.551707+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:43:10.134037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.546345+0000","last_change":"2024-07-26T13:36:59.460421+0000","last_active":"2024-07-26T13:37:17.546345+0000","last_peered":"2024-07-26T13:37:17.546345+0000","last_clean":"2024-07-26T13:37:17.546345+0000","last_became_active":"2024-07-26T13:36:59.460010+0000","last_became_peered":"2024-07-26T13:36:59.460010+0000","last_unstale":"2024-07-26T13:37:17.546345+0000","last_undegraded":"2024-07-26T13:37:17.546345+0000","last_fullsized":"2024-07-26T13:37:17.546345+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:16:21.651673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.553536+0000","last_change":"2024-07-26T13:36:59.468826+0000","last_active":"2024-07-26T13:37:17.553536+0000","last_peered":"2024-07-26T13:37:17.553536+0000","last_clean":"2024-07-26T13:37:17.553536+0000","last_became_active":"2024-07-26T13:36:59.468406+0000","last_became_peered":"2024-07-26T13:36:59.468406+0000","last_unstale":"2024-07-26T13:37:17.553536+0000","last_undegraded":"2024-07-26T13:37:17.553536+0000","last_fullsized":"2024-07-26T13:37:17.553536+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:22:10.590545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":74,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:40.598532+0000","last_change":"2024-07-26T13:37:17.559014+0000","last_active":"2024-07-26T13:37:40.598532+0000","last_peered":"2024-07-26T13:37:40.598532+0000","last_clean":"2024-07-26T13:37:40.598532+0000","last_became_active":"2024-07-26T13:37:17.552325+0000","last_became_peered":"2024-07-26T13:37:17.552325+0000","last_unstale":"2024-07-26T13:37:40.598532+0000","last_undegraded":"2024-07-26T13:37:40.598532+0000","last_fullsized":"2024-07-26T13:37:40.598532+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:26:26.790337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.563761+0000","last_change":"2024-07-26T13:37:17.562871+0000","last_active":"2024-07-26T13:37:17.563761+0000","last_peered":"2024-07-26T13:37:17.563761+0000","last_clean":"2024-07-26T13:37:17.563761+0000","last_became_active":"2024-07-26T13:37:17.562363+0000","last_became_peered":"2024-07-26T13:37:17.562363+0000","last_unstale":"2024-07-26T13:37:17.563761+0000","last_undegraded":"2024-07-26T13:37:17.563761+0000","last_fullsized":"2024-07-26T13:37:17.563761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:25:36.172091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"79'3","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.547720+0000","last_change":"2024-07-26T13:36:56.434778+0000","last_active":"2024-07-26T13:37:17.547720+0000","last_peered":"2024-07-26T13:37:17.547720+0000","last_clean":"2024-07-26T13:37:17.547720+0000","last_became_active":"2024-07-26T13:36:56.434288+0000","last_became_peered":"2024-07-26T13:36:56.434288+0000","last_unstale":"2024-07-26T13:37:17.547720+0000","last_undegraded":"2024-07-26T13:37:17.547720+0000","last_fullsized":"2024-07-26T13:37:17.547720+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:24:22.990358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.565086+0000","last_change":"2024-07-26T13:36:56.424665+0000","last_active":"2024-07-26T13:37:17.565086+0000","last_peered":"2024-07-26T13:37:17.565086+0000","last_clean":"2024-07-26T13:37:17.565086+0000","last_became_active":"2024-07-26T13:36:56.424290+0000","last_became_peered":"2024-07-26T13:36:56.424290+0000","last_unstale":"2024-07-26T13:37:17.565086+0000","last_undegraded":"2024-07-26T13:37:17.565086+0000","last_fullsized":"2024-07-26T13:37:17.565086+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:49:59.520194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.542987+0000","last_change":"2024-07-26T13:36:59.454609+0000","last_active":"2024-07-26T13:37:17.542987+0000","last_peered":"2024-07-26T13:37:17.542987+0000","last_clean":"2024-07-26T13:37:17.542987+0000","last_became_active":"2024-07-26T13:36:59.454228+0000","last_became_peered":"2024-07-26T13:36:59.454228+0000","last_unstale":"2024-07-26T13:37:17.542987+0000","last_undegraded":"2024-07-26T13:37:17.542987+0000","last_fullsized":"2024-07-26T13:37:17.542987+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:02:06.244433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.547076+0000","last_change":"2024-07-26T13:36:56.430268+0000","last_active":"2024-07-26T13:37:17.547076+0000","last_peered":"2024-07-26T13:37:17.547076+0000","last_clean":"2024-07-26T13:37:17.547076+0000","last_became_active":"2024-07-26T13:36:56.428583+0000","last_became_peered":"2024-07-26T13:36:56.428583+0000","last_unstale":"2024-07-26T13:37:17.547076+0000","last_undegraded":"2024-07-26T13:37:17.547076+0000","last_fullsized":"2024-07-26T13:37:17.547076+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:07:05.864886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.544501+0000","last_change":"2024-07-26T13:36:59.456188+0000","last_active":"2024-07-26T13:37:17.544501+0000","last_peered":"2024-07-26T13:37:17.544501+0000","last_clean":"2024-07-26T13:37:17.544501+0000","last_became_active":"2024-07-26T13:36:59.455842+0000","last_became_peered":"2024-07-26T13:36:59.455842+0000","last_unstale":"2024-07-26T13:37:17.544501+0000","last_undegraded":"2024-07-26T13:37:17.544501+0000","last_fullsized":"2024-07-26T13:37:17.544501+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:00:19.162239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"79'3","reported_seq":55,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:41.599642+0000","last_change":"2024-07-26T13:36:56.432818+0000","last_active":"2024-07-26T13:37:41.599642+0000","last_peered":"2024-07-26T13:37:41.599642+0000","last_clean":"2024-07-26T13:37:41.599642+0000","last_became_active":"2024-07-26T13:36:56.432419+0000","last_became_peered":"2024-07-26T13:36:56.432419+0000","last_unstale":"2024-07-26T13:37:41.599642+0000","last_undegraded":"2024-07-26T13:37:41.599642+0000","last_fullsized":"2024-07-26T13:37:41.599642+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:55:51.972470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.544086+0000","last_change":"2024-07-26T13:36:59.460033+0000","last_active":"2024-07-26T13:37:17.544086+0000","last_peered":"2024-07-26T13:37:17.544086+0000","last_clean":"2024-07-26T13:37:17.544086+0000","last_became_active":"2024-07-26T13:36:59.459562+0000","last_became_peered":"2024-07-26T13:36:59.459562+0000","last_unstale":"2024-07-26T13:37:17.544086+0000","last_undegraded":"2024-07-26T13:37:17.544086+0000","last_fullsized":"2024-07-26T13:37:17.544086+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:58:14.155973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551314+0000","last_change":"2024-07-26T13:36:56.432563+0000","last_active":"2024-07-26T13:37:17.551314+0000","last_peered":"2024-07-26T13:37:17.551314+0000","last_clean":"2024-07-26T13:37:17.551314+0000","last_became_active":"2024-07-26T13:36:56.431134+0000","last_became_peered":"2024-07-26T13:36:56.431134+0000","last_unstale":"2024-07-26T13:37:17.551314+0000","last_undegraded":"2024-07-26T13:37:17.551314+0000","last_fullsized":"2024-07-26T13:37:17.551314+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:42:00.653345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.545571+0000","last_change":"2024-07-26T13:36:59.469782+0000","last_active":"2024-07-26T13:37:17.545571+0000","last_peered":"2024-07-26T13:37:17.545571+0000","last_clean":"2024-07-26T13:37:17.545571+0000","last_became_active":"2024-07-26T13:36:59.469412+0000","last_became_peered":"2024-07-26T13:36:59.469412+0000","last_unstale":"2024-07-26T13:37:17.545571+0000","last_undegraded":"2024-07-26T13:37:17.545571+0000","last_fullsized":"2024-07-26T13:37:17.545571+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:28:07.741486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.548023+0000","last_change":"2024-07-26T13:36:56.488981+0000","last_active":"2024-07-26T13:37:17.548023+0000","last_peered":"2024-07-26T13:37:17.548023+0000","last_clean":"2024-07-26T13:37:17.548023+0000","last_became_active":"2024-07-26T13:36:56.488480+0000","last_became_peered":"2024-07-26T13:36:56.488480+0000","last_unstale":"2024-07-26T13:37:17.548023+0000","last_undegraded":"2024-07-26T13:37:17.548023+0000","last_fullsized":"2024-07-26T13:37:17.548023+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:41:10.041007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.543895+0000","last_change":"2024-07-26T13:36:59.473183+0000","last_active":"2024-07-26T13:37:17.543895+0000","last_peered":"2024-07-26T13:37:17.543895+0000","last_clean":"2024-07-26T13:37:17.543895+0000","last_became_active":"2024-07-26T13:36:59.472814+0000","last_became_peered":"2024-07-26T13:36:59.472814+0000","last_unstale":"2024-07-26T13:37:17.543895+0000","last_undegraded":"2024-07-26T13:37:17.543895+0000","last_fullsized":"2024-07-26T13:37:17.543895+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:57:03.396384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'29","reported_seq":115,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:42.611944+0000","last_change":"2024-07-26T13:36:56.435034+0000","last_active":"2024-07-26T13:37:42.611944+0000","last_peered":"2024-07-26T13:37:42.611944+0000","last_clean":"2024-07-26T13:37:42.611944+0000","last_became_active":"2024-07-26T13:36:56.434611+0000","last_became_peered":"2024-07-26T13:36:56.434611+0000","last_unstale":"2024-07-26T13:37:42.611944+0000","last_undegraded":"2024-07-26T13:37:42.611944+0000","last_fullsized":"2024-07-26T13:37:42.611944+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:18:27.272077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62335,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":60,"num_read_kb":90,"num_write":30,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.553386+0000","last_change":"2024-07-26T13:36:59.465831+0000","last_active":"2024-07-26T13:37:17.553386+0000","last_peered":"2024-07-26T13:37:17.553386+0000","last_clean":"2024-07-26T13:37:17.553386+0000","last_became_active":"2024-07-26T13:36:59.465404+0000","last_became_peered":"2024-07-26T13:36:59.465404+0000","last_unstale":"2024-07-26T13:37:17.553386+0000","last_undegraded":"2024-07-26T13:37:17.553386+0000","last_fullsized":"2024-07-26T13:37:17.553386+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:42:53.795926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'6","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.552433+0000","last_change":"2024-07-26T13:36:56.435170+0000","last_active":"2024-07-26T13:37:17.552433+0000","last_peered":"2024-07-26T13:37:17.552433+0000","last_clean":"2024-07-26T13:37:17.552433+0000","last_became_active":"2024-07-26T13:36:56.434715+0000","last_became_peered":"2024-07-26T13:36:56.434715+0000","last_unstale":"2024-07-26T13:37:17.552433+0000","last_undegraded":"2024-07-26T13:37:17.552433+0000","last_fullsized":"2024-07-26T13:37:17.552433+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:05:09.712510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.565171+0000","last_change":"2024-07-26T13:36:59.461926+0000","last_active":"2024-07-26T13:37:17.565171+0000","last_peered":"2024-07-26T13:37:17.565171+0000","last_clean":"2024-07-26T13:37:17.565171+0000","last_became_active":"2024-07-26T13:36:59.461512+0000","last_became_peered":"2024-07-26T13:36:59.461512+0000","last_unstale":"2024-07-26T13:37:17.565171+0000","last_undegraded":"2024-07-26T13:37:17.565171+0000","last_fullsized":"2024-07-26T13:37:17.565171+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:34:24.405663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.546843+0000","last_change":"2024-07-26T13:36:56.430062+0000","last_active":"2024-07-26T13:37:17.546843+0000","last_peered":"2024-07-26T13:37:17.546843+0000","last_clean":"2024-07-26T13:37:17.546843+0000","last_became_active":"2024-07-26T13:36:56.429654+0000","last_became_peered":"2024-07-26T13:36:56.429654+0000","last_unstale":"2024-07-26T13:37:17.546843+0000","last_undegraded":"2024-07-26T13:37:17.546843+0000","last_fullsized":"2024-07-26T13:37:17.546843+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:21:09.514359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550829+0000","last_change":"2024-07-26T13:36:59.474459+0000","last_active":"2024-07-26T13:37:17.550829+0000","last_peered":"2024-07-26T13:37:17.550829+0000","last_clean":"2024-07-26T13:37:17.550829+0000","last_became_active":"2024-07-26T13:36:59.474061+0000","last_became_peered":"2024-07-26T13:36:59.474061+0000","last_unstale":"2024-07-26T13:37:17.550829+0000","last_undegraded":"2024-07-26T13:37:17.550829+0000","last_fullsized":"2024-07-26T13:37:17.550829+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:37:21.691491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'4","reported_seq":60,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:41.599861+0000","last_change":"2024-07-26T13:36:56.432915+0000","last_active":"2024-07-26T13:37:41.599861+0000","last_peered":"2024-07-26T13:37:41.599861+0000","last_clean":"2024-07-26T13:37:41.599861+0000","last_became_active":"2024-07-26T13:36:56.432499+0000","last_became_peered":"2024-07-26T13:36:56.432499+0000","last_unstale":"2024-07-26T13:37:41.599861+0000","last_undegraded":"2024-07-26T13:37:41.599861+0000","last_fullsized":"2024-07-26T13:37:41.599861+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:02:14.306950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":31,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550560+0000","last_change":"2024-07-26T13:36:59.468945+0000","last_active":"2024-07-26T13:37:17.550560+0000","last_peered":"2024-07-26T13:37:17.550560+0000","last_clean":"2024-07-26T13:37:17.550560+0000","last_became_active":"2024-07-26T13:36:59.467970+0000","last_became_peered":"2024-07-26T13:36:59.467970+0000","last_unstale":"2024-07-26T13:37:17.550560+0000","last_undegraded":"2024-07-26T13:37:17.550560+0000","last_fullsized":"2024-07-26T13:37:17.550560+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T13:59:19.514945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'15","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551317+0000","last_change":"2024-07-26T13:36:56.487014+0000","last_active":"2024-07-26T13:37:17.551317+0000","last_peered":"2024-07-26T13:37:17.551317+0000","last_clean":"2024-07-26T13:37:17.551317+0000","last_became_active":"2024-07-26T13:36:56.486470+0000","last_became_peered":"2024-07-26T13:36:56.486470+0000","last_unstale":"2024-07-26T13:37:17.551317+0000","last_undegraded":"2024-07-26T13:37:17.551317+0000","last_fullsized":"2024-07-26T13:37:17.551317+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:07:02.205644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.546513+0000","last_change":"2024-07-26T13:36:59.462465+0000","last_active":"2024-07-26T13:37:17.546513+0000","last_peered":"2024-07-26T13:37:17.546513+0000","last_clean":"2024-07-26T13:37:17.546513+0000","last_became_active":"2024-07-26T13:36:59.462024+0000","last_became_peered":"2024-07-26T13:36:59.462024+0000","last_unstale":"2024-07-26T13:37:17.546513+0000","last_undegraded":"2024-07-26T13:37:17.546513+0000","last_fullsized":"2024-07-26T13:37:17.546513+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:34:14.317762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551923+0000","last_change":"2024-07-26T13:36:56.435007+0000","last_active":"2024-07-26T13:37:17.551923+0000","last_peered":"2024-07-26T13:37:17.551923+0000","last_clean":"2024-07-26T13:37:17.551923+0000","last_became_active":"2024-07-26T13:36:56.434444+0000","last_became_peered":"2024-07-26T13:36:56.434444+0000","last_unstale":"2024-07-26T13:37:17.551923+0000","last_undegraded":"2024-07-26T13:37:17.551923+0000","last_fullsized":"2024-07-26T13:37:17.551923+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:27:51.041349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.546679+0000","last_change":"2024-07-26T13:36:59.464164+0000","last_active":"2024-07-26T13:37:17.546679+0000","last_peered":"2024-07-26T13:37:17.546679+0000","last_clean":"2024-07-26T13:37:17.546679+0000","last_became_active":"2024-07-26T13:36:59.463782+0000","last_became_peered":"2024-07-26T13:36:59.463782+0000","last_unstale":"2024-07-26T13:37:17.546679+0000","last_undegraded":"2024-07-26T13:37:17.546679+0000","last_fullsized":"2024-07-26T13:37:17.546679+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:48:06.481731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.565453+0000","last_change":"2024-07-26T13:36:56.429653+0000","last_active":"2024-07-26T13:37:17.565453+0000","last_peered":"2024-07-26T13:37:17.565453+0000","last_clean":"2024-07-26T13:37:17.565453+0000","last_became_active":"2024-07-26T13:36:56.429060+0000","last_became_peered":"2024-07-26T13:36:56.429060+0000","last_unstale":"2024-07-26T13:37:17.565453+0000","last_undegraded":"2024-07-26T13:37:17.565453+0000","last_fullsized":"2024-07-26T13:37:17.565453+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:41:48.261709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551343+0000","last_change":"2024-07-26T13:36:59.471560+0000","last_active":"2024-07-26T13:37:17.551343+0000","last_peered":"2024-07-26T13:37:17.551343+0000","last_clean":"2024-07-26T13:37:17.551343+0000","last_became_active":"2024-07-26T13:36:59.471151+0000","last_became_peered":"2024-07-26T13:36:59.471151+0000","last_unstale":"2024-07-26T13:37:17.551343+0000","last_undegraded":"2024-07-26T13:37:17.551343+0000","last_fullsized":"2024-07-26T13:37:17.551343+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:49:20.513308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.548108+0000","last_change":"2024-07-26T13:36:56.430010+0000","last_active":"2024-07-26T13:37:17.548108+0000","last_peered":"2024-07-26T13:37:17.548108+0000","last_clean":"2024-07-26T13:37:17.548108+0000","last_became_active":"2024-07-26T13:36:56.429120+0000","last_became_peered":"2024-07-26T13:36:56.429120+0000","last_unstale":"2024-07-26T13:37:17.548108+0000","last_undegraded":"2024-07-26T13:37:17.548108+0000","last_fullsized":"2024-07-26T13:37:17.548108+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:23:43.321975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551149+0000","last_change":"2024-07-26T13:36:59.467788+0000","last_active":"2024-07-26T13:37:17.551149+0000","last_peered":"2024-07-26T13:37:17.551149+0000","last_clean":"2024-07-26T13:37:17.551149+0000","last_became_active":"2024-07-26T13:36:59.467399+0000","last_became_peered":"2024-07-26T13:36:59.467399+0000","last_unstale":"2024-07-26T13:37:17.551149+0000","last_undegraded":"2024-07-26T13:37:17.551149+0000","last_fullsized":"2024-07-26T13:37:17.551149+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:48:51.442999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'3","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.553251+0000","last_change":"2024-07-26T13:36:56.588633+0000","last_active":"2024-07-26T13:37:17.553251+0000","last_peered":"2024-07-26T13:37:17.553251+0000","last_clean":"2024-07-26T13:37:17.553251+0000","last_became_active":"2024-07-26T13:36:56.588113+0000","last_became_peered":"2024-07-26T13:36:56.588113+0000","last_unstale":"2024-07-26T13:37:17.553251+0000","last_undegraded":"2024-07-26T13:37:17.553251+0000","last_fullsized":"2024-07-26T13:37:17.553251+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:03:55.075052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551116+0000","last_change":"2024-07-26T13:36:59.472854+0000","last_active":"2024-07-26T13:37:17.551116+0000","last_peered":"2024-07-26T13:37:17.551116+0000","last_clean":"2024-07-26T13:37:17.551116+0000","last_became_active":"2024-07-26T13:36:59.472466+0000","last_became_peered":"2024-07-26T13:36:59.472466+0000","last_unstale":"2024-07-26T13:37:17.551116+0000","last_undegraded":"2024-07-26T13:37:17.551116+0000","last_fullsized":"2024-07-26T13:37:17.551116+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:50:57.454149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":102,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550777+0000","last_change":"2024-07-26T13:35:34.839121+0000","last_active":"2024-07-26T13:37:17.550777+0000","last_peered":"2024-07-26T13:37:17.550777+0000","last_clean":"2024-07-26T13:37:17.550777+0000","last_became_active":"2024-07-26T13:35:34.833465+0000","last_became_peered":"2024-07-26T13:35:34.833465+0000","last_unstale":"2024-07-26T13:37:17.550777+0000","last_undegraded":"2024-07-26T13:37:17.550777+0000","last_fullsized":"2024-07-26T13:37:17.550777+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:32:23.285428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:32:23.285428+0000","last_clean_scrub_stamp":"2024-07-26T13:32:23.285428+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:29:31.650487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'3","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.561400+0000","last_change":"2024-07-26T13:36:56.428648+0000","last_active":"2024-07-26T13:37:17.561400+0000","last_peered":"2024-07-26T13:37:17.561400+0000","last_clean":"2024-07-26T13:37:17.561400+0000","last_became_active":"2024-07-26T13:36:56.427665+0000","last_became_peered":"2024-07-26T13:36:56.427665+0000","last_unstale":"2024-07-26T13:37:17.561400+0000","last_undegraded":"2024-07-26T13:37:17.561400+0000","last_fullsized":"2024-07-26T13:37:17.561400+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:41:39.371687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.543335+0000","last_change":"2024-07-26T13:36:59.460625+0000","last_active":"2024-07-26T13:37:17.543335+0000","last_peered":"2024-07-26T13:37:17.543335+0000","last_clean":"2024-07-26T13:37:17.543335+0000","last_became_active":"2024-07-26T13:36:59.458658+0000","last_became_peered":"2024-07-26T13:36:59.458658+0000","last_unstale":"2024-07-26T13:37:17.543335+0000","last_undegraded":"2024-07-26T13:37:17.543335+0000","last_fullsized":"2024-07-26T13:37:17.543335+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:46:42.812402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550730+0000","last_change":"2024-07-26T13:36:56.432617+0000","last_active":"2024-07-26T13:37:17.550730+0000","last_peered":"2024-07-26T13:37:17.550730+0000","last_clean":"2024-07-26T13:37:17.550730+0000","last_became_active":"2024-07-26T13:36:56.431519+0000","last_became_peered":"2024-07-26T13:36:56.431519+0000","last_unstale":"2024-07-26T13:37:17.550730+0000","last_undegraded":"2024-07-26T13:37:17.550730+0000","last_fullsized":"2024-07-26T13:37:17.550730+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:32:21.120076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.565072+0000","last_change":"2024-07-26T13:36:59.464084+0000","last_active":"2024-07-26T13:37:17.565072+0000","last_peered":"2024-07-26T13:37:17.565072+0000","last_clean":"2024-07-26T13:37:17.565072+0000","last_became_active":"2024-07-26T13:36:59.463755+0000","last_became_peered":"2024-07-26T13:36:59.463755+0000","last_unstale":"2024-07-26T13:37:17.565072+0000","last_undegraded":"2024-07-26T13:37:17.565072+0000","last_fullsized":"2024-07-26T13:37:17.565072+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:53:20.034188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"79'4","reported_seq":57,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:40.597751+0000","last_change":"2024-07-26T13:36:56.486071+0000","last_active":"2024-07-26T13:37:40.597751+0000","last_peered":"2024-07-26T13:37:40.597751+0000","last_clean":"2024-07-26T13:37:40.597751+0000","last_became_active":"2024-07-26T13:36:56.485672+0000","last_became_peered":"2024-07-26T13:36:56.485672+0000","last_unstale":"2024-07-26T13:37:40.597751+0000","last_undegraded":"2024-07-26T13:37:40.597751+0000","last_fullsized":"2024-07-26T13:37:40.597751+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T21:29:45.666651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.566653+0000","last_change":"2024-07-26T13:37:17.566555+0000","last_active":"2024-07-26T13:37:17.566653+0000","last_peered":"2024-07-26T13:37:17.566653+0000","last_clean":"2024-07-26T13:37:17.566653+0000","last_became_active":"2024-07-26T13:37:17.566040+0000","last_became_peered":"2024-07-26T13:37:17.566040+0000","last_unstale":"2024-07-26T13:37:17.566653+0000","last_undegraded":"2024-07-26T13:37:17.566653+0000","last_fullsized":"2024-07-26T13:37:17.566653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:48:33.078890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'1","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550922+0000","last_change":"2024-07-26T13:36:56.431811+0000","last_active":"2024-07-26T13:37:17.550922+0000","last_peered":"2024-07-26T13:37:17.550922+0000","last_clean":"2024-07-26T13:37:17.550922+0000","last_became_active":"2024-07-26T13:36:56.431223+0000","last_became_peered":"2024-07-26T13:36:56.431223+0000","last_unstale":"2024-07-26T13:37:17.550922+0000","last_undegraded":"2024-07-26T13:37:17.550922+0000","last_fullsized":"2024-07-26T13:37:17.550922+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:08:00.334515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.547257+0000","last_change":"2024-07-26T13:36:59.463972+0000","last_active":"2024-07-26T13:37:17.547257+0000","last_peered":"2024-07-26T13:37:17.547257+0000","last_clean":"2024-07-26T13:37:17.547257+0000","last_became_active":"2024-07-26T13:36:59.463496+0000","last_became_peered":"2024-07-26T13:36:59.463496+0000","last_unstale":"2024-07-26T13:37:17.547257+0000","last_undegraded":"2024-07-26T13:37:17.547257+0000","last_fullsized":"2024-07-26T13:37:17.547257+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:02:06.215372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.545709+0000","last_change":"2024-07-26T13:36:56.486681+0000","last_active":"2024-07-26T13:37:17.545709+0000","last_peered":"2024-07-26T13:37:17.545709+0000","last_clean":"2024-07-26T13:37:17.545709+0000","last_became_active":"2024-07-26T13:36:56.486328+0000","last_became_peered":"2024-07-26T13:36:56.486328+0000","last_unstale":"2024-07-26T13:37:17.545709+0000","last_undegraded":"2024-07-26T13:37:17.545709+0000","last_fullsized":"2024-07-26T13:37:17.545709+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:10:36.557707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.547494+0000","last_change":"2024-07-26T13:36:59.451415+0000","last_active":"2024-07-26T13:37:17.547494+0000","last_peered":"2024-07-26T13:37:17.547494+0000","last_clean":"2024-07-26T13:37:17.547494+0000","last_became_active":"2024-07-26T13:36:59.450992+0000","last_became_peered":"2024-07-26T13:36:59.450992+0000","last_unstale":"2024-07-26T13:37:17.547494+0000","last_undegraded":"2024-07-26T13:37:17.547494+0000","last_fullsized":"2024-07-26T13:37:17.547494+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:29:11.303916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.555734+0000","last_change":"2024-07-26T13:37:17.555643+0000","last_active":"2024-07-26T13:37:17.555734+0000","last_peered":"2024-07-26T13:37:17.555734+0000","last_clean":"2024-07-26T13:37:17.555734+0000","last_became_active":"2024-07-26T13:37:17.549756+0000","last_became_peered":"2024-07-26T13:37:17.549756+0000","last_unstale":"2024-07-26T13:37:17.555734+0000","last_undegraded":"2024-07-26T13:37:17.555734+0000","last_fullsized":"2024-07-26T13:37:17.555734+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:20:16.911286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.565386+0000","last_change":"2024-07-26T13:36:59.465538+0000","last_active":"2024-07-26T13:37:17.565386+0000","last_peered":"2024-07-26T13:37:17.565386+0000","last_clean":"2024-07-26T13:37:17.565386+0000","last_became_active":"2024-07-26T13:36:59.464886+0000","last_became_peered":"2024-07-26T13:36:59.464886+0000","last_unstale":"2024-07-26T13:37:17.565386+0000","last_undegraded":"2024-07-26T13:37:17.565386+0000","last_fullsized":"2024-07-26T13:37:17.565386+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:19:12.273116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'11","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:37.479551+0000","last_change":"2024-07-26T13:36:56.428355+0000","last_active":"2024-07-26T13:37:37.479551+0000","last_peered":"2024-07-26T13:37:37.479551+0000","last_clean":"2024-07-26T13:37:37.479551+0000","last_became_active":"2024-07-26T13:36:56.427213+0000","last_became_peered":"2024-07-26T13:36:56.427213+0000","last_unstale":"2024-07-26T13:37:37.479551+0000","last_undegraded":"2024-07-26T13:37:37.479551+0000","last_fullsized":"2024-07-26T13:37:37.479551+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:48:09.730067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.543752+0000","last_change":"2024-07-26T13:36:59.459378+0000","last_active":"2024-07-26T13:37:17.543752+0000","last_peered":"2024-07-26T13:37:17.543752+0000","last_clean":"2024-07-26T13:37:17.543752+0000","last_became_active":"2024-07-26T13:36:59.458799+0000","last_became_peered":"2024-07-26T13:36:59.458799+0000","last_unstale":"2024-07-26T13:37:17.543752+0000","last_undegraded":"2024-07-26T13:37:17.543752+0000","last_fullsized":"2024-07-26T13:37:17.543752+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:45:19.587800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.561367+0000","last_change":"2024-07-26T13:36:56.588819+0000","last_active":"2024-07-26T13:37:17.561367+0000","last_peered":"2024-07-26T13:37:17.561367+0000","last_clean":"2024-07-26T13:37:17.561367+0000","last_became_active":"2024-07-26T13:36:56.588430+0000","last_became_peered":"2024-07-26T13:36:56.588430+0000","last_unstale":"2024-07-26T13:37:17.561367+0000","last_undegraded":"2024-07-26T13:37:17.561367+0000","last_fullsized":"2024-07-26T13:37:17.561367+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:07:21.387618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.562120+0000","last_change":"2024-07-26T13:37:17.562040+0000","last_active":"2024-07-26T13:37:17.562120+0000","last_peered":"2024-07-26T13:37:17.562120+0000","last_clean":"2024-07-26T13:37:17.562120+0000","last_became_active":"2024-07-26T13:37:17.561542+0000","last_became_peered":"2024-07-26T13:37:17.561542+0000","last_unstale":"2024-07-26T13:37:17.562120+0000","last_undegraded":"2024-07-26T13:37:17.562120+0000","last_fullsized":"2024-07-26T13:37:17.562120+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:21:08.161954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.544058+0000","last_change":"2024-07-26T13:36:56.489106+0000","last_active":"2024-07-26T13:37:17.544058+0000","last_peered":"2024-07-26T13:37:17.544058+0000","last_clean":"2024-07-26T13:37:17.544058+0000","last_became_active":"2024-07-26T13:36:56.488706+0000","last_became_peered":"2024-07-26T13:36:56.488706+0000","last_unstale":"2024-07-26T13:37:17.544058+0000","last_undegraded":"2024-07-26T13:37:17.544058+0000","last_fullsized":"2024-07-26T13:37:17.544058+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:00:59.204434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551358+0000","last_change":"2024-07-26T13:36:59.470412+0000","last_active":"2024-07-26T13:37:17.551358+0000","last_peered":"2024-07-26T13:37:17.551358+0000","last_clean":"2024-07-26T13:37:17.551358+0000","last_became_active":"2024-07-26T13:36:59.470010+0000","last_became_peered":"2024-07-26T13:36:59.470010+0000","last_unstale":"2024-07-26T13:37:17.551358+0000","last_undegraded":"2024-07-26T13:37:17.551358+0000","last_fullsized":"2024-07-26T13:37:17.551358+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:50:57.987600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.543473+0000","last_change":"2024-07-26T13:36:56.428671+0000","last_active":"2024-07-26T13:37:17.543473+0000","last_peered":"2024-07-26T13:37:17.543473+0000","last_clean":"2024-07-26T13:37:17.543473+0000","last_became_active":"2024-07-26T13:36:56.428301+0000","last_became_peered":"2024-07-26T13:36:56.428301+0000","last_unstale":"2024-07-26T13:37:17.543473+0000","last_undegraded":"2024-07-26T13:37:17.543473+0000","last_fullsized":"2024-07-26T13:37:17.543473+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:59:19.593664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.546377+0000","last_change":"2024-07-26T13:36:59.458627+0000","last_active":"2024-07-26T13:37:17.546377+0000","last_peered":"2024-07-26T13:37:17.546377+0000","last_clean":"2024-07-26T13:37:17.546377+0000","last_became_active":"2024-07-26T13:36:59.458199+0000","last_became_peered":"2024-07-26T13:36:59.458199+0000","last_unstale":"2024-07-26T13:37:17.546377+0000","last_undegraded":"2024-07-26T13:37:17.546377+0000","last_fullsized":"2024-07-26T13:37:17.546377+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:54:41.833223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'1","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551478+0000","last_change":"2024-07-26T13:36:56.431548+0000","last_active":"2024-07-26T13:37:17.551478+0000","last_peered":"2024-07-26T13:37:17.551478+0000","last_clean":"2024-07-26T13:37:17.551478+0000","last_became_active":"2024-07-26T13:36:56.430989+0000","last_became_peered":"2024-07-26T13:36:56.430989+0000","last_unstale":"2024-07-26T13:37:17.551478+0000","last_undegraded":"2024-07-26T13:37:17.551478+0000","last_fullsized":"2024-07-26T13:37:17.551478+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:46:51.655324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.543108+0000","last_change":"2024-07-26T13:36:59.458290+0000","last_active":"2024-07-26T13:37:17.543108+0000","last_peered":"2024-07-26T13:37:17.543108+0000","last_clean":"2024-07-26T13:37:17.543108+0000","last_became_active":"2024-07-26T13:36:59.457942+0000","last_became_peered":"2024-07-26T13:36:59.457942+0000","last_unstale":"2024-07-26T13:37:17.543108+0000","last_undegraded":"2024-07-26T13:37:17.543108+0000","last_fullsized":"2024-07-26T13:37:17.543108+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:51:58.797441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.567687+0000","last_change":"2024-07-26T13:37:17.567624+0000","last_active":"2024-07-26T13:37:17.567687+0000","last_peered":"2024-07-26T13:37:17.567687+0000","last_clean":"2024-07-26T13:37:17.567687+0000","last_became_active":"2024-07-26T13:37:17.562455+0000","last_became_peered":"2024-07-26T13:37:17.562455+0000","last_unstale":"2024-07-26T13:37:17.567687+0000","last_undegraded":"2024-07-26T13:37:17.567687+0000","last_fullsized":"2024-07-26T13:37:17.567687+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:50:36.418750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550738+0000","last_change":"2024-07-26T13:36:59.474034+0000","last_active":"2024-07-26T13:37:17.550738+0000","last_peered":"2024-07-26T13:37:17.550738+0000","last_clean":"2024-07-26T13:37:17.550738+0000","last_became_active":"2024-07-26T13:36:59.472358+0000","last_became_peered":"2024-07-26T13:36:59.472358+0000","last_unstale":"2024-07-26T13:37:17.550738+0000","last_undegraded":"2024-07-26T13:37:17.550738+0000","last_fullsized":"2024-07-26T13:37:17.550738+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:36:44.565413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.547175+0000","last_change":"2024-07-26T13:36:59.465546+0000","last_active":"2024-07-26T13:37:17.547175+0000","last_peered":"2024-07-26T13:37:17.547175+0000","last_clean":"2024-07-26T13:37:17.547175+0000","last_became_active":"2024-07-26T13:36:59.465137+0000","last_became_peered":"2024-07-26T13:36:59.465137+0000","last_unstale":"2024-07-26T13:37:17.547175+0000","last_undegraded":"2024-07-26T13:37:17.547175+0000","last_fullsized":"2024-07-26T13:37:17.547175+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:35:13.591354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.562100+0000","last_change":"2024-07-26T13:37:17.562012+0000","last_active":"2024-07-26T13:37:17.562100+0000","last_peered":"2024-07-26T13:37:17.562100+0000","last_clean":"2024-07-26T13:37:17.562100+0000","last_became_active":"2024-07-26T13:37:17.550218+0000","last_became_peered":"2024-07-26T13:37:17.550218+0000","last_unstale":"2024-07-26T13:37:17.562100+0000","last_undegraded":"2024-07-26T13:37:17.562100+0000","last_fullsized":"2024-07-26T13:37:17.562100+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:46:16.559533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.543841+0000","last_change":"2024-07-26T13:36:59.456896+0000","last_active":"2024-07-26T13:37:17.543841+0000","last_peered":"2024-07-26T13:37:17.543841+0000","last_clean":"2024-07-26T13:37:17.543841+0000","last_became_active":"2024-07-26T13:36:59.456530+0000","last_became_peered":"2024-07-26T13:36:59.456530+0000","last_unstale":"2024-07-26T13:37:17.543841+0000","last_undegraded":"2024-07-26T13:37:17.543841+0000","last_fullsized":"2024-07-26T13:37:17.543841+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:59:56.512501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'4","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.553522+0000","last_change":"2024-07-26T13:36:56.433119+0000","last_active":"2024-07-26T13:37:17.553522+0000","last_peered":"2024-07-26T13:37:17.553522+0000","last_clean":"2024-07-26T13:37:17.553522+0000","last_became_active":"2024-07-26T13:36:56.432765+0000","last_became_peered":"2024-07-26T13:36:56.432765+0000","last_unstale":"2024-07-26T13:37:17.553522+0000","last_undegraded":"2024-07-26T13:37:17.553522+0000","last_fullsized":"2024-07-26T13:37:17.553522+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:31:29.197106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.545548+0000","last_change":"2024-07-26T13:36:59.462125+0000","last_active":"2024-07-26T13:37:17.545548+0000","last_peered":"2024-07-26T13:37:17.545548+0000","last_clean":"2024-07-26T13:37:17.545548+0000","last_became_active":"2024-07-26T13:36:59.461796+0000","last_became_peered":"2024-07-26T13:36:59.461796+0000","last_unstale":"2024-07-26T13:37:17.545548+0000","last_undegraded":"2024-07-26T13:37:17.545548+0000","last_fullsized":"2024-07-26T13:37:17.545548+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:44:10.329046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.543605+0000","last_change":"2024-07-26T13:36:56.487251+0000","last_active":"2024-07-26T13:37:17.543605+0000","last_peered":"2024-07-26T13:37:17.543605+0000","last_clean":"2024-07-26T13:37:17.543605+0000","last_became_active":"2024-07-26T13:36:56.486918+0000","last_became_peered":"2024-07-26T13:36:56.486918+0000","last_unstale":"2024-07-26T13:37:17.543605+0000","last_undegraded":"2024-07-26T13:37:17.543605+0000","last_fullsized":"2024-07-26T13:37:17.543605+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:41:31.248895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.543268+0000","last_change":"2024-07-26T13:36:59.470810+0000","last_active":"2024-07-26T13:37:17.543268+0000","last_peered":"2024-07-26T13:37:17.543268+0000","last_clean":"2024-07-26T13:37:17.543268+0000","last_became_active":"2024-07-26T13:36:59.467371+0000","last_became_peered":"2024-07-26T13:36:59.467371+0000","last_unstale":"2024-07-26T13:37:17.543268+0000","last_undegraded":"2024-07-26T13:37:17.543268+0000","last_fullsized":"2024-07-26T13:37:17.543268+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:23:48.572933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'9","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.545993+0000","last_change":"2024-07-26T13:36:56.433598+0000","last_active":"2024-07-26T13:37:17.545993+0000","last_peered":"2024-07-26T13:37:17.545993+0000","last_clean":"2024-07-26T13:37:17.545993+0000","last_became_active":"2024-07-26T13:36:56.433325+0000","last_became_peered":"2024-07-26T13:36:56.433325+0000","last_unstale":"2024-07-26T13:37:17.545993+0000","last_undegraded":"2024-07-26T13:37:17.545993+0000","last_fullsized":"2024-07-26T13:37:17.545993+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:17:33.651765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66727,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":184,"num_read_kb":162,"num_write":165,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":644,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":200181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247480,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678040,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":443451,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874269,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":1097636,"data_compressed":41637,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037794,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":897024,"data_stored":443665,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234022,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":443563,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430250,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":1035906,"data_compressed":41637,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593776,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":443054,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790005,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":1035199,"data_compressed":41637,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953530,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":888832,"data_stored":443882,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149757,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":443365,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378691,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":505910,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542215,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":950272,"data_stored":500800,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705740,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":905216,"data_stored":444325,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34195,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34195,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-26T13:37:46.350 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-07-26T13:37:46.350 INFO:tasks.cephadm:Setting up client nodes... 2024-07-26T13:37:46.351 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-07-26T13:37:46.609 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:37:47.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:47 smithi100 ceph-mon[32599]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-26T13:37:47.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:47 smithi100 ceph-mon[32599]: pgmap v294: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-07-26T13:37:47.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:47 smithi167 ceph-mon[37140]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-26T13:37:47.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:47 smithi167 ceph-mon[37140]: pgmap v294: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-07-26T13:37:47.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:47 smithi138 ceph-mon[36954]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-26T13:37:47.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:47 smithi138 ceph-mon[36954]: pgmap v294: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-07-26T13:37:48.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.360+0000 7fdd7f429640 1 -- 172.21.15.100:0/709352517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd78102100 msgr2=0x7fdd78102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:48.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.360+0000 7fdd7f429640 1 --2- 172.21.15.100:0/709352517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd78102100 0x7fdd78102510 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdd74009900 tx=0x7fdd7402a410 comp rx=0 tx=0).stop 2024-07-26T13:37:48.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.360+0000 7fdd7f429640 1 -- 172.21.15.100:0/709352517 shutdown_connections 2024-07-26T13:37:48.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.360+0000 7fdd7f429640 1 --2- 172.21.15.100:0/709352517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd78107690 0x7fdd78107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:48.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.360+0000 7fdd7f429640 1 --2- 172.21.15.100:0/709352517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd78102b30 0x7fdd78107150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:48.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.360+0000 7fdd7f429640 1 --2- 172.21.15.100:0/709352517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd78102100 0x7fdd78102510 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:48.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.360+0000 7fdd7f429640 1 -- 172.21.15.100:0/709352517 >> 172.21.15.100:0/709352517 conn(0x7fdd780792f0 msgr2=0x7fdd78079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:48.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.360+0000 7fdd7f429640 1 -- 172.21.15.100:0/709352517 shutdown_connections 2024-07-26T13:37:48.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.360+0000 7fdd7f429640 1 -- 172.21.15.100:0/709352517 wait complete. 2024-07-26T13:37:48.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.360+0000 7fdd7f429640 1 Processor -- start 2024-07-26T13:37:48.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.361+0000 7fdd7f429640 1 -- start start 2024-07-26T13:37:48.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.361+0000 7fdd7f429640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd78102100 0x7fdd781a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:48.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.361+0000 7fdd7f429640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd78102b30 0x7fdd781a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:48.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.361+0000 7fdd7f429640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd78107690 0x7fdd781a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:48.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.361+0000 7fdd7f429640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdd78119af0 con 0x7fdd78107690 2024-07-26T13:37:48.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.361+0000 7fdd7f429640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdd78119970 con 0x7fdd78102100 2024-07-26T13:37:48.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.361+0000 7fdd7f429640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdd78119c70 con 0x7fdd78102b30 2024-07-26T13:37:48.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.361+0000 7fdd7d9c6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd78107690 0x7fdd781a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:48.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.361+0000 7fdd7d9c6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd78107690 0x7fdd781a5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55054/0 (socket says 172.21.15.100:55054) 2024-07-26T13:37:48.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.361+0000 7fdd7d9c6640 1 -- 172.21.15.100:0/804699673 learned_addr learned my addr 172.21.15.100:0/804699673 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:37:48.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.361+0000 7fdd7d1c5640 1 --2- 172.21.15.100:0/804699673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd78102100 0x7fdd781a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:48.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.361+0000 7fdd7c9c4640 1 --2- 172.21.15.100:0/804699673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd78102b30 0x7fdd781a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:48.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.361+0000 7fdd7d9c6640 1 -- 172.21.15.100:0/804699673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd78102b30 msgr2=0x7fdd781a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:48.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.361+0000 7fdd7d9c6640 1 --2- 172.21.15.100:0/804699673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd78102b30 0x7fdd781a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:48.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.361+0000 7fdd7d9c6640 1 -- 172.21.15.100:0/804699673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd78102100 msgr2=0x7fdd781a04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:48.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.361+0000 7fdd7d9c6640 1 --2- 172.21.15.100:0/804699673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd78102100 0x7fdd781a04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:48.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.361+0000 7fdd7d9c6640 1 -- 172.21.15.100:0/804699673 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd781a6410 con 0x7fdd78107690 2024-07-26T13:37:48.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.361+0000 7fdd7d9c6640 1 --2- 172.21.15.100:0/804699673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd78107690 0x7fdd781a5ce0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000eaa0 tx=0x7fdd7000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:48.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.362+0000 7fdd6e7fc640 1 -- 172.21.15.100:0/804699673 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd7000cdb0 con 0x7fdd78107690 2024-07-26T13:37:48.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.362+0000 7fdd6e7fc640 1 -- 172.21.15.100:0/804699673 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd70005020 con 0x7fdd78107690 2024-07-26T13:37:48.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.362+0000 7fdd6e7fc640 1 -- 172.21.15.100:0/804699673 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd700116d0 con 0x7fdd78107690 2024-07-26T13:37:48.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.362+0000 7fdd7f429640 1 -- 172.21.15.100:0/804699673 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd781a6700 con 0x7fdd78107690 2024-07-26T13:37:48.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.362+0000 7fdd7f429640 1 -- 172.21.15.100:0/804699673 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd781ace60 con 0x7fdd78107690 2024-07-26T13:37:48.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.363+0000 7fdd7f429640 1 -- 172.21.15.100:0/804699673 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd7806bb50 con 0x7fdd78107690 2024-07-26T13:37:48.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.364+0000 7fdd6e7fc640 1 -- 172.21.15.100:0/804699673 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7fdd700051d0 con 0x7fdd78107690 2024-07-26T13:37:48.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.365+0000 7fdd6e7fc640 1 --2- 172.21.15.100:0/804699673 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fdd640b3d30 0x7fdd640b6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:48.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.365+0000 7fdd6e7fc640 1 -- 172.21.15.100:0/804699673 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fdd700e4310 con 0x7fdd78107690 2024-07-26T13:37:48.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.365+0000 7fdd7d1c5640 1 --2- 172.21.15.100:0/804699673 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fdd640b3d30 0x7fdd640b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:48.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.366+0000 7fdd7d1c5640 1 --2- 172.21.15.100:0/804699673 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fdd640b3d30 0x7fdd640b6240 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdd74031440 tx=0x7fdd74009560 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:48.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.368+0000 7fdd6e7fc640 1 -- 172.21.15.100:0/804699673 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fdd70016030 con 0x7fdd78107690 2024-07-26T13:37:48.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.607+0000 7fdd7f429640 1 -- 172.21.15.100:0/804699673 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fdd781a15d0 con 0x7fdd78107690 2024-07-26T13:37:48.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.610+0000 7fdd6e7fc640 1 -- 172.21.15.100:0/804699673 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fdd700ab130 con 0x7fdd78107690 2024-07-26T13:37:48.612 INFO:teuthology.orchestra.run.smithi100.stdout:[client.0] 2024-07-26T13:37:48.612 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCspqNmuzFVJBAAY7dcB4tbQlZD5cju4v31Hw== 2024-07-26T13:37:48.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.612+0000 7fdd7f429640 1 -- 172.21.15.100:0/804699673 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fdd640b3d30 msgr2=0x7fdd640b6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:48.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.612+0000 7fdd7f429640 1 --2- 172.21.15.100:0/804699673 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fdd640b3d30 0x7fdd640b6240 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdd74031440 tx=0x7fdd74009560 comp rx=0 tx=0).stop 2024-07-26T13:37:48.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.612+0000 7fdd7f429640 1 -- 172.21.15.100:0/804699673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd78107690 msgr2=0x7fdd781a5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:48.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.612+0000 7fdd7f429640 1 --2- 172.21.15.100:0/804699673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd78107690 0x7fdd781a5ce0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000eaa0 tx=0x7fdd7000ef60 comp rx=0 tx=0).stop 2024-07-26T13:37:48.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.612+0000 7fdd7f429640 1 -- 172.21.15.100:0/804699673 shutdown_connections 2024-07-26T13:37:48.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.612+0000 7fdd7f429640 1 --2- 172.21.15.100:0/804699673 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fdd640b3d30 0x7fdd640b6240 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:48.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.612+0000 7fdd7f429640 1 --2- 172.21.15.100:0/804699673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd78107690 0x7fdd781a5ce0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:48.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.612+0000 7fdd7f429640 1 --2- 172.21.15.100:0/804699673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd78102b30 0x7fdd781a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:48.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.612+0000 7fdd7f429640 1 --2- 172.21.15.100:0/804699673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd78102100 0x7fdd781a04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:48.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.612+0000 7fdd7f429640 1 -- 172.21.15.100:0/804699673 >> 172.21.15.100:0/804699673 conn(0x7fdd780792f0 msgr2=0x7fdd781050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:48.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.612+0000 7fdd7f429640 1 -- 172.21.15.100:0/804699673 shutdown_connections 2024-07-26T13:37:48.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:48.612+0000 7fdd7f429640 1 -- 172.21.15.100:0/804699673 wait complete. 2024-07-26T13:37:49.246 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-07-26T13:37:49.246 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-07-26T13:37:49.246 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-07-26T13:37:49.284 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph config log 1 --format=json 2024-07-26T13:37:49.583 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:37:49.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:49 smithi100 ceph-mon[32599]: pgmap v295: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-07-26T13:37:49.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:49 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/804699673' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-07-26T13:37:49.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:49 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/804699673' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-07-26T13:37:49.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:49 smithi167 ceph-mon[37140]: pgmap v295: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-07-26T13:37:49.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:49 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/804699673' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-07-26T13:37:49.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:49 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/804699673' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-07-26T13:37:49.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:49 smithi138 ceph-mon[36954]: pgmap v295: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-07-26T13:37:49.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:49 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/804699673' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-07-26T13:37:49.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:49 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/804699673' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-07-26T13:37:50.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:37:50.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:37:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:37:51.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:51 smithi100 ceph-mon[32599]: pgmap v296: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-07-26T13:37:51.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:51 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:37:51.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:51 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:37:51.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:51 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:37:51.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:51 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:37:51.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:51 smithi167 ceph-mon[37140]: pgmap v296: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-07-26T13:37:51.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:51 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:37:51.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:51 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:37:51.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:51 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:37:51.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:51 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:37:51.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.891+0000 7f77b3311640 1 -- 172.21.15.100:0/3156430804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77ac104be0 msgr2=0x7f77ac104ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:51.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.891+0000 7f77b3311640 1 --2- 172.21.15.100:0/3156430804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77ac104be0 0x7f77ac104ff0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f77a8009900 tx=0x7f77a802a410 comp rx=0 tx=0).stop 2024-07-26T13:37:51.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.891+0000 7f77b3311640 1 -- 172.21.15.100:0/3156430804 shutdown_connections 2024-07-26T13:37:51.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.891+0000 7f77b3311640 1 --2- 172.21.15.100:0/3156430804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77ac105530 0x7f77ac10efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:51.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.891+0000 7f77b3311640 1 --2- 172.21.15.100:0/3156430804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77ac104be0 0x7f77ac104ff0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:51.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.891+0000 7f77b3311640 1 --2- 172.21.15.100:0/3156430804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77ac102560 0x7f77ac102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:51.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.891+0000 7f77b3311640 1 -- 172.21.15.100:0/3156430804 >> 172.21.15.100:0/3156430804 conn(0x7f77ac0793d0 msgr2=0x7f77ac0797e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:51.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.891+0000 7f77b3311640 1 -- 172.21.15.100:0/3156430804 shutdown_connections 2024-07-26T13:37:51.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.891+0000 7f77b3311640 1 -- 172.21.15.100:0/3156430804 wait complete. 2024-07-26T13:37:51.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.892+0000 7f77b3311640 1 Processor -- start 2024-07-26T13:37:51.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.892+0000 7f77b3311640 1 -- start start 2024-07-26T13:37:51.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.892+0000 7f77b3311640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77ac102560 0x7f77ac199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:51.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.892+0000 7f77b3311640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77ac104be0 0x7f77ac19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:51.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.892+0000 7f77b3311640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77ac105530 0x7f77ac19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:51.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.892+0000 7f77b3311640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac112400 con 0x7f77ac105530 2024-07-26T13:37:51.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.892+0000 7f77b3311640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac112280 con 0x7f77ac102560 2024-07-26T13:37:51.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.892+0000 7f77b3311640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac112580 con 0x7f77ac104be0 2024-07-26T13:37:51.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.892+0000 7f77b18ae640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77ac105530 0x7f77ac19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:51.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.892+0000 7f77b18ae640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77ac105530 0x7f77ac19f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55070/0 (socket says 172.21.15.100:55070) 2024-07-26T13:37:51.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.892+0000 7f77b18ae640 1 -- 172.21.15.100:0/2858492369 learned_addr learned my addr 172.21.15.100:0/2858492369 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:37:51.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.892+0000 7f77b10ad640 1 --2- 172.21.15.100:0/2858492369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77ac102560 0x7f77ac199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:51.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.893+0000 7f77b08ac640 1 --2- 172.21.15.100:0/2858492369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77ac104be0 0x7f77ac19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:51.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.893+0000 7f77b18ae640 1 -- 172.21.15.100:0/2858492369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77ac104be0 msgr2=0x7f77ac19a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:51.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.893+0000 7f77b18ae640 1 --2- 172.21.15.100:0/2858492369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77ac104be0 0x7f77ac19a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:51.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.893+0000 7f77b18ae640 1 -- 172.21.15.100:0/2858492369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77ac102560 msgr2=0x7f77ac199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:51.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.893+0000 7f77b18ae640 1 --2- 172.21.15.100:0/2858492369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77ac102560 0x7f77ac199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:51.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.893+0000 7f77b18ae640 1 -- 172.21.15.100:0/2858492369 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77ac19feb0 con 0x7f77ac105530 2024-07-26T13:37:51.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.893+0000 7f77b18ae640 1 --2- 172.21.15.100:0/2858492369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77ac105530 0x7f77ac19f780 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f77a400eb10 tx=0x7f77a400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:51.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.893+0000 7f779e7fc640 1 -- 172.21.15.100:0/2858492369 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a4004b20 con 0x7f77ac105530 2024-07-26T13:37:51.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.893+0000 7f77b3311640 1 -- 172.21.15.100:0/2858492369 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77ac1a01a0 con 0x7f77ac105530 2024-07-26T13:37:51.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.893+0000 7f779e7fc640 1 -- 172.21.15.100:0/2858492369 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77a4004cc0 con 0x7f77ac105530 2024-07-26T13:37:51.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.893+0000 7f779e7fc640 1 -- 172.21.15.100:0/2858492369 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a4010690 con 0x7f77ac105530 2024-07-26T13:37:51.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.893+0000 7f77b3311640 1 -- 172.21.15.100:0/2858492369 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77ac1a6900 con 0x7f77ac105530 2024-07-26T13:37:51.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.894+0000 7f77b3311640 1 -- 172.21.15.100:0/2858492369 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77ac102970 con 0x7f77ac105530 2024-07-26T13:37:51.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.895+0000 7f779e7fc640 1 -- 172.21.15.100:0/2858492369 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f77a4005020 con 0x7f77ac105530 2024-07-26T13:37:51.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.896+0000 7f779e7fc640 1 --2- 172.21.15.100:0/2858492369 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f77980b3cb0 0x7f77980b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:51.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.896+0000 7f77b10ad640 1 --2- 172.21.15.100:0/2858492369 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f77980b3cb0 0x7f77980b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:51.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.896+0000 7f779e7fc640 1 -- 172.21.15.100:0/2858492369 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f77a40e46b0 con 0x7f77ac105530 2024-07-26T13:37:51.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.897+0000 7f77b10ad640 1 --2- 172.21.15.100:0/2858492369 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f77980b3cb0 0x7f77980b61c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f77a0005d00 tx=0x7f77a000b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:51.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:51.899+0000 7f779e7fc640 1 -- 172.21.15.100:0/2858492369 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f77a4055600 con 0x7f77ac105530 2024-07-26T13:37:51.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:51 smithi138 ceph-mon[36954]: pgmap v296: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-07-26T13:37:51.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:51 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:37:51.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:51 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:37:51.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:51 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:37:51.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:51 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:37:52.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:52.096+0000 7f77b3311640 1 -- 172.21.15.100:0/2858492369 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f77ac1a6b90 con 0x7f77ac105530 2024-07-26T13:37:52.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:52.097+0000 7f779e7fc640 1 -- 172.21.15.100:0/2858492369 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v55) ==== 86+0+133 (secure 0 0 0) 0x7f77a40abaa0 con 0x7f77ac105530 2024-07-26T13:37:52.099 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:37:52.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:52.099+0000 7f77b3311640 1 -- 172.21.15.100:0/2858492369 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f77980b3cb0 msgr2=0x7f77980b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:52.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:52.099+0000 7f77b3311640 1 --2- 172.21.15.100:0/2858492369 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f77980b3cb0 0x7f77980b61c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f77a0005d00 tx=0x7f77a000b410 comp rx=0 tx=0).stop 2024-07-26T13:37:52.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:52.099+0000 7f77b3311640 1 -- 172.21.15.100:0/2858492369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77ac105530 msgr2=0x7f77ac19f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:52.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:52.099+0000 7f77b3311640 1 --2- 172.21.15.100:0/2858492369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77ac105530 0x7f77ac19f780 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f77a400eb10 tx=0x7f77a400c2d0 comp rx=0 tx=0).stop 2024-07-26T13:37:52.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:52.099+0000 7f77b3311640 1 -- 172.21.15.100:0/2858492369 shutdown_connections 2024-07-26T13:37:52.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:52.099+0000 7f77b3311640 1 --2- 172.21.15.100:0/2858492369 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f77980b3cb0 0x7f77980b61c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:52.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:52.099+0000 7f77b3311640 1 --2- 172.21.15.100:0/2858492369 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77ac105530 0x7f77ac19f780 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:52.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:52.099+0000 7f77b3311640 1 --2- 172.21.15.100:0/2858492369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77ac104be0 0x7f77ac19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:52.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:52.099+0000 7f77b3311640 1 --2- 172.21.15.100:0/2858492369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77ac102560 0x7f77ac199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:52.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:52.099+0000 7f77b3311640 1 -- 172.21.15.100:0/2858492369 >> 172.21.15.100:0/2858492369 conn(0x7f77ac0793d0 msgr2=0x7f77ac101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:52.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:52.099+0000 7f77b3311640 1 -- 172.21.15.100:0/2858492369 shutdown_connections 2024-07-26T13:37:52.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:52.099+0000 7f77b3311640 1 -- 172.21.15.100:0/2858492369 wait complete. 2024-07-26T13:37:52.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:52 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2858492369' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2024-07-26T13:37:52.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:52 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/2858492369' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2024-07-26T13:37:52.849 INFO:teuthology.orchestra.run.smithi100.stdout:[{"version":55,"timestamp":"2024-07-26T13:36:44.338467+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2024-07-26T13:37:52.849 INFO:tasks.ceph_manager:config epoch is 55 2024-07-26T13:37:52.849 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-07-26T13:37:52.850 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-07-26T13:37:52.850 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph mgr dump --format=json 2024-07-26T13:37:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:52 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/2858492369' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2024-07-26T13:37:53.127 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:37:53.515 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:53 smithi100 ceph-mon[32599]: pgmap v297: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2024-07-26T13:37:53.515 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:53 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:37:53.515 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:53 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:37:53.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:53 smithi167 ceph-mon[37140]: pgmap v297: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2024-07-26T13:37:53.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:53 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:37:53.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:53 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:37:53.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:53 smithi138 ceph-mon[36954]: pgmap v297: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2024-07-26T13:37:53.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:53 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:37:53.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:53 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:37:54.598 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:37:54.598 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:37:54.598 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:37:54.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:37:54.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:37:54.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:37:54.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.812+0000 7f0327b24640 1 -- 172.21.15.100:0/2461636994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03201043e0 msgr2=0x7f03201047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:54.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.812+0000 7f0327b24640 1 --2- 172.21.15.100:0/2461636994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03201043e0 0x7f03201047f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0310009ca0 tx=0x7f031002a8c0 comp rx=0 tx=0).stop 2024-07-26T13:37:54.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.812+0000 7f0327b24640 1 -- 172.21.15.100:0/2461636994 shutdown_connections 2024-07-26T13:37:54.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.812+0000 7f0327b24640 1 --2- 172.21.15.100:0/2461636994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f032010d840 0x7f032010fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:54.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.812+0000 7f0327b24640 1 --2- 172.21.15.100:0/2461636994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0320104d30 0x7f032010d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:54.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.812+0000 7f0327b24640 1 --2- 172.21.15.100:0/2461636994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03201043e0 0x7f03201047f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:54.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.812+0000 7f0327b24640 1 -- 172.21.15.100:0/2461636994 >> 172.21.15.100:0/2461636994 conn(0x7f0320079160 msgr2=0x7f0320079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:54.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.812+0000 7f0327b24640 1 -- 172.21.15.100:0/2461636994 shutdown_connections 2024-07-26T13:37:54.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.812+0000 7f0327b24640 1 -- 172.21.15.100:0/2461636994 wait complete. 2024-07-26T13:37:54.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.813+0000 7f0327b24640 1 Processor -- start 2024-07-26T13:37:54.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.813+0000 7f0327b24640 1 -- start start 2024-07-26T13:37:54.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.813+0000 7f0327b24640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03201043e0 0x7f0320199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:54.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.813+0000 7f0327b24640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0320104d30 0x7f032019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:54.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.813+0000 7f0327b24640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f032010d840 0x7f032019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:54.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.813+0000 7f0327b24640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f03201134b0 con 0x7f0320104d30 2024-07-26T13:37:54.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.813+0000 7f0327b24640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0320113330 con 0x7f032010d840 2024-07-26T13:37:54.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.813+0000 7f0327b24640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0320113630 con 0x7f03201043e0 2024-07-26T13:37:54.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.813+0000 7f03250bf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0320104d30 0x7f032019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:54.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.813+0000 7f03250bf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0320104d30 0x7f032019a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55094/0 (socket says 172.21.15.100:55094) 2024-07-26T13:37:54.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.813+0000 7f03250bf640 1 -- 172.21.15.100:0/955525316 learned_addr learned my addr 172.21.15.100:0/955525316 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:37:54.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.814+0000 7f03258c0640 1 --2- 172.21.15.100:0/955525316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03201043e0 0x7f0320199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:54.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.814+0000 7f03260c1640 1 --2- 172.21.15.100:0/955525316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f032010d840 0x7f032019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:54.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.814+0000 7f03250bf640 1 -- 172.21.15.100:0/955525316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03201043e0 msgr2=0x7f0320199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:54.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.814+0000 7f03250bf640 1 --2- 172.21.15.100:0/955525316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03201043e0 0x7f0320199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:54.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.814+0000 7f03250bf640 1 -- 172.21.15.100:0/955525316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f032010d840 msgr2=0x7f032019f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:54.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.814+0000 7f03250bf640 1 --2- 172.21.15.100:0/955525316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f032010d840 0x7f032019f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:54.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.814+0000 7f03250bf640 1 -- 172.21.15.100:0/955525316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f032019fef0 con 0x7f0320104d30 2024-07-26T13:37:54.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.814+0000 7f03250bf640 1 --2- 172.21.15.100:0/955525316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0320104d30 0x7f032019a520 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f031400d740 tx=0x7f031400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:54.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.814+0000 7f031effd640 1 -- 172.21.15.100:0/955525316 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0314017070 con 0x7f0320104d30 2024-07-26T13:37:54.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.814+0000 7f0327b24640 1 -- 172.21.15.100:0/955525316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03201a01e0 con 0x7f0320104d30 2024-07-26T13:37:54.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.814+0000 7f031effd640 1 -- 172.21.15.100:0/955525316 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f031400de50 con 0x7f0320104d30 2024-07-26T13:37:54.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.814+0000 7f031effd640 1 -- 172.21.15.100:0/955525316 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031400f620 con 0x7f0320104d30 2024-07-26T13:37:54.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.814+0000 7f0327b24640 1 -- 172.21.15.100:0/955525316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03201a6940 con 0x7f0320104d30 2024-07-26T13:37:54.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.815+0000 7f0327b24640 1 -- 172.21.15.100:0/955525316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03201047f0 con 0x7f0320104d30 2024-07-26T13:37:54.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.817+0000 7f031effd640 1 -- 172.21.15.100:0/955525316 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f031401a070 con 0x7f0320104d30 2024-07-26T13:37:54.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.818+0000 7f031effd640 1 --2- 172.21.15.100:0/955525316 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f03080b3cb0 0x7f03080b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:54.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.818+0000 7f03258c0640 1 --2- 172.21.15.100:0/955525316 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f03080b3cb0 0x7f03080b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:54.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.818+0000 7f031effd640 1 -- 172.21.15.100:0/955525316 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f03140e38e0 con 0x7f0320104d30 2024-07-26T13:37:54.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.818+0000 7f03258c0640 1 --2- 172.21.15.100:0/955525316 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f03080b3cb0 0x7f03080b61c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0310005e30 tx=0x7f0310009360 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:54.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:54.821+0000 7f031effd640 1 -- 172.21.15.100:0/955525316 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f03140aacd0 con 0x7f0320104d30 2024-07-26T13:37:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:37:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:37:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:37:55.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:55.039+0000 7f0327b24640 1 -- 172.21.15.100:0/955525316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f032002e0c0 con 0x7f0320104d30 2024-07-26T13:37:55.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:55.044+0000 7f031effd640 1 -- 172.21.15.100:0/955525316 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v21) ==== 74+0+291045 (secure 0 0 0) 0x7f03140aa5d0 con 0x7f0320104d30 2024-07-26T13:37:55.048 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:37:55.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:55.048+0000 7f0327b24640 1 -- 172.21.15.100:0/955525316 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f03080b3cb0 msgr2=0x7f03080b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:55.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:55.048+0000 7f0327b24640 1 --2- 172.21.15.100:0/955525316 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f03080b3cb0 0x7f03080b61c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0310005e30 tx=0x7f0310009360 comp rx=0 tx=0).stop 2024-07-26T13:37:55.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:55.048+0000 7f0327b24640 1 -- 172.21.15.100:0/955525316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0320104d30 msgr2=0x7f032019a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:55.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:55.048+0000 7f0327b24640 1 --2- 172.21.15.100:0/955525316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0320104d30 0x7f032019a520 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f031400d740 tx=0x7f031400dc00 comp rx=0 tx=0).stop 2024-07-26T13:37:55.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:55.048+0000 7f0327b24640 1 -- 172.21.15.100:0/955525316 shutdown_connections 2024-07-26T13:37:55.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:55.048+0000 7f0327b24640 1 --2- 172.21.15.100:0/955525316 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f03080b3cb0 0x7f03080b61c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:55.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:55.048+0000 7f0327b24640 1 --2- 172.21.15.100:0/955525316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f032010d840 0x7f032019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:55.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:55.048+0000 7f0327b24640 1 --2- 172.21.15.100:0/955525316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0320104d30 0x7f032019a520 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:55.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:55.048+0000 7f0327b24640 1 --2- 172.21.15.100:0/955525316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03201043e0 0x7f0320199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:55.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:55.048+0000 7f0327b24640 1 -- 172.21.15.100:0/955525316 >> 172.21.15.100:0/955525316 conn(0x7f0320079160 msgr2=0x7f03201002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:55.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:55.048+0000 7f0327b24640 1 -- 172.21.15.100:0/955525316 shutdown_connections 2024-07-26T13:37:55.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:55.048+0000 7f0327b24640 1 -- 172.21.15.100:0/955525316 wait complete. 2024-07-26T13:37:55.721 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:55 smithi100 ceph-mon[32599]: pgmap v298: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2024-07-26T13:37:55.721 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:55 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/955525316' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-07-26T13:37:55.749 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":21,"flags":0,"active_gid":14214,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3937009534},{"type":"v1","addr":"172.21.15.100:6801","nonce":3937009534}]},"active_addr":"172.21.15.100:6801/3937009534","active_change":"2024-07-26T13:30:16.253258+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24146,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24148,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.100:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":1169745306}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":3698016472}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":2649239615}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":3541637056}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":2715372015}]}]} 2024-07-26T13:37:55.755 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-07-26T13:37:55.755 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-07-26T13:37:55.755 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph osd dump --format=json 2024-07-26T13:37:55.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:55 smithi167 ceph-mon[37140]: pgmap v298: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2024-07-26T13:37:55.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:55 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/955525316' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-07-26T13:37:55.841 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:55 smithi138 ceph-mon[36954]: pgmap v298: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2024-07-26T13:37:55.842 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:55 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/955525316' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-07-26T13:37:56.011 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:37:57.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:57 smithi100 ceph-mon[32599]: pgmap v299: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T13:37:57.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:57 smithi167 ceph-mon[37140]: pgmap v299: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T13:37:57.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.730+0000 7f9db701c640 1 -- 172.21.15.100:0/1465797533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9db0103590 msgr2=0x7f9db01039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:57.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.730+0000 7f9db701c640 1 --2- 172.21.15.100:0/1465797533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9db0103590 0x7f9db01039a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9da0009900 tx=0x7f9da002a410 comp rx=0 tx=0).stop 2024-07-26T13:37:57.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.730+0000 7f9db701c640 1 -- 172.21.15.100:0/1465797533 shutdown_connections 2024-07-26T13:37:57.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.730+0000 7f9db701c640 1 --2- 172.21.15.100:0/1465797533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9db0109820 0x7f9db0110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:57.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.730+0000 7f9db701c640 1 --2- 172.21.15.100:0/1465797533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9db0108ed0 0x7f9db01092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:57.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.730+0000 7f9db701c640 1 --2- 172.21.15.100:0/1465797533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9db0103590 0x7f9db01039a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:57.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.730+0000 7f9db701c640 1 -- 172.21.15.100:0/1465797533 >> 172.21.15.100:0/1465797533 conn(0x7f9db007b280 msgr2=0x7f9db007b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:57.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.730+0000 7f9db701c640 1 -- 172.21.15.100:0/1465797533 shutdown_connections 2024-07-26T13:37:57.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.730+0000 7f9db701c640 1 -- 172.21.15.100:0/1465797533 wait complete. 2024-07-26T13:37:57.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.731+0000 7f9db701c640 1 Processor -- start 2024-07-26T13:37:57.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.731+0000 7f9db701c640 1 -- start start 2024-07-26T13:37:57.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.731+0000 7f9db701c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9db0103590 0x7f9db0199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:57.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.731+0000 7f9db701c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9db0108ed0 0x7f9db019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:57.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.731+0000 7f9db701c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9db0109820 0x7f9db019f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:57.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.731+0000 7f9db701c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9db01134d0 con 0x7f9db0103590 2024-07-26T13:37:57.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.731+0000 7f9db701c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9db0113350 con 0x7f9db0108ed0 2024-07-26T13:37:57.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.731+0000 7f9db701c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9db0113650 con 0x7f9db0109820 2024-07-26T13:37:57.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.731+0000 7f9db4db8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9db0103590 0x7f9db0199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:57.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.731+0000 7f9db4db8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9db0103590 0x7f9db0199ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47482/0 (socket says 172.21.15.100:47482) 2024-07-26T13:37:57.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.731+0000 7f9db4db8640 1 -- 172.21.15.100:0/1993581846 learned_addr learned my addr 172.21.15.100:0/1993581846 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:37:57.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.732+0000 7f9daffff640 1 --2- 172.21.15.100:0/1993581846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9db0108ed0 0x7f9db019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:57.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.732+0000 7f9db55b9640 1 --2- 172.21.15.100:0/1993581846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9db0109820 0x7f9db019f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:57.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.732+0000 7f9db4db8640 1 -- 172.21.15.100:0/1993581846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9db0109820 msgr2=0x7f9db019f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:57.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.732+0000 7f9db4db8640 1 --2- 172.21.15.100:0/1993581846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9db0109820 0x7f9db019f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:57.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.732+0000 7f9db4db8640 1 -- 172.21.15.100:0/1993581846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9db0108ed0 msgr2=0x7f9db019a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:57.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.732+0000 7f9db4db8640 1 --2- 172.21.15.100:0/1993581846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9db0108ed0 0x7f9db019a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:57.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.732+0000 7f9db4db8640 1 -- 172.21.15.100:0/1993581846 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db019fe30 con 0x7f9db0103590 2024-07-26T13:37:57.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.732+0000 7f9db55b9640 1 --2- 172.21.15.100:0/1993581846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9db0109820 0x7f9db019f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:37:57.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.732+0000 7f9db4db8640 1 --2- 172.21.15.100:0/1993581846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9db0103590 0x7f9db0199ec0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9da0005940 tx=0x7f9da00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:57.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.732+0000 7f9dadffb640 1 -- 172.21.15.100:0/1993581846 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da003f030 con 0x7f9db0103590 2024-07-26T13:37:57.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.732+0000 7f9db701c640 1 -- 172.21.15.100:0/1993581846 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9db01a00c0 con 0x7f9db0103590 2024-07-26T13:37:57.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.732+0000 7f9dadffb640 1 -- 172.21.15.100:0/1993581846 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9da0003cb0 con 0x7f9db0103590 2024-07-26T13:37:57.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.732+0000 7f9dadffb640 1 -- 172.21.15.100:0/1993581846 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da0035070 con 0x7f9db0103590 2024-07-26T13:37:57.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.732+0000 7f9db701c640 1 -- 172.21.15.100:0/1993581846 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9db01a6880 con 0x7f9db0103590 2024-07-26T13:37:57.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.733+0000 7f9db701c640 1 -- 172.21.15.100:0/1993581846 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9db01039a0 con 0x7f9db0103590 2024-07-26T13:37:57.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.735+0000 7f9dadffb640 1 -- 172.21.15.100:0/1993581846 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f9da0042030 con 0x7f9db0103590 2024-07-26T13:37:57.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.735+0000 7f9dadffb640 1 --2- 172.21.15.100:0/1993581846 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9d980b3cb0 0x7f9d980b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:37:57.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.735+0000 7f9daffff640 1 --2- 172.21.15.100:0/1993581846 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9d980b3cb0 0x7f9d980b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:37:57.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.735+0000 7f9dadffb640 1 -- 172.21.15.100:0/1993581846 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f9da0104a70 con 0x7f9db0103590 2024-07-26T13:37:57.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.736+0000 7f9daffff640 1 --2- 172.21.15.100:0/1993581846 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9d980b3cb0 0x7f9d980b61c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9db0105f70 tx=0x7f9da400a4c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:37:57.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.738+0000 7f9dadffb640 1 -- 172.21.15.100:0/1993581846 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f9da00cbe60 con 0x7f9db0103590 2024-07-26T13:37:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.912+0000 7f9db701c640 1 -- 172.21.15.100:0/1993581846 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9db010b600 con 0x7f9db0103590 2024-07-26T13:37:57.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.914+0000 7f9dadffb640 1 -- 172.21.15.100:0/1993581846 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+23121 (secure 0 0 0) 0x7f9da00cb760 con 0x7f9db0103590 2024-07-26T13:37:57.915 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:37:57.915 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":81,"fsid":"bf83e2e6-4b52-11ef-bcb7-c7b262605968","created":"2024-07-26T13:28:23.639785+0000","modified":"2024-07-26T13:37:17.539521+0000","last_up_change":"2024-07-26T13:35:56.475554+0000","last_in_change":"2024-07-26T13:35:37.472564+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-26T13:32:22.329969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-26T13:36:54.590525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-26T13:36:58.149545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d714036d-9dda-4cb5-af16-29f83a09dd70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6803","nonce":1480401850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6805","nonce":1480401850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6809","nonce":1480401850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6807","nonce":1480401850}]},"public_addr":"172.21.15.100:6803/1480401850","cluster_addr":"172.21.15.100:6805/1480401850","heartbeat_back_addr":"172.21.15.100:6809/1480401850","heartbeat_front_addr":"172.21.15.100:6807/1480401850","state":["exists","up"]},{"osd":1,"uuid":"3dd0eaa1-618a-495a-9819-e01c0f869ae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6801","nonce":3090877671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6803","nonce":3090877671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6807","nonce":3090877671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6805","nonce":3090877671}]},"public_addr":"172.21.15.138:6801/3090877671","cluster_addr":"172.21.15.138:6803/3090877671","heartbeat_back_addr":"172.21.15.138:6807/3090877671","heartbeat_front_addr":"172.21.15.138:6805/3090877671","state":["exists","up"]},{"osd":2,"uuid":"5c710a78-17ae-41a8-ac16-a678f5d35672","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6801","nonce":3583424790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6803","nonce":3583424790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6807","nonce":3583424790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6805","nonce":3583424790}]},"public_addr":"172.21.15.167:6801/3583424790","cluster_addr":"172.21.15.167:6803/3583424790","heartbeat_back_addr":"172.21.15.167:6807/3583424790","heartbeat_front_addr":"172.21.15.167:6805/3583424790","state":["exists","up"]},{"osd":3,"uuid":"be845495-8c31-4834-a45c-bdc576711962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6811","nonce":3626762419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6813","nonce":3626762419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6817","nonce":3626762419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6815","nonce":3626762419}]},"public_addr":"172.21.15.100:6811/3626762419","cluster_addr":"172.21.15.100:6813/3626762419","heartbeat_back_addr":"172.21.15.100:6817/3626762419","heartbeat_front_addr":"172.21.15.100:6815/3626762419","state":["exists","up"]},{"osd":4,"uuid":"520554cc-e846-46f9-af77-05a1de65a452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6809","nonce":3757334356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6811","nonce":3757334356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6815","nonce":3757334356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6813","nonce":3757334356}]},"public_addr":"172.21.15.138:6809/3757334356","cluster_addr":"172.21.15.138:6811/3757334356","heartbeat_back_addr":"172.21.15.138:6815/3757334356","heartbeat_front_addr":"172.21.15.138:6813/3757334356","state":["exists","up"]},{"osd":5,"uuid":"68dc8ad6-31d2-45b5-9aa7-62c4218d46e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6809","nonce":3239482836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6811","nonce":3239482836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6815","nonce":3239482836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6813","nonce":3239482836}]},"public_addr":"172.21.15.167:6809/3239482836","cluster_addr":"172.21.15.167:6811/3239482836","heartbeat_back_addr":"172.21.15.167:6815/3239482836","heartbeat_front_addr":"172.21.15.167:6813/3239482836","state":["exists","up"]},{"osd":6,"uuid":"420e4471-5626-41cc-a8e1-69c801196b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6819","nonce":3218415484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6821","nonce":3218415484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6825","nonce":3218415484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6823","nonce":3218415484}]},"public_addr":"172.21.15.100:6819/3218415484","cluster_addr":"172.21.15.100:6821/3218415484","heartbeat_back_addr":"172.21.15.100:6825/3218415484","heartbeat_front_addr":"172.21.15.100:6823/3218415484","state":["exists","up"]},{"osd":7,"uuid":"a27b097e-207d-44c3-af63-544f228b83ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6817","nonce":45541749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6819","nonce":45541749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6823","nonce":45541749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6821","nonce":45541749}]},"public_addr":"172.21.15.138:6817/45541749","cluster_addr":"172.21.15.138:6819/45541749","heartbeat_back_addr":"172.21.15.138:6823/45541749","heartbeat_front_addr":"172.21.15.138:6821/45541749","state":["exists","up"]},{"osd":8,"uuid":"3e865858-92cc-48a0-a464-5767d19e8ac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6817","nonce":4031159803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6819","nonce":4031159803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6823","nonce":4031159803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6821","nonce":4031159803}]},"public_addr":"172.21.15.167:6817/4031159803","cluster_addr":"172.21.15.167:6819/4031159803","heartbeat_back_addr":"172.21.15.167:6823/4031159803","heartbeat_front_addr":"172.21.15.167:6821/4031159803","state":["exists","up"]},{"osd":9,"uuid":"8b05b14a-1195-42f5-ac01-d5634cb9c888","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6827","nonce":1840140273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6829","nonce":1840140273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6833","nonce":1840140273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6831","nonce":1840140273}]},"public_addr":"172.21.15.100:6827/1840140273","cluster_addr":"172.21.15.100:6829/1840140273","heartbeat_back_addr":"172.21.15.100:6833/1840140273","heartbeat_front_addr":"172.21.15.100:6831/1840140273","state":["exists","up"]},{"osd":10,"uuid":"ab77ebe7-5254-4dc3-bc54-625698e84afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6825","nonce":703382899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6827","nonce":703382899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6831","nonce":703382899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6829","nonce":703382899}]},"public_addr":"172.21.15.138:6825/703382899","cluster_addr":"172.21.15.138:6827/703382899","heartbeat_back_addr":"172.21.15.138:6831/703382899","heartbeat_front_addr":"172.21.15.138:6829/703382899","state":["exists","up"]},{"osd":11,"uuid":"229cf91e-e9ef-48f6-a677-c5c5705b315a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6825","nonce":1032343068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6827","nonce":1032343068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6831","nonce":1032343068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6829","nonce":1032343068}]},"public_addr":"172.21.15.167:6825/1032343068","cluster_addr":"172.21.15.167:6827/1032343068","heartbeat_back_addr":"172.21.15.167:6831/1032343068","heartbeat_front_addr":"172.21.15.167:6829/1032343068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:31:35.800702+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:31:58.087922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:32:20.463183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:32:42.920598+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:33:06.332655+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:33:29.381407+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:33:53.975963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:34:18.226813+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:34:41.845770+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:35:05.722275+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:35:30.862704+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:35:54.738697+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:0/1874993112":"2024-07-27T13:30:16.253096+0000","172.21.15.100:0/2364620305":"2024-07-27T13:28:45.340334+0000","172.21.15.100:6800/3096076075":"2024-07-27T13:29:07.979178+0000","172.21.15.100:0/1894966501":"2024-07-27T13:30:16.253096+0000","172.21.15.100:0/3411776672":"2024-07-27T13:28:45.340334+0000","172.21.15.100:0/2853423120":"2024-07-27T13:29:07.979178+0000","172.21.15.100:6800/3025873570":"2024-07-27T13:28:45.340334+0000","172.21.15.100:0/4887181":"2024-07-27T13:29:07.979178+0000","172.21.15.100:6801/908675140":"2024-07-27T13:30:16.253096+0000","172.21.15.100:6801/3025873570":"2024-07-27T13:28:45.340334+0000","172.21.15.100:6800/908675140":"2024-07-27T13:30:16.253096+0000","172.21.15.100:6801/3096076075":"2024-07-27T13:29:07.979178+0000","172.21.15.100:0/2302632052":"2024-07-27T13:29:07.979178+0000","172.21.15.100:0/3736098915":"2024-07-27T13:28:45.340334+0000","172.21.15.100:0/3243030560":"2024-07-27T13:30:16.253096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-26T13:37:57.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.916+0000 7f9db701c640 1 -- 172.21.15.100:0/1993581846 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9d980b3cb0 msgr2=0x7f9d980b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:57.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.916+0000 7f9db701c640 1 --2- 172.21.15.100:0/1993581846 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9d980b3cb0 0x7f9d980b61c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9db0105f70 tx=0x7f9da400a4c0 comp rx=0 tx=0).stop 2024-07-26T13:37:57.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.916+0000 7f9db701c640 1 -- 172.21.15.100:0/1993581846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9db0103590 msgr2=0x7f9db0199ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:37:57.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.916+0000 7f9db701c640 1 --2- 172.21.15.100:0/1993581846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9db0103590 0x7f9db0199ec0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9da0005940 tx=0x7f9da00042f0 comp rx=0 tx=0).stop 2024-07-26T13:37:57.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.916+0000 7f9db701c640 1 -- 172.21.15.100:0/1993581846 shutdown_connections 2024-07-26T13:37:57.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.916+0000 7f9db701c640 1 --2- 172.21.15.100:0/1993581846 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9d980b3cb0 0x7f9d980b61c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:57.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.916+0000 7f9db701c640 1 --2- 172.21.15.100:0/1993581846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9db0109820 0x7f9db019f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:57.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.916+0000 7f9db701c640 1 --2- 172.21.15.100:0/1993581846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9db0108ed0 0x7f9db019a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:57.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.916+0000 7f9db701c640 1 --2- 172.21.15.100:0/1993581846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9db0103590 0x7f9db0199ec0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:37:57.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.916+0000 7f9db701c640 1 -- 172.21.15.100:0/1993581846 >> 172.21.15.100:0/1993581846 conn(0x7f9db007b280 msgr2=0x7f9db0106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:37:57.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.916+0000 7f9db701c640 1 -- 172.21.15.100:0/1993581846 shutdown_connections 2024-07-26T13:37:57.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:37:57.916+0000 7f9db701c640 1 -- 172.21.15.100:0/1993581846 wait complete. 2024-07-26T13:37:57.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:57 smithi138 ceph-mon[36954]: pgmap v299: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T13:37:58.543 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-07-26T13:37:58.543 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph osd dump --format=json 2024-07-26T13:37:58.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:58 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/1993581846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-26T13:37:58.797 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:37:58.920 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:58 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/1993581846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-26T13:37:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:58 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/1993581846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-26T13:37:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:37:59 smithi167 ceph-mon[37140]: pgmap v300: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:37:59.823 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:37:59 smithi100 ceph-mon[32599]: pgmap v300: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:37:59.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:37:59 smithi138 ceph-mon[36954]: pgmap v300: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:38:00.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.341+0000 7f5df2095640 1 -- 172.21.15.100:0/2054623421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dec1055a0 msgr2=0x7f5dec10f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:00.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.341+0000 7f5df2095640 1 --2- 172.21.15.100:0/2054623421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dec1055a0 0x7f5dec10f030 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5de4009900 tx=0x7f5de402a3f0 comp rx=0 tx=0).stop 2024-07-26T13:38:00.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.341+0000 7f5df2095640 1 -- 172.21.15.100:0/2054623421 shutdown_connections 2024-07-26T13:38:00.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.341+0000 7f5df2095640 1 --2- 172.21.15.100:0/2054623421 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dec1055a0 0x7f5dec10f030 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:00.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.341+0000 7f5df2095640 1 --2- 172.21.15.100:0/2054623421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dec104c50 0x7f5dec105060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:00.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.341+0000 7f5df2095640 1 --2- 172.21.15.100:0/2054623421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dec102c80 0x7f5dec103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:00.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.341+0000 7f5df2095640 1 -- 172.21.15.100:0/2054623421 >> 172.21.15.100:0/2054623421 conn(0x7f5dec0798d0 msgr2=0x7f5dec079ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:00.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.341+0000 7f5df2095640 1 -- 172.21.15.100:0/2054623421 shutdown_connections 2024-07-26T13:38:00.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.341+0000 7f5df2095640 1 -- 172.21.15.100:0/2054623421 wait complete. 2024-07-26T13:38:00.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.341+0000 7f5df2095640 1 Processor -- start 2024-07-26T13:38:00.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.342+0000 7f5df2095640 1 -- start start 2024-07-26T13:38:00.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.342+0000 7f5df2095640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dec102c80 0x7f5dec199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:00.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.342+0000 7f5df2095640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dec104c50 0x7f5dec19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:00.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.342+0000 7f5df2095640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dec1055a0 0x7f5dec19f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:00.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.342+0000 7f5df2095640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec112470 con 0x7f5dec104c50 2024-07-26T13:38:00.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.342+0000 7f5df2095640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec1122f0 con 0x7f5dec1055a0 2024-07-26T13:38:00.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.342+0000 7f5df2095640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec1125f0 con 0x7f5dec102c80 2024-07-26T13:38:00.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.342+0000 7f5deaffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dec104c50 0x7f5dec19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:00.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.342+0000 7f5deaffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dec104c50 0x7f5dec19a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47490/0 (socket says 172.21.15.100:47490) 2024-07-26T13:38:00.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.342+0000 7f5deaffd640 1 -- 172.21.15.100:0/3607958440 learned_addr learned my addr 172.21.15.100:0/3607958440 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:38:00.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.342+0000 7f5deb7fe640 1 --2- 172.21.15.100:0/3607958440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dec102c80 0x7f5dec199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:00.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.342+0000 7f5debfff640 1 --2- 172.21.15.100:0/3607958440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dec1055a0 0x7f5dec19f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:00.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.342+0000 7f5deaffd640 1 -- 172.21.15.100:0/3607958440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dec102c80 msgr2=0x7f5dec199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:00.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.342+0000 7f5deaffd640 1 --2- 172.21.15.100:0/3607958440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dec102c80 0x7f5dec199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:00.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.342+0000 7f5deaffd640 1 -- 172.21.15.100:0/3607958440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dec1055a0 msgr2=0x7f5dec19f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:00.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.342+0000 7f5deaffd640 1 --2- 172.21.15.100:0/3607958440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dec1055a0 0x7f5dec19f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:00.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.342+0000 7f5deaffd640 1 -- 172.21.15.100:0/3607958440 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dec19fe00 con 0x7f5dec104c50 2024-07-26T13:38:00.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.343+0000 7f5deaffd640 1 --2- 172.21.15.100:0/3607958440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dec104c50 0x7f5dec19a4f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5de000c8c0 tx=0x7f5de000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:00.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.343+0000 7f5de8ff9640 1 -- 172.21.15.100:0/3607958440 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de0007bb0 con 0x7f5dec104c50 2024-07-26T13:38:00.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.343+0000 7f5df2095640 1 -- 172.21.15.100:0/3607958440 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dec1a00f0 con 0x7f5dec104c50 2024-07-26T13:38:00.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.343+0000 7f5de8ff9640 1 -- 172.21.15.100:0/3607958440 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5de0007d50 con 0x7f5dec104c50 2024-07-26T13:38:00.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.343+0000 7f5de8ff9640 1 -- 172.21.15.100:0/3607958440 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de00187c0 con 0x7f5dec104c50 2024-07-26T13:38:00.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.343+0000 7f5df2095640 1 -- 172.21.15.100:0/3607958440 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dec1a6910 con 0x7f5dec104c50 2024-07-26T13:38:00.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.344+0000 7f5df2095640 1 -- 172.21.15.100:0/3607958440 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dec103090 con 0x7f5dec104c50 2024-07-26T13:38:00.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.346+0000 7f5de8ff9640 1 -- 172.21.15.100:0/3607958440 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f5de000fa20 con 0x7f5dec104c50 2024-07-26T13:38:00.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.347+0000 7f5de8ff9640 1 --2- 172.21.15.100:0/3607958440 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5dd40b3ce0 0x7f5dd40b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:00.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.347+0000 7f5deb7fe640 1 --2- 172.21.15.100:0/3607958440 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5dd40b3ce0 0x7f5dd40b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:00.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.347+0000 7f5de8ff9640 1 -- 172.21.15.100:0/3607958440 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f5de00e4120 con 0x7f5dec104c50 2024-07-26T13:38:00.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.347+0000 7f5deb7fe640 1 --2- 172.21.15.100:0/3607958440 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5dd40b3ce0 0x7f5dd40b61f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5dec06b5c0 tx=0x7f5ddc007600 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:00.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.350+0000 7f5de8ff9640 1 -- 172.21.15.100:0/3607958440 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f5de0014030 con 0x7f5dec104c50 2024-07-26T13:38:00.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.531+0000 7f5df2095640 1 -- 172.21.15.100:0/3607958440 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5dec19b080 con 0x7f5dec104c50 2024-07-26T13:38:00.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.533+0000 7f5de8ff9640 1 -- 172.21.15.100:0/3607958440 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+23121 (secure 0 0 0) 0x7f5dec19b080 con 0x7f5dec104c50 2024-07-26T13:38:00.534 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:38:00.534 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":81,"fsid":"bf83e2e6-4b52-11ef-bcb7-c7b262605968","created":"2024-07-26T13:28:23.639785+0000","modified":"2024-07-26T13:37:17.539521+0000","last_up_change":"2024-07-26T13:35:56.475554+0000","last_in_change":"2024-07-26T13:35:37.472564+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-26T13:32:22.329969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-26T13:36:54.590525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-26T13:36:58.149545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d714036d-9dda-4cb5-af16-29f83a09dd70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6803","nonce":1480401850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6805","nonce":1480401850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6809","nonce":1480401850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6807","nonce":1480401850}]},"public_addr":"172.21.15.100:6803/1480401850","cluster_addr":"172.21.15.100:6805/1480401850","heartbeat_back_addr":"172.21.15.100:6809/1480401850","heartbeat_front_addr":"172.21.15.100:6807/1480401850","state":["exists","up"]},{"osd":1,"uuid":"3dd0eaa1-618a-495a-9819-e01c0f869ae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6801","nonce":3090877671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6803","nonce":3090877671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6807","nonce":3090877671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6805","nonce":3090877671}]},"public_addr":"172.21.15.138:6801/3090877671","cluster_addr":"172.21.15.138:6803/3090877671","heartbeat_back_addr":"172.21.15.138:6807/3090877671","heartbeat_front_addr":"172.21.15.138:6805/3090877671","state":["exists","up"]},{"osd":2,"uuid":"5c710a78-17ae-41a8-ac16-a678f5d35672","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6801","nonce":3583424790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6803","nonce":3583424790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6807","nonce":3583424790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6805","nonce":3583424790}]},"public_addr":"172.21.15.167:6801/3583424790","cluster_addr":"172.21.15.167:6803/3583424790","heartbeat_back_addr":"172.21.15.167:6807/3583424790","heartbeat_front_addr":"172.21.15.167:6805/3583424790","state":["exists","up"]},{"osd":3,"uuid":"be845495-8c31-4834-a45c-bdc576711962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6811","nonce":3626762419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6813","nonce":3626762419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6817","nonce":3626762419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6815","nonce":3626762419}]},"public_addr":"172.21.15.100:6811/3626762419","cluster_addr":"172.21.15.100:6813/3626762419","heartbeat_back_addr":"172.21.15.100:6817/3626762419","heartbeat_front_addr":"172.21.15.100:6815/3626762419","state":["exists","up"]},{"osd":4,"uuid":"520554cc-e846-46f9-af77-05a1de65a452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6809","nonce":3757334356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6811","nonce":3757334356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6815","nonce":3757334356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6813","nonce":3757334356}]},"public_addr":"172.21.15.138:6809/3757334356","cluster_addr":"172.21.15.138:6811/3757334356","heartbeat_back_addr":"172.21.15.138:6815/3757334356","heartbeat_front_addr":"172.21.15.138:6813/3757334356","state":["exists","up"]},{"osd":5,"uuid":"68dc8ad6-31d2-45b5-9aa7-62c4218d46e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6809","nonce":3239482836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6811","nonce":3239482836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6815","nonce":3239482836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6813","nonce":3239482836}]},"public_addr":"172.21.15.167:6809/3239482836","cluster_addr":"172.21.15.167:6811/3239482836","heartbeat_back_addr":"172.21.15.167:6815/3239482836","heartbeat_front_addr":"172.21.15.167:6813/3239482836","state":["exists","up"]},{"osd":6,"uuid":"420e4471-5626-41cc-a8e1-69c801196b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6819","nonce":3218415484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6821","nonce":3218415484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6825","nonce":3218415484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6823","nonce":3218415484}]},"public_addr":"172.21.15.100:6819/3218415484","cluster_addr":"172.21.15.100:6821/3218415484","heartbeat_back_addr":"172.21.15.100:6825/3218415484","heartbeat_front_addr":"172.21.15.100:6823/3218415484","state":["exists","up"]},{"osd":7,"uuid":"a27b097e-207d-44c3-af63-544f228b83ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6817","nonce":45541749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6819","nonce":45541749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6823","nonce":45541749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6821","nonce":45541749}]},"public_addr":"172.21.15.138:6817/45541749","cluster_addr":"172.21.15.138:6819/45541749","heartbeat_back_addr":"172.21.15.138:6823/45541749","heartbeat_front_addr":"172.21.15.138:6821/45541749","state":["exists","up"]},{"osd":8,"uuid":"3e865858-92cc-48a0-a464-5767d19e8ac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6817","nonce":4031159803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6819","nonce":4031159803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6823","nonce":4031159803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6821","nonce":4031159803}]},"public_addr":"172.21.15.167:6817/4031159803","cluster_addr":"172.21.15.167:6819/4031159803","heartbeat_back_addr":"172.21.15.167:6823/4031159803","heartbeat_front_addr":"172.21.15.167:6821/4031159803","state":["exists","up"]},{"osd":9,"uuid":"8b05b14a-1195-42f5-ac01-d5634cb9c888","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6827","nonce":1840140273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6829","nonce":1840140273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6833","nonce":1840140273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6831","nonce":1840140273}]},"public_addr":"172.21.15.100:6827/1840140273","cluster_addr":"172.21.15.100:6829/1840140273","heartbeat_back_addr":"172.21.15.100:6833/1840140273","heartbeat_front_addr":"172.21.15.100:6831/1840140273","state":["exists","up"]},{"osd":10,"uuid":"ab77ebe7-5254-4dc3-bc54-625698e84afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6825","nonce":703382899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6827","nonce":703382899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6831","nonce":703382899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6829","nonce":703382899}]},"public_addr":"172.21.15.138:6825/703382899","cluster_addr":"172.21.15.138:6827/703382899","heartbeat_back_addr":"172.21.15.138:6831/703382899","heartbeat_front_addr":"172.21.15.138:6829/703382899","state":["exists","up"]},{"osd":11,"uuid":"229cf91e-e9ef-48f6-a677-c5c5705b315a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6825","nonce":1032343068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6827","nonce":1032343068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6831","nonce":1032343068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6829","nonce":1032343068}]},"public_addr":"172.21.15.167:6825/1032343068","cluster_addr":"172.21.15.167:6827/1032343068","heartbeat_back_addr":"172.21.15.167:6831/1032343068","heartbeat_front_addr":"172.21.15.167:6829/1032343068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:31:35.800702+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:31:58.087922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:32:20.463183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:32:42.920598+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:33:06.332655+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:33:29.381407+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:33:53.975963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:34:18.226813+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:34:41.845770+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:35:05.722275+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:35:30.862704+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:35:54.738697+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:0/1874993112":"2024-07-27T13:30:16.253096+0000","172.21.15.100:0/2364620305":"2024-07-27T13:28:45.340334+0000","172.21.15.100:6800/3096076075":"2024-07-27T13:29:07.979178+0000","172.21.15.100:0/1894966501":"2024-07-27T13:30:16.253096+0000","172.21.15.100:0/3411776672":"2024-07-27T13:28:45.340334+0000","172.21.15.100:0/2853423120":"2024-07-27T13:29:07.979178+0000","172.21.15.100:6800/3025873570":"2024-07-27T13:28:45.340334+0000","172.21.15.100:0/4887181":"2024-07-27T13:29:07.979178+0000","172.21.15.100:6801/908675140":"2024-07-27T13:30:16.253096+0000","172.21.15.100:6801/3025873570":"2024-07-27T13:28:45.340334+0000","172.21.15.100:6800/908675140":"2024-07-27T13:30:16.253096+0000","172.21.15.100:6801/3096076075":"2024-07-27T13:29:07.979178+0000","172.21.15.100:0/2302632052":"2024-07-27T13:29:07.979178+0000","172.21.15.100:0/3736098915":"2024-07-27T13:28:45.340334+0000","172.21.15.100:0/3243030560":"2024-07-27T13:30:16.253096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-26T13:38:00.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.535+0000 7f5df2095640 1 -- 172.21.15.100:0/3607958440 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5dd40b3ce0 msgr2=0x7f5dd40b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:00.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.535+0000 7f5df2095640 1 --2- 172.21.15.100:0/3607958440 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5dd40b3ce0 0x7f5dd40b61f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5dec06b5c0 tx=0x7f5ddc007600 comp rx=0 tx=0).stop 2024-07-26T13:38:00.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.535+0000 7f5df2095640 1 -- 172.21.15.100:0/3607958440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dec104c50 msgr2=0x7f5dec19a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:00.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.535+0000 7f5df2095640 1 --2- 172.21.15.100:0/3607958440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dec104c50 0x7f5dec19a4f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5de000c8c0 tx=0x7f5de000cd80 comp rx=0 tx=0).stop 2024-07-26T13:38:00.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.535+0000 7f5df2095640 1 -- 172.21.15.100:0/3607958440 shutdown_connections 2024-07-26T13:38:00.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.535+0000 7f5df2095640 1 --2- 172.21.15.100:0/3607958440 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5dd40b3ce0 0x7f5dd40b61f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:00.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.535+0000 7f5df2095640 1 --2- 172.21.15.100:0/3607958440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dec1055a0 0x7f5dec19f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:00.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.535+0000 7f5df2095640 1 --2- 172.21.15.100:0/3607958440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5dec104c50 0x7f5dec19a4f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:00.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.535+0000 7f5df2095640 1 --2- 172.21.15.100:0/3607958440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dec102c80 0x7f5dec199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:00.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.535+0000 7f5df2095640 1 -- 172.21.15.100:0/3607958440 >> 172.21.15.100:0/3607958440 conn(0x7f5dec0798d0 msgr2=0x7f5dec105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:00.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.535+0000 7f5df2095640 1 -- 172.21.15.100:0/3607958440 shutdown_connections 2024-07-26T13:38:00.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:00.535+0000 7f5df2095640 1 -- 172.21.15.100:0/3607958440 wait complete. 2024-07-26T13:38:01.263 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-07-26T13:38:01.264 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-07-26T13:38:01.264 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-07-26T13:38:01.265 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-07-26T13:38:01.265 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-07-26T13:38:01.266 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-07-26T13:38:01.266 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-07-26T13:38:01.266 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-07-26T13:38:01.267 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-07-26T13:38:01.267 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-07-26T13:38:01.268 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-07-26T13:38:01.268 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-07-26T13:38:01.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:01 smithi167 ceph-mon[37140]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:38:01.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:01 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/3607958440' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-26T13:38:01.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:01 smithi138 ceph-mon[36954]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:38:01.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:01 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/3607958440' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-26T13:38:01.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:01 smithi100 ceph-mon[32599]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:38:01.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:01 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/3607958440' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-26T13:38:02.063 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:38:02.064 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:38:02.074 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:38:02.074 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:38:02.077 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:38:02.554 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:38:02.554 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:38:02.556 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:38:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:03 smithi167 ceph-mon[37140]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:38:03.791 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:03 smithi100 ceph-mon[32599]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:38:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:03 smithi138 ceph-mon[36954]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:38:04.142 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:38:04.920 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:04 smithi100 ceph-mon[32599]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:38:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:04 smithi138 ceph-mon[36954]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:38:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:04 smithi167 ceph-mon[37140]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:38:06.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.166+0000 7f76833a8640 1 -- 172.21.15.100:0/1325839077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f767c103590 msgr2=0x7f767c1039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:06.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.166+0000 7f76833a8640 1 --2- 172.21.15.100:0/1325839077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f767c103590 0x7f767c1039a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f766c009900 tx=0x7f766c02a410 comp rx=0 tx=0).stop 2024-07-26T13:38:06.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.166+0000 7f76833a8640 1 -- 172.21.15.100:0/1325839077 shutdown_connections 2024-07-26T13:38:06.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.166+0000 7f76833a8640 1 --2- 172.21.15.100:0/1325839077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f767c109820 0x7f767c110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:06.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.166+0000 7f76833a8640 1 --2- 172.21.15.100:0/1325839077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f767c108ed0 0x7f767c1092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:06.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.166+0000 7f76833a8640 1 --2- 172.21.15.100:0/1325839077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f767c103590 0x7f767c1039a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:06.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.166+0000 7f76833a8640 1 -- 172.21.15.100:0/1325839077 >> 172.21.15.100:0/1325839077 conn(0x7f767c07b280 msgr2=0x7f767c07b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:06.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.166+0000 7f76833a8640 1 -- 172.21.15.100:0/1325839077 shutdown_connections 2024-07-26T13:38:06.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.166+0000 7f76833a8640 1 -- 172.21.15.100:0/1325839077 wait complete. 2024-07-26T13:38:06.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.167+0000 7f76833a8640 1 Processor -- start 2024-07-26T13:38:06.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.167+0000 7f76833a8640 1 -- start start 2024-07-26T13:38:06.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.167+0000 7f76833a8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f767c103590 0x7f767c075140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:06.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.167+0000 7f76833a8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f767c108ed0 0x7f767c075680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:06.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.167+0000 7f76833a8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f767c109820 0x7f767c075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:06.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.167+0000 7f76833a8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f767c1134f0 con 0x7f767c103590 2024-07-26T13:38:06.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.167+0000 7f76833a8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f767c113370 con 0x7f767c108ed0 2024-07-26T13:38:06.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.167+0000 7f76833a8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f767c113670 con 0x7f767c109820 2024-07-26T13:38:06.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.167+0000 7f7681144640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f767c103590 0x7f767c075140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:06.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.167+0000 7f7681144640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f767c103590 0x7f767c075140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53380/0 (socket says 172.21.15.100:53380) 2024-07-26T13:38:06.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.167+0000 7f7681144640 1 -- 172.21.15.100:0/4039435847 learned_addr learned my addr 172.21.15.100:0/4039435847 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:38:06.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.167+0000 7f7680943640 1 --2- 172.21.15.100:0/4039435847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f767c108ed0 0x7f767c075680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:06.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.167+0000 7f7681945640 1 --2- 172.21.15.100:0/4039435847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f767c109820 0x7f767c075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:06.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.167+0000 7f7681144640 1 -- 172.21.15.100:0/4039435847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f767c109820 msgr2=0x7f767c075bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:06.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.167+0000 7f7681144640 1 --2- 172.21.15.100:0/4039435847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f767c109820 0x7f767c075bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:06.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.167+0000 7f7681144640 1 -- 172.21.15.100:0/4039435847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f767c108ed0 msgr2=0x7f767c075680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:06.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.167+0000 7f7681144640 1 --2- 172.21.15.100:0/4039435847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f767c108ed0 0x7f767c075680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:06.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.167+0000 7f7681144640 1 -- 172.21.15.100:0/4039435847 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f767c1a8240 con 0x7f767c103590 2024-07-26T13:38:06.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.167+0000 7f7680943640 1 --2- 172.21.15.100:0/4039435847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f767c108ed0 0x7f767c075680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:38:06.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.168+0000 7f7681144640 1 --2- 172.21.15.100:0/4039435847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f767c103590 0x7f767c075140 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f766c005940 tx=0x7f766c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:06.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.168+0000 7f767a7fc640 1 -- 172.21.15.100:0/4039435847 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c03f030 con 0x7f767c103590 2024-07-26T13:38:06.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.168+0000 7f767a7fc640 1 -- 172.21.15.100:0/4039435847 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f766c003aa0 con 0x7f767c103590 2024-07-26T13:38:06.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.168+0000 7f76833a8640 1 -- 172.21.15.100:0/4039435847 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f767c1a84d0 con 0x7f767c103590 2024-07-26T13:38:06.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.168+0000 7f767a7fc640 1 -- 172.21.15.100:0/4039435847 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c035070 con 0x7f767c103590 2024-07-26T13:38:06.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.168+0000 7f76833a8640 1 -- 172.21.15.100:0/4039435847 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f767c1a8a00 con 0x7f767c103590 2024-07-26T13:38:06.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.169+0000 7f76833a8640 1 -- 172.21.15.100:0/4039435847 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f767c063540 con 0x7f767c103590 2024-07-26T13:38:06.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.170+0000 7f767a7fc640 1 -- 172.21.15.100:0/4039435847 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f766c042030 con 0x7f767c103590 2024-07-26T13:38:06.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.171+0000 7f767a7fc640 1 --2- 172.21.15.100:0/4039435847 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f76640b3d30 0x7f76640b6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:06.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.171+0000 7f7680943640 1 --2- 172.21.15.100:0/4039435847 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f76640b3d30 0x7f76640b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:06.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.171+0000 7f767a7fc640 1 -- 172.21.15.100:0/4039435847 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f766c104d40 con 0x7f767c103590 2024-07-26T13:38:06.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.171+0000 7f767a7fc640 1 --2- 172.21.15.100:0/4039435847 >> [v2:172.21.15.167:6800/3583424790,v1:172.21.15.167:6801/3583424790] conn(0x7f76640bdcb0 0x7f76640c01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:06.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.171+0000 7f767a7fc640 1 -- 172.21.15.100:0/4039435847 --> [v2:172.21.15.167:6800/3583424790,v1:172.21.15.167:6801/3583424790] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f766c03ce50 con 0x7f76640bdcb0 2024-07-26T13:38:06.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.171+0000 7f767a7fc640 1 -- 172.21.15.100:0/4039435847 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f766c105190 con 0x7f767c103590 2024-07-26T13:38:06.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.172+0000 7f7680943640 1 --2- 172.21.15.100:0/4039435847 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f76640b3d30 0x7f76640b6240 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f767c108c50 tx=0x7f7670005ec0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:06.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.172+0000 7f7681945640 1 --2- 172.21.15.100:0/4039435847 >> [v2:172.21.15.167:6800/3583424790,v1:172.21.15.167:6801/3583424790] conn(0x7f76640bdcb0 0x7f76640c01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:06.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.172+0000 7f7681945640 1 --2- 172.21.15.100:0/4039435847 >> [v2:172.21.15.167:6800/3583424790,v1:172.21.15.167:6801/3583424790] conn(0x7f76640bdcb0 0x7f76640c01a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:06.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.173+0000 7f767a7fc640 1 -- 172.21.15.100:0/4039435847 <== osd.2 v2:172.21.15.167:6800/3583424790 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f766c03ce50 con 0x7f76640bdcb0 2024-07-26T13:38:06.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.187+0000 7f76833a8640 1 -- 172.21.15.100:0/4039435847 --> [v2:172.21.15.167:6800/3583424790,v1:172.21.15.167:6801/3583424790] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f767c063940 con 0x7f76640bdcb0 2024-07-26T13:38:06.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.187+0000 7f767a7fc640 1 -- 172.21.15.100:0/4039435847 <== osd.2 v2:172.21.15.167:6800/3583424790 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f767c063940 con 0x7f76640bdcb0 2024-07-26T13:38:06.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.188+0000 7f76833a8640 1 -- 172.21.15.100:0/4039435847 >> [v2:172.21.15.167:6800/3583424790,v1:172.21.15.167:6801/3583424790] conn(0x7f76640bdcb0 msgr2=0x7f76640c01a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:06.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.188+0000 7f76833a8640 1 --2- 172.21.15.100:0/4039435847 >> [v2:172.21.15.167:6800/3583424790,v1:172.21.15.167:6801/3583424790] conn(0x7f76640bdcb0 0x7f76640c01a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:06.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.188+0000 7f76833a8640 1 -- 172.21.15.100:0/4039435847 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f76640b3d30 msgr2=0x7f76640b6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:06.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.188+0000 7f76833a8640 1 --2- 172.21.15.100:0/4039435847 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f76640b3d30 0x7f76640b6240 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f767c108c50 tx=0x7f7670005ec0 comp rx=0 tx=0).stop 2024-07-26T13:38:06.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.188+0000 7f76833a8640 1 -- 172.21.15.100:0/4039435847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f767c103590 msgr2=0x7f767c075140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:06.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.188+0000 7f76833a8640 1 --2- 172.21.15.100:0/4039435847 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f767c103590 0x7f767c075140 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f766c005940 tx=0x7f766c004270 comp rx=0 tx=0).stop 2024-07-26T13:38:06.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.188+0000 7f7681945640 1 -- 172.21.15.100:0/4039435847 reap_dead start 2024-07-26T13:38:06.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.188+0000 7f76833a8640 1 -- 172.21.15.100:0/4039435847 shutdown_connections 2024-07-26T13:38:06.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.188+0000 7f76833a8640 1 -- 172.21.15.100:0/4039435847 >> 172.21.15.100:0/4039435847 conn(0x7f767c07b280 msgr2=0x7f767c107080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:06.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.188+0000 7f76833a8640 1 -- 172.21.15.100:0/4039435847 shutdown_connections 2024-07-26T13:38:06.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:06.188+0000 7f76833a8640 1 -- 172.21.15.100:0/4039435847 wait complete. 2024-07-26T13:38:07.500 INFO:teuthology.orchestra.run.smithi100.stdout:81604378696 2024-07-26T13:38:07.500 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-07-26T13:38:07.523 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:07 smithi100 ceph-mon[32599]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T13:38:07.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:07 smithi138 ceph-mon[36954]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T13:38:07.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:07 smithi167 ceph-mon[37140]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T13:38:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:09 smithi138 ceph-mon[36954]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:38:09.707 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:38:09.716 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:09 smithi100 ceph-mon[32599]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:38:09.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:09 smithi167 ceph-mon[37140]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:38:10.955 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:38:10.956 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:38:11.684 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:11 smithi100 ceph-mon[32599]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:38:11.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:11 smithi138 ceph-mon[36954]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:38:11.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:11 smithi167 ceph-mon[37140]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:38:11.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.775+0000 7f5cace45640 1 -- 172.21.15.100:0/1524919955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ca8105510 msgr2=0x7f5ca810ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:11.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.775+0000 7f5cace45640 1 --2- 172.21.15.100:0/1524919955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ca8105510 0x7f5ca810ef60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0009ca0 tx=0x7f5ca002a8c0 comp rx=0 tx=0).stop 2024-07-26T13:38:11.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.775+0000 7f5cace45640 1 -- 172.21.15.100:0/1524919955 shutdown_connections 2024-07-26T13:38:11.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.775+0000 7f5cace45640 1 --2- 172.21.15.100:0/1524919955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ca8105510 0x7f5ca810ef60 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:11.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.775+0000 7f5cace45640 1 --2- 172.21.15.100:0/1524919955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ca8104bc0 0x7f5ca8104fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:11.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.775+0000 7f5cace45640 1 --2- 172.21.15.100:0/1524919955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ca8102bf0 0x7f5ca8103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:11.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.775+0000 7f5cace45640 1 -- 172.21.15.100:0/1524919955 >> 172.21.15.100:0/1524919955 conn(0x7f5ca8077690 msgr2=0x7f5ca8077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:11.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.775+0000 7f5cace45640 1 -- 172.21.15.100:0/1524919955 shutdown_connections 2024-07-26T13:38:11.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.775+0000 7f5cace45640 1 -- 172.21.15.100:0/1524919955 wait complete. 2024-07-26T13:38:11.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.775+0000 7f5cace45640 1 Processor -- start 2024-07-26T13:38:11.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.776+0000 7f5cace45640 1 -- start start 2024-07-26T13:38:11.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.776+0000 7f5cace45640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ca8102bf0 0x7f5ca8199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:11.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.776+0000 7f5cace45640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ca8104bc0 0x7f5ca819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:11.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.776+0000 7f5cace45640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ca8105510 0x7f5ca819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:11.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.776+0000 7f5cace45640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca81134d0 con 0x7f5ca8105510 2024-07-26T13:38:11.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.776+0000 7f5cace45640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca8113350 con 0x7f5ca8104bc0 2024-07-26T13:38:11.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.776+0000 7f5cace45640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca8113650 con 0x7f5ca8102bf0 2024-07-26T13:38:11.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.776+0000 7f5ca6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ca8105510 0x7f5ca819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:11.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.776+0000 7f5ca6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ca8105510 0x7f5ca819f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53406/0 (socket says 172.21.15.100:53406) 2024-07-26T13:38:11.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.776+0000 7f5ca6d9d640 1 -- 172.21.15.100:0/1081120537 learned_addr learned my addr 172.21.15.100:0/1081120537 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:38:11.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.776+0000 7f5ca659c640 1 --2- 172.21.15.100:0/1081120537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ca8102bf0 0x7f5ca8199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:11.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.776+0000 7f5c9ffff640 1 --2- 172.21.15.100:0/1081120537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ca8104bc0 0x7f5ca819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:11.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.776+0000 7f5ca6d9d640 1 -- 172.21.15.100:0/1081120537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ca8102bf0 msgr2=0x7f5ca8199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:11.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.776+0000 7f5ca6d9d640 1 --2- 172.21.15.100:0/1081120537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ca8102bf0 0x7f5ca8199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:11.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.776+0000 7f5ca6d9d640 1 -- 172.21.15.100:0/1081120537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ca8104bc0 msgr2=0x7f5ca819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:11.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.776+0000 7f5ca6d9d640 1 --2- 172.21.15.100:0/1081120537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ca8104bc0 0x7f5ca819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:11.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.776+0000 7f5ca6d9d640 1 -- 172.21.15.100:0/1081120537 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca819fe50 con 0x7f5ca8105510 2024-07-26T13:38:11.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.776+0000 7f5c9ffff640 1 --2- 172.21.15.100:0/1081120537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ca8104bc0 0x7f5ca819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:38:11.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.777+0000 7f5ca6d9d640 1 --2- 172.21.15.100:0/1081120537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ca8105510 0x7f5ca819f720 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0005cc0 tx=0x7f5ca0031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:11.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.777+0000 7f5c9f7fe640 1 -- 172.21.15.100:0/1081120537 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca0003b70 con 0x7f5ca8105510 2024-07-26T13:38:11.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.777+0000 7f5cace45640 1 -- 172.21.15.100:0/1081120537 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ca81a00e0 con 0x7f5ca8105510 2024-07-26T13:38:11.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.777+0000 7f5c9f7fe640 1 -- 172.21.15.100:0/1081120537 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ca00318f0 con 0x7f5ca8105510 2024-07-26T13:38:11.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.777+0000 7f5c9f7fe640 1 -- 172.21.15.100:0/1081120537 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca002f950 con 0x7f5ca8105510 2024-07-26T13:38:11.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.777+0000 7f5cace45640 1 -- 172.21.15.100:0/1081120537 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ca81a68a0 con 0x7f5ca8105510 2024-07-26T13:38:11.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.778+0000 7f5cace45640 1 -- 172.21.15.100:0/1081120537 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5ca8063580 con 0x7f5ca8105510 2024-07-26T13:38:11.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.779+0000 7f5c9f7fe640 1 -- 172.21.15.100:0/1081120537 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f5ca002faf0 con 0x7f5ca8105510 2024-07-26T13:38:11.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.780+0000 7f5c9f7fe640 1 --2- 172.21.15.100:0/1081120537 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5c940b3cb0 0x7f5c940b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:11.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.780+0000 7f5ca659c640 1 --2- 172.21.15.100:0/1081120537 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5c940b3cb0 0x7f5c940b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:11.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.780+0000 7f5c9f7fe640 1 -- 172.21.15.100:0/1081120537 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f5ca0105420 con 0x7f5ca8105510 2024-07-26T13:38:11.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.780+0000 7f5c9f7fe640 1 --2- 172.21.15.100:0/1081120537 >> [v2:172.21.15.167:6824/1032343068,v1:172.21.15.167:6825/1032343068] conn(0x7f5c940bdc30 0x7f5c940c0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:11.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.780+0000 7f5c9f7fe640 1 -- 172.21.15.100:0/1081120537 --> [v2:172.21.15.167:6824/1032343068,v1:172.21.15.167:6825/1032343068] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5ca003fec0 con 0x7f5c940bdc30 2024-07-26T13:38:11.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.780+0000 7f5c9f7fe640 1 -- 172.21.15.100:0/1081120537 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f5ca0105870 con 0x7f5ca8105510 2024-07-26T13:38:11.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.780+0000 7f5ca659c640 1 --2- 172.21.15.100:0/1081120537 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5c940b3cb0 0x7f5c940b61c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5c98009bd0 tx=0x7f5c98005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:11.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.780+0000 7f5c9ffff640 1 --2- 172.21.15.100:0/1081120537 >> [v2:172.21.15.167:6824/1032343068,v1:172.21.15.167:6825/1032343068] conn(0x7f5c940bdc30 0x7f5c940c0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:11.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.781+0000 7f5c9ffff640 1 --2- 172.21.15.100:0/1081120537 >> [v2:172.21.15.167:6824/1032343068,v1:172.21.15.167:6825/1032343068] conn(0x7f5c940bdc30 0x7f5c940c0120 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:11.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.782+0000 7f5c9f7fe640 1 -- 172.21.15.100:0/1081120537 <== osd.11 v2:172.21.15.167:6824/1032343068 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5ca003fec0 con 0x7f5c940bdc30 2024-07-26T13:38:11.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.796+0000 7f5cace45640 1 -- 172.21.15.100:0/1081120537 --> [v2:172.21.15.167:6824/1032343068,v1:172.21.15.167:6825/1032343068] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5ca80639b0 con 0x7f5c940bdc30 2024-07-26T13:38:11.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.796+0000 7f5c9f7fe640 1 -- 172.21.15.100:0/1081120537 <== osd.11 v2:172.21.15.167:6824/1032343068 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5ca80639b0 con 0x7f5c940bdc30 2024-07-26T13:38:11.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.796+0000 7f5cace45640 1 -- 172.21.15.100:0/1081120537 >> [v2:172.21.15.167:6824/1032343068,v1:172.21.15.167:6825/1032343068] conn(0x7f5c940bdc30 msgr2=0x7f5c940c0120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:11.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.796+0000 7f5cace45640 1 --2- 172.21.15.100:0/1081120537 >> [v2:172.21.15.167:6824/1032343068,v1:172.21.15.167:6825/1032343068] conn(0x7f5c940bdc30 0x7f5c940c0120 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:11.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.796+0000 7f5cace45640 1 -- 172.21.15.100:0/1081120537 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5c940b3cb0 msgr2=0x7f5c940b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:11.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.797+0000 7f5cace45640 1 --2- 172.21.15.100:0/1081120537 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5c940b3cb0 0x7f5c940b61c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5c98009bd0 tx=0x7f5c98005f90 comp rx=0 tx=0).stop 2024-07-26T13:38:11.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.797+0000 7f5cace45640 1 -- 172.21.15.100:0/1081120537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ca8105510 msgr2=0x7f5ca819f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:11.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.797+0000 7f5cace45640 1 --2- 172.21.15.100:0/1081120537 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ca8105510 0x7f5ca819f720 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0005cc0 tx=0x7f5ca0031700 comp rx=0 tx=0).stop 2024-07-26T13:38:11.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.797+0000 7f5ca6d9d640 1 -- 172.21.15.100:0/1081120537 reap_dead start 2024-07-26T13:38:11.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.797+0000 7f5cace45640 1 -- 172.21.15.100:0/1081120537 shutdown_connections 2024-07-26T13:38:11.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.797+0000 7f5cace45640 1 -- 172.21.15.100:0/1081120537 >> 172.21.15.100:0/1081120537 conn(0x7f5ca8077690 msgr2=0x7f5ca8105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:11.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.797+0000 7f5cace45640 1 -- 172.21.15.100:0/1081120537 shutdown_connections 2024-07-26T13:38:11.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:11.797+0000 7f5cace45640 1 -- 172.21.15.100:0/1081120537 wait complete. 2024-07-26T13:38:12.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.431+0000 7f46f9082640 1 -- 172.21.15.100:0/4231622044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46f4101a20 msgr2=0x7f46f4101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:12.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.431+0000 7f46f9082640 1 --2- 172.21.15.100:0/4231622044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46f4101a20 0x7f46f4101e30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f46e4009900 tx=0x7f46e402a410 comp rx=0 tx=0).stop 2024-07-26T13:38:12.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.431+0000 7f46f9082640 1 -- 172.21.15.100:0/4231622044 shutdown_connections 2024-07-26T13:38:12.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.431+0000 7f46f9082640 1 --2- 172.21.15.100:0/4231622044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46f4106650 0x7f46f41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:12.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.431+0000 7f46f9082640 1 --2- 172.21.15.100:0/4231622044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46f4105d00 0x7f46f4106110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:12.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.431+0000 7f46f9082640 1 --2- 172.21.15.100:0/4231622044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46f4101a20 0x7f46f4101e30 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:12.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.431+0000 7f46f9082640 1 -- 172.21.15.100:0/4231622044 >> 172.21.15.100:0/4231622044 conn(0x7f46f40776f0 msgr2=0x7f46f4077b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:12.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.431+0000 7f46f9082640 1 -- 172.21.15.100:0/4231622044 shutdown_connections 2024-07-26T13:38:12.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.431+0000 7f46f9082640 1 -- 172.21.15.100:0/4231622044 wait complete. 2024-07-26T13:38:12.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.432+0000 7f46f9082640 1 Processor -- start 2024-07-26T13:38:12.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.432+0000 7f46f9082640 1 -- start start 2024-07-26T13:38:12.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.432+0000 7f46f9082640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46f4101a20 0x7f46f4199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:12.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.432+0000 7f46f9082640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46f4105d00 0x7f46f419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:12.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.432+0000 7f46f9082640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46f4106650 0x7f46f419f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:12.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.432+0000 7f46f9082640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f46f41135a0 con 0x7f46f4106650 2024-07-26T13:38:12.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.432+0000 7f46f9082640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f46f4113420 con 0x7f46f4105d00 2024-07-26T13:38:12.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.432+0000 7f46f9082640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f46f4113720 con 0x7f46f4101a20 2024-07-26T13:38:12.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.432+0000 7f46f359e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46f4106650 0x7f46f419f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:12.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.432+0000 7f46f359e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46f4106650 0x7f46f419f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53420/0 (socket says 172.21.15.100:53420) 2024-07-26T13:38:12.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.432+0000 7f46f359e640 1 -- 172.21.15.100:0/1401375956 learned_addr learned my addr 172.21.15.100:0/1401375956 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:38:12.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.432+0000 7f46f2d9d640 1 --2- 172.21.15.100:0/1401375956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46f4101a20 0x7f46f4199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:12.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.432+0000 7f46ebfff640 1 --2- 172.21.15.100:0/1401375956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46f4105d00 0x7f46f419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:12.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.432+0000 7f46f359e640 1 -- 172.21.15.100:0/1401375956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46f4101a20 msgr2=0x7f46f4199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:12.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.432+0000 7f46f359e640 1 --2- 172.21.15.100:0/1401375956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46f4101a20 0x7f46f4199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:12.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.432+0000 7f46f359e640 1 -- 172.21.15.100:0/1401375956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46f4105d00 msgr2=0x7f46f419a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:12.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.432+0000 7f46f359e640 1 --2- 172.21.15.100:0/1401375956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46f4105d00 0x7f46f419a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:12.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.432+0000 7f46f359e640 1 -- 172.21.15.100:0/1401375956 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f419fec0 con 0x7f46f4106650 2024-07-26T13:38:12.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.433+0000 7f46f359e640 1 --2- 172.21.15.100:0/1401375956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46f4106650 0x7f46f419f790 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f46ec00b970 tx=0x7f46ec00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:12.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.433+0000 7f46f0d99640 1 -- 172.21.15.100:0/1401375956 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec019070 con 0x7f46f4106650 2024-07-26T13:38:12.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.433+0000 7f46f9082640 1 -- 172.21.15.100:0/1401375956 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46f41a01b0 con 0x7f46f4106650 2024-07-26T13:38:12.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.433+0000 7f46f0d99640 1 -- 172.21.15.100:0/1401375956 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46ec014440 con 0x7f46f4106650 2024-07-26T13:38:12.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.433+0000 7f46f0d99640 1 -- 172.21.15.100:0/1401375956 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec0075d0 con 0x7f46f4106650 2024-07-26T13:38:12.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.433+0000 7f46f9082640 1 -- 172.21.15.100:0/1401375956 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46f41a6910 con 0x7f46f4106650 2024-07-26T13:38:12.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.434+0000 7f46f9082640 1 -- 172.21.15.100:0/1401375956 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f46f4063580 con 0x7f46f4106650 2024-07-26T13:38:12.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.435+0000 7f46f0d99640 1 -- 172.21.15.100:0/1401375956 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f46ec01c070 con 0x7f46f4106650 2024-07-26T13:38:12.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.436+0000 7f46f0d99640 1 --2- 172.21.15.100:0/1401375956 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f46e00b3cb0 0x7f46e00b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:12.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.436+0000 7f46f2d9d640 1 --2- 172.21.15.100:0/1401375956 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f46e00b3cb0 0x7f46e00b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:12.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.436+0000 7f46f0d99640 1 -- 172.21.15.100:0/1401375956 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f46ec0e3c40 con 0x7f46f4106650 2024-07-26T13:38:12.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.437+0000 7f46f0d99640 1 --2- 172.21.15.100:0/1401375956 >> [v2:172.21.15.100:6826/1840140273,v1:172.21.15.100:6827/1840140273] conn(0x7f46e00bdc30 0x7f46e00c0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:12.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.437+0000 7f46f0d99640 1 -- 172.21.15.100:0/1401375956 --> [v2:172.21.15.100:6826/1840140273,v1:172.21.15.100:6827/1840140273] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f46ec025ea0 con 0x7f46e00bdc30 2024-07-26T13:38:12.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.437+0000 7f46f0d99640 1 -- 172.21.15.100:0/1401375956 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f46ec0e4090 con 0x7f46f4106650 2024-07-26T13:38:12.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.437+0000 7f46ebfff640 1 --2- 172.21.15.100:0/1401375956 >> [v2:172.21.15.100:6826/1840140273,v1:172.21.15.100:6827/1840140273] conn(0x7f46e00bdc30 0x7f46e00c0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:12.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.437+0000 7f46f2d9d640 1 --2- 172.21.15.100:0/1401375956 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f46e00b3cb0 0x7f46e00b61c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f46e4031440 tx=0x7f46e4005d50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:12.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.437+0000 7f46ebfff640 1 --2- 172.21.15.100:0/1401375956 >> [v2:172.21.15.100:6826/1840140273,v1:172.21.15.100:6827/1840140273] conn(0x7f46e00bdc30 0x7f46e00c0120 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:12.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.438+0000 7f46f0d99640 1 -- 172.21.15.100:0/1401375956 <== osd.9 v2:172.21.15.100:6826/1840140273 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f46ec025ea0 con 0x7f46e00bdc30 2024-07-26T13:38:12.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.451+0000 7f46f9082640 1 -- 172.21.15.100:0/1401375956 --> [v2:172.21.15.100:6826/1840140273,v1:172.21.15.100:6827/1840140273] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f46f4101e30 con 0x7f46e00bdc30 2024-07-26T13:38:12.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.452+0000 7f46f0d99640 1 -- 172.21.15.100:0/1401375956 <== osd.9 v2:172.21.15.100:6826/1840140273 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f46f4101e30 con 0x7f46e00bdc30 2024-07-26T13:38:12.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.452+0000 7f46f9082640 1 -- 172.21.15.100:0/1401375956 >> [v2:172.21.15.100:6826/1840140273,v1:172.21.15.100:6827/1840140273] conn(0x7f46e00bdc30 msgr2=0x7f46e00c0120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:12.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.452+0000 7f46f9082640 1 --2- 172.21.15.100:0/1401375956 >> [v2:172.21.15.100:6826/1840140273,v1:172.21.15.100:6827/1840140273] conn(0x7f46e00bdc30 0x7f46e00c0120 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:12.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.452+0000 7f46f9082640 1 -- 172.21.15.100:0/1401375956 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f46e00b3cb0 msgr2=0x7f46e00b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:12.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.452+0000 7f46f9082640 1 --2- 172.21.15.100:0/1401375956 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f46e00b3cb0 0x7f46e00b61c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f46e4031440 tx=0x7f46e4005d50 comp rx=0 tx=0).stop 2024-07-26T13:38:12.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.452+0000 7f46f9082640 1 -- 172.21.15.100:0/1401375956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46f4106650 msgr2=0x7f46f419f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:12.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.452+0000 7f46f9082640 1 --2- 172.21.15.100:0/1401375956 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46f4106650 0x7f46f419f790 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f46ec00b970 tx=0x7f46ec00be30 comp rx=0 tx=0).stop 2024-07-26T13:38:12.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.452+0000 7f46f359e640 1 -- 172.21.15.100:0/1401375956 reap_dead start 2024-07-26T13:38:12.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.452+0000 7f46f9082640 1 -- 172.21.15.100:0/1401375956 shutdown_connections 2024-07-26T13:38:12.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.452+0000 7f46f9082640 1 -- 172.21.15.100:0/1401375956 >> 172.21.15.100:0/1401375956 conn(0x7f46f40776f0 msgr2=0x7f46f4107010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:12.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.452+0000 7f46f9082640 1 -- 172.21.15.100:0/1401375956 shutdown_connections 2024-07-26T13:38:12.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:12.452+0000 7f46f9082640 1 -- 172.21.15.100:0/1401375956 wait complete. 2024-07-26T13:38:13.636 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:13 smithi100 ceph-mon[32599]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:38:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:13 smithi138 ceph-mon[36954]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:38:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:13 smithi167 ceph-mon[37140]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:38:14.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.372+0000 7fbcc5123640 1 -- 172.21.15.100:0/2016762353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcc0107f00 msgr2=0x7fbcc010e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:14.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.372+0000 7fbcc5123640 1 --2- 172.21.15.100:0/2016762353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcc0107f00 0x7fbcc010e730 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbcb800aa10 tx=0x7fbcb802b720 comp rx=0 tx=0).stop 2024-07-26T13:38:14.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.373+0000 7fbcc5123640 1 -- 172.21.15.100:0/2016762353 shutdown_connections 2024-07-26T13:38:14.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.373+0000 7fbcc5123640 1 --2- 172.21.15.100:0/2016762353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcc0107f00 0x7fbcc010e730 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:14.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.373+0000 7fbcc5123640 1 --2- 172.21.15.100:0/2016762353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcc01075b0 0x7fbcc01079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:14.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.373+0000 7fbcc5123640 1 --2- 172.21.15.100:0/2016762353 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcc0101c70 0x7fbcc0102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:14.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.373+0000 7fbcc5123640 1 -- 172.21.15.100:0/2016762353 >> 172.21.15.100:0/2016762353 conn(0x7fbcc00799c0 msgr2=0x7fbcc0079dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:14.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.373+0000 7fbcc5123640 1 -- 172.21.15.100:0/2016762353 shutdown_connections 2024-07-26T13:38:14.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.373+0000 7fbcc5123640 1 -- 172.21.15.100:0/2016762353 wait complete. 2024-07-26T13:38:14.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.373+0000 7fbcc5123640 1 Processor -- start 2024-07-26T13:38:14.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.374+0000 7fbcc5123640 1 -- start start 2024-07-26T13:38:14.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.374+0000 7fbcc5123640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcc0101c70 0x7fbcc0199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:14.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.374+0000 7fbcc5123640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcc01075b0 0x7fbcc019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:14.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.374+0000 7fbcc5123640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcc0107f00 0x7fbcc019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:14.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.374+0000 7fbcc5123640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc0111340 con 0x7fbcc0101c70 2024-07-26T13:38:14.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.374+0000 7fbcc5123640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc01111c0 con 0x7fbcc01075b0 2024-07-26T13:38:14.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.374+0000 7fbcc5123640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc01114c0 con 0x7fbcc0107f00 2024-07-26T13:38:14.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.374+0000 7fbcbed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcc0101c70 0x7fbcc0199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:14.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.374+0000 7fbcbed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcc0101c70 0x7fbcc0199eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53432/0 (socket says 172.21.15.100:53432) 2024-07-26T13:38:14.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.374+0000 7fbcbed9d640 1 -- 172.21.15.100:0/715471689 learned_addr learned my addr 172.21.15.100:0/715471689 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:38:14.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.374+0000 7fbcbe59c640 1 --2- 172.21.15.100:0/715471689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcc01075b0 0x7fbcc019a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:14.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.374+0000 7fbcbf59e640 1 --2- 172.21.15.100:0/715471689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcc0107f00 0x7fbcc019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:14.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.374+0000 7fbcbed9d640 1 -- 172.21.15.100:0/715471689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcc0107f00 msgr2=0x7fbcc019f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:14.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.374+0000 7fbcbed9d640 1 --2- 172.21.15.100:0/715471689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcc0107f00 0x7fbcc019f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:14.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.374+0000 7fbcbed9d640 1 -- 172.21.15.100:0/715471689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcc01075b0 msgr2=0x7fbcc019a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:14.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.374+0000 7fbcbed9d640 1 --2- 172.21.15.100:0/715471689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcc01075b0 0x7fbcc019a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:14.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.374+0000 7fbcbed9d640 1 -- 172.21.15.100:0/715471689 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc019fd60 con 0x7fbcc0101c70 2024-07-26T13:38:14.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.374+0000 7fbcbe59c640 1 --2- 172.21.15.100:0/715471689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcc01075b0 0x7fbcc019a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:38:14.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.374+0000 7fbcbed9d640 1 --2- 172.21.15.100:0/715471689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcc0101c70 0x7fbcc0199eb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000ea80 tx=0x7fbcb000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:14.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.375+0000 7fbca7fff640 1 -- 172.21.15.100:0/715471689 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb000ce40 con 0x7fbcc0101c70 2024-07-26T13:38:14.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.375+0000 7fbcc5123640 1 -- 172.21.15.100:0/715471689 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcc01a0050 con 0x7fbcc0101c70 2024-07-26T13:38:14.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.375+0000 7fbca7fff640 1 -- 172.21.15.100:0/715471689 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcb0005020 con 0x7fbcc0101c70 2024-07-26T13:38:14.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.375+0000 7fbca7fff640 1 -- 172.21.15.100:0/715471689 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb00116d0 con 0x7fbcc0101c70 2024-07-26T13:38:14.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.375+0000 7fbcc5123640 1 -- 172.21.15.100:0/715471689 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcc01a6870 con 0x7fbcc0101c70 2024-07-26T13:38:14.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.376+0000 7fbcc5123640 1 -- 172.21.15.100:0/715471689 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbcc0063580 con 0x7fbcc0101c70 2024-07-26T13:38:14.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.377+0000 7fbca7fff640 1 -- 172.21.15.100:0/715471689 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7fbcb00054d0 con 0x7fbcc0101c70 2024-07-26T13:38:14.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.378+0000 7fbca7fff640 1 --2- 172.21.15.100:0/715471689 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fbca80b3ce0 0x7fbca80b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:14.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.378+0000 7fbcbe59c640 1 --2- 172.21.15.100:0/715471689 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fbca80b3ce0 0x7fbca80b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:14.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.378+0000 7fbca7fff640 1 -- 172.21.15.100:0/715471689 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fbcb00e47b0 con 0x7fbcc0101c70 2024-07-26T13:38:14.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.378+0000 7fbca7fff640 1 --2- 172.21.15.100:0/715471689 >> [v2:172.21.15.100:6810/3626762419,v1:172.21.15.100:6811/3626762419] conn(0x7fbca80bdc60 0x7fbca80c0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:14.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.378+0000 7fbca7fff640 1 -- 172.21.15.100:0/715471689 --> [v2:172.21.15.100:6810/3626762419,v1:172.21.15.100:6811/3626762419] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbca80c0820 con 0x7fbca80bdc60 2024-07-26T13:38:14.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.378+0000 7fbca7fff640 1 -- 172.21.15.100:0/715471689 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fbcb00e4c00 con 0x7fbcc0101c70 2024-07-26T13:38:14.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.378+0000 7fbcbf59e640 1 --2- 172.21.15.100:0/715471689 >> [v2:172.21.15.100:6810/3626762419,v1:172.21.15.100:6811/3626762419] conn(0x7fbca80bdc60 0x7fbca80c0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:14.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.379+0000 7fbcbe59c640 1 --2- 172.21.15.100:0/715471689 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fbca80b3ce0 0x7fbca80b61f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0079e10 tx=0x7fbcb4005d90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:14.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.379+0000 7fbcbf59e640 1 --2- 172.21.15.100:0/715471689 >> [v2:172.21.15.100:6810/3626762419,v1:172.21.15.100:6811/3626762419] conn(0x7fbca80bdc60 0x7fbca80c0150 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:14.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.380+0000 7fbca7fff640 1 -- 172.21.15.100:0/715471689 <== osd.3 v2:172.21.15.100:6810/3626762419 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbca80c0820 con 0x7fbca80bdc60 2024-07-26T13:38:14.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.393+0000 7fbcc5123640 1 -- 172.21.15.100:0/715471689 --> [v2:172.21.15.100:6810/3626762419,v1:172.21.15.100:6811/3626762419] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbcc0063980 con 0x7fbca80bdc60 2024-07-26T13:38:14.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.393+0000 7fbca7fff640 1 -- 172.21.15.100:0/715471689 <== osd.3 v2:172.21.15.100:6810/3626762419 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbcc0063980 con 0x7fbca80bdc60 2024-07-26T13:38:14.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.394+0000 7fbcc5123640 1 -- 172.21.15.100:0/715471689 >> [v2:172.21.15.100:6810/3626762419,v1:172.21.15.100:6811/3626762419] conn(0x7fbca80bdc60 msgr2=0x7fbca80c0150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:14.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.394+0000 7fbcc5123640 1 --2- 172.21.15.100:0/715471689 >> [v2:172.21.15.100:6810/3626762419,v1:172.21.15.100:6811/3626762419] conn(0x7fbca80bdc60 0x7fbca80c0150 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:14.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.394+0000 7fbcc5123640 1 -- 172.21.15.100:0/715471689 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fbca80b3ce0 msgr2=0x7fbca80b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:14.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.394+0000 7fbcc5123640 1 --2- 172.21.15.100:0/715471689 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fbca80b3ce0 0x7fbca80b61f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0079e10 tx=0x7fbcb4005d90 comp rx=0 tx=0).stop 2024-07-26T13:38:14.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.394+0000 7fbcc5123640 1 -- 172.21.15.100:0/715471689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcc0101c70 msgr2=0x7fbcc0199eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:14.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.394+0000 7fbcc5123640 1 --2- 172.21.15.100:0/715471689 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbcc0101c70 0x7fbcc0199eb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000ea80 tx=0x7fbcb000ef40 comp rx=0 tx=0).stop 2024-07-26T13:38:14.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.394+0000 7fbcbf59e640 1 -- 172.21.15.100:0/715471689 reap_dead start 2024-07-26T13:38:14.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.394+0000 7fbcc5123640 1 -- 172.21.15.100:0/715471689 shutdown_connections 2024-07-26T13:38:14.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.394+0000 7fbcc5123640 1 -- 172.21.15.100:0/715471689 >> 172.21.15.100:0/715471689 conn(0x7fbcc00799c0 msgr2=0x7fbcc0100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:14.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.394+0000 7fbcc5123640 1 -- 172.21.15.100:0/715471689 shutdown_connections 2024-07-26T13:38:14.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:14.394+0000 7fbcc5123640 1 -- 172.21.15.100:0/715471689 wait complete. 2024-07-26T13:38:15.640 INFO:teuthology.orchestra.run.smithi100.stdout:257698037800 2024-07-26T13:38:15.641 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-07-26T13:38:15.704 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:15 smithi100 ceph-mon[32599]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:38:15.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:15 smithi167 ceph-mon[37140]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:38:15.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.805+0000 7f5fca50a640 1 -- 172.21.15.100:0/1757633805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fc4102bb0 msgr2=0x7f5fc4102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:15.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.805+0000 7f5fca50a640 1 --2- 172.21.15.100:0/1757633805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fc4102bb0 0x7f5fc4102fc0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4009900 tx=0x7f5fb402a410 comp rx=0 tx=0).stop 2024-07-26T13:38:15.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.805+0000 7f5fca50a640 1 -- 172.21.15.100:0/1757633805 shutdown_connections 2024-07-26T13:38:15.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.805+0000 7f5fca50a640 1 --2- 172.21.15.100:0/1757633805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fc4108730 0x7f5fc410ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:15.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.805+0000 7f5fca50a640 1 --2- 172.21.15.100:0/1757633805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fc4107de0 0x7f5fc41081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:15.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.805+0000 7f5fca50a640 1 --2- 172.21.15.100:0/1757633805 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fc4102bb0 0x7f5fc4102fc0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:15.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.805+0000 7f5fca50a640 1 -- 172.21.15.100:0/1757633805 >> 172.21.15.100:0/1757633805 conn(0x7f5fc4077690 msgr2=0x7f5fc4077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:15.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.805+0000 7f5fca50a640 1 -- 172.21.15.100:0/1757633805 shutdown_connections 2024-07-26T13:38:15.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.805+0000 7f5fca50a640 1 -- 172.21.15.100:0/1757633805 wait complete. 2024-07-26T13:38:15.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.806+0000 7f5fca50a640 1 Processor -- start 2024-07-26T13:38:15.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.806+0000 7f5fca50a640 1 -- start start 2024-07-26T13:38:15.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.806+0000 7f5fca50a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fc4102bb0 0x7f5fc419ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:15.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.806+0000 7f5fca50a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fc4107de0 0x7f5fc419b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:15.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.806+0000 7f5fca50a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fc4108730 0x7f5fc419b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:15.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.806+0000 7f5fca50a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc4112400 con 0x7f5fc4108730 2024-07-26T13:38:15.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.806+0000 7f5fca50a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc4112280 con 0x7f5fc4102bb0 2024-07-26T13:38:15.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.806+0000 7f5fca50a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc4112580 con 0x7f5fc4107de0 2024-07-26T13:38:15.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.806+0000 7f5fc8aa7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fc4108730 0x7f5fc419b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:15.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.806+0000 7f5fc8aa7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fc4108730 0x7f5fc419b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48868/0 (socket says 172.21.15.100:48868) 2024-07-26T13:38:15.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.806+0000 7f5fc8aa7640 1 -- 172.21.15.100:0/512539823 learned_addr learned my addr 172.21.15.100:0/512539823 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:38:15.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.806+0000 7f5fc3fff640 1 --2- 172.21.15.100:0/512539823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fc4102bb0 0x7f5fc419ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:15.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.806+0000 7f5fc37fe640 1 --2- 172.21.15.100:0/512539823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fc4107de0 0x7f5fc419b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:15.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.807+0000 7f5fc8aa7640 1 -- 172.21.15.100:0/512539823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fc4107de0 msgr2=0x7f5fc419b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:15.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.807+0000 7f5fc8aa7640 1 --2- 172.21.15.100:0/512539823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fc4107de0 0x7f5fc419b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:15.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.807+0000 7f5fc8aa7640 1 -- 172.21.15.100:0/512539823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fc4102bb0 msgr2=0x7f5fc419ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:15.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.807+0000 7f5fc8aa7640 1 --2- 172.21.15.100:0/512539823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fc4102bb0 0x7f5fc419ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:15.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.807+0000 7f5fc8aa7640 1 -- 172.21.15.100:0/512539823 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fc41a5fa0 con 0x7f5fc4108730 2024-07-26T13:38:15.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.807+0000 7f5fc8aa7640 1 --2- 172.21.15.100:0/512539823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fc4108730 0x7f5fc419b790 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc00cad0 tx=0x7f5fbc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:15.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.807+0000 7f5fc17fa640 1 -- 172.21.15.100:0/512539823 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc00ae20 con 0x7f5fc4108730 2024-07-26T13:38:15.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.807+0000 7f5fca50a640 1 -- 172.21.15.100:0/512539823 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fc41a6290 con 0x7f5fc4108730 2024-07-26T13:38:15.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.807+0000 7f5fc17fa640 1 -- 172.21.15.100:0/512539823 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fbc005020 con 0x7f5fc4108730 2024-07-26T13:38:15.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.807+0000 7f5fca50a640 1 -- 172.21.15.100:0/512539823 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fc41a6820 con 0x7f5fc4108730 2024-07-26T13:38:15.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.807+0000 7f5fc17fa640 1 -- 172.21.15.100:0/512539823 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc00f6b0 con 0x7f5fc4108730 2024-07-26T13:38:15.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.809+0000 7f5fca50a640 1 -- 172.21.15.100:0/512539823 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5fc4063540 con 0x7f5fc4108730 2024-07-26T13:38:15.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.810+0000 7f5fc17fa640 1 -- 172.21.15.100:0/512539823 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f5fbc0051d0 con 0x7f5fc4108730 2024-07-26T13:38:15.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.811+0000 7f5fc17fa640 1 --2- 172.21.15.100:0/512539823 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5fb00b3ce0 0x7f5fb00b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:15.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.811+0000 7f5fc3fff640 1 --2- 172.21.15.100:0/512539823 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5fb00b3ce0 0x7f5fb00b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:15.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.811+0000 7f5fc17fa640 1 -- 172.21.15.100:0/512539823 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f5fbc0e3df0 con 0x7f5fc4108730 2024-07-26T13:38:15.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.811+0000 7f5fc17fa640 1 --2- 172.21.15.100:0/512539823 >> [v2:172.21.15.138:6816/45541749,v1:172.21.15.138:6817/45541749] conn(0x7f5fb00bdc60 0x7f5fb00c0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:15.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.811+0000 7f5fc17fa640 1 -- 172.21.15.100:0/512539823 --> [v2:172.21.15.138:6816/45541749,v1:172.21.15.138:6817/45541749] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5fb00c0820 con 0x7f5fb00bdc60 2024-07-26T13:38:15.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.811+0000 7f5fc17fa640 1 -- 172.21.15.100:0/512539823 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f5fbc0e64b0 con 0x7f5fc4108730 2024-07-26T13:38:15.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.811+0000 7f5fc3fff640 1 --2- 172.21.15.100:0/512539823 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5fb00b3ce0 0x7f5fb00b61f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4000c00 tx=0x7f5fb4031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:15.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.811+0000 7f5fc37fe640 1 --2- 172.21.15.100:0/512539823 >> [v2:172.21.15.138:6816/45541749,v1:172.21.15.138:6817/45541749] conn(0x7f5fb00bdc60 0x7f5fb00c0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:15.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.812+0000 7f5fc37fe640 1 --2- 172.21.15.100:0/512539823 >> [v2:172.21.15.138:6816/45541749,v1:172.21.15.138:6817/45541749] conn(0x7f5fb00bdc60 0x7f5fb00c0150 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:15.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.813+0000 7f5fc17fa640 1 -- 172.21.15.100:0/512539823 <== osd.7 v2:172.21.15.138:6816/45541749 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5fb00c0820 con 0x7f5fb00bdc60 2024-07-26T13:38:15.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.826+0000 7f5fca50a640 1 -- 172.21.15.100:0/512539823 --> [v2:172.21.15.138:6816/45541749,v1:172.21.15.138:6817/45541749] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5fc4063970 con 0x7f5fb00bdc60 2024-07-26T13:38:15.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.827+0000 7f5fc17fa640 1 -- 172.21.15.100:0/512539823 <== osd.7 v2:172.21.15.138:6816/45541749 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5fc4063970 con 0x7f5fb00bdc60 2024-07-26T13:38:15.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.827+0000 7f5fca50a640 1 -- 172.21.15.100:0/512539823 >> [v2:172.21.15.138:6816/45541749,v1:172.21.15.138:6817/45541749] conn(0x7f5fb00bdc60 msgr2=0x7f5fb00c0150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:15.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.827+0000 7f5fca50a640 1 --2- 172.21.15.100:0/512539823 >> [v2:172.21.15.138:6816/45541749,v1:172.21.15.138:6817/45541749] conn(0x7f5fb00bdc60 0x7f5fb00c0150 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:15.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.828+0000 7f5fca50a640 1 -- 172.21.15.100:0/512539823 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5fb00b3ce0 msgr2=0x7f5fb00b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:15.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.828+0000 7f5fca50a640 1 --2- 172.21.15.100:0/512539823 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5fb00b3ce0 0x7f5fb00b61f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4000c00 tx=0x7f5fb4031400 comp rx=0 tx=0).stop 2024-07-26T13:38:15.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.828+0000 7f5fca50a640 1 -- 172.21.15.100:0/512539823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fc4108730 msgr2=0x7f5fc419b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:15.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.828+0000 7f5fca50a640 1 --2- 172.21.15.100:0/512539823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fc4108730 0x7f5fc419b790 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc00cad0 tx=0x7f5fbc00cf90 comp rx=0 tx=0).stop 2024-07-26T13:38:15.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.828+0000 7f5fc8aa7640 1 -- 172.21.15.100:0/512539823 reap_dead start 2024-07-26T13:38:15.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.828+0000 7f5fca50a640 1 -- 172.21.15.100:0/512539823 shutdown_connections 2024-07-26T13:38:15.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.828+0000 7f5fca50a640 1 -- 172.21.15.100:0/512539823 >> 172.21.15.100:0/512539823 conn(0x7f5fc4077690 msgr2=0x7f5fc4105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:15.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.828+0000 7f5fca50a640 1 -- 172.21.15.100:0/512539823 shutdown_connections 2024-07-26T13:38:15.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:15.828+0000 7f5fca50a640 1 -- 172.21.15.100:0/512539823 wait complete. 2024-07-26T13:38:15.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:15 smithi138 ceph-mon[36954]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:38:17.096 INFO:teuthology.orchestra.run.smithi100.stdout:206158430258 2024-07-26T13:38:17.097 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-07-26T13:38:17.554 INFO:teuthology.orchestra.run.smithi100.stdout:304942678046 2024-07-26T13:38:17.555 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-07-26T13:38:17.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:17 smithi100 ceph-mon[32599]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T13:38:17.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:17 smithi167 ceph-mon[37140]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T13:38:17.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:17 smithi138 ceph-mon[36954]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T13:38:18.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.837+0000 7f1f49e76640 1 -- 172.21.15.100:0/196493202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f441024a0 msgr2=0x7f1f441028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:18.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.837+0000 7f1f49e76640 1 --2- 172.21.15.100:0/196493202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f441024a0 0x7f1f441028b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c009ca0 tx=0x7f1f3c02a8c0 comp rx=0 tx=0).stop 2024-07-26T13:38:18.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.837+0000 7f1f49e76640 1 -- 172.21.15.100:0/196493202 shutdown_connections 2024-07-26T13:38:18.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.837+0000 7f1f49e76640 1 --2- 172.21.15.100:0/196493202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f44108730 0x7f1f4410ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:18.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.837+0000 7f1f49e76640 1 --2- 172.21.15.100:0/196493202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f44107de0 0x7f1f441081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:18.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.837+0000 7f1f49e76640 1 --2- 172.21.15.100:0/196493202 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f441024a0 0x7f1f441028b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:18.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.837+0000 7f1f49e76640 1 -- 172.21.15.100:0/196493202 >> 172.21.15.100:0/196493202 conn(0x7f1f44079140 msgr2=0x7f1f44079550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:18.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.837+0000 7f1f49e76640 1 -- 172.21.15.100:0/196493202 shutdown_connections 2024-07-26T13:38:18.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.838+0000 7f1f49e76640 1 -- 172.21.15.100:0/196493202 wait complete. 2024-07-26T13:38:18.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.838+0000 7f1f49e76640 1 Processor -- start 2024-07-26T13:38:18.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.838+0000 7f1f49e76640 1 -- start start 2024-07-26T13:38:18.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.839+0000 7f1f49e76640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f441024a0 0x7f1f44199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:18.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.839+0000 7f1f49e76640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f44107de0 0x7f1f4419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:18.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.839+0000 7f1f49e76640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f44108730 0x7f1f4419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:18.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.839+0000 7f1f49e76640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1f44112400 con 0x7f1f44108730 2024-07-26T13:38:18.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.839+0000 7f1f49e76640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1f44112280 con 0x7f1f44107de0 2024-07-26T13:38:18.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.839+0000 7f1f49e76640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1f44112580 con 0x7f1f441024a0 2024-07-26T13:38:18.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.839+0000 7f1f43fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f44108730 0x7f1f4419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:18.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.839+0000 7f1f43fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f44108730 0x7f1f4419f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48894/0 (socket says 172.21.15.100:48894) 2024-07-26T13:38:18.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.839+0000 7f1f43fff640 1 -- 172.21.15.100:0/2355710877 learned_addr learned my addr 172.21.15.100:0/2355710877 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:38:18.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.839+0000 7f1f437fe640 1 --2- 172.21.15.100:0/2355710877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f441024a0 0x7f1f44199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:18.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.839+0000 7f1f42ffd640 1 --2- 172.21.15.100:0/2355710877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f44107de0 0x7f1f4419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:18.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.839+0000 7f1f43fff640 1 -- 172.21.15.100:0/2355710877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f441024a0 msgr2=0x7f1f44199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:18.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.839+0000 7f1f43fff640 1 --2- 172.21.15.100:0/2355710877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f441024a0 0x7f1f44199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:18.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.839+0000 7f1f43fff640 1 -- 172.21.15.100:0/2355710877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f44107de0 msgr2=0x7f1f4419a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:18.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.839+0000 7f1f43fff640 1 --2- 172.21.15.100:0/2355710877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f44107de0 0x7f1f4419a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:18.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.839+0000 7f1f43fff640 1 -- 172.21.15.100:0/2355710877 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f4419fe90 con 0x7f1f44108730 2024-07-26T13:38:18.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.839+0000 7f1f437fe640 1 --2- 172.21.15.100:0/2355710877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f441024a0 0x7f1f44199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:38:18.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.839+0000 7f1f42ffd640 1 --2- 172.21.15.100:0/2355710877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f44107de0 0x7f1f4419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:38:18.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.839+0000 7f1f43fff640 1 --2- 172.21.15.100:0/2355710877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f44108730 0x7f1f4419f760 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1f3400d750 tx=0x7f1f3400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:18.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.839+0000 7f1f40ff9640 1 -- 172.21.15.100:0/2355710877 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f34017070 con 0x7f1f44108730 2024-07-26T13:38:18.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.839+0000 7f1f49e76640 1 -- 172.21.15.100:0/2355710877 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f441a0180 con 0x7f1f44108730 2024-07-26T13:38:18.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.839+0000 7f1f40ff9640 1 -- 172.21.15.100:0/2355710877 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f340054d0 con 0x7f1f44108730 2024-07-26T13:38:18.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.839+0000 7f1f40ff9640 1 -- 172.21.15.100:0/2355710877 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f34004c60 con 0x7f1f44108730 2024-07-26T13:38:18.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.839+0000 7f1f49e76640 1 -- 172.21.15.100:0/2355710877 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f441a68e0 con 0x7f1f44108730 2024-07-26T13:38:18.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.840+0000 7f1f49e76640 1 -- 172.21.15.100:0/2355710877 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1f44063540 con 0x7f1f44108730 2024-07-26T13:38:18.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.842+0000 7f1f40ff9640 1 -- 172.21.15.100:0/2355710877 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f1f34009bd0 con 0x7f1f44108730 2024-07-26T13:38:18.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.842+0000 7f1f40ff9640 1 --2- 172.21.15.100:0/2355710877 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1f300b3cb0 0x7f1f300b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:18.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.843+0000 7f1f437fe640 1 --2- 172.21.15.100:0/2355710877 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1f300b3cb0 0x7f1f300b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:18.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.843+0000 7f1f40ff9640 1 -- 172.21.15.100:0/2355710877 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f1f340e3e10 con 0x7f1f44108730 2024-07-26T13:38:18.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.843+0000 7f1f40ff9640 1 --2- 172.21.15.100:0/2355710877 >> [v2:172.21.15.100:6818/3218415484,v1:172.21.15.100:6819/3218415484] conn(0x7f1f300bdc30 0x7f1f300c0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:18.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.843+0000 7f1f40ff9640 1 -- 172.21.15.100:0/2355710877 --> [v2:172.21.15.100:6818/3218415484,v1:172.21.15.100:6819/3218415484] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1f300c07f0 con 0x7f1f300bdc30 2024-07-26T13:38:18.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.843+0000 7f1f40ff9640 1 -- 172.21.15.100:0/2355710877 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f1f340e41c0 con 0x7f1f44108730 2024-07-26T13:38:18.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.843+0000 7f1f42ffd640 1 --2- 172.21.15.100:0/2355710877 >> [v2:172.21.15.100:6818/3218415484,v1:172.21.15.100:6819/3218415484] conn(0x7f1f300bdc30 0x7f1f300c0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:18.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.843+0000 7f1f437fe640 1 --2- 172.21.15.100:0/2355710877 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1f300b3cb0 0x7f1f300b61c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c006010 tx=0x7f1f3c000f40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:18.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.843+0000 7f1f42ffd640 1 --2- 172.21.15.100:0/2355710877 >> [v2:172.21.15.100:6818/3218415484,v1:172.21.15.100:6819/3218415484] conn(0x7f1f300bdc30 0x7f1f300c0120 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:18.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.844+0000 7f1f40ff9640 1 -- 172.21.15.100:0/2355710877 <== osd.6 v2:172.21.15.100:6818/3218415484 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1f300c07f0 con 0x7f1f300bdc30 2024-07-26T13:38:18.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.858+0000 7f1f49e76640 1 -- 172.21.15.100:0/2355710877 --> [v2:172.21.15.100:6818/3218415484,v1:172.21.15.100:6819/3218415484] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1f44063970 con 0x7f1f300bdc30 2024-07-26T13:38:18.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.858+0000 7f1f40ff9640 1 -- 172.21.15.100:0/2355710877 <== osd.6 v2:172.21.15.100:6818/3218415484 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1f44063970 con 0x7f1f300bdc30 2024-07-26T13:38:18.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.858+0000 7f1f49e76640 1 -- 172.21.15.100:0/2355710877 >> [v2:172.21.15.100:6818/3218415484,v1:172.21.15.100:6819/3218415484] conn(0x7f1f300bdc30 msgr2=0x7f1f300c0120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:18.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.858+0000 7f1f49e76640 1 --2- 172.21.15.100:0/2355710877 >> [v2:172.21.15.100:6818/3218415484,v1:172.21.15.100:6819/3218415484] conn(0x7f1f300bdc30 0x7f1f300c0120 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:18.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.858+0000 7f1f49e76640 1 -- 172.21.15.100:0/2355710877 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1f300b3cb0 msgr2=0x7f1f300b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:18.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.858+0000 7f1f49e76640 1 --2- 172.21.15.100:0/2355710877 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1f300b3cb0 0x7f1f300b61c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c006010 tx=0x7f1f3c000f40 comp rx=0 tx=0).stop 2024-07-26T13:38:18.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.859+0000 7f1f49e76640 1 -- 172.21.15.100:0/2355710877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f44108730 msgr2=0x7f1f4419f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:18.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.859+0000 7f1f49e76640 1 --2- 172.21.15.100:0/2355710877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f44108730 0x7f1f4419f760 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1f3400d750 tx=0x7f1f3400dc10 comp rx=0 tx=0).stop 2024-07-26T13:38:18.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.859+0000 7f1f43fff640 1 -- 172.21.15.100:0/2355710877 reap_dead start 2024-07-26T13:38:18.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.859+0000 7f1f49e76640 1 -- 172.21.15.100:0/2355710877 shutdown_connections 2024-07-26T13:38:18.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.859+0000 7f1f49e76640 1 -- 172.21.15.100:0/2355710877 >> 172.21.15.100:0/2355710877 conn(0x7f1f44079140 msgr2=0x7f1f44101510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:18.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.859+0000 7f1f49e76640 1 -- 172.21.15.100:0/2355710877 shutdown_connections 2024-07-26T13:38:18.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:18.859+0000 7f1f49e76640 1 -- 172.21.15.100:0/2355710877 wait complete. 2024-07-26T13:38:19.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.038+0000 7f574dddf640 1 -- 172.21.15.100:0/1809454514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5748104c50 msgr2=0x7f5748105060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:19.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.038+0000 7f574dddf640 1 --2- 172.21.15.100:0/1809454514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5748104c50 0x7f5748105060 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5730009900 tx=0x7f573002a410 comp rx=0 tx=0).stop 2024-07-26T13:38:19.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.038+0000 7f574dddf640 1 -- 172.21.15.100:0/1809454514 shutdown_connections 2024-07-26T13:38:19.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.038+0000 7f574dddf640 1 --2- 172.21.15.100:0/1809454514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57481055a0 0x7f574810f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:19.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.038+0000 7f574dddf640 1 --2- 172.21.15.100:0/1809454514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5748104c50 0x7f5748105060 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:19.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.038+0000 7f574dddf640 1 --2- 172.21.15.100:0/1809454514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5748102c40 0x7f5748103050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:19.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.038+0000 7f574dddf640 1 -- 172.21.15.100:0/1809454514 >> 172.21.15.100:0/1809454514 conn(0x7f57480798f0 msgr2=0x7f5748079d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:19.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.038+0000 7f574dddf640 1 -- 172.21.15.100:0/1809454514 shutdown_connections 2024-07-26T13:38:19.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.038+0000 7f574dddf640 1 -- 172.21.15.100:0/1809454514 wait complete. 2024-07-26T13:38:19.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.039+0000 7f574dddf640 1 Processor -- start 2024-07-26T13:38:19.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.039+0000 7f574dddf640 1 -- start start 2024-07-26T13:38:19.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.039+0000 7f574dddf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5748102c40 0x7f5748199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:19.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.039+0000 7f574dddf640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5748104c50 0x7f574819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:19.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.039+0000 7f574dddf640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57481055a0 0x7f574819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:19.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.039+0000 7f574dddf640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f57481135c0 con 0x7f5748102c40 2024-07-26T13:38:19.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.039+0000 7f574dddf640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5748113440 con 0x7f57481055a0 2024-07-26T13:38:19.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.039+0000 7f574dddf640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5748113740 con 0x7f5748104c50 2024-07-26T13:38:19.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.039+0000 7f57477fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5748102c40 0x7f5748199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:19.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.039+0000 7f57477fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5748102c40 0x7f5748199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48910/0 (socket says 172.21.15.100:48910) 2024-07-26T13:38:19.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.039+0000 7f57477fe640 1 -- 172.21.15.100:0/1629918640 learned_addr learned my addr 172.21.15.100:0/1629918640 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:38:19.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.039+0000 7f5746ffd640 1 --2- 172.21.15.100:0/1629918640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5748104c50 0x7f574819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:19.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.040+0000 7f5747fff640 1 --2- 172.21.15.100:0/1629918640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57481055a0 0x7f574819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:19.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.040+0000 7f57477fe640 1 -- 172.21.15.100:0/1629918640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5748104c50 msgr2=0x7f574819a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:19.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.040+0000 7f57477fe640 1 --2- 172.21.15.100:0/1629918640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5748104c50 0x7f574819a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:19.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.040+0000 7f57477fe640 1 -- 172.21.15.100:0/1629918640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57481055a0 msgr2=0x7f574819f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:19.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.040+0000 7f57477fe640 1 --2- 172.21.15.100:0/1629918640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57481055a0 0x7f574819f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:19.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.040+0000 7f57477fe640 1 -- 172.21.15.100:0/1629918640 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f574819fec0 con 0x7f5748102c40 2024-07-26T13:38:19.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.040+0000 7f5747fff640 1 --2- 172.21.15.100:0/1629918640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57481055a0 0x7f574819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:38:19.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.040+0000 7f57477fe640 1 --2- 172.21.15.100:0/1629918640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5748102c40 0x7f5748199f50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f573800e9f0 tx=0x7f573800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:19.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.040+0000 7f5744ff9640 1 -- 172.21.15.100:0/1629918640 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573800cd80 con 0x7f5748102c40 2024-07-26T13:38:19.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.040+0000 7f574dddf640 1 -- 172.21.15.100:0/1629918640 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57481a01b0 con 0x7f5748102c40 2024-07-26T13:38:19.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.040+0000 7f5744ff9640 1 -- 172.21.15.100:0/1629918640 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57380108e0 con 0x7f5748102c40 2024-07-26T13:38:19.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.040+0000 7f5744ff9640 1 -- 172.21.15.100:0/1629918640 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5738018980 con 0x7f5748102c40 2024-07-26T13:38:19.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.040+0000 7f574dddf640 1 -- 172.21.15.100:0/1629918640 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57481a6910 con 0x7f5748102c40 2024-07-26T13:38:19.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.041+0000 7f574dddf640 1 -- 172.21.15.100:0/1629918640 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5748063500 con 0x7f5748102c40 2024-07-26T13:38:19.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.043+0000 7f5744ff9640 1 -- 172.21.15.100:0/1629918640 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f5738010430 con 0x7f5748102c40 2024-07-26T13:38:19.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.044+0000 7f5744ff9640 1 --2- 172.21.15.100:0/1629918640 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f57340b3cb0 0x7f57340b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:19.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.044+0000 7f5746ffd640 1 --2- 172.21.15.100:0/1629918640 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f57340b3cb0 0x7f57340b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:19.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.044+0000 7f5744ff9640 1 -- 172.21.15.100:0/1629918640 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f57380e49e0 con 0x7f5748102c40 2024-07-26T13:38:19.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.044+0000 7f5744ff9640 1 --2- 172.21.15.100:0/1629918640 >> [v2:172.21.15.100:6802/1480401850,v1:172.21.15.100:6803/1480401850] conn(0x7f57340bdc30 0x7f57340c0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:19.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.044+0000 7f5744ff9640 1 -- 172.21.15.100:0/1629918640 --> [v2:172.21.15.100:6802/1480401850,v1:172.21.15.100:6803/1480401850] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f57340c07f0 con 0x7f57340bdc30 2024-07-26T13:38:19.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.044+0000 7f5744ff9640 1 -- 172.21.15.100:0/1629918640 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f57380e4e30 con 0x7f5748102c40 2024-07-26T13:38:19.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.044+0000 7f5747fff640 1 --2- 172.21.15.100:0/1629918640 >> [v2:172.21.15.100:6802/1480401850,v1:172.21.15.100:6803/1480401850] conn(0x7f57340bdc30 0x7f57340c0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:19.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.044+0000 7f5746ffd640 1 --2- 172.21.15.100:0/1629918640 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f57340b3cb0 0x7f57340b61c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5730009230 tx=0x7f5730031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:19.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.045+0000 7f5747fff640 1 --2- 172.21.15.100:0/1629918640 >> [v2:172.21.15.100:6802/1480401850,v1:172.21.15.100:6803/1480401850] conn(0x7f57340bdc30 0x7f57340c0120 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:19.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.045+0000 7f5744ff9640 1 -- 172.21.15.100:0/1629918640 <== osd.0 v2:172.21.15.100:6802/1480401850 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f57340c07f0 con 0x7f57340bdc30 2024-07-26T13:38:19.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.059+0000 7f574dddf640 1 -- 172.21.15.100:0/1629918640 --> [v2:172.21.15.100:6802/1480401850,v1:172.21.15.100:6803/1480401850] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5748063900 con 0x7f57340bdc30 2024-07-26T13:38:19.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.060+0000 7f5744ff9640 1 -- 172.21.15.100:0/1629918640 <== osd.0 v2:172.21.15.100:6802/1480401850 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5748063900 con 0x7f57340bdc30 2024-07-26T13:38:19.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.060+0000 7f574dddf640 1 -- 172.21.15.100:0/1629918640 >> [v2:172.21.15.100:6802/1480401850,v1:172.21.15.100:6803/1480401850] conn(0x7f57340bdc30 msgr2=0x7f57340c0120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:19.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.060+0000 7f574dddf640 1 --2- 172.21.15.100:0/1629918640 >> [v2:172.21.15.100:6802/1480401850,v1:172.21.15.100:6803/1480401850] conn(0x7f57340bdc30 0x7f57340c0120 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:19.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.060+0000 7f574dddf640 1 -- 172.21.15.100:0/1629918640 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f57340b3cb0 msgr2=0x7f57340b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:19.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.060+0000 7f574dddf640 1 --2- 172.21.15.100:0/1629918640 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f57340b3cb0 0x7f57340b61c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5730009230 tx=0x7f5730031400 comp rx=0 tx=0).stop 2024-07-26T13:38:19.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.060+0000 7f574dddf640 1 -- 172.21.15.100:0/1629918640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5748102c40 msgr2=0x7f5748199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:19.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.060+0000 7f574dddf640 1 --2- 172.21.15.100:0/1629918640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5748102c40 0x7f5748199f50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f573800e9f0 tx=0x7f573800eeb0 comp rx=0 tx=0).stop 2024-07-26T13:38:19.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.060+0000 7f5747fff640 1 -- 172.21.15.100:0/1629918640 reap_dead start 2024-07-26T13:38:19.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.061+0000 7f574dddf640 1 -- 172.21.15.100:0/1629918640 shutdown_connections 2024-07-26T13:38:19.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.061+0000 7f574dddf640 1 -- 172.21.15.100:0/1629918640 >> 172.21.15.100:0/1629918640 conn(0x7f57480798f0 msgr2=0x7f5748105fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:19.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.061+0000 7f574dddf640 1 -- 172.21.15.100:0/1629918640 shutdown_connections 2024-07-26T13:38:19.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.061+0000 7f574dddf640 1 -- 172.21.15.100:0/1629918640 wait complete. 2024-07-26T13:38:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:19 smithi100 ceph-mon[32599]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:38:19.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:19 smithi167 ceph-mon[37140]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:38:19.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.938+0000 7f568084a640 1 -- 172.21.15.100:0/3322783326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f567c108ed0 msgr2=0x7f567c1092e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:19.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.938+0000 7f568084a640 1 --2- 172.21.15.100:0/3322783326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f567c108ed0 0x7f567c1092e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5670009900 tx=0x7f567002a410 comp rx=0 tx=0).stop 2024-07-26T13:38:19.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.938+0000 7f568084a640 1 -- 172.21.15.100:0/3322783326 shutdown_connections 2024-07-26T13:38:19.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.938+0000 7f568084a640 1 --2- 172.21.15.100:0/3322783326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f567c109820 0x7f567c110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:19.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.938+0000 7f568084a640 1 --2- 172.21.15.100:0/3322783326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f567c108ed0 0x7f567c1092e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:19.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.938+0000 7f568084a640 1 --2- 172.21.15.100:0/3322783326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f567c103590 0x7f567c1039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:19.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.938+0000 7f568084a640 1 -- 172.21.15.100:0/3322783326 >> 172.21.15.100:0/3322783326 conn(0x7f567c07b280 msgr2=0x7f567c07b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:19.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.939+0000 7f568084a640 1 -- 172.21.15.100:0/3322783326 shutdown_connections 2024-07-26T13:38:19.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.939+0000 7f568084a640 1 -- 172.21.15.100:0/3322783326 wait complete. 2024-07-26T13:38:19.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:19 smithi138 ceph-mon[36954]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:38:19.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.939+0000 7f568084a640 1 Processor -- start 2024-07-26T13:38:19.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.939+0000 7f568084a640 1 -- start start 2024-07-26T13:38:19.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.940+0000 7f568084a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f567c103590 0x7f567c199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:19.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.940+0000 7f568084a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f567c108ed0 0x7f567c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:19.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.940+0000 7f568084a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f567c109820 0x7f567c19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:19.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.940+0000 7f568084a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f567c113490 con 0x7f567c109820 2024-07-26T13:38:19.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.940+0000 7f568084a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f567c113310 con 0x7f567c103590 2024-07-26T13:38:19.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.940+0000 7f568084a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f567c113610 con 0x7f567c108ed0 2024-07-26T13:38:19.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.940+0000 7f567ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f567c109820 0x7f567c19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:19.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.940+0000 7f567ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f567c109820 0x7f567c19f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48936/0 (socket says 172.21.15.100:48936) 2024-07-26T13:38:19.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.940+0000 7f567ad9d640 1 -- 172.21.15.100:0/3832328249 learned_addr learned my addr 172.21.15.100:0/3832328249 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:38:19.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.940+0000 7f567a59c640 1 --2- 172.21.15.100:0/3832328249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f567c103590 0x7f567c199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:19.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.940+0000 7f5679d9b640 1 --2- 172.21.15.100:0/3832328249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f567c108ed0 0x7f567c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:19.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.940+0000 7f567ad9d640 1 -- 172.21.15.100:0/3832328249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f567c108ed0 msgr2=0x7f567c19a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:19.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.940+0000 7f567ad9d640 1 --2- 172.21.15.100:0/3832328249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f567c108ed0 0x7f567c19a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:19.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.940+0000 7f567ad9d640 1 -- 172.21.15.100:0/3832328249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f567c103590 msgr2=0x7f567c199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:19.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.940+0000 7f567ad9d640 1 --2- 172.21.15.100:0/3832328249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f567c103590 0x7f567c199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:19.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.940+0000 7f567ad9d640 1 -- 172.21.15.100:0/3832328249 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f567c19fd70 con 0x7f567c109820 2024-07-26T13:38:19.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.941+0000 7f567ad9d640 1 --2- 172.21.15.100:0/3832328249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f567c109820 0x7f567c19f700 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f567400cad0 tx=0x7f567400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:19.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.941+0000 7f566b7fe640 1 -- 172.21.15.100:0/3832328249 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567400ad60 con 0x7f567c109820 2024-07-26T13:38:19.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.941+0000 7f568084a640 1 -- 172.21.15.100:0/3832328249 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f567c1a0060 con 0x7f567c109820 2024-07-26T13:38:19.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.941+0000 7f566b7fe640 1 -- 172.21.15.100:0/3832328249 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5674005020 con 0x7f567c109820 2024-07-26T13:38:19.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.941+0000 7f566b7fe640 1 -- 172.21.15.100:0/3832328249 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567400f6b0 con 0x7f567c109820 2024-07-26T13:38:19.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.941+0000 7f568084a640 1 -- 172.21.15.100:0/3832328249 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f567c1a0520 con 0x7f567c109820 2024-07-26T13:38:19.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.942+0000 7f568084a640 1 -- 172.21.15.100:0/3832328249 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f567c063540 con 0x7f567c109820 2024-07-26T13:38:19.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.943+0000 7f566b7fe640 1 -- 172.21.15.100:0/3832328249 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f56740051d0 con 0x7f567c109820 2024-07-26T13:38:19.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.944+0000 7f566b7fe640 1 --2- 172.21.15.100:0/3832328249 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f56640b3ce0 0x7f56640b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:19.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.944+0000 7f567a59c640 1 --2- 172.21.15.100:0/3832328249 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f56640b3ce0 0x7f56640b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:19.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.944+0000 7f566b7fe640 1 -- 172.21.15.100:0/3832328249 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f56740e42e0 con 0x7f567c109820 2024-07-26T13:38:19.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.944+0000 7f566b7fe640 1 --2- 172.21.15.100:0/3832328249 >> [v2:172.21.15.138:6808/3757334356,v1:172.21.15.138:6809/3757334356] conn(0x7f56640bdc60 0x7f56640c0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:19.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.944+0000 7f566b7fe640 1 -- 172.21.15.100:0/3832328249 --> [v2:172.21.15.138:6808/3757334356,v1:172.21.15.138:6809/3757334356] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5674016ec0 con 0x7f56640bdc60 2024-07-26T13:38:19.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.944+0000 7f566b7fe640 1 -- 172.21.15.100:0/3832328249 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f56740e4730 con 0x7f567c109820 2024-07-26T13:38:19.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.945+0000 7f567a59c640 1 --2- 172.21.15.100:0/3832328249 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f56640b3ce0 0x7f56640b61f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f566c00abe0 tx=0x7f566c00a2d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:19.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.945+0000 7f5679d9b640 1 --2- 172.21.15.100:0/3832328249 >> [v2:172.21.15.138:6808/3757334356,v1:172.21.15.138:6809/3757334356] conn(0x7f56640bdc60 0x7f56640c0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:19.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.945+0000 7f5679d9b640 1 --2- 172.21.15.100:0/3832328249 >> [v2:172.21.15.138:6808/3757334356,v1:172.21.15.138:6809/3757334356] conn(0x7f56640bdc60 0x7f56640c0150 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:19.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.946+0000 7f566b7fe640 1 -- 172.21.15.100:0/3832328249 <== osd.4 v2:172.21.15.138:6808/3757334356 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5674016ec0 con 0x7f56640bdc60 2024-07-26T13:38:19.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.960+0000 7f568084a640 1 -- 172.21.15.100:0/3832328249 --> [v2:172.21.15.138:6808/3757334356,v1:172.21.15.138:6809/3757334356] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f567c063940 con 0x7f56640bdc60 2024-07-26T13:38:19.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.960+0000 7f566b7fe640 1 -- 172.21.15.100:0/3832328249 <== osd.4 v2:172.21.15.138:6808/3757334356 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f567c063940 con 0x7f56640bdc60 2024-07-26T13:38:19.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.960+0000 7f568084a640 1 -- 172.21.15.100:0/3832328249 >> [v2:172.21.15.138:6808/3757334356,v1:172.21.15.138:6809/3757334356] conn(0x7f56640bdc60 msgr2=0x7f56640c0150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:19.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.960+0000 7f568084a640 1 --2- 172.21.15.100:0/3832328249 >> [v2:172.21.15.138:6808/3757334356,v1:172.21.15.138:6809/3757334356] conn(0x7f56640bdc60 0x7f56640c0150 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:19.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.960+0000 7f568084a640 1 -- 172.21.15.100:0/3832328249 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f56640b3ce0 msgr2=0x7f56640b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:19.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.960+0000 7f568084a640 1 --2- 172.21.15.100:0/3832328249 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f56640b3ce0 0x7f56640b61f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f566c00abe0 tx=0x7f566c00a2d0 comp rx=0 tx=0).stop 2024-07-26T13:38:19.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.960+0000 7f568084a640 1 -- 172.21.15.100:0/3832328249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f567c109820 msgr2=0x7f567c19f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:19.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.960+0000 7f568084a640 1 --2- 172.21.15.100:0/3832328249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f567c109820 0x7f567c19f700 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f567400cad0 tx=0x7f567400ced0 comp rx=0 tx=0).stop 2024-07-26T13:38:19.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.960+0000 7f567ad9d640 1 -- 172.21.15.100:0/3832328249 reap_dead start 2024-07-26T13:38:19.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.960+0000 7f568084a640 1 -- 172.21.15.100:0/3832328249 shutdown_connections 2024-07-26T13:38:19.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.961+0000 7f568084a640 1 -- 172.21.15.100:0/3832328249 >> 172.21.15.100:0/3832328249 conn(0x7f567c07b280 msgr2=0x7f567c106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:19.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.961+0000 7f568084a640 1 -- 172.21.15.100:0/3832328249 shutdown_connections 2024-07-26T13:38:19.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:19.961+0000 7f568084a640 1 -- 172.21.15.100:0/3832328249 wait complete. 2024-07-26T13:38:20.592 INFO:teuthology.orchestra.run.smithi100.stdout:111669149764 2024-07-26T13:38:20.592 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-07-26T13:38:21.271 INFO:teuthology.orchestra.run.smithi100.stdout:137438953538 2024-07-26T13:38:21.271 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-07-26T13:38:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:21 smithi100 ceph-mon[32599]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-07-26T13:38:21.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:21 smithi167 ceph-mon[37140]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-07-26T13:38:21.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:21 smithi138 ceph-mon[36954]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-07-26T13:38:22.321 INFO:teuthology.orchestra.run.smithi100.stdout:38654705748 2024-07-26T13:38:22.321 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-07-26T13:38:23.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.397+0000 7f27fe15f640 1 -- 172.21.15.100:0/1016326280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27f8108ed0 msgr2=0x7f27f81092e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:23.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.397+0000 7f27fe15f640 1 --2- 172.21.15.100:0/1016326280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27f8108ed0 0x7f27f81092e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f27ec009900 tx=0x7f27ec02a3f0 comp rx=0 tx=0).stop 2024-07-26T13:38:23.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.398+0000 7f27fe15f640 1 -- 172.21.15.100:0/1016326280 shutdown_connections 2024-07-26T13:38:23.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.398+0000 7f27fe15f640 1 --2- 172.21.15.100:0/1016326280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27f8109820 0x7f27f8110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:23.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.398+0000 7f27fe15f640 1 --2- 172.21.15.100:0/1016326280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27f8108ed0 0x7f27f81092e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:23.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.398+0000 7f27fe15f640 1 --2- 172.21.15.100:0/1016326280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27f8103590 0x7f27f81039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:23.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.398+0000 7f27fe15f640 1 -- 172.21.15.100:0/1016326280 >> 172.21.15.100:0/1016326280 conn(0x7f27f807b280 msgr2=0x7f27f807b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:23.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.398+0000 7f27fe15f640 1 -- 172.21.15.100:0/1016326280 shutdown_connections 2024-07-26T13:38:23.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.398+0000 7f27fe15f640 1 -- 172.21.15.100:0/1016326280 wait complete. 2024-07-26T13:38:23.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.398+0000 7f27fe15f640 1 Processor -- start 2024-07-26T13:38:23.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.398+0000 7f27fe15f640 1 -- start start 2024-07-26T13:38:23.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.399+0000 7f27fe15f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27f8103590 0x7f27f8199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:23.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.399+0000 7f27fe15f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27f8108ed0 0x7f27f819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:23.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.399+0000 7f27fe15f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27f8109820 0x7f27f819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:23.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.399+0000 7f27fe15f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f27f8113490 con 0x7f27f8109820 2024-07-26T13:38:23.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.399+0000 7f27fe15f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f27f8113310 con 0x7f27f8103590 2024-07-26T13:38:23.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.399+0000 7f27fe15f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f27f8113610 con 0x7f27f8108ed0 2024-07-26T13:38:23.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.399+0000 7f27f7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27f8109820 0x7f27f819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:23.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.399+0000 7f27f7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27f8109820 0x7f27f819f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48956/0 (socket says 172.21.15.100:48956) 2024-07-26T13:38:23.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.399+0000 7f27f7fff640 1 -- 172.21.15.100:0/3122938704 learned_addr learned my addr 172.21.15.100:0/3122938704 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:38:23.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.399+0000 7f27f6ffd640 1 --2- 172.21.15.100:0/3122938704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27f8108ed0 0x7f27f819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:23.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.399+0000 7f27f77fe640 1 --2- 172.21.15.100:0/3122938704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27f8103590 0x7f27f8199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:23.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.399+0000 7f27f7fff640 1 -- 172.21.15.100:0/3122938704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27f8108ed0 msgr2=0x7f27f819a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:23.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.399+0000 7f27f7fff640 1 --2- 172.21.15.100:0/3122938704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27f8108ed0 0x7f27f819a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:23.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.399+0000 7f27f7fff640 1 -- 172.21.15.100:0/3122938704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27f8103590 msgr2=0x7f27f8199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:23.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.399+0000 7f27f7fff640 1 --2- 172.21.15.100:0/3122938704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27f8103590 0x7f27f8199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:23.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.399+0000 7f27f7fff640 1 -- 172.21.15.100:0/3122938704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f819fd10 con 0x7f27f8109820 2024-07-26T13:38:23.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.399+0000 7f27f77fe640 1 --2- 172.21.15.100:0/3122938704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27f8103590 0x7f27f8199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:38:23.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.399+0000 7f27f7fff640 1 --2- 172.21.15.100:0/3122938704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27f8109820 0x7f27f819f6a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f27f000cce0 tx=0x7f27f000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:23.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.400+0000 7f27f4ff9640 1 -- 172.21.15.100:0/3122938704 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f000f450 con 0x7f27f8109820 2024-07-26T13:38:23.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.400+0000 7f27f4ff9640 1 -- 172.21.15.100:0/3122938704 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27f000faa0 con 0x7f27f8109820 2024-07-26T13:38:23.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.400+0000 7f27fe15f640 1 -- 172.21.15.100:0/3122938704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27f81a0000 con 0x7f27f8109820 2024-07-26T13:38:23.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.400+0000 7f27f4ff9640 1 -- 172.21.15.100:0/3122938704 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f0018b70 con 0x7f27f8109820 2024-07-26T13:38:23.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.400+0000 7f27fe15f640 1 -- 172.21.15.100:0/3122938704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27f81a04c0 con 0x7f27f8109820 2024-07-26T13:38:23.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.400+0000 7f27fe15f640 1 -- 172.21.15.100:0/3122938704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f27f8063540 con 0x7f27f8109820 2024-07-26T13:38:23.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.402+0000 7f27f4ff9640 1 -- 172.21.15.100:0/3122938704 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f27f0018d10 con 0x7f27f8109820 2024-07-26T13:38:23.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.403+0000 7f27f4ff9640 1 --2- 172.21.15.100:0/3122938704 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f27e00b3ce0 0x7f27e00b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:23.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.403+0000 7f27f77fe640 1 --2- 172.21.15.100:0/3122938704 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f27e00b3ce0 0x7f27e00b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:23.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.403+0000 7f27f4ff9640 1 -- 172.21.15.100:0/3122938704 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f27f00e4590 con 0x7f27f8109820 2024-07-26T13:38:23.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.403+0000 7f27f4ff9640 1 --2- 172.21.15.100:0/3122938704 >> [v2:172.21.15.167:6816/4031159803,v1:172.21.15.167:6817/4031159803] conn(0x7f27e00bdc60 0x7f27e00c0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:23.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.403+0000 7f27f4ff9640 1 -- 172.21.15.100:0/3122938704 --> [v2:172.21.15.167:6816/4031159803,v1:172.21.15.167:6817/4031159803] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f27f0012ec0 con 0x7f27e00bdc60 2024-07-26T13:38:23.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.403+0000 7f27f4ff9640 1 -- 172.21.15.100:0/3122938704 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f27f00e49e0 con 0x7f27f8109820 2024-07-26T13:38:23.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.403+0000 7f27f77fe640 1 --2- 172.21.15.100:0/3122938704 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f27e00b3ce0 0x7f27e00b61f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f27e800abe0 tx=0x7f27e800a5c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:23.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.403+0000 7f27f6ffd640 1 --2- 172.21.15.100:0/3122938704 >> [v2:172.21.15.167:6816/4031159803,v1:172.21.15.167:6817/4031159803] conn(0x7f27e00bdc60 0x7f27e00c0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:23.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.404+0000 7f27f6ffd640 1 --2- 172.21.15.100:0/3122938704 >> [v2:172.21.15.167:6816/4031159803,v1:172.21.15.167:6817/4031159803] conn(0x7f27e00bdc60 0x7f27e00c0150 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:23.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.405+0000 7f27f4ff9640 1 -- 172.21.15.100:0/3122938704 <== osd.8 v2:172.21.15.167:6816/4031159803 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f27f0012ec0 con 0x7f27e00bdc60 2024-07-26T13:38:23.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.418+0000 7f27fe15f640 1 -- 172.21.15.100:0/3122938704 --> [v2:172.21.15.167:6816/4031159803,v1:172.21.15.167:6817/4031159803] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f27f80639f0 con 0x7f27e00bdc60 2024-07-26T13:38:23.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.419+0000 7f27f4ff9640 1 -- 172.21.15.100:0/3122938704 <== osd.8 v2:172.21.15.167:6816/4031159803 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f27f80639f0 con 0x7f27e00bdc60 2024-07-26T13:38:23.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.420+0000 7f27fe15f640 1 -- 172.21.15.100:0/3122938704 >> [v2:172.21.15.167:6816/4031159803,v1:172.21.15.167:6817/4031159803] conn(0x7f27e00bdc60 msgr2=0x7f27e00c0150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:23.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.420+0000 7f27fe15f640 1 --2- 172.21.15.100:0/3122938704 >> [v2:172.21.15.167:6816/4031159803,v1:172.21.15.167:6817/4031159803] conn(0x7f27e00bdc60 0x7f27e00c0150 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:23.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.420+0000 7f27fe15f640 1 -- 172.21.15.100:0/3122938704 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f27e00b3ce0 msgr2=0x7f27e00b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:23.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.420+0000 7f27fe15f640 1 --2- 172.21.15.100:0/3122938704 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f27e00b3ce0 0x7f27e00b61f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f27e800abe0 tx=0x7f27e800a5c0 comp rx=0 tx=0).stop 2024-07-26T13:38:23.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.420+0000 7f27fe15f640 1 -- 172.21.15.100:0/3122938704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27f8109820 msgr2=0x7f27f819f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:23.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.420+0000 7f27fe15f640 1 --2- 172.21.15.100:0/3122938704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f27f8109820 0x7f27f819f6a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f27f000cce0 tx=0x7f27f000a620 comp rx=0 tx=0).stop 2024-07-26T13:38:23.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.420+0000 7f27f7fff640 1 -- 172.21.15.100:0/3122938704 reap_dead start 2024-07-26T13:38:23.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.420+0000 7f27fe15f640 1 -- 172.21.15.100:0/3122938704 shutdown_connections 2024-07-26T13:38:23.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.420+0000 7f27fe15f640 1 -- 172.21.15.100:0/3122938704 >> 172.21.15.100:0/3122938704 conn(0x7f27f807b280 msgr2=0x7f27f8106f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:23.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.420+0000 7f27fe15f640 1 -- 172.21.15.100:0/3122938704 shutdown_connections 2024-07-26T13:38:23.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.420+0000 7f27fe15f640 1 -- 172.21.15.100:0/3122938704 wait complete. 2024-07-26T13:38:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:23 smithi100 ceph-mon[32599]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:38:23.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.694+0000 7f58645ef640 1 -- 172.21.15.100:0/3661846620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f585c101b00 msgr2=0x7f585c101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:23.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.694+0000 7f58645ef640 1 --2- 172.21.15.100:0/3661846620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f585c101b00 0x7f585c101f10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5858009900 tx=0x7f585802a410 comp rx=0 tx=0).stop 2024-07-26T13:38:23.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.694+0000 7f58645ef640 1 -- 172.21.15.100:0/3661846620 shutdown_connections 2024-07-26T13:38:23.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.694+0000 7f58645ef640 1 --2- 172.21.15.100:0/3661846620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f585c106640 0x7f585c110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:23.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.694+0000 7f58645ef640 1 --2- 172.21.15.100:0/3661846620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f585c105cf0 0x7f585c106100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:23.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.694+0000 7f58645ef640 1 --2- 172.21.15.100:0/3661846620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f585c101b00 0x7f585c101f10 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:23.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.694+0000 7f58645ef640 1 -- 172.21.15.100:0/3661846620 >> 172.21.15.100:0/3661846620 conn(0x7f585c079860 msgr2=0x7f585c079c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:23.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.694+0000 7f58645ef640 1 -- 172.21.15.100:0/3661846620 shutdown_connections 2024-07-26T13:38:23.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.694+0000 7f58645ef640 1 -- 172.21.15.100:0/3661846620 wait complete. 2024-07-26T13:38:23.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.695+0000 7f58645ef640 1 Processor -- start 2024-07-26T13:38:23.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.695+0000 7f58645ef640 1 -- start start 2024-07-26T13:38:23.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.695+0000 7f58645ef640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f585c101b00 0x7f585c102aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:23.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.696+0000 7f58645ef640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f585c105cf0 0x7f585c102fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:23.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.696+0000 7f58645ef640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f585c106640 0x7f585c103520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:23.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.696+0000 7f58645ef640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f585c1134e0 con 0x7f585c105cf0 2024-07-26T13:38:23.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.696+0000 7f58645ef640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f585c113360 con 0x7f585c101b00 2024-07-26T13:38:23.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.696+0000 7f58645ef640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f585c113660 con 0x7f585c106640 2024-07-26T13:38:23.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.696+0000 7f5861b8a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f585c105cf0 0x7f585c102fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:23.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.696+0000 7f5861b8a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f585c105cf0 0x7f585c102fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48966/0 (socket says 172.21.15.100:48966) 2024-07-26T13:38:23.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.696+0000 7f5861b8a640 1 -- 172.21.15.100:0/4009518276 learned_addr learned my addr 172.21.15.100:0/4009518276 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:38:23.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.696+0000 7f586238b640 1 --2- 172.21.15.100:0/4009518276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f585c101b00 0x7f585c102aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:23.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.696+0000 7f5862b8c640 1 --2- 172.21.15.100:0/4009518276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f585c106640 0x7f585c103520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:23.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.696+0000 7f5861b8a640 1 -- 172.21.15.100:0/4009518276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f585c106640 msgr2=0x7f585c103520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:23.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.696+0000 7f5861b8a640 1 --2- 172.21.15.100:0/4009518276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f585c106640 0x7f585c103520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:23.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.696+0000 7f5861b8a640 1 -- 172.21.15.100:0/4009518276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f585c101b00 msgr2=0x7f585c102aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:23.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.696+0000 7f5861b8a640 1 --2- 172.21.15.100:0/4009518276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f585c101b00 0x7f585c102aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:23.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.696+0000 7f5861b8a640 1 -- 172.21.15.100:0/4009518276 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f585c1a81d0 con 0x7f585c105cf0 2024-07-26T13:38:23.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.696+0000 7f5861b8a640 1 --2- 172.21.15.100:0/4009518276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f585c105cf0 0x7f585c102fe0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f584c005fd0 tx=0x7f584c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:23.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.696+0000 7f58537fe640 1 -- 172.21.15.100:0/4009518276 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c017070 con 0x7f585c105cf0 2024-07-26T13:38:23.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.696+0000 7f58645ef640 1 -- 172.21.15.100:0/4009518276 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f585c1a84c0 con 0x7f585c105cf0 2024-07-26T13:38:23.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.696+0000 7f58537fe640 1 -- 172.21.15.100:0/4009518276 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f584c009c30 con 0x7f585c105cf0 2024-07-26T13:38:23.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.696+0000 7f58537fe640 1 -- 172.21.15.100:0/4009518276 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c003d30 con 0x7f585c105cf0 2024-07-26T13:38:23.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.697+0000 7f58645ef640 1 -- 172.21.15.100:0/4009518276 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f585c1a8a00 con 0x7f585c105cf0 2024-07-26T13:38:23.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.697+0000 7f58645ef640 1 -- 172.21.15.100:0/4009518276 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f585c063540 con 0x7f585c105cf0 2024-07-26T13:38:23.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.699+0000 7f58537fe640 1 -- 172.21.15.100:0/4009518276 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f584c003ed0 con 0x7f585c105cf0 2024-07-26T13:38:23.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.700+0000 7f58537fe640 1 --2- 172.21.15.100:0/4009518276 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f58480b3cb0 0x7f58480b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:23.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.700+0000 7f586238b640 1 --2- 172.21.15.100:0/4009518276 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f58480b3cb0 0x7f58480b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:23.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.700+0000 7f58537fe640 1 -- 172.21.15.100:0/4009518276 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f584c0e3e60 con 0x7f585c105cf0 2024-07-26T13:38:23.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.700+0000 7f58537fe640 1 --2- 172.21.15.100:0/4009518276 >> [v2:172.21.15.138:6824/703382899,v1:172.21.15.138:6825/703382899] conn(0x7f58480bdc30 0x7f58480c0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:23.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.700+0000 7f58537fe640 1 -- 172.21.15.100:0/4009518276 --> [v2:172.21.15.138:6824/703382899,v1:172.21.15.138:6825/703382899] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f58480c07f0 con 0x7f58480bdc30 2024-07-26T13:38:23.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.700+0000 7f58537fe640 1 -- 172.21.15.100:0/4009518276 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f584c0e42b0 con 0x7f585c105cf0 2024-07-26T13:38:23.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.700+0000 7f586238b640 1 --2- 172.21.15.100:0/4009518276 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f58480b3cb0 0x7f58480b61c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5858000c00 tx=0x7f5858031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:23.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.700+0000 7f5862b8c640 1 --2- 172.21.15.100:0/4009518276 >> [v2:172.21.15.138:6824/703382899,v1:172.21.15.138:6825/703382899] conn(0x7f58480bdc30 0x7f58480c0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:23.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.701+0000 7f5862b8c640 1 --2- 172.21.15.100:0/4009518276 >> [v2:172.21.15.138:6824/703382899,v1:172.21.15.138:6825/703382899] conn(0x7f58480bdc30 0x7f58480c0120 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:23.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.702+0000 7f58537fe640 1 -- 172.21.15.100:0/4009518276 <== osd.10 v2:172.21.15.138:6824/703382899 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f58480c07f0 con 0x7f58480bdc30 2024-07-26T13:38:23.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.716+0000 7f58645ef640 1 -- 172.21.15.100:0/4009518276 --> [v2:172.21.15.138:6824/703382899,v1:172.21.15.138:6825/703382899] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f585c063940 con 0x7f58480bdc30 2024-07-26T13:38:23.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.717+0000 7f58537fe640 1 -- 172.21.15.100:0/4009518276 <== osd.10 v2:172.21.15.138:6824/703382899 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f585c063940 con 0x7f58480bdc30 2024-07-26T13:38:23.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.717+0000 7f58645ef640 1 -- 172.21.15.100:0/4009518276 >> [v2:172.21.15.138:6824/703382899,v1:172.21.15.138:6825/703382899] conn(0x7f58480bdc30 msgr2=0x7f58480c0120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:23.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.717+0000 7f58645ef640 1 --2- 172.21.15.100:0/4009518276 >> [v2:172.21.15.138:6824/703382899,v1:172.21.15.138:6825/703382899] conn(0x7f58480bdc30 0x7f58480c0120 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:23.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.717+0000 7f58645ef640 1 -- 172.21.15.100:0/4009518276 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f58480b3cb0 msgr2=0x7f58480b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:23.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.717+0000 7f58645ef640 1 --2- 172.21.15.100:0/4009518276 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f58480b3cb0 0x7f58480b61c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5858000c00 tx=0x7f5858031400 comp rx=0 tx=0).stop 2024-07-26T13:38:23.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.717+0000 7f58645ef640 1 -- 172.21.15.100:0/4009518276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f585c105cf0 msgr2=0x7f585c102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:23.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.717+0000 7f58645ef640 1 --2- 172.21.15.100:0/4009518276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f585c105cf0 0x7f585c102fe0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f584c005fd0 tx=0x7f584c00dde0 comp rx=0 tx=0).stop 2024-07-26T13:38:23.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.717+0000 7f5862b8c640 1 -- 172.21.15.100:0/4009518276 reap_dead start 2024-07-26T13:38:23.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.717+0000 7f58645ef640 1 -- 172.21.15.100:0/4009518276 shutdown_connections 2024-07-26T13:38:23.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.718+0000 7f58645ef640 1 -- 172.21.15.100:0/4009518276 >> 172.21.15.100:0/4009518276 conn(0x7f585c079860 msgr2=0x7f585c107c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:23.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.718+0000 7f58645ef640 1 -- 172.21.15.100:0/4009518276 shutdown_connections 2024-07-26T13:38:23.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:23.718+0000 7f58645ef640 1 -- 172.21.15.100:0/4009518276 wait complete. 2024-07-26T13:38:23.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:23 smithi167 ceph-mon[37140]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:38:23.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:23 smithi138 ceph-mon[36954]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:38:24.681 INFO:teuthology.orchestra.run.smithi100.stdout:184683593784 2024-07-26T13:38:24.681 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-07-26T13:38:25.255 INFO:teuthology.orchestra.run.smithi100.stdout:279172874278 2024-07-26T13:38:25.255 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-07-26T13:38:25.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:25 smithi100 ceph-mon[32599]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-07-26T13:38:25.742 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:25 smithi138 ceph-mon[36954]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-07-26T13:38:25.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:25 smithi167 ceph-mon[37140]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-07-26T13:38:25.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.946+0000 7f57f4062640 1 -- 172.21.15.100:0/3254566674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57ec10dff0 msgr2=0x7f57ec10e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:25.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.946+0000 7f57f4062640 1 --2- 172.21.15.100:0/3254566674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57ec10dff0 0x7f57ec10e400 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f57e8009900 tx=0x7f57e802a410 comp rx=0 tx=0).stop 2024-07-26T13:38:25.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.946+0000 7f57f4062640 1 -- 172.21.15.100:0/3254566674 shutdown_connections 2024-07-26T13:38:25.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.946+0000 7f57f4062640 1 --2- 172.21.15.100:0/3254566674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57ec114230 0x7f57ec11aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:25.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.946+0000 7f57f4062640 1 --2- 172.21.15.100:0/3254566674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57ec1138e0 0x7f57ec113cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:25.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.946+0000 7f57f4062640 1 --2- 172.21.15.100:0/3254566674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57ec10dff0 0x7f57ec10e400 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:25.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.946+0000 7f57f4062640 1 -- 172.21.15.100:0/3254566674 >> 172.21.15.100:0/3254566674 conn(0x7f57ec109df0 msgr2=0x7f57ec10c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:25.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.947+0000 7f57f4062640 1 -- 172.21.15.100:0/3254566674 shutdown_connections 2024-07-26T13:38:25.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.947+0000 7f57f4062640 1 -- 172.21.15.100:0/3254566674 wait complete. 2024-07-26T13:38:25.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.947+0000 7f57f4062640 1 Processor -- start 2024-07-26T13:38:25.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.947+0000 7f57f4062640 1 -- start start 2024-07-26T13:38:25.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.947+0000 7f57f4062640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57ec10dff0 0x7f57ec1a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:25.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.947+0000 7f57f4062640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57ec1138e0 0x7f57ec1a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:25.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.947+0000 7f57f4062640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57ec114230 0x7f57ec1aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:25.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.947+0000 7f57f4062640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f57ec11def0 con 0x7f57ec114230 2024-07-26T13:38:25.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.947+0000 7f57f4062640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f57ec11dd70 con 0x7f57ec10dff0 2024-07-26T13:38:25.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.947+0000 7f57f4062640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f57ec11e070 con 0x7f57ec1138e0 2024-07-26T13:38:25.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.948+0000 7f57f25ff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57ec114230 0x7f57ec1aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:25.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.948+0000 7f57f25ff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57ec114230 0x7f57ec1aa0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40946/0 (socket says 172.21.15.100:40946) 2024-07-26T13:38:25.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.948+0000 7f57f25ff640 1 -- 172.21.15.100:0/4249672221 learned_addr learned my addr 172.21.15.100:0/4249672221 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:38:25.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.948+0000 7f57f15fd640 1 --2- 172.21.15.100:0/4249672221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57ec1138e0 0x7f57ec1a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:25.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.948+0000 7f57f1dfe640 1 --2- 172.21.15.100:0/4249672221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57ec10dff0 0x7f57ec1a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:25.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.948+0000 7f57f25ff640 1 -- 172.21.15.100:0/4249672221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57ec1138e0 msgr2=0x7f57ec1a4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:25.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.948+0000 7f57f25ff640 1 --2- 172.21.15.100:0/4249672221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57ec1138e0 0x7f57ec1a4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:25.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.948+0000 7f57f25ff640 1 -- 172.21.15.100:0/4249672221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57ec10dff0 msgr2=0x7f57ec1a48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:25.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.948+0000 7f57f25ff640 1 --2- 172.21.15.100:0/4249672221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57ec10dff0 0x7f57ec1a48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:25.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.948+0000 7f57f25ff640 1 -- 172.21.15.100:0/4249672221 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57ec1aa750 con 0x7f57ec114230 2024-07-26T13:38:25.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.948+0000 7f57f1dfe640 1 --2- 172.21.15.100:0/4249672221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57ec10dff0 0x7f57ec1a48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:38:25.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.948+0000 7f57f25ff640 1 --2- 172.21.15.100:0/4249672221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57ec114230 0x7f57ec1aa0e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f57e400eaa0 tx=0x7f57e400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:25.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.948+0000 7f57e2ffd640 1 -- 172.21.15.100:0/4249672221 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e4004d20 con 0x7f57ec114230 2024-07-26T13:38:25.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.948+0000 7f57f4062640 1 -- 172.21.15.100:0/4249672221 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57ec1aaa40 con 0x7f57ec114230 2024-07-26T13:38:25.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.948+0000 7f57e2ffd640 1 -- 172.21.15.100:0/4249672221 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57e4005020 con 0x7f57ec114230 2024-07-26T13:38:25.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.948+0000 7f57e2ffd640 1 -- 172.21.15.100:0/4249672221 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e40116d0 con 0x7f57ec114230 2024-07-26T13:38:25.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.948+0000 7f57f4062640 1 -- 172.21.15.100:0/4249672221 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57ec1b1260 con 0x7f57ec114230 2024-07-26T13:38:25.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.949+0000 7f57f4062640 1 -- 172.21.15.100:0/4249672221 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f57ec063580 con 0x7f57ec114230 2024-07-26T13:38:25.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.951+0000 7f57e2ffd640 1 -- 172.21.15.100:0/4249672221 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f57e400caf0 con 0x7f57ec114230 2024-07-26T13:38:25.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.951+0000 7f57e2ffd640 1 --2- 172.21.15.100:0/4249672221 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f57d80b3ce0 0x7f57d80b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:25.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.952+0000 7f57f1dfe640 1 --2- 172.21.15.100:0/4249672221 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f57d80b3ce0 0x7f57d80b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:25.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.952+0000 7f57e2ffd640 1 -- 172.21.15.100:0/4249672221 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f57e40e4430 con 0x7f57ec114230 2024-07-26T13:38:25.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.952+0000 7f57e2ffd640 1 --2- 172.21.15.100:0/4249672221 >> [v2:172.21.15.167:6808/3239482836,v1:172.21.15.167:6809/3239482836] conn(0x7f57d80bdc60 0x7f57d80c0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:25.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.952+0000 7f57e2ffd640 1 -- 172.21.15.100:0/4249672221 --> [v2:172.21.15.167:6808/3239482836,v1:172.21.15.167:6809/3239482836] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f57e4014ec0 con 0x7f57d80bdc60 2024-07-26T13:38:25.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.952+0000 7f57e2ffd640 1 -- 172.21.15.100:0/4249672221 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f57e40e4880 con 0x7f57ec114230 2024-07-26T13:38:25.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.952+0000 7f57f1dfe640 1 --2- 172.21.15.100:0/4249672221 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f57d80b3ce0 0x7f57d80b61f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f57e80096c0 tx=0x7f57e8031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:25.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.952+0000 7f57f15fd640 1 --2- 172.21.15.100:0/4249672221 >> [v2:172.21.15.167:6808/3239482836,v1:172.21.15.167:6809/3239482836] conn(0x7f57d80bdc60 0x7f57d80c0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:25.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.953+0000 7f57f15fd640 1 --2- 172.21.15.100:0/4249672221 >> [v2:172.21.15.167:6808/3239482836,v1:172.21.15.167:6809/3239482836] conn(0x7f57d80bdc60 0x7f57d80c0150 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:25.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.954+0000 7f57e2ffd640 1 -- 172.21.15.100:0/4249672221 <== osd.5 v2:172.21.15.167:6808/3239482836 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f57e4014ec0 con 0x7f57d80bdc60 2024-07-26T13:38:25.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.967+0000 7f57f4062640 1 -- 172.21.15.100:0/4249672221 --> [v2:172.21.15.167:6808/3239482836,v1:172.21.15.167:6809/3239482836] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f57ec063980 con 0x7f57d80bdc60 2024-07-26T13:38:25.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.968+0000 7f57e2ffd640 1 -- 172.21.15.100:0/4249672221 <== osd.5 v2:172.21.15.167:6808/3239482836 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f57ec063980 con 0x7f57d80bdc60 2024-07-26T13:38:25.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.968+0000 7f57f4062640 1 -- 172.21.15.100:0/4249672221 >> [v2:172.21.15.167:6808/3239482836,v1:172.21.15.167:6809/3239482836] conn(0x7f57d80bdc60 msgr2=0x7f57d80c0150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:25.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.968+0000 7f57f4062640 1 --2- 172.21.15.100:0/4249672221 >> [v2:172.21.15.167:6808/3239482836,v1:172.21.15.167:6809/3239482836] conn(0x7f57d80bdc60 0x7f57d80c0150 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:25.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.968+0000 7f57f4062640 1 -- 172.21.15.100:0/4249672221 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f57d80b3ce0 msgr2=0x7f57d80b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:25.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.968+0000 7f57f4062640 1 --2- 172.21.15.100:0/4249672221 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f57d80b3ce0 0x7f57d80b61f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f57e80096c0 tx=0x7f57e8031400 comp rx=0 tx=0).stop 2024-07-26T13:38:25.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.968+0000 7f57f4062640 1 -- 172.21.15.100:0/4249672221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57ec114230 msgr2=0x7f57ec1aa0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:25.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.968+0000 7f57f4062640 1 --2- 172.21.15.100:0/4249672221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57ec114230 0x7f57ec1aa0e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f57e400eaa0 tx=0x7f57e400eea0 comp rx=0 tx=0).stop 2024-07-26T13:38:25.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.968+0000 7f57f25ff640 1 -- 172.21.15.100:0/4249672221 reap_dead start 2024-07-26T13:38:25.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.968+0000 7f57f4062640 1 -- 172.21.15.100:0/4249672221 shutdown_connections 2024-07-26T13:38:25.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.968+0000 7f57f4062640 1 -- 172.21.15.100:0/4249672221 >> 172.21.15.100:0/4249672221 conn(0x7f57ec109df0 msgr2=0x7f57ec112eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:25.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.968+0000 7f57f4062640 1 -- 172.21.15.100:0/4249672221 shutdown_connections 2024-07-26T13:38:25.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:25.968+0000 7f57f4062640 1 -- 172.21.15.100:0/4249672221 wait complete. 2024-07-26T13:38:26.692 INFO:teuthology.orchestra.run.smithi100.stdout:158913790014 2024-07-26T13:38:26.692 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-07-26T13:38:27.500 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:38:27.501 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:38:27.501 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:38:27.504 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:38:27.504 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:38:27.505 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:38:27.551 INFO:teuthology.orchestra.run.smithi100.stdout:231928234030 2024-07-26T13:38:27.552 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-07-26T13:38:27.723 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:27 smithi100 ceph-mon[32599]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T13:38:27.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.752+0000 7fdf8ce74640 1 -- 172.21.15.100:0/375143001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf8810ba20 msgr2=0x7fdf8810be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:27.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.752+0000 7fdf8ce74640 1 --2- 172.21.15.100:0/375143001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf8810ba20 0x7fdf8810be30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdf78009860 tx=0x7fdf7802a340 comp rx=0 tx=0).stop 2024-07-26T13:38:27.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.752+0000 7fdf8ce74640 1 -- 172.21.15.100:0/375143001 shutdown_connections 2024-07-26T13:38:27.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.752+0000 7fdf8ce74640 1 --2- 172.21.15.100:0/375143001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf88115150 0x7fdf88117570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:27.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.752+0000 7fdf8ce74640 1 --2- 172.21.15.100:0/375143001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf881127f0 0x7fdf88114c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:27.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.752+0000 7fdf8ce74640 1 --2- 172.21.15.100:0/375143001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf8810ba20 0x7fdf8810be30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:27.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.752+0000 7fdf8ce74640 1 -- 172.21.15.100:0/375143001 >> 172.21.15.100:0/375143001 conn(0x7fdf880708f0 msgr2=0x7fdf88070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:27.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.753+0000 7fdf8ce74640 1 -- 172.21.15.100:0/375143001 shutdown_connections 2024-07-26T13:38:27.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.753+0000 7fdf8ce74640 1 -- 172.21.15.100:0/375143001 wait complete. 2024-07-26T13:38:27.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.754+0000 7fdf8ce74640 1 Processor -- start 2024-07-26T13:38:27.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.755+0000 7fdf8ce74640 1 -- start start 2024-07-26T13:38:27.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.755+0000 7fdf8ce74640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf8810ba20 0x7fdf881202a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:27.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.755+0000 7fdf8ce74640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf881127f0 0x7fdf8811a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:27.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.756+0000 7fdf8659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf8810ba20 0x7fdf881202a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:27.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.756+0000 7fdf8ce74640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf88115150 0x7fdf8811aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:27.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.756+0000 7fdf8ce74640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdf88118c70 con 0x7fdf8810ba20 2024-07-26T13:38:27.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.756+0000 7fdf8ce74640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdf88118af0 con 0x7fdf88115150 2024-07-26T13:38:27.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.756+0000 7fdf8ce74640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdf88118df0 con 0x7fdf881127f0 2024-07-26T13:38:27.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.756+0000 7fdf8659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf8810ba20 0x7fdf881202a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40970/0 (socket says 172.21.15.100:40970) 2024-07-26T13:38:27.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.756+0000 7fdf8659c640 1 -- 172.21.15.100:0/776938243 learned_addr learned my addr 172.21.15.100:0/776938243 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:38:27.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.756+0000 7fdf7dd9b640 1 --2- 172.21.15.100:0/776938243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf881127f0 0x7fdf8811a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:27.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.757+0000 7fdf7dd9b640 1 -- 172.21.15.100:0/776938243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf88115150 msgr2=0x7fdf8811aaa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-07-26T13:38:27.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.757+0000 7fdf7dd9b640 1 --2- 172.21.15.100:0/776938243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf88115150 0x7fdf8811aaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:27.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.757+0000 7fdf7dd9b640 1 -- 172.21.15.100:0/776938243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf8810ba20 msgr2=0x7fdf881202a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:27.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.757+0000 7fdf7dd9b640 1 --2- 172.21.15.100:0/776938243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdf8810ba20 0x7fdf881202a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:27.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.757+0000 7fdf7dd9b640 1 -- 172.21.15.100:0/776938243 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf8811b1d0 con 0x7fdf881127f0 2024-07-26T13:38:27.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.758+0000 7fdf7dd9b640 1 --2- 172.21.15.100:0/776938243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf881127f0 0x7fdf8811a560 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdf8806c3a0 tx=0x7fdf7000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:27.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.758+0000 7fdf7ffff640 1 -- 172.21.15.100:0/776938243 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7000cc60 con 0x7fdf881127f0 2024-07-26T13:38:27.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.758+0000 7fdf7ffff640 1 -- 172.21.15.100:0/776938243 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf7000ce00 con 0x7fdf881127f0 2024-07-26T13:38:27.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.758+0000 7fdf7ffff640 1 -- 172.21.15.100:0/776938243 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7001a6f0 con 0x7fdf881127f0 2024-07-26T13:38:27.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.758+0000 7fdf8ce74640 1 -- 172.21.15.100:0/776938243 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf881bd7d0 con 0x7fdf881127f0 2024-07-26T13:38:27.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.758+0000 7fdf8ce74640 1 -- 172.21.15.100:0/776938243 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf881bdbd0 con 0x7fdf881127f0 2024-07-26T13:38:27.761 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:38:27.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.759+0000 7fdf8ce74640 1 -- 172.21.15.100:0/776938243 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdf880742c0 con 0x7fdf881127f0 2024-07-26T13:38:27.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.761+0000 7fdf7ffff640 1 -- 172.21.15.100:0/776938243 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7fdf7001a890 con 0x7fdf881127f0 2024-07-26T13:38:27.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.762+0000 7fdf7ffff640 1 --2- 172.21.15.100:0/776938243 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fdf740b3d60 0x7fdf740b6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:27.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.762+0000 7fdf8659c640 1 --2- 172.21.15.100:0/776938243 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fdf740b3d60 0x7fdf740b6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:27.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.762+0000 7fdf7ffff640 1 -- 172.21.15.100:0/776938243 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fdf700e47e0 con 0x7fdf881127f0 2024-07-26T13:38:27.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.762+0000 7fdf7ffff640 1 --2- 172.21.15.100:0/776938243 >> [v2:172.21.15.138:6800/3090877671,v1:172.21.15.138:6801/3090877671] conn(0x7fdf740bdce0 0x7fdf740c01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:27.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.762+0000 7fdf7ffff640 1 -- 172.21.15.100:0/776938243 --> [v2:172.21.15.138:6800/3090877671,v1:172.21.15.138:6801/3090877671] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdf740c08a0 con 0x7fdf740bdce0 2024-07-26T13:38:27.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.763+0000 7fdf7ffff640 1 -- 172.21.15.100:0/776938243 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fdf700e4b90 con 0x7fdf881127f0 2024-07-26T13:38:27.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.763+0000 7fdf8659c640 1 --2- 172.21.15.100:0/776938243 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fdf740b3d60 0x7fdf740b6270 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdf780095f0 tx=0x7fdf7800b040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:27.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.763+0000 7fdf86d9d640 1 --2- 172.21.15.100:0/776938243 >> [v2:172.21.15.138:6800/3090877671,v1:172.21.15.138:6801/3090877671] conn(0x7fdf740bdce0 0x7fdf740c01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:27.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.764+0000 7fdf86d9d640 1 --2- 172.21.15.100:0/776938243 >> [v2:172.21.15.138:6800/3090877671,v1:172.21.15.138:6801/3090877671] conn(0x7fdf740bdce0 0x7fdf740c01d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:27.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.765+0000 7fdf7ffff640 1 -- 172.21.15.100:0/776938243 <== osd.1 v2:172.21.15.138:6800/3090877671 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdf740c08a0 con 0x7fdf740bdce0 2024-07-26T13:38:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:27 smithi167 ceph-mon[37140]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T13:38:27.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.778+0000 7fdf8ce74640 1 -- 172.21.15.100:0/776938243 --> [v2:172.21.15.138:6800/3090877671,v1:172.21.15.138:6801/3090877671] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdf8810a000 con 0x7fdf740bdce0 2024-07-26T13:38:27.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.779+0000 7fdf7ffff640 1 -- 172.21.15.100:0/776938243 <== osd.1 v2:172.21.15.138:6800/3090877671 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdf8810a000 con 0x7fdf740bdce0 2024-07-26T13:38:27.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.780+0000 7fdf8ce74640 1 -- 172.21.15.100:0/776938243 >> [v2:172.21.15.138:6800/3090877671,v1:172.21.15.138:6801/3090877671] conn(0x7fdf740bdce0 msgr2=0x7fdf740c01d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:27.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.780+0000 7fdf8ce74640 1 --2- 172.21.15.100:0/776938243 >> [v2:172.21.15.138:6800/3090877671,v1:172.21.15.138:6801/3090877671] conn(0x7fdf740bdce0 0x7fdf740c01d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:27.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.780+0000 7fdf8ce74640 1 -- 172.21.15.100:0/776938243 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fdf740b3d60 msgr2=0x7fdf740b6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:27.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.780+0000 7fdf8ce74640 1 --2- 172.21.15.100:0/776938243 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fdf740b3d60 0x7fdf740b6270 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdf780095f0 tx=0x7fdf7800b040 comp rx=0 tx=0).stop 2024-07-26T13:38:27.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.780+0000 7fdf8ce74640 1 -- 172.21.15.100:0/776938243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf881127f0 msgr2=0x7fdf8811a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:27.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.780+0000 7fdf8ce74640 1 --2- 172.21.15.100:0/776938243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf881127f0 0x7fdf8811a560 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdf8806c3a0 tx=0x7fdf7000edf0 comp rx=0 tx=0).stop 2024-07-26T13:38:27.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.780+0000 7fdf86d9d640 1 -- 172.21.15.100:0/776938243 reap_dead start 2024-07-26T13:38:27.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.780+0000 7fdf8ce74640 1 -- 172.21.15.100:0/776938243 shutdown_connections 2024-07-26T13:38:27.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.780+0000 7fdf8ce74640 1 -- 172.21.15.100:0/776938243 >> 172.21.15.100:0/776938243 conn(0x7fdf880708f0 msgr2=0x7fdf88073860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:27.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.780+0000 7fdf8ce74640 1 -- 172.21.15.100:0/776938243 shutdown_connections 2024-07-26T13:38:27.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:27.780+0000 7fdf8ce74640 1 -- 172.21.15.100:0/776938243 wait complete. 2024-07-26T13:38:27.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:27 smithi138 ceph-mon[36954]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T13:38:28.625 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:38:29.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:29 smithi100 ceph-mon[32599]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T13:38:29.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:29 smithi167 ceph-mon[37140]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T13:38:29.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:29 smithi138 ceph-mon[36954]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T13:38:30.083 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:38:31.718 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:31 smithi100 ceph-mon[32599]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:38:31.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:31 smithi167 ceph-mon[37140]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:38:31.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:31 smithi138 ceph-mon[36954]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:38:33.564 INFO:teuthology.orchestra.run.smithi100.stdout:60129542225 2024-07-26T13:38:33.565 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-07-26T13:38:33.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:33 smithi100 ceph-mon[32599]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T13:38:33.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:33 smithi167 ceph-mon[37140]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T13:38:33.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:33 smithi138 ceph-mon[36954]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T13:38:34.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.906+0000 7f8d3007c640 1 -- 172.21.15.100:0/2823384519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d28107dd0 msgr2=0x7f8d281081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:34.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.906+0000 7f8d3007c640 1 --2- 172.21.15.100:0/2823384519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d28107dd0 0x7f8d281081e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8d18009900 tx=0x7f8d1802a410 comp rx=0 tx=0).stop 2024-07-26T13:38:34.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.906+0000 7f8d3007c640 1 -- 172.21.15.100:0/2823384519 shutdown_connections 2024-07-26T13:38:34.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.906+0000 7f8d3007c640 1 --2- 172.21.15.100:0/2823384519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d28108720 0x7f8d2810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:34.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.906+0000 7f8d3007c640 1 --2- 172.21.15.100:0/2823384519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d28107dd0 0x7f8d281081e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:34.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.906+0000 7f8d3007c640 1 --2- 172.21.15.100:0/2823384519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d28102490 0x7f8d281028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:34.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.906+0000 7f8d3007c640 1 -- 172.21.15.100:0/2823384519 >> 172.21.15.100:0/2823384519 conn(0x7f8d28079160 msgr2=0x7f8d28079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:34.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.906+0000 7f8d3007c640 1 -- 172.21.15.100:0/2823384519 shutdown_connections 2024-07-26T13:38:34.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.906+0000 7f8d3007c640 1 -- 172.21.15.100:0/2823384519 wait complete. 2024-07-26T13:38:34.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.907+0000 7f8d3007c640 1 Processor -- start 2024-07-26T13:38:34.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.907+0000 7f8d3007c640 1 -- start start 2024-07-26T13:38:34.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.907+0000 7f8d3007c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d28102490 0x7f8d28197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:34.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.907+0000 7f8d3007c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d28107dd0 0x7f8d28198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:34.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.907+0000 7f8d3007c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d28108720 0x7f8d2819d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:34.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.907+0000 7f8d3007c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8d28111290 con 0x7f8d28102490 2024-07-26T13:38:34.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.907+0000 7f8d3007c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8d28111110 con 0x7f8d28107dd0 2024-07-26T13:38:34.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.907+0000 7f8d3007c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8d28111410 con 0x7f8d28108720 2024-07-26T13:38:34.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.907+0000 7f8d2de18640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d28102490 0x7f8d28197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:34.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.907+0000 7f8d2de18640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d28102490 0x7f8d28197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40992/0 (socket says 172.21.15.100:40992) 2024-07-26T13:38:34.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.907+0000 7f8d2de18640 1 -- 172.21.15.100:0/3919982474 learned_addr learned my addr 172.21.15.100:0/3919982474 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:38:34.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.907+0000 7f8d2d617640 1 --2- 172.21.15.100:0/3919982474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d28107dd0 0x7f8d28198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:34.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.908+0000 7f8d2e619640 1 --2- 172.21.15.100:0/3919982474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d28108720 0x7f8d2819d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:34.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.908+0000 7f8d2de18640 1 -- 172.21.15.100:0/3919982474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d28108720 msgr2=0x7f8d2819d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:34.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.908+0000 7f8d2de18640 1 --2- 172.21.15.100:0/3919982474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d28108720 0x7f8d2819d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:34.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.908+0000 7f8d2de18640 1 -- 172.21.15.100:0/3919982474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d28107dd0 msgr2=0x7f8d28198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:34.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.908+0000 7f8d2de18640 1 --2- 172.21.15.100:0/3919982474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d28107dd0 0x7f8d28198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:34.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.908+0000 7f8d2de18640 1 -- 172.21.15.100:0/3919982474 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d2819dbf0 con 0x7f8d28102490 2024-07-26T13:38:34.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.908+0000 7f8d2e619640 1 --2- 172.21.15.100:0/3919982474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d28108720 0x7f8d2819d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:38:34.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.908+0000 7f8d2de18640 1 --2- 172.21.15.100:0/3919982474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d28102490 0x7f8d28197c80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400ea60 tx=0x7f8d2400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:34.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.908+0000 7f8d1effd640 1 -- 172.21.15.100:0/3919982474 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d2400cde0 con 0x7f8d28102490 2024-07-26T13:38:34.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.908+0000 7f8d1effd640 1 -- 172.21.15.100:0/3919982474 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d240035f0 con 0x7f8d28102490 2024-07-26T13:38:34.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.908+0000 7f8d3007c640 1 -- 172.21.15.100:0/3919982474 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d2819dee0 con 0x7f8d28102490 2024-07-26T13:38:34.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.908+0000 7f8d1effd640 1 -- 172.21.15.100:0/3919982474 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d24010690 con 0x7f8d28102490 2024-07-26T13:38:34.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.908+0000 7f8d3007c640 1 -- 172.21.15.100:0/3919982474 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d281a4640 con 0x7f8d28102490 2024-07-26T13:38:34.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.909+0000 7f8d3007c640 1 -- 172.21.15.100:0/3919982474 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d28103710 con 0x7f8d28102490 2024-07-26T13:38:34.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.911+0000 7f8d1effd640 1 -- 172.21.15.100:0/3919982474 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f8d240037a0 con 0x7f8d28102490 2024-07-26T13:38:34.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.911+0000 7f8d1effd640 1 --2- 172.21.15.100:0/3919982474 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f8d140b3d30 0x7f8d140b6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:34.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.912+0000 7f8d2d617640 1 --2- 172.21.15.100:0/3919982474 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f8d140b3d30 0x7f8d140b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:34.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.912+0000 7f8d1effd640 1 -- 172.21.15.100:0/3919982474 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f8d240e46e0 con 0x7f8d28102490 2024-07-26T13:38:34.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.912+0000 7f8d2d617640 1 --2- 172.21.15.100:0/3919982474 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f8d140b3d30 0x7f8d140b6240 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8d18000c00 tx=0x7f8d18031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:34.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:34.915+0000 7f8d1effd640 1 -- 172.21.15.100:0/3919982474 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f8d240aba70 con 0x7f8d28102490 2024-07-26T13:38:35.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.099+0000 7f8d3007c640 1 -- 172.21.15.100:0/3919982474 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f8d281028a0 con 0x7f8d28102490 2024-07-26T13:38:35.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.100+0000 7f8d1effd640 1 -- 172.21.15.100:0/3919982474 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8d281028a0 con 0x7f8d28102490 2024-07-26T13:38:35.101 INFO:teuthology.orchestra.run.smithi100.stdout:257698037805 2024-07-26T13:38:35.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.102+0000 7f8d3007c640 1 -- 172.21.15.100:0/3919982474 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f8d140b3d30 msgr2=0x7f8d140b6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:35.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.102+0000 7f8d3007c640 1 --2- 172.21.15.100:0/3919982474 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f8d140b3d30 0x7f8d140b6240 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8d18000c00 tx=0x7f8d18031400 comp rx=0 tx=0).stop 2024-07-26T13:38:35.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.102+0000 7f8d3007c640 1 -- 172.21.15.100:0/3919982474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d28102490 msgr2=0x7f8d28197c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:35.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.102+0000 7f8d3007c640 1 --2- 172.21.15.100:0/3919982474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d28102490 0x7f8d28197c80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400ea60 tx=0x7f8d2400ef20 comp rx=0 tx=0).stop 2024-07-26T13:38:35.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.102+0000 7f8d3007c640 1 -- 172.21.15.100:0/3919982474 shutdown_connections 2024-07-26T13:38:35.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.102+0000 7f8d3007c640 1 --2- 172.21.15.100:0/3919982474 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f8d140b3d30 0x7f8d140b6240 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:35.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.102+0000 7f8d3007c640 1 --2- 172.21.15.100:0/3919982474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d28108720 0x7f8d2819d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:35.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.102+0000 7f8d3007c640 1 --2- 172.21.15.100:0/3919982474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d28107dd0 0x7f8d28198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:35.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.102+0000 7f8d3007c640 1 --2- 172.21.15.100:0/3919982474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d28102490 0x7f8d28197c80 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:35.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.102+0000 7f8d3007c640 1 -- 172.21.15.100:0/3919982474 >> 172.21.15.100:0/3919982474 conn(0x7f8d28079160 msgr2=0x7f8d28103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:35.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.102+0000 7f8d3007c640 1 -- 172.21.15.100:0/3919982474 shutdown_connections 2024-07-26T13:38:35.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.102+0000 7f8d3007c640 1 -- 172.21.15.100:0/3919982474 wait complete. 2024-07-26T13:38:35.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.166+0000 7fc159c7d640 1 -- 172.21.15.100:0/3798907445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc154102490 msgr2=0x7fc1541028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:35.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.166+0000 7fc159c7d640 1 --2- 172.21.15.100:0/3798907445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc154102490 0x7fc1541028a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc144009ca0 tx=0x7fc14402a8c0 comp rx=0 tx=0).stop 2024-07-26T13:38:35.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.166+0000 7fc159c7d640 1 -- 172.21.15.100:0/3798907445 shutdown_connections 2024-07-26T13:38:35.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.166+0000 7fc159c7d640 1 --2- 172.21.15.100:0/3798907445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc154108720 0x7fc15410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:35.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.166+0000 7fc159c7d640 1 --2- 172.21.15.100:0/3798907445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc154107dd0 0x7fc1541081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:35.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.166+0000 7fc159c7d640 1 --2- 172.21.15.100:0/3798907445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc154102490 0x7fc1541028a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:35.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.166+0000 7fc159c7d640 1 -- 172.21.15.100:0/3798907445 >> 172.21.15.100:0/3798907445 conn(0x7fc154079160 msgr2=0x7fc154079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:35.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.166+0000 7fc159c7d640 1 -- 172.21.15.100:0/3798907445 shutdown_connections 2024-07-26T13:38:35.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.166+0000 7fc159c7d640 1 -- 172.21.15.100:0/3798907445 wait complete. 2024-07-26T13:38:35.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.167+0000 7fc159c7d640 1 Processor -- start 2024-07-26T13:38:35.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.167+0000 7fc159c7d640 1 -- start start 2024-07-26T13:38:35.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.168+0000 7fc159c7d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc154102490 0x7fc154199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:35.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.168+0000 7fc159c7d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc154107dd0 0x7fc15419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:35.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.168+0000 7fc159c7d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc154108720 0x7fc15419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:35.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.168+0000 7fc159c7d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc1541123a0 con 0x7fc154102490 2024-07-26T13:38:35.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.168+0000 7fc159c7d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc154112220 con 0x7fc154108720 2024-07-26T13:38:35.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.168+0000 7fc159c7d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc154112520 con 0x7fc154107dd0 2024-07-26T13:38:35.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.168+0000 7fc1537fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc154102490 0x7fc154199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:35.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.168+0000 7fc1537fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc154102490 0x7fc154199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55142/0 (socket says 172.21.15.100:55142) 2024-07-26T13:38:35.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.168+0000 7fc1537fe640 1 -- 172.21.15.100:0/4130669533 learned_addr learned my addr 172.21.15.100:0/4130669533 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:38:35.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.168+0000 7fc152ffd640 1 --2- 172.21.15.100:0/4130669533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc154107dd0 0x7fc15419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:35.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.168+0000 7fc153fff640 1 --2- 172.21.15.100:0/4130669533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc154108720 0x7fc15419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:35.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.168+0000 7fc1537fe640 1 -- 172.21.15.100:0/4130669533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc154107dd0 msgr2=0x7fc15419a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:35.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.168+0000 7fc1537fe640 1 --2- 172.21.15.100:0/4130669533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc154107dd0 0x7fc15419a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:35.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.168+0000 7fc1537fe640 1 -- 172.21.15.100:0/4130669533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc154108720 msgr2=0x7fc15419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:35.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.168+0000 7fc1537fe640 1 --2- 172.21.15.100:0/4130669533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc154108720 0x7fc15419f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:35.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.168+0000 7fc1537fe640 1 -- 172.21.15.100:0/4130669533 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc15419fdf0 con 0x7fc154102490 2024-07-26T13:38:35.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.168+0000 7fc1537fe640 1 --2- 172.21.15.100:0/4130669533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc154102490 0x7fc154199e80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc14402ade0 tx=0x7fc144031770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:35.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.169+0000 7fc150ff9640 1 -- 172.21.15.100:0/4130669533 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc144003b70 con 0x7fc154102490 2024-07-26T13:38:35.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.169+0000 7fc159c7d640 1 -- 172.21.15.100:0/4130669533 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1541a0080 con 0x7fc154102490 2024-07-26T13:38:35.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.169+0000 7fc150ff9640 1 -- 172.21.15.100:0/4130669533 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc144003d10 con 0x7fc154102490 2024-07-26T13:38:35.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.169+0000 7fc150ff9640 1 -- 172.21.15.100:0/4130669533 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14402fa50 con 0x7fc154102490 2024-07-26T13:38:35.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.169+0000 7fc159c7d640 1 -- 172.21.15.100:0/4130669533 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1541a6840 con 0x7fc154102490 2024-07-26T13:38:35.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.170+0000 7fc159c7d640 1 -- 172.21.15.100:0/4130669533 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1541028a0 con 0x7fc154102490 2024-07-26T13:38:35.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.171+0000 7fc150ff9640 1 -- 172.21.15.100:0/4130669533 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7fc144004580 con 0x7fc154102490 2024-07-26T13:38:35.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.172+0000 7fc150ff9640 1 --2- 172.21.15.100:0/4130669533 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc1400b3cb0 0x7fc1400b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:35.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.172+0000 7fc152ffd640 1 --2- 172.21.15.100:0/4130669533 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc1400b3cb0 0x7fc1400b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:35.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.172+0000 7fc150ff9640 1 -- 172.21.15.100:0/4130669533 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fc144105330 con 0x7fc154102490 2024-07-26T13:38:35.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.173+0000 7fc152ffd640 1 --2- 172.21.15.100:0/4130669533 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc1400b3cb0 0x7fc1400b61c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc154079620 tx=0x7fc13c00b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:35.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.177+0000 7fc150ff9640 1 -- 172.21.15.100:0/4130669533 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fc1440cc7d0 con 0x7fc154102490 2024-07-26T13:38:35.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.364+0000 7fc159c7d640 1 -- 172.21.15.100:0/4130669533 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fc1541a6ae0 con 0x7fc154102490 2024-07-26T13:38:35.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.365+0000 7fc150ff9640 1 -- 172.21.15.100:0/4130669533 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc1440cc0d0 con 0x7fc154102490 2024-07-26T13:38:35.367 INFO:teuthology.orchestra.run.smithi100.stdout:206158430261 2024-07-26T13:38:35.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.367+0000 7fc159c7d640 1 -- 172.21.15.100:0/4130669533 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc1400b3cb0 msgr2=0x7fc1400b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:35.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.367+0000 7fc159c7d640 1 --2- 172.21.15.100:0/4130669533 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc1400b3cb0 0x7fc1400b61c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc154079620 tx=0x7fc13c00b390 comp rx=0 tx=0).stop 2024-07-26T13:38:35.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.367+0000 7fc159c7d640 1 -- 172.21.15.100:0/4130669533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc154102490 msgr2=0x7fc154199e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:35.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.367+0000 7fc159c7d640 1 --2- 172.21.15.100:0/4130669533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc154102490 0x7fc154199e80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc14402ade0 tx=0x7fc144031770 comp rx=0 tx=0).stop 2024-07-26T13:38:35.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.367+0000 7fc159c7d640 1 -- 172.21.15.100:0/4130669533 shutdown_connections 2024-07-26T13:38:35.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.367+0000 7fc159c7d640 1 --2- 172.21.15.100:0/4130669533 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc1400b3cb0 0x7fc1400b61c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:35.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.367+0000 7fc159c7d640 1 --2- 172.21.15.100:0/4130669533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc154108720 0x7fc15419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:35.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.367+0000 7fc159c7d640 1 --2- 172.21.15.100:0/4130669533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc154107dd0 0x7fc15419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:35.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.367+0000 7fc159c7d640 1 --2- 172.21.15.100:0/4130669533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc154102490 0x7fc154199e80 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:35.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.367+0000 7fc159c7d640 1 -- 172.21.15.100:0/4130669533 >> 172.21.15.100:0/4130669533 conn(0x7fc154079160 msgr2=0x7fc1541014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:35.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.367+0000 7fc159c7d640 1 -- 172.21.15.100:0/4130669533 shutdown_connections 2024-07-26T13:38:35.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:35.367+0000 7fc159c7d640 1 -- 172.21.15.100:0/4130669533 wait complete. 2024-07-26T13:38:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:35 smithi100 ceph-mon[32599]: pgmap v318: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:38:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:35 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/3919982474' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-07-26T13:38:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:35 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/4130669533' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-07-26T13:38:35.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:35 smithi167 ceph-mon[37140]: pgmap v318: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:38:35.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:35 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/3919982474' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-07-26T13:38:35.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:35 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/4130669533' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-07-26T13:38:35.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:35 smithi138 ceph-mon[36954]: pgmap v318: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:38:35.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:35 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/3919982474' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-07-26T13:38:35.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:35 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/4130669533' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-07-26T13:38:36.788 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430258 got 206158430261 for osd.7 2024-07-26T13:38:36.789 DEBUG:teuthology.parallel:result is None 2024-07-26T13:38:37.709 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:37 smithi100 ceph-mon[32599]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-07-26T13:38:37.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:37 smithi167 ceph-mon[37140]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-07-26T13:38:37.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:37 smithi138 ceph-mon[36954]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-07-26T13:38:38.643 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037800 got 257698037805 for osd.9 2024-07-26T13:38:38.643 DEBUG:teuthology.parallel:result is None 2024-07-26T13:38:39.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.223+0000 7ff653e9d640 1 -- 172.21.15.100:0/1659512318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff64c102490 msgr2=0x7ff64c1028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:39.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.223+0000 7ff653e9d640 1 --2- 172.21.15.100:0/1659512318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff64c102490 0x7ff64c1028a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff648009900 tx=0x7ff64802a410 comp rx=0 tx=0).stop 2024-07-26T13:38:39.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.223+0000 7ff653e9d640 1 -- 172.21.15.100:0/1659512318 shutdown_connections 2024-07-26T13:38:39.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.223+0000 7ff653e9d640 1 --2- 172.21.15.100:0/1659512318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff64c108720 0x7ff64c10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:39.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.223+0000 7ff653e9d640 1 --2- 172.21.15.100:0/1659512318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff64c107dd0 0x7ff64c1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:39.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.223+0000 7ff653e9d640 1 --2- 172.21.15.100:0/1659512318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff64c102490 0x7ff64c1028a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:39.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.223+0000 7ff653e9d640 1 -- 172.21.15.100:0/1659512318 >> 172.21.15.100:0/1659512318 conn(0x7ff64c079160 msgr2=0x7ff64c079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:39.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.224+0000 7ff653e9d640 1 -- 172.21.15.100:0/1659512318 shutdown_connections 2024-07-26T13:38:39.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.224+0000 7ff653e9d640 1 -- 172.21.15.100:0/1659512318 wait complete. 2024-07-26T13:38:39.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.224+0000 7ff653e9d640 1 Processor -- start 2024-07-26T13:38:39.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.224+0000 7ff653e9d640 1 -- start start 2024-07-26T13:38:39.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.225+0000 7ff653e9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff64c102490 0x7ff64c199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:39.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.225+0000 7ff653e9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff64c107dd0 0x7ff64c19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:39.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.225+0000 7ff653e9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff64c108720 0x7ff64c19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:39.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.225+0000 7ff653e9d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c1123a0 con 0x7ff64c108720 2024-07-26T13:38:39.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.225+0000 7ff653e9d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c112220 con 0x7ff64c102490 2024-07-26T13:38:39.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.225+0000 7ff653e9d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c112520 con 0x7ff64c107dd0 2024-07-26T13:38:39.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.225+0000 7ff65243a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff64c108720 0x7ff64c19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:39.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.225+0000 7ff65243a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff64c108720 0x7ff64c19f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55162/0 (socket says 172.21.15.100:55162) 2024-07-26T13:38:39.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.225+0000 7ff65243a640 1 -- 172.21.15.100:0/3507004213 learned_addr learned my addr 172.21.15.100:0/3507004213 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:38:39.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.225+0000 7ff651c39640 1 --2- 172.21.15.100:0/3507004213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff64c102490 0x7ff64c199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:39.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.225+0000 7ff651438640 1 --2- 172.21.15.100:0/3507004213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff64c107dd0 0x7ff64c19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:39.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.225+0000 7ff65243a640 1 -- 172.21.15.100:0/3507004213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff64c107dd0 msgr2=0x7ff64c19a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:39.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.225+0000 7ff65243a640 1 --2- 172.21.15.100:0/3507004213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff64c107dd0 0x7ff64c19a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:39.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.225+0000 7ff65243a640 1 -- 172.21.15.100:0/3507004213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff64c102490 msgr2=0x7ff64c199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:39.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.225+0000 7ff65243a640 1 --2- 172.21.15.100:0/3507004213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff64c102490 0x7ff64c199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:39.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.225+0000 7ff65243a640 1 -- 172.21.15.100:0/3507004213 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff64c19fdf0 con 0x7ff64c108720 2024-07-26T13:38:39.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.225+0000 7ff65243a640 1 --2- 172.21.15.100:0/3507004213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff64c108720 0x7ff64c19f6c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff64400eaa0 tx=0x7ff64400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:39.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.225+0000 7ff642ffd640 1 -- 172.21.15.100:0/3507004213 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64400cdb0 con 0x7ff64c108720 2024-07-26T13:38:39.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.225+0000 7ff642ffd640 1 -- 172.21.15.100:0/3507004213 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff644005020 con 0x7ff64c108720 2024-07-26T13:38:39.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.225+0000 7ff653e9d640 1 -- 172.21.15.100:0/3507004213 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff64c1a00e0 con 0x7ff64c108720 2024-07-26T13:38:39.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.225+0000 7ff642ffd640 1 -- 172.21.15.100:0/3507004213 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6440116d0 con 0x7ff64c108720 2024-07-26T13:38:39.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.225+0000 7ff653e9d640 1 -- 172.21.15.100:0/3507004213 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff64c1a6840 con 0x7ff64c108720 2024-07-26T13:38:39.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.227+0000 7ff653e9d640 1 -- 172.21.15.100:0/3507004213 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff64c1028a0 con 0x7ff64c108720 2024-07-26T13:38:39.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.228+0000 7ff642ffd640 1 -- 172.21.15.100:0/3507004213 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7ff6440051d0 con 0x7ff64c108720 2024-07-26T13:38:39.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.229+0000 7ff642ffd640 1 --2- 172.21.15.100:0/3507004213 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff6380b3d30 0x7ff6380b6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:39.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.229+0000 7ff651c39640 1 --2- 172.21.15.100:0/3507004213 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff6380b3d30 0x7ff6380b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:39.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.229+0000 7ff642ffd640 1 -- 172.21.15.100:0/3507004213 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7ff6440e4330 con 0x7ff64c108720 2024-07-26T13:38:39.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.230+0000 7ff651c39640 1 --2- 172.21.15.100:0/3507004213 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff6380b3d30 0x7ff6380b6240 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff648031440 tx=0x7ff648009560 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:39.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.232+0000 7ff642ffd640 1 -- 172.21.15.100:0/3507004213 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7ff644016030 con 0x7ff64c108720 2024-07-26T13:38:39.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.413+0000 7ff653e9d640 1 -- 172.21.15.100:0/3507004213 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7ff64c19afb0 con 0x7ff64c108720 2024-07-26T13:38:39.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.413+0000 7ff642ffd640 1 -- 172.21.15.100:0/3507004213 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7ff64c19afb0 con 0x7ff64c108720 2024-07-26T13:38:39.415 INFO:teuthology.orchestra.run.smithi100.stdout:279172874280 2024-07-26T13:38:39.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.416+0000 7ff653e9d640 1 -- 172.21.15.100:0/3507004213 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff6380b3d30 msgr2=0x7ff6380b6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:39.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.416+0000 7ff653e9d640 1 --2- 172.21.15.100:0/3507004213 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff6380b3d30 0x7ff6380b6240 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff648031440 tx=0x7ff648009560 comp rx=0 tx=0).stop 2024-07-26T13:38:39.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.416+0000 7ff653e9d640 1 -- 172.21.15.100:0/3507004213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff64c108720 msgr2=0x7ff64c19f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:39.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.416+0000 7ff653e9d640 1 --2- 172.21.15.100:0/3507004213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff64c108720 0x7ff64c19f6c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff64400eaa0 tx=0x7ff64400ef60 comp rx=0 tx=0).stop 2024-07-26T13:38:39.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.416+0000 7ff653e9d640 1 -- 172.21.15.100:0/3507004213 shutdown_connections 2024-07-26T13:38:39.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.416+0000 7ff653e9d640 1 --2- 172.21.15.100:0/3507004213 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff6380b3d30 0x7ff6380b6240 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:39.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.416+0000 7ff653e9d640 1 --2- 172.21.15.100:0/3507004213 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff64c108720 0x7ff64c19f6c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:39.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.416+0000 7ff653e9d640 1 --2- 172.21.15.100:0/3507004213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff64c107dd0 0x7ff64c19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:39.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.416+0000 7ff653e9d640 1 --2- 172.21.15.100:0/3507004213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff64c102490 0x7ff64c199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:39.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.416+0000 7ff653e9d640 1 -- 172.21.15.100:0/3507004213 >> 172.21.15.100:0/3507004213 conn(0x7ff64c079160 msgr2=0x7ff64c1014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:39.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.416+0000 7ff653e9d640 1 -- 172.21.15.100:0/3507004213 shutdown_connections 2024-07-26T13:38:39.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.416+0000 7ff653e9d640 1 -- 172.21.15.100:0/3507004213 wait complete. 2024-07-26T13:38:39.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.616+0000 7f35af2e8640 1 -- 172.21.15.100:0/2152731495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35a8107f60 msgr2=0x7f35a810e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:39.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.616+0000 7f35af2e8640 1 --2- 172.21.15.100:0/2152731495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35a8107f60 0x7f35a810e790 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f35a0009900 tx=0x7f35a002a410 comp rx=0 tx=0).stop 2024-07-26T13:38:39.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.617+0000 7f35af2e8640 1 -- 172.21.15.100:0/2152731495 shutdown_connections 2024-07-26T13:38:39.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.617+0000 7f35af2e8640 1 --2- 172.21.15.100:0/2152731495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35a8107f60 0x7f35a810e790 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:39.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.617+0000 7f35af2e8640 1 --2- 172.21.15.100:0/2152731495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35a8107610 0x7f35a8107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:39.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.617+0000 7f35af2e8640 1 --2- 172.21.15.100:0/2152731495 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35a8101cd0 0x7f35a81020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:39.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.617+0000 7f35af2e8640 1 -- 172.21.15.100:0/2152731495 >> 172.21.15.100:0/2152731495 conn(0x7f35a80798f0 msgr2=0x7f35a8079d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:39.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.617+0000 7f35af2e8640 1 -- 172.21.15.100:0/2152731495 shutdown_connections 2024-07-26T13:38:39.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.617+0000 7f35af2e8640 1 -- 172.21.15.100:0/2152731495 wait complete. 2024-07-26T13:38:39.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.617+0000 7f35af2e8640 1 Processor -- start 2024-07-26T13:38:39.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.617+0000 7f35af2e8640 1 -- start start 2024-07-26T13:38:39.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.618+0000 7f35af2e8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35a8101cd0 0x7f35a819adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:39.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.618+0000 7f35af2e8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35a8107610 0x7f35a819b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:39.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.618+0000 7f35af2e8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35a8107f60 0x7f35a819b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:39.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.618+0000 7f35af2e8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f35a8111c80 con 0x7f35a8107f60 2024-07-26T13:38:39.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.618+0000 7f35af2e8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f35a8111b00 con 0x7f35a8101cd0 2024-07-26T13:38:39.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.618+0000 7f35af2e8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f35a8111e00 con 0x7f35a8107610 2024-07-26T13:38:39.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.618+0000 7f35ad885640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35a8107f60 0x7f35a819b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:39.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.618+0000 7f35ad885640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35a8107f60 0x7f35a819b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55180/0 (socket says 172.21.15.100:55180) 2024-07-26T13:38:39.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.618+0000 7f35ad885640 1 -- 172.21.15.100:0/611422008 learned_addr learned my addr 172.21.15.100:0/611422008 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:38:39.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.618+0000 7f35ad885640 1 -- 172.21.15.100:0/611422008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35a8107610 msgr2=0x7f35a819b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:39.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.618+0000 7f35ad084640 1 --2- 172.21.15.100:0/611422008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35a8101cd0 0x7f35a819adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:39.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.618+0000 7f35ac883640 1 --2- 172.21.15.100:0/611422008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35a8107610 0x7f35a819b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:39.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.618+0000 7f35ad885640 1 --2- 172.21.15.100:0/611422008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35a8107610 0x7f35a819b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:39.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.618+0000 7f35ad885640 1 -- 172.21.15.100:0/611422008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35a8101cd0 msgr2=0x7f35a819adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:39.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.618+0000 7f35ad885640 1 --2- 172.21.15.100:0/611422008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35a8101cd0 0x7f35a819adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:39.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.618+0000 7f35ad885640 1 -- 172.21.15.100:0/611422008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a81a6050 con 0x7f35a8107f60 2024-07-26T13:38:39.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.618+0000 7f35ac883640 1 --2- 172.21.15.100:0/611422008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35a8107610 0x7f35a819b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:38:39.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.618+0000 7f35ad084640 1 --2- 172.21.15.100:0/611422008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35a8101cd0 0x7f35a819adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:38:39.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.618+0000 7f35ad885640 1 --2- 172.21.15.100:0/611422008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35a8107f60 0x7f35a819b840 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f35a0009a20 tx=0x7f35a0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:39.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.618+0000 7f35a67fc640 1 -- 172.21.15.100:0/611422008 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a003f030 con 0x7f35a8107f60 2024-07-26T13:38:39.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.619+0000 7f35af2e8640 1 -- 172.21.15.100:0/611422008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35a81a62e0 con 0x7f35a8107f60 2024-07-26T13:38:39.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.619+0000 7f35a67fc640 1 -- 172.21.15.100:0/611422008 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35a0003aa0 con 0x7f35a8107f60 2024-07-26T13:38:39.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.619+0000 7f35a67fc640 1 -- 172.21.15.100:0/611422008 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a0035070 con 0x7f35a8107f60 2024-07-26T13:38:39.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.619+0000 7f35af2e8640 1 -- 172.21.15.100:0/611422008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35a81a67c0 con 0x7f35a8107f60 2024-07-26T13:38:39.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.619+0000 7f35af2e8640 1 -- 172.21.15.100:0/611422008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35a81020e0 con 0x7f35a8107f60 2024-07-26T13:38:39.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.621+0000 7f35a67fc640 1 -- 172.21.15.100:0/611422008 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f35a0042030 con 0x7f35a8107f60 2024-07-26T13:38:39.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.622+0000 7f35a67fc640 1 --2- 172.21.15.100:0/611422008 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f35900b3ce0 0x7f35900b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:39.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.622+0000 7f35ad084640 1 --2- 172.21.15.100:0/611422008 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f35900b3ce0 0x7f35900b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:39.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.622+0000 7f35a67fc640 1 -- 172.21.15.100:0/611422008 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f35a0104990 con 0x7f35a8107f60 2024-07-26T13:38:39.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.622+0000 7f35ad084640 1 --2- 172.21.15.100:0/611422008 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f35900b3ce0 0x7f35900b61f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3598005fd0 tx=0x7f3598008310 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:39.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.625+0000 7f35a67fc640 1 -- 172.21.15.100:0/611422008 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f35a00cbd80 con 0x7f35a8107f60 2024-07-26T13:38:39.737 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:39 smithi100 ceph-mon[32599]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T13:38:39.737 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:39 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/3507004213' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-07-26T13:38:39.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:39 smithi167 ceph-mon[37140]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T13:38:39.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:39 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/3507004213' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-07-26T13:38:39.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.806+0000 7f35af2e8640 1 -- 172.21.15.100:0/611422008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f35a81a6b30 con 0x7f35a8107f60 2024-07-26T13:38:39.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.806+0000 7f35a67fc640 1 -- 172.21.15.100:0/611422008 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f35a00cb680 con 0x7f35a8107f60 2024-07-26T13:38:39.807 INFO:teuthology.orchestra.run.smithi100.stdout:304942678052 2024-07-26T13:38:39.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.808+0000 7f35af2e8640 1 -- 172.21.15.100:0/611422008 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f35900b3ce0 msgr2=0x7f35900b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:39.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.808+0000 7f35af2e8640 1 --2- 172.21.15.100:0/611422008 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f35900b3ce0 0x7f35900b61f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3598005fd0 tx=0x7f3598008310 comp rx=0 tx=0).stop 2024-07-26T13:38:39.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.808+0000 7f35af2e8640 1 -- 172.21.15.100:0/611422008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35a8107f60 msgr2=0x7f35a819b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:39.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.808+0000 7f35af2e8640 1 --2- 172.21.15.100:0/611422008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35a8107f60 0x7f35a819b840 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f35a0009a20 tx=0x7f35a0004270 comp rx=0 tx=0).stop 2024-07-26T13:38:39.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.808+0000 7f35af2e8640 1 -- 172.21.15.100:0/611422008 shutdown_connections 2024-07-26T13:38:39.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.808+0000 7f35af2e8640 1 --2- 172.21.15.100:0/611422008 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f35900b3ce0 0x7f35900b61f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:39.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.808+0000 7f35af2e8640 1 --2- 172.21.15.100:0/611422008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35a8107f60 0x7f35a819b840 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:39.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.808+0000 7f35af2e8640 1 --2- 172.21.15.100:0/611422008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35a8107610 0x7f35a819b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:39.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.808+0000 7f35af2e8640 1 --2- 172.21.15.100:0/611422008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35a8101cd0 0x7f35a819adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:39.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.808+0000 7f35af2e8640 1 -- 172.21.15.100:0/611422008 >> 172.21.15.100:0/611422008 conn(0x7f35a80798f0 msgr2=0x7f35a8100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:39.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.808+0000 7f35af2e8640 1 -- 172.21.15.100:0/611422008 shutdown_connections 2024-07-26T13:38:39.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:39.808+0000 7f35af2e8640 1 -- 172.21.15.100:0/611422008 wait complete. 2024-07-26T13:38:39.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:39 smithi138 ceph-mon[36954]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T13:38:39.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:39 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/3507004213' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-07-26T13:38:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:40 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/611422008' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-07-26T13:38:40.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:40 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/611422008' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-07-26T13:38:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:40 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/611422008' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-07-26T13:38:41.131 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678046 got 304942678052 for osd.11 2024-07-26T13:38:41.131 DEBUG:teuthology.parallel:result is None 2024-07-26T13:38:41.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.501+0000 7feb23c70640 1 -- 172.21.15.100:0/2505275353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb1c108720 msgr2=0x7feb1c10ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:41.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.501+0000 7feb23c70640 1 --2- 172.21.15.100:0/2505275353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb1c108720 0x7feb1c10ef50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7feb14009ca0 tx=0x7feb1402a8c0 comp rx=0 tx=0).stop 2024-07-26T13:38:41.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.501+0000 7feb23c70640 1 -- 172.21.15.100:0/2505275353 shutdown_connections 2024-07-26T13:38:41.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.501+0000 7feb23c70640 1 --2- 172.21.15.100:0/2505275353 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb1c108720 0x7feb1c10ef50 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:41.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.501+0000 7feb23c70640 1 --2- 172.21.15.100:0/2505275353 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb1c107dd0 0x7feb1c1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:41.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.501+0000 7feb23c70640 1 --2- 172.21.15.100:0/2505275353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb1c102490 0x7feb1c1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:41.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.501+0000 7feb23c70640 1 -- 172.21.15.100:0/2505275353 >> 172.21.15.100:0/2505275353 conn(0x7feb1c079160 msgr2=0x7feb1c079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:41.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.501+0000 7feb23c70640 1 -- 172.21.15.100:0/2505275353 shutdown_connections 2024-07-26T13:38:41.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.501+0000 7feb23c70640 1 -- 172.21.15.100:0/2505275353 wait complete. 2024-07-26T13:38:41.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.502+0000 7feb23c70640 1 Processor -- start 2024-07-26T13:38:41.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.502+0000 7feb23c70640 1 -- start start 2024-07-26T13:38:41.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.502+0000 7feb23c70640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb1c102490 0x7feb1c199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:41.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.502+0000 7feb23c70640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb1c107dd0 0x7feb1c19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:41.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.502+0000 7feb23c70640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb1c108720 0x7feb1c19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:41.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.502+0000 7feb23c70640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1123f0 con 0x7feb1c107dd0 2024-07-26T13:38:41.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.502+0000 7feb23c70640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c112270 con 0x7feb1c108720 2024-07-26T13:38:41.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.502+0000 7feb23c70640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c112570 con 0x7feb1c102490 2024-07-26T13:38:41.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.502+0000 7feb2120b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb1c107dd0 0x7feb1c19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:41.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.502+0000 7feb2120b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb1c107dd0 0x7feb1c19a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55200/0 (socket says 172.21.15.100:55200) 2024-07-26T13:38:41.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.502+0000 7feb2120b640 1 -- 172.21.15.100:0/2054628119 learned_addr learned my addr 172.21.15.100:0/2054628119 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:38:41.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.502+0000 7feb21a0c640 1 --2- 172.21.15.100:0/2054628119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb1c102490 0x7feb1c199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:41.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.502+0000 7feb2220d640 1 --2- 172.21.15.100:0/2054628119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb1c108720 0x7feb1c19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:41.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.502+0000 7feb2120b640 1 -- 172.21.15.100:0/2054628119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb1c102490 msgr2=0x7feb1c199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:41.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.503+0000 7feb2120b640 1 --2- 172.21.15.100:0/2054628119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb1c102490 0x7feb1c199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:41.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.503+0000 7feb2120b640 1 -- 172.21.15.100:0/2054628119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb1c108720 msgr2=0x7feb1c19f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:41.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.503+0000 7feb2120b640 1 --2- 172.21.15.100:0/2054628119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb1c108720 0x7feb1c19f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:41.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.503+0000 7feb2120b640 1 -- 172.21.15.100:0/2054628119 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb1c19fdc0 con 0x7feb1c107dd0 2024-07-26T13:38:41.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.503+0000 7feb2120b640 1 --2- 172.21.15.100:0/2054628119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb1c107dd0 0x7feb1c19a4b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7feb0c00b450 tx=0x7feb0c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:41.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.503+0000 7feb12ffd640 1 -- 172.21.15.100:0/2054628119 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c015020 con 0x7feb1c107dd0 2024-07-26T13:38:41.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.503+0000 7feb12ffd640 1 -- 172.21.15.100:0/2054628119 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb0c011d40 con 0x7feb1c107dd0 2024-07-26T13:38:41.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.503+0000 7feb12ffd640 1 -- 172.21.15.100:0/2054628119 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c00f340 con 0x7feb1c107dd0 2024-07-26T13:38:41.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.503+0000 7feb23c70640 1 -- 172.21.15.100:0/2054628119 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb1c1a00b0 con 0x7feb1c107dd0 2024-07-26T13:38:41.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.504+0000 7feb23c70640 1 -- 172.21.15.100:0/2054628119 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb1c1a68d0 con 0x7feb1c107dd0 2024-07-26T13:38:41.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.504+0000 7feb23c70640 1 -- 172.21.15.100:0/2054628119 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb1c1028a0 con 0x7feb1c107dd0 2024-07-26T13:38:41.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.506+0000 7feb12ffd640 1 -- 172.21.15.100:0/2054628119 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7feb0c018020 con 0x7feb1c107dd0 2024-07-26T13:38:41.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.507+0000 7feb12ffd640 1 --2- 172.21.15.100:0/2054628119 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7feb080b3e30 0x7feb080b6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:41.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.507+0000 7feb21a0c640 1 --2- 172.21.15.100:0/2054628119 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7feb080b3e30 0x7feb080b6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:41.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.507+0000 7feb12ffd640 1 -- 172.21.15.100:0/2054628119 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7feb0c0e3df0 con 0x7feb1c107dd0 2024-07-26T13:38:41.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.507+0000 7feb21a0c640 1 --2- 172.21.15.100:0/2054628119 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7feb080b3e30 0x7feb080b6340 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7feb18005e10 tx=0x7feb18005da0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:41.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.510+0000 7feb12ffd640 1 -- 172.21.15.100:0/2054628119 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7feb0c01f020 con 0x7feb1c107dd0 2024-07-26T13:38:41.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.686+0000 7feb23c70640 1 -- 172.21.15.100:0/2054628119 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7feb1c1a6bb0 con 0x7feb1c107dd0 2024-07-26T13:38:41.687 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:41 smithi100 ceph-mon[32599]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T13:38:41.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.686+0000 7feb12ffd640 1 -- 172.21.15.100:0/2054628119 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7feb0c00d040 con 0x7feb1c107dd0 2024-07-26T13:38:41.688 INFO:teuthology.orchestra.run.smithi100.stdout:137438953542 2024-07-26T13:38:41.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.689+0000 7feb23c70640 1 -- 172.21.15.100:0/2054628119 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7feb080b3e30 msgr2=0x7feb080b6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:41.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.689+0000 7feb23c70640 1 --2- 172.21.15.100:0/2054628119 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7feb080b3e30 0x7feb080b6340 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7feb18005e10 tx=0x7feb18005da0 comp rx=0 tx=0).stop 2024-07-26T13:38:41.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.689+0000 7feb23c70640 1 -- 172.21.15.100:0/2054628119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb1c107dd0 msgr2=0x7feb1c19a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:41.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.689+0000 7feb23c70640 1 --2- 172.21.15.100:0/2054628119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb1c107dd0 0x7feb1c19a4b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7feb0c00b450 tx=0x7feb0c00b910 comp rx=0 tx=0).stop 2024-07-26T13:38:41.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.689+0000 7feb23c70640 1 -- 172.21.15.100:0/2054628119 shutdown_connections 2024-07-26T13:38:41.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.689+0000 7feb23c70640 1 --2- 172.21.15.100:0/2054628119 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7feb080b3e30 0x7feb080b6340 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:41.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.689+0000 7feb23c70640 1 --2- 172.21.15.100:0/2054628119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb1c108720 0x7feb1c19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:41.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.689+0000 7feb23c70640 1 --2- 172.21.15.100:0/2054628119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb1c107dd0 0x7feb1c19a4b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:41.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.689+0000 7feb23c70640 1 --2- 172.21.15.100:0/2054628119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb1c102490 0x7feb1c199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:41.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.689+0000 7feb23c70640 1 -- 172.21.15.100:0/2054628119 >> 172.21.15.100:0/2054628119 conn(0x7feb1c079160 msgr2=0x7feb1c1005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:41.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.689+0000 7feb23c70640 1 -- 172.21.15.100:0/2054628119 shutdown_connections 2024-07-26T13:38:41.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:41.689+0000 7feb23c70640 1 -- 172.21.15.100:0/2054628119 wait complete. 2024-07-26T13:38:41.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:41 smithi167 ceph-mon[37140]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T13:38:41.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:41 smithi138 ceph-mon[36954]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T13:38:42.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:42 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2054628119' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-07-26T13:38:42.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:42 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/2054628119' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-07-26T13:38:42.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.794+0000 7f4123207640 1 -- 172.21.15.100:0/650296399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f411c107e00 msgr2=0x7f411c108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:42.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.794+0000 7f4123207640 1 --2- 172.21.15.100:0/650296399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f411c107e00 0x7f411c108210 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4108009900 tx=0x7f410802a410 comp rx=0 tx=0).stop 2024-07-26T13:38:42.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.794+0000 7f4123207640 1 -- 172.21.15.100:0/650296399 shutdown_connections 2024-07-26T13:38:42.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.794+0000 7f4123207640 1 --2- 172.21.15.100:0/650296399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f411c108750 0x7f411c10ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:42.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.794+0000 7f4123207640 1 --2- 172.21.15.100:0/650296399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f411c107e00 0x7f411c108210 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:42.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.794+0000 7f4123207640 1 --2- 172.21.15.100:0/650296399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f411c1024c0 0x7f411c1028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:42.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.794+0000 7f4123207640 1 -- 172.21.15.100:0/650296399 >> 172.21.15.100:0/650296399 conn(0x7f411c079180 msgr2=0x7f411c079590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:42.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.794+0000 7f4123207640 1 -- 172.21.15.100:0/650296399 shutdown_connections 2024-07-26T13:38:42.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.794+0000 7f4123207640 1 -- 172.21.15.100:0/650296399 wait complete. 2024-07-26T13:38:42.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.795+0000 7f4123207640 1 Processor -- start 2024-07-26T13:38:42.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.795+0000 7f4123207640 1 -- start start 2024-07-26T13:38:42.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.795+0000 7f4123207640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f411c1024c0 0x7f411c198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:42.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.795+0000 7f4123207640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f411c107e00 0x7f411c199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:42.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.795+0000 7f4123207640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f411c108750 0x7f411c19e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:42.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.795+0000 7f4123207640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f411c112400 con 0x7f411c1024c0 2024-07-26T13:38:42.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.795+0000 7f4123207640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f411c112280 con 0x7f411c108750 2024-07-26T13:38:42.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.795+0000 7f4123207640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f411c112580 con 0x7f411c107e00 2024-07-26T13:38:42.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.795+0000 7f4120fa3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f411c1024c0 0x7f411c198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:42.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.795+0000 7f4120fa3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f411c1024c0 0x7f411c198d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55212/0 (socket says 172.21.15.100:55212) 2024-07-26T13:38:42.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.795+0000 7f4120fa3640 1 -- 172.21.15.100:0/2153503406 learned_addr learned my addr 172.21.15.100:0/2153503406 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:38:42.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.796+0000 7f411bfff640 1 --2- 172.21.15.100:0/2153503406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f411c107e00 0x7f411c199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:42.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.796+0000 7f41217a4640 1 --2- 172.21.15.100:0/2153503406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f411c108750 0x7f411c19e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:42.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.796+0000 7f4120fa3640 1 -- 172.21.15.100:0/2153503406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f411c107e00 msgr2=0x7f411c199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:42.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.796+0000 7f4120fa3640 1 --2- 172.21.15.100:0/2153503406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f411c107e00 0x7f411c199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:42.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.796+0000 7f4120fa3640 1 -- 172.21.15.100:0/2153503406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f411c108750 msgr2=0x7f411c19e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:42.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.796+0000 7f4120fa3640 1 --2- 172.21.15.100:0/2153503406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f411c108750 0x7f411c19e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:42.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.796+0000 7f4120fa3640 1 -- 172.21.15.100:0/2153503406 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f411c19ec40 con 0x7f411c1024c0 2024-07-26T13:38:42.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.796+0000 7f41217a4640 1 --2- 172.21.15.100:0/2153503406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f411c108750 0x7f411c19e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:38:42.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.796+0000 7f4120fa3640 1 --2- 172.21.15.100:0/2153503406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f411c1024c0 0x7f411c198d90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f411000c920 tx=0x7f411000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:42.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.796+0000 7f4119ffb640 1 -- 172.21.15.100:0/2153503406 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4110007c70 con 0x7f411c1024c0 2024-07-26T13:38:42.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.796+0000 7f4119ffb640 1 -- 172.21.15.100:0/2153503406 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4110007e10 con 0x7f411c1024c0 2024-07-26T13:38:42.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.796+0000 7f4119ffb640 1 -- 172.21.15.100:0/2153503406 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4110018820 con 0x7f411c1024c0 2024-07-26T13:38:42.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.796+0000 7f4123207640 1 -- 172.21.15.100:0/2153503406 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f411c19ef30 con 0x7f411c1024c0 2024-07-26T13:38:42.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.796+0000 7f4123207640 1 -- 172.21.15.100:0/2153503406 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f411c1a5750 con 0x7f411c1024c0 2024-07-26T13:38:42.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.797+0000 7f4123207640 1 -- 172.21.15.100:0/2153503406 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f411c103740 con 0x7f411c1024c0 2024-07-26T13:38:42.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.799+0000 7f4119ffb640 1 -- 172.21.15.100:0/2153503406 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f411000f450 con 0x7f411c1024c0 2024-07-26T13:38:42.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.800+0000 7f4119ffb640 1 --2- 172.21.15.100:0/2153503406 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f410c0b3ce0 0x7f410c0b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:42.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.800+0000 7f411bfff640 1 --2- 172.21.15.100:0/2153503406 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f410c0b3ce0 0x7f410c0b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:42.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.800+0000 7f4119ffb640 1 -- 172.21.15.100:0/2153503406 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f41100e4380 con 0x7f411c1024c0 2024-07-26T13:38:42.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.800+0000 7f411bfff640 1 --2- 172.21.15.100:0/2153503406 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f410c0b3ce0 0x7f410c0b61f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4108000c00 tx=0x7f4108031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:42.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.803+0000 7f4119ffb640 1 -- 172.21.15.100:0/2153503406 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f4110014030 con 0x7f411c1024c0 2024-07-26T13:38:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:42 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/2054628119' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-07-26T13:38:42.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.980+0000 7f4123207640 1 -- 172.21.15.100:0/2153503406 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f411c1028d0 con 0x7f411c1024c0 2024-07-26T13:38:42.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.980+0000 7f4119ffb640 1 -- 172.21.15.100:0/2153503406 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f41100ab800 con 0x7f411c1024c0 2024-07-26T13:38:42.981 INFO:teuthology.orchestra.run.smithi100.stdout:38654705752 2024-07-26T13:38:42.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.982+0000 7f4123207640 1 -- 172.21.15.100:0/2153503406 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f410c0b3ce0 msgr2=0x7f410c0b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:42.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.982+0000 7f4123207640 1 --2- 172.21.15.100:0/2153503406 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f410c0b3ce0 0x7f410c0b61f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4108000c00 tx=0x7f4108031400 comp rx=0 tx=0).stop 2024-07-26T13:38:42.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.982+0000 7f4123207640 1 -- 172.21.15.100:0/2153503406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f411c1024c0 msgr2=0x7f411c198d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:42.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.982+0000 7f4123207640 1 --2- 172.21.15.100:0/2153503406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f411c1024c0 0x7f411c198d90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f411000c920 tx=0x7f411000cde0 comp rx=0 tx=0).stop 2024-07-26T13:38:42.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.982+0000 7f4123207640 1 -- 172.21.15.100:0/2153503406 shutdown_connections 2024-07-26T13:38:42.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.982+0000 7f4123207640 1 --2- 172.21.15.100:0/2153503406 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f410c0b3ce0 0x7f410c0b61f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:42.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.982+0000 7f4123207640 1 --2- 172.21.15.100:0/2153503406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f411c108750 0x7f411c19e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:42.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.982+0000 7f4123207640 1 --2- 172.21.15.100:0/2153503406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f411c107e00 0x7f411c199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:42.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.982+0000 7f4123207640 1 --2- 172.21.15.100:0/2153503406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f411c1024c0 0x7f411c198d90 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:42.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.982+0000 7f4123207640 1 -- 172.21.15.100:0/2153503406 >> 172.21.15.100:0/2153503406 conn(0x7f411c079180 msgr2=0x7f411c104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:42.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.982+0000 7f4123207640 1 -- 172.21.15.100:0/2153503406 shutdown_connections 2024-07-26T13:38:42.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:42.982+0000 7f4123207640 1 -- 172.21.15.100:0/2153503406 wait complete. 2024-07-26T13:38:43.245 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874278 got 279172874280 for osd.10 2024-07-26T13:38:43.245 DEBUG:teuthology.parallel:result is None 2024-07-26T13:38:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:43 smithi100 ceph-mon[32599]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:38:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:43 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2153503406' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-07-26T13:38:43.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:43 smithi167 ceph-mon[37140]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:38:43.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:43 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/2153503406' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-07-26T13:38:43.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:43 smithi138 ceph-mon[36954]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:38:43.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:43 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/2153503406' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-07-26T13:38:44.506 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705748 got 38654705752 for osd.0 2024-07-26T13:38:44.507 DEBUG:teuthology.parallel:result is None 2024-07-26T13:38:44.947 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:38:45.495 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953538 got 137438953542 for osd.4 2024-07-26T13:38:45.495 DEBUG:teuthology.parallel:result is None 2024-07-26T13:38:45.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.678+0000 7facfe4a6640 1 -- 172.21.15.100:0/2804554832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facf8103590 msgr2=0x7facf81039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:45.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.678+0000 7facfe4a6640 1 --2- 172.21.15.100:0/2804554832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facf8103590 0x7facf81039a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7face8009900 tx=0x7face802a410 comp rx=0 tx=0).stop 2024-07-26T13:38:45.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.678+0000 7facfe4a6640 1 -- 172.21.15.100:0/2804554832 shutdown_connections 2024-07-26T13:38:45.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.678+0000 7facfe4a6640 1 --2- 172.21.15.100:0/2804554832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7facf8109820 0x7facf8110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:45.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.678+0000 7facfe4a6640 1 --2- 172.21.15.100:0/2804554832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facf8108ed0 0x7facf81092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:45.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.678+0000 7facfe4a6640 1 --2- 172.21.15.100:0/2804554832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facf8103590 0x7facf81039a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:45.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.678+0000 7facfe4a6640 1 -- 172.21.15.100:0/2804554832 >> 172.21.15.100:0/2804554832 conn(0x7facf807b280 msgr2=0x7facf807b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:45.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.678+0000 7facfe4a6640 1 -- 172.21.15.100:0/2804554832 shutdown_connections 2024-07-26T13:38:45.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.678+0000 7facfe4a6640 1 -- 172.21.15.100:0/2804554832 wait complete. 2024-07-26T13:38:45.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.678+0000 7facfe4a6640 1 Processor -- start 2024-07-26T13:38:45.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.679+0000 7facfe4a6640 1 -- start start 2024-07-26T13:38:45.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.679+0000 7facfe4a6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7facf8103590 0x7facf8199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:45.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.679+0000 7facfe4a6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facf8108ed0 0x7facf819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:45.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.679+0000 7facfe4a6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facf8109820 0x7facf819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:45.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.679+0000 7facfe4a6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7facf8113540 con 0x7facf8109820 2024-07-26T13:38:45.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.679+0000 7facfe4a6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7facf81133c0 con 0x7facf8108ed0 2024-07-26T13:38:45.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.679+0000 7facfe4a6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7facf81136c0 con 0x7facf8103590 2024-07-26T13:38:45.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.679+0000 7facfca43640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facf8109820 0x7facf819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:45.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.679+0000 7facfca43640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facf8109820 0x7facf819f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37412/0 (socket says 172.21.15.100:37412) 2024-07-26T13:38:45.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.679+0000 7facfca43640 1 -- 172.21.15.100:0/2009586512 learned_addr learned my addr 172.21.15.100:0/2009586512 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:38:45.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.679+0000 7facf77fe640 1 --2- 172.21.15.100:0/2009586512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facf8108ed0 0x7facf819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:45.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.679+0000 7facf7fff640 1 --2- 172.21.15.100:0/2009586512 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7facf8103590 0x7facf8199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:45.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.680+0000 7facfca43640 1 -- 172.21.15.100:0/2009586512 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7facf8103590 msgr2=0x7facf8199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:45.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.680+0000 7facfca43640 1 --2- 172.21.15.100:0/2009586512 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7facf8103590 0x7facf8199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:45.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.680+0000 7facfca43640 1 -- 172.21.15.100:0/2009586512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facf8108ed0 msgr2=0x7facf819a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:45.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.680+0000 7facfca43640 1 --2- 172.21.15.100:0/2009586512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facf8108ed0 0x7facf819a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:45.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.680+0000 7facfca43640 1 -- 172.21.15.100:0/2009586512 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf819fe80 con 0x7facf8109820 2024-07-26T13:38:45.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.680+0000 7facfca43640 1 --2- 172.21.15.100:0/2009586512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facf8109820 0x7facf819f750 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7facf0012eb0 tx=0x7facf000c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:45.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.680+0000 7facf57fa640 1 -- 172.21.15.100:0/2009586512 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf0004b20 con 0x7facf8109820 2024-07-26T13:38:45.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.680+0000 7facf57fa640 1 -- 172.21.15.100:0/2009586512 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facf0004cc0 con 0x7facf8109820 2024-07-26T13:38:45.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.680+0000 7facf57fa640 1 -- 172.21.15.100:0/2009586512 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf001a790 con 0x7facf8109820 2024-07-26T13:38:45.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.680+0000 7facfe4a6640 1 -- 172.21.15.100:0/2009586512 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf81a0170 con 0x7facf8109820 2024-07-26T13:38:45.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.680+0000 7facfe4a6640 1 -- 172.21.15.100:0/2009586512 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf8075020 con 0x7facf8109820 2024-07-26T13:38:45.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.682+0000 7facfe4a6640 1 -- 172.21.15.100:0/2009586512 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facf8104810 con 0x7facf8109820 2024-07-26T13:38:45.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.684+0000 7facf57fa640 1 -- 172.21.15.100:0/2009586512 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7facf0005020 con 0x7facf8109820 2024-07-26T13:38:45.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.685+0000 7facf57fa640 1 --2- 172.21.15.100:0/2009586512 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7face00b3cb0 0x7face00b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:45.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.685+0000 7facf7fff640 1 --2- 172.21.15.100:0/2009586512 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7face00b3cb0 0x7face00b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:45.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.685+0000 7facf57fa640 1 -- 172.21.15.100:0/2009586512 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7facf00e7af0 con 0x7facf8109820 2024-07-26T13:38:45.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.685+0000 7facf7fff640 1 --2- 172.21.15.100:0/2009586512 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7face00b3cb0 0x7face00b61c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7face8031440 tx=0x7face8009560 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:45.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.691+0000 7facf57fa640 1 -- 172.21.15.100:0/2009586512 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7facf00aeee0 con 0x7facf8109820 2024-07-26T13:38:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:45 smithi100 ceph-mon[32599]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:38:45.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:45 smithi167 ceph-mon[37140]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:38:45.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.878+0000 7facfe4a6640 1 -- 172.21.15.100:0/2009586512 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7facf810b5a0 con 0x7facf8109820 2024-07-26T13:38:45.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.878+0000 7facf57fa640 1 -- 172.21.15.100:0/2009586512 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7facf00ae7e0 con 0x7facf8109820 2024-07-26T13:38:45.879 INFO:teuthology.orchestra.run.smithi100.stdout:184683593789 2024-07-26T13:38:45.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.880+0000 7facfe4a6640 1 -- 172.21.15.100:0/2009586512 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7face00b3cb0 msgr2=0x7face00b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:45.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.880+0000 7facfe4a6640 1 --2- 172.21.15.100:0/2009586512 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7face00b3cb0 0x7face00b61c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7face8031440 tx=0x7face8009560 comp rx=0 tx=0).stop 2024-07-26T13:38:45.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.880+0000 7facfe4a6640 1 -- 172.21.15.100:0/2009586512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facf8109820 msgr2=0x7facf819f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:45.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.880+0000 7facfe4a6640 1 --2- 172.21.15.100:0/2009586512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facf8109820 0x7facf819f750 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7facf0012eb0 tx=0x7facf000c640 comp rx=0 tx=0).stop 2024-07-26T13:38:45.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.880+0000 7facfe4a6640 1 -- 172.21.15.100:0/2009586512 shutdown_connections 2024-07-26T13:38:45.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.880+0000 7facfe4a6640 1 --2- 172.21.15.100:0/2009586512 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7face00b3cb0 0x7face00b61c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:45.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.880+0000 7facfe4a6640 1 --2- 172.21.15.100:0/2009586512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facf8109820 0x7facf819f750 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:45.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.880+0000 7facfe4a6640 1 --2- 172.21.15.100:0/2009586512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facf8108ed0 0x7facf819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:45.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.880+0000 7facfe4a6640 1 --2- 172.21.15.100:0/2009586512 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7facf8103590 0x7facf8199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:45.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.880+0000 7facfe4a6640 1 -- 172.21.15.100:0/2009586512 >> 172.21.15.100:0/2009586512 conn(0x7facf807b280 msgr2=0x7facf8106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:45.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.880+0000 7facfe4a6640 1 -- 172.21.15.100:0/2009586512 shutdown_connections 2024-07-26T13:38:45.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:45.880+0000 7facfe4a6640 1 -- 172.21.15.100:0/2009586512 wait complete. 2024-07-26T13:38:45.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:45 smithi138 ceph-mon[36954]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:38:46.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.423+0000 7fde8785c640 1 -- 172.21.15.100:0/2040880036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde80109820 msgr2=0x7fde80110050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:46.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.423+0000 7fde8785c640 1 --2- 172.21.15.100:0/2040880036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde80109820 0x7fde80110050 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fde78009900 tx=0x7fde7802a410 comp rx=0 tx=0).stop 2024-07-26T13:38:46.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.423+0000 7fde8785c640 1 -- 172.21.15.100:0/2040880036 shutdown_connections 2024-07-26T13:38:46.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.423+0000 7fde8785c640 1 --2- 172.21.15.100:0/2040880036 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde80109820 0x7fde80110050 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:46.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.423+0000 7fde8785c640 1 --2- 172.21.15.100:0/2040880036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde80108ed0 0x7fde801092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:46.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.423+0000 7fde8785c640 1 --2- 172.21.15.100:0/2040880036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde80103590 0x7fde801039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:46.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.423+0000 7fde8785c640 1 -- 172.21.15.100:0/2040880036 >> 172.21.15.100:0/2040880036 conn(0x7fde8007b280 msgr2=0x7fde8007b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:46.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.423+0000 7fde8785c640 1 -- 172.21.15.100:0/2040880036 shutdown_connections 2024-07-26T13:38:46.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.423+0000 7fde8785c640 1 -- 172.21.15.100:0/2040880036 wait complete. 2024-07-26T13:38:46.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.424+0000 7fde8785c640 1 Processor -- start 2024-07-26T13:38:46.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.424+0000 7fde8785c640 1 -- start start 2024-07-26T13:38:46.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.424+0000 7fde8785c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde80103590 0x7fde80075140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:46.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.424+0000 7fde8785c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde80108ed0 0x7fde80075680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:46.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.424+0000 7fde8785c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde80109820 0x7fde80075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:46.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.424+0000 7fde8785c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fde801134f0 con 0x7fde80103590 2024-07-26T13:38:46.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.424+0000 7fde8785c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fde80113370 con 0x7fde80109820 2024-07-26T13:38:46.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.424+0000 7fde8785c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fde80113670 con 0x7fde80108ed0 2024-07-26T13:38:46.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.424+0000 7fde855f8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde80103590 0x7fde80075140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:46.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.424+0000 7fde855f8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde80103590 0x7fde80075140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37424/0 (socket says 172.21.15.100:37424) 2024-07-26T13:38:46.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.424+0000 7fde855f8640 1 -- 172.21.15.100:0/1828652056 learned_addr learned my addr 172.21.15.100:0/1828652056 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:38:46.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.424+0000 7fde84df7640 1 --2- 172.21.15.100:0/1828652056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde80108ed0 0x7fde80075680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:46.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.424+0000 7fde85df9640 1 --2- 172.21.15.100:0/1828652056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde80109820 0x7fde80075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:46.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.424+0000 7fde855f8640 1 -- 172.21.15.100:0/1828652056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde80108ed0 msgr2=0x7fde80075680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:46.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.424+0000 7fde855f8640 1 --2- 172.21.15.100:0/1828652056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde80108ed0 0x7fde80075680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:46.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.424+0000 7fde855f8640 1 -- 172.21.15.100:0/1828652056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde80109820 msgr2=0x7fde80075bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:46.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.424+0000 7fde855f8640 1 --2- 172.21.15.100:0/1828652056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde80109820 0x7fde80075bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:46.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.424+0000 7fde855f8640 1 -- 172.21.15.100:0/1828652056 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde801a8160 con 0x7fde80103590 2024-07-26T13:38:46.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.425+0000 7fde855f8640 1 --2- 172.21.15.100:0/1828652056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde80103590 0x7fde80075140 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fde7c00eb90 tx=0x7fde7c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:46.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.425+0000 7fde767fc640 1 -- 172.21.15.100:0/1828652056 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7c004d20 con 0x7fde80103590 2024-07-26T13:38:46.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.425+0000 7fde767fc640 1 -- 172.21.15.100:0/1828652056 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde7c00ee90 con 0x7fde80103590 2024-07-26T13:38:46.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.425+0000 7fde767fc640 1 -- 172.21.15.100:0/1828652056 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7c010670 con 0x7fde80103590 2024-07-26T13:38:46.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.425+0000 7fde8785c640 1 -- 172.21.15.100:0/1828652056 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde801a8450 con 0x7fde80103590 2024-07-26T13:38:46.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.425+0000 7fde8785c640 1 -- 172.21.15.100:0/1828652056 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde801a8930 con 0x7fde80103590 2024-07-26T13:38:46.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.426+0000 7fde8785c640 1 -- 172.21.15.100:0/1828652056 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde801039a0 con 0x7fde80103590 2024-07-26T13:38:46.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.428+0000 7fde767fc640 1 -- 172.21.15.100:0/1828652056 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7fde7c010870 con 0x7fde80103590 2024-07-26T13:38:46.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.428+0000 7fde767fc640 1 --2- 172.21.15.100:0/1828652056 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fde6c0b3d30 0x7fde6c0b6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:46.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.429+0000 7fde84df7640 1 --2- 172.21.15.100:0/1828652056 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fde6c0b3d30 0x7fde6c0b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:46.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.429+0000 7fde767fc640 1 -- 172.21.15.100:0/1828652056 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fde7c0e46e0 con 0x7fde80103590 2024-07-26T13:38:46.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.429+0000 7fde84df7640 1 --2- 172.21.15.100:0/1828652056 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fde6c0b3d30 0x7fde6c0b6240 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fde801069f0 tx=0x7fde7000b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:46.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.431+0000 7fde767fc640 1 -- 172.21.15.100:0/1828652056 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fde7c055600 con 0x7fde80103590 2024-07-26T13:38:46.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.511+0000 7fde767fc640 1 -- 172.21.15.100:0/1828652056 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7fde7c0a7660 con 0x7fde80103590 2024-07-26T13:38:46.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.620+0000 7fde8785c640 1 -- 172.21.15.100:0/1828652056 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fde8002e0c0 con 0x7fde80103590 2024-07-26T13:38:46.621 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:46 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2009586512' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-07-26T13:38:46.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.620+0000 7fde767fc640 1 -- 172.21.15.100:0/1828652056 <== mon.0 v2:172.21.15.100:3300/0 8 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fde7c021440 con 0x7fde80103590 2024-07-26T13:38:46.622 INFO:teuthology.orchestra.run.smithi100.stdout:81604378704 2024-07-26T13:38:46.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.623+0000 7fde8785c640 1 -- 172.21.15.100:0/1828652056 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fde6c0b3d30 msgr2=0x7fde6c0b6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:46.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.623+0000 7fde8785c640 1 --2- 172.21.15.100:0/1828652056 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fde6c0b3d30 0x7fde6c0b6240 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fde801069f0 tx=0x7fde7000b390 comp rx=0 tx=0).stop 2024-07-26T13:38:46.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.623+0000 7fde8785c640 1 -- 172.21.15.100:0/1828652056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde80103590 msgr2=0x7fde80075140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:46.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.623+0000 7fde8785c640 1 --2- 172.21.15.100:0/1828652056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde80103590 0x7fde80075140 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fde7c00eb90 tx=0x7fde7c00c6a0 comp rx=0 tx=0).stop 2024-07-26T13:38:46.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.623+0000 7fde8785c640 1 -- 172.21.15.100:0/1828652056 shutdown_connections 2024-07-26T13:38:46.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.623+0000 7fde8785c640 1 --2- 172.21.15.100:0/1828652056 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fde6c0b3d30 0x7fde6c0b6240 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:46.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.623+0000 7fde8785c640 1 --2- 172.21.15.100:0/1828652056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde80109820 0x7fde80075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:46.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.623+0000 7fde8785c640 1 --2- 172.21.15.100:0/1828652056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde80108ed0 0x7fde80075680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:46.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.623+0000 7fde8785c640 1 --2- 172.21.15.100:0/1828652056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fde80103590 0x7fde80075140 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:46.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.623+0000 7fde8785c640 1 -- 172.21.15.100:0/1828652056 >> 172.21.15.100:0/1828652056 conn(0x7fde8007b280 msgr2=0x7fde80104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:46.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.623+0000 7fde8785c640 1 -- 172.21.15.100:0/1828652056 shutdown_connections 2024-07-26T13:38:46.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:46.623+0000 7fde8785c640 1 -- 172.21.15.100:0/1828652056 wait complete. 2024-07-26T13:38:46.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:46 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/2009586512' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-07-26T13:38:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:46 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/2009586512' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-07-26T13:38:47.596 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:38:47.596 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:38:47.626 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593784 got 184683593789 for osd.6 2024-07-26T13:38:47.627 DEBUG:teuthology.parallel:result is None 2024-07-26T13:38:47.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:47 smithi167 ceph-mon[37140]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T13:38:47.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:47 smithi167 ceph-mon[37140]: mgrmap e22: x(active, since 8m), standbys: y, z 2024-07-26T13:38:47.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:47 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/1828652056' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-07-26T13:38:47.790 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:47 smithi100 ceph-mon[32599]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T13:38:47.790 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:47 smithi100 ceph-mon[32599]: mgrmap e22: x(active, since 8m), standbys: y, z 2024-07-26T13:38:47.790 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:47 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/1828652056' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-07-26T13:38:47.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:47 smithi138 ceph-mon[36954]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T13:38:47.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:47 smithi138 ceph-mon[36954]: mgrmap e22: x(active, since 8m), standbys: y, z 2024-07-26T13:38:47.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:47 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/1828652056' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-07-26T13:38:48.443 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378696 got 81604378704 for osd.2 2024-07-26T13:38:48.443 DEBUG:teuthology.parallel:result is None 2024-07-26T13:38:49.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.116+0000 7f76a1da8640 1 -- 172.21.15.100:0/1289751797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f769c108ed0 msgr2=0x7f769c1092e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:49.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.116+0000 7f76a1da8640 1 --2- 172.21.15.100:0/1289751797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f769c108ed0 0x7f769c1092e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7690009900 tx=0x7f769002a410 comp rx=0 tx=0).stop 2024-07-26T13:38:49.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.116+0000 7f76a1da8640 1 -- 172.21.15.100:0/1289751797 shutdown_connections 2024-07-26T13:38:49.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.116+0000 7f76a1da8640 1 --2- 172.21.15.100:0/1289751797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f769c109820 0x7f769c110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:49.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.116+0000 7f76a1da8640 1 --2- 172.21.15.100:0/1289751797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f769c108ed0 0x7f769c1092e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:49.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.116+0000 7f76a1da8640 1 --2- 172.21.15.100:0/1289751797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f769c103590 0x7f769c1039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:49.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.116+0000 7f76a1da8640 1 -- 172.21.15.100:0/1289751797 >> 172.21.15.100:0/1289751797 conn(0x7f769c07b280 msgr2=0x7f769c07b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:49.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.116+0000 7f76a1da8640 1 -- 172.21.15.100:0/1289751797 shutdown_connections 2024-07-26T13:38:49.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.117+0000 7f76a1da8640 1 -- 172.21.15.100:0/1289751797 wait complete. 2024-07-26T13:38:49.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.117+0000 7f76a1da8640 1 Processor -- start 2024-07-26T13:38:49.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.117+0000 7f76a1da8640 1 -- start start 2024-07-26T13:38:49.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.118+0000 7f76a1da8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f769c103590 0x7f769c199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:49.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.118+0000 7f76a1da8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f769c108ed0 0x7f769c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:49.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.118+0000 7f76a1da8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f769c109820 0x7f769c19f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:49.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.118+0000 7f76a1da8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f769c113500 con 0x7f769c103590 2024-07-26T13:38:49.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.118+0000 7f76a1da8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f769c113380 con 0x7f769c109820 2024-07-26T13:38:49.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.118+0000 7f76a1da8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f769c113680 con 0x7f769c108ed0 2024-07-26T13:38:49.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.118+0000 7f769b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f769c103590 0x7f769c199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:49.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.118+0000 7f769b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f769c103590 0x7f769c199ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37438/0 (socket says 172.21.15.100:37438) 2024-07-26T13:38:49.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.118+0000 7f769b7fe640 1 -- 172.21.15.100:0/1652566864 learned_addr learned my addr 172.21.15.100:0/1652566864 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:38:49.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.118+0000 7f769affd640 1 --2- 172.21.15.100:0/1652566864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f769c108ed0 0x7f769c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:49.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.118+0000 7f769bfff640 1 --2- 172.21.15.100:0/1652566864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f769c109820 0x7f769c19f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:49.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.118+0000 7f769b7fe640 1 -- 172.21.15.100:0/1652566864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f769c108ed0 msgr2=0x7f769c19a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:49.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.118+0000 7f769b7fe640 1 --2- 172.21.15.100:0/1652566864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f769c108ed0 0x7f769c19a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:49.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.118+0000 7f769b7fe640 1 -- 172.21.15.100:0/1652566864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f769c109820 msgr2=0x7f769c19f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:49.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.118+0000 7f769b7fe640 1 --2- 172.21.15.100:0/1652566864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f769c109820 0x7f769c19f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:49.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.118+0000 7f769b7fe640 1 -- 172.21.15.100:0/1652566864 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f769c19fe60 con 0x7f769c103590 2024-07-26T13:38:49.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.118+0000 7f769b7fe640 1 --2- 172.21.15.100:0/1652566864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f769c103590 0x7f769c199ef0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f768c00c890 tx=0x7f768c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:49.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.118+0000 7f7698ff9640 1 -- 172.21.15.100:0/1652566864 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768c004220 con 0x7f769c103590 2024-07-26T13:38:49.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.118+0000 7f7698ff9640 1 -- 172.21.15.100:0/1652566864 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f768c0043c0 con 0x7f769c103590 2024-07-26T13:38:49.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.119+0000 7f76a1da8640 1 -- 172.21.15.100:0/1652566864 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f769c1a0150 con 0x7f769c103590 2024-07-26T13:38:49.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.119+0000 7f7698ff9640 1 -- 172.21.15.100:0/1652566864 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768c004a00 con 0x7f769c103590 2024-07-26T13:38:49.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.119+0000 7f76a1da8640 1 -- 172.21.15.100:0/1652566864 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f769c1a68b0 con 0x7f769c103590 2024-07-26T13:38:49.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.120+0000 7f76a1da8640 1 -- 172.21.15.100:0/1652566864 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f769c1039a0 con 0x7f769c103590 2024-07-26T13:38:49.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.121+0000 7f7698ff9640 1 -- 172.21.15.100:0/1652566864 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f768c004ba0 con 0x7f769c103590 2024-07-26T13:38:49.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.122+0000 7f7698ff9640 1 --2- 172.21.15.100:0/1652566864 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f76840b3c20 0x7f76840b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:49.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.122+0000 7f769affd640 1 --2- 172.21.15.100:0/1652566864 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f76840b3c20 0x7f76840b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:49.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.122+0000 7f7698ff9640 1 -- 172.21.15.100:0/1652566864 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f768c0e3aa0 con 0x7f769c103590 2024-07-26T13:38:49.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.122+0000 7f769affd640 1 --2- 172.21.15.100:0/1652566864 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f76840b3c20 0x7f76840b6130 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7690009230 tx=0x7f7690031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:49.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.126+0000 7f7698ff9640 1 -- 172.21.15.100:0/1652566864 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f768c014030 con 0x7f769c103590 2024-07-26T13:38:49.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.302+0000 7f76a1da8640 1 -- 172.21.15.100:0/1652566864 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f769c10b080 con 0x7f769c103590 2024-07-26T13:38:49.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.303+0000 7f7698ff9640 1 -- 172.21.15.100:0/1652566864 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f768c0aaff0 con 0x7f769c103590 2024-07-26T13:38:49.304 INFO:teuthology.orchestra.run.smithi100.stdout:111669149771 2024-07-26T13:38:49.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.305+0000 7f76a1da8640 1 -- 172.21.15.100:0/1652566864 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f76840b3c20 msgr2=0x7f76840b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:49.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.305+0000 7f76a1da8640 1 --2- 172.21.15.100:0/1652566864 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f76840b3c20 0x7f76840b6130 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7690009230 tx=0x7f7690031400 comp rx=0 tx=0).stop 2024-07-26T13:38:49.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.305+0000 7f76a1da8640 1 -- 172.21.15.100:0/1652566864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f769c103590 msgr2=0x7f769c199ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:49.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.305+0000 7f76a1da8640 1 --2- 172.21.15.100:0/1652566864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f769c103590 0x7f769c199ef0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f768c00c890 tx=0x7f768c00cd50 comp rx=0 tx=0).stop 2024-07-26T13:38:49.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.305+0000 7f76a1da8640 1 -- 172.21.15.100:0/1652566864 shutdown_connections 2024-07-26T13:38:49.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.305+0000 7f76a1da8640 1 --2- 172.21.15.100:0/1652566864 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f76840b3c20 0x7f76840b6130 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:49.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.305+0000 7f76a1da8640 1 --2- 172.21.15.100:0/1652566864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f769c109820 0x7f769c19f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:49.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.305+0000 7f76a1da8640 1 --2- 172.21.15.100:0/1652566864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f769c108ed0 0x7f769c19a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:49.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.305+0000 7f76a1da8640 1 --2- 172.21.15.100:0/1652566864 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f769c103590 0x7f769c199ef0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:49.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.305+0000 7f76a1da8640 1 -- 172.21.15.100:0/1652566864 >> 172.21.15.100:0/1652566864 conn(0x7f769c07b280 msgr2=0x7f769c106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:49.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.305+0000 7f76a1da8640 1 -- 172.21.15.100:0/1652566864 shutdown_connections 2024-07-26T13:38:49.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.305+0000 7f76a1da8640 1 -- 172.21.15.100:0/1652566864 wait complete. 2024-07-26T13:38:49.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.737+0000 7f394d510640 1 -- 172.21.15.100:0/3791793765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3948108750 msgr2=0x7f394810ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:49.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.737+0000 7f394d510640 1 --2- 172.21.15.100:0/3791793765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3948108750 0x7f394810ef80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3940009ca0 tx=0x7f394002a8c0 comp rx=0 tx=0).stop 2024-07-26T13:38:49.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.737+0000 7f394d510640 1 -- 172.21.15.100:0/3791793765 shutdown_connections 2024-07-26T13:38:49.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.737+0000 7f394d510640 1 --2- 172.21.15.100:0/3791793765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3948108750 0x7f394810ef80 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:49.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.737+0000 7f394d510640 1 --2- 172.21.15.100:0/3791793765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3948107e00 0x7f3948108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:49.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.737+0000 7f394d510640 1 --2- 172.21.15.100:0/3791793765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3948102510 0x7f3948102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:49.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.737+0000 7f394d510640 1 -- 172.21.15.100:0/3791793765 >> 172.21.15.100:0/3791793765 conn(0x7f3948079190 msgr2=0x7f39480795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:49.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.737+0000 7f394d510640 1 -- 172.21.15.100:0/3791793765 shutdown_connections 2024-07-26T13:38:49.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.737+0000 7f394d510640 1 -- 172.21.15.100:0/3791793765 wait complete. 2024-07-26T13:38:49.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.738+0000 7f394d510640 1 Processor -- start 2024-07-26T13:38:49.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.738+0000 7f394d510640 1 -- start start 2024-07-26T13:38:49.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.738+0000 7f394d510640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3948102510 0x7f3948199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:49.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.738+0000 7f394d510640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3948107e00 0x7f394819a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:49.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.738+0000 7f394d510640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3948108750 0x7f394819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:49.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.738+0000 7f394d510640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f39481123b0 con 0x7f3948102510 2024-07-26T13:38:49.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.738+0000 7f394d510640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3948112230 con 0x7f3948108750 2024-07-26T13:38:49.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.738+0000 7f394d510640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3948112530 con 0x7f3948107e00 2024-07-26T13:38:49.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.739+0000 7f3946ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3948102510 0x7f3948199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:49.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.739+0000 7f3946ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3948102510 0x7f3948199e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37456/0 (socket says 172.21.15.100:37456) 2024-07-26T13:38:49.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.739+0000 7f3946ffd640 1 -- 172.21.15.100:0/1899812302 learned_addr learned my addr 172.21.15.100:0/1899812302 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:38:49.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.739+0000 7f393e7fc640 1 --2- 172.21.15.100:0/1899812302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3948107e00 0x7f394819a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:49.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.739+0000 7f3946ffd640 1 -- 172.21.15.100:0/1899812302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3948107e00 msgr2=0x7f394819a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:49.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.739+0000 7f3946ffd640 1 --2- 172.21.15.100:0/1899812302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3948107e00 0x7f394819a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:49.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.739+0000 7f3946ffd640 1 -- 172.21.15.100:0/1899812302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3948108750 msgr2=0x7f394819f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:49.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.739+0000 7f3946ffd640 1 --2- 172.21.15.100:0/1899812302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3948108750 0x7f394819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:49.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.739+0000 7f3946ffd640 1 -- 172.21.15.100:0/1899812302 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f394819fde0 con 0x7f3948102510 2024-07-26T13:38:49.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.739+0000 7f393e7fc640 1 --2- 172.21.15.100:0/1899812302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3948107e00 0x7f394819a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:38:49.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.739+0000 7f3946ffd640 1 --2- 172.21.15.100:0/1899812302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3948102510 0x7f3948199e70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f393800c890 tx=0x7f393800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:49.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.739+0000 7f3944ff9640 1 -- 172.21.15.100:0/1899812302 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3938004220 con 0x7f3948102510 2024-07-26T13:38:49.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.739+0000 7f394d510640 1 -- 172.21.15.100:0/1899812302 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39481a00d0 con 0x7f3948102510 2024-07-26T13:38:49.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.739+0000 7f3944ff9640 1 -- 172.21.15.100:0/1899812302 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39380043c0 con 0x7f3948102510 2024-07-26T13:38:49.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.739+0000 7f3944ff9640 1 -- 172.21.15.100:0/1899812302 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3938004a00 con 0x7f3948102510 2024-07-26T13:38:49.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.739+0000 7f394d510640 1 -- 172.21.15.100:0/1899812302 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39481a6830 con 0x7f3948102510 2024-07-26T13:38:49.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.740+0000 7f394d510640 1 -- 172.21.15.100:0/1899812302 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3948102920 con 0x7f3948102510 2024-07-26T13:38:49.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.742+0000 7f3944ff9640 1 -- 172.21.15.100:0/1899812302 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f39380079b0 con 0x7f3948102510 2024-07-26T13:38:49.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.742+0000 7f3944ff9640 1 --2- 172.21.15.100:0/1899812302 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f39340b3c20 0x7f39340b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:49.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.742+0000 7f393e7fc640 1 --2- 172.21.15.100:0/1899812302 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f39340b3c20 0x7f39340b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:49.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.742+0000 7f3944ff9640 1 -- 172.21.15.100:0/1899812302 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f39380e3f80 con 0x7f3948102510 2024-07-26T13:38:49.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.743+0000 7f393e7fc640 1 --2- 172.21.15.100:0/1899812302 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f39340b3c20 0x7f39340b6130 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3948079800 tx=0x7f3930005d30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:49.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.746+0000 7f3944ff9640 1 -- 172.21.15.100:0/1899812302 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f3938014030 con 0x7f3948102510 2024-07-26T13:38:49.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:49 smithi167 ceph-mon[37140]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-07-26T13:38:49.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:49 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/1652566864' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-07-26T13:38:49.884 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:49 smithi100 ceph-mon[32599]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-07-26T13:38:49.884 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:49 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/1652566864' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-07-26T13:38:49.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.928+0000 7f394d510640 1 -- 172.21.15.100:0/1899812302 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f394819afa0 con 0x7f3948102510 2024-07-26T13:38:49.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.928+0000 7f3944ff9640 1 -- 172.21.15.100:0/1899812302 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f39380ab4d0 con 0x7f3948102510 2024-07-26T13:38:49.929 INFO:teuthology.orchestra.run.smithi100.stdout:158913790019 2024-07-26T13:38:49.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.930+0000 7f394d510640 1 -- 172.21.15.100:0/1899812302 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f39340b3c20 msgr2=0x7f39340b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:49.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.930+0000 7f394d510640 1 --2- 172.21.15.100:0/1899812302 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f39340b3c20 0x7f39340b6130 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3948079800 tx=0x7f3930005d30 comp rx=0 tx=0).stop 2024-07-26T13:38:49.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.930+0000 7f394d510640 1 -- 172.21.15.100:0/1899812302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3948102510 msgr2=0x7f3948199e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:49.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.930+0000 7f394d510640 1 --2- 172.21.15.100:0/1899812302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3948102510 0x7f3948199e70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f393800c890 tx=0x7f393800cd50 comp rx=0 tx=0).stop 2024-07-26T13:38:49.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.930+0000 7f394d510640 1 -- 172.21.15.100:0/1899812302 shutdown_connections 2024-07-26T13:38:49.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.930+0000 7f394d510640 1 --2- 172.21.15.100:0/1899812302 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f39340b3c20 0x7f39340b6130 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:49.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.930+0000 7f394d510640 1 --2- 172.21.15.100:0/1899812302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3948108750 0x7f394819f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:49.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.930+0000 7f394d510640 1 --2- 172.21.15.100:0/1899812302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3948107e00 0x7f394819a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:49.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.930+0000 7f394d510640 1 --2- 172.21.15.100:0/1899812302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3948102510 0x7f3948199e70 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:49.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.930+0000 7f394d510640 1 -- 172.21.15.100:0/1899812302 >> 172.21.15.100:0/1899812302 conn(0x7f3948079190 msgr2=0x7f3948100580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:49.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.930+0000 7f394d510640 1 -- 172.21.15.100:0/1899812302 shutdown_connections 2024-07-26T13:38:49.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:49.930+0000 7f394d510640 1 -- 172.21.15.100:0/1899812302 wait complete. 2024-07-26T13:38:49.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:49 smithi138 ceph-mon[36954]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-07-26T13:38:49.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:49 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/1652566864' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-07-26T13:38:50.358 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149764 got 111669149771 for osd.3 2024-07-26T13:38:50.358 DEBUG:teuthology.parallel:result is None 2024-07-26T13:38:50.715 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:50 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/1899812302' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-07-26T13:38:50.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:50 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/1899812302' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-07-26T13:38:50.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:50 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/1899812302' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-07-26T13:38:51.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:51 smithi167 ceph-mon[37140]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-07-26T13:38:51.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:51 smithi138 ceph-mon[36954]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-07-26T13:38:51.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:51 smithi100 ceph-mon[32599]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-07-26T13:38:52.445 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790014 got 158913790019 for osd.5 2024-07-26T13:38:52.445 DEBUG:teuthology.parallel:result is None 2024-07-26T13:38:53.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:53 smithi167 ceph-mon[37140]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-07-26T13:38:53.924 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:53 smithi100 ceph-mon[32599]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-07-26T13:38:53.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:53 smithi138 ceph-mon[36954]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-07-26T13:38:54.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.019+0000 7f2ae0e26640 1 -- 172.21.15.100:0/3940064924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2adc109820 msgr2=0x7f2adc110050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:54.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.019+0000 7f2ae0e26640 1 --2- 172.21.15.100:0/3940064924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2adc109820 0x7f2adc110050 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4009ca0 tx=0x7f2ad402a8c0 comp rx=0 tx=0).stop 2024-07-26T13:38:54.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.019+0000 7f2ae0e26640 1 -- 172.21.15.100:0/3940064924 shutdown_connections 2024-07-26T13:38:54.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.019+0000 7f2ae0e26640 1 --2- 172.21.15.100:0/3940064924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2adc109820 0x7f2adc110050 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:54.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.019+0000 7f2ae0e26640 1 --2- 172.21.15.100:0/3940064924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2adc108ed0 0x7f2adc1092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:54.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.019+0000 7f2ae0e26640 1 --2- 172.21.15.100:0/3940064924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2adc103590 0x7f2adc1039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:54.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.019+0000 7f2ae0e26640 1 -- 172.21.15.100:0/3940064924 >> 172.21.15.100:0/3940064924 conn(0x7f2adc07b280 msgr2=0x7f2adc07b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:54.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.019+0000 7f2ae0e26640 1 -- 172.21.15.100:0/3940064924 shutdown_connections 2024-07-26T13:38:54.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.019+0000 7f2ae0e26640 1 -- 172.21.15.100:0/3940064924 wait complete. 2024-07-26T13:38:54.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.020+0000 7f2ae0e26640 1 Processor -- start 2024-07-26T13:38:54.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.020+0000 7f2ae0e26640 1 -- start start 2024-07-26T13:38:54.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.020+0000 7f2ae0e26640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2adc103590 0x7f2adc19c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:54.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.020+0000 7f2ae0e26640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2adc108ed0 0x7f2adc19c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:54.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.020+0000 7f2ae0e26640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2adc109820 0x7f2adc1a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:54.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.020+0000 7f2ae0e26640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc1134f0 con 0x7f2adc108ed0 2024-07-26T13:38:54.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.020+0000 7f2ae0e26640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc113370 con 0x7f2adc109820 2024-07-26T13:38:54.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.020+0000 7f2ae0e26640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc113670 con 0x7f2adc103590 2024-07-26T13:38:54.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.020+0000 7f2ad9d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2adc108ed0 0x7f2adc19c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:54.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.020+0000 7f2ad9d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2adc108ed0 0x7f2adc19c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37474/0 (socket says 172.21.15.100:37474) 2024-07-26T13:38:54.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.020+0000 7f2ad9d9b640 1 -- 172.21.15.100:0/694767937 learned_addr learned my addr 172.21.15.100:0/694767937 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:38:54.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.020+0000 7f2ada59c640 1 --2- 172.21.15.100:0/694767937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2adc103590 0x7f2adc19c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:54.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.020+0000 7f2adad9d640 1 --2- 172.21.15.100:0/694767937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2adc109820 0x7f2adc1a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:54.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.021+0000 7f2ad9d9b640 1 -- 172.21.15.100:0/694767937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2adc103590 msgr2=0x7f2adc19c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:54.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.021+0000 7f2ad9d9b640 1 --2- 172.21.15.100:0/694767937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2adc103590 0x7f2adc19c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:54.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.021+0000 7f2ad9d9b640 1 -- 172.21.15.100:0/694767937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2adc109820 msgr2=0x7f2adc1a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:54.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.021+0000 7f2ad9d9b640 1 --2- 172.21.15.100:0/694767937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2adc109820 0x7f2adc1a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:54.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.021+0000 7f2ad9d9b640 1 -- 172.21.15.100:0/694767937 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2adc1a1f70 con 0x7f2adc108ed0 2024-07-26T13:38:54.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.021+0000 7f2ad9d9b640 1 --2- 172.21.15.100:0/694767937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2adc108ed0 0x7f2adc19c660 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2ad000e9c0 tx=0x7f2ad000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:54.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.021+0000 7f2acb7fe640 1 -- 172.21.15.100:0/694767937 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad000ccc0 con 0x7f2adc108ed0 2024-07-26T13:38:54.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.021+0000 7f2acb7fe640 1 -- 172.21.15.100:0/694767937 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ad00054d0 con 0x7f2adc108ed0 2024-07-26T13:38:54.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.021+0000 7f2ae0e26640 1 -- 172.21.15.100:0/694767937 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2adc1a2260 con 0x7f2adc108ed0 2024-07-26T13:38:54.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.021+0000 7f2acb7fe640 1 -- 172.21.15.100:0/694767937 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad0010430 con 0x7f2adc108ed0 2024-07-26T13:38:54.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.021+0000 7f2ae0e26640 1 -- 172.21.15.100:0/694767937 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2adc1a8a80 con 0x7f2adc108ed0 2024-07-26T13:38:54.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.022+0000 7f2ae0e26640 1 -- 172.21.15.100:0/694767937 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2adc1039a0 con 0x7f2adc108ed0 2024-07-26T13:38:54.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.024+0000 7f2acb7fe640 1 -- 172.21.15.100:0/694767937 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f2ad0005000 con 0x7f2adc108ed0 2024-07-26T13:38:54.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.024+0000 7f2acb7fe640 1 --2- 172.21.15.100:0/694767937 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f2ac40b3c50 0x7f2ac40b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:54.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.025+0000 7f2ada59c640 1 --2- 172.21.15.100:0/694767937 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f2ac40b3c50 0x7f2ac40b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:54.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.025+0000 7f2acb7fe640 1 -- 172.21.15.100:0/694767937 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f2ad00e43e0 con 0x7f2adc108ed0 2024-07-26T13:38:54.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.025+0000 7f2ada59c640 1 --2- 172.21.15.100:0/694767937 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f2ac40b3c50 0x7f2ac40b6160 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2acc005e10 tx=0x7f2acc005da0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:54.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.028+0000 7f2acb7fe640 1 -- 172.21.15.100:0/694767937 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f2ad0055600 con 0x7f2adc108ed0 2024-07-26T13:38:54.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.206+0000 7f2ae0e26640 1 -- 172.21.15.100:0/694767937 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f2adc19d1f0 con 0x7f2adc108ed0 2024-07-26T13:38:54.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.206+0000 7f2acb7fe640 1 -- 172.21.15.100:0/694767937 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2adc19d1f0 con 0x7f2adc108ed0 2024-07-26T13:38:54.207 INFO:teuthology.orchestra.run.smithi100.stdout:231928234036 2024-07-26T13:38:54.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.208+0000 7f2ae0e26640 1 -- 172.21.15.100:0/694767937 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f2ac40b3c50 msgr2=0x7f2ac40b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:54.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.208+0000 7f2ae0e26640 1 --2- 172.21.15.100:0/694767937 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f2ac40b3c50 0x7f2ac40b6160 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2acc005e10 tx=0x7f2acc005da0 comp rx=0 tx=0).stop 2024-07-26T13:38:54.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.208+0000 7f2ae0e26640 1 -- 172.21.15.100:0/694767937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2adc108ed0 msgr2=0x7f2adc19c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:54.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.208+0000 7f2ae0e26640 1 --2- 172.21.15.100:0/694767937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2adc108ed0 0x7f2adc19c660 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2ad000e9c0 tx=0x7f2ad000ee80 comp rx=0 tx=0).stop 2024-07-26T13:38:54.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.208+0000 7f2ae0e26640 1 -- 172.21.15.100:0/694767937 shutdown_connections 2024-07-26T13:38:54.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.208+0000 7f2ae0e26640 1 --2- 172.21.15.100:0/694767937 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f2ac40b3c50 0x7f2ac40b6160 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:54.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.208+0000 7f2ae0e26640 1 --2- 172.21.15.100:0/694767937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2adc109820 0x7f2adc1a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:54.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.208+0000 7f2ae0e26640 1 --2- 172.21.15.100:0/694767937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2adc108ed0 0x7f2adc19c660 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:54.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.208+0000 7f2ae0e26640 1 --2- 172.21.15.100:0/694767937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2adc103590 0x7f2adc19c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:54.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.208+0000 7f2ae0e26640 1 -- 172.21.15.100:0/694767937 >> 172.21.15.100:0/694767937 conn(0x7f2adc07b280 msgr2=0x7f2adc106ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:54.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.209+0000 7f2ae0e26640 1 -- 172.21.15.100:0/694767937 shutdown_connections 2024-07-26T13:38:54.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:54.209+0000 7f2ae0e26640 1 -- 172.21.15.100:0/694767937 wait complete. 2024-07-26T13:38:54.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:54 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/694767937' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-07-26T13:38:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:54 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/694767937' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-07-26T13:38:54.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:54 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/694767937' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-07-26T13:38:55.040 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234030 got 231928234036 for osd.8 2024-07-26T13:38:55.040 DEBUG:teuthology.parallel:result is None 2024-07-26T13:38:55.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.246+0000 7ff0323f3640 1 -- 172.21.15.100:0/2423714782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff02c105cd0 msgr2=0x7ff02c1060e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:55.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.246+0000 7ff0323f3640 1 --2- 172.21.15.100:0/2423714782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff02c105cd0 0x7ff02c1060e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff020009900 tx=0x7ff02002a410 comp rx=0 tx=0).stop 2024-07-26T13:38:55.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.246+0000 7ff0323f3640 1 -- 172.21.15.100:0/2423714782 shutdown_connections 2024-07-26T13:38:55.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.246+0000 7ff0323f3640 1 --2- 172.21.15.100:0/2423714782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff02c106620 0x7ff02c110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:55.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.246+0000 7ff0323f3640 1 --2- 172.21.15.100:0/2423714782 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff02c105cd0 0x7ff02c1060e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:55.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.246+0000 7ff0323f3640 1 --2- 172.21.15.100:0/2423714782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff02c1019f0 0x7ff02c101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:55.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.246+0000 7ff0323f3640 1 -- 172.21.15.100:0/2423714782 >> 172.21.15.100:0/2423714782 conn(0x7ff02c0776f0 msgr2=0x7ff02c077b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:55.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.246+0000 7ff0323f3640 1 -- 172.21.15.100:0/2423714782 shutdown_connections 2024-07-26T13:38:55.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.246+0000 7ff0323f3640 1 -- 172.21.15.100:0/2423714782 wait complete. 2024-07-26T13:38:55.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.247+0000 7ff0323f3640 1 Processor -- start 2024-07-26T13:38:55.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.247+0000 7ff0323f3640 1 -- start start 2024-07-26T13:38:55.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.247+0000 7ff0323f3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff02c1019f0 0x7ff02c199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:55.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.247+0000 7ff0323f3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff02c105cd0 0x7ff02c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:55.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.247+0000 7ff0323f3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff02c106620 0x7ff02c19f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:55.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.247+0000 7ff0323f3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1134f0 con 0x7ff02c106620 2024-07-26T13:38:55.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.247+0000 7ff0323f3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c113370 con 0x7ff02c1019f0 2024-07-26T13:38:55.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.247+0000 7ff0323f3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c113670 con 0x7ff02c105cd0 2024-07-26T13:38:55.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.247+0000 7ff030990640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff02c106620 0x7ff02c19f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:55.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.247+0000 7ff030990640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff02c106620 0x7ff02c19f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46148/0 (socket says 172.21.15.100:46148) 2024-07-26T13:38:55.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.247+0000 7ff030990640 1 -- 172.21.15.100:0/2205245945 learned_addr learned my addr 172.21.15.100:0/2205245945 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:38:55.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.247+0000 7ff02bfff640 1 --2- 172.21.15.100:0/2205245945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff02c1019f0 0x7ff02c199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:55.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.247+0000 7ff02b7fe640 1 --2- 172.21.15.100:0/2205245945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff02c105cd0 0x7ff02c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:55.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.247+0000 7ff030990640 1 -- 172.21.15.100:0/2205245945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff02c105cd0 msgr2=0x7ff02c19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:55.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.247+0000 7ff030990640 1 --2- 172.21.15.100:0/2205245945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff02c105cd0 0x7ff02c19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:55.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.247+0000 7ff030990640 1 -- 172.21.15.100:0/2205245945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff02c1019f0 msgr2=0x7ff02c199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:55.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.247+0000 7ff030990640 1 --2- 172.21.15.100:0/2205245945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff02c1019f0 0x7ff02c199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:55.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.247+0000 7ff030990640 1 -- 172.21.15.100:0/2205245945 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff02c19fe90 con 0x7ff02c106620 2024-07-26T13:38:55.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.248+0000 7ff030990640 1 --2- 172.21.15.100:0/2205245945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff02c106620 0x7ff02c19f760 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff02400eb10 tx=0x7ff02400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:55.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.248+0000 7ff0297fa640 1 -- 172.21.15.100:0/2205245945 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff024004b20 con 0x7ff02c106620 2024-07-26T13:38:55.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.248+0000 7ff0323f3640 1 -- 172.21.15.100:0/2205245945 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff02c1a0180 con 0x7ff02c106620 2024-07-26T13:38:55.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.248+0000 7ff0297fa640 1 -- 172.21.15.100:0/2205245945 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff024004cc0 con 0x7ff02c106620 2024-07-26T13:38:55.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.248+0000 7ff0297fa640 1 -- 172.21.15.100:0/2205245945 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff024010670 con 0x7ff02c106620 2024-07-26T13:38:55.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.248+0000 7ff0323f3640 1 -- 172.21.15.100:0/2205245945 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff02c1a68e0 con 0x7ff02c106620 2024-07-26T13:38:55.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.249+0000 7ff0323f3640 1 -- 172.21.15.100:0/2205245945 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff02c101e00 con 0x7ff02c106620 2024-07-26T13:38:55.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.250+0000 7ff0297fa640 1 -- 172.21.15.100:0/2205245945 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7ff024005000 con 0x7ff02c106620 2024-07-26T13:38:55.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.251+0000 7ff0297fa640 1 --2- 172.21.15.100:0/2205245945 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff0140b3c20 0x7ff0140b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:55.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.251+0000 7ff02bfff640 1 --2- 172.21.15.100:0/2205245945 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff0140b3c20 0x7ff0140b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:55.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.251+0000 7ff0297fa640 1 -- 172.21.15.100:0/2205245945 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7ff0240e4470 con 0x7ff02c106620 2024-07-26T13:38:55.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.252+0000 7ff02bfff640 1 --2- 172.21.15.100:0/2205245945 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff0140b3c20 0x7ff0140b6130 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff01c009bd0 tx=0x7ff01c005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:55.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.254+0000 7ff0297fa640 1 -- 172.21.15.100:0/2205245945 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7ff024055600 con 0x7ff02c106620 2024-07-26T13:38:55.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.469+0000 7ff0323f3640 1 -- 172.21.15.100:0/2205245945 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7ff02c02e0c0 con 0x7ff02c106620 2024-07-26T13:38:55.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.469+0000 7ff0297fa640 1 -- 172.21.15.100:0/2205245945 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff0240ab9c0 con 0x7ff02c106620 2024-07-26T13:38:55.470 INFO:teuthology.orchestra.run.smithi100.stdout:60129542230 2024-07-26T13:38:55.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.471+0000 7ff0323f3640 1 -- 172.21.15.100:0/2205245945 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff0140b3c20 msgr2=0x7ff0140b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:55.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.471+0000 7ff0323f3640 1 --2- 172.21.15.100:0/2205245945 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff0140b3c20 0x7ff0140b6130 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff01c009bd0 tx=0x7ff01c005f90 comp rx=0 tx=0).stop 2024-07-26T13:38:55.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.471+0000 7ff0323f3640 1 -- 172.21.15.100:0/2205245945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff02c106620 msgr2=0x7ff02c19f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:55.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.471+0000 7ff0323f3640 1 --2- 172.21.15.100:0/2205245945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff02c106620 0x7ff02c19f760 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff02400eb10 tx=0x7ff02400c2d0 comp rx=0 tx=0).stop 2024-07-26T13:38:55.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.471+0000 7ff0323f3640 1 -- 172.21.15.100:0/2205245945 shutdown_connections 2024-07-26T13:38:55.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.471+0000 7ff0323f3640 1 --2- 172.21.15.100:0/2205245945 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff0140b3c20 0x7ff0140b6130 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:55.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.471+0000 7ff0323f3640 1 --2- 172.21.15.100:0/2205245945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff02c106620 0x7ff02c19f760 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:55.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.471+0000 7ff0323f3640 1 --2- 172.21.15.100:0/2205245945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff02c105cd0 0x7ff02c19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:55.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.471+0000 7ff0323f3640 1 --2- 172.21.15.100:0/2205245945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff02c1019f0 0x7ff02c199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:55.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.471+0000 7ff0323f3640 1 -- 172.21.15.100:0/2205245945 >> 172.21.15.100:0/2205245945 conn(0x7ff02c0776f0 msgr2=0x7ff02c106f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:55.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.471+0000 7ff0323f3640 1 -- 172.21.15.100:0/2205245945 shutdown_connections 2024-07-26T13:38:55.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:55.471+0000 7ff0323f3640 1 -- 172.21.15.100:0/2205245945 wait complete. 2024-07-26T13:38:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:55 smithi100 ceph-mon[32599]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-07-26T13:38:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:38:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:55 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2205245945' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-07-26T13:38:55.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:55 smithi167 ceph-mon[37140]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-07-26T13:38:55.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:38:55.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:55 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/2205245945' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-07-26T13:38:55.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:55 smithi138 ceph-mon[36954]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-07-26T13:38:55.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:38:55.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:55 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/2205245945' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-07-26T13:38:56.928 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542225 got 60129542230 for osd.1 2024-07-26T13:38:56.928 DEBUG:teuthology.parallel:result is None 2024-07-26T13:38:56.928 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-07-26T13:38:56.929 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph pg dump --format=json 2024-07-26T13:38:57.216 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:38:57.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:38:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:38:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:38:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:38:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:57 smithi100 ceph-mon[32599]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-07-26T13:38:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:38:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:38:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:38:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:38:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:57 smithi138 ceph-mon[36954]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-07-26T13:38:57.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:38:57.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:38:57.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:38:57.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:38:57.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:57 smithi167 ceph-mon[37140]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-07-26T13:38:58.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:38:58.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:38:58.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:38:58.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:38:58.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:38:58.462 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:38:58.462 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:38:58.462 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:38:58.462 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:38:58.462 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:38:58.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:38:58.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:38:58.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:38:58.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:38:58.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:38:58.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.887+0000 7f6b432e7640 1 -- 172.21.15.100:0/488857293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b3c102490 msgr2=0x7f6b3c1028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:58.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.887+0000 7f6b432e7640 1 --2- 172.21.15.100:0/488857293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b3c102490 0x7f6b3c1028a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c009ca0 tx=0x7f6b2c02a8c0 comp rx=0 tx=0).stop 2024-07-26T13:38:58.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.887+0000 7f6b432e7640 1 -- 172.21.15.100:0/488857293 shutdown_connections 2024-07-26T13:38:58.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.887+0000 7f6b432e7640 1 --2- 172.21.15.100:0/488857293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b3c108720 0x7f6b3c10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:58.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.887+0000 7f6b432e7640 1 --2- 172.21.15.100:0/488857293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b3c107dd0 0x7f6b3c1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:58.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.887+0000 7f6b432e7640 1 --2- 172.21.15.100:0/488857293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b3c102490 0x7f6b3c1028a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:58.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.887+0000 7f6b432e7640 1 -- 172.21.15.100:0/488857293 >> 172.21.15.100:0/488857293 conn(0x7f6b3c079160 msgr2=0x7f6b3c079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:58.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.887+0000 7f6b432e7640 1 -- 172.21.15.100:0/488857293 shutdown_connections 2024-07-26T13:38:58.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.887+0000 7f6b432e7640 1 -- 172.21.15.100:0/488857293 wait complete. 2024-07-26T13:38:58.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.888+0000 7f6b432e7640 1 Processor -- start 2024-07-26T13:38:58.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.888+0000 7f6b432e7640 1 -- start start 2024-07-26T13:38:58.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.888+0000 7f6b432e7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b3c102490 0x7f6b3c199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:58.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.888+0000 7f6b432e7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b3c107dd0 0x7f6b3c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:58.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.888+0000 7f6b432e7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b3c108720 0x7f6b3c19f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:58.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.888+0000 7f6b432e7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c1134a0 con 0x7f6b3c107dd0 2024-07-26T13:38:58.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.888+0000 7f6b432e7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c113320 con 0x7f6b3c108720 2024-07-26T13:38:58.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.888+0000 7f6b432e7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c113620 con 0x7f6b3c102490 2024-07-26T13:38:58.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.888+0000 7f6b40882640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b3c107dd0 0x7f6b3c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:58.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.888+0000 7f6b40882640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b3c107dd0 0x7f6b3c19a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46168/0 (socket says 172.21.15.100:46168) 2024-07-26T13:38:58.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.888+0000 7f6b40882640 1 -- 172.21.15.100:0/3975045565 learned_addr learned my addr 172.21.15.100:0/3975045565 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:38:58.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.888+0000 7f6b41083640 1 --2- 172.21.15.100:0/3975045565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b3c102490 0x7f6b3c199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:58.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.888+0000 7f6b41884640 1 --2- 172.21.15.100:0/3975045565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b3c108720 0x7f6b3c19f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:58.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.888+0000 7f6b40882640 1 -- 172.21.15.100:0/3975045565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b3c102490 msgr2=0x7f6b3c199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:58.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.888+0000 7f6b40882640 1 --2- 172.21.15.100:0/3975045565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b3c102490 0x7f6b3c199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:58.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.888+0000 7f6b40882640 1 -- 172.21.15.100:0/3975045565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b3c108720 msgr2=0x7f6b3c19f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:58.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.888+0000 7f6b40882640 1 --2- 172.21.15.100:0/3975045565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b3c108720 0x7f6b3c19f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:58.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.888+0000 7f6b40882640 1 -- 172.21.15.100:0/3975045565 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b3c19fda0 con 0x7f6b3c107dd0 2024-07-26T13:38:58.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.889+0000 7f6b41884640 1 --2- 172.21.15.100:0/3975045565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b3c108720 0x7f6b3c19f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:38:58.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.889+0000 7f6b40882640 1 --2- 172.21.15.100:0/3975045565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b3c107dd0 0x7f6b3c19a490 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6b3000d740 tx=0x7f6b3000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:58.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.889+0000 7f6b3a7fc640 1 -- 172.21.15.100:0/3975045565 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b30017070 con 0x7f6b3c107dd0 2024-07-26T13:38:58.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.889+0000 7f6b3a7fc640 1 -- 172.21.15.100:0/3975045565 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b3000de50 con 0x7f6b3c107dd0 2024-07-26T13:38:58.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.889+0000 7f6b432e7640 1 -- 172.21.15.100:0/3975045565 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b3c1a0090 con 0x7f6b3c107dd0 2024-07-26T13:38:58.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.889+0000 7f6b3a7fc640 1 -- 172.21.15.100:0/3975045565 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b30004ce0 con 0x7f6b3c107dd0 2024-07-26T13:38:58.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.889+0000 7f6b432e7640 1 -- 172.21.15.100:0/3975045565 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b3c1a68b0 con 0x7f6b3c107dd0 2024-07-26T13:38:58.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.890+0000 7f6b432e7640 1 -- 172.21.15.100:0/3975045565 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b3c103710 con 0x7f6b3c107dd0 2024-07-26T13:38:58.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.891+0000 7f6b3a7fc640 1 -- 172.21.15.100:0/3975045565 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f6b30004e80 con 0x7f6b3c107dd0 2024-07-26T13:38:58.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.892+0000 7f6b3a7fc640 1 --2- 172.21.15.100:0/3975045565 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6b240b3c50 0x7f6b240b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:38:58.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.892+0000 7f6b41083640 1 --2- 172.21.15.100:0/3975045565 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6b240b3c50 0x7f6b240b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:38:58.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.892+0000 7f6b3a7fc640 1 -- 172.21.15.100:0/3975045565 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f6b300e3b70 con 0x7f6b3c107dd0 2024-07-26T13:38:58.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.893+0000 7f6b41083640 1 --2- 172.21.15.100:0/3975045565 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6b240b3c50 0x7f6b240b6160 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c005e30 tx=0x7f6b2c009360 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:38:58.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:58.895+0000 7f6b3a7fc640 1 -- 172.21.15.100:0/3975045565 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f6b300ab060 con 0x7f6b3c107dd0 2024-07-26T13:38:59.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:59.064+0000 7f6b432e7640 1 -- 172.21.15.100:0/3975045565 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6b3c063230 con 0x7f6b240b3c50 2024-07-26T13:38:59.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:59.069+0000 7f6b3a7fc640 1 -- 172.21.15.100:0/3975045565 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259056 (secure 0 0 0) 0x7f6b3c063230 con 0x7f6b240b3c50 2024-07-26T13:38:59.072 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:38:59.073 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-07-26T13:38:59.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:59.071+0000 7f6b432e7640 1 -- 172.21.15.100:0/3975045565 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6b240b3c50 msgr2=0x7f6b240b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:59.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:59.071+0000 7f6b432e7640 1 --2- 172.21.15.100:0/3975045565 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6b240b3c50 0x7f6b240b6160 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c005e30 tx=0x7f6b2c009360 comp rx=0 tx=0).stop 2024-07-26T13:38:59.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:59.071+0000 7f6b432e7640 1 -- 172.21.15.100:0/3975045565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b3c107dd0 msgr2=0x7f6b3c19a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:38:59.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:59.071+0000 7f6b432e7640 1 --2- 172.21.15.100:0/3975045565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b3c107dd0 0x7f6b3c19a490 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6b3000d740 tx=0x7f6b3000dc00 comp rx=0 tx=0).stop 2024-07-26T13:38:59.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:59.072+0000 7f6b432e7640 1 -- 172.21.15.100:0/3975045565 shutdown_connections 2024-07-26T13:38:59.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:59.072+0000 7f6b432e7640 1 --2- 172.21.15.100:0/3975045565 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6b240b3c50 0x7f6b240b6160 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:59.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:59.072+0000 7f6b432e7640 1 --2- 172.21.15.100:0/3975045565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b3c108720 0x7f6b3c19f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:59.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:59.072+0000 7f6b432e7640 1 --2- 172.21.15.100:0/3975045565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b3c107dd0 0x7f6b3c19a490 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:59.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:59.072+0000 7f6b432e7640 1 --2- 172.21.15.100:0/3975045565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b3c102490 0x7f6b3c199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:38:59.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:59.072+0000 7f6b432e7640 1 -- 172.21.15.100:0/3975045565 >> 172.21.15.100:0/3975045565 conn(0x7f6b3c079160 msgr2=0x7f6b3c100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:38:59.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:59.072+0000 7f6b432e7640 1 -- 172.21.15.100:0/3975045565 shutdown_connections 2024-07-26T13:38:59.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:38:59.072+0000 7f6b432e7640 1 -- 172.21.15.100:0/3975045565 wait complete. 2024-07-26T13:38:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:38:59 smithi138 ceph-mon[36954]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-07-26T13:38:59.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:38:59 smithi100 ceph-mon[32599]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-07-26T13:38:59.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:38:59 smithi167 ceph-mon[37140]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-07-26T13:38:59.925 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2024-07-26T13:38:58.390606+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657860,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":698,"num_read_kb":449,"num_write":299,"num_write_kb":2272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":2952484,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":742,"ondisk_log_size":742,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372492,"kb_used_data":10860,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569332,"statfs":{"total":1151940427776,"available":1151558995968,"internally_reserved":0,"allocated":11120640,"data_stored":7287744,"data_compressed":177282,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":531,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":38,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003792"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387423+0000","last_change":"2024-07-26T13:36:59.462289+0000","last_active":"2024-07-26T13:38:46.387423+0000","last_peered":"2024-07-26T13:38:46.387423+0000","last_clean":"2024-07-26T13:38:46.387423+0000","last_became_active":"2024-07-26T13:36:59.460311+0000","last_became_peered":"2024-07-26T13:36:59.460311+0000","last_unstale":"2024-07-26T13:38:46.387423+0000","last_undegraded":"2024-07-26T13:38:46.387423+0000","last_fullsized":"2024-07-26T13:38:46.387423+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:04:59.123716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383883+0000","last_change":"2024-07-26T13:36:59.463218+0000","last_active":"2024-07-26T13:38:46.383883+0000","last_peered":"2024-07-26T13:38:46.383883+0000","last_clean":"2024-07-26T13:38:46.383883+0000","last_became_active":"2024-07-26T13:36:59.462889+0000","last_became_peered":"2024-07-26T13:36:59.462889+0000","last_unstale":"2024-07-26T13:38:46.383883+0000","last_undegraded":"2024-07-26T13:38:46.383883+0000","last_fullsized":"2024-07-26T13:38:46.383883+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:30:46.788669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383357+0000","last_change":"2024-07-26T13:36:59.456113+0000","last_active":"2024-07-26T13:38:46.383357+0000","last_peered":"2024-07-26T13:38:46.383357+0000","last_clean":"2024-07-26T13:38:46.383357+0000","last_became_active":"2024-07-26T13:36:59.455622+0000","last_became_peered":"2024-07-26T13:36:59.455622+0000","last_unstale":"2024-07-26T13:38:46.383357+0000","last_undegraded":"2024-07-26T13:38:46.383357+0000","last_fullsized":"2024-07-26T13:38:46.383357+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:36:26.474799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.368653+0000","last_change":"2024-07-26T13:37:17.555947+0000","last_active":"2024-07-26T13:38:46.368653+0000","last_peered":"2024-07-26T13:38:46.368653+0000","last_clean":"2024-07-26T13:38:46.368653+0000","last_became_active":"2024-07-26T13:37:17.555607+0000","last_became_peered":"2024-07-26T13:37:17.555607+0000","last_unstale":"2024-07-26T13:38:46.368653+0000","last_undegraded":"2024-07-26T13:38:46.368653+0000","last_fullsized":"2024-07-26T13:38:46.368653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:19:59.603225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.372013+0000","last_change":"2024-07-26T13:36:59.474896+0000","last_active":"2024-07-26T13:38:46.372013+0000","last_peered":"2024-07-26T13:38:46.372013+0000","last_clean":"2024-07-26T13:38:46.372013+0000","last_became_active":"2024-07-26T13:36:59.474516+0000","last_became_peered":"2024-07-26T13:36:59.474516+0000","last_unstale":"2024-07-26T13:38:46.372013+0000","last_undegraded":"2024-07-26T13:38:46.372013+0000","last_fullsized":"2024-07-26T13:38:46.372013+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:38:18.994280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.357551+0000","last_change":"2024-07-26T13:37:17.556191+0000","last_active":"2024-07-26T13:38:46.357551+0000","last_peered":"2024-07-26T13:38:46.357551+0000","last_clean":"2024-07-26T13:38:46.357551+0000","last_became_active":"2024-07-26T13:37:17.555715+0000","last_became_peered":"2024-07-26T13:37:17.555715+0000","last_unstale":"2024-07-26T13:38:46.357551+0000","last_undegraded":"2024-07-26T13:38:46.357551+0000","last_fullsized":"2024-07-26T13:38:46.357551+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:03:18.352049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383606+0000","last_change":"2024-07-26T13:36:59.469515+0000","last_active":"2024-07-26T13:38:46.383606+0000","last_peered":"2024-07-26T13:38:46.383606+0000","last_clean":"2024-07-26T13:38:46.383606+0000","last_became_active":"2024-07-26T13:36:59.469106+0000","last_became_peered":"2024-07-26T13:36:59.469106+0000","last_unstale":"2024-07-26T13:38:46.383606+0000","last_undegraded":"2024-07-26T13:38:46.383606+0000","last_fullsized":"2024-07-26T13:38:46.383606+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:10:05.455329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.372753+0000","last_change":"2024-07-26T13:37:17.549365+0000","last_active":"2024-07-26T13:38:46.372753+0000","last_peered":"2024-07-26T13:38:46.372753+0000","last_clean":"2024-07-26T13:38:46.372753+0000","last_became_active":"2024-07-26T13:37:17.549036+0000","last_became_peered":"2024-07-26T13:37:17.549036+0000","last_unstale":"2024-07-26T13:38:46.372753+0000","last_undegraded":"2024-07-26T13:38:46.372753+0000","last_fullsized":"2024-07-26T13:38:46.372753+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:27:43.930981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387552+0000","last_change":"2024-07-26T13:36:59.459306+0000","last_active":"2024-07-26T13:38:46.387552+0000","last_peered":"2024-07-26T13:38:46.387552+0000","last_clean":"2024-07-26T13:38:46.387552+0000","last_became_active":"2024-07-26T13:36:59.458714+0000","last_became_peered":"2024-07-26T13:36:59.458714+0000","last_unstale":"2024-07-26T13:38:46.387552+0000","last_undegraded":"2024-07-26T13:38:46.387552+0000","last_fullsized":"2024-07-26T13:38:46.387552+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:34:18.512454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.376639+0000","last_change":"2024-07-26T13:36:59.466556+0000","last_active":"2024-07-26T13:38:46.376639+0000","last_peered":"2024-07-26T13:38:46.376639+0000","last_clean":"2024-07-26T13:38:46.376639+0000","last_became_active":"2024-07-26T13:36:59.466147+0000","last_became_peered":"2024-07-26T13:36:59.466147+0000","last_unstale":"2024-07-26T13:38:46.376639+0000","last_undegraded":"2024-07-26T13:38:46.376639+0000","last_fullsized":"2024-07-26T13:38:46.376639+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:43:11.359314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387429+0000","last_change":"2024-07-26T13:36:59.473685+0000","last_active":"2024-07-26T13:38:46.387429+0000","last_peered":"2024-07-26T13:38:46.387429+0000","last_clean":"2024-07-26T13:38:46.387429+0000","last_became_active":"2024-07-26T13:36:59.473314+0000","last_became_peered":"2024-07-26T13:36:59.473314+0000","last_unstale":"2024-07-26T13:38:46.387429+0000","last_undegraded":"2024-07-26T13:38:46.387429+0000","last_fullsized":"2024-07-26T13:38:46.387429+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:41:51.612348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.367874+0000","last_change":"2024-07-26T13:36:59.471586+0000","last_active":"2024-07-26T13:38:46.367874+0000","last_peered":"2024-07-26T13:38:46.367874+0000","last_clean":"2024-07-26T13:38:46.367874+0000","last_became_active":"2024-07-26T13:36:59.470995+0000","last_became_peered":"2024-07-26T13:36:59.470995+0000","last_unstale":"2024-07-26T13:38:46.367874+0000","last_undegraded":"2024-07-26T13:38:46.367874+0000","last_fullsized":"2024-07-26T13:38:46.367874+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:54:07.957937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.357393+0000","last_change":"2024-07-26T13:36:59.460018+0000","last_active":"2024-07-26T13:38:46.357393+0000","last_peered":"2024-07-26T13:38:46.357393+0000","last_clean":"2024-07-26T13:38:46.357393+0000","last_became_active":"2024-07-26T13:36:59.459671+0000","last_became_peered":"2024-07-26T13:36:59.459671+0000","last_unstale":"2024-07-26T13:38:46.357393+0000","last_undegraded":"2024-07-26T13:38:46.357393+0000","last_fullsized":"2024-07-26T13:38:46.357393+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:19:33.923360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.374398+0000","last_change":"2024-07-26T13:36:59.470271+0000","last_active":"2024-07-26T13:38:46.374398+0000","last_peered":"2024-07-26T13:38:46.374398+0000","last_clean":"2024-07-26T13:38:46.374398+0000","last_became_active":"2024-07-26T13:36:59.469941+0000","last_became_peered":"2024-07-26T13:36:59.469941+0000","last_unstale":"2024-07-26T13:38:46.374398+0000","last_undegraded":"2024-07-26T13:38:46.374398+0000","last_fullsized":"2024-07-26T13:38:46.374398+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:38:39.329366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383031+0000","last_change":"2024-07-26T13:36:59.460133+0000","last_active":"2024-07-26T13:38:46.383031+0000","last_peered":"2024-07-26T13:38:46.383031+0000","last_clean":"2024-07-26T13:38:46.383031+0000","last_became_active":"2024-07-26T13:36:59.459794+0000","last_became_peered":"2024-07-26T13:36:59.459794+0000","last_unstale":"2024-07-26T13:38:46.383031+0000","last_undegraded":"2024-07-26T13:38:46.383031+0000","last_fullsized":"2024-07-26T13:38:46.383031+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:15:36.024347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.385893+0000","last_change":"2024-07-26T13:36:59.462390+0000","last_active":"2024-07-26T13:38:46.385893+0000","last_peered":"2024-07-26T13:38:46.385893+0000","last_clean":"2024-07-26T13:38:46.385893+0000","last_became_active":"2024-07-26T13:36:59.461968+0000","last_became_peered":"2024-07-26T13:36:59.461968+0000","last_unstale":"2024-07-26T13:38:46.385893+0000","last_undegraded":"2024-07-26T13:38:46.385893+0000","last_fullsized":"2024-07-26T13:38:46.385893+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:49:47.142472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387521+0000","last_change":"2024-07-26T13:36:59.460206+0000","last_active":"2024-07-26T13:38:46.387521+0000","last_peered":"2024-07-26T13:38:46.387521+0000","last_clean":"2024-07-26T13:38:46.387521+0000","last_became_active":"2024-07-26T13:36:59.459811+0000","last_became_peered":"2024-07-26T13:36:59.459811+0000","last_unstale":"2024-07-26T13:38:46.387521+0000","last_undegraded":"2024-07-26T13:38:46.387521+0000","last_fullsized":"2024-07-26T13:38:46.387521+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:46:57.112314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387875+0000","last_change":"2024-07-26T13:36:59.460197+0000","last_active":"2024-07-26T13:38:46.387875+0000","last_peered":"2024-07-26T13:38:46.387875+0000","last_clean":"2024-07-26T13:38:46.387875+0000","last_became_active":"2024-07-26T13:36:59.459686+0000","last_became_peered":"2024-07-26T13:36:59.459686+0000","last_unstale":"2024-07-26T13:38:46.387875+0000","last_undegraded":"2024-07-26T13:38:46.387875+0000","last_fullsized":"2024-07-26T13:38:46.387875+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T13:39:22.407376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.373295+0000","last_change":"2024-07-26T13:36:59.462545+0000","last_active":"2024-07-26T13:38:46.373295+0000","last_peered":"2024-07-26T13:38:46.373295+0000","last_clean":"2024-07-26T13:38:46.373295+0000","last_became_active":"2024-07-26T13:36:59.462115+0000","last_became_peered":"2024-07-26T13:36:59.462115+0000","last_unstale":"2024-07-26T13:38:46.373295+0000","last_undegraded":"2024-07-26T13:38:46.373295+0000","last_fullsized":"2024-07-26T13:38:46.373295+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T21:44:13.715076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383157+0000","last_change":"2024-07-26T13:36:59.465317+0000","last_active":"2024-07-26T13:38:46.383157+0000","last_peered":"2024-07-26T13:38:46.383157+0000","last_clean":"2024-07-26T13:38:46.383157+0000","last_became_active":"2024-07-26T13:36:59.464943+0000","last_became_peered":"2024-07-26T13:36:59.464943+0000","last_unstale":"2024-07-26T13:38:46.383157+0000","last_undegraded":"2024-07-26T13:38:46.383157+0000","last_fullsized":"2024-07-26T13:38:46.383157+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T13:53:54.160584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.352946+0000","last_change":"2024-07-26T13:36:59.462884+0000","last_active":"2024-07-26T13:38:46.352946+0000","last_peered":"2024-07-26T13:38:46.352946+0000","last_clean":"2024-07-26T13:38:46.352946+0000","last_became_active":"2024-07-26T13:36:59.462564+0000","last_became_peered":"2024-07-26T13:36:59.462564+0000","last_unstale":"2024-07-26T13:38:46.352946+0000","last_undegraded":"2024-07-26T13:38:46.352946+0000","last_fullsized":"2024-07-26T13:38:46.352946+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:50:19.549148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.386880+0000","last_change":"2024-07-26T13:36:59.468716+0000","last_active":"2024-07-26T13:38:46.386880+0000","last_peered":"2024-07-26T13:38:46.386880+0000","last_clean":"2024-07-26T13:38:46.386880+0000","last_became_active":"2024-07-26T13:36:59.468048+0000","last_became_peered":"2024-07-26T13:36:59.468048+0000","last_unstale":"2024-07-26T13:38:46.386880+0000","last_undegraded":"2024-07-26T13:38:46.386880+0000","last_fullsized":"2024-07-26T13:38:46.386880+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:51:29.510047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383080+0000","last_change":"2024-07-26T13:36:59.466565+0000","last_active":"2024-07-26T13:38:46.383080+0000","last_peered":"2024-07-26T13:38:46.383080+0000","last_clean":"2024-07-26T13:38:46.383080+0000","last_became_active":"2024-07-26T13:36:59.466235+0000","last_became_peered":"2024-07-26T13:36:59.466235+0000","last_unstale":"2024-07-26T13:38:46.383080+0000","last_undegraded":"2024-07-26T13:38:46.383080+0000","last_fullsized":"2024-07-26T13:38:46.383080+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:28:44.326821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.371778+0000","last_change":"2024-07-26T13:36:59.458753+0000","last_active":"2024-07-26T13:38:46.371778+0000","last_peered":"2024-07-26T13:38:46.371778+0000","last_clean":"2024-07-26T13:38:46.371778+0000","last_became_active":"2024-07-26T13:36:59.458333+0000","last_became_peered":"2024-07-26T13:36:59.458333+0000","last_unstale":"2024-07-26T13:38:46.371778+0000","last_undegraded":"2024-07-26T13:38:46.371778+0000","last_fullsized":"2024-07-26T13:38:46.371778+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:33:54.806212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.366828+0000","last_change":"2024-07-26T13:36:59.468657+0000","last_active":"2024-07-26T13:38:46.366828+0000","last_peered":"2024-07-26T13:38:46.366828+0000","last_clean":"2024-07-26T13:38:46.366828+0000","last_became_active":"2024-07-26T13:36:59.468121+0000","last_became_peered":"2024-07-26T13:36:59.468121+0000","last_unstale":"2024-07-26T13:38:46.366828+0000","last_undegraded":"2024-07-26T13:38:46.366828+0000","last_fullsized":"2024-07-26T13:38:46.366828+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:23:53.959836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.377483+0000","last_change":"2024-07-26T13:36:59.469067+0000","last_active":"2024-07-26T13:38:46.377483+0000","last_peered":"2024-07-26T13:38:46.377483+0000","last_clean":"2024-07-26T13:38:46.377483+0000","last_became_active":"2024-07-26T13:36:59.468706+0000","last_became_peered":"2024-07-26T13:36:59.468706+0000","last_unstale":"2024-07-26T13:38:46.377483+0000","last_undegraded":"2024-07-26T13:38:46.377483+0000","last_fullsized":"2024-07-26T13:38:46.377483+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:21:35.974618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.378127+0000","last_change":"2024-07-26T13:36:59.469807+0000","last_active":"2024-07-26T13:38:46.378127+0000","last_peered":"2024-07-26T13:38:46.378127+0000","last_clean":"2024-07-26T13:38:46.378127+0000","last_became_active":"2024-07-26T13:36:59.469020+0000","last_became_peered":"2024-07-26T13:36:59.469020+0000","last_unstale":"2024-07-26T13:38:46.378127+0000","last_undegraded":"2024-07-26T13:38:46.378127+0000","last_fullsized":"2024-07-26T13:38:46.378127+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:29:57.931598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.372785+0000","last_change":"2024-07-26T13:36:59.465236+0000","last_active":"2024-07-26T13:38:46.372785+0000","last_peered":"2024-07-26T13:38:46.372785+0000","last_clean":"2024-07-26T13:38:46.372785+0000","last_became_active":"2024-07-26T13:36:59.464415+0000","last_became_peered":"2024-07-26T13:36:59.464415+0000","last_unstale":"2024-07-26T13:38:46.372785+0000","last_undegraded":"2024-07-26T13:38:46.372785+0000","last_fullsized":"2024-07-26T13:38:46.372785+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:46:20.373688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.375861+0000","last_change":"2024-07-26T13:36:59.466113+0000","last_active":"2024-07-26T13:38:46.375861+0000","last_peered":"2024-07-26T13:38:46.375861+0000","last_clean":"2024-07-26T13:38:46.375861+0000","last_became_active":"2024-07-26T13:36:59.465743+0000","last_became_peered":"2024-07-26T13:36:59.465743+0000","last_unstale":"2024-07-26T13:38:46.375861+0000","last_undegraded":"2024-07-26T13:38:46.375861+0000","last_fullsized":"2024-07-26T13:38:46.375861+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:14:41.839204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.380300+0000","last_change":"2024-07-26T13:36:59.460672+0000","last_active":"2024-07-26T13:38:46.380300+0000","last_peered":"2024-07-26T13:38:46.380300+0000","last_clean":"2024-07-26T13:38:46.380300+0000","last_became_active":"2024-07-26T13:36:59.460247+0000","last_became_peered":"2024-07-26T13:36:59.460247+0000","last_unstale":"2024-07-26T13:38:46.380300+0000","last_undegraded":"2024-07-26T13:38:46.380300+0000","last_fullsized":"2024-07-26T13:38:46.380300+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:37:51.109933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.373301+0000","last_change":"2024-07-26T13:36:59.464784+0000","last_active":"2024-07-26T13:38:46.373301+0000","last_peered":"2024-07-26T13:38:46.373301+0000","last_clean":"2024-07-26T13:38:46.373301+0000","last_became_active":"2024-07-26T13:36:59.464261+0000","last_became_peered":"2024-07-26T13:36:59.464261+0000","last_unstale":"2024-07-26T13:38:46.373301+0000","last_undegraded":"2024-07-26T13:38:46.373301+0000","last_fullsized":"2024-07-26T13:38:46.373301+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:43:10.134037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.384861+0000","last_change":"2024-07-26T13:36:59.460421+0000","last_active":"2024-07-26T13:38:46.384861+0000","last_peered":"2024-07-26T13:38:46.384861+0000","last_clean":"2024-07-26T13:38:46.384861+0000","last_became_active":"2024-07-26T13:36:59.460010+0000","last_became_peered":"2024-07-26T13:36:59.460010+0000","last_unstale":"2024-07-26T13:38:46.384861+0000","last_undegraded":"2024-07-26T13:38:46.384861+0000","last_fullsized":"2024-07-26T13:38:46.384861+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:16:21.651673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.358087+0000","last_change":"2024-07-26T13:36:59.468826+0000","last_active":"2024-07-26T13:38:46.358087+0000","last_peered":"2024-07-26T13:38:46.358087+0000","last_clean":"2024-07-26T13:38:46.358087+0000","last_became_active":"2024-07-26T13:36:59.468406+0000","last_became_peered":"2024-07-26T13:36:59.468406+0000","last_unstale":"2024-07-26T13:38:46.358087+0000","last_undegraded":"2024-07-26T13:38:46.358087+0000","last_fullsized":"2024-07-26T13:38:46.358087+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:22:10.590545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'7","reported_seq":151,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:55.716965+0000","last_change":"2024-07-26T13:37:17.559014+0000","last_active":"2024-07-26T13:38:55.716965+0000","last_peered":"2024-07-26T13:38:55.716965+0000","last_clean":"2024-07-26T13:38:55.716965+0000","last_became_active":"2024-07-26T13:37:17.552325+0000","last_became_peered":"2024-07-26T13:37:17.552325+0000","last_unstale":"2024-07-26T13:38:55.716965+0000","last_undegraded":"2024-07-26T13:38:55.716965+0000","last_fullsized":"2024-07-26T13:38:55.716965+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:26:26.790337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.378262+0000","last_change":"2024-07-26T13:37:17.562871+0000","last_active":"2024-07-26T13:38:46.378262+0000","last_peered":"2024-07-26T13:38:46.378262+0000","last_clean":"2024-07-26T13:38:46.378262+0000","last_became_active":"2024-07-26T13:37:17.562363+0000","last_became_peered":"2024-07-26T13:37:17.562363+0000","last_unstale":"2024-07-26T13:38:46.378262+0000","last_undegraded":"2024-07-26T13:38:46.378262+0000","last_fullsized":"2024-07-26T13:38:46.378262+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:25:36.172091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"79'3","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.547720+0000","last_change":"2024-07-26T13:36:56.434778+0000","last_active":"2024-07-26T13:37:17.547720+0000","last_peered":"2024-07-26T13:37:17.547720+0000","last_clean":"2024-07-26T13:37:17.547720+0000","last_became_active":"2024-07-26T13:36:56.434288+0000","last_became_peered":"2024-07-26T13:36:56.434288+0000","last_unstale":"2024-07-26T13:37:17.547720+0000","last_undegraded":"2024-07-26T13:37:17.547720+0000","last_fullsized":"2024-07-26T13:37:17.547720+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:24:22.990358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.565086+0000","last_change":"2024-07-26T13:36:56.424665+0000","last_active":"2024-07-26T13:37:17.565086+0000","last_peered":"2024-07-26T13:37:17.565086+0000","last_clean":"2024-07-26T13:37:17.565086+0000","last_became_active":"2024-07-26T13:36:56.424290+0000","last_became_peered":"2024-07-26T13:36:56.424290+0000","last_unstale":"2024-07-26T13:37:17.565086+0000","last_undegraded":"2024-07-26T13:37:17.565086+0000","last_fullsized":"2024-07-26T13:37:17.565086+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:49:59.520194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383641+0000","last_change":"2024-07-26T13:36:59.454609+0000","last_active":"2024-07-26T13:38:46.383641+0000","last_peered":"2024-07-26T13:38:46.383641+0000","last_clean":"2024-07-26T13:38:46.383641+0000","last_became_active":"2024-07-26T13:36:59.454228+0000","last_became_peered":"2024-07-26T13:36:59.454228+0000","last_unstale":"2024-07-26T13:38:46.383641+0000","last_undegraded":"2024-07-26T13:38:46.383641+0000","last_fullsized":"2024-07-26T13:38:46.383641+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:02:06.244433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.547076+0000","last_change":"2024-07-26T13:36:56.430268+0000","last_active":"2024-07-26T13:37:17.547076+0000","last_peered":"2024-07-26T13:37:17.547076+0000","last_clean":"2024-07-26T13:37:17.547076+0000","last_became_active":"2024-07-26T13:36:56.428583+0000","last_became_peered":"2024-07-26T13:36:56.428583+0000","last_unstale":"2024-07-26T13:37:17.547076+0000","last_undegraded":"2024-07-26T13:37:17.547076+0000","last_fullsized":"2024-07-26T13:37:17.547076+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:07:05.864886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.379574+0000","last_change":"2024-07-26T13:36:59.456188+0000","last_active":"2024-07-26T13:38:46.379574+0000","last_peered":"2024-07-26T13:38:46.379574+0000","last_clean":"2024-07-26T13:38:46.379574+0000","last_became_active":"2024-07-26T13:36:59.455842+0000","last_became_peered":"2024-07-26T13:36:59.455842+0000","last_unstale":"2024-07-26T13:38:46.379574+0000","last_undegraded":"2024-07-26T13:38:46.379574+0000","last_fullsized":"2024-07-26T13:38:46.379574+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:00:19.162239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":126,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:51.710172+0000","last_change":"2024-07-26T13:36:56.432818+0000","last_active":"2024-07-26T13:38:51.710172+0000","last_peered":"2024-07-26T13:38:51.710172+0000","last_clean":"2024-07-26T13:38:51.710172+0000","last_became_active":"2024-07-26T13:36:56.432419+0000","last_became_peered":"2024-07-26T13:36:56.432419+0000","last_unstale":"2024-07-26T13:38:51.710172+0000","last_undegraded":"2024-07-26T13:38:51.710172+0000","last_fullsized":"2024-07-26T13:38:51.710172+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:55:51.972470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.386803+0000","last_change":"2024-07-26T13:36:59.460033+0000","last_active":"2024-07-26T13:38:46.386803+0000","last_peered":"2024-07-26T13:38:46.386803+0000","last_clean":"2024-07-26T13:38:46.386803+0000","last_became_active":"2024-07-26T13:36:59.459562+0000","last_became_peered":"2024-07-26T13:36:59.459562+0000","last_unstale":"2024-07-26T13:38:46.386803+0000","last_undegraded":"2024-07-26T13:38:46.386803+0000","last_fullsized":"2024-07-26T13:38:46.386803+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:58:14.155973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551314+0000","last_change":"2024-07-26T13:36:56.432563+0000","last_active":"2024-07-26T13:37:17.551314+0000","last_peered":"2024-07-26T13:37:17.551314+0000","last_clean":"2024-07-26T13:37:17.551314+0000","last_became_active":"2024-07-26T13:36:56.431134+0000","last_became_peered":"2024-07-26T13:36:56.431134+0000","last_unstale":"2024-07-26T13:37:17.551314+0000","last_undegraded":"2024-07-26T13:37:17.551314+0000","last_fullsized":"2024-07-26T13:37:17.551314+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:42:00.653345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.377720+0000","last_change":"2024-07-26T13:36:59.469782+0000","last_active":"2024-07-26T13:38:46.377720+0000","last_peered":"2024-07-26T13:38:46.377720+0000","last_clean":"2024-07-26T13:38:46.377720+0000","last_became_active":"2024-07-26T13:36:59.469412+0000","last_became_peered":"2024-07-26T13:36:59.469412+0000","last_unstale":"2024-07-26T13:38:46.377720+0000","last_undegraded":"2024-07-26T13:38:46.377720+0000","last_fullsized":"2024-07-26T13:38:46.377720+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:28:07.741486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.548023+0000","last_change":"2024-07-26T13:36:56.488981+0000","last_active":"2024-07-26T13:37:17.548023+0000","last_peered":"2024-07-26T13:37:17.548023+0000","last_clean":"2024-07-26T13:37:17.548023+0000","last_became_active":"2024-07-26T13:36:56.488480+0000","last_became_peered":"2024-07-26T13:36:56.488480+0000","last_unstale":"2024-07-26T13:37:17.548023+0000","last_undegraded":"2024-07-26T13:37:17.548023+0000","last_fullsized":"2024-07-26T13:37:17.548023+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:41:10.041007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.386815+0000","last_change":"2024-07-26T13:36:59.473183+0000","last_active":"2024-07-26T13:38:46.386815+0000","last_peered":"2024-07-26T13:38:46.386815+0000","last_clean":"2024-07-26T13:38:46.386815+0000","last_became_active":"2024-07-26T13:36:59.472814+0000","last_became_peered":"2024-07-26T13:36:59.472814+0000","last_unstale":"2024-07-26T13:38:46.386815+0000","last_undegraded":"2024-07-26T13:38:46.386815+0000","last_fullsized":"2024-07-26T13:38:46.386815+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:57:03.396384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'32","reported_seq":259,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:52.730574+0000","last_change":"2024-07-26T13:36:56.435034+0000","last_active":"2024-07-26T13:38:52.730574+0000","last_peered":"2024-07-26T13:38:52.730574+0000","last_clean":"2024-07-26T13:38:52.730574+0000","last_became_active":"2024-07-26T13:36:56.434611+0000","last_became_peered":"2024-07-26T13:36:56.434611+0000","last_unstale":"2024-07-26T13:38:52.730574+0000","last_undegraded":"2024-07-26T13:38:52.730574+0000","last_fullsized":"2024-07-26T13:38:52.730574+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:18:27.272077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62584,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":201,"num_read_kb":161,"num_write":33,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.382186+0000","last_change":"2024-07-26T13:36:59.465831+0000","last_active":"2024-07-26T13:38:46.382186+0000","last_peered":"2024-07-26T13:38:46.382186+0000","last_clean":"2024-07-26T13:38:46.382186+0000","last_became_active":"2024-07-26T13:36:59.465404+0000","last_became_peered":"2024-07-26T13:36:59.465404+0000","last_unstale":"2024-07-26T13:38:46.382186+0000","last_undegraded":"2024-07-26T13:38:46.382186+0000","last_fullsized":"2024-07-26T13:38:46.382186+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:42:53.795926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'6","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.552433+0000","last_change":"2024-07-26T13:36:56.435170+0000","last_active":"2024-07-26T13:37:17.552433+0000","last_peered":"2024-07-26T13:37:17.552433+0000","last_clean":"2024-07-26T13:37:17.552433+0000","last_became_active":"2024-07-26T13:36:56.434715+0000","last_became_peered":"2024-07-26T13:36:56.434715+0000","last_unstale":"2024-07-26T13:37:17.552433+0000","last_undegraded":"2024-07-26T13:37:17.552433+0000","last_fullsized":"2024-07-26T13:37:17.552433+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:05:09.712510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.354602+0000","last_change":"2024-07-26T13:36:59.461926+0000","last_active":"2024-07-26T13:38:46.354602+0000","last_peered":"2024-07-26T13:38:46.354602+0000","last_clean":"2024-07-26T13:38:46.354602+0000","last_became_active":"2024-07-26T13:36:59.461512+0000","last_became_peered":"2024-07-26T13:36:59.461512+0000","last_unstale":"2024-07-26T13:38:46.354602+0000","last_undegraded":"2024-07-26T13:38:46.354602+0000","last_fullsized":"2024-07-26T13:38:46.354602+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:34:24.405663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.546843+0000","last_change":"2024-07-26T13:36:56.430062+0000","last_active":"2024-07-26T13:37:17.546843+0000","last_peered":"2024-07-26T13:37:17.546843+0000","last_clean":"2024-07-26T13:37:17.546843+0000","last_became_active":"2024-07-26T13:36:56.429654+0000","last_became_peered":"2024-07-26T13:36:56.429654+0000","last_unstale":"2024-07-26T13:37:17.546843+0000","last_undegraded":"2024-07-26T13:37:17.546843+0000","last_fullsized":"2024-07-26T13:37:17.546843+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:21:09.514359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.379830+0000","last_change":"2024-07-26T13:36:59.474459+0000","last_active":"2024-07-26T13:38:46.379830+0000","last_peered":"2024-07-26T13:38:46.379830+0000","last_clean":"2024-07-26T13:38:46.379830+0000","last_became_active":"2024-07-26T13:36:59.474061+0000","last_became_peered":"2024-07-26T13:36:59.474061+0000","last_unstale":"2024-07-26T13:38:46.379830+0000","last_undegraded":"2024-07-26T13:38:46.379830+0000","last_fullsized":"2024-07-26T13:38:46.379830+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:37:21.691491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'5","reported_seq":132,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:51.710177+0000","last_change":"2024-07-26T13:36:56.432915+0000","last_active":"2024-07-26T13:38:51.710177+0000","last_peered":"2024-07-26T13:38:51.710177+0000","last_clean":"2024-07-26T13:38:51.710177+0000","last_became_active":"2024-07-26T13:36:56.432499+0000","last_became_peered":"2024-07-26T13:36:56.432499+0000","last_unstale":"2024-07-26T13:38:51.710177+0000","last_undegraded":"2024-07-26T13:38:51.710177+0000","last_fullsized":"2024-07-26T13:38:51.710177+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:02:14.306950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383440+0000","last_change":"2024-07-26T13:36:59.468945+0000","last_active":"2024-07-26T13:38:46.383440+0000","last_peered":"2024-07-26T13:38:46.383440+0000","last_clean":"2024-07-26T13:38:46.383440+0000","last_became_active":"2024-07-26T13:36:59.467970+0000","last_became_peered":"2024-07-26T13:36:59.467970+0000","last_unstale":"2024-07-26T13:38:46.383440+0000","last_undegraded":"2024-07-26T13:38:46.383440+0000","last_fullsized":"2024-07-26T13:38:46.383440+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T13:59:19.514945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'15","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551317+0000","last_change":"2024-07-26T13:36:56.487014+0000","last_active":"2024-07-26T13:37:17.551317+0000","last_peered":"2024-07-26T13:37:17.551317+0000","last_clean":"2024-07-26T13:37:17.551317+0000","last_became_active":"2024-07-26T13:36:56.486470+0000","last_became_peered":"2024-07-26T13:36:56.486470+0000","last_unstale":"2024-07-26T13:37:17.551317+0000","last_undegraded":"2024-07-26T13:37:17.551317+0000","last_fullsized":"2024-07-26T13:37:17.551317+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:07:02.205644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.366132+0000","last_change":"2024-07-26T13:36:59.462465+0000","last_active":"2024-07-26T13:38:46.366132+0000","last_peered":"2024-07-26T13:38:46.366132+0000","last_clean":"2024-07-26T13:38:46.366132+0000","last_became_active":"2024-07-26T13:36:59.462024+0000","last_became_peered":"2024-07-26T13:36:59.462024+0000","last_unstale":"2024-07-26T13:38:46.366132+0000","last_undegraded":"2024-07-26T13:38:46.366132+0000","last_fullsized":"2024-07-26T13:38:46.366132+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:34:14.317762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551923+0000","last_change":"2024-07-26T13:36:56.435007+0000","last_active":"2024-07-26T13:37:17.551923+0000","last_peered":"2024-07-26T13:37:17.551923+0000","last_clean":"2024-07-26T13:37:17.551923+0000","last_became_active":"2024-07-26T13:36:56.434444+0000","last_became_peered":"2024-07-26T13:36:56.434444+0000","last_unstale":"2024-07-26T13:37:17.551923+0000","last_undegraded":"2024-07-26T13:37:17.551923+0000","last_fullsized":"2024-07-26T13:37:17.551923+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:27:51.041349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.384358+0000","last_change":"2024-07-26T13:36:59.464164+0000","last_active":"2024-07-26T13:38:46.384358+0000","last_peered":"2024-07-26T13:38:46.384358+0000","last_clean":"2024-07-26T13:38:46.384358+0000","last_became_active":"2024-07-26T13:36:59.463782+0000","last_became_peered":"2024-07-26T13:36:59.463782+0000","last_unstale":"2024-07-26T13:38:46.384358+0000","last_undegraded":"2024-07-26T13:38:46.384358+0000","last_fullsized":"2024-07-26T13:38:46.384358+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:48:06.481731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.565453+0000","last_change":"2024-07-26T13:36:56.429653+0000","last_active":"2024-07-26T13:37:17.565453+0000","last_peered":"2024-07-26T13:37:17.565453+0000","last_clean":"2024-07-26T13:37:17.565453+0000","last_became_active":"2024-07-26T13:36:56.429060+0000","last_became_peered":"2024-07-26T13:36:56.429060+0000","last_unstale":"2024-07-26T13:37:17.565453+0000","last_undegraded":"2024-07-26T13:37:17.565453+0000","last_fullsized":"2024-07-26T13:37:17.565453+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:41:48.261709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383390+0000","last_change":"2024-07-26T13:36:59.471560+0000","last_active":"2024-07-26T13:38:46.383390+0000","last_peered":"2024-07-26T13:38:46.383390+0000","last_clean":"2024-07-26T13:38:46.383390+0000","last_became_active":"2024-07-26T13:36:59.471151+0000","last_became_peered":"2024-07-26T13:36:59.471151+0000","last_unstale":"2024-07-26T13:38:46.383390+0000","last_undegraded":"2024-07-26T13:38:46.383390+0000","last_fullsized":"2024-07-26T13:38:46.383390+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:49:20.513308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.548108+0000","last_change":"2024-07-26T13:36:56.430010+0000","last_active":"2024-07-26T13:37:17.548108+0000","last_peered":"2024-07-26T13:37:17.548108+0000","last_clean":"2024-07-26T13:37:17.548108+0000","last_became_active":"2024-07-26T13:36:56.429120+0000","last_became_peered":"2024-07-26T13:36:56.429120+0000","last_unstale":"2024-07-26T13:37:17.548108+0000","last_undegraded":"2024-07-26T13:37:17.548108+0000","last_fullsized":"2024-07-26T13:37:17.548108+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:23:43.321975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383377+0000","last_change":"2024-07-26T13:36:59.467788+0000","last_active":"2024-07-26T13:38:46.383377+0000","last_peered":"2024-07-26T13:38:46.383377+0000","last_clean":"2024-07-26T13:38:46.383377+0000","last_became_active":"2024-07-26T13:36:59.467399+0000","last_became_peered":"2024-07-26T13:36:59.467399+0000","last_unstale":"2024-07-26T13:38:46.383377+0000","last_undegraded":"2024-07-26T13:38:46.383377+0000","last_fullsized":"2024-07-26T13:38:46.383377+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:48:51.442999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'3","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.553251+0000","last_change":"2024-07-26T13:36:56.588633+0000","last_active":"2024-07-26T13:37:17.553251+0000","last_peered":"2024-07-26T13:37:17.553251+0000","last_clean":"2024-07-26T13:37:17.553251+0000","last_became_active":"2024-07-26T13:36:56.588113+0000","last_became_peered":"2024-07-26T13:36:56.588113+0000","last_unstale":"2024-07-26T13:37:17.553251+0000","last_undegraded":"2024-07-26T13:37:17.553251+0000","last_fullsized":"2024-07-26T13:37:17.553251+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:03:55.075052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387396+0000","last_change":"2024-07-26T13:36:59.472854+0000","last_active":"2024-07-26T13:38:46.387396+0000","last_peered":"2024-07-26T13:38:46.387396+0000","last_clean":"2024-07-26T13:38:46.387396+0000","last_became_active":"2024-07-26T13:36:59.472466+0000","last_became_peered":"2024-07-26T13:36:59.472466+0000","last_unstale":"2024-07-26T13:38:46.387396+0000","last_undegraded":"2024-07-26T13:38:46.387396+0000","last_fullsized":"2024-07-26T13:38:46.387396+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:50:57.454149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":103,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550829+0000","last_change":"2024-07-26T13:35:34.839121+0000","last_active":"2024-07-26T13:37:17.550829+0000","last_peered":"2024-07-26T13:37:17.550829+0000","last_clean":"2024-07-26T13:37:17.550829+0000","last_became_active":"2024-07-26T13:35:34.833465+0000","last_became_peered":"2024-07-26T13:35:34.833465+0000","last_unstale":"2024-07-26T13:37:17.550829+0000","last_undegraded":"2024-07-26T13:37:17.550829+0000","last_fullsized":"2024-07-26T13:37:17.550829+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:32:23.285428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:32:23.285428+0000","last_clean_scrub_stamp":"2024-07-26T13:32:23.285428+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:29:31.650487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'3","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.561400+0000","last_change":"2024-07-26T13:36:56.428648+0000","last_active":"2024-07-26T13:37:17.561400+0000","last_peered":"2024-07-26T13:37:17.561400+0000","last_clean":"2024-07-26T13:37:17.561400+0000","last_became_active":"2024-07-26T13:36:56.427665+0000","last_became_peered":"2024-07-26T13:36:56.427665+0000","last_unstale":"2024-07-26T13:37:17.561400+0000","last_undegraded":"2024-07-26T13:37:17.561400+0000","last_fullsized":"2024-07-26T13:37:17.561400+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:41:39.371687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.376509+0000","last_change":"2024-07-26T13:36:59.460625+0000","last_active":"2024-07-26T13:38:46.376509+0000","last_peered":"2024-07-26T13:38:46.376509+0000","last_clean":"2024-07-26T13:38:46.376509+0000","last_became_active":"2024-07-26T13:36:59.458658+0000","last_became_peered":"2024-07-26T13:36:59.458658+0000","last_unstale":"2024-07-26T13:38:46.376509+0000","last_undegraded":"2024-07-26T13:38:46.376509+0000","last_fullsized":"2024-07-26T13:38:46.376509+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:46:42.812402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550730+0000","last_change":"2024-07-26T13:36:56.432617+0000","last_active":"2024-07-26T13:37:17.550730+0000","last_peered":"2024-07-26T13:37:17.550730+0000","last_clean":"2024-07-26T13:37:17.550730+0000","last_became_active":"2024-07-26T13:36:56.431519+0000","last_became_peered":"2024-07-26T13:36:56.431519+0000","last_unstale":"2024-07-26T13:37:17.550730+0000","last_undegraded":"2024-07-26T13:37:17.550730+0000","last_fullsized":"2024-07-26T13:37:17.550730+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:32:21.120076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.364540+0000","last_change":"2024-07-26T13:36:59.464084+0000","last_active":"2024-07-26T13:38:46.364540+0000","last_peered":"2024-07-26T13:38:46.364540+0000","last_clean":"2024-07-26T13:38:46.364540+0000","last_became_active":"2024-07-26T13:36:59.463755+0000","last_became_peered":"2024-07-26T13:36:59.463755+0000","last_unstale":"2024-07-26T13:38:46.364540+0000","last_undegraded":"2024-07-26T13:38:46.364540+0000","last_fullsized":"2024-07-26T13:38:46.364540+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:53:20.034188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":133,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:55.715881+0000","last_change":"2024-07-26T13:36:56.486071+0000","last_active":"2024-07-26T13:38:55.715881+0000","last_peered":"2024-07-26T13:38:55.715881+0000","last_clean":"2024-07-26T13:38:55.715881+0000","last_became_active":"2024-07-26T13:36:56.485672+0000","last_became_peered":"2024-07-26T13:36:56.485672+0000","last_unstale":"2024-07-26T13:38:55.715881+0000","last_undegraded":"2024-07-26T13:38:55.715881+0000","last_fullsized":"2024-07-26T13:38:55.715881+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T21:29:45.666651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.370979+0000","last_change":"2024-07-26T13:37:17.566555+0000","last_active":"2024-07-26T13:38:46.370979+0000","last_peered":"2024-07-26T13:38:46.370979+0000","last_clean":"2024-07-26T13:38:46.370979+0000","last_became_active":"2024-07-26T13:37:17.566040+0000","last_became_peered":"2024-07-26T13:37:17.566040+0000","last_unstale":"2024-07-26T13:38:46.370979+0000","last_undegraded":"2024-07-26T13:38:46.370979+0000","last_fullsized":"2024-07-26T13:38:46.370979+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:48:33.078890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'1","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550922+0000","last_change":"2024-07-26T13:36:56.431811+0000","last_active":"2024-07-26T13:37:17.550922+0000","last_peered":"2024-07-26T13:37:17.550922+0000","last_clean":"2024-07-26T13:37:17.550922+0000","last_became_active":"2024-07-26T13:36:56.431223+0000","last_became_peered":"2024-07-26T13:36:56.431223+0000","last_unstale":"2024-07-26T13:37:17.550922+0000","last_undegraded":"2024-07-26T13:37:17.550922+0000","last_fullsized":"2024-07-26T13:37:17.550922+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:08:00.334515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.363722+0000","last_change":"2024-07-26T13:36:59.463972+0000","last_active":"2024-07-26T13:38:46.363722+0000","last_peered":"2024-07-26T13:38:46.363722+0000","last_clean":"2024-07-26T13:38:46.363722+0000","last_became_active":"2024-07-26T13:36:59.463496+0000","last_became_peered":"2024-07-26T13:36:59.463496+0000","last_unstale":"2024-07-26T13:38:46.363722+0000","last_undegraded":"2024-07-26T13:38:46.363722+0000","last_fullsized":"2024-07-26T13:38:46.363722+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:02:06.215372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.545709+0000","last_change":"2024-07-26T13:36:56.486681+0000","last_active":"2024-07-26T13:37:17.545709+0000","last_peered":"2024-07-26T13:37:17.545709+0000","last_clean":"2024-07-26T13:37:17.545709+0000","last_became_active":"2024-07-26T13:36:56.486328+0000","last_became_peered":"2024-07-26T13:36:56.486328+0000","last_unstale":"2024-07-26T13:37:17.545709+0000","last_undegraded":"2024-07-26T13:37:17.545709+0000","last_fullsized":"2024-07-26T13:37:17.545709+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:10:36.557707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.355226+0000","last_change":"2024-07-26T13:36:59.451415+0000","last_active":"2024-07-26T13:38:46.355226+0000","last_peered":"2024-07-26T13:38:46.355226+0000","last_clean":"2024-07-26T13:38:46.355226+0000","last_became_active":"2024-07-26T13:36:59.450992+0000","last_became_peered":"2024-07-26T13:36:59.450992+0000","last_unstale":"2024-07-26T13:38:46.355226+0000","last_undegraded":"2024-07-26T13:38:46.355226+0000","last_fullsized":"2024-07-26T13:38:46.355226+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:29:11.303916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.555734+0000","last_change":"2024-07-26T13:37:17.555643+0000","last_active":"2024-07-26T13:37:17.555734+0000","last_peered":"2024-07-26T13:37:17.555734+0000","last_clean":"2024-07-26T13:37:17.555734+0000","last_became_active":"2024-07-26T13:37:17.549756+0000","last_became_peered":"2024-07-26T13:37:17.549756+0000","last_unstale":"2024-07-26T13:37:17.555734+0000","last_undegraded":"2024-07-26T13:37:17.555734+0000","last_fullsized":"2024-07-26T13:37:17.555734+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:20:16.911286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.368420+0000","last_change":"2024-07-26T13:36:59.465538+0000","last_active":"2024-07-26T13:38:46.368420+0000","last_peered":"2024-07-26T13:38:46.368420+0000","last_clean":"2024-07-26T13:38:46.368420+0000","last_became_active":"2024-07-26T13:36:59.464886+0000","last_became_peered":"2024-07-26T13:36:59.464886+0000","last_unstale":"2024-07-26T13:38:46.368420+0000","last_undegraded":"2024-07-26T13:38:46.368420+0000","last_fullsized":"2024-07-26T13:38:46.368420+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:19:12.273116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'12","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:40.480485+0000","last_change":"2024-07-26T13:36:56.428355+0000","last_active":"2024-07-26T13:37:40.480485+0000","last_peered":"2024-07-26T13:37:40.480485+0000","last_clean":"2024-07-26T13:37:40.480485+0000","last_became_active":"2024-07-26T13:36:56.427213+0000","last_became_peered":"2024-07-26T13:36:56.427213+0000","last_unstale":"2024-07-26T13:37:40.480485+0000","last_undegraded":"2024-07-26T13:37:40.480485+0000","last_fullsized":"2024-07-26T13:37:40.480485+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:48:09.730067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.378381+0000","last_change":"2024-07-26T13:36:59.459378+0000","last_active":"2024-07-26T13:38:46.378381+0000","last_peered":"2024-07-26T13:38:46.378381+0000","last_clean":"2024-07-26T13:38:46.378381+0000","last_became_active":"2024-07-26T13:36:59.458799+0000","last_became_peered":"2024-07-26T13:36:59.458799+0000","last_unstale":"2024-07-26T13:38:46.378381+0000","last_undegraded":"2024-07-26T13:38:46.378381+0000","last_fullsized":"2024-07-26T13:38:46.378381+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:45:19.587800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.561367+0000","last_change":"2024-07-26T13:36:56.588819+0000","last_active":"2024-07-26T13:37:17.561367+0000","last_peered":"2024-07-26T13:37:17.561367+0000","last_clean":"2024-07-26T13:37:17.561367+0000","last_became_active":"2024-07-26T13:36:56.588430+0000","last_became_peered":"2024-07-26T13:36:56.588430+0000","last_unstale":"2024-07-26T13:37:17.561367+0000","last_undegraded":"2024-07-26T13:37:17.561367+0000","last_fullsized":"2024-07-26T13:37:17.561367+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:07:21.387618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.380368+0000","last_change":"2024-07-26T13:37:17.562040+0000","last_active":"2024-07-26T13:38:46.380368+0000","last_peered":"2024-07-26T13:38:46.380368+0000","last_clean":"2024-07-26T13:38:46.380368+0000","last_became_active":"2024-07-26T13:37:17.561542+0000","last_became_peered":"2024-07-26T13:37:17.561542+0000","last_unstale":"2024-07-26T13:38:46.380368+0000","last_undegraded":"2024-07-26T13:38:46.380368+0000","last_fullsized":"2024-07-26T13:38:46.380368+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:21:08.161954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.544058+0000","last_change":"2024-07-26T13:36:56.489106+0000","last_active":"2024-07-26T13:37:17.544058+0000","last_peered":"2024-07-26T13:37:17.544058+0000","last_clean":"2024-07-26T13:37:17.544058+0000","last_became_active":"2024-07-26T13:36:56.488706+0000","last_became_peered":"2024-07-26T13:36:56.488706+0000","last_unstale":"2024-07-26T13:37:17.544058+0000","last_undegraded":"2024-07-26T13:37:17.544058+0000","last_fullsized":"2024-07-26T13:37:17.544058+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:00:59.204434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.336503+0000","last_change":"2024-07-26T13:36:59.470412+0000","last_active":"2024-07-26T13:38:46.336503+0000","last_peered":"2024-07-26T13:38:46.336503+0000","last_clean":"2024-07-26T13:38:46.336503+0000","last_became_active":"2024-07-26T13:36:59.470010+0000","last_became_peered":"2024-07-26T13:36:59.470010+0000","last_unstale":"2024-07-26T13:38:46.336503+0000","last_undegraded":"2024-07-26T13:38:46.336503+0000","last_fullsized":"2024-07-26T13:38:46.336503+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:50:57.987600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.543473+0000","last_change":"2024-07-26T13:36:56.428671+0000","last_active":"2024-07-26T13:37:17.543473+0000","last_peered":"2024-07-26T13:37:17.543473+0000","last_clean":"2024-07-26T13:37:17.543473+0000","last_became_active":"2024-07-26T13:36:56.428301+0000","last_became_peered":"2024-07-26T13:36:56.428301+0000","last_unstale":"2024-07-26T13:37:17.543473+0000","last_undegraded":"2024-07-26T13:37:17.543473+0000","last_fullsized":"2024-07-26T13:37:17.543473+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:59:19.593664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383551+0000","last_change":"2024-07-26T13:36:59.458627+0000","last_active":"2024-07-26T13:38:46.383551+0000","last_peered":"2024-07-26T13:38:46.383551+0000","last_clean":"2024-07-26T13:38:46.383551+0000","last_became_active":"2024-07-26T13:36:59.458199+0000","last_became_peered":"2024-07-26T13:36:59.458199+0000","last_unstale":"2024-07-26T13:38:46.383551+0000","last_undegraded":"2024-07-26T13:38:46.383551+0000","last_fullsized":"2024-07-26T13:38:46.383551+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:54:41.833223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'1","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551478+0000","last_change":"2024-07-26T13:36:56.431548+0000","last_active":"2024-07-26T13:37:17.551478+0000","last_peered":"2024-07-26T13:37:17.551478+0000","last_clean":"2024-07-26T13:37:17.551478+0000","last_became_active":"2024-07-26T13:36:56.430989+0000","last_became_peered":"2024-07-26T13:36:56.430989+0000","last_unstale":"2024-07-26T13:37:17.551478+0000","last_undegraded":"2024-07-26T13:37:17.551478+0000","last_fullsized":"2024-07-26T13:37:17.551478+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:46:51.655324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.371547+0000","last_change":"2024-07-26T13:36:59.458290+0000","last_active":"2024-07-26T13:38:46.371547+0000","last_peered":"2024-07-26T13:38:46.371547+0000","last_clean":"2024-07-26T13:38:46.371547+0000","last_became_active":"2024-07-26T13:36:59.457942+0000","last_became_peered":"2024-07-26T13:36:59.457942+0000","last_unstale":"2024-07-26T13:38:46.371547+0000","last_undegraded":"2024-07-26T13:38:46.371547+0000","last_fullsized":"2024-07-26T13:38:46.371547+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:51:58.797441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.567687+0000","last_change":"2024-07-26T13:37:17.567624+0000","last_active":"2024-07-26T13:37:17.567687+0000","last_peered":"2024-07-26T13:37:17.567687+0000","last_clean":"2024-07-26T13:37:17.567687+0000","last_became_active":"2024-07-26T13:37:17.562455+0000","last_became_peered":"2024-07-26T13:37:17.562455+0000","last_unstale":"2024-07-26T13:37:17.567687+0000","last_undegraded":"2024-07-26T13:37:17.567687+0000","last_fullsized":"2024-07-26T13:37:17.567687+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:50:36.418750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387702+0000","last_change":"2024-07-26T13:36:59.474034+0000","last_active":"2024-07-26T13:38:46.387702+0000","last_peered":"2024-07-26T13:38:46.387702+0000","last_clean":"2024-07-26T13:38:46.387702+0000","last_became_active":"2024-07-26T13:36:59.472358+0000","last_became_peered":"2024-07-26T13:36:59.472358+0000","last_unstale":"2024-07-26T13:38:46.387702+0000","last_undegraded":"2024-07-26T13:38:46.387702+0000","last_fullsized":"2024-07-26T13:38:46.387702+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:36:44.565413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.367956+0000","last_change":"2024-07-26T13:36:59.465546+0000","last_active":"2024-07-26T13:38:46.367956+0000","last_peered":"2024-07-26T13:38:46.367956+0000","last_clean":"2024-07-26T13:38:46.367956+0000","last_became_active":"2024-07-26T13:36:59.465137+0000","last_became_peered":"2024-07-26T13:36:59.465137+0000","last_unstale":"2024-07-26T13:38:46.367956+0000","last_undegraded":"2024-07-26T13:38:46.367956+0000","last_fullsized":"2024-07-26T13:38:46.367956+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:35:13.591354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.562100+0000","last_change":"2024-07-26T13:37:17.562012+0000","last_active":"2024-07-26T13:37:17.562100+0000","last_peered":"2024-07-26T13:37:17.562100+0000","last_clean":"2024-07-26T13:37:17.562100+0000","last_became_active":"2024-07-26T13:37:17.550218+0000","last_became_peered":"2024-07-26T13:37:17.550218+0000","last_unstale":"2024-07-26T13:37:17.562100+0000","last_undegraded":"2024-07-26T13:37:17.562100+0000","last_fullsized":"2024-07-26T13:37:17.562100+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:46:16.559533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.374985+0000","last_change":"2024-07-26T13:36:59.456896+0000","last_active":"2024-07-26T13:38:46.374985+0000","last_peered":"2024-07-26T13:38:46.374985+0000","last_clean":"2024-07-26T13:38:46.374985+0000","last_became_active":"2024-07-26T13:36:59.456530+0000","last_became_peered":"2024-07-26T13:36:59.456530+0000","last_unstale":"2024-07-26T13:38:46.374985+0000","last_undegraded":"2024-07-26T13:38:46.374985+0000","last_fullsized":"2024-07-26T13:38:46.374985+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:59:56.512501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'4","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.553522+0000","last_change":"2024-07-26T13:36:56.433119+0000","last_active":"2024-07-26T13:37:17.553522+0000","last_peered":"2024-07-26T13:37:17.553522+0000","last_clean":"2024-07-26T13:37:17.553522+0000","last_became_active":"2024-07-26T13:36:56.432765+0000","last_became_peered":"2024-07-26T13:36:56.432765+0000","last_unstale":"2024-07-26T13:37:17.553522+0000","last_undegraded":"2024-07-26T13:37:17.553522+0000","last_fullsized":"2024-07-26T13:37:17.553522+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:31:29.197106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.381903+0000","last_change":"2024-07-26T13:36:59.462125+0000","last_active":"2024-07-26T13:38:46.381903+0000","last_peered":"2024-07-26T13:38:46.381903+0000","last_clean":"2024-07-26T13:38:46.381903+0000","last_became_active":"2024-07-26T13:36:59.461796+0000","last_became_peered":"2024-07-26T13:36:59.461796+0000","last_unstale":"2024-07-26T13:38:46.381903+0000","last_undegraded":"2024-07-26T13:38:46.381903+0000","last_fullsized":"2024-07-26T13:38:46.381903+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:44:10.329046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.543605+0000","last_change":"2024-07-26T13:36:56.487251+0000","last_active":"2024-07-26T13:37:17.543605+0000","last_peered":"2024-07-26T13:37:17.543605+0000","last_clean":"2024-07-26T13:37:17.543605+0000","last_became_active":"2024-07-26T13:36:56.486918+0000","last_became_peered":"2024-07-26T13:36:56.486918+0000","last_unstale":"2024-07-26T13:37:17.543605+0000","last_undegraded":"2024-07-26T13:37:17.543605+0000","last_fullsized":"2024-07-26T13:37:17.543605+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:41:31.248895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.369380+0000","last_change":"2024-07-26T13:36:59.470810+0000","last_active":"2024-07-26T13:38:46.369380+0000","last_peered":"2024-07-26T13:38:46.369380+0000","last_clean":"2024-07-26T13:38:46.369380+0000","last_became_active":"2024-07-26T13:36:59.467371+0000","last_became_peered":"2024-07-26T13:36:59.467371+0000","last_unstale":"2024-07-26T13:38:46.369380+0000","last_undegraded":"2024-07-26T13:38:46.369380+0000","last_fullsized":"2024-07-26T13:38:46.369380+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:23:48.572933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'9","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.545993+0000","last_change":"2024-07-26T13:36:56.433598+0000","last_active":"2024-07-26T13:37:17.545993+0000","last_peered":"2024-07-26T13:37:17.545993+0000","last_clean":"2024-07-26T13:37:17.545993+0000","last_became_active":"2024-07-26T13:36:56.433325+0000","last_became_peered":"2024-07-26T13:36:56.433325+0000","last_unstale":"2024-07-26T13:37:17.545993+0000","last_undegraded":"2024-07-26T13:37:17.545993+0000","last_fullsized":"2024-07-26T13:37:17.545993+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:17:33.651765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67258,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":617,"num_read_kb":380,"num_write":174,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":644,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":201774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247480,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678056,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":443451,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874284,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":1098026,"data_compressed":41637,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037809,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":897024,"data_stored":443665,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234037,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":443563,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430266,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":1036188,"data_compressed":41637,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593792,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":443054,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790021,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":1035340,"data_compressed":41637,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953545,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":888832,"data_stored":444023,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149773,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":443365,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378707,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":506159,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542231,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":506585,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705756,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":444325,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34195,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34195,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-26T13:38:59.930 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph pg dump --format=json 2024-07-26T13:39:00.185 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:39:00.439 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:00 smithi100 ceph-mon[32599]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-26T13:39:00.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:00 smithi138 ceph-mon[36954]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-26T13:39:00.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:00 smithi167 ceph-mon[37140]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-26T13:39:01.305 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:01 smithi100 ceph-mon[32599]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-07-26T13:39:01.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:01 smithi167 ceph-mon[37140]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-07-26T13:39:01.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:01 smithi138 ceph-mon[36954]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-07-26T13:39:01.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.799+0000 7f9911c5a640 1 -- 172.21.15.100:0/2873331181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f990c108ed0 msgr2=0x7f990c1092e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:01.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.799+0000 7f9911c5a640 1 --2- 172.21.15.100:0/2873331181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f990c108ed0 0x7f990c1092e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009900 tx=0x7f98f402a410 comp rx=0 tx=0).stop 2024-07-26T13:39:01.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.799+0000 7f9911c5a640 1 -- 172.21.15.100:0/2873331181 shutdown_connections 2024-07-26T13:39:01.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.799+0000 7f9911c5a640 1 --2- 172.21.15.100:0/2873331181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f990c109820 0x7f990c110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:01.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.799+0000 7f9911c5a640 1 --2- 172.21.15.100:0/2873331181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f990c108ed0 0x7f990c1092e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:01.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.799+0000 7f9911c5a640 1 --2- 172.21.15.100:0/2873331181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f990c103590 0x7f990c1039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:01.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.799+0000 7f9911c5a640 1 -- 172.21.15.100:0/2873331181 >> 172.21.15.100:0/2873331181 conn(0x7f990c07b280 msgr2=0x7f990c07b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:01.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.800+0000 7f9911c5a640 1 -- 172.21.15.100:0/2873331181 shutdown_connections 2024-07-26T13:39:01.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.800+0000 7f9911c5a640 1 -- 172.21.15.100:0/2873331181 wait complete. 2024-07-26T13:39:01.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.800+0000 7f9911c5a640 1 Processor -- start 2024-07-26T13:39:01.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.800+0000 7f9911c5a640 1 -- start start 2024-07-26T13:39:01.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.801+0000 7f9911c5a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f990c103590 0x7f990c199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:01.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.801+0000 7f9911c5a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f990c108ed0 0x7f990c19a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:01.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.801+0000 7f9911c5a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f990c109820 0x7f990c19f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:01.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.801+0000 7f9911c5a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f990c1134b0 con 0x7f990c109820 2024-07-26T13:39:01.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.801+0000 7f9911c5a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f990c113330 con 0x7f990c108ed0 2024-07-26T13:39:01.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.801+0000 7f9911c5a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f990c113630 con 0x7f990c103590 2024-07-26T13:39:01.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.801+0000 7f990bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f990c109820 0x7f990c19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:01.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.801+0000 7f990bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f990c109820 0x7f990c19f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46190/0 (socket says 172.21.15.100:46190) 2024-07-26T13:39:01.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.801+0000 7f990bfff640 1 -- 172.21.15.100:0/2751634571 learned_addr learned my addr 172.21.15.100:0/2751634571 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:01.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.801+0000 7f990b7fe640 1 --2- 172.21.15.100:0/2751634571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f990c103590 0x7f990c199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:01.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.801+0000 7f990affd640 1 --2- 172.21.15.100:0/2751634571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f990c108ed0 0x7f990c19a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:01.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.801+0000 7f990bfff640 1 -- 172.21.15.100:0/2751634571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f990c103590 msgr2=0x7f990c199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:01.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.801+0000 7f990bfff640 1 --2- 172.21.15.100:0/2751634571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f990c103590 0x7f990c199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:01.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.801+0000 7f990bfff640 1 -- 172.21.15.100:0/2751634571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f990c108ed0 msgr2=0x7f990c19a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:01.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.801+0000 7f990bfff640 1 --2- 172.21.15.100:0/2751634571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f990c108ed0 0x7f990c19a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:01.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.801+0000 7f990bfff640 1 -- 172.21.15.100:0/2751634571 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f990c19ff50 con 0x7f990c109820 2024-07-26T13:39:01.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.801+0000 7f990b7fe640 1 --2- 172.21.15.100:0/2751634571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f990c103590 0x7f990c199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:39:01.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.801+0000 7f990bfff640 1 --2- 172.21.15.100:0/2751634571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f990c109820 0x7f990c19f820 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f990400c970 tx=0x7f990400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:01.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.801+0000 7f9908ff9640 1 -- 172.21.15.100:0/2751634571 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9904007bb0 con 0x7f990c109820 2024-07-26T13:39:01.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.802+0000 7f9908ff9640 1 -- 172.21.15.100:0/2751634571 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9904007d50 con 0x7f990c109820 2024-07-26T13:39:01.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.802+0000 7f9908ff9640 1 -- 172.21.15.100:0/2751634571 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9904004e60 con 0x7f990c109820 2024-07-26T13:39:01.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.802+0000 7f9911c5a640 1 -- 172.21.15.100:0/2751634571 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f990c1a0240 con 0x7f990c109820 2024-07-26T13:39:01.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.802+0000 7f9911c5a640 1 -- 172.21.15.100:0/2751634571 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f990c1a69a0 con 0x7f990c109820 2024-07-26T13:39:01.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.803+0000 7f9911c5a640 1 -- 172.21.15.100:0/2751634571 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f990c1039a0 con 0x7f990c109820 2024-07-26T13:39:01.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.805+0000 7f9908ff9640 1 -- 172.21.15.100:0/2751634571 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f9904005000 con 0x7f990c109820 2024-07-26T13:39:01.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.806+0000 7f9908ff9640 1 --2- 172.21.15.100:0/2751634571 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f98f80b3c20 0x7f98f80b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:01.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.806+0000 7f990b7fe640 1 --2- 172.21.15.100:0/2751634571 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f98f80b3c20 0x7f98f80b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:01.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.806+0000 7f9908ff9640 1 -- 172.21.15.100:0/2751634571 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f99040e3cd0 con 0x7f990c109820 2024-07-26T13:39:01.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.806+0000 7f990b7fe640 1 --2- 172.21.15.100:0/2751634571 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f98f80b3c20 0x7f98f80b6130 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f98fc005fd0 tx=0x7f98fc009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:01.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.809+0000 7f9908ff9640 1 -- 172.21.15.100:0/2751634571 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f9904014030 con 0x7f990c109820 2024-07-26T13:39:01.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.978+0000 7f9911c5a640 1 -- 172.21.15.100:0/2751634571 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f990c063230 con 0x7f98f80b3c20 2024-07-26T13:39:01.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.983+0000 7f9908ff9640 1 -- 172.21.15.100:0/2751634571 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259058 (secure 0 0 0) 0x7f990c063230 con 0x7f98f80b3c20 2024-07-26T13:39:01.986 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:39:01.987 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-07-26T13:39:01.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.985+0000 7f9911c5a640 1 -- 172.21.15.100:0/2751634571 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f98f80b3c20 msgr2=0x7f98f80b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:01.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.985+0000 7f9911c5a640 1 --2- 172.21.15.100:0/2751634571 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f98f80b3c20 0x7f98f80b6130 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f98fc005fd0 tx=0x7f98fc009500 comp rx=0 tx=0).stop 2024-07-26T13:39:01.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.985+0000 7f9911c5a640 1 -- 172.21.15.100:0/2751634571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f990c109820 msgr2=0x7f990c19f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:01.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.985+0000 7f9911c5a640 1 --2- 172.21.15.100:0/2751634571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f990c109820 0x7f990c19f820 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f990400c970 tx=0x7f990400ce30 comp rx=0 tx=0).stop 2024-07-26T13:39:01.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.985+0000 7f9911c5a640 1 -- 172.21.15.100:0/2751634571 shutdown_connections 2024-07-26T13:39:01.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.985+0000 7f9911c5a640 1 --2- 172.21.15.100:0/2751634571 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f98f80b3c20 0x7f98f80b6130 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:01.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.985+0000 7f9911c5a640 1 --2- 172.21.15.100:0/2751634571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f990c109820 0x7f990c19f820 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:01.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.985+0000 7f9911c5a640 1 --2- 172.21.15.100:0/2751634571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f990c108ed0 0x7f990c19a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:01.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.985+0000 7f9911c5a640 1 --2- 172.21.15.100:0/2751634571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f990c103590 0x7f990c199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:01.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.986+0000 7f9911c5a640 1 -- 172.21.15.100:0/2751634571 >> 172.21.15.100:0/2751634571 conn(0x7f990c07b280 msgr2=0x7f990c105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:01.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.986+0000 7f9911c5a640 1 -- 172.21.15.100:0/2751634571 shutdown_connections 2024-07-26T13:39:01.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:01.986+0000 7f9911c5a640 1 -- 172.21.15.100:0/2751634571 wait complete. 2024-07-26T13:39:02.679 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-07-26T13:39:00.391173+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657860,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":718,"num_read_kb":459,"num_write":299,"num_write_kb":2272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":2952484,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":742,"ondisk_log_size":742,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372492,"kb_used_data":10860,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569332,"statfs":{"total":1151940427776,"available":1151558995968,"internally_reserved":0,"allocated":11120640,"data_stored":7287744,"data_compressed":177282,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":42,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003691"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387423+0000","last_change":"2024-07-26T13:36:59.462289+0000","last_active":"2024-07-26T13:38:46.387423+0000","last_peered":"2024-07-26T13:38:46.387423+0000","last_clean":"2024-07-26T13:38:46.387423+0000","last_became_active":"2024-07-26T13:36:59.460311+0000","last_became_peered":"2024-07-26T13:36:59.460311+0000","last_unstale":"2024-07-26T13:38:46.387423+0000","last_undegraded":"2024-07-26T13:38:46.387423+0000","last_fullsized":"2024-07-26T13:38:46.387423+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:04:59.123716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383883+0000","last_change":"2024-07-26T13:36:59.463218+0000","last_active":"2024-07-26T13:38:46.383883+0000","last_peered":"2024-07-26T13:38:46.383883+0000","last_clean":"2024-07-26T13:38:46.383883+0000","last_became_active":"2024-07-26T13:36:59.462889+0000","last_became_peered":"2024-07-26T13:36:59.462889+0000","last_unstale":"2024-07-26T13:38:46.383883+0000","last_undegraded":"2024-07-26T13:38:46.383883+0000","last_fullsized":"2024-07-26T13:38:46.383883+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:30:46.788669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383357+0000","last_change":"2024-07-26T13:36:59.456113+0000","last_active":"2024-07-26T13:38:46.383357+0000","last_peered":"2024-07-26T13:38:46.383357+0000","last_clean":"2024-07-26T13:38:46.383357+0000","last_became_active":"2024-07-26T13:36:59.455622+0000","last_became_peered":"2024-07-26T13:36:59.455622+0000","last_unstale":"2024-07-26T13:38:46.383357+0000","last_undegraded":"2024-07-26T13:38:46.383357+0000","last_fullsized":"2024-07-26T13:38:46.383357+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:36:26.474799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.368653+0000","last_change":"2024-07-26T13:37:17.555947+0000","last_active":"2024-07-26T13:38:46.368653+0000","last_peered":"2024-07-26T13:38:46.368653+0000","last_clean":"2024-07-26T13:38:46.368653+0000","last_became_active":"2024-07-26T13:37:17.555607+0000","last_became_peered":"2024-07-26T13:37:17.555607+0000","last_unstale":"2024-07-26T13:38:46.368653+0000","last_undegraded":"2024-07-26T13:38:46.368653+0000","last_fullsized":"2024-07-26T13:38:46.368653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:19:59.603225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.372013+0000","last_change":"2024-07-26T13:36:59.474896+0000","last_active":"2024-07-26T13:38:46.372013+0000","last_peered":"2024-07-26T13:38:46.372013+0000","last_clean":"2024-07-26T13:38:46.372013+0000","last_became_active":"2024-07-26T13:36:59.474516+0000","last_became_peered":"2024-07-26T13:36:59.474516+0000","last_unstale":"2024-07-26T13:38:46.372013+0000","last_undegraded":"2024-07-26T13:38:46.372013+0000","last_fullsized":"2024-07-26T13:38:46.372013+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:38:18.994280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.357551+0000","last_change":"2024-07-26T13:37:17.556191+0000","last_active":"2024-07-26T13:38:46.357551+0000","last_peered":"2024-07-26T13:38:46.357551+0000","last_clean":"2024-07-26T13:38:46.357551+0000","last_became_active":"2024-07-26T13:37:17.555715+0000","last_became_peered":"2024-07-26T13:37:17.555715+0000","last_unstale":"2024-07-26T13:38:46.357551+0000","last_undegraded":"2024-07-26T13:38:46.357551+0000","last_fullsized":"2024-07-26T13:38:46.357551+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:03:18.352049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383606+0000","last_change":"2024-07-26T13:36:59.469515+0000","last_active":"2024-07-26T13:38:46.383606+0000","last_peered":"2024-07-26T13:38:46.383606+0000","last_clean":"2024-07-26T13:38:46.383606+0000","last_became_active":"2024-07-26T13:36:59.469106+0000","last_became_peered":"2024-07-26T13:36:59.469106+0000","last_unstale":"2024-07-26T13:38:46.383606+0000","last_undegraded":"2024-07-26T13:38:46.383606+0000","last_fullsized":"2024-07-26T13:38:46.383606+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:10:05.455329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.372753+0000","last_change":"2024-07-26T13:37:17.549365+0000","last_active":"2024-07-26T13:38:46.372753+0000","last_peered":"2024-07-26T13:38:46.372753+0000","last_clean":"2024-07-26T13:38:46.372753+0000","last_became_active":"2024-07-26T13:37:17.549036+0000","last_became_peered":"2024-07-26T13:37:17.549036+0000","last_unstale":"2024-07-26T13:38:46.372753+0000","last_undegraded":"2024-07-26T13:38:46.372753+0000","last_fullsized":"2024-07-26T13:38:46.372753+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:27:43.930981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387552+0000","last_change":"2024-07-26T13:36:59.459306+0000","last_active":"2024-07-26T13:38:46.387552+0000","last_peered":"2024-07-26T13:38:46.387552+0000","last_clean":"2024-07-26T13:38:46.387552+0000","last_became_active":"2024-07-26T13:36:59.458714+0000","last_became_peered":"2024-07-26T13:36:59.458714+0000","last_unstale":"2024-07-26T13:38:46.387552+0000","last_undegraded":"2024-07-26T13:38:46.387552+0000","last_fullsized":"2024-07-26T13:38:46.387552+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:34:18.512454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.376639+0000","last_change":"2024-07-26T13:36:59.466556+0000","last_active":"2024-07-26T13:38:46.376639+0000","last_peered":"2024-07-26T13:38:46.376639+0000","last_clean":"2024-07-26T13:38:46.376639+0000","last_became_active":"2024-07-26T13:36:59.466147+0000","last_became_peered":"2024-07-26T13:36:59.466147+0000","last_unstale":"2024-07-26T13:38:46.376639+0000","last_undegraded":"2024-07-26T13:38:46.376639+0000","last_fullsized":"2024-07-26T13:38:46.376639+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:43:11.359314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387429+0000","last_change":"2024-07-26T13:36:59.473685+0000","last_active":"2024-07-26T13:38:46.387429+0000","last_peered":"2024-07-26T13:38:46.387429+0000","last_clean":"2024-07-26T13:38:46.387429+0000","last_became_active":"2024-07-26T13:36:59.473314+0000","last_became_peered":"2024-07-26T13:36:59.473314+0000","last_unstale":"2024-07-26T13:38:46.387429+0000","last_undegraded":"2024-07-26T13:38:46.387429+0000","last_fullsized":"2024-07-26T13:38:46.387429+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:41:51.612348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.367874+0000","last_change":"2024-07-26T13:36:59.471586+0000","last_active":"2024-07-26T13:38:46.367874+0000","last_peered":"2024-07-26T13:38:46.367874+0000","last_clean":"2024-07-26T13:38:46.367874+0000","last_became_active":"2024-07-26T13:36:59.470995+0000","last_became_peered":"2024-07-26T13:36:59.470995+0000","last_unstale":"2024-07-26T13:38:46.367874+0000","last_undegraded":"2024-07-26T13:38:46.367874+0000","last_fullsized":"2024-07-26T13:38:46.367874+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:54:07.957937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.357393+0000","last_change":"2024-07-26T13:36:59.460018+0000","last_active":"2024-07-26T13:38:46.357393+0000","last_peered":"2024-07-26T13:38:46.357393+0000","last_clean":"2024-07-26T13:38:46.357393+0000","last_became_active":"2024-07-26T13:36:59.459671+0000","last_became_peered":"2024-07-26T13:36:59.459671+0000","last_unstale":"2024-07-26T13:38:46.357393+0000","last_undegraded":"2024-07-26T13:38:46.357393+0000","last_fullsized":"2024-07-26T13:38:46.357393+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:19:33.923360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.374398+0000","last_change":"2024-07-26T13:36:59.470271+0000","last_active":"2024-07-26T13:38:46.374398+0000","last_peered":"2024-07-26T13:38:46.374398+0000","last_clean":"2024-07-26T13:38:46.374398+0000","last_became_active":"2024-07-26T13:36:59.469941+0000","last_became_peered":"2024-07-26T13:36:59.469941+0000","last_unstale":"2024-07-26T13:38:46.374398+0000","last_undegraded":"2024-07-26T13:38:46.374398+0000","last_fullsized":"2024-07-26T13:38:46.374398+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:38:39.329366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383031+0000","last_change":"2024-07-26T13:36:59.460133+0000","last_active":"2024-07-26T13:38:46.383031+0000","last_peered":"2024-07-26T13:38:46.383031+0000","last_clean":"2024-07-26T13:38:46.383031+0000","last_became_active":"2024-07-26T13:36:59.459794+0000","last_became_peered":"2024-07-26T13:36:59.459794+0000","last_unstale":"2024-07-26T13:38:46.383031+0000","last_undegraded":"2024-07-26T13:38:46.383031+0000","last_fullsized":"2024-07-26T13:38:46.383031+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:15:36.024347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.385893+0000","last_change":"2024-07-26T13:36:59.462390+0000","last_active":"2024-07-26T13:38:46.385893+0000","last_peered":"2024-07-26T13:38:46.385893+0000","last_clean":"2024-07-26T13:38:46.385893+0000","last_became_active":"2024-07-26T13:36:59.461968+0000","last_became_peered":"2024-07-26T13:36:59.461968+0000","last_unstale":"2024-07-26T13:38:46.385893+0000","last_undegraded":"2024-07-26T13:38:46.385893+0000","last_fullsized":"2024-07-26T13:38:46.385893+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:49:47.142472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387521+0000","last_change":"2024-07-26T13:36:59.460206+0000","last_active":"2024-07-26T13:38:46.387521+0000","last_peered":"2024-07-26T13:38:46.387521+0000","last_clean":"2024-07-26T13:38:46.387521+0000","last_became_active":"2024-07-26T13:36:59.459811+0000","last_became_peered":"2024-07-26T13:36:59.459811+0000","last_unstale":"2024-07-26T13:38:46.387521+0000","last_undegraded":"2024-07-26T13:38:46.387521+0000","last_fullsized":"2024-07-26T13:38:46.387521+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:46:57.112314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387875+0000","last_change":"2024-07-26T13:36:59.460197+0000","last_active":"2024-07-26T13:38:46.387875+0000","last_peered":"2024-07-26T13:38:46.387875+0000","last_clean":"2024-07-26T13:38:46.387875+0000","last_became_active":"2024-07-26T13:36:59.459686+0000","last_became_peered":"2024-07-26T13:36:59.459686+0000","last_unstale":"2024-07-26T13:38:46.387875+0000","last_undegraded":"2024-07-26T13:38:46.387875+0000","last_fullsized":"2024-07-26T13:38:46.387875+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T13:39:22.407376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.373295+0000","last_change":"2024-07-26T13:36:59.462545+0000","last_active":"2024-07-26T13:38:46.373295+0000","last_peered":"2024-07-26T13:38:46.373295+0000","last_clean":"2024-07-26T13:38:46.373295+0000","last_became_active":"2024-07-26T13:36:59.462115+0000","last_became_peered":"2024-07-26T13:36:59.462115+0000","last_unstale":"2024-07-26T13:38:46.373295+0000","last_undegraded":"2024-07-26T13:38:46.373295+0000","last_fullsized":"2024-07-26T13:38:46.373295+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T21:44:13.715076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383157+0000","last_change":"2024-07-26T13:36:59.465317+0000","last_active":"2024-07-26T13:38:46.383157+0000","last_peered":"2024-07-26T13:38:46.383157+0000","last_clean":"2024-07-26T13:38:46.383157+0000","last_became_active":"2024-07-26T13:36:59.464943+0000","last_became_peered":"2024-07-26T13:36:59.464943+0000","last_unstale":"2024-07-26T13:38:46.383157+0000","last_undegraded":"2024-07-26T13:38:46.383157+0000","last_fullsized":"2024-07-26T13:38:46.383157+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T13:53:54.160584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.352946+0000","last_change":"2024-07-26T13:36:59.462884+0000","last_active":"2024-07-26T13:38:46.352946+0000","last_peered":"2024-07-26T13:38:46.352946+0000","last_clean":"2024-07-26T13:38:46.352946+0000","last_became_active":"2024-07-26T13:36:59.462564+0000","last_became_peered":"2024-07-26T13:36:59.462564+0000","last_unstale":"2024-07-26T13:38:46.352946+0000","last_undegraded":"2024-07-26T13:38:46.352946+0000","last_fullsized":"2024-07-26T13:38:46.352946+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:50:19.549148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.386880+0000","last_change":"2024-07-26T13:36:59.468716+0000","last_active":"2024-07-26T13:38:46.386880+0000","last_peered":"2024-07-26T13:38:46.386880+0000","last_clean":"2024-07-26T13:38:46.386880+0000","last_became_active":"2024-07-26T13:36:59.468048+0000","last_became_peered":"2024-07-26T13:36:59.468048+0000","last_unstale":"2024-07-26T13:38:46.386880+0000","last_undegraded":"2024-07-26T13:38:46.386880+0000","last_fullsized":"2024-07-26T13:38:46.386880+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:51:29.510047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383080+0000","last_change":"2024-07-26T13:36:59.466565+0000","last_active":"2024-07-26T13:38:46.383080+0000","last_peered":"2024-07-26T13:38:46.383080+0000","last_clean":"2024-07-26T13:38:46.383080+0000","last_became_active":"2024-07-26T13:36:59.466235+0000","last_became_peered":"2024-07-26T13:36:59.466235+0000","last_unstale":"2024-07-26T13:38:46.383080+0000","last_undegraded":"2024-07-26T13:38:46.383080+0000","last_fullsized":"2024-07-26T13:38:46.383080+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:28:44.326821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.371778+0000","last_change":"2024-07-26T13:36:59.458753+0000","last_active":"2024-07-26T13:38:46.371778+0000","last_peered":"2024-07-26T13:38:46.371778+0000","last_clean":"2024-07-26T13:38:46.371778+0000","last_became_active":"2024-07-26T13:36:59.458333+0000","last_became_peered":"2024-07-26T13:36:59.458333+0000","last_unstale":"2024-07-26T13:38:46.371778+0000","last_undegraded":"2024-07-26T13:38:46.371778+0000","last_fullsized":"2024-07-26T13:38:46.371778+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:33:54.806212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.366828+0000","last_change":"2024-07-26T13:36:59.468657+0000","last_active":"2024-07-26T13:38:46.366828+0000","last_peered":"2024-07-26T13:38:46.366828+0000","last_clean":"2024-07-26T13:38:46.366828+0000","last_became_active":"2024-07-26T13:36:59.468121+0000","last_became_peered":"2024-07-26T13:36:59.468121+0000","last_unstale":"2024-07-26T13:38:46.366828+0000","last_undegraded":"2024-07-26T13:38:46.366828+0000","last_fullsized":"2024-07-26T13:38:46.366828+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:23:53.959836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.377483+0000","last_change":"2024-07-26T13:36:59.469067+0000","last_active":"2024-07-26T13:38:46.377483+0000","last_peered":"2024-07-26T13:38:46.377483+0000","last_clean":"2024-07-26T13:38:46.377483+0000","last_became_active":"2024-07-26T13:36:59.468706+0000","last_became_peered":"2024-07-26T13:36:59.468706+0000","last_unstale":"2024-07-26T13:38:46.377483+0000","last_undegraded":"2024-07-26T13:38:46.377483+0000","last_fullsized":"2024-07-26T13:38:46.377483+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:21:35.974618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.378127+0000","last_change":"2024-07-26T13:36:59.469807+0000","last_active":"2024-07-26T13:38:46.378127+0000","last_peered":"2024-07-26T13:38:46.378127+0000","last_clean":"2024-07-26T13:38:46.378127+0000","last_became_active":"2024-07-26T13:36:59.469020+0000","last_became_peered":"2024-07-26T13:36:59.469020+0000","last_unstale":"2024-07-26T13:38:46.378127+0000","last_undegraded":"2024-07-26T13:38:46.378127+0000","last_fullsized":"2024-07-26T13:38:46.378127+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:29:57.931598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.372785+0000","last_change":"2024-07-26T13:36:59.465236+0000","last_active":"2024-07-26T13:38:46.372785+0000","last_peered":"2024-07-26T13:38:46.372785+0000","last_clean":"2024-07-26T13:38:46.372785+0000","last_became_active":"2024-07-26T13:36:59.464415+0000","last_became_peered":"2024-07-26T13:36:59.464415+0000","last_unstale":"2024-07-26T13:38:46.372785+0000","last_undegraded":"2024-07-26T13:38:46.372785+0000","last_fullsized":"2024-07-26T13:38:46.372785+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:46:20.373688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.375861+0000","last_change":"2024-07-26T13:36:59.466113+0000","last_active":"2024-07-26T13:38:46.375861+0000","last_peered":"2024-07-26T13:38:46.375861+0000","last_clean":"2024-07-26T13:38:46.375861+0000","last_became_active":"2024-07-26T13:36:59.465743+0000","last_became_peered":"2024-07-26T13:36:59.465743+0000","last_unstale":"2024-07-26T13:38:46.375861+0000","last_undegraded":"2024-07-26T13:38:46.375861+0000","last_fullsized":"2024-07-26T13:38:46.375861+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:14:41.839204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.380300+0000","last_change":"2024-07-26T13:36:59.460672+0000","last_active":"2024-07-26T13:38:46.380300+0000","last_peered":"2024-07-26T13:38:46.380300+0000","last_clean":"2024-07-26T13:38:46.380300+0000","last_became_active":"2024-07-26T13:36:59.460247+0000","last_became_peered":"2024-07-26T13:36:59.460247+0000","last_unstale":"2024-07-26T13:38:46.380300+0000","last_undegraded":"2024-07-26T13:38:46.380300+0000","last_fullsized":"2024-07-26T13:38:46.380300+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:37:51.109933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.373301+0000","last_change":"2024-07-26T13:36:59.464784+0000","last_active":"2024-07-26T13:38:46.373301+0000","last_peered":"2024-07-26T13:38:46.373301+0000","last_clean":"2024-07-26T13:38:46.373301+0000","last_became_active":"2024-07-26T13:36:59.464261+0000","last_became_peered":"2024-07-26T13:36:59.464261+0000","last_unstale":"2024-07-26T13:38:46.373301+0000","last_undegraded":"2024-07-26T13:38:46.373301+0000","last_fullsized":"2024-07-26T13:38:46.373301+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:43:10.134037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.384861+0000","last_change":"2024-07-26T13:36:59.460421+0000","last_active":"2024-07-26T13:38:46.384861+0000","last_peered":"2024-07-26T13:38:46.384861+0000","last_clean":"2024-07-26T13:38:46.384861+0000","last_became_active":"2024-07-26T13:36:59.460010+0000","last_became_peered":"2024-07-26T13:36:59.460010+0000","last_unstale":"2024-07-26T13:38:46.384861+0000","last_undegraded":"2024-07-26T13:38:46.384861+0000","last_fullsized":"2024-07-26T13:38:46.384861+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:16:21.651673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.358087+0000","last_change":"2024-07-26T13:36:59.468826+0000","last_active":"2024-07-26T13:38:46.358087+0000","last_peered":"2024-07-26T13:38:46.358087+0000","last_clean":"2024-07-26T13:38:46.358087+0000","last_became_active":"2024-07-26T13:36:59.468406+0000","last_became_peered":"2024-07-26T13:36:59.468406+0000","last_unstale":"2024-07-26T13:38:46.358087+0000","last_undegraded":"2024-07-26T13:38:46.358087+0000","last_fullsized":"2024-07-26T13:38:46.358087+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:22:10.590545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'7","reported_seq":151,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:55.716965+0000","last_change":"2024-07-26T13:37:17.559014+0000","last_active":"2024-07-26T13:38:55.716965+0000","last_peered":"2024-07-26T13:38:55.716965+0000","last_clean":"2024-07-26T13:38:55.716965+0000","last_became_active":"2024-07-26T13:37:17.552325+0000","last_became_peered":"2024-07-26T13:37:17.552325+0000","last_unstale":"2024-07-26T13:38:55.716965+0000","last_undegraded":"2024-07-26T13:38:55.716965+0000","last_fullsized":"2024-07-26T13:38:55.716965+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:26:26.790337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.378262+0000","last_change":"2024-07-26T13:37:17.562871+0000","last_active":"2024-07-26T13:38:46.378262+0000","last_peered":"2024-07-26T13:38:46.378262+0000","last_clean":"2024-07-26T13:38:46.378262+0000","last_became_active":"2024-07-26T13:37:17.562363+0000","last_became_peered":"2024-07-26T13:37:17.562363+0000","last_unstale":"2024-07-26T13:38:46.378262+0000","last_undegraded":"2024-07-26T13:38:46.378262+0000","last_fullsized":"2024-07-26T13:38:46.378262+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:25:36.172091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"79'3","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.547720+0000","last_change":"2024-07-26T13:36:56.434778+0000","last_active":"2024-07-26T13:37:17.547720+0000","last_peered":"2024-07-26T13:37:17.547720+0000","last_clean":"2024-07-26T13:37:17.547720+0000","last_became_active":"2024-07-26T13:36:56.434288+0000","last_became_peered":"2024-07-26T13:36:56.434288+0000","last_unstale":"2024-07-26T13:37:17.547720+0000","last_undegraded":"2024-07-26T13:37:17.547720+0000","last_fullsized":"2024-07-26T13:37:17.547720+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:24:22.990358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.565086+0000","last_change":"2024-07-26T13:36:56.424665+0000","last_active":"2024-07-26T13:37:17.565086+0000","last_peered":"2024-07-26T13:37:17.565086+0000","last_clean":"2024-07-26T13:37:17.565086+0000","last_became_active":"2024-07-26T13:36:56.424290+0000","last_became_peered":"2024-07-26T13:36:56.424290+0000","last_unstale":"2024-07-26T13:37:17.565086+0000","last_undegraded":"2024-07-26T13:37:17.565086+0000","last_fullsized":"2024-07-26T13:37:17.565086+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:49:59.520194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383641+0000","last_change":"2024-07-26T13:36:59.454609+0000","last_active":"2024-07-26T13:38:46.383641+0000","last_peered":"2024-07-26T13:38:46.383641+0000","last_clean":"2024-07-26T13:38:46.383641+0000","last_became_active":"2024-07-26T13:36:59.454228+0000","last_became_peered":"2024-07-26T13:36:59.454228+0000","last_unstale":"2024-07-26T13:38:46.383641+0000","last_undegraded":"2024-07-26T13:38:46.383641+0000","last_fullsized":"2024-07-26T13:38:46.383641+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:02:06.244433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.547076+0000","last_change":"2024-07-26T13:36:56.430268+0000","last_active":"2024-07-26T13:37:17.547076+0000","last_peered":"2024-07-26T13:37:17.547076+0000","last_clean":"2024-07-26T13:37:17.547076+0000","last_became_active":"2024-07-26T13:36:56.428583+0000","last_became_peered":"2024-07-26T13:36:56.428583+0000","last_unstale":"2024-07-26T13:37:17.547076+0000","last_undegraded":"2024-07-26T13:37:17.547076+0000","last_fullsized":"2024-07-26T13:37:17.547076+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:07:05.864886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.379574+0000","last_change":"2024-07-26T13:36:59.456188+0000","last_active":"2024-07-26T13:38:46.379574+0000","last_peered":"2024-07-26T13:38:46.379574+0000","last_clean":"2024-07-26T13:38:46.379574+0000","last_became_active":"2024-07-26T13:36:59.455842+0000","last_became_peered":"2024-07-26T13:36:59.455842+0000","last_unstale":"2024-07-26T13:38:46.379574+0000","last_undegraded":"2024-07-26T13:38:46.379574+0000","last_fullsized":"2024-07-26T13:38:46.379574+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:00:19.162239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":131,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:56.717922+0000","last_change":"2024-07-26T13:36:56.432818+0000","last_active":"2024-07-26T13:38:56.717922+0000","last_peered":"2024-07-26T13:38:56.717922+0000","last_clean":"2024-07-26T13:38:56.717922+0000","last_became_active":"2024-07-26T13:36:56.432419+0000","last_became_peered":"2024-07-26T13:36:56.432419+0000","last_unstale":"2024-07-26T13:38:56.717922+0000","last_undegraded":"2024-07-26T13:38:56.717922+0000","last_fullsized":"2024-07-26T13:38:56.717922+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:55:51.972470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.386803+0000","last_change":"2024-07-26T13:36:59.460033+0000","last_active":"2024-07-26T13:38:46.386803+0000","last_peered":"2024-07-26T13:38:46.386803+0000","last_clean":"2024-07-26T13:38:46.386803+0000","last_became_active":"2024-07-26T13:36:59.459562+0000","last_became_peered":"2024-07-26T13:36:59.459562+0000","last_unstale":"2024-07-26T13:38:46.386803+0000","last_undegraded":"2024-07-26T13:38:46.386803+0000","last_fullsized":"2024-07-26T13:38:46.386803+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:58:14.155973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551314+0000","last_change":"2024-07-26T13:36:56.432563+0000","last_active":"2024-07-26T13:37:17.551314+0000","last_peered":"2024-07-26T13:37:17.551314+0000","last_clean":"2024-07-26T13:37:17.551314+0000","last_became_active":"2024-07-26T13:36:56.431134+0000","last_became_peered":"2024-07-26T13:36:56.431134+0000","last_unstale":"2024-07-26T13:37:17.551314+0000","last_undegraded":"2024-07-26T13:37:17.551314+0000","last_fullsized":"2024-07-26T13:37:17.551314+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:42:00.653345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.377720+0000","last_change":"2024-07-26T13:36:59.469782+0000","last_active":"2024-07-26T13:38:46.377720+0000","last_peered":"2024-07-26T13:38:46.377720+0000","last_clean":"2024-07-26T13:38:46.377720+0000","last_became_active":"2024-07-26T13:36:59.469412+0000","last_became_peered":"2024-07-26T13:36:59.469412+0000","last_unstale":"2024-07-26T13:38:46.377720+0000","last_undegraded":"2024-07-26T13:38:46.377720+0000","last_fullsized":"2024-07-26T13:38:46.377720+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:28:07.741486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.548023+0000","last_change":"2024-07-26T13:36:56.488981+0000","last_active":"2024-07-26T13:37:17.548023+0000","last_peered":"2024-07-26T13:37:17.548023+0000","last_clean":"2024-07-26T13:37:17.548023+0000","last_became_active":"2024-07-26T13:36:56.488480+0000","last_became_peered":"2024-07-26T13:36:56.488480+0000","last_unstale":"2024-07-26T13:37:17.548023+0000","last_undegraded":"2024-07-26T13:37:17.548023+0000","last_fullsized":"2024-07-26T13:37:17.548023+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:41:10.041007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.386815+0000","last_change":"2024-07-26T13:36:59.473183+0000","last_active":"2024-07-26T13:38:46.386815+0000","last_peered":"2024-07-26T13:38:46.386815+0000","last_clean":"2024-07-26T13:38:46.386815+0000","last_became_active":"2024-07-26T13:36:59.472814+0000","last_became_peered":"2024-07-26T13:36:59.472814+0000","last_unstale":"2024-07-26T13:38:46.386815+0000","last_undegraded":"2024-07-26T13:38:46.386815+0000","last_fullsized":"2024-07-26T13:38:46.386815+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:57:03.396384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'32","reported_seq":269,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:57.738215+0000","last_change":"2024-07-26T13:36:56.435034+0000","last_active":"2024-07-26T13:38:57.738215+0000","last_peered":"2024-07-26T13:38:57.738215+0000","last_clean":"2024-07-26T13:38:57.738215+0000","last_became_active":"2024-07-26T13:36:56.434611+0000","last_became_peered":"2024-07-26T13:36:56.434611+0000","last_unstale":"2024-07-26T13:38:57.738215+0000","last_undegraded":"2024-07-26T13:38:57.738215+0000","last_fullsized":"2024-07-26T13:38:57.738215+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:18:27.272077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62584,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":211,"num_read_kb":166,"num_write":33,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.382186+0000","last_change":"2024-07-26T13:36:59.465831+0000","last_active":"2024-07-26T13:38:46.382186+0000","last_peered":"2024-07-26T13:38:46.382186+0000","last_clean":"2024-07-26T13:38:46.382186+0000","last_became_active":"2024-07-26T13:36:59.465404+0000","last_became_peered":"2024-07-26T13:36:59.465404+0000","last_unstale":"2024-07-26T13:38:46.382186+0000","last_undegraded":"2024-07-26T13:38:46.382186+0000","last_fullsized":"2024-07-26T13:38:46.382186+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:42:53.795926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'6","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.552433+0000","last_change":"2024-07-26T13:36:56.435170+0000","last_active":"2024-07-26T13:37:17.552433+0000","last_peered":"2024-07-26T13:37:17.552433+0000","last_clean":"2024-07-26T13:37:17.552433+0000","last_became_active":"2024-07-26T13:36:56.434715+0000","last_became_peered":"2024-07-26T13:36:56.434715+0000","last_unstale":"2024-07-26T13:37:17.552433+0000","last_undegraded":"2024-07-26T13:37:17.552433+0000","last_fullsized":"2024-07-26T13:37:17.552433+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:05:09.712510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.354602+0000","last_change":"2024-07-26T13:36:59.461926+0000","last_active":"2024-07-26T13:38:46.354602+0000","last_peered":"2024-07-26T13:38:46.354602+0000","last_clean":"2024-07-26T13:38:46.354602+0000","last_became_active":"2024-07-26T13:36:59.461512+0000","last_became_peered":"2024-07-26T13:36:59.461512+0000","last_unstale":"2024-07-26T13:38:46.354602+0000","last_undegraded":"2024-07-26T13:38:46.354602+0000","last_fullsized":"2024-07-26T13:38:46.354602+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:34:24.405663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.546843+0000","last_change":"2024-07-26T13:36:56.430062+0000","last_active":"2024-07-26T13:37:17.546843+0000","last_peered":"2024-07-26T13:37:17.546843+0000","last_clean":"2024-07-26T13:37:17.546843+0000","last_became_active":"2024-07-26T13:36:56.429654+0000","last_became_peered":"2024-07-26T13:36:56.429654+0000","last_unstale":"2024-07-26T13:37:17.546843+0000","last_undegraded":"2024-07-26T13:37:17.546843+0000","last_fullsized":"2024-07-26T13:37:17.546843+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:21:09.514359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.379830+0000","last_change":"2024-07-26T13:36:59.474459+0000","last_active":"2024-07-26T13:38:46.379830+0000","last_peered":"2024-07-26T13:38:46.379830+0000","last_clean":"2024-07-26T13:38:46.379830+0000","last_became_active":"2024-07-26T13:36:59.474061+0000","last_became_peered":"2024-07-26T13:36:59.474061+0000","last_unstale":"2024-07-26T13:38:46.379830+0000","last_undegraded":"2024-07-26T13:38:46.379830+0000","last_fullsized":"2024-07-26T13:38:46.379830+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:37:21.691491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'5","reported_seq":137,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:56.717864+0000","last_change":"2024-07-26T13:36:56.432915+0000","last_active":"2024-07-26T13:38:56.717864+0000","last_peered":"2024-07-26T13:38:56.717864+0000","last_clean":"2024-07-26T13:38:56.717864+0000","last_became_active":"2024-07-26T13:36:56.432499+0000","last_became_peered":"2024-07-26T13:36:56.432499+0000","last_unstale":"2024-07-26T13:38:56.717864+0000","last_undegraded":"2024-07-26T13:38:56.717864+0000","last_fullsized":"2024-07-26T13:38:56.717864+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:02:14.306950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383440+0000","last_change":"2024-07-26T13:36:59.468945+0000","last_active":"2024-07-26T13:38:46.383440+0000","last_peered":"2024-07-26T13:38:46.383440+0000","last_clean":"2024-07-26T13:38:46.383440+0000","last_became_active":"2024-07-26T13:36:59.467970+0000","last_became_peered":"2024-07-26T13:36:59.467970+0000","last_unstale":"2024-07-26T13:38:46.383440+0000","last_undegraded":"2024-07-26T13:38:46.383440+0000","last_fullsized":"2024-07-26T13:38:46.383440+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T13:59:19.514945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'15","reported_seq":43,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551317+0000","last_change":"2024-07-26T13:36:56.487014+0000","last_active":"2024-07-26T13:37:17.551317+0000","last_peered":"2024-07-26T13:37:17.551317+0000","last_clean":"2024-07-26T13:37:17.551317+0000","last_became_active":"2024-07-26T13:36:56.486470+0000","last_became_peered":"2024-07-26T13:36:56.486470+0000","last_unstale":"2024-07-26T13:37:17.551317+0000","last_undegraded":"2024-07-26T13:37:17.551317+0000","last_fullsized":"2024-07-26T13:37:17.551317+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:07:02.205644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.366132+0000","last_change":"2024-07-26T13:36:59.462465+0000","last_active":"2024-07-26T13:38:46.366132+0000","last_peered":"2024-07-26T13:38:46.366132+0000","last_clean":"2024-07-26T13:38:46.366132+0000","last_became_active":"2024-07-26T13:36:59.462024+0000","last_became_peered":"2024-07-26T13:36:59.462024+0000","last_unstale":"2024-07-26T13:38:46.366132+0000","last_undegraded":"2024-07-26T13:38:46.366132+0000","last_fullsized":"2024-07-26T13:38:46.366132+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:34:14.317762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551923+0000","last_change":"2024-07-26T13:36:56.435007+0000","last_active":"2024-07-26T13:37:17.551923+0000","last_peered":"2024-07-26T13:37:17.551923+0000","last_clean":"2024-07-26T13:37:17.551923+0000","last_became_active":"2024-07-26T13:36:56.434444+0000","last_became_peered":"2024-07-26T13:36:56.434444+0000","last_unstale":"2024-07-26T13:37:17.551923+0000","last_undegraded":"2024-07-26T13:37:17.551923+0000","last_fullsized":"2024-07-26T13:37:17.551923+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:27:51.041349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.384358+0000","last_change":"2024-07-26T13:36:59.464164+0000","last_active":"2024-07-26T13:38:46.384358+0000","last_peered":"2024-07-26T13:38:46.384358+0000","last_clean":"2024-07-26T13:38:46.384358+0000","last_became_active":"2024-07-26T13:36:59.463782+0000","last_became_peered":"2024-07-26T13:36:59.463782+0000","last_unstale":"2024-07-26T13:38:46.384358+0000","last_undegraded":"2024-07-26T13:38:46.384358+0000","last_fullsized":"2024-07-26T13:38:46.384358+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:48:06.481731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.565453+0000","last_change":"2024-07-26T13:36:56.429653+0000","last_active":"2024-07-26T13:37:17.565453+0000","last_peered":"2024-07-26T13:37:17.565453+0000","last_clean":"2024-07-26T13:37:17.565453+0000","last_became_active":"2024-07-26T13:36:56.429060+0000","last_became_peered":"2024-07-26T13:36:56.429060+0000","last_unstale":"2024-07-26T13:37:17.565453+0000","last_undegraded":"2024-07-26T13:37:17.565453+0000","last_fullsized":"2024-07-26T13:37:17.565453+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:41:48.261709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383390+0000","last_change":"2024-07-26T13:36:59.471560+0000","last_active":"2024-07-26T13:38:46.383390+0000","last_peered":"2024-07-26T13:38:46.383390+0000","last_clean":"2024-07-26T13:38:46.383390+0000","last_became_active":"2024-07-26T13:36:59.471151+0000","last_became_peered":"2024-07-26T13:36:59.471151+0000","last_unstale":"2024-07-26T13:38:46.383390+0000","last_undegraded":"2024-07-26T13:38:46.383390+0000","last_fullsized":"2024-07-26T13:38:46.383390+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:49:20.513308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.548108+0000","last_change":"2024-07-26T13:36:56.430010+0000","last_active":"2024-07-26T13:37:17.548108+0000","last_peered":"2024-07-26T13:37:17.548108+0000","last_clean":"2024-07-26T13:37:17.548108+0000","last_became_active":"2024-07-26T13:36:56.429120+0000","last_became_peered":"2024-07-26T13:36:56.429120+0000","last_unstale":"2024-07-26T13:37:17.548108+0000","last_undegraded":"2024-07-26T13:37:17.548108+0000","last_fullsized":"2024-07-26T13:37:17.548108+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:23:43.321975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383377+0000","last_change":"2024-07-26T13:36:59.467788+0000","last_active":"2024-07-26T13:38:46.383377+0000","last_peered":"2024-07-26T13:38:46.383377+0000","last_clean":"2024-07-26T13:38:46.383377+0000","last_became_active":"2024-07-26T13:36:59.467399+0000","last_became_peered":"2024-07-26T13:36:59.467399+0000","last_unstale":"2024-07-26T13:38:46.383377+0000","last_undegraded":"2024-07-26T13:38:46.383377+0000","last_fullsized":"2024-07-26T13:38:46.383377+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:48:51.442999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'3","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.553251+0000","last_change":"2024-07-26T13:36:56.588633+0000","last_active":"2024-07-26T13:37:17.553251+0000","last_peered":"2024-07-26T13:37:17.553251+0000","last_clean":"2024-07-26T13:37:17.553251+0000","last_became_active":"2024-07-26T13:36:56.588113+0000","last_became_peered":"2024-07-26T13:36:56.588113+0000","last_unstale":"2024-07-26T13:37:17.553251+0000","last_undegraded":"2024-07-26T13:37:17.553251+0000","last_fullsized":"2024-07-26T13:37:17.553251+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:03:55.075052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387396+0000","last_change":"2024-07-26T13:36:59.472854+0000","last_active":"2024-07-26T13:38:46.387396+0000","last_peered":"2024-07-26T13:38:46.387396+0000","last_clean":"2024-07-26T13:38:46.387396+0000","last_became_active":"2024-07-26T13:36:59.472466+0000","last_became_peered":"2024-07-26T13:36:59.472466+0000","last_unstale":"2024-07-26T13:38:46.387396+0000","last_undegraded":"2024-07-26T13:38:46.387396+0000","last_fullsized":"2024-07-26T13:38:46.387396+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:50:57.454149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":103,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550829+0000","last_change":"2024-07-26T13:35:34.839121+0000","last_active":"2024-07-26T13:37:17.550829+0000","last_peered":"2024-07-26T13:37:17.550829+0000","last_clean":"2024-07-26T13:37:17.550829+0000","last_became_active":"2024-07-26T13:35:34.833465+0000","last_became_peered":"2024-07-26T13:35:34.833465+0000","last_unstale":"2024-07-26T13:37:17.550829+0000","last_undegraded":"2024-07-26T13:37:17.550829+0000","last_fullsized":"2024-07-26T13:37:17.550829+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:32:23.285428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:32:23.285428+0000","last_clean_scrub_stamp":"2024-07-26T13:32:23.285428+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:29:31.650487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'3","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.561400+0000","last_change":"2024-07-26T13:36:56.428648+0000","last_active":"2024-07-26T13:37:17.561400+0000","last_peered":"2024-07-26T13:37:17.561400+0000","last_clean":"2024-07-26T13:37:17.561400+0000","last_became_active":"2024-07-26T13:36:56.427665+0000","last_became_peered":"2024-07-26T13:36:56.427665+0000","last_unstale":"2024-07-26T13:37:17.561400+0000","last_undegraded":"2024-07-26T13:37:17.561400+0000","last_fullsized":"2024-07-26T13:37:17.561400+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:41:39.371687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.376509+0000","last_change":"2024-07-26T13:36:59.460625+0000","last_active":"2024-07-26T13:38:46.376509+0000","last_peered":"2024-07-26T13:38:46.376509+0000","last_clean":"2024-07-26T13:38:46.376509+0000","last_became_active":"2024-07-26T13:36:59.458658+0000","last_became_peered":"2024-07-26T13:36:59.458658+0000","last_unstale":"2024-07-26T13:38:46.376509+0000","last_undegraded":"2024-07-26T13:38:46.376509+0000","last_fullsized":"2024-07-26T13:38:46.376509+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:46:42.812402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'3","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550796+0000","last_change":"2024-07-26T13:36:56.432617+0000","last_active":"2024-07-26T13:37:17.550796+0000","last_peered":"2024-07-26T13:37:17.550796+0000","last_clean":"2024-07-26T13:37:17.550796+0000","last_became_active":"2024-07-26T13:36:56.431519+0000","last_became_peered":"2024-07-26T13:36:56.431519+0000","last_unstale":"2024-07-26T13:37:17.550796+0000","last_undegraded":"2024-07-26T13:37:17.550796+0000","last_fullsized":"2024-07-26T13:37:17.550796+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:32:21.120076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.364540+0000","last_change":"2024-07-26T13:36:59.464084+0000","last_active":"2024-07-26T13:38:46.364540+0000","last_peered":"2024-07-26T13:38:46.364540+0000","last_clean":"2024-07-26T13:38:46.364540+0000","last_became_active":"2024-07-26T13:36:59.463755+0000","last_became_peered":"2024-07-26T13:36:59.463755+0000","last_unstale":"2024-07-26T13:38:46.364540+0000","last_undegraded":"2024-07-26T13:38:46.364540+0000","last_fullsized":"2024-07-26T13:38:46.364540+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:53:20.034188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":133,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:55.715881+0000","last_change":"2024-07-26T13:36:56.486071+0000","last_active":"2024-07-26T13:38:55.715881+0000","last_peered":"2024-07-26T13:38:55.715881+0000","last_clean":"2024-07-26T13:38:55.715881+0000","last_became_active":"2024-07-26T13:36:56.485672+0000","last_became_peered":"2024-07-26T13:36:56.485672+0000","last_unstale":"2024-07-26T13:38:55.715881+0000","last_undegraded":"2024-07-26T13:38:55.715881+0000","last_fullsized":"2024-07-26T13:38:55.715881+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T21:29:45.666651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.370979+0000","last_change":"2024-07-26T13:37:17.566555+0000","last_active":"2024-07-26T13:38:46.370979+0000","last_peered":"2024-07-26T13:38:46.370979+0000","last_clean":"2024-07-26T13:38:46.370979+0000","last_became_active":"2024-07-26T13:37:17.566040+0000","last_became_peered":"2024-07-26T13:37:17.566040+0000","last_unstale":"2024-07-26T13:38:46.370979+0000","last_undegraded":"2024-07-26T13:38:46.370979+0000","last_fullsized":"2024-07-26T13:38:46.370979+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:48:33.078890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'1","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550922+0000","last_change":"2024-07-26T13:36:56.431811+0000","last_active":"2024-07-26T13:37:17.550922+0000","last_peered":"2024-07-26T13:37:17.550922+0000","last_clean":"2024-07-26T13:37:17.550922+0000","last_became_active":"2024-07-26T13:36:56.431223+0000","last_became_peered":"2024-07-26T13:36:56.431223+0000","last_unstale":"2024-07-26T13:37:17.550922+0000","last_undegraded":"2024-07-26T13:37:17.550922+0000","last_fullsized":"2024-07-26T13:37:17.550922+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:08:00.334515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.363722+0000","last_change":"2024-07-26T13:36:59.463972+0000","last_active":"2024-07-26T13:38:46.363722+0000","last_peered":"2024-07-26T13:38:46.363722+0000","last_clean":"2024-07-26T13:38:46.363722+0000","last_became_active":"2024-07-26T13:36:59.463496+0000","last_became_peered":"2024-07-26T13:36:59.463496+0000","last_unstale":"2024-07-26T13:38:46.363722+0000","last_undegraded":"2024-07-26T13:38:46.363722+0000","last_fullsized":"2024-07-26T13:38:46.363722+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:02:06.215372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.545709+0000","last_change":"2024-07-26T13:36:56.486681+0000","last_active":"2024-07-26T13:37:17.545709+0000","last_peered":"2024-07-26T13:37:17.545709+0000","last_clean":"2024-07-26T13:37:17.545709+0000","last_became_active":"2024-07-26T13:36:56.486328+0000","last_became_peered":"2024-07-26T13:36:56.486328+0000","last_unstale":"2024-07-26T13:37:17.545709+0000","last_undegraded":"2024-07-26T13:37:17.545709+0000","last_fullsized":"2024-07-26T13:37:17.545709+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:10:36.557707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.355226+0000","last_change":"2024-07-26T13:36:59.451415+0000","last_active":"2024-07-26T13:38:46.355226+0000","last_peered":"2024-07-26T13:38:46.355226+0000","last_clean":"2024-07-26T13:38:46.355226+0000","last_became_active":"2024-07-26T13:36:59.450992+0000","last_became_peered":"2024-07-26T13:36:59.450992+0000","last_unstale":"2024-07-26T13:38:46.355226+0000","last_undegraded":"2024-07-26T13:38:46.355226+0000","last_fullsized":"2024-07-26T13:38:46.355226+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:29:11.303916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.555734+0000","last_change":"2024-07-26T13:37:17.555643+0000","last_active":"2024-07-26T13:37:17.555734+0000","last_peered":"2024-07-26T13:37:17.555734+0000","last_clean":"2024-07-26T13:37:17.555734+0000","last_became_active":"2024-07-26T13:37:17.549756+0000","last_became_peered":"2024-07-26T13:37:17.549756+0000","last_unstale":"2024-07-26T13:37:17.555734+0000","last_undegraded":"2024-07-26T13:37:17.555734+0000","last_fullsized":"2024-07-26T13:37:17.555734+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:20:16.911286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.368420+0000","last_change":"2024-07-26T13:36:59.465538+0000","last_active":"2024-07-26T13:38:46.368420+0000","last_peered":"2024-07-26T13:38:46.368420+0000","last_clean":"2024-07-26T13:38:46.368420+0000","last_became_active":"2024-07-26T13:36:59.464886+0000","last_became_peered":"2024-07-26T13:36:59.464886+0000","last_unstale":"2024-07-26T13:38:46.368420+0000","last_undegraded":"2024-07-26T13:38:46.368420+0000","last_fullsized":"2024-07-26T13:38:46.368420+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:19:12.273116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'12","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:40.480485+0000","last_change":"2024-07-26T13:36:56.428355+0000","last_active":"2024-07-26T13:37:40.480485+0000","last_peered":"2024-07-26T13:37:40.480485+0000","last_clean":"2024-07-26T13:37:40.480485+0000","last_became_active":"2024-07-26T13:36:56.427213+0000","last_became_peered":"2024-07-26T13:36:56.427213+0000","last_unstale":"2024-07-26T13:37:40.480485+0000","last_undegraded":"2024-07-26T13:37:40.480485+0000","last_fullsized":"2024-07-26T13:37:40.480485+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:48:09.730067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.378381+0000","last_change":"2024-07-26T13:36:59.459378+0000","last_active":"2024-07-26T13:38:46.378381+0000","last_peered":"2024-07-26T13:38:46.378381+0000","last_clean":"2024-07-26T13:38:46.378381+0000","last_became_active":"2024-07-26T13:36:59.458799+0000","last_became_peered":"2024-07-26T13:36:59.458799+0000","last_unstale":"2024-07-26T13:38:46.378381+0000","last_undegraded":"2024-07-26T13:38:46.378381+0000","last_fullsized":"2024-07-26T13:38:46.378381+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:45:19.587800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.561367+0000","last_change":"2024-07-26T13:36:56.588819+0000","last_active":"2024-07-26T13:37:17.561367+0000","last_peered":"2024-07-26T13:37:17.561367+0000","last_clean":"2024-07-26T13:37:17.561367+0000","last_became_active":"2024-07-26T13:36:56.588430+0000","last_became_peered":"2024-07-26T13:36:56.588430+0000","last_unstale":"2024-07-26T13:37:17.561367+0000","last_undegraded":"2024-07-26T13:37:17.561367+0000","last_fullsized":"2024-07-26T13:37:17.561367+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:07:21.387618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.380368+0000","last_change":"2024-07-26T13:37:17.562040+0000","last_active":"2024-07-26T13:38:46.380368+0000","last_peered":"2024-07-26T13:38:46.380368+0000","last_clean":"2024-07-26T13:38:46.380368+0000","last_became_active":"2024-07-26T13:37:17.561542+0000","last_became_peered":"2024-07-26T13:37:17.561542+0000","last_unstale":"2024-07-26T13:38:46.380368+0000","last_undegraded":"2024-07-26T13:38:46.380368+0000","last_fullsized":"2024-07-26T13:38:46.380368+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:21:08.161954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.544058+0000","last_change":"2024-07-26T13:36:56.489106+0000","last_active":"2024-07-26T13:37:17.544058+0000","last_peered":"2024-07-26T13:37:17.544058+0000","last_clean":"2024-07-26T13:37:17.544058+0000","last_became_active":"2024-07-26T13:36:56.488706+0000","last_became_peered":"2024-07-26T13:36:56.488706+0000","last_unstale":"2024-07-26T13:37:17.544058+0000","last_undegraded":"2024-07-26T13:37:17.544058+0000","last_fullsized":"2024-07-26T13:37:17.544058+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:00:59.204434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.336503+0000","last_change":"2024-07-26T13:36:59.470412+0000","last_active":"2024-07-26T13:38:46.336503+0000","last_peered":"2024-07-26T13:38:46.336503+0000","last_clean":"2024-07-26T13:38:46.336503+0000","last_became_active":"2024-07-26T13:36:59.470010+0000","last_became_peered":"2024-07-26T13:36:59.470010+0000","last_unstale":"2024-07-26T13:38:46.336503+0000","last_undegraded":"2024-07-26T13:38:46.336503+0000","last_fullsized":"2024-07-26T13:38:46.336503+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:50:57.987600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.543473+0000","last_change":"2024-07-26T13:36:56.428671+0000","last_active":"2024-07-26T13:37:17.543473+0000","last_peered":"2024-07-26T13:37:17.543473+0000","last_clean":"2024-07-26T13:37:17.543473+0000","last_became_active":"2024-07-26T13:36:56.428301+0000","last_became_peered":"2024-07-26T13:36:56.428301+0000","last_unstale":"2024-07-26T13:37:17.543473+0000","last_undegraded":"2024-07-26T13:37:17.543473+0000","last_fullsized":"2024-07-26T13:37:17.543473+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:59:19.593664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383551+0000","last_change":"2024-07-26T13:36:59.458627+0000","last_active":"2024-07-26T13:38:46.383551+0000","last_peered":"2024-07-26T13:38:46.383551+0000","last_clean":"2024-07-26T13:38:46.383551+0000","last_became_active":"2024-07-26T13:36:59.458199+0000","last_became_peered":"2024-07-26T13:36:59.458199+0000","last_unstale":"2024-07-26T13:38:46.383551+0000","last_undegraded":"2024-07-26T13:38:46.383551+0000","last_fullsized":"2024-07-26T13:38:46.383551+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:54:41.833223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'1","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551478+0000","last_change":"2024-07-26T13:36:56.431548+0000","last_active":"2024-07-26T13:37:17.551478+0000","last_peered":"2024-07-26T13:37:17.551478+0000","last_clean":"2024-07-26T13:37:17.551478+0000","last_became_active":"2024-07-26T13:36:56.430989+0000","last_became_peered":"2024-07-26T13:36:56.430989+0000","last_unstale":"2024-07-26T13:37:17.551478+0000","last_undegraded":"2024-07-26T13:37:17.551478+0000","last_fullsized":"2024-07-26T13:37:17.551478+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:46:51.655324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.371547+0000","last_change":"2024-07-26T13:36:59.458290+0000","last_active":"2024-07-26T13:38:46.371547+0000","last_peered":"2024-07-26T13:38:46.371547+0000","last_clean":"2024-07-26T13:38:46.371547+0000","last_became_active":"2024-07-26T13:36:59.457942+0000","last_became_peered":"2024-07-26T13:36:59.457942+0000","last_unstale":"2024-07-26T13:38:46.371547+0000","last_undegraded":"2024-07-26T13:38:46.371547+0000","last_fullsized":"2024-07-26T13:38:46.371547+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:51:58.797441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.567687+0000","last_change":"2024-07-26T13:37:17.567624+0000","last_active":"2024-07-26T13:37:17.567687+0000","last_peered":"2024-07-26T13:37:17.567687+0000","last_clean":"2024-07-26T13:37:17.567687+0000","last_became_active":"2024-07-26T13:37:17.562455+0000","last_became_peered":"2024-07-26T13:37:17.562455+0000","last_unstale":"2024-07-26T13:37:17.567687+0000","last_undegraded":"2024-07-26T13:37:17.567687+0000","last_fullsized":"2024-07-26T13:37:17.567687+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:50:36.418750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387702+0000","last_change":"2024-07-26T13:36:59.474034+0000","last_active":"2024-07-26T13:38:46.387702+0000","last_peered":"2024-07-26T13:38:46.387702+0000","last_clean":"2024-07-26T13:38:46.387702+0000","last_became_active":"2024-07-26T13:36:59.472358+0000","last_became_peered":"2024-07-26T13:36:59.472358+0000","last_unstale":"2024-07-26T13:38:46.387702+0000","last_undegraded":"2024-07-26T13:38:46.387702+0000","last_fullsized":"2024-07-26T13:38:46.387702+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:36:44.565413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.367956+0000","last_change":"2024-07-26T13:36:59.465546+0000","last_active":"2024-07-26T13:38:46.367956+0000","last_peered":"2024-07-26T13:38:46.367956+0000","last_clean":"2024-07-26T13:38:46.367956+0000","last_became_active":"2024-07-26T13:36:59.465137+0000","last_became_peered":"2024-07-26T13:36:59.465137+0000","last_unstale":"2024-07-26T13:38:46.367956+0000","last_undegraded":"2024-07-26T13:38:46.367956+0000","last_fullsized":"2024-07-26T13:38:46.367956+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:35:13.591354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.562100+0000","last_change":"2024-07-26T13:37:17.562012+0000","last_active":"2024-07-26T13:37:17.562100+0000","last_peered":"2024-07-26T13:37:17.562100+0000","last_clean":"2024-07-26T13:37:17.562100+0000","last_became_active":"2024-07-26T13:37:17.550218+0000","last_became_peered":"2024-07-26T13:37:17.550218+0000","last_unstale":"2024-07-26T13:37:17.562100+0000","last_undegraded":"2024-07-26T13:37:17.562100+0000","last_fullsized":"2024-07-26T13:37:17.562100+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:46:16.559533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.374985+0000","last_change":"2024-07-26T13:36:59.456896+0000","last_active":"2024-07-26T13:38:46.374985+0000","last_peered":"2024-07-26T13:38:46.374985+0000","last_clean":"2024-07-26T13:38:46.374985+0000","last_became_active":"2024-07-26T13:36:59.456530+0000","last_became_peered":"2024-07-26T13:36:59.456530+0000","last_unstale":"2024-07-26T13:38:46.374985+0000","last_undegraded":"2024-07-26T13:38:46.374985+0000","last_fullsized":"2024-07-26T13:38:46.374985+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:59:56.512501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'4","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.553522+0000","last_change":"2024-07-26T13:36:56.433119+0000","last_active":"2024-07-26T13:37:17.553522+0000","last_peered":"2024-07-26T13:37:17.553522+0000","last_clean":"2024-07-26T13:37:17.553522+0000","last_became_active":"2024-07-26T13:36:56.432765+0000","last_became_peered":"2024-07-26T13:36:56.432765+0000","last_unstale":"2024-07-26T13:37:17.553522+0000","last_undegraded":"2024-07-26T13:37:17.553522+0000","last_fullsized":"2024-07-26T13:37:17.553522+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:31:29.197106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.381903+0000","last_change":"2024-07-26T13:36:59.462125+0000","last_active":"2024-07-26T13:38:46.381903+0000","last_peered":"2024-07-26T13:38:46.381903+0000","last_clean":"2024-07-26T13:38:46.381903+0000","last_became_active":"2024-07-26T13:36:59.461796+0000","last_became_peered":"2024-07-26T13:36:59.461796+0000","last_unstale":"2024-07-26T13:38:46.381903+0000","last_undegraded":"2024-07-26T13:38:46.381903+0000","last_fullsized":"2024-07-26T13:38:46.381903+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:44:10.329046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.543646+0000","last_change":"2024-07-26T13:36:56.487251+0000","last_active":"2024-07-26T13:37:17.543646+0000","last_peered":"2024-07-26T13:37:17.543646+0000","last_clean":"2024-07-26T13:37:17.543646+0000","last_became_active":"2024-07-26T13:36:56.486918+0000","last_became_peered":"2024-07-26T13:36:56.486918+0000","last_unstale":"2024-07-26T13:37:17.543646+0000","last_undegraded":"2024-07-26T13:37:17.543646+0000","last_fullsized":"2024-07-26T13:37:17.543646+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:41:31.248895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.380839+0000","last_change":"2024-07-26T13:36:59.470810+0000","last_active":"2024-07-26T13:38:46.380839+0000","last_peered":"2024-07-26T13:38:46.380839+0000","last_clean":"2024-07-26T13:38:46.380839+0000","last_became_active":"2024-07-26T13:36:59.467371+0000","last_became_peered":"2024-07-26T13:36:59.467371+0000","last_unstale":"2024-07-26T13:38:46.380839+0000","last_undegraded":"2024-07-26T13:38:46.380839+0000","last_fullsized":"2024-07-26T13:38:46.380839+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:23:48.572933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'9","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.545993+0000","last_change":"2024-07-26T13:36:56.433598+0000","last_active":"2024-07-26T13:37:17.545993+0000","last_peered":"2024-07-26T13:37:17.545993+0000","last_clean":"2024-07-26T13:37:17.545993+0000","last_became_active":"2024-07-26T13:36:56.433325+0000","last_became_peered":"2024-07-26T13:36:56.433325+0000","last_unstale":"2024-07-26T13:37:17.545993+0000","last_undegraded":"2024-07-26T13:37:17.545993+0000","last_fullsized":"2024-07-26T13:37:17.545993+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:17:33.651765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67258,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":637,"num_read_kb":390,"num_write":174,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":644,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":201774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247480,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678056,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":443451,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874285,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":1098026,"data_compressed":41637,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037810,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":897024,"data_stored":443665,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234038,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":443563,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430267,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":1036188,"data_compressed":41637,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593792,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":443054,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790021,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":1035340,"data_compressed":41637,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953546,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":888832,"data_stored":444023,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149774,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":443365,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378707,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":506159,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542231,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":506585,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705756,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":444325,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34195,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34195,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-26T13:39:02.683 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-07-26T13:39:02.683 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-07-26T13:39:02.683 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-07-26T13:39:02.683 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph health --format=json 2024-07-26T13:39:02.939 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:39:03.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:03 smithi100 ceph-mon[32599]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-26T13:39:03.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:03 smithi100 ceph-mon[32599]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:39:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:03 smithi167 ceph-mon[37140]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-26T13:39:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:03 smithi167 ceph-mon[37140]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:39:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:03 smithi138 ceph-mon[36954]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-26T13:39:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:03 smithi138 ceph-mon[36954]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:39:04.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.625+0000 7fdd42afb640 1 -- 172.21.15.100:0/3146676940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd3c102490 msgr2=0x7fdd3c1028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:04.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.625+0000 7fdd42afb640 1 --2- 172.21.15.100:0/3146676940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd3c102490 0x7fdd3c1028a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdd30009900 tx=0x7fdd3002a410 comp rx=0 tx=0).stop 2024-07-26T13:39:04.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.625+0000 7fdd42afb640 1 -- 172.21.15.100:0/3146676940 shutdown_connections 2024-07-26T13:39:04.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.625+0000 7fdd42afb640 1 --2- 172.21.15.100:0/3146676940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd3c108720 0x7fdd3c10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:04.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.625+0000 7fdd42afb640 1 --2- 172.21.15.100:0/3146676940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd3c107dd0 0x7fdd3c1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:04.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.625+0000 7fdd42afb640 1 --2- 172.21.15.100:0/3146676940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd3c102490 0x7fdd3c1028a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:04.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.625+0000 7fdd42afb640 1 -- 172.21.15.100:0/3146676940 >> 172.21.15.100:0/3146676940 conn(0x7fdd3c079160 msgr2=0x7fdd3c079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:04.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.625+0000 7fdd42afb640 1 -- 172.21.15.100:0/3146676940 shutdown_connections 2024-07-26T13:39:04.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.625+0000 7fdd42afb640 1 -- 172.21.15.100:0/3146676940 wait complete. 2024-07-26T13:39:04.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.626+0000 7fdd42afb640 1 Processor -- start 2024-07-26T13:39:04.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.626+0000 7fdd42afb640 1 -- start start 2024-07-26T13:39:04.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.626+0000 7fdd42afb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd3c102490 0x7fdd3c198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:04.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.626+0000 7fdd42afb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd3c107dd0 0x7fdd3c1990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:04.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.626+0000 7fdd42afb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd3c108720 0x7fdd3c199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:04.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.626+0000 7fdd42afb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3c112400 con 0x7fdd3c108720 2024-07-26T13:39:04.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.626+0000 7fdd42afb640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3c112280 con 0x7fdd3c102490 2024-07-26T13:39:04.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.626+0000 7fdd42afb640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3c112580 con 0x7fdd3c107dd0 2024-07-26T13:39:04.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.627+0000 7fdd41098640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd3c108720 0x7fdd3c199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:04.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.627+0000 7fdd41098640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd3c108720 0x7fdd3c199600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46204/0 (socket says 172.21.15.100:46204) 2024-07-26T13:39:04.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.627+0000 7fdd41098640 1 -- 172.21.15.100:0/475257917 learned_addr learned my addr 172.21.15.100:0/475257917 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:04.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.627+0000 7fdd40897640 1 --2- 172.21.15.100:0/475257917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd3c102490 0x7fdd3c198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:04.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.627+0000 7fdd3bfff640 1 --2- 172.21.15.100:0/475257917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd3c107dd0 0x7fdd3c1990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:04.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.627+0000 7fdd41098640 1 -- 172.21.15.100:0/475257917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd3c107dd0 msgr2=0x7fdd3c1990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:04.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.627+0000 7fdd41098640 1 --2- 172.21.15.100:0/475257917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd3c107dd0 0x7fdd3c1990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:04.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.627+0000 7fdd41098640 1 -- 172.21.15.100:0/475257917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd3c102490 msgr2=0x7fdd3c198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:04.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.627+0000 7fdd41098640 1 --2- 172.21.15.100:0/475257917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd3c102490 0x7fdd3c198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:04.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.627+0000 7fdd41098640 1 -- 172.21.15.100:0/475257917 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd3c1a8230 con 0x7fdd3c108720 2024-07-26T13:39:04.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.627+0000 7fdd41098640 1 --2- 172.21.15.100:0/475257917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd3c108720 0x7fdd3c199600 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdd3400b970 tx=0x7fdd3400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:04.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.627+0000 7fdd39ffb640 1 -- 172.21.15.100:0/475257917 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd34019070 con 0x7fdd3c108720 2024-07-26T13:39:04.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.627+0000 7fdd39ffb640 1 -- 172.21.15.100:0/475257917 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd34014440 con 0x7fdd3c108720 2024-07-26T13:39:04.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.627+0000 7fdd39ffb640 1 -- 172.21.15.100:0/475257917 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd340075d0 con 0x7fdd3c108720 2024-07-26T13:39:04.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.627+0000 7fdd42afb640 1 -- 172.21.15.100:0/475257917 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd3c1a8520 con 0x7fdd3c108720 2024-07-26T13:39:04.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.628+0000 7fdd42afb640 1 -- 172.21.15.100:0/475257917 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd3c1a8a90 con 0x7fdd3c108720 2024-07-26T13:39:04.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.629+0000 7fdd42afb640 1 -- 172.21.15.100:0/475257917 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd3c1028a0 con 0x7fdd3c108720 2024-07-26T13:39:04.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.630+0000 7fdd39ffb640 1 -- 172.21.15.100:0/475257917 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7fdd3401c070 con 0x7fdd3c108720 2024-07-26T13:39:04.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.631+0000 7fdd39ffb640 1 --2- 172.21.15.100:0/475257917 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fdd2c0b3c20 0x7fdd2c0b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:04.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.631+0000 7fdd40897640 1 --2- 172.21.15.100:0/475257917 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fdd2c0b3c20 0x7fdd2c0b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:04.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.631+0000 7fdd39ffb640 1 -- 172.21.15.100:0/475257917 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fdd340e37a0 con 0x7fdd3c108720 2024-07-26T13:39:04.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.632+0000 7fdd40897640 1 --2- 172.21.15.100:0/475257917 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fdd2c0b3c20 0x7fdd2c0b6130 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdd30000c00 tx=0x7fdd30031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:04.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.635+0000 7fdd39ffb640 1 -- 172.21.15.100:0/475257917 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fdd340aa6a0 con 0x7fdd3c108720 2024-07-26T13:39:04.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.857+0000 7fdd42afb640 1 -- 172.21.15.100:0/475257917 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fdd3c06aca0 con 0x7fdd3c108720 2024-07-26T13:39:04.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.858+0000 7fdd39ffb640 1 -- 172.21.15.100:0/475257917 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fdd340aebf0 con 0x7fdd3c108720 2024-07-26T13:39:04.859 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:39:04.859 INFO:teuthology.orchestra.run.smithi100.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-07-26T13:39:04.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.860+0000 7fdd42afb640 1 -- 172.21.15.100:0/475257917 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fdd2c0b3c20 msgr2=0x7fdd2c0b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:04.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.860+0000 7fdd42afb640 1 --2- 172.21.15.100:0/475257917 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fdd2c0b3c20 0x7fdd2c0b6130 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdd30000c00 tx=0x7fdd30031400 comp rx=0 tx=0).stop 2024-07-26T13:39:04.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.860+0000 7fdd42afb640 1 -- 172.21.15.100:0/475257917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd3c108720 msgr2=0x7fdd3c199600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:04.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.860+0000 7fdd42afb640 1 --2- 172.21.15.100:0/475257917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd3c108720 0x7fdd3c199600 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdd3400b970 tx=0x7fdd3400be30 comp rx=0 tx=0).stop 2024-07-26T13:39:04.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.860+0000 7fdd42afb640 1 -- 172.21.15.100:0/475257917 shutdown_connections 2024-07-26T13:39:04.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.860+0000 7fdd42afb640 1 --2- 172.21.15.100:0/475257917 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fdd2c0b3c20 0x7fdd2c0b6130 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:04.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.860+0000 7fdd42afb640 1 --2- 172.21.15.100:0/475257917 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd3c108720 0x7fdd3c199600 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:04.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.860+0000 7fdd42afb640 1 --2- 172.21.15.100:0/475257917 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd3c107dd0 0x7fdd3c1990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:04.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.860+0000 7fdd42afb640 1 --2- 172.21.15.100:0/475257917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd3c102490 0x7fdd3c198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:04.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.860+0000 7fdd42afb640 1 -- 172.21.15.100:0/475257917 >> 172.21.15.100:0/475257917 conn(0x7fdd3c079160 msgr2=0x7fdd3c100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:04.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.860+0000 7fdd42afb640 1 -- 172.21.15.100:0/475257917 shutdown_connections 2024-07-26T13:39:04.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:04.860+0000 7fdd42afb640 1 -- 172.21.15.100:0/475257917 wait complete. 2024-07-26T13:39:05.545 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-07-26T13:39:05.546 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:39:05.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:05 smithi100 ceph-mon[32599]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:39:05.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:05 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/475257917' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-07-26T13:39:05.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:05 smithi167 ceph-mon[37140]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:39:05.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:05 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/475257917' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-07-26T13:39:05.807 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:39:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:05 smithi138 ceph-mon[36954]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:39:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:05 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/475257917' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-07-26T13:39:07.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.495+0000 7f6ef871d640 1 -- 172.21.15.100:0/1719081573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ef0107610 msgr2=0x7f6ef0107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:07.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.495+0000 7f6ef871d640 1 --2- 172.21.15.100:0/1719081573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ef0107610 0x7f6ef0107a20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6eec009900 tx=0x7f6eec02a410 comp rx=0 tx=0).stop 2024-07-26T13:39:07.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.495+0000 7f6ef871d640 1 -- 172.21.15.100:0/1719081573 shutdown_connections 2024-07-26T13:39:07.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.495+0000 7f6ef871d640 1 --2- 172.21.15.100:0/1719081573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ef0107f60 0x7f6ef010e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:07.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.495+0000 7f6ef871d640 1 --2- 172.21.15.100:0/1719081573 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ef0107610 0x7f6ef0107a20 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:07.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.495+0000 7f6ef871d640 1 --2- 172.21.15.100:0/1719081573 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ef0101cd0 0x7f6ef01020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:07.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.495+0000 7f6ef871d640 1 -- 172.21.15.100:0/1719081573 >> 172.21.15.100:0/1719081573 conn(0x7f6ef00798f0 msgr2=0x7f6ef0079d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:07.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.495+0000 7f6ef871d640 1 -- 172.21.15.100:0/1719081573 shutdown_connections 2024-07-26T13:39:07.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.495+0000 7f6ef871d640 1 -- 172.21.15.100:0/1719081573 wait complete. 2024-07-26T13:39:07.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.496+0000 7f6ef871d640 1 Processor -- start 2024-07-26T13:39:07.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.496+0000 7f6ef871d640 1 -- start start 2024-07-26T13:39:07.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.496+0000 7f6ef871d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ef0101cd0 0x7f6ef0197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:07.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.496+0000 7f6ef871d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ef0107610 0x7f6ef0198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:07.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.496+0000 7f6ef871d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ef0107f60 0x7f6ef019d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:07.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.496+0000 7f6ef871d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef01113a0 con 0x7f6ef0101cd0 2024-07-26T13:39:07.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.496+0000 7f6ef871d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef0111220 con 0x7f6ef0107610 2024-07-26T13:39:07.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.496+0000 7f6ef871d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef0111520 con 0x7f6ef0107f60 2024-07-26T13:39:07.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.496+0000 7f6ef64b9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ef0101cd0 0x7f6ef0197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:07.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.496+0000 7f6ef64b9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ef0101cd0 0x7f6ef0197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44904/0 (socket says 172.21.15.100:44904) 2024-07-26T13:39:07.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.496+0000 7f6ef64b9640 1 -- 172.21.15.100:0/2818738933 learned_addr learned my addr 172.21.15.100:0/2818738933 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:07.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.497+0000 7f6ef5cb8640 1 --2- 172.21.15.100:0/2818738933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ef0107610 0x7f6ef0198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:07.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.497+0000 7f6ef6cba640 1 --2- 172.21.15.100:0/2818738933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ef0107f60 0x7f6ef019d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:07.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.497+0000 7f6ef64b9640 1 -- 172.21.15.100:0/2818738933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ef0107f60 msgr2=0x7f6ef019d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:07.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.497+0000 7f6ef64b9640 1 --2- 172.21.15.100:0/2818738933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ef0107f60 0x7f6ef019d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:07.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.497+0000 7f6ef64b9640 1 -- 172.21.15.100:0/2818738933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ef0107610 msgr2=0x7f6ef0198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:07.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.497+0000 7f6ef64b9640 1 --2- 172.21.15.100:0/2818738933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ef0107610 0x7f6ef0198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:07.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.497+0000 7f6ef64b9640 1 -- 172.21.15.100:0/2818738933 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef019dd20 con 0x7f6ef0101cd0 2024-07-26T13:39:07.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.497+0000 7f6ef64b9640 1 --2- 172.21.15.100:0/2818738933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ef0101cd0 0x7f6ef0197db0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6ee400edd0 tx=0x7f6ee400a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:07.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.497+0000 7f6ee37fe640 1 -- 172.21.15.100:0/2818738933 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee4004220 con 0x7f6ef0101cd0 2024-07-26T13:39:07.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.497+0000 7f6ee37fe640 1 -- 172.21.15.100:0/2818738933 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ee400ae30 con 0x7f6ef0101cd0 2024-07-26T13:39:07.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.497+0000 7f6ef871d640 1 -- 172.21.15.100:0/2818738933 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ef019e010 con 0x7f6ef0101cd0 2024-07-26T13:39:07.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.497+0000 7f6ee37fe640 1 -- 172.21.15.100:0/2818738933 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee4004d00 con 0x7f6ef0101cd0 2024-07-26T13:39:07.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.497+0000 7f6ef871d640 1 -- 172.21.15.100:0/2818738933 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ef01a4770 con 0x7f6ef0101cd0 2024-07-26T13:39:07.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.498+0000 7f6ef871d640 1 -- 172.21.15.100:0/2818738933 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ef01020e0 con 0x7f6ef0101cd0 2024-07-26T13:39:07.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.500+0000 7f6ee37fe640 1 -- 172.21.15.100:0/2818738933 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f6ee4004ea0 con 0x7f6ef0101cd0 2024-07-26T13:39:07.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.500+0000 7f6ee37fe640 1 --2- 172.21.15.100:0/2818738933 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6edc0b3c20 0x7f6edc0b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:07.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.500+0000 7f6ef5cb8640 1 --2- 172.21.15.100:0/2818738933 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6edc0b3c20 0x7f6edc0b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:07.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.500+0000 7f6ee37fe640 1 -- 172.21.15.100:0/2818738933 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f6ee40e5110 con 0x7f6ef0101cd0 2024-07-26T13:39:07.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.501+0000 7f6ef5cb8640 1 --2- 172.21.15.100:0/2818738933 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6edc0b3c20 0x7f6edc0b6130 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6eec006010 tx=0x7f6eec000bc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:07.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.503+0000 7f6ee37fe640 1 -- 172.21.15.100:0/2818738933 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f6ee40ac6a0 con 0x7f6ef0101cd0 2024-07-26T13:39:07.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.710+0000 7f6ef871d640 1 -- 172.21.15.100:0/2818738933 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ef0198ee0 con 0x7f6ef0101cd0 2024-07-26T13:39:07.713 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:07 smithi100 ceph-mon[32599]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T13:39:07.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.711+0000 7f6ee37fe640 1 -- 172.21.15.100:0/2818738933 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11927 (secure 0 0 0) 0x7f6ee40abfa0 con 0x7f6ef0101cd0 2024-07-26T13:39:07.714 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:39:07.714 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":25,"btime":"2024-07-26T13:37:19:558299+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14790,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/313501409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":313501409},{"type":"v1","addr":"172.21.15.100:6839","nonce":313501409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/1402975506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":1402975506},{"type":"v1","addr":"172.21.15.100:6841","nonce":1402975506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-26T13:37:01.281843+0000","modified":"2024-07-26T13:37:19.272258+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24607,"mds_1":24605,"mds_2":24595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24557":{"gid":24557,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/3290105300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3290105300},{"type":"v1","addr":"172.21.15.138:6835","nonce":3290105300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/3393225483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3393225483},{"type":"v1","addr":"172.21.15.138:6837","nonce":3393225483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24595":{"gid":24595,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6837/3828634468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3828634468},{"type":"v1","addr":"172.21.15.167:6837","nonce":3828634468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.138:6839/1580925471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1580925471},{"type":"v1","addr":"172.21.15.138:6839","nonce":1580925471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/398150514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":398150514},{"type":"v1","addr":"172.21.15.167:6839","nonce":398150514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24607,"qdb_cluster":[24607,24605,24595]},"id":1}]} 2024-07-26T13:39:07.714 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 25 2024-07-26T13:39:07.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.713+0000 7f6ef871d640 1 -- 172.21.15.100:0/2818738933 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6edc0b3c20 msgr2=0x7f6edc0b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:07.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.713+0000 7f6ef871d640 1 --2- 172.21.15.100:0/2818738933 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6edc0b3c20 0x7f6edc0b6130 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6eec006010 tx=0x7f6eec000bc0 comp rx=0 tx=0).stop 2024-07-26T13:39:07.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.713+0000 7f6ef871d640 1 -- 172.21.15.100:0/2818738933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ef0101cd0 msgr2=0x7f6ef0197db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:07.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.713+0000 7f6ef871d640 1 --2- 172.21.15.100:0/2818738933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ef0101cd0 0x7f6ef0197db0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6ee400edd0 tx=0x7f6ee400a720 comp rx=0 tx=0).stop 2024-07-26T13:39:07.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.713+0000 7f6ef871d640 1 -- 172.21.15.100:0/2818738933 shutdown_connections 2024-07-26T13:39:07.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.713+0000 7f6ef871d640 1 --2- 172.21.15.100:0/2818738933 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6edc0b3c20 0x7f6edc0b6130 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:07.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.713+0000 7f6ef871d640 1 --2- 172.21.15.100:0/2818738933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ef0107f60 0x7f6ef019d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:07.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.713+0000 7f6ef871d640 1 --2- 172.21.15.100:0/2818738933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ef0107610 0x7f6ef0198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:07.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.713+0000 7f6ef871d640 1 --2- 172.21.15.100:0/2818738933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ef0101cd0 0x7f6ef0197db0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:07.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.713+0000 7f6ef871d640 1 -- 172.21.15.100:0/2818738933 >> 172.21.15.100:0/2818738933 conn(0x7f6ef00798f0 msgr2=0x7f6ef0105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:07.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.713+0000 7f6ef871d640 1 -- 172.21.15.100:0/2818738933 shutdown_connections 2024-07-26T13:39:07.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:07.713+0000 7f6ef871d640 1 -- 172.21.15.100:0/2818738933 wait complete. 2024-07-26T13:39:07.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:07 smithi167 ceph-mon[37140]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T13:39:07.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:07 smithi138 ceph-mon[36954]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T13:39:08.346 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:39:08.607 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:39:08.730 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:08 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2818738933' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:39:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:08 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/2818738933' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:39:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:08 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/2818738933' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:39:09.742 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:09 smithi100 ceph-mon[32599]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:39:09.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:09 smithi167 ceph-mon[37140]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:39:09.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:09 smithi138 ceph-mon[36954]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:39:10.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.301+0000 7feb795b0640 1 -- 172.21.15.100:0/3056137166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb74109840 msgr2=0x7feb74110070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:10.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.301+0000 7feb795b0640 1 --2- 172.21.15.100:0/3056137166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb74109840 0x7feb74110070 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7feb64009900 tx=0x7feb6402a410 comp rx=0 tx=0).stop 2024-07-26T13:39:10.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.301+0000 7feb795b0640 1 -- 172.21.15.100:0/3056137166 shutdown_connections 2024-07-26T13:39:10.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.301+0000 7feb795b0640 1 --2- 172.21.15.100:0/3056137166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb74109840 0x7feb74110070 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:10.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.301+0000 7feb795b0640 1 --2- 172.21.15.100:0/3056137166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb74108ef0 0x7feb74109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:10.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.301+0000 7feb795b0640 1 --2- 172.21.15.100:0/3056137166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb741035b0 0x7feb741039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:10.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.301+0000 7feb795b0640 1 -- 172.21.15.100:0/3056137166 >> 172.21.15.100:0/3056137166 conn(0x7feb740ff000 msgr2=0x7feb74101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:10.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.301+0000 7feb795b0640 1 -- 172.21.15.100:0/3056137166 shutdown_connections 2024-07-26T13:39:10.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.301+0000 7feb795b0640 1 -- 172.21.15.100:0/3056137166 wait complete. 2024-07-26T13:39:10.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.302+0000 7feb795b0640 1 Processor -- start 2024-07-26T13:39:10.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.302+0000 7feb795b0640 1 -- start start 2024-07-26T13:39:10.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.302+0000 7feb795b0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb741035b0 0x7feb74199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:10.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.302+0000 7feb795b0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb74108ef0 0x7feb7419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:10.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.302+0000 7feb795b0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb74109840 0x7feb7419f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:10.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.302+0000 7feb795b0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7feb74113540 con 0x7feb74109840 2024-07-26T13:39:10.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.302+0000 7feb795b0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feb741133c0 con 0x7feb741035b0 2024-07-26T13:39:10.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.302+0000 7feb795b0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7feb741136c0 con 0x7feb74108ef0 2024-07-26T13:39:10.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.302+0000 7feb737fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb74109840 0x7feb7419f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:10.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.302+0000 7feb737fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb74109840 0x7feb7419f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44930/0 (socket says 172.21.15.100:44930) 2024-07-26T13:39:10.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.302+0000 7feb737fe640 1 -- 172.21.15.100:0/974987241 learned_addr learned my addr 172.21.15.100:0/974987241 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:10.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.303+0000 7feb727fc640 1 --2- 172.21.15.100:0/974987241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb74108ef0 0x7feb7419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:10.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.303+0000 7feb72ffd640 1 --2- 172.21.15.100:0/974987241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb741035b0 0x7feb74199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:10.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.303+0000 7feb737fe640 1 -- 172.21.15.100:0/974987241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb74108ef0 msgr2=0x7feb7419a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:10.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.303+0000 7feb737fe640 1 --2- 172.21.15.100:0/974987241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb74108ef0 0x7feb7419a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:10.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.303+0000 7feb737fe640 1 -- 172.21.15.100:0/974987241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb741035b0 msgr2=0x7feb74199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:10.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.303+0000 7feb737fe640 1 --2- 172.21.15.100:0/974987241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb741035b0 0x7feb74199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:10.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.303+0000 7feb737fe640 1 -- 172.21.15.100:0/974987241 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb7419fee0 con 0x7feb74109840 2024-07-26T13:39:10.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.303+0000 7feb737fe640 1 --2- 172.21.15.100:0/974987241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb74109840 0x7feb7419f7b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7feb64004770 tx=0x7feb64004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:10.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.303+0000 7feb6bfff640 1 -- 172.21.15.100:0/974987241 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6403f030 con 0x7feb74109840 2024-07-26T13:39:10.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.303+0000 7feb6bfff640 1 -- 172.21.15.100:0/974987241 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb64003aa0 con 0x7feb74109840 2024-07-26T13:39:10.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.303+0000 7feb795b0640 1 -- 172.21.15.100:0/974987241 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb741a0170 con 0x7feb74109840 2024-07-26T13:39:10.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.303+0000 7feb6bfff640 1 -- 172.21.15.100:0/974987241 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb64035070 con 0x7feb74109840 2024-07-26T13:39:10.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.303+0000 7feb795b0640 1 -- 172.21.15.100:0/974987241 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb741a6930 con 0x7feb74109840 2024-07-26T13:39:10.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.304+0000 7feb795b0640 1 -- 172.21.15.100:0/974987241 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb74104830 con 0x7feb74109840 2024-07-26T13:39:10.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.306+0000 7feb6bfff640 1 -- 172.21.15.100:0/974987241 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7feb64042030 con 0x7feb74109840 2024-07-26T13:39:10.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.307+0000 7feb6bfff640 1 --2- 172.21.15.100:0/974987241 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7feb600b3c20 0x7feb600b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:10.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.307+0000 7feb72ffd640 1 --2- 172.21.15.100:0/974987241 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7feb600b3c20 0x7feb600b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:10.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.307+0000 7feb6bfff640 1 -- 172.21.15.100:0/974987241 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7feb64104700 con 0x7feb74109840 2024-07-26T13:39:10.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.307+0000 7feb72ffd640 1 --2- 172.21.15.100:0/974987241 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7feb600b3c20 0x7feb600b6130 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7feb6c005fd0 tx=0x7feb6c009870 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:10.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.310+0000 7feb6bfff640 1 -- 172.21.15.100:0/974987241 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7feb640cbc50 con 0x7feb74109840 2024-07-26T13:39:10.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.520+0000 7feb795b0640 1 -- 172.21.15.100:0/974987241 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb741039c0 con 0x7feb74109840 2024-07-26T13:39:10.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.521+0000 7feb6bfff640 1 -- 172.21.15.100:0/974987241 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11927 (secure 0 0 0) 0x7feb741039c0 con 0x7feb74109840 2024-07-26T13:39:10.522 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:39:10.522 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":25,"btime":"2024-07-26T13:37:19:558299+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14790,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/313501409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":313501409},{"type":"v1","addr":"172.21.15.100:6839","nonce":313501409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/1402975506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":1402975506},{"type":"v1","addr":"172.21.15.100:6841","nonce":1402975506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-26T13:37:01.281843+0000","modified":"2024-07-26T13:37:19.272258+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24607,"mds_1":24605,"mds_2":24595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24557":{"gid":24557,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/3290105300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3290105300},{"type":"v1","addr":"172.21.15.138:6835","nonce":3290105300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/3393225483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3393225483},{"type":"v1","addr":"172.21.15.138:6837","nonce":3393225483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24595":{"gid":24595,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6837/3828634468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3828634468},{"type":"v1","addr":"172.21.15.167:6837","nonce":3828634468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.138:6839/1580925471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1580925471},{"type":"v1","addr":"172.21.15.138:6839","nonce":1580925471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/398150514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":398150514},{"type":"v1","addr":"172.21.15.167:6839","nonce":398150514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24607,"qdb_cluster":[24607,24605,24595]},"id":1}]} 2024-07-26T13:39:10.522 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 25 2024-07-26T13:39:10.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.523+0000 7feb795b0640 1 -- 172.21.15.100:0/974987241 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7feb600b3c20 msgr2=0x7feb600b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:10.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.523+0000 7feb795b0640 1 --2- 172.21.15.100:0/974987241 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7feb600b3c20 0x7feb600b6130 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7feb6c005fd0 tx=0x7feb6c009870 comp rx=0 tx=0).stop 2024-07-26T13:39:10.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.523+0000 7feb795b0640 1 -- 172.21.15.100:0/974987241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb74109840 msgr2=0x7feb7419f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:10.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.523+0000 7feb795b0640 1 --2- 172.21.15.100:0/974987241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb74109840 0x7feb7419f7b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7feb64004770 tx=0x7feb64004270 comp rx=0 tx=0).stop 2024-07-26T13:39:10.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.523+0000 7feb795b0640 1 -- 172.21.15.100:0/974987241 shutdown_connections 2024-07-26T13:39:10.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.523+0000 7feb795b0640 1 --2- 172.21.15.100:0/974987241 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7feb600b3c20 0x7feb600b6130 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:10.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.523+0000 7feb795b0640 1 --2- 172.21.15.100:0/974987241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb74109840 0x7feb7419f7b0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:10.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.523+0000 7feb795b0640 1 --2- 172.21.15.100:0/974987241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb74108ef0 0x7feb7419a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:10.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.523+0000 7feb795b0640 1 --2- 172.21.15.100:0/974987241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb741035b0 0x7feb74199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:10.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.523+0000 7feb795b0640 1 -- 172.21.15.100:0/974987241 >> 172.21.15.100:0/974987241 conn(0x7feb740ff000 msgr2=0x7feb74101420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:10.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.523+0000 7feb795b0640 1 -- 172.21.15.100:0/974987241 shutdown_connections 2024-07-26T13:39:10.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:10.523+0000 7feb795b0640 1 -- 172.21.15.100:0/974987241 wait complete. 2024-07-26T13:39:11.226 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph osd dump --format=json 2024-07-26T13:39:11.483 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:39:11.733 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:11 smithi100 ceph-mon[32599]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:39:11.733 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:11 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/974987241' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:39:11.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:11 smithi167 ceph-mon[37140]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:39:11.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:11 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/974987241' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:39:11.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:11 smithi138 ceph-mon[36954]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:39:11.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:11 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/974987241' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:39:13.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.136+0000 7fba31ae4640 1 -- 172.21.15.100:0/148472394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba2c107e50 msgr2=0x7fba2c108260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:13.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.136+0000 7fba31ae4640 1 --2- 172.21.15.100:0/148472394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba2c107e50 0x7fba2c108260 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fba14009900 tx=0x7fba1402a410 comp rx=0 tx=0).stop 2024-07-26T13:39:13.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.136+0000 7fba31ae4640 1 -- 172.21.15.100:0/148472394 shutdown_connections 2024-07-26T13:39:13.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.136+0000 7fba31ae4640 1 --2- 172.21.15.100:0/148472394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba2c1087a0 0x7fba2c10efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:13.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.136+0000 7fba31ae4640 1 --2- 172.21.15.100:0/148472394 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba2c107e50 0x7fba2c108260 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:13.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.136+0000 7fba31ae4640 1 --2- 172.21.15.100:0/148472394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba2c102510 0x7fba2c102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:13.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.136+0000 7fba31ae4640 1 -- 172.21.15.100:0/148472394 >> 172.21.15.100:0/148472394 conn(0x7fba2c0790e0 msgr2=0x7fba2c0794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:13.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.136+0000 7fba31ae4640 1 -- 172.21.15.100:0/148472394 shutdown_connections 2024-07-26T13:39:13.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.136+0000 7fba31ae4640 1 -- 172.21.15.100:0/148472394 wait complete. 2024-07-26T13:39:13.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.136+0000 7fba31ae4640 1 Processor -- start 2024-07-26T13:39:13.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.136+0000 7fba31ae4640 1 -- start start 2024-07-26T13:39:13.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.137+0000 7fba31ae4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba2c102510 0x7fba2c19c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:13.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.137+0000 7fba31ae4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba2c107e50 0x7fba2c19c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:13.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.137+0000 7fba31ae4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba2c1087a0 0x7fba2c1a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:13.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.137+0000 7fba31ae4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fba2c112410 con 0x7fba2c102510 2024-07-26T13:39:13.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.137+0000 7fba31ae4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fba2c112290 con 0x7fba2c107e50 2024-07-26T13:39:13.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.137+0000 7fba31ae4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fba2c112590 con 0x7fba2c1087a0 2024-07-26T13:39:13.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.137+0000 7fba2b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba2c102510 0x7fba2c19c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:13.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.137+0000 7fba2b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba2c102510 0x7fba2c19c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44958/0 (socket says 172.21.15.100:44958) 2024-07-26T13:39:13.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.137+0000 7fba2b7fe640 1 -- 172.21.15.100:0/3423577002 learned_addr learned my addr 172.21.15.100:0/3423577002 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:13.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.137+0000 7fba22ffd640 1 --2- 172.21.15.100:0/3423577002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba2c107e50 0x7fba2c19c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:13.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.137+0000 7fba2bfff640 1 --2- 172.21.15.100:0/3423577002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba2c1087a0 0x7fba2c1a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:13.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.137+0000 7fba2b7fe640 1 -- 172.21.15.100:0/3423577002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba2c1087a0 msgr2=0x7fba2c1a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:13.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.137+0000 7fba2b7fe640 1 --2- 172.21.15.100:0/3423577002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba2c1087a0 0x7fba2c1a1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:13.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.137+0000 7fba2b7fe640 1 -- 172.21.15.100:0/3423577002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba2c107e50 msgr2=0x7fba2c19c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:13.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.137+0000 7fba2b7fe640 1 --2- 172.21.15.100:0/3423577002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba2c107e50 0x7fba2c19c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:13.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.137+0000 7fba2b7fe640 1 -- 172.21.15.100:0/3423577002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba2c1a2200 con 0x7fba2c102510 2024-07-26T13:39:13.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.137+0000 7fba2b7fe640 1 --2- 172.21.15.100:0/3423577002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba2c102510 0x7fba2c19c290 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fba1c00d950 tx=0x7fba1c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:13.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.138+0000 7fba297fa640 1 -- 172.21.15.100:0/3423577002 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c017070 con 0x7fba2c102510 2024-07-26T13:39:13.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.138+0000 7fba31ae4640 1 -- 172.21.15.100:0/3423577002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba2c1a24f0 con 0x7fba2c102510 2024-07-26T13:39:13.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.138+0000 7fba297fa640 1 -- 172.21.15.100:0/3423577002 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba1c009c30 con 0x7fba2c102510 2024-07-26T13:39:13.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.138+0000 7fba297fa640 1 -- 172.21.15.100:0/3423577002 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c00f620 con 0x7fba2c102510 2024-07-26T13:39:13.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.138+0000 7fba31ae4640 1 -- 172.21.15.100:0/3423577002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba2c1a8c50 con 0x7fba2c102510 2024-07-26T13:39:13.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.139+0000 7fba31ae4640 1 -- 172.21.15.100:0/3423577002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba2c103790 con 0x7fba2c102510 2024-07-26T13:39:13.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.140+0000 7fba297fa640 1 -- 172.21.15.100:0/3423577002 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7fba1c005030 con 0x7fba2c102510 2024-07-26T13:39:13.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.141+0000 7fba297fa640 1 --2- 172.21.15.100:0/3423577002 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fba180b3c20 0x7fba180b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:13.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.141+0000 7fba22ffd640 1 --2- 172.21.15.100:0/3423577002 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fba180b3c20 0x7fba180b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:13.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.141+0000 7fba297fa640 1 -- 172.21.15.100:0/3423577002 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fba1c0e3a20 con 0x7fba2c102510 2024-07-26T13:39:13.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.141+0000 7fba22ffd640 1 --2- 172.21.15.100:0/3423577002 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fba180b3c20 0x7fba180b6130 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fba14009230 tx=0x7fba14031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:13.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.144+0000 7fba297fa640 1 -- 172.21.15.100:0/3423577002 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fba1c0aaf70 con 0x7fba2c102510 2024-07-26T13:39:13.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.320+0000 7fba31ae4640 1 -- 172.21.15.100:0/3423577002 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fba2c1006b0 con 0x7fba2c102510 2024-07-26T13:39:13.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.322+0000 7fba297fa640 1 -- 172.21.15.100:0/3423577002 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+23121 (secure 0 0 0) 0x7fba1c009de0 con 0x7fba2c102510 2024-07-26T13:39:13.323 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:39:13.323 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":81,"fsid":"bf83e2e6-4b52-11ef-bcb7-c7b262605968","created":"2024-07-26T13:28:23.639785+0000","modified":"2024-07-26T13:37:17.539521+0000","last_up_change":"2024-07-26T13:35:56.475554+0000","last_in_change":"2024-07-26T13:35:37.472564+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-26T13:32:22.329969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-26T13:36:54.590525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-26T13:36:58.149545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d714036d-9dda-4cb5-af16-29f83a09dd70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6803","nonce":1480401850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6805","nonce":1480401850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6809","nonce":1480401850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6807","nonce":1480401850}]},"public_addr":"172.21.15.100:6803/1480401850","cluster_addr":"172.21.15.100:6805/1480401850","heartbeat_back_addr":"172.21.15.100:6809/1480401850","heartbeat_front_addr":"172.21.15.100:6807/1480401850","state":["exists","up"]},{"osd":1,"uuid":"3dd0eaa1-618a-495a-9819-e01c0f869ae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6801","nonce":3090877671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6803","nonce":3090877671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6807","nonce":3090877671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6805","nonce":3090877671}]},"public_addr":"172.21.15.138:6801/3090877671","cluster_addr":"172.21.15.138:6803/3090877671","heartbeat_back_addr":"172.21.15.138:6807/3090877671","heartbeat_front_addr":"172.21.15.138:6805/3090877671","state":["exists","up"]},{"osd":2,"uuid":"5c710a78-17ae-41a8-ac16-a678f5d35672","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6801","nonce":3583424790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6803","nonce":3583424790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6807","nonce":3583424790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6805","nonce":3583424790}]},"public_addr":"172.21.15.167:6801/3583424790","cluster_addr":"172.21.15.167:6803/3583424790","heartbeat_back_addr":"172.21.15.167:6807/3583424790","heartbeat_front_addr":"172.21.15.167:6805/3583424790","state":["exists","up"]},{"osd":3,"uuid":"be845495-8c31-4834-a45c-bdc576711962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6811","nonce":3626762419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6813","nonce":3626762419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6817","nonce":3626762419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6815","nonce":3626762419}]},"public_addr":"172.21.15.100:6811/3626762419","cluster_addr":"172.21.15.100:6813/3626762419","heartbeat_back_addr":"172.21.15.100:6817/3626762419","heartbeat_front_addr":"172.21.15.100:6815/3626762419","state":["exists","up"]},{"osd":4,"uuid":"520554cc-e846-46f9-af77-05a1de65a452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6809","nonce":3757334356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6811","nonce":3757334356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6815","nonce":3757334356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6813","nonce":3757334356}]},"public_addr":"172.21.15.138:6809/3757334356","cluster_addr":"172.21.15.138:6811/3757334356","heartbeat_back_addr":"172.21.15.138:6815/3757334356","heartbeat_front_addr":"172.21.15.138:6813/3757334356","state":["exists","up"]},{"osd":5,"uuid":"68dc8ad6-31d2-45b5-9aa7-62c4218d46e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6809","nonce":3239482836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6811","nonce":3239482836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6815","nonce":3239482836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6813","nonce":3239482836}]},"public_addr":"172.21.15.167:6809/3239482836","cluster_addr":"172.21.15.167:6811/3239482836","heartbeat_back_addr":"172.21.15.167:6815/3239482836","heartbeat_front_addr":"172.21.15.167:6813/3239482836","state":["exists","up"]},{"osd":6,"uuid":"420e4471-5626-41cc-a8e1-69c801196b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6819","nonce":3218415484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6821","nonce":3218415484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6825","nonce":3218415484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6823","nonce":3218415484}]},"public_addr":"172.21.15.100:6819/3218415484","cluster_addr":"172.21.15.100:6821/3218415484","heartbeat_back_addr":"172.21.15.100:6825/3218415484","heartbeat_front_addr":"172.21.15.100:6823/3218415484","state":["exists","up"]},{"osd":7,"uuid":"a27b097e-207d-44c3-af63-544f228b83ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6817","nonce":45541749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6819","nonce":45541749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6823","nonce":45541749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6821","nonce":45541749}]},"public_addr":"172.21.15.138:6817/45541749","cluster_addr":"172.21.15.138:6819/45541749","heartbeat_back_addr":"172.21.15.138:6823/45541749","heartbeat_front_addr":"172.21.15.138:6821/45541749","state":["exists","up"]},{"osd":8,"uuid":"3e865858-92cc-48a0-a464-5767d19e8ac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6817","nonce":4031159803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6819","nonce":4031159803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6823","nonce":4031159803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6821","nonce":4031159803}]},"public_addr":"172.21.15.167:6817/4031159803","cluster_addr":"172.21.15.167:6819/4031159803","heartbeat_back_addr":"172.21.15.167:6823/4031159803","heartbeat_front_addr":"172.21.15.167:6821/4031159803","state":["exists","up"]},{"osd":9,"uuid":"8b05b14a-1195-42f5-ac01-d5634cb9c888","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6827","nonce":1840140273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6829","nonce":1840140273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6833","nonce":1840140273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6831","nonce":1840140273}]},"public_addr":"172.21.15.100:6827/1840140273","cluster_addr":"172.21.15.100:6829/1840140273","heartbeat_back_addr":"172.21.15.100:6833/1840140273","heartbeat_front_addr":"172.21.15.100:6831/1840140273","state":["exists","up"]},{"osd":10,"uuid":"ab77ebe7-5254-4dc3-bc54-625698e84afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6825","nonce":703382899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6827","nonce":703382899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6831","nonce":703382899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6829","nonce":703382899}]},"public_addr":"172.21.15.138:6825/703382899","cluster_addr":"172.21.15.138:6827/703382899","heartbeat_back_addr":"172.21.15.138:6831/703382899","heartbeat_front_addr":"172.21.15.138:6829/703382899","state":["exists","up"]},{"osd":11,"uuid":"229cf91e-e9ef-48f6-a677-c5c5705b315a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6825","nonce":1032343068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6827","nonce":1032343068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6831","nonce":1032343068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6829","nonce":1032343068}]},"public_addr":"172.21.15.167:6825/1032343068","cluster_addr":"172.21.15.167:6827/1032343068","heartbeat_back_addr":"172.21.15.167:6831/1032343068","heartbeat_front_addr":"172.21.15.167:6829/1032343068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:31:35.800702+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:31:58.087922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:32:20.463183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:32:42.920598+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:33:06.332655+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:33:29.381407+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:33:53.975963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:34:18.226813+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:34:41.845770+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:35:05.722275+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:35:30.862704+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:35:54.738697+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:0/1874993112":"2024-07-27T13:30:16.253096+0000","172.21.15.100:0/2364620305":"2024-07-27T13:28:45.340334+0000","172.21.15.100:6800/3096076075":"2024-07-27T13:29:07.979178+0000","172.21.15.100:0/1894966501":"2024-07-27T13:30:16.253096+0000","172.21.15.100:0/3411776672":"2024-07-27T13:28:45.340334+0000","172.21.15.100:0/2853423120":"2024-07-27T13:29:07.979178+0000","172.21.15.100:6800/3025873570":"2024-07-27T13:28:45.340334+0000","172.21.15.100:0/4887181":"2024-07-27T13:29:07.979178+0000","172.21.15.100:6801/908675140":"2024-07-27T13:30:16.253096+0000","172.21.15.100:6801/3025873570":"2024-07-27T13:28:45.340334+0000","172.21.15.100:6800/908675140":"2024-07-27T13:30:16.253096+0000","172.21.15.100:6801/3096076075":"2024-07-27T13:29:07.979178+0000","172.21.15.100:0/2302632052":"2024-07-27T13:29:07.979178+0000","172.21.15.100:0/3736098915":"2024-07-27T13:28:45.340334+0000","172.21.15.100:0/3243030560":"2024-07-27T13:30:16.253096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-26T13:39:13.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.324+0000 7fba31ae4640 1 -- 172.21.15.100:0/3423577002 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fba180b3c20 msgr2=0x7fba180b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:13.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.324+0000 7fba31ae4640 1 --2- 172.21.15.100:0/3423577002 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fba180b3c20 0x7fba180b6130 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fba14009230 tx=0x7fba14031400 comp rx=0 tx=0).stop 2024-07-26T13:39:13.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.324+0000 7fba31ae4640 1 -- 172.21.15.100:0/3423577002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba2c102510 msgr2=0x7fba2c19c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:13.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.324+0000 7fba31ae4640 1 --2- 172.21.15.100:0/3423577002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba2c102510 0x7fba2c19c290 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fba1c00d950 tx=0x7fba1c00de10 comp rx=0 tx=0).stop 2024-07-26T13:39:13.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.324+0000 7fba31ae4640 1 -- 172.21.15.100:0/3423577002 shutdown_connections 2024-07-26T13:39:13.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.324+0000 7fba31ae4640 1 --2- 172.21.15.100:0/3423577002 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fba180b3c20 0x7fba180b6130 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:13.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.324+0000 7fba31ae4640 1 --2- 172.21.15.100:0/3423577002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba2c1087a0 0x7fba2c1a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:13.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.324+0000 7fba31ae4640 1 --2- 172.21.15.100:0/3423577002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba2c107e50 0x7fba2c19c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:13.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.324+0000 7fba31ae4640 1 --2- 172.21.15.100:0/3423577002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba2c102510 0x7fba2c19c290 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:13.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.324+0000 7fba31ae4640 1 -- 172.21.15.100:0/3423577002 >> 172.21.15.100:0/3423577002 conn(0x7fba2c0790e0 msgr2=0x7fba2c06b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:13.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.324+0000 7fba31ae4640 1 -- 172.21.15.100:0/3423577002 shutdown_connections 2024-07-26T13:39:13.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:13.324+0000 7fba31ae4640 1 -- 172.21.15.100:0/3423577002 wait complete. 2024-07-26T13:39:13.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:13 smithi100 ceph-mon[32599]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:39:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:13 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/3423577002' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-26T13:39:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:13 smithi167 ceph-mon[37140]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:39:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:13 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/3423577002' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-26T13:39:13.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:13 smithi138 ceph-mon[36954]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:39:13.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:13 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/3423577002' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-26T13:39:13.990 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:39:14.245 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:39:15.615 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:15 smithi100 ceph-mon[32599]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:39:15.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:15 smithi167 ceph-mon[37140]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:39:15.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.838+0000 7f6db4df8640 1 -- 172.21.15.100:0/3843239082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6db0101b70 msgr2=0x7f6db0101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:15.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.838+0000 7f6db4df8640 1 --2- 172.21.15.100:0/3843239082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6db0101b70 0x7f6db0101f80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009900 tx=0x7f6da002a410 comp rx=0 tx=0).stop 2024-07-26T13:39:15.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.838+0000 7f6db4df8640 1 -- 172.21.15.100:0/3843239082 shutdown_connections 2024-07-26T13:39:15.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.838+0000 7f6db4df8640 1 --2- 172.21.15.100:0/3843239082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6db01066b0 0x7f6db0110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:15.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.838+0000 7f6db4df8640 1 --2- 172.21.15.100:0/3843239082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6db0105d60 0x7f6db0106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:15.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.838+0000 7f6db4df8640 1 --2- 172.21.15.100:0/3843239082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6db0101b70 0x7f6db0101f80 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:15.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.838+0000 7f6db4df8640 1 -- 172.21.15.100:0/3843239082 >> 172.21.15.100:0/3843239082 conn(0x7f6db00798d0 msgr2=0x7f6db0079ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:15.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.838+0000 7f6db4df8640 1 -- 172.21.15.100:0/3843239082 shutdown_connections 2024-07-26T13:39:15.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.838+0000 7f6db4df8640 1 -- 172.21.15.100:0/3843239082 wait complete. 2024-07-26T13:39:15.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.839+0000 7f6db4df8640 1 Processor -- start 2024-07-26T13:39:15.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.839+0000 7f6db4df8640 1 -- start start 2024-07-26T13:39:15.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.839+0000 7f6db4df8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6db0101b70 0x7f6db0102b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:15.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.839+0000 7f6db4df8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6db0105d60 0x7f6db0103050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:15.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.839+0000 7f6db4df8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6db01066b0 0x7f6db0103590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:15.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.839+0000 7f6db4df8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6db0113570 con 0x7f6db0101b70 2024-07-26T13:39:15.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.839+0000 7f6db4df8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6db01133f0 con 0x7f6db0105d60 2024-07-26T13:39:15.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.839+0000 7f6db4df8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6db01136f0 con 0x7f6db01066b0 2024-07-26T13:39:15.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.839+0000 7f6dae59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6db0101b70 0x7f6db0102b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:15.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.839+0000 7f6dae59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6db0101b70 0x7f6db0102b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60374/0 (socket says 172.21.15.100:60374) 2024-07-26T13:39:15.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.839+0000 7f6dae59c640 1 -- 172.21.15.100:0/3092618540 learned_addr learned my addr 172.21.15.100:0/3092618540 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:15.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.839+0000 7f6dadd9b640 1 --2- 172.21.15.100:0/3092618540 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6db0105d60 0x7f6db0103050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:15.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.839+0000 7f6dae59c640 1 -- 172.21.15.100:0/3092618540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6db01066b0 msgr2=0x7f6db0103590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:15.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.839+0000 7f6daed9d640 1 --2- 172.21.15.100:0/3092618540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6db01066b0 0x7f6db0103590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:15.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.839+0000 7f6dae59c640 1 --2- 172.21.15.100:0/3092618540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6db01066b0 0x7f6db0103590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:15.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.840+0000 7f6dae59c640 1 -- 172.21.15.100:0/3092618540 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6db0105d60 msgr2=0x7f6db0103050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:15.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.840+0000 7f6dae59c640 1 --2- 172.21.15.100:0/3092618540 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6db0105d60 0x7f6db0103050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:15.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.840+0000 7f6dae59c640 1 -- 172.21.15.100:0/3092618540 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db01a8260 con 0x7f6db0101b70 2024-07-26T13:39:15.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.840+0000 7f6dadd9b640 1 --2- 172.21.15.100:0/3092618540 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6db0105d60 0x7f6db0103050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:39:15.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.840+0000 7f6daed9d640 1 --2- 172.21.15.100:0/3092618540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6db01066b0 0x7f6db0103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:39:15.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.840+0000 7f6dae59c640 1 --2- 172.21.15.100:0/3092618540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6db0101b70 0x7f6db0102b10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6da0006010 tx=0x7f6da00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:15.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.840+0000 7f6d9f7fe640 1 -- 172.21.15.100:0/3092618540 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da003f030 con 0x7f6db0101b70 2024-07-26T13:39:15.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.840+0000 7f6db4df8640 1 -- 172.21.15.100:0/3092618540 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6db01a84f0 con 0x7f6db0101b70 2024-07-26T13:39:15.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.840+0000 7f6d9f7fe640 1 -- 172.21.15.100:0/3092618540 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6da0003800 con 0x7f6db0101b70 2024-07-26T13:39:15.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.840+0000 7f6d9f7fe640 1 -- 172.21.15.100:0/3092618540 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da0035070 con 0x7f6db0101b70 2024-07-26T13:39:15.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.840+0000 7f6db4df8640 1 -- 172.21.15.100:0/3092618540 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6db01a89d0 con 0x7f6db0101b70 2024-07-26T13:39:15.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.841+0000 7f6db4df8640 1 -- 172.21.15.100:0/3092618540 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6db0101f80 con 0x7f6db0101b70 2024-07-26T13:39:15.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.842+0000 7f6d9f7fe640 1 -- 172.21.15.100:0/3092618540 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f6da0003cb0 con 0x7f6db0101b70 2024-07-26T13:39:15.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.843+0000 7f6d9f7fe640 1 --2- 172.21.15.100:0/3092618540 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6d980b3c20 0x7f6d980b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:15.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.843+0000 7f6dadd9b640 1 --2- 172.21.15.100:0/3092618540 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6d980b3c20 0x7f6d980b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:15.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.843+0000 7f6d9f7fe640 1 -- 172.21.15.100:0/3092618540 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f6da0104a30 con 0x7f6db0101b70 2024-07-26T13:39:15.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.844+0000 7f6dadd9b640 1 --2- 172.21.15.100:0/3092618540 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6d980b3c20 0x7f6d980b6130 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6db006ab90 tx=0x7f6da4005d30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:15.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:15.846+0000 7f6d9f7fe640 1 -- 172.21.15.100:0/3092618540 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f6da00cbf80 con 0x7f6db0101b70 2024-07-26T13:39:15.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:15 smithi138 ceph-mon[36954]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:39:16.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:16.055+0000 7f6db4df8640 1 -- 172.21.15.100:0/3092618540 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6db0103e10 con 0x7f6db0101b70 2024-07-26T13:39:16.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:16.056+0000 7f6d9f7fe640 1 -- 172.21.15.100:0/3092618540 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11927 (secure 0 0 0) 0x7f6da00cb880 con 0x7f6db0101b70 2024-07-26T13:39:16.057 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:39:16.057 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":25,"btime":"2024-07-26T13:37:19:558299+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14790,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/313501409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":313501409},{"type":"v1","addr":"172.21.15.100:6839","nonce":313501409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/1402975506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":1402975506},{"type":"v1","addr":"172.21.15.100:6841","nonce":1402975506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-26T13:37:01.281843+0000","modified":"2024-07-26T13:37:19.272258+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24607,"mds_1":24605,"mds_2":24595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24557":{"gid":24557,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/3290105300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3290105300},{"type":"v1","addr":"172.21.15.138:6835","nonce":3290105300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/3393225483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3393225483},{"type":"v1","addr":"172.21.15.138:6837","nonce":3393225483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24595":{"gid":24595,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6837/3828634468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3828634468},{"type":"v1","addr":"172.21.15.167:6837","nonce":3828634468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.138:6839/1580925471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1580925471},{"type":"v1","addr":"172.21.15.138:6839","nonce":1580925471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/398150514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":398150514},{"type":"v1","addr":"172.21.15.167:6839","nonce":398150514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24607,"qdb_cluster":[24607,24605,24595]},"id":1}]} 2024-07-26T13:39:16.057 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 25 2024-07-26T13:39:16.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:16.058+0000 7f6db4df8640 1 -- 172.21.15.100:0/3092618540 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6d980b3c20 msgr2=0x7f6d980b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:16.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:16.058+0000 7f6db4df8640 1 --2- 172.21.15.100:0/3092618540 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6d980b3c20 0x7f6d980b6130 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6db006ab90 tx=0x7f6da4005d30 comp rx=0 tx=0).stop 2024-07-26T13:39:16.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:16.058+0000 7f6db4df8640 1 -- 172.21.15.100:0/3092618540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6db0101b70 msgr2=0x7f6db0102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:16.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:16.058+0000 7f6db4df8640 1 --2- 172.21.15.100:0/3092618540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6db0101b70 0x7f6db0102b10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6da0006010 tx=0x7f6da00042f0 comp rx=0 tx=0).stop 2024-07-26T13:39:16.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:16.058+0000 7f6db4df8640 1 -- 172.21.15.100:0/3092618540 shutdown_connections 2024-07-26T13:39:16.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:16.058+0000 7f6db4df8640 1 --2- 172.21.15.100:0/3092618540 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6d980b3c20 0x7f6d980b6130 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:16.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:16.058+0000 7f6db4df8640 1 --2- 172.21.15.100:0/3092618540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6db01066b0 0x7f6db0103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:16.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:16.058+0000 7f6db4df8640 1 --2- 172.21.15.100:0/3092618540 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6db0105d60 0x7f6db0103050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:16.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:16.058+0000 7f6db4df8640 1 --2- 172.21.15.100:0/3092618540 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6db0101b70 0x7f6db0102b10 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:16.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:16.058+0000 7f6db4df8640 1 -- 172.21.15.100:0/3092618540 >> 172.21.15.100:0/3092618540 conn(0x7f6db00798d0 msgr2=0x7f6db0107be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:16.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:16.058+0000 7f6db4df8640 1 -- 172.21.15.100:0/3092618540 shutdown_connections 2024-07-26T13:39:16.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:16.058+0000 7f6db4df8640 1 -- 172.21.15.100:0/3092618540 wait complete. 2024-07-26T13:39:16.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:16 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/3092618540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:39:16.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:16 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/3092618540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:39:16.776 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-07-26T13:37:01.281843+0000', 'modified': '2024-07-26T13:37:19.272258+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24607, 'mds_1': 24605, 'mds_2': 24595}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24557': {'gid': 24557, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.138:6835/3290105300', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6834', 'nonce': 3290105300}, {'type': 'v1', 'addr': '172.21.15.138:6835', 'nonce': 3290105300}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24575': {'gid': 24575, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.138:6837/3393225483', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6836', 'nonce': 3393225483}, {'type': 'v1', 'addr': '172.21.15.138:6837', 'nonce': 3393225483}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24586': {'gid': 24586, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6835/641259145', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6834', 'nonce': 641259145}, {'type': 'v1', 'addr': '172.21.15.167:6835', 'nonce': 641259145}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24595': {'gid': 24595, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.167:6837/3828634468', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6836', 'nonce': 3828634468}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 3828634468}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24605': {'gid': 24605, 'name': 'b', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.138:6839/1580925471', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6838', 'nonce': 1580925471}, {'type': 'v1', 'addr': '172.21.15.138:6839', 'nonce': 1580925471}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24607': {'gid': 24607, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.167:6839/398150514', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6838', 'nonce': 398150514}, {'type': 'v1', 'addr': '172.21.15.167:6839', 'nonce': 398150514}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24607, 'qdb_cluster': [24607, 24605, 24595]} 2024-07-26T13:39:16.776 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-07-26T13:39:16.776 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.c status 2024-07-26T13:39:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:16 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/3092618540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:39:17.029 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:39:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:17 smithi100 ceph-mon[32599]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T13:39:17.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:17 smithi167 ceph-mon[37140]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T13:39:17.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:17 smithi138 ceph-mon[36954]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T13:39:18.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.708+0000 7fb307ddf640 1 -- 172.21.15.100:0/4139510247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb300109820 msgr2=0x7fb300110050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:18.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.708+0000 7fb307ddf640 1 --2- 172.21.15.100:0/4139510247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb300109820 0x7fb300110050 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8009ca0 tx=0x7fb2f802a8c0 comp rx=0 tx=0).stop 2024-07-26T13:39:18.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.708+0000 7fb307ddf640 1 -- 172.21.15.100:0/4139510247 shutdown_connections 2024-07-26T13:39:18.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.708+0000 7fb307ddf640 1 --2- 172.21.15.100:0/4139510247 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb300109820 0x7fb300110050 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:18.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.708+0000 7fb307ddf640 1 --2- 172.21.15.100:0/4139510247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb300108ed0 0x7fb3001092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:18.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.708+0000 7fb307ddf640 1 --2- 172.21.15.100:0/4139510247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb300103590 0x7fb3001039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:18.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.708+0000 7fb307ddf640 1 -- 172.21.15.100:0/4139510247 >> 172.21.15.100:0/4139510247 conn(0x7fb30007b280 msgr2=0x7fb30007b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:18.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.708+0000 7fb307ddf640 1 -- 172.21.15.100:0/4139510247 shutdown_connections 2024-07-26T13:39:18.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.708+0000 7fb307ddf640 1 -- 172.21.15.100:0/4139510247 wait complete. 2024-07-26T13:39:18.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.709+0000 7fb307ddf640 1 Processor -- start 2024-07-26T13:39:18.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.709+0000 7fb307ddf640 1 -- start start 2024-07-26T13:39:18.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.709+0000 7fb307ddf640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb300103590 0x7fb300199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:18.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.709+0000 7fb307ddf640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb300108ed0 0x7fb30019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:18.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.709+0000 7fb307ddf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb300109820 0x7fb30019f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:18.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.709+0000 7fb307ddf640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb3001134f0 con 0x7fb300109820 2024-07-26T13:39:18.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.709+0000 7fb307ddf640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb300113370 con 0x7fb300103590 2024-07-26T13:39:18.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.709+0000 7fb307ddf640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb300113670 con 0x7fb300108ed0 2024-07-26T13:39:18.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.709+0000 7fb30637c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb300109820 0x7fb30019f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:18.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.709+0000 7fb30637c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb300109820 0x7fb30019f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60402/0 (socket says 172.21.15.100:60402) 2024-07-26T13:39:18.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.709+0000 7fb30637c640 1 -- 172.21.15.100:0/1866665804 learned_addr learned my addr 172.21.15.100:0/1866665804 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:18.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.709+0000 7fb30537a640 1 --2- 172.21.15.100:0/1866665804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb300108ed0 0x7fb30019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:18.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.709+0000 7fb305b7b640 1 --2- 172.21.15.100:0/1866665804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb300103590 0x7fb300199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:18.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.709+0000 7fb30637c640 1 -- 172.21.15.100:0/1866665804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb300108ed0 msgr2=0x7fb30019a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:18.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.709+0000 7fb30637c640 1 --2- 172.21.15.100:0/1866665804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb300108ed0 0x7fb30019a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:18.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.709+0000 7fb30637c640 1 -- 172.21.15.100:0/1866665804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb300103590 msgr2=0x7fb300199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:18.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.709+0000 7fb30637c640 1 --2- 172.21.15.100:0/1866665804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb300103590 0x7fb300199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:18.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.710+0000 7fb30637c640 1 -- 172.21.15.100:0/1866665804 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb30019fe10 con 0x7fb300109820 2024-07-26T13:39:18.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.710+0000 7fb30637c640 1 --2- 172.21.15.100:0/1866665804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb300109820 0x7fb30019f6e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb2f80098d0 tx=0x7fb2f80317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:18.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.710+0000 7fb2f6ffd640 1 -- 172.21.15.100:0/1866665804 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f8003b70 con 0x7fb300109820 2024-07-26T13:39:18.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.710+0000 7fb307ddf640 1 -- 172.21.15.100:0/1866665804 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3001a00a0 con 0x7fb300109820 2024-07-26T13:39:18.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.710+0000 7fb2f6ffd640 1 -- 172.21.15.100:0/1866665804 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2f8003d10 con 0x7fb300109820 2024-07-26T13:39:18.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.710+0000 7fb2f6ffd640 1 -- 172.21.15.100:0/1866665804 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f802fa20 con 0x7fb300109820 2024-07-26T13:39:18.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.710+0000 7fb307ddf640 1 -- 172.21.15.100:0/1866665804 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3001a6860 con 0x7fb300109820 2024-07-26T13:39:18.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.712+0000 7fb2f6ffd640 1 -- 172.21.15.100:0/1866665804 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7fb2f802fc40 con 0x7fb300109820 2024-07-26T13:39:18.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.713+0000 7fb30e363740 1 Processor -- start 2024-07-26T13:39:18.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.713+0000 7fb30e363740 1 -- start start 2024-07-26T13:39:18.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.713+0000 7fb30e363740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7e6e43f60 0x55b7e71e93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:18.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.713+0000 7fb30e363740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e6e1b0a0 0x55b7e71e98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:18.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.713+0000 7fb30e363740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7e6e23280 0x55b7e71e9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:18.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.713+0000 7fb30e363740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55b7e715f940 con 0x55b7e6e1b0a0 2024-07-26T13:39:18.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.713+0000 7fb30e363740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b7e714ee40 con 0x55b7e6e43f60 2024-07-26T13:39:18.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.713+0000 7fb30e363740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b7e7152e80 con 0x55b7e6e23280 2024-07-26T13:39:18.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.713+0000 7fb305b7b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e6e1b0a0 0x55b7e71e98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:18.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.713+0000 7fb305b7b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e6e1b0a0 0x55b7e71e98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60414/0 (socket says 172.21.15.100:60414) 2024-07-26T13:39:18.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.713+0000 7fb305b7b640 1 -- 172.21.15.100:0/3894477770 learned_addr learned my addr 172.21.15.100:0/3894477770 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:18.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.713+0000 7fb30537a640 1 --2- 172.21.15.100:0/3894477770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7e6e43f60 0x55b7e71e93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:18.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.713+0000 7fb30537a640 1 --2- 172.21.15.100:0/3894477770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7e6e23280 0x55b7e71e9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:18.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.714+0000 7fb305b7b640 1 -- 172.21.15.100:0/3894477770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7e6e23280 msgr2=0x55b7e71e9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:18.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.714+0000 7fb305b7b640 1 --2- 172.21.15.100:0/3894477770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7e6e23280 0x55b7e71e9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:18.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.714+0000 7fb305b7b640 1 -- 172.21.15.100:0/3894477770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7e6e43f60 msgr2=0x55b7e71e93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:18.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.714+0000 7fb305b7b640 1 --2- 172.21.15.100:0/3894477770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7e6e43f60 0x55b7e71e93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:18.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.714+0000 7fb305b7b640 1 -- 172.21.15.100:0/3894477770 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7e70f5a60 con 0x55b7e6e1b0a0 2024-07-26T13:39:18.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.714+0000 7fb30537a640 1 --2- 172.21.15.100:0/3894477770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7e6e23280 0x55b7e71e9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:39:18.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.714+0000 7fb2f6ffd640 1 --2- 172.21.15.100:0/1866665804 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb2ec0b3c20 0x7fb2ec0b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:18.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.714+0000 7fb305b7b640 1 --2- 172.21.15.100:0/3894477770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e6e1b0a0 0x55b7e71e98e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc009d20 tx=0x7fb2fc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3325120ac126d9b server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:18.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.714+0000 7fb30537a640 1 --2- 172.21.15.100:0/1866665804 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb2ec0b3c20 0x7fb2ec0b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:18.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.714+0000 7fb2f6ffd640 1 -- 172.21.15.100:0/1866665804 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fb2f81051b0 con 0x7fb300109820 2024-07-26T13:39:18.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.714+0000 7fb2ebfff640 1 -- 172.21.15.100:0/3894477770 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc01e070 con 0x55b7e6e1b0a0 2024-07-26T13:39:18.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.714+0000 7fb2ebfff640 1 -- 172.21.15.100:0/3894477770 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2fc019ae0 con 0x55b7e6e1b0a0 2024-07-26T13:39:18.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.714+0000 7fb2ebfff640 1 -- 172.21.15.100:0/3894477770 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc00eac0 con 0x55b7e6e1b0a0 2024-07-26T13:39:18.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.715+0000 7fb30e363740 1 -- 172.21.15.100:0/3894477770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e6e1b0a0 msgr2=0x55b7e71e98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:18.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.715+0000 7fb30537a640 1 --2- 172.21.15.100:0/1866665804 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb2ec0b3c20 0x7fb2ec0b6130 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb300108a70 tx=0x7fb2f0013040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:18.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.715+0000 7fb30e363740 1 --2- 172.21.15.100:0/3894477770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e6e1b0a0 0x55b7e71e98e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc009d20 tx=0x7fb2fc011810 comp rx=0 tx=0).stop 2024-07-26T13:39:18.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.715+0000 7fb30e363740 1 -- 172.21.15.100:0/3894477770 shutdown_connections 2024-07-26T13:39:18.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.715+0000 7fb30e363740 1 --2- 172.21.15.100:0/3894477770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7e6e23280 0x55b7e71e9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:18.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.715+0000 7fb30e363740 1 --2- 172.21.15.100:0/3894477770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e6e1b0a0 0x55b7e71e98e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:18.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.715+0000 7fb30e363740 1 --2- 172.21.15.100:0/3894477770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7e6e43f60 0x55b7e71e93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:18.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.715+0000 7fb30e363740 1 -- 172.21.15.100:0/3894477770 >> 172.21.15.100:0/3894477770 conn(0x55b7e6e38350 msgr2=0x55b7e71e7290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:18.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.715+0000 7fb30e363740 1 -- 172.21.15.100:0/3894477770 shutdown_connections 2024-07-26T13:39:18.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.715+0000 7fb30e363740 1 -- 172.21.15.100:0/3894477770 wait complete. 2024-07-26T13:39:18.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.715+0000 7fb30e363740 1 -- start start 2024-07-26T13:39:18.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.716+0000 7fb30e363740 1 Processor -- start 2024-07-26T13:39:18.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.716+0000 7fb30e363740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7e6e1b0a0 0x55b7e71f4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:18.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.716+0000 7fb30e363740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e6e23280 0x55b7e71f54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:18.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.716+0000 7fb30e363740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7e6e43f60 0x55b7e7200ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:18.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.716+0000 7fb30e363740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55b7e6fb25f0 con 0x55b7e6e23280 2024-07-26T13:39:18.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.716+0000 7fb30e363740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b7e70f18b0 con 0x55b7e6e43f60 2024-07-26T13:39:18.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.716+0000 7fb30e363740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b7e71f7b80 con 0x55b7e6e1b0a0 2024-07-26T13:39:18.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.716+0000 7fb30e363740 10 client.0 _ll_register_callbacks cb 0x55b7e70fb7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:39:18.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.716+0000 7fb30537a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e6e23280 0x55b7e71f54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:18.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.716+0000 7fb30537a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e6e23280 0x55b7e71f54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60428/0 (socket says 172.21.15.100:60428) 2024-07-26T13:39:18.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.716+0000 7fb30537a640 1 -- 172.21.15.100:0/1441034254 learned_addr learned my addr 172.21.15.100:0/1441034254 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:18.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.716+0000 7fb305b7b640 1 --2- 172.21.15.100:0/1441034254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7e6e1b0a0 0x55b7e71f4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:18.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.716+0000 7fb30637c640 1 --2- 172.21.15.100:0/1441034254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7e6e43f60 0x55b7e7200ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:18.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.716+0000 7fb30537a640 1 -- 172.21.15.100:0/1441034254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7e6e1b0a0 msgr2=0x55b7e71f4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:18.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.716+0000 7fb30537a640 1 --2- 172.21.15.100:0/1441034254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7e6e1b0a0 0x55b7e71f4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:18.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.716+0000 7fb30537a640 1 -- 172.21.15.100:0/1441034254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7e6e43f60 msgr2=0x55b7e7200ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:18.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.716+0000 7fb30537a640 1 --2- 172.21.15.100:0/1441034254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7e6e43f60 0x55b7e7200ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:18.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.716+0000 7fb30537a640 1 -- 172.21.15.100:0/1441034254 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7e6eaf540 con 0x55b7e6e23280 2024-07-26T13:39:18.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.717+0000 7fb30537a640 1 --2- 172.21.15.100:0/1441034254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e6e23280 0x55b7e71f54c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb2f001fd80 tx=0x7fb2f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a051b6ae2826cbe server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:18.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.717+0000 7fb2e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:39:18.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.717+0000 7fb2e97fa640 1 -- 172.21.15.100:0/1441034254 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f0004020 con 0x55b7e6e23280 2024-07-26T13:39:18.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.717+0000 7fb2e97fa640 1 -- 172.21.15.100:0/1441034254 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fb2f00041c0 con 0x55b7e6e23280 2024-07-26T13:39:18.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.717+0000 7fb30e363740 1 -- 172.21.15.100:0/1441034254 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2fc019ae0 con 0x55b7e6e23280 2024-07-26T13:39:18.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.717+0000 7fb2e97fa640 1 -- 172.21.15.100:0/1441034254 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb2f0004400 con 0x55b7e6e23280 2024-07-26T13:39:18.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.717+0000 7fb30e363740 10 client.15162 fetch_fsmap learned FSMap version 25 2024-07-26T13:39:18.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.717+0000 7fb30e363740 1 -- 172.21.15.100:0/1441034254 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b7e7203bf0 con 0x55b7e6e23280 2024-07-26T13:39:18.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.717+0000 7fb2e97fa640 1 -- 172.21.15.100:0/1441034254 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2fc019ae0 con 0x55b7e6e23280 2024-07-26T13:39:18.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.718+0000 7fb2e97fa640 1 -- 172.21.15.100:0/1441034254 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb2f0030020 con 0x55b7e6e23280 2024-07-26T13:39:18.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.718+0000 7fb30e363740 10 client.15162 fetch_fsmap finished waiting for FSMap version 25 2024-07-26T13:39:18.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.718+0000 7fb30e363740 10 client.15162 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-26T13:39:18.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.718+0000 7fb30e363740 20 client.15162 populate_metadata read hostname 'smithi100' 2024-07-26T13:39:18.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.718+0000 7fb30e363740 1 --2- 172.21.15.100:0/1441034254 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55b7e6e028e0 0x55b7e72075d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:18.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.718+0000 7fb30e363740 4 client.15162 mds_command: new command op to 24595 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:39:18.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.718+0000 7fb30e363740 1 -- 172.21.15.100:0/1441034254 --> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7e7161890 con 0x55b7e6e028e0 2024-07-26T13:39:18.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.718+0000 7fb305b7b640 1 --2- 172.21.15.100:0/1441034254 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55b7e6e028e0 0x55b7e72075d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:18.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.718+0000 7fb305b7b640 1 --2- 172.21.15.100:0/1441034254 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55b7e6e028e0 0x55b7e72075d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a4d56a785b2ee4f4 server_cookie=198114a06cde73b5 in_seq=0 out_seq=0 2024-07-26T13:39:18.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.718+0000 7fb2e97fa640 10 client.15162 ms_handle_connect on v2:172.21.15.167:6836/3828634468 2024-07-26T13:39:18.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.719+0000 7fb2e97fa640 1 -- 172.21.15.100:0/1441034254 <== mds.2 v2:172.21.15.167:6836/3828634468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b7e7161890 con 0x55b7e6e028e0 2024-07-26T13:39:18.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.719+0000 7fb2e97fa640 10 client.15162 handle_command_reply: tid=0 2024-07-26T13:39:18.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.719+0000 7fb30e363740 1 client.15162 shutdown 2024-07-26T13:39:18.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.719+0000 7fb30e363740 1 -- 172.21.15.100:0/1441034254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e6e23280 msgr2=0x55b7e71f54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:18.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.719+0000 7fb30e363740 1 --2- 172.21.15.100:0/1441034254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e6e23280 0x55b7e71f54c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb2f001fd80 tx=0x7fb2f0004730 comp rx=0 tx=0).stop 2024-07-26T13:39:18.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.720+0000 7fb30e363740 1 -- 172.21.15.100:0/1441034254 shutdown_connections 2024-07-26T13:39:18.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.720+0000 7fb30e363740 1 --2- 172.21.15.100:0/1441034254 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55b7e6e028e0 0x55b7e72075d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:18.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.720+0000 7fb30e363740 1 --2- 172.21.15.100:0/1441034254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7e6e43f60 0x55b7e7200ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:18.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.720+0000 7fb30e363740 1 --2- 172.21.15.100:0/1441034254 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e6e23280 0x55b7e71f54c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:18.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.720+0000 7fb2e97fa640 0 client.15162 ms_handle_reset on v2:172.21.15.167:6836/3828634468 2024-07-26T13:39:18.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.720+0000 7fb30e363740 1 --2- 172.21.15.100:0/1441034254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7e6e1b0a0 0x55b7e71f4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:18.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.720+0000 7fb30e363740 1 -- 172.21.15.100:0/1441034254 >> 172.21.15.100:0/1441034254 conn(0x55b7e6e38350 msgr2=0x55b7e71ec530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:18.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.720+0000 7fb30e363740 1 -- 172.21.15.100:0/1441034254 shutdown_connections 2024-07-26T13:39:18.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.720+0000 7fb30e363740 1 -- 172.21.15.100:0/1441034254 wait complete. 2024-07-26T13:39:18.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.720+0000 7fb30e363740 20 client.15162 trim_cache size 0 max 16384 2024-07-26T13:39:18.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.738+0000 7fb30e363740 1 Processor -- start 2024-07-26T13:39:18.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.738+0000 7fb30e363740 1 -- start start 2024-07-26T13:39:18.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.738+0000 7fb30e363740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e6e028e0 0x55b7e71e1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:18.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.738+0000 7fb30e363740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7e6e1b0a0 0x55b7e7200e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:18.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.738+0000 7fb30e363740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7e6e23280 0x55b7e7201350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:18.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.738+0000 7fb30e363740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55b7e7205910 con 0x55b7e6e028e0 2024-07-26T13:39:18.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.739+0000 7fb30e363740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b7e7205fe0 con 0x55b7e6e23280 2024-07-26T13:39:18.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.739+0000 7fb30e363740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b7e72066b0 con 0x55b7e6e1b0a0 2024-07-26T13:39:18.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.739+0000 7fb305b7b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e6e028e0 0x55b7e71e1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:18.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.739+0000 7fb305b7b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e6e028e0 0x55b7e71e1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60430/0 (socket says 172.21.15.100:60430) 2024-07-26T13:39:18.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.739+0000 7fb305b7b640 1 -- 172.21.15.100:0/545712998 learned_addr learned my addr 172.21.15.100:0/545712998 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:18.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.739+0000 7fb30537a640 1 --2- 172.21.15.100:0/545712998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7e6e1b0a0 0x55b7e7200e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:18.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.739+0000 7fb305b7b640 1 -- 172.21.15.100:0/545712998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7e6e1b0a0 msgr2=0x55b7e7200e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:18.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.739+0000 7fb30637c640 1 --2- 172.21.15.100:0/545712998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7e6e23280 0x55b7e7201350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:18.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.739+0000 7fb305b7b640 1 --2- 172.21.15.100:0/545712998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7e6e1b0a0 0x55b7e7200e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:18.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.739+0000 7fb305b7b640 1 -- 172.21.15.100:0/545712998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7e6e23280 msgr2=0x55b7e7201350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:18.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.739+0000 7fb305b7b640 1 --2- 172.21.15.100:0/545712998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7e6e23280 0x55b7e7201350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:18.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.739+0000 7fb305b7b640 1 -- 172.21.15.100:0/545712998 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7e71f5b10 con 0x55b7e6e028e0 2024-07-26T13:39:18.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.739+0000 7fb305b7b640 1 --2- 172.21.15.100:0/545712998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e6e028e0 0x55b7e71e1ed0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x55b7e71e8020 tx=0x7fb2fc00e700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6684be218fe6437 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:18.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.739+0000 7fb2eb7fe640 1 -- 172.21.15.100:0/545712998 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc01e040 con 0x55b7e6e028e0 2024-07-26T13:39:18.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.739+0000 7fb2eb7fe640 1 -- 172.21.15.100:0/545712998 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2fc0194b0 con 0x55b7e6e028e0 2024-07-26T13:39:18.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.739+0000 7fb2eb7fe640 1 -- 172.21.15.100:0/545712998 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc00e990 con 0x55b7e6e028e0 2024-07-26T13:39:18.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.740+0000 7fb30e363740 1 -- 172.21.15.100:0/545712998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e6e028e0 msgr2=0x55b7e71e1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:18.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.740+0000 7fb30e363740 1 --2- 172.21.15.100:0/545712998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e6e028e0 0x55b7e71e1ed0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x55b7e71e8020 tx=0x7fb2fc00e700 comp rx=0 tx=0).stop 2024-07-26T13:39:18.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.740+0000 7fb30e363740 1 -- 172.21.15.100:0/545712998 shutdown_connections 2024-07-26T13:39:18.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.740+0000 7fb30e363740 1 --2- 172.21.15.100:0/545712998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7e6e23280 0x55b7e7201350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:18.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.740+0000 7fb30e363740 1 --2- 172.21.15.100:0/545712998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7e6e1b0a0 0x55b7e7200e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:18.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.740+0000 7fb30e363740 1 --2- 172.21.15.100:0/545712998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e6e028e0 0x55b7e71e1ed0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:18.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.740+0000 7fb30e363740 1 -- 172.21.15.100:0/545712998 >> 172.21.15.100:0/545712998 conn(0x55b7e6e38350 msgr2=0x55b7e71f3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:18.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.740+0000 7fb30e363740 1 -- 172.21.15.100:0/545712998 shutdown_connections 2024-07-26T13:39:18.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.740+0000 7fb30e363740 1 -- 172.21.15.100:0/545712998 wait complete. 2024-07-26T13:39:18.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.741+0000 7fb30e363740 1 -- start start 2024-07-26T13:39:18.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.741+0000 7fb30e363740 1 Processor -- start 2024-07-26T13:39:18.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.741+0000 7fb30e363740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7e6e028e0 0x55b7e720d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:18.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.741+0000 7fb30e363740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7e6e1b0a0 0x55b7e720dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:18.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.741+0000 7fb30e363740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e6e23280 0x55b7e7208de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:18.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.741+0000 7fb30e363740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55b7e7207450 con 0x55b7e6e23280 2024-07-26T13:39:18.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.741+0000 7fb30e363740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b7e7206d80 con 0x55b7e6e1b0a0 2024-07-26T13:39:18.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.741+0000 7fb30e363740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b7e7207b20 con 0x55b7e6e028e0 2024-07-26T13:39:18.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.741+0000 7fb30e363740 10 client.0 _ll_register_callbacks cb 0x7fb2f001fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:39:18.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.741+0000 7fb30637c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e6e23280 0x55b7e7208de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:18.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.741+0000 7fb305b7b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7e6e028e0 0x55b7e720d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:18.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.741+0000 7fb30637c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e6e23280 0x55b7e7208de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60444/0 (socket says 172.21.15.100:60444) 2024-07-26T13:39:18.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.741+0000 7fb305b7b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7e6e028e0 0x55b7e720d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:59498/0 (socket says 172.21.15.100:59498) 2024-07-26T13:39:18.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.741+0000 7fb30637c640 1 -- 172.21.15.100:0/261634623 learned_addr learned my addr 172.21.15.100:0/261634623 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:18.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.741+0000 7fb30537a640 1 --2- 172.21.15.100:0/261634623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7e6e1b0a0 0x55b7e720dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:18.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.742+0000 7fb30637c640 1 -- 172.21.15.100:0/261634623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7e6e028e0 msgr2=0x55b7e720d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:18.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.742+0000 7fb30637c640 1 --2- 172.21.15.100:0/261634623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7e6e028e0 0x55b7e720d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:18.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.742+0000 7fb30637c640 1 -- 172.21.15.100:0/261634623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7e6e1b0a0 msgr2=0x55b7e720dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:18.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.742+0000 7fb30637c640 1 --2- 172.21.15.100:0/261634623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7e6e1b0a0 0x55b7e720dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:18.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.742+0000 7fb30637c640 1 -- 172.21.15.100:0/261634623 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7e72094a0 con 0x55b7e6e23280 2024-07-26T13:39:18.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.742+0000 7fb305b7b640 1 --2- 172.21.15.100:0/261634623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7e6e028e0 0x55b7e720d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:39:18.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.742+0000 7fb30537a640 1 --2- 172.21.15.100:0/261634623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7e6e1b0a0 0x55b7e720dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:39:18.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.742+0000 7fb30637c640 1 --2- 172.21.15.100:0/261634623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e6e23280 0x55b7e7208de0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8102c00 tx=0x7fb2f80cba80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23c812cbb9743953 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:18.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.742+0000 7fb2e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:39:18.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.742+0000 7fb2e97fa640 1 -- 172.21.15.100:0/261634623 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f80cc000 con 0x55b7e6e23280 2024-07-26T13:39:18.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.742+0000 7fb2e97fa640 1 -- 172.21.15.100:0/261634623 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fb2f80cc470 con 0x55b7e6e23280 2024-07-26T13:39:18.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.742+0000 7fb30e363740 1 -- 172.21.15.100:0/261634623 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2fc0194b0 con 0x55b7e6e23280 2024-07-26T13:39:18.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.742+0000 7fb2e97fa640 1 -- 172.21.15.100:0/261634623 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb2f80d0600 con 0x55b7e6e23280 2024-07-26T13:39:18.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.742+0000 7fb30e363740 10 client.15168 fetch_fsmap learned FSMap version 25 2024-07-26T13:39:18.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.742+0000 7fb30e363740 1 -- 172.21.15.100:0/261634623 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b7e720eea0 con 0x55b7e6e23280 2024-07-26T13:39:18.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.742+0000 7fb2e97fa640 1 -- 172.21.15.100:0/261634623 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2fc0194b0 con 0x55b7e6e23280 2024-07-26T13:39:18.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.743+0000 7fb2e97fa640 1 -- 172.21.15.100:0/261634623 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb2f80ed020 con 0x55b7e6e23280 2024-07-26T13:39:18.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.743+0000 7fb30e363740 10 client.15168 fetch_fsmap finished waiting for FSMap version 25 2024-07-26T13:39:18.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.743+0000 7fb30e363740 10 client.15168 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-26T13:39:18.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.743+0000 7fb30e363740 20 client.15168 populate_metadata read hostname 'smithi100' 2024-07-26T13:39:18.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.743+0000 7fb30e363740 1 --2- 172.21.15.100:0/261634623 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55b7e6e43f60 0x55b7e7273b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:18.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.743+0000 7fb30e363740 4 client.15168 mds_command: new command op to 24595 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-26T13:39:18.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.743+0000 7fb30e363740 1 -- 172.21.15.100:0/261634623 --> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] -- command(tid 0: {"prefix": "status"}) -- 0x55b7e715d330 con 0x55b7e6e43f60 2024-07-26T13:39:18.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.743+0000 7fb305b7b640 1 --2- 172.21.15.100:0/261634623 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55b7e6e43f60 0x55b7e7273b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:18.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.743+0000 7fb2e97fa640 10 client.15168 ms_handle_connect on v2:172.21.15.167:6836/3828634468 2024-07-26T13:39:18.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.743+0000 7fb305b7b640 1 --2- 172.21.15.100:0/261634623 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55b7e6e43f60 0x55b7e7273b00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a1ad2718571147f7 server_cookie=4aa310eb84ccb9bb in_seq=0 out_seq=0 2024-07-26T13:39:18.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.743+0000 7fb2e97fa640 1 -- 172.21.15.100:0/261634623 <== mds.2 v2:172.21.15.167:6836/3828634468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55b7e715d330 con 0x55b7e6e43f60 2024-07-26T13:39:18.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.743+0000 7fb2e97fa640 10 client.15168 handle_command_reply: tid=0 2024-07-26T13:39:18.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.743+0000 7fb30e363740 1 client.15168 shutdown 2024-07-26T13:39:18.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.744+0000 7fb30e363740 1 -- 172.21.15.100:0/261634623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e6e23280 msgr2=0x55b7e7208de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:18.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.744+0000 7fb30e363740 1 --2- 172.21.15.100:0/261634623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e6e23280 0x55b7e7208de0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8102c00 tx=0x7fb2f80cba80 comp rx=0 tx=0).stop 2024-07-26T13:39:18.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.744+0000 7fb30e363740 1 -- 172.21.15.100:0/261634623 shutdown_connections 2024-07-26T13:39:18.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.744+0000 7fb30e363740 1 --2- 172.21.15.100:0/261634623 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55b7e6e43f60 0x55b7e7273b00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:18.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.744+0000 7fb30e363740 1 --2- 172.21.15.100:0/261634623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b7e6e23280 0x55b7e7208de0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:18.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.744+0000 7fb2e97fa640 0 client.15168 ms_handle_reset on v2:172.21.15.167:6836/3828634468 2024-07-26T13:39:18.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.744+0000 7fb30e363740 1 --2- 172.21.15.100:0/261634623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b7e6e1b0a0 0x55b7e720dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:18.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.744+0000 7fb30e363740 1 --2- 172.21.15.100:0/261634623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b7e6e028e0 0x55b7e720d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:18.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.744+0000 7fb30e363740 1 -- 172.21.15.100:0/261634623 >> 172.21.15.100:0/261634623 conn(0x55b7e6e38350 msgr2=0x55b7e71ec4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:18.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.744+0000 7fb30e363740 1 -- 172.21.15.100:0/261634623 shutdown_connections 2024-07-26T13:39:18.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.744+0000 7fb30e363740 1 -- 172.21.15.100:0/261634623 wait complete. 2024-07-26T13:39:18.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.744+0000 7fb30e363740 20 client.15168 trim_cache size 0 max 16384 2024-07-26T13:39:18.754 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:39:18.754 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "bf83e2e6-4b52-11ef-bcb7-c7b262605968", 2024-07-26T13:39:18.754 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 2, 2024-07-26T13:39:18.754 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 24595, 2024-07-26T13:39:18.754 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-07-26T13:39:18.754 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-07-26T13:39:18.754 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-07-26T13:39:18.754 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 124.21188348299999, 2024-07-26T13:39:18.754 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 25, 2024-07-26T13:39:18.754 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 81, 2024-07-26T13:39:18.755 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 81, 2024-07-26T13:39:18.755 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 169.435837115 2024-07-26T13:39:18.755 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:39:18.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.744+0000 7fb307ddf640 1 -- 172.21.15.100:0/1866665804 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb2ec0b3c20 msgr2=0x7fb2ec0b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:18.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.744+0000 7fb307ddf640 1 --2- 172.21.15.100:0/1866665804 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb2ec0b3c20 0x7fb2ec0b6130 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb300108a70 tx=0x7fb2f0013040 comp rx=0 tx=0).stop 2024-07-26T13:39:18.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.744+0000 7fb307ddf640 1 -- 172.21.15.100:0/1866665804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb300109820 msgr2=0x7fb30019f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:18.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.744+0000 7fb307ddf640 1 --2- 172.21.15.100:0/1866665804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb300109820 0x7fb30019f6e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb2f80098d0 tx=0x7fb2f80317b0 comp rx=0 tx=0).stop 2024-07-26T13:39:18.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.744+0000 7fb307ddf640 1 -- 172.21.15.100:0/1866665804 shutdown_connections 2024-07-26T13:39:18.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.744+0000 7fb307ddf640 1 --2- 172.21.15.100:0/1866665804 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb2ec0b3c20 0x7fb2ec0b6130 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:18.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.744+0000 7fb307ddf640 1 --2- 172.21.15.100:0/1866665804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb300109820 0x7fb30019f6e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:18.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.744+0000 7fb307ddf640 1 --2- 172.21.15.100:0/1866665804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb300108ed0 0x7fb30019a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:18.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.744+0000 7fb307ddf640 1 --2- 172.21.15.100:0/1866665804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb300103590 0x7fb300199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:18.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.744+0000 7fb307ddf640 1 -- 172.21.15.100:0/1866665804 >> 172.21.15.100:0/1866665804 conn(0x7fb30007b280 msgr2=0x7fb300106f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:18.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.745+0000 7fb307ddf640 1 -- 172.21.15.100:0/1866665804 shutdown_connections 2024-07-26T13:39:18.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:18.745+0000 7fb307ddf640 1 -- 172.21.15.100:0/1866665804 wait complete. 2024-07-26T13:39:19.412 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.b status 2024-07-26T13:39:19.670 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:39:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:19 smithi100 ceph-mon[32599]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:39:19.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:19 smithi167 ceph-mon[37140]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:39:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:19 smithi138 ceph-mon[36954]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:39:21.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.316+0000 7f2833a43640 1 -- 172.21.15.100:0/3175819256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f282c107e50 msgr2=0x7f282c108260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:21.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.316+0000 7f2833a43640 1 --2- 172.21.15.100:0/3175819256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f282c107e50 0x7f282c108260 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f281c009900 tx=0x7f281c02a410 comp rx=0 tx=0).stop 2024-07-26T13:39:21.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.316+0000 7f2833a43640 1 -- 172.21.15.100:0/3175819256 shutdown_connections 2024-07-26T13:39:21.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.316+0000 7f2833a43640 1 --2- 172.21.15.100:0/3175819256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f282c1087a0 0x7f282c10efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:21.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.316+0000 7f2833a43640 1 --2- 172.21.15.100:0/3175819256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f282c107e50 0x7f282c108260 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:21.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.316+0000 7f2833a43640 1 --2- 172.21.15.100:0/3175819256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f282c102510 0x7f282c102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:21.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.316+0000 7f2833a43640 1 -- 172.21.15.100:0/3175819256 >> 172.21.15.100:0/3175819256 conn(0x7f282c0791a0 msgr2=0x7f282c0795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:21.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.316+0000 7f2833a43640 1 -- 172.21.15.100:0/3175819256 shutdown_connections 2024-07-26T13:39:21.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.316+0000 7f2833a43640 1 -- 172.21.15.100:0/3175819256 wait complete. 2024-07-26T13:39:21.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.317+0000 7f2833a43640 1 Processor -- start 2024-07-26T13:39:21.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.317+0000 7f2833a43640 1 -- start start 2024-07-26T13:39:21.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.317+0000 7f2833a43640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f282c102510 0x7f282c198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:21.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.317+0000 7f2833a43640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f282c107e50 0x7f282c199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:21.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.317+0000 7f2833a43640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f282c1087a0 0x7f282c19e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:21.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.317+0000 7f2833a43640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f282c112410 con 0x7f282c102510 2024-07-26T13:39:21.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.317+0000 7f2833a43640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f282c112290 con 0x7f282c1087a0 2024-07-26T13:39:21.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.317+0000 7f2833a43640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f282c112590 con 0x7f282c107e50 2024-07-26T13:39:21.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.317+0000 7f28317df640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f282c102510 0x7f282c198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:21.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.317+0000 7f28317df640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f282c102510 0x7f282c198e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60466/0 (socket says 172.21.15.100:60466) 2024-07-26T13:39:21.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.317+0000 7f28317df640 1 -- 172.21.15.100:0/1876543069 learned_addr learned my addr 172.21.15.100:0/1876543069 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:21.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.317+0000 7f2831fe0640 1 --2- 172.21.15.100:0/1876543069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f282c1087a0 0x7f282c19e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:21.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.317+0000 7f2830fde640 1 --2- 172.21.15.100:0/1876543069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f282c107e50 0x7f282c199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:21.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.317+0000 7f28317df640 1 -- 172.21.15.100:0/1876543069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f282c107e50 msgr2=0x7f282c199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:21.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.317+0000 7f28317df640 1 --2- 172.21.15.100:0/1876543069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f282c107e50 0x7f282c199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:21.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.317+0000 7f28317df640 1 -- 172.21.15.100:0/1876543069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f282c1087a0 msgr2=0x7f282c19e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:21.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.317+0000 7f28317df640 1 --2- 172.21.15.100:0/1876543069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f282c1087a0 0x7f282c19e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:21.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.317+0000 7f28317df640 1 -- 172.21.15.100:0/1876543069 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f282c19ed30 con 0x7f282c102510 2024-07-26T13:39:21.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.317+0000 7f2831fe0640 1 --2- 172.21.15.100:0/1876543069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f282c1087a0 0x7f282c19e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:39:21.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.318+0000 7f28317df640 1 --2- 172.21.15.100:0/1876543069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f282c102510 0x7f282c198e80 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f282800d950 tx=0x7f282800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:21.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.318+0000 7f28227fc640 1 -- 172.21.15.100:0/1876543069 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2828017070 con 0x7f282c102510 2024-07-26T13:39:21.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.318+0000 7f2833a43640 1 -- 172.21.15.100:0/1876543069 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f282c19f020 con 0x7f282c102510 2024-07-26T13:39:21.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.318+0000 7f28227fc640 1 -- 172.21.15.100:0/1876543069 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2828009c30 con 0x7f282c102510 2024-07-26T13:39:21.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.318+0000 7f28227fc640 1 -- 172.21.15.100:0/1876543069 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282800f620 con 0x7f282c102510 2024-07-26T13:39:21.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.318+0000 7f2833a43640 1 -- 172.21.15.100:0/1876543069 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f282c1a5840 con 0x7f282c102510 2024-07-26T13:39:21.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.320+0000 7f28227fc640 1 -- 172.21.15.100:0/1876543069 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f2828010460 con 0x7f282c102510 2024-07-26T13:39:21.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.321+0000 7f2839fc7740 1 Processor -- start 2024-07-26T13:39:21.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.321+0000 7f2839fc7740 1 -- start start 2024-07-26T13:39:21.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.321+0000 7f2839fc7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc0802df60 0x55dc083d33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:21.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.321+0000 7f2839fc7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc080050a0 0x55dc083d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:21.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.321+0000 7f2839fc7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc0800d280 0x55dc083d3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:21.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.321+0000 7f2839fc7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55dc08349940 con 0x55dc080050a0 2024-07-26T13:39:21.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.321+0000 7f2839fc7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dc08338e40 con 0x55dc0800d280 2024-07-26T13:39:21.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.321+0000 7f2839fc7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55dc0833ce80 con 0x55dc0802df60 2024-07-26T13:39:21.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.321+0000 7f28317df640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc080050a0 0x55dc083d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:21.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.321+0000 7f2831fe0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc0802df60 0x55dc083d33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:21.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.321+0000 7f28317df640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc080050a0 0x55dc083d38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60478/0 (socket says 172.21.15.100:60478) 2024-07-26T13:39:21.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.321+0000 7f28317df640 1 -- 172.21.15.100:0/1236772217 learned_addr learned my addr 172.21.15.100:0/1236772217 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:21.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.321+0000 7f2830fde640 1 --2- 172.21.15.100:0/1236772217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc0800d280 0x55dc083d3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:21.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.322+0000 7f28227fc640 1 --2- 172.21.15.100:0/1876543069 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f28180b3c50 0x7f28180b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:21.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.322+0000 7f28317df640 1 -- 172.21.15.100:0/1236772217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc0802df60 msgr2=0x55dc083d33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:21.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.322+0000 7f28317df640 1 --2- 172.21.15.100:0/1236772217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc0802df60 0x55dc083d33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:21.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.322+0000 7f28317df640 1 -- 172.21.15.100:0/1236772217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc0800d280 msgr2=0x55dc083d3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:21.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.322+0000 7f28317df640 1 --2- 172.21.15.100:0/1236772217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc0800d280 0x55dc083d3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:21.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.322+0000 7f2831fe0640 1 --2- 172.21.15.100:0/1876543069 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f28180b3c50 0x7f28180b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:21.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.322+0000 7f28317df640 1 -- 172.21.15.100:0/1236772217 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc082dfa60 con 0x55dc080050a0 2024-07-26T13:39:21.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.322+0000 7f28227fc640 1 -- 172.21.15.100:0/1876543069 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f28280e39f0 con 0x7f282c102510 2024-07-26T13:39:21.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.322+0000 7f28317df640 1 --2- 172.21.15.100:0/1236772217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc080050a0 0x55dc083d38e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2828051050 tx=0x7f2828010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=542c7b2b97e43a03 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:21.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.322+0000 7f28177fe640 1 -- 172.21.15.100:0/1236772217 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28280aa190 con 0x55dc080050a0 2024-07-26T13:39:21.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.322+0000 7f28177fe640 1 -- 172.21.15.100:0/1236772217 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28280aa350 con 0x55dc080050a0 2024-07-26T13:39:21.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.322+0000 7f28177fe640 1 -- 172.21.15.100:0/1236772217 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2828022300 con 0x55dc080050a0 2024-07-26T13:39:21.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.323+0000 7f2839fc7740 1 -- 172.21.15.100:0/1236772217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc080050a0 msgr2=0x55dc083d38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:21.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.323+0000 7f2831fe0640 1 --2- 172.21.15.100:0/1876543069 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f28180b3c50 0x7f28180b6160 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2824012930 tx=0x7f2824004ab0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:21.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.323+0000 7f2839fc7740 1 --2- 172.21.15.100:0/1236772217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc080050a0 0x55dc083d38e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2828051050 tx=0x7f2828010cb0 comp rx=0 tx=0).stop 2024-07-26T13:39:21.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.323+0000 7f2839fc7740 1 -- 172.21.15.100:0/1236772217 shutdown_connections 2024-07-26T13:39:21.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.323+0000 7f2839fc7740 1 --2- 172.21.15.100:0/1236772217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc0800d280 0x55dc083d3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:21.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.323+0000 7f2839fc7740 1 --2- 172.21.15.100:0/1236772217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc080050a0 0x55dc083d38e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:21.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.323+0000 7f2839fc7740 1 --2- 172.21.15.100:0/1236772217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc0802df60 0x55dc083d33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:21.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.323+0000 7f2839fc7740 1 -- 172.21.15.100:0/1236772217 >> 172.21.15.100:0/1236772217 conn(0x55dc08022350 msgr2=0x55dc083d1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:21.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.323+0000 7f2839fc7740 1 -- 172.21.15.100:0/1236772217 shutdown_connections 2024-07-26T13:39:21.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.323+0000 7f2839fc7740 1 -- 172.21.15.100:0/1236772217 wait complete. 2024-07-26T13:39:21.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.324+0000 7f2839fc7740 1 -- start start 2024-07-26T13:39:21.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.324+0000 7f2839fc7740 1 Processor -- start 2024-07-26T13:39:21.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.324+0000 7f2839fc7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc080050a0 0x55dc083def50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:21.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.324+0000 7f2839fc7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc0800d280 0x55dc083df490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:21.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.324+0000 7f2839fc7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc0802df60 0x55dc083eae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:21.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.324+0000 7f2839fc7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55dc0819c5f0 con 0x55dc0802df60 2024-07-26T13:39:21.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.324+0000 7f2839fc7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dc082db8b0 con 0x55dc0800d280 2024-07-26T13:39:21.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.324+0000 7f2839fc7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55dc083e1b20 con 0x55dc080050a0 2024-07-26T13:39:21.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.324+0000 7f2839fc7740 10 client.0 _ll_register_callbacks cb 0x55dc082e57f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:39:21.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.324+0000 7f2831fe0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc0802df60 0x55dc083eae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:21.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.324+0000 7f2831fe0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc0802df60 0x55dc083eae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60492/0 (socket says 172.21.15.100:60492) 2024-07-26T13:39:21.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.324+0000 7f2831fe0640 1 -- 172.21.15.100:0/4143947743 learned_addr learned my addr 172.21.15.100:0/4143947743 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:21.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.325+0000 7f28317df640 1 --2- 172.21.15.100:0/4143947743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc080050a0 0x55dc083def50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:21.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.325+0000 7f2830fde640 1 --2- 172.21.15.100:0/4143947743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc0800d280 0x55dc083df490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:21.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.325+0000 7f2831fe0640 1 -- 172.21.15.100:0/4143947743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc080050a0 msgr2=0x55dc083def50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:21.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.325+0000 7f2831fe0640 1 --2- 172.21.15.100:0/4143947743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc080050a0 0x55dc083def50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:21.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.325+0000 7f2831fe0640 1 -- 172.21.15.100:0/4143947743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc0800d280 msgr2=0x55dc083df490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:21.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.325+0000 7f2831fe0640 1 --2- 172.21.15.100:0/4143947743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc0800d280 0x55dc083df490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:21.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.325+0000 7f2831fe0640 1 -- 172.21.15.100:0/4143947743 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc08099540 con 0x55dc0802df60 2024-07-26T13:39:21.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.325+0000 7f2831fe0640 1 --2- 172.21.15.100:0/4143947743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc0802df60 0x55dc083eae10 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f28240106a0 tx=0x7f2824004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b63c9ba3b12cb187 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:21.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.325+0000 7f2814ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:39:21.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.325+0000 7f2814ff9640 1 -- 172.21.15.100:0/4143947743 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2824024020 con 0x55dc0802df60 2024-07-26T13:39:21.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.325+0000 7f2814ff9640 1 -- 172.21.15.100:0/4143947743 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f2824020820 con 0x55dc0802df60 2024-07-26T13:39:21.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.325+0000 7f2839fc7740 1 -- 172.21.15.100:0/4143947743 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28280aa350 con 0x55dc0802df60 2024-07-26T13:39:21.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.325+0000 7f2814ff9640 1 -- 172.21.15.100:0/4143947743 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f282402f050 con 0x55dc0802df60 2024-07-26T13:39:21.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.325+0000 7f2839fc7740 10 client.15180 fetch_fsmap learned FSMap version 25 2024-07-26T13:39:21.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.325+0000 7f2839fc7740 1 -- 172.21.15.100:0/4143947743 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dc083edbb0 con 0x55dc0802df60 2024-07-26T13:39:21.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.325+0000 7f2814ff9640 1 -- 172.21.15.100:0/4143947743 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28280aa350 con 0x55dc0802df60 2024-07-26T13:39:21.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.326+0000 7f2814ff9640 1 -- 172.21.15.100:0/4143947743 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f282400fa20 con 0x55dc0802df60 2024-07-26T13:39:21.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.326+0000 7f2839fc7740 10 client.15180 fetch_fsmap finished waiting for FSMap version 25 2024-07-26T13:39:21.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.326+0000 7f2839fc7740 10 client.15180 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-26T13:39:21.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.326+0000 7f2839fc7740 20 client.15180 populate_metadata read hostname 'smithi100' 2024-07-26T13:39:21.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.326+0000 7f2839fc7740 1 --2- 172.21.15.100:0/4143947743 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55dc07fec8e0 0x55dc083f16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:21.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.326+0000 7f2839fc7740 4 client.15180 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:39:21.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.326+0000 7f2839fc7740 1 -- 172.21.15.100:0/4143947743 --> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc0834b890 con 0x55dc07fec8e0 2024-07-26T13:39:21.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.326+0000 7f28317df640 1 --2- 172.21.15.100:0/4143947743 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55dc07fec8e0 0x55dc083f16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:21.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.327+0000 7f2814ff9640 10 client.15180 ms_handle_connect on v2:172.21.15.138:6838/1580925471 2024-07-26T13:39:21.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.327+0000 7f28317df640 1 --2- 172.21.15.100:0/4143947743 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55dc07fec8e0 0x55dc083f16b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9324376e90d7f3ea server_cookie=92130b74f8609033 in_seq=0 out_seq=0 2024-07-26T13:39:21.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.327+0000 7f2814ff9640 1 -- 172.21.15.100:0/4143947743 <== mds.1 v2:172.21.15.138:6838/1580925471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dc0834b890 con 0x55dc07fec8e0 2024-07-26T13:39:21.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.327+0000 7f2814ff9640 10 client.15180 handle_command_reply: tid=0 2024-07-26T13:39:21.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.328+0000 7f2839fc7740 1 client.15180 shutdown 2024-07-26T13:39:21.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.328+0000 7f2839fc7740 1 -- 172.21.15.100:0/4143947743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc0802df60 msgr2=0x55dc083eae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:21.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.328+0000 7f2839fc7740 1 --2- 172.21.15.100:0/4143947743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc0802df60 0x55dc083eae10 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f28240106a0 tx=0x7f2824004a40 comp rx=0 tx=0).stop 2024-07-26T13:39:21.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.328+0000 7f2839fc7740 1 -- 172.21.15.100:0/4143947743 shutdown_connections 2024-07-26T13:39:21.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.328+0000 7f2839fc7740 1 --2- 172.21.15.100:0/4143947743 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55dc07fec8e0 0x55dc083f16b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:21.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.328+0000 7f2839fc7740 1 --2- 172.21.15.100:0/4143947743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc0802df60 0x55dc083eae10 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:21.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.328+0000 7f2839fc7740 1 --2- 172.21.15.100:0/4143947743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc0800d280 0x55dc083df490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:21.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.328+0000 7f2814ff9640 0 client.15180 ms_handle_reset on v2:172.21.15.138:6838/1580925471 2024-07-26T13:39:21.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.328+0000 7f2839fc7740 1 --2- 172.21.15.100:0/4143947743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc080050a0 0x55dc083def50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:21.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.328+0000 7f2839fc7740 1 -- 172.21.15.100:0/4143947743 >> 172.21.15.100:0/4143947743 conn(0x55dc08022350 msgr2=0x55dc083d6530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:21.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.328+0000 7f2839fc7740 1 -- 172.21.15.100:0/4143947743 shutdown_connections 2024-07-26T13:39:21.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.328+0000 7f2839fc7740 1 -- 172.21.15.100:0/4143947743 wait complete. 2024-07-26T13:39:21.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.328+0000 7f2839fc7740 20 client.15180 trim_cache size 0 max 16384 2024-07-26T13:39:21.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.346+0000 7f2839fc7740 1 Processor -- start 2024-07-26T13:39:21.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.346+0000 7f2839fc7740 1 -- start start 2024-07-26T13:39:21.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.346+0000 7f2839fc7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc07fec8e0 0x55dc083ead80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:21.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.346+0000 7f2839fc7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc080050a0 0x55dc083de240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:21.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.346+0000 7f2839fc7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc0800d280 0x55dc083d9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:21.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.346+0000 7f2839fc7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55dc083ef9f0 con 0x55dc07fec8e0 2024-07-26T13:39:21.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.346+0000 7f2839fc7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dc083f00c0 con 0x55dc0800d280 2024-07-26T13:39:21.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.346+0000 7f2839fc7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55dc083f0790 con 0x55dc080050a0 2024-07-26T13:39:21.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.346+0000 7f28317df640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc07fec8e0 0x55dc083ead80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:21.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.346+0000 7f28317df640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc07fec8e0 0x55dc083ead80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60504/0 (socket says 172.21.15.100:60504) 2024-07-26T13:39:21.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.346+0000 7f28317df640 1 -- 172.21.15.100:0/519862377 learned_addr learned my addr 172.21.15.100:0/519862377 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:21.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.347+0000 7f28317df640 1 -- 172.21.15.100:0/519862377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc080050a0 msgr2=0x55dc083de240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:21.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.347+0000 7f28317df640 1 --2- 172.21.15.100:0/519862377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc080050a0 0x55dc083de240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:21.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.347+0000 7f28317df640 1 -- 172.21.15.100:0/519862377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc0800d280 msgr2=0x55dc083d9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:21.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.347+0000 7f2831fe0640 1 --2- 172.21.15.100:0/519862377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc0800d280 0x55dc083d9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:21.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.347+0000 7f28317df640 1 --2- 172.21.15.100:0/519862377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc0800d280 0x55dc083d9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:21.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.347+0000 7f28317df640 1 -- 172.21.15.100:0/519862377 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc083d5690 con 0x55dc07fec8e0 2024-07-26T13:39:21.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.347+0000 7f2831fe0640 1 --2- 172.21.15.100:0/519862377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc0800d280 0x55dc083d9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:39:21.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.347+0000 7f28317df640 1 --2- 172.21.15.100:0/519862377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc07fec8e0 0x55dc083ead80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2828019fd0 tx=0x7f28280e9530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7066325fc9536b4a server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:21.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.347+0000 7f2816ffd640 1 -- 172.21.15.100:0/519862377 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2828020040 con 0x55dc07fec8e0 2024-07-26T13:39:21.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.347+0000 7f2816ffd640 1 -- 172.21.15.100:0/519862377 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28280e8070 con 0x55dc07fec8e0 2024-07-26T13:39:21.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.347+0000 7f2816ffd640 1 -- 172.21.15.100:0/519862377 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28280a9fb0 con 0x55dc07fec8e0 2024-07-26T13:39:21.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.348+0000 7f2839fc7740 1 -- 172.21.15.100:0/519862377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc07fec8e0 msgr2=0x55dc083ead80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:21.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.348+0000 7f2839fc7740 1 --2- 172.21.15.100:0/519862377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc07fec8e0 0x55dc083ead80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2828019fd0 tx=0x7f28280e9530 comp rx=0 tx=0).stop 2024-07-26T13:39:21.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.348+0000 7f2839fc7740 1 -- 172.21.15.100:0/519862377 shutdown_connections 2024-07-26T13:39:21.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.348+0000 7f2839fc7740 1 --2- 172.21.15.100:0/519862377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc0800d280 0x55dc083d9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:21.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.348+0000 7f2839fc7740 1 --2- 172.21.15.100:0/519862377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc080050a0 0x55dc083de240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:21.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.348+0000 7f2839fc7740 1 --2- 172.21.15.100:0/519862377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc07fec8e0 0x55dc083ead80 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:21.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.348+0000 7f2839fc7740 1 -- 172.21.15.100:0/519862377 >> 172.21.15.100:0/519862377 conn(0x55dc08022350 msgr2=0x55dc083def30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:21.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.348+0000 7f2839fc7740 1 -- 172.21.15.100:0/519862377 shutdown_connections 2024-07-26T13:39:21.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.348+0000 7f2839fc7740 1 -- 172.21.15.100:0/519862377 wait complete. 2024-07-26T13:39:21.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.348+0000 7f2839fc7740 1 -- start start 2024-07-26T13:39:21.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.348+0000 7f2839fc7740 1 Processor -- start 2024-07-26T13:39:21.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.349+0000 7f2839fc7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc07fec8e0 0x55dc083f2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:21.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.349+0000 7f2839fc7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc080050a0 0x55dc083f3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:21.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.349+0000 7f2839fc7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc0800d280 0x55dc083f8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:21.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.349+0000 7f2839fc7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55dc083f1530 con 0x55dc080050a0 2024-07-26T13:39:21.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.349+0000 7f2839fc7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dc083f0e60 con 0x55dc0800d280 2024-07-26T13:39:21.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.349+0000 7f2839fc7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55dc083f1c00 con 0x55dc07fec8e0 2024-07-26T13:39:21.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.349+0000 7f2839fc7740 10 client.0 _ll_register_callbacks cb 0x7f282400eec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:39:21.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.349+0000 7f2830fde640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc080050a0 0x55dc083f3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:21.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.349+0000 7f2830fde640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc080050a0 0x55dc083f3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60506/0 (socket says 172.21.15.100:60506) 2024-07-26T13:39:21.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.349+0000 7f2830fde640 1 -- 172.21.15.100:0/498667598 learned_addr learned my addr 172.21.15.100:0/498667598 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:21.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.349+0000 7f2831fe0640 1 --2- 172.21.15.100:0/498667598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc0800d280 0x55dc083f8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:21.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.349+0000 7f28317df640 1 --2- 172.21.15.100:0/498667598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc07fec8e0 0x55dc083f2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:21.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.349+0000 7f2830fde640 1 -- 172.21.15.100:0/498667598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc07fec8e0 msgr2=0x55dc083f2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:21.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.349+0000 7f2830fde640 1 --2- 172.21.15.100:0/498667598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc07fec8e0 0x55dc083f2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:21.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.349+0000 7f2830fde640 1 -- 172.21.15.100:0/498667598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc0800d280 msgr2=0x55dc083f8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:21.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.349+0000 7f2830fde640 1 --2- 172.21.15.100:0/498667598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc0800d280 0x55dc083f8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:21.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.349+0000 7f2830fde640 1 -- 172.21.15.100:0/498667598 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc083f8660 con 0x55dc080050a0 2024-07-26T13:39:21.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.349+0000 7f28317df640 1 --2- 172.21.15.100:0/498667598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc07fec8e0 0x55dc083f2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:39:21.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.349+0000 7f2830fde640 1 --2- 172.21.15.100:0/498667598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc080050a0 0x55dc083f3050 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f281c005940 tx=0x7f281c0317a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b73f754871782cd server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:21.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.349+0000 7f2814ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:39:21.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.350+0000 7f2814ff9640 1 -- 172.21.15.100:0/498667598 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281c035040 con 0x55dc080050a0 2024-07-26T13:39:21.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.350+0000 7f2839fc7740 1 -- 172.21.15.100:0/498667598 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28280e8070 con 0x55dc080050a0 2024-07-26T13:39:21.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.350+0000 7f2814ff9640 1 -- 172.21.15.100:0/498667598 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f281c004240 con 0x55dc080050a0 2024-07-26T13:39:21.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.350+0000 7f2814ff9640 1 -- 172.21.15.100:0/498667598 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f281c031910 con 0x55dc080050a0 2024-07-26T13:39:21.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.350+0000 7f2839fc7740 10 client.15186 fetch_fsmap learned FSMap version 25 2024-07-26T13:39:21.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.350+0000 7f2839fc7740 1 -- 172.21.15.100:0/498667598 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dc083f9a30 con 0x55dc080050a0 2024-07-26T13:39:21.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.350+0000 7f2814ff9640 1 -- 172.21.15.100:0/498667598 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28280e8070 con 0x55dc080050a0 2024-07-26T13:39:21.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.350+0000 7f2814ff9640 1 -- 172.21.15.100:0/498667598 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f281c04c070 con 0x55dc080050a0 2024-07-26T13:39:21.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.350+0000 7f2839fc7740 10 client.15186 fetch_fsmap finished waiting for FSMap version 25 2024-07-26T13:39:21.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.350+0000 7f2839fc7740 10 client.15186 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-26T13:39:21.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.350+0000 7f2839fc7740 20 client.15186 populate_metadata read hostname 'smithi100' 2024-07-26T13:39:21.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.350+0000 7f2839fc7740 1 --2- 172.21.15.100:0/498667598 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55dc0802df60 0x55dc0845d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:21.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.350+0000 7f2839fc7740 4 client.15186 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-26T13:39:21.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.350+0000 7f2839fc7740 1 -- 172.21.15.100:0/498667598 --> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] -- command(tid 0: {"prefix": "status"}) -- 0x55dc08347330 con 0x55dc0802df60 2024-07-26T13:39:21.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.351+0000 7f28317df640 1 --2- 172.21.15.100:0/498667598 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55dc0802df60 0x55dc0845d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:21.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.351+0000 7f28317df640 1 --2- 172.21.15.100:0/498667598 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55dc0802df60 0x55dc0845d490 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=39d2532b575c0baf server_cookie=2f5d1439d2423aa6 in_seq=0 out_seq=0 2024-07-26T13:39:21.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.351+0000 7f2814ff9640 10 client.15186 ms_handle_connect on v2:172.21.15.138:6838/1580925471 2024-07-26T13:39:21.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.351+0000 7f2814ff9640 1 -- 172.21.15.100:0/498667598 <== mds.1 v2:172.21.15.138:6838/1580925471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55dc08347330 con 0x55dc0802df60 2024-07-26T13:39:21.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.351+0000 7f2814ff9640 10 client.15186 handle_command_reply: tid=0 2024-07-26T13:39:21.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.351+0000 7f2839fc7740 1 client.15186 shutdown 2024-07-26T13:39:21.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.351+0000 7f2839fc7740 1 -- 172.21.15.100:0/498667598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc080050a0 msgr2=0x55dc083f3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:21.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.351+0000 7f2839fc7740 1 --2- 172.21.15.100:0/498667598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc080050a0 0x55dc083f3050 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f281c005940 tx=0x7f281c0317a0 comp rx=0 tx=0).stop 2024-07-26T13:39:21.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.351+0000 7f2839fc7740 1 -- 172.21.15.100:0/498667598 shutdown_connections 2024-07-26T13:39:21.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.351+0000 7f2839fc7740 1 --2- 172.21.15.100:0/498667598 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55dc0802df60 0x55dc0845d490 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:21.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.351+0000 7f2839fc7740 1 --2- 172.21.15.100:0/498667598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dc0800d280 0x55dc083f8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:21.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.351+0000 7f2839fc7740 1 --2- 172.21.15.100:0/498667598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dc080050a0 0x55dc083f3050 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:21.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.351+0000 7f2814ff9640 0 client.15186 ms_handle_reset on v2:172.21.15.138:6838/1580925471 2024-07-26T13:39:21.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.351+0000 7f2839fc7740 1 --2- 172.21.15.100:0/498667598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dc07fec8e0 0x55dc083f2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:21.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.351+0000 7f2839fc7740 1 -- 172.21.15.100:0/498667598 >> 172.21.15.100:0/498667598 conn(0x55dc08022350 msgr2=0x55dc083d9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:21.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.351+0000 7f2839fc7740 1 -- 172.21.15.100:0/498667598 shutdown_connections 2024-07-26T13:39:21.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.352+0000 7f2839fc7740 1 -- 172.21.15.100:0/498667598 wait complete. 2024-07-26T13:39:21.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.352+0000 7f2839fc7740 20 client.15186 trim_cache size 0 max 16384 2024-07-26T13:39:21.358 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:39:21.359 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "bf83e2e6-4b52-11ef-bcb7-c7b262605968", 2024-07-26T13:39:21.359 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 1, 2024-07-26T13:39:21.359 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 24605, 2024-07-26T13:39:21.359 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-07-26T13:39:21.359 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-07-26T13:39:21.359 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-07-26T13:39:21.359 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 127.828560002, 2024-07-26T13:39:21.359 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 25, 2024-07-26T13:39:21.359 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 81, 2024-07-26T13:39:21.359 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 81, 2024-07-26T13:39:21.359 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 157.150850429 2024-07-26T13:39:21.359 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:39:21.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.352+0000 7f2833a43640 1 -- 172.21.15.100:0/1876543069 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f28180b3c50 msgr2=0x7f28180b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:21.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.352+0000 7f2833a43640 1 --2- 172.21.15.100:0/1876543069 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f28180b3c50 0x7f28180b6160 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2824012930 tx=0x7f2824004ab0 comp rx=0 tx=0).stop 2024-07-26T13:39:21.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.352+0000 7f2833a43640 1 -- 172.21.15.100:0/1876543069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f282c102510 msgr2=0x7f282c198e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:21.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.352+0000 7f2833a43640 1 --2- 172.21.15.100:0/1876543069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f282c102510 0x7f282c198e80 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f282800d950 tx=0x7f282800de10 comp rx=0 tx=0).stop 2024-07-26T13:39:21.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.352+0000 7f2833a43640 1 -- 172.21.15.100:0/1876543069 shutdown_connections 2024-07-26T13:39:21.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.352+0000 7f2833a43640 1 --2- 172.21.15.100:0/1876543069 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f28180b3c50 0x7f28180b6160 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:21.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.352+0000 7f2833a43640 1 --2- 172.21.15.100:0/1876543069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f282c1087a0 0x7f282c19e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:21.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.352+0000 7f2833a43640 1 --2- 172.21.15.100:0/1876543069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f282c107e50 0x7f282c199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:21.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.352+0000 7f2833a43640 1 --2- 172.21.15.100:0/1876543069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f282c102510 0x7f282c198e80 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:21.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.352+0000 7f2833a43640 1 -- 172.21.15.100:0/1876543069 >> 172.21.15.100:0/1876543069 conn(0x7f282c0791a0 msgr2=0x7f282c105fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:21.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.352+0000 7f2833a43640 1 -- 172.21.15.100:0/1876543069 shutdown_connections 2024-07-26T13:39:21.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:21.352+0000 7f2833a43640 1 -- 172.21.15.100:0/1876543069 wait complete. 2024-07-26T13:39:21.740 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:21 smithi100 ceph-mon[32599]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:39:21.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:21 smithi167 ceph-mon[37140]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:39:21.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:21 smithi138 ceph-mon[36954]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:39:22.088 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.i status 2024-07-26T13:39:22.343 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:39:23.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:23 smithi100 ceph-mon[32599]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:39:23.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:23 smithi167 ceph-mon[37140]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:39:23.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:23 smithi138 ceph-mon[36954]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:39:24.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.069+0000 7f595f2a0640 1 -- 172.21.15.100:0/64435875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5958107dd0 msgr2=0x7f59581081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:24.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.069+0000 7f595f2a0640 1 --2- 172.21.15.100:0/64435875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5958107dd0 0x7f59581081e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5948009900 tx=0x7f594802a410 comp rx=0 tx=0).stop 2024-07-26T13:39:24.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.069+0000 7f595f2a0640 1 -- 172.21.15.100:0/64435875 shutdown_connections 2024-07-26T13:39:24.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.069+0000 7f595f2a0640 1 --2- 172.21.15.100:0/64435875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5958108720 0x7f595810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:24.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.069+0000 7f595f2a0640 1 --2- 172.21.15.100:0/64435875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5958107dd0 0x7f59581081e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:24.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.069+0000 7f595f2a0640 1 --2- 172.21.15.100:0/64435875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5958102490 0x7f59581028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:24.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.069+0000 7f595f2a0640 1 -- 172.21.15.100:0/64435875 >> 172.21.15.100:0/64435875 conn(0x7f5958079190 msgr2=0x7f59580795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:24.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.069+0000 7f595f2a0640 1 -- 172.21.15.100:0/64435875 shutdown_connections 2024-07-26T13:39:24.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.069+0000 7f595f2a0640 1 -- 172.21.15.100:0/64435875 wait complete. 2024-07-26T13:39:24.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.070+0000 7f595f2a0640 1 Processor -- start 2024-07-26T13:39:24.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.070+0000 7f595f2a0640 1 -- start start 2024-07-26T13:39:24.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.070+0000 7f595f2a0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5958102490 0x7f5958198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:24.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.070+0000 7f595f2a0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5958107dd0 0x7f5958199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:24.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.070+0000 7f595f2a0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5958108720 0x7f595819e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:24.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.070+0000 7f595f2a0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f59581123b0 con 0x7f5958102490 2024-07-26T13:39:24.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.070+0000 7f595f2a0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5958112230 con 0x7f5958108720 2024-07-26T13:39:24.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.070+0000 7f595f2a0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5958112530 con 0x7f5958107dd0 2024-07-26T13:39:24.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.070+0000 7f595d03c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5958102490 0x7f5958198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:24.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.070+0000 7f595d03c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5958102490 0x7f5958198ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60520/0 (socket says 172.21.15.100:60520) 2024-07-26T13:39:24.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.070+0000 7f595d03c640 1 -- 172.21.15.100:0/2818072527 learned_addr learned my addr 172.21.15.100:0/2818072527 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:24.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.070+0000 7f595c83b640 1 --2- 172.21.15.100:0/2818072527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5958107dd0 0x7f5958199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:24.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.070+0000 7f595d83d640 1 --2- 172.21.15.100:0/2818072527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5958108720 0x7f595819e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:24.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.071+0000 7f595d03c640 1 -- 172.21.15.100:0/2818072527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5958107dd0 msgr2=0x7f5958199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:24.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.071+0000 7f595d03c640 1 --2- 172.21.15.100:0/2818072527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5958107dd0 0x7f5958199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:24.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.071+0000 7f595d03c640 1 -- 172.21.15.100:0/2818072527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5958108720 msgr2=0x7f595819e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:24.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.071+0000 7f595d03c640 1 --2- 172.21.15.100:0/2818072527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5958108720 0x7f595819e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:24.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.071+0000 7f595d03c640 1 -- 172.21.15.100:0/2818072527 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f595819ed70 con 0x7f5958102490 2024-07-26T13:39:24.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.071+0000 7f595d03c640 1 --2- 172.21.15.100:0/2818072527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5958102490 0x7f5958198ec0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f595400c920 tx=0x7f595400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:24.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.071+0000 7f594e7fc640 1 -- 172.21.15.100:0/2818072527 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5954007c70 con 0x7f5958102490 2024-07-26T13:39:24.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.071+0000 7f595f2a0640 1 -- 172.21.15.100:0/2818072527 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f595819f060 con 0x7f5958102490 2024-07-26T13:39:24.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.071+0000 7f594e7fc640 1 -- 172.21.15.100:0/2818072527 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5954007e10 con 0x7f5958102490 2024-07-26T13:39:24.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.071+0000 7f594e7fc640 1 -- 172.21.15.100:0/2818072527 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5954018820 con 0x7f5958102490 2024-07-26T13:39:24.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.071+0000 7f595f2a0640 1 -- 172.21.15.100:0/2818072527 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59581a5880 con 0x7f5958102490 2024-07-26T13:39:24.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.074+0000 7f5965824740 1 Processor -- start 2024-07-26T13:39:24.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.074+0000 7f5965824740 1 -- start start 2024-07-26T13:39:24.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.074+0000 7f594e7fc640 1 -- 172.21.15.100:0/2818072527 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f595400f450 con 0x7f5958102490 2024-07-26T13:39:24.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.074+0000 7f5965824740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556025485f60 0x55602582b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:24.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.074+0000 7f5965824740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55602545d0a0 0x55602582b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:24.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.074+0000 7f5965824740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556025465280 0x55602582be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:24.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.074+0000 7f5965824740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5560257a1940 con 0x556025485f60 2024-07-26T13:39:24.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.074+0000 7f5965824740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556025790e40 con 0x55602545d0a0 2024-07-26T13:39:24.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.074+0000 7f5965824740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556025794e80 con 0x556025465280 2024-07-26T13:39:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.074+0000 7f595d83d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556025485f60 0x55602582b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.074+0000 7f595d83d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556025485f60 0x55602582b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60522/0 (socket says 172.21.15.100:60522) 2024-07-26T13:39:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.074+0000 7f595d83d640 1 -- 172.21.15.100:0/2297566993 learned_addr learned my addr 172.21.15.100:0/2297566993 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.074+0000 7f595d03c640 1 --2- 172.21.15.100:0/2297566993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55602545d0a0 0x55602582b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.074+0000 7f595c83b640 1 --2- 172.21.15.100:0/2297566993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556025465280 0x55602582be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.075+0000 7f595d83d640 1 -- 172.21.15.100:0/2297566993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556025465280 msgr2=0x55602582be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.075+0000 7f595d83d640 1 --2- 172.21.15.100:0/2297566993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556025465280 0x55602582be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.075+0000 7f595d83d640 1 -- 172.21.15.100:0/2297566993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55602545d0a0 msgr2=0x55602582b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.075+0000 7f595d83d640 1 --2- 172.21.15.100:0/2297566993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55602545d0a0 0x55602582b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.075+0000 7f595d83d640 1 -- 172.21.15.100:0/2297566993 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556025737a60 con 0x556025485f60 2024-07-26T13:39:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.075+0000 7f595d03c640 1 --2- 172.21.15.100:0/2297566993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55602545d0a0 0x55602582b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:39:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.075+0000 7f595d83d640 1 --2- 172.21.15.100:0/2297566993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556025485f60 0x55602582b3a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5950009e60 tx=0x7f5950010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e8b8f24beba0da server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:24.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.075+0000 7f59437fe640 1 -- 172.21.15.100:0/2297566993 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950016070 con 0x556025485f60 2024-07-26T13:39:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.075+0000 7f59437fe640 1 -- 172.21.15.100:0/2297566993 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5950004d20 con 0x556025485f60 2024-07-26T13:39:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.075+0000 7f59437fe640 1 -- 172.21.15.100:0/2297566993 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950012740 con 0x556025485f60 2024-07-26T13:39:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.075+0000 7f594e7fc640 1 --2- 172.21.15.100:0/2818072527 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f59440b3c50 0x7f59440b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.075+0000 7f595d03c640 1 --2- 172.21.15.100:0/2818072527 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f59440b3c50 0x7f59440b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.075+0000 7f594e7fc640 1 -- 172.21.15.100:0/2818072527 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f59540e3e90 con 0x7f5958102490 2024-07-26T13:39:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.076+0000 7f5965824740 1 -- 172.21.15.100:0/2297566993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556025485f60 msgr2=0x55602582b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.076+0000 7f595d03c640 1 --2- 172.21.15.100:0/2818072527 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f59440b3c50 0x7f59440b6160 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x55602582a020 tx=0x7f5954000bc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.076+0000 7f5965824740 1 --2- 172.21.15.100:0/2297566993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556025485f60 0x55602582b3a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5950009e60 tx=0x7f5950010740 comp rx=0 tx=0).stop 2024-07-26T13:39:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.076+0000 7f5965824740 1 -- 172.21.15.100:0/2297566993 shutdown_connections 2024-07-26T13:39:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.076+0000 7f5965824740 1 --2- 172.21.15.100:0/2297566993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556025465280 0x55602582be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.076+0000 7f5965824740 1 --2- 172.21.15.100:0/2297566993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55602545d0a0 0x55602582b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.076+0000 7f5965824740 1 --2- 172.21.15.100:0/2297566993 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556025485f60 0x55602582b3a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.076+0000 7f5965824740 1 -- 172.21.15.100:0/2297566993 >> 172.21.15.100:0/2297566993 conn(0x55602547a350 msgr2=0x556025829290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.076+0000 7f5965824740 1 -- 172.21.15.100:0/2297566993 shutdown_connections 2024-07-26T13:39:24.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.076+0000 7f5965824740 1 -- 172.21.15.100:0/2297566993 wait complete. 2024-07-26T13:39:24.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.076+0000 7f5965824740 1 -- start start 2024-07-26T13:39:24.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.077+0000 7f5965824740 1 Processor -- start 2024-07-26T13:39:24.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.077+0000 7f5965824740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55602545d0a0 0x556025837020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:24.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.077+0000 7f5965824740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556025465280 0x5560258375c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:24.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.077+0000 7f5965824740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556025485f60 0x556025842ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:24.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.077+0000 7f5965824740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5560255f45f0 con 0x556025465280 2024-07-26T13:39:24.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.077+0000 7f5965824740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5560257338b0 con 0x55602545d0a0 2024-07-26T13:39:24.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.077+0000 7f5965824740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556025839b80 con 0x556025485f60 2024-07-26T13:39:24.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.077+0000 7f5965824740 10 client.0 _ll_register_callbacks cb 0x55602573d7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:39:24.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.077+0000 7f595c83b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556025465280 0x5560258375c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:24.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.077+0000 7f595c83b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556025465280 0x5560258375c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60528/0 (socket says 172.21.15.100:60528) 2024-07-26T13:39:24.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.077+0000 7f595c83b640 1 -- 172.21.15.100:0/1500618867 learned_addr learned my addr 172.21.15.100:0/1500618867 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:24.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.077+0000 7f595d83d640 1 --2- 172.21.15.100:0/1500618867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55602545d0a0 0x556025837020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:24.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.077+0000 7f595d83d640 1 --2- 172.21.15.100:0/1500618867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556025485f60 0x556025842ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:24.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.078+0000 7f595c83b640 1 -- 172.21.15.100:0/1500618867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556025485f60 msgr2=0x556025842ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:24.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.078+0000 7f595c83b640 1 --2- 172.21.15.100:0/1500618867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556025485f60 0x556025842ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:24.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.078+0000 7f595c83b640 1 -- 172.21.15.100:0/1500618867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55602545d0a0 msgr2=0x556025837020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:24.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.078+0000 7f595c83b640 1 --2- 172.21.15.100:0/1500618867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55602545d0a0 0x556025837020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:24.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.078+0000 7f595c83b640 1 -- 172.21.15.100:0/1500618867 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560254f1540 con 0x556025465280 2024-07-26T13:39:24.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.078+0000 7f595d83d640 1 --2- 172.21.15.100:0/1500618867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556025485f60 0x556025842ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:39:24.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.078+0000 7f595c83b640 1 --2- 172.21.15.100:0/1500618867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556025465280 0x5560258375c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f595806b380 tx=0x7f59480314c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9bca2cc57d8d263 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:24.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.078+0000 7f5940ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:39:24.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.078+0000 7f5940ff9640 1 -- 172.21.15.100:0/1500618867 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5948035040 con 0x556025465280 2024-07-26T13:39:24.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.078+0000 7f5940ff9640 1 -- 172.21.15.100:0/1500618867 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f59480317b0 con 0x556025465280 2024-07-26T13:39:24.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.078+0000 7f5965824740 1 -- 172.21.15.100:0/1500618867 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5950004d20 con 0x556025465280 2024-07-26T13:39:24.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.078+0000 7f5940ff9640 1 -- 172.21.15.100:0/1500618867 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5948004040 con 0x556025465280 2024-07-26T13:39:24.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.078+0000 7f5965824740 10 client.15198 fetch_fsmap learned FSMap version 25 2024-07-26T13:39:24.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.078+0000 7f5965824740 1 -- 172.21.15.100:0/1500618867 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556025845c40 con 0x556025465280 2024-07-26T13:39:24.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.078+0000 7f5940ff9640 1 -- 172.21.15.100:0/1500618867 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5950004d20 con 0x556025465280 2024-07-26T13:39:24.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.078+0000 7f5940ff9640 1 -- 172.21.15.100:0/1500618867 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5948004870 con 0x556025465280 2024-07-26T13:39:24.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.079+0000 7f5965824740 10 client.15198 fetch_fsmap finished waiting for FSMap version 25 2024-07-26T13:39:24.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.079+0000 7f5965824740 10 client.15198 resolve_mds: resolved name 'i' to daemon mds.i 2024-07-26T13:39:24.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.079+0000 7f5965824740 20 client.15198 populate_metadata read hostname 'smithi100' 2024-07-26T13:39:24.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.079+0000 7f5965824740 1 --2- 172.21.15.100:0/1500618867 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5560254448e0 0x556025849720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:24.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.079+0000 7f5965824740 4 client.15198 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:39:24.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.079+0000 7f5965824740 1 -- 172.21.15.100:0/1500618867 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560257a3890 con 0x5560254448e0 2024-07-26T13:39:24.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.079+0000 7f595d83d640 1 --2- 172.21.15.100:0/1500618867 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5560254448e0 0x556025849720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:24.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.079+0000 7f5940ff9640 10 client.15198 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:39:24.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.079+0000 7f595d83d640 1 --2- 172.21.15.100:0/1500618867 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5560254448e0 0x556025849720 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b36ec5bd8d4ba58 server_cookie=5fba1535e4a62c66 in_seq=0 out_seq=0 2024-07-26T13:39:24.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.080+0000 7f5940ff9640 1 -- 172.21.15.100:0/1500618867 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5560257a3890 con 0x5560254448e0 2024-07-26T13:39:24.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.080+0000 7f5940ff9640 10 client.15198 handle_command_reply: tid=0 2024-07-26T13:39:24.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.080+0000 7f5965824740 1 client.15198 shutdown 2024-07-26T13:39:24.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.080+0000 7f5965824740 1 -- 172.21.15.100:0/1500618867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556025465280 msgr2=0x5560258375c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:24.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.081+0000 7f5965824740 1 --2- 172.21.15.100:0/1500618867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556025465280 0x5560258375c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f595806b380 tx=0x7f59480314c0 comp rx=0 tx=0).stop 2024-07-26T13:39:24.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.081+0000 7f5965824740 1 -- 172.21.15.100:0/1500618867 shutdown_connections 2024-07-26T13:39:24.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.081+0000 7f5965824740 1 --2- 172.21.15.100:0/1500618867 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5560254448e0 0x556025849720 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:24.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.081+0000 7f5965824740 1 --2- 172.21.15.100:0/1500618867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556025485f60 0x556025842ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:24.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.081+0000 7f5965824740 1 --2- 172.21.15.100:0/1500618867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556025465280 0x5560258375c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:24.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.081+0000 7f5965824740 1 --2- 172.21.15.100:0/1500618867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55602545d0a0 0x556025837020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:24.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.081+0000 7f5940ff9640 0 client.15198 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:39:24.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.081+0000 7f5965824740 1 -- 172.21.15.100:0/1500618867 >> 172.21.15.100:0/1500618867 conn(0x55602547a350 msgr2=0x55602582e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:24.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.081+0000 7f5965824740 1 -- 172.21.15.100:0/1500618867 shutdown_connections 2024-07-26T13:39:24.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.081+0000 7f5965824740 1 -- 172.21.15.100:0/1500618867 wait complete. 2024-07-26T13:39:24.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.081+0000 7f5965824740 20 client.15198 trim_cache size 0 max 16384 2024-07-26T13:39:24.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.099+0000 7f5965824740 1 Processor -- start 2024-07-26T13:39:24.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.099+0000 7f5965824740 1 -- start start 2024-07-26T13:39:24.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.099+0000 7f5965824740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55602545d0a0 0x556025830bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:24.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.099+0000 7f5965824740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556025465280 0x556025842e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:24.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.099+0000 7f5965824740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556025485f60 0x556025823ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:24.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.099+0000 7f5965824740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x556025847a60 con 0x556025485f60 2024-07-26T13:39:24.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.099+0000 7f5965824740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556025848130 con 0x556025465280 2024-07-26T13:39:24.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.099+0000 7f5965824740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556025848800 con 0x55602545d0a0 2024-07-26T13:39:24.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.099+0000 7f595d83d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556025485f60 0x556025823ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:24.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.099+0000 7f595d83d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556025485f60 0x556025823ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60542/0 (socket says 172.21.15.100:60542) 2024-07-26T13:39:24.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.099+0000 7f595d83d640 1 -- 172.21.15.100:0/4108149554 learned_addr learned my addr 172.21.15.100:0/4108149554 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:24.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.099+0000 7f595d83d640 1 --2- 172.21.15.100:0/4108149554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55602545d0a0 0x556025830bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:24.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.099+0000 7f595c83b640 1 --2- 172.21.15.100:0/4108149554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556025465280 0x556025842e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:24.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.099+0000 7f595d83d640 1 -- 172.21.15.100:0/4108149554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55602545d0a0 msgr2=0x556025830bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:24.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.099+0000 7f595d83d640 1 --2- 172.21.15.100:0/4108149554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55602545d0a0 0x556025830bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:24.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.099+0000 7f595d83d640 1 -- 172.21.15.100:0/4108149554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556025465280 msgr2=0x556025842e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:24.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.099+0000 7f595d83d640 1 --2- 172.21.15.100:0/4108149554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556025465280 0x556025842e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:24.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.099+0000 7f595d83d640 1 -- 172.21.15.100:0/4108149554 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560258380e0 con 0x556025485f60 2024-07-26T13:39:24.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.100+0000 7f595d83d640 1 --2- 172.21.15.100:0/4108149554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556025485f60 0x556025823ed0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5950010c40 tx=0x7f5950005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6adcc0435a33d3df server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:24.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.100+0000 7f5942ffd640 1 -- 172.21.15.100:0/4108149554 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595002f070 con 0x556025485f60 2024-07-26T13:39:24.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.100+0000 7f5942ffd640 1 -- 172.21.15.100:0/4108149554 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5950028790 con 0x556025485f60 2024-07-26T13:39:24.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.100+0000 7f5942ffd640 1 -- 172.21.15.100:0/4108149554 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950016040 con 0x556025485f60 2024-07-26T13:39:24.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.101+0000 7f5965824740 1 -- 172.21.15.100:0/4108149554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556025485f60 msgr2=0x556025823ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:24.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.101+0000 7f5965824740 1 --2- 172.21.15.100:0/4108149554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556025485f60 0x556025823ed0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5950010c40 tx=0x7f5950005a50 comp rx=0 tx=0).stop 2024-07-26T13:39:24.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.101+0000 7f5965824740 1 -- 172.21.15.100:0/4108149554 shutdown_connections 2024-07-26T13:39:24.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.101+0000 7f5965824740 1 --2- 172.21.15.100:0/4108149554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556025485f60 0x556025823ed0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:24.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.101+0000 7f5965824740 1 --2- 172.21.15.100:0/4108149554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556025465280 0x556025842e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:24.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.101+0000 7f5965824740 1 --2- 172.21.15.100:0/4108149554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55602545d0a0 0x556025830bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:24.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.101+0000 7f5965824740 1 -- 172.21.15.100:0/4108149554 >> 172.21.15.100:0/4108149554 conn(0x55602547a350 msgr2=0x5560258304b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:24.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.101+0000 7f5965824740 1 -- 172.21.15.100:0/4108149554 shutdown_connections 2024-07-26T13:39:24.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.101+0000 7f5965824740 1 -- 172.21.15.100:0/4108149554 wait complete. 2024-07-26T13:39:24.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.101+0000 7f5965824740 1 -- start start 2024-07-26T13:39:24.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.101+0000 7f5965824740 1 Processor -- start 2024-07-26T13:39:24.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.102+0000 7f5965824740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55602545d0a0 0x55602584ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:24.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.102+0000 7f5965824740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556025465280 0x55602584b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:24.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.102+0000 7f5965824740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556025485f60 0x5560258b09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:24.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.102+0000 7f5965824740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5560258495a0 con 0x556025465280 2024-07-26T13:39:24.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.102+0000 7f5965824740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556025848ed0 con 0x55602545d0a0 2024-07-26T13:39:24.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.102+0000 7f5965824740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556025849c70 con 0x556025485f60 2024-07-26T13:39:24.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.102+0000 7f5965824740 10 client.0 _ll_register_callbacks cb 0x7f5948032cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:39:24.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.102+0000 7f595c83b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556025465280 0x55602584b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:24.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.102+0000 7f595c83b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556025465280 0x55602584b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60548/0 (socket says 172.21.15.100:60548) 2024-07-26T13:39:24.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.102+0000 7f595c83b640 1 -- 172.21.15.100:0/89060916 learned_addr learned my addr 172.21.15.100:0/89060916 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:24.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.102+0000 7f595d83d640 1 --2- 172.21.15.100:0/89060916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55602545d0a0 0x55602584ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:24.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.102+0000 7f595d83d640 1 --2- 172.21.15.100:0/89060916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556025485f60 0x5560258b09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:24.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.102+0000 7f595c83b640 1 -- 172.21.15.100:0/89060916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556025485f60 msgr2=0x5560258b09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:24.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.102+0000 7f595c83b640 1 --2- 172.21.15.100:0/89060916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556025485f60 0x5560258b09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:24.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.102+0000 7f595c83b640 1 -- 172.21.15.100:0/89060916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55602545d0a0 msgr2=0x55602584ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:24.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.102+0000 7f595c83b640 1 --2- 172.21.15.100:0/89060916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55602545d0a0 0x55602584ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:24.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.102+0000 7f595c83b640 1 -- 172.21.15.100:0/89060916 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560258b0ef0 con 0x556025465280 2024-07-26T13:39:24.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.102+0000 7f595c83b640 1 --2- 172.21.15.100:0/89060916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556025465280 0x55602584b290 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5948004c40 tx=0x7f5948005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97173e8991bad6c5 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:24.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.102+0000 7f5940ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:39:24.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.102+0000 7f5940ff9640 1 -- 172.21.15.100:0/89060916 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5948035030 con 0x556025465280 2024-07-26T13:39:24.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.102+0000 7f5940ff9640 1 -- 172.21.15.100:0/89060916 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f5948003a30 con 0x556025465280 2024-07-26T13:39:24.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.103+0000 7f5965824740 1 -- 172.21.15.100:0/89060916 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5950028790 con 0x556025465280 2024-07-26T13:39:24.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.103+0000 7f5940ff9640 1 -- 172.21.15.100:0/89060916 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5948052040 con 0x556025465280 2024-07-26T13:39:24.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.103+0000 7f5965824740 10 client.15204 fetch_fsmap learned FSMap version 25 2024-07-26T13:39:24.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.103+0000 7f5965824740 1 -- 172.21.15.100:0/89060916 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5560258b2010 con 0x556025465280 2024-07-26T13:39:24.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.103+0000 7f5940ff9640 1 -- 172.21.15.100:0/89060916 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5950028790 con 0x556025465280 2024-07-26T13:39:24.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.103+0000 7f5940ff9640 1 -- 172.21.15.100:0/89060916 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f594805f020 con 0x556025465280 2024-07-26T13:39:24.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.103+0000 7f5965824740 10 client.15204 fetch_fsmap finished waiting for FSMap version 25 2024-07-26T13:39:24.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.103+0000 7f5965824740 10 client.15204 resolve_mds: resolved name 'i' to daemon mds.i 2024-07-26T13:39:24.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.103+0000 7f5965824740 20 client.15204 populate_metadata read hostname 'smithi100' 2024-07-26T13:39:24.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.103+0000 7f5965824740 1 --2- 172.21.15.100:0/89060916 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x556025419c90 0x5560258b5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:24.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.103+0000 7f5965824740 4 client.15204 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-26T13:39:24.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.103+0000 7f5965824740 1 -- 172.21.15.100:0/89060916 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "status"}) -- 0x55602579f330 con 0x556025419c90 2024-07-26T13:39:24.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.103+0000 7f595d83d640 1 --2- 172.21.15.100:0/89060916 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x556025419c90 0x5560258b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:24.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.104+0000 7f5940ff9640 10 client.15204 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:39:24.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.104+0000 7f595d83d640 1 --2- 172.21.15.100:0/89060916 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x556025419c90 0x5560258b5870 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f93bd5b7a9acc0fa server_cookie=5e5a89ae1aa17f53 in_seq=0 out_seq=0 2024-07-26T13:39:24.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.104+0000 7f5940ff9640 1 -- 172.21.15.100:0/89060916 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55602579f330 con 0x556025419c90 2024-07-26T13:39:24.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.104+0000 7f5940ff9640 10 client.15204 handle_command_reply: tid=0 2024-07-26T13:39:24.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.104+0000 7f5965824740 1 client.15204 shutdown 2024-07-26T13:39:24.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.104+0000 7f5965824740 1 -- 172.21.15.100:0/89060916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556025465280 msgr2=0x55602584b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:24.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.104+0000 7f5965824740 1 --2- 172.21.15.100:0/89060916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556025465280 0x55602584b290 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5948004c40 tx=0x7f5948005fd0 comp rx=0 tx=0).stop 2024-07-26T13:39:24.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.104+0000 7f5965824740 1 -- 172.21.15.100:0/89060916 shutdown_connections 2024-07-26T13:39:24.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.104+0000 7f5965824740 1 --2- 172.21.15.100:0/89060916 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x556025419c90 0x5560258b5870 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:24.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.104+0000 7f5965824740 1 --2- 172.21.15.100:0/89060916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556025485f60 0x5560258b09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:24.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.104+0000 7f5965824740 1 --2- 172.21.15.100:0/89060916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556025465280 0x55602584b290 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:24.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.104+0000 7f5965824740 1 --2- 172.21.15.100:0/89060916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55602545d0a0 0x55602584ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:24.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.104+0000 7f5940ff9640 0 client.15204 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:39:24.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.104+0000 7f5965824740 1 -- 172.21.15.100:0/89060916 >> 172.21.15.100:0/89060916 conn(0x55602547a350 msgr2=0x556025837000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:24.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.104+0000 7f5965824740 1 -- 172.21.15.100:0/89060916 shutdown_connections 2024-07-26T13:39:24.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.104+0000 7f5965824740 1 -- 172.21.15.100:0/89060916 wait complete. 2024-07-26T13:39:24.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.105+0000 7f5965824740 20 client.15204 trim_cache size 0 max 16384 2024-07-26T13:39:24.113 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:39:24.113 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "bf83e2e6-4b52-11ef-bcb7-c7b262605968", 2024-07-26T13:39:24.113 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 0, 2024-07-26T13:39:24.113 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 24607, 2024-07-26T13:39:24.113 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-07-26T13:39:24.113 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-07-26T13:39:24.113 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-07-26T13:39:24.113 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 142.63620208899999, 2024-07-26T13:39:24.113 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 25, 2024-07-26T13:39:24.113 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 81, 2024-07-26T13:39:24.113 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 81, 2024-07-26T13:39:24.113 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 169.406072962 2024-07-26T13:39:24.114 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:39:24.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.105+0000 7f595f2a0640 1 -- 172.21.15.100:0/2818072527 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f59440b3c50 msgr2=0x7f59440b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:24.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.105+0000 7f595f2a0640 1 --2- 172.21.15.100:0/2818072527 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f59440b3c50 0x7f59440b6160 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x55602582a020 tx=0x7f5954000bc0 comp rx=0 tx=0).stop 2024-07-26T13:39:24.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.105+0000 7f595f2a0640 1 -- 172.21.15.100:0/2818072527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5958102490 msgr2=0x7f5958198ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:24.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.105+0000 7f595f2a0640 1 --2- 172.21.15.100:0/2818072527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5958102490 0x7f5958198ec0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f595400c920 tx=0x7f595400cde0 comp rx=0 tx=0).stop 2024-07-26T13:39:24.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.105+0000 7f595f2a0640 1 -- 172.21.15.100:0/2818072527 shutdown_connections 2024-07-26T13:39:24.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.105+0000 7f595f2a0640 1 --2- 172.21.15.100:0/2818072527 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f59440b3c50 0x7f59440b6160 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:24.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.105+0000 7f595f2a0640 1 --2- 172.21.15.100:0/2818072527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5958108720 0x7f595819e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:24.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.105+0000 7f595f2a0640 1 --2- 172.21.15.100:0/2818072527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5958107dd0 0x7f5958199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:24.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.105+0000 7f595f2a0640 1 --2- 172.21.15.100:0/2818072527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5958102490 0x7f5958198ec0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:24.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.105+0000 7f595f2a0640 1 -- 172.21.15.100:0/2818072527 >> 172.21.15.100:0/2818072527 conn(0x7f5958079190 msgr2=0x7f5958105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:24.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.105+0000 7f595f2a0640 1 -- 172.21.15.100:0/2818072527 shutdown_connections 2024-07-26T13:39:24.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:24.105+0000 7f595f2a0640 1 -- 172.21.15.100:0/2818072527 wait complete. 2024-07-26T13:39:24.814 INFO:tasks.cephadm:Setup complete, yielding 2024-07-26T13:39:24.814 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-07-26T13:39:24.825 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi100.front.sepia.ceph.com 2024-07-26T13:39:24.825 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- bash -c 'ceph orch status' 2024-07-26T13:39:25.080 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:39:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:25 smithi100 ceph-mon[32599]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:39:25.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:25 smithi167 ceph-mon[37140]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:39:25.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:25 smithi138 ceph-mon[36954]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:39:26.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.696+0000 7fbe5d0f9640 1 -- 172.21.15.100:0/3161972525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe58103590 msgr2=0x7fbe581039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:26.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.696+0000 7fbe5d0f9640 1 --2- 172.21.15.100:0/3161972525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe58103590 0x7fbe581039a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbe48009900 tx=0x7fbe4802a3f0 comp rx=0 tx=0).stop 2024-07-26T13:39:26.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.696+0000 7fbe5d0f9640 1 -- 172.21.15.100:0/3161972525 shutdown_connections 2024-07-26T13:39:26.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.696+0000 7fbe5d0f9640 1 --2- 172.21.15.100:0/3161972525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe58109820 0x7fbe58110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:26.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.696+0000 7fbe5d0f9640 1 --2- 172.21.15.100:0/3161972525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe58108ed0 0x7fbe581092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:26.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.696+0000 7fbe5d0f9640 1 --2- 172.21.15.100:0/3161972525 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe58103590 0x7fbe581039a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:26.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.696+0000 7fbe5d0f9640 1 -- 172.21.15.100:0/3161972525 >> 172.21.15.100:0/3161972525 conn(0x7fbe5807b280 msgr2=0x7fbe5807b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:26.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.696+0000 7fbe5d0f9640 1 -- 172.21.15.100:0/3161972525 shutdown_connections 2024-07-26T13:39:26.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.696+0000 7fbe5d0f9640 1 -- 172.21.15.100:0/3161972525 wait complete. 2024-07-26T13:39:26.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.697+0000 7fbe5d0f9640 1 Processor -- start 2024-07-26T13:39:26.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.697+0000 7fbe5d0f9640 1 -- start start 2024-07-26T13:39:26.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.697+0000 7fbe5d0f9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe58103590 0x7fbe58075140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:26.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.698+0000 7fbe5d0f9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe58108ed0 0x7fbe58075680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:26.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.698+0000 7fbe5d0f9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe58109820 0x7fbe58075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:26.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.698+0000 7fbe5d0f9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbe58113540 con 0x7fbe58103590 2024-07-26T13:39:26.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.698+0000 7fbe5d0f9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbe581133c0 con 0x7fbe58109820 2024-07-26T13:39:26.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.698+0000 7fbe5d0f9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbe581136c0 con 0x7fbe58108ed0 2024-07-26T13:39:26.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.698+0000 7fbe56d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe58103590 0x7fbe58075140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:26.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.698+0000 7fbe56d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe58103590 0x7fbe58075140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33454/0 (socket says 172.21.15.100:33454) 2024-07-26T13:39:26.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.698+0000 7fbe56d9d640 1 -- 172.21.15.100:0/2079654381 learned_addr learned my addr 172.21.15.100:0/2079654381 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:26.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.698+0000 7fbe5659c640 1 --2- 172.21.15.100:0/2079654381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe58108ed0 0x7fbe58075680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:26.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.698+0000 7fbe56d9d640 1 -- 172.21.15.100:0/2079654381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe58108ed0 msgr2=0x7fbe58075680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:26.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.698+0000 7fbe56d9d640 1 --2- 172.21.15.100:0/2079654381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe58108ed0 0x7fbe58075680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:26.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.698+0000 7fbe56d9d640 1 -- 172.21.15.100:0/2079654381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe58109820 msgr2=0x7fbe58075bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:26.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.698+0000 7fbe5759e640 1 --2- 172.21.15.100:0/2079654381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe58109820 0x7fbe58075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:26.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.698+0000 7fbe56d9d640 1 --2- 172.21.15.100:0/2079654381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe58109820 0x7fbe58075bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:26.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.698+0000 7fbe56d9d640 1 -- 172.21.15.100:0/2079654381 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe581a81e0 con 0x7fbe58103590 2024-07-26T13:39:26.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.699+0000 7fbe56d9d640 1 --2- 172.21.15.100:0/2079654381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe58103590 0x7fbe58075140 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbe48006010 tx=0x7fbe480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:26.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.699+0000 7fbe3ffff640 1 -- 172.21.15.100:0/2079654381 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4803f030 con 0x7fbe58103590 2024-07-26T13:39:26.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.699+0000 7fbe3ffff640 1 -- 172.21.15.100:0/2079654381 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe48003800 con 0x7fbe58103590 2024-07-26T13:39:26.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.699+0000 7fbe3ffff640 1 -- 172.21.15.100:0/2079654381 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe48035070 con 0x7fbe58103590 2024-07-26T13:39:26.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.699+0000 7fbe5d0f9640 1 -- 172.21.15.100:0/2079654381 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe581a8470 con 0x7fbe58103590 2024-07-26T13:39:26.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.699+0000 7fbe5d0f9640 1 -- 172.21.15.100:0/2079654381 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe581a8950 con 0x7fbe58103590 2024-07-26T13:39:26.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.700+0000 7fbe5d0f9640 1 -- 172.21.15.100:0/2079654381 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe581039a0 con 0x7fbe58103590 2024-07-26T13:39:26.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.701+0000 7fbe3ffff640 1 -- 172.21.15.100:0/2079654381 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7fbe48031b10 con 0x7fbe58103590 2024-07-26T13:39:26.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.702+0000 7fbe3ffff640 1 --2- 172.21.15.100:0/2079654381 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fbe400b3c20 0x7fbe400b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:26.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.702+0000 7fbe5659c640 1 --2- 172.21.15.100:0/2079654381 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fbe400b3c20 0x7fbe400b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:26.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.702+0000 7fbe3ffff640 1 -- 172.21.15.100:0/2079654381 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fbe481047d0 con 0x7fbe58103590 2024-07-26T13:39:26.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.702+0000 7fbe5659c640 1 --2- 172.21.15.100:0/2079654381 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fbe400b3c20 0x7fbe400b6130 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbe58105ef0 tx=0x7fbe4c005cc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:26.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.706+0000 7fbe3ffff640 1 -- 172.21.15.100:0/2079654381 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fbe480cbd20 con 0x7fbe58103590 2024-07-26T13:39:26.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.896+0000 7fbe5d0f9640 1 -- 172.21.15.100:0/2079654381 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fbe58063230 con 0x7fbe400b3c20 2024-07-26T13:39:26.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.897+0000 7fbe3ffff640 1 -- 172.21.15.100:0/2079654381 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fbe58063230 con 0x7fbe400b3c20 2024-07-26T13:39:26.898 INFO:teuthology.orchestra.run.smithi100.stdout:Backend: cephadm 2024-07-26T13:39:26.898 INFO:teuthology.orchestra.run.smithi100.stdout:Available: Yes 2024-07-26T13:39:26.898 INFO:teuthology.orchestra.run.smithi100.stdout:Paused: No 2024-07-26T13:39:26.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.899+0000 7fbe5d0f9640 1 -- 172.21.15.100:0/2079654381 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fbe400b3c20 msgr2=0x7fbe400b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:26.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.899+0000 7fbe5d0f9640 1 --2- 172.21.15.100:0/2079654381 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fbe400b3c20 0x7fbe400b6130 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbe58105ef0 tx=0x7fbe4c005cc0 comp rx=0 tx=0).stop 2024-07-26T13:39:26.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.899+0000 7fbe5d0f9640 1 -- 172.21.15.100:0/2079654381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe58103590 msgr2=0x7fbe58075140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:26.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.899+0000 7fbe5d0f9640 1 --2- 172.21.15.100:0/2079654381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe58103590 0x7fbe58075140 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbe48006010 tx=0x7fbe480042f0 comp rx=0 tx=0).stop 2024-07-26T13:39:26.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.899+0000 7fbe5d0f9640 1 -- 172.21.15.100:0/2079654381 shutdown_connections 2024-07-26T13:39:26.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.899+0000 7fbe5d0f9640 1 --2- 172.21.15.100:0/2079654381 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fbe400b3c20 0x7fbe400b6130 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:26.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.899+0000 7fbe5d0f9640 1 --2- 172.21.15.100:0/2079654381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe58109820 0x7fbe58075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:26.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.899+0000 7fbe5d0f9640 1 --2- 172.21.15.100:0/2079654381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe58108ed0 0x7fbe58075680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:26.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.899+0000 7fbe5d0f9640 1 --2- 172.21.15.100:0/2079654381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe58103590 0x7fbe58075140 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:26.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.899+0000 7fbe5d0f9640 1 -- 172.21.15.100:0/2079654381 >> 172.21.15.100:0/2079654381 conn(0x7fbe5807b280 msgr2=0x7fbe58107020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:26.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.899+0000 7fbe5d0f9640 1 -- 172.21.15.100:0/2079654381 shutdown_connections 2024-07-26T13:39:26.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:26.899+0000 7fbe5d0f9640 1 -- 172.21.15.100:0/2079654381 wait complete. 2024-07-26T13:39:27.599 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- bash -c 'ceph orch ps' 2024-07-26T13:39:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:27 smithi167 ceph-mon[37140]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T13:39:27.854 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:39:27.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:27 smithi138 ceph-mon[36954]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T13:39:27.975 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:27 smithi100 ceph-mon[32599]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T13:39:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:28 smithi100 ceph-mon[32599]: from='client.15210 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:39:28.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:28 smithi167 ceph-mon[37140]: from='client.15210 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:39:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:28 smithi138 ceph-mon[36954]: from='client.15210 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:39:29.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.506+0000 7f979cd7a640 1 -- 172.21.15.100:0/3057391603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9798108740 msgr2=0x7f979810ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:29.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.506+0000 7f979cd7a640 1 --2- 172.21.15.100:0/3057391603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9798108740 0x7f979810ef70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9790009900 tx=0x7f979002a410 comp rx=0 tx=0).stop 2024-07-26T13:39:29.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.506+0000 7f979cd7a640 1 -- 172.21.15.100:0/3057391603 shutdown_connections 2024-07-26T13:39:29.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.506+0000 7f979cd7a640 1 --2- 172.21.15.100:0/3057391603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9798108740 0x7f979810ef70 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:29.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.506+0000 7f979cd7a640 1 --2- 172.21.15.100:0/3057391603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9798107df0 0x7f9798108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:29.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.506+0000 7f979cd7a640 1 --2- 172.21.15.100:0/3057391603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97981024b0 0x7f97981028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:29.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.506+0000 7f979cd7a640 1 -- 172.21.15.100:0/3057391603 >> 172.21.15.100:0/3057391603 conn(0x7f9798079170 msgr2=0x7f9798079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:29.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.506+0000 7f979cd7a640 1 -- 172.21.15.100:0/3057391603 shutdown_connections 2024-07-26T13:39:29.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.506+0000 7f979cd7a640 1 -- 172.21.15.100:0/3057391603 wait complete. 2024-07-26T13:39:29.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.507+0000 7f979cd7a640 1 Processor -- start 2024-07-26T13:39:29.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.507+0000 7f979cd7a640 1 -- start start 2024-07-26T13:39:29.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.507+0000 7f979cd7a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97981024b0 0x7f979819adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:29.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.507+0000 7f979cd7a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9798107df0 0x7f979819b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:29.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.507+0000 7f979cd7a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9798108740 0x7f979819b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:29.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.507+0000 7f979cd7a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9798112490 con 0x7f97981024b0 2024-07-26T13:39:29.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.507+0000 7f979cd7a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9798112310 con 0x7f9798108740 2024-07-26T13:39:29.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.507+0000 7f979cd7a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9798112610 con 0x7f9798107df0 2024-07-26T13:39:29.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.507+0000 7f979659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97981024b0 0x7f979819adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:29.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.507+0000 7f979659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97981024b0 0x7f979819adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33478/0 (socket says 172.21.15.100:33478) 2024-07-26T13:39:29.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.507+0000 7f979659c640 1 -- 172.21.15.100:0/4186617253 learned_addr learned my addr 172.21.15.100:0/4186617253 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:29.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.507+0000 7f9796d9d640 1 --2- 172.21.15.100:0/4186617253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9798108740 0x7f979819b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:29.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.507+0000 7f9795d9b640 1 --2- 172.21.15.100:0/4186617253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9798107df0 0x7f979819b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:29.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.508+0000 7f979659c640 1 -- 172.21.15.100:0/4186617253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9798107df0 msgr2=0x7f979819b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:29.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.508+0000 7f979659c640 1 --2- 172.21.15.100:0/4186617253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9798107df0 0x7f979819b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:29.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.508+0000 7f979659c640 1 -- 172.21.15.100:0/4186617253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9798108740 msgr2=0x7f979819b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:29.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.508+0000 7f979659c640 1 --2- 172.21.15.100:0/4186617253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9798108740 0x7f979819b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:29.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.508+0000 7f979659c640 1 -- 172.21.15.100:0/4186617253 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97981a6080 con 0x7f97981024b0 2024-07-26T13:39:29.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.508+0000 7f979659c640 1 --2- 172.21.15.100:0/4186617253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97981024b0 0x7f979819adc0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f978800ba50 tx=0x7f978800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:29.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.508+0000 7f97877fe640 1 -- 172.21.15.100:0/4186617253 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9788019070 con 0x7f97981024b0 2024-07-26T13:39:29.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.508+0000 7f97877fe640 1 -- 172.21.15.100:0/4186617253 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f978800cbe0 con 0x7f97981024b0 2024-07-26T13:39:29.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.508+0000 7f979cd7a640 1 -- 172.21.15.100:0/4186617253 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97981a6370 con 0x7f97981024b0 2024-07-26T13:39:29.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.508+0000 7f97877fe640 1 -- 172.21.15.100:0/4186617253 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97880075d0 con 0x7f97981024b0 2024-07-26T13:39:29.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.508+0000 7f979cd7a640 1 -- 172.21.15.100:0/4186617253 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97981a6930 con 0x7f97981024b0 2024-07-26T13:39:29.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.509+0000 7f979cd7a640 1 -- 172.21.15.100:0/4186617253 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9798103730 con 0x7f97981024b0 2024-07-26T13:39:29.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.510+0000 7f97877fe640 1 -- 172.21.15.100:0/4186617253 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f978801c070 con 0x7f97981024b0 2024-07-26T13:39:29.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.511+0000 7f97877fe640 1 --2- 172.21.15.100:0/4186617253 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f97800b3ca0 0x7f97800b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:29.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.511+0000 7f9795d9b640 1 --2- 172.21.15.100:0/4186617253 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f97800b3ca0 0x7f97800b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:29.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.511+0000 7f97877fe640 1 -- 172.21.15.100:0/4186617253 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f97880e37f0 con 0x7f97981024b0 2024-07-26T13:39:29.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.512+0000 7f9795d9b640 1 --2- 172.21.15.100:0/4186617253 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f97800b3ca0 0x7f97800b61b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9798107450 tx=0x7f978c008040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:29.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.514+0000 7f97877fe640 1 -- 172.21.15.100:0/4186617253 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f97880aa6f0 con 0x7f97981024b0 2024-07-26T13:39:29.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.697+0000 7f979cd7a640 1 -- 172.21.15.100:0/4186617253 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f9798063230 con 0x7f97800b3ca0 2024-07-26T13:39:29.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.709+0000 7f97877fe640 1 -- 172.21.15.100:0/4186617253 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f9798063230 con 0x7f97800b3ca0 2024-07-26T13:39:29.710 INFO:teuthology.orchestra.run.smithi100.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-26T13:39:29.710 INFO:teuthology.orchestra.run.smithi100.stdout:mds.a smithi100 running (2m) 32s ago 2m 84.7M - 19.0.0-5320-ge8f88bb1 ae51f432d4fa e5620637ae0a 2024-07-26T13:39:29.710 INFO:teuthology.orchestra.run.smithi100.stdout:mds.b smithi138 running (2m) 33s ago 2m 87.5M - 19.0.0-5320-ge8f88bb1 ae51f432d4fa e336832bdc01 2024-07-26T13:39:29.710 INFO:teuthology.orchestra.run.smithi100.stdout:mds.c smithi167 running (3m) 33s ago 3m 87.8M - 19.0.0-5320-ge8f88bb1 ae51f432d4fa 6876119001b6 2024-07-26T13:39:29.710 INFO:teuthology.orchestra.run.smithi100.stdout:mds.d smithi100 running (3m) 32s ago 3m 84.8M - 19.0.0-5320-ge8f88bb1 ae51f432d4fa 2af282dd5169 2024-07-26T13:39:29.711 INFO:teuthology.orchestra.run.smithi100.stdout:mds.e smithi138 running (3m) 33s ago 3m 87.6M - 19.0.0-5320-ge8f88bb1 ae51f432d4fa b5b2b94fbed0 2024-07-26T13:39:29.711 INFO:teuthology.orchestra.run.smithi100.stdout:mds.f smithi167 running (3m) 33s ago 3m 88.4M - 19.0.0-5320-ge8f88bb1 ae51f432d4fa 3b039426e42c 2024-07-26T13:39:29.711 INFO:teuthology.orchestra.run.smithi100.stdout:mds.g smithi100 running (2m) 32s ago 2m 86.8M - 19.0.0-5320-ge8f88bb1 ae51f432d4fa 3e857d35b949 2024-07-26T13:39:29.711 INFO:teuthology.orchestra.run.smithi100.stdout:mds.h smithi138 running (2m) 33s ago 2m 87.0M - 19.0.0-5320-ge8f88bb1 ae51f432d4fa 10a4495ac07b 2024-07-26T13:39:29.711 INFO:teuthology.orchestra.run.smithi100.stdout:mds.i smithi167 running (2m) 33s ago 2m 96.2M - 19.0.0-5320-ge8f88bb1 ae51f432d4fa dd8ee84abc2d 2024-07-26T13:39:29.711 INFO:teuthology.orchestra.run.smithi100.stdout:mds.j smithi100 running (3m) 32s ago 3m 82.3M - 19.0.0-5320-ge8f88bb1 ae51f432d4fa 086b81ab7116 2024-07-26T13:39:29.711 INFO:teuthology.orchestra.run.smithi100.stdout:mds.k smithi138 running (3m) 33s ago 3m 82.7M - 19.0.0-5320-ge8f88bb1 ae51f432d4fa e713efffd2cf 2024-07-26T13:39:29.711 INFO:teuthology.orchestra.run.smithi100.stdout:mds.l smithi167 running (3m) 33s ago 3m 82.1M - 19.0.0-5320-ge8f88bb1 ae51f432d4fa dad30bd935ac 2024-07-26T13:39:29.711 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.x smithi100 *:9283,8765 running (10m) 32s ago 10m 586M - 19.0.0-5320-ge8f88bb1 ae51f432d4fa 448c8fb66ee4 2024-07-26T13:39:29.711 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.y smithi138 *:8443,8765 running (9m) 33s ago 9m 522M - 19.0.0-5320-ge8f88bb1 ae51f432d4fa 0052381d13c6 2024-07-26T13:39:29.711 INFO:teuthology.orchestra.run.smithi100.stdout:mgr.z smithi167 *:8443,8765 running (9m) 33s ago 9m 525M - 19.0.0-5320-ge8f88bb1 ae51f432d4fa 3286874f14a6 2024-07-26T13:39:29.711 INFO:teuthology.orchestra.run.smithi100.stdout:mon.a smithi100 running (11m) 32s ago 11m 122M 2048M 19.0.0-5320-ge8f88bb1 ae51f432d4fa 304ecaf8e642 2024-07-26T13:39:29.711 INFO:teuthology.orchestra.run.smithi100.stdout:mon.b smithi138 running (9m) 33s ago 9m 112M 2048M 19.0.0-5320-ge8f88bb1 ae51f432d4fa 258042bba51c 2024-07-26T13:39:29.712 INFO:teuthology.orchestra.run.smithi100.stdout:mon.c smithi167 running (9m) 33s ago 9m 109M 2048M 19.0.0-5320-ge8f88bb1 ae51f432d4fa cab5f4f90d37 2024-07-26T13:39:29.712 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 smithi100 running (7m) 32s ago 7m 114M 4271M 19.0.0-5320-ge8f88bb1 ae51f432d4fa 326b1ccbc8b8 2024-07-26T13:39:29.712 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 smithi138 running (7m) 33s ago 7m 126M 4272M 19.0.0-5320-ge8f88bb1 ae51f432d4fa c86507fb796a 2024-07-26T13:39:29.712 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 smithi167 running (7m) 33s ago 7m 126M 4272M 19.0.0-5320-ge8f88bb1 ae51f432d4fa 10f42a3cc7dd 2024-07-26T13:39:29.712 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 smithi100 running (6m) 32s ago 6m 122M 4271M 19.0.0-5320-ge8f88bb1 ae51f432d4fa b5ecfd533f86 2024-07-26T13:39:29.712 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 smithi138 running (6m) 33s ago 6m 121M 4272M 19.0.0-5320-ge8f88bb1 ae51f432d4fa 3bdd5d30b339 2024-07-26T13:39:29.712 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 smithi167 running (6m) 33s ago 6m 125M 4272M 19.0.0-5320-ge8f88bb1 ae51f432d4fa b18f3c6049b4 2024-07-26T13:39:29.712 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 smithi100 running (5m) 32s ago 5m 110M 4271M 19.0.0-5320-ge8f88bb1 ae51f432d4fa 02aa28cc53ea 2024-07-26T13:39:29.712 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 smithi138 running (5m) 33s ago 5m 112M 4272M 19.0.0-5320-ge8f88bb1 ae51f432d4fa 557c7d2503c4 2024-07-26T13:39:29.712 INFO:teuthology.orchestra.run.smithi100.stdout:osd.8 smithi167 running (4m) 33s ago 4m 114M 4272M 19.0.0-5320-ge8f88bb1 ae51f432d4fa 9e3e3301127b 2024-07-26T13:39:29.712 INFO:teuthology.orchestra.run.smithi100.stdout:osd.9 smithi100 running (4m) 32s ago 4m 121M 4271M 19.0.0-5320-ge8f88bb1 ae51f432d4fa 839d92b96897 2024-07-26T13:39:29.712 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10 smithi138 running (4m) 33s ago 4m 122M 4272M 19.0.0-5320-ge8f88bb1 ae51f432d4fa 2d492fab9297 2024-07-26T13:39:29.712 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11 smithi167 running (3m) 33s ago 3m 110M 4272M 19.0.0-5320-ge8f88bb1 ae51f432d4fa 3f6f2e83cdfb 2024-07-26T13:39:29.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.711+0000 7f979cd7a640 1 -- 172.21.15.100:0/4186617253 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f97800b3ca0 msgr2=0x7f97800b61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:29.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.711+0000 7f979cd7a640 1 --2- 172.21.15.100:0/4186617253 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f97800b3ca0 0x7f97800b61b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9798107450 tx=0x7f978c008040 comp rx=0 tx=0).stop 2024-07-26T13:39:29.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.711+0000 7f979cd7a640 1 -- 172.21.15.100:0/4186617253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97981024b0 msgr2=0x7f979819adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:29.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.711+0000 7f979cd7a640 1 --2- 172.21.15.100:0/4186617253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97981024b0 0x7f979819adc0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f978800ba50 tx=0x7f978800bf10 comp rx=0 tx=0).stop 2024-07-26T13:39:29.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.711+0000 7f979cd7a640 1 -- 172.21.15.100:0/4186617253 shutdown_connections 2024-07-26T13:39:29.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.711+0000 7f979cd7a640 1 --2- 172.21.15.100:0/4186617253 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f97800b3ca0 0x7f97800b61b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:29.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.711+0000 7f979cd7a640 1 --2- 172.21.15.100:0/4186617253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9798108740 0x7f979819b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:29.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.711+0000 7f979cd7a640 1 --2- 172.21.15.100:0/4186617253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9798107df0 0x7f979819b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:29.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.711+0000 7f979cd7a640 1 --2- 172.21.15.100:0/4186617253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97981024b0 0x7f979819adc0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:29.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.711+0000 7f979cd7a640 1 -- 172.21.15.100:0/4186617253 >> 172.21.15.100:0/4186617253 conn(0x7f9798079170 msgr2=0x7f9798104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:29.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.711+0000 7f979cd7a640 1 -- 172.21.15.100:0/4186617253 shutdown_connections 2024-07-26T13:39:29.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:29.711+0000 7f979cd7a640 1 -- 172.21.15.100:0/4186617253 wait complete. 2024-07-26T13:39:29.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:29 smithi138 ceph-mon[36954]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:39:29.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:29 smithi100 ceph-mon[32599]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:39:30.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:29 smithi167 ceph-mon[37140]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:39:30.445 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- bash -c 'ceph orch ls' 2024-07-26T13:39:30.703 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:39:31.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:31 smithi100 ceph-mon[32599]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:39:31.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:31 smithi100 ceph-mon[32599]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:39:31.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:31 smithi138 ceph-mon[36954]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:39:31.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:31 smithi138 ceph-mon[36954]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:39:32.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:31 smithi167 ceph-mon[37140]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:39:32.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:31 smithi167 ceph-mon[37140]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:39:32.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.410+0000 7f6d142fd640 1 -- 172.21.15.100:0/494691621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d0c1035b0 msgr2=0x7f6d0c1039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:32.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.410+0000 7f6d142fd640 1 --2- 172.21.15.100:0/494691621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d0c1035b0 0x7f6d0c1039c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc009900 tx=0x7f6cfc02a410 comp rx=0 tx=0).stop 2024-07-26T13:39:32.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.410+0000 7f6d142fd640 1 -- 172.21.15.100:0/494691621 shutdown_connections 2024-07-26T13:39:32.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.410+0000 7f6d142fd640 1 --2- 172.21.15.100:0/494691621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d0c109840 0x7f6d0c110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:32.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.410+0000 7f6d142fd640 1 --2- 172.21.15.100:0/494691621 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d0c108ef0 0x7f6d0c109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:32.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.410+0000 7f6d142fd640 1 --2- 172.21.15.100:0/494691621 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d0c1035b0 0x7f6d0c1039c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:32.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.410+0000 7f6d142fd640 1 -- 172.21.15.100:0/494691621 >> 172.21.15.100:0/494691621 conn(0x7f6d0c0ff040 msgr2=0x7f6d0c101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:32.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.410+0000 7f6d142fd640 1 -- 172.21.15.100:0/494691621 shutdown_connections 2024-07-26T13:39:32.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.410+0000 7f6d142fd640 1 -- 172.21.15.100:0/494691621 wait complete. 2024-07-26T13:39:32.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.411+0000 7f6d142fd640 1 Processor -- start 2024-07-26T13:39:32.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.411+0000 7f6d142fd640 1 -- start start 2024-07-26T13:39:32.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.411+0000 7f6d142fd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d0c1035b0 0x7f6d0c075140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:32.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.411+0000 7f6d142fd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d0c108ef0 0x7f6d0c075680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:32.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.411+0000 7f6d142fd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d0c109840 0x7f6d0c075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:32.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.411+0000 7f6d142fd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c1135d0 con 0x7f6d0c109840 2024-07-26T13:39:32.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.411+0000 7f6d142fd640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c113450 con 0x7f6d0c1035b0 2024-07-26T13:39:32.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.411+0000 7f6d142fd640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c113750 con 0x7f6d0c108ef0 2024-07-26T13:39:32.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.411+0000 7f6d1289a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d0c109840 0x7f6d0c075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:32.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.411+0000 7f6d1289a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d0c109840 0x7f6d0c075bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33490/0 (socket says 172.21.15.100:33490) 2024-07-26T13:39:32.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.411+0000 7f6d1289a640 1 -- 172.21.15.100:0/347200049 learned_addr learned my addr 172.21.15.100:0/347200049 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:32.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.411+0000 7f6d12099640 1 --2- 172.21.15.100:0/347200049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d0c1035b0 0x7f6d0c075140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:32.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.411+0000 7f6d11898640 1 --2- 172.21.15.100:0/347200049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d0c108ef0 0x7f6d0c075680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:32.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.411+0000 7f6d1289a640 1 -- 172.21.15.100:0/347200049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d0c108ef0 msgr2=0x7f6d0c075680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:32.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.411+0000 7f6d1289a640 1 --2- 172.21.15.100:0/347200049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d0c108ef0 0x7f6d0c075680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:32.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.411+0000 7f6d1289a640 1 -- 172.21.15.100:0/347200049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d0c1035b0 msgr2=0x7f6d0c075140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:32.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.411+0000 7f6d1289a640 1 --2- 172.21.15.100:0/347200049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d0c1035b0 0x7f6d0c075140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:32.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.411+0000 7f6d1289a640 1 -- 172.21.15.100:0/347200049 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d0c1a8220 con 0x7f6d0c109840 2024-07-26T13:39:32.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.411+0000 7f6d12099640 1 --2- 172.21.15.100:0/347200049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d0c1035b0 0x7f6d0c075140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:39:32.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.411+0000 7f6d11898640 1 --2- 172.21.15.100:0/347200049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d0c108ef0 0x7f6d0c075680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:39:32.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.412+0000 7f6d1289a640 1 --2- 172.21.15.100:0/347200049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d0c109840 0x7f6d0c075bc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6d0400b970 tx=0x7f6d0400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:32.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.412+0000 7f6d0b7fe640 1 -- 172.21.15.100:0/347200049 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d04019070 con 0x7f6d0c109840 2024-07-26T13:39:32.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.412+0000 7f6d142fd640 1 -- 172.21.15.100:0/347200049 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d0c1a8510 con 0x7f6d0c109840 2024-07-26T13:39:32.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.412+0000 7f6d142fd640 1 -- 172.21.15.100:0/347200049 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d0c1a8ad0 con 0x7f6d0c109840 2024-07-26T13:39:32.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.412+0000 7f6d0b7fe640 1 -- 172.21.15.100:0/347200049 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d04014440 con 0x7f6d0c109840 2024-07-26T13:39:32.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.412+0000 7f6d0b7fe640 1 -- 172.21.15.100:0/347200049 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d040075d0 con 0x7f6d0c109840 2024-07-26T13:39:32.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.413+0000 7f6d142fd640 1 -- 172.21.15.100:0/347200049 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d0c1039c0 con 0x7f6d0c109840 2024-07-26T13:39:32.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.414+0000 7f6d0b7fe640 1 -- 172.21.15.100:0/347200049 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f6d0401c070 con 0x7f6d0c109840 2024-07-26T13:39:32.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.415+0000 7f6d0b7fe640 1 --2- 172.21.15.100:0/347200049 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6cf40b3c20 0x7f6cf40b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:32.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.415+0000 7f6d12099640 1 --2- 172.21.15.100:0/347200049 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6cf40b3c20 0x7f6cf40b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:32.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.415+0000 7f6d0b7fe640 1 -- 172.21.15.100:0/347200049 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f6d040e38c0 con 0x7f6d0c109840 2024-07-26T13:39:32.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.416+0000 7f6d12099640 1 --2- 172.21.15.100:0/347200049 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6cf40b3c20 0x7f6cf40b6130 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc006010 tx=0x7f6cfc031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:32.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.418+0000 7f6d0b7fe640 1 -- 172.21.15.100:0/347200049 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f6d040aae10 con 0x7f6d0c109840 2024-07-26T13:39:32.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.607+0000 7f6d142fd640 1 -- 172.21.15.100:0/347200049 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f6d0c063230 con 0x7f6cf40b3c20 2024-07-26T13:39:32.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.612+0000 7f6d0b7fe640 1 -- 172.21.15.100:0/347200049 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f6d0c063230 con 0x7f6cf40b3c20 2024-07-26T13:39:32.613 INFO:teuthology.orchestra.run.smithi100.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-07-26T13:39:32.613 INFO:teuthology.orchestra.run.smithi100.stdout:mds.all 12/12 36s ago 3m smithi100=a;smithi100=d;smithi100=g;smithi100=j;smithi138=b;smithi138=e;smithi138=h;smithi138=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2024-07-26T13:39:32.614 INFO:teuthology.orchestra.run.smithi100.stdout:mgr 3/3 36s ago 9m smithi100=x;smithi138=y;smithi167=z;count:3 2024-07-26T13:39:32.614 INFO:teuthology.orchestra.run.smithi100.stdout:mon 3/3 36s ago 9m smithi100:172.21.15.100=a;smithi138:172.21.15.138=b;smithi167:172.21.15.167=c;count:3 2024-07-26T13:39:32.614 INFO:teuthology.orchestra.run.smithi100.stdout:osd 12 36s ago - 2024-07-26T13:39:32.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.614+0000 7f6d142fd640 1 -- 172.21.15.100:0/347200049 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6cf40b3c20 msgr2=0x7f6cf40b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:32.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.614+0000 7f6d142fd640 1 --2- 172.21.15.100:0/347200049 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6cf40b3c20 0x7f6cf40b6130 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc006010 tx=0x7f6cfc031400 comp rx=0 tx=0).stop 2024-07-26T13:39:32.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.615+0000 7f6d142fd640 1 -- 172.21.15.100:0/347200049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d0c109840 msgr2=0x7f6d0c075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:32.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.615+0000 7f6d142fd640 1 --2- 172.21.15.100:0/347200049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d0c109840 0x7f6d0c075bc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6d0400b970 tx=0x7f6d0400be30 comp rx=0 tx=0).stop 2024-07-26T13:39:32.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.615+0000 7f6d142fd640 1 -- 172.21.15.100:0/347200049 shutdown_connections 2024-07-26T13:39:32.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.615+0000 7f6d142fd640 1 --2- 172.21.15.100:0/347200049 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6cf40b3c20 0x7f6cf40b6130 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:32.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.615+0000 7f6d142fd640 1 --2- 172.21.15.100:0/347200049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d0c109840 0x7f6d0c075bc0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:32.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.615+0000 7f6d142fd640 1 --2- 172.21.15.100:0/347200049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d0c108ef0 0x7f6d0c075680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:32.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.615+0000 7f6d142fd640 1 --2- 172.21.15.100:0/347200049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d0c1035b0 0x7f6d0c075140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:32.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.615+0000 7f6d142fd640 1 -- 172.21.15.100:0/347200049 >> 172.21.15.100:0/347200049 conn(0x7f6d0c0ff040 msgr2=0x7f6d0c101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:32.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.615+0000 7f6d142fd640 1 -- 172.21.15.100:0/347200049 shutdown_connections 2024-07-26T13:39:32.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:32.615+0000 7f6d142fd640 1 -- 172.21.15.100:0/347200049 wait complete. 2024-07-26T13:39:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:32 smithi138 ceph-mon[36954]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:39:32.992 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:32 smithi100 ceph-mon[32599]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:39:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:32 smithi167 ceph-mon[37140]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:39:33.338 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- bash -c 'ceph orch host ls' 2024-07-26T13:39:33.591 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:39:33.843 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:33 smithi100 ceph-mon[32599]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:39:33.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:33 smithi138 ceph-mon[36954]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:39:34.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:33 smithi167 ceph-mon[37140]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:39:34.726 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:34 smithi100 ceph-mon[32599]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:39:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:34 smithi138 ceph-mon[36954]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:39:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:34 smithi167 ceph-mon[37140]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:39:35.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.248+0000 7f5000573640 1 -- 172.21.15.100:0/1170976086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ff8107e10 msgr2=0x7f4ff8108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:35.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.248+0000 7f5000573640 1 --2- 172.21.15.100:0/1170976086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ff8107e10 0x7f4ff8108220 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8009900 tx=0x7f4fe802a410 comp rx=0 tx=0).stop 2024-07-26T13:39:35.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.248+0000 7f5000573640 1 -- 172.21.15.100:0/1170976086 shutdown_connections 2024-07-26T13:39:35.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.248+0000 7f5000573640 1 --2- 172.21.15.100:0/1170976086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ff8108760 0x7f4ff810ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:35.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.248+0000 7f5000573640 1 --2- 172.21.15.100:0/1170976086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ff8107e10 0x7f4ff8108220 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:35.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.248+0000 7f5000573640 1 --2- 172.21.15.100:0/1170976086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ff81024d0 0x7f4ff81028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:35.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.248+0000 7f5000573640 1 -- 172.21.15.100:0/1170976086 >> 172.21.15.100:0/1170976086 conn(0x7f4ff8079180 msgr2=0x7f4ff8079590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:35.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.248+0000 7f5000573640 1 -- 172.21.15.100:0/1170976086 shutdown_connections 2024-07-26T13:39:35.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.248+0000 7f5000573640 1 -- 172.21.15.100:0/1170976086 wait complete. 2024-07-26T13:39:35.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.249+0000 7f5000573640 1 Processor -- start 2024-07-26T13:39:35.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.249+0000 7f5000573640 1 -- start start 2024-07-26T13:39:35.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.250+0000 7f5000573640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ff81024d0 0x7f4ff819ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:35.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.250+0000 7f5000573640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ff8107e10 0x7f4ff819b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:35.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.250+0000 7f5000573640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ff8108760 0x7f4ff819b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:35.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.250+0000 7f5000573640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff81123d0 con 0x7f4ff8108760 2024-07-26T13:39:35.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.250+0000 7f5000573640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff8112250 con 0x7f4ff81024d0 2024-07-26T13:39:35.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.250+0000 7f5000573640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff8112550 con 0x7f4ff8107e10 2024-07-26T13:39:35.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.250+0000 7f4ffeb10640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ff8108760 0x7f4ff819b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:35.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.250+0000 7f4ffeb10640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ff8108760 0x7f4ff819b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:32796/0 (socket says 172.21.15.100:32796) 2024-07-26T13:39:35.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.250+0000 7f4ffeb10640 1 -- 172.21.15.100:0/466715326 learned_addr learned my addr 172.21.15.100:0/466715326 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:35.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.250+0000 7f4ffe30f640 1 --2- 172.21.15.100:0/466715326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ff81024d0 0x7f4ff819ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:35.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.250+0000 7f4ffdb0e640 1 --2- 172.21.15.100:0/466715326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ff8107e10 0x7f4ff819b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:35.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.250+0000 7f4ffeb10640 1 -- 172.21.15.100:0/466715326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ff8107e10 msgr2=0x7f4ff819b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:35.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.250+0000 7f4ffeb10640 1 --2- 172.21.15.100:0/466715326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ff8107e10 0x7f4ff819b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:35.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.250+0000 7f4ffeb10640 1 -- 172.21.15.100:0/466715326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ff81024d0 msgr2=0x7f4ff819ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:35.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.250+0000 7f4ffeb10640 1 --2- 172.21.15.100:0/466715326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ff81024d0 0x7f4ff819ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:35.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.250+0000 7f4ffeb10640 1 -- 172.21.15.100:0/466715326 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ff81a5f90 con 0x7f4ff8108760 2024-07-26T13:39:35.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.250+0000 7f4ffeb10640 1 --2- 172.21.15.100:0/466715326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ff8108760 0x7f4ff819b780 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4ff000d750 tx=0x7f4ff000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:35.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.251+0000 7f4fef7fe640 1 -- 172.21.15.100:0/466715326 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff0017070 con 0x7f4ff8108760 2024-07-26T13:39:35.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.251+0000 7f5000573640 1 -- 172.21.15.100:0/466715326 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ff81a6280 con 0x7f4ff8108760 2024-07-26T13:39:35.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.251+0000 7f4fef7fe640 1 -- 172.21.15.100:0/466715326 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ff00054d0 con 0x7f4ff8108760 2024-07-26T13:39:35.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.251+0000 7f4fef7fe640 1 -- 172.21.15.100:0/466715326 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff0004c60 con 0x7f4ff8108760 2024-07-26T13:39:35.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.251+0000 7f5000573640 1 -- 172.21.15.100:0/466715326 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ff81a6790 con 0x7f4ff8108760 2024-07-26T13:39:35.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.252+0000 7f5000573640 1 -- 172.21.15.100:0/466715326 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ff81028e0 con 0x7f4ff8108760 2024-07-26T13:39:35.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.254+0000 7f4fef7fe640 1 -- 172.21.15.100:0/466715326 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f4ff0009bd0 con 0x7f4ff8108760 2024-07-26T13:39:35.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.255+0000 7f4fef7fe640 1 --2- 172.21.15.100:0/466715326 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f4fe40b3c50 0x7f4fe40b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:35.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.255+0000 7f4ffe30f640 1 --2- 172.21.15.100:0/466715326 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f4fe40b3c50 0x7f4fe40b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:35.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.255+0000 7f4fef7fe640 1 -- 172.21.15.100:0/466715326 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f4ff00e3aa0 con 0x7f4ff8108760 2024-07-26T13:39:35.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.256+0000 7f4ffe30f640 1 --2- 172.21.15.100:0/466715326 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f4fe40b3c50 0x7f4fe40b6160 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4ff806bdb0 tx=0x7f4ff4009680 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:35.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.258+0000 7f4fef7fe640 1 -- 172.21.15.100:0/466715326 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f4ff00aa940 con 0x7f4ff8108760 2024-07-26T13:39:35.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.443+0000 7f5000573640 1 -- 172.21.15.100:0/466715326 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f4ff810a5a0 con 0x7f4fe40b3c50 2024-07-26T13:39:35.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.444+0000 7f4fef7fe640 1 -- 172.21.15.100:0/466715326 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f4ff810a5a0 con 0x7f4fe40b3c50 2024-07-26T13:39:35.445 INFO:teuthology.orchestra.run.smithi100.stdout:HOST ADDR LABELS STATUS 2024-07-26T13:39:35.445 INFO:teuthology.orchestra.run.smithi100.stdout:smithi100 172.21.15.100 2024-07-26T13:39:35.445 INFO:teuthology.orchestra.run.smithi100.stdout:smithi138 172.21.15.138 2024-07-26T13:39:35.445 INFO:teuthology.orchestra.run.smithi100.stdout:smithi167 172.21.15.167 2024-07-26T13:39:35.446 INFO:teuthology.orchestra.run.smithi100.stdout:3 hosts in cluster 2024-07-26T13:39:35.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.446+0000 7f5000573640 1 -- 172.21.15.100:0/466715326 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f4fe40b3c50 msgr2=0x7f4fe40b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:35.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.446+0000 7f5000573640 1 --2- 172.21.15.100:0/466715326 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f4fe40b3c50 0x7f4fe40b6160 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4ff806bdb0 tx=0x7f4ff4009680 comp rx=0 tx=0).stop 2024-07-26T13:39:35.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.447+0000 7f5000573640 1 -- 172.21.15.100:0/466715326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ff8108760 msgr2=0x7f4ff819b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:35.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.447+0000 7f5000573640 1 --2- 172.21.15.100:0/466715326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ff8108760 0x7f4ff819b780 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4ff000d750 tx=0x7f4ff000dc10 comp rx=0 tx=0).stop 2024-07-26T13:39:35.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.447+0000 7f5000573640 1 -- 172.21.15.100:0/466715326 shutdown_connections 2024-07-26T13:39:35.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.447+0000 7f5000573640 1 --2- 172.21.15.100:0/466715326 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f4fe40b3c50 0x7f4fe40b6160 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:35.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.447+0000 7f5000573640 1 --2- 172.21.15.100:0/466715326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ff8108760 0x7f4ff819b780 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:35.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.447+0000 7f5000573640 1 --2- 172.21.15.100:0/466715326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ff8107e10 0x7f4ff819b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:35.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.447+0000 7f5000573640 1 --2- 172.21.15.100:0/466715326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ff81024d0 0x7f4ff819ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:35.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.447+0000 7f5000573640 1 -- 172.21.15.100:0/466715326 >> 172.21.15.100:0/466715326 conn(0x7f4ff8079180 msgr2=0x7f4ff8105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:35.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.447+0000 7f5000573640 1 -- 172.21.15.100:0/466715326 shutdown_connections 2024-07-26T13:39:35.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:35.447+0000 7f5000573640 1 -- 172.21.15.100:0/466715326 wait complete. 2024-07-26T13:39:35.594 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:35 smithi100 ceph-mon[32599]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:39:35.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:35 smithi138 ceph-mon[36954]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:39:36.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:35 smithi167 ceph-mon[37140]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:39:36.125 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- bash -c 'ceph orch device ls' 2024-07-26T13:39:36.394 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:39:36.771 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:36 smithi100 ceph-mon[32599]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T13:39:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:36 smithi138 ceph-mon[36954]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T13:39:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:36 smithi167 ceph-mon[37140]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T13:39:38.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.181+0000 7f8fcb26a640 1 -- 172.21.15.100:0/4277839954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fc4107e50 msgr2=0x7f8fc4108260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:38.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.181+0000 7f8fcb26a640 1 --2- 172.21.15.100:0/4277839954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fc4107e50 0x7f8fc4108260 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0009900 tx=0x7f8fc002a410 comp rx=0 tx=0).stop 2024-07-26T13:39:38.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.182+0000 7f8fcb26a640 1 -- 172.21.15.100:0/4277839954 shutdown_connections 2024-07-26T13:39:38.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.182+0000 7f8fcb26a640 1 --2- 172.21.15.100:0/4277839954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fc41087a0 0x7f8fc410efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:38.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.182+0000 7f8fcb26a640 1 --2- 172.21.15.100:0/4277839954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fc4107e50 0x7f8fc4108260 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:38.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.182+0000 7f8fcb26a640 1 --2- 172.21.15.100:0/4277839954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fc4102510 0x7f8fc4102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:38.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.182+0000 7f8fcb26a640 1 -- 172.21.15.100:0/4277839954 >> 172.21.15.100:0/4277839954 conn(0x7f8fc40791a0 msgr2=0x7f8fc40795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:38.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.182+0000 7f8fcb26a640 1 -- 172.21.15.100:0/4277839954 shutdown_connections 2024-07-26T13:39:38.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.182+0000 7f8fcb26a640 1 -- 172.21.15.100:0/4277839954 wait complete. 2024-07-26T13:39:38.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.183+0000 7f8fcb26a640 1 Processor -- start 2024-07-26T13:39:38.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.183+0000 7f8fcb26a640 1 -- start start 2024-07-26T13:39:38.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.183+0000 7f8fcb26a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fc4102510 0x7f8fc4199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:38.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.183+0000 7f8fcb26a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fc4107e50 0x7f8fc419a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:38.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.183+0000 7f8fcb26a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fc41087a0 0x7f8fc419f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:38.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.183+0000 7f8fcb26a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc4113580 con 0x7f8fc41087a0 2024-07-26T13:39:38.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.183+0000 7f8fcb26a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc4113400 con 0x7f8fc4107e50 2024-07-26T13:39:38.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.183+0000 7f8fcb26a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc4113700 con 0x7f8fc4102510 2024-07-26T13:39:38.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.183+0000 7f8fc9807640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fc41087a0 0x7f8fc419f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:38.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.183+0000 7f8fc9807640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fc41087a0 0x7f8fc419f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:32822/0 (socket says 172.21.15.100:32822) 2024-07-26T13:39:38.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.183+0000 7f8fc9807640 1 -- 172.21.15.100:0/3165691887 learned_addr learned my addr 172.21.15.100:0/3165691887 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:38.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.183+0000 7f8fc9006640 1 --2- 172.21.15.100:0/3165691887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fc4102510 0x7f8fc4199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:38.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.183+0000 7f8fc8805640 1 --2- 172.21.15.100:0/3165691887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fc4107e50 0x7f8fc419a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:38.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.184+0000 7f8fc9807640 1 -- 172.21.15.100:0/3165691887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fc4102510 msgr2=0x7f8fc4199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:38.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.184+0000 7f8fc9807640 1 --2- 172.21.15.100:0/3165691887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fc4102510 0x7f8fc4199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:38.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.184+0000 7f8fc9807640 1 -- 172.21.15.100:0/3165691887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fc4107e50 msgr2=0x7f8fc419a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:38.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.184+0000 7f8fc9807640 1 --2- 172.21.15.100:0/3165691887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fc4107e50 0x7f8fc419a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:38.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.184+0000 7f8fc9807640 1 -- 172.21.15.100:0/3165691887 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc419ff60 con 0x7f8fc41087a0 2024-07-26T13:39:38.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.184+0000 7f8fc8805640 1 --2- 172.21.15.100:0/3165691887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fc4107e50 0x7f8fc419a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:39:38.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.184+0000 7f8fc9807640 1 --2- 172.21.15.100:0/3165691887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fc41087a0 0x7f8fc419f830 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00eb10 tx=0x7f8fbc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:38.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.184+0000 7f8fb67fc640 1 -- 172.21.15.100:0/3165691887 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc004b20 con 0x7f8fc41087a0 2024-07-26T13:39:38.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.184+0000 7f8fcb26a640 1 -- 172.21.15.100:0/3165691887 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fc41a0250 con 0x7f8fc41087a0 2024-07-26T13:39:38.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.184+0000 7f8fb67fc640 1 -- 172.21.15.100:0/3165691887 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fbc004cc0 con 0x7f8fc41087a0 2024-07-26T13:39:38.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.184+0000 7f8fb67fc640 1 -- 172.21.15.100:0/3165691887 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc010670 con 0x7f8fc41087a0 2024-07-26T13:39:38.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.184+0000 7f8fcb26a640 1 -- 172.21.15.100:0/3165691887 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fc41a69b0 con 0x7f8fc41087a0 2024-07-26T13:39:38.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.185+0000 7f8fcb26a640 1 -- 172.21.15.100:0/3165691887 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fc4103790 con 0x7f8fc41087a0 2024-07-26T13:39:38.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.186+0000 7f8fb67fc640 1 -- 172.21.15.100:0/3165691887 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f8fbc005020 con 0x7f8fc41087a0 2024-07-26T13:39:38.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.187+0000 7f8fb67fc640 1 --2- 172.21.15.100:0/3165691887 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f8fb00b3c20 0x7f8fb00b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:38.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.187+0000 7f8fc9006640 1 --2- 172.21.15.100:0/3165691887 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f8fb00b3c20 0x7f8fb00b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:38.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.187+0000 7f8fb67fc640 1 -- 172.21.15.100:0/3165691887 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f8fbc0e4350 con 0x7f8fc41087a0 2024-07-26T13:39:38.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.187+0000 7f8fc9006640 1 --2- 172.21.15.100:0/3165691887 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f8fb00b3c20 0x7f8fb00b6130 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8009bd0 tx=0x7f8fb8005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:38.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.192+0000 7f8fb67fc640 1 -- 172.21.15.100:0/3165691887 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f8fbc0ab8a0 con 0x7f8fc41087a0 2024-07-26T13:39:38.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.371+0000 7f8fcb26a640 1 -- 172.21.15.100:0/3165691887 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f8fc4063230 con 0x7f8fb00b3c20 2024-07-26T13:39:38.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.373+0000 7f8fb67fc640 1 -- 172.21.15.100:0/3165691887 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f8fc4063230 con 0x7f8fb00b3c20 2024-07-26T13:39:38.374 INFO:teuthology.orchestra.run.smithi100.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-07-26T13:39:38.374 INFO:teuthology.orchestra.run.smithi100.stdout:smithi100 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400WR400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-07-26T13:39:38.374 INFO:teuthology.orchestra.run.smithi100.stdout:smithi138 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6161003C400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-07-26T13:39:38.374 INFO:teuthology.orchestra.run.smithi100.stdout:smithi167 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT62330080400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-07-26T13:39:38.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.375+0000 7f8fcb26a640 1 -- 172.21.15.100:0/3165691887 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f8fb00b3c20 msgr2=0x7f8fb00b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:38.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.375+0000 7f8fcb26a640 1 --2- 172.21.15.100:0/3165691887 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f8fb00b3c20 0x7f8fb00b6130 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8009bd0 tx=0x7f8fb8005f90 comp rx=0 tx=0).stop 2024-07-26T13:39:38.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.375+0000 7f8fcb26a640 1 -- 172.21.15.100:0/3165691887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fc41087a0 msgr2=0x7f8fc419f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:38.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.375+0000 7f8fcb26a640 1 --2- 172.21.15.100:0/3165691887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fc41087a0 0x7f8fc419f830 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00eb10 tx=0x7f8fbc00c2d0 comp rx=0 tx=0).stop 2024-07-26T13:39:38.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.375+0000 7f8fcb26a640 1 -- 172.21.15.100:0/3165691887 shutdown_connections 2024-07-26T13:39:38.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.375+0000 7f8fcb26a640 1 --2- 172.21.15.100:0/3165691887 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f8fb00b3c20 0x7f8fb00b6130 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:38.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.375+0000 7f8fcb26a640 1 --2- 172.21.15.100:0/3165691887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8fc41087a0 0x7f8fc419f830 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:38.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.375+0000 7f8fcb26a640 1 --2- 172.21.15.100:0/3165691887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fc4107e50 0x7f8fc419a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:38.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.375+0000 7f8fcb26a640 1 --2- 172.21.15.100:0/3165691887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fc4102510 0x7f8fc4199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:38.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.375+0000 7f8fcb26a640 1 -- 172.21.15.100:0/3165691887 >> 172.21.15.100:0/3165691887 conn(0x7f8fc40791a0 msgr2=0x7f8fc4103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:38.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.375+0000 7f8fcb26a640 1 -- 172.21.15.100:0/3165691887 shutdown_connections 2024-07-26T13:39:38.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:38.375+0000 7f8fcb26a640 1 -- 172.21.15.100:0/3165691887 wait complete. 2024-07-26T13:39:39.065 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-07-26T13:39:39.076 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi100.front.sepia.ceph.com 2024-07-26T13:39:39.076 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- bash -c 'ceph fs dump' 2024-07-26T13:39:39.330 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:39:39.725 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:39 smithi100 ceph-mon[32599]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:39:39.725 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:39 smithi100 ceph-mon[32599]: pgmap v350: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:39:39.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:39 smithi167 ceph-mon[37140]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:39:39.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:39 smithi167 ceph-mon[37140]: pgmap v350: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:39:39.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:39 smithi138 ceph-mon[36954]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:39:39.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:39 smithi138 ceph-mon[36954]: pgmap v350: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:39:41.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.056+0000 7f9b0b3d0640 1 -- 172.21.15.100:0/368833683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b04102b60 msgr2=0x7f9b04107090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:41.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.056+0000 7f9b0b3d0640 1 --2- 172.21.15.100:0/368833683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b04102b60 0x7f9b04107090 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9af4009930 tx=0x7f9af402a420 comp rx=0 tx=0).stop 2024-07-26T13:39:41.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.056+0000 7f9b0b3d0640 1 -- 172.21.15.100:0/368833683 shutdown_connections 2024-07-26T13:39:41.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.056+0000 7f9b0b3d0640 1 --2- 172.21.15.100:0/368833683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b04107650 0x7f9b04107b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:41.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.056+0000 7f9b0b3d0640 1 --2- 172.21.15.100:0/368833683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b04102b60 0x7f9b04107090 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:41.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.056+0000 7f9b0b3d0640 1 --2- 172.21.15.100:0/368833683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b04102210 0x7f9b04102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:41.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.056+0000 7f9b0b3d0640 1 -- 172.21.15.100:0/368833683 >> 172.21.15.100:0/368833683 conn(0x7f9b04079180 msgr2=0x7f9b04079590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:41.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.056+0000 7f9b0b3d0640 1 -- 172.21.15.100:0/368833683 shutdown_connections 2024-07-26T13:39:41.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.056+0000 7f9b0b3d0640 1 -- 172.21.15.100:0/368833683 wait complete. 2024-07-26T13:39:41.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.057+0000 7f9b0b3d0640 1 Processor -- start 2024-07-26T13:39:41.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.057+0000 7f9b0b3d0640 1 -- start start 2024-07-26T13:39:41.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.057+0000 7f9b0b3d0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b04102210 0x7f9b041a0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:41.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.057+0000 7f9b0b3d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b04102b60 0x7f9b041a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:41.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.057+0000 7f9b0b3d0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b04107650 0x7f9b041a5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:41.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.057+0000 7f9b0b3d0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9b04119ab0 con 0x7f9b04102b60 2024-07-26T13:39:41.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.057+0000 7f9b0b3d0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9b04119930 con 0x7f9b04102210 2024-07-26T13:39:41.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.057+0000 7f9b0b3d0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9b04119c30 con 0x7f9b04107650 2024-07-26T13:39:41.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.057+0000 7f9b0896b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b04102b60 0x7f9b041a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:41.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.057+0000 7f9b0896b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b04102b60 0x7f9b041a0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:32836/0 (socket says 172.21.15.100:32836) 2024-07-26T13:39:41.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.057+0000 7f9b0896b640 1 -- 172.21.15.100:0/2938584642 learned_addr learned my addr 172.21.15.100:0/2938584642 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:41.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.057+0000 7f9b0996d640 1 --2- 172.21.15.100:0/2938584642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b04107650 0x7f9b041a5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:41.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.058+0000 7f9b0916c640 1 --2- 172.21.15.100:0/2938584642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b04102210 0x7f9b041a0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:41.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.058+0000 7f9b0896b640 1 -- 172.21.15.100:0/2938584642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b04107650 msgr2=0x7f9b041a5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:41.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.058+0000 7f9b0896b640 1 --2- 172.21.15.100:0/2938584642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b04107650 0x7f9b041a5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:41.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.058+0000 7f9b0896b640 1 -- 172.21.15.100:0/2938584642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b04102210 msgr2=0x7f9b041a0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:41.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.058+0000 7f9b0896b640 1 --2- 172.21.15.100:0/2938584642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b04102210 0x7f9b041a0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:41.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.058+0000 7f9b0896b640 1 -- 172.21.15.100:0/2938584642 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b041a6470 con 0x7f9b04102b60 2024-07-26T13:39:41.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.058+0000 7f9b0916c640 1 --2- 172.21.15.100:0/2938584642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b04102210 0x7f9b041a0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:39:41.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.058+0000 7f9b0896b640 1 --2- 172.21.15.100:0/2938584642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b04102b60 0x7f9b041a0aa0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9af4031440 tx=0x7f9af4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:41.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.058+0000 7f9afa7fc640 1 -- 172.21.15.100:0/2938584642 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af403f030 con 0x7f9b04102b60 2024-07-26T13:39:41.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.058+0000 7f9b0b3d0640 1 -- 172.21.15.100:0/2938584642 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b041a6700 con 0x7f9b04102b60 2024-07-26T13:39:41.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.058+0000 7f9afa7fc640 1 -- 172.21.15.100:0/2938584642 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9af4003aa0 con 0x7f9b04102b60 2024-07-26T13:39:41.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.058+0000 7f9afa7fc640 1 -- 172.21.15.100:0/2938584642 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af4035070 con 0x7f9b04102b60 2024-07-26T13:39:41.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.058+0000 7f9b0b3d0640 1 -- 172.21.15.100:0/2938584642 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b041acec0 con 0x7f9b04102b60 2024-07-26T13:39:41.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.059+0000 7f9b0b3d0640 1 -- 172.21.15.100:0/2938584642 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b0406bc70 con 0x7f9b04102b60 2024-07-26T13:39:41.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.061+0000 7f9afa7fc640 1 -- 172.21.15.100:0/2938584642 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f9af4042030 con 0x7f9b04102b60 2024-07-26T13:39:41.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.062+0000 7f9afa7fc640 1 --2- 172.21.15.100:0/2938584642 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9af00b3c20 0x7f9af00b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:41.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.062+0000 7f9b0916c640 1 --2- 172.21.15.100:0/2938584642 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9af00b3c20 0x7f9af00b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:41.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.062+0000 7f9afa7fc640 1 -- 172.21.15.100:0/2938584642 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f9af41048e0 con 0x7f9b04102b60 2024-07-26T13:39:41.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.063+0000 7f9b0916c640 1 --2- 172.21.15.100:0/2938584642 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9af00b3c20 0x7f9af00b6130 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9b000058c0 tx=0x7f9b00005d30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:41.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.065+0000 7f9afa7fc640 1 -- 172.21.15.100:0/2938584642 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f9af40cbe30 con 0x7f9b04102b60 2024-07-26T13:39:41.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.274+0000 7f9b0b3d0640 1 -- 172.21.15.100:0/2938584642 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f9b041ad090 con 0x7f9b04102b60 2024-07-26T13:39:41.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.274+0000 7f9afa7fc640 1 -- 172.21.15.100:0/2938584642 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3097 (secure 0 0 0) 0x7f9af40d4100 con 0x7f9b04102b60 2024-07-26T13:39:41.275 INFO:teuthology.orchestra.run.smithi100.stdout:e25 2024-07-26T13:39:41.276 INFO:teuthology.orchestra.run.smithi100.stdout:btime 2024-07-26T13:37:19:558299+0000 2024-07-26T13:39:41.276 INFO:teuthology.orchestra.run.smithi100.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-26T13:39:41.276 INFO:teuthology.orchestra.run.smithi100.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-07-26T13:39:41.276 INFO:teuthology.orchestra.run.smithi100.stdout:legacy client fscid: 1 2024-07-26T13:39:41.276 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:39:41.276 INFO:teuthology.orchestra.run.smithi100.stdout:Filesystem 'cephfs' (1) 2024-07-26T13:39:41.276 INFO:teuthology.orchestra.run.smithi100.stdout:fs_name cephfs 2024-07-26T13:39:41.276 INFO:teuthology.orchestra.run.smithi100.stdout:epoch 25 2024-07-26T13:39:41.276 INFO:teuthology.orchestra.run.smithi100.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-26T13:39:41.276 INFO:teuthology.orchestra.run.smithi100.stdout:created 2024-07-26T13:37:01.281843+0000 2024-07-26T13:39:41.277 INFO:teuthology.orchestra.run.smithi100.stdout:modified 2024-07-26T13:37:19.272258+0000 2024-07-26T13:39:41.277 INFO:teuthology.orchestra.run.smithi100.stdout:tableserver 0 2024-07-26T13:39:41.277 INFO:teuthology.orchestra.run.smithi100.stdout:root 0 2024-07-26T13:39:41.277 INFO:teuthology.orchestra.run.smithi100.stdout:session_timeout 300 2024-07-26T13:39:41.277 INFO:teuthology.orchestra.run.smithi100.stdout:session_autoclose 300 2024-07-26T13:39:41.277 INFO:teuthology.orchestra.run.smithi100.stdout:max_file_size 1099511627776 2024-07-26T13:39:41.277 INFO:teuthology.orchestra.run.smithi100.stdout:max_xattr_size 65536 2024-07-26T13:39:41.277 INFO:teuthology.orchestra.run.smithi100.stdout:required_client_features {} 2024-07-26T13:39:41.278 INFO:teuthology.orchestra.run.smithi100.stdout:last_failure 0 2024-07-26T13:39:41.278 INFO:teuthology.orchestra.run.smithi100.stdout:last_failure_osd_epoch 0 2024-07-26T13:39:41.278 INFO:teuthology.orchestra.run.smithi100.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-07-26T13:39:41.278 INFO:teuthology.orchestra.run.smithi100.stdout:max_mds 3 2024-07-26T13:39:41.278 INFO:teuthology.orchestra.run.smithi100.stdout:in 0,1,2 2024-07-26T13:39:41.278 INFO:teuthology.orchestra.run.smithi100.stdout:up {0=24607,1=24605,2=24595} 2024-07-26T13:39:41.278 INFO:teuthology.orchestra.run.smithi100.stdout:failed 2024-07-26T13:39:41.278 INFO:teuthology.orchestra.run.smithi100.stdout:damaged 2024-07-26T13:39:41.278 INFO:teuthology.orchestra.run.smithi100.stdout:stopped 2024-07-26T13:39:41.278 INFO:teuthology.orchestra.run.smithi100.stdout:data_pools [3] 2024-07-26T13:39:41.279 INFO:teuthology.orchestra.run.smithi100.stdout:metadata_pool 2 2024-07-26T13:39:41.279 INFO:teuthology.orchestra.run.smithi100.stdout:inline_data disabled 2024-07-26T13:39:41.279 INFO:teuthology.orchestra.run.smithi100.stdout:balancer 2024-07-26T13:39:41.279 INFO:teuthology.orchestra.run.smithi100.stdout:bal_rank_mask -1 2024-07-26T13:39:41.279 INFO:teuthology.orchestra.run.smithi100.stdout:standby_count_wanted 0 2024-07-26T13:39:41.279 INFO:teuthology.orchestra.run.smithi100.stdout:qdb_cluster leader: 24607 members: 24607,24605,24595 2024-07-26T13:39:41.279 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.i{0:24607} state up:active seq 8 addr [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] compat {c=[1],r=[1],i=[1fff]}] 2024-07-26T13:39:41.279 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.f{0:24586} state up:standby-replay seq 1 addr [v2:172.21.15.167:6834/641259145,v1:172.21.15.167:6835/641259145] compat {c=[1],r=[1],i=[1fff]}] 2024-07-26T13:39:41.279 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.b{1:24605} state up:active seq 9 addr [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] compat {c=[1],r=[1],i=[1fff]}] 2024-07-26T13:39:41.279 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.h{1:24575} state up:standby-replay seq 1 addr [v2:172.21.15.138:6836/3393225483,v1:172.21.15.138:6837/3393225483] compat {c=[1],r=[1],i=[1fff]}] 2024-07-26T13:39:41.279 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.c{2:24595} state up:active seq 13 addr [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] compat {c=[1],r=[1],i=[1fff]}] 2024-07-26T13:39:41.280 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.e{2:24557} state up:standby-replay seq 1 addr [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] compat {c=[1],r=[1],i=[1fff]}] 2024-07-26T13:39:41.280 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:39:41.280 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:39:41.280 INFO:teuthology.orchestra.run.smithi100.stdout:Standby daemons: 2024-07-26T13:39:41.280 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:39:41.280 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.j{-1:14727} state up:standby seq 1 addr [v2:172.21.15.100:6834/2380716258,v1:172.21.15.100:6835/2380716258] compat {c=[1],r=[1],i=[1fff]}] 2024-07-26T13:39:41.280 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.d{-1:14733} state up:standby seq 1 addr [v2:172.21.15.100:6836/3079467433,v1:172.21.15.100:6837/3079467433] compat {c=[1],r=[1],i=[1fff]}] 2024-07-26T13:39:41.280 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.g{-1:14790} state up:standby seq 1 addr [v2:172.21.15.100:6838/313501409,v1:172.21.15.100:6839/313501409] compat {c=[1],r=[1],i=[1fff]}] 2024-07-26T13:39:41.280 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.a{-1:14796} state up:standby seq 1 addr [v2:172.21.15.100:6840/1402975506,v1:172.21.15.100:6841/1402975506] compat {c=[1],r=[1],i=[1fff]}] 2024-07-26T13:39:41.281 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.k{-1:24542} state up:standby seq 1 addr [v2:172.21.15.138:6832/2139594730,v1:172.21.15.138:6833/2139594730] compat {c=[1],r=[1],i=[1fff]}] 2024-07-26T13:39:41.281 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.l{-1:24547} state up:standby seq 1 addr [v2:172.21.15.167:6832/174302302,v1:172.21.15.167:6833/174302302] compat {c=[1],r=[1],i=[1fff]}] 2024-07-26T13:39:41.282 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 25 2024-07-26T13:39:41.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.276+0000 7f9b0b3d0640 1 -- 172.21.15.100:0/2938584642 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9af00b3c20 msgr2=0x7f9af00b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:41.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.276+0000 7f9b0b3d0640 1 --2- 172.21.15.100:0/2938584642 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9af00b3c20 0x7f9af00b6130 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9b000058c0 tx=0x7f9b00005d30 comp rx=0 tx=0).stop 2024-07-26T13:39:41.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.276+0000 7f9b0b3d0640 1 -- 172.21.15.100:0/2938584642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b04102b60 msgr2=0x7f9b041a0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:41.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.276+0000 7f9b0b3d0640 1 --2- 172.21.15.100:0/2938584642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b04102b60 0x7f9b041a0aa0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9af4031440 tx=0x7f9af4004270 comp rx=0 tx=0).stop 2024-07-26T13:39:41.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.276+0000 7f9b0b3d0640 1 -- 172.21.15.100:0/2938584642 shutdown_connections 2024-07-26T13:39:41.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.277+0000 7f9b0b3d0640 1 --2- 172.21.15.100:0/2938584642 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9af00b3c20 0x7f9af00b6130 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:41.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.277+0000 7f9b0b3d0640 1 --2- 172.21.15.100:0/2938584642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b04107650 0x7f9b041a5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:41.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.277+0000 7f9b0b3d0640 1 --2- 172.21.15.100:0/2938584642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b04102b60 0x7f9b041a0aa0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:41.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.277+0000 7f9b0b3d0640 1 --2- 172.21.15.100:0/2938584642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b04102210 0x7f9b041a0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:41.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.277+0000 7f9b0b3d0640 1 -- 172.21.15.100:0/2938584642 >> 172.21.15.100:0/2938584642 conn(0x7f9b04079180 msgr2=0x7f9b04105060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:41.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.277+0000 7f9b0b3d0640 1 -- 172.21.15.100:0/2938584642 shutdown_connections 2024-07-26T13:39:41.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:41.277+0000 7f9b0b3d0640 1 -- 172.21.15.100:0/2938584642 wait complete. 2024-07-26T13:39:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:41 smithi100 ceph-mon[32599]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:39:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:41 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2938584642' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-26T13:39:41.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:41 smithi167 ceph-mon[37140]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:39:41.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:41 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/2938584642' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-26T13:39:41.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:41 smithi138 ceph-mon[36954]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:39:41.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:41 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/2938584642' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-26T13:39:42.021 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- bash -c 'ceph osd dump' 2024-07-26T13:39:42.278 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:39:43.701 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:43 smithi100 ceph-mon[32599]: pgmap v352: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:39:43.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:43 smithi167 ceph-mon[37140]: pgmap v352: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:39:43.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:43 smithi138 ceph-mon[36954]: pgmap v352: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:39:44.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.061+0000 7f6c7a93d640 1 -- 172.21.15.100:0/3561206726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c74108720 msgr2=0x7f6c7410ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:44.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.061+0000 7f6c7a93d640 1 --2- 172.21.15.100:0/3561206726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c74108720 0x7f6c7410ef50 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c009ca0 tx=0x7f6c6c02a8c0 comp rx=0 tx=0).stop 2024-07-26T13:39:44.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.061+0000 7f6c7a93d640 1 -- 172.21.15.100:0/3561206726 shutdown_connections 2024-07-26T13:39:44.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.061+0000 7f6c7a93d640 1 --2- 172.21.15.100:0/3561206726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c74108720 0x7f6c7410ef50 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:44.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.061+0000 7f6c7a93d640 1 --2- 172.21.15.100:0/3561206726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c74107dd0 0x7f6c741081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:44.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.061+0000 7f6c7a93d640 1 --2- 172.21.15.100:0/3561206726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c74102490 0x7f6c741028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:44.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.061+0000 7f6c7a93d640 1 -- 172.21.15.100:0/3561206726 >> 172.21.15.100:0/3561206726 conn(0x7f6c74079160 msgr2=0x7f6c74079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:44.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.061+0000 7f6c7a93d640 1 -- 172.21.15.100:0/3561206726 shutdown_connections 2024-07-26T13:39:44.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.061+0000 7f6c7a93d640 1 -- 172.21.15.100:0/3561206726 wait complete. 2024-07-26T13:39:44.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.062+0000 7f6c7a93d640 1 Processor -- start 2024-07-26T13:39:44.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.062+0000 7f6c7a93d640 1 -- start start 2024-07-26T13:39:44.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.063+0000 7f6c7a93d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c74102490 0x7f6c7419ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:44.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.063+0000 7f6c7a93d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c74107dd0 0x7f6c7419b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:44.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.063+0000 7f6c7a93d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c74108720 0x7f6c7419b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:44.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.063+0000 7f6c7a93d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6c741123f0 con 0x7f6c74102490 2024-07-26T13:39:44.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.063+0000 7f6c7a93d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6c74112270 con 0x7f6c74108720 2024-07-26T13:39:44.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.063+0000 7f6c7a93d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6c74112570 con 0x7f6c74107dd0 2024-07-26T13:39:44.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.063+0000 7f6c73fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c74102490 0x7f6c7419ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:44.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.063+0000 7f6c73fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c74102490 0x7f6c7419ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:32846/0 (socket says 172.21.15.100:32846) 2024-07-26T13:39:44.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.063+0000 7f6c73fff640 1 -- 172.21.15.100:0/2771737539 learned_addr learned my addr 172.21.15.100:0/2771737539 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:44.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.063+0000 7f6c78eda640 1 --2- 172.21.15.100:0/2771737539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c74108720 0x7f6c7419b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:44.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.063+0000 7f6c737fe640 1 --2- 172.21.15.100:0/2771737539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c74107dd0 0x7f6c7419b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:44.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.063+0000 7f6c73fff640 1 -- 172.21.15.100:0/2771737539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c74107dd0 msgr2=0x7f6c7419b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:44.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.063+0000 7f6c73fff640 1 --2- 172.21.15.100:0/2771737539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c74107dd0 0x7f6c7419b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:44.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.063+0000 7f6c73fff640 1 -- 172.21.15.100:0/2771737539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c74108720 msgr2=0x7f6c7419b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:44.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.063+0000 7f6c73fff640 1 --2- 172.21.15.100:0/2771737539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c74108720 0x7f6c7419b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:44.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.063+0000 7f6c73fff640 1 -- 172.21.15.100:0/2771737539 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c741a5f70 con 0x7f6c74102490 2024-07-26T13:39:44.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.063+0000 7f6c78eda640 1 --2- 172.21.15.100:0/2771737539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c74108720 0x7f6c7419b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:39:44.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.063+0000 7f6c737fe640 1 --2- 172.21.15.100:0/2771737539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c74107dd0 0x7f6c7419b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:39:44.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.063+0000 7f6c73fff640 1 --2- 172.21.15.100:0/2771737539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c74102490 0x7f6c7419ace0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6c6400d950 tx=0x7f6c6400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:44.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.063+0000 7f6c717fa640 1 -- 172.21.15.100:0/2771737539 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c64017070 con 0x7f6c74102490 2024-07-26T13:39:44.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.063+0000 7f6c717fa640 1 -- 172.21.15.100:0/2771737539 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c64009c30 con 0x7f6c74102490 2024-07-26T13:39:44.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.063+0000 7f6c7a93d640 1 -- 172.21.15.100:0/2771737539 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c741a6260 con 0x7f6c74102490 2024-07-26T13:39:44.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.063+0000 7f6c717fa640 1 -- 172.21.15.100:0/2771737539 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6400f620 con 0x7f6c74102490 2024-07-26T13:39:44.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.063+0000 7f6c7a93d640 1 -- 172.21.15.100:0/2771737539 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c741a6870 con 0x7f6c74102490 2024-07-26T13:39:44.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.064+0000 7f6c7a93d640 1 -- 172.21.15.100:0/2771737539 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c741028a0 con 0x7f6c74102490 2024-07-26T13:39:44.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.066+0000 7f6c717fa640 1 -- 172.21.15.100:0/2771737539 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f6c64005030 con 0x7f6c74102490 2024-07-26T13:39:44.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.067+0000 7f6c717fa640 1 --2- 172.21.15.100:0/2771737539 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6c600b3cd0 0x7f6c600b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:44.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.067+0000 7f6c737fe640 1 --2- 172.21.15.100:0/2771737539 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6c600b3cd0 0x7f6c600b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:44.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.067+0000 7f6c717fa640 1 -- 172.21.15.100:0/2771737539 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f6c640e39a0 con 0x7f6c74102490 2024-07-26T13:39:44.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.067+0000 7f6c737fe640 1 --2- 172.21.15.100:0/2771737539 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6c600b3cd0 0x7f6c600b61e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c006fd0 tx=0x7f6c5c008040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:44.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.070+0000 7f6c717fa640 1 -- 172.21.15.100:0/2771737539 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f6c640aaf70 con 0x7f6c74102490 2024-07-26T13:39:44.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.244+0000 7f6c7a93d640 1 -- 172.21.15.100:0/2771737539 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f6c7402e0c0 con 0x7f6c74102490 2024-07-26T13:39:44.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.245+0000 7f6c717fa640 1 -- 172.21.15.100:0/2771737539 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v81) ==== 56+0+5748 (secure 0 0 0) 0x7f6c64009de0 con 0x7f6c74102490 2024-07-26T13:39:44.246 INFO:teuthology.orchestra.run.smithi100.stdout:epoch 81 2024-07-26T13:39:44.247 INFO:teuthology.orchestra.run.smithi100.stdout:fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 2024-07-26T13:39:44.247 INFO:teuthology.orchestra.run.smithi100.stdout:created 2024-07-26T13:28:23.639785+0000 2024-07-26T13:39:44.247 INFO:teuthology.orchestra.run.smithi100.stdout:modified 2024-07-26T13:37:17.539521+0000 2024-07-26T13:39:44.247 INFO:teuthology.orchestra.run.smithi100.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-07-26T13:39:44.247 INFO:teuthology.orchestra.run.smithi100.stdout:crush_version 26 2024-07-26T13:39:44.247 INFO:teuthology.orchestra.run.smithi100.stdout:full_ratio 0.95 2024-07-26T13:39:44.247 INFO:teuthology.orchestra.run.smithi100.stdout:backfillfull_ratio 0.9 2024-07-26T13:39:44.247 INFO:teuthology.orchestra.run.smithi100.stdout:nearfull_ratio 0.85 2024-07-26T13:39:44.247 INFO:teuthology.orchestra.run.smithi100.stdout:require_min_compat_client luminous 2024-07-26T13:39:44.247 INFO:teuthology.orchestra.run.smithi100.stdout:min_compat_client luminous 2024-07-26T13:39:44.247 INFO:teuthology.orchestra.run.smithi100.stdout:require_osd_release squid 2024-07-26T13:39:44.247 INFO:teuthology.orchestra.run.smithi100.stdout:stretch_mode_enabled false 2024-07-26T13:39:44.248 INFO:teuthology.orchestra.run.smithi100.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-07-26T13:39:44.248 INFO:teuthology.orchestra.run.smithi100.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-07-26T13:39:44.248 INFO:teuthology.orchestra.run.smithi100.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-07-26T13:39:44.248 INFO:teuthology.orchestra.run.smithi100.stdout:max_osd 12 2024-07-26T13:39:44.248 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 up in weight 1 up_from 9 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6802/1480401850,v1:172.21.15.100:6803/1480401850] [v2:172.21.15.100:6804/1480401850,v1:172.21.15.100:6805/1480401850] exists,up d714036d-9dda-4cb5-af16-29f83a09dd70 2024-07-26T13:39:44.248 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 up in weight 1 up_from 14 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6800/3090877671,v1:172.21.15.138:6801/3090877671] [v2:172.21.15.138:6802/3090877671,v1:172.21.15.138:6803/3090877671] exists,up 3dd0eaa1-618a-495a-9819-e01c0f869ae8 2024-07-26T13:39:44.248 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 up in weight 1 up_from 19 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6800/3583424790,v1:172.21.15.167:6801/3583424790] [v2:172.21.15.167:6802/3583424790,v1:172.21.15.167:6803/3583424790] exists,up 5c710a78-17ae-41a8-ac16-a678f5d35672 2024-07-26T13:39:44.248 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 up in weight 1 up_from 26 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6810/3626762419,v1:172.21.15.100:6811/3626762419] [v2:172.21.15.100:6812/3626762419,v1:172.21.15.100:6813/3626762419] exists,up be845495-8c31-4834-a45c-bdc576711962 2024-07-26T13:39:44.248 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 up in weight 1 up_from 32 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6808/3757334356,v1:172.21.15.138:6809/3757334356] [v2:172.21.15.138:6810/3757334356,v1:172.21.15.138:6811/3757334356] exists,up 520554cc-e846-46f9-af77-05a1de65a452 2024-07-26T13:39:44.248 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 up in weight 1 up_from 37 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6808/3239482836,v1:172.21.15.167:6809/3239482836] [v2:172.21.15.167:6810/3239482836,v1:172.21.15.167:6811/3239482836] exists,up 68dc8ad6-31d2-45b5-9aa7-62c4218d46e2 2024-07-26T13:39:44.248 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 up in weight 1 up_from 43 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6818/3218415484,v1:172.21.15.100:6819/3218415484] [v2:172.21.15.100:6820/3218415484,v1:172.21.15.100:6821/3218415484] exists,up 420e4471-5626-41cc-a8e1-69c801196b2c 2024-07-26T13:39:44.248 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 up in weight 1 up_from 48 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6816/45541749,v1:172.21.15.138:6817/45541749] [v2:172.21.15.138:6818/45541749,v1:172.21.15.138:6819/45541749] exists,up a27b097e-207d-44c3-af63-544f228b83ec 2024-07-26T13:39:44.249 INFO:teuthology.orchestra.run.smithi100.stdout:osd.8 up in weight 1 up_from 54 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6816/4031159803,v1:172.21.15.167:6817/4031159803] [v2:172.21.15.167:6818/4031159803,v1:172.21.15.167:6819/4031159803] exists,up 3e865858-92cc-48a0-a464-5767d19e8ac0 2024-07-26T13:39:44.249 INFO:teuthology.orchestra.run.smithi100.stdout:osd.9 up in weight 1 up_from 60 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6826/1840140273,v1:172.21.15.100:6827/1840140273] [v2:172.21.15.100:6828/1840140273,v1:172.21.15.100:6829/1840140273] exists,up 8b05b14a-1195-42f5-ac01-d5634cb9c888 2024-07-26T13:39:44.249 INFO:teuthology.orchestra.run.smithi100.stdout:osd.10 up in weight 1 up_from 65 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6824/703382899,v1:172.21.15.138:6825/703382899] [v2:172.21.15.138:6826/703382899,v1:172.21.15.138:6827/703382899] exists,up ab77ebe7-5254-4dc3-bc54-625698e84afc 2024-07-26T13:39:44.249 INFO:teuthology.orchestra.run.smithi100.stdout:osd.11 up in weight 1 up_from 71 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6824/1032343068,v1:172.21.15.167:6825/1032343068] [v2:172.21.15.167:6826/1032343068,v1:172.21.15.167:6827/1032343068] exists,up 229cf91e-e9ef-48f6-a677-c5c5705b315a 2024-07-26T13:39:44.249 INFO:teuthology.orchestra.run.smithi100.stdout:pg_upmap_items 2.5 [8,11] 2024-07-26T13:39:44.249 INFO:teuthology.orchestra.run.smithi100.stdout:pg_upmap_items 2.b [2,4] 2024-07-26T13:39:44.249 INFO:teuthology.orchestra.run.smithi100.stdout:pg_upmap_items 2.1b [2,1] 2024-07-26T13:39:44.249 INFO:teuthology.orchestra.run.smithi100.stdout:pg_upmap_items 2.1c [8,11] 2024-07-26T13:39:44.249 INFO:teuthology.orchestra.run.smithi100.stdout:pg_upmap_items 3.0 [4,6] 2024-07-26T13:39:44.249 INFO:teuthology.orchestra.run.smithi100.stdout:pg_upmap_items 3.6 [3,7] 2024-07-26T13:39:44.249 INFO:teuthology.orchestra.run.smithi100.stdout:pg_upmap_items 3.1b [5,9] 2024-07-26T13:39:44.249 INFO:teuthology.orchestra.run.smithi100.stdout:pg_upmap_items 3.38 [3,9] 2024-07-26T13:39:44.249 INFO:teuthology.orchestra.run.smithi100.stdout:pg_upmap_items 3.3a [4,8] 2024-07-26T13:39:44.250 INFO:teuthology.orchestra.run.smithi100.stdout:pg_upmap_items 3.3c [4,9] 2024-07-26T13:39:44.250 INFO:teuthology.orchestra.run.smithi100.stdout:blocklist 172.21.15.100:0/1874993112 expires 2024-07-27T13:30:16.253096+0000 2024-07-26T13:39:44.250 INFO:teuthology.orchestra.run.smithi100.stdout:blocklist 172.21.15.100:0/2364620305 expires 2024-07-27T13:28:45.340334+0000 2024-07-26T13:39:44.250 INFO:teuthology.orchestra.run.smithi100.stdout:blocklist 172.21.15.100:6800/3096076075 expires 2024-07-27T13:29:07.979178+0000 2024-07-26T13:39:44.250 INFO:teuthology.orchestra.run.smithi100.stdout:blocklist 172.21.15.100:0/1894966501 expires 2024-07-27T13:30:16.253096+0000 2024-07-26T13:39:44.250 INFO:teuthology.orchestra.run.smithi100.stdout:blocklist 172.21.15.100:0/3411776672 expires 2024-07-27T13:28:45.340334+0000 2024-07-26T13:39:44.250 INFO:teuthology.orchestra.run.smithi100.stdout:blocklist 172.21.15.100:0/2853423120 expires 2024-07-27T13:29:07.979178+0000 2024-07-26T13:39:44.250 INFO:teuthology.orchestra.run.smithi100.stdout:blocklist 172.21.15.100:6800/3025873570 expires 2024-07-27T13:28:45.340334+0000 2024-07-26T13:39:44.250 INFO:teuthology.orchestra.run.smithi100.stdout:blocklist 172.21.15.100:0/4887181 expires 2024-07-27T13:29:07.979178+0000 2024-07-26T13:39:44.250 INFO:teuthology.orchestra.run.smithi100.stdout:blocklist 172.21.15.100:6801/908675140 expires 2024-07-27T13:30:16.253096+0000 2024-07-26T13:39:44.250 INFO:teuthology.orchestra.run.smithi100.stdout:blocklist 172.21.15.100:6801/3025873570 expires 2024-07-27T13:28:45.340334+0000 2024-07-26T13:39:44.250 INFO:teuthology.orchestra.run.smithi100.stdout:blocklist 172.21.15.100:6800/908675140 expires 2024-07-27T13:30:16.253096+0000 2024-07-26T13:39:44.251 INFO:teuthology.orchestra.run.smithi100.stdout:blocklist 172.21.15.100:6801/3096076075 expires 2024-07-27T13:29:07.979178+0000 2024-07-26T13:39:44.251 INFO:teuthology.orchestra.run.smithi100.stdout:blocklist 172.21.15.100:0/2302632052 expires 2024-07-27T13:29:07.979178+0000 2024-07-26T13:39:44.251 INFO:teuthology.orchestra.run.smithi100.stdout:blocklist 172.21.15.100:0/3736098915 expires 2024-07-27T13:28:45.340334+0000 2024-07-26T13:39:44.251 INFO:teuthology.orchestra.run.smithi100.stdout:blocklist 172.21.15.100:0/3243030560 expires 2024-07-27T13:30:16.253096+0000 2024-07-26T13:39:44.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.248+0000 7f6c7a93d640 1 -- 172.21.15.100:0/2771737539 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6c600b3cd0 msgr2=0x7f6c600b61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:44.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.248+0000 7f6c7a93d640 1 --2- 172.21.15.100:0/2771737539 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6c600b3cd0 0x7f6c600b61e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c006fd0 tx=0x7f6c5c008040 comp rx=0 tx=0).stop 2024-07-26T13:39:44.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.248+0000 7f6c7a93d640 1 -- 172.21.15.100:0/2771737539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c74102490 msgr2=0x7f6c7419ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:44.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.248+0000 7f6c7a93d640 1 --2- 172.21.15.100:0/2771737539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c74102490 0x7f6c7419ace0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6c6400d950 tx=0x7f6c6400de10 comp rx=0 tx=0).stop 2024-07-26T13:39:44.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.248+0000 7f6c7a93d640 1 -- 172.21.15.100:0/2771737539 shutdown_connections 2024-07-26T13:39:44.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.248+0000 7f6c7a93d640 1 --2- 172.21.15.100:0/2771737539 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6c600b3cd0 0x7f6c600b61e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:44.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.248+0000 7f6c7a93d640 1 --2- 172.21.15.100:0/2771737539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c74108720 0x7f6c7419b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:44.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.248+0000 7f6c7a93d640 1 --2- 172.21.15.100:0/2771737539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c74107dd0 0x7f6c7419b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:44.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.248+0000 7f6c7a93d640 1 --2- 172.21.15.100:0/2771737539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c74102490 0x7f6c7419ace0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:44.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.248+0000 7f6c7a93d640 1 -- 172.21.15.100:0/2771737539 >> 172.21.15.100:0/2771737539 conn(0x7f6c74079160 msgr2=0x7f6c74104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:44.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.248+0000 7f6c7a93d640 1 -- 172.21.15.100:0/2771737539 shutdown_connections 2024-07-26T13:39:44.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:44.248+0000 7f6c7a93d640 1 -- 172.21.15.100:0/2771737539 wait complete. 2024-07-26T13:39:44.568 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:44 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2771737539' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-07-26T13:39:44.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:44 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/2771737539' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-07-26T13:39:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:44 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/2771737539' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-07-26T13:39:44.981 INFO:teuthology.run_tasks:Running task fs.ready... 2024-07-26T13:39:44.993 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:39:45.247 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:39:45.688 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:45 smithi100 ceph-mon[32599]: pgmap v353: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:39:45.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:45 smithi167 ceph-mon[37140]: pgmap v353: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:39:45.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:45 smithi138 ceph-mon[36954]: pgmap v353: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:39:47.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.003+0000 7f55597f3640 1 -- 172.21.15.100:0/1207197605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5554107dd0 msgr2=0x7f55541081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:47.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.003+0000 7f55597f3640 1 --2- 172.21.15.100:0/1207197605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5554107dd0 0x7f55541081e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5548009ca0 tx=0x7f554802a8c0 comp rx=0 tx=0).stop 2024-07-26T13:39:47.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.003+0000 7f55597f3640 1 -- 172.21.15.100:0/1207197605 shutdown_connections 2024-07-26T13:39:47.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.003+0000 7f55597f3640 1 --2- 172.21.15.100:0/1207197605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5554108720 0x7f555410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:47.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.003+0000 7f55597f3640 1 --2- 172.21.15.100:0/1207197605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5554107dd0 0x7f55541081e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:47.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.003+0000 7f55597f3640 1 --2- 172.21.15.100:0/1207197605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5554102490 0x7f55541028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:47.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.003+0000 7f55597f3640 1 -- 172.21.15.100:0/1207197605 >> 172.21.15.100:0/1207197605 conn(0x7f5554079160 msgr2=0x7f5554079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:47.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.004+0000 7f55597f3640 1 -- 172.21.15.100:0/1207197605 shutdown_connections 2024-07-26T13:39:47.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.004+0000 7f55597f3640 1 -- 172.21.15.100:0/1207197605 wait complete. 2024-07-26T13:39:47.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.004+0000 7f55597f3640 1 Processor -- start 2024-07-26T13:39:47.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.004+0000 7f55597f3640 1 -- start start 2024-07-26T13:39:47.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.005+0000 7f55597f3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5554102490 0x7f5554198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:47.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.005+0000 7f55597f3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5554107dd0 0x7f5554198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:47.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.005+0000 7f55597f3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5554108720 0x7f5554199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:47.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.005+0000 7f55597f3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5554112370 con 0x7f5554107dd0 2024-07-26T13:39:47.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.005+0000 7f55597f3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f55541121f0 con 0x7f5554108720 2024-07-26T13:39:47.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.005+0000 7f55597f3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f55541124f0 con 0x7f5554102490 2024-07-26T13:39:47.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.005+0000 7f55527fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5554107dd0 0x7f5554198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:47.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.005+0000 7f55527fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5554107dd0 0x7f5554198ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46912/0 (socket says 172.21.15.100:46912) 2024-07-26T13:39:47.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.005+0000 7f55527fc640 1 -- 172.21.15.100:0/2785428927 learned_addr learned my addr 172.21.15.100:0/2785428927 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:47.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.005+0000 7f5552ffd640 1 --2- 172.21.15.100:0/2785428927 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5554102490 0x7f5554198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:47.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.005+0000 7f55527fc640 1 -- 172.21.15.100:0/2785428927 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5554102490 msgr2=0x7f5554198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:47.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.005+0000 7f55527fc640 1 --2- 172.21.15.100:0/2785428927 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5554102490 0x7f5554198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:47.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.005+0000 7f55527fc640 1 -- 172.21.15.100:0/2785428927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5554108720 msgr2=0x7f5554199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:47.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.005+0000 7f55537fe640 1 --2- 172.21.15.100:0/2785428927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5554108720 0x7f5554199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:47.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.005+0000 7f55527fc640 1 --2- 172.21.15.100:0/2785428927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5554108720 0x7f5554199530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:47.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.005+0000 7f55527fc640 1 -- 172.21.15.100:0/2785428927 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55541a8160 con 0x7f5554107dd0 2024-07-26T13:39:47.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.005+0000 7f55527fc640 1 --2- 172.21.15.100:0/2785428927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5554107dd0 0x7f5554198ff0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5548033040 tx=0x7f55480316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:47.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.005+0000 7f553bfff640 1 -- 172.21.15.100:0/2785428927 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5548003b70 con 0x7f5554107dd0 2024-07-26T13:39:47.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.006+0000 7f55597f3640 1 -- 172.21.15.100:0/2785428927 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55541a83f0 con 0x7f5554107dd0 2024-07-26T13:39:47.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.006+0000 7f553bfff640 1 -- 172.21.15.100:0/2785428927 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5548003d10 con 0x7f5554107dd0 2024-07-26T13:39:47.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.006+0000 7f553bfff640 1 -- 172.21.15.100:0/2785428927 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554802fa80 con 0x7f5554107dd0 2024-07-26T13:39:47.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.006+0000 7f55597f3640 1 -- 172.21.15.100:0/2785428927 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55541a8980 con 0x7f5554107dd0 2024-07-26T13:39:47.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.007+0000 7f55597f3640 1 -- 172.21.15.100:0/2785428927 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55541028a0 con 0x7f5554107dd0 2024-07-26T13:39:47.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.008+0000 7f553bfff640 1 -- 172.21.15.100:0/2785428927 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f554802fc20 con 0x7f5554107dd0 2024-07-26T13:39:47.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.009+0000 7f553bfff640 1 --2- 172.21.15.100:0/2785428927 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f553c0b3c20 0x7f553c0b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:47.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.009+0000 7f5552ffd640 1 --2- 172.21.15.100:0/2785428927 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f553c0b3c20 0x7f553c0b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:47.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.009+0000 7f553bfff640 1 -- 172.21.15.100:0/2785428927 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f5548105380 con 0x7f5554107dd0 2024-07-26T13:39:47.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.009+0000 7f5552ffd640 1 --2- 172.21.15.100:0/2785428927 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f553c0b3c20 0x7f553c0b6130 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f55541074a0 tx=0x7f554c005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:47.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.012+0000 7f553bfff640 1 -- 172.21.15.100:0/2785428927 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f55480cc960 con 0x7f5554107dd0 2024-07-26T13:39:47.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.220+0000 7f55597f3640 1 -- 172.21.15.100:0/2785428927 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f555406aca0 con 0x7f5554107dd0 2024-07-26T13:39:47.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.221+0000 7f553bfff640 1 -- 172.21.15.100:0/2785428927 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11927 (secure 0 0 0) 0x7f555406aca0 con 0x7f5554107dd0 2024-07-26T13:39:47.223 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:39:47.223 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":25,"btime":"2024-07-26T13:37:19:558299+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14790,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/313501409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":313501409},{"type":"v1","addr":"172.21.15.100:6839","nonce":313501409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/1402975506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":1402975506},{"type":"v1","addr":"172.21.15.100:6841","nonce":1402975506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-26T13:37:01.281843+0000","modified":"2024-07-26T13:37:19.272258+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24607,"mds_1":24605,"mds_2":24595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24557":{"gid":24557,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/3290105300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3290105300},{"type":"v1","addr":"172.21.15.138:6835","nonce":3290105300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/3393225483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3393225483},{"type":"v1","addr":"172.21.15.138:6837","nonce":3393225483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24595":{"gid":24595,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6837/3828634468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3828634468},{"type":"v1","addr":"172.21.15.167:6837","nonce":3828634468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.138:6839/1580925471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1580925471},{"type":"v1","addr":"172.21.15.138:6839","nonce":1580925471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/398150514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":398150514},{"type":"v1","addr":"172.21.15.167:6839","nonce":398150514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24607,"qdb_cluster":[24607,24605,24595]},"id":1}]} 2024-07-26T13:39:47.223 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 25 2024-07-26T13:39:47.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.224+0000 7f55597f3640 1 -- 172.21.15.100:0/2785428927 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f553c0b3c20 msgr2=0x7f553c0b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:47.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.224+0000 7f55597f3640 1 --2- 172.21.15.100:0/2785428927 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f553c0b3c20 0x7f553c0b6130 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f55541074a0 tx=0x7f554c005f90 comp rx=0 tx=0).stop 2024-07-26T13:39:47.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.224+0000 7f55597f3640 1 -- 172.21.15.100:0/2785428927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5554107dd0 msgr2=0x7f5554198ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:47.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.224+0000 7f55597f3640 1 --2- 172.21.15.100:0/2785428927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5554107dd0 0x7f5554198ff0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5548033040 tx=0x7f55480316f0 comp rx=0 tx=0).stop 2024-07-26T13:39:47.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.224+0000 7f55597f3640 1 -- 172.21.15.100:0/2785428927 shutdown_connections 2024-07-26T13:39:47.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.224+0000 7f55597f3640 1 --2- 172.21.15.100:0/2785428927 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f553c0b3c20 0x7f553c0b6130 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:47.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.224+0000 7f55597f3640 1 --2- 172.21.15.100:0/2785428927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5554108720 0x7f5554199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:47.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.224+0000 7f55597f3640 1 --2- 172.21.15.100:0/2785428927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5554107dd0 0x7f5554198ff0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:47.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.224+0000 7f55597f3640 1 --2- 172.21.15.100:0/2785428927 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5554102490 0x7f5554198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:47.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.224+0000 7f55597f3640 1 -- 172.21.15.100:0/2785428927 >> 172.21.15.100:0/2785428927 conn(0x7f5554079160 msgr2=0x7f5554103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:47.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.224+0000 7f55597f3640 1 -- 172.21.15.100:0/2785428927 shutdown_connections 2024-07-26T13:39:47.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:47.224+0000 7f55597f3640 1 -- 172.21.15.100:0/2785428927 wait complete. 2024-07-26T13:39:47.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:47 smithi100 ceph-mon[32599]: pgmap v354: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T13:39:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:47 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2785428927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:39:47.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:47 smithi167 ceph-mon[37140]: pgmap v354: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T13:39:47.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:47 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/2785428927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:39:47.883 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:39:47.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:47 smithi138 ceph-mon[36954]: pgmap v354: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T13:39:47.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:47 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/2785428927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:39:48.141 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:39:49.628 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:49 smithi100 ceph-mon[32599]: pgmap v355: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:39:49.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:49 smithi167 ceph-mon[37140]: pgmap v355: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:39:49.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.846+0000 7f3906f71640 1 -- 172.21.15.100:0/3180826647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39001024c0 msgr2=0x7f39001028d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:49.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.846+0000 7f3906f71640 1 --2- 172.21.15.100:0/3180826647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39001024c0 0x7f39001028d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f38f4009900 tx=0x7f38f402a410 comp rx=0 tx=0).stop 2024-07-26T13:39:49.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.846+0000 7f3906f71640 1 -- 172.21.15.100:0/3180826647 shutdown_connections 2024-07-26T13:39:49.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.846+0000 7f3906f71640 1 --2- 172.21.15.100:0/3180826647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3900108750 0x7f390010ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:49.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.846+0000 7f3906f71640 1 --2- 172.21.15.100:0/3180826647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3900107e00 0x7f3900108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:49.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.846+0000 7f3906f71640 1 --2- 172.21.15.100:0/3180826647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39001024c0 0x7f39001028d0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:49.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.846+0000 7f3906f71640 1 -- 172.21.15.100:0/3180826647 >> 172.21.15.100:0/3180826647 conn(0x7f3900079180 msgr2=0x7f3900079590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:49.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.846+0000 7f3906f71640 1 -- 172.21.15.100:0/3180826647 shutdown_connections 2024-07-26T13:39:49.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.846+0000 7f3906f71640 1 -- 172.21.15.100:0/3180826647 wait complete. 2024-07-26T13:39:49.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.847+0000 7f3906f71640 1 Processor -- start 2024-07-26T13:39:49.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.847+0000 7f3906f71640 1 -- start start 2024-07-26T13:39:49.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.847+0000 7f3906f71640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39001024c0 0x7f3900198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:49.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.847+0000 7f3906f71640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3900107e00 0x7f3900199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:49.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.847+0000 7f3906f71640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3900108750 0x7f390019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:49.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.848+0000 7f3906f71640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f39001123d0 con 0x7f39001024c0 2024-07-26T13:39:49.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.848+0000 7f3906f71640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3900112250 con 0x7f3900107e00 2024-07-26T13:39:49.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.848+0000 7f3906f71640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3900112550 con 0x7f3900108750 2024-07-26T13:39:49.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.848+0000 7f3904d0d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39001024c0 0x7f3900198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:49.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.848+0000 7f3904d0d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39001024c0 0x7f3900198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46918/0 (socket says 172.21.15.100:46918) 2024-07-26T13:39:49.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.848+0000 7f3904d0d640 1 -- 172.21.15.100:0/3023956858 learned_addr learned my addr 172.21.15.100:0/3023956858 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:49.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.848+0000 7f390550e640 1 --2- 172.21.15.100:0/3023956858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3900108750 0x7f390019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:49.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.848+0000 7f38fffff640 1 --2- 172.21.15.100:0/3023956858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3900107e00 0x7f3900199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:49.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.848+0000 7f3904d0d640 1 -- 172.21.15.100:0/3023956858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3900108750 msgr2=0x7f390019e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:49.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.848+0000 7f3904d0d640 1 --2- 172.21.15.100:0/3023956858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3900108750 0x7f390019e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:49.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.848+0000 7f3904d0d640 1 -- 172.21.15.100:0/3023956858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3900107e00 msgr2=0x7f3900199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:49.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.848+0000 7f3904d0d640 1 --2- 172.21.15.100:0/3023956858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3900107e00 0x7f3900199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:49.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.848+0000 7f3904d0d640 1 -- 172.21.15.100:0/3023956858 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f390019ed10 con 0x7f39001024c0 2024-07-26T13:39:49.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.848+0000 7f3904d0d640 1 --2- 172.21.15.100:0/3023956858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39001024c0 0x7f3900198e60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f38f4005940 tx=0x7f38f40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:49.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.848+0000 7f38fdffb640 1 -- 172.21.15.100:0/3023956858 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f403f030 con 0x7f39001024c0 2024-07-26T13:39:49.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.848+0000 7f38fdffb640 1 -- 172.21.15.100:0/3023956858 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38f4003cb0 con 0x7f39001024c0 2024-07-26T13:39:49.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.848+0000 7f3906f71640 1 -- 172.21.15.100:0/3023956858 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f390019efa0 con 0x7f39001024c0 2024-07-26T13:39:49.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.848+0000 7f38fdffb640 1 -- 172.21.15.100:0/3023956858 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f4035070 con 0x7f39001024c0 2024-07-26T13:39:49.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.848+0000 7f3906f71640 1 -- 172.21.15.100:0/3023956858 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f390019f480 con 0x7f39001024c0 2024-07-26T13:39:49.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.849+0000 7f3906f71640 1 -- 172.21.15.100:0/3023956858 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3900103740 con 0x7f39001024c0 2024-07-26T13:39:49.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.851+0000 7f38fdffb640 1 -- 172.21.15.100:0/3023956858 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f38f4042030 con 0x7f39001024c0 2024-07-26T13:39:49.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.852+0000 7f38fdffb640 1 --2- 172.21.15.100:0/3023956858 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f38f00b3c50 0x7f38f00b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:49.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.852+0000 7f38fffff640 1 --2- 172.21.15.100:0/3023956858 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f38f00b3c50 0x7f38f00b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:49.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.852+0000 7f38fdffb640 1 -- 172.21.15.100:0/3023956858 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f38f41045a0 con 0x7f39001024c0 2024-07-26T13:39:49.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.852+0000 7f38fffff640 1 --2- 172.21.15.100:0/3023956858 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f38f00b3c50 0x7f38f00b6160 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f38ec005fd0 tx=0x7f38ec005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:49.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:49.855+0000 7f38fdffb640 1 -- 172.21.15.100:0/3023956858 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f38f40cbaf0 con 0x7f39001024c0 2024-07-26T13:39:49.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:49 smithi138 ceph-mon[36954]: pgmap v355: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:39:50.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:50.065+0000 7f3906f71640 1 -- 172.21.15.100:0/3023956858 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39001028d0 con 0x7f39001024c0 2024-07-26T13:39:50.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:50.065+0000 7f38fdffb640 1 -- 172.21.15.100:0/3023956858 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11927 (secure 0 0 0) 0x7f38f40cb3f0 con 0x7f39001024c0 2024-07-26T13:39:50.067 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:39:50.067 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":25,"btime":"2024-07-26T13:37:19:558299+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14790,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/313501409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":313501409},{"type":"v1","addr":"172.21.15.100:6839","nonce":313501409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/1402975506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":1402975506},{"type":"v1","addr":"172.21.15.100:6841","nonce":1402975506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-26T13:37:01.281843+0000","modified":"2024-07-26T13:37:19.272258+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24607,"mds_1":24605,"mds_2":24595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24557":{"gid":24557,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/3290105300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3290105300},{"type":"v1","addr":"172.21.15.138:6835","nonce":3290105300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/3393225483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3393225483},{"type":"v1","addr":"172.21.15.138:6837","nonce":3393225483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24595":{"gid":24595,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6837/3828634468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3828634468},{"type":"v1","addr":"172.21.15.167:6837","nonce":3828634468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.138:6839/1580925471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1580925471},{"type":"v1","addr":"172.21.15.138:6839","nonce":1580925471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/398150514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":398150514},{"type":"v1","addr":"172.21.15.167:6839","nonce":398150514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24607,"qdb_cluster":[24607,24605,24595]},"id":1}]} 2024-07-26T13:39:50.067 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 25 2024-07-26T13:39:50.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:50.068+0000 7f3906f71640 1 -- 172.21.15.100:0/3023956858 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f38f00b3c50 msgr2=0x7f38f00b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:50.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:50.068+0000 7f3906f71640 1 --2- 172.21.15.100:0/3023956858 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f38f00b3c50 0x7f38f00b6160 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f38ec005fd0 tx=0x7f38ec005850 comp rx=0 tx=0).stop 2024-07-26T13:39:50.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:50.068+0000 7f3906f71640 1 -- 172.21.15.100:0/3023956858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39001024c0 msgr2=0x7f3900198e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:50.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:50.068+0000 7f3906f71640 1 --2- 172.21.15.100:0/3023956858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39001024c0 0x7f3900198e60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f38f4005940 tx=0x7f38f40042f0 comp rx=0 tx=0).stop 2024-07-26T13:39:50.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:50.068+0000 7f3906f71640 1 -- 172.21.15.100:0/3023956858 shutdown_connections 2024-07-26T13:39:50.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:50.068+0000 7f3906f71640 1 --2- 172.21.15.100:0/3023956858 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f38f00b3c50 0x7f38f00b6160 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:50.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:50.068+0000 7f3906f71640 1 --2- 172.21.15.100:0/3023956858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3900108750 0x7f390019e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:50.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:50.068+0000 7f3906f71640 1 --2- 172.21.15.100:0/3023956858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3900107e00 0x7f3900199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:50.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:50.068+0000 7f3906f71640 1 --2- 172.21.15.100:0/3023956858 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39001024c0 0x7f3900198e60 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:50.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:50.068+0000 7f3906f71640 1 -- 172.21.15.100:0/3023956858 >> 172.21.15.100:0/3023956858 conn(0x7f3900079180 msgr2=0x7f3900100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:50.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:50.068+0000 7f3906f71640 1 -- 172.21.15.100:0/3023956858 shutdown_connections 2024-07-26T13:39:50.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:50.068+0000 7f3906f71640 1 -- 172.21.15.100:0/3023956858 wait complete. 2024-07-26T13:39:50.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:50 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/3023956858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:39:50.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:50 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/3023956858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:39:50.793 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph osd dump --format=json 2024-07-26T13:39:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:50 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/3023956858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:39:51.048 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:39:51.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:51 smithi100 ceph-mon[32599]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:39:51.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:51 smithi167 ceph-mon[37140]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:39:51.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:51 smithi138 ceph-mon[36954]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:39:52.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.752+0000 7fce97efa640 1 -- 172.21.15.100:0/2187953927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce90101cd0 msgr2=0x7fce901020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:52.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.752+0000 7fce97efa640 1 --2- 172.21.15.100:0/2187953927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce90101cd0 0x7fce901020e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fce8c00aa10 tx=0x7fce8c02b750 comp rx=0 tx=0).stop 2024-07-26T13:39:52.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.752+0000 7fce97efa640 1 -- 172.21.15.100:0/2187953927 shutdown_connections 2024-07-26T13:39:52.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.752+0000 7fce97efa640 1 --2- 172.21.15.100:0/2187953927 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce90107f60 0x7fce9010e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:52.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.752+0000 7fce97efa640 1 --2- 172.21.15.100:0/2187953927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce90107610 0x7fce90107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:52.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.752+0000 7fce97efa640 1 --2- 172.21.15.100:0/2187953927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce90101cd0 0x7fce901020e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:52.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.752+0000 7fce97efa640 1 -- 172.21.15.100:0/2187953927 >> 172.21.15.100:0/2187953927 conn(0x7fce900798f0 msgr2=0x7fce90079d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:52.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.752+0000 7fce97efa640 1 -- 172.21.15.100:0/2187953927 shutdown_connections 2024-07-26T13:39:52.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.752+0000 7fce97efa640 1 -- 172.21.15.100:0/2187953927 wait complete. 2024-07-26T13:39:52.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.753+0000 7fce97efa640 1 Processor -- start 2024-07-26T13:39:52.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.753+0000 7fce97efa640 1 -- start start 2024-07-26T13:39:52.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.753+0000 7fce97efa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce90101cd0 0x7fce90199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:52.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.753+0000 7fce97efa640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce90107610 0x7fce9019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:52.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.753+0000 7fce97efa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce90107f60 0x7fce9019f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:52.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.753+0000 7fce97efa640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fce90111400 con 0x7fce90101cd0 2024-07-26T13:39:52.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.753+0000 7fce97efa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fce90111280 con 0x7fce90107f60 2024-07-26T13:39:52.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.753+0000 7fce97efa640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fce90111580 con 0x7fce90107610 2024-07-26T13:39:52.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.753+0000 7fce95c96640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce90101cd0 0x7fce90199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:52.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.753+0000 7fce95c96640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce90101cd0 0x7fce90199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46946/0 (socket says 172.21.15.100:46946) 2024-07-26T13:39:52.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.753+0000 7fce95c96640 1 -- 172.21.15.100:0/3537044445 learned_addr learned my addr 172.21.15.100:0/3537044445 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:52.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.754+0000 7fce95495640 1 --2- 172.21.15.100:0/3537044445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce90107610 0x7fce9019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:52.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.754+0000 7fce96497640 1 --2- 172.21.15.100:0/3537044445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce90107f60 0x7fce9019f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:52.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.754+0000 7fce95c96640 1 -- 172.21.15.100:0/3537044445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce90107610 msgr2=0x7fce9019a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:52.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.754+0000 7fce95c96640 1 --2- 172.21.15.100:0/3537044445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce90107610 0x7fce9019a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:52.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.754+0000 7fce95c96640 1 -- 172.21.15.100:0/3537044445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce90107f60 msgr2=0x7fce9019f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:52.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.754+0000 7fce95c96640 1 --2- 172.21.15.100:0/3537044445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce90107f60 0x7fce9019f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:52.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.754+0000 7fce95c96640 1 -- 172.21.15.100:0/3537044445 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce9019ff30 con 0x7fce90101cd0 2024-07-26T13:39:52.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.754+0000 7fce96497640 1 --2- 172.21.15.100:0/3537044445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce90107f60 0x7fce9019f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:39:52.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.754+0000 7fce95c96640 1 --2- 172.21.15.100:0/3537044445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce90101cd0 0x7fce90199fc0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fce8c006010 tx=0x7fce8c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:52.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.754+0000 7fce86ffd640 1 -- 172.21.15.100:0/3537044445 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8c03f070 con 0x7fce90101cd0 2024-07-26T13:39:52.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.754+0000 7fce86ffd640 1 -- 172.21.15.100:0/3537044445 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce8c032400 con 0x7fce90101cd0 2024-07-26T13:39:52.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.754+0000 7fce97efa640 1 -- 172.21.15.100:0/3537044445 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce901a01c0 con 0x7fce90101cd0 2024-07-26T13:39:52.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.754+0000 7fce86ffd640 1 -- 172.21.15.100:0/3537044445 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8c036040 con 0x7fce90101cd0 2024-07-26T13:39:52.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.754+0000 7fce97efa640 1 -- 172.21.15.100:0/3537044445 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce901a6980 con 0x7fce90101cd0 2024-07-26T13:39:52.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.755+0000 7fce97efa640 1 -- 172.21.15.100:0/3537044445 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce901020e0 con 0x7fce90101cd0 2024-07-26T13:39:52.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.757+0000 7fce86ffd640 1 -- 172.21.15.100:0/3537044445 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7fce8c042070 con 0x7fce90101cd0 2024-07-26T13:39:52.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.758+0000 7fce86ffd640 1 --2- 172.21.15.100:0/3537044445 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fce7c0b3ca0 0x7fce7c0b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:52.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.758+0000 7fce95495640 1 --2- 172.21.15.100:0/3537044445 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fce7c0b3ca0 0x7fce7c0b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:52.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.758+0000 7fce86ffd640 1 -- 172.21.15.100:0/3537044445 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fce8c104890 con 0x7fce90101cd0 2024-07-26T13:39:52.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.758+0000 7fce95495640 1 --2- 172.21.15.100:0/3537044445 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fce7c0b3ca0 0x7fce7c0b61b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fce90107250 tx=0x7fce80008040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:52.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.761+0000 7fce86ffd640 1 -- 172.21.15.100:0/3537044445 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fce8c0cbdc0 con 0x7fce90101cd0 2024-07-26T13:39:52.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.937+0000 7fce97efa640 1 -- 172.21.15.100:0/3537044445 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce9019b0f0 con 0x7fce90101cd0 2024-07-26T13:39:52.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.939+0000 7fce86ffd640 1 -- 172.21.15.100:0/3537044445 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+23121 (secure 0 0 0) 0x7fce9019b0f0 con 0x7fce90101cd0 2024-07-26T13:39:52.940 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:39:52.940 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":81,"fsid":"bf83e2e6-4b52-11ef-bcb7-c7b262605968","created":"2024-07-26T13:28:23.639785+0000","modified":"2024-07-26T13:37:17.539521+0000","last_up_change":"2024-07-26T13:35:56.475554+0000","last_in_change":"2024-07-26T13:35:37.472564+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-26T13:32:22.329969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-26T13:36:54.590525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-26T13:36:58.149545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d714036d-9dda-4cb5-af16-29f83a09dd70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6803","nonce":1480401850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6805","nonce":1480401850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6809","nonce":1480401850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6807","nonce":1480401850}]},"public_addr":"172.21.15.100:6803/1480401850","cluster_addr":"172.21.15.100:6805/1480401850","heartbeat_back_addr":"172.21.15.100:6809/1480401850","heartbeat_front_addr":"172.21.15.100:6807/1480401850","state":["exists","up"]},{"osd":1,"uuid":"3dd0eaa1-618a-495a-9819-e01c0f869ae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6801","nonce":3090877671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6803","nonce":3090877671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6807","nonce":3090877671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6805","nonce":3090877671}]},"public_addr":"172.21.15.138:6801/3090877671","cluster_addr":"172.21.15.138:6803/3090877671","heartbeat_back_addr":"172.21.15.138:6807/3090877671","heartbeat_front_addr":"172.21.15.138:6805/3090877671","state":["exists","up"]},{"osd":2,"uuid":"5c710a78-17ae-41a8-ac16-a678f5d35672","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6801","nonce":3583424790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6803","nonce":3583424790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6807","nonce":3583424790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6805","nonce":3583424790}]},"public_addr":"172.21.15.167:6801/3583424790","cluster_addr":"172.21.15.167:6803/3583424790","heartbeat_back_addr":"172.21.15.167:6807/3583424790","heartbeat_front_addr":"172.21.15.167:6805/3583424790","state":["exists","up"]},{"osd":3,"uuid":"be845495-8c31-4834-a45c-bdc576711962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6811","nonce":3626762419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6813","nonce":3626762419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6817","nonce":3626762419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6815","nonce":3626762419}]},"public_addr":"172.21.15.100:6811/3626762419","cluster_addr":"172.21.15.100:6813/3626762419","heartbeat_back_addr":"172.21.15.100:6817/3626762419","heartbeat_front_addr":"172.21.15.100:6815/3626762419","state":["exists","up"]},{"osd":4,"uuid":"520554cc-e846-46f9-af77-05a1de65a452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6809","nonce":3757334356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6811","nonce":3757334356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6815","nonce":3757334356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6813","nonce":3757334356}]},"public_addr":"172.21.15.138:6809/3757334356","cluster_addr":"172.21.15.138:6811/3757334356","heartbeat_back_addr":"172.21.15.138:6815/3757334356","heartbeat_front_addr":"172.21.15.138:6813/3757334356","state":["exists","up"]},{"osd":5,"uuid":"68dc8ad6-31d2-45b5-9aa7-62c4218d46e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6809","nonce":3239482836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6811","nonce":3239482836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6815","nonce":3239482836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6813","nonce":3239482836}]},"public_addr":"172.21.15.167:6809/3239482836","cluster_addr":"172.21.15.167:6811/3239482836","heartbeat_back_addr":"172.21.15.167:6815/3239482836","heartbeat_front_addr":"172.21.15.167:6813/3239482836","state":["exists","up"]},{"osd":6,"uuid":"420e4471-5626-41cc-a8e1-69c801196b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6819","nonce":3218415484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6821","nonce":3218415484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6825","nonce":3218415484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6823","nonce":3218415484}]},"public_addr":"172.21.15.100:6819/3218415484","cluster_addr":"172.21.15.100:6821/3218415484","heartbeat_back_addr":"172.21.15.100:6825/3218415484","heartbeat_front_addr":"172.21.15.100:6823/3218415484","state":["exists","up"]},{"osd":7,"uuid":"a27b097e-207d-44c3-af63-544f228b83ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6817","nonce":45541749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6819","nonce":45541749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6823","nonce":45541749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6821","nonce":45541749}]},"public_addr":"172.21.15.138:6817/45541749","cluster_addr":"172.21.15.138:6819/45541749","heartbeat_back_addr":"172.21.15.138:6823/45541749","heartbeat_front_addr":"172.21.15.138:6821/45541749","state":["exists","up"]},{"osd":8,"uuid":"3e865858-92cc-48a0-a464-5767d19e8ac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6817","nonce":4031159803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6819","nonce":4031159803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6823","nonce":4031159803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6821","nonce":4031159803}]},"public_addr":"172.21.15.167:6817/4031159803","cluster_addr":"172.21.15.167:6819/4031159803","heartbeat_back_addr":"172.21.15.167:6823/4031159803","heartbeat_front_addr":"172.21.15.167:6821/4031159803","state":["exists","up"]},{"osd":9,"uuid":"8b05b14a-1195-42f5-ac01-d5634cb9c888","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6827","nonce":1840140273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6829","nonce":1840140273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6833","nonce":1840140273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6831","nonce":1840140273}]},"public_addr":"172.21.15.100:6827/1840140273","cluster_addr":"172.21.15.100:6829/1840140273","heartbeat_back_addr":"172.21.15.100:6833/1840140273","heartbeat_front_addr":"172.21.15.100:6831/1840140273","state":["exists","up"]},{"osd":10,"uuid":"ab77ebe7-5254-4dc3-bc54-625698e84afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6825","nonce":703382899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6827","nonce":703382899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6831","nonce":703382899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6829","nonce":703382899}]},"public_addr":"172.21.15.138:6825/703382899","cluster_addr":"172.21.15.138:6827/703382899","heartbeat_back_addr":"172.21.15.138:6831/703382899","heartbeat_front_addr":"172.21.15.138:6829/703382899","state":["exists","up"]},{"osd":11,"uuid":"229cf91e-e9ef-48f6-a677-c5c5705b315a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6825","nonce":1032343068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6827","nonce":1032343068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6831","nonce":1032343068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6829","nonce":1032343068}]},"public_addr":"172.21.15.167:6825/1032343068","cluster_addr":"172.21.15.167:6827/1032343068","heartbeat_back_addr":"172.21.15.167:6831/1032343068","heartbeat_front_addr":"172.21.15.167:6829/1032343068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:31:35.800702+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:31:58.087922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:32:20.463183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:32:42.920598+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:33:06.332655+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:33:29.381407+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:33:53.975963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:34:18.226813+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:34:41.845770+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:35:05.722275+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:35:30.862704+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:35:54.738697+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:0/1874993112":"2024-07-27T13:30:16.253096+0000","172.21.15.100:0/2364620305":"2024-07-27T13:28:45.340334+0000","172.21.15.100:6800/3096076075":"2024-07-27T13:29:07.979178+0000","172.21.15.100:0/1894966501":"2024-07-27T13:30:16.253096+0000","172.21.15.100:0/3411776672":"2024-07-27T13:28:45.340334+0000","172.21.15.100:0/2853423120":"2024-07-27T13:29:07.979178+0000","172.21.15.100:6800/3025873570":"2024-07-27T13:28:45.340334+0000","172.21.15.100:0/4887181":"2024-07-27T13:29:07.979178+0000","172.21.15.100:6801/908675140":"2024-07-27T13:30:16.253096+0000","172.21.15.100:6801/3025873570":"2024-07-27T13:28:45.340334+0000","172.21.15.100:6800/908675140":"2024-07-27T13:30:16.253096+0000","172.21.15.100:6801/3096076075":"2024-07-27T13:29:07.979178+0000","172.21.15.100:0/2302632052":"2024-07-27T13:29:07.979178+0000","172.21.15.100:0/3736098915":"2024-07-27T13:28:45.340334+0000","172.21.15.100:0/3243030560":"2024-07-27T13:30:16.253096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-26T13:39:52.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.941+0000 7fce97efa640 1 -- 172.21.15.100:0/3537044445 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fce7c0b3ca0 msgr2=0x7fce7c0b61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:52.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.941+0000 7fce97efa640 1 --2- 172.21.15.100:0/3537044445 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fce7c0b3ca0 0x7fce7c0b61b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fce90107250 tx=0x7fce80008040 comp rx=0 tx=0).stop 2024-07-26T13:39:52.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.941+0000 7fce97efa640 1 -- 172.21.15.100:0/3537044445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce90101cd0 msgr2=0x7fce90199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:52.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.941+0000 7fce97efa640 1 --2- 172.21.15.100:0/3537044445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce90101cd0 0x7fce90199fc0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fce8c006010 tx=0x7fce8c0042f0 comp rx=0 tx=0).stop 2024-07-26T13:39:52.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.941+0000 7fce97efa640 1 -- 172.21.15.100:0/3537044445 shutdown_connections 2024-07-26T13:39:52.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.941+0000 7fce97efa640 1 --2- 172.21.15.100:0/3537044445 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fce7c0b3ca0 0x7fce7c0b61b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:52.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.941+0000 7fce97efa640 1 --2- 172.21.15.100:0/3537044445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce90107f60 0x7fce9019f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:52.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.941+0000 7fce97efa640 1 --2- 172.21.15.100:0/3537044445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce90107610 0x7fce9019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:52.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.941+0000 7fce97efa640 1 --2- 172.21.15.100:0/3537044445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce90101cd0 0x7fce90199fc0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:52.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.941+0000 7fce97efa640 1 -- 172.21.15.100:0/3537044445 >> 172.21.15.100:0/3537044445 conn(0x7fce900798f0 msgr2=0x7fce90105660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:52.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.941+0000 7fce97efa640 1 -- 172.21.15.100:0/3537044445 shutdown_connections 2024-07-26T13:39:52.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:52.941+0000 7fce97efa640 1 -- 172.21.15.100:0/3537044445 wait complete. 2024-07-26T13:39:53.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:53 smithi100 ceph-mon[32599]: pgmap v357: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:39:53.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:53 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/3537044445' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-26T13:39:53.725 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-07-26T13:37:01.281843+0000', 'modified': '2024-07-26T13:37:19.272258+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24607, 'mds_1': 24605, 'mds_2': 24595}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24557': {'gid': 24557, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.138:6835/3290105300', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6834', 'nonce': 3290105300}, {'type': 'v1', 'addr': '172.21.15.138:6835', 'nonce': 3290105300}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24575': {'gid': 24575, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.138:6837/3393225483', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6836', 'nonce': 3393225483}, {'type': 'v1', 'addr': '172.21.15.138:6837', 'nonce': 3393225483}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24586': {'gid': 24586, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6835/641259145', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6834', 'nonce': 641259145}, {'type': 'v1', 'addr': '172.21.15.167:6835', 'nonce': 641259145}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24595': {'gid': 24595, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.167:6837/3828634468', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6836', 'nonce': 3828634468}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 3828634468}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24605': {'gid': 24605, 'name': 'b', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.138:6839/1580925471', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6838', 'nonce': 1580925471}, {'type': 'v1', 'addr': '172.21.15.138:6839', 'nonce': 1580925471}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24607': {'gid': 24607, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.167:6839/398150514', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6838', 'nonce': 398150514}, {'type': 'v1', 'addr': '172.21.15.167:6839', 'nonce': 398150514}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24607, 'qdb_cluster': [24607, 24605, 24595]} 2024-07-26T13:39:53.725 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-07-26T13:39:53.725 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.c status 2024-07-26T13:39:53.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:53 smithi167 ceph-mon[37140]: pgmap v357: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:39:53.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:53 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/3537044445' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-26T13:39:53.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:53 smithi138 ceph-mon[36954]: pgmap v357: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:39:53.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:53 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/3537044445' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-26T13:39:53.982 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:39:55.742 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:55 smithi138 ceph-mon[36954]: pgmap v358: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:39:55.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:55 smithi100 ceph-mon[32599]: pgmap v358: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:39:55.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:55 smithi167 ceph-mon[37140]: pgmap v358: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:39:55.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.758+0000 7f658cae9640 1 -- 172.21.15.100:0/540187397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6588108ed0 msgr2=0x7f65881092e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:55.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.758+0000 7f658cae9640 1 --2- 172.21.15.100:0/540187397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6588108ed0 0x7f65881092e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6570009900 tx=0x7f657002a410 comp rx=0 tx=0).stop 2024-07-26T13:39:55.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.758+0000 7f658cae9640 1 -- 172.21.15.100:0/540187397 shutdown_connections 2024-07-26T13:39:55.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.758+0000 7f658cae9640 1 --2- 172.21.15.100:0/540187397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6588109820 0x7f6588110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:55.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.758+0000 7f658cae9640 1 --2- 172.21.15.100:0/540187397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6588108ed0 0x7f65881092e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:55.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.758+0000 7f658cae9640 1 --2- 172.21.15.100:0/540187397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6588103590 0x7f65881039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:55.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.758+0000 7f658cae9640 1 -- 172.21.15.100:0/540187397 >> 172.21.15.100:0/540187397 conn(0x7f658807b280 msgr2=0x7f658807b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:55.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.758+0000 7f658cae9640 1 -- 172.21.15.100:0/540187397 shutdown_connections 2024-07-26T13:39:55.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.758+0000 7f658cae9640 1 -- 172.21.15.100:0/540187397 wait complete. 2024-07-26T13:39:55.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.758+0000 7f658cae9640 1 Processor -- start 2024-07-26T13:39:55.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.759+0000 7f658cae9640 1 -- start start 2024-07-26T13:39:55.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.759+0000 7f658cae9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6588103590 0x7f6588199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:55.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.759+0000 7f658cae9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6588108ed0 0x7f658819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:55.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.759+0000 7f658cae9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6588109820 0x7f658819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:55.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.759+0000 7f658cae9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f65881134b0 con 0x7f6588108ed0 2024-07-26T13:39:55.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.759+0000 7f658cae9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6588113330 con 0x7f6588109820 2024-07-26T13:39:55.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.759+0000 7f658cae9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6588113630 con 0x7f6588103590 2024-07-26T13:39:55.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.759+0000 7f657dd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6588108ed0 0x7f658819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:55.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.759+0000 7f657dd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6588108ed0 0x7f658819a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38368/0 (socket says 172.21.15.100:38368) 2024-07-26T13:39:55.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.759+0000 7f657dd9b640 1 -- 172.21.15.100:0/3387455343 learned_addr learned my addr 172.21.15.100:0/3387455343 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.759+0000 7f6586d9d640 1 --2- 172.21.15.100:0/3387455343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6588109820 0x7f658819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.759+0000 7f658659c640 1 --2- 172.21.15.100:0/3387455343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6588103590 0x7f6588199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.759+0000 7f657dd9b640 1 -- 172.21.15.100:0/3387455343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6588103590 msgr2=0x7f6588199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.759+0000 7f657dd9b640 1 --2- 172.21.15.100:0/3387455343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6588103590 0x7f6588199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.759+0000 7f657dd9b640 1 -- 172.21.15.100:0/3387455343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6588109820 msgr2=0x7f658819f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.759+0000 7f657dd9b640 1 --2- 172.21.15.100:0/3387455343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6588109820 0x7f658819f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.759+0000 7f657dd9b640 1 -- 172.21.15.100:0/3387455343 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f658819feb0 con 0x7f6588108ed0 2024-07-26T13:39:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.759+0000 7f658659c640 1 --2- 172.21.15.100:0/3387455343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6588103590 0x7f6588199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:39:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.760+0000 7f657dd9b640 1 --2- 172.21.15.100:0/3387455343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6588108ed0 0x7f658819a4e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f657002a930 tx=0x7f6570004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.760+0000 7f657ffff640 1 -- 172.21.15.100:0/3387455343 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657003f030 con 0x7f6588108ed0 2024-07-26T13:39:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.760+0000 7f658cae9640 1 -- 172.21.15.100:0/3387455343 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65881a0140 con 0x7f6588108ed0 2024-07-26T13:39:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.760+0000 7f657ffff640 1 -- 172.21.15.100:0/3387455343 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6570003aa0 con 0x7f6588108ed0 2024-07-26T13:39:55.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.760+0000 7f657ffff640 1 -- 172.21.15.100:0/3387455343 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6570035070 con 0x7f6588108ed0 2024-07-26T13:39:55.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.760+0000 7f658cae9640 1 -- 172.21.15.100:0/3387455343 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65881a6900 con 0x7f6588108ed0 2024-07-26T13:39:55.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.763+0000 7f659306d740 1 Processor -- start 2024-07-26T13:39:55.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.763+0000 7f657ffff640 1 -- 172.21.15.100:0/3387455343 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f6570042030 con 0x7f6588108ed0 2024-07-26T13:39:55.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.763+0000 7f659306d740 1 -- start start 2024-07-26T13:39:55.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.763+0000 7f659306d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3462bff60 0x55c3466653a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:55.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.763+0000 7f659306d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3462970a0 0x55c3466658e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:55.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.763+0000 7f659306d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c34629f280 0x55c346665e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:55.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.763+0000 7f659306d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c3465db940 con 0x55c3462bff60 2024-07-26T13:39:55.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.763+0000 7f659306d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c3465cae40 con 0x55c3462970a0 2024-07-26T13:39:55.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.763+0000 7f659306d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c3465cee80 con 0x55c34629f280 2024-07-26T13:39:55.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.763+0000 7f6586d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3462bff60 0x55c3466653a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:55.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.763+0000 7f6586d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3462bff60 0x55c3466653a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38376/0 (socket says 172.21.15.100:38376) 2024-07-26T13:39:55.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.763+0000 7f6586d9d640 1 -- 172.21.15.100:0/1873714898 learned_addr learned my addr 172.21.15.100:0/1873714898 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:55.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.763+0000 7f658659c640 1 --2- 172.21.15.100:0/1873714898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3462970a0 0x55c3466658e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:55.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.763+0000 7f657dd9b640 1 --2- 172.21.15.100:0/1873714898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c34629f280 0x55c346665e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:55.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.764+0000 7f6586d9d640 1 -- 172.21.15.100:0/1873714898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c34629f280 msgr2=0x55c346665e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:55.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.764+0000 7f6586d9d640 1 --2- 172.21.15.100:0/1873714898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c34629f280 0x55c346665e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:55.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.764+0000 7f6586d9d640 1 -- 172.21.15.100:0/1873714898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3462970a0 msgr2=0x55c3466658e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:55.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.764+0000 7f6586d9d640 1 --2- 172.21.15.100:0/1873714898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3462970a0 0x55c3466658e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:55.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.764+0000 7f6586d9d640 1 -- 172.21.15.100:0/1873714898 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c346571a60 con 0x55c3462bff60 2024-07-26T13:39:55.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.764+0000 7f657dd9b640 1 --2- 172.21.15.100:0/1873714898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c34629f280 0x55c346665e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:39:55.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.764+0000 7f657ffff640 1 --2- 172.21.15.100:0/3387455343 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f65740b3c20 0x7f65740b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:55.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.764+0000 7f6586d9d640 1 --2- 172.21.15.100:0/1873714898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3462bff60 0x55c3466653a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6580009f70 tx=0x7f6580010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e95f2aeda87db2ab server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:55.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.764+0000 7f658659c640 1 --2- 172.21.15.100:0/3387455343 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f65740b3c20 0x7f65740b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:55.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.764+0000 7f657ffff640 1 -- 172.21.15.100:0/3387455343 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f6570104410 con 0x7f6588108ed0 2024-07-26T13:39:55.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.764+0000 7f657cd99640 1 -- 172.21.15.100:0/1873714898 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6580016070 con 0x55c3462bff60 2024-07-26T13:39:55.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.764+0000 7f657cd99640 1 -- 172.21.15.100:0/1873714898 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6580004d20 con 0x55c3462bff60 2024-07-26T13:39:55.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.764+0000 7f657cd99640 1 -- 172.21.15.100:0/1873714898 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6580012790 con 0x55c3462bff60 2024-07-26T13:39:55.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.765+0000 7f659306d740 1 -- 172.21.15.100:0/1873714898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3462bff60 msgr2=0x55c3466653a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:55.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.765+0000 7f658659c640 1 --2- 172.21.15.100:0/3387455343 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f65740b3c20 0x7f65740b6130 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x55c346664020 tx=0x7f6578009330 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:55.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.765+0000 7f659306d740 1 --2- 172.21.15.100:0/1873714898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3462bff60 0x55c3466653a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6580009f70 tx=0x7f6580010800 comp rx=0 tx=0).stop 2024-07-26T13:39:55.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.765+0000 7f659306d740 1 -- 172.21.15.100:0/1873714898 shutdown_connections 2024-07-26T13:39:55.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.765+0000 7f659306d740 1 --2- 172.21.15.100:0/1873714898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c34629f280 0x55c346665e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:55.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.765+0000 7f659306d740 1 --2- 172.21.15.100:0/1873714898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3462970a0 0x55c3466658e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:55.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.765+0000 7f659306d740 1 --2- 172.21.15.100:0/1873714898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3462bff60 0x55c3466653a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:55.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.765+0000 7f659306d740 1 -- 172.21.15.100:0/1873714898 >> 172.21.15.100:0/1873714898 conn(0x55c3462b4350 msgr2=0x55c346663290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:55.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.765+0000 7f659306d740 1 -- 172.21.15.100:0/1873714898 shutdown_connections 2024-07-26T13:39:55.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.765+0000 7f659306d740 1 -- 172.21.15.100:0/1873714898 wait complete. 2024-07-26T13:39:55.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.765+0000 7f659306d740 1 -- start start 2024-07-26T13:39:55.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.766+0000 7f659306d740 1 Processor -- start 2024-07-26T13:39:55.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.766+0000 7f659306d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3462970a0 0x55c346670f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:55.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.766+0000 7f659306d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c34629f280 0x55c3466714c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:55.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.766+0000 7f659306d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3462bff60 0x55c34667ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:55.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.766+0000 7f659306d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c34642e5f0 con 0x55c3462970a0 2024-07-26T13:39:55.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.766+0000 7f659306d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c34656d8b0 con 0x55c3462bff60 2024-07-26T13:39:55.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.766+0000 7f659306d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c346673b80 con 0x55c34629f280 2024-07-26T13:39:55.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.766+0000 7f659306d740 10 client.0 _ll_register_callbacks cb 0x55c3465777f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:39:55.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.766+0000 7f658659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3462970a0 0x55c346670f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:55.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.766+0000 7f658659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3462970a0 0x55c346670f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38390/0 (socket says 172.21.15.100:38390) 2024-07-26T13:39:55.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.766+0000 7f658659c640 1 -- 172.21.15.100:0/3388325606 learned_addr learned my addr 172.21.15.100:0/3388325606 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:55.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.766+0000 7f6586d9d640 1 --2- 172.21.15.100:0/3388325606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3462bff60 0x55c34667ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:55.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.766+0000 7f657dd9b640 1 --2- 172.21.15.100:0/3388325606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c34629f280 0x55c3466714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:55.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.766+0000 7f658659c640 1 -- 172.21.15.100:0/3388325606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c34629f280 msgr2=0x55c3466714c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:55.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.766+0000 7f658659c640 1 --2- 172.21.15.100:0/3388325606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c34629f280 0x55c3466714c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:55.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.766+0000 7f658659c640 1 -- 172.21.15.100:0/3388325606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3462bff60 msgr2=0x55c34667ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:55.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.766+0000 7f658659c640 1 --2- 172.21.15.100:0/3388325606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3462bff60 0x55c34667ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:55.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.766+0000 7f658659c640 1 -- 172.21.15.100:0/3388325606 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c34632b540 con 0x55c3462970a0 2024-07-26T13:39:55.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.766+0000 7f657dd9b640 1 --2- 172.21.15.100:0/3388325606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c34629f280 0x55c3466714c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:39:55.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.766+0000 7f658659c640 1 --2- 172.21.15.100:0/3388325606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3462970a0 0x55c346670f80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f65780124a0 tx=0x7f6578004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=313a003be8e37b23 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:55.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.766+0000 7f656dffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:39:55.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.767+0000 7f656dffb640 1 -- 172.21.15.100:0/3388325606 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6578023070 con 0x55c3462970a0 2024-07-26T13:39:55.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.767+0000 7f656dffb640 1 -- 172.21.15.100:0/3388325606 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f657801f860 con 0x55c3462970a0 2024-07-26T13:39:55.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.767+0000 7f659306d740 1 -- 172.21.15.100:0/3388325606 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6580004d20 con 0x55c3462970a0 2024-07-26T13:39:55.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.767+0000 7f656dffb640 1 -- 172.21.15.100:0/3388325606 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f657802f050 con 0x55c3462970a0 2024-07-26T13:39:55.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.767+0000 7f659306d740 10 client.15276 fetch_fsmap learned FSMap version 25 2024-07-26T13:39:55.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.767+0000 7f659306d740 1 -- 172.21.15.100:0/3388325606 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c34667fb90 con 0x55c3462970a0 2024-07-26T13:39:55.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.767+0000 7f656dffb640 1 -- 172.21.15.100:0/3388325606 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6580004d20 con 0x55c3462970a0 2024-07-26T13:39:55.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.767+0000 7f656dffb640 1 -- 172.21.15.100:0/3388325606 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f657800f040 con 0x55c3462970a0 2024-07-26T13:39:55.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.767+0000 7f659306d740 10 client.15276 fetch_fsmap finished waiting for FSMap version 25 2024-07-26T13:39:55.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.767+0000 7f659306d740 10 client.15276 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-26T13:39:55.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.767+0000 7f659306d740 20 client.15276 populate_metadata read hostname 'smithi100' 2024-07-26T13:39:55.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.767+0000 7f659306d740 1 --2- 172.21.15.100:0/3388325606 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55c34627e8e0 0x55c346683680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:55.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.767+0000 7f659306d740 4 client.15276 mds_command: new command op to 24595 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:39:55.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.767+0000 7f659306d740 1 -- 172.21.15.100:0/3388325606 --> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3465dd890 con 0x55c34627e8e0 2024-07-26T13:39:55.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.767+0000 7f657dd9b640 1 --2- 172.21.15.100:0/3388325606 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55c34627e8e0 0x55c346683680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:55.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.768+0000 7f656dffb640 10 client.15276 ms_handle_connect on v2:172.21.15.167:6836/3828634468 2024-07-26T13:39:55.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.768+0000 7f657dd9b640 1 --2- 172.21.15.100:0/3388325606 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55c34627e8e0 0x55c346683680 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=87534e783f3e6d0a server_cookie=33217d856aea50d6 in_seq=0 out_seq=0 2024-07-26T13:39:55.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.769+0000 7f656dffb640 1 -- 172.21.15.100:0/3388325606 <== mds.2 v2:172.21.15.167:6836/3828634468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c3465dd890 con 0x55c34627e8e0 2024-07-26T13:39:55.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.769+0000 7f656dffb640 10 client.15276 handle_command_reply: tid=0 2024-07-26T13:39:55.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.769+0000 7f659306d740 1 client.15276 shutdown 2024-07-26T13:39:55.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.769+0000 7f659306d740 1 -- 172.21.15.100:0/3388325606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3462970a0 msgr2=0x55c346670f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:55.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.769+0000 7f659306d740 1 --2- 172.21.15.100:0/3388325606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3462970a0 0x55c346670f80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f65780124a0 tx=0x7f6578004730 comp rx=0 tx=0).stop 2024-07-26T13:39:55.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.769+0000 7f659306d740 1 -- 172.21.15.100:0/3388325606 shutdown_connections 2024-07-26T13:39:55.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.769+0000 7f659306d740 1 --2- 172.21.15.100:0/3388325606 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55c34627e8e0 0x55c346683680 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:55.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.769+0000 7f659306d740 1 --2- 172.21.15.100:0/3388325606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3462bff60 0x55c34667ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:55.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.769+0000 7f659306d740 1 --2- 172.21.15.100:0/3388325606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c34629f280 0x55c3466714c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:55.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.769+0000 7f656dffb640 0 client.15276 ms_handle_reset on v2:172.21.15.167:6836/3828634468 2024-07-26T13:39:55.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.769+0000 7f659306d740 1 --2- 172.21.15.100:0/3388325606 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3462970a0 0x55c346670f80 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:55.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.769+0000 7f659306d740 1 -- 172.21.15.100:0/3388325606 >> 172.21.15.100:0/3388325606 conn(0x55c3462b4350 msgr2=0x55c346668580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:55.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.769+0000 7f659306d740 1 -- 172.21.15.100:0/3388325606 shutdown_connections 2024-07-26T13:39:55.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.769+0000 7f659306d740 1 -- 172.21.15.100:0/3388325606 wait complete. 2024-07-26T13:39:55.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.769+0000 7f659306d740 20 client.15276 trim_cache size 0 max 16384 2024-07-26T13:39:55.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.787+0000 7f659306d740 1 Processor -- start 2024-07-26T13:39:55.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.787+0000 7f659306d740 1 -- start start 2024-07-26T13:39:55.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.787+0000 7f659306d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3462970a0 0x55c34667ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:55.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.787+0000 7f659306d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c34629f280 0x55c34667d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:55.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.787+0000 7f659306d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c3462bff60 0x55c346670f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:55.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.787+0000 7f659306d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c3466819c0 con 0x55c34629f280 2024-07-26T13:39:55.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.787+0000 7f659306d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c346682090 con 0x55c3462970a0 2024-07-26T13:39:55.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.787+0000 7f659306d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c346682760 con 0x55c3462bff60 2024-07-26T13:39:55.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.787+0000 7f657dd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c34629f280 0x55c34667d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:55.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.787+0000 7f657dd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c34629f280 0x55c34667d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38394/0 (socket says 172.21.15.100:38394) 2024-07-26T13:39:55.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.787+0000 7f657dd9b640 1 -- 172.21.15.100:0/4153392201 learned_addr learned my addr 172.21.15.100:0/4153392201 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:55.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.787+0000 7f658659c640 1 --2- 172.21.15.100:0/4153392201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3462970a0 0x55c34667ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:55.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.787+0000 7f657dd9b640 1 -- 172.21.15.100:0/4153392201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c3462bff60 msgr2=0x55c346670f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:55.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.787+0000 7f657dd9b640 1 --2- 172.21.15.100:0/4153392201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c3462bff60 0x55c346670f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:55.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.787+0000 7f657dd9b640 1 -- 172.21.15.100:0/4153392201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3462970a0 msgr2=0x55c34667ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:55.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.787+0000 7f657dd9b640 1 --2- 172.21.15.100:0/4153392201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3462970a0 0x55c34667ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:55.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.787+0000 7f657dd9b640 1 -- 172.21.15.100:0/4153392201 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c346667690 con 0x55c34629f280 2024-07-26T13:39:55.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.788+0000 7f657dd9b640 1 --2- 172.21.15.100:0/4153392201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c34629f280 0x55c34667d380 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6570031440 tx=0x7f6570003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a070f3d72d7032e9 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:55.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.788+0000 7f656ffff640 1 -- 172.21.15.100:0/4153392201 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6570049040 con 0x55c34629f280 2024-07-26T13:39:55.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.788+0000 7f656ffff640 1 -- 172.21.15.100:0/4153392201 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65700d4310 con 0x55c34629f280 2024-07-26T13:39:55.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.788+0000 7f656ffff640 1 -- 172.21.15.100:0/4153392201 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65700c74f0 con 0x55c34629f280 2024-07-26T13:39:55.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.788+0000 7f659306d740 1 -- 172.21.15.100:0/4153392201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c34629f280 msgr2=0x55c34667d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:55.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.788+0000 7f659306d740 1 --2- 172.21.15.100:0/4153392201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c34629f280 0x55c34667d380 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6570031440 tx=0x7f6570003f50 comp rx=0 tx=0).stop 2024-07-26T13:39:55.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.789+0000 7f659306d740 1 -- 172.21.15.100:0/4153392201 shutdown_connections 2024-07-26T13:39:55.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.789+0000 7f659306d740 1 --2- 172.21.15.100:0/4153392201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c3462bff60 0x55c346670f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:55.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.789+0000 7f659306d740 1 --2- 172.21.15.100:0/4153392201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c34629f280 0x55c34667d380 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:55.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.789+0000 7f659306d740 1 --2- 172.21.15.100:0/4153392201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3462970a0 0x55c34667ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:55.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.789+0000 7f659306d740 1 -- 172.21.15.100:0/4153392201 >> 172.21.15.100:0/4153392201 conn(0x55c3462b4350 msgr2=0x55c34666f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:55.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.789+0000 7f659306d740 1 -- 172.21.15.100:0/4153392201 shutdown_connections 2024-07-26T13:39:55.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.789+0000 7f659306d740 1 -- 172.21.15.100:0/4153392201 wait complete. 2024-07-26T13:39:55.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.789+0000 7f659306d740 1 -- start start 2024-07-26T13:39:55.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.789+0000 7f659306d740 1 Processor -- start 2024-07-26T13:39:55.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.789+0000 7f659306d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c3462970a0 0x55c34668be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:55.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.789+0000 7f659306d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c34629f280 0x55c3466852b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:55.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.789+0000 7f659306d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3462bff60 0x55c3466857f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:55.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.789+0000 7f659306d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c346683500 con 0x55c3462bff60 2024-07-26T13:39:55.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.789+0000 7f659306d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c346682e30 con 0x55c34629f280 2024-07-26T13:39:55.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.789+0000 7f659306d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c346683bd0 con 0x55c3462970a0 2024-07-26T13:39:55.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.790+0000 7f659306d740 10 client.0 _ll_register_callbacks cb 0x7f6578009d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:39:55.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.790+0000 7f6586d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3462bff60 0x55c3466857f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:55.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.790+0000 7f6586d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3462bff60 0x55c3466857f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38402/0 (socket says 172.21.15.100:38402) 2024-07-26T13:39:55.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.790+0000 7f6586d9d640 1 -- 172.21.15.100:0/2516162237 learned_addr learned my addr 172.21.15.100:0/2516162237 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:55.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.790+0000 7f658659c640 1 --2- 172.21.15.100:0/2516162237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c3462970a0 0x55c34668be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:55.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.790+0000 7f657dd9b640 1 --2- 172.21.15.100:0/2516162237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c34629f280 0x55c3466852b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:55.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.790+0000 7f6586d9d640 1 -- 172.21.15.100:0/2516162237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c3462970a0 msgr2=0x55c34668be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:55.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.790+0000 7f6586d9d640 1 --2- 172.21.15.100:0/2516162237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c3462970a0 0x55c34668be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:55.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.790+0000 7f6586d9d640 1 -- 172.21.15.100:0/2516162237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c34629f280 msgr2=0x55c3466852b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:55.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.790+0000 7f6586d9d640 1 --2- 172.21.15.100:0/2516162237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c34629f280 0x55c3466852b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:55.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.790+0000 7f6586d9d640 1 -- 172.21.15.100:0/2516162237 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c346685d80 con 0x55c3462bff60 2024-07-26T13:39:55.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.790+0000 7f658659c640 1 --2- 172.21.15.100:0/2516162237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c3462970a0 0x55c34668be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:39:55.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.790+0000 7f6586d9d640 1 --2- 172.21.15.100:0/2516162237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3462bff60 0x55c3466857f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6580011010 tx=0x7f6580004c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f63694a719df0e2 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:55.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.790+0000 7f656dffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:39:55.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.790+0000 7f656dffb640 1 -- 172.21.15.100:0/2516162237 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6580016070 con 0x55c3462bff60 2024-07-26T13:39:55.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.790+0000 7f656dffb640 1 -- 172.21.15.100:0/2516162237 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f6580005240 con 0x55c3462bff60 2024-07-26T13:39:55.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.790+0000 7f659306d740 1 -- 172.21.15.100:0/2516162237 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65700d4310 con 0x55c3462bff60 2024-07-26T13:39:55.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.790+0000 7f656dffb640 1 -- 172.21.15.100:0/2516162237 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6580005480 con 0x55c3462bff60 2024-07-26T13:39:55.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.790+0000 7f659306d740 10 client.15282 fetch_fsmap learned FSMap version 25 2024-07-26T13:39:55.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.790+0000 7f659306d740 1 -- 172.21.15.100:0/2516162237 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c34668b780 con 0x55c3462bff60 2024-07-26T13:39:55.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.791+0000 7f656dffb640 1 -- 172.21.15.100:0/2516162237 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65700d4310 con 0x55c3462bff60 2024-07-26T13:39:55.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.791+0000 7f656dffb640 1 -- 172.21.15.100:0/2516162237 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6580024070 con 0x55c3462bff60 2024-07-26T13:39:55.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.791+0000 7f659306d740 10 client.15282 fetch_fsmap finished waiting for FSMap version 25 2024-07-26T13:39:55.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.791+0000 7f659306d740 10 client.15282 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-26T13:39:55.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.791+0000 7f659306d740 20 client.15282 populate_metadata read hostname 'smithi100' 2024-07-26T13:39:55.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.791+0000 7f659306d740 1 --2- 172.21.15.100:0/2516162237 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55c346253c90 0x55c3466efd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:55.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.791+0000 7f659306d740 4 client.15282 mds_command: new command op to 24595 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-26T13:39:55.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.791+0000 7f659306d740 1 -- 172.21.15.100:0/2516162237 --> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] -- command(tid 0: {"prefix": "status"}) -- 0x55c3465d9330 con 0x55c346253c90 2024-07-26T13:39:55.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.791+0000 7f658659c640 1 --2- 172.21.15.100:0/2516162237 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55c346253c90 0x55c3466efd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:55.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.792+0000 7f658659c640 1 --2- 172.21.15.100:0/2516162237 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55c346253c90 0x55c3466efd50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=23205347b1f6507f server_cookie=6981c21165d2c207 in_seq=0 out_seq=0 2024-07-26T13:39:55.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.792+0000 7f656dffb640 10 client.15282 ms_handle_connect on v2:172.21.15.167:6836/3828634468 2024-07-26T13:39:55.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.792+0000 7f656dffb640 1 -- 172.21.15.100:0/2516162237 <== mds.2 v2:172.21.15.167:6836/3828634468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c3465d9330 con 0x55c346253c90 2024-07-26T13:39:55.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.792+0000 7f656dffb640 10 client.15282 handle_command_reply: tid=0 2024-07-26T13:39:55.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.792+0000 7f659306d740 1 client.15282 shutdown 2024-07-26T13:39:55.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.792+0000 7f659306d740 1 -- 172.21.15.100:0/2516162237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3462bff60 msgr2=0x55c3466857f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:55.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.792+0000 7f659306d740 1 --2- 172.21.15.100:0/2516162237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3462bff60 0x55c3466857f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6580011010 tx=0x7f6580004c30 comp rx=0 tx=0).stop 2024-07-26T13:39:55.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.792+0000 7f659306d740 1 -- 172.21.15.100:0/2516162237 shutdown_connections 2024-07-26T13:39:55.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.792+0000 7f659306d740 1 --2- 172.21.15.100:0/2516162237 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55c346253c90 0x55c3466efd50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:55.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.792+0000 7f659306d740 1 --2- 172.21.15.100:0/2516162237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c3462bff60 0x55c3466857f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:55.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.792+0000 7f659306d740 1 --2- 172.21.15.100:0/2516162237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c34629f280 0x55c3466852b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:55.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.792+0000 7f659306d740 1 --2- 172.21.15.100:0/2516162237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c3462970a0 0x55c34668be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:55.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.792+0000 7f656dffb640 0 client.15282 ms_handle_reset on v2:172.21.15.167:6836/3828634468 2024-07-26T13:39:55.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.792+0000 7f659306d740 1 -- 172.21.15.100:0/2516162237 >> 172.21.15.100:0/2516162237 conn(0x55c3462b4350 msgr2=0x55c346670f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:55.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.792+0000 7f659306d740 1 -- 172.21.15.100:0/2516162237 shutdown_connections 2024-07-26T13:39:55.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.792+0000 7f659306d740 1 -- 172.21.15.100:0/2516162237 wait complete. 2024-07-26T13:39:55.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.792+0000 7f659306d740 20 client.15282 trim_cache size 0 max 16384 2024-07-26T13:39:55.803 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:39:55.803 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "bf83e2e6-4b52-11ef-bcb7-c7b262605968", 2024-07-26T13:39:55.804 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 2, 2024-07-26T13:39:55.804 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 24595, 2024-07-26T13:39:55.804 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-07-26T13:39:55.804 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-07-26T13:39:55.804 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-07-26T13:39:55.804 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 161.260358938, 2024-07-26T13:39:55.804 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 25, 2024-07-26T13:39:55.804 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 81, 2024-07-26T13:39:55.804 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 81, 2024-07-26T13:39:55.804 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 206.48431149199999 2024-07-26T13:39:55.804 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:39:55.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.793+0000 7f658cae9640 1 -- 172.21.15.100:0/3387455343 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f65740b3c20 msgr2=0x7f65740b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:55.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.793+0000 7f658cae9640 1 --2- 172.21.15.100:0/3387455343 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f65740b3c20 0x7f65740b6130 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x55c346664020 tx=0x7f6578009330 comp rx=0 tx=0).stop 2024-07-26T13:39:55.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.793+0000 7f658cae9640 1 -- 172.21.15.100:0/3387455343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6588108ed0 msgr2=0x7f658819a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:55.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.793+0000 7f658cae9640 1 --2- 172.21.15.100:0/3387455343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6588108ed0 0x7f658819a4e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f657002a930 tx=0x7f6570004270 comp rx=0 tx=0).stop 2024-07-26T13:39:55.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.793+0000 7f658cae9640 1 -- 172.21.15.100:0/3387455343 shutdown_connections 2024-07-26T13:39:55.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.793+0000 7f658cae9640 1 --2- 172.21.15.100:0/3387455343 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f65740b3c20 0x7f65740b6130 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:55.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.793+0000 7f658cae9640 1 --2- 172.21.15.100:0/3387455343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6588109820 0x7f658819f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:55.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.793+0000 7f658cae9640 1 --2- 172.21.15.100:0/3387455343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6588108ed0 0x7f658819a4e0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:55.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.793+0000 7f658cae9640 1 --2- 172.21.15.100:0/3387455343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6588103590 0x7f6588199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:55.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.793+0000 7f658cae9640 1 -- 172.21.15.100:0/3387455343 >> 172.21.15.100:0/3387455343 conn(0x7f658807b280 msgr2=0x7f6588107010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:55.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.793+0000 7f658cae9640 1 -- 172.21.15.100:0/3387455343 shutdown_connections 2024-07-26T13:39:55.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:55.793+0000 7f658cae9640 1 -- 172.21.15.100:0/3387455343 wait complete. 2024-07-26T13:39:56.552 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.b status 2024-07-26T13:39:56.806 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:39:57.630 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:57 smithi100 ceph-mon[32599]: pgmap v359: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T13:39:57.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:57 smithi167 ceph-mon[37140]: pgmap v359: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T13:39:57.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:57 smithi138 ceph-mon[36954]: pgmap v359: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T13:39:58.618 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:39:58.618 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:39:58.618 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:39:58.618 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:39:58.618 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:39:58.618 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:39:58.618 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:39:58.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.588+0000 7f79257c8640 1 -- 172.21.15.100:0/262587924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7920108790 msgr2=0x7f792010efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:58.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.588+0000 7f79257c8640 1 --2- 172.21.15.100:0/262587924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7920108790 0x7f792010efc0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7918009ca0 tx=0x7f791802a8c0 comp rx=0 tx=0).stop 2024-07-26T13:39:58.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.588+0000 7f79257c8640 1 -- 172.21.15.100:0/262587924 shutdown_connections 2024-07-26T13:39:58.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.588+0000 7f79257c8640 1 --2- 172.21.15.100:0/262587924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7920108790 0x7f792010efc0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:58.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.588+0000 7f79257c8640 1 --2- 172.21.15.100:0/262587924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7920107e40 0x7f7920108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:58.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.588+0000 7f79257c8640 1 --2- 172.21.15.100:0/262587924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7920102550 0x7f7920102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:58.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.588+0000 7f79257c8640 1 -- 172.21.15.100:0/262587924 >> 172.21.15.100:0/262587924 conn(0x7f7920079130 msgr2=0x7f7920079540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:58.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.588+0000 7f79257c8640 1 -- 172.21.15.100:0/262587924 shutdown_connections 2024-07-26T13:39:58.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.588+0000 7f79257c8640 1 -- 172.21.15.100:0/262587924 wait complete. 2024-07-26T13:39:58.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.589+0000 7f79257c8640 1 Processor -- start 2024-07-26T13:39:58.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.589+0000 7f79257c8640 1 -- start start 2024-07-26T13:39:58.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.589+0000 7f79257c8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7920102550 0x7f792019ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:58.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.589+0000 7f79257c8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7920107e40 0x7f792019b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:58.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.590+0000 7f79257c8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7920108790 0x7f792019b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:58.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.590+0000 7f79257c8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7920112440 con 0x7f7920108790 2024-07-26T13:39:58.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.590+0000 7f79257c8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f79201122c0 con 0x7f7920102550 2024-07-26T13:39:58.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.590+0000 7f79257c8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f79201125c0 con 0x7f7920107e40 2024-07-26T13:39:58.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.590+0000 7f791f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7920108790 0x7f792019b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:58.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.590+0000 7f791f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7920108790 0x7f792019b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38426/0 (socket says 172.21.15.100:38426) 2024-07-26T13:39:58.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.590+0000 7f791f7fe640 1 -- 172.21.15.100:0/349910089 learned_addr learned my addr 172.21.15.100:0/349910089 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:58.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.590+0000 7f791effd640 1 --2- 172.21.15.100:0/349910089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7920102550 0x7f792019ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:58.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.590+0000 7f791e7fc640 1 --2- 172.21.15.100:0/349910089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7920107e40 0x7f792019b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:58.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.590+0000 7f791f7fe640 1 -- 172.21.15.100:0/349910089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7920107e40 msgr2=0x7f792019b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:58.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.590+0000 7f791f7fe640 1 --2- 172.21.15.100:0/349910089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7920107e40 0x7f792019b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:58.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.590+0000 7f791f7fe640 1 -- 172.21.15.100:0/349910089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7920102550 msgr2=0x7f792019ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:58.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.590+0000 7f791f7fe640 1 --2- 172.21.15.100:0/349910089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7920102550 0x7f792019ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:58.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.590+0000 7f791f7fe640 1 -- 172.21.15.100:0/349910089 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7920198b00 con 0x7f7920108790 2024-07-26T13:39:58.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.590+0000 7f791f7fe640 1 --2- 172.21.15.100:0/349910089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7920108790 0x7f792019b710 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7918005cc0 tx=0x7f791802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:58.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.590+0000 7f7907fff640 1 -- 172.21.15.100:0/349910089 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7918003bf0 con 0x7f7920108790 2024-07-26T13:39:58.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.590+0000 7f7907fff640 1 -- 172.21.15.100:0/349910089 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7918003d90 con 0x7f7920108790 2024-07-26T13:39:58.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.590+0000 7f7907fff640 1 -- 172.21.15.100:0/349910089 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791802fbc0 con 0x7f7920108790 2024-07-26T13:39:58.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.590+0000 7f79257c8640 1 -- 172.21.15.100:0/349910089 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7920198d90 con 0x7f7920108790 2024-07-26T13:39:58.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.590+0000 7f79257c8640 1 -- 172.21.15.100:0/349910089 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7920199280 con 0x7f7920108790 2024-07-26T13:39:58.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.593+0000 7f792bd4c740 1 Processor -- start 2024-07-26T13:39:58.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.593+0000 7f7907fff640 1 -- 172.21.15.100:0/349910089 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f791802fd60 con 0x7f7920108790 2024-07-26T13:39:58.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.593+0000 7f792bd4c740 1 -- start start 2024-07-26T13:39:58.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.593+0000 7f792bd4c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f888e2af60 0x55f8891d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:58.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.593+0000 7f792bd4c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f888e020a0 0x55f8891d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:58.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.593+0000 7f792bd4c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f888e0a280 0x55f8891d0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:58.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.593+0000 7f792bd4c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55f889146920 con 0x55f888e2af60 2024-07-26T13:39:58.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.593+0000 7f792bd4c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f889135e20 con 0x55f888e020a0 2024-07-26T13:39:58.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.593+0000 7f792bd4c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f889139e60 con 0x55f888e0a280 2024-07-26T13:39:58.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.594+0000 7f791e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f888e2af60 0x55f8891d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:58.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.594+0000 7f791effd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f888e020a0 0x55f8891d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:58.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.594+0000 7f791e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f888e2af60 0x55f8891d0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38438/0 (socket says 172.21.15.100:38438) 2024-07-26T13:39:58.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.594+0000 7f791e7fc640 1 -- 172.21.15.100:0/2434388945 learned_addr learned my addr 172.21.15.100:0/2434388945 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:58.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.594+0000 7f791e7fc640 1 --2- 172.21.15.100:0/2434388945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f888e0a280 0x55f8891d0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:58.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.594+0000 7f791e7fc640 1 -- 172.21.15.100:0/2434388945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f888e0a280 msgr2=0x55f8891d0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:58.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.594+0000 7f791e7fc640 1 --2- 172.21.15.100:0/2434388945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f888e0a280 0x55f8891d0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:58.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.594+0000 7f791e7fc640 1 -- 172.21.15.100:0/2434388945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f888e020a0 msgr2=0x55f8891d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:58.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.594+0000 7f791e7fc640 1 --2- 172.21.15.100:0/2434388945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f888e020a0 0x55f8891d08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:58.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.594+0000 7f791e7fc640 1 -- 172.21.15.100:0/2434388945 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8890dca40 con 0x55f888e2af60 2024-07-26T13:39:58.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.594+0000 7f791effd640 1 --2- 172.21.15.100:0/2434388945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f888e020a0 0x55f8891d08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:39:58.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.594+0000 7f791e7fc640 1 --2- 172.21.15.100:0/2434388945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f888e2af60 0x55f8891d0380 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f792006b410 tx=0x7f791400b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e408fbce838fcf8 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:58.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.594+0000 7f7907fff640 1 --2- 172.21.15.100:0/349910089 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f79080b3c50 0x7f79080b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:58.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.594+0000 7f79057fa640 1 -- 172.21.15.100:0/2434388945 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7914004020 con 0x55f888e2af60 2024-07-26T13:39:58.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.594+0000 7f791effd640 1 --2- 172.21.15.100:0/349910089 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f79080b3c50 0x7f79080b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:58.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.594+0000 7f79057fa640 1 -- 172.21.15.100:0/2434388945 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7914004c50 con 0x55f888e2af60 2024-07-26T13:39:58.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.594+0000 7f79057fa640 1 -- 172.21.15.100:0/2434388945 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7914005310 con 0x55f888e2af60 2024-07-26T13:39:58.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.594+0000 7f7907fff640 1 -- 172.21.15.100:0/349910089 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f79181053d0 con 0x7f7920108790 2024-07-26T13:39:58.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.595+0000 7f792bd4c740 1 -- 172.21.15.100:0/2434388945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f888e2af60 msgr2=0x55f8891d0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:58.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.595+0000 7f791effd640 1 --2- 172.21.15.100:0/349910089 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f79080b3c50 0x7f79080b6160 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x55f8891cf000 tx=0x7f791000b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:58.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.595+0000 7f792bd4c740 1 --2- 172.21.15.100:0/2434388945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f888e2af60 0x55f8891d0380 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f792006b410 tx=0x7f791400b650 comp rx=0 tx=0).stop 2024-07-26T13:39:58.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.595+0000 7f792bd4c740 1 -- 172.21.15.100:0/2434388945 shutdown_connections 2024-07-26T13:39:58.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.595+0000 7f792bd4c740 1 --2- 172.21.15.100:0/2434388945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f888e0a280 0x55f8891d0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:58.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.595+0000 7f792bd4c740 1 --2- 172.21.15.100:0/2434388945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f888e020a0 0x55f8891d08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:58.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.595+0000 7f792bd4c740 1 --2- 172.21.15.100:0/2434388945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f888e2af60 0x55f8891d0380 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:58.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.595+0000 7f792bd4c740 1 -- 172.21.15.100:0/2434388945 >> 172.21.15.100:0/2434388945 conn(0x55f888e1f350 msgr2=0x55f8891ce270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:58.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.595+0000 7f792bd4c740 1 -- 172.21.15.100:0/2434388945 shutdown_connections 2024-07-26T13:39:58.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.595+0000 7f792bd4c740 1 -- 172.21.15.100:0/2434388945 wait complete. 2024-07-26T13:39:58.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.596+0000 7f792bd4c740 1 -- start start 2024-07-26T13:39:58.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.596+0000 7f792bd4c740 1 Processor -- start 2024-07-26T13:39:58.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.596+0000 7f792bd4c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f888e020a0 0x55f8891dbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:58.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.596+0000 7f792bd4c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f888e0a280 0x55f8891dc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:58.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.596+0000 7f792bd4c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f888e2af60 0x55f8891e7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:58.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.596+0000 7f792bd4c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55f888f995f0 con 0x55f888e0a280 2024-07-26T13:39:58.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.596+0000 7f792bd4c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f8890d8890 con 0x55f888e020a0 2024-07-26T13:39:58.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.596+0000 7f792bd4c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f8891deb60 con 0x55f888e2af60 2024-07-26T13:39:58.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.596+0000 7f792bd4c740 10 client.0 _ll_register_callbacks cb 0x55f8890e27d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:39:58.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.596+0000 7f791e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f888e0a280 0x55f8891dc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:58.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.596+0000 7f791e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f888e0a280 0x55f8891dc440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38452/0 (socket says 172.21.15.100:38452) 2024-07-26T13:39:58.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.596+0000 7f791effd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f888e020a0 0x55f8891dbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:58.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.596+0000 7f791e7fc640 1 -- 172.21.15.100:0/893510933 learned_addr learned my addr 172.21.15.100:0/893510933 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:58.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.596+0000 7f791f7fe640 1 --2- 172.21.15.100:0/893510933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f888e2af60 0x55f8891e7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:58.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.597+0000 7f791e7fc640 1 -- 172.21.15.100:0/893510933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f888e2af60 msgr2=0x55f8891e7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:58.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.597+0000 7f791e7fc640 1 --2- 172.21.15.100:0/893510933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f888e2af60 0x55f8891e7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:58.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.597+0000 7f791e7fc640 1 -- 172.21.15.100:0/893510933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f888e020a0 msgr2=0x55f8891dbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:58.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.597+0000 7f791e7fc640 1 --2- 172.21.15.100:0/893510933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f888e020a0 0x55f8891dbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:58.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.597+0000 7f791e7fc640 1 -- 172.21.15.100:0/893510933 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f888e96540 con 0x55f888e0a280 2024-07-26T13:39:58.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.597+0000 7f791effd640 1 --2- 172.21.15.100:0/893510933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f888e020a0 0x55f8891dbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:39:58.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.597+0000 7f791e7fc640 1 --2- 172.21.15.100:0/893510933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f888e0a280 0x55f8891dc440 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7914005d00 tx=0x7f791400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df42842b92e02e38 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:58.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.597+0000 7f7902ff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:39:58.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.597+0000 7f7902ff5640 1 -- 172.21.15.100:0/893510933 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791402d060 con 0x55f888e0a280 2024-07-26T13:39:58.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.597+0000 7f7902ff5640 1 -- 172.21.15.100:0/893510933 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f79140046c0 con 0x55f888e0a280 2024-07-26T13:39:58.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.597+0000 7f792bd4c740 1 -- 172.21.15.100:0/893510933 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7914004c50 con 0x55f888e0a280 2024-07-26T13:39:58.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.597+0000 7f7902ff5640 1 -- 172.21.15.100:0/893510933 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7914004e80 con 0x55f888e0a280 2024-07-26T13:39:58.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.597+0000 7f792bd4c740 10 client.15294 fetch_fsmap learned FSMap version 25 2024-07-26T13:39:58.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.597+0000 7f792bd4c740 1 -- 172.21.15.100:0/893510933 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f8891eab30 con 0x55f888e0a280 2024-07-26T13:39:58.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.597+0000 7f7902ff5640 1 -- 172.21.15.100:0/893510933 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7914004c50 con 0x55f888e0a280 2024-07-26T13:39:58.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.597+0000 7f7902ff5640 1 -- 172.21.15.100:0/893510933 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f79140035f0 con 0x55f888e0a280 2024-07-26T13:39:58.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.597+0000 7f792bd4c740 10 client.15294 fetch_fsmap finished waiting for FSMap version 25 2024-07-26T13:39:58.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.598+0000 7f792bd4c740 10 client.15294 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-26T13:39:58.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.598+0000 7f792bd4c740 20 client.15294 populate_metadata read hostname 'smithi100' 2024-07-26T13:39:58.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.598+0000 7f792bd4c740 1 --2- 172.21.15.100:0/893510933 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55f888de98e0 0x55f8891ee330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:58.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.598+0000 7f792bd4c740 4 client.15294 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:39:58.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.598+0000 7f792bd4c740 1 -- 172.21.15.100:0/893510933 --> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f889148870 con 0x55f888de98e0 2024-07-26T13:39:58.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.598+0000 7f791effd640 1 --2- 172.21.15.100:0/893510933 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55f888de98e0 0x55f8891ee330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:58.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.598+0000 7f7902ff5640 10 client.15294 ms_handle_connect on v2:172.21.15.138:6838/1580925471 2024-07-26T13:39:58.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.598+0000 7f791effd640 1 --2- 172.21.15.100:0/893510933 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55f888de98e0 0x55f8891ee330 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a14a339897dca637 server_cookie=29749036a17e4113 in_seq=0 out_seq=0 2024-07-26T13:39:58.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.599+0000 7f7902ff5640 1 -- 172.21.15.100:0/893510933 <== mds.1 v2:172.21.15.138:6838/1580925471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f889148870 con 0x55f888de98e0 2024-07-26T13:39:58.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.599+0000 7f7902ff5640 10 client.15294 handle_command_reply: tid=0 2024-07-26T13:39:58.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.599+0000 7f792bd4c740 1 client.15294 shutdown 2024-07-26T13:39:58.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.599+0000 7f792bd4c740 1 -- 172.21.15.100:0/893510933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f888e0a280 msgr2=0x55f8891dc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:58.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.599+0000 7f792bd4c740 1 --2- 172.21.15.100:0/893510933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f888e0a280 0x55f8891dc440 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7914005d00 tx=0x7f791400bd30 comp rx=0 tx=0).stop 2024-07-26T13:39:58.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.599+0000 7f792bd4c740 1 -- 172.21.15.100:0/893510933 shutdown_connections 2024-07-26T13:39:58.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.599+0000 7f792bd4c740 1 --2- 172.21.15.100:0/893510933 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55f888de98e0 0x55f8891ee330 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:58.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.599+0000 7f792bd4c740 1 --2- 172.21.15.100:0/893510933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f888e2af60 0x55f8891e7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:58.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.599+0000 7f792bd4c740 1 --2- 172.21.15.100:0/893510933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f888e0a280 0x55f8891dc440 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:58.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.599+0000 7f7902ff5640 0 client.15294 ms_handle_reset on v2:172.21.15.138:6838/1580925471 2024-07-26T13:39:58.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.599+0000 7f792bd4c740 1 --2- 172.21.15.100:0/893510933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f888e020a0 0x55f8891dbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:58.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.599+0000 7f792bd4c740 1 -- 172.21.15.100:0/893510933 >> 172.21.15.100:0/893510933 conn(0x55f888e1f350 msgr2=0x55f8891d3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:58.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.599+0000 7f792bd4c740 1 -- 172.21.15.100:0/893510933 shutdown_connections 2024-07-26T13:39:58.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.599+0000 7f792bd4c740 1 -- 172.21.15.100:0/893510933 wait complete. 2024-07-26T13:39:58.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.599+0000 7f792bd4c740 20 client.15294 trim_cache size 0 max 16384 2024-07-26T13:39:58.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.617+0000 7f792bd4c740 1 Processor -- start 2024-07-26T13:39:58.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.617+0000 7f792bd4c740 1 -- start start 2024-07-26T13:39:58.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.617+0000 7f792bd4c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f888de98e0 0x55f8891e7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:58.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.617+0000 7f792bd4c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f888e020a0 0x55f8891e8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:58.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.617+0000 7f792bd4c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f888e0a280 0x55f8891d5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:58.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.617+0000 7f792bd4c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55f8891ec670 con 0x55f888de98e0 2024-07-26T13:39:58.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.617+0000 7f792bd4c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f8891ecd40 con 0x55f888e020a0 2024-07-26T13:39:58.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.617+0000 7f791effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f888de98e0 0x55f8891e7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:58.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.617+0000 7f792bd4c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f8891ed410 con 0x55f888e0a280 2024-07-26T13:39:58.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.617+0000 7f791e7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f888e020a0 0x55f8891e8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:58.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.617+0000 7f791effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f888de98e0 0x55f8891e7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38468/0 (socket says 172.21.15.100:38468) 2024-07-26T13:39:58.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.617+0000 7f791effd640 1 -- 172.21.15.100:0/1842963006 learned_addr learned my addr 172.21.15.100:0/1842963006 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:58.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.617+0000 7f791f7fe640 1 --2- 172.21.15.100:0/1842963006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f888e0a280 0x55f8891d5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:58.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.618+0000 7f791effd640 1 -- 172.21.15.100:0/1842963006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f888e0a280 msgr2=0x55f8891d5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:58.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.618+0000 7f791effd640 1 --2- 172.21.15.100:0/1842963006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f888e0a280 0x55f8891d5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:58.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.618+0000 7f791effd640 1 -- 172.21.15.100:0/1842963006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f888e020a0 msgr2=0x55f8891e8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:58.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.618+0000 7f791effd640 1 --2- 172.21.15.100:0/1842963006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f888e020a0 0x55f8891e8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:58.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.618+0000 7f791effd640 1 -- 172.21.15.100:0/1842963006 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8891d7870 con 0x55f888de98e0 2024-07-26T13:39:58.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.618+0000 7f791e7fc640 1 --2- 172.21.15.100:0/1842963006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f888e020a0 0x55f8891e8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:39:58.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.618+0000 7f791effd640 1 --2- 172.21.15.100:0/1842963006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f888de98e0 0x55f8891e7e20 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f792006c0f0 tx=0x7f7910019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a3a08a1e746141 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:58.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.618+0000 7f7904ff9640 1 -- 172.21.15.100:0/1842963006 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791001ebb0 con 0x55f888de98e0 2024-07-26T13:39:58.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.618+0000 7f7904ff9640 1 -- 172.21.15.100:0/1842963006 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f791001ed50 con 0x55f888de98e0 2024-07-26T13:39:58.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.618+0000 7f7904ff9640 1 -- 172.21.15.100:0/1842963006 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791002a420 con 0x55f888de98e0 2024-07-26T13:39:58.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.619+0000 7f792bd4c740 1 -- 172.21.15.100:0/1842963006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f888de98e0 msgr2=0x55f8891e7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:58.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.619+0000 7f792bd4c740 1 --2- 172.21.15.100:0/1842963006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f888de98e0 0x55f8891e7e20 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f792006c0f0 tx=0x7f7910019000 comp rx=0 tx=0).stop 2024-07-26T13:39:58.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.619+0000 7f792bd4c740 1 -- 172.21.15.100:0/1842963006 shutdown_connections 2024-07-26T13:39:58.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.619+0000 7f792bd4c740 1 --2- 172.21.15.100:0/1842963006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f888e0a280 0x55f8891d5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:58.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.619+0000 7f792bd4c740 1 --2- 172.21.15.100:0/1842963006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f888e020a0 0x55f8891e8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:58.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.619+0000 7f792bd4c740 1 --2- 172.21.15.100:0/1842963006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f888de98e0 0x55f8891e7e20 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:58.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.619+0000 7f792bd4c740 1 -- 172.21.15.100:0/1842963006 >> 172.21.15.100:0/1842963006 conn(0x55f888e1f350 msgr2=0x55f8891da580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:58.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.619+0000 7f792bd4c740 1 -- 172.21.15.100:0/1842963006 shutdown_connections 2024-07-26T13:39:58.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.619+0000 7f792bd4c740 1 -- 172.21.15.100:0/1842963006 wait complete. 2024-07-26T13:39:58.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.619+0000 7f792bd4c740 1 -- start start 2024-07-26T13:39:58.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.620+0000 7f792bd4c740 1 Processor -- start 2024-07-26T13:39:58.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.620+0000 7f792bd4c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f888de98e0 0x55f8891efa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:58.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.620+0000 7f792bd4c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f888e020a0 0x55f8891effc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:58.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.620+0000 7f792bd4c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f888e0a280 0x55f8891f4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:58.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.620+0000 7f792bd4c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55f8891ee1b0 con 0x55f888de98e0 2024-07-26T13:39:58.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.620+0000 7f792bd4c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f8891edae0 con 0x55f888e020a0 2024-07-26T13:39:58.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.620+0000 7f792bd4c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f8891ee880 con 0x55f888e0a280 2024-07-26T13:39:58.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.620+0000 7f792bd4c740 10 client.0 _ll_register_callbacks cb 0x7f79140042d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:39:58.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.620+0000 7f791effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f888de98e0 0x55f8891efa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:58.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.620+0000 7f791effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f888de98e0 0x55f8891efa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38474/0 (socket says 172.21.15.100:38474) 2024-07-26T13:39:58.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.620+0000 7f791effd640 1 -- 172.21.15.100:0/3597767390 learned_addr learned my addr 172.21.15.100:0/3597767390 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:39:58.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.620+0000 7f791e7fc640 1 --2- 172.21.15.100:0/3597767390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f888e020a0 0x55f8891effc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:58.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.620+0000 7f791f7fe640 1 --2- 172.21.15.100:0/3597767390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f888e0a280 0x55f8891f4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:58.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.620+0000 7f791effd640 1 -- 172.21.15.100:0/3597767390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f888e0a280 msgr2=0x55f8891f4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:58.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.620+0000 7f791effd640 1 --2- 172.21.15.100:0/3597767390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f888e0a280 0x55f8891f4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:58.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.620+0000 7f791effd640 1 -- 172.21.15.100:0/3597767390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f888e020a0 msgr2=0x55f8891effc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:39:58.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.620+0000 7f791effd640 1 --2- 172.21.15.100:0/3597767390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f888e020a0 0x55f8891effc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:58.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.620+0000 7f791effd640 1 -- 172.21.15.100:0/3597767390 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8891f54b0 con 0x55f888de98e0 2024-07-26T13:39:58.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.621+0000 7f791effd640 1 --2- 172.21.15.100:0/3597767390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f888de98e0 0x55f8891efa00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7910018040 tx=0x7f7910004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=becdd69a7c54ae1 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:39:58.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.621+0000 7f7902ff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:39:58.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.621+0000 7f7902ff5640 1 -- 172.21.15.100:0/3597767390 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7910038070 con 0x55f888de98e0 2024-07-26T13:39:58.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.621+0000 7f7902ff5640 1 -- 172.21.15.100:0/3597767390 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f791002a420 con 0x55f888de98e0 2024-07-26T13:39:58.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.621+0000 7f792bd4c740 1 -- 172.21.15.100:0/3597767390 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f791001ed50 con 0x55f888de98e0 2024-07-26T13:39:58.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.621+0000 7f7902ff5640 1 -- 172.21.15.100:0/3597767390 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f791002a660 con 0x55f888de98e0 2024-07-26T13:39:58.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.621+0000 7f792bd4c740 10 client.15300 fetch_fsmap learned FSMap version 25 2024-07-26T13:39:58.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.621+0000 7f792bd4c740 1 -- 172.21.15.100:0/3597767390 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f8891f68a0 con 0x55f888de98e0 2024-07-26T13:39:58.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.622+0000 7f7902ff5640 1 -- 172.21.15.100:0/3597767390 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f791002acb0 con 0x55f888de98e0 2024-07-26T13:39:58.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.622+0000 7f7902ff5640 1 -- 172.21.15.100:0/3597767390 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7910034440 con 0x55f888de98e0 2024-07-26T13:39:58.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.622+0000 7f792bd4c740 10 client.15300 fetch_fsmap finished waiting for FSMap version 25 2024-07-26T13:39:58.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.622+0000 7f792bd4c740 10 client.15300 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-26T13:39:58.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.622+0000 7f792bd4c740 20 client.15300 populate_metadata read hostname 'smithi100' 2024-07-26T13:39:58.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.622+0000 7f792bd4c740 1 --2- 172.21.15.100:0/3597767390 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55f888e2af60 0x55f88925a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:39:58.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.622+0000 7f792bd4c740 4 client.15300 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-26T13:39:58.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.622+0000 7f792bd4c740 1 -- 172.21.15.100:0/3597767390 --> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] -- command(tid 0: {"prefix": "status"}) -- 0x7f791001ed50 con 0x55f888e2af60 2024-07-26T13:39:58.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.622+0000 7f791e7fc640 1 --2- 172.21.15.100:0/3597767390 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55f888e2af60 0x55f88925a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:39:58.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.622+0000 7f7902ff5640 10 client.15300 ms_handle_connect on v2:172.21.15.138:6838/1580925471 2024-07-26T13:39:58.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.622+0000 7f791e7fc640 1 --2- 172.21.15.100:0/3597767390 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55f888e2af60 0x55f88925a110 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4efe28f0e46c804a server_cookie=8efe6f5608878f14 in_seq=0 out_seq=0 2024-07-26T13:39:58.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.623+0000 7f7902ff5640 1 -- 172.21.15.100:0/3597767390 <== mds.1 v2:172.21.15.138:6838/1580925471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x7f791001ed50 con 0x55f888e2af60 2024-07-26T13:39:58.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.623+0000 7f7902ff5640 10 client.15300 handle_command_reply: tid=0 2024-07-26T13:39:58.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.623+0000 7f792bd4c740 1 client.15300 shutdown 2024-07-26T13:39:58.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.623+0000 7f792bd4c740 1 -- 172.21.15.100:0/3597767390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f888de98e0 msgr2=0x55f8891efa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:58.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.623+0000 7f792bd4c740 1 --2- 172.21.15.100:0/3597767390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f888de98e0 0x55f8891efa00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7910018040 tx=0x7f7910004840 comp rx=0 tx=0).stop 2024-07-26T13:39:58.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.623+0000 7f792bd4c740 1 -- 172.21.15.100:0/3597767390 shutdown_connections 2024-07-26T13:39:58.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.623+0000 7f792bd4c740 1 --2- 172.21.15.100:0/3597767390 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55f888e2af60 0x55f88925a110 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:58.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.623+0000 7f792bd4c740 1 --2- 172.21.15.100:0/3597767390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f888e0a280 0x55f8891f4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:58.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.623+0000 7f792bd4c740 1 --2- 172.21.15.100:0/3597767390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f888e020a0 0x55f8891effc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:58.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.623+0000 7f7902ff5640 0 client.15300 ms_handle_reset on v2:172.21.15.138:6838/1580925471 2024-07-26T13:39:58.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.623+0000 7f792bd4c740 1 --2- 172.21.15.100:0/3597767390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f888de98e0 0x55f8891efa00 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:58.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.623+0000 7f792bd4c740 1 -- 172.21.15.100:0/3597767390 >> 172.21.15.100:0/3597767390 conn(0x55f888e1f350 msgr2=0x55f8891dbee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:58.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.623+0000 7f792bd4c740 1 -- 172.21.15.100:0/3597767390 shutdown_connections 2024-07-26T13:39:58.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.623+0000 7f792bd4c740 1 -- 172.21.15.100:0/3597767390 wait complete. 2024-07-26T13:39:58.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.623+0000 7f792bd4c740 20 client.15300 trim_cache size 0 max 16384 2024-07-26T13:39:58.641 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:39:58.641 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "bf83e2e6-4b52-11ef-bcb7-c7b262605968", 2024-07-26T13:39:58.641 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 1, 2024-07-26T13:39:58.641 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 24605, 2024-07-26T13:39:58.641 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-07-26T13:39:58.642 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-07-26T13:39:58.642 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-07-26T13:39:58.642 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 165.100068236, 2024-07-26T13:39:58.642 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 25, 2024-07-26T13:39:58.642 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 81, 2024-07-26T13:39:58.642 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 81, 2024-07-26T13:39:58.642 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 194.42235875399999 2024-07-26T13:39:58.642 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:39:58.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.624+0000 7f79257c8640 1 -- 172.21.15.100:0/349910089 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f79080b3c50 msgr2=0x7f79080b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:58.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.624+0000 7f79257c8640 1 --2- 172.21.15.100:0/349910089 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f79080b3c50 0x7f79080b6160 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x55f8891cf000 tx=0x7f791000b390 comp rx=0 tx=0).stop 2024-07-26T13:39:58.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.624+0000 7f79257c8640 1 -- 172.21.15.100:0/349910089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7920108790 msgr2=0x7f792019b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:39:58.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.624+0000 7f79257c8640 1 --2- 172.21.15.100:0/349910089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7920108790 0x7f792019b710 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7918005cc0 tx=0x7f791802f380 comp rx=0 tx=0).stop 2024-07-26T13:39:58.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.624+0000 7f79257c8640 1 -- 172.21.15.100:0/349910089 shutdown_connections 2024-07-26T13:39:58.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.624+0000 7f79257c8640 1 --2- 172.21.15.100:0/349910089 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f79080b3c50 0x7f79080b6160 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:58.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.624+0000 7f79257c8640 1 --2- 172.21.15.100:0/349910089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7920108790 0x7f792019b710 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:58.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.624+0000 7f79257c8640 1 --2- 172.21.15.100:0/349910089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7920107e40 0x7f792019b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:58.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.624+0000 7f79257c8640 1 --2- 172.21.15.100:0/349910089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7920102550 0x7f792019ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:39:58.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.624+0000 7f79257c8640 1 -- 172.21.15.100:0/349910089 >> 172.21.15.100:0/349910089 conn(0x7f7920079130 msgr2=0x7f7920105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:39:58.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.624+0000 7f79257c8640 1 -- 172.21.15.100:0/349910089 shutdown_connections 2024-07-26T13:39:58.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:39:58.624+0000 7f79257c8640 1 -- 172.21.15.100:0/349910089 wait complete. 2024-07-26T13:39:58.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:39:58.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:39:58.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:39:58.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:39:58.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:39:58.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:39:58.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:39:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:39:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:39:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:39:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:39:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:39:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:39:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:39:59.421 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.i status 2024-07-26T13:39:59.679 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:39:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:59 smithi100 ceph-mon[32599]: pgmap v360: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:39:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:39:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:39:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:39:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:39:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:59 smithi167 ceph-mon[37140]: pgmap v360: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:39:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:39:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:39:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:39:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:39:59.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:59 smithi138 ceph-mon[36954]: pgmap v360: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:39:59.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:39:59.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:39:59.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:39:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:40:00.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:00 smithi167 ceph-mon[37140]: overall HEALTH_OK 2024-07-26T13:40:00.868 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:00 smithi100 ceph-mon[32599]: overall HEALTH_OK 2024-07-26T13:40:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:00 smithi138 ceph-mon[36954]: overall HEALTH_OK 2024-07-26T13:40:01.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.429+0000 7f3767298640 1 -- 172.21.15.100:0/2901601245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3760107e50 msgr2=0x7f3760108260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:01.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.429+0000 7f3767298640 1 --2- 172.21.15.100:0/2901601245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3760107e50 0x7f3760108260 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3754009900 tx=0x7f375402a410 comp rx=0 tx=0).stop 2024-07-26T13:40:01.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.429+0000 7f3767298640 1 -- 172.21.15.100:0/2901601245 shutdown_connections 2024-07-26T13:40:01.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.429+0000 7f3767298640 1 --2- 172.21.15.100:0/2901601245 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37601087a0 0x7f376010efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:01.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.429+0000 7f3767298640 1 --2- 172.21.15.100:0/2901601245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3760107e50 0x7f3760108260 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:01.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.429+0000 7f3767298640 1 --2- 172.21.15.100:0/2901601245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3760102510 0x7f3760102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:01.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.429+0000 7f3767298640 1 -- 172.21.15.100:0/2901601245 >> 172.21.15.100:0/2901601245 conn(0x7f37600791a0 msgr2=0x7f37600795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:01.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.429+0000 7f3767298640 1 -- 172.21.15.100:0/2901601245 shutdown_connections 2024-07-26T13:40:01.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.429+0000 7f3767298640 1 -- 172.21.15.100:0/2901601245 wait complete. 2024-07-26T13:40:01.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.430+0000 7f3767298640 1 Processor -- start 2024-07-26T13:40:01.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.430+0000 7f3767298640 1 -- start start 2024-07-26T13:40:01.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.430+0000 7f3767298640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3760102510 0x7f3760198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:01.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.430+0000 7f3767298640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3760107e50 0x7f3760199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:01.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.430+0000 7f3767298640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37601087a0 0x7f376019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:01.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.430+0000 7f3767298640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3760112430 con 0x7f3760107e50 2024-07-26T13:40:01.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.430+0000 7f3767298640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f37601122b0 con 0x7f37601087a0 2024-07-26T13:40:01.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.430+0000 7f3767298640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f37601125b0 con 0x7f3760102510 2024-07-26T13:40:01.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.430+0000 7f3764833640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3760107e50 0x7f3760199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:01.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.430+0000 7f3764833640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3760107e50 0x7f3760199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38506/0 (socket says 172.21.15.100:38506) 2024-07-26T13:40:01.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.430+0000 7f3764833640 1 -- 172.21.15.100:0/2883895692 learned_addr learned my addr 172.21.15.100:0/2883895692 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:01.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.430+0000 7f3765835640 1 --2- 172.21.15.100:0/2883895692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37601087a0 0x7f376019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:01.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.430+0000 7f3765034640 1 --2- 172.21.15.100:0/2883895692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3760102510 0x7f3760198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:01.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.430+0000 7f3764833640 1 -- 172.21.15.100:0/2883895692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3760102510 msgr2=0x7f3760198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:01.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.430+0000 7f3764833640 1 --2- 172.21.15.100:0/2883895692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3760102510 0x7f3760198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:01.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.430+0000 7f3764833640 1 -- 172.21.15.100:0/2883895692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37601087a0 msgr2=0x7f376019e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:01.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.430+0000 7f3764833640 1 --2- 172.21.15.100:0/2883895692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37601087a0 0x7f376019e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:01.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.430+0000 7f3764833640 1 -- 172.21.15.100:0/2883895692 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376019edd0 con 0x7f3760107e50 2024-07-26T13:40:01.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.431+0000 7f3764833640 1 --2- 172.21.15.100:0/2883895692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3760107e50 0x7f3760199400 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3754006010 tx=0x7f3754004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:01.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.431+0000 7f375e7fc640 1 -- 172.21.15.100:0/2883895692 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3754046070 con 0x7f3760107e50 2024-07-26T13:40:01.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.431+0000 7f375e7fc640 1 -- 172.21.15.100:0/2883895692 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37540044a0 con 0x7f3760107e50 2024-07-26T13:40:01.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.431+0000 7f3767298640 1 -- 172.21.15.100:0/2883895692 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f376019f060 con 0x7f3760107e50 2024-07-26T13:40:01.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.431+0000 7f375e7fc640 1 -- 172.21.15.100:0/2883895692 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3754035050 con 0x7f3760107e50 2024-07-26T13:40:01.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.431+0000 7f3767298640 1 -- 172.21.15.100:0/2883895692 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f376006b590 con 0x7f3760107e50 2024-07-26T13:40:01.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.434+0000 7f376d81c740 1 Processor -- start 2024-07-26T13:40:01.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.434+0000 7f376d81c740 1 -- start start 2024-07-26T13:40:01.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.434+0000 7f375e7fc640 1 -- 172.21.15.100:0/2883895692 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f3754049070 con 0x7f3760107e50 2024-07-26T13:40:01.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.434+0000 7f376d81c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd635bf60 0x560bd67013a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:01.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.434+0000 7f376d81c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd63330a0 0x560bd67018e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:01.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.434+0000 7f376d81c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd633b280 0x560bd6701e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:01.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.434+0000 7f376d81c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x560bd6677940 con 0x560bd635bf60 2024-07-26T13:40:01.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.434+0000 7f376d81c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560bd6666e40 con 0x560bd633b280 2024-07-26T13:40:01.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.434+0000 7f376d81c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560bd666ae80 con 0x560bd63330a0 2024-07-26T13:40:01.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.434+0000 7f3765835640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd635bf60 0x560bd67013a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:01.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.434+0000 7f3765835640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd635bf60 0x560bd67013a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38514/0 (socket says 172.21.15.100:38514) 2024-07-26T13:40:01.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.434+0000 7f3765835640 1 -- 172.21.15.100:0/2956032304 learned_addr learned my addr 172.21.15.100:0/2956032304 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:01.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.434+0000 7f3764833640 1 --2- 172.21.15.100:0/2956032304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd633b280 0x560bd6701e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:01.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.434+0000 7f3765034640 1 --2- 172.21.15.100:0/2956032304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd63330a0 0x560bd67018e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:01.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.434+0000 7f3765835640 1 -- 172.21.15.100:0/2956032304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd63330a0 msgr2=0x560bd67018e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:01.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.434+0000 7f3765835640 1 --2- 172.21.15.100:0/2956032304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd63330a0 0x560bd67018e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:01.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.434+0000 7f3765835640 1 -- 172.21.15.100:0/2956032304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd633b280 msgr2=0x560bd6701e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:01.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.434+0000 7f3765835640 1 --2- 172.21.15.100:0/2956032304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd633b280 0x560bd6701e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:01.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.434+0000 7f3765835640 1 -- 172.21.15.100:0/2956032304 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bd660da60 con 0x560bd635bf60 2024-07-26T13:40:01.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.434+0000 7f3765034640 1 --2- 172.21.15.100:0/2956032304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd63330a0 0x560bd67018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:40:01.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.435+0000 7f375e7fc640 1 --2- 172.21.15.100:0/2883895692 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f37480b3c20 0x7f37480b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:01.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.435+0000 7f3765835640 1 --2- 172.21.15.100:0/2956032304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd635bf60 0x560bd67013a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3758009fd0 tx=0x7f37580118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0b192ac51651bbd server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:01.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.435+0000 7f3765034640 1 --2- 172.21.15.100:0/2883895692 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f37480b3c20 0x7f37480b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:01.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.435+0000 7f375e7fc640 1 -- 172.21.15.100:0/2883895692 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f3754105760 con 0x7f3760107e50 2024-07-26T13:40:01.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.435+0000 7f37477fe640 1 -- 172.21.15.100:0/2956032304 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3758004d20 con 0x560bd635bf60 2024-07-26T13:40:01.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.435+0000 7f37477fe640 1 -- 172.21.15.100:0/2956032304 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3758004050 con 0x560bd635bf60 2024-07-26T13:40:01.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.435+0000 7f37477fe640 1 -- 172.21.15.100:0/2956032304 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3758005280 con 0x560bd635bf60 2024-07-26T13:40:01.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.436+0000 7f376d81c740 1 -- 172.21.15.100:0/2956032304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd635bf60 msgr2=0x560bd67013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:01.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.436+0000 7f3765034640 1 --2- 172.21.15.100:0/2883895692 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f37480b3c20 0x7f37480b6130 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x560bd6700020 tx=0x7f375000b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:01.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.436+0000 7f376d81c740 1 --2- 172.21.15.100:0/2956032304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd635bf60 0x560bd67013a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3758009fd0 tx=0x7f37580118d0 comp rx=0 tx=0).stop 2024-07-26T13:40:01.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.436+0000 7f376d81c740 1 -- 172.21.15.100:0/2956032304 shutdown_connections 2024-07-26T13:40:01.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.436+0000 7f376d81c740 1 --2- 172.21.15.100:0/2956032304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd633b280 0x560bd6701e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:01.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.436+0000 7f376d81c740 1 --2- 172.21.15.100:0/2956032304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd63330a0 0x560bd67018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:01.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.436+0000 7f376d81c740 1 --2- 172.21.15.100:0/2956032304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd635bf60 0x560bd67013a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:01.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.436+0000 7f376d81c740 1 -- 172.21.15.100:0/2956032304 >> 172.21.15.100:0/2956032304 conn(0x560bd6350350 msgr2=0x560bd66ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:01.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.436+0000 7f376d81c740 1 -- 172.21.15.100:0/2956032304 shutdown_connections 2024-07-26T13:40:01.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.436+0000 7f376d81c740 1 -- 172.21.15.100:0/2956032304 wait complete. 2024-07-26T13:40:01.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.436+0000 7f376d81c740 1 -- start start 2024-07-26T13:40:01.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.437+0000 7f376d81c740 1 Processor -- start 2024-07-26T13:40:01.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.437+0000 7f376d81c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd63330a0 0x560bd670d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:01.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.437+0000 7f376d81c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd633b280 0x560bd670d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:01.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.437+0000 7f376d81c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd635bf60 0x560bd6718ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:01.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.437+0000 7f376d81c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x560bd64ca5f0 con 0x560bd633b280 2024-07-26T13:40:01.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.437+0000 7f376d81c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560bd66098b0 con 0x560bd63330a0 2024-07-26T13:40:01.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.437+0000 7f376d81c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560bd670fb80 con 0x560bd635bf60 2024-07-26T13:40:01.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.437+0000 7f376d81c740 10 client.0 _ll_register_callbacks cb 0x560bd66137f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:40:01.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.437+0000 7f3764833640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd633b280 0x560bd670d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:01.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.437+0000 7f3764833640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd633b280 0x560bd670d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38526/0 (socket says 172.21.15.100:38526) 2024-07-26T13:40:01.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.437+0000 7f3764833640 1 -- 172.21.15.100:0/2991791015 learned_addr learned my addr 172.21.15.100:0/2991791015 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:01.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.437+0000 7f3765034640 1 --2- 172.21.15.100:0/2991791015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd63330a0 0x560bd670d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:01.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.437+0000 7f3765835640 1 --2- 172.21.15.100:0/2991791015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd635bf60 0x560bd6718ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:01.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.437+0000 7f3764833640 1 -- 172.21.15.100:0/2991791015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd635bf60 msgr2=0x560bd6718ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:01.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.437+0000 7f3764833640 1 --2- 172.21.15.100:0/2991791015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd635bf60 0x560bd6718ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:01.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.437+0000 7f3764833640 1 -- 172.21.15.100:0/2991791015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd63330a0 msgr2=0x560bd670d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:01.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.437+0000 7f3764833640 1 --2- 172.21.15.100:0/2991791015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd63330a0 0x560bd670d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:01.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.437+0000 7f3764833640 1 -- 172.21.15.100:0/2991791015 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bd63c7540 con 0x560bd633b280 2024-07-26T13:40:01.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.437+0000 7f3765835640 1 --2- 172.21.15.100:0/2991791015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd635bf60 0x560bd6718ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:40:01.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.438+0000 7f3764833640 1 --2- 172.21.15.100:0/2991791015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd633b280 0x560bd670d5d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3754000c00 tx=0x7f37540cc4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a1e020ae7c211a server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:01.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.438+0000 7f3744ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:40:01.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.438+0000 7f3744ff9640 1 -- 172.21.15.100:0/2991791015 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37540cc660 con 0x560bd633b280 2024-07-26T13:40:01.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.438+0000 7f3744ff9640 1 -- 172.21.15.100:0/2991791015 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f375410b0c0 con 0x560bd633b280 2024-07-26T13:40:01.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.438+0000 7f376d81c740 1 -- 172.21.15.100:0/2991791015 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3758004050 con 0x560bd633b280 2024-07-26T13:40:01.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.438+0000 7f3744ff9640 1 -- 172.21.15.100:0/2991791015 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f37540d0bb0 con 0x560bd633b280 2024-07-26T13:40:01.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.438+0000 7f376d81c740 10 client.15312 fetch_fsmap learned FSMap version 25 2024-07-26T13:40:01.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.438+0000 7f376d81c740 1 -- 172.21.15.100:0/2991791015 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560bd671bbb0 con 0x560bd633b280 2024-07-26T13:40:01.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.438+0000 7f3744ff9640 1 -- 172.21.15.100:0/2991791015 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3758004050 con 0x560bd633b280 2024-07-26T13:40:01.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.438+0000 7f3744ff9640 1 -- 172.21.15.100:0/2991791015 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3754107ce0 con 0x560bd633b280 2024-07-26T13:40:01.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.438+0000 7f376d81c740 10 client.15312 fetch_fsmap finished waiting for FSMap version 25 2024-07-26T13:40:01.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.439+0000 7f376d81c740 10 client.15312 resolve_mds: resolved name 'i' to daemon mds.i 2024-07-26T13:40:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.439+0000 7f376d81c740 20 client.15312 populate_metadata read hostname 'smithi100' 2024-07-26T13:40:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.439+0000 7f376d81c740 1 --2- 172.21.15.100:0/2991791015 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560bd631a8e0 0x560bd671f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.439+0000 7f376d81c740 4 client.15312 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:40:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.439+0000 7f376d81c740 1 -- 172.21.15.100:0/2991791015 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bd6679890 con 0x560bd631a8e0 2024-07-26T13:40:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.439+0000 7f3765034640 1 --2- 172.21.15.100:0/2991791015 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560bd631a8e0 0x560bd671f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.439+0000 7f3765034640 1 --2- 172.21.15.100:0/2991791015 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560bd631a8e0 0x560bd671f6b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7189569e1907a060 server_cookie=33b818baa7df4dc2 in_seq=0 out_seq=0 2024-07-26T13:40:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.439+0000 7f3744ff9640 10 client.15312 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:40:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.440+0000 7f3744ff9640 1 -- 172.21.15.100:0/2991791015 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560bd6679890 con 0x560bd631a8e0 2024-07-26T13:40:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.440+0000 7f3744ff9640 10 client.15312 handle_command_reply: tid=0 2024-07-26T13:40:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.440+0000 7f376d81c740 1 client.15312 shutdown 2024-07-26T13:40:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.440+0000 7f376d81c740 1 -- 172.21.15.100:0/2991791015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd633b280 msgr2=0x560bd670d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:01.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.440+0000 7f376d81c740 1 --2- 172.21.15.100:0/2991791015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd633b280 0x560bd670d5d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3754000c00 tx=0x7f37540cc4f0 comp rx=0 tx=0).stop 2024-07-26T13:40:01.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.440+0000 7f376d81c740 1 -- 172.21.15.100:0/2991791015 shutdown_connections 2024-07-26T13:40:01.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.440+0000 7f376d81c740 1 --2- 172.21.15.100:0/2991791015 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560bd631a8e0 0x560bd671f6b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:01.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.440+0000 7f376d81c740 1 --2- 172.21.15.100:0/2991791015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd635bf60 0x560bd6718ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:01.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.440+0000 7f376d81c740 1 --2- 172.21.15.100:0/2991791015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd633b280 0x560bd670d5d0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:01.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.440+0000 7f376d81c740 1 --2- 172.21.15.100:0/2991791015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd63330a0 0x560bd670d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:01.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.440+0000 7f3744ff9640 0 client.15312 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:40:01.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.440+0000 7f376d81c740 1 -- 172.21.15.100:0/2991791015 >> 172.21.15.100:0/2991791015 conn(0x560bd6350350 msgr2=0x560bd6704580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:01.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.441+0000 7f376d81c740 1 -- 172.21.15.100:0/2991791015 shutdown_connections 2024-07-26T13:40:01.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.441+0000 7f376d81c740 1 -- 172.21.15.100:0/2991791015 wait complete. 2024-07-26T13:40:01.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.441+0000 7f376d81c740 20 client.15312 trim_cache size 0 max 16384 2024-07-26T13:40:01.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.458+0000 7f376d81c740 1 Processor -- start 2024-07-26T13:40:01.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.458+0000 7f376d81c740 1 -- start start 2024-07-26T13:40:01.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.459+0000 7f376d81c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd63330a0 0x560bd6727d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:01.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.459+0000 7f376d81c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd633b280 0x560bd66febd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:01.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.459+0000 7f376d81c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd635bf60 0x560bd66ff110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:01.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.459+0000 7f376d81c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x560bd671d9f0 con 0x560bd63330a0 2024-07-26T13:40:01.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.459+0000 7f376d81c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560bd671e0c0 con 0x560bd633b280 2024-07-26T13:40:01.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.459+0000 7f376d81c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560bd671e790 con 0x560bd635bf60 2024-07-26T13:40:01.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.459+0000 7f3765034640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd63330a0 0x560bd6727d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:01.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.459+0000 7f3765034640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd63330a0 0x560bd6727d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38542/0 (socket says 172.21.15.100:38542) 2024-07-26T13:40:01.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.459+0000 7f3765034640 1 -- 172.21.15.100:0/1159734018 learned_addr learned my addr 172.21.15.100:0/1159734018 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:01.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.459+0000 7f3765835640 1 --2- 172.21.15.100:0/1159734018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd635bf60 0x560bd66ff110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:01.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.459+0000 7f3765034640 1 -- 172.21.15.100:0/1159734018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd635bf60 msgr2=0x560bd66ff110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:01.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.459+0000 7f3765034640 1 --2- 172.21.15.100:0/1159734018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd635bf60 0x560bd66ff110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:01.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.459+0000 7f3765034640 1 -- 172.21.15.100:0/1159734018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd633b280 msgr2=0x560bd66febd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:01.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.459+0000 7f3765034640 1 --2- 172.21.15.100:0/1159734018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd633b280 0x560bd66febd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:01.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.459+0000 7f3765034640 1 -- 172.21.15.100:0/1159734018 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bd67038c0 con 0x560bd63330a0 2024-07-26T13:40:01.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.459+0000 7f3765835640 1 --2- 172.21.15.100:0/1159734018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd635bf60 0x560bd66ff110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:40:01.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.459+0000 7f3765034640 1 --2- 172.21.15.100:0/1159734018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd63330a0 0x560bd6727d80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f375000b9b0 tx=0x7f3750014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15af79d6101211c1 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:01.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.459+0000 7f3746ffd640 1 -- 172.21.15.100:0/1159734018 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375002d070 con 0x560bd63330a0 2024-07-26T13:40:01.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.459+0000 7f3746ffd640 1 -- 172.21.15.100:0/1159734018 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f375001fc50 con 0x560bd63330a0 2024-07-26T13:40:01.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.459+0000 7f3746ffd640 1 -- 172.21.15.100:0/1159734018 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3750028bd0 con 0x560bd63330a0 2024-07-26T13:40:01.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.460+0000 7f376d81c740 1 -- 172.21.15.100:0/1159734018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd63330a0 msgr2=0x560bd6727d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:01.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.460+0000 7f376d81c740 1 --2- 172.21.15.100:0/1159734018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd63330a0 0x560bd6727d80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f375000b9b0 tx=0x7f3750014000 comp rx=0 tx=0).stop 2024-07-26T13:40:01.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.460+0000 7f376d81c740 1 -- 172.21.15.100:0/1159734018 shutdown_connections 2024-07-26T13:40:01.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.460+0000 7f376d81c740 1 --2- 172.21.15.100:0/1159734018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd635bf60 0x560bd66ff110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:01.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.460+0000 7f376d81c740 1 --2- 172.21.15.100:0/1159734018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd633b280 0x560bd66febd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:01.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.460+0000 7f376d81c740 1 --2- 172.21.15.100:0/1159734018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd63330a0 0x560bd6727d80 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:01.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.460+0000 7f376d81c740 1 -- 172.21.15.100:0/1159734018 >> 172.21.15.100:0/1159734018 conn(0x560bd6350350 msgr2=0x560bd670d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:01.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.460+0000 7f376d81c740 1 -- 172.21.15.100:0/1159734018 shutdown_connections 2024-07-26T13:40:01.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.460+0000 7f376d81c740 1 -- 172.21.15.100:0/1159734018 wait complete. 2024-07-26T13:40:01.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.461+0000 7f376d81c740 1 -- start start 2024-07-26T13:40:01.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.461+0000 7f376d81c740 1 Processor -- start 2024-07-26T13:40:01.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.461+0000 7f376d81c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd63330a0 0x560bd6717e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:01.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.461+0000 7f376d81c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd633b280 0x560bd6718380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:01.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.461+0000 7f376d81c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd635bf60 0x560bd67188c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:01.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.461+0000 7f376d81c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x560bd671f530 con 0x560bd63330a0 2024-07-26T13:40:01.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.461+0000 7f376d81c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560bd671ee60 con 0x560bd635bf60 2024-07-26T13:40:01.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.461+0000 7f376d81c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560bd671fc00 con 0x560bd633b280 2024-07-26T13:40:01.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.461+0000 7f376d81c740 10 client.0 _ll_register_callbacks cb 0x7f37540c8520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:40:01.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.461+0000 7f3765034640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd63330a0 0x560bd6717e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:01.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.461+0000 7f3765034640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd63330a0 0x560bd6717e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38556/0 (socket says 172.21.15.100:38556) 2024-07-26T13:40:01.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.461+0000 7f3765034640 1 -- 172.21.15.100:0/3812593603 learned_addr learned my addr 172.21.15.100:0/3812593603 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:01.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.462+0000 7f3764833640 1 --2- 172.21.15.100:0/3812593603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd633b280 0x560bd6718380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:01.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.462+0000 7f3765034640 1 -- 172.21.15.100:0/3812593603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd633b280 msgr2=0x560bd6718380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:01.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.462+0000 7f3765034640 1 --2- 172.21.15.100:0/3812593603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd633b280 0x560bd6718380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:01.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.462+0000 7f3765034640 1 -- 172.21.15.100:0/3812593603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd635bf60 msgr2=0x560bd67188c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:01.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.462+0000 7f3765034640 1 --2- 172.21.15.100:0/3812593603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd635bf60 0x560bd67188c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:01.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.462+0000 7f3765034640 1 -- 172.21.15.100:0/3812593603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bd678aa40 con 0x560bd63330a0 2024-07-26T13:40:01.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.462+0000 7f3765034640 1 --2- 172.21.15.100:0/3812593603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd63330a0 0x560bd6717e40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3750005d30 tx=0x7f37500282d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=349e6483c339536e server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:01.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.462+0000 7f3744ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:40:01.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.462+0000 7f3744ff9640 1 -- 172.21.15.100:0/3812593603 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375002d050 con 0x560bd63330a0 2024-07-26T13:40:01.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.462+0000 7f3744ff9640 1 -- 172.21.15.100:0/3812593603 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f3750028c60 con 0x560bd63330a0 2024-07-26T13:40:01.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.462+0000 7f376d81c740 1 -- 172.21.15.100:0/3812593603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f375001fc50 con 0x560bd63330a0 2024-07-26T13:40:01.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.462+0000 7f3744ff9640 1 -- 172.21.15.100:0/3812593603 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3750040050 con 0x560bd63330a0 2024-07-26T13:40:01.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.462+0000 7f376d81c740 10 client.15318 fetch_fsmap learned FSMap version 25 2024-07-26T13:40:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.462+0000 7f376d81c740 1 -- 172.21.15.100:0/3812593603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560bd678bc60 con 0x560bd63330a0 2024-07-26T13:40:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.462+0000 7f3744ff9640 1 -- 172.21.15.100:0/3812593603 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f375001fc50 con 0x560bd63330a0 2024-07-26T13:40:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.463+0000 7f3744ff9640 1 -- 172.21.15.100:0/3812593603 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3750005430 con 0x560bd63330a0 2024-07-26T13:40:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.463+0000 7f376d81c740 10 client.15318 fetch_fsmap finished waiting for FSMap version 25 2024-07-26T13:40:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.463+0000 7f376d81c740 10 client.15318 resolve_mds: resolved name 'i' to daemon mds.i 2024-07-26T13:40:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.463+0000 7f376d81c740 20 client.15318 populate_metadata read hostname 'smithi100' 2024-07-26T13:40:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.463+0000 7f376d81c740 1 --2- 172.21.15.100:0/3812593603 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560bd62efc90 0x560bd678f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:01.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.463+0000 7f376d81c740 4 client.15318 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-26T13:40:01.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.463+0000 7f376d81c740 1 -- 172.21.15.100:0/3812593603 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "status"}) -- 0x560bd6675330 con 0x560bd62efc90 2024-07-26T13:40:01.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.463+0000 7f3764833640 1 --2- 172.21.15.100:0/3812593603 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560bd62efc90 0x560bd678f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:01.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.463+0000 7f3764833640 1 --2- 172.21.15.100:0/3812593603 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560bd62efc90 0x560bd678f2b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82409eeeb8044d97 server_cookie=6cc7ba83a62eec56 in_seq=0 out_seq=0 2024-07-26T13:40:01.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.463+0000 7f3744ff9640 10 client.15318 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:40:01.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.464+0000 7f3744ff9640 1 -- 172.21.15.100:0/3812593603 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560bd6675330 con 0x560bd62efc90 2024-07-26T13:40:01.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.464+0000 7f3744ff9640 10 client.15318 handle_command_reply: tid=0 2024-07-26T13:40:01.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.464+0000 7f376d81c740 1 client.15318 shutdown 2024-07-26T13:40:01.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.464+0000 7f376d81c740 1 -- 172.21.15.100:0/3812593603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd63330a0 msgr2=0x560bd6717e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:01.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.464+0000 7f376d81c740 1 --2- 172.21.15.100:0/3812593603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd63330a0 0x560bd6717e40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3750005d30 tx=0x7f37500282d0 comp rx=0 tx=0).stop 2024-07-26T13:40:01.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.464+0000 7f376d81c740 1 -- 172.21.15.100:0/3812593603 shutdown_connections 2024-07-26T13:40:01.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.464+0000 7f376d81c740 1 --2- 172.21.15.100:0/3812593603 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560bd62efc90 0x560bd678f2b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:01.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.464+0000 7f376d81c740 1 --2- 172.21.15.100:0/3812593603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd635bf60 0x560bd67188c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:01.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.464+0000 7f376d81c740 1 --2- 172.21.15.100:0/3812593603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd633b280 0x560bd6718380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:01.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.464+0000 7f376d81c740 1 --2- 172.21.15.100:0/3812593603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd63330a0 0x560bd6717e40 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:01.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.464+0000 7f376d81c740 1 -- 172.21.15.100:0/3812593603 >> 172.21.15.100:0/3812593603 conn(0x560bd6350350 msgr2=0x560bd6707f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:01.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.464+0000 7f3744ff9640 0 client.15318 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:40:01.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.464+0000 7f376d81c740 1 -- 172.21.15.100:0/3812593603 shutdown_connections 2024-07-26T13:40:01.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.464+0000 7f376d81c740 1 -- 172.21.15.100:0/3812593603 wait complete. 2024-07-26T13:40:01.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.464+0000 7f376d81c740 20 client.15318 trim_cache size 0 max 16384 2024-07-26T13:40:01.473 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:40:01.473 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "bf83e2e6-4b52-11ef-bcb7-c7b262605968", 2024-07-26T13:40:01.473 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 0, 2024-07-26T13:40:01.474 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 24607, 2024-07-26T13:40:01.474 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-07-26T13:40:01.474 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-07-26T13:40:01.474 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-07-26T13:40:01.474 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 179.99582077900001, 2024-07-26T13:40:01.474 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 25, 2024-07-26T13:40:01.474 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 81, 2024-07-26T13:40:01.474 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 81, 2024-07-26T13:40:01.475 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 206.765691433 2024-07-26T13:40:01.475 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:40:01.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.465+0000 7f3767298640 1 -- 172.21.15.100:0/2883895692 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f37480b3c20 msgr2=0x7f37480b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:01.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.465+0000 7f3767298640 1 --2- 172.21.15.100:0/2883895692 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f37480b3c20 0x7f37480b6130 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x560bd6700020 tx=0x7f375000b390 comp rx=0 tx=0).stop 2024-07-26T13:40:01.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.465+0000 7f3767298640 1 -- 172.21.15.100:0/2883895692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3760107e50 msgr2=0x7f3760199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:01.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.465+0000 7f3767298640 1 --2- 172.21.15.100:0/2883895692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3760107e50 0x7f3760199400 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3754006010 tx=0x7f3754004060 comp rx=0 tx=0).stop 2024-07-26T13:40:01.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.465+0000 7f3767298640 1 -- 172.21.15.100:0/2883895692 shutdown_connections 2024-07-26T13:40:01.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.465+0000 7f3767298640 1 --2- 172.21.15.100:0/2883895692 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f37480b3c20 0x7f37480b6130 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:01.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.465+0000 7f3767298640 1 --2- 172.21.15.100:0/2883895692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37601087a0 0x7f376019e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:01.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.465+0000 7f3767298640 1 --2- 172.21.15.100:0/2883895692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3760107e50 0x7f3760199400 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:01.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.465+0000 7f3767298640 1 --2- 172.21.15.100:0/2883895692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3760102510 0x7f3760198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:01.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.465+0000 7f3767298640 1 -- 172.21.15.100:0/2883895692 >> 172.21.15.100:0/2883895692 conn(0x7f37600791a0 msgr2=0x7f3760105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:01.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.465+0000 7f3767298640 1 -- 172.21.15.100:0/2883895692 shutdown_connections 2024-07-26T13:40:01.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:01.465+0000 7f3767298640 1 -- 172.21.15.100:0/2883895692 wait complete. 2024-07-26T13:40:01.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:01 smithi100 ceph-mon[32599]: pgmap v361: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:40:01.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:01 smithi167 ceph-mon[37140]: pgmap v361: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:40:01.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:01 smithi138 ceph-mon[36954]: pgmap v361: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:40:02.326 INFO:teuthology.run_tasks:Running task sequential... 2024-07-26T13:40:02.339 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-07-26T13:40:02.339 INFO:teuthology.task.sequential:In sequential, running task print... 2024-07-26T13:40:02.339 INFO:teuthology.task.print:Enabling mgr modules 2024-07-26T13:40:02.339 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-07-26T13:40:02.340 INFO:teuthology.task.exec:Executing custom commands... 2024-07-26T13:40:02.341 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi100.front.sepia.ceph.com 2024-07-26T13:40:02.341 DEBUG:teuthology.orchestra.run.smithi100:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-07-26T13:40:02.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.455+0000 7efecc45b640 1 -- 172.21.15.100:0/1369289493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efec4108e30 msgr2=0x7efec4109240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:02.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.455+0000 7efecc45b640 1 --2- 172.21.15.100:0/1369289493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efec4108e30 0x7efec4109240 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7efec0009900 tx=0x7efec002a410 comp rx=0 tx=0).stop 2024-07-26T13:40:02.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.455+0000 7efecc45b640 1 -- 172.21.15.100:0/1369289493 shutdown_connections 2024-07-26T13:40:02.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.455+0000 7efecc45b640 1 --2- 172.21.15.100:0/1369289493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efec4109780 0x7efec410ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:02.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.455+0000 7efecc45b640 1 --2- 172.21.15.100:0/1369289493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efec4108e30 0x7efec4109240 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:02.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.455+0000 7efecc45b640 1 --2- 172.21.15.100:0/1369289493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efec4103500 0x7efec4103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:02.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.455+0000 7efecc45b640 1 -- 172.21.15.100:0/1369289493 >> 172.21.15.100:0/1369289493 conn(0x7efec40fef40 msgr2=0x7efec4101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:02.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.455+0000 7efecc45b640 1 -- 172.21.15.100:0/1369289493 shutdown_connections 2024-07-26T13:40:02.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.455+0000 7efecc45b640 1 -- 172.21.15.100:0/1369289493 wait complete. 2024-07-26T13:40:02.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.456+0000 7efecc45b640 1 Processor -- start 2024-07-26T13:40:02.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.456+0000 7efecc45b640 1 -- start start 2024-07-26T13:40:02.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.456+0000 7efecc45b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efec4103500 0x7efec4199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:02.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.456+0000 7efecc45b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efec4108e30 0x7efec419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:02.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.456+0000 7efecc45b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efec4109780 0x7efec419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:02.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.456+0000 7efecc45b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efec4113460 con 0x7efec4108e30 2024-07-26T13:40:02.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.456+0000 7efecc45b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efec41132e0 con 0x7efec4109780 2024-07-26T13:40:02.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.456+0000 7efecc45b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efec41135e0 con 0x7efec4103500 2024-07-26T13:40:02.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.456+0000 7efec99f6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efec4108e30 0x7efec419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:02.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.456+0000 7efec99f6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efec4108e30 0x7efec419a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38572/0 (socket says 172.21.15.100:38572) 2024-07-26T13:40:02.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.456+0000 7efec99f6640 1 -- 172.21.15.100:0/54391584 learned_addr learned my addr 172.21.15.100:0/54391584 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:02.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.456+0000 7efec99f6640 1 -- 172.21.15.100:0/54391584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efec4103500 msgr2=0x7efec4199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:02.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.456+0000 7efec99f6640 1 --2- 172.21.15.100:0/54391584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efec4103500 0x7efec4199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:02.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.456+0000 7efec99f6640 1 -- 172.21.15.100:0/54391584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efec4109780 msgr2=0x7efec419f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:02.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.456+0000 7efeca9f8640 1 --2- 172.21.15.100:0/54391584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efec4109780 0x7efec419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:02.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.456+0000 7efec99f6640 1 --2- 172.21.15.100:0/54391584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efec4109780 0x7efec419f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:02.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.456+0000 7efec99f6640 1 -- 172.21.15.100:0/54391584 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec419fd60 con 0x7efec4108e30 2024-07-26T13:40:02.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.457+0000 7efec99f6640 1 --2- 172.21.15.100:0/54391584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efec4108e30 0x7efec419a460 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7efec0031440 tx=0x7efec0003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:02.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.457+0000 7efeb77fe640 1 -- 172.21.15.100:0/54391584 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec003f070 con 0x7efec4108e30 2024-07-26T13:40:02.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.457+0000 7efecc45b640 1 -- 172.21.15.100:0/54391584 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efec419fff0 con 0x7efec4108e30 2024-07-26T13:40:02.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.457+0000 7efeb77fe640 1 -- 172.21.15.100:0/54391584 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efec0031d60 con 0x7efec4108e30 2024-07-26T13:40:02.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.457+0000 7efeb77fe640 1 -- 172.21.15.100:0/54391584 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec0035030 con 0x7efec4108e30 2024-07-26T13:40:02.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.457+0000 7efecc45b640 1 -- 172.21.15.100:0/54391584 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efec41a04e0 con 0x7efec4108e30 2024-07-26T13:40:02.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.458+0000 7efecc45b640 1 -- 172.21.15.100:0/54391584 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efec4103910 con 0x7efec4108e30 2024-07-26T13:40:02.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.459+0000 7efeb77fe640 1 -- 172.21.15.100:0/54391584 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7efec0042070 con 0x7efec4108e30 2024-07-26T13:40:02.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.460+0000 7efeb77fe640 1 --2- 172.21.15.100:0/54391584 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7efeb00b3c20 0x7efeb00b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:02.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.460+0000 7efeca1f7640 1 --2- 172.21.15.100:0/54391584 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7efeb00b3c20 0x7efeb00b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:02.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.460+0000 7efeb77fe640 1 -- 172.21.15.100:0/54391584 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7efec0104aa0 con 0x7efec4108e30 2024-07-26T13:40:02.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.461+0000 7efeca1f7640 1 --2- 172.21.15.100:0/54391584 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7efeb00b3c20 0x7efeb00b6130 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efeb800a8f0 tx=0x7efeb8005bf0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:02.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.463+0000 7efeb77fe640 1 -- 172.21.15.100:0/54391584 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7efec00cbff0 con 0x7efec4108e30 2024-07-26T13:40:02.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:02.699+0000 7efecc45b640 1 -- 172.21.15.100:0/54391584 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7efec402e0c0 con 0x7efec4108e30 2024-07-26T13:40:03.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:03.514+0000 7efeb77fe640 1 -- 172.21.15.100:0/54391584 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 module 'snap_schedule' is already enabled v23) ==== 133+0+0 (secure 0 0 0) 0x7efec00cb8f0 con 0x7efec4108e30 2024-07-26T13:40:03.515 INFO:teuthology.orchestra.run.smithi100.stderr:module 'snap_schedule' is already enabled 2024-07-26T13:40:03.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:03.517+0000 7efecc45b640 1 -- 172.21.15.100:0/54391584 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7efeb00b3c20 msgr2=0x7efeb00b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:03.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:03.517+0000 7efecc45b640 1 --2- 172.21.15.100:0/54391584 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7efeb00b3c20 0x7efeb00b6130 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efeb800a8f0 tx=0x7efeb8005bf0 comp rx=0 tx=0).stop 2024-07-26T13:40:03.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:03.517+0000 7efecc45b640 1 -- 172.21.15.100:0/54391584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efec4108e30 msgr2=0x7efec419a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:03.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:03.517+0000 7efecc45b640 1 --2- 172.21.15.100:0/54391584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efec4108e30 0x7efec419a460 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7efec0031440 tx=0x7efec0003d50 comp rx=0 tx=0).stop 2024-07-26T13:40:03.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:03.517+0000 7efecc45b640 1 -- 172.21.15.100:0/54391584 shutdown_connections 2024-07-26T13:40:03.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:03.517+0000 7efecc45b640 1 --2- 172.21.15.100:0/54391584 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7efeb00b3c20 0x7efeb00b6130 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:03.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:03.517+0000 7efecc45b640 1 --2- 172.21.15.100:0/54391584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efec4109780 0x7efec419f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:03.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:03.517+0000 7efecc45b640 1 --2- 172.21.15.100:0/54391584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efec4108e30 0x7efec419a460 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:03.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:03.517+0000 7efecc45b640 1 --2- 172.21.15.100:0/54391584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efec4103500 0x7efec4199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:03.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:03.517+0000 7efecc45b640 1 -- 172.21.15.100:0/54391584 >> 172.21.15.100:0/54391584 conn(0x7efec40fef40 msgr2=0x7efec4101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:03.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:03.517+0000 7efecc45b640 1 -- 172.21.15.100:0/54391584 shutdown_connections 2024-07-26T13:40:03.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:03.517+0000 7efecc45b640 1 -- 172.21.15.100:0/54391584 wait complete. 2024-07-26T13:40:03.537 INFO:teuthology.run_tasks:Running task kclient... 2024-07-26T13:40:03.565 INFO:tasks.kclient:Mounting kernel clients... 2024-07-26T13:40:03.565 INFO:tasks.kclient:config is {'client.0': {}} 2024-07-26T13:40:03.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-26T13:40:03.565 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-07-26T13:40:03.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-26T13:40:03.583 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link delete ceph-brx 2024-07-26T13:40:03.652 INFO:teuthology.orchestra.run.smithi100.stderr:Cannot find device "ceph-brx" 2024-07-26T13:40:03.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-26T13:40:03.654 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-07-26T13:40:03.654 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-07-26T13:40:03.654 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-26T13:40:03.655 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs ls 2024-07-26T13:40:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:03 smithi167 ceph-mon[37140]: pgmap v362: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:40:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:03 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/54391584' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-07-26T13:40:03.926 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:03 smithi100 ceph-mon[32599]: pgmap v362: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:40:03.926 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:03 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/54391584' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-07-26T13:40:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:03 smithi138 ceph-mon[36954]: pgmap v362: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:40:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:03 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/54391584' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-07-26T13:40:03.958 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:40:04.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:04 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/54391584' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-07-26T13:40:04.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:04 smithi167 ceph-mon[37140]: mgrmap e23: x(active, since 9m), standbys: y, z 2024-07-26T13:40:04.885 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:04 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/54391584' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-07-26T13:40:04.886 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:04 smithi100 ceph-mon[32599]: mgrmap e23: x(active, since 9m), standbys: y, z 2024-07-26T13:40:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:04 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/54391584' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-07-26T13:40:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:04 smithi138 ceph-mon[36954]: mgrmap e23: x(active, since 9m), standbys: y, z 2024-07-26T13:40:05.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.749+0000 7fd5d5a20640 1 -- 172.21.15.100:0/3560945961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d0108ee0 msgr2=0x7fd5d01092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:05.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.749+0000 7fd5d5a20640 1 --2- 172.21.15.100:0/3560945961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d0108ee0 0x7fd5d01092f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4009900 tx=0x7fd5c402a410 comp rx=0 tx=0).stop 2024-07-26T13:40:05.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.749+0000 7fd5d5a20640 1 -- 172.21.15.100:0/3560945961 shutdown_connections 2024-07-26T13:40:05.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.749+0000 7fd5d5a20640 1 --2- 172.21.15.100:0/3560945961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5d0109830 0x7fd5d0110060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:05.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.749+0000 7fd5d5a20640 1 --2- 172.21.15.100:0/3560945961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d0108ee0 0x7fd5d01092f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:05.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.749+0000 7fd5d5a20640 1 --2- 172.21.15.100:0/3560945961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5d01035a0 0x7fd5d01039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:05.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.749+0000 7fd5d5a20640 1 -- 172.21.15.100:0/3560945961 >> 172.21.15.100:0/3560945961 conn(0x7fd5d00ff010 msgr2=0x7fd5d0101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:05.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.749+0000 7fd5d5a20640 1 -- 172.21.15.100:0/3560945961 shutdown_connections 2024-07-26T13:40:05.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.749+0000 7fd5d5a20640 1 -- 172.21.15.100:0/3560945961 wait complete. 2024-07-26T13:40:05.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.750+0000 7fd5d5a20640 1 Processor -- start 2024-07-26T13:40:05.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.750+0000 7fd5d5a20640 1 -- start start 2024-07-26T13:40:05.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.750+0000 7fd5d5a20640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5d01035a0 0x7fd5d0199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:05.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.750+0000 7fd5d5a20640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5d0108ee0 0x7fd5d019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:05.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.750+0000 7fd5d5a20640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d0109830 0x7fd5d019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:05.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.750+0000 7fd5d5a20640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d01134c0 con 0x7fd5d0109830 2024-07-26T13:40:05.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.750+0000 7fd5d5a20640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d0113340 con 0x7fd5d01035a0 2024-07-26T13:40:05.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.750+0000 7fd5d5a20640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d0113640 con 0x7fd5d0108ee0 2024-07-26T13:40:05.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.750+0000 7fd5cf7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d0109830 0x7fd5d019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:05.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.750+0000 7fd5cf7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d0109830 0x7fd5d019f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57630/0 (socket says 172.21.15.100:57630) 2024-07-26T13:40:05.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.750+0000 7fd5cf7fe640 1 -- 172.21.15.100:0/2342522558 learned_addr learned my addr 172.21.15.100:0/2342522558 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:05.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.750+0000 7fd5ceffd640 1 --2- 172.21.15.100:0/2342522558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5d01035a0 0x7fd5d0199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:05.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.751+0000 7fd5ce7fc640 1 --2- 172.21.15.100:0/2342522558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5d0108ee0 0x7fd5d019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:05.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.751+0000 7fd5cf7fe640 1 -- 172.21.15.100:0/2342522558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5d0108ee0 msgr2=0x7fd5d019a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:05.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.751+0000 7fd5cf7fe640 1 --2- 172.21.15.100:0/2342522558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5d0108ee0 0x7fd5d019a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:05.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.751+0000 7fd5cf7fe640 1 -- 172.21.15.100:0/2342522558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5d01035a0 msgr2=0x7fd5d0199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:05.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.751+0000 7fd5cf7fe640 1 --2- 172.21.15.100:0/2342522558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5d01035a0 0x7fd5d0199ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:05.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.751+0000 7fd5cf7fe640 1 -- 172.21.15.100:0/2342522558 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d019fe40 con 0x7fd5d0109830 2024-07-26T13:40:05.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.751+0000 7fd5ceffd640 1 --2- 172.21.15.100:0/2342522558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5d01035a0 0x7fd5d0199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:40:05.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.751+0000 7fd5cf7fe640 1 --2- 172.21.15.100:0/2342522558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d0109830 0x7fd5d019f710 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd5c800d750 tx=0x7fd5c800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:05.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.751+0000 7fd5d4a1e640 1 -- 172.21.15.100:0/2342522558 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c8017070 con 0x7fd5d0109830 2024-07-26T13:40:05.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.751+0000 7fd5d4a1e640 1 -- 172.21.15.100:0/2342522558 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5c80054d0 con 0x7fd5d0109830 2024-07-26T13:40:05.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.751+0000 7fd5d5a20640 1 -- 172.21.15.100:0/2342522558 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5d01a0130 con 0x7fd5d0109830 2024-07-26T13:40:05.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.751+0000 7fd5d4a1e640 1 -- 172.21.15.100:0/2342522558 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c8004c60 con 0x7fd5d0109830 2024-07-26T13:40:05.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.751+0000 7fd5d5a20640 1 -- 172.21.15.100:0/2342522558 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5d01a6890 con 0x7fd5d0109830 2024-07-26T13:40:05.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.752+0000 7fd5d5a20640 1 -- 172.21.15.100:0/2342522558 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5d0104820 con 0x7fd5d0109830 2024-07-26T13:40:05.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.754+0000 7fd5d4a1e640 1 -- 172.21.15.100:0/2342522558 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 151077+0+0 (secure 0 0 0) 0x7fd5c8009bd0 con 0x7fd5d0109830 2024-07-26T13:40:05.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.754+0000 7fd5d4a1e640 1 --2- 172.21.15.100:0/2342522558 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd5b80b3ca0 0x7fd5b80b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:05.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.754+0000 7fd5ceffd640 1 --2- 172.21.15.100:0/2342522558 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd5b80b3ca0 0x7fd5b80b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:05.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.754+0000 7fd5d4a1e640 1 -- 172.21.15.100:0/2342522558 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fd5c80e3790 con 0x7fd5d0109830 2024-07-26T13:40:05.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.755+0000 7fd5ceffd640 1 --2- 172.21.15.100:0/2342522558 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd5b80b3ca0 0x7fd5b80b61b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0005fd0 tx=0x7fd5c0009520 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:05.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.758+0000 7fd5d4a1e640 1 -- 172.21.15.100:0/2342522558 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fd5c80aace0 con 0x7fd5d0109830 2024-07-26T13:40:05.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:05 smithi167 ceph-mon[37140]: pgmap v363: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:40:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:05 smithi138 ceph-mon[36954]: pgmap v363: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:40:05.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.968+0000 7fd5d5a20640 1 -- 172.21.15.100:0/2342522558 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd5d010b050 con 0x7fd5d0109830 2024-07-26T13:40:05.969 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:05 smithi100 ceph-mon[32599]: pgmap v363: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:40:05.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.969+0000 7fd5d4a1e640 1 -- 172.21.15.100:0/2342522558 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+73 (secure 0 0 0) 0x7fd5c80aa5e0 con 0x7fd5d0109830 2024-07-26T13:40:05.970 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-07-26T13:40:05.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.971+0000 7fd5d5a20640 1 -- 172.21.15.100:0/2342522558 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd5b80b3ca0 msgr2=0x7fd5b80b61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:05.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.971+0000 7fd5d5a20640 1 --2- 172.21.15.100:0/2342522558 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd5b80b3ca0 0x7fd5b80b61b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0005fd0 tx=0x7fd5c0009520 comp rx=0 tx=0).stop 2024-07-26T13:40:05.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.971+0000 7fd5d5a20640 1 -- 172.21.15.100:0/2342522558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d0109830 msgr2=0x7fd5d019f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:05.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.971+0000 7fd5d5a20640 1 --2- 172.21.15.100:0/2342522558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d0109830 0x7fd5d019f710 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd5c800d750 tx=0x7fd5c800dc10 comp rx=0 tx=0).stop 2024-07-26T13:40:05.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.971+0000 7fd5d5a20640 1 -- 172.21.15.100:0/2342522558 shutdown_connections 2024-07-26T13:40:05.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.971+0000 7fd5d5a20640 1 --2- 172.21.15.100:0/2342522558 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd5b80b3ca0 0x7fd5b80b61b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:05.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.971+0000 7fd5d5a20640 1 --2- 172.21.15.100:0/2342522558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5d0109830 0x7fd5d019f710 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:05.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.971+0000 7fd5d5a20640 1 --2- 172.21.15.100:0/2342522558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5d0108ee0 0x7fd5d019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:05.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.971+0000 7fd5d5a20640 1 --2- 172.21.15.100:0/2342522558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5d01035a0 0x7fd5d0199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:05.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.971+0000 7fd5d5a20640 1 -- 172.21.15.100:0/2342522558 >> 172.21.15.100:0/2342522558 conn(0x7fd5d00ff010 msgr2=0x7fd5d0101430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:05.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.972+0000 7fd5d5a20640 1 -- 172.21.15.100:0/2342522558 shutdown_connections 2024-07-26T13:40:05.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:05.972+0000 7fd5d5a20640 1 -- 172.21.15.100:0/2342522558 wait complete. 2024-07-26T13:40:06.742 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:06 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2342522558' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-07-26T13:40:06.771 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-07-26T13:40:06.772 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-26T13:40:06.772 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-07-26T13:40:06.772 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-07-26T13:40:06.772 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-26T13:40:06.772 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-07-26T13:40:06.772 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-07-26T13:40:06.772 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-26T13:40:06.772 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-26T13:40:06.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-26T13:40:06.772 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-07-26T13:40:06.790 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-26T13:40:06.790 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-26T13:40:06.790 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-07-26T13:40:06.790 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-07-26T13:40:06.790 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-07-26T13:40:06.790 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-07-26T13:40:06.790 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-26T13:40:06.790 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-07-26T13:40:06.790 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-07-26T13:40:06.791 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-26T13:40:06.791 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-07-26T13:40:06.791 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-07-26T13:40:06.791 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-26T13:40:06.791 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-07-26T13:40:06.791 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-26T13:40:06.791 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-07-26T13:40:06.791 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-07-26T13:40:06.791 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42008sec preferred_lft 42008sec 2024-07-26T13:40:06.791 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link 2024-07-26T13:40:06.791 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-07-26T13:40:06.792 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-07-26T13:40:06.792 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-26T13:40:06.792 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-07-26T13:40:06.792 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link add name ceph-brx type bridge 2024-07-26T13:40:06.792 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip addr flush dev ceph-brx 2024-07-26T13:40:06.792 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set ceph-brx up 2024-07-26T13:40:06.792 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-07-26T13:40:06.792 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-07-26T13:40:06.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-26T13:40:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:06 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/2342522558' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-07-26T13:40:06.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-26T13:40:06.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-26T13:40:06.995 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-26T13:40:07.019 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-07-26T13:40:07.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-26T13:40:07.021 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-07-26T13:40:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:06 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/2342522558' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-07-26T13:40:07.088 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-07-26T13:40:07.089 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-26T13:40:07.089 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-07-26T13:40:07.089 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-07-26T13:40:07.089 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-26T13:40:07.089 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-26T13:40:07.089 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-07-26T13:40:07.090 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-07-26T13:40:07.090 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-07-26T13:40:07.090 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-07-26T13:40:07.090 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-07-26T13:40:07.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-26T13:40:07.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-26T13:40:07.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-26T13:40:07.411 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-07-26T13:40:07.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-26T13:40:07.427 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list-id 2024-07-26T13:40:07.482 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-26T13:40:07.482 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-07-26T13:40:07.482 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-26T13:40:07.482 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-07-26T13:40:07.482 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-07-26T13:40:07.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-26T13:40:07.578 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:07 smithi100 ceph-mon[32599]: pgmap v364: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T13:40:07.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-26T13:40:07.600 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-07-26T13:40:07.600 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-26T13:40:07.600 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-07-26T13:40:07.600 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-07-26T13:40:07.600 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-07-26T13:40:07.600 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-07-26T13:40:07.600 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-07-26T13:40:07.600 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-07-26T13:40:07.600 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-07-26T13:40:07.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-26T13:40:07.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-26T13:40:07.730 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-26T13:40:07.730 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-07-26T13:40:07.730 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-07-26T13:40:07.731 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set dev brx.0 master ceph-brx 2024-07-26T13:40:07.731 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-07-26T13:40:07.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-26T13:40:07.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-26T13:40:07.807 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-26T13:40:07.807 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-07-26T13:40:07.822 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-07-26T13:40:07.822 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-26T13:40:07.822 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-07-26T13:40:07.877 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-07-26T13:40:07.877 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-07-26T13:40:07.877 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/425a8b37-4935-4bf6-a775-12ed75a40837 2024-07-26T13:40:07.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-26T13:40:07.878 DEBUG:teuthology.orchestra.run.smithi100:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/425a8b37-4935-4bf6-a775-12ed75a40837 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-07-26T13:40:07.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:07 smithi138 ceph-mon[36954]: pgmap v364: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T13:40:07.989 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-07-26T13:40:07.989 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-07-26T13:40:07.989 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-07-26T13:40:08.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:07 smithi167 ceph-mon[37140]: pgmap v364: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T13:40:08.127 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-07-26T13:40:08.464 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-07-26T13:40:08.464 INFO:teuthology.orchestra.run.smithi100.stdout:invalid new device string format 2024-07-26T13:40:08.464 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:6789,172.21.15.138:6789,172.21.15.167:6789" 2024-07-26T13:40:08.464 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.100:6789,172.21.15.138:6789,172.21.15.167:6789:/volumes/qa/sv_1/425a8b37-4935-4bf6-a775-12ed75a40837 2024-07-26T13:40:08.464 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,key=0,fsid=bf83e2e6-4b52-11ef-bcb7-c7b262605968" will pass to kernel 2024-07-26T13:40:08.465 INFO:teuthology.orchestra.run.smithi100.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-07-26T13:40:08.465 INFO:teuthology.orchestra.run.smithi100.stdout: You just mounted a file system that supports labels which does not 2024-07-26T13:40:08.466 INFO:teuthology.orchestra.run.smithi100.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-07-26T13:40:08.466 INFO:teuthology.orchestra.run.smithi100.stdout: applications will generate AVC messages and not be allowed access to 2024-07-26T13:40:08.466 INFO:teuthology.orchestra.run.smithi100.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-07-26T13:40:08.466 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-07-26T13:40:08.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-26T13:40:08.466 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-07-26T13:40:08.494 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-26T13:40:08.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-26T13:40:08.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-26T13:40:08.593 INFO:teuthology.orchestra.run.smithi100.stdout:client15333 2024-07-26T13:40:08.593 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-07-26T13:40:08.593 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-26T13:40:08.610 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-26T13:40:08.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-26T13:40:08.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-26T13:40:08.707 INFO:teuthology.orchestra.run.smithi100.stdout:client15333 2024-07-26T13:40:08.708 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-26T13:40:08.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-26T13:40:08.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-26T13:40:08.765 INFO:teuthology.orchestra.run.smithi100.stdout:client15333 2024-07-26T13:40:08.765 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968.client15333/status') 2024-07-26T13:40:08.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-26T13:40:08.823 INFO:teuthology.orchestra.run.smithi100.stdout:instance: client.15333 (3)192.168.0.1:0/1052122651 2024-07-26T13:40:08.823 INFO:teuthology.orchestra.run.smithi100.stdout:blocklisted: false 2024-07-26T13:40:08.823 INFO:teuthology.orchestra.run.smithi100.stderr:0+1 records in 2024-07-26T13:40:08.823 INFO:teuthology.orchestra.run.smithi100.stderr:0+1 records out 2024-07-26T13:40:08.824 INFO:teuthology.orchestra.run.smithi100.stderr:70 bytes copied, 2.3271e-05 s, 3.0 MB/s 2024-07-26T13:40:08.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-26T13:40:08.831 INFO:teuthology.run_tasks:Running task exec... 2024-07-26T13:40:08.841 INFO:teuthology.task.exec:Executing custom commands... 2024-07-26T13:40:08.841 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi100.front.sepia.ceph.com 2024-07-26T13:40:08.841 DEBUG:teuthology.orchestra.run.smithi100:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-07-26T13:40:08.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.953+0000 7f2618de8640 1 -- 172.21.15.100:0/2105835477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f261410d7c0 msgr2=0x7f261410fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:08.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.953+0000 7f2618de8640 1 --2- 172.21.15.100:0/2105835477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f261410d7c0 0x7f261410fcb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f260c009ca0 tx=0x7f260c02a8c0 comp rx=0 tx=0).stop 2024-07-26T13:40:08.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.953+0000 7f2618de8640 1 -- 172.21.15.100:0/2105835477 shutdown_connections 2024-07-26T13:40:08.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.953+0000 7f2618de8640 1 --2- 172.21.15.100:0/2105835477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f261410d7c0 0x7f261410fcb0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:08.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.953+0000 7f2618de8640 1 --2- 172.21.15.100:0/2105835477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2614104cb0 0x7f261410d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:08.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.953+0000 7f2618de8640 1 --2- 172.21.15.100:0/2105835477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2614104360 0x7f2614104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:08.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.953+0000 7f2618de8640 1 -- 172.21.15.100:0/2105835477 >> 172.21.15.100:0/2105835477 conn(0x7f26140fddf0 msgr2=0x7f2614100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:08.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.953+0000 7f2618de8640 1 -- 172.21.15.100:0/2105835477 shutdown_connections 2024-07-26T13:40:08.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.953+0000 7f2618de8640 1 -- 172.21.15.100:0/2105835477 wait complete. 2024-07-26T13:40:08.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.954+0000 7f2618de8640 1 Processor -- start 2024-07-26T13:40:08.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.954+0000 7f2618de8640 1 -- start start 2024-07-26T13:40:08.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.954+0000 7f2618de8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2614104360 0x7f2614199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:08.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.954+0000 7f2618de8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2614104cb0 0x7f261419a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:08.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.954+0000 7f2618de8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f261410d7c0 0x7f261419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:08.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.954+0000 7f2618de8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2614113450 con 0x7f2614104360 2024-07-26T13:40:08.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.954+0000 7f2618de8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f26141132d0 con 0x7f2614104cb0 2024-07-26T13:40:08.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.954+0000 7f2618de8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f26141135d0 con 0x7f261410d7c0 2024-07-26T13:40:08.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.954+0000 7f261259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2614104360 0x7f2614199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:08.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.954+0000 7f261259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2614104360 0x7f2614199df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57644/0 (socket says 172.21.15.100:57644) 2024-07-26T13:40:08.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.954+0000 7f261259c640 1 -- 172.21.15.100:0/1667860229 learned_addr learned my addr 172.21.15.100:0/1667860229 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:08.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.954+0000 7f2611d9b640 1 --2- 172.21.15.100:0/1667860229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2614104cb0 0x7f261419a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:08.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.955+0000 7f2612d9d640 1 --2- 172.21.15.100:0/1667860229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f261410d7c0 0x7f261419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:08.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.955+0000 7f261259c640 1 -- 172.21.15.100:0/1667860229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f261410d7c0 msgr2=0x7f261419f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:08.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.955+0000 7f261259c640 1 --2- 172.21.15.100:0/1667860229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f261410d7c0 0x7f261419f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:08.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.955+0000 7f261259c640 1 -- 172.21.15.100:0/1667860229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2614104cb0 msgr2=0x7f261419a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:08.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.955+0000 7f261259c640 1 --2- 172.21.15.100:0/1667860229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2614104cb0 0x7f261419a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:08.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.955+0000 7f261259c640 1 -- 172.21.15.100:0/1667860229 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f261419fd20 con 0x7f2614104360 2024-07-26T13:40:08.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.955+0000 7f2611d9b640 1 --2- 172.21.15.100:0/1667860229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2614104cb0 0x7f261419a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:40:08.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.955+0000 7f261259c640 1 --2- 172.21.15.100:0/1667860229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2614104360 0x7f2614199df0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f260400e9f0 tx=0x7f260400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:08.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.955+0000 7f26037fe640 1 -- 172.21.15.100:0/1667860229 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260400cd80 con 0x7f2614104360 2024-07-26T13:40:08.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.955+0000 7f26037fe640 1 -- 172.21.15.100:0/1667860229 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26040108e0 con 0x7f2614104360 2024-07-26T13:40:08.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.955+0000 7f2618de8640 1 -- 172.21.15.100:0/1667860229 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26141a0010 con 0x7f2614104360 2024-07-26T13:40:08.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.955+0000 7f26037fe640 1 -- 172.21.15.100:0/1667860229 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2604018980 con 0x7f2614104360 2024-07-26T13:40:08.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.955+0000 7f2618de8640 1 -- 172.21.15.100:0/1667860229 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26141a67a0 con 0x7f2614104360 2024-07-26T13:40:08.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.956+0000 7f2618de8640 1 -- 172.21.15.100:0/1667860229 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2614104770 con 0x7f2614104360 2024-07-26T13:40:08.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.958+0000 7f26037fe640 1 -- 172.21.15.100:0/1667860229 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 151077+0+0 (secure 0 0 0) 0x7f2604010430 con 0x7f2614104360 2024-07-26T13:40:08.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.958+0000 7f26037fe640 1 --2- 172.21.15.100:0/1667860229 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f25fc0b3c20 0x7f25fc0b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:08.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.958+0000 7f2611d9b640 1 --2- 172.21.15.100:0/1667860229 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f25fc0b3c20 0x7f25fc0b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:08.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.958+0000 7f26037fe640 1 -- 172.21.15.100:0/1667860229 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f26040e4690 con 0x7f2614104360 2024-07-26T13:40:08.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.959+0000 7f2611d9b640 1 --2- 172.21.15.100:0/1667860229 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f25fc0b3c20 0x7f25fc0b6130 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f261410ee50 tx=0x7f2608005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:08.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:08.961+0000 7f26037fe640 1 -- 172.21.15.100:0/1667860229 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f26040abbe0 con 0x7f2614104360 2024-07-26T13:40:09.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.185+0000 7f2618de8640 1 -- 172.21.15.100:0/1667860229 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f26141a6b10 con 0x7f2614104360 2024-07-26T13:40:09.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.531+0000 7f26037fe640 1 -- 172.21.15.100:0/1667860229 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v26) ==== 117+0+0 (secure 0 0 0) 0x7f26040ab4e0 con 0x7f2614104360 2024-07-26T13:40:09.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.533+0000 7f2618de8640 1 -- 172.21.15.100:0/1667860229 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f25fc0b3c20 msgr2=0x7f25fc0b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:09.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.533+0000 7f2618de8640 1 --2- 172.21.15.100:0/1667860229 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f25fc0b3c20 0x7f25fc0b6130 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f261410ee50 tx=0x7f2608005f90 comp rx=0 tx=0).stop 2024-07-26T13:40:09.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.533+0000 7f2618de8640 1 -- 172.21.15.100:0/1667860229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2614104360 msgr2=0x7f2614199df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:09.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.533+0000 7f2618de8640 1 --2- 172.21.15.100:0/1667860229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2614104360 0x7f2614199df0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f260400e9f0 tx=0x7f260400eeb0 comp rx=0 tx=0).stop 2024-07-26T13:40:09.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.533+0000 7f2618de8640 1 -- 172.21.15.100:0/1667860229 shutdown_connections 2024-07-26T13:40:09.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.533+0000 7f2618de8640 1 --2- 172.21.15.100:0/1667860229 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f25fc0b3c20 0x7f25fc0b6130 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:09.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.533+0000 7f2618de8640 1 --2- 172.21.15.100:0/1667860229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f261410d7c0 0x7f261419f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:09.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.533+0000 7f2618de8640 1 --2- 172.21.15.100:0/1667860229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2614104cb0 0x7f261419a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:09.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.533+0000 7f2618de8640 1 --2- 172.21.15.100:0/1667860229 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2614104360 0x7f2614199df0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:09.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.533+0000 7f2618de8640 1 -- 172.21.15.100:0/1667860229 >> 172.21.15.100:0/1667860229 conn(0x7f26140fddf0 msgr2=0x7f2614107240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:09.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.534+0000 7f2618de8640 1 -- 172.21.15.100:0/1667860229 shutdown_connections 2024-07-26T13:40:09.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.534+0000 7f2618de8640 1 -- 172.21.15.100:0/1667860229 wait complete. 2024-07-26T13:40:09.547 INFO:teuthology.run_tasks:Running task check-counter... 2024-07-26T13:40:09.558 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-07-26T13:40:09.571 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-26T13:40:09.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.682+0000 7f05f9aea640 1 -- 172.21.15.100:0/1493549044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05f4107cf0 msgr2=0x7f05f4108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:09.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.682+0000 7f05f9aea640 1 --2- 172.21.15.100:0/1493549044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05f4107cf0 0x7f05f4108100 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f05e8009900 tx=0x7f05e802a410 comp rx=0 tx=0).stop 2024-07-26T13:40:09.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.682+0000 7f05f9aea640 1 -- 172.21.15.100:0/1493549044 shutdown_connections 2024-07-26T13:40:09.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.682+0000 7f05f9aea640 1 --2- 172.21.15.100:0/1493549044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05f4108640 0x7f05f410ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:09.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.682+0000 7f05f9aea640 1 --2- 172.21.15.100:0/1493549044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05f4107cf0 0x7f05f4108100 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:09.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.682+0000 7f05f9aea640 1 --2- 172.21.15.100:0/1493549044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05f41023c0 0x7f05f41027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:09.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.682+0000 7f05f9aea640 1 -- 172.21.15.100:0/1493549044 >> 172.21.15.100:0/1493549044 conn(0x7f05f40fdd80 msgr2=0x7f05f41001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:09.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.682+0000 7f05f9aea640 1 -- 172.21.15.100:0/1493549044 shutdown_connections 2024-07-26T13:40:09.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.682+0000 7f05f9aea640 1 -- 172.21.15.100:0/1493549044 wait complete. 2024-07-26T13:40:09.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.682+0000 7f05f9aea640 1 Processor -- start 2024-07-26T13:40:09.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.683+0000 7f05f9aea640 1 -- start start 2024-07-26T13:40:09.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.683+0000 7f05f9aea640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05f41023c0 0x7f05f4199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:09.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.683+0000 7f05f9aea640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05f4107cf0 0x7f05f419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:09.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.683+0000 7f05f9aea640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05f4108640 0x7f05f419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:09.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.683+0000 7f05f9aea640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f05f41123a0 con 0x7f05f41023c0 2024-07-26T13:40:09.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.683+0000 7f05f9aea640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f05f4112220 con 0x7f05f4107cf0 2024-07-26T13:40:09.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.683+0000 7f05f9aea640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f05f4112520 con 0x7f05f4108640 2024-07-26T13:40:09.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.683+0000 7f05f37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05f41023c0 0x7f05f4199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:09.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.683+0000 7f05f37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05f41023c0 0x7f05f4199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57662/0 (socket says 172.21.15.100:57662) 2024-07-26T13:40:09.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.683+0000 7f05f37fe640 1 -- 172.21.15.100:0/631457510 learned_addr learned my addr 172.21.15.100:0/631457510 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:09.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.683+0000 7f05f2ffd640 1 --2- 172.21.15.100:0/631457510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05f4107cf0 0x7f05f419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:09.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.683+0000 7f05f37fe640 1 -- 172.21.15.100:0/631457510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05f4108640 msgr2=0x7f05f419f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:09.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.683+0000 7f05f3fff640 1 --2- 172.21.15.100:0/631457510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05f4108640 0x7f05f419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:09.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.683+0000 7f05f37fe640 1 --2- 172.21.15.100:0/631457510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05f4108640 0x7f05f419f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:09.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.683+0000 7f05f37fe640 1 -- 172.21.15.100:0/631457510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05f4107cf0 msgr2=0x7f05f419a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:09.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.683+0000 7f05f37fe640 1 --2- 172.21.15.100:0/631457510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05f4107cf0 0x7f05f419a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:09.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.683+0000 7f05f37fe640 1 -- 172.21.15.100:0/631457510 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f419fee0 con 0x7f05f41023c0 2024-07-26T13:40:09.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.683+0000 7f05f2ffd640 1 --2- 172.21.15.100:0/631457510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05f4107cf0 0x7f05f419a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:40:09.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.683+0000 7f05f3fff640 1 --2- 172.21.15.100:0/631457510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05f4108640 0x7f05f419f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:40:09.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.684+0000 7f05f37fe640 1 --2- 172.21.15.100:0/631457510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05f41023c0 0x7f05f4199fb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f05e400b4a0 tx=0x7f05e400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:09.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.684+0000 7f05f0ff9640 1 -- 172.21.15.100:0/631457510 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e4014070 con 0x7f05f41023c0 2024-07-26T13:40:09.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.684+0000 7f05f9aea640 1 -- 172.21.15.100:0/631457510 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05f41a01d0 con 0x7f05f41023c0 2024-07-26T13:40:09.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.684+0000 7f05f0ff9640 1 -- 172.21.15.100:0/631457510 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05e4010ac0 con 0x7f05f41023c0 2024-07-26T13:40:09.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.684+0000 7f05f0ff9640 1 -- 172.21.15.100:0/631457510 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e4017c30 con 0x7f05f41023c0 2024-07-26T13:40:09.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.684+0000 7f05f9aea640 1 -- 172.21.15.100:0/631457510 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05f41a6960 con 0x7f05f41023c0 2024-07-26T13:40:09.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.685+0000 7f05f9aea640 1 -- 172.21.15.100:0/631457510 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05f4102850 con 0x7f05f41023c0 2024-07-26T13:40:09.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.686+0000 7f05f0ff9640 1 -- 172.21.15.100:0/631457510 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 151077+0+0 (secure 0 0 0) 0x7f05e4017070 con 0x7f05f41023c0 2024-07-26T13:40:09.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.687+0000 7f05f0ff9640 1 --2- 172.21.15.100:0/631457510 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f05dc0b3c20 0x7f05dc0b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:09.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.687+0000 7f05f2ffd640 1 --2- 172.21.15.100:0/631457510 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f05dc0b3c20 0x7f05dc0b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:09.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.687+0000 7f05f0ff9640 1 -- 172.21.15.100:0/631457510 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f05e40e3520 con 0x7f05f41023c0 2024-07-26T13:40:09.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.688+0000 7f05f2ffd640 1 --2- 172.21.15.100:0/631457510 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f05dc0b3c20 0x7f05dc0b6130 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f05e8009230 tx=0x7f05e8000bc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:09.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.691+0000 7f05f0ff9640 1 -- 172.21.15.100:0/631457510 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f05e40aaa70 con 0x7f05f41023c0 2024-07-26T13:40:09.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.876+0000 7f05f9aea640 1 -- 172.21.15.100:0/631457510 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f05f402e0c0 con 0x7f05f41023c0 2024-07-26T13:40:09.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.877+0000 7f05f0ff9640 1 -- 172.21.15.100:0/631457510 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+23121 (secure 0 0 0) 0x7f05e40aa370 con 0x7f05f41023c0 2024-07-26T13:40:09.879 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:40:09.879 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":81,"fsid":"bf83e2e6-4b52-11ef-bcb7-c7b262605968","created":"2024-07-26T13:28:23.639785+0000","modified":"2024-07-26T13:37:17.539521+0000","last_up_change":"2024-07-26T13:35:56.475554+0000","last_in_change":"2024-07-26T13:35:37.472564+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-26T13:32:22.329969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-26T13:36:54.590525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-26T13:36:58.149545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d714036d-9dda-4cb5-af16-29f83a09dd70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6803","nonce":1480401850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6805","nonce":1480401850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6809","nonce":1480401850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6807","nonce":1480401850}]},"public_addr":"172.21.15.100:6803/1480401850","cluster_addr":"172.21.15.100:6805/1480401850","heartbeat_back_addr":"172.21.15.100:6809/1480401850","heartbeat_front_addr":"172.21.15.100:6807/1480401850","state":["exists","up"]},{"osd":1,"uuid":"3dd0eaa1-618a-495a-9819-e01c0f869ae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6801","nonce":3090877671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6803","nonce":3090877671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6807","nonce":3090877671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6805","nonce":3090877671}]},"public_addr":"172.21.15.138:6801/3090877671","cluster_addr":"172.21.15.138:6803/3090877671","heartbeat_back_addr":"172.21.15.138:6807/3090877671","heartbeat_front_addr":"172.21.15.138:6805/3090877671","state":["exists","up"]},{"osd":2,"uuid":"5c710a78-17ae-41a8-ac16-a678f5d35672","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6801","nonce":3583424790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6803","nonce":3583424790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6807","nonce":3583424790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6805","nonce":3583424790}]},"public_addr":"172.21.15.167:6801/3583424790","cluster_addr":"172.21.15.167:6803/3583424790","heartbeat_back_addr":"172.21.15.167:6807/3583424790","heartbeat_front_addr":"172.21.15.167:6805/3583424790","state":["exists","up"]},{"osd":3,"uuid":"be845495-8c31-4834-a45c-bdc576711962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6811","nonce":3626762419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6813","nonce":3626762419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6817","nonce":3626762419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6815","nonce":3626762419}]},"public_addr":"172.21.15.100:6811/3626762419","cluster_addr":"172.21.15.100:6813/3626762419","heartbeat_back_addr":"172.21.15.100:6817/3626762419","heartbeat_front_addr":"172.21.15.100:6815/3626762419","state":["exists","up"]},{"osd":4,"uuid":"520554cc-e846-46f9-af77-05a1de65a452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6809","nonce":3757334356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6811","nonce":3757334356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6815","nonce":3757334356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6813","nonce":3757334356}]},"public_addr":"172.21.15.138:6809/3757334356","cluster_addr":"172.21.15.138:6811/3757334356","heartbeat_back_addr":"172.21.15.138:6815/3757334356","heartbeat_front_addr":"172.21.15.138:6813/3757334356","state":["exists","up"]},{"osd":5,"uuid":"68dc8ad6-31d2-45b5-9aa7-62c4218d46e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6809","nonce":3239482836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6811","nonce":3239482836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6815","nonce":3239482836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6813","nonce":3239482836}]},"public_addr":"172.21.15.167:6809/3239482836","cluster_addr":"172.21.15.167:6811/3239482836","heartbeat_back_addr":"172.21.15.167:6815/3239482836","heartbeat_front_addr":"172.21.15.167:6813/3239482836","state":["exists","up"]},{"osd":6,"uuid":"420e4471-5626-41cc-a8e1-69c801196b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6819","nonce":3218415484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6821","nonce":3218415484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6825","nonce":3218415484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6823","nonce":3218415484}]},"public_addr":"172.21.15.100:6819/3218415484","cluster_addr":"172.21.15.100:6821/3218415484","heartbeat_back_addr":"172.21.15.100:6825/3218415484","heartbeat_front_addr":"172.21.15.100:6823/3218415484","state":["exists","up"]},{"osd":7,"uuid":"a27b097e-207d-44c3-af63-544f228b83ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6817","nonce":45541749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6819","nonce":45541749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6823","nonce":45541749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6821","nonce":45541749}]},"public_addr":"172.21.15.138:6817/45541749","cluster_addr":"172.21.15.138:6819/45541749","heartbeat_back_addr":"172.21.15.138:6823/45541749","heartbeat_front_addr":"172.21.15.138:6821/45541749","state":["exists","up"]},{"osd":8,"uuid":"3e865858-92cc-48a0-a464-5767d19e8ac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6817","nonce":4031159803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6819","nonce":4031159803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6823","nonce":4031159803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6821","nonce":4031159803}]},"public_addr":"172.21.15.167:6817/4031159803","cluster_addr":"172.21.15.167:6819/4031159803","heartbeat_back_addr":"172.21.15.167:6823/4031159803","heartbeat_front_addr":"172.21.15.167:6821/4031159803","state":["exists","up"]},{"osd":9,"uuid":"8b05b14a-1195-42f5-ac01-d5634cb9c888","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6827","nonce":1840140273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6829","nonce":1840140273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6833","nonce":1840140273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6831","nonce":1840140273}]},"public_addr":"172.21.15.100:6827/1840140273","cluster_addr":"172.21.15.100:6829/1840140273","heartbeat_back_addr":"172.21.15.100:6833/1840140273","heartbeat_front_addr":"172.21.15.100:6831/1840140273","state":["exists","up"]},{"osd":10,"uuid":"ab77ebe7-5254-4dc3-bc54-625698e84afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6825","nonce":703382899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6827","nonce":703382899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6831","nonce":703382899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6829","nonce":703382899}]},"public_addr":"172.21.15.138:6825/703382899","cluster_addr":"172.21.15.138:6827/703382899","heartbeat_back_addr":"172.21.15.138:6831/703382899","heartbeat_front_addr":"172.21.15.138:6829/703382899","state":["exists","up"]},{"osd":11,"uuid":"229cf91e-e9ef-48f6-a677-c5c5705b315a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6825","nonce":1032343068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6827","nonce":1032343068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6831","nonce":1032343068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6829","nonce":1032343068}]},"public_addr":"172.21.15.167:6825/1032343068","cluster_addr":"172.21.15.167:6827/1032343068","heartbeat_back_addr":"172.21.15.167:6831/1032343068","heartbeat_front_addr":"172.21.15.167:6829/1032343068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:31:35.800702+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:31:58.087922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:32:20.463183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:32:42.920598+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:33:06.332655+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:33:29.381407+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:33:53.975963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:34:18.226813+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:34:41.845770+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:35:05.722275+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:35:30.862704+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:35:54.738697+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:0/1874993112":"2024-07-27T13:30:16.253096+0000","172.21.15.100:0/2364620305":"2024-07-27T13:28:45.340334+0000","172.21.15.100:6800/3096076075":"2024-07-27T13:29:07.979178+0000","172.21.15.100:0/1894966501":"2024-07-27T13:30:16.253096+0000","172.21.15.100:0/3411776672":"2024-07-27T13:28:45.340334+0000","172.21.15.100:0/2853423120":"2024-07-27T13:29:07.979178+0000","172.21.15.100:6800/3025873570":"2024-07-27T13:28:45.340334+0000","172.21.15.100:0/4887181":"2024-07-27T13:29:07.979178+0000","172.21.15.100:6801/908675140":"2024-07-27T13:30:16.253096+0000","172.21.15.100:6801/3025873570":"2024-07-27T13:28:45.340334+0000","172.21.15.100:6800/908675140":"2024-07-27T13:30:16.253096+0000","172.21.15.100:6801/3096076075":"2024-07-27T13:29:07.979178+0000","172.21.15.100:0/2302632052":"2024-07-27T13:29:07.979178+0000","172.21.15.100:0/3736098915":"2024-07-27T13:28:45.340334+0000","172.21.15.100:0/3243030560":"2024-07-27T13:30:16.253096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-26T13:40:09.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.880+0000 7f05f9aea640 1 -- 172.21.15.100:0/631457510 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f05dc0b3c20 msgr2=0x7f05dc0b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:09.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.880+0000 7f05f9aea640 1 --2- 172.21.15.100:0/631457510 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f05dc0b3c20 0x7f05dc0b6130 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f05e8009230 tx=0x7f05e8000bc0 comp rx=0 tx=0).stop 2024-07-26T13:40:09.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.880+0000 7f05f9aea640 1 -- 172.21.15.100:0/631457510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05f41023c0 msgr2=0x7f05f4199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:09.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.880+0000 7f05f9aea640 1 --2- 172.21.15.100:0/631457510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05f41023c0 0x7f05f4199fb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f05e400b4a0 tx=0x7f05e400b960 comp rx=0 tx=0).stop 2024-07-26T13:40:09.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.880+0000 7f05f9aea640 1 -- 172.21.15.100:0/631457510 shutdown_connections 2024-07-26T13:40:09.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.880+0000 7f05f9aea640 1 --2- 172.21.15.100:0/631457510 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f05dc0b3c20 0x7f05dc0b6130 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:09.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.880+0000 7f05f9aea640 1 --2- 172.21.15.100:0/631457510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05f4108640 0x7f05f419f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:09.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.880+0000 7f05f9aea640 1 --2- 172.21.15.100:0/631457510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05f4107cf0 0x7f05f419a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:09.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.880+0000 7f05f9aea640 1 --2- 172.21.15.100:0/631457510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f05f41023c0 0x7f05f4199fb0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:09.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.880+0000 7f05f9aea640 1 -- 172.21.15.100:0/631457510 >> 172.21.15.100:0/631457510 conn(0x7f05f40fdd80 msgr2=0x7f05f406b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:09.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.880+0000 7f05f9aea640 1 -- 172.21.15.100:0/631457510 shutdown_connections 2024-07-26T13:40:09.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:09.880+0000 7f05f9aea640 1 -- 172.21.15.100:0/631457510 wait complete. 2024-07-26T13:40:09.891 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:09 smithi100 ceph-mon[32599]: pgmap v365: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:40:09.891 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:09 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/1667860229' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-07-26T13:40:09.892 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-26T13:32:22.329969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-07-26T13:36:54.590525+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-07-26T13:36:58.149545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-26T13:40:09.893 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-26T13:40:09.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:09 smithi138 ceph-mon[36954]: pgmap v365: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:40:09.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:09 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/1667860229' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-07-26T13:40:10.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.006+0000 7f2ce66d3640 1 -- 172.21.15.100:0/3839768132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ce0102b20 msgr2=0x7f2ce0102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:10.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.006+0000 7f2ce66d3640 1 --2- 172.21.15.100:0/3839768132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ce0102b20 0x7f2ce0102f30 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0009900 tx=0x7f2cd002a410 comp rx=0 tx=0).stop 2024-07-26T13:40:10.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.006+0000 7f2ce66d3640 1 -- 172.21.15.100:0/3839768132 shutdown_connections 2024-07-26T13:40:10.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.006+0000 7f2ce66d3640 1 --2- 172.21.15.100:0/3839768132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ce0105430 0x7f2ce010eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:10.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.006+0000 7f2ce66d3640 1 --2- 172.21.15.100:0/3839768132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ce0104ae0 0x7f2ce0104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:10.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.006+0000 7f2ce66d3640 1 --2- 172.21.15.100:0/3839768132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ce0102b20 0x7f2ce0102f30 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:10.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.006+0000 7f2ce66d3640 1 -- 172.21.15.100:0/3839768132 >> 172.21.15.100:0/3839768132 conn(0x7f2ce0079840 msgr2=0x7f2ce0079c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:10.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.006+0000 7f2ce66d3640 1 -- 172.21.15.100:0/3839768132 shutdown_connections 2024-07-26T13:40:10.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.006+0000 7f2ce66d3640 1 -- 172.21.15.100:0/3839768132 wait complete. 2024-07-26T13:40:10.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.007+0000 7f2ce66d3640 1 Processor -- start 2024-07-26T13:40:10.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.007+0000 7f2ce66d3640 1 -- start start 2024-07-26T13:40:10.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.007+0000 7f2ce66d3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ce0102b20 0x7f2ce0198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:10.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.007+0000 7f2ce66d3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ce0104ae0 0x7f2ce01992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:10.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.007+0000 7f2ce66d3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ce0105430 0x7f2ce019e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:10.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.007+0000 7f2ce66d3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce0112340 con 0x7f2ce0104ae0 2024-07-26T13:40:10.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.007+0000 7f2ce66d3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce01121c0 con 0x7f2ce0105430 2024-07-26T13:40:10.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.007+0000 7f2ce66d3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce01124c0 con 0x7f2ce0102b20 2024-07-26T13:40:10.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.007+0000 7f2cdf7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ce0104ae0 0x7f2ce01992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:10.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.007+0000 7f2cdf7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ce0104ae0 0x7f2ce01992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57686/0 (socket says 172.21.15.100:57686) 2024-07-26T13:40:10.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.007+0000 7f2cdf7fe640 1 -- 172.21.15.100:0/2576982628 learned_addr learned my addr 172.21.15.100:0/2576982628 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:10.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.007+0000 7f2ce4c70640 1 --2- 172.21.15.100:0/2576982628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ce0105430 0x7f2ce019e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:10.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.007+0000 7f2cdffff640 1 --2- 172.21.15.100:0/2576982628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ce0102b20 0x7f2ce0198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:10.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.007+0000 7f2cdf7fe640 1 -- 172.21.15.100:0/2576982628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ce0102b20 msgr2=0x7f2ce0198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:10.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.007+0000 7f2cdf7fe640 1 --2- 172.21.15.100:0/2576982628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ce0102b20 0x7f2ce0198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:10.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.007+0000 7f2cdf7fe640 1 -- 172.21.15.100:0/2576982628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ce0105430 msgr2=0x7f2ce019e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:10.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.007+0000 7f2cdf7fe640 1 --2- 172.21.15.100:0/2576982628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ce0105430 0x7f2ce019e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:10.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.007+0000 7f2cdf7fe640 1 -- 172.21.15.100:0/2576982628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce019ec50 con 0x7f2ce0104ae0 2024-07-26T13:40:10.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.008+0000 7f2cdf7fe640 1 --2- 172.21.15.100:0/2576982628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ce0104ae0 0x7f2ce01992c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2cd400e9f0 tx=0x7f2cd400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:10.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.008+0000 7f2cdd7fa640 1 -- 172.21.15.100:0/2576982628 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd400ccc0 con 0x7f2ce0104ae0 2024-07-26T13:40:10.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.008+0000 7f2cdd7fa640 1 -- 172.21.15.100:0/2576982628 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cd40108e0 con 0x7f2ce0104ae0 2024-07-26T13:40:10.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.008+0000 7f2cdd7fa640 1 -- 172.21.15.100:0/2576982628 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4018860 con 0x7f2ce0104ae0 2024-07-26T13:40:10.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.008+0000 7f2ce66d3640 1 -- 172.21.15.100:0/2576982628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ce019ef40 con 0x7f2ce0104ae0 2024-07-26T13:40:10.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.008+0000 7f2ce66d3640 1 -- 172.21.15.100:0/2576982628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ce01a56d0 con 0x7f2ce0104ae0 2024-07-26T13:40:10.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.009+0000 7f2ce66d3640 1 -- 172.21.15.100:0/2576982628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ce0199e40 con 0x7f2ce0104ae0 2024-07-26T13:40:10.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.010+0000 7f2cdd7fa640 1 -- 172.21.15.100:0/2576982628 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 151077+0+0 (secure 0 0 0) 0x7f2cd4010430 con 0x7f2ce0104ae0 2024-07-26T13:40:10.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.011+0000 7f2cdd7fa640 1 --2- 172.21.15.100:0/2576982628 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f2cc80b3c20 0x7f2cc80b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:10.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.011+0000 7f2cdffff640 1 --2- 172.21.15.100:0/2576982628 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f2cc80b3c20 0x7f2cc80b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:10.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.011+0000 7f2cdd7fa640 1 -- 172.21.15.100:0/2576982628 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f2cd40e4640 con 0x7f2ce0104ae0 2024-07-26T13:40:10.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.012+0000 7f2cdffff640 1 --2- 172.21.15.100:0/2576982628 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f2cc80b3c20 0x7f2cc80b6130 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0005f70 tx=0x7f2cd0000bc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:10.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.014+0000 7f2cdd7fa640 1 -- 172.21.15.100:0/2576982628 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f2cd40abb90 con 0x7f2ce0104ae0 2024-07-26T13:40:10.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:09 smithi167 ceph-mon[37140]: pgmap v365: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:40:10.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:09 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/1667860229' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-07-26T13:40:10.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.198+0000 7f2ce66d3640 1 -- 172.21.15.100:0/2576982628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2ce002e0c0 con 0x7f2ce0104ae0 2024-07-26T13:40:10.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.198+0000 7f2cdd7fa640 1 -- 172.21.15.100:0/2576982628 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f2cd40ab490 con 0x7f2ce0104ae0 2024-07-26T13:40:10.199 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-07-26T13:40:10.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.200+0000 7f2ce66d3640 1 -- 172.21.15.100:0/2576982628 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f2cc80b3c20 msgr2=0x7f2cc80b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:10.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.200+0000 7f2ce66d3640 1 --2- 172.21.15.100:0/2576982628 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f2cc80b3c20 0x7f2cc80b6130 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0005f70 tx=0x7f2cd0000bc0 comp rx=0 tx=0).stop 2024-07-26T13:40:10.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.200+0000 7f2ce66d3640 1 -- 172.21.15.100:0/2576982628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ce0104ae0 msgr2=0x7f2ce01992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:10.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.200+0000 7f2ce66d3640 1 --2- 172.21.15.100:0/2576982628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ce0104ae0 0x7f2ce01992c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2cd400e9f0 tx=0x7f2cd400eeb0 comp rx=0 tx=0).stop 2024-07-26T13:40:10.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.200+0000 7f2ce66d3640 1 -- 172.21.15.100:0/2576982628 shutdown_connections 2024-07-26T13:40:10.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.200+0000 7f2ce66d3640 1 --2- 172.21.15.100:0/2576982628 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f2cc80b3c20 0x7f2cc80b6130 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:10.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.200+0000 7f2ce66d3640 1 --2- 172.21.15.100:0/2576982628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ce0105430 0x7f2ce019e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:10.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.200+0000 7f2ce66d3640 1 --2- 172.21.15.100:0/2576982628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ce0104ae0 0x7f2ce01992c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:10.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.200+0000 7f2ce66d3640 1 --2- 172.21.15.100:0/2576982628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ce0102b20 0x7f2ce0198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:10.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.200+0000 7f2ce66d3640 1 -- 172.21.15.100:0/2576982628 >> 172.21.15.100:0/2576982628 conn(0x7f2ce0079840 msgr2=0x7f2ce0106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:10.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.200+0000 7f2ce66d3640 1 -- 172.21.15.100:0/2576982628 shutdown_connections 2024-07-26T13:40:10.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.201+0000 7f2ce66d3640 1 -- 172.21.15.100:0/2576982628 wait complete. 2024-07-26T13:40:10.213 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-07-26T13:40:10.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.326+0000 7fe1776ec640 1 -- 172.21.15.100:0/1909876966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe170108e20 msgr2=0x7fe170109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:10.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.326+0000 7fe1776ec640 1 --2- 172.21.15.100:0/1909876966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe170108e20 0x7fe170109230 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe160009900 tx=0x7fe16002a3f0 comp rx=0 tx=0).stop 2024-07-26T13:40:10.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.326+0000 7fe1776ec640 1 -- 172.21.15.100:0/1909876966 shutdown_connections 2024-07-26T13:40:10.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.326+0000 7fe1776ec640 1 --2- 172.21.15.100:0/1909876966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe170109770 0x7fe17010ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:10.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.326+0000 7fe1776ec640 1 --2- 172.21.15.100:0/1909876966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe170108e20 0x7fe170109230 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:10.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.326+0000 7fe1776ec640 1 --2- 172.21.15.100:0/1909876966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1701018f0 0x7fe170101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:10.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.326+0000 7fe1776ec640 1 -- 172.21.15.100:0/1909876966 >> 172.21.15.100:0/1909876966 conn(0x7fe1700796e0 msgr2=0x7fe170079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:10.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.326+0000 7fe1776ec640 1 -- 172.21.15.100:0/1909876966 shutdown_connections 2024-07-26T13:40:10.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.326+0000 7fe1776ec640 1 -- 172.21.15.100:0/1909876966 wait complete. 2024-07-26T13:40:10.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.327+0000 7fe1776ec640 1 Processor -- start 2024-07-26T13:40:10.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.327+0000 7fe1776ec640 1 -- start start 2024-07-26T13:40:10.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.327+0000 7fe1776ec640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1701018f0 0x7fe170102880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:10.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.327+0000 7fe1776ec640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe170108e20 0x7fe170102dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:10.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.327+0000 7fe1776ec640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe170109770 0x7fe170103300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:10.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.327+0000 7fe1776ec640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe1701133e0 con 0x7fe170108e20 2024-07-26T13:40:10.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.327+0000 7fe1776ec640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe170113260 con 0x7fe1701018f0 2024-07-26T13:40:10.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.327+0000 7fe1776ec640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe170113560 con 0x7fe170109770 2024-07-26T13:40:10.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.327+0000 7fe174c87640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe170108e20 0x7fe170102dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:10.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.327+0000 7fe174c87640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe170108e20 0x7fe170102dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57690/0 (socket says 172.21.15.100:57690) 2024-07-26T13:40:10.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.327+0000 7fe174c87640 1 -- 172.21.15.100:0/1484446566 learned_addr learned my addr 172.21.15.100:0/1484446566 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:10.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.327+0000 7fe175488640 1 --2- 172.21.15.100:0/1484446566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1701018f0 0x7fe170102880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:10.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.327+0000 7fe175c89640 1 --2- 172.21.15.100:0/1484446566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe170109770 0x7fe170103300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:10.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.328+0000 7fe175488640 1 -- 172.21.15.100:0/1484446566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe170109770 msgr2=0x7fe170103300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:10.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.328+0000 7fe175488640 1 --2- 172.21.15.100:0/1484446566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe170109770 0x7fe170103300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:10.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.328+0000 7fe175488640 1 -- 172.21.15.100:0/1484446566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe170108e20 msgr2=0x7fe170102dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:10.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.328+0000 7fe175488640 1 --2- 172.21.15.100:0/1484446566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe170108e20 0x7fe170102dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:10.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.328+0000 7fe175488640 1 -- 172.21.15.100:0/1484446566 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1701a8090 con 0x7fe1701018f0 2024-07-26T13:40:10.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.328+0000 7fe175488640 1 --2- 172.21.15.100:0/1484446566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1701018f0 0x7fe170102880 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe16c00c970 tx=0x7fe16c00ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:10.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.328+0000 7fe1667fc640 1 -- 172.21.15.100:0/1484446566 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16c007bb0 con 0x7fe1701018f0 2024-07-26T13:40:10.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.328+0000 7fe1776ec640 1 -- 172.21.15.100:0/1484446566 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1701a8380 con 0x7fe1701018f0 2024-07-26T13:40:10.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.328+0000 7fe1667fc640 1 -- 172.21.15.100:0/1484446566 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe16c007d50 con 0x7fe1701018f0 2024-07-26T13:40:10.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.328+0000 7fe1667fc640 1 -- 172.21.15.100:0/1484446566 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16c018780 con 0x7fe1701018f0 2024-07-26T13:40:10.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.328+0000 7fe1776ec640 1 -- 172.21.15.100:0/1484446566 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1701a8920 con 0x7fe1701018f0 2024-07-26T13:40:10.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.329+0000 7fe1776ec640 1 -- 172.21.15.100:0/1484446566 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe170101d00 con 0x7fe1701018f0 2024-07-26T13:40:10.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.331+0000 7fe1667fc640 1 -- 172.21.15.100:0/1484446566 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 23) ==== 151077+0+0 (secure 0 0 0) 0x7fe16c00f450 con 0x7fe1701018f0 2024-07-26T13:40:10.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.332+0000 7fe1667fc640 1 --2- 172.21.15.100:0/1484446566 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe15c0b3c20 0x7fe15c0b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:10.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.332+0000 7fe174c87640 1 --2- 172.21.15.100:0/1484446566 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe15c0b3c20 0x7fe15c0b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:10.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.332+0000 7fe1667fc640 1 -- 172.21.15.100:0/1484446566 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fe16c0e4000 con 0x7fe1701018f0 2024-07-26T13:40:10.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.332+0000 7fe174c87640 1 --2- 172.21.15.100:0/1484446566 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe15c0b3c20 0x7fe15c0b6130 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe160005f70 tx=0x7fe160005e80 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:10.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.335+0000 7fe1667fc640 1 -- 172.21.15.100:0/1484446566 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fe16c014030 con 0x7fe1701018f0 2024-07-26T13:40:10.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.517+0000 7fe1776ec640 1 -- 172.21.15.100:0/1484446566 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe17002e0c0 con 0x7fe1701018f0 2024-07-26T13:40:10.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.517+0000 7fe1667fc640 1 -- 172.21.15.100:0/1484446566 <== mon.2 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) ==== 104+0+11 (secure 0 0 0) 0x7fe16c0ab5e0 con 0x7fe1701018f0 2024-07-26T13:40:10.518 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 32 2024-07-26T13:40:10.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.520+0000 7fe1776ec640 1 -- 172.21.15.100:0/1484446566 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe15c0b3c20 msgr2=0x7fe15c0b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:10.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.520+0000 7fe1776ec640 1 --2- 172.21.15.100:0/1484446566 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe15c0b3c20 0x7fe15c0b6130 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe160005f70 tx=0x7fe160005e80 comp rx=0 tx=0).stop 2024-07-26T13:40:10.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.520+0000 7fe1776ec640 1 -- 172.21.15.100:0/1484446566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1701018f0 msgr2=0x7fe170102880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:10.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.520+0000 7fe1776ec640 1 --2- 172.21.15.100:0/1484446566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1701018f0 0x7fe170102880 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe16c00c970 tx=0x7fe16c00ce30 comp rx=0 tx=0).stop 2024-07-26T13:40:10.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.520+0000 7fe1776ec640 1 -- 172.21.15.100:0/1484446566 shutdown_connections 2024-07-26T13:40:10.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.520+0000 7fe1776ec640 1 --2- 172.21.15.100:0/1484446566 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe15c0b3c20 0x7fe15c0b6130 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:10.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.520+0000 7fe1776ec640 1 --2- 172.21.15.100:0/1484446566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe170109770 0x7fe170103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:10.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.520+0000 7fe1776ec640 1 --2- 172.21.15.100:0/1484446566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe170108e20 0x7fe170102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:10.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.520+0000 7fe1776ec640 1 --2- 172.21.15.100:0/1484446566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1701018f0 0x7fe170102880 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:10.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.520+0000 7fe1776ec640 1 -- 172.21.15.100:0/1484446566 >> 172.21.15.100:0/1484446566 conn(0x7fe1700796e0 msgr2=0x7fe170105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:10.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.520+0000 7fe1776ec640 1 -- 172.21.15.100:0/1484446566 shutdown_connections 2024-07-26T13:40:10.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.520+0000 7fe1776ec640 1 -- 172.21.15.100:0/1484446566 wait complete. 2024-07-26T13:40:10.532 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-07-26T13:40:10.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.646+0000 7f980337e640 1 -- 172.21.15.100:0/661621967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97fc108650 msgr2=0x7f97fc10ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:10.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.646+0000 7f980337e640 1 --2- 172.21.15.100:0/661621967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97fc108650 0x7f97fc10ee80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f97f4009ca0 tx=0x7f97f402a8e0 comp rx=0 tx=0).stop 2024-07-26T13:40:10.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.646+0000 7f980337e640 1 -- 172.21.15.100:0/661621967 shutdown_connections 2024-07-26T13:40:10.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.646+0000 7f980337e640 1 --2- 172.21.15.100:0/661621967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97fc108650 0x7f97fc10ee80 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:10.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.646+0000 7f980337e640 1 --2- 172.21.15.100:0/661621967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97fc107d00 0x7f97fc108110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:10.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.646+0000 7f980337e640 1 --2- 172.21.15.100:0/661621967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97fc1023d0 0x7f97fc1027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:10.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.646+0000 7f980337e640 1 -- 172.21.15.100:0/661621967 >> 172.21.15.100:0/661621967 conn(0x7f97fc0fddf0 msgr2=0x7f97fc100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:10.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.646+0000 7f980337e640 1 -- 172.21.15.100:0/661621967 shutdown_connections 2024-07-26T13:40:10.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.646+0000 7f980337e640 1 -- 172.21.15.100:0/661621967 wait complete. 2024-07-26T13:40:10.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.647+0000 7f980337e640 1 Processor -- start 2024-07-26T13:40:10.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.647+0000 7f980337e640 1 -- start start 2024-07-26T13:40:10.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.647+0000 7f980337e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97fc1023d0 0x7f97fc198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:10.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.647+0000 7f980337e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97fc107d00 0x7f97fc199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:10.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.647+0000 7f980337e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97fc108650 0x7f97fc19e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:10.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.647+0000 7f980337e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc1123d0 con 0x7f97fc1023d0 2024-07-26T13:40:10.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.647+0000 7f980337e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc112250 con 0x7f97fc108650 2024-07-26T13:40:10.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.647+0000 7f980337e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc112550 con 0x7f97fc107d00 2024-07-26T13:40:10.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.647+0000 7f980111a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97fc1023d0 0x7f97fc198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:10.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.647+0000 7f980111a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97fc1023d0 0x7f97fc198ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57710/0 (socket says 172.21.15.100:57710) 2024-07-26T13:40:10.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.647+0000 7f980111a640 1 -- 172.21.15.100:0/1553656014 learned_addr learned my addr 172.21.15.100:0/1553656014 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:10.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.647+0000 7f980191b640 1 --2- 172.21.15.100:0/1553656014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97fc108650 0x7f97fc19e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:10.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.648+0000 7f9800919640 1 --2- 172.21.15.100:0/1553656014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97fc107d00 0x7f97fc199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:10.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.648+0000 7f980111a640 1 -- 172.21.15.100:0/1553656014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97fc107d00 msgr2=0x7f97fc199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:10.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.648+0000 7f980111a640 1 --2- 172.21.15.100:0/1553656014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97fc107d00 0x7f97fc199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:10.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.648+0000 7f980111a640 1 -- 172.21.15.100:0/1553656014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97fc108650 msgr2=0x7f97fc19e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:10.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.648+0000 7f980111a640 1 --2- 172.21.15.100:0/1553656014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97fc108650 0x7f97fc19e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:10.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.648+0000 7f980111a640 1 -- 172.21.15.100:0/1553656014 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97fc19ee00 con 0x7f97fc1023d0 2024-07-26T13:40:10.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.648+0000 7f980191b640 1 --2- 172.21.15.100:0/1553656014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97fc108650 0x7f97fc19e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:40:10.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.648+0000 7f980111a640 1 --2- 172.21.15.100:0/1553656014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97fc1023d0 0x7f97fc198ed0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f97f800b4a0 tx=0x7f97f800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:10.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.648+0000 7f97f27fc640 1 -- 172.21.15.100:0/1553656014 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f8014070 con 0x7f97fc1023d0 2024-07-26T13:40:10.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.648+0000 7f97f27fc640 1 -- 172.21.15.100:0/1553656014 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97f8010ac0 con 0x7f97fc1023d0 2024-07-26T13:40:10.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.648+0000 7f97f27fc640 1 -- 172.21.15.100:0/1553656014 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f8017c30 con 0x7f97fc1023d0 2024-07-26T13:40:10.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.648+0000 7f980337e640 1 -- 172.21.15.100:0/1553656014 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97fc19f0f0 con 0x7f97fc1023d0 2024-07-26T13:40:10.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.648+0000 7f980337e640 1 -- 172.21.15.100:0/1553656014 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97fc1a5880 con 0x7f97fc1023d0 2024-07-26T13:40:10.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.649+0000 7f980337e640 1 -- 172.21.15.100:0/1553656014 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97fc102860 con 0x7f97fc1023d0 2024-07-26T13:40:10.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.650+0000 7f97f27fc640 1 -- 172.21.15.100:0/1553656014 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 151077+0+0 (secure 0 0 0) 0x7f97f8017070 con 0x7f97fc1023d0 2024-07-26T13:40:10.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.651+0000 7f97f27fc640 1 --2- 172.21.15.100:0/1553656014 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f97e80b3c20 0x7f97e80b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:10.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.651+0000 7f97f27fc640 1 -- 172.21.15.100:0/1553656014 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f97f80e34f0 con 0x7f97fc1023d0 2024-07-26T13:40:10.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.651+0000 7f9800919640 1 --2- 172.21.15.100:0/1553656014 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f97e80b3c20 0x7f97e80b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:10.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.652+0000 7f9800919640 1 --2- 172.21.15.100:0/1553656014 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f97e80b3c20 0x7f97e80b6130 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f97fc1077e0 tx=0x7f97ec008040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:10.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.654+0000 7f97f27fc640 1 -- 172.21.15.100:0/1553656014 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f97f80aaa40 con 0x7f97fc1023d0 2024-07-26T13:40:10.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.838+0000 7f980337e640 1 -- 172.21.15.100:0/1553656014 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f97fc02e0c0 con 0x7f97fc1023d0 2024-07-26T13:40:10.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.838+0000 7f97f27fc640 1 -- 172.21.15.100:0/1553656014 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) ==== 100+0+11 (secure 0 0 0) 0x7f97f80aa340 con 0x7f97fc1023d0 2024-07-26T13:40:10.839 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 64 2024-07-26T13:40:10.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.840+0000 7f980337e640 1 -- 172.21.15.100:0/1553656014 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f97e80b3c20 msgr2=0x7f97e80b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:10.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.840+0000 7f980337e640 1 --2- 172.21.15.100:0/1553656014 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f97e80b3c20 0x7f97e80b6130 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f97fc1077e0 tx=0x7f97ec008040 comp rx=0 tx=0).stop 2024-07-26T13:40:10.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.840+0000 7f980337e640 1 -- 172.21.15.100:0/1553656014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97fc1023d0 msgr2=0x7f97fc198ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:10.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.840+0000 7f980337e640 1 --2- 172.21.15.100:0/1553656014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97fc1023d0 0x7f97fc198ed0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f97f800b4a0 tx=0x7f97f800b960 comp rx=0 tx=0).stop 2024-07-26T13:40:10.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.840+0000 7f980337e640 1 -- 172.21.15.100:0/1553656014 shutdown_connections 2024-07-26T13:40:10.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.840+0000 7f980337e640 1 --2- 172.21.15.100:0/1553656014 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f97e80b3c20 0x7f97e80b6130 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:10.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.840+0000 7f980337e640 1 --2- 172.21.15.100:0/1553656014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97fc108650 0x7f97fc19e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:10.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.840+0000 7f980337e640 1 --2- 172.21.15.100:0/1553656014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97fc107d00 0x7f97fc199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:10.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.840+0000 7f980337e640 1 --2- 172.21.15.100:0/1553656014 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97fc1023d0 0x7f97fc198ed0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:10.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.840+0000 7f980337e640 1 -- 172.21.15.100:0/1553656014 >> 172.21.15.100:0/1553656014 conn(0x7f97fc0fddf0 msgr2=0x7f97fc100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:10.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.841+0000 7f980337e640 1 -- 172.21.15.100:0/1553656014 shutdown_connections 2024-07-26T13:40:10.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:10.841+0000 7f980337e640 1 -- 172.21.15.100:0/1553656014 wait complete. 2024-07-26T13:40:10.852 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:10 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/1667860229' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-07-26T13:40:10.852 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:10 smithi100 ceph-mon[32599]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:40:10.852 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:10 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/631457510' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-26T13:40:10.852 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:10 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2576982628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-26T13:40:10.852 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:10 smithi100 ceph-mon[32599]: pgmap v366: 97 pgs: 97 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T13:40:10.852 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:10 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/1484446566' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-07-26T13:40:10.853 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-07-26T13:40:10.854 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:40:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:10 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/1667860229' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-07-26T13:40:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:10 smithi138 ceph-mon[36954]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:40:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:10 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/631457510' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-26T13:40:10.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:10 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/2576982628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-26T13:40:10.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:10 smithi138 ceph-mon[36954]: pgmap v366: 97 pgs: 97 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T13:40:10.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:10 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/1484446566' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-07-26T13:40:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:10 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/1667860229' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-07-26T13:40:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:10 smithi167 ceph-mon[37140]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:40:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:10 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/631457510' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-26T13:40:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:10 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/2576982628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-26T13:40:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:10 smithi167 ceph-mon[37140]: pgmap v366: 97 pgs: 97 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T13:40:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:10 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/1484446566' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-07-26T13:40:11.108 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:40:11.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:11 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/1553656014' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-07-26T13:40:11.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:11 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/1553656014' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-07-26T13:40:12.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:11 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/1553656014' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-07-26T13:40:12.842 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:12 smithi100 ceph-mon[32599]: pgmap v367: 97 pgs: 97 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T13:40:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:12 smithi138 ceph-mon[36954]: pgmap v367: 97 pgs: 97 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T13:40:12.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.931+0000 7f0b20557640 1 -- 172.21.15.100:0/326290559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b181035b0 msgr2=0x7f0b181039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:12.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.931+0000 7f0b20557640 1 --2- 172.21.15.100:0/326290559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b181035b0 0x7f0b181039c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c009900 tx=0x7f0b0c02a410 comp rx=0 tx=0).stop 2024-07-26T13:40:12.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.931+0000 7f0b20557640 1 -- 172.21.15.100:0/326290559 shutdown_connections 2024-07-26T13:40:12.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.931+0000 7f0b20557640 1 --2- 172.21.15.100:0/326290559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b18109840 0x7f0b18110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:12.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.931+0000 7f0b20557640 1 --2- 172.21.15.100:0/326290559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b18108ef0 0x7f0b18109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:12.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.931+0000 7f0b20557640 1 --2- 172.21.15.100:0/326290559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b181035b0 0x7f0b181039c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:12.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.931+0000 7f0b20557640 1 -- 172.21.15.100:0/326290559 >> 172.21.15.100:0/326290559 conn(0x7f0b180ff000 msgr2=0x7f0b18101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:12.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.931+0000 7f0b20557640 1 -- 172.21.15.100:0/326290559 shutdown_connections 2024-07-26T13:40:12.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.931+0000 7f0b20557640 1 -- 172.21.15.100:0/326290559 wait complete. 2024-07-26T13:40:12.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.932+0000 7f0b20557640 1 Processor -- start 2024-07-26T13:40:12.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.932+0000 7f0b20557640 1 -- start start 2024-07-26T13:40:12.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.932+0000 7f0b20557640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b181035b0 0x7f0b18199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:12.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.932+0000 7f0b20557640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b18108ef0 0x7f0b1819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:12.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.932+0000 7f0b20557640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b18109840 0x7f0b1819f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:12.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.932+0000 7f0b20557640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0b181134f0 con 0x7f0b181035b0 2024-07-26T13:40:12.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.932+0000 7f0b20557640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0b18113370 con 0x7f0b18108ef0 2024-07-26T13:40:12.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.932+0000 7f0b20557640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0b18113670 con 0x7f0b18109840 2024-07-26T13:40:12.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.932+0000 7f0b1e2f3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b181035b0 0x7f0b18199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:12.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.932+0000 7f0b1e2f3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b181035b0 0x7f0b18199f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57742/0 (socket says 172.21.15.100:57742) 2024-07-26T13:40:12.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.932+0000 7f0b1e2f3640 1 -- 172.21.15.100:0/824457611 learned_addr learned my addr 172.21.15.100:0/824457611 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:12.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.933+0000 7f0b1eaf4640 1 --2- 172.21.15.100:0/824457611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b18109840 0x7f0b1819f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:12.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.933+0000 7f0b1daf2640 1 --2- 172.21.15.100:0/824457611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b18108ef0 0x7f0b1819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:12.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.933+0000 7f0b1e2f3640 1 -- 172.21.15.100:0/824457611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b18109840 msgr2=0x7f0b1819f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:12.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.933+0000 7f0b1e2f3640 1 --2- 172.21.15.100:0/824457611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b18109840 0x7f0b1819f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:12.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.933+0000 7f0b1e2f3640 1 -- 172.21.15.100:0/824457611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b18108ef0 msgr2=0x7f0b1819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:12.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.933+0000 7f0b1e2f3640 1 --2- 172.21.15.100:0/824457611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b18108ef0 0x7f0b1819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:12.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.933+0000 7f0b1e2f3640 1 -- 172.21.15.100:0/824457611 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b1819fdb0 con 0x7f0b181035b0 2024-07-26T13:40:12.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.933+0000 7f0b1daf2640 1 --2- 172.21.15.100:0/824457611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b18108ef0 0x7f0b1819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:40:12.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.933+0000 7f0b1e2f3640 1 --2- 172.21.15.100:0/824457611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b181035b0 0x7f0b18199f00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c005940 tx=0x7f0b0c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:12.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.933+0000 7f0b0b7fe640 1 -- 172.21.15.100:0/824457611 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c03f030 con 0x7f0b181035b0 2024-07-26T13:40:12.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.933+0000 7f0b0b7fe640 1 -- 172.21.15.100:0/824457611 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b0c003cb0 con 0x7f0b181035b0 2024-07-26T13:40:12.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.933+0000 7f0b0b7fe640 1 -- 172.21.15.100:0/824457611 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c035070 con 0x7f0b181035b0 2024-07-26T13:40:12.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.933+0000 7f0b20557640 1 -- 172.21.15.100:0/824457611 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b181a0040 con 0x7f0b181035b0 2024-07-26T13:40:12.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.933+0000 7f0b20557640 1 -- 172.21.15.100:0/824457611 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b181a0520 con 0x7f0b181035b0 2024-07-26T13:40:12.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.934+0000 7f0b20557640 1 -- 172.21.15.100:0/824457611 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b181039c0 con 0x7f0b181035b0 2024-07-26T13:40:12.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.937+0000 7f0b0b7fe640 1 -- 172.21.15.100:0/824457611 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 151077+0+0 (secure 0 0 0) 0x7f0b0c042030 con 0x7f0b181035b0 2024-07-26T13:40:12.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.937+0000 7f0b0b7fe640 1 --2- 172.21.15.100:0/824457611 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f0b040b3c50 0x7f0b040b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:12.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.937+0000 7f0b1daf2640 1 --2- 172.21.15.100:0/824457611 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f0b040b3c50 0x7f0b040b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:12.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.937+0000 7f0b0b7fe640 1 -- 172.21.15.100:0/824457611 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f0b0c104580 con 0x7f0b181035b0 2024-07-26T13:40:12.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.938+0000 7f0b1daf2640 1 --2- 172.21.15.100:0/824457611 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f0b040b3c50 0x7f0b040b6160 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0b18108b50 tx=0x7f0b14005e50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:12.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:12.940+0000 7f0b0b7fe640 1 -- 172.21.15.100:0/824457611 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f0b0c0cbad0 con 0x7f0b181035b0 2024-07-26T13:40:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:12 smithi167 ceph-mon[37140]: pgmap v367: 97 pgs: 97 active+clean; 643 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T13:40:13.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.149+0000 7f0b20557640 1 -- 172.21.15.100:0/824457611 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b1819b030 con 0x7f0b181035b0 2024-07-26T13:40:13.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.150+0000 7f0b0b7fe640 1 -- 172.21.15.100:0/824457611 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11927 (secure 0 0 0) 0x7f0b0c0cb3d0 con 0x7f0b181035b0 2024-07-26T13:40:13.151 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:40:13.151 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":26,"btime":"2024-07-26T13:40:09:530496+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14790,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/313501409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":313501409},{"type":"v1","addr":"172.21.15.100:6839","nonce":313501409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/1402975506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":1402975506},{"type":"v1","addr":"172.21.15.100:6841","nonce":1402975506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-26T13:37:01.281843+0000","modified":"2024-07-26T13:40:09.187125+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24607,"mds_1":24605,"mds_2":24595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24557":{"gid":24557,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/3290105300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3290105300},{"type":"v1","addr":"172.21.15.138:6835","nonce":3290105300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/3393225483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3393225483},{"type":"v1","addr":"172.21.15.138:6837","nonce":3393225483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24595":{"gid":24595,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6837/3828634468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3828634468},{"type":"v1","addr":"172.21.15.167:6837","nonce":3828634468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.138:6839/1580925471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1580925471},{"type":"v1","addr":"172.21.15.138:6839","nonce":1580925471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/398150514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":398150514},{"type":"v1","addr":"172.21.15.167:6839","nonce":398150514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24607,"qdb_cluster":[24595,24605,24607]},"id":1}]} 2024-07-26T13:40:13.151 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 26 2024-07-26T13:40:13.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.152+0000 7f0b20557640 1 -- 172.21.15.100:0/824457611 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f0b040b3c50 msgr2=0x7f0b040b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:13.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.152+0000 7f0b20557640 1 --2- 172.21.15.100:0/824457611 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f0b040b3c50 0x7f0b040b6160 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0b18108b50 tx=0x7f0b14005e50 comp rx=0 tx=0).stop 2024-07-26T13:40:13.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.152+0000 7f0b20557640 1 -- 172.21.15.100:0/824457611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b181035b0 msgr2=0x7f0b18199f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:13.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.152+0000 7f0b20557640 1 --2- 172.21.15.100:0/824457611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b181035b0 0x7f0b18199f00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c005940 tx=0x7f0b0c0042f0 comp rx=0 tx=0).stop 2024-07-26T13:40:13.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.152+0000 7f0b20557640 1 -- 172.21.15.100:0/824457611 shutdown_connections 2024-07-26T13:40:13.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.152+0000 7f0b20557640 1 --2- 172.21.15.100:0/824457611 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f0b040b3c50 0x7f0b040b6160 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:13.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.152+0000 7f0b20557640 1 --2- 172.21.15.100:0/824457611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b18109840 0x7f0b1819f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:13.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.152+0000 7f0b20557640 1 --2- 172.21.15.100:0/824457611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b18108ef0 0x7f0b1819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:13.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.152+0000 7f0b20557640 1 --2- 172.21.15.100:0/824457611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b181035b0 0x7f0b18199f00 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:13.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.152+0000 7f0b20557640 1 -- 172.21.15.100:0/824457611 >> 172.21.15.100:0/824457611 conn(0x7f0b180ff000 msgr2=0x7f0b18101420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:13.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.153+0000 7f0b20557640 1 -- 172.21.15.100:0/824457611 shutdown_connections 2024-07-26T13:40:13.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.153+0000 7f0b20557640 1 -- 172.21.15.100:0/824457611 wait complete. 2024-07-26T13:40:13.848 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:13 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/824457611' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:40:13.878 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-07-26T13:40:13.878 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-07-26T13:40:13.878 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-26T13:40:13.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:13 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/824457611' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:40:13.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.991+0000 7f8e2f9da640 1 -- 172.21.15.100:0/911466478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e28108630 msgr2=0x7f8e2810ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:13.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.991+0000 7f8e2f9da640 1 --2- 172.21.15.100:0/911466478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e28108630 0x7f8e2810ee60 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8e20009ca0 tx=0x7f8e2002a8c0 comp rx=0 tx=0).stop 2024-07-26T13:40:13.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.991+0000 7f8e2f9da640 1 -- 172.21.15.100:0/911466478 shutdown_connections 2024-07-26T13:40:13.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.991+0000 7f8e2f9da640 1 --2- 172.21.15.100:0/911466478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e28108630 0x7f8e2810ee60 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:13.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.991+0000 7f8e2f9da640 1 --2- 172.21.15.100:0/911466478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e28107ce0 0x7f8e281080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:13.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.991+0000 7f8e2f9da640 1 --2- 172.21.15.100:0/911466478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e281023b0 0x7f8e281027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:13.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.991+0000 7f8e2f9da640 1 -- 172.21.15.100:0/911466478 >> 172.21.15.100:0/911466478 conn(0x7f8e280fddb0 msgr2=0x7f8e28100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:13.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.991+0000 7f8e2f9da640 1 -- 172.21.15.100:0/911466478 shutdown_connections 2024-07-26T13:40:13.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.991+0000 7f8e2f9da640 1 -- 172.21.15.100:0/911466478 wait complete. 2024-07-26T13:40:13.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.992+0000 7f8e2f9da640 1 Processor -- start 2024-07-26T13:40:13.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.992+0000 7f8e2f9da640 1 -- start start 2024-07-26T13:40:13.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.992+0000 7f8e2f9da640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e281023b0 0x7f8e28199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:13.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.992+0000 7f8e2f9da640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e28107ce0 0x7f8e2819a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:13.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.992+0000 7f8e2f9da640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e28108630 0x7f8e2819f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:13.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.992+0000 7f8e2f9da640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8e28112300 con 0x7f8e28108630 2024-07-26T13:40:13.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.992+0000 7f8e2f9da640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8e28112180 con 0x7f8e28107ce0 2024-07-26T13:40:13.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.992+0000 7f8e2f9da640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8e28112480 con 0x7f8e281023b0 2024-07-26T13:40:13.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.992+0000 7f8e2df77640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e28108630 0x7f8e2819f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:13.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.992+0000 7f8e2df77640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e28108630 0x7f8e2819f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57758/0 (socket says 172.21.15.100:57758) 2024-07-26T13:40:13.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.992+0000 7f8e2df77640 1 -- 172.21.15.100:0/4249256534 learned_addr learned my addr 172.21.15.100:0/4249256534 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:13.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.993+0000 7f8e2d776640 1 --2- 172.21.15.100:0/4249256534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e281023b0 0x7f8e28199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:13.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.993+0000 7f8e2df77640 1 -- 172.21.15.100:0/4249256534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e281023b0 msgr2=0x7f8e28199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:13.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.993+0000 7f8e2cf75640 1 --2- 172.21.15.100:0/4249256534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e28107ce0 0x7f8e2819a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:13.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.993+0000 7f8e2df77640 1 --2- 172.21.15.100:0/4249256534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e281023b0 0x7f8e28199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:13.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.993+0000 7f8e2df77640 1 -- 172.21.15.100:0/4249256534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e28107ce0 msgr2=0x7f8e2819a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:13.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.993+0000 7f8e2df77640 1 --2- 172.21.15.100:0/4249256534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e28107ce0 0x7f8e2819a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:13.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.993+0000 7f8e2df77640 1 -- 172.21.15.100:0/4249256534 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e2819fc50 con 0x7f8e28108630 2024-07-26T13:40:13.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.993+0000 7f8e2d776640 1 --2- 172.21.15.100:0/4249256534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e281023b0 0x7f8e28199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:40:13.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.993+0000 7f8e2df77640 1 --2- 172.21.15.100:0/4249256534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e28108630 0x7f8e2819f5e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8e20005cc0 tx=0x7f8e2002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:13.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.993+0000 7f8e1a7fc640 1 -- 172.21.15.100:0/4249256534 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e20003bf0 con 0x7f8e28108630 2024-07-26T13:40:13.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.993+0000 7f8e1a7fc640 1 -- 172.21.15.100:0/4249256534 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e20003d90 con 0x7f8e28108630 2024-07-26T13:40:13.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.993+0000 7f8e2f9da640 1 -- 172.21.15.100:0/4249256534 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e2819fee0 con 0x7f8e28108630 2024-07-26T13:40:13.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.993+0000 7f8e1a7fc640 1 -- 172.21.15.100:0/4249256534 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2002fbc0 con 0x7f8e28108630 2024-07-26T13:40:13.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.993+0000 7f8e2f9da640 1 -- 172.21.15.100:0/4249256534 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e281a6760 con 0x7f8e28108630 2024-07-26T13:40:13.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.994+0000 7f8e2f9da640 1 -- 172.21.15.100:0/4249256534 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e28103620 con 0x7f8e28108630 2024-07-26T13:40:13.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.996+0000 7f8e1a7fc640 1 -- 172.21.15.100:0/4249256534 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 151077+0+0 (secure 0 0 0) 0x7f8e2002fd60 con 0x7f8e28108630 2024-07-26T13:40:13.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.997+0000 7f8e1a7fc640 1 --2- 172.21.15.100:0/4249256534 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f8e140b3c20 0x7f8e140b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:13.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.997+0000 7f8e2d776640 1 --2- 172.21.15.100:0/4249256534 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f8e140b3c20 0x7f8e140b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:13.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.997+0000 7f8e1a7fc640 1 -- 172.21.15.100:0/4249256534 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f8e20105600 con 0x7f8e28108630 2024-07-26T13:40:13.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:13.997+0000 7f8e2d776640 1 --2- 172.21.15.100:0/4249256534 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f8e140b3c20 0x7f8e140b6130 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8e240058c0 tx=0x7f8e24005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:14.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.001+0000 7f8e1a7fc640 1 -- 172.21.15.100:0/4249256534 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f8e200ccaf0 con 0x7f8e28108630 2024-07-26T13:40:14.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:13 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/824457611' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:40:14.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.182+0000 7f8e2f9da640 1 -- 172.21.15.100:0/4249256534 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8e28063230 con 0x7f8e140b3c20 2024-07-26T13:40:14.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.187+0000 7f8e1a7fc640 1 -- 172.21.15.100:0/4249256534 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259060 (secure 0 0 0) 0x7f8e28063230 con 0x7f8e140b3c20 2024-07-26T13:40:14.191 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:40:14.193 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-07-26T13:40:14.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.190+0000 7f8e2f9da640 1 -- 172.21.15.100:0/4249256534 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f8e140b3c20 msgr2=0x7f8e140b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:14.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.190+0000 7f8e2f9da640 1 --2- 172.21.15.100:0/4249256534 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f8e140b3c20 0x7f8e140b6130 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8e240058c0 tx=0x7f8e24005850 comp rx=0 tx=0).stop 2024-07-26T13:40:14.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.190+0000 7f8e2f9da640 1 -- 172.21.15.100:0/4249256534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e28108630 msgr2=0x7f8e2819f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:14.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.190+0000 7f8e2f9da640 1 --2- 172.21.15.100:0/4249256534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e28108630 0x7f8e2819f5e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8e20005cc0 tx=0x7f8e2002f380 comp rx=0 tx=0).stop 2024-07-26T13:40:14.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.190+0000 7f8e2f9da640 1 -- 172.21.15.100:0/4249256534 shutdown_connections 2024-07-26T13:40:14.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.190+0000 7f8e2f9da640 1 --2- 172.21.15.100:0/4249256534 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f8e140b3c20 0x7f8e140b6130 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:14.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.190+0000 7f8e2f9da640 1 --2- 172.21.15.100:0/4249256534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e28108630 0x7f8e2819f5e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:14.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.190+0000 7f8e2f9da640 1 --2- 172.21.15.100:0/4249256534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e28107ce0 0x7f8e2819a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:14.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.190+0000 7f8e2f9da640 1 --2- 172.21.15.100:0/4249256534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e281023b0 0x7f8e28199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:14.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.190+0000 7f8e2f9da640 1 -- 172.21.15.100:0/4249256534 >> 172.21.15.100:0/4249256534 conn(0x7f8e280fddb0 msgr2=0x7f8e2806a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:14.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.190+0000 7f8e2f9da640 1 -- 172.21.15.100:0/4249256534 shutdown_connections 2024-07-26T13:40:14.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.190+0000 7f8e2f9da640 1 -- 172.21.15.100:0/4249256534 wait complete. 2024-07-26T13:40:14.202 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":367,"stamp":"2024-07-26T13:40:12.409975+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658168,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1148,"num_read_kb":674,"num_write":303,"num_write_kb":2275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":2952484,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":745,"ondisk_log_size":745,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372492,"kb_used_data":10860,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569332,"statfs":{"total":1151940427776,"available":1151558995968,"internally_reserved":0,"allocated":11120640,"data_stored":7288360,"data_compressed":177282,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":308,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":35,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003254"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387528+0000","last_change":"2024-07-26T13:36:59.462289+0000","last_active":"2024-07-26T13:38:46.387528+0000","last_peered":"2024-07-26T13:38:46.387528+0000","last_clean":"2024-07-26T13:38:46.387528+0000","last_became_active":"2024-07-26T13:36:59.460311+0000","last_became_peered":"2024-07-26T13:36:59.460311+0000","last_unstale":"2024-07-26T13:38:46.387528+0000","last_undegraded":"2024-07-26T13:38:46.387528+0000","last_fullsized":"2024-07-26T13:38:46.387528+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:04:59.123716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.384825+0000","last_change":"2024-07-26T13:36:59.463218+0000","last_active":"2024-07-26T13:38:46.384825+0000","last_peered":"2024-07-26T13:38:46.384825+0000","last_clean":"2024-07-26T13:38:46.384825+0000","last_became_active":"2024-07-26T13:36:59.462889+0000","last_became_peered":"2024-07-26T13:36:59.462889+0000","last_unstale":"2024-07-26T13:38:46.384825+0000","last_undegraded":"2024-07-26T13:38:46.384825+0000","last_fullsized":"2024-07-26T13:38:46.384825+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:30:46.788669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383503+0000","last_change":"2024-07-26T13:36:59.456113+0000","last_active":"2024-07-26T13:38:46.383503+0000","last_peered":"2024-07-26T13:38:46.383503+0000","last_clean":"2024-07-26T13:38:46.383503+0000","last_became_active":"2024-07-26T13:36:59.455622+0000","last_became_peered":"2024-07-26T13:36:59.455622+0000","last_unstale":"2024-07-26T13:38:46.383503+0000","last_undegraded":"2024-07-26T13:38:46.383503+0000","last_fullsized":"2024-07-26T13:38:46.383503+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:36:26.474799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.368792+0000","last_change":"2024-07-26T13:37:17.555947+0000","last_active":"2024-07-26T13:38:46.368792+0000","last_peered":"2024-07-26T13:38:46.368792+0000","last_clean":"2024-07-26T13:38:46.368792+0000","last_became_active":"2024-07-26T13:37:17.555607+0000","last_became_peered":"2024-07-26T13:37:17.555607+0000","last_unstale":"2024-07-26T13:38:46.368792+0000","last_undegraded":"2024-07-26T13:38:46.368792+0000","last_fullsized":"2024-07-26T13:38:46.368792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:19:59.603225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.372136+0000","last_change":"2024-07-26T13:36:59.474896+0000","last_active":"2024-07-26T13:38:46.372136+0000","last_peered":"2024-07-26T13:38:46.372136+0000","last_clean":"2024-07-26T13:38:46.372136+0000","last_became_active":"2024-07-26T13:36:59.474516+0000","last_became_peered":"2024-07-26T13:36:59.474516+0000","last_unstale":"2024-07-26T13:38:46.372136+0000","last_undegraded":"2024-07-26T13:38:46.372136+0000","last_fullsized":"2024-07-26T13:38:46.372136+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:38:18.994280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.358362+0000","last_change":"2024-07-26T13:37:17.556191+0000","last_active":"2024-07-26T13:38:46.358362+0000","last_peered":"2024-07-26T13:38:46.358362+0000","last_clean":"2024-07-26T13:38:46.358362+0000","last_became_active":"2024-07-26T13:37:17.555715+0000","last_became_peered":"2024-07-26T13:37:17.555715+0000","last_unstale":"2024-07-26T13:38:46.358362+0000","last_undegraded":"2024-07-26T13:38:46.358362+0000","last_fullsized":"2024-07-26T13:38:46.358362+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:03:18.352049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383784+0000","last_change":"2024-07-26T13:36:59.469515+0000","last_active":"2024-07-26T13:38:46.383784+0000","last_peered":"2024-07-26T13:38:46.383784+0000","last_clean":"2024-07-26T13:38:46.383784+0000","last_became_active":"2024-07-26T13:36:59.469106+0000","last_became_peered":"2024-07-26T13:36:59.469106+0000","last_unstale":"2024-07-26T13:38:46.383784+0000","last_undegraded":"2024-07-26T13:38:46.383784+0000","last_fullsized":"2024-07-26T13:38:46.383784+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:10:05.455329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.373036+0000","last_change":"2024-07-26T13:37:17.549365+0000","last_active":"2024-07-26T13:38:46.373036+0000","last_peered":"2024-07-26T13:38:46.373036+0000","last_clean":"2024-07-26T13:38:46.373036+0000","last_became_active":"2024-07-26T13:37:17.549036+0000","last_became_peered":"2024-07-26T13:37:17.549036+0000","last_unstale":"2024-07-26T13:38:46.373036+0000","last_undegraded":"2024-07-26T13:38:46.373036+0000","last_fullsized":"2024-07-26T13:38:46.373036+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:27:43.930981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387690+0000","last_change":"2024-07-26T13:36:59.459306+0000","last_active":"2024-07-26T13:38:46.387690+0000","last_peered":"2024-07-26T13:38:46.387690+0000","last_clean":"2024-07-26T13:38:46.387690+0000","last_became_active":"2024-07-26T13:36:59.458714+0000","last_became_peered":"2024-07-26T13:36:59.458714+0000","last_unstale":"2024-07-26T13:38:46.387690+0000","last_undegraded":"2024-07-26T13:38:46.387690+0000","last_fullsized":"2024-07-26T13:38:46.387690+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:34:18.512454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.376748+0000","last_change":"2024-07-26T13:36:59.466556+0000","last_active":"2024-07-26T13:38:46.376748+0000","last_peered":"2024-07-26T13:38:46.376748+0000","last_clean":"2024-07-26T13:38:46.376748+0000","last_became_active":"2024-07-26T13:36:59.466147+0000","last_became_peered":"2024-07-26T13:36:59.466147+0000","last_unstale":"2024-07-26T13:38:46.376748+0000","last_undegraded":"2024-07-26T13:38:46.376748+0000","last_fullsized":"2024-07-26T13:38:46.376748+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:43:11.359314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387571+0000","last_change":"2024-07-26T13:36:59.473685+0000","last_active":"2024-07-26T13:38:46.387571+0000","last_peered":"2024-07-26T13:38:46.387571+0000","last_clean":"2024-07-26T13:38:46.387571+0000","last_became_active":"2024-07-26T13:36:59.473314+0000","last_became_peered":"2024-07-26T13:36:59.473314+0000","last_unstale":"2024-07-26T13:38:46.387571+0000","last_undegraded":"2024-07-26T13:38:46.387571+0000","last_fullsized":"2024-07-26T13:38:46.387571+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:41:51.612348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.368487+0000","last_change":"2024-07-26T13:36:59.471586+0000","last_active":"2024-07-26T13:38:46.368487+0000","last_peered":"2024-07-26T13:38:46.368487+0000","last_clean":"2024-07-26T13:38:46.368487+0000","last_became_active":"2024-07-26T13:36:59.470995+0000","last_became_peered":"2024-07-26T13:36:59.470995+0000","last_unstale":"2024-07-26T13:38:46.368487+0000","last_undegraded":"2024-07-26T13:38:46.368487+0000","last_fullsized":"2024-07-26T13:38:46.368487+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:54:07.957937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.358125+0000","last_change":"2024-07-26T13:36:59.460018+0000","last_active":"2024-07-26T13:38:46.358125+0000","last_peered":"2024-07-26T13:38:46.358125+0000","last_clean":"2024-07-26T13:38:46.358125+0000","last_became_active":"2024-07-26T13:36:59.459671+0000","last_became_peered":"2024-07-26T13:36:59.459671+0000","last_unstale":"2024-07-26T13:38:46.358125+0000","last_undegraded":"2024-07-26T13:38:46.358125+0000","last_fullsized":"2024-07-26T13:38:46.358125+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:19:33.923360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.374512+0000","last_change":"2024-07-26T13:36:59.470271+0000","last_active":"2024-07-26T13:38:46.374512+0000","last_peered":"2024-07-26T13:38:46.374512+0000","last_clean":"2024-07-26T13:38:46.374512+0000","last_became_active":"2024-07-26T13:36:59.469941+0000","last_became_peered":"2024-07-26T13:36:59.469941+0000","last_unstale":"2024-07-26T13:38:46.374512+0000","last_undegraded":"2024-07-26T13:38:46.374512+0000","last_fullsized":"2024-07-26T13:38:46.374512+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:38:39.329366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383168+0000","last_change":"2024-07-26T13:36:59.460133+0000","last_active":"2024-07-26T13:38:46.383168+0000","last_peered":"2024-07-26T13:38:46.383168+0000","last_clean":"2024-07-26T13:38:46.383168+0000","last_became_active":"2024-07-26T13:36:59.459794+0000","last_became_peered":"2024-07-26T13:36:59.459794+0000","last_unstale":"2024-07-26T13:38:46.383168+0000","last_undegraded":"2024-07-26T13:38:46.383168+0000","last_fullsized":"2024-07-26T13:38:46.383168+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:15:36.024347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.386026+0000","last_change":"2024-07-26T13:36:59.462390+0000","last_active":"2024-07-26T13:38:46.386026+0000","last_peered":"2024-07-26T13:38:46.386026+0000","last_clean":"2024-07-26T13:38:46.386026+0000","last_became_active":"2024-07-26T13:36:59.461968+0000","last_became_peered":"2024-07-26T13:36:59.461968+0000","last_unstale":"2024-07-26T13:38:46.386026+0000","last_undegraded":"2024-07-26T13:38:46.386026+0000","last_fullsized":"2024-07-26T13:38:46.386026+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:49:47.142472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387645+0000","last_change":"2024-07-26T13:36:59.460206+0000","last_active":"2024-07-26T13:38:46.387645+0000","last_peered":"2024-07-26T13:38:46.387645+0000","last_clean":"2024-07-26T13:38:46.387645+0000","last_became_active":"2024-07-26T13:36:59.459811+0000","last_became_peered":"2024-07-26T13:36:59.459811+0000","last_unstale":"2024-07-26T13:38:46.387645+0000","last_undegraded":"2024-07-26T13:38:46.387645+0000","last_fullsized":"2024-07-26T13:38:46.387645+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:46:57.112314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387993+0000","last_change":"2024-07-26T13:36:59.460197+0000","last_active":"2024-07-26T13:38:46.387993+0000","last_peered":"2024-07-26T13:38:46.387993+0000","last_clean":"2024-07-26T13:38:46.387993+0000","last_became_active":"2024-07-26T13:36:59.459686+0000","last_became_peered":"2024-07-26T13:36:59.459686+0000","last_unstale":"2024-07-26T13:38:46.387993+0000","last_undegraded":"2024-07-26T13:38:46.387993+0000","last_fullsized":"2024-07-26T13:38:46.387993+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T13:39:22.407376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.373506+0000","last_change":"2024-07-26T13:36:59.462545+0000","last_active":"2024-07-26T13:38:46.373506+0000","last_peered":"2024-07-26T13:38:46.373506+0000","last_clean":"2024-07-26T13:38:46.373506+0000","last_became_active":"2024-07-26T13:36:59.462115+0000","last_became_peered":"2024-07-26T13:36:59.462115+0000","last_unstale":"2024-07-26T13:38:46.373506+0000","last_undegraded":"2024-07-26T13:38:46.373506+0000","last_fullsized":"2024-07-26T13:38:46.373506+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T21:44:13.715076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.384982+0000","last_change":"2024-07-26T13:36:59.465317+0000","last_active":"2024-07-26T13:38:46.384982+0000","last_peered":"2024-07-26T13:38:46.384982+0000","last_clean":"2024-07-26T13:38:46.384982+0000","last_became_active":"2024-07-26T13:36:59.464943+0000","last_became_peered":"2024-07-26T13:36:59.464943+0000","last_unstale":"2024-07-26T13:38:46.384982+0000","last_undegraded":"2024-07-26T13:38:46.384982+0000","last_fullsized":"2024-07-26T13:38:46.384982+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T13:53:54.160584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.370482+0000","last_change":"2024-07-26T13:36:59.462884+0000","last_active":"2024-07-26T13:38:46.370482+0000","last_peered":"2024-07-26T13:38:46.370482+0000","last_clean":"2024-07-26T13:38:46.370482+0000","last_became_active":"2024-07-26T13:36:59.462564+0000","last_became_peered":"2024-07-26T13:36:59.462564+0000","last_unstale":"2024-07-26T13:38:46.370482+0000","last_undegraded":"2024-07-26T13:38:46.370482+0000","last_fullsized":"2024-07-26T13:38:46.370482+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:50:19.549148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387141+0000","last_change":"2024-07-26T13:36:59.468716+0000","last_active":"2024-07-26T13:38:46.387141+0000","last_peered":"2024-07-26T13:38:46.387141+0000","last_clean":"2024-07-26T13:38:46.387141+0000","last_became_active":"2024-07-26T13:36:59.468048+0000","last_became_peered":"2024-07-26T13:36:59.468048+0000","last_unstale":"2024-07-26T13:38:46.387141+0000","last_undegraded":"2024-07-26T13:38:46.387141+0000","last_fullsized":"2024-07-26T13:38:46.387141+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:51:29.510047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383235+0000","last_change":"2024-07-26T13:36:59.466565+0000","last_active":"2024-07-26T13:38:46.383235+0000","last_peered":"2024-07-26T13:38:46.383235+0000","last_clean":"2024-07-26T13:38:46.383235+0000","last_became_active":"2024-07-26T13:36:59.466235+0000","last_became_peered":"2024-07-26T13:36:59.466235+0000","last_unstale":"2024-07-26T13:38:46.383235+0000","last_undegraded":"2024-07-26T13:38:46.383235+0000","last_fullsized":"2024-07-26T13:38:46.383235+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:28:44.326821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.371918+0000","last_change":"2024-07-26T13:36:59.458753+0000","last_active":"2024-07-26T13:38:46.371918+0000","last_peered":"2024-07-26T13:38:46.371918+0000","last_clean":"2024-07-26T13:38:46.371918+0000","last_became_active":"2024-07-26T13:36:59.458333+0000","last_became_peered":"2024-07-26T13:36:59.458333+0000","last_unstale":"2024-07-26T13:38:46.371918+0000","last_undegraded":"2024-07-26T13:38:46.371918+0000","last_fullsized":"2024-07-26T13:38:46.371918+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:33:54.806212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.369463+0000","last_change":"2024-07-26T13:36:59.468657+0000","last_active":"2024-07-26T13:38:46.369463+0000","last_peered":"2024-07-26T13:38:46.369463+0000","last_clean":"2024-07-26T13:38:46.369463+0000","last_became_active":"2024-07-26T13:36:59.468121+0000","last_became_peered":"2024-07-26T13:36:59.468121+0000","last_unstale":"2024-07-26T13:38:46.369463+0000","last_undegraded":"2024-07-26T13:38:46.369463+0000","last_fullsized":"2024-07-26T13:38:46.369463+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:23:53.959836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.377710+0000","last_change":"2024-07-26T13:36:59.469067+0000","last_active":"2024-07-26T13:38:46.377710+0000","last_peered":"2024-07-26T13:38:46.377710+0000","last_clean":"2024-07-26T13:38:46.377710+0000","last_became_active":"2024-07-26T13:36:59.468706+0000","last_became_peered":"2024-07-26T13:36:59.468706+0000","last_unstale":"2024-07-26T13:38:46.377710+0000","last_undegraded":"2024-07-26T13:38:46.377710+0000","last_fullsized":"2024-07-26T13:38:46.377710+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:21:35.974618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.378393+0000","last_change":"2024-07-26T13:36:59.469807+0000","last_active":"2024-07-26T13:38:46.378393+0000","last_peered":"2024-07-26T13:38:46.378393+0000","last_clean":"2024-07-26T13:38:46.378393+0000","last_became_active":"2024-07-26T13:36:59.469020+0000","last_became_peered":"2024-07-26T13:36:59.469020+0000","last_unstale":"2024-07-26T13:38:46.378393+0000","last_undegraded":"2024-07-26T13:38:46.378393+0000","last_fullsized":"2024-07-26T13:38:46.378393+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:29:57.931598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.373015+0000","last_change":"2024-07-26T13:36:59.465236+0000","last_active":"2024-07-26T13:38:46.373015+0000","last_peered":"2024-07-26T13:38:46.373015+0000","last_clean":"2024-07-26T13:38:46.373015+0000","last_became_active":"2024-07-26T13:36:59.464415+0000","last_became_peered":"2024-07-26T13:36:59.464415+0000","last_unstale":"2024-07-26T13:38:46.373015+0000","last_undegraded":"2024-07-26T13:38:46.373015+0000","last_fullsized":"2024-07-26T13:38:46.373015+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:46:20.373688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.375987+0000","last_change":"2024-07-26T13:36:59.466113+0000","last_active":"2024-07-26T13:38:46.375987+0000","last_peered":"2024-07-26T13:38:46.375987+0000","last_clean":"2024-07-26T13:38:46.375987+0000","last_became_active":"2024-07-26T13:36:59.465743+0000","last_became_peered":"2024-07-26T13:36:59.465743+0000","last_unstale":"2024-07-26T13:38:46.375987+0000","last_undegraded":"2024-07-26T13:38:46.375987+0000","last_fullsized":"2024-07-26T13:38:46.375987+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:14:41.839204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.380451+0000","last_change":"2024-07-26T13:36:59.460672+0000","last_active":"2024-07-26T13:38:46.380451+0000","last_peered":"2024-07-26T13:38:46.380451+0000","last_clean":"2024-07-26T13:38:46.380451+0000","last_became_active":"2024-07-26T13:36:59.460247+0000","last_became_peered":"2024-07-26T13:36:59.460247+0000","last_unstale":"2024-07-26T13:38:46.380451+0000","last_undegraded":"2024-07-26T13:38:46.380451+0000","last_fullsized":"2024-07-26T13:38:46.380451+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:37:51.109933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.378083+0000","last_change":"2024-07-26T13:36:59.464784+0000","last_active":"2024-07-26T13:38:46.378083+0000","last_peered":"2024-07-26T13:38:46.378083+0000","last_clean":"2024-07-26T13:38:46.378083+0000","last_became_active":"2024-07-26T13:36:59.464261+0000","last_became_peered":"2024-07-26T13:36:59.464261+0000","last_unstale":"2024-07-26T13:38:46.378083+0000","last_undegraded":"2024-07-26T13:38:46.378083+0000","last_fullsized":"2024-07-26T13:38:46.378083+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:43:10.134037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.385005+0000","last_change":"2024-07-26T13:36:59.460421+0000","last_active":"2024-07-26T13:38:46.385005+0000","last_peered":"2024-07-26T13:38:46.385005+0000","last_clean":"2024-07-26T13:38:46.385005+0000","last_became_active":"2024-07-26T13:36:59.460010+0000","last_became_peered":"2024-07-26T13:36:59.460010+0000","last_unstale":"2024-07-26T13:38:46.385005+0000","last_undegraded":"2024-07-26T13:38:46.385005+0000","last_fullsized":"2024-07-26T13:38:46.385005+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:16:21.651673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.358312+0000","last_change":"2024-07-26T13:36:59.468826+0000","last_active":"2024-07-26T13:38:46.358312+0000","last_peered":"2024-07-26T13:38:46.358312+0000","last_clean":"2024-07-26T13:38:46.358312+0000","last_became_active":"2024-07-26T13:36:59.468406+0000","last_became_peered":"2024-07-26T13:36:59.468406+0000","last_unstale":"2024-07-26T13:38:46.358312+0000","last_undegraded":"2024-07-26T13:38:46.358312+0000","last_fullsized":"2024-07-26T13:38:46.358312+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:22:10.590545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'7","reported_seq":227,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:40:10.849058+0000","last_change":"2024-07-26T13:37:17.559014+0000","last_active":"2024-07-26T13:40:10.849058+0000","last_peered":"2024-07-26T13:40:10.849058+0000","last_clean":"2024-07-26T13:40:10.849058+0000","last_became_active":"2024-07-26T13:37:17.552325+0000","last_became_peered":"2024-07-26T13:37:17.552325+0000","last_unstale":"2024-07-26T13:40:10.849058+0000","last_undegraded":"2024-07-26T13:40:10.849058+0000","last_fullsized":"2024-07-26T13:40:10.849058+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:26:26.790337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":179,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.379560+0000","last_change":"2024-07-26T13:37:17.562871+0000","last_active":"2024-07-26T13:38:46.379560+0000","last_peered":"2024-07-26T13:38:46.379560+0000","last_clean":"2024-07-26T13:38:46.379560+0000","last_became_active":"2024-07-26T13:37:17.562363+0000","last_became_peered":"2024-07-26T13:37:17.562363+0000","last_unstale":"2024-07-26T13:38:46.379560+0000","last_undegraded":"2024-07-26T13:38:46.379560+0000","last_fullsized":"2024-07-26T13:38:46.379560+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:25:36.172091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"79'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.547762+0000","last_change":"2024-07-26T13:36:56.434778+0000","last_active":"2024-07-26T13:37:17.547762+0000","last_peered":"2024-07-26T13:37:17.547762+0000","last_clean":"2024-07-26T13:37:17.547762+0000","last_became_active":"2024-07-26T13:36:56.434288+0000","last_became_peered":"2024-07-26T13:36:56.434288+0000","last_unstale":"2024-07-26T13:37:17.547762+0000","last_undegraded":"2024-07-26T13:37:17.547762+0000","last_fullsized":"2024-07-26T13:37:17.547762+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:24:22.990358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.565163+0000","last_change":"2024-07-26T13:36:56.424665+0000","last_active":"2024-07-26T13:37:17.565163+0000","last_peered":"2024-07-26T13:37:17.565163+0000","last_clean":"2024-07-26T13:37:17.565163+0000","last_became_active":"2024-07-26T13:36:56.424290+0000","last_became_peered":"2024-07-26T13:36:56.424290+0000","last_unstale":"2024-07-26T13:37:17.565163+0000","last_undegraded":"2024-07-26T13:37:17.565163+0000","last_fullsized":"2024-07-26T13:37:17.565163+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:49:59.520194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383766+0000","last_change":"2024-07-26T13:36:59.454609+0000","last_active":"2024-07-26T13:38:46.383766+0000","last_peered":"2024-07-26T13:38:46.383766+0000","last_clean":"2024-07-26T13:38:46.383766+0000","last_became_active":"2024-07-26T13:36:59.454228+0000","last_became_peered":"2024-07-26T13:36:59.454228+0000","last_unstale":"2024-07-26T13:38:46.383766+0000","last_undegraded":"2024-07-26T13:38:46.383766+0000","last_fullsized":"2024-07-26T13:38:46.383766+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:02:06.244433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.547136+0000","last_change":"2024-07-26T13:36:56.430268+0000","last_active":"2024-07-26T13:37:17.547136+0000","last_peered":"2024-07-26T13:37:17.547136+0000","last_clean":"2024-07-26T13:37:17.547136+0000","last_became_active":"2024-07-26T13:36:56.428583+0000","last_became_peered":"2024-07-26T13:36:56.428583+0000","last_unstale":"2024-07-26T13:37:17.547136+0000","last_undegraded":"2024-07-26T13:37:17.547136+0000","last_fullsized":"2024-07-26T13:37:17.547136+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:07:05.864886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.379724+0000","last_change":"2024-07-26T13:36:59.456188+0000","last_active":"2024-07-26T13:38:46.379724+0000","last_peered":"2024-07-26T13:38:46.379724+0000","last_clean":"2024-07-26T13:38:46.379724+0000","last_became_active":"2024-07-26T13:36:59.455842+0000","last_became_peered":"2024-07-26T13:36:59.455842+0000","last_unstale":"2024-07-26T13:38:46.379724+0000","last_undegraded":"2024-07-26T13:38:46.379724+0000","last_fullsized":"2024-07-26T13:38:46.379724+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:00:19.162239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":202,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:40:06.841828+0000","last_change":"2024-07-26T13:36:56.432818+0000","last_active":"2024-07-26T13:40:06.841828+0000","last_peered":"2024-07-26T13:40:06.841828+0000","last_clean":"2024-07-26T13:40:06.841828+0000","last_became_active":"2024-07-26T13:36:56.432419+0000","last_became_peered":"2024-07-26T13:36:56.432419+0000","last_unstale":"2024-07-26T13:40:06.841828+0000","last_undegraded":"2024-07-26T13:40:06.841828+0000","last_fullsized":"2024-07-26T13:40:06.841828+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:55:51.972470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":171,"num_read_kb":171,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.386929+0000","last_change":"2024-07-26T13:36:59.460033+0000","last_active":"2024-07-26T13:38:46.386929+0000","last_peered":"2024-07-26T13:38:46.386929+0000","last_clean":"2024-07-26T13:38:46.386929+0000","last_became_active":"2024-07-26T13:36:59.459562+0000","last_became_peered":"2024-07-26T13:36:59.459562+0000","last_unstale":"2024-07-26T13:38:46.386929+0000","last_undegraded":"2024-07-26T13:38:46.386929+0000","last_fullsized":"2024-07-26T13:38:46.386929+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:58:14.155973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551399+0000","last_change":"2024-07-26T13:36:56.432563+0000","last_active":"2024-07-26T13:37:17.551399+0000","last_peered":"2024-07-26T13:37:17.551399+0000","last_clean":"2024-07-26T13:37:17.551399+0000","last_became_active":"2024-07-26T13:36:56.431134+0000","last_became_peered":"2024-07-26T13:36:56.431134+0000","last_unstale":"2024-07-26T13:37:17.551399+0000","last_undegraded":"2024-07-26T13:37:17.551399+0000","last_fullsized":"2024-07-26T13:37:17.551399+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:42:00.653345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.377839+0000","last_change":"2024-07-26T13:36:59.469782+0000","last_active":"2024-07-26T13:38:46.377839+0000","last_peered":"2024-07-26T13:38:46.377839+0000","last_clean":"2024-07-26T13:38:46.377839+0000","last_became_active":"2024-07-26T13:36:59.469412+0000","last_became_peered":"2024-07-26T13:36:59.469412+0000","last_unstale":"2024-07-26T13:38:46.377839+0000","last_undegraded":"2024-07-26T13:38:46.377839+0000","last_fullsized":"2024-07-26T13:38:46.377839+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:28:07.741486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.558844+0000","last_change":"2024-07-26T13:36:56.488981+0000","last_active":"2024-07-26T13:37:17.558844+0000","last_peered":"2024-07-26T13:37:17.558844+0000","last_clean":"2024-07-26T13:37:17.558844+0000","last_became_active":"2024-07-26T13:36:56.488480+0000","last_became_peered":"2024-07-26T13:36:56.488480+0000","last_unstale":"2024-07-26T13:37:17.558844+0000","last_undegraded":"2024-07-26T13:37:17.558844+0000","last_fullsized":"2024-07-26T13:37:17.558844+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:41:10.041007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387010+0000","last_change":"2024-07-26T13:36:59.473183+0000","last_active":"2024-07-26T13:38:46.387010+0000","last_peered":"2024-07-26T13:38:46.387010+0000","last_clean":"2024-07-26T13:38:46.387010+0000","last_became_active":"2024-07-26T13:36:59.472814+0000","last_became_peered":"2024-07-26T13:36:59.472814+0000","last_unstale":"2024-07-26T13:38:46.387010+0000","last_undegraded":"2024-07-26T13:38:46.387010+0000","last_fullsized":"2024-07-26T13:38:46.387010+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:57:03.396384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'34","reported_seq":412,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:40:08.116107+0000","last_change":"2024-07-26T13:36:56.435034+0000","last_active":"2024-07-26T13:40:08.116107+0000","last_peered":"2024-07-26T13:40:08.116107+0000","last_clean":"2024-07-26T13:40:08.116107+0000","last_became_active":"2024-07-26T13:36:56.434611+0000","last_became_peered":"2024-07-26T13:36:56.434611+0000","last_unstale":"2024-07-26T13:40:08.116107+0000","last_undegraded":"2024-07-26T13:40:08.116107+0000","last_fullsized":"2024-07-26T13:40:08.116107+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:18:27.272077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62892,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":351,"num_read_kb":236,"num_write":35,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.382422+0000","last_change":"2024-07-26T13:36:59.465831+0000","last_active":"2024-07-26T13:38:46.382422+0000","last_peered":"2024-07-26T13:38:46.382422+0000","last_clean":"2024-07-26T13:38:46.382422+0000","last_became_active":"2024-07-26T13:36:59.465404+0000","last_became_peered":"2024-07-26T13:36:59.465404+0000","last_unstale":"2024-07-26T13:38:46.382422+0000","last_undegraded":"2024-07-26T13:38:46.382422+0000","last_fullsized":"2024-07-26T13:38:46.382422+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:42:53.795926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'6","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.552497+0000","last_change":"2024-07-26T13:36:56.435170+0000","last_active":"2024-07-26T13:37:17.552497+0000","last_peered":"2024-07-26T13:37:17.552497+0000","last_clean":"2024-07-26T13:37:17.552497+0000","last_became_active":"2024-07-26T13:36:56.434715+0000","last_became_peered":"2024-07-26T13:36:56.434715+0000","last_unstale":"2024-07-26T13:37:17.552497+0000","last_undegraded":"2024-07-26T13:37:17.552497+0000","last_fullsized":"2024-07-26T13:37:17.552497+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:05:09.712510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.370726+0000","last_change":"2024-07-26T13:36:59.461926+0000","last_active":"2024-07-26T13:38:46.370726+0000","last_peered":"2024-07-26T13:38:46.370726+0000","last_clean":"2024-07-26T13:38:46.370726+0000","last_became_active":"2024-07-26T13:36:59.461512+0000","last_became_peered":"2024-07-26T13:36:59.461512+0000","last_unstale":"2024-07-26T13:38:46.370726+0000","last_undegraded":"2024-07-26T13:38:46.370726+0000","last_fullsized":"2024-07-26T13:38:46.370726+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:34:24.405663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.546907+0000","last_change":"2024-07-26T13:36:56.430062+0000","last_active":"2024-07-26T13:37:17.546907+0000","last_peered":"2024-07-26T13:37:17.546907+0000","last_clean":"2024-07-26T13:37:17.546907+0000","last_became_active":"2024-07-26T13:36:56.429654+0000","last_became_peered":"2024-07-26T13:36:56.429654+0000","last_unstale":"2024-07-26T13:37:17.546907+0000","last_undegraded":"2024-07-26T13:37:17.546907+0000","last_fullsized":"2024-07-26T13:37:17.546907+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:21:09.514359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.380005+0000","last_change":"2024-07-26T13:36:59.474459+0000","last_active":"2024-07-26T13:38:46.380005+0000","last_peered":"2024-07-26T13:38:46.380005+0000","last_clean":"2024-07-26T13:38:46.380005+0000","last_became_active":"2024-07-26T13:36:59.474061+0000","last_became_peered":"2024-07-26T13:36:59.474061+0000","last_unstale":"2024-07-26T13:38:46.380005+0000","last_undegraded":"2024-07-26T13:38:46.380005+0000","last_fullsized":"2024-07-26T13:38:46.380005+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:37:21.691491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'5","reported_seq":208,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:40:06.841807+0000","last_change":"2024-07-26T13:36:56.432915+0000","last_active":"2024-07-26T13:40:06.841807+0000","last_peered":"2024-07-26T13:40:06.841807+0000","last_clean":"2024-07-26T13:40:06.841807+0000","last_became_active":"2024-07-26T13:36:56.432499+0000","last_became_peered":"2024-07-26T13:36:56.432499+0000","last_unstale":"2024-07-26T13:40:06.841807+0000","last_undegraded":"2024-07-26T13:40:06.841807+0000","last_fullsized":"2024-07-26T13:40:06.841807+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:02:14.306950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":177,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383574+0000","last_change":"2024-07-26T13:36:59.468945+0000","last_active":"2024-07-26T13:38:46.383574+0000","last_peered":"2024-07-26T13:38:46.383574+0000","last_clean":"2024-07-26T13:38:46.383574+0000","last_became_active":"2024-07-26T13:36:59.467970+0000","last_became_peered":"2024-07-26T13:36:59.467970+0000","last_unstale":"2024-07-26T13:38:46.383574+0000","last_undegraded":"2024-07-26T13:38:46.383574+0000","last_fullsized":"2024-07-26T13:38:46.383574+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T13:59:19.514945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'15","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551412+0000","last_change":"2024-07-26T13:36:56.487014+0000","last_active":"2024-07-26T13:37:17.551412+0000","last_peered":"2024-07-26T13:37:17.551412+0000","last_clean":"2024-07-26T13:37:17.551412+0000","last_became_active":"2024-07-26T13:36:56.486470+0000","last_became_peered":"2024-07-26T13:36:56.486470+0000","last_unstale":"2024-07-26T13:37:17.551412+0000","last_undegraded":"2024-07-26T13:37:17.551412+0000","last_fullsized":"2024-07-26T13:37:17.551412+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:07:02.205644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.366501+0000","last_change":"2024-07-26T13:36:59.462465+0000","last_active":"2024-07-26T13:38:46.366501+0000","last_peered":"2024-07-26T13:38:46.366501+0000","last_clean":"2024-07-26T13:38:46.366501+0000","last_became_active":"2024-07-26T13:36:59.462024+0000","last_became_peered":"2024-07-26T13:36:59.462024+0000","last_unstale":"2024-07-26T13:38:46.366501+0000","last_undegraded":"2024-07-26T13:38:46.366501+0000","last_fullsized":"2024-07-26T13:38:46.366501+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:34:14.317762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551992+0000","last_change":"2024-07-26T13:36:56.435007+0000","last_active":"2024-07-26T13:37:17.551992+0000","last_peered":"2024-07-26T13:37:17.551992+0000","last_clean":"2024-07-26T13:37:17.551992+0000","last_became_active":"2024-07-26T13:36:56.434444+0000","last_became_peered":"2024-07-26T13:36:56.434444+0000","last_unstale":"2024-07-26T13:37:17.551992+0000","last_undegraded":"2024-07-26T13:37:17.551992+0000","last_fullsized":"2024-07-26T13:37:17.551992+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:27:51.041349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.384490+0000","last_change":"2024-07-26T13:36:59.464164+0000","last_active":"2024-07-26T13:38:46.384490+0000","last_peered":"2024-07-26T13:38:46.384490+0000","last_clean":"2024-07-26T13:38:46.384490+0000","last_became_active":"2024-07-26T13:36:59.463782+0000","last_became_peered":"2024-07-26T13:36:59.463782+0000","last_unstale":"2024-07-26T13:38:46.384490+0000","last_undegraded":"2024-07-26T13:38:46.384490+0000","last_fullsized":"2024-07-26T13:38:46.384490+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:48:06.481731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.565510+0000","last_change":"2024-07-26T13:36:56.429653+0000","last_active":"2024-07-26T13:37:17.565510+0000","last_peered":"2024-07-26T13:37:17.565510+0000","last_clean":"2024-07-26T13:37:17.565510+0000","last_became_active":"2024-07-26T13:36:56.429060+0000","last_became_peered":"2024-07-26T13:36:56.429060+0000","last_unstale":"2024-07-26T13:37:17.565510+0000","last_undegraded":"2024-07-26T13:37:17.565510+0000","last_fullsized":"2024-07-26T13:37:17.565510+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:41:48.261709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.384430+0000","last_change":"2024-07-26T13:36:59.471560+0000","last_active":"2024-07-26T13:38:46.384430+0000","last_peered":"2024-07-26T13:38:46.384430+0000","last_clean":"2024-07-26T13:38:46.384430+0000","last_became_active":"2024-07-26T13:36:59.471151+0000","last_became_peered":"2024-07-26T13:36:59.471151+0000","last_unstale":"2024-07-26T13:38:46.384430+0000","last_undegraded":"2024-07-26T13:38:46.384430+0000","last_fullsized":"2024-07-26T13:38:46.384430+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:49:20.513308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.548185+0000","last_change":"2024-07-26T13:36:56.430010+0000","last_active":"2024-07-26T13:37:17.548185+0000","last_peered":"2024-07-26T13:37:17.548185+0000","last_clean":"2024-07-26T13:37:17.548185+0000","last_became_active":"2024-07-26T13:36:56.429120+0000","last_became_peered":"2024-07-26T13:36:56.429120+0000","last_unstale":"2024-07-26T13:37:17.548185+0000","last_undegraded":"2024-07-26T13:37:17.548185+0000","last_fullsized":"2024-07-26T13:37:17.548185+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:23:43.321975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383589+0000","last_change":"2024-07-26T13:36:59.467788+0000","last_active":"2024-07-26T13:38:46.383589+0000","last_peered":"2024-07-26T13:38:46.383589+0000","last_clean":"2024-07-26T13:38:46.383589+0000","last_became_active":"2024-07-26T13:36:59.467399+0000","last_became_peered":"2024-07-26T13:36:59.467399+0000","last_unstale":"2024-07-26T13:38:46.383589+0000","last_undegraded":"2024-07-26T13:38:46.383589+0000","last_fullsized":"2024-07-26T13:38:46.383589+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:48:51.442999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'3","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.553308+0000","last_change":"2024-07-26T13:36:56.588633+0000","last_active":"2024-07-26T13:37:17.553308+0000","last_peered":"2024-07-26T13:37:17.553308+0000","last_clean":"2024-07-26T13:37:17.553308+0000","last_became_active":"2024-07-26T13:36:56.588113+0000","last_became_peered":"2024-07-26T13:36:56.588113+0000","last_unstale":"2024-07-26T13:37:17.553308+0000","last_undegraded":"2024-07-26T13:37:17.553308+0000","last_fullsized":"2024-07-26T13:37:17.553308+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:03:55.075052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387531+0000","last_change":"2024-07-26T13:36:59.472854+0000","last_active":"2024-07-26T13:38:46.387531+0000","last_peered":"2024-07-26T13:38:46.387531+0000","last_clean":"2024-07-26T13:38:46.387531+0000","last_became_active":"2024-07-26T13:36:59.472466+0000","last_became_peered":"2024-07-26T13:36:59.472466+0000","last_unstale":"2024-07-26T13:38:46.387531+0000","last_undegraded":"2024-07-26T13:38:46.387531+0000","last_fullsized":"2024-07-26T13:38:46.387531+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:50:57.454149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":103,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550829+0000","last_change":"2024-07-26T13:35:34.839121+0000","last_active":"2024-07-26T13:37:17.550829+0000","last_peered":"2024-07-26T13:37:17.550829+0000","last_clean":"2024-07-26T13:37:17.550829+0000","last_became_active":"2024-07-26T13:35:34.833465+0000","last_became_peered":"2024-07-26T13:35:34.833465+0000","last_unstale":"2024-07-26T13:37:17.550829+0000","last_undegraded":"2024-07-26T13:37:17.550829+0000","last_fullsized":"2024-07-26T13:37:17.550829+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:32:23.285428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:32:23.285428+0000","last_clean_scrub_stamp":"2024-07-26T13:32:23.285428+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:29:31.650487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'3","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.561459+0000","last_change":"2024-07-26T13:36:56.428648+0000","last_active":"2024-07-26T13:37:17.561459+0000","last_peered":"2024-07-26T13:37:17.561459+0000","last_clean":"2024-07-26T13:37:17.561459+0000","last_became_active":"2024-07-26T13:36:56.427665+0000","last_became_peered":"2024-07-26T13:36:56.427665+0000","last_unstale":"2024-07-26T13:37:17.561459+0000","last_undegraded":"2024-07-26T13:37:17.561459+0000","last_fullsized":"2024-07-26T13:37:17.561459+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:41:39.371687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.376696+0000","last_change":"2024-07-26T13:36:59.460625+0000","last_active":"2024-07-26T13:38:46.376696+0000","last_peered":"2024-07-26T13:38:46.376696+0000","last_clean":"2024-07-26T13:38:46.376696+0000","last_became_active":"2024-07-26T13:36:59.458658+0000","last_became_peered":"2024-07-26T13:36:59.458658+0000","last_unstale":"2024-07-26T13:38:46.376696+0000","last_undegraded":"2024-07-26T13:38:46.376696+0000","last_fullsized":"2024-07-26T13:38:46.376696+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:46:42.812402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'3","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550796+0000","last_change":"2024-07-26T13:36:56.432617+0000","last_active":"2024-07-26T13:37:17.550796+0000","last_peered":"2024-07-26T13:37:17.550796+0000","last_clean":"2024-07-26T13:37:17.550796+0000","last_became_active":"2024-07-26T13:36:56.431519+0000","last_became_peered":"2024-07-26T13:36:56.431519+0000","last_unstale":"2024-07-26T13:37:17.550796+0000","last_undegraded":"2024-07-26T13:37:17.550796+0000","last_fullsized":"2024-07-26T13:37:17.550796+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:32:21.120076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.364664+0000","last_change":"2024-07-26T13:36:59.464084+0000","last_active":"2024-07-26T13:38:46.364664+0000","last_peered":"2024-07-26T13:38:46.364664+0000","last_clean":"2024-07-26T13:38:46.364664+0000","last_became_active":"2024-07-26T13:36:59.463755+0000","last_became_peered":"2024-07-26T13:36:59.463755+0000","last_unstale":"2024-07-26T13:38:46.364664+0000","last_undegraded":"2024-07-26T13:38:46.364664+0000","last_fullsized":"2024-07-26T13:38:46.364664+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:53:20.034188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":209,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:40:10.847516+0000","last_change":"2024-07-26T13:36:56.486071+0000","last_active":"2024-07-26T13:40:10.847516+0000","last_peered":"2024-07-26T13:40:10.847516+0000","last_clean":"2024-07-26T13:40:10.847516+0000","last_became_active":"2024-07-26T13:36:56.485672+0000","last_became_peered":"2024-07-26T13:36:56.485672+0000","last_unstale":"2024-07-26T13:40:10.847516+0000","last_undegraded":"2024-07-26T13:40:10.847516+0000","last_fullsized":"2024-07-26T13:40:10.847516+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T21:29:45.666651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":176,"num_read_kb":176,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.372811+0000","last_change":"2024-07-26T13:37:17.566555+0000","last_active":"2024-07-26T13:38:46.372811+0000","last_peered":"2024-07-26T13:38:46.372811+0000","last_clean":"2024-07-26T13:38:46.372811+0000","last_became_active":"2024-07-26T13:37:17.566040+0000","last_became_peered":"2024-07-26T13:37:17.566040+0000","last_unstale":"2024-07-26T13:38:46.372811+0000","last_undegraded":"2024-07-26T13:38:46.372811+0000","last_fullsized":"2024-07-26T13:38:46.372811+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:48:33.078890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550989+0000","last_change":"2024-07-26T13:36:56.431811+0000","last_active":"2024-07-26T13:37:17.550989+0000","last_peered":"2024-07-26T13:37:17.550989+0000","last_clean":"2024-07-26T13:37:17.550989+0000","last_became_active":"2024-07-26T13:36:56.431223+0000","last_became_peered":"2024-07-26T13:36:56.431223+0000","last_unstale":"2024-07-26T13:37:17.550989+0000","last_undegraded":"2024-07-26T13:37:17.550989+0000","last_fullsized":"2024-07-26T13:37:17.550989+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:08:00.334515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.363881+0000","last_change":"2024-07-26T13:36:59.463972+0000","last_active":"2024-07-26T13:38:46.363881+0000","last_peered":"2024-07-26T13:38:46.363881+0000","last_clean":"2024-07-26T13:38:46.363881+0000","last_became_active":"2024-07-26T13:36:59.463496+0000","last_became_peered":"2024-07-26T13:36:59.463496+0000","last_unstale":"2024-07-26T13:38:46.363881+0000","last_undegraded":"2024-07-26T13:38:46.363881+0000","last_fullsized":"2024-07-26T13:38:46.363881+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:02:06.215372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.545780+0000","last_change":"2024-07-26T13:36:56.486681+0000","last_active":"2024-07-26T13:37:17.545780+0000","last_peered":"2024-07-26T13:37:17.545780+0000","last_clean":"2024-07-26T13:37:17.545780+0000","last_became_active":"2024-07-26T13:36:56.486328+0000","last_became_peered":"2024-07-26T13:36:56.486328+0000","last_unstale":"2024-07-26T13:37:17.545780+0000","last_undegraded":"2024-07-26T13:37:17.545780+0000","last_fullsized":"2024-07-26T13:37:17.545780+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:10:36.557707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.355361+0000","last_change":"2024-07-26T13:36:59.451415+0000","last_active":"2024-07-26T13:38:46.355361+0000","last_peered":"2024-07-26T13:38:46.355361+0000","last_clean":"2024-07-26T13:38:46.355361+0000","last_became_active":"2024-07-26T13:36:59.450992+0000","last_became_peered":"2024-07-26T13:36:59.450992+0000","last_unstale":"2024-07-26T13:38:46.355361+0000","last_undegraded":"2024-07-26T13:38:46.355361+0000","last_fullsized":"2024-07-26T13:38:46.355361+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:29:11.303916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.557777+0000","last_change":"2024-07-26T13:37:17.555643+0000","last_active":"2024-07-26T13:37:17.557777+0000","last_peered":"2024-07-26T13:37:17.557777+0000","last_clean":"2024-07-26T13:37:17.557777+0000","last_became_active":"2024-07-26T13:37:17.549756+0000","last_became_peered":"2024-07-26T13:37:17.549756+0000","last_unstale":"2024-07-26T13:37:17.557777+0000","last_undegraded":"2024-07-26T13:37:17.557777+0000","last_fullsized":"2024-07-26T13:37:17.557777+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:20:16.911286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.368534+0000","last_change":"2024-07-26T13:36:59.465538+0000","last_active":"2024-07-26T13:38:46.368534+0000","last_peered":"2024-07-26T13:38:46.368534+0000","last_clean":"2024-07-26T13:38:46.368534+0000","last_became_active":"2024-07-26T13:36:59.464886+0000","last_became_peered":"2024-07-26T13:36:59.464886+0000","last_unstale":"2024-07-26T13:38:46.368534+0000","last_undegraded":"2024-07-26T13:38:46.368534+0000","last_fullsized":"2024-07-26T13:38:46.368534+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:19:12.273116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'13","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:39:08.749398+0000","last_change":"2024-07-26T13:36:56.428355+0000","last_active":"2024-07-26T13:39:08.749398+0000","last_peered":"2024-07-26T13:39:08.749398+0000","last_clean":"2024-07-26T13:39:08.749398+0000","last_became_active":"2024-07-26T13:36:56.427213+0000","last_became_peered":"2024-07-26T13:36:56.427213+0000","last_unstale":"2024-07-26T13:39:08.749398+0000","last_undegraded":"2024-07-26T13:39:08.749398+0000","last_fullsized":"2024-07-26T13:39:08.749398+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:48:09.730067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.378498+0000","last_change":"2024-07-26T13:36:59.459378+0000","last_active":"2024-07-26T13:38:46.378498+0000","last_peered":"2024-07-26T13:38:46.378498+0000","last_clean":"2024-07-26T13:38:46.378498+0000","last_became_active":"2024-07-26T13:36:59.458799+0000","last_became_peered":"2024-07-26T13:36:59.458799+0000","last_unstale":"2024-07-26T13:38:46.378498+0000","last_undegraded":"2024-07-26T13:38:46.378498+0000","last_fullsized":"2024-07-26T13:38:46.378498+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:45:19.587800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.561441+0000","last_change":"2024-07-26T13:36:56.588819+0000","last_active":"2024-07-26T13:37:17.561441+0000","last_peered":"2024-07-26T13:37:17.561441+0000","last_clean":"2024-07-26T13:37:17.561441+0000","last_became_active":"2024-07-26T13:36:56.588430+0000","last_became_peered":"2024-07-26T13:36:56.588430+0000","last_unstale":"2024-07-26T13:37:17.561441+0000","last_undegraded":"2024-07-26T13:37:17.561441+0000","last_fullsized":"2024-07-26T13:37:17.561441+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:07:21.387618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.380460+0000","last_change":"2024-07-26T13:37:17.562040+0000","last_active":"2024-07-26T13:38:46.380460+0000","last_peered":"2024-07-26T13:38:46.380460+0000","last_clean":"2024-07-26T13:38:46.380460+0000","last_became_active":"2024-07-26T13:37:17.561542+0000","last_became_peered":"2024-07-26T13:37:17.561542+0000","last_unstale":"2024-07-26T13:38:46.380460+0000","last_undegraded":"2024-07-26T13:38:46.380460+0000","last_fullsized":"2024-07-26T13:38:46.380460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:21:08.161954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.544169+0000","last_change":"2024-07-26T13:36:56.489106+0000","last_active":"2024-07-26T13:37:17.544169+0000","last_peered":"2024-07-26T13:37:17.544169+0000","last_clean":"2024-07-26T13:37:17.544169+0000","last_became_active":"2024-07-26T13:36:56.488706+0000","last_became_peered":"2024-07-26T13:36:56.488706+0000","last_unstale":"2024-07-26T13:37:17.544169+0000","last_undegraded":"2024-07-26T13:37:17.544169+0000","last_fullsized":"2024-07-26T13:37:17.544169+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:00:59.204434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.343705+0000","last_change":"2024-07-26T13:36:59.470412+0000","last_active":"2024-07-26T13:38:46.343705+0000","last_peered":"2024-07-26T13:38:46.343705+0000","last_clean":"2024-07-26T13:38:46.343705+0000","last_became_active":"2024-07-26T13:36:59.470010+0000","last_became_peered":"2024-07-26T13:36:59.470010+0000","last_unstale":"2024-07-26T13:38:46.343705+0000","last_undegraded":"2024-07-26T13:38:46.343705+0000","last_fullsized":"2024-07-26T13:38:46.343705+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:50:57.987600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.543528+0000","last_change":"2024-07-26T13:36:56.428671+0000","last_active":"2024-07-26T13:37:17.543528+0000","last_peered":"2024-07-26T13:37:17.543528+0000","last_clean":"2024-07-26T13:37:17.543528+0000","last_became_active":"2024-07-26T13:36:56.428301+0000","last_became_peered":"2024-07-26T13:36:56.428301+0000","last_unstale":"2024-07-26T13:37:17.543528+0000","last_undegraded":"2024-07-26T13:37:17.543528+0000","last_fullsized":"2024-07-26T13:37:17.543528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:59:19.593664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.385021+0000","last_change":"2024-07-26T13:36:59.458627+0000","last_active":"2024-07-26T13:38:46.385021+0000","last_peered":"2024-07-26T13:38:46.385021+0000","last_clean":"2024-07-26T13:38:46.385021+0000","last_became_active":"2024-07-26T13:36:59.458199+0000","last_became_peered":"2024-07-26T13:36:59.458199+0000","last_unstale":"2024-07-26T13:38:46.385021+0000","last_undegraded":"2024-07-26T13:38:46.385021+0000","last_fullsized":"2024-07-26T13:38:46.385021+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:54:41.833223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551540+0000","last_change":"2024-07-26T13:36:56.431548+0000","last_active":"2024-07-26T13:37:17.551540+0000","last_peered":"2024-07-26T13:37:17.551540+0000","last_clean":"2024-07-26T13:37:17.551540+0000","last_became_active":"2024-07-26T13:36:56.430989+0000","last_became_peered":"2024-07-26T13:36:56.430989+0000","last_unstale":"2024-07-26T13:37:17.551540+0000","last_undegraded":"2024-07-26T13:37:17.551540+0000","last_fullsized":"2024-07-26T13:37:17.551540+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:46:51.655324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.375762+0000","last_change":"2024-07-26T13:36:59.458290+0000","last_active":"2024-07-26T13:38:46.375762+0000","last_peered":"2024-07-26T13:38:46.375762+0000","last_clean":"2024-07-26T13:38:46.375762+0000","last_became_active":"2024-07-26T13:36:59.457942+0000","last_became_peered":"2024-07-26T13:36:59.457942+0000","last_unstale":"2024-07-26T13:38:46.375762+0000","last_undegraded":"2024-07-26T13:38:46.375762+0000","last_fullsized":"2024-07-26T13:38:46.375762+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:51:58.797441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":46,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.568059+0000","last_change":"2024-07-26T13:37:17.567624+0000","last_active":"2024-07-26T13:37:17.568059+0000","last_peered":"2024-07-26T13:37:17.568059+0000","last_clean":"2024-07-26T13:37:17.568059+0000","last_became_active":"2024-07-26T13:37:17.562455+0000","last_became_peered":"2024-07-26T13:37:17.562455+0000","last_unstale":"2024-07-26T13:37:17.568059+0000","last_undegraded":"2024-07-26T13:37:17.568059+0000","last_fullsized":"2024-07-26T13:37:17.568059+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:50:36.418750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387835+0000","last_change":"2024-07-26T13:36:59.474034+0000","last_active":"2024-07-26T13:38:46.387835+0000","last_peered":"2024-07-26T13:38:46.387835+0000","last_clean":"2024-07-26T13:38:46.387835+0000","last_became_active":"2024-07-26T13:36:59.472358+0000","last_became_peered":"2024-07-26T13:36:59.472358+0000","last_unstale":"2024-07-26T13:38:46.387835+0000","last_undegraded":"2024-07-26T13:38:46.387835+0000","last_fullsized":"2024-07-26T13:38:46.387835+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:36:44.565413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.368595+0000","last_change":"2024-07-26T13:36:59.465546+0000","last_active":"2024-07-26T13:38:46.368595+0000","last_peered":"2024-07-26T13:38:46.368595+0000","last_clean":"2024-07-26T13:38:46.368595+0000","last_became_active":"2024-07-26T13:36:59.465137+0000","last_became_peered":"2024-07-26T13:36:59.465137+0000","last_unstale":"2024-07-26T13:38:46.368595+0000","last_undegraded":"2024-07-26T13:38:46.368595+0000","last_fullsized":"2024-07-26T13:38:46.368595+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:35:13.591354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.562664+0000","last_change":"2024-07-26T13:37:17.562012+0000","last_active":"2024-07-26T13:37:17.562664+0000","last_peered":"2024-07-26T13:37:17.562664+0000","last_clean":"2024-07-26T13:37:17.562664+0000","last_became_active":"2024-07-26T13:37:17.550218+0000","last_became_peered":"2024-07-26T13:37:17.550218+0000","last_unstale":"2024-07-26T13:37:17.562664+0000","last_undegraded":"2024-07-26T13:37:17.562664+0000","last_fullsized":"2024-07-26T13:37:17.562664+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:46:16.559533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.375095+0000","last_change":"2024-07-26T13:36:59.456896+0000","last_active":"2024-07-26T13:38:46.375095+0000","last_peered":"2024-07-26T13:38:46.375095+0000","last_clean":"2024-07-26T13:38:46.375095+0000","last_became_active":"2024-07-26T13:36:59.456530+0000","last_became_peered":"2024-07-26T13:36:59.456530+0000","last_unstale":"2024-07-26T13:38:46.375095+0000","last_undegraded":"2024-07-26T13:38:46.375095+0000","last_fullsized":"2024-07-26T13:38:46.375095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:59:56.512501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'4","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.553596+0000","last_change":"2024-07-26T13:36:56.433119+0000","last_active":"2024-07-26T13:37:17.553596+0000","last_peered":"2024-07-26T13:37:17.553596+0000","last_clean":"2024-07-26T13:37:17.553596+0000","last_became_active":"2024-07-26T13:36:56.432765+0000","last_became_peered":"2024-07-26T13:36:56.432765+0000","last_unstale":"2024-07-26T13:37:17.553596+0000","last_undegraded":"2024-07-26T13:37:17.553596+0000","last_fullsized":"2024-07-26T13:37:17.553596+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:31:29.197106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.382051+0000","last_change":"2024-07-26T13:36:59.462125+0000","last_active":"2024-07-26T13:38:46.382051+0000","last_peered":"2024-07-26T13:38:46.382051+0000","last_clean":"2024-07-26T13:38:46.382051+0000","last_became_active":"2024-07-26T13:36:59.461796+0000","last_became_peered":"2024-07-26T13:36:59.461796+0000","last_unstale":"2024-07-26T13:38:46.382051+0000","last_undegraded":"2024-07-26T13:38:46.382051+0000","last_fullsized":"2024-07-26T13:38:46.382051+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:44:10.329046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.543646+0000","last_change":"2024-07-26T13:36:56.487251+0000","last_active":"2024-07-26T13:37:17.543646+0000","last_peered":"2024-07-26T13:37:17.543646+0000","last_clean":"2024-07-26T13:37:17.543646+0000","last_became_active":"2024-07-26T13:36:56.486918+0000","last_became_peered":"2024-07-26T13:36:56.486918+0000","last_unstale":"2024-07-26T13:37:17.543646+0000","last_undegraded":"2024-07-26T13:37:17.543646+0000","last_fullsized":"2024-07-26T13:37:17.543646+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:41:31.248895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.380839+0000","last_change":"2024-07-26T13:36:59.470810+0000","last_active":"2024-07-26T13:38:46.380839+0000","last_peered":"2024-07-26T13:38:46.380839+0000","last_clean":"2024-07-26T13:38:46.380839+0000","last_became_active":"2024-07-26T13:36:59.467371+0000","last_became_peered":"2024-07-26T13:36:59.467371+0000","last_unstale":"2024-07-26T13:38:46.380839+0000","last_undegraded":"2024-07-26T13:38:46.380839+0000","last_fullsized":"2024-07-26T13:38:46.380839+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:23:48.572933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'9","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.566327+0000","last_change":"2024-07-26T13:36:56.433598+0000","last_active":"2024-07-26T13:37:17.566327+0000","last_peered":"2024-07-26T13:37:17.566327+0000","last_clean":"2024-07-26T13:37:17.566327+0000","last_became_active":"2024-07-26T13:36:56.433325+0000","last_became_peered":"2024-07-26T13:36:56.433325+0000","last_unstale":"2024-07-26T13:37:17.566327+0000","last_undegraded":"2024-07-26T13:37:17.566327+0000","last_fullsized":"2024-07-26T13:37:17.566327+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:17:33.651765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67566,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1067,"num_read_kb":605,"num_write":178,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":644,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":434176,"data_stored":207763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247480,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678070,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":443451,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874299,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":1098334,"data_compressed":41637,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037824,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":897024,"data_stored":443665,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234052,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":443563,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430281,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":1036188,"data_compressed":41637,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593807,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":443054,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790036,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":1035340,"data_compressed":41637,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953560,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":888832,"data_stored":444023,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149788,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":443365,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378721,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":506159,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542246,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":506893,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705770,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":444325,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34195,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34195,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":69419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-26T13:40:14.207 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-26T13:40:14.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.352+0000 7f9bf8d7e640 1 -- 172.21.15.100:0/2161113087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bf4107d20 msgr2=0x7f9bf4108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:14.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.352+0000 7f9bf8d7e640 1 --2- 172.21.15.100:0/2161113087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bf4107d20 0x7f9bf4108130 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9be4009900 tx=0x7f9be402a410 comp rx=0 tx=0).stop 2024-07-26T13:40:14.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.352+0000 7f9bf8d7e640 1 -- 172.21.15.100:0/2161113087 shutdown_connections 2024-07-26T13:40:14.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.352+0000 7f9bf8d7e640 1 --2- 172.21.15.100:0/2161113087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bf4108670 0x7f9bf410eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:14.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.352+0000 7f9bf8d7e640 1 --2- 172.21.15.100:0/2161113087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bf4107d20 0x7f9bf4108130 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:14.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.352+0000 7f9bf8d7e640 1 --2- 172.21.15.100:0/2161113087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bf41023f0 0x7f9bf4102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:14.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.352+0000 7f9bf8d7e640 1 -- 172.21.15.100:0/2161113087 >> 172.21.15.100:0/2161113087 conn(0x7f9bf407a0a0 msgr2=0x7f9bf407a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:14.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.352+0000 7f9bf8d7e640 1 -- 172.21.15.100:0/2161113087 shutdown_connections 2024-07-26T13:40:14.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.352+0000 7f9bf8d7e640 1 -- 172.21.15.100:0/2161113087 wait complete. 2024-07-26T13:40:14.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.353+0000 7f9bf8d7e640 1 Processor -- start 2024-07-26T13:40:14.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.353+0000 7f9bf8d7e640 1 -- start start 2024-07-26T13:40:14.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.353+0000 7f9bf8d7e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bf41023f0 0x7f9bf4199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:14.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.353+0000 7f9bf8d7e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bf4107d20 0x7f9bf419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:14.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.353+0000 7f9bf8d7e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bf4108670 0x7f9bf419f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:14.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.353+0000 7f9bf8d7e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf4113450 con 0x7f9bf4108670 2024-07-26T13:40:14.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.353+0000 7f9bf8d7e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf41132d0 con 0x7f9bf41023f0 2024-07-26T13:40:14.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.353+0000 7f9bf8d7e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf41135d0 con 0x7f9bf4107d20 2024-07-26T13:40:14.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.353+0000 7f9bf2d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bf4108670 0x7f9bf419f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:14.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.353+0000 7f9bf2d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bf4108670 0x7f9bf419f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57768/0 (socket says 172.21.15.100:57768) 2024-07-26T13:40:14.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.353+0000 7f9bf2d9d640 1 -- 172.21.15.100:0/66014604 learned_addr learned my addr 172.21.15.100:0/66014604 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:14.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.353+0000 7f9bf1d9b640 1 --2- 172.21.15.100:0/66014604 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bf4107d20 0x7f9bf419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:14.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.353+0000 7f9bf259c640 1 --2- 172.21.15.100:0/66014604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bf41023f0 0x7f9bf4199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:14.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.354+0000 7f9bf2d9d640 1 -- 172.21.15.100:0/66014604 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bf4107d20 msgr2=0x7f9bf419a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:14.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.354+0000 7f9bf2d9d640 1 --2- 172.21.15.100:0/66014604 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bf4107d20 0x7f9bf419a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:14.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.354+0000 7f9bf2d9d640 1 -- 172.21.15.100:0/66014604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bf41023f0 msgr2=0x7f9bf4199e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:14.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.354+0000 7f9bf2d9d640 1 --2- 172.21.15.100:0/66014604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bf41023f0 0x7f9bf4199e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:14.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.354+0000 7f9bf2d9d640 1 -- 172.21.15.100:0/66014604 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf419fd80 con 0x7f9bf4108670 2024-07-26T13:40:14.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.354+0000 7f9bf2d9d640 1 --2- 172.21.15.100:0/66014604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bf4108670 0x7f9bf419f680 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00eb10 tx=0x7f9bec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:14.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.354+0000 7f9beb7fe640 1 -- 172.21.15.100:0/66014604 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec004b20 con 0x7f9bf4108670 2024-07-26T13:40:14.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.354+0000 7f9bf8d7e640 1 -- 172.21.15.100:0/66014604 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bf41a0070 con 0x7f9bf4108670 2024-07-26T13:40:14.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.354+0000 7f9beb7fe640 1 -- 172.21.15.100:0/66014604 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bec004cc0 con 0x7f9bf4108670 2024-07-26T13:40:14.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.354+0000 7f9beb7fe640 1 -- 172.21.15.100:0/66014604 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec010670 con 0x7f9bf4108670 2024-07-26T13:40:14.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.354+0000 7f9bf8d7e640 1 -- 172.21.15.100:0/66014604 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bf41a6800 con 0x7f9bf4108670 2024-07-26T13:40:14.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.355+0000 7f9bf8d7e640 1 -- 172.21.15.100:0/66014604 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bf4102800 con 0x7f9bf4108670 2024-07-26T13:40:14.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.356+0000 7f9beb7fe640 1 -- 172.21.15.100:0/66014604 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 151077+0+0 (secure 0 0 0) 0x7f9bec005000 con 0x7f9bf4108670 2024-07-26T13:40:14.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.357+0000 7f9beb7fe640 1 --2- 172.21.15.100:0/66014604 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9be00b3c20 0x7f9be00b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:14.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.357+0000 7f9bf259c640 1 --2- 172.21.15.100:0/66014604 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9be00b3c20 0x7f9be00b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:14.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.357+0000 7f9beb7fe640 1 -- 172.21.15.100:0/66014604 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f9bec0e45a0 con 0x7f9bf4108670 2024-07-26T13:40:14.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.358+0000 7f9bf259c640 1 --2- 172.21.15.100:0/66014604 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9be00b3c20 0x7f9be00b6130 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc009b60 tx=0x7f9bdc005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:14.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.360+0000 7f9beb7fe640 1 -- 172.21.15.100:0/66014604 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f9bec055600 con 0x7f9bf4108670 2024-07-26T13:40:14.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.541+0000 7f9bf8d7e640 1 -- 172.21.15.100:0/66014604 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9bf4063230 con 0x7f9be00b3c20 2024-07-26T13:40:14.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.546+0000 7f9beb7fe640 1 -- 172.21.15.100:0/66014604 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259061 (secure 0 0 0) 0x7f9bf4063230 con 0x7f9be00b3c20 2024-07-26T13:40:14.549 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:40:14.551 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-07-26T13:40:14.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.549+0000 7f9bf8d7e640 1 -- 172.21.15.100:0/66014604 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9be00b3c20 msgr2=0x7f9be00b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:14.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.549+0000 7f9bf8d7e640 1 --2- 172.21.15.100:0/66014604 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9be00b3c20 0x7f9be00b6130 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc009b60 tx=0x7f9bdc005f90 comp rx=0 tx=0).stop 2024-07-26T13:40:14.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.549+0000 7f9bf8d7e640 1 -- 172.21.15.100:0/66014604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bf4108670 msgr2=0x7f9bf419f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:14.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.549+0000 7f9bf8d7e640 1 --2- 172.21.15.100:0/66014604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bf4108670 0x7f9bf419f680 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00eb10 tx=0x7f9bec00c2d0 comp rx=0 tx=0).stop 2024-07-26T13:40:14.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.549+0000 7f9bf8d7e640 1 -- 172.21.15.100:0/66014604 shutdown_connections 2024-07-26T13:40:14.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.549+0000 7f9bf8d7e640 1 --2- 172.21.15.100:0/66014604 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9be00b3c20 0x7f9be00b6130 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:14.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.549+0000 7f9bf8d7e640 1 --2- 172.21.15.100:0/66014604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bf4108670 0x7f9bf419f680 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:14.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.549+0000 7f9bf8d7e640 1 --2- 172.21.15.100:0/66014604 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bf4107d20 0x7f9bf419a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:14.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.549+0000 7f9bf8d7e640 1 --2- 172.21.15.100:0/66014604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bf41023f0 0x7f9bf4199e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:14.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.549+0000 7f9bf8d7e640 1 -- 172.21.15.100:0/66014604 >> 172.21.15.100:0/66014604 conn(0x7f9bf407a0a0 msgr2=0x7f9bf4103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:14.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.549+0000 7f9bf8d7e640 1 -- 172.21.15.100:0/66014604 shutdown_connections 2024-07-26T13:40:14.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.549+0000 7f9bf8d7e640 1 -- 172.21.15.100:0/66014604 wait complete. 2024-07-26T13:40:14.562 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2024-07-26T13:40:14.410593+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663541,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1170,"num_read_kb":691,"num_write":304,"num_write_kb":2281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":2952484,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":746,"ondisk_log_size":746,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372500,"kb_used_data":10868,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569324,"statfs":{"total":1151940427776,"available":1151558987776,"internally_reserved":0,"allocated":11128832,"data_stored":7299414,"data_compressed":177282,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5681,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":47,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003202"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387528+0000","last_change":"2024-07-26T13:36:59.462289+0000","last_active":"2024-07-26T13:38:46.387528+0000","last_peered":"2024-07-26T13:38:46.387528+0000","last_clean":"2024-07-26T13:38:46.387528+0000","last_became_active":"2024-07-26T13:36:59.460311+0000","last_became_peered":"2024-07-26T13:36:59.460311+0000","last_unstale":"2024-07-26T13:38:46.387528+0000","last_undegraded":"2024-07-26T13:38:46.387528+0000","last_fullsized":"2024-07-26T13:38:46.387528+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:04:59.123716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.384825+0000","last_change":"2024-07-26T13:36:59.463218+0000","last_active":"2024-07-26T13:38:46.384825+0000","last_peered":"2024-07-26T13:38:46.384825+0000","last_clean":"2024-07-26T13:38:46.384825+0000","last_became_active":"2024-07-26T13:36:59.462889+0000","last_became_peered":"2024-07-26T13:36:59.462889+0000","last_unstale":"2024-07-26T13:38:46.384825+0000","last_undegraded":"2024-07-26T13:38:46.384825+0000","last_fullsized":"2024-07-26T13:38:46.384825+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:30:46.788669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383503+0000","last_change":"2024-07-26T13:36:59.456113+0000","last_active":"2024-07-26T13:38:46.383503+0000","last_peered":"2024-07-26T13:38:46.383503+0000","last_clean":"2024-07-26T13:38:46.383503+0000","last_became_active":"2024-07-26T13:36:59.455622+0000","last_became_peered":"2024-07-26T13:36:59.455622+0000","last_unstale":"2024-07-26T13:38:46.383503+0000","last_undegraded":"2024-07-26T13:38:46.383503+0000","last_fullsized":"2024-07-26T13:38:46.383503+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:36:26.474799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.368792+0000","last_change":"2024-07-26T13:37:17.555947+0000","last_active":"2024-07-26T13:38:46.368792+0000","last_peered":"2024-07-26T13:38:46.368792+0000","last_clean":"2024-07-26T13:38:46.368792+0000","last_became_active":"2024-07-26T13:37:17.555607+0000","last_became_peered":"2024-07-26T13:37:17.555607+0000","last_unstale":"2024-07-26T13:38:46.368792+0000","last_undegraded":"2024-07-26T13:38:46.368792+0000","last_fullsized":"2024-07-26T13:38:46.368792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:19:59.603225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.372136+0000","last_change":"2024-07-26T13:36:59.474896+0000","last_active":"2024-07-26T13:38:46.372136+0000","last_peered":"2024-07-26T13:38:46.372136+0000","last_clean":"2024-07-26T13:38:46.372136+0000","last_became_active":"2024-07-26T13:36:59.474516+0000","last_became_peered":"2024-07-26T13:36:59.474516+0000","last_unstale":"2024-07-26T13:38:46.372136+0000","last_undegraded":"2024-07-26T13:38:46.372136+0000","last_fullsized":"2024-07-26T13:38:46.372136+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:38:18.994280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.358362+0000","last_change":"2024-07-26T13:37:17.556191+0000","last_active":"2024-07-26T13:38:46.358362+0000","last_peered":"2024-07-26T13:38:46.358362+0000","last_clean":"2024-07-26T13:38:46.358362+0000","last_became_active":"2024-07-26T13:37:17.555715+0000","last_became_peered":"2024-07-26T13:37:17.555715+0000","last_unstale":"2024-07-26T13:38:46.358362+0000","last_undegraded":"2024-07-26T13:38:46.358362+0000","last_fullsized":"2024-07-26T13:38:46.358362+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:03:18.352049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383784+0000","last_change":"2024-07-26T13:36:59.469515+0000","last_active":"2024-07-26T13:38:46.383784+0000","last_peered":"2024-07-26T13:38:46.383784+0000","last_clean":"2024-07-26T13:38:46.383784+0000","last_became_active":"2024-07-26T13:36:59.469106+0000","last_became_peered":"2024-07-26T13:36:59.469106+0000","last_unstale":"2024-07-26T13:38:46.383784+0000","last_undegraded":"2024-07-26T13:38:46.383784+0000","last_fullsized":"2024-07-26T13:38:46.383784+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:10:05.455329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.373036+0000","last_change":"2024-07-26T13:37:17.549365+0000","last_active":"2024-07-26T13:38:46.373036+0000","last_peered":"2024-07-26T13:38:46.373036+0000","last_clean":"2024-07-26T13:38:46.373036+0000","last_became_active":"2024-07-26T13:37:17.549036+0000","last_became_peered":"2024-07-26T13:37:17.549036+0000","last_unstale":"2024-07-26T13:38:46.373036+0000","last_undegraded":"2024-07-26T13:38:46.373036+0000","last_fullsized":"2024-07-26T13:38:46.373036+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:27:43.930981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387690+0000","last_change":"2024-07-26T13:36:59.459306+0000","last_active":"2024-07-26T13:38:46.387690+0000","last_peered":"2024-07-26T13:38:46.387690+0000","last_clean":"2024-07-26T13:38:46.387690+0000","last_became_active":"2024-07-26T13:36:59.458714+0000","last_became_peered":"2024-07-26T13:36:59.458714+0000","last_unstale":"2024-07-26T13:38:46.387690+0000","last_undegraded":"2024-07-26T13:38:46.387690+0000","last_fullsized":"2024-07-26T13:38:46.387690+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:34:18.512454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.376748+0000","last_change":"2024-07-26T13:36:59.466556+0000","last_active":"2024-07-26T13:38:46.376748+0000","last_peered":"2024-07-26T13:38:46.376748+0000","last_clean":"2024-07-26T13:38:46.376748+0000","last_became_active":"2024-07-26T13:36:59.466147+0000","last_became_peered":"2024-07-26T13:36:59.466147+0000","last_unstale":"2024-07-26T13:38:46.376748+0000","last_undegraded":"2024-07-26T13:38:46.376748+0000","last_fullsized":"2024-07-26T13:38:46.376748+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:43:11.359314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387571+0000","last_change":"2024-07-26T13:36:59.473685+0000","last_active":"2024-07-26T13:38:46.387571+0000","last_peered":"2024-07-26T13:38:46.387571+0000","last_clean":"2024-07-26T13:38:46.387571+0000","last_became_active":"2024-07-26T13:36:59.473314+0000","last_became_peered":"2024-07-26T13:36:59.473314+0000","last_unstale":"2024-07-26T13:38:46.387571+0000","last_undegraded":"2024-07-26T13:38:46.387571+0000","last_fullsized":"2024-07-26T13:38:46.387571+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:41:51.612348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.368487+0000","last_change":"2024-07-26T13:36:59.471586+0000","last_active":"2024-07-26T13:38:46.368487+0000","last_peered":"2024-07-26T13:38:46.368487+0000","last_clean":"2024-07-26T13:38:46.368487+0000","last_became_active":"2024-07-26T13:36:59.470995+0000","last_became_peered":"2024-07-26T13:36:59.470995+0000","last_unstale":"2024-07-26T13:38:46.368487+0000","last_undegraded":"2024-07-26T13:38:46.368487+0000","last_fullsized":"2024-07-26T13:38:46.368487+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:54:07.957937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.358125+0000","last_change":"2024-07-26T13:36:59.460018+0000","last_active":"2024-07-26T13:38:46.358125+0000","last_peered":"2024-07-26T13:38:46.358125+0000","last_clean":"2024-07-26T13:38:46.358125+0000","last_became_active":"2024-07-26T13:36:59.459671+0000","last_became_peered":"2024-07-26T13:36:59.459671+0000","last_unstale":"2024-07-26T13:38:46.358125+0000","last_undegraded":"2024-07-26T13:38:46.358125+0000","last_fullsized":"2024-07-26T13:38:46.358125+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:19:33.923360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.374512+0000","last_change":"2024-07-26T13:36:59.470271+0000","last_active":"2024-07-26T13:38:46.374512+0000","last_peered":"2024-07-26T13:38:46.374512+0000","last_clean":"2024-07-26T13:38:46.374512+0000","last_became_active":"2024-07-26T13:36:59.469941+0000","last_became_peered":"2024-07-26T13:36:59.469941+0000","last_unstale":"2024-07-26T13:38:46.374512+0000","last_undegraded":"2024-07-26T13:38:46.374512+0000","last_fullsized":"2024-07-26T13:38:46.374512+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:38:39.329366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383168+0000","last_change":"2024-07-26T13:36:59.460133+0000","last_active":"2024-07-26T13:38:46.383168+0000","last_peered":"2024-07-26T13:38:46.383168+0000","last_clean":"2024-07-26T13:38:46.383168+0000","last_became_active":"2024-07-26T13:36:59.459794+0000","last_became_peered":"2024-07-26T13:36:59.459794+0000","last_unstale":"2024-07-26T13:38:46.383168+0000","last_undegraded":"2024-07-26T13:38:46.383168+0000","last_fullsized":"2024-07-26T13:38:46.383168+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:15:36.024347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.386026+0000","last_change":"2024-07-26T13:36:59.462390+0000","last_active":"2024-07-26T13:38:46.386026+0000","last_peered":"2024-07-26T13:38:46.386026+0000","last_clean":"2024-07-26T13:38:46.386026+0000","last_became_active":"2024-07-26T13:36:59.461968+0000","last_became_peered":"2024-07-26T13:36:59.461968+0000","last_unstale":"2024-07-26T13:38:46.386026+0000","last_undegraded":"2024-07-26T13:38:46.386026+0000","last_fullsized":"2024-07-26T13:38:46.386026+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:49:47.142472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387645+0000","last_change":"2024-07-26T13:36:59.460206+0000","last_active":"2024-07-26T13:38:46.387645+0000","last_peered":"2024-07-26T13:38:46.387645+0000","last_clean":"2024-07-26T13:38:46.387645+0000","last_became_active":"2024-07-26T13:36:59.459811+0000","last_became_peered":"2024-07-26T13:36:59.459811+0000","last_unstale":"2024-07-26T13:38:46.387645+0000","last_undegraded":"2024-07-26T13:38:46.387645+0000","last_fullsized":"2024-07-26T13:38:46.387645+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:46:57.112314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387993+0000","last_change":"2024-07-26T13:36:59.460197+0000","last_active":"2024-07-26T13:38:46.387993+0000","last_peered":"2024-07-26T13:38:46.387993+0000","last_clean":"2024-07-26T13:38:46.387993+0000","last_became_active":"2024-07-26T13:36:59.459686+0000","last_became_peered":"2024-07-26T13:36:59.459686+0000","last_unstale":"2024-07-26T13:38:46.387993+0000","last_undegraded":"2024-07-26T13:38:46.387993+0000","last_fullsized":"2024-07-26T13:38:46.387993+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T13:39:22.407376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.373506+0000","last_change":"2024-07-26T13:36:59.462545+0000","last_active":"2024-07-26T13:38:46.373506+0000","last_peered":"2024-07-26T13:38:46.373506+0000","last_clean":"2024-07-26T13:38:46.373506+0000","last_became_active":"2024-07-26T13:36:59.462115+0000","last_became_peered":"2024-07-26T13:36:59.462115+0000","last_unstale":"2024-07-26T13:38:46.373506+0000","last_undegraded":"2024-07-26T13:38:46.373506+0000","last_fullsized":"2024-07-26T13:38:46.373506+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T21:44:13.715076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.384982+0000","last_change":"2024-07-26T13:36:59.465317+0000","last_active":"2024-07-26T13:38:46.384982+0000","last_peered":"2024-07-26T13:38:46.384982+0000","last_clean":"2024-07-26T13:38:46.384982+0000","last_became_active":"2024-07-26T13:36:59.464943+0000","last_became_peered":"2024-07-26T13:36:59.464943+0000","last_unstale":"2024-07-26T13:38:46.384982+0000","last_undegraded":"2024-07-26T13:38:46.384982+0000","last_fullsized":"2024-07-26T13:38:46.384982+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T13:53:54.160584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.370482+0000","last_change":"2024-07-26T13:36:59.462884+0000","last_active":"2024-07-26T13:38:46.370482+0000","last_peered":"2024-07-26T13:38:46.370482+0000","last_clean":"2024-07-26T13:38:46.370482+0000","last_became_active":"2024-07-26T13:36:59.462564+0000","last_became_peered":"2024-07-26T13:36:59.462564+0000","last_unstale":"2024-07-26T13:38:46.370482+0000","last_undegraded":"2024-07-26T13:38:46.370482+0000","last_fullsized":"2024-07-26T13:38:46.370482+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:50:19.549148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387141+0000","last_change":"2024-07-26T13:36:59.468716+0000","last_active":"2024-07-26T13:38:46.387141+0000","last_peered":"2024-07-26T13:38:46.387141+0000","last_clean":"2024-07-26T13:38:46.387141+0000","last_became_active":"2024-07-26T13:36:59.468048+0000","last_became_peered":"2024-07-26T13:36:59.468048+0000","last_unstale":"2024-07-26T13:38:46.387141+0000","last_undegraded":"2024-07-26T13:38:46.387141+0000","last_fullsized":"2024-07-26T13:38:46.387141+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:51:29.510047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383235+0000","last_change":"2024-07-26T13:36:59.466565+0000","last_active":"2024-07-26T13:38:46.383235+0000","last_peered":"2024-07-26T13:38:46.383235+0000","last_clean":"2024-07-26T13:38:46.383235+0000","last_became_active":"2024-07-26T13:36:59.466235+0000","last_became_peered":"2024-07-26T13:36:59.466235+0000","last_unstale":"2024-07-26T13:38:46.383235+0000","last_undegraded":"2024-07-26T13:38:46.383235+0000","last_fullsized":"2024-07-26T13:38:46.383235+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:28:44.326821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.371918+0000","last_change":"2024-07-26T13:36:59.458753+0000","last_active":"2024-07-26T13:38:46.371918+0000","last_peered":"2024-07-26T13:38:46.371918+0000","last_clean":"2024-07-26T13:38:46.371918+0000","last_became_active":"2024-07-26T13:36:59.458333+0000","last_became_peered":"2024-07-26T13:36:59.458333+0000","last_unstale":"2024-07-26T13:38:46.371918+0000","last_undegraded":"2024-07-26T13:38:46.371918+0000","last_fullsized":"2024-07-26T13:38:46.371918+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:33:54.806212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.369463+0000","last_change":"2024-07-26T13:36:59.468657+0000","last_active":"2024-07-26T13:38:46.369463+0000","last_peered":"2024-07-26T13:38:46.369463+0000","last_clean":"2024-07-26T13:38:46.369463+0000","last_became_active":"2024-07-26T13:36:59.468121+0000","last_became_peered":"2024-07-26T13:36:59.468121+0000","last_unstale":"2024-07-26T13:38:46.369463+0000","last_undegraded":"2024-07-26T13:38:46.369463+0000","last_fullsized":"2024-07-26T13:38:46.369463+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:23:53.959836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.377710+0000","last_change":"2024-07-26T13:36:59.469067+0000","last_active":"2024-07-26T13:38:46.377710+0000","last_peered":"2024-07-26T13:38:46.377710+0000","last_clean":"2024-07-26T13:38:46.377710+0000","last_became_active":"2024-07-26T13:36:59.468706+0000","last_became_peered":"2024-07-26T13:36:59.468706+0000","last_unstale":"2024-07-26T13:38:46.377710+0000","last_undegraded":"2024-07-26T13:38:46.377710+0000","last_fullsized":"2024-07-26T13:38:46.377710+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:21:35.974618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.378393+0000","last_change":"2024-07-26T13:36:59.469807+0000","last_active":"2024-07-26T13:38:46.378393+0000","last_peered":"2024-07-26T13:38:46.378393+0000","last_clean":"2024-07-26T13:38:46.378393+0000","last_became_active":"2024-07-26T13:36:59.469020+0000","last_became_peered":"2024-07-26T13:36:59.469020+0000","last_unstale":"2024-07-26T13:38:46.378393+0000","last_undegraded":"2024-07-26T13:38:46.378393+0000","last_fullsized":"2024-07-26T13:38:46.378393+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:29:57.931598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.373015+0000","last_change":"2024-07-26T13:36:59.465236+0000","last_active":"2024-07-26T13:38:46.373015+0000","last_peered":"2024-07-26T13:38:46.373015+0000","last_clean":"2024-07-26T13:38:46.373015+0000","last_became_active":"2024-07-26T13:36:59.464415+0000","last_became_peered":"2024-07-26T13:36:59.464415+0000","last_unstale":"2024-07-26T13:38:46.373015+0000","last_undegraded":"2024-07-26T13:38:46.373015+0000","last_fullsized":"2024-07-26T13:38:46.373015+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:46:20.373688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.375987+0000","last_change":"2024-07-26T13:36:59.466113+0000","last_active":"2024-07-26T13:38:46.375987+0000","last_peered":"2024-07-26T13:38:46.375987+0000","last_clean":"2024-07-26T13:38:46.375987+0000","last_became_active":"2024-07-26T13:36:59.465743+0000","last_became_peered":"2024-07-26T13:36:59.465743+0000","last_unstale":"2024-07-26T13:38:46.375987+0000","last_undegraded":"2024-07-26T13:38:46.375987+0000","last_fullsized":"2024-07-26T13:38:46.375987+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:14:41.839204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.380451+0000","last_change":"2024-07-26T13:36:59.460672+0000","last_active":"2024-07-26T13:38:46.380451+0000","last_peered":"2024-07-26T13:38:46.380451+0000","last_clean":"2024-07-26T13:38:46.380451+0000","last_became_active":"2024-07-26T13:36:59.460247+0000","last_became_peered":"2024-07-26T13:36:59.460247+0000","last_unstale":"2024-07-26T13:38:46.380451+0000","last_undegraded":"2024-07-26T13:38:46.380451+0000","last_fullsized":"2024-07-26T13:38:46.380451+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:37:51.109933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.378083+0000","last_change":"2024-07-26T13:36:59.464784+0000","last_active":"2024-07-26T13:38:46.378083+0000","last_peered":"2024-07-26T13:38:46.378083+0000","last_clean":"2024-07-26T13:38:46.378083+0000","last_became_active":"2024-07-26T13:36:59.464261+0000","last_became_peered":"2024-07-26T13:36:59.464261+0000","last_unstale":"2024-07-26T13:38:46.378083+0000","last_undegraded":"2024-07-26T13:38:46.378083+0000","last_fullsized":"2024-07-26T13:38:46.378083+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:43:10.134037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.385005+0000","last_change":"2024-07-26T13:36:59.460421+0000","last_active":"2024-07-26T13:38:46.385005+0000","last_peered":"2024-07-26T13:38:46.385005+0000","last_clean":"2024-07-26T13:38:46.385005+0000","last_became_active":"2024-07-26T13:36:59.460010+0000","last_became_peered":"2024-07-26T13:36:59.460010+0000","last_unstale":"2024-07-26T13:38:46.385005+0000","last_undegraded":"2024-07-26T13:38:46.385005+0000","last_fullsized":"2024-07-26T13:38:46.385005+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:16:21.651673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.358312+0000","last_change":"2024-07-26T13:36:59.468826+0000","last_active":"2024-07-26T13:38:46.358312+0000","last_peered":"2024-07-26T13:38:46.358312+0000","last_clean":"2024-07-26T13:38:46.358312+0000","last_became_active":"2024-07-26T13:36:59.468406+0000","last_became_peered":"2024-07-26T13:36:59.468406+0000","last_unstale":"2024-07-26T13:38:46.358312+0000","last_undegraded":"2024-07-26T13:38:46.358312+0000","last_fullsized":"2024-07-26T13:38:46.358312+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:22:10.590545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'7","reported_seq":227,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:40:10.849058+0000","last_change":"2024-07-26T13:37:17.559014+0000","last_active":"2024-07-26T13:40:10.849058+0000","last_peered":"2024-07-26T13:40:10.849058+0000","last_clean":"2024-07-26T13:40:10.849058+0000","last_became_active":"2024-07-26T13:37:17.552325+0000","last_became_peered":"2024-07-26T13:37:17.552325+0000","last_unstale":"2024-07-26T13:40:10.849058+0000","last_undegraded":"2024-07-26T13:40:10.849058+0000","last_fullsized":"2024-07-26T13:40:10.849058+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:26:26.790337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":179,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.379560+0000","last_change":"2024-07-26T13:37:17.562871+0000","last_active":"2024-07-26T13:38:46.379560+0000","last_peered":"2024-07-26T13:38:46.379560+0000","last_clean":"2024-07-26T13:38:46.379560+0000","last_became_active":"2024-07-26T13:37:17.562363+0000","last_became_peered":"2024-07-26T13:37:17.562363+0000","last_unstale":"2024-07-26T13:38:46.379560+0000","last_undegraded":"2024-07-26T13:38:46.379560+0000","last_fullsized":"2024-07-26T13:38:46.379560+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:25:36.172091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"79'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.547762+0000","last_change":"2024-07-26T13:36:56.434778+0000","last_active":"2024-07-26T13:37:17.547762+0000","last_peered":"2024-07-26T13:37:17.547762+0000","last_clean":"2024-07-26T13:37:17.547762+0000","last_became_active":"2024-07-26T13:36:56.434288+0000","last_became_peered":"2024-07-26T13:36:56.434288+0000","last_unstale":"2024-07-26T13:37:17.547762+0000","last_undegraded":"2024-07-26T13:37:17.547762+0000","last_fullsized":"2024-07-26T13:37:17.547762+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:24:22.990358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.565163+0000","last_change":"2024-07-26T13:36:56.424665+0000","last_active":"2024-07-26T13:37:17.565163+0000","last_peered":"2024-07-26T13:37:17.565163+0000","last_clean":"2024-07-26T13:37:17.565163+0000","last_became_active":"2024-07-26T13:36:56.424290+0000","last_became_peered":"2024-07-26T13:36:56.424290+0000","last_unstale":"2024-07-26T13:37:17.565163+0000","last_undegraded":"2024-07-26T13:37:17.565163+0000","last_fullsized":"2024-07-26T13:37:17.565163+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:49:59.520194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383766+0000","last_change":"2024-07-26T13:36:59.454609+0000","last_active":"2024-07-26T13:38:46.383766+0000","last_peered":"2024-07-26T13:38:46.383766+0000","last_clean":"2024-07-26T13:38:46.383766+0000","last_became_active":"2024-07-26T13:36:59.454228+0000","last_became_peered":"2024-07-26T13:36:59.454228+0000","last_unstale":"2024-07-26T13:38:46.383766+0000","last_undegraded":"2024-07-26T13:38:46.383766+0000","last_fullsized":"2024-07-26T13:38:46.383766+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:02:06.244433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.547136+0000","last_change":"2024-07-26T13:36:56.430268+0000","last_active":"2024-07-26T13:37:17.547136+0000","last_peered":"2024-07-26T13:37:17.547136+0000","last_clean":"2024-07-26T13:37:17.547136+0000","last_became_active":"2024-07-26T13:36:56.428583+0000","last_became_peered":"2024-07-26T13:36:56.428583+0000","last_unstale":"2024-07-26T13:37:17.547136+0000","last_undegraded":"2024-07-26T13:37:17.547136+0000","last_fullsized":"2024-07-26T13:37:17.547136+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:07:05.864886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.379724+0000","last_change":"2024-07-26T13:36:59.456188+0000","last_active":"2024-07-26T13:38:46.379724+0000","last_peered":"2024-07-26T13:38:46.379724+0000","last_clean":"2024-07-26T13:38:46.379724+0000","last_became_active":"2024-07-26T13:36:59.455842+0000","last_became_peered":"2024-07-26T13:36:59.455842+0000","last_unstale":"2024-07-26T13:38:46.379724+0000","last_undegraded":"2024-07-26T13:38:46.379724+0000","last_fullsized":"2024-07-26T13:38:46.379724+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:00:19.162239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":207,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:40:11.849930+0000","last_change":"2024-07-26T13:36:56.432818+0000","last_active":"2024-07-26T13:40:11.849930+0000","last_peered":"2024-07-26T13:40:11.849930+0000","last_clean":"2024-07-26T13:40:11.849930+0000","last_became_active":"2024-07-26T13:36:56.432419+0000","last_became_peered":"2024-07-26T13:36:56.432419+0000","last_unstale":"2024-07-26T13:40:11.849930+0000","last_undegraded":"2024-07-26T13:40:11.849930+0000","last_fullsized":"2024-07-26T13:40:11.849930+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:55:51.972470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":176,"num_read_kb":176,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.386929+0000","last_change":"2024-07-26T13:36:59.460033+0000","last_active":"2024-07-26T13:38:46.386929+0000","last_peered":"2024-07-26T13:38:46.386929+0000","last_clean":"2024-07-26T13:38:46.386929+0000","last_became_active":"2024-07-26T13:36:59.459562+0000","last_became_peered":"2024-07-26T13:36:59.459562+0000","last_unstale":"2024-07-26T13:38:46.386929+0000","last_undegraded":"2024-07-26T13:38:46.386929+0000","last_fullsized":"2024-07-26T13:38:46.386929+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:58:14.155973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551399+0000","last_change":"2024-07-26T13:36:56.432563+0000","last_active":"2024-07-26T13:37:17.551399+0000","last_peered":"2024-07-26T13:37:17.551399+0000","last_clean":"2024-07-26T13:37:17.551399+0000","last_became_active":"2024-07-26T13:36:56.431134+0000","last_became_peered":"2024-07-26T13:36:56.431134+0000","last_unstale":"2024-07-26T13:37:17.551399+0000","last_undegraded":"2024-07-26T13:37:17.551399+0000","last_fullsized":"2024-07-26T13:37:17.551399+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:42:00.653345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.377839+0000","last_change":"2024-07-26T13:36:59.469782+0000","last_active":"2024-07-26T13:38:46.377839+0000","last_peered":"2024-07-26T13:38:46.377839+0000","last_clean":"2024-07-26T13:38:46.377839+0000","last_became_active":"2024-07-26T13:36:59.469412+0000","last_became_peered":"2024-07-26T13:36:59.469412+0000","last_unstale":"2024-07-26T13:38:46.377839+0000","last_undegraded":"2024-07-26T13:38:46.377839+0000","last_fullsized":"2024-07-26T13:38:46.377839+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:28:07.741486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.558844+0000","last_change":"2024-07-26T13:36:56.488981+0000","last_active":"2024-07-26T13:37:17.558844+0000","last_peered":"2024-07-26T13:37:17.558844+0000","last_clean":"2024-07-26T13:37:17.558844+0000","last_became_active":"2024-07-26T13:36:56.488480+0000","last_became_peered":"2024-07-26T13:36:56.488480+0000","last_unstale":"2024-07-26T13:37:17.558844+0000","last_undegraded":"2024-07-26T13:37:17.558844+0000","last_fullsized":"2024-07-26T13:37:17.558844+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:41:10.041007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387010+0000","last_change":"2024-07-26T13:36:59.473183+0000","last_active":"2024-07-26T13:38:46.387010+0000","last_peered":"2024-07-26T13:38:46.387010+0000","last_clean":"2024-07-26T13:38:46.387010+0000","last_became_active":"2024-07-26T13:36:59.472814+0000","last_became_peered":"2024-07-26T13:36:59.472814+0000","last_unstale":"2024-07-26T13:38:46.387010+0000","last_undegraded":"2024-07-26T13:38:46.387010+0000","last_fullsized":"2024-07-26T13:38:46.387010+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:57:03.396384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'35","reported_seq":425,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:40:12.866875+0000","last_change":"2024-07-26T13:36:56.435034+0000","last_active":"2024-07-26T13:40:12.866875+0000","last_peered":"2024-07-26T13:40:12.866875+0000","last_clean":"2024-07-26T13:40:12.866875+0000","last_became_active":"2024-07-26T13:36:56.434611+0000","last_became_peered":"2024-07-26T13:36:56.434611+0000","last_unstale":"2024-07-26T13:40:12.866875+0000","last_undegraded":"2024-07-26T13:40:12.866875+0000","last_fullsized":"2024-07-26T13:40:12.866875+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:18:27.272077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68265,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":363,"num_read_kb":248,"num_write":36,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.382422+0000","last_change":"2024-07-26T13:36:59.465831+0000","last_active":"2024-07-26T13:38:46.382422+0000","last_peered":"2024-07-26T13:38:46.382422+0000","last_clean":"2024-07-26T13:38:46.382422+0000","last_became_active":"2024-07-26T13:36:59.465404+0000","last_became_peered":"2024-07-26T13:36:59.465404+0000","last_unstale":"2024-07-26T13:38:46.382422+0000","last_undegraded":"2024-07-26T13:38:46.382422+0000","last_fullsized":"2024-07-26T13:38:46.382422+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:42:53.795926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'6","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.552497+0000","last_change":"2024-07-26T13:36:56.435170+0000","last_active":"2024-07-26T13:37:17.552497+0000","last_peered":"2024-07-26T13:37:17.552497+0000","last_clean":"2024-07-26T13:37:17.552497+0000","last_became_active":"2024-07-26T13:36:56.434715+0000","last_became_peered":"2024-07-26T13:36:56.434715+0000","last_unstale":"2024-07-26T13:37:17.552497+0000","last_undegraded":"2024-07-26T13:37:17.552497+0000","last_fullsized":"2024-07-26T13:37:17.552497+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:05:09.712510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.370726+0000","last_change":"2024-07-26T13:36:59.461926+0000","last_active":"2024-07-26T13:38:46.370726+0000","last_peered":"2024-07-26T13:38:46.370726+0000","last_clean":"2024-07-26T13:38:46.370726+0000","last_became_active":"2024-07-26T13:36:59.461512+0000","last_became_peered":"2024-07-26T13:36:59.461512+0000","last_unstale":"2024-07-26T13:38:46.370726+0000","last_undegraded":"2024-07-26T13:38:46.370726+0000","last_fullsized":"2024-07-26T13:38:46.370726+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:34:24.405663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.546907+0000","last_change":"2024-07-26T13:36:56.430062+0000","last_active":"2024-07-26T13:37:17.546907+0000","last_peered":"2024-07-26T13:37:17.546907+0000","last_clean":"2024-07-26T13:37:17.546907+0000","last_became_active":"2024-07-26T13:36:56.429654+0000","last_became_peered":"2024-07-26T13:36:56.429654+0000","last_unstale":"2024-07-26T13:37:17.546907+0000","last_undegraded":"2024-07-26T13:37:17.546907+0000","last_fullsized":"2024-07-26T13:37:17.546907+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:21:09.514359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.380005+0000","last_change":"2024-07-26T13:36:59.474459+0000","last_active":"2024-07-26T13:38:46.380005+0000","last_peered":"2024-07-26T13:38:46.380005+0000","last_clean":"2024-07-26T13:38:46.380005+0000","last_became_active":"2024-07-26T13:36:59.474061+0000","last_became_peered":"2024-07-26T13:36:59.474061+0000","last_unstale":"2024-07-26T13:38:46.380005+0000","last_undegraded":"2024-07-26T13:38:46.380005+0000","last_fullsized":"2024-07-26T13:38:46.380005+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:37:21.691491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'5","reported_seq":213,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:40:11.849698+0000","last_change":"2024-07-26T13:36:56.432915+0000","last_active":"2024-07-26T13:40:11.849698+0000","last_peered":"2024-07-26T13:40:11.849698+0000","last_clean":"2024-07-26T13:40:11.849698+0000","last_became_active":"2024-07-26T13:36:56.432499+0000","last_became_peered":"2024-07-26T13:36:56.432499+0000","last_unstale":"2024-07-26T13:40:11.849698+0000","last_undegraded":"2024-07-26T13:40:11.849698+0000","last_fullsized":"2024-07-26T13:40:11.849698+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:02:14.306950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":182,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383574+0000","last_change":"2024-07-26T13:36:59.468945+0000","last_active":"2024-07-26T13:38:46.383574+0000","last_peered":"2024-07-26T13:38:46.383574+0000","last_clean":"2024-07-26T13:38:46.383574+0000","last_became_active":"2024-07-26T13:36:59.467970+0000","last_became_peered":"2024-07-26T13:36:59.467970+0000","last_unstale":"2024-07-26T13:38:46.383574+0000","last_undegraded":"2024-07-26T13:38:46.383574+0000","last_fullsized":"2024-07-26T13:38:46.383574+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T13:59:19.514945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'15","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551412+0000","last_change":"2024-07-26T13:36:56.487014+0000","last_active":"2024-07-26T13:37:17.551412+0000","last_peered":"2024-07-26T13:37:17.551412+0000","last_clean":"2024-07-26T13:37:17.551412+0000","last_became_active":"2024-07-26T13:36:56.486470+0000","last_became_peered":"2024-07-26T13:36:56.486470+0000","last_unstale":"2024-07-26T13:37:17.551412+0000","last_undegraded":"2024-07-26T13:37:17.551412+0000","last_fullsized":"2024-07-26T13:37:17.551412+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:07:02.205644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.366501+0000","last_change":"2024-07-26T13:36:59.462465+0000","last_active":"2024-07-26T13:38:46.366501+0000","last_peered":"2024-07-26T13:38:46.366501+0000","last_clean":"2024-07-26T13:38:46.366501+0000","last_became_active":"2024-07-26T13:36:59.462024+0000","last_became_peered":"2024-07-26T13:36:59.462024+0000","last_unstale":"2024-07-26T13:38:46.366501+0000","last_undegraded":"2024-07-26T13:38:46.366501+0000","last_fullsized":"2024-07-26T13:38:46.366501+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:34:14.317762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551992+0000","last_change":"2024-07-26T13:36:56.435007+0000","last_active":"2024-07-26T13:37:17.551992+0000","last_peered":"2024-07-26T13:37:17.551992+0000","last_clean":"2024-07-26T13:37:17.551992+0000","last_became_active":"2024-07-26T13:36:56.434444+0000","last_became_peered":"2024-07-26T13:36:56.434444+0000","last_unstale":"2024-07-26T13:37:17.551992+0000","last_undegraded":"2024-07-26T13:37:17.551992+0000","last_fullsized":"2024-07-26T13:37:17.551992+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:27:51.041349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.384490+0000","last_change":"2024-07-26T13:36:59.464164+0000","last_active":"2024-07-26T13:38:46.384490+0000","last_peered":"2024-07-26T13:38:46.384490+0000","last_clean":"2024-07-26T13:38:46.384490+0000","last_became_active":"2024-07-26T13:36:59.463782+0000","last_became_peered":"2024-07-26T13:36:59.463782+0000","last_unstale":"2024-07-26T13:38:46.384490+0000","last_undegraded":"2024-07-26T13:38:46.384490+0000","last_fullsized":"2024-07-26T13:38:46.384490+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:48:06.481731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.565510+0000","last_change":"2024-07-26T13:36:56.429653+0000","last_active":"2024-07-26T13:37:17.565510+0000","last_peered":"2024-07-26T13:37:17.565510+0000","last_clean":"2024-07-26T13:37:17.565510+0000","last_became_active":"2024-07-26T13:36:56.429060+0000","last_became_peered":"2024-07-26T13:36:56.429060+0000","last_unstale":"2024-07-26T13:37:17.565510+0000","last_undegraded":"2024-07-26T13:37:17.565510+0000","last_fullsized":"2024-07-26T13:37:17.565510+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:41:48.261709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.384430+0000","last_change":"2024-07-26T13:36:59.471560+0000","last_active":"2024-07-26T13:38:46.384430+0000","last_peered":"2024-07-26T13:38:46.384430+0000","last_clean":"2024-07-26T13:38:46.384430+0000","last_became_active":"2024-07-26T13:36:59.471151+0000","last_became_peered":"2024-07-26T13:36:59.471151+0000","last_unstale":"2024-07-26T13:38:46.384430+0000","last_undegraded":"2024-07-26T13:38:46.384430+0000","last_fullsized":"2024-07-26T13:38:46.384430+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:49:20.513308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.548185+0000","last_change":"2024-07-26T13:36:56.430010+0000","last_active":"2024-07-26T13:37:17.548185+0000","last_peered":"2024-07-26T13:37:17.548185+0000","last_clean":"2024-07-26T13:37:17.548185+0000","last_became_active":"2024-07-26T13:36:56.429120+0000","last_became_peered":"2024-07-26T13:36:56.429120+0000","last_unstale":"2024-07-26T13:37:17.548185+0000","last_undegraded":"2024-07-26T13:37:17.548185+0000","last_fullsized":"2024-07-26T13:37:17.548185+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:23:43.321975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383589+0000","last_change":"2024-07-26T13:36:59.467788+0000","last_active":"2024-07-26T13:38:46.383589+0000","last_peered":"2024-07-26T13:38:46.383589+0000","last_clean":"2024-07-26T13:38:46.383589+0000","last_became_active":"2024-07-26T13:36:59.467399+0000","last_became_peered":"2024-07-26T13:36:59.467399+0000","last_unstale":"2024-07-26T13:38:46.383589+0000","last_undegraded":"2024-07-26T13:38:46.383589+0000","last_fullsized":"2024-07-26T13:38:46.383589+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:48:51.442999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'3","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.553308+0000","last_change":"2024-07-26T13:36:56.588633+0000","last_active":"2024-07-26T13:37:17.553308+0000","last_peered":"2024-07-26T13:37:17.553308+0000","last_clean":"2024-07-26T13:37:17.553308+0000","last_became_active":"2024-07-26T13:36:56.588113+0000","last_became_peered":"2024-07-26T13:36:56.588113+0000","last_unstale":"2024-07-26T13:37:17.553308+0000","last_undegraded":"2024-07-26T13:37:17.553308+0000","last_fullsized":"2024-07-26T13:37:17.553308+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:03:55.075052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387531+0000","last_change":"2024-07-26T13:36:59.472854+0000","last_active":"2024-07-26T13:38:46.387531+0000","last_peered":"2024-07-26T13:38:46.387531+0000","last_clean":"2024-07-26T13:38:46.387531+0000","last_became_active":"2024-07-26T13:36:59.472466+0000","last_became_peered":"2024-07-26T13:36:59.472466+0000","last_unstale":"2024-07-26T13:38:46.387531+0000","last_undegraded":"2024-07-26T13:38:46.387531+0000","last_fullsized":"2024-07-26T13:38:46.387531+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:50:57.454149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":103,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550829+0000","last_change":"2024-07-26T13:35:34.839121+0000","last_active":"2024-07-26T13:37:17.550829+0000","last_peered":"2024-07-26T13:37:17.550829+0000","last_clean":"2024-07-26T13:37:17.550829+0000","last_became_active":"2024-07-26T13:35:34.833465+0000","last_became_peered":"2024-07-26T13:35:34.833465+0000","last_unstale":"2024-07-26T13:37:17.550829+0000","last_undegraded":"2024-07-26T13:37:17.550829+0000","last_fullsized":"2024-07-26T13:37:17.550829+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:32:23.285428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:32:23.285428+0000","last_clean_scrub_stamp":"2024-07-26T13:32:23.285428+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:29:31.650487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'3","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.561459+0000","last_change":"2024-07-26T13:36:56.428648+0000","last_active":"2024-07-26T13:37:17.561459+0000","last_peered":"2024-07-26T13:37:17.561459+0000","last_clean":"2024-07-26T13:37:17.561459+0000","last_became_active":"2024-07-26T13:36:56.427665+0000","last_became_peered":"2024-07-26T13:36:56.427665+0000","last_unstale":"2024-07-26T13:37:17.561459+0000","last_undegraded":"2024-07-26T13:37:17.561459+0000","last_fullsized":"2024-07-26T13:37:17.561459+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:41:39.371687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.376696+0000","last_change":"2024-07-26T13:36:59.460625+0000","last_active":"2024-07-26T13:38:46.376696+0000","last_peered":"2024-07-26T13:38:46.376696+0000","last_clean":"2024-07-26T13:38:46.376696+0000","last_became_active":"2024-07-26T13:36:59.458658+0000","last_became_peered":"2024-07-26T13:36:59.458658+0000","last_unstale":"2024-07-26T13:38:46.376696+0000","last_undegraded":"2024-07-26T13:38:46.376696+0000","last_fullsized":"2024-07-26T13:38:46.376696+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:46:42.812402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'3","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550796+0000","last_change":"2024-07-26T13:36:56.432617+0000","last_active":"2024-07-26T13:37:17.550796+0000","last_peered":"2024-07-26T13:37:17.550796+0000","last_clean":"2024-07-26T13:37:17.550796+0000","last_became_active":"2024-07-26T13:36:56.431519+0000","last_became_peered":"2024-07-26T13:36:56.431519+0000","last_unstale":"2024-07-26T13:37:17.550796+0000","last_undegraded":"2024-07-26T13:37:17.550796+0000","last_fullsized":"2024-07-26T13:37:17.550796+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:32:21.120076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.364664+0000","last_change":"2024-07-26T13:36:59.464084+0000","last_active":"2024-07-26T13:38:46.364664+0000","last_peered":"2024-07-26T13:38:46.364664+0000","last_clean":"2024-07-26T13:38:46.364664+0000","last_became_active":"2024-07-26T13:36:59.463755+0000","last_became_peered":"2024-07-26T13:36:59.463755+0000","last_unstale":"2024-07-26T13:38:46.364664+0000","last_undegraded":"2024-07-26T13:38:46.364664+0000","last_fullsized":"2024-07-26T13:38:46.364664+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:53:20.034188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":209,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:40:10.847516+0000","last_change":"2024-07-26T13:36:56.486071+0000","last_active":"2024-07-26T13:40:10.847516+0000","last_peered":"2024-07-26T13:40:10.847516+0000","last_clean":"2024-07-26T13:40:10.847516+0000","last_became_active":"2024-07-26T13:36:56.485672+0000","last_became_peered":"2024-07-26T13:36:56.485672+0000","last_unstale":"2024-07-26T13:40:10.847516+0000","last_undegraded":"2024-07-26T13:40:10.847516+0000","last_fullsized":"2024-07-26T13:40:10.847516+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T21:29:45.666651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":176,"num_read_kb":176,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.372811+0000","last_change":"2024-07-26T13:37:17.566555+0000","last_active":"2024-07-26T13:38:46.372811+0000","last_peered":"2024-07-26T13:38:46.372811+0000","last_clean":"2024-07-26T13:38:46.372811+0000","last_became_active":"2024-07-26T13:37:17.566040+0000","last_became_peered":"2024-07-26T13:37:17.566040+0000","last_unstale":"2024-07-26T13:38:46.372811+0000","last_undegraded":"2024-07-26T13:38:46.372811+0000","last_fullsized":"2024-07-26T13:38:46.372811+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:48:33.078890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550989+0000","last_change":"2024-07-26T13:36:56.431811+0000","last_active":"2024-07-26T13:37:17.550989+0000","last_peered":"2024-07-26T13:37:17.550989+0000","last_clean":"2024-07-26T13:37:17.550989+0000","last_became_active":"2024-07-26T13:36:56.431223+0000","last_became_peered":"2024-07-26T13:36:56.431223+0000","last_unstale":"2024-07-26T13:37:17.550989+0000","last_undegraded":"2024-07-26T13:37:17.550989+0000","last_fullsized":"2024-07-26T13:37:17.550989+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:08:00.334515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.363881+0000","last_change":"2024-07-26T13:36:59.463972+0000","last_active":"2024-07-26T13:38:46.363881+0000","last_peered":"2024-07-26T13:38:46.363881+0000","last_clean":"2024-07-26T13:38:46.363881+0000","last_became_active":"2024-07-26T13:36:59.463496+0000","last_became_peered":"2024-07-26T13:36:59.463496+0000","last_unstale":"2024-07-26T13:38:46.363881+0000","last_undegraded":"2024-07-26T13:38:46.363881+0000","last_fullsized":"2024-07-26T13:38:46.363881+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:02:06.215372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.545780+0000","last_change":"2024-07-26T13:36:56.486681+0000","last_active":"2024-07-26T13:37:17.545780+0000","last_peered":"2024-07-26T13:37:17.545780+0000","last_clean":"2024-07-26T13:37:17.545780+0000","last_became_active":"2024-07-26T13:36:56.486328+0000","last_became_peered":"2024-07-26T13:36:56.486328+0000","last_unstale":"2024-07-26T13:37:17.545780+0000","last_undegraded":"2024-07-26T13:37:17.545780+0000","last_fullsized":"2024-07-26T13:37:17.545780+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:10:36.557707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.355361+0000","last_change":"2024-07-26T13:36:59.451415+0000","last_active":"2024-07-26T13:38:46.355361+0000","last_peered":"2024-07-26T13:38:46.355361+0000","last_clean":"2024-07-26T13:38:46.355361+0000","last_became_active":"2024-07-26T13:36:59.450992+0000","last_became_peered":"2024-07-26T13:36:59.450992+0000","last_unstale":"2024-07-26T13:38:46.355361+0000","last_undegraded":"2024-07-26T13:38:46.355361+0000","last_fullsized":"2024-07-26T13:38:46.355361+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:29:11.303916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.557777+0000","last_change":"2024-07-26T13:37:17.555643+0000","last_active":"2024-07-26T13:37:17.557777+0000","last_peered":"2024-07-26T13:37:17.557777+0000","last_clean":"2024-07-26T13:37:17.557777+0000","last_became_active":"2024-07-26T13:37:17.549756+0000","last_became_peered":"2024-07-26T13:37:17.549756+0000","last_unstale":"2024-07-26T13:37:17.557777+0000","last_undegraded":"2024-07-26T13:37:17.557777+0000","last_fullsized":"2024-07-26T13:37:17.557777+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:20:16.911286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.368534+0000","last_change":"2024-07-26T13:36:59.465538+0000","last_active":"2024-07-26T13:38:46.368534+0000","last_peered":"2024-07-26T13:38:46.368534+0000","last_clean":"2024-07-26T13:38:46.368534+0000","last_became_active":"2024-07-26T13:36:59.464886+0000","last_became_peered":"2024-07-26T13:36:59.464886+0000","last_unstale":"2024-07-26T13:38:46.368534+0000","last_undegraded":"2024-07-26T13:38:46.368534+0000","last_fullsized":"2024-07-26T13:38:46.368534+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:19:12.273116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'13","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:39:08.749398+0000","last_change":"2024-07-26T13:36:56.428355+0000","last_active":"2024-07-26T13:39:08.749398+0000","last_peered":"2024-07-26T13:39:08.749398+0000","last_clean":"2024-07-26T13:39:08.749398+0000","last_became_active":"2024-07-26T13:36:56.427213+0000","last_became_peered":"2024-07-26T13:36:56.427213+0000","last_unstale":"2024-07-26T13:39:08.749398+0000","last_undegraded":"2024-07-26T13:39:08.749398+0000","last_fullsized":"2024-07-26T13:39:08.749398+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:48:09.730067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.378498+0000","last_change":"2024-07-26T13:36:59.459378+0000","last_active":"2024-07-26T13:38:46.378498+0000","last_peered":"2024-07-26T13:38:46.378498+0000","last_clean":"2024-07-26T13:38:46.378498+0000","last_became_active":"2024-07-26T13:36:59.458799+0000","last_became_peered":"2024-07-26T13:36:59.458799+0000","last_unstale":"2024-07-26T13:38:46.378498+0000","last_undegraded":"2024-07-26T13:38:46.378498+0000","last_fullsized":"2024-07-26T13:38:46.378498+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:45:19.587800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.561441+0000","last_change":"2024-07-26T13:36:56.588819+0000","last_active":"2024-07-26T13:37:17.561441+0000","last_peered":"2024-07-26T13:37:17.561441+0000","last_clean":"2024-07-26T13:37:17.561441+0000","last_became_active":"2024-07-26T13:36:56.588430+0000","last_became_peered":"2024-07-26T13:36:56.588430+0000","last_unstale":"2024-07-26T13:37:17.561441+0000","last_undegraded":"2024-07-26T13:37:17.561441+0000","last_fullsized":"2024-07-26T13:37:17.561441+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:07:21.387618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.380460+0000","last_change":"2024-07-26T13:37:17.562040+0000","last_active":"2024-07-26T13:38:46.380460+0000","last_peered":"2024-07-26T13:38:46.380460+0000","last_clean":"2024-07-26T13:38:46.380460+0000","last_became_active":"2024-07-26T13:37:17.561542+0000","last_became_peered":"2024-07-26T13:37:17.561542+0000","last_unstale":"2024-07-26T13:38:46.380460+0000","last_undegraded":"2024-07-26T13:38:46.380460+0000","last_fullsized":"2024-07-26T13:38:46.380460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:21:08.161954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.544169+0000","last_change":"2024-07-26T13:36:56.489106+0000","last_active":"2024-07-26T13:37:17.544169+0000","last_peered":"2024-07-26T13:37:17.544169+0000","last_clean":"2024-07-26T13:37:17.544169+0000","last_became_active":"2024-07-26T13:36:56.488706+0000","last_became_peered":"2024-07-26T13:36:56.488706+0000","last_unstale":"2024-07-26T13:37:17.544169+0000","last_undegraded":"2024-07-26T13:37:17.544169+0000","last_fullsized":"2024-07-26T13:37:17.544169+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:00:59.204434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.343705+0000","last_change":"2024-07-26T13:36:59.470412+0000","last_active":"2024-07-26T13:38:46.343705+0000","last_peered":"2024-07-26T13:38:46.343705+0000","last_clean":"2024-07-26T13:38:46.343705+0000","last_became_active":"2024-07-26T13:36:59.470010+0000","last_became_peered":"2024-07-26T13:36:59.470010+0000","last_unstale":"2024-07-26T13:38:46.343705+0000","last_undegraded":"2024-07-26T13:38:46.343705+0000","last_fullsized":"2024-07-26T13:38:46.343705+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:50:57.987600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.543528+0000","last_change":"2024-07-26T13:36:56.428671+0000","last_active":"2024-07-26T13:37:17.543528+0000","last_peered":"2024-07-26T13:37:17.543528+0000","last_clean":"2024-07-26T13:37:17.543528+0000","last_became_active":"2024-07-26T13:36:56.428301+0000","last_became_peered":"2024-07-26T13:36:56.428301+0000","last_unstale":"2024-07-26T13:37:17.543528+0000","last_undegraded":"2024-07-26T13:37:17.543528+0000","last_fullsized":"2024-07-26T13:37:17.543528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:59:19.593664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.385021+0000","last_change":"2024-07-26T13:36:59.458627+0000","last_active":"2024-07-26T13:38:46.385021+0000","last_peered":"2024-07-26T13:38:46.385021+0000","last_clean":"2024-07-26T13:38:46.385021+0000","last_became_active":"2024-07-26T13:36:59.458199+0000","last_became_peered":"2024-07-26T13:36:59.458199+0000","last_unstale":"2024-07-26T13:38:46.385021+0000","last_undegraded":"2024-07-26T13:38:46.385021+0000","last_fullsized":"2024-07-26T13:38:46.385021+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:54:41.833223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551540+0000","last_change":"2024-07-26T13:36:56.431548+0000","last_active":"2024-07-26T13:37:17.551540+0000","last_peered":"2024-07-26T13:37:17.551540+0000","last_clean":"2024-07-26T13:37:17.551540+0000","last_became_active":"2024-07-26T13:36:56.430989+0000","last_became_peered":"2024-07-26T13:36:56.430989+0000","last_unstale":"2024-07-26T13:37:17.551540+0000","last_undegraded":"2024-07-26T13:37:17.551540+0000","last_fullsized":"2024-07-26T13:37:17.551540+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:46:51.655324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.375762+0000","last_change":"2024-07-26T13:36:59.458290+0000","last_active":"2024-07-26T13:38:46.375762+0000","last_peered":"2024-07-26T13:38:46.375762+0000","last_clean":"2024-07-26T13:38:46.375762+0000","last_became_active":"2024-07-26T13:36:59.457942+0000","last_became_peered":"2024-07-26T13:36:59.457942+0000","last_unstale":"2024-07-26T13:38:46.375762+0000","last_undegraded":"2024-07-26T13:38:46.375762+0000","last_fullsized":"2024-07-26T13:38:46.375762+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:51:58.797441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":46,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.568059+0000","last_change":"2024-07-26T13:37:17.567624+0000","last_active":"2024-07-26T13:37:17.568059+0000","last_peered":"2024-07-26T13:37:17.568059+0000","last_clean":"2024-07-26T13:37:17.568059+0000","last_became_active":"2024-07-26T13:37:17.562455+0000","last_became_peered":"2024-07-26T13:37:17.562455+0000","last_unstale":"2024-07-26T13:37:17.568059+0000","last_undegraded":"2024-07-26T13:37:17.568059+0000","last_fullsized":"2024-07-26T13:37:17.568059+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:50:36.418750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387835+0000","last_change":"2024-07-26T13:36:59.474034+0000","last_active":"2024-07-26T13:38:46.387835+0000","last_peered":"2024-07-26T13:38:46.387835+0000","last_clean":"2024-07-26T13:38:46.387835+0000","last_became_active":"2024-07-26T13:36:59.472358+0000","last_became_peered":"2024-07-26T13:36:59.472358+0000","last_unstale":"2024-07-26T13:38:46.387835+0000","last_undegraded":"2024-07-26T13:38:46.387835+0000","last_fullsized":"2024-07-26T13:38:46.387835+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:36:44.565413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.368595+0000","last_change":"2024-07-26T13:36:59.465546+0000","last_active":"2024-07-26T13:38:46.368595+0000","last_peered":"2024-07-26T13:38:46.368595+0000","last_clean":"2024-07-26T13:38:46.368595+0000","last_became_active":"2024-07-26T13:36:59.465137+0000","last_became_peered":"2024-07-26T13:36:59.465137+0000","last_unstale":"2024-07-26T13:38:46.368595+0000","last_undegraded":"2024-07-26T13:38:46.368595+0000","last_fullsized":"2024-07-26T13:38:46.368595+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:35:13.591354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.562664+0000","last_change":"2024-07-26T13:37:17.562012+0000","last_active":"2024-07-26T13:37:17.562664+0000","last_peered":"2024-07-26T13:37:17.562664+0000","last_clean":"2024-07-26T13:37:17.562664+0000","last_became_active":"2024-07-26T13:37:17.550218+0000","last_became_peered":"2024-07-26T13:37:17.550218+0000","last_unstale":"2024-07-26T13:37:17.562664+0000","last_undegraded":"2024-07-26T13:37:17.562664+0000","last_fullsized":"2024-07-26T13:37:17.562664+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:46:16.559533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.375095+0000","last_change":"2024-07-26T13:36:59.456896+0000","last_active":"2024-07-26T13:38:46.375095+0000","last_peered":"2024-07-26T13:38:46.375095+0000","last_clean":"2024-07-26T13:38:46.375095+0000","last_became_active":"2024-07-26T13:36:59.456530+0000","last_became_peered":"2024-07-26T13:36:59.456530+0000","last_unstale":"2024-07-26T13:38:46.375095+0000","last_undegraded":"2024-07-26T13:38:46.375095+0000","last_fullsized":"2024-07-26T13:38:46.375095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:59:56.512501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'4","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.553596+0000","last_change":"2024-07-26T13:36:56.433119+0000","last_active":"2024-07-26T13:37:17.553596+0000","last_peered":"2024-07-26T13:37:17.553596+0000","last_clean":"2024-07-26T13:37:17.553596+0000","last_became_active":"2024-07-26T13:36:56.432765+0000","last_became_peered":"2024-07-26T13:36:56.432765+0000","last_unstale":"2024-07-26T13:37:17.553596+0000","last_undegraded":"2024-07-26T13:37:17.553596+0000","last_fullsized":"2024-07-26T13:37:17.553596+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:31:29.197106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.382051+0000","last_change":"2024-07-26T13:36:59.462125+0000","last_active":"2024-07-26T13:38:46.382051+0000","last_peered":"2024-07-26T13:38:46.382051+0000","last_clean":"2024-07-26T13:38:46.382051+0000","last_became_active":"2024-07-26T13:36:59.461796+0000","last_became_peered":"2024-07-26T13:36:59.461796+0000","last_unstale":"2024-07-26T13:38:46.382051+0000","last_undegraded":"2024-07-26T13:38:46.382051+0000","last_fullsized":"2024-07-26T13:38:46.382051+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:44:10.329046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.543646+0000","last_change":"2024-07-26T13:36:56.487251+0000","last_active":"2024-07-26T13:37:17.543646+0000","last_peered":"2024-07-26T13:37:17.543646+0000","last_clean":"2024-07-26T13:37:17.543646+0000","last_became_active":"2024-07-26T13:36:56.486918+0000","last_became_peered":"2024-07-26T13:36:56.486918+0000","last_unstale":"2024-07-26T13:37:17.543646+0000","last_undegraded":"2024-07-26T13:37:17.543646+0000","last_fullsized":"2024-07-26T13:37:17.543646+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:41:31.248895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.380839+0000","last_change":"2024-07-26T13:36:59.470810+0000","last_active":"2024-07-26T13:38:46.380839+0000","last_peered":"2024-07-26T13:38:46.380839+0000","last_clean":"2024-07-26T13:38:46.380839+0000","last_became_active":"2024-07-26T13:36:59.467371+0000","last_became_peered":"2024-07-26T13:36:59.467371+0000","last_unstale":"2024-07-26T13:38:46.380839+0000","last_undegraded":"2024-07-26T13:38:46.380839+0000","last_fullsized":"2024-07-26T13:38:46.380839+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:23:48.572933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'9","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.566327+0000","last_change":"2024-07-26T13:36:56.433598+0000","last_active":"2024-07-26T13:37:17.566327+0000","last_peered":"2024-07-26T13:37:17.566327+0000","last_clean":"2024-07-26T13:37:17.566327+0000","last_became_active":"2024-07-26T13:36:56.433325+0000","last_became_peered":"2024-07-26T13:36:56.433325+0000","last_unstale":"2024-07-26T13:37:17.566327+0000","last_undegraded":"2024-07-26T13:37:17.566327+0000","last_fullsized":"2024-07-26T13:37:17.566327+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:17:33.651765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72939,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1089,"num_read_kb":622,"num_write":179,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":644,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":218817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247480,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678071,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":443451,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874300,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":1103707,"data_compressed":41637,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037825,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":897024,"data_stored":443665,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234053,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":443563,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430281,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":1036188,"data_compressed":41637,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593807,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":443054,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790036,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":1035340,"data_compressed":41637,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953561,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":888832,"data_stored":444023,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149788,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":443365,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378722,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":511840,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542246,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":506893,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705771,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":444325,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34195,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34195,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":69419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":68993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-26T13:40:14.566 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-07-26T13:40:14.566 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-26T13:40:14.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.713+0000 7f6f5767e640 1 -- 172.21.15.100:0/3641443198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f50107ce0 msgr2=0x7f6f501080f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:14.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.713+0000 7f6f5767e640 1 --2- 172.21.15.100:0/3641443198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f50107ce0 0x7f6f501080f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6f40009900 tx=0x7f6f4002a410 comp rx=0 tx=0).stop 2024-07-26T13:40:14.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.713+0000 7f6f5767e640 1 -- 172.21.15.100:0/3641443198 shutdown_connections 2024-07-26T13:40:14.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.713+0000 7f6f5767e640 1 --2- 172.21.15.100:0/3641443198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f50108630 0x7f6f5010ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:14.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.713+0000 7f6f5767e640 1 --2- 172.21.15.100:0/3641443198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f50107ce0 0x7f6f501080f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:14.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.713+0000 7f6f5767e640 1 --2- 172.21.15.100:0/3641443198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f501023b0 0x7f6f501027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:14.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.713+0000 7f6f5767e640 1 -- 172.21.15.100:0/3641443198 >> 172.21.15.100:0/3641443198 conn(0x7f6f500fddb0 msgr2=0x7f6f50100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:14.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.714+0000 7f6f5767e640 1 -- 172.21.15.100:0/3641443198 shutdown_connections 2024-07-26T13:40:14.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.714+0000 7f6f5767e640 1 -- 172.21.15.100:0/3641443198 wait complete. 2024-07-26T13:40:14.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.714+0000 7f6f5767e640 1 Processor -- start 2024-07-26T13:40:14.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.714+0000 7f6f5767e640 1 -- start start 2024-07-26T13:40:14.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.714+0000 7f6f5767e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f501023b0 0x7f6f5019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:14.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.714+0000 7f6f5767e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f50107ce0 0x7f6f5019b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:14.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.714+0000 7f6f5767e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f50108630 0x7f6f5019b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:14.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.714+0000 7f6f5767e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6f501122c0 con 0x7f6f501023b0 2024-07-26T13:40:14.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.714+0000 7f6f5767e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6f50112140 con 0x7f6f50108630 2024-07-26T13:40:14.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.714+0000 7f6f5767e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6f50112440 con 0x7f6f50107ce0 2024-07-26T13:40:14.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.715+0000 7f6f5541a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f501023b0 0x7f6f5019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:14.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.715+0000 7f6f5541a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f501023b0 0x7f6f5019ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57782/0 (socket says 172.21.15.100:57782) 2024-07-26T13:40:14.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.715+0000 7f6f5541a640 1 -- 172.21.15.100:0/2721976181 learned_addr learned my addr 172.21.15.100:0/2721976181 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:14.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.715+0000 7f6f54c19640 1 --2- 172.21.15.100:0/2721976181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f50107ce0 0x7f6f5019b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:14.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.715+0000 7f6f55c1b640 1 --2- 172.21.15.100:0/2721976181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f50108630 0x7f6f5019b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:14.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.715+0000 7f6f5541a640 1 -- 172.21.15.100:0/2721976181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f50107ce0 msgr2=0x7f6f5019b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:14.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.715+0000 7f6f5541a640 1 --2- 172.21.15.100:0/2721976181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f50107ce0 0x7f6f5019b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:14.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.715+0000 7f6f5541a640 1 -- 172.21.15.100:0/2721976181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f50108630 msgr2=0x7f6f5019b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:14.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.715+0000 7f6f5541a640 1 --2- 172.21.15.100:0/2721976181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f50108630 0x7f6f5019b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:14.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.715+0000 7f6f5541a640 1 -- 172.21.15.100:0/2721976181 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f501a5ec0 con 0x7f6f501023b0 2024-07-26T13:40:14.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.715+0000 7f6f5541a640 1 --2- 172.21.15.100:0/2721976181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f501023b0 0x7f6f5019ac30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c00d770 tx=0x7f6f4c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:14.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.715+0000 7f6f467fc640 1 -- 172.21.15.100:0/2721976181 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c017070 con 0x7f6f501023b0 2024-07-26T13:40:14.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.715+0000 7f6f467fc640 1 -- 172.21.15.100:0/2721976181 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f4c0054d0 con 0x7f6f501023b0 2024-07-26T13:40:14.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.715+0000 7f6f5767e640 1 -- 172.21.15.100:0/2721976181 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f501a61b0 con 0x7f6f501023b0 2024-07-26T13:40:14.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.715+0000 7f6f467fc640 1 -- 172.21.15.100:0/2721976181 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c004c30 con 0x7f6f501023b0 2024-07-26T13:40:14.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.715+0000 7f6f5767e640 1 -- 172.21.15.100:0/2721976181 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f501a6680 con 0x7f6f501023b0 2024-07-26T13:40:14.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.716+0000 7f6f5767e640 1 -- 172.21.15.100:0/2721976181 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f50103620 con 0x7f6f501023b0 2024-07-26T13:40:14.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.718+0000 7f6f467fc640 1 -- 172.21.15.100:0/2721976181 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 151077+0+0 (secure 0 0 0) 0x7f6f4c004dd0 con 0x7f6f501023b0 2024-07-26T13:40:14.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.719+0000 7f6f467fc640 1 --2- 172.21.15.100:0/2721976181 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6f3c0b3c20 0x7f6f3c0b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:14.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.719+0000 7f6f54c19640 1 --2- 172.21.15.100:0/2721976181 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6f3c0b3c20 0x7f6f3c0b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:14.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.719+0000 7f6f467fc640 1 -- 172.21.15.100:0/2721976181 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f6f4c0e3c00 con 0x7f6f501023b0 2024-07-26T13:40:14.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.719+0000 7f6f54c19640 1 --2- 172.21.15.100:0/2721976181 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6f3c0b3c20 0x7f6f3c0b6130 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6f40009230 tx=0x7f6f40031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:14.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.722+0000 7f6f467fc640 1 -- 172.21.15.100:0/2721976181 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f6f4c0ab150 con 0x7f6f501023b0 2024-07-26T13:40:14.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.904+0000 7f6f5767e640 1 -- 172.21.15.100:0/2721976181 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6f50063230 con 0x7f6f3c0b3c20 2024-07-26T13:40:14.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.908+0000 7f6f467fc640 1 -- 172.21.15.100:0/2721976181 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259061 (secure 0 0 0) 0x7f6f50063230 con 0x7f6f3c0b3c20 2024-07-26T13:40:14.912 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:40:14.913 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-07-26T13:40:14.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.912+0000 7f6f5767e640 1 -- 172.21.15.100:0/2721976181 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6f3c0b3c20 msgr2=0x7f6f3c0b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:14.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.912+0000 7f6f5767e640 1 --2- 172.21.15.100:0/2721976181 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6f3c0b3c20 0x7f6f3c0b6130 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6f40009230 tx=0x7f6f40031400 comp rx=0 tx=0).stop 2024-07-26T13:40:14.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.912+0000 7f6f5767e640 1 -- 172.21.15.100:0/2721976181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f501023b0 msgr2=0x7f6f5019ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:14.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.912+0000 7f6f5767e640 1 --2- 172.21.15.100:0/2721976181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f501023b0 0x7f6f5019ac30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c00d770 tx=0x7f6f4c00dc30 comp rx=0 tx=0).stop 2024-07-26T13:40:14.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.912+0000 7f6f5767e640 1 -- 172.21.15.100:0/2721976181 shutdown_connections 2024-07-26T13:40:14.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.912+0000 7f6f5767e640 1 --2- 172.21.15.100:0/2721976181 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6f3c0b3c20 0x7f6f3c0b6130 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:14.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.912+0000 7f6f5767e640 1 --2- 172.21.15.100:0/2721976181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f50108630 0x7f6f5019b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:14.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.912+0000 7f6f5767e640 1 --2- 172.21.15.100:0/2721976181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f50107ce0 0x7f6f5019b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:14.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.912+0000 7f6f5767e640 1 --2- 172.21.15.100:0/2721976181 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f501023b0 0x7f6f5019ac30 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:14.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.912+0000 7f6f5767e640 1 -- 172.21.15.100:0/2721976181 >> 172.21.15.100:0/2721976181 conn(0x7f6f500fddb0 msgr2=0x7f6f501052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:14.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.912+0000 7f6f5767e640 1 -- 172.21.15.100:0/2721976181 shutdown_connections 2024-07-26T13:40:14.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:14.912+0000 7f6f5767e640 1 -- 172.21.15.100:0/2721976181 wait complete. 2024-07-26T13:40:14.923 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:14 smithi100 ceph-mon[32599]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-26T13:40:14.923 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:14 smithi100 ceph-mon[32599]: pgmap v368: 97 pgs: 97 active+clean; 648 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 7 op/s 2024-07-26T13:40:14.924 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2024-07-26T13:40:14.410593+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663541,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1170,"num_read_kb":691,"num_write":304,"num_write_kb":2281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":2952484,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":746,"ondisk_log_size":746,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372500,"kb_used_data":10868,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569324,"statfs":{"total":1151940427776,"available":1151558987776,"internally_reserved":0,"allocated":11128832,"data_stored":7299414,"data_compressed":177282,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5681,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":47,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003202"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387528+0000","last_change":"2024-07-26T13:36:59.462289+0000","last_active":"2024-07-26T13:38:46.387528+0000","last_peered":"2024-07-26T13:38:46.387528+0000","last_clean":"2024-07-26T13:38:46.387528+0000","last_became_active":"2024-07-26T13:36:59.460311+0000","last_became_peered":"2024-07-26T13:36:59.460311+0000","last_unstale":"2024-07-26T13:38:46.387528+0000","last_undegraded":"2024-07-26T13:38:46.387528+0000","last_fullsized":"2024-07-26T13:38:46.387528+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:04:59.123716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.384825+0000","last_change":"2024-07-26T13:36:59.463218+0000","last_active":"2024-07-26T13:38:46.384825+0000","last_peered":"2024-07-26T13:38:46.384825+0000","last_clean":"2024-07-26T13:38:46.384825+0000","last_became_active":"2024-07-26T13:36:59.462889+0000","last_became_peered":"2024-07-26T13:36:59.462889+0000","last_unstale":"2024-07-26T13:38:46.384825+0000","last_undegraded":"2024-07-26T13:38:46.384825+0000","last_fullsized":"2024-07-26T13:38:46.384825+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:30:46.788669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383503+0000","last_change":"2024-07-26T13:36:59.456113+0000","last_active":"2024-07-26T13:38:46.383503+0000","last_peered":"2024-07-26T13:38:46.383503+0000","last_clean":"2024-07-26T13:38:46.383503+0000","last_became_active":"2024-07-26T13:36:59.455622+0000","last_became_peered":"2024-07-26T13:36:59.455622+0000","last_unstale":"2024-07-26T13:38:46.383503+0000","last_undegraded":"2024-07-26T13:38:46.383503+0000","last_fullsized":"2024-07-26T13:38:46.383503+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:36:26.474799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.368792+0000","last_change":"2024-07-26T13:37:17.555947+0000","last_active":"2024-07-26T13:38:46.368792+0000","last_peered":"2024-07-26T13:38:46.368792+0000","last_clean":"2024-07-26T13:38:46.368792+0000","last_became_active":"2024-07-26T13:37:17.555607+0000","last_became_peered":"2024-07-26T13:37:17.555607+0000","last_unstale":"2024-07-26T13:38:46.368792+0000","last_undegraded":"2024-07-26T13:38:46.368792+0000","last_fullsized":"2024-07-26T13:38:46.368792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:19:59.603225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.372136+0000","last_change":"2024-07-26T13:36:59.474896+0000","last_active":"2024-07-26T13:38:46.372136+0000","last_peered":"2024-07-26T13:38:46.372136+0000","last_clean":"2024-07-26T13:38:46.372136+0000","last_became_active":"2024-07-26T13:36:59.474516+0000","last_became_peered":"2024-07-26T13:36:59.474516+0000","last_unstale":"2024-07-26T13:38:46.372136+0000","last_undegraded":"2024-07-26T13:38:46.372136+0000","last_fullsized":"2024-07-26T13:38:46.372136+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:38:18.994280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.358362+0000","last_change":"2024-07-26T13:37:17.556191+0000","last_active":"2024-07-26T13:38:46.358362+0000","last_peered":"2024-07-26T13:38:46.358362+0000","last_clean":"2024-07-26T13:38:46.358362+0000","last_became_active":"2024-07-26T13:37:17.555715+0000","last_became_peered":"2024-07-26T13:37:17.555715+0000","last_unstale":"2024-07-26T13:38:46.358362+0000","last_undegraded":"2024-07-26T13:38:46.358362+0000","last_fullsized":"2024-07-26T13:38:46.358362+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:03:18.352049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383784+0000","last_change":"2024-07-26T13:36:59.469515+0000","last_active":"2024-07-26T13:38:46.383784+0000","last_peered":"2024-07-26T13:38:46.383784+0000","last_clean":"2024-07-26T13:38:46.383784+0000","last_became_active":"2024-07-26T13:36:59.469106+0000","last_became_peered":"2024-07-26T13:36:59.469106+0000","last_unstale":"2024-07-26T13:38:46.383784+0000","last_undegraded":"2024-07-26T13:38:46.383784+0000","last_fullsized":"2024-07-26T13:38:46.383784+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:10:05.455329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.373036+0000","last_change":"2024-07-26T13:37:17.549365+0000","last_active":"2024-07-26T13:38:46.373036+0000","last_peered":"2024-07-26T13:38:46.373036+0000","last_clean":"2024-07-26T13:38:46.373036+0000","last_became_active":"2024-07-26T13:37:17.549036+0000","last_became_peered":"2024-07-26T13:37:17.549036+0000","last_unstale":"2024-07-26T13:38:46.373036+0000","last_undegraded":"2024-07-26T13:38:46.373036+0000","last_fullsized":"2024-07-26T13:38:46.373036+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:27:43.930981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387690+0000","last_change":"2024-07-26T13:36:59.459306+0000","last_active":"2024-07-26T13:38:46.387690+0000","last_peered":"2024-07-26T13:38:46.387690+0000","last_clean":"2024-07-26T13:38:46.387690+0000","last_became_active":"2024-07-26T13:36:59.458714+0000","last_became_peered":"2024-07-26T13:36:59.458714+0000","last_unstale":"2024-07-26T13:38:46.387690+0000","last_undegraded":"2024-07-26T13:38:46.387690+0000","last_fullsized":"2024-07-26T13:38:46.387690+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:34:18.512454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.376748+0000","last_change":"2024-07-26T13:36:59.466556+0000","last_active":"2024-07-26T13:38:46.376748+0000","last_peered":"2024-07-26T13:38:46.376748+0000","last_clean":"2024-07-26T13:38:46.376748+0000","last_became_active":"2024-07-26T13:36:59.466147+0000","last_became_peered":"2024-07-26T13:36:59.466147+0000","last_unstale":"2024-07-26T13:38:46.376748+0000","last_undegraded":"2024-07-26T13:38:46.376748+0000","last_fullsized":"2024-07-26T13:38:46.376748+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:43:11.359314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387571+0000","last_change":"2024-07-26T13:36:59.473685+0000","last_active":"2024-07-26T13:38:46.387571+0000","last_peered":"2024-07-26T13:38:46.387571+0000","last_clean":"2024-07-26T13:38:46.387571+0000","last_became_active":"2024-07-26T13:36:59.473314+0000","last_became_peered":"2024-07-26T13:36:59.473314+0000","last_unstale":"2024-07-26T13:38:46.387571+0000","last_undegraded":"2024-07-26T13:38:46.387571+0000","last_fullsized":"2024-07-26T13:38:46.387571+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:41:51.612348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.368487+0000","last_change":"2024-07-26T13:36:59.471586+0000","last_active":"2024-07-26T13:38:46.368487+0000","last_peered":"2024-07-26T13:38:46.368487+0000","last_clean":"2024-07-26T13:38:46.368487+0000","last_became_active":"2024-07-26T13:36:59.470995+0000","last_became_peered":"2024-07-26T13:36:59.470995+0000","last_unstale":"2024-07-26T13:38:46.368487+0000","last_undegraded":"2024-07-26T13:38:46.368487+0000","last_fullsized":"2024-07-26T13:38:46.368487+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:54:07.957937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.358125+0000","last_change":"2024-07-26T13:36:59.460018+0000","last_active":"2024-07-26T13:38:46.358125+0000","last_peered":"2024-07-26T13:38:46.358125+0000","last_clean":"2024-07-26T13:38:46.358125+0000","last_became_active":"2024-07-26T13:36:59.459671+0000","last_became_peered":"2024-07-26T13:36:59.459671+0000","last_unstale":"2024-07-26T13:38:46.358125+0000","last_undegraded":"2024-07-26T13:38:46.358125+0000","last_fullsized":"2024-07-26T13:38:46.358125+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:19:33.923360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.374512+0000","last_change":"2024-07-26T13:36:59.470271+0000","last_active":"2024-07-26T13:38:46.374512+0000","last_peered":"2024-07-26T13:38:46.374512+0000","last_clean":"2024-07-26T13:38:46.374512+0000","last_became_active":"2024-07-26T13:36:59.469941+0000","last_became_peered":"2024-07-26T13:36:59.469941+0000","last_unstale":"2024-07-26T13:38:46.374512+0000","last_undegraded":"2024-07-26T13:38:46.374512+0000","last_fullsized":"2024-07-26T13:38:46.374512+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:38:39.329366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383168+0000","last_change":"2024-07-26T13:36:59.460133+0000","last_active":"2024-07-26T13:38:46.383168+0000","last_peered":"2024-07-26T13:38:46.383168+0000","last_clean":"2024-07-26T13:38:46.383168+0000","last_became_active":"2024-07-26T13:36:59.459794+0000","last_became_peered":"2024-07-26T13:36:59.459794+0000","last_unstale":"2024-07-26T13:38:46.383168+0000","last_undegraded":"2024-07-26T13:38:46.383168+0000","last_fullsized":"2024-07-26T13:38:46.383168+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:15:36.024347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.386026+0000","last_change":"2024-07-26T13:36:59.462390+0000","last_active":"2024-07-26T13:38:46.386026+0000","last_peered":"2024-07-26T13:38:46.386026+0000","last_clean":"2024-07-26T13:38:46.386026+0000","last_became_active":"2024-07-26T13:36:59.461968+0000","last_became_peered":"2024-07-26T13:36:59.461968+0000","last_unstale":"2024-07-26T13:38:46.386026+0000","last_undegraded":"2024-07-26T13:38:46.386026+0000","last_fullsized":"2024-07-26T13:38:46.386026+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:49:47.142472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387645+0000","last_change":"2024-07-26T13:36:59.460206+0000","last_active":"2024-07-26T13:38:46.387645+0000","last_peered":"2024-07-26T13:38:46.387645+0000","last_clean":"2024-07-26T13:38:46.387645+0000","last_became_active":"2024-07-26T13:36:59.459811+0000","last_became_peered":"2024-07-26T13:36:59.459811+0000","last_unstale":"2024-07-26T13:38:46.387645+0000","last_undegraded":"2024-07-26T13:38:46.387645+0000","last_fullsized":"2024-07-26T13:38:46.387645+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:46:57.112314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387993+0000","last_change":"2024-07-26T13:36:59.460197+0000","last_active":"2024-07-26T13:38:46.387993+0000","last_peered":"2024-07-26T13:38:46.387993+0000","last_clean":"2024-07-26T13:38:46.387993+0000","last_became_active":"2024-07-26T13:36:59.459686+0000","last_became_peered":"2024-07-26T13:36:59.459686+0000","last_unstale":"2024-07-26T13:38:46.387993+0000","last_undegraded":"2024-07-26T13:38:46.387993+0000","last_fullsized":"2024-07-26T13:38:46.387993+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T13:39:22.407376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.373506+0000","last_change":"2024-07-26T13:36:59.462545+0000","last_active":"2024-07-26T13:38:46.373506+0000","last_peered":"2024-07-26T13:38:46.373506+0000","last_clean":"2024-07-26T13:38:46.373506+0000","last_became_active":"2024-07-26T13:36:59.462115+0000","last_became_peered":"2024-07-26T13:36:59.462115+0000","last_unstale":"2024-07-26T13:38:46.373506+0000","last_undegraded":"2024-07-26T13:38:46.373506+0000","last_fullsized":"2024-07-26T13:38:46.373506+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T21:44:13.715076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.384982+0000","last_change":"2024-07-26T13:36:59.465317+0000","last_active":"2024-07-26T13:38:46.384982+0000","last_peered":"2024-07-26T13:38:46.384982+0000","last_clean":"2024-07-26T13:38:46.384982+0000","last_became_active":"2024-07-26T13:36:59.464943+0000","last_became_peered":"2024-07-26T13:36:59.464943+0000","last_unstale":"2024-07-26T13:38:46.384982+0000","last_undegraded":"2024-07-26T13:38:46.384982+0000","last_fullsized":"2024-07-26T13:38:46.384982+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T13:53:54.160584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.370482+0000","last_change":"2024-07-26T13:36:59.462884+0000","last_active":"2024-07-26T13:38:46.370482+0000","last_peered":"2024-07-26T13:38:46.370482+0000","last_clean":"2024-07-26T13:38:46.370482+0000","last_became_active":"2024-07-26T13:36:59.462564+0000","last_became_peered":"2024-07-26T13:36:59.462564+0000","last_unstale":"2024-07-26T13:38:46.370482+0000","last_undegraded":"2024-07-26T13:38:46.370482+0000","last_fullsized":"2024-07-26T13:38:46.370482+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:50:19.549148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387141+0000","last_change":"2024-07-26T13:36:59.468716+0000","last_active":"2024-07-26T13:38:46.387141+0000","last_peered":"2024-07-26T13:38:46.387141+0000","last_clean":"2024-07-26T13:38:46.387141+0000","last_became_active":"2024-07-26T13:36:59.468048+0000","last_became_peered":"2024-07-26T13:36:59.468048+0000","last_unstale":"2024-07-26T13:38:46.387141+0000","last_undegraded":"2024-07-26T13:38:46.387141+0000","last_fullsized":"2024-07-26T13:38:46.387141+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:51:29.510047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383235+0000","last_change":"2024-07-26T13:36:59.466565+0000","last_active":"2024-07-26T13:38:46.383235+0000","last_peered":"2024-07-26T13:38:46.383235+0000","last_clean":"2024-07-26T13:38:46.383235+0000","last_became_active":"2024-07-26T13:36:59.466235+0000","last_became_peered":"2024-07-26T13:36:59.466235+0000","last_unstale":"2024-07-26T13:38:46.383235+0000","last_undegraded":"2024-07-26T13:38:46.383235+0000","last_fullsized":"2024-07-26T13:38:46.383235+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:28:44.326821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.371918+0000","last_change":"2024-07-26T13:36:59.458753+0000","last_active":"2024-07-26T13:38:46.371918+0000","last_peered":"2024-07-26T13:38:46.371918+0000","last_clean":"2024-07-26T13:38:46.371918+0000","last_became_active":"2024-07-26T13:36:59.458333+0000","last_became_peered":"2024-07-26T13:36:59.458333+0000","last_unstale":"2024-07-26T13:38:46.371918+0000","last_undegraded":"2024-07-26T13:38:46.371918+0000","last_fullsized":"2024-07-26T13:38:46.371918+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:33:54.806212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.369463+0000","last_change":"2024-07-26T13:36:59.468657+0000","last_active":"2024-07-26T13:38:46.369463+0000","last_peered":"2024-07-26T13:38:46.369463+0000","last_clean":"2024-07-26T13:38:46.369463+0000","last_became_active":"2024-07-26T13:36:59.468121+0000","last_became_peered":"2024-07-26T13:36:59.468121+0000","last_unstale":"2024-07-26T13:38:46.369463+0000","last_undegraded":"2024-07-26T13:38:46.369463+0000","last_fullsized":"2024-07-26T13:38:46.369463+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:23:53.959836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.377710+0000","last_change":"2024-07-26T13:36:59.469067+0000","last_active":"2024-07-26T13:38:46.377710+0000","last_peered":"2024-07-26T13:38:46.377710+0000","last_clean":"2024-07-26T13:38:46.377710+0000","last_became_active":"2024-07-26T13:36:59.468706+0000","last_became_peered":"2024-07-26T13:36:59.468706+0000","last_unstale":"2024-07-26T13:38:46.377710+0000","last_undegraded":"2024-07-26T13:38:46.377710+0000","last_fullsized":"2024-07-26T13:38:46.377710+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:21:35.974618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.378393+0000","last_change":"2024-07-26T13:36:59.469807+0000","last_active":"2024-07-26T13:38:46.378393+0000","last_peered":"2024-07-26T13:38:46.378393+0000","last_clean":"2024-07-26T13:38:46.378393+0000","last_became_active":"2024-07-26T13:36:59.469020+0000","last_became_peered":"2024-07-26T13:36:59.469020+0000","last_unstale":"2024-07-26T13:38:46.378393+0000","last_undegraded":"2024-07-26T13:38:46.378393+0000","last_fullsized":"2024-07-26T13:38:46.378393+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:29:57.931598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.373015+0000","last_change":"2024-07-26T13:36:59.465236+0000","last_active":"2024-07-26T13:38:46.373015+0000","last_peered":"2024-07-26T13:38:46.373015+0000","last_clean":"2024-07-26T13:38:46.373015+0000","last_became_active":"2024-07-26T13:36:59.464415+0000","last_became_peered":"2024-07-26T13:36:59.464415+0000","last_unstale":"2024-07-26T13:38:46.373015+0000","last_undegraded":"2024-07-26T13:38:46.373015+0000","last_fullsized":"2024-07-26T13:38:46.373015+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:46:20.373688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.375987+0000","last_change":"2024-07-26T13:36:59.466113+0000","last_active":"2024-07-26T13:38:46.375987+0000","last_peered":"2024-07-26T13:38:46.375987+0000","last_clean":"2024-07-26T13:38:46.375987+0000","last_became_active":"2024-07-26T13:36:59.465743+0000","last_became_peered":"2024-07-26T13:36:59.465743+0000","last_unstale":"2024-07-26T13:38:46.375987+0000","last_undegraded":"2024-07-26T13:38:46.375987+0000","last_fullsized":"2024-07-26T13:38:46.375987+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:14:41.839204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.380451+0000","last_change":"2024-07-26T13:36:59.460672+0000","last_active":"2024-07-26T13:38:46.380451+0000","last_peered":"2024-07-26T13:38:46.380451+0000","last_clean":"2024-07-26T13:38:46.380451+0000","last_became_active":"2024-07-26T13:36:59.460247+0000","last_became_peered":"2024-07-26T13:36:59.460247+0000","last_unstale":"2024-07-26T13:38:46.380451+0000","last_undegraded":"2024-07-26T13:38:46.380451+0000","last_fullsized":"2024-07-26T13:38:46.380451+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:37:51.109933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.378083+0000","last_change":"2024-07-26T13:36:59.464784+0000","last_active":"2024-07-26T13:38:46.378083+0000","last_peered":"2024-07-26T13:38:46.378083+0000","last_clean":"2024-07-26T13:38:46.378083+0000","last_became_active":"2024-07-26T13:36:59.464261+0000","last_became_peered":"2024-07-26T13:36:59.464261+0000","last_unstale":"2024-07-26T13:38:46.378083+0000","last_undegraded":"2024-07-26T13:38:46.378083+0000","last_fullsized":"2024-07-26T13:38:46.378083+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:43:10.134037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.385005+0000","last_change":"2024-07-26T13:36:59.460421+0000","last_active":"2024-07-26T13:38:46.385005+0000","last_peered":"2024-07-26T13:38:46.385005+0000","last_clean":"2024-07-26T13:38:46.385005+0000","last_became_active":"2024-07-26T13:36:59.460010+0000","last_became_peered":"2024-07-26T13:36:59.460010+0000","last_unstale":"2024-07-26T13:38:46.385005+0000","last_undegraded":"2024-07-26T13:38:46.385005+0000","last_fullsized":"2024-07-26T13:38:46.385005+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:16:21.651673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.358312+0000","last_change":"2024-07-26T13:36:59.468826+0000","last_active":"2024-07-26T13:38:46.358312+0000","last_peered":"2024-07-26T13:38:46.358312+0000","last_clean":"2024-07-26T13:38:46.358312+0000","last_became_active":"2024-07-26T13:36:59.468406+0000","last_became_peered":"2024-07-26T13:36:59.468406+0000","last_unstale":"2024-07-26T13:38:46.358312+0000","last_undegraded":"2024-07-26T13:38:46.358312+0000","last_fullsized":"2024-07-26T13:38:46.358312+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:22:10.590545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'7","reported_seq":227,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:40:10.849058+0000","last_change":"2024-07-26T13:37:17.559014+0000","last_active":"2024-07-26T13:40:10.849058+0000","last_peered":"2024-07-26T13:40:10.849058+0000","last_clean":"2024-07-26T13:40:10.849058+0000","last_became_active":"2024-07-26T13:37:17.552325+0000","last_became_peered":"2024-07-26T13:37:17.552325+0000","last_unstale":"2024-07-26T13:40:10.849058+0000","last_undegraded":"2024-07-26T13:40:10.849058+0000","last_fullsized":"2024-07-26T13:40:10.849058+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:26:26.790337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":179,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.379560+0000","last_change":"2024-07-26T13:37:17.562871+0000","last_active":"2024-07-26T13:38:46.379560+0000","last_peered":"2024-07-26T13:38:46.379560+0000","last_clean":"2024-07-26T13:38:46.379560+0000","last_became_active":"2024-07-26T13:37:17.562363+0000","last_became_peered":"2024-07-26T13:37:17.562363+0000","last_unstale":"2024-07-26T13:38:46.379560+0000","last_undegraded":"2024-07-26T13:38:46.379560+0000","last_fullsized":"2024-07-26T13:38:46.379560+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:25:36.172091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"79'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.547762+0000","last_change":"2024-07-26T13:36:56.434778+0000","last_active":"2024-07-26T13:37:17.547762+0000","last_peered":"2024-07-26T13:37:17.547762+0000","last_clean":"2024-07-26T13:37:17.547762+0000","last_became_active":"2024-07-26T13:36:56.434288+0000","last_became_peered":"2024-07-26T13:36:56.434288+0000","last_unstale":"2024-07-26T13:37:17.547762+0000","last_undegraded":"2024-07-26T13:37:17.547762+0000","last_fullsized":"2024-07-26T13:37:17.547762+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:24:22.990358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.565163+0000","last_change":"2024-07-26T13:36:56.424665+0000","last_active":"2024-07-26T13:37:17.565163+0000","last_peered":"2024-07-26T13:37:17.565163+0000","last_clean":"2024-07-26T13:37:17.565163+0000","last_became_active":"2024-07-26T13:36:56.424290+0000","last_became_peered":"2024-07-26T13:36:56.424290+0000","last_unstale":"2024-07-26T13:37:17.565163+0000","last_undegraded":"2024-07-26T13:37:17.565163+0000","last_fullsized":"2024-07-26T13:37:17.565163+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:49:59.520194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383766+0000","last_change":"2024-07-26T13:36:59.454609+0000","last_active":"2024-07-26T13:38:46.383766+0000","last_peered":"2024-07-26T13:38:46.383766+0000","last_clean":"2024-07-26T13:38:46.383766+0000","last_became_active":"2024-07-26T13:36:59.454228+0000","last_became_peered":"2024-07-26T13:36:59.454228+0000","last_unstale":"2024-07-26T13:38:46.383766+0000","last_undegraded":"2024-07-26T13:38:46.383766+0000","last_fullsized":"2024-07-26T13:38:46.383766+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:02:06.244433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.547136+0000","last_change":"2024-07-26T13:36:56.430268+0000","last_active":"2024-07-26T13:37:17.547136+0000","last_peered":"2024-07-26T13:37:17.547136+0000","last_clean":"2024-07-26T13:37:17.547136+0000","last_became_active":"2024-07-26T13:36:56.428583+0000","last_became_peered":"2024-07-26T13:36:56.428583+0000","last_unstale":"2024-07-26T13:37:17.547136+0000","last_undegraded":"2024-07-26T13:37:17.547136+0000","last_fullsized":"2024-07-26T13:37:17.547136+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:07:05.864886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.379724+0000","last_change":"2024-07-26T13:36:59.456188+0000","last_active":"2024-07-26T13:38:46.379724+0000","last_peered":"2024-07-26T13:38:46.379724+0000","last_clean":"2024-07-26T13:38:46.379724+0000","last_became_active":"2024-07-26T13:36:59.455842+0000","last_became_peered":"2024-07-26T13:36:59.455842+0000","last_unstale":"2024-07-26T13:38:46.379724+0000","last_undegraded":"2024-07-26T13:38:46.379724+0000","last_fullsized":"2024-07-26T13:38:46.379724+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:00:19.162239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":207,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:40:11.849930+0000","last_change":"2024-07-26T13:36:56.432818+0000","last_active":"2024-07-26T13:40:11.849930+0000","last_peered":"2024-07-26T13:40:11.849930+0000","last_clean":"2024-07-26T13:40:11.849930+0000","last_became_active":"2024-07-26T13:36:56.432419+0000","last_became_peered":"2024-07-26T13:36:56.432419+0000","last_unstale":"2024-07-26T13:40:11.849930+0000","last_undegraded":"2024-07-26T13:40:11.849930+0000","last_fullsized":"2024-07-26T13:40:11.849930+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:55:51.972470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":176,"num_read_kb":176,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.386929+0000","last_change":"2024-07-26T13:36:59.460033+0000","last_active":"2024-07-26T13:38:46.386929+0000","last_peered":"2024-07-26T13:38:46.386929+0000","last_clean":"2024-07-26T13:38:46.386929+0000","last_became_active":"2024-07-26T13:36:59.459562+0000","last_became_peered":"2024-07-26T13:36:59.459562+0000","last_unstale":"2024-07-26T13:38:46.386929+0000","last_undegraded":"2024-07-26T13:38:46.386929+0000","last_fullsized":"2024-07-26T13:38:46.386929+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:58:14.155973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551399+0000","last_change":"2024-07-26T13:36:56.432563+0000","last_active":"2024-07-26T13:37:17.551399+0000","last_peered":"2024-07-26T13:37:17.551399+0000","last_clean":"2024-07-26T13:37:17.551399+0000","last_became_active":"2024-07-26T13:36:56.431134+0000","last_became_peered":"2024-07-26T13:36:56.431134+0000","last_unstale":"2024-07-26T13:37:17.551399+0000","last_undegraded":"2024-07-26T13:37:17.551399+0000","last_fullsized":"2024-07-26T13:37:17.551399+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:42:00.653345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.377839+0000","last_change":"2024-07-26T13:36:59.469782+0000","last_active":"2024-07-26T13:38:46.377839+0000","last_peered":"2024-07-26T13:38:46.377839+0000","last_clean":"2024-07-26T13:38:46.377839+0000","last_became_active":"2024-07-26T13:36:59.469412+0000","last_became_peered":"2024-07-26T13:36:59.469412+0000","last_unstale":"2024-07-26T13:38:46.377839+0000","last_undegraded":"2024-07-26T13:38:46.377839+0000","last_fullsized":"2024-07-26T13:38:46.377839+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:28:07.741486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.558844+0000","last_change":"2024-07-26T13:36:56.488981+0000","last_active":"2024-07-26T13:37:17.558844+0000","last_peered":"2024-07-26T13:37:17.558844+0000","last_clean":"2024-07-26T13:37:17.558844+0000","last_became_active":"2024-07-26T13:36:56.488480+0000","last_became_peered":"2024-07-26T13:36:56.488480+0000","last_unstale":"2024-07-26T13:37:17.558844+0000","last_undegraded":"2024-07-26T13:37:17.558844+0000","last_fullsized":"2024-07-26T13:37:17.558844+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:41:10.041007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387010+0000","last_change":"2024-07-26T13:36:59.473183+0000","last_active":"2024-07-26T13:38:46.387010+0000","last_peered":"2024-07-26T13:38:46.387010+0000","last_clean":"2024-07-26T13:38:46.387010+0000","last_became_active":"2024-07-26T13:36:59.472814+0000","last_became_peered":"2024-07-26T13:36:59.472814+0000","last_unstale":"2024-07-26T13:38:46.387010+0000","last_undegraded":"2024-07-26T13:38:46.387010+0000","last_fullsized":"2024-07-26T13:38:46.387010+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:57:03.396384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'35","reported_seq":425,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:40:12.866875+0000","last_change":"2024-07-26T13:36:56.435034+0000","last_active":"2024-07-26T13:40:12.866875+0000","last_peered":"2024-07-26T13:40:12.866875+0000","last_clean":"2024-07-26T13:40:12.866875+0000","last_became_active":"2024-07-26T13:36:56.434611+0000","last_became_peered":"2024-07-26T13:36:56.434611+0000","last_unstale":"2024-07-26T13:40:12.866875+0000","last_undegraded":"2024-07-26T13:40:12.866875+0000","last_fullsized":"2024-07-26T13:40:12.866875+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:18:27.272077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68265,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":363,"num_read_kb":248,"num_write":36,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.382422+0000","last_change":"2024-07-26T13:36:59.465831+0000","last_active":"2024-07-26T13:38:46.382422+0000","last_peered":"2024-07-26T13:38:46.382422+0000","last_clean":"2024-07-26T13:38:46.382422+0000","last_became_active":"2024-07-26T13:36:59.465404+0000","last_became_peered":"2024-07-26T13:36:59.465404+0000","last_unstale":"2024-07-26T13:38:46.382422+0000","last_undegraded":"2024-07-26T13:38:46.382422+0000","last_fullsized":"2024-07-26T13:38:46.382422+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:42:53.795926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'6","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.552497+0000","last_change":"2024-07-26T13:36:56.435170+0000","last_active":"2024-07-26T13:37:17.552497+0000","last_peered":"2024-07-26T13:37:17.552497+0000","last_clean":"2024-07-26T13:37:17.552497+0000","last_became_active":"2024-07-26T13:36:56.434715+0000","last_became_peered":"2024-07-26T13:36:56.434715+0000","last_unstale":"2024-07-26T13:37:17.552497+0000","last_undegraded":"2024-07-26T13:37:17.552497+0000","last_fullsized":"2024-07-26T13:37:17.552497+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:05:09.712510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.370726+0000","last_change":"2024-07-26T13:36:59.461926+0000","last_active":"2024-07-26T13:38:46.370726+0000","last_peered":"2024-07-26T13:38:46.370726+0000","last_clean":"2024-07-26T13:38:46.370726+0000","last_became_active":"2024-07-26T13:36:59.461512+0000","last_became_peered":"2024-07-26T13:36:59.461512+0000","last_unstale":"2024-07-26T13:38:46.370726+0000","last_undegraded":"2024-07-26T13:38:46.370726+0000","last_fullsized":"2024-07-26T13:38:46.370726+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:34:24.405663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.546907+0000","last_change":"2024-07-26T13:36:56.430062+0000","last_active":"2024-07-26T13:37:17.546907+0000","last_peered":"2024-07-26T13:37:17.546907+0000","last_clean":"2024-07-26T13:37:17.546907+0000","last_became_active":"2024-07-26T13:36:56.429654+0000","last_became_peered":"2024-07-26T13:36:56.429654+0000","last_unstale":"2024-07-26T13:37:17.546907+0000","last_undegraded":"2024-07-26T13:37:17.546907+0000","last_fullsized":"2024-07-26T13:37:17.546907+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:21:09.514359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.380005+0000","last_change":"2024-07-26T13:36:59.474459+0000","last_active":"2024-07-26T13:38:46.380005+0000","last_peered":"2024-07-26T13:38:46.380005+0000","last_clean":"2024-07-26T13:38:46.380005+0000","last_became_active":"2024-07-26T13:36:59.474061+0000","last_became_peered":"2024-07-26T13:36:59.474061+0000","last_unstale":"2024-07-26T13:38:46.380005+0000","last_undegraded":"2024-07-26T13:38:46.380005+0000","last_fullsized":"2024-07-26T13:38:46.380005+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:37:21.691491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'5","reported_seq":213,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:40:11.849698+0000","last_change":"2024-07-26T13:36:56.432915+0000","last_active":"2024-07-26T13:40:11.849698+0000","last_peered":"2024-07-26T13:40:11.849698+0000","last_clean":"2024-07-26T13:40:11.849698+0000","last_became_active":"2024-07-26T13:36:56.432499+0000","last_became_peered":"2024-07-26T13:36:56.432499+0000","last_unstale":"2024-07-26T13:40:11.849698+0000","last_undegraded":"2024-07-26T13:40:11.849698+0000","last_fullsized":"2024-07-26T13:40:11.849698+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:02:14.306950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":182,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383574+0000","last_change":"2024-07-26T13:36:59.468945+0000","last_active":"2024-07-26T13:38:46.383574+0000","last_peered":"2024-07-26T13:38:46.383574+0000","last_clean":"2024-07-26T13:38:46.383574+0000","last_became_active":"2024-07-26T13:36:59.467970+0000","last_became_peered":"2024-07-26T13:36:59.467970+0000","last_unstale":"2024-07-26T13:38:46.383574+0000","last_undegraded":"2024-07-26T13:38:46.383574+0000","last_fullsized":"2024-07-26T13:38:46.383574+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T13:59:19.514945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'15","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551412+0000","last_change":"2024-07-26T13:36:56.487014+0000","last_active":"2024-07-26T13:37:17.551412+0000","last_peered":"2024-07-26T13:37:17.551412+0000","last_clean":"2024-07-26T13:37:17.551412+0000","last_became_active":"2024-07-26T13:36:56.486470+0000","last_became_peered":"2024-07-26T13:36:56.486470+0000","last_unstale":"2024-07-26T13:37:17.551412+0000","last_undegraded":"2024-07-26T13:37:17.551412+0000","last_fullsized":"2024-07-26T13:37:17.551412+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:07:02.205644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.366501+0000","last_change":"2024-07-26T13:36:59.462465+0000","last_active":"2024-07-26T13:38:46.366501+0000","last_peered":"2024-07-26T13:38:46.366501+0000","last_clean":"2024-07-26T13:38:46.366501+0000","last_became_active":"2024-07-26T13:36:59.462024+0000","last_became_peered":"2024-07-26T13:36:59.462024+0000","last_unstale":"2024-07-26T13:38:46.366501+0000","last_undegraded":"2024-07-26T13:38:46.366501+0000","last_fullsized":"2024-07-26T13:38:46.366501+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:34:14.317762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551992+0000","last_change":"2024-07-26T13:36:56.435007+0000","last_active":"2024-07-26T13:37:17.551992+0000","last_peered":"2024-07-26T13:37:17.551992+0000","last_clean":"2024-07-26T13:37:17.551992+0000","last_became_active":"2024-07-26T13:36:56.434444+0000","last_became_peered":"2024-07-26T13:36:56.434444+0000","last_unstale":"2024-07-26T13:37:17.551992+0000","last_undegraded":"2024-07-26T13:37:17.551992+0000","last_fullsized":"2024-07-26T13:37:17.551992+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:27:51.041349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.384490+0000","last_change":"2024-07-26T13:36:59.464164+0000","last_active":"2024-07-26T13:38:46.384490+0000","last_peered":"2024-07-26T13:38:46.384490+0000","last_clean":"2024-07-26T13:38:46.384490+0000","last_became_active":"2024-07-26T13:36:59.463782+0000","last_became_peered":"2024-07-26T13:36:59.463782+0000","last_unstale":"2024-07-26T13:38:46.384490+0000","last_undegraded":"2024-07-26T13:38:46.384490+0000","last_fullsized":"2024-07-26T13:38:46.384490+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:48:06.481731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.565510+0000","last_change":"2024-07-26T13:36:56.429653+0000","last_active":"2024-07-26T13:37:17.565510+0000","last_peered":"2024-07-26T13:37:17.565510+0000","last_clean":"2024-07-26T13:37:17.565510+0000","last_became_active":"2024-07-26T13:36:56.429060+0000","last_became_peered":"2024-07-26T13:36:56.429060+0000","last_unstale":"2024-07-26T13:37:17.565510+0000","last_undegraded":"2024-07-26T13:37:17.565510+0000","last_fullsized":"2024-07-26T13:37:17.565510+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:41:48.261709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.384430+0000","last_change":"2024-07-26T13:36:59.471560+0000","last_active":"2024-07-26T13:38:46.384430+0000","last_peered":"2024-07-26T13:38:46.384430+0000","last_clean":"2024-07-26T13:38:46.384430+0000","last_became_active":"2024-07-26T13:36:59.471151+0000","last_became_peered":"2024-07-26T13:36:59.471151+0000","last_unstale":"2024-07-26T13:38:46.384430+0000","last_undegraded":"2024-07-26T13:38:46.384430+0000","last_fullsized":"2024-07-26T13:38:46.384430+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:49:20.513308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.548185+0000","last_change":"2024-07-26T13:36:56.430010+0000","last_active":"2024-07-26T13:37:17.548185+0000","last_peered":"2024-07-26T13:37:17.548185+0000","last_clean":"2024-07-26T13:37:17.548185+0000","last_became_active":"2024-07-26T13:36:56.429120+0000","last_became_peered":"2024-07-26T13:36:56.429120+0000","last_unstale":"2024-07-26T13:37:17.548185+0000","last_undegraded":"2024-07-26T13:37:17.548185+0000","last_fullsized":"2024-07-26T13:37:17.548185+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:23:43.321975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383589+0000","last_change":"2024-07-26T13:36:59.467788+0000","last_active":"2024-07-26T13:38:46.383589+0000","last_peered":"2024-07-26T13:38:46.383589+0000","last_clean":"2024-07-26T13:38:46.383589+0000","last_became_active":"2024-07-26T13:36:59.467399+0000","last_became_peered":"2024-07-26T13:36:59.467399+0000","last_unstale":"2024-07-26T13:38:46.383589+0000","last_undegraded":"2024-07-26T13:38:46.383589+0000","last_fullsized":"2024-07-26T13:38:46.383589+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:48:51.442999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'3","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.553308+0000","last_change":"2024-07-26T13:36:56.588633+0000","last_active":"2024-07-26T13:37:17.553308+0000","last_peered":"2024-07-26T13:37:17.553308+0000","last_clean":"2024-07-26T13:37:17.553308+0000","last_became_active":"2024-07-26T13:36:56.588113+0000","last_became_peered":"2024-07-26T13:36:56.588113+0000","last_unstale":"2024-07-26T13:37:17.553308+0000","last_undegraded":"2024-07-26T13:37:17.553308+0000","last_fullsized":"2024-07-26T13:37:17.553308+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:03:55.075052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387531+0000","last_change":"2024-07-26T13:36:59.472854+0000","last_active":"2024-07-26T13:38:46.387531+0000","last_peered":"2024-07-26T13:38:46.387531+0000","last_clean":"2024-07-26T13:38:46.387531+0000","last_became_active":"2024-07-26T13:36:59.472466+0000","last_became_peered":"2024-07-26T13:36:59.472466+0000","last_unstale":"2024-07-26T13:38:46.387531+0000","last_undegraded":"2024-07-26T13:38:46.387531+0000","last_fullsized":"2024-07-26T13:38:46.387531+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:50:57.454149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":103,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550829+0000","last_change":"2024-07-26T13:35:34.839121+0000","last_active":"2024-07-26T13:37:17.550829+0000","last_peered":"2024-07-26T13:37:17.550829+0000","last_clean":"2024-07-26T13:37:17.550829+0000","last_became_active":"2024-07-26T13:35:34.833465+0000","last_became_peered":"2024-07-26T13:35:34.833465+0000","last_unstale":"2024-07-26T13:37:17.550829+0000","last_undegraded":"2024-07-26T13:37:17.550829+0000","last_fullsized":"2024-07-26T13:37:17.550829+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:32:23.285428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:32:23.285428+0000","last_clean_scrub_stamp":"2024-07-26T13:32:23.285428+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:29:31.650487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'3","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.561459+0000","last_change":"2024-07-26T13:36:56.428648+0000","last_active":"2024-07-26T13:37:17.561459+0000","last_peered":"2024-07-26T13:37:17.561459+0000","last_clean":"2024-07-26T13:37:17.561459+0000","last_became_active":"2024-07-26T13:36:56.427665+0000","last_became_peered":"2024-07-26T13:36:56.427665+0000","last_unstale":"2024-07-26T13:37:17.561459+0000","last_undegraded":"2024-07-26T13:37:17.561459+0000","last_fullsized":"2024-07-26T13:37:17.561459+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:41:39.371687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.376696+0000","last_change":"2024-07-26T13:36:59.460625+0000","last_active":"2024-07-26T13:38:46.376696+0000","last_peered":"2024-07-26T13:38:46.376696+0000","last_clean":"2024-07-26T13:38:46.376696+0000","last_became_active":"2024-07-26T13:36:59.458658+0000","last_became_peered":"2024-07-26T13:36:59.458658+0000","last_unstale":"2024-07-26T13:38:46.376696+0000","last_undegraded":"2024-07-26T13:38:46.376696+0000","last_fullsized":"2024-07-26T13:38:46.376696+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:46:42.812402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'3","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550796+0000","last_change":"2024-07-26T13:36:56.432617+0000","last_active":"2024-07-26T13:37:17.550796+0000","last_peered":"2024-07-26T13:37:17.550796+0000","last_clean":"2024-07-26T13:37:17.550796+0000","last_became_active":"2024-07-26T13:36:56.431519+0000","last_became_peered":"2024-07-26T13:36:56.431519+0000","last_unstale":"2024-07-26T13:37:17.550796+0000","last_undegraded":"2024-07-26T13:37:17.550796+0000","last_fullsized":"2024-07-26T13:37:17.550796+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:32:21.120076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.364664+0000","last_change":"2024-07-26T13:36:59.464084+0000","last_active":"2024-07-26T13:38:46.364664+0000","last_peered":"2024-07-26T13:38:46.364664+0000","last_clean":"2024-07-26T13:38:46.364664+0000","last_became_active":"2024-07-26T13:36:59.463755+0000","last_became_peered":"2024-07-26T13:36:59.463755+0000","last_unstale":"2024-07-26T13:38:46.364664+0000","last_undegraded":"2024-07-26T13:38:46.364664+0000","last_fullsized":"2024-07-26T13:38:46.364664+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:53:20.034188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":209,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:40:10.847516+0000","last_change":"2024-07-26T13:36:56.486071+0000","last_active":"2024-07-26T13:40:10.847516+0000","last_peered":"2024-07-26T13:40:10.847516+0000","last_clean":"2024-07-26T13:40:10.847516+0000","last_became_active":"2024-07-26T13:36:56.485672+0000","last_became_peered":"2024-07-26T13:36:56.485672+0000","last_unstale":"2024-07-26T13:40:10.847516+0000","last_undegraded":"2024-07-26T13:40:10.847516+0000","last_fullsized":"2024-07-26T13:40:10.847516+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T21:29:45.666651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":176,"num_read_kb":176,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.372811+0000","last_change":"2024-07-26T13:37:17.566555+0000","last_active":"2024-07-26T13:38:46.372811+0000","last_peered":"2024-07-26T13:38:46.372811+0000","last_clean":"2024-07-26T13:38:46.372811+0000","last_became_active":"2024-07-26T13:37:17.566040+0000","last_became_peered":"2024-07-26T13:37:17.566040+0000","last_unstale":"2024-07-26T13:38:46.372811+0000","last_undegraded":"2024-07-26T13:38:46.372811+0000","last_fullsized":"2024-07-26T13:38:46.372811+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:48:33.078890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550989+0000","last_change":"2024-07-26T13:36:56.431811+0000","last_active":"2024-07-26T13:37:17.550989+0000","last_peered":"2024-07-26T13:37:17.550989+0000","last_clean":"2024-07-26T13:37:17.550989+0000","last_became_active":"2024-07-26T13:36:56.431223+0000","last_became_peered":"2024-07-26T13:36:56.431223+0000","last_unstale":"2024-07-26T13:37:17.550989+0000","last_undegraded":"2024-07-26T13:37:17.550989+0000","last_fullsized":"2024-07-26T13:37:17.550989+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:08:00.334515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.363881+0000","last_change":"2024-07-26T13:36:59.463972+0000","last_active":"2024-07-26T13:38:46.363881+0000","last_peered":"2024-07-26T13:38:46.363881+0000","last_clean":"2024-07-26T13:38:46.363881+0000","last_became_active":"2024-07-26T13:36:59.463496+0000","last_became_peered":"2024-07-26T13:36:59.463496+0000","last_unstale":"2024-07-26T13:38:46.363881+0000","last_undegraded":"2024-07-26T13:38:46.363881+0000","last_fullsized":"2024-07-26T13:38:46.363881+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:02:06.215372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.545780+0000","last_change":"2024-07-26T13:36:56.486681+0000","last_active":"2024-07-26T13:37:17.545780+0000","last_peered":"2024-07-26T13:37:17.545780+0000","last_clean":"2024-07-26T13:37:17.545780+0000","last_became_active":"2024-07-26T13:36:56.486328+0000","last_became_peered":"2024-07-26T13:36:56.486328+0000","last_unstale":"2024-07-26T13:37:17.545780+0000","last_undegraded":"2024-07-26T13:37:17.545780+0000","last_fullsized":"2024-07-26T13:37:17.545780+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:10:36.557707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.355361+0000","last_change":"2024-07-26T13:36:59.451415+0000","last_active":"2024-07-26T13:38:46.355361+0000","last_peered":"2024-07-26T13:38:46.355361+0000","last_clean":"2024-07-26T13:38:46.355361+0000","last_became_active":"2024-07-26T13:36:59.450992+0000","last_became_peered":"2024-07-26T13:36:59.450992+0000","last_unstale":"2024-07-26T13:38:46.355361+0000","last_undegraded":"2024-07-26T13:38:46.355361+0000","last_fullsized":"2024-07-26T13:38:46.355361+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:29:11.303916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.557777+0000","last_change":"2024-07-26T13:37:17.555643+0000","last_active":"2024-07-26T13:37:17.557777+0000","last_peered":"2024-07-26T13:37:17.557777+0000","last_clean":"2024-07-26T13:37:17.557777+0000","last_became_active":"2024-07-26T13:37:17.549756+0000","last_became_peered":"2024-07-26T13:37:17.549756+0000","last_unstale":"2024-07-26T13:37:17.557777+0000","last_undegraded":"2024-07-26T13:37:17.557777+0000","last_fullsized":"2024-07-26T13:37:17.557777+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:20:16.911286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.368534+0000","last_change":"2024-07-26T13:36:59.465538+0000","last_active":"2024-07-26T13:38:46.368534+0000","last_peered":"2024-07-26T13:38:46.368534+0000","last_clean":"2024-07-26T13:38:46.368534+0000","last_became_active":"2024-07-26T13:36:59.464886+0000","last_became_peered":"2024-07-26T13:36:59.464886+0000","last_unstale":"2024-07-26T13:38:46.368534+0000","last_undegraded":"2024-07-26T13:38:46.368534+0000","last_fullsized":"2024-07-26T13:38:46.368534+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:19:12.273116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'13","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:39:08.749398+0000","last_change":"2024-07-26T13:36:56.428355+0000","last_active":"2024-07-26T13:39:08.749398+0000","last_peered":"2024-07-26T13:39:08.749398+0000","last_clean":"2024-07-26T13:39:08.749398+0000","last_became_active":"2024-07-26T13:36:56.427213+0000","last_became_peered":"2024-07-26T13:36:56.427213+0000","last_unstale":"2024-07-26T13:39:08.749398+0000","last_undegraded":"2024-07-26T13:39:08.749398+0000","last_fullsized":"2024-07-26T13:39:08.749398+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:48:09.730067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.378498+0000","last_change":"2024-07-26T13:36:59.459378+0000","last_active":"2024-07-26T13:38:46.378498+0000","last_peered":"2024-07-26T13:38:46.378498+0000","last_clean":"2024-07-26T13:38:46.378498+0000","last_became_active":"2024-07-26T13:36:59.458799+0000","last_became_peered":"2024-07-26T13:36:59.458799+0000","last_unstale":"2024-07-26T13:38:46.378498+0000","last_undegraded":"2024-07-26T13:38:46.378498+0000","last_fullsized":"2024-07-26T13:38:46.378498+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:45:19.587800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.561441+0000","last_change":"2024-07-26T13:36:56.588819+0000","last_active":"2024-07-26T13:37:17.561441+0000","last_peered":"2024-07-26T13:37:17.561441+0000","last_clean":"2024-07-26T13:37:17.561441+0000","last_became_active":"2024-07-26T13:36:56.588430+0000","last_became_peered":"2024-07-26T13:36:56.588430+0000","last_unstale":"2024-07-26T13:37:17.561441+0000","last_undegraded":"2024-07-26T13:37:17.561441+0000","last_fullsized":"2024-07-26T13:37:17.561441+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:07:21.387618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.380460+0000","last_change":"2024-07-26T13:37:17.562040+0000","last_active":"2024-07-26T13:38:46.380460+0000","last_peered":"2024-07-26T13:38:46.380460+0000","last_clean":"2024-07-26T13:38:46.380460+0000","last_became_active":"2024-07-26T13:37:17.561542+0000","last_became_peered":"2024-07-26T13:37:17.561542+0000","last_unstale":"2024-07-26T13:38:46.380460+0000","last_undegraded":"2024-07-26T13:38:46.380460+0000","last_fullsized":"2024-07-26T13:38:46.380460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:21:08.161954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.544169+0000","last_change":"2024-07-26T13:36:56.489106+0000","last_active":"2024-07-26T13:37:17.544169+0000","last_peered":"2024-07-26T13:37:17.544169+0000","last_clean":"2024-07-26T13:37:17.544169+0000","last_became_active":"2024-07-26T13:36:56.488706+0000","last_became_peered":"2024-07-26T13:36:56.488706+0000","last_unstale":"2024-07-26T13:37:17.544169+0000","last_undegraded":"2024-07-26T13:37:17.544169+0000","last_fullsized":"2024-07-26T13:37:17.544169+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:00:59.204434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.343705+0000","last_change":"2024-07-26T13:36:59.470412+0000","last_active":"2024-07-26T13:38:46.343705+0000","last_peered":"2024-07-26T13:38:46.343705+0000","last_clean":"2024-07-26T13:38:46.343705+0000","last_became_active":"2024-07-26T13:36:59.470010+0000","last_became_peered":"2024-07-26T13:36:59.470010+0000","last_unstale":"2024-07-26T13:38:46.343705+0000","last_undegraded":"2024-07-26T13:38:46.343705+0000","last_fullsized":"2024-07-26T13:38:46.343705+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:50:57.987600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.543528+0000","last_change":"2024-07-26T13:36:56.428671+0000","last_active":"2024-07-26T13:37:17.543528+0000","last_peered":"2024-07-26T13:37:17.543528+0000","last_clean":"2024-07-26T13:37:17.543528+0000","last_became_active":"2024-07-26T13:36:56.428301+0000","last_became_peered":"2024-07-26T13:36:56.428301+0000","last_unstale":"2024-07-26T13:37:17.543528+0000","last_undegraded":"2024-07-26T13:37:17.543528+0000","last_fullsized":"2024-07-26T13:37:17.543528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:59:19.593664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.385021+0000","last_change":"2024-07-26T13:36:59.458627+0000","last_active":"2024-07-26T13:38:46.385021+0000","last_peered":"2024-07-26T13:38:46.385021+0000","last_clean":"2024-07-26T13:38:46.385021+0000","last_became_active":"2024-07-26T13:36:59.458199+0000","last_became_peered":"2024-07-26T13:36:59.458199+0000","last_unstale":"2024-07-26T13:38:46.385021+0000","last_undegraded":"2024-07-26T13:38:46.385021+0000","last_fullsized":"2024-07-26T13:38:46.385021+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:54:41.833223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551540+0000","last_change":"2024-07-26T13:36:56.431548+0000","last_active":"2024-07-26T13:37:17.551540+0000","last_peered":"2024-07-26T13:37:17.551540+0000","last_clean":"2024-07-26T13:37:17.551540+0000","last_became_active":"2024-07-26T13:36:56.430989+0000","last_became_peered":"2024-07-26T13:36:56.430989+0000","last_unstale":"2024-07-26T13:37:17.551540+0000","last_undegraded":"2024-07-26T13:37:17.551540+0000","last_fullsized":"2024-07-26T13:37:17.551540+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:46:51.655324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.375762+0000","last_change":"2024-07-26T13:36:59.458290+0000","last_active":"2024-07-26T13:38:46.375762+0000","last_peered":"2024-07-26T13:38:46.375762+0000","last_clean":"2024-07-26T13:38:46.375762+0000","last_became_active":"2024-07-26T13:36:59.457942+0000","last_became_peered":"2024-07-26T13:36:59.457942+0000","last_unstale":"2024-07-26T13:38:46.375762+0000","last_undegraded":"2024-07-26T13:38:46.375762+0000","last_fullsized":"2024-07-26T13:38:46.375762+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:51:58.797441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":46,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.568059+0000","last_change":"2024-07-26T13:37:17.567624+0000","last_active":"2024-07-26T13:37:17.568059+0000","last_peered":"2024-07-26T13:37:17.568059+0000","last_clean":"2024-07-26T13:37:17.568059+0000","last_became_active":"2024-07-26T13:37:17.562455+0000","last_became_peered":"2024-07-26T13:37:17.562455+0000","last_unstale":"2024-07-26T13:37:17.568059+0000","last_undegraded":"2024-07-26T13:37:17.568059+0000","last_fullsized":"2024-07-26T13:37:17.568059+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:50:36.418750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387835+0000","last_change":"2024-07-26T13:36:59.474034+0000","last_active":"2024-07-26T13:38:46.387835+0000","last_peered":"2024-07-26T13:38:46.387835+0000","last_clean":"2024-07-26T13:38:46.387835+0000","last_became_active":"2024-07-26T13:36:59.472358+0000","last_became_peered":"2024-07-26T13:36:59.472358+0000","last_unstale":"2024-07-26T13:38:46.387835+0000","last_undegraded":"2024-07-26T13:38:46.387835+0000","last_fullsized":"2024-07-26T13:38:46.387835+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:36:44.565413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.368595+0000","last_change":"2024-07-26T13:36:59.465546+0000","last_active":"2024-07-26T13:38:46.368595+0000","last_peered":"2024-07-26T13:38:46.368595+0000","last_clean":"2024-07-26T13:38:46.368595+0000","last_became_active":"2024-07-26T13:36:59.465137+0000","last_became_peered":"2024-07-26T13:36:59.465137+0000","last_unstale":"2024-07-26T13:38:46.368595+0000","last_undegraded":"2024-07-26T13:38:46.368595+0000","last_fullsized":"2024-07-26T13:38:46.368595+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:35:13.591354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.562664+0000","last_change":"2024-07-26T13:37:17.562012+0000","last_active":"2024-07-26T13:37:17.562664+0000","last_peered":"2024-07-26T13:37:17.562664+0000","last_clean":"2024-07-26T13:37:17.562664+0000","last_became_active":"2024-07-26T13:37:17.550218+0000","last_became_peered":"2024-07-26T13:37:17.550218+0000","last_unstale":"2024-07-26T13:37:17.562664+0000","last_undegraded":"2024-07-26T13:37:17.562664+0000","last_fullsized":"2024-07-26T13:37:17.562664+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:46:16.559533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.375095+0000","last_change":"2024-07-26T13:36:59.456896+0000","last_active":"2024-07-26T13:38:46.375095+0000","last_peered":"2024-07-26T13:38:46.375095+0000","last_clean":"2024-07-26T13:38:46.375095+0000","last_became_active":"2024-07-26T13:36:59.456530+0000","last_became_peered":"2024-07-26T13:36:59.456530+0000","last_unstale":"2024-07-26T13:38:46.375095+0000","last_undegraded":"2024-07-26T13:38:46.375095+0000","last_fullsized":"2024-07-26T13:38:46.375095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:59:56.512501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'4","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.553596+0000","last_change":"2024-07-26T13:36:56.433119+0000","last_active":"2024-07-26T13:37:17.553596+0000","last_peered":"2024-07-26T13:37:17.553596+0000","last_clean":"2024-07-26T13:37:17.553596+0000","last_became_active":"2024-07-26T13:36:56.432765+0000","last_became_peered":"2024-07-26T13:36:56.432765+0000","last_unstale":"2024-07-26T13:37:17.553596+0000","last_undegraded":"2024-07-26T13:37:17.553596+0000","last_fullsized":"2024-07-26T13:37:17.553596+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:31:29.197106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.382051+0000","last_change":"2024-07-26T13:36:59.462125+0000","last_active":"2024-07-26T13:38:46.382051+0000","last_peered":"2024-07-26T13:38:46.382051+0000","last_clean":"2024-07-26T13:38:46.382051+0000","last_became_active":"2024-07-26T13:36:59.461796+0000","last_became_peered":"2024-07-26T13:36:59.461796+0000","last_unstale":"2024-07-26T13:38:46.382051+0000","last_undegraded":"2024-07-26T13:38:46.382051+0000","last_fullsized":"2024-07-26T13:38:46.382051+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:44:10.329046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.543646+0000","last_change":"2024-07-26T13:36:56.487251+0000","last_active":"2024-07-26T13:37:17.543646+0000","last_peered":"2024-07-26T13:37:17.543646+0000","last_clean":"2024-07-26T13:37:17.543646+0000","last_became_active":"2024-07-26T13:36:56.486918+0000","last_became_peered":"2024-07-26T13:36:56.486918+0000","last_unstale":"2024-07-26T13:37:17.543646+0000","last_undegraded":"2024-07-26T13:37:17.543646+0000","last_fullsized":"2024-07-26T13:37:17.543646+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:41:31.248895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.380839+0000","last_change":"2024-07-26T13:36:59.470810+0000","last_active":"2024-07-26T13:38:46.380839+0000","last_peered":"2024-07-26T13:38:46.380839+0000","last_clean":"2024-07-26T13:38:46.380839+0000","last_became_active":"2024-07-26T13:36:59.467371+0000","last_became_peered":"2024-07-26T13:36:59.467371+0000","last_unstale":"2024-07-26T13:38:46.380839+0000","last_undegraded":"2024-07-26T13:38:46.380839+0000","last_fullsized":"2024-07-26T13:38:46.380839+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:23:48.572933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'9","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.566327+0000","last_change":"2024-07-26T13:36:56.433598+0000","last_active":"2024-07-26T13:37:17.566327+0000","last_peered":"2024-07-26T13:37:17.566327+0000","last_clean":"2024-07-26T13:37:17.566327+0000","last_became_active":"2024-07-26T13:36:56.433325+0000","last_became_peered":"2024-07-26T13:36:56.433325+0000","last_unstale":"2024-07-26T13:37:17.566327+0000","last_undegraded":"2024-07-26T13:37:17.566327+0000","last_fullsized":"2024-07-26T13:37:17.566327+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:17:33.651765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72939,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1089,"num_read_kb":622,"num_write":179,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":644,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":218817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247480,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678071,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":443451,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874300,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":1103707,"data_compressed":41637,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037825,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":897024,"data_stored":443665,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234053,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":443563,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430281,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":1036188,"data_compressed":41637,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593807,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":443054,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790036,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":1035340,"data_compressed":41637,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953561,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":888832,"data_stored":444023,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149788,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":443365,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378722,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":511840,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542246,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":506893,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705771,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":444325,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34195,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34195,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":69419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":68993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-26T13:40:14.929 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:40:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:14 smithi138 ceph-mon[36954]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-26T13:40:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:14 smithi138 ceph-mon[36954]: pgmap v368: 97 pgs: 97 active+clean; 648 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 7 op/s 2024-07-26T13:40:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:14 smithi167 ceph-mon[37140]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-26T13:40:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:14 smithi167 ceph-mon[37140]: pgmap v368: 97 pgs: 97 active+clean; 648 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 7 op/s 2024-07-26T13:40:15.218 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:40:15.602 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:15 smithi100 ceph-mon[32599]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-26T13:40:15.602 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:15 smithi100 ceph-mon[32599]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-26T13:40:15.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:15 smithi138 ceph-mon[36954]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-26T13:40:15.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:15 smithi138 ceph-mon[36954]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-26T13:40:16.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:15 smithi167 ceph-mon[37140]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-26T13:40:16.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:15 smithi167 ceph-mon[37140]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-26T13:40:16.723 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:16 smithi100 ceph-mon[32599]: pgmap v369: 97 pgs: 97 active+clean; 648 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-07-26T13:40:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:16 smithi138 ceph-mon[36954]: pgmap v369: 97 pgs: 97 active+clean; 648 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-07-26T13:40:16.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.967+0000 7ffae0287640 1 -- 172.21.15.100:0/688459590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad8108ed0 msgr2=0x7ffad81092e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:16.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.967+0000 7ffae0287640 1 --2- 172.21.15.100:0/688459590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad8108ed0 0x7ffad81092e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ffac8009900 tx=0x7ffac802a410 comp rx=0 tx=0).stop 2024-07-26T13:40:16.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.967+0000 7ffae0287640 1 -- 172.21.15.100:0/688459590 shutdown_connections 2024-07-26T13:40:16.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.967+0000 7ffae0287640 1 --2- 172.21.15.100:0/688459590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad8109820 0x7ffad8110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:16.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.967+0000 7ffae0287640 1 --2- 172.21.15.100:0/688459590 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad8108ed0 0x7ffad81092e0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:16.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.967+0000 7ffae0287640 1 --2- 172.21.15.100:0/688459590 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffad8103590 0x7ffad81039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:16.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.967+0000 7ffae0287640 1 -- 172.21.15.100:0/688459590 >> 172.21.15.100:0/688459590 conn(0x7ffad807b280 msgr2=0x7ffad807b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:16.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.967+0000 7ffae0287640 1 -- 172.21.15.100:0/688459590 shutdown_connections 2024-07-26T13:40:16.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.967+0000 7ffae0287640 1 -- 172.21.15.100:0/688459590 wait complete. 2024-07-26T13:40:16.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.968+0000 7ffae0287640 1 Processor -- start 2024-07-26T13:40:16.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.968+0000 7ffae0287640 1 -- start start 2024-07-26T13:40:16.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.968+0000 7ffae0287640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad8103590 0x7ffad8199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:16.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.968+0000 7ffae0287640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad8108ed0 0x7ffad819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:16.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.968+0000 7ffae0287640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffad8109820 0x7ffad819f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:16.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.968+0000 7ffae0287640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ffad8113490 con 0x7ffad8103590 2024-07-26T13:40:16.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.968+0000 7ffae0287640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffad8113310 con 0x7ffad8108ed0 2024-07-26T13:40:16.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.968+0000 7ffae0287640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ffad8113610 con 0x7ffad8109820 2024-07-26T13:40:16.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.968+0000 7ffade023640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad8103590 0x7ffad8199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:16.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.968+0000 7ffade023640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad8103590 0x7ffad8199f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57018/0 (socket says 172.21.15.100:57018) 2024-07-26T13:40:16.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.968+0000 7ffade023640 1 -- 172.21.15.100:0/1068032282 learned_addr learned my addr 172.21.15.100:0/1068032282 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:16.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.968+0000 7ffade824640 1 --2- 172.21.15.100:0/1068032282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffad8109820 0x7ffad819f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:16.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.968+0000 7ffadd822640 1 --2- 172.21.15.100:0/1068032282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad8108ed0 0x7ffad819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:16.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.969+0000 7ffade023640 1 -- 172.21.15.100:0/1068032282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffad8109820 msgr2=0x7ffad819f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:16.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.969+0000 7ffade023640 1 --2- 172.21.15.100:0/1068032282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffad8109820 0x7ffad819f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:16.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.969+0000 7ffade023640 1 -- 172.21.15.100:0/1068032282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad8108ed0 msgr2=0x7ffad819a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:16.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.969+0000 7ffade023640 1 --2- 172.21.15.100:0/1068032282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad8108ed0 0x7ffad819a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:16.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.969+0000 7ffade023640 1 -- 172.21.15.100:0/1068032282 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad819fe90 con 0x7ffad8103590 2024-07-26T13:40:16.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.969+0000 7ffadd822640 1 --2- 172.21.15.100:0/1068032282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad8108ed0 0x7ffad819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:40:16.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.969+0000 7ffade023640 1 --2- 172.21.15.100:0/1068032282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad8103590 0x7ffad8199f20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ffad400d950 tx=0x7ffad400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:16.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.969+0000 7ffacf7fe640 1 -- 172.21.15.100:0/1068032282 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad4017030 con 0x7ffad8103590 2024-07-26T13:40:16.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.969+0000 7ffae0287640 1 -- 172.21.15.100:0/1068032282 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffad81a0180 con 0x7ffad8103590 2024-07-26T13:40:16.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.969+0000 7ffacf7fe640 1 -- 172.21.15.100:0/1068032282 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffad4004020 con 0x7ffad8103590 2024-07-26T13:40:16.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.969+0000 7ffacf7fe640 1 -- 172.21.15.100:0/1068032282 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad40047a0 con 0x7ffad8103590 2024-07-26T13:40:16.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.969+0000 7ffae0287640 1 -- 172.21.15.100:0/1068032282 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffad81a68e0 con 0x7ffad8103590 2024-07-26T13:40:16.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.970+0000 7ffae0287640 1 -- 172.21.15.100:0/1068032282 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffad81039a0 con 0x7ffad8103590 2024-07-26T13:40:16.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.972+0000 7ffacf7fe640 1 -- 172.21.15.100:0/1068032282 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 151077+0+0 (secure 0 0 0) 0x7ffad4009a30 con 0x7ffad8103590 2024-07-26T13:40:16.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.973+0000 7ffacf7fe640 1 --2- 172.21.15.100:0/1068032282 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ffac40b3c20 0x7ffac40b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:16.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.973+0000 7ffadd822640 1 --2- 172.21.15.100:0/1068032282 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ffac40b3c20 0x7ffac40b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:16.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.973+0000 7ffacf7fe640 1 -- 172.21.15.100:0/1068032282 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7ffad40e3e20 con 0x7ffad8103590 2024-07-26T13:40:16.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.973+0000 7ffadd822640 1 --2- 172.21.15.100:0/1068032282 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ffac40b3c20 0x7ffac40b6130 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ffac8009230 tx=0x7ffac8000bc0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:16.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:16.976+0000 7ffacf7fe640 1 -- 172.21.15.100:0/1068032282 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7ffad40ab310 con 0x7ffad8103590 2024-07-26T13:40:17.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:16 smithi167 ceph-mon[37140]: pgmap v369: 97 pgs: 97 active+clean; 648 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-07-26T13:40:17.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:17.187+0000 7ffae0287640 1 -- 172.21.15.100:0/1068032282 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffad819b050 con 0x7ffad8103590 2024-07-26T13:40:17.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:17.188+0000 7ffacf7fe640 1 -- 172.21.15.100:0/1068032282 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11927 (secure 0 0 0) 0x7ffad40aac10 con 0x7ffad8103590 2024-07-26T13:40:17.189 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:40:17.189 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":26,"btime":"2024-07-26T13:40:09:530496+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14790,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/313501409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":313501409},{"type":"v1","addr":"172.21.15.100:6839","nonce":313501409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/1402975506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":1402975506},{"type":"v1","addr":"172.21.15.100:6841","nonce":1402975506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-26T13:37:01.281843+0000","modified":"2024-07-26T13:40:09.187125+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24607,"mds_1":24605,"mds_2":24595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24557":{"gid":24557,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/3290105300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3290105300},{"type":"v1","addr":"172.21.15.138:6835","nonce":3290105300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/3393225483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3393225483},{"type":"v1","addr":"172.21.15.138:6837","nonce":3393225483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24595":{"gid":24595,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6837/3828634468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3828634468},{"type":"v1","addr":"172.21.15.167:6837","nonce":3828634468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.138:6839/1580925471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1580925471},{"type":"v1","addr":"172.21.15.138:6839","nonce":1580925471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/398150514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":398150514},{"type":"v1","addr":"172.21.15.167:6839","nonce":398150514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24607,"qdb_cluster":[24595,24605,24607]},"id":1}]} 2024-07-26T13:40:17.190 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 26 2024-07-26T13:40:17.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:17.190+0000 7ffae0287640 1 -- 172.21.15.100:0/1068032282 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ffac40b3c20 msgr2=0x7ffac40b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:17.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:17.190+0000 7ffae0287640 1 --2- 172.21.15.100:0/1068032282 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ffac40b3c20 0x7ffac40b6130 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ffac8009230 tx=0x7ffac8000bc0 comp rx=0 tx=0).stop 2024-07-26T13:40:17.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:17.190+0000 7ffae0287640 1 -- 172.21.15.100:0/1068032282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad8103590 msgr2=0x7ffad8199f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:17.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:17.190+0000 7ffae0287640 1 --2- 172.21.15.100:0/1068032282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad8103590 0x7ffad8199f20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ffad400d950 tx=0x7ffad400de10 comp rx=0 tx=0).stop 2024-07-26T13:40:17.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:17.190+0000 7ffae0287640 1 -- 172.21.15.100:0/1068032282 shutdown_connections 2024-07-26T13:40:17.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:17.191+0000 7ffae0287640 1 --2- 172.21.15.100:0/1068032282 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ffac40b3c20 0x7ffac40b6130 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:17.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:17.191+0000 7ffae0287640 1 --2- 172.21.15.100:0/1068032282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffad8109820 0x7ffad819f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:17.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:17.191+0000 7ffae0287640 1 --2- 172.21.15.100:0/1068032282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad8108ed0 0x7ffad819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:17.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:17.191+0000 7ffae0287640 1 --2- 172.21.15.100:0/1068032282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad8103590 0x7ffad8199f20 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:17.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:17.191+0000 7ffae0287640 1 -- 172.21.15.100:0/1068032282 >> 172.21.15.100:0/1068032282 conn(0x7ffad807b280 msgr2=0x7ffad8107010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:17.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:17.191+0000 7ffae0287640 1 -- 172.21.15.100:0/1068032282 shutdown_connections 2024-07-26T13:40:17.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:17.191+0000 7ffae0287640 1 -- 172.21.15.100:0/1068032282 wait complete. 2024-07-26T13:40:17.924 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:17 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/1068032282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:40:17.925 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:17 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/1068032282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:40:17.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:17 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/1068032282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:40:17.956 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph osd dump --format=json 2024-07-26T13:40:18.214 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:40:18.587 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:18 smithi100 ceph-mon[32599]: pgmap v370: 97 pgs: 97 active+clean; 648 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 6 op/s 2024-07-26T13:40:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:18 smithi138 ceph-mon[36954]: pgmap v370: 97 pgs: 97 active+clean; 648 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 6 op/s 2024-07-26T13:40:19.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:18 smithi167 ceph-mon[37140]: pgmap v370: 97 pgs: 97 active+clean; 648 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 6 op/s 2024-07-26T13:40:19.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.919+0000 7fef93550640 1 -- 172.21.15.100:0/337448853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef8c102490 msgr2=0x7fef8c1028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:19.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.919+0000 7fef93550640 1 --2- 172.21.15.100:0/337448853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef8c102490 0x7fef8c1028a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fef7c009ca0 tx=0x7fef7c02a8c0 comp rx=0 tx=0).stop 2024-07-26T13:40:19.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.919+0000 7fef93550640 1 -- 172.21.15.100:0/337448853 shutdown_connections 2024-07-26T13:40:19.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.919+0000 7fef93550640 1 --2- 172.21.15.100:0/337448853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef8c108720 0x7fef8c10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:19.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.919+0000 7fef93550640 1 --2- 172.21.15.100:0/337448853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef8c107dd0 0x7fef8c1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:19.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.919+0000 7fef93550640 1 --2- 172.21.15.100:0/337448853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef8c102490 0x7fef8c1028a0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:19.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.919+0000 7fef93550640 1 -- 172.21.15.100:0/337448853 >> 172.21.15.100:0/337448853 conn(0x7fef8c079150 msgr2=0x7fef8c079560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:19.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.919+0000 7fef93550640 1 -- 172.21.15.100:0/337448853 shutdown_connections 2024-07-26T13:40:19.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.919+0000 7fef93550640 1 -- 172.21.15.100:0/337448853 wait complete. 2024-07-26T13:40:19.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.920+0000 7fef93550640 1 Processor -- start 2024-07-26T13:40:19.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.920+0000 7fef93550640 1 -- start start 2024-07-26T13:40:19.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.920+0000 7fef93550640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef8c102490 0x7fef8c199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:19.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.920+0000 7fef93550640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef8c107dd0 0x7fef8c19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:19.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.920+0000 7fef93550640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef8c108720 0x7fef8c19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:19.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.920+0000 7fef93550640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c112380 con 0x7fef8c108720 2024-07-26T13:40:19.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.920+0000 7fef93550640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c112200 con 0x7fef8c102490 2024-07-26T13:40:19.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.920+0000 7fef93550640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c112500 con 0x7fef8c107dd0 2024-07-26T13:40:19.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.920+0000 7fef91aed640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef8c108720 0x7fef8c19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:19.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.920+0000 7fef91aed640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef8c108720 0x7fef8c19f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57036/0 (socket says 172.21.15.100:57036) 2024-07-26T13:40:19.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.920+0000 7fef91aed640 1 -- 172.21.15.100:0/1109989225 learned_addr learned my addr 172.21.15.100:0/1109989225 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:19.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.920+0000 7fef90aeb640 1 --2- 172.21.15.100:0/1109989225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef8c107dd0 0x7fef8c19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:19.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.921+0000 7fef912ec640 1 --2- 172.21.15.100:0/1109989225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef8c102490 0x7fef8c199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:19.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.921+0000 7fef91aed640 1 -- 172.21.15.100:0/1109989225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef8c107dd0 msgr2=0x7fef8c19a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:19.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.921+0000 7fef91aed640 1 --2- 172.21.15.100:0/1109989225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef8c107dd0 0x7fef8c19a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:19.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.921+0000 7fef91aed640 1 -- 172.21.15.100:0/1109989225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef8c102490 msgr2=0x7fef8c199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:19.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.921+0000 7fef91aed640 1 --2- 172.21.15.100:0/1109989225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef8c102490 0x7fef8c199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:19.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.921+0000 7fef91aed640 1 -- 172.21.15.100:0/1109989225 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef8c19fd30 con 0x7fef8c108720 2024-07-26T13:40:19.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.921+0000 7fef912ec640 1 --2- 172.21.15.100:0/1109989225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef8c102490 0x7fef8c199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:40:19.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.921+0000 7fef91aed640 1 --2- 172.21.15.100:0/1109989225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef8c108720 0x7fef8c19f6c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fef8400d750 tx=0x7fef8400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:19.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.921+0000 7fef8a7fc640 1 -- 172.21.15.100:0/1109989225 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef84017070 con 0x7fef8c108720 2024-07-26T13:40:19.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.921+0000 7fef93550640 1 -- 172.21.15.100:0/1109989225 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef8c1a0020 con 0x7fef8c108720 2024-07-26T13:40:19.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.921+0000 7fef93550640 1 -- 172.21.15.100:0/1109989225 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef8c1a6840 con 0x7fef8c108720 2024-07-26T13:40:19.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.921+0000 7fef8a7fc640 1 -- 172.21.15.100:0/1109989225 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef840054d0 con 0x7fef8c108720 2024-07-26T13:40:19.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.921+0000 7fef8a7fc640 1 -- 172.21.15.100:0/1109989225 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef84004c60 con 0x7fef8c108720 2024-07-26T13:40:19.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.922+0000 7fef93550640 1 -- 172.21.15.100:0/1109989225 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef8c103710 con 0x7fef8c108720 2024-07-26T13:40:19.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.924+0000 7fef8a7fc640 1 -- 172.21.15.100:0/1109989225 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 151077+0+0 (secure 0 0 0) 0x7fef84009bd0 con 0x7fef8c108720 2024-07-26T13:40:19.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.924+0000 7fef8a7fc640 1 --2- 172.21.15.100:0/1109989225 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fef740b3c50 0x7fef740b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:19.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.924+0000 7fef912ec640 1 --2- 172.21.15.100:0/1109989225 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fef740b3c50 0x7fef740b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:19.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.924+0000 7fef8a7fc640 1 -- 172.21.15.100:0/1109989225 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fef840e37f0 con 0x7fef8c108720 2024-07-26T13:40:19.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.925+0000 7fef912ec640 1 --2- 172.21.15.100:0/1109989225 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fef740b3c50 0x7fef740b6160 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fef7c006010 tx=0x7fef7c000f40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:19.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:19.928+0000 7fef8a7fc640 1 -- 172.21.15.100:0/1109989225 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fef840aad40 con 0x7fef8c108720 2024-07-26T13:40:20.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.106+0000 7fef93550640 1 -- 172.21.15.100:0/1109989225 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef8c1a6a40 con 0x7fef8c108720 2024-07-26T13:40:20.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.107+0000 7fef8a7fc640 1 -- 172.21.15.100:0/1109989225 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+23121 (secure 0 0 0) 0x7fef840aa640 con 0x7fef8c108720 2024-07-26T13:40:20.108 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:40:20.108 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":81,"fsid":"bf83e2e6-4b52-11ef-bcb7-c7b262605968","created":"2024-07-26T13:28:23.639785+0000","modified":"2024-07-26T13:37:17.539521+0000","last_up_change":"2024-07-26T13:35:56.475554+0000","last_in_change":"2024-07-26T13:35:37.472564+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-26T13:32:22.329969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-26T13:36:54.590525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-26T13:36:58.149545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d714036d-9dda-4cb5-af16-29f83a09dd70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6803","nonce":1480401850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6805","nonce":1480401850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6809","nonce":1480401850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6807","nonce":1480401850}]},"public_addr":"172.21.15.100:6803/1480401850","cluster_addr":"172.21.15.100:6805/1480401850","heartbeat_back_addr":"172.21.15.100:6809/1480401850","heartbeat_front_addr":"172.21.15.100:6807/1480401850","state":["exists","up"]},{"osd":1,"uuid":"3dd0eaa1-618a-495a-9819-e01c0f869ae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6801","nonce":3090877671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6803","nonce":3090877671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6807","nonce":3090877671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6805","nonce":3090877671}]},"public_addr":"172.21.15.138:6801/3090877671","cluster_addr":"172.21.15.138:6803/3090877671","heartbeat_back_addr":"172.21.15.138:6807/3090877671","heartbeat_front_addr":"172.21.15.138:6805/3090877671","state":["exists","up"]},{"osd":2,"uuid":"5c710a78-17ae-41a8-ac16-a678f5d35672","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6801","nonce":3583424790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6803","nonce":3583424790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6807","nonce":3583424790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6805","nonce":3583424790}]},"public_addr":"172.21.15.167:6801/3583424790","cluster_addr":"172.21.15.167:6803/3583424790","heartbeat_back_addr":"172.21.15.167:6807/3583424790","heartbeat_front_addr":"172.21.15.167:6805/3583424790","state":["exists","up"]},{"osd":3,"uuid":"be845495-8c31-4834-a45c-bdc576711962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6811","nonce":3626762419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6813","nonce":3626762419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6817","nonce":3626762419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6815","nonce":3626762419}]},"public_addr":"172.21.15.100:6811/3626762419","cluster_addr":"172.21.15.100:6813/3626762419","heartbeat_back_addr":"172.21.15.100:6817/3626762419","heartbeat_front_addr":"172.21.15.100:6815/3626762419","state":["exists","up"]},{"osd":4,"uuid":"520554cc-e846-46f9-af77-05a1de65a452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6809","nonce":3757334356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6811","nonce":3757334356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6815","nonce":3757334356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6813","nonce":3757334356}]},"public_addr":"172.21.15.138:6809/3757334356","cluster_addr":"172.21.15.138:6811/3757334356","heartbeat_back_addr":"172.21.15.138:6815/3757334356","heartbeat_front_addr":"172.21.15.138:6813/3757334356","state":["exists","up"]},{"osd":5,"uuid":"68dc8ad6-31d2-45b5-9aa7-62c4218d46e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6809","nonce":3239482836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6811","nonce":3239482836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6815","nonce":3239482836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6813","nonce":3239482836}]},"public_addr":"172.21.15.167:6809/3239482836","cluster_addr":"172.21.15.167:6811/3239482836","heartbeat_back_addr":"172.21.15.167:6815/3239482836","heartbeat_front_addr":"172.21.15.167:6813/3239482836","state":["exists","up"]},{"osd":6,"uuid":"420e4471-5626-41cc-a8e1-69c801196b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6819","nonce":3218415484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6821","nonce":3218415484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6825","nonce":3218415484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6823","nonce":3218415484}]},"public_addr":"172.21.15.100:6819/3218415484","cluster_addr":"172.21.15.100:6821/3218415484","heartbeat_back_addr":"172.21.15.100:6825/3218415484","heartbeat_front_addr":"172.21.15.100:6823/3218415484","state":["exists","up"]},{"osd":7,"uuid":"a27b097e-207d-44c3-af63-544f228b83ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6817","nonce":45541749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6819","nonce":45541749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6823","nonce":45541749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6821","nonce":45541749}]},"public_addr":"172.21.15.138:6817/45541749","cluster_addr":"172.21.15.138:6819/45541749","heartbeat_back_addr":"172.21.15.138:6823/45541749","heartbeat_front_addr":"172.21.15.138:6821/45541749","state":["exists","up"]},{"osd":8,"uuid":"3e865858-92cc-48a0-a464-5767d19e8ac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6817","nonce":4031159803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6819","nonce":4031159803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6823","nonce":4031159803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6821","nonce":4031159803}]},"public_addr":"172.21.15.167:6817/4031159803","cluster_addr":"172.21.15.167:6819/4031159803","heartbeat_back_addr":"172.21.15.167:6823/4031159803","heartbeat_front_addr":"172.21.15.167:6821/4031159803","state":["exists","up"]},{"osd":9,"uuid":"8b05b14a-1195-42f5-ac01-d5634cb9c888","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6827","nonce":1840140273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6829","nonce":1840140273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6833","nonce":1840140273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6831","nonce":1840140273}]},"public_addr":"172.21.15.100:6827/1840140273","cluster_addr":"172.21.15.100:6829/1840140273","heartbeat_back_addr":"172.21.15.100:6833/1840140273","heartbeat_front_addr":"172.21.15.100:6831/1840140273","state":["exists","up"]},{"osd":10,"uuid":"ab77ebe7-5254-4dc3-bc54-625698e84afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6825","nonce":703382899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6827","nonce":703382899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6831","nonce":703382899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6829","nonce":703382899}]},"public_addr":"172.21.15.138:6825/703382899","cluster_addr":"172.21.15.138:6827/703382899","heartbeat_back_addr":"172.21.15.138:6831/703382899","heartbeat_front_addr":"172.21.15.138:6829/703382899","state":["exists","up"]},{"osd":11,"uuid":"229cf91e-e9ef-48f6-a677-c5c5705b315a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6825","nonce":1032343068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6827","nonce":1032343068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6831","nonce":1032343068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6829","nonce":1032343068}]},"public_addr":"172.21.15.167:6825/1032343068","cluster_addr":"172.21.15.167:6827/1032343068","heartbeat_back_addr":"172.21.15.167:6831/1032343068","heartbeat_front_addr":"172.21.15.167:6829/1032343068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:31:35.800702+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:31:58.087922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:32:20.463183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:32:42.920598+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:33:06.332655+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:33:29.381407+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:33:53.975963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:34:18.226813+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:34:41.845770+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:35:05.722275+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:35:30.862704+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:35:54.738697+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:0/1874993112":"2024-07-27T13:30:16.253096+0000","172.21.15.100:0/2364620305":"2024-07-27T13:28:45.340334+0000","172.21.15.100:6800/3096076075":"2024-07-27T13:29:07.979178+0000","172.21.15.100:0/1894966501":"2024-07-27T13:30:16.253096+0000","172.21.15.100:0/3411776672":"2024-07-27T13:28:45.340334+0000","172.21.15.100:0/2853423120":"2024-07-27T13:29:07.979178+0000","172.21.15.100:6800/3025873570":"2024-07-27T13:28:45.340334+0000","172.21.15.100:0/4887181":"2024-07-27T13:29:07.979178+0000","172.21.15.100:6801/908675140":"2024-07-27T13:30:16.253096+0000","172.21.15.100:6801/3025873570":"2024-07-27T13:28:45.340334+0000","172.21.15.100:6800/908675140":"2024-07-27T13:30:16.253096+0000","172.21.15.100:6801/3096076075":"2024-07-27T13:29:07.979178+0000","172.21.15.100:0/2302632052":"2024-07-27T13:29:07.979178+0000","172.21.15.100:0/3736098915":"2024-07-27T13:28:45.340334+0000","172.21.15.100:0/3243030560":"2024-07-27T13:30:16.253096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-26T13:40:20.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.109+0000 7fef93550640 1 -- 172.21.15.100:0/1109989225 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fef740b3c50 msgr2=0x7fef740b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:20.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.109+0000 7fef93550640 1 --2- 172.21.15.100:0/1109989225 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fef740b3c50 0x7fef740b6160 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fef7c006010 tx=0x7fef7c000f40 comp rx=0 tx=0).stop 2024-07-26T13:40:20.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.109+0000 7fef93550640 1 -- 172.21.15.100:0/1109989225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef8c108720 msgr2=0x7fef8c19f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:20.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.109+0000 7fef93550640 1 --2- 172.21.15.100:0/1109989225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef8c108720 0x7fef8c19f6c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fef8400d750 tx=0x7fef8400dc10 comp rx=0 tx=0).stop 2024-07-26T13:40:20.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.109+0000 7fef93550640 1 -- 172.21.15.100:0/1109989225 shutdown_connections 2024-07-26T13:40:20.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.109+0000 7fef93550640 1 --2- 172.21.15.100:0/1109989225 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fef740b3c50 0x7fef740b6160 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:20.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.109+0000 7fef93550640 1 --2- 172.21.15.100:0/1109989225 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef8c108720 0x7fef8c19f6c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:20.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.109+0000 7fef93550640 1 --2- 172.21.15.100:0/1109989225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef8c107dd0 0x7fef8c19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:20.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.109+0000 7fef93550640 1 --2- 172.21.15.100:0/1109989225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef8c102490 0x7fef8c199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:20.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.109+0000 7fef93550640 1 -- 172.21.15.100:0/1109989225 >> 172.21.15.100:0/1109989225 conn(0x7fef8c079150 msgr2=0x7fef8c1014f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:20.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.109+0000 7fef93550640 1 -- 172.21.15.100:0/1109989225 shutdown_connections 2024-07-26T13:40:20.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.109+0000 7fef93550640 1 -- 172.21.15.100:0/1109989225 wait complete. 2024-07-26T13:40:20.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:20 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/1109989225' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-26T13:40:20.445 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:20 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/1109989225' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-26T13:40:20.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:20 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/1109989225' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-26T13:40:20.755 DEBUG:tasks.fwd_scrub:Yielding 2024-07-26T13:40:20.755 INFO:teuthology.run_tasks:Running task exec... 2024-07-26T13:40:20.766 INFO:teuthology.task.exec:Executing custom commands... 2024-07-26T13:40:20.766 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi100.front.sepia.ceph.com 2024-07-26T13:40:20.766 DEBUG:teuthology.orchestra.run.smithi100:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-07-26T13:40:20.767 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-07-26T13:40:20.767 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-26T13:40:20.767 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-26T13:40:20.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.883+0000 7f08ce0c8640 1 -- 172.21.15.100:0/2288849257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08c81086b0 msgr2=0x7f08c810eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:20.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.883+0000 7f08ce0c8640 1 --2- 172.21.15.100:0/2288849257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08c81086b0 0x7f08c810eee0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f08c0009ca0 tx=0x7f08c002a8c0 comp rx=0 tx=0).stop 2024-07-26T13:40:20.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.883+0000 7f08ce0c8640 1 -- 172.21.15.100:0/2288849257 shutdown_connections 2024-07-26T13:40:20.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.883+0000 7f08ce0c8640 1 --2- 172.21.15.100:0/2288849257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08c81086b0 0x7f08c810eee0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:20.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.883+0000 7f08ce0c8640 1 --2- 172.21.15.100:0/2288849257 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08c8107d60 0x7f08c8108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:20.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.883+0000 7f08ce0c8640 1 --2- 172.21.15.100:0/2288849257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08c8102ba0 0x7f08c8102fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:20.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.883+0000 7f08ce0c8640 1 -- 172.21.15.100:0/2288849257 >> 172.21.15.100:0/2288849257 conn(0x7f08c80798c0 msgr2=0x7f08c8079cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:20.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.883+0000 7f08ce0c8640 1 -- 172.21.15.100:0/2288849257 shutdown_connections 2024-07-26T13:40:20.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.883+0000 7f08ce0c8640 1 -- 172.21.15.100:0/2288849257 wait complete. 2024-07-26T13:40:20.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.884+0000 7f08ce0c8640 1 Processor -- start 2024-07-26T13:40:20.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.884+0000 7f08ce0c8640 1 -- start start 2024-07-26T13:40:20.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.885+0000 7f08ce0c8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08c8102ba0 0x7f08c8198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:20.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.885+0000 7f08ce0c8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08c8107d60 0x7f08c8199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:20.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.885+0000 7f08ce0c8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08c81086b0 0x7f08c819e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:20.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.885+0000 7f08ce0c8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f08c8112380 con 0x7f08c8107d60 2024-07-26T13:40:20.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.885+0000 7f08ce0c8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f08c8112200 con 0x7f08c8102ba0 2024-07-26T13:40:20.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.885+0000 7f08ce0c8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f08c8112500 con 0x7f08c81086b0 2024-07-26T13:40:20.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.885+0000 7f08c6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08c8107d60 0x7f08c8199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:20.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.885+0000 7f08c6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08c8107d60 0x7f08c8199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57042/0 (socket says 172.21.15.100:57042) 2024-07-26T13:40:20.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.885+0000 7f08c6ffd640 1 -- 172.21.15.100:0/2103865828 learned_addr learned my addr 172.21.15.100:0/2103865828 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:20.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.885+0000 7f08c7fff640 1 --2- 172.21.15.100:0/2103865828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08c81086b0 0x7f08c819e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:20.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.885+0000 7f08c77fe640 1 --2- 172.21.15.100:0/2103865828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08c8102ba0 0x7f08c8198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:20.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.885+0000 7f08c6ffd640 1 -- 172.21.15.100:0/2103865828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08c81086b0 msgr2=0x7f08c819e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:20.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.885+0000 7f08c6ffd640 1 --2- 172.21.15.100:0/2103865828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08c81086b0 0x7f08c819e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:20.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.885+0000 7f08c6ffd640 1 -- 172.21.15.100:0/2103865828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08c8102ba0 msgr2=0x7f08c8198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:20.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.885+0000 7f08c6ffd640 1 --2- 172.21.15.100:0/2103865828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08c8102ba0 0x7f08c8198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:20.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.885+0000 7f08c6ffd640 1 -- 172.21.15.100:0/2103865828 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c819ec90 con 0x7f08c8107d60 2024-07-26T13:40:20.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.885+0000 7f08c77fe640 1 --2- 172.21.15.100:0/2103865828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08c8102ba0 0x7f08c8198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:40:20.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.885+0000 7f08c6ffd640 1 --2- 172.21.15.100:0/2103865828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08c8107d60 0x7f08c8199300 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f08b000e9c0 tx=0x7f08b000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:20.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.885+0000 7f08c4ff9640 1 -- 172.21.15.100:0/2103865828 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b000ccc0 con 0x7f08c8107d60 2024-07-26T13:40:20.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.885+0000 7f08ce0c8640 1 -- 172.21.15.100:0/2103865828 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08c819ef80 con 0x7f08c8107d60 2024-07-26T13:40:20.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.885+0000 7f08c4ff9640 1 -- 172.21.15.100:0/2103865828 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08b00054d0 con 0x7f08c8107d60 2024-07-26T13:40:20.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.885+0000 7f08c4ff9640 1 -- 172.21.15.100:0/2103865828 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b0010430 con 0x7f08c8107d60 2024-07-26T13:40:20.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.886+0000 7f08ce0c8640 1 -- 172.21.15.100:0/2103865828 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08c81a5710 con 0x7f08c8107d60 2024-07-26T13:40:20.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.886+0000 7f08ce0c8640 1 -- 172.21.15.100:0/2103865828 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08c8102fb0 con 0x7f08c8107d60 2024-07-26T13:40:20.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.889+0000 7f08c4ff9640 1 -- 172.21.15.100:0/2103865828 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 151077+0+0 (secure 0 0 0) 0x7f08b0005000 con 0x7f08c8107d60 2024-07-26T13:40:20.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.890+0000 7f08c4ff9640 1 --2- 172.21.15.100:0/2103865828 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f08b40b3c20 0x7f08b40b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:20.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.890+0000 7f08c4ff9640 1 -- 172.21.15.100:0/2103865828 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f08b00e4320 con 0x7f08c8107d60 2024-07-26T13:40:20.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.890+0000 7f08c77fe640 1 --2- 172.21.15.100:0/2103865828 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f08b40b3c20 0x7f08b40b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:20.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.891+0000 7f08c77fe640 1 --2- 172.21.15.100:0/2103865828 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f08b40b3c20 0x7f08b40b6130 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009b40 tx=0x7f08b8005e50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:20.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:20.892+0000 7f08c4ff9640 1 -- 172.21.15.100:0/2103865828 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f08b0055600 con 0x7f08c8107d60 2024-07-26T13:40:21.031 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:40:21.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.108+0000 7f08ce0c8640 1 -- 172.21.15.100:0/2103865828 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f08c810a500 con 0x7f08b40b3c20 2024-07-26T13:40:21.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.271+0000 7f08c4ff9640 1 -- 172.21.15.100:0/2103865828 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f08c810a500 con 0x7f08b40b3c20 2024-07-26T13:40:21.272 INFO:teuthology.orchestra.run.smithi100.stdout:Schedule set for path / 2024-07-26T13:40:21.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.274+0000 7f08ce0c8640 1 -- 172.21.15.100:0/2103865828 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f08b40b3c20 msgr2=0x7f08b40b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:21.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.274+0000 7f08ce0c8640 1 --2- 172.21.15.100:0/2103865828 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f08b40b3c20 0x7f08b40b6130 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009b40 tx=0x7f08b8005e50 comp rx=0 tx=0).stop 2024-07-26T13:40:21.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.274+0000 7f08ce0c8640 1 -- 172.21.15.100:0/2103865828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08c8107d60 msgr2=0x7f08c8199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:21.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.274+0000 7f08ce0c8640 1 --2- 172.21.15.100:0/2103865828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08c8107d60 0x7f08c8199300 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f08b000e9c0 tx=0x7f08b000ee80 comp rx=0 tx=0).stop 2024-07-26T13:40:21.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.274+0000 7f08ce0c8640 1 -- 172.21.15.100:0/2103865828 shutdown_connections 2024-07-26T13:40:21.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.274+0000 7f08ce0c8640 1 --2- 172.21.15.100:0/2103865828 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f08b40b3c20 0x7f08b40b6130 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:21.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.274+0000 7f08ce0c8640 1 --2- 172.21.15.100:0/2103865828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08c81086b0 0x7f08c819e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:21.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.274+0000 7f08ce0c8640 1 --2- 172.21.15.100:0/2103865828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f08c8107d60 0x7f08c8199300 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:21.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.274+0000 7f08ce0c8640 1 --2- 172.21.15.100:0/2103865828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08c8102ba0 0x7f08c8198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:21.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.274+0000 7f08ce0c8640 1 -- 172.21.15.100:0/2103865828 >> 172.21.15.100:0/2103865828 conn(0x7f08c80798c0 msgr2=0x7f08c8105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:21.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.274+0000 7f08ce0c8640 1 -- 172.21.15.100:0/2103865828 shutdown_connections 2024-07-26T13:40:21.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.274+0000 7f08ce0c8640 1 -- 172.21.15.100:0/2103865828 wait complete. 2024-07-26T13:40:21.287 DEBUG:teuthology.orchestra.run.smithi100:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-07-26T13:40:21.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.400+0000 7fc8fad02640 1 -- 172.21.15.100:0/1575410532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8f41086b0 msgr2=0x7fc8f410eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:21.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.400+0000 7fc8fad02640 1 --2- 172.21.15.100:0/1575410532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8f41086b0 0x7fc8f410eee0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec009ca0 tx=0x7fc8ec02a8c0 comp rx=0 tx=0).stop 2024-07-26T13:40:21.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.400+0000 7fc8fad02640 1 -- 172.21.15.100:0/1575410532 shutdown_connections 2024-07-26T13:40:21.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.400+0000 7fc8fad02640 1 --2- 172.21.15.100:0/1575410532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8f41086b0 0x7fc8f410eee0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:21.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.400+0000 7fc8fad02640 1 --2- 172.21.15.100:0/1575410532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8f4107d60 0x7fc8f4108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:21.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.400+0000 7fc8fad02640 1 --2- 172.21.15.100:0/1575410532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8f4102430 0x7fc8f4102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:21.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.400+0000 7fc8fad02640 1 -- 172.21.15.100:0/1575410532 >> 172.21.15.100:0/1575410532 conn(0x7fc8f40fde10 msgr2=0x7fc8f4100260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:21.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.400+0000 7fc8fad02640 1 -- 172.21.15.100:0/1575410532 shutdown_connections 2024-07-26T13:40:21.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.401+0000 7fc8fad02640 1 -- 172.21.15.100:0/1575410532 wait complete. 2024-07-26T13:40:21.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.401+0000 7fc8fad02640 1 Processor -- start 2024-07-26T13:40:21.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.401+0000 7fc8fad02640 1 -- start start 2024-07-26T13:40:21.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.401+0000 7fc8fad02640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8f4102430 0x7fc8f419acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:21.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.401+0000 7fc8fad02640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8f4107d60 0x7fc8f419b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:21.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.401+0000 7fc8fad02640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8f41086b0 0x7fc8f419b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:21.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.401+0000 7fc8fad02640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f41123f0 con 0x7fc8f4102430 2024-07-26T13:40:21.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.401+0000 7fc8fad02640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f4112270 con 0x7fc8f4107d60 2024-07-26T13:40:21.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.401+0000 7fc8fad02640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f4112570 con 0x7fc8f41086b0 2024-07-26T13:40:21.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.402+0000 7fc8f8a9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8f4102430 0x7fc8f419acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:21.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.402+0000 7fc8f8a9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8f4102430 0x7fc8f419acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57070/0 (socket says 172.21.15.100:57070) 2024-07-26T13:40:21.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.402+0000 7fc8f8a9e640 1 -- 172.21.15.100:0/2980177429 learned_addr learned my addr 172.21.15.100:0/2980177429 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:21.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.402+0000 7fc8f929f640 1 --2- 172.21.15.100:0/2980177429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8f41086b0 0x7fc8f419b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:21.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.402+0000 7fc8f8a9e640 1 -- 172.21.15.100:0/2980177429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8f41086b0 msgr2=0x7fc8f419b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:21.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.402+0000 7fc8f8a9e640 1 --2- 172.21.15.100:0/2980177429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8f41086b0 0x7fc8f419b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:21.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.402+0000 7fc8f3fff640 1 --2- 172.21.15.100:0/2980177429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8f4107d60 0x7fc8f419b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:21.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.402+0000 7fc8f8a9e640 1 -- 172.21.15.100:0/2980177429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8f4107d60 msgr2=0x7fc8f419b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:21.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.402+0000 7fc8f8a9e640 1 --2- 172.21.15.100:0/2980177429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8f4107d60 0x7fc8f419b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:21.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.402+0000 7fc8f8a9e640 1 -- 172.21.15.100:0/2980177429 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f41a5f80 con 0x7fc8f4102430 2024-07-26T13:40:21.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.402+0000 7fc8f8a9e640 1 --2- 172.21.15.100:0/2980177429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8f4102430 0x7fc8f419acc0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc8e800b4a0 tx=0x7fc8e800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:21.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.402+0000 7fc8f1ffb640 1 -- 172.21.15.100:0/2980177429 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8014070 con 0x7fc8f4102430 2024-07-26T13:40:21.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.402+0000 7fc8f1ffb640 1 -- 172.21.15.100:0/2980177429 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8e8010ac0 con 0x7fc8f4102430 2024-07-26T13:40:21.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.402+0000 7fc8fad02640 1 -- 172.21.15.100:0/2980177429 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8f41a6270 con 0x7fc8f4102430 2024-07-26T13:40:21.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.402+0000 7fc8f1ffb640 1 -- 172.21.15.100:0/2980177429 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8017c30 con 0x7fc8f4102430 2024-07-26T13:40:21.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.402+0000 7fc8fad02640 1 -- 172.21.15.100:0/2980177429 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8f41a6890 con 0x7fc8f4102430 2024-07-26T13:40:21.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.403+0000 7fc8fad02640 1 -- 172.21.15.100:0/2980177429 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8f4102840 con 0x7fc8f4102430 2024-07-26T13:40:21.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.405+0000 7fc8f1ffb640 1 -- 172.21.15.100:0/2980177429 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 151077+0+0 (secure 0 0 0) 0x7fc8e8017070 con 0x7fc8f4102430 2024-07-26T13:40:21.407 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:21 smithi100 ceph-mon[32599]: pgmap v371: 97 pgs: 97 active+clean; 648 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 7 op/s 2024-07-26T13:40:21.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.406+0000 7fc8f1ffb640 1 --2- 172.21.15.100:0/2980177429 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc8e40b3ca0 0x7fc8e40b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:21.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.406+0000 7fc8f3fff640 1 --2- 172.21.15.100:0/2980177429 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc8e40b3ca0 0x7fc8e40b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:21.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.406+0000 7fc8f1ffb640 1 -- 172.21.15.100:0/2980177429 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fc8e80e3410 con 0x7fc8f4102430 2024-07-26T13:40:21.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.407+0000 7fc8f3fff640 1 --2- 172.21.15.100:0/2980177429 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc8e40b3ca0 0x7fc8e40b61b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4100650 tx=0x7fc8e0005c60 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:21.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.411+0000 7fc8f1ffb640 1 -- 172.21.15.100:0/2980177429 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fc8e80aa960 con 0x7fc8f4102430 2024-07-26T13:40:21.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:21 smithi138 ceph-mon[36954]: pgmap v371: 97 pgs: 97 active+clean; 648 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 7 op/s 2024-07-26T13:40:21.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:21 smithi167 ceph-mon[37140]: pgmap v371: 97 pgs: 97 active+clean; 648 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 7 op/s 2024-07-26T13:40:21.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.625+0000 7fc8fad02640 1 -- 172.21.15.100:0/2980177429 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fc8f4063230 con 0x7fc8e40b3ca0 2024-07-26T13:40:21.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.694+0000 7fc8f1ffb640 1 -- 172.21.15.100:0/2980177429 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fc8f4063230 con 0x7fc8e40b3ca0 2024-07-26T13:40:21.695 INFO:teuthology.orchestra.run.smithi100.stdout:Retention added to path / 2024-07-26T13:40:21.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.696+0000 7fc8fad02640 1 -- 172.21.15.100:0/2980177429 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc8e40b3ca0 msgr2=0x7fc8e40b61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:21.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.696+0000 7fc8fad02640 1 --2- 172.21.15.100:0/2980177429 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc8e40b3ca0 0x7fc8e40b61b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4100650 tx=0x7fc8e0005c60 comp rx=0 tx=0).stop 2024-07-26T13:40:21.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.696+0000 7fc8fad02640 1 -- 172.21.15.100:0/2980177429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8f4102430 msgr2=0x7fc8f419acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:21.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.696+0000 7fc8fad02640 1 --2- 172.21.15.100:0/2980177429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8f4102430 0x7fc8f419acc0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc8e800b4a0 tx=0x7fc8e800b960 comp rx=0 tx=0).stop 2024-07-26T13:40:21.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.696+0000 7fc8fad02640 1 -- 172.21.15.100:0/2980177429 shutdown_connections 2024-07-26T13:40:21.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.696+0000 7fc8fad02640 1 --2- 172.21.15.100:0/2980177429 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc8e40b3ca0 0x7fc8e40b61b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:21.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.696+0000 7fc8fad02640 1 --2- 172.21.15.100:0/2980177429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8f41086b0 0x7fc8f419b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:21.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.696+0000 7fc8fad02640 1 --2- 172.21.15.100:0/2980177429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8f4107d60 0x7fc8f419b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:21.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.696+0000 7fc8fad02640 1 --2- 172.21.15.100:0/2980177429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8f4102430 0x7fc8f419acc0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:21.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.696+0000 7fc8fad02640 1 -- 172.21.15.100:0/2980177429 >> 172.21.15.100:0/2980177429 conn(0x7fc8f40fde10 msgr2=0x7fc8f4101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:21.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.697+0000 7fc8fad02640 1 -- 172.21.15.100:0/2980177429 shutdown_connections 2024-07-26T13:40:21.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.697+0000 7fc8fad02640 1 -- 172.21.15.100:0/2980177429 wait complete. 2024-07-26T13:40:21.709 DEBUG:teuthology.orchestra.run.smithi100:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-07-26T13:40:21.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.821+0000 7f47733cf640 1 -- 172.21.15.100:0/1814534783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f476c10d7b0 msgr2=0x7f476c10fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:21.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.821+0000 7f47733cf640 1 --2- 172.21.15.100:0/1814534783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f476c10d7b0 0x7f476c10fcc0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4764009ca0 tx=0x7f476402a8c0 comp rx=0 tx=0).stop 2024-07-26T13:40:21.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.822+0000 7f47733cf640 1 -- 172.21.15.100:0/1814534783 shutdown_connections 2024-07-26T13:40:21.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.822+0000 7f47733cf640 1 --2- 172.21.15.100:0/1814534783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f476c10d7b0 0x7f476c10fcc0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:21.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.822+0000 7f47733cf640 1 --2- 172.21.15.100:0/1814534783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f476c104c30 0x7f476c10d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:21.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.822+0000 7f47733cf640 1 --2- 172.21.15.100:0/1814534783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f476c104200 0x7f476c104610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:21.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.822+0000 7f47733cf640 1 -- 172.21.15.100:0/1814534783 >> 172.21.15.100:0/1814534783 conn(0x7f476c07a0a0 msgr2=0x7f476c07a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:21.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.822+0000 7f47733cf640 1 -- 172.21.15.100:0/1814534783 shutdown_connections 2024-07-26T13:40:21.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.822+0000 7f47733cf640 1 -- 172.21.15.100:0/1814534783 wait complete. 2024-07-26T13:40:21.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.822+0000 7f47733cf640 1 Processor -- start 2024-07-26T13:40:21.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.822+0000 7f47733cf640 1 -- start start 2024-07-26T13:40:21.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.823+0000 7f47733cf640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f476c104200 0x7f476c199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:21.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.823+0000 7f47733cf640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f476c104c30 0x7f476c19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:21.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.823+0000 7f47733cf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f476c10d7b0 0x7f476c19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:21.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.823+0000 7f47733cf640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f476c113460 con 0x7f476c10d7b0 2024-07-26T13:40:21.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.823+0000 7f47733cf640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f476c1132e0 con 0x7f476c104200 2024-07-26T13:40:21.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.823+0000 7f47733cf640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f476c1135e0 con 0x7f476c104c30 2024-07-26T13:40:21.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.823+0000 7f477196c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f476c10d7b0 0x7f476c19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:21.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.823+0000 7f477196c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f476c10d7b0 0x7f476c19f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57080/0 (socket says 172.21.15.100:57080) 2024-07-26T13:40:21.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.823+0000 7f477196c640 1 -- 172.21.15.100:0/2214380261 learned_addr learned my addr 172.21.15.100:0/2214380261 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:21.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.823+0000 7f477096a640 1 --2- 172.21.15.100:0/2214380261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f476c104c30 0x7f476c19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:21.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.823+0000 7f477196c640 1 -- 172.21.15.100:0/2214380261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f476c104c30 msgr2=0x7f476c19a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:21.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.823+0000 7f477196c640 1 --2- 172.21.15.100:0/2214380261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f476c104c30 0x7f476c19a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:21.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.823+0000 7f477196c640 1 -- 172.21.15.100:0/2214380261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f476c104200 msgr2=0x7f476c199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:21.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.823+0000 7f477196c640 1 --2- 172.21.15.100:0/2214380261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f476c104200 0x7f476c199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:21.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.823+0000 7f477196c640 1 -- 172.21.15.100:0/2214380261 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f476c19fd70 con 0x7f476c10d7b0 2024-07-26T13:40:21.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.823+0000 7f477196c640 1 --2- 172.21.15.100:0/2214380261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f476c10d7b0 0x7f476c19f670 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4764005cc0 tx=0x7f476402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:21.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.823+0000 7f47627fc640 1 -- 172.21.15.100:0/2214380261 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4764003bf0 con 0x7f476c10d7b0 2024-07-26T13:40:21.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.823+0000 7f47627fc640 1 -- 172.21.15.100:0/2214380261 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4764003d90 con 0x7f476c10d7b0 2024-07-26T13:40:21.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.823+0000 7f47733cf640 1 -- 172.21.15.100:0/2214380261 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f476c1a0000 con 0x7f476c10d7b0 2024-07-26T13:40:21.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.823+0000 7f47627fc640 1 -- 172.21.15.100:0/2214380261 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476402fbc0 con 0x7f476c10d7b0 2024-07-26T13:40:21.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.823+0000 7f47733cf640 1 -- 172.21.15.100:0/2214380261 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f476c1a67f0 con 0x7f476c10d7b0 2024-07-26T13:40:21.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.824+0000 7f47733cf640 1 -- 172.21.15.100:0/2214380261 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f476c104610 con 0x7f476c10d7b0 2024-07-26T13:40:21.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.826+0000 7f47627fc640 1 -- 172.21.15.100:0/2214380261 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 151077+0+0 (secure 0 0 0) 0x7f476402fd60 con 0x7f476c10d7b0 2024-07-26T13:40:21.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.827+0000 7f47627fc640 1 --2- 172.21.15.100:0/2214380261 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f47580b3c20 0x7f47580b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:21.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.827+0000 7f477116b640 1 --2- 172.21.15.100:0/2214380261 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f47580b3c20 0x7f47580b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:21.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.827+0000 7f47627fc640 1 -- 172.21.15.100:0/2214380261 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f4764105710 con 0x7f476c10d7b0 2024-07-26T13:40:21.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.827+0000 7f477116b640 1 --2- 172.21.15.100:0/2214380261 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f47580b3c20 0x7f47580b6130 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4768005fd0 tx=0x7f4768005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:21.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:21.830+0000 7f47627fc640 1 -- 172.21.15.100:0/2214380261 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f47640ccc60 con 0x7f476c10d7b0 2024-07-26T13:40:22.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.038+0000 7f47733cf640 1 -- 172.21.15.100:0/2214380261 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f476c100610 con 0x7f47580b3c20 2024-07-26T13:40:22.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.044+0000 7f47627fc640 1 -- 172.21.15.100:0/2214380261 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f476c100610 con 0x7f47580b3c20 2024-07-26T13:40:22.045 INFO:teuthology.orchestra.run.smithi100.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-07-26T00:00:00", "created": "2024-07-26T13:40:21", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-07-26T13:40:22.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.047+0000 7f47733cf640 1 -- 172.21.15.100:0/2214380261 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f47580b3c20 msgr2=0x7f47580b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:22.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.047+0000 7f47733cf640 1 --2- 172.21.15.100:0/2214380261 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f47580b3c20 0x7f47580b6130 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4768005fd0 tx=0x7f4768005850 comp rx=0 tx=0).stop 2024-07-26T13:40:22.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.047+0000 7f47733cf640 1 -- 172.21.15.100:0/2214380261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f476c10d7b0 msgr2=0x7f476c19f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:22.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.047+0000 7f47733cf640 1 --2- 172.21.15.100:0/2214380261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f476c10d7b0 0x7f476c19f670 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4764005cc0 tx=0x7f476402f380 comp rx=0 tx=0).stop 2024-07-26T13:40:22.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.047+0000 7f47733cf640 1 -- 172.21.15.100:0/2214380261 shutdown_connections 2024-07-26T13:40:22.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.047+0000 7f47733cf640 1 --2- 172.21.15.100:0/2214380261 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f47580b3c20 0x7f47580b6130 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.047+0000 7f47733cf640 1 --2- 172.21.15.100:0/2214380261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f476c10d7b0 0x7f476c19f670 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.047+0000 7f47733cf640 1 --2- 172.21.15.100:0/2214380261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f476c104c30 0x7f476c19a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.047+0000 7f47733cf640 1 --2- 172.21.15.100:0/2214380261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f476c104200 0x7f476c199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.047+0000 7f47733cf640 1 -- 172.21.15.100:0/2214380261 >> 172.21.15.100:0/2214380261 conn(0x7f476c07a0a0 msgr2=0x7f476c0feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:22.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.047+0000 7f47733cf640 1 -- 172.21.15.100:0/2214380261 shutdown_connections 2024-07-26T13:40:22.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.047+0000 7f47733cf640 1 -- 172.21.15.100:0/2214380261 wait complete. 2024-07-26T13:40:22.059 DEBUG:teuthology.orchestra.run.smithi100:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-07-26T13:40:22.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.171+0000 7f20eae15640 1 -- 172.21.15.100:0/4286489358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20e4102ba0 msgr2=0x7f20e4102fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:22.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.171+0000 7f20eae15640 1 --2- 172.21.15.100:0/4286489358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20e4102ba0 0x7f20e4102fb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009900 tx=0x7f20d802a410 comp rx=0 tx=0).stop 2024-07-26T13:40:22.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.171+0000 7f20eae15640 1 -- 172.21.15.100:0/4286489358 shutdown_connections 2024-07-26T13:40:22.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.171+0000 7f20eae15640 1 --2- 172.21.15.100:0/4286489358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20e41086b0 0x7f20e410eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.171+0000 7f20eae15640 1 --2- 172.21.15.100:0/4286489358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20e4107d60 0x7f20e4108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.171+0000 7f20eae15640 1 --2- 172.21.15.100:0/4286489358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20e4102ba0 0x7f20e4102fb0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.171+0000 7f20eae15640 1 -- 172.21.15.100:0/4286489358 >> 172.21.15.100:0/4286489358 conn(0x7f20e40798c0 msgr2=0x7f20e4079cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:22.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.171+0000 7f20eae15640 1 -- 172.21.15.100:0/4286489358 shutdown_connections 2024-07-26T13:40:22.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.171+0000 7f20eae15640 1 -- 172.21.15.100:0/4286489358 wait complete. 2024-07-26T13:40:22.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.172+0000 7f20eae15640 1 Processor -- start 2024-07-26T13:40:22.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.172+0000 7f20eae15640 1 -- start start 2024-07-26T13:40:22.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.172+0000 7f20eae15640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20e4102ba0 0x7f20e410f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:22.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.172+0000 7f20eae15640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20e4107d60 0x7f20e410ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:22.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.172+0000 7f20eae15640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20e41086b0 0x7f20e4110440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:22.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.172+0000 7f20eae15640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f20e4113490 con 0x7f20e41086b0 2024-07-26T13:40:22.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.172+0000 7f20eae15640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f20e4113310 con 0x7f20e4107d60 2024-07-26T13:40:22.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.172+0000 7f20eae15640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f20e4113610 con 0x7f20e4102ba0 2024-07-26T13:40:22.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.172+0000 7f20e93b2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20e41086b0 0x7f20e4110440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:22.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.173+0000 7f20e93b2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20e41086b0 0x7f20e4110440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57100/0 (socket says 172.21.15.100:57100) 2024-07-26T13:40:22.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.173+0000 7f20e93b2640 1 -- 172.21.15.100:0/365834931 learned_addr learned my addr 172.21.15.100:0/365834931 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:22.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.173+0000 7f20e8bb1640 1 --2- 172.21.15.100:0/365834931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20e4102ba0 0x7f20e410f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:22.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.173+0000 7f20e3fff640 1 --2- 172.21.15.100:0/365834931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20e4107d60 0x7f20e410ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:22.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.173+0000 7f20e93b2640 1 -- 172.21.15.100:0/365834931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20e4102ba0 msgr2=0x7f20e410f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:22.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.173+0000 7f20e93b2640 1 --2- 172.21.15.100:0/365834931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20e4102ba0 0x7f20e410f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.173+0000 7f20e93b2640 1 -- 172.21.15.100:0/365834931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20e4107d60 msgr2=0x7f20e410ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:22.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.173+0000 7f20e93b2640 1 --2- 172.21.15.100:0/365834931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20e4107d60 0x7f20e410ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.173+0000 7f20e93b2640 1 -- 172.21.15.100:0/365834931 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e41a81a0 con 0x7f20e41086b0 2024-07-26T13:40:22.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.173+0000 7f20e3fff640 1 --2- 172.21.15.100:0/365834931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20e4107d60 0x7f20e410ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:40:22.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.173+0000 7f20e93b2640 1 --2- 172.21.15.100:0/365834931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20e41086b0 0x7f20e4110440 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f20dc009fd0 tx=0x7f20dc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:22.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.173+0000 7f20e1ffb640 1 -- 172.21.15.100:0/365834931 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20dc017070 con 0x7f20e41086b0 2024-07-26T13:40:22.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.173+0000 7f20e1ffb640 1 -- 172.21.15.100:0/365834931 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20dc003e20 con 0x7f20e41086b0 2024-07-26T13:40:22.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.173+0000 7f20eae15640 1 -- 172.21.15.100:0/365834931 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20e41a8490 con 0x7f20e41086b0 2024-07-26T13:40:22.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.173+0000 7f20e1ffb640 1 -- 172.21.15.100:0/365834931 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20dc010460 con 0x7f20e41086b0 2024-07-26T13:40:22.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.173+0000 7f20eae15640 1 -- 172.21.15.100:0/365834931 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20e41a89e0 con 0x7f20e41086b0 2024-07-26T13:40:22.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.174+0000 7f20eae15640 1 -- 172.21.15.100:0/365834931 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20e4102fb0 con 0x7f20e41086b0 2024-07-26T13:40:22.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.176+0000 7f20e1ffb640 1 -- 172.21.15.100:0/365834931 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 151077+0+0 (secure 0 0 0) 0x7f20dc003970 con 0x7f20e41086b0 2024-07-26T13:40:22.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.176+0000 7f20e1ffb640 1 --2- 172.21.15.100:0/365834931 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f20d40b3cd0 0x7f20d40b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:22.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.177+0000 7f20e8bb1640 1 --2- 172.21.15.100:0/365834931 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f20d40b3cd0 0x7f20d40b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:22.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.177+0000 7f20e1ffb640 1 -- 172.21.15.100:0/365834931 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f20dc0e39e0 con 0x7f20e41086b0 2024-07-26T13:40:22.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.177+0000 7f20e8bb1640 1 --2- 172.21.15.100:0/365834931 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f20d40b3cd0 0x7f20d40b61e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f20d8000c00 tx=0x7f20d8031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:22.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.180+0000 7f20e1ffb640 1 -- 172.21.15.100:0/365834931 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f20dc0aafb0 con 0x7f20e41086b0 2024-07-26T13:40:22.369 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:22 smithi100 ceph-mon[32599]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:40:22.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.414+0000 7f20eae15640 1 -- 172.21.15.100:0/365834931 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f20e4063230 con 0x7f20d40b3cd0 2024-07-26T13:40:22.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.422+0000 7f20e1ffb640 1 -- 172.21.15.100:0/365834931 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f20e4063230 con 0x7f20d40b3cd0 2024-07-26T13:40:22.423 INFO:teuthology.orchestra.run.smithi100.stdout:/ 1m 6m3h 2024-07-26T13:40:22.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.425+0000 7f20eae15640 1 -- 172.21.15.100:0/365834931 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f20d40b3cd0 msgr2=0x7f20d40b61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:22.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.425+0000 7f20eae15640 1 --2- 172.21.15.100:0/365834931 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f20d40b3cd0 0x7f20d40b61e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f20d8000c00 tx=0x7f20d8031400 comp rx=0 tx=0).stop 2024-07-26T13:40:22.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.425+0000 7f20eae15640 1 -- 172.21.15.100:0/365834931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20e41086b0 msgr2=0x7f20e4110440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:22.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.425+0000 7f20eae15640 1 --2- 172.21.15.100:0/365834931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20e41086b0 0x7f20e4110440 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f20dc009fd0 tx=0x7f20dc00b260 comp rx=0 tx=0).stop 2024-07-26T13:40:22.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.425+0000 7f20eae15640 1 -- 172.21.15.100:0/365834931 shutdown_connections 2024-07-26T13:40:22.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.425+0000 7f20eae15640 1 --2- 172.21.15.100:0/365834931 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f20d40b3cd0 0x7f20d40b61e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.425+0000 7f20eae15640 1 --2- 172.21.15.100:0/365834931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20e41086b0 0x7f20e4110440 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.425+0000 7f20eae15640 1 --2- 172.21.15.100:0/365834931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20e4107d60 0x7f20e410ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.425+0000 7f20eae15640 1 --2- 172.21.15.100:0/365834931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20e4102ba0 0x7f20e410f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.425+0000 7f20eae15640 1 -- 172.21.15.100:0/365834931 >> 172.21.15.100:0/365834931 conn(0x7f20e40798c0 msgr2=0x7f20e4105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:22.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.425+0000 7f20eae15640 1 -- 172.21.15.100:0/365834931 shutdown_connections 2024-07-26T13:40:22.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.425+0000 7f20eae15640 1 -- 172.21.15.100:0/365834931 wait complete. 2024-07-26T13:40:22.437 DEBUG:teuthology.orchestra.run.smithi100:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-07-26T13:40:22.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:22 smithi138 ceph-mon[36954]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:40:22.463 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-07-26T13:40:22.476 INFO:teuthology.run_tasks:Running task background_exec... 2024-07-26T13:40:22.514 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi100.front.sepia.ceph.com 2024-07-26T13:40:22.514 DEBUG:teuthology.orchestra.run.smithi100:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-07-26T13:40:22.517 INFO:teuthology.run_tasks:Running task quiescer... 2024-07-26T13:40:22.530 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-07-26T13:40:22.530 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph pg dump --format=json 2024-07-26T13:40:22.531 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:22 smithi167 ceph-mon[37140]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:40:22.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.746+0000 7fc8d0797640 1 -- 172.21.15.100:0/644314785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8c8107dd0 msgr2=0x7fc8c81081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:22.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.746+0000 7fc8d0797640 1 --2- 172.21.15.100:0/644314785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8c8107dd0 0x7fc8c81081e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8009900 tx=0x7fc8b802a410 comp rx=0 tx=0).stop 2024-07-26T13:40:22.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.747+0000 7fc8d0797640 1 -- 172.21.15.100:0/644314785 shutdown_connections 2024-07-26T13:40:22.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.747+0000 7fc8d0797640 1 --2- 172.21.15.100:0/644314785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8c8108720 0x7fc8c810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.747+0000 7fc8d0797640 1 --2- 172.21.15.100:0/644314785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8c8107dd0 0x7fc8c81081e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.747+0000 7fc8d0797640 1 --2- 172.21.15.100:0/644314785 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8c8102490 0x7fc8c81028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.747+0000 7fc8d0797640 1 -- 172.21.15.100:0/644314785 >> 172.21.15.100:0/644314785 conn(0x7fc8c8079160 msgr2=0x7fc8c8079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:22.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.747+0000 7fc8d0797640 1 -- 172.21.15.100:0/644314785 shutdown_connections 2024-07-26T13:40:22.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.747+0000 7fc8d0797640 1 -- 172.21.15.100:0/644314785 wait complete. 2024-07-26T13:40:22.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.747+0000 7fc8d0797640 1 Processor -- start 2024-07-26T13:40:22.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.747+0000 7fc8d0797640 1 -- start start 2024-07-26T13:40:22.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.748+0000 7fc8d0797640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8c8102490 0x7fc8c819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:22.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.748+0000 7fc8d0797640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8c8107dd0 0x7fc8c819b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:22.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.748+0000 7fc8d0797640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8c8108720 0x7fc8c819b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:22.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.748+0000 7fc8d0797640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c8112370 con 0x7fc8c8108720 2024-07-26T13:40:22.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.748+0000 7fc8d0797640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c81121f0 con 0x7fc8c8107dd0 2024-07-26T13:40:22.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.748+0000 7fc8d0797640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c81124f0 con 0x7fc8c8102490 2024-07-26T13:40:22.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.748+0000 7fc8ced34640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8c8108720 0x7fc8c819b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:22.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.748+0000 7fc8ced34640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8c8108720 0x7fc8c819b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57120/0 (socket says 172.21.15.100:57120) 2024-07-26T13:40:22.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.748+0000 7fc8ced34640 1 -- 172.21.15.100:0/817909380 learned_addr learned my addr 172.21.15.100:0/817909380 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:22.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.748+0000 7fc8cdd32640 1 --2- 172.21.15.100:0/817909380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8c8107dd0 0x7fc8c819b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:22.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.748+0000 7fc8ce533640 1 --2- 172.21.15.100:0/817909380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8c8102490 0x7fc8c819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:22.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.748+0000 7fc8ced34640 1 -- 172.21.15.100:0/817909380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8c8102490 msgr2=0x7fc8c819ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:22.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.748+0000 7fc8ced34640 1 --2- 172.21.15.100:0/817909380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8c8102490 0x7fc8c819ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.748+0000 7fc8ced34640 1 -- 172.21.15.100:0/817909380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8c8107dd0 msgr2=0x7fc8c819b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:22.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.748+0000 7fc8ced34640 1 --2- 172.21.15.100:0/817909380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8c8107dd0 0x7fc8c819b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.748+0000 7fc8ced34640 1 -- 172.21.15.100:0/817909380 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c81a5f90 con 0x7fc8c8108720 2024-07-26T13:40:22.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.748+0000 7fc8ce533640 1 --2- 172.21.15.100:0/817909380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8c8102490 0x7fc8c819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:40:22.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.748+0000 7fc8ced34640 1 --2- 172.21.15.100:0/817909380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8c8108720 0x7fc8c819b7e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc8c000cce0 tx=0x7fc8c000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:22.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.748+0000 7fc8bf7fe640 1 -- 172.21.15.100:0/817909380 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c000f450 con 0x7fc8c8108720 2024-07-26T13:40:22.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.748+0000 7fc8bf7fe640 1 -- 172.21.15.100:0/817909380 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8c000faa0 con 0x7fc8c8108720 2024-07-26T13:40:22.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.749+0000 7fc8d0797640 1 -- 172.21.15.100:0/817909380 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8c81a6280 con 0x7fc8c8108720 2024-07-26T13:40:22.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.749+0000 7fc8bf7fe640 1 -- 172.21.15.100:0/817909380 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0018b70 con 0x7fc8c8108720 2024-07-26T13:40:22.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.749+0000 7fc8d0797640 1 -- 172.21.15.100:0/817909380 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8c81a6790 con 0x7fc8c8108720 2024-07-26T13:40:22.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.751+0000 7fc8d6d1b740 1 Processor -- start 2024-07-26T13:40:22.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.751+0000 7fc8bf7fe640 1 -- 172.21.15.100:0/817909380 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 23) ==== 151077+0+0 (secure 0 0 0) 0x7fc8c0018d10 con 0x7fc8c8108720 2024-07-26T13:40:22.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.752+0000 7fc8d6d1b740 1 -- start start 2024-07-26T13:40:22.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.752+0000 7fc8d6d1b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640e4f3c350 0x5640e52e13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:22.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.752+0000 7fc8d6d1b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e4f13490 0x5640e52e18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:22.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.752+0000 7fc8d6d1b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e4f1b670 0x5640e52e1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:22.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.752+0000 7fc8d6d1b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5640e51c92f0 con 0x5640e4f1b670 2024-07-26T13:40:22.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.752+0000 7fc8d6d1b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5640e5257a80 con 0x5640e4f3c350 2024-07-26T13:40:22.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.752+0000 7fc8d6d1b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5640e5246f80 con 0x5640e4f13490 2024-07-26T13:40:22.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.752+0000 7fc8cdd32640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e4f1b670 0x5640e52e1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:22.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.752+0000 7fc8cdd32640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640e4f3c350 0x5640e52e13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:22.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.752+0000 7fc8cdd32640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640e4f3c350 0x5640e52e13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:56318/0 (socket says 172.21.15.100:56318) 2024-07-26T13:40:22.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.752+0000 7fc8cdd32640 1 -- 172.21.15.100:0/2338459371 learned_addr learned my addr 172.21.15.100:0/2338459371 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:22.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.752+0000 7fc8ce533640 1 --2- 172.21.15.100:0/2338459371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e4f13490 0x5640e52e18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:22.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.752+0000 7fc8bf7fe640 1 --2- 172.21.15.100:0/817909380 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc8b40b3ca0 0x7fc8b40b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:22.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.753+0000 7fc8cdd32640 1 -- 172.21.15.100:0/2338459371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e4f13490 msgr2=0x5640e52e18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:22.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.753+0000 7fc8cdd32640 1 --2- 172.21.15.100:0/2338459371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e4f13490 0x5640e52e18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.753+0000 7fc8cdd32640 1 -- 172.21.15.100:0/2338459371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e4f1b670 msgr2=0x5640e52e1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:22.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.753+0000 7fc8cdd32640 1 --2- 172.21.15.100:0/2338459371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e4f1b670 0x5640e52e1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.753+0000 7fc8cdd32640 1 -- 172.21.15.100:0/2338459371 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640e4fa7930 con 0x5640e4f3c350 2024-07-26T13:40:22.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.753+0000 7fc8bf7fe640 1 -- 172.21.15.100:0/817909380 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fc8c00e40c0 con 0x7fc8c8108720 2024-07-26T13:40:22.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.753+0000 7fc8ced34640 1 --2- 172.21.15.100:0/817909380 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc8b40b3ca0 0x7fc8b40b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:22.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.753+0000 7fc8cdd32640 1 --2- 172.21.15.100:0/2338459371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640e4f3c350 0x5640e52e13b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc8b80096c0 tx=0x7fc8b8031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad074b5fbfb066c7 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:22.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.753+0000 7fc8ced34640 1 --2- 172.21.15.100:0/817909380 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc8b40b3ca0 0x7fc8b40b61b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc8c00a6f90 tx=0x7fc8c000ff50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:22.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.753+0000 7fc8bcff9640 1 -- 172.21.15.100:0/2338459371 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8035050 con 0x5640e4f3c350 2024-07-26T13:40:22.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.754+0000 7fc8bcff9640 1 -- 172.21.15.100:0/2338459371 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8b80035f0 con 0x5640e4f3c350 2024-07-26T13:40:22.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.754+0000 7fc8d6d1b740 1 -- 172.21.15.100:0/2338459371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640e4f3c350 msgr2=0x5640e52e13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:22.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.754+0000 7fc8d6d1b740 1 --2- 172.21.15.100:0/2338459371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640e4f3c350 0x5640e52e13b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc8b80096c0 tx=0x7fc8b8031400 comp rx=0 tx=0).stop 2024-07-26T13:40:22.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.754+0000 7fc8d6d1b740 1 -- 172.21.15.100:0/2338459371 shutdown_connections 2024-07-26T13:40:22.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.754+0000 7fc8d6d1b740 1 --2- 172.21.15.100:0/2338459371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e4f1b670 0x5640e52e1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.754+0000 7fc8d6d1b740 1 --2- 172.21.15.100:0/2338459371 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e4f13490 0x5640e52e18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.754+0000 7fc8d6d1b740 1 --2- 172.21.15.100:0/2338459371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640e4f3c350 0x5640e52e13b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.754+0000 7fc8d6d1b740 1 -- 172.21.15.100:0/2338459371 >> 172.21.15.100:0/2338459371 conn(0x5640e4f30740 msgr2=0x5640e52df2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:22.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.755+0000 7fc8d6d1b740 1 -- 172.21.15.100:0/2338459371 shutdown_connections 2024-07-26T13:40:22.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.755+0000 7fc8d6d1b740 1 -- 172.21.15.100:0/2338459371 wait complete. 2024-07-26T13:40:22.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.755+0000 7fc8d6d1b740 1 -- start start 2024-07-26T13:40:22.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.755+0000 7fc8d6d1b740 1 Processor -- start 2024-07-26T13:40:22.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.755+0000 7fc8d6d1b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e4f13490 0x5640e52ecc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:22.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.755+0000 7fc8d6d1b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640e4f1b670 0x5640e52ed200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:22.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.755+0000 7fc8d6d1b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e4f3c350 0x5640e52f8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:22.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.755+0000 7fc8d6d1b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5640e51e94f0 con 0x5640e4f3c350 2024-07-26T13:40:22.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.755+0000 7fc8d6d1b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5640e524afc0 con 0x5640e4f1b670 2024-07-26T13:40:22.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.755+0000 7fc8d6d1b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5640e50aa890 con 0x5640e4f13490 2024-07-26T13:40:22.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.755+0000 7fc8d6d1b740 10 client.0 _ll_register_callbacks cb 0x5640e51f3500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:40:22.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.755+0000 7fc8cdd32640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e4f3c350 0x5640e52f8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:22.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.755+0000 7fc8cdd32640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e4f3c350 0x5640e52f8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57142/0 (socket says 172.21.15.100:57142) 2024-07-26T13:40:22.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.755+0000 7fc8cdd32640 1 -- 172.21.15.100:0/4146399681 learned_addr learned my addr 172.21.15.100:0/4146399681 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:22.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.756+0000 7fc8cdd32640 1 --2- 172.21.15.100:0/4146399681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e4f13490 0x5640e52ecc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:22.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.756+0000 7fc8ce533640 1 --2- 172.21.15.100:0/4146399681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640e4f1b670 0x5640e52ed200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:22.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.756+0000 7fc8cdd32640 1 -- 172.21.15.100:0/4146399681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e4f13490 msgr2=0x5640e52ecc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:22.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.756+0000 7fc8cdd32640 1 --2- 172.21.15.100:0/4146399681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e4f13490 0x5640e52ecc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.756+0000 7fc8cdd32640 1 -- 172.21.15.100:0/4146399681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640e4f1b670 msgr2=0x5640e52ed200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:22.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.756+0000 7fc8cdd32640 1 --2- 172.21.15.100:0/4146399681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640e4f1b670 0x5640e52ed200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.756+0000 7fc8cdd32640 1 -- 172.21.15.100:0/4146399681 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640e51f8c50 con 0x5640e4f3c350 2024-07-26T13:40:22.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.756+0000 7fc8ce533640 1 --2- 172.21.15.100:0/4146399681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640e4f1b670 0x5640e52ed200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:40:22.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.756+0000 7fc8cdd32640 1 --2- 172.21.15.100:0/4146399681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e4f3c350 0x5640e52f8d90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc8b802a930 tx=0x7fc8b8041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11bc21ede5beeca4 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:22.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.756+0000 7fc8b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:40:22.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.756+0000 7fc8b1ffb640 1 -- 172.21.15.100:0/4146399681 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8031d10 con 0x5640e4f3c350 2024-07-26T13:40:22.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.756+0000 7fc8b1ffb640 1 -- 172.21.15.100:0/4146399681 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fc8b80037a0 con 0x5640e4f3c350 2024-07-26T13:40:22.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.756+0000 7fc8d6d1b740 1 -- 172.21.15.100:0/4146399681 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8b80035f0 con 0x5640e4f3c350 2024-07-26T13:40:22.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.756+0000 7fc8b1ffb640 1 -- 172.21.15.100:0/4146399681 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc8b80039a0 con 0x5640e4f3c350 2024-07-26T13:40:22.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.756+0000 7fc8d6d1b740 10 client.15441 fetch_fsmap learned FSMap version 26 2024-07-26T13:40:22.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.756+0000 7fc8d6d1b740 1 -- 172.21.15.100:0/4146399681 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5640e52fb960 con 0x5640e4f3c350 2024-07-26T13:40:22.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.757+0000 7fc8b1ffb640 1 -- 172.21.15.100:0/4146399681 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8b8005500 con 0x5640e4f3c350 2024-07-26T13:40:22.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.757+0000 7fc8b1ffb640 1 -- 172.21.15.100:0/4146399681 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fc8b8004750 con 0x5640e4f3c350 2024-07-26T13:40:22.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.757+0000 7fc8d6d1b740 10 client.15441 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:40:22.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.757+0000 7fc8d6d1b740 10 client.15441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:40:22.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.757+0000 7fc8d6d1b740 20 client.15441 populate_metadata read hostname 'smithi100' 2024-07-26T13:40:22.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.757+0000 7fc8d6d1b740 1 --2- 172.21.15.100:0/4146399681 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5640e4efacd0 0x5640e52fecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:22.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.757+0000 7fc8d6d1b740 4 client.15441 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:40:22.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.757+0000 7fc8d6d1b740 1 -- 172.21.15.100:0/4146399681 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc8b80035f0 con 0x5640e4efacd0 2024-07-26T13:40:22.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.757+0000 7fc8ce533640 1 --2- 172.21.15.100:0/4146399681 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5640e4efacd0 0x5640e52fecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:22.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.758+0000 7fc8ce533640 1 --2- 172.21.15.100:0/4146399681 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5640e4efacd0 0x5640e52fecc0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c666ab74e36d892c server_cookie=c3c98d14bb779ebe in_seq=0 out_seq=0 2024-07-26T13:40:22.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.758+0000 7fc8b1ffb640 10 client.15441 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:40:22.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.759+0000 7fc8b1ffb640 1 -- 172.21.15.100:0/4146399681 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fc8b80035f0 con 0x5640e4efacd0 2024-07-26T13:40:22.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.759+0000 7fc8b1ffb640 10 client.15441 handle_command_reply: tid=0 2024-07-26T13:40:22.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.759+0000 7fc8d6d1b740 1 client.15441 shutdown 2024-07-26T13:40:22.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.759+0000 7fc8d6d1b740 1 -- 172.21.15.100:0/4146399681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e4f3c350 msgr2=0x5640e52f8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:22.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.759+0000 7fc8d6d1b740 1 --2- 172.21.15.100:0/4146399681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e4f3c350 0x5640e52f8d90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc8b802a930 tx=0x7fc8b8041040 comp rx=0 tx=0).stop 2024-07-26T13:40:22.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.759+0000 7fc8d6d1b740 1 -- 172.21.15.100:0/4146399681 shutdown_connections 2024-07-26T13:40:22.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.759+0000 7fc8d6d1b740 1 --2- 172.21.15.100:0/4146399681 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5640e4efacd0 0x5640e52fecc0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.759+0000 7fc8d6d1b740 1 --2- 172.21.15.100:0/4146399681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e4f3c350 0x5640e52f8d90 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.759+0000 7fc8d6d1b740 1 --2- 172.21.15.100:0/4146399681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640e4f1b670 0x5640e52ed200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.759+0000 7fc8d6d1b740 1 --2- 172.21.15.100:0/4146399681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e4f13490 0x5640e52ecc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.759+0000 7fc8d6d1b740 1 -- 172.21.15.100:0/4146399681 >> 172.21.15.100:0/4146399681 conn(0x5640e4f30740 msgr2=0x5640e52e4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:22.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.760+0000 7fc8b1ffb640 0 client.15441 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:40:22.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.760+0000 7fc8d6d1b740 1 -- 172.21.15.100:0/4146399681 shutdown_connections 2024-07-26T13:40:22.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.760+0000 7fc8d6d1b740 1 -- 172.21.15.100:0/4146399681 wait complete. 2024-07-26T13:40:22.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.760+0000 7fc8d6d1b740 20 client.15441 trim_cache size 0 max 16384 2024-07-26T13:40:22.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.777+0000 7fc8d6d1b740 1 Processor -- start 2024-07-26T13:40:22.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.777+0000 7fc8d6d1b740 1 -- start start 2024-07-26T13:40:22.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.778+0000 7fc8d6d1b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640e4f30740 0x5640e52eeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:22.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.778+0000 7fc8d6d1b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e4f13490 0x5640e52e29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:22.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.778+0000 7fc8d6d1b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e4f1b670 0x5640e52e2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:22.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.778+0000 7fc8ce533640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e4f13490 0x5640e52e29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:22.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.778+0000 7fc8ce533640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e4f13490 0x5640e52e29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57144/0 (socket says 172.21.15.100:57144) 2024-07-26T13:40:22.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.778+0000 7fc8ce533640 1 -- 172.21.15.100:0/1764593410 learned_addr learned my addr 172.21.15.100:0/1764593410 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:22.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.778+0000 7fc8d6d1b740 1 -- 172.21.15.100:0/1764593410 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c81b1910 con 0x5640e4f13490 2024-07-26T13:40:22.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.778+0000 7fc8d6d1b740 1 -- 172.21.15.100:0/1764593410 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c81af1e0 con 0x5640e4f30740 2024-07-26T13:40:22.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.778+0000 7fc8d6d1b740 1 -- 172.21.15.100:0/1764593410 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c81ad030 con 0x5640e4f1b670 2024-07-26T13:40:22.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.778+0000 7fc8cdd32640 1 --2- 172.21.15.100:0/1764593410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640e4f30740 0x5640e52eeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:22.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.778+0000 7fc8cdd32640 1 --2- 172.21.15.100:0/1764593410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e4f1b670 0x5640e52e2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:22.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.779+0000 7fc8cdd32640 1 -- 172.21.15.100:0/1764593410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e4f1b670 msgr2=0x5640e52e2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:22.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.779+0000 7fc8cdd32640 1 --2- 172.21.15.100:0/1764593410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e4f1b670 0x5640e52e2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.779+0000 7fc8cdd32640 1 -- 172.21.15.100:0/1764593410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e4f13490 msgr2=0x5640e52e29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:22.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.779+0000 7fc8cdd32640 1 --2- 172.21.15.100:0/1764593410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e4f13490 0x5640e52e29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.779+0000 7fc8cdd32640 1 -- 172.21.15.100:0/1764593410 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640e52e38d0 con 0x5640e4f30740 2024-07-26T13:40:22.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.779+0000 7fc8cdd32640 1 --2- 172.21.15.100:0/1764593410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640e4f30740 0x5640e52eeb00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8004bf0 tx=0x7fc8b8005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3a5e80d13b72458 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:22.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.779+0000 7fc8b3fff640 1 -- 172.21.15.100:0/1764593410 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8035070 con 0x5640e4f30740 2024-07-26T13:40:22.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.779+0000 7fc8b3fff640 1 -- 172.21.15.100:0/1764593410 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8b80037f0 con 0x5640e4f30740 2024-07-26T13:40:22.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.779+0000 7fc8b3fff640 1 -- 172.21.15.100:0/1764593410 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8031be0 con 0x5640e4f30740 2024-07-26T13:40:22.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.780+0000 7fc8d6d1b740 1 -- 172.21.15.100:0/1764593410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640e4f30740 msgr2=0x5640e52eeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:22.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.780+0000 7fc8d6d1b740 1 --2- 172.21.15.100:0/1764593410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640e4f30740 0x5640e52eeb00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8004bf0 tx=0x7fc8b8005fd0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.780+0000 7fc8d6d1b740 1 -- 172.21.15.100:0/1764593410 shutdown_connections 2024-07-26T13:40:22.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.780+0000 7fc8d6d1b740 1 --2- 172.21.15.100:0/1764593410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e4f1b670 0x5640e52e2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.780+0000 7fc8d6d1b740 1 --2- 172.21.15.100:0/1764593410 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e4f13490 0x5640e52e29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.780+0000 7fc8d6d1b740 1 --2- 172.21.15.100:0/1764593410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640e4f30740 0x5640e52eeb00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.780+0000 7fc8d6d1b740 1 -- 172.21.15.100:0/1764593410 >> 172.21.15.100:0/1764593410 conn(0x5640e52df420 msgr2=0x5640e52e2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:22.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.780+0000 7fc8d6d1b740 1 -- 172.21.15.100:0/1764593410 shutdown_connections 2024-07-26T13:40:22.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.780+0000 7fc8d6d1b740 1 -- 172.21.15.100:0/1764593410 wait complete. 2024-07-26T13:40:22.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.781+0000 7fc8d6d1b740 1 -- start start 2024-07-26T13:40:22.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.781+0000 7fc8d6d1b740 1 Processor -- start 2024-07-26T13:40:22.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.781+0000 7fc8d6d1b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640e4f13490 0x5640e52ffdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:22.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.781+0000 7fc8d6d1b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e4f1b670 0x5640e53002f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:22.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.781+0000 7fc8d6d1b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e4f30740 0x5640e5300830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:22.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.781+0000 7fc8d6d1b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c81ad6a0 con 0x5640e4f30740 2024-07-26T13:40:22.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.781+0000 7fc8d6d1b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c81aeb10 con 0x5640e4f13490 2024-07-26T13:40:22.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.781+0000 7fc8d6d1b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c81add70 con 0x5640e4f1b670 2024-07-26T13:40:22.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.781+0000 7fc8d6d1b740 10 client.0 _ll_register_callbacks cb 0x7fc8b8031840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:40:22.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.782+0000 7fc8cdd32640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e4f30740 0x5640e5300830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:22.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.782+0000 7fc8cdd32640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e4f30740 0x5640e5300830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57154/0 (socket says 172.21.15.100:57154) 2024-07-26T13:40:22.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.782+0000 7fc8cdd32640 1 -- 172.21.15.100:0/1802740379 learned_addr learned my addr 172.21.15.100:0/1802740379 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:22.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.782+0000 7fc8ce533640 1 --2- 172.21.15.100:0/1802740379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e4f1b670 0x5640e53002f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:22.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.782+0000 7fc8cdd32640 1 --2- 172.21.15.100:0/1802740379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640e4f13490 0x5640e52ffdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:22.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.782+0000 7fc8cdd32640 1 -- 172.21.15.100:0/1802740379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e4f1b670 msgr2=0x5640e53002f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:22.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.782+0000 7fc8cdd32640 1 --2- 172.21.15.100:0/1802740379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e4f1b670 0x5640e53002f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.782+0000 7fc8cdd32640 1 -- 172.21.15.100:0/1802740379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e4f30740 msgr2=0x5640e5300830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:22.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.782+0000 7fc8cdd32640 1 --2- 172.21.15.100:0/1802740379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e4f30740 0x5640e5300830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.782+0000 7fc8cdd32640 1 -- 172.21.15.100:0/1802740379 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640e536a820 con 0x5640e4f13490 2024-07-26T13:40:22.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.782+0000 7fc8ce533640 1 --2- 172.21.15.100:0/1802740379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e4f1b670 0x5640e53002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:40:22.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.782+0000 7fc8cdd32640 1 --2- 172.21.15.100:0/1802740379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640e4f13490 0x5640e52ffdb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8009900 tx=0x7fc8b803b6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61035a58c23512d6 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:22.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.782+0000 7fc8b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-07-26T13:40:22.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.782+0000 7fc8b1ffb640 1 -- 172.21.15.100:0/1802740379 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8035030 con 0x5640e4f13490 2024-07-26T13:40:22.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.782+0000 7fc8b1ffb640 1 -- 172.21.15.100:0/1802740379 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fc8b80048d0 con 0x5640e4f13490 2024-07-26T13:40:22.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.782+0000 7fc8d6d1b740 1 -- 172.21.15.100:0/1802740379 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8b80037f0 con 0x5640e4f13490 2024-07-26T13:40:22.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.783+0000 7fc8b1ffb640 1 -- 172.21.15.100:0/1802740379 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc8b803b770 con 0x5640e4f13490 2024-07-26T13:40:22.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.783+0000 7fc8d6d1b740 10 client.24983 fetch_fsmap learned FSMap version 26 2024-07-26T13:40:22.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.783+0000 7fc8d6d1b740 1 -- 172.21.15.100:0/1802740379 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5640e536bac0 con 0x5640e4f13490 2024-07-26T13:40:22.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.783+0000 7fc8b1ffb640 1 -- 172.21.15.100:0/1802740379 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8b80037f0 con 0x5640e4f13490 2024-07-26T13:40:22.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.783+0000 7fc8b1ffb640 1 -- 172.21.15.100:0/1802740379 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fc8b80050b0 con 0x5640e4f13490 2024-07-26T13:40:22.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.783+0000 7fc8d6d1b740 10 client.24983 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:40:22.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.783+0000 7fc8d6d1b740 10 client.24983 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:40:22.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.783+0000 7fc8d6d1b740 20 client.24983 populate_metadata read hostname 'smithi100' 2024-07-26T13:40:22.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.783+0000 7fc8d6d1b740 1 --2- 172.21.15.100:0/1802740379 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5640e4f3c350 0x5640e536dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:22.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.783+0000 7fc8d6d1b740 4 client.24983 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-26T13:40:22.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.783+0000 7fc8d6d1b740 1 -- 172.21.15.100:0/1802740379 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5640e52599d0 con 0x5640e4f3c350 2024-07-26T13:40:22.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.783+0000 7fc8ce533640 1 --2- 172.21.15.100:0/1802740379 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5640e4f3c350 0x5640e536dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:22.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.784+0000 7fc8b1ffb640 10 client.24983 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:40:22.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.784+0000 7fc8ce533640 1 --2- 172.21.15.100:0/1802740379 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5640e4f3c350 0x5640e536dfe0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2749aa720dbe3b2 server_cookie=a1d058f763c2cf91 in_seq=0 out_seq=0 2024-07-26T13:40:22.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.785+0000 7fc8b1ffb640 1 -- 172.21.15.100:0/1802740379 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5640e52599d0 con 0x5640e4f3c350 2024-07-26T13:40:22.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.785+0000 7fc8b1ffb640 10 client.24983 handle_command_reply: tid=0 2024-07-26T13:40:22.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.785+0000 7fc8d6d1b740 1 client.24983 shutdown 2024-07-26T13:40:22.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.785+0000 7fc8d6d1b740 1 -- 172.21.15.100:0/1802740379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640e4f13490 msgr2=0x5640e52ffdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:22.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.785+0000 7fc8d6d1b740 1 --2- 172.21.15.100:0/1802740379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640e4f13490 0x5640e52ffdb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8009900 tx=0x7fc8b803b6b0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.785+0000 7fc8d6d1b740 1 -- 172.21.15.100:0/1802740379 shutdown_connections 2024-07-26T13:40:22.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.785+0000 7fc8d6d1b740 1 --2- 172.21.15.100:0/1802740379 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5640e4f3c350 0x5640e536dfe0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.785+0000 7fc8d6d1b740 1 --2- 172.21.15.100:0/1802740379 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640e4f30740 0x5640e5300830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.785+0000 7fc8d6d1b740 1 --2- 172.21.15.100:0/1802740379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640e4f1b670 0x5640e53002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.785+0000 7fc8d6d1b740 1 --2- 172.21.15.100:0/1802740379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640e4f13490 0x5640e52ffdb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.785+0000 7fc8d6d1b740 1 -- 172.21.15.100:0/1802740379 >> 172.21.15.100:0/1802740379 conn(0x5640e52df420 msgr2=0x5640e52e2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:22.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.785+0000 7fc8b1ffb640 0 client.24983 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:40:22.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.786+0000 7fc8d6d1b740 1 -- 172.21.15.100:0/1802740379 shutdown_connections 2024-07-26T13:40:22.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.786+0000 7fc8d6d1b740 1 -- 172.21.15.100:0/1802740379 wait complete. 2024-07-26T13:40:22.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.786+0000 7fc8d6d1b740 20 client.24983 trim_cache size 0 max 16384 2024-07-26T13:40:22.790 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:40:22.790 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-07-26T13:40:22.790 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "cef38ea4-fdd8-460a-89cf-1770cd63e758", 2024-07-26T13:40:22.790 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-07-26T13:40:22.790 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:40:22.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.786+0000 7fc8d0797640 1 -- 172.21.15.100:0/817909380 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc8b40b3ca0 msgr2=0x7fc8b40b61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:22.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.786+0000 7fc8d0797640 1 --2- 172.21.15.100:0/817909380 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc8b40b3ca0 0x7fc8b40b61b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc8c00a6f90 tx=0x7fc8c000ff50 comp rx=0 tx=0).stop 2024-07-26T13:40:22.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.786+0000 7fc8d0797640 1 -- 172.21.15.100:0/817909380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8c8108720 msgr2=0x7fc8c819b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:22.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.786+0000 7fc8d0797640 1 --2- 172.21.15.100:0/817909380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8c8108720 0x7fc8c819b7e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc8c000cce0 tx=0x7fc8c000a620 comp rx=0 tx=0).stop 2024-07-26T13:40:22.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.786+0000 7fc8d0797640 1 -- 172.21.15.100:0/817909380 shutdown_connections 2024-07-26T13:40:22.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.786+0000 7fc8d0797640 1 --2- 172.21.15.100:0/817909380 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc8b40b3ca0 0x7fc8b40b61b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.786+0000 7fc8d0797640 1 --2- 172.21.15.100:0/817909380 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8c8108720 0x7fc8c819b7e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.786+0000 7fc8d0797640 1 --2- 172.21.15.100:0/817909380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8c8107dd0 0x7fc8c819b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.786+0000 7fc8d0797640 1 --2- 172.21.15.100:0/817909380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8c8102490 0x7fc8c819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:22.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.787+0000 7fc8d0797640 1 -- 172.21.15.100:0/817909380 >> 172.21.15.100:0/817909380 conn(0x7fc8c8079160 msgr2=0x7fc8c8103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:22.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.787+0000 7fc8d0797640 1 -- 172.21.15.100:0/817909380 shutdown_connections 2024-07-26T13:40:22.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:22.787+0000 7fc8d0797640 1 -- 172.21.15.100:0/817909380 wait complete. 2024-07-26T13:40:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:23 smithi138 ceph-mon[36954]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:40:23.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:23 smithi138 ceph-mon[36954]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:40:23.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:23 smithi138 ceph-mon[36954]: pgmap v372: 97 pgs: 97 active+clean; 1.0 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 73 KiB/s wr, 14 op/s 2024-07-26T13:40:23.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:23 smithi138 ceph-mon[36954]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:40:23.691 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:23 smithi167 ceph-mon[37140]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:40:23.691 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:23 smithi167 ceph-mon[37140]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:40:23.691 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:23 smithi167 ceph-mon[37140]: pgmap v372: 97 pgs: 97 active+clean; 1.0 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 73 KiB/s wr, 14 op/s 2024-07-26T13:40:23.691 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:23 smithi167 ceph-mon[37140]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:40:23.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:23 smithi100 ceph-mon[32599]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:40:23.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:23 smithi100 ceph-mon[32599]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:40:23.691 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:23 smithi100 ceph-mon[32599]: pgmap v372: 97 pgs: 97 active+clean; 1.0 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 73 KiB/s wr, 14 op/s 2024-07-26T13:40:23.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:23 smithi100 ceph-mon[32599]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:40:23.692 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:40:23.693 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-26T13:40:24.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:24 smithi138 ceph-mon[36954]: scrub queued for path: / 2024-07-26T13:40:24.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:24 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:40:24.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:24 smithi138 ceph-mon[36954]: scrub summary: active paths [/] 2024-07-26T13:40:24.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:24 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:40:24.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:24 smithi138 ceph-mon[36954]: mgrmap e24: x(active, since 10m), standbys: y, z 2024-07-26T13:40:24.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:24 smithi100 ceph-mon[32599]: scrub queued for path: / 2024-07-26T13:40:24.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:24 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:40:24.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:24 smithi100 ceph-mon[32599]: scrub summary: active paths [/] 2024-07-26T13:40:24.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:24 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:40:24.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:24 smithi100 ceph-mon[32599]: mgrmap e24: x(active, since 10m), standbys: y, z 2024-07-26T13:40:24.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:24 smithi167 ceph-mon[37140]: scrub queued for path: / 2024-07-26T13:40:24.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:24 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:40:24.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:24 smithi167 ceph-mon[37140]: scrub summary: active paths [/] 2024-07-26T13:40:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:24 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:40:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:24 smithi167 ceph-mon[37140]: mgrmap e24: x(active, since 10m), standbys: y, z 2024-07-26T13:40:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:25 smithi100 ceph-mon[32599]: pgmap v373: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 174 KiB/s wr, 53 op/s 2024-07-26T13:40:25.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.190+0000 7fd715aff640 1 -- 172.21.15.100:0/2818723214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd710107dd0 msgr2=0x7fd7101081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:25.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.190+0000 7fd715aff640 1 --2- 172.21.15.100:0/2818723214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd710107dd0 0x7fd7101081e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8009900 tx=0x7fd6f802a410 comp rx=0 tx=0).stop 2024-07-26T13:40:25.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.190+0000 7fd715aff640 1 -- 172.21.15.100:0/2818723214 shutdown_connections 2024-07-26T13:40:25.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.190+0000 7fd715aff640 1 --2- 172.21.15.100:0/2818723214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd710108720 0x7fd71010ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:25.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.190+0000 7fd715aff640 1 --2- 172.21.15.100:0/2818723214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd710107dd0 0x7fd7101081e0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:25.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.190+0000 7fd715aff640 1 --2- 172.21.15.100:0/2818723214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd710102490 0x7fd7101028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:25.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.190+0000 7fd715aff640 1 -- 172.21.15.100:0/2818723214 >> 172.21.15.100:0/2818723214 conn(0x7fd710079160 msgr2=0x7fd710079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:25.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.190+0000 7fd715aff640 1 -- 172.21.15.100:0/2818723214 shutdown_connections 2024-07-26T13:40:25.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.190+0000 7fd715aff640 1 -- 172.21.15.100:0/2818723214 wait complete. 2024-07-26T13:40:25.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.191+0000 7fd715aff640 1 Processor -- start 2024-07-26T13:40:25.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.191+0000 7fd715aff640 1 -- start start 2024-07-26T13:40:25.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.191+0000 7fd715aff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd710102490 0x7fd71019c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:25.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.191+0000 7fd715aff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd710107dd0 0x7fd71019c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:25.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.191+0000 7fd715aff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd710108720 0x7fd7101a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:25.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.191+0000 7fd715aff640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd7101123f0 con 0x7fd710102490 2024-07-26T13:40:25.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.191+0000 7fd715aff640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd710112270 con 0x7fd710107dd0 2024-07-26T13:40:25.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.191+0000 7fd715aff640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd710112570 con 0x7fd710108720 2024-07-26T13:40:25.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.191+0000 7fd70f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd710102490 0x7fd71019c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:25.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.191+0000 7fd70f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd710102490 0x7fd71019c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44884/0 (socket says 172.21.15.100:44884) 2024-07-26T13:40:25.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.191+0000 7fd70f7fe640 1 -- 172.21.15.100:0/1630922907 learned_addr learned my addr 172.21.15.100:0/1630922907 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:25.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.191+0000 7fd70ffff640 1 --2- 172.21.15.100:0/1630922907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd710108720 0x7fd7101a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:25.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.191+0000 7fd707fff640 1 --2- 172.21.15.100:0/1630922907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd710107dd0 0x7fd71019c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:25.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.191+0000 7fd70f7fe640 1 -- 172.21.15.100:0/1630922907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd710108720 msgr2=0x7fd7101a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:25.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.191+0000 7fd70f7fe640 1 --2- 172.21.15.100:0/1630922907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd710108720 0x7fd7101a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:25.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.191+0000 7fd70f7fe640 1 -- 172.21.15.100:0/1630922907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd710107dd0 msgr2=0x7fd71019c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:25.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.191+0000 7fd70f7fe640 1 --2- 172.21.15.100:0/1630922907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd710107dd0 0x7fd71019c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:25.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.191+0000 7fd70f7fe640 1 -- 172.21.15.100:0/1630922907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7101a20e0 con 0x7fd710102490 2024-07-26T13:40:25.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.191+0000 7fd70ffff640 1 --2- 172.21.15.100:0/1630922907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd710108720 0x7fd7101a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:40:25.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.192+0000 7fd707fff640 1 --2- 172.21.15.100:0/1630922907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd710107dd0 0x7fd71019c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:40:25.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.192+0000 7fd70f7fe640 1 --2- 172.21.15.100:0/1630922907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd710102490 0x7fd71019c170 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd70000d770 tx=0x7fd70000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:25.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.192+0000 7fd70d7fa640 1 -- 172.21.15.100:0/1630922907 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd700017070 con 0x7fd710102490 2024-07-26T13:40:25.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.192+0000 7fd715aff640 1 -- 172.21.15.100:0/1630922907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7101a23d0 con 0x7fd710102490 2024-07-26T13:40:25.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.192+0000 7fd70d7fa640 1 -- 172.21.15.100:0/1630922907 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7000054d0 con 0x7fd710102490 2024-07-26T13:40:25.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.192+0000 7fd70d7fa640 1 -- 172.21.15.100:0/1630922907 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd700004c30 con 0x7fd710102490 2024-07-26T13:40:25.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.192+0000 7fd715aff640 1 -- 172.21.15.100:0/1630922907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7101a8b30 con 0x7fd710102490 2024-07-26T13:40:25.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.193+0000 7fd715aff640 1 -- 172.21.15.100:0/1630922907 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7101028a0 con 0x7fd710102490 2024-07-26T13:40:25.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.194+0000 7fd70d7fa640 1 -- 172.21.15.100:0/1630922907 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 151134+0+0 (secure 0 0 0) 0x7fd700004e10 con 0x7fd710102490 2024-07-26T13:40:25.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.195+0000 7fd70d7fa640 1 --2- 172.21.15.100:0/1630922907 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd6fc0b3cb0 0x7fd6fc0b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:25.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.195+0000 7fd707fff640 1 --2- 172.21.15.100:0/1630922907 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd6fc0b3cb0 0x7fd6fc0b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:25.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.195+0000 7fd70d7fa640 1 -- 172.21.15.100:0/1630922907 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fd7000e3ab0 con 0x7fd710102490 2024-07-26T13:40:25.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.195+0000 7fd707fff640 1 --2- 172.21.15.100:0/1630922907 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd6fc0b3cb0 0x7fd6fc0b61c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd6f80096c0 tx=0x7fd6f8031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:25.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.199+0000 7fd70d7fa640 1 -- 172.21.15.100:0/1630922907 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fd7000aaea0 con 0x7fd710102490 2024-07-26T13:40:25.327 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:40:25.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.379+0000 7fd715aff640 1 -- 172.21.15.100:0/1630922907 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd71006b0e0 con 0x7fd6fc0b3cb0 2024-07-26T13:40:25.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.384+0000 7fd70d7fa640 1 -- 172.21.15.100:0/1630922907 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259183 (secure 0 0 0) 0x7fd71006b0e0 con 0x7fd6fc0b3cb0 2024-07-26T13:40:25.388 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:40:25.389 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-07-26T13:40:25.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.387+0000 7fd715aff640 1 -- 172.21.15.100:0/1630922907 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd6fc0b3cb0 msgr2=0x7fd6fc0b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:25.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.387+0000 7fd715aff640 1 --2- 172.21.15.100:0/1630922907 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd6fc0b3cb0 0x7fd6fc0b61c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd6f80096c0 tx=0x7fd6f8031400 comp rx=0 tx=0).stop 2024-07-26T13:40:25.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.387+0000 7fd715aff640 1 -- 172.21.15.100:0/1630922907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd710102490 msgr2=0x7fd71019c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:25.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.387+0000 7fd715aff640 1 --2- 172.21.15.100:0/1630922907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd710102490 0x7fd71019c170 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd70000d770 tx=0x7fd70000dc30 comp rx=0 tx=0).stop 2024-07-26T13:40:25.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.387+0000 7fd715aff640 1 -- 172.21.15.100:0/1630922907 shutdown_connections 2024-07-26T13:40:25.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.387+0000 7fd715aff640 1 --2- 172.21.15.100:0/1630922907 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd6fc0b3cb0 0x7fd6fc0b61c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:25.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.387+0000 7fd715aff640 1 --2- 172.21.15.100:0/1630922907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd710108720 0x7fd7101a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:25.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.387+0000 7fd715aff640 1 --2- 172.21.15.100:0/1630922907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd710107dd0 0x7fd71019c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:25.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.387+0000 7fd715aff640 1 --2- 172.21.15.100:0/1630922907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd710102490 0x7fd71019c170 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:25.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.387+0000 7fd715aff640 1 -- 172.21.15.100:0/1630922907 >> 172.21.15.100:0/1630922907 conn(0x7fd710079160 msgr2=0x7fd71006a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:25.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.388+0000 7fd715aff640 1 -- 172.21.15.100:0/1630922907 shutdown_connections 2024-07-26T13:40:25.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:25.388+0000 7fd715aff640 1 -- 172.21.15.100:0/1630922907 wait complete. 2024-07-26T13:40:25.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:25 smithi138 ceph-mon[36954]: pgmap v373: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 174 KiB/s wr, 53 op/s 2024-07-26T13:40:25.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:25 smithi167 ceph-mon[37140]: pgmap v373: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 174 KiB/s wr, 53 op/s 2024-07-26T13:40:26.220 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":373,"stamp":"2024-07-26T13:40:24.413193+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1450533,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1564,"num_read_kb":1020,"num_write":528,"num_write_kb":4369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":2952484,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":889,"ondisk_log_size":889,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376768,"kb_used_data":10972,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124565056,"statfs":{"total":1151940427776,"available":1151554617344,"internally_reserved":0,"allocated":11235328,"data_stored":8484995,"data_compressed":233951,"data_compressed_allocated":307200,"data_compressed_original":3121152,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":792365,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":416,"num_read_kb":346,"num_write":225,"num_write_kb":2094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003217"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387528+0000","last_change":"2024-07-26T13:36:59.462289+0000","last_active":"2024-07-26T13:38:46.387528+0000","last_peered":"2024-07-26T13:38:46.387528+0000","last_clean":"2024-07-26T13:38:46.387528+0000","last_became_active":"2024-07-26T13:36:59.460311+0000","last_became_peered":"2024-07-26T13:36:59.460311+0000","last_unstale":"2024-07-26T13:38:46.387528+0000","last_undegraded":"2024-07-26T13:38:46.387528+0000","last_fullsized":"2024-07-26T13:38:46.387528+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:04:59.123716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.384825+0000","last_change":"2024-07-26T13:36:59.463218+0000","last_active":"2024-07-26T13:38:46.384825+0000","last_peered":"2024-07-26T13:38:46.384825+0000","last_clean":"2024-07-26T13:38:46.384825+0000","last_became_active":"2024-07-26T13:36:59.462889+0000","last_became_peered":"2024-07-26T13:36:59.462889+0000","last_unstale":"2024-07-26T13:38:46.384825+0000","last_undegraded":"2024-07-26T13:38:46.384825+0000","last_fullsized":"2024-07-26T13:38:46.384825+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:30:46.788669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383503+0000","last_change":"2024-07-26T13:36:59.456113+0000","last_active":"2024-07-26T13:38:46.383503+0000","last_peered":"2024-07-26T13:38:46.383503+0000","last_clean":"2024-07-26T13:38:46.383503+0000","last_became_active":"2024-07-26T13:36:59.455622+0000","last_became_peered":"2024-07-26T13:36:59.455622+0000","last_unstale":"2024-07-26T13:38:46.383503+0000","last_undegraded":"2024-07-26T13:38:46.383503+0000","last_fullsized":"2024-07-26T13:38:46.383503+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:36:26.474799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.368792+0000","last_change":"2024-07-26T13:37:17.555947+0000","last_active":"2024-07-26T13:38:46.368792+0000","last_peered":"2024-07-26T13:38:46.368792+0000","last_clean":"2024-07-26T13:38:46.368792+0000","last_became_active":"2024-07-26T13:37:17.555607+0000","last_became_peered":"2024-07-26T13:37:17.555607+0000","last_unstale":"2024-07-26T13:38:46.368792+0000","last_undegraded":"2024-07-26T13:38:46.368792+0000","last_fullsized":"2024-07-26T13:38:46.368792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:19:59.603225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.372136+0000","last_change":"2024-07-26T13:36:59.474896+0000","last_active":"2024-07-26T13:38:46.372136+0000","last_peered":"2024-07-26T13:38:46.372136+0000","last_clean":"2024-07-26T13:38:46.372136+0000","last_became_active":"2024-07-26T13:36:59.474516+0000","last_became_peered":"2024-07-26T13:36:59.474516+0000","last_unstale":"2024-07-26T13:38:46.372136+0000","last_undegraded":"2024-07-26T13:38:46.372136+0000","last_fullsized":"2024-07-26T13:38:46.372136+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:38:18.994280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.358362+0000","last_change":"2024-07-26T13:37:17.556191+0000","last_active":"2024-07-26T13:38:46.358362+0000","last_peered":"2024-07-26T13:38:46.358362+0000","last_clean":"2024-07-26T13:38:46.358362+0000","last_became_active":"2024-07-26T13:37:17.555715+0000","last_became_peered":"2024-07-26T13:37:17.555715+0000","last_unstale":"2024-07-26T13:38:46.358362+0000","last_undegraded":"2024-07-26T13:38:46.358362+0000","last_fullsized":"2024-07-26T13:38:46.358362+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:03:18.352049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383784+0000","last_change":"2024-07-26T13:36:59.469515+0000","last_active":"2024-07-26T13:38:46.383784+0000","last_peered":"2024-07-26T13:38:46.383784+0000","last_clean":"2024-07-26T13:38:46.383784+0000","last_became_active":"2024-07-26T13:36:59.469106+0000","last_became_peered":"2024-07-26T13:36:59.469106+0000","last_unstale":"2024-07-26T13:38:46.383784+0000","last_undegraded":"2024-07-26T13:38:46.383784+0000","last_fullsized":"2024-07-26T13:38:46.383784+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:10:05.455329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.373036+0000","last_change":"2024-07-26T13:37:17.549365+0000","last_active":"2024-07-26T13:38:46.373036+0000","last_peered":"2024-07-26T13:38:46.373036+0000","last_clean":"2024-07-26T13:38:46.373036+0000","last_became_active":"2024-07-26T13:37:17.549036+0000","last_became_peered":"2024-07-26T13:37:17.549036+0000","last_unstale":"2024-07-26T13:38:46.373036+0000","last_undegraded":"2024-07-26T13:38:46.373036+0000","last_fullsized":"2024-07-26T13:38:46.373036+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:27:43.930981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387690+0000","last_change":"2024-07-26T13:36:59.459306+0000","last_active":"2024-07-26T13:38:46.387690+0000","last_peered":"2024-07-26T13:38:46.387690+0000","last_clean":"2024-07-26T13:38:46.387690+0000","last_became_active":"2024-07-26T13:36:59.458714+0000","last_became_peered":"2024-07-26T13:36:59.458714+0000","last_unstale":"2024-07-26T13:38:46.387690+0000","last_undegraded":"2024-07-26T13:38:46.387690+0000","last_fullsized":"2024-07-26T13:38:46.387690+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:34:18.512454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.376748+0000","last_change":"2024-07-26T13:36:59.466556+0000","last_active":"2024-07-26T13:38:46.376748+0000","last_peered":"2024-07-26T13:38:46.376748+0000","last_clean":"2024-07-26T13:38:46.376748+0000","last_became_active":"2024-07-26T13:36:59.466147+0000","last_became_peered":"2024-07-26T13:36:59.466147+0000","last_unstale":"2024-07-26T13:38:46.376748+0000","last_undegraded":"2024-07-26T13:38:46.376748+0000","last_fullsized":"2024-07-26T13:38:46.376748+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:43:11.359314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387571+0000","last_change":"2024-07-26T13:36:59.473685+0000","last_active":"2024-07-26T13:38:46.387571+0000","last_peered":"2024-07-26T13:38:46.387571+0000","last_clean":"2024-07-26T13:38:46.387571+0000","last_became_active":"2024-07-26T13:36:59.473314+0000","last_became_peered":"2024-07-26T13:36:59.473314+0000","last_unstale":"2024-07-26T13:38:46.387571+0000","last_undegraded":"2024-07-26T13:38:46.387571+0000","last_fullsized":"2024-07-26T13:38:46.387571+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:41:51.612348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.368487+0000","last_change":"2024-07-26T13:36:59.471586+0000","last_active":"2024-07-26T13:38:46.368487+0000","last_peered":"2024-07-26T13:38:46.368487+0000","last_clean":"2024-07-26T13:38:46.368487+0000","last_became_active":"2024-07-26T13:36:59.470995+0000","last_became_peered":"2024-07-26T13:36:59.470995+0000","last_unstale":"2024-07-26T13:38:46.368487+0000","last_undegraded":"2024-07-26T13:38:46.368487+0000","last_fullsized":"2024-07-26T13:38:46.368487+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:54:07.957937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.358125+0000","last_change":"2024-07-26T13:36:59.460018+0000","last_active":"2024-07-26T13:38:46.358125+0000","last_peered":"2024-07-26T13:38:46.358125+0000","last_clean":"2024-07-26T13:38:46.358125+0000","last_became_active":"2024-07-26T13:36:59.459671+0000","last_became_peered":"2024-07-26T13:36:59.459671+0000","last_unstale":"2024-07-26T13:38:46.358125+0000","last_undegraded":"2024-07-26T13:38:46.358125+0000","last_fullsized":"2024-07-26T13:38:46.358125+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:19:33.923360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.374512+0000","last_change":"2024-07-26T13:36:59.470271+0000","last_active":"2024-07-26T13:38:46.374512+0000","last_peered":"2024-07-26T13:38:46.374512+0000","last_clean":"2024-07-26T13:38:46.374512+0000","last_became_active":"2024-07-26T13:36:59.469941+0000","last_became_peered":"2024-07-26T13:36:59.469941+0000","last_unstale":"2024-07-26T13:38:46.374512+0000","last_undegraded":"2024-07-26T13:38:46.374512+0000","last_fullsized":"2024-07-26T13:38:46.374512+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:38:39.329366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383168+0000","last_change":"2024-07-26T13:36:59.460133+0000","last_active":"2024-07-26T13:38:46.383168+0000","last_peered":"2024-07-26T13:38:46.383168+0000","last_clean":"2024-07-26T13:38:46.383168+0000","last_became_active":"2024-07-26T13:36:59.459794+0000","last_became_peered":"2024-07-26T13:36:59.459794+0000","last_unstale":"2024-07-26T13:38:46.383168+0000","last_undegraded":"2024-07-26T13:38:46.383168+0000","last_fullsized":"2024-07-26T13:38:46.383168+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:15:36.024347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.386026+0000","last_change":"2024-07-26T13:36:59.462390+0000","last_active":"2024-07-26T13:38:46.386026+0000","last_peered":"2024-07-26T13:38:46.386026+0000","last_clean":"2024-07-26T13:38:46.386026+0000","last_became_active":"2024-07-26T13:36:59.461968+0000","last_became_peered":"2024-07-26T13:36:59.461968+0000","last_unstale":"2024-07-26T13:38:46.386026+0000","last_undegraded":"2024-07-26T13:38:46.386026+0000","last_fullsized":"2024-07-26T13:38:46.386026+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:49:47.142472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387645+0000","last_change":"2024-07-26T13:36:59.460206+0000","last_active":"2024-07-26T13:38:46.387645+0000","last_peered":"2024-07-26T13:38:46.387645+0000","last_clean":"2024-07-26T13:38:46.387645+0000","last_became_active":"2024-07-26T13:36:59.459811+0000","last_became_peered":"2024-07-26T13:36:59.459811+0000","last_unstale":"2024-07-26T13:38:46.387645+0000","last_undegraded":"2024-07-26T13:38:46.387645+0000","last_fullsized":"2024-07-26T13:38:46.387645+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:46:57.112314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387993+0000","last_change":"2024-07-26T13:36:59.460197+0000","last_active":"2024-07-26T13:38:46.387993+0000","last_peered":"2024-07-26T13:38:46.387993+0000","last_clean":"2024-07-26T13:38:46.387993+0000","last_became_active":"2024-07-26T13:36:59.459686+0000","last_became_peered":"2024-07-26T13:36:59.459686+0000","last_unstale":"2024-07-26T13:38:46.387993+0000","last_undegraded":"2024-07-26T13:38:46.387993+0000","last_fullsized":"2024-07-26T13:38:46.387993+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T13:39:22.407376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.373506+0000","last_change":"2024-07-26T13:36:59.462545+0000","last_active":"2024-07-26T13:38:46.373506+0000","last_peered":"2024-07-26T13:38:46.373506+0000","last_clean":"2024-07-26T13:38:46.373506+0000","last_became_active":"2024-07-26T13:36:59.462115+0000","last_became_peered":"2024-07-26T13:36:59.462115+0000","last_unstale":"2024-07-26T13:38:46.373506+0000","last_undegraded":"2024-07-26T13:38:46.373506+0000","last_fullsized":"2024-07-26T13:38:46.373506+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T21:44:13.715076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.384982+0000","last_change":"2024-07-26T13:36:59.465317+0000","last_active":"2024-07-26T13:38:46.384982+0000","last_peered":"2024-07-26T13:38:46.384982+0000","last_clean":"2024-07-26T13:38:46.384982+0000","last_became_active":"2024-07-26T13:36:59.464943+0000","last_became_peered":"2024-07-26T13:36:59.464943+0000","last_unstale":"2024-07-26T13:38:46.384982+0000","last_undegraded":"2024-07-26T13:38:46.384982+0000","last_fullsized":"2024-07-26T13:38:46.384982+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T13:53:54.160584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.370482+0000","last_change":"2024-07-26T13:36:59.462884+0000","last_active":"2024-07-26T13:38:46.370482+0000","last_peered":"2024-07-26T13:38:46.370482+0000","last_clean":"2024-07-26T13:38:46.370482+0000","last_became_active":"2024-07-26T13:36:59.462564+0000","last_became_peered":"2024-07-26T13:36:59.462564+0000","last_unstale":"2024-07-26T13:38:46.370482+0000","last_undegraded":"2024-07-26T13:38:46.370482+0000","last_fullsized":"2024-07-26T13:38:46.370482+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:50:19.549148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387141+0000","last_change":"2024-07-26T13:36:59.468716+0000","last_active":"2024-07-26T13:38:46.387141+0000","last_peered":"2024-07-26T13:38:46.387141+0000","last_clean":"2024-07-26T13:38:46.387141+0000","last_became_active":"2024-07-26T13:36:59.468048+0000","last_became_peered":"2024-07-26T13:36:59.468048+0000","last_unstale":"2024-07-26T13:38:46.387141+0000","last_undegraded":"2024-07-26T13:38:46.387141+0000","last_fullsized":"2024-07-26T13:38:46.387141+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:51:29.510047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383235+0000","last_change":"2024-07-26T13:36:59.466565+0000","last_active":"2024-07-26T13:38:46.383235+0000","last_peered":"2024-07-26T13:38:46.383235+0000","last_clean":"2024-07-26T13:38:46.383235+0000","last_became_active":"2024-07-26T13:36:59.466235+0000","last_became_peered":"2024-07-26T13:36:59.466235+0000","last_unstale":"2024-07-26T13:38:46.383235+0000","last_undegraded":"2024-07-26T13:38:46.383235+0000","last_fullsized":"2024-07-26T13:38:46.383235+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:28:44.326821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.371918+0000","last_change":"2024-07-26T13:36:59.458753+0000","last_active":"2024-07-26T13:38:46.371918+0000","last_peered":"2024-07-26T13:38:46.371918+0000","last_clean":"2024-07-26T13:38:46.371918+0000","last_became_active":"2024-07-26T13:36:59.458333+0000","last_became_peered":"2024-07-26T13:36:59.458333+0000","last_unstale":"2024-07-26T13:38:46.371918+0000","last_undegraded":"2024-07-26T13:38:46.371918+0000","last_fullsized":"2024-07-26T13:38:46.371918+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:33:54.806212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.369463+0000","last_change":"2024-07-26T13:36:59.468657+0000","last_active":"2024-07-26T13:38:46.369463+0000","last_peered":"2024-07-26T13:38:46.369463+0000","last_clean":"2024-07-26T13:38:46.369463+0000","last_became_active":"2024-07-26T13:36:59.468121+0000","last_became_peered":"2024-07-26T13:36:59.468121+0000","last_unstale":"2024-07-26T13:38:46.369463+0000","last_undegraded":"2024-07-26T13:38:46.369463+0000","last_fullsized":"2024-07-26T13:38:46.369463+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:23:53.959836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.377710+0000","last_change":"2024-07-26T13:36:59.469067+0000","last_active":"2024-07-26T13:38:46.377710+0000","last_peered":"2024-07-26T13:38:46.377710+0000","last_clean":"2024-07-26T13:38:46.377710+0000","last_became_active":"2024-07-26T13:36:59.468706+0000","last_became_peered":"2024-07-26T13:36:59.468706+0000","last_unstale":"2024-07-26T13:38:46.377710+0000","last_undegraded":"2024-07-26T13:38:46.377710+0000","last_fullsized":"2024-07-26T13:38:46.377710+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:21:35.974618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.378393+0000","last_change":"2024-07-26T13:36:59.469807+0000","last_active":"2024-07-26T13:38:46.378393+0000","last_peered":"2024-07-26T13:38:46.378393+0000","last_clean":"2024-07-26T13:38:46.378393+0000","last_became_active":"2024-07-26T13:36:59.469020+0000","last_became_peered":"2024-07-26T13:36:59.469020+0000","last_unstale":"2024-07-26T13:38:46.378393+0000","last_undegraded":"2024-07-26T13:38:46.378393+0000","last_fullsized":"2024-07-26T13:38:46.378393+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:29:57.931598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.373015+0000","last_change":"2024-07-26T13:36:59.465236+0000","last_active":"2024-07-26T13:38:46.373015+0000","last_peered":"2024-07-26T13:38:46.373015+0000","last_clean":"2024-07-26T13:38:46.373015+0000","last_became_active":"2024-07-26T13:36:59.464415+0000","last_became_peered":"2024-07-26T13:36:59.464415+0000","last_unstale":"2024-07-26T13:38:46.373015+0000","last_undegraded":"2024-07-26T13:38:46.373015+0000","last_fullsized":"2024-07-26T13:38:46.373015+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:46:20.373688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.375987+0000","last_change":"2024-07-26T13:36:59.466113+0000","last_active":"2024-07-26T13:38:46.375987+0000","last_peered":"2024-07-26T13:38:46.375987+0000","last_clean":"2024-07-26T13:38:46.375987+0000","last_became_active":"2024-07-26T13:36:59.465743+0000","last_became_peered":"2024-07-26T13:36:59.465743+0000","last_unstale":"2024-07-26T13:38:46.375987+0000","last_undegraded":"2024-07-26T13:38:46.375987+0000","last_fullsized":"2024-07-26T13:38:46.375987+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:14:41.839204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.380451+0000","last_change":"2024-07-26T13:36:59.460672+0000","last_active":"2024-07-26T13:38:46.380451+0000","last_peered":"2024-07-26T13:38:46.380451+0000","last_clean":"2024-07-26T13:38:46.380451+0000","last_became_active":"2024-07-26T13:36:59.460247+0000","last_became_peered":"2024-07-26T13:36:59.460247+0000","last_unstale":"2024-07-26T13:38:46.380451+0000","last_undegraded":"2024-07-26T13:38:46.380451+0000","last_fullsized":"2024-07-26T13:38:46.380451+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:37:51.109933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.378083+0000","last_change":"2024-07-26T13:36:59.464784+0000","last_active":"2024-07-26T13:38:46.378083+0000","last_peered":"2024-07-26T13:38:46.378083+0000","last_clean":"2024-07-26T13:38:46.378083+0000","last_became_active":"2024-07-26T13:36:59.464261+0000","last_became_peered":"2024-07-26T13:36:59.464261+0000","last_unstale":"2024-07-26T13:38:46.378083+0000","last_undegraded":"2024-07-26T13:38:46.378083+0000","last_fullsized":"2024-07-26T13:38:46.378083+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:43:10.134037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.385005+0000","last_change":"2024-07-26T13:36:59.460421+0000","last_active":"2024-07-26T13:38:46.385005+0000","last_peered":"2024-07-26T13:38:46.385005+0000","last_clean":"2024-07-26T13:38:46.385005+0000","last_became_active":"2024-07-26T13:36:59.460010+0000","last_became_peered":"2024-07-26T13:36:59.460010+0000","last_unstale":"2024-07-26T13:38:46.385005+0000","last_undegraded":"2024-07-26T13:38:46.385005+0000","last_fullsized":"2024-07-26T13:38:46.385005+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:16:21.651673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.358312+0000","last_change":"2024-07-26T13:36:59.468826+0000","last_active":"2024-07-26T13:38:46.358312+0000","last_peered":"2024-07-26T13:38:46.358312+0000","last_clean":"2024-07-26T13:38:46.358312+0000","last_became_active":"2024-07-26T13:36:59.468406+0000","last_became_peered":"2024-07-26T13:36:59.468406+0000","last_unstale":"2024-07-26T13:38:46.358312+0000","last_undegraded":"2024-07-26T13:38:46.358312+0000","last_fullsized":"2024-07-26T13:38:46.358312+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:22:10.590545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'7","reported_seq":237,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:40:20.865098+0000","last_change":"2024-07-26T13:37:17.559014+0000","last_active":"2024-07-26T13:40:20.865098+0000","last_peered":"2024-07-26T13:40:20.865098+0000","last_clean":"2024-07-26T13:40:20.865098+0000","last_became_active":"2024-07-26T13:37:17.552325+0000","last_became_peered":"2024-07-26T13:37:17.552325+0000","last_unstale":"2024-07-26T13:40:20.865098+0000","last_undegraded":"2024-07-26T13:40:20.865098+0000","last_fullsized":"2024-07-26T13:40:20.865098+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:26:26.790337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":189,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.379560+0000","last_change":"2024-07-26T13:37:17.562871+0000","last_active":"2024-07-26T13:38:46.379560+0000","last_peered":"2024-07-26T13:38:46.379560+0000","last_clean":"2024-07-26T13:38:46.379560+0000","last_became_active":"2024-07-26T13:37:17.562363+0000","last_became_peered":"2024-07-26T13:37:17.562363+0000","last_unstale":"2024-07-26T13:38:46.379560+0000","last_undegraded":"2024-07-26T13:38:46.379560+0000","last_fullsized":"2024-07-26T13:38:46.379560+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:25:36.172091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"79'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.547762+0000","last_change":"2024-07-26T13:36:56.434778+0000","last_active":"2024-07-26T13:37:17.547762+0000","last_peered":"2024-07-26T13:37:17.547762+0000","last_clean":"2024-07-26T13:37:17.547762+0000","last_became_active":"2024-07-26T13:36:56.434288+0000","last_became_peered":"2024-07-26T13:36:56.434288+0000","last_unstale":"2024-07-26T13:37:17.547762+0000","last_undegraded":"2024-07-26T13:37:17.547762+0000","last_fullsized":"2024-07-26T13:37:17.547762+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:24:22.990358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.565163+0000","last_change":"2024-07-26T13:36:56.424665+0000","last_active":"2024-07-26T13:37:17.565163+0000","last_peered":"2024-07-26T13:37:17.565163+0000","last_clean":"2024-07-26T13:37:17.565163+0000","last_became_active":"2024-07-26T13:36:56.424290+0000","last_became_peered":"2024-07-26T13:36:56.424290+0000","last_unstale":"2024-07-26T13:37:17.565163+0000","last_undegraded":"2024-07-26T13:37:17.565163+0000","last_fullsized":"2024-07-26T13:37:17.565163+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:49:59.520194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383766+0000","last_change":"2024-07-26T13:36:59.454609+0000","last_active":"2024-07-26T13:38:46.383766+0000","last_peered":"2024-07-26T13:38:46.383766+0000","last_clean":"2024-07-26T13:38:46.383766+0000","last_became_active":"2024-07-26T13:36:59.454228+0000","last_became_peered":"2024-07-26T13:36:59.454228+0000","last_unstale":"2024-07-26T13:38:46.383766+0000","last_undegraded":"2024-07-26T13:38:46.383766+0000","last_fullsized":"2024-07-26T13:38:46.383766+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:02:06.244433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.547136+0000","last_change":"2024-07-26T13:36:56.430268+0000","last_active":"2024-07-26T13:37:17.547136+0000","last_peered":"2024-07-26T13:37:17.547136+0000","last_clean":"2024-07-26T13:37:17.547136+0000","last_became_active":"2024-07-26T13:36:56.428583+0000","last_became_peered":"2024-07-26T13:36:56.428583+0000","last_unstale":"2024-07-26T13:37:17.547136+0000","last_undegraded":"2024-07-26T13:37:17.547136+0000","last_fullsized":"2024-07-26T13:37:17.547136+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:07:05.864886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.379724+0000","last_change":"2024-07-26T13:36:59.456188+0000","last_active":"2024-07-26T13:38:46.379724+0000","last_peered":"2024-07-26T13:38:46.379724+0000","last_clean":"2024-07-26T13:38:46.379724+0000","last_became_active":"2024-07-26T13:36:59.455842+0000","last_became_peered":"2024-07-26T13:36:59.455842+0000","last_unstale":"2024-07-26T13:38:46.379724+0000","last_undegraded":"2024-07-26T13:38:46.379724+0000","last_fullsized":"2024-07-26T13:38:46.379724+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:00:19.162239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":217,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:40:21.865881+0000","last_change":"2024-07-26T13:36:56.432818+0000","last_active":"2024-07-26T13:40:21.865881+0000","last_peered":"2024-07-26T13:40:21.865881+0000","last_clean":"2024-07-26T13:40:21.865881+0000","last_became_active":"2024-07-26T13:36:56.432419+0000","last_became_peered":"2024-07-26T13:36:56.432419+0000","last_unstale":"2024-07-26T13:40:21.865881+0000","last_undegraded":"2024-07-26T13:40:21.865881+0000","last_fullsized":"2024-07-26T13:40:21.865881+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:55:51.972470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":186,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.386929+0000","last_change":"2024-07-26T13:36:59.460033+0000","last_active":"2024-07-26T13:38:46.386929+0000","last_peered":"2024-07-26T13:38:46.386929+0000","last_clean":"2024-07-26T13:38:46.386929+0000","last_became_active":"2024-07-26T13:36:59.459562+0000","last_became_peered":"2024-07-26T13:36:59.459562+0000","last_unstale":"2024-07-26T13:38:46.386929+0000","last_undegraded":"2024-07-26T13:38:46.386929+0000","last_fullsized":"2024-07-26T13:38:46.386929+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:58:14.155973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551399+0000","last_change":"2024-07-26T13:36:56.432563+0000","last_active":"2024-07-26T13:37:17.551399+0000","last_peered":"2024-07-26T13:37:17.551399+0000","last_clean":"2024-07-26T13:37:17.551399+0000","last_became_active":"2024-07-26T13:36:56.431134+0000","last_became_peered":"2024-07-26T13:36:56.431134+0000","last_unstale":"2024-07-26T13:37:17.551399+0000","last_undegraded":"2024-07-26T13:37:17.551399+0000","last_fullsized":"2024-07-26T13:37:17.551399+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:42:00.653345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.377839+0000","last_change":"2024-07-26T13:36:59.469782+0000","last_active":"2024-07-26T13:38:46.377839+0000","last_peered":"2024-07-26T13:38:46.377839+0000","last_clean":"2024-07-26T13:38:46.377839+0000","last_became_active":"2024-07-26T13:36:59.469412+0000","last_became_peered":"2024-07-26T13:36:59.469412+0000","last_unstale":"2024-07-26T13:38:46.377839+0000","last_undegraded":"2024-07-26T13:38:46.377839+0000","last_fullsized":"2024-07-26T13:38:46.377839+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:28:07.741486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.558844+0000","last_change":"2024-07-26T13:36:56.488981+0000","last_active":"2024-07-26T13:37:17.558844+0000","last_peered":"2024-07-26T13:37:17.558844+0000","last_clean":"2024-07-26T13:37:17.558844+0000","last_became_active":"2024-07-26T13:36:56.488480+0000","last_became_peered":"2024-07-26T13:36:56.488480+0000","last_unstale":"2024-07-26T13:37:17.558844+0000","last_undegraded":"2024-07-26T13:37:17.558844+0000","last_fullsized":"2024-07-26T13:37:17.558844+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:41:10.041007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387010+0000","last_change":"2024-07-26T13:36:59.473183+0000","last_active":"2024-07-26T13:38:46.387010+0000","last_peered":"2024-07-26T13:38:46.387010+0000","last_clean":"2024-07-26T13:38:46.387010+0000","last_became_active":"2024-07-26T13:36:59.472814+0000","last_became_peered":"2024-07-26T13:36:59.472814+0000","last_unstale":"2024-07-26T13:38:46.387010+0000","last_undegraded":"2024-07-26T13:38:46.387010+0000","last_fullsized":"2024-07-26T13:38:46.387010+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:57:03.396384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'35","reported_seq":446,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:40:22.882789+0000","last_change":"2024-07-26T13:36:56.435034+0000","last_active":"2024-07-26T13:40:22.882789+0000","last_peered":"2024-07-26T13:40:22.882789+0000","last_clean":"2024-07-26T13:40:22.882789+0000","last_became_active":"2024-07-26T13:36:56.434611+0000","last_became_peered":"2024-07-26T13:36:56.434611+0000","last_unstale":"2024-07-26T13:40:22.882789+0000","last_undegraded":"2024-07-26T13:40:22.882789+0000","last_fullsized":"2024-07-26T13:40:22.882789+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:18:27.272077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68265,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":384,"num_read_kb":259,"num_write":36,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.382422+0000","last_change":"2024-07-26T13:36:59.465831+0000","last_active":"2024-07-26T13:38:46.382422+0000","last_peered":"2024-07-26T13:38:46.382422+0000","last_clean":"2024-07-26T13:38:46.382422+0000","last_became_active":"2024-07-26T13:36:59.465404+0000","last_became_peered":"2024-07-26T13:36:59.465404+0000","last_unstale":"2024-07-26T13:38:46.382422+0000","last_undegraded":"2024-07-26T13:38:46.382422+0000","last_fullsized":"2024-07-26T13:38:46.382422+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:42:53.795926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'6","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.552497+0000","last_change":"2024-07-26T13:36:56.435170+0000","last_active":"2024-07-26T13:37:17.552497+0000","last_peered":"2024-07-26T13:37:17.552497+0000","last_clean":"2024-07-26T13:37:17.552497+0000","last_became_active":"2024-07-26T13:36:56.434715+0000","last_became_peered":"2024-07-26T13:36:56.434715+0000","last_unstale":"2024-07-26T13:37:17.552497+0000","last_undegraded":"2024-07-26T13:37:17.552497+0000","last_fullsized":"2024-07-26T13:37:17.552497+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:05:09.712510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.370726+0000","last_change":"2024-07-26T13:36:59.461926+0000","last_active":"2024-07-26T13:38:46.370726+0000","last_peered":"2024-07-26T13:38:46.370726+0000","last_clean":"2024-07-26T13:38:46.370726+0000","last_became_active":"2024-07-26T13:36:59.461512+0000","last_became_peered":"2024-07-26T13:36:59.461512+0000","last_unstale":"2024-07-26T13:38:46.370726+0000","last_undegraded":"2024-07-26T13:38:46.370726+0000","last_fullsized":"2024-07-26T13:38:46.370726+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:34:24.405663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.546907+0000","last_change":"2024-07-26T13:36:56.430062+0000","last_active":"2024-07-26T13:37:17.546907+0000","last_peered":"2024-07-26T13:37:17.546907+0000","last_clean":"2024-07-26T13:37:17.546907+0000","last_became_active":"2024-07-26T13:36:56.429654+0000","last_became_peered":"2024-07-26T13:36:56.429654+0000","last_unstale":"2024-07-26T13:37:17.546907+0000","last_undegraded":"2024-07-26T13:37:17.546907+0000","last_fullsized":"2024-07-26T13:37:17.546907+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:21:09.514359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.380005+0000","last_change":"2024-07-26T13:36:59.474459+0000","last_active":"2024-07-26T13:38:46.380005+0000","last_peered":"2024-07-26T13:38:46.380005+0000","last_clean":"2024-07-26T13:38:46.380005+0000","last_became_active":"2024-07-26T13:36:59.474061+0000","last_became_peered":"2024-07-26T13:36:59.474061+0000","last_unstale":"2024-07-26T13:38:46.380005+0000","last_undegraded":"2024-07-26T13:38:46.380005+0000","last_fullsized":"2024-07-26T13:38:46.380005+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:37:21.691491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'5","reported_seq":223,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:40:21.865501+0000","last_change":"2024-07-26T13:36:56.432915+0000","last_active":"2024-07-26T13:40:21.865501+0000","last_peered":"2024-07-26T13:40:21.865501+0000","last_clean":"2024-07-26T13:40:21.865501+0000","last_became_active":"2024-07-26T13:36:56.432499+0000","last_became_peered":"2024-07-26T13:36:56.432499+0000","last_unstale":"2024-07-26T13:40:21.865501+0000","last_undegraded":"2024-07-26T13:40:21.865501+0000","last_fullsized":"2024-07-26T13:40:21.865501+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:02:14.306950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":192,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383574+0000","last_change":"2024-07-26T13:36:59.468945+0000","last_active":"2024-07-26T13:38:46.383574+0000","last_peered":"2024-07-26T13:38:46.383574+0000","last_clean":"2024-07-26T13:38:46.383574+0000","last_became_active":"2024-07-26T13:36:59.467970+0000","last_became_peered":"2024-07-26T13:36:59.467970+0000","last_unstale":"2024-07-26T13:38:46.383574+0000","last_undegraded":"2024-07-26T13:38:46.383574+0000","last_fullsized":"2024-07-26T13:38:46.383574+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T13:59:19.514945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'15","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551412+0000","last_change":"2024-07-26T13:36:56.487014+0000","last_active":"2024-07-26T13:37:17.551412+0000","last_peered":"2024-07-26T13:37:17.551412+0000","last_clean":"2024-07-26T13:37:17.551412+0000","last_became_active":"2024-07-26T13:36:56.486470+0000","last_became_peered":"2024-07-26T13:36:56.486470+0000","last_unstale":"2024-07-26T13:37:17.551412+0000","last_undegraded":"2024-07-26T13:37:17.551412+0000","last_fullsized":"2024-07-26T13:37:17.551412+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:07:02.205644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.366501+0000","last_change":"2024-07-26T13:36:59.462465+0000","last_active":"2024-07-26T13:38:46.366501+0000","last_peered":"2024-07-26T13:38:46.366501+0000","last_clean":"2024-07-26T13:38:46.366501+0000","last_became_active":"2024-07-26T13:36:59.462024+0000","last_became_peered":"2024-07-26T13:36:59.462024+0000","last_unstale":"2024-07-26T13:38:46.366501+0000","last_undegraded":"2024-07-26T13:38:46.366501+0000","last_fullsized":"2024-07-26T13:38:46.366501+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:34:14.317762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551992+0000","last_change":"2024-07-26T13:36:56.435007+0000","last_active":"2024-07-26T13:37:17.551992+0000","last_peered":"2024-07-26T13:37:17.551992+0000","last_clean":"2024-07-26T13:37:17.551992+0000","last_became_active":"2024-07-26T13:36:56.434444+0000","last_became_peered":"2024-07-26T13:36:56.434444+0000","last_unstale":"2024-07-26T13:37:17.551992+0000","last_undegraded":"2024-07-26T13:37:17.551992+0000","last_fullsized":"2024-07-26T13:37:17.551992+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:27:51.041349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.384490+0000","last_change":"2024-07-26T13:36:59.464164+0000","last_active":"2024-07-26T13:38:46.384490+0000","last_peered":"2024-07-26T13:38:46.384490+0000","last_clean":"2024-07-26T13:38:46.384490+0000","last_became_active":"2024-07-26T13:36:59.463782+0000","last_became_peered":"2024-07-26T13:36:59.463782+0000","last_unstale":"2024-07-26T13:38:46.384490+0000","last_undegraded":"2024-07-26T13:38:46.384490+0000","last_fullsized":"2024-07-26T13:38:46.384490+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:48:06.481731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.565510+0000","last_change":"2024-07-26T13:36:56.429653+0000","last_active":"2024-07-26T13:37:17.565510+0000","last_peered":"2024-07-26T13:37:17.565510+0000","last_clean":"2024-07-26T13:37:17.565510+0000","last_became_active":"2024-07-26T13:36:56.429060+0000","last_became_peered":"2024-07-26T13:36:56.429060+0000","last_unstale":"2024-07-26T13:37:17.565510+0000","last_undegraded":"2024-07-26T13:37:17.565510+0000","last_fullsized":"2024-07-26T13:37:17.565510+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:41:48.261709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.384430+0000","last_change":"2024-07-26T13:36:59.471560+0000","last_active":"2024-07-26T13:38:46.384430+0000","last_peered":"2024-07-26T13:38:46.384430+0000","last_clean":"2024-07-26T13:38:46.384430+0000","last_became_active":"2024-07-26T13:36:59.471151+0000","last_became_peered":"2024-07-26T13:36:59.471151+0000","last_unstale":"2024-07-26T13:38:46.384430+0000","last_undegraded":"2024-07-26T13:38:46.384430+0000","last_fullsized":"2024-07-26T13:38:46.384430+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:49:20.513308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'60","reported_seq":117,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:40:21.692927+0000","last_change":"2024-07-26T13:36:56.430010+0000","last_active":"2024-07-26T13:40:21.692927+0000","last_peered":"2024-07-26T13:40:21.692927+0000","last_clean":"2024-07-26T13:40:21.692927+0000","last_became_active":"2024-07-26T13:36:56.429120+0000","last_became_peered":"2024-07-26T13:36:56.429120+0000","last_unstale":"2024-07-26T13:40:21.692927+0000","last_undegraded":"2024-07-26T13:40:21.692927+0000","last_fullsized":"2024-07-26T13:40:21.692927+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:23:43.321975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393846,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":32,"num_read_kb":31,"num_write":71,"num_write_kb":873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383589+0000","last_change":"2024-07-26T13:36:59.467788+0000","last_active":"2024-07-26T13:38:46.383589+0000","last_peered":"2024-07-26T13:38:46.383589+0000","last_clean":"2024-07-26T13:38:46.383589+0000","last_became_active":"2024-07-26T13:36:59.467399+0000","last_became_peered":"2024-07-26T13:36:59.467399+0000","last_unstale":"2024-07-26T13:38:46.383589+0000","last_undegraded":"2024-07-26T13:38:46.383589+0000","last_fullsized":"2024-07-26T13:38:46.383589+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:48:51.442999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'3","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.553308+0000","last_change":"2024-07-26T13:36:56.588633+0000","last_active":"2024-07-26T13:37:17.553308+0000","last_peered":"2024-07-26T13:37:17.553308+0000","last_clean":"2024-07-26T13:37:17.553308+0000","last_became_active":"2024-07-26T13:36:56.588113+0000","last_became_peered":"2024-07-26T13:36:56.588113+0000","last_unstale":"2024-07-26T13:37:17.553308+0000","last_undegraded":"2024-07-26T13:37:17.553308+0000","last_fullsized":"2024-07-26T13:37:17.553308+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:03:55.075052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387531+0000","last_change":"2024-07-26T13:36:59.472854+0000","last_active":"2024-07-26T13:38:46.387531+0000","last_peered":"2024-07-26T13:38:46.387531+0000","last_clean":"2024-07-26T13:38:46.387531+0000","last_became_active":"2024-07-26T13:36:59.472466+0000","last_became_peered":"2024-07-26T13:36:59.472466+0000","last_unstale":"2024-07-26T13:38:46.387531+0000","last_undegraded":"2024-07-26T13:38:46.387531+0000","last_fullsized":"2024-07-26T13:38:46.387531+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:50:57.454149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":103,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550829+0000","last_change":"2024-07-26T13:35:34.839121+0000","last_active":"2024-07-26T13:37:17.550829+0000","last_peered":"2024-07-26T13:37:17.550829+0000","last_clean":"2024-07-26T13:37:17.550829+0000","last_became_active":"2024-07-26T13:35:34.833465+0000","last_became_peered":"2024-07-26T13:35:34.833465+0000","last_unstale":"2024-07-26T13:37:17.550829+0000","last_undegraded":"2024-07-26T13:37:17.550829+0000","last_fullsized":"2024-07-26T13:37:17.550829+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:32:23.285428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:32:23.285428+0000","last_clean_scrub_stamp":"2024-07-26T13:32:23.285428+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:29:31.650487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'3","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.561459+0000","last_change":"2024-07-26T13:36:56.428648+0000","last_active":"2024-07-26T13:37:17.561459+0000","last_peered":"2024-07-26T13:37:17.561459+0000","last_clean":"2024-07-26T13:37:17.561459+0000","last_became_active":"2024-07-26T13:36:56.427665+0000","last_became_peered":"2024-07-26T13:36:56.427665+0000","last_unstale":"2024-07-26T13:37:17.561459+0000","last_undegraded":"2024-07-26T13:37:17.561459+0000","last_fullsized":"2024-07-26T13:37:17.561459+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:41:39.371687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.376696+0000","last_change":"2024-07-26T13:36:59.460625+0000","last_active":"2024-07-26T13:38:46.376696+0000","last_peered":"2024-07-26T13:38:46.376696+0000","last_clean":"2024-07-26T13:38:46.376696+0000","last_became_active":"2024-07-26T13:36:59.458658+0000","last_became_peered":"2024-07-26T13:36:59.458658+0000","last_unstale":"2024-07-26T13:38:46.376696+0000","last_undegraded":"2024-07-26T13:38:46.376696+0000","last_fullsized":"2024-07-26T13:38:46.376696+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:46:42.812402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'3","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550796+0000","last_change":"2024-07-26T13:36:56.432617+0000","last_active":"2024-07-26T13:37:17.550796+0000","last_peered":"2024-07-26T13:37:17.550796+0000","last_clean":"2024-07-26T13:37:17.550796+0000","last_became_active":"2024-07-26T13:36:56.431519+0000","last_became_peered":"2024-07-26T13:36:56.431519+0000","last_unstale":"2024-07-26T13:37:17.550796+0000","last_undegraded":"2024-07-26T13:37:17.550796+0000","last_fullsized":"2024-07-26T13:37:17.550796+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:32:21.120076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.364664+0000","last_change":"2024-07-26T13:36:59.464084+0000","last_active":"2024-07-26T13:38:46.364664+0000","last_peered":"2024-07-26T13:38:46.364664+0000","last_clean":"2024-07-26T13:38:46.364664+0000","last_became_active":"2024-07-26T13:36:59.463755+0000","last_became_peered":"2024-07-26T13:36:59.463755+0000","last_unstale":"2024-07-26T13:38:46.364664+0000","last_undegraded":"2024-07-26T13:38:46.364664+0000","last_fullsized":"2024-07-26T13:38:46.364664+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:53:20.034188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":218,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:40:19.861654+0000","last_change":"2024-07-26T13:36:56.486071+0000","last_active":"2024-07-26T13:40:19.861654+0000","last_peered":"2024-07-26T13:40:19.861654+0000","last_clean":"2024-07-26T13:40:19.861654+0000","last_became_active":"2024-07-26T13:36:56.485672+0000","last_became_peered":"2024-07-26T13:36:56.485672+0000","last_unstale":"2024-07-26T13:40:19.861654+0000","last_undegraded":"2024-07-26T13:40:19.861654+0000","last_fullsized":"2024-07-26T13:40:19.861654+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T21:29:45.666651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":185,"num_read_kb":185,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.372811+0000","last_change":"2024-07-26T13:37:17.566555+0000","last_active":"2024-07-26T13:38:46.372811+0000","last_peered":"2024-07-26T13:38:46.372811+0000","last_clean":"2024-07-26T13:38:46.372811+0000","last_became_active":"2024-07-26T13:37:17.566040+0000","last_became_peered":"2024-07-26T13:37:17.566040+0000","last_unstale":"2024-07-26T13:38:46.372811+0000","last_undegraded":"2024-07-26T13:38:46.372811+0000","last_fullsized":"2024-07-26T13:38:46.372811+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:48:33.078890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550989+0000","last_change":"2024-07-26T13:36:56.431811+0000","last_active":"2024-07-26T13:37:17.550989+0000","last_peered":"2024-07-26T13:37:17.550989+0000","last_clean":"2024-07-26T13:37:17.550989+0000","last_became_active":"2024-07-26T13:36:56.431223+0000","last_became_peered":"2024-07-26T13:36:56.431223+0000","last_unstale":"2024-07-26T13:37:17.550989+0000","last_undegraded":"2024-07-26T13:37:17.550989+0000","last_fullsized":"2024-07-26T13:37:17.550989+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:08:00.334515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.363881+0000","last_change":"2024-07-26T13:36:59.463972+0000","last_active":"2024-07-26T13:38:46.363881+0000","last_peered":"2024-07-26T13:38:46.363881+0000","last_clean":"2024-07-26T13:38:46.363881+0000","last_became_active":"2024-07-26T13:36:59.463496+0000","last_became_peered":"2024-07-26T13:36:59.463496+0000","last_unstale":"2024-07-26T13:38:46.363881+0000","last_undegraded":"2024-07-26T13:38:46.363881+0000","last_fullsized":"2024-07-26T13:38:46.363881+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:02:06.215372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.545780+0000","last_change":"2024-07-26T13:36:56.486681+0000","last_active":"2024-07-26T13:37:17.545780+0000","last_peered":"2024-07-26T13:37:17.545780+0000","last_clean":"2024-07-26T13:37:17.545780+0000","last_became_active":"2024-07-26T13:36:56.486328+0000","last_became_peered":"2024-07-26T13:36:56.486328+0000","last_unstale":"2024-07-26T13:37:17.545780+0000","last_undegraded":"2024-07-26T13:37:17.545780+0000","last_fullsized":"2024-07-26T13:37:17.545780+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:10:36.557707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.355361+0000","last_change":"2024-07-26T13:36:59.451415+0000","last_active":"2024-07-26T13:38:46.355361+0000","last_peered":"2024-07-26T13:38:46.355361+0000","last_clean":"2024-07-26T13:38:46.355361+0000","last_became_active":"2024-07-26T13:36:59.450992+0000","last_became_peered":"2024-07-26T13:36:59.450992+0000","last_unstale":"2024-07-26T13:38:46.355361+0000","last_undegraded":"2024-07-26T13:38:46.355361+0000","last_fullsized":"2024-07-26T13:38:46.355361+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:29:11.303916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.557777+0000","last_change":"2024-07-26T13:37:17.555643+0000","last_active":"2024-07-26T13:37:17.557777+0000","last_peered":"2024-07-26T13:37:17.557777+0000","last_clean":"2024-07-26T13:37:17.557777+0000","last_became_active":"2024-07-26T13:37:17.549756+0000","last_became_peered":"2024-07-26T13:37:17.549756+0000","last_unstale":"2024-07-26T13:37:17.557777+0000","last_undegraded":"2024-07-26T13:37:17.557777+0000","last_fullsized":"2024-07-26T13:37:17.557777+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:20:16.911286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.368534+0000","last_change":"2024-07-26T13:36:59.465538+0000","last_active":"2024-07-26T13:38:46.368534+0000","last_peered":"2024-07-26T13:38:46.368534+0000","last_clean":"2024-07-26T13:38:46.368534+0000","last_became_active":"2024-07-26T13:36:59.464886+0000","last_became_peered":"2024-07-26T13:36:59.464886+0000","last_unstale":"2024-07-26T13:38:46.368534+0000","last_undegraded":"2024-07-26T13:38:46.368534+0000","last_fullsized":"2024-07-26T13:38:46.368534+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:19:12.273116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'97","reported_seq":213,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:40:22.420742+0000","last_change":"2024-07-26T13:36:56.428355+0000","last_active":"2024-07-26T13:40:22.420742+0000","last_peered":"2024-07-26T13:40:22.420742+0000","last_clean":"2024-07-26T13:40:22.420742+0000","last_became_active":"2024-07-26T13:36:56.427213+0000","last_became_peered":"2024-07-26T13:36:56.427213+0000","last_unstale":"2024-07-26T13:40:22.420742+0000","last_undegraded":"2024-07-26T13:40:22.420742+0000","last_fullsized":"2024-07-26T13:40:22.420742+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:48:09.730067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":179,"num_write_kb":1222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.378498+0000","last_change":"2024-07-26T13:36:59.459378+0000","last_active":"2024-07-26T13:38:46.378498+0000","last_peered":"2024-07-26T13:38:46.378498+0000","last_clean":"2024-07-26T13:38:46.378498+0000","last_became_active":"2024-07-26T13:36:59.458799+0000","last_became_peered":"2024-07-26T13:36:59.458799+0000","last_unstale":"2024-07-26T13:38:46.378498+0000","last_undegraded":"2024-07-26T13:38:46.378498+0000","last_fullsized":"2024-07-26T13:38:46.378498+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:45:19.587800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.561441+0000","last_change":"2024-07-26T13:36:56.588819+0000","last_active":"2024-07-26T13:37:17.561441+0000","last_peered":"2024-07-26T13:37:17.561441+0000","last_clean":"2024-07-26T13:37:17.561441+0000","last_became_active":"2024-07-26T13:36:56.588430+0000","last_became_peered":"2024-07-26T13:36:56.588430+0000","last_unstale":"2024-07-26T13:37:17.561441+0000","last_undegraded":"2024-07-26T13:37:17.561441+0000","last_fullsized":"2024-07-26T13:37:17.561441+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:07:21.387618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.380460+0000","last_change":"2024-07-26T13:37:17.562040+0000","last_active":"2024-07-26T13:38:46.380460+0000","last_peered":"2024-07-26T13:38:46.380460+0000","last_clean":"2024-07-26T13:38:46.380460+0000","last_became_active":"2024-07-26T13:37:17.561542+0000","last_became_peered":"2024-07-26T13:37:17.561542+0000","last_unstale":"2024-07-26T13:38:46.380460+0000","last_undegraded":"2024-07-26T13:38:46.380460+0000","last_fullsized":"2024-07-26T13:38:46.380460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:21:08.161954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.544169+0000","last_change":"2024-07-26T13:36:56.489106+0000","last_active":"2024-07-26T13:37:17.544169+0000","last_peered":"2024-07-26T13:37:17.544169+0000","last_clean":"2024-07-26T13:37:17.544169+0000","last_became_active":"2024-07-26T13:36:56.488706+0000","last_became_peered":"2024-07-26T13:36:56.488706+0000","last_unstale":"2024-07-26T13:37:17.544169+0000","last_undegraded":"2024-07-26T13:37:17.544169+0000","last_fullsized":"2024-07-26T13:37:17.544169+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:00:59.204434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.343705+0000","last_change":"2024-07-26T13:36:59.470412+0000","last_active":"2024-07-26T13:38:46.343705+0000","last_peered":"2024-07-26T13:38:46.343705+0000","last_clean":"2024-07-26T13:38:46.343705+0000","last_became_active":"2024-07-26T13:36:59.470010+0000","last_became_peered":"2024-07-26T13:36:59.470010+0000","last_unstale":"2024-07-26T13:38:46.343705+0000","last_undegraded":"2024-07-26T13:38:46.343705+0000","last_fullsized":"2024-07-26T13:38:46.343705+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:50:57.987600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.543528+0000","last_change":"2024-07-26T13:36:56.428671+0000","last_active":"2024-07-26T13:37:17.543528+0000","last_peered":"2024-07-26T13:37:17.543528+0000","last_clean":"2024-07-26T13:37:17.543528+0000","last_became_active":"2024-07-26T13:36:56.428301+0000","last_became_peered":"2024-07-26T13:36:56.428301+0000","last_unstale":"2024-07-26T13:37:17.543528+0000","last_undegraded":"2024-07-26T13:37:17.543528+0000","last_fullsized":"2024-07-26T13:37:17.543528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:59:19.593664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.385021+0000","last_change":"2024-07-26T13:36:59.458627+0000","last_active":"2024-07-26T13:38:46.385021+0000","last_peered":"2024-07-26T13:38:46.385021+0000","last_clean":"2024-07-26T13:38:46.385021+0000","last_became_active":"2024-07-26T13:36:59.458199+0000","last_became_peered":"2024-07-26T13:36:59.458199+0000","last_unstale":"2024-07-26T13:38:46.385021+0000","last_undegraded":"2024-07-26T13:38:46.385021+0000","last_fullsized":"2024-07-26T13:38:46.385021+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:54:41.833223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551540+0000","last_change":"2024-07-26T13:36:56.431548+0000","last_active":"2024-07-26T13:37:17.551540+0000","last_peered":"2024-07-26T13:37:17.551540+0000","last_clean":"2024-07-26T13:37:17.551540+0000","last_became_active":"2024-07-26T13:36:56.430989+0000","last_became_peered":"2024-07-26T13:36:56.430989+0000","last_unstale":"2024-07-26T13:37:17.551540+0000","last_undegraded":"2024-07-26T13:37:17.551540+0000","last_fullsized":"2024-07-26T13:37:17.551540+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:46:51.655324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.375762+0000","last_change":"2024-07-26T13:36:59.458290+0000","last_active":"2024-07-26T13:38:46.375762+0000","last_peered":"2024-07-26T13:38:46.375762+0000","last_clean":"2024-07-26T13:38:46.375762+0000","last_became_active":"2024-07-26T13:36:59.457942+0000","last_became_peered":"2024-07-26T13:36:59.457942+0000","last_unstale":"2024-07-26T13:38:46.375762+0000","last_undegraded":"2024-07-26T13:38:46.375762+0000","last_fullsized":"2024-07-26T13:38:46.375762+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:51:58.797441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":46,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.568059+0000","last_change":"2024-07-26T13:37:17.567624+0000","last_active":"2024-07-26T13:37:17.568059+0000","last_peered":"2024-07-26T13:37:17.568059+0000","last_clean":"2024-07-26T13:37:17.568059+0000","last_became_active":"2024-07-26T13:37:17.562455+0000","last_became_peered":"2024-07-26T13:37:17.562455+0000","last_unstale":"2024-07-26T13:37:17.568059+0000","last_undegraded":"2024-07-26T13:37:17.568059+0000","last_fullsized":"2024-07-26T13:37:17.568059+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:50:36.418750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387835+0000","last_change":"2024-07-26T13:36:59.474034+0000","last_active":"2024-07-26T13:38:46.387835+0000","last_peered":"2024-07-26T13:38:46.387835+0000","last_clean":"2024-07-26T13:38:46.387835+0000","last_became_active":"2024-07-26T13:36:59.472358+0000","last_became_peered":"2024-07-26T13:36:59.472358+0000","last_unstale":"2024-07-26T13:38:46.387835+0000","last_undegraded":"2024-07-26T13:38:46.387835+0000","last_fullsized":"2024-07-26T13:38:46.387835+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:36:44.565413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.368595+0000","last_change":"2024-07-26T13:36:59.465546+0000","last_active":"2024-07-26T13:38:46.368595+0000","last_peered":"2024-07-26T13:38:46.368595+0000","last_clean":"2024-07-26T13:38:46.368595+0000","last_became_active":"2024-07-26T13:36:59.465137+0000","last_became_peered":"2024-07-26T13:36:59.465137+0000","last_unstale":"2024-07-26T13:38:46.368595+0000","last_undegraded":"2024-07-26T13:38:46.368595+0000","last_fullsized":"2024-07-26T13:38:46.368595+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:35:13.591354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.562664+0000","last_change":"2024-07-26T13:37:17.562012+0000","last_active":"2024-07-26T13:37:17.562664+0000","last_peered":"2024-07-26T13:37:17.562664+0000","last_clean":"2024-07-26T13:37:17.562664+0000","last_became_active":"2024-07-26T13:37:17.550218+0000","last_became_peered":"2024-07-26T13:37:17.550218+0000","last_unstale":"2024-07-26T13:37:17.562664+0000","last_undegraded":"2024-07-26T13:37:17.562664+0000","last_fullsized":"2024-07-26T13:37:17.562664+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:46:16.559533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.375095+0000","last_change":"2024-07-26T13:36:59.456896+0000","last_active":"2024-07-26T13:38:46.375095+0000","last_peered":"2024-07-26T13:38:46.375095+0000","last_clean":"2024-07-26T13:38:46.375095+0000","last_became_active":"2024-07-26T13:36:59.456530+0000","last_became_peered":"2024-07-26T13:36:59.456530+0000","last_unstale":"2024-07-26T13:38:46.375095+0000","last_undegraded":"2024-07-26T13:38:46.375095+0000","last_fullsized":"2024-07-26T13:38:46.375095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:59:56.512501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'4","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.553596+0000","last_change":"2024-07-26T13:36:56.433119+0000","last_active":"2024-07-26T13:37:17.553596+0000","last_peered":"2024-07-26T13:37:17.553596+0000","last_clean":"2024-07-26T13:37:17.553596+0000","last_became_active":"2024-07-26T13:36:56.432765+0000","last_became_peered":"2024-07-26T13:36:56.432765+0000","last_unstale":"2024-07-26T13:37:17.553596+0000","last_undegraded":"2024-07-26T13:37:17.553596+0000","last_fullsized":"2024-07-26T13:37:17.553596+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:31:29.197106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.382051+0000","last_change":"2024-07-26T13:36:59.462125+0000","last_active":"2024-07-26T13:38:46.382051+0000","last_peered":"2024-07-26T13:38:46.382051+0000","last_clean":"2024-07-26T13:38:46.382051+0000","last_became_active":"2024-07-26T13:36:59.461796+0000","last_became_peered":"2024-07-26T13:36:59.461796+0000","last_unstale":"2024-07-26T13:38:46.382051+0000","last_undegraded":"2024-07-26T13:38:46.382051+0000","last_fullsized":"2024-07-26T13:38:46.382051+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:44:10.329046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.543646+0000","last_change":"2024-07-26T13:36:56.487251+0000","last_active":"2024-07-26T13:37:17.543646+0000","last_peered":"2024-07-26T13:37:17.543646+0000","last_clean":"2024-07-26T13:37:17.543646+0000","last_became_active":"2024-07-26T13:36:56.486918+0000","last_became_peered":"2024-07-26T13:36:56.486918+0000","last_unstale":"2024-07-26T13:37:17.543646+0000","last_undegraded":"2024-07-26T13:37:17.543646+0000","last_fullsized":"2024-07-26T13:37:17.543646+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:41:31.248895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.380839+0000","last_change":"2024-07-26T13:36:59.470810+0000","last_active":"2024-07-26T13:38:46.380839+0000","last_peered":"2024-07-26T13:38:46.380839+0000","last_clean":"2024-07-26T13:38:46.380839+0000","last_became_active":"2024-07-26T13:36:59.467371+0000","last_became_peered":"2024-07-26T13:36:59.467371+0000","last_unstale":"2024-07-26T13:38:46.380839+0000","last_undegraded":"2024-07-26T13:38:46.380839+0000","last_fullsized":"2024-07-26T13:38:46.380839+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:23:48.572933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'9","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:39:03.748968+0000","last_change":"2024-07-26T13:36:56.433598+0000","last_active":"2024-07-26T13:39:03.748968+0000","last_peered":"2024-07-26T13:39:03.748968+0000","last_clean":"2024-07-26T13:39:03.748968+0000","last_became_active":"2024-07-26T13:36:56.433325+0000","last_became_peered":"2024-07-26T13:36:56.433325+0000","last_unstale":"2024-07-26T13:39:03.748968+0000","last_undegraded":"2024-07-26T13:39:03.748968+0000","last_fullsized":"2024-07-26T13:39:03.748968+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:17:33.651765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":859931,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1483,"num_read_kb":951,"num_write":403,"num_write_kb":2217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":644,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":598016,"data_stored":1792801,"data_compressed":74760,"data_compressed_allocated":98304,"data_compressed_original":1523712,"omap_allocated":0,"internal_metadata":0},"log_size":295,"ondisk_log_size":295,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247480,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678073,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":443451,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874302,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1093632,"data_stored":1497483,"data_compressed":59728,"data_compressed_allocated":77824,"data_compressed_original":974848,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037827,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":921600,"data_stored":836881,"data_compressed":25108,"data_compressed_allocated":32768,"data_compressed_original":409600,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234055,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":836779,"data_compressed":25108,"data_compressed_allocated":32768,"data_compressed_original":409600,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430283,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":1036188,"data_compressed":41637,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593809,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":443054,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790038,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":1035340,"data_compressed":41637,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953563,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":888832,"data_stored":444023,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149790,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":443365,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378724,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":511840,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542248,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":962560,"data_stored":512266,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705773,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":444325,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34195,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34195,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":69419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":68993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":395784,"data_compressed":18091,"data_compressed_allocated":24576,"data_compressed_original":368640,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":464151,"data_compressed":18091,"data_compressed_allocated":24576,"data_compressed_original":368640,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-26T13:40:26.224 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph pg dump --format=json 2024-07-26T13:40:26.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:26 smithi138 ceph-mon[36954]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-26T13:40:26.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:26 smithi100 ceph-mon[32599]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-26T13:40:26.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:26 smithi167 ceph-mon[37140]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-26T13:40:27.122 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:40:27.388 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:27 smithi100 ceph-mon[32599]: pgmap v374: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 174 KiB/s wr, 51 op/s 2024-07-26T13:40:27.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:27 smithi167 ceph-mon[37140]: pgmap v374: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 174 KiB/s wr, 51 op/s 2024-07-26T13:40:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:27 smithi138 ceph-mon[36954]: pgmap v374: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 174 KiB/s wr, 51 op/s 2024-07-26T13:40:28.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.235+0000 7fd58b93e640 1 -- 172.21.15.100:0/2109929041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd584104140 msgr2=0x7fd584104550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:28.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.235+0000 7fd58b93e640 1 --2- 172.21.15.100:0/2109929041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd584104140 0x7fd584104550 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd580009900 tx=0x7fd58002a480 comp rx=0 tx=0).stop 2024-07-26T13:40:28.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.235+0000 7fd58b93e640 1 -- 172.21.15.100:0/2109929041 shutdown_connections 2024-07-26T13:40:28.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.235+0000 7fd58b93e640 1 --2- 172.21.15.100:0/2109929041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd584102370 0x7fd584111ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:28.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.235+0000 7fd58b93e640 1 --2- 172.21.15.100:0/2109929041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd584101820 0x7fd584101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:28.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.235+0000 7fd58b93e640 1 --2- 172.21.15.100:0/2109929041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd584104140 0x7fd584104550 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:28.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.235+0000 7fd58b93e640 1 -- 172.21.15.100:0/2109929041 >> 172.21.15.100:0/2109929041 conn(0x7fd5840798f0 msgr2=0x7fd584079d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:28.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.235+0000 7fd58b93e640 1 -- 172.21.15.100:0/2109929041 shutdown_connections 2024-07-26T13:40:28.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.235+0000 7fd58b93e640 1 -- 172.21.15.100:0/2109929041 wait complete. 2024-07-26T13:40:28.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.236+0000 7fd58b93e640 1 Processor -- start 2024-07-26T13:40:28.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.236+0000 7fd58b93e640 1 -- start start 2024-07-26T13:40:28.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.236+0000 7fd58b93e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd584101820 0x7fd58419c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:28.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.236+0000 7fd58b93e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd584102370 0x7fd58419c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:28.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.236+0000 7fd58b93e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd584104140 0x7fd5841a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:28.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.236+0000 7fd58b93e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd584115790 con 0x7fd584101820 2024-07-26T13:40:28.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.236+0000 7fd58b93e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd584115610 con 0x7fd584104140 2024-07-26T13:40:28.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.236+0000 7fd58b93e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd584115910 con 0x7fd584102370 2024-07-26T13:40:28.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.236+0000 7fd5896da640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd584101820 0x7fd58419c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:28.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.236+0000 7fd5896da640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd584101820 0x7fd58419c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44908/0 (socket says 172.21.15.100:44908) 2024-07-26T13:40:28.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.236+0000 7fd5896da640 1 -- 172.21.15.100:0/3268049061 learned_addr learned my addr 172.21.15.100:0/3268049061 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:28.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.236+0000 7fd588ed9640 1 --2- 172.21.15.100:0/3268049061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd584102370 0x7fd58419c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:28.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.236+0000 7fd5896da640 1 -- 172.21.15.100:0/3268049061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd584102370 msgr2=0x7fd58419c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:28.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.236+0000 7fd5896da640 1 --2- 172.21.15.100:0/3268049061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd584102370 0x7fd58419c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:28.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.237+0000 7fd5896da640 1 -- 172.21.15.100:0/3268049061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd584104140 msgr2=0x7fd5841a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:28.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.237+0000 7fd5896da640 1 --2- 172.21.15.100:0/3268049061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd584104140 0x7fd5841a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:28.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.237+0000 7fd5896da640 1 -- 172.21.15.100:0/3268049061 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5841a2090 con 0x7fd584101820 2024-07-26T13:40:28.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.237+0000 7fd588ed9640 1 --2- 172.21.15.100:0/3268049061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd584102370 0x7fd58419c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:40:28.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.237+0000 7fd5896da640 1 --2- 172.21.15.100:0/3268049061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd584101820 0x7fd58419c120 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd580005940 tx=0x7fd5800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:28.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.237+0000 7fd57a7fc640 1 -- 172.21.15.100:0/3268049061 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58003f030 con 0x7fd584101820 2024-07-26T13:40:28.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.237+0000 7fd57a7fc640 1 -- 172.21.15.100:0/3268049061 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd580003cb0 con 0x7fd584101820 2024-07-26T13:40:28.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.237+0000 7fd58b93e640 1 -- 172.21.15.100:0/3268049061 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5841a2320 con 0x7fd584101820 2024-07-26T13:40:28.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.237+0000 7fd57a7fc640 1 -- 172.21.15.100:0/3268049061 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd580035070 con 0x7fd584101820 2024-07-26T13:40:28.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.237+0000 7fd58b93e640 1 -- 172.21.15.100:0/3268049061 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5841a8ae0 con 0x7fd584101820 2024-07-26T13:40:28.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.239+0000 7fd57a7fc640 1 -- 172.21.15.100:0/3268049061 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 151134+0+0 (secure 0 0 0) 0x7fd580042030 con 0x7fd584101820 2024-07-26T13:40:28.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.240+0000 7fd591ec2740 1 Processor -- start 2024-07-26T13:40:28.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.240+0000 7fd591ec2740 1 -- start start 2024-07-26T13:40:28.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.240+0000 7fd591ec2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557827ace070 0x557827e73670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:28.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.240+0000 7fd591ec2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557827aa51b0 0x557827e73bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:28.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.240+0000 7fd591ec2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557827aad390 0x557827e740f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:28.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.240+0000 7fd591ec2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x557827de9b90 con 0x557827aad390 2024-07-26T13:40:28.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.240+0000 7fd591ec2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557827dd9090 con 0x557827ace070 2024-07-26T13:40:28.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.240+0000 7fd591ec2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557827ddd0d0 con 0x557827aa51b0 2024-07-26T13:40:28.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.240+0000 7fd57a7fc640 1 --2- 172.21.15.100:0/3268049061 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd5700b3cb0 0x7fd5700b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:28.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.240+0000 7fd588ed9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557827aad390 0x557827e740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:28.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.240+0000 7fd588ed9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557827aad390 0x557827e740f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44924/0 (socket says 172.21.15.100:44924) 2024-07-26T13:40:28.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.240+0000 7fd588ed9640 1 -- 172.21.15.100:0/3137759659 learned_addr learned my addr 172.21.15.100:0/3137759659 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:28.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.241+0000 7fd589edb640 1 --2- 172.21.15.100:0/3137759659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557827ace070 0x557827e73670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:28.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.241+0000 7fd589edb640 1 --2- 172.21.15.100:0/3268049061 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd5700b3cb0 0x7fd5700b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:28.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.241+0000 7fd5896da640 1 --2- 172.21.15.100:0/3137759659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557827aa51b0 0x557827e73bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:28.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.241+0000 7fd57a7fc640 1 -- 172.21.15.100:0/3268049061 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fd580104870 con 0x7fd584101820 2024-07-26T13:40:28.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.241+0000 7fd588ed9640 1 -- 172.21.15.100:0/3137759659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557827aa51b0 msgr2=0x557827e73bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:28.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.241+0000 7fd588ed9640 1 --2- 172.21.15.100:0/3137759659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557827aa51b0 0x557827e73bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:28.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.241+0000 7fd588ed9640 1 -- 172.21.15.100:0/3137759659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557827ace070 msgr2=0x557827e73670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:28.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.241+0000 7fd588ed9640 1 --2- 172.21.15.100:0/3137759659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557827ace070 0x557827e73670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:28.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.241+0000 7fd588ed9640 1 -- 172.21.15.100:0/3137759659 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557827d7fb70 con 0x557827aad390 2024-07-26T13:40:28.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.241+0000 7fd589edb640 1 --2- 172.21.15.100:0/3137759659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557827ace070 0x557827e73670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:40:28.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.241+0000 7fd588ed9640 1 --2- 172.21.15.100:0/3137759659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557827aad390 0x557827e740f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd574009bf0 tx=0x7fd57400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=558bed2154cc0e88 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:28.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.241+0000 7fd589edb640 1 --2- 172.21.15.100:0/3268049061 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd5700b3cb0 0x7fd5700b61c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009fd0 tx=0x7fd57c005bd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:28.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.241+0000 7fd56f7fe640 1 -- 172.21.15.100:0/3137759659 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd574016070 con 0x557827aad390 2024-07-26T13:40:28.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.241+0000 7fd56f7fe640 1 -- 172.21.15.100:0/3137759659 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd574011920 con 0x557827aad390 2024-07-26T13:40:28.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.241+0000 7fd56f7fe640 1 -- 172.21.15.100:0/3137759659 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5740107d0 con 0x557827aad390 2024-07-26T13:40:28.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.242+0000 7fd591ec2740 1 -- 172.21.15.100:0/3137759659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557827aad390 msgr2=0x557827e740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:28.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.242+0000 7fd591ec2740 1 --2- 172.21.15.100:0/3137759659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557827aad390 0x557827e740f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd574009bf0 tx=0x7fd57400e6c0 comp rx=0 tx=0).stop 2024-07-26T13:40:28.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.242+0000 7fd591ec2740 1 -- 172.21.15.100:0/3137759659 shutdown_connections 2024-07-26T13:40:28.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.242+0000 7fd591ec2740 1 --2- 172.21.15.100:0/3137759659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557827aad390 0x557827e740f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:28.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.242+0000 7fd591ec2740 1 --2- 172.21.15.100:0/3137759659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557827aa51b0 0x557827e73bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:28.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.242+0000 7fd591ec2740 1 --2- 172.21.15.100:0/3137759659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557827ace070 0x557827e73670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:28.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.242+0000 7fd591ec2740 1 -- 172.21.15.100:0/3137759659 >> 172.21.15.100:0/3137759659 conn(0x557827ac2460 msgr2=0x557827e71560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:28.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.242+0000 7fd591ec2740 1 -- 172.21.15.100:0/3137759659 shutdown_connections 2024-07-26T13:40:28.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.242+0000 7fd591ec2740 1 -- 172.21.15.100:0/3137759659 wait complete. 2024-07-26T13:40:28.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.243+0000 7fd591ec2740 1 -- start start 2024-07-26T13:40:28.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.243+0000 7fd591ec2740 1 Processor -- start 2024-07-26T13:40:28.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.243+0000 7fd591ec2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557827aa51b0 0x557827e7f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:28.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.243+0000 7fd591ec2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557827aad390 0x557827e7f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:28.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.243+0000 7fd591ec2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557827ace070 0x557827e8b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:28.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.243+0000 7fd591ec2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x557827c3c700 con 0x557827aa51b0 2024-07-26T13:40:28.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.243+0000 7fd591ec2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557827d7b9c0 con 0x557827ace070 2024-07-26T13:40:28.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.243+0000 7fd591ec2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557827e81e50 con 0x557827aad390 2024-07-26T13:40:28.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.243+0000 7fd591ec2740 10 client.0 _ll_register_callbacks cb 0x557827d964d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:40:28.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.243+0000 7fd5896da640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557827aa51b0 0x557827e7f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:28.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.244+0000 7fd5896da640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557827aa51b0 0x557827e7f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44934/0 (socket says 172.21.15.100:44934) 2024-07-26T13:40:28.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.244+0000 7fd5896da640 1 -- 172.21.15.100:0/3335513201 learned_addr learned my addr 172.21.15.100:0/3335513201 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:28.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.244+0000 7fd588ed9640 1 --2- 172.21.15.100:0/3335513201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557827aad390 0x557827e7f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:28.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.244+0000 7fd589edb640 1 --2- 172.21.15.100:0/3335513201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557827ace070 0x557827e8b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:28.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.244+0000 7fd5896da640 1 -- 172.21.15.100:0/3335513201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557827aad390 msgr2=0x557827e7f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:28.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.244+0000 7fd5896da640 1 --2- 172.21.15.100:0/3335513201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557827aad390 0x557827e7f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:28.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.244+0000 7fd5896da640 1 -- 172.21.15.100:0/3335513201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557827ace070 msgr2=0x557827e8b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:28.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.244+0000 7fd5896da640 1 --2- 172.21.15.100:0/3335513201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557827ace070 0x557827e8b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:28.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.244+0000 7fd5896da640 1 -- 172.21.15.100:0/3335513201 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557827b39650 con 0x557827aa51b0 2024-07-26T13:40:28.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.244+0000 7fd5896da640 1 --2- 172.21.15.100:0/3335513201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557827aa51b0 0x557827e7f180 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd580102150 tx=0x7fd5800414b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c0a22e5eacb7f8d server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:28.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.244+0000 7fd56cff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:40:28.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.244+0000 7fd56cff9640 1 -- 172.21.15.100:0/3335513201 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd580049070 con 0x557827aa51b0 2024-07-26T13:40:28.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.244+0000 7fd56cff9640 1 -- 172.21.15.100:0/3335513201 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fd58002f300 con 0x557827aa51b0 2024-07-26T13:40:28.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.244+0000 7fd591ec2740 1 -- 172.21.15.100:0/3335513201 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd574011920 con 0x557827aa51b0 2024-07-26T13:40:28.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.244+0000 7fd56cff9640 1 -- 172.21.15.100:0/3335513201 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd580003e60 con 0x557827aa51b0 2024-07-26T13:40:28.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.244+0000 7fd591ec2740 10 client.15465 fetch_fsmap learned FSMap version 26 2024-07-26T13:40:28.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.244+0000 7fd591ec2740 1 -- 172.21.15.100:0/3335513201 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557827e8ddb0 con 0x557827aa51b0 2024-07-26T13:40:28.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.245+0000 7fd56cff9640 1 -- 172.21.15.100:0/3335513201 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd574011920 con 0x557827aa51b0 2024-07-26T13:40:28.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.245+0000 7fd56cff9640 1 -- 172.21.15.100:0/3335513201 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fd5800cb7c0 con 0x557827aa51b0 2024-07-26T13:40:28.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.245+0000 7fd591ec2740 10 client.15465 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:40:28.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.245+0000 7fd591ec2740 10 client.15465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:40:28.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.245+0000 7fd591ec2740 20 client.15465 populate_metadata read hostname 'smithi100' 2024-07-26T13:40:28.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.245+0000 7fd591ec2740 1 --2- 172.21.15.100:0/3335513201 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x557827a8c9f0 0x557827e912b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:28.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.245+0000 7fd591ec2740 4 client.15465 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:40:28.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.245+0000 7fd591ec2740 1 -- 172.21.15.100:0/3335513201 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557827debae0 con 0x557827a8c9f0 2024-07-26T13:40:28.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.245+0000 7fd588ed9640 1 --2- 172.21.15.100:0/3335513201 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x557827a8c9f0 0x557827e912b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:28.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.246+0000 7fd56cff9640 10 client.15465 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:40:28.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.246+0000 7fd588ed9640 1 --2- 172.21.15.100:0/3335513201 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x557827a8c9f0 0x557827e912b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6518d000eba0a22 server_cookie=db1585dad7bb3385 in_seq=0 out_seq=0 2024-07-26T13:40:28.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.246+0000 7fd56cff9640 1 -- 172.21.15.100:0/3335513201 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557827debae0 con 0x557827a8c9f0 2024-07-26T13:40:28.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.246+0000 7fd56cff9640 10 client.15465 handle_command_reply: tid=0 2024-07-26T13:40:28.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.246+0000 7fd591ec2740 1 client.15465 shutdown 2024-07-26T13:40:28.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.247+0000 7fd591ec2740 1 -- 172.21.15.100:0/3335513201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557827aa51b0 msgr2=0x557827e7f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:28.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.247+0000 7fd591ec2740 1 --2- 172.21.15.100:0/3335513201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557827aa51b0 0x557827e7f180 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd580102150 tx=0x7fd5800414b0 comp rx=0 tx=0).stop 2024-07-26T13:40:28.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.247+0000 7fd591ec2740 1 -- 172.21.15.100:0/3335513201 shutdown_connections 2024-07-26T13:40:28.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.247+0000 7fd591ec2740 1 --2- 172.21.15.100:0/3335513201 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x557827a8c9f0 0x557827e912b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:28.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.247+0000 7fd591ec2740 1 --2- 172.21.15.100:0/3335513201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557827ace070 0x557827e8b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:28.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.247+0000 7fd591ec2740 1 --2- 172.21.15.100:0/3335513201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557827aad390 0x557827e7f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:28.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.247+0000 7fd56cff9640 0 client.15465 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:40:28.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.247+0000 7fd591ec2740 1 --2- 172.21.15.100:0/3335513201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557827aa51b0 0x557827e7f180 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:28.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.247+0000 7fd591ec2740 1 -- 172.21.15.100:0/3335513201 >> 172.21.15.100:0/3335513201 conn(0x557827ac2460 msgr2=0x557827e6fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:28.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.247+0000 7fd591ec2740 1 -- 172.21.15.100:0/3335513201 shutdown_connections 2024-07-26T13:40:28.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.247+0000 7fd591ec2740 1 -- 172.21.15.100:0/3335513201 wait complete. 2024-07-26T13:40:28.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.247+0000 7fd591ec2740 20 client.15465 trim_cache size 0 max 16384 2024-07-26T13:40:28.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.264+0000 7fd591ec2740 1 Processor -- start 2024-07-26T13:40:28.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.264+0000 7fd591ec2740 1 -- start start 2024-07-26T13:40:28.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.264+0000 7fd591ec2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557827aa51b0 0x557827e715d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:28.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.264+0000 7fd591ec2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557827aad390 0x557827e8af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:28.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.264+0000 7fd591ec2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557827ace070 0x557827e8b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:28.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.264+0000 7fd591ec2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd5841b3b80 con 0x557827ace070 2024-07-26T13:40:28.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.264+0000 7fd591ec2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd5841b1450 con 0x557827aad390 2024-07-26T13:40:28.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.264+0000 7fd591ec2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd5841af2a0 con 0x557827aa51b0 2024-07-26T13:40:28.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.264+0000 7fd589edb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557827ace070 0x557827e8b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:28.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.264+0000 7fd589edb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557827ace070 0x557827e8b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44944/0 (socket says 172.21.15.100:44944) 2024-07-26T13:40:28.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.264+0000 7fd589edb640 1 -- 172.21.15.100:0/1370426088 learned_addr learned my addr 172.21.15.100:0/1370426088 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:28.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.264+0000 7fd588ed9640 1 --2- 172.21.15.100:0/1370426088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557827aad390 0x557827e8af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:28.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.264+0000 7fd5896da640 1 --2- 172.21.15.100:0/1370426088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557827aa51b0 0x557827e715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:28.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.264+0000 7fd589edb640 1 -- 172.21.15.100:0/1370426088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557827aa51b0 msgr2=0x557827e715d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:28.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.264+0000 7fd589edb640 1 --2- 172.21.15.100:0/1370426088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557827aa51b0 0x557827e715d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:28.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.264+0000 7fd589edb640 1 -- 172.21.15.100:0/1370426088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557827aad390 msgr2=0x557827e8af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:28.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.264+0000 7fd589edb640 1 --2- 172.21.15.100:0/1370426088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557827aad390 0x557827e8af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:28.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.264+0000 7fd589edb640 1 -- 172.21.15.100:0/1370426088 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557827e8c0d0 con 0x557827ace070 2024-07-26T13:40:28.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.265+0000 7fd589edb640 1 --2- 172.21.15.100:0/1370426088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557827ace070 0x557827e8b470 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009e70 tx=0x7fd57c014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=998ff0600b9f04cf server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:28.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.265+0000 7fd56effd640 1 -- 172.21.15.100:0/1370426088 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c004b20 con 0x557827ace070 2024-07-26T13:40:28.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.265+0000 7fd56effd640 1 -- 172.21.15.100:0/1370426088 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd57c004cc0 con 0x557827ace070 2024-07-26T13:40:28.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.265+0000 7fd56effd640 1 -- 172.21.15.100:0/1370426088 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c0286e0 con 0x557827ace070 2024-07-26T13:40:28.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.266+0000 7fd591ec2740 1 -- 172.21.15.100:0/1370426088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557827ace070 msgr2=0x557827e8b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:28.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.266+0000 7fd591ec2740 1 --2- 172.21.15.100:0/1370426088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557827ace070 0x557827e8b470 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009e70 tx=0x7fd57c014040 comp rx=0 tx=0).stop 2024-07-26T13:40:28.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.266+0000 7fd591ec2740 1 -- 172.21.15.100:0/1370426088 shutdown_connections 2024-07-26T13:40:28.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.266+0000 7fd591ec2740 1 --2- 172.21.15.100:0/1370426088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557827ace070 0x557827e8b470 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:28.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.266+0000 7fd591ec2740 1 --2- 172.21.15.100:0/1370426088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557827aad390 0x557827e8af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:28.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.266+0000 7fd591ec2740 1 --2- 172.21.15.100:0/1370426088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557827aa51b0 0x557827e715d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:28.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.266+0000 7fd591ec2740 1 -- 172.21.15.100:0/1370426088 >> 172.21.15.100:0/1370426088 conn(0x557827ac2460 msgr2=0x557827e70ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:28.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.266+0000 7fd591ec2740 1 -- 172.21.15.100:0/1370426088 shutdown_connections 2024-07-26T13:40:28.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.266+0000 7fd591ec2740 1 -- 172.21.15.100:0/1370426088 wait complete. 2024-07-26T13:40:28.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.266+0000 7fd591ec2740 1 -- start start 2024-07-26T13:40:28.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.266+0000 7fd591ec2740 1 Processor -- start 2024-07-26T13:40:28.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.266+0000 7fd591ec2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557827aa51b0 0x557827e87670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:28.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.266+0000 7fd591ec2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557827aad390 0x557827e82b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:28.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.267+0000 7fd591ec2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557827ace070 0x557827e830b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:28.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.267+0000 7fd591ec2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd5841af910 con 0x557827aa51b0 2024-07-26T13:40:28.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.267+0000 7fd591ec2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd5841b0d80 con 0x557827ace070 2024-07-26T13:40:28.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.267+0000 7fd591ec2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd5841affe0 con 0x557827aad390 2024-07-26T13:40:28.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.267+0000 7fd5896da640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557827aa51b0 0x557827e87670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:28.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.267+0000 7fd591ec2740 10 client.0 _ll_register_callbacks cb 0x7fd580009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:40:28.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.267+0000 7fd5896da640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557827aa51b0 0x557827e87670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44960/0 (socket says 172.21.15.100:44960) 2024-07-26T13:40:28.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.267+0000 7fd5896da640 1 -- 172.21.15.100:0/2029049432 learned_addr learned my addr 172.21.15.100:0/2029049432 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:28.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.267+0000 7fd589edb640 1 --2- 172.21.15.100:0/2029049432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557827ace070 0x557827e830b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:28.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.267+0000 7fd588ed9640 1 --2- 172.21.15.100:0/2029049432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557827aad390 0x557827e82b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:28.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.267+0000 7fd5896da640 1 -- 172.21.15.100:0/2029049432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557827aad390 msgr2=0x557827e82b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:28.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.267+0000 7fd5896da640 1 --2- 172.21.15.100:0/2029049432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557827aad390 0x557827e82b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:28.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.267+0000 7fd5896da640 1 -- 172.21.15.100:0/2029049432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557827ace070 msgr2=0x557827e830b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:28.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.267+0000 7fd5896da640 1 --2- 172.21.15.100:0/2029049432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557827ace070 0x557827e830b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:28.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.267+0000 7fd5896da640 1 -- 172.21.15.100:0/2029049432 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557827e83750 con 0x557827aa51b0 2024-07-26T13:40:28.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.267+0000 7fd589edb640 1 --2- 172.21.15.100:0/2029049432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557827ace070 0x557827e830b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:40:28.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.267+0000 7fd5896da640 1 --2- 172.21.15.100:0/2029049432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557827aa51b0 0x557827e87670 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd5800c87d0 tx=0x7fd5800c7420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c04369da131de3 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:28.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.267+0000 7fd56cff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:40:28.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.267+0000 7fd56cff9640 1 -- 172.21.15.100:0/2029049432 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd580049070 con 0x557827aa51b0 2024-07-26T13:40:28.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.267+0000 7fd56cff9640 1 -- 172.21.15.100:0/2029049432 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fd5800c7b50 con 0x557827aa51b0 2024-07-26T13:40:28.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.267+0000 7fd591ec2740 1 -- 172.21.15.100:0/2029049432 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd57c004cc0 con 0x557827aa51b0 2024-07-26T13:40:28.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.267+0000 7fd56cff9640 1 -- 172.21.15.100:0/2029049432 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd5800d4660 con 0x557827aa51b0 2024-07-26T13:40:28.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.267+0000 7fd591ec2740 10 client.15471 fetch_fsmap learned FSMap version 26 2024-07-26T13:40:28.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.267+0000 7fd591ec2740 1 -- 172.21.15.100:0/2029049432 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557827ef9850 con 0x557827aa51b0 2024-07-26T13:40:28.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.268+0000 7fd56cff9640 1 -- 172.21.15.100:0/2029049432 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd57c004cc0 con 0x557827aa51b0 2024-07-26T13:40:28.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.268+0000 7fd56cff9640 1 -- 172.21.15.100:0/2029049432 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fd58003bbe0 con 0x557827aa51b0 2024-07-26T13:40:28.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.268+0000 7fd591ec2740 10 client.15471 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:40:28.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.268+0000 7fd591ec2740 10 client.15471 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:40:28.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.268+0000 7fd591ec2740 20 client.15471 populate_metadata read hostname 'smithi100' 2024-07-26T13:40:28.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.268+0000 7fd591ec2740 1 --2- 172.21.15.100:0/2029049432 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x557827a61da0 0x557827efbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:28.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.268+0000 7fd591ec2740 4 client.15471 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-26T13:40:28.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.268+0000 7fd591ec2740 1 -- 172.21.15.100:0/2029049432 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557827de7580 con 0x557827a61da0 2024-07-26T13:40:28.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.268+0000 7fd588ed9640 1 --2- 172.21.15.100:0/2029049432 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x557827a61da0 0x557827efbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:28.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.269+0000 7fd56cff9640 10 client.15471 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:40:28.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.269+0000 7fd588ed9640 1 --2- 172.21.15.100:0/2029049432 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x557827a61da0 0x557827efbeb0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65fe9649c280be9c server_cookie=83db7fe2efe631c4 in_seq=0 out_seq=0 2024-07-26T13:40:28.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.269+0000 7fd56cff9640 1 -- 172.21.15.100:0/2029049432 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557827de7580 con 0x557827a61da0 2024-07-26T13:40:28.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.269+0000 7fd56cff9640 10 client.15471 handle_command_reply: tid=0 2024-07-26T13:40:28.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.269+0000 7fd591ec2740 1 client.15471 shutdown 2024-07-26T13:40:28.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.269+0000 7fd591ec2740 1 -- 172.21.15.100:0/2029049432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557827aa51b0 msgr2=0x557827e87670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:28.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.269+0000 7fd591ec2740 1 --2- 172.21.15.100:0/2029049432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557827aa51b0 0x557827e87670 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd5800c87d0 tx=0x7fd5800c7420 comp rx=0 tx=0).stop 2024-07-26T13:40:28.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.269+0000 7fd591ec2740 1 -- 172.21.15.100:0/2029049432 shutdown_connections 2024-07-26T13:40:28.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.269+0000 7fd591ec2740 1 --2- 172.21.15.100:0/2029049432 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x557827a61da0 0x557827efbeb0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:28.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.269+0000 7fd591ec2740 1 --2- 172.21.15.100:0/2029049432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557827ace070 0x557827e830b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:28.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.269+0000 7fd591ec2740 1 --2- 172.21.15.100:0/2029049432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557827aad390 0x557827e82b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:28.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.269+0000 7fd591ec2740 1 --2- 172.21.15.100:0/2029049432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557827aa51b0 0x557827e87670 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:28.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.269+0000 7fd591ec2740 1 -- 172.21.15.100:0/2029049432 >> 172.21.15.100:0/2029049432 conn(0x557827ac2460 msgr2=0x557827e746a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:28.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.269+0000 7fd56cff9640 0 client.15471 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:40:28.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.269+0000 7fd591ec2740 1 -- 172.21.15.100:0/2029049432 shutdown_connections 2024-07-26T13:40:28.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.269+0000 7fd591ec2740 1 -- 172.21.15.100:0/2029049432 wait complete. 2024-07-26T13:40:28.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.269+0000 7fd591ec2740 20 client.15471 trim_cache size 0 max 16384 2024-07-26T13:40:28.276 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:40:28.276 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-26T13:40:28.276 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-07-26T13:40:28.276 INFO:teuthology.orchestra.run.smithi100.stdout: "cef38ea4-fdd8-460a-89cf-1770cd63e758": { 2024-07-26T13:40:28.276 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-07-26T13:40:28.276 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "cef38ea4-fdd8-460a-89cf-1770cd63e758", 2024-07-26T13:40:28.277 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-07-26T13:40:28.277 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:40:28.277 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:40:28.277 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:40:28.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.270+0000 7fd58b93e640 1 -- 172.21.15.100:0/3268049061 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd5700b3cb0 msgr2=0x7fd5700b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:28.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.270+0000 7fd58b93e640 1 --2- 172.21.15.100:0/3268049061 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd5700b3cb0 0x7fd5700b61c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009fd0 tx=0x7fd57c005bd0 comp rx=0 tx=0).stop 2024-07-26T13:40:28.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.270+0000 7fd58b93e640 1 -- 172.21.15.100:0/3268049061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd584101820 msgr2=0x7fd58419c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:28.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.270+0000 7fd58b93e640 1 --2- 172.21.15.100:0/3268049061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd584101820 0x7fd58419c120 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd580005940 tx=0x7fd5800042f0 comp rx=0 tx=0).stop 2024-07-26T13:40:28.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.270+0000 7fd58b93e640 1 -- 172.21.15.100:0/3268049061 shutdown_connections 2024-07-26T13:40:28.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.270+0000 7fd58b93e640 1 --2- 172.21.15.100:0/3268049061 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd5700b3cb0 0x7fd5700b61c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:28.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.270+0000 7fd58b93e640 1 --2- 172.21.15.100:0/3268049061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd584104140 0x7fd5841a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:28.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.270+0000 7fd58b93e640 1 --2- 172.21.15.100:0/3268049061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd584102370 0x7fd58419c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:28.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.270+0000 7fd58b93e640 1 --2- 172.21.15.100:0/3268049061 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd584101820 0x7fd58419c120 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:28.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.270+0000 7fd58b93e640 1 -- 172.21.15.100:0/3268049061 >> 172.21.15.100:0/3268049061 conn(0x7fd5840798f0 msgr2=0x7fd5840ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:28.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.270+0000 7fd58b93e640 1 -- 172.21.15.100:0/3268049061 shutdown_connections 2024-07-26T13:40:28.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:28.270+0000 7fd58b93e640 1 -- 172.21.15.100:0/3268049061 wait complete. 2024-07-26T13:40:29.024 INFO:tasks.cephfs.filesystem:scrub status for tag:cef38ea4-fdd8-460a-89cf-1770cd63e758 - {'path': '/', 'tag': 'cef38ea4-fdd8-460a-89cf-1770cd63e758', 'options': 'recursive,force'} 2024-07-26T13:40:29.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:29 smithi100 ceph-mon[32599]: pgmap v375: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 174 KiB/s wr, 52 op/s 2024-07-26T13:40:29.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:29 smithi167 ceph-mon[37140]: pgmap v375: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 174 KiB/s wr, 52 op/s 2024-07-26T13:40:29.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:29 smithi138 ceph-mon[36954]: pgmap v375: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 174 KiB/s wr, 52 op/s 2024-07-26T13:40:30.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.013+0000 7f8070c91640 1 -- 172.21.15.100:0/2984576207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f806c106d90 msgr2=0x7f806c1071a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:30.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.013+0000 7f8070c91640 1 --2- 172.21.15.100:0/2984576207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f806c106d90 0x7f806c1071a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8060009900 tx=0x7f806002a410 comp rx=0 tx=0).stop 2024-07-26T13:40:30.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.013+0000 7f8070c91640 1 -- 172.21.15.100:0/2984576207 shutdown_connections 2024-07-26T13:40:30.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.013+0000 7f8070c91640 1 --2- 172.21.15.100:0/2984576207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f806c1076e0 0x7f806c110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:30.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.013+0000 7f8070c91640 1 --2- 172.21.15.100:0/2984576207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f806c106d90 0x7f806c1071a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:30.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.013+0000 7f8070c91640 1 --2- 172.21.15.100:0/2984576207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f806c101b90 0x7f806c101fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:30.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.013+0000 7f8070c91640 1 -- 172.21.15.100:0/2984576207 >> 172.21.15.100:0/2984576207 conn(0x7f806c0798d0 msgr2=0x7f806c079ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:30.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.013+0000 7f8070c91640 1 -- 172.21.15.100:0/2984576207 shutdown_connections 2024-07-26T13:40:30.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.013+0000 7f8070c91640 1 -- 172.21.15.100:0/2984576207 wait complete. 2024-07-26T13:40:30.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.014+0000 7f8070c91640 1 Processor -- start 2024-07-26T13:40:30.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.014+0000 7f8070c91640 1 -- start start 2024-07-26T13:40:30.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.014+0000 7f8070c91640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f806c101b90 0x7f806c102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:30.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.014+0000 7f8070c91640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f806c106d90 0x7f806c103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:30.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.014+0000 7f8070c91640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f806c1076e0 0x7f806c1035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:30.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.014+0000 7f8070c91640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f806c113570 con 0x7f806c1076e0 2024-07-26T13:40:30.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.014+0000 7f8070c91640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f806c1133f0 con 0x7f806c106d90 2024-07-26T13:40:30.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.014+0000 7f8070c91640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f806c1136f0 con 0x7f806c101b90 2024-07-26T13:40:30.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.014+0000 7f806ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f806c1076e0 0x7f806c1035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:30.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.014+0000 7f806ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f806c1076e0 0x7f806c1035b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44988/0 (socket says 172.21.15.100:44988) 2024-07-26T13:40:30.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.014+0000 7f8069d9b640 1 --2- 172.21.15.100:0/4187189306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f806c106d90 0x7f806c103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:30.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.014+0000 7f806ad9d640 1 -- 172.21.15.100:0/4187189306 learned_addr learned my addr 172.21.15.100:0/4187189306 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:30.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.014+0000 7f806a59c640 1 --2- 172.21.15.100:0/4187189306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f806c101b90 0x7f806c102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:30.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.015+0000 7f806ad9d640 1 -- 172.21.15.100:0/4187189306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f806c101b90 msgr2=0x7f806c102b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:30.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.015+0000 7f806ad9d640 1 --2- 172.21.15.100:0/4187189306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f806c101b90 0x7f806c102b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:30.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.015+0000 7f806ad9d640 1 -- 172.21.15.100:0/4187189306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f806c106d90 msgr2=0x7f806c103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:30.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.015+0000 7f806ad9d640 1 --2- 172.21.15.100:0/4187189306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f806c106d90 0x7f806c103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:30.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.015+0000 7f806ad9d640 1 -- 172.21.15.100:0/4187189306 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f806c1a81b0 con 0x7f806c1076e0 2024-07-26T13:40:30.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.015+0000 7f8069d9b640 1 --2- 172.21.15.100:0/4187189306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f806c106d90 0x7f806c103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:40:30.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.015+0000 7f806ad9d640 1 --2- 172.21.15.100:0/4187189306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f806c1076e0 0x7f806c1035b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f806400c970 tx=0x7f806400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:30.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.015+0000 7f805b7fe640 1 -- 172.21.15.100:0/4187189306 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8064007bb0 con 0x7f806c1076e0 2024-07-26T13:40:30.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.015+0000 7f8070c91640 1 -- 172.21.15.100:0/4187189306 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f806c1a84a0 con 0x7f806c1076e0 2024-07-26T13:40:30.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.015+0000 7f805b7fe640 1 -- 172.21.15.100:0/4187189306 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8064007d50 con 0x7f806c1076e0 2024-07-26T13:40:30.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.015+0000 7f805b7fe640 1 -- 172.21.15.100:0/4187189306 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8064004e60 con 0x7f806c1076e0 2024-07-26T13:40:30.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.015+0000 7f8070c91640 1 -- 172.21.15.100:0/4187189306 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f806c1a8a00 con 0x7f806c1076e0 2024-07-26T13:40:30.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.016+0000 7f8070c91640 1 -- 172.21.15.100:0/4187189306 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f806c101fa0 con 0x7f806c1076e0 2024-07-26T13:40:30.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.018+0000 7f805b7fe640 1 -- 172.21.15.100:0/4187189306 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 151134+0+0 (secure 0 0 0) 0x7f8064005000 con 0x7f806c1076e0 2024-07-26T13:40:30.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.018+0000 7f805b7fe640 1 --2- 172.21.15.100:0/4187189306 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f80540b3cb0 0x7f80540b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:30.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.018+0000 7f806a59c640 1 --2- 172.21.15.100:0/4187189306 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f80540b3cb0 0x7f80540b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:30.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.018+0000 7f805b7fe640 1 -- 172.21.15.100:0/4187189306 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f80640e3f20 con 0x7f806c1076e0 2024-07-26T13:40:30.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.019+0000 7f806a59c640 1 --2- 172.21.15.100:0/4187189306 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f80540b3cb0 0x7f80540b61c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f806c104580 tx=0x7f805c0075b0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:30.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.021+0000 7f805b7fe640 1 -- 172.21.15.100:0/4187189306 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f8064014030 con 0x7f806c1076e0 2024-07-26T13:40:30.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.190+0000 7f8070c91640 1 -- 172.21.15.100:0/4187189306 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f806c063230 con 0x7f80540b3cb0 2024-07-26T13:40:30.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.195+0000 7f805b7fe640 1 -- 172.21.15.100:0/4187189306 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259206 (secure 0 0 0) 0x7f806c063230 con 0x7f80540b3cb0 2024-07-26T13:40:30.198 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:40:30.199 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-07-26T13:40:30.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.198+0000 7f8070c91640 1 -- 172.21.15.100:0/4187189306 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f80540b3cb0 msgr2=0x7f80540b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:30.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.198+0000 7f8070c91640 1 --2- 172.21.15.100:0/4187189306 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f80540b3cb0 0x7f80540b61c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f806c104580 tx=0x7f805c0075b0 comp rx=0 tx=0).stop 2024-07-26T13:40:30.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.198+0000 7f8070c91640 1 -- 172.21.15.100:0/4187189306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f806c1076e0 msgr2=0x7f806c1035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:30.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.198+0000 7f8070c91640 1 --2- 172.21.15.100:0/4187189306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f806c1076e0 0x7f806c1035b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f806400c970 tx=0x7f806400ce30 comp rx=0 tx=0).stop 2024-07-26T13:40:30.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.198+0000 7f8070c91640 1 -- 172.21.15.100:0/4187189306 shutdown_connections 2024-07-26T13:40:30.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.198+0000 7f8070c91640 1 --2- 172.21.15.100:0/4187189306 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f80540b3cb0 0x7f80540b61c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:30.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.198+0000 7f8070c91640 1 --2- 172.21.15.100:0/4187189306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f806c1076e0 0x7f806c1035b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:30.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.198+0000 7f8070c91640 1 --2- 172.21.15.100:0/4187189306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f806c106d90 0x7f806c103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:30.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.198+0000 7f8070c91640 1 --2- 172.21.15.100:0/4187189306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f806c101b90 0x7f806c102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:30.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.198+0000 7f8070c91640 1 -- 172.21.15.100:0/4187189306 >> 172.21.15.100:0/4187189306 conn(0x7f806c0798d0 msgr2=0x7f806c105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:30.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.198+0000 7f8070c91640 1 -- 172.21.15.100:0/4187189306 shutdown_connections 2024-07-26T13:40:30.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:30.198+0000 7f8070c91640 1 -- 172.21.15.100:0/4187189306 wait complete. 2024-07-26T13:40:30.916 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":375,"stamp":"2024-07-26T13:40:28.414210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1450533,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1575,"num_read_kb":1026,"num_write":528,"num_write_kb":4369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":2952484,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":889,"ondisk_log_size":889,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381064,"kb_used_data":11100,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560760,"statfs":{"total":1151940427776,"available":1151550218240,"internally_reserved":0,"allocated":11366400,"data_stored":9665763,"data_compressed":289422,"data_compressed_allocated":380928,"data_compressed_original":4251648,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":786992,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":405,"num_read_kb":335,"num_write":224,"num_write_kb":2088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003103"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387528+0000","last_change":"2024-07-26T13:36:59.462289+0000","last_active":"2024-07-26T13:38:46.387528+0000","last_peered":"2024-07-26T13:38:46.387528+0000","last_clean":"2024-07-26T13:38:46.387528+0000","last_became_active":"2024-07-26T13:36:59.460311+0000","last_became_peered":"2024-07-26T13:36:59.460311+0000","last_unstale":"2024-07-26T13:38:46.387528+0000","last_undegraded":"2024-07-26T13:38:46.387528+0000","last_fullsized":"2024-07-26T13:38:46.387528+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:04:59.123716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.384825+0000","last_change":"2024-07-26T13:36:59.463218+0000","last_active":"2024-07-26T13:38:46.384825+0000","last_peered":"2024-07-26T13:38:46.384825+0000","last_clean":"2024-07-26T13:38:46.384825+0000","last_became_active":"2024-07-26T13:36:59.462889+0000","last_became_peered":"2024-07-26T13:36:59.462889+0000","last_unstale":"2024-07-26T13:38:46.384825+0000","last_undegraded":"2024-07-26T13:38:46.384825+0000","last_fullsized":"2024-07-26T13:38:46.384825+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:30:46.788669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383503+0000","last_change":"2024-07-26T13:36:59.456113+0000","last_active":"2024-07-26T13:38:46.383503+0000","last_peered":"2024-07-26T13:38:46.383503+0000","last_clean":"2024-07-26T13:38:46.383503+0000","last_became_active":"2024-07-26T13:36:59.455622+0000","last_became_peered":"2024-07-26T13:36:59.455622+0000","last_unstale":"2024-07-26T13:38:46.383503+0000","last_undegraded":"2024-07-26T13:38:46.383503+0000","last_fullsized":"2024-07-26T13:38:46.383503+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:36:26.474799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.368792+0000","last_change":"2024-07-26T13:37:17.555947+0000","last_active":"2024-07-26T13:38:46.368792+0000","last_peered":"2024-07-26T13:38:46.368792+0000","last_clean":"2024-07-26T13:38:46.368792+0000","last_became_active":"2024-07-26T13:37:17.555607+0000","last_became_peered":"2024-07-26T13:37:17.555607+0000","last_unstale":"2024-07-26T13:38:46.368792+0000","last_undegraded":"2024-07-26T13:38:46.368792+0000","last_fullsized":"2024-07-26T13:38:46.368792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:19:59.603225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.372136+0000","last_change":"2024-07-26T13:36:59.474896+0000","last_active":"2024-07-26T13:38:46.372136+0000","last_peered":"2024-07-26T13:38:46.372136+0000","last_clean":"2024-07-26T13:38:46.372136+0000","last_became_active":"2024-07-26T13:36:59.474516+0000","last_became_peered":"2024-07-26T13:36:59.474516+0000","last_unstale":"2024-07-26T13:38:46.372136+0000","last_undegraded":"2024-07-26T13:38:46.372136+0000","last_fullsized":"2024-07-26T13:38:46.372136+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:38:18.994280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.358362+0000","last_change":"2024-07-26T13:37:17.556191+0000","last_active":"2024-07-26T13:38:46.358362+0000","last_peered":"2024-07-26T13:38:46.358362+0000","last_clean":"2024-07-26T13:38:46.358362+0000","last_became_active":"2024-07-26T13:37:17.555715+0000","last_became_peered":"2024-07-26T13:37:17.555715+0000","last_unstale":"2024-07-26T13:38:46.358362+0000","last_undegraded":"2024-07-26T13:38:46.358362+0000","last_fullsized":"2024-07-26T13:38:46.358362+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:03:18.352049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383784+0000","last_change":"2024-07-26T13:36:59.469515+0000","last_active":"2024-07-26T13:38:46.383784+0000","last_peered":"2024-07-26T13:38:46.383784+0000","last_clean":"2024-07-26T13:38:46.383784+0000","last_became_active":"2024-07-26T13:36:59.469106+0000","last_became_peered":"2024-07-26T13:36:59.469106+0000","last_unstale":"2024-07-26T13:38:46.383784+0000","last_undegraded":"2024-07-26T13:38:46.383784+0000","last_fullsized":"2024-07-26T13:38:46.383784+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:10:05.455329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.373036+0000","last_change":"2024-07-26T13:37:17.549365+0000","last_active":"2024-07-26T13:38:46.373036+0000","last_peered":"2024-07-26T13:38:46.373036+0000","last_clean":"2024-07-26T13:38:46.373036+0000","last_became_active":"2024-07-26T13:37:17.549036+0000","last_became_peered":"2024-07-26T13:37:17.549036+0000","last_unstale":"2024-07-26T13:38:46.373036+0000","last_undegraded":"2024-07-26T13:38:46.373036+0000","last_fullsized":"2024-07-26T13:38:46.373036+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:27:43.930981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387690+0000","last_change":"2024-07-26T13:36:59.459306+0000","last_active":"2024-07-26T13:38:46.387690+0000","last_peered":"2024-07-26T13:38:46.387690+0000","last_clean":"2024-07-26T13:38:46.387690+0000","last_became_active":"2024-07-26T13:36:59.458714+0000","last_became_peered":"2024-07-26T13:36:59.458714+0000","last_unstale":"2024-07-26T13:38:46.387690+0000","last_undegraded":"2024-07-26T13:38:46.387690+0000","last_fullsized":"2024-07-26T13:38:46.387690+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:34:18.512454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.376748+0000","last_change":"2024-07-26T13:36:59.466556+0000","last_active":"2024-07-26T13:38:46.376748+0000","last_peered":"2024-07-26T13:38:46.376748+0000","last_clean":"2024-07-26T13:38:46.376748+0000","last_became_active":"2024-07-26T13:36:59.466147+0000","last_became_peered":"2024-07-26T13:36:59.466147+0000","last_unstale":"2024-07-26T13:38:46.376748+0000","last_undegraded":"2024-07-26T13:38:46.376748+0000","last_fullsized":"2024-07-26T13:38:46.376748+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:43:11.359314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387571+0000","last_change":"2024-07-26T13:36:59.473685+0000","last_active":"2024-07-26T13:38:46.387571+0000","last_peered":"2024-07-26T13:38:46.387571+0000","last_clean":"2024-07-26T13:38:46.387571+0000","last_became_active":"2024-07-26T13:36:59.473314+0000","last_became_peered":"2024-07-26T13:36:59.473314+0000","last_unstale":"2024-07-26T13:38:46.387571+0000","last_undegraded":"2024-07-26T13:38:46.387571+0000","last_fullsized":"2024-07-26T13:38:46.387571+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:41:51.612348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.368487+0000","last_change":"2024-07-26T13:36:59.471586+0000","last_active":"2024-07-26T13:38:46.368487+0000","last_peered":"2024-07-26T13:38:46.368487+0000","last_clean":"2024-07-26T13:38:46.368487+0000","last_became_active":"2024-07-26T13:36:59.470995+0000","last_became_peered":"2024-07-26T13:36:59.470995+0000","last_unstale":"2024-07-26T13:38:46.368487+0000","last_undegraded":"2024-07-26T13:38:46.368487+0000","last_fullsized":"2024-07-26T13:38:46.368487+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:54:07.957937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.358125+0000","last_change":"2024-07-26T13:36:59.460018+0000","last_active":"2024-07-26T13:38:46.358125+0000","last_peered":"2024-07-26T13:38:46.358125+0000","last_clean":"2024-07-26T13:38:46.358125+0000","last_became_active":"2024-07-26T13:36:59.459671+0000","last_became_peered":"2024-07-26T13:36:59.459671+0000","last_unstale":"2024-07-26T13:38:46.358125+0000","last_undegraded":"2024-07-26T13:38:46.358125+0000","last_fullsized":"2024-07-26T13:38:46.358125+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:19:33.923360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.374512+0000","last_change":"2024-07-26T13:36:59.470271+0000","last_active":"2024-07-26T13:38:46.374512+0000","last_peered":"2024-07-26T13:38:46.374512+0000","last_clean":"2024-07-26T13:38:46.374512+0000","last_became_active":"2024-07-26T13:36:59.469941+0000","last_became_peered":"2024-07-26T13:36:59.469941+0000","last_unstale":"2024-07-26T13:38:46.374512+0000","last_undegraded":"2024-07-26T13:38:46.374512+0000","last_fullsized":"2024-07-26T13:38:46.374512+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:38:39.329366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383168+0000","last_change":"2024-07-26T13:36:59.460133+0000","last_active":"2024-07-26T13:38:46.383168+0000","last_peered":"2024-07-26T13:38:46.383168+0000","last_clean":"2024-07-26T13:38:46.383168+0000","last_became_active":"2024-07-26T13:36:59.459794+0000","last_became_peered":"2024-07-26T13:36:59.459794+0000","last_unstale":"2024-07-26T13:38:46.383168+0000","last_undegraded":"2024-07-26T13:38:46.383168+0000","last_fullsized":"2024-07-26T13:38:46.383168+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:15:36.024347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.386026+0000","last_change":"2024-07-26T13:36:59.462390+0000","last_active":"2024-07-26T13:38:46.386026+0000","last_peered":"2024-07-26T13:38:46.386026+0000","last_clean":"2024-07-26T13:38:46.386026+0000","last_became_active":"2024-07-26T13:36:59.461968+0000","last_became_peered":"2024-07-26T13:36:59.461968+0000","last_unstale":"2024-07-26T13:38:46.386026+0000","last_undegraded":"2024-07-26T13:38:46.386026+0000","last_fullsized":"2024-07-26T13:38:46.386026+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:49:47.142472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387645+0000","last_change":"2024-07-26T13:36:59.460206+0000","last_active":"2024-07-26T13:38:46.387645+0000","last_peered":"2024-07-26T13:38:46.387645+0000","last_clean":"2024-07-26T13:38:46.387645+0000","last_became_active":"2024-07-26T13:36:59.459811+0000","last_became_peered":"2024-07-26T13:36:59.459811+0000","last_unstale":"2024-07-26T13:38:46.387645+0000","last_undegraded":"2024-07-26T13:38:46.387645+0000","last_fullsized":"2024-07-26T13:38:46.387645+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:46:57.112314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387993+0000","last_change":"2024-07-26T13:36:59.460197+0000","last_active":"2024-07-26T13:38:46.387993+0000","last_peered":"2024-07-26T13:38:46.387993+0000","last_clean":"2024-07-26T13:38:46.387993+0000","last_became_active":"2024-07-26T13:36:59.459686+0000","last_became_peered":"2024-07-26T13:36:59.459686+0000","last_unstale":"2024-07-26T13:38:46.387993+0000","last_undegraded":"2024-07-26T13:38:46.387993+0000","last_fullsized":"2024-07-26T13:38:46.387993+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T13:39:22.407376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.373506+0000","last_change":"2024-07-26T13:36:59.462545+0000","last_active":"2024-07-26T13:38:46.373506+0000","last_peered":"2024-07-26T13:38:46.373506+0000","last_clean":"2024-07-26T13:38:46.373506+0000","last_became_active":"2024-07-26T13:36:59.462115+0000","last_became_peered":"2024-07-26T13:36:59.462115+0000","last_unstale":"2024-07-26T13:38:46.373506+0000","last_undegraded":"2024-07-26T13:38:46.373506+0000","last_fullsized":"2024-07-26T13:38:46.373506+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T21:44:13.715076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.384982+0000","last_change":"2024-07-26T13:36:59.465317+0000","last_active":"2024-07-26T13:38:46.384982+0000","last_peered":"2024-07-26T13:38:46.384982+0000","last_clean":"2024-07-26T13:38:46.384982+0000","last_became_active":"2024-07-26T13:36:59.464943+0000","last_became_peered":"2024-07-26T13:36:59.464943+0000","last_unstale":"2024-07-26T13:38:46.384982+0000","last_undegraded":"2024-07-26T13:38:46.384982+0000","last_fullsized":"2024-07-26T13:38:46.384982+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T13:53:54.160584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.370482+0000","last_change":"2024-07-26T13:36:59.462884+0000","last_active":"2024-07-26T13:38:46.370482+0000","last_peered":"2024-07-26T13:38:46.370482+0000","last_clean":"2024-07-26T13:38:46.370482+0000","last_became_active":"2024-07-26T13:36:59.462564+0000","last_became_peered":"2024-07-26T13:36:59.462564+0000","last_unstale":"2024-07-26T13:38:46.370482+0000","last_undegraded":"2024-07-26T13:38:46.370482+0000","last_fullsized":"2024-07-26T13:38:46.370482+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:50:19.549148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387141+0000","last_change":"2024-07-26T13:36:59.468716+0000","last_active":"2024-07-26T13:38:46.387141+0000","last_peered":"2024-07-26T13:38:46.387141+0000","last_clean":"2024-07-26T13:38:46.387141+0000","last_became_active":"2024-07-26T13:36:59.468048+0000","last_became_peered":"2024-07-26T13:36:59.468048+0000","last_unstale":"2024-07-26T13:38:46.387141+0000","last_undegraded":"2024-07-26T13:38:46.387141+0000","last_fullsized":"2024-07-26T13:38:46.387141+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:51:29.510047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383235+0000","last_change":"2024-07-26T13:36:59.466565+0000","last_active":"2024-07-26T13:38:46.383235+0000","last_peered":"2024-07-26T13:38:46.383235+0000","last_clean":"2024-07-26T13:38:46.383235+0000","last_became_active":"2024-07-26T13:36:59.466235+0000","last_became_peered":"2024-07-26T13:36:59.466235+0000","last_unstale":"2024-07-26T13:38:46.383235+0000","last_undegraded":"2024-07-26T13:38:46.383235+0000","last_fullsized":"2024-07-26T13:38:46.383235+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:28:44.326821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.371918+0000","last_change":"2024-07-26T13:36:59.458753+0000","last_active":"2024-07-26T13:38:46.371918+0000","last_peered":"2024-07-26T13:38:46.371918+0000","last_clean":"2024-07-26T13:38:46.371918+0000","last_became_active":"2024-07-26T13:36:59.458333+0000","last_became_peered":"2024-07-26T13:36:59.458333+0000","last_unstale":"2024-07-26T13:38:46.371918+0000","last_undegraded":"2024-07-26T13:38:46.371918+0000","last_fullsized":"2024-07-26T13:38:46.371918+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:33:54.806212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.369463+0000","last_change":"2024-07-26T13:36:59.468657+0000","last_active":"2024-07-26T13:38:46.369463+0000","last_peered":"2024-07-26T13:38:46.369463+0000","last_clean":"2024-07-26T13:38:46.369463+0000","last_became_active":"2024-07-26T13:36:59.468121+0000","last_became_peered":"2024-07-26T13:36:59.468121+0000","last_unstale":"2024-07-26T13:38:46.369463+0000","last_undegraded":"2024-07-26T13:38:46.369463+0000","last_fullsized":"2024-07-26T13:38:46.369463+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:23:53.959836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.377710+0000","last_change":"2024-07-26T13:36:59.469067+0000","last_active":"2024-07-26T13:38:46.377710+0000","last_peered":"2024-07-26T13:38:46.377710+0000","last_clean":"2024-07-26T13:38:46.377710+0000","last_became_active":"2024-07-26T13:36:59.468706+0000","last_became_peered":"2024-07-26T13:36:59.468706+0000","last_unstale":"2024-07-26T13:38:46.377710+0000","last_undegraded":"2024-07-26T13:38:46.377710+0000","last_fullsized":"2024-07-26T13:38:46.377710+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:21:35.974618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.378393+0000","last_change":"2024-07-26T13:36:59.469807+0000","last_active":"2024-07-26T13:38:46.378393+0000","last_peered":"2024-07-26T13:38:46.378393+0000","last_clean":"2024-07-26T13:38:46.378393+0000","last_became_active":"2024-07-26T13:36:59.469020+0000","last_became_peered":"2024-07-26T13:36:59.469020+0000","last_unstale":"2024-07-26T13:38:46.378393+0000","last_undegraded":"2024-07-26T13:38:46.378393+0000","last_fullsized":"2024-07-26T13:38:46.378393+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:29:57.931598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.373015+0000","last_change":"2024-07-26T13:36:59.465236+0000","last_active":"2024-07-26T13:38:46.373015+0000","last_peered":"2024-07-26T13:38:46.373015+0000","last_clean":"2024-07-26T13:38:46.373015+0000","last_became_active":"2024-07-26T13:36:59.464415+0000","last_became_peered":"2024-07-26T13:36:59.464415+0000","last_unstale":"2024-07-26T13:38:46.373015+0000","last_undegraded":"2024-07-26T13:38:46.373015+0000","last_fullsized":"2024-07-26T13:38:46.373015+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:46:20.373688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.375987+0000","last_change":"2024-07-26T13:36:59.466113+0000","last_active":"2024-07-26T13:38:46.375987+0000","last_peered":"2024-07-26T13:38:46.375987+0000","last_clean":"2024-07-26T13:38:46.375987+0000","last_became_active":"2024-07-26T13:36:59.465743+0000","last_became_peered":"2024-07-26T13:36:59.465743+0000","last_unstale":"2024-07-26T13:38:46.375987+0000","last_undegraded":"2024-07-26T13:38:46.375987+0000","last_fullsized":"2024-07-26T13:38:46.375987+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:14:41.839204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.380451+0000","last_change":"2024-07-26T13:36:59.460672+0000","last_active":"2024-07-26T13:38:46.380451+0000","last_peered":"2024-07-26T13:38:46.380451+0000","last_clean":"2024-07-26T13:38:46.380451+0000","last_became_active":"2024-07-26T13:36:59.460247+0000","last_became_peered":"2024-07-26T13:36:59.460247+0000","last_unstale":"2024-07-26T13:38:46.380451+0000","last_undegraded":"2024-07-26T13:38:46.380451+0000","last_fullsized":"2024-07-26T13:38:46.380451+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:37:51.109933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.378083+0000","last_change":"2024-07-26T13:36:59.464784+0000","last_active":"2024-07-26T13:38:46.378083+0000","last_peered":"2024-07-26T13:38:46.378083+0000","last_clean":"2024-07-26T13:38:46.378083+0000","last_became_active":"2024-07-26T13:36:59.464261+0000","last_became_peered":"2024-07-26T13:36:59.464261+0000","last_unstale":"2024-07-26T13:38:46.378083+0000","last_undegraded":"2024-07-26T13:38:46.378083+0000","last_fullsized":"2024-07-26T13:38:46.378083+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:43:10.134037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.385005+0000","last_change":"2024-07-26T13:36:59.460421+0000","last_active":"2024-07-26T13:38:46.385005+0000","last_peered":"2024-07-26T13:38:46.385005+0000","last_clean":"2024-07-26T13:38:46.385005+0000","last_became_active":"2024-07-26T13:36:59.460010+0000","last_became_peered":"2024-07-26T13:36:59.460010+0000","last_unstale":"2024-07-26T13:38:46.385005+0000","last_undegraded":"2024-07-26T13:38:46.385005+0000","last_fullsized":"2024-07-26T13:38:46.385005+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:16:21.651673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.358312+0000","last_change":"2024-07-26T13:36:59.468826+0000","last_active":"2024-07-26T13:38:46.358312+0000","last_peered":"2024-07-26T13:38:46.358312+0000","last_clean":"2024-07-26T13:38:46.358312+0000","last_became_active":"2024-07-26T13:36:59.468406+0000","last_became_peered":"2024-07-26T13:36:59.468406+0000","last_unstale":"2024-07-26T13:38:46.358312+0000","last_undegraded":"2024-07-26T13:38:46.358312+0000","last_fullsized":"2024-07-26T13:38:46.358312+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:22:10.590545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'7","reported_seq":242,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:40:25.872488+0000","last_change":"2024-07-26T13:37:17.559014+0000","last_active":"2024-07-26T13:40:25.872488+0000","last_peered":"2024-07-26T13:40:25.872488+0000","last_clean":"2024-07-26T13:40:25.872488+0000","last_became_active":"2024-07-26T13:37:17.552325+0000","last_became_peered":"2024-07-26T13:37:17.552325+0000","last_unstale":"2024-07-26T13:40:25.872488+0000","last_undegraded":"2024-07-26T13:40:25.872488+0000","last_fullsized":"2024-07-26T13:40:25.872488+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:26:26.790337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":194,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.379560+0000","last_change":"2024-07-26T13:37:17.562871+0000","last_active":"2024-07-26T13:38:46.379560+0000","last_peered":"2024-07-26T13:38:46.379560+0000","last_clean":"2024-07-26T13:38:46.379560+0000","last_became_active":"2024-07-26T13:37:17.562363+0000","last_became_peered":"2024-07-26T13:37:17.562363+0000","last_unstale":"2024-07-26T13:38:46.379560+0000","last_undegraded":"2024-07-26T13:38:46.379560+0000","last_fullsized":"2024-07-26T13:38:46.379560+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:25:36.172091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"79'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.547762+0000","last_change":"2024-07-26T13:36:56.434778+0000","last_active":"2024-07-26T13:37:17.547762+0000","last_peered":"2024-07-26T13:37:17.547762+0000","last_clean":"2024-07-26T13:37:17.547762+0000","last_became_active":"2024-07-26T13:36:56.434288+0000","last_became_peered":"2024-07-26T13:36:56.434288+0000","last_unstale":"2024-07-26T13:37:17.547762+0000","last_undegraded":"2024-07-26T13:37:17.547762+0000","last_fullsized":"2024-07-26T13:37:17.547762+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:24:22.990358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.565163+0000","last_change":"2024-07-26T13:36:56.424665+0000","last_active":"2024-07-26T13:37:17.565163+0000","last_peered":"2024-07-26T13:37:17.565163+0000","last_clean":"2024-07-26T13:37:17.565163+0000","last_became_active":"2024-07-26T13:36:56.424290+0000","last_became_peered":"2024-07-26T13:36:56.424290+0000","last_unstale":"2024-07-26T13:37:17.565163+0000","last_undegraded":"2024-07-26T13:37:17.565163+0000","last_fullsized":"2024-07-26T13:37:17.565163+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:49:59.520194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383766+0000","last_change":"2024-07-26T13:36:59.454609+0000","last_active":"2024-07-26T13:38:46.383766+0000","last_peered":"2024-07-26T13:38:46.383766+0000","last_clean":"2024-07-26T13:38:46.383766+0000","last_became_active":"2024-07-26T13:36:59.454228+0000","last_became_peered":"2024-07-26T13:36:59.454228+0000","last_unstale":"2024-07-26T13:38:46.383766+0000","last_undegraded":"2024-07-26T13:38:46.383766+0000","last_fullsized":"2024-07-26T13:38:46.383766+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:02:06.244433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.547136+0000","last_change":"2024-07-26T13:36:56.430268+0000","last_active":"2024-07-26T13:37:17.547136+0000","last_peered":"2024-07-26T13:37:17.547136+0000","last_clean":"2024-07-26T13:37:17.547136+0000","last_became_active":"2024-07-26T13:36:56.428583+0000","last_became_peered":"2024-07-26T13:36:56.428583+0000","last_unstale":"2024-07-26T13:37:17.547136+0000","last_undegraded":"2024-07-26T13:37:17.547136+0000","last_fullsized":"2024-07-26T13:37:17.547136+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:07:05.864886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.379724+0000","last_change":"2024-07-26T13:36:59.456188+0000","last_active":"2024-07-26T13:38:46.379724+0000","last_peered":"2024-07-26T13:38:46.379724+0000","last_clean":"2024-07-26T13:38:46.379724+0000","last_became_active":"2024-07-26T13:36:59.455842+0000","last_became_peered":"2024-07-26T13:36:59.455842+0000","last_unstale":"2024-07-26T13:38:46.379724+0000","last_undegraded":"2024-07-26T13:38:46.379724+0000","last_fullsized":"2024-07-26T13:38:46.379724+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:00:19.162239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":217,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:40:21.865881+0000","last_change":"2024-07-26T13:36:56.432818+0000","last_active":"2024-07-26T13:40:21.865881+0000","last_peered":"2024-07-26T13:40:21.865881+0000","last_clean":"2024-07-26T13:40:21.865881+0000","last_became_active":"2024-07-26T13:36:56.432419+0000","last_became_peered":"2024-07-26T13:36:56.432419+0000","last_unstale":"2024-07-26T13:40:21.865881+0000","last_undegraded":"2024-07-26T13:40:21.865881+0000","last_fullsized":"2024-07-26T13:40:21.865881+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:55:51.972470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":186,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.386929+0000","last_change":"2024-07-26T13:36:59.460033+0000","last_active":"2024-07-26T13:38:46.386929+0000","last_peered":"2024-07-26T13:38:46.386929+0000","last_clean":"2024-07-26T13:38:46.386929+0000","last_became_active":"2024-07-26T13:36:59.459562+0000","last_became_peered":"2024-07-26T13:36:59.459562+0000","last_unstale":"2024-07-26T13:38:46.386929+0000","last_undegraded":"2024-07-26T13:38:46.386929+0000","last_fullsized":"2024-07-26T13:38:46.386929+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:58:14.155973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551399+0000","last_change":"2024-07-26T13:36:56.432563+0000","last_active":"2024-07-26T13:37:17.551399+0000","last_peered":"2024-07-26T13:37:17.551399+0000","last_clean":"2024-07-26T13:37:17.551399+0000","last_became_active":"2024-07-26T13:36:56.431134+0000","last_became_peered":"2024-07-26T13:36:56.431134+0000","last_unstale":"2024-07-26T13:37:17.551399+0000","last_undegraded":"2024-07-26T13:37:17.551399+0000","last_fullsized":"2024-07-26T13:37:17.551399+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:42:00.653345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.377839+0000","last_change":"2024-07-26T13:36:59.469782+0000","last_active":"2024-07-26T13:38:46.377839+0000","last_peered":"2024-07-26T13:38:46.377839+0000","last_clean":"2024-07-26T13:38:46.377839+0000","last_became_active":"2024-07-26T13:36:59.469412+0000","last_became_peered":"2024-07-26T13:36:59.469412+0000","last_unstale":"2024-07-26T13:38:46.377839+0000","last_undegraded":"2024-07-26T13:38:46.377839+0000","last_fullsized":"2024-07-26T13:38:46.377839+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:28:07.741486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.558844+0000","last_change":"2024-07-26T13:36:56.488981+0000","last_active":"2024-07-26T13:37:17.558844+0000","last_peered":"2024-07-26T13:37:17.558844+0000","last_clean":"2024-07-26T13:37:17.558844+0000","last_became_active":"2024-07-26T13:36:56.488480+0000","last_became_peered":"2024-07-26T13:36:56.488480+0000","last_unstale":"2024-07-26T13:37:17.558844+0000","last_undegraded":"2024-07-26T13:37:17.558844+0000","last_fullsized":"2024-07-26T13:37:17.558844+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:41:10.041007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387010+0000","last_change":"2024-07-26T13:36:59.473183+0000","last_active":"2024-07-26T13:38:46.387010+0000","last_peered":"2024-07-26T13:38:46.387010+0000","last_clean":"2024-07-26T13:38:46.387010+0000","last_became_active":"2024-07-26T13:36:59.472814+0000","last_became_peered":"2024-07-26T13:36:59.472814+0000","last_unstale":"2024-07-26T13:38:46.387010+0000","last_undegraded":"2024-07-26T13:38:46.387010+0000","last_fullsized":"2024-07-26T13:38:46.387010+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:57:03.396384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'35","reported_seq":446,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:40:22.882789+0000","last_change":"2024-07-26T13:36:56.435034+0000","last_active":"2024-07-26T13:40:22.882789+0000","last_peered":"2024-07-26T13:40:22.882789+0000","last_clean":"2024-07-26T13:40:22.882789+0000","last_became_active":"2024-07-26T13:36:56.434611+0000","last_became_peered":"2024-07-26T13:36:56.434611+0000","last_unstale":"2024-07-26T13:40:22.882789+0000","last_undegraded":"2024-07-26T13:40:22.882789+0000","last_fullsized":"2024-07-26T13:40:22.882789+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:18:27.272077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68265,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":384,"num_read_kb":259,"num_write":36,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.382422+0000","last_change":"2024-07-26T13:36:59.465831+0000","last_active":"2024-07-26T13:38:46.382422+0000","last_peered":"2024-07-26T13:38:46.382422+0000","last_clean":"2024-07-26T13:38:46.382422+0000","last_became_active":"2024-07-26T13:36:59.465404+0000","last_became_peered":"2024-07-26T13:36:59.465404+0000","last_unstale":"2024-07-26T13:38:46.382422+0000","last_undegraded":"2024-07-26T13:38:46.382422+0000","last_fullsized":"2024-07-26T13:38:46.382422+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:42:53.795926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'6","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.552497+0000","last_change":"2024-07-26T13:36:56.435170+0000","last_active":"2024-07-26T13:37:17.552497+0000","last_peered":"2024-07-26T13:37:17.552497+0000","last_clean":"2024-07-26T13:37:17.552497+0000","last_became_active":"2024-07-26T13:36:56.434715+0000","last_became_peered":"2024-07-26T13:36:56.434715+0000","last_unstale":"2024-07-26T13:37:17.552497+0000","last_undegraded":"2024-07-26T13:37:17.552497+0000","last_fullsized":"2024-07-26T13:37:17.552497+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:05:09.712510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.370726+0000","last_change":"2024-07-26T13:36:59.461926+0000","last_active":"2024-07-26T13:38:46.370726+0000","last_peered":"2024-07-26T13:38:46.370726+0000","last_clean":"2024-07-26T13:38:46.370726+0000","last_became_active":"2024-07-26T13:36:59.461512+0000","last_became_peered":"2024-07-26T13:36:59.461512+0000","last_unstale":"2024-07-26T13:38:46.370726+0000","last_undegraded":"2024-07-26T13:38:46.370726+0000","last_fullsized":"2024-07-26T13:38:46.370726+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:34:24.405663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.546907+0000","last_change":"2024-07-26T13:36:56.430062+0000","last_active":"2024-07-26T13:37:17.546907+0000","last_peered":"2024-07-26T13:37:17.546907+0000","last_clean":"2024-07-26T13:37:17.546907+0000","last_became_active":"2024-07-26T13:36:56.429654+0000","last_became_peered":"2024-07-26T13:36:56.429654+0000","last_unstale":"2024-07-26T13:37:17.546907+0000","last_undegraded":"2024-07-26T13:37:17.546907+0000","last_fullsized":"2024-07-26T13:37:17.546907+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:21:09.514359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.380005+0000","last_change":"2024-07-26T13:36:59.474459+0000","last_active":"2024-07-26T13:38:46.380005+0000","last_peered":"2024-07-26T13:38:46.380005+0000","last_clean":"2024-07-26T13:38:46.380005+0000","last_became_active":"2024-07-26T13:36:59.474061+0000","last_became_peered":"2024-07-26T13:36:59.474061+0000","last_unstale":"2024-07-26T13:38:46.380005+0000","last_undegraded":"2024-07-26T13:38:46.380005+0000","last_fullsized":"2024-07-26T13:38:46.380005+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:37:21.691491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'5","reported_seq":223,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:40:21.865501+0000","last_change":"2024-07-26T13:36:56.432915+0000","last_active":"2024-07-26T13:40:21.865501+0000","last_peered":"2024-07-26T13:40:21.865501+0000","last_clean":"2024-07-26T13:40:21.865501+0000","last_became_active":"2024-07-26T13:36:56.432499+0000","last_became_peered":"2024-07-26T13:36:56.432499+0000","last_unstale":"2024-07-26T13:40:21.865501+0000","last_undegraded":"2024-07-26T13:40:21.865501+0000","last_fullsized":"2024-07-26T13:40:21.865501+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:02:14.306950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":192,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383574+0000","last_change":"2024-07-26T13:36:59.468945+0000","last_active":"2024-07-26T13:38:46.383574+0000","last_peered":"2024-07-26T13:38:46.383574+0000","last_clean":"2024-07-26T13:38:46.383574+0000","last_became_active":"2024-07-26T13:36:59.467970+0000","last_became_peered":"2024-07-26T13:36:59.467970+0000","last_unstale":"2024-07-26T13:38:46.383574+0000","last_undegraded":"2024-07-26T13:38:46.383574+0000","last_fullsized":"2024-07-26T13:38:46.383574+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T13:59:19.514945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'15","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551412+0000","last_change":"2024-07-26T13:36:56.487014+0000","last_active":"2024-07-26T13:37:17.551412+0000","last_peered":"2024-07-26T13:37:17.551412+0000","last_clean":"2024-07-26T13:37:17.551412+0000","last_became_active":"2024-07-26T13:36:56.486470+0000","last_became_peered":"2024-07-26T13:36:56.486470+0000","last_unstale":"2024-07-26T13:37:17.551412+0000","last_undegraded":"2024-07-26T13:37:17.551412+0000","last_fullsized":"2024-07-26T13:37:17.551412+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:07:02.205644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.366501+0000","last_change":"2024-07-26T13:36:59.462465+0000","last_active":"2024-07-26T13:38:46.366501+0000","last_peered":"2024-07-26T13:38:46.366501+0000","last_clean":"2024-07-26T13:38:46.366501+0000","last_became_active":"2024-07-26T13:36:59.462024+0000","last_became_peered":"2024-07-26T13:36:59.462024+0000","last_unstale":"2024-07-26T13:38:46.366501+0000","last_undegraded":"2024-07-26T13:38:46.366501+0000","last_fullsized":"2024-07-26T13:38:46.366501+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:34:14.317762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551992+0000","last_change":"2024-07-26T13:36:56.435007+0000","last_active":"2024-07-26T13:37:17.551992+0000","last_peered":"2024-07-26T13:37:17.551992+0000","last_clean":"2024-07-26T13:37:17.551992+0000","last_became_active":"2024-07-26T13:36:56.434444+0000","last_became_peered":"2024-07-26T13:36:56.434444+0000","last_unstale":"2024-07-26T13:37:17.551992+0000","last_undegraded":"2024-07-26T13:37:17.551992+0000","last_fullsized":"2024-07-26T13:37:17.551992+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:27:51.041349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.384490+0000","last_change":"2024-07-26T13:36:59.464164+0000","last_active":"2024-07-26T13:38:46.384490+0000","last_peered":"2024-07-26T13:38:46.384490+0000","last_clean":"2024-07-26T13:38:46.384490+0000","last_became_active":"2024-07-26T13:36:59.463782+0000","last_became_peered":"2024-07-26T13:36:59.463782+0000","last_unstale":"2024-07-26T13:38:46.384490+0000","last_undegraded":"2024-07-26T13:38:46.384490+0000","last_fullsized":"2024-07-26T13:38:46.384490+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:48:06.481731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.565510+0000","last_change":"2024-07-26T13:36:56.429653+0000","last_active":"2024-07-26T13:37:17.565510+0000","last_peered":"2024-07-26T13:37:17.565510+0000","last_clean":"2024-07-26T13:37:17.565510+0000","last_became_active":"2024-07-26T13:36:56.429060+0000","last_became_peered":"2024-07-26T13:36:56.429060+0000","last_unstale":"2024-07-26T13:37:17.565510+0000","last_undegraded":"2024-07-26T13:37:17.565510+0000","last_fullsized":"2024-07-26T13:37:17.565510+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:41:48.261709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.384430+0000","last_change":"2024-07-26T13:36:59.471560+0000","last_active":"2024-07-26T13:38:46.384430+0000","last_peered":"2024-07-26T13:38:46.384430+0000","last_clean":"2024-07-26T13:38:46.384430+0000","last_became_active":"2024-07-26T13:36:59.471151+0000","last_became_peered":"2024-07-26T13:36:59.471151+0000","last_unstale":"2024-07-26T13:38:46.384430+0000","last_undegraded":"2024-07-26T13:38:46.384430+0000","last_fullsized":"2024-07-26T13:38:46.384430+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:49:20.513308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'60","reported_seq":118,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:40:22.043007+0000","last_change":"2024-07-26T13:36:56.430010+0000","last_active":"2024-07-26T13:40:22.043007+0000","last_peered":"2024-07-26T13:40:22.043007+0000","last_clean":"2024-07-26T13:40:22.043007+0000","last_became_active":"2024-07-26T13:36:56.429120+0000","last_became_peered":"2024-07-26T13:36:56.429120+0000","last_unstale":"2024-07-26T13:40:22.043007+0000","last_undegraded":"2024-07-26T13:40:22.043007+0000","last_fullsized":"2024-07-26T13:40:22.043007+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:23:43.321975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393846,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":71,"num_write_kb":873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383589+0000","last_change":"2024-07-26T13:36:59.467788+0000","last_active":"2024-07-26T13:38:46.383589+0000","last_peered":"2024-07-26T13:38:46.383589+0000","last_clean":"2024-07-26T13:38:46.383589+0000","last_became_active":"2024-07-26T13:36:59.467399+0000","last_became_peered":"2024-07-26T13:36:59.467399+0000","last_unstale":"2024-07-26T13:38:46.383589+0000","last_undegraded":"2024-07-26T13:38:46.383589+0000","last_fullsized":"2024-07-26T13:38:46.383589+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:48:51.442999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'3","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.553308+0000","last_change":"2024-07-26T13:36:56.588633+0000","last_active":"2024-07-26T13:37:17.553308+0000","last_peered":"2024-07-26T13:37:17.553308+0000","last_clean":"2024-07-26T13:37:17.553308+0000","last_became_active":"2024-07-26T13:36:56.588113+0000","last_became_peered":"2024-07-26T13:36:56.588113+0000","last_unstale":"2024-07-26T13:37:17.553308+0000","last_undegraded":"2024-07-26T13:37:17.553308+0000","last_fullsized":"2024-07-26T13:37:17.553308+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:03:55.075052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387531+0000","last_change":"2024-07-26T13:36:59.472854+0000","last_active":"2024-07-26T13:38:46.387531+0000","last_peered":"2024-07-26T13:38:46.387531+0000","last_clean":"2024-07-26T13:38:46.387531+0000","last_became_active":"2024-07-26T13:36:59.472466+0000","last_became_peered":"2024-07-26T13:36:59.472466+0000","last_unstale":"2024-07-26T13:38:46.387531+0000","last_undegraded":"2024-07-26T13:38:46.387531+0000","last_fullsized":"2024-07-26T13:38:46.387531+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:50:57.454149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":104,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:25.192739+0000","last_change":"2024-07-26T13:35:34.839121+0000","last_active":"2024-07-26T13:38:25.192739+0000","last_peered":"2024-07-26T13:38:25.192739+0000","last_clean":"2024-07-26T13:38:25.192739+0000","last_became_active":"2024-07-26T13:35:34.833465+0000","last_became_peered":"2024-07-26T13:35:34.833465+0000","last_unstale":"2024-07-26T13:38:25.192739+0000","last_undegraded":"2024-07-26T13:38:25.192739+0000","last_fullsized":"2024-07-26T13:38:25.192739+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:32:23.285428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:32:23.285428+0000","last_clean_scrub_stamp":"2024-07-26T13:32:23.285428+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:29:31.650487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'3","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.561459+0000","last_change":"2024-07-26T13:36:56.428648+0000","last_active":"2024-07-26T13:37:17.561459+0000","last_peered":"2024-07-26T13:37:17.561459+0000","last_clean":"2024-07-26T13:37:17.561459+0000","last_became_active":"2024-07-26T13:36:56.427665+0000","last_became_peered":"2024-07-26T13:36:56.427665+0000","last_unstale":"2024-07-26T13:37:17.561459+0000","last_undegraded":"2024-07-26T13:37:17.561459+0000","last_fullsized":"2024-07-26T13:37:17.561459+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:41:39.371687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.376696+0000","last_change":"2024-07-26T13:36:59.460625+0000","last_active":"2024-07-26T13:38:46.376696+0000","last_peered":"2024-07-26T13:38:46.376696+0000","last_clean":"2024-07-26T13:38:46.376696+0000","last_became_active":"2024-07-26T13:36:59.458658+0000","last_became_peered":"2024-07-26T13:36:59.458658+0000","last_unstale":"2024-07-26T13:38:46.376696+0000","last_undegraded":"2024-07-26T13:38:46.376696+0000","last_fullsized":"2024-07-26T13:38:46.376696+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:46:42.812402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'3","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550796+0000","last_change":"2024-07-26T13:36:56.432617+0000","last_active":"2024-07-26T13:37:17.550796+0000","last_peered":"2024-07-26T13:37:17.550796+0000","last_clean":"2024-07-26T13:37:17.550796+0000","last_became_active":"2024-07-26T13:36:56.431519+0000","last_became_peered":"2024-07-26T13:36:56.431519+0000","last_unstale":"2024-07-26T13:37:17.550796+0000","last_undegraded":"2024-07-26T13:37:17.550796+0000","last_fullsized":"2024-07-26T13:37:17.550796+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:32:21.120076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.364664+0000","last_change":"2024-07-26T13:36:59.464084+0000","last_active":"2024-07-26T13:38:46.364664+0000","last_peered":"2024-07-26T13:38:46.364664+0000","last_clean":"2024-07-26T13:38:46.364664+0000","last_became_active":"2024-07-26T13:36:59.463755+0000","last_became_peered":"2024-07-26T13:36:59.463755+0000","last_unstale":"2024-07-26T13:38:46.364664+0000","last_undegraded":"2024-07-26T13:38:46.364664+0000","last_fullsized":"2024-07-26T13:38:46.364664+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:53:20.034188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":223,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:40:24.869696+0000","last_change":"2024-07-26T13:36:56.486071+0000","last_active":"2024-07-26T13:40:24.869696+0000","last_peered":"2024-07-26T13:40:24.869696+0000","last_clean":"2024-07-26T13:40:24.869696+0000","last_became_active":"2024-07-26T13:36:56.485672+0000","last_became_peered":"2024-07-26T13:36:56.485672+0000","last_unstale":"2024-07-26T13:40:24.869696+0000","last_undegraded":"2024-07-26T13:40:24.869696+0000","last_fullsized":"2024-07-26T13:40:24.869696+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T21:29:45.666651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":190,"num_read_kb":190,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.372811+0000","last_change":"2024-07-26T13:37:17.566555+0000","last_active":"2024-07-26T13:38:46.372811+0000","last_peered":"2024-07-26T13:38:46.372811+0000","last_clean":"2024-07-26T13:38:46.372811+0000","last_became_active":"2024-07-26T13:37:17.566040+0000","last_became_peered":"2024-07-26T13:37:17.566040+0000","last_unstale":"2024-07-26T13:38:46.372811+0000","last_undegraded":"2024-07-26T13:38:46.372811+0000","last_fullsized":"2024-07-26T13:38:46.372811+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:48:33.078890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550989+0000","last_change":"2024-07-26T13:36:56.431811+0000","last_active":"2024-07-26T13:37:17.550989+0000","last_peered":"2024-07-26T13:37:17.550989+0000","last_clean":"2024-07-26T13:37:17.550989+0000","last_became_active":"2024-07-26T13:36:56.431223+0000","last_became_peered":"2024-07-26T13:36:56.431223+0000","last_unstale":"2024-07-26T13:37:17.550989+0000","last_undegraded":"2024-07-26T13:37:17.550989+0000","last_fullsized":"2024-07-26T13:37:17.550989+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:08:00.334515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.363881+0000","last_change":"2024-07-26T13:36:59.463972+0000","last_active":"2024-07-26T13:38:46.363881+0000","last_peered":"2024-07-26T13:38:46.363881+0000","last_clean":"2024-07-26T13:38:46.363881+0000","last_became_active":"2024-07-26T13:36:59.463496+0000","last_became_peered":"2024-07-26T13:36:59.463496+0000","last_unstale":"2024-07-26T13:38:46.363881+0000","last_undegraded":"2024-07-26T13:38:46.363881+0000","last_fullsized":"2024-07-26T13:38:46.363881+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:02:06.215372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.545780+0000","last_change":"2024-07-26T13:36:56.486681+0000","last_active":"2024-07-26T13:37:17.545780+0000","last_peered":"2024-07-26T13:37:17.545780+0000","last_clean":"2024-07-26T13:37:17.545780+0000","last_became_active":"2024-07-26T13:36:56.486328+0000","last_became_peered":"2024-07-26T13:36:56.486328+0000","last_unstale":"2024-07-26T13:37:17.545780+0000","last_undegraded":"2024-07-26T13:37:17.545780+0000","last_fullsized":"2024-07-26T13:37:17.545780+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:10:36.557707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.355361+0000","last_change":"2024-07-26T13:36:59.451415+0000","last_active":"2024-07-26T13:38:46.355361+0000","last_peered":"2024-07-26T13:38:46.355361+0000","last_clean":"2024-07-26T13:38:46.355361+0000","last_became_active":"2024-07-26T13:36:59.450992+0000","last_became_peered":"2024-07-26T13:36:59.450992+0000","last_unstale":"2024-07-26T13:38:46.355361+0000","last_undegraded":"2024-07-26T13:38:46.355361+0000","last_fullsized":"2024-07-26T13:38:46.355361+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:29:11.303916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.557777+0000","last_change":"2024-07-26T13:37:17.555643+0000","last_active":"2024-07-26T13:37:17.557777+0000","last_peered":"2024-07-26T13:37:17.557777+0000","last_clean":"2024-07-26T13:37:17.557777+0000","last_became_active":"2024-07-26T13:37:17.549756+0000","last_became_peered":"2024-07-26T13:37:17.549756+0000","last_unstale":"2024-07-26T13:37:17.557777+0000","last_undegraded":"2024-07-26T13:37:17.557777+0000","last_fullsized":"2024-07-26T13:37:17.557777+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:20:16.911286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.368534+0000","last_change":"2024-07-26T13:36:59.465538+0000","last_active":"2024-07-26T13:38:46.368534+0000","last_peered":"2024-07-26T13:38:46.368534+0000","last_clean":"2024-07-26T13:38:46.368534+0000","last_became_active":"2024-07-26T13:36:59.464886+0000","last_became_peered":"2024-07-26T13:36:59.464886+0000","last_unstale":"2024-07-26T13:38:46.368534+0000","last_undegraded":"2024-07-26T13:38:46.368534+0000","last_fullsized":"2024-07-26T13:38:46.368534+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:19:12.273116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'97","reported_seq":213,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:40:22.420742+0000","last_change":"2024-07-26T13:36:56.428355+0000","last_active":"2024-07-26T13:40:22.420742+0000","last_peered":"2024-07-26T13:40:22.420742+0000","last_clean":"2024-07-26T13:40:22.420742+0000","last_became_active":"2024-07-26T13:36:56.427213+0000","last_became_peered":"2024-07-26T13:36:56.427213+0000","last_unstale":"2024-07-26T13:40:22.420742+0000","last_undegraded":"2024-07-26T13:40:22.420742+0000","last_fullsized":"2024-07-26T13:40:22.420742+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:48:09.730067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":179,"num_write_kb":1222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.378498+0000","last_change":"2024-07-26T13:36:59.459378+0000","last_active":"2024-07-26T13:38:46.378498+0000","last_peered":"2024-07-26T13:38:46.378498+0000","last_clean":"2024-07-26T13:38:46.378498+0000","last_became_active":"2024-07-26T13:36:59.458799+0000","last_became_peered":"2024-07-26T13:36:59.458799+0000","last_unstale":"2024-07-26T13:38:46.378498+0000","last_undegraded":"2024-07-26T13:38:46.378498+0000","last_fullsized":"2024-07-26T13:38:46.378498+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:45:19.587800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.561441+0000","last_change":"2024-07-26T13:36:56.588819+0000","last_active":"2024-07-26T13:37:17.561441+0000","last_peered":"2024-07-26T13:37:17.561441+0000","last_clean":"2024-07-26T13:37:17.561441+0000","last_became_active":"2024-07-26T13:36:56.588430+0000","last_became_peered":"2024-07-26T13:36:56.588430+0000","last_unstale":"2024-07-26T13:37:17.561441+0000","last_undegraded":"2024-07-26T13:37:17.561441+0000","last_fullsized":"2024-07-26T13:37:17.561441+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:07:21.387618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.380460+0000","last_change":"2024-07-26T13:37:17.562040+0000","last_active":"2024-07-26T13:38:46.380460+0000","last_peered":"2024-07-26T13:38:46.380460+0000","last_clean":"2024-07-26T13:38:46.380460+0000","last_became_active":"2024-07-26T13:37:17.561542+0000","last_became_peered":"2024-07-26T13:37:17.561542+0000","last_unstale":"2024-07-26T13:38:46.380460+0000","last_undegraded":"2024-07-26T13:38:46.380460+0000","last_fullsized":"2024-07-26T13:38:46.380460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:21:08.161954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.544169+0000","last_change":"2024-07-26T13:36:56.489106+0000","last_active":"2024-07-26T13:37:17.544169+0000","last_peered":"2024-07-26T13:37:17.544169+0000","last_clean":"2024-07-26T13:37:17.544169+0000","last_became_active":"2024-07-26T13:36:56.488706+0000","last_became_peered":"2024-07-26T13:36:56.488706+0000","last_unstale":"2024-07-26T13:37:17.544169+0000","last_undegraded":"2024-07-26T13:37:17.544169+0000","last_fullsized":"2024-07-26T13:37:17.544169+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:00:59.204434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.343705+0000","last_change":"2024-07-26T13:36:59.470412+0000","last_active":"2024-07-26T13:38:46.343705+0000","last_peered":"2024-07-26T13:38:46.343705+0000","last_clean":"2024-07-26T13:38:46.343705+0000","last_became_active":"2024-07-26T13:36:59.470010+0000","last_became_peered":"2024-07-26T13:36:59.470010+0000","last_unstale":"2024-07-26T13:38:46.343705+0000","last_undegraded":"2024-07-26T13:38:46.343705+0000","last_fullsized":"2024-07-26T13:38:46.343705+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:50:57.987600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.543528+0000","last_change":"2024-07-26T13:36:56.428671+0000","last_active":"2024-07-26T13:37:17.543528+0000","last_peered":"2024-07-26T13:37:17.543528+0000","last_clean":"2024-07-26T13:37:17.543528+0000","last_became_active":"2024-07-26T13:36:56.428301+0000","last_became_peered":"2024-07-26T13:36:56.428301+0000","last_unstale":"2024-07-26T13:37:17.543528+0000","last_undegraded":"2024-07-26T13:37:17.543528+0000","last_fullsized":"2024-07-26T13:37:17.543528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:59:19.593664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.385021+0000","last_change":"2024-07-26T13:36:59.458627+0000","last_active":"2024-07-26T13:38:46.385021+0000","last_peered":"2024-07-26T13:38:46.385021+0000","last_clean":"2024-07-26T13:38:46.385021+0000","last_became_active":"2024-07-26T13:36:59.458199+0000","last_became_peered":"2024-07-26T13:36:59.458199+0000","last_unstale":"2024-07-26T13:38:46.385021+0000","last_undegraded":"2024-07-26T13:38:46.385021+0000","last_fullsized":"2024-07-26T13:38:46.385021+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:54:41.833223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551540+0000","last_change":"2024-07-26T13:36:56.431548+0000","last_active":"2024-07-26T13:37:17.551540+0000","last_peered":"2024-07-26T13:37:17.551540+0000","last_clean":"2024-07-26T13:37:17.551540+0000","last_became_active":"2024-07-26T13:36:56.430989+0000","last_became_peered":"2024-07-26T13:36:56.430989+0000","last_unstale":"2024-07-26T13:37:17.551540+0000","last_undegraded":"2024-07-26T13:37:17.551540+0000","last_fullsized":"2024-07-26T13:37:17.551540+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:46:51.655324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.375762+0000","last_change":"2024-07-26T13:36:59.458290+0000","last_active":"2024-07-26T13:38:46.375762+0000","last_peered":"2024-07-26T13:38:46.375762+0000","last_clean":"2024-07-26T13:38:46.375762+0000","last_became_active":"2024-07-26T13:36:59.457942+0000","last_became_peered":"2024-07-26T13:36:59.457942+0000","last_unstale":"2024-07-26T13:38:46.375762+0000","last_undegraded":"2024-07-26T13:38:46.375762+0000","last_fullsized":"2024-07-26T13:38:46.375762+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:51:58.797441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":46,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.568059+0000","last_change":"2024-07-26T13:37:17.567624+0000","last_active":"2024-07-26T13:37:17.568059+0000","last_peered":"2024-07-26T13:37:17.568059+0000","last_clean":"2024-07-26T13:37:17.568059+0000","last_became_active":"2024-07-26T13:37:17.562455+0000","last_became_peered":"2024-07-26T13:37:17.562455+0000","last_unstale":"2024-07-26T13:37:17.568059+0000","last_undegraded":"2024-07-26T13:37:17.568059+0000","last_fullsized":"2024-07-26T13:37:17.568059+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:50:36.418750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387835+0000","last_change":"2024-07-26T13:36:59.474034+0000","last_active":"2024-07-26T13:38:46.387835+0000","last_peered":"2024-07-26T13:38:46.387835+0000","last_clean":"2024-07-26T13:38:46.387835+0000","last_became_active":"2024-07-26T13:36:59.472358+0000","last_became_peered":"2024-07-26T13:36:59.472358+0000","last_unstale":"2024-07-26T13:38:46.387835+0000","last_undegraded":"2024-07-26T13:38:46.387835+0000","last_fullsized":"2024-07-26T13:38:46.387835+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:36:44.565413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.368595+0000","last_change":"2024-07-26T13:36:59.465546+0000","last_active":"2024-07-26T13:38:46.368595+0000","last_peered":"2024-07-26T13:38:46.368595+0000","last_clean":"2024-07-26T13:38:46.368595+0000","last_became_active":"2024-07-26T13:36:59.465137+0000","last_became_peered":"2024-07-26T13:36:59.465137+0000","last_unstale":"2024-07-26T13:38:46.368595+0000","last_undegraded":"2024-07-26T13:38:46.368595+0000","last_fullsized":"2024-07-26T13:38:46.368595+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:35:13.591354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.562664+0000","last_change":"2024-07-26T13:37:17.562012+0000","last_active":"2024-07-26T13:37:17.562664+0000","last_peered":"2024-07-26T13:37:17.562664+0000","last_clean":"2024-07-26T13:37:17.562664+0000","last_became_active":"2024-07-26T13:37:17.550218+0000","last_became_peered":"2024-07-26T13:37:17.550218+0000","last_unstale":"2024-07-26T13:37:17.562664+0000","last_undegraded":"2024-07-26T13:37:17.562664+0000","last_fullsized":"2024-07-26T13:37:17.562664+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:46:16.559533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.375095+0000","last_change":"2024-07-26T13:36:59.456896+0000","last_active":"2024-07-26T13:38:46.375095+0000","last_peered":"2024-07-26T13:38:46.375095+0000","last_clean":"2024-07-26T13:38:46.375095+0000","last_became_active":"2024-07-26T13:36:59.456530+0000","last_became_peered":"2024-07-26T13:36:59.456530+0000","last_unstale":"2024-07-26T13:38:46.375095+0000","last_undegraded":"2024-07-26T13:38:46.375095+0000","last_fullsized":"2024-07-26T13:38:46.375095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:59:56.512501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'4","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.553596+0000","last_change":"2024-07-26T13:36:56.433119+0000","last_active":"2024-07-26T13:37:17.553596+0000","last_peered":"2024-07-26T13:37:17.553596+0000","last_clean":"2024-07-26T13:37:17.553596+0000","last_became_active":"2024-07-26T13:36:56.432765+0000","last_became_peered":"2024-07-26T13:36:56.432765+0000","last_unstale":"2024-07-26T13:37:17.553596+0000","last_undegraded":"2024-07-26T13:37:17.553596+0000","last_fullsized":"2024-07-26T13:37:17.553596+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:31:29.197106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.382051+0000","last_change":"2024-07-26T13:36:59.462125+0000","last_active":"2024-07-26T13:38:46.382051+0000","last_peered":"2024-07-26T13:38:46.382051+0000","last_clean":"2024-07-26T13:38:46.382051+0000","last_became_active":"2024-07-26T13:36:59.461796+0000","last_became_peered":"2024-07-26T13:36:59.461796+0000","last_unstale":"2024-07-26T13:38:46.382051+0000","last_undegraded":"2024-07-26T13:38:46.382051+0000","last_fullsized":"2024-07-26T13:38:46.382051+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:44:10.329046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.543646+0000","last_change":"2024-07-26T13:36:56.487251+0000","last_active":"2024-07-26T13:37:17.543646+0000","last_peered":"2024-07-26T13:37:17.543646+0000","last_clean":"2024-07-26T13:37:17.543646+0000","last_became_active":"2024-07-26T13:36:56.486918+0000","last_became_peered":"2024-07-26T13:36:56.486918+0000","last_unstale":"2024-07-26T13:37:17.543646+0000","last_undegraded":"2024-07-26T13:37:17.543646+0000","last_fullsized":"2024-07-26T13:37:17.543646+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:41:31.248895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.380839+0000","last_change":"2024-07-26T13:36:59.470810+0000","last_active":"2024-07-26T13:38:46.380839+0000","last_peered":"2024-07-26T13:38:46.380839+0000","last_clean":"2024-07-26T13:38:46.380839+0000","last_became_active":"2024-07-26T13:36:59.467371+0000","last_became_peered":"2024-07-26T13:36:59.467371+0000","last_unstale":"2024-07-26T13:38:46.380839+0000","last_undegraded":"2024-07-26T13:38:46.380839+0000","last_fullsized":"2024-07-26T13:38:46.380839+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:23:48.572933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'9","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:39:03.748968+0000","last_change":"2024-07-26T13:36:56.433598+0000","last_active":"2024-07-26T13:39:03.748968+0000","last_peered":"2024-07-26T13:39:03.748968+0000","last_clean":"2024-07-26T13:39:03.748968+0000","last_became_active":"2024-07-26T13:36:56.433325+0000","last_became_peered":"2024-07-26T13:36:56.433325+0000","last_unstale":"2024-07-26T13:39:03.748968+0000","last_undegraded":"2024-07-26T13:39:03.748968+0000","last_fullsized":"2024-07-26T13:39:03.748968+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:17:33.651765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":859931,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1494,"num_read_kb":957,"num_write":403,"num_write_kb":2217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":644,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":675840,"data_stored":2579793,"data_compressed":112140,"data_compressed_allocated":147456,"data_compressed_original":2285568,"omap_allocated":0,"internal_metadata":0},"log_size":295,"ondisk_log_size":295,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247480,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678074,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":443451,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874302,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1093632,"data_stored":1497483,"data_compressed":59728,"data_compressed_allocated":77824,"data_compressed_original":974848,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037827,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":921600,"data_stored":836881,"data_compressed":25108,"data_compressed_allocated":32768,"data_compressed_original":409600,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234055,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":836779,"data_compressed":25108,"data_compressed_allocated":32768,"data_compressed_original":409600,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430284,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":1036188,"data_compressed":41637,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593810,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":443054,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790039,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1011712,"data_stored":1429116,"data_compressed":59728,"data_compressed_allocated":77824,"data_compressed_original":974848,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953563,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":888832,"data_stored":444023,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149791,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":978944,"data_stored":1230357,"data_compressed":43199,"data_compressed_allocated":57344,"data_compressed_original":778240,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378725,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":511840,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542249,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":962560,"data_stored":512266,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705774,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":444325,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34195,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34195,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":69419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":68993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":787276,"data_compressed":37380,"data_compressed_allocated":49152,"data_compressed_original":761856,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":395784,"data_compressed":18091,"data_compressed_allocated":24576,"data_compressed_original":368640,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":464151,"data_compressed":18091,"data_compressed_allocated":24576,"data_compressed_original":368640,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-26T13:40:30.921 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-07-26T13:40:30.921 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph pg dump --format=json 2024-07-26T13:40:31.178 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:40:31.575 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:31 smithi100 ceph-mon[32599]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-26T13:40:31.575 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:31 smithi100 ceph-mon[32599]: pgmap v376: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 174 KiB/s wr, 53 op/s 2024-07-26T13:40:31.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:31 smithi167 ceph-mon[37140]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-26T13:40:31.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:31 smithi167 ceph-mon[37140]: pgmap v376: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 174 KiB/s wr, 53 op/s 2024-07-26T13:40:31.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:31 smithi138 ceph-mon[36954]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-26T13:40:31.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:31 smithi138 ceph-mon[36954]: pgmap v376: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 174 KiB/s wr, 53 op/s 2024-07-26T13:40:32.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.865+0000 7fe58f39e640 1 -- 172.21.15.100:0/3951371094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe588108ee0 msgr2=0x7fe5881092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:32.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.865+0000 7fe58f39e640 1 --2- 172.21.15.100:0/3951371094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe588108ee0 0x7fe5881092f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe578009900 tx=0x7fe57802a410 comp rx=0 tx=0).stop 2024-07-26T13:40:32.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.865+0000 7fe58f39e640 1 -- 172.21.15.100:0/3951371094 shutdown_connections 2024-07-26T13:40:32.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.865+0000 7fe58f39e640 1 --2- 172.21.15.100:0/3951371094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe588109830 0x7fe588110060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:32.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.865+0000 7fe58f39e640 1 --2- 172.21.15.100:0/3951371094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe588108ee0 0x7fe5881092f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:32.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.865+0000 7fe58f39e640 1 --2- 172.21.15.100:0/3951371094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5881035a0 0x7fe5881039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:32.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.865+0000 7fe58f39e640 1 -- 172.21.15.100:0/3951371094 >> 172.21.15.100:0/3951371094 conn(0x7fe5880ff010 msgr2=0x7fe588101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:32.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.865+0000 7fe58f39e640 1 -- 172.21.15.100:0/3951371094 shutdown_connections 2024-07-26T13:40:32.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.865+0000 7fe58f39e640 1 -- 172.21.15.100:0/3951371094 wait complete. 2024-07-26T13:40:32.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.866+0000 7fe58f39e640 1 Processor -- start 2024-07-26T13:40:32.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.866+0000 7fe58f39e640 1 -- start start 2024-07-26T13:40:32.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.866+0000 7fe58f39e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5881035a0 0x7fe588075140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:32.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.866+0000 7fe58f39e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe588108ee0 0x7fe588075680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:32.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.866+0000 7fe58f39e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe588109830 0x7fe588075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:32.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.866+0000 7fe58f39e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe5881134c0 con 0x7fe588108ee0 2024-07-26T13:40:32.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.866+0000 7fe58f39e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe588113340 con 0x7fe5881035a0 2024-07-26T13:40:32.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.866+0000 7fe58f39e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe588113640 con 0x7fe588109830 2024-07-26T13:40:32.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.866+0000 7fe58c939640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe588108ee0 0x7fe588075680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:32.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.866+0000 7fe58c939640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe588108ee0 0x7fe588075680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45020/0 (socket says 172.21.15.100:45020) 2024-07-26T13:40:32.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.866+0000 7fe58c939640 1 -- 172.21.15.100:0/3278664348 learned_addr learned my addr 172.21.15.100:0/3278664348 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:32.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.866+0000 7fe58d93b640 1 --2- 172.21.15.100:0/3278664348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe588109830 0x7fe588075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:32.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.866+0000 7fe58d13a640 1 --2- 172.21.15.100:0/3278664348 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5881035a0 0x7fe588075140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:32.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.866+0000 7fe58c939640 1 -- 172.21.15.100:0/3278664348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe588109830 msgr2=0x7fe588075bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:32.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.866+0000 7fe58c939640 1 --2- 172.21.15.100:0/3278664348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe588109830 0x7fe588075bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:32.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.866+0000 7fe58c939640 1 -- 172.21.15.100:0/3278664348 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5881035a0 msgr2=0x7fe588075140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:32.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.866+0000 7fe58c939640 1 --2- 172.21.15.100:0/3278664348 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5881035a0 0x7fe588075140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:32.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.866+0000 7fe58c939640 1 -- 172.21.15.100:0/3278664348 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5881a8100 con 0x7fe588108ee0 2024-07-26T13:40:32.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.866+0000 7fe58d13a640 1 --2- 172.21.15.100:0/3278664348 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5881035a0 0x7fe588075140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:40:32.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.866+0000 7fe58c939640 1 --2- 172.21.15.100:0/3278664348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe588108ee0 0x7fe588075680 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe578006010 tx=0x7fe578004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:32.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.867+0000 7fe57e7fc640 1 -- 172.21.15.100:0/3278664348 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57803f030 con 0x7fe588108ee0 2024-07-26T13:40:32.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.867+0000 7fe58f39e640 1 -- 172.21.15.100:0/3278664348 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5881a8390 con 0x7fe588108ee0 2024-07-26T13:40:32.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.867+0000 7fe57e7fc640 1 -- 172.21.15.100:0/3278664348 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe578003aa0 con 0x7fe588108ee0 2024-07-26T13:40:32.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.867+0000 7fe57e7fc640 1 -- 172.21.15.100:0/3278664348 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe578035070 con 0x7fe588108ee0 2024-07-26T13:40:32.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.867+0000 7fe58f39e640 1 -- 172.21.15.100:0/3278664348 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5881a8870 con 0x7fe588108ee0 2024-07-26T13:40:32.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.868+0000 7fe58f39e640 1 -- 172.21.15.100:0/3278664348 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5881039b0 con 0x7fe588108ee0 2024-07-26T13:40:32.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.869+0000 7fe57e7fc640 1 -- 172.21.15.100:0/3278664348 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 151134+0+0 (secure 0 0 0) 0x7fe578042030 con 0x7fe588108ee0 2024-07-26T13:40:32.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.870+0000 7fe57e7fc640 1 --2- 172.21.15.100:0/3278664348 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5740b3cb0 0x7fe5740b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:32.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.870+0000 7fe58d13a640 1 --2- 172.21.15.100:0/3278664348 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5740b3cb0 0x7fe5740b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:32.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.870+0000 7fe57e7fc640 1 -- 172.21.15.100:0/3278664348 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fe578104820 con 0x7fe588108ee0 2024-07-26T13:40:32.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.871+0000 7fe58d13a640 1 --2- 172.21.15.100:0/3278664348 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5740b3cb0 0x7fe5740b61c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe588076b90 tx=0x7fe584005db0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:32.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:32.873+0000 7fe57e7fc640 1 -- 172.21.15.100:0/3278664348 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fe5780cbc10 con 0x7fe588108ee0 2024-07-26T13:40:33.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:33.044+0000 7fe58f39e640 1 -- 172.21.15.100:0/3278664348 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe588063230 con 0x7fe5740b3cb0 2024-07-26T13:40:33.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:33.049+0000 7fe57e7fc640 1 -- 172.21.15.100:0/3278664348 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259206 (secure 0 0 0) 0x7fe588063230 con 0x7fe5740b3cb0 2024-07-26T13:40:33.052 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:40:33.053 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-07-26T13:40:33.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:33.051+0000 7fe58f39e640 1 -- 172.21.15.100:0/3278664348 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5740b3cb0 msgr2=0x7fe5740b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:33.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:33.051+0000 7fe58f39e640 1 --2- 172.21.15.100:0/3278664348 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5740b3cb0 0x7fe5740b61c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe588076b90 tx=0x7fe584005db0 comp rx=0 tx=0).stop 2024-07-26T13:40:33.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:33.051+0000 7fe58f39e640 1 -- 172.21.15.100:0/3278664348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe588108ee0 msgr2=0x7fe588075680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:33.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:33.051+0000 7fe58f39e640 1 --2- 172.21.15.100:0/3278664348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe588108ee0 0x7fe588075680 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe578006010 tx=0x7fe578004060 comp rx=0 tx=0).stop 2024-07-26T13:40:33.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:33.051+0000 7fe58f39e640 1 -- 172.21.15.100:0/3278664348 shutdown_connections 2024-07-26T13:40:33.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:33.051+0000 7fe58f39e640 1 --2- 172.21.15.100:0/3278664348 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5740b3cb0 0x7fe5740b61c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:33.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:33.051+0000 7fe58f39e640 1 --2- 172.21.15.100:0/3278664348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe588109830 0x7fe588075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:33.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:33.051+0000 7fe58f39e640 1 --2- 172.21.15.100:0/3278664348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe588108ee0 0x7fe588075680 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:33.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:33.051+0000 7fe58f39e640 1 --2- 172.21.15.100:0/3278664348 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5881035a0 0x7fe588075140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:33.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:33.051+0000 7fe58f39e640 1 -- 172.21.15.100:0/3278664348 >> 172.21.15.100:0/3278664348 conn(0x7fe5880ff010 msgr2=0x7fe588101430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:33.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:33.052+0000 7fe58f39e640 1 -- 172.21.15.100:0/3278664348 shutdown_connections 2024-07-26T13:40:33.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:33.052+0000 7fe58f39e640 1 -- 172.21.15.100:0/3278664348 wait complete. 2024-07-26T13:40:33.666 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:33 smithi100 ceph-mon[32599]: pgmap v377: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 174 KiB/s wr, 52 op/s 2024-07-26T13:40:33.694 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":377,"stamp":"2024-07-26T13:40:32.415531+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1450533,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1605,"num_read_kb":1041,"num_write":528,"num_write_kb":4369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":2952484,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":889,"ondisk_log_size":889,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381064,"kb_used_data":11100,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560760,"statfs":{"total":1151940427776,"available":1151550218240,"internally_reserved":0,"allocated":11366400,"data_stored":9665763,"data_compressed":289422,"data_compressed_allocated":380928,"data_compressed_original":4251648,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":786992,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":405,"num_read_kb":335,"num_write":224,"num_write_kb":2088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003271"},"pg_stats":[{"pgid":"3.3f","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387528+0000","last_change":"2024-07-26T13:36:59.462289+0000","last_active":"2024-07-26T13:38:46.387528+0000","last_peered":"2024-07-26T13:38:46.387528+0000","last_clean":"2024-07-26T13:38:46.387528+0000","last_became_active":"2024-07-26T13:36:59.460311+0000","last_became_peered":"2024-07-26T13:36:59.460311+0000","last_unstale":"2024-07-26T13:38:46.387528+0000","last_undegraded":"2024-07-26T13:38:46.387528+0000","last_fullsized":"2024-07-26T13:38:46.387528+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:04:59.123716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"81'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.384825+0000","last_change":"2024-07-26T13:36:59.463218+0000","last_active":"2024-07-26T13:38:46.384825+0000","last_peered":"2024-07-26T13:38:46.384825+0000","last_clean":"2024-07-26T13:38:46.384825+0000","last_became_active":"2024-07-26T13:36:59.462889+0000","last_became_peered":"2024-07-26T13:36:59.462889+0000","last_unstale":"2024-07-26T13:38:46.384825+0000","last_undegraded":"2024-07-26T13:38:46.384825+0000","last_fullsized":"2024-07-26T13:38:46.384825+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:30:46.788669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383503+0000","last_change":"2024-07-26T13:36:59.456113+0000","last_active":"2024-07-26T13:38:46.383503+0000","last_peered":"2024-07-26T13:38:46.383503+0000","last_clean":"2024-07-26T13:38:46.383503+0000","last_became_active":"2024-07-26T13:36:59.455622+0000","last_became_peered":"2024-07-26T13:36:59.455622+0000","last_unstale":"2024-07-26T13:38:46.383503+0000","last_undegraded":"2024-07-26T13:38:46.383503+0000","last_fullsized":"2024-07-26T13:38:46.383503+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:36:26.474799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"81'8","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.368792+0000","last_change":"2024-07-26T13:37:17.555947+0000","last_active":"2024-07-26T13:38:46.368792+0000","last_peered":"2024-07-26T13:38:46.368792+0000","last_clean":"2024-07-26T13:38:46.368792+0000","last_became_active":"2024-07-26T13:37:17.555607+0000","last_became_peered":"2024-07-26T13:37:17.555607+0000","last_unstale":"2024-07-26T13:38:46.368792+0000","last_undegraded":"2024-07-26T13:38:46.368792+0000","last_fullsized":"2024-07-26T13:38:46.368792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:19:59.603225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"81'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.372136+0000","last_change":"2024-07-26T13:36:59.474896+0000","last_active":"2024-07-26T13:38:46.372136+0000","last_peered":"2024-07-26T13:38:46.372136+0000","last_clean":"2024-07-26T13:38:46.372136+0000","last_became_active":"2024-07-26T13:36:59.474516+0000","last_became_peered":"2024-07-26T13:36:59.474516+0000","last_unstale":"2024-07-26T13:38:46.372136+0000","last_undegraded":"2024-07-26T13:38:46.372136+0000","last_fullsized":"2024-07-26T13:38:46.372136+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:38:18.994280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"81'6","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.358362+0000","last_change":"2024-07-26T13:37:17.556191+0000","last_active":"2024-07-26T13:38:46.358362+0000","last_peered":"2024-07-26T13:38:46.358362+0000","last_clean":"2024-07-26T13:38:46.358362+0000","last_became_active":"2024-07-26T13:37:17.555715+0000","last_became_peered":"2024-07-26T13:37:17.555715+0000","last_unstale":"2024-07-26T13:38:46.358362+0000","last_undegraded":"2024-07-26T13:38:46.358362+0000","last_fullsized":"2024-07-26T13:38:46.358362+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:03:18.352049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383784+0000","last_change":"2024-07-26T13:36:59.469515+0000","last_active":"2024-07-26T13:38:46.383784+0000","last_peered":"2024-07-26T13:38:46.383784+0000","last_clean":"2024-07-26T13:38:46.383784+0000","last_became_active":"2024-07-26T13:36:59.469106+0000","last_became_peered":"2024-07-26T13:36:59.469106+0000","last_unstale":"2024-07-26T13:38:46.383784+0000","last_undegraded":"2024-07-26T13:38:46.383784+0000","last_fullsized":"2024-07-26T13:38:46.383784+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:10:05.455329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"81'9","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.373036+0000","last_change":"2024-07-26T13:37:17.549365+0000","last_active":"2024-07-26T13:38:46.373036+0000","last_peered":"2024-07-26T13:38:46.373036+0000","last_clean":"2024-07-26T13:38:46.373036+0000","last_became_active":"2024-07-26T13:37:17.549036+0000","last_became_peered":"2024-07-26T13:37:17.549036+0000","last_unstale":"2024-07-26T13:38:46.373036+0000","last_undegraded":"2024-07-26T13:38:46.373036+0000","last_fullsized":"2024-07-26T13:38:46.373036+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:27:43.930981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"81'11","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387690+0000","last_change":"2024-07-26T13:36:59.459306+0000","last_active":"2024-07-26T13:38:46.387690+0000","last_peered":"2024-07-26T13:38:46.387690+0000","last_clean":"2024-07-26T13:38:46.387690+0000","last_became_active":"2024-07-26T13:36:59.458714+0000","last_became_peered":"2024-07-26T13:36:59.458714+0000","last_unstale":"2024-07-26T13:38:46.387690+0000","last_undegraded":"2024-07-26T13:38:46.387690+0000","last_fullsized":"2024-07-26T13:38:46.387690+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:34:18.512454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"81'16","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.376748+0000","last_change":"2024-07-26T13:36:59.466556+0000","last_active":"2024-07-26T13:38:46.376748+0000","last_peered":"2024-07-26T13:38:46.376748+0000","last_clean":"2024-07-26T13:38:46.376748+0000","last_became_active":"2024-07-26T13:36:59.466147+0000","last_became_peered":"2024-07-26T13:36:59.466147+0000","last_unstale":"2024-07-26T13:38:46.376748+0000","last_undegraded":"2024-07-26T13:38:46.376748+0000","last_fullsized":"2024-07-26T13:38:46.376748+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:43:11.359314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387571+0000","last_change":"2024-07-26T13:36:59.473685+0000","last_active":"2024-07-26T13:38:46.387571+0000","last_peered":"2024-07-26T13:38:46.387571+0000","last_clean":"2024-07-26T13:38:46.387571+0000","last_became_active":"2024-07-26T13:36:59.473314+0000","last_became_peered":"2024-07-26T13:36:59.473314+0000","last_unstale":"2024-07-26T13:38:46.387571+0000","last_undegraded":"2024-07-26T13:38:46.387571+0000","last_fullsized":"2024-07-26T13:38:46.387571+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:41:51.612348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.368487+0000","last_change":"2024-07-26T13:36:59.471586+0000","last_active":"2024-07-26T13:38:46.368487+0000","last_peered":"2024-07-26T13:38:46.368487+0000","last_clean":"2024-07-26T13:38:46.368487+0000","last_became_active":"2024-07-26T13:36:59.470995+0000","last_became_peered":"2024-07-26T13:36:59.470995+0000","last_unstale":"2024-07-26T13:38:46.368487+0000","last_undegraded":"2024-07-26T13:38:46.368487+0000","last_fullsized":"2024-07-26T13:38:46.368487+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:54:07.957937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"81'10","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.358125+0000","last_change":"2024-07-26T13:36:59.460018+0000","last_active":"2024-07-26T13:38:46.358125+0000","last_peered":"2024-07-26T13:38:46.358125+0000","last_clean":"2024-07-26T13:38:46.358125+0000","last_became_active":"2024-07-26T13:36:59.459671+0000","last_became_peered":"2024-07-26T13:36:59.459671+0000","last_unstale":"2024-07-26T13:38:46.358125+0000","last_undegraded":"2024-07-26T13:38:46.358125+0000","last_fullsized":"2024-07-26T13:38:46.358125+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:19:33.923360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.374512+0000","last_change":"2024-07-26T13:36:59.470271+0000","last_active":"2024-07-26T13:38:46.374512+0000","last_peered":"2024-07-26T13:38:46.374512+0000","last_clean":"2024-07-26T13:38:46.374512+0000","last_became_active":"2024-07-26T13:36:59.469941+0000","last_became_peered":"2024-07-26T13:36:59.469941+0000","last_unstale":"2024-07-26T13:38:46.374512+0000","last_undegraded":"2024-07-26T13:38:46.374512+0000","last_fullsized":"2024-07-26T13:38:46.374512+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:38:39.329366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383168+0000","last_change":"2024-07-26T13:36:59.460133+0000","last_active":"2024-07-26T13:38:46.383168+0000","last_peered":"2024-07-26T13:38:46.383168+0000","last_clean":"2024-07-26T13:38:46.383168+0000","last_became_active":"2024-07-26T13:36:59.459794+0000","last_became_peered":"2024-07-26T13:36:59.459794+0000","last_unstale":"2024-07-26T13:38:46.383168+0000","last_undegraded":"2024-07-26T13:38:46.383168+0000","last_fullsized":"2024-07-26T13:38:46.383168+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:15:36.024347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.386026+0000","last_change":"2024-07-26T13:36:59.462390+0000","last_active":"2024-07-26T13:38:46.386026+0000","last_peered":"2024-07-26T13:38:46.386026+0000","last_clean":"2024-07-26T13:38:46.386026+0000","last_became_active":"2024-07-26T13:36:59.461968+0000","last_became_peered":"2024-07-26T13:36:59.461968+0000","last_unstale":"2024-07-26T13:38:46.386026+0000","last_undegraded":"2024-07-26T13:38:46.386026+0000","last_fullsized":"2024-07-26T13:38:46.386026+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:49:47.142472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387645+0000","last_change":"2024-07-26T13:36:59.460206+0000","last_active":"2024-07-26T13:38:46.387645+0000","last_peered":"2024-07-26T13:38:46.387645+0000","last_clean":"2024-07-26T13:38:46.387645+0000","last_became_active":"2024-07-26T13:36:59.459811+0000","last_became_peered":"2024-07-26T13:36:59.459811+0000","last_unstale":"2024-07-26T13:38:46.387645+0000","last_undegraded":"2024-07-26T13:38:46.387645+0000","last_fullsized":"2024-07-26T13:38:46.387645+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:46:57.112314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"81'10","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387993+0000","last_change":"2024-07-26T13:36:59.460197+0000","last_active":"2024-07-26T13:38:46.387993+0000","last_peered":"2024-07-26T13:38:46.387993+0000","last_clean":"2024-07-26T13:38:46.387993+0000","last_became_active":"2024-07-26T13:36:59.459686+0000","last_became_peered":"2024-07-26T13:36:59.459686+0000","last_unstale":"2024-07-26T13:38:46.387993+0000","last_undegraded":"2024-07-26T13:38:46.387993+0000","last_fullsized":"2024-07-26T13:38:46.387993+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T13:39:22.407376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.373506+0000","last_change":"2024-07-26T13:36:59.462545+0000","last_active":"2024-07-26T13:38:46.373506+0000","last_peered":"2024-07-26T13:38:46.373506+0000","last_clean":"2024-07-26T13:38:46.373506+0000","last_became_active":"2024-07-26T13:36:59.462115+0000","last_became_peered":"2024-07-26T13:36:59.462115+0000","last_unstale":"2024-07-26T13:38:46.373506+0000","last_undegraded":"2024-07-26T13:38:46.373506+0000","last_fullsized":"2024-07-26T13:38:46.373506+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T21:44:13.715076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.384982+0000","last_change":"2024-07-26T13:36:59.465317+0000","last_active":"2024-07-26T13:38:46.384982+0000","last_peered":"2024-07-26T13:38:46.384982+0000","last_clean":"2024-07-26T13:38:46.384982+0000","last_became_active":"2024-07-26T13:36:59.464943+0000","last_became_peered":"2024-07-26T13:36:59.464943+0000","last_unstale":"2024-07-26T13:38:46.384982+0000","last_undegraded":"2024-07-26T13:38:46.384982+0000","last_fullsized":"2024-07-26T13:38:46.384982+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T13:53:54.160584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"81'11","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.370482+0000","last_change":"2024-07-26T13:36:59.462884+0000","last_active":"2024-07-26T13:38:46.370482+0000","last_peered":"2024-07-26T13:38:46.370482+0000","last_clean":"2024-07-26T13:38:46.370482+0000","last_became_active":"2024-07-26T13:36:59.462564+0000","last_became_peered":"2024-07-26T13:36:59.462564+0000","last_unstale":"2024-07-26T13:38:46.370482+0000","last_undegraded":"2024-07-26T13:38:46.370482+0000","last_fullsized":"2024-07-26T13:38:46.370482+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:50:19.549148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"81'10","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387141+0000","last_change":"2024-07-26T13:36:59.468716+0000","last_active":"2024-07-26T13:38:46.387141+0000","last_peered":"2024-07-26T13:38:46.387141+0000","last_clean":"2024-07-26T13:38:46.387141+0000","last_became_active":"2024-07-26T13:36:59.468048+0000","last_became_peered":"2024-07-26T13:36:59.468048+0000","last_unstale":"2024-07-26T13:38:46.387141+0000","last_undegraded":"2024-07-26T13:38:46.387141+0000","last_fullsized":"2024-07-26T13:38:46.387141+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:51:29.510047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383235+0000","last_change":"2024-07-26T13:36:59.466565+0000","last_active":"2024-07-26T13:38:46.383235+0000","last_peered":"2024-07-26T13:38:46.383235+0000","last_clean":"2024-07-26T13:38:46.383235+0000","last_became_active":"2024-07-26T13:36:59.466235+0000","last_became_peered":"2024-07-26T13:36:59.466235+0000","last_unstale":"2024-07-26T13:38:46.383235+0000","last_undegraded":"2024-07-26T13:38:46.383235+0000","last_fullsized":"2024-07-26T13:38:46.383235+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:28:44.326821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.371918+0000","last_change":"2024-07-26T13:36:59.458753+0000","last_active":"2024-07-26T13:38:46.371918+0000","last_peered":"2024-07-26T13:38:46.371918+0000","last_clean":"2024-07-26T13:38:46.371918+0000","last_became_active":"2024-07-26T13:36:59.458333+0000","last_became_peered":"2024-07-26T13:36:59.458333+0000","last_unstale":"2024-07-26T13:38:46.371918+0000","last_undegraded":"2024-07-26T13:38:46.371918+0000","last_fullsized":"2024-07-26T13:38:46.371918+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:33:54.806212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"81'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.369463+0000","last_change":"2024-07-26T13:36:59.468657+0000","last_active":"2024-07-26T13:38:46.369463+0000","last_peered":"2024-07-26T13:38:46.369463+0000","last_clean":"2024-07-26T13:38:46.369463+0000","last_became_active":"2024-07-26T13:36:59.468121+0000","last_became_peered":"2024-07-26T13:36:59.468121+0000","last_unstale":"2024-07-26T13:38:46.369463+0000","last_undegraded":"2024-07-26T13:38:46.369463+0000","last_fullsized":"2024-07-26T13:38:46.369463+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:23:53.959836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"81'15","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.377710+0000","last_change":"2024-07-26T13:36:59.469067+0000","last_active":"2024-07-26T13:38:46.377710+0000","last_peered":"2024-07-26T13:38:46.377710+0000","last_clean":"2024-07-26T13:38:46.377710+0000","last_became_active":"2024-07-26T13:36:59.468706+0000","last_became_peered":"2024-07-26T13:36:59.468706+0000","last_unstale":"2024-07-26T13:38:46.377710+0000","last_undegraded":"2024-07-26T13:38:46.377710+0000","last_fullsized":"2024-07-26T13:38:46.377710+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:21:35.974618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.378393+0000","last_change":"2024-07-26T13:36:59.469807+0000","last_active":"2024-07-26T13:38:46.378393+0000","last_peered":"2024-07-26T13:38:46.378393+0000","last_clean":"2024-07-26T13:38:46.378393+0000","last_became_active":"2024-07-26T13:36:59.469020+0000","last_became_peered":"2024-07-26T13:36:59.469020+0000","last_unstale":"2024-07-26T13:38:46.378393+0000","last_undegraded":"2024-07-26T13:38:46.378393+0000","last_fullsized":"2024-07-26T13:38:46.378393+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:29:57.931598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"81'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.373015+0000","last_change":"2024-07-26T13:36:59.465236+0000","last_active":"2024-07-26T13:38:46.373015+0000","last_peered":"2024-07-26T13:38:46.373015+0000","last_clean":"2024-07-26T13:38:46.373015+0000","last_became_active":"2024-07-26T13:36:59.464415+0000","last_became_peered":"2024-07-26T13:36:59.464415+0000","last_unstale":"2024-07-26T13:38:46.373015+0000","last_undegraded":"2024-07-26T13:38:46.373015+0000","last_fullsized":"2024-07-26T13:38:46.373015+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:46:20.373688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.375987+0000","last_change":"2024-07-26T13:36:59.466113+0000","last_active":"2024-07-26T13:38:46.375987+0000","last_peered":"2024-07-26T13:38:46.375987+0000","last_clean":"2024-07-26T13:38:46.375987+0000","last_became_active":"2024-07-26T13:36:59.465743+0000","last_became_peered":"2024-07-26T13:36:59.465743+0000","last_unstale":"2024-07-26T13:38:46.375987+0000","last_undegraded":"2024-07-26T13:38:46.375987+0000","last_fullsized":"2024-07-26T13:38:46.375987+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:14:41.839204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.380451+0000","last_change":"2024-07-26T13:36:59.460672+0000","last_active":"2024-07-26T13:38:46.380451+0000","last_peered":"2024-07-26T13:38:46.380451+0000","last_clean":"2024-07-26T13:38:46.380451+0000","last_became_active":"2024-07-26T13:36:59.460247+0000","last_became_peered":"2024-07-26T13:36:59.460247+0000","last_unstale":"2024-07-26T13:38:46.380451+0000","last_undegraded":"2024-07-26T13:38:46.380451+0000","last_fullsized":"2024-07-26T13:38:46.380451+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:37:51.109933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"81'12","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.378083+0000","last_change":"2024-07-26T13:36:59.464784+0000","last_active":"2024-07-26T13:38:46.378083+0000","last_peered":"2024-07-26T13:38:46.378083+0000","last_clean":"2024-07-26T13:38:46.378083+0000","last_became_active":"2024-07-26T13:36:59.464261+0000","last_became_peered":"2024-07-26T13:36:59.464261+0000","last_unstale":"2024-07-26T13:38:46.378083+0000","last_undegraded":"2024-07-26T13:38:46.378083+0000","last_fullsized":"2024-07-26T13:38:46.378083+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:43:10.134037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.385005+0000","last_change":"2024-07-26T13:36:59.460421+0000","last_active":"2024-07-26T13:38:46.385005+0000","last_peered":"2024-07-26T13:38:46.385005+0000","last_clean":"2024-07-26T13:38:46.385005+0000","last_became_active":"2024-07-26T13:36:59.460010+0000","last_became_peered":"2024-07-26T13:36:59.460010+0000","last_unstale":"2024-07-26T13:38:46.385005+0000","last_undegraded":"2024-07-26T13:38:46.385005+0000","last_fullsized":"2024-07-26T13:38:46.385005+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:16:21.651673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.358312+0000","last_change":"2024-07-26T13:36:59.468826+0000","last_active":"2024-07-26T13:38:46.358312+0000","last_peered":"2024-07-26T13:38:46.358312+0000","last_clean":"2024-07-26T13:38:46.358312+0000","last_became_active":"2024-07-26T13:36:59.468406+0000","last_became_peered":"2024-07-26T13:36:59.468406+0000","last_unstale":"2024-07-26T13:38:46.358312+0000","last_undegraded":"2024-07-26T13:38:46.358312+0000","last_fullsized":"2024-07-26T13:38:46.358312+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:22:10.590545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'7","reported_seq":247,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:40:30.880711+0000","last_change":"2024-07-26T13:37:17.559014+0000","last_active":"2024-07-26T13:40:30.880711+0000","last_peered":"2024-07-26T13:40:30.880711+0000","last_clean":"2024-07-26T13:40:30.880711+0000","last_became_active":"2024-07-26T13:37:17.552325+0000","last_became_peered":"2024-07-26T13:37:17.552325+0000","last_unstale":"2024-07-26T13:40:30.880711+0000","last_undegraded":"2024-07-26T13:40:30.880711+0000","last_fullsized":"2024-07-26T13:40:30.880711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:26:26.790337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":199,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"81'8","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.379560+0000","last_change":"2024-07-26T13:37:17.562871+0000","last_active":"2024-07-26T13:38:46.379560+0000","last_peered":"2024-07-26T13:38:46.379560+0000","last_clean":"2024-07-26T13:38:46.379560+0000","last_became_active":"2024-07-26T13:37:17.562363+0000","last_became_peered":"2024-07-26T13:37:17.562363+0000","last_unstale":"2024-07-26T13:38:46.379560+0000","last_undegraded":"2024-07-26T13:38:46.379560+0000","last_fullsized":"2024-07-26T13:38:46.379560+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:25:36.172091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"79'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.547762+0000","last_change":"2024-07-26T13:36:56.434778+0000","last_active":"2024-07-26T13:37:17.547762+0000","last_peered":"2024-07-26T13:37:17.547762+0000","last_clean":"2024-07-26T13:37:17.547762+0000","last_became_active":"2024-07-26T13:36:56.434288+0000","last_became_peered":"2024-07-26T13:36:56.434288+0000","last_unstale":"2024-07-26T13:37:17.547762+0000","last_undegraded":"2024-07-26T13:37:17.547762+0000","last_fullsized":"2024-07-26T13:37:17.547762+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:24:22.990358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"79'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.565163+0000","last_change":"2024-07-26T13:36:56.424665+0000","last_active":"2024-07-26T13:37:17.565163+0000","last_peered":"2024-07-26T13:37:17.565163+0000","last_clean":"2024-07-26T13:37:17.565163+0000","last_became_active":"2024-07-26T13:36:56.424290+0000","last_became_peered":"2024-07-26T13:36:56.424290+0000","last_unstale":"2024-07-26T13:37:17.565163+0000","last_undegraded":"2024-07-26T13:37:17.565163+0000","last_fullsized":"2024-07-26T13:37:17.565163+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:49:59.520194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383766+0000","last_change":"2024-07-26T13:36:59.454609+0000","last_active":"2024-07-26T13:38:46.383766+0000","last_peered":"2024-07-26T13:38:46.383766+0000","last_clean":"2024-07-26T13:38:46.383766+0000","last_became_active":"2024-07-26T13:36:59.454228+0000","last_became_peered":"2024-07-26T13:36:59.454228+0000","last_unstale":"2024-07-26T13:38:46.383766+0000","last_undegraded":"2024-07-26T13:38:46.383766+0000","last_fullsized":"2024-07-26T13:38:46.383766+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:02:06.244433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"79'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.547136+0000","last_change":"2024-07-26T13:36:56.430268+0000","last_active":"2024-07-26T13:37:17.547136+0000","last_peered":"2024-07-26T13:37:17.547136+0000","last_clean":"2024-07-26T13:37:17.547136+0000","last_became_active":"2024-07-26T13:36:56.428583+0000","last_became_peered":"2024-07-26T13:36:56.428583+0000","last_unstale":"2024-07-26T13:37:17.547136+0000","last_undegraded":"2024-07-26T13:37:17.547136+0000","last_fullsized":"2024-07-26T13:37:17.547136+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:07:05.864886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"81'11","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.379724+0000","last_change":"2024-07-26T13:36:59.456188+0000","last_active":"2024-07-26T13:38:46.379724+0000","last_peered":"2024-07-26T13:38:46.379724+0000","last_clean":"2024-07-26T13:38:46.379724+0000","last_became_active":"2024-07-26T13:36:59.455842+0000","last_became_peered":"2024-07-26T13:36:59.455842+0000","last_unstale":"2024-07-26T13:38:46.379724+0000","last_undegraded":"2024-07-26T13:38:46.379724+0000","last_fullsized":"2024-07-26T13:38:46.379724+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:00:19.162239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":222,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:40:26.873335+0000","last_change":"2024-07-26T13:36:56.432818+0000","last_active":"2024-07-26T13:40:26.873335+0000","last_peered":"2024-07-26T13:40:26.873335+0000","last_clean":"2024-07-26T13:40:26.873335+0000","last_became_active":"2024-07-26T13:36:56.432419+0000","last_became_peered":"2024-07-26T13:36:56.432419+0000","last_unstale":"2024-07-26T13:40:26.873335+0000","last_undegraded":"2024-07-26T13:40:26.873335+0000","last_fullsized":"2024-07-26T13:40:26.873335+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:55:51.972470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":191,"num_read_kb":191,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"81'10","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.386929+0000","last_change":"2024-07-26T13:36:59.460033+0000","last_active":"2024-07-26T13:38:46.386929+0000","last_peered":"2024-07-26T13:38:46.386929+0000","last_clean":"2024-07-26T13:38:46.386929+0000","last_became_active":"2024-07-26T13:36:59.459562+0000","last_became_peered":"2024-07-26T13:36:59.459562+0000","last_unstale":"2024-07-26T13:38:46.386929+0000","last_undegraded":"2024-07-26T13:38:46.386929+0000","last_fullsized":"2024-07-26T13:38:46.386929+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:58:14.155973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"79'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551399+0000","last_change":"2024-07-26T13:36:56.432563+0000","last_active":"2024-07-26T13:37:17.551399+0000","last_peered":"2024-07-26T13:37:17.551399+0000","last_clean":"2024-07-26T13:37:17.551399+0000","last_became_active":"2024-07-26T13:36:56.431134+0000","last_became_peered":"2024-07-26T13:36:56.431134+0000","last_unstale":"2024-07-26T13:37:17.551399+0000","last_undegraded":"2024-07-26T13:37:17.551399+0000","last_fullsized":"2024-07-26T13:37:17.551399+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:42:00.653345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.377839+0000","last_change":"2024-07-26T13:36:59.469782+0000","last_active":"2024-07-26T13:38:46.377839+0000","last_peered":"2024-07-26T13:38:46.377839+0000","last_clean":"2024-07-26T13:38:46.377839+0000","last_became_active":"2024-07-26T13:36:59.469412+0000","last_became_peered":"2024-07-26T13:36:59.469412+0000","last_unstale":"2024-07-26T13:38:46.377839+0000","last_undegraded":"2024-07-26T13:38:46.377839+0000","last_fullsized":"2024-07-26T13:38:46.377839+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:28:07.741486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'4","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.558844+0000","last_change":"2024-07-26T13:36:56.488981+0000","last_active":"2024-07-26T13:37:17.558844+0000","last_peered":"2024-07-26T13:37:17.558844+0000","last_clean":"2024-07-26T13:37:17.558844+0000","last_became_active":"2024-07-26T13:36:56.488480+0000","last_became_peered":"2024-07-26T13:36:56.488480+0000","last_unstale":"2024-07-26T13:37:17.558844+0000","last_undegraded":"2024-07-26T13:37:17.558844+0000","last_fullsized":"2024-07-26T13:37:17.558844+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:41:10.041007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"81'11","reported_seq":33,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387010+0000","last_change":"2024-07-26T13:36:59.473183+0000","last_active":"2024-07-26T13:38:46.387010+0000","last_peered":"2024-07-26T13:38:46.387010+0000","last_clean":"2024-07-26T13:38:46.387010+0000","last_became_active":"2024-07-26T13:36:59.472814+0000","last_became_peered":"2024-07-26T13:36:59.472814+0000","last_unstale":"2024-07-26T13:38:46.387010+0000","last_undegraded":"2024-07-26T13:38:46.387010+0000","last_fullsized":"2024-07-26T13:38:46.387010+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:57:03.396384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'35","reported_seq":456,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:40:27.890457+0000","last_change":"2024-07-26T13:36:56.435034+0000","last_active":"2024-07-26T13:40:27.890457+0000","last_peered":"2024-07-26T13:40:27.890457+0000","last_clean":"2024-07-26T13:40:27.890457+0000","last_became_active":"2024-07-26T13:36:56.434611+0000","last_became_peered":"2024-07-26T13:36:56.434611+0000","last_unstale":"2024-07-26T13:40:27.890457+0000","last_undegraded":"2024-07-26T13:40:27.890457+0000","last_fullsized":"2024-07-26T13:40:27.890457+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:18:27.272077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68265,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":394,"num_read_kb":264,"num_write":36,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.382422+0000","last_change":"2024-07-26T13:36:59.465831+0000","last_active":"2024-07-26T13:38:46.382422+0000","last_peered":"2024-07-26T13:38:46.382422+0000","last_clean":"2024-07-26T13:38:46.382422+0000","last_became_active":"2024-07-26T13:36:59.465404+0000","last_became_peered":"2024-07-26T13:36:59.465404+0000","last_unstale":"2024-07-26T13:38:46.382422+0000","last_undegraded":"2024-07-26T13:38:46.382422+0000","last_fullsized":"2024-07-26T13:38:46.382422+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:42:53.795926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'6","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.552497+0000","last_change":"2024-07-26T13:36:56.435170+0000","last_active":"2024-07-26T13:37:17.552497+0000","last_peered":"2024-07-26T13:37:17.552497+0000","last_clean":"2024-07-26T13:37:17.552497+0000","last_became_active":"2024-07-26T13:36:56.434715+0000","last_became_peered":"2024-07-26T13:36:56.434715+0000","last_unstale":"2024-07-26T13:37:17.552497+0000","last_undegraded":"2024-07-26T13:37:17.552497+0000","last_fullsized":"2024-07-26T13:37:17.552497+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:05:09.712510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.370726+0000","last_change":"2024-07-26T13:36:59.461926+0000","last_active":"2024-07-26T13:38:46.370726+0000","last_peered":"2024-07-26T13:38:46.370726+0000","last_clean":"2024-07-26T13:38:46.370726+0000","last_became_active":"2024-07-26T13:36:59.461512+0000","last_became_peered":"2024-07-26T13:36:59.461512+0000","last_unstale":"2024-07-26T13:38:46.370726+0000","last_undegraded":"2024-07-26T13:38:46.370726+0000","last_fullsized":"2024-07-26T13:38:46.370726+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:34:24.405663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"79'3","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.546907+0000","last_change":"2024-07-26T13:36:56.430062+0000","last_active":"2024-07-26T13:37:17.546907+0000","last_peered":"2024-07-26T13:37:17.546907+0000","last_clean":"2024-07-26T13:37:17.546907+0000","last_became_active":"2024-07-26T13:36:56.429654+0000","last_became_peered":"2024-07-26T13:36:56.429654+0000","last_unstale":"2024-07-26T13:37:17.546907+0000","last_undegraded":"2024-07-26T13:37:17.546907+0000","last_fullsized":"2024-07-26T13:37:17.546907+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:21:09.514359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"81'15","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.380005+0000","last_change":"2024-07-26T13:36:59.474459+0000","last_active":"2024-07-26T13:38:46.380005+0000","last_peered":"2024-07-26T13:38:46.380005+0000","last_clean":"2024-07-26T13:38:46.380005+0000","last_became_active":"2024-07-26T13:36:59.474061+0000","last_became_peered":"2024-07-26T13:36:59.474061+0000","last_unstale":"2024-07-26T13:38:46.380005+0000","last_undegraded":"2024-07-26T13:38:46.380005+0000","last_fullsized":"2024-07-26T13:38:46.380005+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:37:21.691491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'5","reported_seq":228,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:40:26.873436+0000","last_change":"2024-07-26T13:36:56.432915+0000","last_active":"2024-07-26T13:40:26.873436+0000","last_peered":"2024-07-26T13:40:26.873436+0000","last_clean":"2024-07-26T13:40:26.873436+0000","last_became_active":"2024-07-26T13:36:56.432499+0000","last_became_peered":"2024-07-26T13:36:56.432499+0000","last_unstale":"2024-07-26T13:40:26.873436+0000","last_undegraded":"2024-07-26T13:40:26.873436+0000","last_fullsized":"2024-07-26T13:40:26.873436+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:02:14.306950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":197,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383574+0000","last_change":"2024-07-26T13:36:59.468945+0000","last_active":"2024-07-26T13:38:46.383574+0000","last_peered":"2024-07-26T13:38:46.383574+0000","last_clean":"2024-07-26T13:38:46.383574+0000","last_became_active":"2024-07-26T13:36:59.467970+0000","last_became_peered":"2024-07-26T13:36:59.467970+0000","last_unstale":"2024-07-26T13:38:46.383574+0000","last_undegraded":"2024-07-26T13:38:46.383574+0000","last_fullsized":"2024-07-26T13:38:46.383574+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T13:59:19.514945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'15","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551412+0000","last_change":"2024-07-26T13:36:56.487014+0000","last_active":"2024-07-26T13:37:17.551412+0000","last_peered":"2024-07-26T13:37:17.551412+0000","last_clean":"2024-07-26T13:37:17.551412+0000","last_became_active":"2024-07-26T13:36:56.486470+0000","last_became_peered":"2024-07-26T13:36:56.486470+0000","last_unstale":"2024-07-26T13:37:17.551412+0000","last_undegraded":"2024-07-26T13:37:17.551412+0000","last_fullsized":"2024-07-26T13:37:17.551412+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:07:02.205644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.366501+0000","last_change":"2024-07-26T13:36:59.462465+0000","last_active":"2024-07-26T13:38:46.366501+0000","last_peered":"2024-07-26T13:38:46.366501+0000","last_clean":"2024-07-26T13:38:46.366501+0000","last_became_active":"2024-07-26T13:36:59.462024+0000","last_became_peered":"2024-07-26T13:36:59.462024+0000","last_unstale":"2024-07-26T13:38:46.366501+0000","last_undegraded":"2024-07-26T13:38:46.366501+0000","last_fullsized":"2024-07-26T13:38:46.366501+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:34:14.317762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'2","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551992+0000","last_change":"2024-07-26T13:36:56.435007+0000","last_active":"2024-07-26T13:37:17.551992+0000","last_peered":"2024-07-26T13:37:17.551992+0000","last_clean":"2024-07-26T13:37:17.551992+0000","last_became_active":"2024-07-26T13:36:56.434444+0000","last_became_peered":"2024-07-26T13:36:56.434444+0000","last_unstale":"2024-07-26T13:37:17.551992+0000","last_undegraded":"2024-07-26T13:37:17.551992+0000","last_fullsized":"2024-07-26T13:37:17.551992+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:27:51.041349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.384490+0000","last_change":"2024-07-26T13:36:59.464164+0000","last_active":"2024-07-26T13:38:46.384490+0000","last_peered":"2024-07-26T13:38:46.384490+0000","last_clean":"2024-07-26T13:38:46.384490+0000","last_became_active":"2024-07-26T13:36:59.463782+0000","last_became_peered":"2024-07-26T13:36:59.463782+0000","last_unstale":"2024-07-26T13:38:46.384490+0000","last_undegraded":"2024-07-26T13:38:46.384490+0000","last_fullsized":"2024-07-26T13:38:46.384490+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:48:06.481731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.565510+0000","last_change":"2024-07-26T13:36:56.429653+0000","last_active":"2024-07-26T13:37:17.565510+0000","last_peered":"2024-07-26T13:37:17.565510+0000","last_clean":"2024-07-26T13:37:17.565510+0000","last_became_active":"2024-07-26T13:36:56.429060+0000","last_became_peered":"2024-07-26T13:36:56.429060+0000","last_unstale":"2024-07-26T13:37:17.565510+0000","last_undegraded":"2024-07-26T13:37:17.565510+0000","last_fullsized":"2024-07-26T13:37:17.565510+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:41:48.261709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.384430+0000","last_change":"2024-07-26T13:36:59.471560+0000","last_active":"2024-07-26T13:38:46.384430+0000","last_peered":"2024-07-26T13:38:46.384430+0000","last_clean":"2024-07-26T13:38:46.384430+0000","last_became_active":"2024-07-26T13:36:59.471151+0000","last_became_peered":"2024-07-26T13:36:59.471151+0000","last_unstale":"2024-07-26T13:38:46.384430+0000","last_undegraded":"2024-07-26T13:38:46.384430+0000","last_fullsized":"2024-07-26T13:38:46.384430+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:49:20.513308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'60","reported_seq":118,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:40:22.043007+0000","last_change":"2024-07-26T13:36:56.430010+0000","last_active":"2024-07-26T13:40:22.043007+0000","last_peered":"2024-07-26T13:40:22.043007+0000","last_clean":"2024-07-26T13:40:22.043007+0000","last_became_active":"2024-07-26T13:36:56.429120+0000","last_became_peered":"2024-07-26T13:36:56.429120+0000","last_unstale":"2024-07-26T13:40:22.043007+0000","last_undegraded":"2024-07-26T13:40:22.043007+0000","last_fullsized":"2024-07-26T13:40:22.043007+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:23:43.321975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393846,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":71,"num_write_kb":873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.383589+0000","last_change":"2024-07-26T13:36:59.467788+0000","last_active":"2024-07-26T13:38:46.383589+0000","last_peered":"2024-07-26T13:38:46.383589+0000","last_clean":"2024-07-26T13:38:46.383589+0000","last_became_active":"2024-07-26T13:36:59.467399+0000","last_became_peered":"2024-07-26T13:36:59.467399+0000","last_unstale":"2024-07-26T13:38:46.383589+0000","last_undegraded":"2024-07-26T13:38:46.383589+0000","last_fullsized":"2024-07-26T13:38:46.383589+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:48:51.442999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"79'3","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.553308+0000","last_change":"2024-07-26T13:36:56.588633+0000","last_active":"2024-07-26T13:37:17.553308+0000","last_peered":"2024-07-26T13:37:17.553308+0000","last_clean":"2024-07-26T13:37:17.553308+0000","last_became_active":"2024-07-26T13:36:56.588113+0000","last_became_peered":"2024-07-26T13:36:56.588113+0000","last_unstale":"2024-07-26T13:37:17.553308+0000","last_undegraded":"2024-07-26T13:37:17.553308+0000","last_fullsized":"2024-07-26T13:37:17.553308+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:03:55.075052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387531+0000","last_change":"2024-07-26T13:36:59.472854+0000","last_active":"2024-07-26T13:38:46.387531+0000","last_peered":"2024-07-26T13:38:46.387531+0000","last_clean":"2024-07-26T13:38:46.387531+0000","last_became_active":"2024-07-26T13:36:59.472466+0000","last_became_peered":"2024-07-26T13:36:59.472466+0000","last_unstale":"2024-07-26T13:38:46.387531+0000","last_undegraded":"2024-07-26T13:38:46.387531+0000","last_fullsized":"2024-07-26T13:38:46.387531+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:50:57.454149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":104,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:25.192739+0000","last_change":"2024-07-26T13:35:34.839121+0000","last_active":"2024-07-26T13:38:25.192739+0000","last_peered":"2024-07-26T13:38:25.192739+0000","last_clean":"2024-07-26T13:38:25.192739+0000","last_became_active":"2024-07-26T13:35:34.833465+0000","last_became_peered":"2024-07-26T13:35:34.833465+0000","last_unstale":"2024-07-26T13:38:25.192739+0000","last_undegraded":"2024-07-26T13:38:25.192739+0000","last_fullsized":"2024-07-26T13:38:25.192739+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:32:23.285428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:32:23.285428+0000","last_clean_scrub_stamp":"2024-07-26T13:32:23.285428+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:29:31.650487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"79'3","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.561459+0000","last_change":"2024-07-26T13:36:56.428648+0000","last_active":"2024-07-26T13:37:17.561459+0000","last_peered":"2024-07-26T13:37:17.561459+0000","last_clean":"2024-07-26T13:37:17.561459+0000","last_became_active":"2024-07-26T13:36:56.427665+0000","last_became_peered":"2024-07-26T13:36:56.427665+0000","last_unstale":"2024-07-26T13:37:17.561459+0000","last_undegraded":"2024-07-26T13:37:17.561459+0000","last_fullsized":"2024-07-26T13:37:17.561459+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:41:39.371687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.376696+0000","last_change":"2024-07-26T13:36:59.460625+0000","last_active":"2024-07-26T13:38:46.376696+0000","last_peered":"2024-07-26T13:38:46.376696+0000","last_clean":"2024-07-26T13:38:46.376696+0000","last_became_active":"2024-07-26T13:36:59.458658+0000","last_became_peered":"2024-07-26T13:36:59.458658+0000","last_unstale":"2024-07-26T13:38:46.376696+0000","last_undegraded":"2024-07-26T13:38:46.376696+0000","last_fullsized":"2024-07-26T13:38:46.376696+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:46:42.812402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"79'3","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550796+0000","last_change":"2024-07-26T13:36:56.432617+0000","last_active":"2024-07-26T13:37:17.550796+0000","last_peered":"2024-07-26T13:37:17.550796+0000","last_clean":"2024-07-26T13:37:17.550796+0000","last_became_active":"2024-07-26T13:36:56.431519+0000","last_became_peered":"2024-07-26T13:36:56.431519+0000","last_unstale":"2024-07-26T13:37:17.550796+0000","last_undegraded":"2024-07-26T13:37:17.550796+0000","last_fullsized":"2024-07-26T13:37:17.550796+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:32:21.120076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.364664+0000","last_change":"2024-07-26T13:36:59.464084+0000","last_active":"2024-07-26T13:38:46.364664+0000","last_peered":"2024-07-26T13:38:46.364664+0000","last_clean":"2024-07-26T13:38:46.364664+0000","last_became_active":"2024-07-26T13:36:59.463755+0000","last_became_peered":"2024-07-26T13:36:59.463755+0000","last_unstale":"2024-07-26T13:38:46.364664+0000","last_undegraded":"2024-07-26T13:38:46.364664+0000","last_fullsized":"2024-07-26T13:38:46.364664+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:53:20.034188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":228,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:40:29.877634+0000","last_change":"2024-07-26T13:36:56.486071+0000","last_active":"2024-07-26T13:40:29.877634+0000","last_peered":"2024-07-26T13:40:29.877634+0000","last_clean":"2024-07-26T13:40:29.877634+0000","last_became_active":"2024-07-26T13:36:56.485672+0000","last_became_peered":"2024-07-26T13:36:56.485672+0000","last_unstale":"2024-07-26T13:40:29.877634+0000","last_undegraded":"2024-07-26T13:40:29.877634+0000","last_fullsized":"2024-07-26T13:40:29.877634+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T21:29:45.666651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":195,"num_read_kb":195,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"81'8","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.372811+0000","last_change":"2024-07-26T13:37:17.566555+0000","last_active":"2024-07-26T13:38:46.372811+0000","last_peered":"2024-07-26T13:38:46.372811+0000","last_clean":"2024-07-26T13:38:46.372811+0000","last_became_active":"2024-07-26T13:37:17.566040+0000","last_became_peered":"2024-07-26T13:37:17.566040+0000","last_unstale":"2024-07-26T13:38:46.372811+0000","last_undegraded":"2024-07-26T13:38:46.372811+0000","last_fullsized":"2024-07-26T13:38:46.372811+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:48:33.078890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"79'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.550989+0000","last_change":"2024-07-26T13:36:56.431811+0000","last_active":"2024-07-26T13:37:17.550989+0000","last_peered":"2024-07-26T13:37:17.550989+0000","last_clean":"2024-07-26T13:37:17.550989+0000","last_became_active":"2024-07-26T13:36:56.431223+0000","last_became_peered":"2024-07-26T13:36:56.431223+0000","last_unstale":"2024-07-26T13:37:17.550989+0000","last_undegraded":"2024-07-26T13:37:17.550989+0000","last_fullsized":"2024-07-26T13:37:17.550989+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:08:00.334515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"81'7","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.363881+0000","last_change":"2024-07-26T13:36:59.463972+0000","last_active":"2024-07-26T13:38:46.363881+0000","last_peered":"2024-07-26T13:38:46.363881+0000","last_clean":"2024-07-26T13:38:46.363881+0000","last_became_active":"2024-07-26T13:36:59.463496+0000","last_became_peered":"2024-07-26T13:36:59.463496+0000","last_unstale":"2024-07-26T13:38:46.363881+0000","last_undegraded":"2024-07-26T13:38:46.363881+0000","last_fullsized":"2024-07-26T13:38:46.363881+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T00:02:06.215372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.545780+0000","last_change":"2024-07-26T13:36:56.486681+0000","last_active":"2024-07-26T13:37:17.545780+0000","last_peered":"2024-07-26T13:37:17.545780+0000","last_clean":"2024-07-26T13:37:17.545780+0000","last_became_active":"2024-07-26T13:36:56.486328+0000","last_became_peered":"2024-07-26T13:36:56.486328+0000","last_unstale":"2024-07-26T13:37:17.545780+0000","last_undegraded":"2024-07-26T13:37:17.545780+0000","last_fullsized":"2024-07-26T13:37:17.545780+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:10:36.557707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.355361+0000","last_change":"2024-07-26T13:36:59.451415+0000","last_active":"2024-07-26T13:38:46.355361+0000","last_peered":"2024-07-26T13:38:46.355361+0000","last_clean":"2024-07-26T13:38:46.355361+0000","last_became_active":"2024-07-26T13:36:59.450992+0000","last_became_peered":"2024-07-26T13:36:59.450992+0000","last_unstale":"2024-07-26T13:38:46.355361+0000","last_undegraded":"2024-07-26T13:38:46.355361+0000","last_fullsized":"2024-07-26T13:38:46.355361+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T16:29:11.303916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"79'1","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.557777+0000","last_change":"2024-07-26T13:37:17.555643+0000","last_active":"2024-07-26T13:37:17.557777+0000","last_peered":"2024-07-26T13:37:17.557777+0000","last_clean":"2024-07-26T13:37:17.557777+0000","last_became_active":"2024-07-26T13:37:17.549756+0000","last_became_peered":"2024-07-26T13:37:17.549756+0000","last_unstale":"2024-07-26T13:37:17.557777+0000","last_undegraded":"2024-07-26T13:37:17.557777+0000","last_fullsized":"2024-07-26T13:37:17.557777+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:20:16.911286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.368534+0000","last_change":"2024-07-26T13:36:59.465538+0000","last_active":"2024-07-26T13:38:46.368534+0000","last_peered":"2024-07-26T13:38:46.368534+0000","last_clean":"2024-07-26T13:38:46.368534+0000","last_became_active":"2024-07-26T13:36:59.464886+0000","last_became_peered":"2024-07-26T13:36:59.464886+0000","last_unstale":"2024-07-26T13:38:46.368534+0000","last_undegraded":"2024-07-26T13:38:46.368534+0000","last_fullsized":"2024-07-26T13:38:46.368534+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:19:12.273116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'97","reported_seq":213,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:40:22.420742+0000","last_change":"2024-07-26T13:36:56.428355+0000","last_active":"2024-07-26T13:40:22.420742+0000","last_peered":"2024-07-26T13:40:22.420742+0000","last_clean":"2024-07-26T13:40:22.420742+0000","last_became_active":"2024-07-26T13:36:56.427213+0000","last_became_peered":"2024-07-26T13:36:56.427213+0000","last_unstale":"2024-07-26T13:40:22.420742+0000","last_undegraded":"2024-07-26T13:40:22.420742+0000","last_fullsized":"2024-07-26T13:40:22.420742+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:48:09.730067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":179,"num_write_kb":1222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.378498+0000","last_change":"2024-07-26T13:36:59.459378+0000","last_active":"2024-07-26T13:38:46.378498+0000","last_peered":"2024-07-26T13:38:46.378498+0000","last_clean":"2024-07-26T13:38:46.378498+0000","last_became_active":"2024-07-26T13:36:59.458799+0000","last_became_peered":"2024-07-26T13:36:59.458799+0000","last_unstale":"2024-07-26T13:38:46.378498+0000","last_undegraded":"2024-07-26T13:38:46.378498+0000","last_fullsized":"2024-07-26T13:38:46.378498+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:45:19.587800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'4","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.561441+0000","last_change":"2024-07-26T13:36:56.588819+0000","last_active":"2024-07-26T13:37:17.561441+0000","last_peered":"2024-07-26T13:37:17.561441+0000","last_clean":"2024-07-26T13:37:17.561441+0000","last_became_active":"2024-07-26T13:36:56.588430+0000","last_became_peered":"2024-07-26T13:36:56.588430+0000","last_unstale":"2024-07-26T13:37:17.561441+0000","last_undegraded":"2024-07-26T13:37:17.561441+0000","last_fullsized":"2024-07-26T13:37:17.561441+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:07:21.387618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"81'7","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.380460+0000","last_change":"2024-07-26T13:37:17.562040+0000","last_active":"2024-07-26T13:38:46.380460+0000","last_peered":"2024-07-26T13:38:46.380460+0000","last_clean":"2024-07-26T13:38:46.380460+0000","last_became_active":"2024-07-26T13:37:17.561542+0000","last_became_peered":"2024-07-26T13:37:17.561542+0000","last_unstale":"2024-07-26T13:38:46.380460+0000","last_undegraded":"2024-07-26T13:38:46.380460+0000","last_fullsized":"2024-07-26T13:38:46.380460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:21:08.161954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.544169+0000","last_change":"2024-07-26T13:36:56.489106+0000","last_active":"2024-07-26T13:37:17.544169+0000","last_peered":"2024-07-26T13:37:17.544169+0000","last_clean":"2024-07-26T13:37:17.544169+0000","last_became_active":"2024-07-26T13:36:56.488706+0000","last_became_peered":"2024-07-26T13:36:56.488706+0000","last_unstale":"2024-07-26T13:37:17.544169+0000","last_undegraded":"2024-07-26T13:37:17.544169+0000","last_fullsized":"2024-07-26T13:37:17.544169+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:00:59.204434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"81'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.343705+0000","last_change":"2024-07-26T13:36:59.470412+0000","last_active":"2024-07-26T13:38:46.343705+0000","last_peered":"2024-07-26T13:38:46.343705+0000","last_clean":"2024-07-26T13:38:46.343705+0000","last_became_active":"2024-07-26T13:36:59.470010+0000","last_became_peered":"2024-07-26T13:36:59.470010+0000","last_unstale":"2024-07-26T13:38:46.343705+0000","last_undegraded":"2024-07-26T13:38:46.343705+0000","last_fullsized":"2024-07-26T13:38:46.343705+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:50:57.987600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"79'3","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.543528+0000","last_change":"2024-07-26T13:36:56.428671+0000","last_active":"2024-07-26T13:37:17.543528+0000","last_peered":"2024-07-26T13:37:17.543528+0000","last_clean":"2024-07-26T13:37:17.543528+0000","last_became_active":"2024-07-26T13:36:56.428301+0000","last_became_peered":"2024-07-26T13:36:56.428301+0000","last_unstale":"2024-07-26T13:37:17.543528+0000","last_undegraded":"2024-07-26T13:37:17.543528+0000","last_fullsized":"2024-07-26T13:37:17.543528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:59:19.593664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.385021+0000","last_change":"2024-07-26T13:36:59.458627+0000","last_active":"2024-07-26T13:38:46.385021+0000","last_peered":"2024-07-26T13:38:46.385021+0000","last_clean":"2024-07-26T13:38:46.385021+0000","last_became_active":"2024-07-26T13:36:59.458199+0000","last_became_peered":"2024-07-26T13:36:59.458199+0000","last_unstale":"2024-07-26T13:38:46.385021+0000","last_undegraded":"2024-07-26T13:38:46.385021+0000","last_fullsized":"2024-07-26T13:38:46.385021+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T22:54:41.833223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"79'1","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.551540+0000","last_change":"2024-07-26T13:36:56.431548+0000","last_active":"2024-07-26T13:37:17.551540+0000","last_peered":"2024-07-26T13:37:17.551540+0000","last_clean":"2024-07-26T13:37:17.551540+0000","last_became_active":"2024-07-26T13:36:56.430989+0000","last_became_peered":"2024-07-26T13:36:56.430989+0000","last_unstale":"2024-07-26T13:37:17.551540+0000","last_undegraded":"2024-07-26T13:37:17.551540+0000","last_fullsized":"2024-07-26T13:37:17.551540+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:46:51.655324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"81'8","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.375762+0000","last_change":"2024-07-26T13:36:59.458290+0000","last_active":"2024-07-26T13:38:46.375762+0000","last_peered":"2024-07-26T13:38:46.375762+0000","last_clean":"2024-07-26T13:38:46.375762+0000","last_became_active":"2024-07-26T13:36:59.457942+0000","last_became_peered":"2024-07-26T13:36:59.457942+0000","last_unstale":"2024-07-26T13:38:46.375762+0000","last_undegraded":"2024-07-26T13:38:46.375762+0000","last_fullsized":"2024-07-26T13:38:46.375762+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:51:58.797441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'4","reported_seq":46,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.568059+0000","last_change":"2024-07-26T13:37:17.567624+0000","last_active":"2024-07-26T13:37:17.568059+0000","last_peered":"2024-07-26T13:37:17.568059+0000","last_clean":"2024-07-26T13:37:17.568059+0000","last_became_active":"2024-07-26T13:37:17.562455+0000","last_became_peered":"2024-07-26T13:37:17.562455+0000","last_unstale":"2024-07-26T13:37:17.568059+0000","last_undegraded":"2024-07-26T13:37:17.568059+0000","last_fullsized":"2024-07-26T13:37:17.568059+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T14:50:36.418750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.387835+0000","last_change":"2024-07-26T13:36:59.474034+0000","last_active":"2024-07-26T13:38:46.387835+0000","last_peered":"2024-07-26T13:38:46.387835+0000","last_clean":"2024-07-26T13:38:46.387835+0000","last_became_active":"2024-07-26T13:36:59.472358+0000","last_became_peered":"2024-07-26T13:36:59.472358+0000","last_unstale":"2024-07-26T13:38:46.387835+0000","last_undegraded":"2024-07-26T13:38:46.387835+0000","last_fullsized":"2024-07-26T13:38:46.387835+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:36:44.565413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"81'15","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.368595+0000","last_change":"2024-07-26T13:36:59.465546+0000","last_active":"2024-07-26T13:38:46.368595+0000","last_peered":"2024-07-26T13:38:46.368595+0000","last_clean":"2024-07-26T13:38:46.368595+0000","last_became_active":"2024-07-26T13:36:59.465137+0000","last_became_peered":"2024-07-26T13:36:59.465137+0000","last_unstale":"2024-07-26T13:38:46.368595+0000","last_undegraded":"2024-07-26T13:38:46.368595+0000","last_fullsized":"2024-07-26T13:38:46.368595+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:35:13.591354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.562664+0000","last_change":"2024-07-26T13:37:17.562012+0000","last_active":"2024-07-26T13:37:17.562664+0000","last_peered":"2024-07-26T13:37:17.562664+0000","last_clean":"2024-07-26T13:37:17.562664+0000","last_became_active":"2024-07-26T13:37:17.550218+0000","last_became_peered":"2024-07-26T13:37:17.550218+0000","last_unstale":"2024-07-26T13:37:17.562664+0000","last_undegraded":"2024-07-26T13:37:17.562664+0000","last_fullsized":"2024-07-26T13:37:17.562664+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T17:46:16.559533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.375095+0000","last_change":"2024-07-26T13:36:59.456896+0000","last_active":"2024-07-26T13:38:46.375095+0000","last_peered":"2024-07-26T13:38:46.375095+0000","last_clean":"2024-07-26T13:38:46.375095+0000","last_became_active":"2024-07-26T13:36:59.456530+0000","last_became_peered":"2024-07-26T13:36:59.456530+0000","last_unstale":"2024-07-26T13:38:46.375095+0000","last_undegraded":"2024-07-26T13:38:46.375095+0000","last_fullsized":"2024-07-26T13:38:46.375095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T19:59:56.512501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"79'4","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.553596+0000","last_change":"2024-07-26T13:36:56.433119+0000","last_active":"2024-07-26T13:37:17.553596+0000","last_peered":"2024-07-26T13:37:17.553596+0000","last_clean":"2024-07-26T13:37:17.553596+0000","last_became_active":"2024-07-26T13:36:56.432765+0000","last_became_peered":"2024-07-26T13:36:56.432765+0000","last_unstale":"2024-07-26T13:37:17.553596+0000","last_undegraded":"2024-07-26T13:37:17.553596+0000","last_fullsized":"2024-07-26T13:37:17.553596+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T23:31:29.197106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"81'6","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.382051+0000","last_change":"2024-07-26T13:36:59.462125+0000","last_active":"2024-07-26T13:38:46.382051+0000","last_peered":"2024-07-26T13:38:46.382051+0000","last_clean":"2024-07-26T13:38:46.382051+0000","last_became_active":"2024-07-26T13:36:59.461796+0000","last_became_peered":"2024-07-26T13:36:59.461796+0000","last_unstale":"2024-07-26T13:38:46.382051+0000","last_undegraded":"2024-07-26T13:38:46.382051+0000","last_fullsized":"2024-07-26T13:38:46.382051+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T15:44:10.329046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"79'2","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:37:17.543646+0000","last_change":"2024-07-26T13:36:56.487251+0000","last_active":"2024-07-26T13:37:17.543646+0000","last_peered":"2024-07-26T13:37:17.543646+0000","last_clean":"2024-07-26T13:37:17.543646+0000","last_became_active":"2024-07-26T13:36:56.486918+0000","last_became_peered":"2024-07-26T13:36:56.486918+0000","last_unstale":"2024-07-26T13:37:17.543646+0000","last_undegraded":"2024-07-26T13:37:17.543646+0000","last_fullsized":"2024-07-26T13:37:17.543646+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T20:41:31.248895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"81'5","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:38:46.380839+0000","last_change":"2024-07-26T13:36:59.470810+0000","last_active":"2024-07-26T13:38:46.380839+0000","last_peered":"2024-07-26T13:38:46.380839+0000","last_clean":"2024-07-26T13:38:46.380839+0000","last_became_active":"2024-07-26T13:36:59.467371+0000","last_became_peered":"2024-07-26T13:36:59.467371+0000","last_unstale":"2024-07-26T13:38:46.380839+0000","last_undegraded":"2024-07-26T13:38:46.380839+0000","last_fullsized":"2024-07-26T13:38:46.380839+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:58.430687+0000","last_clean_scrub_stamp":"2024-07-26T13:36:58.430687+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-27T18:23:48.572933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"79'9","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-07-26T13:39:03.748968+0000","last_change":"2024-07-26T13:36:56.433598+0000","last_active":"2024-07-26T13:39:03.748968+0000","last_peered":"2024-07-26T13:39:03.748968+0000","last_clean":"2024-07-26T13:39:03.748968+0000","last_became_active":"2024-07-26T13:36:56.433325+0000","last_became_peered":"2024-07-26T13:36:56.433325+0000","last_unstale":"2024-07-26T13:39:03.748968+0000","last_undegraded":"2024-07-26T13:39:03.748968+0000","last_fullsized":"2024-07-26T13:39:03.748968+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-26T13:36:55.412975+0000","last_clean_scrub_stamp":"2024-07-26T13:36:55.412975+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T01:17:33.651765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":859931,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1524,"num_read_kb":972,"num_write":403,"num_write_kb":2217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":644,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":675840,"data_stored":2579793,"data_compressed":112140,"data_compressed_allocated":147456,"data_compressed_original":2285568,"omap_allocated":0,"internal_metadata":0},"log_size":295,"ondisk_log_size":295,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247480,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678074,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":443451,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874303,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1093632,"data_stored":1497483,"data_compressed":59728,"data_compressed_allocated":77824,"data_compressed_original":974848,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037828,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":921600,"data_stored":836881,"data_compressed":25108,"data_compressed_allocated":32768,"data_compressed_original":409600,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234056,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":836779,"data_compressed":25108,"data_compressed_allocated":32768,"data_compressed_original":409600,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430285,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":1036188,"data_compressed":41637,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593811,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":443054,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790040,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1011712,"data_stored":1429116,"data_compressed":59728,"data_compressed_allocated":77824,"data_compressed_original":974848,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953564,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":888832,"data_stored":444023,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149792,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":978944,"data_stored":1230357,"data_compressed":43199,"data_compressed_allocated":57344,"data_compressed_original":778240,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378725,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":511840,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542250,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":962560,"data_stored":512266,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705774,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":905216,"data_stored":444325,"data_compressed":5819,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34195,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34195,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":69419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":68993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":787276,"data_compressed":37380,"data_compressed_allocated":49152,"data_compressed_original":761856,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":395784,"data_compressed":18091,"data_compressed_allocated":24576,"data_compressed_original":368640,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":464151,"data_compressed":18091,"data_compressed_allocated":24576,"data_compressed_original":368640,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-26T13:40:33.699 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:40:33.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:33 smithi167 ceph-mon[37140]: pgmap v377: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 174 KiB/s wr, 52 op/s 2024-07-26T13:40:33.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:33 smithi138 ceph-mon[36954]: pgmap v377: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 174 KiB/s wr, 52 op/s 2024-07-26T13:40:33.955 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:40:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:34 smithi100 ceph-mon[32599]: from='client.15483 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-26T13:40:34.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:34 smithi167 ceph-mon[37140]: from='client.15483 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-26T13:40:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:34 smithi138 ceph-mon[36954]: from='client.15483 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-26T13:40:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.663+0000 7fceea55e640 1 -- 172.21.15.100:0/3673541593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcee4107d30 msgr2=0x7fcee4108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.663+0000 7fceea55e640 1 --2- 172.21.15.100:0/3673541593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcee4107d30 0x7fcee4108140 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fced8009900 tx=0x7fced802a410 comp rx=0 tx=0).stop 2024-07-26T13:40:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.663+0000 7fceea55e640 1 -- 172.21.15.100:0/3673541593 shutdown_connections 2024-07-26T13:40:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.663+0000 7fceea55e640 1 --2- 172.21.15.100:0/3673541593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcee4108680 0x7fcee410eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.663+0000 7fceea55e640 1 --2- 172.21.15.100:0/3673541593 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcee4107d30 0x7fcee4108140 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.663+0000 7fceea55e640 1 --2- 172.21.15.100:0/3673541593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcee4102400 0x7fcee4102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.664+0000 7fceea55e640 1 -- 172.21.15.100:0/3673541593 >> 172.21.15.100:0/3673541593 conn(0x7fcee40fde40 msgr2=0x7fcee4100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.664+0000 7fceea55e640 1 -- 172.21.15.100:0/3673541593 shutdown_connections 2024-07-26T13:40:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.664+0000 7fceea55e640 1 -- 172.21.15.100:0/3673541593 wait complete. 2024-07-26T13:40:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.664+0000 7fceea55e640 1 Processor -- start 2024-07-26T13:40:35.666 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.664+0000 7fceea55e640 1 -- start start 2024-07-26T13:40:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.665+0000 7fceea55e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcee4102400 0x7fcee4199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.665+0000 7fceea55e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcee4107d30 0x7fcee419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.665+0000 7fceea55e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcee4108680 0x7fcee419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.665+0000 7fceea55e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcee4113410 con 0x7fcee4107d30 2024-07-26T13:40:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.665+0000 7fceea55e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcee4113290 con 0x7fcee4108680 2024-07-26T13:40:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.665+0000 7fceea55e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcee4113590 con 0x7fcee4102400 2024-07-26T13:40:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.665+0000 7fcee37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcee4107d30 0x7fcee419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.665+0000 7fcee37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcee4107d30 0x7fcee419a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36396/0 (socket says 172.21.15.100:36396) 2024-07-26T13:40:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.665+0000 7fcee37fe640 1 -- 172.21.15.100:0/3573331354 learned_addr learned my addr 172.21.15.100:0/3573331354 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.665+0000 7fcee8afb640 1 --2- 172.21.15.100:0/3573331354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcee4108680 0x7fcee419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.665+0000 7fcee37fe640 1 -- 172.21.15.100:0/3573331354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcee4102400 msgr2=0x7fcee4199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.665+0000 7fcee37fe640 1 --2- 172.21.15.100:0/3573331354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcee4102400 0x7fcee4199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.665+0000 7fcee37fe640 1 -- 172.21.15.100:0/3573331354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcee4108680 msgr2=0x7fcee419f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.665+0000 7fcee37fe640 1 --2- 172.21.15.100:0/3573331354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcee4108680 0x7fcee419f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.665+0000 7fcee37fe640 1 -- 172.21.15.100:0/3573331354 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee419fe40 con 0x7fcee4107d30 2024-07-26T13:40:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.666+0000 7fcee37fe640 1 --2- 172.21.15.100:0/3573331354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcee4107d30 0x7fcee419a4b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fced802a930 tx=0x7fced8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.666+0000 7fcee17fa640 1 -- 172.21.15.100:0/3573331354 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced803f030 con 0x7fcee4107d30 2024-07-26T13:40:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.666+0000 7fcee17fa640 1 -- 172.21.15.100:0/3573331354 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fced8003aa0 con 0x7fcee4107d30 2024-07-26T13:40:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.666+0000 7fceea55e640 1 -- 172.21.15.100:0/3573331354 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcee41a00d0 con 0x7fcee4107d30 2024-07-26T13:40:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.666+0000 7fcee17fa640 1 -- 172.21.15.100:0/3573331354 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced8035070 con 0x7fcee4107d30 2024-07-26T13:40:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.666+0000 7fceea55e640 1 -- 172.21.15.100:0/3573331354 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcee41a68c0 con 0x7fcee4107d30 2024-07-26T13:40:35.671 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:35 smithi100 ceph-mon[32599]: pgmap v378: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 101 KiB/s wr, 44 op/s 2024-07-26T13:40:35.671 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:35 smithi100 ceph-mon[32599]: scrub summary: idle 2024-07-26T13:40:35.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.666+0000 7ffad8091640 1 -- 172.21.15.100:0/2361700869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad0107eb0 msgr2=0x7ffad010e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:35.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.666+0000 7ffad8091640 1 --2- 172.21.15.100:0/2361700869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad0107eb0 0x7ffad010e6e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ffac800adb0 tx=0x7ffac802b7a0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.667+0000 7ffad8091640 1 -- 172.21.15.100:0/2361700869 shutdown_connections 2024-07-26T13:40:35.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.667+0000 7ffad8091640 1 --2- 172.21.15.100:0/2361700869 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad0107eb0 0x7ffad010e6e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.667+0000 7ffad8091640 1 --2- 172.21.15.100:0/2361700869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffad0107560 0x7ffad0107970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.667+0000 7ffad8091640 1 --2- 172.21.15.100:0/2361700869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad0101c20 0x7ffad0102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.667+0000 7ffad8091640 1 -- 172.21.15.100:0/2361700869 >> 172.21.15.100:0/2361700869 conn(0x7ffad0077690 msgr2=0x7ffad0077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:35.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.667+0000 7ffad8091640 1 -- 172.21.15.100:0/2361700869 shutdown_connections 2024-07-26T13:40:35.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.668+0000 7ffad8091640 1 -- 172.21.15.100:0/2361700869 wait complete. 2024-07-26T13:40:35.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.669+0000 7ffad8091640 1 Processor -- start 2024-07-26T13:40:35.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.669+0000 7ffad8091640 1 -- start start 2024-07-26T13:40:35.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.669+0000 7ffad8091640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffad0101c20 0x7ffad0199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:35.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.669+0000 7ffad8091640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad0107560 0x7ffad019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.670+0000 7fcee17fa640 1 -- 172.21.15.100:0/3573331354 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 151134+0+0 (secure 0 0 0) 0x7fced8042030 con 0x7fcee4107d30 2024-07-26T13:40:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.671+0000 7fcef1479740 1 Processor -- start 2024-07-26T13:40:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.671+0000 7fcef1479740 1 -- start start 2024-07-26T13:40:35.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.669+0000 7ffad8091640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad0107eb0 0x7ffad019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:35.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.669+0000 7ffad8091640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ffad0111340 con 0x7ffad0107eb0 2024-07-26T13:40:35.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.669+0000 7ffad8091640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffad01111c0 con 0x7ffad0107560 2024-07-26T13:40:35.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.669+0000 7ffad8091640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ffad01114c0 con 0x7ffad0101c20 2024-07-26T13:40:35.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.669+0000 7ffad662e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad0107eb0 0x7ffad019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:35.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.669+0000 7ffad662e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad0107eb0 0x7ffad019f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36408/0 (socket says 172.21.15.100:36408) 2024-07-26T13:40:35.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.669+0000 7ffad662e640 1 -- 172.21.15.100:0/2156486735 learned_addr learned my addr 172.21.15.100:0/2156486735 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:35.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.669+0000 7ffad562c640 1 --2- 172.21.15.100:0/2156486735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad0107560 0x7ffad019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:35.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.669+0000 7ffad5e2d640 1 --2- 172.21.15.100:0/2156486735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffad0101c20 0x7ffad0199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:35.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.669+0000 7ffad662e640 1 -- 172.21.15.100:0/2156486735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffad0101c20 msgr2=0x7ffad0199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:35.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.669+0000 7ffad662e640 1 --2- 172.21.15.100:0/2156486735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffad0101c20 0x7ffad0199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.670+0000 7ffad662e640 1 -- 172.21.15.100:0/2156486735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad0107560 msgr2=0x7ffad019a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:35.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.670+0000 7ffad662e640 1 --2- 172.21.15.100:0/2156486735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad0107560 0x7ffad019a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.670+0000 7ffad662e640 1 -- 172.21.15.100:0/2156486735 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad019fdf0 con 0x7ffad0107eb0 2024-07-26T13:40:35.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.670+0000 7ffad662e640 1 --2- 172.21.15.100:0/2156486735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad0107eb0 0x7ffad019f6c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ffac8030f50 tx=0x7ffac8003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:35.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.670+0000 7ffac6ffd640 1 -- 172.21.15.100:0/2156486735 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac8040070 con 0x7ffad0107eb0 2024-07-26T13:40:35.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.670+0000 7ffac6ffd640 1 -- 172.21.15.100:0/2156486735 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffac8032890 con 0x7ffad0107eb0 2024-07-26T13:40:35.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.670+0000 7ffac6ffd640 1 -- 172.21.15.100:0/2156486735 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac80047c0 con 0x7ffad0107eb0 2024-07-26T13:40:35.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.670+0000 7ffad8091640 1 -- 172.21.15.100:0/2156486735 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffad01a0080 con 0x7ffad0107eb0 2024-07-26T13:40:35.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.670+0000 7ffad8091640 1 -- 172.21.15.100:0/2156486735 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffad006c810 con 0x7ffad0107eb0 2024-07-26T13:40:35.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.672+0000 7ffad8091640 1 -- 172.21.15.100:0/2156486735 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffad0102030 con 0x7ffad0107eb0 2024-07-26T13:40:35.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.671+0000 7fcef1479740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e1d19a1b0 0x562e1d53c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:35.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.671+0000 7fcef1479740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e1d1712f0 0x562e1d53cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.671+0000 7fcef1479740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e1d1794d0 0x562e1d53d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.671+0000 7fcef1479740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x562e1d4b23b0 con 0x562e1d1794d0 2024-07-26T13:40:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.672+0000 7fcef1479740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562e1d4a18b0 con 0x562e1d1712f0 2024-07-26T13:40:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.672+0000 7fcef1479740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562e1d4a58f0 con 0x562e1d19a1b0 2024-07-26T13:40:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.672+0000 7fcee17fa640 1 --2- 172.21.15.100:0/3573331354 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fcecc0b3db0 0x7fcecc0b62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.672+0000 7fcee8afb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e1d19a1b0 0x562e1d53c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.672+0000 7fcee3fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e1d1712f0 0x562e1d53cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.672+0000 7fcee8afb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e1d19a1b0 0x562e1d53c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:60262/0 (socket says 172.21.15.100:60262) 2024-07-26T13:40:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.672+0000 7fcee8afb640 1 -- 172.21.15.100:0/78670770 learned_addr learned my addr 172.21.15.100:0/78670770 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.672+0000 7fcee8afb640 1 --2- 172.21.15.100:0/3573331354 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fcecc0b3db0 0x7fcecc0b62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.672+0000 7fcee17fa640 1 -- 172.21.15.100:0/3573331354 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fced8104670 con 0x7fcee4107d30 2024-07-26T13:40:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.672+0000 7fcee37fe640 1 --2- 172.21.15.100:0/78670770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e1d1794d0 0x562e1d53d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.673+0000 7fcee8afb640 1 -- 172.21.15.100:0/78670770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e1d1712f0 msgr2=0x562e1d53cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.673+0000 7fcee8afb640 1 --2- 172.21.15.100:0/78670770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e1d1712f0 0x562e1d53cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.673+0000 7fcee8afb640 1 -- 172.21.15.100:0/78670770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e1d1794d0 msgr2=0x562e1d53d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.673+0000 7fcee8afb640 1 --2- 172.21.15.100:0/78670770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e1d1794d0 0x562e1d53d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.673+0000 7fcee8afb640 1 -- 172.21.15.100:0/78670770 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e1d4489a0 con 0x562e1d19a1b0 2024-07-26T13:40:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.673+0000 7fcee8afb640 1 --2- 172.21.15.100:0/3573331354 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fcecc0b3db0 0x7fcecc0b62c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcedc012b80 tx=0x7fcedc0099b0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.673+0000 7fcee8afb640 1 --2- 172.21.15.100:0/78670770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e1d19a1b0 0x562e1d53c6b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcedc012880 tx=0x7fcedc01c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad35c7ba96c2ea45 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.673+0000 7fceca7fc640 1 -- 172.21.15.100:0/78670770 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcedc027020 con 0x562e1d19a1b0 2024-07-26T13:40:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.673+0000 7fceca7fc640 1 -- 172.21.15.100:0/78670770 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcedc023c50 con 0x562e1d19a1b0 2024-07-26T13:40:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.673+0000 7fceca7fc640 1 -- 172.21.15.100:0/78670770 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcedc00dda0 con 0x562e1d19a1b0 2024-07-26T13:40:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.674+0000 7fcef1479740 1 -- 172.21.15.100:0/78670770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e1d19a1b0 msgr2=0x562e1d53c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.674+0000 7fcef1479740 1 --2- 172.21.15.100:0/78670770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e1d19a1b0 0x562e1d53c6b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcedc012880 tx=0x7fcedc01c000 comp rx=0 tx=0).stop 2024-07-26T13:40:35.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.675+0000 7ffac6ffd640 1 -- 172.21.15.100:0/2156486735 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 151134+0+0 (secure 0 0 0) 0x7ffac8030bc0 con 0x7ffad0107eb0 2024-07-26T13:40:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.675+0000 7fcef1479740 1 -- 172.21.15.100:0/78670770 shutdown_connections 2024-07-26T13:40:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.675+0000 7fcef1479740 1 --2- 172.21.15.100:0/78670770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e1d1794d0 0x562e1d53d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.675+0000 7fcef1479740 1 --2- 172.21.15.100:0/78670770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e1d1712f0 0x562e1d53cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.675+0000 7fcef1479740 1 --2- 172.21.15.100:0/78670770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e1d19a1b0 0x562e1d53c6b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.675+0000 7fcef1479740 1 -- 172.21.15.100:0/78670770 >> 172.21.15.100:0/78670770 conn(0x562e1d18e300 msgr2=0x562e1d53a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:35.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.675+0000 7fcef1479740 1 -- 172.21.15.100:0/78670770 shutdown_connections 2024-07-26T13:40:35.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.675+0000 7fcef1479740 1 -- 172.21.15.100:0/78670770 wait complete. 2024-07-26T13:40:35.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.675+0000 7fcef1479740 1 -- start start 2024-07-26T13:40:35.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.676+0000 7ffac6ffd640 1 --2- 172.21.15.100:0/2156486735 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ffabc0b3d30 0x7ffabc0b6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.676+0000 7fcef1479740 1 Processor -- start 2024-07-26T13:40:35.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.676+0000 7ffad5e2d640 1 --2- 172.21.15.100:0/2156486735 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ffabc0b3d30 0x7ffabc0b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.676+0000 7fcef1479740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e1d1712f0 0x562e1d548300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.676+0000 7fcef1479740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e1d1794d0 0x562e1d548840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:35.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.676+0000 7ffac6ffd640 1 -- 172.21.15.100:0/2156486735 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7ffac81053b0 con 0x7ffad0107eb0 2024-07-26T13:40:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.676+0000 7fcef1479740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e1d19a1b0 0x562e1d5540d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.676+0000 7fcee3fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e1d1712f0 0x562e1d548300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.677+0000 7fcee3fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e1d1712f0 0x562e1d548300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:60268/0 (socket says 172.21.15.100:60268) 2024-07-26T13:40:35.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.677+0000 7ffad5e2d640 1 --2- 172.21.15.100:0/2156486735 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ffabc0b3d30 0x7ffabc0b6240 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ffacc009bd0 tx=0x7ffacc005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.677+0000 7fcee8afb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e1d19a1b0 0x562e1d5540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.677+0000 7fcee8afb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e1d19a1b0 0x562e1d5540d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36428/0 (socket says 172.21.15.100:36428) 2024-07-26T13:40:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.677+0000 7fcee37fe640 1 --2- 172.21.15.100:0/4232072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e1d1794d0 0x562e1d548840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.677+0000 7fcee3fff640 1 -- 172.21.15.100:0/4232072 learned_addr learned my addr 172.21.15.100:0/4232072 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.677+0000 7fcef1479740 1 -- 172.21.15.100:0/4232072 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x562e1d3dbab0 con 0x562e1d19a1b0 2024-07-26T13:40:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.677+0000 7fcef1479740 1 -- 172.21.15.100:0/4232072 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562e1d429cf0 con 0x562e1d1794d0 2024-07-26T13:40:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.677+0000 7fcef1479740 1 -- 172.21.15.100:0/4232072 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562e1d54add0 con 0x562e1d1712f0 2024-07-26T13:40:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.677+0000 7fcef1479740 10 client.0 _ll_register_callbacks cb 0x562e1d49b3e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:40:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.677+0000 7fcee3fff640 1 -- 172.21.15.100:0/4232072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e1d1794d0 msgr2=0x562e1d548840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.677+0000 7fcee3fff640 1 --2- 172.21.15.100:0/4232072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e1d1794d0 0x562e1d548840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.677+0000 7fcee3fff640 1 -- 172.21.15.100:0/4232072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e1d19a1b0 msgr2=0x562e1d5540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.677+0000 7fcee3fff640 1 --2- 172.21.15.100:0/4232072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e1d19a1b0 0x562e1d5540d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.677+0000 7fcee3fff640 1 -- 172.21.15.100:0/4232072 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e1d205790 con 0x562e1d1712f0 2024-07-26T13:40:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.677+0000 7fcee37fe640 1 --2- 172.21.15.100:0/4232072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e1d1794d0 0x562e1d548840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:40:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.677+0000 7fcef1479740 1 -- 172.21.15.100:0/4232072 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcedc023c50 con 0x562e1d1712f0 2024-07-26T13:40:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.678+0000 7fcee3fff640 1 --2- 172.21.15.100:0/4232072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e1d1712f0 0x562e1d548300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fced401cbc0 tx=0x7fced40039c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b893c95b7ac4d178 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.678+0000 7fcec37fe640 10 client.25120 ms_handle_connect on v2:172.21.15.167:3300/0 2024-07-26T13:40:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.678+0000 7fcec37fe640 1 -- 172.21.15.100:0/4232072 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced402b670 con 0x562e1d1712f0 2024-07-26T13:40:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.678+0000 7fcec37fe640 1 -- 172.21.15.100:0/4232072 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fced402b810 con 0x562e1d1712f0 2024-07-26T13:40:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.678+0000 7fcec37fe640 1 -- 172.21.15.100:0/4232072 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fced402ba30 con 0x562e1d1712f0 2024-07-26T13:40:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.678+0000 7fcef1479740 10 client.25120 fetch_fsmap learned FSMap version 26 2024-07-26T13:40:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.678+0000 7fcef1479740 1 -- 172.21.15.100:0/4232072 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({fsmap=26}) -- 0x562e1d27e620 con 0x562e1d1712f0 2024-07-26T13:40:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.678+0000 7fcec37fe640 1 -- 172.21.15.100:0/4232072 <== mon.1 v2:172.21.15.167:3300/0 4 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fced401f040 con 0x562e1d1712f0 2024-07-26T13:40:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.679+0000 7fcef1479740 10 client.25120 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:40:35.687 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.679+0000 7fcef1479740 10 client.25120 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:40:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.679+0000 7fcef1479740 20 client.25120 populate_metadata read hostname 'smithi100' 2024-07-26T13:40:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.679+0000 7fcef1479740 1 --2- 172.21.15.100:0/4232072 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x562e1d158d80 0x562e1d55a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.679+0000 7fcef1479740 4 client.25120 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:40:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.679+0000 7fcef1479740 1 -- 172.21.15.100:0/4232072 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e1d4b4300 con 0x562e1d158d80 2024-07-26T13:40:35.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.679+0000 7ffac6ffd640 1 -- 172.21.15.100:0/2156486735 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7ffac800a9c0 con 0x7ffad0107eb0 2024-07-26T13:40:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.679+0000 7fcee37fe640 1 --2- 172.21.15.100:0/4232072 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x562e1d158d80 0x562e1d55a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.679+0000 7fcec37fe640 10 client.25120 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:40:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.679+0000 7fcee37fe640 1 --2- 172.21.15.100:0/4232072 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x562e1d158d80 0x562e1d55a030 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14526420890de964 server_cookie=f72a2fc808a8f77a in_seq=0 out_seq=0 2024-07-26T13:40:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.680+0000 7fcec37fe640 1 -- 172.21.15.100:0/4232072 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562e1d4b4300 con 0x562e1d158d80 2024-07-26T13:40:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.680+0000 7fcec37fe640 10 client.25120 handle_command_reply: tid=0 2024-07-26T13:40:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.680+0000 7fcef1479740 1 client.25120 shutdown 2024-07-26T13:40:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.680+0000 7fcef1479740 1 -- 172.21.15.100:0/4232072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e1d1712f0 msgr2=0x562e1d548300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.680+0000 7fcef1479740 1 --2- 172.21.15.100:0/4232072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e1d1712f0 0x562e1d548300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fced401cbc0 tx=0x7fced40039c0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.680+0000 7fcef1479740 1 -- 172.21.15.100:0/4232072 shutdown_connections 2024-07-26T13:40:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.680+0000 7fcef1479740 1 --2- 172.21.15.100:0/4232072 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x562e1d158d80 0x562e1d55a030 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.680+0000 7fcef1479740 1 --2- 172.21.15.100:0/4232072 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e1d19a1b0 0x562e1d5540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.690 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.680+0000 7fcec37fe640 0 client.25120 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:40:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.680+0000 7fcef1479740 1 --2- 172.21.15.100:0/4232072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e1d1794d0 0x562e1d548840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.680+0000 7fcef1479740 1 --2- 172.21.15.100:0/4232072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e1d1712f0 0x562e1d548300 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.680+0000 7fcef1479740 1 -- 172.21.15.100:0/4232072 >> 172.21.15.100:0/4232072 conn(0x562e1d18e300 msgr2=0x562e1d538c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.681+0000 7fcef1479740 1 -- 172.21.15.100:0/4232072 shutdown_connections 2024-07-26T13:40:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.681+0000 7fcef1479740 1 -- 172.21.15.100:0/4232072 wait complete. 2024-07-26T13:40:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.681+0000 7fcef1479740 20 client.25120 trim_cache size 0 max 16384 2024-07-26T13:40:35.711 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.709+0000 7fcef1479740 1 Processor -- start 2024-07-26T13:40:35.711 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.709+0000 7fcef1479740 1 -- start start 2024-07-26T13:40:35.711 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.709+0000 7fcef1479740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e1d158d80 0x562e1d541e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:35.711 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.709+0000 7fcef1479740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e1d1712f0 0x562e1d5485d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:35.711 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.709+0000 7fcef1479740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e1d1794d0 0x562e1d553f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:35.711 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.709+0000 7fcef1479740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcee41a6f60 con 0x562e1d1712f0 2024-07-26T13:40:35.711 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.709+0000 7fcef1479740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcee4072dc0 con 0x562e1d158d80 2024-07-26T13:40:35.711 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.709+0000 7fcef1479740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcee4070c10 con 0x562e1d1794d0 2024-07-26T13:40:35.712 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.709+0000 7fcee37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e1d1712f0 0x562e1d5485d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:35.712 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.709+0000 7fcee37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e1d1712f0 0x562e1d5485d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36438/0 (socket says 172.21.15.100:36438) 2024-07-26T13:40:35.712 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.709+0000 7fcee37fe640 1 -- 172.21.15.100:0/2897140057 learned_addr learned my addr 172.21.15.100:0/2897140057 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:35.712 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.709+0000 7fcee3fff640 1 --2- 172.21.15.100:0/2897140057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e1d158d80 0x562e1d541e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:35.712 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.709+0000 7fcee8afb640 1 --2- 172.21.15.100:0/2897140057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e1d1794d0 0x562e1d553f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:35.712 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.709+0000 7fcee37fe640 1 -- 172.21.15.100:0/2897140057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e1d1794d0 msgr2=0x562e1d553f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:35.712 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.709+0000 7fcee37fe640 1 --2- 172.21.15.100:0/2897140057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e1d1794d0 0x562e1d553f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.712 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.709+0000 7fcee37fe640 1 -- 172.21.15.100:0/2897140057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e1d158d80 msgr2=0x562e1d541e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:35.712 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.709+0000 7fcee37fe640 1 --2- 172.21.15.100:0/2897140057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e1d158d80 0x562e1d541e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.712 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.709+0000 7fcee37fe640 1 -- 172.21.15.100:0/2897140057 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e1d5465d0 con 0x562e1d1712f0 2024-07-26T13:40:35.712 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.710+0000 7fcee37fe640 1 --2- 172.21.15.100:0/2897140057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e1d1712f0 0x562e1d5485d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fced8031440 tx=0x7fced8003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2db70419de34fcc server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:35.712 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.710+0000 7fcec9ffb640 1 -- 172.21.15.100:0/2897140057 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced8049070 con 0x562e1d1712f0 2024-07-26T13:40:35.712 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.710+0000 7fcec9ffb640 1 -- 172.21.15.100:0/2897140057 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fced80d43b0 con 0x562e1d1712f0 2024-07-26T13:40:35.712 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.710+0000 7fcec9ffb640 1 -- 172.21.15.100:0/2897140057 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced80c7590 con 0x562e1d1712f0 2024-07-26T13:40:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.711+0000 7fcef1479740 1 -- 172.21.15.100:0/2897140057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e1d1712f0 msgr2=0x562e1d5485d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.711+0000 7fcef1479740 1 --2- 172.21.15.100:0/2897140057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e1d1712f0 0x562e1d5485d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fced8031440 tx=0x7fced8003f50 comp rx=0 tx=0).stop 2024-07-26T13:40:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.711+0000 7fcef1479740 1 -- 172.21.15.100:0/2897140057 shutdown_connections 2024-07-26T13:40:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.711+0000 7fcef1479740 1 --2- 172.21.15.100:0/2897140057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e1d1794d0 0x562e1d553f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.711+0000 7fcef1479740 1 --2- 172.21.15.100:0/2897140057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e1d1712f0 0x562e1d5485d0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.711+0000 7fcef1479740 1 --2- 172.21.15.100:0/2897140057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e1d158d80 0x562e1d541e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.711+0000 7fcef1479740 1 -- 172.21.15.100:0/2897140057 >> 172.21.15.100:0/2897140057 conn(0x562e1d18e300 msgr2=0x562e1d541770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.711+0000 7fcef1479740 1 -- 172.21.15.100:0/2897140057 shutdown_connections 2024-07-26T13:40:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.711+0000 7fcef1479740 1 -- 172.21.15.100:0/2897140057 wait complete. 2024-07-26T13:40:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.711+0000 7fcef1479740 1 -- start start 2024-07-26T13:40:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.712+0000 7fcef1479740 1 Processor -- start 2024-07-26T13:40:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.712+0000 7fcef1479740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e1d158d80 0x562e1d5507d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.712+0000 7fcef1479740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e1d1712f0 0x562e1d54e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.712+0000 7fcef1479740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e1d1794d0 0x562e1d54e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.712+0000 7fcef1479740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcee4071280 con 0x562e1d1712f0 2024-07-26T13:40:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.712+0000 7fcef1479740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcee40726f0 con 0x562e1d158d80 2024-07-26T13:40:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.712+0000 7fcef1479740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcee4071950 con 0x562e1d1794d0 2024-07-26T13:40:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.712+0000 7fcef1479740 10 client.0 _ll_register_callbacks cb 0x7fced401c800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:40:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.712+0000 7fcee37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e1d1712f0 0x562e1d54e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.712+0000 7fcee37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e1d1712f0 0x562e1d54e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36454/0 (socket says 172.21.15.100:36454) 2024-07-26T13:40:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.712+0000 7fcee37fe640 1 -- 172.21.15.100:0/4180733972 learned_addr learned my addr 172.21.15.100:0/4180733972 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.712+0000 7fcee3fff640 1 --2- 172.21.15.100:0/4180733972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e1d158d80 0x562e1d5507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.712+0000 7fcee8afb640 1 --2- 172.21.15.100:0/4180733972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e1d1794d0 0x562e1d54e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.713+0000 7fcee37fe640 1 -- 172.21.15.100:0/4180733972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e1d1794d0 msgr2=0x562e1d54e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.713+0000 7fcee37fe640 1 --2- 172.21.15.100:0/4180733972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e1d1794d0 0x562e1d54e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.713+0000 7fcee37fe640 1 -- 172.21.15.100:0/4180733972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e1d158d80 msgr2=0x562e1d5507d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.713+0000 7fcee37fe640 1 --2- 172.21.15.100:0/4180733972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e1d158d80 0x562e1d5507d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.713+0000 7fcee37fe640 1 -- 172.21.15.100:0/4180733972 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e1d54ef60 con 0x562e1d1712f0 2024-07-26T13:40:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.713+0000 7fcee37fe640 1 --2- 172.21.15.100:0/4180733972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e1d1712f0 0x562e1d54e210 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fced803b6f0 tx=0x7fced803bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db5d2ff4187d6bae server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.713+0000 7fcec37fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:40:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.713+0000 7fcec37fe640 1 -- 172.21.15.100:0/4180733972 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced80e4020 con 0x562e1d1712f0 2024-07-26T13:40:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.713+0000 7fcec37fe640 1 -- 172.21.15.100:0/4180733972 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fced80c7590 con 0x562e1d1712f0 2024-07-26T13:40:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.713+0000 7fcef1479740 1 -- 172.21.15.100:0/4180733972 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fced80d43b0 con 0x562e1d1712f0 2024-07-26T13:40:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.713+0000 7fcec37fe640 1 -- 172.21.15.100:0/4180733972 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fced8049040 con 0x562e1d1712f0 2024-07-26T13:40:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.713+0000 7fcef1479740 10 client.15504 fetch_fsmap learned FSMap version 26 2024-07-26T13:40:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.713+0000 7fcef1479740 1 -- 172.21.15.100:0/4180733972 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562e1d5c1710 con 0x562e1d1712f0 2024-07-26T13:40:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.714+0000 7fcec37fe640 1 -- 172.21.15.100:0/4180733972 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fced80d43b0 con 0x562e1d1712f0 2024-07-26T13:40:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.714+0000 7fcec37fe640 1 -- 172.21.15.100:0/4180733972 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fced8077ea0 con 0x562e1d1712f0 2024-07-26T13:40:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.714+0000 7fcef1479740 10 client.15504 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:40:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.714+0000 7fcef1479740 10 client.15504 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:40:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.714+0000 7fcef1479740 20 client.15504 populate_metadata read hostname 'smithi100' 2024-07-26T13:40:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.714+0000 7fcef1479740 1 --2- 172.21.15.100:0/4180733972 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x562e1d19a1b0 0x562e1d5c3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.714+0000 7fcef1479740 4 client.15504 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-26T13:40:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.714+0000 7fcef1479740 1 -- 172.21.15.100:0/4180733972 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562e1d4afda0 con 0x562e1d19a1b0 2024-07-26T13:40:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.714+0000 7fcee3fff640 1 --2- 172.21.15.100:0/4180733972 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x562e1d19a1b0 0x562e1d5c3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.714+0000 7fcee3fff640 1 --2- 172.21.15.100:0/4180733972 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x562e1d19a1b0 0x562e1d5c3c50 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28aeb481a8df1ef6 server_cookie=d189671a617dc495 in_seq=0 out_seq=0 2024-07-26T13:40:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.714+0000 7fcec37fe640 10 client.15504 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:40:35.727 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.726+0000 7fcec37fe640 1 -- 172.21.15.100:0/4180733972 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562e1d4afda0 con 0x562e1d19a1b0 2024-07-26T13:40:35.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.726+0000 7fcec37fe640 10 client.15504 handle_command_reply: tid=0 2024-07-26T13:40:35.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.726+0000 7fcef1479740 1 client.15504 shutdown 2024-07-26T13:40:35.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.726+0000 7fcef1479740 1 -- 172.21.15.100:0/4180733972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e1d1712f0 msgr2=0x562e1d54e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:35.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.726+0000 7fcef1479740 1 --2- 172.21.15.100:0/4180733972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e1d1712f0 0x562e1d54e210 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fced803b6f0 tx=0x7fced803bf70 comp rx=0 tx=0).stop 2024-07-26T13:40:35.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.726+0000 7fcef1479740 1 -- 172.21.15.100:0/4180733972 shutdown_connections 2024-07-26T13:40:35.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.726+0000 7fcef1479740 1 --2- 172.21.15.100:0/4180733972 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x562e1d19a1b0 0x562e1d5c3c50 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.726+0000 7fcef1479740 1 --2- 172.21.15.100:0/4180733972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e1d1794d0 0x562e1d54e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.726+0000 7fcec37fe640 0 client.15504 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:40:35.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.726+0000 7fcef1479740 1 --2- 172.21.15.100:0/4180733972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e1d1712f0 0x562e1d54e210 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.726+0000 7fcef1479740 1 --2- 172.21.15.100:0/4180733972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e1d158d80 0x562e1d5507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.726+0000 7fcef1479740 1 -- 172.21.15.100:0/4180733972 >> 172.21.15.100:0/4180733972 conn(0x562e1d18e300 msgr2=0x562e1d53d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:35.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.726+0000 7fcef1479740 1 -- 172.21.15.100:0/4180733972 shutdown_connections 2024-07-26T13:40:35.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.726+0000 7fcef1479740 1 -- 172.21.15.100:0/4180733972 wait complete. 2024-07-26T13:40:35.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.726+0000 7fcef1479740 20 client.15504 trim_cache size 0 max 16384 2024-07-26T13:40:35.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-07-26T13:40:35.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-07-26T13:40:35.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-07-26T13:40:35.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-07-26T13:40:35.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.727+0000 7fceea55e640 1 -- 172.21.15.100:0/3573331354 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fcecc0b3db0 msgr2=0x7fcecc0b62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:35.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.727+0000 7fceea55e640 1 --2- 172.21.15.100:0/3573331354 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fcecc0b3db0 0x7fcecc0b62c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcedc012b80 tx=0x7fcedc0099b0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.727+0000 7fceea55e640 1 -- 172.21.15.100:0/3573331354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcee4107d30 msgr2=0x7fcee419a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:35.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.727+0000 7fceea55e640 1 --2- 172.21.15.100:0/3573331354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcee4107d30 0x7fcee419a4b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fced802a930 tx=0x7fced8004270 comp rx=0 tx=0).stop 2024-07-26T13:40:35.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.727+0000 7fceea55e640 1 -- 172.21.15.100:0/3573331354 shutdown_connections 2024-07-26T13:40:35.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.727+0000 7fceea55e640 1 --2- 172.21.15.100:0/3573331354 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fcecc0b3db0 0x7fcecc0b62c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.727+0000 7fceea55e640 1 --2- 172.21.15.100:0/3573331354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcee4108680 0x7fcee419f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.727+0000 7fceea55e640 1 --2- 172.21.15.100:0/3573331354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcee4107d30 0x7fcee419a4b0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.727+0000 7fceea55e640 1 --2- 172.21.15.100:0/3573331354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcee4102400 0x7fcee4199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.727+0000 7fceea55e640 1 -- 172.21.15.100:0/3573331354 >> 172.21.15.100:0/3573331354 conn(0x7fcee40fde40 msgr2=0x7fcee4100260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.727+0000 7fceea55e640 1 -- 172.21.15.100:0/3573331354 shutdown_connections 2024-07-26T13:40:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:35.727+0000 7fceea55e640 1 -- 172.21.15.100:0/3573331354 wait complete. 2024-07-26T13:40:35.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:35 smithi167 ceph-mon[37140]: pgmap v378: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 101 KiB/s wr, 44 op/s 2024-07-26T13:40:35.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:35 smithi167 ceph-mon[37140]: scrub summary: idle 2024-07-26T13:40:35.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.903+0000 7ffad8091640 1 -- 172.21.15.100:0/2156486735 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffad019afb0 con 0x7ffad0107eb0 2024-07-26T13:40:35.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.904+0000 7ffac6ffd640 1 -- 172.21.15.100:0/2156486735 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11927 (secure 0 0 0) 0x7ffad019afb0 con 0x7ffad0107eb0 2024-07-26T13:40:35.905 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:40:35.905 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":26,"btime":"2024-07-26T13:40:09:530496+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14790,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/313501409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":313501409},{"type":"v1","addr":"172.21.15.100:6839","nonce":313501409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/1402975506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":1402975506},{"type":"v1","addr":"172.21.15.100:6841","nonce":1402975506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-26T13:37:01.281843+0000","modified":"2024-07-26T13:40:09.187125+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24607,"mds_1":24605,"mds_2":24595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24557":{"gid":24557,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/3290105300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3290105300},{"type":"v1","addr":"172.21.15.138:6835","nonce":3290105300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/3393225483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3393225483},{"type":"v1","addr":"172.21.15.138:6837","nonce":3393225483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24595":{"gid":24595,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6837/3828634468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3828634468},{"type":"v1","addr":"172.21.15.167:6837","nonce":3828634468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.138:6839/1580925471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1580925471},{"type":"v1","addr":"172.21.15.138:6839","nonce":1580925471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/398150514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":398150514},{"type":"v1","addr":"172.21.15.167:6839","nonce":398150514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24607,"qdb_cluster":[24595,24605,24607]},"id":1}]} 2024-07-26T13:40:35.905 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 26 2024-07-26T13:40:35.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.906+0000 7ffad8091640 1 -- 172.21.15.100:0/2156486735 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ffabc0b3d30 msgr2=0x7ffabc0b6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:35.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.906+0000 7ffad8091640 1 --2- 172.21.15.100:0/2156486735 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ffabc0b3d30 0x7ffabc0b6240 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ffacc009bd0 tx=0x7ffacc005f90 comp rx=0 tx=0).stop 2024-07-26T13:40:35.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.906+0000 7ffad8091640 1 -- 172.21.15.100:0/2156486735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad0107eb0 msgr2=0x7ffad019f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:35.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.906+0000 7ffad8091640 1 --2- 172.21.15.100:0/2156486735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad0107eb0 0x7ffad019f6c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ffac8030f50 tx=0x7ffac8003f10 comp rx=0 tx=0).stop 2024-07-26T13:40:35.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.906+0000 7ffad8091640 1 -- 172.21.15.100:0/2156486735 shutdown_connections 2024-07-26T13:40:35.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.906+0000 7ffad8091640 1 --2- 172.21.15.100:0/2156486735 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ffabc0b3d30 0x7ffabc0b6240 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.906+0000 7ffad8091640 1 --2- 172.21.15.100:0/2156486735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffad0107eb0 0x7ffad019f6c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.906+0000 7ffad8091640 1 --2- 172.21.15.100:0/2156486735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffad0107560 0x7ffad019a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.906+0000 7ffad8091640 1 --2- 172.21.15.100:0/2156486735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffad0101c20 0x7ffad0199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:35.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.906+0000 7ffad8091640 1 -- 172.21.15.100:0/2156486735 >> 172.21.15.100:0/2156486735 conn(0x7ffad0077690 msgr2=0x7ffad01034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:35.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.906+0000 7ffad8091640 1 -- 172.21.15.100:0/2156486735 shutdown_connections 2024-07-26T13:40:35.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:35.906+0000 7ffad8091640 1 -- 172.21.15.100:0/2156486735 wait complete. 2024-07-26T13:40:35.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:35 smithi138 ceph-mon[36954]: pgmap v378: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 101 KiB/s wr, 44 op/s 2024-07-26T13:40:35.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:35 smithi138 ceph-mon[36954]: scrub summary: idle 2024-07-26T13:40:36.662 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:40:36.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:36 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/2156486735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:40:36.919 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:40:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:36 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/2156486735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:40:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:36 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2156486735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:40:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:37 smithi100 ceph-mon[32599]: pgmap v379: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T13:40:37.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:37 smithi167 ceph-mon[37140]: pgmap v379: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T13:40:37.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:37 smithi138 ceph-mon[36954]: pgmap v379: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T13:40:38.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.562+0000 7f26c2623640 1 -- 172.21.15.100:0/4154094638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26bc107e00 msgr2=0x7f26bc108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:38.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.562+0000 7f26c2623640 1 --2- 172.21.15.100:0/4154094638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26bc107e00 0x7f26bc108210 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f26b0009900 tx=0x7f26b002a410 comp rx=0 tx=0).stop 2024-07-26T13:40:38.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.562+0000 7f26c2623640 1 -- 172.21.15.100:0/4154094638 shutdown_connections 2024-07-26T13:40:38.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.562+0000 7f26c2623640 1 --2- 172.21.15.100:0/4154094638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26bc108750 0x7f26bc10ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:38.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.562+0000 7f26c2623640 1 --2- 172.21.15.100:0/4154094638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26bc107e00 0x7f26bc108210 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:38.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.562+0000 7f26c2623640 1 --2- 172.21.15.100:0/4154094638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26bc1024c0 0x7f26bc1028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:38.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.562+0000 7f26c2623640 1 -- 172.21.15.100:0/4154094638 >> 172.21.15.100:0/4154094638 conn(0x7f26bc079180 msgr2=0x7f26bc079590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:38.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.562+0000 7f26c2623640 1 -- 172.21.15.100:0/4154094638 shutdown_connections 2024-07-26T13:40:38.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.562+0000 7f26c2623640 1 -- 172.21.15.100:0/4154094638 wait complete. 2024-07-26T13:40:38.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.563+0000 7f26c2623640 1 Processor -- start 2024-07-26T13:40:38.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.563+0000 7f26c2623640 1 -- start start 2024-07-26T13:40:38.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.563+0000 7f26c2623640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26bc1024c0 0x7f26bc199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:38.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.563+0000 7f26c2623640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26bc107e00 0x7f26bc19a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:38.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.563+0000 7f26c2623640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26bc108750 0x7f26bc19f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:38.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.563+0000 7f26c2623640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc113500 con 0x7f26bc108750 2024-07-26T13:40:38.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.563+0000 7f26c2623640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc113380 con 0x7f26bc107e00 2024-07-26T13:40:38.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.563+0000 7f26c2623640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f26bc113680 con 0x7f26bc1024c0 2024-07-26T13:40:38.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.564+0000 7f26c0bc0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26bc108750 0x7f26bc19f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:38.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.564+0000 7f26c0bc0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26bc108750 0x7f26bc19f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36484/0 (socket says 172.21.15.100:36484) 2024-07-26T13:40:38.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.564+0000 7f26c0bc0640 1 -- 172.21.15.100:0/2759440833 learned_addr learned my addr 172.21.15.100:0/2759440833 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:38.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.564+0000 7f26bbfff640 1 --2- 172.21.15.100:0/2759440833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26bc1024c0 0x7f26bc199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:38.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.564+0000 7f26bb7fe640 1 --2- 172.21.15.100:0/2759440833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26bc107e00 0x7f26bc19a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:38.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.564+0000 7f26c0bc0640 1 -- 172.21.15.100:0/2759440833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26bc1024c0 msgr2=0x7f26bc199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:38.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.564+0000 7f26c0bc0640 1 --2- 172.21.15.100:0/2759440833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26bc1024c0 0x7f26bc199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:38.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.564+0000 7f26c0bc0640 1 -- 172.21.15.100:0/2759440833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26bc107e00 msgr2=0x7f26bc19a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:38.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.564+0000 7f26c0bc0640 1 --2- 172.21.15.100:0/2759440833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26bc107e00 0x7f26bc19a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:38.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.564+0000 7f26c0bc0640 1 -- 172.21.15.100:0/2759440833 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26bc19ff40 con 0x7f26bc108750 2024-07-26T13:40:38.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.564+0000 7f26c0bc0640 1 --2- 172.21.15.100:0/2759440833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26bc108750 0x7f26bc19f810 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f26b400b970 tx=0x7f26b400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:38.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.564+0000 7f26b97fa640 1 -- 172.21.15.100:0/2759440833 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b4019070 con 0x7f26bc108750 2024-07-26T13:40:38.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.564+0000 7f26b97fa640 1 -- 172.21.15.100:0/2759440833 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26b4014440 con 0x7f26bc108750 2024-07-26T13:40:38.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.564+0000 7f26c2623640 1 -- 172.21.15.100:0/2759440833 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26bc1a0230 con 0x7f26bc108750 2024-07-26T13:40:38.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.564+0000 7f26b97fa640 1 -- 172.21.15.100:0/2759440833 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b40075d0 con 0x7f26bc108750 2024-07-26T13:40:38.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.564+0000 7f26c2623640 1 -- 172.21.15.100:0/2759440833 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26bc1a6990 con 0x7f26bc108750 2024-07-26T13:40:38.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.566+0000 7f26c2623640 1 -- 172.21.15.100:0/2759440833 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26bc103740 con 0x7f26bc108750 2024-07-26T13:40:38.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.567+0000 7f26b97fa640 1 -- 172.21.15.100:0/2759440833 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 151134+0+0 (secure 0 0 0) 0x7f26b401c070 con 0x7f26bc108750 2024-07-26T13:40:38.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.568+0000 7f26b97fa640 1 --2- 172.21.15.100:0/2759440833 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f26a40b3cb0 0x7f26a40b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:38.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.568+0000 7f26bbfff640 1 --2- 172.21.15.100:0/2759440833 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f26a40b3cb0 0x7f26a40b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:38.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.568+0000 7f26b97fa640 1 -- 172.21.15.100:0/2759440833 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f26b40e3390 con 0x7f26bc108750 2024-07-26T13:40:38.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.568+0000 7f26bbfff640 1 --2- 172.21.15.100:0/2759440833 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f26a40b3cb0 0x7f26a40b61c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f26ac0058c0 tx=0x7f26ac00a2d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:38.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.572+0000 7f26b97fa640 1 -- 172.21.15.100:0/2759440833 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f26b40aa240 con 0x7f26bc108750 2024-07-26T13:40:38.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.781+0000 7f26c2623640 1 -- 172.21.15.100:0/2759440833 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26bc1028d0 con 0x7f26bc108750 2024-07-26T13:40:38.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.781+0000 7f26b97fa640 1 -- 172.21.15.100:0/2759440833 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11927 (secure 0 0 0) 0x7f26bc1028d0 con 0x7f26bc108750 2024-07-26T13:40:38.782 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:40:38.783 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":26,"btime":"2024-07-26T13:40:09:530496+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14790,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/313501409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":313501409},{"type":"v1","addr":"172.21.15.100:6839","nonce":313501409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/1402975506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":1402975506},{"type":"v1","addr":"172.21.15.100:6841","nonce":1402975506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-26T13:37:01.281843+0000","modified":"2024-07-26T13:40:09.187125+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24607,"mds_1":24605,"mds_2":24595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24557":{"gid":24557,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/3290105300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3290105300},{"type":"v1","addr":"172.21.15.138:6835","nonce":3290105300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/3393225483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3393225483},{"type":"v1","addr":"172.21.15.138:6837","nonce":3393225483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24595":{"gid":24595,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6837/3828634468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3828634468},{"type":"v1","addr":"172.21.15.167:6837","nonce":3828634468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.138:6839/1580925471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1580925471},{"type":"v1","addr":"172.21.15.138:6839","nonce":1580925471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/398150514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":398150514},{"type":"v1","addr":"172.21.15.167:6839","nonce":398150514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24607,"qdb_cluster":[24595,24605,24607]},"id":1}]} 2024-07-26T13:40:38.783 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 26 2024-07-26T13:40:38.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.783+0000 7f26c2623640 1 -- 172.21.15.100:0/2759440833 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f26a40b3cb0 msgr2=0x7f26a40b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:38.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.783+0000 7f26c2623640 1 --2- 172.21.15.100:0/2759440833 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f26a40b3cb0 0x7f26a40b61c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f26ac0058c0 tx=0x7f26ac00a2d0 comp rx=0 tx=0).stop 2024-07-26T13:40:38.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.783+0000 7f26c2623640 1 -- 172.21.15.100:0/2759440833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26bc108750 msgr2=0x7f26bc19f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:38.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.783+0000 7f26c2623640 1 --2- 172.21.15.100:0/2759440833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26bc108750 0x7f26bc19f810 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f26b400b970 tx=0x7f26b400be30 comp rx=0 tx=0).stop 2024-07-26T13:40:38.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.783+0000 7f26c2623640 1 -- 172.21.15.100:0/2759440833 shutdown_connections 2024-07-26T13:40:38.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.783+0000 7f26c2623640 1 --2- 172.21.15.100:0/2759440833 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f26a40b3cb0 0x7f26a40b61c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:38.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.783+0000 7f26c2623640 1 --2- 172.21.15.100:0/2759440833 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f26bc108750 0x7f26bc19f810 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:38.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.783+0000 7f26c2623640 1 --2- 172.21.15.100:0/2759440833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26bc107e00 0x7f26bc19a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:38.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.783+0000 7f26c2623640 1 --2- 172.21.15.100:0/2759440833 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26bc1024c0 0x7f26bc199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:38.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.783+0000 7f26c2623640 1 -- 172.21.15.100:0/2759440833 >> 172.21.15.100:0/2759440833 conn(0x7f26bc079180 msgr2=0x7f26bc103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:38.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.784+0000 7f26c2623640 1 -- 172.21.15.100:0/2759440833 shutdown_connections 2024-07-26T13:40:38.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:38.784+0000 7f26c2623640 1 -- 172.21.15.100:0/2759440833 wait complete. 2024-07-26T13:40:39.501 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph osd dump --format=json 2024-07-26T13:40:39.757 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:40:39.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:39 smithi167 ceph-mon[37140]: pgmap v380: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T13:40:39.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:39 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/2759440833' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:40:39.880 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:39 smithi100 ceph-mon[32599]: pgmap v380: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T13:40:39.880 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:39 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2759440833' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:40:39.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:39 smithi138 ceph-mon[36954]: pgmap v380: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T13:40:39.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:39 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/2759440833' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:40:41.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.463+0000 7f096c685640 1 -- 172.21.15.100:0/2053079681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0964108730 msgr2=0x7f096410ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:41.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.463+0000 7f096c685640 1 --2- 172.21.15.100:0/2053079681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0964108730 0x7f096410ef60 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f095c009ca0 tx=0x7f095c02a8c0 comp rx=0 tx=0).stop 2024-07-26T13:40:41.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.464+0000 7f096c685640 1 -- 172.21.15.100:0/2053079681 shutdown_connections 2024-07-26T13:40:41.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.464+0000 7f096c685640 1 --2- 172.21.15.100:0/2053079681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0964108730 0x7f096410ef60 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:41.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.464+0000 7f096c685640 1 --2- 172.21.15.100:0/2053079681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0964107de0 0x7f09641081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:41.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.464+0000 7f096c685640 1 --2- 172.21.15.100:0/2053079681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09641024a0 0x7f09641028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:41.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.464+0000 7f096c685640 1 -- 172.21.15.100:0/2053079681 >> 172.21.15.100:0/2053079681 conn(0x7f0964079140 msgr2=0x7f0964079550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:41.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.464+0000 7f096c685640 1 -- 172.21.15.100:0/2053079681 shutdown_connections 2024-07-26T13:40:41.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.464+0000 7f096c685640 1 -- 172.21.15.100:0/2053079681 wait complete. 2024-07-26T13:40:41.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.464+0000 7f096c685640 1 Processor -- start 2024-07-26T13:40:41.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.465+0000 7f096c685640 1 -- start start 2024-07-26T13:40:41.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.465+0000 7f096c685640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09641024a0 0x7f0964199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:41.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.465+0000 7f096c685640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0964107de0 0x7f096419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:41.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.465+0000 7f096c685640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0964108730 0x7f096419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:41.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.465+0000 7f096c685640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0964112400 con 0x7f09641024a0 2024-07-26T13:40:41.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.465+0000 7f096c685640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0964112280 con 0x7f0964107de0 2024-07-26T13:40:41.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.465+0000 7f096c685640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0964112580 con 0x7f0964108730 2024-07-26T13:40:41.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.465+0000 7f096a421640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09641024a0 0x7f0964199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:41.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.465+0000 7f096a421640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09641024a0 0x7f0964199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36502/0 (socket says 172.21.15.100:36502) 2024-07-26T13:40:41.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.465+0000 7f096a421640 1 -- 172.21.15.100:0/1212992793 learned_addr learned my addr 172.21.15.100:0/1212992793 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:41.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.465+0000 7f0969c20640 1 --2- 172.21.15.100:0/1212992793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0964107de0 0x7f096419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:41.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.465+0000 7f096ac22640 1 --2- 172.21.15.100:0/1212992793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0964108730 0x7f096419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:41.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.465+0000 7f096a421640 1 -- 172.21.15.100:0/1212992793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0964108730 msgr2=0x7f096419f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:41.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.465+0000 7f096a421640 1 --2- 172.21.15.100:0/1212992793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0964108730 0x7f096419f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:41.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.465+0000 7f096a421640 1 -- 172.21.15.100:0/1212992793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0964107de0 msgr2=0x7f096419a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:41.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.465+0000 7f096a421640 1 --2- 172.21.15.100:0/1212992793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0964107de0 0x7f096419a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:41.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.465+0000 7f096a421640 1 -- 172.21.15.100:0/1212992793 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f096419feb0 con 0x7f09641024a0 2024-07-26T13:40:41.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.465+0000 7f096a421640 1 --2- 172.21.15.100:0/1212992793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09641024a0 0x7f0964199f40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f096000d770 tx=0x7f096000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:41.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.466+0000 7f09577fe640 1 -- 172.21.15.100:0/1212992793 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0960017070 con 0x7f09641024a0 2024-07-26T13:40:41.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.466+0000 7f09577fe640 1 -- 172.21.15.100:0/1212992793 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09600054d0 con 0x7f09641024a0 2024-07-26T13:40:41.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.466+0000 7f09577fe640 1 -- 172.21.15.100:0/1212992793 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0960004c30 con 0x7f09641024a0 2024-07-26T13:40:41.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.466+0000 7f096c685640 1 -- 172.21.15.100:0/1212992793 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09641a01a0 con 0x7f09641024a0 2024-07-26T13:40:41.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.466+0000 7f096c685640 1 -- 172.21.15.100:0/1212992793 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09641a6900 con 0x7f09641024a0 2024-07-26T13:40:41.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.467+0000 7f096c685640 1 -- 172.21.15.100:0/1212992793 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0964103720 con 0x7f09641024a0 2024-07-26T13:40:41.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.468+0000 7f09577fe640 1 -- 172.21.15.100:0/1212992793 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 151134+0+0 (secure 0 0 0) 0x7f0960004e10 con 0x7f09641024a0 2024-07-26T13:40:41.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.469+0000 7f09577fe640 1 --2- 172.21.15.100:0/1212992793 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f09500b3d30 0x7f09500b6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:41.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.469+0000 7f0969c20640 1 --2- 172.21.15.100:0/1212992793 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f09500b3d30 0x7f09500b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:41.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.469+0000 7f09577fe640 1 -- 172.21.15.100:0/1212992793 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f09600e3930 con 0x7f09641024a0 2024-07-26T13:40:41.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.469+0000 7f0969c20640 1 --2- 172.21.15.100:0/1212992793 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f09500b3d30 0x7f09500b6240 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f096406c480 tx=0x7f095800a2d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:41.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.473+0000 7f09577fe640 1 -- 172.21.15.100:0/1212992793 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f09600aad20 con 0x7f09641024a0 2024-07-26T13:40:41.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.651+0000 7f096c685640 1 -- 172.21.15.100:0/1212992793 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f096419b070 con 0x7f09641024a0 2024-07-26T13:40:41.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.652+0000 7f09577fe640 1 -- 172.21.15.100:0/1212992793 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+23121 (secure 0 0 0) 0x7f096419b070 con 0x7f09641024a0 2024-07-26T13:40:41.654 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:40:41.654 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":81,"fsid":"bf83e2e6-4b52-11ef-bcb7-c7b262605968","created":"2024-07-26T13:28:23.639785+0000","modified":"2024-07-26T13:37:17.539521+0000","last_up_change":"2024-07-26T13:35:56.475554+0000","last_in_change":"2024-07-26T13:35:37.472564+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-26T13:32:22.329969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-26T13:36:54.590525+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-26T13:36:58.149545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d714036d-9dda-4cb5-af16-29f83a09dd70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6803","nonce":1480401850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6805","nonce":1480401850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6809","nonce":1480401850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":1480401850},{"type":"v1","addr":"172.21.15.100:6807","nonce":1480401850}]},"public_addr":"172.21.15.100:6803/1480401850","cluster_addr":"172.21.15.100:6805/1480401850","heartbeat_back_addr":"172.21.15.100:6809/1480401850","heartbeat_front_addr":"172.21.15.100:6807/1480401850","state":["exists","up"]},{"osd":1,"uuid":"3dd0eaa1-618a-495a-9819-e01c0f869ae8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6801","nonce":3090877671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6803","nonce":3090877671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6807","nonce":3090877671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3090877671},{"type":"v1","addr":"172.21.15.138:6805","nonce":3090877671}]},"public_addr":"172.21.15.138:6801/3090877671","cluster_addr":"172.21.15.138:6803/3090877671","heartbeat_back_addr":"172.21.15.138:6807/3090877671","heartbeat_front_addr":"172.21.15.138:6805/3090877671","state":["exists","up"]},{"osd":2,"uuid":"5c710a78-17ae-41a8-ac16-a678f5d35672","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6801","nonce":3583424790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6803","nonce":3583424790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6807","nonce":3583424790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3583424790},{"type":"v1","addr":"172.21.15.167:6805","nonce":3583424790}]},"public_addr":"172.21.15.167:6801/3583424790","cluster_addr":"172.21.15.167:6803/3583424790","heartbeat_back_addr":"172.21.15.167:6807/3583424790","heartbeat_front_addr":"172.21.15.167:6805/3583424790","state":["exists","up"]},{"osd":3,"uuid":"be845495-8c31-4834-a45c-bdc576711962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6811","nonce":3626762419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6813","nonce":3626762419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6817","nonce":3626762419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3626762419},{"type":"v1","addr":"172.21.15.100:6815","nonce":3626762419}]},"public_addr":"172.21.15.100:6811/3626762419","cluster_addr":"172.21.15.100:6813/3626762419","heartbeat_back_addr":"172.21.15.100:6817/3626762419","heartbeat_front_addr":"172.21.15.100:6815/3626762419","state":["exists","up"]},{"osd":4,"uuid":"520554cc-e846-46f9-af77-05a1de65a452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6809","nonce":3757334356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6811","nonce":3757334356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6815","nonce":3757334356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":3757334356},{"type":"v1","addr":"172.21.15.138:6813","nonce":3757334356}]},"public_addr":"172.21.15.138:6809/3757334356","cluster_addr":"172.21.15.138:6811/3757334356","heartbeat_back_addr":"172.21.15.138:6815/3757334356","heartbeat_front_addr":"172.21.15.138:6813/3757334356","state":["exists","up"]},{"osd":5,"uuid":"68dc8ad6-31d2-45b5-9aa7-62c4218d46e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6809","nonce":3239482836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6811","nonce":3239482836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6815","nonce":3239482836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3239482836},{"type":"v1","addr":"172.21.15.167:6813","nonce":3239482836}]},"public_addr":"172.21.15.167:6809/3239482836","cluster_addr":"172.21.15.167:6811/3239482836","heartbeat_back_addr":"172.21.15.167:6815/3239482836","heartbeat_front_addr":"172.21.15.167:6813/3239482836","state":["exists","up"]},{"osd":6,"uuid":"420e4471-5626-41cc-a8e1-69c801196b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6819","nonce":3218415484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6821","nonce":3218415484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6825","nonce":3218415484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":3218415484},{"type":"v1","addr":"172.21.15.100:6823","nonce":3218415484}]},"public_addr":"172.21.15.100:6819/3218415484","cluster_addr":"172.21.15.100:6821/3218415484","heartbeat_back_addr":"172.21.15.100:6825/3218415484","heartbeat_front_addr":"172.21.15.100:6823/3218415484","state":["exists","up"]},{"osd":7,"uuid":"a27b097e-207d-44c3-af63-544f228b83ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6817","nonce":45541749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6819","nonce":45541749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6823","nonce":45541749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":45541749},{"type":"v1","addr":"172.21.15.138:6821","nonce":45541749}]},"public_addr":"172.21.15.138:6817/45541749","cluster_addr":"172.21.15.138:6819/45541749","heartbeat_back_addr":"172.21.15.138:6823/45541749","heartbeat_front_addr":"172.21.15.138:6821/45541749","state":["exists","up"]},{"osd":8,"uuid":"3e865858-92cc-48a0-a464-5767d19e8ac0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6817","nonce":4031159803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6819","nonce":4031159803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6823","nonce":4031159803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":4031159803},{"type":"v1","addr":"172.21.15.167:6821","nonce":4031159803}]},"public_addr":"172.21.15.167:6817/4031159803","cluster_addr":"172.21.15.167:6819/4031159803","heartbeat_back_addr":"172.21.15.167:6823/4031159803","heartbeat_front_addr":"172.21.15.167:6821/4031159803","state":["exists","up"]},{"osd":9,"uuid":"8b05b14a-1195-42f5-ac01-d5634cb9c888","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6827","nonce":1840140273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6829","nonce":1840140273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6833","nonce":1840140273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1840140273},{"type":"v1","addr":"172.21.15.100:6831","nonce":1840140273}]},"public_addr":"172.21.15.100:6827/1840140273","cluster_addr":"172.21.15.100:6829/1840140273","heartbeat_back_addr":"172.21.15.100:6833/1840140273","heartbeat_front_addr":"172.21.15.100:6831/1840140273","state":["exists","up"]},{"osd":10,"uuid":"ab77ebe7-5254-4dc3-bc54-625698e84afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6825","nonce":703382899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6827","nonce":703382899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6831","nonce":703382899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":703382899},{"type":"v1","addr":"172.21.15.138:6829","nonce":703382899}]},"public_addr":"172.21.15.138:6825/703382899","cluster_addr":"172.21.15.138:6827/703382899","heartbeat_back_addr":"172.21.15.138:6831/703382899","heartbeat_front_addr":"172.21.15.138:6829/703382899","state":["exists","up"]},{"osd":11,"uuid":"229cf91e-e9ef-48f6-a677-c5c5705b315a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6825","nonce":1032343068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6827","nonce":1032343068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6831","nonce":1032343068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1032343068},{"type":"v1","addr":"172.21.15.167:6829","nonce":1032343068}]},"public_addr":"172.21.15.167:6825/1032343068","cluster_addr":"172.21.15.167:6827/1032343068","heartbeat_back_addr":"172.21.15.167:6831/1032343068","heartbeat_front_addr":"172.21.15.167:6829/1032343068","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:31:35.800702+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:31:58.087922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:32:20.463183+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:32:42.920598+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:33:06.332655+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:33:29.381407+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:33:53.975963+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:34:18.226813+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:34:41.845770+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:35:05.722275+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:35:30.862704+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-26T13:35:54.738697+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:0/1874993112":"2024-07-27T13:30:16.253096+0000","172.21.15.100:0/2364620305":"2024-07-27T13:28:45.340334+0000","172.21.15.100:6800/3096076075":"2024-07-27T13:29:07.979178+0000","172.21.15.100:0/1894966501":"2024-07-27T13:30:16.253096+0000","172.21.15.100:0/3411776672":"2024-07-27T13:28:45.340334+0000","172.21.15.100:0/2853423120":"2024-07-27T13:29:07.979178+0000","172.21.15.100:6800/3025873570":"2024-07-27T13:28:45.340334+0000","172.21.15.100:0/4887181":"2024-07-27T13:29:07.979178+0000","172.21.15.100:6801/908675140":"2024-07-27T13:30:16.253096+0000","172.21.15.100:6801/3025873570":"2024-07-27T13:28:45.340334+0000","172.21.15.100:6800/908675140":"2024-07-27T13:30:16.253096+0000","172.21.15.100:6801/3096076075":"2024-07-27T13:29:07.979178+0000","172.21.15.100:0/2302632052":"2024-07-27T13:29:07.979178+0000","172.21.15.100:0/3736098915":"2024-07-27T13:28:45.340334+0000","172.21.15.100:0/3243030560":"2024-07-27T13:30:16.253096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-26T13:40:41.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.654+0000 7f096c685640 1 -- 172.21.15.100:0/1212992793 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f09500b3d30 msgr2=0x7f09500b6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:41.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.654+0000 7f096c685640 1 --2- 172.21.15.100:0/1212992793 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f09500b3d30 0x7f09500b6240 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f096406c480 tx=0x7f095800a2d0 comp rx=0 tx=0).stop 2024-07-26T13:40:41.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.654+0000 7f096c685640 1 -- 172.21.15.100:0/1212992793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09641024a0 msgr2=0x7f0964199f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:41.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.654+0000 7f096c685640 1 --2- 172.21.15.100:0/1212992793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09641024a0 0x7f0964199f40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f096000d770 tx=0x7f096000dc30 comp rx=0 tx=0).stop 2024-07-26T13:40:41.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.655+0000 7f096c685640 1 -- 172.21.15.100:0/1212992793 shutdown_connections 2024-07-26T13:40:41.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.655+0000 7f096c685640 1 --2- 172.21.15.100:0/1212992793 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f09500b3d30 0x7f09500b6240 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:41.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.655+0000 7f096c685640 1 --2- 172.21.15.100:0/1212992793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0964108730 0x7f096419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:41.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.655+0000 7f096c685640 1 --2- 172.21.15.100:0/1212992793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0964107de0 0x7f096419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:41.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.655+0000 7f096c685640 1 --2- 172.21.15.100:0/1212992793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09641024a0 0x7f0964199f40 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:41.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.655+0000 7f096c685640 1 -- 172.21.15.100:0/1212992793 >> 172.21.15.100:0/1212992793 conn(0x7f0964079140 msgr2=0x7f0964101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:41.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.655+0000 7f096c685640 1 -- 172.21.15.100:0/1212992793 shutdown_connections 2024-07-26T13:40:41.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:41.655+0000 7f096c685640 1 -- 172.21.15.100:0/1212992793 wait complete. 2024-07-26T13:40:41.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:41 smithi167 ceph-mon[37140]: pgmap v381: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-07-26T13:40:41.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:41 smithi138 ceph-mon[36954]: pgmap v381: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-07-26T13:40:41.963 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:41 smithi100 ceph-mon[32599]: pgmap v381: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-07-26T13:40:42.344 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-07-26T13:40:42.602 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:40:42.724 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:42 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/1212992793' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-26T13:40:42.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:42 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/1212992793' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-26T13:40:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:42 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/1212992793' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-26T13:40:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:43 smithi100 ceph-mon[32599]: pgmap v382: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-07-26T13:40:43.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:43 smithi167 ceph-mon[37140]: pgmap v382: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-07-26T13:40:43.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:43 smithi138 ceph-mon[36954]: pgmap v382: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-07-26T13:40:44.970 INFO:teuthology.orchestra.run.smithi100.stdout:{"run_dir":"/var/run/ceph"} 2024-07-26T13:40:44.971 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-07-26T13:40:44.971 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 5532 2024-07-26T13:40:44.971 DEBUG:tasks.quiescer:Yielding 2024-07-26T13:40:44.972 INFO:teuthology.run_tasks:Running task workunit... 2024-07-26T13:40:44.987 INFO:tasks.workunit:Pulling workunits from ref e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 2024-07-26T13:40:44.987 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-07-26T13:40:44.987 INFO:tasks.workunit:timeout=3h 2024-07-26T13:40:44.987 INFO:tasks.workunit:cleanup=True 2024-07-26T13:40:44.987 DEBUG:teuthology.orchestra.run.smithi100:> stat -- /home/ubuntu/cephtest/mnt.0 2024-07-26T13:40:44.988 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:40:45.004 INFO:teuthology.orchestra.run.smithi100.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-07-26T13:40:45.005 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-07-26T13:40:45.005 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-07-26T13:40:45.005 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-26T13:40:45.005 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-07-26T13:40:45.005 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-07-26 13:37:30.896629130 +0000 2024-07-26T13:40:45.005 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-07-26 13:37:30.896629130 +0000 2024-07-26T13:40:45.005 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-07-26 13:40:08.490376383 +0000 2024-07-26T13:40:45.005 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-07-26 13:37:30.896629130 +0000 2024-07-26T13:40:45.005 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-07-26T13:40:45.005 DEBUG:teuthology.orchestra.run.smithi100:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-07-26T13:40:45.082 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 2024-07-26T13:40:45.138 INFO:tasks.workunit.client.0.smithi100.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-07-26T13:40:45.257 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:40:45.637 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:45 smithi100 ceph-mon[32599]: pgmap v383: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-07-26T13:40:45.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:45 smithi167 ceph-mon[37140]: pgmap v383: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-07-26T13:40:45.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:45 smithi138 ceph-mon[36954]: pgmap v383: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-07-26T13:40:46.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.985+0000 7f88e9b75640 1 -- 172.21.15.100:0/1088000173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88e41035e0 msgr2=0x7f88e41039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:46.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.985+0000 7f88e9b75640 1 --2- 172.21.15.100:0/1088000173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88e41035e0 0x7f88e41039f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f88d4009900 tx=0x7f88d402a410 comp rx=0 tx=0).stop 2024-07-26T13:40:46.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.985+0000 7f88e9b75640 1 -- 172.21.15.100:0/1088000173 shutdown_connections 2024-07-26T13:40:46.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.985+0000 7f88e9b75640 1 --2- 172.21.15.100:0/1088000173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88e4109870 0x7f88e41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:46.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.985+0000 7f88e9b75640 1 --2- 172.21.15.100:0/1088000173 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88e4108f20 0x7f88e4109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:46.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.985+0000 7f88e9b75640 1 --2- 172.21.15.100:0/1088000173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88e41035e0 0x7f88e41039f0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:46.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.985+0000 7f88e9b75640 1 -- 172.21.15.100:0/1088000173 >> 172.21.15.100:0/1088000173 conn(0x7f88e40ff030 msgr2=0x7f88e4101480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:46.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.985+0000 7f88e9b75640 1 -- 172.21.15.100:0/1088000173 shutdown_connections 2024-07-26T13:40:46.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.986+0000 7f88e9b75640 1 -- 172.21.15.100:0/1088000173 wait complete. 2024-07-26T13:40:46.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.986+0000 7f88e9b75640 1 Processor -- start 2024-07-26T13:40:46.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.986+0000 7f88e9b75640 1 -- start start 2024-07-26T13:40:46.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.987+0000 7f88e9b75640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88e41035e0 0x7f88e4075140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:46.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.987+0000 7f88e9b75640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88e4108f20 0x7f88e4075680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:46.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.987+0000 7f88e9b75640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88e4109870 0x7f88e4075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:46.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.987+0000 7f88e9b75640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f88e4113590 con 0x7f88e4108f20 2024-07-26T13:40:46.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.987+0000 7f88e9b75640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f88e4113410 con 0x7f88e41035e0 2024-07-26T13:40:46.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.987+0000 7f88e9b75640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f88e4113710 con 0x7f88e4109870 2024-07-26T13:40:46.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.987+0000 7f88e2ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88e4108f20 0x7f88e4075680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:46.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.987+0000 7f88e2ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88e4108f20 0x7f88e4075680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42234/0 (socket says 172.21.15.100:42234) 2024-07-26T13:40:46.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.987+0000 7f88e2ffd640 1 -- 172.21.15.100:0/2101698523 learned_addr learned my addr 172.21.15.100:0/2101698523 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:46.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.987+0000 7f88e3fff640 1 --2- 172.21.15.100:0/2101698523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88e4109870 0x7f88e4075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:46.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.987+0000 7f88e37fe640 1 --2- 172.21.15.100:0/2101698523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88e41035e0 0x7f88e4075140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:46.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.987+0000 7f88e2ffd640 1 -- 172.21.15.100:0/2101698523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88e4109870 msgr2=0x7f88e4075bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:46.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.987+0000 7f88e2ffd640 1 --2- 172.21.15.100:0/2101698523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88e4109870 0x7f88e4075bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:46.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.987+0000 7f88e2ffd640 1 -- 172.21.15.100:0/2101698523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88e41035e0 msgr2=0x7f88e4075140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:46.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.987+0000 7f88e2ffd640 1 --2- 172.21.15.100:0/2101698523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88e41035e0 0x7f88e4075140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:46.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.987+0000 7f88e2ffd640 1 -- 172.21.15.100:0/2101698523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e41a8220 con 0x7f88e4108f20 2024-07-26T13:40:46.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.987+0000 7f88e3fff640 1 --2- 172.21.15.100:0/2101698523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88e4109870 0x7f88e4075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:40:46.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.987+0000 7f88e2ffd640 1 --2- 172.21.15.100:0/2101698523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88e4108f20 0x7f88e4075680 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f88d8006fd0 tx=0x7f88d800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:46.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.988+0000 7f88e0ff9640 1 -- 172.21.15.100:0/2101698523 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d8011830 con 0x7f88e4108f20 2024-07-26T13:40:46.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.988+0000 7f88e0ff9640 1 -- 172.21.15.100:0/2101698523 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88d8016070 con 0x7f88e4108f20 2024-07-26T13:40:46.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.988+0000 7f88e9b75640 1 -- 172.21.15.100:0/2101698523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88e41a8510 con 0x7f88e4108f20 2024-07-26T13:40:46.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.988+0000 7f88e0ff9640 1 -- 172.21.15.100:0/2101698523 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d8010e00 con 0x7f88e4108f20 2024-07-26T13:40:46.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.988+0000 7f88e9b75640 1 -- 172.21.15.100:0/2101698523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88e41a8ad0 con 0x7f88e4108f20 2024-07-26T13:40:46.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.989+0000 7f88e9b75640 1 -- 172.21.15.100:0/2101698523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88e41039f0 con 0x7f88e4108f20 2024-07-26T13:40:46.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.990+0000 7f88e0ff9640 1 -- 172.21.15.100:0/2101698523 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 151134+0+0 (secure 0 0 0) 0x7f88d801b460 con 0x7f88e4108f20 2024-07-26T13:40:46.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.991+0000 7f88e0ff9640 1 --2- 172.21.15.100:0/2101698523 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f88cc0b3cb0 0x7f88cc0b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:46.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.991+0000 7f88e37fe640 1 --2- 172.21.15.100:0/2101698523 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f88cc0b3cb0 0x7f88cc0b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:46.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.991+0000 7f88e0ff9640 1 -- 172.21.15.100:0/2101698523 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f88d80e3b80 con 0x7f88e4108f20 2024-07-26T13:40:46.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.992+0000 7f88e37fe640 1 --2- 172.21.15.100:0/2101698523 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f88cc0b3cb0 0x7f88cc0b61c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f88d4000c00 tx=0x7f88d4032ce0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:46.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:46.994+0000 7f88e0ff9640 1 -- 172.21.15.100:0/2101698523 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f88d80ab080 con 0x7f88e4108f20 2024-07-26T13:40:47.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:47.203+0000 7f88e9b75640 1 -- 172.21.15.100:0/2101698523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88e4076440 con 0x7f88e4108f20 2024-07-26T13:40:47.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:47.203+0000 7f88e0ff9640 1 -- 172.21.15.100:0/2101698523 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11927 (secure 0 0 0) 0x7f88d80aa980 con 0x7f88e4108f20 2024-07-26T13:40:47.204 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:40:47.205 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":26,"btime":"2024-07-26T13:40:09:530496+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14790,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/313501409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":313501409},{"type":"v1","addr":"172.21.15.100:6839","nonce":313501409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/1402975506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":1402975506},{"type":"v1","addr":"172.21.15.100:6841","nonce":1402975506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-26T13:37:01.281843+0000","modified":"2024-07-26T13:40:09.187125+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24607,"mds_1":24605,"mds_2":24595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24557":{"gid":24557,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/3290105300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3290105300},{"type":"v1","addr":"172.21.15.138:6835","nonce":3290105300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/3393225483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3393225483},{"type":"v1","addr":"172.21.15.138:6837","nonce":3393225483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24595":{"gid":24595,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6837/3828634468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3828634468},{"type":"v1","addr":"172.21.15.167:6837","nonce":3828634468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.138:6839/1580925471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1580925471},{"type":"v1","addr":"172.21.15.138:6839","nonce":1580925471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/398150514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":398150514},{"type":"v1","addr":"172.21.15.167:6839","nonce":398150514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24607,"qdb_cluster":[24595,24605,24607]},"id":1}]} 2024-07-26T13:40:47.205 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 26 2024-07-26T13:40:47.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:47.205+0000 7f88e9b75640 1 -- 172.21.15.100:0/2101698523 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f88cc0b3cb0 msgr2=0x7f88cc0b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:47.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:47.205+0000 7f88e9b75640 1 --2- 172.21.15.100:0/2101698523 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f88cc0b3cb0 0x7f88cc0b61c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f88d4000c00 tx=0x7f88d4032ce0 comp rx=0 tx=0).stop 2024-07-26T13:40:47.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:47.205+0000 7f88e9b75640 1 -- 172.21.15.100:0/2101698523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88e4108f20 msgr2=0x7f88e4075680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:47.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:47.205+0000 7f88e9b75640 1 --2- 172.21.15.100:0/2101698523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88e4108f20 0x7f88e4075680 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f88d8006fd0 tx=0x7f88d800b8e0 comp rx=0 tx=0).stop 2024-07-26T13:40:47.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:47.205+0000 7f88e9b75640 1 -- 172.21.15.100:0/2101698523 shutdown_connections 2024-07-26T13:40:47.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:47.205+0000 7f88e9b75640 1 --2- 172.21.15.100:0/2101698523 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f88cc0b3cb0 0x7f88cc0b61c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:47.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:47.205+0000 7f88e9b75640 1 --2- 172.21.15.100:0/2101698523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88e4109870 0x7f88e4075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:47.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:47.205+0000 7f88e9b75640 1 --2- 172.21.15.100:0/2101698523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88e4108f20 0x7f88e4075680 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:47.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:47.205+0000 7f88e9b75640 1 --2- 172.21.15.100:0/2101698523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88e41035e0 0x7f88e4075140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:47.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:47.205+0000 7f88e9b75640 1 -- 172.21.15.100:0/2101698523 >> 172.21.15.100:0/2101698523 conn(0x7f88e40ff030 msgr2=0x7f88e4101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:47.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:47.206+0000 7f88e9b75640 1 -- 172.21.15.100:0/2101698523 shutdown_connections 2024-07-26T13:40:47.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:47.206+0000 7f88e9b75640 1 -- 172.21.15.100:0/2101698523 wait complete. 2024-07-26T13:40:47.568 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:47 smithi100 ceph-mon[32599]: pgmap v384: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-07-26T13:40:47.568 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:47 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2101698523' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:40:47.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:47 smithi167 ceph-mon[37140]: pgmap v384: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-07-26T13:40:47.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:47 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/2101698523' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:40:47.899 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-07-26T13:37:01.281843+0000', 'modified': '2024-07-26T13:40:09.187125+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24607, 'mds_1': 24605, 'mds_2': 24595}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24557': {'gid': 24557, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.138:6835/3290105300', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6834', 'nonce': 3290105300}, {'type': 'v1', 'addr': '172.21.15.138:6835', 'nonce': 3290105300}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24575': {'gid': 24575, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.138:6837/3393225483', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6836', 'nonce': 3393225483}, {'type': 'v1', 'addr': '172.21.15.138:6837', 'nonce': 3393225483}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24586': {'gid': 24586, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6835/641259145', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6834', 'nonce': 641259145}, {'type': 'v1', 'addr': '172.21.15.167:6835', 'nonce': 641259145}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24595': {'gid': 24595, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.167:6837/3828634468', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6836', 'nonce': 3828634468}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 3828634468}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24605': {'gid': 24605, 'name': 'b', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.138:6839/1580925471', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6838', 'nonce': 1580925471}, {'type': 'v1', 'addr': '172.21.15.138:6839', 'nonce': 1580925471}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24607': {'gid': 24607, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.167:6839/398150514', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6838', 'nonce': 398150514}, {'type': 'v1', 'addr': '172.21.15.167:6839', 'nonce': 398150514}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24607, 'qdb_cluster': [24595, 24605, 24607]} 2024-07-26T13:40:47.900 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-07-26T13:40:47.900 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.c status 2024-07-26T13:40:47.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:47 smithi138 ceph-mon[36954]: pgmap v384: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-07-26T13:40:47.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:47 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/2101698523' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:40:48.198 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:40:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.826+0000 7f791b5ad640 1 -- 172.21.15.100:0/4132550937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7914103500 msgr2=0x7f7914103910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.826+0000 7f791b5ad640 1 --2- 172.21.15.100:0/4132550937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7914103500 0x7f7914103910 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f790c009900 tx=0x7f790c02a410 comp rx=0 tx=0).stop 2024-07-26T13:40:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.826+0000 7f791b5ad640 1 -- 172.21.15.100:0/4132550937 shutdown_connections 2024-07-26T13:40:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.826+0000 7f791b5ad640 1 --2- 172.21.15.100:0/4132550937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7914109780 0x7f791410ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.826+0000 7f791b5ad640 1 --2- 172.21.15.100:0/4132550937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7914108e30 0x7f7914109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.826+0000 7f791b5ad640 1 --2- 172.21.15.100:0/4132550937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7914103500 0x7f7914103910 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.826+0000 7f791b5ad640 1 -- 172.21.15.100:0/4132550937 >> 172.21.15.100:0/4132550937 conn(0x7f79140fef40 msgr2=0x7f7914101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.826+0000 7f791b5ad640 1 -- 172.21.15.100:0/4132550937 shutdown_connections 2024-07-26T13:40:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.826+0000 7f791b5ad640 1 -- 172.21.15.100:0/4132550937 wait complete. 2024-07-26T13:40:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.827+0000 7f791b5ad640 1 Processor -- start 2024-07-26T13:40:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.827+0000 7f791b5ad640 1 -- start start 2024-07-26T13:40:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.827+0000 7f791b5ad640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7914103500 0x7f7914199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.827+0000 7f791b5ad640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7914108e30 0x7f791419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.827+0000 7f791b5ad640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7914109780 0x7f791419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.827+0000 7f791b5ad640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7914113450 con 0x7f7914108e30 2024-07-26T13:40:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.827+0000 7f791b5ad640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f79141132d0 con 0x7f7914109780 2024-07-26T13:40:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.827+0000 7f791b5ad640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f79141135d0 con 0x7f7914103500 2024-07-26T13:40:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.827+0000 7f7918b48640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7914108e30 0x7f791419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.827+0000 7f7918b48640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7914108e30 0x7f791419a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42252/0 (socket says 172.21.15.100:42252) 2024-07-26T13:40:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.827+0000 7f7918b48640 1 -- 172.21.15.100:0/2088332283 learned_addr learned my addr 172.21.15.100:0/2088332283 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.828+0000 7f7919349640 1 --2- 172.21.15.100:0/2088332283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7914103500 0x7f7914199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.828+0000 7f7919b4a640 1 --2- 172.21.15.100:0/2088332283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7914109780 0x7f791419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.828+0000 7f7918b48640 1 -- 172.21.15.100:0/2088332283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7914103500 msgr2=0x7f7914199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:48.831 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.828+0000 7f7918b48640 1 --2- 172.21.15.100:0/2088332283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7914103500 0x7f7914199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.828+0000 7f7918b48640 1 -- 172.21.15.100:0/2088332283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7914109780 msgr2=0x7f791419f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.828+0000 7f7918b48640 1 --2- 172.21.15.100:0/2088332283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7914109780 0x7f791419f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.828+0000 7f7918b48640 1 -- 172.21.15.100:0/2088332283 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f791419fde0 con 0x7f7914108e30 2024-07-26T13:40:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.828+0000 7f7918b48640 1 --2- 172.21.15.100:0/2088332283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7914108e30 0x7f791419a450 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f790800c970 tx=0x7f790800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.828+0000 7f79127fc640 1 -- 172.21.15.100:0/2088332283 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908004220 con 0x7f7914108e30 2024-07-26T13:40:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.828+0000 7f791b5ad640 1 -- 172.21.15.100:0/2088332283 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79141a00d0 con 0x7f7914108e30 2024-07-26T13:40:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.828+0000 7f79127fc640 1 -- 172.21.15.100:0/2088332283 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79080043c0 con 0x7f7914108e30 2024-07-26T13:40:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.828+0000 7f79127fc640 1 -- 172.21.15.100:0/2088332283 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908004a00 con 0x7f7914108e30 2024-07-26T13:40:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.828+0000 7f791b5ad640 1 -- 172.21.15.100:0/2088332283 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79141a6860 con 0x7f7914108e30 2024-07-26T13:40:48.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.831+0000 7f79127fc640 1 -- 172.21.15.100:0/2088332283 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 151134+0+0 (secure 0 0 0) 0x7f79080079b0 con 0x7f7914108e30 2024-07-26T13:40:48.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.831+0000 7f79225a7740 1 Processor -- start 2024-07-26T13:40:48.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.831+0000 7f79225a7740 1 -- start start 2024-07-26T13:40:48.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.831+0000 7f79225a7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640078b71b0 0x564007c59ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:48.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.831+0000 7f79225a7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56400788e2f0 0x564007c5a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:48.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.831+0000 7f79225a7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640078964d0 0x564007c5a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:48.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.831+0000 7f79225a7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x564007bcf3b0 con 0x5640078b71b0 2024-07-26T13:40:48.839 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.831+0000 7f79225a7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564007bbe8b0 con 0x5640078964d0 2024-07-26T13:40:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.831+0000 7f79225a7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564007bc28f0 con 0x56400788e2f0 2024-07-26T13:40:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.832+0000 7f7919b4a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640078b71b0 0x564007c59ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.832+0000 7f7919b4a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640078b71b0 0x564007c59ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42256/0 (socket says 172.21.15.100:42256) 2024-07-26T13:40:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.832+0000 7f7919b4a640 1 -- 172.21.15.100:0/146741339 learned_addr learned my addr 172.21.15.100:0/146741339 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.832+0000 7f79127fc640 1 --2- 172.21.15.100:0/2088332283 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f78fc0b3cb0 0x7f78fc0b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.832+0000 7f7919349640 1 --2- 172.21.15.100:0/146741339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56400788e2f0 0x564007c5a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.832+0000 7f7918b48640 1 --2- 172.21.15.100:0/146741339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640078964d0 0x564007c5a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.832+0000 7f7919b4a640 1 -- 172.21.15.100:0/146741339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56400788e2f0 msgr2=0x564007c5a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.832+0000 7f7919b4a640 1 --2- 172.21.15.100:0/146741339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56400788e2f0 0x564007c5a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.832+0000 7f7919b4a640 1 -- 172.21.15.100:0/146741339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640078964d0 msgr2=0x564007c5a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.832+0000 7f7919b4a640 1 --2- 172.21.15.100:0/146741339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640078964d0 0x564007c5a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.832+0000 7f7919b4a640 1 -- 172.21.15.100:0/146741339 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564007b659a0 con 0x5640078b71b0 2024-07-26T13:40:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.832+0000 7f7918b48640 1 --2- 172.21.15.100:0/146741339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640078964d0 0x564007c5a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:40:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.832+0000 7f79127fc640 1 -- 172.21.15.100:0/2088332283 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f79080e3bb0 con 0x7f7914108e30 2024-07-26T13:40:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.832+0000 7f7919b4a640 1 --2- 172.21.15.100:0/2088332283 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f78fc0b3cb0 0x7f78fc0b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.832+0000 7f7919b4a640 1 --2- 172.21.15.100:0/146741339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640078b71b0 0x564007c59ce0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7904011590 tx=0x7f7904011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=695e3ce2ae67354c server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.832+0000 7f78fb7fe640 1 -- 172.21.15.100:0/146741339 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7904017050 con 0x5640078b71b0 2024-07-26T13:40:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.832+0000 7f78fb7fe640 1 -- 172.21.15.100:0/146741339 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f790400fcf0 con 0x5640078b71b0 2024-07-26T13:40:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.832+0000 7f78fb7fe640 1 -- 172.21.15.100:0/146741339 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790401b9c0 con 0x5640078b71b0 2024-07-26T13:40:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.833+0000 7f79225a7740 1 -- 172.21.15.100:0/146741339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640078b71b0 msgr2=0x564007c59ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.833+0000 7f79225a7740 1 --2- 172.21.15.100:0/146741339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640078b71b0 0x564007c59ce0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7904011590 tx=0x7f7904011a50 comp rx=0 tx=0).stop 2024-07-26T13:40:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.833+0000 7f7919b4a640 1 --2- 172.21.15.100:0/2088332283 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f78fc0b3cb0 0x7f78fc0b61c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7904022000 tx=0x7f7904023000 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.833+0000 7f79225a7740 1 -- 172.21.15.100:0/146741339 shutdown_connections 2024-07-26T13:40:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.833+0000 7f79225a7740 1 --2- 172.21.15.100:0/146741339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640078964d0 0x564007c5a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.833+0000 7f79225a7740 1 --2- 172.21.15.100:0/146741339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56400788e2f0 0x564007c5a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:48.841 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.833+0000 7f79225a7740 1 --2- 172.21.15.100:0/146741339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640078b71b0 0x564007c59ce0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.833+0000 7f79225a7740 1 -- 172.21.15.100:0/146741339 >> 172.21.15.100:0/146741339 conn(0x5640078ab300 msgr2=0x564007c57bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.833+0000 7f79225a7740 1 -- 172.21.15.100:0/146741339 shutdown_connections 2024-07-26T13:40:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.833+0000 7f79225a7740 1 -- 172.21.15.100:0/146741339 wait complete. 2024-07-26T13:40:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.834+0000 7f79225a7740 1 -- start start 2024-07-26T13:40:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.834+0000 7f79225a7740 1 Processor -- start 2024-07-26T13:40:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.834+0000 7f79225a7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56400788e2f0 0x564007c69c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.834+0000 7f79225a7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640078964d0 0x564007c6a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.834+0000 7f79225a7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640078b71b0 0x564007c75b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.834+0000 7f79225a7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x564007af8ab0 con 0x56400788e2f0 2024-07-26T13:40:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.834+0000 7f79225a7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564007b46cf0 con 0x5640078964d0 2024-07-26T13:40:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.834+0000 7f79225a7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564007c6c840 con 0x5640078b71b0 2024-07-26T13:40:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.834+0000 7f79225a7740 10 client.0 _ll_register_callbacks cb 0x564007bb89f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:40:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.834+0000 7f7919349640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56400788e2f0 0x564007c69c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.834+0000 7f7919349640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56400788e2f0 0x564007c69c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42266/0 (socket says 172.21.15.100:42266) 2024-07-26T13:40:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.834+0000 7f7919349640 1 -- 172.21.15.100:0/2530535412 learned_addr learned my addr 172.21.15.100:0/2530535412 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.835+0000 7f7919b4a640 1 --2- 172.21.15.100:0/2530535412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640078b71b0 0x564007c75b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.835+0000 7f7918b48640 1 --2- 172.21.15.100:0/2530535412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640078964d0 0x564007c6a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.835+0000 7f7919349640 1 -- 172.21.15.100:0/2530535412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640078b71b0 msgr2=0x564007c75b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.835+0000 7f7919349640 1 --2- 172.21.15.100:0/2530535412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640078b71b0 0x564007c75b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.835+0000 7f7919349640 1 -- 172.21.15.100:0/2530535412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640078964d0 msgr2=0x564007c6a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.835+0000 7f7919349640 1 --2- 172.21.15.100:0/2530535412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640078964d0 0x564007c6a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.835+0000 7f7919349640 1 -- 172.21.15.100:0/2530535412 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564007922790 con 0x56400788e2f0 2024-07-26T13:40:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.835+0000 7f7918b48640 1 --2- 172.21.15.100:0/2530535412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640078964d0 0x564007c6a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:40:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.835+0000 7f7919349640 1 --2- 172.21.15.100:0/2530535412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56400788e2f0 0x564007c69c40 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f790c009900 tx=0x7f790c0314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e5c5d2aaa89aa1 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.835+0000 7f78f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:40:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.835+0000 7f78f8ff9640 1 -- 172.21.15.100:0/2530535412 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c035040 con 0x56400788e2f0 2024-07-26T13:40:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.835+0000 7f78f8ff9640 1 -- 172.21.15.100:0/2530535412 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f790c031790 con 0x56400788e2f0 2024-07-26T13:40:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.835+0000 7f79225a7740 1 -- 172.21.15.100:0/2530535412 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f790400fcf0 con 0x56400788e2f0 2024-07-26T13:40:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.835+0000 7f78f8ff9640 1 -- 172.21.15.100:0/2530535412 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f790c004040 con 0x56400788e2f0 2024-07-26T13:40:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.835+0000 7f79225a7740 10 client.15534 fetch_fsmap learned FSMap version 26 2024-07-26T13:40:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.835+0000 7f79225a7740 1 -- 172.21.15.100:0/2530535412 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56400799b620 con 0x56400788e2f0 2024-07-26T13:40:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.835+0000 7f78f8ff9640 1 -- 172.21.15.100:0/2530535412 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f790400fcf0 con 0x56400788e2f0 2024-07-26T13:40:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.836+0000 7f78f8ff9640 1 -- 172.21.15.100:0/2530535412 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f790c0047e0 con 0x56400788e2f0 2024-07-26T13:40:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.836+0000 7f79225a7740 10 client.15534 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:40:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.836+0000 7f79225a7740 10 client.15534 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:40:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.836+0000 7f79225a7740 20 client.15534 populate_metadata read hostname 'smithi100' 2024-07-26T13:40:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.836+0000 7f79225a7740 1 --2- 172.21.15.100:0/2530535412 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x564007875d80 0x564007c7ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.836+0000 7f79225a7740 4 client.15534 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:40:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.836+0000 7f79225a7740 1 -- 172.21.15.100:0/2530535412 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564007bd1300 con 0x564007875d80 2024-07-26T13:40:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.836+0000 7f7918b48640 1 --2- 172.21.15.100:0/2530535412 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x564007875d80 0x564007c7ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.837+0000 7f78f8ff9640 10 client.15534 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:40:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.837+0000 7f7918b48640 1 --2- 172.21.15.100:0/2530535412 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x564007875d80 0x564007c7ba90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ebb3e31d2f715e server_cookie=d499ce6b2919d67e in_seq=0 out_seq=0 2024-07-26T13:40:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.837+0000 7f78f8ff9640 1 -- 172.21.15.100:0/2530535412 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564007bd1300 con 0x564007875d80 2024-07-26T13:40:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.837+0000 7f78f8ff9640 10 client.15534 handle_command_reply: tid=0 2024-07-26T13:40:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.837+0000 7f79225a7740 1 client.15534 shutdown 2024-07-26T13:40:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.837+0000 7f79225a7740 1 -- 172.21.15.100:0/2530535412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56400788e2f0 msgr2=0x564007c69c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.837+0000 7f79225a7740 1 --2- 172.21.15.100:0/2530535412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56400788e2f0 0x564007c69c40 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f790c009900 tx=0x7f790c0314a0 comp rx=0 tx=0).stop 2024-07-26T13:40:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.838+0000 7f79225a7740 1 -- 172.21.15.100:0/2530535412 shutdown_connections 2024-07-26T13:40:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.838+0000 7f79225a7740 1 --2- 172.21.15.100:0/2530535412 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x564007875d80 0x564007c7ba90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.838+0000 7f79225a7740 1 --2- 172.21.15.100:0/2530535412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5640078b71b0 0x564007c75b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.838+0000 7f79225a7740 1 --2- 172.21.15.100:0/2530535412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640078964d0 0x564007c6a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.838+0000 7f79225a7740 1 --2- 172.21.15.100:0/2530535412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56400788e2f0 0x564007c69c40 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.838+0000 7f78f8ff9640 0 client.15534 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:40:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.838+0000 7f79225a7740 1 -- 172.21.15.100:0/2530535412 >> 172.21.15.100:0/2530535412 conn(0x5640078ab300 msgr2=0x564007c562b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.838+0000 7f79225a7740 1 -- 172.21.15.100:0/2530535412 shutdown_connections 2024-07-26T13:40:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.838+0000 7f79225a7740 1 -- 172.21.15.100:0/2530535412 wait complete. 2024-07-26T13:40:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.838+0000 7f79225a7740 20 client.15534 trim_cache size 0 max 16384 2024-07-26T13:40:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.853+0000 7f79225a7740 1 Processor -- start 2024-07-26T13:40:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.853+0000 7f79225a7740 1 -- start start 2024-07-26T13:40:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.853+0000 7f79225a7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56400788e2f0 0x564007c57b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.853+0000 7f79225a7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640078964d0 0x564007c69f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.853+0000 7f79225a7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640078b71b0 0x564007c5b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.853+0000 7f79225a7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f79141af7e0 con 0x5640078b71b0 2024-07-26T13:40:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.853+0000 7f79225a7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f79141ad0b0 con 0x5640078964d0 2024-07-26T13:40:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.853+0000 7f79225a7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f79141aaf00 con 0x56400788e2f0 2024-07-26T13:40:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.853+0000 7f7919b4a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640078b71b0 0x564007c5b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.853+0000 7f7919b4a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640078b71b0 0x564007c5b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42282/0 (socket says 172.21.15.100:42282) 2024-07-26T13:40:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.853+0000 7f7919b4a640 1 -- 172.21.15.100:0/1641677905 learned_addr learned my addr 172.21.15.100:0/1641677905 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.853+0000 7f7918b48640 1 --2- 172.21.15.100:0/1641677905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640078964d0 0x564007c69f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.853+0000 7f7919349640 1 --2- 172.21.15.100:0/1641677905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56400788e2f0 0x564007c57b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.854+0000 7f7919b4a640 1 -- 172.21.15.100:0/1641677905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56400788e2f0 msgr2=0x564007c57b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.854+0000 7f7919b4a640 1 --2- 172.21.15.100:0/1641677905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56400788e2f0 0x564007c57b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.854+0000 7f7919b4a640 1 -- 172.21.15.100:0/1641677905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640078964d0 msgr2=0x564007c69f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.854+0000 7f7919b4a640 1 --2- 172.21.15.100:0/1641677905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640078964d0 0x564007c69f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.854+0000 7f7919b4a640 1 -- 172.21.15.100:0/1641677905 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564007c5cad0 con 0x5640078b71b0 2024-07-26T13:40:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.854+0000 7f7919349640 1 --2- 172.21.15.100:0/1641677905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56400788e2f0 0x564007c57b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:40:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.854+0000 7f7919b4a640 1 --2- 172.21.15.100:0/1641677905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640078b71b0 0x564007c5b420 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7904005fe0 tx=0x7f7904009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ead6baf9c56fbd server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.854+0000 7f78faffd640 1 -- 172.21.15.100:0/1641677905 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7904036070 con 0x5640078b71b0 2024-07-26T13:40:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.854+0000 7f78faffd640 1 -- 172.21.15.100:0/1641677905 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f790401b430 con 0x5640078b71b0 2024-07-26T13:40:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.854+0000 7f78faffd640 1 -- 172.21.15.100:0/1641677905 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790401be60 con 0x5640078b71b0 2024-07-26T13:40:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.855+0000 7f79225a7740 1 -- 172.21.15.100:0/1641677905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640078b71b0 msgr2=0x564007c5b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:48.858 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.855+0000 7f79225a7740 1 --2- 172.21.15.100:0/1641677905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640078b71b0 0x564007c5b420 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7904005fe0 tx=0x7f7904009b30 comp rx=0 tx=0).stop 2024-07-26T13:40:48.858 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.855+0000 7f79225a7740 1 -- 172.21.15.100:0/1641677905 shutdown_connections 2024-07-26T13:40:48.858 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.855+0000 7f79225a7740 1 --2- 172.21.15.100:0/1641677905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640078b71b0 0x564007c5b420 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:48.858 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.855+0000 7f79225a7740 1 --2- 172.21.15.100:0/1641677905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640078964d0 0x564007c69f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:48.858 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.855+0000 7f79225a7740 1 --2- 172.21.15.100:0/1641677905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56400788e2f0 0x564007c57b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:48.858 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.855+0000 7f79225a7740 1 -- 172.21.15.100:0/1641677905 >> 172.21.15.100:0/1641677905 conn(0x5640078ab300 msgr2=0x564007c5ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:48.858 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.855+0000 7f79225a7740 1 -- 172.21.15.100:0/1641677905 shutdown_connections 2024-07-26T13:40:48.858 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.855+0000 7f79225a7740 1 -- 172.21.15.100:0/1641677905 wait complete. 2024-07-26T13:40:48.858 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.855+0000 7f79225a7740 1 -- start start 2024-07-26T13:40:48.858 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.855+0000 7f79225a7740 1 Processor -- start 2024-07-26T13:40:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.855+0000 7f79225a7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56400788e2f0 0x564007c6eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.855+0000 7f79225a7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640078964d0 0x564007c6f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.856+0000 7f79225a7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640078b71b0 0x564007ce2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.856+0000 7f79225a7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f79141ab570 con 0x5640078b71b0 2024-07-26T13:40:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.856+0000 7f79225a7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f79141ac9e0 con 0x5640078964d0 2024-07-26T13:40:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.856+0000 7f79225a7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f79141abc40 con 0x56400788e2f0 2024-07-26T13:40:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.856+0000 7f79225a7740 10 client.0 _ll_register_callbacks cb 0x7f790c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:40:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.856+0000 7f7919b4a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640078b71b0 0x564007ce2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.856+0000 7f7919b4a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640078b71b0 0x564007ce2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42284/0 (socket says 172.21.15.100:42284) 2024-07-26T13:40:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.856+0000 7f7919b4a640 1 -- 172.21.15.100:0/1127692973 learned_addr learned my addr 172.21.15.100:0/1127692973 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.856+0000 7f7918b48640 1 --2- 172.21.15.100:0/1127692973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640078964d0 0x564007c6f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.856+0000 7f7919349640 1 --2- 172.21.15.100:0/1127692973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56400788e2f0 0x564007c6eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.856+0000 7f7919b4a640 1 -- 172.21.15.100:0/1127692973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56400788e2f0 msgr2=0x564007c6eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.856+0000 7f7919b4a640 1 --2- 172.21.15.100:0/1127692973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56400788e2f0 0x564007c6eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.856+0000 7f7919b4a640 1 -- 172.21.15.100:0/1127692973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640078964d0 msgr2=0x564007c6f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.856+0000 7f7919b4a640 1 --2- 172.21.15.100:0/1127692973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640078964d0 0x564007c6f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.856+0000 7f7919b4a640 1 -- 172.21.15.100:0/1127692973 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564007c6fcf0 con 0x5640078b71b0 2024-07-26T13:40:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.856+0000 7f7919b4a640 1 --2- 172.21.15.100:0/1127692973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640078b71b0 0x564007ce2260 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f790401bfd0 tx=0x7f7904005ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8a3794d065be62 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.856+0000 7f78f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:40:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.856+0000 7f78f8ff9640 1 -- 172.21.15.100:0/1127692973 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7904036070 con 0x5640078b71b0 2024-07-26T13:40:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.856+0000 7f78f8ff9640 1 -- 172.21.15.100:0/1127692973 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f7904004df0 con 0x5640078b71b0 2024-07-26T13:40:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.856+0000 7f79225a7740 1 -- 172.21.15.100:0/1127692973 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f790401b430 con 0x5640078b71b0 2024-07-26T13:40:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.856+0000 7f78f8ff9640 1 -- 172.21.15.100:0/1127692973 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7904017070 con 0x5640078b71b0 2024-07-26T13:40:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.856+0000 7f79225a7740 10 client.15540 fetch_fsmap learned FSMap version 26 2024-07-26T13:40:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.856+0000 7f79225a7740 1 -- 172.21.15.100:0/1127692973 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564007ce3780 con 0x5640078b71b0 2024-07-26T13:40:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.857+0000 7f78f8ff9640 1 -- 172.21.15.100:0/1127692973 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f790401b430 con 0x5640078b71b0 2024-07-26T13:40:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.857+0000 7f78f8ff9640 1 -- 172.21.15.100:0/1127692973 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f7904041020 con 0x5640078b71b0 2024-07-26T13:40:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.857+0000 7f79225a7740 10 client.15540 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:40:48.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.857+0000 7f79225a7740 10 client.15540 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:40:48.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.857+0000 7f79225a7740 20 client.15540 populate_metadata read hostname 'smithi100' 2024-07-26T13:40:48.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.857+0000 7f79225a7740 1 --2- 172.21.15.100:0/1127692973 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56400784b1c0 0x564007ce5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:48.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.857+0000 7f79225a7740 4 client.15540 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-26T13:40:48.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.857+0000 7f79225a7740 1 -- 172.21.15.100:0/1127692973 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564007bccda0 con 0x56400784b1c0 2024-07-26T13:40:48.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.857+0000 7f7919349640 1 --2- 172.21.15.100:0/1127692973 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56400784b1c0 0x564007ce5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:48.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.858+0000 7f78f8ff9640 10 client.15540 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:40:48.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.858+0000 7f7919349640 1 --2- 172.21.15.100:0/1127692973 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56400784b1c0 0x564007ce5dd0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1c65ad523d7bc1 server_cookie=ffc7aab99c186045 in_seq=0 out_seq=0 2024-07-26T13:40:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.865+0000 7f78f8ff9640 1 -- 172.21.15.100:0/1127692973 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564007bccda0 con 0x56400784b1c0 2024-07-26T13:40:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.865+0000 7f78f8ff9640 10 client.15540 handle_command_reply: tid=0 2024-07-26T13:40:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.865+0000 7f79225a7740 1 client.15540 shutdown 2024-07-26T13:40:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.865+0000 7f79225a7740 1 -- 172.21.15.100:0/1127692973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640078b71b0 msgr2=0x564007ce2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.865+0000 7f79225a7740 1 --2- 172.21.15.100:0/1127692973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640078b71b0 0x564007ce2260 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f790401bfd0 tx=0x7f7904005ac0 comp rx=0 tx=0).stop 2024-07-26T13:40:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.865+0000 7f79225a7740 1 -- 172.21.15.100:0/1127692973 shutdown_connections 2024-07-26T13:40:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.865+0000 7f79225a7740 1 --2- 172.21.15.100:0/1127692973 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56400784b1c0 0x564007ce5dd0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.865+0000 7f79225a7740 1 --2- 172.21.15.100:0/1127692973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5640078b71b0 0x564007ce2260 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.865+0000 7f79225a7740 1 --2- 172.21.15.100:0/1127692973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5640078964d0 0x564007c6f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.865+0000 7f78f8ff9640 0 client.15540 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:40:48.868 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.865+0000 7f79225a7740 1 --2- 172.21.15.100:0/1127692973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56400788e2f0 0x564007c6eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:48.868 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.865+0000 7f79225a7740 1 -- 172.21.15.100:0/1127692973 >> 172.21.15.100:0/1127692973 conn(0x5640078ab300 msgr2=0x564007c57930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:48.868 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.866+0000 7f79225a7740 1 -- 172.21.15.100:0/1127692973 shutdown_connections 2024-07-26T13:40:48.868 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.866+0000 7f79225a7740 1 -- 172.21.15.100:0/1127692973 wait complete. 2024-07-26T13:40:48.868 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.866+0000 7f79225a7740 20 client.15540 trim_cache size 0 max 16384 2024-07-26T13:40:48.868 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-07-26T13:40:48.868 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-07-26T13:40:48.868 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-07-26T13:40:48.868 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-07-26T13:40:48.869 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.866+0000 7f791b5ad640 1 -- 172.21.15.100:0/2088332283 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f78fc0b3cb0 msgr2=0x7f78fc0b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:48.869 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.866+0000 7f791b5ad640 1 --2- 172.21.15.100:0/2088332283 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f78fc0b3cb0 0x7f78fc0b61c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7904022000 tx=0x7f7904023000 comp rx=0 tx=0).stop 2024-07-26T13:40:48.869 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.866+0000 7f791b5ad640 1 -- 172.21.15.100:0/2088332283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7914108e30 msgr2=0x7f791419a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.866+0000 7f791b5ad640 1 --2- 172.21.15.100:0/2088332283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7914108e30 0x7f791419a450 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f790800c970 tx=0x7f790800ce30 comp rx=0 tx=0).stop 2024-07-26T13:40:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.866+0000 7f791b5ad640 1 -- 172.21.15.100:0/2088332283 shutdown_connections 2024-07-26T13:40:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.866+0000 7f791b5ad640 1 --2- 172.21.15.100:0/2088332283 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f78fc0b3cb0 0x7f78fc0b61c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.866+0000 7f791b5ad640 1 --2- 172.21.15.100:0/2088332283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7914109780 0x7f791419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.866+0000 7f791b5ad640 1 --2- 172.21.15.100:0/2088332283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7914108e30 0x7f791419a450 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.866+0000 7f791b5ad640 1 --2- 172.21.15.100:0/2088332283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7914103500 0x7f7914199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.866+0000 7f791b5ad640 1 -- 172.21.15.100:0/2088332283 >> 172.21.15.100:0/2088332283 conn(0x7f79140fef40 msgr2=0x7f7914101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.866+0000 7f791b5ad640 1 -- 172.21.15.100:0/2088332283 shutdown_connections 2024-07-26T13:40:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:40:48.866+0000 7f791b5ad640 1 -- 172.21.15.100:0/2088332283 wait complete. 2024-07-26T13:40:49.621 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:49 smithi100 ceph-mon[32599]: pgmap v385: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-07-26T13:40:49.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:49 smithi167 ceph-mon[37140]: pgmap v385: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-07-26T13:40:49.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:49 smithi138 ceph-mon[36954]: pgmap v385: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-07-26T13:40:50.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.005+0000 7f11d7853640 1 -- 172.21.15.100:0/56989009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11d0107dd0 msgr2=0x7f11d01081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:50.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.005+0000 7f11d7853640 1 --2- 172.21.15.100:0/56989009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11d0107dd0 0x7f11d01081e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f11c0009900 tx=0x7f11c002a410 comp rx=0 tx=0).stop 2024-07-26T13:40:50.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.005+0000 7f11d7853640 1 -- 172.21.15.100:0/56989009 shutdown_connections 2024-07-26T13:40:50.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.005+0000 7f11d7853640 1 --2- 172.21.15.100:0/56989009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11d0108720 0x7f11d010ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:50.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.005+0000 7f11d7853640 1 --2- 172.21.15.100:0/56989009 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11d0107dd0 0x7f11d01081e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:50.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.005+0000 7f11d7853640 1 --2- 172.21.15.100:0/56989009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11d0102490 0x7f11d01028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:50.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.005+0000 7f11d7853640 1 -- 172.21.15.100:0/56989009 >> 172.21.15.100:0/56989009 conn(0x7f11d0079160 msgr2=0x7f11d0079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:50.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.005+0000 7f11d7853640 1 -- 172.21.15.100:0/56989009 shutdown_connections 2024-07-26T13:40:50.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.005+0000 7f11d7853640 1 -- 172.21.15.100:0/56989009 wait complete. 2024-07-26T13:40:50.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.006+0000 7f11d7853640 1 Processor -- start 2024-07-26T13:40:50.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.006+0000 7f11d7853640 1 -- start start 2024-07-26T13:40:50.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.006+0000 7f11d7853640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11d0102490 0x7f11d0199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:50.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.006+0000 7f11d7853640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11d0107dd0 0x7f11d019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:50.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.006+0000 7f11d7853640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11d0108720 0x7f11d019f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:50.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.006+0000 7f11d7853640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f11d0112350 con 0x7f11d0107dd0 2024-07-26T13:40:50.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.006+0000 7f11d7853640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f11d01121d0 con 0x7f11d0102490 2024-07-26T13:40:50.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.006+0000 7f11d7853640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f11d01124d0 con 0x7f11d0108720 2024-07-26T13:40:50.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.006+0000 7f11d4dee640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11d0107dd0 0x7f11d019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:50.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.006+0000 7f11d4dee640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11d0107dd0 0x7f11d019a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42312/0 (socket says 172.21.15.100:42312) 2024-07-26T13:40:50.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.006+0000 7f11d4dee640 1 -- 172.21.15.100:0/1994037649 learned_addr learned my addr 172.21.15.100:0/1994037649 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:50.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.006+0000 7f11d55ef640 1 --2- 172.21.15.100:0/1994037649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11d0102490 0x7f11d0199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:50.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.007+0000 7f11d5df0640 1 --2- 172.21.15.100:0/1994037649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11d0108720 0x7f11d019f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:50.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.007+0000 7f11d4dee640 1 -- 172.21.15.100:0/1994037649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11d0108720 msgr2=0x7f11d019f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:50.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.007+0000 7f11d4dee640 1 --2- 172.21.15.100:0/1994037649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11d0108720 0x7f11d019f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:50.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.007+0000 7f11d4dee640 1 -- 172.21.15.100:0/1994037649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11d0102490 msgr2=0x7f11d0199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:50.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.007+0000 7f11d4dee640 1 --2- 172.21.15.100:0/1994037649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11d0102490 0x7f11d0199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:50.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.007+0000 7f11d4dee640 1 -- 172.21.15.100:0/1994037649 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d019fee0 con 0x7f11d0107dd0 2024-07-26T13:40:50.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.007+0000 7f11d4dee640 1 --2- 172.21.15.100:0/1994037649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11d0107dd0 0x7f11d019a510 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f11c0032fd0 tx=0x7f11c0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:50.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.007+0000 7f11c67fc640 1 -- 172.21.15.100:0/1994037649 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c003f030 con 0x7f11d0107dd0 2024-07-26T13:40:50.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.007+0000 7f11c67fc640 1 -- 172.21.15.100:0/1994037649 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11c0004490 con 0x7f11d0107dd0 2024-07-26T13:40:50.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.007+0000 7f11d7853640 1 -- 172.21.15.100:0/1994037649 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d01a0170 con 0x7f11d0107dd0 2024-07-26T13:40:50.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.007+0000 7f11c67fc640 1 -- 172.21.15.100:0/1994037649 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c0035070 con 0x7f11d0107dd0 2024-07-26T13:40:50.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.007+0000 7f11d7853640 1 -- 172.21.15.100:0/1994037649 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d01a6930 con 0x7f11d0107dd0 2024-07-26T13:40:50.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.010+0000 7f11dddd7740 1 Processor -- start 2024-07-26T13:40:50.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.010+0000 7f11dddd7740 1 -- start start 2024-07-26T13:40:50.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.011+0000 7f11dddd7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9100c9f60 0x55e91046f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:50.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.011+0000 7f11dddd7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9100a10a0 0x55e91046f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:50.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.011+0000 7f11dddd7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9100a9280 0x55e91046fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:50.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.011+0000 7f11dddd7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55e9103e5920 con 0x55e9100c9f60 2024-07-26T13:40:50.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.011+0000 7f11dddd7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e9103d4e20 con 0x55e9100a10a0 2024-07-26T13:40:50.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.011+0000 7f11dddd7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e9103d8e60 con 0x55e9100a9280 2024-07-26T13:40:50.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.011+0000 7f11c67fc640 1 -- 172.21.15.100:0/1994037649 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 151134+0+0 (secure 0 0 0) 0x7f11c0042030 con 0x7f11d0107dd0 2024-07-26T13:40:50.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.011+0000 7f11d5df0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9100c9f60 0x55e91046f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:50.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.011+0000 7f11d5df0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9100c9f60 0x55e91046f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42324/0 (socket says 172.21.15.100:42324) 2024-07-26T13:40:50.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.011+0000 7f11d5df0640 1 -- 172.21.15.100:0/2388903705 learned_addr learned my addr 172.21.15.100:0/2388903705 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:50.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.011+0000 7f11d55ef640 1 --2- 172.21.15.100:0/2388903705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9100a10a0 0x55e91046f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:50.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.011+0000 7f11d4dee640 1 --2- 172.21.15.100:0/2388903705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9100a9280 0x55e91046fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:50.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.011+0000 7f11d5df0640 1 -- 172.21.15.100:0/2388903705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9100a9280 msgr2=0x55e91046fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:50.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.011+0000 7f11d5df0640 1 --2- 172.21.15.100:0/2388903705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9100a9280 0x55e91046fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:50.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.011+0000 7f11d5df0640 1 -- 172.21.15.100:0/2388903705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9100a10a0 msgr2=0x55e91046f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:50.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.011+0000 7f11d5df0640 1 --2- 172.21.15.100:0/2388903705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9100a10a0 0x55e91046f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:50.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.011+0000 7f11d5df0640 1 -- 172.21.15.100:0/2388903705 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e91037ba40 con 0x55e9100c9f60 2024-07-26T13:40:50.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.011+0000 7f11d4dee640 1 --2- 172.21.15.100:0/2388903705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9100a9280 0x55e91046fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:40:50.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.011+0000 7f11d5df0640 1 --2- 172.21.15.100:0/2388903705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9100c9f60 0x55e91046f380 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f11c8009f20 tx=0x7f11c800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4478967f8015a4c server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:50.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.012+0000 7f11bb7fe640 1 -- 172.21.15.100:0/2388903705 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c8015070 con 0x55e9100c9f60 2024-07-26T13:40:50.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.012+0000 7f11bb7fe640 1 -- 172.21.15.100:0/2388903705 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11c801ab10 con 0x55e9100c9f60 2024-07-26T13:40:50.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.012+0000 7f11bb7fe640 1 -- 172.21.15.100:0/2388903705 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c8011bc0 con 0x55e9100c9f60 2024-07-26T13:40:50.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.012+0000 7f11c67fc640 1 --2- 172.21.15.100:0/1994037649 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f11bc0b3cb0 0x7f11bc0b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:50.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.012+0000 7f11d55ef640 1 --2- 172.21.15.100:0/1994037649 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f11bc0b3cb0 0x7f11bc0b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:50.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.012+0000 7f11c67fc640 1 -- 172.21.15.100:0/1994037649 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f11c01043e0 con 0x7f11d0107dd0 2024-07-26T13:40:50.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.013+0000 7f11dddd7740 1 -- 172.21.15.100:0/2388903705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9100c9f60 msgr2=0x55e91046f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:50.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.013+0000 7f11d55ef640 1 --2- 172.21.15.100:0/1994037649 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f11bc0b3cb0 0x7f11bc0b61c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x55e91046e000 tx=0x7f11cc00b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:50.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.013+0000 7f11dddd7740 1 --2- 172.21.15.100:0/2388903705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9100c9f60 0x55e91046f380 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f11c8009f20 tx=0x7f11c800f7f0 comp rx=0 tx=0).stop 2024-07-26T13:40:50.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.013+0000 7f11dddd7740 1 -- 172.21.15.100:0/2388903705 shutdown_connections 2024-07-26T13:40:50.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.013+0000 7f11dddd7740 1 --2- 172.21.15.100:0/2388903705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9100a9280 0x55e91046fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:50.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.013+0000 7f11dddd7740 1 --2- 172.21.15.100:0/2388903705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9100a10a0 0x55e91046f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:50.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.013+0000 7f11dddd7740 1 --2- 172.21.15.100:0/2388903705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9100c9f60 0x55e91046f380 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:50.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.013+0000 7f11dddd7740 1 -- 172.21.15.100:0/2388903705 >> 172.21.15.100:0/2388903705 conn(0x55e9100be350 msgr2=0x55e91046d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:50.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.013+0000 7f11dddd7740 1 -- 172.21.15.100:0/2388903705 shutdown_connections 2024-07-26T13:40:50.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.013+0000 7f11dddd7740 1 -- 172.21.15.100:0/2388903705 wait complete. 2024-07-26T13:40:50.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.013+0000 7f11dddd7740 1 -- start start 2024-07-26T13:40:50.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.013+0000 7f11dddd7740 1 Processor -- start 2024-07-26T13:40:50.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.014+0000 7f11dddd7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9100a10a0 0x55e91047afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:50.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.014+0000 7f11dddd7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9100a9280 0x55e91047b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:50.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.014+0000 7f11dddd7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9100c9f60 0x55e910486e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:50.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.014+0000 7f11dddd7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55e9102385f0 con 0x55e9100a10a0 2024-07-26T13:40:50.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.014+0000 7f11dddd7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e910377890 con 0x55e9100a9280 2024-07-26T13:40:50.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.014+0000 7f11dddd7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e91047db60 con 0x55e9100c9f60 2024-07-26T13:40:50.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.014+0000 7f11dddd7740 10 client.0 _ll_register_callbacks cb 0x55e9103817d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:40:50.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.014+0000 7f11d55ef640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9100a10a0 0x55e91047afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:50.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.014+0000 7f11d55ef640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9100a10a0 0x55e91047afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42334/0 (socket says 172.21.15.100:42334) 2024-07-26T13:40:50.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.014+0000 7f11d55ef640 1 -- 172.21.15.100:0/2573164237 learned_addr learned my addr 172.21.15.100:0/2573164237 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:50.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.014+0000 7f11d5df0640 1 --2- 172.21.15.100:0/2573164237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9100c9f60 0x55e910486e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:50.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.014+0000 7f11d4dee640 1 --2- 172.21.15.100:0/2573164237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9100a9280 0x55e91047b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:50.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.014+0000 7f11d55ef640 1 -- 172.21.15.100:0/2573164237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9100c9f60 msgr2=0x55e910486e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:50.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.014+0000 7f11d55ef640 1 --2- 172.21.15.100:0/2573164237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9100c9f60 0x55e910486e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:50.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.014+0000 7f11d55ef640 1 -- 172.21.15.100:0/2573164237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9100a9280 msgr2=0x55e91047b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:50.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.014+0000 7f11d55ef640 1 --2- 172.21.15.100:0/2573164237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9100a9280 0x55e91047b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:50.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.014+0000 7f11d55ef640 1 -- 172.21.15.100:0/2573164237 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e910135540 con 0x55e9100a10a0 2024-07-26T13:40:50.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.014+0000 7f11d4dee640 1 --2- 172.21.15.100:0/2573164237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9100a9280 0x55e91047b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:40:50.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.014+0000 7f11d55ef640 1 --2- 172.21.15.100:0/2573164237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9100a10a0 0x55e91047afd0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f11cc01f4a0 tx=0x7f11cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe4789a69c22163d server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:50.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.014+0000 7f11b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:40:50.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.014+0000 7f11b8ff9640 1 -- 172.21.15.100:0/2573164237 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11cc02c060 con 0x55e9100a10a0 2024-07-26T13:40:50.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.014+0000 7f11b8ff9640 1 -- 172.21.15.100:0/2573164237 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f11cc017030 con 0x55e9100a10a0 2024-07-26T13:40:50.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.014+0000 7f11dddd7740 1 -- 172.21.15.100:0/2573164237 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11c801ab10 con 0x55e9100a10a0 2024-07-26T13:40:50.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.015+0000 7f11b8ff9640 1 -- 172.21.15.100:0/2573164237 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f11cc01c600 con 0x55e9100a10a0 2024-07-26T13:40:50.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.015+0000 7f11dddd7740 10 client.15552 fetch_fsmap learned FSMap version 26 2024-07-26T13:40:50.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.015+0000 7f11dddd7740 1 -- 172.21.15.100:0/2573164237 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e910489bc0 con 0x55e9100a10a0 2024-07-26T13:40:50.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.015+0000 7f11b8ff9640 1 -- 172.21.15.100:0/2573164237 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11c801ab10 con 0x55e9100a10a0 2024-07-26T13:40:50.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.015+0000 7f11b8ff9640 1 -- 172.21.15.100:0/2573164237 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f11cc0308e0 con 0x55e9100a10a0 2024-07-26T13:40:50.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.015+0000 7f11dddd7740 10 client.15552 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:40:50.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.015+0000 7f11dddd7740 10 client.15552 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-26T13:40:50.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.015+0000 7f11dddd7740 20 client.15552 populate_metadata read hostname 'smithi100' 2024-07-26T13:40:50.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.015+0000 7f11dddd7740 1 --2- 172.21.15.100:0/2573164237 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55e9100888e0 0x55e91048d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:50.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.015+0000 7f11dddd7740 4 client.15552 mds_command: new command op to 24595 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:40:50.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.015+0000 7f11dddd7740 1 -- 172.21.15.100:0/2573164237 --> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9103e7870 con 0x55e9100888e0 2024-07-26T13:40:50.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.016+0000 7f11d4dee640 1 --2- 172.21.15.100:0/2573164237 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55e9100888e0 0x55e91048d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:50.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.016+0000 7f11d4dee640 1 --2- 172.21.15.100:0/2573164237 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55e9100888e0 0x55e91048d6d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=791612614bcd066 server_cookie=8ac5107bf46891bb in_seq=0 out_seq=0 2024-07-26T13:40:50.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.016+0000 7f11b8ff9640 10 client.15552 ms_handle_connect on v2:172.21.15.167:6836/3828634468 2024-07-26T13:40:50.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.017+0000 7f11b8ff9640 1 -- 172.21.15.100:0/2573164237 <== mds.2 v2:172.21.15.167:6836/3828634468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e9103e7870 con 0x55e9100888e0 2024-07-26T13:40:50.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.017+0000 7f11b8ff9640 10 client.15552 handle_command_reply: tid=0 2024-07-26T13:40:50.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.017+0000 7f11dddd7740 1 client.15552 shutdown 2024-07-26T13:40:50.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.017+0000 7f11dddd7740 1 -- 172.21.15.100:0/2573164237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9100a10a0 msgr2=0x55e91047afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:50.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.017+0000 7f11dddd7740 1 --2- 172.21.15.100:0/2573164237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9100a10a0 0x55e91047afd0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f11cc01f4a0 tx=0x7f11cc004730 comp rx=0 tx=0).stop 2024-07-26T13:40:50.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.017+0000 7f11dddd7740 1 -- 172.21.15.100:0/2573164237 shutdown_connections 2024-07-26T13:40:50.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.017+0000 7f11dddd7740 1 --2- 172.21.15.100:0/2573164237 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55e9100888e0 0x55e91048d6d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:50.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.017+0000 7f11dddd7740 1 --2- 172.21.15.100:0/2573164237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9100c9f60 0x55e910486e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:50.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.017+0000 7f11dddd7740 1 --2- 172.21.15.100:0/2573164237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9100a9280 0x55e91047b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:50.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.017+0000 7f11dddd7740 1 --2- 172.21.15.100:0/2573164237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9100a10a0 0x55e91047afd0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:50.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.017+0000 7f11dddd7740 1 -- 172.21.15.100:0/2573164237 >> 172.21.15.100:0/2573164237 conn(0x55e9100be350 msgr2=0x55e910472560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:50.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.017+0000 7f11b8ff9640 0 client.15552 ms_handle_reset on v2:172.21.15.167:6836/3828634468 2024-07-26T13:40:50.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.017+0000 7f11dddd7740 1 -- 172.21.15.100:0/2573164237 shutdown_connections 2024-07-26T13:40:50.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.017+0000 7f11dddd7740 1 -- 172.21.15.100:0/2573164237 wait complete. 2024-07-26T13:40:50.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.017+0000 7f11dddd7740 20 client.15552 trim_cache size 0 max 16384 2024-07-26T13:40:50.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.035+0000 7f11dddd7740 1 Processor -- start 2024-07-26T13:40:50.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.035+0000 7f11dddd7740 1 -- start start 2024-07-26T13:40:50.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.035+0000 7f11dddd7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9100a10a0 0x55e91048c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:50.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.035+0000 7f11dddd7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9100a9280 0x55e910486df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:50.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.035+0000 7f11dddd7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9100c9f60 0x55e910467eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:50.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.035+0000 7f11dddd7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55e91048ba10 con 0x55e9100a9280 2024-07-26T13:40:50.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.035+0000 7f11dddd7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e91048c0e0 con 0x55e9100c9f60 2024-07-26T13:40:50.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.035+0000 7f11dddd7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e91048c7b0 con 0x55e9100a10a0 2024-07-26T13:40:50.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.035+0000 7f11d4dee640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9100a9280 0x55e910486df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:50.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.035+0000 7f11d4dee640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9100a9280 0x55e910486df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42346/0 (socket says 172.21.15.100:42346) 2024-07-26T13:40:50.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.035+0000 7f11d4dee640 1 -- 172.21.15.100:0/53777305 learned_addr learned my addr 172.21.15.100:0/53777305 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:50.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.036+0000 7f11d5df0640 1 --2- 172.21.15.100:0/53777305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9100c9f60 0x55e910467eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:50.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.036+0000 7f11d55ef640 1 --2- 172.21.15.100:0/53777305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9100a10a0 0x55e91048c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:50.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.036+0000 7f11d4dee640 1 -- 172.21.15.100:0/53777305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9100a10a0 msgr2=0x55e91048c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:50.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.036+0000 7f11d4dee640 1 --2- 172.21.15.100:0/53777305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9100a10a0 0x55e91048c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:50.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.036+0000 7f11d4dee640 1 -- 172.21.15.100:0/53777305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9100c9f60 msgr2=0x55e910467eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:50.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.036+0000 7f11d4dee640 1 --2- 172.21.15.100:0/53777305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9100c9f60 0x55e910467eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:50.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.036+0000 7f11d4dee640 1 -- 172.21.15.100:0/53777305 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9104792d0 con 0x55e9100a9280 2024-07-26T13:40:50.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.036+0000 7f11d4dee640 1 --2- 172.21.15.100:0/53777305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9100a9280 0x55e910486df0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f11c0102210 tx=0x7f11c002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4dde40ed3da600d server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:50.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.036+0000 7f11baffd640 1 -- 172.21.15.100:0/53777305 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c00b2070 con 0x55e9100a9280 2024-07-26T13:40:50.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.036+0000 7f11baffd640 1 -- 172.21.15.100:0/53777305 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11c00cf6d0 con 0x55e9100a9280 2024-07-26T13:40:50.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.036+0000 7f11baffd640 1 -- 172.21.15.100:0/53777305 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c0049040 con 0x55e9100a9280 2024-07-26T13:40:50.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.037+0000 7f11dddd7740 1 -- 172.21.15.100:0/53777305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9100a9280 msgr2=0x55e910486df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:50.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.037+0000 7f11dddd7740 1 --2- 172.21.15.100:0/53777305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9100a9280 0x55e910486df0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f11c0102210 tx=0x7f11c002a410 comp rx=0 tx=0).stop 2024-07-26T13:40:50.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.037+0000 7f11dddd7740 1 -- 172.21.15.100:0/53777305 shutdown_connections 2024-07-26T13:40:50.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.037+0000 7f11dddd7740 1 --2- 172.21.15.100:0/53777305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9100c9f60 0x55e910467eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:50.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.037+0000 7f11dddd7740 1 --2- 172.21.15.100:0/53777305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9100a9280 0x55e910486df0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:50.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.037+0000 7f11dddd7740 1 --2- 172.21.15.100:0/53777305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9100a10a0 0x55e91048c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:50.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.037+0000 7f11dddd7740 1 -- 172.21.15.100:0/53777305 >> 172.21.15.100:0/53777305 conn(0x55e9100be350 msgr2=0x55e91046cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:50.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.037+0000 7f11dddd7740 1 -- 172.21.15.100:0/53777305 shutdown_connections 2024-07-26T13:40:50.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.037+0000 7f11dddd7740 1 -- 172.21.15.100:0/53777305 wait complete. 2024-07-26T13:40:50.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.037+0000 7f11dddd7740 1 -- start start 2024-07-26T13:40:50.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.037+0000 7f11dddd7740 1 Processor -- start 2024-07-26T13:40:50.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.038+0000 7f11dddd7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9100a10a0 0x55e91048ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:50.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.038+0000 7f11dddd7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9100a9280 0x55e91048f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:50.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.038+0000 7f11dddd7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9100c9f60 0x55e910493bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:50.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.038+0000 7f11dddd7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55e91048d550 con 0x55e9100a9280 2024-07-26T13:40:50.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.038+0000 7f11dddd7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e91048ce80 con 0x55e9100a10a0 2024-07-26T13:40:50.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.038+0000 7f11dddd7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e91048dc20 con 0x55e9100c9f60 2024-07-26T13:40:50.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.038+0000 7f11dddd7740 10 client.0 _ll_register_callbacks cb 0x7f11cc00bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:40:50.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.038+0000 7f11d4dee640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9100a9280 0x55e91048f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:50.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.038+0000 7f11d4dee640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9100a9280 0x55e91048f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42358/0 (socket says 172.21.15.100:42358) 2024-07-26T13:40:50.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.038+0000 7f11d4dee640 1 -- 172.21.15.100:0/2872632278 learned_addr learned my addr 172.21.15.100:0/2872632278 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:50.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.038+0000 7f11d55ef640 1 --2- 172.21.15.100:0/2872632278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9100a10a0 0x55e91048ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:50.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.038+0000 7f11d5df0640 1 --2- 172.21.15.100:0/2872632278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9100c9f60 0x55e910493bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:50.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.038+0000 7f11d4dee640 1 -- 172.21.15.100:0/2872632278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9100c9f60 msgr2=0x55e910493bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:50.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.038+0000 7f11d4dee640 1 --2- 172.21.15.100:0/2872632278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9100c9f60 0x55e910493bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:50.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.038+0000 7f11d4dee640 1 -- 172.21.15.100:0/2872632278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9100a10a0 msgr2=0x55e91048ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:50.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.038+0000 7f11d4dee640 1 --2- 172.21.15.100:0/2872632278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9100a10a0 0x55e91048ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:50.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.038+0000 7f11d4dee640 1 -- 172.21.15.100:0/2872632278 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e910494230 con 0x55e9100a9280 2024-07-26T13:40:50.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.038+0000 7f11d4dee640 1 --2- 172.21.15.100:0/2872632278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9100a9280 0x55e91048f090 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f11c0031440 tx=0x7f11c0102110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f887f21a68335c3a server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:50.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.038+0000 7f11b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:40:50.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.039+0000 7f11b8ff9640 1 -- 172.21.15.100:0/2872632278 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c00e4020 con 0x55e9100a9280 2024-07-26T13:40:50.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.039+0000 7f11b8ff9640 1 -- 172.21.15.100:0/2872632278 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f11c00c73d0 con 0x55e9100a9280 2024-07-26T13:40:50.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.039+0000 7f11dddd7740 1 -- 172.21.15.100:0/2872632278 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11c00cf6d0 con 0x55e9100a9280 2024-07-26T13:40:50.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.039+0000 7f11b8ff9640 1 -- 172.21.15.100:0/2872632278 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f11c00b2070 con 0x55e9100a9280 2024-07-26T13:40:50.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.039+0000 7f11dddd7740 10 client.15558 fetch_fsmap learned FSMap version 26 2024-07-26T13:40:50.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.039+0000 7f11dddd7740 1 -- 172.21.15.100:0/2872632278 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e9104955f0 con 0x55e9100a9280 2024-07-26T13:40:50.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.039+0000 7f11b8ff9640 1 -- 172.21.15.100:0/2872632278 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11c00cf6d0 con 0x55e9100a9280 2024-07-26T13:40:50.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.039+0000 7f11b8ff9640 1 -- 172.21.15.100:0/2872632278 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f11c00ec070 con 0x55e9100a9280 2024-07-26T13:40:50.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.039+0000 7f11dddd7740 10 client.15558 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:40:50.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.039+0000 7f11dddd7740 10 client.15558 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-26T13:40:50.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.039+0000 7f11dddd7740 20 client.15558 populate_metadata read hostname 'smithi100' 2024-07-26T13:40:50.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.039+0000 7f11dddd7740 1 --2- 172.21.15.100:0/2872632278 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55e91005dc90 0x55e9104f8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:50.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.039+0000 7f11dddd7740 4 client.15558 mds_command: new command op to 24595 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-26T13:40:50.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.039+0000 7f11dddd7740 1 -- 172.21.15.100:0/2872632278 --> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] -- command(tid 0: {"prefix": "status"}) -- 0x55e9103e3310 con 0x55e91005dc90 2024-07-26T13:40:50.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.040+0000 7f11d55ef640 1 --2- 172.21.15.100:0/2872632278 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55e91005dc90 0x55e9104f8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:50.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.040+0000 7f11d55ef640 1 --2- 172.21.15.100:0/2872632278 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55e91005dc90 0x55e9104f8d80 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=15bf39fda3f3d3bc server_cookie=8c201390eb4b3631 in_seq=0 out_seq=0 2024-07-26T13:40:50.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.040+0000 7f11b8ff9640 10 client.15558 ms_handle_connect on v2:172.21.15.167:6836/3828634468 2024-07-26T13:40:50.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.040+0000 7f11b8ff9640 1 -- 172.21.15.100:0/2872632278 <== mds.2 v2:172.21.15.167:6836/3828634468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e9103e3310 con 0x55e91005dc90 2024-07-26T13:40:50.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.040+0000 7f11b8ff9640 10 client.15558 handle_command_reply: tid=0 2024-07-26T13:40:50.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.040+0000 7f11dddd7740 1 client.15558 shutdown 2024-07-26T13:40:50.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.040+0000 7f11dddd7740 1 -- 172.21.15.100:0/2872632278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9100a9280 msgr2=0x55e91048f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:50.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.040+0000 7f11dddd7740 1 --2- 172.21.15.100:0/2872632278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9100a9280 0x55e91048f090 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f11c0031440 tx=0x7f11c0102110 comp rx=0 tx=0).stop 2024-07-26T13:40:50.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.040+0000 7f11dddd7740 1 -- 172.21.15.100:0/2872632278 shutdown_connections 2024-07-26T13:40:50.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.040+0000 7f11dddd7740 1 --2- 172.21.15.100:0/2872632278 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55e91005dc90 0x55e9104f8d80 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:50.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.040+0000 7f11dddd7740 1 --2- 172.21.15.100:0/2872632278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9100c9f60 0x55e910493bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:50.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.040+0000 7f11b8ff9640 0 client.15558 ms_handle_reset on v2:172.21.15.167:6836/3828634468 2024-07-26T13:40:50.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.040+0000 7f11dddd7740 1 --2- 172.21.15.100:0/2872632278 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e9100a9280 0x55e91048f090 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:50.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.040+0000 7f11dddd7740 1 --2- 172.21.15.100:0/2872632278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9100a10a0 0x55e91048ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:50.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.040+0000 7f11dddd7740 1 -- 172.21.15.100:0/2872632278 >> 172.21.15.100:0/2872632278 conn(0x55e9100be350 msgr2=0x55e9104794e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:50.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.041+0000 7f11dddd7740 1 -- 172.21.15.100:0/2872632278 shutdown_connections 2024-07-26T13:40:50.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.041+0000 7f11dddd7740 1 -- 172.21.15.100:0/2872632278 wait complete. 2024-07-26T13:40:50.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.041+0000 7f11dddd7740 20 client.15558 trim_cache size 0 max 16384 2024-07-26T13:40:50.047 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:40:50.048 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "bf83e2e6-4b52-11ef-bcb7-c7b262605968", 2024-07-26T13:40:50.048 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 2, 2024-07-26T13:40:50.048 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 24595, 2024-07-26T13:40:50.048 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-07-26T13:40:50.048 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-07-26T13:40:50.048 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-07-26T13:40:50.048 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 215.50872834500001, 2024-07-26T13:40:50.048 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 26, 2024-07-26T13:40:50.048 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 81, 2024-07-26T13:40:50.048 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 81, 2024-07-26T13:40:50.048 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 260.73268060200002 2024-07-26T13:40:50.048 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:40:50.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.041+0000 7f11d7853640 1 -- 172.21.15.100:0/1994037649 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f11bc0b3cb0 msgr2=0x7f11bc0b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:50.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.041+0000 7f11d7853640 1 --2- 172.21.15.100:0/1994037649 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f11bc0b3cb0 0x7f11bc0b61c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x55e91046e000 tx=0x7f11cc00b410 comp rx=0 tx=0).stop 2024-07-26T13:40:50.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.041+0000 7f11d7853640 1 -- 172.21.15.100:0/1994037649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11d0107dd0 msgr2=0x7f11d019a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:50.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.041+0000 7f11d7853640 1 --2- 172.21.15.100:0/1994037649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11d0107dd0 0x7f11d019a510 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f11c0032fd0 tx=0x7f11c0004270 comp rx=0 tx=0).stop 2024-07-26T13:40:50.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.041+0000 7f11d7853640 1 -- 172.21.15.100:0/1994037649 shutdown_connections 2024-07-26T13:40:50.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.041+0000 7f11d7853640 1 --2- 172.21.15.100:0/1994037649 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f11bc0b3cb0 0x7f11bc0b61c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:50.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.041+0000 7f11d7853640 1 --2- 172.21.15.100:0/1994037649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11d0108720 0x7f11d019f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:50.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.041+0000 7f11d7853640 1 --2- 172.21.15.100:0/1994037649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11d0107dd0 0x7f11d019a510 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:50.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.041+0000 7f11d7853640 1 --2- 172.21.15.100:0/1994037649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11d0102490 0x7f11d0199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:50.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.041+0000 7f11d7853640 1 -- 172.21.15.100:0/1994037649 >> 172.21.15.100:0/1994037649 conn(0x7f11d0079160 msgr2=0x7f11d006ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:50.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.041+0000 7f11d7853640 1 -- 172.21.15.100:0/1994037649 shutdown_connections 2024-07-26T13:40:50.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:50.041+0000 7f11d7853640 1 -- 172.21.15.100:0/1994037649 wait complete. 2024-07-26T13:40:50.759 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.b status 2024-07-26T13:40:51.014 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:40:51.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:51 smithi100 ceph-mon[32599]: pgmap v386: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-07-26T13:40:51.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:51 smithi167 ceph-mon[37140]: pgmap v386: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-07-26T13:40:51.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:51 smithi138 ceph-mon[36954]: pgmap v386: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-07-26T13:40:52.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.659+0000 7f84f0087640 1 -- 172.21.15.100:0/1677945501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84e8107dd0 msgr2=0x7f84e81081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:52.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.659+0000 7f84f0087640 1 --2- 172.21.15.100:0/1677945501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84e8107dd0 0x7f84e81081e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f84e4009900 tx=0x7f84e402a410 comp rx=0 tx=0).stop 2024-07-26T13:40:52.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.659+0000 7f84f0087640 1 -- 172.21.15.100:0/1677945501 shutdown_connections 2024-07-26T13:40:52.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.659+0000 7f84f0087640 1 --2- 172.21.15.100:0/1677945501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84e8108720 0x7f84e810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:52.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.659+0000 7f84f0087640 1 --2- 172.21.15.100:0/1677945501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84e8107dd0 0x7f84e81081e0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:52.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.659+0000 7f84f0087640 1 --2- 172.21.15.100:0/1677945501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84e8102490 0x7f84e81028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:52.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.659+0000 7f84f0087640 1 -- 172.21.15.100:0/1677945501 >> 172.21.15.100:0/1677945501 conn(0x7f84e8079150 msgr2=0x7f84e8079560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:52.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.659+0000 7f84f0087640 1 -- 172.21.15.100:0/1677945501 shutdown_connections 2024-07-26T13:40:52.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.659+0000 7f84f0087640 1 -- 172.21.15.100:0/1677945501 wait complete. 2024-07-26T13:40:52.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.660+0000 7f84f0087640 1 Processor -- start 2024-07-26T13:40:52.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.660+0000 7f84f0087640 1 -- start start 2024-07-26T13:40:52.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.660+0000 7f84f0087640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84e8102490 0x7f84e8198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:52.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.660+0000 7f84f0087640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84e8107dd0 0x7f84e8199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:52.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.660+0000 7f84f0087640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84e8108720 0x7f84e819e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:52.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.660+0000 7f84f0087640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f84e8112370 con 0x7f84e8107dd0 2024-07-26T13:40:52.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.660+0000 7f84f0087640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f84e81121f0 con 0x7f84e8108720 2024-07-26T13:40:52.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.660+0000 7f84f0087640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f84e81124f0 con 0x7f84e8102490 2024-07-26T13:40:52.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.660+0000 7f84ed622640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84e8107dd0 0x7f84e8199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:52.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.660+0000 7f84ed622640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84e8107dd0 0x7f84e8199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42372/0 (socket says 172.21.15.100:42372) 2024-07-26T13:40:52.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.660+0000 7f84ed622640 1 -- 172.21.15.100:0/709393182 learned_addr learned my addr 172.21.15.100:0/709393182 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:52.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.661+0000 7f84ede23640 1 --2- 172.21.15.100:0/709393182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84e8102490 0x7f84e8198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:52.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.661+0000 7f84ee624640 1 --2- 172.21.15.100:0/709393182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84e8108720 0x7f84e819e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:52.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.661+0000 7f84ed622640 1 -- 172.21.15.100:0/709393182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84e8102490 msgr2=0x7f84e8198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:52.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.661+0000 7f84ed622640 1 --2- 172.21.15.100:0/709393182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84e8102490 0x7f84e8198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:52.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.661+0000 7f84ed622640 1 -- 172.21.15.100:0/709393182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84e8108720 msgr2=0x7f84e819e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:52.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.661+0000 7f84ed622640 1 --2- 172.21.15.100:0/709393182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84e8108720 0x7f84e819e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:52.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.661+0000 7f84ed622640 1 -- 172.21.15.100:0/709393182 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e819ec50 con 0x7f84e8107dd0 2024-07-26T13:40:52.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.661+0000 7f84ed622640 1 --2- 172.21.15.100:0/709393182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84e8107dd0 0x7f84e8199340 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f84e4031a90 tx=0x7f84e4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:52.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.661+0000 7f84daffd640 1 -- 172.21.15.100:0/709393182 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e403f030 con 0x7f84e8107dd0 2024-07-26T13:40:52.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.661+0000 7f84daffd640 1 -- 172.21.15.100:0/709393182 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84e4003cb0 con 0x7f84e8107dd0 2024-07-26T13:40:52.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.661+0000 7f84f0087640 1 -- 172.21.15.100:0/709393182 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84e819eee0 con 0x7f84e8107dd0 2024-07-26T13:40:52.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.661+0000 7f84daffd640 1 -- 172.21.15.100:0/709393182 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e4035070 con 0x7f84e8107dd0 2024-07-26T13:40:52.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.661+0000 7f84f0087640 1 -- 172.21.15.100:0/709393182 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84e819f3c0 con 0x7f84e8107dd0 2024-07-26T13:40:52.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.664+0000 7f84f660b740 1 Processor -- start 2024-07-26T13:40:52.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.664+0000 7f84f660b740 1 -- start start 2024-07-26T13:40:52.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.665+0000 7f84f660b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55997cb63f60 0x55997cf093a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:52.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.665+0000 7f84daffd640 1 -- 172.21.15.100:0/709393182 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 151134+0+0 (secure 0 0 0) 0x7f84e4042030 con 0x7f84e8107dd0 2024-07-26T13:40:52.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.665+0000 7f84f660b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55997cb3b0a0 0x55997cf098e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:52.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.665+0000 7f84f660b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55997cb43280 0x55997cf09e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:52.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.665+0000 7f84f660b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55997ce7f940 con 0x55997cb43280 2024-07-26T13:40:52.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.665+0000 7f84f660b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55997ce6ee40 con 0x55997cb63f60 2024-07-26T13:40:52.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.665+0000 7f84f660b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55997ce72e80 con 0x55997cb3b0a0 2024-07-26T13:40:52.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.665+0000 7f84ed622640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55997cb43280 0x55997cf09e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:52.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.665+0000 7f84ed622640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55997cb43280 0x55997cf09e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42384/0 (socket says 172.21.15.100:42384) 2024-07-26T13:40:52.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.665+0000 7f84ed622640 1 -- 172.21.15.100:0/2519996624 learned_addr learned my addr 172.21.15.100:0/2519996624 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:52.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.665+0000 7f84ee624640 1 --2- 172.21.15.100:0/2519996624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55997cb63f60 0x55997cf093a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:52.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.665+0000 7f84ede23640 1 --2- 172.21.15.100:0/2519996624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55997cb3b0a0 0x55997cf098e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:52.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.665+0000 7f84ed622640 1 -- 172.21.15.100:0/2519996624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55997cb3b0a0 msgr2=0x55997cf098e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:52.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.665+0000 7f84ed622640 1 --2- 172.21.15.100:0/2519996624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55997cb3b0a0 0x55997cf098e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:52.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.665+0000 7f84ed622640 1 -- 172.21.15.100:0/2519996624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55997cb63f60 msgr2=0x55997cf093a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:52.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.665+0000 7f84ed622640 1 --2- 172.21.15.100:0/2519996624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55997cb63f60 0x55997cf093a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:52.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.665+0000 7f84ed622640 1 -- 172.21.15.100:0/2519996624 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55997ce15a60 con 0x55997cb43280 2024-07-26T13:40:52.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.665+0000 7f84ede23640 1 --2- 172.21.15.100:0/2519996624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55997cb3b0a0 0x55997cf098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:40:52.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.665+0000 7f84ed622640 1 --2- 172.21.15.100:0/2519996624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55997cb43280 0x55997cf09e20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f84e4101dc0 tx=0x7f84e410acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d20d2eb977e407 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:52.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.666+0000 7f84daffd640 1 --2- 172.21.15.100:0/709393182 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f84d40b3ce0 0x7f84d40b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:52.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.666+0000 7f84d3fff640 1 -- 172.21.15.100:0/2519996624 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e40c7910 con 0x55997cb43280 2024-07-26T13:40:52.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.666+0000 7f84ee624640 1 --2- 172.21.15.100:0/709393182 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f84d40b3ce0 0x7f84d40b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:52.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.666+0000 7f84d3fff640 1 -- 172.21.15.100:0/2519996624 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84e4114020 con 0x55997cb43280 2024-07-26T13:40:52.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.666+0000 7f84d3fff640 1 -- 172.21.15.100:0/2519996624 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e40cb090 con 0x55997cb43280 2024-07-26T13:40:52.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.666+0000 7f84daffd640 1 -- 172.21.15.100:0/709393182 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f84e41045d0 con 0x7f84e8107dd0 2024-07-26T13:40:52.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.667+0000 7f84f660b740 1 -- 172.21.15.100:0/2519996624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55997cb43280 msgr2=0x55997cf09e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:52.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.667+0000 7f84ee624640 1 --2- 172.21.15.100:0/709393182 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f84d40b3ce0 0x7f84d40b61f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f84e0008e40 tx=0x7f84e00085f0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:52.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.667+0000 7f84f660b740 1 --2- 172.21.15.100:0/2519996624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55997cb43280 0x55997cf09e20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f84e4101dc0 tx=0x7f84e410acb0 comp rx=0 tx=0).stop 2024-07-26T13:40:52.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.667+0000 7f84f660b740 1 -- 172.21.15.100:0/2519996624 shutdown_connections 2024-07-26T13:40:52.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.667+0000 7f84f660b740 1 --2- 172.21.15.100:0/2519996624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55997cb43280 0x55997cf09e20 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:52.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.667+0000 7f84f660b740 1 --2- 172.21.15.100:0/2519996624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55997cb3b0a0 0x55997cf098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:52.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.667+0000 7f84f660b740 1 --2- 172.21.15.100:0/2519996624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55997cb63f60 0x55997cf093a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:52.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.667+0000 7f84f660b740 1 -- 172.21.15.100:0/2519996624 >> 172.21.15.100:0/2519996624 conn(0x55997cb58350 msgr2=0x55997cf07290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:52.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.667+0000 7f84f660b740 1 -- 172.21.15.100:0/2519996624 shutdown_connections 2024-07-26T13:40:52.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.667+0000 7f84f660b740 1 -- 172.21.15.100:0/2519996624 wait complete. 2024-07-26T13:40:52.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.667+0000 7f84f660b740 1 -- start start 2024-07-26T13:40:52.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.667+0000 7f84f660b740 1 Processor -- start 2024-07-26T13:40:52.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.668+0000 7f84f660b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55997cb3b0a0 0x55997cf170c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:52.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.668+0000 7f84f660b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55997cb43280 0x55997cf17600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:52.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.668+0000 7f84f660b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55997cb63f60 0x55997cf22d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:52.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.668+0000 7f84f660b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55997ccd25f0 con 0x55997cb43280 2024-07-26T13:40:52.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.668+0000 7f84f660b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55997ce118b0 con 0x55997cb63f60 2024-07-26T13:40:52.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.668+0000 7f84f660b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55997cf19aa0 con 0x55997cb3b0a0 2024-07-26T13:40:52.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.668+0000 7f84f660b740 10 client.0 _ll_register_callbacks cb 0x55997ce1b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:40:52.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.668+0000 7f84ed622640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55997cb43280 0x55997cf17600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:52.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.668+0000 7f84ed622640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55997cb43280 0x55997cf17600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42386/0 (socket says 172.21.15.100:42386) 2024-07-26T13:40:52.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.668+0000 7f84ed622640 1 -- 172.21.15.100:0/4263817440 learned_addr learned my addr 172.21.15.100:0/4263817440 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:52.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.668+0000 7f84ee624640 1 --2- 172.21.15.100:0/4263817440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55997cb63f60 0x55997cf22d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:52.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.668+0000 7f84ede23640 1 --2- 172.21.15.100:0/4263817440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55997cb3b0a0 0x55997cf170c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:52.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.668+0000 7f84ed622640 1 -- 172.21.15.100:0/4263817440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55997cb3b0a0 msgr2=0x55997cf170c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:52.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.668+0000 7f84ed622640 1 --2- 172.21.15.100:0/4263817440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55997cb3b0a0 0x55997cf170c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:52.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.668+0000 7f84ed622640 1 -- 172.21.15.100:0/4263817440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55997cb63f60 msgr2=0x55997cf22d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:52.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.668+0000 7f84ed622640 1 --2- 172.21.15.100:0/4263817440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55997cb63f60 0x55997cf22d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:52.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.668+0000 7f84ed622640 1 -- 172.21.15.100:0/4263817440 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55997cbcf540 con 0x55997cb43280 2024-07-26T13:40:52.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.668+0000 7f84ede23640 1 --2- 172.21.15.100:0/4263817440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55997cb3b0a0 0x55997cf170c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:40:52.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.669+0000 7f84ed622640 1 --2- 172.21.15.100:0/4263817440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55997cb43280 0x55997cf17600 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f84e40cafc0 tx=0x7f84e402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64368512de0a435 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:52.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.669+0000 7f84d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:40:52.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.669+0000 7f84d17fa640 1 -- 172.21.15.100:0/4263817440 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e407a790 con 0x55997cb43280 2024-07-26T13:40:52.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.669+0000 7f84d17fa640 1 -- 172.21.15.100:0/4263817440 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f84e40c78b0 con 0x55997cb43280 2024-07-26T13:40:52.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.669+0000 7f84f660b740 1 -- 172.21.15.100:0/4263817440 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84e4114020 con 0x55997cb43280 2024-07-26T13:40:52.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.669+0000 7f84d17fa640 1 -- 172.21.15.100:0/4263817440 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f84e4049030 con 0x55997cb43280 2024-07-26T13:40:52.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.669+0000 7f84f660b740 10 client.15570 fetch_fsmap learned FSMap version 26 2024-07-26T13:40:52.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.669+0000 7f84f660b740 1 -- 172.21.15.100:0/4263817440 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55997cf259a0 con 0x55997cb43280 2024-07-26T13:40:52.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.669+0000 7f84d17fa640 1 -- 172.21.15.100:0/4263817440 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84e4114020 con 0x55997cb43280 2024-07-26T13:40:52.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.669+0000 7f84d17fa640 1 -- 172.21.15.100:0/4263817440 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f84e40c7550 con 0x55997cb43280 2024-07-26T13:40:52.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.669+0000 7f84f660b740 10 client.15570 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:40:52.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.669+0000 7f84f660b740 10 client.15570 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-26T13:40:52.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.669+0000 7f84f660b740 20 client.15570 populate_metadata read hostname 'smithi100' 2024-07-26T13:40:52.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.669+0000 7f84f660b740 1 --2- 172.21.15.100:0/4263817440 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55997cb228e0 0x55997cf29160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:52.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.669+0000 7f84f660b740 4 client.15570 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:40:52.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.669+0000 7f84f660b740 1 -- 172.21.15.100:0/4263817440 --> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55997ce81890 con 0x55997cb228e0 2024-07-26T13:40:52.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.670+0000 7f84ede23640 1 --2- 172.21.15.100:0/4263817440 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55997cb228e0 0x55997cf29160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:52.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.670+0000 7f84d17fa640 10 client.15570 ms_handle_connect on v2:172.21.15.138:6838/1580925471 2024-07-26T13:40:52.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.670+0000 7f84ede23640 1 --2- 172.21.15.100:0/4263817440 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55997cb228e0 0x55997cf29160 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4001ce8ced38e25d server_cookie=e8624224f443ce33 in_seq=0 out_seq=0 2024-07-26T13:40:52.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.671+0000 7f84d17fa640 1 -- 172.21.15.100:0/4263817440 <== mds.1 v2:172.21.15.138:6838/1580925471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55997ce81890 con 0x55997cb228e0 2024-07-26T13:40:52.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.671+0000 7f84d17fa640 10 client.15570 handle_command_reply: tid=0 2024-07-26T13:40:52.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.671+0000 7f84f660b740 1 client.15570 shutdown 2024-07-26T13:40:52.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.671+0000 7f84f660b740 1 -- 172.21.15.100:0/4263817440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55997cb43280 msgr2=0x55997cf17600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:52.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.671+0000 7f84f660b740 1 --2- 172.21.15.100:0/4263817440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55997cb43280 0x55997cf17600 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f84e40cafc0 tx=0x7f84e402a410 comp rx=0 tx=0).stop 2024-07-26T13:40:52.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.672+0000 7f84f660b740 1 -- 172.21.15.100:0/4263817440 shutdown_connections 2024-07-26T13:40:52.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.672+0000 7f84f660b740 1 --2- 172.21.15.100:0/4263817440 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55997cb228e0 0x55997cf29160 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:52.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.672+0000 7f84f660b740 1 --2- 172.21.15.100:0/4263817440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55997cb63f60 0x55997cf22d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:52.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.672+0000 7f84f660b740 1 --2- 172.21.15.100:0/4263817440 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55997cb43280 0x55997cf17600 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:52.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.672+0000 7f84f660b740 1 --2- 172.21.15.100:0/4263817440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55997cb3b0a0 0x55997cf170c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:52.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.672+0000 7f84d17fa640 0 client.15570 ms_handle_reset on v2:172.21.15.138:6838/1580925471 2024-07-26T13:40:52.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.672+0000 7f84f660b740 1 -- 172.21.15.100:0/4263817440 >> 172.21.15.100:0/4263817440 conn(0x55997cb58350 msgr2=0x55997cf0ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:52.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.672+0000 7f84f660b740 1 -- 172.21.15.100:0/4263817440 shutdown_connections 2024-07-26T13:40:52.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.672+0000 7f84f660b740 1 -- 172.21.15.100:0/4263817440 wait complete. 2024-07-26T13:40:52.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.672+0000 7f84f660b740 20 client.15570 trim_cache size 0 max 16384 2024-07-26T13:40:52.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.689+0000 7f84f660b740 1 Processor -- start 2024-07-26T13:40:52.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.689+0000 7f84f660b740 1 -- start start 2024-07-26T13:40:52.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.690+0000 7f84f660b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55997cb3b0a0 0x55997cf27cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:52.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.690+0000 7f84f660b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55997cb43280 0x55997cf22d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:52.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.690+0000 7f84f660b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55997cb63f60 0x55997cf23240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:52.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.690+0000 7f84f660b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55997cf274a0 con 0x55997cb43280 2024-07-26T13:40:52.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.690+0000 7f84f660b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55997cf27b70 con 0x55997cb3b0a0 2024-07-26T13:40:52.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.690+0000 7f84f660b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55997cf28240 con 0x55997cb63f60 2024-07-26T13:40:52.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.690+0000 7f84ed622640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55997cb43280 0x55997cf22d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:52.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.690+0000 7f84ed622640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55997cb43280 0x55997cf22d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42402/0 (socket says 172.21.15.100:42402) 2024-07-26T13:40:52.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.690+0000 7f84ed622640 1 -- 172.21.15.100:0/998017031 learned_addr learned my addr 172.21.15.100:0/998017031 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:52.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.690+0000 7f84ee624640 1 --2- 172.21.15.100:0/998017031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55997cb63f60 0x55997cf23240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:52.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.690+0000 7f84ede23640 1 --2- 172.21.15.100:0/998017031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55997cb3b0a0 0x55997cf27cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:52.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.690+0000 7f84ed622640 1 -- 172.21.15.100:0/998017031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55997cb63f60 msgr2=0x55997cf23240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:52.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.690+0000 7f84ed622640 1 --2- 172.21.15.100:0/998017031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55997cb63f60 0x55997cf23240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:52.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.690+0000 7f84ed622640 1 -- 172.21.15.100:0/998017031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55997cb3b0a0 msgr2=0x55997cf27cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:52.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.690+0000 7f84ed622640 1 --2- 172.21.15.100:0/998017031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55997cb3b0a0 0x55997cf27cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:52.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.690+0000 7f84ed622640 1 -- 172.21.15.100:0/998017031 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55997cf153c0 con 0x55997cb43280 2024-07-26T13:40:52.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.690+0000 7f84ede23640 1 --2- 172.21.15.100:0/998017031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55997cb3b0a0 0x55997cf27cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:40:52.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.690+0000 7f84ed622640 1 --2- 172.21.15.100:0/998017031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55997cb43280 0x55997cf22d00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f84e40cb250 tx=0x7f84e403b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f6bd34c024cc2ea server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:52.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.691+0000 7f84d37fe640 1 -- 172.21.15.100:0/998017031 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e4077b00 con 0x55997cb43280 2024-07-26T13:40:52.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.691+0000 7f84d37fe640 1 -- 172.21.15.100:0/998017031 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84e410a730 con 0x55997cb43280 2024-07-26T13:40:52.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.691+0000 7f84d37fe640 1 -- 172.21.15.100:0/998017031 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e4078cc0 con 0x55997cb43280 2024-07-26T13:40:52.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.691+0000 7f84f660b740 1 -- 172.21.15.100:0/998017031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55997cb43280 msgr2=0x55997cf22d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:52.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.691+0000 7f84f660b740 1 --2- 172.21.15.100:0/998017031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55997cb43280 0x55997cf22d00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f84e40cb250 tx=0x7f84e403b720 comp rx=0 tx=0).stop 2024-07-26T13:40:52.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.692+0000 7f84f660b740 1 -- 172.21.15.100:0/998017031 shutdown_connections 2024-07-26T13:40:52.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.692+0000 7f84f660b740 1 --2- 172.21.15.100:0/998017031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55997cb63f60 0x55997cf23240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:52.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.692+0000 7f84f660b740 1 --2- 172.21.15.100:0/998017031 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55997cb43280 0x55997cf22d00 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:52.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.692+0000 7f84f660b740 1 --2- 172.21.15.100:0/998017031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55997cb3b0a0 0x55997cf27cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:52.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.692+0000 7f84f660b740 1 -- 172.21.15.100:0/998017031 >> 172.21.15.100:0/998017031 conn(0x55997cb58350 msgr2=0x55997cf0b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:52.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.692+0000 7f84f660b740 1 -- 172.21.15.100:0/998017031 shutdown_connections 2024-07-26T13:40:52.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.692+0000 7f84f660b740 1 -- 172.21.15.100:0/998017031 wait complete. 2024-07-26T13:40:52.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.692+0000 7f84f660b740 1 -- start start 2024-07-26T13:40:52.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.692+0000 7f84f660b740 1 Processor -- start 2024-07-26T13:40:52.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.693+0000 7f84f660b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55997cb3b0a0 0x55997cf31c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:52.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.693+0000 7f84f660b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55997cb43280 0x55997cf0a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:52.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.693+0000 7f84f660b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55997cb63f60 0x55997cf0aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:52.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.693+0000 7f84f660b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55997cf28fe0 con 0x55997cb43280 2024-07-26T13:40:52.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.693+0000 7f84f660b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55997cf28910 con 0x55997cb63f60 2024-07-26T13:40:52.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.693+0000 7f84f660b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55997cf296b0 con 0x55997cb3b0a0 2024-07-26T13:40:52.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.693+0000 7f84f660b740 10 client.0 _ll_register_callbacks cb 0x7f84e410af90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:40:52.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.693+0000 7f84ed622640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55997cb43280 0x55997cf0a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:52.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.693+0000 7f84ed622640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55997cb43280 0x55997cf0a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42406/0 (socket says 172.21.15.100:42406) 2024-07-26T13:40:52.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.693+0000 7f84ed622640 1 -- 172.21.15.100:0/3621539960 learned_addr learned my addr 172.21.15.100:0/3621539960 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:52.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.693+0000 7f84ede23640 1 --2- 172.21.15.100:0/3621539960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55997cb3b0a0 0x55997cf31c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:52.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.693+0000 7f84ee624640 1 --2- 172.21.15.100:0/3621539960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55997cb63f60 0x55997cf0aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:52.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.693+0000 7f84ed622640 1 -- 172.21.15.100:0/3621539960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55997cb3b0a0 msgr2=0x55997cf31c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:52.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.693+0000 7f84ed622640 1 --2- 172.21.15.100:0/3621539960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55997cb3b0a0 0x55997cf31c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:52.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.693+0000 7f84ed622640 1 -- 172.21.15.100:0/3621539960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55997cb63f60 msgr2=0x55997cf0aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:52.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.693+0000 7f84ed622640 1 --2- 172.21.15.100:0/3621539960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55997cb63f60 0x55997cf0aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:52.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.693+0000 7f84ed622640 1 -- 172.21.15.100:0/3621539960 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55997cf0b190 con 0x55997cb43280 2024-07-26T13:40:52.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.693+0000 7f84ee624640 1 --2- 172.21.15.100:0/3621539960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55997cb63f60 0x55997cf0aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:40:52.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.693+0000 7f84ed622640 1 --2- 172.21.15.100:0/3621539960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55997cb43280 0x55997cf0a380 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f84e4079060 tx=0x7f84e402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72225d3393a208b4 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:52.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.693+0000 7f84d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:40:52.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.694+0000 7f84d17fa640 1 -- 172.21.15.100:0/3621539960 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e4113070 con 0x55997cb43280 2024-07-26T13:40:52.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.694+0000 7f84d17fa640 1 -- 172.21.15.100:0/3621539960 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f84e40d3ee0 con 0x55997cb43280 2024-07-26T13:40:52.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.694+0000 7f84f660b740 1 -- 172.21.15.100:0/3621539960 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84e410a730 con 0x55997cb43280 2024-07-26T13:40:52.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.694+0000 7f84d17fa640 1 -- 172.21.15.100:0/3621539960 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f84e4078ea0 con 0x55997cb43280 2024-07-26T13:40:52.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.694+0000 7f84f660b740 10 client.15576 fetch_fsmap learned FSMap version 26 2024-07-26T13:40:52.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.694+0000 7f84f660b740 1 -- 172.21.15.100:0/3621539960 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55997cf0b190 con 0x55997cb43280 2024-07-26T13:40:52.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.694+0000 7f84d17fa640 1 -- 172.21.15.100:0/3621539960 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84e410a730 con 0x55997cb43280 2024-07-26T13:40:52.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.694+0000 7f84d17fa640 1 -- 172.21.15.100:0/3621539960 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f84e40cb520 con 0x55997cb43280 2024-07-26T13:40:52.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.694+0000 7f84f660b740 10 client.15576 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:40:52.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.694+0000 7f84f660b740 10 client.15576 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-26T13:40:52.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.694+0000 7f84f660b740 20 client.15576 populate_metadata read hostname 'smithi100' 2024-07-26T13:40:52.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.694+0000 7f84f660b740 1 --2- 172.21.15.100:0/3621539960 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55997caf7c90 0x55997cf92270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:52.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.694+0000 7f84f660b740 4 client.15576 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-26T13:40:52.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.694+0000 7f84f660b740 1 -- 172.21.15.100:0/3621539960 --> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] -- command(tid 0: {"prefix": "status"}) -- 0x55997ce7d330 con 0x55997caf7c90 2024-07-26T13:40:52.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.695+0000 7f84ede23640 1 --2- 172.21.15.100:0/3621539960 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55997caf7c90 0x55997cf92270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:52.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.695+0000 7f84d17fa640 10 client.15576 ms_handle_connect on v2:172.21.15.138:6838/1580925471 2024-07-26T13:40:52.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.695+0000 7f84ede23640 1 --2- 172.21.15.100:0/3621539960 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55997caf7c90 0x55997cf92270 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8a88b98c02b454 server_cookie=abb74beaeed7f042 in_seq=0 out_seq=0 2024-07-26T13:40:52.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.695+0000 7f84d17fa640 1 -- 172.21.15.100:0/3621539960 <== mds.1 v2:172.21.15.138:6838/1580925471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55997ce7d330 con 0x55997caf7c90 2024-07-26T13:40:52.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.695+0000 7f84d17fa640 10 client.15576 handle_command_reply: tid=0 2024-07-26T13:40:52.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.695+0000 7f84f660b740 1 client.15576 shutdown 2024-07-26T13:40:52.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.695+0000 7f84f660b740 1 -- 172.21.15.100:0/3621539960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55997cb43280 msgr2=0x55997cf0a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:52.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.695+0000 7f84f660b740 1 --2- 172.21.15.100:0/3621539960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55997cb43280 0x55997cf0a380 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f84e4079060 tx=0x7f84e402a410 comp rx=0 tx=0).stop 2024-07-26T13:40:52.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.696+0000 7f84f660b740 1 -- 172.21.15.100:0/3621539960 shutdown_connections 2024-07-26T13:40:52.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.696+0000 7f84f660b740 1 --2- 172.21.15.100:0/3621539960 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55997caf7c90 0x55997cf92270 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:52.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.696+0000 7f84f660b740 1 --2- 172.21.15.100:0/3621539960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55997cb63f60 0x55997cf0aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:52.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.696+0000 7f84f660b740 1 --2- 172.21.15.100:0/3621539960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55997cb43280 0x55997cf0a380 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:52.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.696+0000 7f84d17fa640 0 client.15576 ms_handle_reset on v2:172.21.15.138:6838/1580925471 2024-07-26T13:40:52.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.696+0000 7f84f660b740 1 --2- 172.21.15.100:0/3621539960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55997cb3b0a0 0x55997cf31c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:52.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.696+0000 7f84f660b740 1 -- 172.21.15.100:0/3621539960 >> 172.21.15.100:0/3621539960 conn(0x55997cb58350 msgr2=0x55997cf12000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:52.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.696+0000 7f84f660b740 1 -- 172.21.15.100:0/3621539960 shutdown_connections 2024-07-26T13:40:52.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.696+0000 7f84f660b740 1 -- 172.21.15.100:0/3621539960 wait complete. 2024-07-26T13:40:52.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.696+0000 7f84f660b740 20 client.15576 trim_cache size 0 max 16384 2024-07-26T13:40:52.704 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:40:52.704 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "bf83e2e6-4b52-11ef-bcb7-c7b262605968", 2024-07-26T13:40:52.704 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 1, 2024-07-26T13:40:52.704 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 24605, 2024-07-26T13:40:52.704 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-07-26T13:40:52.705 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-07-26T13:40:52.705 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-07-26T13:40:52.705 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 219.17270155700001, 2024-07-26T13:40:52.705 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 26, 2024-07-26T13:40:52.705 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 81, 2024-07-26T13:40:52.705 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 81, 2024-07-26T13:40:52.705 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 248.494991392 2024-07-26T13:40:52.705 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:40:52.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.696+0000 7f84f0087640 1 -- 172.21.15.100:0/709393182 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f84d40b3ce0 msgr2=0x7f84d40b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:52.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.696+0000 7f84f0087640 1 --2- 172.21.15.100:0/709393182 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f84d40b3ce0 0x7f84d40b61f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f84e0008e40 tx=0x7f84e00085f0 comp rx=0 tx=0).stop 2024-07-26T13:40:52.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.696+0000 7f84f0087640 1 -- 172.21.15.100:0/709393182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84e8107dd0 msgr2=0x7f84e8199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:52.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.696+0000 7f84f0087640 1 --2- 172.21.15.100:0/709393182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84e8107dd0 0x7f84e8199340 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f84e4031a90 tx=0x7f84e4004270 comp rx=0 tx=0).stop 2024-07-26T13:40:52.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.696+0000 7f84f0087640 1 -- 172.21.15.100:0/709393182 shutdown_connections 2024-07-26T13:40:52.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.696+0000 7f84f0087640 1 --2- 172.21.15.100:0/709393182 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f84d40b3ce0 0x7f84d40b61f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:52.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.696+0000 7f84f0087640 1 --2- 172.21.15.100:0/709393182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84e8108720 0x7f84e819e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:52.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.696+0000 7f84f0087640 1 --2- 172.21.15.100:0/709393182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84e8107dd0 0x7f84e8199340 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:52.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.696+0000 7f84f0087640 1 --2- 172.21.15.100:0/709393182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84e8102490 0x7f84e8198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:52.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.696+0000 7f84f0087640 1 -- 172.21.15.100:0/709393182 >> 172.21.15.100:0/709393182 conn(0x7f84e8079150 msgr2=0x7f84e8104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:52.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.697+0000 7f84f0087640 1 -- 172.21.15.100:0/709393182 shutdown_connections 2024-07-26T13:40:52.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:52.697+0000 7f84f0087640 1 -- 172.21.15.100:0/709393182 wait complete. 2024-07-26T13:40:53.411 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.i status 2024-07-26T13:40:53.667 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:40:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:53 smithi100 ceph-mon[32599]: pgmap v387: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 938 B/s wr, 6 op/s 2024-07-26T13:40:53.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:53 smithi138 ceph-mon[36954]: pgmap v387: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 938 B/s wr, 6 op/s 2024-07-26T13:40:54.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:53 smithi167 ceph-mon[37140]: pgmap v387: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 938 B/s wr, 6 op/s 2024-07-26T13:40:55.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.477+0000 7fc4c4da2640 1 -- 172.21.15.100:0/3997463674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4c0108720 msgr2=0x7fc4c010ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:55.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.477+0000 7fc4c4da2640 1 --2- 172.21.15.100:0/3997463674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4c0108720 0x7fc4c010ef50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8009ca0 tx=0x7fc4b802a8c0 comp rx=0 tx=0).stop 2024-07-26T13:40:55.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.477+0000 7fc4c4da2640 1 -- 172.21.15.100:0/3997463674 shutdown_connections 2024-07-26T13:40:55.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.477+0000 7fc4c4da2640 1 --2- 172.21.15.100:0/3997463674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4c0108720 0x7fc4c010ef50 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:55.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.477+0000 7fc4c4da2640 1 --2- 172.21.15.100:0/3997463674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4c0107dd0 0x7fc4c01081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:55.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.477+0000 7fc4c4da2640 1 --2- 172.21.15.100:0/3997463674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4c0102490 0x7fc4c01028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:55.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.477+0000 7fc4c4da2640 1 -- 172.21.15.100:0/3997463674 >> 172.21.15.100:0/3997463674 conn(0x7fc4c0079150 msgr2=0x7fc4c0079560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:55.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.477+0000 7fc4c4da2640 1 -- 172.21.15.100:0/3997463674 shutdown_connections 2024-07-26T13:40:55.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.477+0000 7fc4c4da2640 1 -- 172.21.15.100:0/3997463674 wait complete. 2024-07-26T13:40:55.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.478+0000 7fc4c4da2640 1 Processor -- start 2024-07-26T13:40:55.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.478+0000 7fc4c4da2640 1 -- start start 2024-07-26T13:40:55.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.478+0000 7fc4c4da2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4c0102490 0x7fc4c019acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:55.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.478+0000 7fc4c4da2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4c0107dd0 0x7fc4c019b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:55.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.478+0000 7fc4c4da2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4c0108720 0x7fc4c019b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:55.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.478+0000 7fc4c4da2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c01123a0 con 0x7fc4c0102490 2024-07-26T13:40:55.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.478+0000 7fc4c4da2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c0112220 con 0x7fc4c0107dd0 2024-07-26T13:40:55.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.478+0000 7fc4c4da2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c0112520 con 0x7fc4c0108720 2024-07-26T13:40:55.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.479+0000 7fc4be59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4c0102490 0x7fc4c019acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:55.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.479+0000 7fc4be59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4c0102490 0x7fc4c019acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46896/0 (socket says 172.21.15.100:46896) 2024-07-26T13:40:55.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.479+0000 7fc4be59c640 1 -- 172.21.15.100:0/2193936809 learned_addr learned my addr 172.21.15.100:0/2193936809 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:55.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.479+0000 7fc4bdd9b640 1 --2- 172.21.15.100:0/2193936809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4c0107dd0 0x7fc4c019b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:55.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.479+0000 7fc4bed9d640 1 --2- 172.21.15.100:0/2193936809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4c0108720 0x7fc4c019b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:55.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.479+0000 7fc4be59c640 1 -- 172.21.15.100:0/2193936809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4c0108720 msgr2=0x7fc4c019b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:55.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.479+0000 7fc4be59c640 1 --2- 172.21.15.100:0/2193936809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4c0108720 0x7fc4c019b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:55.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.479+0000 7fc4be59c640 1 -- 172.21.15.100:0/2193936809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4c0107dd0 msgr2=0x7fc4c019b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:55.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.479+0000 7fc4be59c640 1 --2- 172.21.15.100:0/2193936809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4c0107dd0 0x7fc4c019b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:55.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.479+0000 7fc4be59c640 1 -- 172.21.15.100:0/2193936809 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c01a5f60 con 0x7fc4c0102490 2024-07-26T13:40:55.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.479+0000 7fc4be59c640 1 --2- 172.21.15.100:0/2193936809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4c0102490 0x7fc4c019acd0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc4b400d770 tx=0x7fc4b400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:55.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.479+0000 7fc4af7fe640 1 -- 172.21.15.100:0/2193936809 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4017070 con 0x7fc4c0102490 2024-07-26T13:40:55.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.479+0000 7fc4af7fe640 1 -- 172.21.15.100:0/2193936809 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4b40054d0 con 0x7fc4c0102490 2024-07-26T13:40:55.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.479+0000 7fc4af7fe640 1 -- 172.21.15.100:0/2193936809 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4004c30 con 0x7fc4c0102490 2024-07-26T13:40:55.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.479+0000 7fc4c4da2640 1 -- 172.21.15.100:0/2193936809 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4c01a6250 con 0x7fc4c0102490 2024-07-26T13:40:55.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.479+0000 7fc4c4da2640 1 -- 172.21.15.100:0/2193936809 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4c01a6790 con 0x7fc4c0102490 2024-07-26T13:40:55.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.482+0000 7fc4cb326740 1 Processor -- start 2024-07-26T13:40:55.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.482+0000 7fc4cb326740 1 -- start start 2024-07-26T13:40:55.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.482+0000 7fc4af7fe640 1 -- 172.21.15.100:0/2193936809 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 151134+0+0 (secure 0 0 0) 0x7fc4b4004dd0 con 0x7fc4c0102490 2024-07-26T13:40:55.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.482+0000 7fc4cb326740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f28ca6f60 0x561f2904c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:55.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.482+0000 7fc4cb326740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f28c7e0a0 0x561f2904c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:55.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.482+0000 7fc4cb326740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f28c86280 0x561f2904ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:55.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.482+0000 7fc4cb326740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x561f28fc2920 con 0x561f28ca6f60 2024-07-26T13:40:55.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.482+0000 7fc4cb326740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561f28fb1e20 con 0x561f28c7e0a0 2024-07-26T13:40:55.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.482+0000 7fc4cb326740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561f28fb5e60 con 0x561f28c86280 2024-07-26T13:40:55.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.482+0000 7fc4bed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f28ca6f60 0x561f2904c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:55.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.483+0000 7fc4bed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f28ca6f60 0x561f2904c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46906/0 (socket says 172.21.15.100:46906) 2024-07-26T13:40:55.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.483+0000 7fc4bed9d640 1 -- 172.21.15.100:0/1786305931 learned_addr learned my addr 172.21.15.100:0/1786305931 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:55.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.483+0000 7fc4bdd9b640 1 --2- 172.21.15.100:0/1786305931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f28c86280 0x561f2904ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:55.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.483+0000 7fc4be59c640 1 --2- 172.21.15.100:0/1786305931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f28c7e0a0 0x561f2904c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:55.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.483+0000 7fc4bed9d640 1 -- 172.21.15.100:0/1786305931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f28c86280 msgr2=0x561f2904ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:55.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.483+0000 7fc4bed9d640 1 --2- 172.21.15.100:0/1786305931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f28c86280 0x561f2904ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:55.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.483+0000 7fc4bed9d640 1 -- 172.21.15.100:0/1786305931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f28c7e0a0 msgr2=0x561f2904c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:55.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.483+0000 7fc4bed9d640 1 --2- 172.21.15.100:0/1786305931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f28c7e0a0 0x561f2904c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:55.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.483+0000 7fc4bed9d640 1 -- 172.21.15.100:0/1786305931 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f28f58a40 con 0x561f28ca6f60 2024-07-26T13:40:55.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.483+0000 7fc4bed9d640 1 --2- 172.21.15.100:0/1786305931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f28ca6f60 0x561f2904c380 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8009920 tx=0x7fc4b8009950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0983ca0333457c server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:55.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.483+0000 7fc4af7fe640 1 --2- 172.21.15.100:0/2193936809 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc4a80b3d60 0x7fc4a80b6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:55.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.483+0000 7fc4acff9640 1 -- 172.21.15.100:0/1786305931 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8031e30 con 0x561f28ca6f60 2024-07-26T13:40:55.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.483+0000 7fc4be59c640 1 --2- 172.21.15.100:0/2193936809 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc4a80b3d60 0x7fc4a80b6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:55.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.483+0000 7fc4acff9640 1 -- 172.21.15.100:0/1786305931 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4b8036030 con 0x561f28ca6f60 2024-07-26T13:40:55.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.483+0000 7fc4acff9640 1 -- 172.21.15.100:0/1786305931 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b80042a0 con 0x561f28ca6f60 2024-07-26T13:40:55.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.483+0000 7fc4af7fe640 1 -- 172.21.15.100:0/2193936809 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fc4b40e38a0 con 0x7fc4c0102490 2024-07-26T13:40:55.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.484+0000 7fc4cb326740 1 -- 172.21.15.100:0/1786305931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f28ca6f60 msgr2=0x561f2904c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:55.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.484+0000 7fc4be59c640 1 --2- 172.21.15.100:0/2193936809 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc4a80b3d60 0x7fc4a80b6270 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x561f2904b000 tx=0x7fc4b4009f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:55.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.484+0000 7fc4cb326740 1 --2- 172.21.15.100:0/1786305931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f28ca6f60 0x561f2904c380 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8009920 tx=0x7fc4b8009950 comp rx=0 tx=0).stop 2024-07-26T13:40:55.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.484+0000 7fc4cb326740 1 -- 172.21.15.100:0/1786305931 shutdown_connections 2024-07-26T13:40:55.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.484+0000 7fc4cb326740 1 --2- 172.21.15.100:0/1786305931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f28c86280 0x561f2904ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:55.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.484+0000 7fc4cb326740 1 --2- 172.21.15.100:0/1786305931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f28c7e0a0 0x561f2904c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:55.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.484+0000 7fc4cb326740 1 --2- 172.21.15.100:0/1786305931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f28ca6f60 0x561f2904c380 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:55.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.484+0000 7fc4cb326740 1 -- 172.21.15.100:0/1786305931 >> 172.21.15.100:0/1786305931 conn(0x561f28c9b350 msgr2=0x561f2904a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:55.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.484+0000 7fc4cb326740 1 -- 172.21.15.100:0/1786305931 shutdown_connections 2024-07-26T13:40:55.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.484+0000 7fc4cb326740 1 -- 172.21.15.100:0/1786305931 wait complete. 2024-07-26T13:40:55.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.485+0000 7fc4cb326740 1 -- start start 2024-07-26T13:40:55.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.485+0000 7fc4cb326740 1 Processor -- start 2024-07-26T13:40:55.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.485+0000 7fc4cb326740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f28c7e0a0 0x561f29057f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:55.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.485+0000 7fc4cb326740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f28c86280 0x561f29058470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:55.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.485+0000 7fc4cb326740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f28ca6f60 0x561f29063ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:55.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.485+0000 7fc4cb326740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x561f28e155f0 con 0x561f28c7e0a0 2024-07-26T13:40:55.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.485+0000 7fc4cb326740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561f28f54890 con 0x561f28c86280 2024-07-26T13:40:55.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.485+0000 7fc4cb326740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561f2905a980 con 0x561f28ca6f60 2024-07-26T13:40:55.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.485+0000 7fc4cb326740 10 client.0 _ll_register_callbacks cb 0x561f28f5e7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:40:55.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.485+0000 7fc4bed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f28c7e0a0 0x561f29057f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:55.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.485+0000 7fc4bed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f28c7e0a0 0x561f29057f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46922/0 (socket says 172.21.15.100:46922) 2024-07-26T13:40:55.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.485+0000 7fc4bed9d640 1 -- 172.21.15.100:0/2234101133 learned_addr learned my addr 172.21.15.100:0/2234101133 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:55.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.485+0000 7fc4bdd9b640 1 --2- 172.21.15.100:0/2234101133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f28c86280 0x561f29058470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:55.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.486+0000 7fc4bed9d640 1 --2- 172.21.15.100:0/2234101133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f28ca6f60 0x561f29063ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:55.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.486+0000 7fc4bed9d640 1 -- 172.21.15.100:0/2234101133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f28ca6f60 msgr2=0x561f29063ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:55.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.486+0000 7fc4bed9d640 1 --2- 172.21.15.100:0/2234101133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f28ca6f60 0x561f29063ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:55.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.486+0000 7fc4bed9d640 1 -- 172.21.15.100:0/2234101133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f28c86280 msgr2=0x561f29058470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:55.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.486+0000 7fc4bed9d640 1 --2- 172.21.15.100:0/2234101133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f28c86280 0x561f29058470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:55.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.486+0000 7fc4bed9d640 1 -- 172.21.15.100:0/2234101133 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f28d12540 con 0x561f28c7e0a0 2024-07-26T13:40:55.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.486+0000 7fc4bed9d640 1 --2- 172.21.15.100:0/2234101133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f28c7e0a0 0x561f29057f30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fc4b802f420 tx=0x7fc4b8003e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e4994ea689dd62 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:55.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.486+0000 7fc4a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:40:55.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.486+0000 7fc4a5ffb640 1 -- 172.21.15.100:0/2234101133 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8058060 con 0x561f28c7e0a0 2024-07-26T13:40:55.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.486+0000 7fc4a5ffb640 1 -- 172.21.15.100:0/2234101133 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fc4b8004000 con 0x561f28c7e0a0 2024-07-26T13:40:55.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.486+0000 7fc4cb326740 1 -- 172.21.15.100:0/2234101133 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4b8036030 con 0x561f28c7e0a0 2024-07-26T13:40:55.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.486+0000 7fc4a5ffb640 1 -- 172.21.15.100:0/2234101133 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc4b8004200 con 0x561f28c7e0a0 2024-07-26T13:40:55.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.486+0000 7fc4cb326740 10 client.15588 fetch_fsmap learned FSMap version 26 2024-07-26T13:40:55.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.486+0000 7fc4cb326740 1 -- 172.21.15.100:0/2234101133 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561f29066970 con 0x561f28c7e0a0 2024-07-26T13:40:55.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.487+0000 7fc4a5ffb640 1 -- 172.21.15.100:0/2234101133 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4b8036030 con 0x561f28c7e0a0 2024-07-26T13:40:55.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.487+0000 7fc4a5ffb640 1 -- 172.21.15.100:0/2234101133 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fc4b8043070 con 0x561f28c7e0a0 2024-07-26T13:40:55.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.487+0000 7fc4cb326740 10 client.15588 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:40:55.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.487+0000 7fc4cb326740 10 client.15588 resolve_mds: resolved name 'i' to daemon mds.i 2024-07-26T13:40:55.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.487+0000 7fc4cb326740 20 client.15588 populate_metadata read hostname 'smithi100' 2024-07-26T13:40:55.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.487+0000 7fc4cb326740 1 --2- 172.21.15.100:0/2234101133 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x561f28c658e0 0x561f2906a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:55.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.487+0000 7fc4cb326740 4 client.15588 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:40:55.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.487+0000 7fc4cb326740 1 -- 172.21.15.100:0/2234101133 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f28fc4870 con 0x561f28c658e0 2024-07-26T13:40:55.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.487+0000 7fc4bdd9b640 1 --2- 172.21.15.100:0/2234101133 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x561f28c658e0 0x561f2906a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:55.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.488+0000 7fc4bdd9b640 1 --2- 172.21.15.100:0/2234101133 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x561f28c658e0 0x561f2906a1a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2ef6980dc349916 server_cookie=9721af3bec6ac1e9 in_seq=0 out_seq=0 2024-07-26T13:40:55.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.488+0000 7fc4a5ffb640 10 client.15588 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:40:55.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.489+0000 7fc4a5ffb640 1 -- 172.21.15.100:0/2234101133 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561f28fc4870 con 0x561f28c658e0 2024-07-26T13:40:55.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.489+0000 7fc4a5ffb640 10 client.15588 handle_command_reply: tid=0 2024-07-26T13:40:55.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.489+0000 7fc4cb326740 1 client.15588 shutdown 2024-07-26T13:40:55.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.489+0000 7fc4cb326740 1 -- 172.21.15.100:0/2234101133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f28c7e0a0 msgr2=0x561f29057f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:55.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.489+0000 7fc4cb326740 1 --2- 172.21.15.100:0/2234101133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f28c7e0a0 0x561f29057f30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fc4b802f420 tx=0x7fc4b8003e50 comp rx=0 tx=0).stop 2024-07-26T13:40:55.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.489+0000 7fc4cb326740 1 -- 172.21.15.100:0/2234101133 shutdown_connections 2024-07-26T13:40:55.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.489+0000 7fc4cb326740 1 --2- 172.21.15.100:0/2234101133 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x561f28c658e0 0x561f2906a1a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:55.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.489+0000 7fc4a5ffb640 0 client.15588 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:40:55.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.489+0000 7fc4cb326740 1 --2- 172.21.15.100:0/2234101133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f28ca6f60 0x561f29063ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:55.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.489+0000 7fc4cb326740 1 --2- 172.21.15.100:0/2234101133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f28c86280 0x561f29058470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:55.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.489+0000 7fc4cb326740 1 --2- 172.21.15.100:0/2234101133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f28c7e0a0 0x561f29057f30 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:55.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.489+0000 7fc4cb326740 1 -- 172.21.15.100:0/2234101133 >> 172.21.15.100:0/2234101133 conn(0x561f28c9b350 msgr2=0x561f2904f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:55.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.489+0000 7fc4cb326740 1 -- 172.21.15.100:0/2234101133 shutdown_connections 2024-07-26T13:40:55.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.489+0000 7fc4cb326740 1 -- 172.21.15.100:0/2234101133 wait complete. 2024-07-26T13:40:55.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.489+0000 7fc4cb326740 20 client.15588 trim_cache size 0 max 16384 2024-07-26T13:40:55.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.506+0000 7fc4cb326740 1 Processor -- start 2024-07-26T13:40:55.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.507+0000 7fc4cb326740 1 -- start start 2024-07-26T13:40:55.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.507+0000 7fc4cb326740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f28c658e0 0x561f29051ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:55.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.507+0000 7fc4cb326740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f28c7e0a0 0x561f29063c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:55.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.507+0000 7fc4cb326740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f28c86280 0x561f29064150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:55.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.507+0000 7fc4cb326740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x561f290684e0 con 0x561f28c86280 2024-07-26T13:40:55.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.507+0000 7fc4cb326740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561f29068bb0 con 0x561f28c658e0 2024-07-26T13:40:55.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.507+0000 7fc4cb326740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561f29069280 con 0x561f28c7e0a0 2024-07-26T13:40:55.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.507+0000 7fc4bed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f28c86280 0x561f29064150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:55.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.507+0000 7fc4bed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f28c86280 0x561f29064150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46924/0 (socket says 172.21.15.100:46924) 2024-07-26T13:40:55.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.507+0000 7fc4bed9d640 1 -- 172.21.15.100:0/1944639090 learned_addr learned my addr 172.21.15.100:0/1944639090 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:55.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.507+0000 7fc4bdd9b640 1 --2- 172.21.15.100:0/1944639090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f28c7e0a0 0x561f29063c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:55.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.507+0000 7fc4bed9d640 1 --2- 172.21.15.100:0/1944639090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f28c658e0 0x561f29051ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:55.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.507+0000 7fc4bed9d640 1 -- 172.21.15.100:0/1944639090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f28c7e0a0 msgr2=0x561f29063c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:55.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.507+0000 7fc4bed9d640 1 --2- 172.21.15.100:0/1944639090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f28c7e0a0 0x561f29063c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:55.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.507+0000 7fc4bed9d640 1 -- 172.21.15.100:0/1944639090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f28c658e0 msgr2=0x561f29051ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:55.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.507+0000 7fc4bed9d640 1 --2- 172.21.15.100:0/1944639090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f28c658e0 0x561f29051ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:55.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.507+0000 7fc4bed9d640 1 -- 172.21.15.100:0/1944639090 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f2904e940 con 0x561f28c86280 2024-07-26T13:40:55.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.507+0000 7fc4bdd9b640 1 --2- 172.21.15.100:0/1944639090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f28c7e0a0 0x561f29063c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:40:55.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.507+0000 7fc4bed9d640 1 --2- 172.21.15.100:0/1944639090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f28c86280 0x561f29064150 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8002230 tx=0x7fc4b802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5056da7dab2158f server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:55.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.508+0000 7fc4a7fff640 1 -- 172.21.15.100:0/1944639090 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8058020 con 0x561f28c86280 2024-07-26T13:40:55.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.508+0000 7fc4a7fff640 1 -- 172.21.15.100:0/1944639090 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4b8036040 con 0x561f28c86280 2024-07-26T13:40:55.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.508+0000 7fc4a7fff640 1 -- 172.21.15.100:0/1944639090 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8031760 con 0x561f28c86280 2024-07-26T13:40:55.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.508+0000 7fc4cb326740 1 -- 172.21.15.100:0/1944639090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f28c86280 msgr2=0x561f29064150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:55.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.508+0000 7fc4cb326740 1 --2- 172.21.15.100:0/1944639090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f28c86280 0x561f29064150 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8002230 tx=0x7fc4b802a8c0 comp rx=0 tx=0).stop 2024-07-26T13:40:55.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.508+0000 7fc4cb326740 1 -- 172.21.15.100:0/1944639090 shutdown_connections 2024-07-26T13:40:55.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.508+0000 7fc4cb326740 1 --2- 172.21.15.100:0/1944639090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f28c86280 0x561f29064150 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:55.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.508+0000 7fc4cb326740 1 --2- 172.21.15.100:0/1944639090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f28c7e0a0 0x561f29063c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:55.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.508+0000 7fc4cb326740 1 --2- 172.21.15.100:0/1944639090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f28c658e0 0x561f29051ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:55.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.508+0000 7fc4cb326740 1 -- 172.21.15.100:0/1944639090 >> 172.21.15.100:0/1944639090 conn(0x561f28c9b350 msgr2=0x561f29051490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:55.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.508+0000 7fc4cb326740 1 -- 172.21.15.100:0/1944639090 shutdown_connections 2024-07-26T13:40:55.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.509+0000 7fc4cb326740 1 -- 172.21.15.100:0/1944639090 wait complete. 2024-07-26T13:40:55.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.509+0000 7fc4cb326740 1 -- start start 2024-07-26T13:40:55.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.509+0000 7fc4cb326740 1 Processor -- start 2024-07-26T13:40:55.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.509+0000 7fc4cb326740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f28c658e0 0x561f29072a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:55.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.509+0000 7fc4cb326740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f28c7e0a0 0x561f2906b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:55.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.509+0000 7fc4cb326740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f28c86280 0x561f2906b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:55.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.509+0000 7fc4cb326740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x561f2906a020 con 0x561f28c658e0 2024-07-26T13:40:55.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.509+0000 7fc4cb326740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561f29069950 con 0x561f28c86280 2024-07-26T13:40:55.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.509+0000 7fc4cb326740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561f2906a6f0 con 0x561f28c7e0a0 2024-07-26T13:40:55.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.509+0000 7fc4cb326740 10 client.0 _ll_register_callbacks cb 0x7fc4b8003970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:40:55.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.509+0000 7fc4bed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f28c658e0 0x561f29072a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:55.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.509+0000 7fc4bed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f28c658e0 0x561f29072a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46928/0 (socket says 172.21.15.100:46928) 2024-07-26T13:40:55.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.509+0000 7fc4bed9d640 1 -- 172.21.15.100:0/1757557283 learned_addr learned my addr 172.21.15.100:0/1757557283 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:40:55.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.510+0000 7fc4bdd9b640 1 --2- 172.21.15.100:0/1757557283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f28c7e0a0 0x561f2906b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:55.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.510+0000 7fc4bed9d640 1 -- 172.21.15.100:0/1757557283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f28c7e0a0 msgr2=0x561f2906b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:55.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.510+0000 7fc4bed9d640 1 --2- 172.21.15.100:0/1757557283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f28c7e0a0 0x561f2906b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:55.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.510+0000 7fc4bed9d640 1 -- 172.21.15.100:0/1757557283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f28c86280 msgr2=0x561f2906b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:40:55.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.510+0000 7fc4bed9d640 1 --2- 172.21.15.100:0/1757557283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f28c86280 0x561f2906b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:55.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.510+0000 7fc4bed9d640 1 -- 172.21.15.100:0/1757557283 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f2906bfa0 con 0x561f28c658e0 2024-07-26T13:40:55.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.510+0000 7fc4bed9d640 1 --2- 172.21.15.100:0/1757557283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f28c658e0 0x561f29072a70 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc4b802fcf0 tx=0x7fc4b8005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f855997a164b9c21 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:40:55.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.510+0000 7fc4a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:40:55.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.510+0000 7fc4a5ffb640 1 -- 172.21.15.100:0/1757557283 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8058030 con 0x561f28c658e0 2024-07-26T13:40:55.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.510+0000 7fc4a5ffb640 1 -- 172.21.15.100:0/1757557283 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fc4b802f6e0 con 0x561f28c658e0 2024-07-26T13:40:55.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.510+0000 7fc4cb326740 1 -- 172.21.15.100:0/1757557283 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4b8036040 con 0x561f28c658e0 2024-07-26T13:40:55.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.510+0000 7fc4a5ffb640 1 -- 172.21.15.100:0/1757557283 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc4b8004050 con 0x561f28c658e0 2024-07-26T13:40:55.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.510+0000 7fc4cb326740 10 client.15594 fetch_fsmap learned FSMap version 26 2024-07-26T13:40:55.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.510+0000 7fc4cb326740 1 -- 172.21.15.100:0/1757557283 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561f290d22a0 con 0x561f28c658e0 2024-07-26T13:40:55.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.510+0000 7fc4a5ffb640 1 -- 172.21.15.100:0/1757557283 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4b8036040 con 0x561f28c658e0 2024-07-26T13:40:55.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.511+0000 7fc4a5ffb640 1 -- 172.21.15.100:0/1757557283 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fc4b806c070 con 0x561f28c658e0 2024-07-26T13:40:55.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.511+0000 7fc4cb326740 10 client.15594 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:40:55.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.511+0000 7fc4cb326740 10 client.15594 resolve_mds: resolved name 'i' to daemon mds.i 2024-07-26T13:40:55.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.511+0000 7fc4cb326740 20 client.15594 populate_metadata read hostname 'smithi100' 2024-07-26T13:40:55.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.511+0000 7fc4cb326740 1 --2- 172.21.15.100:0/1757557283 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x561f28ca6f60 0x561f290d5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:40:55.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.511+0000 7fc4cb326740 4 client.15594 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-26T13:40:55.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.511+0000 7fc4cb326740 1 -- 172.21.15.100:0/1757557283 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "status"}) -- 0x561f28fc0310 con 0x561f28ca6f60 2024-07-26T13:40:55.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.511+0000 7fc4bdd9b640 1 --2- 172.21.15.100:0/1757557283 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x561f28ca6f60 0x561f290d5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:40:55.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.512+0000 7fc4a5ffb640 10 client.15594 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:40:55.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.512+0000 7fc4bdd9b640 1 --2- 172.21.15.100:0/1757557283 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x561f28ca6f60 0x561f290d5570 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e98feccca9cada53 server_cookie=cd0d7b63ac0cf815 in_seq=0 out_seq=0 2024-07-26T13:40:55.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.512+0000 7fc4a5ffb640 1 -- 172.21.15.100:0/1757557283 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x561f28fc0310 con 0x561f28ca6f60 2024-07-26T13:40:55.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.512+0000 7fc4a5ffb640 10 client.15594 handle_command_reply: tid=0 2024-07-26T13:40:55.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.512+0000 7fc4cb326740 1 client.15594 shutdown 2024-07-26T13:40:55.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.512+0000 7fc4cb326740 1 -- 172.21.15.100:0/1757557283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f28c658e0 msgr2=0x561f29072a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:55.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.512+0000 7fc4cb326740 1 --2- 172.21.15.100:0/1757557283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f28c658e0 0x561f29072a70 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc4b802fcf0 tx=0x7fc4b8005bf0 comp rx=0 tx=0).stop 2024-07-26T13:40:55.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.512+0000 7fc4cb326740 1 -- 172.21.15.100:0/1757557283 shutdown_connections 2024-07-26T13:40:55.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.512+0000 7fc4cb326740 1 --2- 172.21.15.100:0/1757557283 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x561f28ca6f60 0x561f290d5570 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:55.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.512+0000 7fc4cb326740 1 --2- 172.21.15.100:0/1757557283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f28c86280 0x561f2906b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:55.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.512+0000 7fc4cb326740 1 --2- 172.21.15.100:0/1757557283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561f28c7e0a0 0x561f2906b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:55.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.512+0000 7fc4cb326740 1 --2- 172.21.15.100:0/1757557283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561f28c658e0 0x561f29072a70 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:55.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.512+0000 7fc4cb326740 1 -- 172.21.15.100:0/1757557283 >> 172.21.15.100:0/1757557283 conn(0x561f28c9b350 msgr2=0x561f290591e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:55.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.512+0000 7fc4a5ffb640 0 client.15594 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:40:55.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.512+0000 7fc4cb326740 1 -- 172.21.15.100:0/1757557283 shutdown_connections 2024-07-26T13:40:55.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.512+0000 7fc4cb326740 1 -- 172.21.15.100:0/1757557283 wait complete. 2024-07-26T13:40:55.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.512+0000 7fc4cb326740 20 client.15594 trim_cache size 0 max 16384 2024-07-26T13:40:55.520 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:40:55.520 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "bf83e2e6-4b52-11ef-bcb7-c7b262605968", 2024-07-26T13:40:55.520 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 0, 2024-07-26T13:40:55.520 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 24607, 2024-07-26T13:40:55.520 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-07-26T13:40:55.520 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-07-26T13:40:55.520 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-07-26T13:40:55.520 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 234.04399960699999, 2024-07-26T13:40:55.520 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 26, 2024-07-26T13:40:55.520 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 81, 2024-07-26T13:40:55.521 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 81, 2024-07-26T13:40:55.521 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 260.81387008600001 2024-07-26T13:40:55.521 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:40:55.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.513+0000 7fc4c4da2640 1 -- 172.21.15.100:0/2193936809 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc4a80b3d60 msgr2=0x7fc4a80b6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:55.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.513+0000 7fc4c4da2640 1 --2- 172.21.15.100:0/2193936809 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc4a80b3d60 0x7fc4a80b6270 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x561f2904b000 tx=0x7fc4b4009f90 comp rx=0 tx=0).stop 2024-07-26T13:40:55.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.513+0000 7fc4c4da2640 1 -- 172.21.15.100:0/2193936809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4c0102490 msgr2=0x7fc4c019acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:40:55.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.513+0000 7fc4c4da2640 1 --2- 172.21.15.100:0/2193936809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4c0102490 0x7fc4c019acd0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc4b400d770 tx=0x7fc4b400dc30 comp rx=0 tx=0).stop 2024-07-26T13:40:55.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.513+0000 7fc4c4da2640 1 -- 172.21.15.100:0/2193936809 shutdown_connections 2024-07-26T13:40:55.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.513+0000 7fc4c4da2640 1 --2- 172.21.15.100:0/2193936809 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc4a80b3d60 0x7fc4a80b6270 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:55.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.513+0000 7fc4c4da2640 1 --2- 172.21.15.100:0/2193936809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4c0108720 0x7fc4c019b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:55.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.513+0000 7fc4c4da2640 1 --2- 172.21.15.100:0/2193936809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4c0107dd0 0x7fc4c019b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:55.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.513+0000 7fc4c4da2640 1 --2- 172.21.15.100:0/2193936809 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4c0102490 0x7fc4c019acd0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:40:55.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.513+0000 7fc4c4da2640 1 -- 172.21.15.100:0/2193936809 >> 172.21.15.100:0/2193936809 conn(0x7fc4c0079150 msgr2=0x7fc4c0103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:40:55.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.513+0000 7fc4c4da2640 1 -- 172.21.15.100:0/2193936809 shutdown_connections 2024-07-26T13:40:55.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:40:55.513+0000 7fc4c4da2640 1 -- 172.21.15.100:0/2193936809 wait complete. 2024-07-26T13:40:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:55 smithi100 ceph-mon[32599]: pgmap v388: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-07-26T13:40:55.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:55 smithi138 ceph-mon[36954]: pgmap v388: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-07-26T13:40:56.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:55 smithi167 ceph-mon[37140]: pgmap v388: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-07-26T13:40:56.232 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-07-26T13:40:57.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:57 smithi138 ceph-mon[36954]: pgmap v389: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-07-26T13:40:57.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:57 smithi100 ceph-mon[32599]: pgmap v389: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-07-26T13:40:58.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:57 smithi167 ceph-mon[37140]: pgmap v389: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-07-26T13:40:59.025 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-26T13:40:59.309 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:40:59.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:59 smithi100 ceph-mon[32599]: pgmap v390: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-07-26T13:40:59.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:40:59.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:40:59.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:40:59.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:40:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:40:59.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:59 smithi138 ceph-mon[36954]: pgmap v390: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-07-26T13:40:59.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:40:59.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:40:59.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:40:59.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:40:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:41:00.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:59 smithi167 ceph-mon[37140]: pgmap v390: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-07-26T13:41:00.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:41:00.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:41:00.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:41:00.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:40:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:41:00.744 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:41:00 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: 2024-07-26T13:41:00.705+0000 7f9e3cccf640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-26T13:41:00.744 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:41:00 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: 2024-07-26T13:41:00.705+0000 7f9e3cccf640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-26T13:41:00.744 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:41:00 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: 2024-07-26T13:41:00.705+0000 7f9e3cccf640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-26T13:41:00.744 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:41:00 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: 2024-07-26T13:41:00.705+0000 7f9e3cccf640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-26T13:41:00.744 INFO:journalctl@ceph.mgr.x.smithi100.stdout:Jul 26 13:41:00 smithi100 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mgr-x[32829]: 2024-07-26T13:41:00.705+0000 7f9e3cccf640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-26T13:41:01.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.130+0000 7ff855650640 1 -- 172.21.15.100:0/1707439129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8501024a0 msgr2=0x7ff8501028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:01.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.130+0000 7ff855650640 1 --2- 172.21.15.100:0/1707439129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8501024a0 0x7ff8501028b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff840009900 tx=0x7ff84002a410 comp rx=0 tx=0).stop 2024-07-26T13:41:01.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.130+0000 7ff855650640 1 -- 172.21.15.100:0/1707439129 shutdown_connections 2024-07-26T13:41:01.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.130+0000 7ff855650640 1 --2- 172.21.15.100:0/1707439129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff850108730 0x7ff85010ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.130+0000 7ff855650640 1 --2- 172.21.15.100:0/1707439129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff850107de0 0x7ff8501081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.130+0000 7ff855650640 1 --2- 172.21.15.100:0/1707439129 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8501024a0 0x7ff8501028b0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.130+0000 7ff855650640 1 -- 172.21.15.100:0/1707439129 >> 172.21.15.100:0/1707439129 conn(0x7ff850079160 msgr2=0x7ff850079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.130+0000 7ff855650640 1 -- 172.21.15.100:0/1707439129 shutdown_connections 2024-07-26T13:41:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.130+0000 7ff855650640 1 -- 172.21.15.100:0/1707439129 wait complete. 2024-07-26T13:41:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.131+0000 7ff855650640 1 Processor -- start 2024-07-26T13:41:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.131+0000 7ff855650640 1 -- start start 2024-07-26T13:41:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.131+0000 7ff855650640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8501024a0 0x7ff850198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.131+0000 7ff855650640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff850107de0 0x7ff850199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.131+0000 7ff855650640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff850108730 0x7ff85019e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.131+0000 7ff855650640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff8501123e0 con 0x7ff8501024a0 2024-07-26T13:41:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.131+0000 7ff855650640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff850112260 con 0x7ff850107de0 2024-07-26T13:41:01.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.131+0000 7ff855650640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff850112560 con 0x7ff850108730 2024-07-26T13:41:01.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.132+0000 7ff84effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8501024a0 0x7ff850198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:01.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.132+0000 7ff84effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8501024a0 0x7ff850198df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46960/0 (socket says 172.21.15.100:46960) 2024-07-26T13:41:01.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.132+0000 7ff84effd640 1 -- 172.21.15.100:0/3570878697 learned_addr learned my addr 172.21.15.100:0/3570878697 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:01.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.132+0000 7ff84f7fe640 1 --2- 172.21.15.100:0/3570878697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff850108730 0x7ff85019e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:01.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.132+0000 7ff84e7fc640 1 --2- 172.21.15.100:0/3570878697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff850107de0 0x7ff850199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:01.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.132+0000 7ff84effd640 1 -- 172.21.15.100:0/3570878697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff850108730 msgr2=0x7ff85019e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:01.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.132+0000 7ff84effd640 1 --2- 172.21.15.100:0/3570878697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff850108730 0x7ff85019e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.132+0000 7ff84effd640 1 -- 172.21.15.100:0/3570878697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff850107de0 msgr2=0x7ff850199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:01.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.132+0000 7ff84effd640 1 --2- 172.21.15.100:0/3570878697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff850107de0 0x7ff850199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.132+0000 7ff84effd640 1 -- 172.21.15.100:0/3570878697 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff85019eca0 con 0x7ff8501024a0 2024-07-26T13:41:01.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.132+0000 7ff84e7fc640 1 --2- 172.21.15.100:0/3570878697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff850107de0 0x7ff850199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:41:01.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.132+0000 7ff84effd640 1 --2- 172.21.15.100:0/3570878697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8501024a0 0x7ff850198df0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff840005ab0 tx=0x7ff840003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:01.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.132+0000 7ff837fff640 1 -- 172.21.15.100:0/3570878697 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84003f070 con 0x7ff8501024a0 2024-07-26T13:41:01.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.132+0000 7ff855650640 1 -- 172.21.15.100:0/3570878697 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff85019ef30 con 0x7ff8501024a0 2024-07-26T13:41:01.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.132+0000 7ff837fff640 1 -- 172.21.15.100:0/3570878697 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff840031d60 con 0x7ff8501024a0 2024-07-26T13:41:01.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.132+0000 7ff855650640 1 -- 172.21.15.100:0/3570878697 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff85019f460 con 0x7ff8501024a0 2024-07-26T13:41:01.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.132+0000 7ff837fff640 1 -- 172.21.15.100:0/3570878697 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff840035030 con 0x7ff8501024a0 2024-07-26T13:41:01.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.135+0000 7ff85bbd4740 1 Processor -- start 2024-07-26T13:41:01.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.135+0000 7ff85bbd4740 1 -- start start 2024-07-26T13:41:01.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.135+0000 7ff837fff640 1 -- 172.21.15.100:0/3570878697 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 151134+0+0 (secure 0 0 0) 0x7ff840042070 con 0x7ff8501024a0 2024-07-26T13:41:01.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.135+0000 7ff85bbd4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559addc9c070 0x559ade041520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:01.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.135+0000 7ff85bbd4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559addc731b0 0x559ade041a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.135+0000 7ff85bbd4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559addc7b390 0x559ade041fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.135+0000 7ff85bbd4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x559addfb7b60 con 0x559addc731b0 2024-07-26T13:41:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.135+0000 7ff85bbd4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559addfa7060 con 0x559addc9c070 2024-07-26T13:41:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.135+0000 7ff85bbd4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559addfab0a0 con 0x559addc7b390 2024-07-26T13:41:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.136+0000 7ff84effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559addc731b0 0x559ade041a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.136+0000 7ff84effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559addc731b0 0x559ade041a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46966/0 (socket says 172.21.15.100:46966) 2024-07-26T13:41:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.136+0000 7ff84effd640 1 -- 172.21.15.100:0/3979278289 learned_addr learned my addr 172.21.15.100:0/3979278289 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.136+0000 7ff84e7fc640 1 --2- 172.21.15.100:0/3979278289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559addc7b390 0x559ade041fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.136+0000 7ff84f7fe640 1 --2- 172.21.15.100:0/3979278289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559addc9c070 0x559ade041520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.136+0000 7ff84f7fe640 1 -- 172.21.15.100:0/3979278289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559addc7b390 msgr2=0x559ade041fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.136+0000 7ff84f7fe640 1 --2- 172.21.15.100:0/3979278289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559addc7b390 0x559ade041fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.136+0000 7ff84f7fe640 1 -- 172.21.15.100:0/3979278289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559addc731b0 msgr2=0x559ade041a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:01.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.136+0000 7ff84f7fe640 1 --2- 172.21.15.100:0/3979278289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559addc731b0 0x559ade041a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.136+0000 7ff84f7fe640 1 -- 172.21.15.100:0/3979278289 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559addf4db40 con 0x559addc9c070 2024-07-26T13:41:01.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.136+0000 7ff84e7fc640 1 --2- 172.21.15.100:0/3979278289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559addc7b390 0x559ade041fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:41:01.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.136+0000 7ff837fff640 1 --2- 172.21.15.100:0/3570878697 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff8380b3ce0 0x7ff8380b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:01.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.136+0000 7ff84effd640 1 --2- 172.21.15.100:0/3979278289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559addc731b0 0x559ade041a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:41:01.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.136+0000 7ff84f7fe640 1 --2- 172.21.15.100:0/3979278289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559addc9c070 0x559ade041520 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff848007d90 tx=0x7ff84800e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=125383cc88961310 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:01.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.136+0000 7ff84effd640 1 --2- 172.21.15.100:0/3570878697 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff8380b3ce0 0x7ff8380b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:01.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.137+0000 7ff8357fa640 1 -- 172.21.15.100:0/3979278289 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff848014070 con 0x559addc9c070 2024-07-26T13:41:01.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.137+0000 7ff837fff640 1 -- 172.21.15.100:0/3570878697 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7ff8401044d0 con 0x7ff8501024a0 2024-07-26T13:41:01.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.137+0000 7ff8357fa640 1 -- 172.21.15.100:0/3979278289 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff848003e20 con 0x559addc9c070 2024-07-26T13:41:01.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.137+0000 7ff8357fa640 1 -- 172.21.15.100:0/3979278289 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff848005020 con 0x559addc9c070 2024-07-26T13:41:01.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.138+0000 7ff85bbd4740 1 -- 172.21.15.100:0/3979278289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559addc9c070 msgr2=0x559ade041520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:01.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.138+0000 7ff85bbd4740 1 --2- 172.21.15.100:0/3979278289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559addc9c070 0x559ade041520 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff848007d90 tx=0x7ff84800e720 comp rx=0 tx=0).stop 2024-07-26T13:41:01.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.138+0000 7ff84effd640 1 --2- 172.21.15.100:0/3570878697 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff8380b3ce0 0x7ff8380b61f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff8400cb150 tx=0x7ff84010a0e0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:01.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.138+0000 7ff85bbd4740 1 -- 172.21.15.100:0/3979278289 shutdown_connections 2024-07-26T13:41:01.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.138+0000 7ff85bbd4740 1 --2- 172.21.15.100:0/3979278289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559addc7b390 0x559ade041fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.138+0000 7ff85bbd4740 1 --2- 172.21.15.100:0/3979278289 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559addc731b0 0x559ade041a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.138+0000 7ff85bbd4740 1 --2- 172.21.15.100:0/3979278289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559addc9c070 0x559ade041520 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.138+0000 7ff85bbd4740 1 -- 172.21.15.100:0/3979278289 >> 172.21.15.100:0/3979278289 conn(0x559addc90460 msgr2=0x559ade03f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:01.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.138+0000 7ff85bbd4740 1 -- 172.21.15.100:0/3979278289 shutdown_connections 2024-07-26T13:41:01.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.138+0000 7ff85bbd4740 1 -- 172.21.15.100:0/3979278289 wait complete. 2024-07-26T13:41:01.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.138+0000 7ff85bbd4740 1 -- start start 2024-07-26T13:41:01.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.138+0000 7ff85bbd4740 1 Processor -- start 2024-07-26T13:41:01.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.139+0000 7ff85bbd4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559addc731b0 0x559ade051020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:01.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.139+0000 7ff85bbd4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559addc7b390 0x559ade04d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:01.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.139+0000 7ff85bbd4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559addc9c070 0x559ade04da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:01.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.139+0000 7ff85bbd4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x559adde0a700 con 0x559addc9c070 2024-07-26T13:41:01.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.139+0000 7ff85bbd4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559addf49990 con 0x559addc7b390 2024-07-26T13:41:01.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.139+0000 7ff85bbd4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559ade047b50 con 0x559addc731b0 2024-07-26T13:41:01.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.139+0000 7ff85bbd4740 10 client.0 _ll_register_callbacks cb 0x559addf644a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:41:01.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.139+0000 7ff84f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559addc9c070 0x559ade04da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:01.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.139+0000 7ff84f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559addc9c070 0x559ade04da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46980/0 (socket says 172.21.15.100:46980) 2024-07-26T13:41:01.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.139+0000 7ff84f7fe640 1 -- 172.21.15.100:0/3004654401 learned_addr learned my addr 172.21.15.100:0/3004654401 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:01.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.139+0000 7ff84e7fc640 1 --2- 172.21.15.100:0/3004654401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559addc7b390 0x559ade04d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:01.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.139+0000 7ff84f7fe640 1 --2- 172.21.15.100:0/3004654401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559addc731b0 0x559ade051020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:01.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.139+0000 7ff84f7fe640 1 -- 172.21.15.100:0/3004654401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559addc731b0 msgr2=0x559ade051020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:01.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.139+0000 7ff84f7fe640 1 --2- 172.21.15.100:0/3004654401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559addc731b0 0x559ade051020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.139+0000 7ff84f7fe640 1 -- 172.21.15.100:0/3004654401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559addc7b390 msgr2=0x559ade04d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:01.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.139+0000 7ff84f7fe640 1 --2- 172.21.15.100:0/3004654401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559addc7b390 0x559ade04d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.139+0000 7ff84f7fe640 1 -- 172.21.15.100:0/3004654401 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559addd07650 con 0x559addc9c070 2024-07-26T13:41:01.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.139+0000 7ff84e7fc640 1 --2- 172.21.15.100:0/3004654401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559addc7b390 0x559ade04d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:41:01.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.140+0000 7ff84f7fe640 1 --2- 172.21.15.100:0/3004654401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559addc9c070 0x559ade04da20 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff8480074f0 tx=0x7ff848004490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7d2e5b88ea97ad server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:01.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.140+0000 7ff832ff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:41:01.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.140+0000 7ff832ff5640 1 -- 172.21.15.100:0/3004654401 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff848014030 con 0x559addc9c070 2024-07-26T13:41:01.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.140+0000 7ff832ff5640 1 -- 172.21.15.100:0/3004654401 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7ff848004c40 con 0x559addc9c070 2024-07-26T13:41:01.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.140+0000 7ff85bbd4740 1 -- 172.21.15.100:0/3004654401 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff848003e20 con 0x559addc9c070 2024-07-26T13:41:01.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.140+0000 7ff832ff5640 1 -- 172.21.15.100:0/3004654401 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff848022040 con 0x559addc9c070 2024-07-26T13:41:01.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.140+0000 7ff85bbd4740 10 client.15612 fetch_fsmap learned FSMap version 26 2024-07-26T13:41:01.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.140+0000 7ff85bbd4740 1 -- 172.21.15.100:0/3004654401 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559ade0600c0 con 0x559addc9c070 2024-07-26T13:41:01.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.140+0000 7ff832ff5640 1 -- 172.21.15.100:0/3004654401 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff848003e20 con 0x559addc9c070 2024-07-26T13:41:01.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.140+0000 7ff832ff5640 1 -- 172.21.15.100:0/3004654401 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7ff848010cc0 con 0x559addc9c070 2024-07-26T13:41:01.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.140+0000 7ff85bbd4740 10 client.15612 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:41:01.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.140+0000 7ff85bbd4740 10 client.15612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:41:01.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.140+0000 7ff85bbd4740 20 client.15612 populate_metadata read hostname 'smithi100' 2024-07-26T13:41:01.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.140+0000 7ff85bbd4740 1 --2- 172.21.15.100:0/3004654401 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559addc5a9f0 0x559ade063530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:01.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.141+0000 7ff85bbd4740 4 client.15612 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:41:01.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.141+0000 7ff85bbd4740 1 -- 172.21.15.100:0/3004654401 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559addfb9ab0 con 0x559addc5a9f0 2024-07-26T13:41:01.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.141+0000 7ff84e7fc640 1 --2- 172.21.15.100:0/3004654401 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559addc5a9f0 0x559ade063530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:01.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.141+0000 7ff832ff5640 10 client.15612 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:01.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.141+0000 7ff84e7fc640 1 --2- 172.21.15.100:0/3004654401 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559addc5a9f0 0x559ade063530 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b14f026eaf1ef0 server_cookie=e4073bce9db5b13a in_seq=0 out_seq=0 2024-07-26T13:41:01.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.142+0000 7ff832ff5640 1 -- 172.21.15.100:0/3004654401 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559addfb9ab0 con 0x559addc5a9f0 2024-07-26T13:41:01.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.142+0000 7ff832ff5640 10 client.15612 handle_command_reply: tid=0 2024-07-26T13:41:01.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.142+0000 7ff85bbd4740 1 client.15612 shutdown 2024-07-26T13:41:01.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.142+0000 7ff85bbd4740 1 -- 172.21.15.100:0/3004654401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559addc9c070 msgr2=0x559ade04da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:01.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.142+0000 7ff85bbd4740 1 --2- 172.21.15.100:0/3004654401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559addc9c070 0x559ade04da20 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff8480074f0 tx=0x7ff848004490 comp rx=0 tx=0).stop 2024-07-26T13:41:01.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.142+0000 7ff85bbd4740 1 -- 172.21.15.100:0/3004654401 shutdown_connections 2024-07-26T13:41:01.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.142+0000 7ff85bbd4740 1 --2- 172.21.15.100:0/3004654401 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559addc5a9f0 0x559ade063530 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.142+0000 7ff85bbd4740 1 --2- 172.21.15.100:0/3004654401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559addc9c070 0x559ade04da20 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.142+0000 7ff85bbd4740 1 --2- 172.21.15.100:0/3004654401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559addc7b390 0x559ade04d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.142+0000 7ff85bbd4740 1 --2- 172.21.15.100:0/3004654401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559addc731b0 0x559ade051020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.142+0000 7ff85bbd4740 1 -- 172.21.15.100:0/3004654401 >> 172.21.15.100:0/3004654401 conn(0x559addc90460 msgr2=0x559ade03dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:01.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.142+0000 7ff832ff5640 0 client.15612 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:01.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.142+0000 7ff85bbd4740 1 -- 172.21.15.100:0/3004654401 shutdown_connections 2024-07-26T13:41:01.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.142+0000 7ff85bbd4740 1 -- 172.21.15.100:0/3004654401 wait complete. 2024-07-26T13:41:01.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.143+0000 7ff85bbd4740 20 client.15612 trim_cache size 0 max 16384 2024-07-26T13:41:01.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.159+0000 7ff85bbd4740 1 Processor -- start 2024-07-26T13:41:01.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.160+0000 7ff85bbd4740 1 -- start start 2024-07-26T13:41:01.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.160+0000 7ff85bbd4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559addc90460 0x559ade042c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:01.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.160+0000 7ff85bbd4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559addc731b0 0x559ade046810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:01.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.160+0000 7ff85bbd4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559addc7b390 0x559ade046d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:01.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.160+0000 7ff85bbd4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff8501af810 con 0x559addc731b0 2024-07-26T13:41:01.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.160+0000 7ff85bbd4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff8501ad0e0 con 0x559addc90460 2024-07-26T13:41:01.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.160+0000 7ff85bbd4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff8501aaf30 con 0x559addc7b390 2024-07-26T13:41:01.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.160+0000 7ff84e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559addc731b0 0x559ade046810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:01.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.160+0000 7ff84e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559addc731b0 0x559ade046810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46984/0 (socket says 172.21.15.100:46984) 2024-07-26T13:41:01.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.160+0000 7ff84e7fc640 1 -- 172.21.15.100:0/3452246486 learned_addr learned my addr 172.21.15.100:0/3452246486 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:01.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.160+0000 7ff84f7fe640 1 --2- 172.21.15.100:0/3452246486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559addc7b390 0x559ade046d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:01.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.160+0000 7ff84f7fe640 1 --2- 172.21.15.100:0/3452246486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559addc90460 0x559ade042c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:01.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.160+0000 7ff84f7fe640 1 -- 172.21.15.100:0/3452246486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559addc90460 msgr2=0x559ade042c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:01.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.160+0000 7ff84f7fe640 1 --2- 172.21.15.100:0/3452246486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559addc90460 0x559ade042c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.160+0000 7ff84f7fe640 1 -- 172.21.15.100:0/3452246486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559addc731b0 msgr2=0x559ade046810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:01.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.160+0000 7ff84f7fe640 1 --2- 172.21.15.100:0/3452246486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559addc731b0 0x559ade046810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.160+0000 7ff84f7fe640 1 -- 172.21.15.100:0/3452246486 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ade051390 con 0x559addc7b390 2024-07-26T13:41:01.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.160+0000 7ff84e7fc640 1 --2- 172.21.15.100:0/3452246486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559addc731b0 0x559ade046810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:41:01.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.161+0000 7ff84f7fe640 1 --2- 172.21.15.100:0/3452246486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559addc7b390 0x559ade046d50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff848007b20 tx=0x7ff84800e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7262e94a2dfa0af7 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:01.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.161+0000 7ff834ff9640 1 -- 172.21.15.100:0/3452246486 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff848014040 con 0x559addc7b390 2024-07-26T13:41:01.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.161+0000 7ff834ff9640 1 -- 172.21.15.100:0/3452246486 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff848004080 con 0x559addc7b390 2024-07-26T13:41:01.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.161+0000 7ff834ff9640 1 -- 172.21.15.100:0/3452246486 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff848029420 con 0x559addc7b390 2024-07-26T13:41:01.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.162+0000 7ff85bbd4740 1 -- 172.21.15.100:0/3452246486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559addc7b390 msgr2=0x559ade046d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:01.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.162+0000 7ff85bbd4740 1 --2- 172.21.15.100:0/3452246486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559addc7b390 0x559ade046d50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff848007b20 tx=0x7ff84800e720 comp rx=0 tx=0).stop 2024-07-26T13:41:01.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.162+0000 7ff85bbd4740 1 -- 172.21.15.100:0/3452246486 shutdown_connections 2024-07-26T13:41:01.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.162+0000 7ff85bbd4740 1 --2- 172.21.15.100:0/3452246486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559addc7b390 0x559ade046d50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.162+0000 7ff85bbd4740 1 --2- 172.21.15.100:0/3452246486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559addc731b0 0x559ade046810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.162+0000 7ff85bbd4740 1 --2- 172.21.15.100:0/3452246486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559addc90460 0x559ade042c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.162+0000 7ff85bbd4740 1 -- 172.21.15.100:0/3452246486 >> 172.21.15.100:0/3452246486 conn(0x559ade0506c0 msgr2=0x559ade042550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:01.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.162+0000 7ff85bbd4740 1 -- 172.21.15.100:0/3452246486 shutdown_connections 2024-07-26T13:41:01.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.162+0000 7ff85bbd4740 1 -- 172.21.15.100:0/3452246486 wait complete. 2024-07-26T13:41:01.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.163+0000 7ff85bbd4740 1 -- start start 2024-07-26T13:41:01.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.163+0000 7ff85bbd4740 1 Processor -- start 2024-07-26T13:41:01.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.163+0000 7ff85bbd4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559addc731b0 0x559ade05b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:01.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.163+0000 7ff85bbd4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559addc7b390 0x559ade056af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:01.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.163+0000 7ff85bbd4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559addc90460 0x559ade057030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:01.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.163+0000 7ff85bbd4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff8501ab5a0 con 0x559addc90460 2024-07-26T13:41:01.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.163+0000 7ff85bbd4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff8501aca10 con 0x559addc7b390 2024-07-26T13:41:01.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.163+0000 7ff85bbd4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff8501abc70 con 0x559addc731b0 2024-07-26T13:41:01.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.163+0000 7ff85bbd4740 10 client.0 _ll_register_callbacks cb 0x7ff840107ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:41:01.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.163+0000 7ff84f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559addc90460 0x559ade057030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:01.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.163+0000 7ff84f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559addc90460 0x559ade057030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46988/0 (socket says 172.21.15.100:46988) 2024-07-26T13:41:01.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.163+0000 7ff84f7fe640 1 -- 172.21.15.100:0/2231327267 learned_addr learned my addr 172.21.15.100:0/2231327267 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:01.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.164+0000 7ff84e7fc640 1 --2- 172.21.15.100:0/2231327267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559addc7b390 0x559ade056af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:01.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.164+0000 7ff84f7fe640 1 --2- 172.21.15.100:0/2231327267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559addc731b0 0x559ade05b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:01.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.164+0000 7ff84f7fe640 1 -- 172.21.15.100:0/2231327267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559addc731b0 msgr2=0x559ade05b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:01.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.164+0000 7ff84f7fe640 1 --2- 172.21.15.100:0/2231327267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559addc731b0 0x559ade05b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.164+0000 7ff84f7fe640 1 -- 172.21.15.100:0/2231327267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559addc7b390 msgr2=0x559ade056af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:01.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.164+0000 7ff84f7fe640 1 --2- 172.21.15.100:0/2231327267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559addc7b390 0x559ade056af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.164+0000 7ff84f7fe640 1 -- 172.21.15.100:0/2231327267 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ade057770 con 0x559addc90460 2024-07-26T13:41:01.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.164+0000 7ff84f7fe640 1 --2- 172.21.15.100:0/2231327267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559addc90460 0x559ade057030 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x559ade05e760 tx=0x7ff848004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d1e577f91a6a4c server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:01.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.164+0000 7ff832ff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:41:01.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.164+0000 7ff832ff5640 1 -- 172.21.15.100:0/2231327267 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84801d070 con 0x559addc90460 2024-07-26T13:41:01.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.164+0000 7ff832ff5640 1 -- 172.21.15.100:0/2231327267 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7ff8480049d0 con 0x559addc90460 2024-07-26T13:41:01.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.164+0000 7ff85bbd4740 1 -- 172.21.15.100:0/2231327267 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff848004080 con 0x559addc90460 2024-07-26T13:41:01.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.164+0000 7ff832ff5640 1 -- 172.21.15.100:0/2231327267 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff848014030 con 0x559addc90460 2024-07-26T13:41:01.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.164+0000 7ff85bbd4740 10 client.15618 fetch_fsmap learned FSMap version 26 2024-07-26T13:41:01.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.164+0000 7ff85bbd4740 1 -- 172.21.15.100:0/2231327267 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559ade0cb950 con 0x559addc90460 2024-07-26T13:41:01.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.165+0000 7ff832ff5640 1 -- 172.21.15.100:0/2231327267 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff848004080 con 0x559addc90460 2024-07-26T13:41:01.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.165+0000 7ff832ff5640 1 -- 172.21.15.100:0/2231327267 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7ff848010760 con 0x559addc90460 2024-07-26T13:41:01.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.165+0000 7ff85bbd4740 10 client.15618 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:41:01.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.165+0000 7ff85bbd4740 10 client.15618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:41:01.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.165+0000 7ff85bbd4740 20 client.15618 populate_metadata read hostname 'smithi100' 2024-07-26T13:41:01.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.165+0000 7ff85bbd4740 1 --2- 172.21.15.100:0/2231327267 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559addc9c070 0x559ade0ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:01.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.165+0000 7ff85bbd4740 4 client.15618 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-26T13:41:01.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.165+0000 7ff85bbd4740 1 -- 172.21.15.100:0/2231327267 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559addfb5550 con 0x559addc9c070 2024-07-26T13:41:01.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.165+0000 7ff84e7fc640 1 --2- 172.21.15.100:0/2231327267 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559addc9c070 0x559ade0ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:01.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.166+0000 7ff832ff5640 10 client.15618 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:01.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.166+0000 7ff84e7fc640 1 --2- 172.21.15.100:0/2231327267 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559addc9c070 0x559ade0ce0d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a803adb5f2f71224 server_cookie=d72cc9d19e3bf843 in_seq=0 out_seq=0 2024-07-26T13:41:01.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.166+0000 7ff832ff5640 1 -- 172.21.15.100:0/2231327267 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559addfb5550 con 0x559addc9c070 2024-07-26T13:41:01.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.166+0000 7ff832ff5640 10 client.15618 handle_command_reply: tid=0 2024-07-26T13:41:01.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.166+0000 7ff85bbd4740 1 client.15618 shutdown 2024-07-26T13:41:01.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.166+0000 7ff85bbd4740 1 -- 172.21.15.100:0/2231327267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559addc90460 msgr2=0x559ade057030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:01.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.166+0000 7ff85bbd4740 1 --2- 172.21.15.100:0/2231327267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559addc90460 0x559ade057030 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x559ade05e760 tx=0x7ff848004c60 comp rx=0 tx=0).stop 2024-07-26T13:41:01.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.166+0000 7ff85bbd4740 1 -- 172.21.15.100:0/2231327267 shutdown_connections 2024-07-26T13:41:01.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.166+0000 7ff85bbd4740 1 --2- 172.21.15.100:0/2231327267 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559addc9c070 0x559ade0ce0d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.166+0000 7ff85bbd4740 1 --2- 172.21.15.100:0/2231327267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559addc90460 0x559ade057030 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.166+0000 7ff85bbd4740 1 --2- 172.21.15.100:0/2231327267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559addc7b390 0x559ade056af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.166+0000 7ff85bbd4740 1 --2- 172.21.15.100:0/2231327267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559addc731b0 0x559ade05b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.166+0000 7ff85bbd4740 1 -- 172.21.15.100:0/2231327267 >> 172.21.15.100:0/2231327267 conn(0x559ade0506c0 msgr2=0x559ade042f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:01.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.166+0000 7ff832ff5640 0 client.15618 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:01.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.166+0000 7ff85bbd4740 1 -- 172.21.15.100:0/2231327267 shutdown_connections 2024-07-26T13:41:01.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.166+0000 7ff85bbd4740 1 -- 172.21.15.100:0/2231327267 wait complete. 2024-07-26T13:41:01.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.167+0000 7ff85bbd4740 20 client.15618 trim_cache size 0 max 16384 2024-07-26T13:41:01.174 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:41:01.174 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-07-26T13:41:01.174 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-07-26T13:41:01.174 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:41:01.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.167+0000 7ff855650640 1 -- 172.21.15.100:0/3570878697 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff8380b3ce0 msgr2=0x7ff8380b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:01.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.167+0000 7ff855650640 1 --2- 172.21.15.100:0/3570878697 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff8380b3ce0 0x7ff8380b61f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff8400cb150 tx=0x7ff84010a0e0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.167+0000 7ff855650640 1 -- 172.21.15.100:0/3570878697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8501024a0 msgr2=0x7ff850198df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:01.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.167+0000 7ff855650640 1 --2- 172.21.15.100:0/3570878697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8501024a0 0x7ff850198df0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff840005ab0 tx=0x7ff840003c80 comp rx=0 tx=0).stop 2024-07-26T13:41:01.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.167+0000 7ff855650640 1 -- 172.21.15.100:0/3570878697 shutdown_connections 2024-07-26T13:41:01.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.167+0000 7ff855650640 1 --2- 172.21.15.100:0/3570878697 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff8380b3ce0 0x7ff8380b61f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.167+0000 7ff855650640 1 --2- 172.21.15.100:0/3570878697 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff850108730 0x7ff85019e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.167+0000 7ff855650640 1 --2- 172.21.15.100:0/3570878697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff850107de0 0x7ff850199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.167+0000 7ff855650640 1 --2- 172.21.15.100:0/3570878697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8501024a0 0x7ff850198df0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.167+0000 7ff855650640 1 -- 172.21.15.100:0/3570878697 >> 172.21.15.100:0/3570878697 conn(0x7ff850079160 msgr2=0x7ff850100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:01.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.167+0000 7ff855650640 1 -- 172.21.15.100:0/3570878697 shutdown_connections 2024-07-26T13:41:01.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:01.167+0000 7ff855650640 1 -- 172.21.15.100:0/3570878697 wait complete. 2024-07-26T13:41:01.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:01 smithi138 ceph-mon[36954]: pgmap v391: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-07-26T13:41:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.967+0000 7faa57f39640 1 -- 172.21.15.100:0/2133158794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa501097b0 msgr2=0x7faa5010ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.967+0000 7faa57f39640 1 --2- 172.21.15.100:0/2133158794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa501097b0 0x7faa5010ffe0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7faa48009ca0 tx=0x7faa4802a8c0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.967+0000 7faa57f39640 1 -- 172.21.15.100:0/2133158794 shutdown_connections 2024-07-26T13:41:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.967+0000 7faa57f39640 1 --2- 172.21.15.100:0/2133158794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa501097b0 0x7faa5010ffe0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.967+0000 7faa57f39640 1 --2- 172.21.15.100:0/2133158794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa50108e60 0x7faa50109270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.967+0000 7faa57f39640 1 --2- 172.21.15.100:0/2133158794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa50101a90 0x7faa50101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.967+0000 7faa57f39640 1 -- 172.21.15.100:0/2133158794 >> 172.21.15.100:0/2133158794 conn(0x7faa500798c0 msgr2=0x7faa50079cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:01.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.967+0000 7faa57f39640 1 -- 172.21.15.100:0/2133158794 shutdown_connections 2024-07-26T13:41:01.970 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.967+0000 7faa57f39640 1 -- 172.21.15.100:0/2133158794 wait complete. 2024-07-26T13:41:01.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.968+0000 7faa57f39640 1 Processor -- start 2024-07-26T13:41:01.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.968+0000 7faa57f39640 1 -- start start 2024-07-26T13:41:01.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.968+0000 7faa57f39640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa50101a90 0x7faa50199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:01.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.968+0000 7faa57f39640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa50108e60 0x7faa5019a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:01.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.969+0000 7faa57f39640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa501097b0 0x7faa5019f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:01.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.969+0000 7faa57f39640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7faa50113480 con 0x7faa501097b0 2024-07-26T13:41:01.972 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.969+0000 7faa57f39640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faa50113300 con 0x7faa50101a90 2024-07-26T13:41:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.969+0000 7faa57f39640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faa50113600 con 0x7faa50108e60 2024-07-26T13:41:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.969+0000 7faa564d6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa501097b0 0x7faa5019f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.969+0000 7faa564d6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa501097b0 0x7faa5019f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47010/0 (socket says 172.21.15.100:47010) 2024-07-26T13:41:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.969+0000 7faa564d6640 1 -- 172.21.15.100:0/2839677474 learned_addr learned my addr 172.21.15.100:0/2839677474 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.969+0000 7faa554d4640 1 --2- 172.21.15.100:0/2839677474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa50108e60 0x7faa5019a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.969+0000 7faa55cd5640 1 --2- 172.21.15.100:0/2839677474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa50101a90 0x7faa50199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.969+0000 7faa564d6640 1 -- 172.21.15.100:0/2839677474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa50108e60 msgr2=0x7faa5019a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.969+0000 7faa564d6640 1 --2- 172.21.15.100:0/2839677474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa50108e60 0x7faa5019a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.969+0000 7faa564d6640 1 -- 172.21.15.100:0/2839677474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa50101a90 msgr2=0x7faa50199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.969+0000 7faa564d6640 1 --2- 172.21.15.100:0/2839677474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa50101a90 0x7faa50199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.969+0000 7faa564d6640 1 -- 172.21.15.100:0/2839677474 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa5019fd70 con 0x7faa501097b0 2024-07-26T13:41:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.969+0000 7faa564d6640 1 --2- 172.21.15.100:0/2839677474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa501097b0 0x7faa5019f670 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7faa480098d0 tx=0x7faa480317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.969+0000 7faa46ffd640 1 -- 172.21.15.100:0/2839677474 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa48003b70 con 0x7faa501097b0 2024-07-26T13:41:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.969+0000 7faa57f39640 1 -- 172.21.15.100:0/2839677474 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa501a0000 con 0x7faa501097b0 2024-07-26T13:41:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.969+0000 7faa46ffd640 1 -- 172.21.15.100:0/2839677474 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa48003d10 con 0x7faa501097b0 2024-07-26T13:41:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.969+0000 7faa46ffd640 1 -- 172.21.15.100:0/2839677474 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4802fa20 con 0x7faa501097b0 2024-07-26T13:41:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.970+0000 7faa57f39640 1 -- 172.21.15.100:0/2839677474 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa501a67f0 con 0x7faa501097b0 2024-07-26T13:41:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.972+0000 7faa46ffd640 1 -- 172.21.15.100:0/2839677474 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 24) ==== 151134+0+0 (secure 0 0 0) 0x7faa4802fc20 con 0x7faa501097b0 2024-07-26T13:41:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.972+0000 7faa5ee54740 1 Processor -- start 2024-07-26T13:41:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.972+0000 7faa5ee54740 1 -- start start 2024-07-26T13:41:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.973+0000 7faa5ee54740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc55fff1b0 0x55bc563a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.973+0000 7faa5ee54740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc55fd62f0 0x55bc563a1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.973+0000 7faa5ee54740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc55fde4d0 0x55bc563a2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.973+0000 7faa5ee54740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55bc563173b0 con 0x55bc55fde4d0 2024-07-26T13:41:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.973+0000 7faa5ee54740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bc563068b0 con 0x55bc55fd62f0 2024-07-26T13:41:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.973+0000 7faa5ee54740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bc5630a8f0 con 0x55bc55fff1b0 2024-07-26T13:41:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.973+0000 7faa554d4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc55fde4d0 0x55bc563a2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.973+0000 7faa554d4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc55fde4d0 0x55bc563a2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47024/0 (socket says 172.21.15.100:47024) 2024-07-26T13:41:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.973+0000 7faa554d4640 1 -- 172.21.15.100:0/3285358193 learned_addr learned my addr 172.21.15.100:0/3285358193 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.973+0000 7faa554d4640 1 --2- 172.21.15.100:0/3285358193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc55fff1b0 0x55bc563a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.973+0000 7faa46ffd640 1 --2- 172.21.15.100:0/2839677474 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7faa3c0b3cb0 0x7faa3c0b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.973+0000 7faa55cd5640 1 --2- 172.21.15.100:0/3285358193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc55fd62f0 0x55bc563a1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.973+0000 7faa554d4640 1 -- 172.21.15.100:0/3285358193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc55fff1b0 msgr2=0x55bc563a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.973+0000 7faa554d4640 1 --2- 172.21.15.100:0/3285358193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc55fff1b0 0x55bc563a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.973+0000 7faa554d4640 1 -- 172.21.15.100:0/3285358193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc55fd62f0 msgr2=0x55bc563a1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.973+0000 7faa554d4640 1 --2- 172.21.15.100:0/3285358193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc55fd62f0 0x55bc563a1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.973+0000 7faa564d6640 1 --2- 172.21.15.100:0/2839677474 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7faa3c0b3cb0 0x7faa3c0b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.973+0000 7faa554d4640 1 -- 172.21.15.100:0/3285358193 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc562ad9a0 con 0x55bc55fde4d0 2024-07-26T13:41:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.973+0000 7faa55cd5640 1 --2- 172.21.15.100:0/3285358193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc55fd62f0 0x55bc563a1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:41:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.973+0000 7faa46ffd640 1 -- 172.21.15.100:0/2839677474 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7faa481054b0 con 0x7faa501097b0 2024-07-26T13:41:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.974+0000 7faa554d4640 1 --2- 172.21.15.100:0/3285358193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc55fde4d0 0x55bc563a2130 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7faa400116b0 tx=0x7faa4000a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fafdac785c9a5e9 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.974+0000 7faa3bfff640 1 -- 172.21.15.100:0/3285358193 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa40004020 con 0x55bc55fde4d0 2024-07-26T13:41:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.974+0000 7faa564d6640 1 --2- 172.21.15.100:0/2839677474 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7faa3c0b3cb0 0x7faa3c0b61c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7faa480c9520 tx=0x7faa48099a00 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.974+0000 7faa3bfff640 1 -- 172.21.15.100:0/3285358193 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa400054d0 con 0x55bc55fde4d0 2024-07-26T13:41:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.974+0000 7faa3bfff640 1 -- 172.21.15.100:0/3285358193 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa40004020 con 0x55bc55fde4d0 2024-07-26T13:41:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.975+0000 7faa5ee54740 1 -- 172.21.15.100:0/3285358193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc55fde4d0 msgr2=0x55bc563a2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.975+0000 7faa5ee54740 1 --2- 172.21.15.100:0/3285358193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc55fde4d0 0x55bc563a2130 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7faa400116b0 tx=0x7faa4000a640 comp rx=0 tx=0).stop 2024-07-26T13:41:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.975+0000 7faa5ee54740 1 -- 172.21.15.100:0/3285358193 shutdown_connections 2024-07-26T13:41:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.975+0000 7faa5ee54740 1 --2- 172.21.15.100:0/3285358193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc55fde4d0 0x55bc563a2130 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.975+0000 7faa5ee54740 1 --2- 172.21.15.100:0/3285358193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc55fd62f0 0x55bc563a1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.975+0000 7faa5ee54740 1 --2- 172.21.15.100:0/3285358193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc55fff1b0 0x55bc563a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.975+0000 7faa5ee54740 1 -- 172.21.15.100:0/3285358193 >> 172.21.15.100:0/3285358193 conn(0x55bc55ff3300 msgr2=0x55bc5639f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.975+0000 7faa5ee54740 1 -- 172.21.15.100:0/3285358193 shutdown_connections 2024-07-26T13:41:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.975+0000 7faa5ee54740 1 -- 172.21.15.100:0/3285358193 wait complete. 2024-07-26T13:41:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.975+0000 7faa5ee54740 1 -- start start 2024-07-26T13:41:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.976+0000 7faa5ee54740 1 Processor -- start 2024-07-26T13:41:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.976+0000 7faa5ee54740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc55fd62f0 0x55bc563ad250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.976+0000 7faa5ee54740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc55fde4d0 0x55bc563ad790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.976+0000 7faa5ee54740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc55fff1b0 0x55bc563b9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.976+0000 7faa5ee54740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55bc56240ab0 con 0x55bc55fd62f0 2024-07-26T13:41:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.976+0000 7faa5ee54740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bc5628ecf0 con 0x55bc55fff1b0 2024-07-26T13:41:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.976+0000 7faa5ee54740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bc563afe30 con 0x55bc55fde4d0 2024-07-26T13:41:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.976+0000 7faa5ee54740 10 client.0 _ll_register_callbacks cb 0x55bc563003e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:41:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.976+0000 7faa554d4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc55fd62f0 0x55bc563ad250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.976+0000 7faa554d4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc55fd62f0 0x55bc563ad250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47040/0 (socket says 172.21.15.100:47040) 2024-07-26T13:41:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.976+0000 7faa554d4640 1 -- 172.21.15.100:0/1891059701 learned_addr learned my addr 172.21.15.100:0/1891059701 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.976+0000 7faa55cd5640 1 --2- 172.21.15.100:0/1891059701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc55fde4d0 0x55bc563ad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.976+0000 7faa554d4640 1 --2- 172.21.15.100:0/1891059701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc55fff1b0 0x55bc563b9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.977+0000 7faa554d4640 1 -- 172.21.15.100:0/1891059701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc55fde4d0 msgr2=0x55bc563ad790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.977+0000 7faa554d4640 1 --2- 172.21.15.100:0/1891059701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc55fde4d0 0x55bc563ad790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.977+0000 7faa554d4640 1 -- 172.21.15.100:0/1891059701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc55fff1b0 msgr2=0x55bc563b9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.977+0000 7faa554d4640 1 --2- 172.21.15.100:0/1891059701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc55fff1b0 0x55bc563b9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.977+0000 7faa554d4640 1 -- 172.21.15.100:0/1891059701 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc5606a790 con 0x55bc55fd62f0 2024-07-26T13:41:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.977+0000 7faa554d4640 1 --2- 172.21.15.100:0/1891059701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc55fd62f0 0x55bc563ad250 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7faa40004940 tx=0x7faa40004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2619856412a36549 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.977+0000 7faa397fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:41:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.977+0000 7faa397fa640 1 -- 172.21.15.100:0/1891059701 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4002b060 con 0x55bc55fd62f0 2024-07-26T13:41:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.977+0000 7faa397fa640 1 -- 172.21.15.100:0/1891059701 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7faa4000f6c0 con 0x55bc55fd62f0 2024-07-26T13:41:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.977+0000 7faa5ee54740 1 -- 172.21.15.100:0/1891059701 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa400054d0 con 0x55bc55fd62f0 2024-07-26T13:41:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.977+0000 7faa397fa640 1 -- 172.21.15.100:0/1891059701 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faa4000abb0 con 0x55bc55fd62f0 2024-07-26T13:41:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.977+0000 7faa5ee54740 10 client.15630 fetch_fsmap learned FSMap version 26 2024-07-26T13:41:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.977+0000 7faa5ee54740 1 -- 172.21.15.100:0/1891059701 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bc560e3620 con 0x55bc55fd62f0 2024-07-26T13:41:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.977+0000 7faa397fa640 1 -- 172.21.15.100:0/1891059701 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa400054d0 con 0x55bc55fd62f0 2024-07-26T13:41:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.978+0000 7faa397fa640 1 -- 172.21.15.100:0/1891059701 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7faa4001e920 con 0x55bc55fd62f0 2024-07-26T13:41:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.978+0000 7faa5ee54740 10 client.15630 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:41:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.978+0000 7faa5ee54740 10 client.15630 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:41:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.978+0000 7faa5ee54740 20 client.15630 populate_metadata read hostname 'smithi100' 2024-07-26T13:41:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.978+0000 7faa5ee54740 1 --2- 172.21.15.100:0/1891059701 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55bc55fbdd80 0x55bc563bef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.978+0000 7faa5ee54740 4 client.15630 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:41:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.978+0000 7faa5ee54740 1 -- 172.21.15.100:0/1891059701 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc56319300 con 0x55bc55fbdd80 2024-07-26T13:41:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.978+0000 7faa55cd5640 1 --2- 172.21.15.100:0/1891059701 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55bc55fbdd80 0x55bc563bef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.978+0000 7faa55cd5640 1 --2- 172.21.15.100:0/1891059701 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55bc55fbdd80 0x55bc563bef80 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0c57d5fa841e536 server_cookie=57b8b346df22881a in_seq=0 out_seq=0 2024-07-26T13:41:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.978+0000 7faa397fa640 10 client.15630 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.979+0000 7faa397fa640 1 -- 172.21.15.100:0/1891059701 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bc56319300 con 0x55bc55fbdd80 2024-07-26T13:41:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.979+0000 7faa397fa640 10 client.15630 handle_command_reply: tid=0 2024-07-26T13:41:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.979+0000 7faa5ee54740 1 client.15630 shutdown 2024-07-26T13:41:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.979+0000 7faa5ee54740 1 -- 172.21.15.100:0/1891059701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc55fd62f0 msgr2=0x55bc563ad250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.979+0000 7faa5ee54740 1 --2- 172.21.15.100:0/1891059701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc55fd62f0 0x55bc563ad250 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7faa40004940 tx=0x7faa40004840 comp rx=0 tx=0).stop 2024-07-26T13:41:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.979+0000 7faa5ee54740 1 -- 172.21.15.100:0/1891059701 shutdown_connections 2024-07-26T13:41:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.979+0000 7faa5ee54740 1 --2- 172.21.15.100:0/1891059701 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55bc55fbdd80 0x55bc563bef80 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.979+0000 7faa5ee54740 1 --2- 172.21.15.100:0/1891059701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc55fff1b0 0x55bc563b9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.979+0000 7faa5ee54740 1 --2- 172.21.15.100:0/1891059701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc55fde4d0 0x55bc563ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.980+0000 7faa5ee54740 1 --2- 172.21.15.100:0/1891059701 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc55fd62f0 0x55bc563ad250 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.979+0000 7faa397fa640 0 client.15630 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.980+0000 7faa5ee54740 1 -- 172.21.15.100:0/1891059701 >> 172.21.15.100:0/1891059701 conn(0x55bc55ff3300 msgr2=0x55bc563a26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.980+0000 7faa5ee54740 1 -- 172.21.15.100:0/1891059701 shutdown_connections 2024-07-26T13:41:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.980+0000 7faa5ee54740 1 -- 172.21.15.100:0/1891059701 wait complete. 2024-07-26T13:41:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.980+0000 7faa5ee54740 20 client.15630 trim_cache size 0 max 16384 2024-07-26T13:41:01.992 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:01 smithi100 ceph-mon[32599]: pgmap v391: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-07-26T13:41:01.999 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.996+0000 7faa5ee54740 1 Processor -- start 2024-07-26T13:41:01.999 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.996+0000 7faa5ee54740 1 -- start start 2024-07-26T13:41:01.999 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.996+0000 7faa5ee54740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc55fd62f0 0x55bc563a6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:02.000 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.996+0000 7faa5ee54740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc55fde4d0 0x55bc563b8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:02.000 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.996+0000 7faa5ee54740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc55fff1b0 0x55bc563ad520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:02.000 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.996+0000 7faa5ee54740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7faa501b1900 con 0x55bc55fff1b0 2024-07-26T13:41:02.000 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.996+0000 7faa5ee54740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faa501af1d0 con 0x55bc55fde4d0 2024-07-26T13:41:02.000 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.996+0000 7faa5ee54740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faa501ad020 con 0x55bc55fd62f0 2024-07-26T13:41:02.000 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.996+0000 7faa554d4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc55fff1b0 0x55bc563ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:02.000 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.997+0000 7faa554d4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc55fff1b0 0x55bc563ad520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47048/0 (socket says 172.21.15.100:47048) 2024-07-26T13:41:02.000 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.997+0000 7faa554d4640 1 -- 172.21.15.100:0/2911086682 learned_addr learned my addr 172.21.15.100:0/2911086682 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:02.000 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.997+0000 7faa55cd5640 1 --2- 172.21.15.100:0/2911086682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc55fde4d0 0x55bc563b8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:02.000 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.997+0000 7faa554d4640 1 --2- 172.21.15.100:0/2911086682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc55fd62f0 0x55bc563a6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:02.000 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.997+0000 7faa554d4640 1 -- 172.21.15.100:0/2911086682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc55fd62f0 msgr2=0x55bc563a6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:02.000 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.997+0000 7faa554d4640 1 --2- 172.21.15.100:0/2911086682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc55fd62f0 0x55bc563a6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.997+0000 7faa554d4640 1 -- 172.21.15.100:0/2911086682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc55fde4d0 msgr2=0x55bc563b8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.997+0000 7faa554d4640 1 --2- 172.21.15.100:0/2911086682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc55fde4d0 0x55bc563b8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.997+0000 7faa554d4640 1 -- 172.21.15.100:0/2911086682 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc563a4560 con 0x55bc55fff1b0 2024-07-26T13:41:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.997+0000 7faa55cd5640 1 --2- 172.21.15.100:0/2911086682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc55fde4d0 0x55bc563b8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:41:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.997+0000 7faa554d4640 1 --2- 172.21.15.100:0/2911086682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc55fff1b0 0x55bc563ad520 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7faa400114f0 tx=0x7faa4001a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2c5a0a0ae6de25 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.997+0000 7faa3b7fe640 1 -- 172.21.15.100:0/2911086682 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4002b070 con 0x55bc55fff1b0 2024-07-26T13:41:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.997+0000 7faa3b7fe640 1 -- 172.21.15.100:0/2911086682 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa4001c370 con 0x55bc55fff1b0 2024-07-26T13:41:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.997+0000 7faa3b7fe640 1 -- 172.21.15.100:0/2911086682 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4000f6c0 con 0x55bc55fff1b0 2024-07-26T13:41:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.999+0000 7faa5ee54740 1 -- 172.21.15.100:0/2911086682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc55fff1b0 msgr2=0x55bc563ad520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.999+0000 7faa5ee54740 1 --2- 172.21.15.100:0/2911086682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc55fff1b0 0x55bc563ad520 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7faa400114f0 tx=0x7faa4001a040 comp rx=0 tx=0).stop 2024-07-26T13:41:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.999+0000 7faa5ee54740 1 -- 172.21.15.100:0/2911086682 shutdown_connections 2024-07-26T13:41:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.999+0000 7faa5ee54740 1 --2- 172.21.15.100:0/2911086682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc55fff1b0 0x55bc563ad520 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.999+0000 7faa5ee54740 1 --2- 172.21.15.100:0/2911086682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc55fde4d0 0x55bc563b8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.999+0000 7faa5ee54740 1 --2- 172.21.15.100:0/2911086682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc55fd62f0 0x55bc563a6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.999+0000 7faa5ee54740 1 -- 172.21.15.100:0/2911086682 >> 172.21.15.100:0/2911086682 conn(0x55bc55ff3300 msgr2=0x55bc563a6770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.999+0000 7faa5ee54740 1 -- 172.21.15.100:0/2911086682 shutdown_connections 2024-07-26T13:41:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.999+0000 7faa5ee54740 1 -- 172.21.15.100:0/2911086682 wait complete. 2024-07-26T13:41:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.999+0000 7faa5ee54740 1 -- start start 2024-07-26T13:41:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:01.999+0000 7faa5ee54740 1 Processor -- start 2024-07-26T13:41:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:02.000+0000 7faa5ee54740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc55fd62f0 0x55bc563b5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:02.000+0000 7faa5ee54740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc55fde4d0 0x55bc563b2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:02.000+0000 7faa5ee54740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc55fff1b0 0x55bc563b2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:02.000+0000 7faa5ee54740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7faa501ad690 con 0x55bc55fff1b0 2024-07-26T13:41:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:02.000+0000 7faa5ee54740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faa501aeb00 con 0x55bc55fd62f0 2024-07-26T13:41:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:02.000+0000 7faa5ee54740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faa501add60 con 0x55bc55fde4d0 2024-07-26T13:41:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:02.000+0000 7faa5ee54740 10 client.0 _ll_register_callbacks cb 0x7faa4000ad50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:41:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:02.000+0000 7faa554d4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc55fff1b0 0x55bc563b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:02.000+0000 7faa55cd5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc55fde4d0 0x55bc563b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:02.000+0000 7faa554d4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc55fff1b0 0x55bc563b2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47056/0 (socket says 172.21.15.100:47056) 2024-07-26T13:41:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:02.000+0000 7faa554d4640 1 -- 172.21.15.100:0/1528591165 learned_addr learned my addr 172.21.15.100:0/1528591165 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:02.000+0000 7faa554d4640 1 --2- 172.21.15.100:0/1528591165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc55fd62f0 0x55bc563b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:02.000+0000 7faa55cd5640 1 -- 172.21.15.100:0/1528591165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc55fd62f0 msgr2=0x55bc563b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:02.000+0000 7faa55cd5640 1 --2- 172.21.15.100:0/1528591165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc55fd62f0 0x55bc563b5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:02.000+0000 7faa55cd5640 1 -- 172.21.15.100:0/1528591165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc55fff1b0 msgr2=0x55bc563b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:02.000+0000 7faa55cd5640 1 --2- 172.21.15.100:0/1528591165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc55fff1b0 0x55bc563b2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:02.000+0000 7faa55cd5640 1 -- 172.21.15.100:0/1528591165 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc563a3be0 con 0x55bc55fde4d0 2024-07-26T13:41:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:02.000+0000 7faa554d4640 1 --2- 172.21.15.100:0/1528591165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc55fff1b0 0x55bc563b2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:41:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:02.001+0000 7faa55cd5640 1 --2- 172.21.15.100:0/1528591165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc55fde4d0 0x55bc563b2050 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x55bc563a4490 tx=0x7faa4c0138f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e227c7fdbaee428 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:02.001+0000 7faa397fa640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-07-26T13:41:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:02.001+0000 7faa397fa640 1 -- 172.21.15.100:0/1528591165 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4c004250 con 0x55bc55fde4d0 2024-07-26T13:41:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:02.001+0000 7faa397fa640 1 -- 172.21.15.100:0/1528591165 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7faa4c0043f0 con 0x55bc55fde4d0 2024-07-26T13:41:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:02.001+0000 7faa5ee54740 1 -- 172.21.15.100:0/1528591165 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa4001c370 con 0x55bc55fde4d0 2024-07-26T13:41:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:02.001+0000 7faa397fa640 1 -- 172.21.15.100:0/1528591165 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faa4c02d050 con 0x55bc55fde4d0 2024-07-26T13:41:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:02.001+0000 7faa5ee54740 10 client.25228 fetch_fsmap learned FSMap version 26 2024-07-26T13:41:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:02.001+0000 7faa5ee54740 1 -- 172.21.15.100:0/1528591165 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bc564260a0 con 0x55bc55fde4d0 2024-07-26T13:41:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:02.002+0000 7faa397fa640 1 -- 172.21.15.100:0/1528591165 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa4001c370 con 0x55bc55fde4d0 2024-07-26T13:41:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:02.002+0000 7faa397fa640 1 -- 172.21.15.100:0/1528591165 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7faa4c003860 con 0x55bc55fde4d0 2024-07-26T13:41:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:02.002+0000 7faa5ee54740 10 client.25228 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:41:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:02.002+0000 7faa5ee54740 10 client.25228 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:41:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:02.002+0000 7faa5ee54740 20 client.25228 populate_metadata read hostname 'smithi100' 2024-07-26T13:41:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:02.002+0000 7faa5ee54740 1 --2- 172.21.15.100:0/1528591165 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55bc55f931c0 0x55bc56428810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:02.002+0000 7faa5ee54740 4 client.25228 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-26T13:41:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:02.002+0000 7faa5ee54740 1 -- 172.21.15.100:0/1528591165 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bc56314da0 con 0x55bc55f931c0 2024-07-26T13:41:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:02.002+0000 7faa554d4640 1 --2- 172.21.15.100:0/1528591165 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55bc55f931c0 0x55bc56428810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:02.003+0000 7faa554d4640 1 --2- 172.21.15.100:0/1528591165 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55bc55f931c0 0x55bc56428810 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1d87a202f157775 server_cookie=ff7c6bd585298610 in_seq=0 out_seq=0 2024-07-26T13:41:02.010 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:02.003+0000 7faa397fa640 10 client.25228 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:02.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:01 smithi167 ceph-mon[37140]: pgmap v391: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-07-26T13:41:02.023 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-26T13:41:02.024 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:41:02.293 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:41:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:02.551+0000 7faa46ffd640 1 -- 172.21.15.100:0/2839677474 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7faa480c8540 con 0x7faa501097b0 2024-07-26T13:41:03.770 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:03 smithi100 ceph-mon[32599]: pgmap v392: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-07-26T13:41:03.770 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:03 smithi100 ceph-mon[32599]: mgrmap e25: x(active, since 10m), standbys: y, z 2024-07-26T13:41:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:03 smithi138 ceph-mon[36954]: pgmap v392: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-07-26T13:41:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:03 smithi138 ceph-mon[36954]: mgrmap e25: x(active, since 10m), standbys: y, z 2024-07-26T13:41:04.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:03 smithi167 ceph-mon[37140]: pgmap v392: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-07-26T13:41:04.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:03 smithi167 ceph-mon[37140]: mgrmap e25: x(active, since 10m), standbys: y, z 2024-07-26T13:41:04.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.131+0000 7f5d85aff640 1 -- 172.21.15.100:0/814869755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d80075b30 msgr2=0x7f5d8010f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:04.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.131+0000 7f5d85aff640 1 --2- 172.21.15.100:0/814869755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d80075b30 0x7f5d8010f5c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5d70009900 tx=0x7f5d7002a410 comp rx=0 tx=0).stop 2024-07-26T13:41:04.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.131+0000 7f5d85aff640 1 -- 172.21.15.100:0/814869755 shutdown_connections 2024-07-26T13:41:04.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.131+0000 7f5d85aff640 1 --2- 172.21.15.100:0/814869755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d8010fb00 0x7f5d80111f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:04.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.131+0000 7f5d85aff640 1 --2- 172.21.15.100:0/814869755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d80075b30 0x7f5d8010f5c0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:04.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.131+0000 7f5d85aff640 1 --2- 172.21.15.100:0/814869755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d800751e0 0x7f5d800755f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:04.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.131+0000 7f5d85aff640 1 -- 172.21.15.100:0/814869755 >> 172.21.15.100:0/814869755 conn(0x7f5d800ff010 msgr2=0x7f5d80101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:04.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.131+0000 7f5d85aff640 1 -- 172.21.15.100:0/814869755 shutdown_connections 2024-07-26T13:41:04.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.131+0000 7f5d85aff640 1 -- 172.21.15.100:0/814869755 wait complete. 2024-07-26T13:41:04.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.132+0000 7f5d85aff640 1 Processor -- start 2024-07-26T13:41:04.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.132+0000 7f5d85aff640 1 -- start start 2024-07-26T13:41:04.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.133+0000 7f5d85aff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d800751e0 0x7f5d8019c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:04.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.133+0000 7f5d85aff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d80075b30 0x7f5d8019c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:04.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.133+0000 7f5d85aff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d8010fb00 0x7f5d801a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:04.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.133+0000 7f5d85aff640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5d801156a0 con 0x7f5d800751e0 2024-07-26T13:41:04.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.133+0000 7f5d85aff640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5d80115520 con 0x7f5d8010fb00 2024-07-26T13:41:04.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.133+0000 7f5d85aff640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5d80115820 con 0x7f5d80075b30 2024-07-26T13:41:04.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.133+0000 7f5d7f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d800751e0 0x7f5d8019c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:04.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.133+0000 7f5d7f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d800751e0 0x7f5d8019c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47084/0 (socket says 172.21.15.100:47084) 2024-07-26T13:41:04.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.133+0000 7f5d7f7fe640 1 -- 172.21.15.100:0/2090177808 learned_addr learned my addr 172.21.15.100:0/2090177808 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:04.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.133+0000 7f5d7effd640 1 --2- 172.21.15.100:0/2090177808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d80075b30 0x7f5d8019c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:04.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.133+0000 7f5d7f7fe640 1 -- 172.21.15.100:0/2090177808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d80075b30 msgr2=0x7f5d8019c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:04.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.133+0000 7f5d7f7fe640 1 --2- 172.21.15.100:0/2090177808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d80075b30 0x7f5d8019c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:04.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.133+0000 7f5d7ffff640 1 --2- 172.21.15.100:0/2090177808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d8010fb00 0x7f5d801a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:04.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.133+0000 7f5d7f7fe640 1 -- 172.21.15.100:0/2090177808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d8010fb00 msgr2=0x7f5d801a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:04.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.133+0000 7f5d7f7fe640 1 --2- 172.21.15.100:0/2090177808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d8010fb00 0x7f5d801a1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:04.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.133+0000 7f5d7f7fe640 1 -- 172.21.15.100:0/2090177808 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d801a2140 con 0x7f5d800751e0 2024-07-26T13:41:04.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.133+0000 7f5d7effd640 1 --2- 172.21.15.100:0/2090177808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d80075b30 0x7f5d8019c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:41:04.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.133+0000 7f5d7ffff640 1 --2- 172.21.15.100:0/2090177808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d8010fb00 0x7f5d801a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:41:04.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.133+0000 7f5d7f7fe640 1 --2- 172.21.15.100:0/2090177808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d800751e0 0x7f5d8019c1d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5d6800d950 tx=0x7f5d6800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:04.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.133+0000 7f5d7cff9640 1 -- 172.21.15.100:0/2090177808 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d68017070 con 0x7f5d800751e0 2024-07-26T13:41:04.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.133+0000 7f5d7cff9640 1 -- 172.21.15.100:0/2090177808 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d68009c30 con 0x7f5d800751e0 2024-07-26T13:41:04.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.134+0000 7f5d85aff640 1 -- 172.21.15.100:0/2090177808 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d801a2430 con 0x7f5d800751e0 2024-07-26T13:41:04.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.134+0000 7f5d7cff9640 1 -- 172.21.15.100:0/2090177808 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6800f620 con 0x7f5d800751e0 2024-07-26T13:41:04.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.134+0000 7f5d85aff640 1 -- 172.21.15.100:0/2090177808 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d801a8b90 con 0x7f5d800751e0 2024-07-26T13:41:04.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.135+0000 7f5d85aff640 1 -- 172.21.15.100:0/2090177808 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d800755f0 con 0x7f5d800751e0 2024-07-26T13:41:04.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.136+0000 7f5d7cff9640 1 -- 172.21.15.100:0/2090177808 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f5d68005030 con 0x7f5d800751e0 2024-07-26T13:41:04.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.137+0000 7f5d7cff9640 1 --2- 172.21.15.100:0/2090177808 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5d6c0b3dc0 0x7f5d6c0b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:04.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.137+0000 7f5d7effd640 1 --2- 172.21.15.100:0/2090177808 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5d6c0b3dc0 0x7f5d6c0b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:04.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.138+0000 7f5d7cff9640 1 -- 172.21.15.100:0/2090177808 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f5d680e3c30 con 0x7f5d800751e0 2024-07-26T13:41:04.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.138+0000 7f5d7effd640 1 --2- 172.21.15.100:0/2090177808 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5d6c0b3dc0 0x7f5d6c0b62d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5d70000c00 tx=0x7f5d70032da0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:04.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.140+0000 7f5d7cff9640 1 -- 172.21.15.100:0/2090177808 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f5d680aafe0 con 0x7f5d800751e0 2024-07-26T13:41:04.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.398+0000 7f5d85aff640 1 -- 172.21.15.100:0/2090177808 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d801a8ec0 con 0x7f5d800751e0 2024-07-26T13:41:04.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.399+0000 7f5d7cff9640 1 -- 172.21.15.100:0/2090177808 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11927 (secure 0 0 0) 0x7f5d68009de0 con 0x7f5d800751e0 2024-07-26T13:41:04.401 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:41:04.401 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":26,"btime":"2024-07-26T13:40:09:530496+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14790,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/313501409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":313501409},{"type":"v1","addr":"172.21.15.100:6839","nonce":313501409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/1402975506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":1402975506},{"type":"v1","addr":"172.21.15.100:6841","nonce":1402975506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-26T13:37:01.281843+0000","modified":"2024-07-26T13:40:09.187125+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24607,"mds_1":24605,"mds_2":24595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24557":{"gid":24557,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/3290105300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3290105300},{"type":"v1","addr":"172.21.15.138:6835","nonce":3290105300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/3393225483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3393225483},{"type":"v1","addr":"172.21.15.138:6837","nonce":3393225483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24595":{"gid":24595,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6837/3828634468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3828634468},{"type":"v1","addr":"172.21.15.167:6837","nonce":3828634468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.138:6839/1580925471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1580925471},{"type":"v1","addr":"172.21.15.138:6839","nonce":1580925471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/398150514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":398150514},{"type":"v1","addr":"172.21.15.167:6839","nonce":398150514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24607,"qdb_cluster":[24595,24605,24607]},"id":1}]} 2024-07-26T13:41:04.401 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 26 2024-07-26T13:41:04.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.401+0000 7f5d85aff640 1 -- 172.21.15.100:0/2090177808 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5d6c0b3dc0 msgr2=0x7f5d6c0b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:04.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.401+0000 7f5d85aff640 1 --2- 172.21.15.100:0/2090177808 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5d6c0b3dc0 0x7f5d6c0b62d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5d70000c00 tx=0x7f5d70032da0 comp rx=0 tx=0).stop 2024-07-26T13:41:04.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.401+0000 7f5d85aff640 1 -- 172.21.15.100:0/2090177808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d800751e0 msgr2=0x7f5d8019c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:04.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.401+0000 7f5d85aff640 1 --2- 172.21.15.100:0/2090177808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d800751e0 0x7f5d8019c1d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5d6800d950 tx=0x7f5d6800de10 comp rx=0 tx=0).stop 2024-07-26T13:41:04.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.401+0000 7f5d85aff640 1 -- 172.21.15.100:0/2090177808 shutdown_connections 2024-07-26T13:41:04.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.401+0000 7f5d85aff640 1 --2- 172.21.15.100:0/2090177808 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5d6c0b3dc0 0x7f5d6c0b62d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:04.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.401+0000 7f5d85aff640 1 --2- 172.21.15.100:0/2090177808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d8010fb00 0x7f5d801a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:04.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.401+0000 7f5d85aff640 1 --2- 172.21.15.100:0/2090177808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d80075b30 0x7f5d8019c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:04.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.401+0000 7f5d85aff640 1 --2- 172.21.15.100:0/2090177808 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d800751e0 0x7f5d8019c1d0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:04.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.401+0000 7f5d85aff640 1 -- 172.21.15.100:0/2090177808 >> 172.21.15.100:0/2090177808 conn(0x7f5d800ff010 msgr2=0x7f5d801096e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:04.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.402+0000 7f5d85aff640 1 -- 172.21.15.100:0/2090177808 shutdown_connections 2024-07-26T13:41:04.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:04.402+0000 7f5d85aff640 1 -- 172.21.15.100:0/2090177808 wait complete. 2024-07-26T13:41:04.702 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:04 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2090177808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:41:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:04.713+0000 7faa397fa640 1 -- 172.21.15.100:0/1528591165 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bc56314da0 con 0x55bc55f931c0 2024-07-26T13:41:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:04.713+0000 7faa397fa640 10 client.25228 handle_command_reply: tid=0 2024-07-26T13:41:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:04.713+0000 7faa5ee54740 1 client.25228 shutdown 2024-07-26T13:41:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:04.713+0000 7faa5ee54740 1 -- 172.21.15.100:0/1528591165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc55fde4d0 msgr2=0x55bc563b2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:04.713+0000 7faa5ee54740 1 --2- 172.21.15.100:0/1528591165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc55fde4d0 0x55bc563b2050 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x55bc563a4490 tx=0x7faa4c0138f0 comp rx=0 tx=0).stop 2024-07-26T13:41:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:04.713+0000 7faa5ee54740 1 -- 172.21.15.100:0/1528591165 shutdown_connections 2024-07-26T13:41:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:04.713+0000 7faa5ee54740 1 --2- 172.21.15.100:0/1528591165 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55bc55f931c0 0x55bc56428810 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:04.713+0000 7faa5ee54740 1 --2- 172.21.15.100:0/1528591165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55bc55fff1b0 0x55bc563b2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:04.713+0000 7faa5ee54740 1 --2- 172.21.15.100:0/1528591165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc55fde4d0 0x55bc563b2050 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:04.713+0000 7faa397fa640 0 client.25228 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:04.713+0000 7faa5ee54740 1 --2- 172.21.15.100:0/1528591165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc55fd62f0 0x55bc563b5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:04.713+0000 7faa5ee54740 1 -- 172.21.15.100:0/1528591165 >> 172.21.15.100:0/1528591165 conn(0x55bc55ff3300 msgr2=0x55bc5639ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:04.713+0000 7faa5ee54740 1 -- 172.21.15.100:0/1528591165 shutdown_connections 2024-07-26T13:41:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:04.713+0000 7faa5ee54740 1 -- 172.21.15.100:0/1528591165 wait complete. 2024-07-26T13:41:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:04.713+0000 7faa5ee54740 20 client.25228 trim_cache size 0 max 16384 2024-07-26T13:41:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-07-26T13:41:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-07-26T13:41:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-07-26T13:41:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-07-26T13:41:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:04.714+0000 7faa57f39640 1 -- 172.21.15.100:0/2839677474 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7faa3c0b3cb0 msgr2=0x7faa3c0b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:04.714+0000 7faa57f39640 1 --2- 172.21.15.100:0/2839677474 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7faa3c0b3cb0 0x7faa3c0b61c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7faa480c9520 tx=0x7faa48099a00 comp rx=0 tx=0).stop 2024-07-26T13:41:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:04.714+0000 7faa57f39640 1 -- 172.21.15.100:0/2839677474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa501097b0 msgr2=0x7faa5019f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:04.714+0000 7faa57f39640 1 --2- 172.21.15.100:0/2839677474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa501097b0 0x7faa5019f670 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7faa480098d0 tx=0x7faa480317b0 comp rx=0 tx=0).stop 2024-07-26T13:41:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:04.714+0000 7faa57f39640 1 -- 172.21.15.100:0/2839677474 shutdown_connections 2024-07-26T13:41:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:04.714+0000 7faa57f39640 1 --2- 172.21.15.100:0/2839677474 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7faa3c0b3cb0 0x7faa3c0b61c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:04.714+0000 7faa57f39640 1 --2- 172.21.15.100:0/2839677474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa501097b0 0x7faa5019f670 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:04.714+0000 7faa57f39640 1 --2- 172.21.15.100:0/2839677474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa50108e60 0x7faa5019a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:04.714+0000 7faa57f39640 1 --2- 172.21.15.100:0/2839677474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa50101a90 0x7faa50199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:04.714+0000 7faa57f39640 1 -- 172.21.15.100:0/2839677474 >> 172.21.15.100:0/2839677474 conn(0x7faa500798c0 msgr2=0x7faa50104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:04.714+0000 7faa57f39640 1 -- 172.21.15.100:0/2839677474 shutdown_connections 2024-07-26T13:41:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:04.714+0000 7faa57f39640 1 -- 172.21.15.100:0/2839677474 wait complete. 2024-07-26T13:41:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:04 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/2090177808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:41:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:04 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/2090177808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:41:05.219 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-26T13:41:05.510 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:41:05.910 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:05 smithi100 ceph-mon[32599]: pgmap v393: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-07-26T13:41:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:05 smithi138 ceph-mon[36954]: pgmap v393: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-07-26T13:41:06.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:05 smithi167 ceph-mon[37140]: pgmap v393: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-07-26T13:41:06.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:06 smithi100 ceph-mon[32599]: pgmap v394: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-07-26T13:41:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:06 smithi138 ceph-mon[36954]: pgmap v394: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-07-26T13:41:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:06 smithi167 ceph-mon[37140]: pgmap v394: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-07-26T13:41:07.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.332+0000 7f0928a6c640 1 -- 172.21.15.100:0/891858282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09241057d0 msgr2=0x7f0924105be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:07.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.332+0000 7f0928a6c640 1 --2- 172.21.15.100:0/891858282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09241057d0 0x7f0924105be0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0914009900 tx=0x7f091402a410 comp rx=0 tx=0).stop 2024-07-26T13:41:07.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.332+0000 7f0928a6c640 1 -- 172.21.15.100:0/891858282 shutdown_connections 2024-07-26T13:41:07.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.332+0000 7f0928a6c640 1 --2- 172.21.15.100:0/891858282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f092410ba60 0x7f0924112290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:07.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.332+0000 7f0928a6c640 1 --2- 172.21.15.100:0/891858282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f092410b110 0x7f092410b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:07.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.332+0000 7f0928a6c640 1 --2- 172.21.15.100:0/891858282 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09241057d0 0x7f0924105be0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:07.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.332+0000 7f0928a6c640 1 -- 172.21.15.100:0/891858282 >> 172.21.15.100:0/891858282 conn(0x7f0924101200 msgr2=0x7f0924103650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:07.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.332+0000 7f0928a6c640 1 -- 172.21.15.100:0/891858282 shutdown_connections 2024-07-26T13:41:07.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.332+0000 7f0928a6c640 1 -- 172.21.15.100:0/891858282 wait complete. 2024-07-26T13:41:07.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.333+0000 7f0928a6c640 1 Processor -- start 2024-07-26T13:41:07.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.333+0000 7f0928a6c640 1 -- start start 2024-07-26T13:41:07.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.333+0000 7f0928a6c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09241057d0 0x7f092419c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:07.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.333+0000 7f0928a6c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f092410b110 0x7f092419c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:07.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.333+0000 7f0928a6c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f092410ba60 0x7f09241a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:07.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.333+0000 7f0928a6c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0924115710 con 0x7f09241057d0 2024-07-26T13:41:07.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.333+0000 7f0928a6c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0924115590 con 0x7f092410ba60 2024-07-26T13:41:07.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.333+0000 7f0928a6c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0924115890 con 0x7f092410b110 2024-07-26T13:41:07.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.333+0000 7f092259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09241057d0 0x7f092419c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:07.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.333+0000 7f092259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09241057d0 0x7f092419c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44546/0 (socket says 172.21.15.100:44546) 2024-07-26T13:41:07.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.333+0000 7f092259c640 1 -- 172.21.15.100:0/405528601 learned_addr learned my addr 172.21.15.100:0/405528601 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:07.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.333+0000 7f0921d9b640 1 --2- 172.21.15.100:0/405528601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f092410b110 0x7f092419c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:07.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.333+0000 7f0922d9d640 1 --2- 172.21.15.100:0/405528601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f092410ba60 0x7f09241a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:07.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.334+0000 7f092259c640 1 -- 172.21.15.100:0/405528601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f092410b110 msgr2=0x7f092419c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:07.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.334+0000 7f092259c640 1 --2- 172.21.15.100:0/405528601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f092410b110 0x7f092419c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:07.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.334+0000 7f092259c640 1 -- 172.21.15.100:0/405528601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f092410ba60 msgr2=0x7f09241a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:07.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.334+0000 7f092259c640 1 --2- 172.21.15.100:0/405528601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f092410ba60 0x7f09241a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:07.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.334+0000 7f092259c640 1 -- 172.21.15.100:0/405528601 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09241a2090 con 0x7f09241057d0 2024-07-26T13:41:07.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.334+0000 7f0922d9d640 1 --2- 172.21.15.100:0/405528601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f092410ba60 0x7f09241a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:41:07.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.334+0000 7f092259c640 1 --2- 172.21.15.100:0/405528601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09241057d0 0x7f092419c120 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0914005940 tx=0x7f0914004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:07.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.334+0000 7f09137fe640 1 -- 172.21.15.100:0/405528601 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091403f030 con 0x7f09241057d0 2024-07-26T13:41:07.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.334+0000 7f0928a6c640 1 -- 172.21.15.100:0/405528601 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09241a2320 con 0x7f09241057d0 2024-07-26T13:41:07.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.334+0000 7f09137fe640 1 -- 172.21.15.100:0/405528601 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0914003aa0 con 0x7f09241057d0 2024-07-26T13:41:07.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.334+0000 7f09137fe640 1 -- 172.21.15.100:0/405528601 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0914035070 con 0x7f09241057d0 2024-07-26T13:41:07.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.334+0000 7f0928a6c640 1 -- 172.21.15.100:0/405528601 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09241a8ae0 con 0x7f09241057d0 2024-07-26T13:41:07.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.337+0000 7f092eff0740 1 Processor -- start 2024-07-26T13:41:07.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.337+0000 7f09137fe640 1 -- 172.21.15.100:0/405528601 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f0914042030 con 0x7f09241057d0 2024-07-26T13:41:07.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.337+0000 7f092eff0740 1 -- start start 2024-07-26T13:41:07.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.338+0000 7f092eff0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f346619070 0x55f3469be570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:07.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.338+0000 7f092eff0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3465f01b0 0x55f3469beab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:07.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.338+0000 7f092eff0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3465f8390 0x55f3469beff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:07.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.338+0000 7f092eff0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55f346934bb0 con 0x55f3465f8390 2024-07-26T13:41:07.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.338+0000 7f092eff0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f3469240b0 con 0x55f346619070 2024-07-26T13:41:07.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.338+0000 7f092eff0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f3469280f0 con 0x55f3465f01b0 2024-07-26T13:41:07.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.338+0000 7f0921d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3465f8390 0x55f3469beff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:07.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.338+0000 7f0921d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3465f8390 0x55f3469beff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44552/0 (socket says 172.21.15.100:44552) 2024-07-26T13:41:07.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.338+0000 7f0921d9b640 1 -- 172.21.15.100:0/4219654246 learned_addr learned my addr 172.21.15.100:0/4219654246 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:07.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.338+0000 7f092259c640 1 --2- 172.21.15.100:0/4219654246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3465f01b0 0x55f3469beab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:07.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.338+0000 7f0922d9d640 1 --2- 172.21.15.100:0/4219654246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f346619070 0x55f3469be570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:07.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.338+0000 7f0921d9b640 1 -- 172.21.15.100:0/4219654246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3465f01b0 msgr2=0x55f3469beab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:07.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.338+0000 7f0921d9b640 1 --2- 172.21.15.100:0/4219654246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3465f01b0 0x55f3469beab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:07.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.338+0000 7f0921d9b640 1 -- 172.21.15.100:0/4219654246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f346619070 msgr2=0x55f3469be570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:07.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.338+0000 7f0921d9b640 1 --2- 172.21.15.100:0/4219654246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f346619070 0x55f3469be570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:07.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.338+0000 7f0921d9b640 1 -- 172.21.15.100:0/4219654246 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3468cab90 con 0x55f3465f8390 2024-07-26T13:41:07.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.338+0000 7f09137fe640 1 --2- 172.21.15.100:0/405528601 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f090c0b3d40 0x7f090c0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:07.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.339+0000 7f0921d9b640 1 --2- 172.21.15.100:0/4219654246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3465f8390 0x55f3469beff0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0918009c20 tx=0x7f0918011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab05b4955b252f5 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:07.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.339+0000 7f0922d9d640 1 --2- 172.21.15.100:0/405528601 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f090c0b3d40 0x7f090c0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:07.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.339+0000 7f09137fe640 1 -- 172.21.15.100:0/405528601 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f0914104790 con 0x7f09241057d0 2024-07-26T13:41:07.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.339+0000 7f0910ff9640 1 -- 172.21.15.100:0/4219654246 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0918004230 con 0x55f3465f8390 2024-07-26T13:41:07.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.339+0000 7f0910ff9640 1 -- 172.21.15.100:0/4219654246 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09180043d0 con 0x55f3465f8390 2024-07-26T13:41:07.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.339+0000 7f0910ff9640 1 -- 172.21.15.100:0/4219654246 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0918004d50 con 0x55f3465f8390 2024-07-26T13:41:07.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.340+0000 7f092eff0740 1 -- 172.21.15.100:0/4219654246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3465f8390 msgr2=0x55f3469beff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:07.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.340+0000 7f092eff0740 1 --2- 172.21.15.100:0/4219654246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3465f8390 0x55f3469beff0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0918009c20 tx=0x7f0918011810 comp rx=0 tx=0).stop 2024-07-26T13:41:07.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.340+0000 7f0922d9d640 1 --2- 172.21.15.100:0/405528601 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f090c0b3d40 0x7f090c0b6250 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f091c005fd0 tx=0x7f091c009590 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:07.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.340+0000 7f092eff0740 1 -- 172.21.15.100:0/4219654246 shutdown_connections 2024-07-26T13:41:07.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.340+0000 7f092eff0740 1 --2- 172.21.15.100:0/4219654246 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3465f8390 0x55f3469beff0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:07.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.340+0000 7f092eff0740 1 --2- 172.21.15.100:0/4219654246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3465f01b0 0x55f3469beab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:07.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.340+0000 7f092eff0740 1 --2- 172.21.15.100:0/4219654246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f346619070 0x55f3469be570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:07.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.340+0000 7f092eff0740 1 -- 172.21.15.100:0/4219654246 >> 172.21.15.100:0/4219654246 conn(0x55f34660d460 msgr2=0x55f3469bc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:07.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.340+0000 7f092eff0740 1 -- 172.21.15.100:0/4219654246 shutdown_connections 2024-07-26T13:41:07.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.340+0000 7f092eff0740 1 -- 172.21.15.100:0/4219654246 wait complete. 2024-07-26T13:41:07.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.341+0000 7f092eff0740 1 -- start start 2024-07-26T13:41:07.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.341+0000 7f092eff0740 1 Processor -- start 2024-07-26T13:41:07.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.341+0000 7f092eff0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3465f01b0 0x55f3469ca140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:07.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.341+0000 7f092eff0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3465f8390 0x55f3469ca680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:07.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.341+0000 7f092eff0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f346619070 0x55f3469d60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:07.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.341+0000 7f092eff0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55f346787700 con 0x55f346619070 2024-07-26T13:41:07.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.341+0000 7f092eff0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f3468c69e0 con 0x55f3465f8390 2024-07-26T13:41:07.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.341+0000 7f092eff0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f3469ccd50 con 0x55f3465f01b0 2024-07-26T13:41:07.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.341+0000 7f092eff0740 10 client.0 _ll_register_callbacks cb 0x55f3468e14f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:41:07.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.341+0000 7f0922d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f346619070 0x55f3469d60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:07.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.341+0000 7f0922d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f346619070 0x55f3469d60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44564/0 (socket says 172.21.15.100:44564) 2024-07-26T13:41:07.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.341+0000 7f0922d9d640 1 -- 172.21.15.100:0/3889592705 learned_addr learned my addr 172.21.15.100:0/3889592705 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:07.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.341+0000 7f092259c640 1 --2- 172.21.15.100:0/3889592705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3465f01b0 0x55f3469ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:07.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.341+0000 7f0921d9b640 1 --2- 172.21.15.100:0/3889592705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3465f8390 0x55f3469ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:07.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.342+0000 7f0922d9d640 1 -- 172.21.15.100:0/3889592705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3465f01b0 msgr2=0x55f3469ca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:07.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.342+0000 7f0922d9d640 1 --2- 172.21.15.100:0/3889592705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3465f01b0 0x55f3469ca140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:07.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.342+0000 7f0922d9d640 1 -- 172.21.15.100:0/3889592705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3465f8390 msgr2=0x55f3469ca680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:07.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.342+0000 7f0922d9d640 1 --2- 172.21.15.100:0/3889592705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3465f8390 0x55f3469ca680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:07.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.342+0000 7f0922d9d640 1 -- 172.21.15.100:0/3889592705 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f346684650 con 0x55f346619070 2024-07-26T13:41:07.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.342+0000 7f0922d9d640 1 --2- 172.21.15.100:0/3889592705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f346619070 0x55f3469d60a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f091c012920 tx=0x7f091c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53ed31f0a35b1a8 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:07.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.342+0000 7f0909ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:41:07.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.342+0000 7f0909ffb640 1 -- 172.21.15.100:0/3889592705 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091c00fd60 con 0x55f346619070 2024-07-26T13:41:07.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.342+0000 7f0909ffb640 1 -- 172.21.15.100:0/3889592705 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f091c01f420 con 0x55f346619070 2024-07-26T13:41:07.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.342+0000 7f092eff0740 1 -- 172.21.15.100:0/3889592705 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09180043d0 con 0x55f346619070 2024-07-26T13:41:07.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.342+0000 7f0909ffb640 1 -- 172.21.15.100:0/3889592705 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f091c01f620 con 0x55f346619070 2024-07-26T13:41:07.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.342+0000 7f092eff0740 10 client.15654 fetch_fsmap learned FSMap version 26 2024-07-26T13:41:07.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.342+0000 7f092eff0740 1 -- 172.21.15.100:0/3889592705 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f3469d8e30 con 0x55f346619070 2024-07-26T13:41:07.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.343+0000 7f0909ffb640 1 -- 172.21.15.100:0/3889592705 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09180043d0 con 0x55f346619070 2024-07-26T13:41:07.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.343+0000 7f0909ffb640 1 -- 172.21.15.100:0/3889592705 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f091c0307b0 con 0x55f346619070 2024-07-26T13:41:07.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.343+0000 7f092eff0740 10 client.15654 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:41:07.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.343+0000 7f092eff0740 10 client.15654 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-26T13:41:07.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.343+0000 7f092eff0740 20 client.15654 populate_metadata read hostname 'smithi100' 2024-07-26T13:41:07.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.343+0000 7f092eff0740 1 --2- 172.21.15.100:0/3889592705 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55f3465d79f0 0x55f3469dc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:07.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.343+0000 7f092eff0740 4 client.15654 mds_command: new command op to 24595 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:41:07.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.343+0000 7f092eff0740 1 -- 172.21.15.100:0/3889592705 --> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f346936b00 con 0x55f3465d79f0 2024-07-26T13:41:07.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.343+0000 7f092259c640 1 --2- 172.21.15.100:0/3889592705 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55f3465d79f0 0x55f3469dc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:07.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.344+0000 7f0909ffb640 10 client.15654 ms_handle_connect on v2:172.21.15.167:6836/3828634468 2024-07-26T13:41:07.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.344+0000 7f092259c640 1 --2- 172.21.15.100:0/3889592705 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55f3465d79f0 0x55f3469dc2a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2a0a5fe9f6152610 server_cookie=ed61009e415ecd4c in_seq=0 out_seq=0 2024-07-26T13:41:07.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.345+0000 7f0909ffb640 1 -- 172.21.15.100:0/3889592705 <== mds.2 v2:172.21.15.167:6836/3828634468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f346936b00 con 0x55f3465d79f0 2024-07-26T13:41:07.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.345+0000 7f0909ffb640 10 client.15654 handle_command_reply: tid=0 2024-07-26T13:41:07.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.345+0000 7f092eff0740 1 client.15654 shutdown 2024-07-26T13:41:07.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.345+0000 7f092eff0740 1 -- 172.21.15.100:0/3889592705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f346619070 msgr2=0x55f3469d60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:07.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.345+0000 7f092eff0740 1 --2- 172.21.15.100:0/3889592705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f346619070 0x55f3469d60a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f091c012920 tx=0x7f091c004a40 comp rx=0 tx=0).stop 2024-07-26T13:41:07.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.345+0000 7f092eff0740 1 -- 172.21.15.100:0/3889592705 shutdown_connections 2024-07-26T13:41:07.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.345+0000 7f092eff0740 1 --2- 172.21.15.100:0/3889592705 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55f3465d79f0 0x55f3469dc2a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:07.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.345+0000 7f092eff0740 1 --2- 172.21.15.100:0/3889592705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f346619070 0x55f3469d60a0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:07.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.345+0000 7f092eff0740 1 --2- 172.21.15.100:0/3889592705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3465f8390 0x55f3469ca680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:07.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.345+0000 7f092eff0740 1 --2- 172.21.15.100:0/3889592705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3465f01b0 0x55f3469ca140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:07.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.345+0000 7f092eff0740 1 -- 172.21.15.100:0/3889592705 >> 172.21.15.100:0/3889592705 conn(0x55f34660d460 msgr2=0x55f3469bac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:07.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.345+0000 7f0909ffb640 0 client.15654 ms_handle_reset on v2:172.21.15.167:6836/3828634468 2024-07-26T13:41:07.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.345+0000 7f092eff0740 1 -- 172.21.15.100:0/3889592705 shutdown_connections 2024-07-26T13:41:07.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.345+0000 7f092eff0740 1 -- 172.21.15.100:0/3889592705 wait complete. 2024-07-26T13:41:07.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.345+0000 7f092eff0740 20 client.15654 trim_cache size 0 max 16384 2024-07-26T13:41:07.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.371+0000 7f092eff0740 1 Processor -- start 2024-07-26T13:41:07.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.371+0000 7f092eff0740 1 -- start start 2024-07-26T13:41:07.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.371+0000 7f092eff0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3465f01b0 0x55f3469bb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:07.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.371+0000 7f092eff0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3465f8390 0x55f3469c4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:07.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.372+0000 7f092eff0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f346619070 0x55f3469bbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:07.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.372+0000 7f092eff0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f09241b3c10 con 0x55f3465f01b0 2024-07-26T13:41:07.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.372+0000 7f092eff0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f09241b14e0 con 0x55f346619070 2024-07-26T13:41:07.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.372+0000 7f092eff0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f09241af330 con 0x55f3465f8390 2024-07-26T13:41:07.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.372+0000 7f092259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3465f01b0 0x55f3469bb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:07.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.372+0000 7f092259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3465f01b0 0x55f3469bb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44580/0 (socket says 172.21.15.100:44580) 2024-07-26T13:41:07.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.372+0000 7f092259c640 1 -- 172.21.15.100:0/3773810141 learned_addr learned my addr 172.21.15.100:0/3773810141 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:07.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.372+0000 7f0921d9b640 1 --2- 172.21.15.100:0/3773810141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3465f8390 0x55f3469c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:07.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.372+0000 7f0922d9d640 1 --2- 172.21.15.100:0/3773810141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f346619070 0x55f3469bbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:07.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.372+0000 7f092259c640 1 -- 172.21.15.100:0/3773810141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3465f8390 msgr2=0x55f3469c4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:07.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.372+0000 7f092259c640 1 --2- 172.21.15.100:0/3773810141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3465f8390 0x55f3469c4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:07.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.372+0000 7f092259c640 1 -- 172.21.15.100:0/3773810141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f346619070 msgr2=0x55f3469bbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:07.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.372+0000 7f092259c640 1 --2- 172.21.15.100:0/3773810141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f346619070 0x55f3469bbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:07.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.372+0000 7f092259c640 1 -- 172.21.15.100:0/3773810141 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3469bce00 con 0x55f3465f01b0 2024-07-26T13:41:07.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.372+0000 7f0921d9b640 1 --2- 172.21.15.100:0/3773810141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3465f8390 0x55f3469c4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:41:07.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.372+0000 7f092259c640 1 --2- 172.21.15.100:0/3773810141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3465f01b0 0x55f3469bb350 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f09140c85d0 tx=0x7f09140ee040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa7f92e6110ee490 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:07.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.372+0000 7f090bfff640 1 -- 172.21.15.100:0/3773810141 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0914049040 con 0x55f3465f01b0 2024-07-26T13:41:07.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.373+0000 7f090bfff640 1 -- 172.21.15.100:0/3773810141 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09140cb360 con 0x55f3465f01b0 2024-07-26T13:41:07.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.373+0000 7f090bfff640 1 -- 172.21.15.100:0/3773810141 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091403bcc0 con 0x55f3465f01b0 2024-07-26T13:41:07.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.373+0000 7f092eff0740 1 -- 172.21.15.100:0/3773810141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3465f01b0 msgr2=0x55f3469bb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:07.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.373+0000 7f092eff0740 1 --2- 172.21.15.100:0/3773810141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3465f01b0 0x55f3469bb350 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f09140c85d0 tx=0x7f09140ee040 comp rx=0 tx=0).stop 2024-07-26T13:41:07.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.373+0000 7f092eff0740 1 -- 172.21.15.100:0/3773810141 shutdown_connections 2024-07-26T13:41:07.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.373+0000 7f092eff0740 1 --2- 172.21.15.100:0/3773810141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f346619070 0x55f3469bbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:07.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.373+0000 7f092eff0740 1 --2- 172.21.15.100:0/3773810141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3465f8390 0x55f3469c4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:07.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.373+0000 7f092eff0740 1 --2- 172.21.15.100:0/3773810141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3465f01b0 0x55f3469bb350 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:07.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.373+0000 7f092eff0740 1 -- 172.21.15.100:0/3773810141 >> 172.21.15.100:0/3773810141 conn(0x55f34660d460 msgr2=0x55f3469bac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:07.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.374+0000 7f092eff0740 1 -- 172.21.15.100:0/3773810141 shutdown_connections 2024-07-26T13:41:07.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.374+0000 7f092eff0740 1 -- 172.21.15.100:0/3773810141 wait complete. 2024-07-26T13:41:07.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.374+0000 7f092eff0740 1 -- start start 2024-07-26T13:41:07.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.374+0000 7f092eff0740 1 Processor -- start 2024-07-26T13:41:07.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.374+0000 7f092eff0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3465f01b0 0x55f3469ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:07.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.374+0000 7f092eff0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3465f8390 0x55f3469cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:07.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.374+0000 7f092eff0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f346619070 0x55f3469cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:07.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.374+0000 7f092eff0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f09241af9a0 con 0x55f3465f8390 2024-07-26T13:41:07.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.374+0000 7f092eff0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f09241b0e10 con 0x55f346619070 2024-07-26T13:41:07.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.374+0000 7f092eff0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f09241b0070 con 0x55f3465f01b0 2024-07-26T13:41:07.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.374+0000 7f092eff0740 10 client.0 _ll_register_callbacks cb 0x7f091c0096a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:41:07.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.374+0000 7f0921d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3465f8390 0x55f3469cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:07.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.374+0000 7f0921d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3465f8390 0x55f3469cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44584/0 (socket says 172.21.15.100:44584) 2024-07-26T13:41:07.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.374+0000 7f0921d9b640 1 -- 172.21.15.100:0/3062150429 learned_addr learned my addr 172.21.15.100:0/3062150429 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:07.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.375+0000 7f092259c640 1 --2- 172.21.15.100:0/3062150429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3465f01b0 0x55f3469ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:07.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.375+0000 7f0922d9d640 1 --2- 172.21.15.100:0/3062150429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f346619070 0x55f3469cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:07.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.375+0000 7f0921d9b640 1 -- 172.21.15.100:0/3062150429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3465f01b0 msgr2=0x55f3469ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:07.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.375+0000 7f0921d9b640 1 --2- 172.21.15.100:0/3062150429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3465f01b0 0x55f3469ccfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:07.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.375+0000 7f0921d9b640 1 -- 172.21.15.100:0/3062150429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f346619070 msgr2=0x55f3469cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:07.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.375+0000 7f0921d9b640 1 --2- 172.21.15.100:0/3062150429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f346619070 0x55f3469cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:07.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.375+0000 7f0921d9b640 1 -- 172.21.15.100:0/3062150429 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f346a42b60 con 0x55f3465f8390 2024-07-26T13:41:07.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.375+0000 7f0922d9d640 1 --2- 172.21.15.100:0/3062150429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f346619070 0x55f3469cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:41:07.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.375+0000 7f0921d9b640 1 --2- 172.21.15.100:0/3062150429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3465f8390 0x55f3469cd500 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0918009c20 tx=0x7f0918004800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2551a4f991060633 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:07.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.375+0000 7f0909ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:41:07.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.375+0000 7f0909ffb640 1 -- 172.21.15.100:0/3062150429 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091802a070 con 0x55f3465f8390 2024-07-26T13:41:07.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.375+0000 7f0909ffb640 1 -- 172.21.15.100:0/3062150429 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f09180037f0 con 0x55f3465f8390 2024-07-26T13:41:07.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.375+0000 7f092eff0740 1 -- 172.21.15.100:0/3062150429 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09140cb360 con 0x55f3465f8390 2024-07-26T13:41:07.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.375+0000 7f0909ffb640 1 -- 172.21.15.100:0/3062150429 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0918036050 con 0x55f3465f8390 2024-07-26T13:41:07.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.375+0000 7f092eff0740 10 client.15660 fetch_fsmap learned FSMap version 26 2024-07-26T13:41:07.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.375+0000 7f092eff0740 1 -- 172.21.15.100:0/3062150429 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f346a43dc0 con 0x55f3465f8390 2024-07-26T13:41:07.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.376+0000 7f0909ffb640 1 -- 172.21.15.100:0/3062150429 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f091801f040 con 0x55f3465f8390 2024-07-26T13:41:07.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.376+0000 7f0909ffb640 1 -- 172.21.15.100:0/3062150429 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f0918024040 con 0x55f3465f8390 2024-07-26T13:41:07.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.376+0000 7f092eff0740 10 client.15660 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:41:07.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.376+0000 7f092eff0740 10 client.15660 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-26T13:41:07.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.376+0000 7f092eff0740 20 client.15660 populate_metadata read hostname 'smithi100' 2024-07-26T13:41:07.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.376+0000 7f092eff0740 1 --2- 172.21.15.100:0/3062150429 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55f3465acda0 0x55f346a463d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:07.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.376+0000 7f092eff0740 4 client.15660 mds_command: new command op to 24595 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-26T13:41:07.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.376+0000 7f092eff0740 1 -- 172.21.15.100:0/3062150429 --> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f09140cb360 con 0x55f3465acda0 2024-07-26T13:41:07.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.376+0000 7f092259c640 1 --2- 172.21.15.100:0/3062150429 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55f3465acda0 0x55f346a463d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:07.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.377+0000 7f0909ffb640 10 client.15660 ms_handle_connect on v2:172.21.15.167:6836/3828634468 2024-07-26T13:41:07.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.377+0000 7f092259c640 1 --2- 172.21.15.100:0/3062150429 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55f3465acda0 0x55f346a463d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c59714536d15f5ce server_cookie=5be5464c6340dc79 in_seq=0 out_seq=0 2024-07-26T13:41:07.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.377+0000 7f0909ffb640 1 -- 172.21.15.100:0/3062150429 <== mds.2 v2:172.21.15.167:6836/3828634468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f09140cb360 con 0x55f3465acda0 2024-07-26T13:41:07.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.377+0000 7f0909ffb640 10 client.15660 handle_command_reply: tid=0 2024-07-26T13:41:07.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.377+0000 7f092eff0740 1 client.15660 shutdown 2024-07-26T13:41:07.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.377+0000 7f092eff0740 1 -- 172.21.15.100:0/3062150429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3465f8390 msgr2=0x55f3469cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:07.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.377+0000 7f092eff0740 1 --2- 172.21.15.100:0/3062150429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3465f8390 0x55f3469cd500 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0918009c20 tx=0x7f0918004800 comp rx=0 tx=0).stop 2024-07-26T13:41:07.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.377+0000 7f092eff0740 1 -- 172.21.15.100:0/3062150429 shutdown_connections 2024-07-26T13:41:07.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.377+0000 7f092eff0740 1 --2- 172.21.15.100:0/3062150429 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55f3465acda0 0x55f346a463d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:07.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.377+0000 7f092eff0740 1 --2- 172.21.15.100:0/3062150429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f346619070 0x55f3469cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:07.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.377+0000 7f092eff0740 1 --2- 172.21.15.100:0/3062150429 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3465f8390 0x55f3469cd500 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:07.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.377+0000 7f092eff0740 1 --2- 172.21.15.100:0/3062150429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3465f01b0 0x55f3469ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:07.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.377+0000 7f092eff0740 1 -- 172.21.15.100:0/3062150429 >> 172.21.15.100:0/3062150429 conn(0x55f34660d460 msgr2=0x55f3469bbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:07.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.377+0000 7f0909ffb640 0 client.15660 ms_handle_reset on v2:172.21.15.167:6836/3828634468 2024-07-26T13:41:07.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.377+0000 7f092eff0740 1 -- 172.21.15.100:0/3062150429 shutdown_connections 2024-07-26T13:41:07.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.377+0000 7f092eff0740 1 -- 172.21.15.100:0/3062150429 wait complete. 2024-07-26T13:41:07.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.377+0000 7f092eff0740 20 client.15660 trim_cache size 0 max 16384 2024-07-26T13:41:07.383 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-07-26T13:41:07.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.378+0000 7f0928a6c640 1 -- 172.21.15.100:0/405528601 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f090c0b3d40 msgr2=0x7f090c0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:07.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.378+0000 7f0928a6c640 1 --2- 172.21.15.100:0/405528601 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f090c0b3d40 0x7f090c0b6250 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f091c005fd0 tx=0x7f091c009590 comp rx=0 tx=0).stop 2024-07-26T13:41:07.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.378+0000 7f0928a6c640 1 -- 172.21.15.100:0/405528601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09241057d0 msgr2=0x7f092419c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:07.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.378+0000 7f0928a6c640 1 --2- 172.21.15.100:0/405528601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09241057d0 0x7f092419c120 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0914005940 tx=0x7f0914004270 comp rx=0 tx=0).stop 2024-07-26T13:41:07.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.378+0000 7f0928a6c640 1 -- 172.21.15.100:0/405528601 shutdown_connections 2024-07-26T13:41:07.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.378+0000 7f0928a6c640 1 --2- 172.21.15.100:0/405528601 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f090c0b3d40 0x7f090c0b6250 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:07.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.378+0000 7f0928a6c640 1 --2- 172.21.15.100:0/405528601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f092410ba60 0x7f09241a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:07.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.378+0000 7f0928a6c640 1 --2- 172.21.15.100:0/405528601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f092410b110 0x7f092419c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:07.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.378+0000 7f0928a6c640 1 --2- 172.21.15.100:0/405528601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09241057d0 0x7f092419c120 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:07.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.378+0000 7f0928a6c640 1 -- 172.21.15.100:0/405528601 >> 172.21.15.100:0/405528601 conn(0x7f0924101200 msgr2=0x7f092410a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:07.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.378+0000 7f0928a6c640 1 -- 172.21.15.100:0/405528601 shutdown_connections 2024-07-26T13:41:07.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:07.378+0000 7f0928a6c640 1 -- 172.21.15.100:0/405528601 wait complete. 2024-07-26T13:41:08.047 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-26T13:41:08.328 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:41:09.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:09 smithi100 ceph-mon[32599]: pgmap v395: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-07-26T13:41:09.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:09 smithi167 ceph-mon[37140]: pgmap v395: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-07-26T13:41:09.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:09 smithi138 ceph-mon[36954]: pgmap v395: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-07-26T13:41:10.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.035+0000 7f454a79f640 1 -- 172.21.15.100:0/3839352589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4544108530 msgr2=0x7f454410ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:10.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.035+0000 7f454a79f640 1 --2- 172.21.15.100:0/3839352589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4544108530 0x7f454410ed60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f453c009ca0 tx=0x7f453c02a8c0 comp rx=0 tx=0).stop 2024-07-26T13:41:10.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.035+0000 7f454a79f640 1 -- 172.21.15.100:0/3839352589 shutdown_connections 2024-07-26T13:41:10.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.035+0000 7f454a79f640 1 --2- 172.21.15.100:0/3839352589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4544108530 0x7f454410ed60 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:10.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.035+0000 7f454a79f640 1 --2- 172.21.15.100:0/3839352589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4544107be0 0x7f4544107ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:10.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.035+0000 7f454a79f640 1 --2- 172.21.15.100:0/3839352589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45441022a0 0x7f45441026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:10.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.035+0000 7f454a79f640 1 -- 172.21.15.100:0/3839352589 >> 172.21.15.100:0/3839352589 conn(0x7f45440fdcf0 msgr2=0x7f4544100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:10.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.035+0000 7f454a79f640 1 -- 172.21.15.100:0/3839352589 shutdown_connections 2024-07-26T13:41:10.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.035+0000 7f454a79f640 1 -- 172.21.15.100:0/3839352589 wait complete. 2024-07-26T13:41:10.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.036+0000 7f454a79f640 1 Processor -- start 2024-07-26T13:41:10.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.036+0000 7f454a79f640 1 -- start start 2024-07-26T13:41:10.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.037+0000 7f454a79f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45441022a0 0x7f4544199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:10.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.037+0000 7f454a79f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4544107be0 0x7f454419a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:10.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.037+0000 7f454a79f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4544108530 0x7f454419f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:10.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.037+0000 7f454a79f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4544113310 con 0x7f4544107be0 2024-07-26T13:41:10.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.037+0000 7f454a79f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4544113190 con 0x7f45441022a0 2024-07-26T13:41:10.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.037+0000 7f454a79f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4544113490 con 0x7f4544108530 2024-07-26T13:41:10.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.037+0000 7f4548f9c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4544107be0 0x7f454419a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:10.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.037+0000 7f4548f9c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4544107be0 0x7f454419a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44596/0 (socket says 172.21.15.100:44596) 2024-07-26T13:41:10.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.037+0000 7f4548f9c640 1 -- 172.21.15.100:0/1642532983 learned_addr learned my addr 172.21.15.100:0/1642532983 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:10.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.037+0000 7f454979d640 1 --2- 172.21.15.100:0/1642532983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45441022a0 0x7f4544199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:10.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.038+0000 7f4548f9c640 1 -- 172.21.15.100:0/1642532983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4544108530 msgr2=0x7f454419f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:10.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.038+0000 7f4548f9c640 1 --2- 172.21.15.100:0/1642532983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4544108530 0x7f454419f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:10.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.038+0000 7f4548f9c640 1 -- 172.21.15.100:0/1642532983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45441022a0 msgr2=0x7f4544199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:10.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.038+0000 7f4548f9c640 1 --2- 172.21.15.100:0/1642532983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45441022a0 0x7f4544199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:10.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.038+0000 7f4548f9c640 1 -- 172.21.15.100:0/1642532983 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f454419fbf0 con 0x7f4544107be0 2024-07-26T13:41:10.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.038+0000 7f454979d640 1 --2- 172.21.15.100:0/1642532983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45441022a0 0x7f4544199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:41:10.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.038+0000 7f4548f9c640 1 --2- 172.21.15.100:0/1642532983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4544107be0 0x7f454419a220 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f453400e9f0 tx=0x7f453400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:10.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.038+0000 7f453a7fc640 1 -- 172.21.15.100:0/1642532983 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453400ccc0 con 0x7f4544107be0 2024-07-26T13:41:10.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.038+0000 7f453a7fc640 1 -- 172.21.15.100:0/1642532983 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45340108e0 con 0x7f4544107be0 2024-07-26T13:41:10.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.038+0000 7f454a79f640 1 -- 172.21.15.100:0/1642532983 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f454419fee0 con 0x7f4544107be0 2024-07-26T13:41:10.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.040+0000 7f453a7fc640 1 -- 172.21.15.100:0/1642532983 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45340188e0 con 0x7f4544107be0 2024-07-26T13:41:10.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.040+0000 7f454a79f640 1 -- 172.21.15.100:0/1642532983 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45441a6640 con 0x7f4544107be0 2024-07-26T13:41:10.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.042+0000 7f453a7fc640 1 -- 172.21.15.100:0/1642532983 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f4534010430 con 0x7f4544107be0 2024-07-26T13:41:10.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.043+0000 7f453a7fc640 1 --2- 172.21.15.100:0/1642532983 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f45300b80b0 0x7f45300ba5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:10.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.043+0000 7f4551f85740 1 Processor -- start 2024-07-26T13:41:10.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.043+0000 7f4551f85740 1 -- start start 2024-07-26T13:41:10.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.043+0000 7f453a7fc640 1 -- 172.21.15.100:0/1642532983 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f45340e40e0 con 0x7f4544107be0 2024-07-26T13:41:10.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.044+0000 7f454979d640 1 --2- 172.21.15.100:0/1642532983 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f45300b80b0 0x7f45300ba5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:10.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.044+0000 7f4551f85740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559623912070 0x559623cb76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:10.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.044+0000 7f4551f85740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596238e91b0 0x559623cb7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:10.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.044+0000 7f4551f85740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596238f1390 0x559623cb8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:10.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.044+0000 7f4551f85740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x559623c2dbb0 con 0x559623912070 2024-07-26T13:41:10.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.044+0000 7f4551f85740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559623c1d0b0 con 0x5596238f1390 2024-07-26T13:41:10.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.044+0000 7f4551f85740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559623c210f0 con 0x5596238e91b0 2024-07-26T13:41:10.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.044+0000 7f4548f9c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596238e91b0 0x559623cb7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:10.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.044+0000 7f4548f9c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596238e91b0 0x559623cb7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:45084/0 (socket says 172.21.15.100:45084) 2024-07-26T13:41:10.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.044+0000 7f4548f9c640 1 -- 172.21.15.100:0/2514851761 learned_addr learned my addr 172.21.15.100:0/2514851761 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:10.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.044+0000 7f454979d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559623912070 0x559623cb76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:10.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.044+0000 7f4549f9e640 1 --2- 172.21.15.100:0/2514851761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596238f1390 0x559623cb8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:10.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.044+0000 7f454979d640 1 --2- 172.21.15.100:0/1642532983 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f45300b80b0 0x7f45300ba5c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f454000bfd0 tx=0x7f454000b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:10.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.044+0000 7f4548f9c640 1 -- 172.21.15.100:0/2514851761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596238f1390 msgr2=0x559623cb8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:10.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.044+0000 7f4548f9c640 1 --2- 172.21.15.100:0/2514851761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596238f1390 0x559623cb8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:10.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.045+0000 7f4548f9c640 1 -- 172.21.15.100:0/2514851761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559623912070 msgr2=0x559623cb76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:10.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.045+0000 7f4548f9c640 1 --2- 172.21.15.100:0/2514851761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559623912070 0x559623cb76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:10.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.045+0000 7f4548f9c640 1 -- 172.21.15.100:0/2514851761 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559623bc3b90 con 0x5596238e91b0 2024-07-26T13:41:10.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.045+0000 7f454979d640 1 --2- 172.21.15.100:0/2514851761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559623912070 0x559623cb76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:41:10.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.045+0000 7f4548f9c640 1 --2- 172.21.15.100:0/2514851761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596238e91b0 0x559623cb7c30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f45340aaca0 tx=0x7f45340aa960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=988a7da6f6572f49 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:10.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.045+0000 7f452f7fe640 1 -- 172.21.15.100:0/2514851761 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45340af310 con 0x5596238e91b0 2024-07-26T13:41:10.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.045+0000 7f452f7fe640 1 -- 172.21.15.100:0/2514851761 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45340b3dc0 con 0x5596238e91b0 2024-07-26T13:41:10.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.045+0000 7f452f7fe640 1 -- 172.21.15.100:0/2514851761 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45340a99e0 con 0x5596238e91b0 2024-07-26T13:41:10.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.046+0000 7f4551f85740 1 -- 172.21.15.100:0/2514851761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596238e91b0 msgr2=0x559623cb7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:10.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.046+0000 7f4551f85740 1 --2- 172.21.15.100:0/2514851761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596238e91b0 0x559623cb7c30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f45340aaca0 tx=0x7f45340aa960 comp rx=0 tx=0).stop 2024-07-26T13:41:10.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.046+0000 7f4551f85740 1 -- 172.21.15.100:0/2514851761 shutdown_connections 2024-07-26T13:41:10.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.046+0000 7f4551f85740 1 --2- 172.21.15.100:0/2514851761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596238f1390 0x559623cb8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:10.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.046+0000 7f4551f85740 1 --2- 172.21.15.100:0/2514851761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596238e91b0 0x559623cb7c30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:10.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.046+0000 7f4551f85740 1 --2- 172.21.15.100:0/2514851761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559623912070 0x559623cb76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:10.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.046+0000 7f4551f85740 1 -- 172.21.15.100:0/2514851761 >> 172.21.15.100:0/2514851761 conn(0x559623906460 msgr2=0x559623cb55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:10.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.047+0000 7f4551f85740 1 -- 172.21.15.100:0/2514851761 shutdown_connections 2024-07-26T13:41:10.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.047+0000 7f4551f85740 1 -- 172.21.15.100:0/2514851761 wait complete. 2024-07-26T13:41:10.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.047+0000 7f4551f85740 1 -- start start 2024-07-26T13:41:10.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.047+0000 7f4551f85740 1 Processor -- start 2024-07-26T13:41:10.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.048+0000 7f4551f85740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596238e91b0 0x559623ccbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:10.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.048+0000 7f454979d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596238e91b0 0x559623ccbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:10.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.048+0000 7f454979d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596238e91b0 0x559623ccbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44608/0 (socket says 172.21.15.100:44608) 2024-07-26T13:41:10.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.048+0000 7f454979d640 1 -- 172.21.15.100:0/3360118673 learned_addr learned my addr 172.21.15.100:0/3360118673 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:10.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.048+0000 7f4551f85740 1 --2- 172.21.15.100:0/3360118673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596238f1390 0x559623ccc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:10.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.048+0000 7f4551f85740 1 --2- 172.21.15.100:0/3360118673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559623912070 0x559623cd78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:10.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.048+0000 7f4551f85740 1 -- 172.21.15.100:0/3360118673 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x559623a80700 con 0x5596238e91b0 2024-07-26T13:41:10.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.048+0000 7f4551f85740 1 -- 172.21.15.100:0/3360118673 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559623bbf9e0 con 0x5596238f1390 2024-07-26T13:41:10.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.048+0000 7f4551f85740 1 -- 172.21.15.100:0/3360118673 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559623cce5a0 con 0x559623912070 2024-07-26T13:41:10.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.048+0000 7f4549f9e640 1 --2- 172.21.15.100:0/3360118673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559623912070 0x559623cd78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:10.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.049+0000 7f4548f9c640 1 --2- 172.21.15.100:0/3360118673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596238f1390 0x559623ccc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:10.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.049+0000 7f454979d640 1 -- 172.21.15.100:0/3360118673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559623912070 msgr2=0x559623cd78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:10.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.049+0000 7f454979d640 1 --2- 172.21.15.100:0/3360118673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559623912070 0x559623cd78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:10.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.049+0000 7f454979d640 1 -- 172.21.15.100:0/3360118673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596238f1390 msgr2=0x559623ccc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:10.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.049+0000 7f454979d640 1 --2- 172.21.15.100:0/3360118673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596238f1390 0x559623ccc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:10.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.049+0000 7f454979d640 1 -- 172.21.15.100:0/3360118673 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55962397d650 con 0x5596238e91b0 2024-07-26T13:41:10.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.049+0000 7f454979d640 1 --2- 172.21.15.100:0/3360118673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596238e91b0 0x559623ccbaf0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4540011e00 tx=0x7f4540004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de181890878c7f92 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:10.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.049+0000 7f452cff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:41:10.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.049+0000 7f452cff9640 1 -- 172.21.15.100:0/3360118673 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4540005020 con 0x5596238e91b0 2024-07-26T13:41:10.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.050+0000 7f4551f85740 10 client.0 _ll_register_callbacks cb 0x559623bda4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:41:10.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.050+0000 7f452cff9640 1 -- 172.21.15.100:0/3360118673 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f45400051c0 con 0x5596238e91b0 2024-07-26T13:41:10.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.050+0000 7f4551f85740 1 -- 172.21.15.100:0/3360118673 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45340b3dc0 con 0x5596238e91b0 2024-07-26T13:41:10.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.050+0000 7f452cff9640 1 -- 172.21.15.100:0/3360118673 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f45400053c0 con 0x5596238e91b0 2024-07-26T13:41:10.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.050+0000 7f4551f85740 10 client.15672 fetch_fsmap learned FSMap version 26 2024-07-26T13:41:10.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.050+0000 7f4551f85740 1 -- 172.21.15.100:0/3360118673 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({fsmap=26}) -- 0x559623cda430 con 0x5596238e91b0 2024-07-26T13:41:10.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.050+0000 7f452cff9640 1 -- 172.21.15.100:0/3360118673 <== mon.0 v2:172.21.15.100:3300/0 4 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f45400047a0 con 0x5596238e91b0 2024-07-26T13:41:10.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.051+0000 7f4551f85740 10 client.15672 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:41:10.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.051+0000 7f4551f85740 10 client.15672 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-26T13:41:10.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.051+0000 7f4551f85740 20 client.15672 populate_metadata read hostname 'smithi100' 2024-07-26T13:41:10.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.051+0000 7f4551f85740 1 --2- 172.21.15.100:0/3360118673 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x5596238d09f0 0x559623cdde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:10.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.051+0000 7f4551f85740 4 client.15672 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:41:10.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.051+0000 7f4551f85740 1 -- 172.21.15.100:0/3360118673 --> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f45340b3dc0 con 0x5596238d09f0 2024-07-26T13:41:10.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.051+0000 7f4548f9c640 1 --2- 172.21.15.100:0/3360118673 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x5596238d09f0 0x559623cdde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:10.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.051+0000 7f452cff9640 10 client.15672 ms_handle_connect on v2:172.21.15.138:6838/1580925471 2024-07-26T13:41:10.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.051+0000 7f4548f9c640 1 --2- 172.21.15.100:0/3360118673 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x5596238d09f0 0x559623cdde80 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=11f755c2c261e773 server_cookie=3d83093b559ae018 in_seq=0 out_seq=0 2024-07-26T13:41:10.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.052+0000 7f452cff9640 1 -- 172.21.15.100:0/3360118673 <== mds.1 v2:172.21.15.138:6838/1580925471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f45340b3dc0 con 0x5596238d09f0 2024-07-26T13:41:10.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.052+0000 7f452cff9640 10 client.15672 handle_command_reply: tid=0 2024-07-26T13:41:10.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.052+0000 7f4551f85740 1 client.15672 shutdown 2024-07-26T13:41:10.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.052+0000 7f4551f85740 1 -- 172.21.15.100:0/3360118673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596238e91b0 msgr2=0x559623ccbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:10.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.052+0000 7f4551f85740 1 --2- 172.21.15.100:0/3360118673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596238e91b0 0x559623ccbaf0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4540011e00 tx=0x7f4540004730 comp rx=0 tx=0).stop 2024-07-26T13:41:10.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.053+0000 7f4551f85740 1 -- 172.21.15.100:0/3360118673 shutdown_connections 2024-07-26T13:41:10.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.053+0000 7f4551f85740 1 --2- 172.21.15.100:0/3360118673 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x5596238d09f0 0x559623cdde80 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:10.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.053+0000 7f4551f85740 1 --2- 172.21.15.100:0/3360118673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559623912070 0x559623cd78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:10.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.053+0000 7f452cff9640 0 client.15672 ms_handle_reset on v2:172.21.15.138:6838/1580925471 2024-07-26T13:41:10.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.053+0000 7f4551f85740 1 --2- 172.21.15.100:0/3360118673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596238f1390 0x559623ccc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:10.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.053+0000 7f4551f85740 1 --2- 172.21.15.100:0/3360118673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596238e91b0 0x559623ccbaf0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:10.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.053+0000 7f4551f85740 1 -- 172.21.15.100:0/3360118673 >> 172.21.15.100:0/3360118673 conn(0x559623906460 msgr2=0x559623cb3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:10.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.053+0000 7f4551f85740 1 -- 172.21.15.100:0/3360118673 shutdown_connections 2024-07-26T13:41:10.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.053+0000 7f4551f85740 1 -- 172.21.15.100:0/3360118673 wait complete. 2024-07-26T13:41:10.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.053+0000 7f4551f85740 20 client.15672 trim_cache size 0 max 16384 2024-07-26T13:41:10.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.078+0000 7f4551f85740 1 Processor -- start 2024-07-26T13:41:10.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.078+0000 7f4551f85740 1 -- start start 2024-07-26T13:41:10.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.078+0000 7f4551f85740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596238d09f0 0x559623cd7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:10.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.078+0000 7f4551f85740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596238e91b0 0x559623cb5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:10.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.078+0000 7f4551f85740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596238f1390 0x559623ce0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:10.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.078+0000 7f4551f85740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f45441af350 con 0x5596238d09f0 2024-07-26T13:41:10.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.079+0000 7f4551f85740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f45441acc20 con 0x5596238f1390 2024-07-26T13:41:10.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.079+0000 7f4551f85740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f45441aaa70 con 0x5596238e91b0 2024-07-26T13:41:10.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.079+0000 7f454979d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596238d09f0 0x559623cd7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:10.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.079+0000 7f454979d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596238d09f0 0x559623cd7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44614/0 (socket says 172.21.15.100:44614) 2024-07-26T13:41:10.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.079+0000 7f454979d640 1 -- 172.21.15.100:0/2703459710 learned_addr learned my addr 172.21.15.100:0/2703459710 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:10.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.079+0000 7f4549f9e640 1 --2- 172.21.15.100:0/2703459710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596238f1390 0x559623ce0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:10.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.079+0000 7f4548f9c640 1 --2- 172.21.15.100:0/2703459710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596238e91b0 0x559623cb5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:10.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.079+0000 7f454979d640 1 -- 172.21.15.100:0/2703459710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596238e91b0 msgr2=0x559623cb5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:10.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.079+0000 7f454979d640 1 --2- 172.21.15.100:0/2703459710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596238e91b0 0x559623cb5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:10.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.079+0000 7f454979d640 1 -- 172.21.15.100:0/2703459710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596238f1390 msgr2=0x559623ce0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:10.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.079+0000 7f454979d640 1 --2- 172.21.15.100:0/2703459710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596238f1390 0x559623ce0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:10.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.079+0000 7f454979d640 1 -- 172.21.15.100:0/2703459710 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559623cb9f40 con 0x5596238d09f0 2024-07-26T13:41:10.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.079+0000 7f4549f9e640 1 --2- 172.21.15.100:0/2703459710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596238f1390 0x559623ce0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:41:10.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.079+0000 7f454979d640 1 --2- 172.21.15.100:0/2703459710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596238d09f0 0x559623cd7840 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4540011850 tx=0x7f4540021070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5e4ade4f9192a8d server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:10.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.080+0000 7f452effd640 1 -- 172.21.15.100:0/2703459710 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4540005560 con 0x5596238d09f0 2024-07-26T13:41:10.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.080+0000 7f452effd640 1 -- 172.21.15.100:0/2703459710 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45400047a0 con 0x5596238d09f0 2024-07-26T13:41:10.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.080+0000 7f452effd640 1 -- 172.21.15.100:0/2703459710 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454001ec20 con 0x5596238d09f0 2024-07-26T13:41:10.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.081+0000 7f4551f85740 1 -- 172.21.15.100:0/2703459710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596238d09f0 msgr2=0x559623cd7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:10.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.081+0000 7f4551f85740 1 --2- 172.21.15.100:0/2703459710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596238d09f0 0x559623cd7840 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4540011850 tx=0x7f4540021070 comp rx=0 tx=0).stop 2024-07-26T13:41:10.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.081+0000 7f4551f85740 1 -- 172.21.15.100:0/2703459710 shutdown_connections 2024-07-26T13:41:10.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.081+0000 7f4551f85740 1 --2- 172.21.15.100:0/2703459710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596238f1390 0x559623ce0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:10.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.081+0000 7f4551f85740 1 --2- 172.21.15.100:0/2703459710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596238e91b0 0x559623cb5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:10.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.081+0000 7f4551f85740 1 --2- 172.21.15.100:0/2703459710 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596238d09f0 0x559623cd7840 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:10.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.081+0000 7f4551f85740 1 -- 172.21.15.100:0/2703459710 >> 172.21.15.100:0/2703459710 conn(0x559623906460 msgr2=0x559623cb4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:10.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.081+0000 7f4551f85740 1 -- 172.21.15.100:0/2703459710 shutdown_connections 2024-07-26T13:41:10.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.081+0000 7f4551f85740 1 -- 172.21.15.100:0/2703459710 wait complete. 2024-07-26T13:41:10.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.081+0000 7f4551f85740 1 -- start start 2024-07-26T13:41:10.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.082+0000 7f4551f85740 1 Processor -- start 2024-07-26T13:41:10.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.082+0000 7f4551f85740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596238d09f0 0x559623ce1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:10.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.082+0000 7f4551f85740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596238e91b0 0x559623cde950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:10.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.082+0000 7f4551f85740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596238f1390 0x559623cdee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:10.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.082+0000 7f4548f9c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596238e91b0 0x559623cde950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:10.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.082+0000 7f4551f85740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f45441ab0e0 con 0x5596238e91b0 2024-07-26T13:41:10.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.082+0000 7f4551f85740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f45441ac550 con 0x5596238d09f0 2024-07-26T13:41:10.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.082+0000 7f4551f85740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f45441ab7b0 con 0x5596238f1390 2024-07-26T13:41:10.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.082+0000 7f4548f9c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596238e91b0 0x559623cde950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44630/0 (socket says 172.21.15.100:44630) 2024-07-26T13:41:10.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.082+0000 7f4548f9c640 1 -- 172.21.15.100:0/2038015944 learned_addr learned my addr 172.21.15.100:0/2038015944 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:10.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.082+0000 7f4551f85740 10 client.0 _ll_register_callbacks cb 0x7f4540011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:41:10.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.083+0000 7f454979d640 1 --2- 172.21.15.100:0/2038015944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596238d09f0 0x559623ce1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:10.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.083+0000 7f4549f9e640 1 --2- 172.21.15.100:0/2038015944 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596238f1390 0x559623cdee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:10.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.083+0000 7f4548f9c640 1 -- 172.21.15.100:0/2038015944 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596238f1390 msgr2=0x559623cdee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:10.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.083+0000 7f4548f9c640 1 --2- 172.21.15.100:0/2038015944 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596238f1390 0x559623cdee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:10.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.083+0000 7f4548f9c640 1 -- 172.21.15.100:0/2038015944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596238d09f0 msgr2=0x559623ce1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:10.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.083+0000 7f4548f9c640 1 --2- 172.21.15.100:0/2038015944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596238d09f0 0x559623ce1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:10.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.083+0000 7f4548f9c640 1 -- 172.21.15.100:0/2038015944 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559623cdf600 con 0x5596238e91b0 2024-07-26T13:41:10.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.083+0000 7f454979d640 1 --2- 172.21.15.100:0/2038015944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596238d09f0 0x559623ce1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:41:10.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.083+0000 7f4548f9c640 1 --2- 172.21.15.100:0/2038015944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596238e91b0 0x559623cde950 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f453400cfd0 tx=0x7f4534000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50c78cc976bb2834 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:10.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.083+0000 7f452cff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:41:10.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.083+0000 7f452cff9640 1 -- 172.21.15.100:0/2038015944 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45340aaae0 con 0x5596238e91b0 2024-07-26T13:41:10.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.083+0000 7f452cff9640 1 -- 172.21.15.100:0/2038015944 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f45340a9f70 con 0x5596238e91b0 2024-07-26T13:41:10.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.083+0000 7f4551f85740 1 -- 172.21.15.100:0/2038015944 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45400047a0 con 0x5596238e91b0 2024-07-26T13:41:10.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.084+0000 7f452cff9640 1 -- 172.21.15.100:0/2038015944 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f45340b3dc0 con 0x5596238e91b0 2024-07-26T13:41:10.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.084+0000 7f4551f85740 10 client.15678 fetch_fsmap learned FSMap version 26 2024-07-26T13:41:10.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.084+0000 7f4551f85740 1 -- 172.21.15.100:0/2038015944 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559623d46440 con 0x5596238e91b0 2024-07-26T13:41:10.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.084+0000 7f452cff9640 1 -- 172.21.15.100:0/2038015944 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45400047a0 con 0x5596238e91b0 2024-07-26T13:41:10.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.084+0000 7f452cff9640 1 -- 172.21.15.100:0/2038015944 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f45340a6810 con 0x5596238e91b0 2024-07-26T13:41:10.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.084+0000 7f4551f85740 10 client.15678 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:41:10.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.084+0000 7f4551f85740 10 client.15678 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-26T13:41:10.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.084+0000 7f4551f85740 20 client.15678 populate_metadata read hostname 'smithi100' 2024-07-26T13:41:10.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.084+0000 7f4551f85740 1 --2- 172.21.15.100:0/2038015944 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x559623912070 0x559623d48790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:10.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.084+0000 7f4551f85740 4 client.15678 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-26T13:41:10.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.084+0000 7f4551f85740 1 -- 172.21.15.100:0/2038015944 --> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559623c2fb00 con 0x559623912070 2024-07-26T13:41:10.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.084+0000 7f454979d640 1 --2- 172.21.15.100:0/2038015944 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x559623912070 0x559623d48790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:10.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.085+0000 7f452cff9640 10 client.15678 ms_handle_connect on v2:172.21.15.138:6838/1580925471 2024-07-26T13:41:10.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.085+0000 7f454979d640 1 --2- 172.21.15.100:0/2038015944 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x559623912070 0x559623d48790 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=329885e4ac9b58bf server_cookie=a70185988dd06014 in_seq=0 out_seq=0 2024-07-26T13:41:10.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.085+0000 7f452cff9640 1 -- 172.21.15.100:0/2038015944 <== mds.1 v2:172.21.15.138:6838/1580925471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559623c2fb00 con 0x559623912070 2024-07-26T13:41:10.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.085+0000 7f452cff9640 10 client.15678 handle_command_reply: tid=0 2024-07-26T13:41:10.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.085+0000 7f4551f85740 1 client.15678 shutdown 2024-07-26T13:41:10.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.086+0000 7f4551f85740 1 -- 172.21.15.100:0/2038015944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596238e91b0 msgr2=0x559623cde950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:10.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.086+0000 7f4551f85740 1 --2- 172.21.15.100:0/2038015944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596238e91b0 0x559623cde950 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f453400cfd0 tx=0x7f4534000bc0 comp rx=0 tx=0).stop 2024-07-26T13:41:10.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.086+0000 7f4551f85740 1 -- 172.21.15.100:0/2038015944 shutdown_connections 2024-07-26T13:41:10.090 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-07-26T13:41:10.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.086+0000 7f4551f85740 1 --2- 172.21.15.100:0/2038015944 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x559623912070 0x559623d48790 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:10.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.086+0000 7f4551f85740 1 --2- 172.21.15.100:0/2038015944 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5596238f1390 0x559623cdee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:10.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.086+0000 7f4551f85740 1 --2- 172.21.15.100:0/2038015944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5596238e91b0 0x559623cde950 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:10.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.086+0000 7f4551f85740 1 --2- 172.21.15.100:0/2038015944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5596238d09f0 0x559623ce1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:10.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.086+0000 7f4551f85740 1 -- 172.21.15.100:0/2038015944 >> 172.21.15.100:0/2038015944 conn(0x559623906460 msgr2=0x559623cb4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:10.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.086+0000 7f452cff9640 0 client.15678 ms_handle_reset on v2:172.21.15.138:6838/1580925471 2024-07-26T13:41:10.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.086+0000 7f4551f85740 1 -- 172.21.15.100:0/2038015944 shutdown_connections 2024-07-26T13:41:10.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.086+0000 7f4551f85740 1 -- 172.21.15.100:0/2038015944 wait complete. 2024-07-26T13:41:10.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.086+0000 7f4551f85740 20 client.15678 trim_cache size 0 max 16384 2024-07-26T13:41:10.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.087+0000 7f454a79f640 1 -- 172.21.15.100:0/1642532983 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f45300b80b0 msgr2=0x7f45300ba5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:10.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.087+0000 7f454a79f640 1 --2- 172.21.15.100:0/1642532983 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f45300b80b0 0x7f45300ba5c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f454000bfd0 tx=0x7f454000b390 comp rx=0 tx=0).stop 2024-07-26T13:41:10.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.087+0000 7f454a79f640 1 -- 172.21.15.100:0/1642532983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4544107be0 msgr2=0x7f454419a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:10.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.087+0000 7f454a79f640 1 --2- 172.21.15.100:0/1642532983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4544107be0 0x7f454419a220 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f453400e9f0 tx=0x7f453400eeb0 comp rx=0 tx=0).stop 2024-07-26T13:41:10.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.087+0000 7f454a79f640 1 -- 172.21.15.100:0/1642532983 shutdown_connections 2024-07-26T13:41:10.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.087+0000 7f454a79f640 1 --2- 172.21.15.100:0/1642532983 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f45300b80b0 0x7f45300ba5c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:10.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.087+0000 7f454a79f640 1 --2- 172.21.15.100:0/1642532983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4544108530 0x7f454419f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:10.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.087+0000 7f454a79f640 1 --2- 172.21.15.100:0/1642532983 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4544107be0 0x7f454419a220 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:10.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.087+0000 7f454a79f640 1 --2- 172.21.15.100:0/1642532983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45441022a0 0x7f4544199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:10.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.087+0000 7f454a79f640 1 -- 172.21.15.100:0/1642532983 >> 172.21.15.100:0/1642532983 conn(0x7f45440fdcf0 msgr2=0x7f4544100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:10.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.087+0000 7f454a79f640 1 -- 172.21.15.100:0/1642532983 shutdown_connections 2024-07-26T13:41:10.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:10.087+0000 7f454a79f640 1 -- 172.21.15.100:0/1642532983 wait complete. 2024-07-26T13:41:10.951 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-26T13:41:11.234 INFO:tasks.quiescer.fs.[cephfs]:Total duration (45.1) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-07-26T13:41:11.234 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.55 2024-07-26T13:41:11.234 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:41:11.305 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:41:11.577 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:11 smithi100 ceph-mon[32599]: pgmap v396: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-07-26T13:41:11.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:11 smithi167 ceph-mon[37140]: pgmap v396: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-07-26T13:41:11.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:11 smithi138 ceph-mon[36954]: pgmap v396: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-07-26T13:41:12.091 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:41:13.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.206+0000 7f187b4f0640 1 -- 172.21.15.100:0/4157910365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1874102bd0 msgr2=0x7f1874102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:13.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.206+0000 7f187b4f0640 1 --2- 172.21.15.100:0/4157910365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1874102bd0 0x7f1874102fe0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1870009900 tx=0x7f187002a410 comp rx=0 tx=0).stop 2024-07-26T13:41:13.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.206+0000 7f187b4f0640 1 -- 172.21.15.100:0/4157910365 shutdown_connections 2024-07-26T13:41:13.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.206+0000 7f187b4f0640 1 --2- 172.21.15.100:0/4157910365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18741054f0 0x7f187410ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:13.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.206+0000 7f187b4f0640 1 --2- 172.21.15.100:0/4157910365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1874104ba0 0x7f1874104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:13.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.206+0000 7f187b4f0640 1 --2- 172.21.15.100:0/4157910365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1874102bd0 0x7f1874102fe0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:13.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.206+0000 7f187b4f0640 1 -- 172.21.15.100:0/4157910365 >> 172.21.15.100:0/4157910365 conn(0x7f1874077690 msgr2=0x7f1874077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:13.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.206+0000 7f187b4f0640 1 -- 172.21.15.100:0/4157910365 shutdown_connections 2024-07-26T13:41:13.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.206+0000 7f187b4f0640 1 -- 172.21.15.100:0/4157910365 wait complete. 2024-07-26T13:41:13.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.207+0000 7f187b4f0640 1 Processor -- start 2024-07-26T13:41:13.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.207+0000 7f187b4f0640 1 -- start start 2024-07-26T13:41:13.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.207+0000 7f187b4f0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1874102bd0 0x7f1874199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:13.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.208+0000 7f187b4f0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1874104ba0 0x7f187419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:13.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.208+0000 7f187b4f0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18741054f0 0x7f187419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:13.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.208+0000 7f187b4f0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1874112400 con 0x7f1874104ba0 2024-07-26T13:41:13.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.208+0000 7f187b4f0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1874112280 con 0x7f1874102bd0 2024-07-26T13:41:13.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.208+0000 7f187b4f0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1874112580 con 0x7f18741054f0 2024-07-26T13:41:13.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.208+0000 7f1878a8b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1874104ba0 0x7f187419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:13.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.208+0000 7f1878a8b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1874104ba0 0x7f187419a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44652/0 (socket says 172.21.15.100:44652) 2024-07-26T13:41:13.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.208+0000 7f1878a8b640 1 -- 172.21.15.100:0/3730344982 learned_addr learned my addr 172.21.15.100:0/3730344982 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:13.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.208+0000 7f187928c640 1 --2- 172.21.15.100:0/3730344982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1874102bd0 0x7f1874199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:13.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.208+0000 7f1879a8d640 1 --2- 172.21.15.100:0/3730344982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18741054f0 0x7f187419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:13.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.208+0000 7f1878a8b640 1 -- 172.21.15.100:0/3730344982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18741054f0 msgr2=0x7f187419f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:13.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.208+0000 7f1878a8b640 1 --2- 172.21.15.100:0/3730344982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18741054f0 0x7f187419f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:13.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.208+0000 7f1878a8b640 1 -- 172.21.15.100:0/3730344982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1874102bd0 msgr2=0x7f1874199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:13.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.208+0000 7f1878a8b640 1 --2- 172.21.15.100:0/3730344982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1874102bd0 0x7f1874199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:13.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.208+0000 7f1878a8b640 1 -- 172.21.15.100:0/3730344982 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f187419fe50 con 0x7f1874104ba0 2024-07-26T13:41:13.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.208+0000 7f1878a8b640 1 --2- 172.21.15.100:0/3730344982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1874104ba0 0x7f187419a490 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f186400d740 tx=0x7f186400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:13.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.209+0000 7f186a7fc640 1 -- 172.21.15.100:0/3730344982 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864017070 con 0x7f1874104ba0 2024-07-26T13:41:13.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.209+0000 7f187b4f0640 1 -- 172.21.15.100:0/3730344982 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18741a0140 con 0x7f1874104ba0 2024-07-26T13:41:13.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.209+0000 7f187b4f0640 1 -- 172.21.15.100:0/3730344982 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18741a68a0 con 0x7f1874104ba0 2024-07-26T13:41:13.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.209+0000 7f186a7fc640 1 -- 172.21.15.100:0/3730344982 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f186400de50 con 0x7f1874104ba0 2024-07-26T13:41:13.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.209+0000 7f186a7fc640 1 -- 172.21.15.100:0/3730344982 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186400f620 con 0x7f1874104ba0 2024-07-26T13:41:13.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.212+0000 7f1881a74740 1 Processor -- start 2024-07-26T13:41:13.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.212+0000 7f1881a74740 1 -- start start 2024-07-26T13:41:13.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.213+0000 7f1881a74740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eafeaf9070 0x55eafee9e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:13.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.213+0000 7f1881a74740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eafead01b0 0x55eafee9eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:13.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.213+0000 7f1881a74740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eafead8390 0x55eafee9eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:13.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.213+0000 7f1881a74740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55eafee14bb0 con 0x55eafead01b0 2024-07-26T13:41:13.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.213+0000 7f1881a74740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55eafee040b0 con 0x55eafead8390 2024-07-26T13:41:13.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.213+0000 7f1881a74740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55eafee080f0 con 0x55eafeaf9070 2024-07-26T13:41:13.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.213+0000 7f186a7fc640 1 -- 172.21.15.100:0/3730344982 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f186401a070 con 0x7f1874104ba0 2024-07-26T13:41:13.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.213+0000 7f187928c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eafead01b0 0x55eafee9eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:13.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.213+0000 7f187928c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eafead01b0 0x55eafee9eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44660/0 (socket says 172.21.15.100:44660) 2024-07-26T13:41:13.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.213+0000 7f187928c640 1 -- 172.21.15.100:0/3873586156 learned_addr learned my addr 172.21.15.100:0/3873586156 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:13.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.213+0000 7f1878a8b640 1 --2- 172.21.15.100:0/3873586156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eafead8390 0x55eafee9eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:13.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.213+0000 7f1879a8d640 1 --2- 172.21.15.100:0/3873586156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eafeaf9070 0x55eafee9e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:13.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.214+0000 7f1878a8b640 1 -- 172.21.15.100:0/3873586156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eafeaf9070 msgr2=0x55eafee9e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:13.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.214+0000 7f1878a8b640 1 --2- 172.21.15.100:0/3873586156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eafeaf9070 0x55eafee9e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:13.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.214+0000 7f1878a8b640 1 -- 172.21.15.100:0/3873586156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eafead01b0 msgr2=0x55eafee9eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:13.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.214+0000 7f1878a8b640 1 --2- 172.21.15.100:0/3873586156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eafead01b0 0x55eafee9eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:13.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.214+0000 7f1878a8b640 1 -- 172.21.15.100:0/3873586156 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eafedaab90 con 0x55eafead8390 2024-07-26T13:41:13.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.214+0000 7f187928c640 1 --2- 172.21.15.100:0/3873586156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eafead01b0 0x55eafee9eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:41:13.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.214+0000 7f1878a8b640 1 --2- 172.21.15.100:0/3873586156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eafead8390 0x55eafee9eff0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f18640e10f0 tx=0x7f18640eb2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb623fe5d11cac6b server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:13.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.214+0000 7f185f7fe640 1 -- 172.21.15.100:0/3873586156 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18640f2070 con 0x55eafead8390 2024-07-26T13:41:13.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.214+0000 7f185f7fe640 1 -- 172.21.15.100:0/3873586156 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18640ee8d0 con 0x55eafead8390 2024-07-26T13:41:13.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.214+0000 7f186a7fc640 1 --2- 172.21.15.100:0/3730344982 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f18600b3e40 0x7f18600b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:13.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.214+0000 7f185f7fe640 1 -- 172.21.15.100:0/3873586156 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18640ed7e0 con 0x55eafead8390 2024-07-26T13:41:13.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.214+0000 7f1879a8d640 1 --2- 172.21.15.100:0/3730344982 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f18600b3e40 0x7f18600b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:13.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.215+0000 7f186a7fc640 1 -- 172.21.15.100:0/3730344982 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f18640e39e0 con 0x7f1874104ba0 2024-07-26T13:41:13.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.215+0000 7f1881a74740 1 -- 172.21.15.100:0/3873586156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eafead8390 msgr2=0x55eafee9eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:13.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.215+0000 7f1879a8d640 1 --2- 172.21.15.100:0/3730344982 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f18600b3e40 0x7f18600b6350 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f186c005fd0 tx=0x7f186c009740 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:13.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.215+0000 7f1881a74740 1 --2- 172.21.15.100:0/3873586156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eafead8390 0x55eafee9eff0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f18640e10f0 tx=0x7f18640eb2a0 comp rx=0 tx=0).stop 2024-07-26T13:41:13.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.216+0000 7f1881a74740 1 -- 172.21.15.100:0/3873586156 shutdown_connections 2024-07-26T13:41:13.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.216+0000 7f1881a74740 1 --2- 172.21.15.100:0/3873586156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eafead8390 0x55eafee9eff0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:13.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.216+0000 7f1881a74740 1 --2- 172.21.15.100:0/3873586156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eafead01b0 0x55eafee9eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:13.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.216+0000 7f1881a74740 1 --2- 172.21.15.100:0/3873586156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eafeaf9070 0x55eafee9e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:13.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.216+0000 7f1881a74740 1 -- 172.21.15.100:0/3873586156 >> 172.21.15.100:0/3873586156 conn(0x55eafeaed460 msgr2=0x55eafee9c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:13.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.216+0000 7f1881a74740 1 -- 172.21.15.100:0/3873586156 shutdown_connections 2024-07-26T13:41:13.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.216+0000 7f1881a74740 1 -- 172.21.15.100:0/3873586156 wait complete. 2024-07-26T13:41:13.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.217+0000 7f1881a74740 1 -- start start 2024-07-26T13:41:13.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.217+0000 7f1881a74740 1 Processor -- start 2024-07-26T13:41:13.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.217+0000 7f1881a74740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eafead01b0 0x55eafeeae490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:13.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.218+0000 7f1881a74740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eafead8390 0x55eafeeae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:13.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.218+0000 7f1881a74740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eafeaf9070 0x55eafeeba380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:13.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.218+0000 7f1881a74740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55eafec67700 con 0x55eafead8390 2024-07-26T13:41:13.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.218+0000 7f1881a74740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55eafeda69e0 con 0x55eafead01b0 2024-07-26T13:41:13.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.218+0000 7f1881a74740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55eafeeb0ff0 con 0x55eafeaf9070 2024-07-26T13:41:13.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.218+0000 7f1881a74740 10 client.0 _ll_register_callbacks cb 0x55eafedc14f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:41:13.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.218+0000 7f1878a8b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eafead8390 0x55eafeeae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:13.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.218+0000 7f1878a8b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eafead8390 0x55eafeeae9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44664/0 (socket says 172.21.15.100:44664) 2024-07-26T13:41:13.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.218+0000 7f1878a8b640 1 -- 172.21.15.100:0/258815611 learned_addr learned my addr 172.21.15.100:0/258815611 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:13.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.218+0000 7f187928c640 1 --2- 172.21.15.100:0/258815611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eafead01b0 0x55eafeeae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:13.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.218+0000 7f1879a8d640 1 --2- 172.21.15.100:0/258815611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eafeaf9070 0x55eafeeba380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:13.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.218+0000 7f1878a8b640 1 -- 172.21.15.100:0/258815611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eafeaf9070 msgr2=0x55eafeeba380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:13.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.218+0000 7f1878a8b640 1 --2- 172.21.15.100:0/258815611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eafeaf9070 0x55eafeeba380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:13.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.218+0000 7f1878a8b640 1 -- 172.21.15.100:0/258815611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eafead01b0 msgr2=0x55eafeeae490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:13.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.218+0000 7f1878a8b640 1 --2- 172.21.15.100:0/258815611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eafead01b0 0x55eafeeae490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:13.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.218+0000 7f1878a8b640 1 -- 172.21.15.100:0/258815611 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eafeb64650 con 0x55eafead8390 2024-07-26T13:41:13.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.218+0000 7f1878a8b640 1 --2- 172.21.15.100:0/258815611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eafead8390 0x55eafeeae9d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f18640aa540 tx=0x7f1864005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f63c267a75179d9f server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:13.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.218+0000 7f185cff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:41:13.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.219+0000 7f185cff9640 1 -- 172.21.15.100:0/258815611 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864021020 con 0x55eafead8390 2024-07-26T13:41:13.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.219+0000 7f185cff9640 1 -- 172.21.15.100:0/258815611 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f18640a6840 con 0x55eafead8390 2024-07-26T13:41:13.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.219+0000 7f1881a74740 1 -- 172.21.15.100:0/258815611 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18640ee8d0 con 0x55eafead8390 2024-07-26T13:41:13.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.219+0000 7f185cff9640 1 -- 172.21.15.100:0/258815611 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f18640f2070 con 0x55eafead8390 2024-07-26T13:41:13.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.219+0000 7f1881a74740 10 client.15690 fetch_fsmap learned FSMap version 26 2024-07-26T13:41:13.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.219+0000 7f1881a74740 1 -- 172.21.15.100:0/258815611 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55eafeebcfd0 con 0x55eafead8390 2024-07-26T13:41:13.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.219+0000 7f185cff9640 1 -- 172.21.15.100:0/258815611 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18640ee8d0 con 0x55eafead8390 2024-07-26T13:41:13.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.220+0000 7f185cff9640 1 -- 172.21.15.100:0/258815611 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f18640e1370 con 0x55eafead8390 2024-07-26T13:41:13.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.220+0000 7f1881a74740 10 client.15690 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:41:13.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.220+0000 7f1881a74740 10 client.15690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:41:13.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.220+0000 7f1881a74740 20 client.15690 populate_metadata read hostname 'smithi100' 2024-07-26T13:41:13.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.220+0000 7f1881a74740 1 --2- 172.21.15.100:0/258815611 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55eafeab79f0 0x55eafeec0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:13.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.220+0000 7f1881a74740 4 client.15690 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:41:13.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.220+0000 7f1881a74740 1 -- 172.21.15.100:0/258815611 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eafee16b00 con 0x55eafeab79f0 2024-07-26T13:41:13.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.220+0000 7f187928c640 1 --2- 172.21.15.100:0/258815611 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55eafeab79f0 0x55eafeec0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:13.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.221+0000 7f187928c640 1 --2- 172.21.15.100:0/258815611 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55eafeab79f0 0x55eafeec0330 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=723e1fe550496645 server_cookie=e9698325f6a01c8b in_seq=0 out_seq=0 2024-07-26T13:41:13.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.221+0000 7f185cff9640 10 client.15690 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:13.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.222+0000 7f185cff9640 1 -- 172.21.15.100:0/258815611 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eafee16b00 con 0x55eafeab79f0 2024-07-26T13:41:13.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.222+0000 7f185cff9640 10 client.15690 handle_command_reply: tid=0 2024-07-26T13:41:13.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.222+0000 7f1881a74740 1 client.15690 shutdown 2024-07-26T13:41:13.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.222+0000 7f1881a74740 1 -- 172.21.15.100:0/258815611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eafead8390 msgr2=0x55eafeeae9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:13.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.222+0000 7f1881a74740 1 --2- 172.21.15.100:0/258815611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eafead8390 0x55eafeeae9d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f18640aa540 tx=0x7f1864005cc0 comp rx=0 tx=0).stop 2024-07-26T13:41:13.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.222+0000 7f1881a74740 1 -- 172.21.15.100:0/258815611 shutdown_connections 2024-07-26T13:41:13.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.222+0000 7f1881a74740 1 --2- 172.21.15.100:0/258815611 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55eafeab79f0 0x55eafeec0330 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:13.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.222+0000 7f1881a74740 1 --2- 172.21.15.100:0/258815611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eafeaf9070 0x55eafeeba380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:13.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.222+0000 7f185cff9640 0 client.15690 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:13.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.222+0000 7f1881a74740 1 --2- 172.21.15.100:0/258815611 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eafead8390 0x55eafeeae9d0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:13.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.222+0000 7f1881a74740 1 --2- 172.21.15.100:0/258815611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eafead01b0 0x55eafeeae490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:13.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.222+0000 7f1881a74740 1 -- 172.21.15.100:0/258815611 >> 172.21.15.100:0/258815611 conn(0x55eafeaed460 msgr2=0x55eafee9ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:13.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.222+0000 7f1881a74740 1 -- 172.21.15.100:0/258815611 shutdown_connections 2024-07-26T13:41:13.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.222+0000 7f1881a74740 1 -- 172.21.15.100:0/258815611 wait complete. 2024-07-26T13:41:13.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.222+0000 7f1881a74740 20 client.15690 trim_cache size 0 max 16384 2024-07-26T13:41:13.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.245+0000 7f1881a74740 1 Processor -- start 2024-07-26T13:41:13.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.245+0000 7f1881a74740 1 -- start start 2024-07-26T13:41:13.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.245+0000 7f1881a74740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eafeab79f0 0x55eafeeba2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:13.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.245+0000 7f1881a74740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eafead01b0 0x55eafeeba830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:13.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.245+0000 7f1881a74740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eafead8390 0x55eafeec3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:13.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.245+0000 7f1881a74740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f18741af7c0 con 0x55eafead01b0 2024-07-26T13:41:13.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.245+0000 7f1881a74740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f18741ad090 con 0x55eafeab79f0 2024-07-26T13:41:13.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.245+0000 7f1881a74740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f18741aaee0 con 0x55eafead8390 2024-07-26T13:41:13.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.245+0000 7f1878a8b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eafead01b0 0x55eafeeba830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:13.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.245+0000 7f1878a8b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eafead01b0 0x55eafeeba830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44678/0 (socket says 172.21.15.100:44678) 2024-07-26T13:41:13.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.245+0000 7f1878a8b640 1 -- 172.21.15.100:0/3446984972 learned_addr learned my addr 172.21.15.100:0/3446984972 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:13.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.246+0000 7f187928c640 1 --2- 172.21.15.100:0/3446984972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eafeab79f0 0x55eafeeba2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:13.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.246+0000 7f1879a8d640 1 --2- 172.21.15.100:0/3446984972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eafead8390 0x55eafeec3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:13.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.246+0000 7f1878a8b640 1 -- 172.21.15.100:0/3446984972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eafead8390 msgr2=0x55eafeec3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:13.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.246+0000 7f1878a8b640 1 --2- 172.21.15.100:0/3446984972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eafead8390 0x55eafeec3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:13.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.246+0000 7f1878a8b640 1 -- 172.21.15.100:0/3446984972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eafeab79f0 msgr2=0x55eafeeba2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:13.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.246+0000 7f1878a8b640 1 --2- 172.21.15.100:0/3446984972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eafeab79f0 0x55eafeeba2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:13.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.246+0000 7f1878a8b640 1 -- 172.21.15.100:0/3446984972 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eafee9f3b0 con 0x55eafead01b0 2024-07-26T13:41:13.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.246+0000 7f187928c640 1 --2- 172.21.15.100:0/3446984972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eafeab79f0 0x55eafeeba2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:41:13.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.246+0000 7f1878a8b640 1 --2- 172.21.15.100:0/3446984972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eafead01b0 0x55eafeeba830 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f18640e9210 tx=0x7f18640ed380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b1b7aecc60be30 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:13.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.246+0000 7f185effd640 1 -- 172.21.15.100:0/3446984972 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864021030 con 0x55eafead01b0 2024-07-26T13:41:13.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.247+0000 7f185effd640 1 -- 172.21.15.100:0/3446984972 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18640aacb0 con 0x55eafead01b0 2024-07-26T13:41:13.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.247+0000 7f185effd640 1 -- 172.21.15.100:0/3446984972 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18640e9300 con 0x55eafead01b0 2024-07-26T13:41:13.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.248+0000 7f1881a74740 1 -- 172.21.15.100:0/3446984972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eafead01b0 msgr2=0x55eafeeba830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:13.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.248+0000 7f1881a74740 1 --2- 172.21.15.100:0/3446984972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eafead01b0 0x55eafeeba830 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f18640e9210 tx=0x7f18640ed380 comp rx=0 tx=0).stop 2024-07-26T13:41:13.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.248+0000 7f1881a74740 1 -- 172.21.15.100:0/3446984972 shutdown_connections 2024-07-26T13:41:13.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.248+0000 7f1881a74740 1 --2- 172.21.15.100:0/3446984972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eafead8390 0x55eafeec3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:13.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.248+0000 7f1881a74740 1 --2- 172.21.15.100:0/3446984972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eafead01b0 0x55eafeeba830 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:13.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.248+0000 7f1881a74740 1 --2- 172.21.15.100:0/3446984972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eafeab79f0 0x55eafeeba2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:13.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.248+0000 7f1881a74740 1 -- 172.21.15.100:0/3446984972 >> 172.21.15.100:0/3446984972 conn(0x55eafeaed460 msgr2=0x55eafeeacb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:13.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.248+0000 7f1881a74740 1 -- 172.21.15.100:0/3446984972 shutdown_connections 2024-07-26T13:41:13.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.248+0000 7f1881a74740 1 -- 172.21.15.100:0/3446984972 wait complete. 2024-07-26T13:41:13.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.249+0000 7f1881a74740 1 -- start start 2024-07-26T13:41:13.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.249+0000 7f1881a74740 1 Processor -- start 2024-07-26T13:41:13.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.249+0000 7f1881a74740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eafeab79f0 0x55eafeec1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:13.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.249+0000 7f1881a74740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eafead01b0 0x55eafeec1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:13.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.250+0000 7f1881a74740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eafead8390 0x55eafeec1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:13.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.250+0000 7f1878a8b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eafead01b0 0x55eafeec1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:13.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.250+0000 7f1878a8b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eafead01b0 0x55eafeec1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44686/0 (socket says 172.21.15.100:44686) 2024-07-26T13:41:13.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.250+0000 7f187928c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eafeab79f0 0x55eafeec1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:13.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.250+0000 7f1881a74740 1 -- 172.21.15.100:0/2748376283 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f18741ab550 con 0x55eafead01b0 2024-07-26T13:41:13.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.250+0000 7f1881a74740 1 -- 172.21.15.100:0/2748376283 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f18741ac9c0 con 0x55eafead8390 2024-07-26T13:41:13.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.250+0000 7f1881a74740 1 -- 172.21.15.100:0/2748376283 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f18741abc20 con 0x55eafeab79f0 2024-07-26T13:41:13.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.250+0000 7f1878a8b640 1 -- 172.21.15.100:0/2748376283 learned_addr learned my addr 172.21.15.100:0/2748376283 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:13.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.250+0000 7f1881a74740 10 client.0 _ll_register_callbacks cb 0x7f1870009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:41:13.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.250+0000 7f1879a8d640 1 --2- 172.21.15.100:0/2748376283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eafead8390 0x55eafeec1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:13.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.250+0000 7f1878a8b640 1 -- 172.21.15.100:0/2748376283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eafeab79f0 msgr2=0x55eafeec1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:13.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.250+0000 7f1878a8b640 1 --2- 172.21.15.100:0/2748376283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eafeab79f0 0x55eafeec1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:13.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.250+0000 7f1878a8b640 1 -- 172.21.15.100:0/2748376283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eafead8390 msgr2=0x55eafeec1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:13.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.250+0000 7f1878a8b640 1 --2- 172.21.15.100:0/2748376283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eafead8390 0x55eafeec1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:13.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.250+0000 7f1878a8b640 1 -- 172.21.15.100:0/2748376283 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eafeeaed40 con 0x55eafead01b0 2024-07-26T13:41:13.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.250+0000 7f1879a8d640 1 --2- 172.21.15.100:0/2748376283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eafead8390 0x55eafeec1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:41:13.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.251+0000 7f1878a8b640 1 --2- 172.21.15.100:0/2748376283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eafead01b0 0x55eafeec1990 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1864009610 tx=0x7f1864005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8cc7eb4b02bf4b server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:13.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.251+0000 7f185cff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:41:13.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.251+0000 7f185cff9640 1 -- 172.21.15.100:0/2748376283 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18640c3030 con 0x55eafead01b0 2024-07-26T13:41:13.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.251+0000 7f185cff9640 1 -- 172.21.15.100:0/2748376283 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f18640a6ba0 con 0x55eafead01b0 2024-07-26T13:41:13.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.251+0000 7f1881a74740 1 -- 172.21.15.100:0/2748376283 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18640aacb0 con 0x55eafead01b0 2024-07-26T13:41:13.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.251+0000 7f185cff9640 1 -- 172.21.15.100:0/2748376283 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f18640a6470 con 0x55eafead01b0 2024-07-26T13:41:13.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.251+0000 7f1881a74740 10 client.15696 fetch_fsmap learned FSMap version 26 2024-07-26T13:41:13.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.251+0000 7f1881a74740 1 -- 172.21.15.100:0/2748376283 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55eafeeaed40 con 0x55eafead01b0 2024-07-26T13:41:13.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.251+0000 7f185cff9640 1 -- 172.21.15.100:0/2748376283 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18640aacb0 con 0x55eafead01b0 2024-07-26T13:41:13.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.252+0000 7f185cff9640 1 -- 172.21.15.100:0/2748376283 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f18640e1400 con 0x55eafead01b0 2024-07-26T13:41:13.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.252+0000 7f1881a74740 10 client.15696 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:41:13.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.252+0000 7f1881a74740 10 client.15696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:41:13.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.252+0000 7f1881a74740 20 client.15696 populate_metadata read hostname 'smithi100' 2024-07-26T13:41:13.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.252+0000 7f1881a74740 1 --2- 172.21.15.100:0/2748376283 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55eafeaf9070 0x55eafef2ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:13.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.252+0000 7f1881a74740 4 client.15696 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-26T13:41:13.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.252+0000 7f1881a74740 1 -- 172.21.15.100:0/2748376283 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eafee125a0 con 0x55eafeaf9070 2024-07-26T13:41:13.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.252+0000 7f187928c640 1 --2- 172.21.15.100:0/2748376283 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55eafeaf9070 0x55eafef2ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:13.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.253+0000 7f185cff9640 10 client.15696 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:13.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.253+0000 7f187928c640 1 --2- 172.21.15.100:0/2748376283 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55eafeaf9070 0x55eafef2ae60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54f0bb44bd6bceac server_cookie=8307fc47f66a16c3 in_seq=0 out_seq=0 2024-07-26T13:41:13.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.253+0000 7f185cff9640 1 -- 172.21.15.100:0/2748376283 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eafee125a0 con 0x55eafeaf9070 2024-07-26T13:41:13.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.253+0000 7f185cff9640 10 client.15696 handle_command_reply: tid=0 2024-07-26T13:41:13.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.253+0000 7f1881a74740 1 client.15696 shutdown 2024-07-26T13:41:13.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.253+0000 7f1881a74740 1 -- 172.21.15.100:0/2748376283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eafead01b0 msgr2=0x55eafeec1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:13.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.253+0000 7f1881a74740 1 --2- 172.21.15.100:0/2748376283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eafead01b0 0x55eafeec1990 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1864009610 tx=0x7f1864005ab0 comp rx=0 tx=0).stop 2024-07-26T13:41:13.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.253+0000 7f1881a74740 1 -- 172.21.15.100:0/2748376283 shutdown_connections 2024-07-26T13:41:13.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.253+0000 7f1881a74740 1 --2- 172.21.15.100:0/2748376283 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55eafeaf9070 0x55eafef2ae60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:13.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.253+0000 7f185cff9640 0 client.15696 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:13.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.253+0000 7f1881a74740 1 --2- 172.21.15.100:0/2748376283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eafead8390 0x55eafeec1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:13.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.253+0000 7f1881a74740 1 --2- 172.21.15.100:0/2748376283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eafead01b0 0x55eafeec1990 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:13.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.253+0000 7f1881a74740 1 --2- 172.21.15.100:0/2748376283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eafeab79f0 0x55eafeec1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:13.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.253+0000 7f1881a74740 1 -- 172.21.15.100:0/2748376283 >> 172.21.15.100:0/2748376283 conn(0x55eafeaed460 msgr2=0x55eafeeaf740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:13.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.253+0000 7f1881a74740 1 -- 172.21.15.100:0/2748376283 shutdown_connections 2024-07-26T13:41:13.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.253+0000 7f1881a74740 1 -- 172.21.15.100:0/2748376283 wait complete. 2024-07-26T13:41:13.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.253+0000 7f1881a74740 20 client.15696 trim_cache size 0 max 16384 2024-07-26T13:41:13.258 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-07-26T13:41:13.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.254+0000 7f187b4f0640 1 -- 172.21.15.100:0/3730344982 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f18600b3e40 msgr2=0x7f18600b6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:13.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.254+0000 7f187b4f0640 1 --2- 172.21.15.100:0/3730344982 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f18600b3e40 0x7f18600b6350 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f186c005fd0 tx=0x7f186c009740 comp rx=0 tx=0).stop 2024-07-26T13:41:13.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.254+0000 7f187b4f0640 1 -- 172.21.15.100:0/3730344982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1874104ba0 msgr2=0x7f187419a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:13.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.254+0000 7f187b4f0640 1 --2- 172.21.15.100:0/3730344982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1874104ba0 0x7f187419a490 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f186400d740 tx=0x7f186400dc00 comp rx=0 tx=0).stop 2024-07-26T13:41:13.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.255+0000 7f187b4f0640 1 -- 172.21.15.100:0/3730344982 shutdown_connections 2024-07-26T13:41:13.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.255+0000 7f187b4f0640 1 --2- 172.21.15.100:0/3730344982 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f18600b3e40 0x7f18600b6350 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:13.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.255+0000 7f187b4f0640 1 --2- 172.21.15.100:0/3730344982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18741054f0 0x7f187419f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:13.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.255+0000 7f187b4f0640 1 --2- 172.21.15.100:0/3730344982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1874104ba0 0x7f187419a490 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:13.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.255+0000 7f187b4f0640 1 --2- 172.21.15.100:0/3730344982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1874102bd0 0x7f1874199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:13.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.255+0000 7f187b4f0640 1 -- 172.21.15.100:0/3730344982 >> 172.21.15.100:0/3730344982 conn(0x7f1874077690 msgr2=0x7f1874106980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:13.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.256+0000 7f187b4f0640 1 -- 172.21.15.100:0/3730344982 shutdown_connections 2024-07-26T13:41:13.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:13.256+0000 7f187b4f0640 1 -- 172.21.15.100:0/3730344982 wait complete. 2024-07-26T13:41:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:13 smithi100 ceph-mon[32599]: pgmap v397: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-07-26T13:41:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:13 smithi167 ceph-mon[37140]: pgmap v397: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-07-26T13:41:13.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:13 smithi138 ceph-mon[36954]: pgmap v397: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-07-26T13:41:15.146 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-26T13:41:15.147 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-26T13:41:15.573 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:15 smithi100 ceph-mon[32599]: pgmap v398: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 22 op/s 2024-07-26T13:41:15.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.607+0000 7fb5e85d2640 1 -- 172.21.15.100:0/1180954817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5e0104090 msgr2=0x7fb5e01044a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:15.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.607+0000 7fb5e85d2640 1 --2- 172.21.15.100:0/1180954817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5e0104090 0x7fb5e01044a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc009900 tx=0x7fb5dc02a410 comp rx=0 tx=0).stop 2024-07-26T13:41:15.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.607+0000 7fb5e85d2640 1 -- 172.21.15.100:0/1180954817 shutdown_connections 2024-07-26T13:41:15.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.607+0000 7fb5e85d2640 1 --2- 172.21.15.100:0/1180954817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5e01022c0 0x7fb5e0111f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:15.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.607+0000 7fb5e85d2640 1 --2- 172.21.15.100:0/1180954817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5e01017a0 0x7fb5e0101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:15.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.607+0000 7fb5e85d2640 1 --2- 172.21.15.100:0/1180954817 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5e0104090 0x7fb5e01044a0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:15.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.607+0000 7fb5e85d2640 1 -- 172.21.15.100:0/1180954817 >> 172.21.15.100:0/1180954817 conn(0x7fb5e0077690 msgr2=0x7fb5e0077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:15.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.607+0000 7fb5e85d2640 1 -- 172.21.15.100:0/1180954817 shutdown_connections 2024-07-26T13:41:15.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.607+0000 7fb5e85d2640 1 -- 172.21.15.100:0/1180954817 wait complete. 2024-07-26T13:41:15.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.608+0000 7fb5e85d2640 1 Processor -- start 2024-07-26T13:41:15.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.609+0000 7fb5e85d2640 1 -- start start 2024-07-26T13:41:15.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.609+0000 7fb5e85d2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5e01017a0 0x7fb5e019bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:15.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.609+0000 7fb5e85d2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5e01022c0 0x7fb5e019c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:15.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.609+0000 7fb5e85d2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5e0104090 0x7fb5e01a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:15.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.609+0000 7fb5e85d2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e0115740 con 0x7fb5e01022c0 2024-07-26T13:41:15.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.609+0000 7fb5e85d2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e01155c0 con 0x7fb5e01017a0 2024-07-26T13:41:15.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.609+0000 7fb5e636e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5e01017a0 0x7fb5e019bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:15.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.609+0000 7fb5e636e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5e01017a0 0x7fb5e019bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:60308/0 (socket says 172.21.15.100:60308) 2024-07-26T13:41:15.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.609+0000 7fb5e636e640 1 -- 172.21.15.100:0/898908731 learned_addr learned my addr 172.21.15.100:0/898908731 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:15.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.609+0000 7fb5e85d2640 1 -- 172.21.15.100:0/898908731 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e01158c0 con 0x7fb5e0104090 2024-07-26T13:41:15.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.610+0000 7fb5e5b6d640 1 --2- 172.21.15.100:0/898908731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5e01022c0 0x7fb5e019c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:15.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.610+0000 7fb5e6b6f640 1 --2- 172.21.15.100:0/898908731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5e0104090 0x7fb5e01a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:15.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.610+0000 7fb5e636e640 1 -- 172.21.15.100:0/898908731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5e0104090 msgr2=0x7fb5e01a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:15.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.610+0000 7fb5e636e640 1 --2- 172.21.15.100:0/898908731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5e0104090 0x7fb5e01a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:15.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.610+0000 7fb5e636e640 1 -- 172.21.15.100:0/898908731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5e01022c0 msgr2=0x7fb5e019c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:15.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.610+0000 7fb5e636e640 1 --2- 172.21.15.100:0/898908731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5e01022c0 0x7fb5e019c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:15.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.610+0000 7fb5e636e640 1 -- 172.21.15.100:0/898908731 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e01a1e10 con 0x7fb5e01017a0 2024-07-26T13:41:15.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.610+0000 7fb5e636e640 1 --2- 172.21.15.100:0/898908731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5e01017a0 0x7fb5e019bea0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc009850 tx=0x7fb5dc030650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:15.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.611+0000 7fb5d77fe640 1 -- 172.21.15.100:0/898908731 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc03f070 con 0x7fb5e01017a0 2024-07-26T13:41:15.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.611+0000 7fb5d77fe640 1 -- 172.21.15.100:0/898908731 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5dc004460 con 0x7fb5e01017a0 2024-07-26T13:41:15.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.611+0000 7fb5d77fe640 1 -- 172.21.15.100:0/898908731 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc035030 con 0x7fb5e01017a0 2024-07-26T13:41:15.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.611+0000 7fb5e85d2640 1 -- 172.21.15.100:0/898908731 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e01a20a0 con 0x7fb5e01017a0 2024-07-26T13:41:15.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.611+0000 7fb5e85d2640 1 -- 172.21.15.100:0/898908731 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e01a8860 con 0x7fb5e01017a0 2024-07-26T13:41:15.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.612+0000 7fb5e85d2640 1 -- 172.21.15.100:0/898908731 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5e0102720 con 0x7fb5e01017a0 2024-07-26T13:41:15.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.614+0000 7fb5d77fe640 1 -- 172.21.15.100:0/898908731 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fb5dc042070 con 0x7fb5e01017a0 2024-07-26T13:41:15.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.614+0000 7fb5d77fe640 1 --2- 172.21.15.100:0/898908731 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb5cc0b3e40 0x7fb5cc0b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:15.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.615+0000 7fb5d77fe640 1 -- 172.21.15.100:0/898908731 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fb5dc104e40 con 0x7fb5e01017a0 2024-07-26T13:41:15.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.615+0000 7fb5e5b6d640 1 --2- 172.21.15.100:0/898908731 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb5cc0b3e40 0x7fb5cc0b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:15.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.616+0000 7fb5e5b6d640 1 --2- 172.21.15.100:0/898908731 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb5cc0b3e40 0x7fb5cc0b6350 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb5e010ffc0 tx=0x7fb5d000b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:15.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.617+0000 7fb5d77fe640 1 -- 172.21.15.100:0/898908731 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fb5dc0cc110 con 0x7fb5e01017a0 2024-07-26T13:41:15.675 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:41:15.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:15 smithi167 ceph-mon[37140]: pgmap v398: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 22 op/s 2024-07-26T13:41:15.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.929+0000 7fb5e85d2640 1 -- 172.21.15.100:0/898908731 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5e019cfd0 con 0x7fb5e01017a0 2024-07-26T13:41:15.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.930+0000 7fb5d77fe640 1 -- 172.21.15.100:0/898908731 <== mon.2 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11927 (secure 0 0 0) 0x7fb5dc0d43e0 con 0x7fb5e01017a0 2024-07-26T13:41:15.931 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:41:15.931 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":26,"btime":"2024-07-26T13:40:09:530496+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14790,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/313501409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":313501409},{"type":"v1","addr":"172.21.15.100:6839","nonce":313501409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/1402975506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":1402975506},{"type":"v1","addr":"172.21.15.100:6841","nonce":1402975506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-26T13:37:01.281843+0000","modified":"2024-07-26T13:40:09.187125+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24607,"mds_1":24605,"mds_2":24595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24557":{"gid":24557,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/3290105300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3290105300},{"type":"v1","addr":"172.21.15.138:6835","nonce":3290105300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/3393225483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3393225483},{"type":"v1","addr":"172.21.15.138:6837","nonce":3393225483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24595":{"gid":24595,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6837/3828634468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3828634468},{"type":"v1","addr":"172.21.15.167:6837","nonce":3828634468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.138:6839/1580925471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1580925471},{"type":"v1","addr":"172.21.15.138:6839","nonce":1580925471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/398150514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":398150514},{"type":"v1","addr":"172.21.15.167:6839","nonce":398150514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24607,"qdb_cluster":[24595,24605,24607]},"id":1}]} 2024-07-26T13:41:15.931 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 26 2024-07-26T13:41:15.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.933+0000 7fb5e85d2640 1 -- 172.21.15.100:0/898908731 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb5cc0b3e40 msgr2=0x7fb5cc0b6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:15.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.933+0000 7fb5e85d2640 1 --2- 172.21.15.100:0/898908731 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb5cc0b3e40 0x7fb5cc0b6350 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb5e010ffc0 tx=0x7fb5d000b410 comp rx=0 tx=0).stop 2024-07-26T13:41:15.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.933+0000 7fb5e85d2640 1 -- 172.21.15.100:0/898908731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5e01017a0 msgr2=0x7fb5e019bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:15.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.933+0000 7fb5e85d2640 1 --2- 172.21.15.100:0/898908731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5e01017a0 0x7fb5e019bea0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc009850 tx=0x7fb5dc030650 comp rx=0 tx=0).stop 2024-07-26T13:41:15.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.933+0000 7fb5e85d2640 1 -- 172.21.15.100:0/898908731 shutdown_connections 2024-07-26T13:41:15.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.933+0000 7fb5e85d2640 1 --2- 172.21.15.100:0/898908731 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb5cc0b3e40 0x7fb5cc0b6350 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:15.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.933+0000 7fb5e85d2640 1 --2- 172.21.15.100:0/898908731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5e0104090 0x7fb5e01a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:15.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.933+0000 7fb5e85d2640 1 --2- 172.21.15.100:0/898908731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5e01022c0 0x7fb5e019c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:15.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.933+0000 7fb5e85d2640 1 --2- 172.21.15.100:0/898908731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5e01017a0 0x7fb5e019bea0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:15.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.933+0000 7fb5e85d2640 1 -- 172.21.15.100:0/898908731 >> 172.21.15.100:0/898908731 conn(0x7fb5e0077690 msgr2=0x7fb5e0100700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:15.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.933+0000 7fb5e85d2640 1 -- 172.21.15.100:0/898908731 shutdown_connections 2024-07-26T13:41:15.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:15.933+0000 7fb5e85d2640 1 -- 172.21.15.100:0/898908731 wait complete. 2024-07-26T13:41:15.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:15 smithi138 ceph-mon[36954]: pgmap v398: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 22 op/s 2024-07-26T13:41:16.673 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:16 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/898908731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:41:16.717 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24607 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0' 2024-07-26T13:41:16.717 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- timeout 105 ceph tell mds.24607 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0 2024-07-26T13:41:16.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:16 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/898908731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:41:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:16 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/898908731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:41:17.664 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:17 smithi100 ceph-mon[32599]: pgmap v399: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-07-26T13:41:17.749 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:41:17.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:17 smithi167 ceph-mon[37140]: pgmap v399: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-07-26T13:41:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.870+0000 7fed84432640 1 -- 172.21.15.100:0/1985440688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed7c1023e0 msgr2=0x7fed7c1027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.870+0000 7fed84432640 1 --2- 172.21.15.100:0/1985440688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed7c1023e0 0x7fed7c1027f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fed78009900 tx=0x7fed7802a410 comp rx=0 tx=0).stop 2024-07-26T13:41:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.870+0000 7fed84432640 1 -- 172.21.15.100:0/1985440688 shutdown_connections 2024-07-26T13:41:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.870+0000 7fed84432640 1 --2- 172.21.15.100:0/1985440688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed7c108660 0x7fed7c10ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.870+0000 7fed84432640 1 --2- 172.21.15.100:0/1985440688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed7c107d10 0x7fed7c108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.870+0000 7fed84432640 1 --2- 172.21.15.100:0/1985440688 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed7c1023e0 0x7fed7c1027f0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.870+0000 7fed84432640 1 -- 172.21.15.100:0/1985440688 >> 172.21.15.100:0/1985440688 conn(0x7fed7c07a170 msgr2=0x7fed7c07a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.870+0000 7fed84432640 1 -- 172.21.15.100:0/1985440688 shutdown_connections 2024-07-26T13:41:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.870+0000 7fed84432640 1 -- 172.21.15.100:0/1985440688 wait complete. 2024-07-26T13:41:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.871+0000 7fed84432640 1 Processor -- start 2024-07-26T13:41:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.871+0000 7fed84432640 1 -- start start 2024-07-26T13:41:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.871+0000 7fed84432640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed7c1023e0 0x7fed7c19ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.871+0000 7fed84432640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed7c107d10 0x7fed7c19b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.871+0000 7fed84432640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed7c108660 0x7fed7c19b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.871+0000 7fed84432640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c112360 con 0x7fed7c1023e0 2024-07-26T13:41:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.871+0000 7fed84432640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c1121e0 con 0x7fed7c107d10 2024-07-26T13:41:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.871+0000 7fed84432640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c1124e0 con 0x7fed7c108660 2024-07-26T13:41:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.871+0000 7fed821ce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed7c1023e0 0x7fed7c19ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.871+0000 7fed821ce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed7c1023e0 0x7fed7c19ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35708/0 (socket says 172.21.15.100:35708) 2024-07-26T13:41:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.871+0000 7fed821ce640 1 -- 172.21.15.100:0/2575509823 learned_addr learned my addr 172.21.15.100:0/2575509823 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.871+0000 7fed829cf640 1 --2- 172.21.15.100:0/2575509823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed7c108660 0x7fed7c19b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.872+0000 7fed819cd640 1 --2- 172.21.15.100:0/2575509823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed7c107d10 0x7fed7c19b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.872+0000 7fed821ce640 1 -- 172.21.15.100:0/2575509823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed7c108660 msgr2=0x7fed7c19b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.872+0000 7fed821ce640 1 --2- 172.21.15.100:0/2575509823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed7c108660 0x7fed7c19b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.872+0000 7fed821ce640 1 -- 172.21.15.100:0/2575509823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed7c107d10 msgr2=0x7fed7c19b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.872+0000 7fed821ce640 1 --2- 172.21.15.100:0/2575509823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed7c107d10 0x7fed7c19b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.872+0000 7fed821ce640 1 -- 172.21.15.100:0/2575509823 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed7c1a5f20 con 0x7fed7c1023e0 2024-07-26T13:41:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.872+0000 7fed829cf640 1 --2- 172.21.15.100:0/2575509823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed7c108660 0x7fed7c19b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:41:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.872+0000 7fed821ce640 1 --2- 172.21.15.100:0/2575509823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed7c1023e0 0x7fed7c19ac90 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fed78005ab0 tx=0x7fed78003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.872+0000 7fed737fe640 1 -- 172.21.15.100:0/2575509823 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7803f070 con 0x7fed7c1023e0 2024-07-26T13:41:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.872+0000 7fed84432640 1 -- 172.21.15.100:0/2575509823 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed7c1a61b0 con 0x7fed7c1023e0 2024-07-26T13:41:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.873+0000 7fed737fe640 1 -- 172.21.15.100:0/2575509823 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed78031d60 con 0x7fed7c1023e0 2024-07-26T13:41:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.873+0000 7fed737fe640 1 -- 172.21.15.100:0/2575509823 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed78035030 con 0x7fed7c1023e0 2024-07-26T13:41:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.873+0000 7fed84432640 1 -- 172.21.15.100:0/2575509823 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed7c198d70 con 0x7fed7c1023e0 2024-07-26T13:41:17.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.877+0000 7fed8b420740 1 Processor -- start 2024-07-26T13:41:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.877+0000 7fed8b420740 1 -- start start 2024-07-26T13:41:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.877+0000 7fed737fe640 1 -- 172.21.15.100:0/2575509823 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fed78042070 con 0x7fed7c1023e0 2024-07-26T13:41:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.877+0000 7fed8b420740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625ddfc51b0 0x5625de367ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:17.880 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.878+0000 7fed829cf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625ddfc51b0 0x5625de367ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:17.880 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.878+0000 7fed829cf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625ddfc51b0 0x5625de367ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35722/0 (socket says 172.21.15.100:35722) 2024-07-26T13:41:17.880 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.878+0000 7fed829cf640 1 -- 172.21.15.100:0/4023989953 learned_addr learned my addr 172.21.15.100:0/4023989953 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:17.880 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.878+0000 7fed8b420740 1 --2- 172.21.15.100:0/4023989953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ddf9c2f0 0x5625de368220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:17.880 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.878+0000 7fed8b420740 1 --2- 172.21.15.100:0/4023989953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625ddfa44d0 0x5625de368760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:17.881 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.878+0000 7fed8b420740 1 -- 172.21.15.100:0/4023989953 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5625de2dd3b0 con 0x5625ddfc51b0 2024-07-26T13:41:17.881 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.878+0000 7fed8b420740 1 -- 172.21.15.100:0/4023989953 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5625de2cc8b0 con 0x5625ddf9c2f0 2024-07-26T13:41:17.881 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.878+0000 7fed8b420740 1 -- 172.21.15.100:0/4023989953 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5625de2d08f0 con 0x5625ddfa44d0 2024-07-26T13:41:17.881 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.878+0000 7fed821ce640 1 --2- 172.21.15.100:0/4023989953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ddf9c2f0 0x5625de368220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:17.881 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.878+0000 7fed829cf640 1 -- 172.21.15.100:0/4023989953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625ddfa44d0 msgr2=0x5625de368760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:17.881 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.878+0000 7fed819cd640 1 --2- 172.21.15.100:0/4023989953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625ddfa44d0 0x5625de368760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:17.881 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.878+0000 7fed829cf640 1 --2- 172.21.15.100:0/4023989953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625ddfa44d0 0x5625de368760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:17.881 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.878+0000 7fed829cf640 1 -- 172.21.15.100:0/4023989953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ddf9c2f0 msgr2=0x5625de368220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:17.881 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.878+0000 7fed829cf640 1 --2- 172.21.15.100:0/4023989953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ddf9c2f0 0x5625de368220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:17.881 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.878+0000 7fed737fe640 1 --2- 172.21.15.100:0/2575509823 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fed680b3dc0 0x7fed680b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.878+0000 7fed829cf640 1 -- 172.21.15.100:0/4023989953 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625de2739a0 con 0x5625ddfc51b0 2024-07-26T13:41:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.879+0000 7fed737fe640 1 -- 172.21.15.100:0/2575509823 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fed78104330 con 0x7fed7c1023e0 2024-07-26T13:41:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.879+0000 7fed829cf640 1 --2- 172.21.15.100:0/4023989953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625ddfc51b0 0x5625de367ce0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fed740107a0 tx=0x7fed74010c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c612769c340b78c1 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.879+0000 7fed70bfe640 1 -- 172.21.15.100:0/4023989953 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7400ead0 con 0x5625ddfc51b0 2024-07-26T13:41:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.879+0000 7fed70bfe640 1 -- 172.21.15.100:0/4023989953 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed7400ec70 con 0x5625ddfc51b0 2024-07-26T13:41:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.879+0000 7fed819cd640 1 --2- 172.21.15.100:0/2575509823 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fed680b3dc0 0x7fed680b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.880+0000 7fed70bfe640 1 -- 172.21.15.100:0/4023989953 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed74011640 con 0x5625ddfc51b0 2024-07-26T13:41:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.880+0000 7fed8b420740 1 -- 172.21.15.100:0/4023989953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625ddfc51b0 msgr2=0x5625de367ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.880+0000 7fed8b420740 1 --2- 172.21.15.100:0/4023989953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625ddfc51b0 0x5625de367ce0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fed740107a0 tx=0x7fed74010c60 comp rx=0 tx=0).stop 2024-07-26T13:41:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.880+0000 7fed819cd640 1 --2- 172.21.15.100:0/2575509823 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fed680b3dc0 0x7fed680b62d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fed7c07abf0 tx=0x7fed6c00b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.881+0000 7fed8b420740 1 -- 172.21.15.100:0/4023989953 shutdown_connections 2024-07-26T13:41:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.881+0000 7fed8b420740 1 --2- 172.21.15.100:0/4023989953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625ddfa44d0 0x5625de368760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.881+0000 7fed8b420740 1 --2- 172.21.15.100:0/4023989953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ddf9c2f0 0x5625de368220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.881+0000 7fed8b420740 1 --2- 172.21.15.100:0/4023989953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625ddfc51b0 0x5625de367ce0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.881+0000 7fed8b420740 1 -- 172.21.15.100:0/4023989953 >> 172.21.15.100:0/4023989953 conn(0x5625ddfb9300 msgr2=0x5625de365bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.881+0000 7fed8b420740 1 -- 172.21.15.100:0/4023989953 shutdown_connections 2024-07-26T13:41:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.881+0000 7fed8b420740 1 -- 172.21.15.100:0/4023989953 wait complete. 2024-07-26T13:41:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.881+0000 7fed8b420740 1 -- start start 2024-07-26T13:41:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.882+0000 7fed8b420740 1 Processor -- start 2024-07-26T13:41:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.882+0000 7fed8b420740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625ddf9c2f0 0x5625de373800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.882+0000 7fed8b420740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ddfa44d0 0x5625de373d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.882+0000 7fed8b420740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625ddfc51b0 0x5625de37f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.882+0000 7fed829cf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625ddfc51b0 0x5625de37f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.882+0000 7fed829cf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625ddfc51b0 0x5625de37f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35738/0 (socket says 172.21.15.100:35738) 2024-07-26T13:41:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.882+0000 7fed829cf640 1 -- 172.21.15.100:0/798856969 learned_addr learned my addr 172.21.15.100:0/798856969 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.882+0000 7fed8b420740 1 -- 172.21.15.100:0/798856969 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5625de206ab0 con 0x5625ddfc51b0 2024-07-26T13:41:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.882+0000 7fed821ce640 1 --2- 172.21.15.100:0/798856969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625ddf9c2f0 0x5625de373800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.883+0000 7fed819cd640 1 --2- 172.21.15.100:0/798856969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ddfa44d0 0x5625de373d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.883+0000 7fed8b420740 1 -- 172.21.15.100:0/798856969 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5625de254cf0 con 0x5625ddfa44d0 2024-07-26T13:41:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.883+0000 7fed8b420740 1 -- 172.21.15.100:0/798856969 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5625de3764b0 con 0x5625ddf9c2f0 2024-07-26T13:41:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.883+0000 7fed819cd640 1 -- 172.21.15.100:0/798856969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625ddf9c2f0 msgr2=0x5625de373800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.883+0000 7fed819cd640 1 --2- 172.21.15.100:0/798856969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625ddf9c2f0 0x5625de373800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.883+0000 7fed819cd640 1 -- 172.21.15.100:0/798856969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625ddfc51b0 msgr2=0x5625de37f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.883+0000 7fed819cd640 1 --2- 172.21.15.100:0/798856969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625ddfc51b0 0x5625de37f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.883+0000 7fed819cd640 1 -- 172.21.15.100:0/798856969 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625de030790 con 0x5625ddfa44d0 2024-07-26T13:41:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.883+0000 7fed829cf640 1 --2- 172.21.15.100:0/798856969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625ddfc51b0 0x5625de37f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:41:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.884+0000 7fed8b420740 10 client.0 _ll_register_callbacks cb 0x5625de2c69f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:41:17.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.884+0000 7fed819cd640 1 --2- 172.21.15.100:0/798856969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ddfa44d0 0x5625de373d40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00bfd0 tx=0x7fed6c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d26e576a4aa42ca0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.884+0000 7fed8b420740 1 -- 172.21.15.100:0/798856969 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed7400ec70 con 0x5625ddfa44d0 2024-07-26T13:41:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.884+0000 7fed65ffb640 10 client.25181 ms_handle_connect on v2:172.21.15.138:3300/0 2024-07-26T13:41:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.884+0000 7fed65ffb640 1 -- 172.21.15.100:0/798856969 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c01f830 con 0x5625ddfa44d0 2024-07-26T13:41:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.884+0000 7fed65ffb640 1 -- 172.21.15.100:0/798856969 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fed6c01f9d0 con 0x5625ddfa44d0 2024-07-26T13:41:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.884+0000 7fed65ffb640 1 -- 172.21.15.100:0/798856969 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fed6c01fbd0 con 0x5625ddfa44d0 2024-07-26T13:41:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.884+0000 7fed8b420740 10 client.25181 fetch_fsmap learned FSMap version 26 2024-07-26T13:41:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.884+0000 7fed8b420740 1 -- 172.21.15.100:0/798856969 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({fsmap=26}) -- 0x5625de0a9620 con 0x5625ddfa44d0 2024-07-26T13:41:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.885+0000 7fed65ffb640 1 -- 172.21.15.100:0/798856969 <== mon.2 v2:172.21.15.138:3300/0 4 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fed6c031420 con 0x5625ddfa44d0 2024-07-26T13:41:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.885+0000 7fed8b420740 10 client.25181 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:41:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.885+0000 7fed8b420740 10 client.25181 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:41:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.885+0000 7fed8b420740 20 client.25181 populate_metadata read hostname 'smithi100' 2024-07-26T13:41:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.885+0000 7fed8b420740 1 --2- 172.21.15.100:0/798856969 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5625ddf83d80 0x5625de385620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.885+0000 7fed8b420740 4 client.25181 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:41:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.885+0000 7fed8b420740 1 -- 172.21.15.100:0/798856969 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625de2df300 con 0x5625ddf83d80 2024-07-26T13:41:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.886+0000 7fed821ce640 1 --2- 172.21.15.100:0/798856969 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5625ddf83d80 0x5625de385620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.886+0000 7fed821ce640 1 --2- 172.21.15.100:0/798856969 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5625ddf83d80 0x5625de385620 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12f560638b411c5c server_cookie=bc42198ebe325ea2 in_seq=0 out_seq=0 2024-07-26T13:41:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.886+0000 7fed65ffb640 10 client.25181 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.887+0000 7fed65ffb640 1 -- 172.21.15.100:0/798856969 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5625de2df300 con 0x5625ddf83d80 2024-07-26T13:41:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.887+0000 7fed65ffb640 10 client.25181 handle_command_reply: tid=0 2024-07-26T13:41:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.887+0000 7fed8b420740 1 client.25181 shutdown 2024-07-26T13:41:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.887+0000 7fed8b420740 1 -- 172.21.15.100:0/798856969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ddfa44d0 msgr2=0x5625de373d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.887+0000 7fed8b420740 1 --2- 172.21.15.100:0/798856969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ddfa44d0 0x5625de373d40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00bfd0 tx=0x7fed6c0046c0 comp rx=0 tx=0).stop 2024-07-26T13:41:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.887+0000 7fed8b420740 1 -- 172.21.15.100:0/798856969 shutdown_connections 2024-07-26T13:41:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.887+0000 7fed8b420740 1 --2- 172.21.15.100:0/798856969 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5625ddf83d80 0x5625de385620 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.887+0000 7fed8b420740 1 --2- 172.21.15.100:0/798856969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625ddfc51b0 0x5625de37f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.887+0000 7fed8b420740 1 --2- 172.21.15.100:0/798856969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ddfa44d0 0x5625de373d40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.887+0000 7fed8b420740 1 --2- 172.21.15.100:0/798856969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625ddf9c2f0 0x5625de373800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.887+0000 7fed8b420740 1 -- 172.21.15.100:0/798856969 >> 172.21.15.100:0/798856969 conn(0x5625ddfb9300 msgr2=0x5625de368d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.887+0000 7fed65ffb640 0 client.25181 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.887+0000 7fed8b420740 1 -- 172.21.15.100:0/798856969 shutdown_connections 2024-07-26T13:41:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.887+0000 7fed8b420740 1 -- 172.21.15.100:0/798856969 wait complete. 2024-07-26T13:41:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.888+0000 7fed8b420740 20 client.25181 trim_cache size 0 max 16384 2024-07-26T13:41:17.918 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.917+0000 7fed8b420740 1 Processor -- start 2024-07-26T13:41:17.918 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.917+0000 7fed8b420740 1 -- start start 2024-07-26T13:41:17.919 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.917+0000 7fed8b420740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625ddf9c2f0 0x5625de373ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:17.919 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.917+0000 7fed8b420740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625ddfa44d0 0x5625de3649c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:17.919 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.918+0000 7fed819cd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625ddfa44d0 0x5625de3649c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:17.919 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.918+0000 7fed819cd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625ddfa44d0 0x5625de3649c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35744/0 (socket says 172.21.15.100:35744) 2024-07-26T13:41:17.919 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.918+0000 7fed8b420740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ddfc51b0 0x5625de364f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:17.919 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.918+0000 7fed821ce640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625ddf9c2f0 0x5625de373ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:17.919 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.918+0000 7fed819cd640 1 -- 172.21.15.100:0/4142113006 learned_addr learned my addr 172.21.15.100:0/4142113006 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:17.919 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.918+0000 7fed8b420740 1 -- 172.21.15.100:0/4142113006 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c1b3c60 con 0x5625ddfa44d0 2024-07-26T13:41:17.920 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.918+0000 7fed8b420740 1 -- 172.21.15.100:0/4142113006 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c1b1530 con 0x5625ddfc51b0 2024-07-26T13:41:17.920 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.918+0000 7fed8b420740 1 -- 172.21.15.100:0/4142113006 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c1af380 con 0x5625ddf9c2f0 2024-07-26T13:41:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.918+0000 7fed829cf640 1 --2- 172.21.15.100:0/4142113006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ddfc51b0 0x5625de364f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.918+0000 7fed821ce640 1 -- 172.21.15.100:0/4142113006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ddfc51b0 msgr2=0x5625de364f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.918+0000 7fed821ce640 1 --2- 172.21.15.100:0/4142113006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ddfc51b0 0x5625de364f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.918+0000 7fed821ce640 1 -- 172.21.15.100:0/4142113006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625ddfa44d0 msgr2=0x5625de3649c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:17.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.918+0000 7fed821ce640 1 --2- 172.21.15.100:0/4142113006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625ddfa44d0 0x5625de3649c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:17.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.918+0000 7fed821ce640 1 -- 172.21.15.100:0/4142113006 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625de3697e0 con 0x5625ddf9c2f0 2024-07-26T13:41:17.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.918+0000 7fed829cf640 1 --2- 172.21.15.100:0/4142113006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ddfc51b0 0x5625de364f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:41:17.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.918+0000 7fed819cd640 1 --2- 172.21.15.100:0/4142113006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625ddfa44d0 0x5625de3649c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:41:17.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.918+0000 7fed821ce640 1 --2- 172.21.15.100:0/4142113006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625ddf9c2f0 0x5625de373ad0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fed78101aa0 tx=0x7fed78031f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a47271a4195a2d5 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:17.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.919+0000 7fed67fff640 1 -- 172.21.15.100:0/4142113006 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed78049040 con 0x5625ddf9c2f0 2024-07-26T13:41:17.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.919+0000 7fed67fff640 1 -- 172.21.15.100:0/4142113006 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed7803be50 con 0x5625ddf9c2f0 2024-07-26T13:41:17.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.919+0000 7fed67fff640 1 -- 172.21.15.100:0/4142113006 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed780d4010 con 0x5625ddf9c2f0 2024-07-26T13:41:17.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.920+0000 7fed8b420740 1 -- 172.21.15.100:0/4142113006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625ddf9c2f0 msgr2=0x5625de373ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:17.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.920+0000 7fed8b420740 1 --2- 172.21.15.100:0/4142113006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625ddf9c2f0 0x5625de373ad0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fed78101aa0 tx=0x7fed78031f10 comp rx=0 tx=0).stop 2024-07-26T13:41:17.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.920+0000 7fed8b420740 1 -- 172.21.15.100:0/4142113006 shutdown_connections 2024-07-26T13:41:17.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.920+0000 7fed8b420740 1 --2- 172.21.15.100:0/4142113006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ddfc51b0 0x5625de364f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:17.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.920+0000 7fed8b420740 1 --2- 172.21.15.100:0/4142113006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625ddfa44d0 0x5625de3649c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:17.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.920+0000 7fed8b420740 1 --2- 172.21.15.100:0/4142113006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625ddf9c2f0 0x5625de373ad0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:17.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.920+0000 7fed8b420740 1 -- 172.21.15.100:0/4142113006 >> 172.21.15.100:0/4142113006 conn(0x5625ddfb9300 msgr2=0x5625de3642b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:17.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.920+0000 7fed8b420740 1 -- 172.21.15.100:0/4142113006 shutdown_connections 2024-07-26T13:41:17.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.920+0000 7fed8b420740 1 -- 172.21.15.100:0/4142113006 wait complete. 2024-07-26T13:41:17.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.920+0000 7fed8b420740 1 -- start start 2024-07-26T13:41:17.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.921+0000 7fed8b420740 1 Processor -- start 2024-07-26T13:41:17.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.921+0000 7fed8b420740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625ddf9c2f0 0x5625de37bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:17.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.921+0000 7fed8b420740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625ddfa44d0 0x5625de378720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:17.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.921+0000 7fed8b420740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ddfc51b0 0x5625de378c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.921+0000 7fed821ce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625ddf9c2f0 0x5625de37bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.921+0000 7fed8b420740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c1af9f0 con 0x5625ddf9c2f0 2024-07-26T13:41:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.921+0000 7fed821ce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625ddf9c2f0 0x5625de37bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35748/0 (socket says 172.21.15.100:35748) 2024-07-26T13:41:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.921+0000 7fed821ce640 1 -- 172.21.15.100:0/3005626131 learned_addr learned my addr 172.21.15.100:0/3005626131 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.921+0000 7fed8b420740 1 -- 172.21.15.100:0/3005626131 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c1b0e60 con 0x5625ddfc51b0 2024-07-26T13:41:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.921+0000 7fed8b420740 1 -- 172.21.15.100:0/3005626131 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c1b00c0 con 0x5625ddfa44d0 2024-07-26T13:41:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.921+0000 7fed829cf640 1 --2- 172.21.15.100:0/3005626131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ddfc51b0 0x5625de378c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.921+0000 7fed819cd640 1 --2- 172.21.15.100:0/3005626131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625ddfa44d0 0x5625de378720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.921+0000 7fed8b420740 10 client.0 _ll_register_callbacks cb 0x7fed6c00b910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:41:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.921+0000 7fed821ce640 1 -- 172.21.15.100:0/3005626131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625ddfa44d0 msgr2=0x5625de378720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.921+0000 7fed821ce640 1 --2- 172.21.15.100:0/3005626131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625ddfa44d0 0x5625de378720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.921+0000 7fed821ce640 1 -- 172.21.15.100:0/3005626131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ddfc51b0 msgr2=0x5625de378c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.921+0000 7fed821ce640 1 --2- 172.21.15.100:0/3005626131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ddfc51b0 0x5625de378c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:17.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.921+0000 7fed821ce640 1 -- 172.21.15.100:0/3005626131 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625de3791a0 con 0x5625ddf9c2f0 2024-07-26T13:41:17.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.922+0000 7fed819cd640 1 --2- 172.21.15.100:0/3005626131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625ddfa44d0 0x5625de378720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:41:17.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.922+0000 7fed821ce640 1 --2- 172.21.15.100:0/3005626131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625ddf9c2f0 0x5625de37bb30 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fed780cad10 tx=0x7fed78101f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0595d85f4b400f0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:17.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.922+0000 7fed65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:41:17.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.922+0000 7fed65ffb640 1 -- 172.21.15.100:0/3005626131 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed78049040 con 0x5625ddf9c2f0 2024-07-26T13:41:17.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.922+0000 7fed65ffb640 1 -- 172.21.15.100:0/3005626131 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fed7803b580 con 0x5625ddf9c2f0 2024-07-26T13:41:17.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.922+0000 7fed8b420740 1 -- 172.21.15.100:0/3005626131 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed7803be50 con 0x5625ddf9c2f0 2024-07-26T13:41:17.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.922+0000 7fed65ffb640 1 -- 172.21.15.100:0/3005626131 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fed780cb010 con 0x5625ddf9c2f0 2024-07-26T13:41:17.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.922+0000 7fed8b420740 10 client.15720 fetch_fsmap learned FSMap version 26 2024-07-26T13:41:17.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.922+0000 7fed8b420740 1 -- 172.21.15.100:0/3005626131 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5625de3ec970 con 0x5625ddf9c2f0 2024-07-26T13:41:17.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.923+0000 7fed65ffb640 1 -- 172.21.15.100:0/3005626131 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed780cab20 con 0x5625ddf9c2f0 2024-07-26T13:41:17.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.923+0000 7fed65ffb640 1 -- 172.21.15.100:0/3005626131 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fed780cb270 con 0x5625ddf9c2f0 2024-07-26T13:41:17.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.923+0000 7fed8b420740 10 client.15720 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:41:17.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.923+0000 7fed8b420740 10 client.15720 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:41:17.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.923+0000 7fed8b420740 20 client.15720 populate_metadata read hostname 'smithi100' 2024-07-26T13:41:17.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.923+0000 7fed8b420740 1 --2- 172.21.15.100:0/3005626131 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5625ddf591c0 0x5625de3eeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:17.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.923+0000 7fed8b420740 4 client.15720 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-26T13:41:17.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.923+0000 7fed8b420740 1 -- 172.21.15.100:0/3005626131 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fed7803be50 con 0x5625ddf591c0 2024-07-26T13:41:17.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.923+0000 7fed819cd640 1 --2- 172.21.15.100:0/3005626131 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5625ddf591c0 0x5625de3eeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:17.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.924+0000 7fed65ffb640 10 client.15720 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:17.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.924+0000 7fed819cd640 1 --2- 172.21.15.100:0/3005626131 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5625ddf591c0 0x5625de3eeef0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=959aa1515f43450 server_cookie=b38b297d9c77fc22 in_seq=0 out_seq=0 2024-07-26T13:41:17.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.928+0000 7fed65ffb640 1 -- 172.21.15.100:0/3005626131 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fed7803be50 con 0x5625ddf591c0 2024-07-26T13:41:17.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.928+0000 7fed65ffb640 10 client.15720 handle_command_reply: tid=0 2024-07-26T13:41:17.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.928+0000 7fed8b420740 1 client.15720 shutdown 2024-07-26T13:41:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.928+0000 7fed8b420740 1 -- 172.21.15.100:0/3005626131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625ddf9c2f0 msgr2=0x5625de37bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.928+0000 7fed8b420740 1 --2- 172.21.15.100:0/3005626131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625ddf9c2f0 0x5625de37bb30 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fed780cad10 tx=0x7fed78101f50 comp rx=0 tx=0).stop 2024-07-26T13:41:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.929+0000 7fed8b420740 1 -- 172.21.15.100:0/3005626131 shutdown_connections 2024-07-26T13:41:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.929+0000 7fed8b420740 1 --2- 172.21.15.100:0/3005626131 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5625ddf591c0 0x5625de3eeef0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.929+0000 7fed8b420740 1 --2- 172.21.15.100:0/3005626131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5625ddfc51b0 0x5625de378c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.929+0000 7fed8b420740 1 --2- 172.21.15.100:0/3005626131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5625ddfa44d0 0x5625de378720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.929+0000 7fed8b420740 1 --2- 172.21.15.100:0/3005626131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5625ddf9c2f0 0x5625de37bb30 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.929+0000 7fed8b420740 1 -- 172.21.15.100:0/3005626131 >> 172.21.15.100:0/3005626131 conn(0x5625ddfb9300 msgr2=0x5625de364c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.929+0000 7fed65ffb640 0 client.15720 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.929+0000 7fed8b420740 1 -- 172.21.15.100:0/3005626131 shutdown_connections 2024-07-26T13:41:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.929+0000 7fed8b420740 1 -- 172.21.15.100:0/3005626131 wait complete. 2024-07-26T13:41:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.929+0000 7fed8b420740 20 client.15720 trim_cache size 0 max 16384 2024-07-26T13:41:17.934 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-07-26T13:41:17.934 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-07-26T13:41:17.934 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-07-26T13:41:17.934 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-07-26T13:41:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.930+0000 7fed84432640 1 -- 172.21.15.100:0/2575509823 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fed680b3dc0 msgr2=0x7fed680b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.930+0000 7fed84432640 1 --2- 172.21.15.100:0/2575509823 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fed680b3dc0 0x7fed680b62d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fed7c07abf0 tx=0x7fed6c00b390 comp rx=0 tx=0).stop 2024-07-26T13:41:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.930+0000 7fed84432640 1 -- 172.21.15.100:0/2575509823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed7c1023e0 msgr2=0x7fed7c19ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.930+0000 7fed84432640 1 --2- 172.21.15.100:0/2575509823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed7c1023e0 0x7fed7c19ac90 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fed78005ab0 tx=0x7fed78003cd0 comp rx=0 tx=0).stop 2024-07-26T13:41:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.930+0000 7fed84432640 1 -- 172.21.15.100:0/2575509823 shutdown_connections 2024-07-26T13:41:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.930+0000 7fed84432640 1 --2- 172.21.15.100:0/2575509823 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fed680b3dc0 0x7fed680b62d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.930+0000 7fed84432640 1 --2- 172.21.15.100:0/2575509823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed7c108660 0x7fed7c19b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.930+0000 7fed84432640 1 --2- 172.21.15.100:0/2575509823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed7c107d10 0x7fed7c19b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.930+0000 7fed84432640 1 --2- 172.21.15.100:0/2575509823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed7c1023e0 0x7fed7c19ac90 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.930+0000 7fed84432640 1 -- 172.21.15.100:0/2575509823 >> 172.21.15.100:0/2575509823 conn(0x7fed7c07a170 msgr2=0x7fed7c105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.930+0000 7fed84432640 1 -- 172.21.15.100:0/2575509823 shutdown_connections 2024-07-26T13:41:17.936 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:17.930+0000 7fed84432640 1 -- 172.21.15.100:0/2575509823 wait complete. 2024-07-26T13:41:17.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:17 smithi138 ceph-mon[36954]: pgmap v399: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-07-26T13:41:18.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.599+0000 7f04851ef640 1 -- 172.21.15.100:0/38797192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0480107610 msgr2=0x7f0480107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:18.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.599+0000 7f04851ef640 1 --2- 172.21.15.100:0/38797192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0480107610 0x7f0480107af0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0478009cd0 tx=0x7f047802a8e0 comp rx=0 tx=0).stop 2024-07-26T13:41:18.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.599+0000 7f04851ef640 1 -- 172.21.15.100:0/38797192 shutdown_connections 2024-07-26T13:41:18.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.599+0000 7f04851ef640 1 --2- 172.21.15.100:0/38797192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0480107610 0x7f0480107af0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:18.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.599+0000 7f04851ef640 1 --2- 172.21.15.100:0/38797192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0480102b20 0x7f0480107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:18.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.599+0000 7f04851ef640 1 --2- 172.21.15.100:0/38797192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04801021d0 0x7f04801025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:18.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.599+0000 7f04851ef640 1 -- 172.21.15.100:0/38797192 >> 172.21.15.100:0/38797192 conn(0x7f0480079160 msgr2=0x7f0480079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:18.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.599+0000 7f04851ef640 1 -- 172.21.15.100:0/38797192 shutdown_connections 2024-07-26T13:41:18.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.599+0000 7f04851ef640 1 -- 172.21.15.100:0/38797192 wait complete. 2024-07-26T13:41:18.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.600+0000 7f04851ef640 1 Processor -- start 2024-07-26T13:41:18.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.600+0000 7f04851ef640 1 -- start start 2024-07-26T13:41:18.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.601+0000 7f04851ef640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04801021d0 0x7f04801166b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:18.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.601+0000 7f04851ef640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0480102b20 0x7f048010f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:18.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.601+0000 7f04851ef640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0480107610 0x7f048010fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:18.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.601+0000 7f04851ef640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0480119b20 con 0x7f04801021d0 2024-07-26T13:41:18.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.601+0000 7f04851ef640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f04801199a0 con 0x7f0480107610 2024-07-26T13:41:18.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.601+0000 7f04851ef640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0480119ca0 con 0x7f0480102b20 2024-07-26T13:41:18.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.601+0000 7f047ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04801021d0 0x7f04801166b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:18.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.601+0000 7f047ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04801021d0 0x7f04801166b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35764/0 (socket says 172.21.15.100:35764) 2024-07-26T13:41:18.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.601+0000 7f047ed9d640 1 -- 172.21.15.100:0/537326011 learned_addr learned my addr 172.21.15.100:0/537326011 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:18.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.601+0000 7f047f59e640 1 --2- 172.21.15.100:0/537326011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0480107610 0x7f048010fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:18.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.601+0000 7f047e59c640 1 --2- 172.21.15.100:0/537326011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0480102b20 0x7f048010f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:18.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.602+0000 7f047f59e640 1 -- 172.21.15.100:0/537326011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0480102b20 msgr2=0x7f048010f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:18.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.602+0000 7f047f59e640 1 --2- 172.21.15.100:0/537326011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0480102b20 0x7f048010f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:18.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.602+0000 7f047f59e640 1 -- 172.21.15.100:0/537326011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04801021d0 msgr2=0x7f04801166b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:18.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.602+0000 7f047f59e640 1 --2- 172.21.15.100:0/537326011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04801021d0 0x7f04801166b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:18.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.602+0000 7f047f59e640 1 -- 172.21.15.100:0/537326011 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0480110700 con 0x7f0480107610 2024-07-26T13:41:18.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.602+0000 7f047f59e640 1 --2- 172.21.15.100:0/537326011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0480107610 0x7f048010fee0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0478009ca0 tx=0x7f0478003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:18.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.602+0000 7f0467fff640 1 -- 172.21.15.100:0/537326011 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0478004240 con 0x7f0480107610 2024-07-26T13:41:18.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.602+0000 7f0467fff640 1 -- 172.21.15.100:0/537326011 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0478031d00 con 0x7f0480107610 2024-07-26T13:41:18.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.602+0000 7f0467fff640 1 -- 172.21.15.100:0/537326011 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0478030e70 con 0x7f0480107610 2024-07-26T13:41:18.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.602+0000 7f04851ef640 1 -- 172.21.15.100:0/537326011 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0480115320 con 0x7f0480107610 2024-07-26T13:41:18.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.602+0000 7f04851ef640 1 -- 172.21.15.100:0/537326011 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04801157e0 con 0x7f0480107610 2024-07-26T13:41:18.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.606+0000 7f048b773740 1 Processor -- start 2024-07-26T13:41:18.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.606+0000 7f0467fff640 1 -- 172.21.15.100:0/537326011 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f047803a400 con 0x7f0480107610 2024-07-26T13:41:18.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.606+0000 7f048b773740 1 -- start start 2024-07-26T13:41:18.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.606+0000 7f048b773740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef35e3350 0x555ef39883b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:18.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.606+0000 7f048b773740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ef35ba490 0x555ef39888f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:18.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.606+0000 7f048b773740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ef35c2670 0x555ef3988e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:18.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.607+0000 7f048b773740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x555ef38702f0 con 0x555ef35e3350 2024-07-26T13:41:18.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.607+0000 7f048b773740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555ef38fea80 con 0x555ef35c2670 2024-07-26T13:41:18.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.607+0000 7f048b773740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555ef38edf80 con 0x555ef35ba490 2024-07-26T13:41:18.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.607+0000 7f047e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef35e3350 0x555ef39883b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:18.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.607+0000 7f047e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef35e3350 0x555ef39883b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35768/0 (socket says 172.21.15.100:35768) 2024-07-26T13:41:18.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.607+0000 7f047e59c640 1 -- 172.21.15.100:0/1822735662 learned_addr learned my addr 172.21.15.100:0/1822735662 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:18.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.607+0000 7f047e59c640 1 --2- 172.21.15.100:0/1822735662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ef35c2670 0x555ef3988e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:18.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.607+0000 7f0467fff640 1 --2- 172.21.15.100:0/537326011 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f04680b3dc0 0x7f04680b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:18.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.607+0000 7f047e59c640 1 -- 172.21.15.100:0/1822735662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ef35ba490 msgr2=0x555ef39888f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-26T13:41:18.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.607+0000 7f047e59c640 1 --2- 172.21.15.100:0/1822735662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ef35ba490 0x555ef39888f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:18.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.607+0000 7f047e59c640 1 -- 172.21.15.100:0/1822735662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ef35c2670 msgr2=0x555ef3988e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:18.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.607+0000 7f047e59c640 1 --2- 172.21.15.100:0/1822735662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ef35c2670 0x555ef3988e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:18.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.607+0000 7f047e59c640 1 -- 172.21.15.100:0/1822735662 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ef364e930 con 0x555ef35e3350 2024-07-26T13:41:18.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.607+0000 7f0467fff640 1 -- 172.21.15.100:0/537326011 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f047802e030 con 0x7f0480107610 2024-07-26T13:41:18.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.607+0000 7f047f59e640 1 --2- 172.21.15.100:0/537326011 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f04680b3dc0 0x7f04680b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:18.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.608+0000 7f047e59c640 1 --2- 172.21.15.100:0/1822735662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef35e3350 0x555ef39883b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f04740135b0 tx=0x7f0474005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e92d02596f3a3bb0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:18.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.608+0000 7f04657fa640 1 -- 172.21.15.100:0/1822735662 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0474004050 con 0x555ef35e3350 2024-07-26T13:41:18.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.608+0000 7f04657fa640 1 -- 172.21.15.100:0/1822735662 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04740041f0 con 0x555ef35e3350 2024-07-26T13:41:18.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.608+0000 7f04657fa640 1 -- 172.21.15.100:0/1822735662 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047400f760 con 0x555ef35e3350 2024-07-26T13:41:18.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.609+0000 7f048b773740 1 -- 172.21.15.100:0/1822735662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef35e3350 msgr2=0x555ef39883b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:18.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.609+0000 7f047f59e640 1 --2- 172.21.15.100:0/537326011 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f04680b3dc0 0x7f04680b62d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0478003fd0 tx=0x7f0478005df0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:18.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.609+0000 7f048b773740 1 --2- 172.21.15.100:0/1822735662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef35e3350 0x555ef39883b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f04740135b0 tx=0x7f0474005850 comp rx=0 tx=0).stop 2024-07-26T13:41:18.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.609+0000 7f048b773740 1 -- 172.21.15.100:0/1822735662 shutdown_connections 2024-07-26T13:41:18.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.609+0000 7f048b773740 1 --2- 172.21.15.100:0/1822735662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ef35c2670 0x555ef3988e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:18.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.609+0000 7f048b773740 1 --2- 172.21.15.100:0/1822735662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ef35ba490 0x555ef39888f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:18.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.609+0000 7f048b773740 1 --2- 172.21.15.100:0/1822735662 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef35e3350 0x555ef39883b0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:18.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.609+0000 7f048b773740 1 -- 172.21.15.100:0/1822735662 >> 172.21.15.100:0/1822735662 conn(0x555ef35d7740 msgr2=0x555ef39862a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:18.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.610+0000 7f048b773740 1 -- 172.21.15.100:0/1822735662 shutdown_connections 2024-07-26T13:41:18.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.610+0000 7f048b773740 1 -- 172.21.15.100:0/1822735662 wait complete. 2024-07-26T13:41:18.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.610+0000 7f048b773740 1 -- start start 2024-07-26T13:41:18.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.610+0000 7f048b773740 1 Processor -- start 2024-07-26T13:41:18.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.611+0000 7f048b773740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ef35ba490 0x555ef3994000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:18.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.611+0000 7f048b773740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef35c2670 0x555ef39945c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:18.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.611+0000 7f048b773740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ef35e3350 0x555ef399fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:18.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.611+0000 7f047e59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ef35ba490 0x555ef3994000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:18.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.611+0000 7f047ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef35c2670 0x555ef39945c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:18.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.611+0000 7f047ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef35c2670 0x555ef39945c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35770/0 (socket says 172.21.15.100:35770) 2024-07-26T13:41:18.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.611+0000 7f047ed9d640 1 -- 172.21.15.100:0/1301018532 learned_addr learned my addr 172.21.15.100:0/1301018532 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:18.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.611+0000 7f048b773740 1 -- 172.21.15.100:0/1301018532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x555ef38904f0 con 0x555ef35c2670 2024-07-26T13:41:18.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.612+0000 7f048b773740 1 -- 172.21.15.100:0/1301018532 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555ef38f1fc0 con 0x555ef35ba490 2024-07-26T13:41:18.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.612+0000 7f048b773740 1 -- 172.21.15.100:0/1301018532 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555ef3751890 con 0x555ef35e3350 2024-07-26T13:41:18.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.612+0000 7f047e59c640 1 --2- 172.21.15.100:0/1301018532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ef35e3350 0x555ef399fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:18.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.612+0000 7f047ed9d640 1 -- 172.21.15.100:0/1301018532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ef35e3350 msgr2=0x555ef399fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:18.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.612+0000 7f047ed9d640 1 --2- 172.21.15.100:0/1301018532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ef35e3350 0x555ef399fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:18.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.612+0000 7f047ed9d640 1 -- 172.21.15.100:0/1301018532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ef35ba490 msgr2=0x555ef3994000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:18.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.612+0000 7f047ed9d640 1 --2- 172.21.15.100:0/1301018532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ef35ba490 0x555ef3994000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:18.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.612+0000 7f047ed9d640 1 -- 172.21.15.100:0/1301018532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ef389fc50 con 0x555ef35c2670 2024-07-26T13:41:18.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.612+0000 7f048b773740 10 client.0 _ll_register_callbacks cb 0x555ef389a500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:41:18.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.612+0000 7f048b773740 1 -- 172.21.15.100:0/1301018532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04740041f0 con 0x555ef35c2670 2024-07-26T13:41:18.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.612+0000 7f047ed9d640 1 --2- 172.21.15.100:0/1301018532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef35c2670 0x555ef39945c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f047000bf80 tx=0x7f0470010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f97c679546fb43c3 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:18.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.612+0000 7f0462ff5640 10 client.15732 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:41:18.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.613+0000 7f0462ff5640 1 -- 172.21.15.100:0/1301018532 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0470011850 con 0x555ef35c2670 2024-07-26T13:41:18.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.613+0000 7f0462ff5640 1 -- 172.21.15.100:0/1301018532 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f04700119f0 con 0x555ef35c2670 2024-07-26T13:41:18.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.613+0000 7f0462ff5640 1 -- 172.21.15.100:0/1301018532 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0470011c30 con 0x555ef35c2670 2024-07-26T13:41:18.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.613+0000 7f048b773740 10 client.15732 fetch_fsmap learned FSMap version 26 2024-07-26T13:41:18.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.613+0000 7f048b773740 1 -- 172.21.15.100:0/1301018532 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({fsmap=26}) -- 0x555ef39a2a40 con 0x555ef35c2670 2024-07-26T13:41:18.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.614+0000 7f0462ff5640 1 -- 172.21.15.100:0/1301018532 <== mon.0 v2:172.21.15.100:3300/0 4 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f0470026420 con 0x555ef35c2670 2024-07-26T13:41:18.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.614+0000 7f048b773740 10 client.15732 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:41:18.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.614+0000 7f048b773740 10 client.15732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:41:18.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.614+0000 7f048b773740 20 client.15732 populate_metadata read hostname 'smithi100' 2024-07-26T13:41:18.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.614+0000 7f048b773740 1 --2- 172.21.15.100:0/1301018532 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555ef35a1cd0 0x555ef39a6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:18.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.614+0000 7f048b773740 4 client.15732 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:41:18.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.614+0000 7f048b773740 1 -- 172.21.15.100:0/1301018532 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ef39009d0 con 0x555ef35a1cd0 2024-07-26T13:41:18.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.614+0000 7f047e59c640 1 --2- 172.21.15.100:0/1301018532 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555ef35a1cd0 0x555ef39a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:18.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.615+0000 7f047e59c640 1 --2- 172.21.15.100:0/1301018532 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555ef35a1cd0 0x555ef39a6360 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e61d446d7e918ef server_cookie=f415b3492907fa9b in_seq=0 out_seq=0 2024-07-26T13:41:18.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.615+0000 7f0462ff5640 10 client.15732 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:18.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.615+0000 7f0462ff5640 1 -- 172.21.15.100:0/1301018532 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555ef39009d0 con 0x555ef35a1cd0 2024-07-26T13:41:18.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.615+0000 7f0462ff5640 10 client.15732 handle_command_reply: tid=0 2024-07-26T13:41:18.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.615+0000 7f048b773740 1 client.15732 shutdown 2024-07-26T13:41:18.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.616+0000 7f048b773740 1 -- 172.21.15.100:0/1301018532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef35c2670 msgr2=0x555ef39945c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:18.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.616+0000 7f048b773740 1 --2- 172.21.15.100:0/1301018532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef35c2670 0x555ef39945c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f047000bf80 tx=0x7f0470010860 comp rx=0 tx=0).stop 2024-07-26T13:41:18.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.616+0000 7f048b773740 1 -- 172.21.15.100:0/1301018532 shutdown_connections 2024-07-26T13:41:18.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.616+0000 7f048b773740 1 --2- 172.21.15.100:0/1301018532 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555ef35a1cd0 0x555ef39a6360 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:18.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.616+0000 7f048b773740 1 --2- 172.21.15.100:0/1301018532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ef35e3350 0x555ef399fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:18.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.616+0000 7f048b773740 1 --2- 172.21.15.100:0/1301018532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef35c2670 0x555ef39945c0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:18.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.616+0000 7f048b773740 1 --2- 172.21.15.100:0/1301018532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ef35ba490 0x555ef3994000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:18.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.616+0000 7f048b773740 1 -- 172.21.15.100:0/1301018532 >> 172.21.15.100:0/1301018532 conn(0x555ef35d7740 msgr2=0x555ef398b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:18.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.616+0000 7f0462ff5640 0 client.15732 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:18.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.616+0000 7f048b773740 1 -- 172.21.15.100:0/1301018532 shutdown_connections 2024-07-26T13:41:18.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.616+0000 7f048b773740 1 -- 172.21.15.100:0/1301018532 wait complete. 2024-07-26T13:41:18.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.616+0000 7f048b773740 20 client.15732 trim_cache size 0 max 16384 2024-07-26T13:41:18.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.650+0000 7f048b773740 1 Processor -- start 2024-07-26T13:41:18.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.650+0000 7f048b773740 1 -- start start 2024-07-26T13:41:18.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.650+0000 7f048b773740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ef35ba490 0x555ef3994670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:18.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.650+0000 7f048b773740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef35c2670 0x555ef3995a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:18.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.650+0000 7f048b773740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ef35e3350 0x555ef398d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:18.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.651+0000 7f048b773740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f04801c0770 con 0x555ef35c2670 2024-07-26T13:41:18.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.651+0000 7f048b773740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f04801be040 con 0x555ef35e3350 2024-07-26T13:41:18.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.651+0000 7f048b773740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f04801bbe90 con 0x555ef35ba490 2024-07-26T13:41:18.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.651+0000 7f047ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef35c2670 0x555ef3995a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:18.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.651+0000 7f047ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef35c2670 0x555ef3995a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35786/0 (socket says 172.21.15.100:35786) 2024-07-26T13:41:18.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.651+0000 7f047ed9d640 1 -- 172.21.15.100:0/2691057392 learned_addr learned my addr 172.21.15.100:0/2691057392 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:18.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.651+0000 7f047e59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ef35ba490 0x555ef3994670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:18.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.651+0000 7f047e59c640 1 --2- 172.21.15.100:0/2691057392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ef35e3350 0x555ef398d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:18.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.651+0000 7f047e59c640 1 -- 172.21.15.100:0/2691057392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ef35e3350 msgr2=0x555ef398d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:18.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.651+0000 7f047e59c640 1 --2- 172.21.15.100:0/2691057392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ef35e3350 0x555ef398d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:18.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.651+0000 7f047e59c640 1 -- 172.21.15.100:0/2691057392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef35c2670 msgr2=0x555ef3995a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:18.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.651+0000 7f047e59c640 1 --2- 172.21.15.100:0/2691057392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef35c2670 0x555ef3995a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:18.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.651+0000 7f047e59c640 1 -- 172.21.15.100:0/2691057392 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ef398a6a0 con 0x555ef35ba490 2024-07-26T13:41:18.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.651+0000 7f047e59c640 1 --2- 172.21.15.100:0/2691057392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ef35ba490 0x555ef3994670 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f04740096a0 tx=0x7f0474003d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30fc014d9d83be72 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:18.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.651+0000 7f0464ff9640 1 -- 172.21.15.100:0/2691057392 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0474003f50 con 0x555ef35ba490 2024-07-26T13:41:18.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.652+0000 7f0464ff9640 1 -- 172.21.15.100:0/2691057392 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0474003aa0 con 0x555ef35ba490 2024-07-26T13:41:18.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.652+0000 7f0464ff9640 1 -- 172.21.15.100:0/2691057392 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047400f7d0 con 0x555ef35ba490 2024-07-26T13:41:18.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.653+0000 7f048b773740 1 -- 172.21.15.100:0/2691057392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ef35ba490 msgr2=0x555ef3994670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:18.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.653+0000 7f048b773740 1 --2- 172.21.15.100:0/2691057392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ef35ba490 0x555ef3994670 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f04740096a0 tx=0x7f0474003d90 comp rx=0 tx=0).stop 2024-07-26T13:41:18.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.653+0000 7f048b773740 1 -- 172.21.15.100:0/2691057392 shutdown_connections 2024-07-26T13:41:18.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.653+0000 7f048b773740 1 --2- 172.21.15.100:0/2691057392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ef35e3350 0x555ef398d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:18.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.653+0000 7f048b773740 1 --2- 172.21.15.100:0/2691057392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef35c2670 0x555ef3995a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:18.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.653+0000 7f048b773740 1 --2- 172.21.15.100:0/2691057392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ef35ba490 0x555ef3994670 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:18.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.653+0000 7f048b773740 1 -- 172.21.15.100:0/2691057392 >> 172.21.15.100:0/2691057392 conn(0x555ef35d7740 msgr2=0x555ef3995320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:18.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.653+0000 7f048b773740 1 -- 172.21.15.100:0/2691057392 shutdown_connections 2024-07-26T13:41:18.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.653+0000 7f048b773740 1 -- 172.21.15.100:0/2691057392 wait complete. 2024-07-26T13:41:18.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.653+0000 7f048b773740 1 -- start start 2024-07-26T13:41:18.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.654+0000 7f048b773740 1 Processor -- start 2024-07-26T13:41:18.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.654+0000 7f048b773740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ef35ba490 0x555ef399c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:18.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.654+0000 7f048b773740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ef35c2670 0x555ef399f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:18.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.654+0000 7f048b773740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef35e3350 0x555ef399fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:18.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.654+0000 7f048b773740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f04801bc500 con 0x555ef35e3350 2024-07-26T13:41:18.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.654+0000 7f048b773740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f04801bd970 con 0x555ef35c2670 2024-07-26T13:41:18.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.654+0000 7f048b773740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f04801bcbd0 con 0x555ef35ba490 2024-07-26T13:41:18.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.654+0000 7f048b773740 10 client.0 _ll_register_callbacks cb 0x7f047000bbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:41:18.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.654+0000 7f047e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef35e3350 0x555ef399fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:18.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.654+0000 7f047ed9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ef35c2670 0x555ef399f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:18.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.654+0000 7f047ed9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ef35c2670 0x555ef399f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:60412/0 (socket says 172.21.15.100:60412) 2024-07-26T13:41:18.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.654+0000 7f047e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef35e3350 0x555ef399fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35794/0 (socket says 172.21.15.100:35794) 2024-07-26T13:41:18.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.654+0000 7f047ed9d640 1 -- 172.21.15.100:0/2699029407 learned_addr learned my addr 172.21.15.100:0/2699029407 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:18.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.655+0000 7f047e59c640 1 --2- 172.21.15.100:0/2699029407 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ef35ba490 0x555ef399c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:18.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.655+0000 7f047ed9d640 1 -- 172.21.15.100:0/2699029407 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ef35ba490 msgr2=0x555ef399c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:18.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.655+0000 7f047ed9d640 1 --2- 172.21.15.100:0/2699029407 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ef35ba490 0x555ef399c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:18.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.655+0000 7f047ed9d640 1 -- 172.21.15.100:0/2699029407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef35e3350 msgr2=0x555ef399fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:18.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.655+0000 7f047ed9d640 1 --2- 172.21.15.100:0/2699029407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef35e3350 0x555ef399fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:18.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.655+0000 7f047ed9d640 1 -- 172.21.15.100:0/2699029407 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ef399ff50 con 0x555ef35c2670 2024-07-26T13:41:18.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.655+0000 7f047ed9d640 1 --2- 172.21.15.100:0/2699029407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ef35c2670 0x555ef399f4d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f047000bcd0 tx=0x7f047000b410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e54105f90adde245 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:18.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.655+0000 7f0462ff5640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-07-26T13:41:18.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.655+0000 7f0462ff5640 1 -- 172.21.15.100:0/2699029407 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047000ba10 con 0x555ef35c2670 2024-07-26T13:41:18.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.655+0000 7f0462ff5640 1 -- 172.21.15.100:0/2699029407 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f0470011650 con 0x555ef35c2670 2024-07-26T13:41:18.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.655+0000 7f048b773740 1 -- 172.21.15.100:0/2699029407 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0474003aa0 con 0x555ef35c2670 2024-07-26T13:41:18.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.656+0000 7f0462ff5640 1 -- 172.21.15.100:0/2699029407 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f047003f050 con 0x555ef35c2670 2024-07-26T13:41:18.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.656+0000 7f048b773740 10 client.25202 fetch_fsmap learned FSMap version 26 2024-07-26T13:41:18.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.656+0000 7f048b773740 1 -- 172.21.15.100:0/2699029407 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555ef399a970 con 0x555ef35c2670 2024-07-26T13:41:18.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.656+0000 7f0462ff5640 1 -- 172.21.15.100:0/2699029407 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0470036070 con 0x555ef35c2670 2024-07-26T13:41:18.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.656+0000 7f0462ff5640 1 -- 172.21.15.100:0/2699029407 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f04700319e0 con 0x555ef35c2670 2024-07-26T13:41:18.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.656+0000 7f048b773740 10 client.25202 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:41:18.946 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:41:18.946 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-07-26T13:41:18.946 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "04358fed-5625-44f4-9858-cb794a5ee345", 2024-07-26T13:41:18.946 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-07-26T13:41:18.946 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:41:18.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.656+0000 7f048b773740 10 client.25202 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:41:18.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.656+0000 7f048b773740 20 client.25202 populate_metadata read hostname 'smithi100' 2024-07-26T13:41:18.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.656+0000 7f048b773740 1 --2- 172.21.15.100:0/2699029407 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555ef3577080 0x555ef3a10f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:18.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.656+0000 7f048b773740 4 client.25202 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-26T13:41:18.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.656+0000 7f048b773740 1 -- 172.21.15.100:0/2699029407 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f0474003aa0 con 0x555ef3577080 2024-07-26T13:41:18.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.657+0000 7f047e59c640 1 --2- 172.21.15.100:0/2699029407 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555ef3577080 0x555ef3a10f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:18.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.658+0000 7f047e59c640 1 --2- 172.21.15.100:0/2699029407 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555ef3577080 0x555ef3a10f80 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d64ac57a34c503f server_cookie=9069c2a35d2e4d70 in_seq=0 out_seq=0 2024-07-26T13:41:18.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.658+0000 7f0462ff5640 10 client.25202 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:18.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.659+0000 7f0462ff5640 1 -- 172.21.15.100:0/2699029407 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f0474003aa0 con 0x555ef3577080 2024-07-26T13:41:18.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.659+0000 7f0462ff5640 10 client.25202 handle_command_reply: tid=0 2024-07-26T13:41:18.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.659+0000 7f048b773740 1 client.25202 shutdown 2024-07-26T13:41:18.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.659+0000 7f048b773740 1 -- 172.21.15.100:0/2699029407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ef35c2670 msgr2=0x555ef399f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:18.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.659+0000 7f048b773740 1 --2- 172.21.15.100:0/2699029407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ef35c2670 0x555ef399f4d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f047000bcd0 tx=0x7f047000b410 comp rx=0 tx=0).stop 2024-07-26T13:41:18.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.659+0000 7f048b773740 1 -- 172.21.15.100:0/2699029407 shutdown_connections 2024-07-26T13:41:18.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.659+0000 7f048b773740 1 --2- 172.21.15.100:0/2699029407 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555ef3577080 0x555ef3a10f80 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:18.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.659+0000 7f048b773740 1 --2- 172.21.15.100:0/2699029407 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ef35e3350 0x555ef399fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:18.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.659+0000 7f048b773740 1 --2- 172.21.15.100:0/2699029407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ef35c2670 0x555ef399f4d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:18.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.659+0000 7f0462ff5640 0 client.25202 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:18.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.659+0000 7f048b773740 1 --2- 172.21.15.100:0/2699029407 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ef35ba490 0x555ef399c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:18.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.659+0000 7f048b773740 1 -- 172.21.15.100:0/2699029407 >> 172.21.15.100:0/2699029407 conn(0x555ef35d7740 msgr2=0x555ef39952b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:18.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.660+0000 7f048b773740 1 -- 172.21.15.100:0/2699029407 shutdown_connections 2024-07-26T13:41:18.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.660+0000 7f048b773740 1 -- 172.21.15.100:0/2699029407 wait complete. 2024-07-26T13:41:18.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.660+0000 7f048b773740 20 client.25202 trim_cache size 0 max 16384 2024-07-26T13:41:18.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.660+0000 7f04851ef640 1 -- 172.21.15.100:0/537326011 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f04680b3dc0 msgr2=0x7f04680b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:18.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.660+0000 7f04851ef640 1 --2- 172.21.15.100:0/537326011 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f04680b3dc0 0x7f04680b62d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0478003fd0 tx=0x7f0478005df0 comp rx=0 tx=0).stop 2024-07-26T13:41:18.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.660+0000 7f04851ef640 1 -- 172.21.15.100:0/537326011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0480107610 msgr2=0x7f048010fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:18.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.660+0000 7f04851ef640 1 --2- 172.21.15.100:0/537326011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0480107610 0x7f048010fee0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0478009ca0 tx=0x7f0478003ba0 comp rx=0 tx=0).stop 2024-07-26T13:41:18.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.661+0000 7f04851ef640 1 -- 172.21.15.100:0/537326011 shutdown_connections 2024-07-26T13:41:18.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.661+0000 7f04851ef640 1 --2- 172.21.15.100:0/537326011 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f04680b3dc0 0x7f04680b62d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:18.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.661+0000 7f04851ef640 1 --2- 172.21.15.100:0/537326011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0480107610 0x7f048010fee0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:18.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.661+0000 7f04851ef640 1 --2- 172.21.15.100:0/537326011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0480102b20 0x7f048010f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:18.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.661+0000 7f04851ef640 1 --2- 172.21.15.100:0/537326011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04801021d0 0x7f04801166b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:18.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.661+0000 7f04851ef640 1 -- 172.21.15.100:0/537326011 >> 172.21.15.100:0/537326011 conn(0x7f0480079160 msgr2=0x7f0480105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:18.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.661+0000 7f04851ef640 1 -- 172.21.15.100:0/537326011 shutdown_connections 2024-07-26T13:41:18.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:18.661+0000 7f04851ef640 1 -- 172.21.15.100:0/537326011 wait complete. 2024-07-26T13:41:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:19 smithi100 ceph-mon[32599]: pgmap v400: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-07-26T13:41:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:19 smithi100 ceph-mon[32599]: scrub queued for path: / 2024-07-26T13:41:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:19 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:41:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:19 smithi100 ceph-mon[32599]: scrub summary: active paths [/] 2024-07-26T13:41:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:19 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:41:19.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:19 smithi167 ceph-mon[37140]: pgmap v400: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-07-26T13:41:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:19 smithi167 ceph-mon[37140]: scrub queued for path: / 2024-07-26T13:41:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:19 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:41:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:19 smithi167 ceph-mon[37140]: scrub summary: active paths [/] 2024-07-26T13:41:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:19 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:41:19.873 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-26T13:41:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:19 smithi138 ceph-mon[36954]: pgmap v400: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-07-26T13:41:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:19 smithi138 ceph-mon[36954]: scrub queued for path: / 2024-07-26T13:41:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:19 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:41:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:19 smithi138 ceph-mon[36954]: scrub summary: active paths [/] 2024-07-26T13:41:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:19 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:41:20.672 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:20 smithi100 ceph-mon[32599]: scrub summary: idle 2024-07-26T13:41:20.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:20 smithi167 ceph-mon[37140]: scrub summary: idle 2024-07-26T13:41:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:20 smithi138 ceph-mon[36954]: scrub summary: idle 2024-07-26T13:41:20.969 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:41:20.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.974+0000 7f5e11569640 1 -- 172.21.15.100:0/3365771992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e0c107df0 msgr2=0x7f5e0c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:20.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.974+0000 7f5e11569640 1 --2- 172.21.15.100:0/3365771992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e0c107df0 0x7f5e0c108200 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5e00009860 tx=0x7f5e0002a340 comp rx=0 tx=0).stop 2024-07-26T13:41:20.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.974+0000 7f5e11569640 1 -- 172.21.15.100:0/3365771992 shutdown_connections 2024-07-26T13:41:20.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.974+0000 7f5e11569640 1 --2- 172.21.15.100:0/3365771992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e0c108740 0x7f5e0c10ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:20.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.974+0000 7f5e11569640 1 --2- 172.21.15.100:0/3365771992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e0c107df0 0x7f5e0c108200 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:20.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.974+0000 7f5e11569640 1 --2- 172.21.15.100:0/3365771992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e0c1024b0 0x7f5e0c1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:20.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.974+0000 7f5e11569640 1 -- 172.21.15.100:0/3365771992 >> 172.21.15.100:0/3365771992 conn(0x7f5e0c079190 msgr2=0x7f5e0c0795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:20.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.976+0000 7f5e11569640 1 -- 172.21.15.100:0/3365771992 shutdown_connections 2024-07-26T13:41:20.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.976+0000 7f5e11569640 1 -- 172.21.15.100:0/3365771992 wait complete. 2024-07-26T13:41:20.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.977+0000 7f5e11569640 1 Processor -- start 2024-07-26T13:41:20.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.978+0000 7f5e11569640 1 -- start start 2024-07-26T13:41:20.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.978+0000 7f5e11569640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e0c1024b0 0x7f5e0c198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:20.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.978+0000 7f5e11569640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e0c107df0 0x7f5e0c1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:20.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.978+0000 7f5e0affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e0c1024b0 0x7f5e0c198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:20.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.979+0000 7f5e0affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e0c1024b0 0x7f5e0c198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35820/0 (socket says 172.21.15.100:35820) 2024-07-26T13:41:20.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.979+0000 7f5e0affd640 1 -- 172.21.15.100:0/4067603707 learned_addr learned my addr 172.21.15.100:0/4067603707 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:20.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.979+0000 7f5e0a7fc640 1 --2- 172.21.15.100:0/4067603707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e0c107df0 0x7f5e0c1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:20.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.979+0000 7f5e11569640 1 --2- 172.21.15.100:0/4067603707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e0c108740 0x7f5e0c19e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:20.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.979+0000 7f5e11569640 1 -- 172.21.15.100:0/4067603707 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c112370 con 0x7f5e0c1024b0 2024-07-26T13:41:20.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.979+0000 7f5e11569640 1 -- 172.21.15.100:0/4067603707 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c1121f0 con 0x7f5e0c108740 2024-07-26T13:41:20.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.979+0000 7f5e11569640 1 -- 172.21.15.100:0/4067603707 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c1124f0 con 0x7f5e0c107df0 2024-07-26T13:41:20.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.979+0000 7f5e0b7fe640 1 --2- 172.21.15.100:0/4067603707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e0c108740 0x7f5e0c19e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:20.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.980+0000 7f5e0affd640 1 -- 172.21.15.100:0/4067603707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e0c107df0 msgr2=0x7f5e0c1992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:20.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.980+0000 7f5e0affd640 1 --2- 172.21.15.100:0/4067603707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e0c107df0 0x7f5e0c1992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:20.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.980+0000 7f5e0affd640 1 -- 172.21.15.100:0/4067603707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e0c108740 msgr2=0x7f5e0c19e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:20.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.980+0000 7f5e0affd640 1 --2- 172.21.15.100:0/4067603707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e0c108740 0x7f5e0c19e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:20.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.980+0000 7f5e0affd640 1 -- 172.21.15.100:0/4067603707 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e0c19ec70 con 0x7f5e0c1024b0 2024-07-26T13:41:20.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.980+0000 7f5e0a7fc640 1 --2- 172.21.15.100:0/4067603707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e0c107df0 0x7f5e0c1992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:41:20.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.980+0000 7f5e0affd640 1 --2- 172.21.15.100:0/4067603707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e0c1024b0 0x7f5e0c198d00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc00e9c0 tx=0x7f5dfc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:20.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.980+0000 7f5df3fff640 1 -- 172.21.15.100:0/4067603707 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc00ccc0 con 0x7f5e0c1024b0 2024-07-26T13:41:20.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.980+0000 7f5df3fff640 1 -- 172.21.15.100:0/4067603707 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dfc0054d0 con 0x7f5e0c1024b0 2024-07-26T13:41:20.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.980+0000 7f5df3fff640 1 -- 172.21.15.100:0/4067603707 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc010430 con 0x7f5e0c1024b0 2024-07-26T13:41:20.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.980+0000 7f5e11569640 1 -- 172.21.15.100:0/4067603707 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e0c19ef60 con 0x7f5e0c1024b0 2024-07-26T13:41:20.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.980+0000 7f5e11569640 1 -- 172.21.15.100:0/4067603707 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e0c06a890 con 0x7f5e0c1024b0 2024-07-26T13:41:20.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.984+0000 7f5e17aed740 1 Processor -- start 2024-07-26T13:41:20.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.984+0000 7f5e17aed740 1 -- start start 2024-07-26T13:41:20.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.984+0000 7f5e17aed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559aa5e02850 0x559aa61a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:20.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.984+0000 7f5e17aed740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559aa5dd9990 0x559aa61a8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:20.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.984+0000 7f5e17aed740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559aa5de1b70 0x559aa61a8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:20.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.985+0000 7f5e17aed740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x559aa611d690 con 0x559aa5de1b70 2024-07-26T13:41:20.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.985+0000 7f5e17aed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559aa610cb90 con 0x559aa5e02850 2024-07-26T13:41:20.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.985+0000 7f5e17aed740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559aa6110bd0 con 0x559aa5dd9990 2024-07-26T13:41:20.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.985+0000 7f5e0b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559aa5e02850 0x559aa61a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:20.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.985+0000 7f5e0b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559aa5e02850 0x559aa61a8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:60430/0 (socket says 172.21.15.100:60430) 2024-07-26T13:41:20.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.985+0000 7f5e0b7fe640 1 -- 172.21.15.100:0/2561988133 learned_addr learned my addr 172.21.15.100:0/2561988133 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:20.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.985+0000 7f5e0a7fc640 1 --2- 172.21.15.100:0/2561988133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559aa5de1b70 0x559aa61a8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:20.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.985+0000 7f5e0b7fe640 1 -- 172.21.15.100:0/2561988133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559aa5dd9990 msgr2=0x559aa61a8a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-26T13:41:20.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.985+0000 7f5e0b7fe640 1 --2- 172.21.15.100:0/2561988133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559aa5dd9990 0x559aa61a8a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:20.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.985+0000 7f5e0b7fe640 1 -- 172.21.15.100:0/2561988133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559aa5de1b70 msgr2=0x559aa61a8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:20.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.985+0000 7f5e0b7fe640 1 --2- 172.21.15.100:0/2561988133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559aa5de1b70 0x559aa61a8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:20.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.985+0000 7f5e0b7fe640 1 -- 172.21.15.100:0/2561988133 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559aa6098a10 con 0x559aa5e02850 2024-07-26T13:41:20.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.986+0000 7f5e0b7fe640 1 --2- 172.21.15.100:0/2561988133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559aa5e02850 0x559aa61a8540 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5e04009630 tx=0x7f5e0400c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=373381b9d04669b0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:20.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.986+0000 7f5e0a7fc640 1 --2- 172.21.15.100:0/2561988133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559aa5de1b70 0x559aa61a8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:41:20.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.986+0000 7f5df17fa640 1 -- 172.21.15.100:0/2561988133 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e04004d20 con 0x559aa5e02850 2024-07-26T13:41:20.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.986+0000 7f5df3fff640 1 -- 172.21.15.100:0/4067603707 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f5dfc004f80 con 0x7f5e0c1024b0 2024-07-26T13:41:20.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.986+0000 7f5df17fa640 1 -- 172.21.15.100:0/2561988133 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e0400c6a0 con 0x559aa5e02850 2024-07-26T13:41:20.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.986+0000 7f5df17fa640 1 -- 172.21.15.100:0/2561988133 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e04005280 con 0x559aa5e02850 2024-07-26T13:41:20.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.987+0000 7f5df3fff640 1 --2- 172.21.15.100:0/4067603707 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5df40b3d40 0x7f5df40b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:20.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.987+0000 7f5e17aed740 1 -- 172.21.15.100:0/2561988133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559aa5e02850 msgr2=0x559aa61a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:20.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.987+0000 7f5e17aed740 1 --2- 172.21.15.100:0/2561988133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559aa5e02850 0x559aa61a8540 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5e04009630 tx=0x7f5e0400c3d0 comp rx=0 tx=0).stop 2024-07-26T13:41:20.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.988+0000 7f5df3fff640 1 -- 172.21.15.100:0/4067603707 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f5dfc0e4450 con 0x7f5e0c1024b0 2024-07-26T13:41:20.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.988+0000 7f5e17aed740 1 -- 172.21.15.100:0/2561988133 shutdown_connections 2024-07-26T13:41:20.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.988+0000 7f5e0affd640 1 --2- 172.21.15.100:0/4067603707 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5df40b3d40 0x7f5df40b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:20.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.988+0000 7f5e17aed740 1 --2- 172.21.15.100:0/2561988133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559aa5de1b70 0x559aa61a8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:20.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.988+0000 7f5e17aed740 1 --2- 172.21.15.100:0/2561988133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559aa5dd9990 0x559aa61a8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:20.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.988+0000 7f5e17aed740 1 --2- 172.21.15.100:0/2561988133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559aa5e02850 0x559aa61a8540 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:20.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.988+0000 7f5e17aed740 1 -- 172.21.15.100:0/2561988133 >> 172.21.15.100:0/2561988133 conn(0x559aa5df6c40 msgr2=0x559aa61a6430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:20.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.989+0000 7f5e17aed740 1 -- 172.21.15.100:0/2561988133 shutdown_connections 2024-07-26T13:41:20.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.989+0000 7f5e17aed740 1 -- 172.21.15.100:0/2561988133 wait complete. 2024-07-26T13:41:20.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.989+0000 7f5e0affd640 1 --2- 172.21.15.100:0/4067603707 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5df40b3d40 0x7f5df40b6250 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x559aa61a71c0 tx=0x7f5dfc00efb0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:20.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.990+0000 7f5e17aed740 1 -- start start 2024-07-26T13:41:20.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.990+0000 7f5e17aed740 1 Processor -- start 2024-07-26T13:41:20.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.990+0000 7f5e17aed740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559aa5dd9990 0x559aa61bc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:20.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.990+0000 7f5e17aed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559aa5de1b70 0x559aa61bc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:20.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.990+0000 7f5e17aed740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559aa5e02850 0x559aa61c8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:20.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.990+0000 7f5e17aed740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x559aa5f707b0 con 0x559aa5dd9990 2024-07-26T13:41:20.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.991+0000 7f5e17aed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559aa60af210 con 0x559aa5de1b70 2024-07-26T13:41:20.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.991+0000 7f5e17aed740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559aa61bf450 con 0x559aa5e02850 2024-07-26T13:41:20.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.991+0000 7f5e17aed740 10 client.0 _ll_register_callbacks cb 0x559aa60902c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:41:20.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.991+0000 7f5e0b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559aa5dd9990 0x559aa61bc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:20.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.991+0000 7f5e0b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559aa5dd9990 0x559aa61bc3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35836/0 (socket says 172.21.15.100:35836) 2024-07-26T13:41:20.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.991+0000 7f5e0b7fe640 1 -- 172.21.15.100:0/2971383147 learned_addr learned my addr 172.21.15.100:0/2971383147 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:20.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.991+0000 7f5e0a7fc640 1 --2- 172.21.15.100:0/2971383147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559aa5de1b70 0x559aa61bc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:20.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.992+0000 7f5e0b7fe640 1 -- 172.21.15.100:0/2971383147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559aa5e02850 msgr2=0x559aa61c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:20.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.992+0000 7f5e0b7fe640 1 --2- 172.21.15.100:0/2971383147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559aa5e02850 0x559aa61c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:20.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.992+0000 7f5e0b7fe640 1 -- 172.21.15.100:0/2971383147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559aa5de1b70 msgr2=0x559aa61bc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:20.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.992+0000 7f5e0b7fe640 1 --2- 172.21.15.100:0/2971383147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559aa5de1b70 0x559aa61bc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:20.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.992+0000 7f5e0b7fe640 1 -- 172.21.15.100:0/2971383147 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559aa60c0710 con 0x559aa5dd9990 2024-07-26T13:41:20.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.992+0000 7f5e0a7fc640 1 --2- 172.21.15.100:0/2971383147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559aa5de1b70 0x559aa61bc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:41:20.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.992+0000 7f5e0b7fe640 1 --2- 172.21.15.100:0/2971383147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559aa5dd9990 0x559aa61bc3d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5e04010a60 tx=0x7f5e04010f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=586aac2624eed7ec server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:20.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.992+0000 7f5deeff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:41:20.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.992+0000 7f5deeff5640 1 -- 172.21.15.100:0/2971383147 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e04029060 con 0x559aa5dd9990 2024-07-26T13:41:20.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.992+0000 7f5deeff5640 1 -- 172.21.15.100:0/2971383147 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f5e0400c960 con 0x559aa5dd9990 2024-07-26T13:41:20.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.992+0000 7f5e17aed740 1 -- 172.21.15.100:0/2971383147 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559aa611f5e0 con 0x559aa5dd9990 2024-07-26T13:41:20.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.993+0000 7f5deeff5640 1 -- 172.21.15.100:0/2971383147 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5e0400cb60 con 0x559aa5dd9990 2024-07-26T13:41:20.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.993+0000 7f5e17aed740 10 client.15750 fetch_fsmap learned FSMap version 26 2024-07-26T13:41:20.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.993+0000 7f5e17aed740 1 -- 172.21.15.100:0/2971383147 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559aa5e7cba0 con 0x559aa5dd9990 2024-07-26T13:41:20.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.993+0000 7f5deeff5640 1 -- 172.21.15.100:0/2971383147 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e04024920 con 0x559aa5dd9990 2024-07-26T13:41:20.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.993+0000 7f5deeff5640 1 -- 172.21.15.100:0/2971383147 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f5e0402d5e0 con 0x559aa5dd9990 2024-07-26T13:41:20.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.993+0000 7f5e17aed740 10 client.15750 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:41:20.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.993+0000 7f5e17aed740 10 client.15750 resolve_mds: validated gid 24607 aka daemon mds.i 2024-07-26T13:41:20.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.994+0000 7f5e17aed740 20 client.15750 populate_metadata read hostname 'smithi100' 2024-07-26T13:41:20.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.994+0000 7f5e17aed740 1 --2- 172.21.15.100:0/2971383147 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559aa5dc11d0 0x559aa61ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:20.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.994+0000 7f5e17aed740 4 client.15750 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:41:20.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.994+0000 7f5e17aed740 1 -- 172.21.15.100:0/2971383147 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559aa611f5e0 con 0x559aa5dc11d0 2024-07-26T13:41:20.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.994+0000 7f5e0a7fc640 1 --2- 172.21.15.100:0/2971383147 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559aa5dc11d0 0x559aa61ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:20.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.995+0000 7f5e0a7fc640 1 --2- 172.21.15.100:0/2971383147 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559aa5dc11d0 0x559aa61ce3a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7daf4a4b87b1e57d server_cookie=aaedbfbf11dae6bb in_seq=0 out_seq=0 2024-07-26T13:41:20.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.995+0000 7f5deeff5640 10 client.15750 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:21.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.996+0000 7f5deeff5640 1 -- 172.21.15.100:0/2971383147 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559aa611f5e0 con 0x559aa5dc11d0 2024-07-26T13:41:21.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.996+0000 7f5deeff5640 10 client.15750 handle_command_reply: tid=0 2024-07-26T13:41:21.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.996+0000 7f5e17aed740 1 client.15750 shutdown 2024-07-26T13:41:21.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.996+0000 7f5e17aed740 1 -- 172.21.15.100:0/2971383147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559aa5dd9990 msgr2=0x559aa61bc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:21.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.996+0000 7f5e17aed740 1 --2- 172.21.15.100:0/2971383147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559aa5dd9990 0x559aa61bc3d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5e04010a60 tx=0x7f5e04010f50 comp rx=0 tx=0).stop 2024-07-26T13:41:21.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.997+0000 7f5e17aed740 1 -- 172.21.15.100:0/2971383147 shutdown_connections 2024-07-26T13:41:21.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.997+0000 7f5e17aed740 1 --2- 172.21.15.100:0/2971383147 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559aa5dc11d0 0x559aa61ce3a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:21.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.997+0000 7f5e17aed740 1 --2- 172.21.15.100:0/2971383147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559aa5e02850 0x559aa61c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:21.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.997+0000 7f5e17aed740 1 --2- 172.21.15.100:0/2971383147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559aa5de1b70 0x559aa61bc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:21.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.997+0000 7f5e17aed740 1 --2- 172.21.15.100:0/2971383147 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559aa5dd9990 0x559aa61bc3d0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:21.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.997+0000 7f5e17aed740 1 -- 172.21.15.100:0/2971383147 >> 172.21.15.100:0/2971383147 conn(0x559aa5df6c40 msgr2=0x559aa61a4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:21.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.997+0000 7f5deeff5640 0 client.15750 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:21.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.997+0000 7f5e17aed740 1 -- 172.21.15.100:0/2971383147 shutdown_connections 2024-07-26T13:41:21.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.997+0000 7f5e17aed740 1 -- 172.21.15.100:0/2971383147 wait complete. 2024-07-26T13:41:21.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:20.997+0000 7f5e17aed740 20 client.15750 trim_cache size 0 max 16384 2024-07-26T13:41:21.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.030+0000 7f5e17aed740 1 Processor -- start 2024-07-26T13:41:21.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.030+0000 7f5e17aed740 1 -- start start 2024-07-26T13:41:21.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.031+0000 7f5e17aed740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559aa5dd9990 0x559aa61bdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:21.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.031+0000 7f5e17aed740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559aa5de1b70 0x559aa61ad600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:21.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.031+0000 7f5e17aed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559aa5e02850 0x559aa61b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:21.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.031+0000 7f5e17aed740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c1af7d0 con 0x559aa5de1b70 2024-07-26T13:41:21.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.031+0000 7f5e17aed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c1ad0a0 con 0x559aa5e02850 2024-07-26T13:41:21.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.031+0000 7f5e17aed740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c1aaef0 con 0x559aa5dd9990 2024-07-26T13:41:21.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.031+0000 7f5e0b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559aa5e02850 0x559aa61b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:21.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.031+0000 7f5e0b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559aa5e02850 0x559aa61b7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:60446/0 (socket says 172.21.15.100:60446) 2024-07-26T13:41:21.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.031+0000 7f5e0a7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559aa5de1b70 0x559aa61ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:21.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.031+0000 7f5e0b7fe640 1 -- 172.21.15.100:0/3383043235 learned_addr learned my addr 172.21.15.100:0/3383043235 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:21.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.031+0000 7f5e0b7fe640 1 --2- 172.21.15.100:0/3383043235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559aa5dd9990 0x559aa61bdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:21.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.031+0000 7f5e0b7fe640 1 -- 172.21.15.100:0/3383043235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559aa5dd9990 msgr2=0x559aa61bdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:21.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.031+0000 7f5e0b7fe640 1 --2- 172.21.15.100:0/3383043235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559aa5dd9990 0x559aa61bdf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:21.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.031+0000 7f5e0b7fe640 1 -- 172.21.15.100:0/3383043235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559aa5de1b70 msgr2=0x559aa61ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:21.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.031+0000 7f5e0b7fe640 1 --2- 172.21.15.100:0/3383043235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559aa5de1b70 0x559aa61ad600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:21.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.031+0000 7f5e0b7fe640 1 -- 172.21.15.100:0/3383043235 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559aa61ae140 con 0x559aa5e02850 2024-07-26T13:41:21.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.032+0000 7f5e0b7fe640 1 --2- 172.21.15.100:0/3383043235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559aa5e02850 0x559aa61b7a40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x559aa61a6e10 tx=0x7f5e040107c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b07a4e9f99b3015 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:21.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.032+0000 7f5df0ff9640 1 -- 172.21.15.100:0/3383043235 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e04029070 con 0x559aa5e02850 2024-07-26T13:41:21.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.032+0000 7f5df0ff9640 1 -- 172.21.15.100:0/3383043235 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e0400c760 con 0x559aa5e02850 2024-07-26T13:41:21.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.032+0000 7f5df0ff9640 1 -- 172.21.15.100:0/3383043235 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e04005250 con 0x559aa5e02850 2024-07-26T13:41:21.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.033+0000 7f5e17aed740 1 -- 172.21.15.100:0/3383043235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559aa5e02850 msgr2=0x559aa61b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:21.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.033+0000 7f5e17aed740 1 --2- 172.21.15.100:0/3383043235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559aa5e02850 0x559aa61b7a40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x559aa61a6e10 tx=0x7f5e040107c0 comp rx=0 tx=0).stop 2024-07-26T13:41:21.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.033+0000 7f5e17aed740 1 -- 172.21.15.100:0/3383043235 shutdown_connections 2024-07-26T13:41:21.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.033+0000 7f5e17aed740 1 --2- 172.21.15.100:0/3383043235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559aa5e02850 0x559aa61b7a40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:21.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.033+0000 7f5e17aed740 1 --2- 172.21.15.100:0/3383043235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559aa5de1b70 0x559aa61ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:21.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.033+0000 7f5e17aed740 1 --2- 172.21.15.100:0/3383043235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559aa5dd9990 0x559aa61bdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:21.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.033+0000 7f5e17aed740 1 -- 172.21.15.100:0/3383043235 >> 172.21.15.100:0/3383043235 conn(0x559aa5df6c40 msgr2=0x559aa61a5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:21.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.033+0000 7f5e17aed740 1 -- 172.21.15.100:0/3383043235 shutdown_connections 2024-07-26T13:41:21.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.034+0000 7f5e17aed740 1 -- 172.21.15.100:0/3383043235 wait complete. 2024-07-26T13:41:21.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.034+0000 7f5e17aed740 1 -- start start 2024-07-26T13:41:21.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.034+0000 7f5e17aed740 1 Processor -- start 2024-07-26T13:41:21.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.035+0000 7f5e17aed740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559aa5dd9990 0x559aa61c0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:21.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.035+0000 7f5e0b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559aa5dd9990 0x559aa61c0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:21.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.035+0000 7f5e0b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559aa5dd9990 0x559aa61c0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35856/0 (socket says 172.21.15.100:35856) 2024-07-26T13:41:21.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.035+0000 7f5e0b7fe640 1 -- 172.21.15.100:0/2299293286 learned_addr learned my addr 172.21.15.100:0/2299293286 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:21.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.035+0000 7f5e17aed740 1 --2- 172.21.15.100:0/2299293286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559aa5de1b70 0x559aa61c15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:21.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.036+0000 7f5e17aed740 1 --2- 172.21.15.100:0/2299293286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559aa5e02850 0x559aa622f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:21.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.036+0000 7f5e17aed740 1 -- 172.21.15.100:0/2299293286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c1ab560 con 0x559aa5dd9990 2024-07-26T13:41:21.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.036+0000 7f5e17aed740 1 -- 172.21.15.100:0/2299293286 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c1ac9d0 con 0x559aa5de1b70 2024-07-26T13:41:21.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.036+0000 7f5e17aed740 1 -- 172.21.15.100:0/2299293286 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c1abc30 con 0x559aa5e02850 2024-07-26T13:41:21.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.036+0000 7f5e0a7fc640 1 --2- 172.21.15.100:0/2299293286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559aa5de1b70 0x559aa61c15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:21.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.036+0000 7f5e0a7fc640 1 -- 172.21.15.100:0/2299293286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559aa5e02850 msgr2=0x559aa622f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:21.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.036+0000 7f5e0a7fc640 1 --2- 172.21.15.100:0/2299293286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559aa5e02850 0x559aa622f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:21.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.036+0000 7f5e0a7fc640 1 -- 172.21.15.100:0/2299293286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559aa5dd9990 msgr2=0x559aa61c0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:21.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.036+0000 7f5e0a7fc640 1 --2- 172.21.15.100:0/2299293286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559aa5dd9990 0x559aa61c0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:21.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.036+0000 7f5e0a7fc640 1 -- 172.21.15.100:0/2299293286 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559aa6230030 con 0x559aa5de1b70 2024-07-26T13:41:21.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.036+0000 7f5e0b7fe640 1 --2- 172.21.15.100:0/2299293286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559aa5dd9990 0x559aa61c0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:41:21.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.037+0000 7f5e0a7fc640 1 --2- 172.21.15.100:0/2299293286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559aa5de1b70 0x559aa61c15c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5e0002a860 tx=0x7f5e00004130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4ef538124ebdca9 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:21.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.036+0000 7f5e17aed740 10 client.0 _ll_register_callbacks cb 0x7f5e04009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:41:21.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.037+0000 7f5e17aed740 1 -- 172.21.15.100:0/2299293286 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e0400c760 con 0x559aa5de1b70 2024-07-26T13:41:21.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.037+0000 7f5deeff5640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-07-26T13:41:21.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.037+0000 7f5deeff5640 1 -- 172.21.15.100:0/2299293286 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0004d060 con 0x559aa5de1b70 2024-07-26T13:41:21.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.037+0000 7f5deeff5640 1 -- 172.21.15.100:0/2299293286 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f5e00030450 con 0x559aa5de1b70 2024-07-26T13:41:21.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.037+0000 7f5deeff5640 1 -- 172.21.15.100:0/2299293286 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5e00037040 con 0x559aa5de1b70 2024-07-26T13:41:21.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.037+0000 7f5e17aed740 10 client.25220 fetch_fsmap learned FSMap version 26 2024-07-26T13:41:21.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.037+0000 7f5e17aed740 1 -- 172.21.15.100:0/2299293286 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({fsmap=26}) -- 0x559aa62372a0 con 0x559aa5de1b70 2024-07-26T13:41:21.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.038+0000 7f5deeff5640 1 -- 172.21.15.100:0/2299293286 <== mon.2 v2:172.21.15.138:3300/0 4 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f5e00030650 con 0x559aa5de1b70 2024-07-26T13:41:21.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.038+0000 7f5e17aed740 10 client.25220 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:41:21.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.038+0000 7f5e17aed740 10 client.25220 resolve_mds: validated gid 24607 aka daemon mds.i 2024-07-26T13:41:21.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.038+0000 7f5e17aed740 20 client.25220 populate_metadata read hostname 'smithi100' 2024-07-26T13:41:21.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.038+0000 7f5e17aed740 1 --2- 172.21.15.100:0/2299293286 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559aa5d96580 0x559aa6239910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:21.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.038+0000 7f5e17aed740 4 client.25220 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}] 2024-07-26T13:41:21.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.038+0000 7f5e17aed740 1 -- 172.21.15.100:0/2299293286 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}) -- 0x559aa611b080 con 0x559aa5d96580 2024-07-26T13:41:21.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.038+0000 7f5e0b7fe640 1 --2- 172.21.15.100:0/2299293286 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559aa5d96580 0x559aa6239910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:21.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.039+0000 7f5deeff5640 10 client.25220 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:21.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.039+0000 7f5e0b7fe640 1 --2- 172.21.15.100:0/2299293286 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559aa5d96580 0x559aa6239910 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76e580794f1711d3 server_cookie=95eb26d2aad99751 in_seq=0 out_seq=0 2024-07-26T13:41:21.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.048+0000 7f5deeff5640 1 -- 172.21.15.100:0/2299293286 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x559aa611b080 con 0x559aa5d96580 2024-07-26T13:41:21.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.048+0000 7f5deeff5640 10 client.25220 handle_command_reply: tid=0 2024-07-26T13:41:21.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.048+0000 7f5e17aed740 1 client.25220 shutdown 2024-07-26T13:41:21.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.048+0000 7f5e17aed740 1 -- 172.21.15.100:0/2299293286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559aa5de1b70 msgr2=0x559aa61c15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:21.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.048+0000 7f5e17aed740 1 --2- 172.21.15.100:0/2299293286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559aa5de1b70 0x559aa61c15c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5e0002a860 tx=0x7f5e00004130 comp rx=0 tx=0).stop 2024-07-26T13:41:21.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.048+0000 7f5e17aed740 1 -- 172.21.15.100:0/2299293286 shutdown_connections 2024-07-26T13:41:21.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.048+0000 7f5e17aed740 1 --2- 172.21.15.100:0/2299293286 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559aa5d96580 0x559aa6239910 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:21.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.048+0000 7f5e17aed740 1 --2- 172.21.15.100:0/2299293286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559aa5e02850 0x559aa622f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:21.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.048+0000 7f5e17aed740 1 --2- 172.21.15.100:0/2299293286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559aa5de1b70 0x559aa61c15c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:21.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.048+0000 7f5e17aed740 1 --2- 172.21.15.100:0/2299293286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559aa5dd9990 0x559aa61c0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:21.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.048+0000 7f5e17aed740 1 -- 172.21.15.100:0/2299293286 >> 172.21.15.100:0/2299293286 conn(0x559aa5df6c40 msgr2=0x559aa61a6210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:21.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.048+0000 7f5deeff5640 0 client.25220 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:21.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.048+0000 7f5e17aed740 1 -- 172.21.15.100:0/2299293286 shutdown_connections 2024-07-26T13:41:21.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.048+0000 7f5e17aed740 1 -- 172.21.15.100:0/2299293286 wait complete. 2024-07-26T13:41:21.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.049+0000 7f5e17aed740 20 client.25220 trim_cache size 0 max 16384 2024-07-26T13:41:21.051 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:41:21.051 INFO:teuthology.orchestra.run.smithi100.stdout: "epoch": 26, 2024-07-26T13:41:21.051 INFO:teuthology.orchestra.run.smithi100.stdout: "leader": 24607, 2024-07-26T13:41:21.051 INFO:teuthology.orchestra.run.smithi100.stdout: "set_version": 2, 2024-07-26T13:41:21.051 INFO:teuthology.orchestra.run.smithi100.stdout: "sets": { 2024-07-26T13:41:21.051 INFO:teuthology.orchestra.run.smithi100.stdout: "28b446d7": { 2024-07-26T13:41:21.051 INFO:teuthology.orchestra.run.smithi100.stdout: "version": 2, 2024-07-26T13:41:21.051 INFO:teuthology.orchestra.run.smithi100.stdout: "age_ref": 0.0, 2024-07-26T13:41:21.052 INFO:teuthology.orchestra.run.smithi100.stdout: "state": { 2024-07-26T13:41:21.052 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "QUIESCED", 2024-07-26T13:41:21.052 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 0.0 2024-07-26T13:41:21.052 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-07-26T13:41:21.052 INFO:teuthology.orchestra.run.smithi100.stdout: "timeout": 90.0, 2024-07-26T13:41:21.052 INFO:teuthology.orchestra.run.smithi100.stdout: "expiration": 142.6, 2024-07-26T13:41:21.052 INFO:teuthology.orchestra.run.smithi100.stdout: "members": { 2024-07-26T13:41:21.052 INFO:teuthology.orchestra.run.smithi100.stdout: "file:/": { 2024-07-26T13:41:21.052 INFO:teuthology.orchestra.run.smithi100.stdout: "excluded": false, 2024-07-26T13:41:21.052 INFO:teuthology.orchestra.run.smithi100.stdout: "state": { 2024-07-26T13:41:21.052 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "QUIESCED", 2024-07-26T13:41:21.052 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 0.0 2024-07-26T13:41:21.052 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:41:21.052 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:41:21.053 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:41:21.053 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:41:21.053 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:41:21.053 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:41:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.049+0000 7f5e11569640 1 -- 172.21.15.100:0/4067603707 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5df40b3d40 msgr2=0x7f5df40b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.049+0000 7f5e11569640 1 --2- 172.21.15.100:0/4067603707 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5df40b3d40 0x7f5df40b6250 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x559aa61a71c0 tx=0x7f5dfc00efb0 comp rx=0 tx=0).stop 2024-07-26T13:41:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.049+0000 7f5e11569640 1 -- 172.21.15.100:0/4067603707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e0c1024b0 msgr2=0x7f5e0c198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:21.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.049+0000 7f5e11569640 1 --2- 172.21.15.100:0/4067603707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e0c1024b0 0x7f5e0c198d00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc00e9c0 tx=0x7f5dfc00ee80 comp rx=0 tx=0).stop 2024-07-26T13:41:21.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.050+0000 7f5e11569640 1 -- 172.21.15.100:0/4067603707 shutdown_connections 2024-07-26T13:41:21.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.050+0000 7f5e11569640 1 --2- 172.21.15.100:0/4067603707 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5df40b3d40 0x7f5df40b6250 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:21.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.050+0000 7f5e11569640 1 --2- 172.21.15.100:0/4067603707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e0c108740 0x7f5e0c19e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:21.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.050+0000 7f5e11569640 1 --2- 172.21.15.100:0/4067603707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e0c107df0 0x7f5e0c1992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:21.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.050+0000 7f5e11569640 1 --2- 172.21.15.100:0/4067603707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e0c1024b0 0x7f5e0c198d00 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:21.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.050+0000 7f5e11569640 1 -- 172.21.15.100:0/4067603707 >> 172.21.15.100:0/4067603707 conn(0x7f5e0c079190 msgr2=0x7f5e0c103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:21.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.050+0000 7f5e11569640 1 -- 172.21.15.100:0/4067603707 shutdown_connections 2024-07-26T13:41:21.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:21.050+0000 7f5e11569640 1 -- 172.21.15.100:0/4067603707 wait complete. 2024-07-26T13:41:21.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:21 smithi167 ceph-mon[37140]: pgmap v401: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-07-26T13:41:21.922 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:21 smithi100 ceph-mon[32599]: pgmap v401: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-07-26T13:41:21.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:21 smithi138 ceph-mon[36954]: pgmap v401: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-07-26T13:41:21.964 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '28b446d7', quiesce took 10.7 seconds. Will release after: 11.850000000000001 2024-07-26T13:41:23.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.670+0000 7f5975955640 1 -- 172.21.15.100:0/4240972466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5970109830 msgr2=0x7f5970110060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:23.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.670+0000 7f5975955640 1 --2- 172.21.15.100:0/4240972466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5970109830 0x7f5970110060 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5968009ca0 tx=0x7f596802a8c0 comp rx=0 tx=0).stop 2024-07-26T13:41:23.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.670+0000 7f5975955640 1 -- 172.21.15.100:0/4240972466 shutdown_connections 2024-07-26T13:41:23.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.670+0000 7f5975955640 1 --2- 172.21.15.100:0/4240972466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5970109830 0x7f5970110060 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:23.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.670+0000 7f5975955640 1 --2- 172.21.15.100:0/4240972466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5970108ee0 0x7f59701092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:23.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.670+0000 7f5975955640 1 --2- 172.21.15.100:0/4240972466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59701019a0 0x7f5970101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:23.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.670+0000 7f5975955640 1 -- 172.21.15.100:0/4240972466 >> 172.21.15.100:0/4240972466 conn(0x7f5970077690 msgr2=0x7f5970077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:23.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.671+0000 7f5975955640 1 -- 172.21.15.100:0/4240972466 shutdown_connections 2024-07-26T13:41:23.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.671+0000 7f5975955640 1 -- 172.21.15.100:0/4240972466 wait complete. 2024-07-26T13:41:23.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.672+0000 7f5975955640 1 Processor -- start 2024-07-26T13:41:23.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.672+0000 7f5975955640 1 -- start start 2024-07-26T13:41:23.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.672+0000 7f5975955640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59701019a0 0x7f5970199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:23.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.672+0000 7f5975955640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5970108ee0 0x7f597019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:23.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.672+0000 7f5975955640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5970109830 0x7f597019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:23.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.673+0000 7f596effd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59701019a0 0x7f5970199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:23.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.673+0000 7f596effd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59701019a0 0x7f5970199f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:52450/0 (socket says 172.21.15.100:52450) 2024-07-26T13:41:23.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.673+0000 7f596f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5970109830 0x7f597019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:23.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.673+0000 7f596effd640 1 -- 172.21.15.100:0/2798079589 learned_addr learned my addr 172.21.15.100:0/2798079589 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:23.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.673+0000 7f5975955640 1 -- 172.21.15.100:0/2798079589 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5970113500 con 0x7f5970109830 2024-07-26T13:41:23.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.673+0000 7f5975955640 1 -- 172.21.15.100:0/2798079589 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5970113380 con 0x7f5970108ee0 2024-07-26T13:41:23.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.673+0000 7f5975955640 1 -- 172.21.15.100:0/2798079589 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5970113680 con 0x7f59701019a0 2024-07-26T13:41:23.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.673+0000 7f596e7fc640 1 --2- 172.21.15.100:0/2798079589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5970108ee0 0x7f597019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:23.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.673+0000 7f596effd640 1 -- 172.21.15.100:0/2798079589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5970108ee0 msgr2=0x7f597019a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:23.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.673+0000 7f596effd640 1 --2- 172.21.15.100:0/2798079589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5970108ee0 0x7f597019a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:23.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.673+0000 7f596effd640 1 -- 172.21.15.100:0/2798079589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5970109830 msgr2=0x7f597019f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:23.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.673+0000 7f596effd640 1 --2- 172.21.15.100:0/2798079589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5970109830 0x7f597019f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:23.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.673+0000 7f596effd640 1 -- 172.21.15.100:0/2798079589 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f597019fe80 con 0x7f59701019a0 2024-07-26T13:41:23.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.673+0000 7f596f7fe640 1 --2- 172.21.15.100:0/2798079589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5970109830 0x7f597019f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:41:23.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.673+0000 7f596effd640 1 --2- 172.21.15.100:0/2798079589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59701019a0 0x7f5970199f10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f596000d950 tx=0x7f596000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:23.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.674+0000 7f5974953640 1 -- 172.21.15.100:0/2798079589 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5960017070 con 0x7f59701019a0 2024-07-26T13:41:23.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.674+0000 7f5975955640 1 -- 172.21.15.100:0/2798079589 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59701a0170 con 0x7f59701019a0 2024-07-26T13:41:23.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.674+0000 7f5974953640 1 -- 172.21.15.100:0/2798079589 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5960009c30 con 0x7f59701019a0 2024-07-26T13:41:23.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.674+0000 7f5974953640 1 -- 172.21.15.100:0/2798079589 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596000f620 con 0x7f59701019a0 2024-07-26T13:41:23.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.674+0000 7f5975955640 1 -- 172.21.15.100:0/2798079589 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59701a68d0 con 0x7f59701019a0 2024-07-26T13:41:23.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.678+0000 7f5974953640 1 -- 172.21.15.100:0/2798079589 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f5960010460 con 0x7f59701019a0 2024-07-26T13:41:23.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.678+0000 7f597bed9740 1 Processor -- start 2024-07-26T13:41:23.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.678+0000 7f597bed9740 1 -- start start 2024-07-26T13:41:23.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.679+0000 7f597bed9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aca558f070 0x55aca5934550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:23.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.679+0000 7f597bed9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aca55661b0 0x55aca5934a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:23.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.679+0000 7f597bed9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aca556e390 0x55aca5934fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:23.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.679+0000 7f597bed9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55aca58aab90 con 0x55aca55661b0 2024-07-26T13:41:23.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.679+0000 7f597bed9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55aca589a090 con 0x55aca558f070 2024-07-26T13:41:23.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.679+0000 7f597bed9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55aca589e0d0 con 0x55aca556e390 2024-07-26T13:41:23.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.679+0000 7f596e7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aca556e390 0x55aca5934fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:23.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.679+0000 7f596e7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aca556e390 0x55aca5934fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:52464/0 (socket says 172.21.15.100:52464) 2024-07-26T13:41:23.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.679+0000 7f596e7fc640 1 -- 172.21.15.100:0/4066141650 learned_addr learned my addr 172.21.15.100:0/4066141650 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:23.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.679+0000 7f596f7fe640 1 --2- 172.21.15.100:0/4066141650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aca558f070 0x55aca5934550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:23.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.679+0000 7f596effd640 1 --2- 172.21.15.100:0/4066141650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aca55661b0 0x55aca5934a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:23.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.680+0000 7f596e7fc640 1 -- 172.21.15.100:0/4066141650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aca558f070 msgr2=0x55aca5934550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:23.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.680+0000 7f596e7fc640 1 --2- 172.21.15.100:0/4066141650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aca558f070 0x55aca5934550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:23.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.680+0000 7f596e7fc640 1 -- 172.21.15.100:0/4066141650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aca55661b0 msgr2=0x55aca5934a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:23.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.680+0000 7f5974953640 1 --2- 172.21.15.100:0/2798079589 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f59580b3dc0 0x7f59580b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:23.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.680+0000 7f596e7fc640 1 --2- 172.21.15.100:0/4066141650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aca55661b0 0x55aca5934a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:23.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.680+0000 7f596e7fc640 1 -- 172.21.15.100:0/4066141650 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aca5840b70 con 0x55aca556e390 2024-07-26T13:41:23.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.680+0000 7f596effd640 1 --2- 172.21.15.100:0/4066141650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aca55661b0 0x55aca5934a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:41:23.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.680+0000 7f596f7fe640 1 --2- 172.21.15.100:0/2798079589 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f59580b3dc0 0x7f59580b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:23.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.680+0000 7f5974953640 1 -- 172.21.15.100:0/2798079589 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f59600e3f30 con 0x7f59701019a0 2024-07-26T13:41:23.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.681+0000 7f596e7fc640 1 --2- 172.21.15.100:0/4066141650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aca556e390 0x55aca5934fd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f59640045e0 tx=0x7f5964004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d45ddca16b1d040 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:23.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.681+0000 7f5955ffb640 1 -- 172.21.15.100:0/4066141650 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5964003a90 con 0x55aca556e390 2024-07-26T13:41:23.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.681+0000 7f596f7fe640 1 --2- 172.21.15.100:0/2798079589 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f59580b3dc0 0x7f59580b62d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5968004af0 tx=0x7f5968005d80 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:23.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.681+0000 7f5955ffb640 1 -- 172.21.15.100:0/4066141650 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5964015070 con 0x55aca556e390 2024-07-26T13:41:23.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.682+0000 7f5955ffb640 1 -- 172.21.15.100:0/4066141650 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5964004c80 con 0x55aca556e390 2024-07-26T13:41:23.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.683+0000 7f597bed9740 1 -- 172.21.15.100:0/4066141650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aca556e390 msgr2=0x55aca5934fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:23.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.683+0000 7f597bed9740 1 --2- 172.21.15.100:0/4066141650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aca556e390 0x55aca5934fd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f59640045e0 tx=0x7f5964004610 comp rx=0 tx=0).stop 2024-07-26T13:41:23.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.683+0000 7f597bed9740 1 -- 172.21.15.100:0/4066141650 shutdown_connections 2024-07-26T13:41:23.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.683+0000 7f597bed9740 1 --2- 172.21.15.100:0/4066141650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aca556e390 0x55aca5934fd0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:23.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.683+0000 7f597bed9740 1 --2- 172.21.15.100:0/4066141650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aca55661b0 0x55aca5934a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:23.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.683+0000 7f597bed9740 1 --2- 172.21.15.100:0/4066141650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aca558f070 0x55aca5934550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:23.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.683+0000 7f597bed9740 1 -- 172.21.15.100:0/4066141650 >> 172.21.15.100:0/4066141650 conn(0x55aca5583460 msgr2=0x55aca5932440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:23.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.683+0000 7f597bed9740 1 -- 172.21.15.100:0/4066141650 shutdown_connections 2024-07-26T13:41:23.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.683+0000 7f597bed9740 1 -- 172.21.15.100:0/4066141650 wait complete. 2024-07-26T13:41:23.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.684+0000 7f597bed9740 1 -- start start 2024-07-26T13:41:23.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.684+0000 7f597bed9740 1 Processor -- start 2024-07-26T13:41:23.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.684+0000 7f597bed9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aca55661b0 0x55aca593ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:23.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.684+0000 7f597bed9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aca556e390 0x55aca5940510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:23.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.684+0000 7f597bed9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aca558f070 0x55aca5945760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:23.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.684+0000 7f597bed9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55aca56fd700 con 0x55aca556e390 2024-07-26T13:41:23.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.684+0000 7f597bed9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55aca583c9c0 con 0x55aca55661b0 2024-07-26T13:41:23.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.684+0000 7f597bed9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55aca59470e0 con 0x55aca558f070 2024-07-26T13:41:23.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.684+0000 7f597bed9740 10 client.0 _ll_register_callbacks cb 0x55aca58574d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:41:23.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.685+0000 7f596f7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aca558f070 0x55aca5945760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:23.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.685+0000 7f596f7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aca558f070 0x55aca5945760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:52478/0 (socket says 172.21.15.100:52478) 2024-07-26T13:41:23.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.685+0000 7f596f7fe640 1 -- 172.21.15.100:0/2587318784 learned_addr learned my addr 172.21.15.100:0/2587318784 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:23.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.685+0000 7f596e7fc640 1 --2- 172.21.15.100:0/2587318784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aca556e390 0x55aca5940510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:23.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.685+0000 7f596f7fe640 1 -- 172.21.15.100:0/2587318784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aca55661b0 msgr2=0x55aca593ffd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-26T13:41:23.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.685+0000 7f596f7fe640 1 --2- 172.21.15.100:0/2587318784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aca55661b0 0x55aca593ffd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:23.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.685+0000 7f596f7fe640 1 -- 172.21.15.100:0/2587318784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aca556e390 msgr2=0x55aca5940510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:23.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.685+0000 7f596f7fe640 1 --2- 172.21.15.100:0/2587318784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aca556e390 0x55aca5940510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:23.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.685+0000 7f596f7fe640 1 -- 172.21.15.100:0/2587318784 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aca55fa650 con 0x55aca558f070 2024-07-26T13:41:23.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.686+0000 7f596f7fe640 1 --2- 172.21.15.100:0/2587318784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aca558f070 0x55aca5945760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f59680093f0 tx=0x7f596802a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ecda224e16a916a server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:23.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.686+0000 7f594effd640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-07-26T13:41:23.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.686+0000 7f594effd640 1 -- 172.21.15.100:0/2587318784 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59680318e0 con 0x55aca558f070 2024-07-26T13:41:23.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.686+0000 7f597bed9740 1 -- 172.21.15.100:0/2587318784 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5964015070 con 0x55aca558f070 2024-07-26T13:41:23.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.686+0000 7f594effd640 1 -- 172.21.15.100:0/2587318784 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f5968004250 con 0x55aca558f070 2024-07-26T13:41:23.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.686+0000 7f594effd640 1 -- 172.21.15.100:0/2587318784 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5968004450 con 0x55aca558f070 2024-07-26T13:41:23.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.686+0000 7f597bed9740 10 client.25336 fetch_fsmap learned FSMap version 26 2024-07-26T13:41:23.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.686+0000 7f597bed9740 1 -- 172.21.15.100:0/2587318784 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55aca59575c0 con 0x55aca558f070 2024-07-26T13:41:23.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.687+0000 7f594effd640 1 -- 172.21.15.100:0/2587318784 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5964015070 con 0x55aca558f070 2024-07-26T13:41:23.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.687+0000 7f594effd640 1 -- 172.21.15.100:0/2587318784 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f5968059070 con 0x55aca558f070 2024-07-26T13:41:23.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.687+0000 7f597bed9740 10 client.25336 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:41:23.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.687+0000 7f597bed9740 10 client.25336 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:41:23.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.687+0000 7f597bed9740 20 client.25336 populate_metadata read hostname 'smithi100' 2024-07-26T13:41:23.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.687+0000 7f597bed9740 1 --2- 172.21.15.100:0/2587318784 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55aca554d9f0 0x55aca595ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:23.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.687+0000 7f597bed9740 4 client.25336 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:41:23.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.687+0000 7f597bed9740 1 -- 172.21.15.100:0/2587318784 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aca58acae0 con 0x55aca554d9f0 2024-07-26T13:41:23.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.687+0000 7f596effd640 1 --2- 172.21.15.100:0/2587318784 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55aca554d9f0 0x55aca595ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:23.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.688+0000 7f594effd640 10 client.25336 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:23.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.688+0000 7f596effd640 1 --2- 172.21.15.100:0/2587318784 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55aca554d9f0 0x55aca595ab20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f91056525ae8fcf6 server_cookie=b7090792877ef0c9 in_seq=0 out_seq=0 2024-07-26T13:41:23.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.689+0000 7f594effd640 1 -- 172.21.15.100:0/2587318784 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aca58acae0 con 0x55aca554d9f0 2024-07-26T13:41:23.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.689+0000 7f594effd640 10 client.25336 handle_command_reply: tid=0 2024-07-26T13:41:23.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.689+0000 7f597bed9740 1 client.25336 shutdown 2024-07-26T13:41:23.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.689+0000 7f597bed9740 1 -- 172.21.15.100:0/2587318784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aca558f070 msgr2=0x55aca5945760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:23.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.689+0000 7f597bed9740 1 --2- 172.21.15.100:0/2587318784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aca558f070 0x55aca5945760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f59680093f0 tx=0x7f596802a8c0 comp rx=0 tx=0).stop 2024-07-26T13:41:23.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.689+0000 7f597bed9740 1 -- 172.21.15.100:0/2587318784 shutdown_connections 2024-07-26T13:41:23.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.689+0000 7f597bed9740 1 --2- 172.21.15.100:0/2587318784 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55aca554d9f0 0x55aca595ab20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:23.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.689+0000 7f597bed9740 1 --2- 172.21.15.100:0/2587318784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aca558f070 0x55aca5945760 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:23.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.689+0000 7f597bed9740 1 --2- 172.21.15.100:0/2587318784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aca556e390 0x55aca5940510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:23.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.689+0000 7f597bed9740 1 --2- 172.21.15.100:0/2587318784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aca55661b0 0x55aca593ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:23.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.689+0000 7f597bed9740 1 -- 172.21.15.100:0/2587318784 >> 172.21.15.100:0/2587318784 conn(0x55aca5583460 msgr2=0x55aca5935f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:23.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.689+0000 7f594effd640 0 client.25336 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:23.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.689+0000 7f597bed9740 1 -- 172.21.15.100:0/2587318784 shutdown_connections 2024-07-26T13:41:23.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.689+0000 7f597bed9740 1 -- 172.21.15.100:0/2587318784 wait complete. 2024-07-26T13:41:23.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.690+0000 7f597bed9740 20 client.25336 trim_cache size 0 max 16384 2024-07-26T13:41:23.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.723+0000 7f597bed9740 1 Processor -- start 2024-07-26T13:41:23.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.723+0000 7f597bed9740 1 -- start start 2024-07-26T13:41:23.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.724+0000 7f597bed9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aca55661b0 0x55aca5939d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:23.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.724+0000 7f597bed9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aca556e390 0x55aca5960320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:23.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.724+0000 7f597bed9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aca558f070 0x55aca5960860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:23.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.724+0000 7f597bed9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f59701af7f0 con 0x55aca558f070 2024-07-26T13:41:23.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.724+0000 7f597bed9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f59701ad0c0 con 0x55aca55661b0 2024-07-26T13:41:23.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.724+0000 7f597bed9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f59701aaf10 con 0x55aca556e390 2024-07-26T13:41:23.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.724+0000 7f596f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aca558f070 0x55aca5960860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:23.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.724+0000 7f596f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aca558f070 0x55aca5960860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35916/0 (socket says 172.21.15.100:35916) 2024-07-26T13:41:23.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.724+0000 7f596f7fe640 1 -- 172.21.15.100:0/1920646965 learned_addr learned my addr 172.21.15.100:0/1920646965 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:23.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.724+0000 7f596e7fc640 1 --2- 172.21.15.100:0/1920646965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aca556e390 0x55aca5960320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:23.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.724+0000 7f596effd640 1 --2- 172.21.15.100:0/1920646965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aca55661b0 0x55aca5939d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:23.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.725+0000 7f596e7fc640 1 -- 172.21.15.100:0/1920646965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aca55661b0 msgr2=0x55aca5939d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:23.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.725+0000 7f596e7fc640 1 --2- 172.21.15.100:0/1920646965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aca55661b0 0x55aca5939d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:23.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.725+0000 7f596e7fc640 1 -- 172.21.15.100:0/1920646965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aca558f070 msgr2=0x55aca5960860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:23.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.725+0000 7f596e7fc640 1 --2- 172.21.15.100:0/1920646965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aca558f070 0x55aca5960860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:23.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.725+0000 7f596e7fc640 1 -- 172.21.15.100:0/1920646965 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aca593ef30 con 0x55aca556e390 2024-07-26T13:41:23.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.725+0000 7f596f7fe640 1 --2- 172.21.15.100:0/1920646965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aca558f070 0x55aca5960860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:41:23.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.725+0000 7f596e7fc640 1 --2- 172.21.15.100:0/1920646965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aca556e390 0x55aca5960320 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f596400b5f0 tx=0x7f596401b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=440f72721408b75d server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:23.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.725+0000 7f59557fa640 1 -- 172.21.15.100:0/1920646965 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5964029070 con 0x55aca556e390 2024-07-26T13:41:23.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.725+0000 7f59557fa640 1 -- 172.21.15.100:0/1920646965 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5964015050 con 0x55aca556e390 2024-07-26T13:41:23.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.725+0000 7f59557fa640 1 -- 172.21.15.100:0/1920646965 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5964011e40 con 0x55aca556e390 2024-07-26T13:41:23.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.727+0000 7f597bed9740 1 -- 172.21.15.100:0/1920646965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aca556e390 msgr2=0x55aca5960320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:23.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.727+0000 7f597bed9740 1 --2- 172.21.15.100:0/1920646965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aca556e390 0x55aca5960320 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f596400b5f0 tx=0x7f596401b040 comp rx=0 tx=0).stop 2024-07-26T13:41:23.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.727+0000 7f597bed9740 1 -- 172.21.15.100:0/1920646965 shutdown_connections 2024-07-26T13:41:23.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.727+0000 7f597bed9740 1 --2- 172.21.15.100:0/1920646965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aca558f070 0x55aca5960860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:23.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.727+0000 7f597bed9740 1 --2- 172.21.15.100:0/1920646965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aca556e390 0x55aca5960320 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:23.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.727+0000 7f597bed9740 1 --2- 172.21.15.100:0/1920646965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aca55661b0 0x55aca5939d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:23.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.727+0000 7f597bed9740 1 -- 172.21.15.100:0/1920646965 >> 172.21.15.100:0/1920646965 conn(0x55aca5583460 msgr2=0x55aca5939680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:23.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.727+0000 7f597bed9740 1 -- 172.21.15.100:0/1920646965 shutdown_connections 2024-07-26T13:41:23.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.727+0000 7f597bed9740 1 -- 172.21.15.100:0/1920646965 wait complete. 2024-07-26T13:41:23.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.727+0000 7f597bed9740 1 -- start start 2024-07-26T13:41:23.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.728+0000 7f597bed9740 1 Processor -- start 2024-07-26T13:41:23.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.728+0000 7f597bed9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aca55661b0 0x55aca5952d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:23.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.728+0000 7f597bed9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aca556e390 0x55aca59532c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:23.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.728+0000 7f596effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aca55661b0 0x55aca5952d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:23.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.728+0000 7f596effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aca55661b0 0x55aca5952d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35930/0 (socket says 172.21.15.100:35930) 2024-07-26T13:41:23.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.728+0000 7f596effd640 1 -- 172.21.15.100:0/2744711248 learned_addr learned my addr 172.21.15.100:0/2744711248 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:23.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.728+0000 7f597bed9740 1 --2- 172.21.15.100:0/2744711248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aca558f070 0x55aca5953800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:23.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.729+0000 7f597bed9740 1 -- 172.21.15.100:0/2744711248 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f59701ab580 con 0x55aca55661b0 2024-07-26T13:41:23.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.729+0000 7f597bed9740 1 -- 172.21.15.100:0/2744711248 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f59701ac9f0 con 0x55aca556e390 2024-07-26T13:41:23.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.729+0000 7f597bed9740 1 -- 172.21.15.100:0/2744711248 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f59701abc50 con 0x55aca558f070 2024-07-26T13:41:23.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.729+0000 7f596e7fc640 1 --2- 172.21.15.100:0/2744711248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aca556e390 0x55aca59532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:23.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.729+0000 7f596f7fe640 1 --2- 172.21.15.100:0/2744711248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aca558f070 0x55aca5953800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:23.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.729+0000 7f596effd640 1 -- 172.21.15.100:0/2744711248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aca558f070 msgr2=0x55aca5953800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:23.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.729+0000 7f596effd640 1 --2- 172.21.15.100:0/2744711248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aca558f070 0x55aca5953800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:23.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.729+0000 7f596effd640 1 -- 172.21.15.100:0/2744711248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aca556e390 msgr2=0x55aca59532c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:23.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.729+0000 7f596effd640 1 --2- 172.21.15.100:0/2744711248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aca556e390 0x55aca59532c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:23.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.729+0000 7f596effd640 1 -- 172.21.15.100:0/2744711248 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aca59c2430 con 0x55aca55661b0 2024-07-26T13:41:23.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.729+0000 7f596e7fc640 1 --2- 172.21.15.100:0/2744711248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aca556e390 0x55aca59532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:41:23.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.729+0000 7f597bed9740 10 client.0 _ll_register_callbacks cb 0x7f596802f430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:41:23.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.729+0000 7f597bed9740 1 -- 172.21.15.100:0/2744711248 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5964015050 con 0x55aca55661b0 2024-07-26T13:41:23.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.729+0000 7f596effd640 1 --2- 172.21.15.100:0/2744711248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aca55661b0 0x55aca5952d80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f59600e1700 tx=0x7f59600aa390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=270b67ff1f84380f server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:23.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.730+0000 7f594effd640 10 client.15771 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:41:23.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.730+0000 7f594effd640 1 -- 172.21.15.100:0/2744711248 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5960020040 con 0x55aca55661b0 2024-07-26T13:41:23.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.730+0000 7f594effd640 1 -- 172.21.15.100:0/2744711248 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f59600e1920 con 0x55aca55661b0 2024-07-26T13:41:23.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.730+0000 7f594effd640 1 -- 172.21.15.100:0/2744711248 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f59600aa670 con 0x55aca55661b0 2024-07-26T13:41:23.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.730+0000 7f597bed9740 10 client.15771 fetch_fsmap learned FSMap version 26 2024-07-26T13:41:23.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.730+0000 7f597bed9740 1 -- 172.21.15.100:0/2744711248 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55aca59c35e0 con 0x55aca55661b0 2024-07-26T13:41:23.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.731+0000 7f594effd640 1 -- 172.21.15.100:0/2744711248 <== mon.0 v2:172.21.15.100:3300/0 4 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f59600a6df0 con 0x55aca55661b0 2024-07-26T13:41:23.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.731+0000 7f597bed9740 10 client.15771 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:41:23.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.731+0000 7f597bed9740 10 client.15771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:41:23.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.731+0000 7f597bed9740 20 client.15771 populate_metadata read hostname 'smithi100' 2024-07-26T13:41:23.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.731+0000 7f597bed9740 1 --2- 172.21.15.100:0/2744711248 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55aca5522da0 0x55aca59c5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:23.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.731+0000 7f597bed9740 4 client.15771 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-26T13:41:23.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.731+0000 7f597bed9740 1 -- 172.21.15.100:0/2744711248 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55aca58a8580 con 0x55aca5522da0 2024-07-26T13:41:23.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.731+0000 7f596e7fc640 1 --2- 172.21.15.100:0/2744711248 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55aca5522da0 0x55aca59c5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:23.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.732+0000 7f596e7fc640 1 --2- 172.21.15.100:0/2744711248 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55aca5522da0 0x55aca59c5c40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1657342894e625c6 server_cookie=dea3785101d5af46 in_seq=0 out_seq=0 2024-07-26T13:41:23.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.732+0000 7f594effd640 10 client.15771 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:23.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.732+0000 7f594effd640 1 -- 172.21.15.100:0/2744711248 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55aca58a8580 con 0x55aca5522da0 2024-07-26T13:41:23.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.732+0000 7f594effd640 10 client.15771 handle_command_reply: tid=0 2024-07-26T13:41:23.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.732+0000 7f597bed9740 1 client.15771 shutdown 2024-07-26T13:41:23.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.732+0000 7f597bed9740 1 -- 172.21.15.100:0/2744711248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aca55661b0 msgr2=0x55aca5952d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:23.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.732+0000 7f597bed9740 1 --2- 172.21.15.100:0/2744711248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aca55661b0 0x55aca5952d80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f59600e1700 tx=0x7f59600aa390 comp rx=0 tx=0).stop 2024-07-26T13:41:23.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.732+0000 7f597bed9740 1 -- 172.21.15.100:0/2744711248 shutdown_connections 2024-07-26T13:41:23.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.732+0000 7f597bed9740 1 --2- 172.21.15.100:0/2744711248 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55aca5522da0 0x55aca59c5c40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:23.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.733+0000 7f597bed9740 1 --2- 172.21.15.100:0/2744711248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aca558f070 0x55aca5953800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:23.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.733+0000 7f597bed9740 1 --2- 172.21.15.100:0/2744711248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aca556e390 0x55aca59532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:23.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.733+0000 7f597bed9740 1 --2- 172.21.15.100:0/2744711248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aca55661b0 0x55aca5952d80 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:23.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.733+0000 7f597bed9740 1 -- 172.21.15.100:0/2744711248 >> 172.21.15.100:0/2744711248 conn(0x55aca5583460 msgr2=0x55aca5935390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:23.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.733+0000 7f594effd640 0 client.15771 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:23.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.733+0000 7f597bed9740 1 -- 172.21.15.100:0/2744711248 shutdown_connections 2024-07-26T13:41:23.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.733+0000 7f597bed9740 1 -- 172.21.15.100:0/2744711248 wait complete. 2024-07-26T13:41:23.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.733+0000 7f597bed9740 20 client.15771 trim_cache size 0 max 16384 2024-07-26T13:41:23.739 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:41:23.739 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-07-26T13:41:23.740 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-07-26T13:41:23.740 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:41:23.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.734+0000 7f5975955640 1 -- 172.21.15.100:0/2798079589 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f59580b3dc0 msgr2=0x7f59580b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:23.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.734+0000 7f5975955640 1 --2- 172.21.15.100:0/2798079589 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f59580b3dc0 0x7f59580b62d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5968004af0 tx=0x7f5968005d80 comp rx=0 tx=0).stop 2024-07-26T13:41:23.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.734+0000 7f5975955640 1 -- 172.21.15.100:0/2798079589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59701019a0 msgr2=0x7f5970199f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:23.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.734+0000 7f5975955640 1 --2- 172.21.15.100:0/2798079589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59701019a0 0x7f5970199f10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f596000d950 tx=0x7f596000de10 comp rx=0 tx=0).stop 2024-07-26T13:41:23.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.734+0000 7f5975955640 1 -- 172.21.15.100:0/2798079589 shutdown_connections 2024-07-26T13:41:23.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.734+0000 7f5975955640 1 --2- 172.21.15.100:0/2798079589 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f59580b3dc0 0x7f59580b62d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:23.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.734+0000 7f5975955640 1 --2- 172.21.15.100:0/2798079589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5970109830 0x7f597019f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:23.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.734+0000 7f5975955640 1 --2- 172.21.15.100:0/2798079589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5970108ee0 0x7f597019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:23.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.734+0000 7f5975955640 1 --2- 172.21.15.100:0/2798079589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59701019a0 0x7f5970199f10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:23.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.734+0000 7f5975955640 1 -- 172.21.15.100:0/2798079589 >> 172.21.15.100:0/2798079589 conn(0x7f5970077690 msgr2=0x7f5970106fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:23.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.734+0000 7f5975955640 1 -- 172.21.15.100:0/2798079589 shutdown_connections 2024-07-26T13:41:23.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:23.734+0000 7f5975955640 1 -- 172.21.15.100:0/2798079589 wait complete. 2024-07-26T13:41:23.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:23 smithi167 ceph-mon[37140]: pgmap v402: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T13:41:23.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:23 smithi138 ceph-mon[36954]: pgmap v402: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T13:41:23.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:23 smithi100 ceph-mon[32599]: pgmap v402: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T13:41:24.589 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-26T13:41:24.589 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:41:24.883 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:41:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:25 smithi100 ceph-mon[32599]: pgmap v403: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-07-26T13:41:25.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:25 smithi167 ceph-mon[37140]: pgmap v403: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-07-26T13:41:25.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:25 smithi138 ceph-mon[36954]: pgmap v403: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-07-26T13:41:26.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.700+0000 7fc1103c9640 1 -- 172.21.15.100:0/786990996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc108104d80 msgr2=0x7fc10810d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:26.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.700+0000 7fc1103c9640 1 --2- 172.21.15.100:0/786990996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc108104d80 0x7fc10810d2d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009900 tx=0x7fc0f802a410 comp rx=0 tx=0).stop 2024-07-26T13:41:26.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.700+0000 7fc1103c9640 1 -- 172.21.15.100:0/786990996 shutdown_connections 2024-07-26T13:41:26.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.700+0000 7fc1103c9640 1 --2- 172.21.15.100:0/786990996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc10810d860 0x7fc10810fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:26.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.700+0000 7fc1103c9640 1 --2- 172.21.15.100:0/786990996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc108104d80 0x7fc10810d2d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:26.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.700+0000 7fc1103c9640 1 --2- 172.21.15.100:0/786990996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc108104430 0x7fc108104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:26.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.700+0000 7fc1103c9640 1 -- 172.21.15.100:0/786990996 >> 172.21.15.100:0/786990996 conn(0x7fc108079140 msgr2=0x7fc108079550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:26.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.700+0000 7fc1103c9640 1 -- 172.21.15.100:0/786990996 shutdown_connections 2024-07-26T13:41:26.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.700+0000 7fc1103c9640 1 -- 172.21.15.100:0/786990996 wait complete. 2024-07-26T13:41:26.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.701+0000 7fc1103c9640 1 Processor -- start 2024-07-26T13:41:26.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.701+0000 7fc1103c9640 1 -- start start 2024-07-26T13:41:26.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.701+0000 7fc1103c9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc108104430 0x7fc108199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:26.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.702+0000 7fc1103c9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc108104d80 0x7fc10819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:26.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.702+0000 7fc1103c9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc10810d860 0x7fc10819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:26.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.702+0000 7fc1103c9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc108113490 con 0x7fc108104430 2024-07-26T13:41:26.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.702+0000 7fc1103c9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc108113310 con 0x7fc108104d80 2024-07-26T13:41:26.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.702+0000 7fc1103c9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc108113610 con 0x7fc10810d860 2024-07-26T13:41:26.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.702+0000 7fc10e165640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc108104430 0x7fc108199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:26.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.702+0000 7fc10e165640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc108104430 0x7fc108199ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49972/0 (socket says 172.21.15.100:49972) 2024-07-26T13:41:26.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.702+0000 7fc10e165640 1 -- 172.21.15.100:0/2033091293 learned_addr learned my addr 172.21.15.100:0/2033091293 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:26.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.702+0000 7fc10d964640 1 --2- 172.21.15.100:0/2033091293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc108104d80 0x7fc10819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:26.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.702+0000 7fc10e966640 1 --2- 172.21.15.100:0/2033091293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc10810d860 0x7fc10819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:26.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.702+0000 7fc10e165640 1 -- 172.21.15.100:0/2033091293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc10810d860 msgr2=0x7fc10819f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:26.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.702+0000 7fc10e165640 1 --2- 172.21.15.100:0/2033091293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc10810d860 0x7fc10819f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:26.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.702+0000 7fc10e165640 1 -- 172.21.15.100:0/2033091293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc108104d80 msgr2=0x7fc10819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:26.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.702+0000 7fc10e165640 1 --2- 172.21.15.100:0/2033091293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc108104d80 0x7fc10819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:26.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.702+0000 7fc10e165640 1 -- 172.21.15.100:0/2033091293 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc10819fe30 con 0x7fc108104430 2024-07-26T13:41:26.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.702+0000 7fc10e165640 1 --2- 172.21.15.100:0/2033091293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc108104430 0x7fc108199ec0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc10400e9f0 tx=0x7fc10400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:26.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.703+0000 7fc0ff7fe640 1 -- 172.21.15.100:0/2033091293 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10400cd80 con 0x7fc108104430 2024-07-26T13:41:26.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.703+0000 7fc1103c9640 1 -- 172.21.15.100:0/2033091293 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1081a0120 con 0x7fc108104430 2024-07-26T13:41:26.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.703+0000 7fc1103c9640 1 -- 172.21.15.100:0/2033091293 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1081a6880 con 0x7fc108104430 2024-07-26T13:41:26.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.703+0000 7fc0ff7fe640 1 -- 172.21.15.100:0/2033091293 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1040108e0 con 0x7fc108104430 2024-07-26T13:41:26.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.703+0000 7fc0ff7fe640 1 -- 172.21.15.100:0/2033091293 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc104018980 con 0x7fc108104430 2024-07-26T13:41:26.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.704+0000 7fc1103c9640 1 -- 172.21.15.100:0/2033091293 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0ec0052c0 con 0x7fc108104430 2024-07-26T13:41:26.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.707+0000 7fc0ff7fe640 1 -- 172.21.15.100:0/2033091293 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fc104010430 con 0x7fc108104430 2024-07-26T13:41:26.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.708+0000 7fc0ff7fe640 1 --2- 172.21.15.100:0/2033091293 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc0f40b81d0 0x7fc0f40ba6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:26.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.708+0000 7fc0ff7fe640 1 -- 172.21.15.100:0/2033091293 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fc1040e4090 con 0x7fc108104430 2024-07-26T13:41:26.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.708+0000 7fc10d964640 1 --2- 172.21.15.100:0/2033091293 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc0f40b81d0 0x7fc0f40ba6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:26.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.709+0000 7fc10d964640 1 --2- 172.21.15.100:0/2033091293 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc0f40b81d0 0x7fc0f40ba6e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009230 tx=0x7fc0f8031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:26.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.713+0000 7fc0ff7fe640 1 -- 172.21.15.100:0/2033091293 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fc1040aad70 con 0x7fc108104430 2024-07-26T13:41:26.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.929+0000 7fc1103c9640 1 -- 172.21.15.100:0/2033091293 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0ec005880 con 0x7fc108104430 2024-07-26T13:41:26.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.930+0000 7fc0ff7fe640 1 -- 172.21.15.100:0/2033091293 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11927 (secure 0 0 0) 0x7fc1040af2c0 con 0x7fc108104430 2024-07-26T13:41:26.931 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:41:26.931 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":26,"btime":"2024-07-26T13:40:09:530496+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14790,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/313501409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":313501409},{"type":"v1","addr":"172.21.15.100:6839","nonce":313501409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/1402975506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":1402975506},{"type":"v1","addr":"172.21.15.100:6841","nonce":1402975506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-26T13:37:01.281843+0000","modified":"2024-07-26T13:40:09.187125+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24607,"mds_1":24605,"mds_2":24595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24557":{"gid":24557,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/3290105300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3290105300},{"type":"v1","addr":"172.21.15.138:6835","nonce":3290105300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/3393225483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3393225483},{"type":"v1","addr":"172.21.15.138:6837","nonce":3393225483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24595":{"gid":24595,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6837/3828634468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3828634468},{"type":"v1","addr":"172.21.15.167:6837","nonce":3828634468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.138:6839/1580925471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1580925471},{"type":"v1","addr":"172.21.15.138:6839","nonce":1580925471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/398150514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":398150514},{"type":"v1","addr":"172.21.15.167:6839","nonce":398150514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24607,"qdb_cluster":[24595,24605,24607]},"id":1}]} 2024-07-26T13:41:26.931 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 26 2024-07-26T13:41:26.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.932+0000 7fc1103c9640 1 -- 172.21.15.100:0/2033091293 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc0f40b81d0 msgr2=0x7fc0f40ba6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:26.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.932+0000 7fc1103c9640 1 --2- 172.21.15.100:0/2033091293 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc0f40b81d0 0x7fc0f40ba6e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009230 tx=0x7fc0f8031400 comp rx=0 tx=0).stop 2024-07-26T13:41:26.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.932+0000 7fc1103c9640 1 -- 172.21.15.100:0/2033091293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc108104430 msgr2=0x7fc108199ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:26.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.932+0000 7fc1103c9640 1 --2- 172.21.15.100:0/2033091293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc108104430 0x7fc108199ec0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc10400e9f0 tx=0x7fc10400eeb0 comp rx=0 tx=0).stop 2024-07-26T13:41:26.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.932+0000 7fc1103c9640 1 -- 172.21.15.100:0/2033091293 shutdown_connections 2024-07-26T13:41:26.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.932+0000 7fc1103c9640 1 --2- 172.21.15.100:0/2033091293 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc0f40b81d0 0x7fc0f40ba6e0 secure :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8009230 tx=0x7fc0f8031400 comp rx=0 tx=0).stop 2024-07-26T13:41:26.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.932+0000 7fc1103c9640 1 --2- 172.21.15.100:0/2033091293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc10810d860 0x7fc10819f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:26.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.932+0000 7fc1103c9640 1 --2- 172.21.15.100:0/2033091293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc108104d80 0x7fc10819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:26.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.932+0000 7fc1103c9640 1 --2- 172.21.15.100:0/2033091293 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc108104430 0x7fc108199ec0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:26.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.932+0000 7fc1103c9640 1 -- 172.21.15.100:0/2033091293 >> 172.21.15.100:0/2033091293 conn(0x7fc108079140 msgr2=0x7fc108100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:26.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.933+0000 7fc1103c9640 1 -- 172.21.15.100:0/2033091293 shutdown_connections 2024-07-26T13:41:26.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:26.933+0000 7fc1103c9640 1 -- 172.21.15.100:0/2033091293 wait complete. 2024-07-26T13:41:27.516 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:27 smithi100 ceph-mon[32599]: pgmap v404: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-07-26T13:41:27.516 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:27 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2033091293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:41:27.736 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-26T13:41:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:27 smithi167 ceph-mon[37140]: pgmap v404: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-07-26T13:41:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:27 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/2033091293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:41:27.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:27 smithi138 ceph-mon[36954]: pgmap v404: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-07-26T13:41:27.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:27 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/2033091293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:41:28.080 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:41:29.518 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:29 smithi100 ceph-mon[32599]: pgmap v405: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-07-26T13:41:29.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:29 smithi167 ceph-mon[37140]: pgmap v405: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-07-26T13:41:29.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:29 smithi138 ceph-mon[36954]: pgmap v405: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-07-26T13:41:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.085+0000 7f46a71e8640 1 -- 172.21.15.100:0/216059762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46a0107d60 msgr2=0x7f46a0108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.085+0000 7f46a71e8640 1 --2- 172.21.15.100:0/216059762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46a0107d60 0x7f46a0108170 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f468c009900 tx=0x7f468c02a3f0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.085+0000 7f46a71e8640 1 -- 172.21.15.100:0/216059762 shutdown_connections 2024-07-26T13:41:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.085+0000 7f46a71e8640 1 --2- 172.21.15.100:0/216059762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46a01086b0 0x7f46a010eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.085+0000 7f46a71e8640 1 --2- 172.21.15.100:0/216059762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46a0107d60 0x7f46a0108170 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.085+0000 7f46a71e8640 1 --2- 172.21.15.100:0/216059762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46a0102ba0 0x7f46a0102fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.085+0000 7f46a71e8640 1 -- 172.21.15.100:0/216059762 >> 172.21.15.100:0/216059762 conn(0x7f46a00798c0 msgr2=0x7f46a0079cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.086+0000 7f46a71e8640 1 -- 172.21.15.100:0/216059762 shutdown_connections 2024-07-26T13:41:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.086+0000 7f46a71e8640 1 -- 172.21.15.100:0/216059762 wait complete. 2024-07-26T13:41:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.086+0000 7f46a71e8640 1 Processor -- start 2024-07-26T13:41:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.087+0000 7f46a71e8640 1 -- start start 2024-07-26T13:41:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.087+0000 7f46a71e8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46a0102ba0 0x7f46a0197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.087+0000 7f46a71e8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46a0107d60 0x7f46a01982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:31.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.087+0000 7f46a71e8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46a01086b0 0x7f46a019d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:31.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.087+0000 7f46a71e8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f46a0111270 con 0x7f46a0107d60 2024-07-26T13:41:31.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.087+0000 7f46a71e8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f46a01110f0 con 0x7f46a0102ba0 2024-07-26T13:41:31.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.087+0000 7f46a71e8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f46a01113f0 con 0x7f46a01086b0 2024-07-26T13:41:31.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.087+0000 7f469ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46a0107d60 0x7f46a01982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:31.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.087+0000 7f46a4f84640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46a0102ba0 0x7f46a0197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:31.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.087+0000 7f46a4f84640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46a0102ba0 0x7f46a0197d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:39546/0 (socket says 172.21.15.100:39546) 2024-07-26T13:41:31.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.087+0000 7f46a4f84640 1 -- 172.21.15.100:0/1123861893 learned_addr learned my addr 172.21.15.100:0/1123861893 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:31.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.087+0000 7f46a5785640 1 --2- 172.21.15.100:0/1123861893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46a01086b0 0x7f46a019d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:31.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.088+0000 7f46a4f84640 1 -- 172.21.15.100:0/1123861893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46a01086b0 msgr2=0x7f46a019d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:31.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.088+0000 7f46a4f84640 1 --2- 172.21.15.100:0/1123861893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46a01086b0 0x7f46a019d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.088+0000 7f46a4f84640 1 -- 172.21.15.100:0/1123861893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46a0107d60 msgr2=0x7f46a01982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:31.091 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.088+0000 7f46a4f84640 1 --2- 172.21.15.100:0/1123861893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46a0107d60 0x7f46a01982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.091 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.088+0000 7f46a4f84640 1 -- 172.21.15.100:0/1123861893 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a019dc40 con 0x7f46a0102ba0 2024-07-26T13:41:31.091 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.088+0000 7f46a4f84640 1 --2- 172.21.15.100:0/1123861893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46a0102ba0 0x7f46a0197d10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4694009c40 tx=0x7f469400b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:31.091 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.088+0000 7f469dffb640 1 -- 172.21.15.100:0/1123861893 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4694005020 con 0x7f46a0102ba0 2024-07-26T13:41:31.091 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.088+0000 7f469dffb640 1 -- 172.21.15.100:0/1123861893 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4694005670 con 0x7f46a0102ba0 2024-07-26T13:41:31.091 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.088+0000 7f46a71e8640 1 -- 172.21.15.100:0/1123861893 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46a019df30 con 0x7f46a0102ba0 2024-07-26T13:41:31.091 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.088+0000 7f469dffb640 1 -- 172.21.15.100:0/1123861893 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4694019670 con 0x7f46a0102ba0 2024-07-26T13:41:31.091 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.088+0000 7f46a71e8640 1 -- 172.21.15.100:0/1123861893 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46a01a46c0 con 0x7f46a0102ba0 2024-07-26T13:41:31.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.092+0000 7f46ae1da740 1 Processor -- start 2024-07-26T13:41:31.094 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.092+0000 7f469dffb640 1 -- 172.21.15.100:0/1123861893 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f4694004830 con 0x7f46a0102ba0 2024-07-26T13:41:31.094 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.093+0000 7f469dffb640 1 --2- 172.21.15.100:0/1123861893 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f46900b3db0 0x7f46900b62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:31.094 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.093+0000 7f469dffb640 1 -- 172.21.15.100:0/1123861893 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f46940e7ac0 con 0x7f46a0102ba0 2024-07-26T13:41:31.094 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.093+0000 7f46a5785640 1 --2- 172.21.15.100:0/1123861893 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f46900b3db0 0x7f46900b62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:31.095 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.094+0000 7f46ae1da740 1 -- start start 2024-07-26T13:41:31.095 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.094+0000 7f46ae1da740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cab9ae1b0 0x555cabd50ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:31.095 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.094+0000 7f46ae1da740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cab9852f0 0x555cabd51220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:31.095 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.094+0000 7f46ae1da740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555cab98d4d0 0x555cabd51760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:31.095 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.094+0000 7f46ae1da740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x555cabcc63b0 con 0x555cab9ae1b0 2024-07-26T13:41:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.094+0000 7f46ae1da740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555cabcb58b0 con 0x555cab9852f0 2024-07-26T13:41:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.094+0000 7f46ae1da740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555cabcb98f0 con 0x555cab98d4d0 2024-07-26T13:41:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.094+0000 7f46a4f84640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cab9ae1b0 0x555cabd50ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.094+0000 7f46a4f84640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cab9ae1b0 0x555cabd50ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50012/0 (socket says 172.21.15.100:50012) 2024-07-26T13:41:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.094+0000 7f46a4f84640 1 -- 172.21.15.100:0/2918565813 learned_addr learned my addr 172.21.15.100:0/2918565813 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.094+0000 7f469ffff640 1 --2- 172.21.15.100:0/2918565813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cab9852f0 0x555cabd51220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.094+0000 7f46a4f84640 1 -- 172.21.15.100:0/2918565813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555cab98d4d0 msgr2=0x555cabd51760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-26T13:41:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.094+0000 7f46a4f84640 1 --2- 172.21.15.100:0/2918565813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555cab98d4d0 0x555cabd51760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.094+0000 7f46a4f84640 1 -- 172.21.15.100:0/2918565813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cab9852f0 msgr2=0x555cabd51220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.094+0000 7f46a4f84640 1 --2- 172.21.15.100:0/2918565813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cab9852f0 0x555cabd51220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.096 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.094+0000 7f46a4f84640 1 -- 172.21.15.100:0/2918565813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cabc5c9a0 con 0x555cab9ae1b0 2024-07-26T13:41:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.095+0000 7f46a4f84640 1 --2- 172.21.15.100:0/2918565813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cab9ae1b0 0x555cabd50ce0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4694004b80 tx=0x7f46940adeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64df3af363b45b4 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.095+0000 7f46a5785640 1 --2- 172.21.15.100:0/1123861893 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f46900b3db0 0x7f46900b62c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4698005fd0 tx=0x7f4698009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.095+0000 7f468affd640 1 -- 172.21.15.100:0/2918565813 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46940a8e60 con 0x555cab9ae1b0 2024-07-26T13:41:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.095+0000 7f468affd640 1 -- 172.21.15.100:0/2918565813 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46940ae740 con 0x555cab9ae1b0 2024-07-26T13:41:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.095+0000 7f468affd640 1 -- 172.21.15.100:0/2918565813 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46940e5650 con 0x555cab9ae1b0 2024-07-26T13:41:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.096+0000 7f46ae1da740 1 -- 172.21.15.100:0/2918565813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cab9ae1b0 msgr2=0x555cabd50ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.096+0000 7f46ae1da740 1 --2- 172.21.15.100:0/2918565813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cab9ae1b0 0x555cabd50ce0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4694004b80 tx=0x7f46940adeb0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.097+0000 7f46ae1da740 1 -- 172.21.15.100:0/2918565813 shutdown_connections 2024-07-26T13:41:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.097+0000 7f46ae1da740 1 --2- 172.21.15.100:0/2918565813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555cab98d4d0 0x555cabd51760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.097+0000 7f46ae1da740 1 --2- 172.21.15.100:0/2918565813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cab9852f0 0x555cabd51220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.097+0000 7f46ae1da740 1 --2- 172.21.15.100:0/2918565813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cab9ae1b0 0x555cabd50ce0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.097+0000 7f46ae1da740 1 -- 172.21.15.100:0/2918565813 >> 172.21.15.100:0/2918565813 conn(0x555cab9a2300 msgr2=0x555cabd4ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.097+0000 7f46ae1da740 1 -- 172.21.15.100:0/2918565813 shutdown_connections 2024-07-26T13:41:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.097+0000 7f46ae1da740 1 -- 172.21.15.100:0/2918565813 wait complete. 2024-07-26T13:41:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.098+0000 7f46ae1da740 1 -- start start 2024-07-26T13:41:31.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.098+0000 7f46ae1da740 1 Processor -- start 2024-07-26T13:41:31.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.099+0000 7f46ae1da740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cab9852f0 0x555cabd5c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.099+0000 7f46ae1da740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cab98d4d0 0x555cabd5cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.099+0000 7f46ae1da740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555cab9ae1b0 0x555cabd796a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.099+0000 7f46a4f84640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cab9852f0 0x555cabd5c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.100+0000 7f46a4f84640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cab9852f0 0x555cabd5c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:39574/0 (socket says 172.21.15.100:39574) 2024-07-26T13:41:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.100+0000 7f469ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cab98d4d0 0x555cabd5cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.100+0000 7f469ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cab98d4d0 0x555cabd5cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50024/0 (socket says 172.21.15.100:50024) 2024-07-26T13:41:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.100+0000 7f469ffff640 1 -- 172.21.15.100:0/2555584667 learned_addr learned my addr 172.21.15.100:0/2555584667 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.100+0000 7f46ae1da740 1 -- 172.21.15.100:0/2555584667 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x555cabbefab0 con 0x555cab98d4d0 2024-07-26T13:41:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.100+0000 7f46ae1da740 1 -- 172.21.15.100:0/2555584667 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555cabc3dcf0 con 0x555cab9852f0 2024-07-26T13:41:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.100+0000 7f46ae1da740 1 -- 172.21.15.100:0/2555584667 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555cabd703b0 con 0x555cab9ae1b0 2024-07-26T13:41:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.100+0000 7f46ae1da740 10 client.0 _ll_register_callbacks cb 0x555cabcaf9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:41:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.100+0000 7f46a5785640 1 --2- 172.21.15.100:0/2555584667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555cab9ae1b0 0x555cabd796a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.100+0000 7f46a4f84640 1 -- 172.21.15.100:0/2555584667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555cab9ae1b0 msgr2=0x555cabd796a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.100+0000 7f46a4f84640 1 --2- 172.21.15.100:0/2555584667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555cab9ae1b0 0x555cabd796a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.100+0000 7f46a4f84640 1 -- 172.21.15.100:0/2555584667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cab98d4d0 msgr2=0x555cabd5cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.100+0000 7f46a4f84640 1 --2- 172.21.15.100:0/2555584667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cab98d4d0 0x555cabd5cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.100+0000 7f46a4f84640 1 -- 172.21.15.100:0/2555584667 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555caba19790 con 0x555cab9852f0 2024-07-26T13:41:31.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.101+0000 7f46a4f84640 1 --2- 172.21.15.100:0/2555584667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cab9852f0 0x555cabd5c8b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4694010790 tx=0x7f46940cfdc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7e837d0fdb3e5d7 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:31.104 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.101+0000 7f467ffff640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-07-26T13:41:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.101+0000 7f467ffff640 1 -- 172.21.15.100:0/2555584667 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46940a8e60 con 0x555cab9852f0 2024-07-26T13:41:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.101+0000 7f467ffff640 1 -- 172.21.15.100:0/2555584667 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f46940aad30 con 0x555cab9852f0 2024-07-26T13:41:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.101+0000 7f46ae1da740 1 -- 172.21.15.100:0/2555584667 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46940ae740 con 0x555cab9852f0 2024-07-26T13:41:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.101+0000 7f467ffff640 1 -- 172.21.15.100:0/2555584667 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f46940b7740 con 0x555cab9852f0 2024-07-26T13:41:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.101+0000 7f46ae1da740 10 client.25262 fetch_fsmap learned FSMap version 26 2024-07-26T13:41:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.101+0000 7f46ae1da740 1 -- 172.21.15.100:0/2555584667 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555caba92620 con 0x555cab9852f0 2024-07-26T13:41:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.102+0000 7f467ffff640 1 -- 172.21.15.100:0/2555584667 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46940ae740 con 0x555cab9852f0 2024-07-26T13:41:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.102+0000 7f467ffff640 1 -- 172.21.15.100:0/2555584667 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f46940ad410 con 0x555cab9852f0 2024-07-26T13:41:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.102+0000 7f46ae1da740 10 client.25262 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:41:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.102+0000 7f46ae1da740 10 client.25262 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:41:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.102+0000 7f46ae1da740 20 client.25262 populate_metadata read hostname 'smithi100' 2024-07-26T13:41:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.102+0000 7f46ae1da740 1 --2- 172.21.15.100:0/2555584667 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555cab96cd80 0x555cabd7f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.102+0000 7f46ae1da740 4 client.25262 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:41:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.103+0000 7f46ae1da740 1 -- 172.21.15.100:0/2555584667 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cabcc8300 con 0x555cab96cd80 2024-07-26T13:41:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.103+0000 7f469ffff640 1 --2- 172.21.15.100:0/2555584667 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555cab96cd80 0x555cabd7f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.103+0000 7f467ffff640 10 client.25262 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.103+0000 7f469ffff640 1 --2- 172.21.15.100:0/2555584667 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555cab96cd80 0x555cabd7f560 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a9423f85d3be26e server_cookie=c11be75819763e02 in_seq=0 out_seq=0 2024-07-26T13:41:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.104+0000 7f467ffff640 1 -- 172.21.15.100:0/2555584667 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555cabcc8300 con 0x555cab96cd80 2024-07-26T13:41:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.104+0000 7f467ffff640 10 client.25262 handle_command_reply: tid=0 2024-07-26T13:41:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.104+0000 7f46ae1da740 1 client.25262 shutdown 2024-07-26T13:41:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.105+0000 7f46ae1da740 1 -- 172.21.15.100:0/2555584667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cab9852f0 msgr2=0x555cabd5c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.105+0000 7f46ae1da740 1 --2- 172.21.15.100:0/2555584667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cab9852f0 0x555cabd5c8b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4694010790 tx=0x7f46940cfdc0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.105+0000 7f46ae1da740 1 -- 172.21.15.100:0/2555584667 shutdown_connections 2024-07-26T13:41:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.105+0000 7f46ae1da740 1 --2- 172.21.15.100:0/2555584667 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555cab96cd80 0x555cabd7f560 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.105+0000 7f467ffff640 0 client.25262 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.105+0000 7f46ae1da740 1 --2- 172.21.15.100:0/2555584667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555cab9ae1b0 0x555cabd796a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.105+0000 7f46ae1da740 1 --2- 172.21.15.100:0/2555584667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cab98d4d0 0x555cabd5cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.105+0000 7f46ae1da740 1 --2- 172.21.15.100:0/2555584667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cab9852f0 0x555cabd5c8b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.105+0000 7f46ae1da740 1 -- 172.21.15.100:0/2555584667 >> 172.21.15.100:0/2555584667 conn(0x555cab9a2300 msgr2=0x555cabd51d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.105+0000 7f46ae1da740 1 -- 172.21.15.100:0/2555584667 shutdown_connections 2024-07-26T13:41:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.105+0000 7f46ae1da740 1 -- 172.21.15.100:0/2555584667 wait complete. 2024-07-26T13:41:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.105+0000 7f46ae1da740 20 client.25262 trim_cache size 0 max 16384 2024-07-26T13:41:31.122 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.122+0000 7f46ae1da740 1 Processor -- start 2024-07-26T13:41:31.123 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.122+0000 7f46ae1da740 1 -- start start 2024-07-26T13:41:31.123 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.122+0000 7f46ae1da740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cab96cd80 0x555cabd56db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.122+0000 7f46ae1da740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cab9852f0 0x555cabd5cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.123+0000 7f46ae1da740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555cab98d4d0 0x555cabd79610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.123+0000 7f469ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cab9852f0 0x555cabd5cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.123+0000 7f469ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cab9852f0 0x555cabd5cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50050/0 (socket says 172.21.15.100:50050) 2024-07-26T13:41:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.123+0000 7f469ffff640 1 -- 172.21.15.100:0/2419671167 learned_addr learned my addr 172.21.15.100:0/2419671167 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.123+0000 7f46a4f84640 1 --2- 172.21.15.100:0/2419671167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cab96cd80 0x555cabd56db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.123+0000 7f46ae1da740 1 -- 172.21.15.100:0/2419671167 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f46a01a6fe0 con 0x555cab9852f0 2024-07-26T13:41:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.123+0000 7f46ae1da740 1 -- 172.21.15.100:0/2419671167 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f46a01a48e0 con 0x555cab96cd80 2024-07-26T13:41:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.123+0000 7f46ae1da740 1 -- 172.21.15.100:0/2419671167 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f46a0072e80 con 0x555cab98d4d0 2024-07-26T13:41:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.123+0000 7f46a5785640 1 --2- 172.21.15.100:0/2419671167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555cab98d4d0 0x555cabd79610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.123+0000 7f46a4f84640 1 -- 172.21.15.100:0/2419671167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555cab98d4d0 msgr2=0x555cabd79610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.123+0000 7f46a4f84640 1 --2- 172.21.15.100:0/2419671167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555cab98d4d0 0x555cabd79610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.123+0000 7f46a4f84640 1 -- 172.21.15.100:0/2419671167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cab9852f0 msgr2=0x555cabd5cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.123+0000 7f46a4f84640 1 --2- 172.21.15.100:0/2419671167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cab9852f0 0x555cabd5cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.123+0000 7f46a4f84640 1 -- 172.21.15.100:0/2419671167 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cabd53af0 con 0x555cab96cd80 2024-07-26T13:41:31.126 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.123+0000 7f46a4f84640 1 --2- 172.21.15.100:0/2419671167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cab96cd80 0x555cabd56db0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f46940ae6d0 tx=0x7f4694004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c8a0fa245eac687 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.124+0000 7f468a7fc640 1 -- 172.21.15.100:0/2419671167 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46940a8e60 con 0x555cab96cd80 2024-07-26T13:41:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.124+0000 7f468a7fc640 1 -- 172.21.15.100:0/2419671167 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46940ae740 con 0x555cab96cd80 2024-07-26T13:41:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.124+0000 7f468a7fc640 1 -- 172.21.15.100:0/2419671167 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46940aa0b0 con 0x555cab96cd80 2024-07-26T13:41:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.125+0000 7f46ae1da740 1 -- 172.21.15.100:0/2419671167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cab96cd80 msgr2=0x555cabd56db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.125+0000 7f46ae1da740 1 --2- 172.21.15.100:0/2419671167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cab96cd80 0x555cabd56db0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f46940ae6d0 tx=0x7f4694004b40 comp rx=0 tx=0).stop 2024-07-26T13:41:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.125+0000 7f46ae1da740 1 -- 172.21.15.100:0/2419671167 shutdown_connections 2024-07-26T13:41:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.125+0000 7f46ae1da740 1 --2- 172.21.15.100:0/2419671167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555cab98d4d0 0x555cabd79610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.125+0000 7f46ae1da740 1 --2- 172.21.15.100:0/2419671167 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cab9852f0 0x555cabd5cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.125+0000 7f46ae1da740 1 --2- 172.21.15.100:0/2419671167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cab96cd80 0x555cabd56db0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.125+0000 7f46ae1da740 1 -- 172.21.15.100:0/2419671167 >> 172.21.15.100:0/2419671167 conn(0x555cab9a2300 msgr2=0x555cabd5c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.125+0000 7f46ae1da740 1 -- 172.21.15.100:0/2419671167 shutdown_connections 2024-07-26T13:41:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.125+0000 7f46ae1da740 1 -- 172.21.15.100:0/2419671167 wait complete. 2024-07-26T13:41:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.125+0000 7f46ae1da740 1 -- start start 2024-07-26T13:41:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.125+0000 7f46ae1da740 1 Processor -- start 2024-07-26T13:41:31.130 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.126+0000 7f46ae1da740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cab96cd80 0x555cabd82a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:31.130 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.126+0000 7f46ae1da740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555cab9852f0 0x555cabd80ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:31.130 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.126+0000 7f46ae1da740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cab98d4d0 0x555cabd81410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:31.130 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.126+0000 7f46ae1da740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f46a00734f0 con 0x555cab96cd80 2024-07-26T13:41:31.130 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.126+0000 7f46ae1da740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f46a0074960 con 0x555cab98d4d0 2024-07-26T13:41:31.130 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.126+0000 7f46ae1da740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f46a0073bc0 con 0x555cab9852f0 2024-07-26T13:41:31.130 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.126+0000 7f46ae1da740 10 client.0 _ll_register_callbacks cb 0x7f46940aabe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:41:31.130 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.126+0000 7f46a4f84640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cab96cd80 0x555cabd82a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.126+0000 7f46a4f84640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cab96cd80 0x555cabd82a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50058/0 (socket says 172.21.15.100:50058) 2024-07-26T13:41:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.126+0000 7f46a4f84640 1 -- 172.21.15.100:0/458348438 learned_addr learned my addr 172.21.15.100:0/458348438 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.126+0000 7f46a5785640 1 --2- 172.21.15.100:0/458348438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cab98d4d0 0x555cabd81410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.127+0000 7f469ffff640 1 --2- 172.21.15.100:0/458348438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555cab9852f0 0x555cabd80ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.127+0000 7f46a4f84640 1 -- 172.21.15.100:0/458348438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555cab9852f0 msgr2=0x555cabd80ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.127+0000 7f46a4f84640 1 --2- 172.21.15.100:0/458348438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555cab9852f0 0x555cabd80ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.127+0000 7f46a4f84640 1 -- 172.21.15.100:0/458348438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cab98d4d0 msgr2=0x555cabd81410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.127+0000 7f46a4f84640 1 --2- 172.21.15.100:0/458348438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cab98d4d0 0x555cabd81410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.127+0000 7f46a4f84640 1 -- 172.21.15.100:0/458348438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cabd5c420 con 0x555cab96cd80 2024-07-26T13:41:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.127+0000 7f46a4f84640 1 --2- 172.21.15.100:0/458348438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cab96cd80 0x555cabd82a00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f46940ad800 tx=0x7f46940adbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d5e2646eae9d9d server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.127+0000 7f467ffff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:41:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.127+0000 7f467ffff640 1 -- 172.21.15.100:0/458348438 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46940a8e60 con 0x555cab96cd80 2024-07-26T13:41:31.131 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.127+0000 7f467ffff640 1 -- 172.21.15.100:0/458348438 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f46940aad30 con 0x555cab96cd80 2024-07-26T13:41:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.127+0000 7f46ae1da740 1 -- 172.21.15.100:0/458348438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46940ae740 con 0x555cab96cd80 2024-07-26T13:41:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.128+0000 7f467ffff640 1 -- 172.21.15.100:0/458348438 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f46940b3e20 con 0x555cab96cd80 2024-07-26T13:41:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.128+0000 7f46ae1da740 10 client.15807 fetch_fsmap learned FSMap version 26 2024-07-26T13:41:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.128+0000 7f46ae1da740 1 -- 172.21.15.100:0/458348438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555cabd5c420 con 0x555cab96cd80 2024-07-26T13:41:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.128+0000 7f467ffff640 1 -- 172.21.15.100:0/458348438 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46940ae740 con 0x555cab96cd80 2024-07-26T13:41:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.128+0000 7f467ffff640 1 -- 172.21.15.100:0/458348438 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f4694088980 con 0x555cab96cd80 2024-07-26T13:41:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.128+0000 7f46ae1da740 10 client.15807 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:41:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.128+0000 7f46ae1da740 10 client.15807 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:41:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.128+0000 7f46ae1da740 20 client.15807 populate_metadata read hostname 'smithi100' 2024-07-26T13:41:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.128+0000 7f46ae1da740 1 --2- 172.21.15.100:0/458348438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555cab9ae1b0 0x555cabde88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.128+0000 7f46ae1da740 4 client.15807 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-26T13:41:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.128+0000 7f46ae1da740 1 -- 172.21.15.100:0/458348438 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555cabcc3da0 con 0x555cab9ae1b0 2024-07-26T13:41:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.128+0000 7f469ffff640 1 --2- 172.21.15.100:0/458348438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555cab9ae1b0 0x555cabde88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:31.133 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.129+0000 7f467ffff640 10 client.15807 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:31.133 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.129+0000 7f469ffff640 1 --2- 172.21.15.100:0/458348438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555cab9ae1b0 0x555cabde88d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23f2f8dfb996f747 server_cookie=c57ea1ff6383abc7 in_seq=0 out_seq=0 2024-07-26T13:41:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.138+0000 7f467ffff640 1 -- 172.21.15.100:0/458348438 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555cabcc3da0 con 0x555cab9ae1b0 2024-07-26T13:41:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.138+0000 7f467ffff640 10 client.15807 handle_command_reply: tid=0 2024-07-26T13:41:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.138+0000 7f46ae1da740 1 client.15807 shutdown 2024-07-26T13:41:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.138+0000 7f46ae1da740 1 -- 172.21.15.100:0/458348438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cab96cd80 msgr2=0x555cabd82a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.138+0000 7f46ae1da740 1 --2- 172.21.15.100:0/458348438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cab96cd80 0x555cabd82a00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f46940ad800 tx=0x7f46940adbb0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.138+0000 7f46ae1da740 1 -- 172.21.15.100:0/458348438 shutdown_connections 2024-07-26T13:41:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.139+0000 7f46ae1da740 1 --2- 172.21.15.100:0/458348438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555cab9ae1b0 0x555cabde88d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.139+0000 7f467ffff640 0 client.15807 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.139+0000 7f46ae1da740 1 --2- 172.21.15.100:0/458348438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555cab98d4d0 0x555cabd81410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.139+0000 7f46ae1da740 1 --2- 172.21.15.100:0/458348438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555cab9852f0 0x555cabd80ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.139+0000 7f46ae1da740 1 --2- 172.21.15.100:0/458348438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555cab96cd80 0x555cabd82a00 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.139+0000 7f46ae1da740 1 -- 172.21.15.100:0/458348438 >> 172.21.15.100:0/458348438 conn(0x555cab9a2300 msgr2=0x555cabd4d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:31.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.139+0000 7f46ae1da740 1 -- 172.21.15.100:0/458348438 shutdown_connections 2024-07-26T13:41:31.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.139+0000 7f46ae1da740 1 -- 172.21.15.100:0/458348438 wait complete. 2024-07-26T13:41:31.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.139+0000 7f46ae1da740 20 client.15807 trim_cache size 0 max 16384 2024-07-26T13:41:31.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-07-26T13:41:31.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-07-26T13:41:31.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-07-26T13:41:31.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-07-26T13:41:31.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.140+0000 7f46a71e8640 1 -- 172.21.15.100:0/1123861893 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f46900b3db0 msgr2=0x7f46900b62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:31.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.140+0000 7f46a71e8640 1 --2- 172.21.15.100:0/1123861893 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f46900b3db0 0x7f46900b62c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4698005fd0 tx=0x7f4698009500 comp rx=0 tx=0).stop 2024-07-26T13:41:31.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.140+0000 7f46a71e8640 1 -- 172.21.15.100:0/1123861893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46a0102ba0 msgr2=0x7f46a0197d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:31.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.140+0000 7f46a71e8640 1 --2- 172.21.15.100:0/1123861893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46a0102ba0 0x7f46a0197d10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4694009c40 tx=0x7f469400b690 comp rx=0 tx=0).stop 2024-07-26T13:41:31.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.140+0000 7f46a71e8640 1 -- 172.21.15.100:0/1123861893 shutdown_connections 2024-07-26T13:41:31.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.140+0000 7f46a71e8640 1 --2- 172.21.15.100:0/1123861893 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f46900b3db0 0x7f46900b62c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.140+0000 7f46a71e8640 1 --2- 172.21.15.100:0/1123861893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46a01086b0 0x7f46a019d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.140+0000 7f46a71e8640 1 --2- 172.21.15.100:0/1123861893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46a0107d60 0x7f46a01982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.140+0000 7f46a71e8640 1 --2- 172.21.15.100:0/1123861893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46a0102ba0 0x7f46a0197d10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.141+0000 7f46a71e8640 1 -- 172.21.15.100:0/1123861893 >> 172.21.15.100:0/1123861893 conn(0x7f46a00798c0 msgr2=0x7f46a0103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:31.143 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.141+0000 7f46a71e8640 1 -- 172.21.15.100:0/1123861893 shutdown_connections 2024-07-26T13:41:31.144 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:31.141+0000 7f46a71e8640 1 -- 172.21.15.100:0/1123861893 wait complete. 2024-07-26T13:41:31.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:31 smithi167 ceph-mon[37140]: pgmap v406: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-07-26T13:41:31.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.091+0000 7f9c427e7640 1 -- 172.21.15.100:0/1463026963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c3c1035d0 msgr2=0x7f9c3c1039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:31.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.091+0000 7f9c427e7640 1 --2- 172.21.15.100:0/1463026963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c3c1035d0 0x7f9c3c1039e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c009860 tx=0x7f9c2c02a340 comp rx=0 tx=0).stop 2024-07-26T13:41:31.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.092+0000 7f9c427e7640 1 -- 172.21.15.100:0/1463026963 shutdown_connections 2024-07-26T13:41:31.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.092+0000 7f9c427e7640 1 --2- 172.21.15.100:0/1463026963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c3c109860 0x7f9c3c110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.092+0000 7f9c427e7640 1 --2- 172.21.15.100:0/1463026963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c3c108f10 0x7f9c3c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.092+0000 7f9c427e7640 1 --2- 172.21.15.100:0/1463026963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c3c1035d0 0x7f9c3c1039e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.092+0000 7f9c427e7640 1 -- 172.21.15.100:0/1463026963 >> 172.21.15.100:0/1463026963 conn(0x7f9c3c0ff060 msgr2=0x7f9c3c1014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:31.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.092+0000 7f9c427e7640 1 -- 172.21.15.100:0/1463026963 shutdown_connections 2024-07-26T13:41:31.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.092+0000 7f9c427e7640 1 -- 172.21.15.100:0/1463026963 wait complete. 2024-07-26T13:41:31.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.093+0000 7f9c427e7640 1 Processor -- start 2024-07-26T13:41:31.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.093+0000 7f9c427e7640 1 -- start start 2024-07-26T13:41:31.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.093+0000 7f9c427e7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c3c1035d0 0x7f9c3c10d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:31.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.093+0000 7f9c427e7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c3c108f10 0x7f9c3c10da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:31.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.093+0000 7f9c427e7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c3c109860 0x7f9c3c10df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:31.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.094+0000 7f9c3bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c3c1035d0 0x7f9c3c10d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:31.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.094+0000 7f9c3bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c3c1035d0 0x7f9c3c10d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:56084/0 (socket says 172.21.15.100:56084) 2024-07-26T13:41:31.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.094+0000 7f9c3b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c3c108f10 0x7f9c3c10da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:31.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.094+0000 7f9c427e7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3c076ce0 con 0x7f9c3c108f10 2024-07-26T13:41:31.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.094+0000 7f9c427e7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3c076b60 con 0x7f9c3c109860 2024-07-26T13:41:31.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.094+0000 7f9c427e7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3c076e60 con 0x7f9c3c1035d0 2024-07-26T13:41:31.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.094+0000 7f9c3b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c3c108f10 0x7f9c3c10da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50008/0 (socket says 172.21.15.100:50008) 2024-07-26T13:41:31.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.094+0000 7f9c3b7fe640 1 -- 172.21.15.100:0/481044981 learned_addr learned my addr 172.21.15.100:0/481044981 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:31.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.094+0000 7f9c40d84640 1 --2- 172.21.15.100:0/481044981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c3c109860 0x7f9c3c10df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:31.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.094+0000 7f9c3b7fe640 1 -- 172.21.15.100:0/481044981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c3c1035d0 msgr2=0x7f9c3c10d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:31.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.094+0000 7f9c3b7fe640 1 --2- 172.21.15.100:0/481044981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c3c1035d0 0x7f9c3c10d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.094+0000 7f9c3b7fe640 1 -- 172.21.15.100:0/481044981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c3c109860 msgr2=0x7f9c3c10df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:31.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.094+0000 7f9c3b7fe640 1 --2- 172.21.15.100:0/481044981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c3c109860 0x7f9c3c10df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.094+0000 7f9c3b7fe640 1 -- 172.21.15.100:0/481044981 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c3c1a1d50 con 0x7f9c3c108f10 2024-07-26T13:41:31.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.095+0000 7f9c3b7fe640 1 --2- 172.21.15.100:0/481044981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c3c108f10 0x7f9c3c10da30 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9c3000e970 tx=0x7f9c3000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:31.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.095+0000 7f9c397fa640 1 -- 172.21.15.100:0/481044981 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3000cd80 con 0x7f9c3c108f10 2024-07-26T13:41:31.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.095+0000 7f9c427e7640 1 -- 172.21.15.100:0/481044981 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c3c1a2040 con 0x7f9c3c108f10 2024-07-26T13:41:31.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.095+0000 7f9c427e7640 1 -- 172.21.15.100:0/481044981 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c3c1a25d0 con 0x7f9c3c108f10 2024-07-26T13:41:31.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.096+0000 7f9c397fa640 1 -- 172.21.15.100:0/481044981 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c300035f0 con 0x7f9c3c108f10 2024-07-26T13:41:31.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.096+0000 7f9c397fa640 1 -- 172.21.15.100:0/481044981 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c30010690 con 0x7f9c3c108f10 2024-07-26T13:41:31.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.099+0000 7f9c48d6b740 1 Processor -- start 2024-07-26T13:41:31.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.099+0000 7f9c48d6b740 1 -- start start 2024-07-26T13:41:31.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.099+0000 7f9c48d6b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560283384070 0x560283729570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:31.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.099+0000 7f9c48d6b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56028335b1b0 0x560283729ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:31.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.099+0000 7f9c48d6b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560283363390 0x560283729ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:31.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.099+0000 7f9c48d6b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56028369fbb0 con 0x56028335b1b0 2024-07-26T13:41:31.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.099+0000 7f9c48d6b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56028368f0b0 con 0x560283384070 2024-07-26T13:41:31.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.099+0000 7f9c48d6b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5602836930f0 con 0x560283363390 2024-07-26T13:41:31.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.100+0000 7f9c3bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56028335b1b0 0x560283729ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:31.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.100+0000 7f9c3bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56028335b1b0 0x560283729ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50032/0 (socket says 172.21.15.100:50032) 2024-07-26T13:41:31.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.100+0000 7f9c3bfff640 1 -- 172.21.15.100:0/3645489457 learned_addr learned my addr 172.21.15.100:0/3645489457 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:31.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.100+0000 7f9c40d84640 1 --2- 172.21.15.100:0/3645489457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560283384070 0x560283729570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:31.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.100+0000 7f9c40d84640 1 -- 172.21.15.100:0/3645489457 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560283363390 msgr2=0x560283729ff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-26T13:41:31.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.100+0000 7f9c40d84640 1 --2- 172.21.15.100:0/3645489457 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560283363390 0x560283729ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.100+0000 7f9c40d84640 1 -- 172.21.15.100:0/3645489457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56028335b1b0 msgr2=0x560283729ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:31.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.100+0000 7f9c40d84640 1 --2- 172.21.15.100:0/3645489457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56028335b1b0 0x560283729ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.100+0000 7f9c40d84640 1 -- 172.21.15.100:0/3645489457 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560283635b90 con 0x560283384070 2024-07-26T13:41:31.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.100+0000 7f9c3bfff640 1 --2- 172.21.15.100:0/3645489457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56028335b1b0 0x560283729ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:41:31.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.101+0000 7f9c397fa640 1 -- 172.21.15.100:0/481044981 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f9c30010830 con 0x7f9c3c108f10 2024-07-26T13:41:31.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.101+0000 7f9c40d84640 1 --2- 172.21.15.100:0/3645489457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560283384070 0x560283729570 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9c3400e450 tx=0x7f9c340095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d93626b27f77f664 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:31.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.101+0000 7f9c227fc640 1 -- 172.21.15.100:0/3645489457 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c34004b20 con 0x560283384070 2024-07-26T13:41:31.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.101+0000 7f9c227fc640 1 -- 172.21.15.100:0/3645489457 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c34004cc0 con 0x560283384070 2024-07-26T13:41:31.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.101+0000 7f9c227fc640 1 -- 172.21.15.100:0/3645489457 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c340055e0 con 0x560283384070 2024-07-26T13:41:31.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.102+0000 7f9c397fa640 1 --2- 172.21.15.100:0/481044981 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9c240b3e40 0x7f9c240b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:31.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.102+0000 7f9c3bfff640 1 --2- 172.21.15.100:0/481044981 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9c240b3e40 0x7f9c240b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:31.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.102+0000 7f9c48d6b740 1 -- 172.21.15.100:0/3645489457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560283384070 msgr2=0x560283729570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:31.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.102+0000 7f9c48d6b740 1 --2- 172.21.15.100:0/3645489457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560283384070 0x560283729570 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9c3400e450 tx=0x7f9c340095c0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.103+0000 7f9c48d6b740 1 -- 172.21.15.100:0/3645489457 shutdown_connections 2024-07-26T13:41:31.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.103+0000 7f9c48d6b740 1 --2- 172.21.15.100:0/3645489457 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560283363390 0x560283729ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.103+0000 7f9c48d6b740 1 --2- 172.21.15.100:0/3645489457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56028335b1b0 0x560283729ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.103+0000 7f9c48d6b740 1 --2- 172.21.15.100:0/3645489457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560283384070 0x560283729570 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.103+0000 7f9c48d6b740 1 -- 172.21.15.100:0/3645489457 >> 172.21.15.100:0/3645489457 conn(0x560283378460 msgr2=0x560283727460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:31.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.103+0000 7f9c3bfff640 1 --2- 172.21.15.100:0/481044981 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9c240b3e40 0x7f9c240b6350 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x5602837281f0 tx=0x7f9c2c02c040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:31.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.103+0000 7f9c397fa640 1 -- 172.21.15.100:0/481044981 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f9c300e4010 con 0x7f9c3c108f10 2024-07-26T13:41:31.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.104+0000 7f9c48d6b740 1 -- 172.21.15.100:0/3645489457 shutdown_connections 2024-07-26T13:41:31.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.104+0000 7f9c48d6b740 1 -- 172.21.15.100:0/3645489457 wait complete. 2024-07-26T13:41:31.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.104+0000 7f9c48d6b740 1 -- start start 2024-07-26T13:41:31.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.105+0000 7f9c48d6b740 1 Processor -- start 2024-07-26T13:41:31.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.105+0000 7f9c48d6b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028335b1b0 0x5602837351e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:31.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.105+0000 7f9c48d6b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560283363390 0x560283735760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:31.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.105+0000 7f9c3bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028335b1b0 0x5602837351e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:31.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.105+0000 7f9c3bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028335b1b0 0x5602837351e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:39582/0 (socket says 172.21.15.100:39582) 2024-07-26T13:41:31.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.105+0000 7f9c3b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560283363390 0x560283735760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:31.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.105+0000 7f9c3b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560283363390 0x560283735760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50038/0 (socket says 172.21.15.100:50038) 2024-07-26T13:41:31.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.105+0000 7f9c3bfff640 1 -- 172.21.15.100:0/866344694 learned_addr learned my addr 172.21.15.100:0/866344694 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:31.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.105+0000 7f9c48d6b740 1 --2- 172.21.15.100:0/866344694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560283384070 0x5602837410d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:31.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.106+0000 7f9c48d6b740 1 -- 172.21.15.100:0/866344694 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5602834f2700 con 0x560283363390 2024-07-26T13:41:31.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.106+0000 7f9c48d6b740 1 -- 172.21.15.100:0/866344694 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5602836319e0 con 0x56028335b1b0 2024-07-26T13:41:31.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.106+0000 7f9c48d6b740 1 -- 172.21.15.100:0/866344694 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560283737d50 con 0x560283384070 2024-07-26T13:41:31.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.106+0000 7f9c40d84640 1 --2- 172.21.15.100:0/866344694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560283384070 0x5602837410d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:31.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.106+0000 7f9c3bfff640 1 -- 172.21.15.100:0/866344694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560283384070 msgr2=0x5602837410d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:31.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.106+0000 7f9c3bfff640 1 --2- 172.21.15.100:0/866344694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560283384070 0x5602837410d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.106+0000 7f9c3bfff640 1 -- 172.21.15.100:0/866344694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560283363390 msgr2=0x560283735760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:31.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.106+0000 7f9c3bfff640 1 --2- 172.21.15.100:0/866344694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560283363390 0x560283735760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.106+0000 7f9c3bfff640 1 -- 172.21.15.100:0/866344694 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602833ef650 con 0x56028335b1b0 2024-07-26T13:41:31.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.106+0000 7f9c3b7fe640 1 --2- 172.21.15.100:0/866344694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560283363390 0x560283735760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:41:31.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.106+0000 7f9c48d6b740 10 client.0 _ll_register_callbacks cb 0x56028364c4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:41:31.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.107+0000 7f9c48d6b740 1 -- 172.21.15.100:0/866344694 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c34004cc0 con 0x56028335b1b0 2024-07-26T13:41:31.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.107+0000 7f9c3bfff640 1 --2- 172.21.15.100:0/866344694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028335b1b0 0x5602837351e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c009860 tx=0x7f9c2c02a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86f2bfd73a58cf1d server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:31.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.107+0000 7f9c1b7fe640 10 client.25265 ms_handle_connect on v2:172.21.15.138:3300/0 2024-07-26T13:41:31.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.107+0000 7f9c1b7fe640 1 -- 172.21.15.100:0/866344694 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c037040 con 0x56028335b1b0 2024-07-26T13:41:31.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.107+0000 7f9c1b7fe640 1 -- 172.21.15.100:0/866344694 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f9c2c0035f0 con 0x56028335b1b0 2024-07-26T13:41:31.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.107+0000 7f9c1b7fe640 1 -- 172.21.15.100:0/866344694 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9c2c003850 con 0x56028335b1b0 2024-07-26T13:41:31.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.107+0000 7f9c48d6b740 10 client.25265 fetch_fsmap learned FSMap version 26 2024-07-26T13:41:31.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.107+0000 7f9c48d6b740 1 -- 172.21.15.100:0/866344694 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({fsmap=26}) -- 0x560283743bf0 con 0x56028335b1b0 2024-07-26T13:41:31.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.108+0000 7f9c1b7fe640 1 -- 172.21.15.100:0/866344694 <== mon.2 v2:172.21.15.138:3300/0 4 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f9c2c0039f0 con 0x56028335b1b0 2024-07-26T13:41:31.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.108+0000 7f9c48d6b740 10 client.25265 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:41:31.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.108+0000 7f9c48d6b740 10 client.25265 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-26T13:41:31.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.108+0000 7f9c48d6b740 20 client.25265 populate_metadata read hostname 'smithi100' 2024-07-26T13:41:31.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.108+0000 7f9c48d6b740 1 --2- 172.21.15.100:0/866344694 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x5602833429f0 0x560283747160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:31.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.108+0000 7f9c48d6b740 4 client.25265 mds_command: new command op to 24595 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:41:31.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.108+0000 7f9c48d6b740 1 -- 172.21.15.100:0/866344694 --> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602836a1b00 con 0x5602833429f0 2024-07-26T13:41:31.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.108+0000 7f9c3b7fe640 1 --2- 172.21.15.100:0/866344694 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x5602833429f0 0x560283747160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:31.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.109+0000 7f9c1b7fe640 10 client.25265 ms_handle_connect on v2:172.21.15.167:6836/3828634468 2024-07-26T13:41:31.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.109+0000 7f9c3b7fe640 1 --2- 172.21.15.100:0/866344694 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x5602833429f0 0x560283747160 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=381864ae329322c8 server_cookie=e822a921569486c9 in_seq=0 out_seq=0 2024-07-26T13:41:31.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.110+0000 7f9c1b7fe640 1 -- 172.21.15.100:0/866344694 <== mds.2 v2:172.21.15.167:6836/3828634468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5602836a1b00 con 0x5602833429f0 2024-07-26T13:41:31.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.110+0000 7f9c1b7fe640 10 client.25265 handle_command_reply: tid=0 2024-07-26T13:41:31.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.110+0000 7f9c48d6b740 1 client.25265 shutdown 2024-07-26T13:41:31.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.110+0000 7f9c48d6b740 1 -- 172.21.15.100:0/866344694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028335b1b0 msgr2=0x5602837351e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:31.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.110+0000 7f9c48d6b740 1 --2- 172.21.15.100:0/866344694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028335b1b0 0x5602837351e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c009860 tx=0x7f9c2c02a340 comp rx=0 tx=0).stop 2024-07-26T13:41:31.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.110+0000 7f9c48d6b740 1 -- 172.21.15.100:0/866344694 shutdown_connections 2024-07-26T13:41:31.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.110+0000 7f9c48d6b740 1 --2- 172.21.15.100:0/866344694 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x5602833429f0 0x560283747160 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.110+0000 7f9c48d6b740 1 --2- 172.21.15.100:0/866344694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560283384070 0x5602837410d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.110+0000 7f9c48d6b740 1 --2- 172.21.15.100:0/866344694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560283363390 0x560283735760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.110+0000 7f9c1b7fe640 0 client.25265 ms_handle_reset on v2:172.21.15.167:6836/3828634468 2024-07-26T13:41:31.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.110+0000 7f9c48d6b740 1 --2- 172.21.15.100:0/866344694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028335b1b0 0x5602837351e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.912 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-07-26T13:41:31.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.111+0000 7f9c48d6b740 1 -- 172.21.15.100:0/866344694 >> 172.21.15.100:0/866344694 conn(0x560283378460 msgr2=0x560283725c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:31.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.111+0000 7f9c48d6b740 1 -- 172.21.15.100:0/866344694 shutdown_connections 2024-07-26T13:41:31.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.111+0000 7f9c48d6b740 1 -- 172.21.15.100:0/866344694 wait complete. 2024-07-26T13:41:31.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.111+0000 7f9c48d6b740 20 client.25265 trim_cache size 0 max 16384 2024-07-26T13:41:31.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.137+0000 7f9c48d6b740 1 Processor -- start 2024-07-26T13:41:31.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.137+0000 7f9c48d6b740 1 -- start start 2024-07-26T13:41:31.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.137+0000 7f9c48d6b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5602833429f0 0x560283733860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:31.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.137+0000 7f9c48d6b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028335b1b0 0x560283726e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:31.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.137+0000 7f9c48d6b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560283363390 0x560283727390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:31.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.137+0000 7f9c48d6b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9c28011060 con 0x560283363390 2024-07-26T13:41:31.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.137+0000 7f9c48d6b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2800e930 con 0x56028335b1b0 2024-07-26T13:41:31.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.137+0000 7f9c48d6b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2800c780 con 0x5602833429f0 2024-07-26T13:41:31.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.137+0000 7f9c3bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5602833429f0 0x560283733860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:31.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.137+0000 7f9c3bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5602833429f0 0x560283733860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:56128/0 (socket says 172.21.15.100:56128) 2024-07-26T13:41:31.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.137+0000 7f9c3bfff640 1 -- 172.21.15.100:0/1854998228 learned_addr learned my addr 172.21.15.100:0/1854998228 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:31.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.137+0000 7f9c3b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028335b1b0 0x560283726e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:31.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.137+0000 7f9c40d84640 1 --2- 172.21.15.100:0/1854998228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560283363390 0x560283727390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:31.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.138+0000 7f9c3b7fe640 1 -- 172.21.15.100:0/1854998228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5602833429f0 msgr2=0x560283733860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:31.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.138+0000 7f9c3b7fe640 1 --2- 172.21.15.100:0/1854998228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5602833429f0 0x560283733860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.138+0000 7f9c3b7fe640 1 -- 172.21.15.100:0/1854998228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560283363390 msgr2=0x560283727390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:31.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.138+0000 7f9c3b7fe640 1 --2- 172.21.15.100:0/1854998228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560283363390 0x560283727390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.138+0000 7f9c3b7fe640 1 -- 172.21.15.100:0/1854998228 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560283728220 con 0x56028335b1b0 2024-07-26T13:41:31.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.138+0000 7f9c3b7fe640 1 --2- 172.21.15.100:0/1854998228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028335b1b0 0x560283726e50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9c300e1720 tx=0x7f9c300e3040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a45958613a21d84 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:31.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.138+0000 7f9c21ffb640 1 -- 172.21.15.100:0/1854998228 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c300cb070 con 0x56028335b1b0 2024-07-26T13:41:31.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.138+0000 7f9c21ffb640 1 -- 172.21.15.100:0/1854998228 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c300aabe0 con 0x56028335b1b0 2024-07-26T13:41:31.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.138+0000 7f9c21ffb640 1 -- 172.21.15.100:0/1854998228 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c30010b20 con 0x56028335b1b0 2024-07-26T13:41:31.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.139+0000 7f9c48d6b740 1 -- 172.21.15.100:0/1854998228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028335b1b0 msgr2=0x560283726e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:31.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.139+0000 7f9c48d6b740 1 --2- 172.21.15.100:0/1854998228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028335b1b0 0x560283726e50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9c300e1720 tx=0x7f9c300e3040 comp rx=0 tx=0).stop 2024-07-26T13:41:31.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.140+0000 7f9c48d6b740 1 -- 172.21.15.100:0/1854998228 shutdown_connections 2024-07-26T13:41:31.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.140+0000 7f9c48d6b740 1 --2- 172.21.15.100:0/1854998228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560283363390 0x560283727390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.140+0000 7f9c48d6b740 1 --2- 172.21.15.100:0/1854998228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028335b1b0 0x560283726e50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.140+0000 7f9c48d6b740 1 --2- 172.21.15.100:0/1854998228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5602833429f0 0x560283733860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.140+0000 7f9c48d6b740 1 -- 172.21.15.100:0/1854998228 >> 172.21.15.100:0/1854998228 conn(0x560283378460 msgr2=0x560283730190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:31.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.140+0000 7f9c48d6b740 1 -- 172.21.15.100:0/1854998228 shutdown_connections 2024-07-26T13:41:31.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.140+0000 7f9c48d6b740 1 -- 172.21.15.100:0/1854998228 wait complete. 2024-07-26T13:41:31.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.140+0000 7f9c48d6b740 1 -- start start 2024-07-26T13:41:31.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.141+0000 7f9c48d6b740 1 Processor -- start 2024-07-26T13:41:31.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.141+0000 7f9c48d6b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5602833429f0 0x56028373d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:31.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.141+0000 7f9c48d6b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028335b1b0 0x560283738cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:31.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.141+0000 7f9c48d6b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560283363390 0x560283739390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:31.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.141+0000 7f9c48d6b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2800cdf0 con 0x560283363390 2024-07-26T13:41:31.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.141+0000 7f9c48d6b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2800e260 con 0x56028335b1b0 2024-07-26T13:41:31.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.141+0000 7f9c48d6b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2800d4c0 con 0x5602833429f0 2024-07-26T13:41:31.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.141+0000 7f9c48d6b740 10 client.0 _ll_register_callbacks cb 0x7f9c34009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:41:31.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.141+0000 7f9c3bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5602833429f0 0x56028373d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:31.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.141+0000 7f9c3b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028335b1b0 0x560283738cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:31.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.142+0000 7f9c3bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5602833429f0 0x56028373d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:56132/0 (socket says 172.21.15.100:56132) 2024-07-26T13:41:31.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.142+0000 7f9c3bfff640 1 -- 172.21.15.100:0/393209597 learned_addr learned my addr 172.21.15.100:0/393209597 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:31.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.142+0000 7f9c40d84640 1 --2- 172.21.15.100:0/393209597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560283363390 0x560283739390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:31.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.142+0000 7f9c3bfff640 1 -- 172.21.15.100:0/393209597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028335b1b0 msgr2=0x560283738cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:31.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.142+0000 7f9c3bfff640 1 --2- 172.21.15.100:0/393209597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028335b1b0 0x560283738cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.142+0000 7f9c3bfff640 1 -- 172.21.15.100:0/393209597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560283363390 msgr2=0x560283739390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:31.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.142+0000 7f9c3bfff640 1 --2- 172.21.15.100:0/393209597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560283363390 0x560283739390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.142+0000 7f9c3bfff640 1 -- 172.21.15.100:0/393209597 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560283739b00 con 0x5602833429f0 2024-07-26T13:41:31.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.142+0000 7f9c3bfff640 1 --2- 172.21.15.100:0/393209597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5602833429f0 0x56028373d840 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c009860 tx=0x7f9c2c03f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41d44f40abee3444 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:31.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.142+0000 7f9c1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-07-26T13:41:31.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.143+0000 7f9c1b7fe640 1 -- 172.21.15.100:0/393209597 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c060060 con 0x5602833429f0 2024-07-26T13:41:31.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.143+0000 7f9c1b7fe640 1 -- 172.21.15.100:0/393209597 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f9c2c048690 con 0x5602833429f0 2024-07-26T13:41:31.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.143+0000 7f9c48d6b740 1 -- 172.21.15.100:0/393209597 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c300aabe0 con 0x5602833429f0 2024-07-26T13:41:31.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.143+0000 7f9c1b7fe640 1 -- 172.21.15.100:0/393209597 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9c2c037050 con 0x5602833429f0 2024-07-26T13:41:31.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.143+0000 7f9c48d6b740 10 client.25378 fetch_fsmap learned FSMap version 26 2024-07-26T13:41:31.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.143+0000 7f9c48d6b740 1 -- 172.21.15.100:0/393209597 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5602837af9b0 con 0x5602833429f0 2024-07-26T13:41:31.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.143+0000 7f9c1b7fe640 1 -- 172.21.15.100:0/393209597 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c300aabe0 con 0x5602833429f0 2024-07-26T13:41:31.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.144+0000 7f9c1b7fe640 1 -- 172.21.15.100:0/393209597 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f9c2c05a070 con 0x5602833429f0 2024-07-26T13:41:31.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.144+0000 7f9c48d6b740 10 client.25378 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:41:31.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.144+0000 7f9c48d6b740 10 client.25378 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-26T13:41:31.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.144+0000 7f9c48d6b740 20 client.25378 populate_metadata read hostname 'smithi100' 2024-07-26T13:41:31.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.144+0000 7f9c48d6b740 1 --2- 172.21.15.100:0/393209597 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x560283384070 0x5602837b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:31.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.144+0000 7f9c48d6b740 4 client.25378 mds_command: new command op to 24595 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-26T13:41:31.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.144+0000 7f9c48d6b740 1 -- 172.21.15.100:0/393209597 --> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56028369d5a0 con 0x560283384070 2024-07-26T13:41:31.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.144+0000 7f9c3b7fe640 1 --2- 172.21.15.100:0/393209597 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x560283384070 0x5602837b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:31.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.145+0000 7f9c1b7fe640 10 client.25378 ms_handle_connect on v2:172.21.15.167:6836/3828634468 2024-07-26T13:41:31.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.145+0000 7f9c3b7fe640 1 --2- 172.21.15.100:0/393209597 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x560283384070 0x5602837b2140 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=885fcbacf0ecbe70 server_cookie=58f3d983fecccdff in_seq=0 out_seq=0 2024-07-26T13:41:31.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.145+0000 7f9c1b7fe640 1 -- 172.21.15.100:0/393209597 <== mds.2 v2:172.21.15.167:6836/3828634468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56028369d5a0 con 0x560283384070 2024-07-26T13:41:31.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.145+0000 7f9c1b7fe640 10 client.25378 handle_command_reply: tid=0 2024-07-26T13:41:31.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.145+0000 7f9c48d6b740 1 client.25378 shutdown 2024-07-26T13:41:31.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.146+0000 7f9c48d6b740 1 -- 172.21.15.100:0/393209597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5602833429f0 msgr2=0x56028373d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:31.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.146+0000 7f9c48d6b740 1 --2- 172.21.15.100:0/393209597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5602833429f0 0x56028373d840 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c009860 tx=0x7f9c2c03f000 comp rx=0 tx=0).stop 2024-07-26T13:41:31.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.146+0000 7f9c48d6b740 1 -- 172.21.15.100:0/393209597 shutdown_connections 2024-07-26T13:41:31.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.146+0000 7f9c48d6b740 1 --2- 172.21.15.100:0/393209597 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x560283384070 0x5602837b2140 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.146+0000 7f9c48d6b740 1 --2- 172.21.15.100:0/393209597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560283363390 0x560283739390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.146+0000 7f9c48d6b740 1 --2- 172.21.15.100:0/393209597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56028335b1b0 0x560283738cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.146+0000 7f9c48d6b740 1 --2- 172.21.15.100:0/393209597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5602833429f0 0x56028373d840 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.146+0000 7f9c1b7fe640 0 client.25378 ms_handle_reset on v2:172.21.15.167:6836/3828634468 2024-07-26T13:41:31.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.146+0000 7f9c48d6b740 1 -- 172.21.15.100:0/393209597 >> 172.21.15.100:0/393209597 conn(0x560283378460 msgr2=0x560283725c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:31.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.147+0000 7f9c48d6b740 1 -- 172.21.15.100:0/393209597 shutdown_connections 2024-07-26T13:41:31.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.147+0000 7f9c48d6b740 1 -- 172.21.15.100:0/393209597 wait complete. 2024-07-26T13:41:31.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.147+0000 7f9c48d6b740 20 client.25378 trim_cache size 0 max 16384 2024-07-26T13:41:31.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.147+0000 7f9c427e7640 1 -- 172.21.15.100:0/481044981 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9c240b3e40 msgr2=0x7f9c240b6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:31.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.147+0000 7f9c427e7640 1 --2- 172.21.15.100:0/481044981 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9c240b3e40 0x7f9c240b6350 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x5602837281f0 tx=0x7f9c2c02c040 comp rx=0 tx=0).stop 2024-07-26T13:41:31.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.147+0000 7f9c427e7640 1 -- 172.21.15.100:0/481044981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c3c108f10 msgr2=0x7f9c3c10da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:31.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.147+0000 7f9c427e7640 1 --2- 172.21.15.100:0/481044981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c3c108f10 0x7f9c3c10da30 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9c3000e970 tx=0x7f9c3000ee30 comp rx=0 tx=0).stop 2024-07-26T13:41:31.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.148+0000 7f9c427e7640 1 -- 172.21.15.100:0/481044981 shutdown_connections 2024-07-26T13:41:31.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.148+0000 7f9c427e7640 1 --2- 172.21.15.100:0/481044981 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9c240b3e40 0x7f9c240b6350 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.148+0000 7f9c427e7640 1 --2- 172.21.15.100:0/481044981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c3c109860 0x7f9c3c10df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.148+0000 7f9c427e7640 1 --2- 172.21.15.100:0/481044981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c3c108f10 0x7f9c3c10da30 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.148+0000 7f9c427e7640 1 --2- 172.21.15.100:0/481044981 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c3c1035d0 0x7f9c3c10d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:31.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.148+0000 7f9c427e7640 1 -- 172.21.15.100:0/481044981 >> 172.21.15.100:0/481044981 conn(0x7f9c3c0ff060 msgr2=0x7f9c3c101480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:31.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.148+0000 7f9c427e7640 1 -- 172.21.15.100:0/481044981 shutdown_connections 2024-07-26T13:41:31.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:31.148+0000 7f9c427e7640 1 -- 172.21.15.100:0/481044981 wait complete. 2024-07-26T13:41:31.920 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:31 smithi100 ceph-mon[32599]: pgmap v406: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-07-26T13:41:31.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:31 smithi138 ceph-mon[36954]: pgmap v406: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-07-26T13:41:33.815 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:41:33.848 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:33 smithi100 ceph-mon[32599]: pgmap v407: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-07-26T13:41:33.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:33 smithi138 ceph-mon[36954]: pgmap v407: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-07-26T13:41:34.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:33 smithi167 ceph-mon[37140]: pgmap v407: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-07-26T13:41:34.043 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-26T13:41:34.244 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:41:35.576 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:41:35.851 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:35 smithi100 ceph-mon[32599]: pgmap v408: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.6 KiB/s wr, 9 op/s 2024-07-26T13:41:35.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:35 smithi138 ceph-mon[36954]: pgmap v408: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.6 KiB/s wr, 9 op/s 2024-07-26T13:41:36.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:35 smithi167 ceph-mon[37140]: pgmap v408: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.6 KiB/s wr, 9 op/s 2024-07-26T13:41:37.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.234+0000 7fd31f92e640 1 -- 172.21.15.100:0/166536959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd318106600 msgr2=0x7fd318110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:37.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.234+0000 7fd31f92e640 1 --2- 172.21.15.100:0/166536959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd318106600 0x7fd318110090 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd310009ca0 tx=0x7fd31002a8c0 comp rx=0 tx=0).stop 2024-07-26T13:41:37.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.234+0000 7fd31f92e640 1 -- 172.21.15.100:0/166536959 shutdown_connections 2024-07-26T13:41:37.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.234+0000 7fd31f92e640 1 --2- 172.21.15.100:0/166536959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd318106600 0x7fd318110090 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:37.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.234+0000 7fd31f92e640 1 --2- 172.21.15.100:0/166536959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd318105cb0 0x7fd3181060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:37.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.234+0000 7fd31f92e640 1 --2- 172.21.15.100:0/166536959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd318101a60 0x7fd318101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:37.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.234+0000 7fd31f92e640 1 -- 172.21.15.100:0/166536959 >> 172.21.15.100:0/166536959 conn(0x7fd318077690 msgr2=0x7fd318077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:37.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.234+0000 7fd31f92e640 1 -- 172.21.15.100:0/166536959 shutdown_connections 2024-07-26T13:41:37.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.234+0000 7fd31f92e640 1 -- 172.21.15.100:0/166536959 wait complete. 2024-07-26T13:41:37.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.235+0000 7fd31f92e640 1 Processor -- start 2024-07-26T13:41:37.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.236+0000 7fd31f92e640 1 -- start start 2024-07-26T13:41:37.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.236+0000 7fd31f92e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd318101a60 0x7fd318199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:37.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.236+0000 7fd31f92e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd318105cb0 0x7fd31819a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:37.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.236+0000 7fd31f92e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd318106600 0x7fd31819f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:37.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.236+0000 7fd31f92e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd318113520 con 0x7fd318105cb0 2024-07-26T13:41:37.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.236+0000 7fd31f92e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd3181133a0 con 0x7fd318101a60 2024-07-26T13:41:37.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.236+0000 7fd31f92e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd3181136a0 con 0x7fd318106600 2024-07-26T13:41:37.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.236+0000 7fd31cec9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd318105cb0 0x7fd31819a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:37.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.236+0000 7fd31d6ca640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd318101a60 0x7fd318199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:37.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.236+0000 7fd31d6ca640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd318101a60 0x7fd318199e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:55642/0 (socket says 172.21.15.100:55642) 2024-07-26T13:41:37.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.236+0000 7fd31d6ca640 1 -- 172.21.15.100:0/2113341940 learned_addr learned my addr 172.21.15.100:0/2113341940 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:37.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.237+0000 7fd31decb640 1 --2- 172.21.15.100:0/2113341940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd318106600 0x7fd31819f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:37.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.237+0000 7fd31d6ca640 1 -- 172.21.15.100:0/2113341940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd318106600 msgr2=0x7fd31819f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:37.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.237+0000 7fd31d6ca640 1 --2- 172.21.15.100:0/2113341940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd318106600 0x7fd31819f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:37.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.237+0000 7fd31d6ca640 1 -- 172.21.15.100:0/2113341940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd318105cb0 msgr2=0x7fd31819a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:37.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.237+0000 7fd31d6ca640 1 --2- 172.21.15.100:0/2113341940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd318105cb0 0x7fd31819a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:37.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.237+0000 7fd31d6ca640 1 -- 172.21.15.100:0/2113341940 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31819fda0 con 0x7fd318101a60 2024-07-26T13:41:37.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.237+0000 7fd31decb640 1 --2- 172.21.15.100:0/2113341940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd318106600 0x7fd31819f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:41:37.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.237+0000 7fd31cec9640 1 --2- 172.21.15.100:0/2113341940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd318105cb0 0x7fd31819a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:41:37.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.238+0000 7fd31d6ca640 1 --2- 172.21.15.100:0/2113341940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd318101a60 0x7fd318199e30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd31400e9c0 tx=0x7fd31400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:37.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.238+0000 7fd30e7fc640 1 -- 172.21.15.100:0/2113341940 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31400cd80 con 0x7fd318101a60 2024-07-26T13:41:37.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.238+0000 7fd31f92e640 1 -- 172.21.15.100:0/2113341940 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3181a0090 con 0x7fd318101a60 2024-07-26T13:41:37.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.238+0000 7fd30e7fc640 1 -- 172.21.15.100:0/2113341940 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3140035f0 con 0x7fd318101a60 2024-07-26T13:41:37.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.238+0000 7fd30e7fc640 1 -- 172.21.15.100:0/2113341940 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd314010690 con 0x7fd318101a60 2024-07-26T13:41:37.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.238+0000 7fd31f92e640 1 -- 172.21.15.100:0/2113341940 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3181028e0 con 0x7fd318101a60 2024-07-26T13:41:37.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.239+0000 7fd31f92e640 1 -- 172.21.15.100:0/2113341940 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd318101e70 con 0x7fd318101a60 2024-07-26T13:41:37.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.242+0000 7fd30e7fc640 1 -- 172.21.15.100:0/2113341940 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fd3140037a0 con 0x7fd318101a60 2024-07-26T13:41:37.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.243+0000 7fd30e7fc640 1 --2- 172.21.15.100:0/2113341940 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd3040b3d40 0x7fd3040b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:37.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.243+0000 7fd31cec9640 1 --2- 172.21.15.100:0/2113341940 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd3040b3d40 0x7fd3040b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:37.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.243+0000 7fd30e7fc640 1 -- 172.21.15.100:0/2113341940 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fd3140e4530 con 0x7fd318101a60 2024-07-26T13:41:37.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.244+0000 7fd31cec9640 1 --2- 172.21.15.100:0/2113341940 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd3040b3d40 0x7fd3040b6250 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd3080095f0 tx=0x7fd308005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:37.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.245+0000 7fd30e7fc640 1 -- 172.21.15.100:0/2113341940 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fd3140ab800 con 0x7fd318101a60 2024-07-26T13:41:37.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.483+0000 7fd31f92e640 1 -- 172.21.15.100:0/2113341940 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd318102bf0 con 0x7fd318101a60 2024-07-26T13:41:37.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.484+0000 7fd30e7fc640 1 -- 172.21.15.100:0/2113341940 <== mon.2 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11927 (secure 0 0 0) 0x7fd3140ab100 con 0x7fd318101a60 2024-07-26T13:41:37.486 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:41:37.486 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":26,"btime":"2024-07-26T13:40:09:530496+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14790,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/313501409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":313501409},{"type":"v1","addr":"172.21.15.100:6839","nonce":313501409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/1402975506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":1402975506},{"type":"v1","addr":"172.21.15.100:6841","nonce":1402975506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-26T13:37:01.281843+0000","modified":"2024-07-26T13:40:09.187125+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24607,"mds_1":24605,"mds_2":24595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24557":{"gid":24557,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/3290105300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3290105300},{"type":"v1","addr":"172.21.15.138:6835","nonce":3290105300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/3393225483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3393225483},{"type":"v1","addr":"172.21.15.138:6837","nonce":3393225483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24595":{"gid":24595,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6837/3828634468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3828634468},{"type":"v1","addr":"172.21.15.167:6837","nonce":3828634468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.138:6839/1580925471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1580925471},{"type":"v1","addr":"172.21.15.138:6839","nonce":1580925471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/398150514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":398150514},{"type":"v1","addr":"172.21.15.167:6839","nonce":398150514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24607,"qdb_cluster":[24595,24605,24607]},"id":1}]} 2024-07-26T13:41:37.486 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 26 2024-07-26T13:41:37.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.487+0000 7fd31f92e640 1 -- 172.21.15.100:0/2113341940 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd3040b3d40 msgr2=0x7fd3040b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:37.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.487+0000 7fd31f92e640 1 --2- 172.21.15.100:0/2113341940 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd3040b3d40 0x7fd3040b6250 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd3080095f0 tx=0x7fd308005f90 comp rx=0 tx=0).stop 2024-07-26T13:41:37.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.487+0000 7fd31f92e640 1 -- 172.21.15.100:0/2113341940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd318101a60 msgr2=0x7fd318199e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:37.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.487+0000 7fd31f92e640 1 --2- 172.21.15.100:0/2113341940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd318101a60 0x7fd318199e30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd31400e9c0 tx=0x7fd31400ee80 comp rx=0 tx=0).stop 2024-07-26T13:41:37.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.487+0000 7fd31f92e640 1 -- 172.21.15.100:0/2113341940 shutdown_connections 2024-07-26T13:41:37.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.487+0000 7fd31f92e640 1 --2- 172.21.15.100:0/2113341940 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd3040b3d40 0x7fd3040b6250 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:37.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.487+0000 7fd31f92e640 1 --2- 172.21.15.100:0/2113341940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd318106600 0x7fd31819f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:37.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.487+0000 7fd31f92e640 1 --2- 172.21.15.100:0/2113341940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd318105cb0 0x7fd31819a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:37.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.487+0000 7fd31f92e640 1 --2- 172.21.15.100:0/2113341940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd318101a60 0x7fd318199e30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:37.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.487+0000 7fd31f92e640 1 -- 172.21.15.100:0/2113341940 >> 172.21.15.100:0/2113341940 conn(0x7fd318077690 msgr2=0x7fd318106e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:37.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.487+0000 7fd31f92e640 1 -- 172.21.15.100:0/2113341940 shutdown_connections 2024-07-26T13:41:37.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:37.487+0000 7fd31f92e640 1 -- 172.21.15.100:0/2113341940 wait complete. 2024-07-26T13:41:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:37 smithi100 ceph-mon[32599]: pgmap v409: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-07-26T13:41:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:37 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2113341940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:41:37.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:37 smithi138 ceph-mon[36954]: pgmap v409: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-07-26T13:41:37.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:37 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/2113341940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:41:38.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:37 smithi167 ceph-mon[37140]: pgmap v409: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-07-26T13:41:38.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:37 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/2113341940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:41:38.294 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24607 quiesce db --set-id 28b446d7 --release --await' 2024-07-26T13:41:38.295 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- timeout 105 ceph tell mds.24607 quiesce db --set-id 28b446d7 --release --await 2024-07-26T13:41:38.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.491+0000 7f3423e87640 1 -- 172.21.15.100:0/2395995555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f341c107de0 msgr2=0x7f341c1081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:38.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.491+0000 7f3423e87640 1 --2- 172.21.15.100:0/2395995555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f341c107de0 0x7f341c1081f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f340c009900 tx=0x7f340c02a410 comp rx=0 tx=0).stop 2024-07-26T13:41:38.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.492+0000 7f3423e87640 1 -- 172.21.15.100:0/2395995555 shutdown_connections 2024-07-26T13:41:38.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.492+0000 7f3423e87640 1 --2- 172.21.15.100:0/2395995555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f341c108730 0x7f341c10ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:38.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.492+0000 7f3423e87640 1 --2- 172.21.15.100:0/2395995555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f341c107de0 0x7f341c1081f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:38.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.492+0000 7f3423e87640 1 --2- 172.21.15.100:0/2395995555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f341c1024a0 0x7f341c1028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:38.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.492+0000 7f3423e87640 1 -- 172.21.15.100:0/2395995555 >> 172.21.15.100:0/2395995555 conn(0x7f341c079160 msgr2=0x7f341c079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:38.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.492+0000 7f3423e87640 1 -- 172.21.15.100:0/2395995555 shutdown_connections 2024-07-26T13:41:38.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.492+0000 7f3423e87640 1 -- 172.21.15.100:0/2395995555 wait complete. 2024-07-26T13:41:38.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.493+0000 7f3423e87640 1 Processor -- start 2024-07-26T13:41:38.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.493+0000 7f3423e87640 1 -- start start 2024-07-26T13:41:38.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.493+0000 7f3423e87640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f341c1024a0 0x7f341c197cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:38.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.493+0000 7f3423e87640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f341c107de0 0x7f341c198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:38.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.493+0000 7f3423e87640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f341c108730 0x7f341c19d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:38.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.493+0000 7f3423e87640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f341c1112e0 con 0x7f341c108730 2024-07-26T13:41:38.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.493+0000 7f3423e87640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f341c111160 con 0x7f341c1024a0 2024-07-26T13:41:38.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.493+0000 7f3423e87640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f341c111460 con 0x7f341c107de0 2024-07-26T13:41:38.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.494+0000 7f3421422640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f341c107de0 0x7f341c198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:38.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.494+0000 7f3421422640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f341c107de0 0x7f341c198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:37078/0 (socket says 172.21.15.100:37078) 2024-07-26T13:41:38.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.494+0000 7f3421422640 1 -- 172.21.15.100:0/4260419 learned_addr learned my addr 172.21.15.100:0/4260419 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:38.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.494+0000 7f3422424640 1 --2- 172.21.15.100:0/4260419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f341c108730 0x7f341c19d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:38.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.494+0000 7f3421c23640 1 --2- 172.21.15.100:0/4260419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f341c1024a0 0x7f341c197cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:38.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.494+0000 7f3421422640 1 -- 172.21.15.100:0/4260419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f341c1024a0 msgr2=0x7f341c197cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:38.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.494+0000 7f3421422640 1 --2- 172.21.15.100:0/4260419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f341c1024a0 0x7f341c197cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:38.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.494+0000 7f3421422640 1 -- 172.21.15.100:0/4260419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f341c108730 msgr2=0x7f341c19d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:38.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.494+0000 7f3421422640 1 --2- 172.21.15.100:0/4260419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f341c108730 0x7f341c19d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:38.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.494+0000 7f3421422640 1 -- 172.21.15.100:0/4260419 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f341c19dc40 con 0x7f341c107de0 2024-07-26T13:41:38.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.494+0000 7f3422424640 1 --2- 172.21.15.100:0/4260419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f341c108730 0x7f341c19d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:41:38.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.495+0000 7f3421422640 1 --2- 172.21.15.100:0/4260419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f341c107de0 0x7f341c198270 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f340c005ab0 tx=0x7f340c0035f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:38.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.495+0000 7f3412ffd640 1 -- 172.21.15.100:0/4260419 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340c03e070 con 0x7f341c107de0 2024-07-26T13:41:38.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.495+0000 7f3423e87640 1 -- 172.21.15.100:0/4260419 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f341c19ded0 con 0x7f341c107de0 2024-07-26T13:41:38.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.495+0000 7f3423e87640 1 -- 172.21.15.100:0/4260419 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f341c1a4690 con 0x7f341c107de0 2024-07-26T13:41:38.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.495+0000 7f3412ffd640 1 -- 172.21.15.100:0/4260419 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f340c040770 con 0x7f341c107de0 2024-07-26T13:41:38.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.495+0000 7f3412ffd640 1 -- 172.21.15.100:0/4260419 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340c035070 con 0x7f341c107de0 2024-07-26T13:41:38.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.499+0000 7f342a40b740 1 Processor -- start 2024-07-26T13:41:38.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.499+0000 7f342a40b740 1 -- start start 2024-07-26T13:41:38.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.499+0000 7f342a40b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623c8d3a070 0x5623c90df690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:38.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.499+0000 7f342a40b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623c8d111b0 0x5623c90dfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:38.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.499+0000 7f342a40b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623c8d19390 0x5623c90e0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:38.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.499+0000 7f342a40b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5623c9055bb0 con 0x5623c8d3a070 2024-07-26T13:41:38.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.499+0000 7f342a40b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5623c90450b0 con 0x5623c8d111b0 2024-07-26T13:41:38.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.499+0000 7f342a40b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5623c90490f0 con 0x5623c8d19390 2024-07-26T13:41:38.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.500+0000 7f3421c23640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623c8d111b0 0x5623c90dfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:38.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.500+0000 7f3421c23640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623c8d111b0 0x5623c90dfbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:55670/0 (socket says 172.21.15.100:55670) 2024-07-26T13:41:38.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.500+0000 7f3421c23640 1 -- 172.21.15.100:0/1332401016 learned_addr learned my addr 172.21.15.100:0/1332401016 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:38.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.500+0000 7f3412ffd640 1 -- 172.21.15.100:0/4260419 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f340c031a70 con 0x7f341c107de0 2024-07-26T13:41:38.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.500+0000 7f3422424640 1 --2- 172.21.15.100:0/1332401016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623c8d3a070 0x5623c90df690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:38.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.500+0000 7f3421422640 1 --2- 172.21.15.100:0/1332401016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623c8d19390 0x5623c90e0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:38.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.500+0000 7f3421c23640 1 -- 172.21.15.100:0/1332401016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623c8d19390 msgr2=0x5623c90e0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:38.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.501+0000 7f3421c23640 1 --2- 172.21.15.100:0/1332401016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623c8d19390 0x5623c90e0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:38.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.501+0000 7f3421c23640 1 -- 172.21.15.100:0/1332401016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623c8d3a070 msgr2=0x5623c90df690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:38.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.501+0000 7f3421c23640 1 --2- 172.21.15.100:0/1332401016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623c8d3a070 0x5623c90df690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:38.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.501+0000 7f3421c23640 1 -- 172.21.15.100:0/1332401016 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623c8febb90 con 0x5623c8d111b0 2024-07-26T13:41:38.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.501+0000 7f3412ffd640 1 --2- 172.21.15.100:0/4260419 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f34080b3e40 0x7f34080b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:38.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.501+0000 7f3421c23640 1 --2- 172.21.15.100:0/1332401016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623c8d111b0 0x5623c90dfbd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3418009d20 tx=0x7f3418011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5efb863f2a68b4e2 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:38.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.502+0000 7f3412ffd640 1 -- 172.21.15.100:0/4260419 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f340c1154e0 con 0x7f341c107de0 2024-07-26T13:41:38.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.502+0000 7f3422424640 1 --2- 172.21.15.100:0/4260419 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f34080b3e40 0x7f34080b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:38.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.502+0000 7f3407fff640 1 -- 172.21.15.100:0/1332401016 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341801e070 con 0x5623c8d111b0 2024-07-26T13:41:38.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.502+0000 7f3407fff640 1 -- 172.21.15.100:0/1332401016 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3418019ae0 con 0x5623c8d111b0 2024-07-26T13:41:38.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.502+0000 7f3407fff640 1 -- 172.21.15.100:0/1332401016 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341800ea80 con 0x5623c8d111b0 2024-07-26T13:41:38.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.503+0000 7f342a40b740 1 -- 172.21.15.100:0/1332401016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623c8d111b0 msgr2=0x5623c90dfbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:38.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.503+0000 7f342a40b740 1 --2- 172.21.15.100:0/1332401016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623c8d111b0 0x5623c90dfbd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3418009d20 tx=0x7f3418011810 comp rx=0 tx=0).stop 2024-07-26T13:41:38.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.504+0000 7f3422424640 1 --2- 172.21.15.100:0/4260419 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f34080b3e40 0x7f34080b6350 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f341400bfd0 tx=0x7f341400d040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:38.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.504+0000 7f342a40b740 1 -- 172.21.15.100:0/1332401016 shutdown_connections 2024-07-26T13:41:38.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.504+0000 7f342a40b740 1 --2- 172.21.15.100:0/1332401016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623c8d19390 0x5623c90e0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:38.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.504+0000 7f342a40b740 1 --2- 172.21.15.100:0/1332401016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623c8d111b0 0x5623c90dfbd0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:38.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.504+0000 7f342a40b740 1 --2- 172.21.15.100:0/1332401016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623c8d3a070 0x5623c90df690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:38.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.504+0000 7f342a40b740 1 -- 172.21.15.100:0/1332401016 >> 172.21.15.100:0/1332401016 conn(0x5623c8d2e460 msgr2=0x5623c90dd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:38.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.504+0000 7f342a40b740 1 -- 172.21.15.100:0/1332401016 shutdown_connections 2024-07-26T13:41:38.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.505+0000 7f342a40b740 1 -- 172.21.15.100:0/1332401016 wait complete. 2024-07-26T13:41:38.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.505+0000 7f342a40b740 1 -- start start 2024-07-26T13:41:38.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.505+0000 7f342a40b740 1 Processor -- start 2024-07-26T13:41:38.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.506+0000 7f342a40b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623c8d111b0 0x5623c90eb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:38.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.506+0000 7f342a40b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623c8d19390 0x5623c90eb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:38.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.506+0000 7f342a40b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623c8d3a070 0x5623c90f7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:38.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.506+0000 7f342a40b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5623c8ea8700 con 0x5623c8d3a070 2024-07-26T13:41:38.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.506+0000 7f342a40b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5623c8fe79e0 con 0x5623c8d19390 2024-07-26T13:41:38.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.506+0000 7f342a40b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5623c90ede70 con 0x5623c8d111b0 2024-07-26T13:41:38.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.506+0000 7f342a40b740 10 client.0 _ll_register_callbacks cb 0x5623c90024f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:41:38.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.506+0000 7f3421422640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623c8d19390 0x5623c90eb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:38.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.506+0000 7f3422424640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623c8d3a070 0x5623c90f7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:38.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.506+0000 7f3421422640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623c8d19390 0x5623c90eb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:55680/0 (socket says 172.21.15.100:55680) 2024-07-26T13:41:38.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.506+0000 7f3421422640 1 -- 172.21.15.100:0/2597688599 learned_addr learned my addr 172.21.15.100:0/2597688599 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:38.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.507+0000 7f3421c23640 1 --2- 172.21.15.100:0/2597688599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623c8d111b0 0x5623c90eb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:38.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.507+0000 7f3421422640 1 -- 172.21.15.100:0/2597688599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623c8d111b0 msgr2=0x5623c90eb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:38.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.507+0000 7f3421422640 1 --2- 172.21.15.100:0/2597688599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623c8d111b0 0x5623c90eb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:38.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.507+0000 7f3421422640 1 -- 172.21.15.100:0/2597688599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623c8d3a070 msgr2=0x5623c90f7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:38.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.507+0000 7f3421422640 1 --2- 172.21.15.100:0/2597688599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623c8d3a070 0x5623c90f7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:38.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.507+0000 7f3421422640 1 -- 172.21.15.100:0/2597688599 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623c8da5650 con 0x5623c8d19390 2024-07-26T13:41:38.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.507+0000 7f3421c23640 1 --2- 172.21.15.100:0/2597688599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623c8d111b0 0x5623c90eb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:41:38.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.507+0000 7f3421422640 1 --2- 172.21.15.100:0/2597688599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623c8d19390 0x5623c90eb8c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f340c006010 tx=0x7f340c009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84a9c59eb9358ef0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:38.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.507+0000 7f34057fa640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-07-26T13:41:38.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.507+0000 7f34057fa640 1 -- 172.21.15.100:0/2597688599 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340c0db8d0 con 0x5623c8d19390 2024-07-26T13:41:38.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.507+0000 7f342a40b740 1 -- 172.21.15.100:0/2597688599 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3418019ae0 con 0x5623c8d19390 2024-07-26T13:41:38.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.507+0000 7f34057fa640 1 -- 172.21.15.100:0/2597688599 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f340c040570 con 0x5623c8d19390 2024-07-26T13:41:38.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.507+0000 7f34057fa640 1 -- 172.21.15.100:0/2597688599 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f340c0dbc70 con 0x5623c8d19390 2024-07-26T13:41:38.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.507+0000 7f342a40b740 10 client.25295 fetch_fsmap learned FSMap version 26 2024-07-26T13:41:38.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.507+0000 7f342a40b740 1 -- 172.21.15.100:0/2597688599 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5623c90f9f00 con 0x5623c8d19390 2024-07-26T13:41:38.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.508+0000 7f34057fa640 1 -- 172.21.15.100:0/2597688599 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3418019ae0 con 0x5623c8d19390 2024-07-26T13:41:38.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.508+0000 7f34057fa640 1 -- 172.21.15.100:0/2597688599 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f340c003c80 con 0x5623c8d19390 2024-07-26T13:41:38.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.508+0000 7f342a40b740 10 client.25295 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:41:38.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.508+0000 7f342a40b740 10 client.25295 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-26T13:41:38.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.508+0000 7f342a40b740 20 client.25295 populate_metadata read hostname 'smithi100' 2024-07-26T13:41:38.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.508+0000 7f342a40b740 1 --2- 172.21.15.100:0/2597688599 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x5623c8cf89f0 0x5623c90fd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:38.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.508+0000 7f342a40b740 4 client.25295 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:41:38.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.508+0000 7f342a40b740 1 -- 172.21.15.100:0/2597688599 --> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623c9057b00 con 0x5623c8cf89f0 2024-07-26T13:41:38.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.509+0000 7f3421c23640 1 --2- 172.21.15.100:0/2597688599 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x5623c8cf89f0 0x5623c90fd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:38.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.510+0000 7f3421c23640 1 --2- 172.21.15.100:0/2597688599 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x5623c8cf89f0 0x5623c90fd380 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1200401354dcd441 server_cookie=871fb18cc7068e42 in_seq=0 out_seq=0 2024-07-26T13:41:38.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.510+0000 7f34057fa640 10 client.25295 ms_handle_connect on v2:172.21.15.138:6838/1580925471 2024-07-26T13:41:38.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.510+0000 7f34057fa640 1 -- 172.21.15.100:0/2597688599 <== mds.1 v2:172.21.15.138:6838/1580925471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5623c9057b00 con 0x5623c8cf89f0 2024-07-26T13:41:38.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.510+0000 7f34057fa640 10 client.25295 handle_command_reply: tid=0 2024-07-26T13:41:38.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.510+0000 7f342a40b740 1 client.25295 shutdown 2024-07-26T13:41:38.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.511+0000 7f342a40b740 1 -- 172.21.15.100:0/2597688599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623c8d19390 msgr2=0x5623c90eb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:38.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.511+0000 7f342a40b740 1 --2- 172.21.15.100:0/2597688599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623c8d19390 0x5623c90eb8c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f340c006010 tx=0x7f340c009620 comp rx=0 tx=0).stop 2024-07-26T13:41:38.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.511+0000 7f342a40b740 1 -- 172.21.15.100:0/2597688599 shutdown_connections 2024-07-26T13:41:38.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.511+0000 7f342a40b740 1 --2- 172.21.15.100:0/2597688599 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x5623c8cf89f0 0x5623c90fd380 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:38.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.511+0000 7f34057fa640 0 client.25295 ms_handle_reset on v2:172.21.15.138:6838/1580925471 2024-07-26T13:41:38.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.511+0000 7f342a40b740 1 --2- 172.21.15.100:0/2597688599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623c8d3a070 0x5623c90f7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:38.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.511+0000 7f342a40b740 1 --2- 172.21.15.100:0/2597688599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623c8d19390 0x5623c90eb8c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:38.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.511+0000 7f342a40b740 1 --2- 172.21.15.100:0/2597688599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623c8d111b0 0x5623c90eb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:38.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.511+0000 7f342a40b740 1 -- 172.21.15.100:0/2597688599 >> 172.21.15.100:0/2597688599 conn(0x5623c8d2e460 msgr2=0x5623c90e2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:38.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.511+0000 7f342a40b740 1 -- 172.21.15.100:0/2597688599 shutdown_connections 2024-07-26T13:41:38.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.511+0000 7f342a40b740 1 -- 172.21.15.100:0/2597688599 wait complete. 2024-07-26T13:41:38.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.511+0000 7f342a40b740 20 client.25295 trim_cache size 0 max 16384 2024-07-26T13:41:38.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.534+0000 7f342a40b740 1 Processor -- start 2024-07-26T13:41:38.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.535+0000 7f342a40b740 1 -- start start 2024-07-26T13:41:38.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.535+0000 7f342a40b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623c8cf89f0 0x5623c90e2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:38.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.536+0000 7f342a40b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623c8d111b0 0x5623c90f6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:38.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.536+0000 7f3421c23640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623c8cf89f0 0x5623c90e2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:38.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.536+0000 7f3421c23640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623c8cf89f0 0x5623c90e2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45242/0 (socket says 172.21.15.100:45242) 2024-07-26T13:41:38.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.536+0000 7f3421c23640 1 -- 172.21.15.100:0/496680668 learned_addr learned my addr 172.21.15.100:0/496680668 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:38.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.536+0000 7f3421422640 1 --2- 172.21.15.100:0/496680668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623c8d111b0 0x5623c90f6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:38.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.536+0000 7f342a40b740 1 --2- 172.21.15.100:0/496680668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623c8d19390 0x5623c90e4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:38.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.536+0000 7f342a40b740 1 -- 172.21.15.100:0/496680668 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f341c1af7f0 con 0x5623c8cf89f0 2024-07-26T13:41:38.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.537+0000 7f342a40b740 1 -- 172.21.15.100:0/496680668 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f341c1ad0c0 con 0x5623c8d111b0 2024-07-26T13:41:38.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.537+0000 7f3422424640 1 --2- 172.21.15.100:0/496680668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623c8d19390 0x5623c90e4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:38.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.537+0000 7f342a40b740 1 -- 172.21.15.100:0/496680668 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f341c1aaf10 con 0x5623c8d19390 2024-07-26T13:41:38.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.537+0000 7f3421c23640 1 -- 172.21.15.100:0/496680668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623c8d19390 msgr2=0x5623c90e4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:38.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.537+0000 7f3421c23640 1 --2- 172.21.15.100:0/496680668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623c8d19390 0x5623c90e4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:38.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.537+0000 7f3421c23640 1 -- 172.21.15.100:0/496680668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623c8d111b0 msgr2=0x5623c90f6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:38.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.537+0000 7f3421c23640 1 --2- 172.21.15.100:0/496680668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623c8d111b0 0x5623c90f6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:38.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.537+0000 7f3421c23640 1 -- 172.21.15.100:0/496680668 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623c90ebc30 con 0x5623c8cf89f0 2024-07-26T13:41:38.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.538+0000 7f3421c23640 1 --2- 172.21.15.100:0/496680668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623c8cf89f0 0x5623c90e2ea0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3418009250 tx=0x7f3418011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb2c1c642d4b7654 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:38.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.538+0000 7f34077fe640 1 -- 172.21.15.100:0/496680668 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341801e040 con 0x5623c8cf89f0 2024-07-26T13:41:38.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.538+0000 7f34077fe640 1 -- 172.21.15.100:0/496680668 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f341801f040 con 0x5623c8cf89f0 2024-07-26T13:41:38.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.538+0000 7f34077fe640 1 -- 172.21.15.100:0/496680668 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341800ec30 con 0x5623c8cf89f0 2024-07-26T13:41:38.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.539+0000 7f342a40b740 1 -- 172.21.15.100:0/496680668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623c8cf89f0 msgr2=0x5623c90e2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:38.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.539+0000 7f342a40b740 1 --2- 172.21.15.100:0/496680668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623c8cf89f0 0x5623c90e2ea0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3418009250 tx=0x7f3418011d10 comp rx=0 tx=0).stop 2024-07-26T13:41:38.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.539+0000 7f342a40b740 1 -- 172.21.15.100:0/496680668 shutdown_connections 2024-07-26T13:41:38.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.539+0000 7f342a40b740 1 --2- 172.21.15.100:0/496680668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623c8d19390 0x5623c90e4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:38.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.539+0000 7f342a40b740 1 --2- 172.21.15.100:0/496680668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623c8d111b0 0x5623c90f6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:38.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.539+0000 7f342a40b740 1 --2- 172.21.15.100:0/496680668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623c8cf89f0 0x5623c90e2ea0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:38.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.539+0000 7f342a40b740 1 -- 172.21.15.100:0/496680668 >> 172.21.15.100:0/496680668 conn(0x5623c8d2e460 msgr2=0x5623c90e2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:38.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.540+0000 7f342a40b740 1 -- 172.21.15.100:0/496680668 shutdown_connections 2024-07-26T13:41:38.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.540+0000 7f342a40b740 1 -- 172.21.15.100:0/496680668 wait complete. 2024-07-26T13:41:38.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.540+0000 7f342a40b740 1 -- start start 2024-07-26T13:41:38.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.540+0000 7f342a40b740 1 Processor -- start 2024-07-26T13:41:38.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.541+0000 7f342a40b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623c8cf89f0 0x5623c90f3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:38.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.541+0000 7f342a40b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623c8d111b0 0x5623c90dc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:38.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.541+0000 7f342a40b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623c8d19390 0x5623c90dcb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:38.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.541+0000 7f342a40b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f341c1ab580 con 0x5623c8d111b0 2024-07-26T13:41:38.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.541+0000 7f342a40b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f341c1ac9f0 con 0x5623c8d19390 2024-07-26T13:41:38.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.541+0000 7f342a40b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f341c1abc50 con 0x5623c8cf89f0 2024-07-26T13:41:38.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.541+0000 7f342a40b740 10 client.0 _ll_register_callbacks cb 0x7f340c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:41:38.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.541+0000 7f3421422640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623c8d111b0 0x5623c90dc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:38.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.541+0000 7f3421422640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623c8d111b0 0x5623c90dc5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45246/0 (socket says 172.21.15.100:45246) 2024-07-26T13:41:38.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.541+0000 7f3421422640 1 -- 172.21.15.100:0/1944263764 learned_addr learned my addr 172.21.15.100:0/1944263764 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:38.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.541+0000 7f3422424640 1 --2- 172.21.15.100:0/1944263764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623c8d19390 0x5623c90dcb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:38.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.541+0000 7f3421c23640 1 --2- 172.21.15.100:0/1944263764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623c8cf89f0 0x5623c90f3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:38.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.542+0000 7f3421422640 1 -- 172.21.15.100:0/1944263764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623c8cf89f0 msgr2=0x5623c90f3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:38.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.542+0000 7f3421422640 1 --2- 172.21.15.100:0/1944263764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623c8cf89f0 0x5623c90f3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:38.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.542+0000 7f3421422640 1 -- 172.21.15.100:0/1944263764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623c8d19390 msgr2=0x5623c90dcb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:38.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.542+0000 7f3421422640 1 --2- 172.21.15.100:0/1944263764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623c8d19390 0x5623c90dcb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:38.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.542+0000 7f3421422640 1 -- 172.21.15.100:0/1944263764 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623c90dd070 con 0x5623c8d111b0 2024-07-26T13:41:38.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.543+0000 7f3421422640 1 --2- 172.21.15.100:0/1944263764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623c8d111b0 0x5623c90dc5f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f340c031440 tx=0x7f340c005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c29b93725d6f4d0f server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:38.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.543+0000 7f34057fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:41:38.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.543+0000 7f34057fa640 1 -- 172.21.15.100:0/1944263764 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340c0402c0 con 0x5623c8d111b0 2024-07-26T13:41:38.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.543+0000 7f34057fa640 1 -- 172.21.15.100:0/1944263764 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f340c0db3c0 con 0x5623c8d111b0 2024-07-26T13:41:38.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.543+0000 7f342a40b740 1 -- 172.21.15.100:0/1944263764 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f341801f040 con 0x5623c8d111b0 2024-07-26T13:41:38.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.543+0000 7f34057fa640 1 -- 172.21.15.100:0/1944263764 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f340c0dbf30 con 0x5623c8d111b0 2024-07-26T13:41:38.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.543+0000 7f342a40b740 10 client.15837 fetch_fsmap learned FSMap version 26 2024-07-26T13:41:38.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.543+0000 7f342a40b740 1 -- 172.21.15.100:0/1944263764 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5623c90f1da0 con 0x5623c8d111b0 2024-07-26T13:41:38.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.544+0000 7f34057fa640 1 -- 172.21.15.100:0/1944263764 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f340c0fd350 con 0x5623c8d111b0 2024-07-26T13:41:38.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.544+0000 7f34057fa640 1 -- 172.21.15.100:0/1944263764 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f340c0fd030 con 0x5623c8d111b0 2024-07-26T13:41:38.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.544+0000 7f342a40b740 10 client.15837 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:41:38.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.544+0000 7f342a40b740 10 client.15837 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-26T13:41:38.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.544+0000 7f342a40b740 20 client.15837 populate_metadata read hostname 'smithi100' 2024-07-26T13:41:38.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.544+0000 7f342a40b740 1 --2- 172.21.15.100:0/1944263764 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x5623c8d3a070 0x5623c9167e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:38.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.544+0000 7f342a40b740 4 client.15837 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-26T13:41:38.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.544+0000 7f342a40b740 1 -- 172.21.15.100:0/1944263764 --> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f341801f040 con 0x5623c8d3a070 2024-07-26T13:41:38.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.544+0000 7f3421c23640 1 --2- 172.21.15.100:0/1944263764 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x5623c8d3a070 0x5623c9167e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:38.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.545+0000 7f34057fa640 10 client.15837 ms_handle_connect on v2:172.21.15.138:6838/1580925471 2024-07-26T13:41:38.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.545+0000 7f3421c23640 1 --2- 172.21.15.100:0/1944263764 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x5623c8d3a070 0x5623c9167e80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=72d131e4bd025937 server_cookie=a933edfefbcf0376 in_seq=0 out_seq=0 2024-07-26T13:41:38.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.545+0000 7f34057fa640 1 -- 172.21.15.100:0/1944263764 <== mds.1 v2:172.21.15.138:6838/1580925471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f341801f040 con 0x5623c8d3a070 2024-07-26T13:41:38.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.545+0000 7f34057fa640 10 client.15837 handle_command_reply: tid=0 2024-07-26T13:41:38.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.545+0000 7f342a40b740 1 client.15837 shutdown 2024-07-26T13:41:38.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.545+0000 7f342a40b740 1 -- 172.21.15.100:0/1944263764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623c8d111b0 msgr2=0x5623c90dc5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:38.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.545+0000 7f342a40b740 1 --2- 172.21.15.100:0/1944263764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623c8d111b0 0x5623c90dc5f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f340c031440 tx=0x7f340c005c90 comp rx=0 tx=0).stop 2024-07-26T13:41:38.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.546+0000 7f342a40b740 1 -- 172.21.15.100:0/1944263764 shutdown_connections 2024-07-26T13:41:38.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.546+0000 7f342a40b740 1 --2- 172.21.15.100:0/1944263764 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x5623c8d3a070 0x5623c9167e80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:38.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.546+0000 7f34057fa640 0 client.15837 ms_handle_reset on v2:172.21.15.138:6838/1580925471 2024-07-26T13:41:38.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.546+0000 7f342a40b740 1 --2- 172.21.15.100:0/1944263764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623c8d19390 0x5623c90dcb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:38.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.546+0000 7f342a40b740 1 --2- 172.21.15.100:0/1944263764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5623c8d111b0 0x5623c90dc5f0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:38.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.546+0000 7f342a40b740 1 --2- 172.21.15.100:0/1944263764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623c8cf89f0 0x5623c90f3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:38.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.546+0000 7f342a40b740 1 -- 172.21.15.100:0/1944263764 >> 172.21.15.100:0/1944263764 conn(0x5623c8d2e460 msgr2=0x5623c90e0660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:38.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.546+0000 7f342a40b740 1 -- 172.21.15.100:0/1944263764 shutdown_connections 2024-07-26T13:41:38.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.546+0000 7f342a40b740 1 -- 172.21.15.100:0/1944263764 wait complete. 2024-07-26T13:41:38.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.546+0000 7f342a40b740 20 client.15837 trim_cache size 0 max 16384 2024-07-26T13:41:38.551 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-07-26T13:41:38.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.547+0000 7f3423e87640 1 -- 172.21.15.100:0/4260419 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f34080b3e40 msgr2=0x7f34080b6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:38.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.547+0000 7f3423e87640 1 --2- 172.21.15.100:0/4260419 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f34080b3e40 0x7f34080b6350 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f341400bfd0 tx=0x7f341400d040 comp rx=0 tx=0).stop 2024-07-26T13:41:38.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.547+0000 7f3423e87640 1 -- 172.21.15.100:0/4260419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f341c107de0 msgr2=0x7f341c198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:38.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.547+0000 7f3423e87640 1 --2- 172.21.15.100:0/4260419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f341c107de0 0x7f341c198270 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f340c005ab0 tx=0x7f340c0035f0 comp rx=0 tx=0).stop 2024-07-26T13:41:38.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.547+0000 7f3423e87640 1 -- 172.21.15.100:0/4260419 shutdown_connections 2024-07-26T13:41:38.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.547+0000 7f3423e87640 1 --2- 172.21.15.100:0/4260419 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f34080b3e40 0x7f34080b6350 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:38.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.547+0000 7f3423e87640 1 --2- 172.21.15.100:0/4260419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f341c108730 0x7f341c19d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:38.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.547+0000 7f3423e87640 1 --2- 172.21.15.100:0/4260419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f341c107de0 0x7f341c198270 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:38.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.547+0000 7f3423e87640 1 --2- 172.21.15.100:0/4260419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f341c1024a0 0x7f341c197cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:38.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.547+0000 7f3423e87640 1 -- 172.21.15.100:0/4260419 >> 172.21.15.100:0/4260419 conn(0x7f341c079160 msgr2=0x7f341c103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:38.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.547+0000 7f3423e87640 1 -- 172.21.15.100:0/4260419 shutdown_connections 2024-07-26T13:41:38.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:38.547+0000 7f3423e87640 1 -- 172.21.15.100:0/4260419 wait complete. 2024-07-26T13:41:39.309 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:41:39.336 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-26T13:41:39.582 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:39 smithi100 ceph-mon[32599]: pgmap v410: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-07-26T13:41:39.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:39 smithi138 ceph-mon[36954]: pgmap v410: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-07-26T13:41:40.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:39 smithi167 ceph-mon[37140]: pgmap v410: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-07-26T13:41:40.093 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:41:41.810 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:41 smithi100 ceph-mon[32599]: pgmap v411: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-07-26T13:41:41.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.817+0000 7f758897e640 1 -- 172.21.15.100:0/1807863849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7584106600 msgr2=0x7f7584110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:41.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.817+0000 7f758897e640 1 --2- 172.21.15.100:0/1807863849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7584106600 0x7f7584110090 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f757c009ca0 tx=0x7f757c02a8c0 comp rx=0 tx=0).stop 2024-07-26T13:41:41.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.817+0000 7f758897e640 1 -- 172.21.15.100:0/1807863849 shutdown_connections 2024-07-26T13:41:41.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.817+0000 7f758897e640 1 --2- 172.21.15.100:0/1807863849 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7584106600 0x7f7584110090 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:41.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.817+0000 7f758897e640 1 --2- 172.21.15.100:0/1807863849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7584105cb0 0x7f75841060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:41.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.817+0000 7f758897e640 1 --2- 172.21.15.100:0/1807863849 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7584101a60 0x7f7584101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:41.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.817+0000 7f758897e640 1 -- 172.21.15.100:0/1807863849 >> 172.21.15.100:0/1807863849 conn(0x7f7584077690 msgr2=0x7f7584077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:41.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.817+0000 7f758897e640 1 -- 172.21.15.100:0/1807863849 shutdown_connections 2024-07-26T13:41:41.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.817+0000 7f758897e640 1 -- 172.21.15.100:0/1807863849 wait complete. 2024-07-26T13:41:41.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.818+0000 7f758897e640 1 Processor -- start 2024-07-26T13:41:41.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.818+0000 7f758897e640 1 -- start start 2024-07-26T13:41:41.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.818+0000 7f758897e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7584101a60 0x7f7584102a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:41.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.818+0000 7f758897e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7584105cb0 0x7f7584102f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:41.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.818+0000 7f758897e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7584106600 0x7f7584103480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:41.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.818+0000 7f758897e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f75841134f0 con 0x7f7584101a60 2024-07-26T13:41:41.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.818+0000 7f758897e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7584113370 con 0x7f7584105cb0 2024-07-26T13:41:41.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.818+0000 7f758897e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7584113670 con 0x7f7584106600 2024-07-26T13:41:41.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.818+0000 7f758259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7584101a60 0x7f7584102a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:41.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.818+0000 7f758259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7584101a60 0x7f7584102a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45262/0 (socket says 172.21.15.100:45262) 2024-07-26T13:41:41.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.818+0000 7f758259c640 1 -- 172.21.15.100:0/1662835386 learned_addr learned my addr 172.21.15.100:0/1662835386 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:41.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.819+0000 7f7582d9d640 1 --2- 172.21.15.100:0/1662835386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7584106600 0x7f7584103480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:41.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.819+0000 7f7581d9b640 1 --2- 172.21.15.100:0/1662835386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7584105cb0 0x7f7584102f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:41.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.819+0000 7f758259c640 1 -- 172.21.15.100:0/1662835386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7584106600 msgr2=0x7f7584103480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:41.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.819+0000 7f758259c640 1 --2- 172.21.15.100:0/1662835386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7584106600 0x7f7584103480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:41.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.819+0000 7f758259c640 1 -- 172.21.15.100:0/1662835386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7584105cb0 msgr2=0x7f7584102f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:41.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.819+0000 7f758259c640 1 --2- 172.21.15.100:0/1662835386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7584105cb0 0x7f7584102f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:41.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.819+0000 7f758259c640 1 -- 172.21.15.100:0/1662835386 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75841a8180 con 0x7f7584101a60 2024-07-26T13:41:41.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.819+0000 7f758259c640 1 --2- 172.21.15.100:0/1662835386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7584101a60 0x7f7584102a00 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f757400d950 tx=0x7f757400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:41.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.819+0000 7f75737fe640 1 -- 172.21.15.100:0/1662835386 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7574017070 con 0x7f7584101a60 2024-07-26T13:41:41.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.819+0000 7f75737fe640 1 -- 172.21.15.100:0/1662835386 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7574009c30 con 0x7f7584101a60 2024-07-26T13:41:41.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.819+0000 7f758897e640 1 -- 172.21.15.100:0/1662835386 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75841a8470 con 0x7f7584101a60 2024-07-26T13:41:41.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.819+0000 7f75737fe640 1 -- 172.21.15.100:0/1662835386 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757400f620 con 0x7f7584101a60 2024-07-26T13:41:41.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.819+0000 7f758897e640 1 -- 172.21.15.100:0/1662835386 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75841a8960 con 0x7f7584101a60 2024-07-26T13:41:41.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.822+0000 7f75737fe640 1 -- 172.21.15.100:0/1662835386 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f757401a070 con 0x7f7584101a60 2024-07-26T13:41:41.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.822+0000 7f758ef02740 1 Processor -- start 2024-07-26T13:41:41.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.823+0000 7f758ef02740 1 -- start start 2024-07-26T13:41:41.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.823+0000 7f758ef02740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564252afc520 0x564252ea2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:41.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.823+0000 7f758ef02740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564252ad3660 0x564252ea25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:41.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.823+0000 7f758ef02740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564252adb840 0x564252ea2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:41.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.823+0000 7f758ef02740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x564252e17e70 con 0x564252adb840 2024-07-26T13:41:41.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.823+0000 7f758ef02740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564252e07370 con 0x564252afc520 2024-07-26T13:41:41.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.823+0000 7f758ef02740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564252e0b3b0 con 0x564252ad3660 2024-07-26T13:41:41.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.823+0000 7f7582d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564252afc520 0x564252ea2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:41.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.823+0000 7f7581d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564252adb840 0x564252ea2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:41.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.823+0000 7f7582d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564252afc520 0x564252ea2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:55718/0 (socket says 172.21.15.100:55718) 2024-07-26T13:41:41.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.823+0000 7f7582d9d640 1 -- 172.21.15.100:0/1629463602 learned_addr learned my addr 172.21.15.100:0/1629463602 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:41.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.823+0000 7f758259c640 1 --2- 172.21.15.100:0/1629463602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564252ad3660 0x564252ea25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:41.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.823+0000 7f75737fe640 1 --2- 172.21.15.100:0/1662835386 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f756c0b3dc0 0x7f756c0b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:41.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.824+0000 7f7582d9d640 1 -- 172.21.15.100:0/1629463602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564252ad3660 msgr2=0x564252ea25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:41.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.824+0000 7f7582d9d640 1 --2- 172.21.15.100:0/1629463602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564252ad3660 0x564252ea25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:41.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.824+0000 7f7582d9d640 1 -- 172.21.15.100:0/1629463602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564252adb840 msgr2=0x564252ea2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:41.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.824+0000 7f7582d9d640 1 --2- 172.21.15.100:0/1629463602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564252adb840 0x564252ea2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:41.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.824+0000 7f7582d9d640 1 -- 172.21.15.100:0/1629463602 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564252dd4040 con 0x564252afc520 2024-07-26T13:41:41.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.824+0000 7f7582d9d640 1 --2- 172.21.15.100:0/1662835386 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f756c0b3dc0 0x7f756c0b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:41.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.824+0000 7f75737fe640 1 -- 172.21.15.100:0/1662835386 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f75740e3a70 con 0x7f7584101a60 2024-07-26T13:41:41.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.824+0000 7f7582d9d640 1 --2- 172.21.15.100:0/1629463602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564252afc520 0x564252ea2070 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f757c0099b0 tx=0x7f757c0099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5acb20f03093fac0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:41.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.824+0000 7f7582d9d640 1 --2- 172.21.15.100:0/1662835386 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f756c0b3dc0 0x7f756c0b62d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f757c04d000 tx=0x7f757c04e000 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:41.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.824+0000 7f7570ff9640 1 -- 172.21.15.100:0/1629463602 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c042040 con 0x564252afc520 2024-07-26T13:41:41.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.824+0000 7f7570ff9640 1 -- 172.21.15.100:0/1629463602 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f757c036070 con 0x564252afc520 2024-07-26T13:41:41.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.824+0000 7f7570ff9640 1 -- 172.21.15.100:0/1629463602 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c02f8a0 con 0x564252afc520 2024-07-26T13:41:41.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.825+0000 7f758ef02740 1 -- 172.21.15.100:0/1629463602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564252afc520 msgr2=0x564252ea2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:41.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.825+0000 7f758ef02740 1 --2- 172.21.15.100:0/1629463602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564252afc520 0x564252ea2070 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f757c0099b0 tx=0x7f757c0099e0 comp rx=0 tx=0).stop 2024-07-26T13:41:41.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.825+0000 7f758ef02740 1 -- 172.21.15.100:0/1629463602 shutdown_connections 2024-07-26T13:41:41.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.825+0000 7f758ef02740 1 --2- 172.21.15.100:0/1629463602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564252adb840 0x564252ea2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:41.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.825+0000 7f758ef02740 1 --2- 172.21.15.100:0/1629463602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564252ad3660 0x564252ea25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:41.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.825+0000 7f758ef02740 1 --2- 172.21.15.100:0/1629463602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564252afc520 0x564252ea2070 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:41.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.825+0000 7f758ef02740 1 -- 172.21.15.100:0/1629463602 >> 172.21.15.100:0/1629463602 conn(0x564252af0910 msgr2=0x564252e9ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:41.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.825+0000 7f758ef02740 1 -- 172.21.15.100:0/1629463602 shutdown_connections 2024-07-26T13:41:41.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.825+0000 7f758ef02740 1 -- 172.21.15.100:0/1629463602 wait complete. 2024-07-26T13:41:41.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.826+0000 7f758ef02740 1 -- start start 2024-07-26T13:41:41.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.826+0000 7f758ef02740 1 Processor -- start 2024-07-26T13:41:41.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.826+0000 7f758ef02740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564252ad3660 0x564252eadbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:41.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.826+0000 7f758ef02740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564252adb840 0x564252eae100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:41.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.826+0000 7f758ef02740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564252afc520 0x564252eb9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:41.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.826+0000 7f758ef02740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x564252c6abb0 con 0x564252adb840 2024-07-26T13:41:41.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.826+0000 7f758ef02740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564252da9ac0 con 0x564252ad3660 2024-07-26T13:41:41.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.826+0000 7f758ef02740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564252eb0850 con 0x564252afc520 2024-07-26T13:41:41.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.826+0000 7f758ef02740 10 client.0 _ll_register_callbacks cb 0x564252db23b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:41:41.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.826+0000 7f7581d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564252adb840 0x564252eae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:41.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.826+0000 7f7581d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564252adb840 0x564252eae100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45288/0 (socket says 172.21.15.100:45288) 2024-07-26T13:41:41.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.826+0000 7f7581d9b640 1 -- 172.21.15.100:0/588568127 learned_addr learned my addr 172.21.15.100:0/588568127 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:41.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.826+0000 7f758259c640 1 --2- 172.21.15.100:0/588568127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564252ad3660 0x564252eadbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:41.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.826+0000 7f7582d9d640 1 --2- 172.21.15.100:0/588568127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564252afc520 0x564252eb9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:41.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.827+0000 7f7581d9b640 1 -- 172.21.15.100:0/588568127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564252afc520 msgr2=0x564252eb9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:41.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.827+0000 7f7581d9b640 1 --2- 172.21.15.100:0/588568127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564252afc520 0x564252eb9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:41.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.827+0000 7f7581d9b640 1 -- 172.21.15.100:0/588568127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564252ad3660 msgr2=0x564252eadbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:41.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.827+0000 7f7581d9b640 1 --2- 172.21.15.100:0/588568127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564252ad3660 0x564252eadbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:41.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.827+0000 7f7581d9b640 1 -- 172.21.15.100:0/588568127 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564252b67b00 con 0x564252adb840 2024-07-26T13:41:41.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.827+0000 7f758259c640 1 --2- 172.21.15.100:0/588568127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564252ad3660 0x564252eadbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:41:41.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.827+0000 7f7581d9b640 1 --2- 172.21.15.100:0/588568127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564252adb840 0x564252eae100 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7578011710 tx=0x7f7578011b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98172d98d30ae0c7 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:41.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.827+0000 7f7569ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:41:41.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.827+0000 7f7569ffb640 1 -- 172.21.15.100:0/588568127 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757800fd50 con 0x564252adb840 2024-07-26T13:41:41.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.827+0000 7f7569ffb640 1 -- 172.21.15.100:0/588568127 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f7578005020 con 0x564252adb840 2024-07-26T13:41:41.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.827+0000 7f758ef02740 1 -- 172.21.15.100:0/588568127 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f757c036070 con 0x564252adb840 2024-07-26T13:41:41.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.827+0000 7f7569ffb640 1 -- 172.21.15.100:0/588568127 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7578005220 con 0x564252adb840 2024-07-26T13:41:41.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.827+0000 7f758ef02740 10 client.15849 fetch_fsmap learned FSMap version 26 2024-07-26T13:41:41.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.827+0000 7f758ef02740 1 -- 172.21.15.100:0/588568127 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564252bde960 con 0x564252adb840 2024-07-26T13:41:41.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.828+0000 7f7569ffb640 1 -- 172.21.15.100:0/588568127 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f757c036070 con 0x564252adb840 2024-07-26T13:41:41.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.828+0000 7f7569ffb640 1 -- 172.21.15.100:0/588568127 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f7578020020 con 0x564252adb840 2024-07-26T13:41:41.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.828+0000 7f758ef02740 10 client.15849 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:41:41.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.828+0000 7f758ef02740 10 client.15849 resolve_mds: validated gid 24607 aka daemon mds.i 2024-07-26T13:41:41.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.828+0000 7f758ef02740 20 client.15849 populate_metadata read hostname 'smithi100' 2024-07-26T13:41:41.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.828+0000 7f758ef02740 1 --2- 172.21.15.100:0/588568127 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x564252abaea0 0x564252ebfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:41.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.828+0000 7f758ef02740 4 client.15849 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:41:41.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.828+0000 7f758ef02740 1 -- 172.21.15.100:0/588568127 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564252e19dc0 con 0x564252abaea0 2024-07-26T13:41:41.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.828+0000 7f758259c640 1 --2- 172.21.15.100:0/588568127 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x564252abaea0 0x564252ebfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:41.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.829+0000 7f7569ffb640 10 client.15849 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:41.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.829+0000 7f758259c640 1 --2- 172.21.15.100:0/588568127 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x564252abaea0 0x564252ebfdd0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55d774c8910bba1b server_cookie=a348649fbe6d6948 in_seq=0 out_seq=0 2024-07-26T13:41:41.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.830+0000 7f7569ffb640 1 -- 172.21.15.100:0/588568127 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564252e19dc0 con 0x564252abaea0 2024-07-26T13:41:41.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.830+0000 7f7569ffb640 10 client.15849 handle_command_reply: tid=0 2024-07-26T13:41:41.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.830+0000 7f758ef02740 1 client.15849 shutdown 2024-07-26T13:41:41.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.830+0000 7f758ef02740 1 -- 172.21.15.100:0/588568127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564252adb840 msgr2=0x564252eae100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:41.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.830+0000 7f758ef02740 1 --2- 172.21.15.100:0/588568127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564252adb840 0x564252eae100 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7578011710 tx=0x7f7578011b10 comp rx=0 tx=0).stop 2024-07-26T13:41:41.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.830+0000 7f758ef02740 1 -- 172.21.15.100:0/588568127 shutdown_connections 2024-07-26T13:41:41.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.830+0000 7f758ef02740 1 --2- 172.21.15.100:0/588568127 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x564252abaea0 0x564252ebfdd0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:41.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.830+0000 7f758ef02740 1 --2- 172.21.15.100:0/588568127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564252afc520 0x564252eb9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:41.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.830+0000 7f7569ffb640 0 client.15849 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:41.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.830+0000 7f758ef02740 1 --2- 172.21.15.100:0/588568127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564252adb840 0x564252eae100 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:41.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.830+0000 7f758ef02740 1 --2- 172.21.15.100:0/588568127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564252ad3660 0x564252eadbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:41.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.830+0000 7f758ef02740 1 -- 172.21.15.100:0/588568127 >> 172.21.15.100:0/588568127 conn(0x564252af0910 msgr2=0x564252ea5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:41.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.830+0000 7f758ef02740 1 -- 172.21.15.100:0/588568127 shutdown_connections 2024-07-26T13:41:41.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.830+0000 7f758ef02740 1 -- 172.21.15.100:0/588568127 wait complete. 2024-07-26T13:41:41.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.830+0000 7f758ef02740 20 client.15849 trim_cache size 0 max 16384 2024-07-26T13:41:41.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.847+0000 7f758ef02740 1 Processor -- start 2024-07-26T13:41:41.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.847+0000 7f758ef02740 1 -- start start 2024-07-26T13:41:41.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.848+0000 7f758ef02740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564252ad3660 0x564252eaf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:41.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.848+0000 7f758ef02740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564252adb840 0x564252eb9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:41.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.848+0000 7f758ef02740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564252afc520 0x564252eb9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:41.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.848+0000 7f758ef02740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f75841b3b40 con 0x564252ad3660 2024-07-26T13:41:41.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.848+0000 7f758ef02740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f75841b1410 con 0x564252afc520 2024-07-26T13:41:41.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.848+0000 7f758ef02740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f75841af260 con 0x564252adb840 2024-07-26T13:41:41.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.848+0000 7f758259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564252ad3660 0x564252eaf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:41.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.848+0000 7f758259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564252ad3660 0x564252eaf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45300/0 (socket says 172.21.15.100:45300) 2024-07-26T13:41:41.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.848+0000 7f758259c640 1 -- 172.21.15.100:0/3981523471 learned_addr learned my addr 172.21.15.100:0/3981523471 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:41.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.848+0000 7f7581d9b640 1 --2- 172.21.15.100:0/3981523471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564252adb840 0x564252eb9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:41.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.848+0000 7f7582d9d640 1 --2- 172.21.15.100:0/3981523471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564252afc520 0x564252eb9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:41.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.848+0000 7f758259c640 1 -- 172.21.15.100:0/3981523471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564252adb840 msgr2=0x564252eb9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:41.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.848+0000 7f758259c640 1 --2- 172.21.15.100:0/3981523471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564252adb840 0x564252eb9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:41.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.848+0000 7f758259c640 1 -- 172.21.15.100:0/3981523471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564252afc520 msgr2=0x564252eb9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:41.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.848+0000 7f758259c640 1 --2- 172.21.15.100:0/3981523471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564252afc520 0x564252eb9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:41.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.848+0000 7f758259c640 1 -- 172.21.15.100:0/3981523471 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564252e9ac90 con 0x564252ad3660 2024-07-26T13:41:41.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.848+0000 7f7582d9d640 1 --2- 172.21.15.100:0/3981523471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564252afc520 0x564252eb9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:41:41.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.848+0000 7f7581d9b640 1 --2- 172.21.15.100:0/3981523471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564252adb840 0x564252eb9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:41:41.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.848+0000 7f758259c640 1 --2- 172.21.15.100:0/3981523471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564252ad3660 0x564252eaf5f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7574005fd0 tx=0x7f75740b3040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3179867d7d20af5 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:41.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.849+0000 7f756bfff640 1 -- 172.21.15.100:0/3981523471 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75740e3e60 con 0x564252ad3660 2024-07-26T13:41:41.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.849+0000 7f756bfff640 1 -- 172.21.15.100:0/3981523471 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75740aa640 con 0x564252ad3660 2024-07-26T13:41:41.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.849+0000 7f756bfff640 1 -- 172.21.15.100:0/3981523471 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7574021040 con 0x564252ad3660 2024-07-26T13:41:41.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.849+0000 7f758ef02740 1 -- 172.21.15.100:0/3981523471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564252ad3660 msgr2=0x564252eaf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:41.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.849+0000 7f758ef02740 1 --2- 172.21.15.100:0/3981523471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564252ad3660 0x564252eaf5f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7574005fd0 tx=0x7f75740b3040 comp rx=0 tx=0).stop 2024-07-26T13:41:41.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.850+0000 7f758ef02740 1 -- 172.21.15.100:0/3981523471 shutdown_connections 2024-07-26T13:41:41.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.850+0000 7f758ef02740 1 --2- 172.21.15.100:0/3981523471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564252afc520 0x564252eb9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:41.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.850+0000 7f758ef02740 1 --2- 172.21.15.100:0/3981523471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564252adb840 0x564252eb9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:41.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.850+0000 7f758ef02740 1 --2- 172.21.15.100:0/3981523471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564252ad3660 0x564252eaf5f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:41.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.850+0000 7f758ef02740 1 -- 172.21.15.100:0/3981523471 >> 172.21.15.100:0/3981523471 conn(0x564252af0910 msgr2=0x564252eaeee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:41.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.850+0000 7f758ef02740 1 -- 172.21.15.100:0/3981523471 shutdown_connections 2024-07-26T13:41:41.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.850+0000 7f758ef02740 1 -- 172.21.15.100:0/3981523471 wait complete. 2024-07-26T13:41:41.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.850+0000 7f758ef02740 1 -- start start 2024-07-26T13:41:41.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.850+0000 7f758ef02740 1 Processor -- start 2024-07-26T13:41:41.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.851+0000 7f758ef02740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564252ad3660 0x564252eb2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:41.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.851+0000 7f758ef02740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564252adb840 0x564252eb2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:41.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.851+0000 7f758ef02740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564252afc520 0x564252f273e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:41.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.851+0000 7f758ef02740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f75841af8d0 con 0x564252afc520 2024-07-26T13:41:41.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.851+0000 7f758ef02740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f75841b0d40 con 0x564252adb840 2024-07-26T13:41:41.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.851+0000 7f758ef02740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f75841affa0 con 0x564252ad3660 2024-07-26T13:41:41.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.851+0000 7f758ef02740 10 client.0 _ll_register_callbacks cb 0x7f757800cf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:41:41.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.851+0000 7f7582d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564252afc520 0x564252f273e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:41.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.851+0000 7f7582d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564252afc520 0x564252f273e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45312/0 (socket says 172.21.15.100:45312) 2024-07-26T13:41:41.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.851+0000 7f7582d9d640 1 -- 172.21.15.100:0/1761760138 learned_addr learned my addr 172.21.15.100:0/1761760138 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:41.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.851+0000 7f758259c640 1 --2- 172.21.15.100:0/1761760138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564252ad3660 0x564252eb2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:41.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.851+0000 7f7581d9b640 1 --2- 172.21.15.100:0/1761760138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564252adb840 0x564252eb2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:41.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.851+0000 7f7582d9d640 1 -- 172.21.15.100:0/1761760138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564252ad3660 msgr2=0x564252eb2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:41.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.851+0000 7f7582d9d640 1 --2- 172.21.15.100:0/1761760138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564252ad3660 0x564252eb2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:41.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.851+0000 7f7582d9d640 1 -- 172.21.15.100:0/1761760138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564252adb840 msgr2=0x564252eb2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:41.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.851+0000 7f7582d9d640 1 --2- 172.21.15.100:0/1761760138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564252adb840 0x564252eb2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:41.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.851+0000 7f7582d9d640 1 -- 172.21.15.100:0/1761760138 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564252f27970 con 0x564252afc520 2024-07-26T13:41:41.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.851+0000 7f7582d9d640 1 --2- 172.21.15.100:0/1761760138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564252afc520 0x564252f273e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f757c004780 tx=0x7f757c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a65455babc41ef36 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:41.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.851+0000 7f7569ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:41:41.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.852+0000 7f7569ffb640 1 -- 172.21.15.100:0/1761760138 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c042040 con 0x564252afc520 2024-07-26T13:41:41.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.852+0000 7f7569ffb640 1 -- 172.21.15.100:0/1761760138 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f757c003a30 con 0x564252afc520 2024-07-26T13:41:41.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.852+0000 7f758ef02740 1 -- 172.21.15.100:0/1761760138 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75740aa640 con 0x564252afc520 2024-07-26T13:41:41.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.852+0000 7f7569ffb640 1 -- 172.21.15.100:0/1761760138 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f757c009360 con 0x564252afc520 2024-07-26T13:41:41.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.852+0000 7f758ef02740 10 client.15855 fetch_fsmap learned FSMap version 26 2024-07-26T13:41:41.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.852+0000 7f758ef02740 1 -- 172.21.15.100:0/1761760138 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564252f28ba0 con 0x564252afc520 2024-07-26T13:41:41.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.852+0000 7f7569ffb640 1 -- 172.21.15.100:0/1761760138 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75740aa640 con 0x564252afc520 2024-07-26T13:41:41.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.852+0000 7f7569ffb640 1 -- 172.21.15.100:0/1761760138 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f757c0042e0 con 0x564252afc520 2024-07-26T13:41:41.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.852+0000 7f758ef02740 10 client.15855 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:41:41.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.852+0000 7f758ef02740 10 client.15855 resolve_mds: validated gid 24607 aka daemon mds.i 2024-07-26T13:41:41.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.852+0000 7f758ef02740 20 client.15855 populate_metadata read hostname 'smithi100' 2024-07-26T13:41:41.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.852+0000 7f758ef02740 1 --2- 172.21.15.100:0/1761760138 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x564252a90250 0x564252f2af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:41.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.852+0000 7f758ef02740 4 client.15855 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "28b446d7", "release": true, "await": true}] 2024-07-26T13:41:41.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.852+0000 7f758ef02740 1 -- 172.21.15.100:0/1761760138 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "quiesce db", "set_id": "28b446d7", "release": true, "await": true}) -- 0x564252e15860 con 0x564252a90250 2024-07-26T13:41:41.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.853+0000 7f758259c640 1 --2- 172.21.15.100:0/1761760138 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x564252a90250 0x564252f2af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:41.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.853+0000 7f758259c640 1 --2- 172.21.15.100:0/1761760138 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x564252a90250 0x564252f2af40 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd0ff46bd0b2ba57 server_cookie=83c4210b739d45e in_seq=0 out_seq=0 2024-07-26T13:41:41.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.853+0000 7f7569ffb640 10 client.15855 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:41.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.855+0000 7f7569ffb640 1 -- 172.21.15.100:0/1761760138 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x564252e15860 con 0x564252a90250 2024-07-26T13:41:41.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.855+0000 7f7569ffb640 10 client.15855 handle_command_reply: tid=0 2024-07-26T13:41:41.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.855+0000 7f758ef02740 1 client.15855 shutdown 2024-07-26T13:41:41.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.855+0000 7f758ef02740 1 -- 172.21.15.100:0/1761760138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564252afc520 msgr2=0x564252f273e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:41.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.855+0000 7f758ef02740 1 --2- 172.21.15.100:0/1761760138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564252afc520 0x564252f273e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f757c004780 tx=0x7f757c0095c0 comp rx=0 tx=0).stop 2024-07-26T13:41:41.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.855+0000 7f758ef02740 1 -- 172.21.15.100:0/1761760138 shutdown_connections 2024-07-26T13:41:41.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.855+0000 7f758ef02740 1 --2- 172.21.15.100:0/1761760138 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x564252a90250 0x564252f2af40 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:41.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.855+0000 7f758ef02740 1 --2- 172.21.15.100:0/1761760138 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564252afc520 0x564252f273e0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:41.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.855+0000 7f758ef02740 1 --2- 172.21.15.100:0/1761760138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564252adb840 0x564252eb2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:41.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.855+0000 7f758ef02740 1 --2- 172.21.15.100:0/1761760138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564252ad3660 0x564252eb2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:41.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.855+0000 7f758ef02740 1 -- 172.21.15.100:0/1761760138 >> 172.21.15.100:0/1761760138 conn(0x564252af0910 msgr2=0x564252ea30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:41.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.855+0000 7f7569ffb640 0 client.15855 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:41.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.856+0000 7f758ef02740 1 -- 172.21.15.100:0/1761760138 shutdown_connections 2024-07-26T13:41:41.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.856+0000 7f758ef02740 1 -- 172.21.15.100:0/1761760138 wait complete. 2024-07-26T13:41:41.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.856+0000 7f758ef02740 20 client.15855 trim_cache size 0 max 16384 2024-07-26T13:41:41.861 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:41:41.861 INFO:teuthology.orchestra.run.smithi100.stdout: "epoch": 26, 2024-07-26T13:41:41.861 INFO:teuthology.orchestra.run.smithi100.stdout: "leader": 24607, 2024-07-26T13:41:41.862 INFO:teuthology.orchestra.run.smithi100.stdout: "set_version": 4, 2024-07-26T13:41:41.862 INFO:teuthology.orchestra.run.smithi100.stdout: "sets": { 2024-07-26T13:41:41.862 INFO:teuthology.orchestra.run.smithi100.stdout: "28b446d7": { 2024-07-26T13:41:41.862 INFO:teuthology.orchestra.run.smithi100.stdout: "version": 4, 2024-07-26T13:41:41.862 INFO:teuthology.orchestra.run.smithi100.stdout: "age_ref": 0.0, 2024-07-26T13:41:41.862 INFO:teuthology.orchestra.run.smithi100.stdout: "state": { 2024-07-26T13:41:41.862 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RELEASED", 2024-07-26T13:41:41.862 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 0.0 2024-07-26T13:41:41.862 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-07-26T13:41:41.862 INFO:teuthology.orchestra.run.smithi100.stdout: "timeout": 90.0, 2024-07-26T13:41:41.862 INFO:teuthology.orchestra.run.smithi100.stdout: "expiration": 142.6, 2024-07-26T13:41:41.862 INFO:teuthology.orchestra.run.smithi100.stdout: "members": { 2024-07-26T13:41:41.862 INFO:teuthology.orchestra.run.smithi100.stdout: "file:/": { 2024-07-26T13:41:41.862 INFO:teuthology.orchestra.run.smithi100.stdout: "excluded": false, 2024-07-26T13:41:41.863 INFO:teuthology.orchestra.run.smithi100.stdout: "state": { 2024-07-26T13:41:41.863 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RELEASED", 2024-07-26T13:41:41.863 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 0.0 2024-07-26T13:41:41.863 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:41:41.863 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:41:41.863 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:41:41.863 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:41:41.863 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:41:41.863 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:41:41.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.856+0000 7f758897e640 1 -- 172.21.15.100:0/1662835386 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f756c0b3dc0 msgr2=0x7f756c0b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:41.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.856+0000 7f758897e640 1 --2- 172.21.15.100:0/1662835386 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f756c0b3dc0 0x7f756c0b62d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f757c04d000 tx=0x7f757c04e000 comp rx=0 tx=0).stop 2024-07-26T13:41:41.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.856+0000 7f758897e640 1 -- 172.21.15.100:0/1662835386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7584101a60 msgr2=0x7f7584102a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:41.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.856+0000 7f758897e640 1 --2- 172.21.15.100:0/1662835386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7584101a60 0x7f7584102a00 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f757400d950 tx=0x7f757400de10 comp rx=0 tx=0).stop 2024-07-26T13:41:41.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.857+0000 7f758897e640 1 -- 172.21.15.100:0/1662835386 shutdown_connections 2024-07-26T13:41:41.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.857+0000 7f758897e640 1 --2- 172.21.15.100:0/1662835386 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f756c0b3dc0 0x7f756c0b62d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:41.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.857+0000 7f758897e640 1 --2- 172.21.15.100:0/1662835386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7584106600 0x7f7584103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:41.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.857+0000 7f758897e640 1 --2- 172.21.15.100:0/1662835386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7584105cb0 0x7f7584102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:41.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.857+0000 7f758897e640 1 --2- 172.21.15.100:0/1662835386 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7584101a60 0x7f7584102a00 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:41.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.857+0000 7f758897e640 1 -- 172.21.15.100:0/1662835386 >> 172.21.15.100:0/1662835386 conn(0x7f7584077690 msgr2=0x7f7584106ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:41.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.857+0000 7f758897e640 1 -- 172.21.15.100:0/1662835386 shutdown_connections 2024-07-26T13:41:41.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:41.857+0000 7f758897e640 1 -- 172.21.15.100:0/1662835386 wait complete. 2024-07-26T13:41:41.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:41 smithi138 ceph-mon[36954]: pgmap v411: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-07-26T13:41:42.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:41 smithi167 ceph-mon[37140]: pgmap v411: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-07-26T13:41:43.422 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '28b446d7', total seconds elapsed: 32.2 2024-07-26T13:41:43.422 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.55 2024-07-26T13:41:43.423 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:41:43.488 INFO:tasks.workunit.client.0.smithi100.stderr:Updating files: 93% (11878/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-07-26T13:41:43.644 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:43 smithi100 ceph-mon[32599]: pgmap v412: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-07-26T13:41:43.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.878+0000 7f4af3606640 1 -- 172.21.15.100:0/583252456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4aec107dd0 msgr2=0x7f4aec1081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:43.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.878+0000 7f4af3606640 1 --2- 172.21.15.100:0/583252456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4aec107dd0 0x7f4aec1081e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009900 tx=0x7f4ae002a410 comp rx=0 tx=0).stop 2024-07-26T13:41:43.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.878+0000 7f4af3606640 1 -- 172.21.15.100:0/583252456 shutdown_connections 2024-07-26T13:41:43.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.878+0000 7f4af3606640 1 --2- 172.21.15.100:0/583252456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4aec108720 0x7f4aec10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:43.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.878+0000 7f4af3606640 1 --2- 172.21.15.100:0/583252456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4aec107dd0 0x7f4aec1081e0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:43.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.878+0000 7f4af3606640 1 --2- 172.21.15.100:0/583252456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4aec102490 0x7f4aec1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:43.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.878+0000 7f4af3606640 1 -- 172.21.15.100:0/583252456 >> 172.21.15.100:0/583252456 conn(0x7f4aec079160 msgr2=0x7f4aec079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:43.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.878+0000 7f4af3606640 1 -- 172.21.15.100:0/583252456 shutdown_connections 2024-07-26T13:41:43.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.878+0000 7f4af3606640 1 -- 172.21.15.100:0/583252456 wait complete. 2024-07-26T13:41:43.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.879+0000 7f4af3606640 1 Processor -- start 2024-07-26T13:41:43.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.879+0000 7f4af3606640 1 -- start start 2024-07-26T13:41:43.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.879+0000 7f4af3606640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4aec102490 0x7f4aec19acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:43.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.879+0000 7f4af3606640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4aec107dd0 0x7f4aec19b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:43.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.879+0000 7f4af3606640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4aec108720 0x7f4aec19b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:43.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.879+0000 7f4af3606640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec112350 con 0x7f4aec107dd0 2024-07-26T13:41:43.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.879+0000 7f4af3606640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec1121d0 con 0x7f4aec102490 2024-07-26T13:41:43.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.879+0000 7f4af3606640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec1124d0 con 0x7f4aec108720 2024-07-26T13:41:43.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.879+0000 7f4af0ba1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4aec107dd0 0x7f4aec19b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:43.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.879+0000 7f4af0ba1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4aec107dd0 0x7f4aec19b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45336/0 (socket says 172.21.15.100:45336) 2024-07-26T13:41:43.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.879+0000 7f4af0ba1640 1 -- 172.21.15.100:0/341230497 learned_addr learned my addr 172.21.15.100:0/341230497 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:43.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.880+0000 7f4af1ba3640 1 --2- 172.21.15.100:0/341230497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4aec108720 0x7f4aec19b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:43.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.880+0000 7f4af13a2640 1 --2- 172.21.15.100:0/341230497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4aec102490 0x7f4aec19acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:43.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.880+0000 7f4af0ba1640 1 -- 172.21.15.100:0/341230497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4aec108720 msgr2=0x7f4aec19b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:43.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.880+0000 7f4af0ba1640 1 --2- 172.21.15.100:0/341230497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4aec108720 0x7f4aec19b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:43.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.880+0000 7f4af0ba1640 1 -- 172.21.15.100:0/341230497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4aec102490 msgr2=0x7f4aec19acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:43.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.880+0000 7f4af0ba1640 1 --2- 172.21.15.100:0/341230497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4aec102490 0x7f4aec19acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:43.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.880+0000 7f4af0ba1640 1 -- 172.21.15.100:0/341230497 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aec1a5f80 con 0x7f4aec107dd0 2024-07-26T13:41:43.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.880+0000 7f4af0ba1640 1 --2- 172.21.15.100:0/341230497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4aec107dd0 0x7f4aec19b200 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0032fd0 tx=0x7f4ae0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:43.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.880+0000 7f4ade7fc640 1 -- 172.21.15.100:0/341230497 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae003f030 con 0x7f4aec107dd0 2024-07-26T13:41:43.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.880+0000 7f4ade7fc640 1 -- 172.21.15.100:0/341230497 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ae0004490 con 0x7f4aec107dd0 2024-07-26T13:41:43.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.880+0000 7f4af3606640 1 -- 172.21.15.100:0/341230497 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aec1a6210 con 0x7f4aec107dd0 2024-07-26T13:41:43.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.880+0000 7f4ade7fc640 1 -- 172.21.15.100:0/341230497 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0035070 con 0x7f4aec107dd0 2024-07-26T13:41:43.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.880+0000 7f4af3606640 1 -- 172.21.15.100:0/341230497 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aec1a66f0 con 0x7f4aec107dd0 2024-07-26T13:41:43.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.883+0000 7f4ade7fc640 1 -- 172.21.15.100:0/341230497 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f4ae0042030 con 0x7f4aec107dd0 2024-07-26T13:41:43.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.884+0000 7f4ade7fc640 1 --2- 172.21.15.100:0/341230497 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f4ad80b3dc0 0x7f4ad80b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:43.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.884+0000 7f4af9b8a740 1 Processor -- start 2024-07-26T13:41:43.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.884+0000 7f4af9b8a740 1 -- start start 2024-07-26T13:41:43.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.884+0000 7f4af13a2640 1 --2- 172.21.15.100:0/341230497 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f4ad80b3dc0 0x7f4ad80b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:43.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.884+0000 7f4ade7fc640 1 -- 172.21.15.100:0/341230497 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f4ae0104150 con 0x7f4aec107dd0 2024-07-26T13:41:43.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.884+0000 7f4af9b8a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580c80c4070 0x5580c8469570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:43.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.884+0000 7f4af9b8a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c809b1b0 0x5580c8469ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:43.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.884+0000 7f4af9b8a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580c80a3390 0x5580c8469ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:43.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.884+0000 7f4af9b8a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5580c83dfbb0 con 0x5580c809b1b0 2024-07-26T13:41:43.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.884+0000 7f4af9b8a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5580c83cf0b0 con 0x5580c80a3390 2024-07-26T13:41:43.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.884+0000 7f4af9b8a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5580c83d30f0 con 0x5580c80c4070 2024-07-26T13:41:43.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.884+0000 7f4af0ba1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c809b1b0 0x5580c8469ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:43.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.884+0000 7f4af13a2640 1 --2- 172.21.15.100:0/341230497 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f4ad80b3dc0 0x7f4ad80b62d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4aec06b410 tx=0x7f4ae800b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:43.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.884+0000 7f4af0ba1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c809b1b0 0x5580c8469ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45342/0 (socket says 172.21.15.100:45342) 2024-07-26T13:41:43.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.885+0000 7f4af0ba1640 1 -- 172.21.15.100:0/1946641529 learned_addr learned my addr 172.21.15.100:0/1946641529 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:43.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.885+0000 7f4af13a2640 1 --2- 172.21.15.100:0/1946641529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580c80c4070 0x5580c8469570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:43.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.885+0000 7f4af1ba3640 1 --2- 172.21.15.100:0/1946641529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580c80a3390 0x5580c8469ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:43.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.885+0000 7f4af0ba1640 1 -- 172.21.15.100:0/1946641529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580c80c4070 msgr2=0x5580c8469570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:43.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.885+0000 7f4af0ba1640 1 --2- 172.21.15.100:0/1946641529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580c80c4070 0x5580c8469570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:43.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.885+0000 7f4af0ba1640 1 -- 172.21.15.100:0/1946641529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580c80a3390 msgr2=0x5580c8469ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:43.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.885+0000 7f4af0ba1640 1 --2- 172.21.15.100:0/1946641529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580c80a3390 0x5580c8469ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:43.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.885+0000 7f4af0ba1640 1 -- 172.21.15.100:0/1946641529 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580c8375b90 con 0x5580c809b1b0 2024-07-26T13:41:43.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.885+0000 7f4af1ba3640 1 --2- 172.21.15.100:0/1946641529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580c80a3390 0x5580c8469ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:41:43.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.885+0000 7f4af0ba1640 1 --2- 172.21.15.100:0/1946641529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c809b1b0 0x5580c8469ab0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4ae00cad60 tx=0x7f4ae0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e56587ff46acc6 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:43.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.885+0000 7f4ad77fe640 1 -- 172.21.15.100:0/1946641529 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0049070 con 0x5580c809b1b0 2024-07-26T13:41:43.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.885+0000 7f4ad77fe640 1 -- 172.21.15.100:0/1946641529 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ae00cf320 con 0x5580c809b1b0 2024-07-26T13:41:43.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.885+0000 7f4ad77fe640 1 -- 172.21.15.100:0/1946641529 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae003bc20 con 0x5580c809b1b0 2024-07-26T13:41:43.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.886+0000 7f4af9b8a740 1 -- 172.21.15.100:0/1946641529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c809b1b0 msgr2=0x5580c8469ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:43.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.886+0000 7f4af9b8a740 1 --2- 172.21.15.100:0/1946641529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c809b1b0 0x5580c8469ab0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4ae00cad60 tx=0x7f4ae0000bc0 comp rx=0 tx=0).stop 2024-07-26T13:41:43.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.886+0000 7f4af9b8a740 1 -- 172.21.15.100:0/1946641529 shutdown_connections 2024-07-26T13:41:43.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.886+0000 7f4af9b8a740 1 --2- 172.21.15.100:0/1946641529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580c80a3390 0x5580c8469ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:43.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.886+0000 7f4af9b8a740 1 --2- 172.21.15.100:0/1946641529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c809b1b0 0x5580c8469ab0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:43.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.886+0000 7f4af9b8a740 1 --2- 172.21.15.100:0/1946641529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580c80c4070 0x5580c8469570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:43.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.886+0000 7f4af9b8a740 1 -- 172.21.15.100:0/1946641529 >> 172.21.15.100:0/1946641529 conn(0x5580c80b8460 msgr2=0x5580c8467460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:43.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.886+0000 7f4af9b8a740 1 -- 172.21.15.100:0/1946641529 shutdown_connections 2024-07-26T13:41:43.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.886+0000 7f4af9b8a740 1 -- 172.21.15.100:0/1946641529 wait complete. 2024-07-26T13:41:43.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.887+0000 7f4af9b8a740 1 -- start start 2024-07-26T13:41:43.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.887+0000 7f4af9b8a740 1 Processor -- start 2024-07-26T13:41:43.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.887+0000 7f4af9b8a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c809b1b0 0x5580c8475230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:43.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.887+0000 7f4af9b8a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580c80a3390 0x5580c84757f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:43.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.887+0000 7f4af9b8a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580c80c4070 0x5580c847f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:43.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.887+0000 7f4af9b8a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5580c8232700 con 0x5580c809b1b0 2024-07-26T13:41:43.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.887+0000 7f4af9b8a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5580c83719e0 con 0x5580c80a3390 2024-07-26T13:41:43.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.887+0000 7f4af9b8a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5580c847bf20 con 0x5580c80c4070 2024-07-26T13:41:43.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.887+0000 7f4af9b8a740 10 client.0 _ll_register_callbacks cb 0x5580c838c4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:41:43.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.887+0000 7f4af13a2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c809b1b0 0x5580c8475230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:43.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.887+0000 7f4af13a2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c809b1b0 0x5580c8475230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45344/0 (socket says 172.21.15.100:45344) 2024-07-26T13:41:43.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.887+0000 7f4af13a2640 1 -- 172.21.15.100:0/825057400 learned_addr learned my addr 172.21.15.100:0/825057400 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:43.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.887+0000 7f4af0ba1640 1 --2- 172.21.15.100:0/825057400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580c80a3390 0x5580c84757f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:43.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.888+0000 7f4af13a2640 1 -- 172.21.15.100:0/825057400 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580c80c4070 msgr2=0x5580c847f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:43.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.888+0000 7f4af1ba3640 1 --2- 172.21.15.100:0/825057400 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580c80c4070 0x5580c847f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:43.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.888+0000 7f4af13a2640 1 --2- 172.21.15.100:0/825057400 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580c80c4070 0x5580c847f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:43.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.888+0000 7f4af13a2640 1 -- 172.21.15.100:0/825057400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580c80a3390 msgr2=0x5580c84757f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:43.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.888+0000 7f4af13a2640 1 --2- 172.21.15.100:0/825057400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580c80a3390 0x5580c84757f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:43.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.888+0000 7f4af13a2640 1 -- 172.21.15.100:0/825057400 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580c812f650 con 0x5580c809b1b0 2024-07-26T13:41:43.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.888+0000 7f4af0ba1640 1 --2- 172.21.15.100:0/825057400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580c80a3390 0x5580c84757f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:41:43.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.888+0000 7f4af1ba3640 1 --2- 172.21.15.100:0/825057400 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580c80c4070 0x5580c847f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:41:43.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.888+0000 7f4af13a2640 1 --2- 172.21.15.100:0/825057400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c809b1b0 0x5580c8475230 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8011890 tx=0x7f4ae8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77cbcd956b1eb0d7 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:43.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.888+0000 7f4ad4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:41:43.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.888+0000 7f4ad4ff9640 1 -- 172.21.15.100:0/825057400 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae801eb40 con 0x5580c809b1b0 2024-07-26T13:41:43.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.888+0000 7f4af9b8a740 1 -- 172.21.15.100:0/825057400 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ae00cf320 con 0x5580c809b1b0 2024-07-26T13:41:43.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.888+0000 7f4ad4ff9640 1 -- 172.21.15.100:0/825057400 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f4ae801ece0 con 0x5580c809b1b0 2024-07-26T13:41:43.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.888+0000 7f4ad4ff9640 1 -- 172.21.15.100:0/825057400 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4ae8011e10 con 0x5580c809b1b0 2024-07-26T13:41:43.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.888+0000 7f4af9b8a740 10 client.15867 fetch_fsmap learned FSMap version 26 2024-07-26T13:41:43.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.888+0000 7f4af9b8a740 1 -- 172.21.15.100:0/825057400 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5580c8487fe0 con 0x5580c809b1b0 2024-07-26T13:41:43.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.889+0000 7f4ad4ff9640 1 -- 172.21.15.100:0/825057400 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ae00cf320 con 0x5580c809b1b0 2024-07-26T13:41:43.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.889+0000 7f4ad4ff9640 1 -- 172.21.15.100:0/825057400 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f4ae802f070 con 0x5580c809b1b0 2024-07-26T13:41:43.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.889+0000 7f4af9b8a740 10 client.15867 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:41:43.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.889+0000 7f4af9b8a740 10 client.15867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:41:43.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.889+0000 7f4af9b8a740 20 client.15867 populate_metadata read hostname 'smithi100' 2024-07-26T13:41:43.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.889+0000 7f4af9b8a740 1 --2- 172.21.15.100:0/825057400 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5580c80829f0 0x5580c848b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:43.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.889+0000 7f4af9b8a740 4 client.15867 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:41:43.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.889+0000 7f4af9b8a740 1 -- 172.21.15.100:0/825057400 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580c83e1b00 con 0x5580c80829f0 2024-07-26T13:41:43.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.889+0000 7f4af0ba1640 1 --2- 172.21.15.100:0/825057400 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5580c80829f0 0x5580c848b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:43.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.890+0000 7f4ad4ff9640 10 client.15867 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:43.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.890+0000 7f4af0ba1640 1 --2- 172.21.15.100:0/825057400 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5580c80829f0 0x5580c848b530 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb20fb33f817062b server_cookie=d7d76f3422317091 in_seq=0 out_seq=0 2024-07-26T13:41:43.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.891+0000 7f4ad4ff9640 1 -- 172.21.15.100:0/825057400 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5580c83e1b00 con 0x5580c80829f0 2024-07-26T13:41:43.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.891+0000 7f4ad4ff9640 10 client.15867 handle_command_reply: tid=0 2024-07-26T13:41:43.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.891+0000 7f4af9b8a740 1 client.15867 shutdown 2024-07-26T13:41:43.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.891+0000 7f4af9b8a740 1 -- 172.21.15.100:0/825057400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c809b1b0 msgr2=0x5580c8475230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:43.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.891+0000 7f4af9b8a740 1 --2- 172.21.15.100:0/825057400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c809b1b0 0x5580c8475230 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8011890 tx=0x7f4ae8004730 comp rx=0 tx=0).stop 2024-07-26T13:41:43.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.891+0000 7f4af9b8a740 1 -- 172.21.15.100:0/825057400 shutdown_connections 2024-07-26T13:41:43.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.891+0000 7f4af9b8a740 1 --2- 172.21.15.100:0/825057400 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5580c80829f0 0x5580c848b530 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:43.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.891+0000 7f4af9b8a740 1 --2- 172.21.15.100:0/825057400 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580c80c4070 0x5580c847f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:43.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.891+0000 7f4ad4ff9640 0 client.15867 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:43.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.891+0000 7f4af9b8a740 1 --2- 172.21.15.100:0/825057400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580c80a3390 0x5580c84757f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:43.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.891+0000 7f4af9b8a740 1 --2- 172.21.15.100:0/825057400 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c809b1b0 0x5580c8475230 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:43.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.891+0000 7f4af9b8a740 1 -- 172.21.15.100:0/825057400 >> 172.21.15.100:0/825057400 conn(0x5580c80b8460 msgr2=0x5580c8465c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:43.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.891+0000 7f4af9b8a740 1 -- 172.21.15.100:0/825057400 shutdown_connections 2024-07-26T13:41:43.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.891+0000 7f4af9b8a740 1 -- 172.21.15.100:0/825057400 wait complete. 2024-07-26T13:41:43.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.891+0000 7f4af9b8a740 20 client.15867 trim_cache size 0 max 16384 2024-07-26T13:41:43.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.904+0000 7f4af9b8a740 1 Processor -- start 2024-07-26T13:41:43.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.904+0000 7f4af9b8a740 1 -- start start 2024-07-26T13:41:43.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.905+0000 7f4af9b8a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580c80829f0 0x5580c847f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:43.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.905+0000 7f4af9b8a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580c809b1b0 0x5580c84674f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:43.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.905+0000 7f4af9b8a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c80a3390 0x5580c846acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:43.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.905+0000 7f4af9b8a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec1af770 con 0x5580c80a3390 2024-07-26T13:41:43.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.905+0000 7f4af9b8a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec1ad040 con 0x5580c80829f0 2024-07-26T13:41:43.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.905+0000 7f4af9b8a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec1aae90 con 0x5580c809b1b0 2024-07-26T13:41:43.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.905+0000 7f4af1ba3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c80a3390 0x5580c846acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:43.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.905+0000 7f4af1ba3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c80a3390 0x5580c846acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45346/0 (socket says 172.21.15.100:45346) 2024-07-26T13:41:43.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.905+0000 7f4af1ba3640 1 -- 172.21.15.100:0/527692013 learned_addr learned my addr 172.21.15.100:0/527692013 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:43.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.905+0000 7f4af13a2640 1 --2- 172.21.15.100:0/527692013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580c80829f0 0x5580c847f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:43.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.905+0000 7f4af1ba3640 1 -- 172.21.15.100:0/527692013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580c809b1b0 msgr2=0x5580c84674f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:43.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.905+0000 7f4af0ba1640 1 --2- 172.21.15.100:0/527692013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580c809b1b0 0x5580c84674f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:43.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.905+0000 7f4af1ba3640 1 --2- 172.21.15.100:0/527692013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580c809b1b0 0x5580c84674f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:43.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.905+0000 7f4af1ba3640 1 -- 172.21.15.100:0/527692013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580c80829f0 msgr2=0x5580c847f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:43.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.905+0000 7f4af1ba3640 1 --2- 172.21.15.100:0/527692013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580c80829f0 0x5580c847f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:43.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.905+0000 7f4af1ba3640 1 -- 172.21.15.100:0/527692013 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580c8462b10 con 0x5580c80a3390 2024-07-26T13:41:43.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.905+0000 7f4af13a2640 1 --2- 172.21.15.100:0/527692013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580c80829f0 0x5580c847f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:41:43.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.905+0000 7f4af0ba1640 1 --2- 172.21.15.100:0/527692013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580c809b1b0 0x5580c84674f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:41:43.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.906+0000 7f4af1ba3640 1 --2- 172.21.15.100:0/527692013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c80a3390 0x5580c846acb0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4ae400af00 tx=0x7f4ae4012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caef5c82258aa43a server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:43.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.906+0000 7f4ad6ffd640 1 -- 172.21.15.100:0/527692013 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae401f070 con 0x5580c80a3390 2024-07-26T13:41:43.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.906+0000 7f4ad6ffd640 1 -- 172.21.15.100:0/527692013 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ae4010d50 con 0x5580c80a3390 2024-07-26T13:41:43.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.906+0000 7f4ad6ffd640 1 -- 172.21.15.100:0/527692013 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae401ae20 con 0x5580c80a3390 2024-07-26T13:41:43.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.907+0000 7f4af9b8a740 1 -- 172.21.15.100:0/527692013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c80a3390 msgr2=0x5580c846acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:43.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.907+0000 7f4af9b8a740 1 --2- 172.21.15.100:0/527692013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c80a3390 0x5580c846acb0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4ae400af00 tx=0x7f4ae4012820 comp rx=0 tx=0).stop 2024-07-26T13:41:43.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.907+0000 7f4af9b8a740 1 -- 172.21.15.100:0/527692013 shutdown_connections 2024-07-26T13:41:43.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.907+0000 7f4af9b8a740 1 --2- 172.21.15.100:0/527692013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c80a3390 0x5580c846acb0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:43.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.907+0000 7f4af9b8a740 1 --2- 172.21.15.100:0/527692013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580c809b1b0 0x5580c84674f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:43.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.907+0000 7f4af9b8a740 1 --2- 172.21.15.100:0/527692013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580c80829f0 0x5580c847f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:43.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.907+0000 7f4af9b8a740 1 -- 172.21.15.100:0/527692013 >> 172.21.15.100:0/527692013 conn(0x5580c80b8460 msgr2=0x5580c846a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:43.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.907+0000 7f4af9b8a740 1 -- 172.21.15.100:0/527692013 shutdown_connections 2024-07-26T13:41:43.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.907+0000 7f4af9b8a740 1 -- 172.21.15.100:0/527692013 wait complete. 2024-07-26T13:41:43.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.907+0000 7f4af9b8a740 1 -- start start 2024-07-26T13:41:43.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.908+0000 7f4af9b8a740 1 Processor -- start 2024-07-26T13:41:43.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.908+0000 7f4af9b8a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c80829f0 0x5580c8479400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:43.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.908+0000 7f4af9b8a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580c809b1b0 0x5580c8479980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:43.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.908+0000 7f4af9b8a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580c80a3390 0x5580c8484d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:43.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.908+0000 7f4af9b8a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec1ab500 con 0x5580c80829f0 2024-07-26T13:41:43.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.908+0000 7f4af9b8a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec1ac970 con 0x5580c809b1b0 2024-07-26T13:41:43.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.908+0000 7f4af9b8a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec1abbd0 con 0x5580c80a3390 2024-07-26T13:41:43.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.908+0000 7f4af9b8a740 10 client.0 _ll_register_callbacks cb 0x7f4ae800bd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:41:43.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.908+0000 7f4af13a2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c80829f0 0x5580c8479400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:43.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.908+0000 7f4af13a2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c80829f0 0x5580c8479400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45352/0 (socket says 172.21.15.100:45352) 2024-07-26T13:41:43.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.908+0000 7f4af13a2640 1 -- 172.21.15.100:0/1604181754 learned_addr learned my addr 172.21.15.100:0/1604181754 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:43.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.908+0000 7f4af1ba3640 1 --2- 172.21.15.100:0/1604181754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580c80a3390 0x5580c8484d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:43.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.909+0000 7f4af0ba1640 1 --2- 172.21.15.100:0/1604181754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580c809b1b0 0x5580c8479980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:43.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.909+0000 7f4af13a2640 1 -- 172.21.15.100:0/1604181754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580c80a3390 msgr2=0x5580c8484d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:43.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.909+0000 7f4af13a2640 1 --2- 172.21.15.100:0/1604181754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580c80a3390 0x5580c8484d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:43.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.909+0000 7f4af13a2640 1 -- 172.21.15.100:0/1604181754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580c809b1b0 msgr2=0x5580c8479980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:43.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.909+0000 7f4af13a2640 1 --2- 172.21.15.100:0/1604181754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580c809b1b0 0x5580c8479980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:43.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.909+0000 7f4af13a2640 1 -- 172.21.15.100:0/1604181754 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580c8485270 con 0x5580c80829f0 2024-07-26T13:41:43.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.909+0000 7f4af1ba3640 1 --2- 172.21.15.100:0/1604181754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580c80a3390 0x5580c8484d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:41:43.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.909+0000 7f4af13a2640 1 --2- 172.21.15.100:0/1604181754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c80829f0 0x5580c8479400 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4ae801f590 tx=0x7f4ae801fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44dc30ff94ba3596 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:43.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.909+0000 7f4ad4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:41:43.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.909+0000 7f4ad4ff9640 1 -- 172.21.15.100:0/1604181754 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae803f020 con 0x5580c80829f0 2024-07-26T13:41:43.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.909+0000 7f4af9b8a740 1 -- 172.21.15.100:0/1604181754 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ae4010d50 con 0x5580c80829f0 2024-07-26T13:41:43.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.909+0000 7f4ad4ff9640 1 -- 172.21.15.100:0/1604181754 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f4ae801fdb0 con 0x5580c80829f0 2024-07-26T13:41:43.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.909+0000 7f4ad4ff9640 1 -- 172.21.15.100:0/1604181754 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4ae8035040 con 0x5580c80829f0 2024-07-26T13:41:43.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.909+0000 7f4af9b8a740 10 client.15873 fetch_fsmap learned FSMap version 26 2024-07-26T13:41:43.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.909+0000 7f4af9b8a740 1 -- 172.21.15.100:0/1604181754 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5580c84f84f0 con 0x5580c80829f0 2024-07-26T13:41:43.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.910+0000 7f4ad4ff9640 1 -- 172.21.15.100:0/1604181754 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ae4010d50 con 0x5580c80829f0 2024-07-26T13:41:43.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.910+0000 7f4ad4ff9640 1 -- 172.21.15.100:0/1604181754 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f4ae802f040 con 0x5580c80829f0 2024-07-26T13:41:43.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.910+0000 7f4af9b8a740 10 client.15873 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:41:43.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.910+0000 7f4af9b8a740 10 client.15873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:41:43.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.910+0000 7f4af9b8a740 20 client.15873 populate_metadata read hostname 'smithi100' 2024-07-26T13:41:43.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.910+0000 7f4af9b8a740 1 --2- 172.21.15.100:0/1604181754 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5580c80c4070 0x5580c84faa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:43.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.910+0000 7f4af9b8a740 4 client.15873 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-26T13:41:43.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.910+0000 7f4af9b8a740 1 -- 172.21.15.100:0/1604181754 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5580c83dd5a0 con 0x5580c80c4070 2024-07-26T13:41:43.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.910+0000 7f4af0ba1640 1 --2- 172.21.15.100:0/1604181754 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5580c80c4070 0x5580c84faa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:43.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.911+0000 7f4ad4ff9640 10 client.15873 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:43.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.911+0000 7f4af0ba1640 1 --2- 172.21.15.100:0/1604181754 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5580c80c4070 0x5580c84faa60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3becec898fb2505 server_cookie=e989b69afa43db7e in_seq=0 out_seq=0 2024-07-26T13:41:43.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.911+0000 7f4ad4ff9640 1 -- 172.21.15.100:0/1604181754 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5580c83dd5a0 con 0x5580c80c4070 2024-07-26T13:41:43.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.911+0000 7f4ad4ff9640 10 client.15873 handle_command_reply: tid=0 2024-07-26T13:41:43.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.911+0000 7f4af9b8a740 1 client.15873 shutdown 2024-07-26T13:41:43.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.911+0000 7f4af9b8a740 1 -- 172.21.15.100:0/1604181754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c80829f0 msgr2=0x5580c8479400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:43.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.911+0000 7f4af9b8a740 1 --2- 172.21.15.100:0/1604181754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c80829f0 0x5580c8479400 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4ae801f590 tx=0x7f4ae801fab0 comp rx=0 tx=0).stop 2024-07-26T13:41:43.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.912+0000 7f4af9b8a740 1 -- 172.21.15.100:0/1604181754 shutdown_connections 2024-07-26T13:41:43.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.912+0000 7f4af9b8a740 1 --2- 172.21.15.100:0/1604181754 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5580c80c4070 0x5580c84faa60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:43.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.912+0000 7f4af9b8a740 1 --2- 172.21.15.100:0/1604181754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5580c80a3390 0x5580c8484d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:43.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.912+0000 7f4ad4ff9640 0 client.15873 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:43.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.912+0000 7f4af9b8a740 1 --2- 172.21.15.100:0/1604181754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580c809b1b0 0x5580c8479980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:43.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.912+0000 7f4af9b8a740 1 --2- 172.21.15.100:0/1604181754 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5580c80829f0 0x5580c8479400 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:43.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.912+0000 7f4af9b8a740 1 -- 172.21.15.100:0/1604181754 >> 172.21.15.100:0/1604181754 conn(0x5580c80b8460 msgr2=0x5580c846aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:43.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.912+0000 7f4af9b8a740 1 -- 172.21.15.100:0/1604181754 shutdown_connections 2024-07-26T13:41:43.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.912+0000 7f4af9b8a740 1 -- 172.21.15.100:0/1604181754 wait complete. 2024-07-26T13:41:43.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.912+0000 7f4af9b8a740 20 client.15873 trim_cache size 0 max 16384 2024-07-26T13:41:43.918 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-07-26T13:41:43.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.913+0000 7f4af3606640 1 -- 172.21.15.100:0/341230497 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f4ad80b3dc0 msgr2=0x7f4ad80b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:43.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.913+0000 7f4af3606640 1 --2- 172.21.15.100:0/341230497 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f4ad80b3dc0 0x7f4ad80b62d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4aec06b410 tx=0x7f4ae800b410 comp rx=0 tx=0).stop 2024-07-26T13:41:43.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.913+0000 7f4af3606640 1 -- 172.21.15.100:0/341230497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4aec107dd0 msgr2=0x7f4aec19b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:43.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.913+0000 7f4af3606640 1 --2- 172.21.15.100:0/341230497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4aec107dd0 0x7f4aec19b200 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0032fd0 tx=0x7f4ae0004270 comp rx=0 tx=0).stop 2024-07-26T13:41:43.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.913+0000 7f4af3606640 1 -- 172.21.15.100:0/341230497 shutdown_connections 2024-07-26T13:41:43.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.913+0000 7f4af3606640 1 --2- 172.21.15.100:0/341230497 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f4ad80b3dc0 0x7f4ad80b62d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:43.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.913+0000 7f4af3606640 1 --2- 172.21.15.100:0/341230497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4aec108720 0x7f4aec19b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:43.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.913+0000 7f4af3606640 1 --2- 172.21.15.100:0/341230497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4aec107dd0 0x7f4aec19b200 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:43.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.913+0000 7f4af3606640 1 --2- 172.21.15.100:0/341230497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4aec102490 0x7f4aec19acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:43.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.913+0000 7f4af3606640 1 -- 172.21.15.100:0/341230497 >> 172.21.15.100:0/341230497 conn(0x7f4aec079160 msgr2=0x7f4aec104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:43.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.913+0000 7f4af3606640 1 -- 172.21.15.100:0/341230497 shutdown_connections 2024-07-26T13:41:43.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:43.913+0000 7f4af3606640 1 -- 172.21.15.100:0/341230497 wait complete. 2024-07-26T13:41:43.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:43 smithi138 ceph-mon[36954]: pgmap v412: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-07-26T13:41:44.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:43 smithi167 ceph-mon[37140]: pgmap v412: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-07-26T13:41:44.100 INFO:tasks.workunit.client.0.smithi100.stderr:Note: switching to 'e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528'. 2024-07-26T13:41:44.100 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-07-26T13:41:44.100 INFO:tasks.workunit.client.0.smithi100.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-07-26T13:41:44.100 INFO:tasks.workunit.client.0.smithi100.stderr:changes and commit them, and you can discard any commits you make in this 2024-07-26T13:41:44.100 INFO:tasks.workunit.client.0.smithi100.stderr:state without impacting any branches by switching back to a branch. 2024-07-26T13:41:44.100 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-07-26T13:41:44.101 INFO:tasks.workunit.client.0.smithi100.stderr:If you want to create a new branch to retain commits you create, you may 2024-07-26T13:41:44.101 INFO:tasks.workunit.client.0.smithi100.stderr:do so (now or later) by using -c with the switch command. Example: 2024-07-26T13:41:44.101 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-07-26T13:41:44.101 INFO:tasks.workunit.client.0.smithi100.stderr: git switch -c 2024-07-26T13:41:44.101 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-07-26T13:41:44.101 INFO:tasks.workunit.client.0.smithi100.stderr:Or undo this operation with: 2024-07-26T13:41:44.101 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-07-26T13:41:44.101 INFO:tasks.workunit.client.0.smithi100.stderr: git switch - 2024-07-26T13:41:44.101 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-07-26T13:41:44.101 INFO:tasks.workunit.client.0.smithi100.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-07-26T13:41:44.101 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-07-26T13:41:44.101 INFO:tasks.workunit.client.0.smithi100.stderr:HEAD is now at e8f88bb1fe0 Merge PR #54435 into wip-vshankar-testing-20240725.121331-debug 2024-07-26T13:41:44.103 DEBUG:teuthology.orchestra.run.smithi100:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-07-26T13:41:44.198 INFO:tasks.workunit.client.0.smithi100.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-07-26T13:41:44.200 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-07-26T13:41:44.200 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-07-26T13:41:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.244+0000 7f0108ccb640 1 -- 172.21.15.100:0/1072552793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01041097b0 msgr2=0x7f010410ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.244+0000 7f0108ccb640 1 --2- 172.21.15.100:0/1072552793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01041097b0 0x7f010410ffe0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f00f4009900 tx=0x7f00f402a410 comp rx=0 tx=0).stop 2024-07-26T13:41:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.245+0000 7f0108ccb640 1 -- 172.21.15.100:0/1072552793 shutdown_connections 2024-07-26T13:41:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.245+0000 7f0108ccb640 1 --2- 172.21.15.100:0/1072552793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01041097b0 0x7f010410ffe0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.245+0000 7f0108ccb640 1 --2- 172.21.15.100:0/1072552793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0104108e60 0x7f0104109270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:44.247 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.245+0000 7f0108ccb640 1 --2- 172.21.15.100:0/1072552793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0104101a90 0x7f0104101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.245+0000 7f0108ccb640 1 -- 172.21.15.100:0/1072552793 >> 172.21.15.100:0/1072552793 conn(0x7f01040798c0 msgr2=0x7f0104079cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.245+0000 7f0108ccb640 1 -- 172.21.15.100:0/1072552793 shutdown_connections 2024-07-26T13:41:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.245+0000 7f0108ccb640 1 -- 172.21.15.100:0/1072552793 wait complete. 2024-07-26T13:41:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.245+0000 7f0108ccb640 1 Processor -- start 2024-07-26T13:41:44.248 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.245+0000 7f0108ccb640 1 -- start start 2024-07-26T13:41:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.246+0000 7f0108ccb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0104101a90 0x7f0104199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.246+0000 7f0108ccb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0104108e60 0x7f010419a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.246+0000 7f0108ccb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01041097b0 0x7f010419f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.246+0000 7f0108ccb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0104113480 con 0x7f01041097b0 2024-07-26T13:41:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.246+0000 7f0108ccb640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0104113300 con 0x7f0104108e60 2024-07-26T13:41:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.246+0000 7f0108ccb640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0104113600 con 0x7f0104101a90 2024-07-26T13:41:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.246+0000 7f0102d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01041097b0 0x7f010419f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.246+0000 7f0102d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01041097b0 0x7f010419f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45370/0 (socket says 172.21.15.100:45370) 2024-07-26T13:41:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.246+0000 7f0102d9d640 1 -- 172.21.15.100:0/412995142 learned_addr learned my addr 172.21.15.100:0/412995142 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:44.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.246+0000 7f010259c640 1 --2- 172.21.15.100:0/412995142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0104101a90 0x7f0104199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.246+0000 7f0102d9d640 1 -- 172.21.15.100:0/412995142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0104101a90 msgr2=0x7f0104199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.246+0000 7f0102d9d640 1 --2- 172.21.15.100:0/412995142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0104101a90 0x7f0104199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.246+0000 7f0101d9b640 1 --2- 172.21.15.100:0/412995142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0104108e60 0x7f010419a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.246+0000 7f0102d9d640 1 -- 172.21.15.100:0/412995142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0104108e60 msgr2=0x7f010419a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.246+0000 7f0102d9d640 1 --2- 172.21.15.100:0/412995142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0104108e60 0x7f010419a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.246+0000 7f0102d9d640 1 -- 172.21.15.100:0/412995142 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f010419fd70 con 0x7f01041097b0 2024-07-26T13:41:44.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.246+0000 7f0101d9b640 1 --2- 172.21.15.100:0/412995142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0104108e60 0x7f010419a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:41:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.246+0000 7f0102d9d640 1 --2- 172.21.15.100:0/412995142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01041097b0 0x7f010419f670 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f00f4004770 tx=0x7f00f4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.246+0000 7f00fb7fe640 1 -- 172.21.15.100:0/412995142 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f403f030 con 0x7f01041097b0 2024-07-26T13:41:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.247+0000 7f00fb7fe640 1 -- 172.21.15.100:0/412995142 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00f4003aa0 con 0x7f01041097b0 2024-07-26T13:41:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.247+0000 7f0108ccb640 1 -- 172.21.15.100:0/412995142 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01041a0000 con 0x7f01041097b0 2024-07-26T13:41:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.247+0000 7f00fb7fe640 1 -- 172.21.15.100:0/412995142 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f4035070 con 0x7f01041097b0 2024-07-26T13:41:44.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.247+0000 7f0108ccb640 1 -- 172.21.15.100:0/412995142 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01041a67f0 con 0x7f01041097b0 2024-07-26T13:41:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.249+0000 7f010fbde740 1 Processor -- start 2024-07-26T13:41:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.249+0000 7f00fb7fe640 1 -- 172.21.15.100:0/412995142 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f00f4042030 con 0x7f01041097b0 2024-07-26T13:41:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.250+0000 7f010fbde740 1 -- start start 2024-07-26T13:41:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.250+0000 7f010fbde740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e220381b0 0x560e223da6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.250+0000 7f010fbde740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e2200f2f0 0x560e223dabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.250+0000 7f010fbde740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e220174d0 0x560e223db130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.250+0000 7f010fbde740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x560e223503b0 con 0x560e220381b0 2024-07-26T13:41:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.250+0000 7f010fbde740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560e2233f8b0 con 0x560e220174d0 2024-07-26T13:41:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.250+0000 7f010fbde740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560e223438f0 con 0x560e2200f2f0 2024-07-26T13:41:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.250+0000 7f0101d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e220381b0 0x560e223da6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.250+0000 7f0101d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e220381b0 0x560e223da6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45382/0 (socket says 172.21.15.100:45382) 2024-07-26T13:41:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.250+0000 7f0101d9b640 1 -- 172.21.15.100:0/3523469649 learned_addr learned my addr 172.21.15.100:0/3523469649 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.250+0000 7f0101d9b640 1 --2- 172.21.15.100:0/3523469649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e220174d0 0x560e223db130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.250+0000 7f0101d9b640 1 -- 172.21.15.100:0/3523469649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e2200f2f0 msgr2=0x560e223dabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.250+0000 7f010259c640 1 --2- 172.21.15.100:0/3523469649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e2200f2f0 0x560e223dabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.250+0000 7f0101d9b640 1 --2- 172.21.15.100:0/3523469649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e2200f2f0 0x560e223dabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.250+0000 7f0101d9b640 1 -- 172.21.15.100:0/3523469649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e220174d0 msgr2=0x560e223db130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.250+0000 7f0101d9b640 1 --2- 172.21.15.100:0/3523469649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e220174d0 0x560e223db130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.250+0000 7f0101d9b640 1 -- 172.21.15.100:0/3523469649 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e222e69a0 con 0x560e220381b0 2024-07-26T13:41:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.250+0000 7f00fb7fe640 1 --2- 172.21.15.100:0/412995142 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f00f00b3d40 0x7f00f00b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.251+0000 7f0101d9b640 1 --2- 172.21.15.100:0/3523469649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e220381b0 0x560e223da6b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f00ec009390 tx=0x7f00ec0096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2f508fbde70f5b9 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.251+0000 7f010259c640 1 --2- 172.21.15.100:0/412995142 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f00f00b3d40 0x7f00f00b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.251+0000 7f00fb7fe640 1 -- 172.21.15.100:0/412995142 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f00f4104730 con 0x7f01041097b0 2024-07-26T13:41:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.251+0000 7f00f8ff9640 1 -- 172.21.15.100:0/3523469649 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec012da0 con 0x560e220381b0 2024-07-26T13:41:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.251+0000 7f00f8ff9640 1 -- 172.21.15.100:0/3523469649 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00ec018020 con 0x560e220381b0 2024-07-26T13:41:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.251+0000 7f00f8ff9640 1 -- 172.21.15.100:0/3523469649 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec003bc0 con 0x560e220381b0 2024-07-26T13:41:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.252+0000 7f010fbde740 1 -- 172.21.15.100:0/3523469649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e220381b0 msgr2=0x560e223da6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.252+0000 7f010259c640 1 --2- 172.21.15.100:0/412995142 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f00f00b3d40 0x7f00f00b6250 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f00fc005c40 tx=0x7f00fc005bd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.252+0000 7f010fbde740 1 --2- 172.21.15.100:0/3523469649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e220381b0 0x560e223da6b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f00ec009390 tx=0x7f00ec0096a0 comp rx=0 tx=0).stop 2024-07-26T13:41:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.252+0000 7f010fbde740 1 -- 172.21.15.100:0/3523469649 shutdown_connections 2024-07-26T13:41:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.252+0000 7f010fbde740 1 --2- 172.21.15.100:0/3523469649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e220174d0 0x560e223db130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.252+0000 7f010fbde740 1 --2- 172.21.15.100:0/3523469649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e2200f2f0 0x560e223dabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.252+0000 7f010fbde740 1 --2- 172.21.15.100:0/3523469649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e220381b0 0x560e223da6b0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.252+0000 7f010fbde740 1 -- 172.21.15.100:0/3523469649 >> 172.21.15.100:0/3523469649 conn(0x560e2202c300 msgr2=0x560e223d85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.252+0000 7f010fbde740 1 -- 172.21.15.100:0/3523469649 shutdown_connections 2024-07-26T13:41:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.252+0000 7f010fbde740 1 -- 172.21.15.100:0/3523469649 wait complete. 2024-07-26T13:41:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.252+0000 7f010fbde740 1 -- start start 2024-07-26T13:41:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.252+0000 7f010fbde740 1 Processor -- start 2024-07-26T13:41:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.253+0000 7f010fbde740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e2200f2f0 0x560e223e61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.253+0000 7f010fbde740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e220174d0 0x560e223e6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.253+0000 7f010fbde740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e220381b0 0x560e223f2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.253+0000 7f010fbde740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x560e22279ab0 con 0x560e220174d0 2024-07-26T13:41:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.253+0000 7f010fbde740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560e222c7cf0 con 0x560e2200f2f0 2024-07-26T13:41:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.253+0000 7f010fbde740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560e223e8e30 con 0x560e220381b0 2024-07-26T13:41:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.253+0000 7f010fbde740 10 client.0 _ll_register_callbacks cb 0x560e223393e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:41:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.253+0000 7f0101d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e220174d0 0x560e223e6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.253+0000 7f0101d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e220174d0 0x560e223e6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45396/0 (socket says 172.21.15.100:45396) 2024-07-26T13:41:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.253+0000 7f0101d9b640 1 -- 172.21.15.100:0/92311315 learned_addr learned my addr 172.21.15.100:0/92311315 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.253+0000 7f0102d9d640 1 --2- 172.21.15.100:0/92311315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e220381b0 0x560e223f2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.253+0000 7f0101d9b640 1 -- 172.21.15.100:0/92311315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e220381b0 msgr2=0x560e223f2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.253+0000 7f010259c640 1 --2- 172.21.15.100:0/92311315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e2200f2f0 0x560e223e61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.253+0000 7f0101d9b640 1 --2- 172.21.15.100:0/92311315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e220381b0 0x560e223f2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.253+0000 7f0101d9b640 1 -- 172.21.15.100:0/92311315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e2200f2f0 msgr2=0x560e223e61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.253+0000 7f0101d9b640 1 --2- 172.21.15.100:0/92311315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e2200f2f0 0x560e223e61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.253+0000 7f0101d9b640 1 -- 172.21.15.100:0/92311315 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e220a3790 con 0x560e220174d0 2024-07-26T13:41:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.253+0000 7f010259c640 1 --2- 172.21.15.100:0/92311315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e2200f2f0 0x560e223e61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:41:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.253+0000 7f0101d9b640 1 --2- 172.21.15.100:0/92311315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e220174d0 0x560e223e6720 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f00ec014b70 tx=0x7f00ec014ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df8b14dec9e59b40 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.253+0000 7f00e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:41:44.262 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.253+0000 7f00e9ffb640 1 -- 172.21.15.100:0/92311315 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec009980 con 0x560e220174d0 2024-07-26T13:41:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.253+0000 7f00e9ffb640 1 -- 172.21.15.100:0/92311315 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f00ec012da0 con 0x560e220174d0 2024-07-26T13:41:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.253+0000 7f010fbde740 1 -- 172.21.15.100:0/92311315 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00ec018020 con 0x560e220174d0 2024-07-26T13:41:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.254+0000 7f00e9ffb640 1 -- 172.21.15.100:0/92311315 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f00ec02e440 con 0x560e220174d0 2024-07-26T13:41:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.254+0000 7f010fbde740 10 client.15885 fetch_fsmap learned FSMap version 26 2024-07-26T13:41:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.254+0000 7f010fbde740 1 -- 172.21.15.100:0/92311315 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560e2211c620 con 0x560e220174d0 2024-07-26T13:41:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.254+0000 7f00e9ffb640 1 -- 172.21.15.100:0/92311315 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00ec018020 con 0x560e220174d0 2024-07-26T13:41:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.254+0000 7f00e9ffb640 1 -- 172.21.15.100:0/92311315 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f00ec02c020 con 0x560e220174d0 2024-07-26T13:41:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.254+0000 7f010fbde740 10 client.15885 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:41:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.254+0000 7f010fbde740 10 client.15885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:41:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.254+0000 7f010fbde740 20 client.15885 populate_metadata read hostname 'smithi100' 2024-07-26T13:41:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.254+0000 7f010fbde740 1 --2- 172.21.15.100:0/92311315 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560e21ff6d80 0x560e223f8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.254+0000 7f010fbde740 4 client.15885 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:41:44.263 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.254+0000 7f010fbde740 1 -- 172.21.15.100:0/92311315 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e22352300 con 0x560e21ff6d80 2024-07-26T13:41:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.254+0000 7f010259c640 1 --2- 172.21.15.100:0/92311315 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560e21ff6d80 0x560e223f8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.255+0000 7f010259c640 1 --2- 172.21.15.100:0/92311315 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560e21ff6d80 0x560e223f8050 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3375abf797041420 server_cookie=a9a18b4132f73efa in_seq=0 out_seq=0 2024-07-26T13:41:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.255+0000 7f00e9ffb640 10 client.15885 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.256+0000 7f00e9ffb640 1 -- 172.21.15.100:0/92311315 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560e22352300 con 0x560e21ff6d80 2024-07-26T13:41:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.256+0000 7f00e9ffb640 10 client.15885 handle_command_reply: tid=0 2024-07-26T13:41:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.256+0000 7f010fbde740 1 client.15885 shutdown 2024-07-26T13:41:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.256+0000 7f010fbde740 1 -- 172.21.15.100:0/92311315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e220174d0 msgr2=0x560e223e6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.256+0000 7f010fbde740 1 --2- 172.21.15.100:0/92311315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e220174d0 0x560e223e6720 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f00ec014b70 tx=0x7f00ec014ba0 comp rx=0 tx=0).stop 2024-07-26T13:41:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.256+0000 7f010fbde740 1 -- 172.21.15.100:0/92311315 shutdown_connections 2024-07-26T13:41:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.256+0000 7f010fbde740 1 --2- 172.21.15.100:0/92311315 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560e21ff6d80 0x560e223f8050 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.256+0000 7f010fbde740 1 --2- 172.21.15.100:0/92311315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e220381b0 0x560e223f2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.256+0000 7f010fbde740 1 --2- 172.21.15.100:0/92311315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e220174d0 0x560e223e6720 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:44.264 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.256+0000 7f00e9ffb640 0 client.15885 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:44.265 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.256+0000 7f010fbde740 1 --2- 172.21.15.100:0/92311315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e2200f2f0 0x560e223e61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:44.265 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.256+0000 7f010fbde740 1 -- 172.21.15.100:0/92311315 >> 172.21.15.100:0/92311315 conn(0x560e2202c300 msgr2=0x560e223d6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:44.265 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.256+0000 7f010fbde740 1 -- 172.21.15.100:0/92311315 shutdown_connections 2024-07-26T13:41:44.265 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.256+0000 7f010fbde740 1 -- 172.21.15.100:0/92311315 wait complete. 2024-07-26T13:41:44.265 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.256+0000 7f010fbde740 20 client.15885 trim_cache size 0 max 16384 2024-07-26T13:41:44.265 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:41:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.273+0000 7f010fbde740 1 Processor -- start 2024-07-26T13:41:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.273+0000 7f010fbde740 1 -- start start 2024-07-26T13:41:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.273+0000 7f010fbde740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e2200f2f0 0x560e223e5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.273+0000 7f010fbde740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e220174d0 0x560e223e0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.273+0000 7f010fbde740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e220381b0 0x560e223f2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.273+0000 7f010fbde740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f01041b1950 con 0x560e2200f2f0 2024-07-26T13:41:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.273+0000 7f010fbde740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f01041af220 con 0x560e220381b0 2024-07-26T13:41:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.273+0000 7f010fbde740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f01041ad070 con 0x560e220174d0 2024-07-26T13:41:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.273+0000 7f010259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e2200f2f0 0x560e223e5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.273+0000 7f010259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e2200f2f0 0x560e223e5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45406/0 (socket says 172.21.15.100:45406) 2024-07-26T13:41:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.273+0000 7f010259c640 1 -- 172.21.15.100:0/4215570519 learned_addr learned my addr 172.21.15.100:0/4215570519 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.273+0000 7f0102d9d640 1 --2- 172.21.15.100:0/4215570519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e220381b0 0x560e223f2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.273+0000 7f0101d9b640 1 --2- 172.21.15.100:0/4215570519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e220174d0 0x560e223e0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.273+0000 7f010259c640 1 -- 172.21.15.100:0/4215570519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e220174d0 msgr2=0x560e223e0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.273+0000 7f010259c640 1 --2- 172.21.15.100:0/4215570519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e220174d0 0x560e223e0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.273+0000 7f010259c640 1 -- 172.21.15.100:0/4215570519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e220381b0 msgr2=0x560e223f2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.273+0000 7f010259c640 1 --2- 172.21.15.100:0/4215570519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e220381b0 0x560e223f2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.273+0000 7f010259c640 1 -- 172.21.15.100:0/4215570519 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e223dd4c0 con 0x560e2200f2f0 2024-07-26T13:41:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.273+0000 7f0102d9d640 1 --2- 172.21.15.100:0/4215570519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e220381b0 0x560e223f2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:41:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.274+0000 7f010259c640 1 --2- 172.21.15.100:0/4215570519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e2200f2f0 0x560e223e5bb0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x560e223d90e0 tx=0x7f00fc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2977763a1e088899 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.274+0000 7f00ebfff640 1 -- 172.21.15.100:0/4215570519 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00fc028530 con 0x560e2200f2f0 2024-07-26T13:41:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.274+0000 7f00ebfff640 1 -- 172.21.15.100:0/4215570519 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00fc028b80 con 0x560e2200f2f0 2024-07-26T13:41:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.274+0000 7f00ebfff640 1 -- 172.21.15.100:0/4215570519 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00fc01fdc0 con 0x560e2200f2f0 2024-07-26T13:41:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.275+0000 7f010fbde740 1 -- 172.21.15.100:0/4215570519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e2200f2f0 msgr2=0x560e223e5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.275+0000 7f010fbde740 1 --2- 172.21.15.100:0/4215570519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e2200f2f0 0x560e223e5bb0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x560e223d90e0 tx=0x7f00fc014000 comp rx=0 tx=0).stop 2024-07-26T13:41:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.275+0000 7f010fbde740 1 -- 172.21.15.100:0/4215570519 shutdown_connections 2024-07-26T13:41:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.275+0000 7f010fbde740 1 --2- 172.21.15.100:0/4215570519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e220381b0 0x560e223f2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.275+0000 7f010fbde740 1 --2- 172.21.15.100:0/4215570519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e220174d0 0x560e223e0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.275+0000 7f010fbde740 1 --2- 172.21.15.100:0/4215570519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e2200f2f0 0x560e223e5bb0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.275+0000 7f010fbde740 1 -- 172.21.15.100:0/4215570519 >> 172.21.15.100:0/4215570519 conn(0x560e2202c300 msgr2=0x560e223e54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.275+0000 7f010fbde740 1 -- 172.21.15.100:0/4215570519 shutdown_connections 2024-07-26T13:41:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.275+0000 7f010fbde740 1 -- 172.21.15.100:0/4215570519 wait complete. 2024-07-26T13:41:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.275+0000 7f010fbde740 1 -- start start 2024-07-26T13:41:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.275+0000 7f010fbde740 1 Processor -- start 2024-07-26T13:41:44.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.276+0000 7f010fbde740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e2200f2f0 0x560e223f9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:44.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.276+0000 7f010fbde740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e220174d0 0x560e223fa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:44.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.276+0000 7f010fbde740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e220381b0 0x560e22455540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:44.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.276+0000 7f010fbde740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f01041ad6e0 con 0x560e2200f2f0 2024-07-26T13:41:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.276+0000 7f010fbde740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f01041aeb50 con 0x560e220381b0 2024-07-26T13:41:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.276+0000 7f010fbde740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f01041addb0 con 0x560e220174d0 2024-07-26T13:41:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.276+0000 7f010fbde740 10 client.0 _ll_register_callbacks cb 0x7f00ec009cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:41:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.276+0000 7f010259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e2200f2f0 0x560e223f9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.276+0000 7f010259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e2200f2f0 0x560e223f9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45410/0 (socket says 172.21.15.100:45410) 2024-07-26T13:41:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.276+0000 7f010259c640 1 -- 172.21.15.100:0/4151558122 learned_addr learned my addr 172.21.15.100:0/4151558122 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.276+0000 7f0102d9d640 1 --2- 172.21.15.100:0/4151558122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e220381b0 0x560e22455540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.276+0000 7f0101d9b640 1 --2- 172.21.15.100:0/4151558122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e220174d0 0x560e223fa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.276+0000 7f010259c640 1 -- 172.21.15.100:0/4151558122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e220174d0 msgr2=0x560e223fa0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.276+0000 7f010259c640 1 --2- 172.21.15.100:0/4151558122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e220174d0 0x560e223fa0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.276+0000 7f010259c640 1 -- 172.21.15.100:0/4151558122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e220381b0 msgr2=0x560e22455540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.276+0000 7f010259c640 1 --2- 172.21.15.100:0/4151558122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e220381b0 0x560e22455540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.276+0000 7f010259c640 1 -- 172.21.15.100:0/4151558122 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e22455be0 con 0x560e2200f2f0 2024-07-26T13:41:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.276+0000 7f0102d9d640 1 --2- 172.21.15.100:0/4151558122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e220381b0 0x560e22455540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:41:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.276+0000 7f010259c640 1 --2- 172.21.15.100:0/4151558122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e2200f2f0 0x560e223f9b00 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f00fc01f9a0 tx=0x7f00fc0126d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6278f66bdd11ead3 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.276+0000 7f00e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:41:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.276+0000 7f00e9ffb640 1 -- 172.21.15.100:0/4151558122 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00fc01fd10 con 0x560e2200f2f0 2024-07-26T13:41:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.276+0000 7f00e9ffb640 1 -- 172.21.15.100:0/4151558122 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f00fc01f420 con 0x560e2200f2f0 2024-07-26T13:41:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.276+0000 7f010fbde740 1 -- 172.21.15.100:0/4151558122 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00fc028b80 con 0x560e2200f2f0 2024-07-26T13:41:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.277+0000 7f00e9ffb640 1 -- 172.21.15.100:0/4151558122 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f00fc01f620 con 0x560e2200f2f0 2024-07-26T13:41:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.277+0000 7f010fbde740 10 client.15891 fetch_fsmap learned FSMap version 26 2024-07-26T13:41:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.277+0000 7f010fbde740 1 -- 172.21.15.100:0/4151558122 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560e2245f1f0 con 0x560e2200f2f0 2024-07-26T13:41:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.277+0000 7f00e9ffb640 1 -- 172.21.15.100:0/4151558122 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00fc028b80 con 0x560e2200f2f0 2024-07-26T13:41:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.277+0000 7f00e9ffb640 1 -- 172.21.15.100:0/4151558122 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f00fc041070 con 0x560e2200f2f0 2024-07-26T13:41:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.277+0000 7f010fbde740 10 client.15891 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:41:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.277+0000 7f010fbde740 10 client.15891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:41:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.277+0000 7f010fbde740 20 client.15891 populate_metadata read hostname 'smithi100' 2024-07-26T13:41:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.277+0000 7f010fbde740 1 --2- 172.21.15.100:0/4151558122 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560e21fcc1c0 0x560e224619d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.277+0000 7f010fbde740 4 client.15891 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-26T13:41:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.277+0000 7f010fbde740 1 -- 172.21.15.100:0/4151558122 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560e2234dda0 con 0x560e21fcc1c0 2024-07-26T13:41:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.277+0000 7f0101d9b640 1 --2- 172.21.15.100:0/4151558122 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560e21fcc1c0 0x560e224619d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.278+0000 7f00e9ffb640 10 client.15891 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.278+0000 7f0101d9b640 1 --2- 172.21.15.100:0/4151558122 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560e21fcc1c0 0x560e224619d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f7030865f443276 server_cookie=8200fbf315387e23 in_seq=0 out_seq=0 2024-07-26T13:41:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.282+0000 7f00e9ffb640 1 -- 172.21.15.100:0/4151558122 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560e2234dda0 con 0x560e21fcc1c0 2024-07-26T13:41:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.282+0000 7f00e9ffb640 10 client.15891 handle_command_reply: tid=0 2024-07-26T13:41:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.282+0000 7f010fbde740 1 client.15891 shutdown 2024-07-26T13:41:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.282+0000 7f010fbde740 1 -- 172.21.15.100:0/4151558122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e2200f2f0 msgr2=0x560e223f9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.282+0000 7f010fbde740 1 --2- 172.21.15.100:0/4151558122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e2200f2f0 0x560e223f9b00 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f00fc01f9a0 tx=0x7f00fc0126d0 comp rx=0 tx=0).stop 2024-07-26T13:41:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.282+0000 7f010fbde740 1 -- 172.21.15.100:0/4151558122 shutdown_connections 2024-07-26T13:41:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.282+0000 7f010fbde740 1 --2- 172.21.15.100:0/4151558122 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560e21fcc1c0 0x560e224619d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.282+0000 7f010fbde740 1 --2- 172.21.15.100:0/4151558122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e220381b0 0x560e22455540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.282+0000 7f010fbde740 1 --2- 172.21.15.100:0/4151558122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e220174d0 0x560e223fa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.282+0000 7f010fbde740 1 --2- 172.21.15.100:0/4151558122 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e2200f2f0 0x560e223f9b00 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.282+0000 7f010fbde740 1 -- 172.21.15.100:0/4151558122 >> 172.21.15.100:0/4151558122 conn(0x560e2202c300 msgr2=0x560e223e7480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.282+0000 7f00e9ffb640 0 client.15891 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.282+0000 7f010fbde740 1 -- 172.21.15.100:0/4151558122 shutdown_connections 2024-07-26T13:41:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.283+0000 7f010fbde740 1 -- 172.21.15.100:0/4151558122 wait complete. 2024-07-26T13:41:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.283+0000 7f010fbde740 20 client.15891 trim_cache size 0 max 16384 2024-07-26T13:41:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-07-26T13:41:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-07-26T13:41:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-07-26T13:41:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-07-26T13:41:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.283+0000 7f0108ccb640 1 -- 172.21.15.100:0/412995142 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f00f00b3d40 msgr2=0x7f00f00b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.283+0000 7f0108ccb640 1 --2- 172.21.15.100:0/412995142 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f00f00b3d40 0x7f00f00b6250 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f00fc005c40 tx=0x7f00fc005bd0 comp rx=0 tx=0).stop 2024-07-26T13:41:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.283+0000 7f0108ccb640 1 -- 172.21.15.100:0/412995142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01041097b0 msgr2=0x7f010419f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.283+0000 7f0108ccb640 1 --2- 172.21.15.100:0/412995142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01041097b0 0x7f010419f670 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f00f4004770 tx=0x7f00f4004270 comp rx=0 tx=0).stop 2024-07-26T13:41:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.284+0000 7f0108ccb640 1 -- 172.21.15.100:0/412995142 shutdown_connections 2024-07-26T13:41:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.284+0000 7f0108ccb640 1 --2- 172.21.15.100:0/412995142 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f00f00b3d40 0x7f00f00b6250 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:44.290 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.284+0000 7f0108ccb640 1 --2- 172.21.15.100:0/412995142 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f01041097b0 0x7f010419f670 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:44.290 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.284+0000 7f0108ccb640 1 --2- 172.21.15.100:0/412995142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0104108e60 0x7f010419a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:44.290 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.284+0000 7f0108ccb640 1 --2- 172.21.15.100:0/412995142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0104101a90 0x7f0104199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:44.290 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.284+0000 7f0108ccb640 1 -- 172.21.15.100:0/412995142 >> 172.21.15.100:0/412995142 conn(0x7f01040798c0 msgr2=0x7f0104106ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:44.290 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.284+0000 7f0108ccb640 1 -- 172.21.15.100:0/412995142 shutdown_connections 2024-07-26T13:41:44.290 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:44.284+0000 7f0108ccb640 1 -- 172.21.15.100:0/412995142 wait complete. 2024-07-26T13:41:45.874 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-26T13:41:45.875 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-26T13:41:45.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:45 smithi100 ceph-mon[32599]: pgmap v413: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-07-26T13:41:45.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:45 smithi138 ceph-mon[36954]: pgmap v413: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-07-26T13:41:45.979 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-07-26T13:41:46.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:45 smithi167 ceph-mon[37140]: pgmap v413: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-07-26T13:41:46.081 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-07-26T13:41:46.120 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-07-26T13:41:46.121 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-07-26T13:41:46.122 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-07-26T13:41:46.161 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-07-26T13:41:46.164 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-07-26T13:41:46.164 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-07-26T13:41:46.220 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-07-26T13:41:46.222 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-07-26T13:41:46.222 DEBUG:teuthology.orchestra.run.smithi100:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-07-26T13:41:46.286 INFO:tasks.workunit.client.0.smithi100.stderr:+ dbench 1 2024-07-26T13:41:46.298 INFO:tasks.workunit.client.0.smithi100.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-07-26T13:41:46.298 INFO:tasks.workunit.client.0.smithi100.stdout: 2024-07-26T13:41:46.298 INFO:tasks.workunit.client.0.smithi100.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-07-26T13:41:46.303 INFO:tasks.workunit.client.0.smithi100.stdout:0 of 1 processes prepared for launch 0 sec 2024-07-26T13:41:46.332 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:41:46.402 INFO:tasks.workunit.client.0.smithi100.stdout:1 of 1 processes prepared for launch 0 sec 2024-07-26T13:41:46.402 INFO:tasks.workunit.client.0.smithi100.stdout:releasing clients 2024-07-26T13:41:47.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.399+0000 7f9666675640 1 -- 172.21.15.100:0/369424774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96601024b0 msgr2=0x7f96601028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:47.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.399+0000 7f9666675640 1 --2- 172.21.15.100:0/369424774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96601024b0 0x7f96601028c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9650009900 tx=0x7f965002a410 comp rx=0 tx=0).stop 2024-07-26T13:41:47.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.399+0000 7f9666675640 1 -- 172.21.15.100:0/369424774 shutdown_connections 2024-07-26T13:41:47.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.399+0000 7f9666675640 1 --2- 172.21.15.100:0/369424774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9660108740 0x7f966010ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:47.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.399+0000 7f9666675640 1 --2- 172.21.15.100:0/369424774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9660107df0 0x7f9660108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:47.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.399+0000 7f9666675640 1 --2- 172.21.15.100:0/369424774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f96601024b0 0x7f96601028c0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:47.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.399+0000 7f9666675640 1 -- 172.21.15.100:0/369424774 >> 172.21.15.100:0/369424774 conn(0x7f9660079170 msgr2=0x7f9660079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:47.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.399+0000 7f9666675640 1 -- 172.21.15.100:0/369424774 shutdown_connections 2024-07-26T13:41:47.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.399+0000 7f9666675640 1 -- 172.21.15.100:0/369424774 wait complete. 2024-07-26T13:41:47.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.399+0000 7f9666675640 1 Processor -- start 2024-07-26T13:41:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.399+0000 7f9666675640 1 -- start start 2024-07-26T13:41:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.400+0000 7f9666675640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96601024b0 0x7f966019ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.400+0000 7f9666675640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9660107df0 0x7f966019b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.400+0000 7f9666675640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9660108740 0x7f966019b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.400+0000 7f9666675640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f96601123a0 con 0x7f9660107df0 2024-07-26T13:41:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.400+0000 7f9666675640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9660112220 con 0x7f96601024b0 2024-07-26T13:41:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.400+0000 7f9666675640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9660112520 con 0x7f9660108740 2024-07-26T13:41:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.400+0000 7f9657fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9660107df0 0x7f966019b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.400+0000 7f9664c12640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9660108740 0x7f966019b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.400+0000 7f965ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96601024b0 0x7f966019ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.400+0000 7f9657fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9660107df0 0x7f966019b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35350/0 (socket says 172.21.15.100:35350) 2024-07-26T13:41:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.400+0000 7f9657fff640 1 -- 172.21.15.100:0/3354859402 learned_addr learned my addr 172.21.15.100:0/3354859402 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:47.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.400+0000 7f965ffff640 1 -- 172.21.15.100:0/3354859402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9660108740 msgr2=0x7f966019b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:47.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.400+0000 7f965ffff640 1 --2- 172.21.15.100:0/3354859402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9660108740 0x7f966019b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:47.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.400+0000 7f965ffff640 1 -- 172.21.15.100:0/3354859402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9660107df0 msgr2=0x7f966019b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:47.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.400+0000 7f965ffff640 1 --2- 172.21.15.100:0/3354859402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9660107df0 0x7f966019b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:47.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.400+0000 7f965ffff640 1 -- 172.21.15.100:0/3354859402 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96601a6000 con 0x7f96601024b0 2024-07-26T13:41:47.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.400+0000 7f9664c12640 1 --2- 172.21.15.100:0/3354859402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9660108740 0x7f966019b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:41:47.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.400+0000 7f965ffff640 1 --2- 172.21.15.100:0/3354859402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96601024b0 0x7f966019ad70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f965002a930 tx=0x7f9650031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:47.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.401+0000 7f965dffb640 1 -- 172.21.15.100:0/3354859402 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965003f030 con 0x7f96601024b0 2024-07-26T13:41:47.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.401+0000 7f9666675640 1 -- 172.21.15.100:0/3354859402 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96601a6290 con 0x7f96601024b0 2024-07-26T13:41:47.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.401+0000 7f965dffb640 1 -- 172.21.15.100:0/3354859402 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96500043d0 con 0x7f96601024b0 2024-07-26T13:41:47.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.401+0000 7f965dffb640 1 -- 172.21.15.100:0/3354859402 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9650035070 con 0x7f96601024b0 2024-07-26T13:41:47.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.401+0000 7f9666675640 1 -- 172.21.15.100:0/3354859402 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96601a6770 con 0x7f96601024b0 2024-07-26T13:41:47.404 INFO:tasks.workunit.client.0.smithi100.stdout: 1 3859 39.76 MB/sec warmup 1 sec latency 6.163 ms 2024-07-26T13:41:47.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.402+0000 7f9666675640 1 -- 172.21.15.100:0/3354859402 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96601028c0 con 0x7f96601024b0 2024-07-26T13:41:47.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.403+0000 7f965dffb640 1 -- 172.21.15.100:0/3354859402 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f9650042030 con 0x7f96601024b0 2024-07-26T13:41:47.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.404+0000 7f965dffb640 1 --2- 172.21.15.100:0/3354859402 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f964c0b3d70 0x7f964c0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:47.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.404+0000 7f9657fff640 1 --2- 172.21.15.100:0/3354859402 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f964c0b3d70 0x7f964c0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:47.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.404+0000 7f965dffb640 1 -- 172.21.15.100:0/3354859402 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f96501044f0 con 0x7f96601024b0 2024-07-26T13:41:47.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.405+0000 7f9657fff640 1 --2- 172.21.15.100:0/3354859402 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f964c0b3d70 0x7f964c0b6280 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9660100630 tx=0x7f9648005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:47.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.407+0000 7f965dffb640 1 -- 172.21.15.100:0/3354859402 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f96500cb7c0 con 0x7f96601024b0 2024-07-26T13:41:47.569 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:47 smithi100 ceph-mon[32599]: pgmap v414: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T13:41:47.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.639+0000 7f9666675640 1 -- 172.21.15.100:0/3354859402 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f966002e0c0 con 0x7f96601024b0 2024-07-26T13:41:47.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.640+0000 7f965dffb640 1 -- 172.21.15.100:0/3354859402 <== mon.2 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11927 (secure 0 0 0) 0x7f96500cb0c0 con 0x7f96601024b0 2024-07-26T13:41:47.641 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:41:47.641 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":26,"btime":"2024-07-26T13:40:09:530496+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14790,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/313501409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":313501409},{"type":"v1","addr":"172.21.15.100:6839","nonce":313501409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/1402975506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":1402975506},{"type":"v1","addr":"172.21.15.100:6841","nonce":1402975506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-26T13:37:01.281843+0000","modified":"2024-07-26T13:40:09.187125+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24607,"mds_1":24605,"mds_2":24595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24557":{"gid":24557,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/3290105300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3290105300},{"type":"v1","addr":"172.21.15.138:6835","nonce":3290105300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/3393225483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3393225483},{"type":"v1","addr":"172.21.15.138:6837","nonce":3393225483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24595":{"gid":24595,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6837/3828634468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3828634468},{"type":"v1","addr":"172.21.15.167:6837","nonce":3828634468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.138:6839/1580925471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1580925471},{"type":"v1","addr":"172.21.15.138:6839","nonce":1580925471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/398150514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":398150514},{"type":"v1","addr":"172.21.15.167:6839","nonce":398150514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24607,"qdb_cluster":[24595,24605,24607]},"id":1}]} 2024-07-26T13:41:47.642 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 26 2024-07-26T13:41:47.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.642+0000 7f9666675640 1 -- 172.21.15.100:0/3354859402 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f964c0b3d70 msgr2=0x7f964c0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:47.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.642+0000 7f9666675640 1 --2- 172.21.15.100:0/3354859402 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f964c0b3d70 0x7f964c0b6280 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9660100630 tx=0x7f9648005f90 comp rx=0 tx=0).stop 2024-07-26T13:41:47.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.642+0000 7f9666675640 1 -- 172.21.15.100:0/3354859402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96601024b0 msgr2=0x7f966019ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:47.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.642+0000 7f9666675640 1 --2- 172.21.15.100:0/3354859402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96601024b0 0x7f966019ad70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f965002a930 tx=0x7f9650031a20 comp rx=0 tx=0).stop 2024-07-26T13:41:47.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.642+0000 7f9666675640 1 -- 172.21.15.100:0/3354859402 shutdown_connections 2024-07-26T13:41:47.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.642+0000 7f9666675640 1 --2- 172.21.15.100:0/3354859402 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f964c0b3d70 0x7f964c0b6280 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:47.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.642+0000 7f9666675640 1 --2- 172.21.15.100:0/3354859402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9660108740 0x7f966019b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:47.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.642+0000 7f9666675640 1 --2- 172.21.15.100:0/3354859402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9660107df0 0x7f966019b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:47.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.642+0000 7f9666675640 1 --2- 172.21.15.100:0/3354859402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96601024b0 0x7f966019ad70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:47.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.642+0000 7f9666675640 1 -- 172.21.15.100:0/3354859402 >> 172.21.15.100:0/3354859402 conn(0x7f9660079170 msgr2=0x7f9660100ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:47.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.643+0000 7f9666675640 1 -- 172.21.15.100:0/3354859402 shutdown_connections 2024-07-26T13:41:47.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:47.643+0000 7f9666675640 1 -- 172.21.15.100:0/3354859402 wait complete. 2024-07-26T13:41:47.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:47 smithi167 ceph-mon[37140]: pgmap v414: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T13:41:47.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:47 smithi138 ceph-mon[36954]: pgmap v414: 97 pgs: 97 active+clean; 1.4 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T13:41:48.402 INFO:tasks.workunit.client.0.smithi100.stdout: 1 7336 29.44 MB/sec warmup 2 sec latency 13.140 ms 2024-07-26T13:41:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:48 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/3354859402' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:41:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:48 smithi100 ceph-mon[32599]: pgmap v415: 97 pgs: 97 active+clean; 1.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 27 KiB/s wr, 6 op/s 2024-07-26T13:41:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:48 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/3354859402' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:41:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:48 smithi138 ceph-mon[36954]: pgmap v415: 97 pgs: 97 active+clean; 1.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 27 KiB/s wr, 6 op/s 2024-07-26T13:41:48.959 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24607 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0' 2024-07-26T13:41:48.960 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- timeout 105 ceph tell mds.24607 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0 2024-07-26T13:41:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:48 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/3354859402' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:41:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:48 smithi167 ceph-mon[37140]: pgmap v415: 97 pgs: 97 active+clean; 1.7 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 27 KiB/s wr, 6 op/s 2024-07-26T13:41:49.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.170+0000 7f7ee9e5c640 1 -- 172.21.15.100:0/1260578318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ee41035c0 msgr2=0x7f7ee41039d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:49.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.170+0000 7f7ee9e5c640 1 --2- 172.21.15.100:0/1260578318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ee41035c0 0x7f7ee41039d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4009900 tx=0x7f7ed402a410 comp rx=0 tx=0).stop 2024-07-26T13:41:49.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.170+0000 7f7ee9e5c640 1 -- 172.21.15.100:0/1260578318 shutdown_connections 2024-07-26T13:41:49.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.170+0000 7f7ee9e5c640 1 --2- 172.21.15.100:0/1260578318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ee4109850 0x7f7ee4110080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:49.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.170+0000 7f7ee9e5c640 1 --2- 172.21.15.100:0/1260578318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ee4108f00 0x7f7ee4109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:49.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.170+0000 7f7ee9e5c640 1 --2- 172.21.15.100:0/1260578318 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ee41035c0 0x7f7ee41039d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:49.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.170+0000 7f7ee9e5c640 1 -- 172.21.15.100:0/1260578318 >> 172.21.15.100:0/1260578318 conn(0x7f7ee40ff030 msgr2=0x7f7ee4101480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:49.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.170+0000 7f7ee9e5c640 1 -- 172.21.15.100:0/1260578318 shutdown_connections 2024-07-26T13:41:49.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.170+0000 7f7ee9e5c640 1 -- 172.21.15.100:0/1260578318 wait complete. 2024-07-26T13:41:49.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.171+0000 7f7ee9e5c640 1 Processor -- start 2024-07-26T13:41:49.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.171+0000 7f7ee9e5c640 1 -- start start 2024-07-26T13:41:49.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.171+0000 7f7ee9e5c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ee41035c0 0x7f7ee4199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:49.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.171+0000 7f7ee9e5c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ee4108f00 0x7f7ee419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:49.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.171+0000 7f7ee9e5c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ee4109850 0x7f7ee419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:49.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.171+0000 7f7ee9e5c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee4113520 con 0x7f7ee4108f00 2024-07-26T13:41:49.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.171+0000 7f7ee9e5c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee41133a0 con 0x7f7ee41035c0 2024-07-26T13:41:49.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.171+0000 7f7ee9e5c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee41136a0 con 0x7f7ee4109850 2024-07-26T13:41:49.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.171+0000 7f7ee2ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ee4108f00 0x7f7ee419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:49.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.171+0000 7f7ee37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ee41035c0 0x7f7ee4199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:49.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.171+0000 7f7ee2ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ee4108f00 0x7f7ee419a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35370/0 (socket says 172.21.15.100:35370) 2024-07-26T13:41:49.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.171+0000 7f7ee2ffd640 1 -- 172.21.15.100:0/1745484402 learned_addr learned my addr 172.21.15.100:0/1745484402 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:49.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.171+0000 7f7ee3fff640 1 --2- 172.21.15.100:0/1745484402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ee4109850 0x7f7ee419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:49.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.172+0000 7f7ee37fe640 1 -- 172.21.15.100:0/1745484402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ee4109850 msgr2=0x7f7ee419f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:49.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.172+0000 7f7ee37fe640 1 --2- 172.21.15.100:0/1745484402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ee4109850 0x7f7ee419f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:49.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.172+0000 7f7ee37fe640 1 -- 172.21.15.100:0/1745484402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ee4108f00 msgr2=0x7f7ee419a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:49.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.172+0000 7f7ee37fe640 1 --2- 172.21.15.100:0/1745484402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ee4108f00 0x7f7ee419a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:49.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.172+0000 7f7ee37fe640 1 -- 172.21.15.100:0/1745484402 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee419fe60 con 0x7f7ee41035c0 2024-07-26T13:41:49.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.172+0000 7f7ee37fe640 1 --2- 172.21.15.100:0/1745484402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ee41035c0 0x7f7ee4199ef0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7ed402a930 tx=0x7f7ed4031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:49.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.172+0000 7f7ee0ff9640 1 -- 172.21.15.100:0/1745484402 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed403f030 con 0x7f7ee41035c0 2024-07-26T13:41:49.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.172+0000 7f7ee9e5c640 1 -- 172.21.15.100:0/1745484402 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ee41a00f0 con 0x7f7ee41035c0 2024-07-26T13:41:49.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.172+0000 7f7ee0ff9640 1 -- 172.21.15.100:0/1745484402 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ed4004490 con 0x7f7ee41035c0 2024-07-26T13:41:49.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.172+0000 7f7ee0ff9640 1 -- 172.21.15.100:0/1745484402 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed4035070 con 0x7f7ee41035c0 2024-07-26T13:41:49.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.172+0000 7f7ee9e5c640 1 -- 172.21.15.100:0/1745484402 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ee41a68b0 con 0x7f7ee41035c0 2024-07-26T13:41:49.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.175+0000 7f7ef03e0740 1 Processor -- start 2024-07-26T13:41:49.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.175+0000 7f7ef03e0740 1 -- start start 2024-07-26T13:41:49.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.175+0000 7f7ee0ff9640 1 -- 172.21.15.100:0/1745484402 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f7ed4042030 con 0x7f7ee41035c0 2024-07-26T13:41:49.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.175+0000 7f7ef03e0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2d7f86350 0x55d2d832b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:49.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.175+0000 7f7ef03e0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2d7f5d490 0x55d2d832b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:49.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.175+0000 7f7ef03e0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d7f65670 0x55d2d832be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:49.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.175+0000 7f7ef03e0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55d2d82132f0 con 0x55d2d7f86350 2024-07-26T13:41:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.175+0000 7f7ef03e0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d2d82a1a80 con 0x55d2d7f65670 2024-07-26T13:41:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.175+0000 7f7ef03e0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d2d8290f80 con 0x55d2d7f5d490 2024-07-26T13:41:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.175+0000 7f7ee3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2d7f86350 0x55d2d832b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.175+0000 7f7ee37fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2d7f5d490 0x55d2d832b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.175+0000 7f7ee37fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2d7f5d490 0x55d2d832b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:50540/0 (socket says 172.21.15.100:50540) 2024-07-26T13:41:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.175+0000 7f7ee37fe640 1 -- 172.21.15.100:0/593389428 learned_addr learned my addr 172.21.15.100:0/593389428 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.175+0000 7f7ee3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2d7f86350 0x55d2d832b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35378/0 (socket says 172.21.15.100:35378) 2024-07-26T13:41:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.176+0000 7f7ee2ffd640 1 --2- 172.21.15.100:0/593389428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d7f65670 0x55d2d832be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.176+0000 7f7ee37fe640 1 -- 172.21.15.100:0/593389428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d7f65670 msgr2=0x55d2d832be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.176+0000 7f7ee37fe640 1 --2- 172.21.15.100:0/593389428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d7f65670 0x55d2d832be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.176+0000 7f7ee37fe640 1 -- 172.21.15.100:0/593389428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2d7f86350 msgr2=0x55d2d832b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.176+0000 7f7ee37fe640 1 --2- 172.21.15.100:0/593389428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2d7f86350 0x55d2d832b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.176+0000 7f7ee37fe640 1 -- 172.21.15.100:0/593389428 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2d7ff1930 con 0x55d2d7f5d490 2024-07-26T13:41:49.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.176+0000 7f7ee0ff9640 1 --2- 172.21.15.100:0/1745484402 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f7ecc0b3d40 0x7f7ecc0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:49.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.176+0000 7f7ee3fff640 1 --2- 172.21.15.100:0/1745484402 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f7ecc0b3d40 0x7f7ecc0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:49.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.176+0000 7f7ee37fe640 1 --2- 172.21.15.100:0/593389428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2d7f5d490 0x55d2d832b8f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4101940 tx=0x7f7ed410c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a59045c82f6c3ad4 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:49.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.176+0000 7f7ee0ff9640 1 -- 172.21.15.100:0/1745484402 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f7ed4104230 con 0x7f7ee41035c0 2024-07-26T13:41:49.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.176+0000 7f7ec9ffb640 1 -- 172.21.15.100:0/593389428 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed4049070 con 0x55d2d7f5d490 2024-07-26T13:41:49.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.176+0000 7f7ec9ffb640 1 -- 172.21.15.100:0/593389428 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ed40caa20 con 0x55d2d7f5d490 2024-07-26T13:41:49.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.176+0000 7f7ec9ffb640 1 -- 172.21.15.100:0/593389428 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed40c6dd0 con 0x55d2d7f5d490 2024-07-26T13:41:49.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.177+0000 7f7ef03e0740 1 -- 172.21.15.100:0/593389428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2d7f5d490 msgr2=0x55d2d832b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:49.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.177+0000 7f7ee3fff640 1 --2- 172.21.15.100:0/1745484402 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f7ecc0b3d40 0x7f7ecc0b6250 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7edc005fd0 tx=0x7f7edc009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:49.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.177+0000 7f7ef03e0740 1 --2- 172.21.15.100:0/593389428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2d7f5d490 0x55d2d832b8f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4101940 tx=0x7f7ed410c560 comp rx=0 tx=0).stop 2024-07-26T13:41:49.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.177+0000 7f7ef03e0740 1 -- 172.21.15.100:0/593389428 shutdown_connections 2024-07-26T13:41:49.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.177+0000 7f7ef03e0740 1 --2- 172.21.15.100:0/593389428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d7f65670 0x55d2d832be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:49.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.177+0000 7f7ef03e0740 1 --2- 172.21.15.100:0/593389428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2d7f5d490 0x55d2d832b8f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:49.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.177+0000 7f7ef03e0740 1 --2- 172.21.15.100:0/593389428 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2d7f86350 0x55d2d832b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:49.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.177+0000 7f7ef03e0740 1 -- 172.21.15.100:0/593389428 >> 172.21.15.100:0/593389428 conn(0x55d2d7f7a740 msgr2=0x55d2d83292a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:49.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.178+0000 7f7ef03e0740 1 -- 172.21.15.100:0/593389428 shutdown_connections 2024-07-26T13:41:49.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.178+0000 7f7ef03e0740 1 -- 172.21.15.100:0/593389428 wait complete. 2024-07-26T13:41:49.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.178+0000 7f7ef03e0740 1 -- start start 2024-07-26T13:41:49.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.178+0000 7f7ef03e0740 1 Processor -- start 2024-07-26T13:41:49.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.178+0000 7f7ef03e0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d7f5d490 0x55d2d833b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:49.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.178+0000 7f7ef03e0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2d7f65670 0x55d2d833b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:49.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.178+0000 7f7ef03e0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2d7f86350 0x55d2d8346e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:49.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.178+0000 7f7ef03e0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55d2d82334f0 con 0x55d2d7f65670 2024-07-26T13:41:49.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.178+0000 7f7ef03e0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d2d8294fc0 con 0x55d2d7f5d490 2024-07-26T13:41:49.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.178+0000 7f7ef03e0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d2d80f4890 con 0x55d2d7f86350 2024-07-26T13:41:49.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.179+0000 7f7ef03e0740 10 client.0 _ll_register_callbacks cb 0x55d2d823d500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:41:49.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.179+0000 7f7ee37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d7f5d490 0x55d2d833b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:49.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.179+0000 7f7ee2ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2d7f65670 0x55d2d833b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:49.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.179+0000 7f7ee37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d7f5d490 0x55d2d833b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:33386/0 (socket says 172.21.15.100:33386) 2024-07-26T13:41:49.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.179+0000 7f7ee37fe640 1 -- 172.21.15.100:0/3157320996 learned_addr learned my addr 172.21.15.100:0/3157320996 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:49.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.179+0000 7f7ee3fff640 1 --2- 172.21.15.100:0/3157320996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2d7f86350 0x55d2d8346e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:49.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.179+0000 7f7ee37fe640 1 -- 172.21.15.100:0/3157320996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2d7f86350 msgr2=0x55d2d8346e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:49.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.179+0000 7f7ee37fe640 1 --2- 172.21.15.100:0/3157320996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2d7f86350 0x55d2d8346e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:49.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.179+0000 7f7ee37fe640 1 -- 172.21.15.100:0/3157320996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2d7f65670 msgr2=0x55d2d833b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:49.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.179+0000 7f7ee37fe640 1 --2- 172.21.15.100:0/3157320996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2d7f65670 0x55d2d833b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:49.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.179+0000 7f7ee37fe640 1 -- 172.21.15.100:0/3157320996 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2d8242c50 con 0x55d2d7f5d490 2024-07-26T13:41:49.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.179+0000 7f7ee37fe640 1 --2- 172.21.15.100:0/3157320996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d7f5d490 0x55d2d833b2f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4101a50 tx=0x7f7ed402a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf7a9c519771f74e server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:49.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.179+0000 7f7ec2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-07-26T13:41:49.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.179+0000 7f7ec2ffd640 1 -- 172.21.15.100:0/3157320996 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed4049040 con 0x55d2d7f5d490 2024-07-26T13:41:49.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.180+0000 7f7ec2ffd640 1 -- 172.21.15.100:0/3157320996 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f7ed40cb170 con 0x55d2d7f5d490 2024-07-26T13:41:49.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.180+0000 7f7ef03e0740 1 -- 172.21.15.100:0/3157320996 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ed40caa20 con 0x55d2d7f5d490 2024-07-26T13:41:49.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.180+0000 7f7ec2ffd640 1 -- 172.21.15.100:0/3157320996 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7ed40ca680 con 0x55d2d7f5d490 2024-07-26T13:41:49.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.180+0000 7f7ef03e0740 10 client.25352 fetch_fsmap learned FSMap version 26 2024-07-26T13:41:49.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.180+0000 7f7ef03e0740 1 -- 172.21.15.100:0/3157320996 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d2d8349a90 con 0x55d2d7f5d490 2024-07-26T13:41:49.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.180+0000 7f7ec2ffd640 1 -- 172.21.15.100:0/3157320996 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ed40caa20 con 0x55d2d7f5d490 2024-07-26T13:41:49.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.180+0000 7f7ec2ffd640 1 -- 172.21.15.100:0/3157320996 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f7ed40d3a80 con 0x55d2d7f5d490 2024-07-26T13:41:49.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.180+0000 7f7ef03e0740 10 client.25352 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:41:49.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.180+0000 7f7ef03e0740 10 client.25352 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:41:49.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.180+0000 7f7ef03e0740 20 client.25352 populate_metadata read hostname 'smithi100' 2024-07-26T13:41:49.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.180+0000 7f7ef03e0740 1 --2- 172.21.15.100:0/3157320996 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55d2d7f44cd0 0x55d2d834cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:49.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.180+0000 7f7ef03e0740 4 client.25352 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:41:49.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.180+0000 7f7ef03e0740 1 -- 172.21.15.100:0/3157320996 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2d82a39d0 con 0x55d2d7f44cd0 2024-07-26T13:41:49.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.181+0000 7f7ee2ffd640 1 --2- 172.21.15.100:0/3157320996 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55d2d7f44cd0 0x55d2d834cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:49.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.181+0000 7f7ec2ffd640 10 client.25352 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:49.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.181+0000 7f7ee2ffd640 1 --2- 172.21.15.100:0/3157320996 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55d2d7f44cd0 0x55d2d834cdf0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=204fc2e427c9a659 server_cookie=b20ead44faeb28c7 in_seq=0 out_seq=0 2024-07-26T13:41:49.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.182+0000 7f7ec2ffd640 1 -- 172.21.15.100:0/3157320996 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d2d82a39d0 con 0x55d2d7f44cd0 2024-07-26T13:41:49.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.182+0000 7f7ec2ffd640 10 client.25352 handle_command_reply: tid=0 2024-07-26T13:41:49.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.182+0000 7f7ef03e0740 1 client.25352 shutdown 2024-07-26T13:41:49.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.182+0000 7f7ef03e0740 1 -- 172.21.15.100:0/3157320996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d7f5d490 msgr2=0x55d2d833b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:49.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.182+0000 7f7ef03e0740 1 --2- 172.21.15.100:0/3157320996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d7f5d490 0x55d2d833b2f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4101a50 tx=0x7f7ed402a410 comp rx=0 tx=0).stop 2024-07-26T13:41:49.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.182+0000 7f7ef03e0740 1 -- 172.21.15.100:0/3157320996 shutdown_connections 2024-07-26T13:41:49.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.182+0000 7f7ef03e0740 1 --2- 172.21.15.100:0/3157320996 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55d2d7f44cd0 0x55d2d834cdf0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:49.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.182+0000 7f7ef03e0740 1 --2- 172.21.15.100:0/3157320996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2d7f86350 0x55d2d8346e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:49.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.182+0000 7f7ef03e0740 1 --2- 172.21.15.100:0/3157320996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2d7f65670 0x55d2d833b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:49.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.182+0000 7f7ec2ffd640 0 client.25352 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:49.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.182+0000 7f7ef03e0740 1 --2- 172.21.15.100:0/3157320996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d7f5d490 0x55d2d833b2f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:49.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.182+0000 7f7ef03e0740 1 -- 172.21.15.100:0/3157320996 >> 172.21.15.100:0/3157320996 conn(0x55d2d7f7a740 msgr2=0x55d2d832cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:49.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.183+0000 7f7ef03e0740 1 -- 172.21.15.100:0/3157320996 shutdown_connections 2024-07-26T13:41:49.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.183+0000 7f7ef03e0740 1 -- 172.21.15.100:0/3157320996 wait complete. 2024-07-26T13:41:49.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.183+0000 7f7ef03e0740 20 client.25352 trim_cache size 0 max 16384 2024-07-26T13:41:49.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.202+0000 7f7ef03e0740 1 Processor -- start 2024-07-26T13:41:49.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.202+0000 7f7ef03e0740 1 -- start start 2024-07-26T13:41:49.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.202+0000 7f7ef03e0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d7f5d490 0x55d2d8329180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:49.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.202+0000 7f7ef03e0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2d7f65670 0x55d2d832caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:49.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.202+0000 7f7ef03e0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2d7f86350 0x55d2d8346c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:49.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.202+0000 7f7ef03e0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee41af7f0 con 0x55d2d7f86350 2024-07-26T13:41:49.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.202+0000 7f7ef03e0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee41ad0c0 con 0x55d2d7f5d490 2024-07-26T13:41:49.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.202+0000 7f7ef03e0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee41aaf10 con 0x55d2d7f65670 2024-07-26T13:41:49.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.202+0000 7f7ee2ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2d7f65670 0x55d2d832caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:49.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.202+0000 7f7ee37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d7f5d490 0x55d2d8329180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:49.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.202+0000 7f7ee37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d7f5d490 0x55d2d8329180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:33402/0 (socket says 172.21.15.100:33402) 2024-07-26T13:41:49.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.203+0000 7f7ee37fe640 1 -- 172.21.15.100:0/4127659612 learned_addr learned my addr 172.21.15.100:0/4127659612 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:49.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.202+0000 7f7ee2ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2d7f65670 0x55d2d832caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:50560/0 (socket says 172.21.15.100:50560) 2024-07-26T13:41:49.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.203+0000 7f7ee3fff640 1 --2- 172.21.15.100:0/4127659612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2d7f86350 0x55d2d8346c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:49.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.203+0000 7f7ee37fe640 1 -- 172.21.15.100:0/4127659612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2d7f65670 msgr2=0x55d2d832caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:49.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.203+0000 7f7ee37fe640 1 --2- 172.21.15.100:0/4127659612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2d7f65670 0x55d2d832caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:49.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.203+0000 7f7ee37fe640 1 -- 172.21.15.100:0/4127659612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2d7f86350 msgr2=0x55d2d8346c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:49.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.203+0000 7f7ee37fe640 1 --2- 172.21.15.100:0/4127659612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2d7f86350 0x55d2d8346c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:49.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.203+0000 7f7ee37fe640 1 -- 172.21.15.100:0/4127659612 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2d833c3b0 con 0x55d2d7f5d490 2024-07-26T13:41:49.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.203+0000 7f7ee3fff640 1 --2- 172.21.15.100:0/4127659612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2d7f86350 0x55d2d8346c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:41:49.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.203+0000 7f7ee2ffd640 1 --2- 172.21.15.100:0/4127659612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2d7f65670 0x55d2d832caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:41:49.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.203+0000 7f7ee37fe640 1 --2- 172.21.15.100:0/4127659612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d7f5d490 0x55d2d8329180 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7ed40c8180 tx=0x7f7ed403b560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cdc7268e6458f14 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:49.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.203+0000 7f7ec97fa640 1 -- 172.21.15.100:0/4127659612 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed4049070 con 0x55d2d7f5d490 2024-07-26T13:41:49.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.204+0000 7f7ec97fa640 1 -- 172.21.15.100:0/4127659612 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ed40e4020 con 0x55d2d7f5d490 2024-07-26T13:41:49.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.204+0000 7f7ec97fa640 1 -- 172.21.15.100:0/4127659612 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed40ca860 con 0x55d2d7f5d490 2024-07-26T13:41:49.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.205+0000 7f7ef03e0740 1 -- 172.21.15.100:0/4127659612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d7f5d490 msgr2=0x55d2d8329180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:49.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.205+0000 7f7ef03e0740 1 --2- 172.21.15.100:0/4127659612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d7f5d490 0x55d2d8329180 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7ed40c8180 tx=0x7f7ed403b560 comp rx=0 tx=0).stop 2024-07-26T13:41:49.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.205+0000 7f7ef03e0740 1 -- 172.21.15.100:0/4127659612 shutdown_connections 2024-07-26T13:41:49.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.205+0000 7f7ef03e0740 1 --2- 172.21.15.100:0/4127659612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2d7f86350 0x55d2d8346c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:49.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.205+0000 7f7ef03e0740 1 --2- 172.21.15.100:0/4127659612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2d7f65670 0x55d2d832caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:49.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.205+0000 7f7ef03e0740 1 --2- 172.21.15.100:0/4127659612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d7f5d490 0x55d2d8329180 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:49.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.205+0000 7f7ef03e0740 1 -- 172.21.15.100:0/4127659612 >> 172.21.15.100:0/4127659612 conn(0x55d2d7f7a740 msgr2=0x55d2d832c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:49.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.205+0000 7f7ef03e0740 1 -- 172.21.15.100:0/4127659612 shutdown_connections 2024-07-26T13:41:49.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.205+0000 7f7ef03e0740 1 -- 172.21.15.100:0/4127659612 wait complete. 2024-07-26T13:41:49.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.206+0000 7f7ef03e0740 1 -- start start 2024-07-26T13:41:49.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.206+0000 7f7ef03e0740 1 Processor -- start 2024-07-26T13:41:49.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.206+0000 7f7ef03e0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d7f5d490 0x55d2d833f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:49.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.206+0000 7f7ef03e0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2d7f65670 0x55d2d833fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:49.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.206+0000 7f7ef03e0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2d7f86350 0x55d2d83ae4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:49.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.206+0000 7f7ee37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d7f5d490 0x55d2d833f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:49.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.206+0000 7f7ee2ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2d7f65670 0x55d2d833fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:49.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.206+0000 7f7ee2ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2d7f65670 0x55d2d833fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:50562/0 (socket says 172.21.15.100:50562) 2024-07-26T13:41:49.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.206+0000 7f7ee37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d7f5d490 0x55d2d833f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:33406/0 (socket says 172.21.15.100:33406) 2024-07-26T13:41:49.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.207+0000 7f7ee3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2d7f86350 0x55d2d83ae4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:49.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.207+0000 7f7ee3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2d7f86350 0x55d2d83ae4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35402/0 (socket says 172.21.15.100:35402) 2024-07-26T13:41:49.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.207+0000 7f7ee37fe640 1 -- 172.21.15.100:0/2033668489 learned_addr learned my addr 172.21.15.100:0/2033668489 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:49.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.207+0000 7f7ef03e0740 1 -- 172.21.15.100:0/2033668489 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee41ab580 con 0x55d2d7f86350 2024-07-26T13:41:49.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.207+0000 7f7ef03e0740 1 -- 172.21.15.100:0/2033668489 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee41ac9f0 con 0x55d2d7f5d490 2024-07-26T13:41:49.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.207+0000 7f7ef03e0740 1 -- 172.21.15.100:0/2033668489 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee41abc50 con 0x55d2d7f65670 2024-07-26T13:41:49.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.207+0000 7f7ee2ffd640 1 -- 172.21.15.100:0/2033668489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d7f5d490 msgr2=0x55d2d833f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:49.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.207+0000 7f7ee2ffd640 1 --2- 172.21.15.100:0/2033668489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d7f5d490 0x55d2d833f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:49.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.207+0000 7f7ee2ffd640 1 -- 172.21.15.100:0/2033668489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2d7f86350 msgr2=0x55d2d83ae4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:49.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.207+0000 7f7ee2ffd640 1 --2- 172.21.15.100:0/2033668489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2d7f86350 0x55d2d83ae4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:49.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.207+0000 7f7ee2ffd640 1 -- 172.21.15.100:0/2033668489 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2d83aeb10 con 0x55d2d7f65670 2024-07-26T13:41:49.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.207+0000 7f7ee37fe640 1 --2- 172.21.15.100:0/2033668489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d7f5d490 0x55d2d833f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:41:49.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.207+0000 7f7ee3fff640 1 --2- 172.21.15.100:0/2033668489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2d7f86350 0x55d2d83ae4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:41:49.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.207+0000 7f7ef03e0740 10 client.0 _ll_register_callbacks cb 0x7f7ed410c320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:41:49.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.207+0000 7f7ef03e0740 1 -- 172.21.15.100:0/2033668489 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ed40e4020 con 0x55d2d7f65670 2024-07-26T13:41:49.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.207+0000 7f7ee2ffd640 1 --2- 172.21.15.100:0/2033668489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2d7f65670 0x55d2d833fc00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x55d2d834d3d0 tx=0x7f7ed8010950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac25ecc92b240adf server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:49.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.207+0000 7f7ec2ffd640 10 client.25471 ms_handle_connect on v2:172.21.15.167:3300/0 2024-07-26T13:41:49.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.208+0000 7f7ec2ffd640 1 -- 172.21.15.100:0/2033668489 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed802a020 con 0x55d2d7f65670 2024-07-26T13:41:49.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.208+0000 7f7ec2ffd640 1 -- 172.21.15.100:0/2033668489 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f7ed8005220 con 0x55d2d7f65670 2024-07-26T13:41:49.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.208+0000 7f7ec2ffd640 1 -- 172.21.15.100:0/2033668489 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7ed8005420 con 0x55d2d7f65670 2024-07-26T13:41:49.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.208+0000 7f7ef03e0740 10 client.25471 fetch_fsmap learned FSMap version 26 2024-07-26T13:41:49.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.208+0000 7f7ef03e0740 1 -- 172.21.15.100:0/2033668489 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55d2d83b5e10 con 0x55d2d7f65670 2024-07-26T13:41:49.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.208+0000 7f7ec2ffd640 1 -- 172.21.15.100:0/2033668489 <== mon.1 v2:172.21.15.167:3300/0 4 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f7ed8025070 con 0x55d2d7f65670 2024-07-26T13:41:49.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.208+0000 7f7ef03e0740 10 client.25471 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:41:49.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.208+0000 7f7ef03e0740 10 client.25471 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:41:49.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.208+0000 7f7ef03e0740 20 client.25471 populate_metadata read hostname 'smithi100' 2024-07-26T13:41:49.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.208+0000 7f7ef03e0740 1 --2- 172.21.15.100:0/2033668489 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55d2d7f1a080 0x55d2d83b84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:49.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.208+0000 7f7ef03e0740 4 client.25471 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-26T13:41:49.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.208+0000 7f7ef03e0740 1 -- 172.21.15.100:0/2033668489 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d2d829f470 con 0x55d2d7f1a080 2024-07-26T13:41:49.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.209+0000 7f7ee37fe640 1 --2- 172.21.15.100:0/2033668489 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55d2d7f1a080 0x55d2d83b84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:49.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.209+0000 7f7ec2ffd640 10 client.25471 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:49.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.209+0000 7f7ee37fe640 1 --2- 172.21.15.100:0/2033668489 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55d2d7f1a080 0x55d2d83b84a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d977222c52f015eb server_cookie=17292304bed007db in_seq=0 out_seq=0 2024-07-26T13:41:49.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.210+0000 7f7ec2ffd640 1 -- 172.21.15.100:0/2033668489 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d2d829f470 con 0x55d2d7f1a080 2024-07-26T13:41:49.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.211+0000 7f7ec2ffd640 10 client.25471 handle_command_reply: tid=0 2024-07-26T13:41:49.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.211+0000 7f7ef03e0740 1 client.25471 shutdown 2024-07-26T13:41:49.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.212+0000 7f7ef03e0740 1 -- 172.21.15.100:0/2033668489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2d7f65670 msgr2=0x55d2d833fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:49.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.212+0000 7f7ef03e0740 1 --2- 172.21.15.100:0/2033668489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2d7f65670 0x55d2d833fc00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x55d2d834d3d0 tx=0x7f7ed8010950 comp rx=0 tx=0).stop 2024-07-26T13:41:49.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.212+0000 7f7ef03e0740 1 -- 172.21.15.100:0/2033668489 shutdown_connections 2024-07-26T13:41:49.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.212+0000 7f7ef03e0740 1 --2- 172.21.15.100:0/2033668489 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55d2d7f1a080 0x55d2d83b84a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:49.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.212+0000 7f7ef03e0740 1 --2- 172.21.15.100:0/2033668489 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2d7f86350 0x55d2d83ae4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:49.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.212+0000 7f7ef03e0740 1 --2- 172.21.15.100:0/2033668489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2d7f65670 0x55d2d833fc00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:49.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.212+0000 7f7ec2ffd640 0 client.25471 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:49.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.212+0000 7f7ef03e0740 1 --2- 172.21.15.100:0/2033668489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2d7f5d490 0x55d2d833f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:49.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.212+0000 7f7ef03e0740 1 -- 172.21.15.100:0/2033668489 >> 172.21.15.100:0/2033668489 conn(0x55d2d7f7a740 msgr2=0x55d2d8328050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:49.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.212+0000 7f7ef03e0740 1 -- 172.21.15.100:0/2033668489 shutdown_connections 2024-07-26T13:41:49.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.212+0000 7f7ef03e0740 1 -- 172.21.15.100:0/2033668489 wait complete. 2024-07-26T13:41:49.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.212+0000 7f7ef03e0740 20 client.25471 trim_cache size 0 max 16384 2024-07-26T13:41:49.215 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:41:49.216 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-07-26T13:41:49.216 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "08350ab4-89a9-444b-add3-1f68c54b31d6", 2024-07-26T13:41:49.216 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-07-26T13:41:49.216 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:41:49.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.212+0000 7f7ee9e5c640 1 -- 172.21.15.100:0/1745484402 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f7ecc0b3d40 msgr2=0x7f7ecc0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:49.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.213+0000 7f7ee9e5c640 1 --2- 172.21.15.100:0/1745484402 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f7ecc0b3d40 0x7f7ecc0b6250 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7edc005fd0 tx=0x7f7edc009500 comp rx=0 tx=0).stop 2024-07-26T13:41:49.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.213+0000 7f7ee9e5c640 1 -- 172.21.15.100:0/1745484402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ee41035c0 msgr2=0x7f7ee4199ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:49.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.213+0000 7f7ee9e5c640 1 --2- 172.21.15.100:0/1745484402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ee41035c0 0x7f7ee4199ef0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7ed402a930 tx=0x7f7ed4031a20 comp rx=0 tx=0).stop 2024-07-26T13:41:49.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.213+0000 7f7ee9e5c640 1 -- 172.21.15.100:0/1745484402 shutdown_connections 2024-07-26T13:41:49.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.213+0000 7f7ee9e5c640 1 --2- 172.21.15.100:0/1745484402 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f7ecc0b3d40 0x7f7ecc0b6250 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:49.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.213+0000 7f7ee9e5c640 1 --2- 172.21.15.100:0/1745484402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ee4109850 0x7f7ee419f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:49.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.213+0000 7f7ee9e5c640 1 --2- 172.21.15.100:0/1745484402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ee4108f00 0x7f7ee419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:49.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.213+0000 7f7ee9e5c640 1 --2- 172.21.15.100:0/1745484402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ee41035c0 0x7f7ee4199ef0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:49.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.213+0000 7f7ee9e5c640 1 -- 172.21.15.100:0/1745484402 >> 172.21.15.100:0/1745484402 conn(0x7f7ee40ff030 msgr2=0x7f7ee4101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:49.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.213+0000 7f7ee9e5c640 1 -- 172.21.15.100:0/1745484402 shutdown_connections 2024-07-26T13:41:49.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:49.213+0000 7f7ee9e5c640 1 -- 172.21.15.100:0/1745484402 wait complete. 2024-07-26T13:41:49.402 INFO:tasks.workunit.client.0.smithi100.stdout: 1 10629 25.54 MB/sec warmup 3 sec latency 59.200 ms 2024-07-26T13:41:49.972 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:49 smithi100 ceph-mon[32599]: scrub queued for path: / 2024-07-26T13:41:49.973 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:49 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:41:49.973 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:49 smithi100 ceph-mon[32599]: scrub summary: active paths [/] 2024-07-26T13:41:49.973 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:49 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:41:49.973 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:49 smithi100 ceph-mon[32599]: scrub summary: idle 2024-07-26T13:41:50.000 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-26T13:41:50.116 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:41:50.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:49 smithi138 ceph-mon[36954]: scrub queued for path: / 2024-07-26T13:41:50.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:49 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:41:50.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:49 smithi138 ceph-mon[36954]: scrub summary: active paths [/] 2024-07-26T13:41:50.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:49 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:41:50.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:49 smithi138 ceph-mon[36954]: scrub summary: idle 2024-07-26T13:41:50.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:49 smithi167 ceph-mon[37140]: scrub queued for path: / 2024-07-26T13:41:50.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:49 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:41:50.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:49 smithi167 ceph-mon[37140]: scrub summary: active paths [/] 2024-07-26T13:41:50.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:49 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:41:50.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:49 smithi167 ceph-mon[37140]: scrub summary: idle 2024-07-26T13:41:50.402 INFO:tasks.workunit.client.0.smithi100.stdout: 1 13530 22.39 MB/sec warmup 4 sec latency 24.202 ms 2024-07-26T13:41:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:50 smithi138 ceph-mon[36954]: pgmap v416: 97 pgs: 97 active+clean; 4.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 329 KiB/s wr, 48 op/s 2024-07-26T13:41:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:50 smithi100 ceph-mon[32599]: pgmap v416: 97 pgs: 97 active+clean; 4.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 329 KiB/s wr, 48 op/s 2024-07-26T13:41:51.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:50 smithi167 ceph-mon[37140]: pgmap v416: 97 pgs: 97 active+clean; 4.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 329 KiB/s wr, 48 op/s 2024-07-26T13:41:51.402 INFO:tasks.workunit.client.0.smithi100.stdout: 1 16173 20.59 MB/sec warmup 5 sec latency 23.039 ms 2024-07-26T13:41:51.438 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:41:52.403 INFO:tasks.workunit.client.0.smithi100.stdout: 1 19403 20.30 MB/sec warmup 6 sec latency 12.649 ms 2024-07-26T13:41:52.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.608+0000 7fea4a3cb640 1 -- 172.21.15.100:0/147618735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea44107df0 msgr2=0x7fea44108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:52.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.608+0000 7fea4a3cb640 1 --2- 172.21.15.100:0/147618735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea44107df0 0x7fea44108200 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fea38009900 tx=0x7fea3802a410 comp rx=0 tx=0).stop 2024-07-26T13:41:52.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.608+0000 7fea4a3cb640 1 -- 172.21.15.100:0/147618735 shutdown_connections 2024-07-26T13:41:52.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.608+0000 7fea4a3cb640 1 --2- 172.21.15.100:0/147618735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea44108740 0x7fea4410ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:52.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.608+0000 7fea4a3cb640 1 --2- 172.21.15.100:0/147618735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea44107df0 0x7fea44108200 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:52.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.608+0000 7fea4a3cb640 1 --2- 172.21.15.100:0/147618735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea441024b0 0x7fea441028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:52.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.608+0000 7fea4a3cb640 1 -- 172.21.15.100:0/147618735 >> 172.21.15.100:0/147618735 conn(0x7fea440792f0 msgr2=0x7fea44079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:52.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.608+0000 7fea4a3cb640 1 -- 172.21.15.100:0/147618735 shutdown_connections 2024-07-26T13:41:52.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.608+0000 7fea4a3cb640 1 -- 172.21.15.100:0/147618735 wait complete. 2024-07-26T13:41:52.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.608+0000 7fea4a3cb640 1 Processor -- start 2024-07-26T13:41:52.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.608+0000 7fea4a3cb640 1 -- start start 2024-07-26T13:41:52.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.609+0000 7fea4a3cb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea441024b0 0x7fea44199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:52.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.609+0000 7fea4a3cb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea44107df0 0x7fea4419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:52.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.609+0000 7fea4a3cb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea44108740 0x7fea4419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:52.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.609+0000 7fea4a3cb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fea441123b0 con 0x7fea441024b0 2024-07-26T13:41:52.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.609+0000 7fea4a3cb640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fea44112230 con 0x7fea44108740 2024-07-26T13:41:52.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.609+0000 7fea4a3cb640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fea44112530 con 0x7fea44107df0 2024-07-26T13:41:52.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.609+0000 7fea43fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea441024b0 0x7fea44199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:52.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.609+0000 7fea43fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea441024b0 0x7fea44199f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35416/0 (socket says 172.21.15.100:35416) 2024-07-26T13:41:52.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.609+0000 7fea43fff640 1 -- 172.21.15.100:0/2814250667 learned_addr learned my addr 172.21.15.100:0/2814250667 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:52.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.609+0000 7fea437fe640 1 --2- 172.21.15.100:0/2814250667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea44107df0 0x7fea4419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:52.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.609+0000 7fea48968640 1 --2- 172.21.15.100:0/2814250667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea44108740 0x7fea4419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:52.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.609+0000 7fea43fff640 1 -- 172.21.15.100:0/2814250667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea44107df0 msgr2=0x7fea4419a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:52.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.609+0000 7fea43fff640 1 --2- 172.21.15.100:0/2814250667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea44107df0 0x7fea4419a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:52.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.609+0000 7fea43fff640 1 -- 172.21.15.100:0/2814250667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea44108740 msgr2=0x7fea4419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:52.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.609+0000 7fea43fff640 1 --2- 172.21.15.100:0/2814250667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea44108740 0x7fea4419f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:52.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.609+0000 7fea43fff640 1 -- 172.21.15.100:0/2814250667 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea4419fe80 con 0x7fea441024b0 2024-07-26T13:41:52.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.609+0000 7fea48968640 1 --2- 172.21.15.100:0/2814250667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea44108740 0x7fea4419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:41:52.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.610+0000 7fea43fff640 1 --2- 172.21.15.100:0/2814250667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea441024b0 0x7fea44199f10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fea3400ea60 tx=0x7fea3400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:52.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.610+0000 7fea417fa640 1 -- 172.21.15.100:0/2814250667 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3400cde0 con 0x7fea441024b0 2024-07-26T13:41:52.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.610+0000 7fea4a3cb640 1 -- 172.21.15.100:0/2814250667 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea441a0170 con 0x7fea441024b0 2024-07-26T13:41:52.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.610+0000 7fea4a3cb640 1 -- 172.21.15.100:0/2814250667 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea441a68d0 con 0x7fea441024b0 2024-07-26T13:41:52.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.610+0000 7fea417fa640 1 -- 172.21.15.100:0/2814250667 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea340035f0 con 0x7fea441024b0 2024-07-26T13:41:52.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.610+0000 7fea417fa640 1 -- 172.21.15.100:0/2814250667 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea34018690 con 0x7fea441024b0 2024-07-26T13:41:52.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.613+0000 7fea5094f740 1 Processor -- start 2024-07-26T13:41:52.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.613+0000 7fea5094f740 1 -- start start 2024-07-26T13:41:52.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.613+0000 7fea5094f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c08a80850 0x561c08e26510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:52.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.613+0000 7fea5094f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c08a57990 0x561c08e26a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:52.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.613+0000 7fea5094f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c08a5fb70 0x561c08e26f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:52.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.613+0000 7fea5094f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x561c08d9b690 con 0x561c08a80850 2024-07-26T13:41:52.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.613+0000 7fea5094f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561c08d8ab90 con 0x561c08a5fb70 2024-07-26T13:41:52.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.613+0000 7fea5094f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561c08d8ebd0 con 0x561c08a57990 2024-07-26T13:41:52.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.613+0000 7fea417fa640 1 -- 172.21.15.100:0/2814250667 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fea34004020 con 0x7fea441024b0 2024-07-26T13:41:52.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.613+0000 7fea48968640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c08a80850 0x561c08e26510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:52.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.613+0000 7fea48968640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c08a80850 0x561c08e26510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35428/0 (socket says 172.21.15.100:35428) 2024-07-26T13:41:52.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.613+0000 7fea48968640 1 -- 172.21.15.100:0/1681552227 learned_addr learned my addr 172.21.15.100:0/1681552227 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:52.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.613+0000 7fea43fff640 1 --2- 172.21.15.100:0/1681552227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c08a57990 0x561c08e26a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:52.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.613+0000 7fea437fe640 1 --2- 172.21.15.100:0/1681552227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c08a5fb70 0x561c08e26f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:52.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.614+0000 7fea48968640 1 -- 172.21.15.100:0/1681552227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c08a57990 msgr2=0x561c08e26a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:52.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.614+0000 7fea48968640 1 --2- 172.21.15.100:0/1681552227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c08a57990 0x561c08e26a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:52.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.614+0000 7fea48968640 1 -- 172.21.15.100:0/1681552227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c08a5fb70 msgr2=0x561c08e26f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:52.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.614+0000 7fea48968640 1 --2- 172.21.15.100:0/1681552227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c08a5fb70 0x561c08e26f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:52.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.614+0000 7fea48968640 1 -- 172.21.15.100:0/1681552227 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c08d16a10 con 0x561c08a80850 2024-07-26T13:41:52.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.614+0000 7fea48968640 1 --2- 172.21.15.100:0/1681552227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c08a80850 0x561c08e26510 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fea3c010480 tx=0x7fea3c010940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf9eaf51979fa3a server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:52.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.614+0000 7fea2a7fc640 1 -- 172.21.15.100:0/1681552227 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c017070 con 0x561c08a80850 2024-07-26T13:41:52.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.614+0000 7fea2a7fc640 1 -- 172.21.15.100:0/1681552227 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea3c011c10 con 0x561c08a80850 2024-07-26T13:41:52.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.614+0000 7fea2a7fc640 1 -- 172.21.15.100:0/1681552227 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c00edc0 con 0x561c08a80850 2024-07-26T13:41:52.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.614+0000 7fea417fa640 1 --2- 172.21.15.100:0/2814250667 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fea2c0b3d40 0x7fea2c0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:52.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.615+0000 7fea43fff640 1 --2- 172.21.15.100:0/2814250667 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fea2c0b3d40 0x7fea2c0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:52.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.615+0000 7fea417fa640 1 -- 172.21.15.100:0/2814250667 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fea340e4250 con 0x7fea441024b0 2024-07-26T13:41:52.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.615+0000 7fea5094f740 1 -- 172.21.15.100:0/1681552227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c08a80850 msgr2=0x561c08e26510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:52.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.615+0000 7fea43fff640 1 --2- 172.21.15.100:0/2814250667 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fea2c0b3d40 0x7fea2c0b6250 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x561c08e25190 tx=0x7fea34092040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:52.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.615+0000 7fea5094f740 1 --2- 172.21.15.100:0/1681552227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c08a80850 0x561c08e26510 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fea3c010480 tx=0x7fea3c010940 comp rx=0 tx=0).stop 2024-07-26T13:41:52.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.615+0000 7fea5094f740 1 -- 172.21.15.100:0/1681552227 shutdown_connections 2024-07-26T13:41:52.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.615+0000 7fea5094f740 1 --2- 172.21.15.100:0/1681552227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c08a5fb70 0x561c08e26f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:52.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.615+0000 7fea5094f740 1 --2- 172.21.15.100:0/1681552227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c08a57990 0x561c08e26a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:52.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.615+0000 7fea5094f740 1 --2- 172.21.15.100:0/1681552227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c08a80850 0x561c08e26510 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:52.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.615+0000 7fea5094f740 1 -- 172.21.15.100:0/1681552227 >> 172.21.15.100:0/1681552227 conn(0x561c08a74c40 msgr2=0x561c08e24400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:52.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.615+0000 7fea5094f740 1 -- 172.21.15.100:0/1681552227 shutdown_connections 2024-07-26T13:41:52.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.615+0000 7fea5094f740 1 -- 172.21.15.100:0/1681552227 wait complete. 2024-07-26T13:41:52.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.616+0000 7fea5094f740 1 -- start start 2024-07-26T13:41:52.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.616+0000 7fea5094f740 1 Processor -- start 2024-07-26T13:41:52.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.616+0000 7fea5094f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c08a57990 0x561c08e31b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:52.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.616+0000 7fea5094f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c08a5fb70 0x561c08e320c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:52.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.616+0000 7fea5094f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c08a80850 0x561c08e3dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:52.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.616+0000 7fea5094f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x561c08bee7b0 con 0x561c08a5fb70 2024-07-26T13:41:52.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.616+0000 7fea5094f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561c08d2d210 con 0x561c08a80850 2024-07-26T13:41:52.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.616+0000 7fea5094f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561c08e34c60 con 0x561c08a57990 2024-07-26T13:41:52.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.616+0000 7fea5094f740 10 client.0 _ll_register_callbacks cb 0x561c08d0e2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:41:52.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.617+0000 7fea48968640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c08a57990 0x561c08e31b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:52.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.617+0000 7fea437fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c08a5fb70 0x561c08e320c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:52.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.617+0000 7fea48968640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c08a57990 0x561c08e31b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:50602/0 (socket says 172.21.15.100:50602) 2024-07-26T13:41:52.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.617+0000 7fea48968640 1 -- 172.21.15.100:0/3777209008 learned_addr learned my addr 172.21.15.100:0/3777209008 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:52.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.617+0000 7fea48968640 1 --2- 172.21.15.100:0/3777209008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c08a80850 0x561c08e3dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:52.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.617+0000 7fea437fe640 1 -- 172.21.15.100:0/3777209008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c08a57990 msgr2=0x561c08e31b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:52.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.617+0000 7fea437fe640 1 --2- 172.21.15.100:0/3777209008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c08a57990 0x561c08e31b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:52.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.617+0000 7fea437fe640 1 -- 172.21.15.100:0/3777209008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c08a80850 msgr2=0x561c08e3dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:52.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.617+0000 7fea437fe640 1 --2- 172.21.15.100:0/3777209008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c08a80850 0x561c08e3dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:52.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.617+0000 7fea437fe640 1 -- 172.21.15.100:0/3777209008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c08d3e710 con 0x561c08a5fb70 2024-07-26T13:41:52.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.617+0000 7fea48968640 1 --2- 172.21.15.100:0/3777209008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c08a57990 0x561c08e31b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:41:52.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.617+0000 7fea437fe640 1 --2- 172.21.15.100:0/3777209008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c08a5fb70 0x561c08e320c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fea3802a930 tx=0x7fea38032e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ba65e456082719 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:52.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.617+0000 7fea237fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:41:52.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.617+0000 7fea237fe640 1 -- 172.21.15.100:0/3777209008 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38035040 con 0x561c08a5fb70 2024-07-26T13:41:52.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.617+0000 7fea237fe640 1 -- 172.21.15.100:0/3777209008 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fea38031760 con 0x561c08a5fb70 2024-07-26T13:41:52.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.617+0000 7fea5094f740 1 -- 172.21.15.100:0/3777209008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561c08d9d5e0 con 0x561c08a5fb70 2024-07-26T13:41:52.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.617+0000 7fea237fe640 1 -- 172.21.15.100:0/3777209008 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fea38004040 con 0x561c08a5fb70 2024-07-26T13:41:52.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.617+0000 7fea5094f740 10 client.15927 fetch_fsmap learned FSMap version 26 2024-07-26T13:41:52.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.617+0000 7fea5094f740 1 -- 172.21.15.100:0/3777209008 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561c08afaba0 con 0x561c08a5fb70 2024-07-26T13:41:52.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.618+0000 7fea237fe640 1 -- 172.21.15.100:0/3777209008 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561c08d9d5e0 con 0x561c08a5fb70 2024-07-26T13:41:52.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.618+0000 7fea237fe640 1 -- 172.21.15.100:0/3777209008 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fea38004750 con 0x561c08a5fb70 2024-07-26T13:41:52.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.618+0000 7fea5094f740 10 client.15927 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:41:52.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.618+0000 7fea5094f740 10 client.15927 resolve_mds: validated gid 24607 aka daemon mds.i 2024-07-26T13:41:52.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.618+0000 7fea5094f740 20 client.15927 populate_metadata read hostname 'smithi100' 2024-07-26T13:41:52.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.618+0000 7fea5094f740 1 --2- 172.21.15.100:0/3777209008 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x561c08a3f1d0 0x561c08e43d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:52.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.618+0000 7fea5094f740 4 client.15927 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:41:52.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.618+0000 7fea5094f740 1 -- 172.21.15.100:0/3777209008 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c08d99080 con 0x561c08a3f1d0 2024-07-26T13:41:52.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.618+0000 7fea48968640 1 --2- 172.21.15.100:0/3777209008 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x561c08a3f1d0 0x561c08e43d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:52.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.618+0000 7fea237fe640 10 client.15927 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:52.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.619+0000 7fea48968640 1 --2- 172.21.15.100:0/3777209008 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x561c08a3f1d0 0x561c08e43d90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aded1eeb1d5ff086 server_cookie=2fd4f802ce79ee3b in_seq=0 out_seq=0 2024-07-26T13:41:52.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.621+0000 7fea237fe640 1 -- 172.21.15.100:0/3777209008 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561c08d99080 con 0x561c08a3f1d0 2024-07-26T13:41:52.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.621+0000 7fea237fe640 10 client.15927 handle_command_reply: tid=0 2024-07-26T13:41:52.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.621+0000 7fea5094f740 1 client.15927 shutdown 2024-07-26T13:41:52.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.621+0000 7fea5094f740 1 -- 172.21.15.100:0/3777209008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c08a5fb70 msgr2=0x561c08e320c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:52.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.621+0000 7fea5094f740 1 --2- 172.21.15.100:0/3777209008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c08a5fb70 0x561c08e320c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fea3802a930 tx=0x7fea38032e90 comp rx=0 tx=0).stop 2024-07-26T13:41:52.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.621+0000 7fea5094f740 1 -- 172.21.15.100:0/3777209008 shutdown_connections 2024-07-26T13:41:52.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.621+0000 7fea5094f740 1 --2- 172.21.15.100:0/3777209008 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x561c08a3f1d0 0x561c08e43d90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:52.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.621+0000 7fea5094f740 1 --2- 172.21.15.100:0/3777209008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c08a80850 0x561c08e3dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:52.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.621+0000 7fea5094f740 1 --2- 172.21.15.100:0/3777209008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c08a5fb70 0x561c08e320c0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:52.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.621+0000 7fea5094f740 1 --2- 172.21.15.100:0/3777209008 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c08a57990 0x561c08e31b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:52.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.621+0000 7fea5094f740 1 -- 172.21.15.100:0/3777209008 >> 172.21.15.100:0/3777209008 conn(0x561c08a74c40 msgr2=0x561c08e296a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:52.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.621+0000 7fea237fe640 0 client.15927 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:52.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.621+0000 7fea5094f740 1 -- 172.21.15.100:0/3777209008 shutdown_connections 2024-07-26T13:41:52.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.621+0000 7fea5094f740 1 -- 172.21.15.100:0/3777209008 wait complete. 2024-07-26T13:41:52.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.622+0000 7fea5094f740 20 client.15927 trim_cache size 0 max 16384 2024-07-26T13:41:52.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.644+0000 7fea5094f740 1 Processor -- start 2024-07-26T13:41:52.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.644+0000 7fea5094f740 1 -- start start 2024-07-26T13:41:52.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.645+0000 7fea5094f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c08a57990 0x561c08e2d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:52.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.645+0000 7fea5094f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c08a5fb70 0x561c08e27f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:52.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.645+0000 7fea5094f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c08a80850 0x561c08e3ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:52.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.645+0000 7fea5094f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fea30015440 con 0x561c08a80850 2024-07-26T13:41:52.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.645+0000 7fea5094f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fea30012d10 con 0x561c08a5fb70 2024-07-26T13:41:52.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.645+0000 7fea5094f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fea30010b60 con 0x561c08a57990 2024-07-26T13:41:52.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.645+0000 7fea48968640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c08a57990 0x561c08e2d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:52.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.645+0000 7fea48968640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c08a57990 0x561c08e2d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:50606/0 (socket says 172.21.15.100:50606) 2024-07-26T13:41:52.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.645+0000 7fea48968640 1 -- 172.21.15.100:0/4196590881 learned_addr learned my addr 172.21.15.100:0/4196590881 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:52.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.645+0000 7fea48968640 1 --2- 172.21.15.100:0/4196590881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c08a80850 0x561c08e3ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:52.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.646+0000 7fea437fe640 1 --2- 172.21.15.100:0/4196590881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c08a5fb70 0x561c08e27f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:52.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.646+0000 7fea48968640 1 -- 172.21.15.100:0/4196590881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c08a5fb70 msgr2=0x561c08e27f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:52.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.646+0000 7fea48968640 1 --2- 172.21.15.100:0/4196590881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c08a5fb70 0x561c08e27f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:52.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.646+0000 7fea48968640 1 -- 172.21.15.100:0/4196590881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c08a80850 msgr2=0x561c08e3ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:52.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.646+0000 7fea48968640 1 --2- 172.21.15.100:0/4196590881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c08a80850 0x561c08e3ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:52.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.646+0000 7fea48968640 1 -- 172.21.15.100:0/4196590881 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c08e32e40 con 0x561c08a57990 2024-07-26T13:41:52.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.646+0000 7fea48968640 1 --2- 172.21.15.100:0/4196590881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c08a57990 0x561c08e2d350 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fea3c010f40 tx=0x7fea3c000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6abfbab3f7f7c9c6 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:52.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.646+0000 7fea29ffb640 1 -- 172.21.15.100:0/4196590881 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c017040 con 0x561c08a57990 2024-07-26T13:41:52.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.647+0000 7fea29ffb640 1 -- 172.21.15.100:0/4196590881 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea3c00e5d0 con 0x561c08a57990 2024-07-26T13:41:52.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.647+0000 7fea29ffb640 1 -- 172.21.15.100:0/4196590881 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c0333f0 con 0x561c08a57990 2024-07-26T13:41:52.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.648+0000 7fea5094f740 1 -- 172.21.15.100:0/4196590881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c08a57990 msgr2=0x561c08e2d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:52.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.648+0000 7fea5094f740 1 --2- 172.21.15.100:0/4196590881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c08a57990 0x561c08e2d350 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fea3c010f40 tx=0x7fea3c000f40 comp rx=0 tx=0).stop 2024-07-26T13:41:52.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.648+0000 7fea5094f740 1 -- 172.21.15.100:0/4196590881 shutdown_connections 2024-07-26T13:41:52.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.648+0000 7fea5094f740 1 --2- 172.21.15.100:0/4196590881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c08a80850 0x561c08e3ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:52.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.648+0000 7fea5094f740 1 --2- 172.21.15.100:0/4196590881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c08a5fb70 0x561c08e27f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:52.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.648+0000 7fea5094f740 1 --2- 172.21.15.100:0/4196590881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c08a57990 0x561c08e2d350 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:52.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.648+0000 7fea5094f740 1 -- 172.21.15.100:0/4196590881 >> 172.21.15.100:0/4196590881 conn(0x561c08a74c40 msgr2=0x561c08e22a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:52.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.648+0000 7fea5094f740 1 -- 172.21.15.100:0/4196590881 shutdown_connections 2024-07-26T13:41:52.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.648+0000 7fea5094f740 1 -- 172.21.15.100:0/4196590881 wait complete. 2024-07-26T13:41:52.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.649+0000 7fea5094f740 1 -- start start 2024-07-26T13:41:52.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.649+0000 7fea5094f740 1 Processor -- start 2024-07-26T13:41:52.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.650+0000 7fea5094f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c08a57990 0x561c08e3a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:52.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.650+0000 7fea5094f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c08a5fb70 0x561c08e358f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:52.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.650+0000 7fea5094f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c08a80850 0x561c08e35e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:52.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.650+0000 7fea5094f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fea300111d0 con 0x561c08a57990 2024-07-26T13:41:52.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.650+0000 7fea5094f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fea30012640 con 0x561c08a80850 2024-07-26T13:41:52.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.650+0000 7fea5094f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fea300118a0 con 0x561c08a5fb70 2024-07-26T13:41:52.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.650+0000 7fea5094f740 10 client.0 _ll_register_callbacks cb 0x7fea38009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:41:52.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.650+0000 7fea48968640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c08a57990 0x561c08e3a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:52.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.650+0000 7fea437fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c08a5fb70 0x561c08e358f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:52.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.650+0000 7fea48968640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c08a80850 0x561c08e35e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:52.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.650+0000 7fea437fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c08a5fb70 0x561c08e358f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:50614/0 (socket says 172.21.15.100:50614) 2024-07-26T13:41:52.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.650+0000 7fea437fe640 1 -- 172.21.15.100:0/1402614692 learned_addr learned my addr 172.21.15.100:0/1402614692 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:52.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.651+0000 7fea437fe640 1 -- 172.21.15.100:0/1402614692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c08a80850 msgr2=0x561c08e35e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:52.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.651+0000 7fea437fe640 1 --2- 172.21.15.100:0/1402614692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c08a80850 0x561c08e35e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:52.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.651+0000 7fea437fe640 1 -- 172.21.15.100:0/1402614692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c08a57990 msgr2=0x561c08e3a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:52.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.651+0000 7fea437fe640 1 --2- 172.21.15.100:0/1402614692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c08a57990 0x561c08e3a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:52.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.651+0000 7fea437fe640 1 -- 172.21.15.100:0/1402614692 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c08e363f0 con 0x561c08a5fb70 2024-07-26T13:41:52.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.652+0000 7fea437fe640 1 --2- 172.21.15.100:0/1402614692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c08a5fb70 0x561c08e358f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fea3802a450 tx=0x7fea38004410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e839f292c7e9f73 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:52.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.652+0000 7fea237fe640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-07-26T13:41:52.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.652+0000 7fea237fe640 1 -- 172.21.15.100:0/1402614692 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38035030 con 0x561c08a5fb70 2024-07-26T13:41:52.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.652+0000 7fea237fe640 1 -- 172.21.15.100:0/1402614692 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fea38003930 con 0x561c08a5fb70 2024-07-26T13:41:52.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.652+0000 7fea5094f740 1 -- 172.21.15.100:0/1402614692 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea3c00e5d0 con 0x561c08a5fb70 2024-07-26T13:41:52.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.653+0000 7fea237fe640 1 -- 172.21.15.100:0/1402614692 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fea3805e050 con 0x561c08a5fb70 2024-07-26T13:41:52.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.653+0000 7fea5094f740 10 client.25489 fetch_fsmap learned FSMap version 26 2024-07-26T13:41:52.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.653+0000 7fea5094f740 1 -- 172.21.15.100:0/1402614692 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561c08eac270 con 0x561c08a5fb70 2024-07-26T13:41:52.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.653+0000 7fea237fe640 1 -- 172.21.15.100:0/1402614692 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea3c00e5d0 con 0x561c08a5fb70 2024-07-26T13:41:52.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.653+0000 7fea237fe640 1 -- 172.21.15.100:0/1402614692 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fea38061020 con 0x561c08a5fb70 2024-07-26T13:41:52.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.654+0000 7fea5094f740 10 client.25489 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:41:52.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.654+0000 7fea5094f740 10 client.25489 resolve_mds: validated gid 24607 aka daemon mds.i 2024-07-26T13:41:52.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.654+0000 7fea5094f740 20 client.25489 populate_metadata read hostname 'smithi100' 2024-07-26T13:41:52.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.654+0000 7fea5094f740 1 --2- 172.21.15.100:0/1402614692 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x561c08a14580 0x561c08eaeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:52.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.654+0000 7fea5094f740 4 client.25489 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}] 2024-07-26T13:41:52.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.654+0000 7fea5094f740 1 -- 172.21.15.100:0/1402614692 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}) -- 0x561c08d967c0 con 0x561c08a14580 2024-07-26T13:41:52.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.654+0000 7fea48968640 1 --2- 172.21.15.100:0/1402614692 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x561c08a14580 0x561c08eaeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:52.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.655+0000 7fea237fe640 10 client.25489 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:52.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.655+0000 7fea48968640 1 --2- 172.21.15.100:0/1402614692 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x561c08a14580 0x561c08eaeac0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbd27605bd3ea1b5 server_cookie=912948749604ef9e in_seq=0 out_seq=0 2024-07-26T13:41:52.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.788+0000 7fea237fe640 1 -- 172.21.15.100:0/1402614692 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x561c08d967c0 con 0x561c08a14580 2024-07-26T13:41:52.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.788+0000 7fea237fe640 10 client.25489 handle_command_reply: tid=0 2024-07-26T13:41:52.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.788+0000 7fea5094f740 1 client.25489 shutdown 2024-07-26T13:41:52.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.789+0000 7fea5094f740 1 -- 172.21.15.100:0/1402614692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c08a5fb70 msgr2=0x561c08e358f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:52.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.789+0000 7fea5094f740 1 --2- 172.21.15.100:0/1402614692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c08a5fb70 0x561c08e358f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fea3802a450 tx=0x7fea38004410 comp rx=0 tx=0).stop 2024-07-26T13:41:52.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.789+0000 7fea5094f740 1 -- 172.21.15.100:0/1402614692 shutdown_connections 2024-07-26T13:41:52.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.789+0000 7fea5094f740 1 --2- 172.21.15.100:0/1402614692 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x561c08a14580 0x561c08eaeac0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:52.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.789+0000 7fea5094f740 1 --2- 172.21.15.100:0/1402614692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c08a80850 0x561c08e35e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:52.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.789+0000 7fea5094f740 1 --2- 172.21.15.100:0/1402614692 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c08a5fb70 0x561c08e358f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:52.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.789+0000 7fea5094f740 1 --2- 172.21.15.100:0/1402614692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561c08a57990 0x561c08e3a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:52.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.789+0000 7fea237fe640 0 client.25489 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:52.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.789+0000 7fea5094f740 1 -- 172.21.15.100:0/1402614692 >> 172.21.15.100:0/1402614692 conn(0x561c08a74c40 msgr2=0x561c08e22a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:52.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.789+0000 7fea5094f740 1 -- 172.21.15.100:0/1402614692 shutdown_connections 2024-07-26T13:41:52.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.789+0000 7fea5094f740 1 -- 172.21.15.100:0/1402614692 wait complete. 2024-07-26T13:41:52.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.789+0000 7fea5094f740 20 client.25489 trim_cache size 0 max 16384 2024-07-26T13:41:52.795 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:41:52.795 INFO:teuthology.orchestra.run.smithi100.stdout: "epoch": 26, 2024-07-26T13:41:52.795 INFO:teuthology.orchestra.run.smithi100.stdout: "leader": 24607, 2024-07-26T13:41:52.795 INFO:teuthology.orchestra.run.smithi100.stdout: "set_version": 6, 2024-07-26T13:41:52.795 INFO:teuthology.orchestra.run.smithi100.stdout: "sets": { 2024-07-26T13:41:52.795 INFO:teuthology.orchestra.run.smithi100.stdout: "437045fa": { 2024-07-26T13:41:52.795 INFO:teuthology.orchestra.run.smithi100.stdout: "version": 6, 2024-07-26T13:41:52.795 INFO:teuthology.orchestra.run.smithi100.stdout: "age_ref": 0.0, 2024-07-26T13:41:52.795 INFO:teuthology.orchestra.run.smithi100.stdout: "state": { 2024-07-26T13:41:52.795 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "QUIESCED", 2024-07-26T13:41:52.795 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 0.0 2024-07-26T13:41:52.796 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-07-26T13:41:52.796 INFO:teuthology.orchestra.run.smithi100.stdout: "timeout": 90.0, 2024-07-26T13:41:52.796 INFO:teuthology.orchestra.run.smithi100.stdout: "expiration": 142.6, 2024-07-26T13:41:52.796 INFO:teuthology.orchestra.run.smithi100.stdout: "members": { 2024-07-26T13:41:52.796 INFO:teuthology.orchestra.run.smithi100.stdout: "file:/": { 2024-07-26T13:41:52.796 INFO:teuthology.orchestra.run.smithi100.stdout: "excluded": false, 2024-07-26T13:41:52.796 INFO:teuthology.orchestra.run.smithi100.stdout: "state": { 2024-07-26T13:41:52.796 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "QUIESCED", 2024-07-26T13:41:52.796 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 0.0 2024-07-26T13:41:52.796 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:41:52.796 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:41:52.796 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:41:52.796 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:41:52.796 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:41:52.796 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:41:52.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.790+0000 7fea4a3cb640 1 -- 172.21.15.100:0/2814250667 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fea2c0b3d40 msgr2=0x7fea2c0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:52.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.790+0000 7fea4a3cb640 1 --2- 172.21.15.100:0/2814250667 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fea2c0b3d40 0x7fea2c0b6250 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x561c08e25190 tx=0x7fea34092040 comp rx=0 tx=0).stop 2024-07-26T13:41:52.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.790+0000 7fea4a3cb640 1 -- 172.21.15.100:0/2814250667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea441024b0 msgr2=0x7fea44199f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:52.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.790+0000 7fea4a3cb640 1 --2- 172.21.15.100:0/2814250667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea441024b0 0x7fea44199f10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fea3400ea60 tx=0x7fea3400ef20 comp rx=0 tx=0).stop 2024-07-26T13:41:52.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.790+0000 7fea4a3cb640 1 -- 172.21.15.100:0/2814250667 shutdown_connections 2024-07-26T13:41:52.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.790+0000 7fea4a3cb640 1 --2- 172.21.15.100:0/2814250667 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fea2c0b3d40 0x7fea2c0b6250 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:52.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.790+0000 7fea4a3cb640 1 --2- 172.21.15.100:0/2814250667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea44108740 0x7fea4419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:52.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.790+0000 7fea4a3cb640 1 --2- 172.21.15.100:0/2814250667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea44107df0 0x7fea4419a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:52.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.790+0000 7fea4a3cb640 1 --2- 172.21.15.100:0/2814250667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fea441024b0 0x7fea44199f10 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:52.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.790+0000 7fea4a3cb640 1 -- 172.21.15.100:0/2814250667 >> 172.21.15.100:0/2814250667 conn(0x7fea440792f0 msgr2=0x7fea44104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:52.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.790+0000 7fea4a3cb640 1 -- 172.21.15.100:0/2814250667 shutdown_connections 2024-07-26T13:41:52.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:52.790+0000 7fea4a3cb640 1 -- 172.21.15.100:0/2814250667 wait complete. 2024-07-26T13:41:53.402 INFO:tasks.workunit.client.0.smithi100.stdout: 1 20252 17.75 MB/sec warmup 7 sec latency 722.597 ms 2024-07-26T13:41:53.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.592+0000 7fe50085d640 1 -- 172.21.15.100:0/3339065347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4fc107dd0 msgr2=0x7fe4fc1081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:53.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.592+0000 7fe50085d640 1 --2- 172.21.15.100:0/3339065347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4fc107dd0 0x7fe4fc1081e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0009900 tx=0x7fe4f002a410 comp rx=0 tx=0).stop 2024-07-26T13:41:53.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.592+0000 7fe50085d640 1 -- 172.21.15.100:0/3339065347 shutdown_connections 2024-07-26T13:41:53.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.592+0000 7fe50085d640 1 --2- 172.21.15.100:0/3339065347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4fc108720 0x7fe4fc10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:53.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.592+0000 7fe50085d640 1 --2- 172.21.15.100:0/3339065347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4fc107dd0 0x7fe4fc1081e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:53.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.592+0000 7fe50085d640 1 --2- 172.21.15.100:0/3339065347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4fc102490 0x7fe4fc1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:53.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.592+0000 7fe50085d640 1 -- 172.21.15.100:0/3339065347 >> 172.21.15.100:0/3339065347 conn(0x7fe4fc079160 msgr2=0x7fe4fc079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:53.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.592+0000 7fe50085d640 1 -- 172.21.15.100:0/3339065347 shutdown_connections 2024-07-26T13:41:53.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.592+0000 7fe50085d640 1 -- 172.21.15.100:0/3339065347 wait complete. 2024-07-26T13:41:53.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.593+0000 7fe50085d640 1 Processor -- start 2024-07-26T13:41:53.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.593+0000 7fe50085d640 1 -- start start 2024-07-26T13:41:53.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.593+0000 7fe50085d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4fc102490 0x7fe4fc0ff0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:53.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.593+0000 7fe50085d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4fc107dd0 0x7fe4fc0ff5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:53.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.593+0000 7fe50085d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4fc108720 0x7fe4fc0ffb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:53.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.593+0000 7fe50085d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc1123a0 con 0x7fe4fc102490 2024-07-26T13:41:53.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.593+0000 7fe50085d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc112220 con 0x7fe4fc108720 2024-07-26T13:41:53.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.593+0000 7fe50085d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc112520 con 0x7fe4fc107dd0 2024-07-26T13:41:53.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.593+0000 7fe4fa59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4fc102490 0x7fe4fc0ff0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:53.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.594+0000 7fe4fa59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4fc102490 0x7fe4fc0ff0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35470/0 (socket says 172.21.15.100:35470) 2024-07-26T13:41:53.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.594+0000 7fe4fa59c640 1 -- 172.21.15.100:0/655761726 learned_addr learned my addr 172.21.15.100:0/655761726 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:53.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.594+0000 7fe4f9d9b640 1 --2- 172.21.15.100:0/655761726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4fc107dd0 0x7fe4fc0ff5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:53.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.594+0000 7fe4fad9d640 1 --2- 172.21.15.100:0/655761726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4fc108720 0x7fe4fc0ffb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:53.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.594+0000 7fe4fa59c640 1 -- 172.21.15.100:0/655761726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4fc107dd0 msgr2=0x7fe4fc0ff5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:53.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.594+0000 7fe4fa59c640 1 --2- 172.21.15.100:0/655761726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4fc107dd0 0x7fe4fc0ff5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:53.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.594+0000 7fe4fa59c640 1 -- 172.21.15.100:0/655761726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4fc108720 msgr2=0x7fe4fc0ffb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:53.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.594+0000 7fe4fa59c640 1 --2- 172.21.15.100:0/655761726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4fc108720 0x7fe4fc0ffb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:53.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.594+0000 7fe4fa59c640 1 -- 172.21.15.100:0/655761726 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4fc1a80f0 con 0x7fe4fc102490 2024-07-26T13:41:53.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.594+0000 7fe4fad9d640 1 --2- 172.21.15.100:0/655761726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4fc108720 0x7fe4fc0ffb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:41:53.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.594+0000 7fe4fa59c640 1 --2- 172.21.15.100:0/655761726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4fc102490 0x7fe4fc0ff0a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec00eab0 tx=0x7fe4ec00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:53.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.594+0000 7fe4eb7fe640 1 -- 172.21.15.100:0/655761726 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec00ce40 con 0x7fe4fc102490 2024-07-26T13:41:53.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.594+0000 7fe50085d640 1 -- 172.21.15.100:0/655761726 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4fc1a83e0 con 0x7fe4fc102490 2024-07-26T13:41:53.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.594+0000 7fe4eb7fe640 1 -- 172.21.15.100:0/655761726 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4ec011920 con 0x7fe4fc102490 2024-07-26T13:41:53.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.594+0000 7fe4eb7fe640 1 -- 172.21.15.100:0/655761726 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec010a30 con 0x7fe4fc102490 2024-07-26T13:41:53.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.594+0000 7fe50085d640 1 -- 172.21.15.100:0/655761726 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4fc1a8940 con 0x7fe4fc102490 2024-07-26T13:41:53.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.597+0000 7fe4eb7fe640 1 -- 172.21.15.100:0/655761726 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fe4ec010bd0 con 0x7fe4fc102490 2024-07-26T13:41:53.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.597+0000 7fe506de1740 1 Processor -- start 2024-07-26T13:41:53.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.597+0000 7fe506de1740 1 -- start start 2024-07-26T13:41:53.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.597+0000 7fe506de1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e4350070 0x5636e46f5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:53.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.597+0000 7fe506de1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e43271b0 0x5636e46f5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:53.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.597+0000 7fe506de1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636e432f390 0x5636e46f5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:53.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.598+0000 7fe506de1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5636e466bb90 con 0x5636e432f390 2024-07-26T13:41:53.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.598+0000 7fe506de1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5636e465b090 con 0x5636e43271b0 2024-07-26T13:41:53.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.598+0000 7fe506de1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5636e465f0d0 con 0x5636e4350070 2024-07-26T13:41:53.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.598+0000 7fe4f9d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636e432f390 0x5636e46f5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:53.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.598+0000 7fe4f9d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636e432f390 0x5636e46f5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35486/0 (socket says 172.21.15.100:35486) 2024-07-26T13:41:53.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.598+0000 7fe4f9d9b640 1 -- 172.21.15.100:0/4287475029 learned_addr learned my addr 172.21.15.100:0/4287475029 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:53.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.598+0000 7fe4fa59c640 1 --2- 172.21.15.100:0/4287475029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e43271b0 0x5636e46f5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:53.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.598+0000 7fe4fad9d640 1 --2- 172.21.15.100:0/4287475029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e4350070 0x5636e46f5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:53.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.598+0000 7fe4eb7fe640 1 --2- 172.21.15.100:0/655761726 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe4e40b3d40 0x7fe4e40b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:53.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.598+0000 7fe4f9d9b640 1 -- 172.21.15.100:0/4287475029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e4350070 msgr2=0x5636e46f5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:53.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.598+0000 7fe4f9d9b640 1 --2- 172.21.15.100:0/4287475029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e4350070 0x5636e46f5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:53.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.598+0000 7fe4f9d9b640 1 -- 172.21.15.100:0/4287475029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e43271b0 msgr2=0x5636e46f5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:53.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.598+0000 7fe4f9d9b640 1 --2- 172.21.15.100:0/4287475029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e43271b0 0x5636e46f5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:53.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.598+0000 7fe4f9d9b640 1 -- 172.21.15.100:0/4287475029 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636e4601b70 con 0x5636e432f390 2024-07-26T13:41:53.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.598+0000 7fe4fad9d640 1 --2- 172.21.15.100:0/4287475029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e4350070 0x5636e46f5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:41:53.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.598+0000 7fe4fad9d640 1 --2- 172.21.15.100:0/655761726 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe4e40b3d40 0x7fe4e40b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:53.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.598+0000 7fe4eb7fe640 1 -- 172.21.15.100:0/655761726 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fe4ec0e4760 con 0x7fe4fc102490 2024-07-26T13:41:53.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.598+0000 7fe4f9d9b640 1 --2- 172.21.15.100:0/4287475029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636e432f390 0x5636e46f5fd0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc06a4d0 tx=0x7fe4f0004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a009a785338aca8 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:53.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.599+0000 7fe4e8ff9640 1 -- 172.21.15.100:0/4287475029 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0035030 con 0x5636e432f390 2024-07-26T13:41:53.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.599+0000 7fe4e8ff9640 1 -- 172.21.15.100:0/4287475029 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4f00054f0 con 0x5636e432f390 2024-07-26T13:41:53.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.599+0000 7fe4e8ff9640 1 -- 172.21.15.100:0/4287475029 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0003850 con 0x5636e432f390 2024-07-26T13:41:53.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.599+0000 7fe4fad9d640 1 --2- 172.21.15.100:0/655761726 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe4e40b3d40 0x7fe4e40b6250 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe4f400bc50 tx=0x7fe4f40046c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:53.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.600+0000 7fe506de1740 1 -- 172.21.15.100:0/4287475029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636e432f390 msgr2=0x5636e46f5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:53.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.600+0000 7fe506de1740 1 --2- 172.21.15.100:0/4287475029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636e432f390 0x5636e46f5fd0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc06a4d0 tx=0x7fe4f0004390 comp rx=0 tx=0).stop 2024-07-26T13:41:53.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.600+0000 7fe506de1740 1 -- 172.21.15.100:0/4287475029 shutdown_connections 2024-07-26T13:41:53.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.600+0000 7fe506de1740 1 --2- 172.21.15.100:0/4287475029 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636e432f390 0x5636e46f5fd0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:53.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.600+0000 7fe506de1740 1 --2- 172.21.15.100:0/4287475029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e43271b0 0x5636e46f5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:53.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.600+0000 7fe506de1740 1 --2- 172.21.15.100:0/4287475029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e4350070 0x5636e46f5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:53.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.600+0000 7fe506de1740 1 -- 172.21.15.100:0/4287475029 >> 172.21.15.100:0/4287475029 conn(0x5636e4344460 msgr2=0x5636e46f3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:53.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.600+0000 7fe506de1740 1 -- 172.21.15.100:0/4287475029 shutdown_connections 2024-07-26T13:41:53.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.600+0000 7fe506de1740 1 -- 172.21.15.100:0/4287475029 wait complete. 2024-07-26T13:41:53.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.600+0000 7fe506de1740 1 -- start start 2024-07-26T13:41:53.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.600+0000 7fe506de1740 1 Processor -- start 2024-07-26T13:41:53.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.601+0000 7fe506de1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636e43271b0 0x5636e4701180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:53.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.601+0000 7fe506de1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e432f390 0x5636e4701720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:53.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.601+0000 7fe506de1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e4350070 0x5636e470cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:53.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.601+0000 7fe506de1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5636e44be700 con 0x5636e43271b0 2024-07-26T13:41:53.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.601+0000 7fe506de1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5636e45fd9c0 con 0x5636e432f390 2024-07-26T13:41:53.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.601+0000 7fe506de1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5636e4703cd0 con 0x5636e4350070 2024-07-26T13:41:53.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.601+0000 7fe4fa59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636e43271b0 0x5636e4701180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:53.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.601+0000 7fe506de1740 10 client.0 _ll_register_callbacks cb 0x5636e46184d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:41:53.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.601+0000 7fe4fa59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636e43271b0 0x5636e4701180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35500/0 (socket says 172.21.15.100:35500) 2024-07-26T13:41:53.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.601+0000 7fe4fa59c640 1 -- 172.21.15.100:0/4176038481 learned_addr learned my addr 172.21.15.100:0/4176038481 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:53.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.601+0000 7fe4f9d9b640 1 --2- 172.21.15.100:0/4176038481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e432f390 0x5636e4701720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:53.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.601+0000 7fe4fad9d640 1 --2- 172.21.15.100:0/4176038481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e4350070 0x5636e470cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:53.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.601+0000 7fe4fa59c640 1 -- 172.21.15.100:0/4176038481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e4350070 msgr2=0x5636e470cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:53.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.601+0000 7fe4fa59c640 1 --2- 172.21.15.100:0/4176038481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e4350070 0x5636e470cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:53.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.601+0000 7fe4fa59c640 1 -- 172.21.15.100:0/4176038481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e432f390 msgr2=0x5636e4701720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:53.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.601+0000 7fe4fa59c640 1 --2- 172.21.15.100:0/4176038481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e432f390 0x5636e4701720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:53.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.601+0000 7fe4fa59c640 1 -- 172.21.15.100:0/4176038481 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636e43bb650 con 0x5636e43271b0 2024-07-26T13:41:53.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.601+0000 7fe4f9d9b640 1 --2- 172.21.15.100:0/4176038481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e432f390 0x5636e4701720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:41:53.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.601+0000 7fe4fa59c640 1 --2- 172.21.15.100:0/4176038481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636e43271b0 0x5636e4701180 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0e1f30 tx=0x7fe4ec0094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa56e4fcd45353d server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:53.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.601+0000 7fe4e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:41:53.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.601+0000 7fe4e1ffb640 1 -- 172.21.15.100:0/4176038481 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec0aba90 con 0x5636e43271b0 2024-07-26T13:41:53.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.602+0000 7fe4e1ffb640 1 -- 172.21.15.100:0/4176038481 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fe4ec0a7620 con 0x5636e43271b0 2024-07-26T13:41:53.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.602+0000 7fe506de1740 1 -- 172.21.15.100:0/4176038481 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4f00054f0 con 0x5636e43271b0 2024-07-26T13:41:53.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.602+0000 7fe4e1ffb640 1 -- 172.21.15.100:0/4176038481 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe4ec0ab390 con 0x5636e43271b0 2024-07-26T13:41:53.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.602+0000 7fe506de1740 10 client.15945 fetch_fsmap learned FSMap version 26 2024-07-26T13:41:53.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.602+0000 7fe506de1740 1 -- 172.21.15.100:0/4176038481 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5636e470fdc0 con 0x5636e43271b0 2024-07-26T13:41:53.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.602+0000 7fe4e1ffb640 1 -- 172.21.15.100:0/4176038481 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4f00054f0 con 0x5636e43271b0 2024-07-26T13:41:53.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.602+0000 7fe4e1ffb640 1 -- 172.21.15.100:0/4176038481 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fe4ec00f040 con 0x5636e43271b0 2024-07-26T13:41:53.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.602+0000 7fe506de1740 10 client.15945 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:41:53.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.602+0000 7fe506de1740 10 client.15945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:41:53.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.602+0000 7fe506de1740 20 client.15945 populate_metadata read hostname 'smithi100' 2024-07-26T13:41:53.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.602+0000 7fe506de1740 1 --2- 172.21.15.100:0/4176038481 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5636e430e9f0 0x5636e4713220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:53.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.602+0000 7fe506de1740 4 client.15945 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:41:53.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.602+0000 7fe506de1740 1 -- 172.21.15.100:0/4176038481 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636e466dae0 con 0x5636e430e9f0 2024-07-26T13:41:53.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.602+0000 7fe4f9d9b640 1 --2- 172.21.15.100:0/4176038481 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5636e430e9f0 0x5636e4713220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:53.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.603+0000 7fe4e1ffb640 10 client.15945 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:53.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.603+0000 7fe4f9d9b640 1 --2- 172.21.15.100:0/4176038481 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5636e430e9f0 0x5636e4713220 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5271006327de20c7 server_cookie=c4265117ce95d0fb in_seq=0 out_seq=0 2024-07-26T13:41:53.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.604+0000 7fe4e1ffb640 1 -- 172.21.15.100:0/4176038481 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5636e466dae0 con 0x5636e430e9f0 2024-07-26T13:41:53.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.604+0000 7fe4e1ffb640 10 client.15945 handle_command_reply: tid=0 2024-07-26T13:41:53.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.604+0000 7fe506de1740 1 client.15945 shutdown 2024-07-26T13:41:53.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.604+0000 7fe506de1740 1 -- 172.21.15.100:0/4176038481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636e43271b0 msgr2=0x5636e4701180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:53.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.604+0000 7fe506de1740 1 --2- 172.21.15.100:0/4176038481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636e43271b0 0x5636e4701180 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0e1f30 tx=0x7fe4ec0094a0 comp rx=0 tx=0).stop 2024-07-26T13:41:53.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.604+0000 7fe506de1740 1 -- 172.21.15.100:0/4176038481 shutdown_connections 2024-07-26T13:41:53.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.604+0000 7fe506de1740 1 --2- 172.21.15.100:0/4176038481 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5636e430e9f0 0x5636e4713220 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:53.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.604+0000 7fe506de1740 1 --2- 172.21.15.100:0/4176038481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e4350070 0x5636e470cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:53.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.604+0000 7fe506de1740 1 --2- 172.21.15.100:0/4176038481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e432f390 0x5636e4701720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:53.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.604+0000 7fe4e1ffb640 0 client.15945 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:53.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.604+0000 7fe506de1740 1 --2- 172.21.15.100:0/4176038481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636e43271b0 0x5636e4701180 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:53.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.604+0000 7fe506de1740 1 -- 172.21.15.100:0/4176038481 >> 172.21.15.100:0/4176038481 conn(0x5636e4344460 msgr2=0x5636e46f8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:53.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.604+0000 7fe506de1740 1 -- 172.21.15.100:0/4176038481 shutdown_connections 2024-07-26T13:41:53.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.604+0000 7fe506de1740 1 -- 172.21.15.100:0/4176038481 wait complete. 2024-07-26T13:41:53.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.604+0000 7fe506de1740 20 client.15945 trim_cache size 0 max 16384 2024-07-26T13:41:53.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.621+0000 7fe506de1740 1 Processor -- start 2024-07-26T13:41:53.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.621+0000 7fe506de1740 1 -- start start 2024-07-26T13:41:53.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.621+0000 7fe506de1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e43271b0 0x5636e470cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:53.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.621+0000 7fe506de1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636e432f390 0x5636e4702bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:53.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.621+0000 7fe506de1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e4350070 0x5636e46fa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:53.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.621+0000 7fe506de1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc1b3ad0 con 0x5636e432f390 2024-07-26T13:41:53.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.621+0000 7fe506de1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc1b13a0 con 0x5636e4350070 2024-07-26T13:41:53.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.621+0000 7fe506de1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc1af1f0 con 0x5636e43271b0 2024-07-26T13:41:53.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.621+0000 7fe4f9d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636e432f390 0x5636e4702bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:53.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.621+0000 7fe4f9d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636e432f390 0x5636e4702bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35510/0 (socket says 172.21.15.100:35510) 2024-07-26T13:41:53.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.621+0000 7fe4f9d9b640 1 -- 172.21.15.100:0/4058786114 learned_addr learned my addr 172.21.15.100:0/4058786114 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:53.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.622+0000 7fe4fa59c640 1 --2- 172.21.15.100:0/4058786114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e43271b0 0x5636e470cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:53.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.622+0000 7fe4fad9d640 1 --2- 172.21.15.100:0/4058786114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e4350070 0x5636e46fa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:53.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.622+0000 7fe4f9d9b640 1 -- 172.21.15.100:0/4058786114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e43271b0 msgr2=0x5636e470cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:53.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.622+0000 7fe4f9d9b640 1 --2- 172.21.15.100:0/4058786114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e43271b0 0x5636e470cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:53.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.622+0000 7fe4f9d9b640 1 -- 172.21.15.100:0/4058786114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e4350070 msgr2=0x5636e46fa7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:53.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.622+0000 7fe4f9d9b640 1 --2- 172.21.15.100:0/4058786114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e4350070 0x5636e46fa7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:53.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.622+0000 7fe4f9d9b640 1 -- 172.21.15.100:0/4058786114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636e46fbc70 con 0x5636e432f390 2024-07-26T13:41:53.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.622+0000 7fe4fad9d640 1 --2- 172.21.15.100:0/4058786114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e4350070 0x5636e46fa7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:41:53.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.622+0000 7fe4f9d9b640 1 --2- 172.21.15.100:0/4058786114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636e432f390 0x5636e4702bb0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0009900 tx=0x7fe4f0005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857262101b40dd0d server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:53.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.622+0000 7fe4e3fff640 1 -- 172.21.15.100:0/4058786114 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0035040 con 0x5636e432f390 2024-07-26T13:41:53.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.622+0000 7fe4e3fff640 1 -- 172.21.15.100:0/4058786114 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4f0053040 con 0x5636e432f390 2024-07-26T13:41:53.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.622+0000 7fe4e3fff640 1 -- 172.21.15.100:0/4058786114 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0031700 con 0x5636e432f390 2024-07-26T13:41:53.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.623+0000 7fe506de1740 1 -- 172.21.15.100:0/4058786114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636e432f390 msgr2=0x5636e4702bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:53.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.623+0000 7fe506de1740 1 --2- 172.21.15.100:0/4058786114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636e432f390 0x5636e4702bb0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0009900 tx=0x7fe4f0005ee0 comp rx=0 tx=0).stop 2024-07-26T13:41:53.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.623+0000 7fe506de1740 1 -- 172.21.15.100:0/4058786114 shutdown_connections 2024-07-26T13:41:53.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.623+0000 7fe506de1740 1 --2- 172.21.15.100:0/4058786114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e4350070 0x5636e46fa7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:53.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.623+0000 7fe506de1740 1 --2- 172.21.15.100:0/4058786114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636e432f390 0x5636e4702bb0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:53.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.623+0000 7fe506de1740 1 --2- 172.21.15.100:0/4058786114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e43271b0 0x5636e470cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:53.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.623+0000 7fe506de1740 1 -- 172.21.15.100:0/4058786114 >> 172.21.15.100:0/4058786114 conn(0x5636e4344460 msgr2=0x5636e47024a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:53.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.623+0000 7fe506de1740 1 -- 172.21.15.100:0/4058786114 shutdown_connections 2024-07-26T13:41:53.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.623+0000 7fe506de1740 1 -- 172.21.15.100:0/4058786114 wait complete. 2024-07-26T13:41:53.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.623+0000 7fe506de1740 1 -- start start 2024-07-26T13:41:53.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.623+0000 7fe506de1740 1 Processor -- start 2024-07-26T13:41:53.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.624+0000 7fe506de1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e43271b0 0x5636e47095b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:53.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.624+0000 7fe506de1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e432f390 0x5636e46f2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:53.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.624+0000 7fe506de1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636e4350070 0x5636e46f2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:53.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.624+0000 7fe506de1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc1af860 con 0x5636e4350070 2024-07-26T13:41:53.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.624+0000 7fe506de1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc1b0cd0 con 0x5636e43271b0 2024-07-26T13:41:53.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.624+0000 7fe506de1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc1aff30 con 0x5636e432f390 2024-07-26T13:41:53.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.624+0000 7fe506de1740 10 client.0 _ll_register_callbacks cb 0x7fe4ec0aab00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:41:53.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.624+0000 7fe4fad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636e4350070 0x5636e46f2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.624+0000 7fe4fad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636e4350070 0x5636e46f2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35518/0 (socket says 172.21.15.100:35518) 2024-07-26T13:41:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.624+0000 7fe4fad9d640 1 -- 172.21.15.100:0/1908429401 learned_addr learned my addr 172.21.15.100:0/1908429401 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.624+0000 7fe4f9d9b640 1 --2- 172.21.15.100:0/1908429401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e432f390 0x5636e46f2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.624+0000 7fe4fa59c640 1 --2- 172.21.15.100:0/1908429401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e43271b0 0x5636e47095b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.624+0000 7fe4fad9d640 1 -- 172.21.15.100:0/1908429401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e432f390 msgr2=0x5636e46f2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.624+0000 7fe4fad9d640 1 --2- 172.21.15.100:0/1908429401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e432f390 0x5636e46f2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.624+0000 7fe4fad9d640 1 -- 172.21.15.100:0/1908429401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e43271b0 msgr2=0x5636e47095b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.624+0000 7fe4fad9d640 1 --2- 172.21.15.100:0/1908429401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e43271b0 0x5636e47095b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.624+0000 7fe4fad9d640 1 -- 172.21.15.100:0/1908429401 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636e46f3380 con 0x5636e4350070 2024-07-26T13:41:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.624+0000 7fe4fad9d640 1 --2- 172.21.15.100:0/1908429401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636e4350070 0x5636e46f2c40 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe4f401ee40 tx=0x7fe4f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61d91b9a621ca4c server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.624+0000 7fe4e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:41:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.624+0000 7fe4e1ffb640 1 -- 172.21.15.100:0/1908429401 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4027640 con 0x5636e4350070 2024-07-26T13:41:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.624+0000 7fe4e1ffb640 1 -- 172.21.15.100:0/1908429401 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fe4f401a430 con 0x5636e4350070 2024-07-26T13:41:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.624+0000 7fe506de1740 1 -- 172.21.15.100:0/1908429401 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4f0053040 con 0x5636e4350070 2024-07-26T13:41:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.625+0000 7fe4e1ffb640 1 -- 172.21.15.100:0/1908429401 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe4f40277e0 con 0x5636e4350070 2024-07-26T13:41:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.625+0000 7fe506de1740 10 client.15951 fetch_fsmap learned FSMap version 26 2024-07-26T13:41:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.625+0000 7fe506de1740 1 -- 172.21.15.100:0/1908429401 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5636e477b9b0 con 0x5636e4350070 2024-07-26T13:41:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.625+0000 7fe4e1ffb640 1 -- 172.21.15.100:0/1908429401 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4f0053040 con 0x5636e4350070 2024-07-26T13:41:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.625+0000 7fe4e1ffb640 1 -- 172.21.15.100:0/1908429401 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fe4f402e070 con 0x5636e4350070 2024-07-26T13:41:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.625+0000 7fe506de1740 10 client.15951 fetch_fsmap finished waiting for FSMap version 26 2024-07-26T13:41:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.625+0000 7fe506de1740 10 client.15951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:41:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.625+0000 7fe506de1740 20 client.15951 populate_metadata read hostname 'smithi100' 2024-07-26T13:41:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.625+0000 7fe506de1740 1 --2- 172.21.15.100:0/1908429401 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5636e42e3da0 0x5636e477e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.625+0000 7fe506de1740 4 client.15951 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-26T13:41:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.625+0000 7fe506de1740 1 -- 172.21.15.100:0/1908429401 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5636e4669580 con 0x5636e42e3da0 2024-07-26T13:41:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.625+0000 7fe4fa59c640 1 --2- 172.21.15.100:0/1908429401 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5636e42e3da0 0x5636e477e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.626+0000 7fe4e1ffb640 10 client.15951 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.626+0000 7fe4fa59c640 1 --2- 172.21.15.100:0/1908429401 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5636e42e3da0 0x5636e477e100 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ccd94380645df62 server_cookie=28f9ba6ba2975e8 in_seq=0 out_seq=0 2024-07-26T13:41:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.626+0000 7fe4e1ffb640 1 -- 172.21.15.100:0/1908429401 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5636e4669580 con 0x5636e42e3da0 2024-07-26T13:41:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.626+0000 7fe4e1ffb640 10 client.15951 handle_command_reply: tid=0 2024-07-26T13:41:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.626+0000 7fe506de1740 1 client.15951 shutdown 2024-07-26T13:41:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.626+0000 7fe506de1740 1 -- 172.21.15.100:0/1908429401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636e4350070 msgr2=0x5636e46f2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.626+0000 7fe506de1740 1 --2- 172.21.15.100:0/1908429401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636e4350070 0x5636e46f2c40 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe4f401ee40 tx=0x7fe4f4004730 comp rx=0 tx=0).stop 2024-07-26T13:41:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.626+0000 7fe506de1740 1 -- 172.21.15.100:0/1908429401 shutdown_connections 2024-07-26T13:41:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.626+0000 7fe506de1740 1 --2- 172.21.15.100:0/1908429401 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5636e42e3da0 0x5636e477e100 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.626+0000 7fe506de1740 1 --2- 172.21.15.100:0/1908429401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5636e4350070 0x5636e46f2c40 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.626+0000 7fe506de1740 1 --2- 172.21.15.100:0/1908429401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5636e432f390 0x5636e46f2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.626+0000 7fe506de1740 1 --2- 172.21.15.100:0/1908429401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636e43271b0 0x5636e47095b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.626+0000 7fe506de1740 1 -- 172.21.15.100:0/1908429401 >> 172.21.15.100:0/1908429401 conn(0x5636e4344460 msgr2=0x5636e4702df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:53.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.626+0000 7fe4e1ffb640 0 client.15951 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:53.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.626+0000 7fe506de1740 1 -- 172.21.15.100:0/1908429401 shutdown_connections 2024-07-26T13:41:53.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.626+0000 7fe506de1740 1 -- 172.21.15.100:0/1908429401 wait complete. 2024-07-26T13:41:53.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.626+0000 7fe506de1740 20 client.15951 trim_cache size 0 max 16384 2024-07-26T13:41:53.633 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:41:53.633 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-07-26T13:41:53.633 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-07-26T13:41:53.633 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:41:53.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.627+0000 7fe50085d640 1 -- 172.21.15.100:0/655761726 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe4e40b3d40 msgr2=0x7fe4e40b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:53.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.627+0000 7fe50085d640 1 --2- 172.21.15.100:0/655761726 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe4e40b3d40 0x7fe4e40b6250 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe4f400bc50 tx=0x7fe4f40046c0 comp rx=0 tx=0).stop 2024-07-26T13:41:53.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.627+0000 7fe50085d640 1 -- 172.21.15.100:0/655761726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4fc102490 msgr2=0x7fe4fc0ff0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:53.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.627+0000 7fe50085d640 1 --2- 172.21.15.100:0/655761726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4fc102490 0x7fe4fc0ff0a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec00eab0 tx=0x7fe4ec00ef70 comp rx=0 tx=0).stop 2024-07-26T13:41:53.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.627+0000 7fe50085d640 1 -- 172.21.15.100:0/655761726 shutdown_connections 2024-07-26T13:41:53.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.627+0000 7fe50085d640 1 --2- 172.21.15.100:0/655761726 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe4e40b3d40 0x7fe4e40b6250 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:53.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.627+0000 7fe50085d640 1 --2- 172.21.15.100:0/655761726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4fc108720 0x7fe4fc0ffb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:53.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.627+0000 7fe50085d640 1 --2- 172.21.15.100:0/655761726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4fc107dd0 0x7fe4fc0ff5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:53.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.627+0000 7fe50085d640 1 --2- 172.21.15.100:0/655761726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4fc102490 0x7fe4fc0ff0a0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:53.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.627+0000 7fe50085d640 1 -- 172.21.15.100:0/655761726 >> 172.21.15.100:0/655761726 conn(0x7fe4fc079160 msgr2=0x7fe4fc06ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:53.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.627+0000 7fe50085d640 1 -- 172.21.15.100:0/655761726 shutdown_connections 2024-07-26T13:41:53.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:53.627+0000 7fe50085d640 1 -- 172.21.15.100:0/655761726 wait complete. 2024-07-26T13:41:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:53 smithi100 ceph-mon[32599]: pgmap v417: 97 pgs: 97 active+clean; 6.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 542 KiB/s wr, 70 op/s 2024-07-26T13:41:53.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:53 smithi167 ceph-mon[37140]: pgmap v417: 97 pgs: 97 active+clean; 6.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 542 KiB/s wr, 70 op/s 2024-07-26T13:41:53.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:53 smithi138 ceph-mon[36954]: pgmap v417: 97 pgs: 97 active+clean; 6.9 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 542 KiB/s wr, 70 op/s 2024-07-26T13:41:54.402 INFO:tasks.workunit.client.0.smithi100.stdout: 1 20252 15.53 MB/sec warmup 8 sec latency 1722.693 ms 2024-07-26T13:41:54.759 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '437045fa', quiesce took 11.3 seconds. Will release after: 11.25 2024-07-26T13:41:55.402 INFO:tasks.workunit.client.0.smithi100.stdout: 1 20252 13.81 MB/sec warmup 9 sec latency 2722.761 ms 2024-07-26T13:41:55.519 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-26T13:41:55.519 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:41:55.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:55 smithi100 ceph-mon[32599]: pgmap v418: 97 pgs: 97 active+clean; 13 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-07-26T13:41:55.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:55 smithi167 ceph-mon[37140]: pgmap v418: 97 pgs: 97 active+clean; 13 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-07-26T13:41:55.780 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:41:55.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:55 smithi138 ceph-mon[36954]: pgmap v418: 97 pgs: 97 active+clean; 13 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-07-26T13:41:56.403 INFO:tasks.workunit.client.0.smithi100.stdout: 1 20252 12.43 MB/sec warmup 10 sec latency 3722.834 ms 2024-07-26T13:41:56.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:56 smithi167 ceph-mon[37140]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:41:56.911 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:56 smithi100 ceph-mon[32599]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:41:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:56 smithi138 ceph-mon[36954]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:41:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.389+0000 7f1529da1640 1 -- 172.21.15.100:0/2519476171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1524103e30 msgr2=0x7f1524104240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.389+0000 7f1529da1640 1 --2- 172.21.15.100:0/2519476171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1524103e30 0x7f1524104240 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1518009900 tx=0x7f151802a410 comp rx=0 tx=0).stop 2024-07-26T13:41:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.389+0000 7f1529da1640 1 -- 172.21.15.100:0/2519476171 shutdown_connections 2024-07-26T13:41:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.389+0000 7f1529da1640 1 --2- 172.21.15.100:0/2519476171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1524102050 0x7f1524111ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.389+0000 7f1529da1640 1 --2- 172.21.15.100:0/2519476171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1524101580 0x7f1524101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.389+0000 7f1529da1640 1 --2- 172.21.15.100:0/2519476171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1524103e30 0x7f1524104240 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.389+0000 7f1529da1640 1 -- 172.21.15.100:0/2519476171 >> 172.21.15.100:0/2519476171 conn(0x7f1524079530 msgr2=0x7f1524079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.389+0000 7f1529da1640 1 -- 172.21.15.100:0/2519476171 shutdown_connections 2024-07-26T13:41:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.389+0000 7f1529da1640 1 -- 172.21.15.100:0/2519476171 wait complete. 2024-07-26T13:41:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.389+0000 7f1529da1640 1 Processor -- start 2024-07-26T13:41:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.389+0000 7f1529da1640 1 -- start start 2024-07-26T13:41:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.390+0000 7f1529da1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1524101580 0x7f152419bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.390+0000 7f1529da1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1524102050 0x7f152419c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.390+0000 7f1529da1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1524103e30 0x7f15241a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.390+0000 7f1529da1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1524115460 con 0x7f1524101580 2024-07-26T13:41:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.390+0000 7f1529da1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f15241152e0 con 0x7f1524103e30 2024-07-26T13:41:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.390+0000 7f1529da1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f15241155e0 con 0x7f1524102050 2024-07-26T13:41:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.390+0000 7f1528d9f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1524101580 0x7f152419bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.390+0000 7f1528d9f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1524101580 0x7f152419bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46482/0 (socket says 172.21.15.100:46482) 2024-07-26T13:41:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.390+0000 7f1528d9f640 1 -- 172.21.15.100:0/1421476068 learned_addr learned my addr 172.21.15.100:0/1421476068 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.390+0000 7f1523fff640 1 --2- 172.21.15.100:0/1421476068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1524102050 0x7f152419c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:57.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.390+0000 7f15295a0640 1 --2- 172.21.15.100:0/1421476068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1524103e30 0x7f15241a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.390+0000 7f1528d9f640 1 -- 172.21.15.100:0/1421476068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1524102050 msgr2=0x7f152419c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.390+0000 7f1528d9f640 1 --2- 172.21.15.100:0/1421476068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1524102050 0x7f152419c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.390+0000 7f1528d9f640 1 -- 172.21.15.100:0/1421476068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1524103e30 msgr2=0x7f15241a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.390+0000 7f1528d9f640 1 --2- 172.21.15.100:0/1421476068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1524103e30 0x7f15241a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.390+0000 7f1528d9f640 1 -- 172.21.15.100:0/1421476068 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15241a1e10 con 0x7f1524101580 2024-07-26T13:41:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.390+0000 7f15295a0640 1 --2- 172.21.15.100:0/1421476068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1524103e30 0x7f15241a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:41:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.390+0000 7f1528d9f640 1 --2- 172.21.15.100:0/1421476068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1524101580 0x7f152419bee0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1518005940 tx=0x7f15180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.390+0000 7f1521ffb640 1 -- 172.21.15.100:0/1421476068 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151803f030 con 0x7f1524101580 2024-07-26T13:41:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.390+0000 7f1521ffb640 1 -- 172.21.15.100:0/1421476068 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1518003cb0 con 0x7f1524101580 2024-07-26T13:41:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.390+0000 7f1529da1640 1 -- 172.21.15.100:0/1421476068 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15241a20a0 con 0x7f1524101580 2024-07-26T13:41:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.390+0000 7f1521ffb640 1 -- 172.21.15.100:0/1421476068 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1518035070 con 0x7f1524101580 2024-07-26T13:41:57.394 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.391+0000 7f1529da1640 1 -- 172.21.15.100:0/1421476068 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15241a8890 con 0x7f1524101580 2024-07-26T13:41:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.393+0000 7f1531ff1740 1 Processor -- start 2024-07-26T13:41:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.393+0000 7f1521ffb640 1 -- 172.21.15.100:0/1421476068 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f1518042030 con 0x7f1524101580 2024-07-26T13:41:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.393+0000 7f1531ff1740 1 -- start start 2024-07-26T13:41:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.394+0000 7f1531ff1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f5c3cf21b0 0x55f5c40948b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.394+0000 7f1531ff1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5c3cc92f0 0x55f5c4094df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.394+0000 7f1531ff1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5c3cd14d0 0x55f5c4095330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.394+0000 7f1531ff1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55f5c400a3b0 con 0x55f5c3cf21b0 2024-07-26T13:41:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.394+0000 7f1531ff1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f5c3ff98b0 con 0x55f5c3cd14d0 2024-07-26T13:41:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.394+0000 7f1531ff1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f5c3ffd8f0 con 0x55f5c3cc92f0 2024-07-26T13:41:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.394+0000 7f15295a0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f5c3cf21b0 0x55f5c40948b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.394+0000 7f15295a0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f5c3cf21b0 0x55f5c40948b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46492/0 (socket says 172.21.15.100:46492) 2024-07-26T13:41:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.394+0000 7f15295a0640 1 -- 172.21.15.100:0/2011320317 learned_addr learned my addr 172.21.15.100:0/2011320317 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.394+0000 7f1528d9f640 1 --2- 172.21.15.100:0/2011320317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5c3cc92f0 0x55f5c4094df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.394+0000 7f1523fff640 1 --2- 172.21.15.100:0/2011320317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5c3cd14d0 0x55f5c4095330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.394+0000 7f1521ffb640 1 --2- 172.21.15.100:0/1421476068 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f15140b3dc0 0x7f15140b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.394+0000 7f15295a0640 1 -- 172.21.15.100:0/2011320317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5c3cc92f0 msgr2=0x55f5c4094df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.394+0000 7f15295a0640 1 --2- 172.21.15.100:0/2011320317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5c3cc92f0 0x55f5c4094df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.394+0000 7f15295a0640 1 -- 172.21.15.100:0/2011320317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5c3cd14d0 msgr2=0x55f5c4095330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:57.402 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.394+0000 7f15295a0640 1 --2- 172.21.15.100:0/2011320317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5c3cd14d0 0x55f5c4095330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.403 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.394+0000 7f15295a0640 1 -- 172.21.15.100:0/2011320317 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5c3fa09a0 con 0x55f5c3cf21b0 2024-07-26T13:41:57.403 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.394+0000 7f1528d9f640 1 --2- 172.21.15.100:0/2011320317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5c3cc92f0 0x55f5c4094df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:41:57.403 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.394+0000 7f1523fff640 1 --2- 172.21.15.100:0/2011320317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5c3cd14d0 0x55f5c4095330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:41:57.403 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.394+0000 7f1521ffb640 1 -- 172.21.15.100:0/1421476068 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f15181048c0 con 0x7f1524101580 2024-07-26T13:41:57.403 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.394+0000 7f15295a0640 1 --2- 172.21.15.100:0/2011320317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f5c3cf21b0 0x55f5c40948b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f151c009e40 tx=0x7f151c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caffdd3f9c30d392 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:57.403 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.394+0000 7f15295a0640 1 --2- 172.21.15.100:0/1421476068 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f15140b3dc0 0x7f15140b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:57.403 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.395+0000 7f150effd640 1 -- 172.21.15.100:0/2011320317 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c01a680 con 0x55f5c3cf21b0 2024-07-26T13:41:57.403 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.395+0000 7f150effd640 1 -- 172.21.15.100:0/2011320317 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f151c011440 con 0x55f5c3cf21b0 2024-07-26T13:41:57.403 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.395+0000 7f150effd640 1 -- 172.21.15.100:0/2011320317 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c019600 con 0x55f5c3cf21b0 2024-07-26T13:41:57.403 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.395+0000 7f1531ff1740 1 -- 172.21.15.100:0/2011320317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f5c3cf21b0 msgr2=0x55f5c40948b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:57.403 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.395+0000 7f15295a0640 1 --2- 172.21.15.100:0/1421476068 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f15140b3dc0 0x7f15140b62d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f151c009fd0 tx=0x7f151c019c90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:57.403 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.395+0000 7f1531ff1740 1 --2- 172.21.15.100:0/2011320317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f5c3cf21b0 0x55f5c40948b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f151c009e40 tx=0x7f151c00f730 comp rx=0 tx=0).stop 2024-07-26T13:41:57.403 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.396+0000 7f1531ff1740 1 -- 172.21.15.100:0/2011320317 shutdown_connections 2024-07-26T13:41:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.396+0000 7f1531ff1740 1 --2- 172.21.15.100:0/2011320317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5c3cd14d0 0x55f5c4095330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.396+0000 7f1531ff1740 1 --2- 172.21.15.100:0/2011320317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5c3cc92f0 0x55f5c4094df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.396+0000 7f1531ff1740 1 --2- 172.21.15.100:0/2011320317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f5c3cf21b0 0x55f5c40948b0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.396+0000 7f1531ff1740 1 -- 172.21.15.100:0/2011320317 >> 172.21.15.100:0/2011320317 conn(0x55f5c3ce6300 msgr2=0x55f5c40927a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.396+0000 7f1531ff1740 1 -- 172.21.15.100:0/2011320317 shutdown_connections 2024-07-26T13:41:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.396+0000 7f1531ff1740 1 -- 172.21.15.100:0/2011320317 wait complete. 2024-07-26T13:41:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.396+0000 7f1531ff1740 1 -- start start 2024-07-26T13:41:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.396+0000 7f1531ff1740 1 Processor -- start 2024-07-26T13:41:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.396+0000 7f1531ff1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5c3cc92f0 0x55f5c40a0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.396+0000 7f1531ff1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5c3cd14d0 0x55f5c40a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.396+0000 7f1531ff1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f5c3cf21b0 0x55f5c40ac320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.396+0000 7f1531ff1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55f5c3f33ab0 con 0x55f5c3cf21b0 2024-07-26T13:41:57.404 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.396+0000 7f1531ff1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f5c3f81cf0 con 0x55f5c3cc92f0 2024-07-26T13:41:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.397+0000 7f1531ff1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f5c40a3030 con 0x55f5c3cd14d0 2024-07-26T13:41:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.397+0000 7f1531ff1740 10 client.0 _ll_register_callbacks cb 0x55f5c3ff33e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:41:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.397+0000 7f15295a0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f5c3cf21b0 0x55f5c40ac320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.397+0000 7f15295a0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f5c3cf21b0 0x55f5c40ac320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46500/0 (socket says 172.21.15.100:46500) 2024-07-26T13:41:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.397+0000 7f15295a0640 1 -- 172.21.15.100:0/1305988205 learned_addr learned my addr 172.21.15.100:0/1305988205 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.397+0000 7f1523fff640 1 --2- 172.21.15.100:0/1305988205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5c3cd14d0 0x55f5c40a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.397+0000 7f1528d9f640 1 --2- 172.21.15.100:0/1305988205 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5c3cc92f0 0x55f5c40a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.397+0000 7f15295a0640 1 -- 172.21.15.100:0/1305988205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5c3cd14d0 msgr2=0x55f5c40a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.397+0000 7f15295a0640 1 --2- 172.21.15.100:0/1305988205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5c3cd14d0 0x55f5c40a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.397+0000 7f15295a0640 1 -- 172.21.15.100:0/1305988205 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5c3cc92f0 msgr2=0x55f5c40a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.397+0000 7f15295a0640 1 --2- 172.21.15.100:0/1305988205 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5c3cc92f0 0x55f5c40a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.397+0000 7f15295a0640 1 -- 172.21.15.100:0/1305988205 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5c3d5d790 con 0x55f5c3cf21b0 2024-07-26T13:41:57.405 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.397+0000 7f1523fff640 1 --2- 172.21.15.100:0/1305988205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5c3cd14d0 0x55f5c40a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:41:57.406 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.397+0000 7f1528d9f640 1 --2- 172.21.15.100:0/1305988205 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5c3cc92f0 0x55f5c40a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:41:57.406 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.397+0000 7f15295a0640 1 --2- 172.21.15.100:0/1305988205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f5c3cf21b0 0x55f5c40ac320 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f151c011f10 tx=0x7f151c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ba98b5d018c0b1 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:57.406 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.397+0000 7f1503fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:41:57.406 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.397+0000 7f1503fff640 1 -- 172.21.15.100:0/1305988205 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151c035060 con 0x55f5c3cf21b0 2024-07-26T13:41:57.406 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.397+0000 7f1503fff640 1 -- 172.21.15.100:0/1305988205 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f151c0115f0 con 0x55f5c3cf21b0 2024-07-26T13:41:57.406 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.397+0000 7f1531ff1740 1 -- 172.21.15.100:0/1305988205 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f151c011440 con 0x55f5c3cf21b0 2024-07-26T13:41:57.406 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.397+0000 7f1503fff640 1 -- 172.21.15.100:0/1305988205 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f151c02c040 con 0x55f5c3cf21b0 2024-07-26T13:41:57.406 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.397+0000 7f1531ff1740 10 client.15963 fetch_fsmap learned FSMap version 27 2024-07-26T13:41:57.406 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.397+0000 7f1531ff1740 1 -- 172.21.15.100:0/1305988205 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f5c3dd6620 con 0x55f5c3cf21b0 2024-07-26T13:41:57.406 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.398+0000 7f1503fff640 1 -- 172.21.15.100:0/1305988205 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f151c011440 con 0x55f5c3cf21b0 2024-07-26T13:41:57.406 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.398+0000 7f1503fff640 1 -- 172.21.15.100:0/1305988205 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f151c010930 con 0x55f5c3cf21b0 2024-07-26T13:41:57.406 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.398+0000 7f1531ff1740 10 client.15963 fetch_fsmap finished waiting for FSMap version 27 2024-07-26T13:41:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.398+0000 7f1531ff1740 10 client.15963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:41:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.398+0000 7f1531ff1740 20 client.15963 populate_metadata read hostname 'smithi100' 2024-07-26T13:41:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.398+0000 7f1531ff1740 1 --2- 172.21.15.100:0/1305988205 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f5c3cb0d80 0x55f5c40b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.398+0000 7f1531ff1740 4 client.15963 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:41:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.398+0000 7f1531ff1740 1 -- 172.21.15.100:0/1305988205 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5c400c300 con 0x55f5c3cb0d80 2024-07-26T13:41:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.398+0000 7f1528d9f640 1 --2- 172.21.15.100:0/1305988205 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f5c3cb0d80 0x55f5c40b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.399+0000 7f1503fff640 10 client.15963 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.399+0000 7f1528d9f640 1 --2- 172.21.15.100:0/1305988205 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f5c3cb0d80 0x55f5c40b21c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e238e5cd020e0e71 server_cookie=e9f86115bb80636 in_seq=0 out_seq=0 2024-07-26T13:41:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.400+0000 7f1503fff640 1 -- 172.21.15.100:0/1305988205 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f5c400c300 con 0x55f5c3cb0d80 2024-07-26T13:41:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.400+0000 7f1503fff640 10 client.15963 handle_command_reply: tid=0 2024-07-26T13:41:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.400+0000 7f1531ff1740 1 client.15963 shutdown 2024-07-26T13:41:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.400+0000 7f1531ff1740 1 -- 172.21.15.100:0/1305988205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f5c3cf21b0 msgr2=0x55f5c40ac320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:57.407 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.400+0000 7f1531ff1740 1 --2- 172.21.15.100:0/1305988205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f5c3cf21b0 0x55f5c40ac320 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f151c011f10 tx=0x7f151c01a000 comp rx=0 tx=0).stop 2024-07-26T13:41:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.400+0000 7f1531ff1740 1 -- 172.21.15.100:0/1305988205 shutdown_connections 2024-07-26T13:41:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.400+0000 7f1531ff1740 1 --2- 172.21.15.100:0/1305988205 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f5c3cb0d80 0x55f5c40b21c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.400+0000 7f1531ff1740 1 --2- 172.21.15.100:0/1305988205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f5c3cf21b0 0x55f5c40ac320 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.400+0000 7f1531ff1740 1 --2- 172.21.15.100:0/1305988205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5c3cd14d0 0x55f5c40a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.400+0000 7f1503fff640 0 client.15963 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.400+0000 7f1531ff1740 1 --2- 172.21.15.100:0/1305988205 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5c3cc92f0 0x55f5c40a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.400+0000 7f1531ff1740 1 -- 172.21.15.100:0/1305988205 >> 172.21.15.100:0/1305988205 conn(0x55f5c3ce6300 msgr2=0x55f5c40958e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.400+0000 7f1531ff1740 1 -- 172.21.15.100:0/1305988205 shutdown_connections 2024-07-26T13:41:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.400+0000 7f1531ff1740 1 -- 172.21.15.100:0/1305988205 wait complete. 2024-07-26T13:41:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.400+0000 7f1531ff1740 20 client.15963 trim_cache size 0 max 16384 2024-07-26T13:41:57.409 INFO:tasks.workunit.client.0.smithi100.stdout: 1 20252 11.30 MB/sec warmup 11 sec latency 4722.859 ms 2024-07-26T13:41:57.421 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.415+0000 7f1531ff1740 1 Processor -- start 2024-07-26T13:41:57.421 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.415+0000 7f1531ff1740 1 -- start start 2024-07-26T13:41:57.421 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.415+0000 7f1531ff1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f5c3cc92f0 0x55f5c409a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:57.421 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.415+0000 7f1531ff1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5c3cd14d0 0x55f5c40ac110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:57.421 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.415+0000 7f1531ff1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5c3cf21b0 0x55f5c40ac650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:57.421 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.415+0000 7f1531ff1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f15241b3a80 con 0x55f5c3cc92f0 2024-07-26T13:41:57.421 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.415+0000 7f1531ff1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f15241b1350 con 0x55f5c3cd14d0 2024-07-26T13:41:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.415+0000 7f1531ff1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f15241af1a0 con 0x55f5c3cf21b0 2024-07-26T13:41:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.415+0000 7f1528d9f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f5c3cc92f0 0x55f5c409a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.415+0000 7f1528d9f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f5c3cc92f0 0x55f5c409a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46508/0 (socket says 172.21.15.100:46508) 2024-07-26T13:41:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.415+0000 7f1528d9f640 1 -- 172.21.15.100:0/1859473635 learned_addr learned my addr 172.21.15.100:0/1859473635 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.416+0000 7f1523fff640 1 --2- 172.21.15.100:0/1859473635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5c3cd14d0 0x55f5c40ac110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.416+0000 7f15295a0640 1 --2- 172.21.15.100:0/1859473635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5c3cf21b0 0x55f5c40ac650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.416+0000 7f1528d9f640 1 -- 172.21.15.100:0/1859473635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5c3cf21b0 msgr2=0x55f5c40ac650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.416+0000 7f1528d9f640 1 --2- 172.21.15.100:0/1859473635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5c3cf21b0 0x55f5c40ac650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.416+0000 7f1528d9f640 1 -- 172.21.15.100:0/1859473635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5c3cd14d0 msgr2=0x55f5c40ac110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.416+0000 7f1528d9f640 1 --2- 172.21.15.100:0/1859473635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5c3cd14d0 0x55f5c40ac110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.416+0000 7f1528d9f640 1 -- 172.21.15.100:0/1859473635 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5c40970c0 con 0x55f5c3cc92f0 2024-07-26T13:41:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.416+0000 7f1523fff640 1 --2- 172.21.15.100:0/1859473635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5c3cd14d0 0x55f5c40ac110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:41:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.416+0000 7f15295a0640 1 --2- 172.21.15.100:0/1859473635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5c3cf21b0 0x55f5c40ac650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:41:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.416+0000 7f1528d9f640 1 --2- 172.21.15.100:0/1859473635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f5c3cc92f0 0x55f5c409a980 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f1518004650 tx=0x7f15180045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a67f334d382d63f server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.416+0000 7f150e7fc640 1 -- 172.21.15.100:0/1859473635 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1518049040 con 0x55f5c3cc92f0 2024-07-26T13:41:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.416+0000 7f150e7fc640 1 -- 172.21.15.100:0/1859473635 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15180cb000 con 0x55f5c3cc92f0 2024-07-26T13:41:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.416+0000 7f150e7fc640 1 -- 172.21.15.100:0/1859473635 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15181022b0 con 0x55f5c3cc92f0 2024-07-26T13:41:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.417+0000 7f1531ff1740 1 -- 172.21.15.100:0/1859473635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f5c3cc92f0 msgr2=0x55f5c409a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.417+0000 7f1531ff1740 1 --2- 172.21.15.100:0/1859473635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f5c3cc92f0 0x55f5c409a980 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f1518004650 tx=0x7f15180045a0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.417+0000 7f1531ff1740 1 -- 172.21.15.100:0/1859473635 shutdown_connections 2024-07-26T13:41:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.417+0000 7f1531ff1740 1 --2- 172.21.15.100:0/1859473635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5c3cf21b0 0x55f5c40ac650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.417+0000 7f1531ff1740 1 --2- 172.21.15.100:0/1859473635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5c3cd14d0 0x55f5c40ac110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.417+0000 7f1531ff1740 1 --2- 172.21.15.100:0/1859473635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f5c3cc92f0 0x55f5c409a980 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.417+0000 7f1531ff1740 1 -- 172.21.15.100:0/1859473635 >> 172.21.15.100:0/1859473635 conn(0x55f5c3ce6300 msgr2=0x55f5c409eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.417+0000 7f1531ff1740 1 -- 172.21.15.100:0/1859473635 shutdown_connections 2024-07-26T13:41:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.417+0000 7f1531ff1740 1 -- 172.21.15.100:0/1859473635 wait complete. 2024-07-26T13:41:57.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.417+0000 7f1531ff1740 1 -- start start 2024-07-26T13:41:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.417+0000 7f1531ff1740 1 Processor -- start 2024-07-26T13:41:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.418+0000 7f1531ff1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f5c3cc92f0 0x55f5c40a87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.418+0000 7f1531ff1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5c3cd14d0 0x55f5c40a5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.418+0000 7f1531ff1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5c3cf21b0 0x55f5c40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.418+0000 7f1531ff1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f15241af810 con 0x55f5c3cc92f0 2024-07-26T13:41:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.418+0000 7f1531ff1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f15241b0c80 con 0x55f5c3cf21b0 2024-07-26T13:41:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.418+0000 7f1531ff1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f15241afee0 con 0x55f5c3cd14d0 2024-07-26T13:41:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.418+0000 7f1531ff1740 10 client.0 _ll_register_callbacks cb 0x7f151c029040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:41:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.418+0000 7f1528d9f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f5c3cc92f0 0x55f5c40a87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.418+0000 7f1528d9f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f5c3cc92f0 0x55f5c40a87a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46522/0 (socket says 172.21.15.100:46522) 2024-07-26T13:41:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.418+0000 7f1528d9f640 1 -- 172.21.15.100:0/3288423227 learned_addr learned my addr 172.21.15.100:0/3288423227 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:57.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.418+0000 7f1523fff640 1 --2- 172.21.15.100:0/3288423227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5c3cd14d0 0x55f5c40a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.418+0000 7f15295a0640 1 --2- 172.21.15.100:0/3288423227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5c3cf21b0 0x55f5c40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.418+0000 7f1528d9f640 1 -- 172.21.15.100:0/3288423227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5c3cd14d0 msgr2=0x55f5c40a5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.418+0000 7f1528d9f640 1 --2- 172.21.15.100:0/3288423227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5c3cd14d0 0x55f5c40a5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.418+0000 7f1528d9f640 1 -- 172.21.15.100:0/3288423227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5c3cf21b0 msgr2=0x55f5c40a5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:41:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.418+0000 7f1528d9f640 1 --2- 172.21.15.100:0/3288423227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5c3cf21b0 0x55f5c40a5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.418+0000 7f1528d9f640 1 -- 172.21.15.100:0/3288423227 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5c40a5d50 con 0x55f5c3cc92f0 2024-07-26T13:41:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.418+0000 7f15295a0640 1 --2- 172.21.15.100:0/3288423227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5c3cf21b0 0x55f5c40a5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:41:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.418+0000 7f1503fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:41:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.418+0000 7f1528d9f640 1 --2- 172.21.15.100:0/3288423227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f5c3cc92f0 0x55f5c40a87a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1518006010 tx=0x7f1518104c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a976231cf020b866 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.418+0000 7f1503fff640 1 -- 172.21.15.100:0/3288423227 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1518049050 con 0x55f5c3cc92f0 2024-07-26T13:41:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.418+0000 7f1503fff640 1 -- 172.21.15.100:0/3288423227 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f15181022b0 con 0x55f5c3cc92f0 2024-07-26T13:41:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.418+0000 7f1531ff1740 1 -- 172.21.15.100:0/3288423227 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15180cb000 con 0x55f5c3cc92f0 2024-07-26T13:41:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.419+0000 7f1503fff640 1 -- 172.21.15.100:0/3288423227 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f15180cfaf0 con 0x55f5c3cc92f0 2024-07-26T13:41:57.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.419+0000 7f1531ff1740 10 client.15969 fetch_fsmap learned FSMap version 27 2024-07-26T13:41:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.419+0000 7f1531ff1740 1 -- 172.21.15.100:0/3288423227 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f5c41193f0 con 0x55f5c3cc92f0 2024-07-26T13:41:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.419+0000 7f1503fff640 1 -- 172.21.15.100:0/3288423227 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15180cb000 con 0x55f5c3cc92f0 2024-07-26T13:41:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.419+0000 7f1503fff640 1 -- 172.21.15.100:0/3288423227 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f15180cb750 con 0x55f5c3cc92f0 2024-07-26T13:41:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.419+0000 7f1531ff1740 10 client.15969 fetch_fsmap finished waiting for FSMap version 27 2024-07-26T13:41:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.419+0000 7f1531ff1740 10 client.15969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:41:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.419+0000 7f1531ff1740 20 client.15969 populate_metadata read hostname 'smithi100' 2024-07-26T13:41:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.419+0000 7f1531ff1740 1 --2- 172.21.15.100:0/3288423227 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f5c3c861c0 0x55f5c411bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.419+0000 7f1531ff1740 4 client.15969 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-26T13:41:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.419+0000 7f1531ff1740 1 -- 172.21.15.100:0/3288423227 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f5c4007da0 con 0x55f5c3c861c0 2024-07-26T13:41:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.419+0000 7f1523fff640 1 --2- 172.21.15.100:0/3288423227 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f5c3c861c0 0x55f5c411bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.420+0000 7f1503fff640 10 client.15969 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:57.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.420+0000 7f1523fff640 1 --2- 172.21.15.100:0/3288423227 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f5c3c861c0 0x55f5c411bb60 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdf1bb069a97750d server_cookie=b5375d9ab6ea26e1 in_seq=0 out_seq=0 2024-07-26T13:41:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.505+0000 7f1503fff640 1 -- 172.21.15.100:0/3288423227 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f5c4007da0 con 0x55f5c3c861c0 2024-07-26T13:41:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.505+0000 7f1503fff640 10 client.15969 handle_command_reply: tid=0 2024-07-26T13:41:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.505+0000 7f1531ff1740 1 client.15969 shutdown 2024-07-26T13:41:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.505+0000 7f1531ff1740 1 -- 172.21.15.100:0/3288423227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f5c3cc92f0 msgr2=0x55f5c40a87a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.505+0000 7f1531ff1740 1 --2- 172.21.15.100:0/3288423227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f5c3cc92f0 0x55f5c40a87a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1518006010 tx=0x7f1518104c50 comp rx=0 tx=0).stop 2024-07-26T13:41:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.505+0000 7f1531ff1740 1 -- 172.21.15.100:0/3288423227 shutdown_connections 2024-07-26T13:41:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.505+0000 7f1531ff1740 1 --2- 172.21.15.100:0/3288423227 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f5c3c861c0 0x55f5c411bb60 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.505+0000 7f1531ff1740 1 --2- 172.21.15.100:0/3288423227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5c3cf21b0 0x55f5c40a5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.505+0000 7f1503fff640 0 client.15969 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:41:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.505+0000 7f1531ff1740 1 --2- 172.21.15.100:0/3288423227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f5c3cd14d0 0x55f5c40a5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.505+0000 7f1531ff1740 1 --2- 172.21.15.100:0/3288423227 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f5c3cc92f0 0x55f5c40a87a0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.505+0000 7f1531ff1740 1 -- 172.21.15.100:0/3288423227 >> 172.21.15.100:0/3288423227 conn(0x55f5c3ce6300 msgr2=0x55f5c40958e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.506+0000 7f1531ff1740 1 -- 172.21.15.100:0/3288423227 shutdown_connections 2024-07-26T13:41:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.506+0000 7f1531ff1740 1 -- 172.21.15.100:0/3288423227 wait complete. 2024-07-26T13:41:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.506+0000 7f1531ff1740 20 client.15969 trim_cache size 0 max 16384 2024-07-26T13:41:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-07-26T13:41:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-07-26T13:41:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-07-26T13:41:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-07-26T13:41:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.506+0000 7f1529da1640 1 -- 172.21.15.100:0/1421476068 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f15140b3dc0 msgr2=0x7f15140b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.506+0000 7f1529da1640 1 --2- 172.21.15.100:0/1421476068 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f15140b3dc0 0x7f15140b62d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f151c009fd0 tx=0x7f151c019c90 comp rx=0 tx=0).stop 2024-07-26T13:41:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.506+0000 7f1529da1640 1 -- 172.21.15.100:0/1421476068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1524101580 msgr2=0x7f152419bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.506+0000 7f1529da1640 1 --2- 172.21.15.100:0/1421476068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1524101580 0x7f152419bee0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1518005940 tx=0x7f15180042f0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.506+0000 7f1529da1640 1 -- 172.21.15.100:0/1421476068 shutdown_connections 2024-07-26T13:41:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.506+0000 7f1529da1640 1 --2- 172.21.15.100:0/1421476068 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f15140b3dc0 0x7f15140b62d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.506+0000 7f1529da1640 1 --2- 172.21.15.100:0/1421476068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1524103e30 0x7f15241a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.506+0000 7f1529da1640 1 --2- 172.21.15.100:0/1421476068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1524102050 0x7f152419c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.506+0000 7f1529da1640 1 --2- 172.21.15.100:0/1421476068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1524101580 0x7f152419bee0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.506+0000 7f1529da1640 1 -- 172.21.15.100:0/1421476068 >> 172.21.15.100:0/1421476068 conn(0x7f1524079530 msgr2=0x7f1524100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.506+0000 7f1529da1640 1 -- 172.21.15.100:0/1421476068 shutdown_connections 2024-07-26T13:41:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:41:57.506+0000 7f1529da1640 1 -- 172.21.15.100:0/1421476068 wait complete. 2024-07-26T13:41:57.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.560+0000 7f3e1bb3f640 1 -- 172.21.15.100:0/2938006223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e14109840 msgr2=0x7f3e14110070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:57.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.560+0000 7f3e1bb3f640 1 --2- 172.21.15.100:0/2938006223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e14109840 0x7f3e14110070 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009ca0 tx=0x7f3e0c02a8c0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.560+0000 7f3e1bb3f640 1 -- 172.21.15.100:0/2938006223 shutdown_connections 2024-07-26T13:41:57.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.560+0000 7f3e1bb3f640 1 --2- 172.21.15.100:0/2938006223 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e14109840 0x7f3e14110070 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.560+0000 7f3e1bb3f640 1 --2- 172.21.15.100:0/2938006223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e14108ef0 0x7f3e14109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.560+0000 7f3e1bb3f640 1 --2- 172.21.15.100:0/2938006223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e141035b0 0x7f3e141039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.560+0000 7f3e1bb3f640 1 -- 172.21.15.100:0/2938006223 >> 172.21.15.100:0/2938006223 conn(0x7f3e140ff000 msgr2=0x7f3e14101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:57.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.560+0000 7f3e1bb3f640 1 -- 172.21.15.100:0/2938006223 shutdown_connections 2024-07-26T13:41:57.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.560+0000 7f3e1bb3f640 1 -- 172.21.15.100:0/2938006223 wait complete. 2024-07-26T13:41:57.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.561+0000 7f3e1bb3f640 1 Processor -- start 2024-07-26T13:41:57.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.561+0000 7f3e1bb3f640 1 -- start start 2024-07-26T13:41:57.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.561+0000 7f3e1bb3f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e141035b0 0x7f3e14199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:57.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.561+0000 7f3e1bb3f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e14108ef0 0x7f3e1419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:57.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.561+0000 7f3e1bb3f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e14109840 0x7f3e1419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:57.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.561+0000 7f3e1bb3f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3e14113510 con 0x7f3e14108ef0 2024-07-26T13:41:57.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.561+0000 7f3e1bb3f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3e14113390 con 0x7f3e14109840 2024-07-26T13:41:57.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.561+0000 7f3e1bb3f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3e14113690 con 0x7f3e141035b0 2024-07-26T13:41:57.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.561+0000 7f3e190da640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e14108ef0 0x7f3e1419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:57.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.562+0000 7f3e190da640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e14108ef0 0x7f3e1419a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46530/0 (socket says 172.21.15.100:46530) 2024-07-26T13:41:57.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.562+0000 7f3e190da640 1 -- 172.21.15.100:0/3117481374 learned_addr learned my addr 172.21.15.100:0/3117481374 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:41:57.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.562+0000 7f3e198db640 1 --2- 172.21.15.100:0/3117481374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e141035b0 0x7f3e14199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:57.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.562+0000 7f3e1a0dc640 1 --2- 172.21.15.100:0/3117481374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e14109840 0x7f3e1419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:57.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.562+0000 7f3e190da640 1 -- 172.21.15.100:0/3117481374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e141035b0 msgr2=0x7f3e14199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:57.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.562+0000 7f3e190da640 1 --2- 172.21.15.100:0/3117481374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e141035b0 0x7f3e14199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.562+0000 7f3e190da640 1 -- 172.21.15.100:0/3117481374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e14109840 msgr2=0x7f3e1419f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:57.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.562+0000 7f3e190da640 1 --2- 172.21.15.100:0/3117481374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e14109840 0x7f3e1419f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.562+0000 7f3e190da640 1 -- 172.21.15.100:0/3117481374 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e1419fd50 con 0x7f3e14108ef0 2024-07-26T13:41:57.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.562+0000 7f3e190da640 1 --2- 172.21.15.100:0/3117481374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e14108ef0 0x7f3e1419a440 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3e1000c8c0 tx=0x7f3e1000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:57.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.562+0000 7f3e0affd640 1 -- 172.21.15.100:0/3117481374 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e10007bb0 con 0x7f3e14108ef0 2024-07-26T13:41:57.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.562+0000 7f3e0affd640 1 -- 172.21.15.100:0/3117481374 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e10007d50 con 0x7f3e14108ef0 2024-07-26T13:41:57.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.562+0000 7f3e1bb3f640 1 -- 172.21.15.100:0/3117481374 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e141a0040 con 0x7f3e14108ef0 2024-07-26T13:41:57.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.563+0000 7f3e0affd640 1 -- 172.21.15.100:0/3117481374 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e100187c0 con 0x7f3e14108ef0 2024-07-26T13:41:57.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.563+0000 7f3e1bb3f640 1 -- 172.21.15.100:0/3117481374 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e141a6860 con 0x7f3e14108ef0 2024-07-26T13:41:57.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.563+0000 7f3e1bb3f640 1 -- 172.21.15.100:0/3117481374 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e141039c0 con 0x7f3e14108ef0 2024-07-26T13:41:57.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.565+0000 7f3e0affd640 1 -- 172.21.15.100:0/3117481374 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f3e1000fa20 con 0x7f3e14108ef0 2024-07-26T13:41:57.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.565+0000 7f3e0affd640 1 --2- 172.21.15.100:0/3117481374 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3e000b3d70 0x7f3e000b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:41:57.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.566+0000 7f3e198db640 1 --2- 172.21.15.100:0/3117481374 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3e000b3d70 0x7f3e000b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:41:57.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.566+0000 7f3e0affd640 1 -- 172.21.15.100:0/3117481374 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f3e100e4000 con 0x7f3e14108ef0 2024-07-26T13:41:57.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.566+0000 7f3e198db640 1 --2- 172.21.15.100:0/3117481374 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3e000b3d70 0x7f3e000b6280 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3e04009bd0 tx=0x7f3e04005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:41:57.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.570+0000 7f3e0affd640 1 -- 172.21.15.100:0/3117481374 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f3e10014030 con 0x7f3e14108ef0 2024-07-26T13:41:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:57 smithi100 ceph-mon[32599]: pgmap v419: 97 pgs: 97 active+clean; 14 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 1.3 MiB/s wr, 114 op/s 2024-07-26T13:41:57.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:57 smithi167 ceph-mon[37140]: pgmap v419: 97 pgs: 97 active+clean; 14 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 1.3 MiB/s wr, 114 op/s 2024-07-26T13:41:57.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.782+0000 7f3e1bb3f640 1 -- 172.21.15.100:0/3117481374 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e1410b0a0 con 0x7f3e14108ef0 2024-07-26T13:41:57.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.783+0000 7f3e0affd640 1 -- 172.21.15.100:0/3117481374 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11930 (secure 0 0 0) 0x7f3e100ab330 con 0x7f3e14108ef0 2024-07-26T13:41:57.784 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:41:57.784 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":27,"btime":"2024-07-26T13:41:55:493454+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14790,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/313501409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":313501409},{"type":"v1","addr":"172.21.15.100:6839","nonce":313501409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/1402975506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":1402975506},{"type":"v1","addr":"172.21.15.100:6841","nonce":1402975506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-26T13:37:01.281843+0000","modified":"2024-07-26T13:41:54.710630+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24607,"mds_1":24605,"mds_2":24595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24557":{"gid":24557,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/3290105300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3290105300},{"type":"v1","addr":"172.21.15.138:6835","nonce":3290105300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/3393225483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3393225483},{"type":"v1","addr":"172.21.15.138:6837","nonce":3393225483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24595":{"gid":24595,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6837/3828634468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3828634468},{"type":"v1","addr":"172.21.15.167:6837","nonce":3828634468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.138:6839/1580925471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1580925471},{"type":"v1","addr":"172.21.15.138:6839","nonce":1580925471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/398150514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":398150514},{"type":"v1","addr":"172.21.15.167:6839","nonce":398150514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24607,"qdb_cluster":[24607,24605,24595]},"id":1}]} 2024-07-26T13:41:57.784 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 27 2024-07-26T13:41:57.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.785+0000 7f3e1bb3f640 1 -- 172.21.15.100:0/3117481374 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3e000b3d70 msgr2=0x7f3e000b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:57.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.785+0000 7f3e1bb3f640 1 --2- 172.21.15.100:0/3117481374 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3e000b3d70 0x7f3e000b6280 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3e04009bd0 tx=0x7f3e04005f90 comp rx=0 tx=0).stop 2024-07-26T13:41:57.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.785+0000 7f3e1bb3f640 1 -- 172.21.15.100:0/3117481374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e14108ef0 msgr2=0x7f3e1419a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:41:57.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.785+0000 7f3e1bb3f640 1 --2- 172.21.15.100:0/3117481374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e14108ef0 0x7f3e1419a440 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3e1000c8c0 tx=0x7f3e1000cd80 comp rx=0 tx=0).stop 2024-07-26T13:41:57.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.785+0000 7f3e1bb3f640 1 -- 172.21.15.100:0/3117481374 shutdown_connections 2024-07-26T13:41:57.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.785+0000 7f3e1bb3f640 1 --2- 172.21.15.100:0/3117481374 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3e000b3d70 0x7f3e000b6280 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.785+0000 7f3e1bb3f640 1 --2- 172.21.15.100:0/3117481374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e14109840 0x7f3e1419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.785+0000 7f3e1bb3f640 1 --2- 172.21.15.100:0/3117481374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3e14108ef0 0x7f3e1419a440 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.785+0000 7f3e1bb3f640 1 --2- 172.21.15.100:0/3117481374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e141035b0 0x7f3e14199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:41:57.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.785+0000 7f3e1bb3f640 1 -- 172.21.15.100:0/3117481374 >> 172.21.15.100:0/3117481374 conn(0x7f3e140ff000 msgr2=0x7f3e14101420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:41:57.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.785+0000 7f3e1bb3f640 1 -- 172.21.15.100:0/3117481374 shutdown_connections 2024-07-26T13:41:57.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:41:57.785+0000 7f3e1bb3f640 1 -- 172.21.15.100:0/3117481374 wait complete. 2024-07-26T13:41:57.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:57 smithi138 ceph-mon[36954]: pgmap v419: 97 pgs: 97 active+clean; 14 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 1.3 MiB/s wr, 114 op/s 2024-07-26T13:41:58.403 INFO:tasks.workunit.client.0.smithi100.stdout: 1 20252 10.36 MB/sec warmup 12 sec latency 5722.928 ms 2024-07-26T13:41:58.675 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-26T13:41:58.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:58 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/3117481374' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:41:58.932 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:41:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:58 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/3117481374' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:41:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:58 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/3117481374' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:41:59.403 INFO:tasks.workunit.client.0.smithi100.stdout: 1 20252 9.56 MB/sec warmup 13 sec latency 6722.992 ms 2024-07-26T13:41:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:59 smithi100 ceph-mon[32599]: pgmap v420: 97 pgs: 97 active+clean; 16 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-07-26T13:41:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:41:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:41:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:59 smithi167 ceph-mon[37140]: pgmap v420: 97 pgs: 97 active+clean; 16 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-07-26T13:41:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:41:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:41:59.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:59 smithi138 ceph-mon[36954]: pgmap v420: 97 pgs: 97 active+clean; 16 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-07-26T13:41:59.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:41:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:42:00.403 INFO:tasks.workunit.client.0.smithi100.stdout: 1 20252 8.88 MB/sec warmup 14 sec latency 7723.017 ms 2024-07-26T13:42:00.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.666+0000 7f10dca85640 1 -- 172.21.15.100:0/1115738887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10d8108ed0 msgr2=0x7f10d81092e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:00.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.666+0000 7f10dca85640 1 --2- 172.21.15.100:0/1115738887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10d8108ed0 0x7f10d81092e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f10cc009900 tx=0x7f10cc02a410 comp rx=0 tx=0).stop 2024-07-26T13:42:00.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.666+0000 7f10dca85640 1 -- 172.21.15.100:0/1115738887 shutdown_connections 2024-07-26T13:42:00.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.666+0000 7f10dca85640 1 --2- 172.21.15.100:0/1115738887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10d8109820 0x7f10d8110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:00.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.666+0000 7f10dca85640 1 --2- 172.21.15.100:0/1115738887 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10d8108ed0 0x7f10d81092e0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:00.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.666+0000 7f10dca85640 1 --2- 172.21.15.100:0/1115738887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10d8103590 0x7f10d81039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:00.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.666+0000 7f10dca85640 1 -- 172.21.15.100:0/1115738887 >> 172.21.15.100:0/1115738887 conn(0x7f10d807b280 msgr2=0x7f10d807b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:00.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.666+0000 7f10dca85640 1 -- 172.21.15.100:0/1115738887 shutdown_connections 2024-07-26T13:42:00.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.666+0000 7f10dca85640 1 -- 172.21.15.100:0/1115738887 wait complete. 2024-07-26T13:42:00.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.667+0000 7f10dca85640 1 Processor -- start 2024-07-26T13:42:00.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.667+0000 7f10dca85640 1 -- start start 2024-07-26T13:42:00.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.667+0000 7f10dca85640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10d8103590 0x7f10d8075140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:00.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.667+0000 7f10dca85640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10d8108ed0 0x7f10d8075680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:00.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.667+0000 7f10dca85640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10d8109820 0x7f10d8075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:00.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.667+0000 7f10dca85640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f10d8113490 con 0x7f10d8103590 2024-07-26T13:42:00.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.667+0000 7f10dca85640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f10d8113310 con 0x7f10d8108ed0 2024-07-26T13:42:00.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.667+0000 7f10dca85640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f10d8113610 con 0x7f10d8109820 2024-07-26T13:42:00.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.667+0000 7f10d659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10d8103590 0x7f10d8075140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:00.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.667+0000 7f10d659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10d8103590 0x7f10d8075140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46544/0 (socket says 172.21.15.100:46544) 2024-07-26T13:42:00.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.667+0000 7f10d659c640 1 -- 172.21.15.100:0/4145388200 learned_addr learned my addr 172.21.15.100:0/4145388200 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:00.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.667+0000 7f10d5d9b640 1 --2- 172.21.15.100:0/4145388200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10d8108ed0 0x7f10d8075680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:00.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.667+0000 7f10d6d9d640 1 --2- 172.21.15.100:0/4145388200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10d8109820 0x7f10d8075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:00.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.667+0000 7f10d659c640 1 -- 172.21.15.100:0/4145388200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10d8109820 msgr2=0x7f10d8075bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:00.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.667+0000 7f10d659c640 1 --2- 172.21.15.100:0/4145388200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10d8109820 0x7f10d8075bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:00.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.667+0000 7f10d659c640 1 -- 172.21.15.100:0/4145388200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10d8108ed0 msgr2=0x7f10d8075680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:00.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.667+0000 7f10d659c640 1 --2- 172.21.15.100:0/4145388200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10d8108ed0 0x7f10d8075680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:00.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.667+0000 7f10d659c640 1 -- 172.21.15.100:0/4145388200 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d81a8240 con 0x7f10d8103590 2024-07-26T13:42:00.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.668+0000 7f10d659c640 1 --2- 172.21.15.100:0/4145388200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10d8103590 0x7f10d8075140 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f10c800d950 tx=0x7f10c800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:00.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.668+0000 7f10c77fe640 1 -- 172.21.15.100:0/4145388200 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c8017070 con 0x7f10d8103590 2024-07-26T13:42:00.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.668+0000 7f10dca85640 1 -- 172.21.15.100:0/4145388200 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10d81a8530 con 0x7f10d8103590 2024-07-26T13:42:00.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.668+0000 7f10c77fe640 1 -- 172.21.15.100:0/4145388200 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10c8009c30 con 0x7f10d8103590 2024-07-26T13:42:00.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.668+0000 7f10c77fe640 1 -- 172.21.15.100:0/4145388200 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c800f620 con 0x7f10d8103590 2024-07-26T13:42:00.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.668+0000 7f10dca85640 1 -- 172.21.15.100:0/4145388200 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10d81a8a40 con 0x7f10d8103590 2024-07-26T13:42:00.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.671+0000 7f10e3009740 1 Processor -- start 2024-07-26T13:42:00.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.672+0000 7f10e3009740 1 -- start start 2024-07-26T13:42:00.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.672+0000 7f10c77fe640 1 -- 172.21.15.100:0/4145388200 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f10c8010460 con 0x7f10d8103590 2024-07-26T13:42:00.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.672+0000 7f10e3009740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c45e4ba070 0x55c45e85f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:00.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.672+0000 7f10e3009740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c45e4911b0 0x55c45e85fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:00.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.672+0000 7f10e3009740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c45e499390 0x55c45e85fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:00.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.672+0000 7f10e3009740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c45e7d5bb0 con 0x55c45e4ba070 2024-07-26T13:42:00.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.672+0000 7f10e3009740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c45e7c50b0 con 0x55c45e499390 2024-07-26T13:42:00.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.672+0000 7f10e3009740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c45e7c90f0 con 0x55c45e4911b0 2024-07-26T13:42:00.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.672+0000 7f10d6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c45e4ba070 0x55c45e85f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:00.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.672+0000 7f10d6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c45e4ba070 0x55c45e85f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46556/0 (socket says 172.21.15.100:46556) 2024-07-26T13:42:00.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.672+0000 7f10d6d9d640 1 -- 172.21.15.100:0/3790938053 learned_addr learned my addr 172.21.15.100:0/3790938053 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:00.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.672+0000 7f10d5d9b640 1 --2- 172.21.15.100:0/3790938053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c45e499390 0x55c45e85fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:00.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.672+0000 7f10d659c640 1 --2- 172.21.15.100:0/3790938053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c45e4911b0 0x55c45e85fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:00.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.672+0000 7f10d6d9d640 1 -- 172.21.15.100:0/3790938053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c45e4911b0 msgr2=0x55c45e85fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:00.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.672+0000 7f10d6d9d640 1 --2- 172.21.15.100:0/3790938053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c45e4911b0 0x55c45e85fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:00.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.672+0000 7f10d6d9d640 1 -- 172.21.15.100:0/3790938053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c45e499390 msgr2=0x55c45e85fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:00.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.672+0000 7f10d6d9d640 1 --2- 172.21.15.100:0/3790938053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c45e499390 0x55c45e85fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:00.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.672+0000 7f10d6d9d640 1 -- 172.21.15.100:0/3790938053 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c45e76bb90 con 0x55c45e4ba070 2024-07-26T13:42:00.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.673+0000 7f10c77fe640 1 --2- 172.21.15.100:0/4145388200 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f10c00b3d40 0x7f10c00b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:00.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.673+0000 7f10d6d9d640 1 --2- 172.21.15.100:0/3790938053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c45e4ba070 0x55c45e85f570 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f10d0009f20 tx=0x7f10d000f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a252ed995ba3a6f server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:00.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.673+0000 7f10d659c640 1 --2- 172.21.15.100:0/4145388200 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f10c00b3d40 0x7f10c00b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:00.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.673+0000 7f10c4ff9640 1 -- 172.21.15.100:0/3790938053 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d0015070 con 0x55c45e4ba070 2024-07-26T13:42:00.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.673+0000 7f10c77fe640 1 -- 172.21.15.100:0/4145388200 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f10c80e3c20 con 0x7f10d8103590 2024-07-26T13:42:00.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.673+0000 7f10c4ff9640 1 -- 172.21.15.100:0/3790938053 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10d001ab10 con 0x55c45e4ba070 2024-07-26T13:42:00.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.673+0000 7f10c4ff9640 1 -- 172.21.15.100:0/3790938053 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d0011b60 con 0x55c45e4ba070 2024-07-26T13:42:00.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.674+0000 7f10e3009740 1 -- 172.21.15.100:0/3790938053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c45e4ba070 msgr2=0x55c45e85f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:00.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.674+0000 7f10d659c640 1 --2- 172.21.15.100:0/4145388200 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f10c00b3d40 0x7f10c00b6250 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f10c80e1440 tx=0x7f10c8092040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:00.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.674+0000 7f10e3009740 1 --2- 172.21.15.100:0/3790938053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c45e4ba070 0x55c45e85f570 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f10d0009f20 tx=0x7f10d000f7f0 comp rx=0 tx=0).stop 2024-07-26T13:42:00.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.674+0000 7f10e3009740 1 -- 172.21.15.100:0/3790938053 shutdown_connections 2024-07-26T13:42:00.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.674+0000 7f10e3009740 1 --2- 172.21.15.100:0/3790938053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c45e499390 0x55c45e85fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:00.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.674+0000 7f10e3009740 1 --2- 172.21.15.100:0/3790938053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c45e4911b0 0x55c45e85fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:00.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.674+0000 7f10e3009740 1 --2- 172.21.15.100:0/3790938053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c45e4ba070 0x55c45e85f570 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:00.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.674+0000 7f10e3009740 1 -- 172.21.15.100:0/3790938053 >> 172.21.15.100:0/3790938053 conn(0x55c45e4ae460 msgr2=0x55c45e85d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:00.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.674+0000 7f10e3009740 1 -- 172.21.15.100:0/3790938053 shutdown_connections 2024-07-26T13:42:00.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.674+0000 7f10e3009740 1 -- 172.21.15.100:0/3790938053 wait complete. 2024-07-26T13:42:00.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.674+0000 7f10e3009740 1 -- start start 2024-07-26T13:42:00.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.674+0000 7f10e3009740 1 Processor -- start 2024-07-26T13:42:00.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.675+0000 7f10e3009740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c45e4911b0 0x55c45e86b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:00.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.675+0000 7f10e3009740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c45e499390 0x55c45e86b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:00.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.675+0000 7f10e3009740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c45e4ba070 0x55c45e877040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:00.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.675+0000 7f10e3009740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c45e628700 con 0x55c45e4ba070 2024-07-26T13:42:00.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.675+0000 7f10e3009740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c45e7679e0 con 0x55c45e499390 2024-07-26T13:42:00.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.675+0000 7f10e3009740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c45e86dd50 con 0x55c45e4911b0 2024-07-26T13:42:00.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.675+0000 7f10e3009740 10 client.0 _ll_register_callbacks cb 0x55c45e7824f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:42:00.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.675+0000 7f10d6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c45e4ba070 0x55c45e877040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:00.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.675+0000 7f10d6d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c45e4911b0 0x55c45e86b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:00.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.675+0000 7f10d6d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c45e4911b0 0x55c45e86b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:37362/0 (socket says 172.21.15.100:37362) 2024-07-26T13:42:00.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.675+0000 7f10d6d9d640 1 -- 172.21.15.100:0/4061349182 learned_addr learned my addr 172.21.15.100:0/4061349182 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:00.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.675+0000 7f10d5d9b640 1 --2- 172.21.15.100:0/4061349182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c45e499390 0x55c45e86b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:00.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.675+0000 7f10d6d9d640 1 -- 172.21.15.100:0/4061349182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c45e4911b0 msgr2=0x55c45e86b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:00.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.675+0000 7f10d6d9d640 1 --2- 172.21.15.100:0/4061349182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c45e4911b0 0x55c45e86b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:00.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.675+0000 7f10d6d9d640 1 -- 172.21.15.100:0/4061349182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c45e499390 msgr2=0x55c45e86b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:00.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.675+0000 7f10d6d9d640 1 --2- 172.21.15.100:0/4061349182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c45e499390 0x55c45e86b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:00.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.675+0000 7f10d6d9d640 1 -- 172.21.15.100:0/4061349182 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c45e525650 con 0x55c45e4ba070 2024-07-26T13:42:00.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.676+0000 7f10d6d9d640 1 --2- 172.21.15.100:0/4061349182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c45e4ba070 0x55c45e877040 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f10d0011640 tx=0x7f10d0011740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d54c1d1f9c26a3 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:00.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.676+0000 7f10bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:42:00.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.676+0000 7f10bdffb640 1 -- 172.21.15.100:0/4061349182 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d0022070 con 0x55c45e4ba070 2024-07-26T13:42:00.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.676+0000 7f10bdffb640 1 -- 172.21.15.100:0/4061349182 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f10d0004a40 con 0x55c45e4ba070 2024-07-26T13:42:00.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.676+0000 7f10e3009740 1 -- 172.21.15.100:0/4061349182 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10d001ab10 con 0x55c45e4ba070 2024-07-26T13:42:00.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.676+0000 7f10bdffb640 1 -- 172.21.15.100:0/4061349182 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f10d0031050 con 0x55c45e4ba070 2024-07-26T13:42:00.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.676+0000 7f10e3009740 10 client.15987 fetch_fsmap learned FSMap version 27 2024-07-26T13:42:00.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.676+0000 7f10e3009740 1 -- 172.21.15.100:0/4061349182 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c45e879d60 con 0x55c45e4ba070 2024-07-26T13:42:00.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.676+0000 7f10bdffb640 1 -- 172.21.15.100:0/4061349182 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10d001ab10 con 0x55c45e4ba070 2024-07-26T13:42:00.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.676+0000 7f10bdffb640 1 -- 172.21.15.100:0/4061349182 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f10d00108f0 con 0x55c45e4ba070 2024-07-26T13:42:00.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.676+0000 7f10e3009740 10 client.15987 fetch_fsmap finished waiting for FSMap version 27 2024-07-26T13:42:00.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.676+0000 7f10e3009740 10 client.15987 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-26T13:42:00.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.676+0000 7f10e3009740 20 client.15987 populate_metadata read hostname 'smithi100' 2024-07-26T13:42:00.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.676+0000 7f10e3009740 1 --2- 172.21.15.100:0/4061349182 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55c45e4789f0 0x55c45e87d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:00.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.676+0000 7f10e3009740 4 client.15987 mds_command: new command op to 24595 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:42:00.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.676+0000 7f10e3009740 1 -- 172.21.15.100:0/4061349182 --> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c45e7d7b00 con 0x55c45e4789f0 2024-07-26T13:42:00.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.677+0000 7f10d5d9b640 1 --2- 172.21.15.100:0/4061349182 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55c45e4789f0 0x55c45e87d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:00.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.677+0000 7f10d5d9b640 1 --2- 172.21.15.100:0/4061349182 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55c45e4789f0 0x55c45e87d120 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ea992cc932358fe0 server_cookie=665d5878db42ed93 in_seq=0 out_seq=0 2024-07-26T13:42:00.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.677+0000 7f10bdffb640 10 client.15987 ms_handle_connect on v2:172.21.15.167:6836/3828634468 2024-07-26T13:42:00.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.678+0000 7f10bdffb640 1 -- 172.21.15.100:0/4061349182 <== mds.2 v2:172.21.15.167:6836/3828634468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c45e7d7b00 con 0x55c45e4789f0 2024-07-26T13:42:00.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.678+0000 7f10bdffb640 10 client.15987 handle_command_reply: tid=0 2024-07-26T13:42:00.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.678+0000 7f10e3009740 1 client.15987 shutdown 2024-07-26T13:42:00.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.678+0000 7f10e3009740 1 -- 172.21.15.100:0/4061349182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c45e4ba070 msgr2=0x55c45e877040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:00.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.678+0000 7f10e3009740 1 --2- 172.21.15.100:0/4061349182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c45e4ba070 0x55c45e877040 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f10d0011640 tx=0x7f10d0011740 comp rx=0 tx=0).stop 2024-07-26T13:42:00.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.678+0000 7f10e3009740 1 -- 172.21.15.100:0/4061349182 shutdown_connections 2024-07-26T13:42:00.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.678+0000 7f10e3009740 1 --2- 172.21.15.100:0/4061349182 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55c45e4789f0 0x55c45e87d120 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:00.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.678+0000 7f10e3009740 1 --2- 172.21.15.100:0/4061349182 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c45e4ba070 0x55c45e877040 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:00.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.678+0000 7f10e3009740 1 --2- 172.21.15.100:0/4061349182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c45e499390 0x55c45e86b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:00.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.678+0000 7f10bdffb640 0 client.15987 ms_handle_reset on v2:172.21.15.167:6836/3828634468 2024-07-26T13:42:00.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.678+0000 7f10e3009740 1 --2- 172.21.15.100:0/4061349182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c45e4911b0 0x55c45e86b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:00.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.678+0000 7f10e3009740 1 -- 172.21.15.100:0/4061349182 >> 172.21.15.100:0/4061349182 conn(0x55c45e4ae460 msgr2=0x55c45e85bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:00.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.678+0000 7f10e3009740 1 -- 172.21.15.100:0/4061349182 shutdown_connections 2024-07-26T13:42:00.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.678+0000 7f10e3009740 1 -- 172.21.15.100:0/4061349182 wait complete. 2024-07-26T13:42:00.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.678+0000 7f10e3009740 20 client.15987 trim_cache size 0 max 16384 2024-07-26T13:42:00.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.691+0000 7f10e3009740 1 Processor -- start 2024-07-26T13:42:00.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.691+0000 7f10e3009740 1 -- start start 2024-07-26T13:42:00.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:42:00.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:42:00.693 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:42:00.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.692+0000 7f10e3009740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c45e4789f0 0x55c45e86cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:00.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.692+0000 7f10e3009740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c45e4911b0 0x55c45e876fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:00.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.692+0000 7f10e3009740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c45e499390 0x55c45e8774f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:00.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.692+0000 7f10e3009740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f10d81b3d20 con 0x55c45e4789f0 2024-07-26T13:42:00.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.692+0000 7f10e3009740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f10d81b15f0 con 0x55c45e499390 2024-07-26T13:42:00.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.692+0000 7f10e3009740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f10d81af440 con 0x55c45e4911b0 2024-07-26T13:42:00.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.692+0000 7f10d6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c45e4789f0 0x55c45e86cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:00.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.692+0000 7f10d6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c45e4789f0 0x55c45e86cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46576/0 (socket says 172.21.15.100:46576) 2024-07-26T13:42:00.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.692+0000 7f10d6d9d640 1 -- 172.21.15.100:0/1884556086 learned_addr learned my addr 172.21.15.100:0/1884556086 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:00.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.692+0000 7f10d5d9b640 1 --2- 172.21.15.100:0/1884556086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c45e4911b0 0x55c45e876fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:00.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.692+0000 7f10d6d9d640 1 -- 172.21.15.100:0/1884556086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c45e4911b0 msgr2=0x55c45e876fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:00.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.692+0000 7f10d6d9d640 1 --2- 172.21.15.100:0/1884556086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c45e4911b0 0x55c45e876fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:00.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.692+0000 7f10d6d9d640 1 -- 172.21.15.100:0/1884556086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c45e499390 msgr2=0x55c45e8774f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:00.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.692+0000 7f10d6d9d640 1 --2- 172.21.15.100:0/1884556086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c45e499390 0x55c45e8774f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:00.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.692+0000 7f10d6d9d640 1 -- 172.21.15.100:0/1884556086 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c45e86d660 con 0x55c45e4789f0 2024-07-26T13:42:00.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.692+0000 7f10d6d9d640 1 --2- 172.21.15.100:0/1884556086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c45e4789f0 0x55c45e86cb50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f10d0002230 tx=0x7f10d00092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b251eac579b67f server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:00.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.692+0000 7f10bffff640 1 -- 172.21.15.100:0/1884556086 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d003a070 con 0x55c45e4789f0 2024-07-26T13:42:00.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.693+0000 7f10bffff640 1 -- 172.21.15.100:0/1884556086 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10d0009b20 con 0x55c45e4789f0 2024-07-26T13:42:00.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.693+0000 7f10bffff640 1 -- 172.21.15.100:0/1884556086 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d0022040 con 0x55c45e4789f0 2024-07-26T13:42:00.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.693+0000 7f10e3009740 1 -- 172.21.15.100:0/1884556086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c45e4789f0 msgr2=0x55c45e86cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:00.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.693+0000 7f10e3009740 1 --2- 172.21.15.100:0/1884556086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c45e4789f0 0x55c45e86cb50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f10d0002230 tx=0x7f10d00092a0 comp rx=0 tx=0).stop 2024-07-26T13:42:00.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.693+0000 7f10e3009740 1 -- 172.21.15.100:0/1884556086 shutdown_connections 2024-07-26T13:42:00.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.693+0000 7f10e3009740 1 --2- 172.21.15.100:0/1884556086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c45e499390 0x55c45e8774f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:00.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.693+0000 7f10e3009740 1 --2- 172.21.15.100:0/1884556086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c45e4911b0 0x55c45e876fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:00.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.693+0000 7f10e3009740 1 --2- 172.21.15.100:0/1884556086 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c45e4789f0 0x55c45e86cb50 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:00.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.693+0000 7f10e3009740 1 -- 172.21.15.100:0/1884556086 >> 172.21.15.100:0/1884556086 conn(0x55c45e4ae460 msgr2=0x55c45e86c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:00.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.693+0000 7f10e3009740 1 -- 172.21.15.100:0/1884556086 shutdown_connections 2024-07-26T13:42:00.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.693+0000 7f10e3009740 1 -- 172.21.15.100:0/1884556086 wait complete. 2024-07-26T13:42:00.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.694+0000 7f10e3009740 1 -- start start 2024-07-26T13:42:00.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.694+0000 7f10e3009740 1 Processor -- start 2024-07-26T13:42:00.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.694+0000 7f10e3009740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c45e4789f0 0x55c45e87d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:00.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.694+0000 7f10e3009740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c45e4911b0 0x55c45e87dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:00.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.694+0000 7f10e3009740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c45e499390 0x55c45e8e3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:00.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.694+0000 7f10e3009740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f10d81afab0 con 0x55c45e499390 2024-07-26T13:42:00.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.694+0000 7f10e3009740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f10d81b0f20 con 0x55c45e4789f0 2024-07-26T13:42:00.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.694+0000 7f10e3009740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f10d81b0180 con 0x55c45e4911b0 2024-07-26T13:42:00.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.694+0000 7f10e3009740 10 client.0 _ll_register_callbacks cb 0x7f10d0009da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:42:00.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.694+0000 7f10d6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c45e499390 0x55c45e8e3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:00.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.694+0000 7f10d6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c45e499390 0x55c45e8e3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46584/0 (socket says 172.21.15.100:46584) 2024-07-26T13:42:00.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.694+0000 7f10d6d9d640 1 -- 172.21.15.100:0/1261711 learned_addr learned my addr 172.21.15.100:0/1261711 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:00.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.694+0000 7f10d5d9b640 1 --2- 172.21.15.100:0/1261711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c45e4911b0 0x55c45e87dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:00.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.695+0000 7f10d6d9d640 1 --2- 172.21.15.100:0/1261711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c45e4789f0 0x55c45e87d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:00.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.695+0000 7f10d6d9d640 1 -- 172.21.15.100:0/1261711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c45e4911b0 msgr2=0x55c45e87dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:00.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.695+0000 7f10d6d9d640 1 --2- 172.21.15.100:0/1261711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c45e4911b0 0x55c45e87dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:00.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.695+0000 7f10d6d9d640 1 -- 172.21.15.100:0/1261711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c45e4789f0 msgr2=0x55c45e87d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:00.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.695+0000 7f10d6d9d640 1 --2- 172.21.15.100:0/1261711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c45e4789f0 0x55c45e87d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:00.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.695+0000 7f10d6d9d640 1 -- 172.21.15.100:0/1261711 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c45e8e3c20 con 0x55c45e499390 2024-07-26T13:42:00.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.695+0000 7f10d6d9d640 1 --2- 172.21.15.100:0/1261711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c45e499390 0x55c45e8e3660 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f10d0010eb0 tx=0x7f10d0004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fdafba0e411da85 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:00.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.695+0000 7f10bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:42:00.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.695+0000 7f10bdffb640 1 -- 172.21.15.100:0/1261711 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d003a040 con 0x55c45e499390 2024-07-26T13:42:00.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.695+0000 7f10bdffb640 1 -- 172.21.15.100:0/1261711 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f10d0011dc0 con 0x55c45e499390 2024-07-26T13:42:00.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.695+0000 7f10e3009740 1 -- 172.21.15.100:0/1261711 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10d0009b20 con 0x55c45e499390 2024-07-26T13:42:00.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.695+0000 7f10bdffb640 1 -- 172.21.15.100:0/1261711 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f10d0015040 con 0x55c45e499390 2024-07-26T13:42:00.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.695+0000 7f10e3009740 10 client.15993 fetch_fsmap learned FSMap version 27 2024-07-26T13:42:00.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.695+0000 7f10e3009740 1 -- 172.21.15.100:0/1261711 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c45e8e4ea0 con 0x55c45e499390 2024-07-26T13:42:00.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.696+0000 7f10bdffb640 1 -- 172.21.15.100:0/1261711 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10d0009b20 con 0x55c45e499390 2024-07-26T13:42:00.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.696+0000 7f10bdffb640 1 -- 172.21.15.100:0/1261711 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f10d0049050 con 0x55c45e499390 2024-07-26T13:42:00.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.696+0000 7f10e3009740 10 client.15993 fetch_fsmap finished waiting for FSMap version 27 2024-07-26T13:42:00.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.696+0000 7f10e3009740 10 client.15993 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-26T13:42:00.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.696+0000 7f10e3009740 20 client.15993 populate_metadata read hostname 'smithi100' 2024-07-26T13:42:00.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.696+0000 7f10e3009740 1 --2- 172.21.15.100:0/1261711 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55c45e4ba070 0x55c45e8e7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:00.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.696+0000 7f10e3009740 4 client.15993 mds_command: new command op to 24595 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-26T13:42:00.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.696+0000 7f10e3009740 1 -- 172.21.15.100:0/1261711 --> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c45e7d35a0 con 0x55c45e4ba070 2024-07-26T13:42:00.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.696+0000 7f10d5d9b640 1 --2- 172.21.15.100:0/1261711 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55c45e4ba070 0x55c45e8e7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:00.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.697+0000 7f10bdffb640 10 client.15993 ms_handle_connect on v2:172.21.15.167:6836/3828634468 2024-07-26T13:42:00.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.697+0000 7f10d5d9b640 1 --2- 172.21.15.100:0/1261711 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55c45e4ba070 0x55c45e8e7150 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=37c443c28d68da39 server_cookie=7bb34f48836b5c2 in_seq=0 out_seq=0 2024-07-26T13:42:00.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.697+0000 7f10bdffb640 1 -- 172.21.15.100:0/1261711 <== mds.2 v2:172.21.15.167:6836/3828634468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c45e7d35a0 con 0x55c45e4ba070 2024-07-26T13:42:00.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.697+0000 7f10bdffb640 10 client.15993 handle_command_reply: tid=0 2024-07-26T13:42:00.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.697+0000 7f10e3009740 1 client.15993 shutdown 2024-07-26T13:42:00.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.697+0000 7f10e3009740 1 -- 172.21.15.100:0/1261711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c45e499390 msgr2=0x55c45e8e3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:00.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.697+0000 7f10e3009740 1 --2- 172.21.15.100:0/1261711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c45e499390 0x55c45e8e3660 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f10d0010eb0 tx=0x7f10d0004f90 comp rx=0 tx=0).stop 2024-07-26T13:42:00.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.697+0000 7f10e3009740 1 -- 172.21.15.100:0/1261711 shutdown_connections 2024-07-26T13:42:00.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.697+0000 7f10e3009740 1 --2- 172.21.15.100:0/1261711 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x55c45e4ba070 0x55c45e8e7150 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:00.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.697+0000 7f10e3009740 1 --2- 172.21.15.100:0/1261711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c45e499390 0x55c45e8e3660 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:00.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.697+0000 7f10e3009740 1 --2- 172.21.15.100:0/1261711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c45e4911b0 0x55c45e87dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:00.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.697+0000 7f10e3009740 1 --2- 172.21.15.100:0/1261711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c45e4789f0 0x55c45e87d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:00.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.697+0000 7f10bdffb640 0 client.15993 ms_handle_reset on v2:172.21.15.167:6836/3828634468 2024-07-26T13:42:00.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.697+0000 7f10e3009740 1 -- 172.21.15.100:0/1261711 >> 172.21.15.100:0/1261711 conn(0x55c45e4ae460 msgr2=0x55c45e860530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:00.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.697+0000 7f10e3009740 1 -- 172.21.15.100:0/1261711 shutdown_connections 2024-07-26T13:42:00.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.697+0000 7f10e3009740 1 -- 172.21.15.100:0/1261711 wait complete. 2024-07-26T13:42:00.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.697+0000 7f10e3009740 20 client.15993 trim_cache size 0 max 16384 2024-07-26T13:42:00.704 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-07-26T13:42:00.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.698+0000 7f10dca85640 1 -- 172.21.15.100:0/4145388200 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f10c00b3d40 msgr2=0x7f10c00b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:00.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.698+0000 7f10dca85640 1 --2- 172.21.15.100:0/4145388200 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f10c00b3d40 0x7f10c00b6250 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f10c80e1440 tx=0x7f10c8092040 comp rx=0 tx=0).stop 2024-07-26T13:42:00.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.698+0000 7f10dca85640 1 -- 172.21.15.100:0/4145388200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10d8103590 msgr2=0x7f10d8075140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:00.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.698+0000 7f10dca85640 1 --2- 172.21.15.100:0/4145388200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10d8103590 0x7f10d8075140 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f10c800d950 tx=0x7f10c800de10 comp rx=0 tx=0).stop 2024-07-26T13:42:00.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.698+0000 7f10dca85640 1 -- 172.21.15.100:0/4145388200 shutdown_connections 2024-07-26T13:42:00.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.698+0000 7f10dca85640 1 --2- 172.21.15.100:0/4145388200 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f10c00b3d40 0x7f10c00b6250 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:00.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.698+0000 7f10dca85640 1 --2- 172.21.15.100:0/4145388200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10d8109820 0x7f10d8075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:00.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.698+0000 7f10dca85640 1 --2- 172.21.15.100:0/4145388200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10d8108ed0 0x7f10d8075680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:00.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.698+0000 7f10dca85640 1 --2- 172.21.15.100:0/4145388200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10d8103590 0x7f10d8075140 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:00.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.698+0000 7f10dca85640 1 -- 172.21.15.100:0/4145388200 >> 172.21.15.100:0/4145388200 conn(0x7f10d807b280 msgr2=0x7f10d8107090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:00.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.698+0000 7f10dca85640 1 -- 172.21.15.100:0/4145388200 shutdown_connections 2024-07-26T13:42:00.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:00.698+0000 7f10dca85640 1 -- 172.21.15.100:0/4145388200 wait complete. 2024-07-26T13:42:00.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:42:00.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:42:00.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:42:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:42:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:42:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:42:01.403 INFO:tasks.workunit.client.0.smithi100.stdout: 1 20252 8.28 MB/sec warmup 15 sec latency 8723.044 ms 2024-07-26T13:42:01.420 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-26T13:42:01.676 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:42:01.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:01 smithi100 ceph-mon[32599]: pgmap v421: 97 pgs: 97 active+clean; 12 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.4 MiB/s wr, 183 op/s 2024-07-26T13:42:01.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:01 smithi138 ceph-mon[36954]: pgmap v421: 97 pgs: 97 active+clean; 12 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.4 MiB/s wr, 183 op/s 2024-07-26T13:42:02.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:01 smithi167 ceph-mon[37140]: pgmap v421: 97 pgs: 97 active+clean; 12 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.4 MiB/s wr, 183 op/s 2024-07-26T13:42:02.403 INFO:tasks.workunit.client.0.smithi100.stdout: 1 20252 7.77 MB/sec warmup 16 sec latency 9723.152 ms 2024-07-26T13:42:03.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.386+0000 7f4b769e4640 1 -- 172.21.15.100:0/3396998284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b70107e50 msgr2=0x7f4b70108260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:03.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.386+0000 7f4b769e4640 1 --2- 172.21.15.100:0/3396998284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b70107e50 0x7f4b70108260 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009900 tx=0x7f4b6402a410 comp rx=0 tx=0).stop 2024-07-26T13:42:03.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.386+0000 7f4b769e4640 1 -- 172.21.15.100:0/3396998284 shutdown_connections 2024-07-26T13:42:03.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.386+0000 7f4b769e4640 1 --2- 172.21.15.100:0/3396998284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b701087a0 0x7f4b7010efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:03.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.386+0000 7f4b769e4640 1 --2- 172.21.15.100:0/3396998284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b70107e50 0x7f4b70108260 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:03.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.386+0000 7f4b769e4640 1 --2- 172.21.15.100:0/3396998284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b70102510 0x7f4b70102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:03.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.386+0000 7f4b769e4640 1 -- 172.21.15.100:0/3396998284 >> 172.21.15.100:0/3396998284 conn(0x7f4b700791a0 msgr2=0x7f4b700795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:03.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.386+0000 7f4b769e4640 1 -- 172.21.15.100:0/3396998284 shutdown_connections 2024-07-26T13:42:03.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.386+0000 7f4b769e4640 1 -- 172.21.15.100:0/3396998284 wait complete. 2024-07-26T13:42:03.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.386+0000 7f4b769e4640 1 Processor -- start 2024-07-26T13:42:03.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.387+0000 7f4b769e4640 1 -- start start 2024-07-26T13:42:03.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.387+0000 7f4b769e4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b70102510 0x7f4b70199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:03.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.387+0000 7f4b769e4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b70107e50 0x7f4b7019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:03.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.387+0000 7f4b769e4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b701087a0 0x7f4b7019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:03.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.387+0000 7f4b769e4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4b70113580 con 0x7f4b701087a0 2024-07-26T13:42:03.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.387+0000 7f4b769e4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4b70113400 con 0x7f4b70107e50 2024-07-26T13:42:03.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.387+0000 7f4b769e4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4b70113700 con 0x7f4b70102510 2024-07-26T13:42:03.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.387+0000 7f4b74f81640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b701087a0 0x7f4b7019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:03.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.387+0000 7f4b74f81640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b701087a0 0x7f4b7019f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46606/0 (socket says 172.21.15.100:46606) 2024-07-26T13:42:03.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.387+0000 7f4b74f81640 1 -- 172.21.15.100:0/3029914967 learned_addr learned my addr 172.21.15.100:0/3029914967 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:03.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.387+0000 7f4b6f7fe640 1 --2- 172.21.15.100:0/3029914967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b70107e50 0x7f4b7019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:03.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.387+0000 7f4b6ffff640 1 --2- 172.21.15.100:0/3029914967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b70102510 0x7f4b70199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:03.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.388+0000 7f4b74f81640 1 -- 172.21.15.100:0/3029914967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b70102510 msgr2=0x7f4b70199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:03.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.388+0000 7f4b74f81640 1 --2- 172.21.15.100:0/3029914967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b70102510 0x7f4b70199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:03.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.388+0000 7f4b74f81640 1 -- 172.21.15.100:0/3029914967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b70107e50 msgr2=0x7f4b7019a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:03.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.388+0000 7f4b74f81640 1 --2- 172.21.15.100:0/3029914967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b70107e50 0x7f4b7019a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:03.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.388+0000 7f4b74f81640 1 -- 172.21.15.100:0/3029914967 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b7019fec0 con 0x7f4b701087a0 2024-07-26T13:42:03.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.388+0000 7f4b74f81640 1 --2- 172.21.15.100:0/3029914967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b701087a0 0x7f4b7019f790 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f4b6800eaa0 tx=0x7f4b6800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:03.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.388+0000 7f4b6d7fa640 1 -- 172.21.15.100:0/3029914967 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6800cdb0 con 0x7f4b701087a0 2024-07-26T13:42:03.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.388+0000 7f4b769e4640 1 -- 172.21.15.100:0/3029914967 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b701a01b0 con 0x7f4b701087a0 2024-07-26T13:42:03.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.388+0000 7f4b6d7fa640 1 -- 172.21.15.100:0/3029914967 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b68005020 con 0x7f4b701087a0 2024-07-26T13:42:03.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.388+0000 7f4b6d7fa640 1 -- 172.21.15.100:0/3029914967 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b680116d0 con 0x7f4b701087a0 2024-07-26T13:42:03.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.388+0000 7f4b769e4640 1 -- 172.21.15.100:0/3029914967 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b7010f990 con 0x7f4b701087a0 2024-07-26T13:42:03.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.391+0000 7f4b7cf68740 1 Processor -- start 2024-07-26T13:42:03.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.391+0000 7f4b6d7fa640 1 -- 172.21.15.100:0/3029914967 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f4b68011870 con 0x7f4b701087a0 2024-07-26T13:42:03.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.391+0000 7f4b7cf68740 1 -- start start 2024-07-26T13:42:03.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.391+0000 7f4b7cf68740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e97e492070 0x55e97e837570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:03.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.391+0000 7f4b7cf68740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e97e4691b0 0x55e97e837ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:03.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.391+0000 7f4b7cf68740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e97e471390 0x55e97e837ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:03.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.391+0000 7f4b7cf68740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55e97e7adbb0 con 0x55e97e492070 2024-07-26T13:42:03.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.391+0000 7f4b7cf68740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e97e79d0b0 con 0x55e97e471390 2024-07-26T13:42:03.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.391+0000 7f4b7cf68740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e97e7a10f0 con 0x55e97e4691b0 2024-07-26T13:42:03.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.391+0000 7f4b6f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e97e492070 0x55e97e837570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:03.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.392+0000 7f4b6f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e97e492070 0x55e97e837570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46616/0 (socket says 172.21.15.100:46616) 2024-07-26T13:42:03.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.392+0000 7f4b6f7fe640 1 -- 172.21.15.100:0/651984418 learned_addr learned my addr 172.21.15.100:0/651984418 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:03.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.392+0000 7f4b6ffff640 1 --2- 172.21.15.100:0/651984418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e97e4691b0 0x55e97e837ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:03.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.392+0000 7f4b6f7fe640 1 --2- 172.21.15.100:0/651984418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e97e471390 0x55e97e837ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:03.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.392+0000 7f4b6f7fe640 1 -- 172.21.15.100:0/651984418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e97e4691b0 msgr2=0x55e97e837ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:03.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.392+0000 7f4b6f7fe640 1 --2- 172.21.15.100:0/651984418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e97e4691b0 0x55e97e837ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:03.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.392+0000 7f4b6f7fe640 1 -- 172.21.15.100:0/651984418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e97e471390 msgr2=0x55e97e837ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:03.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.392+0000 7f4b6f7fe640 1 --2- 172.21.15.100:0/651984418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e97e471390 0x55e97e837ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:03.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.392+0000 7f4b6f7fe640 1 -- 172.21.15.100:0/651984418 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e97e743b90 con 0x55e97e492070 2024-07-26T13:42:03.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.392+0000 7f4b6d7fa640 1 --2- 172.21.15.100:0/3029914967 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f4b580b3d40 0x7f4b580b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:03.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.392+0000 7f4b6ffff640 1 --2- 172.21.15.100:0/3029914967 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f4b580b3d40 0x7f4b580b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:03.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.392+0000 7f4b6f7fe640 1 --2- 172.21.15.100:0/651984418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e97e492070 0x55e97e837570 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009900 tx=0x7f4b64000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9060cea245bb421d server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:03.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.392+0000 7f4b6d7fa640 1 -- 172.21.15.100:0/3029914967 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f4b680e40a0 con 0x7f4b701087a0 2024-07-26T13:42:03.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.392+0000 7f4b567fc640 1 -- 172.21.15.100:0/651984418 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b64035040 con 0x55e97e492070 2024-07-26T13:42:03.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.392+0000 7f4b567fc640 1 -- 172.21.15.100:0/651984418 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b64045070 con 0x55e97e492070 2024-07-26T13:42:03.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.392+0000 7f4b567fc640 1 -- 172.21.15.100:0/651984418 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b640052a0 con 0x55e97e492070 2024-07-26T13:42:03.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.393+0000 7f4b7cf68740 1 -- 172.21.15.100:0/651984418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e97e492070 msgr2=0x55e97e837570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:03.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.393+0000 7f4b6ffff640 1 --2- 172.21.15.100:0/3029914967 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f4b580b3d40 0x7f4b580b6250 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x55e97e8361f0 tx=0x7f4b6000b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:03.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.393+0000 7f4b7cf68740 1 --2- 172.21.15.100:0/651984418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e97e492070 0x55e97e837570 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009900 tx=0x7f4b64000bc0 comp rx=0 tx=0).stop 2024-07-26T13:42:03.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.393+0000 7f4b7cf68740 1 -- 172.21.15.100:0/651984418 shutdown_connections 2024-07-26T13:42:03.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.393+0000 7f4b7cf68740 1 --2- 172.21.15.100:0/651984418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e97e471390 0x55e97e837ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:03.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.393+0000 7f4b7cf68740 1 --2- 172.21.15.100:0/651984418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e97e4691b0 0x55e97e837ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:03.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.393+0000 7f4b7cf68740 1 --2- 172.21.15.100:0/651984418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e97e492070 0x55e97e837570 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:03.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.393+0000 7f4b7cf68740 1 -- 172.21.15.100:0/651984418 >> 172.21.15.100:0/651984418 conn(0x55e97e486460 msgr2=0x55e97e835460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:03.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.393+0000 7f4b7cf68740 1 -- 172.21.15.100:0/651984418 shutdown_connections 2024-07-26T13:42:03.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.393+0000 7f4b7cf68740 1 -- 172.21.15.100:0/651984418 wait complete. 2024-07-26T13:42:03.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.394+0000 7f4b7cf68740 1 -- start start 2024-07-26T13:42:03.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.394+0000 7f4b7cf68740 1 Processor -- start 2024-07-26T13:42:03.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.394+0000 7f4b7cf68740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e97e4691b0 0x55e97e8431d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:03.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.394+0000 7f4b7cf68740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e97e471390 0x55e97e8437d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:03.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.394+0000 7f4b7cf68740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e97e492070 0x55e97e84efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:03.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.394+0000 7f4b7cf68740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55e97e600700 con 0x55e97e471390 2024-07-26T13:42:03.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.394+0000 7f4b7cf68740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e97e73f9e0 con 0x55e97e4691b0 2024-07-26T13:42:03.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.394+0000 7f4b7cf68740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e97e845cf0 con 0x55e97e492070 2024-07-26T13:42:03.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.394+0000 7f4b7cf68740 10 client.0 _ll_register_callbacks cb 0x55e97e75a4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:42:03.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.395+0000 7f4b6f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e97e471390 0x55e97e8437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:03.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.395+0000 7f4b6f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e97e471390 0x55e97e8437d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46626/0 (socket says 172.21.15.100:46626) 2024-07-26T13:42:03.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.395+0000 7f4b6f7fe640 1 -- 172.21.15.100:0/1621134668 learned_addr learned my addr 172.21.15.100:0/1621134668 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:03.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.395+0000 7f4b74f81640 1 --2- 172.21.15.100:0/1621134668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e97e492070 0x55e97e84efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:03.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.395+0000 7f4b6ffff640 1 --2- 172.21.15.100:0/1621134668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e97e4691b0 0x55e97e8431d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:03.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.395+0000 7f4b6f7fe640 1 -- 172.21.15.100:0/1621134668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e97e492070 msgr2=0x55e97e84efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:03.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.395+0000 7f4b6f7fe640 1 --2- 172.21.15.100:0/1621134668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e97e492070 0x55e97e84efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:03.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.395+0000 7f4b6f7fe640 1 -- 172.21.15.100:0/1621134668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e97e4691b0 msgr2=0x55e97e8431d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:03.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.395+0000 7f4b6f7fe640 1 --2- 172.21.15.100:0/1621134668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e97e4691b0 0x55e97e8431d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:03.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.395+0000 7f4b6f7fe640 1 -- 172.21.15.100:0/1621134668 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e97e4fd650 con 0x55e97e471390 2024-07-26T13:42:03.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.395+0000 7f4b6ffff640 1 --2- 172.21.15.100:0/1621134668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e97e4691b0 0x55e97e8431d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:42:03.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.395+0000 7f4b6f7fe640 1 --2- 172.21.15.100:0/1621134668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e97e471390 0x55e97e8437d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4b6402a450 tx=0x7f4b64004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a3c0ebd9109925 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:03.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.395+0000 7f4b4f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:42:03.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.395+0000 7f4b4f7fe640 1 -- 172.21.15.100:0/1621134668 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6404e060 con 0x55e97e471390 2024-07-26T13:42:03.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.395+0000 7f4b7cf68740 1 -- 172.21.15.100:0/1621134668 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b64045070 con 0x55e97e471390 2024-07-26T13:42:03.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.395+0000 7f4b4f7fe640 1 -- 172.21.15.100:0/1621134668 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f4b64005230 con 0x55e97e471390 2024-07-26T13:42:03.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.395+0000 7f4b4f7fe640 1 -- 172.21.15.100:0/1621134668 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4b64035070 con 0x55e97e471390 2024-07-26T13:42:03.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.395+0000 7f4b7cf68740 10 client.16005 fetch_fsmap learned FSMap version 27 2024-07-26T13:42:03.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.395+0000 7f4b7cf68740 1 -- 172.21.15.100:0/1621134668 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e97e851db0 con 0x55e97e471390 2024-07-26T13:42:03.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.396+0000 7f4b4f7fe640 1 -- 172.21.15.100:0/1621134668 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b64045070 con 0x55e97e471390 2024-07-26T13:42:03.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.396+0000 7f4b4f7fe640 1 -- 172.21.15.100:0/1621134668 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4b6403a3f0 con 0x55e97e471390 2024-07-26T13:42:03.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.396+0000 7f4b7cf68740 10 client.16005 fetch_fsmap finished waiting for FSMap version 27 2024-07-26T13:42:03.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.396+0000 7f4b7cf68740 10 client.16005 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-26T13:42:03.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.396+0000 7f4b7cf68740 20 client.16005 populate_metadata read hostname 'smithi100' 2024-07-26T13:42:03.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.396+0000 7f4b7cf68740 1 --2- 172.21.15.100:0/1621134668 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55e97e4509f0 0x55e97e855110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:03.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.396+0000 7f4b7cf68740 4 client.16005 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:42:03.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.396+0000 7f4b7cf68740 1 -- 172.21.15.100:0/1621134668 --> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e97e7afb00 con 0x55e97e4509f0 2024-07-26T13:42:03.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.396+0000 7f4b6ffff640 1 --2- 172.21.15.100:0/1621134668 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55e97e4509f0 0x55e97e855110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:03.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.397+0000 7f4b4f7fe640 10 client.16005 ms_handle_connect on v2:172.21.15.138:6838/1580925471 2024-07-26T13:42:03.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.397+0000 7f4b6ffff640 1 --2- 172.21.15.100:0/1621134668 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55e97e4509f0 0x55e97e855110 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b635339d64cd4c1e server_cookie=14ed83c897180893 in_seq=0 out_seq=0 2024-07-26T13:42:03.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.397+0000 7f4b4f7fe640 1 -- 172.21.15.100:0/1621134668 <== mds.1 v2:172.21.15.138:6838/1580925471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e97e7afb00 con 0x55e97e4509f0 2024-07-26T13:42:03.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.397+0000 7f4b4f7fe640 10 client.16005 handle_command_reply: tid=0 2024-07-26T13:42:03.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.397+0000 7f4b7cf68740 1 client.16005 shutdown 2024-07-26T13:42:03.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.398+0000 7f4b7cf68740 1 -- 172.21.15.100:0/1621134668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e97e471390 msgr2=0x55e97e8437d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:03.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.398+0000 7f4b7cf68740 1 --2- 172.21.15.100:0/1621134668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e97e471390 0x55e97e8437d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4b6402a450 tx=0x7f4b64004390 comp rx=0 tx=0).stop 2024-07-26T13:42:03.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.398+0000 7f4b7cf68740 1 -- 172.21.15.100:0/1621134668 shutdown_connections 2024-07-26T13:42:03.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.398+0000 7f4b7cf68740 1 --2- 172.21.15.100:0/1621134668 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55e97e4509f0 0x55e97e855110 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:03.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.398+0000 7f4b7cf68740 1 --2- 172.21.15.100:0/1621134668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e97e492070 0x55e97e84efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:03.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.398+0000 7f4b7cf68740 1 --2- 172.21.15.100:0/1621134668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e97e471390 0x55e97e8437d0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:03.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.398+0000 7f4b4f7fe640 0 client.16005 ms_handle_reset on v2:172.21.15.138:6838/1580925471 2024-07-26T13:42:03.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.398+0000 7f4b7cf68740 1 --2- 172.21.15.100:0/1621134668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e97e4691b0 0x55e97e8431d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:03.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.398+0000 7f4b7cf68740 1 -- 172.21.15.100:0/1621134668 >> 172.21.15.100:0/1621134668 conn(0x55e97e486460 msgr2=0x55e97e833c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:03.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.398+0000 7f4b7cf68740 1 -- 172.21.15.100:0/1621134668 shutdown_connections 2024-07-26T13:42:03.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.398+0000 7f4b7cf68740 1 -- 172.21.15.100:0/1621134668 wait complete. 2024-07-26T13:42:03.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.398+0000 7f4b7cf68740 20 client.16005 trim_cache size 0 max 16384 2024-07-26T13:42:03.407 INFO:tasks.workunit.client.0.smithi100.stdout: 1 20252 7.31 MB/sec warmup 17 sec latency 10723.219 ms 2024-07-26T13:42:03.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.411+0000 7f4b7cf68740 1 Processor -- start 2024-07-26T13:42:03.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.411+0000 7f4b7cf68740 1 -- start start 2024-07-26T13:42:03.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.411+0000 7f4b7cf68740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e97e4691b0 0x55e97e834350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:03.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.411+0000 7f4b7cf68740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e97e471390 0x55e97e84ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:03.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.411+0000 7f4b7cf68740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e97e492070 0x55e97e83c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:03.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.411+0000 7f4b7cf68740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4b701b3b10 con 0x55e97e471390 2024-07-26T13:42:03.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.411+0000 7f4b7cf68740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4b701b13e0 con 0x55e97e4691b0 2024-07-26T13:42:03.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.411+0000 7f4b7cf68740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4b701af250 con 0x55e97e492070 2024-07-26T13:42:03.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.411+0000 7f4b6f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e97e471390 0x55e97e84ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:03.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.411+0000 7f4b6f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e97e471390 0x55e97e84ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46628/0 (socket says 172.21.15.100:46628) 2024-07-26T13:42:03.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.411+0000 7f4b6f7fe640 1 -- 172.21.15.100:0/3917052438 learned_addr learned my addr 172.21.15.100:0/3917052438 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:03.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.411+0000 7f4b6f7fe640 1 -- 172.21.15.100:0/3917052438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e97e492070 msgr2=0x55e97e83c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:03.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.411+0000 7f4b6f7fe640 1 --2- 172.21.15.100:0/3917052438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e97e492070 0x55e97e83c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:03.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.411+0000 7f4b6f7fe640 1 -- 172.21.15.100:0/3917052438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e97e4691b0 msgr2=0x55e97e834350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:03.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.411+0000 7f4b6f7fe640 1 --2- 172.21.15.100:0/3917052438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e97e4691b0 0x55e97e834350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:03.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.411+0000 7f4b6f7fe640 1 -- 172.21.15.100:0/3917052438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e97e83a3d0 con 0x55e97e471390 2024-07-26T13:42:03.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.411+0000 7f4b6f7fe640 1 --2- 172.21.15.100:0/3917052438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e97e471390 0x55e97e84ef50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4b6403a7c0 tx=0x7f4b6402ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a71f97557b019f server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:03.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.412+0000 7f4b55ffb640 1 -- 172.21.15.100:0/3917052438 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6404e030 con 0x55e97e471390 2024-07-26T13:42:03.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.412+0000 7f4b55ffb640 1 -- 172.21.15.100:0/3917052438 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b64045040 con 0x55e97e471390 2024-07-26T13:42:03.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.412+0000 7f4b55ffb640 1 -- 172.21.15.100:0/3917052438 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b64032070 con 0x55e97e471390 2024-07-26T13:42:03.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.412+0000 7f4b7cf68740 1 -- 172.21.15.100:0/3917052438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e97e471390 msgr2=0x55e97e84ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:03.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.412+0000 7f4b7cf68740 1 --2- 172.21.15.100:0/3917052438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e97e471390 0x55e97e84ef50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4b6403a7c0 tx=0x7f4b6402ab10 comp rx=0 tx=0).stop 2024-07-26T13:42:03.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.412+0000 7f4b7cf68740 1 -- 172.21.15.100:0/3917052438 shutdown_connections 2024-07-26T13:42:03.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.412+0000 7f4b7cf68740 1 --2- 172.21.15.100:0/3917052438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e97e492070 0x55e97e83c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:03.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.412+0000 7f4b7cf68740 1 --2- 172.21.15.100:0/3917052438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e97e471390 0x55e97e84ef50 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:03.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.412+0000 7f4b7cf68740 1 --2- 172.21.15.100:0/3917052438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e97e4691b0 0x55e97e834350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:03.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.412+0000 7f4b7cf68740 1 -- 172.21.15.100:0/3917052438 >> 172.21.15.100:0/3917052438 conn(0x55e97e486460 msgr2=0x55e97e833c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:03.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.412+0000 7f4b7cf68740 1 -- 172.21.15.100:0/3917052438 shutdown_connections 2024-07-26T13:42:03.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.413+0000 7f4b7cf68740 1 -- 172.21.15.100:0/3917052438 wait complete. 2024-07-26T13:42:03.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.413+0000 7f4b7cf68740 1 -- start start 2024-07-26T13:42:03.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.413+0000 7f4b7cf68740 1 Processor -- start 2024-07-26T13:42:03.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.413+0000 7f4b7cf68740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e97e4691b0 0x55e97e855910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:03.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.413+0000 7f4b7cf68740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e97e471390 0x55e97e855ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:03.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.413+0000 7f4b7cf68740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e97e492070 0x55e97e8b5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:03.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.413+0000 7f4b7cf68740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4b701af8a0 con 0x55e97e492070 2024-07-26T13:42:03.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.413+0000 7f4b7cf68740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4b701b0d10 con 0x55e97e4691b0 2024-07-26T13:42:03.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.413+0000 7f4b7cf68740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4b701aff70 con 0x55e97e471390 2024-07-26T13:42:03.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.413+0000 7f4b7cf68740 10 client.0 _ll_register_callbacks cb 0x7f4b6000b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:42:03.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.414+0000 7f4b74f81640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e97e492070 0x55e97e8b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:03.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.414+0000 7f4b74f81640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e97e492070 0x55e97e8b5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46636/0 (socket says 172.21.15.100:46636) 2024-07-26T13:42:03.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.414+0000 7f4b74f81640 1 -- 172.21.15.100:0/3060138090 learned_addr learned my addr 172.21.15.100:0/3060138090 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:03.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.414+0000 7f4b6f7fe640 1 --2- 172.21.15.100:0/3060138090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e97e471390 0x55e97e855ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:03.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.414+0000 7f4b6ffff640 1 --2- 172.21.15.100:0/3060138090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e97e4691b0 0x55e97e855910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:03.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.414+0000 7f4b74f81640 1 -- 172.21.15.100:0/3060138090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e97e471390 msgr2=0x55e97e855ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:03.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.414+0000 7f4b74f81640 1 --2- 172.21.15.100:0/3060138090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e97e471390 0x55e97e855ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:03.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.414+0000 7f4b74f81640 1 -- 172.21.15.100:0/3060138090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e97e4691b0 msgr2=0x55e97e855910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:03.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.414+0000 7f4b74f81640 1 --2- 172.21.15.100:0/3060138090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e97e4691b0 0x55e97e855910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:03.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.414+0000 7f4b74f81640 1 -- 172.21.15.100:0/3060138090 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e97e8b5c90 con 0x55e97e492070 2024-07-26T13:42:03.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.414+0000 7f4b74f81640 1 --2- 172.21.15.100:0/3060138090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e97e492070 0x55e97e8b5620 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f4b68009270 tx=0x7f4b680aa6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8392af6e1858a25 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:03.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.414+0000 7f4b4f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:42:03.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.414+0000 7f4b4f7fe640 1 -- 172.21.15.100:0/3060138090 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b680c5060 con 0x55e97e492070 2024-07-26T13:42:03.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.414+0000 7f4b4f7fe640 1 -- 172.21.15.100:0/3060138090 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f4b68011b60 con 0x55e97e492070 2024-07-26T13:42:03.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.414+0000 7f4b7cf68740 1 -- 172.21.15.100:0/3060138090 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b64045040 con 0x55e97e492070 2024-07-26T13:42:03.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.414+0000 7f4b4f7fe640 1 -- 172.21.15.100:0/3060138090 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4b6801f040 con 0x55e97e492070 2024-07-26T13:42:03.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.414+0000 7f4b7cf68740 10 client.16011 fetch_fsmap learned FSMap version 27 2024-07-26T13:42:03.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.414+0000 7f4b7cf68740 1 -- 172.21.15.100:0/3060138090 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e97e8bcf60 con 0x55e97e492070 2024-07-26T13:42:03.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.415+0000 7f4b4f7fe640 1 -- 172.21.15.100:0/3060138090 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b64045040 con 0x55e97e492070 2024-07-26T13:42:03.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.415+0000 7f4b4f7fe640 1 -- 172.21.15.100:0/3060138090 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4b6800f040 con 0x55e97e492070 2024-07-26T13:42:03.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.415+0000 7f4b7cf68740 10 client.16011 fetch_fsmap finished waiting for FSMap version 27 2024-07-26T13:42:03.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.415+0000 7f4b7cf68740 10 client.16011 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-26T13:42:03.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.415+0000 7f4b7cf68740 20 client.16011 populate_metadata read hostname 'smithi100' 2024-07-26T13:42:03.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.415+0000 7f4b7cf68740 1 --2- 172.21.15.100:0/3060138090 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55e97e425da0 0x55e97e8bf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:03.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.415+0000 7f4b7cf68740 4 client.16011 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-26T13:42:03.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.415+0000 7f4b7cf68740 1 -- 172.21.15.100:0/3060138090 --> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e97e7ab5a0 con 0x55e97e425da0 2024-07-26T13:42:03.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.415+0000 7f4b6ffff640 1 --2- 172.21.15.100:0/3060138090 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55e97e425da0 0x55e97e8bf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:03.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.416+0000 7f4b4f7fe640 10 client.16011 ms_handle_connect on v2:172.21.15.138:6838/1580925471 2024-07-26T13:42:03.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.416+0000 7f4b6ffff640 1 --2- 172.21.15.100:0/3060138090 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55e97e425da0 0x55e97e8bf750 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f3e922b44e2a3201 server_cookie=365eec0b4a3ad8e5 in_seq=0 out_seq=0 2024-07-26T13:42:03.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.416+0000 7f4b4f7fe640 1 -- 172.21.15.100:0/3060138090 <== mds.1 v2:172.21.15.138:6838/1580925471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e97e7ab5a0 con 0x55e97e425da0 2024-07-26T13:42:03.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.416+0000 7f4b4f7fe640 10 client.16011 handle_command_reply: tid=0 2024-07-26T13:42:03.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.416+0000 7f4b7cf68740 1 client.16011 shutdown 2024-07-26T13:42:03.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.416+0000 7f4b7cf68740 1 -- 172.21.15.100:0/3060138090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e97e492070 msgr2=0x55e97e8b5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:03.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.416+0000 7f4b7cf68740 1 --2- 172.21.15.100:0/3060138090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e97e492070 0x55e97e8b5620 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f4b68009270 tx=0x7f4b680aa6c0 comp rx=0 tx=0).stop 2024-07-26T13:42:03.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.416+0000 7f4b7cf68740 1 -- 172.21.15.100:0/3060138090 shutdown_connections 2024-07-26T13:42:03.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.416+0000 7f4b7cf68740 1 --2- 172.21.15.100:0/3060138090 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55e97e425da0 0x55e97e8bf750 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:03.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.416+0000 7f4b7cf68740 1 --2- 172.21.15.100:0/3060138090 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e97e492070 0x55e97e8b5620 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:03.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.416+0000 7f4b7cf68740 1 --2- 172.21.15.100:0/3060138090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e97e471390 0x55e97e855ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:03.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.416+0000 7f4b7cf68740 1 --2- 172.21.15.100:0/3060138090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e97e4691b0 0x55e97e855910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:03.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.416+0000 7f4b7cf68740 1 -- 172.21.15.100:0/3060138090 >> 172.21.15.100:0/3060138090 conn(0x55e97e486460 msgr2=0x55e97e83c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:03.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.416+0000 7f4b4f7fe640 0 client.16011 ms_handle_reset on v2:172.21.15.138:6838/1580925471 2024-07-26T13:42:03.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.416+0000 7f4b7cf68740 1 -- 172.21.15.100:0/3060138090 shutdown_connections 2024-07-26T13:42:03.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.416+0000 7f4b7cf68740 1 -- 172.21.15.100:0/3060138090 wait complete. 2024-07-26T13:42:03.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.416+0000 7f4b7cf68740 20 client.16011 trim_cache size 0 max 16384 2024-07-26T13:42:03.427 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-07-26T13:42:03.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.417+0000 7f4b769e4640 1 -- 172.21.15.100:0/3029914967 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f4b580b3d40 msgr2=0x7f4b580b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:03.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.417+0000 7f4b769e4640 1 --2- 172.21.15.100:0/3029914967 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f4b580b3d40 0x7f4b580b6250 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x55e97e8361f0 tx=0x7f4b6000b390 comp rx=0 tx=0).stop 2024-07-26T13:42:03.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.417+0000 7f4b769e4640 1 -- 172.21.15.100:0/3029914967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b701087a0 msgr2=0x7f4b7019f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:03.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.417+0000 7f4b769e4640 1 --2- 172.21.15.100:0/3029914967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b701087a0 0x7f4b7019f790 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f4b6800eaa0 tx=0x7f4b6800ef60 comp rx=0 tx=0).stop 2024-07-26T13:42:03.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.417+0000 7f4b769e4640 1 -- 172.21.15.100:0/3029914967 shutdown_connections 2024-07-26T13:42:03.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.417+0000 7f4b769e4640 1 --2- 172.21.15.100:0/3029914967 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f4b580b3d40 0x7f4b580b6250 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:03.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.417+0000 7f4b769e4640 1 --2- 172.21.15.100:0/3029914967 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b701087a0 0x7f4b7019f790 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:03.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.417+0000 7f4b769e4640 1 --2- 172.21.15.100:0/3029914967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b70107e50 0x7f4b7019a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:03.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.417+0000 7f4b769e4640 1 --2- 172.21.15.100:0/3029914967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b70102510 0x7f4b70199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:03.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.417+0000 7f4b769e4640 1 -- 172.21.15.100:0/3029914967 >> 172.21.15.100:0/3029914967 conn(0x7f4b700791a0 msgr2=0x7f4b70105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:03.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.417+0000 7f4b769e4640 1 -- 172.21.15.100:0/3029914967 shutdown_connections 2024-07-26T13:42:03.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:03.417+0000 7f4b769e4640 1 -- 172.21.15.100:0/3029914967 wait complete. 2024-07-26T13:42:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:03 smithi138 ceph-mon[36954]: pgmap v422: 97 pgs: 97 active+clean; 12 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 1.1 MiB/s wr, 161 op/s 2024-07-26T13:42:03.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:03 smithi100 ceph-mon[32599]: pgmap v422: 97 pgs: 97 active+clean; 12 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 1.1 MiB/s wr, 161 op/s 2024-07-26T13:42:04.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:03 smithi167 ceph-mon[37140]: pgmap v422: 97 pgs: 97 active+clean; 12 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 1.1 MiB/s wr, 161 op/s 2024-07-26T13:42:04.403 INFO:tasks.workunit.client.0.smithi100.stdout: 1 20252 6.90 MB/sec warmup 18 sec latency 11723.273 ms 2024-07-26T13:42:04.408 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-26T13:42:04.677 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:42:05.404 INFO:tasks.workunit.client.0.smithi100.stdout: 1 20252 6.54 MB/sec warmup 19 sec latency 12723.306 ms 2024-07-26T13:42:05.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:05 smithi100 ceph-mon[32599]: pgmap v423: 97 pgs: 97 active+clean; 12 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 964 KiB/s wr, 151 op/s 2024-07-26T13:42:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:05 smithi138 ceph-mon[36954]: pgmap v423: 97 pgs: 97 active+clean; 12 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 964 KiB/s wr, 151 op/s 2024-07-26T13:42:06.010 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:42:06.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:05 smithi167 ceph-mon[37140]: pgmap v423: 97 pgs: 97 active+clean; 12 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 964 KiB/s wr, 151 op/s 2024-07-26T13:42:06.403 INFO:tasks.workunit.client.0.smithi100.stdout: 1 20252 6.21 MB/sec warmup 20 sec latency 13723.606 ms 2024-07-26T13:42:06.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.460+0000 7fb176546640 1 -- 172.21.15.100:0/3346032626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb170103590 msgr2=0x7fb1701039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:06.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.460+0000 7fb176546640 1 --2- 172.21.15.100:0/3346032626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb170103590 0x7fb1701039a0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb160009900 tx=0x7fb16002a410 comp rx=0 tx=0).stop 2024-07-26T13:42:06.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.460+0000 7fb176546640 1 -- 172.21.15.100:0/3346032626 shutdown_connections 2024-07-26T13:42:06.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.460+0000 7fb176546640 1 --2- 172.21.15.100:0/3346032626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb170109820 0x7fb170110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:06.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.460+0000 7fb176546640 1 --2- 172.21.15.100:0/3346032626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb170108ed0 0x7fb1701092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:06.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.460+0000 7fb176546640 1 --2- 172.21.15.100:0/3346032626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb170103590 0x7fb1701039a0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:06.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.460+0000 7fb176546640 1 -- 172.21.15.100:0/3346032626 >> 172.21.15.100:0/3346032626 conn(0x7fb17007b280 msgr2=0x7fb17007b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:06.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.460+0000 7fb176546640 1 -- 172.21.15.100:0/3346032626 shutdown_connections 2024-07-26T13:42:06.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.460+0000 7fb176546640 1 -- 172.21.15.100:0/3346032626 wait complete. 2024-07-26T13:42:06.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.461+0000 7fb176546640 1 Processor -- start 2024-07-26T13:42:06.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.461+0000 7fb176546640 1 -- start start 2024-07-26T13:42:06.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.461+0000 7fb176546640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb170103590 0x7fb170199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:06.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.461+0000 7fb176546640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb170108ed0 0x7fb17019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:06.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.461+0000 7fb176546640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb170109820 0x7fb17019f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:06.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.461+0000 7fb176546640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb1701134d0 con 0x7fb170108ed0 2024-07-26T13:42:06.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.461+0000 7fb176546640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb170113350 con 0x7fb170109820 2024-07-26T13:42:06.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.461+0000 7fb176546640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb170113650 con 0x7fb170103590 2024-07-26T13:42:06.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.461+0000 7fb16f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb170108ed0 0x7fb17019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:06.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.462+0000 7fb16f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb170108ed0 0x7fb17019a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55878/0 (socket says 172.21.15.100:55878) 2024-07-26T13:42:06.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.462+0000 7fb16f7fe640 1 -- 172.21.15.100:0/1858321974 learned_addr learned my addr 172.21.15.100:0/1858321974 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:06.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.462+0000 7fb16ffff640 1 --2- 172.21.15.100:0/1858321974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb170103590 0x7fb170199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:06.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.462+0000 7fb174ae3640 1 --2- 172.21.15.100:0/1858321974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb170109820 0x7fb17019f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:06.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.462+0000 7fb16f7fe640 1 -- 172.21.15.100:0/1858321974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb170103590 msgr2=0x7fb170199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:06.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.462+0000 7fb16f7fe640 1 --2- 172.21.15.100:0/1858321974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb170103590 0x7fb170199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:06.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.462+0000 7fb16f7fe640 1 -- 172.21.15.100:0/1858321974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb170109820 msgr2=0x7fb17019f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:06.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.462+0000 7fb16f7fe640 1 --2- 172.21.15.100:0/1858321974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb170109820 0x7fb17019f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:06.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.462+0000 7fb16f7fe640 1 -- 172.21.15.100:0/1858321974 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb17019fd70 con 0x7fb170108ed0 2024-07-26T13:42:06.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.462+0000 7fb16f7fe640 1 --2- 172.21.15.100:0/1858321974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb170108ed0 0x7fb17019a460 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb16400c8c0 tx=0x7fb16400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:06.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.462+0000 7fb16d7fa640 1 -- 172.21.15.100:0/1858321974 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb164007bb0 con 0x7fb170108ed0 2024-07-26T13:42:06.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.462+0000 7fb176546640 1 -- 172.21.15.100:0/1858321974 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1701a0060 con 0x7fb170108ed0 2024-07-26T13:42:06.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.462+0000 7fb16d7fa640 1 -- 172.21.15.100:0/1858321974 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb164007d50 con 0x7fb170108ed0 2024-07-26T13:42:06.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.463+0000 7fb16d7fa640 1 -- 172.21.15.100:0/1858321974 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1640187c0 con 0x7fb170108ed0 2024-07-26T13:42:06.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.463+0000 7fb176546640 1 -- 172.21.15.100:0/1858321974 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1701a6880 con 0x7fb170108ed0 2024-07-26T13:42:06.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.465+0000 7fb16d7fa640 1 -- 172.21.15.100:0/1858321974 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fb16400fa20 con 0x7fb170108ed0 2024-07-26T13:42:06.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.466+0000 7fb16d7fa640 1 --2- 172.21.15.100:0/1858321974 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb1580b3df0 0x7fb1580b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:06.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.466+0000 7fb16d7fa640 1 -- 172.21.15.100:0/1858321974 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fb1640e3f70 con 0x7fb170108ed0 2024-07-26T13:42:06.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.466+0000 7fb17caca740 1 Processor -- start 2024-07-26T13:42:06.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.466+0000 7fb16ffff640 1 --2- 172.21.15.100:0/1858321974 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb1580b3df0 0x7fb1580b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:06.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.466+0000 7fb17caca740 1 -- start start 2024-07-26T13:42:06.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.467+0000 7fb17caca740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56236b9ea070 0x56236bd8f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:06.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.467+0000 7fb17caca740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56236b9c11b0 0x56236bd8fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:06.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.467+0000 7fb17caca740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b9c9390 0x56236bd90110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:06.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.467+0000 7fb17caca740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56236bd05bb0 con 0x56236b9ea070 2024-07-26T13:42:06.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.467+0000 7fb17caca740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56236bcf50b0 con 0x56236b9c9390 2024-07-26T13:42:06.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.467+0000 7fb17caca740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56236bcf90f0 con 0x56236b9c11b0 2024-07-26T13:42:06.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.467+0000 7fb16ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56236b9ea070 0x56236bd8f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:06.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.467+0000 7fb16ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56236b9ea070 0x56236bd8f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55880/0 (socket says 172.21.15.100:55880) 2024-07-26T13:42:06.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.467+0000 7fb16ffff640 1 -- 172.21.15.100:0/2580134588 learned_addr learned my addr 172.21.15.100:0/2580134588 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:06.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.467+0000 7fb16ffff640 1 --2- 172.21.15.100:0/1858321974 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb1580b3df0 0x7fb1580b6300 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb160031440 tx=0x7fb160009560 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:06.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.467+0000 7fb174ae3640 1 --2- 172.21.15.100:0/2580134588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b9c9390 0x56236bd90110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:06.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.467+0000 7fb16f7fe640 1 --2- 172.21.15.100:0/2580134588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56236b9c11b0 0x56236bd8fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:06.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.467+0000 7fb16ffff640 1 -- 172.21.15.100:0/2580134588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56236b9c11b0 msgr2=0x56236bd8fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:06.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.467+0000 7fb16ffff640 1 --2- 172.21.15.100:0/2580134588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56236b9c11b0 0x56236bd8fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:06.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.467+0000 7fb16ffff640 1 -- 172.21.15.100:0/2580134588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b9c9390 msgr2=0x56236bd90110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:06.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.467+0000 7fb16ffff640 1 --2- 172.21.15.100:0/2580134588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b9c9390 0x56236bd90110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:06.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.467+0000 7fb16ffff640 1 -- 172.21.15.100:0/2580134588 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56236bc9bb90 con 0x56236b9ea070 2024-07-26T13:42:06.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.468+0000 7fb16ffff640 1 --2- 172.21.15.100:0/2580134588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56236b9ea070 0x56236bd8f690 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb160009900 tx=0x7fb160041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5fee6030b65d455 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:06.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.468+0000 7fb1567fc640 1 -- 172.21.15.100:0/2580134588 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb160035040 con 0x56236b9ea070 2024-07-26T13:42:06.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.468+0000 7fb1567fc640 1 -- 172.21.15.100:0/2580134588 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb160005040 con 0x56236b9ea070 2024-07-26T13:42:06.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.468+0000 7fb1567fc640 1 -- 172.21.15.100:0/2580134588 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16003eb70 con 0x56236b9ea070 2024-07-26T13:42:06.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.469+0000 7fb17caca740 1 -- 172.21.15.100:0/2580134588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56236b9ea070 msgr2=0x56236bd8f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:06.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.469+0000 7fb17caca740 1 --2- 172.21.15.100:0/2580134588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56236b9ea070 0x56236bd8f690 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb160009900 tx=0x7fb160041000 comp rx=0 tx=0).stop 2024-07-26T13:42:06.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.469+0000 7fb17caca740 1 -- 172.21.15.100:0/2580134588 shutdown_connections 2024-07-26T13:42:06.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.469+0000 7fb17caca740 1 --2- 172.21.15.100:0/2580134588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b9c9390 0x56236bd90110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:06.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.469+0000 7fb17caca740 1 --2- 172.21.15.100:0/2580134588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56236b9c11b0 0x56236bd8fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:06.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.469+0000 7fb17caca740 1 --2- 172.21.15.100:0/2580134588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56236b9ea070 0x56236bd8f690 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:06.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.469+0000 7fb17caca740 1 -- 172.21.15.100:0/2580134588 >> 172.21.15.100:0/2580134588 conn(0x56236b9de460 msgr2=0x56236bd8d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:06.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.469+0000 7fb17caca740 1 -- 172.21.15.100:0/2580134588 shutdown_connections 2024-07-26T13:42:06.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.469+0000 7fb17caca740 1 -- 172.21.15.100:0/2580134588 wait complete. 2024-07-26T13:42:06.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.469+0000 7fb17caca740 1 -- start start 2024-07-26T13:42:06.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.470+0000 7fb17caca740 1 Processor -- start 2024-07-26T13:42:06.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.470+0000 7fb17caca740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56236b9c11b0 0x56236bd9b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:06.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.470+0000 7fb17caca740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b9c9390 0x56236bd9b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:06.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.470+0000 7fb17caca740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56236b9ea070 0x56236bda6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:06.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.470+0000 7fb17caca740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56236bb58700 con 0x56236b9ea070 2024-07-26T13:42:06.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.470+0000 7fb17caca740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56236bc979e0 con 0x56236b9c9390 2024-07-26T13:42:06.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.470+0000 7fb17caca740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56236bd9dc90 con 0x56236b9c11b0 2024-07-26T13:42:06.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.470+0000 7fb17caca740 10 client.0 _ll_register_callbacks cb 0x56236bcb24f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:42:06.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.470+0000 7fb174ae3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56236b9ea070 0x56236bda6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:06.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.470+0000 7fb174ae3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56236b9ea070 0x56236bda6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55886/0 (socket says 172.21.15.100:55886) 2024-07-26T13:42:06.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.470+0000 7fb174ae3640 1 -- 172.21.15.100:0/2451576135 learned_addr learned my addr 172.21.15.100:0/2451576135 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:06.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.470+0000 7fb16ffff640 1 --2- 172.21.15.100:0/2451576135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56236b9c11b0 0x56236bd9b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:06.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.471+0000 7fb16f7fe640 1 --2- 172.21.15.100:0/2451576135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b9c9390 0x56236bd9b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:06.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.471+0000 7fb174ae3640 1 -- 172.21.15.100:0/2451576135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56236b9c11b0 msgr2=0x56236bd9b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:06.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.471+0000 7fb174ae3640 1 --2- 172.21.15.100:0/2451576135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56236b9c11b0 0x56236bd9b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:06.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.471+0000 7fb174ae3640 1 -- 172.21.15.100:0/2451576135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b9c9390 msgr2=0x56236bd9b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:06.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.471+0000 7fb174ae3640 1 --2- 172.21.15.100:0/2451576135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b9c9390 0x56236bd9b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:06.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.471+0000 7fb174ae3640 1 -- 172.21.15.100:0/2451576135 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56236ba55650 con 0x56236b9ea070 2024-07-26T13:42:06.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.471+0000 7fb16f7fe640 1 --2- 172.21.15.100:0/2451576135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b9c9390 0x56236bd9b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:42:06.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.471+0000 7fb174ae3640 1 --2- 172.21.15.100:0/2451576135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56236b9ea070 0x56236bda6fb0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb168011d10 tx=0x7fb16800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ee4b4fc96bb407b server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:06.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.471+0000 7fb14f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:42:06.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.471+0000 7fb14f7fe640 1 -- 172.21.15.100:0/2451576135 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb168003970 con 0x56236b9ea070 2024-07-26T13:42:06.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.471+0000 7fb14f7fe640 1 -- 172.21.15.100:0/2451576135 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fb168017050 con 0x56236b9ea070 2024-07-26T13:42:06.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.471+0000 7fb17caca740 1 -- 172.21.15.100:0/2451576135 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb160005040 con 0x56236b9ea070 2024-07-26T13:42:06.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.471+0000 7fb14f7fe640 1 -- 172.21.15.100:0/2451576135 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb168003b10 con 0x56236b9ea070 2024-07-26T13:42:06.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.472+0000 7fb17caca740 10 client.16023 fetch_fsmap learned FSMap version 27 2024-07-26T13:42:06.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.472+0000 7fb17caca740 1 -- 172.21.15.100:0/2451576135 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56236bda9ce0 con 0x56236b9ea070 2024-07-26T13:42:06.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.472+0000 7fb14f7fe640 1 -- 172.21.15.100:0/2451576135 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb160005040 con 0x56236b9ea070 2024-07-26T13:42:06.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.472+0000 7fb14f7fe640 1 -- 172.21.15.100:0/2451576135 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb168021020 con 0x56236b9ea070 2024-07-26T13:42:06.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.472+0000 7fb17caca740 10 client.16023 fetch_fsmap finished waiting for FSMap version 27 2024-07-26T13:42:06.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.472+0000 7fb17caca740 10 client.16023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:42:06.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.472+0000 7fb17caca740 20 client.16023 populate_metadata read hostname 'smithi100' 2024-07-26T13:42:06.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.473+0000 7fb17caca740 1 --2- 172.21.15.100:0/2451576135 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56236b9a89f0 0x56236bdad370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:06.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.473+0000 7fb17caca740 4 client.16023 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:42:06.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.473+0000 7fb17caca740 1 -- 172.21.15.100:0/2451576135 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56236bd07b00 con 0x56236b9a89f0 2024-07-26T13:42:06.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.473+0000 7fb16ffff640 1 --2- 172.21.15.100:0/2451576135 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56236b9a89f0 0x56236bdad370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:06.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.473+0000 7fb14f7fe640 10 client.16023 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:06.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.473+0000 7fb16ffff640 1 --2- 172.21.15.100:0/2451576135 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56236b9a89f0 0x56236bdad370 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c716e2594274556e server_cookie=e46bbfd5aa5370a7 in_seq=0 out_seq=0 2024-07-26T13:42:06.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.474+0000 7fb14f7fe640 1 -- 172.21.15.100:0/2451576135 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56236bd07b00 con 0x56236b9a89f0 2024-07-26T13:42:06.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.474+0000 7fb14f7fe640 10 client.16023 handle_command_reply: tid=0 2024-07-26T13:42:06.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.474+0000 7fb17caca740 1 client.16023 shutdown 2024-07-26T13:42:06.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.474+0000 7fb17caca740 1 -- 172.21.15.100:0/2451576135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56236b9ea070 msgr2=0x56236bda6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:06.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.474+0000 7fb17caca740 1 --2- 172.21.15.100:0/2451576135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56236b9ea070 0x56236bda6fb0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb168011d10 tx=0x7fb16800f790 comp rx=0 tx=0).stop 2024-07-26T13:42:06.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.474+0000 7fb17caca740 1 -- 172.21.15.100:0/2451576135 shutdown_connections 2024-07-26T13:42:06.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.474+0000 7fb17caca740 1 --2- 172.21.15.100:0/2451576135 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56236b9a89f0 0x56236bdad370 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:06.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.474+0000 7fb17caca740 1 --2- 172.21.15.100:0/2451576135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56236b9ea070 0x56236bda6fb0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:06.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.474+0000 7fb17caca740 1 --2- 172.21.15.100:0/2451576135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b9c9390 0x56236bd9b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:06.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.474+0000 7fb17caca740 1 --2- 172.21.15.100:0/2451576135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56236b9c11b0 0x56236bd9b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:06.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.474+0000 7fb17caca740 1 -- 172.21.15.100:0/2451576135 >> 172.21.15.100:0/2451576135 conn(0x56236b9de460 msgr2=0x56236bd8bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:06.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.474+0000 7fb14f7fe640 0 client.16023 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:06.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.474+0000 7fb17caca740 1 -- 172.21.15.100:0/2451576135 shutdown_connections 2024-07-26T13:42:06.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.474+0000 7fb17caca740 1 -- 172.21.15.100:0/2451576135 wait complete. 2024-07-26T13:42:06.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.475+0000 7fb17caca740 20 client.16023 trim_cache size 0 max 16384 2024-07-26T13:42:06.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.488+0000 7fb17caca740 1 Processor -- start 2024-07-26T13:42:06.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.488+0000 7fb17caca740 1 -- start start 2024-07-26T13:42:06.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.488+0000 7fb17caca740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56236b9c11b0 0x56236bd94eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:06.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.488+0000 7fb17caca740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56236b9c9390 0x56236bda6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:06.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.488+0000 7fb17caca740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b9ea070 0x56236bda72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:06.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.488+0000 7fb17caca740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb1701af780 con 0x56236b9c11b0 2024-07-26T13:42:06.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.488+0000 7fb17caca740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb1701ad050 con 0x56236b9ea070 2024-07-26T13:42:06.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.488+0000 7fb17caca740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb1701aaea0 con 0x56236b9c9390 2024-07-26T13:42:06.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.488+0000 7fb16ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56236b9c11b0 0x56236bd94eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:06.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.488+0000 7fb16ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56236b9c11b0 0x56236bd94eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55892/0 (socket says 172.21.15.100:55892) 2024-07-26T13:42:06.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.488+0000 7fb16ffff640 1 -- 172.21.15.100:0/4056840585 learned_addr learned my addr 172.21.15.100:0/4056840585 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:06.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.488+0000 7fb174ae3640 1 --2- 172.21.15.100:0/4056840585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b9ea070 0x56236bda72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:06.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.488+0000 7fb16f7fe640 1 --2- 172.21.15.100:0/4056840585 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56236b9c9390 0x56236bda6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:06.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.489+0000 7fb16ffff640 1 -- 172.21.15.100:0/4056840585 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56236b9c9390 msgr2=0x56236bda6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:06.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.489+0000 7fb16ffff640 1 --2- 172.21.15.100:0/4056840585 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56236b9c9390 0x56236bda6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:06.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.489+0000 7fb16ffff640 1 -- 172.21.15.100:0/4056840585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b9ea070 msgr2=0x56236bda72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:06.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.489+0000 7fb16ffff640 1 --2- 172.21.15.100:0/4056840585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b9ea070 0x56236bda72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:06.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.489+0000 7fb16ffff640 1 -- 172.21.15.100:0/4056840585 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56236bd8def0 con 0x56236b9c11b0 2024-07-26T13:42:06.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.489+0000 7fb174ae3640 1 --2- 172.21.15.100:0/4056840585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b9ea070 0x56236bda72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:42:06.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.489+0000 7fb16ffff640 1 --2- 172.21.15.100:0/4056840585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56236b9c11b0 0x56236bd94eb0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb16003e7e0 tx=0x7fb16002ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c870904b0a6374c5 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:06.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.489+0000 7fb155ffb640 1 -- 172.21.15.100:0/4056840585 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb160035040 con 0x56236b9c11b0 2024-07-26T13:42:06.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.489+0000 7fb155ffb640 1 -- 172.21.15.100:0/4056840585 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb160062070 con 0x56236b9c11b0 2024-07-26T13:42:06.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.489+0000 7fb155ffb640 1 -- 172.21.15.100:0/4056840585 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16003d430 con 0x56236b9c11b0 2024-07-26T13:42:06.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.490+0000 7fb17caca740 1 -- 172.21.15.100:0/4056840585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56236b9c11b0 msgr2=0x56236bd94eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:06.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.490+0000 7fb17caca740 1 --2- 172.21.15.100:0/4056840585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56236b9c11b0 0x56236bd94eb0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb16003e7e0 tx=0x7fb16002ab10 comp rx=0 tx=0).stop 2024-07-26T13:42:06.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.490+0000 7fb17caca740 1 -- 172.21.15.100:0/4056840585 shutdown_connections 2024-07-26T13:42:06.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.490+0000 7fb17caca740 1 --2- 172.21.15.100:0/4056840585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b9ea070 0x56236bda72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:06.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.490+0000 7fb17caca740 1 --2- 172.21.15.100:0/4056840585 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56236b9c9390 0x56236bda6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:06.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.490+0000 7fb17caca740 1 --2- 172.21.15.100:0/4056840585 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56236b9c11b0 0x56236bd94eb0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:06.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.490+0000 7fb17caca740 1 -- 172.21.15.100:0/4056840585 >> 172.21.15.100:0/4056840585 conn(0x56236b9de460 msgr2=0x56236bd947a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:06.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.490+0000 7fb17caca740 1 -- 172.21.15.100:0/4056840585 shutdown_connections 2024-07-26T13:42:06.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.490+0000 7fb17caca740 1 -- 172.21.15.100:0/4056840585 wait complete. 2024-07-26T13:42:06.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.491+0000 7fb17caca740 1 -- start start 2024-07-26T13:42:06.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.491+0000 7fb17caca740 1 Processor -- start 2024-07-26T13:42:06.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.491+0000 7fb17caca740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56236b9c11b0 0x56236bda3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:06.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.491+0000 7fb17caca740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56236b9c9390 0x56236bda3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:06.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.491+0000 7fb17caca740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b9ea070 0x56236bd9e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:06.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.491+0000 7fb17caca740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb1701ab510 con 0x56236b9c11b0 2024-07-26T13:42:06.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.491+0000 7fb17caca740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb1701ac980 con 0x56236b9ea070 2024-07-26T13:42:06.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.491+0000 7fb17caca740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb1701abbe0 con 0x56236b9c9390 2024-07-26T13:42:06.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.491+0000 7fb17caca740 10 client.0 _ll_register_callbacks cb 0x7fb168009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:42:06.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.491+0000 7fb16ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56236b9c11b0 0x56236bda3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:06.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.491+0000 7fb16ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56236b9c11b0 0x56236bda3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55898/0 (socket says 172.21.15.100:55898) 2024-07-26T13:42:06.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.491+0000 7fb16f7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56236b9c9390 0x56236bda3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:06.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.491+0000 7fb16ffff640 1 -- 172.21.15.100:0/2792567974 learned_addr learned my addr 172.21.15.100:0/2792567974 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:06.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.492+0000 7fb174ae3640 1 --2- 172.21.15.100:0/2792567974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b9ea070 0x56236bd9e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:06.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.492+0000 7fb16ffff640 1 -- 172.21.15.100:0/2792567974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56236b9c9390 msgr2=0x56236bda3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:06.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.492+0000 7fb16ffff640 1 --2- 172.21.15.100:0/2792567974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56236b9c9390 0x56236bda3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:06.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.492+0000 7fb16ffff640 1 -- 172.21.15.100:0/2792567974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b9ea070 msgr2=0x56236bd9e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:06.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.492+0000 7fb16ffff640 1 --2- 172.21.15.100:0/2792567974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b9ea070 0x56236bd9e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:06.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.492+0000 7fb16ffff640 1 -- 172.21.15.100:0/2792567974 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56236bd9f090 con 0x56236b9c11b0 2024-07-26T13:42:06.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.492+0000 7fb16ffff640 1 --2- 172.21.15.100:0/2792567974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56236b9c11b0 0x56236bda3280 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb1600057c0 tx=0x7fb16003d0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9839f111a754ed server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:06.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.492+0000 7fb14f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:42:06.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.492+0000 7fb14f7fe640 1 -- 172.21.15.100:0/2792567974 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb160035040 con 0x56236b9c11b0 2024-07-26T13:42:06.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.492+0000 7fb14f7fe640 1 -- 172.21.15.100:0/2792567974 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fb160004040 con 0x56236b9c11b0 2024-07-26T13:42:06.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.492+0000 7fb17caca740 1 -- 172.21.15.100:0/2792567974 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb160062070 con 0x56236b9c11b0 2024-07-26T13:42:06.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.492+0000 7fb14f7fe640 1 -- 172.21.15.100:0/2792567974 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb16003d5c0 con 0x56236b9c11b0 2024-07-26T13:42:06.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.492+0000 7fb17caca740 10 client.16029 fetch_fsmap learned FSMap version 27 2024-07-26T13:42:06.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.492+0000 7fb17caca740 1 -- 172.21.15.100:0/2792567974 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56236be156e0 con 0x56236b9c11b0 2024-07-26T13:42:06.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.493+0000 7fb14f7fe640 1 -- 172.21.15.100:0/2792567974 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb160062070 con 0x56236b9c11b0 2024-07-26T13:42:06.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.493+0000 7fb14f7fe640 1 -- 172.21.15.100:0/2792567974 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb160045040 con 0x56236b9c11b0 2024-07-26T13:42:06.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.493+0000 7fb17caca740 10 client.16029 fetch_fsmap finished waiting for FSMap version 27 2024-07-26T13:42:06.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.493+0000 7fb17caca740 10 client.16029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:42:06.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.493+0000 7fb17caca740 20 client.16029 populate_metadata read hostname 'smithi100' 2024-07-26T13:42:06.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.493+0000 7fb17caca740 1 --2- 172.21.15.100:0/2792567974 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56236b97dda0 0x56236be17c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:06.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.493+0000 7fb17caca740 4 client.16029 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-26T13:42:06.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.493+0000 7fb17caca740 1 -- 172.21.15.100:0/2792567974 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56236bd035a0 con 0x56236b97dda0 2024-07-26T13:42:06.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.493+0000 7fb16f7fe640 1 --2- 172.21.15.100:0/2792567974 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56236b97dda0 0x56236be17c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:06.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.494+0000 7fb14f7fe640 10 client.16029 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:06.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.494+0000 7fb16f7fe640 1 --2- 172.21.15.100:0/2792567974 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56236b97dda0 0x56236be17c50 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c11248182a307578 server_cookie=b6625b038d03e1c8 in_seq=0 out_seq=0 2024-07-26T13:42:06.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.494+0000 7fb14f7fe640 1 -- 172.21.15.100:0/2792567974 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56236bd035a0 con 0x56236b97dda0 2024-07-26T13:42:06.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.494+0000 7fb14f7fe640 10 client.16029 handle_command_reply: tid=0 2024-07-26T13:42:06.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.494+0000 7fb17caca740 1 client.16029 shutdown 2024-07-26T13:42:06.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.494+0000 7fb17caca740 1 -- 172.21.15.100:0/2792567974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56236b9c11b0 msgr2=0x56236bda3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:06.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.494+0000 7fb17caca740 1 --2- 172.21.15.100:0/2792567974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56236b9c11b0 0x56236bda3280 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb1600057c0 tx=0x7fb16003d0c0 comp rx=0 tx=0).stop 2024-07-26T13:42:06.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.494+0000 7fb17caca740 1 -- 172.21.15.100:0/2792567974 shutdown_connections 2024-07-26T13:42:06.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.494+0000 7fb17caca740 1 --2- 172.21.15.100:0/2792567974 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56236b97dda0 0x56236be17c50 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:06.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.494+0000 7fb17caca740 1 --2- 172.21.15.100:0/2792567974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56236b9ea070 0x56236bd9e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:06.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.494+0000 7fb17caca740 1 --2- 172.21.15.100:0/2792567974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56236b9c9390 0x56236bda3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:06.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.494+0000 7fb14f7fe640 0 client.16029 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:06.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.494+0000 7fb17caca740 1 --2- 172.21.15.100:0/2792567974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56236b9c11b0 0x56236bda3280 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:06.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.494+0000 7fb17caca740 1 -- 172.21.15.100:0/2792567974 >> 172.21.15.100:0/2792567974 conn(0x56236b9de460 msgr2=0x56236bd9b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:06.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.495+0000 7fb17caca740 1 -- 172.21.15.100:0/2792567974 shutdown_connections 2024-07-26T13:42:06.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.495+0000 7fb17caca740 1 -- 172.21.15.100:0/2792567974 wait complete. 2024-07-26T13:42:06.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.495+0000 7fb17caca740 20 client.16029 trim_cache size 0 max 16384 2024-07-26T13:42:06.503 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-07-26T13:42:06.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.495+0000 7fb176546640 1 -- 172.21.15.100:0/1858321974 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb1580b3df0 msgr2=0x7fb1580b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:06.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.495+0000 7fb176546640 1 --2- 172.21.15.100:0/1858321974 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb1580b3df0 0x7fb1580b6300 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb160031440 tx=0x7fb160009560 comp rx=0 tx=0).stop 2024-07-26T13:42:06.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.495+0000 7fb176546640 1 -- 172.21.15.100:0/1858321974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb170108ed0 msgr2=0x7fb17019a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:06.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.495+0000 7fb176546640 1 --2- 172.21.15.100:0/1858321974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb170108ed0 0x7fb17019a460 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb16400c8c0 tx=0x7fb16400cd80 comp rx=0 tx=0).stop 2024-07-26T13:42:06.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.495+0000 7fb176546640 1 -- 172.21.15.100:0/1858321974 shutdown_connections 2024-07-26T13:42:06.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.495+0000 7fb176546640 1 --2- 172.21.15.100:0/1858321974 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb1580b3df0 0x7fb1580b6300 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:06.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.495+0000 7fb176546640 1 --2- 172.21.15.100:0/1858321974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb170109820 0x7fb17019f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:06.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.495+0000 7fb176546640 1 --2- 172.21.15.100:0/1858321974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb170108ed0 0x7fb17019a460 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:06.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.495+0000 7fb176546640 1 --2- 172.21.15.100:0/1858321974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb170103590 0x7fb170199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:06.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.495+0000 7fb176546640 1 -- 172.21.15.100:0/1858321974 >> 172.21.15.100:0/1858321974 conn(0x7fb17007b280 msgr2=0x7fb170106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:06.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.495+0000 7fb176546640 1 -- 172.21.15.100:0/1858321974 shutdown_connections 2024-07-26T13:42:06.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:06.496+0000 7fb176546640 1 -- 172.21.15.100:0/1858321974 wait complete. 2024-07-26T13:42:06.714 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:42:07.403 INFO:tasks.workunit.client.0.smithi100.stdout: 1 20252 5.92 MB/sec warmup 21 sec latency 14723.684 ms 2024-07-26T13:42:07.938 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:07 smithi100 ceph-mon[32599]: pgmap v424: 97 pgs: 97 active+clean; 12 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 348 KiB/s wr, 107 op/s 2024-07-26T13:42:07.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:07 smithi138 ceph-mon[36954]: pgmap v424: 97 pgs: 97 active+clean; 12 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 348 KiB/s wr, 107 op/s 2024-07-26T13:42:08.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:07 smithi167 ceph-mon[37140]: pgmap v424: 97 pgs: 97 active+clean; 12 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 348 KiB/s wr, 107 op/s 2024-07-26T13:42:08.403 INFO:tasks.workunit.client.0.smithi100.stdout: 1 20252 5.65 MB/sec warmup 22 sec latency 15723.727 ms 2024-07-26T13:42:08.970 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-26T13:42:08.970 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-26T13:42:09.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.054+0000 7f9bda664640 1 -- 172.21.15.100:0/3858695389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bd4107610 msgr2=0x7f9bd4107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:09.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.054+0000 7f9bda664640 1 --2- 172.21.15.100:0/3858695389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bd4107610 0x7f9bd4107af0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc009cd0 tx=0x7f9bcc02a8e0 comp rx=0 tx=0).stop 2024-07-26T13:42:09.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.054+0000 7f9bda664640 1 -- 172.21.15.100:0/3858695389 shutdown_connections 2024-07-26T13:42:09.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.054+0000 7f9bda664640 1 --2- 172.21.15.100:0/3858695389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bd4107610 0x7f9bd4107af0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:09.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.054+0000 7f9bda664640 1 --2- 172.21.15.100:0/3858695389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bd4102b20 0x7f9bd4107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:09.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.054+0000 7f9bda664640 1 --2- 172.21.15.100:0/3858695389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bd41021d0 0x7f9bd41025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:09.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.054+0000 7f9bda664640 1 -- 172.21.15.100:0/3858695389 >> 172.21.15.100:0/3858695389 conn(0x7f9bd4079160 msgr2=0x7f9bd4079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:09.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.054+0000 7f9bda664640 1 -- 172.21.15.100:0/3858695389 shutdown_connections 2024-07-26T13:42:09.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.054+0000 7f9bda664640 1 -- 172.21.15.100:0/3858695389 wait complete. 2024-07-26T13:42:09.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.055+0000 7f9bda664640 1 Processor -- start 2024-07-26T13:42:09.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.055+0000 7f9bda664640 1 -- start start 2024-07-26T13:42:09.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.055+0000 7f9bda664640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bd41021d0 0x7f9bd41a0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:09.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.055+0000 7f9bda664640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bd4102b20 0x7f9bd41a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:09.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.055+0000 7f9bda664640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bd4107610 0x7f9bd41a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:09.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.055+0000 7f9bda664640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd4119ab0 con 0x7f9bd4107610 2024-07-26T13:42:09.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.055+0000 7f9bda664640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd4119930 con 0x7f9bd41021d0 2024-07-26T13:42:09.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.056+0000 7f9bda664640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd4119c30 con 0x7f9bd4102b20 2024-07-26T13:42:09.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.056+0000 7f9bd8c01640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bd4107610 0x7f9bd41a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:09.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.056+0000 7f9bd8c01640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bd4107610 0x7f9bd41a5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55920/0 (socket says 172.21.15.100:55920) 2024-07-26T13:42:09.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.056+0000 7f9bd8c01640 1 -- 172.21.15.100:0/3051782797 learned_addr learned my addr 172.21.15.100:0/3051782797 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:09.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.056+0000 7f9bd37fe640 1 --2- 172.21.15.100:0/3051782797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bd4102b20 0x7f9bd41a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:09.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.056+0000 7f9bd3fff640 1 --2- 172.21.15.100:0/3051782797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bd41021d0 0x7f9bd41a0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:09.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.056+0000 7f9bd8c01640 1 -- 172.21.15.100:0/3051782797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bd4102b20 msgr2=0x7f9bd41a0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:09.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.056+0000 7f9bd8c01640 1 --2- 172.21.15.100:0/3051782797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bd4102b20 0x7f9bd41a0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:09.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.056+0000 7f9bd8c01640 1 -- 172.21.15.100:0/3051782797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bd41021d0 msgr2=0x7f9bd41a0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:09.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.056+0000 7f9bd8c01640 1 --2- 172.21.15.100:0/3051782797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bd41021d0 0x7f9bd41a0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:09.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.056+0000 7f9bd8c01640 1 -- 172.21.15.100:0/3051782797 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd41a63f0 con 0x7f9bd4107610 2024-07-26T13:42:09.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.056+0000 7f9bd8c01640 1 --2- 172.21.15.100:0/3051782797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bd4107610 0x7f9bd41a5cc0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc033040 tx=0x7f9bcc003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:09.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.057+0000 7f9bd17fa640 1 -- 172.21.15.100:0/3051782797 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc004040 con 0x7f9bd4107610 2024-07-26T13:42:09.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.057+0000 7f9bda664640 1 -- 172.21.15.100:0/3051782797 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bd41a6680 con 0x7f9bd4107610 2024-07-26T13:42:09.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.057+0000 7f9bd17fa640 1 -- 172.21.15.100:0/3051782797 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bcc004690 con 0x7f9bd4107610 2024-07-26T13:42:09.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.057+0000 7f9bd17fa640 1 -- 172.21.15.100:0/3051782797 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc030e20 con 0x7f9bd4107610 2024-07-26T13:42:09.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.057+0000 7f9bda664640 1 -- 172.21.15.100:0/3051782797 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bd41ace40 con 0x7f9bd4107610 2024-07-26T13:42:09.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.058+0000 7f9bda664640 1 -- 172.21.15.100:0/3051782797 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bd406af90 con 0x7f9bd4107610 2024-07-26T13:42:09.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.059+0000 7f9bd17fa640 1 -- 172.21.15.100:0/3051782797 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f9bcc0305a0 con 0x7f9bd4107610 2024-07-26T13:42:09.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.060+0000 7f9bd17fa640 1 --2- 172.21.15.100:0/3051782797 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9bbc0b3d40 0x7f9bbc0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:09.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.060+0000 7f9bd3fff640 1 --2- 172.21.15.100:0/3051782797 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9bbc0b3d40 0x7f9bbc0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:09.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.060+0000 7f9bd17fa640 1 -- 172.21.15.100:0/3051782797 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f9bcc02e030 con 0x7f9bd4107610 2024-07-26T13:42:09.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.061+0000 7f9bd3fff640 1 --2- 172.21.15.100:0/3051782797 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9bbc0b3d40 0x7f9bbc0b6250 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9bd406b690 tx=0x7f9bc4005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:09.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.065+0000 7f9bd17fa640 1 -- 172.21.15.100:0/3051782797 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f9bcc04b060 con 0x7f9bd4107610 2024-07-26T13:42:09.237 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:42:09.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.288+0000 7f9bda664640 1 -- 172.21.15.100:0/3051782797 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9bd41ad010 con 0x7f9bd4107610 2024-07-26T13:42:09.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.289+0000 7f9bd17fa640 1 -- 172.21.15.100:0/3051782797 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11930 (secure 0 0 0) 0x7f9bcc0cc9f0 con 0x7f9bd4107610 2024-07-26T13:42:09.290 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:42:09.290 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":27,"btime":"2024-07-26T13:41:55:493454+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14790,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/313501409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":313501409},{"type":"v1","addr":"172.21.15.100:6839","nonce":313501409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/1402975506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":1402975506},{"type":"v1","addr":"172.21.15.100:6841","nonce":1402975506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-26T13:37:01.281843+0000","modified":"2024-07-26T13:41:54.710630+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24607,"mds_1":24605,"mds_2":24595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24557":{"gid":24557,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/3290105300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3290105300},{"type":"v1","addr":"172.21.15.138:6835","nonce":3290105300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/3393225483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3393225483},{"type":"v1","addr":"172.21.15.138:6837","nonce":3393225483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24595":{"gid":24595,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6837/3828634468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3828634468},{"type":"v1","addr":"172.21.15.167:6837","nonce":3828634468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.138:6839/1580925471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1580925471},{"type":"v1","addr":"172.21.15.138:6839","nonce":1580925471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/398150514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":398150514},{"type":"v1","addr":"172.21.15.167:6839","nonce":398150514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24607,"qdb_cluster":[24607,24605,24595]},"id":1}]} 2024-07-26T13:42:09.290 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 27 2024-07-26T13:42:09.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.291+0000 7f9bda664640 1 -- 172.21.15.100:0/3051782797 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9bbc0b3d40 msgr2=0x7f9bbc0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:09.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.291+0000 7f9bda664640 1 --2- 172.21.15.100:0/3051782797 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9bbc0b3d40 0x7f9bbc0b6250 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9bd406b690 tx=0x7f9bc4005f90 comp rx=0 tx=0).stop 2024-07-26T13:42:09.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.291+0000 7f9bda664640 1 -- 172.21.15.100:0/3051782797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bd4107610 msgr2=0x7f9bd41a5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:09.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.291+0000 7f9bda664640 1 --2- 172.21.15.100:0/3051782797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bd4107610 0x7f9bd41a5cc0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc033040 tx=0x7f9bcc003970 comp rx=0 tx=0).stop 2024-07-26T13:42:09.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.291+0000 7f9bda664640 1 -- 172.21.15.100:0/3051782797 shutdown_connections 2024-07-26T13:42:09.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.291+0000 7f9bda664640 1 --2- 172.21.15.100:0/3051782797 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9bbc0b3d40 0x7f9bbc0b6250 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:09.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.291+0000 7f9bda664640 1 --2- 172.21.15.100:0/3051782797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9bd4107610 0x7f9bd41a5cc0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:09.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.291+0000 7f9bda664640 1 --2- 172.21.15.100:0/3051782797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bd4102b20 0x7f9bd41a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:09.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.291+0000 7f9bda664640 1 --2- 172.21.15.100:0/3051782797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bd41021d0 0x7f9bd41a0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:09.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.291+0000 7f9bda664640 1 -- 172.21.15.100:0/3051782797 >> 172.21.15.100:0/3051782797 conn(0x7f9bd4079160 msgr2=0x7f9bd4105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:09.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.291+0000 7f9bda664640 1 -- 172.21.15.100:0/3051782797 shutdown_connections 2024-07-26T13:42:09.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:09.291+0000 7f9bda664640 1 -- 172.21.15.100:0/3051782797 wait complete. 2024-07-26T13:42:09.403 INFO:tasks.workunit.client.0.smithi100.stdout: 1 20252 5.40 MB/sec warmup 23 sec latency 16723.778 ms 2024-07-26T13:42:09.785 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:09 smithi100 ceph-mon[32599]: pgmap v425: 97 pgs: 97 active+clean; 12 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 215 KiB/s wr, 106 op/s 2024-07-26T13:42:09.786 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:09 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/3051782797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:42:09.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:09 smithi138 ceph-mon[36954]: pgmap v425: 97 pgs: 97 active+clean; 12 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 215 KiB/s wr, 106 op/s 2024-07-26T13:42:09.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:09 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/3051782797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:42:10.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:09 smithi167 ceph-mon[37140]: pgmap v425: 97 pgs: 97 active+clean; 12 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 215 KiB/s wr, 106 op/s 2024-07-26T13:42:10.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:09 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/3051782797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:42:10.403 INFO:tasks.workunit.client.0.smithi100.stdout: 1 20252 5.18 MB/sec warmup 24 sec latency 17723.860 ms 2024-07-26T13:42:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.625+0000 7fbe6ad99640 1 -- 172.21.15.100:0/2016090865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe64102b20 msgr2=0x7fbe64102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.625+0000 7fbe6ad99640 1 --2- 172.21.15.100:0/2016090865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe64102b20 0x7fbe64102f30 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fbe60009900 tx=0x7fbe6002a410 comp rx=0 tx=0).stop 2024-07-26T13:42:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.625+0000 7fbe6ad99640 1 -- 172.21.15.100:0/2016090865 shutdown_connections 2024-07-26T13:42:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.625+0000 7fbe6ad99640 1 --2- 172.21.15.100:0/2016090865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe64105470 0x7fbe6410eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.625+0000 7fbe6ad99640 1 --2- 172.21.15.100:0/2016090865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe64104b20 0x7fbe64104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.625+0000 7fbe6ad99640 1 --2- 172.21.15.100:0/2016090865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe64102b20 0x7fbe64102f30 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.625+0000 7fbe6ad99640 1 -- 172.21.15.100:0/2016090865 >> 172.21.15.100:0/2016090865 conn(0x7fbe64077500 msgr2=0x7fbe64077940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.625+0000 7fbe6ad99640 1 -- 172.21.15.100:0/2016090865 shutdown_connections 2024-07-26T13:42:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.625+0000 7fbe6ad99640 1 -- 172.21.15.100:0/2016090865 wait complete. 2024-07-26T13:42:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.626+0000 7fbe6ad99640 1 Processor -- start 2024-07-26T13:42:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.626+0000 7fbe6ad99640 1 -- start start 2024-07-26T13:42:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.626+0000 7fbe6ad99640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe64102b20 0x7fbe6419acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.626+0000 7fbe6ad99640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe64104b20 0x7fbe6419b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.626+0000 7fbe6ad99640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe64105470 0x7fbe6419b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.626+0000 7fbe6ad99640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbe64112380 con 0x7fbe64105470 2024-07-26T13:42:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.626+0000 7fbe6ad99640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbe64112200 con 0x7fbe64104b20 2024-07-26T13:42:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.626+0000 7fbe6ad99640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbe64112500 con 0x7fbe64102b20 2024-07-26T13:42:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.626+0000 7fbe69336640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe64105470 0x7fbe6419b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.626+0000 7fbe69336640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe64105470 0x7fbe6419b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55936/0 (socket says 172.21.15.100:55936) 2024-07-26T13:42:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.626+0000 7fbe69336640 1 -- 172.21.15.100:0/239611067 learned_addr learned my addr 172.21.15.100:0/239611067 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.626+0000 7fbe68b35640 1 --2- 172.21.15.100:0/239611067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe64102b20 0x7fbe6419acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.626+0000 7fbe5bfff640 1 --2- 172.21.15.100:0/239611067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe64104b20 0x7fbe6419b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.626+0000 7fbe69336640 1 -- 172.21.15.100:0/239611067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe64102b20 msgr2=0x7fbe6419acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.626+0000 7fbe69336640 1 --2- 172.21.15.100:0/239611067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe64102b20 0x7fbe6419acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.626+0000 7fbe69336640 1 -- 172.21.15.100:0/239611067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe64104b20 msgr2=0x7fbe6419b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.626+0000 7fbe69336640 1 --2- 172.21.15.100:0/239611067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe64104b20 0x7fbe6419b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.627+0000 7fbe69336640 1 -- 172.21.15.100:0/239611067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe641a5f80 con 0x7fbe64105470 2024-07-26T13:42:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.627+0000 7fbe5bfff640 1 --2- 172.21.15.100:0/239611067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe64104b20 0x7fbe6419b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:42:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.627+0000 7fbe69336640 1 --2- 172.21.15.100:0/239611067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe64105470 0x7fbe6419b770 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c00eb10 tx=0x7fbe5c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.627+0000 7fbe59ffb640 1 -- 172.21.15.100:0/239611067 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c004b20 con 0x7fbe64105470 2024-07-26T13:42:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.627+0000 7fbe59ffb640 1 -- 172.21.15.100:0/239611067 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe5c004cc0 con 0x7fbe64105470 2024-07-26T13:42:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.627+0000 7fbe6ad99640 1 -- 172.21.15.100:0/239611067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe641a6270 con 0x7fbe64105470 2024-07-26T13:42:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.627+0000 7fbe59ffb640 1 -- 172.21.15.100:0/239611067 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c010670 con 0x7fbe64105470 2024-07-26T13:42:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.627+0000 7fbe6ad99640 1 -- 172.21.15.100:0/239611067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe641a6770 con 0x7fbe64105470 2024-07-26T13:42:10.637 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.630+0000 7fbe59ffb640 1 -- 172.21.15.100:0/239611067 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fbe5c005020 con 0x7fbe64105470 2024-07-26T13:42:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.630+0000 7fbe71d8c740 1 Processor -- start 2024-07-26T13:42:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.630+0000 7fbe71d8c740 1 -- start start 2024-07-26T13:42:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.630+0000 7fbe71d8c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756b971b0 0x55b756f396b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.630+0000 7fbe71d8c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b756b6e2f0 0x55b756f39bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.630+0000 7fbe71d8c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b756b764d0 0x55b756f3a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.630+0000 7fbe71d8c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55b756eaf3b0 con 0x55b756b764d0 2024-07-26T13:42:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.630+0000 7fbe71d8c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b756e9e8b0 con 0x55b756b971b0 2024-07-26T13:42:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.630+0000 7fbe71d8c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b756ea28f0 con 0x55b756b6e2f0 2024-07-26T13:42:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.630+0000 7fbe5bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b756b764d0 0x55b756f3a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.630+0000 7fbe5bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b756b764d0 0x55b756f3a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55938/0 (socket says 172.21.15.100:55938) 2024-07-26T13:42:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.630+0000 7fbe5bfff640 1 -- 172.21.15.100:0/3082424214 learned_addr learned my addr 172.21.15.100:0/3082424214 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.631+0000 7fbe68b35640 1 --2- 172.21.15.100:0/3082424214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b756b6e2f0 0x55b756f39bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.631+0000 7fbe59ffb640 1 --2- 172.21.15.100:0/239611067 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fbe4c0b3d40 0x7fbe4c0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.631+0000 7fbe5bfff640 1 --2- 172.21.15.100:0/3082424214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756b971b0 0x55b756f396b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.631+0000 7fbe5bfff640 1 -- 172.21.15.100:0/3082424214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b756b6e2f0 msgr2=0x55b756f39bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.631+0000 7fbe5bfff640 1 --2- 172.21.15.100:0/3082424214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b756b6e2f0 0x55b756f39bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.631+0000 7fbe5bfff640 1 -- 172.21.15.100:0/3082424214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756b971b0 msgr2=0x55b756f396b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.631+0000 7fbe5bfff640 1 --2- 172.21.15.100:0/3082424214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756b971b0 0x55b756f396b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.631+0000 7fbe69336640 1 --2- 172.21.15.100:0/239611067 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fbe4c0b3d40 0x7fbe4c0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.631+0000 7fbe5bfff640 1 -- 172.21.15.100:0/3082424214 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b756e459a0 con 0x55b756b764d0 2024-07-26T13:42:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.631+0000 7fbe68b35640 1 --2- 172.21.15.100:0/3082424214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b756b6e2f0 0x55b756f39bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:42:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.631+0000 7fbe59ffb640 1 -- 172.21.15.100:0/239611067 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fbe5c0e4660 con 0x7fbe64105470 2024-07-26T13:42:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.631+0000 7fbe5bfff640 1 --2- 172.21.15.100:0/3082424214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b756b764d0 0x55b756f3a130 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fbe54013630 tx=0x7fbe5400c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5baf8b67554b3fa server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.631+0000 7fbe4affd640 1 -- 172.21.15.100:0/3082424214 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe54019030 con 0x55b756b764d0 2024-07-26T13:42:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.631+0000 7fbe4affd640 1 -- 172.21.15.100:0/3082424214 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe540054d0 con 0x55b756b764d0 2024-07-26T13:42:10.639 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.631+0000 7fbe4affd640 1 -- 172.21.15.100:0/3082424214 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe54011800 con 0x55b756b764d0 2024-07-26T13:42:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.632+0000 7fbe71d8c740 1 -- 172.21.15.100:0/3082424214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b756b764d0 msgr2=0x55b756f3a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.632+0000 7fbe71d8c740 1 --2- 172.21.15.100:0/3082424214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b756b764d0 0x55b756f3a130 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fbe54013630 tx=0x7fbe5400c6f0 comp rx=0 tx=0).stop 2024-07-26T13:42:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.632+0000 7fbe69336640 1 --2- 172.21.15.100:0/239611067 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fbe4c0b3d40 0x7fbe4c0b6250 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c0a8500 tx=0x7fbe5c0789e0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.632+0000 7fbe71d8c740 1 -- 172.21.15.100:0/3082424214 shutdown_connections 2024-07-26T13:42:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.632+0000 7fbe71d8c740 1 --2- 172.21.15.100:0/3082424214 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b756b764d0 0x55b756f3a130 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.632+0000 7fbe71d8c740 1 --2- 172.21.15.100:0/3082424214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b756b6e2f0 0x55b756f39bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.632+0000 7fbe71d8c740 1 --2- 172.21.15.100:0/3082424214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756b971b0 0x55b756f396b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.632+0000 7fbe71d8c740 1 -- 172.21.15.100:0/3082424214 >> 172.21.15.100:0/3082424214 conn(0x55b756b8b300 msgr2=0x55b756f375a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.632+0000 7fbe71d8c740 1 -- 172.21.15.100:0/3082424214 shutdown_connections 2024-07-26T13:42:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.632+0000 7fbe71d8c740 1 -- 172.21.15.100:0/3082424214 wait complete. 2024-07-26T13:42:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.633+0000 7fbe71d8c740 1 -- start start 2024-07-26T13:42:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.633+0000 7fbe71d8c740 1 Processor -- start 2024-07-26T13:42:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.633+0000 7fbe71d8c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756b6e2f0 0x55b756f451e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.633+0000 7fbe71d8c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b756b764d0 0x55b756f45720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.633+0000 7fbe71d8c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b756b971b0 0x55b756f51120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.633+0000 7fbe71d8c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55b756dd8ab0 con 0x55b756b764d0 2024-07-26T13:42:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.633+0000 7fbe71d8c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b756e26cf0 con 0x55b756b6e2f0 2024-07-26T13:42:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.633+0000 7fbe71d8c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b756f47e30 con 0x55b756b971b0 2024-07-26T13:42:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.633+0000 7fbe71d8c740 10 client.0 _ll_register_callbacks cb 0x55b756e983e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:42:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.633+0000 7fbe68b35640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b756b764d0 0x55b756f45720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.633+0000 7fbe68b35640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b756b764d0 0x55b756f45720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55944/0 (socket says 172.21.15.100:55944) 2024-07-26T13:42:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.633+0000 7fbe68b35640 1 -- 172.21.15.100:0/2875095702 learned_addr learned my addr 172.21.15.100:0/2875095702 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.633+0000 7fbe5bfff640 1 --2- 172.21.15.100:0/2875095702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756b6e2f0 0x55b756f451e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.633+0000 7fbe5bfff640 1 --2- 172.21.15.100:0/2875095702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b756b971b0 0x55b756f51120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.634+0000 7fbe68b35640 1 -- 172.21.15.100:0/2875095702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b756b971b0 msgr2=0x55b756f51120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.634+0000 7fbe68b35640 1 --2- 172.21.15.100:0/2875095702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b756b971b0 0x55b756f51120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.634+0000 7fbe68b35640 1 -- 172.21.15.100:0/2875095702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756b6e2f0 msgr2=0x55b756f451e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.634+0000 7fbe68b35640 1 --2- 172.21.15.100:0/2875095702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756b6e2f0 0x55b756f451e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.634+0000 7fbe68b35640 1 -- 172.21.15.100:0/2875095702 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b756c02790 con 0x55b756b764d0 2024-07-26T13:42:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.634+0000 7fbe5bfff640 1 --2- 172.21.15.100:0/2875095702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b756b971b0 0x55b756f51120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:42:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.634+0000 7fbe68b35640 1 --2- 172.21.15.100:0/2875095702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b756b764d0 0x55b756f45720 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fbe60031440 tx=0x7fbe60032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=597868f1fa838e53 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.634+0000 7fbe43fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:42:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.634+0000 7fbe43fff640 1 -- 172.21.15.100:0/2875095702 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe60035040 con 0x55b756b764d0 2024-07-26T13:42:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.634+0000 7fbe43fff640 1 -- 172.21.15.100:0/2875095702 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fbe600316a0 con 0x55b756b764d0 2024-07-26T13:42:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.634+0000 7fbe71d8c740 1 -- 172.21.15.100:0/2875095702 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe540054d0 con 0x55b756b764d0 2024-07-26T13:42:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.634+0000 7fbe43fff640 1 -- 172.21.15.100:0/2875095702 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbe600318a0 con 0x55b756b764d0 2024-07-26T13:42:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.634+0000 7fbe71d8c740 10 client.16047 fetch_fsmap learned FSMap version 27 2024-07-26T13:42:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.634+0000 7fbe71d8c740 1 -- 172.21.15.100:0/2875095702 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b756c7b620 con 0x55b756b764d0 2024-07-26T13:42:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.634+0000 7fbe43fff640 1 -- 172.21.15.100:0/2875095702 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe540054d0 con 0x55b756b764d0 2024-07-26T13:42:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.635+0000 7fbe43fff640 1 -- 172.21.15.100:0/2875095702 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbe6004c020 con 0x55b756b764d0 2024-07-26T13:42:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.635+0000 7fbe71d8c740 10 client.16047 fetch_fsmap finished waiting for FSMap version 27 2024-07-26T13:42:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.635+0000 7fbe71d8c740 10 client.16047 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:42:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.635+0000 7fbe71d8c740 20 client.16047 populate_metadata read hostname 'smithi100' 2024-07-26T13:42:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.635+0000 7fbe71d8c740 1 --2- 172.21.15.100:0/2875095702 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55b756b55d80 0x55b756f57120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.635+0000 7fbe71d8c740 4 client.16047 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:42:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.635+0000 7fbe71d8c740 1 -- 172.21.15.100:0/2875095702 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b756eb1300 con 0x55b756b55d80 2024-07-26T13:42:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.635+0000 7fbe5bfff640 1 --2- 172.21.15.100:0/2875095702 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55b756b55d80 0x55b756f57120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.636+0000 7fbe5bfff640 1 --2- 172.21.15.100:0/2875095702 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55b756b55d80 0x55b756f57120 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=371355f7d0957d7b server_cookie=9ab42e84ebe5ebad in_seq=0 out_seq=0 2024-07-26T13:42:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.636+0000 7fbe43fff640 10 client.16047 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.636+0000 7fbe43fff640 1 -- 172.21.15.100:0/2875095702 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b756eb1300 con 0x55b756b55d80 2024-07-26T13:42:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.636+0000 7fbe43fff640 10 client.16047 handle_command_reply: tid=0 2024-07-26T13:42:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.636+0000 7fbe71d8c740 1 client.16047 shutdown 2024-07-26T13:42:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.636+0000 7fbe71d8c740 1 -- 172.21.15.100:0/2875095702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b756b764d0 msgr2=0x55b756f45720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.636+0000 7fbe71d8c740 1 --2- 172.21.15.100:0/2875095702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b756b764d0 0x55b756f45720 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fbe60031440 tx=0x7fbe60032d50 comp rx=0 tx=0).stop 2024-07-26T13:42:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.636+0000 7fbe71d8c740 1 -- 172.21.15.100:0/2875095702 shutdown_connections 2024-07-26T13:42:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.636+0000 7fbe71d8c740 1 --2- 172.21.15.100:0/2875095702 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55b756b55d80 0x55b756f57120 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.637+0000 7fbe71d8c740 1 --2- 172.21.15.100:0/2875095702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b756b971b0 0x55b756f51120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.637+0000 7fbe71d8c740 1 --2- 172.21.15.100:0/2875095702 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b756b764d0 0x55b756f45720 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.637+0000 7fbe43fff640 0 client.16047 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.637+0000 7fbe71d8c740 1 --2- 172.21.15.100:0/2875095702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756b6e2f0 0x55b756f451e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.637+0000 7fbe71d8c740 1 -- 172.21.15.100:0/2875095702 >> 172.21.15.100:0/2875095702 conn(0x55b756b8b300 msgr2=0x55b756f3a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.637+0000 7fbe71d8c740 1 -- 172.21.15.100:0/2875095702 shutdown_connections 2024-07-26T13:42:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.637+0000 7fbe71d8c740 1 -- 172.21.15.100:0/2875095702 wait complete. 2024-07-26T13:42:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.637+0000 7fbe71d8c740 20 client.16047 trim_cache size 0 max 16384 2024-07-26T13:42:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.652+0000 7fbe71d8c740 1 Processor -- start 2024-07-26T13:42:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.653+0000 7fbe71d8c740 1 -- start start 2024-07-26T13:42:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.653+0000 7fbe71d8c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756b55d80 0x55b756f3adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.653+0000 7fbe71d8c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b756b6e2f0 0x55b756f444a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.653+0000 7fbe71d8c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b756b764d0 0x55b756f449e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.653+0000 7fbe71d8c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbe641b1a20 con 0x55b756b6e2f0 2024-07-26T13:42:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.653+0000 7fbe71d8c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbe641af2f0 con 0x55b756b55d80 2024-07-26T13:42:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.653+0000 7fbe71d8c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbe641ad140 con 0x55b756b764d0 2024-07-26T13:42:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.653+0000 7fbe68b35640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b756b6e2f0 0x55b756f444a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.653+0000 7fbe68b35640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b756b6e2f0 0x55b756f444a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55946/0 (socket says 172.21.15.100:55946) 2024-07-26T13:42:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.653+0000 7fbe68b35640 1 -- 172.21.15.100:0/2017790015 learned_addr learned my addr 172.21.15.100:0/2017790015 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.653+0000 7fbe5bfff640 1 --2- 172.21.15.100:0/2017790015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756b55d80 0x55b756f3adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.653+0000 7fbe5bfff640 1 --2- 172.21.15.100:0/2017790015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b756b764d0 0x55b756f449e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.653+0000 7fbe68b35640 1 -- 172.21.15.100:0/2017790015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b756b764d0 msgr2=0x55b756f449e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.653+0000 7fbe68b35640 1 --2- 172.21.15.100:0/2017790015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b756b764d0 0x55b756f449e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.653+0000 7fbe68b35640 1 -- 172.21.15.100:0/2017790015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756b55d80 msgr2=0x55b756f3adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.653+0000 7fbe68b35640 1 --2- 172.21.15.100:0/2017790015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756b55d80 0x55b756f3adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.653+0000 7fbe68b35640 1 -- 172.21.15.100:0/2017790015 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b756f40820 con 0x55b756b6e2f0 2024-07-26T13:42:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.653+0000 7fbe5bfff640 1 --2- 172.21.15.100:0/2017790015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b756b764d0 0x55b756f449e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:42:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.654+0000 7fbe68b35640 1 --2- 172.21.15.100:0/2017790015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b756b6e2f0 0x55b756f444a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fbe6002a930 tx=0x7fbe60000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c0e823fe8f3a1a server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.654+0000 7fbe4a7fc640 1 -- 172.21.15.100:0/2017790015 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe60035030 con 0x55b756b6e2f0 2024-07-26T13:42:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.654+0000 7fbe4a7fc640 1 -- 172.21.15.100:0/2017790015 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe600316a0 con 0x55b756b6e2f0 2024-07-26T13:42:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.654+0000 7fbe4a7fc640 1 -- 172.21.15.100:0/2017790015 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe60003d50 con 0x55b756b6e2f0 2024-07-26T13:42:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.655+0000 7fbe71d8c740 1 -- 172.21.15.100:0/2017790015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b756b6e2f0 msgr2=0x55b756f444a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.655+0000 7fbe71d8c740 1 --2- 172.21.15.100:0/2017790015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b756b6e2f0 0x55b756f444a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fbe6002a930 tx=0x7fbe60000bc0 comp rx=0 tx=0).stop 2024-07-26T13:42:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.655+0000 7fbe71d8c740 1 -- 172.21.15.100:0/2017790015 shutdown_connections 2024-07-26T13:42:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.655+0000 7fbe71d8c740 1 --2- 172.21.15.100:0/2017790015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b756b764d0 0x55b756f449e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.655+0000 7fbe71d8c740 1 --2- 172.21.15.100:0/2017790015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b756b6e2f0 0x55b756f444a0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.655+0000 7fbe71d8c740 1 --2- 172.21.15.100:0/2017790015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756b55d80 0x55b756f3adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.655+0000 7fbe71d8c740 1 -- 172.21.15.100:0/2017790015 >> 172.21.15.100:0/2017790015 conn(0x55b756b8b300 msgr2=0x55b756f3a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:10.661 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.655+0000 7fbe71d8c740 1 -- 172.21.15.100:0/2017790015 shutdown_connections 2024-07-26T13:42:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.655+0000 7fbe71d8c740 1 -- 172.21.15.100:0/2017790015 wait complete. 2024-07-26T13:42:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.656+0000 7fbe71d8c740 1 -- start start 2024-07-26T13:42:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.656+0000 7fbe71d8c740 1 Processor -- start 2024-07-26T13:42:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.656+0000 7fbe71d8c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b756b55d80 0x55b756f49a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.656+0000 7fbe71d8c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b756b6e2f0 0x55b756f4a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.656+0000 7fbe71d8c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756b764d0 0x55b756fb6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.656+0000 7fbe71d8c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbe641ad7b0 con 0x55b756b6e2f0 2024-07-26T13:42:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.657+0000 7fbe71d8c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbe641aec20 con 0x55b756b764d0 2024-07-26T13:42:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.657+0000 7fbe71d8c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbe641ade80 con 0x55b756b55d80 2024-07-26T13:42:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.657+0000 7fbe71d8c740 10 client.0 _ll_register_callbacks cb 0x7fbe60009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:42:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.657+0000 7fbe68b35640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b756b6e2f0 0x55b756f4a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.657+0000 7fbe68b35640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b756b6e2f0 0x55b756f4a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55954/0 (socket says 172.21.15.100:55954) 2024-07-26T13:42:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.657+0000 7fbe68b35640 1 -- 172.21.15.100:0/2398220999 learned_addr learned my addr 172.21.15.100:0/2398220999 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.657+0000 7fbe5bfff640 1 --2- 172.21.15.100:0/2398220999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b756b55d80 0x55b756f49a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.657+0000 7fbe5bfff640 1 --2- 172.21.15.100:0/2398220999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756b764d0 0x55b756fb6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.657+0000 7fbe68b35640 1 -- 172.21.15.100:0/2398220999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b756b55d80 msgr2=0x55b756f49a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.657+0000 7fbe68b35640 1 --2- 172.21.15.100:0/2398220999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b756b55d80 0x55b756f49a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.657+0000 7fbe68b35640 1 -- 172.21.15.100:0/2398220999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756b764d0 msgr2=0x55b756fb6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.657+0000 7fbe68b35640 1 --2- 172.21.15.100:0/2398220999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756b764d0 0x55b756fb6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.657+0000 7fbe68b35640 1 -- 172.21.15.100:0/2398220999 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b756fb7030 con 0x55b756b6e2f0 2024-07-26T13:42:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.657+0000 7fbe5bfff640 1 --2- 172.21.15.100:0/2398220999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756b764d0 0x55b756fb6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:42:10.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.657+0000 7fbe68b35640 1 --2- 172.21.15.100:0/2398220999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b756b6e2f0 0x55b756f4a010 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fbe60036040 tx=0x7fbe60003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=581d3df76f2af106 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.657+0000 7fbe43fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:42:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.657+0000 7fbe43fff640 1 -- 172.21.15.100:0/2398220999 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe60004fc0 con 0x55b756b6e2f0 2024-07-26T13:42:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.657+0000 7fbe43fff640 1 -- 172.21.15.100:0/2398220999 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fbe60005160 con 0x55b756b6e2f0 2024-07-26T13:42:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.657+0000 7fbe71d8c740 1 -- 172.21.15.100:0/2398220999 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe600316a0 con 0x55b756b6e2f0 2024-07-26T13:42:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.658+0000 7fbe43fff640 1 -- 172.21.15.100:0/2398220999 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbe60067050 con 0x55b756b6e2f0 2024-07-26T13:42:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.658+0000 7fbe71d8c740 10 client.16053 fetch_fsmap learned FSMap version 27 2024-07-26T13:42:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.658+0000 7fbe71d8c740 1 -- 172.21.15.100:0/2398220999 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b756fbe390 con 0x55b756b6e2f0 2024-07-26T13:42:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.658+0000 7fbe43fff640 1 -- 172.21.15.100:0/2398220999 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe600316a0 con 0x55b756b6e2f0 2024-07-26T13:42:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.658+0000 7fbe43fff640 1 -- 172.21.15.100:0/2398220999 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbe6004c070 con 0x55b756b6e2f0 2024-07-26T13:42:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.658+0000 7fbe71d8c740 10 client.16053 fetch_fsmap finished waiting for FSMap version 27 2024-07-26T13:42:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.658+0000 7fbe71d8c740 10 client.16053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:42:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.658+0000 7fbe71d8c740 20 client.16053 populate_metadata read hostname 'smithi100' 2024-07-26T13:42:10.664 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.658+0000 7fbe71d8c740 1 --2- 172.21.15.100:0/2398220999 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55b756b971b0 0x55b756fc0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.658+0000 7fbe71d8c740 4 client.16053 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-26T13:42:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.658+0000 7fbe71d8c740 1 -- 172.21.15.100:0/2398220999 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b756eacda0 con 0x55b756b971b0 2024-07-26T13:42:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.658+0000 7fbe5bfff640 1 --2- 172.21.15.100:0/2398220999 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55b756b971b0 0x55b756fc0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.659+0000 7fbe43fff640 10 client.16053 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:10.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.659+0000 7fbe5bfff640 1 --2- 172.21.15.100:0/2398220999 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55b756b971b0 0x55b756fc0b10 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a36626148105f2a server_cookie=d94f844801b08435 in_seq=0 out_seq=0 2024-07-26T13:42:10.674 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.673+0000 7fbe43fff640 1 -- 172.21.15.100:0/2398220999 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b756eacda0 con 0x55b756b971b0 2024-07-26T13:42:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.673+0000 7fbe43fff640 10 client.16053 handle_command_reply: tid=0 2024-07-26T13:42:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.673+0000 7fbe71d8c740 1 client.16053 shutdown 2024-07-26T13:42:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.673+0000 7fbe71d8c740 1 -- 172.21.15.100:0/2398220999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b756b6e2f0 msgr2=0x55b756f4a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.673+0000 7fbe71d8c740 1 --2- 172.21.15.100:0/2398220999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b756b6e2f0 0x55b756f4a010 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fbe60036040 tx=0x7fbe60003820 comp rx=0 tx=0).stop 2024-07-26T13:42:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.673+0000 7fbe71d8c740 1 -- 172.21.15.100:0/2398220999 shutdown_connections 2024-07-26T13:42:10.675 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.673+0000 7fbe71d8c740 1 --2- 172.21.15.100:0/2398220999 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55b756b971b0 0x55b756fc0b10 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.673+0000 7fbe71d8c740 1 --2- 172.21.15.100:0/2398220999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b756b764d0 0x55b756fb6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.673+0000 7fbe71d8c740 1 --2- 172.21.15.100:0/2398220999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b756b6e2f0 0x55b756f4a010 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.673+0000 7fbe43fff640 0 client.16053 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.673+0000 7fbe71d8c740 1 --2- 172.21.15.100:0/2398220999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b756b55d80 0x55b756f49a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.673+0000 7fbe71d8c740 1 -- 172.21.15.100:0/2398220999 >> 172.21.15.100:0/2398220999 conn(0x55b756b8b300 msgr2=0x55b756f444a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.673+0000 7fbe71d8c740 1 -- 172.21.15.100:0/2398220999 shutdown_connections 2024-07-26T13:42:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.673+0000 7fbe71d8c740 1 -- 172.21.15.100:0/2398220999 wait complete. 2024-07-26T13:42:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.673+0000 7fbe71d8c740 20 client.16053 trim_cache size 0 max 16384 2024-07-26T13:42:10.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-07-26T13:42:10.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-07-26T13:42:10.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-07-26T13:42:10.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-07-26T13:42:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.674+0000 7fbe6ad99640 1 -- 172.21.15.100:0/239611067 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fbe4c0b3d40 msgr2=0x7fbe4c0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.674+0000 7fbe6ad99640 1 --2- 172.21.15.100:0/239611067 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fbe4c0b3d40 0x7fbe4c0b6250 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c0a8500 tx=0x7fbe5c0789e0 comp rx=0 tx=0).stop 2024-07-26T13:42:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.674+0000 7fbe6ad99640 1 -- 172.21.15.100:0/239611067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe64105470 msgr2=0x7fbe6419b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.674+0000 7fbe6ad99640 1 --2- 172.21.15.100:0/239611067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe64105470 0x7fbe6419b770 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c00eb10 tx=0x7fbe5c00c2d0 comp rx=0 tx=0).stop 2024-07-26T13:42:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.674+0000 7fbe6ad99640 1 -- 172.21.15.100:0/239611067 shutdown_connections 2024-07-26T13:42:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.674+0000 7fbe6ad99640 1 --2- 172.21.15.100:0/239611067 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fbe4c0b3d40 0x7fbe4c0b6250 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.674+0000 7fbe6ad99640 1 --2- 172.21.15.100:0/239611067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe64105470 0x7fbe6419b770 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.674+0000 7fbe6ad99640 1 --2- 172.21.15.100:0/239611067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe64104b20 0x7fbe6419b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.674+0000 7fbe6ad99640 1 --2- 172.21.15.100:0/239611067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe64102b20 0x7fbe6419acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.674+0000 7fbe6ad99640 1 -- 172.21.15.100:0/239611067 >> 172.21.15.100:0/239611067 conn(0x7fbe64077500 msgr2=0x7fbe64106a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.674+0000 7fbe6ad99640 1 -- 172.21.15.100:0/239611067 shutdown_connections 2024-07-26T13:42:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:10.674+0000 7fbe6ad99640 1 -- 172.21.15.100:0/239611067 wait complete. 2024-07-26T13:42:11.123 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24607 quiesce db --set-id 437045fa --release --await' 2024-07-26T13:42:11.123 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- timeout 105 ceph tell mds.24607 quiesce db --set-id 437045fa --release --await 2024-07-26T13:42:11.404 INFO:tasks.workunit.client.0.smithi100.stdout: 1 20252 4.97 MB/sec warmup 25 sec latency 18723.931 ms 2024-07-26T13:42:11.689 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:42:11.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.689+0000 7fa01c510640 1 -- 172.21.15.100:0/2618315174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0141024b0 msgr2=0x7fa0141028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:11.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.689+0000 7fa01c510640 1 --2- 172.21.15.100:0/2618315174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0141024b0 0x7fa0141028c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fa004009900 tx=0x7fa00402a410 comp rx=0 tx=0).stop 2024-07-26T13:42:11.690 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:11 smithi100 ceph-mon[32599]: pgmap v426: 97 pgs: 97 active+clean; 12 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 48 KiB/s wr, 70 op/s 2024-07-26T13:42:11.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.689+0000 7fa01c510640 1 -- 172.21.15.100:0/2618315174 shutdown_connections 2024-07-26T13:42:11.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.689+0000 7fa01c510640 1 --2- 172.21.15.100:0/2618315174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa014108740 0x7fa01410ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:11.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.689+0000 7fa01c510640 1 --2- 172.21.15.100:0/2618315174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa014107df0 0x7fa014108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:11.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.689+0000 7fa01c510640 1 --2- 172.21.15.100:0/2618315174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa0141024b0 0x7fa0141028c0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:11.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.689+0000 7fa01c510640 1 -- 172.21.15.100:0/2618315174 >> 172.21.15.100:0/2618315174 conn(0x7fa014079170 msgr2=0x7fa014079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:11.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.689+0000 7fa01c510640 1 -- 172.21.15.100:0/2618315174 shutdown_connections 2024-07-26T13:42:11.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.690+0000 7fa01c510640 1 -- 172.21.15.100:0/2618315174 wait complete. 2024-07-26T13:42:11.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.690+0000 7fa01c510640 1 Processor -- start 2024-07-26T13:42:11.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.690+0000 7fa01c510640 1 -- start start 2024-07-26T13:42:11.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.691+0000 7fa01c510640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0141024b0 0x7fa014103ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:11.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.691+0000 7fa01c510640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa014107df0 0x7fa0141041e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:11.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.691+0000 7fa01c510640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa014108740 0x7fa014104720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:11.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.691+0000 7fa01c510640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa0141123c0 con 0x7fa014107df0 2024-07-26T13:42:11.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.691+0000 7fa01c510640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa014112240 con 0x7fa014108740 2024-07-26T13:42:11.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.691+0000 7fa01c510640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa014112540 con 0x7fa0141024b0 2024-07-26T13:42:11.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.691+0000 7fa019aab640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa014107df0 0x7fa0141041e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:11.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.691+0000 7fa019aab640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa014107df0 0x7fa0141041e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55982/0 (socket says 172.21.15.100:55982) 2024-07-26T13:42:11.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.691+0000 7fa019aab640 1 -- 172.21.15.100:0/2319153448 learned_addr learned my addr 172.21.15.100:0/2319153448 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:11.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.691+0000 7fa01a2ac640 1 --2- 172.21.15.100:0/2319153448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0141024b0 0x7fa014103ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:11.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.691+0000 7fa01aaad640 1 --2- 172.21.15.100:0/2319153448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa014108740 0x7fa014104720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:11.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.691+0000 7fa01a2ac640 1 -- 172.21.15.100:0/2319153448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa014108740 msgr2=0x7fa014104720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:11.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.691+0000 7fa01a2ac640 1 --2- 172.21.15.100:0/2319153448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa014108740 0x7fa014104720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:11.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.691+0000 7fa01a2ac640 1 -- 172.21.15.100:0/2319153448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa014107df0 msgr2=0x7fa0141041e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:11.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.691+0000 7fa01a2ac640 1 --2- 172.21.15.100:0/2319153448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa014107df0 0x7fa0141041e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:11.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.691+0000 7fa01a2ac640 1 -- 172.21.15.100:0/2319153448 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0141a8180 con 0x7fa0141024b0 2024-07-26T13:42:11.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.691+0000 7fa019aab640 1 --2- 172.21.15.100:0/2319153448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa014107df0 0x7fa0141041e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:42:11.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.692+0000 7fa01a2ac640 1 --2- 172.21.15.100:0/2319153448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0141024b0 0x7fa014103ca0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa00402a930 tx=0x7fa004004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:11.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.692+0000 7fa00b7fe640 1 -- 172.21.15.100:0/2319153448 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00403f030 con 0x7fa0141024b0 2024-07-26T13:42:11.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.692+0000 7fa00b7fe640 1 -- 172.21.15.100:0/2319153448 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa004003aa0 con 0x7fa0141024b0 2024-07-26T13:42:11.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.692+0000 7fa01c510640 1 -- 172.21.15.100:0/2319153448 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0141a8410 con 0x7fa0141024b0 2024-07-26T13:42:11.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.692+0000 7fa01c510640 1 -- 172.21.15.100:0/2319153448 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0141a88f0 con 0x7fa0141024b0 2024-07-26T13:42:11.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.692+0000 7fa00b7fe640 1 -- 172.21.15.100:0/2319153448 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa004035070 con 0x7fa0141024b0 2024-07-26T13:42:11.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.695+0000 7fa00b7fe640 1 -- 172.21.15.100:0/2319153448 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fa004042030 con 0x7fa0141024b0 2024-07-26T13:42:11.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.696+0000 7fa00b7fe640 1 --2- 172.21.15.100:0/2319153448 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fa0000b3d40 0x7fa0000b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:11.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.696+0000 7fa019aab640 1 --2- 172.21.15.100:0/2319153448 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fa0000b3d40 0x7fa0000b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:11.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.696+0000 7fa00b7fe640 1 -- 172.21.15.100:0/2319153448 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fa004104590 con 0x7fa0141024b0 2024-07-26T13:42:11.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.696+0000 7fa022a94740 1 Processor -- start 2024-07-26T13:42:11.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.696+0000 7fa022a94740 1 -- start start 2024-07-26T13:42:11.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.696+0000 7fa019aab640 1 --2- 172.21.15.100:0/2319153448 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fa0000b3d40 0x7fa0000b6250 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa0140795c0 tx=0x7fa01000b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:11.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.696+0000 7fa022a94740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ea5f04350 0x561ea62a93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:11.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.696+0000 7fa022a94740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea5edb490 0x561ea62a98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:11.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.696+0000 7fa022a94740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ea5ee3670 0x561ea62a9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:11.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.696+0000 7fa022a94740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x561ea61912f0 con 0x561ea5ee3670 2024-07-26T13:42:11.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.697+0000 7fa022a94740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561ea621fa80 con 0x561ea5f04350 2024-07-26T13:42:11.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.697+0000 7fa022a94740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561ea620ef80 con 0x561ea5edb490 2024-07-26T13:42:11.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.697+0000 7fa01aaad640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ea5ee3670 0x561ea62a9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:11.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.697+0000 7fa01aaad640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ea5ee3670 0x561ea62a9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55988/0 (socket says 172.21.15.100:55988) 2024-07-26T13:42:11.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.697+0000 7fa01aaad640 1 -- 172.21.15.100:0/347429251 learned_addr learned my addr 172.21.15.100:0/347429251 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:11.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.697+0000 7fa019aab640 1 --2- 172.21.15.100:0/347429251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea5edb490 0x561ea62a98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:11.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.697+0000 7fa01a2ac640 1 --2- 172.21.15.100:0/347429251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ea5f04350 0x561ea62a93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:11.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.697+0000 7fa01aaad640 1 -- 172.21.15.100:0/347429251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea5edb490 msgr2=0x561ea62a98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:11.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.697+0000 7fa01aaad640 1 --2- 172.21.15.100:0/347429251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea5edb490 0x561ea62a98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:11.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.697+0000 7fa01aaad640 1 -- 172.21.15.100:0/347429251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ea5f04350 msgr2=0x561ea62a93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:11.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.697+0000 7fa01aaad640 1 --2- 172.21.15.100:0/347429251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ea5f04350 0x561ea62a93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:11.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.697+0000 7fa01aaad640 1 -- 172.21.15.100:0/347429251 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ea5f6f930 con 0x561ea5ee3670 2024-07-26T13:42:11.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.697+0000 7fa01aaad640 1 --2- 172.21.15.100:0/347429251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ea5ee3670 0x561ea62a9e30 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa00c010370 tx=0x7fa00c010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7523a80f1b8abb4 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:11.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.697+0000 7fa008ff9640 1 -- 172.21.15.100:0/347429251 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00c016070 con 0x561ea5ee3670 2024-07-26T13:42:11.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.698+0000 7fa008ff9640 1 -- 172.21.15.100:0/347429251 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa00c004d20 con 0x561ea5ee3670 2024-07-26T13:42:11.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.698+0000 7fa008ff9640 1 -- 172.21.15.100:0/347429251 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00c005350 con 0x561ea5ee3670 2024-07-26T13:42:11.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.698+0000 7fa022a94740 1 -- 172.21.15.100:0/347429251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ea5ee3670 msgr2=0x561ea62a9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:11.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.698+0000 7fa022a94740 1 --2- 172.21.15.100:0/347429251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ea5ee3670 0x561ea62a9e30 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa00c010370 tx=0x7fa00c010830 comp rx=0 tx=0).stop 2024-07-26T13:42:11.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.698+0000 7fa022a94740 1 -- 172.21.15.100:0/347429251 shutdown_connections 2024-07-26T13:42:11.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.699+0000 7fa022a94740 1 --2- 172.21.15.100:0/347429251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ea5ee3670 0x561ea62a9e30 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:11.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.699+0000 7fa022a94740 1 --2- 172.21.15.100:0/347429251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea5edb490 0x561ea62a98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:11.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.699+0000 7fa022a94740 1 --2- 172.21.15.100:0/347429251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ea5f04350 0x561ea62a93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:11.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.699+0000 7fa022a94740 1 -- 172.21.15.100:0/347429251 >> 172.21.15.100:0/347429251 conn(0x561ea5ef8740 msgr2=0x561ea62a72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:11.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.699+0000 7fa022a94740 1 -- 172.21.15.100:0/347429251 shutdown_connections 2024-07-26T13:42:11.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.699+0000 7fa022a94740 1 -- 172.21.15.100:0/347429251 wait complete. 2024-07-26T13:42:11.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.699+0000 7fa022a94740 1 -- start start 2024-07-26T13:42:11.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.699+0000 7fa022a94740 1 Processor -- start 2024-07-26T13:42:11.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.699+0000 7fa022a94740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ea5edb490 0x561ea62b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:11.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.700+0000 7fa022a94740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea5ee3670 0x561ea62b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:11.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.700+0000 7fa022a94740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ea5f04350 0x561ea62c0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:11.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.700+0000 7fa022a94740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x561ea61b14f0 con 0x561ea5edb490 2024-07-26T13:42:11.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.700+0000 7fa022a94740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561ea6212fc0 con 0x561ea5f04350 2024-07-26T13:42:11.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.700+0000 7fa022a94740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561ea6072890 con 0x561ea5ee3670 2024-07-26T13:42:11.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.700+0000 7fa022a94740 10 client.0 _ll_register_callbacks cb 0x561ea61bb500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:42:11.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.700+0000 7fa01a2ac640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ea5edb490 0x561ea62b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:11.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.700+0000 7fa01a2ac640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ea5edb490 0x561ea62b4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55994/0 (socket says 172.21.15.100:55994) 2024-07-26T13:42:11.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.700+0000 7fa01a2ac640 1 -- 172.21.15.100:0/1261889197 learned_addr learned my addr 172.21.15.100:0/1261889197 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:11.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.700+0000 7fa019aab640 1 --2- 172.21.15.100:0/1261889197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea5ee3670 0x561ea62b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:11.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.700+0000 7fa01aaad640 1 --2- 172.21.15.100:0/1261889197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ea5f04350 0x561ea62c0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:11.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.700+0000 7fa01a2ac640 1 -- 172.21.15.100:0/1261889197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea5ee3670 msgr2=0x561ea62b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:11.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.700+0000 7fa01a2ac640 1 --2- 172.21.15.100:0/1261889197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea5ee3670 0x561ea62b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:11.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.700+0000 7fa01a2ac640 1 -- 172.21.15.100:0/1261889197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ea5f04350 msgr2=0x561ea62c0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:11.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.700+0000 7fa01a2ac640 1 --2- 172.21.15.100:0/1261889197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ea5f04350 0x561ea62c0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:11.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.700+0000 7fa01a2ac640 1 -- 172.21.15.100:0/1261889197 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ea61c0c50 con 0x561ea5edb490 2024-07-26T13:42:11.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.700+0000 7fa01aaad640 1 --2- 172.21.15.100:0/1261889197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ea5f04350 0x561ea62c0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:42:11.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.700+0000 7fa01a2ac640 1 --2- 172.21.15.100:0/1261889197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ea5edb490 0x561ea62b4fa0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa0040caf70 tx=0x7fa0040caaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a700770e03114c8a server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:11.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.700+0000 7f9ffdffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:42:11.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.700+0000 7f9ffdffb640 1 -- 172.21.15.100:0/1261889197 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0040e5060 con 0x561ea5edb490 2024-07-26T13:42:11.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.700+0000 7f9ffdffb640 1 -- 172.21.15.100:0/1261889197 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fa004003c50 con 0x561ea5edb490 2024-07-26T13:42:11.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.700+0000 7fa022a94740 1 -- 172.21.15.100:0/1261889197 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa00c004d20 con 0x561ea5edb490 2024-07-26T13:42:11.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.701+0000 7f9ffdffb640 1 -- 172.21.15.100:0/1261889197 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa0040ec050 con 0x561ea5edb490 2024-07-26T13:42:11.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.701+0000 7fa022a94740 10 client.16065 fetch_fsmap learned FSMap version 27 2024-07-26T13:42:11.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.701+0000 7fa022a94740 1 -- 172.21.15.100:0/1261889197 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561ea62c3a70 con 0x561ea5edb490 2024-07-26T13:42:11.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.701+0000 7f9ffdffb640 1 -- 172.21.15.100:0/1261889197 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa00c004d20 con 0x561ea5edb490 2024-07-26T13:42:11.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.701+0000 7f9ffdffb640 1 -- 172.21.15.100:0/1261889197 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa0040cb4d0 con 0x561ea5edb490 2024-07-26T13:42:11.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.701+0000 7fa022a94740 10 client.16065 fetch_fsmap finished waiting for FSMap version 27 2024-07-26T13:42:11.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.701+0000 7fa022a94740 10 client.16065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:42:11.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.701+0000 7fa022a94740 20 client.16065 populate_metadata read hostname 'smithi100' 2024-07-26T13:42:11.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.701+0000 7fa022a94740 1 --2- 172.21.15.100:0/1261889197 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x561ea5ec2cd0 0x561ea62c7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:11.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.701+0000 7fa022a94740 4 client.16065 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:42:11.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.701+0000 7fa022a94740 1 -- 172.21.15.100:0/1261889197 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ea62219d0 con 0x561ea5ec2cd0 2024-07-26T13:42:11.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.701+0000 7fa019aab640 1 --2- 172.21.15.100:0/1261889197 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x561ea5ec2cd0 0x561ea62c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:11.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.702+0000 7f9ffdffb640 10 client.16065 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:11.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.702+0000 7fa019aab640 1 --2- 172.21.15.100:0/1261889197 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x561ea5ec2cd0 0x561ea62c7030 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b6af9b8cab74f42 server_cookie=b889599ca09208dd in_seq=0 out_seq=0 2024-07-26T13:42:11.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.703+0000 7f9ffdffb640 1 -- 172.21.15.100:0/1261889197 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561ea62219d0 con 0x561ea5ec2cd0 2024-07-26T13:42:11.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.703+0000 7f9ffdffb640 10 client.16065 handle_command_reply: tid=0 2024-07-26T13:42:11.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.703+0000 7fa022a94740 1 client.16065 shutdown 2024-07-26T13:42:11.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.703+0000 7fa022a94740 1 -- 172.21.15.100:0/1261889197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ea5edb490 msgr2=0x561ea62b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:11.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.703+0000 7fa022a94740 1 --2- 172.21.15.100:0/1261889197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ea5edb490 0x561ea62b4fa0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa0040caf70 tx=0x7fa0040caaa0 comp rx=0 tx=0).stop 2024-07-26T13:42:11.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.703+0000 7fa022a94740 1 -- 172.21.15.100:0/1261889197 shutdown_connections 2024-07-26T13:42:11.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.703+0000 7fa022a94740 1 --2- 172.21.15.100:0/1261889197 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x561ea5ec2cd0 0x561ea62c7030 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:11.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.703+0000 7fa022a94740 1 --2- 172.21.15.100:0/1261889197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ea5f04350 0x561ea62c0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:11.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.703+0000 7fa022a94740 1 --2- 172.21.15.100:0/1261889197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea5ee3670 0x561ea62b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:11.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.703+0000 7f9ffdffb640 0 client.16065 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:11.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.703+0000 7fa022a94740 1 --2- 172.21.15.100:0/1261889197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ea5edb490 0x561ea62b4fa0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:11.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.703+0000 7fa022a94740 1 -- 172.21.15.100:0/1261889197 >> 172.21.15.100:0/1261889197 conn(0x561ea5ef8740 msgr2=0x561ea62ac590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:11.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.703+0000 7fa022a94740 1 -- 172.21.15.100:0/1261889197 shutdown_connections 2024-07-26T13:42:11.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.703+0000 7fa022a94740 1 -- 172.21.15.100:0/1261889197 wait complete. 2024-07-26T13:42:11.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.703+0000 7fa022a94740 20 client.16065 trim_cache size 0 max 16384 2024-07-26T13:42:11.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.722+0000 7fa022a94740 1 Processor -- start 2024-07-26T13:42:11.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.722+0000 7fa022a94740 1 -- start start 2024-07-26T13:42:11.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.722+0000 7fa022a94740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea5edb490 0x561ea62c0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:11.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.722+0000 7fa022a94740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ea5ee3670 0x561ea62aaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:11.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.722+0000 7fa022a94740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ea5f04350 0x561ea62cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:11.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.722+0000 7fa022a94740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa0141b3bf0 con 0x561ea5f04350 2024-07-26T13:42:11.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.722+0000 7fa022a94740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa0141b14c0 con 0x561ea5ee3670 2024-07-26T13:42:11.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.722+0000 7fa022a94740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa0141af310 con 0x561ea5edb490 2024-07-26T13:42:11.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.722+0000 7fa01aaad640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ea5f04350 0x561ea62cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:11.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.722+0000 7fa01aaad640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ea5f04350 0x561ea62cc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55998/0 (socket says 172.21.15.100:55998) 2024-07-26T13:42:11.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.722+0000 7fa01aaad640 1 -- 172.21.15.100:0/1540955969 learned_addr learned my addr 172.21.15.100:0/1540955969 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:11.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.723+0000 7fa019aab640 1 --2- 172.21.15.100:0/1540955969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ea5ee3670 0x561ea62aaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:11.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.723+0000 7fa01a2ac640 1 --2- 172.21.15.100:0/1540955969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea5edb490 0x561ea62c0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:11.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.723+0000 7fa01aaad640 1 -- 172.21.15.100:0/1540955969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea5edb490 msgr2=0x561ea62c0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:11.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.723+0000 7fa01aaad640 1 --2- 172.21.15.100:0/1540955969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea5edb490 0x561ea62c0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:11.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.723+0000 7fa01aaad640 1 -- 172.21.15.100:0/1540955969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ea5ee3670 msgr2=0x561ea62aaaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:11.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.723+0000 7fa01aaad640 1 --2- 172.21.15.100:0/1540955969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ea5ee3670 0x561ea62aaaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:11.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.723+0000 7fa01aaad640 1 -- 172.21.15.100:0/1540955969 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ea62ac250 con 0x561ea5f04350 2024-07-26T13:42:11.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.723+0000 7fa01a2ac640 1 --2- 172.21.15.100:0/1540955969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea5edb490 0x561ea62c0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:42:11.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.723+0000 7fa019aab640 1 --2- 172.21.15.100:0/1540955969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ea5ee3670 0x561ea62aaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:42:11.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.723+0000 7fa01aaad640 1 --2- 172.21.15.100:0/1540955969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ea5f04350 0x561ea62cc7f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa00c000f80 tx=0x7fa00c009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e1a217a92e3cc2 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:11.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.723+0000 7f9ffffff640 1 -- 172.21.15.100:0/1540955969 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00c016030 con 0x561ea5f04350 2024-07-26T13:42:11.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.723+0000 7f9ffffff640 1 -- 172.21.15.100:0/1540955969 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa00c012510 con 0x561ea5f04350 2024-07-26T13:42:11.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.723+0000 7f9ffffff640 1 -- 172.21.15.100:0/1540955969 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00c005300 con 0x561ea5f04350 2024-07-26T13:42:11.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.724+0000 7fa022a94740 1 -- 172.21.15.100:0/1540955969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ea5f04350 msgr2=0x561ea62cc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:11.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.724+0000 7fa022a94740 1 --2- 172.21.15.100:0/1540955969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ea5f04350 0x561ea62cc7f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa00c000f80 tx=0x7fa00c009d40 comp rx=0 tx=0).stop 2024-07-26T13:42:11.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.724+0000 7fa022a94740 1 -- 172.21.15.100:0/1540955969 shutdown_connections 2024-07-26T13:42:11.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.724+0000 7fa022a94740 1 --2- 172.21.15.100:0/1540955969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ea5f04350 0x561ea62cc7f0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:11.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.724+0000 7fa022a94740 1 --2- 172.21.15.100:0/1540955969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ea5ee3670 0x561ea62aaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:11.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.724+0000 7fa022a94740 1 --2- 172.21.15.100:0/1540955969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea5edb490 0x561ea62c0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:11.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.724+0000 7fa022a94740 1 -- 172.21.15.100:0/1540955969 >> 172.21.15.100:0/1540955969 conn(0x561ea5ef8740 msgr2=0x561ea62aa3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:11.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.724+0000 7fa022a94740 1 -- 172.21.15.100:0/1540955969 shutdown_connections 2024-07-26T13:42:11.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.724+0000 7fa022a94740 1 -- 172.21.15.100:0/1540955969 wait complete. 2024-07-26T13:42:11.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.725+0000 7fa022a94740 1 -- start start 2024-07-26T13:42:11.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.725+0000 7fa022a94740 1 Processor -- start 2024-07-26T13:42:11.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.725+0000 7fa022a94740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ea5edb490 0x561ea62bd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:11.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.725+0000 7fa022a94740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ea5ee3670 0x561ea62b8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:11.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.725+0000 7fa022a94740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea5f04350 0x561ea62b8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:11.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.725+0000 7fa022a94740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa0141af980 con 0x561ea5ee3670 2024-07-26T13:42:11.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.725+0000 7fa022a94740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa0141b0df0 con 0x561ea5edb490 2024-07-26T13:42:11.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.725+0000 7fa022a94740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa0141b0050 con 0x561ea5f04350 2024-07-26T13:42:11.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.725+0000 7fa022a94740 10 client.0 _ll_register_callbacks cb 0x7fa0040cabc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:42:11.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.725+0000 7fa019aab640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ea5ee3670 0x561ea62b8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:11.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.725+0000 7fa019aab640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ea5ee3670 0x561ea62b8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56004/0 (socket says 172.21.15.100:56004) 2024-07-26T13:42:11.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.725+0000 7fa019aab640 1 -- 172.21.15.100:0/3482650363 learned_addr learned my addr 172.21.15.100:0/3482650363 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:11.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.725+0000 7fa01a2ac640 1 --2- 172.21.15.100:0/3482650363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ea5edb490 0x561ea62bd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:11.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.725+0000 7fa01aaad640 1 --2- 172.21.15.100:0/3482650363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea5f04350 0x561ea62b8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:11.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.726+0000 7fa019aab640 1 -- 172.21.15.100:0/3482650363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea5f04350 msgr2=0x561ea62b8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:11.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.726+0000 7fa019aab640 1 --2- 172.21.15.100:0/3482650363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea5f04350 0x561ea62b8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:11.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.726+0000 7fa019aab640 1 -- 172.21.15.100:0/3482650363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ea5edb490 msgr2=0x561ea62bd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:11.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.726+0000 7fa019aab640 1 --2- 172.21.15.100:0/3482650363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ea5edb490 0x561ea62bd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:11.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.726+0000 7fa019aab640 1 -- 172.21.15.100:0/3482650363 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ea62b9730 con 0x561ea5ee3670 2024-07-26T13:42:11.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.726+0000 7fa019aab640 1 --2- 172.21.15.100:0/3482650363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ea5ee3670 0x561ea62b8920 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x561ea62a8030 tx=0x7fa010013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=161e7018f8298a91 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:11.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.726+0000 7f9ffdffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:42:11.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.726+0000 7f9ffdffb640 1 -- 172.21.15.100:0/3482650363 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa01002a580 con 0x561ea5ee3670 2024-07-26T13:42:11.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.726+0000 7f9ffdffb640 1 -- 172.21.15.100:0/3482650363 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fa01002a720 con 0x561ea5ee3670 2024-07-26T13:42:11.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.726+0000 7fa022a94740 1 -- 172.21.15.100:0/3482650363 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa00c012510 con 0x561ea5ee3670 2024-07-26T13:42:11.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.726+0000 7f9ffdffb640 1 -- 172.21.15.100:0/3482650363 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa01002a920 con 0x561ea5ee3670 2024-07-26T13:42:11.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.726+0000 7fa022a94740 10 client.16071 fetch_fsmap learned FSMap version 27 2024-07-26T13:42:11.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.726+0000 7fa022a94740 1 -- 172.21.15.100:0/3482650363 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561ea632f850 con 0x561ea5ee3670 2024-07-26T13:42:11.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.727+0000 7f9ffdffb640 1 -- 172.21.15.100:0/3482650363 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa00c012510 con 0x561ea5ee3670 2024-07-26T13:42:11.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.727+0000 7f9ffdffb640 1 -- 172.21.15.100:0/3482650363 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa01003a020 con 0x561ea5ee3670 2024-07-26T13:42:11.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.727+0000 7fa022a94740 10 client.16071 fetch_fsmap finished waiting for FSMap version 27 2024-07-26T13:42:11.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.727+0000 7fa022a94740 10 client.16071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:42:11.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.727+0000 7fa022a94740 20 client.16071 populate_metadata read hostname 'smithi100' 2024-07-26T13:42:11.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.727+0000 7fa022a94740 1 --2- 172.21.15.100:0/3482650363 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x561ea5e98080 0x561ea6331f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:11.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.727+0000 7fa022a94740 4 client.16071 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-26T13:42:11.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.727+0000 7fa022a94740 1 -- 172.21.15.100:0/3482650363 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561ea621d470 con 0x561ea5e98080 2024-07-26T13:42:11.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.727+0000 7fa01a2ac640 1 --2- 172.21.15.100:0/3482650363 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x561ea5e98080 0x561ea6331f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:11.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.727+0000 7f9ffdffb640 10 client.16071 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:11.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.727+0000 7fa01a2ac640 1 --2- 172.21.15.100:0/3482650363 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x561ea5e98080 0x561ea6331f70 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b7303bb0a298bed server_cookie=f22bf63affb92899 in_seq=0 out_seq=0 2024-07-26T13:42:11.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.728+0000 7f9ffdffb640 1 -- 172.21.15.100:0/3482650363 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561ea621d470 con 0x561ea5e98080 2024-07-26T13:42:11.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.728+0000 7f9ffdffb640 10 client.16071 handle_command_reply: tid=0 2024-07-26T13:42:11.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.728+0000 7fa022a94740 1 client.16071 shutdown 2024-07-26T13:42:11.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.729+0000 7fa022a94740 1 -- 172.21.15.100:0/3482650363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ea5ee3670 msgr2=0x561ea62b8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:11.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.729+0000 7fa022a94740 1 --2- 172.21.15.100:0/3482650363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ea5ee3670 0x561ea62b8920 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x561ea62a8030 tx=0x7fa010013070 comp rx=0 tx=0).stop 2024-07-26T13:42:11.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.729+0000 7fa022a94740 1 -- 172.21.15.100:0/3482650363 shutdown_connections 2024-07-26T13:42:11.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.729+0000 7fa022a94740 1 --2- 172.21.15.100:0/3482650363 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x561ea5e98080 0x561ea6331f70 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:11.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.729+0000 7fa022a94740 1 --2- 172.21.15.100:0/3482650363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea5f04350 0x561ea62b8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:11.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.729+0000 7fa022a94740 1 --2- 172.21.15.100:0/3482650363 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561ea5ee3670 0x561ea62b8920 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:11.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.729+0000 7f9ffdffb640 0 client.16071 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:11.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.729+0000 7fa022a94740 1 --2- 172.21.15.100:0/3482650363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561ea5edb490 0x561ea62bd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:11.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.729+0000 7fa022a94740 1 -- 172.21.15.100:0/3482650363 >> 172.21.15.100:0/3482650363 conn(0x561ea5ef8740 msgr2=0x561ea62a72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:11.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.729+0000 7fa022a94740 1 -- 172.21.15.100:0/3482650363 shutdown_connections 2024-07-26T13:42:11.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.729+0000 7fa022a94740 1 -- 172.21.15.100:0/3482650363 wait complete. 2024-07-26T13:42:11.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.729+0000 7fa022a94740 20 client.16071 trim_cache size 0 max 16384 2024-07-26T13:42:11.735 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:42:11.735 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-07-26T13:42:11.735 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "5fa637e0-5e4e-4c47-a619-4f18b5ba545d", 2024-07-26T13:42:11.735 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-07-26T13:42:11.735 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:42:11.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.729+0000 7fa01c510640 1 -- 172.21.15.100:0/2319153448 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fa0000b3d40 msgr2=0x7fa0000b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:11.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.729+0000 7fa01c510640 1 --2- 172.21.15.100:0/2319153448 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fa0000b3d40 0x7fa0000b6250 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa0140795c0 tx=0x7fa01000b410 comp rx=0 tx=0).stop 2024-07-26T13:42:11.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.729+0000 7fa01c510640 1 -- 172.21.15.100:0/2319153448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0141024b0 msgr2=0x7fa014103ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:11.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.729+0000 7fa01c510640 1 --2- 172.21.15.100:0/2319153448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0141024b0 0x7fa014103ca0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa00402a930 tx=0x7fa004004270 comp rx=0 tx=0).stop 2024-07-26T13:42:11.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.730+0000 7fa01c510640 1 -- 172.21.15.100:0/2319153448 shutdown_connections 2024-07-26T13:42:11.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.730+0000 7fa01c510640 1 --2- 172.21.15.100:0/2319153448 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fa0000b3d40 0x7fa0000b6250 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:11.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.730+0000 7fa01c510640 1 --2- 172.21.15.100:0/2319153448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa014108740 0x7fa014104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:11.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.730+0000 7fa01c510640 1 --2- 172.21.15.100:0/2319153448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa014107df0 0x7fa0141041e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:11.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.730+0000 7fa01c510640 1 --2- 172.21.15.100:0/2319153448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0141024b0 0x7fa014103ca0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:11.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.730+0000 7fa01c510640 1 -- 172.21.15.100:0/2319153448 >> 172.21.15.100:0/2319153448 conn(0x7fa014079170 msgr2=0x7fa014106140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:11.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.730+0000 7fa01c510640 1 -- 172.21.15.100:0/2319153448 shutdown_connections 2024-07-26T13:42:11.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:11.730+0000 7fa01c510640 1 -- 172.21.15.100:0/2319153448 wait complete. 2024-07-26T13:42:11.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:11 smithi138 ceph-mon[36954]: pgmap v426: 97 pgs: 97 active+clean; 12 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 48 KiB/s wr, 70 op/s 2024-07-26T13:42:12.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:11 smithi167 ceph-mon[37140]: pgmap v426: 97 pgs: 97 active+clean; 12 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 48 KiB/s wr, 70 op/s 2024-07-26T13:42:12.404 INFO:tasks.workunit.client.0.smithi100.stdout: 1 20252 4.78 MB/sec warmup 26 sec latency 19724.009 ms 2024-07-26T13:42:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:12 smithi100 ceph-mon[32599]: scrub queued for path: / 2024-07-26T13:42:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:12 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:42:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:12 smithi100 ceph-mon[32599]: scrub summary: active paths [/] 2024-07-26T13:42:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:12 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:42:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:12 smithi138 ceph-mon[36954]: scrub queued for path: / 2024-07-26T13:42:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:12 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:42:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:12 smithi138 ceph-mon[36954]: scrub summary: active paths [/] 2024-07-26T13:42:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:12 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:42:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:12 smithi167 ceph-mon[37140]: scrub queued for path: / 2024-07-26T13:42:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:12 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:42:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:12 smithi167 ceph-mon[37140]: scrub summary: active paths [/] 2024-07-26T13:42:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:12 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:42:13.404 INFO:tasks.workunit.client.0.smithi100.stdout: 1 20252 4.60 MB/sec warmup 27 sec latency 20724.061 ms 2024-07-26T13:42:13.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.729+0000 7fa09960a640 1 -- 172.21.15.100:0/782210521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa094108720 msgr2=0x7fa09410ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:13.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.729+0000 7fa09960a640 1 --2- 172.21.15.100:0/782210521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa094108720 0x7fa09410ef50 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa08c009900 tx=0x7fa08c02a410 comp rx=0 tx=0).stop 2024-07-26T13:42:13.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.729+0000 7fa09960a640 1 -- 172.21.15.100:0/782210521 shutdown_connections 2024-07-26T13:42:13.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.729+0000 7fa09960a640 1 --2- 172.21.15.100:0/782210521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa094108720 0x7fa09410ef50 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:13.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.729+0000 7fa09960a640 1 --2- 172.21.15.100:0/782210521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa094107dd0 0x7fa0941081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:13.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.729+0000 7fa09960a640 1 --2- 172.21.15.100:0/782210521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa094102490 0x7fa0941028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:13.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.729+0000 7fa09960a640 1 -- 172.21.15.100:0/782210521 >> 172.21.15.100:0/782210521 conn(0x7fa094079160 msgr2=0x7fa094079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:13.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.729+0000 7fa09960a640 1 -- 172.21.15.100:0/782210521 shutdown_connections 2024-07-26T13:42:13.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.729+0000 7fa09960a640 1 -- 172.21.15.100:0/782210521 wait complete. 2024-07-26T13:42:13.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.729+0000 7fa09960a640 1 Processor -- start 2024-07-26T13:42:13.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.730+0000 7fa09960a640 1 -- start start 2024-07-26T13:42:13.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.730+0000 7fa09960a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa094102490 0x7fa094104ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:13.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.730+0000 7fa09960a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa094107dd0 0x7fa094105010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:13.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.730+0000 7fa09960a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa094108720 0x7fa094105550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:13.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.730+0000 7fa09960a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa094112380 con 0x7fa094102490 2024-07-26T13:42:13.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.730+0000 7fa09960a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa094112200 con 0x7fa094108720 2024-07-26T13:42:13.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.730+0000 7fa09960a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa094112500 con 0x7fa094107dd0 2024-07-26T13:42:13.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.730+0000 7fa092ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa094102490 0x7fa094104ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:13.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.730+0000 7fa092ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa094102490 0x7fa094104ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56032/0 (socket says 172.21.15.100:56032) 2024-07-26T13:42:13.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.730+0000 7fa092ffd640 1 -- 172.21.15.100:0/2648575681 learned_addr learned my addr 172.21.15.100:0/2648575681 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:13.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.730+0000 7fa0927fc640 1 --2- 172.21.15.100:0/2648575681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa094107dd0 0x7fa094105010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:13.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.730+0000 7fa0937fe640 1 --2- 172.21.15.100:0/2648575681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa094108720 0x7fa094105550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:13.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.730+0000 7fa092ffd640 1 -- 172.21.15.100:0/2648575681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa094107dd0 msgr2=0x7fa094105010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:13.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.730+0000 7fa092ffd640 1 --2- 172.21.15.100:0/2648575681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa094107dd0 0x7fa094105010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:13.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.730+0000 7fa092ffd640 1 -- 172.21.15.100:0/2648575681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa094108720 msgr2=0x7fa094105550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:13.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.730+0000 7fa092ffd640 1 --2- 172.21.15.100:0/2648575681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa094108720 0x7fa094105550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:13.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.730+0000 7fa092ffd640 1 -- 172.21.15.100:0/2648575681 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0941a8100 con 0x7fa094102490 2024-07-26T13:42:13.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.731+0000 7fa092ffd640 1 --2- 172.21.15.100:0/2648575681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa094102490 0x7fa094104ad0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa08400cce0 tx=0x7fa08400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:13.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.731+0000 7fa08bfff640 1 -- 172.21.15.100:0/2648575681 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08400f650 con 0x7fa094102490 2024-07-26T13:42:13.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.731+0000 7fa09960a640 1 -- 172.21.15.100:0/2648575681 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0941a83f0 con 0x7fa094102490 2024-07-26T13:42:13.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.731+0000 7fa08bfff640 1 -- 172.21.15.100:0/2648575681 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa08400fca0 con 0x7fa094102490 2024-07-26T13:42:13.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.731+0000 7fa08bfff640 1 -- 172.21.15.100:0/2648575681 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa084018b40 con 0x7fa094102490 2024-07-26T13:42:13.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.731+0000 7fa09960a640 1 -- 172.21.15.100:0/2648575681 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0941a8980 con 0x7fa094102490 2024-07-26T13:42:13.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.734+0000 7fa09fb8e740 1 Processor -- start 2024-07-26T13:42:13.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.734+0000 7fa09fb8e740 1 -- start start 2024-07-26T13:42:13.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.734+0000 7fa09fb8e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56364a6b6520 0x56364aa5c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:13.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.734+0000 7fa09fb8e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56364a68d660 0x56364aa5c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:13.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.734+0000 7fa08bfff640 1 -- 172.21.15.100:0/2648575681 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fa084018d00 con 0x7fa094102490 2024-07-26T13:42:13.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.735+0000 7fa09fb8e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56364a695840 0x56364aa5caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:13.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.735+0000 7fa09fb8e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56364a9d1e70 con 0x56364a695840 2024-07-26T13:42:13.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.735+0000 7fa09fb8e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56364a9c1370 con 0x56364a68d660 2024-07-26T13:42:13.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.735+0000 7fa09fb8e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56364a9c53b0 con 0x56364a6b6520 2024-07-26T13:42:13.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.735+0000 7fa0927fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56364a695840 0x56364aa5caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:13.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.735+0000 7fa0927fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56364a695840 0x56364aa5caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56038/0 (socket says 172.21.15.100:56038) 2024-07-26T13:42:13.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.735+0000 7fa0927fc640 1 -- 172.21.15.100:0/3918447091 learned_addr learned my addr 172.21.15.100:0/3918447091 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:13.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.735+0000 7fa0937fe640 1 --2- 172.21.15.100:0/3918447091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56364a6b6520 0x56364aa5c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:13.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.735+0000 7fa092ffd640 1 --2- 172.21.15.100:0/3918447091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56364a68d660 0x56364aa5c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:13.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.735+0000 7fa0927fc640 1 -- 172.21.15.100:0/3918447091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56364a6b6520 msgr2=0x56364aa5c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:13.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.735+0000 7fa0927fc640 1 --2- 172.21.15.100:0/3918447091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56364a6b6520 0x56364aa5c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:13.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.735+0000 7fa0927fc640 1 -- 172.21.15.100:0/3918447091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56364a68d660 msgr2=0x56364aa5c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:13.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.735+0000 7fa0927fc640 1 --2- 172.21.15.100:0/3918447091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56364a68d660 0x56364aa5c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:13.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.735+0000 7fa0927fc640 1 -- 172.21.15.100:0/3918447091 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56364a98e040 con 0x56364a695840 2024-07-26T13:42:13.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.735+0000 7fa092ffd640 1 --2- 172.21.15.100:0/3918447091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56364a68d660 0x56364aa5c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:42:13.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.735+0000 7fa0927fc640 1 --2- 172.21.15.100:0/3918447091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56364a695840 0x56364aa5caf0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa07c00bbe0 tx=0x7fa07c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436ae3b0af6a6e45 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:13.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.735+0000 7fa0897fa640 1 -- 172.21.15.100:0/3918447091 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07c015070 con 0x56364a695840 2024-07-26T13:42:13.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.736+0000 7fa0897fa640 1 -- 172.21.15.100:0/3918447091 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa07c0118f0 con 0x56364a695840 2024-07-26T13:42:13.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.736+0000 7fa0897fa640 1 -- 172.21.15.100:0/3918447091 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07c010730 con 0x56364a695840 2024-07-26T13:42:13.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.736+0000 7fa08bfff640 1 --2- 172.21.15.100:0/2648575681 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fa0800b3d40 0x7fa0800b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:13.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.736+0000 7fa0937fe640 1 --2- 172.21.15.100:0/2648575681 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fa0800b3d40 0x7fa0800b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:13.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.736+0000 7fa08bfff640 1 -- 172.21.15.100:0/2648575681 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fa0840e4590 con 0x7fa094102490 2024-07-26T13:42:13.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.736+0000 7fa09fb8e740 1 -- 172.21.15.100:0/3918447091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56364a695840 msgr2=0x56364aa5caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:13.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.736+0000 7fa0937fe640 1 --2- 172.21.15.100:0/2648575681 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fa0800b3d40 0x7fa0800b6250 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa08c005940 tx=0x7fa08c009660 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:13.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.737+0000 7fa09fb8e740 1 --2- 172.21.15.100:0/3918447091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56364a695840 0x56364aa5caf0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa07c00bbe0 tx=0x7fa07c00f730 comp rx=0 tx=0).stop 2024-07-26T13:42:13.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.737+0000 7fa09fb8e740 1 -- 172.21.15.100:0/3918447091 shutdown_connections 2024-07-26T13:42:13.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.737+0000 7fa09fb8e740 1 --2- 172.21.15.100:0/3918447091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56364a695840 0x56364aa5caf0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:13.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.737+0000 7fa09fb8e740 1 --2- 172.21.15.100:0/3918447091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56364a68d660 0x56364aa5c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:13.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.737+0000 7fa09fb8e740 1 --2- 172.21.15.100:0/3918447091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56364a6b6520 0x56364aa5c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:13.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.737+0000 7fa09fb8e740 1 -- 172.21.15.100:0/3918447091 >> 172.21.15.100:0/3918447091 conn(0x56364a6aa910 msgr2=0x56364aa59f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:13.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.737+0000 7fa09fb8e740 1 -- 172.21.15.100:0/3918447091 shutdown_connections 2024-07-26T13:42:13.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.737+0000 7fa09fb8e740 1 -- 172.21.15.100:0/3918447091 wait complete. 2024-07-26T13:42:13.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.737+0000 7fa09fb8e740 1 -- start start 2024-07-26T13:42:13.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.737+0000 7fa09fb8e740 1 Processor -- start 2024-07-26T13:42:13.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.737+0000 7fa09fb8e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56364a68d660 0x56364aa67bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:13.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.737+0000 7fa09fb8e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56364a695840 0x56364aa68100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:13.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.737+0000 7fa09fb8e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56364a6b6520 0x56364aa73b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:13.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.738+0000 7fa09fb8e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56364a824bb0 con 0x56364a695840 2024-07-26T13:42:13.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.738+0000 7fa09fb8e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56364a963ac0 con 0x56364a68d660 2024-07-26T13:42:13.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.738+0000 7fa09fb8e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56364aa6a850 con 0x56364a6b6520 2024-07-26T13:42:13.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.738+0000 7fa09fb8e740 10 client.0 _ll_register_callbacks cb 0x56364a96c3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:42:13.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.738+0000 7fa0927fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56364a695840 0x56364aa68100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:13.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.738+0000 7fa0927fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56364a695840 0x56364aa68100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56054/0 (socket says 172.21.15.100:56054) 2024-07-26T13:42:13.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.738+0000 7fa0927fc640 1 -- 172.21.15.100:0/3552803619 learned_addr learned my addr 172.21.15.100:0/3552803619 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:13.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.738+0000 7fa0937fe640 1 --2- 172.21.15.100:0/3552803619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56364a6b6520 0x56364aa73b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:13.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.738+0000 7fa092ffd640 1 --2- 172.21.15.100:0/3552803619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56364a68d660 0x56364aa67bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:13.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.738+0000 7fa0927fc640 1 -- 172.21.15.100:0/3552803619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56364a6b6520 msgr2=0x56364aa73b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:13.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.738+0000 7fa0927fc640 1 --2- 172.21.15.100:0/3552803619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56364a6b6520 0x56364aa73b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:13.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.738+0000 7fa0927fc640 1 -- 172.21.15.100:0/3552803619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56364a68d660 msgr2=0x56364aa67bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:13.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.738+0000 7fa0927fc640 1 --2- 172.21.15.100:0/3552803619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56364a68d660 0x56364aa67bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:13.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.738+0000 7fa0927fc640 1 -- 172.21.15.100:0/3552803619 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56364a721b00 con 0x56364a695840 2024-07-26T13:42:13.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.738+0000 7fa0927fc640 1 --2- 172.21.15.100:0/3552803619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56364a695840 0x56364aa68100 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa07c010410 tx=0x7fa07c011670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798025123204f3b9 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:13.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.738+0000 7fa07a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:42:13.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.738+0000 7fa07a7fc640 1 -- 172.21.15.100:0/3552803619 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07c028060 con 0x56364a695840 2024-07-26T13:42:13.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.738+0000 7fa07a7fc640 1 -- 172.21.15.100:0/3552803619 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fa07c00bd40 con 0x56364a695840 2024-07-26T13:42:13.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.738+0000 7fa09fb8e740 1 -- 172.21.15.100:0/3552803619 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa07c0118f0 con 0x56364a695840 2024-07-26T13:42:13.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.739+0000 7fa07a7fc640 1 -- 172.21.15.100:0/3552803619 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa07c015070 con 0x56364a695840 2024-07-26T13:42:13.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.739+0000 7fa09fb8e740 10 client.16083 fetch_fsmap learned FSMap version 27 2024-07-26T13:42:13.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.739+0000 7fa09fb8e740 1 -- 172.21.15.100:0/3552803619 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56364a798960 con 0x56364a695840 2024-07-26T13:42:13.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.739+0000 7fa07a7fc640 1 -- 172.21.15.100:0/3552803619 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa07c0118f0 con 0x56364a695840 2024-07-26T13:42:13.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.739+0000 7fa07a7fc640 1 -- 172.21.15.100:0/3552803619 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa07c0047a0 con 0x56364a695840 2024-07-26T13:42:13.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.739+0000 7fa09fb8e740 10 client.16083 fetch_fsmap finished waiting for FSMap version 27 2024-07-26T13:42:13.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.739+0000 7fa09fb8e740 10 client.16083 resolve_mds: validated gid 24607 aka daemon mds.i 2024-07-26T13:42:13.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.739+0000 7fa09fb8e740 20 client.16083 populate_metadata read hostname 'smithi100' 2024-07-26T13:42:13.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.739+0000 7fa09fb8e740 1 --2- 172.21.15.100:0/3552803619 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56364a674ea0 0x56364aa79a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:13.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.739+0000 7fa09fb8e740 4 client.16083 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:42:13.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.739+0000 7fa09fb8e740 1 -- 172.21.15.100:0/3552803619 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56364a9d3dc0 con 0x56364a674ea0 2024-07-26T13:42:13.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.739+0000 7fa092ffd640 1 --2- 172.21.15.100:0/3552803619 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56364a674ea0 0x56364aa79a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:13.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.740+0000 7fa07a7fc640 10 client.16083 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:13.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.740+0000 7fa092ffd640 1 --2- 172.21.15.100:0/3552803619 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56364a674ea0 0x56364aa79a50 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc246f7c8797bdaa server_cookie=ffcf0d3665a08dc9 in_seq=0 out_seq=0 2024-07-26T13:42:13.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.740+0000 7fa07a7fc640 1 -- 172.21.15.100:0/3552803619 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56364a9d3dc0 con 0x56364a674ea0 2024-07-26T13:42:13.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.740+0000 7fa07a7fc640 10 client.16083 handle_command_reply: tid=0 2024-07-26T13:42:13.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.741+0000 7fa09fb8e740 1 client.16083 shutdown 2024-07-26T13:42:13.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.741+0000 7fa09fb8e740 1 -- 172.21.15.100:0/3552803619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56364a695840 msgr2=0x56364aa68100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:13.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.741+0000 7fa09fb8e740 1 --2- 172.21.15.100:0/3552803619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56364a695840 0x56364aa68100 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa07c010410 tx=0x7fa07c011670 comp rx=0 tx=0).stop 2024-07-26T13:42:13.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.741+0000 7fa09fb8e740 1 -- 172.21.15.100:0/3552803619 shutdown_connections 2024-07-26T13:42:13.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.741+0000 7fa09fb8e740 1 --2- 172.21.15.100:0/3552803619 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56364a674ea0 0x56364aa79a50 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:13.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.741+0000 7fa07a7fc640 0 client.16083 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:13.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.741+0000 7fa09fb8e740 1 --2- 172.21.15.100:0/3552803619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56364a6b6520 0x56364aa73b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:13.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.741+0000 7fa09fb8e740 1 --2- 172.21.15.100:0/3552803619 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56364a695840 0x56364aa68100 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:13.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.741+0000 7fa09fb8e740 1 --2- 172.21.15.100:0/3552803619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56364a68d660 0x56364aa67bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:13.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.741+0000 7fa09fb8e740 1 -- 172.21.15.100:0/3552803619 >> 172.21.15.100:0/3552803619 conn(0x56364a6aa910 msgr2=0x56364aa5f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:13.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.741+0000 7fa09fb8e740 1 -- 172.21.15.100:0/3552803619 shutdown_connections 2024-07-26T13:42:13.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.741+0000 7fa09fb8e740 1 -- 172.21.15.100:0/3552803619 wait complete. 2024-07-26T13:42:13.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.741+0000 7fa09fb8e740 20 client.16083 trim_cache size 0 max 16384 2024-07-26T13:42:13.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.758+0000 7fa09fb8e740 1 Processor -- start 2024-07-26T13:42:13.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.758+0000 7fa09fb8e740 1 -- start start 2024-07-26T13:42:13.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.758+0000 7fa09fb8e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56364a674ea0 0x56364aa5a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:13.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.758+0000 7fa09fb8e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56364a68d660 0x56364aa58f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:13.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.758+0000 7fa09fb8e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56364a695840 0x56364aa5f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:13.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.758+0000 7fa09fb8e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa0941b3c00 con 0x56364a695840 2024-07-26T13:42:13.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.758+0000 7fa09fb8e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa0941b14d0 con 0x56364a68d660 2024-07-26T13:42:13.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.758+0000 7fa09fb8e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa0941af320 con 0x56364a674ea0 2024-07-26T13:42:13.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.758+0000 7fa0937fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56364a695840 0x56364aa5f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:13.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.758+0000 7fa0937fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56364a695840 0x56364aa5f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56056/0 (socket says 172.21.15.100:56056) 2024-07-26T13:42:13.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.758+0000 7fa0937fe640 1 -- 172.21.15.100:0/2068797938 learned_addr learned my addr 172.21.15.100:0/2068797938 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:13.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.759+0000 7fa092ffd640 1 --2- 172.21.15.100:0/2068797938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56364a674ea0 0x56364aa5a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:13.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.759+0000 7fa0937fe640 1 -- 172.21.15.100:0/2068797938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56364a674ea0 msgr2=0x56364aa5a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:13.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.759+0000 7fa0927fc640 1 --2- 172.21.15.100:0/2068797938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56364a68d660 0x56364aa58f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:13.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.759+0000 7fa0937fe640 1 --2- 172.21.15.100:0/2068797938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56364a674ea0 0x56364aa5a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:13.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.759+0000 7fa0937fe640 1 -- 172.21.15.100:0/2068797938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56364a68d660 msgr2=0x56364aa58f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:13.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.759+0000 7fa0937fe640 1 --2- 172.21.15.100:0/2068797938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56364a68d660 0x56364aa58f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:13.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.759+0000 7fa0937fe640 1 -- 172.21.15.100:0/2068797938 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56364aa7fc30 con 0x56364a695840 2024-07-26T13:42:13.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.759+0000 7fa092ffd640 1 --2- 172.21.15.100:0/2068797938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56364a674ea0 0x56364aa5a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:42:13.762 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:13 smithi100 ceph-mon[32599]: pgmap v427: 97 pgs: 97 active+clean; 12 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 35 KiB/s wr, 39 op/s 2024-07-26T13:42:13.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.759+0000 7fa0937fe640 1 --2- 172.21.15.100:0/2068797938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56364a695840 0x56364aa5f880 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa08c033000 tx=0x7fa08c03e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ad3d3380675f06 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:13.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.759+0000 7fa088ff9640 1 -- 172.21.15.100:0/2068797938 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08c035040 con 0x56364a695840 2024-07-26T13:42:13.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.759+0000 7fa088ff9640 1 -- 172.21.15.100:0/2068797938 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa08c0043c0 con 0x56364a695840 2024-07-26T13:42:13.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.759+0000 7fa088ff9640 1 -- 172.21.15.100:0/2068797938 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08c004da0 con 0x56364a695840 2024-07-26T13:42:13.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.760+0000 7fa09fb8e740 1 -- 172.21.15.100:0/2068797938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56364a695840 msgr2=0x56364aa5f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:13.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.760+0000 7fa09fb8e740 1 --2- 172.21.15.100:0/2068797938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56364a695840 0x56364aa5f880 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa08c033000 tx=0x7fa08c03e000 comp rx=0 tx=0).stop 2024-07-26T13:42:13.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.760+0000 7fa09fb8e740 1 -- 172.21.15.100:0/2068797938 shutdown_connections 2024-07-26T13:42:13.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.760+0000 7fa09fb8e740 1 --2- 172.21.15.100:0/2068797938 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56364a695840 0x56364aa5f880 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:13.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.760+0000 7fa09fb8e740 1 --2- 172.21.15.100:0/2068797938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56364a68d660 0x56364aa58f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:13.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.760+0000 7fa09fb8e740 1 --2- 172.21.15.100:0/2068797938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56364a674ea0 0x56364aa5a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:13.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.760+0000 7fa09fb8e740 1 -- 172.21.15.100:0/2068797938 >> 172.21.15.100:0/2068797938 conn(0x56364a6aa910 msgr2=0x56364aa5f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:13.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.760+0000 7fa09fb8e740 1 -- 172.21.15.100:0/2068797938 shutdown_connections 2024-07-26T13:42:13.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.760+0000 7fa09fb8e740 1 -- 172.21.15.100:0/2068797938 wait complete. 2024-07-26T13:42:13.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.761+0000 7fa09fb8e740 1 -- start start 2024-07-26T13:42:13.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.761+0000 7fa09fb8e740 1 Processor -- start 2024-07-26T13:42:13.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.761+0000 7fa09fb8e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56364a674ea0 0x56364aa7a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:13.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.761+0000 7fa09fb8e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56364a68d660 0x56364aa7ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:13.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.761+0000 7fa09fb8e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56364a695840 0x56364aa7b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:13.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.761+0000 7fa09fb8e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa0941af990 con 0x56364a674ea0 2024-07-26T13:42:13.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.761+0000 7fa09fb8e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa0941b0e00 con 0x56364a68d660 2024-07-26T13:42:13.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.761+0000 7fa09fb8e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa0941b0060 con 0x56364a695840 2024-07-26T13:42:13.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.761+0000 7fa09fb8e740 10 client.0 _ll_register_callbacks cb 0x7fa08c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:42:13.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.761+0000 7fa092ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56364a674ea0 0x56364aa7a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:13.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.761+0000 7fa092ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56364a674ea0 0x56364aa7a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56072/0 (socket says 172.21.15.100:56072) 2024-07-26T13:42:13.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.761+0000 7fa092ffd640 1 -- 172.21.15.100:0/4172253978 learned_addr learned my addr 172.21.15.100:0/4172253978 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:13.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.761+0000 7fa0927fc640 1 --2- 172.21.15.100:0/4172253978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56364a68d660 0x56364aa7ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:13.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.762+0000 7fa0937fe640 1 --2- 172.21.15.100:0/4172253978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56364a695840 0x56364aa7b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:13.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.762+0000 7fa092ffd640 1 -- 172.21.15.100:0/4172253978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56364a695840 msgr2=0x56364aa7b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:13.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.762+0000 7fa092ffd640 1 --2- 172.21.15.100:0/4172253978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56364a695840 0x56364aa7b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:13.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.762+0000 7fa092ffd640 1 -- 172.21.15.100:0/4172253978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56364a68d660 msgr2=0x56364aa7ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:13.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.762+0000 7fa092ffd640 1 --2- 172.21.15.100:0/4172253978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56364a68d660 0x56364aa7ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:13.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.762+0000 7fa092ffd640 1 -- 172.21.15.100:0/4172253978 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56364aae0fa0 con 0x56364a674ea0 2024-07-26T13:42:13.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.762+0000 7fa0937fe640 1 --2- 172.21.15.100:0/4172253978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56364a695840 0x56364aa7b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:42:13.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.762+0000 7fa092ffd640 1 --2- 172.21.15.100:0/4172253978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56364a674ea0 0x56364aa7a950 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa0840e2300 tx=0x7fa0840aa930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcbdc8c2ec9b4448 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:13.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.762+0000 7fa07a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:42:13.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.762+0000 7fa07a7fc640 1 -- 172.21.15.100:0/4172253978 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0840ab1c0 con 0x56364a674ea0 2024-07-26T13:42:13.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.762+0000 7fa07a7fc640 1 -- 172.21.15.100:0/4172253978 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7fa0840a7450 con 0x56364a674ea0 2024-07-26T13:42:13.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.762+0000 7fa09fb8e740 1 -- 172.21.15.100:0/4172253978 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa08c0043c0 con 0x56364a674ea0 2024-07-26T13:42:13.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.762+0000 7fa07a7fc640 1 -- 172.21.15.100:0/4172253978 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa0840af7c0 con 0x56364a674ea0 2024-07-26T13:42:13.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.762+0000 7fa09fb8e740 10 client.16089 fetch_fsmap learned FSMap version 27 2024-07-26T13:42:13.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.762+0000 7fa09fb8e740 1 -- 172.21.15.100:0/4172253978 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56364aae2210 con 0x56364a674ea0 2024-07-26T13:42:13.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.763+0000 7fa07a7fc640 1 -- 172.21.15.100:0/4172253978 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa08c0043c0 con 0x56364a674ea0 2024-07-26T13:42:13.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.763+0000 7fa07a7fc640 1 -- 172.21.15.100:0/4172253978 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa0840e40d0 con 0x56364a674ea0 2024-07-26T13:42:13.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.763+0000 7fa09fb8e740 10 client.16089 fetch_fsmap finished waiting for FSMap version 27 2024-07-26T13:42:13.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.763+0000 7fa09fb8e740 10 client.16089 resolve_mds: validated gid 24607 aka daemon mds.i 2024-07-26T13:42:13.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.763+0000 7fa09fb8e740 20 client.16089 populate_metadata read hostname 'smithi100' 2024-07-26T13:42:13.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.763+0000 7fa09fb8e740 1 --2- 172.21.15.100:0/4172253978 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56364a6b6520 0x56364aae49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:13.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.763+0000 7fa09fb8e740 4 client.16089 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "437045fa", "release": true, "await": true}] 2024-07-26T13:42:13.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.763+0000 7fa09fb8e740 1 -- 172.21.15.100:0/4172253978 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "quiesce db", "set_id": "437045fa", "release": true, "await": true}) -- 0x56364a9cf860 con 0x56364a6b6520 2024-07-26T13:42:13.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.763+0000 7fa0927fc640 1 --2- 172.21.15.100:0/4172253978 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56364a6b6520 0x56364aae49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:13.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.763+0000 7fa07a7fc640 10 client.16089 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:13.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.763+0000 7fa0927fc640 1 --2- 172.21.15.100:0/4172253978 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56364a6b6520 0x56364aae49e0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ab7097c62c4d723 server_cookie=ea94e5380059acf2 in_seq=0 out_seq=0 2024-07-26T13:42:13.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.785+0000 7fa07a7fc640 1 -- 172.21.15.100:0/4172253978 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x56364a9cf860 con 0x56364a6b6520 2024-07-26T13:42:13.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.785+0000 7fa07a7fc640 10 client.16089 handle_command_reply: tid=0 2024-07-26T13:42:13.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.785+0000 7fa09fb8e740 1 client.16089 shutdown 2024-07-26T13:42:13.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.785+0000 7fa09fb8e740 1 -- 172.21.15.100:0/4172253978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56364a674ea0 msgr2=0x56364aa7a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:13.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.785+0000 7fa09fb8e740 1 --2- 172.21.15.100:0/4172253978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56364a674ea0 0x56364aa7a950 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa0840e2300 tx=0x7fa0840aa930 comp rx=0 tx=0).stop 2024-07-26T13:42:13.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.785+0000 7fa09fb8e740 1 -- 172.21.15.100:0/4172253978 shutdown_connections 2024-07-26T13:42:13.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.785+0000 7fa09fb8e740 1 --2- 172.21.15.100:0/4172253978 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56364a6b6520 0x56364aae49e0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:13.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.785+0000 7fa09fb8e740 1 --2- 172.21.15.100:0/4172253978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56364a695840 0x56364aa7b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:13.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.785+0000 7fa09fb8e740 1 --2- 172.21.15.100:0/4172253978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56364a68d660 0x56364aa7ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:13.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.785+0000 7fa07a7fc640 0 client.16089 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:13.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.785+0000 7fa09fb8e740 1 --2- 172.21.15.100:0/4172253978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56364a674ea0 0x56364aa7a950 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:13.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.785+0000 7fa09fb8e740 1 -- 172.21.15.100:0/4172253978 >> 172.21.15.100:0/4172253978 conn(0x56364a6aa910 msgr2=0x56364aa58d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:13.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.785+0000 7fa09fb8e740 1 -- 172.21.15.100:0/4172253978 shutdown_connections 2024-07-26T13:42:13.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.785+0000 7fa09fb8e740 1 -- 172.21.15.100:0/4172253978 wait complete. 2024-07-26T13:42:13.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.785+0000 7fa09fb8e740 20 client.16089 trim_cache size 0 max 16384 2024-07-26T13:42:13.788 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:42:13.788 INFO:teuthology.orchestra.run.smithi100.stdout: "epoch": 27, 2024-07-26T13:42:13.788 INFO:teuthology.orchestra.run.smithi100.stdout: "leader": 24607, 2024-07-26T13:42:13.789 INFO:teuthology.orchestra.run.smithi100.stdout: "set_version": 8, 2024-07-26T13:42:13.789 INFO:teuthology.orchestra.run.smithi100.stdout: "sets": { 2024-07-26T13:42:13.789 INFO:teuthology.orchestra.run.smithi100.stdout: "437045fa": { 2024-07-26T13:42:13.789 INFO:teuthology.orchestra.run.smithi100.stdout: "version": 8, 2024-07-26T13:42:13.789 INFO:teuthology.orchestra.run.smithi100.stdout: "age_ref": 0.0, 2024-07-26T13:42:13.789 INFO:teuthology.orchestra.run.smithi100.stdout: "state": { 2024-07-26T13:42:13.789 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RELEASED", 2024-07-26T13:42:13.789 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 0.0 2024-07-26T13:42:13.789 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-07-26T13:42:13.789 INFO:teuthology.orchestra.run.smithi100.stdout: "timeout": 90.0, 2024-07-26T13:42:13.789 INFO:teuthology.orchestra.run.smithi100.stdout: "expiration": 142.6, 2024-07-26T13:42:13.789 INFO:teuthology.orchestra.run.smithi100.stdout: "members": { 2024-07-26T13:42:13.789 INFO:teuthology.orchestra.run.smithi100.stdout: "file:/": { 2024-07-26T13:42:13.789 INFO:teuthology.orchestra.run.smithi100.stdout: "excluded": false, 2024-07-26T13:42:13.790 INFO:teuthology.orchestra.run.smithi100.stdout: "state": { 2024-07-26T13:42:13.790 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RELEASED", 2024-07-26T13:42:13.790 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 0.0 2024-07-26T13:42:13.790 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:42:13.790 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:42:13.790 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:42:13.790 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:42:13.790 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:42:13.790 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:42:13.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.786+0000 7fa09960a640 1 -- 172.21.15.100:0/2648575681 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fa0800b3d40 msgr2=0x7fa0800b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:13.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.786+0000 7fa09960a640 1 --2- 172.21.15.100:0/2648575681 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fa0800b3d40 0x7fa0800b6250 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa08c005940 tx=0x7fa08c009660 comp rx=0 tx=0).stop 2024-07-26T13:42:13.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.786+0000 7fa09960a640 1 -- 172.21.15.100:0/2648575681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa094102490 msgr2=0x7fa094104ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:13.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.786+0000 7fa09960a640 1 --2- 172.21.15.100:0/2648575681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa094102490 0x7fa094104ad0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa08400cce0 tx=0x7fa08400a620 comp rx=0 tx=0).stop 2024-07-26T13:42:13.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.786+0000 7fa09960a640 1 -- 172.21.15.100:0/2648575681 shutdown_connections 2024-07-26T13:42:13.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.786+0000 7fa09960a640 1 --2- 172.21.15.100:0/2648575681 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fa0800b3d40 0x7fa0800b6250 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:13.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.786+0000 7fa09960a640 1 --2- 172.21.15.100:0/2648575681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa094108720 0x7fa094105550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:13.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.786+0000 7fa09960a640 1 --2- 172.21.15.100:0/2648575681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa094107dd0 0x7fa094105010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:13.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.786+0000 7fa09960a640 1 --2- 172.21.15.100:0/2648575681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa094102490 0x7fa094104ad0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:13.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.786+0000 7fa09960a640 1 -- 172.21.15.100:0/2648575681 >> 172.21.15.100:0/2648575681 conn(0x7fa094079160 msgr2=0x7fa09406ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:13.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.786+0000 7fa09960a640 1 -- 172.21.15.100:0/2648575681 shutdown_connections 2024-07-26T13:42:13.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:13.786+0000 7fa09960a640 1 -- 172.21.15.100:0/2648575681 wait complete. 2024-07-26T13:42:13.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:13 smithi138 ceph-mon[36954]: pgmap v427: 97 pgs: 97 active+clean; 12 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 35 KiB/s wr, 39 op/s 2024-07-26T13:42:14.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:13 smithi167 ceph-mon[37140]: pgmap v427: 97 pgs: 97 active+clean; 12 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 35 KiB/s wr, 39 op/s 2024-07-26T13:42:14.404 INFO:tasks.workunit.client.0.smithi100.stdout: 1 21866 4.85 MB/sec warmup 28 sec latency 21093.079 ms 2024-07-26T13:42:14.562 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-26T13:42:14.678 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:14 smithi100 ceph-mon[32599]: pgmap v428: 97 pgs: 97 active+clean; 29 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.4 MiB/s wr, 42 op/s 2024-07-26T13:42:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:14 smithi138 ceph-mon[36954]: pgmap v428: 97 pgs: 97 active+clean; 29 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.4 MiB/s wr, 42 op/s 2024-07-26T13:42:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:14 smithi167 ceph-mon[37140]: pgmap v428: 97 pgs: 97 active+clean; 29 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.4 MiB/s wr, 42 op/s 2024-07-26T13:42:15.404 INFO:tasks.workunit.client.0.smithi100.stdout: 1 24934 5.29 MB/sec warmup 29 sec latency 17.411 ms 2024-07-26T13:42:15.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:15 smithi138 ceph-mon[36954]: scrub summary: idle 2024-07-26T13:42:15.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:15 smithi100 ceph-mon[32599]: scrub summary: idle 2024-07-26T13:42:16.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:15 smithi167 ceph-mon[37140]: scrub summary: idle 2024-07-26T13:42:16.404 INFO:tasks.workunit.client.0.smithi100.stdout: 1 25427 5.16 MB/sec warmup 30 sec latency 466.317 ms 2024-07-26T13:42:16.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:16 smithi100 ceph-mon[32599]: pgmap v429: 97 pgs: 97 active+clean; 33 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 2.2 MiB/s wr, 48 op/s 2024-07-26T13:42:16.932 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '437045fa', total seconds elapsed: 33.5 2024-07-26T13:42:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:16 smithi138 ceph-mon[36954]: pgmap v429: 97 pgs: 97 active+clean; 33 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 2.2 MiB/s wr, 48 op/s 2024-07-26T13:42:17.000 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:42:17.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:16 smithi167 ceph-mon[37140]: pgmap v429: 97 pgs: 97 active+clean; 33 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 2.2 MiB/s wr, 48 op/s 2024-07-26T13:42:17.404 INFO:tasks.workunit.client.0.smithi100.stdout: 1 25427 5.00 MB/sec warmup 31 sec latency 1466.357 ms 2024-07-26T13:42:18.404 INFO:tasks.workunit.client.0.smithi100.stdout: 1 25427 4.84 MB/sec warmup 32 sec latency 2466.456 ms 2024-07-26T13:42:18.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.763+0000 7f33e758e640 1 -- 172.21.15.100:0/1998325361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33e0107dd0 msgr2=0x7f33e01081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:18.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.763+0000 7f33e758e640 1 --2- 172.21.15.100:0/1998325361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33e0107dd0 0x7f33e01081e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f33d0009900 tx=0x7f33d002a3f0 comp rx=0 tx=0).stop 2024-07-26T13:42:18.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.763+0000 7f33e758e640 1 -- 172.21.15.100:0/1998325361 shutdown_connections 2024-07-26T13:42:18.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.763+0000 7f33e758e640 1 --2- 172.21.15.100:0/1998325361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33e0108720 0x7f33e010ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:18.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.763+0000 7f33e758e640 1 --2- 172.21.15.100:0/1998325361 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33e0107dd0 0x7f33e01081e0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:18.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.763+0000 7f33e758e640 1 --2- 172.21.15.100:0/1998325361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33e0102490 0x7f33e01028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:18.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.763+0000 7f33e758e640 1 -- 172.21.15.100:0/1998325361 >> 172.21.15.100:0/1998325361 conn(0x7f33e0079160 msgr2=0x7f33e0079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:18.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.763+0000 7f33e758e640 1 -- 172.21.15.100:0/1998325361 shutdown_connections 2024-07-26T13:42:18.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.763+0000 7f33e758e640 1 -- 172.21.15.100:0/1998325361 wait complete. 2024-07-26T13:42:18.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.764+0000 7f33e758e640 1 Processor -- start 2024-07-26T13:42:18.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.764+0000 7f33e758e640 1 -- start start 2024-07-26T13:42:18.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.765+0000 7f33e758e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33e0102490 0x7f33e0199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:18.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.765+0000 7f33e758e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33e0107dd0 0x7f33e019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:18.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.765+0000 7f33e758e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33e0108720 0x7f33e019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:18.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.765+0000 7f33e758e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f33e01134b0 con 0x7f33e0107dd0 2024-07-26T13:42:18.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.765+0000 7f33e758e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f33e0113330 con 0x7f33e0108720 2024-07-26T13:42:18.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.765+0000 7f33e758e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f33e0113630 con 0x7f33e0102490 2024-07-26T13:42:18.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.765+0000 7f33e4b29640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33e0107dd0 0x7f33e019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:18.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.765+0000 7f33e4b29640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33e0107dd0 0x7f33e019a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49362/0 (socket says 172.21.15.100:49362) 2024-07-26T13:42:18.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.765+0000 7f33e4b29640 1 -- 172.21.15.100:0/1584199649 learned_addr learned my addr 172.21.15.100:0/1584199649 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:18.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.765+0000 7f33e532a640 1 --2- 172.21.15.100:0/1584199649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33e0102490 0x7f33e0199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:18.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.765+0000 7f33e5b2b640 1 --2- 172.21.15.100:0/1584199649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33e0108720 0x7f33e019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:18.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.765+0000 7f33e4b29640 1 -- 172.21.15.100:0/1584199649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33e0102490 msgr2=0x7f33e0199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:18.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.765+0000 7f33e4b29640 1 --2- 172.21.15.100:0/1584199649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33e0102490 0x7f33e0199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:18.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.765+0000 7f33e4b29640 1 -- 172.21.15.100:0/1584199649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33e0108720 msgr2=0x7f33e019f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:18.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.765+0000 7f33e4b29640 1 --2- 172.21.15.100:0/1584199649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33e0108720 0x7f33e019f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:18.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.765+0000 7f33e4b29640 1 -- 172.21.15.100:0/1584199649 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33e019fd90 con 0x7f33e0107dd0 2024-07-26T13:42:18.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.765+0000 7f33e5b2b640 1 --2- 172.21.15.100:0/1584199649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33e0108720 0x7f33e019f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:42:18.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.765+0000 7f33e4b29640 1 --2- 172.21.15.100:0/1584199649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33e0107dd0 0x7f33e019a480 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f33d0032fd0 tx=0x7f33d0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:18.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.765+0000 7f33d67fc640 1 -- 172.21.15.100:0/1584199649 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d003f030 con 0x7f33e0107dd0 2024-07-26T13:42:18.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.765+0000 7f33d67fc640 1 -- 172.21.15.100:0/1584199649 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33d0004490 con 0x7f33e0107dd0 2024-07-26T13:42:18.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.765+0000 7f33e758e640 1 -- 172.21.15.100:0/1584199649 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33e01a0020 con 0x7f33e0107dd0 2024-07-26T13:42:18.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.765+0000 7f33d67fc640 1 -- 172.21.15.100:0/1584199649 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d0035070 con 0x7f33e0107dd0 2024-07-26T13:42:18.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.765+0000 7f33e758e640 1 -- 172.21.15.100:0/1584199649 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33e01a0500 con 0x7f33e0107dd0 2024-07-26T13:42:18.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.769+0000 7f33d67fc640 1 -- 172.21.15.100:0/1584199649 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f33d0042030 con 0x7f33e0107dd0 2024-07-26T13:42:18.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.769+0000 7f33edb12740 1 Processor -- start 2024-07-26T13:42:18.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.769+0000 7f33edb12740 1 -- start start 2024-07-26T13:42:18.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.769+0000 7f33edb12740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55978e950070 0x55978ecf5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:18.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.769+0000 7f33edb12740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55978e9271b0 0x55978ecf5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:18.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.769+0000 7f33edb12740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55978e92f390 0x55978ecf5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:18.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.769+0000 7f33d67fc640 1 --2- 172.21.15.100:0/1584199649 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f33cc0b3d70 0x7f33cc0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:18.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.769+0000 7f33edb12740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55978ec6bb60 con 0x55978e950070 2024-07-26T13:42:18.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.769+0000 7f33edb12740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55978ec5b060 con 0x55978e9271b0 2024-07-26T13:42:18.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.769+0000 7f33edb12740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55978ec5f0a0 con 0x55978e92f390 2024-07-26T13:42:18.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.770+0000 7f33e4b29640 1 --2- 172.21.15.100:0/1584199649 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f33cc0b3d70 0x7f33cc0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:18.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.770+0000 7f33e5b2b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55978e950070 0x55978ecf5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:18.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.770+0000 7f33e5b2b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55978e950070 0x55978ecf5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49374/0 (socket says 172.21.15.100:49374) 2024-07-26T13:42:18.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.770+0000 7f33e5b2b640 1 -- 172.21.15.100:0/3419440408 learned_addr learned my addr 172.21.15.100:0/3419440408 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:18.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.770+0000 7f33e532a640 1 --2- 172.21.15.100:0/3419440408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55978e9271b0 0x55978ecf5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:18.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.770+0000 7f33d67fc640 1 -- 172.21.15.100:0/1584199649 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f33d0104560 con 0x7f33e0107dd0 2024-07-26T13:42:18.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.770+0000 7f33e5b2b640 1 --2- 172.21.15.100:0/3419440408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55978e92f390 0x55978ecf5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:18.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.770+0000 7f33e5b2b640 1 -- 172.21.15.100:0/3419440408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55978e92f390 msgr2=0x55978ecf5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:18.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.770+0000 7f33e5b2b640 1 --2- 172.21.15.100:0/3419440408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55978e92f390 0x55978ecf5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:18.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.770+0000 7f33e5b2b640 1 -- 172.21.15.100:0/3419440408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55978e9271b0 msgr2=0x55978ecf5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:18.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.770+0000 7f33e5b2b640 1 --2- 172.21.15.100:0/3419440408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55978e9271b0 0x55978ecf5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:18.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.770+0000 7f33e5b2b640 1 -- 172.21.15.100:0/3419440408 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55978ec01b40 con 0x55978e950070 2024-07-26T13:42:18.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.770+0000 7f33e5b2b640 1 --2- 172.21.15.100:0/3419440408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55978e950070 0x55978ecf5520 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f33d8009f70 tx=0x7f33d8009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7420a7fec769cc13 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:18.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.770+0000 7f33cb7fe640 1 -- 172.21.15.100:0/3419440408 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d8019030 con 0x55978e950070 2024-07-26T13:42:18.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.770+0000 7f33e4b29640 1 --2- 172.21.15.100:0/1584199649 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f33cc0b3d70 0x7f33cc0b6280 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f33d01022d0 tx=0x7f33d00caf60 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:18.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.771+0000 7f33cb7fe640 1 -- 172.21.15.100:0/3419440408 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33d8013d50 con 0x55978e950070 2024-07-26T13:42:18.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.771+0000 7f33cb7fe640 1 -- 172.21.15.100:0/3419440408 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d8005280 con 0x55978e950070 2024-07-26T13:42:18.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.771+0000 7f33edb12740 1 -- 172.21.15.100:0/3419440408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55978e950070 msgr2=0x55978ecf5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:18.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.771+0000 7f33edb12740 1 --2- 172.21.15.100:0/3419440408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55978e950070 0x55978ecf5520 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f33d8009f70 tx=0x7f33d8009810 comp rx=0 tx=0).stop 2024-07-26T13:42:18.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.772+0000 7f33edb12740 1 -- 172.21.15.100:0/3419440408 shutdown_connections 2024-07-26T13:42:18.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.772+0000 7f33edb12740 1 --2- 172.21.15.100:0/3419440408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55978e92f390 0x55978ecf5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:18.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.772+0000 7f33edb12740 1 --2- 172.21.15.100:0/3419440408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55978e9271b0 0x55978ecf5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:18.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.772+0000 7f33edb12740 1 --2- 172.21.15.100:0/3419440408 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55978e950070 0x55978ecf5520 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:18.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.772+0000 7f33edb12740 1 -- 172.21.15.100:0/3419440408 >> 172.21.15.100:0/3419440408 conn(0x55978e944460 msgr2=0x55978ecf3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:18.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.772+0000 7f33edb12740 1 -- 172.21.15.100:0/3419440408 shutdown_connections 2024-07-26T13:42:18.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.772+0000 7f33edb12740 1 -- 172.21.15.100:0/3419440408 wait complete. 2024-07-26T13:42:18.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.772+0000 7f33edb12740 1 -- start start 2024-07-26T13:42:18.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.772+0000 7f33edb12740 1 Processor -- start 2024-07-26T13:42:18.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.773+0000 7f33edb12740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55978e9271b0 0x55978ed05550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:18.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.773+0000 7f33edb12740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55978e92f390 0x55978ed05ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:18.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.773+0000 7f33edb12740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55978e950070 0x55978ed113f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:18.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.773+0000 7f33edb12740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55978eabe700 con 0x55978e9271b0 2024-07-26T13:42:18.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.773+0000 7f33edb12740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55978ebfd990 con 0x55978e950070 2024-07-26T13:42:18.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.773+0000 7f33edb12740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55978ed08060 con 0x55978e92f390 2024-07-26T13:42:18.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.773+0000 7f33e5b2b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55978e9271b0 0x55978ed05550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:18.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.773+0000 7f33edb12740 10 client.0 _ll_register_callbacks cb 0x55978ec184a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:42:18.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.773+0000 7f33e5b2b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55978e9271b0 0x55978ed05550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49382/0 (socket says 172.21.15.100:49382) 2024-07-26T13:42:18.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.773+0000 7f33e5b2b640 1 -- 172.21.15.100:0/3295596779 learned_addr learned my addr 172.21.15.100:0/3295596779 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:18.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.773+0000 7f33e532a640 1 --2- 172.21.15.100:0/3295596779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55978e92f390 0x55978ed05ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:18.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.773+0000 7f33e5b2b640 1 -- 172.21.15.100:0/3295596779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55978e92f390 msgr2=0x55978ed05ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:18.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.773+0000 7f33e5b2b640 1 --2- 172.21.15.100:0/3295596779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55978e92f390 0x55978ed05ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:18.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.773+0000 7f33e5b2b640 1 -- 172.21.15.100:0/3295596779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55978e950070 msgr2=0x55978ed113f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:18.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.773+0000 7f33e5b2b640 1 --2- 172.21.15.100:0/3295596779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55978e950070 0x55978ed113f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:18.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.773+0000 7f33e5b2b640 1 -- 172.21.15.100:0/3295596779 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55978e9bb650 con 0x55978e9271b0 2024-07-26T13:42:18.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.773+0000 7f33e5b2b640 1 --2- 172.21.15.100:0/3295596779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55978e9271b0 0x55978ed05550 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f33d8013520 tx=0x7f33d80094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0ae4ebee4dbe4b server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:18.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.773+0000 7f33c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:42:18.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.773+0000 7f33c8ff9640 1 -- 172.21.15.100:0/3295596779 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d8019070 con 0x55978e9271b0 2024-07-26T13:42:18.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.773+0000 7f33c8ff9640 1 -- 172.21.15.100:0/3295596779 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f33d8004c40 con 0x55978e9271b0 2024-07-26T13:42:18.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.773+0000 7f33edb12740 1 -- 172.21.15.100:0/3295596779 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33d8013d50 con 0x55978e9271b0 2024-07-26T13:42:18.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.774+0000 7f33c8ff9640 1 -- 172.21.15.100:0/3295596779 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f33d8031050 con 0x55978e9271b0 2024-07-26T13:42:18.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.774+0000 7f33edb12740 10 client.16101 fetch_fsmap learned FSMap version 27 2024-07-26T13:42:18.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.774+0000 7f33edb12740 1 -- 172.21.15.100:0/3295596779 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55978ed14160 con 0x55978e9271b0 2024-07-26T13:42:18.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.774+0000 7f33c8ff9640 1 -- 172.21.15.100:0/3295596779 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33d8013d50 con 0x55978e9271b0 2024-07-26T13:42:18.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.774+0000 7f33c8ff9640 1 -- 172.21.15.100:0/3295596779 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f33d801e460 con 0x55978e9271b0 2024-07-26T13:42:18.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.774+0000 7f33edb12740 10 client.16101 fetch_fsmap finished waiting for FSMap version 27 2024-07-26T13:42:18.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.774+0000 7f33edb12740 10 client.16101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:42:18.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.774+0000 7f33edb12740 20 client.16101 populate_metadata read hostname 'smithi100' 2024-07-26T13:42:18.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.774+0000 7f33edb12740 1 --2- 172.21.15.100:0/3295596779 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55978e90e9f0 0x55978ed174c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:18.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.774+0000 7f33edb12740 4 client.16101 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:42:18.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.774+0000 7f33edb12740 1 -- 172.21.15.100:0/3295596779 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55978ec6dab0 con 0x55978e90e9f0 2024-07-26T13:42:18.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.774+0000 7f33e532a640 1 --2- 172.21.15.100:0/3295596779 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55978e90e9f0 0x55978ed174c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:18.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.775+0000 7f33c8ff9640 10 client.16101 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:18.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.775+0000 7f33e532a640 1 --2- 172.21.15.100:0/3295596779 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55978e90e9f0 0x55978ed174c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4723f3c9f62625f6 server_cookie=c2726cb042b4d2d1 in_seq=0 out_seq=0 2024-07-26T13:42:18.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.775+0000 7f33c8ff9640 1 -- 172.21.15.100:0/3295596779 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55978ec6dab0 con 0x55978e90e9f0 2024-07-26T13:42:18.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.775+0000 7f33c8ff9640 10 client.16101 handle_command_reply: tid=0 2024-07-26T13:42:18.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.775+0000 7f33edb12740 1 client.16101 shutdown 2024-07-26T13:42:18.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.776+0000 7f33edb12740 1 -- 172.21.15.100:0/3295596779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55978e9271b0 msgr2=0x55978ed05550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:18.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.776+0000 7f33edb12740 1 --2- 172.21.15.100:0/3295596779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55978e9271b0 0x55978ed05550 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f33d8013520 tx=0x7f33d80094c0 comp rx=0 tx=0).stop 2024-07-26T13:42:18.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.776+0000 7f33edb12740 1 -- 172.21.15.100:0/3295596779 shutdown_connections 2024-07-26T13:42:18.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.776+0000 7f33edb12740 1 --2- 172.21.15.100:0/3295596779 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55978e90e9f0 0x55978ed174c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:18.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.776+0000 7f33edb12740 1 --2- 172.21.15.100:0/3295596779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55978e950070 0x55978ed113f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:18.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.776+0000 7f33edb12740 1 --2- 172.21.15.100:0/3295596779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55978e92f390 0x55978ed05ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:18.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.776+0000 7f33edb12740 1 --2- 172.21.15.100:0/3295596779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55978e9271b0 0x55978ed05550 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:18.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.776+0000 7f33c8ff9640 0 client.16101 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:18.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.776+0000 7f33edb12740 1 -- 172.21.15.100:0/3295596779 >> 172.21.15.100:0/3295596779 conn(0x55978e944460 msgr2=0x55978ecf6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:18.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.776+0000 7f33edb12740 1 -- 172.21.15.100:0/3295596779 shutdown_connections 2024-07-26T13:42:18.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.776+0000 7f33edb12740 1 -- 172.21.15.100:0/3295596779 wait complete. 2024-07-26T13:42:18.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.776+0000 7f33edb12740 20 client.16101 trim_cache size 0 max 16384 2024-07-26T13:42:18.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.793+0000 7f33edb12740 1 Processor -- start 2024-07-26T13:42:18.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.793+0000 7f33edb12740 1 -- start start 2024-07-26T13:42:18.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.793+0000 7f33edb12740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55978e9271b0 0x55978ecfac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:18.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.793+0000 7f33edb12740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55978e92f390 0x55978ecf6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:18.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.793+0000 7f33edb12740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55978e950070 0x55978ed111e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:18.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.793+0000 7f33edb12740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f33e01af7b0 con 0x55978e950070 2024-07-26T13:42:18.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.793+0000 7f33edb12740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f33e01ad080 con 0x55978e92f390 2024-07-26T13:42:18.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.793+0000 7f33edb12740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f33e01aaed0 con 0x55978e9271b0 2024-07-26T13:42:18.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.794+0000 7f33e5b2b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55978e950070 0x55978ed111e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:18.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.794+0000 7f33e5b2b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55978e950070 0x55978ed111e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49386/0 (socket says 172.21.15.100:49386) 2024-07-26T13:42:18.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.794+0000 7f33e5b2b640 1 -- 172.21.15.100:0/1523747391 learned_addr learned my addr 172.21.15.100:0/1523747391 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:18.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.794+0000 7f33e5b2b640 1 --2- 172.21.15.100:0/1523747391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55978e9271b0 0x55978ecfac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:18.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.794+0000 7f33e532a640 1 --2- 172.21.15.100:0/1523747391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55978e92f390 0x55978ecf6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:18.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.794+0000 7f33e5b2b640 1 -- 172.21.15.100:0/1523747391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55978e9271b0 msgr2=0x55978ecfac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:18.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.794+0000 7f33e5b2b640 1 --2- 172.21.15.100:0/1523747391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55978e9271b0 0x55978ecfac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:18.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.794+0000 7f33e5b2b640 1 -- 172.21.15.100:0/1523747391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55978e92f390 msgr2=0x55978ecf6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:18.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.794+0000 7f33e5b2b640 1 --2- 172.21.15.100:0/1523747391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55978e92f390 0x55978ecf6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:18.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.794+0000 7f33e5b2b640 1 -- 172.21.15.100:0/1523747391 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55978ecee870 con 0x55978e950070 2024-07-26T13:42:18.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.794+0000 7f33e5b2b640 1 --2- 172.21.15.100:0/1523747391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55978e950070 0x55978ed111e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f33d8009850 tx=0x7f33d8020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8323cc2bb9d59922 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:18.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.794+0000 7f33caffd640 1 -- 172.21.15.100:0/1523747391 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d8019070 con 0x55978e950070 2024-07-26T13:42:18.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.794+0000 7f33caffd640 1 -- 172.21.15.100:0/1523747391 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33d801a040 con 0x55978e950070 2024-07-26T13:42:18.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.794+0000 7f33caffd640 1 -- 172.21.15.100:0/1523747391 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d802a410 con 0x55978e950070 2024-07-26T13:42:18.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.795+0000 7f33edb12740 1 -- 172.21.15.100:0/1523747391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55978e950070 msgr2=0x55978ed111e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:18.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.795+0000 7f33edb12740 1 --2- 172.21.15.100:0/1523747391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55978e950070 0x55978ed111e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f33d8009850 tx=0x7f33d8020040 comp rx=0 tx=0).stop 2024-07-26T13:42:18.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.795+0000 7f33edb12740 1 -- 172.21.15.100:0/1523747391 shutdown_connections 2024-07-26T13:42:18.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.795+0000 7f33edb12740 1 --2- 172.21.15.100:0/1523747391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55978e950070 0x55978ed111e0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:18.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.795+0000 7f33edb12740 1 --2- 172.21.15.100:0/1523747391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55978e92f390 0x55978ecf6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:18.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.795+0000 7f33edb12740 1 --2- 172.21.15.100:0/1523747391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55978e9271b0 0x55978ecfac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:18.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.795+0000 7f33edb12740 1 -- 172.21.15.100:0/1523747391 >> 172.21.15.100:0/1523747391 conn(0x55978e944460 msgr2=0x55978ecf6550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:18.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.795+0000 7f33edb12740 1 -- 172.21.15.100:0/1523747391 shutdown_connections 2024-07-26T13:42:18.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.795+0000 7f33edb12740 1 -- 172.21.15.100:0/1523747391 wait complete. 2024-07-26T13:42:18.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.796+0000 7f33edb12740 1 -- start start 2024-07-26T13:42:18.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.796+0000 7f33edb12740 1 Processor -- start 2024-07-26T13:42:18.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.796+0000 7f33edb12740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55978e9271b0 0x55978ed08400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:18.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.796+0000 7f33edb12740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55978e92f390 0x55978ed08940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:18.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.796+0000 7f33edb12740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55978e950070 0x55978ed08e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:18.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.796+0000 7f33edb12740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f33e01ab540 con 0x55978e9271b0 2024-07-26T13:42:18.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.796+0000 7f33edb12740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f33e01ac9b0 con 0x55978e950070 2024-07-26T13:42:18.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.796+0000 7f33edb12740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f33e01abc10 con 0x55978e92f390 2024-07-26T13:42:18.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.796+0000 7f33edb12740 10 client.0 _ll_register_callbacks cb 0x7f33dc00c350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:42:18.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.796+0000 7f33e5b2b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55978e9271b0 0x55978ed08400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:18.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.796+0000 7f33e5b2b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55978e9271b0 0x55978ed08400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49400/0 (socket says 172.21.15.100:49400) 2024-07-26T13:42:18.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.796+0000 7f33e5b2b640 1 -- 172.21.15.100:0/3909423165 learned_addr learned my addr 172.21.15.100:0/3909423165 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:18.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.797+0000 7f33e532a640 1 --2- 172.21.15.100:0/3909423165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55978e92f390 0x55978ed08940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:18.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.797+0000 7f33e5b2b640 1 --2- 172.21.15.100:0/3909423165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55978e950070 0x55978ed08e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:18.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.797+0000 7f33e5b2b640 1 -- 172.21.15.100:0/3909423165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55978e92f390 msgr2=0x55978ed08940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:18.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.797+0000 7f33e5b2b640 1 --2- 172.21.15.100:0/3909423165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55978e92f390 0x55978ed08940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:18.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.797+0000 7f33e5b2b640 1 -- 172.21.15.100:0/3909423165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55978e950070 msgr2=0x55978ed08e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:18.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.797+0000 7f33e5b2b640 1 --2- 172.21.15.100:0/3909423165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55978e950070 0x55978ed08e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:18.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.797+0000 7f33e5b2b640 1 -- 172.21.15.100:0/3909423165 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55978ed7dc50 con 0x55978e9271b0 2024-07-26T13:42:18.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.797+0000 7f33e5b2b640 1 --2- 172.21.15.100:0/3909423165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55978e9271b0 0x55978ed08400 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f33d8006010 tx=0x7f33d80092b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39a865345677dd1e server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:18.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.797+0000 7f33c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:42:18.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.797+0000 7f33c8ff9640 1 -- 172.21.15.100:0/3909423165 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d8019040 con 0x55978e9271b0 2024-07-26T13:42:18.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.797+0000 7f33c8ff9640 1 -- 172.21.15.100:0/3909423165 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f33d803c040 con 0x55978e9271b0 2024-07-26T13:42:18.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.797+0000 7f33edb12740 1 -- 172.21.15.100:0/3909423165 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33d801a040 con 0x55978e9271b0 2024-07-26T13:42:18.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.797+0000 7f33c8ff9640 1 -- 172.21.15.100:0/3909423165 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f33d80093c0 con 0x55978e9271b0 2024-07-26T13:42:18.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.797+0000 7f33edb12740 10 client.16107 fetch_fsmap learned FSMap version 27 2024-07-26T13:42:18.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.797+0000 7f33edb12740 1 -- 172.21.15.100:0/3909423165 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55978ed7eee0 con 0x55978e9271b0 2024-07-26T13:42:18.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.798+0000 7f33c8ff9640 1 -- 172.21.15.100:0/3909423165 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33d801a040 con 0x55978e9271b0 2024-07-26T13:42:18.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.798+0000 7f33c8ff9640 1 -- 172.21.15.100:0/3909423165 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f33d8040600 con 0x55978e9271b0 2024-07-26T13:42:18.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.798+0000 7f33edb12740 10 client.16107 fetch_fsmap finished waiting for FSMap version 27 2024-07-26T13:42:18.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.798+0000 7f33edb12740 10 client.16107 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:42:18.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.798+0000 7f33edb12740 20 client.16107 populate_metadata read hostname 'smithi100' 2024-07-26T13:42:18.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.798+0000 7f33edb12740 1 --2- 172.21.15.100:0/3909423165 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55978e8e3da0 0x55978ed81640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:18.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.798+0000 7f33edb12740 4 client.16107 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-26T13:42:18.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.798+0000 7f33edb12740 1 -- 172.21.15.100:0/3909423165 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55978ec69550 con 0x55978e8e3da0 2024-07-26T13:42:18.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.798+0000 7f33e532a640 1 --2- 172.21.15.100:0/3909423165 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55978e8e3da0 0x55978ed81640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:18.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.798+0000 7f33c8ff9640 10 client.16107 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:18.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.798+0000 7f33e532a640 1 --2- 172.21.15.100:0/3909423165 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55978e8e3da0 0x55978ed81640 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3072bddeb880d00 server_cookie=1c57992004ae01f9 in_seq=0 out_seq=0 2024-07-26T13:42:18.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.799+0000 7f33c8ff9640 1 -- 172.21.15.100:0/3909423165 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55978ec69550 con 0x55978e8e3da0 2024-07-26T13:42:18.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.799+0000 7f33c8ff9640 10 client.16107 handle_command_reply: tid=0 2024-07-26T13:42:18.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.799+0000 7f33edb12740 1 client.16107 shutdown 2024-07-26T13:42:18.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.799+0000 7f33edb12740 1 -- 172.21.15.100:0/3909423165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55978e9271b0 msgr2=0x55978ed08400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:18.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.799+0000 7f33edb12740 1 --2- 172.21.15.100:0/3909423165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55978e9271b0 0x55978ed08400 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f33d8006010 tx=0x7f33d80092b0 comp rx=0 tx=0).stop 2024-07-26T13:42:18.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.799+0000 7f33edb12740 1 -- 172.21.15.100:0/3909423165 shutdown_connections 2024-07-26T13:42:18.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.799+0000 7f33edb12740 1 --2- 172.21.15.100:0/3909423165 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55978e8e3da0 0x55978ed81640 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:18.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.799+0000 7f33edb12740 1 --2- 172.21.15.100:0/3909423165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55978e950070 0x55978ed08e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:18.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.799+0000 7f33edb12740 1 --2- 172.21.15.100:0/3909423165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55978e92f390 0x55978ed08940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:18.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.799+0000 7f33edb12740 1 --2- 172.21.15.100:0/3909423165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55978e9271b0 0x55978ed08400 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:18.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.799+0000 7f33edb12740 1 -- 172.21.15.100:0/3909423165 >> 172.21.15.100:0/3909423165 conn(0x55978e944460 msgr2=0x55978ecf6530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:18.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.799+0000 7f33c8ff9640 0 client.16107 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:18.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.799+0000 7f33edb12740 1 -- 172.21.15.100:0/3909423165 shutdown_connections 2024-07-26T13:42:18.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.799+0000 7f33edb12740 1 -- 172.21.15.100:0/3909423165 wait complete. 2024-07-26T13:42:18.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.799+0000 7f33edb12740 20 client.16107 trim_cache size 0 max 16384 2024-07-26T13:42:18.809 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:42:18.810 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-07-26T13:42:18.810 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-07-26T13:42:18.810 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:42:18.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.800+0000 7f33e758e640 1 -- 172.21.15.100:0/1584199649 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f33cc0b3d70 msgr2=0x7f33cc0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:18.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.800+0000 7f33e758e640 1 --2- 172.21.15.100:0/1584199649 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f33cc0b3d70 0x7f33cc0b6280 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f33d01022d0 tx=0x7f33d00caf60 comp rx=0 tx=0).stop 2024-07-26T13:42:18.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.800+0000 7f33e758e640 1 -- 172.21.15.100:0/1584199649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33e0107dd0 msgr2=0x7f33e019a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:18.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.800+0000 7f33e758e640 1 --2- 172.21.15.100:0/1584199649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33e0107dd0 0x7f33e019a480 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f33d0032fd0 tx=0x7f33d0004270 comp rx=0 tx=0).stop 2024-07-26T13:42:18.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.800+0000 7f33e758e640 1 -- 172.21.15.100:0/1584199649 shutdown_connections 2024-07-26T13:42:18.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.800+0000 7f33e758e640 1 --2- 172.21.15.100:0/1584199649 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f33cc0b3d70 0x7f33cc0b6280 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:18.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.800+0000 7f33e758e640 1 --2- 172.21.15.100:0/1584199649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33e0108720 0x7f33e019f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:18.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.800+0000 7f33e758e640 1 --2- 172.21.15.100:0/1584199649 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33e0107dd0 0x7f33e019a480 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:18.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.800+0000 7f33e758e640 1 --2- 172.21.15.100:0/1584199649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33e0102490 0x7f33e0199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:18.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.800+0000 7f33e758e640 1 -- 172.21.15.100:0/1584199649 >> 172.21.15.100:0/1584199649 conn(0x7f33e0079160 msgr2=0x7f33e0104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:18.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.800+0000 7f33e758e640 1 -- 172.21.15.100:0/1584199649 shutdown_connections 2024-07-26T13:42:18.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:18.800+0000 7f33e758e640 1 -- 172.21.15.100:0/1584199649 wait complete. 2024-07-26T13:42:19.404 INFO:tasks.workunit.client.0.smithi100.stdout: 1 25427 4.69 MB/sec warmup 33 sec latency 3466.541 ms 2024-07-26T13:42:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:19 smithi100 ceph-mon[32599]: pgmap v430: 97 pgs: 97 active+clean; 30 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 2.7 MiB/s wr, 89 op/s 2024-07-26T13:42:19.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:19 smithi167 ceph-mon[37140]: pgmap v430: 97 pgs: 97 active+clean; 30 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 2.7 MiB/s wr, 89 op/s 2024-07-26T13:42:19.823 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-26T13:42:19.824 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:42:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:19 smithi138 ceph-mon[36954]: pgmap v430: 97 pgs: 97 active+clean; 30 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 2.7 MiB/s wr, 89 op/s 2024-07-26T13:42:20.083 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:42:20.404 INFO:tasks.workunit.client.0.smithi100.stdout: 1 25427 4.55 MB/sec warmup 34 sec latency 4466.615 ms 2024-07-26T13:42:21.404 INFO:tasks.workunit.client.0.smithi100.stdout: 1 25427 4.42 MB/sec warmup 35 sec latency 5466.690 ms 2024-07-26T13:42:21.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.747+0000 7f0388d9f640 1 -- 172.21.15.100:0/240613765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f038410d840 msgr2=0x7f038410fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:21.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.747+0000 7f0388d9f640 1 --2- 172.21.15.100:0/240613765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f038410d840 0x7f038410fd30 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f037c009900 tx=0x7f037c02a410 comp rx=0 tx=0).stop 2024-07-26T13:42:21.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.747+0000 7f0388d9f640 1 -- 172.21.15.100:0/240613765 shutdown_connections 2024-07-26T13:42:21.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.747+0000 7f0388d9f640 1 --2- 172.21.15.100:0/240613765 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f038410d840 0x7f038410fd30 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:21.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.747+0000 7f0388d9f640 1 --2- 172.21.15.100:0/240613765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0384104d30 0x7f038410d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:21.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.747+0000 7f0388d9f640 1 --2- 172.21.15.100:0/240613765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03841043e0 0x7f03841047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:21.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.747+0000 7f0388d9f640 1 -- 172.21.15.100:0/240613765 >> 172.21.15.100:0/240613765 conn(0x7f0384079190 msgr2=0x7f03840795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:21.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.747+0000 7f0388d9f640 1 -- 172.21.15.100:0/240613765 shutdown_connections 2024-07-26T13:42:21.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.747+0000 7f0388d9f640 1 -- 172.21.15.100:0/240613765 wait complete. 2024-07-26T13:42:21.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.748+0000 7f0388d9f640 1 Processor -- start 2024-07-26T13:42:21.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.748+0000 7f0388d9f640 1 -- start start 2024-07-26T13:42:21.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.748+0000 7f0388d9f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03841043e0 0x7f0384102120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:21.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.748+0000 7f0388d9f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0384104d30 0x7f0384102660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:21.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.748+0000 7f0388d9f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f038410d840 0x7f0384102ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:21.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.748+0000 7f0388d9f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f03841134c0 con 0x7f03841043e0 2024-07-26T13:42:21.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.748+0000 7f0388d9f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0384113340 con 0x7f0384104d30 2024-07-26T13:42:21.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.748+0000 7f0388d9f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0384113640 con 0x7f038410d840 2024-07-26T13:42:21.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.749+0000 7f038259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03841043e0 0x7f0384102120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:21.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.749+0000 7f038259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03841043e0 0x7f0384102120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49430/0 (socket says 172.21.15.100:49430) 2024-07-26T13:42:21.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.749+0000 7f038259c640 1 -- 172.21.15.100:0/3590119547 learned_addr learned my addr 172.21.15.100:0/3590119547 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:21.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.749+0000 7f0381d9b640 1 --2- 172.21.15.100:0/3590119547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0384104d30 0x7f0384102660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:21.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.749+0000 7f0382d9d640 1 --2- 172.21.15.100:0/3590119547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f038410d840 0x7f0384102ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:21.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.749+0000 7f038259c640 1 -- 172.21.15.100:0/3590119547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f038410d840 msgr2=0x7f0384102ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:21.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.749+0000 7f038259c640 1 --2- 172.21.15.100:0/3590119547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f038410d840 0x7f0384102ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:21.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.749+0000 7f038259c640 1 -- 172.21.15.100:0/3590119547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0384104d30 msgr2=0x7f0384102660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:21.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.749+0000 7f038259c640 1 --2- 172.21.15.100:0/3590119547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0384104d30 0x7f0384102660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:21.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.749+0000 7f038259c640 1 -- 172.21.15.100:0/3590119547 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03841a8130 con 0x7f03841043e0 2024-07-26T13:42:21.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.749+0000 7f038259c640 1 --2- 172.21.15.100:0/3590119547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03841043e0 0x7f0384102120 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f037400eb90 tx=0x7f037400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:21.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.749+0000 7f03737fe640 1 -- 172.21.15.100:0/3590119547 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0374004d20 con 0x7f03841043e0 2024-07-26T13:42:21.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.749+0000 7f0388d9f640 1 -- 172.21.15.100:0/3590119547 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03841a8420 con 0x7f03841043e0 2024-07-26T13:42:21.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.749+0000 7f03737fe640 1 -- 172.21.15.100:0/3590119547 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f037400ee90 con 0x7f03841043e0 2024-07-26T13:42:21.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.749+0000 7f03737fe640 1 -- 172.21.15.100:0/3590119547 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0374010670 con 0x7f03841043e0 2024-07-26T13:42:21.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.749+0000 7f0388d9f640 1 -- 172.21.15.100:0/3590119547 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03841a8910 con 0x7f03841043e0 2024-07-26T13:42:21.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.750+0000 7f0388d9f640 1 -- 172.21.15.100:0/3590119547 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f038406bcf0 con 0x7f03841043e0 2024-07-26T13:42:21.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.752+0000 7f03737fe640 1 -- 172.21.15.100:0/3590119547 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f0374010870 con 0x7f03841043e0 2024-07-26T13:42:21.755 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:21 smithi100 ceph-mon[32599]: pgmap v431: 97 pgs: 97 active+clean; 28 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 2.8 MiB/s wr, 112 op/s 2024-07-26T13:42:21.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.753+0000 7f03737fe640 1 --2- 172.21.15.100:0/3590119547 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f036c0b3d40 0x7f036c0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:21.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.753+0000 7f0381d9b640 1 --2- 172.21.15.100:0/3590119547 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f036c0b3d40 0x7f036c0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:21.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.753+0000 7f03737fe640 1 -- 172.21.15.100:0/3590119547 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f03740e49b0 con 0x7f03841043e0 2024-07-26T13:42:21.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.753+0000 7f0381d9b640 1 --2- 172.21.15.100:0/3590119547 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f036c0b3d40 0x7f036c0b6250 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f038410f660 tx=0x7f037800a4c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:21.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.758+0000 7f03737fe640 1 -- 172.21.15.100:0/3590119547 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f0374055600 con 0x7f03841043e0 2024-07-26T13:42:21.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:21 smithi167 ceph-mon[37140]: pgmap v431: 97 pgs: 97 active+clean; 28 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 2.8 MiB/s wr, 112 op/s 2024-07-26T13:42:21.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:21 smithi138 ceph-mon[36954]: pgmap v431: 97 pgs: 97 active+clean; 28 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 2.8 MiB/s wr, 112 op/s 2024-07-26T13:42:21.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.965+0000 7f0388d9f640 1 -- 172.21.15.100:0/3590119547 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03841047f0 con 0x7f03841043e0 2024-07-26T13:42:21.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.966+0000 7f03737fe640 1 -- 172.21.15.100:0/3590119547 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11930 (secure 0 0 0) 0x7f03841047f0 con 0x7f03841043e0 2024-07-26T13:42:21.968 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:42:21.968 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":27,"btime":"2024-07-26T13:41:55:493454+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14790,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/313501409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":313501409},{"type":"v1","addr":"172.21.15.100:6839","nonce":313501409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/1402975506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":1402975506},{"type":"v1","addr":"172.21.15.100:6841","nonce":1402975506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-26T13:37:01.281843+0000","modified":"2024-07-26T13:41:54.710630+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24607,"mds_1":24605,"mds_2":24595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24557":{"gid":24557,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/3290105300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3290105300},{"type":"v1","addr":"172.21.15.138:6835","nonce":3290105300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/3393225483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3393225483},{"type":"v1","addr":"172.21.15.138:6837","nonce":3393225483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24595":{"gid":24595,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.167:6837/3828634468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3828634468},{"type":"v1","addr":"172.21.15.167:6837","nonce":3828634468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.138:6839/1580925471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1580925471},{"type":"v1","addr":"172.21.15.138:6839","nonce":1580925471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/398150514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":398150514},{"type":"v1","addr":"172.21.15.167:6839","nonce":398150514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24607,"qdb_cluster":[24607,24605,24595]},"id":1}]} 2024-07-26T13:42:21.969 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 27 2024-07-26T13:42:21.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.968+0000 7f0388d9f640 1 -- 172.21.15.100:0/3590119547 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f036c0b3d40 msgr2=0x7f036c0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:21.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.968+0000 7f0388d9f640 1 --2- 172.21.15.100:0/3590119547 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f036c0b3d40 0x7f036c0b6250 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f038410f660 tx=0x7f037800a4c0 comp rx=0 tx=0).stop 2024-07-26T13:42:21.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.968+0000 7f0388d9f640 1 -- 172.21.15.100:0/3590119547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03841043e0 msgr2=0x7f0384102120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:21.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.968+0000 7f0388d9f640 1 --2- 172.21.15.100:0/3590119547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03841043e0 0x7f0384102120 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f037400eb90 tx=0x7f037400c6a0 comp rx=0 tx=0).stop 2024-07-26T13:42:21.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.968+0000 7f0388d9f640 1 -- 172.21.15.100:0/3590119547 shutdown_connections 2024-07-26T13:42:21.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.968+0000 7f0388d9f640 1 --2- 172.21.15.100:0/3590119547 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f036c0b3d40 0x7f036c0b6250 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:21.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.968+0000 7f0388d9f640 1 --2- 172.21.15.100:0/3590119547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f038410d840 0x7f0384102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:21.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.968+0000 7f0388d9f640 1 --2- 172.21.15.100:0/3590119547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0384104d30 0x7f0384102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:21.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.968+0000 7f0388d9f640 1 --2- 172.21.15.100:0/3590119547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03841043e0 0x7f0384102120 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:21.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.968+0000 7f0388d9f640 1 -- 172.21.15.100:0/3590119547 >> 172.21.15.100:0/3590119547 conn(0x7f0384079190 msgr2=0x7f0384105220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:21.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.968+0000 7f0388d9f640 1 -- 172.21.15.100:0/3590119547 shutdown_connections 2024-07-26T13:42:21.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:21.968+0000 7f0388d9f640 1 -- 172.21.15.100:0/3590119547 wait complete. 2024-07-26T13:42:22.405 INFO:tasks.workunit.client.0.smithi100.stdout: 1 25427 4.30 MB/sec warmup 36 sec latency 6466.777 ms 2024-07-26T13:42:22.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:22 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/3590119547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:42:22.874 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:22 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/3590119547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:42:22.906 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-26T13:42:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:22 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/3590119547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:42:23.164 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:42:23.404 INFO:tasks.workunit.client.0.smithi100.stdout: 1 25427 4.19 MB/sec warmup 37 sec latency 7466.803 ms 2024-07-26T13:42:23.603 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:23 smithi100 ceph-mon[32599]: pgmap v432: 97 pgs: 97 active+clean; 28 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 2.8 MiB/s wr, 113 op/s 2024-07-26T13:42:23.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:23 smithi167 ceph-mon[37140]: pgmap v432: 97 pgs: 97 active+clean; 28 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 2.8 MiB/s wr, 113 op/s 2024-07-26T13:42:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.773+0000 7fa0031be640 1 -- 172.21.15.100:0/133313239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ffc104390 msgr2=0x7f9ffc1047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.773+0000 7fa0031be640 1 --2- 172.21.15.100:0/133313239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ffc104390 0x7f9ffc1047a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9fec009900 tx=0x7f9fec02a410 comp rx=0 tx=0).stop 2024-07-26T13:42:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.773+0000 7fa0031be640 1 -- 172.21.15.100:0/133313239 shutdown_connections 2024-07-26T13:42:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.773+0000 7fa0031be640 1 --2- 172.21.15.100:0/133313239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ffc10d7b0 0x7f9ffc10fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.773+0000 7fa0031be640 1 --2- 172.21.15.100:0/133313239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ffc104ce0 0x7f9ffc10d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.773+0000 7fa0031be640 1 --2- 172.21.15.100:0/133313239 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ffc104390 0x7f9ffc1047a0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.773+0000 7fa0031be640 1 -- 172.21.15.100:0/133313239 >> 172.21.15.100:0/133313239 conn(0x7f9ffc07a170 msgr2=0x7f9ffc07a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.773+0000 7fa0031be640 1 -- 172.21.15.100:0/133313239 shutdown_connections 2024-07-26T13:42:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.773+0000 7fa0031be640 1 -- 172.21.15.100:0/133313239 wait complete. 2024-07-26T13:42:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.774+0000 7fa0031be640 1 Processor -- start 2024-07-26T13:42:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.774+0000 7fa0031be640 1 -- start start 2024-07-26T13:42:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.774+0000 7fa0031be640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ffc104390 0x7f9ffc199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.774+0000 7fa0031be640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ffc104ce0 0x7f9ffc19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.774+0000 7fa0031be640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ffc10d7b0 0x7f9ffc19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.774+0000 7fa0031be640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9ffc113480 con 0x7f9ffc104ce0 2024-07-26T13:42:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.774+0000 7fa0031be640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9ffc113300 con 0x7f9ffc104390 2024-07-26T13:42:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.774+0000 7fa0031be640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9ffc113600 con 0x7f9ffc10d7b0 2024-07-26T13:42:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.774+0000 7f9ffbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ffc104ce0 0x7f9ffc19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.774+0000 7f9ffbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ffc104ce0 0x7f9ffc19a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49438/0 (socket says 172.21.15.100:49438) 2024-07-26T13:42:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.774+0000 7f9ffbfff640 1 -- 172.21.15.100:0/3263597520 learned_addr learned my addr 172.21.15.100:0/3263597520 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.774+0000 7fa000f5a640 1 --2- 172.21.15.100:0/3263597520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ffc104390 0x7f9ffc199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.775+0000 7f9ffbfff640 1 -- 172.21.15.100:0/3263597520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ffc10d7b0 msgr2=0x7f9ffc19f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.774+0000 7fa00175b640 1 --2- 172.21.15.100:0/3263597520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ffc10d7b0 0x7f9ffc19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.775+0000 7f9ffbfff640 1 --2- 172.21.15.100:0/3263597520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ffc10d7b0 0x7f9ffc19f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.775+0000 7f9ffbfff640 1 -- 172.21.15.100:0/3263597520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ffc104390 msgr2=0x7f9ffc199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.775+0000 7f9ffbfff640 1 --2- 172.21.15.100:0/3263597520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ffc104390 0x7f9ffc199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.775+0000 7f9ffbfff640 1 -- 172.21.15.100:0/3263597520 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ffc19fdd0 con 0x7f9ffc104ce0 2024-07-26T13:42:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.775+0000 7fa00175b640 1 --2- 172.21.15.100:0/3263597520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ffc10d7b0 0x7f9ffc19f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:42:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.775+0000 7f9ffbfff640 1 --2- 172.21.15.100:0/3263597520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ffc104ce0 0x7f9ffc19a440 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f9ff000d740 tx=0x7f9ff000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.775+0000 7f9ff9ffb640 1 -- 172.21.15.100:0/3263597520 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0017070 con 0x7f9ffc104ce0 2024-07-26T13:42:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.775+0000 7f9ff9ffb640 1 -- 172.21.15.100:0/3263597520 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ff000de50 con 0x7f9ffc104ce0 2024-07-26T13:42:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.775+0000 7fa0031be640 1 -- 172.21.15.100:0/3263597520 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ffc1a00c0 con 0x7f9ffc104ce0 2024-07-26T13:42:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.775+0000 7f9ff9ffb640 1 -- 172.21.15.100:0/3263597520 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff000f620 con 0x7f9ffc104ce0 2024-07-26T13:42:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.775+0000 7fa0031be640 1 -- 172.21.15.100:0/3263597520 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ffc1a6850 con 0x7f9ffc104ce0 2024-07-26T13:42:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.778+0000 7fa00a1aa740 1 Processor -- start 2024-07-26T13:42:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.778+0000 7fa00a1aa740 1 -- start start 2024-07-26T13:42:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.778+0000 7fa00a1aa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612b92901b0 0x5612b96326b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.778+0000 7fa00a1aa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612b92672f0 0x5612b9632bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.778+0000 7fa00a1aa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612b926f4d0 0x5612b9633130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.778+0000 7fa00a1aa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5612b95a83b0 con 0x5612b92672f0 2024-07-26T13:42:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.778+0000 7fa00a1aa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5612b95978b0 con 0x5612b926f4d0 2024-07-26T13:42:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.778+0000 7fa00a1aa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5612b959b8f0 con 0x5612b92901b0 2024-07-26T13:42:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.778+0000 7f9ff9ffb640 1 -- 172.21.15.100:0/3263597520 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f9ff001a070 con 0x7f9ffc104ce0 2024-07-26T13:42:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.778+0000 7fa000f5a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612b92672f0 0x5612b9632bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.778+0000 7fa000f5a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612b92672f0 0x5612b9632bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49440/0 (socket says 172.21.15.100:49440) 2024-07-26T13:42:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.778+0000 7fa000f5a640 1 -- 172.21.15.100:0/3541211296 learned_addr learned my addr 172.21.15.100:0/3541211296 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.778+0000 7f9ffbfff640 1 --2- 172.21.15.100:0/3541211296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612b926f4d0 0x5612b9633130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.778+0000 7fa00175b640 1 --2- 172.21.15.100:0/3541211296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612b92901b0 0x5612b96326b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.778+0000 7fa000f5a640 1 -- 172.21.15.100:0/3541211296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612b92901b0 msgr2=0x5612b96326b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.778+0000 7fa000f5a640 1 --2- 172.21.15.100:0/3541211296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612b92901b0 0x5612b96326b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.778+0000 7fa000f5a640 1 -- 172.21.15.100:0/3541211296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612b926f4d0 msgr2=0x5612b9633130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.778+0000 7fa000f5a640 1 --2- 172.21.15.100:0/3541211296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612b926f4d0 0x5612b9633130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.778+0000 7fa000f5a640 1 -- 172.21.15.100:0/3541211296 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612b953e9a0 con 0x5612b92672f0 2024-07-26T13:42:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.778+0000 7fa00175b640 1 --2- 172.21.15.100:0/3541211296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612b92901b0 0x5612b96326b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:42:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.779+0000 7fa000f5a640 1 --2- 172.21.15.100:0/3541211296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612b92672f0 0x5612b9632bf0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9fec009900 tx=0x7f9fec032e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a63162194fb1b9 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.779+0000 7f9fe2ffd640 1 -- 172.21.15.100:0/3541211296 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec040070 con 0x5612b92672f0 2024-07-26T13:42:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.779+0000 7f9fe2ffd640 1 -- 172.21.15.100:0/3541211296 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fec0316a0 con 0x5612b92672f0 2024-07-26T13:42:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.779+0000 7f9fe2ffd640 1 -- 172.21.15.100:0/3541211296 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec035040 con 0x5612b92672f0 2024-07-26T13:42:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.779+0000 7f9ff9ffb640 1 --2- 172.21.15.100:0/3263597520 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9fe40b3d40 0x7f9fe40b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.779+0000 7fa00175b640 1 --2- 172.21.15.100:0/3263597520 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9fe40b3d40 0x7f9fe40b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.779+0000 7f9ff9ffb640 1 -- 172.21.15.100:0/3263597520 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f9ff00e3a00 con 0x7f9ffc104ce0 2024-07-26T13:42:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.780+0000 7fa00a1aa740 1 -- 172.21.15.100:0/3541211296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612b92672f0 msgr2=0x5612b9632bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.780+0000 7fa00a1aa740 1 --2- 172.21.15.100:0/3541211296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612b92672f0 0x5612b9632bf0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9fec009900 tx=0x7f9fec032e60 comp rx=0 tx=0).stop 2024-07-26T13:42:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.780+0000 7fa00175b640 1 --2- 172.21.15.100:0/3263597520 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9fe40b3d40 0x7f9fe40b6250 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9ff4007ee0 tx=0x7f9ff4007430 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.780+0000 7fa00a1aa740 1 -- 172.21.15.100:0/3541211296 shutdown_connections 2024-07-26T13:42:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.780+0000 7fa00a1aa740 1 --2- 172.21.15.100:0/3541211296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612b926f4d0 0x5612b9633130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.780+0000 7fa00a1aa740 1 --2- 172.21.15.100:0/3541211296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612b92672f0 0x5612b9632bf0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.780+0000 7fa00a1aa740 1 --2- 172.21.15.100:0/3541211296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612b92901b0 0x5612b96326b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.780+0000 7fa00a1aa740 1 -- 172.21.15.100:0/3541211296 >> 172.21.15.100:0/3541211296 conn(0x5612b9284300 msgr2=0x5612b96305a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.780+0000 7fa00a1aa740 1 -- 172.21.15.100:0/3541211296 shutdown_connections 2024-07-26T13:42:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.780+0000 7fa00a1aa740 1 -- 172.21.15.100:0/3541211296 wait complete. 2024-07-26T13:42:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.780+0000 7fa00a1aa740 1 -- start start 2024-07-26T13:42:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.780+0000 7fa00a1aa740 1 Processor -- start 2024-07-26T13:42:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.781+0000 7fa00a1aa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612b92672f0 0x5612b963e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.781+0000 7fa00a1aa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612b926f4d0 0x5612b963e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.781+0000 7fa00a1aa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612b92901b0 0x5612b964a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.781+0000 7fa00a1aa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5612b94d1ab0 con 0x5612b92672f0 2024-07-26T13:42:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.781+0000 7fa00a1aa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5612b951fcf0 con 0x5612b92901b0 2024-07-26T13:42:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.781+0000 7fa00a1aa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5612b9640dd0 con 0x5612b926f4d0 2024-07-26T13:42:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.781+0000 7fa00a1aa740 10 client.0 _ll_register_callbacks cb 0x5612b95913e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:42:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.781+0000 7fa000f5a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612b92672f0 0x5612b963e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.781+0000 7fa000f5a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612b92672f0 0x5612b963e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49454/0 (socket says 172.21.15.100:49454) 2024-07-26T13:42:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.781+0000 7fa000f5a640 1 -- 172.21.15.100:0/493767914 learned_addr learned my addr 172.21.15.100:0/493767914 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.781+0000 7fa00175b640 1 --2- 172.21.15.100:0/493767914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612b92901b0 0x5612b964a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.781+0000 7f9ffbfff640 1 --2- 172.21.15.100:0/493767914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612b926f4d0 0x5612b963e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.781+0000 7fa000f5a640 1 -- 172.21.15.100:0/493767914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612b926f4d0 msgr2=0x5612b963e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.781+0000 7fa000f5a640 1 --2- 172.21.15.100:0/493767914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612b926f4d0 0x5612b963e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.781+0000 7fa000f5a640 1 -- 172.21.15.100:0/493767914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612b92901b0 msgr2=0x5612b964a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.781+0000 7fa000f5a640 1 --2- 172.21.15.100:0/493767914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612b92901b0 0x5612b964a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.781+0000 7fa000f5a640 1 -- 172.21.15.100:0/493767914 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612b92fb790 con 0x5612b92672f0 2024-07-26T13:42:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.781+0000 7f9ffbfff640 1 --2- 172.21.15.100:0/493767914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612b926f4d0 0x5612b963e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:42:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.781+0000 7fa00175b640 1 --2- 172.21.15.100:0/493767914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612b92901b0 0x5612b964a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:42:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.781+0000 7fa000f5a640 1 --2- 172.21.15.100:0/493767914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612b92672f0 0x5612b963e1a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9fec031a80 tx=0x7f9fec02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0002c7d76d76da server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.781+0000 7f9fdbfff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:42:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.781+0000 7f9fdbfff640 1 -- 172.21.15.100:0/493767914 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec035040 con 0x5612b92672f0 2024-07-26T13:42:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.781+0000 7f9fdbfff640 1 -- 172.21.15.100:0/493767914 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f9fec004030 con 0x5612b92672f0 2024-07-26T13:42:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.781+0000 7fa00a1aa740 1 -- 172.21.15.100:0/493767914 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fec0316a0 con 0x5612b92672f0 2024-07-26T13:42:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.782+0000 7f9fdbfff640 1 -- 172.21.15.100:0/493767914 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9fec040040 con 0x5612b92672f0 2024-07-26T13:42:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.782+0000 7fa00a1aa740 10 client.16125 fetch_fsmap learned FSMap version 27 2024-07-26T13:42:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.782+0000 7fa00a1aa740 1 -- 172.21.15.100:0/493767914 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5612b9374620 con 0x5612b92672f0 2024-07-26T13:42:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.782+0000 7f9fdbfff640 1 -- 172.21.15.100:0/493767914 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fec0316a0 con 0x5612b92672f0 2024-07-26T13:42:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.782+0000 7f9fdbfff640 1 -- 172.21.15.100:0/493767914 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9fec0035f0 con 0x5612b92672f0 2024-07-26T13:42:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.782+0000 7fa00a1aa740 10 client.16125 fetch_fsmap finished waiting for FSMap version 27 2024-07-26T13:42:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.782+0000 7fa00a1aa740 10 client.16125 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:42:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.782+0000 7fa00a1aa740 20 client.16125 populate_metadata read hostname 'smithi100' 2024-07-26T13:42:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.782+0000 7fa00a1aa740 1 --2- 172.21.15.100:0/493767914 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5612b924ed80 0x5612b964fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.782+0000 7fa00a1aa740 4 client.16125 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:42:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.782+0000 7fa00a1aa740 1 -- 172.21.15.100:0/493767914 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612b95aa300 con 0x5612b924ed80 2024-07-26T13:42:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.782+0000 7f9ffbfff640 1 --2- 172.21.15.100:0/493767914 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5612b924ed80 0x5612b964fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.783+0000 7f9fdbfff640 10 client.16125 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.783+0000 7f9ffbfff640 1 --2- 172.21.15.100:0/493767914 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5612b924ed80 0x5612b964fe60 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13213d850a172cb8 server_cookie=5cc880e4ebebd6f7 in_seq=0 out_seq=0 2024-07-26T13:42:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.784+0000 7f9fdbfff640 1 -- 172.21.15.100:0/493767914 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5612b95aa300 con 0x5612b924ed80 2024-07-26T13:42:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.784+0000 7f9fdbfff640 10 client.16125 handle_command_reply: tid=0 2024-07-26T13:42:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.784+0000 7fa00a1aa740 1 client.16125 shutdown 2024-07-26T13:42:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.784+0000 7fa00a1aa740 1 -- 172.21.15.100:0/493767914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612b92672f0 msgr2=0x5612b963e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.784+0000 7fa00a1aa740 1 --2- 172.21.15.100:0/493767914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612b92672f0 0x5612b963e1a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9fec031a80 tx=0x7f9fec02a410 comp rx=0 tx=0).stop 2024-07-26T13:42:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.784+0000 7fa00a1aa740 1 -- 172.21.15.100:0/493767914 shutdown_connections 2024-07-26T13:42:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.784+0000 7fa00a1aa740 1 --2- 172.21.15.100:0/493767914 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5612b924ed80 0x5612b964fe60 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.784+0000 7fa00a1aa740 1 --2- 172.21.15.100:0/493767914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612b92901b0 0x5612b964a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.784+0000 7fa00a1aa740 1 --2- 172.21.15.100:0/493767914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612b926f4d0 0x5612b963e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.784+0000 7f9fdbfff640 0 client.16125 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.784+0000 7fa00a1aa740 1 --2- 172.21.15.100:0/493767914 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612b92672f0 0x5612b963e1a0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.784+0000 7fa00a1aa740 1 -- 172.21.15.100:0/493767914 >> 172.21.15.100:0/493767914 conn(0x5612b9284300 msgr2=0x5612b962ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.784+0000 7fa00a1aa740 1 -- 172.21.15.100:0/493767914 shutdown_connections 2024-07-26T13:42:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.784+0000 7fa00a1aa740 1 -- 172.21.15.100:0/493767914 wait complete. 2024-07-26T13:42:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.784+0000 7fa00a1aa740 20 client.16125 trim_cache size 0 max 16384 2024-07-26T13:42:23.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.799+0000 7fa00a1aa740 1 Processor -- start 2024-07-26T13:42:23.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.799+0000 7fa00a1aa740 1 -- start start 2024-07-26T13:42:23.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.799+0000 7fa00a1aa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612b9284300 0x5612b9638780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:23.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.799+0000 7fa00a1aa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612b924ed80 0x5612b963fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:23.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.799+0000 7fa00a1aa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612b92672f0 0x5612b9633df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:23.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.799+0000 7fa00a1aa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9ffc1b1980 con 0x5612b9284300 2024-07-26T13:42:23.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.799+0000 7fa00a1aa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9ffc1af250 con 0x5612b92672f0 2024-07-26T13:42:23.802 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.799+0000 7fa00a1aa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9ffc1ad0a0 con 0x5612b924ed80 2024-07-26T13:42:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.799+0000 7fa000f5a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612b9284300 0x5612b9638780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.800+0000 7fa000f5a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612b9284300 0x5612b9638780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49470/0 (socket says 172.21.15.100:49470) 2024-07-26T13:42:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.800+0000 7fa000f5a640 1 -- 172.21.15.100:0/507512273 learned_addr learned my addr 172.21.15.100:0/507512273 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.800+0000 7fa000f5a640 1 -- 172.21.15.100:0/507512273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612b924ed80 msgr2=0x5612b963fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.800+0000 7f9ffbfff640 1 --2- 172.21.15.100:0/507512273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612b924ed80 0x5612b963fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.800+0000 7fa000f5a640 1 --2- 172.21.15.100:0/507512273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612b924ed80 0x5612b963fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.800+0000 7fa00175b640 1 --2- 172.21.15.100:0/507512273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612b92672f0 0x5612b9633df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.800+0000 7fa000f5a640 1 -- 172.21.15.100:0/507512273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612b92672f0 msgr2=0x5612b9633df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.800+0000 7fa000f5a640 1 --2- 172.21.15.100:0/507512273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612b92672f0 0x5612b9633df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.800+0000 7fa000f5a640 1 -- 172.21.15.100:0/507512273 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612b9630ea0 con 0x5612b9284300 2024-07-26T13:42:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.800+0000 7fa00175b640 1 --2- 172.21.15.100:0/507512273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612b92672f0 0x5612b9633df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:42:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.800+0000 7fa000f5a640 1 --2- 172.21.15.100:0/507512273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612b9284300 0x5612b9638780 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f9fec0053e0 tx=0x7f9fec005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4a528785b7674e2 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.800+0000 7f9fe27fc640 1 -- 172.21.15.100:0/507512273 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec05a040 con 0x5612b9284300 2024-07-26T13:42:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.800+0000 7f9fe27fc640 1 -- 172.21.15.100:0/507512273 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fec0314a0 con 0x5612b9284300 2024-07-26T13:42:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.800+0000 7f9fe27fc640 1 -- 172.21.15.100:0/507512273 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec035040 con 0x5612b9284300 2024-07-26T13:42:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.801+0000 7fa00a1aa740 1 -- 172.21.15.100:0/507512273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612b9284300 msgr2=0x5612b9638780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.801+0000 7fa00a1aa740 1 --2- 172.21.15.100:0/507512273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612b9284300 0x5612b9638780 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f9fec0053e0 tx=0x7f9fec005720 comp rx=0 tx=0).stop 2024-07-26T13:42:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.801+0000 7fa00a1aa740 1 -- 172.21.15.100:0/507512273 shutdown_connections 2024-07-26T13:42:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.801+0000 7fa00a1aa740 1 --2- 172.21.15.100:0/507512273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612b92672f0 0x5612b9633df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.801+0000 7fa00a1aa740 1 --2- 172.21.15.100:0/507512273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612b924ed80 0x5612b963fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.801+0000 7fa00a1aa740 1 --2- 172.21.15.100:0/507512273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612b9284300 0x5612b9638780 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.801+0000 7fa00a1aa740 1 -- 172.21.15.100:0/507512273 >> 172.21.15.100:0/507512273 conn(0x5612b96306b0 msgr2=0x5612b96336e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.801+0000 7fa00a1aa740 1 -- 172.21.15.100:0/507512273 shutdown_connections 2024-07-26T13:42:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.801+0000 7fa00a1aa740 1 -- 172.21.15.100:0/507512273 wait complete. 2024-07-26T13:42:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.801+0000 7fa00a1aa740 1 -- start start 2024-07-26T13:42:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.801+0000 7fa00a1aa740 1 Processor -- start 2024-07-26T13:42:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.802+0000 7fa00a1aa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612b924ed80 0x5612b96aa730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.802+0000 7fa00a1aa740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612b92672f0 0x5612b96aad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.802+0000 7fa00a1aa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612b9284300 0x5612b96afd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.802+0000 7fa00a1aa740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9ffc1ad710 con 0x5612b92672f0 2024-07-26T13:42:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.802+0000 7fa00a1aa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9ffc1aeb80 con 0x5612b924ed80 2024-07-26T13:42:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.802+0000 7fa00a1aa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9ffc1adde0 con 0x5612b9284300 2024-07-26T13:42:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.802+0000 7fa00a1aa740 10 client.0 _ll_register_callbacks cb 0x7f9fec032db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:42:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.802+0000 7f9ffbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612b92672f0 0x5612b96aad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.802+0000 7f9ffbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612b92672f0 0x5612b96aad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49478/0 (socket says 172.21.15.100:49478) 2024-07-26T13:42:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.802+0000 7f9ffbfff640 1 -- 172.21.15.100:0/529226126 learned_addr learned my addr 172.21.15.100:0/529226126 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.802+0000 7fa000f5a640 1 --2- 172.21.15.100:0/529226126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612b924ed80 0x5612b96aa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:23.807 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.802+0000 7f9ffbfff640 1 -- 172.21.15.100:0/529226126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612b9284300 msgr2=0x5612b96afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.802+0000 7f9ffbfff640 1 --2- 172.21.15.100:0/529226126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612b9284300 0x5612b96afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.802+0000 7f9ffbfff640 1 -- 172.21.15.100:0/529226126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612b924ed80 msgr2=0x5612b96aa730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.802+0000 7f9ffbfff640 1 --2- 172.21.15.100:0/529226126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612b924ed80 0x5612b96aa730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.802+0000 7f9ffbfff640 1 -- 172.21.15.100:0/529226126 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612b96b03c0 con 0x5612b92672f0 2024-07-26T13:42:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.802+0000 7f9ffbfff640 1 --2- 172.21.15.100:0/529226126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612b92672f0 0x5612b96aad10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f9ff00a78a0 tx=0x7f9ff0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad0b0214a7864f server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.802+0000 7f9fdbfff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:42:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.803+0000 7f9fdbfff640 1 -- 172.21.15.100:0/529226126 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0021040 con 0x5612b92672f0 2024-07-26T13:42:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.803+0000 7f9fdbfff640 1 -- 172.21.15.100:0/529226126 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f9ff00e13f0 con 0x5612b92672f0 2024-07-26T13:42:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.803+0000 7fa00a1aa740 1 -- 172.21.15.100:0/529226126 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fec0314a0 con 0x5612b92672f0 2024-07-26T13:42:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.803+0000 7f9fdbfff640 1 -- 172.21.15.100:0/529226126 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9ff00aa140 con 0x5612b92672f0 2024-07-26T13:42:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.803+0000 7fa00a1aa740 10 client.16131 fetch_fsmap learned FSMap version 27 2024-07-26T13:42:23.808 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.803+0000 7fa00a1aa740 1 -- 172.21.15.100:0/529226126 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5612b96b7720 con 0x5612b92672f0 2024-07-26T13:42:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.803+0000 7f9fdbfff640 1 -- 172.21.15.100:0/529226126 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fec0314a0 con 0x5612b92672f0 2024-07-26T13:42:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.803+0000 7f9fdbfff640 1 -- 172.21.15.100:0/529226126 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9ff00a68c0 con 0x5612b92672f0 2024-07-26T13:42:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.803+0000 7fa00a1aa740 10 client.16131 fetch_fsmap finished waiting for FSMap version 27 2024-07-26T13:42:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.803+0000 7fa00a1aa740 10 client.16131 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:42:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.803+0000 7fa00a1aa740 20 client.16131 populate_metadata read hostname 'smithi100' 2024-07-26T13:42:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.803+0000 7fa00a1aa740 1 --2- 172.21.15.100:0/529226126 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5612b926f4d0 0x5612b96b9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.803+0000 7fa00a1aa740 4 client.16131 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-26T13:42:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.803+0000 7fa00a1aa740 1 -- 172.21.15.100:0/529226126 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5612b95a5da0 con 0x5612b926f4d0 2024-07-26T13:42:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.803+0000 7fa000f5a640 1 --2- 172.21.15.100:0/529226126 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5612b926f4d0 0x5612b96b9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.804+0000 7f9fdbfff640 10 client.16131 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:23.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:23.804+0000 7fa000f5a640 1 --2- 172.21.15.100:0/529226126 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5612b926f4d0 0x5612b96b9e60 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d2459fc668f8e37 server_cookie=27ccfc6e666e2f91 in_seq=0 out_seq=0 2024-07-26T13:42:23.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:23 smithi138 ceph-mon[36954]: pgmap v432: 97 pgs: 97 active+clean; 28 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 2.8 MiB/s wr, 113 op/s 2024-07-26T13:42:24.404 INFO:tasks.workunit.client.0.smithi100.stdout: 1 25427 4.08 MB/sec warmup 38 sec latency 8466.861 ms 2024-07-26T13:42:25.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.062+0000 7f0433618640 1 -- 172.21.15.100:0/3144169703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f042c108750 msgr2=0x7f042c10ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:25.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.062+0000 7f0433618640 1 --2- 172.21.15.100:0/3144169703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f042c108750 0x7f042c10ef80 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0424009ca0 tx=0x7f042402a8c0 comp rx=0 tx=0).stop 2024-07-26T13:42:25.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.062+0000 7f0433618640 1 -- 172.21.15.100:0/3144169703 shutdown_connections 2024-07-26T13:42:25.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.062+0000 7f0433618640 1 --2- 172.21.15.100:0/3144169703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f042c108750 0x7f042c10ef80 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:25.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.062+0000 7f0433618640 1 --2- 172.21.15.100:0/3144169703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f042c107e00 0x7f042c108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:25.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.062+0000 7f0433618640 1 --2- 172.21.15.100:0/3144169703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f042c102510 0x7f042c102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:25.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.062+0000 7f0433618640 1 -- 172.21.15.100:0/3144169703 >> 172.21.15.100:0/3144169703 conn(0x7f042c0791c0 msgr2=0x7f042c0795d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:25.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.062+0000 7f0433618640 1 -- 172.21.15.100:0/3144169703 shutdown_connections 2024-07-26T13:42:25.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.062+0000 7f0433618640 1 -- 172.21.15.100:0/3144169703 wait complete. 2024-07-26T13:42:25.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.063+0000 7f0433618640 1 Processor -- start 2024-07-26T13:42:25.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.063+0000 7f0433618640 1 -- start start 2024-07-26T13:42:25.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.063+0000 7f0433618640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f042c102510 0x7f042c199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:25.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.063+0000 7f0433618640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f042c107e00 0x7f042c19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:25.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.063+0000 7f0433618640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f042c108750 0x7f042c19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:25.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.063+0000 7f0433618640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f042c113530 con 0x7f042c102510 2024-07-26T13:42:25.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.063+0000 7f0433618640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f042c1133b0 con 0x7f042c107e00 2024-07-26T13:42:25.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.063+0000 7f0433618640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f042c1136b0 con 0x7f042c108750 2024-07-26T13:42:25.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.063+0000 7f04313b4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f042c102510 0x7f042c199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:25.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.063+0000 7f04313b4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f042c102510 0x7f042c199ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33806/0 (socket says 172.21.15.100:33806) 2024-07-26T13:42:25.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.063+0000 7f04313b4640 1 -- 172.21.15.100:0/4160846200 learned_addr learned my addr 172.21.15.100:0/4160846200 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:25.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.063+0000 7f0430bb3640 1 --2- 172.21.15.100:0/4160846200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f042c107e00 0x7f042c19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:25.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.063+0000 7f0431bb5640 1 --2- 172.21.15.100:0/4160846200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f042c108750 0x7f042c19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:25.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.063+0000 7f04313b4640 1 -- 172.21.15.100:0/4160846200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f042c108750 msgr2=0x7f042c19f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:25.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.063+0000 7f04313b4640 1 --2- 172.21.15.100:0/4160846200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f042c108750 0x7f042c19f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:25.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.063+0000 7f04313b4640 1 -- 172.21.15.100:0/4160846200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f042c107e00 msgr2=0x7f042c19a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:25.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.063+0000 7f04313b4640 1 --2- 172.21.15.100:0/4160846200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f042c107e00 0x7f042c19a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:25.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.063+0000 7f04313b4640 1 -- 172.21.15.100:0/4160846200 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f042c19fd90 con 0x7f042c102510 2024-07-26T13:42:25.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.064+0000 7f04313b4640 1 --2- 172.21.15.100:0/4160846200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f042c102510 0x7f042c199ee0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f042800d770 tx=0x7f042800db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:25.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.064+0000 7f04227fc640 1 -- 172.21.15.100:0/4160846200 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0428017070 con 0x7f042c102510 2024-07-26T13:42:25.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.064+0000 7f04227fc640 1 -- 172.21.15.100:0/4160846200 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0428009820 con 0x7f042c102510 2024-07-26T13:42:25.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.064+0000 7f0433618640 1 -- 172.21.15.100:0/4160846200 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f042c1a0080 con 0x7f042c102510 2024-07-26T13:42:25.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.064+0000 7f04227fc640 1 -- 172.21.15.100:0/4160846200 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0428004bc0 con 0x7f042c102510 2024-07-26T13:42:25.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.064+0000 7f0433618640 1 -- 172.21.15.100:0/4160846200 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f042c1a68a0 con 0x7f042c102510 2024-07-26T13:42:25.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.067+0000 7f0439b9c740 1 Processor -- start 2024-07-26T13:42:25.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.067+0000 7f0439b9c740 1 -- start start 2024-07-26T13:42:25.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.067+0000 7f04227fc640 1 -- 172.21.15.100:0/4160846200 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f04280099d0 con 0x7f042c102510 2024-07-26T13:42:25.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.067+0000 7f0439b9c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a1e0cd070 0x555a1e472570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:25.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.067+0000 7f0439b9c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a1e0a41b0 0x555a1e472ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:25.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.067+0000 7f0439b9c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a1e0ac390 0x555a1e472ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:25.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.068+0000 7f0439b9c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x555a1e3e8bb0 con 0x555a1e0cd070 2024-07-26T13:42:25.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.068+0000 7f0439b9c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555a1e3d80b0 con 0x555a1e0a41b0 2024-07-26T13:42:25.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.068+0000 7f0439b9c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555a1e3dc0f0 con 0x555a1e0ac390 2024-07-26T13:42:25.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.068+0000 7f0431bb5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a1e0cd070 0x555a1e472570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:25.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.068+0000 7f0431bb5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a1e0cd070 0x555a1e472570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33818/0 (socket says 172.21.15.100:33818) 2024-07-26T13:42:25.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.068+0000 7f0431bb5640 1 -- 172.21.15.100:0/2744392751 learned_addr learned my addr 172.21.15.100:0/2744392751 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:25.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.068+0000 7f04313b4640 1 --2- 172.21.15.100:0/2744392751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a1e0a41b0 0x555a1e472ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:25.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.068+0000 7f0430bb3640 1 --2- 172.21.15.100:0/2744392751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a1e0ac390 0x555a1e472ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:25.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.068+0000 7f0431bb5640 1 -- 172.21.15.100:0/2744392751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a1e0ac390 msgr2=0x555a1e472ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:25.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.068+0000 7f0431bb5640 1 --2- 172.21.15.100:0/2744392751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a1e0ac390 0x555a1e472ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:25.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.068+0000 7f0431bb5640 1 -- 172.21.15.100:0/2744392751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a1e0a41b0 msgr2=0x555a1e472ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:25.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.068+0000 7f0431bb5640 1 --2- 172.21.15.100:0/2744392751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a1e0a41b0 0x555a1e472ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:25.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.068+0000 7f0431bb5640 1 -- 172.21.15.100:0/2744392751 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a1e37eb90 con 0x555a1e0cd070 2024-07-26T13:42:25.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.068+0000 7f0430bb3640 1 --2- 172.21.15.100:0/2744392751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a1e0ac390 0x555a1e472ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:42:25.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.068+0000 7f04313b4640 1 --2- 172.21.15.100:0/2744392751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a1e0a41b0 0x555a1e472ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:42:25.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.068+0000 7f04227fc640 1 --2- 172.21.15.100:0/4160846200 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f04180b3df0 0x7f04180b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:25.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.068+0000 7f0431bb5640 1 --2- 172.21.15.100:0/2744392751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a1e0cd070 0x555a1e472570 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0424009930 tx=0x7f0424009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a9e3b0e69f743b6 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:25.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.068+0000 7f04313b4640 1 --2- 172.21.15.100:0/4160846200 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f04180b3df0 0x7f04180b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:25.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.069+0000 7f04227fc640 1 -- 172.21.15.100:0/4160846200 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f04280e3a30 con 0x7f042c102510 2024-07-26T13:42:25.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.069+0000 7f04177fe640 1 -- 172.21.15.100:0/2744392751 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0424031df0 con 0x555a1e0cd070 2024-07-26T13:42:25.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.069+0000 7f04177fe640 1 -- 172.21.15.100:0/2744392751 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0424036040 con 0x555a1e0cd070 2024-07-26T13:42:25.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.069+0000 7f04313b4640 1 --2- 172.21.15.100:0/4160846200 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f04180b3df0 0x7f04180b6300 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f04280e1250 tx=0x7f04280e72b0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:25.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.069+0000 7f04177fe640 1 -- 172.21.15.100:0/2744392751 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0424004330 con 0x555a1e0cd070 2024-07-26T13:42:25.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.070+0000 7f0439b9c740 1 -- 172.21.15.100:0/2744392751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a1e0cd070 msgr2=0x555a1e472570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:25.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.070+0000 7f0439b9c740 1 --2- 172.21.15.100:0/2744392751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a1e0cd070 0x555a1e472570 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0424009930 tx=0x7f0424009960 comp rx=0 tx=0).stop 2024-07-26T13:42:25.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.070+0000 7f0439b9c740 1 -- 172.21.15.100:0/2744392751 shutdown_connections 2024-07-26T13:42:25.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.070+0000 7f0439b9c740 1 --2- 172.21.15.100:0/2744392751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a1e0ac390 0x555a1e472ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:25.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.070+0000 7f0439b9c740 1 --2- 172.21.15.100:0/2744392751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a1e0a41b0 0x555a1e472ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:25.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.070+0000 7f0439b9c740 1 --2- 172.21.15.100:0/2744392751 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a1e0cd070 0x555a1e472570 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:25.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.070+0000 7f0439b9c740 1 -- 172.21.15.100:0/2744392751 >> 172.21.15.100:0/2744392751 conn(0x555a1e0c1460 msgr2=0x555a1e470460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:25.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.070+0000 7f0439b9c740 1 -- 172.21.15.100:0/2744392751 shutdown_connections 2024-07-26T13:42:25.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.070+0000 7f0439b9c740 1 -- 172.21.15.100:0/2744392751 wait complete. 2024-07-26T13:42:25.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.070+0000 7f0439b9c740 1 -- start start 2024-07-26T13:42:25.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.070+0000 7f0439b9c740 1 Processor -- start 2024-07-26T13:42:25.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.071+0000 7f0439b9c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a1e0a41b0 0x555a1e47e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:25.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.071+0000 7f0439b9c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a1e0ac390 0x555a1e47e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:25.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.071+0000 7f0439b9c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a1e0cd070 0x555a1e489fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:25.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.071+0000 7f0439b9c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x555a1e23b700 con 0x555a1e0cd070 2024-07-26T13:42:25.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.071+0000 7f0439b9c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555a1e37a9e0 con 0x555a1e0a41b0 2024-07-26T13:42:25.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.071+0000 7f0439b9c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555a1e480cf0 con 0x555a1e0ac390 2024-07-26T13:42:25.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.071+0000 7f0439b9c740 10 client.0 _ll_register_callbacks cb 0x555a1e3954f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:42:25.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.071+0000 7f0431bb5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a1e0cd070 0x555a1e489fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:25.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.071+0000 7f0431bb5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a1e0cd070 0x555a1e489fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33824/0 (socket says 172.21.15.100:33824) 2024-07-26T13:42:25.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.071+0000 7f0431bb5640 1 -- 172.21.15.100:0/1154079301 learned_addr learned my addr 172.21.15.100:0/1154079301 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:25.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.071+0000 7f0431bb5640 1 --2- 172.21.15.100:0/1154079301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a1e0a41b0 0x555a1e47e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:25.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.071+0000 7f0430bb3640 1 --2- 172.21.15.100:0/1154079301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a1e0ac390 0x555a1e47e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:25.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.071+0000 7f0431bb5640 1 -- 172.21.15.100:0/1154079301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a1e0ac390 msgr2=0x555a1e47e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:25.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.071+0000 7f0431bb5640 1 --2- 172.21.15.100:0/1154079301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a1e0ac390 0x555a1e47e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:25.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.071+0000 7f0431bb5640 1 -- 172.21.15.100:0/1154079301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a1e0a41b0 msgr2=0x555a1e47e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:25.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.071+0000 7f0431bb5640 1 --2- 172.21.15.100:0/1154079301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a1e0a41b0 0x555a1e47e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:25.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.071+0000 7f0431bb5640 1 -- 172.21.15.100:0/1154079301 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a1e138650 con 0x555a1e0cd070 2024-07-26T13:42:25.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.072+0000 7f0431bb5640 1 --2- 172.21.15.100:0/1154079301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a1e0cd070 0x555a1e489fe0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x555a1e480810 tx=0x7f042403e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48162140296dc0e server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:25.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.072+0000 7f0414ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:42:25.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.072+0000 7f0414ff9640 1 -- 172.21.15.100:0/1154079301 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0424059060 con 0x555a1e0cd070 2024-07-26T13:42:25.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.072+0000 7f0414ff9640 1 -- 172.21.15.100:0/1154079301 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f0424031650 con 0x555a1e0cd070 2024-07-26T13:42:25.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.072+0000 7f0439b9c740 1 -- 172.21.15.100:0/1154079301 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0424036040 con 0x555a1e0cd070 2024-07-26T13:42:25.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.072+0000 7f0414ff9640 1 -- 172.21.15.100:0/1154079301 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0424031e20 con 0x555a1e0cd070 2024-07-26T13:42:25.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.072+0000 7f0439b9c740 10 client.16143 fetch_fsmap learned FSMap version 27 2024-07-26T13:42:25.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.072+0000 7f0439b9c740 1 -- 172.21.15.100:0/1154079301 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555a1e48cd50 con 0x555a1e0cd070 2024-07-26T13:42:25.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.072+0000 7f0414ff9640 1 -- 172.21.15.100:0/1154079301 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0424036040 con 0x555a1e0cd070 2024-07-26T13:42:25.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.072+0000 7f0414ff9640 1 -- 172.21.15.100:0/1154079301 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f042402f660 con 0x555a1e0cd070 2024-07-26T13:42:25.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.072+0000 7f0439b9c740 10 client.16143 fetch_fsmap finished waiting for FSMap version 27 2024-07-26T13:42:25.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.072+0000 7f0439b9c740 10 client.16143 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-26T13:42:25.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.072+0000 7f0439b9c740 20 client.16143 populate_metadata read hostname 'smithi100' 2024-07-26T13:42:25.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.072+0000 7f0439b9c740 1 --2- 172.21.15.100:0/1154079301 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x555a1e08b9f0 0x555a1e4900b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:25.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.072+0000 7f0439b9c740 4 client.16143 mds_command: new command op to 24595 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:42:25.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.072+0000 7f0439b9c740 1 -- 172.21.15.100:0/1154079301 --> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a1e3eab00 con 0x555a1e08b9f0 2024-07-26T13:42:25.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.073+0000 7f0430bb3640 1 --2- 172.21.15.100:0/1154079301 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x555a1e08b9f0 0x555a1e4900b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:25.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.073+0000 7f0414ff9640 10 client.16143 ms_handle_connect on v2:172.21.15.167:6836/3828634468 2024-07-26T13:42:25.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.073+0000 7f0430bb3640 1 --2- 172.21.15.100:0/1154079301 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x555a1e08b9f0 0x555a1e4900b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1b6fbe5e9cb80242 server_cookie=163eff7a2af1e162 in_seq=0 out_seq=0 2024-07-26T13:42:25.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.074+0000 7f0414ff9640 1 -- 172.21.15.100:0/1154079301 <== mds.2 v2:172.21.15.167:6836/3828634468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555a1e3eab00 con 0x555a1e08b9f0 2024-07-26T13:42:25.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.074+0000 7f0414ff9640 10 client.16143 handle_command_reply: tid=0 2024-07-26T13:42:25.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.074+0000 7f0439b9c740 1 client.16143 shutdown 2024-07-26T13:42:25.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.074+0000 7f0439b9c740 1 -- 172.21.15.100:0/1154079301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a1e0cd070 msgr2=0x555a1e489fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:25.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.074+0000 7f0439b9c740 1 --2- 172.21.15.100:0/1154079301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a1e0cd070 0x555a1e489fe0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x555a1e480810 tx=0x7f042403e040 comp rx=0 tx=0).stop 2024-07-26T13:42:25.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.074+0000 7f0439b9c740 1 -- 172.21.15.100:0/1154079301 shutdown_connections 2024-07-26T13:42:25.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.074+0000 7f0439b9c740 1 --2- 172.21.15.100:0/1154079301 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x555a1e08b9f0 0x555a1e4900b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:25.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.074+0000 7f0414ff9640 0 client.16143 ms_handle_reset on v2:172.21.15.167:6836/3828634468 2024-07-26T13:42:25.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.074+0000 7f0439b9c740 1 --2- 172.21.15.100:0/1154079301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a1e0cd070 0x555a1e489fe0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:25.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.074+0000 7f0439b9c740 1 --2- 172.21.15.100:0/1154079301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a1e0ac390 0x555a1e47e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:25.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.074+0000 7f0439b9c740 1 --2- 172.21.15.100:0/1154079301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a1e0a41b0 0x555a1e47e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:25.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.074+0000 7f0439b9c740 1 -- 172.21.15.100:0/1154079301 >> 172.21.15.100:0/1154079301 conn(0x555a1e0c1460 msgr2=0x555a1e475750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:25.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.075+0000 7f0439b9c740 1 -- 172.21.15.100:0/1154079301 shutdown_connections 2024-07-26T13:42:25.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.075+0000 7f0439b9c740 1 -- 172.21.15.100:0/1154079301 wait complete. 2024-07-26T13:42:25.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.075+0000 7f0439b9c740 20 client.16143 trim_cache size 0 max 16384 2024-07-26T13:42:25.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.087+0000 7f0439b9c740 1 Processor -- start 2024-07-26T13:42:25.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.088+0000 7f0439b9c740 1 -- start start 2024-07-26T13:42:25.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.088+0000 7f0439b9c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a1e08b9f0 0x555a1e470560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:25.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.088+0000 7f0439b9c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a1e0a41b0 0x555a1e489f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:25.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.088+0000 7f0439b9c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a1e0ac390 0x555a1e477650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:25.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.088+0000 7f0439b9c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f042c1b1990 con 0x555a1e0ac390 2024-07-26T13:42:25.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.088+0000 7f0439b9c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f042c1af260 con 0x555a1e0a41b0 2024-07-26T13:42:25.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.088+0000 7f0439b9c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f042c1ad0b0 con 0x555a1e08b9f0 2024-07-26T13:42:25.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.088+0000 7f0431bb5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a1e0ac390 0x555a1e477650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:25.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.088+0000 7f0431bb5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a1e0ac390 0x555a1e477650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33838/0 (socket says 172.21.15.100:33838) 2024-07-26T13:42:25.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.088+0000 7f0431bb5640 1 -- 172.21.15.100:0/2824195295 learned_addr learned my addr 172.21.15.100:0/2824195295 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:25.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.088+0000 7f0430bb3640 1 --2- 172.21.15.100:0/2824195295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a1e0a41b0 0x555a1e489f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:25.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.088+0000 7f0431bb5640 1 --2- 172.21.15.100:0/2824195295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a1e08b9f0 0x555a1e470560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:25.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.088+0000 7f0431bb5640 1 -- 172.21.15.100:0/2824195295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a1e08b9f0 msgr2=0x555a1e470560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:25.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.088+0000 7f0431bb5640 1 --2- 172.21.15.100:0/2824195295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a1e08b9f0 0x555a1e470560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:25.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.088+0000 7f0431bb5640 1 -- 172.21.15.100:0/2824195295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a1e0a41b0 msgr2=0x555a1e489f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:25.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.088+0000 7f0431bb5640 1 --2- 172.21.15.100:0/2824195295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a1e0a41b0 0x555a1e489f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:25.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.088+0000 7f0431bb5640 1 -- 172.21.15.100:0/2824195295 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a1e474860 con 0x555a1e0ac390 2024-07-26T13:42:25.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.088+0000 7f0431bb5640 1 --2- 172.21.15.100:0/2824195295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a1e0ac390 0x555a1e477650 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0424009ca0 tx=0x7f042402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e11a03af6174505 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:25.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.089+0000 7f0416ffd640 1 -- 172.21.15.100:0/2824195295 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0424059070 con 0x555a1e0ac390 2024-07-26T13:42:25.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.089+0000 7f0416ffd640 1 -- 172.21.15.100:0/2824195295 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0424036070 con 0x555a1e0ac390 2024-07-26T13:42:25.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.089+0000 7f0416ffd640 1 -- 172.21.15.100:0/2824195295 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0424031450 con 0x555a1e0ac390 2024-07-26T13:42:25.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.090+0000 7f0439b9c740 1 -- 172.21.15.100:0/2824195295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a1e0ac390 msgr2=0x555a1e477650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:25.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.090+0000 7f0439b9c740 1 --2- 172.21.15.100:0/2824195295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a1e0ac390 0x555a1e477650 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0424009ca0 tx=0x7f042402a8c0 comp rx=0 tx=0).stop 2024-07-26T13:42:25.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.090+0000 7f0439b9c740 1 -- 172.21.15.100:0/2824195295 shutdown_connections 2024-07-26T13:42:25.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.090+0000 7f0439b9c740 1 --2- 172.21.15.100:0/2824195295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a1e0ac390 0x555a1e477650 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:25.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.090+0000 7f0439b9c740 1 --2- 172.21.15.100:0/2824195295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a1e0a41b0 0x555a1e489f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:25.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.090+0000 7f0439b9c740 1 --2- 172.21.15.100:0/2824195295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a1e08b9f0 0x555a1e470560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:25.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.090+0000 7f0439b9c740 1 -- 172.21.15.100:0/2824195295 >> 172.21.15.100:0/2824195295 conn(0x555a1e0c1460 msgr2=0x555a1e46fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:25.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.090+0000 7f0439b9c740 1 -- 172.21.15.100:0/2824195295 shutdown_connections 2024-07-26T13:42:25.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.090+0000 7f0439b9c740 1 -- 172.21.15.100:0/2824195295 wait complete. 2024-07-26T13:42:25.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.090+0000 7f0439b9c740 1 -- start start 2024-07-26T13:42:25.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.090+0000 7f0439b9c740 1 Processor -- start 2024-07-26T13:42:25.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.091+0000 7f0439b9c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a1e08b9f0 0x555a1e493980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:25.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.091+0000 7f0439b9c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a1e0a41b0 0x555a1e48ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:25.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.091+0000 7f0439b9c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a1e0ac390 0x555a1e48f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:25.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.091+0000 7f0439b9c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f042c1ad720 con 0x555a1e0ac390 2024-07-26T13:42:25.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.091+0000 7f0439b9c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f042c1aeb90 con 0x555a1e0a41b0 2024-07-26T13:42:25.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.091+0000 7f0439b9c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f042c1addf0 con 0x555a1e08b9f0 2024-07-26T13:42:25.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.091+0000 7f0439b9c740 10 client.0 _ll_register_callbacks cb 0x7f0424009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:42:25.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.091+0000 7f0431bb5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a1e0ac390 0x555a1e48f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:25.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.091+0000 7f0431bb5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a1e0ac390 0x555a1e48f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33846/0 (socket says 172.21.15.100:33846) 2024-07-26T13:42:25.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.091+0000 7f0431bb5640 1 -- 172.21.15.100:0/3191358450 learned_addr learned my addr 172.21.15.100:0/3191358450 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:25.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.091+0000 7f0430bb3640 1 --2- 172.21.15.100:0/3191358450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a1e0a41b0 0x555a1e48ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:25.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.091+0000 7f0431bb5640 1 --2- 172.21.15.100:0/3191358450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a1e08b9f0 0x555a1e493980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:25.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.091+0000 7f0431bb5640 1 -- 172.21.15.100:0/3191358450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a1e08b9f0 msgr2=0x555a1e493980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:25.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.091+0000 7f0431bb5640 1 --2- 172.21.15.100:0/3191358450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a1e08b9f0 0x555a1e493980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:25.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.091+0000 7f0431bb5640 1 -- 172.21.15.100:0/3191358450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a1e0a41b0 msgr2=0x555a1e48ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:25.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.091+0000 7f0431bb5640 1 --2- 172.21.15.100:0/3191358450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a1e0a41b0 0x555a1e48ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:25.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.091+0000 7f0431bb5640 1 -- 172.21.15.100:0/3191358450 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a1e4f75f0 con 0x555a1e0ac390 2024-07-26T13:42:25.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.091+0000 7f0431bb5640 1 --2- 172.21.15.100:0/3191358450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a1e0ac390 0x555a1e48f540 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f042402f550 tx=0x7f042403e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f3774d2e6d22cda server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:25.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.091+0000 7f0414ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:42:25.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.091+0000 7f0414ff9640 1 -- 172.21.15.100:0/3191358450 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0424059040 con 0x555a1e0ac390 2024-07-26T13:42:25.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.091+0000 7f0414ff9640 1 -- 172.21.15.100:0/3191358450 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f0424004b40 con 0x555a1e0ac390 2024-07-26T13:42:25.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.091+0000 7f0439b9c740 1 -- 172.21.15.100:0/3191358450 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0424036070 con 0x555a1e0ac390 2024-07-26T13:42:25.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.092+0000 7f0414ff9640 1 -- 172.21.15.100:0/3191358450 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0424003ba0 con 0x555a1e0ac390 2024-07-26T13:42:25.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.092+0000 7f0439b9c740 10 client.16149 fetch_fsmap learned FSMap version 27 2024-07-26T13:42:25.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.092+0000 7f0439b9c740 1 -- 172.21.15.100:0/3191358450 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555a1e4f8710 con 0x555a1e0ac390 2024-07-26T13:42:25.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.092+0000 7f0414ff9640 1 -- 172.21.15.100:0/3191358450 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0424036070 con 0x555a1e0ac390 2024-07-26T13:42:25.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.092+0000 7f0414ff9640 1 -- 172.21.15.100:0/3191358450 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f04240305c0 con 0x555a1e0ac390 2024-07-26T13:42:25.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.092+0000 7f0439b9c740 10 client.16149 fetch_fsmap finished waiting for FSMap version 27 2024-07-26T13:42:25.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.092+0000 7f0439b9c740 10 client.16149 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-26T13:42:25.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.092+0000 7f0439b9c740 20 client.16149 populate_metadata read hostname 'smithi100' 2024-07-26T13:42:25.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.092+0000 7f0439b9c740 1 --2- 172.21.15.100:0/3191358450 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x555a1e0cd070 0x555a1e4fab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:25.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.092+0000 7f0439b9c740 4 client.16149 mds_command: new command op to 24595 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-26T13:42:25.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.092+0000 7f0439b9c740 1 -- 172.21.15.100:0/3191358450 --> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555a1e3e65a0 con 0x555a1e0cd070 2024-07-26T13:42:25.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.092+0000 7f0430bb3640 1 --2- 172.21.15.100:0/3191358450 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x555a1e0cd070 0x555a1e4fab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:25.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.093+0000 7f0414ff9640 10 client.16149 ms_handle_connect on v2:172.21.15.167:6836/3828634468 2024-07-26T13:42:25.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.093+0000 7f0430bb3640 1 --2- 172.21.15.100:0/3191358450 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x555a1e0cd070 0x555a1e4fab40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8e7c0fa72be158f8 server_cookie=b303c52cacff892d in_seq=0 out_seq=0 2024-07-26T13:42:25.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.093+0000 7f0414ff9640 1 -- 172.21.15.100:0/3191358450 <== mds.2 v2:172.21.15.167:6836/3828634468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555a1e3e65a0 con 0x555a1e0cd070 2024-07-26T13:42:25.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.093+0000 7f0414ff9640 10 client.16149 handle_command_reply: tid=0 2024-07-26T13:42:25.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.093+0000 7f0439b9c740 1 client.16149 shutdown 2024-07-26T13:42:25.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.093+0000 7f0439b9c740 1 -- 172.21.15.100:0/3191358450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a1e0ac390 msgr2=0x555a1e48f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:25.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.093+0000 7f0439b9c740 1 --2- 172.21.15.100:0/3191358450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a1e0ac390 0x555a1e48f540 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f042402f550 tx=0x7f042403e040 comp rx=0 tx=0).stop 2024-07-26T13:42:25.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.093+0000 7f0439b9c740 1 -- 172.21.15.100:0/3191358450 shutdown_connections 2024-07-26T13:42:25.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.093+0000 7f0439b9c740 1 --2- 172.21.15.100:0/3191358450 >> [v2:172.21.15.167:6836/3828634468,v1:172.21.15.167:6837/3828634468] conn(0x555a1e0cd070 0x555a1e4fab40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:25.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.093+0000 7f0414ff9640 0 client.16149 ms_handle_reset on v2:172.21.15.167:6836/3828634468 2024-07-26T13:42:25.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.093+0000 7f0439b9c740 1 --2- 172.21.15.100:0/3191358450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555a1e0ac390 0x555a1e48f540 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:25.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.093+0000 7f0439b9c740 1 --2- 172.21.15.100:0/3191358450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a1e0a41b0 0x555a1e48ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:25.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.093+0000 7f0439b9c740 1 --2- 172.21.15.100:0/3191358450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555a1e08b9f0 0x555a1e493980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:25.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.093+0000 7f0439b9c740 1 -- 172.21.15.100:0/3191358450 >> 172.21.15.100:0/3191358450 conn(0x555a1e0c1460 msgr2=0x555a1e4735a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:25.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.093+0000 7f0439b9c740 1 -- 172.21.15.100:0/3191358450 shutdown_connections 2024-07-26T13:42:25.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.093+0000 7f0439b9c740 1 -- 172.21.15.100:0/3191358450 wait complete. 2024-07-26T13:42:25.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.093+0000 7f0439b9c740 20 client.16149 trim_cache size 0 max 16384 2024-07-26T13:42:25.100 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-07-26T13:42:25.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.094+0000 7f0433618640 1 -- 172.21.15.100:0/4160846200 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f04180b3df0 msgr2=0x7f04180b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:25.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.094+0000 7f0433618640 1 --2- 172.21.15.100:0/4160846200 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f04180b3df0 0x7f04180b6300 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f04280e1250 tx=0x7f04280e72b0 comp rx=0 tx=0).stop 2024-07-26T13:42:25.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.094+0000 7f0433618640 1 -- 172.21.15.100:0/4160846200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f042c102510 msgr2=0x7f042c199ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:25.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.094+0000 7f0433618640 1 --2- 172.21.15.100:0/4160846200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f042c102510 0x7f042c199ee0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f042800d770 tx=0x7f042800db70 comp rx=0 tx=0).stop 2024-07-26T13:42:25.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.094+0000 7f0433618640 1 -- 172.21.15.100:0/4160846200 shutdown_connections 2024-07-26T13:42:25.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.094+0000 7f0433618640 1 --2- 172.21.15.100:0/4160846200 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f04180b3df0 0x7f04180b6300 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:25.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.094+0000 7f0433618640 1 --2- 172.21.15.100:0/4160846200 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f042c108750 0x7f042c19f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:25.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.094+0000 7f0433618640 1 --2- 172.21.15.100:0/4160846200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f042c107e00 0x7f042c19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:25.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.094+0000 7f0433618640 1 --2- 172.21.15.100:0/4160846200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f042c102510 0x7f042c199ee0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:25.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.094+0000 7f0433618640 1 -- 172.21.15.100:0/4160846200 >> 172.21.15.100:0/4160846200 conn(0x7f042c0791c0 msgr2=0x7f042c103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:25.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.094+0000 7f0433618640 1 -- 172.21.15.100:0/4160846200 shutdown_connections 2024-07-26T13:42:25.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:25.094+0000 7f0433618640 1 -- 172.21.15.100:0/4160846200 wait complete. 2024-07-26T13:42:25.404 INFO:tasks.workunit.client.0.smithi100.stdout: 1 25427 3.97 MB/sec warmup 39 sec latency 9466.887 ms 2024-07-26T13:42:25.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:25 smithi167 ceph-mon[37140]: pgmap v433: 97 pgs: 97 active+clean; 28 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 2.8 MiB/s wr, 114 op/s 2024-07-26T13:42:25.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:25 smithi100 ceph-mon[32599]: pgmap v433: 97 pgs: 97 active+clean; 28 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 2.8 MiB/s wr, 114 op/s 2024-07-26T13:42:25.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:25 smithi138 ceph-mon[36954]: pgmap v433: 97 pgs: 97 active+clean; 28 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 2.8 MiB/s wr, 114 op/s 2024-07-26T13:42:25.968 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-26T13:42:26.223 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:42:26.405 INFO:tasks.workunit.client.0.smithi100.stdout: 1 25427 3.87 MB/sec warmup 40 sec latency 10466.967 ms 2024-07-26T13:42:27.405 INFO:tasks.workunit.client.0.smithi100.stdout: 1 25427 3.78 MB/sec warmup 41 sec latency 11467.066 ms 2024-07-26T13:42:27.740 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:27 smithi100 ceph-mon[32599]: pgmap v434: 97 pgs: 97 active+clean; 28 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 1.4 MiB/s wr, 96 op/s 2024-07-26T13:42:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:27 smithi167 ceph-mon[37140]: pgmap v434: 97 pgs: 97 active+clean; 28 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 1.4 MiB/s wr, 96 op/s 2024-07-26T13:42:27.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:27 smithi138 ceph-mon[36954]: pgmap v434: 97 pgs: 97 active+clean; 28 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 1.4 MiB/s wr, 96 op/s 2024-07-26T13:42:28.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.120+0000 7f2219b09640 1 -- 172.21.15.100:0/1585602491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2214104140 msgr2=0x7f2214104550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:28.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.120+0000 7f2219b09640 1 --2- 172.21.15.100:0/1585602491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2214104140 0x7f2214104550 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f2204009900 tx=0x7f220402a480 comp rx=0 tx=0).stop 2024-07-26T13:42:28.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.120+0000 7f2219b09640 1 -- 172.21.15.100:0/1585602491 shutdown_connections 2024-07-26T13:42:28.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.120+0000 7f2219b09640 1 --2- 172.21.15.100:0/1585602491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2214102370 0x7f2214111ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:28.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.120+0000 7f2219b09640 1 --2- 172.21.15.100:0/1585602491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2214101820 0x7f2214101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:28.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.120+0000 7f2219b09640 1 --2- 172.21.15.100:0/1585602491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2214104140 0x7f2214104550 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:28.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.120+0000 7f2219b09640 1 -- 172.21.15.100:0/1585602491 >> 172.21.15.100:0/1585602491 conn(0x7f22140798f0 msgr2=0x7f2214079d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:28.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.120+0000 7f2219b09640 1 -- 172.21.15.100:0/1585602491 shutdown_connections 2024-07-26T13:42:28.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.120+0000 7f2219b09640 1 -- 172.21.15.100:0/1585602491 wait complete. 2024-07-26T13:42:28.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.121+0000 7f2219b09640 1 Processor -- start 2024-07-26T13:42:28.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.122+0000 7f2219b09640 1 -- start start 2024-07-26T13:42:28.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.122+0000 7f2219b09640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2214101820 0x7f221419c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:28.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.122+0000 7f2219b09640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2214102370 0x7f221419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:28.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.122+0000 7f2219b09640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2214104140 0x7f22141a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:28.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.122+0000 7f2219b09640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2214115780 con 0x7f2214101820 2024-07-26T13:42:28.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.122+0000 7f2219b09640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2214115600 con 0x7f2214104140 2024-07-26T13:42:28.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.122+0000 7f2219b09640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2214115900 con 0x7f2214102370 2024-07-26T13:42:28.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.122+0000 7f22137fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2214101820 0x7f221419c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:28.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.122+0000 7f22137fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2214101820 0x7f221419c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33866/0 (socket says 172.21.15.100:33866) 2024-07-26T13:42:28.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.122+0000 7f22137fe640 1 -- 172.21.15.100:0/659869465 learned_addr learned my addr 172.21.15.100:0/659869465 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:28.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.122+0000 7f2213fff640 1 --2- 172.21.15.100:0/659869465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2214104140 0x7f22141a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:28.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.122+0000 7f2212ffd640 1 --2- 172.21.15.100:0/659869465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2214102370 0x7f221419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:28.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.122+0000 7f22137fe640 1 -- 172.21.15.100:0/659869465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2214102370 msgr2=0x7f221419c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:28.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.122+0000 7f22137fe640 1 --2- 172.21.15.100:0/659869465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2214102370 0x7f221419c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:28.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.122+0000 7f22137fe640 1 -- 172.21.15.100:0/659869465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2214104140 msgr2=0x7f22141a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:28.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.122+0000 7f22137fe640 1 --2- 172.21.15.100:0/659869465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2214104140 0x7f22141a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:28.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.122+0000 7f22137fe640 1 -- 172.21.15.100:0/659869465 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22141a20c0 con 0x7f2214101820 2024-07-26T13:42:28.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.122+0000 7f2213fff640 1 --2- 172.21.15.100:0/659869465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2214104140 0x7f22141a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:42:28.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.122+0000 7f2212ffd640 1 --2- 172.21.15.100:0/659869465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2214102370 0x7f221419c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:42:28.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.123+0000 7f22137fe640 1 --2- 172.21.15.100:0/659869465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2214101820 0x7f221419c150 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2204005940 tx=0x7f22040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:28.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.123+0000 7f2210ff9640 1 -- 172.21.15.100:0/659869465 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220403f030 con 0x7f2214101820 2024-07-26T13:42:28.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.123+0000 7f2219b09640 1 -- 172.21.15.100:0/659869465 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22141a2350 con 0x7f2214101820 2024-07-26T13:42:28.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.123+0000 7f2210ff9640 1 -- 172.21.15.100:0/659869465 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2204003cb0 con 0x7f2214101820 2024-07-26T13:42:28.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.123+0000 7f2210ff9640 1 -- 172.21.15.100:0/659869465 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204035070 con 0x7f2214101820 2024-07-26T13:42:28.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.123+0000 7f2219b09640 1 -- 172.21.15.100:0/659869465 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22141a8b10 con 0x7f2214101820 2024-07-26T13:42:28.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.126+0000 7f222008d740 1 Processor -- start 2024-07-26T13:42:28.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.126+0000 7f222008d740 1 -- start start 2024-07-26T13:42:28.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.126+0000 7f2210ff9640 1 -- 172.21.15.100:0/659869465 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f2204042030 con 0x7f2214101820 2024-07-26T13:42:28.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.126+0000 7f222008d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bdb3e4070 0x556bdb789570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:28.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.126+0000 7f222008d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bdb3bb1b0 0x556bdb789ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:28.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.126+0000 7f222008d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556bdb3c3390 0x556bdb789ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:28.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.126+0000 7f222008d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x556bdb6ffbb0 con 0x556bdb3bb1b0 2024-07-26T13:42:28.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.126+0000 7f222008d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556bdb6ef0b0 con 0x556bdb3c3390 2024-07-26T13:42:28.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.126+0000 7f222008d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556bdb6f30f0 con 0x556bdb3e4070 2024-07-26T13:42:28.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.126+0000 7f22137fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bdb3bb1b0 0x556bdb789ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:28.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.126+0000 7f22137fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bdb3bb1b0 0x556bdb789ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33874/0 (socket says 172.21.15.100:33874) 2024-07-26T13:42:28.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.126+0000 7f22137fe640 1 -- 172.21.15.100:0/1231500000 learned_addr learned my addr 172.21.15.100:0/1231500000 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:28.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.126+0000 7f2212ffd640 1 --2- 172.21.15.100:0/1231500000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556bdb3c3390 0x556bdb789ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:28.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.126+0000 7f2213fff640 1 --2- 172.21.15.100:0/1231500000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bdb3e4070 0x556bdb789570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:28.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.127+0000 7f22137fe640 1 -- 172.21.15.100:0/1231500000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bdb3e4070 msgr2=0x556bdb789570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:28.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.127+0000 7f22137fe640 1 --2- 172.21.15.100:0/1231500000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bdb3e4070 0x556bdb789570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:28.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.127+0000 7f22137fe640 1 -- 172.21.15.100:0/1231500000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556bdb3c3390 msgr2=0x556bdb789ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:28.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.127+0000 7f22137fe640 1 --2- 172.21.15.100:0/1231500000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556bdb3c3390 0x556bdb789ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:28.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.127+0000 7f22137fe640 1 -- 172.21.15.100:0/1231500000 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bdb695b90 con 0x556bdb3bb1b0 2024-07-26T13:42:28.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.127+0000 7f22137fe640 1 --2- 172.21.15.100:0/1231500000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bdb3bb1b0 0x556bdb789ab0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2204102020 tx=0x7f220410ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2890ad1dc268cfef server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:28.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.127+0000 7f2210ff9640 1 --2- 172.21.15.100:0/659869465 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f21fc0b3d40 0x7f21fc0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:28.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.127+0000 7f2213fff640 1 --2- 172.21.15.100:0/659869465 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f21fc0b3d40 0x7f21fc0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:28.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.127+0000 7f21f9ffb640 1 -- 172.21.15.100:0/1231500000 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22040c7bc0 con 0x556bdb3bb1b0 2024-07-26T13:42:28.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.127+0000 7f2210ff9640 1 -- 172.21.15.100:0/659869465 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f2204104940 con 0x7f2214101820 2024-07-26T13:42:28.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.127+0000 7f21f9ffb640 1 -- 172.21.15.100:0/1231500000 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f220403b460 con 0x556bdb3bb1b0 2024-07-26T13:42:28.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.127+0000 7f21f9ffb640 1 -- 172.21.15.100:0/1231500000 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22040cace0 con 0x556bdb3bb1b0 2024-07-26T13:42:28.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.128+0000 7f222008d740 1 -- 172.21.15.100:0/1231500000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bdb3bb1b0 msgr2=0x556bdb789ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:28.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.128+0000 7f2213fff640 1 --2- 172.21.15.100:0/659869465 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f21fc0b3d40 0x7f21fc0b6250 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f220c005fd0 tx=0x7f220c009700 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:28.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.128+0000 7f222008d740 1 --2- 172.21.15.100:0/1231500000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bdb3bb1b0 0x556bdb789ab0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2204102020 tx=0x7f220410ac70 comp rx=0 tx=0).stop 2024-07-26T13:42:28.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.128+0000 7f222008d740 1 -- 172.21.15.100:0/1231500000 shutdown_connections 2024-07-26T13:42:28.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.128+0000 7f222008d740 1 --2- 172.21.15.100:0/1231500000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556bdb3c3390 0x556bdb789ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:28.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.128+0000 7f222008d740 1 --2- 172.21.15.100:0/1231500000 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bdb3bb1b0 0x556bdb789ab0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:28.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.128+0000 7f222008d740 1 --2- 172.21.15.100:0/1231500000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bdb3e4070 0x556bdb789570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:28.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.128+0000 7f222008d740 1 -- 172.21.15.100:0/1231500000 >> 172.21.15.100:0/1231500000 conn(0x556bdb3d8460 msgr2=0x556bdb787460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:28.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.128+0000 7f222008d740 1 -- 172.21.15.100:0/1231500000 shutdown_connections 2024-07-26T13:42:28.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.128+0000 7f222008d740 1 -- 172.21.15.100:0/1231500000 wait complete. 2024-07-26T13:42:28.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.129+0000 7f222008d740 1 -- start start 2024-07-26T13:42:28.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.129+0000 7f222008d740 1 Processor -- start 2024-07-26T13:42:28.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.129+0000 7f222008d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bdb3bb1b0 0x556bdb795110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:28.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.129+0000 7f222008d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bdb3c3390 0x556bdb795650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:28.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.129+0000 7f222008d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556bdb3e4070 0x556bdb7a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:28.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.129+0000 7f222008d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x556bdb552700 con 0x556bdb3bb1b0 2024-07-26T13:42:28.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.129+0000 7f222008d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556bdb6919e0 con 0x556bdb3e4070 2024-07-26T13:42:28.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.129+0000 7f222008d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556bdb797b10 con 0x556bdb3c3390 2024-07-26T13:42:28.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.129+0000 7f22137fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bdb3bb1b0 0x556bdb795110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:28.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.129+0000 7f222008d740 10 client.0 _ll_register_callbacks cb 0x556bdb6ac4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:42:28.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.129+0000 7f22137fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bdb3bb1b0 0x556bdb795110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33888/0 (socket says 172.21.15.100:33888) 2024-07-26T13:42:28.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.129+0000 7f22137fe640 1 -- 172.21.15.100:0/1513350859 learned_addr learned my addr 172.21.15.100:0/1513350859 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:28.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.130+0000 7f2212ffd640 1 --2- 172.21.15.100:0/1513350859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bdb3c3390 0x556bdb795650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:28.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.130+0000 7f2213fff640 1 --2- 172.21.15.100:0/1513350859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556bdb3e4070 0x556bdb7a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:28.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.130+0000 7f22137fe640 1 -- 172.21.15.100:0/1513350859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bdb3c3390 msgr2=0x556bdb795650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:28.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.130+0000 7f22137fe640 1 --2- 172.21.15.100:0/1513350859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bdb3c3390 0x556bdb795650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:28.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.130+0000 7f22137fe640 1 -- 172.21.15.100:0/1513350859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556bdb3e4070 msgr2=0x556bdb7a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:28.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.130+0000 7f22137fe640 1 --2- 172.21.15.100:0/1513350859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556bdb3e4070 0x556bdb7a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:28.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.130+0000 7f22137fe640 1 -- 172.21.15.100:0/1513350859 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bdb44f650 con 0x556bdb3bb1b0 2024-07-26T13:42:28.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.130+0000 7f2213fff640 1 --2- 172.21.15.100:0/1513350859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556bdb3e4070 0x556bdb7a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:42:28.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.130+0000 7f22137fe640 1 --2- 172.21.15.100:0/1513350859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bdb3bb1b0 0x556bdb795110 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f2204102160 tx=0x7f220402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520c0acde103e44e server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:28.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.130+0000 7f21f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:42:28.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.130+0000 7f21f2ffd640 1 -- 172.21.15.100:0/1513350859 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204049070 con 0x556bdb3bb1b0 2024-07-26T13:42:28.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.130+0000 7f21f2ffd640 1 -- 172.21.15.100:0/1513350859 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f220410a6d0 con 0x556bdb3bb1b0 2024-07-26T13:42:28.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.130+0000 7f222008d740 1 -- 172.21.15.100:0/1513350859 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556bdb701b00 con 0x556bdb3bb1b0 2024-07-26T13:42:28.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.130+0000 7f21f2ffd640 1 -- 172.21.15.100:0/1513350859 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f22040caec0 con 0x556bdb3bb1b0 2024-07-26T13:42:28.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.130+0000 7f222008d740 10 client.16161 fetch_fsmap learned FSMap version 27 2024-07-26T13:42:28.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.130+0000 7f222008d740 1 -- 172.21.15.100:0/1513350859 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556bdb7a3a10 con 0x556bdb3bb1b0 2024-07-26T13:42:28.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.131+0000 7f21f2ffd640 1 -- 172.21.15.100:0/1513350859 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556bdb701b00 con 0x556bdb3bb1b0 2024-07-26T13:42:28.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.131+0000 7f21f2ffd640 1 -- 172.21.15.100:0/1513350859 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2204102330 con 0x556bdb3bb1b0 2024-07-26T13:42:28.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.131+0000 7f222008d740 10 client.16161 fetch_fsmap finished waiting for FSMap version 27 2024-07-26T13:42:28.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.131+0000 7f222008d740 10 client.16161 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-26T13:42:28.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.131+0000 7f222008d740 20 client.16161 populate_metadata read hostname 'smithi100' 2024-07-26T13:42:28.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.131+0000 7f222008d740 1 --2- 172.21.15.100:0/1513350859 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x556bdb3a29f0 0x556bdb7a6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:28.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.131+0000 7f222008d740 4 client.16161 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:42:28.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.131+0000 7f222008d740 1 -- 172.21.15.100:0/1513350859 --> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bdb6fd5a0 con 0x556bdb3a29f0 2024-07-26T13:42:28.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.131+0000 7f2212ffd640 1 --2- 172.21.15.100:0/1513350859 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x556bdb3a29f0 0x556bdb7a6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:28.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.132+0000 7f21f2ffd640 10 client.16161 ms_handle_connect on v2:172.21.15.138:6838/1580925471 2024-07-26T13:42:28.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.132+0000 7f2212ffd640 1 --2- 172.21.15.100:0/1513350859 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x556bdb3a29f0 0x556bdb7a6d70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=17dc219c8ad526dd server_cookie=731332c01686a9c8 in_seq=0 out_seq=0 2024-07-26T13:42:28.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.132+0000 7f21f2ffd640 1 -- 172.21.15.100:0/1513350859 <== mds.1 v2:172.21.15.138:6838/1580925471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556bdb6fd5a0 con 0x556bdb3a29f0 2024-07-26T13:42:28.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.132+0000 7f21f2ffd640 10 client.16161 handle_command_reply: tid=0 2024-07-26T13:42:28.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.132+0000 7f222008d740 1 client.16161 shutdown 2024-07-26T13:42:28.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.133+0000 7f222008d740 1 -- 172.21.15.100:0/1513350859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bdb3bb1b0 msgr2=0x556bdb795110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:28.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.133+0000 7f222008d740 1 --2- 172.21.15.100:0/1513350859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bdb3bb1b0 0x556bdb795110 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f2204102160 tx=0x7f220402a480 comp rx=0 tx=0).stop 2024-07-26T13:42:28.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.133+0000 7f222008d740 1 -- 172.21.15.100:0/1513350859 shutdown_connections 2024-07-26T13:42:28.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.133+0000 7f222008d740 1 --2- 172.21.15.100:0/1513350859 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x556bdb3a29f0 0x556bdb7a6d70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:28.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.133+0000 7f222008d740 1 --2- 172.21.15.100:0/1513350859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556bdb3e4070 0x556bdb7a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:28.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.133+0000 7f222008d740 1 --2- 172.21.15.100:0/1513350859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bdb3c3390 0x556bdb795650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:28.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.133+0000 7f222008d740 1 --2- 172.21.15.100:0/1513350859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bdb3bb1b0 0x556bdb795110 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:28.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.133+0000 7f21f2ffd640 0 client.16161 ms_handle_reset on v2:172.21.15.138:6838/1580925471 2024-07-26T13:42:28.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.133+0000 7f222008d740 1 -- 172.21.15.100:0/1513350859 >> 172.21.15.100:0/1513350859 conn(0x556bdb3d8460 msgr2=0x556bdb785c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:28.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.133+0000 7f222008d740 1 -- 172.21.15.100:0/1513350859 shutdown_connections 2024-07-26T13:42:28.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.133+0000 7f222008d740 1 -- 172.21.15.100:0/1513350859 wait complete. 2024-07-26T13:42:28.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.133+0000 7f222008d740 20 client.16161 trim_cache size 0 max 16384 2024-07-26T13:42:28.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.146+0000 7f222008d740 1 Processor -- start 2024-07-26T13:42:28.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.146+0000 7f222008d740 1 -- start start 2024-07-26T13:42:28.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.146+0000 7f222008d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bdb3bb1b0 0x556bdb7862b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:28.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.146+0000 7f222008d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bdb3c3390 0x556bdb787470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:28.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.146+0000 7f222008d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556bdb3e4070 0x556bdb7a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:28.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.146+0000 7f222008d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f22141b3c30 con 0x556bdb3bb1b0 2024-07-26T13:42:28.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.146+0000 7f222008d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f22141b1500 con 0x556bdb3e4070 2024-07-26T13:42:28.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.146+0000 7f222008d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f22141af350 con 0x556bdb3c3390 2024-07-26T13:42:28.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.146+0000 7f22137fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bdb3bb1b0 0x556bdb7862b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:28.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.146+0000 7f22137fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bdb3bb1b0 0x556bdb7862b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33904/0 (socket says 172.21.15.100:33904) 2024-07-26T13:42:28.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.146+0000 7f22137fe640 1 -- 172.21.15.100:0/2833022174 learned_addr learned my addr 172.21.15.100:0/2833022174 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:28.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.146+0000 7f2213fff640 1 --2- 172.21.15.100:0/2833022174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556bdb3e4070 0x556bdb7a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:28.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.146+0000 7f22137fe640 1 -- 172.21.15.100:0/2833022174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bdb3c3390 msgr2=0x556bdb787470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:28.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.146+0000 7f22137fe640 1 --2- 172.21.15.100:0/2833022174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bdb3c3390 0x556bdb787470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:28.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.146+0000 7f22137fe640 1 -- 172.21.15.100:0/2833022174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556bdb3e4070 msgr2=0x556bdb7a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:28.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.146+0000 7f22137fe640 1 --2- 172.21.15.100:0/2833022174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556bdb3e4070 0x556bdb7a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:28.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.146+0000 7f22137fe640 1 -- 172.21.15.100:0/2833022174 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bdb795fb0 con 0x556bdb3bb1b0 2024-07-26T13:42:28.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.146+0000 7f22137fe640 1 --2- 172.21.15.100:0/2833022174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bdb3bb1b0 0x556bdb7862b0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f22040cb500 tx=0x7f2204032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d7499a94e56d96c server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:28.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.147+0000 7f21f97fa640 1 -- 172.21.15.100:0/2833022174 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204049030 con 0x556bdb3bb1b0 2024-07-26T13:42:28.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.147+0000 7f21f97fa640 1 -- 172.21.15.100:0/2833022174 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22040cbc70 con 0x556bdb3bb1b0 2024-07-26T13:42:28.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.147+0000 7f21f97fa640 1 -- 172.21.15.100:0/2833022174 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22040cb0b0 con 0x556bdb3bb1b0 2024-07-26T13:42:28.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.147+0000 7f222008d740 1 -- 172.21.15.100:0/2833022174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bdb3bb1b0 msgr2=0x556bdb7862b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:28.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.147+0000 7f222008d740 1 --2- 172.21.15.100:0/2833022174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bdb3bb1b0 0x556bdb7862b0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f22040cb500 tx=0x7f2204032f60 comp rx=0 tx=0).stop 2024-07-26T13:42:28.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.147+0000 7f222008d740 1 -- 172.21.15.100:0/2833022174 shutdown_connections 2024-07-26T13:42:28.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.147+0000 7f222008d740 1 --2- 172.21.15.100:0/2833022174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556bdb3e4070 0x556bdb7a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:28.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.147+0000 7f222008d740 1 --2- 172.21.15.100:0/2833022174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bdb3c3390 0x556bdb787470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:28.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.147+0000 7f222008d740 1 --2- 172.21.15.100:0/2833022174 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bdb3bb1b0 0x556bdb7862b0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.147+0000 7f222008d740 1 -- 172.21.15.100:0/2833022174 >> 172.21.15.100:0/2833022174 conn(0x556bdb3d8460 msgr2=0x556bdb786d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.147+0000 7f222008d740 1 -- 172.21.15.100:0/2833022174 shutdown_connections 2024-07-26T13:42:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.147+0000 7f222008d740 1 -- 172.21.15.100:0/2833022174 wait complete. 2024-07-26T13:42:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.148+0000 7f222008d740 1 -- start start 2024-07-26T13:42:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.148+0000 7f222008d740 1 Processor -- start 2024-07-26T13:42:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.148+0000 7f222008d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bdb3bb1b0 0x556bdb797c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.148+0000 7f222008d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bdb3c3390 0x556bdb7981c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.148+0000 7f222008d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556bdb3e4070 0x556bdb798700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.148+0000 7f222008d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f22141af9c0 con 0x556bdb3bb1b0 2024-07-26T13:42:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.148+0000 7f222008d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f22141b0e30 con 0x556bdb3e4070 2024-07-26T13:42:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.148+0000 7f222008d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f22141b0090 con 0x556bdb3c3390 2024-07-26T13:42:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.148+0000 7f222008d740 10 client.0 _ll_register_callbacks cb 0x7f2208009340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:42:28.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.148+0000 7f22137fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bdb3bb1b0 0x556bdb797c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:28.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.148+0000 7f22137fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bdb3bb1b0 0x556bdb797c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33916/0 (socket says 172.21.15.100:33916) 2024-07-26T13:42:28.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.148+0000 7f22137fe640 1 -- 172.21.15.100:0/128275305 learned_addr learned my addr 172.21.15.100:0/128275305 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:28.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.149+0000 7f2212ffd640 1 --2- 172.21.15.100:0/128275305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bdb3c3390 0x556bdb7981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:28.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.149+0000 7f22137fe640 1 -- 172.21.15.100:0/128275305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bdb3c3390 msgr2=0x556bdb7981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:28.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.149+0000 7f22137fe640 1 --2- 172.21.15.100:0/128275305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bdb3c3390 0x556bdb7981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:28.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.149+0000 7f22137fe640 1 -- 172.21.15.100:0/128275305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556bdb3e4070 msgr2=0x556bdb798700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:28.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.149+0000 7f22137fe640 1 --2- 172.21.15.100:0/128275305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556bdb3e4070 0x556bdb798700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:28.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.149+0000 7f22137fe640 1 -- 172.21.15.100:0/128275305 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bdb80d620 con 0x556bdb3bb1b0 2024-07-26T13:42:28.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.149+0000 7f22137fe640 1 --2- 172.21.15.100:0/128275305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bdb3bb1b0 0x556bdb797c80 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f2204102110 tx=0x7f220410a4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e77a3106706e4c3 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:28.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.149+0000 7f21f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:42:28.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.149+0000 7f21f2ffd640 1 -- 172.21.15.100:0/128275305 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204049070 con 0x556bdb3bb1b0 2024-07-26T13:42:28.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.149+0000 7f21f2ffd640 1 -- 172.21.15.100:0/128275305 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f220410a620 con 0x556bdb3bb1b0 2024-07-26T13:42:28.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.149+0000 7f222008d740 1 -- 172.21.15.100:0/128275305 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22040cbc70 con 0x556bdb3bb1b0 2024-07-26T13:42:28.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.149+0000 7f21f2ffd640 1 -- 172.21.15.100:0/128275305 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f220408f050 con 0x556bdb3bb1b0 2024-07-26T13:42:28.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.149+0000 7f222008d740 10 client.16167 fetch_fsmap learned FSMap version 27 2024-07-26T13:42:28.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.149+0000 7f222008d740 1 -- 172.21.15.100:0/128275305 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556bdb80d620 con 0x556bdb3bb1b0 2024-07-26T13:42:28.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.149+0000 7f21f2ffd640 1 -- 172.21.15.100:0/128275305 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22040cbc70 con 0x556bdb3bb1b0 2024-07-26T13:42:28.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.150+0000 7f21f2ffd640 1 -- 172.21.15.100:0/128275305 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f22040d4190 con 0x556bdb3bb1b0 2024-07-26T13:42:28.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.150+0000 7f222008d740 10 client.16167 fetch_fsmap finished waiting for FSMap version 27 2024-07-26T13:42:28.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.150+0000 7f222008d740 10 client.16167 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-26T13:42:28.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.150+0000 7f222008d740 20 client.16167 populate_metadata read hostname 'smithi100' 2024-07-26T13:42:28.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.150+0000 7f222008d740 1 --2- 172.21.15.100:0/128275305 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x556bdb377da0 0x556bdb810bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:28.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.150+0000 7f222008d740 4 client.16167 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-26T13:42:28.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.150+0000 7f222008d740 1 -- 172.21.15.100:0/128275305 --> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556bdb6face0 con 0x556bdb377da0 2024-07-26T13:42:28.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.150+0000 7f2212ffd640 1 --2- 172.21.15.100:0/128275305 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x556bdb377da0 0x556bdb810bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:28.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.150+0000 7f2212ffd640 1 --2- 172.21.15.100:0/128275305 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x556bdb377da0 0x556bdb810bc0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d3816315591e0e28 server_cookie=415dbbc65c2fe2e1 in_seq=0 out_seq=0 2024-07-26T13:42:28.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.150+0000 7f21f2ffd640 10 client.16167 ms_handle_connect on v2:172.21.15.138:6838/1580925471 2024-07-26T13:42:28.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.151+0000 7f21f2ffd640 1 -- 172.21.15.100:0/128275305 <== mds.1 v2:172.21.15.138:6838/1580925471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556bdb6face0 con 0x556bdb377da0 2024-07-26T13:42:28.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.151+0000 7f21f2ffd640 10 client.16167 handle_command_reply: tid=0 2024-07-26T13:42:28.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.151+0000 7f222008d740 1 client.16167 shutdown 2024-07-26T13:42:28.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.151+0000 7f222008d740 1 -- 172.21.15.100:0/128275305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bdb3bb1b0 msgr2=0x556bdb797c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:28.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.151+0000 7f222008d740 1 --2- 172.21.15.100:0/128275305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bdb3bb1b0 0x556bdb797c80 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f2204102110 tx=0x7f220410a4f0 comp rx=0 tx=0).stop 2024-07-26T13:42:28.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.151+0000 7f222008d740 1 -- 172.21.15.100:0/128275305 shutdown_connections 2024-07-26T13:42:28.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.151+0000 7f222008d740 1 --2- 172.21.15.100:0/128275305 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x556bdb377da0 0x556bdb810bc0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:28.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.151+0000 7f222008d740 1 --2- 172.21.15.100:0/128275305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556bdb3e4070 0x556bdb798700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:28.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.151+0000 7f21f2ffd640 0 client.16167 ms_handle_reset on v2:172.21.15.138:6838/1580925471 2024-07-26T13:42:28.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.151+0000 7f222008d740 1 --2- 172.21.15.100:0/128275305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bdb3c3390 0x556bdb7981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:28.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.151+0000 7f222008d740 1 --2- 172.21.15.100:0/128275305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556bdb3bb1b0 0x556bdb797c80 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:28.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.151+0000 7f222008d740 1 -- 172.21.15.100:0/128275305 >> 172.21.15.100:0/128275305 conn(0x556bdb3d8460 msgr2=0x556bdb78a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:28.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.151+0000 7f222008d740 1 -- 172.21.15.100:0/128275305 shutdown_connections 2024-07-26T13:42:28.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.151+0000 7f222008d740 1 -- 172.21.15.100:0/128275305 wait complete. 2024-07-26T13:42:28.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.151+0000 7f222008d740 20 client.16167 trim_cache size 0 max 16384 2024-07-26T13:42:28.158 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-07-26T13:42:28.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.151+0000 7f2219b09640 1 -- 172.21.15.100:0/659869465 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f21fc0b3d40 msgr2=0x7f21fc0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:28.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.151+0000 7f2219b09640 1 --2- 172.21.15.100:0/659869465 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f21fc0b3d40 0x7f21fc0b6250 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f220c005fd0 tx=0x7f220c009700 comp rx=0 tx=0).stop 2024-07-26T13:42:28.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.151+0000 7f2219b09640 1 -- 172.21.15.100:0/659869465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2214101820 msgr2=0x7f221419c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:28.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.151+0000 7f2219b09640 1 --2- 172.21.15.100:0/659869465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2214101820 0x7f221419c150 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2204005940 tx=0x7f22040042f0 comp rx=0 tx=0).stop 2024-07-26T13:42:28.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.152+0000 7f2219b09640 1 -- 172.21.15.100:0/659869465 shutdown_connections 2024-07-26T13:42:28.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.152+0000 7f2219b09640 1 --2- 172.21.15.100:0/659869465 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f21fc0b3d40 0x7f21fc0b6250 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:28.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.152+0000 7f2219b09640 1 --2- 172.21.15.100:0/659869465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2214104140 0x7f22141a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:28.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.152+0000 7f2219b09640 1 --2- 172.21.15.100:0/659869465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2214102370 0x7f221419c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:28.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.152+0000 7f2219b09640 1 --2- 172.21.15.100:0/659869465 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2214101820 0x7f221419c150 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:28.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.152+0000 7f2219b09640 1 -- 172.21.15.100:0/659869465 >> 172.21.15.100:0/659869465 conn(0x7f22140798f0 msgr2=0x7f2214100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:28.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.152+0000 7f2219b09640 1 -- 172.21.15.100:0/659869465 shutdown_connections 2024-07-26T13:42:28.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:28.152+0000 7f2219b09640 1 -- 172.21.15.100:0/659869465 wait complete. 2024-07-26T13:42:28.405 INFO:tasks.workunit.client.0.smithi100.stdout: 1 25427 3.69 MB/sec warmup 42 sec latency 12467.168 ms 2024-07-26T13:42:28.949 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-26T13:42:29.205 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:42:29.405 INFO:tasks.workunit.client.0.smithi100.stdout: 1 25427 3.60 MB/sec warmup 43 sec latency 13467.205 ms 2024-07-26T13:42:29.592 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:29 smithi100 ceph-mon[32599]: pgmap v435: 97 pgs: 97 active+clean; 28 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 685 KiB/s wr, 83 op/s 2024-07-26T13:42:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:29.739+0000 7f9fdbfff640 1 -- 172.21.15.100:0/529226126 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5612b95a5da0 con 0x5612b926f4d0 2024-07-26T13:42:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:29.739+0000 7f9fdbfff640 10 client.16131 handle_command_reply: tid=0 2024-07-26T13:42:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:29.739+0000 7fa00a1aa740 1 client.16131 shutdown 2024-07-26T13:42:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:29.740+0000 7fa00a1aa740 1 -- 172.21.15.100:0/529226126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612b92672f0 msgr2=0x5612b96aad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:29.741 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:29.740+0000 7fa00a1aa740 1 --2- 172.21.15.100:0/529226126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612b92672f0 0x5612b96aad10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f9ff00a78a0 tx=0x7f9ff0000bc0 comp rx=0 tx=0).stop 2024-07-26T13:42:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:29.740+0000 7fa00a1aa740 1 -- 172.21.15.100:0/529226126 shutdown_connections 2024-07-26T13:42:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:29.740+0000 7fa00a1aa740 1 --2- 172.21.15.100:0/529226126 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5612b926f4d0 0x5612b96b9e60 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:29.740+0000 7fa00a1aa740 1 --2- 172.21.15.100:0/529226126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5612b9284300 0x5612b96afd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:29.740+0000 7fa00a1aa740 1 --2- 172.21.15.100:0/529226126 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5612b92672f0 0x5612b96aad10 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:29.740+0000 7f9fdbfff640 0 client.16131 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:29.740+0000 7fa00a1aa740 1 --2- 172.21.15.100:0/529226126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5612b924ed80 0x5612b96aa730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:29.740+0000 7fa00a1aa740 1 -- 172.21.15.100:0/529226126 >> 172.21.15.100:0/529226126 conn(0x5612b96306b0 msgr2=0x5612b963fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:29.740+0000 7fa00a1aa740 1 -- 172.21.15.100:0/529226126 shutdown_connections 2024-07-26T13:42:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:29.740+0000 7fa00a1aa740 1 -- 172.21.15.100:0/529226126 wait complete. 2024-07-26T13:42:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:29.740+0000 7fa00a1aa740 20 client.16131 trim_cache size 0 max 16384 2024-07-26T13:42:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-07-26T13:42:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-07-26T13:42:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-07-26T13:42:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-07-26T13:42:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:29.740+0000 7fa0031be640 1 -- 172.21.15.100:0/3263597520 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9fe40b3d40 msgr2=0x7f9fe40b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:29.741+0000 7fa0031be640 1 --2- 172.21.15.100:0/3263597520 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9fe40b3d40 0x7f9fe40b6250 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9ff4007ee0 tx=0x7f9ff4007430 comp rx=0 tx=0).stop 2024-07-26T13:42:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:29.741+0000 7fa0031be640 1 -- 172.21.15.100:0/3263597520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ffc104ce0 msgr2=0x7f9ffc19a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:29.741+0000 7fa0031be640 1 --2- 172.21.15.100:0/3263597520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ffc104ce0 0x7f9ffc19a440 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f9ff000d740 tx=0x7f9ff000dc00 comp rx=0 tx=0).stop 2024-07-26T13:42:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:29.741+0000 7fa0031be640 1 -- 172.21.15.100:0/3263597520 shutdown_connections 2024-07-26T13:42:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:29.741+0000 7fa0031be640 1 --2- 172.21.15.100:0/3263597520 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9fe40b3d40 0x7f9fe40b6250 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:29.741+0000 7fa0031be640 1 --2- 172.21.15.100:0/3263597520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ffc10d7b0 0x7f9ffc19f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:29.741+0000 7fa0031be640 1 --2- 172.21.15.100:0/3263597520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ffc104ce0 0x7f9ffc19a440 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:29.741+0000 7fa0031be640 1 --2- 172.21.15.100:0/3263597520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ffc104390 0x7f9ffc199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:29.741+0000 7fa0031be640 1 -- 172.21.15.100:0/3263597520 >> 172.21.15.100:0/3263597520 conn(0x7f9ffc07a170 msgr2=0x7f9ffc100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:29.741+0000 7fa0031be640 1 -- 172.21.15.100:0/3263597520 shutdown_connections 2024-07-26T13:42:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:29.741+0000 7fa0031be640 1 -- 172.21.15.100:0/3263597520 wait complete. 2024-07-26T13:42:29.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:29 smithi167 ceph-mon[37140]: pgmap v435: 97 pgs: 97 active+clean; 28 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 685 KiB/s wr, 83 op/s 2024-07-26T13:42:29.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:29 smithi138 ceph-mon[36954]: pgmap v435: 97 pgs: 97 active+clean; 28 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 685 KiB/s wr, 83 op/s 2024-07-26T13:42:30.405 INFO:tasks.workunit.client.0.smithi100.stdout: 1 27292 3.68 MB/sec warmup 44 sec latency 13804.091 ms 2024-07-26T13:42:31.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.009+0000 7f04d3fc7640 1 -- 172.21.15.100:0/713264101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04cc102510 msgr2=0x7f04cc102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:31.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.009+0000 7f04d3fc7640 1 --2- 172.21.15.100:0/713264101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04cc102510 0x7f04cc102920 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f04c0009900 tx=0x7f04c002a410 comp rx=0 tx=0).stop 2024-07-26T13:42:31.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.009+0000 7f04d3fc7640 1 -- 172.21.15.100:0/713264101 shutdown_connections 2024-07-26T13:42:31.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.009+0000 7f04d3fc7640 1 --2- 172.21.15.100:0/713264101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04cc108750 0x7f04cc10ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:31.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.009+0000 7f04d3fc7640 1 --2- 172.21.15.100:0/713264101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04cc107e00 0x7f04cc108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:31.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.009+0000 7f04d3fc7640 1 --2- 172.21.15.100:0/713264101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04cc102510 0x7f04cc102920 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:31.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.009+0000 7f04d3fc7640 1 -- 172.21.15.100:0/713264101 >> 172.21.15.100:0/713264101 conn(0x7f04cc079190 msgr2=0x7f04cc0795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:31.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.009+0000 7f04d3fc7640 1 -- 172.21.15.100:0/713264101 shutdown_connections 2024-07-26T13:42:31.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.009+0000 7f04d3fc7640 1 -- 172.21.15.100:0/713264101 wait complete. 2024-07-26T13:42:31.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.010+0000 7f04d3fc7640 1 Processor -- start 2024-07-26T13:42:31.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.010+0000 7f04d3fc7640 1 -- start start 2024-07-26T13:42:31.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.010+0000 7f04d3fc7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04cc102510 0x7f04cc199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:31.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.010+0000 7f04d3fc7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04cc107e00 0x7f04cc19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:31.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.010+0000 7f04d3fc7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04cc108750 0x7f04cc19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:31.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.010+0000 7f04d3fc7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f04cc1134d0 con 0x7f04cc102510 2024-07-26T13:42:31.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.010+0000 7f04d3fc7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f04cc113350 con 0x7f04cc108750 2024-07-26T13:42:31.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.010+0000 7f04d3fc7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f04cc113650 con 0x7f04cc107e00 2024-07-26T13:42:31.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.010+0000 7f04d1d63640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04cc102510 0x7f04cc199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:31.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.010+0000 7f04d1d63640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04cc102510 0x7f04cc199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33944/0 (socket says 172.21.15.100:33944) 2024-07-26T13:42:31.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.010+0000 7f04d1d63640 1 -- 172.21.15.100:0/2747781485 learned_addr learned my addr 172.21.15.100:0/2747781485 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:31.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.011+0000 7f04d1562640 1 --2- 172.21.15.100:0/2747781485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04cc107e00 0x7f04cc19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:31.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.011+0000 7f04d2564640 1 --2- 172.21.15.100:0/2747781485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04cc108750 0x7f04cc19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:31.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.011+0000 7f04d1562640 1 -- 172.21.15.100:0/2747781485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04cc108750 msgr2=0x7f04cc19f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:31.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.011+0000 7f04d1562640 1 --2- 172.21.15.100:0/2747781485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04cc108750 0x7f04cc19f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:31.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.011+0000 7f04d1562640 1 -- 172.21.15.100:0/2747781485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04cc102510 msgr2=0x7f04cc199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:31.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.011+0000 7f04d1562640 1 --2- 172.21.15.100:0/2747781485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04cc102510 0x7f04cc199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:31.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.011+0000 7f04d1562640 1 -- 172.21.15.100:0/2747781485 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04cc19fdf0 con 0x7f04cc107e00 2024-07-26T13:42:31.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.011+0000 7f04d1d63640 1 --2- 172.21.15.100:0/2747781485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04cc102510 0x7f04cc199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:42:31.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.011+0000 7f04d2564640 1 --2- 172.21.15.100:0/2747781485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04cc108750 0x7f04cc19f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:42:31.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.011+0000 7f04d1562640 1 --2- 172.21.15.100:0/2747781485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04cc107e00 0x7f04cc19a420 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f04c800e9c0 tx=0x7f04c800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:31.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.011+0000 7f04beffd640 1 -- 172.21.15.100:0/2747781485 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c800ccc0 con 0x7f04cc107e00 2024-07-26T13:42:31.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.011+0000 7f04d3fc7640 1 -- 172.21.15.100:0/2747781485 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04cc1a00e0 con 0x7f04cc107e00 2024-07-26T13:42:31.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.011+0000 7f04beffd640 1 -- 172.21.15.100:0/2747781485 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04c80108e0 con 0x7f04cc107e00 2024-07-26T13:42:31.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.011+0000 7f04beffd640 1 -- 172.21.15.100:0/2747781485 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c80187a0 con 0x7f04cc107e00 2024-07-26T13:42:31.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.011+0000 7f04d3fc7640 1 -- 172.21.15.100:0/2747781485 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04cc1a6840 con 0x7f04cc107e00 2024-07-26T13:42:31.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.014+0000 7f04da54b740 1 Processor -- start 2024-07-26T13:42:31.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.014+0000 7f04da54b740 1 -- start start 2024-07-26T13:42:31.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.014+0000 7f04beffd640 1 -- 172.21.15.100:0/2747781485 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f04c8010430 con 0x7f04cc107e00 2024-07-26T13:42:31.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.014+0000 7f04da54b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064fe16070 0x5606501bb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:31.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.014+0000 7f04da54b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064fded1b0 0x5606501bbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:31.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.014+0000 7f04da54b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56064fdf5390 0x5606501bbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:31.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.014+0000 7f04da54b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x560650131bb0 con 0x56064fe16070 2024-07-26T13:42:31.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.014+0000 7f04da54b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5606501210b0 con 0x56064fded1b0 2024-07-26T13:42:31.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.014+0000 7f04da54b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5606501250f0 con 0x56064fdf5390 2024-07-26T13:42:31.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.015+0000 7f04d1d63640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064fded1b0 0x5606501bbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:31.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.015+0000 7f04d2564640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064fe16070 0x5606501bb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:31.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.015+0000 7f04d2564640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064fe16070 0x5606501bb570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33958/0 (socket says 172.21.15.100:33958) 2024-07-26T13:42:31.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.015+0000 7f04d1d63640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064fded1b0 0x5606501bbab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:46010/0 (socket says 172.21.15.100:46010) 2024-07-26T13:42:31.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.015+0000 7f04d2564640 1 -- 172.21.15.100:0/1423348714 learned_addr learned my addr 172.21.15.100:0/1423348714 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:31.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.015+0000 7f04d1562640 1 --2- 172.21.15.100:0/1423348714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56064fdf5390 0x5606501bbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:31.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.015+0000 7f04d1d63640 1 -- 172.21.15.100:0/1423348714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56064fdf5390 msgr2=0x5606501bbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:31.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.015+0000 7f04d1d63640 1 --2- 172.21.15.100:0/1423348714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56064fdf5390 0x5606501bbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:31.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.015+0000 7f04d1d63640 1 -- 172.21.15.100:0/1423348714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064fe16070 msgr2=0x5606501bb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:31.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.015+0000 7f04d1d63640 1 --2- 172.21.15.100:0/1423348714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064fe16070 0x5606501bb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:31.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.015+0000 7f04d1d63640 1 -- 172.21.15.100:0/1423348714 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606500c7b90 con 0x56064fded1b0 2024-07-26T13:42:31.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.015+0000 7f04d1d63640 1 --2- 172.21.15.100:0/1423348714 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064fded1b0 0x5606501bbab0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f04c0004770 tx=0x7f04c0005ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0ff511fb05343f0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:31.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.015+0000 7f04b7fff640 1 -- 172.21.15.100:0/1423348714 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c0035040 con 0x56064fded1b0 2024-07-26T13:42:31.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.016+0000 7f04b7fff640 1 -- 172.21.15.100:0/1423348714 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04c0031820 con 0x56064fded1b0 2024-07-26T13:42:31.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.016+0000 7f04b7fff640 1 -- 172.21.15.100:0/1423348714 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c00049e0 con 0x56064fded1b0 2024-07-26T13:42:31.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.016+0000 7f04beffd640 1 --2- 172.21.15.100:0/2747781485 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f04b80b3d40 0x7f04b80b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:31.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.016+0000 7f04beffd640 1 -- 172.21.15.100:0/2747781485 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f04c80e4310 con 0x7f04cc107e00 2024-07-26T13:42:31.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.016+0000 7f04d2564640 1 --2- 172.21.15.100:0/2747781485 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f04b80b3d40 0x7f04b80b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:31.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.016+0000 7f04d2564640 1 --2- 172.21.15.100:0/2747781485 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f04b80b3d40 0x7f04b80b6250 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f04c4009fd0 tx=0x7f04c4009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:31.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.016+0000 7f04da54b740 1 -- 172.21.15.100:0/1423348714 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064fded1b0 msgr2=0x5606501bbab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:31.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.016+0000 7f04da54b740 1 --2- 172.21.15.100:0/1423348714 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064fded1b0 0x5606501bbab0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f04c0004770 tx=0x7f04c0005ed0 comp rx=0 tx=0).stop 2024-07-26T13:42:31.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.017+0000 7f04da54b740 1 -- 172.21.15.100:0/1423348714 shutdown_connections 2024-07-26T13:42:31.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.017+0000 7f04da54b740 1 --2- 172.21.15.100:0/1423348714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56064fdf5390 0x5606501bbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:31.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.017+0000 7f04da54b740 1 --2- 172.21.15.100:0/1423348714 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064fded1b0 0x5606501bbab0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:31.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.017+0000 7f04da54b740 1 --2- 172.21.15.100:0/1423348714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064fe16070 0x5606501bb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:31.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.017+0000 7f04da54b740 1 -- 172.21.15.100:0/1423348714 >> 172.21.15.100:0/1423348714 conn(0x56064fe0a460 msgr2=0x5606501b9460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:31.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.017+0000 7f04da54b740 1 -- 172.21.15.100:0/1423348714 shutdown_connections 2024-07-26T13:42:31.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.017+0000 7f04da54b740 1 -- 172.21.15.100:0/1423348714 wait complete. 2024-07-26T13:42:31.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.017+0000 7f04da54b740 1 -- start start 2024-07-26T13:42:31.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.017+0000 7f04da54b740 1 Processor -- start 2024-07-26T13:42:31.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.018+0000 7f04da54b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56064fded1b0 0x5606501c7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:31.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.018+0000 7f04da54b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064fdf5390 0x5606501c7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:31.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.018+0000 7f04da54b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064fe16070 0x5606501d2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:31.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.018+0000 7f04da54b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56064ff84700 con 0x56064fdf5390 2024-07-26T13:42:31.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.018+0000 7f04da54b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5606500c39e0 con 0x56064fe16070 2024-07-26T13:42:31.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.018+0000 7f04da54b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5606501c9cf0 con 0x56064fded1b0 2024-07-26T13:42:31.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.018+0000 7f04da54b740 10 client.0 _ll_register_callbacks cb 0x5606500de4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:42:31.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.018+0000 7f04d1562640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064fdf5390 0x5606501c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:31.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.018+0000 7f04d2564640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064fe16070 0x5606501d2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:31.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.018+0000 7f04d1562640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064fdf5390 0x5606501c7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33962/0 (socket says 172.21.15.100:33962) 2024-07-26T13:42:31.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.018+0000 7f04d1562640 1 -- 172.21.15.100:0/3897654933 learned_addr learned my addr 172.21.15.100:0/3897654933 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:31.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.018+0000 7f04d1d63640 1 --2- 172.21.15.100:0/3897654933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56064fded1b0 0x5606501c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:31.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.018+0000 7f04d2564640 1 -- 172.21.15.100:0/3897654933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56064fded1b0 msgr2=0x5606501c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:31.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.018+0000 7f04d2564640 1 --2- 172.21.15.100:0/3897654933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56064fded1b0 0x5606501c7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:31.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.018+0000 7f04d2564640 1 -- 172.21.15.100:0/3897654933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064fdf5390 msgr2=0x5606501c7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:31.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.018+0000 7f04d2564640 1 --2- 172.21.15.100:0/3897654933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064fdf5390 0x5606501c7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:31.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.018+0000 7f04d2564640 1 -- 172.21.15.100:0/3897654933 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56064fe81650 con 0x56064fe16070 2024-07-26T13:42:31.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.018+0000 7f04d1562640 1 --2- 172.21.15.100:0/3897654933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064fdf5390 0x5606501c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:42:31.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.018+0000 7f04d1d63640 1 --2- 172.21.15.100:0/3897654933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56064fded1b0 0x5606501c7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:42:31.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.018+0000 7f04d2564640 1 --2- 172.21.15.100:0/3897654933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064fe16070 0x5606501d2fe0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f04c40048a0 tx=0x7f04c4004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=184549a278b5cb05 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:31.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.018+0000 7f04b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-07-26T13:42:31.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.019+0000 7f04b57fa640 1 -- 172.21.15.100:0/3897654933 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c402d060 con 0x56064fe16070 2024-07-26T13:42:31.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.019+0000 7f04b57fa640 1 -- 172.21.15.100:0/3897654933 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f04c4017030 con 0x56064fe16070 2024-07-26T13:42:31.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.019+0000 7f04da54b740 1 -- 172.21.15.100:0/3897654933 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04c0031820 con 0x56064fe16070 2024-07-26T13:42:31.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.019+0000 7f04b57fa640 1 -- 172.21.15.100:0/3897654933 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f04c401d620 con 0x56064fe16070 2024-07-26T13:42:31.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.019+0000 7f04da54b740 10 client.25556 fetch_fsmap learned FSMap version 27 2024-07-26T13:42:31.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.019+0000 7f04da54b740 1 -- 172.21.15.100:0/3897654933 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5606501d5d30 con 0x56064fe16070 2024-07-26T13:42:31.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.019+0000 7f04b57fa640 1 -- 172.21.15.100:0/3897654933 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04c0031820 con 0x56064fe16070 2024-07-26T13:42:31.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.019+0000 7f04b57fa640 1 -- 172.21.15.100:0/3897654933 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f04c40289d0 con 0x56064fe16070 2024-07-26T13:42:31.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.019+0000 7f04da54b740 10 client.25556 fetch_fsmap finished waiting for FSMap version 27 2024-07-26T13:42:31.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.019+0000 7f04da54b740 10 client.25556 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:42:31.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.019+0000 7f04da54b740 20 client.25556 populate_metadata read hostname 'smithi100' 2024-07-26T13:42:31.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.019+0000 7f04da54b740 1 --2- 172.21.15.100:0/3897654933 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56064fdd49f0 0x5606501d9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:31.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.019+0000 7f04da54b740 4 client.25556 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:42:31.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.019+0000 7f04da54b740 1 -- 172.21.15.100:0/3897654933 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560650133b00 con 0x56064fdd49f0 2024-07-26T13:42:31.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.019+0000 7f04d1d63640 1 --2- 172.21.15.100:0/3897654933 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56064fdd49f0 0x5606501d9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:31.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.020+0000 7f04b57fa640 10 client.25556 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:31.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.020+0000 7f04d1d63640 1 --2- 172.21.15.100:0/3897654933 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56064fdd49f0 0x5606501d9360 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf6a46075693458 server_cookie=d8243ea08689aa17 in_seq=0 out_seq=0 2024-07-26T13:42:31.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.021+0000 7f04b57fa640 1 -- 172.21.15.100:0/3897654933 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560650133b00 con 0x56064fdd49f0 2024-07-26T13:42:31.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.021+0000 7f04b57fa640 10 client.25556 handle_command_reply: tid=0 2024-07-26T13:42:31.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.021+0000 7f04da54b740 1 client.25556 shutdown 2024-07-26T13:42:31.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.021+0000 7f04da54b740 1 -- 172.21.15.100:0/3897654933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064fe16070 msgr2=0x5606501d2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:31.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.021+0000 7f04da54b740 1 --2- 172.21.15.100:0/3897654933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064fe16070 0x5606501d2fe0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f04c40048a0 tx=0x7f04c4004ab0 comp rx=0 tx=0).stop 2024-07-26T13:42:31.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.021+0000 7f04da54b740 1 -- 172.21.15.100:0/3897654933 shutdown_connections 2024-07-26T13:42:31.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.021+0000 7f04da54b740 1 --2- 172.21.15.100:0/3897654933 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56064fdd49f0 0x5606501d9360 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:31.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.021+0000 7f04da54b740 1 --2- 172.21.15.100:0/3897654933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064fe16070 0x5606501d2fe0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:31.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.021+0000 7f04da54b740 1 --2- 172.21.15.100:0/3897654933 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064fdf5390 0x5606501c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:31.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.021+0000 7f04da54b740 1 --2- 172.21.15.100:0/3897654933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56064fded1b0 0x5606501c7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:31.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.021+0000 7f04b57fa640 0 client.25556 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:31.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.021+0000 7f04da54b740 1 -- 172.21.15.100:0/3897654933 >> 172.21.15.100:0/3897654933 conn(0x56064fe0a460 msgr2=0x5606501b7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:31.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.021+0000 7f04da54b740 1 -- 172.21.15.100:0/3897654933 shutdown_connections 2024-07-26T13:42:31.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.021+0000 7f04da54b740 1 -- 172.21.15.100:0/3897654933 wait complete. 2024-07-26T13:42:31.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.021+0000 7f04da54b740 20 client.25556 trim_cache size 0 max 16384 2024-07-26T13:42:31.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.037+0000 7f04da54b740 1 Processor -- start 2024-07-26T13:42:31.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.037+0000 7f04da54b740 1 -- start start 2024-07-26T13:42:31.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.037+0000 7f04da54b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56064fded1b0 0x5606501b93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:31.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.037+0000 7f04da54b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064fdf5390 0x5606501deb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:31.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.037+0000 7f04da54b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064fe16070 0x5606501df0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:31.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.037+0000 7f04da54b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f04cc1b19e0 con 0x56064fdf5390 2024-07-26T13:42:31.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.037+0000 7f04da54b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f04cc1af2b0 con 0x56064fe16070 2024-07-26T13:42:31.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.037+0000 7f04da54b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f04cc1ad100 con 0x56064fded1b0 2024-07-26T13:42:31.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.037+0000 7f04d1562640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064fdf5390 0x5606501deb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:31.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.037+0000 7f04d1d63640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56064fded1b0 0x5606501b93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:31.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.037+0000 7f04d1d63640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56064fded1b0 0x5606501b93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:44530/0 (socket says 172.21.15.100:44530) 2024-07-26T13:42:31.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.037+0000 7f04d1d63640 1 -- 172.21.15.100:0/4100092977 learned_addr learned my addr 172.21.15.100:0/4100092977 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:31.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.038+0000 7f04d2564640 1 --2- 172.21.15.100:0/4100092977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064fe16070 0x5606501df0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:31.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.038+0000 7f04d1d63640 1 -- 172.21.15.100:0/4100092977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064fe16070 msgr2=0x5606501df0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:31.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.038+0000 7f04d1d63640 1 --2- 172.21.15.100:0/4100092977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064fe16070 0x5606501df0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:31.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.038+0000 7f04d1d63640 1 -- 172.21.15.100:0/4100092977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064fdf5390 msgr2=0x5606501deb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:31.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.038+0000 7f04d1d63640 1 --2- 172.21.15.100:0/4100092977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064fdf5390 0x5606501deb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:31.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.038+0000 7f04d1d63640 1 -- 172.21.15.100:0/4100092977 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606501b9910 con 0x56064fded1b0 2024-07-26T13:42:31.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.038+0000 7f04d1562640 1 --2- 172.21.15.100:0/4100092977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064fdf5390 0x5606501deb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:42:31.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.038+0000 7f04d1d63640 1 --2- 172.21.15.100:0/4100092977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56064fded1b0 0x5606501b93d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f04c0009a20 tx=0x7f04c00319d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1c7ab7070749182 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:31.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.038+0000 7f04b77fe640 1 -- 172.21.15.100:0/4100092977 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c0035040 con 0x56064fded1b0 2024-07-26T13:42:31.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.038+0000 7f04b77fe640 1 -- 172.21.15.100:0/4100092977 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04c0056070 con 0x56064fded1b0 2024-07-26T13:42:31.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.038+0000 7f04b77fe640 1 -- 172.21.15.100:0/4100092977 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c00035f0 con 0x56064fded1b0 2024-07-26T13:42:31.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.039+0000 7f04da54b740 1 -- 172.21.15.100:0/4100092977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56064fded1b0 msgr2=0x5606501b93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:31.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.039+0000 7f04da54b740 1 --2- 172.21.15.100:0/4100092977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56064fded1b0 0x5606501b93d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f04c0009a20 tx=0x7f04c00319d0 comp rx=0 tx=0).stop 2024-07-26T13:42:31.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.039+0000 7f04da54b740 1 -- 172.21.15.100:0/4100092977 shutdown_connections 2024-07-26T13:42:31.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.039+0000 7f04da54b740 1 --2- 172.21.15.100:0/4100092977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064fe16070 0x5606501df0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:31.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.039+0000 7f04da54b740 1 --2- 172.21.15.100:0/4100092977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064fdf5390 0x5606501deb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:31.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.039+0000 7f04da54b740 1 --2- 172.21.15.100:0/4100092977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56064fded1b0 0x5606501b93d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:31.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.039+0000 7f04da54b740 1 -- 172.21.15.100:0/4100092977 >> 172.21.15.100:0/4100092977 conn(0x56064fe0a460 msgr2=0x5606501b8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:31.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.039+0000 7f04da54b740 1 -- 172.21.15.100:0/4100092977 shutdown_connections 2024-07-26T13:42:31.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.039+0000 7f04da54b740 1 -- 172.21.15.100:0/4100092977 wait complete. 2024-07-26T13:42:31.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.039+0000 7f04da54b740 1 -- start start 2024-07-26T13:42:31.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.040+0000 7f04da54b740 1 Processor -- start 2024-07-26T13:42:31.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.040+0000 7f04da54b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064fded1b0 0x5606501cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:31.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.040+0000 7f04da54b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56064fdf5390 0x5606501cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:31.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.040+0000 7f04da54b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064fe16070 0x5606501cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:31.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.040+0000 7f04da54b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f04cc1ad770 con 0x56064fe16070 2024-07-26T13:42:31.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.040+0000 7f04da54b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f04cc1aebe0 con 0x56064fded1b0 2024-07-26T13:42:31.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.040+0000 7f04da54b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f04cc1ade40 con 0x56064fdf5390 2024-07-26T13:42:31.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.040+0000 7f04da54b740 10 client.0 _ll_register_callbacks cb 0x7f04c4003bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:42:31.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.040+0000 7f04d2564640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064fe16070 0x5606501cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:31.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.040+0000 7f04d1562640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56064fdf5390 0x5606501cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:31.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.040+0000 7f04d1d63640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064fded1b0 0x5606501cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:31.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.040+0000 7f04d1562640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56064fdf5390 0x5606501cb4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:44532/0 (socket says 172.21.15.100:44532) 2024-07-26T13:42:31.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.040+0000 7f04d2564640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064fe16070 0x5606501cba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33990/0 (socket says 172.21.15.100:33990) 2024-07-26T13:42:31.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.040+0000 7f04d1562640 1 -- 172.21.15.100:0/3921289694 learned_addr learned my addr 172.21.15.100:0/3921289694 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:31.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.041+0000 7f04d2564640 1 -- 172.21.15.100:0/3921289694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56064fdf5390 msgr2=0x5606501cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:31.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.041+0000 7f04d2564640 1 --2- 172.21.15.100:0/3921289694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56064fdf5390 0x5606501cb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:31.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.041+0000 7f04d2564640 1 -- 172.21.15.100:0/3921289694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064fded1b0 msgr2=0x5606501cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:31.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.041+0000 7f04d2564640 1 --2- 172.21.15.100:0/3921289694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064fded1b0 0x5606501cafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:31.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.041+0000 7f04d2564640 1 -- 172.21.15.100:0/3921289694 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560650240ce0 con 0x56064fe16070 2024-07-26T13:42:31.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.041+0000 7f04d1562640 1 --2- 172.21.15.100:0/3921289694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56064fdf5390 0x5606501cb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:42:31.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.041+0000 7f04d2564640 1 --2- 172.21.15.100:0/3921289694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064fe16070 0x5606501cba20 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f04c4018040 tx=0x7f04c401d270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd2d4fc91ef0a01c server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:31.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.041+0000 7f04b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:42:31.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.041+0000 7f04b57fa640 1 -- 172.21.15.100:0/3921289694 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c400fa40 con 0x56064fe16070 2024-07-26T13:42:31.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.041+0000 7f04da54b740 1 -- 172.21.15.100:0/3921289694 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04c0056070 con 0x56064fe16070 2024-07-26T13:42:31.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.041+0000 7f04b57fa640 1 -- 172.21.15.100:0/3921289694 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f04c400fbe0 con 0x56064fe16070 2024-07-26T13:42:31.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.041+0000 7f04b57fa640 1 -- 172.21.15.100:0/3921289694 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f04c400fde0 con 0x56064fe16070 2024-07-26T13:42:31.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.041+0000 7f04da54b740 10 client.16185 fetch_fsmap learned FSMap version 27 2024-07-26T13:42:31.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.041+0000 7f04da54b740 1 -- 172.21.15.100:0/3921289694 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560650241f40 con 0x56064fe16070 2024-07-26T13:42:31.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.041+0000 7f04b57fa640 1 -- 172.21.15.100:0/3921289694 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04c0056070 con 0x56064fe16070 2024-07-26T13:42:31.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.041+0000 7f04b57fa640 1 -- 172.21.15.100:0/3921289694 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f04c4041030 con 0x56064fe16070 2024-07-26T13:42:31.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.042+0000 7f04da54b740 10 client.16185 fetch_fsmap finished waiting for FSMap version 27 2024-07-26T13:42:31.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.042+0000 7f04da54b740 10 client.16185 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:42:31.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.042+0000 7f04da54b740 20 client.16185 populate_metadata read hostname 'smithi100' 2024-07-26T13:42:31.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.042+0000 7f04da54b740 1 --2- 172.21.15.100:0/3921289694 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56064fda9da0 0x5606502447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:31.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.042+0000 7f04da54b740 4 client.16185 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-26T13:42:31.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.042+0000 7f04da54b740 1 -- 172.21.15.100:0/3921289694 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56065012f5a0 con 0x56064fda9da0 2024-07-26T13:42:31.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.042+0000 7f04d1d63640 1 --2- 172.21.15.100:0/3921289694 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56064fda9da0 0x5606502447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:31.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.042+0000 7f04b57fa640 10 client.16185 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:31.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.042+0000 7f04d1d63640 1 --2- 172.21.15.100:0/3921289694 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56064fda9da0 0x5606502447e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=842311fe245a8081 server_cookie=c7022f6fa4ececf in_seq=0 out_seq=0 2024-07-26T13:42:31.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.042+0000 7f04b57fa640 1 -- 172.21.15.100:0/3921289694 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56065012f5a0 con 0x56064fda9da0 2024-07-26T13:42:31.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.042+0000 7f04b57fa640 10 client.16185 handle_command_reply: tid=0 2024-07-26T13:42:31.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.042+0000 7f04da54b740 1 client.16185 shutdown 2024-07-26T13:42:31.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.043+0000 7f04da54b740 1 -- 172.21.15.100:0/3921289694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064fe16070 msgr2=0x5606501cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:31.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.043+0000 7f04da54b740 1 --2- 172.21.15.100:0/3921289694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064fe16070 0x5606501cba20 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f04c4018040 tx=0x7f04c401d270 comp rx=0 tx=0).stop 2024-07-26T13:42:31.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.043+0000 7f04da54b740 1 -- 172.21.15.100:0/3921289694 shutdown_connections 2024-07-26T13:42:31.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.043+0000 7f04da54b740 1 --2- 172.21.15.100:0/3921289694 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56064fda9da0 0x5606502447e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:31.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.043+0000 7f04da54b740 1 --2- 172.21.15.100:0/3921289694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56064fe16070 0x5606501cba20 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:31.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.043+0000 7f04b57fa640 0 client.16185 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:31.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.043+0000 7f04da54b740 1 --2- 172.21.15.100:0/3921289694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56064fdf5390 0x5606501cb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:31.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.043+0000 7f04da54b740 1 --2- 172.21.15.100:0/3921289694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56064fded1b0 0x5606501cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:31.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.043+0000 7f04da54b740 1 -- 172.21.15.100:0/3921289694 >> 172.21.15.100:0/3921289694 conn(0x56064fe0a460 msgr2=0x5606501bc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:31.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.043+0000 7f04da54b740 1 -- 172.21.15.100:0/3921289694 shutdown_connections 2024-07-26T13:42:31.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.043+0000 7f04da54b740 1 -- 172.21.15.100:0/3921289694 wait complete. 2024-07-26T13:42:31.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.043+0000 7f04da54b740 20 client.16185 trim_cache size 0 max 16384 2024-07-26T13:42:31.052 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-07-26T13:42:31.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.044+0000 7f04d3fc7640 1 -- 172.21.15.100:0/2747781485 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f04b80b3d40 msgr2=0x7f04b80b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:31.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.044+0000 7f04d3fc7640 1 --2- 172.21.15.100:0/2747781485 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f04b80b3d40 0x7f04b80b6250 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f04c4009fd0 tx=0x7f04c4009500 comp rx=0 tx=0).stop 2024-07-26T13:42:31.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.044+0000 7f04d3fc7640 1 -- 172.21.15.100:0/2747781485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04cc107e00 msgr2=0x7f04cc19a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:31.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.044+0000 7f04d3fc7640 1 --2- 172.21.15.100:0/2747781485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04cc107e00 0x7f04cc19a420 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f04c800e9c0 tx=0x7f04c800ee80 comp rx=0 tx=0).stop 2024-07-26T13:42:31.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.044+0000 7f04d3fc7640 1 -- 172.21.15.100:0/2747781485 shutdown_connections 2024-07-26T13:42:31.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.044+0000 7f04d3fc7640 1 --2- 172.21.15.100:0/2747781485 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f04b80b3d40 0x7f04b80b6250 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:31.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.044+0000 7f04d3fc7640 1 --2- 172.21.15.100:0/2747781485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04cc108750 0x7f04cc19f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:31.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.044+0000 7f04d3fc7640 1 --2- 172.21.15.100:0/2747781485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04cc107e00 0x7f04cc19a420 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:31.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.044+0000 7f04d3fc7640 1 --2- 172.21.15.100:0/2747781485 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04cc102510 0x7f04cc199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:31.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.044+0000 7f04d3fc7640 1 -- 172.21.15.100:0/2747781485 >> 172.21.15.100:0/2747781485 conn(0x7f04cc079190 msgr2=0x7f04cc100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:31.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.044+0000 7f04d3fc7640 1 -- 172.21.15.100:0/2747781485 shutdown_connections 2024-07-26T13:42:31.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:31.044+0000 7f04d3fc7640 1 -- 172.21.15.100:0/2747781485 wait complete. 2024-07-26T13:42:31.405 INFO:tasks.workunit.client.0.smithi100.stdout: 1 29036 3.87 MB/sec warmup 45 sec latency 18.433 ms 2024-07-26T13:42:31.672 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:31 smithi100 ceph-mon[32599]: pgmap v436: 97 pgs: 97 active+clean; 29 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 155 KiB/s wr, 53 op/s 2024-07-26T13:42:31.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:31 smithi167 ceph-mon[37140]: pgmap v436: 97 pgs: 97 active+clean; 29 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 155 KiB/s wr, 53 op/s 2024-07-26T13:42:31.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:31 smithi138 ceph-mon[36954]: pgmap v436: 97 pgs: 97 active+clean; 29 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 155 KiB/s wr, 53 op/s 2024-07-26T13:42:32.405 INFO:tasks.workunit.client.0.smithi100.stdout: 1 32070 4.17 MB/sec warmup 46 sec latency 14.514 ms 2024-07-26T13:42:32.706 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-26T13:42:32.706 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-26T13:42:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:42:33.405 INFO:tasks.workunit.client.0.smithi100.stdout: 1 33656 4.22 MB/sec warmup 47 sec latency 12.209 ms 2024-07-26T13:42:33.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:33 smithi100 ceph-mon[32599]: pgmap v437: 97 pgs: 97 active+clean; 25 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 86 KiB/s wr, 36 op/s 2024-07-26T13:42:33.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:33 smithi138 ceph-mon[36954]: pgmap v437: 97 pgs: 97 active+clean; 25 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 86 KiB/s wr, 36 op/s 2024-07-26T13:42:34.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:33 smithi167 ceph-mon[37140]: pgmap v437: 97 pgs: 97 active+clean; 25 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 86 KiB/s wr, 36 op/s 2024-07-26T13:42:34.405 INFO:tasks.workunit.client.0.smithi100.stdout: 1 35961 4.42 MB/sec warmup 48 sec latency 11.427 ms 2024-07-26T13:42:34.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.894+0000 7f400899b640 1 -- 172.21.15.100:0/3161957101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4004107dd0 msgr2=0x7f40041081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:34.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.894+0000 7f400899b640 1 --2- 172.21.15.100:0/3161957101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4004107dd0 0x7f40041081e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8009900 tx=0x7f3ff802a410 comp rx=0 tx=0).stop 2024-07-26T13:42:34.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.894+0000 7f400899b640 1 -- 172.21.15.100:0/3161957101 shutdown_connections 2024-07-26T13:42:34.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.894+0000 7f400899b640 1 --2- 172.21.15.100:0/3161957101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4004108720 0x7f400410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:34.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.894+0000 7f400899b640 1 --2- 172.21.15.100:0/3161957101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4004107dd0 0x7f40041081e0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:34.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.894+0000 7f400899b640 1 --2- 172.21.15.100:0/3161957101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4004102490 0x7f40041028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:34.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.894+0000 7f400899b640 1 -- 172.21.15.100:0/3161957101 >> 172.21.15.100:0/3161957101 conn(0x7f4004079160 msgr2=0x7f4004079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:34.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.894+0000 7f400899b640 1 -- 172.21.15.100:0/3161957101 shutdown_connections 2024-07-26T13:42:34.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.894+0000 7f400899b640 1 -- 172.21.15.100:0/3161957101 wait complete. 2024-07-26T13:42:34.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.895+0000 7f400899b640 1 Processor -- start 2024-07-26T13:42:34.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.895+0000 7f400899b640 1 -- start start 2024-07-26T13:42:34.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.895+0000 7f400899b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4004102490 0x7f4004198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:34.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.895+0000 7f400899b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4004107dd0 0x7f4004199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:34.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.895+0000 7f400899b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4004108720 0x7f400419e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:34.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.895+0000 7f400899b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4004112370 con 0x7f4004108720 2024-07-26T13:42:34.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.895+0000 7f400899b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f40041121f0 con 0x7f4004102490 2024-07-26T13:42:34.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.895+0000 7f400899b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f40041124f0 con 0x7f4004107dd0 2024-07-26T13:42:34.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.896+0000 7f400259c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4004102490 0x7f4004198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:34.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.896+0000 7f4001d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4004107dd0 0x7f4004199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:34.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.896+0000 7f400259c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4004102490 0x7f4004198d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:46068/0 (socket says 172.21.15.100:46068) 2024-07-26T13:42:34.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.896+0000 7f400259c640 1 -- 172.21.15.100:0/2832440539 learned_addr learned my addr 172.21.15.100:0/2832440539 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:34.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.896+0000 7f4002d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4004108720 0x7f400419e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:34.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.896+0000 7f400259c640 1 -- 172.21.15.100:0/2832440539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4004107dd0 msgr2=0x7f4004199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:34.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.896+0000 7f400259c640 1 --2- 172.21.15.100:0/2832440539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4004107dd0 0x7f4004199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:34.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.896+0000 7f400259c640 1 -- 172.21.15.100:0/2832440539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4004108720 msgr2=0x7f400419e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:34.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.896+0000 7f400259c640 1 --2- 172.21.15.100:0/2832440539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4004108720 0x7f400419e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:34.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.896+0000 7f400259c640 1 -- 172.21.15.100:0/2832440539 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f400419ecf0 con 0x7f4004102490 2024-07-26T13:42:34.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.896+0000 7f4001d9b640 1 --2- 172.21.15.100:0/2832440539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4004107dd0 0x7f4004199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:42:34.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.896+0000 7f4002d9d640 1 --2- 172.21.15.100:0/2832440539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4004108720 0x7f400419e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:42:34.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.896+0000 7f400259c640 1 --2- 172.21.15.100:0/2832440539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4004102490 0x7f4004198d80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400e9f0 tx=0x7f3ff400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:34.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.896+0000 7f3ff37fe640 1 -- 172.21.15.100:0/2832440539 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff400cd80 con 0x7f4004102490 2024-07-26T13:42:34.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.896+0000 7f400899b640 1 -- 172.21.15.100:0/2832440539 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f400419efe0 con 0x7f4004102490 2024-07-26T13:42:34.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.896+0000 7f3ff37fe640 1 -- 172.21.15.100:0/2832440539 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ff40108e0 con 0x7f4004102490 2024-07-26T13:42:34.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.896+0000 7f3ff37fe640 1 -- 172.21.15.100:0/2832440539 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4018980 con 0x7f4004102490 2024-07-26T13:42:34.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.896+0000 7f400899b640 1 -- 172.21.15.100:0/2832440539 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f400406a890 con 0x7f4004102490 2024-07-26T13:42:34.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.899+0000 7f3ff37fe640 1 -- 172.21.15.100:0/2832440539 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f3ff4018b20 con 0x7f4004102490 2024-07-26T13:42:34.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.899+0000 7f400ef1f740 1 Processor -- start 2024-07-26T13:42:34.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.899+0000 7f400ef1f740 1 -- start start 2024-07-26T13:42:34.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.900+0000 7f400ef1f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1856da350 0x55f185a7f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:34.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.900+0000 7f400ef1f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1856b1490 0x55f185a7f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:34.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.900+0000 7f400ef1f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1856b9670 0x55f185a7fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:34.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.900+0000 7f400ef1f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55f1859672f0 con 0x55f1856b1490 2024-07-26T13:42:34.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.900+0000 7f400ef1f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f1859f5a80 con 0x55f1856b9670 2024-07-26T13:42:34.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.900+0000 7f400ef1f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f1859e4f80 con 0x55f1856da350 2024-07-26T13:42:34.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.900+0000 7f4002d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1856da350 0x55f185a7f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:34.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.900+0000 7f4002d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1856da350 0x55f185a7f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:44572/0 (socket says 172.21.15.100:44572) 2024-07-26T13:42:34.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.900+0000 7f4002d9d640 1 -- 172.21.15.100:0/2318034413 learned_addr learned my addr 172.21.15.100:0/2318034413 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:34.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.900+0000 7f4001d9b640 1 --2- 172.21.15.100:0/2318034413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1856b9670 0x55f185a7fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:34.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.900+0000 7f400259c640 1 --2- 172.21.15.100:0/2318034413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1856b1490 0x55f185a7f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:34.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.900+0000 7f3ff37fe640 1 --2- 172.21.15.100:0/2832440539 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3fec0b3d40 0x7f3fec0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:34.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.900+0000 7f4001d9b640 1 -- 172.21.15.100:0/2318034413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1856da350 msgr2=0x55f185a7f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:34.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.900+0000 7f4001d9b640 1 --2- 172.21.15.100:0/2318034413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1856da350 0x55f185a7f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:34.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.900+0000 7f4001d9b640 1 -- 172.21.15.100:0/2318034413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1856b1490 msgr2=0x55f185a7f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:34.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.900+0000 7f4001d9b640 1 --2- 172.21.15.100:0/2318034413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1856b1490 0x55f185a7f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:34.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.900+0000 7f4001d9b640 1 -- 172.21.15.100:0/2318034413 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f185745930 con 0x55f1856b9670 2024-07-26T13:42:34.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.900+0000 7f3ff37fe640 1 -- 172.21.15.100:0/2832440539 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f3ff40e4370 con 0x7f4004102490 2024-07-26T13:42:34.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.900+0000 7f4002d9d640 1 --2- 172.21.15.100:0/2318034413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1856da350 0x55f185a7f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:42:34.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.900+0000 7f4002d9d640 1 --2- 172.21.15.100:0/2832440539 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3fec0b3d40 0x7f3fec0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:34.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.901+0000 7f4001d9b640 1 --2- 172.21.15.100:0/2318034413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1856b9670 0x55f185a7fe30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8006010 tx=0x7f3ff8005e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2975388ff7d4b773 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:34.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.901+0000 7f3ff0ff9640 1 -- 172.21.15.100:0/2318034413 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8040070 con 0x55f1856b9670 2024-07-26T13:42:34.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.901+0000 7f3ff0ff9640 1 -- 172.21.15.100:0/2318034413 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ff8031720 con 0x55f1856b9670 2024-07-26T13:42:34.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.901+0000 7f3ff0ff9640 1 -- 172.21.15.100:0/2318034413 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8035040 con 0x55f1856b9670 2024-07-26T13:42:34.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.902+0000 7f400ef1f740 1 -- 172.21.15.100:0/2318034413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1856b9670 msgr2=0x55f185a7fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:34.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.902+0000 7f4002d9d640 1 --2- 172.21.15.100:0/2832440539 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3fec0b3d40 0x7f3fec0b6250 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc009fd0 tx=0x7f3ffc004a40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:34.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.902+0000 7f400ef1f740 1 --2- 172.21.15.100:0/2318034413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1856b9670 0x55f185a7fe30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8006010 tx=0x7f3ff8005e20 comp rx=0 tx=0).stop 2024-07-26T13:42:34.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.902+0000 7f400ef1f740 1 -- 172.21.15.100:0/2318034413 shutdown_connections 2024-07-26T13:42:34.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.902+0000 7f400ef1f740 1 --2- 172.21.15.100:0/2318034413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1856b9670 0x55f185a7fe30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:34.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.902+0000 7f400ef1f740 1 --2- 172.21.15.100:0/2318034413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1856b1490 0x55f185a7f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:34.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.902+0000 7f400ef1f740 1 --2- 172.21.15.100:0/2318034413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1856da350 0x55f185a7f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:34.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.902+0000 7f400ef1f740 1 -- 172.21.15.100:0/2318034413 >> 172.21.15.100:0/2318034413 conn(0x55f1856ce740 msgr2=0x55f185a7d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:34.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.902+0000 7f400ef1f740 1 -- 172.21.15.100:0/2318034413 shutdown_connections 2024-07-26T13:42:34.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.902+0000 7f400ef1f740 1 -- 172.21.15.100:0/2318034413 wait complete. 2024-07-26T13:42:34.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.902+0000 7f400ef1f740 1 -- start start 2024-07-26T13:42:34.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.902+0000 7f400ef1f740 1 Processor -- start 2024-07-26T13:42:34.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.903+0000 7f400ef1f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1856b1490 0x55f185a8afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:34.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.903+0000 7f400ef1f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1856b9670 0x55f185a8b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:34.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.903+0000 7f400ef1f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1856da350 0x55f185a9b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:34.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.903+0000 7f400ef1f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55f1859874f0 con 0x55f1856b9670 2024-07-26T13:42:34.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.903+0000 7f400ef1f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f1859e8fc0 con 0x55f1856b1490 2024-07-26T13:42:34.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.903+0000 7f400ef1f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f185848890 con 0x55f1856da350 2024-07-26T13:42:34.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.903+0000 7f400259c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1856b1490 0x55f185a8afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:34.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.903+0000 7f400259c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1856b1490 0x55f185a8afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:46088/0 (socket says 172.21.15.100:46088) 2024-07-26T13:42:34.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.903+0000 7f400259c640 1 -- 172.21.15.100:0/439809942 learned_addr learned my addr 172.21.15.100:0/439809942 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:34.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.903+0000 7f4002d9d640 1 --2- 172.21.15.100:0/439809942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1856da350 0x55f185a9b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:34.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.903+0000 7f400ef1f740 10 client.0 _ll_register_callbacks cb 0x55f185991500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:42:34.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.903+0000 7f4001d9b640 1 --2- 172.21.15.100:0/439809942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1856b9670 0x55f185a8b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:34.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.903+0000 7f400259c640 1 -- 172.21.15.100:0/439809942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1856da350 msgr2=0x55f185a9b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:34.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.903+0000 7f400259c640 1 --2- 172.21.15.100:0/439809942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1856da350 0x55f185a9b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:34.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.903+0000 7f400259c640 1 -- 172.21.15.100:0/439809942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1856b9670 msgr2=0x55f185a8b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:34.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.903+0000 7f400259c640 1 --2- 172.21.15.100:0/439809942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1856b9670 0x55f185a8b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:34.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.904+0000 7f400259c640 1 -- 172.21.15.100:0/439809942 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f185996c50 con 0x55f1856b1490 2024-07-26T13:42:34.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.904+0000 7f4001d9b640 1 --2- 172.21.15.100:0/439809942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1856b9670 0x55f185a8b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:42:34.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.904+0000 7f400259c640 1 --2- 172.21.15.100:0/439809942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1856b1490 0x55f185a8afc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3ff40e1b40 tx=0x7f3ff4014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6284df8478cb171a server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:34.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.904+0000 7f3fe9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-07-26T13:42:34.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.904+0000 7f3fe9ffb640 1 -- 172.21.15.100:0/439809942 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff40aa7c0 con 0x55f1856b1490 2024-07-26T13:42:34.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.904+0000 7f400ef1f740 1 -- 172.21.15.100:0/439809942 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ff8031720 con 0x55f1856b1490 2024-07-26T13:42:34.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.904+0000 7f3fe9ffb640 1 -- 172.21.15.100:0/439809942 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f3ff40e7bb0 con 0x55f1856b1490 2024-07-26T13:42:34.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.904+0000 7f3fe9ffb640 1 -- 172.21.15.100:0/439809942 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3ff40aab60 con 0x55f1856b1490 2024-07-26T13:42:34.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.904+0000 7f400ef1f740 10 client.25574 fetch_fsmap learned FSMap version 27 2024-07-26T13:42:34.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.904+0000 7f400ef1f740 1 -- 172.21.15.100:0/439809942 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f185a9de80 con 0x55f1856b1490 2024-07-26T13:42:34.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.905+0000 7f3fe9ffb640 1 -- 172.21.15.100:0/439809942 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ff8031720 con 0x55f1856b1490 2024-07-26T13:42:34.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.905+0000 7f3fe9ffb640 1 -- 172.21.15.100:0/439809942 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3ff40aa2b0 con 0x55f1856b1490 2024-07-26T13:42:34.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.905+0000 7f400ef1f740 10 client.25574 fetch_fsmap finished waiting for FSMap version 27 2024-07-26T13:42:34.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.905+0000 7f400ef1f740 10 client.25574 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:42:34.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.905+0000 7f400ef1f740 20 client.25574 populate_metadata read hostname 'smithi100' 2024-07-26T13:42:34.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.905+0000 7f400ef1f740 1 --2- 172.21.15.100:0/439809942 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f185698cd0 0x55f185aa13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:34.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.905+0000 7f400ef1f740 4 client.25574 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:42:34.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.905+0000 7f400ef1f740 1 -- 172.21.15.100:0/439809942 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1859f79d0 con 0x55f185698cd0 2024-07-26T13:42:34.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.905+0000 7f4001d9b640 1 --2- 172.21.15.100:0/439809942 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f185698cd0 0x55f185aa13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:34.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.905+0000 7f3fe9ffb640 10 client.25574 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:34.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.905+0000 7f4001d9b640 1 --2- 172.21.15.100:0/439809942 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f185698cd0 0x55f185aa13d0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c31073d0ce088458 server_cookie=7ade611f4ffcf7e2 in_seq=0 out_seq=0 2024-07-26T13:42:34.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.906+0000 7f3fe9ffb640 1 -- 172.21.15.100:0/439809942 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f1859f79d0 con 0x55f185698cd0 2024-07-26T13:42:34.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.906+0000 7f3fe9ffb640 10 client.25574 handle_command_reply: tid=0 2024-07-26T13:42:34.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.906+0000 7f400ef1f740 1 client.25574 shutdown 2024-07-26T13:42:34.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.906+0000 7f400ef1f740 1 -- 172.21.15.100:0/439809942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1856b1490 msgr2=0x55f185a8afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:34.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.906+0000 7f400ef1f740 1 --2- 172.21.15.100:0/439809942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1856b1490 0x55f185a8afc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3ff40e1b40 tx=0x7f3ff4014000 comp rx=0 tx=0).stop 2024-07-26T13:42:34.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.906+0000 7f400ef1f740 1 -- 172.21.15.100:0/439809942 shutdown_connections 2024-07-26T13:42:34.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.906+0000 7f400ef1f740 1 --2- 172.21.15.100:0/439809942 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f185698cd0 0x55f185aa13d0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:34.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.906+0000 7f400ef1f740 1 --2- 172.21.15.100:0/439809942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1856da350 0x55f185a9b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:34.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.906+0000 7f3fe9ffb640 0 client.25574 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:34.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.906+0000 7f400ef1f740 1 --2- 172.21.15.100:0/439809942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1856b9670 0x55f185a8b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:34.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.906+0000 7f400ef1f740 1 --2- 172.21.15.100:0/439809942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1856b1490 0x55f185a8afc0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:34.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.906+0000 7f400ef1f740 1 -- 172.21.15.100:0/439809942 >> 172.21.15.100:0/439809942 conn(0x55f1856ce740 msgr2=0x55f185a82590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:34.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.906+0000 7f400ef1f740 1 -- 172.21.15.100:0/439809942 shutdown_connections 2024-07-26T13:42:34.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.907+0000 7f400ef1f740 1 -- 172.21.15.100:0/439809942 wait complete. 2024-07-26T13:42:34.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.907+0000 7f400ef1f740 20 client.25574 trim_cache size 0 max 16384 2024-07-26T13:42:34.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.924+0000 7f400ef1f740 1 Processor -- start 2024-07-26T13:42:34.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.924+0000 7f400ef1f740 1 -- start start 2024-07-26T13:42:34.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.924+0000 7f400ef1f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1856b1490 0x55f185a7d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:34.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.924+0000 7f400ef1f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1856b9670 0x55f185a9aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:34.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.924+0000 7f400ef1f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1856da350 0x55f185aa6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:34.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.924+0000 7f400ef1f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f40041af7b0 con 0x55f1856da350 2024-07-26T13:42:34.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.924+0000 7f400ef1f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f40041ad080 con 0x55f1856b9670 2024-07-26T13:42:34.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.924+0000 7f400ef1f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f40041aaed0 con 0x55f1856b1490 2024-07-26T13:42:34.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.925+0000 7f400259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1856b1490 0x55f185a7d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:34.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.925+0000 7f4002d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1856da350 0x55f185aa6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:34.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.925+0000 7f4001d9b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1856b9670 0x55f185a9aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:34.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.925+0000 7f400259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1856b1490 0x55f185a7d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:44592/0 (socket says 172.21.15.100:44592) 2024-07-26T13:42:34.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.925+0000 7f400259c640 1 -- 172.21.15.100:0/898245664 learned_addr learned my addr 172.21.15.100:0/898245664 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:34.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.925+0000 7f400259c640 1 -- 172.21.15.100:0/898245664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1856b9670 msgr2=0x55f185a9aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:34.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.925+0000 7f400259c640 1 --2- 172.21.15.100:0/898245664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1856b9670 0x55f185a9aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:34.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.925+0000 7f400259c640 1 -- 172.21.15.100:0/898245664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1856da350 msgr2=0x55f185aa6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:34.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.925+0000 7f400259c640 1 --2- 172.21.15.100:0/898245664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1856da350 0x55f185aa6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:34.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.925+0000 7f400259c640 1 -- 172.21.15.100:0/898245664 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f185a86650 con 0x55f1856b1490 2024-07-26T13:42:34.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.925+0000 7f4001d9b640 1 --2- 172.21.15.100:0/898245664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1856b9670 0x55f185a9aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:42:34.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.925+0000 7f400259c640 1 --2- 172.21.15.100:0/898245664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1856b1490 0x55f185a7d220 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3ff40aad50 tx=0x7f3ff401f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4e3eef6ed2e3f23 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:34.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.925+0000 7f3febfff640 1 -- 172.21.15.100:0/898245664 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff40af070 con 0x55f1856b1490 2024-07-26T13:42:34.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.925+0000 7f3febfff640 1 -- 172.21.15.100:0/898245664 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ff40af5a0 con 0x55f1856b1490 2024-07-26T13:42:34.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.925+0000 7f3febfff640 1 -- 172.21.15.100:0/898245664 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff40aa3f0 con 0x55f1856b1490 2024-07-26T13:42:34.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.926+0000 7f400ef1f740 1 -- 172.21.15.100:0/898245664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1856b1490 msgr2=0x55f185a7d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:34.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.926+0000 7f400ef1f740 1 --2- 172.21.15.100:0/898245664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1856b1490 0x55f185a7d220 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3ff40aad50 tx=0x7f3ff401f000 comp rx=0 tx=0).stop 2024-07-26T13:42:34.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.926+0000 7f400ef1f740 1 -- 172.21.15.100:0/898245664 shutdown_connections 2024-07-26T13:42:34.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.926+0000 7f400ef1f740 1 --2- 172.21.15.100:0/898245664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1856da350 0x55f185aa6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:34.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.926+0000 7f400ef1f740 1 --2- 172.21.15.100:0/898245664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1856b9670 0x55f185a9aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:34.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.926+0000 7f400ef1f740 1 --2- 172.21.15.100:0/898245664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1856b1490 0x55f185a7d220 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:34.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.926+0000 7f400ef1f740 1 -- 172.21.15.100:0/898245664 >> 172.21.15.100:0/898245664 conn(0x55f1856ce740 msgr2=0x55f185a7cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:34.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.926+0000 7f400ef1f740 1 -- 172.21.15.100:0/898245664 shutdown_connections 2024-07-26T13:42:34.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.926+0000 7f400ef1f740 1 -- 172.21.15.100:0/898245664 wait complete. 2024-07-26T13:42:34.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.926+0000 7f400ef1f740 1 -- start start 2024-07-26T13:42:34.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.927+0000 7f400ef1f740 1 Processor -- start 2024-07-26T13:42:34.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.927+0000 7f400ef1f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1856b1490 0x55f185a93170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:34.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.927+0000 7f400ef1f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1856b9670 0x55f185a936b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:34.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.927+0000 7f400ef1f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1856da350 0x55f185a93c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:34.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.927+0000 7f400ef1f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f40041ab540 con 0x55f1856b9670 2024-07-26T13:42:34.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.927+0000 7f400ef1f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f40041ac9b0 con 0x55f1856da350 2024-07-26T13:42:34.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.927+0000 7f400ef1f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f40041abc10 con 0x55f1856b1490 2024-07-26T13:42:34.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.927+0000 7f400ef1f740 10 client.0 _ll_register_callbacks cb 0x7f3ff40e79d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:42:34.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.927+0000 7f400259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1856b1490 0x55f185a93170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:34.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.927+0000 7f4001d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1856b9670 0x55f185a936b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:34.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.927+0000 7f4001d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1856b9670 0x55f185a936b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34044/0 (socket says 172.21.15.100:34044) 2024-07-26T13:42:34.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.927+0000 7f400259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1856b1490 0x55f185a93170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:44604/0 (socket says 172.21.15.100:44604) 2024-07-26T13:42:34.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.927+0000 7f4001d9b640 1 -- 172.21.15.100:0/1035434248 learned_addr learned my addr 172.21.15.100:0/1035434248 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:34.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.927+0000 7f4002d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1856da350 0x55f185a93c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:34.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.927+0000 7f4001d9b640 1 -- 172.21.15.100:0/1035434248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1856b1490 msgr2=0x55f185a93170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:34.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.927+0000 7f4001d9b640 1 --2- 172.21.15.100:0/1035434248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1856b1490 0x55f185a93170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:34.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.927+0000 7f4001d9b640 1 -- 172.21.15.100:0/1035434248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1856da350 msgr2=0x55f185a93c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:34.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.927+0000 7f4001d9b640 1 --2- 172.21.15.100:0/1035434248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1856da350 0x55f185a93c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:34.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.927+0000 7f4001d9b640 1 -- 172.21.15.100:0/1035434248 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f185b08db0 con 0x55f1856b9670 2024-07-26T13:42:34.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.927+0000 7f4002d9d640 1 --2- 172.21.15.100:0/1035434248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1856da350 0x55f185a93c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:42:34.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.928+0000 7f4001d9b640 1 --2- 172.21.15.100:0/1035434248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1856b9670 0x55f185a936b0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8005510 tx=0x7f3ff8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7333467809e38519 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:34.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.928+0000 7f3fe9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:42:34.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.928+0000 7f3fe9ffb640 1 -- 172.21.15.100:0/1035434248 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff805a020 con 0x55f1856b9670 2024-07-26T13:42:34.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.928+0000 7f3fe9ffb640 1 -- 172.21.15.100:0/1035434248 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f3ff8050860 con 0x55f1856b9670 2024-07-26T13:42:34.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.928+0000 7f400ef1f740 1 -- 172.21.15.100:0/1035434248 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ff40af5a0 con 0x55f1856b9670 2024-07-26T13:42:34.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.928+0000 7f3fe9ffb640 1 -- 172.21.15.100:0/1035434248 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3ff8065050 con 0x55f1856b9670 2024-07-26T13:42:34.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.928+0000 7f400ef1f740 10 client.16203 fetch_fsmap learned FSMap version 27 2024-07-26T13:42:34.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.928+0000 7f400ef1f740 1 -- 172.21.15.100:0/1035434248 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f185b0a070 con 0x55f1856b9670 2024-07-26T13:42:34.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.928+0000 7f3fe9ffb640 1 -- 172.21.15.100:0/1035434248 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ff40af5a0 con 0x55f1856b9670 2024-07-26T13:42:34.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.928+0000 7f3fe9ffb640 1 -- 172.21.15.100:0/1035434248 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3ff803c850 con 0x55f1856b9670 2024-07-26T13:42:34.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.928+0000 7f400ef1f740 10 client.16203 fetch_fsmap finished waiting for FSMap version 27 2024-07-26T13:42:34.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.928+0000 7f400ef1f740 10 client.16203 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:42:34.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.928+0000 7f400ef1f740 20 client.16203 populate_metadata read hostname 'smithi100' 2024-07-26T13:42:34.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.928+0000 7f400ef1f740 1 --2- 172.21.15.100:0/1035434248 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f18566e080 0x55f185b0c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:34.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.929+0000 7f400ef1f740 4 client.16203 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-26T13:42:34.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.929+0000 7f400ef1f740 1 -- 172.21.15.100:0/1035434248 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f1859f3470 con 0x55f18566e080 2024-07-26T13:42:34.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.929+0000 7f400259c640 1 --2- 172.21.15.100:0/1035434248 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f18566e080 0x55f185b0c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:34.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.929+0000 7f400259c640 1 --2- 172.21.15.100:0/1035434248 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f18566e080 0x55f185b0c440 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90e1c27f67cbd783 server_cookie=dbe8467f6d3b116a in_seq=0 out_seq=0 2024-07-26T13:42:34.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.929+0000 7f3fe9ffb640 10 client.16203 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:34.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.930+0000 7f3fe9ffb640 1 -- 172.21.15.100:0/1035434248 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f1859f3470 con 0x55f18566e080 2024-07-26T13:42:34.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.930+0000 7f3fe9ffb640 10 client.16203 handle_command_reply: tid=0 2024-07-26T13:42:34.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.930+0000 7f400ef1f740 1 client.16203 shutdown 2024-07-26T13:42:34.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.930+0000 7f400ef1f740 1 -- 172.21.15.100:0/1035434248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1856b9670 msgr2=0x55f185a936b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:34.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.930+0000 7f400ef1f740 1 --2- 172.21.15.100:0/1035434248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1856b9670 0x55f185a936b0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8005510 tx=0x7f3ff8009260 comp rx=0 tx=0).stop 2024-07-26T13:42:34.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.930+0000 7f400ef1f740 1 -- 172.21.15.100:0/1035434248 shutdown_connections 2024-07-26T13:42:34.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.930+0000 7f400ef1f740 1 --2- 172.21.15.100:0/1035434248 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f18566e080 0x55f185b0c440 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:34.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.930+0000 7f400ef1f740 1 --2- 172.21.15.100:0/1035434248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1856da350 0x55f185a93c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:34.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.930+0000 7f400ef1f740 1 --2- 172.21.15.100:0/1035434248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f1856b9670 0x55f185a936b0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:34.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.930+0000 7f400ef1f740 1 --2- 172.21.15.100:0/1035434248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f1856b1490 0x55f185a93170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:34.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.930+0000 7f3fe9ffb640 0 client.16203 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:34.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.930+0000 7f400ef1f740 1 -- 172.21.15.100:0/1035434248 >> 172.21.15.100:0/1035434248 conn(0x55f1856ce740 msgr2=0x55f185a8c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:34.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.930+0000 7f400ef1f740 1 -- 172.21.15.100:0/1035434248 shutdown_connections 2024-07-26T13:42:34.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.930+0000 7f400ef1f740 1 -- 172.21.15.100:0/1035434248 wait complete. 2024-07-26T13:42:34.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.930+0000 7f400ef1f740 20 client.16203 trim_cache size 0 max 16384 2024-07-26T13:42:34.936 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:42:34.936 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-07-26T13:42:34.936 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "a536d90a-b7ef-44dd-be34-4c34840ab2f5", 2024-07-26T13:42:34.936 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-07-26T13:42:34.936 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:42:34.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.931+0000 7f400899b640 1 -- 172.21.15.100:0/2832440539 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3fec0b3d40 msgr2=0x7f3fec0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:34.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.931+0000 7f400899b640 1 --2- 172.21.15.100:0/2832440539 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3fec0b3d40 0x7f3fec0b6250 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc009fd0 tx=0x7f3ffc004a40 comp rx=0 tx=0).stop 2024-07-26T13:42:34.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.931+0000 7f400899b640 1 -- 172.21.15.100:0/2832440539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4004102490 msgr2=0x7f4004198d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:34.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.931+0000 7f400899b640 1 --2- 172.21.15.100:0/2832440539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4004102490 0x7f4004198d80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400e9f0 tx=0x7f3ff400eeb0 comp rx=0 tx=0).stop 2024-07-26T13:42:34.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.931+0000 7f400899b640 1 -- 172.21.15.100:0/2832440539 shutdown_connections 2024-07-26T13:42:34.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.931+0000 7f400899b640 1 --2- 172.21.15.100:0/2832440539 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3fec0b3d40 0x7f3fec0b6250 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:34.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.931+0000 7f400899b640 1 --2- 172.21.15.100:0/2832440539 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4004108720 0x7f400419e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:34.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.931+0000 7f400899b640 1 --2- 172.21.15.100:0/2832440539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4004107dd0 0x7f4004199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:34.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.931+0000 7f400899b640 1 --2- 172.21.15.100:0/2832440539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4004102490 0x7f4004198d80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:34.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.931+0000 7f400899b640 1 -- 172.21.15.100:0/2832440539 >> 172.21.15.100:0/2832440539 conn(0x7f4004079160 msgr2=0x7f4004104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:34.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.931+0000 7f400899b640 1 -- 172.21.15.100:0/2832440539 shutdown_connections 2024-07-26T13:42:34.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:34.931+0000 7f400899b640 1 -- 172.21.15.100:0/2832440539 wait complete. 2024-07-26T13:42:35.405 INFO:tasks.workunit.client.0.smithi100.stdout: 1 38494 4.59 MB/sec warmup 49 sec latency 34.147 ms 2024-07-26T13:42:35.774 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-26T13:42:35.798 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:35 smithi100 ceph-mon[32599]: pgmap v438: 97 pgs: 97 active+clean; 28 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 434 KiB/s wr, 102 op/s 2024-07-26T13:42:35.798 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:35 smithi100 ceph-mon[32599]: scrub summary: active paths [/] 2024-07-26T13:42:35.798 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:35 smithi100 ceph-mon[32599]: scrub summary: active paths [/] 2024-07-26T13:42:35.798 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:35 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:42:35.798 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:35 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:42:35.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:35 smithi138 ceph-mon[36954]: pgmap v438: 97 pgs: 97 active+clean; 28 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 434 KiB/s wr, 102 op/s 2024-07-26T13:42:35.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:35 smithi138 ceph-mon[36954]: scrub summary: active paths [/] 2024-07-26T13:42:35.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:35 smithi138 ceph-mon[36954]: scrub summary: active paths [/] 2024-07-26T13:42:35.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:35 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:42:35.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:35 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:42:36.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:35 smithi167 ceph-mon[37140]: pgmap v438: 97 pgs: 97 active+clean; 28 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 434 KiB/s wr, 102 op/s 2024-07-26T13:42:36.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:35 smithi167 ceph-mon[37140]: scrub summary: active paths [/] 2024-07-26T13:42:36.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:35 smithi167 ceph-mon[37140]: scrub summary: active paths [/] 2024-07-26T13:42:36.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:35 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:42:36.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:35 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:42:36.040 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:42:36.405 INFO:tasks.workunit.client.0.smithi100.stdout: 1 40191 4.67 MB/sec warmup 50 sec latency 13.802 ms 2024-07-26T13:42:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:36 smithi100 ceph-mon[32599]: scrub queued for path: / 2024-07-26T13:42:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:36 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:42:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:36 smithi100 ceph-mon[32599]: scrub summary: active paths [/] 2024-07-26T13:42:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:36 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:42:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:36 smithi138 ceph-mon[36954]: scrub queued for path: / 2024-07-26T13:42:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:36 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:42:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:36 smithi138 ceph-mon[36954]: scrub summary: active paths [/] 2024-07-26T13:42:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:36 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:42:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:36 smithi167 ceph-mon[37140]: scrub queued for path: / 2024-07-26T13:42:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:36 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:42:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:36 smithi167 ceph-mon[37140]: scrub summary: active paths [/] 2024-07-26T13:42:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:36 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:42:37.405 INFO:tasks.workunit.client.0.smithi100.stdout: 1 42882 4.90 MB/sec warmup 51 sec latency 13.253 ms 2024-07-26T13:42:37.718 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:37 smithi100 ceph-mon[32599]: pgmap v439: 97 pgs: 97 active+clean; 29 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 528 KiB/s wr, 109 op/s 2024-07-26T13:42:37.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.812+0000 7f1b44057640 1 -- 172.21.15.100:0/801818653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b3c107dd0 msgr2=0x7f1b3c1081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:37.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.812+0000 7f1b44057640 1 --2- 172.21.15.100:0/801818653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b3c107dd0 0x7f1b3c1081e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c009900 tx=0x7f1b2c02a410 comp rx=0 tx=0).stop 2024-07-26T13:42:37.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.812+0000 7f1b44057640 1 -- 172.21.15.100:0/801818653 shutdown_connections 2024-07-26T13:42:37.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.812+0000 7f1b44057640 1 --2- 172.21.15.100:0/801818653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b3c108720 0x7f1b3c10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:37.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.812+0000 7f1b44057640 1 --2- 172.21.15.100:0/801818653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b3c107dd0 0x7f1b3c1081e0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:37.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.812+0000 7f1b44057640 1 --2- 172.21.15.100:0/801818653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b3c102490 0x7f1b3c1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:37.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.812+0000 7f1b44057640 1 -- 172.21.15.100:0/801818653 >> 172.21.15.100:0/801818653 conn(0x7f1b3c079160 msgr2=0x7f1b3c079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:37.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.813+0000 7f1b44057640 1 -- 172.21.15.100:0/801818653 shutdown_connections 2024-07-26T13:42:37.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.813+0000 7f1b44057640 1 -- 172.21.15.100:0/801818653 wait complete. 2024-07-26T13:42:37.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.813+0000 7f1b44057640 1 Processor -- start 2024-07-26T13:42:37.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.813+0000 7f1b44057640 1 -- start start 2024-07-26T13:42:37.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.814+0000 7f1b44057640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b3c102490 0x7f1b3c198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:37.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.814+0000 7f1b44057640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b3c107dd0 0x7f1b3c199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:37.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.814+0000 7f1b44057640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b3c108720 0x7f1b3c199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:37.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.814+0000 7f1b44057640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c112350 con 0x7f1b3c102490 2024-07-26T13:42:37.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.814+0000 7f1b44057640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c1121d0 con 0x7f1b3c107dd0 2024-07-26T13:42:37.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.814+0000 7f1b44057640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c1124d0 con 0x7f1b3c108720 2024-07-26T13:42:37.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.814+0000 7f1b41df3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b3c102490 0x7f1b3c198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:37.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.814+0000 7f1b41df3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b3c102490 0x7f1b3c198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38178/0 (socket says 172.21.15.100:38178) 2024-07-26T13:42:37.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.814+0000 7f1b415f2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b3c107dd0 0x7f1b3c199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:37.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.814+0000 7f1b41df3640 1 -- 172.21.15.100:0/232831498 learned_addr learned my addr 172.21.15.100:0/232831498 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:37.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.814+0000 7f1b425f4640 1 --2- 172.21.15.100:0/232831498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b3c108720 0x7f1b3c199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:37.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.814+0000 7f1b41df3640 1 -- 172.21.15.100:0/232831498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b3c108720 msgr2=0x7f1b3c199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:37.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.814+0000 7f1b41df3640 1 --2- 172.21.15.100:0/232831498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b3c108720 0x7f1b3c199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:37.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.814+0000 7f1b41df3640 1 -- 172.21.15.100:0/232831498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b3c107dd0 msgr2=0x7f1b3c199010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:37.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.814+0000 7f1b41df3640 1 --2- 172.21.15.100:0/232831498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b3c107dd0 0x7f1b3c199010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:37.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.814+0000 7f1b41df3640 1 -- 172.21.15.100:0/232831498 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b3c1a81d0 con 0x7f1b3c102490 2024-07-26T13:42:37.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.814+0000 7f1b415f2640 1 --2- 172.21.15.100:0/232831498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b3c107dd0 0x7f1b3c199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:42:37.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.815+0000 7f1b41df3640 1 --2- 172.21.15.100:0/232831498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b3c102490 0x7f1b3c198ad0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800d980 tx=0x7f1b3800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:37.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.815+0000 7f1b32ffd640 1 -- 172.21.15.100:0/232831498 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b38017070 con 0x7f1b3c102490 2024-07-26T13:42:37.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.815+0000 7f1b44057640 1 -- 172.21.15.100:0/232831498 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b3c1a84c0 con 0x7f1b3c102490 2024-07-26T13:42:37.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.815+0000 7f1b32ffd640 1 -- 172.21.15.100:0/232831498 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b38009c30 con 0x7f1b3c102490 2024-07-26T13:42:37.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.815+0000 7f1b32ffd640 1 -- 172.21.15.100:0/232831498 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3800f620 con 0x7f1b3c102490 2024-07-26T13:42:37.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.815+0000 7f1b44057640 1 -- 172.21.15.100:0/232831498 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b3c1a8a20 con 0x7f1b3c102490 2024-07-26T13:42:37.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.817+0000 7f1b4a5db740 1 Processor -- start 2024-07-26T13:42:37.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.818+0000 7f1b32ffd640 1 -- 172.21.15.100:0/232831498 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f1b3800f7c0 con 0x7f1b3c102490 2024-07-26T13:42:37.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.818+0000 7f1b4a5db740 1 -- start start 2024-07-26T13:42:37.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.818+0000 7f1b4a5db740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5557e096d070 0x5557e0d12670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:37.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.818+0000 7f1b4a5db740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557e09441b0 0x5557e0d12bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:37.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.818+0000 7f1b4a5db740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557e094c390 0x5557e0d130f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:37.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.818+0000 7f1b4a5db740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5557e0c88b90 con 0x5557e096d070 2024-07-26T13:42:37.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.818+0000 7f1b4a5db740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5557e0c78090 con 0x5557e09441b0 2024-07-26T13:42:37.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.818+0000 7f1b4a5db740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5557e0c7c0d0 con 0x5557e094c390 2024-07-26T13:42:37.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.818+0000 7f1b425f4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5557e096d070 0x5557e0d12670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:37.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.818+0000 7f1b425f4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5557e096d070 0x5557e0d12670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38194/0 (socket says 172.21.15.100:38194) 2024-07-26T13:42:37.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.818+0000 7f1b425f4640 1 -- 172.21.15.100:0/4208424341 learned_addr learned my addr 172.21.15.100:0/4208424341 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:37.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.818+0000 7f1b415f2640 1 --2- 172.21.15.100:0/4208424341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557e094c390 0x5557e0d130f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:37.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.818+0000 7f1b41df3640 1 --2- 172.21.15.100:0/4208424341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557e09441b0 0x5557e0d12bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:37.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.818+0000 7f1b415f2640 1 -- 172.21.15.100:0/4208424341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557e09441b0 msgr2=0x5557e0d12bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:37.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.818+0000 7f1b415f2640 1 --2- 172.21.15.100:0/4208424341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557e09441b0 0x5557e0d12bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:37.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.818+0000 7f1b415f2640 1 -- 172.21.15.100:0/4208424341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5557e096d070 msgr2=0x5557e0d12670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:37.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.818+0000 7f1b415f2640 1 --2- 172.21.15.100:0/4208424341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5557e096d070 0x5557e0d12670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:37.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.818+0000 7f1b415f2640 1 -- 172.21.15.100:0/4208424341 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557e0c1eb70 con 0x5557e094c390 2024-07-26T13:42:37.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.819+0000 7f1b41df3640 1 --2- 172.21.15.100:0/4208424341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557e09441b0 0x5557e0d12bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:42:37.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.819+0000 7f1b425f4640 1 --2- 172.21.15.100:0/4208424341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5557e096d070 0x5557e0d12670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:42:37.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.819+0000 7f1b32ffd640 1 --2- 172.21.15.100:0/232831498 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1b280b3dc0 0x7f1b280b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:37.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.819+0000 7f1b415f2640 1 --2- 172.21.15.100:0/4208424341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557e094c390 0x5557e0d130f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c006010 tx=0x7f1b2c005ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7aad989ccb483a77 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:37.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.819+0000 7f1b425f4640 1 --2- 172.21.15.100:0/232831498 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1b280b3dc0 0x7f1b280b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:37.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.819+0000 7f1b32ffd640 1 -- 172.21.15.100:0/232831498 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f1b380e3b70 con 0x7f1b3c102490 2024-07-26T13:42:37.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.819+0000 7f1b27fff640 1 -- 172.21.15.100:0/4208424341 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c040070 con 0x5557e094c390 2024-07-26T13:42:37.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.819+0000 7f1b27fff640 1 -- 172.21.15.100:0/4208424341 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b2c031820 con 0x5557e094c390 2024-07-26T13:42:37.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.819+0000 7f1b27fff640 1 -- 172.21.15.100:0/4208424341 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c035040 con 0x5557e094c390 2024-07-26T13:42:37.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.820+0000 7f1b4a5db740 1 -- 172.21.15.100:0/4208424341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557e094c390 msgr2=0x5557e0d130f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:37.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.820+0000 7f1b425f4640 1 --2- 172.21.15.100:0/232831498 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1b280b3dc0 0x7f1b280b62d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1b34009fd0 tx=0x7f1b34009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:37.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.820+0000 7f1b4a5db740 1 --2- 172.21.15.100:0/4208424341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557e094c390 0x5557e0d130f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c006010 tx=0x7f1b2c005ed0 comp rx=0 tx=0).stop 2024-07-26T13:42:37.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.820+0000 7f1b4a5db740 1 -- 172.21.15.100:0/4208424341 shutdown_connections 2024-07-26T13:42:37.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.820+0000 7f1b4a5db740 1 --2- 172.21.15.100:0/4208424341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557e094c390 0x5557e0d130f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:37.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.820+0000 7f1b4a5db740 1 --2- 172.21.15.100:0/4208424341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557e09441b0 0x5557e0d12bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:37.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.820+0000 7f1b4a5db740 1 --2- 172.21.15.100:0/4208424341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5557e096d070 0x5557e0d12670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:37.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.820+0000 7f1b4a5db740 1 -- 172.21.15.100:0/4208424341 >> 172.21.15.100:0/4208424341 conn(0x5557e0961460 msgr2=0x5557e0d10560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:37.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.820+0000 7f1b4a5db740 1 -- 172.21.15.100:0/4208424341 shutdown_connections 2024-07-26T13:42:37.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.820+0000 7f1b4a5db740 1 -- 172.21.15.100:0/4208424341 wait complete. 2024-07-26T13:42:37.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.821+0000 7f1b4a5db740 1 -- start start 2024-07-26T13:42:37.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.821+0000 7f1b4a5db740 1 Processor -- start 2024-07-26T13:42:37.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.821+0000 7f1b4a5db740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557e09441b0 0x5557e0d1e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:37.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.821+0000 7f1b4a5db740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5557e094c390 0x5557e0d1e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:37.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.821+0000 7f1b4a5db740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557e096d070 0x5557e0d2a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:37.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.821+0000 7f1b4a5db740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5557e0adb700 con 0x5557e094c390 2024-07-26T13:42:37.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.821+0000 7f1b4a5db740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5557e0c1a9c0 con 0x5557e09441b0 2024-07-26T13:42:37.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.821+0000 7f1b4a5db740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5557e0d20df0 con 0x5557e096d070 2024-07-26T13:42:37.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.821+0000 7f1b4a5db740 10 client.0 _ll_register_callbacks cb 0x5557e0c354d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:42:37.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.821+0000 7f1b415f2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5557e094c390 0x5557e0d1e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:37.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.821+0000 7f1b415f2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5557e094c390 0x5557e0d1e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38206/0 (socket says 172.21.15.100:38206) 2024-07-26T13:42:37.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.821+0000 7f1b415f2640 1 -- 172.21.15.100:0/150072195 learned_addr learned my addr 172.21.15.100:0/150072195 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:37.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.821+0000 7f1b425f4640 1 --2- 172.21.15.100:0/150072195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557e096d070 0x5557e0d2a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:37.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.822+0000 7f1b41df3640 1 --2- 172.21.15.100:0/150072195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557e09441b0 0x5557e0d1e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:37.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.822+0000 7f1b415f2640 1 -- 172.21.15.100:0/150072195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557e096d070 msgr2=0x5557e0d2a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:37.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.822+0000 7f1b415f2640 1 --2- 172.21.15.100:0/150072195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557e096d070 0x5557e0d2a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:37.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.822+0000 7f1b415f2640 1 -- 172.21.15.100:0/150072195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557e09441b0 msgr2=0x5557e0d1e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:37.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.822+0000 7f1b415f2640 1 --2- 172.21.15.100:0/150072195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557e09441b0 0x5557e0d1e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:37.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.822+0000 7f1b415f2640 1 -- 172.21.15.100:0/150072195 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557e09d8650 con 0x5557e094c390 2024-07-26T13:42:37.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.822+0000 7f1b41df3640 1 --2- 172.21.15.100:0/150072195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557e09441b0 0x5557e0d1e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:42:37.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.822+0000 7f1b425f4640 1 --2- 172.21.15.100:0/150072195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557e096d070 0x5557e0d2a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:42:37.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.822+0000 7f1b415f2640 1 --2- 172.21.15.100:0/150072195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5557e094c390 0x5557e0d1e880 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c0042f0 tx=0x7f1b2c0319d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190b072fe1e1d80 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:37.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.822+0000 7f1b257fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:42:37.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.822+0000 7f1b257fa640 1 -- 172.21.15.100:0/150072195 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c035070 con 0x5557e094c390 2024-07-26T13:42:37.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.822+0000 7f1b257fa640 1 -- 172.21.15.100:0/150072195 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f1b2c003800 con 0x5557e094c390 2024-07-26T13:42:37.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.822+0000 7f1b4a5db740 1 -- 172.21.15.100:0/150072195 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b2c031820 con 0x5557e094c390 2024-07-26T13:42:37.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.822+0000 7f1b257fa640 1 -- 172.21.15.100:0/150072195 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1b2c055050 con 0x5557e094c390 2024-07-26T13:42:37.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.822+0000 7f1b4a5db740 10 client.16215 fetch_fsmap learned FSMap version 27 2024-07-26T13:42:37.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.822+0000 7f1b4a5db740 1 -- 172.21.15.100:0/150072195 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5557e0d2ce90 con 0x5557e094c390 2024-07-26T13:42:37.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.823+0000 7f1b257fa640 1 -- 172.21.15.100:0/150072195 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b2c031820 con 0x5557e094c390 2024-07-26T13:42:37.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.823+0000 7f1b257fa640 1 -- 172.21.15.100:0/150072195 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1b2c03c840 con 0x5557e094c390 2024-07-26T13:42:37.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.823+0000 7f1b4a5db740 10 client.16215 fetch_fsmap finished waiting for FSMap version 27 2024-07-26T13:42:37.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.823+0000 7f1b4a5db740 10 client.16215 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:42:37.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.823+0000 7f1b4a5db740 20 client.16215 populate_metadata read hostname 'smithi100' 2024-07-26T13:42:37.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.823+0000 7f1b4a5db740 1 --2- 172.21.15.100:0/150072195 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5557e092b9f0 0x5557e0d301f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:37.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.823+0000 7f1b4a5db740 4 client.16215 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:42:37.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.823+0000 7f1b4a5db740 1 -- 172.21.15.100:0/150072195 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557e0c8aae0 con 0x5557e092b9f0 2024-07-26T13:42:37.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.823+0000 7f1b41df3640 1 --2- 172.21.15.100:0/150072195 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5557e092b9f0 0x5557e0d301f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:37.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.824+0000 7f1b257fa640 10 client.16215 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:37.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.824+0000 7f1b41df3640 1 --2- 172.21.15.100:0/150072195 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5557e092b9f0 0x5557e0d301f0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71582c8be24a40fa server_cookie=e907ee8e6b61a1dc in_seq=0 out_seq=0 2024-07-26T13:42:37.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.824+0000 7f1b257fa640 1 -- 172.21.15.100:0/150072195 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5557e0c8aae0 con 0x5557e092b9f0 2024-07-26T13:42:37.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.824+0000 7f1b257fa640 10 client.16215 handle_command_reply: tid=0 2024-07-26T13:42:37.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.824+0000 7f1b4a5db740 1 client.16215 shutdown 2024-07-26T13:42:37.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.824+0000 7f1b4a5db740 1 -- 172.21.15.100:0/150072195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5557e094c390 msgr2=0x5557e0d1e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:37.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.824+0000 7f1b4a5db740 1 --2- 172.21.15.100:0/150072195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5557e094c390 0x5557e0d1e880 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c0042f0 tx=0x7f1b2c0319d0 comp rx=0 tx=0).stop 2024-07-26T13:42:37.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.825+0000 7f1b4a5db740 1 -- 172.21.15.100:0/150072195 shutdown_connections 2024-07-26T13:42:37.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.825+0000 7f1b4a5db740 1 --2- 172.21.15.100:0/150072195 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5557e092b9f0 0x5557e0d301f0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:37.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.825+0000 7f1b4a5db740 1 --2- 172.21.15.100:0/150072195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557e096d070 0x5557e0d2a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:37.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.825+0000 7f1b4a5db740 1 --2- 172.21.15.100:0/150072195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5557e094c390 0x5557e0d1e880 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:37.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.825+0000 7f1b4a5db740 1 --2- 172.21.15.100:0/150072195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557e09441b0 0x5557e0d1e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:37.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.825+0000 7f1b257fa640 0 client.16215 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:37.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.825+0000 7f1b4a5db740 1 -- 172.21.15.100:0/150072195 >> 172.21.15.100:0/150072195 conn(0x5557e0961460 msgr2=0x5557e0d0ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:37.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.825+0000 7f1b4a5db740 1 -- 172.21.15.100:0/150072195 shutdown_connections 2024-07-26T13:42:37.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.825+0000 7f1b4a5db740 1 -- 172.21.15.100:0/150072195 wait complete. 2024-07-26T13:42:37.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.825+0000 7f1b4a5db740 20 client.16215 trim_cache size 0 max 16384 2024-07-26T13:42:37.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.843+0000 7f1b4a5db740 1 Processor -- start 2024-07-26T13:42:37.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.843+0000 7f1b4a5db740 1 -- start start 2024-07-26T13:42:37.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.843+0000 7f1b4a5db740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557e09441b0 0x5557e0d0f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:37.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.843+0000 7f1b4a5db740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557e094c390 0x5557e0d178f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:37.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.843+0000 7f1b4a5db740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5557e096d070 0x5557e0d17e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:37.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.843+0000 7f1b4a5db740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c1b3c50 con 0x5557e096d070 2024-07-26T13:42:37.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.843+0000 7f1b4a5db740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c1b1520 con 0x5557e094c390 2024-07-26T13:42:37.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.843+0000 7f1b4a5db740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c1af370 con 0x5557e09441b0 2024-07-26T13:42:37.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.843+0000 7f1b425f4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5557e096d070 0x5557e0d17e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:37.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.843+0000 7f1b425f4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5557e096d070 0x5557e0d17e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38216/0 (socket says 172.21.15.100:38216) 2024-07-26T13:42:37.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.843+0000 7f1b425f4640 1 -- 172.21.15.100:0/3601365189 learned_addr learned my addr 172.21.15.100:0/3601365189 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:37.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.843+0000 7f1b41df3640 1 --2- 172.21.15.100:0/3601365189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557e09441b0 0x5557e0d0f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:37.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.843+0000 7f1b415f2640 1 --2- 172.21.15.100:0/3601365189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557e094c390 0x5557e0d178f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:37.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.843+0000 7f1b425f4640 1 -- 172.21.15.100:0/3601365189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557e09441b0 msgr2=0x5557e0d0f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:37.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.843+0000 7f1b425f4640 1 --2- 172.21.15.100:0/3601365189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557e09441b0 0x5557e0d0f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:37.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.843+0000 7f1b425f4640 1 -- 172.21.15.100:0/3601365189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557e094c390 msgr2=0x5557e0d178f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:37.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.843+0000 7f1b425f4640 1 --2- 172.21.15.100:0/3601365189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557e094c390 0x5557e0d178f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:37.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.843+0000 7f1b425f4640 1 -- 172.21.15.100:0/3601365189 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557e0d1f380 con 0x5557e096d070 2024-07-26T13:42:37.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.844+0000 7f1b425f4640 1 --2- 172.21.15.100:0/3601365189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5557e096d070 0x5557e0d17e30 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400fd30 tx=0x7f1b34016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ccda960914bd22 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:37.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.844+0000 7f1b277fe640 1 -- 172.21.15.100:0/3601365189 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b34004b20 con 0x5557e096d070 2024-07-26T13:42:37.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.844+0000 7f1b277fe640 1 -- 172.21.15.100:0/3601365189 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b34003e20 con 0x5557e096d070 2024-07-26T13:42:37.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.844+0000 7f1b277fe640 1 -- 172.21.15.100:0/3601365189 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3401ce60 con 0x5557e096d070 2024-07-26T13:42:37.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.845+0000 7f1b4a5db740 1 -- 172.21.15.100:0/3601365189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5557e096d070 msgr2=0x5557e0d17e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:37.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.845+0000 7f1b4a5db740 1 --2- 172.21.15.100:0/3601365189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5557e096d070 0x5557e0d17e30 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400fd30 tx=0x7f1b34016000 comp rx=0 tx=0).stop 2024-07-26T13:42:37.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.845+0000 7f1b4a5db740 1 -- 172.21.15.100:0/3601365189 shutdown_connections 2024-07-26T13:42:37.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.845+0000 7f1b4a5db740 1 --2- 172.21.15.100:0/3601365189 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5557e096d070 0x5557e0d17e30 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:37.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.845+0000 7f1b4a5db740 1 --2- 172.21.15.100:0/3601365189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557e094c390 0x5557e0d178f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:37.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.845+0000 7f1b4a5db740 1 --2- 172.21.15.100:0/3601365189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557e09441b0 0x5557e0d0f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:37.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.845+0000 7f1b4a5db740 1 -- 172.21.15.100:0/3601365189 >> 172.21.15.100:0/3601365189 conn(0x5557e0961460 msgr2=0x5557e0d0ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:37.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.845+0000 7f1b4a5db740 1 -- 172.21.15.100:0/3601365189 shutdown_connections 2024-07-26T13:42:37.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.845+0000 7f1b4a5db740 1 -- 172.21.15.100:0/3601365189 wait complete. 2024-07-26T13:42:37.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.845+0000 7f1b4a5db740 1 -- start start 2024-07-26T13:42:37.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.845+0000 7f1b4a5db740 1 Processor -- start 2024-07-26T13:42:37.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.846+0000 7f1b4a5db740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557e09441b0 0x5557e0d339e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:37.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.846+0000 7f1b4a5db740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5557e094c390 0x5557e0d2ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:37.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.846+0000 7f1b4a5db740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557e096d070 0x5557e0d2f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:37.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.846+0000 7f1b4a5db740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c1af9e0 con 0x5557e094c390 2024-07-26T13:42:37.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.846+0000 7f1b4a5db740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c1b0e50 con 0x5557e09441b0 2024-07-26T13:42:37.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.846+0000 7f1b4a5db740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c1b00b0 con 0x5557e096d070 2024-07-26T13:42:37.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.846+0000 7f1b4a5db740 10 client.0 _ll_register_callbacks cb 0x7f1b34009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:42:37.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.846+0000 7f1b415f2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5557e094c390 0x5557e0d2ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:37.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.846+0000 7f1b425f4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557e096d070 0x5557e0d2f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:37.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.846+0000 7f1b41df3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557e09441b0 0x5557e0d339e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:37.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.846+0000 7f1b415f2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5557e094c390 0x5557e0d2ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38222/0 (socket says 172.21.15.100:38222) 2024-07-26T13:42:37.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.846+0000 7f1b425f4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557e096d070 0x5557e0d2f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:36194/0 (socket says 172.21.15.100:36194) 2024-07-26T13:42:37.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.846+0000 7f1b415f2640 1 -- 172.21.15.100:0/1379203992 learned_addr learned my addr 172.21.15.100:0/1379203992 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:37.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.846+0000 7f1b415f2640 1 -- 172.21.15.100:0/1379203992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557e096d070 msgr2=0x5557e0d2f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:37.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.846+0000 7f1b415f2640 1 --2- 172.21.15.100:0/1379203992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557e096d070 0x5557e0d2f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:37.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.846+0000 7f1b415f2640 1 -- 172.21.15.100:0/1379203992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557e09441b0 msgr2=0x5557e0d339e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:37.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.846+0000 7f1b415f2640 1 --2- 172.21.15.100:0/1379203992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557e09441b0 0x5557e0d339e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:37.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.846+0000 7f1b415f2640 1 -- 172.21.15.100:0/1379203992 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557e0d2fc50 con 0x5557e094c390 2024-07-26T13:42:37.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.846+0000 7f1b425f4640 1 --2- 172.21.15.100:0/1379203992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557e096d070 0x5557e0d2f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:42:37.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.847+0000 7f1b415f2640 1 --2- 172.21.15.100:0/1379203992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5557e094c390 0x5557e0d2ee90 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c006010 tx=0x7f1b2c0045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae27c837b2f7065f server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:37.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.847+0000 7f1b257fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:42:37.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.847+0000 7f1b257fa640 1 -- 172.21.15.100:0/1379203992 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c035050 con 0x5557e094c390 2024-07-26T13:42:37.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.847+0000 7f1b257fa640 1 -- 172.21.15.100:0/1379203992 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f1b2c031c40 con 0x5557e094c390 2024-07-26T13:42:37.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.847+0000 7f1b4a5db740 1 -- 172.21.15.100:0/1379203992 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b34003e20 con 0x5557e094c390 2024-07-26T13:42:37.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.847+0000 7f1b257fa640 1 -- 172.21.15.100:0/1379203992 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1b2c067050 con 0x5557e094c390 2024-07-26T13:42:37.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.847+0000 7f1b4a5db740 10 client.16221 fetch_fsmap learned FSMap version 27 2024-07-26T13:42:37.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.847+0000 7f1b4a5db740 1 -- 172.21.15.100:0/1379203992 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5557e0d98780 con 0x5557e094c390 2024-07-26T13:42:37.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.847+0000 7f1b257fa640 1 -- 172.21.15.100:0/1379203992 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b34003e20 con 0x5557e094c390 2024-07-26T13:42:37.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.847+0000 7f1b257fa640 1 -- 172.21.15.100:0/1379203992 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1b2c003800 con 0x5557e094c390 2024-07-26T13:42:37.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.847+0000 7f1b4a5db740 10 client.16221 fetch_fsmap finished waiting for FSMap version 27 2024-07-26T13:42:37.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.847+0000 7f1b4a5db740 10 client.16221 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:42:37.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.847+0000 7f1b4a5db740 20 client.16221 populate_metadata read hostname 'smithi100' 2024-07-26T13:42:37.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.847+0000 7f1b4a5db740 1 --2- 172.21.15.100:0/1379203992 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5557e0900da0 0x5557e0d9af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:37.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.847+0000 7f1b4a5db740 4 client.16221 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-26T13:42:37.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.847+0000 7f1b4a5db740 1 -- 172.21.15.100:0/1379203992 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5557e0c86580 con 0x5557e0900da0 2024-07-26T13:42:37.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.848+0000 7f1b41df3640 1 --2- 172.21.15.100:0/1379203992 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5557e0900da0 0x5557e0d9af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:37.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.848+0000 7f1b257fa640 10 client.16221 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:37.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.848+0000 7f1b41df3640 1 --2- 172.21.15.100:0/1379203992 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5557e0900da0 0x5557e0d9af90 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed05d3ea7f24068c server_cookie=30e1e26b3828e95e in_seq=0 out_seq=0 2024-07-26T13:42:37.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.848+0000 7f1b257fa640 1 -- 172.21.15.100:0/1379203992 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5557e0c86580 con 0x5557e0900da0 2024-07-26T13:42:37.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.848+0000 7f1b257fa640 10 client.16221 handle_command_reply: tid=0 2024-07-26T13:42:37.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.848+0000 7f1b4a5db740 1 client.16221 shutdown 2024-07-26T13:42:37.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.848+0000 7f1b4a5db740 1 -- 172.21.15.100:0/1379203992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5557e094c390 msgr2=0x5557e0d2ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:37.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.848+0000 7f1b4a5db740 1 --2- 172.21.15.100:0/1379203992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5557e094c390 0x5557e0d2ee90 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c006010 tx=0x7f1b2c0045d0 comp rx=0 tx=0).stop 2024-07-26T13:42:37.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.849+0000 7f1b4a5db740 1 -- 172.21.15.100:0/1379203992 shutdown_connections 2024-07-26T13:42:37.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.849+0000 7f1b4a5db740 1 --2- 172.21.15.100:0/1379203992 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5557e0900da0 0x5557e0d9af90 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:37.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.849+0000 7f1b4a5db740 1 --2- 172.21.15.100:0/1379203992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557e096d070 0x5557e0d2f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:37.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.849+0000 7f1b4a5db740 1 --2- 172.21.15.100:0/1379203992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5557e094c390 0x5557e0d2ee90 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:37.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.849+0000 7f1b257fa640 0 client.16221 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:37.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.849+0000 7f1b4a5db740 1 --2- 172.21.15.100:0/1379203992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557e09441b0 0x5557e0d339e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:37.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.849+0000 7f1b4a5db740 1 -- 172.21.15.100:0/1379203992 >> 172.21.15.100:0/1379203992 conn(0x5557e0961460 msgr2=0x5557e0d0fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:37.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.849+0000 7f1b4a5db740 1 -- 172.21.15.100:0/1379203992 shutdown_connections 2024-07-26T13:42:37.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.849+0000 7f1b4a5db740 1 -- 172.21.15.100:0/1379203992 wait complete. 2024-07-26T13:42:37.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.849+0000 7f1b4a5db740 20 client.16221 trim_cache size 0 max 16384 2024-07-26T13:42:37.855 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:42:37.855 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-26T13:42:37.855 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-07-26T13:42:37.855 INFO:teuthology.orchestra.run.smithi100.stdout: "a536d90a-b7ef-44dd-be34-4c34840ab2f5": { 2024-07-26T13:42:37.855 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-07-26T13:42:37.855 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "a536d90a-b7ef-44dd-be34-4c34840ab2f5", 2024-07-26T13:42:37.855 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-07-26T13:42:37.855 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:42:37.855 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:42:37.855 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:42:37.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.849+0000 7f1b44057640 1 -- 172.21.15.100:0/232831498 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1b280b3dc0 msgr2=0x7f1b280b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:37.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.849+0000 7f1b44057640 1 --2- 172.21.15.100:0/232831498 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1b280b3dc0 0x7f1b280b62d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1b34009fd0 tx=0x7f1b34009500 comp rx=0 tx=0).stop 2024-07-26T13:42:37.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.849+0000 7f1b44057640 1 -- 172.21.15.100:0/232831498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b3c102490 msgr2=0x7f1b3c198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:37.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.849+0000 7f1b44057640 1 --2- 172.21.15.100:0/232831498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b3c102490 0x7f1b3c198ad0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800d980 tx=0x7f1b3800de40 comp rx=0 tx=0).stop 2024-07-26T13:42:37.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.849+0000 7f1b44057640 1 -- 172.21.15.100:0/232831498 shutdown_connections 2024-07-26T13:42:37.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.849+0000 7f1b44057640 1 --2- 172.21.15.100:0/232831498 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1b280b3dc0 0x7f1b280b62d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:37.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.849+0000 7f1b44057640 1 --2- 172.21.15.100:0/232831498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b3c108720 0x7f1b3c199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:37.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.849+0000 7f1b44057640 1 --2- 172.21.15.100:0/232831498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b3c107dd0 0x7f1b3c199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:37.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.849+0000 7f1b44057640 1 --2- 172.21.15.100:0/232831498 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b3c102490 0x7f1b3c198ad0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:37.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.849+0000 7f1b44057640 1 -- 172.21.15.100:0/232831498 >> 172.21.15.100:0/232831498 conn(0x7f1b3c079160 msgr2=0x7f1b3c103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:37.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.850+0000 7f1b44057640 1 -- 172.21.15.100:0/232831498 shutdown_connections 2024-07-26T13:42:37.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:42:37.850+0000 7f1b44057640 1 -- 172.21.15.100:0/232831498 wait complete. 2024-07-26T13:42:37.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:37 smithi138 ceph-mon[36954]: pgmap v439: 97 pgs: 97 active+clean; 29 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 528 KiB/s wr, 109 op/s 2024-07-26T13:42:38.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:37 smithi167 ceph-mon[37140]: pgmap v439: 97 pgs: 97 active+clean; 29 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 528 KiB/s wr, 109 op/s 2024-07-26T13:42:38.405 INFO:tasks.workunit.client.0.smithi100.stdout: 1 44999 4.95 MB/sec warmup 52 sec latency 11.909 ms 2024-07-26T13:42:38.673 INFO:tasks.cephfs.filesystem:scrub status for tag:a536d90a-b7ef-44dd-be34-4c34840ab2f5 - {'path': '/', 'tag': 'a536d90a-b7ef-44dd-be34-4c34840ab2f5', 'options': 'recursive,force'} 2024-07-26T13:42:39.405 INFO:tasks.workunit.client.0.smithi100.stdout: 1 46767 5.09 MB/sec warmup 53 sec latency 10.773 ms 2024-07-26T13:42:39.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:39 smithi138 ceph-mon[36954]: pgmap v440: 97 pgs: 97 active+clean; 31 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 714 KiB/s wr, 133 op/s 2024-07-26T13:42:39.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:39 smithi100 ceph-mon[32599]: pgmap v440: 97 pgs: 97 active+clean; 31 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 714 KiB/s wr, 133 op/s 2024-07-26T13:42:40.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:39 smithi167 ceph-mon[37140]: pgmap v440: 97 pgs: 97 active+clean; 31 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 714 KiB/s wr, 133 op/s 2024-07-26T13:42:40.405 INFO:tasks.workunit.client.0.smithi100.stdout: 1 49761 5.32 MB/sec warmup 54 sec latency 13.024 ms 2024-07-26T13:42:41.405 INFO:tasks.workunit.client.0.smithi100.stdout: 1 51273 5.34 MB/sec warmup 55 sec latency 12.738 ms 2024-07-26T13:42:41.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:41 smithi138 ceph-mon[36954]: pgmap v441: 97 pgs: 97 active+clean; 35 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.1 MiB/s wr, 213 op/s 2024-07-26T13:42:41.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:41 smithi100 ceph-mon[32599]: pgmap v441: 97 pgs: 97 active+clean; 35 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.1 MiB/s wr, 213 op/s 2024-07-26T13:42:42.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:41 smithi167 ceph-mon[37140]: pgmap v441: 97 pgs: 97 active+clean; 35 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.1 MiB/s wr, 213 op/s 2024-07-26T13:42:42.405 INFO:tasks.workunit.client.0.smithi100.stdout: 1 53652 5.50 MB/sec warmup 56 sec latency 12.090 ms 2024-07-26T13:42:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.843+0000 7f090496c640 1 -- 172.21.15.100:0/2367714087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09001097b0 msgr2=0x7f090010ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.843+0000 7f090496c640 1 --2- 172.21.15.100:0/2367714087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09001097b0 0x7f090010ffe0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f08f8009ca0 tx=0x7f08f802a8c0 comp rx=0 tx=0).stop 2024-07-26T13:42:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.843+0000 7f090496c640 1 -- 172.21.15.100:0/2367714087 shutdown_connections 2024-07-26T13:42:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.843+0000 7f090496c640 1 --2- 172.21.15.100:0/2367714087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09001097b0 0x7f090010ffe0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.843+0000 7f090496c640 1 --2- 172.21.15.100:0/2367714087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0900108e60 0x7f0900109270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.843+0000 7f090496c640 1 --2- 172.21.15.100:0/2367714087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0900101a90 0x7f0900101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.843+0000 7f090496c640 1 -- 172.21.15.100:0/2367714087 >> 172.21.15.100:0/2367714087 conn(0x7f09000798c0 msgr2=0x7f0900079cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.843+0000 7f090496c640 1 -- 172.21.15.100:0/2367714087 shutdown_connections 2024-07-26T13:42:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.843+0000 7f090496c640 1 -- 172.21.15.100:0/2367714087 wait complete. 2024-07-26T13:42:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.844+0000 7f090496c640 1 Processor -- start 2024-07-26T13:42:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.844+0000 7f090496c640 1 -- start start 2024-07-26T13:42:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.844+0000 7f090496c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0900101a90 0x7f0900199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.844+0000 7f090496c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0900108e60 0x7f090019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.844+0000 7f090496c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09001097b0 0x7f090019f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.844+0000 7f090496c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0900113480 con 0x7f09001097b0 2024-07-26T13:42:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.844+0000 7f090496c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0900113300 con 0x7f0900101a90 2024-07-26T13:42:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.844+0000 7f090496c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0900113600 con 0x7f0900108e60 2024-07-26T13:42:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.844+0000 7f08fed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09001097b0 0x7f090019f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.844+0000 7f08fe59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0900101a90 0x7f0900199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.844+0000 7f08fed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09001097b0 0x7f090019f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38246/0 (socket says 172.21.15.100:38246) 2024-07-26T13:42:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.844+0000 7f08fed9d640 1 -- 172.21.15.100:0/513837224 learned_addr learned my addr 172.21.15.100:0/513837224 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.844+0000 7f08fdd9b640 1 --2- 172.21.15.100:0/513837224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0900108e60 0x7f090019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.844+0000 7f08fe59c640 1 -- 172.21.15.100:0/513837224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0900108e60 msgr2=0x7f090019a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.844+0000 7f08fe59c640 1 --2- 172.21.15.100:0/513837224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0900108e60 0x7f090019a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.844+0000 7f08fe59c640 1 -- 172.21.15.100:0/513837224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09001097b0 msgr2=0x7f090019f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.844+0000 7f08fe59c640 1 --2- 172.21.15.100:0/513837224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09001097b0 0x7f090019f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.844+0000 7f08fe59c640 1 -- 172.21.15.100:0/513837224 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f090019fd90 con 0x7f0900101a90 2024-07-26T13:42:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.844+0000 7f08fed9d640 1 --2- 172.21.15.100:0/513837224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09001097b0 0x7f090019f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:42:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.845+0000 7f08fe59c640 1 --2- 172.21.15.100:0/513837224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0900101a90 0x7f0900199e60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f08f000e9f0 tx=0x7f08f000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.845+0000 7f08ef7fe640 1 -- 172.21.15.100:0/513837224 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f000ccc0 con 0x7f0900101a90 2024-07-26T13:42:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.845+0000 7f090496c640 1 -- 172.21.15.100:0/513837224 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09001a0080 con 0x7f0900101a90 2024-07-26T13:42:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.845+0000 7f08ef7fe640 1 -- 172.21.15.100:0/513837224 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08f00108e0 con 0x7f0900101a90 2024-07-26T13:42:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.845+0000 7f08ef7fe640 1 -- 172.21.15.100:0/513837224 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f0018880 con 0x7f0900101a90 2024-07-26T13:42:42.849 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.845+0000 7f090496c640 1 -- 172.21.15.100:0/513837224 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09001a6810 con 0x7f0900101a90 2024-07-26T13:42:42.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.847+0000 7f090b883740 1 Processor -- start 2024-07-26T13:42:42.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.848+0000 7f090b883740 1 -- start start 2024-07-26T13:42:42.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.848+0000 7f08ef7fe640 1 -- 172.21.15.100:0/513837224 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f08f0010430 con 0x7f0900101a90 2024-07-26T13:42:42.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.848+0000 7f090b883740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561b2fb4c1b0 0x561b2feeece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:42.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.848+0000 7f090b883740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b2fb232f0 0x561b2feef220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:42.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.848+0000 7f090b883740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b2fb2b4d0 0x561b2feef760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:42.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.848+0000 7f090b883740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x561b2fe643b0 con 0x561b2fb2b4d0 2024-07-26T13:42:42.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.848+0000 7f090b883740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561b2fe538b0 con 0x561b2fb232f0 2024-07-26T13:42:42.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.848+0000 7f090b883740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561b2fe578f0 con 0x561b2fb4c1b0 2024-07-26T13:42:42.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.848+0000 7f08fed9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561b2fb4c1b0 0x561b2feeece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:42.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.848+0000 7f08fdd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b2fb2b4d0 0x561b2feef760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.848+0000 7f08fed9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561b2fb4c1b0 0x561b2feeece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:36232/0 (socket says 172.21.15.100:36232) 2024-07-26T13:42:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.848+0000 7f08fed9d640 1 -- 172.21.15.100:0/4006598523 learned_addr learned my addr 172.21.15.100:0/4006598523 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.848+0000 7f08fe59c640 1 --2- 172.21.15.100:0/4006598523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b2fb232f0 0x561b2feef220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.848+0000 7f08fed9d640 1 -- 172.21.15.100:0/4006598523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b2fb232f0 msgr2=0x561b2feef220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.848+0000 7f08fed9d640 1 --2- 172.21.15.100:0/4006598523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b2fb232f0 0x561b2feef220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.848+0000 7f08fed9d640 1 -- 172.21.15.100:0/4006598523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b2fb2b4d0 msgr2=0x561b2feef760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.848+0000 7f08fed9d640 1 --2- 172.21.15.100:0/4006598523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b2fb2b4d0 0x561b2feef760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.849+0000 7f08fed9d640 1 -- 172.21.15.100:0/4006598523 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b2fdfa9a0 con 0x561b2fb4c1b0 2024-07-26T13:42:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.849+0000 7f08fdd9b640 1 --2- 172.21.15.100:0/4006598523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b2fb2b4d0 0x561b2feef760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:42:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.849+0000 7f08ef7fe640 1 --2- 172.21.15.100:0/513837224 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f08e80b3d40 0x7f08e80b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.849+0000 7f08fed9d640 1 --2- 172.21.15.100:0/4006598523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561b2fb4c1b0 0x561b2feeece0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f08f80094e0 tx=0x7f08f802f310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2d9d9942d52656d server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.849+0000 7f08fe59c640 1 --2- 172.21.15.100:0/513837224 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f08e80b3d40 0x7f08e80b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.849+0000 7f08ef7fe640 1 -- 172.21.15.100:0/513837224 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f08f00e4360 con 0x7f0900101a90 2024-07-26T13:42:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.849+0000 7f08ecff9640 1 -- 172.21.15.100:0/4006598523 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f8031450 con 0x561b2fb4c1b0 2024-07-26T13:42:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.849+0000 7f08ecff9640 1 -- 172.21.15.100:0/4006598523 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08f8036040 con 0x561b2fb4c1b0 2024-07-26T13:42:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.849+0000 7f08ecff9640 1 -- 172.21.15.100:0/4006598523 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f8004480 con 0x561b2fb4c1b0 2024-07-26T13:42:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.850+0000 7f090b883740 1 -- 172.21.15.100:0/4006598523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561b2fb4c1b0 msgr2=0x561b2feeece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.850+0000 7f090b883740 1 --2- 172.21.15.100:0/4006598523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561b2fb4c1b0 0x561b2feeece0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f08f80094e0 tx=0x7f08f802f310 comp rx=0 tx=0).stop 2024-07-26T13:42:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.850+0000 7f08fe59c640 1 --2- 172.21.15.100:0/513837224 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f08e80b3d40 0x7f08e80b6250 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f08f00e1b80 tx=0x7f08f00e7eb0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.850+0000 7f090b883740 1 -- 172.21.15.100:0/4006598523 shutdown_connections 2024-07-26T13:42:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.850+0000 7f090b883740 1 --2- 172.21.15.100:0/4006598523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b2fb2b4d0 0x561b2feef760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.850+0000 7f090b883740 1 --2- 172.21.15.100:0/4006598523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b2fb232f0 0x561b2feef220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.850+0000 7f090b883740 1 --2- 172.21.15.100:0/4006598523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561b2fb4c1b0 0x561b2feeece0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.850+0000 7f090b883740 1 -- 172.21.15.100:0/4006598523 >> 172.21.15.100:0/4006598523 conn(0x561b2fb40300 msgr2=0x561b2feecbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.850+0000 7f090b883740 1 -- 172.21.15.100:0/4006598523 shutdown_connections 2024-07-26T13:42:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.850+0000 7f090b883740 1 -- 172.21.15.100:0/4006598523 wait complete. 2024-07-26T13:42:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.850+0000 7f090b883740 1 -- start start 2024-07-26T13:42:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.851+0000 7f090b883740 1 Processor -- start 2024-07-26T13:42:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.851+0000 7f090b883740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b2fb232f0 0x561b2fefa820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.851+0000 7f090b883740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b2fb2b4d0 0x561b2fefad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.851+0000 7f090b883740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561b2fb4c1b0 0x561b2ff06570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.851+0000 7f090b883740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x561b2fd8dab0 con 0x561b2fb232f0 2024-07-26T13:42:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.851+0000 7f090b883740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561b2fddbcf0 con 0x561b2fb2b4d0 2024-07-26T13:42:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.851+0000 7f090b883740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561b2fefd280 con 0x561b2fb4c1b0 2024-07-26T13:42:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.851+0000 7f090b883740 10 client.0 _ll_register_callbacks cb 0x561b2fe4d9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:42:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.851+0000 7f08fed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b2fb232f0 0x561b2fefa820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.851+0000 7f08fed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b2fb232f0 0x561b2fefa820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38276/0 (socket says 172.21.15.100:38276) 2024-07-26T13:42:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.851+0000 7f08fed9d640 1 -- 172.21.15.100:0/3983878434 learned_addr learned my addr 172.21.15.100:0/3983878434 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.851+0000 7f08fdd9b640 1 --2- 172.21.15.100:0/3983878434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b2fb2b4d0 0x561b2fefad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.851+0000 7f08fed9d640 1 --2- 172.21.15.100:0/3983878434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561b2fb4c1b0 0x561b2ff06570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.851+0000 7f08fed9d640 1 -- 172.21.15.100:0/3983878434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561b2fb4c1b0 msgr2=0x561b2ff06570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.851+0000 7f08fed9d640 1 --2- 172.21.15.100:0/3983878434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561b2fb4c1b0 0x561b2ff06570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.851+0000 7f08fed9d640 1 -- 172.21.15.100:0/3983878434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b2fb2b4d0 msgr2=0x561b2fefad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.851+0000 7f08fed9d640 1 --2- 172.21.15.100:0/3983878434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b2fb2b4d0 0x561b2fefad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.851+0000 7f08fed9d640 1 -- 172.21.15.100:0/3983878434 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b2fbb7790 con 0x561b2fb232f0 2024-07-26T13:42:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.852+0000 7f08fed9d640 1 --2- 172.21.15.100:0/3983878434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b2fb232f0 0x561b2fefa820 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x561b2fefd030 tx=0x7f08f8004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da326a2e5eebed3f server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.852+0000 7f08e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:42:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.852+0000 7f08e5ffb640 1 -- 172.21.15.100:0/3983878434 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f804b040 con 0x561b2fb232f0 2024-07-26T13:42:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.852+0000 7f08e5ffb640 1 -- 172.21.15.100:0/3983878434 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f08f8004260 con 0x561b2fb232f0 2024-07-26T13:42:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.852+0000 7f090b883740 1 -- 172.21.15.100:0/3983878434 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08f8036040 con 0x561b2fb232f0 2024-07-26T13:42:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.852+0000 7f08e5ffb640 1 -- 172.21.15.100:0/3983878434 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f08f8004460 con 0x561b2fb232f0 2024-07-26T13:42:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.852+0000 7f090b883740 10 client.16233 fetch_fsmap learned FSMap version 27 2024-07-26T13:42:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.852+0000 7f090b883740 1 -- 172.21.15.100:0/3983878434 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561b2fc30620 con 0x561b2fb232f0 2024-07-26T13:42:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.852+0000 7f08e5ffb640 1 -- 172.21.15.100:0/3983878434 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08f8036040 con 0x561b2fb232f0 2024-07-26T13:42:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.852+0000 7f08e5ffb640 1 -- 172.21.15.100:0/3983878434 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f08f805b070 con 0x561b2fb232f0 2024-07-26T13:42:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.852+0000 7f090b883740 10 client.16233 fetch_fsmap finished waiting for FSMap version 27 2024-07-26T13:42:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.852+0000 7f090b883740 10 client.16233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:42:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.852+0000 7f090b883740 20 client.16233 populate_metadata read hostname 'smithi100' 2024-07-26T13:42:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.852+0000 7f090b883740 1 --2- 172.21.15.100:0/3983878434 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x561b2fb0ad80 0x561b2ff0c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.852+0000 7f090b883740 4 client.16233 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:42:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.852+0000 7f090b883740 1 -- 172.21.15.100:0/3983878434 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b2fe66300 con 0x561b2fb0ad80 2024-07-26T13:42:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.853+0000 7f08fdd9b640 1 --2- 172.21.15.100:0/3983878434 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x561b2fb0ad80 0x561b2ff0c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.853+0000 7f08e5ffb640 10 client.16233 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.853+0000 7f08fdd9b640 1 --2- 172.21.15.100:0/3983878434 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x561b2fb0ad80 0x561b2ff0c3b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edcebf8e89cd10c2 server_cookie=2c472badb7ed195e in_seq=0 out_seq=0 2024-07-26T13:42:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.854+0000 7f08e5ffb640 1 -- 172.21.15.100:0/3983878434 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561b2fe66300 con 0x561b2fb0ad80 2024-07-26T13:42:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.854+0000 7f08e5ffb640 10 client.16233 handle_command_reply: tid=0 2024-07-26T13:42:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.854+0000 7f090b883740 1 client.16233 shutdown 2024-07-26T13:42:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.854+0000 7f090b883740 1 -- 172.21.15.100:0/3983878434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b2fb232f0 msgr2=0x561b2fefa820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.854+0000 7f090b883740 1 --2- 172.21.15.100:0/3983878434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b2fb232f0 0x561b2fefa820 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x561b2fefd030 tx=0x7f08f8004180 comp rx=0 tx=0).stop 2024-07-26T13:42:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.854+0000 7f090b883740 1 -- 172.21.15.100:0/3983878434 shutdown_connections 2024-07-26T13:42:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.854+0000 7f090b883740 1 --2- 172.21.15.100:0/3983878434 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x561b2fb0ad80 0x561b2ff0c3b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.854+0000 7f090b883740 1 --2- 172.21.15.100:0/3983878434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561b2fb4c1b0 0x561b2ff06570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.854+0000 7f090b883740 1 --2- 172.21.15.100:0/3983878434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b2fb2b4d0 0x561b2fefad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.854+0000 7f090b883740 1 --2- 172.21.15.100:0/3983878434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b2fb232f0 0x561b2fefa820 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.854+0000 7f08e5ffb640 0 client.16233 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.854+0000 7f090b883740 1 -- 172.21.15.100:0/3983878434 >> 172.21.15.100:0/3983878434 conn(0x561b2fb40300 msgr2=0x561b2feefd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.854+0000 7f090b883740 1 -- 172.21.15.100:0/3983878434 shutdown_connections 2024-07-26T13:42:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.854+0000 7f090b883740 1 -- 172.21.15.100:0/3983878434 wait complete. 2024-07-26T13:42:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.854+0000 7f090b883740 20 client.16233 trim_cache size 0 max 16384 2024-07-26T13:42:42.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.870+0000 7f090b883740 1 Processor -- start 2024-07-26T13:42:42.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.870+0000 7f090b883740 1 -- start start 2024-07-26T13:42:42.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.870+0000 7f090b883740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b2fb232f0 0x561b2fefac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:42.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.870+0000 7f090b883740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b2fb2b4d0 0x561b2ff11a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.870+0000 7f090b883740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561b2fb4c1b0 0x561b2ff11fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.870+0000 7f090b883740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f09001b1910 con 0x561b2fb2b4d0 2024-07-26T13:42:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.871+0000 7f090b883740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f09001af1e0 con 0x561b2fb232f0 2024-07-26T13:42:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.871+0000 7f090b883740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f09001ad030 con 0x561b2fb4c1b0 2024-07-26T13:42:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.871+0000 7f08fdd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b2fb2b4d0 0x561b2ff11a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.871+0000 7f08fdd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b2fb2b4d0 0x561b2ff11a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38288/0 (socket says 172.21.15.100:38288) 2024-07-26T13:42:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.871+0000 7f08fdd9b640 1 -- 172.21.15.100:0/3572493329 learned_addr learned my addr 172.21.15.100:0/3572493329 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.871+0000 7f08fed9d640 1 --2- 172.21.15.100:0/3572493329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561b2fb4c1b0 0x561b2ff11fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.871+0000 7f08fed9d640 1 --2- 172.21.15.100:0/3572493329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b2fb232f0 0x561b2fefac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.871+0000 7f08fdd9b640 1 -- 172.21.15.100:0/3572493329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561b2fb4c1b0 msgr2=0x561b2ff11fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.871+0000 7f08fdd9b640 1 --2- 172.21.15.100:0/3572493329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561b2fb4c1b0 0x561b2ff11fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.871+0000 7f08fdd9b640 1 -- 172.21.15.100:0/3572493329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b2fb232f0 msgr2=0x561b2fefac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.871+0000 7f08fdd9b640 1 --2- 172.21.15.100:0/3572493329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b2fb232f0 0x561b2fefac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.871+0000 7f08fdd9b640 1 -- 172.21.15.100:0/3572493329 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b2fefa390 con 0x561b2fb2b4d0 2024-07-26T13:42:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.871+0000 7f08fdd9b640 1 --2- 172.21.15.100:0/3572493329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b2fb2b4d0 0x561b2ff11a90 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x561b2ff0ca30 tx=0x7f08f4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b2b80087d32d7d server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.871+0000 7f08e7fff640 1 -- 172.21.15.100:0/3572493329 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f4022020 con 0x561b2fb2b4d0 2024-07-26T13:42:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.871+0000 7f08e7fff640 1 -- 172.21.15.100:0/3572493329 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08f40108e0 con 0x561b2fb2b4d0 2024-07-26T13:42:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.871+0000 7f08e7fff640 1 -- 172.21.15.100:0/3572493329 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f401e8b0 con 0x561b2fb2b4d0 2024-07-26T13:42:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.872+0000 7f090b883740 1 -- 172.21.15.100:0/3572493329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b2fb2b4d0 msgr2=0x561b2ff11a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.872+0000 7f090b883740 1 --2- 172.21.15.100:0/3572493329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b2fb2b4d0 0x561b2ff11a90 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x561b2ff0ca30 tx=0x7f08f4005f90 comp rx=0 tx=0).stop 2024-07-26T13:42:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.872+0000 7f090b883740 1 -- 172.21.15.100:0/3572493329 shutdown_connections 2024-07-26T13:42:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.872+0000 7f090b883740 1 --2- 172.21.15.100:0/3572493329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561b2fb4c1b0 0x561b2ff11fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.872+0000 7f090b883740 1 --2- 172.21.15.100:0/3572493329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b2fb2b4d0 0x561b2ff11a90 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.872+0000 7f090b883740 1 --2- 172.21.15.100:0/3572493329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b2fb232f0 0x561b2fefac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.872+0000 7f090b883740 1 -- 172.21.15.100:0/3572493329 >> 172.21.15.100:0/3572493329 conn(0x561b2fb40300 msgr2=0x561b2fefa580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.872+0000 7f090b883740 1 -- 172.21.15.100:0/3572493329 shutdown_connections 2024-07-26T13:42:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.872+0000 7f090b883740 1 -- 172.21.15.100:0/3572493329 wait complete. 2024-07-26T13:42:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.873+0000 7f090b883740 1 -- start start 2024-07-26T13:42:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.873+0000 7f090b883740 1 Processor -- start 2024-07-26T13:42:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.873+0000 7f090b883740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b2fb232f0 0x561b2ff02d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.873+0000 7f090b883740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561b2fb2b4d0 0x561b2ff01220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.873+0000 7f090b883740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b2fb4c1b0 0x561b2ff01910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.873+0000 7f090b883740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f09001ad6a0 con 0x561b2fb232f0 2024-07-26T13:42:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.873+0000 7f090b883740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f09001aeb10 con 0x561b2fb4c1b0 2024-07-26T13:42:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.873+0000 7f090b883740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f09001add70 con 0x561b2fb2b4d0 2024-07-26T13:42:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.873+0000 7f090b883740 10 client.0 _ll_register_callbacks cb 0x7f08f4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:42:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.873+0000 7f08fed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b2fb232f0 0x561b2ff02d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.873+0000 7f08fed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b2fb232f0 0x561b2ff02d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38304/0 (socket says 172.21.15.100:38304) 2024-07-26T13:42:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.873+0000 7f08fed9d640 1 -- 172.21.15.100:0/1260558516 learned_addr learned my addr 172.21.15.100:0/1260558516 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.873+0000 7f08fdd9b640 1 --2- 172.21.15.100:0/1260558516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561b2fb2b4d0 0x561b2ff01220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.873+0000 7f08fed9d640 1 --2- 172.21.15.100:0/1260558516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b2fb4c1b0 0x561b2ff01910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.873+0000 7f08fed9d640 1 -- 172.21.15.100:0/1260558516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561b2fb2b4d0 msgr2=0x561b2ff01220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.873+0000 7f08fed9d640 1 --2- 172.21.15.100:0/1260558516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561b2fb2b4d0 0x561b2ff01220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.873+0000 7f08fed9d640 1 -- 172.21.15.100:0/1260558516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b2fb4c1b0 msgr2=0x561b2ff01910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.873+0000 7f08fed9d640 1 --2- 172.21.15.100:0/1260558516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b2fb4c1b0 0x561b2ff01910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.873+0000 7f08fed9d640 1 -- 172.21.15.100:0/1260558516 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b2ff72a70 con 0x561b2fb232f0 2024-07-26T13:42:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.874+0000 7f08fed9d640 1 --2- 172.21.15.100:0/1260558516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b2fb232f0 0x561b2ff02d50 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f08f8005e30 tx=0x7f08f8004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce91b8b447ac88b server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.874+0000 7f08e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:42:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.874+0000 7f08e5ffb640 1 -- 172.21.15.100:0/1260558516 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f8061040 con 0x561b2fb232f0 2024-07-26T13:42:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.874+0000 7f08e5ffb640 1 -- 172.21.15.100:0/1260558516 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7f08f8003c80 con 0x561b2fb232f0 2024-07-26T13:42:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.874+0000 7f090b883740 1 -- 172.21.15.100:0/1260558516 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08f40108e0 con 0x561b2fb232f0 2024-07-26T13:42:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.874+0000 7f08e5ffb640 1 -- 172.21.15.100:0/1260558516 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f08f8004810 con 0x561b2fb232f0 2024-07-26T13:42:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.874+0000 7f090b883740 10 client.16239 fetch_fsmap learned FSMap version 27 2024-07-26T13:42:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.874+0000 7f090b883740 1 -- 172.21.15.100:0/1260558516 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561b2ff73b70 con 0x561b2fb232f0 2024-07-26T13:42:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.874+0000 7f08e5ffb640 1 -- 172.21.15.100:0/1260558516 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08f40108e0 con 0x561b2fb232f0 2024-07-26T13:42:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.874+0000 7f08e5ffb640 1 -- 172.21.15.100:0/1260558516 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f08f806f020 con 0x561b2fb232f0 2024-07-26T13:42:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.874+0000 7f090b883740 10 client.16239 fetch_fsmap finished waiting for FSMap version 27 2024-07-26T13:42:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.874+0000 7f090b883740 10 client.16239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:42:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.874+0000 7f090b883740 20 client.16239 populate_metadata read hostname 'smithi100' 2024-07-26T13:42:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.874+0000 7f090b883740 1 --2- 172.21.15.100:0/1260558516 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x561b2fae01c0 0x561b2ff76240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.874+0000 7f090b883740 4 client.16239 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-26T13:42:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.874+0000 7f090b883740 1 -- 172.21.15.100:0/1260558516 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561b2fe61da0 con 0x561b2fae01c0 2024-07-26T13:42:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.875+0000 7f08fdd9b640 1 --2- 172.21.15.100:0/1260558516 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x561b2fae01c0 0x561b2ff76240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.875+0000 7f08e5ffb640 10 client.16239 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.875+0000 7f08fdd9b640 1 --2- 172.21.15.100:0/1260558516 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x561b2fae01c0 0x561b2ff76240 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2fb6a1f83fe2419 server_cookie=24dfaf379f52b5aa in_seq=0 out_seq=0 2024-07-26T13:42:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.931+0000 7f08e5ffb640 1 -- 172.21.15.100:0/1260558516 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561b2fe61da0 con 0x561b2fae01c0 2024-07-26T13:42:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.931+0000 7f08e5ffb640 10 client.16239 handle_command_reply: tid=0 2024-07-26T13:42:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.931+0000 7f090b883740 1 client.16239 shutdown 2024-07-26T13:42:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.931+0000 7f090b883740 1 -- 172.21.15.100:0/1260558516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b2fb232f0 msgr2=0x561b2ff02d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.931+0000 7f090b883740 1 --2- 172.21.15.100:0/1260558516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b2fb232f0 0x561b2ff02d50 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f08f8005e30 tx=0x7f08f8004180 comp rx=0 tx=0).stop 2024-07-26T13:42:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.931+0000 7f090b883740 1 -- 172.21.15.100:0/1260558516 shutdown_connections 2024-07-26T13:42:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.931+0000 7f090b883740 1 --2- 172.21.15.100:0/1260558516 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x561b2fae01c0 0x561b2ff76240 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.931+0000 7f090b883740 1 --2- 172.21.15.100:0/1260558516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b2fb4c1b0 0x561b2ff01910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.931+0000 7f08e5ffb640 0 client.16239 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.931+0000 7f090b883740 1 --2- 172.21.15.100:0/1260558516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561b2fb2b4d0 0x561b2ff01220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.931+0000 7f090b883740 1 --2- 172.21.15.100:0/1260558516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561b2fb232f0 0x561b2ff02d50 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.931+0000 7f090b883740 1 -- 172.21.15.100:0/1260558516 >> 172.21.15.100:0/1260558516 conn(0x561b2fb40300 msgr2=0x561b2fefbac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.931+0000 7f090b883740 1 -- 172.21.15.100:0/1260558516 shutdown_connections 2024-07-26T13:42:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.931+0000 7f090b883740 1 -- 172.21.15.100:0/1260558516 wait complete. 2024-07-26T13:42:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.931+0000 7f090b883740 20 client.16239 trim_cache size 0 max 16384 2024-07-26T13:42:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-07-26T13:42:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-07-26T13:42:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-07-26T13:42:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-07-26T13:42:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.932+0000 7f090496c640 1 -- 172.21.15.100:0/513837224 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f08e80b3d40 msgr2=0x7f08e80b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.932+0000 7f090496c640 1 --2- 172.21.15.100:0/513837224 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f08e80b3d40 0x7f08e80b6250 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f08f00e1b80 tx=0x7f08f00e7eb0 comp rx=0 tx=0).stop 2024-07-26T13:42:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.932+0000 7f090496c640 1 -- 172.21.15.100:0/513837224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0900101a90 msgr2=0x7f0900199e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.932+0000 7f090496c640 1 --2- 172.21.15.100:0/513837224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0900101a90 0x7f0900199e60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f08f000e9f0 tx=0x7f08f000eeb0 comp rx=0 tx=0).stop 2024-07-26T13:42:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.932+0000 7f090496c640 1 -- 172.21.15.100:0/513837224 shutdown_connections 2024-07-26T13:42:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.932+0000 7f090496c640 1 --2- 172.21.15.100:0/513837224 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f08e80b3d40 0x7f08e80b6250 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.932+0000 7f090496c640 1 --2- 172.21.15.100:0/513837224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f09001097b0 0x7f090019f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.932+0000 7f090496c640 1 --2- 172.21.15.100:0/513837224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0900108e60 0x7f090019a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.932+0000 7f090496c640 1 --2- 172.21.15.100:0/513837224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0900101a90 0x7f0900199e60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.932+0000 7f090496c640 1 -- 172.21.15.100:0/513837224 >> 172.21.15.100:0/513837224 conn(0x7f09000798c0 msgr2=0x7f0900106eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.932+0000 7f090496c640 1 -- 172.21.15.100:0/513837224 shutdown_connections 2024-07-26T13:42:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:42.932+0000 7f090496c640 1 -- 172.21.15.100:0/513837224 wait complete. 2024-07-26T13:42:43.405 INFO:tasks.workunit.client.0.smithi100.stdout: 1 55950 5.59 MB/sec warmup 57 sec latency 22.602 ms 2024-07-26T13:42:43.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:43 smithi138 ceph-mon[36954]: pgmap v442: 97 pgs: 97 active+clean; 36 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 225 op/s 2024-07-26T13:42:43.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:43 smithi100 ceph-mon[32599]: pgmap v442: 97 pgs: 97 active+clean; 36 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 225 op/s 2024-07-26T13:42:44.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:43 smithi167 ceph-mon[37140]: pgmap v442: 97 pgs: 97 active+clean; 36 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 225 op/s 2024-07-26T13:42:44.405 INFO:tasks.workunit.client.0.smithi100.stdout: 1 57561 5.64 MB/sec warmup 58 sec latency 10.869 ms 2024-07-26T13:42:45.405 INFO:tasks.workunit.client.0.smithi100.stdout: 1 60395 5.84 MB/sec warmup 59 sec latency 16.854 ms 2024-07-26T13:42:45.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:45 smithi138 ceph-mon[36954]: pgmap v443: 97 pgs: 97 active+clean; 37 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.7 MiB/s wr, 319 op/s 2024-07-26T13:42:45.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:45 smithi138 ceph-mon[36954]: scrub summary: idle 2024-07-26T13:42:45.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:45 smithi138 ceph-mon[36954]: scrub summary: idle 2024-07-26T13:42:45.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:45 smithi138 ceph-mon[36954]: scrub summary: idle 2024-07-26T13:42:45.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:45 smithi100 ceph-mon[32599]: pgmap v443: 97 pgs: 97 active+clean; 37 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.7 MiB/s wr, 319 op/s 2024-07-26T13:42:45.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:45 smithi100 ceph-mon[32599]: scrub summary: idle 2024-07-26T13:42:45.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:45 smithi100 ceph-mon[32599]: scrub summary: idle 2024-07-26T13:42:45.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:45 smithi100 ceph-mon[32599]: scrub summary: idle 2024-07-26T13:42:46.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:45 smithi167 ceph-mon[37140]: pgmap v443: 97 pgs: 97 active+clean; 37 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.7 MiB/s wr, 319 op/s 2024-07-26T13:42:46.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:45 smithi167 ceph-mon[37140]: scrub summary: idle 2024-07-26T13:42:46.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:45 smithi167 ceph-mon[37140]: scrub summary: idle 2024-07-26T13:42:46.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:45 smithi167 ceph-mon[37140]: scrub summary: idle 2024-07-26T13:42:46.405 INFO:tasks.workunit.client.0.smithi100.stdout: 1 61835 5.85 MB/sec warmup 60 sec latency 14.641 ms 2024-07-26T13:42:47.406 INFO:tasks.workunit.client.0.smithi100.stdout: 1 64258 5.99 MB/sec warmup 61 sec latency 11.649 ms 2024-07-26T13:42:47.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:47 smithi138 ceph-mon[36954]: pgmap v444: 97 pgs: 97 active+clean; 37 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.5 MiB/s wr, 267 op/s 2024-07-26T13:42:47.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:47 smithi100 ceph-mon[32599]: pgmap v444: 97 pgs: 97 active+clean; 37 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.5 MiB/s wr, 267 op/s 2024-07-26T13:42:48.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:47 smithi167 ceph-mon[37140]: pgmap v444: 97 pgs: 97 active+clean; 37 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.5 MiB/s wr, 267 op/s 2024-07-26T13:42:48.406 INFO:tasks.workunit.client.0.smithi100.stdout: 1 66453 6.02 MB/sec warmup 62 sec latency 13.726 ms 2024-07-26T13:42:49.406 INFO:tasks.workunit.client.0.smithi100.stdout: 1 68304 6.11 MB/sec warmup 63 sec latency 12.492 ms 2024-07-26T13:42:49.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:49 smithi167 ceph-mon[37140]: pgmap v445: 97 pgs: 97 active+clean; 39 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.7 MiB/s wr, 293 op/s 2024-07-26T13:42:49.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:49 smithi138 ceph-mon[36954]: pgmap v445: 97 pgs: 97 active+clean; 39 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.7 MiB/s wr, 293 op/s 2024-07-26T13:42:49.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:49 smithi100 ceph-mon[32599]: pgmap v445: 97 pgs: 97 active+clean; 39 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.7 MiB/s wr, 293 op/s 2024-07-26T13:42:50.406 INFO:tasks.workunit.client.0.smithi100.stdout: 1 71122 6.28 MB/sec warmup 64 sec latency 38.959 ms 2024-07-26T13:42:51.406 INFO:tasks.workunit.client.0.smithi100.stdout: 1 72521 6.29 MB/sec warmup 65 sec latency 82.368 ms 2024-07-26T13:42:51.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:51 smithi138 ceph-mon[36954]: pgmap v446: 97 pgs: 97 active+clean; 44 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.0 MiB/s wr, 353 op/s 2024-07-26T13:42:51.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:51 smithi100 ceph-mon[32599]: pgmap v446: 97 pgs: 97 active+clean; 44 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.0 MiB/s wr, 353 op/s 2024-07-26T13:42:52.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:51 smithi167 ceph-mon[37140]: pgmap v446: 97 pgs: 97 active+clean; 44 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.0 MiB/s wr, 353 op/s 2024-07-26T13:42:52.406 INFO:tasks.workunit.client.0.smithi100.stdout: 1 74961 6.40 MB/sec warmup 66 sec latency 9.478 ms 2024-07-26T13:42:53.406 INFO:tasks.workunit.client.0.smithi100.stdout: 1 77566 6.50 MB/sec warmup 67 sec latency 11.418 ms 2024-07-26T13:42:53.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:53 smithi138 ceph-mon[36954]: pgmap v447: 97 pgs: 97 active+clean; 45 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.7 MiB/s wr, 290 op/s 2024-07-26T13:42:53.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:53 smithi100 ceph-mon[32599]: pgmap v447: 97 pgs: 97 active+clean; 45 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.7 MiB/s wr, 290 op/s 2024-07-26T13:42:54.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:53 smithi167 ceph-mon[37140]: pgmap v447: 97 pgs: 97 active+clean; 45 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.7 MiB/s wr, 290 op/s 2024-07-26T13:42:54.406 INFO:tasks.workunit.client.0.smithi100.stdout: 1 79156 6.52 MB/sec warmup 68 sec latency 13.994 ms 2024-07-26T13:42:55.406 INFO:tasks.workunit.client.0.smithi100.stdout: 1 81922 6.67 MB/sec warmup 69 sec latency 11.446 ms 2024-07-26T13:42:55.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:55 smithi138 ceph-mon[36954]: pgmap v448: 97 pgs: 97 active+clean; 48 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.1 MiB/s wr, 331 op/s 2024-07-26T13:42:55.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:55 smithi138 ceph-mon[36954]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:42:55.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:55 smithi100 ceph-mon[32599]: pgmap v448: 97 pgs: 97 active+clean; 48 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.1 MiB/s wr, 331 op/s 2024-07-26T13:42:55.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:55 smithi100 ceph-mon[32599]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:42:56.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:55 smithi167 ceph-mon[37140]: pgmap v448: 97 pgs: 97 active+clean; 48 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.1 MiB/s wr, 331 op/s 2024-07-26T13:42:56.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:55 smithi167 ceph-mon[37140]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:42:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.030+0000 7ff25198e640 1 -- 172.21.15.100:0/309515760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff24c108660 msgr2=0x7ff24c10ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.030+0000 7ff25198e640 1 --2- 172.21.15.100:0/309515760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff24c108660 0x7ff24c10ee90 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ff244009900 tx=0x7ff24402a410 comp rx=0 tx=0).stop 2024-07-26T13:42:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.030+0000 7ff25198e640 1 -- 172.21.15.100:0/309515760 shutdown_connections 2024-07-26T13:42:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.030+0000 7ff25198e640 1 --2- 172.21.15.100:0/309515760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff24c108660 0x7ff24c10ee90 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.030+0000 7ff25198e640 1 --2- 172.21.15.100:0/309515760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff24c107d10 0x7ff24c108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.030+0000 7ff25198e640 1 --2- 172.21.15.100:0/309515760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff24c1023e0 0x7ff24c1027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.030+0000 7ff25198e640 1 -- 172.21.15.100:0/309515760 >> 172.21.15.100:0/309515760 conn(0x7ff24c07a170 msgr2=0x7ff24c07a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:56.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.030+0000 7ff25198e640 1 -- 172.21.15.100:0/309515760 shutdown_connections 2024-07-26T13:42:56.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.030+0000 7ff25198e640 1 -- 172.21.15.100:0/309515760 wait complete. 2024-07-26T13:42:56.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.030+0000 7ff25198e640 1 Processor -- start 2024-07-26T13:42:56.034 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.031+0000 7ff25198e640 1 -- start start 2024-07-26T13:42:56.034 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.031+0000 7ff25198e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff24c1023e0 0x7ff24c198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:56.034 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.031+0000 7ff25198e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff24c107d10 0x7ff24c1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:56.034 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.031+0000 7ff25198e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff24c108660 0x7ff24c19e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:56.034 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.031+0000 7ff25198e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c112330 con 0x7ff24c107d10 2024-07-26T13:42:56.034 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.031+0000 7ff25198e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c1121b0 con 0x7ff24c108660 2024-07-26T13:42:56.034 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.031+0000 7ff25198e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c1124b0 con 0x7ff24c1023e0 2024-07-26T13:42:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.031+0000 7ff24a7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff24c107d10 0x7ff24c1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.031+0000 7ff24a7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff24c107d10 0x7ff24c1992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60222/0 (socket says 172.21.15.100:60222) 2024-07-26T13:42:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.031+0000 7ff24a7fc640 1 -- 172.21.15.100:0/3776978518 learned_addr learned my addr 172.21.15.100:0/3776978518 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.031+0000 7ff24b7fe640 1 --2- 172.21.15.100:0/3776978518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff24c108660 0x7ff24c19e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.031+0000 7ff24affd640 1 --2- 172.21.15.100:0/3776978518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff24c1023e0 0x7ff24c198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.031+0000 7ff24a7fc640 1 -- 172.21.15.100:0/3776978518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff24c1023e0 msgr2=0x7ff24c198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.031+0000 7ff24a7fc640 1 --2- 172.21.15.100:0/3776978518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff24c1023e0 0x7ff24c198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.031+0000 7ff24a7fc640 1 -- 172.21.15.100:0/3776978518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff24c108660 msgr2=0x7ff24c19e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.031+0000 7ff24a7fc640 1 --2- 172.21.15.100:0/3776978518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff24c108660 0x7ff24c19e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.031+0000 7ff24a7fc640 1 -- 172.21.15.100:0/3776978518 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff24c19ec60 con 0x7ff24c107d10 2024-07-26T13:42:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.032+0000 7ff24a7fc640 1 --2- 172.21.15.100:0/3776978518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff24c107d10 0x7ff24c1992d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff23400d950 tx=0x7ff23400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.032+0000 7ff25098c640 1 -- 172.21.15.100:0/3776978518 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff234017070 con 0x7ff24c107d10 2024-07-26T13:42:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.032+0000 7ff25198e640 1 -- 172.21.15.100:0/3776978518 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff24c19ef50 con 0x7ff24c107d10 2024-07-26T13:42:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.032+0000 7ff25098c640 1 -- 172.21.15.100:0/3776978518 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff234009c30 con 0x7ff24c107d10 2024-07-26T13:42:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.032+0000 7ff25098c640 1 -- 172.21.15.100:0/3776978518 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2340047a0 con 0x7ff24c107d10 2024-07-26T13:42:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.032+0000 7ff25198e640 1 -- 172.21.15.100:0/3776978518 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff24c1a56e0 con 0x7ff24c107d10 2024-07-26T13:42:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.034+0000 7ff258989740 1 Processor -- start 2024-07-26T13:42:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.035+0000 7ff258989740 1 -- start start 2024-07-26T13:42:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.035+0000 7ff25098c640 1 -- 172.21.15.100:0/3776978518 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7ff234004020 con 0x7ff24c107d10 2024-07-26T13:42:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.035+0000 7ff258989740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2c2e2a1b0 0x55d2c31cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.035+0000 7ff258989740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2c2e012f0 0x55d2c31ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.035+0000 7ff258989740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2c2e094d0 0x55d2c31cd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.035+0000 7ff258989740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55d2c31423b0 con 0x55d2c2e2a1b0 2024-07-26T13:42:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.035+0000 7ff258989740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d2c31318b0 con 0x55d2c2e012f0 2024-07-26T13:42:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.035+0000 7ff258989740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d2c31358f0 con 0x55d2c2e094d0 2024-07-26T13:42:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.035+0000 7ff24b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2c2e2a1b0 0x55d2c31cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.035+0000 7ff24b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2c2e2a1b0 0x55d2c31cc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60232/0 (socket says 172.21.15.100:60232) 2024-07-26T13:42:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.035+0000 7ff24b7fe640 1 -- 172.21.15.100:0/3371592569 learned_addr learned my addr 172.21.15.100:0/3371592569 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.035+0000 7ff24affd640 1 --2- 172.21.15.100:0/3371592569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2c2e012f0 0x55d2c31ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.035+0000 7ff24a7fc640 1 --2- 172.21.15.100:0/3371592569 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2c2e094d0 0x55d2c31cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.035+0000 7ff24b7fe640 1 -- 172.21.15.100:0/3371592569 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2c2e094d0 msgr2=0x55d2c31cd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.035+0000 7ff24b7fe640 1 --2- 172.21.15.100:0/3371592569 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2c2e094d0 0x55d2c31cd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.035+0000 7ff24b7fe640 1 -- 172.21.15.100:0/3371592569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2c2e012f0 msgr2=0x55d2c31ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.035+0000 7ff24b7fe640 1 --2- 172.21.15.100:0/3371592569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2c2e012f0 0x55d2c31ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.035+0000 7ff24b7fe640 1 -- 172.21.15.100:0/3371592569 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2c30d89a0 con 0x55d2c2e2a1b0 2024-07-26T13:42:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.035+0000 7ff24a7fc640 1 --2- 172.21.15.100:0/3371592569 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2c2e094d0 0x55d2c31cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:42:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.035+0000 7ff24affd640 1 --2- 172.21.15.100:0/3371592569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2c2e012f0 0x55d2c31ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:42:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.035+0000 7ff25098c640 1 --2- 172.21.15.100:0/3776978518 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff2380b3d40 0x7ff2380b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.036+0000 7ff24b7fe640 1 --2- 172.21.15.100:0/3371592569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2c2e2a1b0 0x55d2c31cc6b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ff244009900 tx=0x7ff244032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb657747978548b server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.036+0000 7ff24affd640 1 --2- 172.21.15.100:0/3776978518 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff2380b3d40 0x7ff2380b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.036+0000 7ff25098c640 1 -- 172.21.15.100:0/3776978518 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7ff2340e3bd0 con 0x7ff24c107d10 2024-07-26T13:42:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.036+0000 7ff241ffb640 1 -- 172.21.15.100:0/3371592569 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff244040070 con 0x55d2c2e2a1b0 2024-07-26T13:42:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.036+0000 7ff241ffb640 1 -- 172.21.15.100:0/3371592569 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff244032e50 con 0x55d2c2e2a1b0 2024-07-26T13:42:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.036+0000 7ff241ffb640 1 -- 172.21.15.100:0/3371592569 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff244035040 con 0x55d2c2e2a1b0 2024-07-26T13:42:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.037+0000 7ff258989740 1 -- 172.21.15.100:0/3371592569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2c2e2a1b0 msgr2=0x55d2c31cc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.037+0000 7ff24affd640 1 --2- 172.21.15.100:0/3776978518 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff2380b3d40 0x7ff2380b6250 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55d2c31cb330 tx=0x7ff23c009680 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.037+0000 7ff258989740 1 --2- 172.21.15.100:0/3371592569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2c2e2a1b0 0x55d2c31cc6b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ff244009900 tx=0x7ff244032c60 comp rx=0 tx=0).stop 2024-07-26T13:42:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.037+0000 7ff258989740 1 -- 172.21.15.100:0/3371592569 shutdown_connections 2024-07-26T13:42:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.037+0000 7ff258989740 1 --2- 172.21.15.100:0/3371592569 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2c2e094d0 0x55d2c31cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.037+0000 7ff258989740 1 --2- 172.21.15.100:0/3371592569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2c2e012f0 0x55d2c31ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.037+0000 7ff258989740 1 --2- 172.21.15.100:0/3371592569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2c2e2a1b0 0x55d2c31cc6b0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.037+0000 7ff258989740 1 -- 172.21.15.100:0/3371592569 >> 172.21.15.100:0/3371592569 conn(0x55d2c2e1e300 msgr2=0x55d2c31ca5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.037+0000 7ff258989740 1 -- 172.21.15.100:0/3371592569 shutdown_connections 2024-07-26T13:42:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.037+0000 7ff258989740 1 -- 172.21.15.100:0/3371592569 wait complete. 2024-07-26T13:42:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.037+0000 7ff258989740 1 -- start start 2024-07-26T13:42:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.037+0000 7ff258989740 1 Processor -- start 2024-07-26T13:42:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.038+0000 7ff258989740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2c2e012f0 0x55d2c31da410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.038+0000 7ff258989740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2c2e094d0 0x55d2c31da950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.038+0000 7ff258989740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2c2e2a1b0 0x55d2c31e6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.038+0000 7ff258989740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55d2c306bab0 con 0x55d2c2e094d0 2024-07-26T13:42:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.038+0000 7ff258989740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d2c30b9cf0 con 0x55d2c2e2a1b0 2024-07-26T13:42:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.038+0000 7ff258989740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d2c31dcf90 con 0x55d2c2e012f0 2024-07-26T13:42:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.038+0000 7ff258989740 10 client.0 _ll_register_callbacks cb 0x55d2c312b3e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:42:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.038+0000 7ff24a7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2c2e094d0 0x55d2c31da950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.038+0000 7ff24a7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2c2e094d0 0x55d2c31da950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60238/0 (socket says 172.21.15.100:60238) 2024-07-26T13:42:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.038+0000 7ff24a7fc640 1 -- 172.21.15.100:0/2309715230 learned_addr learned my addr 172.21.15.100:0/2309715230 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.038+0000 7ff24affd640 1 --2- 172.21.15.100:0/2309715230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2c2e012f0 0x55d2c31da410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.038+0000 7ff24a7fc640 1 -- 172.21.15.100:0/2309715230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2c2e012f0 msgr2=0x55d2c31da410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.038+0000 7ff24a7fc640 1 --2- 172.21.15.100:0/2309715230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2c2e012f0 0x55d2c31da410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.038+0000 7ff24a7fc640 1 -- 172.21.15.100:0/2309715230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2c2e2a1b0 msgr2=0x55d2c31e6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.038+0000 7ff24a7fc640 1 --2- 172.21.15.100:0/2309715230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2c2e2a1b0 0x55d2c31e6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.038+0000 7ff24a7fc640 1 -- 172.21.15.100:0/2309715230 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2c2e95790 con 0x55d2c2e094d0 2024-07-26T13:42:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.038+0000 7ff24affd640 1 --2- 172.21.15.100:0/2309715230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2c2e012f0 0x55d2c31da410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:42:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.038+0000 7ff24a7fc640 1 --2- 172.21.15.100:0/2309715230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2c2e094d0 0x55d2c31da950 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff234005fd0 tx=0x7ff2340aa690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c176ca45b5f5c95 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.038+0000 7ff22affd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:42:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.038+0000 7ff22affd640 1 -- 172.21.15.100:0/2309715230 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2340aaf00 con 0x55d2c2e094d0 2024-07-26T13:42:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.038+0000 7ff22affd640 1 -- 172.21.15.100:0/2309715230 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7ff2340a6a90 con 0x55d2c2e094d0 2024-07-26T13:42:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.038+0000 7ff258989740 1 -- 172.21.15.100:0/2309715230 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff244032e50 con 0x55d2c2e094d0 2024-07-26T13:42:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.038+0000 7ff22affd640 1 -- 172.21.15.100:0/2309715230 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff234009de0 con 0x55d2c2e094d0 2024-07-26T13:42:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.039+0000 7ff258989740 10 client.16251 fetch_fsmap learned FSMap version 28 2024-07-26T13:42:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.039+0000 7ff258989740 1 -- 172.21.15.100:0/2309715230 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d2c2f0e620 con 0x55d2c2e094d0 2024-07-26T13:42:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.039+0000 7ff22affd640 1 -- 172.21.15.100:0/2309715230 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff244032e50 con 0x55d2c2e094d0 2024-07-26T13:42:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.039+0000 7ff22affd640 1 -- 172.21.15.100:0/2309715230 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7ff2340cd0b0 con 0x55d2c2e094d0 2024-07-26T13:42:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.039+0000 7ff258989740 10 client.16251 fetch_fsmap finished waiting for FSMap version 28 2024-07-26T13:42:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.039+0000 7ff258989740 10 client.16251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:42:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.039+0000 7ff258989740 20 client.16251 populate_metadata read hostname 'smithi100' 2024-07-26T13:42:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.039+0000 7ff258989740 1 --2- 172.21.15.100:0/2309715230 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55d2c2de8d80 0x55d2c31ec3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.039+0000 7ff258989740 4 client.16251 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:42:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.039+0000 7ff258989740 1 -- 172.21.15.100:0/2309715230 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2c3144300 con 0x55d2c2de8d80 2024-07-26T13:42:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.039+0000 7ff24affd640 1 --2- 172.21.15.100:0/2309715230 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55d2c2de8d80 0x55d2c31ec3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.040+0000 7ff22affd640 10 client.16251 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.040+0000 7ff24affd640 1 --2- 172.21.15.100:0/2309715230 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55d2c2de8d80 0x55d2c31ec3a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0696448eccca43 server_cookie=d895a9dfb1342967 in_seq=0 out_seq=0 2024-07-26T13:42:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.040+0000 7ff22affd640 1 -- 172.21.15.100:0/2309715230 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d2c3144300 con 0x55d2c2de8d80 2024-07-26T13:42:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.040+0000 7ff22affd640 10 client.16251 handle_command_reply: tid=0 2024-07-26T13:42:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.040+0000 7ff258989740 1 client.16251 shutdown 2024-07-26T13:42:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.040+0000 7ff258989740 1 -- 172.21.15.100:0/2309715230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2c2e094d0 msgr2=0x55d2c31da950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.040+0000 7ff258989740 1 --2- 172.21.15.100:0/2309715230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2c2e094d0 0x55d2c31da950 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff234005fd0 tx=0x7ff2340aa690 comp rx=0 tx=0).stop 2024-07-26T13:42:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.041+0000 7ff258989740 1 -- 172.21.15.100:0/2309715230 shutdown_connections 2024-07-26T13:42:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.041+0000 7ff258989740 1 --2- 172.21.15.100:0/2309715230 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55d2c2de8d80 0x55d2c31ec3a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.041+0000 7ff258989740 1 --2- 172.21.15.100:0/2309715230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2c2e2a1b0 0x55d2c31e6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.041+0000 7ff258989740 1 --2- 172.21.15.100:0/2309715230 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2c2e094d0 0x55d2c31da950 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.041+0000 7ff258989740 1 --2- 172.21.15.100:0/2309715230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2c2e012f0 0x55d2c31da410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:56.051 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.041+0000 7ff22affd640 0 client.16251 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:56.051 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.041+0000 7ff258989740 1 -- 172.21.15.100:0/2309715230 >> 172.21.15.100:0/2309715230 conn(0x55d2c2e1e300 msgr2=0x55d2c31c8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:56.051 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.041+0000 7ff258989740 1 -- 172.21.15.100:0/2309715230 shutdown_connections 2024-07-26T13:42:56.051 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.041+0000 7ff258989740 1 -- 172.21.15.100:0/2309715230 wait complete. 2024-07-26T13:42:56.051 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.041+0000 7ff258989740 20 client.16251 trim_cache size 0 max 16384 2024-07-26T13:42:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.056+0000 7ff258989740 1 Processor -- start 2024-07-26T13:42:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.056+0000 7ff258989740 1 -- start start 2024-07-26T13:42:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.056+0000 7ff258989740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2c2e012f0 0x55d2c31ca530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.056+0000 7ff258989740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2c2e094d0 0x55d2c31cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.056+0000 7ff258989740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2c2e2a1b0 0x55d2c31d9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.056+0000 7ff258989740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c1af7b0 con 0x55d2c2e094d0 2024-07-26T13:42:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.056+0000 7ff258989740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c1ad050 con 0x55d2c2e2a1b0 2024-07-26T13:42:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.056+0000 7ff258989740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c1aaea0 con 0x55d2c2e012f0 2024-07-26T13:42:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.056+0000 7ff24a7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2c2e094d0 0x55d2c31cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.056+0000 7ff24a7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2c2e094d0 0x55d2c31cddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60246/0 (socket says 172.21.15.100:60246) 2024-07-26T13:42:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.056+0000 7ff24a7fc640 1 -- 172.21.15.100:0/544790879 learned_addr learned my addr 172.21.15.100:0/544790879 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.056+0000 7ff24affd640 1 --2- 172.21.15.100:0/544790879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2c2e012f0 0x55d2c31ca530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.056+0000 7ff24a7fc640 1 -- 172.21.15.100:0/544790879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2c2e012f0 msgr2=0x55d2c31ca530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.056+0000 7ff24a7fc640 1 --2- 172.21.15.100:0/544790879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2c2e012f0 0x55d2c31ca530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.056+0000 7ff24a7fc640 1 -- 172.21.15.100:0/544790879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2c2e2a1b0 msgr2=0x55d2c31d9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.056+0000 7ff24a7fc640 1 --2- 172.21.15.100:0/544790879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2c2e2a1b0 0x55d2c31d9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.056+0000 7ff24a7fc640 1 -- 172.21.15.100:0/544790879 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2c31cd490 con 0x55d2c2e094d0 2024-07-26T13:42:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.056+0000 7ff24affd640 1 --2- 172.21.15.100:0/544790879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2c2e012f0 0x55d2c31ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:42:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.056+0000 7ff24a7fc640 1 --2- 172.21.15.100:0/544790879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2c2e094d0 0x55d2c31cddf0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ff23400bfd0 tx=0x7ff23400b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b0754a69075ad5 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.056+0000 7ff2417fa640 1 -- 172.21.15.100:0/544790879 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2340aaf00 con 0x55d2c2e094d0 2024-07-26T13:42:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.057+0000 7ff2417fa640 1 -- 172.21.15.100:0/544790879 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2340b38b0 con 0x55d2c2e094d0 2024-07-26T13:42:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.057+0000 7ff2417fa640 1 -- 172.21.15.100:0/544790879 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2340a9a60 con 0x55d2c2e094d0 2024-07-26T13:42:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.057+0000 7ff258989740 1 -- 172.21.15.100:0/544790879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2c2e094d0 msgr2=0x55d2c31cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.057+0000 7ff258989740 1 --2- 172.21.15.100:0/544790879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2c2e094d0 0x55d2c31cddf0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ff23400bfd0 tx=0x7ff23400b660 comp rx=0 tx=0).stop 2024-07-26T13:42:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.057+0000 7ff258989740 1 -- 172.21.15.100:0/544790879 shutdown_connections 2024-07-26T13:42:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.057+0000 7ff258989740 1 --2- 172.21.15.100:0/544790879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2c2e2a1b0 0x55d2c31d9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.057+0000 7ff258989740 1 --2- 172.21.15.100:0/544790879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2c2e094d0 0x55d2c31cddf0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.057+0000 7ff258989740 1 --2- 172.21.15.100:0/544790879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2c2e012f0 0x55d2c31ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.057+0000 7ff258989740 1 -- 172.21.15.100:0/544790879 >> 172.21.15.100:0/544790879 conn(0x55d2c2e1e300 msgr2=0x55d2c31cd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.057+0000 7ff258989740 1 -- 172.21.15.100:0/544790879 shutdown_connections 2024-07-26T13:42:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.057+0000 7ff258989740 1 -- 172.21.15.100:0/544790879 wait complete. 2024-07-26T13:42:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.058+0000 7ff258989740 1 -- start start 2024-07-26T13:42:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.058+0000 7ff258989740 1 Processor -- start 2024-07-26T13:42:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.058+0000 7ff258989740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2c2e012f0 0x55d2c31e2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.058+0000 7ff258989740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2c2e094d0 0x55d2c31e0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.058+0000 7ff258989740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2c2e2a1b0 0x55d2c31e0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.058+0000 7ff258989740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c1ab510 con 0x55d2c2e2a1b0 2024-07-26T13:42:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.058+0000 7ff258989740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c1ac980 con 0x55d2c2e012f0 2024-07-26T13:42:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.058+0000 7ff258989740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c1abbe0 con 0x55d2c2e094d0 2024-07-26T13:42:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.058+0000 7ff258989740 10 client.0 _ll_register_callbacks cb 0x7ff234005780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:42:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.058+0000 7ff24b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2c2e2a1b0 0x55d2c31e0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.058+0000 7ff24b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2c2e2a1b0 0x55d2c31e0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:60250/0 (socket says 172.21.15.100:60250) 2024-07-26T13:42:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.058+0000 7ff24b7fe640 1 -- 172.21.15.100:0/4032295152 learned_addr learned my addr 172.21.15.100:0/4032295152 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:42:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.058+0000 7ff24a7fc640 1 --2- 172.21.15.100:0/4032295152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2c2e094d0 0x55d2c31e0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.058+0000 7ff24affd640 1 --2- 172.21.15.100:0/4032295152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2c2e012f0 0x55d2c31e2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.058+0000 7ff24b7fe640 1 -- 172.21.15.100:0/4032295152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2c2e094d0 msgr2=0x55d2c31e0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.058+0000 7ff24b7fe640 1 --2- 172.21.15.100:0/4032295152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2c2e094d0 0x55d2c31e0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.058+0000 7ff24b7fe640 1 -- 172.21.15.100:0/4032295152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2c2e012f0 msgr2=0x55d2c31e2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:42:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.058+0000 7ff24b7fe640 1 --2- 172.21.15.100:0/4032295152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2c2e012f0 0x55d2c31e2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.058+0000 7ff24b7fe640 1 -- 172.21.15.100:0/4032295152 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2c31e1250 con 0x55d2c2e2a1b0 2024-07-26T13:42:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.059+0000 7ff24b7fe640 1 --2- 172.21.15.100:0/4032295152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2c2e2a1b0 0x55d2c31e0ac0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ff244006010 tx=0x7ff2440058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb8f665f38fd7c9 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:42:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.059+0000 7ff22affd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:42:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.059+0000 7ff22affd640 1 -- 172.21.15.100:0/4032295152 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff244040070 con 0x55d2c2e2a1b0 2024-07-26T13:42:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.059+0000 7ff22affd640 1 -- 172.21.15.100:0/4032295152 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8484+0+0 (secure 0 0 0) 0x7ff2440318b0 con 0x55d2c2e2a1b0 2024-07-26T13:42:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.059+0000 7ff258989740 1 -- 172.21.15.100:0/4032295152 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2340b38b0 con 0x55d2c2e2a1b0 2024-07-26T13:42:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.059+0000 7ff22affd640 1 -- 172.21.15.100:0/4032295152 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff244035040 con 0x55d2c2e2a1b0 2024-07-26T13:42:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.059+0000 7ff258989740 10 client.16257 fetch_fsmap learned FSMap version 28 2024-07-26T13:42:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.059+0000 7ff258989740 1 -- 172.21.15.100:0/4032295152 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d2c3253af0 con 0x55d2c2e2a1b0 2024-07-26T13:42:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.059+0000 7ff22affd640 1 -- 172.21.15.100:0/4032295152 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2340b38b0 con 0x55d2c2e2a1b0 2024-07-26T13:42:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.059+0000 7ff22affd640 1 -- 172.21.15.100:0/4032295152 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7ff244054020 con 0x55d2c2e2a1b0 2024-07-26T13:42:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.059+0000 7ff258989740 10 client.16257 fetch_fsmap finished waiting for FSMap version 28 2024-07-26T13:42:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.059+0000 7ff258989740 10 client.16257 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:42:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.059+0000 7ff258989740 20 client.16257 populate_metadata read hostname 'smithi100' 2024-07-26T13:42:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.059+0000 7ff258989740 1 --2- 172.21.15.100:0/4032295152 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55d2c2dbe1c0 0x55d2c3255eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:42:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.059+0000 7ff258989740 4 client.16257 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-26T13:42:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.059+0000 7ff258989740 1 -- 172.21.15.100:0/4032295152 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d2c31d5d70 con 0x55d2c2dbe1c0 2024-07-26T13:42:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.060+0000 7ff24affd640 1 --2- 172.21.15.100:0/4032295152 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55d2c2dbe1c0 0x55d2c3255eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:42:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.060+0000 7ff24affd640 1 --2- 172.21.15.100:0/4032295152 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55d2c2dbe1c0 0x55d2c3255eb0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd242fe7d0fcf3fa server_cookie=c4ab9b532a19b937 in_seq=0 out_seq=0 2024-07-26T13:42:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.060+0000 7ff22affd640 10 client.16257 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.116+0000 7ff22affd640 1 -- 172.21.15.100:0/4032295152 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d2c31d5d70 con 0x55d2c2dbe1c0 2024-07-26T13:42:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.116+0000 7ff22affd640 10 client.16257 handle_command_reply: tid=0 2024-07-26T13:42:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.116+0000 7ff258989740 1 client.16257 shutdown 2024-07-26T13:42:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.116+0000 7ff258989740 1 -- 172.21.15.100:0/4032295152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2c2e2a1b0 msgr2=0x55d2c31e0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.116+0000 7ff258989740 1 --2- 172.21.15.100:0/4032295152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2c2e2a1b0 0x55d2c31e0ac0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ff244006010 tx=0x7ff2440058e0 comp rx=0 tx=0).stop 2024-07-26T13:42:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.116+0000 7ff258989740 1 -- 172.21.15.100:0/4032295152 shutdown_connections 2024-07-26T13:42:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.116+0000 7ff258989740 1 --2- 172.21.15.100:0/4032295152 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55d2c2dbe1c0 0x55d2c3255eb0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.117+0000 7ff258989740 1 --2- 172.21.15.100:0/4032295152 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d2c2e2a1b0 0x55d2c31e0ac0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.117+0000 7ff258989740 1 --2- 172.21.15.100:0/4032295152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2c2e094d0 0x55d2c31e0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.117+0000 7ff258989740 1 --2- 172.21.15.100:0/4032295152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2c2e012f0 0x55d2c31e2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.117+0000 7ff22affd640 0 client.16257 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:42:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.117+0000 7ff258989740 1 -- 172.21.15.100:0/4032295152 >> 172.21.15.100:0/4032295152 conn(0x55d2c2e1e300 msgr2=0x55d2c31ca300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.117+0000 7ff258989740 1 -- 172.21.15.100:0/4032295152 shutdown_connections 2024-07-26T13:42:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.117+0000 7ff258989740 1 -- 172.21.15.100:0/4032295152 wait complete. 2024-07-26T13:42:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.117+0000 7ff258989740 20 client.16257 trim_cache size 0 max 16384 2024-07-26T13:42:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-07-26T13:42:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-07-26T13:42:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-07-26T13:42:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-07-26T13:42:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.117+0000 7ff25198e640 1 -- 172.21.15.100:0/3776978518 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff2380b3d40 msgr2=0x7ff2380b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.117+0000 7ff25198e640 1 --2- 172.21.15.100:0/3776978518 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff2380b3d40 0x7ff2380b6250 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55d2c31cb330 tx=0x7ff23c009680 comp rx=0 tx=0).stop 2024-07-26T13:42:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.117+0000 7ff25198e640 1 -- 172.21.15.100:0/3776978518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff24c107d10 msgr2=0x7ff24c1992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:42:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.117+0000 7ff25198e640 1 --2- 172.21.15.100:0/3776978518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff24c107d10 0x7ff24c1992d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff23400d950 tx=0x7ff23400de10 comp rx=0 tx=0).stop 2024-07-26T13:42:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.118+0000 7ff25198e640 1 -- 172.21.15.100:0/3776978518 shutdown_connections 2024-07-26T13:42:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.118+0000 7ff25198e640 1 --2- 172.21.15.100:0/3776978518 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff2380b3d40 0x7ff2380b6250 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.118+0000 7ff25198e640 1 --2- 172.21.15.100:0/3776978518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff24c108660 0x7ff24c19e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.118+0000 7ff25198e640 1 --2- 172.21.15.100:0/3776978518 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff24c107d10 0x7ff24c1992d0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.118+0000 7ff25198e640 1 --2- 172.21.15.100:0/3776978518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff24c1023e0 0x7ff24c198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:42:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.118+0000 7ff25198e640 1 -- 172.21.15.100:0/3776978518 >> 172.21.15.100:0/3776978518 conn(0x7ff24c07a170 msgr2=0x7ff24c103d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:42:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.118+0000 7ff25198e640 1 -- 172.21.15.100:0/3776978518 shutdown_connections 2024-07-26T13:42:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:42:56.118+0000 7ff25198e640 1 -- 172.21.15.100:0/3776978518 wait complete. 2024-07-26T13:42:56.406 INFO:tasks.workunit.client.0.smithi100.stdout: 1 82357 6.59 MB/sec warmup 70 sec latency 596.793 ms 2024-07-26T13:42:56.523 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: ceph-mds: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-5320-ge8f88bb1/rpm/el9/BUILD/ceph-19.0.0-5320-ge8f88bb1/redhat-linux-build/boost/include/boost/smart_ptr/intrusive_ptr.hpp:201: T* boost::intrusive_ptr::operator->() const [with T = const Message]: Assertion `px != 0' failed. 2024-07-26T13:42:56.523 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: *** Caught signal (Aborted) ** 2024-07-26T13:42:56.523 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: in thread 7ff7bdead640 thread_name:ms_dispatch 2024-07-26T13:42:56.523 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: ceph version 19.0.0-5320-ge8f88bb1 (e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528) squid (dev) 2024-07-26T13:42:56.523 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 1: /lib64/libc.so.6(+0x3e6f0) [0x7ff7c32846f0] 2024-07-26T13:42:56.523 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 2: /lib64/libc.so.6(+0x8b94c) [0x7ff7c32d194c] 2024-07-26T13:42:56.523 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 3: raise() 2024-07-26T13:42:56.523 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 4: abort() 2024-07-26T13:42:56.524 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 5: /lib64/libc.so.6(+0x2871b) [0x7ff7c326e71b] 2024-07-26T13:42:56.524 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 6: /lib64/libc.so.6(+0x37386) [0x7ff7c327d386] 2024-07-26T13:42:56.524 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 7: (MDSRank::get_session(boost::intrusive_ptr const&)+0x89) [0x55d78948d6b9] 2024-07-26T13:42:56.524 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 8: (MDCache::path_traverse(boost::intrusive_ptr const&, ContextFactory&, filepath const&, int, std::vector >*, CInode**, CDir**)+0x58e) [0x55d78964b80c] 2024-07-26T13:42:56.524 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 9: (Server::handle_peer_rename_prep(boost::intrusive_ptr const&)+0x4fe) [0x55d789599cde] 2024-07-26T13:42:56.524 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 10: (Server::dispatch_peer_request(boost::intrusive_ptr const&)+0x933) [0x55d78955b763] 2024-07-26T13:42:56.524 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 11: (Server::handle_peer_request(boost::intrusive_ptr const&)+0x10c3) [0x55d78955ca1b] 2024-07-26T13:42:56.524 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x9ba) [0x55d789540752] 2024-07-26T13:42:56.524 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 13: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x81a) [0x55d78948ea56] 2024-07-26T13:42:56.524 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 14: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55d789492637] 2024-07-26T13:42:56.524 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 15: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55d789492f5e] 2024-07-26T13:42:56.524 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 16: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55d78947603a] 2024-07-26T13:42:56.525 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 17: /usr/lib64/ceph/libceph-common.so.2(+0x39b407) [0x7ff7c3b51407] 2024-07-26T13:42:56.525 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 18: (DispatchQueue::entry()+0x60f) [0x7ff7c3b51cf9] 2024-07-26T13:42:56.525 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 19: /usr/lib64/ceph/libceph-common.so.2(+0x4785a1) [0x7ff7c3c2e5a1] 2024-07-26T13:42:56.525 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 20: (Thread::entry_wrapper()+0x43) [0x7ff7c39dbcdd] 2024-07-26T13:42:56.526 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 21: (Thread::_entry_func(void*)+0xd) [0x7ff7c39dbcf9] 2024-07-26T13:42:56.526 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 22: /lib64/libc.so.6(+0x89c02) [0x7ff7c32cfc02] 2024-07-26T13:42:56.526 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 23: /lib64/libc.so.6(+0x10ec40) [0x7ff7c3354c40] 2024-07-26T13:42:56.526 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 2024-07-26T13:42:56.116+0000 7ff7bdead640 -1 *** Caught signal (Aborted) ** 2024-07-26T13:42:56.526 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: in thread 7ff7bdead640 thread_name:ms_dispatch 2024-07-26T13:42:56.526 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 2024-07-26T13:42:56.526 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: ceph version 19.0.0-5320-ge8f88bb1 (e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528) squid (dev) 2024-07-26T13:42:56.526 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 1: /lib64/libc.so.6(+0x3e6f0) [0x7ff7c32846f0] 2024-07-26T13:42:56.526 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 2: /lib64/libc.so.6(+0x8b94c) [0x7ff7c32d194c] 2024-07-26T13:42:56.526 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 3: raise() 2024-07-26T13:42:56.526 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 4: abort() 2024-07-26T13:42:56.526 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 5: /lib64/libc.so.6(+0x2871b) [0x7ff7c326e71b] 2024-07-26T13:42:56.526 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 6: /lib64/libc.so.6(+0x37386) [0x7ff7c327d386] 2024-07-26T13:42:56.527 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 7: (MDSRank::get_session(boost::intrusive_ptr const&)+0x89) [0x55d78948d6b9] 2024-07-26T13:42:56.527 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 8: (MDCache::path_traverse(boost::intrusive_ptr const&, ContextFactory&, filepath const&, int, std::vector >*, CInode**, CDir**)+0x58e) [0x55d78964b80c] 2024-07-26T13:42:56.527 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 9: (Server::handle_peer_rename_prep(boost::intrusive_ptr const&)+0x4fe) [0x55d789599cde] 2024-07-26T13:42:56.527 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 10: (Server::dispatch_peer_request(boost::intrusive_ptr const&)+0x933) [0x55d78955b763] 2024-07-26T13:42:56.527 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 11: (Server::handle_peer_request(boost::intrusive_ptr const&)+0x10c3) [0x55d78955ca1b] 2024-07-26T13:42:56.527 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x9ba) [0x55d789540752] 2024-07-26T13:42:56.527 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 13: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x81a) [0x55d78948ea56] 2024-07-26T13:42:56.527 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 14: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55d789492637] 2024-07-26T13:42:56.527 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 15: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55d789492f5e] 2024-07-26T13:42:56.527 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 16: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55d78947603a] 2024-07-26T13:42:56.527 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 17: /usr/lib64/ceph/libceph-common.so.2(+0x39b407) [0x7ff7c3b51407] 2024-07-26T13:42:56.527 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 18: (DispatchQueue::entry()+0x60f) [0x7ff7c3b51cf9] 2024-07-26T13:42:56.528 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 19: /usr/lib64/ceph/libceph-common.so.2(+0x4785a1) [0x7ff7c3c2e5a1] 2024-07-26T13:42:56.528 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 20: (Thread::entry_wrapper()+0x43) [0x7ff7c39dbcdd] 2024-07-26T13:42:56.528 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 21: (Thread::_entry_func(void*)+0xd) [0x7ff7c39dbcf9] 2024-07-26T13:42:56.528 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 22: /lib64/libc.so.6(+0x89c02) [0x7ff7c32cfc02] 2024-07-26T13:42:56.528 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 23: /lib64/libc.so.6(+0x10ec40) [0x7ff7c3354c40] 2024-07-26T13:42:56.528 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-07-26T13:42:56.528 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 2024-07-26T13:42:56.528 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 0> 2024-07-26T13:42:56.116+0000 7ff7bdead640 -1 *** Caught signal (Aborted) ** 2024-07-26T13:42:56.528 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: in thread 7ff7bdead640 thread_name:ms_dispatch 2024-07-26T13:42:56.528 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 2024-07-26T13:42:56.528 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: ceph version 19.0.0-5320-ge8f88bb1 (e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528) squid (dev) 2024-07-26T13:42:56.528 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 1: /lib64/libc.so.6(+0x3e6f0) [0x7ff7c32846f0] 2024-07-26T13:42:56.528 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 2: /lib64/libc.so.6(+0x8b94c) [0x7ff7c32d194c] 2024-07-26T13:42:56.528 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 3: raise() 2024-07-26T13:42:56.529 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 4: abort() 2024-07-26T13:42:56.529 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 5: /lib64/libc.so.6(+0x2871b) [0x7ff7c326e71b] 2024-07-26T13:42:56.529 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 6: /lib64/libc.so.6(+0x37386) [0x7ff7c327d386] 2024-07-26T13:42:56.529 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 7: (MDSRank::get_session(boost::intrusive_ptr const&)+0x89) [0x55d78948d6b9] 2024-07-26T13:42:56.529 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 8: (MDCache::path_traverse(boost::intrusive_ptr const&, ContextFactory&, filepath const&, int, std::vector >*, CInode**, CDir**)+0x58e) [0x55d78964b80c] 2024-07-26T13:42:56.529 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 9: (Server::handle_peer_rename_prep(boost::intrusive_ptr const&)+0x4fe) [0x55d789599cde] 2024-07-26T13:42:56.529 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 10: (Server::dispatch_peer_request(boost::intrusive_ptr const&)+0x933) [0x55d78955b763] 2024-07-26T13:42:56.529 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 11: (Server::handle_peer_request(boost::intrusive_ptr const&)+0x10c3) [0x55d78955ca1b] 2024-07-26T13:42:56.529 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x9ba) [0x55d789540752] 2024-07-26T13:42:56.529 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 13: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x81a) [0x55d78948ea56] 2024-07-26T13:42:56.529 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 14: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55d789492637] 2024-07-26T13:42:56.529 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 15: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55d789492f5e] 2024-07-26T13:42:56.529 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 16: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55d78947603a] 2024-07-26T13:42:56.529 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 17: /usr/lib64/ceph/libceph-common.so.2(+0x39b407) [0x7ff7c3b51407] 2024-07-26T13:42:56.530 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 18: (DispatchQueue::entry()+0x60f) [0x7ff7c3b51cf9] 2024-07-26T13:42:56.530 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 19: /usr/lib64/ceph/libceph-common.so.2(+0x4785a1) [0x7ff7c3c2e5a1] 2024-07-26T13:42:56.530 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 20: (Thread::entry_wrapper()+0x43) [0x7ff7c39dbcdd] 2024-07-26T13:42:56.530 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 21: (Thread::_entry_func(void*)+0xd) [0x7ff7c39dbcf9] 2024-07-26T13:42:56.530 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 22: /lib64/libc.so.6(+0x89c02) [0x7ff7c32cfc02] 2024-07-26T13:42:56.530 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 23: /lib64/libc.so.6(+0x10ec40) [0x7ff7c3354c40] 2024-07-26T13:42:56.530 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-07-26T13:42:56.530 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 2024-07-26T13:42:56.530 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 0> 2024-07-26T13:42:56.116+0000 7ff7bdead640 -1 *** Caught signal (Aborted) ** 2024-07-26T13:42:56.530 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: in thread 7ff7bdead640 thread_name:ms_dispatch 2024-07-26T13:42:56.530 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 2024-07-26T13:42:56.530 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: ceph version 19.0.0-5320-ge8f88bb1 (e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528) squid (dev) 2024-07-26T13:42:56.530 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 1: /lib64/libc.so.6(+0x3e6f0) [0x7ff7c32846f0] 2024-07-26T13:42:56.531 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 2: /lib64/libc.so.6(+0x8b94c) [0x7ff7c32d194c] 2024-07-26T13:42:56.531 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 3: raise() 2024-07-26T13:42:56.531 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 4: abort() 2024-07-26T13:42:56.531 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 5: /lib64/libc.so.6(+0x2871b) [0x7ff7c326e71b] 2024-07-26T13:42:56.531 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 6: /lib64/libc.so.6(+0x37386) [0x7ff7c327d386] 2024-07-26T13:42:56.531 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 7: (MDSRank::get_session(boost::intrusive_ptr const&)+0x89) [0x55d78948d6b9] 2024-07-26T13:42:56.531 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 8: (MDCache::path_traverse(boost::intrusive_ptr const&, ContextFactory&, filepath const&, int, std::vector >*, CInode**, CDir**)+0x58e) [0x55d78964b80c] 2024-07-26T13:42:56.531 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 9: (Server::handle_peer_rename_prep(boost::intrusive_ptr const&)+0x4fe) [0x55d789599cde] 2024-07-26T13:42:56.531 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 10: (Server::dispatch_peer_request(boost::intrusive_ptr const&)+0x933) [0x55d78955b763] 2024-07-26T13:42:56.531 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 11: (Server::handle_peer_request(boost::intrusive_ptr const&)+0x10c3) [0x55d78955ca1b] 2024-07-26T13:42:56.531 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x9ba) [0x55d789540752] 2024-07-26T13:42:56.531 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 13: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x81a) [0x55d78948ea56] 2024-07-26T13:42:56.531 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 14: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55d789492637] 2024-07-26T13:42:56.531 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 15: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55d789492f5e] 2024-07-26T13:42:56.532 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 16: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55d78947603a] 2024-07-26T13:42:56.532 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 17: /usr/lib64/ceph/libceph-common.so.2(+0x39b407) [0x7ff7c3b51407] 2024-07-26T13:42:56.532 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 18: (DispatchQueue::entry()+0x60f) [0x7ff7c3b51cf9] 2024-07-26T13:42:56.532 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 19: /usr/lib64/ceph/libceph-common.so.2(+0x4785a1) [0x7ff7c3c2e5a1] 2024-07-26T13:42:56.532 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 20: (Thread::entry_wrapper()+0x43) [0x7ff7c39dbcdd] 2024-07-26T13:42:56.532 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 21: (Thread::_entry_func(void*)+0xd) [0x7ff7c39dbcf9] 2024-07-26T13:42:56.532 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 22: /lib64/libc.so.6(+0x89c02) [0x7ff7c32cfc02] 2024-07-26T13:42:56.532 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 23: /lib64/libc.so.6(+0x10ec40) [0x7ff7c3354c40] 2024-07-26T13:42:56.532 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-07-26T13:42:56.532 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[69031]: 2024-07-26T13:42:56.532 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:56 smithi167 podman[71677]: 2024-07-26 13:42:56.173333928 +0000 UTC m=+0.019109613 container died 6876119001b6632c2578cacbbb590f29bc2c03a736c801fe755d40bdbf818d0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb) 2024-07-26T13:42:57.406 INFO:tasks.workunit.client.0.smithi100.stdout: 1 82357 6.49 MB/sec warmup 71 sec latency 1596.855 ms 2024-07-26T13:42:57.862 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:57 smithi167 ceph-mon[37140]: pgmap v449: 97 pgs: 97 active+clean; 50 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.9 MiB/s wr, 269 op/s 2024-07-26T13:42:57.862 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:57 smithi167 podman[71677]: 2024-07-26 13:42:57.616775549 +0000 UTC m=+1.462551234 container remove 6876119001b6632c2578cacbbb590f29bc2c03a736c801fe755d40bdbf818d0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, org.label-schema.build-date=20240724, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-07-26T13:42:57.862 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:57 smithi167 systemd[1]: ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.c.service: Main process exited, code=exited, status=134/n/a 2024-07-26T13:42:57.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:57 smithi138 ceph-mon[36954]: pgmap v449: 97 pgs: 97 active+clean; 50 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.9 MiB/s wr, 269 op/s 2024-07-26T13:42:57.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:57 smithi100 ceph-mon[32599]: pgmap v449: 97 pgs: 97 active+clean; 50 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.9 MiB/s wr, 269 op/s 2024-07-26T13:42:58.272 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:57 smithi167 systemd[1]: ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-07-26T13:42:58.272 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:42:57 smithi167 systemd[1]: ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.c.service: Consumed 12.204s CPU time. 2024-07-26T13:42:58.406 INFO:tasks.workunit.client.0.smithi100.stdout: 1 82357 6.40 MB/sec warmup 72 sec latency 2596.927 ms 2024-07-26T13:42:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:42:58 smithi138 ceph-mon[36954]: pgmap v450: 97 pgs: 97 active+clean; 51 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 290 op/s 2024-07-26T13:42:58.977 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:42:58 smithi100 ceph-mon[32599]: pgmap v450: 97 pgs: 97 active+clean; 51 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 290 op/s 2024-07-26T13:42:59.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:42:58 smithi167 ceph-mon[37140]: pgmap v450: 97 pgs: 97 active+clean; 51 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 290 op/s 2024-07-26T13:42:59.406 INFO:tasks.workunit.client.0.smithi100.stdout: 1 82357 6.32 MB/sec warmup 73 sec latency 3597.025 ms 2024-07-26T13:43:00.406 INFO:tasks.workunit.client.0.smithi100.stdout: 1 82357 6.23 MB/sec warmup 74 sec latency 4597.157 ms 2024-07-26T13:43:00.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:43:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:43:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:43:01.406 INFO:tasks.workunit.client.0.smithi100.stdout: 1 82357 6.15 MB/sec warmup 75 sec latency 5597.259 ms 2024-07-26T13:43:01.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:01 smithi167 ceph-mon[37140]: pgmap v451: 97 pgs: 97 active+clean; 44 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 295 op/s 2024-07-26T13:43:01.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:43:01.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:43:01.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:43:01.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:01 smithi138 ceph-mon[36954]: pgmap v451: 97 pgs: 97 active+clean; 44 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 295 op/s 2024-07-26T13:43:01.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:43:01.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:43:01.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:43:01.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:01 smithi100 ceph-mon[32599]: pgmap v451: 97 pgs: 97 active+clean; 44 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 295 op/s 2024-07-26T13:43:01.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:43:01.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:43:01.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:43:02.406 INFO:tasks.workunit.client.0.smithi100.stdout: 1 82357 6.07 MB/sec warmup 76 sec latency 6597.358 ms 2024-07-26T13:43:03.407 INFO:tasks.workunit.client.0.smithi100.stdout: 1 82357 5.99 MB/sec warmup 77 sec latency 7597.476 ms 2024-07-26T13:43:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:03 smithi167 ceph-mon[37140]: pgmap v452: 97 pgs: 97 active+clean; 44 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.2 MiB/s wr, 212 op/s 2024-07-26T13:43:03.795 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:03 smithi100 ceph-mon[32599]: pgmap v452: 97 pgs: 97 active+clean; 44 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.2 MiB/s wr, 212 op/s 2024-07-26T13:43:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:03 smithi138 ceph-mon[36954]: pgmap v452: 97 pgs: 97 active+clean; 44 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.2 MiB/s wr, 212 op/s 2024-07-26T13:43:04.407 INFO:tasks.workunit.client.0.smithi100.stdout: 1 82357 5.91 MB/sec warmup 78 sec latency 8597.569 ms 2024-07-26T13:43:05.407 INFO:tasks.workunit.client.0.smithi100.stdout: 1 82357 5.84 MB/sec warmup 79 sec latency 9597.609 ms 2024-07-26T13:43:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:05 smithi138 ceph-mon[36954]: pgmap v453: 97 pgs: 97 active+clean; 44 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 191 op/s 2024-07-26T13:43:05.961 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:05 smithi100 ceph-mon[32599]: pgmap v453: 97 pgs: 97 active+clean; 44 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 191 op/s 2024-07-26T13:43:06.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:05 smithi167 ceph-mon[37140]: pgmap v453: 97 pgs: 97 active+clean; 44 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 191 op/s 2024-07-26T13:43:06.407 INFO:tasks.workunit.client.0.smithi100.stdout: 1 82357 5.76 MB/sec warmup 80 sec latency 10597.682 ms 2024-07-26T13:43:07.407 INFO:tasks.workunit.client.0.smithi100.stdout: 1 82357 5.69 MB/sec warmup 81 sec latency 11597.728 ms 2024-07-26T13:43:07.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:07 smithi138 ceph-mon[36954]: pgmap v454: 97 pgs: 97 active+clean; 46 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 709 KiB/s wr, 134 op/s 2024-07-26T13:43:07.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:07 smithi100 ceph-mon[32599]: pgmap v454: 97 pgs: 97 active+clean; 46 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 709 KiB/s wr, 134 op/s 2024-07-26T13:43:08.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:07 smithi167 ceph-mon[37140]: pgmap v454: 97 pgs: 97 active+clean; 46 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 709 KiB/s wr, 134 op/s 2024-07-26T13:43:08.391 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:43:08 smithi167 systemd[1]: ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.c.service: Scheduled restart job, restart counter is at 1. 2024-07-26T13:43:08.392 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:43:08 smithi167 systemd[1]: Stopped Ceph mds.c for bf83e2e6-4b52-11ef-bcb7-c7b262605968. 2024-07-26T13:43:08.392 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:43:08 smithi167 systemd[1]: ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.c.service: Consumed 12.204s CPU time. 2024-07-26T13:43:08.392 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:43:08 smithi167 systemd[1]: Starting Ceph mds.c for bf83e2e6-4b52-11ef-bcb7-c7b262605968... 2024-07-26T13:43:08.392 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:43:08 smithi167 podman[71900]: 2024-07-26 13:43:08.159825052 +0000 UTC m=+0.017606779 image pull ae51f432d4fa82622bc253ca9e2da7666c617e096e99d43d0a0e4fdaca8865f8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2 2024-07-26T13:43:08.392 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:43:08 smithi167 podman[71900]: 2024-07-26 13:43:08.26798505 +0000 UTC m=+0.125766777 container create e0d4ef191c8d9901c511ebe9fad8969ad51ab96a95f1bcb28c96cc04af6ba9a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb) 2024-07-26T13:43:08.407 INFO:tasks.workunit.client.0.smithi100.stdout: 1 82357 5.62 MB/sec warmup 82 sec latency 12597.775 ms 2024-07-26T13:43:08.656 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:43:08 smithi167 podman[71900]: 2024-07-26 13:43:08.512666505 +0000 UTC m=+0.370448244 container init e0d4ef191c8d9901c511ebe9fad8969ad51ab96a95f1bcb28c96cc04af6ba9a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb) 2024-07-26T13:43:08.656 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:43:08 smithi167 podman[71900]: 2024-07-26 13:43:08.516058691 +0000 UTC m=+0.373840424 container start e0d4ef191c8d9901c511ebe9fad8969ad51ab96a95f1bcb28c96cc04af6ba9a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True) 2024-07-26T13:43:08.656 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:43:08 smithi167 ceph-mds[71920]: -- 172.21.15.167:0/3961282759 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555febfcd860 con 0x555ff016fa80 2024-07-26T13:43:08.656 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:43:08 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: starting mds.c at 2024-07-26T13:43:08.656 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:43:08 smithi167 bash[71900]: e0d4ef191c8d9901c511ebe9fad8969ad51ab96a95f1bcb28c96cc04af6ba9a6 2024-07-26T13:43:08.656 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:43:08 smithi167 systemd[1]: Started Ceph mds.c for bf83e2e6-4b52-11ef-bcb7-c7b262605968. 2024-07-26T13:43:08.674 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-26T13:43:08.949 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:43:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.218+0000 7ff0346e0640 1 -- 172.21.15.100:0/3074850813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff02c108690 msgr2=0x7ff02c10eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.218+0000 7ff0346e0640 1 --2- 172.21.15.100:0/3074850813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff02c108690 0x7ff02c10eec0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7ff024009ca0 tx=0x7ff02402a8c0 comp rx=0 tx=0).stop 2024-07-26T13:43:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.218+0000 7ff0346e0640 1 -- 172.21.15.100:0/3074850813 shutdown_connections 2024-07-26T13:43:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.218+0000 7ff0346e0640 1 --2- 172.21.15.100:0/3074850813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff02c108690 0x7ff02c10eec0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.218+0000 7ff0346e0640 1 --2- 172.21.15.100:0/3074850813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff02c107d40 0x7ff02c108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.218+0000 7ff0346e0640 1 --2- 172.21.15.100:0/3074850813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff02c102410 0x7ff02c102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.218+0000 7ff0346e0640 1 -- 172.21.15.100:0/3074850813 >> 172.21.15.100:0/3074850813 conn(0x7ff02c0fde10 msgr2=0x7ff02c100260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.218+0000 7ff0346e0640 1 -- 172.21.15.100:0/3074850813 shutdown_connections 2024-07-26T13:43:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.218+0000 7ff0346e0640 1 -- 172.21.15.100:0/3074850813 wait complete. 2024-07-26T13:43:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.219+0000 7ff0346e0640 1 Processor -- start 2024-07-26T13:43:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.219+0000 7ff0346e0640 1 -- start start 2024-07-26T13:43:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.219+0000 7ff0346e0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff02c102410 0x7ff02c19ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.219+0000 7ff0346e0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff02c107d40 0x7ff02c19b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.219+0000 7ff0346e0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff02c108690 0x7ff02c19b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.219+0000 7ff0346e0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c112360 con 0x7ff02c102410 2024-07-26T13:43:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.219+0000 7ff0346e0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1121e0 con 0x7ff02c108690 2024-07-26T13:43:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.219+0000 7ff0346e0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1124e0 con 0x7ff02c107d40 2024-07-26T13:43:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.219+0000 7ff03247c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff02c102410 0x7ff02c19ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.219+0000 7ff03247c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff02c102410 0x7ff02c19ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37498/0 (socket says 172.21.15.100:37498) 2024-07-26T13:43:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.219+0000 7ff03247c640 1 -- 172.21.15.100:0/2033379943 learned_addr learned my addr 172.21.15.100:0/2033379943 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.219+0000 7ff031c7b640 1 --2- 172.21.15.100:0/2033379943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff02c107d40 0x7ff02c19b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.219+0000 7ff032c7d640 1 --2- 172.21.15.100:0/2033379943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff02c108690 0x7ff02c19b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.219+0000 7ff03247c640 1 -- 172.21.15.100:0/2033379943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff02c107d40 msgr2=0x7ff02c19b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.219+0000 7ff03247c640 1 --2- 172.21.15.100:0/2033379943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff02c107d40 0x7ff02c19b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.219+0000 7ff03247c640 1 -- 172.21.15.100:0/2033379943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff02c108690 msgr2=0x7ff02c19b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.219+0000 7ff03247c640 1 --2- 172.21.15.100:0/2033379943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff02c108690 0x7ff02c19b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.219+0000 7ff03247c640 1 -- 172.21.15.100:0/2033379943 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff02c1a5f10 con 0x7ff02c102410 2024-07-26T13:43:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.220+0000 7ff03247c640 1 --2- 172.21.15.100:0/2033379943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff02c102410 0x7ff02c19ac50 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff01c00d950 tx=0x7ff01c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.220+0000 7ff0237fe640 1 -- 172.21.15.100:0/2033379943 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01c017030 con 0x7ff02c102410 2024-07-26T13:43:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.220+0000 7ff0346e0640 1 -- 172.21.15.100:0/2033379943 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff02c1a6200 con 0x7ff02c102410 2024-07-26T13:43:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.220+0000 7ff0237fe640 1 -- 172.21.15.100:0/2033379943 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff01c004020 con 0x7ff02c102410 2024-07-26T13:43:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.220+0000 7ff0237fe640 1 -- 172.21.15.100:0/2033379943 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01c0047a0 con 0x7ff02c102410 2024-07-26T13:43:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.220+0000 7ff0346e0640 1 -- 172.21.15.100:0/2033379943 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff02c1a67a0 con 0x7ff02c102410 2024-07-26T13:43:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.222+0000 7ff03b5f1740 1 Processor -- start 2024-07-26T13:43:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.223+0000 7ff03b5f1740 1 -- start start 2024-07-26T13:43:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.223+0000 7ff03b5f1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e80e09c1b0 0x55e80e43ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.223+0000 7ff03b5f1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80e0732f0 0x55e80e43f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.223+0000 7ff0237fe640 1 -- 172.21.15.100:0/2033379943 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7ff01c009a30 con 0x7ff02c102410 2024-07-26T13:43:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.223+0000 7ff03b5f1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e80e07b4d0 0x55e80e43f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.223+0000 7ff03b5f1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55e80e3b43b0 con 0x55e80e07b4d0 2024-07-26T13:43:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.223+0000 7ff03b5f1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e80e3a38b0 con 0x55e80e09c1b0 2024-07-26T13:43:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.223+0000 7ff03b5f1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e80e3a78f0 con 0x55e80e0732f0 2024-07-26T13:43:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.223+0000 7ff031c7b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e80e07b4d0 0x55e80e43f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.223+0000 7ff031c7b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e80e07b4d0 0x55e80e43f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37500/0 (socket says 172.21.15.100:37500) 2024-07-26T13:43:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.223+0000 7ff031c7b640 1 -- 172.21.15.100:0/3819174032 learned_addr learned my addr 172.21.15.100:0/3819174032 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.223+0000 7ff03247c640 1 --2- 172.21.15.100:0/3819174032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80e0732f0 0x55e80e43f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.223+0000 7ff032c7d640 1 --2- 172.21.15.100:0/3819174032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e80e09c1b0 0x55e80e43ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.223+0000 7ff031c7b640 1 -- 172.21.15.100:0/3819174032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80e0732f0 msgr2=0x55e80e43f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.223+0000 7ff031c7b640 1 --2- 172.21.15.100:0/3819174032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80e0732f0 0x55e80e43f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.223+0000 7ff031c7b640 1 -- 172.21.15.100:0/3819174032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e80e09c1b0 msgr2=0x55e80e43ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.223+0000 7ff031c7b640 1 --2- 172.21.15.100:0/3819174032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e80e09c1b0 0x55e80e43ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.223+0000 7ff031c7b640 1 -- 172.21.15.100:0/3819174032 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e80e34a9a0 con 0x55e80e07b4d0 2024-07-26T13:43:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.224+0000 7ff031c7b640 1 --2- 172.21.15.100:0/3819174032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e80e07b4d0 0x55e80e43f760 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff028009c00 tx=0x7ff028011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6167284cf17277c4 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.224+0000 7ff020ff9640 1 -- 172.21.15.100:0/3819174032 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff028004050 con 0x55e80e07b4d0 2024-07-26T13:43:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.224+0000 7ff0237fe640 1 --2- 172.21.15.100:0/2033379943 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff0180b3d40 0x7ff0180b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.224+0000 7ff020ff9640 1 -- 172.21.15.100:0/3819174032 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0280054d0 con 0x55e80e07b4d0 2024-07-26T13:43:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.224+0000 7ff020ff9640 1 -- 172.21.15.100:0/3819174032 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff028004ca0 con 0x55e80e07b4d0 2024-07-26T13:43:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.224+0000 7ff032c7d640 1 --2- 172.21.15.100:0/2033379943 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff0180b3d40 0x7ff0180b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.224+0000 7ff0237fe640 1 -- 172.21.15.100:0/2033379943 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7ff01c0e3b20 con 0x7ff02c102410 2024-07-26T13:43:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.225+0000 7ff03b5f1740 1 -- 172.21.15.100:0/3819174032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e80e07b4d0 msgr2=0x55e80e43f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.225+0000 7ff032c7d640 1 --2- 172.21.15.100:0/2033379943 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff0180b3d40 0x7ff0180b6250 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff024006010 tx=0x7ff0240093f0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.225+0000 7ff03b5f1740 1 --2- 172.21.15.100:0/3819174032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e80e07b4d0 0x55e80e43f760 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff028009c00 tx=0x7ff028011810 comp rx=0 tx=0).stop 2024-07-26T13:43:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.225+0000 7ff03b5f1740 1 -- 172.21.15.100:0/3819174032 shutdown_connections 2024-07-26T13:43:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.225+0000 7ff03b5f1740 1 --2- 172.21.15.100:0/3819174032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e80e07b4d0 0x55e80e43f760 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.225+0000 7ff03b5f1740 1 --2- 172.21.15.100:0/3819174032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80e0732f0 0x55e80e43f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.225+0000 7ff03b5f1740 1 --2- 172.21.15.100:0/3819174032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e80e09c1b0 0x55e80e43ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.225+0000 7ff03b5f1740 1 -- 172.21.15.100:0/3819174032 >> 172.21.15.100:0/3819174032 conn(0x55e80e090300 msgr2=0x55e80e43cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.225+0000 7ff03b5f1740 1 -- 172.21.15.100:0/3819174032 shutdown_connections 2024-07-26T13:43:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.225+0000 7ff03b5f1740 1 -- 172.21.15.100:0/3819174032 wait complete. 2024-07-26T13:43:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.225+0000 7ff03b5f1740 1 -- start start 2024-07-26T13:43:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.225+0000 7ff03b5f1740 1 Processor -- start 2024-07-26T13:43:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.226+0000 7ff03b5f1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e80e0732f0 0x55e80e44a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.226+0000 7ff03b5f1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e80e07b4d0 0x55e80e44ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.226+0000 7ff03b5f1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80e09c1b0 0x55e80e456780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.226+0000 7ff03b5f1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55e80e2ddab0 con 0x55e80e07b4d0 2024-07-26T13:43:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.226+0000 7ff03b5f1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e80e32bcf0 con 0x55e80e0732f0 2024-07-26T13:43:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.226+0000 7ff03b5f1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e80e44d460 con 0x55e80e09c1b0 2024-07-26T13:43:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.226+0000 7ff03b5f1740 10 client.0 _ll_register_callbacks cb 0x55e80e39d9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:43:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.226+0000 7ff031c7b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e80e07b4d0 0x55e80e44ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:09.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.226+0000 7ff031c7b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e80e07b4d0 0x55e80e44ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37512/0 (socket says 172.21.15.100:37512) 2024-07-26T13:43:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.226+0000 7ff031c7b640 1 -- 172.21.15.100:0/2031182066 learned_addr learned my addr 172.21.15.100:0/2031182066 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.226+0000 7ff032c7d640 1 --2- 172.21.15.100:0/2031182066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80e09c1b0 0x55e80e456780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.226+0000 7ff03247c640 1 --2- 172.21.15.100:0/2031182066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e80e0732f0 0x55e80e44a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.226+0000 7ff031c7b640 1 -- 172.21.15.100:0/2031182066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80e09c1b0 msgr2=0x55e80e456780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.226+0000 7ff031c7b640 1 --2- 172.21.15.100:0/2031182066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80e09c1b0 0x55e80e456780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.226+0000 7ff031c7b640 1 -- 172.21.15.100:0/2031182066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e80e0732f0 msgr2=0x55e80e44a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.226+0000 7ff031c7b640 1 --2- 172.21.15.100:0/2031182066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e80e0732f0 0x55e80e44a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.226+0000 7ff031c7b640 1 -- 172.21.15.100:0/2031182066 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e80e107790 con 0x55e80e07b4d0 2024-07-26T13:43:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.226+0000 7ff03247c640 1 --2- 172.21.15.100:0/2031182066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e80e0732f0 0x55e80e44a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:43:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.226+0000 7ff031c7b640 1 --2- 172.21.15.100:0/2031182066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e80e07b4d0 0x55e80e44ace0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff02801b040 tx=0x7ff028004780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c188bd62ea99b2a server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.226+0000 7ff015ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:43:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.226+0000 7ff015ffb640 1 -- 172.21.15.100:0/2031182066 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff028029070 con 0x55e80e07b4d0 2024-07-26T13:43:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.226+0000 7ff015ffb640 1 -- 172.21.15.100:0/2031182066 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7ff028003a00 con 0x55e80e07b4d0 2024-07-26T13:43:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.226+0000 7ff03b5f1740 1 -- 172.21.15.100:0/2031182066 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0280054d0 con 0x55e80e07b4d0 2024-07-26T13:43:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.227+0000 7ff015ffb640 1 -- 172.21.15.100:0/2031182066 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff028035050 con 0x55e80e07b4d0 2024-07-26T13:43:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.227+0000 7ff03b5f1740 10 client.16272 fetch_fsmap learned FSMap version 30 2024-07-26T13:43:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.227+0000 7ff03b5f1740 1 -- 172.21.15.100:0/2031182066 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e80e180620 con 0x55e80e07b4d0 2024-07-26T13:43:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.227+0000 7ff015ffb640 1 -- 172.21.15.100:0/2031182066 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0280054d0 con 0x55e80e07b4d0 2024-07-26T13:43:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.227+0000 7ff015ffb640 1 -- 172.21.15.100:0/2031182066 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7ff028023430 con 0x55e80e07b4d0 2024-07-26T13:43:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.227+0000 7ff03b5f1740 10 client.16272 fetch_fsmap finished waiting for FSMap version 30 2024-07-26T13:43:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.227+0000 7ff03b5f1740 10 client.16272 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:43:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.227+0000 7ff03b5f1740 20 client.16272 populate_metadata read hostname 'smithi100' 2024-07-26T13:43:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.227+0000 7ff03b5f1740 1 --2- 172.21.15.100:0/2031182066 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55e80e05ad80 0x55e80e45c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.227+0000 7ff03b5f1740 4 client.16272 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:43:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.227+0000 7ff03b5f1740 1 -- 172.21.15.100:0/2031182066 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e80e3b6300 con 0x55e80e05ad80 2024-07-26T13:43:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.227+0000 7ff03247c640 1 --2- 172.21.15.100:0/2031182066 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55e80e05ad80 0x55e80e45c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.228+0000 7ff015ffb640 10 client.16272 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:43:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.228+0000 7ff03247c640 1 --2- 172.21.15.100:0/2031182066 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55e80e05ad80 0x55e80e45c590 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e582deadcfcae435 server_cookie=cfd058d7afb35a72 in_seq=0 out_seq=0 2024-07-26T13:43:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.228+0000 7ff015ffb640 1 -- 172.21.15.100:0/2031182066 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e80e3b6300 con 0x55e80e05ad80 2024-07-26T13:43:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.228+0000 7ff015ffb640 10 client.16272 handle_command_reply: tid=0 2024-07-26T13:43:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.228+0000 7ff03b5f1740 1 client.16272 shutdown 2024-07-26T13:43:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.228+0000 7ff03b5f1740 1 -- 172.21.15.100:0/2031182066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e80e07b4d0 msgr2=0x55e80e44ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.228+0000 7ff03b5f1740 1 --2- 172.21.15.100:0/2031182066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e80e07b4d0 0x55e80e44ace0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff02801b040 tx=0x7ff028004780 comp rx=0 tx=0).stop 2024-07-26T13:43:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.228+0000 7ff03b5f1740 1 -- 172.21.15.100:0/2031182066 shutdown_connections 2024-07-26T13:43:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.228+0000 7ff03b5f1740 1 --2- 172.21.15.100:0/2031182066 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55e80e05ad80 0x55e80e45c590 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.229+0000 7ff03b5f1740 1 --2- 172.21.15.100:0/2031182066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80e09c1b0 0x55e80e456780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.229+0000 7ff03b5f1740 1 --2- 172.21.15.100:0/2031182066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e80e07b4d0 0x55e80e44ace0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.229+0000 7ff03b5f1740 1 --2- 172.21.15.100:0/2031182066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e80e0732f0 0x55e80e44a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.229+0000 7ff015ffb640 0 client.16272 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:43:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.229+0000 7ff03b5f1740 1 -- 172.21.15.100:0/2031182066 >> 172.21.15.100:0/2031182066 conn(0x55e80e090300 msgr2=0x55e80e43fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.229+0000 7ff03b5f1740 1 -- 172.21.15.100:0/2031182066 shutdown_connections 2024-07-26T13:43:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.229+0000 7ff03b5f1740 1 -- 172.21.15.100:0/2031182066 wait complete. 2024-07-26T13:43:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.229+0000 7ff03b5f1740 20 client.16272 trim_cache size 0 max 16384 2024-07-26T13:43:09.249 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.243+0000 7ff03b5f1740 1 Processor -- start 2024-07-26T13:43:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.243+0000 7ff03b5f1740 1 -- start start 2024-07-26T13:43:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.244+0000 7ff03b5f1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e80e0732f0 0x55e80e440420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.244+0000 7ff03b5f1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80e07b4d0 0x55e80e456570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.244+0000 7ff03b5f1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e80e09c1b0 0x55e80e456ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.244+0000 7ff03b5f1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1afa70 con 0x55e80e09c1b0 2024-07-26T13:43:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.244+0000 7ff03b5f1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1ad1f0 con 0x55e80e0732f0 2024-07-26T13:43:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.244+0000 7ff03b5f1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1af3a0 con 0x55e80e07b4d0 2024-07-26T13:43:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.244+0000 7ff032c7d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e80e09c1b0 0x55e80e456ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.244+0000 7ff032c7d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e80e09c1b0 0x55e80e456ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37524/0 (socket says 172.21.15.100:37524) 2024-07-26T13:43:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.244+0000 7ff032c7d640 1 -- 172.21.15.100:0/3470881939 learned_addr learned my addr 172.21.15.100:0/3470881939 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.244+0000 7ff031c7b640 1 --2- 172.21.15.100:0/3470881939 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80e07b4d0 0x55e80e456570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.244+0000 7ff032c7d640 1 -- 172.21.15.100:0/3470881939 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80e07b4d0 msgr2=0x55e80e456570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.244+0000 7ff032c7d640 1 --2- 172.21.15.100:0/3470881939 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80e07b4d0 0x55e80e456570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.244+0000 7ff032c7d640 1 -- 172.21.15.100:0/3470881939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e80e0732f0 msgr2=0x55e80e440420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.244+0000 7ff032c7d640 1 --2- 172.21.15.100:0/3470881939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e80e0732f0 0x55e80e440420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.244+0000 7ff032c7d640 1 -- 172.21.15.100:0/3470881939 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e80e44a310 con 0x55e80e09c1b0 2024-07-26T13:43:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.244+0000 7ff032c7d640 1 --2- 172.21.15.100:0/3470881939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e80e09c1b0 0x55e80e456ab0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff024031e40 tx=0x7ff02403b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f98bba14a2dda6 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.244+0000 7ff017fff640 1 -- 172.21.15.100:0/3470881939 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff024056070 con 0x55e80e09c1b0 2024-07-26T13:43:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.244+0000 7ff017fff640 1 -- 172.21.15.100:0/3470881939 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff024036040 con 0x55e80e09c1b0 2024-07-26T13:43:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.244+0000 7ff017fff640 1 -- 172.21.15.100:0/3470881939 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02402f660 con 0x55e80e09c1b0 2024-07-26T13:43:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.245+0000 7ff03b5f1740 1 -- 172.21.15.100:0/3470881939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e80e09c1b0 msgr2=0x55e80e456ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.245+0000 7ff03b5f1740 1 --2- 172.21.15.100:0/3470881939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e80e09c1b0 0x55e80e456ab0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff024031e40 tx=0x7ff02403b000 comp rx=0 tx=0).stop 2024-07-26T13:43:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.245+0000 7ff03b5f1740 1 -- 172.21.15.100:0/3470881939 shutdown_connections 2024-07-26T13:43:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.245+0000 7ff03b5f1740 1 --2- 172.21.15.100:0/3470881939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e80e09c1b0 0x55e80e456ab0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.245+0000 7ff03b5f1740 1 --2- 172.21.15.100:0/3470881939 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80e07b4d0 0x55e80e456570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.245+0000 7ff03b5f1740 1 --2- 172.21.15.100:0/3470881939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e80e0732f0 0x55e80e440420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.245+0000 7ff03b5f1740 1 -- 172.21.15.100:0/3470881939 >> 172.21.15.100:0/3470881939 conn(0x55e80e090300 msgr2=0x55e80e43fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.245+0000 7ff03b5f1740 1 -- 172.21.15.100:0/3470881939 shutdown_connections 2024-07-26T13:43:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.245+0000 7ff03b5f1740 1 -- 172.21.15.100:0/3470881939 wait complete. 2024-07-26T13:43:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.246+0000 7ff03b5f1740 1 -- start start 2024-07-26T13:43:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.246+0000 7ff03b5f1740 1 Processor -- start 2024-07-26T13:43:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.246+0000 7ff03b5f1740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e80e0732f0 0x55e80e44f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.246+0000 7ff03b5f1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e80e07b4d0 0x55e80e44fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.246+0000 7ff03b5f1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80e09c1b0 0x55e80e450130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.246+0000 7ff03b5f1740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1aecd0 con 0x55e80e0732f0 2024-07-26T13:43:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.246+0000 7ff03b5f1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1ad860 con 0x55e80e07b4d0 2024-07-26T13:43:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.246+0000 7ff03b5f1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c1adf30 con 0x55e80e09c1b0 2024-07-26T13:43:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.246+0000 7ff03b5f1740 10 client.0 _ll_register_callbacks cb 0x7ff01c0e7540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:43:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.246+0000 7ff03247c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e80e0732f0 0x55e80e44f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.246+0000 7ff03247c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e80e0732f0 0x55e80e44f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37536/0 (socket says 172.21.15.100:37536) 2024-07-26T13:43:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.246+0000 7ff03247c640 1 -- 172.21.15.100:0/1041393924 learned_addr learned my addr 172.21.15.100:0/1041393924 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.246+0000 7ff032c7d640 1 --2- 172.21.15.100:0/1041393924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80e09c1b0 0x55e80e450130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.246+0000 7ff031c7b640 1 --2- 172.21.15.100:0/1041393924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e80e07b4d0 0x55e80e44fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.246+0000 7ff03247c640 1 -- 172.21.15.100:0/1041393924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80e09c1b0 msgr2=0x55e80e450130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.246+0000 7ff03247c640 1 --2- 172.21.15.100:0/1041393924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80e09c1b0 0x55e80e450130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.246+0000 7ff03247c640 1 -- 172.21.15.100:0/1041393924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e80e07b4d0 msgr2=0x55e80e44fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.246+0000 7ff03247c640 1 --2- 172.21.15.100:0/1041393924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e80e07b4d0 0x55e80e44fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.246+0000 7ff03247c640 1 -- 172.21.15.100:0/1041393924 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e80e441440 con 0x55e80e0732f0 2024-07-26T13:43:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.246+0000 7ff031c7b640 1 --2- 172.21.15.100:0/1041393924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e80e07b4d0 0x55e80e44fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:43:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.246+0000 7ff03247c640 1 --2- 172.21.15.100:0/1041393924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e80e0732f0 0x55e80e44f6b0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff01c0a7970 tx=0x7ff01c0e7290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e5554f9af274181 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.246+0000 7ff015ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:43:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.247+0000 7ff015ffb640 1 -- 172.21.15.100:0/1041393924 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01c014070 con 0x55e80e0732f0 2024-07-26T13:43:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.247+0000 7ff015ffb640 1 -- 172.21.15.100:0/1041393924 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7ff01c0e14c0 con 0x55e80e0732f0 2024-07-26T13:43:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.247+0000 7ff03b5f1740 1 -- 172.21.15.100:0/1041393924 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff024036040 con 0x55e80e0732f0 2024-07-26T13:43:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.247+0000 7ff015ffb640 1 -- 172.21.15.100:0/1041393924 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff01c0c4050 con 0x55e80e0732f0 2024-07-26T13:43:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.247+0000 7ff03b5f1740 10 client.16278 fetch_fsmap learned FSMap version 30 2024-07-26T13:43:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.247+0000 7ff03b5f1740 1 -- 172.21.15.100:0/1041393924 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e80e4c3cc0 con 0x55e80e0732f0 2024-07-26T13:43:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.247+0000 7ff015ffb640 1 -- 172.21.15.100:0/1041393924 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff024036040 con 0x55e80e0732f0 2024-07-26T13:43:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.247+0000 7ff015ffb640 1 -- 172.21.15.100:0/1041393924 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 30) ==== 8358+0+0 (secure 0 0 0) 0x7ff01c0a6990 con 0x55e80e0732f0 2024-07-26T13:43:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.247+0000 7ff03b5f1740 10 client.16278 fetch_fsmap finished waiting for FSMap version 30 2024-07-26T13:43:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.247+0000 7ff03b5f1740 10 client.16278 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:43:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.247+0000 7ff03b5f1740 20 client.16278 populate_metadata read hostname 'smithi100' 2024-07-26T13:43:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.247+0000 7ff03b5f1740 1 --2- 172.21.15.100:0/1041393924 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55e80e0301c0 0x55e80e4c6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.247+0000 7ff03b5f1740 4 client.16278 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-26T13:43:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.247+0000 7ff03b5f1740 1 -- 172.21.15.100:0/1041393924 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e80e3b1da0 con 0x55e80e0301c0 2024-07-26T13:43:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.247+0000 7ff031c7b640 1 --2- 172.21.15.100:0/1041393924 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55e80e0301c0 0x55e80e4c6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.248+0000 7ff015ffb640 10 client.16278 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:43:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:09.248+0000 7ff031c7b640 1 --2- 172.21.15.100:0/1041393924 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55e80e0301c0 0x55e80e4c6490 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b8ea0d9738b2fe2 server_cookie=35f5700cb149c568 in_seq=0 out_seq=0 2024-07-26T13:43:09.407 INFO:tasks.workunit.client.0.smithi100.stdout: 1 82357 5.56 MB/sec warmup 83 sec latency 13597.881 ms 2024-07-26T13:43:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:09 smithi100 ceph-mon[32599]: pgmap v455: 97 pgs: 97 active+clean; 49 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 799 KiB/s wr, 100 op/s 2024-07-26T13:43:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:09 smithi100 ceph-mon[32599]: daemon mds.c restarted 2024-07-26T13:43:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:09 smithi100 ceph-mon[32599]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-26T13:43:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:09 smithi100 ceph-mon[32599]: osdmap e82: 12 total, 12 up, 12 in 2024-07-26T13:43:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:09 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.167:6836/2476068386,v1:172.21.15.167:6837/2476068386] up:boot 2024-07-26T13:43:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:09 smithi100 ceph-mon[32599]: Standby daemon mds.e assigned to filesystem cephfs as rank 2 2024-07-26T13:43:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:09 smithi100 ceph-mon[32599]: fsmap cephfs:2/3 {0=i=up:active,1=b=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-07-26T13:43:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-26T13:43:09.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:09 smithi100 ceph-mon[32599]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=e=up:replay} 2 up:standby-replay 7 up:standby 2024-07-26T13:43:09.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:09 smithi138 ceph-mon[36954]: pgmap v455: 97 pgs: 97 active+clean; 49 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 799 KiB/s wr, 100 op/s 2024-07-26T13:43:09.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:09 smithi138 ceph-mon[36954]: daemon mds.c restarted 2024-07-26T13:43:09.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:09 smithi138 ceph-mon[36954]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-26T13:43:09.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:09 smithi138 ceph-mon[36954]: osdmap e82: 12 total, 12 up, 12 in 2024-07-26T13:43:09.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:09 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.167:6836/2476068386,v1:172.21.15.167:6837/2476068386] up:boot 2024-07-26T13:43:09.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:09 smithi138 ceph-mon[36954]: Standby daemon mds.e assigned to filesystem cephfs as rank 2 2024-07-26T13:43:09.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:09 smithi138 ceph-mon[36954]: fsmap cephfs:2/3 {0=i=up:active,1=b=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-07-26T13:43:09.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-26T13:43:09.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:09 smithi138 ceph-mon[36954]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=e=up:replay} 2 up:standby-replay 7 up:standby 2024-07-26T13:43:10.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:09 smithi167 ceph-mon[37140]: pgmap v455: 97 pgs: 97 active+clean; 49 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 799 KiB/s wr, 100 op/s 2024-07-26T13:43:10.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:09 smithi167 ceph-mon[37140]: daemon mds.c restarted 2024-07-26T13:43:10.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:09 smithi167 ceph-mon[37140]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-26T13:43:10.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:09 smithi167 ceph-mon[37140]: osdmap e82: 12 total, 12 up, 12 in 2024-07-26T13:43:10.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:09 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.167:6836/2476068386,v1:172.21.15.167:6837/2476068386] up:boot 2024-07-26T13:43:10.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:09 smithi167 ceph-mon[37140]: Standby daemon mds.e assigned to filesystem cephfs as rank 2 2024-07-26T13:43:10.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:09 smithi167 ceph-mon[37140]: fsmap cephfs:2/3 {0=i=up:active,1=b=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-07-26T13:43:10.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-26T13:43:10.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:09 smithi167 ceph-mon[37140]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=e=up:replay} 2 up:standby-replay 7 up:standby 2024-07-26T13:43:10.407 INFO:tasks.workunit.client.0.smithi100.stdout: 1 82357 5.49 MB/sec warmup 84 sec latency 14597.937 ms 2024-07-26T13:43:10.721 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:10 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] up:resolve 2024-07-26T13:43:10.721 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:10 smithi100 ceph-mon[32599]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=e=up:resolve} 2 up:standby-replay 7 up:standby 2024-07-26T13:43:10.722 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:10 smithi100 ceph-mon[32599]: pgmap v457: 97 pgs: 97 active+clean; 53 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 966 KiB/s wr, 38 op/s 2024-07-26T13:43:10.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.812+0000 7f2a8b2c0640 1 -- 172.21.15.100:0/54668703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a84103590 msgr2=0x7f2a841039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:10.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.812+0000 7f2a8b2c0640 1 --2- 172.21.15.100:0/54668703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a84103590 0x7f2a841039a0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009900 tx=0x7f2a8002a410 comp rx=0 tx=0).stop 2024-07-26T13:43:10.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.813+0000 7f2a8b2c0640 1 -- 172.21.15.100:0/54668703 shutdown_connections 2024-07-26T13:43:10.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.813+0000 7f2a8b2c0640 1 --2- 172.21.15.100:0/54668703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a84109820 0x7f2a84110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:10.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.813+0000 7f2a8b2c0640 1 --2- 172.21.15.100:0/54668703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a84108ed0 0x7f2a841092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:10.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.813+0000 7f2a8b2c0640 1 --2- 172.21.15.100:0/54668703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a84103590 0x7f2a841039a0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:10.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.813+0000 7f2a8b2c0640 1 -- 172.21.15.100:0/54668703 >> 172.21.15.100:0/54668703 conn(0x7f2a8407b280 msgr2=0x7f2a8407b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:10.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.813+0000 7f2a8b2c0640 1 -- 172.21.15.100:0/54668703 shutdown_connections 2024-07-26T13:43:10.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.813+0000 7f2a8b2c0640 1 -- 172.21.15.100:0/54668703 wait complete. 2024-07-26T13:43:10.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.813+0000 7f2a8b2c0640 1 Processor -- start 2024-07-26T13:43:10.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.813+0000 7f2a8b2c0640 1 -- start start 2024-07-26T13:43:10.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.814+0000 7f2a8b2c0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a84103590 0x7f2a84199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:10.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.814+0000 7f2a8b2c0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a84108ed0 0x7f2a8419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:10.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.814+0000 7f2a8b2c0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a84109820 0x7f2a8419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:10.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.814+0000 7f2a8b2c0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2a841134d0 con 0x7f2a84108ed0 2024-07-26T13:43:10.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.814+0000 7f2a8b2c0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2a84113350 con 0x7f2a84103590 2024-07-26T13:43:10.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.814+0000 7f2a8b2c0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2a84113650 con 0x7f2a84109820 2024-07-26T13:43:10.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.814+0000 7f2a8885b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a84108ed0 0x7f2a8419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:10.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.814+0000 7f2a8885b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a84108ed0 0x7f2a8419a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37558/0 (socket says 172.21.15.100:37558) 2024-07-26T13:43:10.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.814+0000 7f2a8885b640 1 -- 172.21.15.100:0/3029385713 learned_addr learned my addr 172.21.15.100:0/3029385713 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:10.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.814+0000 7f2a8905c640 1 --2- 172.21.15.100:0/3029385713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a84103590 0x7f2a84199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:10.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.814+0000 7f2a8985d640 1 --2- 172.21.15.100:0/3029385713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a84109820 0x7f2a8419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:10.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.814+0000 7f2a8885b640 1 -- 172.21.15.100:0/3029385713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a84109820 msgr2=0x7f2a8419f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:10.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.814+0000 7f2a8885b640 1 --2- 172.21.15.100:0/3029385713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a84109820 0x7f2a8419f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:10.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.814+0000 7f2a8885b640 1 -- 172.21.15.100:0/3029385713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a84103590 msgr2=0x7f2a84199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:10.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.814+0000 7f2a8885b640 1 --2- 172.21.15.100:0/3029385713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a84103590 0x7f2a84199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:10.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.814+0000 7f2a8885b640 1 -- 172.21.15.100:0/3029385713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a8419fe30 con 0x7f2a84108ed0 2024-07-26T13:43:10.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.814+0000 7f2a8985d640 1 --2- 172.21.15.100:0/3029385713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a84109820 0x7f2a8419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:43:10.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.814+0000 7f2a8885b640 1 --2- 172.21.15.100:0/3029385713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a84108ed0 0x7f2a8419a460 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400e9f0 tx=0x7f2a7400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:10.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.814+0000 7f2a7a7fc640 1 -- 172.21.15.100:0/3029385713 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7400ccc0 con 0x7f2a84108ed0 2024-07-26T13:43:10.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.815+0000 7f2a7a7fc640 1 -- 172.21.15.100:0/3029385713 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a740108e0 con 0x7f2a84108ed0 2024-07-26T13:43:10.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.815+0000 7f2a7a7fc640 1 -- 172.21.15.100:0/3029385713 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a74018860 con 0x7f2a84108ed0 2024-07-26T13:43:10.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.815+0000 7f2a8b2c0640 1 -- 172.21.15.100:0/3029385713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a841a0120 con 0x7f2a84108ed0 2024-07-26T13:43:10.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.815+0000 7f2a8b2c0640 1 -- 172.21.15.100:0/3029385713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a841a6880 con 0x7f2a84108ed0 2024-07-26T13:43:10.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.817+0000 7f2a91844740 1 Processor -- start 2024-07-26T13:43:10.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.817+0000 7f2a91844740 1 -- start start 2024-07-26T13:43:10.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.817+0000 7f2a7a7fc640 1 -- 172.21.15.100:0/3029385713 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f2a74010430 con 0x7f2a84108ed0 2024-07-26T13:43:10.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.817+0000 7f2a91844740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606280e5070 0x56062848a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:10.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.817+0000 7f2a91844740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606280bc1b0 0x56062848abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:10.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.817+0000 7f2a91844740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606280c4390 0x56062848b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:10.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.818+0000 7f2a91844740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x560628400b90 con 0x5606280bc1b0 2024-07-26T13:43:10.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.818+0000 7f2a91844740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5606283f0090 con 0x5606280c4390 2024-07-26T13:43:10.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.818+0000 7f2a91844740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5606283f40d0 con 0x5606280e5070 2024-07-26T13:43:10.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.818+0000 7f2a8905c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606280bc1b0 0x56062848abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:10.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.818+0000 7f2a8905c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606280bc1b0 0x56062848abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37574/0 (socket says 172.21.15.100:37574) 2024-07-26T13:43:10.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.818+0000 7f2a8905c640 1 -- 172.21.15.100:0/583017311 learned_addr learned my addr 172.21.15.100:0/583017311 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:10.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.818+0000 7f2a8985d640 1 --2- 172.21.15.100:0/583017311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606280e5070 0x56062848a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:10.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.818+0000 7f2a8885b640 1 --2- 172.21.15.100:0/583017311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606280c4390 0x56062848b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:10.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.818+0000 7f2a8905c640 1 -- 172.21.15.100:0/583017311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606280e5070 msgr2=0x56062848a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:10.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.818+0000 7f2a8905c640 1 --2- 172.21.15.100:0/583017311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606280e5070 0x56062848a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:10.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.818+0000 7f2a8905c640 1 -- 172.21.15.100:0/583017311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606280c4390 msgr2=0x56062848b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:10.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.818+0000 7f2a8905c640 1 --2- 172.21.15.100:0/583017311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606280c4390 0x56062848b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:10.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.818+0000 7f2a8905c640 1 -- 172.21.15.100:0/583017311 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560628396b70 con 0x5606280bc1b0 2024-07-26T13:43:10.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.818+0000 7f2a8985d640 1 --2- 172.21.15.100:0/583017311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606280e5070 0x56062848a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:43:10.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.818+0000 7f2a8905c640 1 --2- 172.21.15.100:0/583017311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606280bc1b0 0x56062848abb0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009900 tx=0x7f2a80032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7bb6d86853c4608 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:10.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.818+0000 7f2a7a7fc640 1 --2- 172.21.15.100:0/3029385713 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f2a700b3d40 0x7f2a700b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:10.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.818+0000 7f2a8985d640 1 --2- 172.21.15.100:0/3029385713 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f2a700b3d40 0x7f2a700b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:10.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.818+0000 7f2a6f7fe640 1 -- 172.21.15.100:0/583017311 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80040070 con 0x5606280bc1b0 2024-07-26T13:43:10.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.819+0000 7f2a7a7fc640 1 -- 172.21.15.100:0/3029385713 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f2a740e4330 con 0x7f2a84108ed0 2024-07-26T13:43:10.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.819+0000 7f2a6f7fe640 1 -- 172.21.15.100:0/583017311 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a80004030 con 0x5606280bc1b0 2024-07-26T13:43:10.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.819+0000 7f2a6f7fe640 1 -- 172.21.15.100:0/583017311 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80035040 con 0x5606280bc1b0 2024-07-26T13:43:10.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.819+0000 7f2a91844740 1 -- 172.21.15.100:0/583017311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606280bc1b0 msgr2=0x56062848abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:10.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.819+0000 7f2a8985d640 1 --2- 172.21.15.100:0/3029385713 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f2a700b3d40 0x7f2a700b6250 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c005fd0 tx=0x7f2a7c009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:10.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.819+0000 7f2a91844740 1 --2- 172.21.15.100:0/583017311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606280bc1b0 0x56062848abb0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009900 tx=0x7f2a80032b90 comp rx=0 tx=0).stop 2024-07-26T13:43:10.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.819+0000 7f2a91844740 1 -- 172.21.15.100:0/583017311 shutdown_connections 2024-07-26T13:43:10.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.819+0000 7f2a91844740 1 --2- 172.21.15.100:0/583017311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606280c4390 0x56062848b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:10.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.819+0000 7f2a91844740 1 --2- 172.21.15.100:0/583017311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606280bc1b0 0x56062848abb0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:10.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.819+0000 7f2a91844740 1 --2- 172.21.15.100:0/583017311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606280e5070 0x56062848a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:10.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.819+0000 7f2a91844740 1 -- 172.21.15.100:0/583017311 >> 172.21.15.100:0/583017311 conn(0x5606280d9460 msgr2=0x560628488560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:10.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.820+0000 7f2a91844740 1 -- 172.21.15.100:0/583017311 shutdown_connections 2024-07-26T13:43:10.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.820+0000 7f2a91844740 1 -- 172.21.15.100:0/583017311 wait complete. 2024-07-26T13:43:10.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.820+0000 7f2a91844740 1 -- start start 2024-07-26T13:43:10.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.820+0000 7f2a91844740 1 Processor -- start 2024-07-26T13:43:10.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.820+0000 7f2a91844740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606280bc1b0 0x56062849a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:10.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.820+0000 7f2a91844740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606280c4390 0x56062849aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:10.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.820+0000 7f2a91844740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606280e5070 0x5606284a6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:10.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.821+0000 7f2a91844740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x560628253700 con 0x5606280c4390 2024-07-26T13:43:10.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.821+0000 7f2a91844740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5606283929c0 con 0x5606280bc1b0 2024-07-26T13:43:10.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.821+0000 7f2a91844740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56062849d150 con 0x5606280e5070 2024-07-26T13:43:10.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.821+0000 7f2a91844740 10 client.0 _ll_register_callbacks cb 0x5606283ad4d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:43:10.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.821+0000 7f2a8885b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606280c4390 0x56062849aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:10.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.821+0000 7f2a8885b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606280c4390 0x56062849aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37586/0 (socket says 172.21.15.100:37586) 2024-07-26T13:43:10.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.821+0000 7f2a8885b640 1 -- 172.21.15.100:0/3890764377 learned_addr learned my addr 172.21.15.100:0/3890764377 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:10.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.821+0000 7f2a8905c640 1 --2- 172.21.15.100:0/3890764377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606280bc1b0 0x56062849a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:10.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.821+0000 7f2a8985d640 1 --2- 172.21.15.100:0/3890764377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606280e5070 0x5606284a6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:10.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.821+0000 7f2a8885b640 1 -- 172.21.15.100:0/3890764377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606280e5070 msgr2=0x5606284a6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:10.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.821+0000 7f2a8885b640 1 --2- 172.21.15.100:0/3890764377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606280e5070 0x5606284a6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:10.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.821+0000 7f2a8885b640 1 -- 172.21.15.100:0/3890764377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606280bc1b0 msgr2=0x56062849a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:10.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.821+0000 7f2a8885b640 1 --2- 172.21.15.100:0/3890764377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606280bc1b0 0x56062849a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:10.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.821+0000 7f2a8885b640 1 -- 172.21.15.100:0/3890764377 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560628150650 con 0x5606280c4390 2024-07-26T13:43:10.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.821+0000 7f2a8905c640 1 --2- 172.21.15.100:0/3890764377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606280bc1b0 0x56062849a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:43:10.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.821+0000 7f2a8885b640 1 --2- 172.21.15.100:0/3890764377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606280c4390 0x56062849aac0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2a74000c00 tx=0x7f2a74009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=830557979b602a3b server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:10.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.821+0000 7f2a6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:43:10.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.821+0000 7f2a6cff9640 1 -- 172.21.15.100:0/3890764377 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a740aaa20 con 0x5606280c4390 2024-07-26T13:43:10.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.821+0000 7f2a6cff9640 1 -- 172.21.15.100:0/3890764377 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f2a740a71a0 con 0x5606280c4390 2024-07-26T13:43:10.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.821+0000 7f2a91844740 1 -- 172.21.15.100:0/3890764377 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a80004030 con 0x5606280c4390 2024-07-26T13:43:10.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.821+0000 7f2a6cff9640 1 -- 172.21.15.100:0/3890764377 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2a740ab610 con 0x5606280c4390 2024-07-26T13:43:10.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.821+0000 7f2a91844740 10 client.16290 fetch_fsmap learned FSMap version 32 2024-07-26T13:43:10.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.821+0000 7f2a91844740 1 -- 172.21.15.100:0/3890764377 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5606284a91a0 con 0x5606280c4390 2024-07-26T13:43:10.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.822+0000 7f2a6cff9640 1 -- 172.21.15.100:0/3890764377 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a80004030 con 0x5606280c4390 2024-07-26T13:43:10.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.822+0000 7f2a6cff9640 1 -- 172.21.15.100:0/3890764377 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f2a740a6880 con 0x5606280c4390 2024-07-26T13:43:10.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.822+0000 7f2a91844740 10 client.16290 fetch_fsmap finished waiting for FSMap version 32 2024-07-26T13:43:10.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.822+0000 7f2a91844740 10 client.16290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:43:10.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.822+0000 7f2a91844740 20 client.16290 populate_metadata read hostname 'smithi100' 2024-07-26T13:43:10.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.822+0000 7f2a91844740 1 --2- 172.21.15.100:0/3890764377 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5606280a39f0 0x5606284ac6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:10.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.822+0000 7f2a91844740 4 client.16290 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:43:10.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.822+0000 7f2a91844740 1 -- 172.21.15.100:0/3890764377 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560628402ae0 con 0x5606280a39f0 2024-07-26T13:43:10.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.822+0000 7f2a8905c640 1 --2- 172.21.15.100:0/3890764377 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5606280a39f0 0x5606284ac6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:10.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.823+0000 7f2a6cff9640 10 client.16290 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:43:10.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.823+0000 7f2a8905c640 1 --2- 172.21.15.100:0/3890764377 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5606280a39f0 0x5606284ac6b0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=344ea09fdaed0734 server_cookie=e9b07ec0dde5eda in_seq=0 out_seq=0 2024-07-26T13:43:10.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.823+0000 7f2a6cff9640 1 -- 172.21.15.100:0/3890764377 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560628402ae0 con 0x5606280a39f0 2024-07-26T13:43:10.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.823+0000 7f2a6cff9640 10 client.16290 handle_command_reply: tid=0 2024-07-26T13:43:10.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.823+0000 7f2a91844740 1 client.16290 shutdown 2024-07-26T13:43:10.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.823+0000 7f2a91844740 1 -- 172.21.15.100:0/3890764377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606280c4390 msgr2=0x56062849aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:10.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.823+0000 7f2a91844740 1 --2- 172.21.15.100:0/3890764377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606280c4390 0x56062849aac0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2a74000c00 tx=0x7f2a74009740 comp rx=0 tx=0).stop 2024-07-26T13:43:10.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.823+0000 7f2a91844740 1 -- 172.21.15.100:0/3890764377 shutdown_connections 2024-07-26T13:43:10.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.823+0000 7f2a91844740 1 --2- 172.21.15.100:0/3890764377 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5606280a39f0 0x5606284ac6b0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:10.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.823+0000 7f2a91844740 1 --2- 172.21.15.100:0/3890764377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606280e5070 0x5606284a6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:10.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.823+0000 7f2a91844740 1 --2- 172.21.15.100:0/3890764377 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606280c4390 0x56062849aac0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:10.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.823+0000 7f2a91844740 1 --2- 172.21.15.100:0/3890764377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606280bc1b0 0x56062849a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:10.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.823+0000 7f2a91844740 1 -- 172.21.15.100:0/3890764377 >> 172.21.15.100:0/3890764377 conn(0x5606280d9460 msgr2=0x56062848c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:10.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.823+0000 7f2a6cff9640 0 client.16290 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:43:10.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.824+0000 7f2a91844740 1 -- 172.21.15.100:0/3890764377 shutdown_connections 2024-07-26T13:43:10.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.824+0000 7f2a91844740 1 -- 172.21.15.100:0/3890764377 wait complete. 2024-07-26T13:43:10.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.824+0000 7f2a91844740 20 client.16290 trim_cache size 0 max 16384 2024-07-26T13:43:10.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.841+0000 7f2a91844740 1 Processor -- start 2024-07-26T13:43:10.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.841+0000 7f2a91844740 1 -- start start 2024-07-26T13:43:10.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.841+0000 7f2a91844740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606280bc1b0 0x5606284a6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:10.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.841+0000 7f2a91844740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606280c4390 0x5606284a6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:10.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.841+0000 7f2a91844740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606280e5070 0x5606284b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:10.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.841+0000 7f2a91844740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2a841ad840 con 0x5606280bc1b0 2024-07-26T13:43:10.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.841+0000 7f2a91844740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2a841aafc0 con 0x5606280e5070 2024-07-26T13:43:10.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.841+0000 7f2a91844740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2a841ad170 con 0x5606280c4390 2024-07-26T13:43:10.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.841+0000 7f2a8905c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606280bc1b0 0x5606284a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:10.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.841+0000 7f2a8905c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606280bc1b0 0x5606284a6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37598/0 (socket says 172.21.15.100:37598) 2024-07-26T13:43:10.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.841+0000 7f2a8905c640 1 -- 172.21.15.100:0/3485795791 learned_addr learned my addr 172.21.15.100:0/3485795791 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:10.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.841+0000 7f2a8885b640 1 --2- 172.21.15.100:0/3485795791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606280c4390 0x5606284a6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:10.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.841+0000 7f2a8985d640 1 --2- 172.21.15.100:0/3485795791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606280e5070 0x5606284b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:10.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.841+0000 7f2a8905c640 1 -- 172.21.15.100:0/3485795791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606280c4390 msgr2=0x5606284a6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:10.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.841+0000 7f2a8905c640 1 --2- 172.21.15.100:0/3485795791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606280c4390 0x5606284a6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:10.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.841+0000 7f2a8905c640 1 -- 172.21.15.100:0/3485795791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606280e5070 msgr2=0x5606284b1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:10.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.841+0000 7f2a8905c640 1 --2- 172.21.15.100:0/3485795791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606280e5070 0x5606284b1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:10.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.841+0000 7f2a8905c640 1 -- 172.21.15.100:0/3485795791 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56062848b460 con 0x5606280bc1b0 2024-07-26T13:43:10.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.842+0000 7f2a8905c640 1 --2- 172.21.15.100:0/3485795791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606280bc1b0 0x5606284a6300 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f2a80005150 tx=0x7f2a80005180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd7097fce05535a4 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:10.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.842+0000 7f2a6effd640 1 -- 172.21.15.100:0/3485795791 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80053020 con 0x5606280bc1b0 2024-07-26T13:43:10.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.842+0000 7f2a6effd640 1 -- 172.21.15.100:0/3485795791 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a800051f0 con 0x5606280bc1b0 2024-07-26T13:43:10.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.842+0000 7f2a6effd640 1 -- 172.21.15.100:0/3485795791 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80035040 con 0x5606280bc1b0 2024-07-26T13:43:10.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.843+0000 7f2a91844740 1 -- 172.21.15.100:0/3485795791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606280bc1b0 msgr2=0x5606284a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:10.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.843+0000 7f2a91844740 1 --2- 172.21.15.100:0/3485795791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606280bc1b0 0x5606284a6300 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f2a80005150 tx=0x7f2a80005180 comp rx=0 tx=0).stop 2024-07-26T13:43:10.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.843+0000 7f2a91844740 1 -- 172.21.15.100:0/3485795791 shutdown_connections 2024-07-26T13:43:10.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.843+0000 7f2a91844740 1 --2- 172.21.15.100:0/3485795791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606280e5070 0x5606284b1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:10.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.843+0000 7f2a91844740 1 --2- 172.21.15.100:0/3485795791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606280c4390 0x5606284a6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:10.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.843+0000 7f2a91844740 1 --2- 172.21.15.100:0/3485795791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606280bc1b0 0x5606284a6300 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:10.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.843+0000 7f2a91844740 1 -- 172.21.15.100:0/3485795791 >> 172.21.15.100:0/3485795791 conn(0x5606280d9460 msgr2=0x560628498c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:10.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.843+0000 7f2a91844740 1 -- 172.21.15.100:0/3485795791 shutdown_connections 2024-07-26T13:43:10.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.843+0000 7f2a91844740 1 -- 172.21.15.100:0/3485795791 wait complete. 2024-07-26T13:43:10.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.843+0000 7f2a91844740 1 -- start start 2024-07-26T13:43:10.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.843+0000 7f2a91844740 1 Processor -- start 2024-07-26T13:43:10.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.844+0000 7f2a91844740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606280bc1b0 0x5606284a2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:10.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.844+0000 7f2a91844740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606280c4390 0x56062849df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:10.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.844+0000 7f2a91844740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606280e5070 0x56062849e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:10.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.844+0000 7f2a91844740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2a841acaa0 con 0x5606280c4390 2024-07-26T13:43:10.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.844+0000 7f2a91844740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2a841ab630 con 0x5606280e5070 2024-07-26T13:43:10.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.844+0000 7f2a91844740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2a841abd00 con 0x5606280bc1b0 2024-07-26T13:43:10.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.844+0000 7f2a91844740 10 client.0 _ll_register_callbacks cb 0x7f2a740e7990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:43:10.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.844+0000 7f2a8885b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606280c4390 0x56062849df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:10.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.844+0000 7f2a8885b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606280c4390 0x56062849df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37614/0 (socket says 172.21.15.100:37614) 2024-07-26T13:43:10.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.844+0000 7f2a8885b640 1 -- 172.21.15.100:0/3072116826 learned_addr learned my addr 172.21.15.100:0/3072116826 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:10.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.844+0000 7f2a8905c640 1 --2- 172.21.15.100:0/3072116826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606280bc1b0 0x5606284a2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:10.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.844+0000 7f2a8985d640 1 --2- 172.21.15.100:0/3072116826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606280e5070 0x56062849e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:10.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.844+0000 7f2a8885b640 1 -- 172.21.15.100:0/3072116826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606280bc1b0 msgr2=0x5606284a2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:10.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.844+0000 7f2a8885b640 1 --2- 172.21.15.100:0/3072116826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606280bc1b0 0x5606284a2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:10.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.844+0000 7f2a8885b640 1 -- 172.21.15.100:0/3072116826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606280e5070 msgr2=0x56062849e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:10.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.844+0000 7f2a8885b640 1 --2- 172.21.15.100:0/3072116826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606280e5070 0x56062849e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:10.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.844+0000 7f2a8885b640 1 -- 172.21.15.100:0/3072116826 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560628494fe0 con 0x5606280c4390 2024-07-26T13:43:10.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.844+0000 7f2a8885b640 1 --2- 172.21.15.100:0/3072116826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606280c4390 0x56062849df00 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f2a740aaea0 tx=0x7f2a740096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc138a9e5dbca73 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:10.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.844+0000 7f2a6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:43:10.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.844+0000 7f2a6cff9640 1 -- 172.21.15.100:0/3072116826 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a740aaa20 con 0x5606280c4390 2024-07-26T13:43:10.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.844+0000 7f2a6cff9640 1 -- 172.21.15.100:0/3072116826 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f2a740a9d20 con 0x5606280c4390 2024-07-26T13:43:10.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.844+0000 7f2a91844740 1 -- 172.21.15.100:0/3072116826 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a800051f0 con 0x5606280c4390 2024-07-26T13:43:10.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.845+0000 7f2a6cff9640 1 -- 172.21.15.100:0/3072116826 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2a740b3fc0 con 0x5606280c4390 2024-07-26T13:43:10.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.845+0000 7f2a91844740 10 client.16296 fetch_fsmap learned FSMap version 32 2024-07-26T13:43:10.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.845+0000 7f2a91844740 1 -- 172.21.15.100:0/3072116826 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560628494fe0 con 0x5606280c4390 2024-07-26T13:43:10.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.845+0000 7f2a6cff9640 1 -- 172.21.15.100:0/3072116826 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a800051f0 con 0x5606280c4390 2024-07-26T13:43:10.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.845+0000 7f2a6cff9640 1 -- 172.21.15.100:0/3072116826 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f2a740b06a0 con 0x5606280c4390 2024-07-26T13:43:10.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.845+0000 7f2a91844740 10 client.16296 fetch_fsmap finished waiting for FSMap version 32 2024-07-26T13:43:10.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.845+0000 7f2a91844740 10 client.16296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:43:10.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.845+0000 7f2a91844740 20 client.16296 populate_metadata read hostname 'smithi100' 2024-07-26T13:43:10.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.845+0000 7f2a91844740 1 --2- 172.21.15.100:0/3072116826 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560628078da0 0x560628516f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:10.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.845+0000 7f2a91844740 4 client.16296 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-26T13:43:10.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.845+0000 7f2a91844740 1 -- 172.21.15.100:0/3072116826 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5606283fe580 con 0x560628078da0 2024-07-26T13:43:10.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.845+0000 7f2a8905c640 1 --2- 172.21.15.100:0/3072116826 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560628078da0 0x560628516f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:10.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.846+0000 7f2a8905c640 1 --2- 172.21.15.100:0/3072116826 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560628078da0 0x560628516f50 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6e86113c8dd1b2f server_cookie=2fd558c7001a32e3 in_seq=0 out_seq=0 2024-07-26T13:43:10.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.846+0000 7f2a6cff9640 10 client.16296 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:43:10.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.846+0000 7f2a6cff9640 1 -- 172.21.15.100:0/3072116826 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5606283fe580 con 0x560628078da0 2024-07-26T13:43:10.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.846+0000 7f2a6cff9640 10 client.16296 handle_command_reply: tid=0 2024-07-26T13:43:10.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.846+0000 7f2a91844740 1 client.16296 shutdown 2024-07-26T13:43:10.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.846+0000 7f2a91844740 1 -- 172.21.15.100:0/3072116826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606280c4390 msgr2=0x56062849df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:10.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.846+0000 7f2a91844740 1 --2- 172.21.15.100:0/3072116826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606280c4390 0x56062849df00 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f2a740aaea0 tx=0x7f2a740096a0 comp rx=0 tx=0).stop 2024-07-26T13:43:10.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.846+0000 7f2a91844740 1 -- 172.21.15.100:0/3072116826 shutdown_connections 2024-07-26T13:43:10.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.846+0000 7f2a91844740 1 --2- 172.21.15.100:0/3072116826 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560628078da0 0x560628516f50 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:10.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.846+0000 7f2a91844740 1 --2- 172.21.15.100:0/3072116826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606280e5070 0x56062849e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:10.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.846+0000 7f2a91844740 1 --2- 172.21.15.100:0/3072116826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5606280c4390 0x56062849df00 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:10.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.846+0000 7f2a91844740 1 --2- 172.21.15.100:0/3072116826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606280bc1b0 0x5606284a2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:10.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.846+0000 7f2a91844740 1 -- 172.21.15.100:0/3072116826 >> 172.21.15.100:0/3072116826 conn(0x5606280d9460 msgr2=0x560628486ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:10.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.846+0000 7f2a6cff9640 0 client.16296 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:43:10.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.846+0000 7f2a91844740 1 -- 172.21.15.100:0/3072116826 shutdown_connections 2024-07-26T13:43:10.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.846+0000 7f2a91844740 1 -- 172.21.15.100:0/3072116826 wait complete. 2024-07-26T13:43:10.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.846+0000 7f2a91844740 20 client.16296 trim_cache size 0 max 16384 2024-07-26T13:43:10.855 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:43:10.855 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-07-26T13:43:10.855 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-07-26T13:43:10.855 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:43:10.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.847+0000 7f2a8b2c0640 1 -- 172.21.15.100:0/3029385713 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f2a700b3d40 msgr2=0x7f2a700b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:10.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.847+0000 7f2a8b2c0640 1 --2- 172.21.15.100:0/3029385713 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f2a700b3d40 0x7f2a700b6250 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c005fd0 tx=0x7f2a7c009500 comp rx=0 tx=0).stop 2024-07-26T13:43:10.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.847+0000 7f2a8b2c0640 1 -- 172.21.15.100:0/3029385713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a84108ed0 msgr2=0x7f2a8419a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:10.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.847+0000 7f2a8b2c0640 1 --2- 172.21.15.100:0/3029385713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a84108ed0 0x7f2a8419a460 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400e9f0 tx=0x7f2a7400eeb0 comp rx=0 tx=0).stop 2024-07-26T13:43:10.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.847+0000 7f2a8b2c0640 1 -- 172.21.15.100:0/3029385713 shutdown_connections 2024-07-26T13:43:10.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.847+0000 7f2a8b2c0640 1 --2- 172.21.15.100:0/3029385713 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f2a700b3d40 0x7f2a700b6250 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:10.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.847+0000 7f2a8b2c0640 1 --2- 172.21.15.100:0/3029385713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a84109820 0x7f2a8419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:10.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.847+0000 7f2a8b2c0640 1 --2- 172.21.15.100:0/3029385713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2a84108ed0 0x7f2a8419a460 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:10.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.847+0000 7f2a8b2c0640 1 --2- 172.21.15.100:0/3029385713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a84103590 0x7f2a84199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:10.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.847+0000 7f2a8b2c0640 1 -- 172.21.15.100:0/3029385713 >> 172.21.15.100:0/3029385713 conn(0x7f2a8407b280 msgr2=0x7f2a84106fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:10.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.847+0000 7f2a8b2c0640 1 -- 172.21.15.100:0/3029385713 shutdown_connections 2024-07-26T13:43:10.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:10.847+0000 7f2a8b2c0640 1 -- 172.21.15.100:0/3029385713 wait complete. 2024-07-26T13:43:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:10 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] up:resolve 2024-07-26T13:43:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:10 smithi138 ceph-mon[36954]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=e=up:resolve} 2 up:standby-replay 7 up:standby 2024-07-26T13:43:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:10 smithi138 ceph-mon[36954]: pgmap v457: 97 pgs: 97 active+clean; 53 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 966 KiB/s wr, 38 op/s 2024-07-26T13:43:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:10 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] up:resolve 2024-07-26T13:43:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:10 smithi167 ceph-mon[37140]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=e=up:resolve} 2 up:standby-replay 7 up:standby 2024-07-26T13:43:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:10 smithi167 ceph-mon[37140]: pgmap v457: 97 pgs: 97 active+clean; 53 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 966 KiB/s wr, 38 op/s 2024-07-26T13:43:11.407 INFO:tasks.workunit.client.0.smithi100.stdout: 1 82357 5.42 MB/sec warmup 85 sec latency 15598.048 ms 2024-07-26T13:43:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:11.611+0000 7ff015ffb640 1 -- 172.21.15.100:0/1041393924 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e80e3b1da0 con 0x55e80e0301c0 2024-07-26T13:43:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:11.611+0000 7ff015ffb640 10 client.16278 handle_command_reply: tid=0 2024-07-26T13:43:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:11.611+0000 7ff03b5f1740 1 client.16278 shutdown 2024-07-26T13:43:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:11.611+0000 7ff03b5f1740 1 -- 172.21.15.100:0/1041393924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e80e0732f0 msgr2=0x55e80e44f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:11.611+0000 7ff03b5f1740 1 --2- 172.21.15.100:0/1041393924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e80e0732f0 0x55e80e44f6b0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff01c0a7970 tx=0x7ff01c0e7290 comp rx=0 tx=0).stop 2024-07-26T13:43:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:11.611+0000 7ff03b5f1740 1 -- 172.21.15.100:0/1041393924 shutdown_connections 2024-07-26T13:43:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:11.611+0000 7ff03b5f1740 1 --2- 172.21.15.100:0/1041393924 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55e80e0301c0 0x55e80e4c6490 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:11.611+0000 7ff03b5f1740 1 --2- 172.21.15.100:0/1041393924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80e09c1b0 0x55e80e450130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:11.611+0000 7ff015ffb640 0 client.16278 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:43:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:11.611+0000 7ff03b5f1740 1 --2- 172.21.15.100:0/1041393924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e80e07b4d0 0x55e80e44fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:11.611+0000 7ff03b5f1740 1 --2- 172.21.15.100:0/1041393924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55e80e0732f0 0x55e80e44f6b0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:11.611+0000 7ff03b5f1740 1 -- 172.21.15.100:0/1041393924 >> 172.21.15.100:0/1041393924 conn(0x55e80e090300 msgr2=0x55e80e44ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:11.614 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:11.611+0000 7ff03b5f1740 1 -- 172.21.15.100:0/1041393924 shutdown_connections 2024-07-26T13:43:11.614 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:11.611+0000 7ff03b5f1740 1 -- 172.21.15.100:0/1041393924 wait complete. 2024-07-26T13:43:11.614 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:11.611+0000 7ff03b5f1740 20 client.16278 trim_cache size 0 max 16384 2024-07-26T13:43:11.614 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-07-26T13:43:11.614 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-07-26T13:43:11.614 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-07-26T13:43:11.614 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-07-26T13:43:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:11.612+0000 7ff0346e0640 1 -- 172.21.15.100:0/2033379943 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff0180b3d40 msgr2=0x7ff0180b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:11.612+0000 7ff0346e0640 1 --2- 172.21.15.100:0/2033379943 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff0180b3d40 0x7ff0180b6250 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff024006010 tx=0x7ff0240093f0 comp rx=0 tx=0).stop 2024-07-26T13:43:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:11.612+0000 7ff0346e0640 1 -- 172.21.15.100:0/2033379943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff02c102410 msgr2=0x7ff02c19ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:11.612+0000 7ff0346e0640 1 --2- 172.21.15.100:0/2033379943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff02c102410 0x7ff02c19ac50 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff01c00d950 tx=0x7ff01c00de10 comp rx=0 tx=0).stop 2024-07-26T13:43:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:11.612+0000 7ff0346e0640 1 -- 172.21.15.100:0/2033379943 shutdown_connections 2024-07-26T13:43:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:11.612+0000 7ff0346e0640 1 --2- 172.21.15.100:0/2033379943 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff0180b3d40 0x7ff0180b6250 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:11.612+0000 7ff0346e0640 1 --2- 172.21.15.100:0/2033379943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff02c108690 0x7ff02c19b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:11.612+0000 7ff0346e0640 1 --2- 172.21.15.100:0/2033379943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff02c107d40 0x7ff02c19b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:11.612+0000 7ff0346e0640 1 --2- 172.21.15.100:0/2033379943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff02c102410 0x7ff02c19ac50 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:11.612+0000 7ff0346e0640 1 -- 172.21.15.100:0/2033379943 >> 172.21.15.100:0/2033379943 conn(0x7ff02c0fde10 msgr2=0x7ff02c107350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:11.612+0000 7ff0346e0640 1 -- 172.21.15.100:0/2033379943 shutdown_connections 2024-07-26T13:43:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:11.612+0000 7ff0346e0640 1 -- 172.21.15.100:0/2033379943 wait complete. 2024-07-26T13:43:11.695 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-26T13:43:11.695 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:43:11.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:11 smithi138 ceph-mon[36954]: reconnect by client.15333 v1:192.168.0.1:0/1052122651 after 0.000999979 2024-07-26T13:43:11.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:11 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] up:reconnect 2024-07-26T13:43:11.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:11 smithi138 ceph-mon[36954]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=e=up:reconnect} 2 up:standby-replay 7 up:standby 2024-07-26T13:43:11.950 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:43:11.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:11 smithi100 ceph-mon[32599]: reconnect by client.15333 v1:192.168.0.1:0/1052122651 after 0.000999979 2024-07-26T13:43:11.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:11 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] up:reconnect 2024-07-26T13:43:11.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:11 smithi100 ceph-mon[32599]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=e=up:reconnect} 2 up:standby-replay 7 up:standby 2024-07-26T13:43:12.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:11 smithi167 ceph-mon[37140]: reconnect by client.15333 v1:192.168.0.1:0/1052122651 after 0.000999979 2024-07-26T13:43:12.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:11 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] up:reconnect 2024-07-26T13:43:12.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:11 smithi167 ceph-mon[37140]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=e=up:reconnect} 2 up:standby-replay 7 up:standby 2024-07-26T13:43:12.407 INFO:tasks.workunit.client.0.smithi100.stdout: 1 82357 5.36 MB/sec warmup 86 sec latency 16598.074 ms 2024-07-26T13:43:12.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:12 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] up:rejoin 2024-07-26T13:43:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:12 smithi100 ceph-mon[32599]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=e=up:rejoin} 2 up:standby-replay 7 up:standby 2024-07-26T13:43:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:12 smithi100 ceph-mon[32599]: pgmap v458: 97 pgs: 97 active+clean; 53 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.1 MiB/s wr, 40 op/s 2024-07-26T13:43:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:12 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] up:rejoin 2024-07-26T13:43:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:12 smithi138 ceph-mon[36954]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=e=up:rejoin} 2 up:standby-replay 7 up:standby 2024-07-26T13:43:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:12 smithi138 ceph-mon[36954]: pgmap v458: 97 pgs: 97 active+clean; 53 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.1 MiB/s wr, 40 op/s 2024-07-26T13:43:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:12 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] up:rejoin 2024-07-26T13:43:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:12 smithi167 ceph-mon[37140]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=e=up:rejoin} 2 up:standby-replay 7 up:standby 2024-07-26T13:43:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:12 smithi167 ceph-mon[37140]: pgmap v458: 97 pgs: 97 active+clean; 53 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.1 MiB/s wr, 40 op/s 2024-07-26T13:43:13.407 INFO:tasks.workunit.client.0.smithi100.stdout: 1 82358 5.30 MB/sec warmup 87 sec latency 16779.181 ms 2024-07-26T13:43:13.626 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:13 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] up:clientreplay 2024-07-26T13:43:13.626 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:13 smithi100 ceph-mon[32599]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=e=up:clientreplay} 2 up:standby-replay 7 up:standby 2024-07-26T13:43:13.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.714+0000 7f11edb37640 1 -- 172.21.15.100:0/3778892506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11e8108750 msgr2=0x7f11e810ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:13.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.714+0000 7f11edb37640 1 --2- 172.21.15.100:0/3778892506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11e8108750 0x7f11e810ef80 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f11e0009900 tx=0x7f11e002a3f0 comp rx=0 tx=0).stop 2024-07-26T13:43:13.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.714+0000 7f11edb37640 1 -- 172.21.15.100:0/3778892506 shutdown_connections 2024-07-26T13:43:13.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.714+0000 7f11edb37640 1 --2- 172.21.15.100:0/3778892506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11e8108750 0x7f11e810ef80 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:13.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.714+0000 7f11edb37640 1 --2- 172.21.15.100:0/3778892506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11e8107e00 0x7f11e8108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:13.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.714+0000 7f11edb37640 1 --2- 172.21.15.100:0/3778892506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11e81024c0 0x7f11e81028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:13.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.714+0000 7f11edb37640 1 -- 172.21.15.100:0/3778892506 >> 172.21.15.100:0/3778892506 conn(0x7f11e8079160 msgr2=0x7f11e8079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:13.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.714+0000 7f11edb37640 1 -- 172.21.15.100:0/3778892506 shutdown_connections 2024-07-26T13:43:13.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.714+0000 7f11edb37640 1 -- 172.21.15.100:0/3778892506 wait complete. 2024-07-26T13:43:13.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.715+0000 7f11edb37640 1 Processor -- start 2024-07-26T13:43:13.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.715+0000 7f11edb37640 1 -- start start 2024-07-26T13:43:13.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.715+0000 7f11edb37640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11e81024c0 0x7f11e8198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:13.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.715+0000 7f11edb37640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11e8107e00 0x7f11e81992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:13.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.715+0000 7f11edb37640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11e8108750 0x7f11e819e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:13.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.715+0000 7f11edb37640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f11e8112400 con 0x7f11e8108750 2024-07-26T13:43:13.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.715+0000 7f11edb37640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f11e8112280 con 0x7f11e81024c0 2024-07-26T13:43:13.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.715+0000 7f11edb37640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f11e8112580 con 0x7f11e8107e00 2024-07-26T13:43:13.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.716+0000 7f11e7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11e8108750 0x7f11e819e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:13.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.716+0000 7f11e7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11e8108750 0x7f11e819e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37634/0 (socket says 172.21.15.100:37634) 2024-07-26T13:43:13.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.716+0000 7f11e7fff640 1 -- 172.21.15.100:0/579199045 learned_addr learned my addr 172.21.15.100:0/579199045 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:13.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.716+0000 7f11e77fe640 1 --2- 172.21.15.100:0/579199045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11e81024c0 0x7f11e8198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:13.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.716+0000 7f11e6ffd640 1 --2- 172.21.15.100:0/579199045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11e8107e00 0x7f11e81992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:13.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.716+0000 7f11e7fff640 1 -- 172.21.15.100:0/579199045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11e8107e00 msgr2=0x7f11e81992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:13.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.716+0000 7f11e7fff640 1 --2- 172.21.15.100:0/579199045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11e8107e00 0x7f11e81992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:13.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.716+0000 7f11e7fff640 1 -- 172.21.15.100:0/579199045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11e81024c0 msgr2=0x7f11e8198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:13.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.716+0000 7f11e7fff640 1 --2- 172.21.15.100:0/579199045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11e81024c0 0x7f11e8198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:13.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.716+0000 7f11e7fff640 1 -- 172.21.15.100:0/579199045 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e819ebd0 con 0x7f11e8108750 2024-07-26T13:43:13.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.716+0000 7f11e77fe640 1 --2- 172.21.15.100:0/579199045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11e81024c0 0x7f11e8198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:43:13.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.716+0000 7f11e7fff640 1 --2- 172.21.15.100:0/579199045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11e8108750 0x7f11e819e560 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f11e0009a20 tx=0x7f11e0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:13.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.716+0000 7f11e4ff9640 1 -- 172.21.15.100:0/579199045 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e003f030 con 0x7f11e8108750 2024-07-26T13:43:13.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.716+0000 7f11edb37640 1 -- 172.21.15.100:0/579199045 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11e819ee60 con 0x7f11e8108750 2024-07-26T13:43:13.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.716+0000 7f11e4ff9640 1 -- 172.21.15.100:0/579199045 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11e0003aa0 con 0x7f11e8108750 2024-07-26T13:43:13.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.716+0000 7f11e4ff9640 1 -- 172.21.15.100:0/579199045 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e0035070 con 0x7f11e8108750 2024-07-26T13:43:13.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.716+0000 7f11edb37640 1 -- 172.21.15.100:0/579199045 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11e819f340 con 0x7f11e8108750 2024-07-26T13:43:13.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.717+0000 7f11edb37640 1 -- 172.21.15.100:0/579199045 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11e8103740 con 0x7f11e8108750 2024-07-26T13:43:13.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.719+0000 7f11e4ff9640 1 -- 172.21.15.100:0/579199045 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f11e0042030 con 0x7f11e8108750 2024-07-26T13:43:13.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.720+0000 7f11e4ff9640 1 --2- 172.21.15.100:0/579199045 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f11d40b3d70 0x7f11d40b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:13.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.720+0000 7f11e77fe640 1 --2- 172.21.15.100:0/579199045 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f11d40b3d70 0x7f11d40b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:13.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.720+0000 7f11e4ff9640 1 -- 172.21.15.100:0/579199045 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f11e0104a40 con 0x7f11e8108750 2024-07-26T13:43:13.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.720+0000 7f11e77fe640 1 --2- 172.21.15.100:0/579199045 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f11d40b3d70 0x7f11d40b6280 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f11d8007e70 tx=0x7f11d8007850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:13.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.723+0000 7f11e4ff9640 1 -- 172.21.15.100:0/579199045 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f11e00cbd20 con 0x7f11e8108750 2024-07-26T13:43:13.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.929+0000 7f11edb37640 1 -- 172.21.15.100:0/579199045 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11e81028d0 con 0x7f11e8108750 2024-07-26T13:43:13.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.930+0000 7f11e4ff9640 1 -- 172.21.15.100:0/579199045 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11925 (secure 0 0 0) 0x7f11e81028d0 con 0x7f11e8108750 2024-07-26T13:43:13.931 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:43:13.931 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":34,"btime":"2024-07-26T13:43:12:582829+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14790,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/313501409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":313501409},{"type":"v1","addr":"172.21.15.100:6839","nonce":313501409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/1402975506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":1402975506},{"type":"v1","addr":"172.21.15.100:6841","nonce":1402975506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":25762,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2476068386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2476068386},{"type":"v1","addr":"172.21.15.167:6837","nonce":2476068386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-26T13:37:01.281843+0000","modified":"2024-07-26T13:43:11.591975+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24607,"mds_1":24605,"mds_2":24557},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24557":{"gid":24557,"name":"e","rank":2,"incarnation":30,"state":"up:clientreplay","state_seq":106,"addr":"172.21.15.138:6835/3290105300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3290105300},{"type":"v1","addr":"172.21.15.138:6835","nonce":3290105300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/3393225483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3393225483},{"type":"v1","addr":"172.21.15.138:6837","nonce":3393225483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.138:6839/1580925471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1580925471},{"type":"v1","addr":"172.21.15.138:6839","nonce":1580925471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/398150514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":398150514},{"type":"v1","addr":"172.21.15.167:6839","nonce":398150514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24607,"qdb_cluster":[24607,24605]},"id":1}]} 2024-07-26T13:43:13.932 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 34 2024-07-26T13:43:13.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.932+0000 7f11edb37640 1 -- 172.21.15.100:0/579199045 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f11d40b3d70 msgr2=0x7f11d40b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:13.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.932+0000 7f11edb37640 1 --2- 172.21.15.100:0/579199045 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f11d40b3d70 0x7f11d40b6280 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f11d8007e70 tx=0x7f11d8007850 comp rx=0 tx=0).stop 2024-07-26T13:43:13.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.932+0000 7f11edb37640 1 -- 172.21.15.100:0/579199045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11e8108750 msgr2=0x7f11e819e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:13.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.932+0000 7f11edb37640 1 --2- 172.21.15.100:0/579199045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11e8108750 0x7f11e819e560 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f11e0009a20 tx=0x7f11e0004270 comp rx=0 tx=0).stop 2024-07-26T13:43:13.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.932+0000 7f11edb37640 1 -- 172.21.15.100:0/579199045 shutdown_connections 2024-07-26T13:43:13.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.932+0000 7f11edb37640 1 --2- 172.21.15.100:0/579199045 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f11d40b3d70 0x7f11d40b6280 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:13.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.932+0000 7f11edb37640 1 --2- 172.21.15.100:0/579199045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11e8108750 0x7f11e819e560 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:13.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.932+0000 7f11edb37640 1 --2- 172.21.15.100:0/579199045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11e8107e00 0x7f11e81992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:13.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.932+0000 7f11edb37640 1 --2- 172.21.15.100:0/579199045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11e81024c0 0x7f11e8198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:13.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.932+0000 7f11edb37640 1 -- 172.21.15.100:0/579199045 >> 172.21.15.100:0/579199045 conn(0x7f11e8079160 msgr2=0x7f11e8105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:13.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.932+0000 7f11edb37640 1 -- 172.21.15.100:0/579199045 shutdown_connections 2024-07-26T13:43:13.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:13.932+0000 7f11edb37640 1 -- 172.21.15.100:0/579199045 wait complete. 2024-07-26T13:43:13.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:13 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] up:clientreplay 2024-07-26T13:43:13.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:13 smithi138 ceph-mon[36954]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=e=up:clientreplay} 2 up:standby-replay 7 up:standby 2024-07-26T13:43:14.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:13 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] up:clientreplay 2024-07-26T13:43:14.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:13 smithi167 ceph-mon[37140]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=e=up:clientreplay} 2 up:standby-replay 7 up:standby 2024-07-26T13:43:14.408 INFO:tasks.workunit.client.0.smithi100.stdout: 1 82916 5.26 MB/sec warmup 88 sec latency 1606.838 ms 2024-07-26T13:43:14.686 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-26T13:43:14.852 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:14 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/579199045' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:43:14.852 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:14 smithi100 ceph-mon[32599]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-07-26T13:43:14.852 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:14 smithi100 ceph-mon[32599]: pgmap v459: 97 pgs: 97 active+clean; 53 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.7 MiB/s wr, 76 op/s 2024-07-26T13:43:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:14 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/579199045' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:43:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:14 smithi138 ceph-mon[36954]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-07-26T13:43:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:14 smithi138 ceph-mon[36954]: pgmap v459: 97 pgs: 97 active+clean; 53 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.7 MiB/s wr, 76 op/s 2024-07-26T13:43:15.022 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:43:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:14 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/579199045' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:43:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:14 smithi167 ceph-mon[37140]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-07-26T13:43:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:14 smithi167 ceph-mon[37140]: pgmap v459: 97 pgs: 97 active+clean; 53 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.7 MiB/s wr, 76 op/s 2024-07-26T13:43:15.408 INFO:tasks.workunit.client.0.smithi100.stdout: 1 84243 5.27 MB/sec warmup 89 sec latency 255.069 ms 2024-07-26T13:43:15.522 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:43:15 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 2024-07-26T13:43:15.265+0000 7f740ace1640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-07-26T13:43:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:15 smithi100 ceph-mon[32599]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-26T13:43:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:15 smithi100 ceph-mon[32599]: Cluster is now healthy 2024-07-26T13:43:15.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:15 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] up:active 2024-07-26T13:43:15.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:15 smithi100 ceph-mon[32599]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:43:15.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:15 smithi100 ceph-mon[32599]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:43:15.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:15 smithi138 ceph-mon[36954]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-26T13:43:15.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:15 smithi138 ceph-mon[36954]: Cluster is now healthy 2024-07-26T13:43:15.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:15 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] up:active 2024-07-26T13:43:15.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:15 smithi138 ceph-mon[36954]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:43:15.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:15 smithi138 ceph-mon[36954]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:43:16.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:15 smithi167 ceph-mon[37140]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-26T13:43:16.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:15 smithi167 ceph-mon[37140]: Cluster is now healthy 2024-07-26T13:43:16.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:15 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] up:active 2024-07-26T13:43:16.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:15 smithi167 ceph-mon[37140]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:43:16.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:15 smithi167 ceph-mon[37140]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:43:16.408 INFO:tasks.workunit.client.0.smithi100.stdout: 1 85788 5.33 MB/sec warmup 90 sec latency 12.927 ms 2024-07-26T13:43:16.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:16 smithi100 ceph-mon[32599]: pgmap v460: 97 pgs: 97 active+clean; 52 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 1.9 MiB/s wr, 113 op/s 2024-07-26T13:43:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:16 smithi138 ceph-mon[36954]: pgmap v460: 97 pgs: 97 active+clean; 52 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 1.9 MiB/s wr, 113 op/s 2024-07-26T13:43:17.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:16 smithi167 ceph-mon[37140]: pgmap v460: 97 pgs: 97 active+clean; 52 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 1.9 MiB/s wr, 113 op/s 2024-07-26T13:43:17.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.041+0000 7f79c1e7c640 1 -- 172.21.15.100:0/1911689256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79bc107dd0 msgr2=0x7f79bc1081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:17.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.041+0000 7f79c1e7c640 1 --2- 172.21.15.100:0/1911689256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79bc107dd0 0x7f79bc1081e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f79b0009900 tx=0x7f79b002a410 comp rx=0 tx=0).stop 2024-07-26T13:43:17.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.041+0000 7f79c1e7c640 1 -- 172.21.15.100:0/1911689256 shutdown_connections 2024-07-26T13:43:17.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.041+0000 7f79c1e7c640 1 --2- 172.21.15.100:0/1911689256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79bc108720 0x7f79bc10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:17.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.041+0000 7f79c1e7c640 1 --2- 172.21.15.100:0/1911689256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79bc107dd0 0x7f79bc1081e0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:17.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.041+0000 7f79c1e7c640 1 --2- 172.21.15.100:0/1911689256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79bc102490 0x7f79bc1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:17.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.041+0000 7f79c1e7c640 1 -- 172.21.15.100:0/1911689256 >> 172.21.15.100:0/1911689256 conn(0x7f79bc079160 msgr2=0x7f79bc079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:17.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.041+0000 7f79c1e7c640 1 -- 172.21.15.100:0/1911689256 shutdown_connections 2024-07-26T13:43:17.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.041+0000 7f79c1e7c640 1 -- 172.21.15.100:0/1911689256 wait complete. 2024-07-26T13:43:17.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.042+0000 7f79c1e7c640 1 Processor -- start 2024-07-26T13:43:17.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.042+0000 7f79c1e7c640 1 -- start start 2024-07-26T13:43:17.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.042+0000 7f79c1e7c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79bc102490 0x7f79bc19ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:17.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.042+0000 7f79c1e7c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79bc107dd0 0x7f79bc19b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:17.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.042+0000 7f79c1e7c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79bc108720 0x7f79bc19b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:17.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.042+0000 7f79c1e7c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f79bc112350 con 0x7f79bc102490 2024-07-26T13:43:17.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.042+0000 7f79c1e7c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f79bc1121d0 con 0x7f79bc107dd0 2024-07-26T13:43:17.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.042+0000 7f79c1e7c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f79bc1124d0 con 0x7f79bc108720 2024-07-26T13:43:17.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.043+0000 7f79bb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79bc102490 0x7f79bc19ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:17.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.043+0000 7f79bb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79bc102490 0x7f79bc19ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56132/0 (socket says 172.21.15.100:56132) 2024-07-26T13:43:17.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.043+0000 7f79bb7fe640 1 -- 172.21.15.100:0/3614282463 learned_addr learned my addr 172.21.15.100:0/3614282463 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:17.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.043+0000 7f79baffd640 1 --2- 172.21.15.100:0/3614282463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79bc107dd0 0x7f79bc19b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:17.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.043+0000 7f79bbfff640 1 --2- 172.21.15.100:0/3614282463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79bc108720 0x7f79bc19b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:17.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.043+0000 7f79bb7fe640 1 -- 172.21.15.100:0/3614282463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79bc108720 msgr2=0x7f79bc19b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:17.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.043+0000 7f79bb7fe640 1 --2- 172.21.15.100:0/3614282463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79bc108720 0x7f79bc19b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:17.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.043+0000 7f79bb7fe640 1 -- 172.21.15.100:0/3614282463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79bc107dd0 msgr2=0x7f79bc19b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:17.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.043+0000 7f79bb7fe640 1 --2- 172.21.15.100:0/3614282463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79bc107dd0 0x7f79bc19b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:17.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.043+0000 7f79bb7fe640 1 -- 172.21.15.100:0/3614282463 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79bc1a6100 con 0x7f79bc102490 2024-07-26T13:43:17.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.043+0000 7f79bb7fe640 1 --2- 172.21.15.100:0/3614282463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79bc102490 0x7f79bc19ae40 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f79ac00e9f0 tx=0x7f79ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:17.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.043+0000 7f79b8ff9640 1 -- 172.21.15.100:0/3614282463 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac00cd80 con 0x7f79bc102490 2024-07-26T13:43:17.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.043+0000 7f79c1e7c640 1 -- 172.21.15.100:0/3614282463 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79bc1a63f0 con 0x7f79bc102490 2024-07-26T13:43:17.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.043+0000 7f79b8ff9640 1 -- 172.21.15.100:0/3614282463 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79ac0108e0 con 0x7f79bc102490 2024-07-26T13:43:17.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.043+0000 7f79b8ff9640 1 -- 172.21.15.100:0/3614282463 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac018980 con 0x7f79bc102490 2024-07-26T13:43:17.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.043+0000 7f79c1e7c640 1 -- 172.21.15.100:0/3614282463 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79bc1a6950 con 0x7f79bc102490 2024-07-26T13:43:17.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.046+0000 7f79c8400740 1 Processor -- start 2024-07-26T13:43:17.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.047+0000 7f79c8400740 1 -- start start 2024-07-26T13:43:17.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.047+0000 7f79b8ff9640 1 -- 172.21.15.100:0/3614282463 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f79ac010430 con 0x7f79bc102490 2024-07-26T13:43:17.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.047+0000 7f79c8400740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba3903070 0x55fba3ca8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:17.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.047+0000 7f79c8400740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fba38da1b0 0x55fba3ca8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:17.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.047+0000 7f79c8400740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fba38e2390 0x55fba3ca8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:17.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.047+0000 7f79c8400740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55fba3c1ebb0 con 0x55fba3903070 2024-07-26T13:43:17.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.047+0000 7f79c8400740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fba3c0e0b0 con 0x55fba38da1b0 2024-07-26T13:43:17.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.047+0000 7f79c8400740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55fba3c120f0 con 0x55fba38e2390 2024-07-26T13:43:17.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.047+0000 7f79bbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba3903070 0x55fba3ca8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:17.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.047+0000 7f79bbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba3903070 0x55fba3ca8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56134/0 (socket says 172.21.15.100:56134) 2024-07-26T13:43:17.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.047+0000 7f79bb7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fba38da1b0 0x55fba3ca8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:17.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.047+0000 7f79bbfff640 1 -- 172.21.15.100:0/2718041053 learned_addr learned my addr 172.21.15.100:0/2718041053 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:17.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.047+0000 7f79baffd640 1 --2- 172.21.15.100:0/2718041053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fba38e2390 0x55fba3ca8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:17.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.048+0000 7f79bbfff640 1 -- 172.21.15.100:0/2718041053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fba38e2390 msgr2=0x55fba3ca8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:17.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.048+0000 7f79bbfff640 1 --2- 172.21.15.100:0/2718041053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fba38e2390 0x55fba3ca8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:17.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.048+0000 7f79bbfff640 1 -- 172.21.15.100:0/2718041053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fba38da1b0 msgr2=0x55fba3ca8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:17.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.048+0000 7f79bbfff640 1 --2- 172.21.15.100:0/2718041053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fba38da1b0 0x55fba3ca8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:17.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.048+0000 7f79bbfff640 1 -- 172.21.15.100:0/2718041053 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fba3bb4b90 con 0x55fba3903070 2024-07-26T13:43:17.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.048+0000 7f79bbfff640 1 --2- 172.21.15.100:0/2718041053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba3903070 0x55fba3ca8570 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f79b4009f90 tx=0x7f79b4010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1924b8498eeb3d30 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:17.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.048+0000 7f79b8ff9640 1 --2- 172.21.15.100:0/3614282463 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f79a40b3dc0 0x7f79a40b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:17.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.048+0000 7f79a1ffb640 1 -- 172.21.15.100:0/2718041053 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b4011700 con 0x55fba3903070 2024-07-26T13:43:17.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.048+0000 7f79b8ff9640 1 -- 172.21.15.100:0/3614282463 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f79ac0e42d0 con 0x7f79bc102490 2024-07-26T13:43:17.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.048+0000 7f79bb7fe640 1 --2- 172.21.15.100:0/3614282463 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f79a40b3dc0 0x7f79a40b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:17.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.048+0000 7f79a1ffb640 1 -- 172.21.15.100:0/2718041053 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79b4011d50 con 0x55fba3903070 2024-07-26T13:43:17.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.048+0000 7f79a1ffb640 1 -- 172.21.15.100:0/2718041053 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b4019d40 con 0x55fba3903070 2024-07-26T13:43:17.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.049+0000 7f79c8400740 1 -- 172.21.15.100:0/2718041053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba3903070 msgr2=0x55fba3ca8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:17.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.049+0000 7f79bb7fe640 1 --2- 172.21.15.100:0/3614282463 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f79a40b3dc0 0x7f79a40b62d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f79ac0e1aa0 tx=0x7f79ac0e7e20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:17.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.049+0000 7f79c8400740 1 --2- 172.21.15.100:0/2718041053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba3903070 0x55fba3ca8570 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f79b4009f90 tx=0x7f79b4010860 comp rx=0 tx=0).stop 2024-07-26T13:43:17.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.049+0000 7f79c8400740 1 -- 172.21.15.100:0/2718041053 shutdown_connections 2024-07-26T13:43:17.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.049+0000 7f79c8400740 1 --2- 172.21.15.100:0/2718041053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fba38e2390 0x55fba3ca8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:17.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.049+0000 7f79c8400740 1 --2- 172.21.15.100:0/2718041053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fba38da1b0 0x55fba3ca8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:17.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.049+0000 7f79c8400740 1 --2- 172.21.15.100:0/2718041053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba3903070 0x55fba3ca8570 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:17.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.050+0000 7f79c8400740 1 -- 172.21.15.100:0/2718041053 >> 172.21.15.100:0/2718041053 conn(0x55fba38f7460 msgr2=0x55fba3ca6460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:17.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.050+0000 7f79c8400740 1 -- 172.21.15.100:0/2718041053 shutdown_connections 2024-07-26T13:43:17.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.050+0000 7f79c8400740 1 -- 172.21.15.100:0/2718041053 wait complete. 2024-07-26T13:43:17.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.050+0000 7f79c8400740 1 -- start start 2024-07-26T13:43:17.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.050+0000 7f79c8400740 1 Processor -- start 2024-07-26T13:43:17.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.051+0000 7f79c8400740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fba38da1b0 0x55fba3cb4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:17.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.051+0000 7f79c8400740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba38e2390 0x55fba3cb4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:17.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.051+0000 7f79c8400740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fba3903070 0x55fba3cc4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:17.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.051+0000 7f79c8400740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55fba3a71700 con 0x55fba38e2390 2024-07-26T13:43:17.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.051+0000 7f79c8400740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fba3bb09e0 con 0x55fba38da1b0 2024-07-26T13:43:17.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.051+0000 7f79c8400740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55fba3cbb160 con 0x55fba3903070 2024-07-26T13:43:17.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.051+0000 7f79c8400740 10 client.0 _ll_register_callbacks cb 0x55fba3bcb4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:43:17.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.051+0000 7f79baffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba38e2390 0x55fba3cb4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:17.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.051+0000 7f79bbfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fba38da1b0 0x55fba3cb4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:17.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.051+0000 7f79baffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba38e2390 0x55fba3cb4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56138/0 (socket says 172.21.15.100:56138) 2024-07-26T13:43:17.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.051+0000 7f79bbfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fba38da1b0 0x55fba3cb4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:58884/0 (socket says 172.21.15.100:58884) 2024-07-26T13:43:17.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.051+0000 7f79baffd640 1 -- 172.21.15.100:0/2748207501 learned_addr learned my addr 172.21.15.100:0/2748207501 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:17.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.051+0000 7f79bbfff640 1 --2- 172.21.15.100:0/2748207501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fba3903070 0x55fba3cc4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:17.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.051+0000 7f79bbfff640 1 -- 172.21.15.100:0/2748207501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fba3903070 msgr2=0x55fba3cc4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:17.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.051+0000 7f79bbfff640 1 --2- 172.21.15.100:0/2748207501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fba3903070 0x55fba3cc4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:17.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.051+0000 7f79bbfff640 1 -- 172.21.15.100:0/2748207501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba38e2390 msgr2=0x55fba3cb4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:17.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.051+0000 7f79bbfff640 1 --2- 172.21.15.100:0/2748207501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba38e2390 0x55fba3cb4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:17.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.051+0000 7f79bbfff640 1 -- 172.21.15.100:0/2748207501 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fba396e650 con 0x55fba38da1b0 2024-07-26T13:43:17.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.051+0000 7f79baffd640 1 --2- 172.21.15.100:0/2748207501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba38e2390 0x55fba3cb4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:43:17.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.051+0000 7f79bbfff640 1 --2- 172.21.15.100:0/2748207501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fba38da1b0 0x55fba3cb4100 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f79b4009f90 tx=0x7f79b4013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c42d1cdfcad410d2 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:17.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.051+0000 7f799affd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-07-26T13:43:17.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.052+0000 7f799affd640 1 -- 172.21.15.100:0/2748207501 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b402d060 con 0x55fba38da1b0 2024-07-26T13:43:17.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.052+0000 7f799affd640 1 -- 172.21.15.100:0/2748207501 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f79b4019ce0 con 0x55fba38da1b0 2024-07-26T13:43:17.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.052+0000 7f79c8400740 1 -- 172.21.15.100:0/2748207501 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79b4011d50 con 0x55fba38da1b0 2024-07-26T13:43:17.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.052+0000 7f799affd640 1 -- 172.21.15.100:0/2748207501 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f79b4004a40 con 0x55fba38da1b0 2024-07-26T13:43:17.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.052+0000 7f79c8400740 10 client.25676 fetch_fsmap learned FSMap version 36 2024-07-26T13:43:17.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.052+0000 7f79c8400740 1 -- 172.21.15.100:0/2748207501 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55fba3cc7140 con 0x55fba38da1b0 2024-07-26T13:43:17.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.052+0000 7f799affd640 1 -- 172.21.15.100:0/2748207501 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79b4011d50 con 0x55fba38da1b0 2024-07-26T13:43:17.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.052+0000 7f799affd640 1 -- 172.21.15.100:0/2748207501 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f79b4025070 con 0x55fba38da1b0 2024-07-26T13:43:17.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.052+0000 7f79c8400740 10 client.25676 fetch_fsmap finished waiting for FSMap version 36 2024-07-26T13:43:17.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.052+0000 7f79c8400740 10 client.25676 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-07-26T13:43:17.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.052+0000 7f79c8400740 20 client.25676 populate_metadata read hostname 'smithi100' 2024-07-26T13:43:17.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.052+0000 7f79c8400740 1 --2- 172.21.15.100:0/2748207501 >> [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] conn(0x55fba38c19f0 0x55fba3cca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:17.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.052+0000 7f79c8400740 4 client.25676 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:43:17.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.052+0000 7f79c8400740 1 -- 172.21.15.100:0/2748207501 --> [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fba3c20b00 con 0x55fba38c19f0 2024-07-26T13:43:17.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.052+0000 7f79baffd640 1 --2- 172.21.15.100:0/2748207501 >> [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] conn(0x55fba38c19f0 0x55fba3cca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:17.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.053+0000 7f799affd640 10 client.25676 ms_handle_connect on v2:172.21.15.138:6834/3290105300 2024-07-26T13:43:17.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.053+0000 7f79baffd640 1 --2- 172.21.15.100:0/2748207501 >> [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] conn(0x55fba38c19f0 0x55fba3cca5b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8baaa2ebb0b3f561 server_cookie=632b2ba0c1cf6bdc in_seq=0 out_seq=0 2024-07-26T13:43:17.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.054+0000 7f799affd640 1 -- 172.21.15.100:0/2748207501 <== mds.2 v2:172.21.15.138:6834/3290105300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fba3c20b00 con 0x55fba38c19f0 2024-07-26T13:43:17.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.054+0000 7f799affd640 10 client.25676 handle_command_reply: tid=0 2024-07-26T13:43:17.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.054+0000 7f79c8400740 1 client.25676 shutdown 2024-07-26T13:43:17.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.054+0000 7f79c8400740 1 -- 172.21.15.100:0/2748207501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fba38da1b0 msgr2=0x55fba3cb4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:17.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.054+0000 7f79c8400740 1 --2- 172.21.15.100:0/2748207501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fba38da1b0 0x55fba3cb4100 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f79b4009f90 tx=0x7f79b4013040 comp rx=0 tx=0).stop 2024-07-26T13:43:17.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.054+0000 7f79c8400740 1 -- 172.21.15.100:0/2748207501 shutdown_connections 2024-07-26T13:43:17.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.054+0000 7f79c8400740 1 --2- 172.21.15.100:0/2748207501 >> [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] conn(0x55fba38c19f0 0x55fba3cca5b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:17.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.054+0000 7f79c8400740 1 --2- 172.21.15.100:0/2748207501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fba3903070 0x55fba3cc4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:17.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.054+0000 7f799affd640 0 client.25676 ms_handle_reset on v2:172.21.15.138:6834/3290105300 2024-07-26T13:43:17.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.054+0000 7f79c8400740 1 --2- 172.21.15.100:0/2748207501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba38e2390 0x55fba3cb4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:17.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.054+0000 7f79c8400740 1 --2- 172.21.15.100:0/2748207501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fba38da1b0 0x55fba3cb4100 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:17.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.054+0000 7f79c8400740 1 -- 172.21.15.100:0/2748207501 >> 172.21.15.100:0/2748207501 conn(0x55fba38f7460 msgr2=0x55fba3cab750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:17.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.054+0000 7f79c8400740 1 -- 172.21.15.100:0/2748207501 shutdown_connections 2024-07-26T13:43:17.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.054+0000 7f79c8400740 1 -- 172.21.15.100:0/2748207501 wait complete. 2024-07-26T13:43:17.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.055+0000 7f79c8400740 20 client.25676 trim_cache size 0 max 16384 2024-07-26T13:43:17.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.068+0000 7f79c8400740 1 Processor -- start 2024-07-26T13:43:17.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.068+0000 7f79c8400740 1 -- start start 2024-07-26T13:43:17.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.068+0000 7f79c8400740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fba38c19f0 0x55fba3cc43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:17.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.068+0000 7f79c8400740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba38da1b0 0x55fba3cb5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:17.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.068+0000 7f79c8400740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fba38e2390 0x55fba3cad650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:17.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.068+0000 7f79c8400740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f79bc1af900 con 0x55fba38da1b0 2024-07-26T13:43:17.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.068+0000 7f79c8400740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f79bc1ad170 con 0x55fba38c19f0 2024-07-26T13:43:17.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.068+0000 7f79c8400740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f79bc1aafc0 con 0x55fba38e2390 2024-07-26T13:43:17.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.068+0000 7f79bbfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fba38c19f0 0x55fba3cc43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:17.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.068+0000 7f79baffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba38da1b0 0x55fba3cb5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:17.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.068+0000 7f79baffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba38da1b0 0x55fba3cb5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56142/0 (socket says 172.21.15.100:56142) 2024-07-26T13:43:17.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.068+0000 7f79bbfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fba38c19f0 0x55fba3cc43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:58892/0 (socket says 172.21.15.100:58892) 2024-07-26T13:43:17.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.068+0000 7f79baffd640 1 -- 172.21.15.100:0/2067328500 learned_addr learned my addr 172.21.15.100:0/2067328500 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:17.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.068+0000 7f79bbfff640 1 --2- 172.21.15.100:0/2067328500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fba38e2390 0x55fba3cad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:17.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.069+0000 7f79baffd640 1 -- 172.21.15.100:0/2067328500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fba38e2390 msgr2=0x55fba3cad650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:17.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.069+0000 7f79baffd640 1 --2- 172.21.15.100:0/2067328500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fba38e2390 0x55fba3cad650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:17.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.069+0000 7f79baffd640 1 -- 172.21.15.100:0/2067328500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fba38c19f0 msgr2=0x55fba3cc43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:17.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.069+0000 7f79baffd640 1 --2- 172.21.15.100:0/2067328500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fba38c19f0 0x55fba3cc43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:17.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.069+0000 7f79baffd640 1 -- 172.21.15.100:0/2067328500 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fba3cd0790 con 0x55fba38da1b0 2024-07-26T13:43:17.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.069+0000 7f79bbfff640 1 --2- 172.21.15.100:0/2067328500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fba38c19f0 0x55fba3cc43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:43:17.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.069+0000 7f79baffd640 1 --2- 172.21.15.100:0/2067328500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba38da1b0 0x55fba3cb5b30 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f79b0005940 tx=0x7f79b0031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=803d9f109a37bb5b server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:17.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.069+0000 7f79a17fa640 1 -- 172.21.15.100:0/2067328500 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b004b070 con 0x55fba38da1b0 2024-07-26T13:43:17.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.069+0000 7f79a17fa640 1 -- 172.21.15.100:0/2067328500 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79b00054f0 con 0x55fba38da1b0 2024-07-26T13:43:17.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.069+0000 7f79a17fa640 1 -- 172.21.15.100:0/2067328500 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b0035040 con 0x55fba38da1b0 2024-07-26T13:43:17.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.070+0000 7f79c8400740 1 -- 172.21.15.100:0/2067328500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba38da1b0 msgr2=0x55fba3cb5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:17.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.070+0000 7f79c8400740 1 --2- 172.21.15.100:0/2067328500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba38da1b0 0x55fba3cb5b30 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f79b0005940 tx=0x7f79b0031400 comp rx=0 tx=0).stop 2024-07-26T13:43:17.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.070+0000 7f79c8400740 1 -- 172.21.15.100:0/2067328500 shutdown_connections 2024-07-26T13:43:17.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.070+0000 7f79c8400740 1 --2- 172.21.15.100:0/2067328500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fba38e2390 0x55fba3cad650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:17.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.070+0000 7f79c8400740 1 --2- 172.21.15.100:0/2067328500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba38da1b0 0x55fba3cb5b30 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:17.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.070+0000 7f79c8400740 1 --2- 172.21.15.100:0/2067328500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fba38c19f0 0x55fba3cc43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:17.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.070+0000 7f79c8400740 1 -- 172.21.15.100:0/2067328500 >> 172.21.15.100:0/2067328500 conn(0x55fba38f7460 msgr2=0x55fba3cb5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:17.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.070+0000 7f79c8400740 1 -- 172.21.15.100:0/2067328500 shutdown_connections 2024-07-26T13:43:17.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.070+0000 7f79c8400740 1 -- 172.21.15.100:0/2067328500 wait complete. 2024-07-26T13:43:17.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.070+0000 7f79c8400740 1 -- start start 2024-07-26T13:43:17.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.071+0000 7f79c8400740 1 Processor -- start 2024-07-26T13:43:17.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.071+0000 7f79c8400740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fba38c19f0 0x55fba3cc8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:17.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.071+0000 7f79c8400740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba38da1b0 0x55fba3cc9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:17.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.071+0000 7f79c8400740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fba38e2390 0x55fba3d2ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:17.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.071+0000 7f79c8400740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f79bc1ab630 con 0x55fba38da1b0 2024-07-26T13:43:17.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.071+0000 7f79c8400740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f79bc1acaa0 con 0x55fba38c19f0 2024-07-26T13:43:17.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.071+0000 7f79c8400740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f79bc1abd00 con 0x55fba38e2390 2024-07-26T13:43:17.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.071+0000 7f79c8400740 10 client.0 _ll_register_callbacks cb 0x7f79ac0e7cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:43:17.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.071+0000 7f79baffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba38da1b0 0x55fba3cc9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:17.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.071+0000 7f79baffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba38da1b0 0x55fba3cc9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56156/0 (socket says 172.21.15.100:56156) 2024-07-26T13:43:17.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.071+0000 7f79baffd640 1 -- 172.21.15.100:0/1604170798 learned_addr learned my addr 172.21.15.100:0/1604170798 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:17.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.071+0000 7f79bbfff640 1 --2- 172.21.15.100:0/1604170798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fba38c19f0 0x55fba3cc8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:17.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.071+0000 7f79bbfff640 1 --2- 172.21.15.100:0/1604170798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fba38e2390 0x55fba3d2ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:17.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.071+0000 7f79baffd640 1 -- 172.21.15.100:0/1604170798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fba38e2390 msgr2=0x55fba3d2ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:17.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.071+0000 7f79baffd640 1 --2- 172.21.15.100:0/1604170798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fba38e2390 0x55fba3d2ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:17.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.071+0000 7f79baffd640 1 -- 172.21.15.100:0/1604170798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fba38c19f0 msgr2=0x55fba3cc8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:17.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.071+0000 7f79baffd640 1 --2- 172.21.15.100:0/1604170798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fba38c19f0 0x55fba3cc8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:17.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.071+0000 7f79baffd640 1 -- 172.21.15.100:0/1604170798 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fba3d2b1b0 con 0x55fba38da1b0 2024-07-26T13:43:17.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.072+0000 7f79baffd640 1 --2- 172.21.15.100:0/1604170798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba38da1b0 0x55fba3cc9510 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f79b0049040 tx=0x7f79b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b2c625011603db server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:17.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.072+0000 7f799affd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:43:17.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.072+0000 7f799affd640 1 -- 172.21.15.100:0/1604170798 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b004b040 con 0x55fba38da1b0 2024-07-26T13:43:17.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.072+0000 7f799affd640 1 -- 172.21.15.100:0/1604170798 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f79b0004300 con 0x55fba38da1b0 2024-07-26T13:43:17.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.072+0000 7f79c8400740 1 -- 172.21.15.100:0/1604170798 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79b00054f0 con 0x55fba38da1b0 2024-07-26T13:43:17.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.072+0000 7f799affd640 1 -- 172.21.15.100:0/1604170798 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f79b005c050 con 0x55fba38da1b0 2024-07-26T13:43:17.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.072+0000 7f79c8400740 10 client.16320 fetch_fsmap learned FSMap version 36 2024-07-26T13:43:17.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.072+0000 7f79c8400740 1 -- 172.21.15.100:0/1604170798 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55fba3d325d0 con 0x55fba38da1b0 2024-07-26T13:43:17.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.072+0000 7f799affd640 1 -- 172.21.15.100:0/1604170798 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79b00054f0 con 0x55fba38da1b0 2024-07-26T13:43:17.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.072+0000 7f799affd640 1 -- 172.21.15.100:0/1604170798 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f79b0057030 con 0x55fba38da1b0 2024-07-26T13:43:17.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.073+0000 7f79c8400740 10 client.16320 fetch_fsmap finished waiting for FSMap version 36 2024-07-26T13:43:17.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.073+0000 7f79c8400740 10 client.16320 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-07-26T13:43:17.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.073+0000 7f79c8400740 20 client.16320 populate_metadata read hostname 'smithi100' 2024-07-26T13:43:17.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.073+0000 7f79c8400740 1 --2- 172.21.15.100:0/1604170798 >> [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] conn(0x55fba3903070 0x55fba3d34da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:17.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.073+0000 7f79c8400740 4 client.16320 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-26T13:43:17.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.073+0000 7f79c8400740 1 -- 172.21.15.100:0/1604170798 --> [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fba3c1c5a0 con 0x55fba3903070 2024-07-26T13:43:17.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.073+0000 7f79bbfff640 1 --2- 172.21.15.100:0/1604170798 >> [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] conn(0x55fba3903070 0x55fba3d34da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:17.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.073+0000 7f799affd640 10 client.16320 ms_handle_connect on v2:172.21.15.138:6834/3290105300 2024-07-26T13:43:17.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.073+0000 7f79bbfff640 1 --2- 172.21.15.100:0/1604170798 >> [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] conn(0x55fba3903070 0x55fba3d34da0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=70dcd4e0a0c62668 server_cookie=ad5d2f82cc4283e3 in_seq=0 out_seq=0 2024-07-26T13:43:17.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.073+0000 7f799affd640 1 -- 172.21.15.100:0/1604170798 <== mds.2 v2:172.21.15.138:6834/3290105300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fba3c1c5a0 con 0x55fba3903070 2024-07-26T13:43:17.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.073+0000 7f799affd640 10 client.16320 handle_command_reply: tid=0 2024-07-26T13:43:17.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.073+0000 7f79c8400740 1 client.16320 shutdown 2024-07-26T13:43:17.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.074+0000 7f79c8400740 1 -- 172.21.15.100:0/1604170798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba38da1b0 msgr2=0x55fba3cc9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:17.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.074+0000 7f79c8400740 1 --2- 172.21.15.100:0/1604170798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba38da1b0 0x55fba3cc9510 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f79b0049040 tx=0x7f79b0004750 comp rx=0 tx=0).stop 2024-07-26T13:43:17.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.074+0000 7f79c8400740 1 -- 172.21.15.100:0/1604170798 shutdown_connections 2024-07-26T13:43:17.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.074+0000 7f79c8400740 1 --2- 172.21.15.100:0/1604170798 >> [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] conn(0x55fba3903070 0x55fba3d34da0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:17.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.074+0000 7f79c8400740 1 --2- 172.21.15.100:0/1604170798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fba38e2390 0x55fba3d2ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:17.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.074+0000 7f799affd640 0 client.16320 ms_handle_reset on v2:172.21.15.138:6834/3290105300 2024-07-26T13:43:17.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.074+0000 7f79c8400740 1 --2- 172.21.15.100:0/1604170798 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fba38da1b0 0x55fba3cc9510 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:17.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.074+0000 7f79c8400740 1 --2- 172.21.15.100:0/1604170798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fba38c19f0 0x55fba3cc8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:17.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.074+0000 7f79c8400740 1 -- 172.21.15.100:0/1604170798 >> 172.21.15.100:0/1604170798 conn(0x55fba38f7460 msgr2=0x55fba3ca9580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:17.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.074+0000 7f79c8400740 1 -- 172.21.15.100:0/1604170798 shutdown_connections 2024-07-26T13:43:17.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.074+0000 7f79c8400740 1 -- 172.21.15.100:0/1604170798 wait complete. 2024-07-26T13:43:17.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.074+0000 7f79c8400740 20 client.16320 trim_cache size 0 max 16384 2024-07-26T13:43:17.095 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-07-26T13:43:17.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.075+0000 7f79c1e7c640 1 -- 172.21.15.100:0/3614282463 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f79a40b3dc0 msgr2=0x7f79a40b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:17.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.075+0000 7f79c1e7c640 1 --2- 172.21.15.100:0/3614282463 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f79a40b3dc0 0x7f79a40b62d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f79ac0e1aa0 tx=0x7f79ac0e7e20 comp rx=0 tx=0).stop 2024-07-26T13:43:17.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.075+0000 7f79c1e7c640 1 -- 172.21.15.100:0/3614282463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79bc102490 msgr2=0x7f79bc19ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:17.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.075+0000 7f79c1e7c640 1 --2- 172.21.15.100:0/3614282463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79bc102490 0x7f79bc19ae40 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f79ac00e9f0 tx=0x7f79ac00eeb0 comp rx=0 tx=0).stop 2024-07-26T13:43:17.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.075+0000 7f79c1e7c640 1 -- 172.21.15.100:0/3614282463 shutdown_connections 2024-07-26T13:43:17.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.075+0000 7f79c1e7c640 1 --2- 172.21.15.100:0/3614282463 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f79a40b3dc0 0x7f79a40b62d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:17.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.075+0000 7f79c1e7c640 1 --2- 172.21.15.100:0/3614282463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79bc108720 0x7f79bc19b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:17.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.075+0000 7f79c1e7c640 1 --2- 172.21.15.100:0/3614282463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79bc107dd0 0x7f79bc19b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:17.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.075+0000 7f79c1e7c640 1 --2- 172.21.15.100:0/3614282463 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79bc102490 0x7f79bc19ae40 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:17.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.075+0000 7f79c1e7c640 1 -- 172.21.15.100:0/3614282463 >> 172.21.15.100:0/3614282463 conn(0x7f79bc079160 msgr2=0x7f79bc103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:17.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.075+0000 7f79c1e7c640 1 -- 172.21.15.100:0/3614282463 shutdown_connections 2024-07-26T13:43:17.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:17.075+0000 7f79c1e7c640 1 -- 172.21.15.100:0/3614282463 wait complete. 2024-07-26T13:43:17.408 INFO:tasks.workunit.client.0.smithi100.stdout: 1 88150 5.41 MB/sec warmup 91 sec latency 58.959 ms 2024-07-26T13:43:17.809 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-26T13:43:18.078 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:43:18.408 INFO:tasks.workunit.client.0.smithi100.stdout: 1 89721 5.43 MB/sec warmup 92 sec latency 10.373 ms 2024-07-26T13:43:19.408 INFO:tasks.workunit.client.0.smithi100.stdout: 1 92340 5.56 MB/sec warmup 93 sec latency 15.087 ms 2024-07-26T13:43:19.620 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:19 smithi100 ceph-mon[32599]: pgmap v461: 97 pgs: 97 active+clean; 47 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 154 op/s 2024-07-26T13:43:19.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:19 smithi167 ceph-mon[37140]: pgmap v461: 97 pgs: 97 active+clean; 47 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 154 op/s 2024-07-26T13:43:19.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.863+0000 7fe0a99b6640 1 -- 172.21.15.100:0/826464706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0a4107dd0 msgr2=0x7fe0a41081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:19.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.863+0000 7fe0a99b6640 1 --2- 172.21.15.100:0/826464706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0a4107dd0 0x7fe0a41081e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fe098009ca0 tx=0x7fe09802a8c0 comp rx=0 tx=0).stop 2024-07-26T13:43:19.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.863+0000 7fe0a99b6640 1 -- 172.21.15.100:0/826464706 shutdown_connections 2024-07-26T13:43:19.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.863+0000 7fe0a99b6640 1 --2- 172.21.15.100:0/826464706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0a4108720 0x7fe0a410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:19.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.864+0000 7fe0a99b6640 1 --2- 172.21.15.100:0/826464706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0a4107dd0 0x7fe0a41081e0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:19.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.864+0000 7fe0a99b6640 1 --2- 172.21.15.100:0/826464706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0a4102490 0x7fe0a41028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:19.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.864+0000 7fe0a99b6640 1 -- 172.21.15.100:0/826464706 >> 172.21.15.100:0/826464706 conn(0x7fe0a4079160 msgr2=0x7fe0a4079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:19.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.864+0000 7fe0a99b6640 1 -- 172.21.15.100:0/826464706 shutdown_connections 2024-07-26T13:43:19.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.864+0000 7fe0a99b6640 1 -- 172.21.15.100:0/826464706 wait complete. 2024-07-26T13:43:19.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.864+0000 7fe0a99b6640 1 Processor -- start 2024-07-26T13:43:19.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.864+0000 7fe0a99b6640 1 -- start start 2024-07-26T13:43:19.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.865+0000 7fe0a99b6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0a4102490 0x7fe0a40ff0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:19.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.865+0000 7fe0a99b6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0a4107dd0 0x7fe0a40ff5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:19.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.865+0000 7fe0a99b6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0a4108720 0x7fe0a40ffb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:19.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.865+0000 7fe0a99b6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a4112370 con 0x7fe0a4108720 2024-07-26T13:43:19.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.865+0000 7fe0a99b6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a41121f0 con 0x7fe0a4102490 2024-07-26T13:43:19.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.865+0000 7fe0a99b6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a41124f0 con 0x7fe0a4107dd0 2024-07-26T13:43:19.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.865+0000 7fe0a2ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0a4102490 0x7fe0a40ff0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:19.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.865+0000 7fe0a37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0a4108720 0x7fe0a40ffb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:19.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.865+0000 7fe0a2ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0a4102490 0x7fe0a40ff0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:58928/0 (socket says 172.21.15.100:58928) 2024-07-26T13:43:19.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.865+0000 7fe0a37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0a4108720 0x7fe0a40ffb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56170/0 (socket says 172.21.15.100:56170) 2024-07-26T13:43:19.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.865+0000 7fe0a2ffd640 1 -- 172.21.15.100:0/3088386319 learned_addr learned my addr 172.21.15.100:0/3088386319 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:19.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.865+0000 7fe0a27fc640 1 --2- 172.21.15.100:0/3088386319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0a4107dd0 0x7fe0a40ff5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:19.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.865+0000 7fe0a2ffd640 1 -- 172.21.15.100:0/3088386319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0a4107dd0 msgr2=0x7fe0a40ff5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:19.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.865+0000 7fe0a2ffd640 1 --2- 172.21.15.100:0/3088386319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0a4107dd0 0x7fe0a40ff5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:19.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.865+0000 7fe0a2ffd640 1 -- 172.21.15.100:0/3088386319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0a4108720 msgr2=0x7fe0a40ffb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:19.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.865+0000 7fe0a2ffd640 1 --2- 172.21.15.100:0/3088386319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0a4108720 0x7fe0a40ffb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:19.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.865+0000 7fe0a2ffd640 1 -- 172.21.15.100:0/3088386319 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0a41a8130 con 0x7fe0a4102490 2024-07-26T13:43:19.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.865+0000 7fe0a37fe640 1 --2- 172.21.15.100:0/3088386319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0a4108720 0x7fe0a40ffb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:43:19.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.865+0000 7fe0a2ffd640 1 --2- 172.21.15.100:0/3088386319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0a4102490 0x7fe0a40ff0a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe09c00e9f0 tx=0x7fe09c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:19.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.866+0000 7fe0a89b4640 1 -- 172.21.15.100:0/3088386319 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09c00cd80 con 0x7fe0a4102490 2024-07-26T13:43:19.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.866+0000 7fe0a99b6640 1 -- 172.21.15.100:0/3088386319 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0a41a8420 con 0x7fe0a4102490 2024-07-26T13:43:19.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.866+0000 7fe0a89b4640 1 -- 172.21.15.100:0/3088386319 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe09c0108e0 con 0x7fe0a4102490 2024-07-26T13:43:19.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.866+0000 7fe0a89b4640 1 -- 172.21.15.100:0/3088386319 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09c018980 con 0x7fe0a4102490 2024-07-26T13:43:19.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.866+0000 7fe0a99b6640 1 -- 172.21.15.100:0/3088386319 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0a41a8980 con 0x7fe0a4102490 2024-07-26T13:43:19.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.869+0000 7fe0aff3a740 1 Processor -- start 2024-07-26T13:43:19.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.869+0000 7fe0aff3a740 1 -- start start 2024-07-26T13:43:19.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.869+0000 7fe0aff3a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de22041070 0x55de223e6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:19.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.869+0000 7fe0aff3a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de220181b0 0x55de223e6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:19.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.869+0000 7fe0aff3a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de22020390 0x55de223e7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:19.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.869+0000 7fe0aff3a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55de2235cbb0 con 0x55de22020390 2024-07-26T13:43:19.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.869+0000 7fe0aff3a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55de2234c0b0 con 0x55de22041070 2024-07-26T13:43:19.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.869+0000 7fe0aff3a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55de223500f0 con 0x55de220181b0 2024-07-26T13:43:19.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.869+0000 7fe0a89b4640 1 -- 172.21.15.100:0/3088386319 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fe09c010430 con 0x7fe0a4102490 2024-07-26T13:43:19.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.869+0000 7fe0a37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de22041070 0x55de223e6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:19.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.869+0000 7fe0a37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de22041070 0x55de223e6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:58934/0 (socket says 172.21.15.100:58934) 2024-07-26T13:43:19.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.869+0000 7fe0a37fe640 1 -- 172.21.15.100:0/609738256 learned_addr learned my addr 172.21.15.100:0/609738256 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:19.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.869+0000 7fe0a27fc640 1 --2- 172.21.15.100:0/609738256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de22020390 0x55de223e7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:19.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.869+0000 7fe0a2ffd640 1 --2- 172.21.15.100:0/609738256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de220181b0 0x55de223e6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:19.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.870+0000 7fe0a37fe640 1 -- 172.21.15.100:0/609738256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de220181b0 msgr2=0x55de223e6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:19.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.870+0000 7fe0a37fe640 1 --2- 172.21.15.100:0/609738256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de220181b0 0x55de223e6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:19.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.870+0000 7fe0a37fe640 1 -- 172.21.15.100:0/609738256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de22020390 msgr2=0x55de223e7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:19.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.870+0000 7fe0a37fe640 1 --2- 172.21.15.100:0/609738256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de22020390 0x55de223e7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:19.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.870+0000 7fe0a37fe640 1 -- 172.21.15.100:0/609738256 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de222f2b90 con 0x55de22041070 2024-07-26T13:43:19.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.870+0000 7fe0a37fe640 1 --2- 172.21.15.100:0/609738256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de22041070 0x55de223e6690 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe094011540 tx=0x7fe094011a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a392823941083c8 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:19.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.870+0000 7fe089ffb640 1 -- 172.21.15.100:0/609738256 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09400f5c0 con 0x55de22041070 2024-07-26T13:43:19.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.870+0000 7fe089ffb640 1 -- 172.21.15.100:0/609738256 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe09400fc10 con 0x55de22041070 2024-07-26T13:43:19.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.870+0000 7fe089ffb640 1 -- 172.21.15.100:0/609738256 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe094019c50 con 0x55de22041070 2024-07-26T13:43:19.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.870+0000 7fe0a89b4640 1 --2- 172.21.15.100:0/3088386319 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe08c0b3dc0 0x7fe08c0b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:19.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.870+0000 7fe0a2ffd640 1 --2- 172.21.15.100:0/3088386319 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe08c0b3dc0 0x7fe08c0b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:19.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.871+0000 7fe0a89b4640 1 -- 172.21.15.100:0/3088386319 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7fe09c0e43a0 con 0x7fe0a4102490 2024-07-26T13:43:19.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.871+0000 7fe0aff3a740 1 -- 172.21.15.100:0/609738256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de22041070 msgr2=0x55de223e6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:19.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.871+0000 7fe0aff3a740 1 --2- 172.21.15.100:0/609738256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de22041070 0x55de223e6690 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe094011540 tx=0x7fe094011a00 comp rx=0 tx=0).stop 2024-07-26T13:43:19.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.871+0000 7fe0a2ffd640 1 --2- 172.21.15.100:0/3088386319 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe08c0b3dc0 0x7fe08c0b62d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x55de223e5310 tx=0x7fe09c0744b0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:19.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.871+0000 7fe0aff3a740 1 -- 172.21.15.100:0/609738256 shutdown_connections 2024-07-26T13:43:19.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.871+0000 7fe0aff3a740 1 --2- 172.21.15.100:0/609738256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de22020390 0x55de223e7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:19.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.871+0000 7fe0aff3a740 1 --2- 172.21.15.100:0/609738256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de220181b0 0x55de223e6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:19.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.871+0000 7fe0aff3a740 1 --2- 172.21.15.100:0/609738256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de22041070 0x55de223e6690 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:19.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.871+0000 7fe0aff3a740 1 -- 172.21.15.100:0/609738256 >> 172.21.15.100:0/609738256 conn(0x55de22035460 msgr2=0x55de223e4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:19.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.871+0000 7fe0aff3a740 1 -- 172.21.15.100:0/609738256 shutdown_connections 2024-07-26T13:43:19.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.871+0000 7fe0aff3a740 1 -- 172.21.15.100:0/609738256 wait complete. 2024-07-26T13:43:19.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.872+0000 7fe0aff3a740 1 -- start start 2024-07-26T13:43:19.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.872+0000 7fe0aff3a740 1 Processor -- start 2024-07-26T13:43:19.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.872+0000 7fe0aff3a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de220181b0 0x55de223f22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:19.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.872+0000 7fe0aff3a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de22020390 0x55de223f2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:19.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.872+0000 7fe0aff3a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de22041070 0x55de223fe160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:19.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.872+0000 7fe0aff3a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55de221af700 con 0x55de22020390 2024-07-26T13:43:19.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.872+0000 7fe0aff3a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55de222ee9e0 con 0x55de220181b0 2024-07-26T13:43:19.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.872+0000 7fe0aff3a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55de223f4e70 con 0x55de22041070 2024-07-26T13:43:19.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.872+0000 7fe0aff3a740 10 client.0 _ll_register_callbacks cb 0x55de223094f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:43:19.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.873+0000 7fe0a37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de220181b0 0x55de223f22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:19.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.873+0000 7fe0a27fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de22020390 0x55de223f2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:19.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.873+0000 7fe0a27fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de22020390 0x55de223f2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56192/0 (socket says 172.21.15.100:56192) 2024-07-26T13:43:19.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.873+0000 7fe0a27fc640 1 -- 172.21.15.100:0/1243542837 learned_addr learned my addr 172.21.15.100:0/1243542837 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:19.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.873+0000 7fe0a37fe640 1 --2- 172.21.15.100:0/1243542837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de22041070 0x55de223fe160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:19.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.873+0000 7fe0a37fe640 1 -- 172.21.15.100:0/1243542837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de22041070 msgr2=0x55de223fe160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:19.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.873+0000 7fe0a37fe640 1 --2- 172.21.15.100:0/1243542837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de22041070 0x55de223fe160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:19.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.873+0000 7fe0a37fe640 1 -- 172.21.15.100:0/1243542837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de22020390 msgr2=0x55de223f2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:19.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.873+0000 7fe0a37fe640 1 --2- 172.21.15.100:0/1243542837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de22020390 0x55de223f2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:19.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.873+0000 7fe0a37fe640 1 -- 172.21.15.100:0/1243542837 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de220ac650 con 0x55de220181b0 2024-07-26T13:43:19.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.873+0000 7fe0a27fc640 1 --2- 172.21.15.100:0/1243542837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de22020390 0x55de223f2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:43:19.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.873+0000 7fe0a37fe640 1 --2- 172.21.15.100:0/1243542837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de220181b0 0x55de223f22a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe0940198b0 tx=0x7fe094009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8086ba039823dee4 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:19.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.873+0000 7fe082ffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-07-26T13:43:19.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.873+0000 7fe082ffd640 1 -- 172.21.15.100:0/1243542837 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09402d060 con 0x55de220181b0 2024-07-26T13:43:19.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.873+0000 7fe0aff3a740 1 -- 172.21.15.100:0/1243542837 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe09400fc10 con 0x55de220181b0 2024-07-26T13:43:19.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.873+0000 7fe082ffd640 1 -- 172.21.15.100:0/1243542837 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7fe09401f040 con 0x55de220181b0 2024-07-26T13:43:19.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.874+0000 7fe082ffd640 1 -- 172.21.15.100:0/1243542837 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe094019e30 con 0x55de220181b0 2024-07-26T13:43:19.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.874+0000 7fe0aff3a740 10 client.25694 fetch_fsmap learned FSMap version 36 2024-07-26T13:43:19.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.874+0000 7fe0aff3a740 1 -- 172.21.15.100:0/1243542837 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55de22400e90 con 0x55de220181b0 2024-07-26T13:43:19.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.874+0000 7fe082ffd640 1 -- 172.21.15.100:0/1243542837 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe09400fc10 con 0x55de220181b0 2024-07-26T13:43:19.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.874+0000 7fe082ffd640 1 -- 172.21.15.100:0/1243542837 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7fe094005020 con 0x55de220181b0 2024-07-26T13:43:19.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.874+0000 7fe0aff3a740 10 client.25694 fetch_fsmap finished waiting for FSMap version 36 2024-07-26T13:43:19.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.874+0000 7fe0aff3a740 10 client.25694 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-26T13:43:19.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.874+0000 7fe0aff3a740 20 client.25694 populate_metadata read hostname 'smithi100' 2024-07-26T13:43:19.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.874+0000 7fe0aff3a740 1 --2- 172.21.15.100:0/1243542837 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55de21fff9f0 0x55de22404250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:19.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.874+0000 7fe0aff3a740 4 client.25694 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:43:19.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.874+0000 7fe0aff3a740 1 -- 172.21.15.100:0/1243542837 --> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de2235eb00 con 0x55de21fff9f0 2024-07-26T13:43:19.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.874+0000 7fe0a27fc640 1 --2- 172.21.15.100:0/1243542837 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55de21fff9f0 0x55de22404250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:19.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.875+0000 7fe082ffd640 10 client.25694 ms_handle_connect on v2:172.21.15.138:6838/1580925471 2024-07-26T13:43:19.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.875+0000 7fe0a27fc640 1 --2- 172.21.15.100:0/1243542837 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55de21fff9f0 0x55de22404250 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7ca1558795691966 server_cookie=90085b78693b799e in_seq=0 out_seq=0 2024-07-26T13:43:19.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.876+0000 7fe082ffd640 1 -- 172.21.15.100:0/1243542837 <== mds.1 v2:172.21.15.138:6838/1580925471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55de2235eb00 con 0x55de21fff9f0 2024-07-26T13:43:19.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.876+0000 7fe082ffd640 10 client.25694 handle_command_reply: tid=0 2024-07-26T13:43:19.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.876+0000 7fe0aff3a740 1 client.25694 shutdown 2024-07-26T13:43:19.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.876+0000 7fe0aff3a740 1 -- 172.21.15.100:0/1243542837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de220181b0 msgr2=0x55de223f22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:19.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.876+0000 7fe0aff3a740 1 --2- 172.21.15.100:0/1243542837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de220181b0 0x55de223f22a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe0940198b0 tx=0x7fe094009310 comp rx=0 tx=0).stop 2024-07-26T13:43:19.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.876+0000 7fe0aff3a740 1 -- 172.21.15.100:0/1243542837 shutdown_connections 2024-07-26T13:43:19.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.876+0000 7fe0aff3a740 1 --2- 172.21.15.100:0/1243542837 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55de21fff9f0 0x55de22404250 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:19.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.876+0000 7fe0aff3a740 1 --2- 172.21.15.100:0/1243542837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de22041070 0x55de223fe160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:19.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.876+0000 7fe0aff3a740 1 --2- 172.21.15.100:0/1243542837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de22020390 0x55de223f2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:19.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.876+0000 7fe082ffd640 0 client.25694 ms_handle_reset on v2:172.21.15.138:6838/1580925471 2024-07-26T13:43:19.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.876+0000 7fe0aff3a740 1 --2- 172.21.15.100:0/1243542837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de220181b0 0x55de223f22a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:19.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.876+0000 7fe0aff3a740 1 -- 172.21.15.100:0/1243542837 >> 172.21.15.100:0/1243542837 conn(0x55de22035460 msgr2=0x55de223e2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:19.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.876+0000 7fe0aff3a740 1 -- 172.21.15.100:0/1243542837 shutdown_connections 2024-07-26T13:43:19.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.876+0000 7fe0aff3a740 1 -- 172.21.15.100:0/1243542837 wait complete. 2024-07-26T13:43:19.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.876+0000 7fe0aff3a740 20 client.25694 trim_cache size 0 max 16384 2024-07-26T13:43:19.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.890+0000 7fe0aff3a740 1 Processor -- start 2024-07-26T13:43:19.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.890+0000 7fe0aff3a740 1 -- start start 2024-07-26T13:43:19.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.890+0000 7fe0aff3a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de21fff9f0 0x55de223fdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:19.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.890+0000 7fe0aff3a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de220181b0 0x55de223e7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:19.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.890+0000 7fe0aff3a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de22020390 0x55de223eb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:19.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.890+0000 7fe0aff3a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a41b3d30 con 0x55de21fff9f0 2024-07-26T13:43:19.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.890+0000 7fe0aff3a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a41b15a0 con 0x55de220181b0 2024-07-26T13:43:19.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.890+0000 7fe0aff3a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a41af3f0 con 0x55de22020390 2024-07-26T13:43:19.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.890+0000 7fe0a37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de21fff9f0 0x55de223fdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:19.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.890+0000 7fe0a37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de21fff9f0 0x55de223fdf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56204/0 (socket says 172.21.15.100:56204) 2024-07-26T13:43:19.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.890+0000 7fe0a27fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de220181b0 0x55de223e7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:19.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.890+0000 7fe0a37fe640 1 -- 172.21.15.100:0/3084293707 learned_addr learned my addr 172.21.15.100:0/3084293707 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:19.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.890+0000 7fe0a37fe640 1 --2- 172.21.15.100:0/3084293707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de22020390 0x55de223eb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:19.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.890+0000 7fe0a27fc640 1 -- 172.21.15.100:0/3084293707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de22020390 msgr2=0x55de223eb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:19.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.890+0000 7fe0a27fc640 1 --2- 172.21.15.100:0/3084293707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de22020390 0x55de223eb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:19.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.890+0000 7fe0a27fc640 1 -- 172.21.15.100:0/3084293707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de21fff9f0 msgr2=0x55de223fdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:19.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.890+0000 7fe0a27fc640 1 --2- 172.21.15.100:0/3084293707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de21fff9f0 0x55de223fdf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:19.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.890+0000 7fe0a27fc640 1 -- 172.21.15.100:0/3084293707 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de223dffa0 con 0x55de220181b0 2024-07-26T13:43:19.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.891+0000 7fe0a27fc640 1 --2- 172.21.15.100:0/3084293707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de220181b0 0x55de223e7dd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x55de22404850 tx=0x7fe098009360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36bbc7f9e9a0084 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:19.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.891+0000 7fe0897fa640 1 -- 172.21.15.100:0/3084293707 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe098031700 con 0x55de220181b0 2024-07-26T13:43:19.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.891+0000 7fe0897fa640 1 -- 172.21.15.100:0/3084293707 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe098036040 con 0x55de220181b0 2024-07-26T13:43:19.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.891+0000 7fe0897fa640 1 -- 172.21.15.100:0/3084293707 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0980046f0 con 0x55de220181b0 2024-07-26T13:43:19.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.892+0000 7fe0aff3a740 1 -- 172.21.15.100:0/3084293707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de220181b0 msgr2=0x55de223e7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:19.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.892+0000 7fe0aff3a740 1 --2- 172.21.15.100:0/3084293707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de220181b0 0x55de223e7dd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x55de22404850 tx=0x7fe098009360 comp rx=0 tx=0).stop 2024-07-26T13:43:19.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.892+0000 7fe0aff3a740 1 -- 172.21.15.100:0/3084293707 shutdown_connections 2024-07-26T13:43:19.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.892+0000 7fe0aff3a740 1 --2- 172.21.15.100:0/3084293707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de22020390 0x55de223eb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:19.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.892+0000 7fe0aff3a740 1 --2- 172.21.15.100:0/3084293707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de220181b0 0x55de223e7dd0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:19.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.892+0000 7fe0aff3a740 1 --2- 172.21.15.100:0/3084293707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de21fff9f0 0x55de223fdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:19.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.892+0000 7fe0aff3a740 1 -- 172.21.15.100:0/3084293707 >> 172.21.15.100:0/3084293707 conn(0x55de22035460 msgr2=0x55de223e76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:19.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.892+0000 7fe0aff3a740 1 -- 172.21.15.100:0/3084293707 shutdown_connections 2024-07-26T13:43:19.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.892+0000 7fe0aff3a740 1 -- 172.21.15.100:0/3084293707 wait complete. 2024-07-26T13:43:19.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.892+0000 7fe0aff3a740 1 -- start start 2024-07-26T13:43:19.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.892+0000 7fe0aff3a740 1 Processor -- start 2024-07-26T13:43:19.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.892+0000 7fe0aff3a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de21fff9f0 0x55de223fa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:19.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.893+0000 7fe0aff3a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de220181b0 0x55de22409010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:19.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.893+0000 7fe0aff3a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de22020390 0x55de22409550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:19.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.893+0000 7fe0aff3a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a41afa60 con 0x55de220181b0 2024-07-26T13:43:19.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.893+0000 7fe0aff3a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a41b0ed0 con 0x55de21fff9f0 2024-07-26T13:43:19.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.893+0000 7fe0aff3a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a41b0130 con 0x55de22020390 2024-07-26T13:43:19.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.893+0000 7fe0aff3a740 10 client.0 _ll_register_callbacks cb 0x7fe094005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:43:19.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.893+0000 7fe0a27fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de220181b0 0x55de22409010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:19.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.893+0000 7fe0a37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de21fff9f0 0x55de223fa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:19.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.893+0000 7fe0a27fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de220181b0 0x55de22409010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56206/0 (socket says 172.21.15.100:56206) 2024-07-26T13:43:19.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.893+0000 7fe0a27fc640 1 -- 172.21.15.100:0/1477933643 learned_addr learned my addr 172.21.15.100:0/1477933643 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:19.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.893+0000 7fe0a37fe640 1 --2- 172.21.15.100:0/1477933643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de22020390 0x55de22409550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:19.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.893+0000 7fe0a27fc640 1 -- 172.21.15.100:0/1477933643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de22020390 msgr2=0x55de22409550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:19.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.893+0000 7fe0a27fc640 1 --2- 172.21.15.100:0/1477933643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de22020390 0x55de22409550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:19.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.893+0000 7fe0a27fc640 1 -- 172.21.15.100:0/1477933643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de21fff9f0 msgr2=0x55de223fa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:19.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.893+0000 7fe0a27fc640 1 --2- 172.21.15.100:0/1477933643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de21fff9f0 0x55de223fa4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:19.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.893+0000 7fe0a27fc640 1 -- 172.21.15.100:0/1477933643 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de22409a90 con 0x55de220181b0 2024-07-26T13:43:19.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.893+0000 7fe0a27fc640 1 --2- 172.21.15.100:0/1477933643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de220181b0 0x55de22409010 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe09802ade0 tx=0x7fe098031470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f5808a6fbe14d42 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:19.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.893+0000 7fe082ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:43:19.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.893+0000 7fe082ffd640 1 -- 172.21.15.100:0/1477933643 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe098056040 con 0x55de220181b0 2024-07-26T13:43:19.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.893+0000 7fe082ffd640 1 -- 172.21.15.100:0/1477933643 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7fe098004000 con 0x55de220181b0 2024-07-26T13:43:19.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.893+0000 7fe0aff3a740 1 -- 172.21.15.100:0/1477933643 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe098036040 con 0x55de220181b0 2024-07-26T13:43:19.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.894+0000 7fe082ffd640 1 -- 172.21.15.100:0/1477933643 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe098068050 con 0x55de220181b0 2024-07-26T13:43:19.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.894+0000 7fe0aff3a740 10 client.16338 fetch_fsmap learned FSMap version 36 2024-07-26T13:43:19.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.894+0000 7fe0aff3a740 1 -- 172.21.15.100:0/1477933643 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55de223f8d40 con 0x55de220181b0 2024-07-26T13:43:19.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.894+0000 7fe082ffd640 1 -- 172.21.15.100:0/1477933643 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe098036040 con 0x55de220181b0 2024-07-26T13:43:19.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.894+0000 7fe082ffd640 1 -- 172.21.15.100:0/1477933643 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7fe098059070 con 0x55de220181b0 2024-07-26T13:43:19.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.894+0000 7fe0aff3a740 10 client.16338 fetch_fsmap finished waiting for FSMap version 36 2024-07-26T13:43:19.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.894+0000 7fe0aff3a740 10 client.16338 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-26T13:43:19.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.894+0000 7fe0aff3a740 20 client.16338 populate_metadata read hostname 'smithi100' 2024-07-26T13:43:19.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.894+0000 7fe0aff3a740 1 --2- 172.21.15.100:0/1477933643 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55de22041070 0x55de2246ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:19.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.894+0000 7fe0aff3a740 4 client.16338 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-26T13:43:19.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.894+0000 7fe0aff3a740 1 -- 172.21.15.100:0/1477933643 --> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55de2235a5a0 con 0x55de22041070 2024-07-26T13:43:19.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.894+0000 7fe0a37fe640 1 --2- 172.21.15.100:0/1477933643 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55de22041070 0x55de2246ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:19.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.895+0000 7fe082ffd640 10 client.16338 ms_handle_connect on v2:172.21.15.138:6838/1580925471 2024-07-26T13:43:19.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.895+0000 7fe0a37fe640 1 --2- 172.21.15.100:0/1477933643 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55de22041070 0x55de2246ec20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5d01e779dc01bd44 server_cookie=705cf2b80b16dee in_seq=0 out_seq=0 2024-07-26T13:43:19.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.895+0000 7fe082ffd640 1 -- 172.21.15.100:0/1477933643 <== mds.1 v2:172.21.15.138:6838/1580925471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55de2235a5a0 con 0x55de22041070 2024-07-26T13:43:19.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.895+0000 7fe082ffd640 10 client.16338 handle_command_reply: tid=0 2024-07-26T13:43:19.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.895+0000 7fe0aff3a740 1 client.16338 shutdown 2024-07-26T13:43:19.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.895+0000 7fe0aff3a740 1 -- 172.21.15.100:0/1477933643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de220181b0 msgr2=0x55de22409010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:19.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.895+0000 7fe0aff3a740 1 --2- 172.21.15.100:0/1477933643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de220181b0 0x55de22409010 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe09802ade0 tx=0x7fe098031470 comp rx=0 tx=0).stop 2024-07-26T13:43:19.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.895+0000 7fe0aff3a740 1 -- 172.21.15.100:0/1477933643 shutdown_connections 2024-07-26T13:43:19.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.895+0000 7fe0aff3a740 1 --2- 172.21.15.100:0/1477933643 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55de22041070 0x55de2246ec20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:19.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.895+0000 7fe0aff3a740 1 --2- 172.21.15.100:0/1477933643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de22020390 0x55de22409550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:19.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.895+0000 7fe0aff3a740 1 --2- 172.21.15.100:0/1477933643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55de220181b0 0x55de22409010 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:19.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.895+0000 7fe0aff3a740 1 --2- 172.21.15.100:0/1477933643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55de21fff9f0 0x55de223fa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:19.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.895+0000 7fe0aff3a740 1 -- 172.21.15.100:0/1477933643 >> 172.21.15.100:0/1477933643 conn(0x55de22035460 msgr2=0x55de223e9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:19.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.895+0000 7fe082ffd640 0 client.16338 ms_handle_reset on v2:172.21.15.138:6838/1580925471 2024-07-26T13:43:19.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.896+0000 7fe0aff3a740 1 -- 172.21.15.100:0/1477933643 shutdown_connections 2024-07-26T13:43:19.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.896+0000 7fe0aff3a740 1 -- 172.21.15.100:0/1477933643 wait complete. 2024-07-26T13:43:19.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.896+0000 7fe0aff3a740 20 client.16338 trim_cache size 0 max 16384 2024-07-26T13:43:19.903 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-07-26T13:43:19.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.896+0000 7fe0a99b6640 1 -- 172.21.15.100:0/3088386319 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe08c0b3dc0 msgr2=0x7fe08c0b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:19.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.896+0000 7fe0a99b6640 1 --2- 172.21.15.100:0/3088386319 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe08c0b3dc0 0x7fe08c0b62d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x55de223e5310 tx=0x7fe09c0744b0 comp rx=0 tx=0).stop 2024-07-26T13:43:19.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.896+0000 7fe0a99b6640 1 -- 172.21.15.100:0/3088386319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0a4102490 msgr2=0x7fe0a40ff0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:19.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.896+0000 7fe0a99b6640 1 --2- 172.21.15.100:0/3088386319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0a4102490 0x7fe0a40ff0a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe09c00e9f0 tx=0x7fe09c00eeb0 comp rx=0 tx=0).stop 2024-07-26T13:43:19.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.896+0000 7fe0a99b6640 1 -- 172.21.15.100:0/3088386319 shutdown_connections 2024-07-26T13:43:19.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.896+0000 7fe0a99b6640 1 --2- 172.21.15.100:0/3088386319 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe08c0b3dc0 0x7fe08c0b62d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:19.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.896+0000 7fe0a99b6640 1 --2- 172.21.15.100:0/3088386319 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0a4108720 0x7fe0a40ffb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:19.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.896+0000 7fe0a99b6640 1 --2- 172.21.15.100:0/3088386319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0a4107dd0 0x7fe0a40ff5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:19.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.896+0000 7fe0a99b6640 1 --2- 172.21.15.100:0/3088386319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0a4102490 0x7fe0a40ff0a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:19.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.896+0000 7fe0a99b6640 1 -- 172.21.15.100:0/3088386319 >> 172.21.15.100:0/3088386319 conn(0x7fe0a4079160 msgr2=0x7fe0a4101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:19.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.896+0000 7fe0a99b6640 1 -- 172.21.15.100:0/3088386319 shutdown_connections 2024-07-26T13:43:19.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:19.896+0000 7fe0a99b6640 1 -- 172.21.15.100:0/3088386319 wait complete. 2024-07-26T13:43:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:19 smithi138 ceph-mon[36954]: pgmap v461: 97 pgs: 97 active+clean; 47 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 154 op/s 2024-07-26T13:43:20.408 INFO:tasks.workunit.client.0.smithi100.stdout: 1 93695 5.57 MB/sec warmup 94 sec latency 70.378 ms 2024-07-26T13:43:21.239 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-26T13:43:21.408 INFO:tasks.workunit.client.0.smithi100.stdout: 1 96094 5.65 MB/sec warmup 95 sec latency 16.249 ms 2024-07-26T13:43:21.502 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:43:21.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:21 smithi167 ceph-mon[37140]: pgmap v462: 97 pgs: 97 active+clean; 46 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 224 op/s 2024-07-26T13:43:21.874 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:21 smithi100 ceph-mon[32599]: pgmap v462: 97 pgs: 97 active+clean; 46 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 224 op/s 2024-07-26T13:43:21.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:21 smithi138 ceph-mon[36954]: pgmap v462: 97 pgs: 97 active+clean; 46 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 224 op/s 2024-07-26T13:43:22.408 INFO:tasks.workunit.client.0.smithi100.stdout: 1 98206 5.67 MB/sec warmup 96 sec latency 11.346 ms 2024-07-26T13:43:23.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.289+0000 7fd373c65640 1 -- 172.21.15.100:0/2316733816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd36c1035b0 msgr2=0x7fd36c1039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:23.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.289+0000 7fd373c65640 1 --2- 172.21.15.100:0/2316733816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd36c1035b0 0x7fd36c1039c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fd368009900 tx=0x7fd36802a410 comp rx=0 tx=0).stop 2024-07-26T13:43:23.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.289+0000 7fd373c65640 1 -- 172.21.15.100:0/2316733816 shutdown_connections 2024-07-26T13:43:23.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.289+0000 7fd373c65640 1 --2- 172.21.15.100:0/2316733816 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c109840 0x7fd36c110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:23.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.289+0000 7fd373c65640 1 --2- 172.21.15.100:0/2316733816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd36c108ef0 0x7fd36c109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:23.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.289+0000 7fd373c65640 1 --2- 172.21.15.100:0/2316733816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd36c1035b0 0x7fd36c1039c0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:23.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.289+0000 7fd373c65640 1 -- 172.21.15.100:0/2316733816 >> 172.21.15.100:0/2316733816 conn(0x7fd36c0fefe0 msgr2=0x7fd36c101430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:23.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.289+0000 7fd373c65640 1 -- 172.21.15.100:0/2316733816 shutdown_connections 2024-07-26T13:43:23.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.289+0000 7fd373c65640 1 -- 172.21.15.100:0/2316733816 wait complete. 2024-07-26T13:43:23.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.290+0000 7fd373c65640 1 Processor -- start 2024-07-26T13:43:23.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.290+0000 7fd373c65640 1 -- start start 2024-07-26T13:43:23.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.290+0000 7fd373c65640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd36c1035b0 0x7fd36c075140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:23.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.290+0000 7fd373c65640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c108ef0 0x7fd36c075680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:23.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.290+0000 7fd373c65640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd36c109840 0x7fd36c075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:23.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.290+0000 7fd373c65640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c113510 con 0x7fd36c1035b0 2024-07-26T13:43:23.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.290+0000 7fd373c65640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c113390 con 0x7fd36c108ef0 2024-07-26T13:43:23.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.290+0000 7fd373c65640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c113690 con 0x7fd36c109840 2024-07-26T13:43:23.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.290+0000 7fd371a01640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd36c1035b0 0x7fd36c075140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:23.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.290+0000 7fd371a01640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd36c1035b0 0x7fd36c075140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56234/0 (socket says 172.21.15.100:56234) 2024-07-26T13:43:23.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.290+0000 7fd371a01640 1 -- 172.21.15.100:0/407101855 learned_addr learned my addr 172.21.15.100:0/407101855 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:23.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.290+0000 7fd371200640 1 --2- 172.21.15.100:0/407101855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c108ef0 0x7fd36c075680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:23.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.290+0000 7fd372202640 1 --2- 172.21.15.100:0/407101855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd36c109840 0x7fd36c075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:23.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.291+0000 7fd371a01640 1 -- 172.21.15.100:0/407101855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd36c109840 msgr2=0x7fd36c075bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:23.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.291+0000 7fd371a01640 1 --2- 172.21.15.100:0/407101855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd36c109840 0x7fd36c075bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:23.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.291+0000 7fd371a01640 1 -- 172.21.15.100:0/407101855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c108ef0 msgr2=0x7fd36c075680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:23.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.291+0000 7fd371a01640 1 --2- 172.21.15.100:0/407101855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c108ef0 0x7fd36c075680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:23.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.291+0000 7fd371a01640 1 -- 172.21.15.100:0/407101855 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd36c1a81c0 con 0x7fd36c1035b0 2024-07-26T13:43:23.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.291+0000 7fd371a01640 1 --2- 172.21.15.100:0/407101855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd36c1035b0 0x7fd36c075140 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fd368005940 tx=0x7fd3680040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:23.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.291+0000 7fd362ffd640 1 -- 172.21.15.100:0/407101855 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36803f030 con 0x7fd36c1035b0 2024-07-26T13:43:23.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.291+0000 7fd362ffd640 1 -- 172.21.15.100:0/407101855 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3680044c0 con 0x7fd36c1035b0 2024-07-26T13:43:23.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.291+0000 7fd373c65640 1 -- 172.21.15.100:0/407101855 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd36c1a8450 con 0x7fd36c1035b0 2024-07-26T13:43:23.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.291+0000 7fd362ffd640 1 -- 172.21.15.100:0/407101855 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd368035070 con 0x7fd36c1035b0 2024-07-26T13:43:23.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.291+0000 7fd373c65640 1 -- 172.21.15.100:0/407101855 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd36c1a8930 con 0x7fd36c1035b0 2024-07-26T13:43:23.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.294+0000 7fd37a1e9740 1 Processor -- start 2024-07-26T13:43:23.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.294+0000 7fd37a1e9740 1 -- start start 2024-07-26T13:43:23.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.295+0000 7fd37a1e9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624de373070 0x5624de718570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:23.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.295+0000 7fd37a1e9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624de34a1b0 0x5624de718ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:23.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.295+0000 7fd37a1e9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624de352390 0x5624de718ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:23.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.295+0000 7fd37a1e9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5624de68ebb0 con 0x5624de352390 2024-07-26T13:43:23.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.295+0000 7fd37a1e9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5624de67e0b0 con 0x5624de34a1b0 2024-07-26T13:43:23.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.295+0000 7fd37a1e9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5624de6820f0 con 0x5624de373070 2024-07-26T13:43:23.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.295+0000 7fd362ffd640 1 -- 172.21.15.100:0/407101855 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fd368042030 con 0x7fd36c1035b0 2024-07-26T13:43:23.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.295+0000 7fd372202640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624de373070 0x5624de718570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:23.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.295+0000 7fd372202640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624de373070 0x5624de718570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:41254/0 (socket says 172.21.15.100:41254) 2024-07-26T13:43:23.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.295+0000 7fd372202640 1 -- 172.21.15.100:0/1001889050 learned_addr learned my addr 172.21.15.100:0/1001889050 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:23.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.295+0000 7fd371200640 1 --2- 172.21.15.100:0/1001889050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624de352390 0x5624de718ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:23.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.295+0000 7fd371a01640 1 --2- 172.21.15.100:0/1001889050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624de34a1b0 0x5624de718ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:23.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.295+0000 7fd372202640 1 -- 172.21.15.100:0/1001889050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624de34a1b0 msgr2=0x5624de718ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:23.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.295+0000 7fd372202640 1 --2- 172.21.15.100:0/1001889050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624de34a1b0 0x5624de718ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:23.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.295+0000 7fd372202640 1 -- 172.21.15.100:0/1001889050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624de352390 msgr2=0x5624de718ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:23.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.295+0000 7fd372202640 1 --2- 172.21.15.100:0/1001889050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624de352390 0x5624de718ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:23.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.295+0000 7fd372202640 1 -- 172.21.15.100:0/1001889050 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624de624b90 con 0x5624de373070 2024-07-26T13:43:23.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.295+0000 7fd371200640 1 --2- 172.21.15.100:0/1001889050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624de352390 0x5624de718ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:43:23.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.296+0000 7fd372202640 1 --2- 172.21.15.100:0/1001889050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624de373070 0x5624de718570 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd364009fd0 tx=0x7fd3640118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73a5af86b6b890e5 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:23.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.296+0000 7fd357fff640 1 -- 172.21.15.100:0/1001889050 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd364004d20 con 0x5624de373070 2024-07-26T13:43:23.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.296+0000 7fd362ffd640 1 --2- 172.21.15.100:0/407101855 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd3580b3d40 0x7fd3580b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:23.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.296+0000 7fd357fff640 1 -- 172.21.15.100:0/1001889050 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd364004050 con 0x5624de373070 2024-07-26T13:43:23.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.296+0000 7fd357fff640 1 -- 172.21.15.100:0/1001889050 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd364005260 con 0x5624de373070 2024-07-26T13:43:23.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.296+0000 7fd371a01640 1 --2- 172.21.15.100:0/407101855 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd3580b3d40 0x7fd3580b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:23.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.296+0000 7fd362ffd640 1 -- 172.21.15.100:0/407101855 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7fd368104050 con 0x7fd36c1035b0 2024-07-26T13:43:23.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.297+0000 7fd37a1e9740 1 -- 172.21.15.100:0/1001889050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624de373070 msgr2=0x5624de718570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:23.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.297+0000 7fd371a01640 1 --2- 172.21.15.100:0/407101855 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd3580b3d40 0x7fd3580b6250 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd368101820 tx=0x7fd3681078d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:23.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.297+0000 7fd37a1e9740 1 --2- 172.21.15.100:0/1001889050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624de373070 0x5624de718570 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd364009fd0 tx=0x7fd3640118d0 comp rx=0 tx=0).stop 2024-07-26T13:43:23.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.297+0000 7fd37a1e9740 1 -- 172.21.15.100:0/1001889050 shutdown_connections 2024-07-26T13:43:23.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.297+0000 7fd37a1e9740 1 --2- 172.21.15.100:0/1001889050 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624de352390 0x5624de718ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:23.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.297+0000 7fd37a1e9740 1 --2- 172.21.15.100:0/1001889050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624de34a1b0 0x5624de718ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:23.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.297+0000 7fd37a1e9740 1 --2- 172.21.15.100:0/1001889050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624de373070 0x5624de718570 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:23.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.297+0000 7fd37a1e9740 1 -- 172.21.15.100:0/1001889050 >> 172.21.15.100:0/1001889050 conn(0x5624de367460 msgr2=0x5624de716460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:23.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.297+0000 7fd37a1e9740 1 -- 172.21.15.100:0/1001889050 shutdown_connections 2024-07-26T13:43:23.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.297+0000 7fd37a1e9740 1 -- 172.21.15.100:0/1001889050 wait complete. 2024-07-26T13:43:23.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.298+0000 7fd37a1e9740 1 -- start start 2024-07-26T13:43:23.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.298+0000 7fd37a1e9740 1 Processor -- start 2024-07-26T13:43:23.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.298+0000 7fd37a1e9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624de34a1b0 0x5624de724110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:23.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.298+0000 7fd37a1e9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624de352390 0x5624de724650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:23.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.298+0000 7fd37a1e9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624de373070 0x5624de730070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:23.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.298+0000 7fd37a1e9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5624de4e1700 con 0x5624de373070 2024-07-26T13:43:23.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.298+0000 7fd37a1e9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5624de6209e0 con 0x5624de34a1b0 2024-07-26T13:43:23.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.298+0000 7fd37a1e9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5624de726d50 con 0x5624de352390 2024-07-26T13:43:23.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.298+0000 7fd37a1e9740 10 client.0 _ll_register_callbacks cb 0x5624de63b4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:43:23.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.298+0000 7fd372202640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624de34a1b0 0x5624de724110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:23.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.298+0000 7fd371200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624de352390 0x5624de724650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:23.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.298+0000 7fd372202640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624de34a1b0 0x5624de724110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:59018/0 (socket says 172.21.15.100:59018) 2024-07-26T13:43:23.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.298+0000 7fd372202640 1 -- 172.21.15.100:0/299978781 learned_addr learned my addr 172.21.15.100:0/299978781 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:23.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.298+0000 7fd372202640 1 --2- 172.21.15.100:0/299978781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624de373070 0x5624de730070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:23.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.299+0000 7fd371200640 1 -- 172.21.15.100:0/299978781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624de34a1b0 msgr2=0x5624de724110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:23.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.299+0000 7fd371200640 1 --2- 172.21.15.100:0/299978781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624de34a1b0 0x5624de724110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:23.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.299+0000 7fd371200640 1 -- 172.21.15.100:0/299978781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624de373070 msgr2=0x5624de730070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:23.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.299+0000 7fd371200640 1 --2- 172.21.15.100:0/299978781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624de373070 0x5624de730070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:23.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.299+0000 7fd371200640 1 -- 172.21.15.100:0/299978781 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624de3de650 con 0x5624de352390 2024-07-26T13:43:23.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.299+0000 7fd372202640 1 --2- 172.21.15.100:0/299978781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624de34a1b0 0x5624de724110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:43:23.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.299+0000 7fd371200640 1 --2- 172.21.15.100:0/299978781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624de352390 0x5624de724650 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd36c06c3f0 tx=0x7fd35c012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5985a14b42634a8 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:23.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.299+0000 7fd3557fa640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-07-26T13:43:23.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.299+0000 7fd3557fa640 1 -- 172.21.15.100:0/299978781 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c004030 con 0x5624de352390 2024-07-26T13:43:23.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.299+0000 7fd3557fa640 1 -- 172.21.15.100:0/299978781 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7fd35c0041d0 con 0x5624de352390 2024-07-26T13:43:23.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.299+0000 7fd37a1e9740 1 -- 172.21.15.100:0/299978781 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd364004050 con 0x5624de352390 2024-07-26T13:43:23.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.299+0000 7fd3557fa640 1 -- 172.21.15.100:0/299978781 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd35c0043d0 con 0x5624de352390 2024-07-26T13:43:23.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.299+0000 7fd37a1e9740 10 client.25846 fetch_fsmap learned FSMap version 36 2024-07-26T13:43:23.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.299+0000 7fd37a1e9740 1 -- 172.21.15.100:0/299978781 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5624de732df0 con 0x5624de352390 2024-07-26T13:43:23.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.300+0000 7fd3557fa640 1 -- 172.21.15.100:0/299978781 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd364004050 con 0x5624de352390 2024-07-26T13:43:23.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.300+0000 7fd3557fa640 1 -- 172.21.15.100:0/299978781 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7fd35c023070 con 0x5624de352390 2024-07-26T13:43:23.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.300+0000 7fd37a1e9740 10 client.25846 fetch_fsmap finished waiting for FSMap version 36 2024-07-26T13:43:23.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.300+0000 7fd37a1e9740 10 client.25846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:43:23.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.300+0000 7fd37a1e9740 20 client.25846 populate_metadata read hostname 'smithi100' 2024-07-26T13:43:23.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.300+0000 7fd37a1e9740 1 --2- 172.21.15.100:0/299978781 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5624de3319f0 0x5624de7363c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:23.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.300+0000 7fd37a1e9740 4 client.25846 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:43:23.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.300+0000 7fd37a1e9740 1 -- 172.21.15.100:0/299978781 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624de690b00 con 0x5624de3319f0 2024-07-26T13:43:23.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.300+0000 7fd372202640 1 --2- 172.21.15.100:0/299978781 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5624de3319f0 0x5624de7363c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:23.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.300+0000 7fd3557fa640 10 client.25846 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:43:23.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.300+0000 7fd372202640 1 --2- 172.21.15.100:0/299978781 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5624de3319f0 0x5624de7363c0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eaab8a999d974c9 server_cookie=ff9d2076cb0c1fbb in_seq=0 out_seq=0 2024-07-26T13:43:23.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.301+0000 7fd3557fa640 1 -- 172.21.15.100:0/299978781 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5624de690b00 con 0x5624de3319f0 2024-07-26T13:43:23.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.301+0000 7fd3557fa640 10 client.25846 handle_command_reply: tid=0 2024-07-26T13:43:23.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.301+0000 7fd37a1e9740 1 client.25846 shutdown 2024-07-26T13:43:23.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.301+0000 7fd37a1e9740 1 -- 172.21.15.100:0/299978781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624de352390 msgr2=0x5624de724650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:23.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.301+0000 7fd37a1e9740 1 --2- 172.21.15.100:0/299978781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624de352390 0x5624de724650 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd36c06c3f0 tx=0x7fd35c012b60 comp rx=0 tx=0).stop 2024-07-26T13:43:23.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.302+0000 7fd37a1e9740 1 -- 172.21.15.100:0/299978781 shutdown_connections 2024-07-26T13:43:23.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.302+0000 7fd37a1e9740 1 --2- 172.21.15.100:0/299978781 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5624de3319f0 0x5624de7363c0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:23.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.302+0000 7fd37a1e9740 1 --2- 172.21.15.100:0/299978781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624de373070 0x5624de730070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:23.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.302+0000 7fd37a1e9740 1 --2- 172.21.15.100:0/299978781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624de352390 0x5624de724650 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:23.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.302+0000 7fd37a1e9740 1 --2- 172.21.15.100:0/299978781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624de34a1b0 0x5624de724110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:23.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.302+0000 7fd3557fa640 0 client.25846 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:43:23.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.302+0000 7fd37a1e9740 1 -- 172.21.15.100:0/299978781 >> 172.21.15.100:0/299978781 conn(0x5624de367460 msgr2=0x5624de71b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:23.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.302+0000 7fd37a1e9740 1 -- 172.21.15.100:0/299978781 shutdown_connections 2024-07-26T13:43:23.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.302+0000 7fd37a1e9740 1 -- 172.21.15.100:0/299978781 wait complete. 2024-07-26T13:43:23.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.302+0000 7fd37a1e9740 20 client.25846 trim_cache size 0 max 16384 2024-07-26T13:43:23.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.315+0000 7fd37a1e9740 1 Processor -- start 2024-07-26T13:43:23.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.315+0000 7fd37a1e9740 1 -- start start 2024-07-26T13:43:23.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.315+0000 7fd37a1e9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624de34a1b0 0x5624de716490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:23.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.315+0000 7fd37a1e9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624de352390 0x5624de7113e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:23.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.315+0000 7fd37a1e9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624de373070 0x5624de71b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:23.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.316+0000 7fd37a1e9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c1b3dc0 con 0x5624de352390 2024-07-26T13:43:23.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.316+0000 7fd37a1e9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c1b1630 con 0x5624de373070 2024-07-26T13:43:23.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.316+0000 7fd37a1e9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c1af480 con 0x5624de34a1b0 2024-07-26T13:43:23.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.316+0000 7fd371200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624de352390 0x5624de7113e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:23.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.316+0000 7fd371200640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624de352390 0x5624de7113e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56264/0 (socket says 172.21.15.100:56264) 2024-07-26T13:43:23.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.316+0000 7fd371200640 1 -- 172.21.15.100:0/1387938276 learned_addr learned my addr 172.21.15.100:0/1387938276 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:23.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.316+0000 7fd372202640 1 --2- 172.21.15.100:0/1387938276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624de373070 0x5624de71b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:23.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.316+0000 7fd372202640 1 --2- 172.21.15.100:0/1387938276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624de34a1b0 0x5624de716490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:23.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.316+0000 7fd371200640 1 -- 172.21.15.100:0/1387938276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624de34a1b0 msgr2=0x5624de716490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:23.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.316+0000 7fd371200640 1 --2- 172.21.15.100:0/1387938276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624de34a1b0 0x5624de716490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:23.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.316+0000 7fd371200640 1 -- 172.21.15.100:0/1387938276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624de373070 msgr2=0x5624de71b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:23.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.316+0000 7fd371200640 1 --2- 172.21.15.100:0/1387938276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624de373070 0x5624de71b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:23.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.316+0000 7fd371200640 1 -- 172.21.15.100:0/1387938276 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624de722e70 con 0x5624de352390 2024-07-26T13:43:23.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.316+0000 7fd372202640 1 --2- 172.21.15.100:0/1387938276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624de34a1b0 0x5624de716490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:43:23.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.316+0000 7fd371200640 1 --2- 172.21.15.100:0/1387938276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624de352390 0x5624de7113e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fd35c009e80 tx=0x7fd35c009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae96df8f2c058c9c server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:23.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.316+0000 7fd3577fe640 1 -- 172.21.15.100:0/1387938276 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c029030 con 0x5624de352390 2024-07-26T13:43:23.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.316+0000 7fd3577fe640 1 -- 172.21.15.100:0/1387938276 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd35c003de0 con 0x5624de352390 2024-07-26T13:43:23.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.316+0000 7fd3577fe640 1 -- 172.21.15.100:0/1387938276 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c00fb20 con 0x5624de352390 2024-07-26T13:43:23.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.317+0000 7fd37a1e9740 1 -- 172.21.15.100:0/1387938276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624de352390 msgr2=0x5624de7113e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:23.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.317+0000 7fd37a1e9740 1 --2- 172.21.15.100:0/1387938276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624de352390 0x5624de7113e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fd35c009e80 tx=0x7fd35c009eb0 comp rx=0 tx=0).stop 2024-07-26T13:43:23.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.317+0000 7fd37a1e9740 1 -- 172.21.15.100:0/1387938276 shutdown_connections 2024-07-26T13:43:23.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.317+0000 7fd37a1e9740 1 --2- 172.21.15.100:0/1387938276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624de373070 0x5624de71b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:23.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.317+0000 7fd37a1e9740 1 --2- 172.21.15.100:0/1387938276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624de352390 0x5624de7113e0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:23.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.317+0000 7fd37a1e9740 1 --2- 172.21.15.100:0/1387938276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624de34a1b0 0x5624de716490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:23.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.317+0000 7fd37a1e9740 1 -- 172.21.15.100:0/1387938276 >> 172.21.15.100:0/1387938276 conn(0x5624de367460 msgr2=0x5624de715d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:23.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.317+0000 7fd37a1e9740 1 -- 172.21.15.100:0/1387938276 shutdown_connections 2024-07-26T13:43:23.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.317+0000 7fd37a1e9740 1 -- 172.21.15.100:0/1387938276 wait complete. 2024-07-26T13:43:23.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.317+0000 7fd37a1e9740 1 -- start start 2024-07-26T13:43:23.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.318+0000 7fd37a1e9740 1 Processor -- start 2024-07-26T13:43:23.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.318+0000 7fd37a1e9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624de34a1b0 0x5624de72c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:23.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.318+0000 7fd37a1e9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624de352390 0x5624de7278e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:23.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.318+0000 7fd37a1e9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624de373070 0x5624de727e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:23.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.318+0000 7fd37a1e9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c1afaf0 con 0x5624de34a1b0 2024-07-26T13:43:23.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.318+0000 7fd37a1e9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c1b0f60 con 0x5624de352390 2024-07-26T13:43:23.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.318+0000 7fd37a1e9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c1b01c0 con 0x5624de373070 2024-07-26T13:43:23.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.318+0000 7fd37a1e9740 10 client.0 _ll_register_callbacks cb 0x7fd35c009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:43:23.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.318+0000 7fd372202640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624de34a1b0 0x5624de72c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:23.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.318+0000 7fd372202640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624de34a1b0 0x5624de72c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56276/0 (socket says 172.21.15.100:56276) 2024-07-26T13:43:23.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.318+0000 7fd372202640 1 -- 172.21.15.100:0/4202588977 learned_addr learned my addr 172.21.15.100:0/4202588977 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:23.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.318+0000 7fd371200640 1 --2- 172.21.15.100:0/4202588977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624de352390 0x5624de7278e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:23.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.318+0000 7fd372202640 1 --2- 172.21.15.100:0/4202588977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624de373070 0x5624de727e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:23.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.318+0000 7fd371200640 1 -- 172.21.15.100:0/4202588977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624de373070 msgr2=0x5624de727e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:23.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.318+0000 7fd371200640 1 --2- 172.21.15.100:0/4202588977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624de373070 0x5624de727e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:23.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.319+0000 7fd371200640 1 -- 172.21.15.100:0/4202588977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624de34a1b0 msgr2=0x5624de72c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:23.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.319+0000 7fd371200640 1 --2- 172.21.15.100:0/4202588977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624de34a1b0 0x5624de72c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:23.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.319+0000 7fd371200640 1 -- 172.21.15.100:0/4202588977 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624de728360 con 0x5624de352390 2024-07-26T13:43:23.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.319+0000 7fd372202640 1 --2- 172.21.15.100:0/4202588977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624de34a1b0 0x5624de72c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:43:23.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.319+0000 7fd371200640 1 --2- 172.21.15.100:0/4202588977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624de352390 0x5624de7278e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x5624de7226a0 tx=0x7fd35c0038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e92c49bf10a10b3 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:23.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.319+0000 7fd3557fa640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-07-26T13:43:23.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.319+0000 7fd3557fa640 1 -- 172.21.15.100:0/4202588977 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c029040 con 0x5624de352390 2024-07-26T13:43:23.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.319+0000 7fd37a1e9740 1 -- 172.21.15.100:0/4202588977 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd35c003de0 con 0x5624de352390 2024-07-26T13:43:23.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.319+0000 7fd3557fa640 1 -- 172.21.15.100:0/4202588977 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7fd35c00faa0 con 0x5624de352390 2024-07-26T13:43:23.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.319+0000 7fd3557fa640 1 -- 172.21.15.100:0/4202588977 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd35c00fca0 con 0x5624de352390 2024-07-26T13:43:23.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.319+0000 7fd37a1e9740 10 client.25718 fetch_fsmap learned FSMap version 36 2024-07-26T13:43:23.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.319+0000 7fd37a1e9740 1 -- 172.21.15.100:0/4202588977 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5624de79e780 con 0x5624de352390 2024-07-26T13:43:23.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.319+0000 7fd3557fa640 1 -- 172.21.15.100:0/4202588977 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd35c003de0 con 0x5624de352390 2024-07-26T13:43:23.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.320+0000 7fd3557fa640 1 -- 172.21.15.100:0/4202588977 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7fd35c03d070 con 0x5624de352390 2024-07-26T13:43:23.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.320+0000 7fd37a1e9740 10 client.25718 fetch_fsmap finished waiting for FSMap version 36 2024-07-26T13:43:23.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.320+0000 7fd37a1e9740 10 client.25718 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:43:23.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.320+0000 7fd37a1e9740 20 client.25718 populate_metadata read hostname 'smithi100' 2024-07-26T13:43:23.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.320+0000 7fd37a1e9740 1 --2- 172.21.15.100:0/4202588977 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5624de306da0 0x5624de7a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:23.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.320+0000 7fd37a1e9740 4 client.25718 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-26T13:43:23.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.320+0000 7fd37a1e9740 1 -- 172.21.15.100:0/4202588977 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5624de68c5a0 con 0x5624de306da0 2024-07-26T13:43:23.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.320+0000 7fd372202640 1 --2- 172.21.15.100:0/4202588977 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5624de306da0 0x5624de7a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:23.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.320+0000 7fd3557fa640 10 client.25718 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:43:23.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.320+0000 7fd372202640 1 --2- 172.21.15.100:0/4202588977 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5624de306da0 0x5624de7a0e90 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4e5e49bee72afd7 server_cookie=682853b53e7f8467 in_seq=0 out_seq=0 2024-07-26T13:43:23.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.320+0000 7fd3557fa640 1 -- 172.21.15.100:0/4202588977 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5624de68c5a0 con 0x5624de306da0 2024-07-26T13:43:23.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.320+0000 7fd3557fa640 10 client.25718 handle_command_reply: tid=0 2024-07-26T13:43:23.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.320+0000 7fd37a1e9740 1 client.25718 shutdown 2024-07-26T13:43:23.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.321+0000 7fd37a1e9740 1 -- 172.21.15.100:0/4202588977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624de352390 msgr2=0x5624de7278e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:23.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.321+0000 7fd37a1e9740 1 --2- 172.21.15.100:0/4202588977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624de352390 0x5624de7278e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x5624de7226a0 tx=0x7fd35c0038a0 comp rx=0 tx=0).stop 2024-07-26T13:43:23.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.321+0000 7fd37a1e9740 1 -- 172.21.15.100:0/4202588977 shutdown_connections 2024-07-26T13:43:23.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.321+0000 7fd37a1e9740 1 --2- 172.21.15.100:0/4202588977 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5624de306da0 0x5624de7a0e90 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:23.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.321+0000 7fd37a1e9740 1 --2- 172.21.15.100:0/4202588977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624de373070 0x5624de727e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:23.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.321+0000 7fd37a1e9740 1 --2- 172.21.15.100:0/4202588977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624de352390 0x5624de7278e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:23.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.321+0000 7fd3557fa640 0 client.25718 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:43:23.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.321+0000 7fd37a1e9740 1 --2- 172.21.15.100:0/4202588977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624de34a1b0 0x5624de72c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:23.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.321+0000 7fd37a1e9740 1 -- 172.21.15.100:0/4202588977 >> 172.21.15.100:0/4202588977 conn(0x5624de367460 msgr2=0x5624de71b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:23.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.321+0000 7fd37a1e9740 1 -- 172.21.15.100:0/4202588977 shutdown_connections 2024-07-26T13:43:23.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.321+0000 7fd37a1e9740 1 -- 172.21.15.100:0/4202588977 wait complete. 2024-07-26T13:43:23.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.321+0000 7fd37a1e9740 20 client.25718 trim_cache size 0 max 16384 2024-07-26T13:43:23.328 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-07-26T13:43:23.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.321+0000 7fd373c65640 1 -- 172.21.15.100:0/407101855 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd3580b3d40 msgr2=0x7fd3580b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:23.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.321+0000 7fd373c65640 1 --2- 172.21.15.100:0/407101855 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd3580b3d40 0x7fd3580b6250 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd368101820 tx=0x7fd3681078d0 comp rx=0 tx=0).stop 2024-07-26T13:43:23.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.321+0000 7fd373c65640 1 -- 172.21.15.100:0/407101855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd36c1035b0 msgr2=0x7fd36c075140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:23.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.321+0000 7fd373c65640 1 --2- 172.21.15.100:0/407101855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd36c1035b0 0x7fd36c075140 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fd368005940 tx=0x7fd3680040c0 comp rx=0 tx=0).stop 2024-07-26T13:43:23.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.321+0000 7fd373c65640 1 -- 172.21.15.100:0/407101855 shutdown_connections 2024-07-26T13:43:23.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.321+0000 7fd373c65640 1 --2- 172.21.15.100:0/407101855 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd3580b3d40 0x7fd3580b6250 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:23.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.321+0000 7fd373c65640 1 --2- 172.21.15.100:0/407101855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd36c109840 0x7fd36c075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:23.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.321+0000 7fd373c65640 1 --2- 172.21.15.100:0/407101855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c108ef0 0x7fd36c075680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:23.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.322+0000 7fd373c65640 1 --2- 172.21.15.100:0/407101855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd36c1035b0 0x7fd36c075140 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:23.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.322+0000 7fd373c65640 1 -- 172.21.15.100:0/407101855 >> 172.21.15.100:0/407101855 conn(0x7fd36c0fefe0 msgr2=0x7fd36c108590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:23.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.322+0000 7fd373c65640 1 -- 172.21.15.100:0/407101855 shutdown_connections 2024-07-26T13:43:23.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:23.322+0000 7fd373c65640 1 -- 172.21.15.100:0/407101855 wait complete. 2024-07-26T13:43:23.408 INFO:tasks.workunit.client.0.smithi100.stdout: 1 99919 5.74 MB/sec warmup 97 sec latency 11.169 ms 2024-07-26T13:43:23.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:23 smithi167 ceph-mon[37140]: pgmap v463: 97 pgs: 97 active+clean; 49 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 250 op/s 2024-07-26T13:43:23.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:23 smithi138 ceph-mon[36954]: pgmap v463: 97 pgs: 97 active+clean; 49 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 250 op/s 2024-07-26T13:43:23.965 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:23 smithi100 ceph-mon[32599]: pgmap v463: 97 pgs: 97 active+clean; 49 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 250 op/s 2024-07-26T13:43:24.408 INFO:tasks.workunit.client.0.smithi100.stdout: 1 102376 5.81 MB/sec warmup 98 sec latency 12.840 ms 2024-07-26T13:43:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.713+0000 7f9d19dac640 1 -- 172.21.15.100:0/3941422341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d14107d30 msgr2=0x7f9d14108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.713+0000 7f9d19dac640 1 --2- 172.21.15.100:0/3941422341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d14107d30 0x7f9d14108140 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc009900 tx=0x7f9cfc02a410 comp rx=0 tx=0).stop 2024-07-26T13:43:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.713+0000 7f9d19dac640 1 -- 172.21.15.100:0/3941422341 shutdown_connections 2024-07-26T13:43:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.713+0000 7f9d19dac640 1 --2- 172.21.15.100:0/3941422341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d14108680 0x7f9d1410eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.713+0000 7f9d19dac640 1 --2- 172.21.15.100:0/3941422341 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d14107d30 0x7f9d14108140 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.713+0000 7f9d19dac640 1 --2- 172.21.15.100:0/3941422341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d14102400 0x7f9d14102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.713+0000 7f9d19dac640 1 -- 172.21.15.100:0/3941422341 >> 172.21.15.100:0/3941422341 conn(0x7f9d140fde20 msgr2=0x7f9d14100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.713+0000 7f9d19dac640 1 -- 172.21.15.100:0/3941422341 shutdown_connections 2024-07-26T13:43:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.713+0000 7f9d19dac640 1 -- 172.21.15.100:0/3941422341 wait complete. 2024-07-26T13:43:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.714+0000 7f9d19dac640 1 Processor -- start 2024-07-26T13:43:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.714+0000 7f9d19dac640 1 -- start start 2024-07-26T13:43:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.714+0000 7f9d19dac640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d14102400 0x7f9d14199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.714+0000 7f9d19dac640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d14107d30 0x7f9d1419a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.714+0000 7f9d19dac640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d14108680 0x7f9d1419f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.714+0000 7f9d19dac640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9d14113480 con 0x7f9d14102400 2024-07-26T13:43:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.714+0000 7f9d19dac640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9d14113300 con 0x7f9d14107d30 2024-07-26T13:43:24.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.714+0000 7f9d19dac640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9d14113600 con 0x7f9d14108680 2024-07-26T13:43:24.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.714+0000 7f9d137fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d14102400 0x7f9d14199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:24.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.714+0000 7f9d137fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d14102400 0x7f9d14199e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56300/0 (socket says 172.21.15.100:56300) 2024-07-26T13:43:24.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.714+0000 7f9d0bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d14107d30 0x7f9d1419a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:24.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.714+0000 7f9d13fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d14108680 0x7f9d1419f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:24.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.714+0000 7f9d137fe640 1 -- 172.21.15.100:0/1216224321 learned_addr learned my addr 172.21.15.100:0/1216224321 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:24.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.715+0000 7f9d137fe640 1 -- 172.21.15.100:0/1216224321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d14108680 msgr2=0x7f9d1419f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:24.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.715+0000 7f9d137fe640 1 --2- 172.21.15.100:0/1216224321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d14108680 0x7f9d1419f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:24.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.715+0000 7f9d137fe640 1 -- 172.21.15.100:0/1216224321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d14107d30 msgr2=0x7f9d1419a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:24.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.715+0000 7f9d137fe640 1 --2- 172.21.15.100:0/1216224321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d14107d30 0x7f9d1419a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:24.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.715+0000 7f9d137fe640 1 -- 172.21.15.100:0/1216224321 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d1419fd70 con 0x7f9d14102400 2024-07-26T13:43:24.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.715+0000 7f9d137fe640 1 --2- 172.21.15.100:0/1216224321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d14102400 0x7f9d14199e40 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f9d0400e9f0 tx=0x7f9d0400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:24.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.715+0000 7f9d117fa640 1 -- 172.21.15.100:0/1216224321 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0400cd80 con 0x7f9d14102400 2024-07-26T13:43:24.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.715+0000 7f9d117fa640 1 -- 172.21.15.100:0/1216224321 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d040108e0 con 0x7f9d14102400 2024-07-26T13:43:24.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.715+0000 7f9d19dac640 1 -- 172.21.15.100:0/1216224321 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d141a0060 con 0x7f9d14102400 2024-07-26T13:43:24.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.715+0000 7f9d117fa640 1 -- 172.21.15.100:0/1216224321 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d04018980 con 0x7f9d14102400 2024-07-26T13:43:24.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.715+0000 7f9d19dac640 1 -- 172.21.15.100:0/1216224321 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d141a67f0 con 0x7f9d14102400 2024-07-26T13:43:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.718+0000 7f9d20d9e740 1 Processor -- start 2024-07-26T13:43:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.718+0000 7f9d20d9e740 1 -- start start 2024-07-26T13:43:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.718+0000 7f9d117fa640 1 -- 172.21.15.100:0/1216224321 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f9d04010430 con 0x7f9d14102400 2024-07-26T13:43:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.718+0000 7f9d20d9e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e7fb611b0 0x560e7ff03ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.718+0000 7f9d20d9e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e7fb382f0 0x560e7ff04220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.718+0000 7f9d20d9e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e7fb404d0 0x560e7ff04760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.718+0000 7f9d20d9e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x560e7fe793b0 con 0x560e7fb404d0 2024-07-26T13:43:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.718+0000 7f9d20d9e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560e7fe688b0 con 0x560e7fb611b0 2024-07-26T13:43:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.718+0000 7f9d20d9e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560e7fe6c8f0 con 0x560e7fb382f0 2024-07-26T13:43:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.718+0000 7f9d0bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e7fb404d0 0x560e7ff04760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.718+0000 7f9d13fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e7fb611b0 0x560e7ff03ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.718+0000 7f9d13fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e7fb611b0 0x560e7ff03ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:59068/0 (socket says 172.21.15.100:59068) 2024-07-26T13:43:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.718+0000 7f9d13fff640 1 -- 172.21.15.100:0/3130405562 learned_addr learned my addr 172.21.15.100:0/3130405562 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.718+0000 7f9d0bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e7fb404d0 0x560e7ff04760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56308/0 (socket says 172.21.15.100:56308) 2024-07-26T13:43:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.718+0000 7f9d137fe640 1 --2- 172.21.15.100:0/3130405562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e7fb382f0 0x560e7ff04220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.718+0000 7f9d13fff640 1 -- 172.21.15.100:0/3130405562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e7fb382f0 msgr2=0x560e7ff04220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.718+0000 7f9d13fff640 1 --2- 172.21.15.100:0/3130405562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e7fb382f0 0x560e7ff04220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.718+0000 7f9d13fff640 1 -- 172.21.15.100:0/3130405562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e7fb404d0 msgr2=0x560e7ff04760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.718+0000 7f9d13fff640 1 --2- 172.21.15.100:0/3130405562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e7fb404d0 0x560e7ff04760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.718+0000 7f9d13fff640 1 -- 172.21.15.100:0/3130405562 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e7fe0f9a0 con 0x560e7fb611b0 2024-07-26T13:43:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.718+0000 7f9d137fe640 1 --2- 172.21.15.100:0/3130405562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e7fb382f0 0x560e7ff04220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:43:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.719+0000 7f9d13fff640 1 --2- 172.21.15.100:0/3130405562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e7fb611b0 0x560e7ff03ce0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c011440 tx=0x7f9d0c011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54c204f5c8a0aea6 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.719+0000 7f9d09ffb640 1 -- 172.21.15.100:0/3130405562 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c004d20 con 0x560e7fb611b0 2024-07-26T13:43:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.719+0000 7f9d09ffb640 1 -- 172.21.15.100:0/3130405562 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d0c004050 con 0x560e7fb611b0 2024-07-26T13:43:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.719+0000 7f9d09ffb640 1 -- 172.21.15.100:0/3130405562 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c00f760 con 0x560e7fb611b0 2024-07-26T13:43:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.719+0000 7f9d117fa640 1 --2- 172.21.15.100:0/1216224321 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9d000b3d40 0x7f9d000b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.719+0000 7f9d137fe640 1 --2- 172.21.15.100:0/1216224321 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9d000b3d40 0x7f9d000b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.719+0000 7f9d117fa640 1 -- 172.21.15.100:0/1216224321 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f9d040e4340 con 0x7f9d14102400 2024-07-26T13:43:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.720+0000 7f9d20d9e740 1 -- 172.21.15.100:0/3130405562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e7fb611b0 msgr2=0x560e7ff03ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.720+0000 7f9d137fe640 1 --2- 172.21.15.100:0/1216224321 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9d000b3d40 0x7f9d000b6250 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9d040e1b10 tx=0x7f9d04025330 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.720+0000 7f9d20d9e740 1 --2- 172.21.15.100:0/3130405562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e7fb611b0 0x560e7ff03ce0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c011440 tx=0x7f9d0c011900 comp rx=0 tx=0).stop 2024-07-26T13:43:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.720+0000 7f9d20d9e740 1 -- 172.21.15.100:0/3130405562 shutdown_connections 2024-07-26T13:43:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.720+0000 7f9d20d9e740 1 --2- 172.21.15.100:0/3130405562 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e7fb404d0 0x560e7ff04760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.720+0000 7f9d20d9e740 1 --2- 172.21.15.100:0/3130405562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e7fb382f0 0x560e7ff04220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.720+0000 7f9d20d9e740 1 --2- 172.21.15.100:0/3130405562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e7fb611b0 0x560e7ff03ce0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.720+0000 7f9d20d9e740 1 -- 172.21.15.100:0/3130405562 >> 172.21.15.100:0/3130405562 conn(0x560e7fb55300 msgr2=0x560e7ff01bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.720+0000 7f9d20d9e740 1 -- 172.21.15.100:0/3130405562 shutdown_connections 2024-07-26T13:43:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.720+0000 7f9d20d9e740 1 -- 172.21.15.100:0/3130405562 wait complete. 2024-07-26T13:43:24.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.720+0000 7f9d20d9e740 1 -- start start 2024-07-26T13:43:24.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.720+0000 7f9d20d9e740 1 Processor -- start 2024-07-26T13:43:24.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.721+0000 7f9d20d9e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e7fb382f0 0x560e7ff0f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:24.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.721+0000 7f9d20d9e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e7fb404d0 0x560e7ff0fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:24.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.721+0000 7f9d20d9e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e7fb611b0 0x560e7ff1b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:24.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.721+0000 7f9d20d9e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x560e7fda2ab0 con 0x560e7fb404d0 2024-07-26T13:43:24.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.721+0000 7f9d20d9e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560e7fdf0cf0 con 0x560e7fb382f0 2024-07-26T13:43:24.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.721+0000 7f9d20d9e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560e7ff12460 con 0x560e7fb611b0 2024-07-26T13:43:24.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.721+0000 7f9d20d9e740 10 client.0 _ll_register_callbacks cb 0x560e7fe629f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:43:24.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.721+0000 7f9d0bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e7fb404d0 0x560e7ff0fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:24.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.721+0000 7f9d0bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e7fb404d0 0x560e7ff0fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56310/0 (socket says 172.21.15.100:56310) 2024-07-26T13:43:24.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.721+0000 7f9d0bfff640 1 -- 172.21.15.100:0/1498212483 learned_addr learned my addr 172.21.15.100:0/1498212483 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:24.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.721+0000 7f9d13fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e7fb382f0 0x560e7ff0f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.721+0000 7f9d13fff640 1 --2- 172.21.15.100:0/1498212483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e7fb611b0 0x560e7ff1b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.721+0000 7f9d0bfff640 1 -- 172.21.15.100:0/1498212483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e7fb611b0 msgr2=0x560e7ff1b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.721+0000 7f9d0bfff640 1 --2- 172.21.15.100:0/1498212483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e7fb611b0 0x560e7ff1b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.721+0000 7f9d0bfff640 1 -- 172.21.15.100:0/1498212483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e7fb382f0 msgr2=0x560e7ff0f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.721+0000 7f9d0bfff640 1 --2- 172.21.15.100:0/1498212483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e7fb382f0 0x560e7ff0f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.721+0000 7f9d0bfff640 1 -- 172.21.15.100:0/1498212483 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e7fbcc790 con 0x560e7fb404d0 2024-07-26T13:43:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.721+0000 7f9d13fff640 1 --2- 172.21.15.100:0/1498212483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e7fb382f0 0x560e7ff0f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:43:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.721+0000 7f9d0bfff640 1 --2- 172.21.15.100:0/1498212483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e7fb404d0 0x560e7ff0fd50 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f9d1406b1d0 tx=0x7f9cfc0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f71af09cb6edb3fd server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.721+0000 7f9cf2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:43:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.722+0000 7f9cf2ffd640 1 -- 172.21.15.100:0/1498212483 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc035040 con 0x560e7fb404d0 2024-07-26T13:43:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.722+0000 7f9cf2ffd640 1 -- 172.21.15.100:0/1498212483 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f9cfc005450 con 0x560e7fb404d0 2024-07-26T13:43:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.722+0000 7f9d20d9e740 1 -- 172.21.15.100:0/1498212483 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d0c004050 con 0x560e7fb404d0 2024-07-26T13:43:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.722+0000 7f9cf2ffd640 1 -- 172.21.15.100:0/1498212483 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9cfc005650 con 0x560e7fb404d0 2024-07-26T13:43:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.722+0000 7f9d20d9e740 10 client.16368 fetch_fsmap learned FSMap version 36 2024-07-26T13:43:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.722+0000 7f9d20d9e740 1 -- 172.21.15.100:0/1498212483 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560e7fc45620 con 0x560e7fb404d0 2024-07-26T13:43:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.722+0000 7f9cf2ffd640 1 -- 172.21.15.100:0/1498212483 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d0c004050 con 0x560e7fb404d0 2024-07-26T13:43:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.722+0000 7f9cf2ffd640 1 -- 172.21.15.100:0/1498212483 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f9cfc04d070 con 0x560e7fb404d0 2024-07-26T13:43:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.722+0000 7f9d20d9e740 10 client.16368 fetch_fsmap finished waiting for FSMap version 36 2024-07-26T13:43:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.722+0000 7f9d20d9e740 10 client.16368 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:43:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.722+0000 7f9d20d9e740 20 client.16368 populate_metadata read hostname 'smithi100' 2024-07-26T13:43:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.722+0000 7f9d20d9e740 1 --2- 172.21.15.100:0/1498212483 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560e7fb1fd80 0x560e7ff217f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.722+0000 7f9d20d9e740 4 client.16368 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:43:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.722+0000 7f9d20d9e740 1 -- 172.21.15.100:0/1498212483 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e7fe7b300 con 0x560e7fb1fd80 2024-07-26T13:43:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.722+0000 7f9d13fff640 1 --2- 172.21.15.100:0/1498212483 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560e7fb1fd80 0x560e7ff217f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.723+0000 7f9d13fff640 1 --2- 172.21.15.100:0/1498212483 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560e7fb1fd80 0x560e7ff217f0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf76f8a939a6b76 server_cookie=d94e759d76e1076b in_seq=0 out_seq=0 2024-07-26T13:43:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.723+0000 7f9cf2ffd640 10 client.16368 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:43:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.724+0000 7f9cf2ffd640 1 -- 172.21.15.100:0/1498212483 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560e7fe7b300 con 0x560e7fb1fd80 2024-07-26T13:43:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.724+0000 7f9cf2ffd640 10 client.16368 handle_command_reply: tid=0 2024-07-26T13:43:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.724+0000 7f9d20d9e740 1 client.16368 shutdown 2024-07-26T13:43:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.724+0000 7f9d20d9e740 1 -- 172.21.15.100:0/1498212483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e7fb404d0 msgr2=0x560e7ff0fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.724+0000 7f9d20d9e740 1 --2- 172.21.15.100:0/1498212483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e7fb404d0 0x560e7ff0fd50 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f9d1406b1d0 tx=0x7f9cfc0043b0 comp rx=0 tx=0).stop 2024-07-26T13:43:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.724+0000 7f9d20d9e740 1 -- 172.21.15.100:0/1498212483 shutdown_connections 2024-07-26T13:43:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.724+0000 7f9d20d9e740 1 --2- 172.21.15.100:0/1498212483 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560e7fb1fd80 0x560e7ff217f0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.724+0000 7f9d20d9e740 1 --2- 172.21.15.100:0/1498212483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e7fb611b0 0x560e7ff1b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.724+0000 7f9d20d9e740 1 --2- 172.21.15.100:0/1498212483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e7fb404d0 0x560e7ff0fd50 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.724+0000 7f9cf2ffd640 0 client.16368 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:43:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.724+0000 7f9d20d9e740 1 --2- 172.21.15.100:0/1498212483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e7fb382f0 0x560e7ff0f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.724+0000 7f9d20d9e740 1 -- 172.21.15.100:0/1498212483 >> 172.21.15.100:0/1498212483 conn(0x560e7fb55300 msgr2=0x560e7ff04d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.724+0000 7f9d20d9e740 1 -- 172.21.15.100:0/1498212483 shutdown_connections 2024-07-26T13:43:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.724+0000 7f9d20d9e740 1 -- 172.21.15.100:0/1498212483 wait complete. 2024-07-26T13:43:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.724+0000 7f9d20d9e740 20 client.16368 trim_cache size 0 max 16384 2024-07-26T13:43:24.743 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.741+0000 7f9d20d9e740 1 Processor -- start 2024-07-26T13:43:24.743 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.741+0000 7f9d20d9e740 1 -- start start 2024-07-26T13:43:24.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.741+0000 7f9d20d9e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e7fb1fd80 0x560e7ff04d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:24.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.742+0000 7f9d20d9e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e7fb382f0 0x560e7ff05250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:24.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.742+0000 7f9d20d9e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e7fb404d0 0x560e7ff1b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:24.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.742+0000 7f9d20d9e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9d141b1a30 con 0x560e7fb382f0 2024-07-26T13:43:24.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.742+0000 7f9d20d9e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9d141af2a0 con 0x560e7fb1fd80 2024-07-26T13:43:24.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.742+0000 7f9d20d9e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9d141ad0f0 con 0x560e7fb404d0 2024-07-26T13:43:24.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.742+0000 7f9d0bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e7fb382f0 0x560e7ff05250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:24.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.742+0000 7f9d0bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e7fb382f0 0x560e7ff05250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56320/0 (socket says 172.21.15.100:56320) 2024-07-26T13:43:24.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.742+0000 7f9d0bfff640 1 -- 172.21.15.100:0/1432414065 learned_addr learned my addr 172.21.15.100:0/1432414065 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:24.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.742+0000 7f9d13fff640 1 --2- 172.21.15.100:0/1432414065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e7fb1fd80 0x560e7ff04d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:24.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.742+0000 7f9d13fff640 1 --2- 172.21.15.100:0/1432414065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e7fb404d0 0x560e7ff1b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:24.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.742+0000 7f9d0bfff640 1 -- 172.21.15.100:0/1432414065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e7fb404d0 msgr2=0x560e7ff1b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:24.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.742+0000 7f9d0bfff640 1 --2- 172.21.15.100:0/1432414065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e7fb404d0 0x560e7ff1b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:24.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.742+0000 7f9d0bfff640 1 -- 172.21.15.100:0/1432414065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e7fb1fd80 msgr2=0x560e7ff04d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:24.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.742+0000 7f9d0bfff640 1 --2- 172.21.15.100:0/1432414065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e7fb1fd80 0x560e7ff04d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:24.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.742+0000 7f9d0bfff640 1 -- 172.21.15.100:0/1432414065 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e7ff0db10 con 0x560e7fb382f0 2024-07-26T13:43:24.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.742+0000 7f9d0bfff640 1 --2- 172.21.15.100:0/1432414065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e7fb382f0 0x560e7ff05250 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc005ab0 tx=0x7f9cfc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=176e996a5e7a31f9 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:24.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.742+0000 7f9d097fa640 1 -- 172.21.15.100:0/1432414065 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc035030 con 0x560e7fb382f0 2024-07-26T13:43:24.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.742+0000 7f9d097fa640 1 -- 172.21.15.100:0/1432414065 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cfc041070 con 0x560e7fb382f0 2024-07-26T13:43:24.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.742+0000 7f9d097fa640 1 -- 172.21.15.100:0/1432414065 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc053070 con 0x560e7fb382f0 2024-07-26T13:43:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.743+0000 7f9d20d9e740 1 -- 172.21.15.100:0/1432414065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e7fb382f0 msgr2=0x560e7ff05250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.743+0000 7f9d20d9e740 1 --2- 172.21.15.100:0/1432414065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e7fb382f0 0x560e7ff05250 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc005ab0 tx=0x7f9cfc000bc0 comp rx=0 tx=0).stop 2024-07-26T13:43:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.743+0000 7f9d20d9e740 1 -- 172.21.15.100:0/1432414065 shutdown_connections 2024-07-26T13:43:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.743+0000 7f9d20d9e740 1 --2- 172.21.15.100:0/1432414065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e7fb404d0 0x560e7ff1b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.743+0000 7f9d20d9e740 1 --2- 172.21.15.100:0/1432414065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e7fb382f0 0x560e7ff05250 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.743+0000 7f9d20d9e740 1 --2- 172.21.15.100:0/1432414065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e7fb1fd80 0x560e7ff04d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.743+0000 7f9d20d9e740 1 -- 172.21.15.100:0/1432414065 >> 172.21.15.100:0/1432414065 conn(0x560e7fb55300 msgr2=0x560e7ff0de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.743+0000 7f9d20d9e740 1 -- 172.21.15.100:0/1432414065 shutdown_connections 2024-07-26T13:43:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.743+0000 7f9d20d9e740 1 -- 172.21.15.100:0/1432414065 wait complete. 2024-07-26T13:43:24.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.744+0000 7f9d20d9e740 1 -- start start 2024-07-26T13:43:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.744+0000 7f9d20d9e740 1 Processor -- start 2024-07-26T13:43:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.744+0000 7f9d20d9e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e7fb1fd80 0x560e7ff148e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.744+0000 7f9d20d9e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e7fb382f0 0x560e7ff14e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.744+0000 7f9d20d9e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e7fb404d0 0x560e7ff87f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.744+0000 7f9d20d9e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9d141ad760 con 0x560e7fb1fd80 2024-07-26T13:43:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.744+0000 7f9d20d9e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9d141aebd0 con 0x560e7fb382f0 2024-07-26T13:43:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.744+0000 7f9d20d9e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9d141ade30 con 0x560e7fb404d0 2024-07-26T13:43:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.744+0000 7f9d20d9e740 10 client.0 _ll_register_callbacks cb 0x7f9cfc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:43:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.744+0000 7f9d13fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e7fb1fd80 0x560e7ff148e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.744+0000 7f9d13fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e7fb1fd80 0x560e7ff148e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56324/0 (socket says 172.21.15.100:56324) 2024-07-26T13:43:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.744+0000 7f9d13fff640 1 -- 172.21.15.100:0/94021064 learned_addr learned my addr 172.21.15.100:0/94021064 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:24.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.744+0000 7f9d0bfff640 1 --2- 172.21.15.100:0/94021064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e7fb382f0 0x560e7ff14e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.744+0000 7f9d13fff640 1 --2- 172.21.15.100:0/94021064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e7fb404d0 0x560e7ff87f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.745+0000 7f9d13fff640 1 -- 172.21.15.100:0/94021064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e7fb404d0 msgr2=0x560e7ff87f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.745+0000 7f9d13fff640 1 --2- 172.21.15.100:0/94021064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e7fb404d0 0x560e7ff87f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.745+0000 7f9d13fff640 1 -- 172.21.15.100:0/94021064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e7fb382f0 msgr2=0x560e7ff14e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.745+0000 7f9d13fff640 1 --2- 172.21.15.100:0/94021064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e7fb382f0 0x560e7ff14e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.745+0000 7f9d13fff640 1 -- 172.21.15.100:0/94021064 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e7ff156f0 con 0x560e7fb1fd80 2024-07-26T13:43:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.745+0000 7f9d0bfff640 1 --2- 172.21.15.100:0/94021064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e7fb382f0 0x560e7ff14e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:43:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.745+0000 7f9d13fff640 1 --2- 172.21.15.100:0/94021064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e7fb1fd80 0x560e7ff148e0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c011e00 tx=0x7f9d0c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ea4a838ddc97c2 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.745+0000 7f9cf2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:43:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.745+0000 7f9cf2ffd640 1 -- 172.21.15.100:0/94021064 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c01d040 con 0x560e7fb1fd80 2024-07-26T13:43:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.745+0000 7f9cf2ffd640 1 -- 172.21.15.100:0/94021064 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f9d0c004bf0 con 0x560e7fb1fd80 2024-07-26T13:43:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.745+0000 7f9d20d9e740 1 -- 172.21.15.100:0/94021064 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cfc041070 con 0x560e7fb1fd80 2024-07-26T13:43:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.745+0000 7f9cf2ffd640 1 -- 172.21.15.100:0/94021064 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9d0c00f540 con 0x560e7fb1fd80 2024-07-26T13:43:24.752 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.745+0000 7f9d20d9e740 10 client.16374 fetch_fsmap learned FSMap version 36 2024-07-26T13:43:24.752 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.745+0000 7f9d20d9e740 1 -- 172.21.15.100:0/94021064 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560e7ff89550 con 0x560e7fb1fd80 2024-07-26T13:43:24.752 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.745+0000 7f9cf2ffd640 1 -- 172.21.15.100:0/94021064 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cfc041070 con 0x560e7fb1fd80 2024-07-26T13:43:24.752 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.745+0000 7f9cf2ffd640 1 -- 172.21.15.100:0/94021064 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f9d0c03b5e0 con 0x560e7fb1fd80 2024-07-26T13:43:24.752 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.745+0000 7f9d20d9e740 10 client.16374 fetch_fsmap finished waiting for FSMap version 36 2024-07-26T13:43:24.752 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.745+0000 7f9d20d9e740 10 client.16374 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:43:24.752 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.745+0000 7f9d20d9e740 20 client.16374 populate_metadata read hostname 'smithi100' 2024-07-26T13:43:24.752 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.745+0000 7f9d20d9e740 1 --2- 172.21.15.100:0/94021064 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560e7fb611b0 0x560e7ff8bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:24.752 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.745+0000 7f9d20d9e740 4 client.16374 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-26T13:43:24.752 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.745+0000 7f9d20d9e740 1 -- 172.21.15.100:0/94021064 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560e7fe76da0 con 0x560e7fb611b0 2024-07-26T13:43:24.753 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.746+0000 7f9d0bfff640 1 --2- 172.21.15.100:0/94021064 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560e7fb611b0 0x560e7ff8bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:24.753 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.746+0000 7f9cf2ffd640 10 client.16374 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:43:24.753 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.746+0000 7f9d0bfff640 1 --2- 172.21.15.100:0/94021064 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560e7fb611b0 0x560e7ff8bb50 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb06c6306764e56 server_cookie=44d9012f7256f373 in_seq=0 out_seq=0 2024-07-26T13:43:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.783+0000 7f9cf2ffd640 1 -- 172.21.15.100:0/94021064 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560e7fe76da0 con 0x560e7fb611b0 2024-07-26T13:43:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.783+0000 7f9cf2ffd640 10 client.16374 handle_command_reply: tid=0 2024-07-26T13:43:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.783+0000 7f9d20d9e740 1 client.16374 shutdown 2024-07-26T13:43:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.784+0000 7f9d20d9e740 1 -- 172.21.15.100:0/94021064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e7fb1fd80 msgr2=0x560e7ff148e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:24.785 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.784+0000 7f9d20d9e740 1 --2- 172.21.15.100:0/94021064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e7fb1fd80 0x560e7ff148e0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c011e00 tx=0x7f9d0c009500 comp rx=0 tx=0).stop 2024-07-26T13:43:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.784+0000 7f9d20d9e740 1 -- 172.21.15.100:0/94021064 shutdown_connections 2024-07-26T13:43:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.784+0000 7f9d20d9e740 1 --2- 172.21.15.100:0/94021064 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560e7fb611b0 0x560e7ff8bb50 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.784+0000 7f9d20d9e740 1 --2- 172.21.15.100:0/94021064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560e7fb404d0 0x560e7ff87f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.784+0000 7f9d20d9e740 1 --2- 172.21.15.100:0/94021064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e7fb382f0 0x560e7ff14e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.784+0000 7f9cf2ffd640 0 client.16374 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:43:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.784+0000 7f9d20d9e740 1 --2- 172.21.15.100:0/94021064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560e7fb1fd80 0x560e7ff148e0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.784+0000 7f9d20d9e740 1 -- 172.21.15.100:0/94021064 >> 172.21.15.100:0/94021064 conn(0x560e7fb55300 msgr2=0x560e7ff0dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.784+0000 7f9d20d9e740 1 -- 172.21.15.100:0/94021064 shutdown_connections 2024-07-26T13:43:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.784+0000 7f9d20d9e740 1 -- 172.21.15.100:0/94021064 wait complete. 2024-07-26T13:43:24.786 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.784+0000 7f9d20d9e740 20 client.16374 trim_cache size 0 max 16384 2024-07-26T13:43:24.787 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-07-26T13:43:24.787 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-07-26T13:43:24.787 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-07-26T13:43:24.787 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-07-26T13:43:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.785+0000 7f9d19dac640 1 -- 172.21.15.100:0/1216224321 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9d000b3d40 msgr2=0x7f9d000b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.785+0000 7f9d19dac640 1 --2- 172.21.15.100:0/1216224321 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9d000b3d40 0x7f9d000b6250 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9d040e1b10 tx=0x7f9d04025330 comp rx=0 tx=0).stop 2024-07-26T13:43:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.785+0000 7f9d19dac640 1 -- 172.21.15.100:0/1216224321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d14102400 msgr2=0x7f9d14199e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.785+0000 7f9d19dac640 1 --2- 172.21.15.100:0/1216224321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d14102400 0x7f9d14199e40 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f9d0400e9f0 tx=0x7f9d0400eeb0 comp rx=0 tx=0).stop 2024-07-26T13:43:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.785+0000 7f9d19dac640 1 -- 172.21.15.100:0/1216224321 shutdown_connections 2024-07-26T13:43:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.785+0000 7f9d19dac640 1 --2- 172.21.15.100:0/1216224321 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9d000b3d40 0x7f9d000b6250 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.785+0000 7f9d19dac640 1 --2- 172.21.15.100:0/1216224321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d14108680 0x7f9d1419f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.785+0000 7f9d19dac640 1 --2- 172.21.15.100:0/1216224321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d14107d30 0x7f9d1419a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.785+0000 7f9d19dac640 1 --2- 172.21.15.100:0/1216224321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9d14102400 0x7f9d14199e40 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.785+0000 7f9d19dac640 1 -- 172.21.15.100:0/1216224321 >> 172.21.15.100:0/1216224321 conn(0x7f9d140fde20 msgr2=0x7f9d14100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.785+0000 7f9d19dac640 1 -- 172.21.15.100:0/1216224321 shutdown_connections 2024-07-26T13:43:24.788 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:24.785+0000 7f9d19dac640 1 -- 172.21.15.100:0/1216224321 wait complete. 2024-07-26T13:43:24.995 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-26T13:43:24.995 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-26T13:43:25.261 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:43:25.408 INFO:tasks.workunit.client.0.smithi100.stdout: 1 103843 5.82 MB/sec warmup 99 sec latency 42.028 ms 2024-07-26T13:43:25.670 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:25 smithi100 ceph-mon[32599]: pgmap v464: 97 pgs: 97 active+clean; 51 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 307 op/s 2024-07-26T13:43:25.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:25 smithi167 ceph-mon[37140]: pgmap v464: 97 pgs: 97 active+clean; 51 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 307 op/s 2024-07-26T13:43:25.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:25 smithi138 ceph-mon[36954]: pgmap v464: 97 pgs: 97 active+clean; 51 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 307 op/s 2024-07-26T13:43:26.408 INFO:tasks.workunit.client.0.smithi100.stdout: 1 106397 5.93 MB/sec warmup 100 sec latency 24.861 ms 2024-07-26T13:43:27.408 INFO:tasks.workunit.client.0.smithi100.stdout: 1 107903 5.94 MB/sec warmup 101 sec latency 12.303 ms 2024-07-26T13:43:27.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.661+0000 7f0027718640 1 -- 172.21.15.100:0/2094037642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0020108720 msgr2=0x7f002010ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:27.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.661+0000 7f0027718640 1 --2- 172.21.15.100:0/2094037642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0020108720 0x7f002010ef50 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f0018009900 tx=0x7f001802a410 comp rx=0 tx=0).stop 2024-07-26T13:43:27.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.661+0000 7f0027718640 1 -- 172.21.15.100:0/2094037642 shutdown_connections 2024-07-26T13:43:27.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.661+0000 7f0027718640 1 --2- 172.21.15.100:0/2094037642 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0020108720 0x7f002010ef50 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:27.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.661+0000 7f0027718640 1 --2- 172.21.15.100:0/2094037642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0020107dd0 0x7f00201081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:27.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.661+0000 7f0027718640 1 --2- 172.21.15.100:0/2094037642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0020102490 0x7f00201028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:27.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.661+0000 7f0027718640 1 -- 172.21.15.100:0/2094037642 >> 172.21.15.100:0/2094037642 conn(0x7f0020079160 msgr2=0x7f0020079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:27.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.661+0000 7f0027718640 1 -- 172.21.15.100:0/2094037642 shutdown_connections 2024-07-26T13:43:27.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.661+0000 7f0027718640 1 -- 172.21.15.100:0/2094037642 wait complete. 2024-07-26T13:43:27.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.661+0000 7f0027718640 1 Processor -- start 2024-07-26T13:43:27.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.662+0000 7f0027718640 1 -- start start 2024-07-26T13:43:27.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.662+0000 7f0027718640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0020102490 0x7f0020198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:27.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.662+0000 7f0027718640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0020107dd0 0x7f00201993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:27.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.662+0000 7f0027718640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0020108720 0x7f002019e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:27.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.662+0000 7f0027718640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0020112380 con 0x7f0020102490 2024-07-26T13:43:27.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.662+0000 7f0027718640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0020112200 con 0x7f0020107dd0 2024-07-26T13:43:27.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.662+0000 7f0027718640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0020112500 con 0x7f0020108720 2024-07-26T13:43:27.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.662+0000 7f00254b4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0020102490 0x7f0020198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:27.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.662+0000 7f00254b4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0020102490 0x7f0020198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48238/0 (socket says 172.21.15.100:48238) 2024-07-26T13:43:27.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.662+0000 7f00254b4640 1 -- 172.21.15.100:0/1418900435 learned_addr learned my addr 172.21.15.100:0/1418900435 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:27.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.662+0000 7f0024cb3640 1 --2- 172.21.15.100:0/1418900435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0020107dd0 0x7f00201993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:27.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.662+0000 7f0025cb5640 1 --2- 172.21.15.100:0/1418900435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0020108720 0x7f002019e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:27.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.662+0000 7f00254b4640 1 -- 172.21.15.100:0/1418900435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0020108720 msgr2=0x7f002019e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:27.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.662+0000 7f00254b4640 1 --2- 172.21.15.100:0/1418900435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0020108720 0x7f002019e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:27.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.662+0000 7f00254b4640 1 -- 172.21.15.100:0/1418900435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0020107dd0 msgr2=0x7f00201993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:27.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.662+0000 7f00254b4640 1 --2- 172.21.15.100:0/1418900435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0020107dd0 0x7f00201993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:27.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.662+0000 7f00254b4640 1 -- 172.21.15.100:0/1418900435 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f002019ed80 con 0x7f0020102490 2024-07-26T13:43:27.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.662+0000 7f00254b4640 1 --2- 172.21.15.100:0/1418900435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0020102490 0x7f0020198e10 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f001000ebc0 tx=0x7f001000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:27.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.663+0000 7f001e7fc640 1 -- 172.21.15.100:0/1418900435 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0010010630 con 0x7f0020102490 2024-07-26T13:43:27.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.663+0000 7f001e7fc640 1 -- 172.21.15.100:0/1418900435 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0010010c80 con 0x7f0020102490 2024-07-26T13:43:27.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.663+0000 7f0027718640 1 -- 172.21.15.100:0/1418900435 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f002019f070 con 0x7f0020102490 2024-07-26T13:43:27.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.663+0000 7f001e7fc640 1 -- 172.21.15.100:0/1418900435 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0010018b60 con 0x7f0020102490 2024-07-26T13:43:27.668 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.663+0000 7f0027718640 1 -- 172.21.15.100:0/1418900435 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f002006a890 con 0x7f0020102490 2024-07-26T13:43:27.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.666+0000 7f001e7fc640 1 -- 172.21.15.100:0/1418900435 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f0010018d00 con 0x7f0020102490 2024-07-26T13:43:27.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.666+0000 7f002dc9c740 1 Processor -- start 2024-07-26T13:43:27.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.666+0000 7f002dc9c740 1 -- start start 2024-07-26T13:43:27.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.666+0000 7f002dc9c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558473b1350 0x5558477563b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:27.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.666+0000 7f002dc9c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555847388490 0x5558477568f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:27.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.666+0000 7f002dc9c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555847390670 0x555847756e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:27.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.666+0000 7f002dc9c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55584763e2f0 con 0x5558473b1350 2024-07-26T13:43:27.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.666+0000 7f002dc9c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5558476cca80 con 0x555847390670 2024-07-26T13:43:27.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.666+0000 7f002dc9c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5558476bbf80 con 0x555847388490 2024-07-26T13:43:27.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.666+0000 7f0025cb5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558473b1350 0x5558477563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:27.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.666+0000 7f00254b4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555847388490 0x5558477568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:27.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.666+0000 7f0025cb5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558473b1350 0x5558477563b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48242/0 (socket says 172.21.15.100:48242) 2024-07-26T13:43:27.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.666+0000 7f0025cb5640 1 -- 172.21.15.100:0/606459764 learned_addr learned my addr 172.21.15.100:0/606459764 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:27.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.667+0000 7f0024cb3640 1 --2- 172.21.15.100:0/606459764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555847390670 0x555847756e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:27.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.667+0000 7f001e7fc640 1 --2- 172.21.15.100:0/1418900435 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f00080b3dc0 0x7f00080b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:27.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.667+0000 7f00254b4640 1 -- 172.21.15.100:0/606459764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555847390670 msgr2=0x555847756e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:27.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.667+0000 7f00254b4640 1 --2- 172.21.15.100:0/606459764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555847390670 0x555847756e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:27.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.667+0000 7f00254b4640 1 -- 172.21.15.100:0/606459764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558473b1350 msgr2=0x5558477563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:27.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.667+0000 7f00254b4640 1 --2- 172.21.15.100:0/606459764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558473b1350 0x5558477563b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:27.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.667+0000 7f00254b4640 1 -- 172.21.15.100:0/606459764 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55584741c930 con 0x555847388490 2024-07-26T13:43:27.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.667+0000 7f0025cb5640 1 --2- 172.21.15.100:0/606459764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558473b1350 0x5558477563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:43:27.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.667+0000 7f0025cb5640 1 --2- 172.21.15.100:0/1418900435 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f00080b3dc0 0x7f00080b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:27.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.667+0000 7f001e7fc640 1 -- 172.21.15.100:0/1418900435 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f00100e4580 con 0x7f0020102490 2024-07-26T13:43:27.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.667+0000 7f00254b4640 1 --2- 172.21.15.100:0/606459764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555847388490 0x5558477568f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f00100e1ca0 tx=0x7f00100700e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aee461846bedbe63 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:27.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.667+0000 7f0025cb5640 1 --2- 172.21.15.100:0/1418900435 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f00080b3dc0 0x7f00080b62d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0018031440 tx=0x7f0018009260 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:27.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.667+0000 7f00077fe640 1 -- 172.21.15.100:0/606459764 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00100aafe0 con 0x555847388490 2024-07-26T13:43:27.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.667+0000 7f00077fe640 1 -- 172.21.15.100:0/606459764 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0010018420 con 0x555847388490 2024-07-26T13:43:27.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.668+0000 7f00077fe640 1 -- 172.21.15.100:0/606459764 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001006e4b0 con 0x555847388490 2024-07-26T13:43:27.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.668+0000 7f002dc9c740 1 -- 172.21.15.100:0/606459764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555847388490 msgr2=0x5558477568f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:27.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.668+0000 7f002dc9c740 1 --2- 172.21.15.100:0/606459764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555847388490 0x5558477568f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f00100e1ca0 tx=0x7f00100700e0 comp rx=0 tx=0).stop 2024-07-26T13:43:27.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.668+0000 7f002dc9c740 1 -- 172.21.15.100:0/606459764 shutdown_connections 2024-07-26T13:43:27.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.668+0000 7f002dc9c740 1 --2- 172.21.15.100:0/606459764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555847390670 0x555847756e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:27.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.668+0000 7f002dc9c740 1 --2- 172.21.15.100:0/606459764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555847388490 0x5558477568f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:27.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.668+0000 7f002dc9c740 1 --2- 172.21.15.100:0/606459764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558473b1350 0x5558477563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:27.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.668+0000 7f002dc9c740 1 -- 172.21.15.100:0/606459764 >> 172.21.15.100:0/606459764 conn(0x5558473a5740 msgr2=0x5558477542a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:27.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.669+0000 7f002dc9c740 1 -- 172.21.15.100:0/606459764 shutdown_connections 2024-07-26T13:43:27.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.669+0000 7f002dc9c740 1 -- 172.21.15.100:0/606459764 wait complete. 2024-07-26T13:43:27.674 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:27 smithi100 ceph-mon[32599]: pgmap v465: 97 pgs: 97 active+clean; 47 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 313 op/s 2024-07-26T13:43:27.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.669+0000 7f002dc9c740 1 -- start start 2024-07-26T13:43:27.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.669+0000 7f002dc9c740 1 Processor -- start 2024-07-26T13:43:27.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.670+0000 7f002dc9c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555847388490 0x555847766450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:27.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.670+0000 7f002dc9c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555847390670 0x5558477669b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:27.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.670+0000 7f002dc9c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558473b1350 0x555847772040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:27.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.670+0000 7f002dc9c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55584765e4f0 con 0x555847390670 2024-07-26T13:43:27.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.670+0000 7f002dc9c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5558476bffc0 con 0x5558473b1350 2024-07-26T13:43:27.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.670+0000 7f002dc9c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55584751f890 con 0x555847388490 2024-07-26T13:43:27.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.670+0000 7f002dc9c740 10 client.0 _ll_register_callbacks cb 0x555847668500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:43:27.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.670+0000 7f00254b4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555847388490 0x555847766450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:27.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.670+0000 7f00254b4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555847388490 0x555847766450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:48808/0 (socket says 172.21.15.100:48808) 2024-07-26T13:43:27.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.670+0000 7f00254b4640 1 -- 172.21.15.100:0/3334343368 learned_addr learned my addr 172.21.15.100:0/3334343368 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:27.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.670+0000 7f0025cb5640 1 --2- 172.21.15.100:0/3334343368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558473b1350 0x555847772040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:27.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.670+0000 7f0024cb3640 1 --2- 172.21.15.100:0/3334343368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555847390670 0x5558477669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:27.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.670+0000 7f00254b4640 1 -- 172.21.15.100:0/3334343368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558473b1350 msgr2=0x555847772040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:27.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.670+0000 7f00254b4640 1 --2- 172.21.15.100:0/3334343368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558473b1350 0x555847772040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:27.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.670+0000 7f00254b4640 1 -- 172.21.15.100:0/3334343368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555847390670 msgr2=0x5558477669b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:27.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.670+0000 7f00254b4640 1 --2- 172.21.15.100:0/3334343368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555847390670 0x5558477669b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:27.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.670+0000 7f00254b4640 1 -- 172.21.15.100:0/3334343368 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55584766dc50 con 0x555847388490 2024-07-26T13:43:27.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.670+0000 7f0024cb3640 1 --2- 172.21.15.100:0/3334343368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555847390670 0x5558477669b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:43:27.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.670+0000 7f00254b4640 1 --2- 172.21.15.100:0/3334343368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555847388490 0x555847766450 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f00100e22a0 tx=0x7f00100099b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=baf8c297577e9345 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:27.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.670+0000 7f0004ff9640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-07-26T13:43:27.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.671+0000 7f0004ff9640 1 -- 172.21.15.100:0/3334343368 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00100aa980 con 0x555847388490 2024-07-26T13:43:27.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.671+0000 7f0004ff9640 1 -- 172.21.15.100:0/3334343368 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f00100b3d80 con 0x555847388490 2024-07-26T13:43:27.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.671+0000 7f002dc9c740 1 -- 172.21.15.100:0/3334343368 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5558476ce9d0 con 0x555847388490 2024-07-26T13:43:27.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.671+0000 7f0004ff9640 1 -- 172.21.15.100:0/3334343368 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f00100aab60 con 0x555847388490 2024-07-26T13:43:27.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.671+0000 7f002dc9c740 10 client.25879 fetch_fsmap learned FSMap version 36 2024-07-26T13:43:27.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.671+0000 7f002dc9c740 1 -- 172.21.15.100:0/3334343368 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x555847774d90 con 0x555847388490 2024-07-26T13:43:27.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.671+0000 7f0004ff9640 1 -- 172.21.15.100:0/3334343368 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5558476ce9d0 con 0x555847388490 2024-07-26T13:43:27.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.671+0000 7f0004ff9640 1 -- 172.21.15.100:0/3334343368 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f00100c02d0 con 0x555847388490 2024-07-26T13:43:27.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.671+0000 7f002dc9c740 10 client.25879 fetch_fsmap finished waiting for FSMap version 36 2024-07-26T13:43:27.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.671+0000 7f002dc9c740 10 client.25879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:43:27.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.671+0000 7f002dc9c740 20 client.25879 populate_metadata read hostname 'smithi100' 2024-07-26T13:43:27.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.671+0000 7f002dc9c740 1 --2- 172.21.15.100:0/3334343368 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55584736fcd0 0x5558477780f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:27.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.671+0000 7f002dc9c740 4 client.25879 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:43:27.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.671+0000 7f002dc9c740 1 -- 172.21.15.100:0/3334343368 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558476ca470 con 0x55584736fcd0 2024-07-26T13:43:27.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.672+0000 7f0024cb3640 1 --2- 172.21.15.100:0/3334343368 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55584736fcd0 0x5558477780f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:27.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.672+0000 7f0004ff9640 10 client.25879 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:43:27.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.672+0000 7f0024cb3640 1 --2- 172.21.15.100:0/3334343368 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55584736fcd0 0x5558477780f0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e7857526379dcd9 server_cookie=8727510eafda2fde in_seq=0 out_seq=0 2024-07-26T13:43:27.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.673+0000 7f0004ff9640 1 -- 172.21.15.100:0/3334343368 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5558476ca470 con 0x55584736fcd0 2024-07-26T13:43:27.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.673+0000 7f0004ff9640 10 client.25879 handle_command_reply: tid=0 2024-07-26T13:43:27.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.673+0000 7f002dc9c740 1 client.25879 shutdown 2024-07-26T13:43:27.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.673+0000 7f002dc9c740 1 -- 172.21.15.100:0/3334343368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555847388490 msgr2=0x555847766450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:27.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.673+0000 7f002dc9c740 1 --2- 172.21.15.100:0/3334343368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555847388490 0x555847766450 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f00100e22a0 tx=0x7f00100099b0 comp rx=0 tx=0).stop 2024-07-26T13:43:27.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.673+0000 7f002dc9c740 1 -- 172.21.15.100:0/3334343368 shutdown_connections 2024-07-26T13:43:27.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.673+0000 7f002dc9c740 1 --2- 172.21.15.100:0/3334343368 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55584736fcd0 0x5558477780f0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:27.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.673+0000 7f002dc9c740 1 --2- 172.21.15.100:0/3334343368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558473b1350 0x555847772040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:27.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.673+0000 7f002dc9c740 1 --2- 172.21.15.100:0/3334343368 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555847390670 0x5558477669b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:27.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.673+0000 7f002dc9c740 1 --2- 172.21.15.100:0/3334343368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555847388490 0x555847766450 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:27.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.673+0000 7f002dc9c740 1 -- 172.21.15.100:0/3334343368 >> 172.21.15.100:0/3334343368 conn(0x5558473a5740 msgr2=0x555847757da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:27.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.673+0000 7f0004ff9640 0 client.25879 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:43:27.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.673+0000 7f002dc9c740 1 -- 172.21.15.100:0/3334343368 shutdown_connections 2024-07-26T13:43:27.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.673+0000 7f002dc9c740 1 -- 172.21.15.100:0/3334343368 wait complete. 2024-07-26T13:43:27.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.674+0000 7f002dc9c740 20 client.25879 trim_cache size 0 max 16384 2024-07-26T13:43:27.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.693+0000 7f002dc9c740 1 Processor -- start 2024-07-26T13:43:27.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.693+0000 7f002dc9c740 1 -- start start 2024-07-26T13:43:27.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.693+0000 7f002dc9c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555847388490 0x55584775bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:27.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.693+0000 7f002dc9c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555847390670 0x555847771fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:27.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.693+0000 7f002dc9c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558473b1350 0x5558477724f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:27.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.693+0000 7f002dc9c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f00201afa20 con 0x555847390670 2024-07-26T13:43:27.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.693+0000 7f002dc9c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f00201ad290 con 0x5558473b1350 2024-07-26T13:43:27.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.693+0000 7f002dc9c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f00201ab0e0 con 0x555847388490 2024-07-26T13:43:27.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.693+0000 7f0024cb3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555847390670 0x555847771fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:27.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.693+0000 7f0024cb3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555847390670 0x555847771fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48252/0 (socket says 172.21.15.100:48252) 2024-07-26T13:43:27.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.693+0000 7f0024cb3640 1 -- 172.21.15.100:0/3478271283 learned_addr learned my addr 172.21.15.100:0/3478271283 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:27.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.693+0000 7f00254b4640 1 --2- 172.21.15.100:0/3478271283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555847388490 0x55584775bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:27.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.693+0000 7f0025cb5640 1 --2- 172.21.15.100:0/3478271283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558473b1350 0x5558477724f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:27.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.694+0000 7f0024cb3640 1 -- 172.21.15.100:0/3478271283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555847388490 msgr2=0x55584775bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:27.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.694+0000 7f0024cb3640 1 --2- 172.21.15.100:0/3478271283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555847388490 0x55584775bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:27.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.694+0000 7f0024cb3640 1 -- 172.21.15.100:0/3478271283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558473b1350 msgr2=0x5558477724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:27.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.694+0000 7f0024cb3640 1 --2- 172.21.15.100:0/3478271283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558473b1350 0x5558477724f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:27.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.694+0000 7f0024cb3640 1 -- 172.21.15.100:0/3478271283 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558477651b0 con 0x555847390670 2024-07-26T13:43:27.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.694+0000 7f0024cb3640 1 --2- 172.21.15.100:0/3478271283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555847390670 0x555847771fb0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f002006a4d0 tx=0x7f0014005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a70c913637f35426 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:27.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.694+0000 7f0006ffd640 1 -- 172.21.15.100:0/3478271283 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0014004250 con 0x555847390670 2024-07-26T13:43:27.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.694+0000 7f0006ffd640 1 -- 172.21.15.100:0/3478271283 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00140043f0 con 0x555847390670 2024-07-26T13:43:27.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.694+0000 7f0006ffd640 1 -- 172.21.15.100:0/3478271283 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00140035f0 con 0x555847390670 2024-07-26T13:43:27.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.695+0000 7f002dc9c740 1 -- 172.21.15.100:0/3478271283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555847390670 msgr2=0x555847771fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:27.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.695+0000 7f002dc9c740 1 --2- 172.21.15.100:0/3478271283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555847390670 0x555847771fb0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f002006a4d0 tx=0x7f0014005f90 comp rx=0 tx=0).stop 2024-07-26T13:43:27.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.695+0000 7f002dc9c740 1 -- 172.21.15.100:0/3478271283 shutdown_connections 2024-07-26T13:43:27.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.695+0000 7f002dc9c740 1 --2- 172.21.15.100:0/3478271283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558473b1350 0x5558477724f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:27.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.695+0000 7f002dc9c740 1 --2- 172.21.15.100:0/3478271283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555847390670 0x555847771fb0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:27.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.695+0000 7f002dc9c740 1 --2- 172.21.15.100:0/3478271283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555847388490 0x55584775bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:27.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.695+0000 7f002dc9c740 1 -- 172.21.15.100:0/3478271283 >> 172.21.15.100:0/3478271283 conn(0x5558473a5740 msgr2=0x55584775b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:27.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.695+0000 7f002dc9c740 1 -- 172.21.15.100:0/3478271283 shutdown_connections 2024-07-26T13:43:27.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.695+0000 7f002dc9c740 1 -- 172.21.15.100:0/3478271283 wait complete. 2024-07-26T13:43:27.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.695+0000 7f002dc9c740 1 -- start start 2024-07-26T13:43:27.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.695+0000 7f002dc9c740 1 Processor -- start 2024-07-26T13:43:27.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.696+0000 7f002dc9c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555847388490 0x555847779120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:27.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.696+0000 7f002dc9c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555847390670 0x555847779660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:27.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.696+0000 7f002dc9c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558473b1350 0x555847779ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:27.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.696+0000 7f002dc9c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f00201ab750 con 0x555847388490 2024-07-26T13:43:27.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.696+0000 7f002dc9c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f00201acbc0 con 0x555847390670 2024-07-26T13:43:27.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.696+0000 7f002dc9c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f00201abe20 con 0x5558473b1350 2024-07-26T13:43:27.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.696+0000 7f002dc9c740 10 client.0 _ll_register_callbacks cb 0x7f00100a7160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:43:27.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.696+0000 7f00254b4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555847388490 0x555847779120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:27.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.696+0000 7f00254b4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555847388490 0x555847779120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48256/0 (socket says 172.21.15.100:48256) 2024-07-26T13:43:27.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.696+0000 7f00254b4640 1 -- 172.21.15.100:0/2445697764 learned_addr learned my addr 172.21.15.100:0/2445697764 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:27.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.696+0000 7f0025cb5640 1 --2- 172.21.15.100:0/2445697764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558473b1350 0x555847779ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:27.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.696+0000 7f0024cb3640 1 --2- 172.21.15.100:0/2445697764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555847390670 0x555847779660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:27.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.696+0000 7f00254b4640 1 -- 172.21.15.100:0/2445697764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558473b1350 msgr2=0x555847779ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:27.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.696+0000 7f00254b4640 1 --2- 172.21.15.100:0/2445697764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558473b1350 0x555847779ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:27.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.697+0000 7f00254b4640 1 -- 172.21.15.100:0/2445697764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555847390670 msgr2=0x555847779660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:27.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.697+0000 7f00254b4640 1 --2- 172.21.15.100:0/2445697764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555847390670 0x555847779660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:27.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.697+0000 7f00254b4640 1 -- 172.21.15.100:0/2445697764 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558477df770 con 0x555847388490 2024-07-26T13:43:27.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.697+0000 7f00254b4640 1 --2- 172.21.15.100:0/2445697764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555847388490 0x555847779120 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f00100b43d0 tx=0x7f00100e4880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae94d1039dfbf5c7 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:27.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.697+0000 7f0004ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:43:27.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.697+0000 7f0004ff9640 1 -- 172.21.15.100:0/2445697764 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00100aa980 con 0x555847388490 2024-07-26T13:43:27.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.697+0000 7f0004ff9640 1 -- 172.21.15.100:0/2445697764 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f00100107d0 con 0x555847388490 2024-07-26T13:43:27.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.697+0000 7f002dc9c740 1 -- 172.21.15.100:0/2445697764 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00140043f0 con 0x555847388490 2024-07-26T13:43:27.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.697+0000 7f0004ff9640 1 -- 172.21.15.100:0/2445697764 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f00100aafe0 con 0x555847388490 2024-07-26T13:43:27.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.697+0000 7f002dc9c740 10 client.16392 fetch_fsmap learned FSMap version 36 2024-07-26T13:43:27.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.697+0000 7f002dc9c740 1 -- 172.21.15.100:0/2445697764 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5558477e0a10 con 0x555847388490 2024-07-26T13:43:27.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.697+0000 7f0004ff9640 1 -- 172.21.15.100:0/2445697764 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00140043f0 con 0x555847388490 2024-07-26T13:43:27.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.697+0000 7f0004ff9640 1 -- 172.21.15.100:0/2445697764 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f00100aa470 con 0x555847388490 2024-07-26T13:43:27.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.698+0000 7f002dc9c740 10 client.16392 fetch_fsmap finished waiting for FSMap version 36 2024-07-26T13:43:27.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.698+0000 7f002dc9c740 10 client.16392 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:43:27.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.698+0000 7f002dc9c740 20 client.16392 populate_metadata read hostname 'smithi100' 2024-07-26T13:43:27.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.698+0000 7f002dc9c740 1 --2- 172.21.15.100:0/2445697764 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555847345080 0x5558477e3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:27.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.698+0000 7f002dc9c740 4 client.16392 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-26T13:43:27.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.698+0000 7f002dc9c740 1 -- 172.21.15.100:0/2445697764 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5558476c7bb0 con 0x555847345080 2024-07-26T13:43:27.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.698+0000 7f0024cb3640 1 --2- 172.21.15.100:0/2445697764 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555847345080 0x5558477e3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:27.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.698+0000 7f0004ff9640 10 client.16392 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:43:27.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.698+0000 7f0024cb3640 1 --2- 172.21.15.100:0/2445697764 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555847345080 0x5558477e3060 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85dc7ca185850009 server_cookie=94c93dc37cc9492c in_seq=0 out_seq=0 2024-07-26T13:43:27.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.699+0000 7f0004ff9640 1 -- 172.21.15.100:0/2445697764 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5558476c7bb0 con 0x555847345080 2024-07-26T13:43:27.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.699+0000 7f0004ff9640 10 client.16392 handle_command_reply: tid=0 2024-07-26T13:43:27.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.699+0000 7f002dc9c740 1 client.16392 shutdown 2024-07-26T13:43:27.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.699+0000 7f002dc9c740 1 -- 172.21.15.100:0/2445697764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555847388490 msgr2=0x555847779120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:27.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.699+0000 7f002dc9c740 1 --2- 172.21.15.100:0/2445697764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555847388490 0x555847779120 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f00100b43d0 tx=0x7f00100e4880 comp rx=0 tx=0).stop 2024-07-26T13:43:27.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.699+0000 7f002dc9c740 1 -- 172.21.15.100:0/2445697764 shutdown_connections 2024-07-26T13:43:27.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.699+0000 7f002dc9c740 1 --2- 172.21.15.100:0/2445697764 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555847345080 0x5558477e3060 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:27.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.699+0000 7f002dc9c740 1 --2- 172.21.15.100:0/2445697764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558473b1350 0x555847779ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:27.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.699+0000 7f0004ff9640 0 client.16392 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:43:27.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.699+0000 7f002dc9c740 1 --2- 172.21.15.100:0/2445697764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555847390670 0x555847779660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:27.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.699+0000 7f002dc9c740 1 --2- 172.21.15.100:0/2445697764 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555847388490 0x555847779120 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:27.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.699+0000 7f002dc9c740 1 -- 172.21.15.100:0/2445697764 >> 172.21.15.100:0/2445697764 conn(0x5558473a5740 msgr2=0x5558477573c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:27.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.699+0000 7f002dc9c740 1 -- 172.21.15.100:0/2445697764 shutdown_connections 2024-07-26T13:43:27.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.699+0000 7f002dc9c740 1 -- 172.21.15.100:0/2445697764 wait complete. 2024-07-26T13:43:27.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.700+0000 7f002dc9c740 20 client.16392 trim_cache size 0 max 16384 2024-07-26T13:43:27.706 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:43:27.706 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-07-26T13:43:27.706 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "181df3ac-9c51-4cdd-89b7-d49758149d30", 2024-07-26T13:43:27.706 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-07-26T13:43:27.706 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:43:27.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.700+0000 7f0027718640 1 -- 172.21.15.100:0/1418900435 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f00080b3dc0 msgr2=0x7f00080b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:27.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.700+0000 7f0027718640 1 --2- 172.21.15.100:0/1418900435 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f00080b3dc0 0x7f00080b62d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0018031440 tx=0x7f0018009260 comp rx=0 tx=0).stop 2024-07-26T13:43:27.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.700+0000 7f0027718640 1 -- 172.21.15.100:0/1418900435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0020102490 msgr2=0x7f0020198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:27.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.700+0000 7f0027718640 1 --2- 172.21.15.100:0/1418900435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0020102490 0x7f0020198e10 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f001000ebc0 tx=0x7f001000c6a0 comp rx=0 tx=0).stop 2024-07-26T13:43:27.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.700+0000 7f0027718640 1 -- 172.21.15.100:0/1418900435 shutdown_connections 2024-07-26T13:43:27.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.700+0000 7f0027718640 1 --2- 172.21.15.100:0/1418900435 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f00080b3dc0 0x7f00080b62d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:27.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.700+0000 7f0027718640 1 --2- 172.21.15.100:0/1418900435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0020108720 0x7f002019e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:27.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.700+0000 7f0027718640 1 --2- 172.21.15.100:0/1418900435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0020107dd0 0x7f00201993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:27.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.700+0000 7f0027718640 1 --2- 172.21.15.100:0/1418900435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0020102490 0x7f0020198e10 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:27.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.700+0000 7f0027718640 1 -- 172.21.15.100:0/1418900435 >> 172.21.15.100:0/1418900435 conn(0x7f0020079160 msgr2=0x7f0020103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:27.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.700+0000 7f0027718640 1 -- 172.21.15.100:0/1418900435 shutdown_connections 2024-07-26T13:43:27.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:27.700+0000 7f0027718640 1 -- 172.21.15.100:0/1418900435 wait complete. 2024-07-26T13:43:27.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:27 smithi138 ceph-mon[36954]: pgmap v465: 97 pgs: 97 active+clean; 47 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 313 op/s 2024-07-26T13:43:28.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:27 smithi167 ceph-mon[37140]: pgmap v465: 97 pgs: 97 active+clean; 47 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 313 op/s 2024-07-26T13:43:28.408 INFO:tasks.workunit.client.0.smithi100.stdout: 1 110315 6.01 MB/sec warmup 102 sec latency 11.178 ms 2024-07-26T13:43:28.510 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-26T13:43:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:28 smithi100 ceph-mon[32599]: scrub queued for path: / 2024-07-26T13:43:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:28 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:43:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:28 smithi100 ceph-mon[32599]: scrub summary: active paths [/] 2024-07-26T13:43:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:28 smithi100 ceph-mon[32599]: scrub summary: active paths [/] 2024-07-26T13:43:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:28 smithi100 ceph-mon[32599]: scrub summary: active paths [/] 2024-07-26T13:43:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:28 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:43:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:28 smithi100 ceph-mon[32599]: scrub summary: active paths [/] 2024-07-26T13:43:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:28 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:43:28.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:28 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:43:28.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:28 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:43:28.780 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:43:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:28 smithi138 ceph-mon[36954]: scrub queued for path: / 2024-07-26T13:43:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:28 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:43:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:28 smithi138 ceph-mon[36954]: scrub summary: active paths [/] 2024-07-26T13:43:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:28 smithi138 ceph-mon[36954]: scrub summary: active paths [/] 2024-07-26T13:43:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:28 smithi138 ceph-mon[36954]: scrub summary: active paths [/] 2024-07-26T13:43:28.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:28 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:43:28.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:28 smithi138 ceph-mon[36954]: scrub summary: active paths [/] 2024-07-26T13:43:28.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:28 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:43:28.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:28 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:43:28.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:28 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:43:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:28 smithi167 ceph-mon[37140]: scrub queued for path: / 2024-07-26T13:43:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:28 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:43:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:28 smithi167 ceph-mon[37140]: scrub summary: active paths [/] 2024-07-26T13:43:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:28 smithi167 ceph-mon[37140]: scrub summary: active paths [/] 2024-07-26T13:43:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:28 smithi167 ceph-mon[37140]: scrub summary: active paths [/] 2024-07-26T13:43:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:28 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:43:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:28 smithi167 ceph-mon[37140]: scrub summary: active paths [/] 2024-07-26T13:43:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:28 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:43:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:28 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:43:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:28 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:43:29.408 INFO:tasks.workunit.client.0.smithi100.stdout: 1 111764 6.01 MB/sec warmup 103 sec latency 181.924 ms 2024-07-26T13:43:29.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:29 smithi138 ceph-mon[36954]: pgmap v466: 97 pgs: 97 active+clean; 48 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 331 op/s 2024-07-26T13:43:29.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:29 smithi100 ceph-mon[32599]: pgmap v466: 97 pgs: 97 active+clean; 48 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 331 op/s 2024-07-26T13:43:30.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:29 smithi167 ceph-mon[37140]: pgmap v466: 97 pgs: 97 active+clean; 48 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 331 op/s 2024-07-26T13:43:30.409 INFO:tasks.workunit.client.0.smithi100.stdout: 1 113916 6.09 MB/sec warmup 104 sec latency 45.356 ms 2024-07-26T13:43:30.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.547+0000 7f2fdfebe640 1 -- 172.21.15.100:0/3237244831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fd8107dd0 msgr2=0x7f2fd81081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:30.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.547+0000 7f2fdfebe640 1 --2- 172.21.15.100:0/3237244831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fd8107dd0 0x7f2fd81081e0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc009900 tx=0x7f2fcc02a410 comp rx=0 tx=0).stop 2024-07-26T13:43:30.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.547+0000 7f2fdfebe640 1 -- 172.21.15.100:0/3237244831 shutdown_connections 2024-07-26T13:43:30.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.547+0000 7f2fdfebe640 1 --2- 172.21.15.100:0/3237244831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fd8108720 0x7f2fd810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:30.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.547+0000 7f2fdfebe640 1 --2- 172.21.15.100:0/3237244831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fd8107dd0 0x7f2fd81081e0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:30.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.547+0000 7f2fdfebe640 1 --2- 172.21.15.100:0/3237244831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fd8102490 0x7f2fd81028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:30.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.547+0000 7f2fdfebe640 1 -- 172.21.15.100:0/3237244831 >> 172.21.15.100:0/3237244831 conn(0x7f2fd8079160 msgr2=0x7f2fd8079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:30.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.547+0000 7f2fdfebe640 1 -- 172.21.15.100:0/3237244831 shutdown_connections 2024-07-26T13:43:30.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.547+0000 7f2fdfebe640 1 -- 172.21.15.100:0/3237244831 wait complete. 2024-07-26T13:43:30.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.548+0000 7f2fdfebe640 1 Processor -- start 2024-07-26T13:43:30.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.548+0000 7f2fdfebe640 1 -- start start 2024-07-26T13:43:30.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.549+0000 7f2fdfebe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fd8102490 0x7f2fd8198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:30.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.549+0000 7f2fdfebe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fd8107dd0 0x7f2fd8199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:30.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.549+0000 7f2fdfebe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fd8108720 0x7f2fd819e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:30.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.549+0000 7f2fdfebe640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd8112370 con 0x7f2fd8102490 2024-07-26T13:43:30.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.549+0000 7f2fdfebe640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd81121f0 con 0x7f2fd8107dd0 2024-07-26T13:43:30.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.549+0000 7f2fdfebe640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd81124f0 con 0x7f2fd8108720 2024-07-26T13:43:30.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.549+0000 7f2fddc5a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fd8102490 0x7f2fd8198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:30.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.549+0000 7f2fddc5a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fd8102490 0x7f2fd8198d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48272/0 (socket says 172.21.15.100:48272) 2024-07-26T13:43:30.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.549+0000 7f2fddc5a640 1 -- 172.21.15.100:0/564194712 learned_addr learned my addr 172.21.15.100:0/564194712 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:30.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.549+0000 7f2fdd459640 1 --2- 172.21.15.100:0/564194712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fd8107dd0 0x7f2fd8199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:30.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.549+0000 7f2fde45b640 1 --2- 172.21.15.100:0/564194712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fd8108720 0x7f2fd819e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:30.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.549+0000 7f2fddc5a640 1 -- 172.21.15.100:0/564194712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fd8108720 msgr2=0x7f2fd819e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:30.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.549+0000 7f2fddc5a640 1 --2- 172.21.15.100:0/564194712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fd8108720 0x7f2fd819e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:30.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.549+0000 7f2fddc5a640 1 -- 172.21.15.100:0/564194712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fd8107dd0 msgr2=0x7f2fd8199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:30.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.549+0000 7f2fddc5a640 1 --2- 172.21.15.100:0/564194712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fd8107dd0 0x7f2fd8199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:30.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.549+0000 7f2fddc5a640 1 -- 172.21.15.100:0/564194712 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd819ecf0 con 0x7f2fd8102490 2024-07-26T13:43:30.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.549+0000 7f2fddc5a640 1 --2- 172.21.15.100:0/564194712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fd8102490 0x7f2fd8198d80 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f2fd000e9f0 tx=0x7f2fd000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:30.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.549+0000 7f2fd6ffd640 1 -- 172.21.15.100:0/564194712 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd000cd80 con 0x7f2fd8102490 2024-07-26T13:43:30.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.549+0000 7f2fdfebe640 1 -- 172.21.15.100:0/564194712 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fd819efe0 con 0x7f2fd8102490 2024-07-26T13:43:30.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.549+0000 7f2fd6ffd640 1 -- 172.21.15.100:0/564194712 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fd00108e0 con 0x7f2fd8102490 2024-07-26T13:43:30.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.549+0000 7f2fd6ffd640 1 -- 172.21.15.100:0/564194712 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0018980 con 0x7f2fd8102490 2024-07-26T13:43:30.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.549+0000 7f2fdfebe640 1 -- 172.21.15.100:0/564194712 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fd81a5740 con 0x7f2fd8102490 2024-07-26T13:43:30.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.552+0000 7f2fe6442740 1 Processor -- start 2024-07-26T13:43:30.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.552+0000 7f2fd6ffd640 1 -- 172.21.15.100:0/564194712 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f2fd0010430 con 0x7f2fd8102490 2024-07-26T13:43:30.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.552+0000 7f2fe6442740 1 -- start start 2024-07-26T13:43:30.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.553+0000 7f2fe6442740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbca9d070 0x559cbce42520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:30.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.553+0000 7f2fe6442740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cbca741b0 0x559cbce42a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:30.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.553+0000 7f2fe6442740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cbca7c390 0x559cbce42fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:30.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.553+0000 7f2fe6442740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x559cbcdb8b60 con 0x559cbca741b0 2024-07-26T13:43:30.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.553+0000 7f2fe6442740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559cbcda8060 con 0x559cbca9d070 2024-07-26T13:43:30.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.553+0000 7f2fe6442740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559cbcdac0a0 con 0x559cbca7c390 2024-07-26T13:43:30.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.553+0000 7f2fddc5a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cbca741b0 0x559cbce42a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:30.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.553+0000 7f2fde45b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbca9d070 0x559cbce42520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:30.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.553+0000 7f2fddc5a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cbca741b0 0x559cbce42a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48276/0 (socket says 172.21.15.100:48276) 2024-07-26T13:43:30.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.553+0000 7f2fddc5a640 1 -- 172.21.15.100:0/3608172684 learned_addr learned my addr 172.21.15.100:0/3608172684 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:30.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.553+0000 7f2fdd459640 1 --2- 172.21.15.100:0/3608172684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cbca7c390 0x559cbce42fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:30.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.553+0000 7f2fde45b640 1 -- 172.21.15.100:0/3608172684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cbca7c390 msgr2=0x559cbce42fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:30.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.553+0000 7f2fde45b640 1 --2- 172.21.15.100:0/3608172684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cbca7c390 0x559cbce42fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:30.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.553+0000 7f2fde45b640 1 -- 172.21.15.100:0/3608172684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cbca741b0 msgr2=0x559cbce42a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:30.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.553+0000 7f2fde45b640 1 --2- 172.21.15.100:0/3608172684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cbca741b0 0x559cbce42a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:30.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.553+0000 7f2fde45b640 1 -- 172.21.15.100:0/3608172684 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cbcd4eb40 con 0x559cbca9d070 2024-07-26T13:43:30.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.553+0000 7f2fd6ffd640 1 --2- 172.21.15.100:0/564194712 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f2fc00b5f50 0x7f2fc00b8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:30.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.553+0000 7f2fddc5a640 1 --2- 172.21.15.100:0/3608172684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cbca741b0 0x559cbce42a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:43:30.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.553+0000 7f2fddc5a640 1 --2- 172.21.15.100:0/564194712 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f2fc00b5f50 0x7f2fc00b8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:30.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.553+0000 7f2fde45b640 1 --2- 172.21.15.100:0/3608172684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbca9d070 0x559cbce42520 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009ed0 tx=0x7f2fc80107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8b1859b3ab302af server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:30.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.554+0000 7f2fd6ffd640 1 -- 172.21.15.100:0/564194712 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f2fd00e43f0 con 0x7f2fd8102490 2024-07-26T13:43:30.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.554+0000 7f2fbffff640 1 -- 172.21.15.100:0/3608172684 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc8003b70 con 0x559cbca9d070 2024-07-26T13:43:30.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.554+0000 7f2fbffff640 1 -- 172.21.15.100:0/3608172684 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fc8015070 con 0x559cbca9d070 2024-07-26T13:43:30.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.554+0000 7f2fbffff640 1 -- 172.21.15.100:0/3608172684 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc8011770 con 0x559cbca9d070 2024-07-26T13:43:30.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.555+0000 7f2fe6442740 1 -- 172.21.15.100:0/3608172684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbca9d070 msgr2=0x559cbce42520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:30.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.555+0000 7f2fe6442740 1 --2- 172.21.15.100:0/3608172684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbca9d070 0x559cbce42520 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009ed0 tx=0x7f2fc80107a0 comp rx=0 tx=0).stop 2024-07-26T13:43:30.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.555+0000 7f2fddc5a640 1 --2- 172.21.15.100:0/564194712 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f2fc00b5f50 0x7f2fc00b8460 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2fd00aacd0 tx=0x7f2fd00e7f40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:30.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.555+0000 7f2fe6442740 1 -- 172.21.15.100:0/3608172684 shutdown_connections 2024-07-26T13:43:30.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.555+0000 7f2fe6442740 1 --2- 172.21.15.100:0/3608172684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cbca7c390 0x559cbce42fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:30.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.555+0000 7f2fe6442740 1 --2- 172.21.15.100:0/3608172684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cbca741b0 0x559cbce42a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:30.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.555+0000 7f2fe6442740 1 --2- 172.21.15.100:0/3608172684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbca9d070 0x559cbce42520 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:30.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.555+0000 7f2fe6442740 1 -- 172.21.15.100:0/3608172684 >> 172.21.15.100:0/3608172684 conn(0x559cbca91460 msgr2=0x559cbce40410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:30.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.555+0000 7f2fe6442740 1 -- 172.21.15.100:0/3608172684 shutdown_connections 2024-07-26T13:43:30.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.555+0000 7f2fe6442740 1 -- 172.21.15.100:0/3608172684 wait complete. 2024-07-26T13:43:30.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.555+0000 7f2fe6442740 1 -- start start 2024-07-26T13:43:30.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.555+0000 7f2fe6442740 1 Processor -- start 2024-07-26T13:43:30.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.556+0000 7f2fe6442740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cbca741b0 0x559cbce4e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:30.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.556+0000 7f2fe6442740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cbca7c390 0x559cbce4e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:30.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.556+0000 7f2fe6442740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbca9d070 0x559cbce59ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:30.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.556+0000 7f2fe6442740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x559cbcc0b700 con 0x559cbca7c390 2024-07-26T13:43:30.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.556+0000 7f2fe6442740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559cbcd4a990 con 0x559cbca9d070 2024-07-26T13:43:30.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.556+0000 7f2fe6442740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559cbce50d00 con 0x559cbca741b0 2024-07-26T13:43:30.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.556+0000 7f2fe6442740 10 client.0 _ll_register_callbacks cb 0x559cbcd654a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:43:30.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.556+0000 7f2fdd459640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cbca7c390 0x559cbce4e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:30.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.556+0000 7f2fde45b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cbca741b0 0x559cbce4e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:30.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.556+0000 7f2fdd459640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cbca7c390 0x559cbce4e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48278/0 (socket says 172.21.15.100:48278) 2024-07-26T13:43:30.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.556+0000 7f2fdd459640 1 -- 172.21.15.100:0/4118717241 learned_addr learned my addr 172.21.15.100:0/4118717241 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:30.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.556+0000 7f2fde45b640 1 --2- 172.21.15.100:0/4118717241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbca9d070 0x559cbce59ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:30.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.556+0000 7f2fdd459640 1 -- 172.21.15.100:0/4118717241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cbca741b0 msgr2=0x559cbce4e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:30.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.556+0000 7f2fdd459640 1 --2- 172.21.15.100:0/4118717241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cbca741b0 0x559cbce4e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:30.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.556+0000 7f2fdd459640 1 -- 172.21.15.100:0/4118717241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbca9d070 msgr2=0x559cbce59ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:30.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.556+0000 7f2fdd459640 1 --2- 172.21.15.100:0/4118717241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbca9d070 0x559cbce59ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:30.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.556+0000 7f2fdd459640 1 -- 172.21.15.100:0/4118717241 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cbcb08650 con 0x559cbca7c390 2024-07-26T13:43:30.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.556+0000 7f2fdd459640 1 --2- 172.21.15.100:0/4118717241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cbca7c390 0x559cbce4e700 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f2fd806a4d0 tx=0x7f2fcc032eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4cfb3122caf5392 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:30.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.556+0000 7f2fbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:43:30.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.556+0000 7f2fbd7fa640 1 -- 172.21.15.100:0/4118717241 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc035040 con 0x559cbca7c390 2024-07-26T13:43:30.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.556+0000 7f2fe6442740 1 -- 172.21.15.100:0/4118717241 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fc8015070 con 0x559cbca7c390 2024-07-26T13:43:30.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.556+0000 7f2fbd7fa640 1 -- 172.21.15.100:0/4118717241 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f2fcc0317e0 con 0x559cbca7c390 2024-07-26T13:43:30.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.557+0000 7f2fbd7fa640 1 -- 172.21.15.100:0/4118717241 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f2fcc004040 con 0x559cbca7c390 2024-07-26T13:43:30.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.557+0000 7f2fe6442740 10 client.16404 fetch_fsmap learned FSMap version 36 2024-07-26T13:43:30.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.557+0000 7f2fe6442740 1 -- 172.21.15.100:0/4118717241 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x559cbce5cd70 con 0x559cbca7c390 2024-07-26T13:43:30.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.557+0000 7f2fbd7fa640 1 -- 172.21.15.100:0/4118717241 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fc8015070 con 0x559cbca7c390 2024-07-26T13:43:30.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.557+0000 7f2fbd7fa640 1 -- 172.21.15.100:0/4118717241 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f2fcc04d070 con 0x559cbca7c390 2024-07-26T13:43:30.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.557+0000 7f2fe6442740 10 client.16404 fetch_fsmap finished waiting for FSMap version 36 2024-07-26T13:43:30.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.557+0000 7f2fe6442740 10 client.16404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:43:30.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.557+0000 7f2fe6442740 20 client.16404 populate_metadata read hostname 'smithi100' 2024-07-26T13:43:30.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.557+0000 7f2fe6442740 1 --2- 172.21.15.100:0/4118717241 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559cbca5b9f0 0x559cbce60260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:30.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.557+0000 7f2fe6442740 4 client.16404 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:43:30.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.557+0000 7f2fe6442740 1 -- 172.21.15.100:0/4118717241 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559cbcdbaab0 con 0x559cbca5b9f0 2024-07-26T13:43:30.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.557+0000 7f2fde45b640 1 --2- 172.21.15.100:0/4118717241 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559cbca5b9f0 0x559cbce60260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:30.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.558+0000 7f2fbd7fa640 10 client.16404 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:43:30.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.558+0000 7f2fde45b640 1 --2- 172.21.15.100:0/4118717241 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559cbca5b9f0 0x559cbce60260 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c09f3db4aa9e89bf server_cookie=4dc480a388a23021 in_seq=0 out_seq=0 2024-07-26T13:43:30.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.558+0000 7f2fbd7fa640 1 -- 172.21.15.100:0/4118717241 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559cbcdbaab0 con 0x559cbca5b9f0 2024-07-26T13:43:30.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.558+0000 7f2fbd7fa640 10 client.16404 handle_command_reply: tid=0 2024-07-26T13:43:30.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.559+0000 7f2fe6442740 1 client.16404 shutdown 2024-07-26T13:43:30.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.559+0000 7f2fe6442740 1 -- 172.21.15.100:0/4118717241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cbca7c390 msgr2=0x559cbce4e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:30.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.559+0000 7f2fe6442740 1 --2- 172.21.15.100:0/4118717241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cbca7c390 0x559cbce4e700 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f2fd806a4d0 tx=0x7f2fcc032eb0 comp rx=0 tx=0).stop 2024-07-26T13:43:30.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.559+0000 7f2fe6442740 1 -- 172.21.15.100:0/4118717241 shutdown_connections 2024-07-26T13:43:30.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.559+0000 7f2fe6442740 1 --2- 172.21.15.100:0/4118717241 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559cbca5b9f0 0x559cbce60260 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:30.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.559+0000 7f2fe6442740 1 --2- 172.21.15.100:0/4118717241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbca9d070 0x559cbce59ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:30.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.559+0000 7f2fe6442740 1 --2- 172.21.15.100:0/4118717241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cbca7c390 0x559cbce4e700 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:30.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.559+0000 7f2fbd7fa640 0 client.16404 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:43:30.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.559+0000 7f2fe6442740 1 --2- 172.21.15.100:0/4118717241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cbca741b0 0x559cbce4e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:30.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.559+0000 7f2fe6442740 1 -- 172.21.15.100:0/4118717241 >> 172.21.15.100:0/4118717241 conn(0x559cbca91460 msgr2=0x559cbce3ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:30.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.559+0000 7f2fe6442740 1 -- 172.21.15.100:0/4118717241 shutdown_connections 2024-07-26T13:43:30.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.559+0000 7f2fe6442740 1 -- 172.21.15.100:0/4118717241 wait complete. 2024-07-26T13:43:30.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.559+0000 7f2fe6442740 20 client.16404 trim_cache size 0 max 16384 2024-07-26T13:43:30.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.577+0000 7f2fe6442740 1 Processor -- start 2024-07-26T13:43:30.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.577+0000 7f2fe6442740 1 -- start start 2024-07-26T13:43:30.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.577+0000 7f2fe6442740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cbca5b9f0 0x559cbce43c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:30.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.577+0000 7f2fe6442740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cbca741b0 0x559cbce477a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:30.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.577+0000 7f2fe6442740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbca7c390 0x559cbce47ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:30.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.577+0000 7f2fe6442740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd81af7c0 con 0x559cbca5b9f0 2024-07-26T13:43:30.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.577+0000 7f2fe6442740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd81ad030 con 0x559cbca7c390 2024-07-26T13:43:30.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.577+0000 7f2fe6442740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd81aae80 con 0x559cbca741b0 2024-07-26T13:43:30.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.577+0000 7f2fde45b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cbca5b9f0 0x559cbce43c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:30.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.577+0000 7f2fde45b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cbca5b9f0 0x559cbce43c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48284/0 (socket says 172.21.15.100:48284) 2024-07-26T13:43:30.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.577+0000 7f2fde45b640 1 -- 172.21.15.100:0/3397240728 learned_addr learned my addr 172.21.15.100:0/3397240728 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:30.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.577+0000 7f2fdd459640 1 --2- 172.21.15.100:0/3397240728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cbca741b0 0x559cbce477a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:30.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.577+0000 7f2fde45b640 1 --2- 172.21.15.100:0/3397240728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbca7c390 0x559cbce47ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:30.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.577+0000 7f2fde45b640 1 -- 172.21.15.100:0/3397240728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cbca741b0 msgr2=0x559cbce477a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:30.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.577+0000 7f2fde45b640 1 --2- 172.21.15.100:0/3397240728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cbca741b0 0x559cbce477a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:30.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.577+0000 7f2fde45b640 1 -- 172.21.15.100:0/3397240728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbca7c390 msgr2=0x559cbce47ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:30.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.577+0000 7f2fde45b640 1 --2- 172.21.15.100:0/3397240728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbca7c390 0x559cbce47ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:30.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.577+0000 7f2fde45b640 1 -- 172.21.15.100:0/3397240728 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cbce4f280 con 0x559cbca5b9f0 2024-07-26T13:43:30.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.578+0000 7f2fde45b640 1 --2- 172.21.15.100:0/3397240728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cbca5b9f0 0x559cbce43c60 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8004870 tx=0x7f2fc8010fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1ddecee1a679e25 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:30.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.578+0000 7f2fbf7fe640 1 -- 172.21.15.100:0/3397240728 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc8025040 con 0x559cbca5b9f0 2024-07-26T13:43:30.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.578+0000 7f2fbf7fe640 1 -- 172.21.15.100:0/3397240728 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fc8015030 con 0x559cbca5b9f0 2024-07-26T13:43:30.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.578+0000 7f2fbf7fe640 1 -- 172.21.15.100:0/3397240728 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc80052b0 con 0x559cbca5b9f0 2024-07-26T13:43:30.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.579+0000 7f2fe6442740 1 -- 172.21.15.100:0/3397240728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cbca5b9f0 msgr2=0x559cbce43c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:30.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.579+0000 7f2fe6442740 1 --2- 172.21.15.100:0/3397240728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cbca5b9f0 0x559cbce43c60 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8004870 tx=0x7f2fc8010fd0 comp rx=0 tx=0).stop 2024-07-26T13:43:30.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.579+0000 7f2fe6442740 1 -- 172.21.15.100:0/3397240728 shutdown_connections 2024-07-26T13:43:30.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.579+0000 7f2fe6442740 1 --2- 172.21.15.100:0/3397240728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbca7c390 0x559cbce47ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:30.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.579+0000 7f2fe6442740 1 --2- 172.21.15.100:0/3397240728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cbca741b0 0x559cbce477a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:30.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.579+0000 7f2fe6442740 1 --2- 172.21.15.100:0/3397240728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cbca5b9f0 0x559cbce43c60 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:30.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.579+0000 7f2fe6442740 1 -- 172.21.15.100:0/3397240728 >> 172.21.15.100:0/3397240728 conn(0x559cbca91460 msgr2=0x559cbce43550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:30.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.579+0000 7f2fe6442740 1 -- 172.21.15.100:0/3397240728 shutdown_connections 2024-07-26T13:43:30.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.579+0000 7f2fe6442740 1 -- 172.21.15.100:0/3397240728 wait complete. 2024-07-26T13:43:30.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.579+0000 7f2fe6442740 1 -- start start 2024-07-26T13:43:30.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.579+0000 7f2fe6442740 1 Processor -- start 2024-07-26T13:43:30.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.580+0000 7f2fe6442740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cbca5b9f0 0x559cbce56630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:30.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.580+0000 7f2fe6442740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbca741b0 0x559cbce598c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:30.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.580+0000 7f2fe6442740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cbca7c390 0x559cbce59e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:30.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.580+0000 7f2fe6442740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd81ab4f0 con 0x559cbca5b9f0 2024-07-26T13:43:30.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.580+0000 7f2fe6442740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd81ac960 con 0x559cbca741b0 2024-07-26T13:43:30.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.580+0000 7f2fe6442740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd81abbc0 con 0x559cbca7c390 2024-07-26T13:43:30.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.580+0000 7f2fe6442740 10 client.0 _ll_register_callbacks cb 0x7f2fcc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:43:30.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.580+0000 7f2fde45b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cbca5b9f0 0x559cbce56630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:30.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.580+0000 7f2fde45b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cbca5b9f0 0x559cbce56630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48296/0 (socket says 172.21.15.100:48296) 2024-07-26T13:43:30.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.580+0000 7f2fde45b640 1 -- 172.21.15.100:0/3265140243 learned_addr learned my addr 172.21.15.100:0/3265140243 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:30.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.580+0000 7f2fdd459640 1 --2- 172.21.15.100:0/3265140243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbca741b0 0x559cbce598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:30.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.580+0000 7f2fde45b640 1 --2- 172.21.15.100:0/3265140243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cbca7c390 0x559cbce59e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:30.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.580+0000 7f2fde45b640 1 -- 172.21.15.100:0/3265140243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cbca7c390 msgr2=0x559cbce59e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:30.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.580+0000 7f2fde45b640 1 --2- 172.21.15.100:0/3265140243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cbca7c390 0x559cbce59e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:30.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.580+0000 7f2fde45b640 1 -- 172.21.15.100:0/3265140243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbca741b0 msgr2=0x559cbce598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:30.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.580+0000 7f2fde45b640 1 --2- 172.21.15.100:0/3265140243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbca741b0 0x559cbce598c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:30.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.580+0000 7f2fde45b640 1 -- 172.21.15.100:0/3265140243 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cbce5a590 con 0x559cbca5b9f0 2024-07-26T13:43:30.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.580+0000 7f2fdd459640 1 --2- 172.21.15.100:0/3265140243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbca741b0 0x559cbce598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:43:30.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.580+0000 7f2fde45b640 1 --2- 172.21.15.100:0/3265140243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cbca5b9f0 0x559cbce56630 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009700 tx=0x7f2fc80114e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd4cf96b9be52c96 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:30.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.580+0000 7f2fbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:43:30.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.581+0000 7f2fbd7fa640 1 -- 172.21.15.100:0/3265140243 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc8011710 con 0x559cbca5b9f0 2024-07-26T13:43:30.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.581+0000 7f2fbd7fa640 1 -- 172.21.15.100:0/3265140243 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f2fc80118b0 con 0x559cbca5b9f0 2024-07-26T13:43:30.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.581+0000 7f2fe6442740 1 -- 172.21.15.100:0/3265140243 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fc8015030 con 0x559cbca5b9f0 2024-07-26T13:43:30.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.581+0000 7f2fbd7fa640 1 -- 172.21.15.100:0/3265140243 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f2fc8011ab0 con 0x559cbca5b9f0 2024-07-26T13:43:30.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.581+0000 7f2fe6442740 10 client.16410 fetch_fsmap learned FSMap version 36 2024-07-26T13:43:30.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.581+0000 7f2fe6442740 1 -- 172.21.15.100:0/3265140243 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x559cbcec8a10 con 0x559cbca5b9f0 2024-07-26T13:43:30.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.581+0000 7f2fbd7fa640 1 -- 172.21.15.100:0/3265140243 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fc8015030 con 0x559cbca5b9f0 2024-07-26T13:43:30.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.581+0000 7f2fbd7fa640 1 -- 172.21.15.100:0/3265140243 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f2fc8044050 con 0x559cbca5b9f0 2024-07-26T13:43:30.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.581+0000 7f2fe6442740 10 client.16410 fetch_fsmap finished waiting for FSMap version 36 2024-07-26T13:43:30.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.581+0000 7f2fe6442740 10 client.16410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:43:30.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.581+0000 7f2fe6442740 20 client.16410 populate_metadata read hostname 'smithi100' 2024-07-26T13:43:30.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.581+0000 7f2fe6442740 1 --2- 172.21.15.100:0/3265140243 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559cbca9d070 0x559cbcecae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:30.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.581+0000 7f2fe6442740 4 client.16410 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-26T13:43:30.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.581+0000 7f2fe6442740 1 -- 172.21.15.100:0/3265140243 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559cbcdb6550 con 0x559cbca9d070 2024-07-26T13:43:30.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.581+0000 7f2fdd459640 1 --2- 172.21.15.100:0/3265140243 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559cbca9d070 0x559cbcecae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:30.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.582+0000 7f2fbd7fa640 10 client.16410 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:43:30.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.582+0000 7f2fdd459640 1 --2- 172.21.15.100:0/3265140243 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559cbca9d070 0x559cbcecae80 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=441c3e88242af7c6 server_cookie=f311b6b2539ff290 in_seq=0 out_seq=0 2024-07-26T13:43:30.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.582+0000 7f2fbd7fa640 1 -- 172.21.15.100:0/3265140243 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559cbcdb6550 con 0x559cbca9d070 2024-07-26T13:43:30.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.582+0000 7f2fbd7fa640 10 client.16410 handle_command_reply: tid=0 2024-07-26T13:43:30.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.582+0000 7f2fe6442740 1 client.16410 shutdown 2024-07-26T13:43:30.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.582+0000 7f2fe6442740 1 -- 172.21.15.100:0/3265140243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cbca5b9f0 msgr2=0x559cbce56630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:30.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.582+0000 7f2fe6442740 1 --2- 172.21.15.100:0/3265140243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cbca5b9f0 0x559cbce56630 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009700 tx=0x7f2fc80114e0 comp rx=0 tx=0).stop 2024-07-26T13:43:30.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.582+0000 7f2fe6442740 1 -- 172.21.15.100:0/3265140243 shutdown_connections 2024-07-26T13:43:30.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.582+0000 7f2fe6442740 1 --2- 172.21.15.100:0/3265140243 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559cbca9d070 0x559cbcecae80 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:30.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.582+0000 7f2fe6442740 1 --2- 172.21.15.100:0/3265140243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559cbca7c390 0x559cbce59e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:30.631 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:43:30.631 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-26T13:43:30.631 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-07-26T13:43:30.631 INFO:teuthology.orchestra.run.smithi100.stdout: "181df3ac-9c51-4cdd-89b7-d49758149d30": { 2024-07-26T13:43:30.631 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-07-26T13:43:30.631 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "181df3ac-9c51-4cdd-89b7-d49758149d30", 2024-07-26T13:43:30.631 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-07-26T13:43:30.631 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:43:30.631 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:43:30.631 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:43:30.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.582+0000 7f2fe6442740 1 --2- 172.21.15.100:0/3265140243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cbca741b0 0x559cbce598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:30.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.582+0000 7f2fe6442740 1 --2- 172.21.15.100:0/3265140243 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559cbca5b9f0 0x559cbce56630 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:30.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.582+0000 7f2fbd7fa640 0 client.16410 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:43:30.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.582+0000 7f2fe6442740 1 -- 172.21.15.100:0/3265140243 >> 172.21.15.100:0/3265140243 conn(0x559cbca91460 msgr2=0x559cbce45620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:30.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.583+0000 7f2fe6442740 1 -- 172.21.15.100:0/3265140243 shutdown_connections 2024-07-26T13:43:30.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.583+0000 7f2fe6442740 1 -- 172.21.15.100:0/3265140243 wait complete. 2024-07-26T13:43:30.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.583+0000 7f2fe6442740 20 client.16410 trim_cache size 0 max 16384 2024-07-26T13:43:30.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.583+0000 7f2fdfebe640 1 -- 172.21.15.100:0/564194712 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f2fc00b5f50 msgr2=0x7f2fc00b8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:30.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.583+0000 7f2fdfebe640 1 --2- 172.21.15.100:0/564194712 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f2fc00b5f50 0x7f2fc00b8460 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2fd00aacd0 tx=0x7f2fd00e7f40 comp rx=0 tx=0).stop 2024-07-26T13:43:30.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.583+0000 7f2fdfebe640 1 -- 172.21.15.100:0/564194712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fd8102490 msgr2=0x7f2fd8198d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:30.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.583+0000 7f2fdfebe640 1 --2- 172.21.15.100:0/564194712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fd8102490 0x7f2fd8198d80 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f2fd000e9f0 tx=0x7f2fd000eeb0 comp rx=0 tx=0).stop 2024-07-26T13:43:30.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.583+0000 7f2fdfebe640 1 -- 172.21.15.100:0/564194712 shutdown_connections 2024-07-26T13:43:30.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.583+0000 7f2fdfebe640 1 --2- 172.21.15.100:0/564194712 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f2fc00b5f50 0x7f2fc00b8460 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:30.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.583+0000 7f2fdfebe640 1 --2- 172.21.15.100:0/564194712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fd8108720 0x7f2fd819e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:30.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.583+0000 7f2fdfebe640 1 --2- 172.21.15.100:0/564194712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fd8107dd0 0x7f2fd8199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:30.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.583+0000 7f2fdfebe640 1 --2- 172.21.15.100:0/564194712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fd8102490 0x7f2fd8198d80 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:30.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.583+0000 7f2fdfebe640 1 -- 172.21.15.100:0/564194712 >> 172.21.15.100:0/564194712 conn(0x7f2fd8079160 msgr2=0x7f2fd8104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:30.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.583+0000 7f2fdfebe640 1 -- 172.21.15.100:0/564194712 shutdown_connections 2024-07-26T13:43:30.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:30.583+0000 7f2fdfebe640 1 -- 172.21.15.100:0/564194712 wait complete. 2024-07-26T13:43:31.409 INFO:tasks.workunit.client.0.smithi100.stdout: 1 116041 6.10 MB/sec warmup 105 sec latency 11.797 ms 2024-07-26T13:43:31.506 INFO:tasks.cephfs.filesystem:scrub status for tag:181df3ac-9c51-4cdd-89b7-d49758149d30 - {'path': '/', 'tag': '181df3ac-9c51-4cdd-89b7-d49758149d30', 'options': 'recursive,force'} 2024-07-26T13:43:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:31 smithi100 ceph-mon[32599]: pgmap v467: 97 pgs: 97 active+clean; 52 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 393 op/s 2024-07-26T13:43:31.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:31 smithi138 ceph-mon[36954]: pgmap v467: 97 pgs: 97 active+clean; 52 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 393 op/s 2024-07-26T13:43:32.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:31 smithi167 ceph-mon[37140]: pgmap v467: 97 pgs: 97 active+clean; 52 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 393 op/s 2024-07-26T13:43:32.409 INFO:tasks.workunit.client.0.smithi100.stdout: 1 117775 6.15 MB/sec warmup 106 sec latency 11.452 ms 2024-07-26T13:43:33.409 INFO:tasks.workunit.client.0.smithi100.stdout: 1 120299 6.22 MB/sec warmup 107 sec latency 13.128 ms 2024-07-26T13:43:33.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:33 smithi138 ceph-mon[36954]: pgmap v468: 97 pgs: 97 active+clean; 53 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 325 op/s 2024-07-26T13:43:33.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:33 smithi100 ceph-mon[32599]: pgmap v468: 97 pgs: 97 active+clean; 53 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 325 op/s 2024-07-26T13:43:34.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:33 smithi167 ceph-mon[37140]: pgmap v468: 97 pgs: 97 active+clean; 53 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 325 op/s 2024-07-26T13:43:34.409 INFO:tasks.workunit.client.0.smithi100.stdout: 1 121854 6.23 MB/sec warmup 108 sec latency 13.744 ms 2024-07-26T13:43:35.409 INFO:tasks.workunit.client.0.smithi100.stdout: 1 124433 6.33 MB/sec warmup 109 sec latency 10.591 ms 2024-07-26T13:43:35.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:35 smithi138 ceph-mon[36954]: pgmap v469: 97 pgs: 97 active+clean; 56 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 350 op/s 2024-07-26T13:43:35.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:35 smithi138 ceph-mon[36954]: scrub summary: idle 2024-07-26T13:43:35.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:35 smithi138 ceph-mon[36954]: scrub summary: idle 2024-07-26T13:43:35.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:35 smithi138 ceph-mon[36954]: scrub summary: idle 2024-07-26T13:43:35.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:35 smithi100 ceph-mon[32599]: pgmap v469: 97 pgs: 97 active+clean; 56 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 350 op/s 2024-07-26T13:43:35.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:35 smithi100 ceph-mon[32599]: scrub summary: idle 2024-07-26T13:43:35.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:35 smithi100 ceph-mon[32599]: scrub summary: idle 2024-07-26T13:43:35.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:35 smithi100 ceph-mon[32599]: scrub summary: idle 2024-07-26T13:43:36.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:35 smithi167 ceph-mon[37140]: pgmap v469: 97 pgs: 97 active+clean; 56 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 350 op/s 2024-07-26T13:43:36.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:35 smithi167 ceph-mon[37140]: scrub summary: idle 2024-07-26T13:43:36.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:35 smithi167 ceph-mon[37140]: scrub summary: idle 2024-07-26T13:43:36.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:35 smithi167 ceph-mon[37140]: scrub summary: idle 2024-07-26T13:43:36.409 INFO:tasks.workunit.client.0.smithi100.stdout: 1 126521 6.34 MB/sec warmup 110 sec latency 13.554 ms 2024-07-26T13:43:37.409 INFO:tasks.workunit.client.0.smithi100.stdout: 1 128181 6.39 MB/sec warmup 111 sec latency 9.534 ms 2024-07-26T13:43:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.886+0000 7f1b1eada640 1 -- 172.21.15.100:0/973782565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b18107d40 msgr2=0x7f1b18108150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.886+0000 7f1b1eada640 1 --2- 172.21.15.100:0/973782565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b18107d40 0x7f1b18108150 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f1b04009900 tx=0x7f1b0402a410 comp rx=0 tx=0).stop 2024-07-26T13:43:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.886+0000 7f1b1eada640 1 -- 172.21.15.100:0/973782565 shutdown_connections 2024-07-26T13:43:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.886+0000 7f1b1eada640 1 --2- 172.21.15.100:0/973782565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b18108690 0x7f1b1810eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.886+0000 7f1b1eada640 1 --2- 172.21.15.100:0/973782565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b18107d40 0x7f1b18108150 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.886+0000 7f1b1eada640 1 --2- 172.21.15.100:0/973782565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b18102410 0x7f1b18102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.886+0000 7f1b1eada640 1 -- 172.21.15.100:0/973782565 >> 172.21.15.100:0/973782565 conn(0x7f1b180fddf0 msgr2=0x7f1b18100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.886+0000 7f1b1eada640 1 -- 172.21.15.100:0/973782565 shutdown_connections 2024-07-26T13:43:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.886+0000 7f1b1eada640 1 -- 172.21.15.100:0/973782565 wait complete. 2024-07-26T13:43:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.887+0000 7f1b1eada640 1 Processor -- start 2024-07-26T13:43:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.887+0000 7f1b1eada640 1 -- start start 2024-07-26T13:43:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.887+0000 7f1b1eada640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b18102410 0x7f1b18198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.887+0000 7f1b1eada640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b18107d40 0x7f1b18199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.887+0000 7f1b1eada640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b18108690 0x7f1b1819e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.887+0000 7f1b1eada640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1b18112370 con 0x7f1b18108690 2024-07-26T13:43:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.887+0000 7f1b1eada640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1b181121f0 con 0x7f1b18102410 2024-07-26T13:43:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.887+0000 7f1b1eada640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1b181124f0 con 0x7f1b18107d40 2024-07-26T13:43:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.887+0000 7f1b1d077640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b18108690 0x7f1b1819e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.887+0000 7f1b1c876640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b18102410 0x7f1b18198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.887+0000 7f1b1c876640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b18102410 0x7f1b18198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:40430/0 (socket says 172.21.15.100:40430) 2024-07-26T13:43:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.887+0000 7f1b1c876640 1 -- 172.21.15.100:0/2999786406 learned_addr learned my addr 172.21.15.100:0/2999786406 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.887+0000 7f1b1d077640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b18108690 0x7f1b1819e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54540/0 (socket says 172.21.15.100:54540) 2024-07-26T13:43:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.887+0000 7f1b17fff640 1 --2- 172.21.15.100:0/2999786406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b18107d40 0x7f1b18199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.887+0000 7f1b1c876640 1 -- 172.21.15.100:0/2999786406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b18107d40 msgr2=0x7f1b18199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.887+0000 7f1b1c876640 1 --2- 172.21.15.100:0/2999786406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b18107d40 0x7f1b18199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.887+0000 7f1b1c876640 1 -- 172.21.15.100:0/2999786406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b18108690 msgr2=0x7f1b1819e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.887+0000 7f1b1c876640 1 --2- 172.21.15.100:0/2999786406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b18108690 0x7f1b1819e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.887+0000 7f1b1c876640 1 -- 172.21.15.100:0/2999786406 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b1819ece0 con 0x7f1b18102410 2024-07-26T13:43:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.887+0000 7f1b1d077640 1 --2- 172.21.15.100:0/2999786406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b18108690 0x7f1b1819e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:43:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.888+0000 7f1b1c876640 1 --2- 172.21.15.100:0/2999786406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b18102410 0x7f1b18198db0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00e9f0 tx=0x7f1b0c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.888+0000 7f1b15ffb640 1 -- 172.21.15.100:0/2999786406 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c00cd80 con 0x7f1b18102410 2024-07-26T13:43:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.888+0000 7f1b1eada640 1 -- 172.21.15.100:0/2999786406 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b1819efd0 con 0x7f1b18102410 2024-07-26T13:43:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.888+0000 7f1b15ffb640 1 -- 172.21.15.100:0/2999786406 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b0c0054d0 con 0x7f1b18102410 2024-07-26T13:43:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.888+0000 7f1b15ffb640 1 -- 172.21.15.100:0/2999786406 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c010690 con 0x7f1b18102410 2024-07-26T13:43:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.888+0000 7f1b1eada640 1 -- 172.21.15.100:0/2999786406 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b181a5760 con 0x7f1b18102410 2024-07-26T13:43:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.890+0000 7f1b259fc740 1 Processor -- start 2024-07-26T13:43:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.890+0000 7f1b15ffb640 1 -- 172.21.15.100:0/2999786406 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f1b0c004e90 con 0x7f1b18102410 2024-07-26T13:43:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.890+0000 7f1b259fc740 1 -- start start 2024-07-26T13:43:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.891+0000 7f1b259fc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6b21121b0 0x55d6b24b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.891+0000 7f1b259fc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6b20e92f0 0x55d6b24b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.891+0000 7f1b259fc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6b20f14d0 0x55d6b24b5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.891+0000 7f1b259fc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55d6b242a3b0 con 0x55d6b21121b0 2024-07-26T13:43:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.891+0000 7f1b259fc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d6b24198b0 con 0x55d6b20e92f0 2024-07-26T13:43:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.891+0000 7f1b259fc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d6b241d8f0 con 0x55d6b20f14d0 2024-07-26T13:43:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.891+0000 7f1b1d077640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6b21121b0 0x55d6b24b46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.891+0000 7f1b1c876640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6b20e92f0 0x55d6b24b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.891+0000 7f1b17fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6b20f14d0 0x55d6b24b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.891+0000 7f1b1d077640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6b21121b0 0x55d6b24b46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54542/0 (socket says 172.21.15.100:54542) 2024-07-26T13:43:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.891+0000 7f1b1c876640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6b20e92f0 0x55d6b24b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:40446/0 (socket says 172.21.15.100:40446) 2024-07-26T13:43:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.891+0000 7f1b1d077640 1 -- 172.21.15.100:0/621664547 learned_addr learned my addr 172.21.15.100:0/621664547 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.891+0000 7f1b1d077640 1 -- 172.21.15.100:0/621664547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6b20f14d0 msgr2=0x55d6b24b5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.891+0000 7f1b1d077640 1 --2- 172.21.15.100:0/621664547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6b20f14d0 0x55d6b24b5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.891+0000 7f1b1d077640 1 -- 172.21.15.100:0/621664547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6b20e92f0 msgr2=0x55d6b24b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.891+0000 7f1b1d077640 1 --2- 172.21.15.100:0/621664547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6b20e92f0 0x55d6b24b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:37.900 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.891+0000 7f1b15ffb640 1 --2- 172.21.15.100:0/2999786406 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1b080b3d40 0x7f1b080b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.891+0000 7f1b1d077640 1 -- 172.21.15.100:0/621664547 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6b23c09a0 con 0x55d6b21121b0 2024-07-26T13:43:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.891+0000 7f1b17fff640 1 --2- 172.21.15.100:0/2999786406 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1b080b3d40 0x7f1b080b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.891+0000 7f1b1d077640 1 --2- 172.21.15.100:0/621664547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6b21121b0 0x55d6b24b46b0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f1b100117e0 tx=0x7f1b10011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a118cae5410087a1 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.891+0000 7f1b15ffb640 1 -- 172.21.15.100:0/2999786406 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f1b0c0e4410 con 0x7f1b18102410 2024-07-26T13:43:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.892+0000 7f1b02ffd640 1 -- 172.21.15.100:0/621664547 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b10011310 con 0x55d6b21121b0 2024-07-26T13:43:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.892+0000 7f1b02ffd640 1 -- 172.21.15.100:0/621664547 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b10003e20 con 0x55d6b21121b0 2024-07-26T13:43:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.892+0000 7f1b02ffd640 1 -- 172.21.15.100:0/621664547 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b10004450 con 0x55d6b21121b0 2024-07-26T13:43:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.892+0000 7f1b259fc740 1 -- 172.21.15.100:0/621664547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6b21121b0 msgr2=0x55d6b24b46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.892+0000 7f1b17fff640 1 --2- 172.21.15.100:0/2999786406 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1b080b3d40 0x7f1b080b6250 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1b0402aa00 tx=0x7f1b04005a20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.892+0000 7f1b259fc740 1 --2- 172.21.15.100:0/621664547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6b21121b0 0x55d6b24b46b0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f1b100117e0 tx=0x7f1b10011ca0 comp rx=0 tx=0).stop 2024-07-26T13:43:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.893+0000 7f1b259fc740 1 -- 172.21.15.100:0/621664547 shutdown_connections 2024-07-26T13:43:37.901 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.893+0000 7f1b259fc740 1 --2- 172.21.15.100:0/621664547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6b20f14d0 0x55d6b24b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.893+0000 7f1b259fc740 1 --2- 172.21.15.100:0/621664547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6b20e92f0 0x55d6b24b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.893+0000 7f1b259fc740 1 --2- 172.21.15.100:0/621664547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6b21121b0 0x55d6b24b46b0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.893+0000 7f1b259fc740 1 -- 172.21.15.100:0/621664547 >> 172.21.15.100:0/621664547 conn(0x55d6b2106300 msgr2=0x55d6b24b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.893+0000 7f1b259fc740 1 -- 172.21.15.100:0/621664547 shutdown_connections 2024-07-26T13:43:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.893+0000 7f1b259fc740 1 -- 172.21.15.100:0/621664547 wait complete. 2024-07-26T13:43:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.893+0000 7f1b259fc740 1 -- start start 2024-07-26T13:43:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.893+0000 7f1b259fc740 1 Processor -- start 2024-07-26T13:43:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.893+0000 7f1b259fc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6b20e92f0 0x55d6b24c01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.894+0000 7f1b259fc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6b20f14d0 0x55d6b24c0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.894+0000 7f1b259fc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6b21121b0 0x55d6b24cc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:37.902 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.894+0000 7f1b259fc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55d6b2353ab0 con 0x55d6b20f14d0 2024-07-26T13:43:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.894+0000 7f1b259fc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d6b23a1cf0 con 0x55d6b21121b0 2024-07-26T13:43:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.894+0000 7f1b259fc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d6b24c2e30 con 0x55d6b20e92f0 2024-07-26T13:43:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.894+0000 7f1b259fc740 10 client.0 _ll_register_callbacks cb 0x55d6b24133e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:43:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.894+0000 7f1b17fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6b20f14d0 0x55d6b24c0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.894+0000 7f1b1c876640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6b20e92f0 0x55d6b24c01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.894+0000 7f1b17fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6b20f14d0 0x55d6b24c0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54552/0 (socket says 172.21.15.100:54552) 2024-07-26T13:43:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.894+0000 7f1b17fff640 1 -- 172.21.15.100:0/3291483581 learned_addr learned my addr 172.21.15.100:0/3291483581 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.894+0000 7f1b1d077640 1 --2- 172.21.15.100:0/3291483581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6b21121b0 0x55d6b24cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.894+0000 7f1b17fff640 1 -- 172.21.15.100:0/3291483581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6b20e92f0 msgr2=0x55d6b24c01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.894+0000 7f1b17fff640 1 --2- 172.21.15.100:0/3291483581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6b20e92f0 0x55d6b24c01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.894+0000 7f1b17fff640 1 -- 172.21.15.100:0/3291483581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6b21121b0 msgr2=0x55d6b24cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.894+0000 7f1b17fff640 1 --2- 172.21.15.100:0/3291483581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6b21121b0 0x55d6b24cc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.894+0000 7f1b17fff640 1 -- 172.21.15.100:0/3291483581 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6b217d790 con 0x55d6b20f14d0 2024-07-26T13:43:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.894+0000 7f1b1c876640 1 --2- 172.21.15.100:0/3291483581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6b20e92f0 0x55d6b24c01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:43:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.894+0000 7f1b17fff640 1 --2- 172.21.15.100:0/3291483581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6b20f14d0 0x55d6b24c0720 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f1b04005ab0 tx=0x7f1b04009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2be523e4f1d2a3a server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.894+0000 7f1af7fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:43:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.894+0000 7f1af7fff640 1 -- 172.21.15.100:0/3291483581 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0404e070 con 0x55d6b20f14d0 2024-07-26T13:43:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.894+0000 7f1af7fff640 1 -- 172.21.15.100:0/3291483581 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f1b04004950 con 0x55d6b20f14d0 2024-07-26T13:43:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.894+0000 7f1b259fc740 1 -- 172.21.15.100:0/3291483581 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b10003e20 con 0x55d6b20f14d0 2024-07-26T13:43:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.895+0000 7f1af7fff640 1 -- 172.21.15.100:0/3291483581 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1b0405a050 con 0x55d6b20f14d0 2024-07-26T13:43:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.895+0000 7f1b259fc740 10 client.16422 fetch_fsmap learned FSMap version 36 2024-07-26T13:43:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.895+0000 7f1b259fc740 1 -- 172.21.15.100:0/3291483581 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d6b21f6620 con 0x55d6b20f14d0 2024-07-26T13:43:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.895+0000 7f1af7fff640 1 -- 172.21.15.100:0/3291483581 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b10003e20 con 0x55d6b20f14d0 2024-07-26T13:43:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.895+0000 7f1af7fff640 1 -- 172.21.15.100:0/3291483581 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f1b040324b0 con 0x55d6b20f14d0 2024-07-26T13:43:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.895+0000 7f1b259fc740 10 client.16422 fetch_fsmap finished waiting for FSMap version 36 2024-07-26T13:43:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.895+0000 7f1b259fc740 10 client.16422 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:43:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.895+0000 7f1b259fc740 20 client.16422 populate_metadata read hostname 'smithi100' 2024-07-26T13:43:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.895+0000 7f1b259fc740 1 --2- 172.21.15.100:0/3291483581 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55d6b20d0d80 0x55d6b24d4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.895+0000 7f1b259fc740 4 client.16422 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:43:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.895+0000 7f1b259fc740 1 -- 172.21.15.100:0/3291483581 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6b242c300 con 0x55d6b20d0d80 2024-07-26T13:43:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.895+0000 7f1b1c876640 1 --2- 172.21.15.100:0/3291483581 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55d6b20d0d80 0x55d6b24d4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.896+0000 7f1af7fff640 10 client.16422 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:43:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.896+0000 7f1b1c876640 1 --2- 172.21.15.100:0/3291483581 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55d6b20d0d80 0x55d6b24d4770 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5eb8b327e54d213 server_cookie=335423c364a018f6 in_seq=0 out_seq=0 2024-07-26T13:43:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.896+0000 7f1af7fff640 1 -- 172.21.15.100:0/3291483581 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d6b242c300 con 0x55d6b20d0d80 2024-07-26T13:43:37.905 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.896+0000 7f1af7fff640 10 client.16422 handle_command_reply: tid=0 2024-07-26T13:43:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.896+0000 7f1b259fc740 1 client.16422 shutdown 2024-07-26T13:43:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.896+0000 7f1b259fc740 1 -- 172.21.15.100:0/3291483581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6b20f14d0 msgr2=0x55d6b24c0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.896+0000 7f1b259fc740 1 --2- 172.21.15.100:0/3291483581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6b20f14d0 0x55d6b24c0720 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f1b04005ab0 tx=0x7f1b04009260 comp rx=0 tx=0).stop 2024-07-26T13:43:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.897+0000 7f1b259fc740 1 -- 172.21.15.100:0/3291483581 shutdown_connections 2024-07-26T13:43:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.897+0000 7f1b259fc740 1 --2- 172.21.15.100:0/3291483581 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55d6b20d0d80 0x55d6b24d4770 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.897+0000 7f1b259fc740 1 --2- 172.21.15.100:0/3291483581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6b21121b0 0x55d6b24cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.897+0000 7f1b259fc740 1 --2- 172.21.15.100:0/3291483581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6b20f14d0 0x55d6b24c0720 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.897+0000 7f1b259fc740 1 --2- 172.21.15.100:0/3291483581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6b20e92f0 0x55d6b24c01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.897+0000 7f1b259fc740 1 -- 172.21.15.100:0/3291483581 >> 172.21.15.100:0/3291483581 conn(0x55d6b2106300 msgr2=0x55d6b24b56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.897+0000 7f1af7fff640 0 client.16422 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:43:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.897+0000 7f1b259fc740 1 -- 172.21.15.100:0/3291483581 shutdown_connections 2024-07-26T13:43:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.897+0000 7f1b259fc740 1 -- 172.21.15.100:0/3291483581 wait complete. 2024-07-26T13:43:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.897+0000 7f1b259fc740 20 client.16422 trim_cache size 0 max 16384 2024-07-26T13:43:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.913+0000 7f1b259fc740 1 Processor -- start 2024-07-26T13:43:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.913+0000 7f1b259fc740 1 -- start start 2024-07-26T13:43:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.913+0000 7f1b259fc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6b20e92f0 0x55d6b24ba780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.913+0000 7f1b259fc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6b20f14d0 0x55d6b24bf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.913+0000 7f1b259fc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6b21121b0 0x55d6b24bf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.913+0000 7f1b259fc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1b181af900 con 0x55d6b20e92f0 2024-07-26T13:43:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.913+0000 7f1b259fc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1b181ad170 con 0x55d6b21121b0 2024-07-26T13:43:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.913+0000 7f1b259fc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1b181aafc0 con 0x55d6b20f14d0 2024-07-26T13:43:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.913+0000 7f1b1c876640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6b20e92f0 0x55d6b24ba780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.913+0000 7f1b1c876640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6b20e92f0 0x55d6b24ba780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54566/0 (socket says 172.21.15.100:54566) 2024-07-26T13:43:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.913+0000 7f1b17fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6b20f14d0 0x55d6b24bf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.913+0000 7f1b1c876640 1 -- 172.21.15.100:0/3026895824 learned_addr learned my addr 172.21.15.100:0/3026895824 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.913+0000 7f1b1d077640 1 --2- 172.21.15.100:0/3026895824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6b21121b0 0x55d6b24bf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.914+0000 7f1b1c876640 1 -- 172.21.15.100:0/3026895824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6b20f14d0 msgr2=0x55d6b24bf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.914+0000 7f1b1c876640 1 --2- 172.21.15.100:0/3026895824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6b20f14d0 0x55d6b24bf4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.914+0000 7f1b1c876640 1 -- 172.21.15.100:0/3026895824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6b21121b0 msgr2=0x55d6b24bf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.914+0000 7f1b1c876640 1 --2- 172.21.15.100:0/3026895824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6b21121b0 0x55d6b24bf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.914+0000 7f1b1c876640 1 -- 172.21.15.100:0/3026895824 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6b24b54e0 con 0x55d6b20e92f0 2024-07-26T13:43:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.914+0000 7f1b1c876640 1 --2- 172.21.15.100:0/3026895824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6b20e92f0 0x55d6b24ba780 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c0a7070 tx=0x7f1b0c0e7f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a3a4a3748d5f23a server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.914+0000 7f1b027fc640 1 -- 172.21.15.100:0/3026895824 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c0aadc0 con 0x55d6b20e92f0 2024-07-26T13:43:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.914+0000 7f1b027fc640 1 -- 172.21.15.100:0/3026895824 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b0c0aab00 con 0x55d6b20e92f0 2024-07-26T13:43:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.914+0000 7f1b027fc640 1 -- 172.21.15.100:0/3026895824 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c005180 con 0x55d6b20e92f0 2024-07-26T13:43:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.915+0000 7f1b259fc740 1 -- 172.21.15.100:0/3026895824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6b20e92f0 msgr2=0x55d6b24ba780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.915+0000 7f1b259fc740 1 --2- 172.21.15.100:0/3026895824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6b20e92f0 0x55d6b24ba780 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c0a7070 tx=0x7f1b0c0e7f60 comp rx=0 tx=0).stop 2024-07-26T13:43:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.915+0000 7f1b259fc740 1 -- 172.21.15.100:0/3026895824 shutdown_connections 2024-07-26T13:43:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.915+0000 7f1b259fc740 1 --2- 172.21.15.100:0/3026895824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6b21121b0 0x55d6b24bf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.915+0000 7f1b259fc740 1 --2- 172.21.15.100:0/3026895824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6b20f14d0 0x55d6b24bf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.915+0000 7f1b259fc740 1 --2- 172.21.15.100:0/3026895824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6b20e92f0 0x55d6b24ba780 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.915+0000 7f1b259fc740 1 -- 172.21.15.100:0/3026895824 >> 172.21.15.100:0/3026895824 conn(0x55d6b2106300 msgr2=0x55d6b24be830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.915+0000 7f1b259fc740 1 -- 172.21.15.100:0/3026895824 shutdown_connections 2024-07-26T13:43:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.915+0000 7f1b259fc740 1 -- 172.21.15.100:0/3026895824 wait complete. 2024-07-26T13:43:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.915+0000 7f1b259fc740 1 -- start start 2024-07-26T13:43:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.915+0000 7f1b259fc740 1 Processor -- start 2024-07-26T13:43:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.916+0000 7f1b259fc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6b20e92f0 0x55d6b24c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.916+0000 7f1b259fc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6b20f14d0 0x55d6b24c8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.916+0000 7f1b259fc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6b21121b0 0x55d6b24c5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.916+0000 7f1b259fc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1b181ab630 con 0x55d6b20e92f0 2024-07-26T13:43:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.916+0000 7f1b259fc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1b181acaa0 con 0x55d6b21121b0 2024-07-26T13:43:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.916+0000 7f1b259fc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1b181abd00 con 0x55d6b20f14d0 2024-07-26T13:43:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.916+0000 7f1b259fc740 10 client.0 _ll_register_callbacks cb 0x7f1b04009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:43:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.916+0000 7f1b1c876640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6b20e92f0 0x55d6b24c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.916+0000 7f1b1c876640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6b20e92f0 0x55d6b24c82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54568/0 (socket says 172.21.15.100:54568) 2024-07-26T13:43:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.916+0000 7f1b1c876640 1 -- 172.21.15.100:0/643124134 learned_addr learned my addr 172.21.15.100:0/643124134 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.916+0000 7f1b17fff640 1 --2- 172.21.15.100:0/643124134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6b20f14d0 0x55d6b24c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.916+0000 7f1b1d077640 1 --2- 172.21.15.100:0/643124134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6b21121b0 0x55d6b24c5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.916+0000 7f1b1c876640 1 -- 172.21.15.100:0/643124134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6b20f14d0 msgr2=0x55d6b24c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.916+0000 7f1b1c876640 1 --2- 172.21.15.100:0/643124134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6b20f14d0 0x55d6b24c8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.916+0000 7f1b1c876640 1 -- 172.21.15.100:0/643124134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6b21121b0 msgr2=0x55d6b24c5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.916+0000 7f1b1c876640 1 --2- 172.21.15.100:0/643124134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6b21121b0 0x55d6b24c5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.916+0000 7f1b1c876640 1 -- 172.21.15.100:0/643124134 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6b24c5720 con 0x55d6b20e92f0 2024-07-26T13:43:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.917+0000 7f1b1c876640 1 --2- 172.21.15.100:0/643124134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6b20e92f0 0x55d6b24c82c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00c6a0 tx=0x7f1b0c0b4110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a262f5ea9474c90b server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.917+0000 7f1af7fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:43:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.917+0000 7f1af7fff640 1 -- 172.21.15.100:0/643124134 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c0c4070 con 0x55d6b20e92f0 2024-07-26T13:43:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.917+0000 7f1af7fff640 1 -- 172.21.15.100:0/643124134 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f1b0c0a7280 con 0x55d6b20e92f0 2024-07-26T13:43:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.917+0000 7f1b259fc740 1 -- 172.21.15.100:0/643124134 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b0c0aab00 con 0x55d6b20e92f0 2024-07-26T13:43:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.917+0000 7f1af7fff640 1 -- 172.21.15.100:0/643124134 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1b0c0aa760 con 0x55d6b20e92f0 2024-07-26T13:43:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.917+0000 7f1b259fc740 10 client.16428 fetch_fsmap learned FSMap version 36 2024-07-26T13:43:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.917+0000 7f1b259fc740 1 -- 172.21.15.100:0/643124134 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d6b253b870 con 0x55d6b20e92f0 2024-07-26T13:43:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.917+0000 7f1af7fff640 1 -- 172.21.15.100:0/643124134 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b0c0aab00 con 0x55d6b20e92f0 2024-07-26T13:43:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.917+0000 7f1af7fff640 1 -- 172.21.15.100:0/643124134 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f1b0c0b5070 con 0x55d6b20e92f0 2024-07-26T13:43:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.917+0000 7f1b259fc740 10 client.16428 fetch_fsmap finished waiting for FSMap version 36 2024-07-26T13:43:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.917+0000 7f1b259fc740 10 client.16428 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:43:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.917+0000 7f1b259fc740 20 client.16428 populate_metadata read hostname 'smithi100' 2024-07-26T13:43:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.917+0000 7f1b259fc740 1 --2- 172.21.15.100:0/643124134 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55d6b20a61c0 0x55d6b253e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.917+0000 7f1b259fc740 4 client.16428 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-26T13:43:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.917+0000 7f1b259fc740 1 -- 172.21.15.100:0/643124134 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d6b2427da0 con 0x55d6b20a61c0 2024-07-26T13:43:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.917+0000 7f1b17fff640 1 --2- 172.21.15.100:0/643124134 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55d6b20a61c0 0x55d6b253e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.918+0000 7f1af7fff640 10 client.16428 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:43:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.918+0000 7f1b17fff640 1 --2- 172.21.15.100:0/643124134 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55d6b20a61c0 0x55d6b253e030 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab24ae8c3deed535 server_cookie=729a99fbbd7e34a5 in_seq=0 out_seq=0 2024-07-26T13:43:37.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:37 smithi138 ceph-mon[36954]: pgmap v470: 97 pgs: 97 active+clean; 58 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 319 op/s 2024-07-26T13:43:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.953+0000 7f1af7fff640 1 -- 172.21.15.100:0/643124134 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d6b2427da0 con 0x55d6b20a61c0 2024-07-26T13:43:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.953+0000 7f1af7fff640 10 client.16428 handle_command_reply: tid=0 2024-07-26T13:43:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.953+0000 7f1b259fc740 1 client.16428 shutdown 2024-07-26T13:43:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.953+0000 7f1b259fc740 1 -- 172.21.15.100:0/643124134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6b20e92f0 msgr2=0x55d6b24c82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.953+0000 7f1b259fc740 1 --2- 172.21.15.100:0/643124134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6b20e92f0 0x55d6b24c82c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00c6a0 tx=0x7f1b0c0b4110 comp rx=0 tx=0).stop 2024-07-26T13:43:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.953+0000 7f1b259fc740 1 -- 172.21.15.100:0/643124134 shutdown_connections 2024-07-26T13:43:37.956 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.953+0000 7f1b259fc740 1 --2- 172.21.15.100:0/643124134 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55d6b20a61c0 0x55d6b253e030 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:37.956 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.953+0000 7f1b259fc740 1 --2- 172.21.15.100:0/643124134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d6b21121b0 0x55d6b24c5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:37.956 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.953+0000 7f1b259fc740 1 --2- 172.21.15.100:0/643124134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6b20f14d0 0x55d6b24c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:37.956 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.953+0000 7f1b259fc740 1 --2- 172.21.15.100:0/643124134 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d6b20e92f0 0x55d6b24c82c0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:37.956 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.953+0000 7f1af7fff640 0 client.16428 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:43:37.956 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.953+0000 7f1b259fc740 1 -- 172.21.15.100:0/643124134 >> 172.21.15.100:0/643124134 conn(0x55d6b2106300 msgr2=0x55d6b24bf4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:37.956 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.953+0000 7f1b259fc740 1 -- 172.21.15.100:0/643124134 shutdown_connections 2024-07-26T13:43:37.956 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.953+0000 7f1b259fc740 1 -- 172.21.15.100:0/643124134 wait complete. 2024-07-26T13:43:37.956 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.953+0000 7f1b259fc740 20 client.16428 trim_cache size 0 max 16384 2024-07-26T13:43:37.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-07-26T13:43:37.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-07-26T13:43:37.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-07-26T13:43:37.957 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-07-26T13:43:37.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.954+0000 7f1b1eada640 1 -- 172.21.15.100:0/2999786406 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1b080b3d40 msgr2=0x7f1b080b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:37.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.954+0000 7f1b1eada640 1 --2- 172.21.15.100:0/2999786406 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1b080b3d40 0x7f1b080b6250 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1b0402aa00 tx=0x7f1b04005a20 comp rx=0 tx=0).stop 2024-07-26T13:43:37.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.954+0000 7f1b1eada640 1 -- 172.21.15.100:0/2999786406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b18102410 msgr2=0x7f1b18198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:37.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.954+0000 7f1b1eada640 1 --2- 172.21.15.100:0/2999786406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b18102410 0x7f1b18198db0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00e9f0 tx=0x7f1b0c00edf0 comp rx=0 tx=0).stop 2024-07-26T13:43:37.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.954+0000 7f1b1eada640 1 -- 172.21.15.100:0/2999786406 shutdown_connections 2024-07-26T13:43:37.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.954+0000 7f1b1eada640 1 --2- 172.21.15.100:0/2999786406 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1b080b3d40 0x7f1b080b6250 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:37.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.954+0000 7f1b1eada640 1 --2- 172.21.15.100:0/2999786406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b18108690 0x7f1b1819e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:37.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.954+0000 7f1b1eada640 1 --2- 172.21.15.100:0/2999786406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b18107d40 0x7f1b18199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:37.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.954+0000 7f1b1eada640 1 --2- 172.21.15.100:0/2999786406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b18102410 0x7f1b18198db0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:37.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.954+0000 7f1b1eada640 1 -- 172.21.15.100:0/2999786406 >> 172.21.15.100:0/2999786406 conn(0x7f1b180fddf0 msgr2=0x7f1b18101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:37.958 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.954+0000 7f1b1eada640 1 -- 172.21.15.100:0/2999786406 shutdown_connections 2024-07-26T13:43:37.959 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:37.954+0000 7f1b1eada640 1 -- 172.21.15.100:0/2999786406 wait complete. 2024-07-26T13:43:37.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:37 smithi100 ceph-mon[32599]: pgmap v470: 97 pgs: 97 active+clean; 58 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 319 op/s 2024-07-26T13:43:38.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:37 smithi167 ceph-mon[37140]: pgmap v470: 97 pgs: 97 active+clean; 58 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 319 op/s 2024-07-26T13:43:38.409 INFO:tasks.workunit.client.0.smithi100.stdout: 1 130605 6.43 MB/sec warmup 112 sec latency 31.258 ms 2024-07-26T13:43:39.409 INFO:tasks.workunit.client.0.smithi100.stdout: 1 132315 6.46 MB/sec warmup 113 sec latency 11.721 ms 2024-07-26T13:43:39.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:39 smithi138 ceph-mon[36954]: pgmap v471: 97 pgs: 97 active+clean; 60 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 309 op/s 2024-07-26T13:43:39.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:39 smithi100 ceph-mon[32599]: pgmap v471: 97 pgs: 97 active+clean; 60 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 309 op/s 2024-07-26T13:43:40.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:39 smithi167 ceph-mon[37140]: pgmap v471: 97 pgs: 97 active+clean; 60 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 309 op/s 2024-07-26T13:43:40.409 INFO:tasks.workunit.client.0.smithi100.stdout: 1 134903 6.55 MB/sec warmup 114 sec latency 13.256 ms 2024-07-26T13:43:40.690 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 31.1 2024-07-26T13:43:40.690 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:43:40.958 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:43:41.409 INFO:tasks.workunit.client.0.smithi100.stdout: 1 136350 6.55 MB/sec warmup 115 sec latency 18.518 ms 2024-07-26T13:43:41.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:41 smithi100 ceph-mon[32599]: pgmap v472: 97 pgs: 97 active+clean; 56 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.2 MiB/s wr, 351 op/s 2024-07-26T13:43:41.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:41 smithi138 ceph-mon[36954]: pgmap v472: 97 pgs: 97 active+clean; 56 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.2 MiB/s wr, 351 op/s 2024-07-26T13:43:42.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:41 smithi167 ceph-mon[37140]: pgmap v472: 97 pgs: 97 active+clean; 56 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.2 MiB/s wr, 351 op/s 2024-07-26T13:43:42.409 INFO:tasks.workunit.client.0.smithi100.stdout: 1 138724 6.61 MB/sec warmup 116 sec latency 12.410 ms 2024-07-26T13:43:42.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.849+0000 7f3c5b9eb640 1 -- 172.21.15.100:0/2500268703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c54107dd0 msgr2=0x7f3c541081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:42.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.849+0000 7f3c5b9eb640 1 --2- 172.21.15.100:0/2500268703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c54107dd0 0x7f3c541081e0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3c44009900 tx=0x7f3c4402a410 comp rx=0 tx=0).stop 2024-07-26T13:43:42.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.849+0000 7f3c5b9eb640 1 -- 172.21.15.100:0/2500268703 shutdown_connections 2024-07-26T13:43:42.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.849+0000 7f3c5b9eb640 1 --2- 172.21.15.100:0/2500268703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c54108720 0x7f3c5410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:42.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.849+0000 7f3c5b9eb640 1 --2- 172.21.15.100:0/2500268703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c54107dd0 0x7f3c541081e0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:42.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.849+0000 7f3c5b9eb640 1 --2- 172.21.15.100:0/2500268703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c54102490 0x7f3c541028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:42.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.849+0000 7f3c5b9eb640 1 -- 172.21.15.100:0/2500268703 >> 172.21.15.100:0/2500268703 conn(0x7f3c54079160 msgr2=0x7f3c54079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:42.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.849+0000 7f3c5b9eb640 1 -- 172.21.15.100:0/2500268703 shutdown_connections 2024-07-26T13:43:42.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.849+0000 7f3c5b9eb640 1 -- 172.21.15.100:0/2500268703 wait complete. 2024-07-26T13:43:42.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.850+0000 7f3c5b9eb640 1 Processor -- start 2024-07-26T13:43:42.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.850+0000 7f3c5b9eb640 1 -- start start 2024-07-26T13:43:42.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.850+0000 7f3c5b9eb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c54102490 0x7f3c54199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:42.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.850+0000 7f3c5b9eb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c54107dd0 0x7f3c5419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:42.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.850+0000 7f3c5b9eb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c54108720 0x7f3c5419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:42.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.850+0000 7f3c5b9eb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3c541134b0 con 0x7f3c54108720 2024-07-26T13:43:42.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.850+0000 7f3c5b9eb640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3c54113330 con 0x7f3c54102490 2024-07-26T13:43:42.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.850+0000 7f3c5b9eb640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3c54113630 con 0x7f3c54107dd0 2024-07-26T13:43:42.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.850+0000 7f3c59f88640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c54108720 0x7f3c5419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:42.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.850+0000 7f3c59787640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c54102490 0x7f3c54199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:42.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.850+0000 7f3c58f86640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c54107dd0 0x7f3c5419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:42.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.850+0000 7f3c59787640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c54102490 0x7f3c54199ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:40488/0 (socket says 172.21.15.100:40488) 2024-07-26T13:43:42.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.850+0000 7f3c59f88640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c54108720 0x7f3c5419f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54574/0 (socket says 172.21.15.100:54574) 2024-07-26T13:43:42.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.850+0000 7f3c59787640 1 -- 172.21.15.100:0/3468692015 learned_addr learned my addr 172.21.15.100:0/3468692015 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:42.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.850+0000 7f3c58f86640 1 -- 172.21.15.100:0/3468692015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c54102490 msgr2=0x7f3c54199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:42.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.850+0000 7f3c58f86640 1 --2- 172.21.15.100:0/3468692015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c54102490 0x7f3c54199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:42.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.850+0000 7f3c58f86640 1 -- 172.21.15.100:0/3468692015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c54108720 msgr2=0x7f3c5419f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:42.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.850+0000 7f3c58f86640 1 --2- 172.21.15.100:0/3468692015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c54108720 0x7f3c5419f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:42.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.850+0000 7f3c58f86640 1 -- 172.21.15.100:0/3468692015 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c5419fe50 con 0x7f3c54107dd0 2024-07-26T13:43:42.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.850+0000 7f3c59787640 1 --2- 172.21.15.100:0/3468692015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c54102490 0x7f3c54199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:43:42.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.850+0000 7f3c59f88640 1 --2- 172.21.15.100:0/3468692015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c54108720 0x7f3c5419f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:43:42.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.851+0000 7f3c58f86640 1 --2- 172.21.15.100:0/3468692015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c54107dd0 0x7f3c5419a480 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3c44006010 tx=0x7f3c44004060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:42.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.851+0000 7f3c4a7fc640 1 -- 172.21.15.100:0/3468692015 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4403f070 con 0x7f3c54107dd0 2024-07-26T13:43:42.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.851+0000 7f3c5b9eb640 1 -- 172.21.15.100:0/3468692015 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c541a00e0 con 0x7f3c54107dd0 2024-07-26T13:43:42.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.851+0000 7f3c4a7fc640 1 -- 172.21.15.100:0/3468692015 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c44003ca0 con 0x7f3c54107dd0 2024-07-26T13:43:42.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.851+0000 7f3c4a7fc640 1 -- 172.21.15.100:0/3468692015 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c44035070 con 0x7f3c54107dd0 2024-07-26T13:43:42.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.851+0000 7f3c5b9eb640 1 -- 172.21.15.100:0/3468692015 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c541a68a0 con 0x7f3c54107dd0 2024-07-26T13:43:42.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.852+0000 7f3c5b9eb640 1 -- 172.21.15.100:0/3468692015 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c541028a0 con 0x7f3c54107dd0 2024-07-26T13:43:42.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.854+0000 7f3c4a7fc640 1 -- 172.21.15.100:0/3468692015 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f3c4403b460 con 0x7f3c54107dd0 2024-07-26T13:43:42.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.854+0000 7f3c4a7fc640 1 --2- 172.21.15.100:0/3468692015 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3c400b3d40 0x7f3c400b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:42.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.855+0000 7f3c59787640 1 --2- 172.21.15.100:0/3468692015 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3c400b3d40 0x7f3c400b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:42.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.855+0000 7f3c4a7fc640 1 -- 172.21.15.100:0/3468692015 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f3c44104650 con 0x7f3c54107dd0 2024-07-26T13:43:42.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.855+0000 7f3c59787640 1 --2- 172.21.15.100:0/3468692015 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3c400b3d40 0x7f3c400b6250 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3c500095f0 tx=0x7f3c50005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:42.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:42.857+0000 7f3c4a7fc640 1 -- 172.21.15.100:0/3468692015 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f3c440cb2e0 con 0x7f3c54107dd0 2024-07-26T13:43:43.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:43.083+0000 7f3c5b9eb640 1 -- 172.21.15.100:0/3468692015 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c5410a510 con 0x7f3c54107dd0 2024-07-26T13:43:43.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:43.084+0000 7f3c4a7fc640 1 -- 172.21.15.100:0/3468692015 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11932 (secure 0 0 0) 0x7f3c440cf830 con 0x7f3c54107dd0 2024-07-26T13:43:43.132 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:43:43.132 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":36,"btime":"2024-07-26T13:43:14:601533+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14790,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/313501409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":313501409},{"type":"v1","addr":"172.21.15.100:6839","nonce":313501409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/1402975506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":1402975506},{"type":"v1","addr":"172.21.15.100:6841","nonce":1402975506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-26T13:37:01.281843+0000","modified":"2024-07-26T13:43:14.601516+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24607,"mds_1":24605,"mds_2":24557},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24557":{"gid":24557,"name":"e","rank":2,"incarnation":30,"state":"up:active","state_seq":107,"addr":"172.21.15.138:6835/3290105300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3290105300},{"type":"v1","addr":"172.21.15.138:6835","nonce":3290105300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/3393225483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3393225483},{"type":"v1","addr":"172.21.15.138:6837","nonce":3393225483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.138:6839/1580925471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1580925471},{"type":"v1","addr":"172.21.15.138:6839","nonce":1580925471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/398150514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":398150514},{"type":"v1","addr":"172.21.15.167:6839","nonce":398150514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25762":{"gid":25762,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2476068386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2476068386},{"type":"v1","addr":"172.21.15.167:6837","nonce":2476068386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24607,"qdb_cluster":[24607,24605,24557]},"id":1}]} 2024-07-26T13:43:43.132 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 36 2024-07-26T13:43:43.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:43.086+0000 7f3c5b9eb640 1 -- 172.21.15.100:0/3468692015 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3c400b3d40 msgr2=0x7f3c400b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:43.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:43.086+0000 7f3c5b9eb640 1 --2- 172.21.15.100:0/3468692015 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3c400b3d40 0x7f3c400b6250 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3c500095f0 tx=0x7f3c50005f90 comp rx=0 tx=0).stop 2024-07-26T13:43:43.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:43.086+0000 7f3c5b9eb640 1 -- 172.21.15.100:0/3468692015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c54107dd0 msgr2=0x7f3c5419a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:43.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:43.086+0000 7f3c5b9eb640 1 --2- 172.21.15.100:0/3468692015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c54107dd0 0x7f3c5419a480 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3c44006010 tx=0x7f3c44004060 comp rx=0 tx=0).stop 2024-07-26T13:43:43.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:43.086+0000 7f3c5b9eb640 1 -- 172.21.15.100:0/3468692015 shutdown_connections 2024-07-26T13:43:43.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:43.086+0000 7f3c5b9eb640 1 --2- 172.21.15.100:0/3468692015 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3c400b3d40 0x7f3c400b6250 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:43.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:43.086+0000 7f3c5b9eb640 1 --2- 172.21.15.100:0/3468692015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3c54108720 0x7f3c5419f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:43.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:43.086+0000 7f3c5b9eb640 1 --2- 172.21.15.100:0/3468692015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c54107dd0 0x7f3c5419a480 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:43.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:43.086+0000 7f3c5b9eb640 1 --2- 172.21.15.100:0/3468692015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c54102490 0x7f3c54199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:43.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:43.086+0000 7f3c5b9eb640 1 -- 172.21.15.100:0/3468692015 >> 172.21.15.100:0/3468692015 conn(0x7f3c54079160 msgr2=0x7f3c54104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:43.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:43.086+0000 7f3c5b9eb640 1 -- 172.21.15.100:0/3468692015 shutdown_connections 2024-07-26T13:43:43.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:43.086+0000 7f3c5b9eb640 1 -- 172.21.15.100:0/3468692015 wait complete. 2024-07-26T13:43:43.411 INFO:tasks.workunit.client.0.smithi100.stdout: 1 140822 6.61 MB/sec warmup 117 sec latency 12.660 ms 2024-07-26T13:43:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:43 smithi100 ceph-mon[32599]: pgmap v473: 97 pgs: 97 active+clean; 58 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.5 MiB/s wr, 276 op/s 2024-07-26T13:43:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:43 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/3468692015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:43:43.927 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24607 quiesce db / --timeout 90 --expiration 151.1 --await-for 45.0' 2024-07-26T13:43:43.927 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- timeout 105 ceph tell mds.24607 quiesce db / --timeout 90 --expiration 151.1 --await-for 45.0 2024-07-26T13:43:43.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:43 smithi138 ceph-mon[36954]: pgmap v473: 97 pgs: 97 active+clean; 58 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.5 MiB/s wr, 276 op/s 2024-07-26T13:43:43.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:43 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/3468692015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:43:44.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:43 smithi167 ceph-mon[37140]: pgmap v473: 97 pgs: 97 active+clean; 58 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.5 MiB/s wr, 276 op/s 2024-07-26T13:43:44.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:43 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/3468692015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:43:44.202 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:43:44.409 INFO:tasks.workunit.client.0.smithi100.stdout: 1 142577 6.66 MB/sec warmup 118 sec latency 9.528 ms 2024-07-26T13:43:45.409 INFO:tasks.workunit.client.0.smithi100.stdout: 1 145119 6.72 MB/sec warmup 119 sec latency 12.430 ms 2024-07-26T13:43:45.628 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:45 smithi100 ceph-mon[32599]: pgmap v474: 97 pgs: 97 active+clean; 60 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.6 MiB/s wr, 287 op/s 2024-07-26T13:43:45.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:45 smithi138 ceph-mon[36954]: pgmap v474: 97 pgs: 97 active+clean; 60 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.6 MiB/s wr, 287 op/s 2024-07-26T13:43:46.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:45 smithi167 ceph-mon[37140]: pgmap v474: 97 pgs: 97 active+clean; 60 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.6 MiB/s wr, 287 op/s 2024-07-26T13:43:46.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.062+0000 7fc151623640 1 -- 172.21.15.100:0/1051503610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc14c1035b0 msgr2=0x7fc14c1039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:46.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.062+0000 7fc151623640 1 --2- 172.21.15.100:0/1051503610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc14c1035b0 0x7fc14c1039c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fc13c009900 tx=0x7fc13c02a410 comp rx=0 tx=0).stop 2024-07-26T13:43:46.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.062+0000 7fc151623640 1 -- 172.21.15.100:0/1051503610 shutdown_connections 2024-07-26T13:43:46.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.062+0000 7fc151623640 1 --2- 172.21.15.100:0/1051503610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc14c109840 0x7fc14c110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:46.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.062+0000 7fc151623640 1 --2- 172.21.15.100:0/1051503610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc14c108ef0 0x7fc14c109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:46.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.062+0000 7fc151623640 1 --2- 172.21.15.100:0/1051503610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc14c1035b0 0x7fc14c1039c0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:46.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.062+0000 7fc151623640 1 -- 172.21.15.100:0/1051503610 >> 172.21.15.100:0/1051503610 conn(0x7fc14c0ff020 msgr2=0x7fc14c101470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:46.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.062+0000 7fc151623640 1 -- 172.21.15.100:0/1051503610 shutdown_connections 2024-07-26T13:43:46.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.062+0000 7fc151623640 1 -- 172.21.15.100:0/1051503610 wait complete. 2024-07-26T13:43:46.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.063+0000 7fc151623640 1 Processor -- start 2024-07-26T13:43:46.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.063+0000 7fc151623640 1 -- start start 2024-07-26T13:43:46.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.064+0000 7fc151623640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc14c1035b0 0x7fc14c075140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:46.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.064+0000 7fc151623640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc14c108ef0 0x7fc14c075680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:46.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.064+0000 7fc151623640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc14c109840 0x7fc14c075bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:46.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.064+0000 7fc151623640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc14c113510 con 0x7fc14c109840 2024-07-26T13:43:46.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.064+0000 7fc151623640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc14c113390 con 0x7fc14c108ef0 2024-07-26T13:43:46.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.064+0000 7fc151623640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc14c113690 con 0x7fc14c1035b0 2024-07-26T13:43:46.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.064+0000 7fc14a7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc14c108ef0 0x7fc14c075680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:46.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.064+0000 7fc14affd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc14c1035b0 0x7fc14c075140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:46.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.064+0000 7fc14a7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc14c108ef0 0x7fc14c075680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:33606/0 (socket says 172.21.15.100:33606) 2024-07-26T13:43:46.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.064+0000 7fc14a7fc640 1 -- 172.21.15.100:0/3199017554 learned_addr learned my addr 172.21.15.100:0/3199017554 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:46.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.064+0000 7fc14b7fe640 1 --2- 172.21.15.100:0/3199017554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc14c109840 0x7fc14c075bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:46.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.064+0000 7fc14a7fc640 1 -- 172.21.15.100:0/3199017554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc14c1035b0 msgr2=0x7fc14c075140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:46.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.064+0000 7fc14a7fc640 1 --2- 172.21.15.100:0/3199017554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc14c1035b0 0x7fc14c075140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:46.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.064+0000 7fc14a7fc640 1 -- 172.21.15.100:0/3199017554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc14c109840 msgr2=0x7fc14c075bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:46.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.064+0000 7fc14a7fc640 1 --2- 172.21.15.100:0/3199017554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc14c109840 0x7fc14c075bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:46.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.064+0000 7fc14a7fc640 1 -- 172.21.15.100:0/3199017554 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc14c1a8180 con 0x7fc14c108ef0 2024-07-26T13:43:46.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.064+0000 7fc14affd640 1 --2- 172.21.15.100:0/3199017554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc14c1035b0 0x7fc14c075140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:43:46.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.064+0000 7fc14b7fe640 1 --2- 172.21.15.100:0/3199017554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc14c109840 0x7fc14c075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:43:46.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.064+0000 7fc14a7fc640 1 --2- 172.21.15.100:0/3199017554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc14c108ef0 0x7fc14c075680 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc14000e9f0 tx=0x7fc14000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:46.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.064+0000 7fc133fff640 1 -- 172.21.15.100:0/3199017554 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14000ccc0 con 0x7fc14c108ef0 2024-07-26T13:43:46.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.065+0000 7fc151623640 1 -- 172.21.15.100:0/3199017554 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc14c1a8470 con 0x7fc14c108ef0 2024-07-26T13:43:46.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.065+0000 7fc133fff640 1 -- 172.21.15.100:0/3199017554 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1400108e0 con 0x7fc14c108ef0 2024-07-26T13:43:46.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.065+0000 7fc133fff640 1 -- 172.21.15.100:0/3199017554 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc140018860 con 0x7fc14c108ef0 2024-07-26T13:43:46.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.065+0000 7fc151623640 1 -- 172.21.15.100:0/3199017554 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc14c1a8a00 con 0x7fc14c108ef0 2024-07-26T13:43:46.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.067+0000 7fc157ba7740 1 Processor -- start 2024-07-26T13:43:46.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.067+0000 7fc157ba7740 1 -- start start 2024-07-26T13:43:46.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.067+0000 7fc133fff640 1 -- 172.21.15.100:0/3199017554 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fc140010430 con 0x7fc14c108ef0 2024-07-26T13:43:46.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.067+0000 7fc157ba7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f936af840 0x559f93a55840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:46.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.067+0000 7fc157ba7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f93686980 0x559f93a55d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:46.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.067+0000 7fc157ba7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9368eb60 0x559f93a562c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:46.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.067+0000 7fc157ba7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x559f939ca780 con 0x559f9368eb60 2024-07-26T13:43:46.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.067+0000 7fc157ba7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559f939b9c80 con 0x559f93686980 2024-07-26T13:43:46.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.067+0000 7fc157ba7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559f939bdcc0 con 0x559f936af840 2024-07-26T13:43:46.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.068+0000 7fc14affd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f93686980 0x559f93a55d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:46.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.068+0000 7fc14b7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f936af840 0x559f93a55840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:46.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.068+0000 7fc14affd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f93686980 0x559f93a55d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:33608/0 (socket says 172.21.15.100:33608) 2024-07-26T13:43:46.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.068+0000 7fc14affd640 1 -- 172.21.15.100:0/2099654467 learned_addr learned my addr 172.21.15.100:0/2099654467 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:46.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.068+0000 7fc14b7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f936af840 0x559f93a55840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:52882/0 (socket says 172.21.15.100:52882) 2024-07-26T13:43:46.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.068+0000 7fc14a7fc640 1 --2- 172.21.15.100:0/2099654467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9368eb60 0x559f93a562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:46.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.068+0000 7fc14affd640 1 -- 172.21.15.100:0/2099654467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f936af840 msgr2=0x559f93a55840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:46.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.068+0000 7fc14affd640 1 --2- 172.21.15.100:0/2099654467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f936af840 0x559f93a55840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:46.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.068+0000 7fc14affd640 1 -- 172.21.15.100:0/2099654467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9368eb60 msgr2=0x559f93a562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:46.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.068+0000 7fc14affd640 1 --2- 172.21.15.100:0/2099654467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9368eb60 0x559f93a562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:46.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.068+0000 7fc14affd640 1 -- 172.21.15.100:0/2099654467 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f9394f080 con 0x559f93686980 2024-07-26T13:43:46.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.068+0000 7fc14b7fe640 1 --2- 172.21.15.100:0/2099654467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f936af840 0x559f93a55840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:43:46.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.068+0000 7fc14affd640 1 --2- 172.21.15.100:0/2099654467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f93686980 0x559f93a55d80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc13c02a930 tx=0x7fc13c005e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cecdda9d1e62ed00 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:46.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.068+0000 7fc133fff640 1 --2- 172.21.15.100:0/3199017554 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc1340b3d40 0x7fc1340b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:46.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.069+0000 7fc14b7fe640 1 --2- 172.21.15.100:0/3199017554 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc1340b3d40 0x7fc1340b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:46.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.069+0000 7fc133fff640 1 -- 172.21.15.100:0/3199017554 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7fc1400e42d0 con 0x7fc14c108ef0 2024-07-26T13:43:46.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.069+0000 7fc1317fa640 1 -- 172.21.15.100:0/2099654467 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c040070 con 0x559f93686980 2024-07-26T13:43:46.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.069+0000 7fc1317fa640 1 -- 172.21.15.100:0/2099654467 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc13c031720 con 0x559f93686980 2024-07-26T13:43:46.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.069+0000 7fc1317fa640 1 -- 172.21.15.100:0/2099654467 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c035040 con 0x559f93686980 2024-07-26T13:43:46.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.070+0000 7fc14b7fe640 1 --2- 172.21.15.100:0/3199017554 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc1340b3d40 0x7fc1340b6250 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc144009fd0 tx=0x7fc144009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:46.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.070+0000 7fc157ba7740 1 -- 172.21.15.100:0/2099654467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f93686980 msgr2=0x559f93a55d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:46.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.070+0000 7fc157ba7740 1 --2- 172.21.15.100:0/2099654467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f93686980 0x559f93a55d80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc13c02a930 tx=0x7fc13c005e20 comp rx=0 tx=0).stop 2024-07-26T13:43:46.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.070+0000 7fc157ba7740 1 -- 172.21.15.100:0/2099654467 shutdown_connections 2024-07-26T13:43:46.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.070+0000 7fc157ba7740 1 --2- 172.21.15.100:0/2099654467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9368eb60 0x559f93a562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:46.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.070+0000 7fc157ba7740 1 --2- 172.21.15.100:0/2099654467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f93686980 0x559f93a55d80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:46.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.070+0000 7fc157ba7740 1 --2- 172.21.15.100:0/2099654467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f936af840 0x559f93a55840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:46.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.070+0000 7fc157ba7740 1 -- 172.21.15.100:0/2099654467 >> 172.21.15.100:0/2099654467 conn(0x559f936a3990 msgr2=0x559f93a53730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:46.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.070+0000 7fc157ba7740 1 -- 172.21.15.100:0/2099654467 shutdown_connections 2024-07-26T13:43:46.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.070+0000 7fc157ba7740 1 -- 172.21.15.100:0/2099654467 wait complete. 2024-07-26T13:43:46.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.070+0000 7fc157ba7740 1 -- start start 2024-07-26T13:43:46.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.070+0000 7fc157ba7740 1 Processor -- start 2024-07-26T13:43:46.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.071+0000 7fc157ba7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f93686980 0x559f93a60d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:46.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.071+0000 7fc157ba7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9368eb60 0x559f93a61280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:46.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.071+0000 7fc157ba7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f936af840 0x559f93a617c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:46.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.071+0000 7fc157ba7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x559f9381d7a0 con 0x559f9368eb60 2024-07-26T13:43:46.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.071+0000 7fc157ba7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559f9395c2f0 con 0x559f93686980 2024-07-26T13:43:46.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.071+0000 7fc157ba7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559f93a63fc0 con 0x559f936af840 2024-07-26T13:43:46.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.071+0000 7fc157ba7740 10 client.0 _ll_register_callbacks cb 0x559f93966110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:43:46.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.071+0000 7fc14a7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9368eb60 0x559f93a61280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:46.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.071+0000 7fc14a7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9368eb60 0x559f93a61280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59502/0 (socket says 172.21.15.100:59502) 2024-07-26T13:43:46.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.071+0000 7fc14a7fc640 1 -- 172.21.15.100:0/1237032728 learned_addr learned my addr 172.21.15.100:0/1237032728 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:46.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.071+0000 7fc14affd640 1 --2- 172.21.15.100:0/1237032728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f93686980 0x559f93a60d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:46.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.071+0000 7fc14b7fe640 1 --2- 172.21.15.100:0/1237032728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f936af840 0x559f93a617c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:46.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.071+0000 7fc14a7fc640 1 -- 172.21.15.100:0/1237032728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f936af840 msgr2=0x559f93a617c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:46.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.071+0000 7fc14a7fc640 1 --2- 172.21.15.100:0/1237032728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f936af840 0x559f93a617c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:46.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.071+0000 7fc14a7fc640 1 -- 172.21.15.100:0/1237032728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f93686980 msgr2=0x559f93a60d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:46.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.071+0000 7fc14a7fc640 1 --2- 172.21.15.100:0/1237032728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f93686980 0x559f93a60d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:46.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.071+0000 7fc14a7fc640 1 -- 172.21.15.100:0/1237032728 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f93960480 con 0x559f9368eb60 2024-07-26T13:43:46.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.071+0000 7fc14a7fc640 1 --2- 172.21.15.100:0/1237032728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9368eb60 0x559f93a61280 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc1400e1a50 tx=0x7fc14000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de3c9b24c0ec3c25 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:46.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.071+0000 7fc12eff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:43:46.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.071+0000 7fc12eff5640 1 -- 172.21.15.100:0/1237032728 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1400aa8b0 con 0x559f9368eb60 2024-07-26T13:43:46.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.072+0000 7fc12eff5640 1 -- 172.21.15.100:0/1237032728 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7fc1400a71f0 con 0x559f9368eb60 2024-07-26T13:43:46.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.072+0000 7fc157ba7740 1 -- 172.21.15.100:0/1237032728 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc13c031720 con 0x559f9368eb60 2024-07-26T13:43:46.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.072+0000 7fc12eff5640 1 -- 172.21.15.100:0/1237032728 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc1400aa6d0 con 0x559f9368eb60 2024-07-26T13:43:46.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.072+0000 7fc157ba7740 10 client.16446 fetch_fsmap learned FSMap version 36 2024-07-26T13:43:46.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.072+0000 7fc157ba7740 1 -- 172.21.15.100:0/1237032728 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x559f93729b90 con 0x559f9368eb60 2024-07-26T13:43:46.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.072+0000 7fc12eff5640 1 -- 172.21.15.100:0/1237032728 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc13c031720 con 0x559f9368eb60 2024-07-26T13:43:46.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.072+0000 7fc12eff5640 1 -- 172.21.15.100:0/1237032728 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7fc1400a68d0 con 0x559f9368eb60 2024-07-26T13:43:46.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.072+0000 7fc157ba7740 10 client.16446 fetch_fsmap finished waiting for FSMap version 36 2024-07-26T13:43:46.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.072+0000 7fc157ba7740 10 client.16446 resolve_mds: validated gid 24607 aka daemon mds.i 2024-07-26T13:43:46.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.072+0000 7fc157ba7740 20 client.16446 populate_metadata read hostname 'smithi100' 2024-07-26T13:43:46.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.072+0000 7fc157ba7740 1 --2- 172.21.15.100:0/1237032728 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559f9366e1c0 0x559f93a72e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:46.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.072+0000 7fc157ba7740 4 client.16446 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:43:46.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.072+0000 7fc157ba7740 1 -- 172.21.15.100:0/1237032728 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f939cc6d0 con 0x559f9366e1c0 2024-07-26T13:43:46.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.072+0000 7fc14affd640 1 --2- 172.21.15.100:0/1237032728 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559f9366e1c0 0x559f93a72e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:46.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.073+0000 7fc12eff5640 10 client.16446 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:43:46.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.073+0000 7fc14affd640 1 --2- 172.21.15.100:0/1237032728 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559f9366e1c0 0x559f93a72e20 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=383cfa0e759670ad server_cookie=a0096dff1605fa0f in_seq=0 out_seq=0 2024-07-26T13:43:46.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.073+0000 7fc12eff5640 1 -- 172.21.15.100:0/1237032728 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559f939cc6d0 con 0x559f9366e1c0 2024-07-26T13:43:46.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.073+0000 7fc12eff5640 10 client.16446 handle_command_reply: tid=0 2024-07-26T13:43:46.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.073+0000 7fc157ba7740 1 client.16446 shutdown 2024-07-26T13:43:46.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.074+0000 7fc157ba7740 1 -- 172.21.15.100:0/1237032728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9368eb60 msgr2=0x559f93a61280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:46.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.074+0000 7fc157ba7740 1 --2- 172.21.15.100:0/1237032728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9368eb60 0x559f93a61280 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc1400e1a50 tx=0x7fc14000ce60 comp rx=0 tx=0).stop 2024-07-26T13:43:46.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.074+0000 7fc157ba7740 1 -- 172.21.15.100:0/1237032728 shutdown_connections 2024-07-26T13:43:46.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.074+0000 7fc157ba7740 1 --2- 172.21.15.100:0/1237032728 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559f9366e1c0 0x559f93a72e20 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:46.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.074+0000 7fc157ba7740 1 --2- 172.21.15.100:0/1237032728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f936af840 0x559f93a617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:46.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.074+0000 7fc157ba7740 1 --2- 172.21.15.100:0/1237032728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f9368eb60 0x559f93a61280 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:46.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.074+0000 7fc157ba7740 1 --2- 172.21.15.100:0/1237032728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f93686980 0x559f93a60d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:46.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.074+0000 7fc157ba7740 1 -- 172.21.15.100:0/1237032728 >> 172.21.15.100:0/1237032728 conn(0x559f936a3990 msgr2=0x559f93a589d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:46.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.074+0000 7fc12eff5640 0 client.16446 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:43:46.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.074+0000 7fc157ba7740 1 -- 172.21.15.100:0/1237032728 shutdown_connections 2024-07-26T13:43:46.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.074+0000 7fc157ba7740 1 -- 172.21.15.100:0/1237032728 wait complete. 2024-07-26T13:43:46.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.074+0000 7fc157ba7740 20 client.16446 trim_cache size 0 max 16384 2024-07-26T13:43:46.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.093+0000 7fc157ba7740 1 Processor -- start 2024-07-26T13:43:46.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.093+0000 7fc157ba7740 1 -- start start 2024-07-26T13:43:46.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.093+0000 7fc157ba7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f93686980 0x559f93a61200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:46.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.093+0000 7fc157ba7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f9368eb60 0x559f93a61740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:46.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.093+0000 7fc157ba7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f936af840 0x559f93a5a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:46.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.093+0000 7fc157ba7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc14c1b3cf0 con 0x559f936af840 2024-07-26T13:43:46.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.093+0000 7fc157ba7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc14c1b1560 con 0x559f93686980 2024-07-26T13:43:46.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.093+0000 7fc157ba7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc14c1af3b0 con 0x559f9368eb60 2024-07-26T13:43:46.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.093+0000 7fc14b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f936af840 0x559f93a5a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:46.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.093+0000 7fc14b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f936af840 0x559f93a5a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59506/0 (socket says 172.21.15.100:59506) 2024-07-26T13:43:46.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.093+0000 7fc14b7fe640 1 -- 172.21.15.100:0/1014920032 learned_addr learned my addr 172.21.15.100:0/1014920032 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:46.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.093+0000 7fc14affd640 1 --2- 172.21.15.100:0/1014920032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f93686980 0x559f93a61200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:46.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.093+0000 7fc14a7fc640 1 --2- 172.21.15.100:0/1014920032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f9368eb60 0x559f93a61740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:46.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.094+0000 7fc14affd640 1 -- 172.21.15.100:0/1014920032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f9368eb60 msgr2=0x559f93a61740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:46.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.094+0000 7fc14affd640 1 --2- 172.21.15.100:0/1014920032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f9368eb60 0x559f93a61740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:46.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.094+0000 7fc14affd640 1 -- 172.21.15.100:0/1014920032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f936af840 msgr2=0x559f93a5a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:46.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.094+0000 7fc14affd640 1 --2- 172.21.15.100:0/1014920032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f936af840 0x559f93a5a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:46.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.094+0000 7fc14affd640 1 -- 172.21.15.100:0/1014920032 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f93a57b30 con 0x559f93686980 2024-07-26T13:43:46.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.094+0000 7fc14b7fe640 1 --2- 172.21.15.100:0/1014920032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f936af840 0x559f93a5a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:43:46.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.094+0000 7fc14affd640 1 --2- 172.21.15.100:0/1014920032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f93686980 0x559f93a61200 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc13c005580 tx=0x7fc13c005480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c55c05d523a34b7 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:46.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.094+0000 7fc130ff9640 1 -- 172.21.15.100:0/1014920032 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c035040 con 0x559f93686980 2024-07-26T13:43:46.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.094+0000 7fc130ff9640 1 -- 172.21.15.100:0/1014920032 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc13c055020 con 0x559f93686980 2024-07-26T13:43:46.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.094+0000 7fc130ff9640 1 -- 172.21.15.100:0/1014920032 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c040040 con 0x559f93686980 2024-07-26T13:43:46.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.095+0000 7fc157ba7740 1 -- 172.21.15.100:0/1014920032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f93686980 msgr2=0x559f93a61200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:46.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.095+0000 7fc157ba7740 1 --2- 172.21.15.100:0/1014920032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f93686980 0x559f93a61200 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc13c005580 tx=0x7fc13c005480 comp rx=0 tx=0).stop 2024-07-26T13:43:46.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.095+0000 7fc157ba7740 1 -- 172.21.15.100:0/1014920032 shutdown_connections 2024-07-26T13:43:46.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.095+0000 7fc157ba7740 1 --2- 172.21.15.100:0/1014920032 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f936af840 0x559f93a5a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:46.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.095+0000 7fc157ba7740 1 --2- 172.21.15.100:0/1014920032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f9368eb60 0x559f93a61740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:46.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.095+0000 7fc157ba7740 1 --2- 172.21.15.100:0/1014920032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f93686980 0x559f93a61200 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:46.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.095+0000 7fc157ba7740 1 -- 172.21.15.100:0/1014920032 >> 172.21.15.100:0/1014920032 conn(0x559f936a3990 msgr2=0x559f93a60af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:46.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.095+0000 7fc157ba7740 1 -- 172.21.15.100:0/1014920032 shutdown_connections 2024-07-26T13:43:46.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.095+0000 7fc157ba7740 1 -- 172.21.15.100:0/1014920032 wait complete. 2024-07-26T13:43:46.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.095+0000 7fc157ba7740 1 -- start start 2024-07-26T13:43:46.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.096+0000 7fc157ba7740 1 Processor -- start 2024-07-26T13:43:46.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.096+0000 7fc157ba7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f93686980 0x559f93a650e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:46.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.096+0000 7fc157ba7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f9368eb60 0x559f93a65620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:46.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.096+0000 7fc157ba7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f936af840 0x559f93a65b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:46.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.096+0000 7fc14affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f93686980 0x559f93a650e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:46.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.096+0000 7fc157ba7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc14c1afa20 con 0x559f93686980 2024-07-26T13:43:46.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.096+0000 7fc14affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f93686980 0x559f93a650e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59514/0 (socket says 172.21.15.100:59514) 2024-07-26T13:43:46.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.096+0000 7fc157ba7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc14c1b0e90 con 0x559f936af840 2024-07-26T13:43:46.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.096+0000 7fc14affd640 1 -- 172.21.15.100:0/2728040367 learned_addr learned my addr 172.21.15.100:0/2728040367 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:46.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.096+0000 7fc157ba7740 1 -- 172.21.15.100:0/2728040367 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc14c1b00f0 con 0x559f9368eb60 2024-07-26T13:43:46.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.096+0000 7fc14a7fc640 1 --2- 172.21.15.100:0/2728040367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f9368eb60 0x559f93a65620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:46.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.096+0000 7fc14b7fe640 1 --2- 172.21.15.100:0/2728040367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f936af840 0x559f93a65b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:46.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.096+0000 7fc157ba7740 10 client.0 _ll_register_callbacks cb 0x7fc1400e7b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:43:46.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.096+0000 7fc14affd640 1 -- 172.21.15.100:0/2728040367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f9368eb60 msgr2=0x559f93a65620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:46.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.097+0000 7fc14affd640 1 --2- 172.21.15.100:0/2728040367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f9368eb60 0x559f93a65620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:46.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.097+0000 7fc14affd640 1 -- 172.21.15.100:0/2728040367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f936af840 msgr2=0x559f93a65b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:46.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.097+0000 7fc14affd640 1 --2- 172.21.15.100:0/2728040367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f936af840 0x559f93a65b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:46.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.097+0000 7fc14affd640 1 -- 172.21.15.100:0/2728040367 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f93ada5f0 con 0x559f93686980 2024-07-26T13:43:46.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.097+0000 7fc14affd640 1 --2- 172.21.15.100:0/2728040367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f93686980 0x559f93a650e0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc13c0318a0 tx=0x7fc13c031750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f3b5f2968dc202f server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:46.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.097+0000 7fc12eff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:43:46.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.097+0000 7fc12eff5640 1 -- 172.21.15.100:0/2728040367 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c060030 con 0x559f93686980 2024-07-26T13:43:46.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.097+0000 7fc12eff5640 1 -- 172.21.15.100:0/2728040367 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7fc13c004250 con 0x559f93686980 2024-07-26T13:43:46.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.097+0000 7fc157ba7740 1 -- 172.21.15.100:0/2728040367 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc13c055020 con 0x559f93686980 2024-07-26T13:43:46.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.097+0000 7fc12eff5640 1 -- 172.21.15.100:0/2728040367 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc13c06a050 con 0x559f93686980 2024-07-26T13:43:46.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.097+0000 7fc157ba7740 10 client.16452 fetch_fsmap learned FSMap version 36 2024-07-26T13:43:46.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.097+0000 7fc157ba7740 1 -- 172.21.15.100:0/2728040367 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x559f93adb800 con 0x559f93686980 2024-07-26T13:43:46.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.097+0000 7fc12eff5640 1 -- 172.21.15.100:0/2728040367 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc13c055020 con 0x559f93686980 2024-07-26T13:43:46.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.098+0000 7fc12eff5640 1 -- 172.21.15.100:0/2728040367 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7fc13c052460 con 0x559f93686980 2024-07-26T13:43:46.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.098+0000 7fc157ba7740 10 client.16452 fetch_fsmap finished waiting for FSMap version 36 2024-07-26T13:43:46.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.098+0000 7fc157ba7740 10 client.16452 resolve_mds: validated gid 24607 aka daemon mds.i 2024-07-26T13:43:46.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.098+0000 7fc157ba7740 20 client.16452 populate_metadata read hostname 'smithi100' 2024-07-26T13:43:46.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.098+0000 7fc157ba7740 1 --2- 172.21.15.100:0/2728040367 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559f93643570 0x559f93addc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:46.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.098+0000 7fc157ba7740 4 client.16452 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 151.1, "await_for": 45.0}] 2024-07-26T13:43:46.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.098+0000 7fc157ba7740 1 -- 172.21.15.100:0/2728040367 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 151.1, "await_for": 45.0}) -- 0x559f939c8170 con 0x559f93643570 2024-07-26T13:43:46.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.098+0000 7fc14a7fc640 1 --2- 172.21.15.100:0/2728040367 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559f93643570 0x559f93addc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:46.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.098+0000 7fc12eff5640 10 client.16452 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:43:46.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.098+0000 7fc14a7fc640 1 --2- 172.21.15.100:0/2728040367 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559f93643570 0x559f93addc20 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24f37fb048baa213 server_cookie=3af9fca3f88bb333 in_seq=0 out_seq=0 2024-07-26T13:43:46.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.157+0000 7fc12eff5640 1 -- 172.21.15.100:0/2728040367 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559f939c8170 con 0x559f93643570 2024-07-26T13:43:46.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.157+0000 7fc12eff5640 10 client.16452 handle_command_reply: tid=0 2024-07-26T13:43:46.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.157+0000 7fc157ba7740 1 client.16452 shutdown 2024-07-26T13:43:46.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.157+0000 7fc157ba7740 1 -- 172.21.15.100:0/2728040367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f93686980 msgr2=0x559f93a650e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:46.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.157+0000 7fc157ba7740 1 --2- 172.21.15.100:0/2728040367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f93686980 0x559f93a650e0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc13c0318a0 tx=0x7fc13c031750 comp rx=0 tx=0).stop 2024-07-26T13:43:46.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.157+0000 7fc157ba7740 1 -- 172.21.15.100:0/2728040367 shutdown_connections 2024-07-26T13:43:46.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.157+0000 7fc157ba7740 1 --2- 172.21.15.100:0/2728040367 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559f93643570 0x559f93addc20 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:46.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.157+0000 7fc157ba7740 1 --2- 172.21.15.100:0/2728040367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f936af840 0x559f93a65b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:46.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.157+0000 7fc12eff5640 0 client.16452 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:43:46.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.157+0000 7fc157ba7740 1 --2- 172.21.15.100:0/2728040367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f9368eb60 0x559f93a65620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:46.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.157+0000 7fc157ba7740 1 --2- 172.21.15.100:0/2728040367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559f93686980 0x559f93a650e0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:46.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.157+0000 7fc157ba7740 1 -- 172.21.15.100:0/2728040367 >> 172.21.15.100:0/2728040367 conn(0x559f936a3990 msgr2=0x559f93a614b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:46.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.157+0000 7fc157ba7740 1 -- 172.21.15.100:0/2728040367 shutdown_connections 2024-07-26T13:43:46.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.157+0000 7fc157ba7740 1 -- 172.21.15.100:0/2728040367 wait complete. 2024-07-26T13:43:46.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.157+0000 7fc157ba7740 20 client.16452 trim_cache size 0 max 16384 2024-07-26T13:43:46.162 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:43:46.163 INFO:teuthology.orchestra.run.smithi100.stdout: "epoch": 36, 2024-07-26T13:43:46.163 INFO:teuthology.orchestra.run.smithi100.stdout: "leader": 24607, 2024-07-26T13:43:46.163 INFO:teuthology.orchestra.run.smithi100.stdout: "set_version": 10, 2024-07-26T13:43:46.163 INFO:teuthology.orchestra.run.smithi100.stdout: "sets": { 2024-07-26T13:43:46.163 INFO:teuthology.orchestra.run.smithi100.stdout: "47f95d77": { 2024-07-26T13:43:46.163 INFO:teuthology.orchestra.run.smithi100.stdout: "version": 10, 2024-07-26T13:43:46.163 INFO:teuthology.orchestra.run.smithi100.stdout: "age_ref": 0.0, 2024-07-26T13:43:46.164 INFO:teuthology.orchestra.run.smithi100.stdout: "state": { 2024-07-26T13:43:46.164 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "QUIESCED", 2024-07-26T13:43:46.164 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 0.0 2024-07-26T13:43:46.164 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-07-26T13:43:46.164 INFO:teuthology.orchestra.run.smithi100.stdout: "timeout": 90.0, 2024-07-26T13:43:46.164 INFO:teuthology.orchestra.run.smithi100.stdout: "expiration": 151.1, 2024-07-26T13:43:46.164 INFO:teuthology.orchestra.run.smithi100.stdout: "members": { 2024-07-26T13:43:46.165 INFO:teuthology.orchestra.run.smithi100.stdout: "file:/": { 2024-07-26T13:43:46.165 INFO:teuthology.orchestra.run.smithi100.stdout: "excluded": false, 2024-07-26T13:43:46.165 INFO:teuthology.orchestra.run.smithi100.stdout: "state": { 2024-07-26T13:43:46.165 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "QUIESCED", 2024-07-26T13:43:46.165 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 0.0 2024-07-26T13:43:46.165 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:43:46.166 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:43:46.166 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:43:46.166 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:43:46.166 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:43:46.166 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:43:46.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.158+0000 7fc151623640 1 -- 172.21.15.100:0/3199017554 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc1340b3d40 msgr2=0x7fc1340b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:46.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.158+0000 7fc151623640 1 --2- 172.21.15.100:0/3199017554 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc1340b3d40 0x7fc1340b6250 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc144009fd0 tx=0x7fc144009500 comp rx=0 tx=0).stop 2024-07-26T13:43:46.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.158+0000 7fc151623640 1 -- 172.21.15.100:0/3199017554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc14c108ef0 msgr2=0x7fc14c075680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:46.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.158+0000 7fc151623640 1 --2- 172.21.15.100:0/3199017554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc14c108ef0 0x7fc14c075680 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc14000e9f0 tx=0x7fc14000eeb0 comp rx=0 tx=0).stop 2024-07-26T13:43:46.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.158+0000 7fc151623640 1 -- 172.21.15.100:0/3199017554 shutdown_connections 2024-07-26T13:43:46.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.158+0000 7fc151623640 1 --2- 172.21.15.100:0/3199017554 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc1340b3d40 0x7fc1340b6250 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:46.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.158+0000 7fc151623640 1 --2- 172.21.15.100:0/3199017554 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc14c109840 0x7fc14c075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:46.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.158+0000 7fc151623640 1 --2- 172.21.15.100:0/3199017554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc14c108ef0 0x7fc14c075680 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:46.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.158+0000 7fc151623640 1 --2- 172.21.15.100:0/3199017554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc14c1035b0 0x7fc14c075140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:46.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.158+0000 7fc151623640 1 -- 172.21.15.100:0/3199017554 >> 172.21.15.100:0/3199017554 conn(0x7fc14c0ff020 msgr2=0x7fc14c101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:46.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.158+0000 7fc151623640 1 -- 172.21.15.100:0/3199017554 shutdown_connections 2024-07-26T13:43:46.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:43:46.158+0000 7fc151623640 1 -- 172.21.15.100:0/3199017554 wait complete. 2024-07-26T13:43:47.290 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '47f95d77', quiesce took 6.6 seconds. Will release after: 24.5 2024-07-26T13:43:47.409 INFO:tasks.workunit.client.0.smithi100.stdout: 1 145991 0.00 MB/sec execute 1 sec latency 1309.211 ms 2024-07-26T13:43:47.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:47 smithi138 ceph-mon[36954]: pgmap v475: 97 pgs: 97 active+clean; 61 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 256 op/s 2024-07-26T13:43:47.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:47 smithi100 ceph-mon[32599]: pgmap v475: 97 pgs: 97 active+clean; 61 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 256 op/s 2024-07-26T13:43:48.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:47 smithi167 ceph-mon[37140]: pgmap v475: 97 pgs: 97 active+clean; 61 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.5 MiB/s wr, 256 op/s 2024-07-26T13:43:48.410 INFO:tasks.workunit.client.0.smithi100.stdout: 1 145991 0.00 MB/sec execute 2 sec latency 2309.292 ms 2024-07-26T13:43:49.410 INFO:tasks.workunit.client.0.smithi100.stdout: 1 145991 0.00 MB/sec execute 3 sec latency 3309.411 ms 2024-07-26T13:43:49.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:49 smithi167 ceph-mon[37140]: pgmap v476: 97 pgs: 97 active+clean; 65 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.7 MiB/s wr, 282 op/s 2024-07-26T13:43:49.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:49 smithi138 ceph-mon[36954]: pgmap v476: 97 pgs: 97 active+clean; 65 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.7 MiB/s wr, 282 op/s 2024-07-26T13:43:49.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:49 smithi100 ceph-mon[32599]: pgmap v476: 97 pgs: 97 active+clean; 65 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.7 MiB/s wr, 282 op/s 2024-07-26T13:43:50.410 INFO:tasks.workunit.client.0.smithi100.stdout: 1 145991 0.00 MB/sec execute 4 sec latency 4309.481 ms 2024-07-26T13:43:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:50 smithi138 ceph-mon[36954]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:43:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:50 smithi100 ceph-mon[32599]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:43:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:50 smithi167 ceph-mon[37140]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:43:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.071+0000 7ff2e7644640 1 -- 172.21.15.100:0/633374925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2e0105450 msgr2=0x7ff2e010eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.071+0000 7ff2e7644640 1 --2- 172.21.15.100:0/633374925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2e0105450 0x7ff2e010eec0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8009ca0 tx=0x7ff2d802a8c0 comp rx=0 tx=0).stop 2024-07-26T13:43:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.071+0000 7ff2e7644640 1 -- 172.21.15.100:0/633374925 shutdown_connections 2024-07-26T13:43:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.071+0000 7ff2e7644640 1 --2- 172.21.15.100:0/633374925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2e0105450 0x7ff2e010eec0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.071+0000 7ff2e7644640 1 --2- 172.21.15.100:0/633374925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2e0104b00 0x7ff2e0104f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.071+0000 7ff2e7644640 1 --2- 172.21.15.100:0/633374925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2e0102bd0 0x7ff2e0102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.071+0000 7ff2e7644640 1 -- 172.21.15.100:0/633374925 >> 172.21.15.100:0/633374925 conn(0x7ff2e00798f0 msgr2=0x7ff2e0079d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.071+0000 7ff2e7644640 1 -- 172.21.15.100:0/633374925 shutdown_connections 2024-07-26T13:43:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.071+0000 7ff2e7644640 1 -- 172.21.15.100:0/633374925 wait complete. 2024-07-26T13:43:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.072+0000 7ff2e7644640 1 Processor -- start 2024-07-26T13:43:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.072+0000 7ff2e7644640 1 -- start start 2024-07-26T13:43:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.072+0000 7ff2e7644640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2e0102bd0 0x7ff2e0198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.072+0000 7ff2e7644640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2e0104b00 0x7ff2e0199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.072+0000 7ff2e7644640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2e0105450 0x7ff2e019e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.072+0000 7ff2e7644640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e0112340 con 0x7ff2e0105450 2024-07-26T13:43:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.072+0000 7ff2e7644640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01121c0 con 0x7ff2e0102bd0 2024-07-26T13:43:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.072+0000 7ff2e7644640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01124c0 con 0x7ff2e0104b00 2024-07-26T13:43:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.072+0000 7ff2e5be1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2e0105450 0x7ff2e019e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.072+0000 7ff2e5be1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2e0105450 0x7ff2e019e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59530/0 (socket says 172.21.15.100:59530) 2024-07-26T13:43:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.072+0000 7ff2e5be1640 1 -- 172.21.15.100:0/1244785785 learned_addr learned my addr 172.21.15.100:0/1244785785 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.072+0000 7ff2e53e0640 1 --2- 172.21.15.100:0/1244785785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2e0102bd0 0x7ff2e0198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.072+0000 7ff2e4bdf640 1 --2- 172.21.15.100:0/1244785785 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2e0104b00 0x7ff2e0199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.072+0000 7ff2e5be1640 1 -- 172.21.15.100:0/1244785785 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2e0104b00 msgr2=0x7ff2e0199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.072+0000 7ff2e5be1640 1 --2- 172.21.15.100:0/1244785785 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2e0104b00 0x7ff2e0199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.072+0000 7ff2e5be1640 1 -- 172.21.15.100:0/1244785785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2e0102bd0 msgr2=0x7ff2e0198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.072+0000 7ff2e5be1640 1 --2- 172.21.15.100:0/1244785785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2e0102bd0 0x7ff2e0198ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.072+0000 7ff2e5be1640 1 -- 172.21.15.100:0/1244785785 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e019ec10 con 0x7ff2e0105450 2024-07-26T13:43:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.073+0000 7ff2e5be1640 1 --2- 172.21.15.100:0/1244785785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2e0105450 0x7ff2e019e510 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8005cc0 tx=0x7ff2d802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.073+0000 7ff2d67fc640 1 -- 172.21.15.100:0/1244785785 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8003bf0 con 0x7ff2e0105450 2024-07-26T13:43:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.073+0000 7ff2d67fc640 1 -- 172.21.15.100:0/1244785785 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2d8003d90 con 0x7ff2e0105450 2024-07-26T13:43:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.073+0000 7ff2d67fc640 1 -- 172.21.15.100:0/1244785785 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d802fbc0 con 0x7ff2e0105450 2024-07-26T13:43:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.073+0000 7ff2e7644640 1 -- 172.21.15.100:0/1244785785 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2e019eea0 con 0x7ff2e0105450 2024-07-26T13:43:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.073+0000 7ff2e7644640 1 -- 172.21.15.100:0/1244785785 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2e01a5690 con 0x7ff2e0105450 2024-07-26T13:43:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.075+0000 7ff2ee564740 1 Processor -- start 2024-07-26T13:43:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.076+0000 7ff2d67fc640 1 -- 172.21.15.100:0/1244785785 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7ff2d802fd60 con 0x7ff2e0105450 2024-07-26T13:43:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.076+0000 7ff2ee564740 1 -- start start 2024-07-26T13:43:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.076+0000 7ff2ee564740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569e99011b0 0x5569e9ca36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.076+0000 7ff2ee564740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569e98d82f0 0x5569e9ca3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.076+0000 7ff2ee564740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e98e04d0 0x5569e9ca4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.076+0000 7ff2ee564740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5569e9c193b0 con 0x5569e99011b0 2024-07-26T13:43:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.076+0000 7ff2ee564740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5569e9c088b0 con 0x5569e98e04d0 2024-07-26T13:43:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.076+0000 7ff2ee564740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5569e9c0c8f0 con 0x5569e98d82f0 2024-07-26T13:43:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.076+0000 7ff2e4bdf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569e99011b0 0x5569e9ca36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.076+0000 7ff2e4bdf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569e99011b0 0x5569e9ca36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59540/0 (socket says 172.21.15.100:59540) 2024-07-26T13:43:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.076+0000 7ff2e4bdf640 1 -- 172.21.15.100:0/3357852718 learned_addr learned my addr 172.21.15.100:0/3357852718 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.076+0000 7ff2e53e0640 1 --2- 172.21.15.100:0/3357852718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569e98d82f0 0x5569e9ca3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.076+0000 7ff2e4bdf640 1 --2- 172.21.15.100:0/3357852718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e98e04d0 0x5569e9ca4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.076+0000 7ff2e4bdf640 1 -- 172.21.15.100:0/3357852718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569e98d82f0 msgr2=0x5569e9ca3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.076+0000 7ff2e4bdf640 1 --2- 172.21.15.100:0/3357852718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569e98d82f0 0x5569e9ca3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.076+0000 7ff2e4bdf640 1 -- 172.21.15.100:0/3357852718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e98e04d0 msgr2=0x5569e9ca4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.076+0000 7ff2e4bdf640 1 --2- 172.21.15.100:0/3357852718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e98e04d0 0x5569e9ca4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.076+0000 7ff2d67fc640 1 --2- 172.21.15.100:0/1244785785 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff2cc0b3dc0 0x7ff2cc0b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.076+0000 7ff2e4bdf640 1 -- 172.21.15.100:0/3357852718 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569e9baf9a0 con 0x5569e99011b0 2024-07-26T13:43:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.077+0000 7ff2e5be1640 1 --2- 172.21.15.100:0/1244785785 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff2cc0b3dc0 0x7ff2cc0b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.077+0000 7ff2e4bdf640 1 --2- 172.21.15.100:0/3357852718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569e99011b0 0x5569e9ca36b0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff2d000b5b0 tx=0x7ff2d000b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a62b185c15f4fdd server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.077+0000 7ff2d67fc640 1 -- 172.21.15.100:0/1244785785 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7ff2d8105610 con 0x7ff2e0105450 2024-07-26T13:43:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.077+0000 7ff2cb7fe640 1 -- 172.21.15.100:0/3357852718 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0004030 con 0x5569e99011b0 2024-07-26T13:43:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.077+0000 7ff2cb7fe640 1 -- 172.21.15.100:0/3357852718 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2d00054d0 con 0x5569e99011b0 2024-07-26T13:43:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.077+0000 7ff2cb7fe640 1 -- 172.21.15.100:0/3357852718 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0004d00 con 0x5569e99011b0 2024-07-26T13:43:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.078+0000 7ff2ee564740 1 -- 172.21.15.100:0/3357852718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569e99011b0 msgr2=0x5569e9ca36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.078+0000 7ff2e5be1640 1 --2- 172.21.15.100:0/1244785785 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff2cc0b3dc0 0x7ff2cc0b62d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8102ef0 tx=0x7ff2d80987b0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.078+0000 7ff2ee564740 1 --2- 172.21.15.100:0/3357852718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569e99011b0 0x5569e9ca36b0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff2d000b5b0 tx=0x7ff2d000b820 comp rx=0 tx=0).stop 2024-07-26T13:43:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.078+0000 7ff2ee564740 1 -- 172.21.15.100:0/3357852718 shutdown_connections 2024-07-26T13:43:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.078+0000 7ff2ee564740 1 --2- 172.21.15.100:0/3357852718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e98e04d0 0x5569e9ca4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.078+0000 7ff2ee564740 1 --2- 172.21.15.100:0/3357852718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569e98d82f0 0x5569e9ca3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.078+0000 7ff2ee564740 1 --2- 172.21.15.100:0/3357852718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569e99011b0 0x5569e9ca36b0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.078+0000 7ff2ee564740 1 -- 172.21.15.100:0/3357852718 >> 172.21.15.100:0/3357852718 conn(0x5569e98f5300 msgr2=0x5569e9ca15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.078+0000 7ff2ee564740 1 -- 172.21.15.100:0/3357852718 shutdown_connections 2024-07-26T13:43:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.078+0000 7ff2ee564740 1 -- 172.21.15.100:0/3357852718 wait complete. 2024-07-26T13:43:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.078+0000 7ff2ee564740 1 -- start start 2024-07-26T13:43:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.078+0000 7ff2ee564740 1 Processor -- start 2024-07-26T13:43:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.079+0000 7ff2ee564740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569e98d82f0 0x5569e9caf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.079+0000 7ff2ee564740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e98e04d0 0x5569e9caf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.079+0000 7ff2ee564740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569e99011b0 0x5569e9cbb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.079+0000 7ff2ee564740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5569e9b42ab0 con 0x5569e99011b0 2024-07-26T13:43:51.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.079+0000 7ff2ee564740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5569e9b90cf0 con 0x5569e98e04d0 2024-07-26T13:43:51.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.079+0000 7ff2ee564740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5569e9cb1e30 con 0x5569e98d82f0 2024-07-26T13:43:51.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.079+0000 7ff2ee564740 10 client.0 _ll_register_callbacks cb 0x5569e9c023e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:43:51.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.079+0000 7ff2e4bdf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569e99011b0 0x5569e9cbb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:51.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.079+0000 7ff2e4bdf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569e99011b0 0x5569e9cbb120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59554/0 (socket says 172.21.15.100:59554) 2024-07-26T13:43:51.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.079+0000 7ff2e4bdf640 1 -- 172.21.15.100:0/1146280003 learned_addr learned my addr 172.21.15.100:0/1146280003 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:51.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.079+0000 7ff2e53e0640 1 --2- 172.21.15.100:0/1146280003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e98e04d0 0x5569e9caf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:51.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.079+0000 7ff2e4bdf640 1 --2- 172.21.15.100:0/1146280003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569e98d82f0 0x5569e9caf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:51.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.079+0000 7ff2e4bdf640 1 -- 172.21.15.100:0/1146280003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569e98d82f0 msgr2=0x5569e9caf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:51.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.079+0000 7ff2e4bdf640 1 --2- 172.21.15.100:0/1146280003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569e98d82f0 0x5569e9caf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:51.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.079+0000 7ff2e4bdf640 1 -- 172.21.15.100:0/1146280003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e98e04d0 msgr2=0x5569e9caf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:51.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.079+0000 7ff2e4bdf640 1 --2- 172.21.15.100:0/1146280003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e98e04d0 0x5569e9caf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:51.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.079+0000 7ff2e4bdf640 1 -- 172.21.15.100:0/1146280003 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569e996c790 con 0x5569e99011b0 2024-07-26T13:43:51.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.079+0000 7ff2e53e0640 1 --2- 172.21.15.100:0/1146280003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e98e04d0 0x5569e9caf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:43:51.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.080+0000 7ff2e4bdf640 1 --2- 172.21.15.100:0/1146280003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569e99011b0 0x5569e9cbb120 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x5569e9ca5b00 tx=0x7ff2d0004d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84e27df94ff128d3 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:51.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.080+0000 7ff2c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:43:51.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.080+0000 7ff2c8ff9640 1 -- 172.21.15.100:0/1146280003 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d001f020 con 0x5569e99011b0 2024-07-26T13:43:51.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.080+0000 7ff2c8ff9640 1 -- 172.21.15.100:0/1146280003 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7ff2d0010450 con 0x5569e99011b0 2024-07-26T13:43:51.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.080+0000 7ff2ee564740 1 -- 172.21.15.100:0/1146280003 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2d00054d0 con 0x5569e99011b0 2024-07-26T13:43:51.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.080+0000 7ff2c8ff9640 1 -- 172.21.15.100:0/1146280003 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff2d0032050 con 0x5569e99011b0 2024-07-26T13:43:51.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.080+0000 7ff2ee564740 10 client.16464 fetch_fsmap learned FSMap version 37 2024-07-26T13:43:51.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.080+0000 7ff2ee564740 1 -- 172.21.15.100:0/1146280003 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5569e99e5620 con 0x5569e99011b0 2024-07-26T13:43:51.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.080+0000 7ff2c8ff9640 1 -- 172.21.15.100:0/1146280003 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2d00054d0 con 0x5569e99011b0 2024-07-26T13:43:51.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.080+0000 7ff2c8ff9640 1 -- 172.21.15.100:0/1146280003 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7ff2d001b480 con 0x5569e99011b0 2024-07-26T13:43:51.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.080+0000 7ff2ee564740 10 client.16464 fetch_fsmap finished waiting for FSMap version 37 2024-07-26T13:43:51.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.080+0000 7ff2ee564740 10 client.16464 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:43:51.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.080+0000 7ff2ee564740 20 client.16464 populate_metadata read hostname 'smithi100' 2024-07-26T13:43:51.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.080+0000 7ff2ee564740 1 --2- 172.21.15.100:0/1146280003 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5569e98bfd80 0x5569e9cc0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:51.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.080+0000 7ff2ee564740 4 client.16464 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:43:51.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.080+0000 7ff2ee564740 1 -- 172.21.15.100:0/1146280003 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569e9c1b300 con 0x5569e98bfd80 2024-07-26T13:43:51.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.081+0000 7ff2e53e0640 1 --2- 172.21.15.100:0/1146280003 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5569e98bfd80 0x5569e9cc0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:51.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.081+0000 7ff2c8ff9640 10 client.16464 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:43:51.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.081+0000 7ff2e53e0640 1 --2- 172.21.15.100:0/1146280003 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5569e98bfd80 0x5569e9cc0fc0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9821fb85dd8aae54 server_cookie=da29225d1c7d37cc in_seq=0 out_seq=0 2024-07-26T13:43:51.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.082+0000 7ff2c8ff9640 1 -- 172.21.15.100:0/1146280003 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5569e9c1b300 con 0x5569e98bfd80 2024-07-26T13:43:51.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.082+0000 7ff2c8ff9640 10 client.16464 handle_command_reply: tid=0 2024-07-26T13:43:51.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.082+0000 7ff2ee564740 1 client.16464 shutdown 2024-07-26T13:43:51.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.082+0000 7ff2ee564740 1 -- 172.21.15.100:0/1146280003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569e99011b0 msgr2=0x5569e9cbb120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:51.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.082+0000 7ff2ee564740 1 --2- 172.21.15.100:0/1146280003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569e99011b0 0x5569e9cbb120 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x5569e9ca5b00 tx=0x7ff2d0004d60 comp rx=0 tx=0).stop 2024-07-26T13:43:51.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.082+0000 7ff2ee564740 1 -- 172.21.15.100:0/1146280003 shutdown_connections 2024-07-26T13:43:51.091 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.082+0000 7ff2ee564740 1 --2- 172.21.15.100:0/1146280003 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5569e98bfd80 0x5569e9cc0fc0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:51.091 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.082+0000 7ff2ee564740 1 --2- 172.21.15.100:0/1146280003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569e99011b0 0x5569e9cbb120 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:51.091 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.082+0000 7ff2c8ff9640 0 client.16464 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:43:51.091 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.082+0000 7ff2ee564740 1 --2- 172.21.15.100:0/1146280003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e98e04d0 0x5569e9caf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:51.091 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.082+0000 7ff2ee564740 1 --2- 172.21.15.100:0/1146280003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569e98d82f0 0x5569e9caf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:51.091 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.082+0000 7ff2ee564740 1 -- 172.21.15.100:0/1146280003 >> 172.21.15.100:0/1146280003 conn(0x5569e98f5300 msgr2=0x5569e9ca46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:51.091 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.082+0000 7ff2ee564740 1 -- 172.21.15.100:0/1146280003 shutdown_connections 2024-07-26T13:43:51.091 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.082+0000 7ff2ee564740 1 -- 172.21.15.100:0/1146280003 wait complete. 2024-07-26T13:43:51.091 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.082+0000 7ff2ee564740 20 client.16464 trim_cache size 0 max 16384 2024-07-26T13:43:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.097+0000 7ff2ee564740 1 Processor -- start 2024-07-26T13:43:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.097+0000 7ff2ee564740 1 -- start start 2024-07-26T13:43:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.097+0000 7ff2ee564740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569e98bfd80 0x5569e9ca4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.097+0000 7ff2ee564740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e98d82f0 0x5569e9ca9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.097+0000 7ff2ee564740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569e98e04d0 0x5569e9cae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.097+0000 7ff2ee564740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01b0920 con 0x5569e98e04d0 2024-07-26T13:43:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.097+0000 7ff2ee564740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01ae190 con 0x5569e98d82f0 2024-07-26T13:43:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.097+0000 7ff2ee564740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01abfe0 con 0x5569e98bfd80 2024-07-26T13:43:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.097+0000 7ff2e4bdf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569e98e04d0 0x5569e9cae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.097+0000 7ff2e4bdf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569e98e04d0 0x5569e9cae4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59560/0 (socket says 172.21.15.100:59560) 2024-07-26T13:43:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.097+0000 7ff2e4bdf640 1 -- 172.21.15.100:0/3561513209 learned_addr learned my addr 172.21.15.100:0/3561513209 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.097+0000 7ff2e4bdf640 1 --2- 172.21.15.100:0/3561513209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569e98bfd80 0x5569e9ca4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.097+0000 7ff2e53e0640 1 --2- 172.21.15.100:0/3561513209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e98d82f0 0x5569e9ca9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.098+0000 7ff2e4bdf640 1 -- 172.21.15.100:0/3561513209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569e98bfd80 msgr2=0x5569e9ca4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.098+0000 7ff2e4bdf640 1 --2- 172.21.15.100:0/3561513209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569e98bfd80 0x5569e9ca4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.098+0000 7ff2e4bdf640 1 -- 172.21.15.100:0/3561513209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e98d82f0 msgr2=0x5569e9ca9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.098+0000 7ff2e4bdf640 1 --2- 172.21.15.100:0/3561513209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e98d82f0 0x5569e9ca9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.098+0000 7ff2e4bdf640 1 -- 172.21.15.100:0/3561513209 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569e9ca64c0 con 0x5569e98e04d0 2024-07-26T13:43:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.098+0000 7ff2e53e0640 1 --2- 172.21.15.100:0/3561513209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e98d82f0 0x5569e9ca9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:43:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.098+0000 7ff2e4bdf640 1 --2- 172.21.15.100:0/3561513209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569e98e04d0 0x5569e9cae4c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0001eb0 tx=0x7ff2d0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e05a7840a5b37c69 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.098+0000 7ff2caffd640 1 -- 172.21.15.100:0/3561513209 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d001f050 con 0x5569e98e04d0 2024-07-26T13:43:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.098+0000 7ff2caffd640 1 -- 172.21.15.100:0/3561513209 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2d0023050 con 0x5569e98e04d0 2024-07-26T13:43:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.098+0000 7ff2caffd640 1 -- 172.21.15.100:0/3561513209 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0025070 con 0x5569e98e04d0 2024-07-26T13:43:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.099+0000 7ff2ee564740 1 -- 172.21.15.100:0/3561513209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569e98e04d0 msgr2=0x5569e9cae4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.099+0000 7ff2ee564740 1 --2- 172.21.15.100:0/3561513209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569e98e04d0 0x5569e9cae4c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0001eb0 tx=0x7ff2d0000bc0 comp rx=0 tx=0).stop 2024-07-26T13:43:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.099+0000 7ff2ee564740 1 -- 172.21.15.100:0/3561513209 shutdown_connections 2024-07-26T13:43:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.099+0000 7ff2ee564740 1 --2- 172.21.15.100:0/3561513209 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569e98e04d0 0x5569e9cae4c0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.099+0000 7ff2ee564740 1 --2- 172.21.15.100:0/3561513209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e98d82f0 0x5569e9ca9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.099+0000 7ff2ee564740 1 --2- 172.21.15.100:0/3561513209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569e98bfd80 0x5569e9ca4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.099+0000 7ff2ee564740 1 -- 172.21.15.100:0/3561513209 >> 172.21.15.100:0/3561513209 conn(0x5569e98f5300 msgr2=0x5569e9ca46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.099+0000 7ff2ee564740 1 -- 172.21.15.100:0/3561513209 shutdown_connections 2024-07-26T13:43:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.099+0000 7ff2ee564740 1 -- 172.21.15.100:0/3561513209 wait complete. 2024-07-26T13:43:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.099+0000 7ff2ee564740 1 -- start start 2024-07-26T13:43:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.099+0000 7ff2ee564740 1 Processor -- start 2024-07-26T13:43:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.100+0000 7ff2ee564740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569e98bfd80 0x5569e9cc4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.100+0000 7ff2ee564740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569e98d82f0 0x5569e9cc2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.100+0000 7ff2ee564740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e98e04d0 0x5569e9cc3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.100+0000 7ff2ee564740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01ac650 con 0x5569e98bfd80 2024-07-26T13:43:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.100+0000 7ff2ee564740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01adac0 con 0x5569e98e04d0 2024-07-26T13:43:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.100+0000 7ff2ee564740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01acd20 con 0x5569e98d82f0 2024-07-26T13:43:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.100+0000 7ff2ee564740 10 client.0 _ll_register_callbacks cb 0x7ff2dc00b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:43:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.100+0000 7ff2e4bdf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569e98bfd80 0x5569e9cc4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.100+0000 7ff2e4bdf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569e98bfd80 0x5569e9cc4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59572/0 (socket says 172.21.15.100:59572) 2024-07-26T13:43:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.100+0000 7ff2e4bdf640 1 -- 172.21.15.100:0/3171088543 learned_addr learned my addr 172.21.15.100:0/3171088543 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:43:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.100+0000 7ff2e4bdf640 1 --2- 172.21.15.100:0/3171088543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e98e04d0 0x5569e9cc3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.100+0000 7ff2e4bdf640 1 -- 172.21.15.100:0/3171088543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569e98d82f0 msgr2=0x5569e9cc2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.100+0000 7ff2e4bdf640 1 --2- 172.21.15.100:0/3171088543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569e98d82f0 0x5569e9cc2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.100+0000 7ff2e4bdf640 1 -- 172.21.15.100:0/3171088543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e98e04d0 msgr2=0x5569e9cc3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:43:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.100+0000 7ff2e4bdf640 1 --2- 172.21.15.100:0/3171088543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e98e04d0 0x5569e9cc3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.100+0000 7ff2e4bdf640 1 -- 172.21.15.100:0/3171088543 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569e9cc3790 con 0x5569e98bfd80 2024-07-26T13:43:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.100+0000 7ff2e4bdf640 1 --2- 172.21.15.100:0/3171088543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569e98bfd80 0x5569e9cc4640 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0004ef0 tx=0x7ff2d001b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aea07a1498e98ea server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:43:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.100+0000 7ff2c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:43:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.100+0000 7ff2c8ff9640 1 -- 172.21.15.100:0/3171088543 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0025050 con 0x5569e98bfd80 2024-07-26T13:43:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.100+0000 7ff2c8ff9640 1 -- 172.21.15.100:0/3171088543 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7ff2d001b3f0 con 0x5569e98bfd80 2024-07-26T13:43:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.100+0000 7ff2ee564740 1 -- 172.21.15.100:0/3171088543 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2d0023050 con 0x5569e98bfd80 2024-07-26T13:43:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.101+0000 7ff2c8ff9640 1 -- 172.21.15.100:0/3171088543 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff2d001f070 con 0x5569e98bfd80 2024-07-26T13:43:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.101+0000 7ff2ee564740 10 client.16470 fetch_fsmap learned FSMap version 37 2024-07-26T13:43:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.101+0000 7ff2ee564740 1 -- 172.21.15.100:0/3171088543 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5569e9d283d0 con 0x5569e98bfd80 2024-07-26T13:43:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.101+0000 7ff2c8ff9640 1 -- 172.21.15.100:0/3171088543 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2d0023050 con 0x5569e98bfd80 2024-07-26T13:43:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.101+0000 7ff2c8ff9640 1 -- 172.21.15.100:0/3171088543 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7ff2d0010a20 con 0x5569e98bfd80 2024-07-26T13:43:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.101+0000 7ff2ee564740 10 client.16470 fetch_fsmap finished waiting for FSMap version 37 2024-07-26T13:43:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.101+0000 7ff2ee564740 10 client.16470 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:43:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.101+0000 7ff2ee564740 20 client.16470 populate_metadata read hostname 'smithi100' 2024-07-26T13:43:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.101+0000 7ff2ee564740 1 --2- 172.21.15.100:0/3171088543 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5569e99011b0 0x5569e9d2aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:43:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.101+0000 7ff2ee564740 4 client.16470 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-26T13:43:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.101+0000 7ff2ee564740 1 -- 172.21.15.100:0/3171088543 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5569e9c16da0 con 0x5569e99011b0 2024-07-26T13:43:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.101+0000 7ff2e53e0640 1 --2- 172.21.15.100:0/3171088543 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5569e99011b0 0x5569e9d2aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:43:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.102+0000 7ff2c8ff9640 10 client.16470 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:43:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.102+0000 7ff2e53e0640 1 --2- 172.21.15.100:0/3171088543 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5569e99011b0 0x5569e9d2aae0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c0ce435b4840539 server_cookie=74c5b303ebc28b2e in_seq=0 out_seq=0 2024-07-26T13:43:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.136+0000 7ff2c8ff9640 1 -- 172.21.15.100:0/3171088543 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5569e9c16da0 con 0x5569e99011b0 2024-07-26T13:43:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.136+0000 7ff2c8ff9640 10 client.16470 handle_command_reply: tid=0 2024-07-26T13:43:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.136+0000 7ff2ee564740 1 client.16470 shutdown 2024-07-26T13:43:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.136+0000 7ff2ee564740 1 -- 172.21.15.100:0/3171088543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569e98bfd80 msgr2=0x5569e9cc4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.136+0000 7ff2ee564740 1 --2- 172.21.15.100:0/3171088543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569e98bfd80 0x5569e9cc4640 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0004ef0 tx=0x7ff2d001b900 comp rx=0 tx=0).stop 2024-07-26T13:43:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.136+0000 7ff2ee564740 1 -- 172.21.15.100:0/3171088543 shutdown_connections 2024-07-26T13:43:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.136+0000 7ff2ee564740 1 --2- 172.21.15.100:0/3171088543 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5569e99011b0 0x5569e9d2aae0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.136+0000 7ff2ee564740 1 --2- 172.21.15.100:0/3171088543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569e98e04d0 0x5569e9cc3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.136+0000 7ff2ee564740 1 --2- 172.21.15.100:0/3171088543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5569e98d82f0 0x5569e9cc2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.136+0000 7ff2c8ff9640 0 client.16470 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:43:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.136+0000 7ff2ee564740 1 --2- 172.21.15.100:0/3171088543 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5569e98bfd80 0x5569e9cc4640 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:51.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.136+0000 7ff2ee564740 1 -- 172.21.15.100:0/3171088543 >> 172.21.15.100:0/3171088543 conn(0x5569e98f5300 msgr2=0x5569e9cae4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:51.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.136+0000 7ff2ee564740 1 -- 172.21.15.100:0/3171088543 shutdown_connections 2024-07-26T13:43:51.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.137+0000 7ff2ee564740 1 -- 172.21.15.100:0/3171088543 wait complete. 2024-07-26T13:43:51.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.137+0000 7ff2ee564740 20 client.16470 trim_cache size 0 max 16384 2024-07-26T13:43:51.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-07-26T13:43:51.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-07-26T13:43:51.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-07-26T13:43:51.139 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-07-26T13:43:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.137+0000 7ff2e7644640 1 -- 172.21.15.100:0/1244785785 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff2cc0b3dc0 msgr2=0x7ff2cc0b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.137+0000 7ff2e7644640 1 --2- 172.21.15.100:0/1244785785 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff2cc0b3dc0 0x7ff2cc0b62d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8102ef0 tx=0x7ff2d80987b0 comp rx=0 tx=0).stop 2024-07-26T13:43:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.137+0000 7ff2e7644640 1 -- 172.21.15.100:0/1244785785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2e0105450 msgr2=0x7ff2e019e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:43:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.137+0000 7ff2e7644640 1 --2- 172.21.15.100:0/1244785785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2e0105450 0x7ff2e019e510 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8005cc0 tx=0x7ff2d802f380 comp rx=0 tx=0).stop 2024-07-26T13:43:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.137+0000 7ff2e7644640 1 -- 172.21.15.100:0/1244785785 shutdown_connections 2024-07-26T13:43:51.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.137+0000 7ff2e7644640 1 --2- 172.21.15.100:0/1244785785 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff2cc0b3dc0 0x7ff2cc0b62d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:51.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.137+0000 7ff2e7644640 1 --2- 172.21.15.100:0/1244785785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff2e0105450 0x7ff2e019e510 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:51.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.137+0000 7ff2e7644640 1 --2- 172.21.15.100:0/1244785785 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2e0104b00 0x7ff2e0199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:51.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.137+0000 7ff2e7644640 1 --2- 172.21.15.100:0/1244785785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2e0102bd0 0x7ff2e0198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:43:51.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.137+0000 7ff2e7644640 1 -- 172.21.15.100:0/1244785785 >> 172.21.15.100:0/1244785785 conn(0x7ff2e00798f0 msgr2=0x7ff2e0105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:43:51.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.137+0000 7ff2e7644640 1 -- 172.21.15.100:0/1244785785 shutdown_connections 2024-07-26T13:43:51.141 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:43:51.137+0000 7ff2e7644640 1 -- 172.21.15.100:0/1244785785 wait complete. 2024-07-26T13:43:51.410 INFO:tasks.workunit.client.0.smithi100.stdout: 1 145991 0.00 MB/sec execute 5 sec latency 5309.604 ms 2024-07-26T13:43:51.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:51 smithi138 ceph-mon[36954]: pgmap v477: 97 pgs: 97 active+clean; 65 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.6 MiB/s wr, 272 op/s 2024-07-26T13:43:51.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:51 smithi100 ceph-mon[32599]: pgmap v477: 97 pgs: 97 active+clean; 65 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.6 MiB/s wr, 272 op/s 2024-07-26T13:43:52.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:51 smithi167 ceph-mon[37140]: pgmap v477: 97 pgs: 97 active+clean; 65 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.6 MiB/s wr, 272 op/s 2024-07-26T13:43:52.410 INFO:tasks.workunit.client.0.smithi100.stdout: 1 145991 0.00 MB/sec execute 6 sec latency 6309.683 ms 2024-07-26T13:43:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:52 smithi138 ceph-mon[36954]: pgmap v478: 97 pgs: 97 active+clean; 66 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 182 op/s 2024-07-26T13:43:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:52 smithi100 ceph-mon[32599]: pgmap v478: 97 pgs: 97 active+clean; 66 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 182 op/s 2024-07-26T13:43:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:52 smithi167 ceph-mon[37140]: pgmap v478: 97 pgs: 97 active+clean; 66 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 182 op/s 2024-07-26T13:43:53.410 INFO:tasks.workunit.client.0.smithi100.stdout: 1 145991 0.00 MB/sec execute 7 sec latency 7309.775 ms 2024-07-26T13:43:54.410 INFO:tasks.workunit.client.0.smithi100.stdout: 1 145991 0.00 MB/sec execute 8 sec latency 8309.875 ms 2024-07-26T13:43:55.410 INFO:tasks.workunit.client.0.smithi100.stdout: 1 145991 0.00 MB/sec execute 9 sec latency 9309.974 ms 2024-07-26T13:43:55.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:55 smithi138 ceph-mon[36954]: pgmap v479: 97 pgs: 97 active+clean; 70 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 174 op/s 2024-07-26T13:43:55.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:55 smithi100 ceph-mon[32599]: pgmap v479: 97 pgs: 97 active+clean; 70 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 174 op/s 2024-07-26T13:43:56.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:55 smithi167 ceph-mon[37140]: pgmap v479: 97 pgs: 97 active+clean; 70 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 174 op/s 2024-07-26T13:43:56.410 INFO:tasks.workunit.client.0.smithi100.stdout: 1 145991 0.00 MB/sec execute 10 sec latency 10310.090 ms 2024-07-26T13:43:57.411 INFO:tasks.workunit.client.0.smithi100.stdout: 1 145991 0.00 MB/sec execute 11 sec latency 11310.136 ms 2024-07-26T13:43:57.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:57 smithi138 ceph-mon[36954]: pgmap v480: 97 pgs: 97 active+clean; 65 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 142 op/s 2024-07-26T13:43:57.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:57 smithi100 ceph-mon[32599]: pgmap v480: 97 pgs: 97 active+clean; 65 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 142 op/s 2024-07-26T13:43:58.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:57 smithi167 ceph-mon[37140]: pgmap v480: 97 pgs: 97 active+clean; 65 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 142 op/s 2024-07-26T13:43:58.410 INFO:tasks.workunit.client.0.smithi100.stdout: 1 145991 0.00 MB/sec execute 12 sec latency 12310.171 ms 2024-07-26T13:43:59.411 INFO:tasks.workunit.client.0.smithi100.stdout: 1 145991 0.00 MB/sec execute 13 sec latency 13310.256 ms 2024-07-26T13:43:59.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:43:59 smithi138 ceph-mon[36954]: pgmap v481: 97 pgs: 97 active+clean; 65 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 1.6 MiB/s wr, 127 op/s 2024-07-26T13:43:59.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:43:59 smithi100 ceph-mon[32599]: pgmap v481: 97 pgs: 97 active+clean; 65 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 1.6 MiB/s wr, 127 op/s 2024-07-26T13:44:00.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:43:59 smithi167 ceph-mon[37140]: pgmap v481: 97 pgs: 97 active+clean; 65 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 1.6 MiB/s wr, 127 op/s 2024-07-26T13:44:00.411 INFO:tasks.workunit.client.0.smithi100.stdout: 1 145991 0.00 MB/sec execute 14 sec latency 14310.305 ms 2024-07-26T13:44:01.411 INFO:tasks.workunit.client.0.smithi100.stdout: 1 145991 0.00 MB/sec execute 15 sec latency 15310.336 ms 2024-07-26T13:44:01.507 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-26T13:44:01.762 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:44:01.883 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:01 smithi100 ceph-mon[32599]: pgmap v482: 97 pgs: 97 active+clean; 64 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 1.5 MiB/s wr, 81 op/s 2024-07-26T13:44:01.883 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:44:01.883 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:44:01.884 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:44:01.884 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:44:01.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:01 smithi138 ceph-mon[36954]: pgmap v482: 97 pgs: 97 active+clean; 64 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 1.5 MiB/s wr, 81 op/s 2024-07-26T13:44:01.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:44:01.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:44:01.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:44:01.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:44:02.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:01 smithi167 ceph-mon[37140]: pgmap v482: 97 pgs: 97 active+clean; 64 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 1.5 MiB/s wr, 81 op/s 2024-07-26T13:44:02.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:44:02.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:44:02.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:44:02.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:44:02.411 INFO:tasks.workunit.client.0.smithi100.stdout: 1 145991 0.00 MB/sec execute 16 sec latency 16310.362 ms 2024-07-26T13:44:03.411 INFO:tasks.workunit.client.0.smithi100.stdout: 1 145991 0.00 MB/sec execute 17 sec latency 17310.457 ms 2024-07-26T13:44:03.730 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:03 smithi100 ceph-mon[32599]: pgmap v483: 97 pgs: 97 active+clean; 66 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 1.6 MiB/s wr, 71 op/s 2024-07-26T13:44:03.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.728+0000 7f6a429bf640 1 -- 172.21.15.100:0/677242149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a3c107e50 msgr2=0x7f6a3c108260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:03.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.728+0000 7f6a429bf640 1 --2- 172.21.15.100:0/677242149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a3c107e50 0x7f6a3c108260 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f6a30009900 tx=0x7f6a3002a410 comp rx=0 tx=0).stop 2024-07-26T13:44:03.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.729+0000 7f6a429bf640 1 -- 172.21.15.100:0/677242149 shutdown_connections 2024-07-26T13:44:03.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.729+0000 7f6a429bf640 1 --2- 172.21.15.100:0/677242149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a3c1087a0 0x7f6a3c10efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:03.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.729+0000 7f6a429bf640 1 --2- 172.21.15.100:0/677242149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a3c107e50 0x7f6a3c108260 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:03.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.729+0000 7f6a429bf640 1 --2- 172.21.15.100:0/677242149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a3c102510 0x7f6a3c102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:03.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.729+0000 7f6a429bf640 1 -- 172.21.15.100:0/677242149 >> 172.21.15.100:0/677242149 conn(0x7f6a3c0791a0 msgr2=0x7f6a3c0795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:03.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.729+0000 7f6a429bf640 1 -- 172.21.15.100:0/677242149 shutdown_connections 2024-07-26T13:44:03.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.729+0000 7f6a429bf640 1 -- 172.21.15.100:0/677242149 wait complete. 2024-07-26T13:44:03.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.729+0000 7f6a429bf640 1 Processor -- start 2024-07-26T13:44:03.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.729+0000 7f6a429bf640 1 -- start start 2024-07-26T13:44:03.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.730+0000 7f6a429bf640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a3c102510 0x7f6a3c199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:03.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.730+0000 7f6a429bf640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a3c107e50 0x7f6a3c19a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:03.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.730+0000 7f6a429bf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a3c1087a0 0x7f6a3c19f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:03.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.730+0000 7f6a429bf640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3c1124a0 con 0x7f6a3c1087a0 2024-07-26T13:44:03.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.730+0000 7f6a429bf640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3c112320 con 0x7f6a3c102510 2024-07-26T13:44:03.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.730+0000 7f6a429bf640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3c112620 con 0x7f6a3c107e50 2024-07-26T13:44:03.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.730+0000 7f6a40f5c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a3c1087a0 0x7f6a3c19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:03.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.730+0000 7f6a40f5c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a3c1087a0 0x7f6a3c19f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40332/0 (socket says 172.21.15.100:40332) 2024-07-26T13:44:03.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.730+0000 7f6a40f5c640 1 -- 172.21.15.100:0/856317266 learned_addr learned my addr 172.21.15.100:0/856317266 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:03.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.730+0000 7f6a3b7fe640 1 --2- 172.21.15.100:0/856317266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a3c107e50 0x7f6a3c19a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:03.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.730+0000 7f6a3bfff640 1 --2- 172.21.15.100:0/856317266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a3c102510 0x7f6a3c199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:03.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.730+0000 7f6a40f5c640 1 -- 172.21.15.100:0/856317266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a3c107e50 msgr2=0x7f6a3c19a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:03.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.730+0000 7f6a40f5c640 1 --2- 172.21.15.100:0/856317266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a3c107e50 0x7f6a3c19a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:03.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.730+0000 7f6a40f5c640 1 -- 172.21.15.100:0/856317266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a3c102510 msgr2=0x7f6a3c199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:03.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.730+0000 7f6a40f5c640 1 --2- 172.21.15.100:0/856317266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a3c102510 0x7f6a3c199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:03.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.730+0000 7f6a40f5c640 1 -- 172.21.15.100:0/856317266 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a3c19ff50 con 0x7f6a3c1087a0 2024-07-26T13:44:03.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.731+0000 7f6a40f5c640 1 --2- 172.21.15.100:0/856317266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a3c1087a0 0x7f6a3c19f820 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f6a3400cad0 tx=0x7f6a3400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:03.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.731+0000 7f6a397fa640 1 -- 172.21.15.100:0/856317266 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3400ad60 con 0x7f6a3c1087a0 2024-07-26T13:44:03.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.731+0000 7f6a397fa640 1 -- 172.21.15.100:0/856317266 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a34005020 con 0x7f6a3c1087a0 2024-07-26T13:44:03.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.731+0000 7f6a429bf640 1 -- 172.21.15.100:0/856317266 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a3c1a0240 con 0x7f6a3c1087a0 2024-07-26T13:44:03.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.731+0000 7f6a397fa640 1 -- 172.21.15.100:0/856317266 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3400f6b0 con 0x7f6a3c1087a0 2024-07-26T13:44:03.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.731+0000 7f6a429bf640 1 -- 172.21.15.100:0/856317266 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a3c1a69a0 con 0x7f6a3c1087a0 2024-07-26T13:44:03.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.733+0000 7f6a48f43740 1 Processor -- start 2024-07-26T13:44:03.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.734+0000 7f6a397fa640 1 -- 172.21.15.100:0/856317266 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f6a34004b20 con 0x7f6a3c1087a0 2024-07-26T13:44:03.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.734+0000 7f6a48f43740 1 -- start start 2024-07-26T13:44:03.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.734+0000 7f6a48f43740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556139e56070 0x55613a1fb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:03.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.734+0000 7f6a48f43740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556139e2d1b0 0x55613a1fba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:03.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.734+0000 7f6a48f43740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556139e35390 0x55613a1fbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:03.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.734+0000 7f6a48f43740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55613a171b60 con 0x556139e35390 2024-07-26T13:44:03.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.734+0000 7f6a48f43740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55613a161060 con 0x556139e56070 2024-07-26T13:44:03.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.734+0000 7f6a48f43740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55613a1650a0 con 0x556139e2d1b0 2024-07-26T13:44:03.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.734+0000 7f6a3b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556139e35390 0x55613a1fbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:03.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.734+0000 7f6a3b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556139e35390 0x55613a1fbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40336/0 (socket says 172.21.15.100:40336) 2024-07-26T13:44:03.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.734+0000 7f6a3b7fe640 1 -- 172.21.15.100:0/660743707 learned_addr learned my addr 172.21.15.100:0/660743707 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:03.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.734+0000 7f6a3b7fe640 1 --2- 172.21.15.100:0/660743707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556139e56070 0x55613a1fb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:03.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.734+0000 7f6a3bfff640 1 --2- 172.21.15.100:0/660743707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556139e2d1b0 0x55613a1fba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:03.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.734+0000 7f6a397fa640 1 --2- 172.21.15.100:0/856317266 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6a240b3d40 0x7f6a240b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:03.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.734+0000 7f6a3b7fe640 1 -- 172.21.15.100:0/660743707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556139e2d1b0 msgr2=0x55613a1fba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:03.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.734+0000 7f6a3b7fe640 1 --2- 172.21.15.100:0/660743707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556139e2d1b0 0x55613a1fba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:03.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.734+0000 7f6a3b7fe640 1 -- 172.21.15.100:0/660743707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556139e56070 msgr2=0x55613a1fb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:03.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.734+0000 7f6a3b7fe640 1 --2- 172.21.15.100:0/660743707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556139e56070 0x55613a1fb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:03.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.735+0000 7f6a3b7fe640 1 -- 172.21.15.100:0/660743707 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55613a107b40 con 0x556139e35390 2024-07-26T13:44:03.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.735+0000 7f6a40f5c640 1 --2- 172.21.15.100:0/856317266 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6a240b3d40 0x7f6a240b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:03.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.735+0000 7f6a3b7fe640 1 --2- 172.21.15.100:0/660743707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556139e35390 0x55613a1fbfa0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c06b3f0 tx=0x7f6a30000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d632ea1ef888f62a server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:03.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.735+0000 7f6a397fa640 1 -- 172.21.15.100:0/856317266 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f6a340e3f40 con 0x7f6a3c1087a0 2024-07-26T13:44:03.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.735+0000 7f6a227fc640 1 -- 172.21.15.100:0/660743707 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a30035070 con 0x556139e35390 2024-07-26T13:44:03.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.735+0000 7f6a227fc640 1 -- 172.21.15.100:0/660743707 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a300318e0 con 0x556139e35390 2024-07-26T13:44:03.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.735+0000 7f6a227fc640 1 -- 172.21.15.100:0/660743707 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a30003b20 con 0x556139e35390 2024-07-26T13:44:03.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.736+0000 7f6a48f43740 1 -- 172.21.15.100:0/660743707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556139e35390 msgr2=0x55613a1fbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:03.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.736+0000 7f6a48f43740 1 --2- 172.21.15.100:0/660743707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556139e35390 0x55613a1fbfa0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c06b3f0 tx=0x7f6a30000bc0 comp rx=0 tx=0).stop 2024-07-26T13:44:03.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.736+0000 7f6a40f5c640 1 --2- 172.21.15.100:0/856317266 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6a240b3d40 0x7f6a240b6250 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6a340e1710 tx=0x7f6a34076fd0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:03.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.736+0000 7f6a48f43740 1 -- 172.21.15.100:0/660743707 shutdown_connections 2024-07-26T13:44:03.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.736+0000 7f6a48f43740 1 --2- 172.21.15.100:0/660743707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556139e35390 0x55613a1fbfa0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:03.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.736+0000 7f6a48f43740 1 --2- 172.21.15.100:0/660743707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556139e2d1b0 0x55613a1fba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:03.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.736+0000 7f6a48f43740 1 --2- 172.21.15.100:0/660743707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556139e56070 0x55613a1fb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:03.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.736+0000 7f6a48f43740 1 -- 172.21.15.100:0/660743707 >> 172.21.15.100:0/660743707 conn(0x556139e4a460 msgr2=0x55613a1f9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:03.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.736+0000 7f6a48f43740 1 -- 172.21.15.100:0/660743707 shutdown_connections 2024-07-26T13:44:03.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.736+0000 7f6a48f43740 1 -- 172.21.15.100:0/660743707 wait complete. 2024-07-26T13:44:03.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.736+0000 7f6a48f43740 1 -- start start 2024-07-26T13:44:03.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.737+0000 7f6a48f43740 1 Processor -- start 2024-07-26T13:44:03.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.737+0000 7f6a48f43740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556139e2d1b0 0x55613a207130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:03.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.737+0000 7f6a48f43740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556139e35390 0x55613a2076b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:03.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.737+0000 7f6a48f43740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556139e56070 0x55613a212fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:03.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.737+0000 7f6a3b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556139e2d1b0 0x55613a207130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:03.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.737+0000 7f6a3b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556139e2d1b0 0x55613a207130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40340/0 (socket says 172.21.15.100:40340) 2024-07-26T13:44:03.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.737+0000 7f6a48f43740 1 -- 172.21.15.100:0/3480986480 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x556139fc4700 con 0x556139e2d1b0 2024-07-26T13:44:03.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.737+0000 7f6a3b7fe640 1 -- 172.21.15.100:0/3480986480 learned_addr learned my addr 172.21.15.100:0/3480986480 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:03.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.737+0000 7f6a48f43740 1 -- 172.21.15.100:0/3480986480 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55613a103990 con 0x556139e35390 2024-07-26T13:44:03.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.737+0000 7f6a48f43740 1 -- 172.21.15.100:0/3480986480 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55613a209ca0 con 0x556139e56070 2024-07-26T13:44:03.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.737+0000 7f6a48f43740 10 client.0 _ll_register_callbacks cb 0x55613a11e4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:44:03.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.738+0000 7f6a3b7fe640 1 -- 172.21.15.100:0/3480986480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556139e56070 msgr2=0x55613a212fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-26T13:44:03.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.738+0000 7f6a3b7fe640 1 --2- 172.21.15.100:0/3480986480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556139e56070 0x55613a212fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:03.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.738+0000 7f6a3b7fe640 1 -- 172.21.15.100:0/3480986480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556139e35390 msgr2=0x55613a2076b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:03.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.738+0000 7f6a3b7fe640 1 --2- 172.21.15.100:0/3480986480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556139e35390 0x55613a2076b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:03.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.738+0000 7f6a3b7fe640 1 -- 172.21.15.100:0/3480986480 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556139ec1650 con 0x556139e2d1b0 2024-07-26T13:44:03.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.738+0000 7f6a3b7fe640 1 --2- 172.21.15.100:0/3480986480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556139e2d1b0 0x55613a207130 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f6a30006010 tx=0x7f6a30031d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec5d10d17285d5a5 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:03.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.738+0000 7f6a1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:44:03.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.738+0000 7f6a1b7fe640 1 -- 172.21.15.100:0/3480986480 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3004e070 con 0x556139e2d1b0 2024-07-26T13:44:03.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.738+0000 7f6a1b7fe640 1 -- 172.21.15.100:0/3480986480 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f6a30032430 con 0x556139e2d1b0 2024-07-26T13:44:03.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.738+0000 7f6a48f43740 1 -- 172.21.15.100:0/3480986480 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a300318e0 con 0x556139e2d1b0 2024-07-26T13:44:03.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.739+0000 7f6a1b7fe640 1 -- 172.21.15.100:0/3480986480 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6a30035030 con 0x556139e2d1b0 2024-07-26T13:44:03.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.739+0000 7f6a48f43740 10 client.16482 fetch_fsmap learned FSMap version 37 2024-07-26T13:44:03.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.739+0000 7f6a48f43740 1 -- 172.21.15.100:0/3480986480 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55613a215d30 con 0x556139e2d1b0 2024-07-26T13:44:03.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.739+0000 7f6a1b7fe640 1 -- 172.21.15.100:0/3480986480 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a300035f0 con 0x556139e2d1b0 2024-07-26T13:44:03.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.739+0000 7f6a1b7fe640 1 -- 172.21.15.100:0/3480986480 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f6a3003d900 con 0x556139e2d1b0 2024-07-26T13:44:03.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.739+0000 7f6a48f43740 10 client.16482 fetch_fsmap finished waiting for FSMap version 37 2024-07-26T13:44:03.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.739+0000 7f6a48f43740 10 client.16482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:44:03.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.739+0000 7f6a48f43740 20 client.16482 populate_metadata read hostname 'smithi100' 2024-07-26T13:44:03.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.739+0000 7f6a48f43740 1 --2- 172.21.15.100:0/3480986480 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x556139e149f0 0x55613a219090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:03.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.739+0000 7f6a48f43740 4 client.16482 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:44:03.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.739+0000 7f6a48f43740 1 -- 172.21.15.100:0/3480986480 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6a300318e0 con 0x556139e149f0 2024-07-26T13:44:03.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.739+0000 7f6a3bfff640 1 --2- 172.21.15.100:0/3480986480 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x556139e149f0 0x55613a219090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:03.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.740+0000 7f6a1b7fe640 10 client.16482 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:03.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.740+0000 7f6a3bfff640 1 --2- 172.21.15.100:0/3480986480 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x556139e149f0 0x55613a219090 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef42765dabcbd715 server_cookie=51be751b16fc0540 in_seq=0 out_seq=0 2024-07-26T13:44:03.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.741+0000 7f6a1b7fe640 1 -- 172.21.15.100:0/3480986480 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f6a300318e0 con 0x556139e149f0 2024-07-26T13:44:03.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.741+0000 7f6a1b7fe640 10 client.16482 handle_command_reply: tid=0 2024-07-26T13:44:03.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.741+0000 7f6a48f43740 1 client.16482 shutdown 2024-07-26T13:44:03.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.741+0000 7f6a48f43740 1 -- 172.21.15.100:0/3480986480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556139e2d1b0 msgr2=0x55613a207130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:03.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.741+0000 7f6a48f43740 1 --2- 172.21.15.100:0/3480986480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556139e2d1b0 0x55613a207130 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f6a30006010 tx=0x7f6a30031d40 comp rx=0 tx=0).stop 2024-07-26T13:44:03.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.741+0000 7f6a48f43740 1 -- 172.21.15.100:0/3480986480 shutdown_connections 2024-07-26T13:44:03.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.741+0000 7f6a48f43740 1 --2- 172.21.15.100:0/3480986480 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x556139e149f0 0x55613a219090 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:03.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.741+0000 7f6a48f43740 1 --2- 172.21.15.100:0/3480986480 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556139e56070 0x55613a212fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:03.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.741+0000 7f6a48f43740 1 --2- 172.21.15.100:0/3480986480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556139e35390 0x55613a2076b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:03.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.741+0000 7f6a48f43740 1 --2- 172.21.15.100:0/3480986480 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556139e2d1b0 0x55613a207130 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:03.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.742+0000 7f6a48f43740 1 -- 172.21.15.100:0/3480986480 >> 172.21.15.100:0/3480986480 conn(0x556139e4a460 msgr2=0x55613a1fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:03.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.742+0000 7f6a1b7fe640 0 client.16482 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:03.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.742+0000 7f6a48f43740 1 -- 172.21.15.100:0/3480986480 shutdown_connections 2024-07-26T13:44:03.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.742+0000 7f6a48f43740 1 -- 172.21.15.100:0/3480986480 wait complete. 2024-07-26T13:44:03.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.742+0000 7f6a48f43740 20 client.16482 trim_cache size 0 max 16384 2024-07-26T13:44:03.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.759+0000 7f6a48f43740 1 Processor -- start 2024-07-26T13:44:03.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.759+0000 7f6a48f43740 1 -- start start 2024-07-26T13:44:03.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.759+0000 7f6a48f43740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556139e2d1b0 0x55613a212f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:03.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.759+0000 7f6a48f43740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556139e35390 0x55613a2020e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:03.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.759+0000 7f6a48f43740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556139e56070 0x55613a2005b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:03.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.759+0000 7f6a48f43740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3c1b1bc0 con 0x556139e2d1b0 2024-07-26T13:44:03.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.759+0000 7f6a48f43740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3c1af430 con 0x556139e56070 2024-07-26T13:44:03.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.759+0000 7f6a48f43740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3c1ad280 con 0x556139e35390 2024-07-26T13:44:03.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.759+0000 7f6a3b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556139e2d1b0 0x55613a212f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:03.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.759+0000 7f6a3b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556139e2d1b0 0x55613a212f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40348/0 (socket says 172.21.15.100:40348) 2024-07-26T13:44:03.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.759+0000 7f6a3b7fe640 1 -- 172.21.15.100:0/3146699302 learned_addr learned my addr 172.21.15.100:0/3146699302 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:03.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.759+0000 7f6a3bfff640 1 --2- 172.21.15.100:0/3146699302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556139e35390 0x55613a2020e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:03.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.759+0000 7f6a3b7fe640 1 -- 172.21.15.100:0/3146699302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556139e35390 msgr2=0x55613a2020e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:03.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.759+0000 7f6a3b7fe640 1 --2- 172.21.15.100:0/3146699302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556139e35390 0x55613a2020e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:03.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.759+0000 7f6a3b7fe640 1 -- 172.21.15.100:0/3146699302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556139e56070 msgr2=0x55613a2005b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:03.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.759+0000 7f6a3b7fe640 1 --2- 172.21.15.100:0/3146699302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556139e56070 0x55613a2005b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:03.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.759+0000 7f6a3b7fe640 1 -- 172.21.15.100:0/3146699302 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55613a2081f0 con 0x556139e2d1b0 2024-07-26T13:44:03.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.759+0000 7f6a3b7fe640 1 --2- 172.21.15.100:0/3146699302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556139e2d1b0 0x55613a212f30 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f6a300096c0 tx=0x7f6a3003fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ead1820b16bdb8 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:03.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.760+0000 7f6a21ffb640 1 -- 172.21.15.100:0/3146699302 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3004e070 con 0x556139e2d1b0 2024-07-26T13:44:03.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.760+0000 7f6a21ffb640 1 -- 172.21.15.100:0/3146699302 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a3005d070 con 0x556139e2d1b0 2024-07-26T13:44:03.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.760+0000 7f6a21ffb640 1 -- 172.21.15.100:0/3146699302 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a30004a00 con 0x556139e2d1b0 2024-07-26T13:44:03.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.760+0000 7f6a48f43740 1 -- 172.21.15.100:0/3146699302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556139e2d1b0 msgr2=0x55613a212f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:03.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.760+0000 7f6a48f43740 1 --2- 172.21.15.100:0/3146699302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556139e2d1b0 0x55613a212f30 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f6a300096c0 tx=0x7f6a3003fec0 comp rx=0 tx=0).stop 2024-07-26T13:44:03.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.760+0000 7f6a48f43740 1 -- 172.21.15.100:0/3146699302 shutdown_connections 2024-07-26T13:44:03.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.760+0000 7f6a48f43740 1 --2- 172.21.15.100:0/3146699302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556139e56070 0x55613a2005b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:03.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.760+0000 7f6a48f43740 1 --2- 172.21.15.100:0/3146699302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556139e35390 0x55613a2020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:03.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.760+0000 7f6a48f43740 1 --2- 172.21.15.100:0/3146699302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556139e2d1b0 0x55613a212f30 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:03.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.760+0000 7f6a48f43740 1 -- 172.21.15.100:0/3146699302 >> 172.21.15.100:0/3146699302 conn(0x556139e4a460 msgr2=0x55613a2057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:03.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.761+0000 7f6a48f43740 1 -- 172.21.15.100:0/3146699302 shutdown_connections 2024-07-26T13:44:03.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.761+0000 7f6a48f43740 1 -- 172.21.15.100:0/3146699302 wait complete. 2024-07-26T13:44:03.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.761+0000 7f6a48f43740 1 -- start start 2024-07-26T13:44:03.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.761+0000 7f6a48f43740 1 Processor -- start 2024-07-26T13:44:03.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.761+0000 7f6a48f43740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556139e2d1b0 0x55613a21c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:03.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.761+0000 7f6a48f43740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556139e35390 0x55613a217ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:03.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.761+0000 7f6a48f43740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556139e56070 0x55613a218240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:03.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.761+0000 7f6a48f43740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3c1ad8f0 con 0x556139e35390 2024-07-26T13:44:03.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.761+0000 7f6a48f43740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3c1aed60 con 0x556139e2d1b0 2024-07-26T13:44:03.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.761+0000 7f6a48f43740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3c1adfc0 con 0x556139e56070 2024-07-26T13:44:03.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.761+0000 7f6a48f43740 10 client.0 _ll_register_callbacks cb 0x7f6a30032ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:44:03.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.762+0000 7f6a3bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556139e35390 0x55613a217ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:03.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.762+0000 7f6a3bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556139e35390 0x55613a217ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40362/0 (socket says 172.21.15.100:40362) 2024-07-26T13:44:03.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.762+0000 7f6a3bfff640 1 -- 172.21.15.100:0/1871171865 learned_addr learned my addr 172.21.15.100:0/1871171865 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:03.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.762+0000 7f6a3b7fe640 1 --2- 172.21.15.100:0/1871171865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556139e56070 0x55613a218240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:03.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.762+0000 7f6a3bfff640 1 -- 172.21.15.100:0/1871171865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556139e56070 msgr2=0x55613a218240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:03.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.762+0000 7f6a3bfff640 1 --2- 172.21.15.100:0/1871171865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556139e56070 0x55613a218240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:03.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.762+0000 7f6a3bfff640 1 -- 172.21.15.100:0/1871171865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556139e2d1b0 msgr2=0x55613a21c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:03.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.762+0000 7f6a3bfff640 1 --2- 172.21.15.100:0/1871171865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556139e2d1b0 0x55613a21c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:03.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.762+0000 7f6a3bfff640 1 -- 172.21.15.100:0/1871171865 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55613a2189d0 con 0x556139e35390 2024-07-26T13:44:03.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.762+0000 7f6a3b7fe640 1 --2- 172.21.15.100:0/1871171865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556139e56070 0x55613a218240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:44:03.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.762+0000 7f6a3bfff640 1 --2- 172.21.15.100:0/1871171865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556139e35390 0x55613a217ca0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c006aa0 tx=0x7f6a2c009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c516b85498e7e094 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:03.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.762+0000 7f6a1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:44:03.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.762+0000 7f6a1b7fe640 1 -- 172.21.15.100:0/1871171865 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c015cc0 con 0x556139e35390 2024-07-26T13:44:03.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.762+0000 7f6a1b7fe640 1 -- 172.21.15.100:0/1871171865 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f6a2c024480 con 0x556139e35390 2024-07-26T13:44:03.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.762+0000 7f6a48f43740 1 -- 172.21.15.100:0/1871171865 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a3005d070 con 0x556139e35390 2024-07-26T13:44:03.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.762+0000 7f6a1b7fe640 1 -- 172.21.15.100:0/1871171865 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6a2c015e60 con 0x556139e35390 2024-07-26T13:44:03.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.762+0000 7f6a48f43740 10 client.16488 fetch_fsmap learned FSMap version 37 2024-07-26T13:44:03.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.762+0000 7f6a48f43740 1 -- 172.21.15.100:0/1871171865 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55613a285a30 con 0x556139e35390 2024-07-26T13:44:03.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.763+0000 7f6a1b7fe640 1 -- 172.21.15.100:0/1871171865 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a3005d070 con 0x556139e35390 2024-07-26T13:44:03.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.763+0000 7f6a1b7fe640 1 -- 172.21.15.100:0/1871171865 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f6a2c02b070 con 0x556139e35390 2024-07-26T13:44:03.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.763+0000 7f6a48f43740 10 client.16488 fetch_fsmap finished waiting for FSMap version 37 2024-07-26T13:44:03.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.763+0000 7f6a48f43740 10 client.16488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:44:03.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.763+0000 7f6a48f43740 20 client.16488 populate_metadata read hostname 'smithi100' 2024-07-26T13:44:03.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.763+0000 7f6a48f43740 1 --2- 172.21.15.100:0/1871171865 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x556139de9da0 0x55613a288230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:03.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.763+0000 7f6a48f43740 4 client.16488 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-26T13:44:03.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.763+0000 7f6a48f43740 1 -- 172.21.15.100:0/1871171865 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55613a173ab0 con 0x556139de9da0 2024-07-26T13:44:03.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.763+0000 7f6a3b7fe640 1 --2- 172.21.15.100:0/1871171865 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x556139de9da0 0x55613a288230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:03.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.764+0000 7f6a1b7fe640 10 client.16488 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:03.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.764+0000 7f6a3b7fe640 1 --2- 172.21.15.100:0/1871171865 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x556139de9da0 0x55613a288230 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b698435fb280623d server_cookie=969fe6145cf76579 in_seq=0 out_seq=0 2024-07-26T13:44:03.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.764+0000 7f6a1b7fe640 1 -- 172.21.15.100:0/1871171865 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55613a173ab0 con 0x556139de9da0 2024-07-26T13:44:03.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.764+0000 7f6a1b7fe640 10 client.16488 handle_command_reply: tid=0 2024-07-26T13:44:03.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.764+0000 7f6a48f43740 1 client.16488 shutdown 2024-07-26T13:44:03.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.764+0000 7f6a48f43740 1 -- 172.21.15.100:0/1871171865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556139e35390 msgr2=0x55613a217ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:03.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.764+0000 7f6a48f43740 1 --2- 172.21.15.100:0/1871171865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556139e35390 0x55613a217ca0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c006aa0 tx=0x7f6a2c009f50 comp rx=0 tx=0).stop 2024-07-26T13:44:03.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.764+0000 7f6a48f43740 1 -- 172.21.15.100:0/1871171865 shutdown_connections 2024-07-26T13:44:03.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.764+0000 7f6a48f43740 1 --2- 172.21.15.100:0/1871171865 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x556139de9da0 0x55613a288230 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:03.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.764+0000 7f6a48f43740 1 --2- 172.21.15.100:0/1871171865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556139e56070 0x55613a218240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:03.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.764+0000 7f6a48f43740 1 --2- 172.21.15.100:0/1871171865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556139e35390 0x55613a217ca0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:03.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.764+0000 7f6a48f43740 1 --2- 172.21.15.100:0/1871171865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556139e2d1b0 0x55613a21c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:03.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.764+0000 7f6a1b7fe640 0 client.16488 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:03.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.764+0000 7f6a48f43740 1 -- 172.21.15.100:0/1871171865 >> 172.21.15.100:0/1871171865 conn(0x556139e4a460 msgr2=0x55613a1fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:03.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.764+0000 7f6a48f43740 1 -- 172.21.15.100:0/1871171865 shutdown_connections 2024-07-26T13:44:03.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.764+0000 7f6a48f43740 1 -- 172.21.15.100:0/1871171865 wait complete. 2024-07-26T13:44:03.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.764+0000 7f6a48f43740 20 client.16488 trim_cache size 0 max 16384 2024-07-26T13:44:03.772 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:44:03.773 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-07-26T13:44:03.773 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-07-26T13:44:03.773 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:44:03.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.765+0000 7f6a429bf640 1 -- 172.21.15.100:0/856317266 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6a240b3d40 msgr2=0x7f6a240b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:03.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.765+0000 7f6a429bf640 1 --2- 172.21.15.100:0/856317266 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6a240b3d40 0x7f6a240b6250 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6a340e1710 tx=0x7f6a34076fd0 comp rx=0 tx=0).stop 2024-07-26T13:44:03.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.765+0000 7f6a429bf640 1 -- 172.21.15.100:0/856317266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a3c1087a0 msgr2=0x7f6a3c19f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:03.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.765+0000 7f6a429bf640 1 --2- 172.21.15.100:0/856317266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a3c1087a0 0x7f6a3c19f820 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f6a3400cad0 tx=0x7f6a3400ced0 comp rx=0 tx=0).stop 2024-07-26T13:44:03.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.765+0000 7f6a429bf640 1 -- 172.21.15.100:0/856317266 shutdown_connections 2024-07-26T13:44:03.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.765+0000 7f6a429bf640 1 --2- 172.21.15.100:0/856317266 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6a240b3d40 0x7f6a240b6250 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:03.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.765+0000 7f6a429bf640 1 --2- 172.21.15.100:0/856317266 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a3c1087a0 0x7f6a3c19f820 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:03.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.765+0000 7f6a429bf640 1 --2- 172.21.15.100:0/856317266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a3c107e50 0x7f6a3c19a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:03.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.765+0000 7f6a429bf640 1 --2- 172.21.15.100:0/856317266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a3c102510 0x7f6a3c199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:03.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.765+0000 7f6a429bf640 1 -- 172.21.15.100:0/856317266 >> 172.21.15.100:0/856317266 conn(0x7f6a3c0791a0 msgr2=0x7f6a3c101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:03.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.765+0000 7f6a429bf640 1 -- 172.21.15.100:0/856317266 shutdown_connections 2024-07-26T13:44:03.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:03.765+0000 7f6a429bf640 1 -- 172.21.15.100:0/856317266 wait complete. 2024-07-26T13:44:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:03 smithi138 ceph-mon[36954]: pgmap v483: 97 pgs: 97 active+clean; 66 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 1.6 MiB/s wr, 71 op/s 2024-07-26T13:44:04.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:03 smithi167 ceph-mon[37140]: pgmap v483: 97 pgs: 97 active+clean; 66 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 1.6 MiB/s wr, 71 op/s 2024-07-26T13:44:04.240 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.237+0000 7efffb559640 1 -- 172.21.15.100:0/660326048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efff4108690 msgr2=0x7efff410eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:04.240 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.237+0000 7efffb559640 1 --2- 172.21.15.100:0/660326048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efff4108690 0x7efff410eec0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7effec009ca0 tx=0x7effec02a8c0 comp rx=0 tx=0).stop 2024-07-26T13:44:04.240 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.237+0000 7efffb559640 1 -- 172.21.15.100:0/660326048 shutdown_connections 2024-07-26T13:44:04.240 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.237+0000 7efffb559640 1 --2- 172.21.15.100:0/660326048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efff4108690 0x7efff410eec0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:04.240 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.237+0000 7efffb559640 1 --2- 172.21.15.100:0/660326048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efff4107d40 0x7efff4108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:04.240 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.237+0000 7efffb559640 1 --2- 172.21.15.100:0/660326048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efff4102410 0x7efff4102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.237+0000 7efffb559640 1 -- 172.21.15.100:0/660326048 >> 172.21.15.100:0/660326048 conn(0x7efff40fddf0 msgr2=0x7efff4100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.237+0000 7efffb559640 1 -- 172.21.15.100:0/660326048 shutdown_connections 2024-07-26T13:44:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.237+0000 7efffb559640 1 -- 172.21.15.100:0/660326048 wait complete. 2024-07-26T13:44:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.237+0000 7efffb559640 1 Processor -- start 2024-07-26T13:44:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.237+0000 7efffb559640 1 -- start start 2024-07-26T13:44:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.238+0000 7efffb559640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efff4102410 0x7efff419ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.238+0000 7efffb559640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efff4107d40 0x7efff419b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.238+0000 7efffb559640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efff4108690 0x7efff419b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.238+0000 7efffb559640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efff4112360 con 0x7efff4102410 2024-07-26T13:44:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.238+0000 7efffb559640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efff41121e0 con 0x7efff4108690 2024-07-26T13:44:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.238+0000 7efffb559640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efff41124e0 con 0x7efff4107d40 2024-07-26T13:44:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.238+0000 7efff92f5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efff4102410 0x7efff419ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.238+0000 7efff92f5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efff4102410 0x7efff419ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40386/0 (socket says 172.21.15.100:40386) 2024-07-26T13:44:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.238+0000 7efff92f5640 1 -- 172.21.15.100:0/1972964412 learned_addr learned my addr 172.21.15.100:0/1972964412 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.238+0000 7efff9af6640 1 --2- 172.21.15.100:0/1972964412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efff4108690 0x7efff419b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.238+0000 7efff8af4640 1 --2- 172.21.15.100:0/1972964412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efff4107d40 0x7efff419b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.238+0000 7efff92f5640 1 -- 172.21.15.100:0/1972964412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efff4107d40 msgr2=0x7efff419b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.238+0000 7efff92f5640 1 --2- 172.21.15.100:0/1972964412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efff4107d40 0x7efff419b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.238+0000 7efff92f5640 1 -- 172.21.15.100:0/1972964412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efff4108690 msgr2=0x7efff419b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.238+0000 7efff92f5640 1 --2- 172.21.15.100:0/1972964412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efff4108690 0x7efff419b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.238+0000 7efff92f5640 1 -- 172.21.15.100:0/1972964412 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efff41a5e30 con 0x7efff4102410 2024-07-26T13:44:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.238+0000 7efff9af6640 1 --2- 172.21.15.100:0/1972964412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efff4108690 0x7efff419b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:44:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.238+0000 7efff92f5640 1 --2- 172.21.15.100:0/1972964412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efff4102410 0x7efff419ac30 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7effe400c920 tx=0x7effe400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.238+0000 7effea7fc640 1 -- 172.21.15.100:0/1972964412 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe4007c70 con 0x7efff4102410 2024-07-26T13:44:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.239+0000 7effea7fc640 1 -- 172.21.15.100:0/1972964412 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effe4007e10 con 0x7efff4102410 2024-07-26T13:44:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.239+0000 7effea7fc640 1 -- 172.21.15.100:0/1972964412 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe4018820 con 0x7efff4102410 2024-07-26T13:44:04.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.239+0000 7efffb559640 1 -- 172.21.15.100:0/1972964412 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efff41a6120 con 0x7efff4102410 2024-07-26T13:44:04.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.239+0000 7efffb559640 1 -- 172.21.15.100:0/1972964412 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efff41a6620 con 0x7efff4102410 2024-07-26T13:44:04.249 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.241+0000 7effea7fc640 1 -- 172.21.15.100:0/1972964412 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7effe400f450 con 0x7efff4102410 2024-07-26T13:44:04.249 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.241+0000 7f000246a740 1 Processor -- start 2024-07-26T13:44:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.241+0000 7f000246a740 1 -- start start 2024-07-26T13:44:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.242+0000 7f000246a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563acc2261b0 0x563acc5c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.242+0000 7f000246a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acc1fd2f0 0x563acc5c9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.242+0000 7f000246a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563acc2054d0 0x563acc5c9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.242+0000 7f000246a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x563acc53e3b0 con 0x563acc2054d0 2024-07-26T13:44:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.242+0000 7f000246a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563acc52d8b0 con 0x563acc1fd2f0 2024-07-26T13:44:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.242+0000 7f000246a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563acc5318f0 con 0x563acc2261b0 2024-07-26T13:44:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.242+0000 7efff8af4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563acc2054d0 0x563acc5c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.242+0000 7efff8af4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563acc2054d0 0x563acc5c9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40400/0 (socket says 172.21.15.100:40400) 2024-07-26T13:44:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.242+0000 7efff8af4640 1 -- 172.21.15.100:0/3230779678 learned_addr learned my addr 172.21.15.100:0/3230779678 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.242+0000 7efff92f5640 1 --2- 172.21.15.100:0/3230779678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acc1fd2f0 0x563acc5c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.242+0000 7efff9af6640 1 --2- 172.21.15.100:0/3230779678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563acc2261b0 0x563acc5c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.242+0000 7effea7fc640 1 --2- 172.21.15.100:0/1972964412 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7effe00b3d40 0x7effe00b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.242+0000 7efff8af4640 1 -- 172.21.15.100:0/3230779678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563acc2261b0 msgr2=0x563acc5c8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.242+0000 7efff8af4640 1 --2- 172.21.15.100:0/3230779678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563acc2261b0 0x563acc5c8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.242+0000 7efff8af4640 1 -- 172.21.15.100:0/3230779678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acc1fd2f0 msgr2=0x563acc5c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.242+0000 7efff8af4640 1 --2- 172.21.15.100:0/3230779678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acc1fd2f0 0x563acc5c9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.242+0000 7efff8af4640 1 -- 172.21.15.100:0/3230779678 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563acc4d49a0 con 0x563acc2054d0 2024-07-26T13:44:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.242+0000 7efff9af6640 1 --2- 172.21.15.100:0/1972964412 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7effe00b3d40 0x7effe00b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.242+0000 7efff8af4640 1 --2- 172.21.15.100:0/3230779678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563acc2054d0 0x563acc5c9760 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7efff0009bc0 tx=0x7efff0010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc501f15ef140cb6 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.242+0000 7effea7fc640 1 -- 172.21.15.100:0/1972964412 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7effe40e4240 con 0x7efff4102410 2024-07-26T13:44:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.242+0000 7effdf7fe640 1 -- 172.21.15.100:0/3230779678 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff001b480 con 0x563acc2054d0 2024-07-26T13:44:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.242+0000 7effdf7fe640 1 -- 172.21.15.100:0/3230779678 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff0012900 con 0x563acc2054d0 2024-07-26T13:44:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.243+0000 7effdf7fe640 1 -- 172.21.15.100:0/3230779678 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff0011800 con 0x563acc2054d0 2024-07-26T13:44:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.243+0000 7f000246a740 1 -- 172.21.15.100:0/3230779678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563acc2054d0 msgr2=0x563acc5c9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.243+0000 7efff9af6640 1 --2- 172.21.15.100:0/1972964412 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7effe00b3d40 0x7effe00b6250 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7effec02f500 tx=0x7effec0095c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.243+0000 7f000246a740 1 --2- 172.21.15.100:0/3230779678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563acc2054d0 0x563acc5c9760 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7efff0009bc0 tx=0x7efff0010740 comp rx=0 tx=0).stop 2024-07-26T13:44:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.243+0000 7f000246a740 1 -- 172.21.15.100:0/3230779678 shutdown_connections 2024-07-26T13:44:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.243+0000 7f000246a740 1 --2- 172.21.15.100:0/3230779678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563acc2054d0 0x563acc5c9760 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.243+0000 7f000246a740 1 --2- 172.21.15.100:0/3230779678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acc1fd2f0 0x563acc5c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.243+0000 7f000246a740 1 --2- 172.21.15.100:0/3230779678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563acc2261b0 0x563acc5c8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.243+0000 7f000246a740 1 -- 172.21.15.100:0/3230779678 >> 172.21.15.100:0/3230779678 conn(0x563acc21a300 msgr2=0x563acc5c6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.243+0000 7f000246a740 1 -- 172.21.15.100:0/3230779678 shutdown_connections 2024-07-26T13:44:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.243+0000 7f000246a740 1 -- 172.21.15.100:0/3230779678 wait complete. 2024-07-26T13:44:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.244+0000 7f000246a740 1 -- start start 2024-07-26T13:44:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.244+0000 7f000246a740 1 Processor -- start 2024-07-26T13:44:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.244+0000 7f000246a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acc1fd2f0 0x563acc5d4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.244+0000 7f000246a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563acc2054d0 0x563acc5d4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.244+0000 7f000246a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563acc2261b0 0x563acc5e0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.244+0000 7f000246a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x563acc467ab0 con 0x563acc2261b0 2024-07-26T13:44:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.244+0000 7f000246a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563acc4b5cf0 con 0x563acc1fd2f0 2024-07-26T13:44:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.244+0000 7f000246a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563acc5d7460 con 0x563acc2054d0 2024-07-26T13:44:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.244+0000 7f000246a740 10 client.0 _ll_register_callbacks cb 0x563acc5279f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:44:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.244+0000 7efff9af6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563acc2261b0 0x563acc5e0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.245+0000 7efff9af6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563acc2261b0 0x563acc5e0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40416/0 (socket says 172.21.15.100:40416) 2024-07-26T13:44:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.245+0000 7efff9af6640 1 -- 172.21.15.100:0/5849260 learned_addr learned my addr 172.21.15.100:0/5849260 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.245+0000 7efff92f5640 1 --2- 172.21.15.100:0/5849260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acc1fd2f0 0x563acc5d4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.245+0000 7efff8af4640 1 --2- 172.21.15.100:0/5849260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563acc2054d0 0x563acc5d4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.245+0000 7efff9af6640 1 -- 172.21.15.100:0/5849260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563acc2054d0 msgr2=0x563acc5d4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.245+0000 7efff9af6640 1 --2- 172.21.15.100:0/5849260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563acc2054d0 0x563acc5d4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.245+0000 7efff9af6640 1 -- 172.21.15.100:0/5849260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acc1fd2f0 msgr2=0x563acc5d4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.245+0000 7efff9af6640 1 --2- 172.21.15.100:0/5849260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acc1fd2f0 0x563acc5d4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.245+0000 7efff9af6640 1 -- 172.21.15.100:0/5849260 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563acc291790 con 0x563acc2261b0 2024-07-26T13:44:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.245+0000 7efff9af6640 1 --2- 172.21.15.100:0/5849260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563acc2261b0 0x563acc5e0750 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7effec0317a0 tx=0x7effec02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806375a301f4b95b server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.245+0000 7effdcff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:44:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.245+0000 7effdcff9640 1 -- 172.21.15.100:0/5849260 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec004240 con 0x563acc2261b0 2024-07-26T13:44:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.245+0000 7effdcff9640 1 -- 172.21.15.100:0/5849260 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7effec0043e0 con 0x563acc2261b0 2024-07-26T13:44:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.245+0000 7f000246a740 1 -- 172.21.15.100:0/5849260 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efff0012900 con 0x563acc2261b0 2024-07-26T13:44:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.245+0000 7effdcff9640 1 -- 172.21.15.100:0/5849260 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7effec0045e0 con 0x563acc2261b0 2024-07-26T13:44:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.245+0000 7f000246a740 10 client.16500 fetch_fsmap learned FSMap version 37 2024-07-26T13:44:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.245+0000 7f000246a740 1 -- 172.21.15.100:0/5849260 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563acc30a620 con 0x563acc2261b0 2024-07-26T13:44:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.246+0000 7effdcff9640 1 -- 172.21.15.100:0/5849260 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff0012900 con 0x563acc2261b0 2024-07-26T13:44:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.246+0000 7effdcff9640 1 -- 172.21.15.100:0/5849260 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7effec057020 con 0x563acc2261b0 2024-07-26T13:44:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.246+0000 7f000246a740 10 client.16500 fetch_fsmap finished waiting for FSMap version 37 2024-07-26T13:44:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.246+0000 7f000246a740 10 client.16500 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:44:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.246+0000 7f000246a740 20 client.16500 populate_metadata read hostname 'smithi100' 2024-07-26T13:44:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.246+0000 7f000246a740 1 --2- 172.21.15.100:0/5849260 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x563acc1e4d80 0x563acc5e6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.246+0000 7f000246a740 4 client.16500 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:44:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.246+0000 7f000246a740 1 -- 172.21.15.100:0/5849260 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563acc540300 con 0x563acc1e4d80 2024-07-26T13:44:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.246+0000 7efff92f5640 1 --2- 172.21.15.100:0/5849260 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x563acc1e4d80 0x563acc5e6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.247+0000 7effdcff9640 10 client.16500 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.247+0000 7efff92f5640 1 --2- 172.21.15.100:0/5849260 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x563acc1e4d80 0x563acc5e6830 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67064849fed07550 server_cookie=c91a3ae58dbbd59c in_seq=0 out_seq=0 2024-07-26T13:44:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.247+0000 7effdcff9640 1 -- 172.21.15.100:0/5849260 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563acc540300 con 0x563acc1e4d80 2024-07-26T13:44:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.247+0000 7effdcff9640 10 client.16500 handle_command_reply: tid=0 2024-07-26T13:44:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.247+0000 7f000246a740 1 client.16500 shutdown 2024-07-26T13:44:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.248+0000 7f000246a740 1 -- 172.21.15.100:0/5849260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563acc2261b0 msgr2=0x563acc5e0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.248+0000 7f000246a740 1 --2- 172.21.15.100:0/5849260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563acc2261b0 0x563acc5e0750 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7effec0317a0 tx=0x7effec02a8c0 comp rx=0 tx=0).stop 2024-07-26T13:44:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.248+0000 7f000246a740 1 -- 172.21.15.100:0/5849260 shutdown_connections 2024-07-26T13:44:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.248+0000 7f000246a740 1 --2- 172.21.15.100:0/5849260 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x563acc1e4d80 0x563acc5e6830 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.248+0000 7f000246a740 1 --2- 172.21.15.100:0/5849260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563acc2261b0 0x563acc5e0750 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.248+0000 7effdcff9640 0 client.16500 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.248+0000 7f000246a740 1 --2- 172.21.15.100:0/5849260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563acc2054d0 0x563acc5d4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.248+0000 7f000246a740 1 --2- 172.21.15.100:0/5849260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acc1fd2f0 0x563acc5d4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.248+0000 7f000246a740 1 -- 172.21.15.100:0/5849260 >> 172.21.15.100:0/5849260 conn(0x563acc21a300 msgr2=0x563acc5c9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.248+0000 7f000246a740 1 -- 172.21.15.100:0/5849260 shutdown_connections 2024-07-26T13:44:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.248+0000 7f000246a740 1 -- 172.21.15.100:0/5849260 wait complete. 2024-07-26T13:44:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.248+0000 7f000246a740 20 client.16500 trim_cache size 0 max 16384 2024-07-26T13:44:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.263+0000 7f000246a740 1 Processor -- start 2024-07-26T13:44:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.263+0000 7f000246a740 1 -- start start 2024-07-26T13:44:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.263+0000 7f000246a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acc1fd2f0 0x563acc5ca420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.263+0000 7f000246a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563acc2054d0 0x563acc5e0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.263+0000 7f000246a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563acc2261b0 0x563acc5e0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.263+0000 7f000246a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efff41b1980 con 0x563acc2054d0 2024-07-26T13:44:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.263+0000 7f000246a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efff41af1f0 con 0x563acc1fd2f0 2024-07-26T13:44:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.263+0000 7f000246a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efff41ad040 con 0x563acc2261b0 2024-07-26T13:44:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.263+0000 7efff8af4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563acc2054d0 0x563acc5e0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.263+0000 7efff8af4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563acc2054d0 0x563acc5e0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40428/0 (socket says 172.21.15.100:40428) 2024-07-26T13:44:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.263+0000 7efff8af4640 1 -- 172.21.15.100:0/400408263 learned_addr learned my addr 172.21.15.100:0/400408263 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.263+0000 7efff9af6640 1 --2- 172.21.15.100:0/400408263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563acc2261b0 0x563acc5e0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.264+0000 7efff92f5640 1 --2- 172.21.15.100:0/400408263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acc1fd2f0 0x563acc5ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.264+0000 7efff8af4640 1 -- 172.21.15.100:0/400408263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563acc2261b0 msgr2=0x563acc5e0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.264+0000 7efff8af4640 1 --2- 172.21.15.100:0/400408263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563acc2261b0 0x563acc5e0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.264+0000 7efff8af4640 1 -- 172.21.15.100:0/400408263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acc1fd2f0 msgr2=0x563acc5ca420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.264+0000 7efff8af4640 1 --2- 172.21.15.100:0/400408263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acc1fd2f0 0x563acc5ca420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.264+0000 7efff8af4640 1 -- 172.21.15.100:0/400408263 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563acc5d4370 con 0x563acc2054d0 2024-07-26T13:44:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.264+0000 7efff8af4640 1 --2- 172.21.15.100:0/400408263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563acc2054d0 0x563acc5e0540 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7efff0009bc0 tx=0x7efff00096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=507dd19a26a1bdad server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.264+0000 7effdeffd640 1 -- 172.21.15.100:0/400408263 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff0005040 con 0x563acc2054d0 2024-07-26T13:44:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.264+0000 7effdeffd640 1 -- 172.21.15.100:0/400408263 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff0004680 con 0x563acc2054d0 2024-07-26T13:44:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.264+0000 7effdeffd640 1 -- 172.21.15.100:0/400408263 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff0011750 con 0x563acc2054d0 2024-07-26T13:44:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.265+0000 7f000246a740 1 -- 172.21.15.100:0/400408263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563acc2054d0 msgr2=0x563acc5e0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.265+0000 7f000246a740 1 --2- 172.21.15.100:0/400408263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563acc2054d0 0x563acc5e0540 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7efff0009bc0 tx=0x7efff00096c0 comp rx=0 tx=0).stop 2024-07-26T13:44:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.265+0000 7f000246a740 1 -- 172.21.15.100:0/400408263 shutdown_connections 2024-07-26T13:44:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.265+0000 7f000246a740 1 --2- 172.21.15.100:0/400408263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563acc2261b0 0x563acc5e0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.265+0000 7f000246a740 1 --2- 172.21.15.100:0/400408263 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563acc2054d0 0x563acc5e0540 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.265+0000 7f000246a740 1 --2- 172.21.15.100:0/400408263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acc1fd2f0 0x563acc5ca420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.265+0000 7f000246a740 1 -- 172.21.15.100:0/400408263 >> 172.21.15.100:0/400408263 conn(0x563acc21a300 msgr2=0x563acc5c9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.265+0000 7f000246a740 1 -- 172.21.15.100:0/400408263 shutdown_connections 2024-07-26T13:44:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.265+0000 7f000246a740 1 -- 172.21.15.100:0/400408263 wait complete. 2024-07-26T13:44:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.265+0000 7f000246a740 1 -- start start 2024-07-26T13:44:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.265+0000 7f000246a740 1 Processor -- start 2024-07-26T13:44:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.266+0000 7f000246a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563acc1fd2f0 0x563acc5d9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.266+0000 7f000246a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563acc2054d0 0x563acc5d9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.266+0000 7f000246a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acc2261b0 0x563acc5da100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.266+0000 7f000246a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efff41ad6b0 con 0x563acc1fd2f0 2024-07-26T13:44:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.266+0000 7f000246a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efff41aeb20 con 0x563acc2261b0 2024-07-26T13:44:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.266+0000 7f000246a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efff41add80 con 0x563acc2054d0 2024-07-26T13:44:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.266+0000 7f000246a740 10 client.0 _ll_register_callbacks cb 0x7efff0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:44:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.266+0000 7efff92f5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563acc1fd2f0 0x563acc5d9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.266+0000 7efff92f5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563acc1fd2f0 0x563acc5d9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40444/0 (socket says 172.21.15.100:40444) 2024-07-26T13:44:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.266+0000 7efff92f5640 1 -- 172.21.15.100:0/266302438 learned_addr learned my addr 172.21.15.100:0/266302438 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.266+0000 7efff8af4640 1 --2- 172.21.15.100:0/266302438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563acc2054d0 0x563acc5d9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.266+0000 7efff92f5640 1 -- 172.21.15.100:0/266302438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563acc2054d0 msgr2=0x563acc5d9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.266+0000 7efff92f5640 1 --2- 172.21.15.100:0/266302438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563acc2054d0 0x563acc5d9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.266+0000 7efff9af6640 1 --2- 172.21.15.100:0/266302438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acc2261b0 0x563acc5da100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.266+0000 7efff92f5640 1 -- 172.21.15.100:0/266302438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acc2261b0 msgr2=0x563acc5da100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.266+0000 7efff92f5640 1 --2- 172.21.15.100:0/266302438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acc2261b0 0x563acc5da100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.266+0000 7efff92f5640 1 -- 172.21.15.100:0/266302438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563acc64d100 con 0x563acc1fd2f0 2024-07-26T13:44:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.266+0000 7efff8af4640 1 --2- 172.21.15.100:0/266302438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563acc2054d0 0x563acc5d9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:44:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.266+0000 7efff92f5640 1 --2- 172.21.15.100:0/266302438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563acc1fd2f0 0x563acc5d9680 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7effe40e1a10 tx=0x7effe400fe60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd384f0522f547e server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.266+0000 7effdcff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:44:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.266+0000 7effdcff9640 1 -- 172.21.15.100:0/266302438 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe40af420 con 0x563acc1fd2f0 2024-07-26T13:44:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.266+0000 7effdcff9640 1 -- 172.21.15.100:0/266302438 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7effe400f740 con 0x563acc1fd2f0 2024-07-26T13:44:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.266+0000 7f000246a740 1 -- 172.21.15.100:0/266302438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efff0004680 con 0x563acc1fd2f0 2024-07-26T13:44:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.267+0000 7effdcff9640 1 -- 172.21.15.100:0/266302438 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7effe40b3ed0 con 0x563acc1fd2f0 2024-07-26T13:44:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.267+0000 7f000246a740 10 client.16506 fetch_fsmap learned FSMap version 37 2024-07-26T13:44:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.267+0000 7f000246a740 1 -- 172.21.15.100:0/266302438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563acc64e3d0 con 0x563acc1fd2f0 2024-07-26T13:44:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.267+0000 7effdcff9640 1 -- 172.21.15.100:0/266302438 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff0004680 con 0x563acc1fd2f0 2024-07-26T13:44:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.267+0000 7effdcff9640 1 -- 172.21.15.100:0/266302438 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7effe40b3a40 con 0x563acc1fd2f0 2024-07-26T13:44:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.267+0000 7f000246a740 10 client.16506 fetch_fsmap finished waiting for FSMap version 37 2024-07-26T13:44:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.267+0000 7f000246a740 10 client.16506 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:44:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.267+0000 7f000246a740 20 client.16506 populate_metadata read hostname 'smithi100' 2024-07-26T13:44:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.267+0000 7f000246a740 1 --2- 172.21.15.100:0/266302438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x563acc1ba1c0 0x563acc650ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.267+0000 7f000246a740 4 client.16506 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-26T13:44:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.267+0000 7f000246a740 1 -- 172.21.15.100:0/266302438 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563acc53bda0 con 0x563acc1ba1c0 2024-07-26T13:44:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.267+0000 7efff8af4640 1 --2- 172.21.15.100:0/266302438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x563acc1ba1c0 0x563acc650ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.268+0000 7effdcff9640 10 client.16506 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.268+0000 7efff8af4640 1 --2- 172.21.15.100:0/266302438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x563acc1ba1c0 0x563acc650ba0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16d15b03ba17b437 server_cookie=3c673e064a9602a3 in_seq=0 out_seq=0 2024-07-26T13:44:04.411 INFO:tasks.workunit.client.0.smithi100.stdout: 1 145991 0.00 MB/sec execute 18 sec latency 18310.511 ms 2024-07-26T13:44:04.513 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-26T13:44:04.513 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:44:04.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.735+0000 7effdcff9640 1 -- 172.21.15.100:0/266302438 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563acc53bda0 con 0x563acc1ba1c0 2024-07-26T13:44:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.735+0000 7effdcff9640 10 client.16506 handle_command_reply: tid=0 2024-07-26T13:44:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.735+0000 7f000246a740 1 client.16506 shutdown 2024-07-26T13:44:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.735+0000 7f000246a740 1 -- 172.21.15.100:0/266302438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563acc1fd2f0 msgr2=0x563acc5d9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.735+0000 7f000246a740 1 --2- 172.21.15.100:0/266302438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563acc1fd2f0 0x563acc5d9680 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7effe40e1a10 tx=0x7effe400fe60 comp rx=0 tx=0).stop 2024-07-26T13:44:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.735+0000 7f000246a740 1 -- 172.21.15.100:0/266302438 shutdown_connections 2024-07-26T13:44:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.735+0000 7f000246a740 1 --2- 172.21.15.100:0/266302438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x563acc1ba1c0 0x563acc650ba0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.735+0000 7f000246a740 1 --2- 172.21.15.100:0/266302438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563acc2261b0 0x563acc5da100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.735+0000 7f000246a740 1 --2- 172.21.15.100:0/266302438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563acc2054d0 0x563acc5d9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.735+0000 7f000246a740 1 --2- 172.21.15.100:0/266302438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563acc1fd2f0 0x563acc5d9680 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.735+0000 7effdcff9640 0 client.16506 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.735+0000 7f000246a740 1 -- 172.21.15.100:0/266302438 >> 172.21.15.100:0/266302438 conn(0x563acc21a300 msgr2=0x563acc5d5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.735+0000 7f000246a740 1 -- 172.21.15.100:0/266302438 shutdown_connections 2024-07-26T13:44:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.735+0000 7f000246a740 1 -- 172.21.15.100:0/266302438 wait complete. 2024-07-26T13:44:04.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.735+0000 7f000246a740 20 client.16506 trim_cache size 0 max 16384 2024-07-26T13:44:04.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-07-26T13:44:04.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-07-26T13:44:04.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-07-26T13:44:04.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-07-26T13:44:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.736+0000 7efffb559640 1 -- 172.21.15.100:0/1972964412 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7effe00b3d40 msgr2=0x7effe00b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.736+0000 7efffb559640 1 --2- 172.21.15.100:0/1972964412 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7effe00b3d40 0x7effe00b6250 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7effec02f500 tx=0x7effec0095c0 comp rx=0 tx=0).stop 2024-07-26T13:44:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.736+0000 7efffb559640 1 -- 172.21.15.100:0/1972964412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efff4102410 msgr2=0x7efff419ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.736+0000 7efffb559640 1 --2- 172.21.15.100:0/1972964412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efff4102410 0x7efff419ac30 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7effe400c920 tx=0x7effe400cde0 comp rx=0 tx=0).stop 2024-07-26T13:44:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.736+0000 7efffb559640 1 -- 172.21.15.100:0/1972964412 shutdown_connections 2024-07-26T13:44:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.736+0000 7efffb559640 1 --2- 172.21.15.100:0/1972964412 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7effe00b3d40 0x7effe00b6250 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.736+0000 7efffb559640 1 --2- 172.21.15.100:0/1972964412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efff4108690 0x7efff419b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.736+0000 7efffb559640 1 --2- 172.21.15.100:0/1972964412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efff4107d40 0x7efff419b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.736+0000 7efffb559640 1 --2- 172.21.15.100:0/1972964412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efff4102410 0x7efff419ac30 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.736+0000 7efffb559640 1 -- 172.21.15.100:0/1972964412 >> 172.21.15.100:0/1972964412 conn(0x7efff40fddf0 msgr2=0x7efff41072e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.736+0000 7efffb559640 1 -- 172.21.15.100:0/1972964412 shutdown_connections 2024-07-26T13:44:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:04.736+0000 7efffb559640 1 -- 172.21.15.100:0/1972964412 wait complete. 2024-07-26T13:44:04.795 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:44:05.411 INFO:tasks.workunit.client.0.smithi100.stdout: 1 145991 0.00 MB/sec execute 19 sec latency 19310.549 ms 2024-07-26T13:44:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:05 smithi138 ceph-mon[36954]: pgmap v484: 97 pgs: 97 active+clean; 63 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 1.7 MiB/s wr, 93 op/s 2024-07-26T13:44:05.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:05 smithi100 ceph-mon[32599]: pgmap v484: 97 pgs: 97 active+clean; 63 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 1.7 MiB/s wr, 93 op/s 2024-07-26T13:44:06.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:05 smithi167 ceph-mon[37140]: pgmap v484: 97 pgs: 97 active+clean; 63 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 1.7 MiB/s wr, 93 op/s 2024-07-26T13:44:06.411 INFO:tasks.workunit.client.0.smithi100.stdout: 1 145991 0.00 MB/sec execute 20 sec latency 20310.575 ms 2024-07-26T13:44:06.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.565+0000 7f131e4e7640 1 -- 172.21.15.100:0/3812751976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1318108750 msgr2=0x7f131810ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:06.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.565+0000 7f131e4e7640 1 --2- 172.21.15.100:0/3812751976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1318108750 0x7f131810ef80 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f1310009ca0 tx=0x7f131002a8c0 comp rx=0 tx=0).stop 2024-07-26T13:44:06.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.566+0000 7f131e4e7640 1 -- 172.21.15.100:0/3812751976 shutdown_connections 2024-07-26T13:44:06.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.566+0000 7f131e4e7640 1 --2- 172.21.15.100:0/3812751976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1318108750 0x7f131810ef80 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:06.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.566+0000 7f131e4e7640 1 --2- 172.21.15.100:0/3812751976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1318107e00 0x7f1318108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:06.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.566+0000 7f131e4e7640 1 --2- 172.21.15.100:0/3812751976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1318102510 0x7f1318102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:06.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.566+0000 7f131e4e7640 1 -- 172.21.15.100:0/3812751976 >> 172.21.15.100:0/3812751976 conn(0x7f1318079190 msgr2=0x7f13180795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:06.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.566+0000 7f131e4e7640 1 -- 172.21.15.100:0/3812751976 shutdown_connections 2024-07-26T13:44:06.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.566+0000 7f131e4e7640 1 -- 172.21.15.100:0/3812751976 wait complete. 2024-07-26T13:44:06.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.566+0000 7f131e4e7640 1 Processor -- start 2024-07-26T13:44:06.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.566+0000 7f131e4e7640 1 -- start start 2024-07-26T13:44:06.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.567+0000 7f131e4e7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1318102510 0x7f13180ff0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:06.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.567+0000 7f131e4e7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1318107e00 0x7f13180ff5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:06.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.567+0000 7f131e4e7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1318108750 0x7f13180ffb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:06.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.567+0000 7f131e4e7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f13181123b0 con 0x7f1318102510 2024-07-26T13:44:06.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.567+0000 7f131e4e7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1318112230 con 0x7f1318107e00 2024-07-26T13:44:06.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.567+0000 7f131e4e7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1318112530 con 0x7f1318108750 2024-07-26T13:44:06.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.567+0000 7f1317fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1318102510 0x7f13180ff0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:06.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.567+0000 7f1317fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1318102510 0x7f13180ff0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37044/0 (socket says 172.21.15.100:37044) 2024-07-26T13:44:06.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.567+0000 7f1317fff640 1 -- 172.21.15.100:0/18068137 learned_addr learned my addr 172.21.15.100:0/18068137 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:06.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.567+0000 7f131ca84640 1 --2- 172.21.15.100:0/18068137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1318108750 0x7f13180ffb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:06.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.567+0000 7f130ffff640 1 --2- 172.21.15.100:0/18068137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1318107e00 0x7f13180ff5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:06.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.567+0000 7f1317fff640 1 -- 172.21.15.100:0/18068137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1318108750 msgr2=0x7f13180ffb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:06.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.567+0000 7f1317fff640 1 --2- 172.21.15.100:0/18068137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1318108750 0x7f13180ffb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:06.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.567+0000 7f1317fff640 1 -- 172.21.15.100:0/18068137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1318107e00 msgr2=0x7f13180ff5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:06.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.567+0000 7f1317fff640 1 --2- 172.21.15.100:0/18068137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1318107e00 0x7f13180ff5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:06.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.567+0000 7f1317fff640 1 -- 172.21.15.100:0/18068137 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13181a8110 con 0x7f1318102510 2024-07-26T13:44:06.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.567+0000 7f131ca84640 1 --2- 172.21.15.100:0/18068137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1318108750 0x7f13180ffb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:44:06.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.567+0000 7f130ffff640 1 --2- 172.21.15.100:0/18068137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1318107e00 0x7f13180ff5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:44:06.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.567+0000 7f1317fff640 1 --2- 172.21.15.100:0/18068137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1318102510 0x7f13180ff0a0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f130800c890 tx=0x7f130800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:06.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.567+0000 7f1315ffb640 1 -- 172.21.15.100:0/18068137 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1308004220 con 0x7f1318102510 2024-07-26T13:44:06.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.567+0000 7f1315ffb640 1 -- 172.21.15.100:0/18068137 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13080043c0 con 0x7f1318102510 2024-07-26T13:44:06.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.568+0000 7f1315ffb640 1 -- 172.21.15.100:0/18068137 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1308004a00 con 0x7f1318102510 2024-07-26T13:44:06.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.568+0000 7f131e4e7640 1 -- 172.21.15.100:0/18068137 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13181a8400 con 0x7f1318102510 2024-07-26T13:44:06.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.568+0000 7f131e4e7640 1 -- 172.21.15.100:0/18068137 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13181a88f0 con 0x7f1318102510 2024-07-26T13:44:06.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.569+0000 7f131e4e7640 1 -- 172.21.15.100:0/18068137 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1318102920 con 0x7f1318102510 2024-07-26T13:44:06.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.570+0000 7f1315ffb640 1 -- 172.21.15.100:0/18068137 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f13080079b0 con 0x7f1318102510 2024-07-26T13:44:06.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.571+0000 7f1315ffb640 1 --2- 172.21.15.100:0/18068137 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f13040b3d40 0x7f13040b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:06.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.571+0000 7f130ffff640 1 --2- 172.21.15.100:0/18068137 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f13040b3d40 0x7f13040b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:06.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.571+0000 7f1315ffb640 1 -- 172.21.15.100:0/18068137 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f13080e3e60 con 0x7f1318102510 2024-07-26T13:44:06.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.571+0000 7f130ffff640 1 --2- 172.21.15.100:0/18068137 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f13040b3d40 0x7f13040b6250 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f13000058c0 tx=0x7f1300005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:06.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.575+0000 7f1315ffb640 1 -- 172.21.15.100:0/18068137 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f1308014030 con 0x7f1318102510 2024-07-26T13:44:06.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.792+0000 7f131e4e7640 1 -- 172.21.15.100:0/18068137 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f131802e0c0 con 0x7f1318102510 2024-07-26T13:44:06.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.793+0000 7f1315ffb640 1 -- 172.21.15.100:0/18068137 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11933 (secure 0 0 0) 0x7f13080ab140 con 0x7f1318102510 2024-07-26T13:44:06.794 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:44:06.794 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":37,"btime":"2024-07-26T13:43:49:549366+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14790,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/313501409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":313501409},{"type":"v1","addr":"172.21.15.100:6839","nonce":313501409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/1402975506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":1402975506},{"type":"v1","addr":"172.21.15.100:6841","nonce":1402975506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-26T13:37:01.281843+0000","modified":"2024-07-26T13:43:49.175512+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24607,"mds_1":24605,"mds_2":24557},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24557":{"gid":24557,"name":"e","rank":2,"incarnation":30,"state":"up:active","state_seq":107,"addr":"172.21.15.138:6835/3290105300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3290105300},{"type":"v1","addr":"172.21.15.138:6835","nonce":3290105300}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/3393225483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3393225483},{"type":"v1","addr":"172.21.15.138:6837","nonce":3393225483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.138:6839/1580925471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1580925471},{"type":"v1","addr":"172.21.15.138:6839","nonce":1580925471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/398150514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":398150514},{"type":"v1","addr":"172.21.15.167:6839","nonce":398150514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25762":{"gid":25762,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2476068386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2476068386},{"type":"v1","addr":"172.21.15.167:6837","nonce":2476068386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24607,"qdb_cluster":[24557,24605,24607]},"id":1}]} 2024-07-26T13:44:06.795 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 37 2024-07-26T13:44:06.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.795+0000 7f131e4e7640 1 -- 172.21.15.100:0/18068137 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f13040b3d40 msgr2=0x7f13040b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:06.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.795+0000 7f131e4e7640 1 --2- 172.21.15.100:0/18068137 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f13040b3d40 0x7f13040b6250 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f13000058c0 tx=0x7f1300005850 comp rx=0 tx=0).stop 2024-07-26T13:44:06.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.795+0000 7f131e4e7640 1 -- 172.21.15.100:0/18068137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1318102510 msgr2=0x7f13180ff0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:06.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.795+0000 7f131e4e7640 1 --2- 172.21.15.100:0/18068137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1318102510 0x7f13180ff0a0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f130800c890 tx=0x7f130800cd50 comp rx=0 tx=0).stop 2024-07-26T13:44:06.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.795+0000 7f131e4e7640 1 -- 172.21.15.100:0/18068137 shutdown_connections 2024-07-26T13:44:06.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.795+0000 7f131e4e7640 1 --2- 172.21.15.100:0/18068137 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f13040b3d40 0x7f13040b6250 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:06.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.795+0000 7f131e4e7640 1 --2- 172.21.15.100:0/18068137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1318108750 0x7f13180ffb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:06.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.795+0000 7f131e4e7640 1 --2- 172.21.15.100:0/18068137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1318107e00 0x7f13180ff5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:06.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.795+0000 7f131e4e7640 1 --2- 172.21.15.100:0/18068137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1318102510 0x7f13180ff0a0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:06.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.795+0000 7f131e4e7640 1 -- 172.21.15.100:0/18068137 >> 172.21.15.100:0/18068137 conn(0x7f1318079190 msgr2=0x7f131806b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:06.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.795+0000 7f131e4e7640 1 -- 172.21.15.100:0/18068137 shutdown_connections 2024-07-26T13:44:06.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:06.795+0000 7f131e4e7640 1 -- 172.21.15.100:0/18068137 wait complete. 2024-07-26T13:44:07.411 INFO:tasks.workunit.client.0.smithi100.stdout: 1 145991 0.00 MB/sec execute 21 sec latency 21310.682 ms 2024-07-26T13:44:07.577 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-26T13:44:07.835 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:44:07.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:07 smithi138 ceph-mon[36954]: pgmap v485: 97 pgs: 97 active+clean; 63 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 1.6 MiB/s wr, 82 op/s 2024-07-26T13:44:07.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:07 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/18068137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:44:07.956 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:07 smithi100 ceph-mon[32599]: pgmap v485: 97 pgs: 97 active+clean; 63 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 1.6 MiB/s wr, 82 op/s 2024-07-26T13:44:07.957 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:07 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/18068137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:44:08.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:07 smithi167 ceph-mon[37140]: pgmap v485: 97 pgs: 97 active+clean; 63 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 1.6 MiB/s wr, 82 op/s 2024-07-26T13:44:08.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:07 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/18068137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:44:08.411 INFO:tasks.workunit.client.0.smithi100.stdout: 1 145991 0.00 MB/sec execute 22 sec latency 22310.771 ms 2024-07-26T13:44:09.412 INFO:tasks.workunit.client.0.smithi100.stdout: 1 145991 0.00 MB/sec execute 23 sec latency 23310.834 ms 2024-07-26T13:44:09.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.551+0000 7f827de7c640 1 -- 172.21.15.100:0/987924885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8278102490 msgr2=0x7f82781028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:09.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.551+0000 7f827de7c640 1 --2- 172.21.15.100:0/987924885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8278102490 0x7f82781028a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f8268009900 tx=0x7f826802a410 comp rx=0 tx=0).stop 2024-07-26T13:44:09.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.551+0000 7f827de7c640 1 -- 172.21.15.100:0/987924885 shutdown_connections 2024-07-26T13:44:09.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.551+0000 7f827de7c640 1 --2- 172.21.15.100:0/987924885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8278108720 0x7f827810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:09.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.552+0000 7f827de7c640 1 --2- 172.21.15.100:0/987924885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8278107dd0 0x7f82781081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:09.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.552+0000 7f827de7c640 1 --2- 172.21.15.100:0/987924885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8278102490 0x7f82781028a0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:09.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.552+0000 7f827de7c640 1 -- 172.21.15.100:0/987924885 >> 172.21.15.100:0/987924885 conn(0x7f8278079160 msgr2=0x7f8278079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:09.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.552+0000 7f827de7c640 1 -- 172.21.15.100:0/987924885 shutdown_connections 2024-07-26T13:44:09.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.552+0000 7f827de7c640 1 -- 172.21.15.100:0/987924885 wait complete. 2024-07-26T13:44:09.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.552+0000 7f827de7c640 1 Processor -- start 2024-07-26T13:44:09.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.552+0000 7f827de7c640 1 -- start start 2024-07-26T13:44:09.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.553+0000 7f827de7c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8278102490 0x7f8278198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:09.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.553+0000 7f827de7c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8278107dd0 0x7f8278199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:09.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.553+0000 7f827de7c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8278108720 0x7f827819e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:09.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.553+0000 7f827de7c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f82781123a0 con 0x7f8278107dd0 2024-07-26T13:44:09.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.553+0000 7f827de7c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8278112220 con 0x7f8278102490 2024-07-26T13:44:09.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.553+0000 7f827de7c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8278112520 con 0x7f8278108720 2024-07-26T13:44:09.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.553+0000 7f8276ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8278107dd0 0x7f8278199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:09.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.553+0000 7f8276ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8278107dd0 0x7f8278199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37062/0 (socket says 172.21.15.100:37062) 2024-07-26T13:44:09.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.553+0000 7f8276ffd640 1 -- 172.21.15.100:0/1028350314 learned_addr learned my addr 172.21.15.100:0/1028350314 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:09.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.553+0000 7f82777fe640 1 --2- 172.21.15.100:0/1028350314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8278102490 0x7f8278198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:09.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.553+0000 7f8277fff640 1 --2- 172.21.15.100:0/1028350314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8278108720 0x7f827819e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:09.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.553+0000 7f8276ffd640 1 -- 172.21.15.100:0/1028350314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8278108720 msgr2=0x7f827819e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:09.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.553+0000 7f8276ffd640 1 --2- 172.21.15.100:0/1028350314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8278108720 0x7f827819e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:09.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.553+0000 7f8276ffd640 1 -- 172.21.15.100:0/1028350314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8278102490 msgr2=0x7f8278198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:09.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.553+0000 7f8276ffd640 1 --2- 172.21.15.100:0/1028350314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8278102490 0x7f8278198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:09.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.553+0000 7f8276ffd640 1 -- 172.21.15.100:0/1028350314 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f827819ece0 con 0x7f8278107dd0 2024-07-26T13:44:09.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.553+0000 7f82777fe640 1 --2- 172.21.15.100:0/1028350314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8278102490 0x7f8278198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:44:09.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.553+0000 7f8276ffd640 1 --2- 172.21.15.100:0/1028350314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8278107dd0 0x7f8278199310 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f826c00e9c0 tx=0x7f826c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:09.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.554+0000 7f8274ff9640 1 -- 172.21.15.100:0/1028350314 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c00cd80 con 0x7f8278107dd0 2024-07-26T13:44:09.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.554+0000 7f827de7c640 1 -- 172.21.15.100:0/1028350314 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f827819efd0 con 0x7f8278107dd0 2024-07-26T13:44:09.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.554+0000 7f8274ff9640 1 -- 172.21.15.100:0/1028350314 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f826c0054d0 con 0x7f8278107dd0 2024-07-26T13:44:09.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.554+0000 7f8274ff9640 1 -- 172.21.15.100:0/1028350314 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c010430 con 0x7f8278107dd0 2024-07-26T13:44:09.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.554+0000 7f827de7c640 1 -- 172.21.15.100:0/1028350314 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f827806a890 con 0x7f8278107dd0 2024-07-26T13:44:09.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.556+0000 7f8284400740 1 Processor -- start 2024-07-26T13:44:09.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.556+0000 7f8284400740 1 -- start start 2024-07-26T13:44:09.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.556+0000 7f8274ff9640 1 -- 172.21.15.100:0/1028350314 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f826c0105d0 con 0x7f8278107dd0 2024-07-26T13:44:09.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.556+0000 7f8284400740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddb2c2b070 0x55ddb2fd0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:09.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.557+0000 7f8284400740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddb2c021b0 0x55ddb2fd0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:09.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.557+0000 7f8284400740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddb2c0a390 0x55ddb2fd0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:09.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.557+0000 7f8284400740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ddb2f46bb0 con 0x55ddb2c2b070 2024-07-26T13:44:09.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.557+0000 7f8284400740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ddb2f360b0 con 0x55ddb2c0a390 2024-07-26T13:44:09.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.557+0000 7f8284400740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ddb2f3a0f0 con 0x55ddb2c021b0 2024-07-26T13:44:09.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.557+0000 7f8277fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddb2c2b070 0x55ddb2fd0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:09.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.557+0000 7f8277fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddb2c2b070 0x55ddb2fd0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37066/0 (socket says 172.21.15.100:37066) 2024-07-26T13:44:09.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.557+0000 7f8277fff640 1 -- 172.21.15.100:0/2440381647 learned_addr learned my addr 172.21.15.100:0/2440381647 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:09.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.557+0000 7f82777fe640 1 --2- 172.21.15.100:0/2440381647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddb2c021b0 0x55ddb2fd0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:09.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.557+0000 7f8276ffd640 1 --2- 172.21.15.100:0/2440381647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddb2c0a390 0x55ddb2fd0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:09.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.557+0000 7f8277fff640 1 -- 172.21.15.100:0/2440381647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddb2c021b0 msgr2=0x55ddb2fd0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:09.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.557+0000 7f8277fff640 1 --2- 172.21.15.100:0/2440381647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddb2c021b0 0x55ddb2fd0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:09.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.557+0000 7f8277fff640 1 -- 172.21.15.100:0/2440381647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddb2c0a390 msgr2=0x55ddb2fd0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:09.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.557+0000 7f8277fff640 1 --2- 172.21.15.100:0/2440381647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddb2c0a390 0x55ddb2fd0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:09.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.557+0000 7f8277fff640 1 -- 172.21.15.100:0/2440381647 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddb2edcb90 con 0x55ddb2c2b070 2024-07-26T13:44:09.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.557+0000 7f82777fe640 1 --2- 172.21.15.100:0/2440381647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddb2c021b0 0x55ddb2fd0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:44:09.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.557+0000 7f8277fff640 1 --2- 172.21.15.100:0/2440381647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddb2c2b070 0x55ddb2fd0570 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f8270011440 tx=0x7f8270011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c37645f8097218 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:09.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.557+0000 7f8274ff9640 1 --2- 172.21.15.100:0/1028350314 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f82600b3d40 0x7f82600b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:09.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.557+0000 7f825dffb640 1 -- 172.21.15.100:0/2440381647 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8270004d20 con 0x55ddb2c2b070 2024-07-26T13:44:09.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.558+0000 7f82777fe640 1 --2- 172.21.15.100:0/1028350314 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f82600b3d40 0x7f82600b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:09.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.558+0000 7f825dffb640 1 -- 172.21.15.100:0/2440381647 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8270004050 con 0x55ddb2c2b070 2024-07-26T13:44:09.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.558+0000 7f825dffb640 1 -- 172.21.15.100:0/2440381647 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827000f7a0 con 0x55ddb2c2b070 2024-07-26T13:44:09.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.558+0000 7f8274ff9640 1 -- 172.21.15.100:0/1028350314 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f826c0e4450 con 0x7f8278107dd0 2024-07-26T13:44:09.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.558+0000 7f8284400740 1 -- 172.21.15.100:0/2440381647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddb2c2b070 msgr2=0x55ddb2fd0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:09.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.558+0000 7f82777fe640 1 --2- 172.21.15.100:0/1028350314 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f82600b3d40 0x7f82600b6250 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8268031440 tx=0x7f8268032ed0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:09.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.558+0000 7f8284400740 1 --2- 172.21.15.100:0/2440381647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddb2c2b070 0x55ddb2fd0570 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f8270011440 tx=0x7f8270011900 comp rx=0 tx=0).stop 2024-07-26T13:44:09.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.558+0000 7f8284400740 1 -- 172.21.15.100:0/2440381647 shutdown_connections 2024-07-26T13:44:09.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.558+0000 7f8284400740 1 --2- 172.21.15.100:0/2440381647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddb2c0a390 0x55ddb2fd0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:09.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.558+0000 7f8284400740 1 --2- 172.21.15.100:0/2440381647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddb2c021b0 0x55ddb2fd0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:09.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.558+0000 7f8284400740 1 --2- 172.21.15.100:0/2440381647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddb2c2b070 0x55ddb2fd0570 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:09.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.558+0000 7f8284400740 1 -- 172.21.15.100:0/2440381647 >> 172.21.15.100:0/2440381647 conn(0x55ddb2c1f460 msgr2=0x55ddb2fce460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:09.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.559+0000 7f8284400740 1 -- 172.21.15.100:0/2440381647 shutdown_connections 2024-07-26T13:44:09.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.559+0000 7f8284400740 1 -- 172.21.15.100:0/2440381647 wait complete. 2024-07-26T13:44:09.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.559+0000 7f8284400740 1 -- start start 2024-07-26T13:44:09.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.559+0000 7f8284400740 1 Processor -- start 2024-07-26T13:44:09.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.559+0000 7f8284400740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddb2c021b0 0x55ddb2fe04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:09.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.559+0000 7f8284400740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddb2c0a390 0x55ddb2fe0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:09.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.559+0000 7f8284400740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddb2c2b070 0x55ddb2fec3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:09.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.560+0000 7f8284400740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ddb2d99700 con 0x55ddb2c0a390 2024-07-26T13:44:09.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.560+0000 7f8284400740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ddb2ed89e0 con 0x55ddb2c2b070 2024-07-26T13:44:09.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.560+0000 7f8284400740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ddb2fe3080 con 0x55ddb2c021b0 2024-07-26T13:44:09.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.560+0000 7f8284400740 10 client.0 _ll_register_callbacks cb 0x55ddb2ef34f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:44:09.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.560+0000 7f8276ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddb2c0a390 0x55ddb2fe0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:09.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.560+0000 7f8276ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddb2c0a390 0x55ddb2fe0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37080/0 (socket says 172.21.15.100:37080) 2024-07-26T13:44:09.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.560+0000 7f8276ffd640 1 -- 172.21.15.100:0/2467609470 learned_addr learned my addr 172.21.15.100:0/2467609470 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:09.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.560+0000 7f82777fe640 1 --2- 172.21.15.100:0/2467609470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddb2c021b0 0x55ddb2fe04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:09.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.560+0000 7f8277fff640 1 --2- 172.21.15.100:0/2467609470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddb2c2b070 0x55ddb2fec3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:09.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.560+0000 7f8276ffd640 1 -- 172.21.15.100:0/2467609470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddb2c021b0 msgr2=0x55ddb2fe04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:09.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.560+0000 7f8276ffd640 1 --2- 172.21.15.100:0/2467609470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddb2c021b0 0x55ddb2fe04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:09.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.560+0000 7f8276ffd640 1 -- 172.21.15.100:0/2467609470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddb2c2b070 msgr2=0x55ddb2fec3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:09.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.560+0000 7f8276ffd640 1 --2- 172.21.15.100:0/2467609470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddb2c2b070 0x55ddb2fec3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:09.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.560+0000 7f8276ffd640 1 -- 172.21.15.100:0/2467609470 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddb2c96650 con 0x55ddb2c0a390 2024-07-26T13:44:09.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.560+0000 7f8277fff640 1 --2- 172.21.15.100:0/2467609470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddb2c2b070 0x55ddb2fec3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:44:09.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.560+0000 7f82777fe640 1 --2- 172.21.15.100:0/2467609470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddb2c021b0 0x55ddb2fe04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:44:09.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.560+0000 7f8276ffd640 1 --2- 172.21.15.100:0/2467609470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddb2c0a390 0x55ddb2fe0a10 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f826c0aafc0 tx=0x7f826c0a6f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3af32be77b35f9 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:09.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.560+0000 7f8256ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:44:09.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.560+0000 7f8256ffd640 1 -- 172.21.15.100:0/2467609470 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c0aa7a0 con 0x55ddb2c0a390 2024-07-26T13:44:09.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.560+0000 7f8256ffd640 1 -- 172.21.15.100:0/2467609470 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f826c0b3e60 con 0x55ddb2c0a390 2024-07-26T13:44:09.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.560+0000 7f8284400740 1 -- 172.21.15.100:0/2467609470 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8270004050 con 0x55ddb2c0a390 2024-07-26T13:44:09.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.561+0000 7f8256ffd640 1 -- 172.21.15.100:0/2467609470 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f826c0046c0 con 0x55ddb2c0a390 2024-07-26T13:44:09.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.561+0000 7f8284400740 10 client.16524 fetch_fsmap learned FSMap version 37 2024-07-26T13:44:09.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.561+0000 7f8284400740 1 -- 172.21.15.100:0/2467609470 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ddb2fef110 con 0x55ddb2c0a390 2024-07-26T13:44:09.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.561+0000 7f8256ffd640 1 -- 172.21.15.100:0/2467609470 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8270004050 con 0x55ddb2c0a390 2024-07-26T13:44:09.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.561+0000 7f8256ffd640 1 -- 172.21.15.100:0/2467609470 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f826c004fd0 con 0x55ddb2c0a390 2024-07-26T13:44:09.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.561+0000 7f8284400740 10 client.16524 fetch_fsmap finished waiting for FSMap version 37 2024-07-26T13:44:09.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.561+0000 7f8284400740 10 client.16524 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-07-26T13:44:09.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.561+0000 7f8284400740 20 client.16524 populate_metadata read hostname 'smithi100' 2024-07-26T13:44:09.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.561+0000 7f8284400740 1 --2- 172.21.15.100:0/2467609470 >> [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] conn(0x55ddb2be99f0 0x55ddb2ff26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:09.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.561+0000 7f8284400740 4 client.16524 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:44:09.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.561+0000 7f8284400740 1 -- 172.21.15.100:0/2467609470 --> [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddb2f48b00 con 0x55ddb2be99f0 2024-07-26T13:44:09.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.561+0000 7f82777fe640 1 --2- 172.21.15.100:0/2467609470 >> [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] conn(0x55ddb2be99f0 0x55ddb2ff26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:09.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.562+0000 7f8256ffd640 10 client.16524 ms_handle_connect on v2:172.21.15.138:6834/3290105300 2024-07-26T13:44:09.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.562+0000 7f82777fe640 1 --2- 172.21.15.100:0/2467609470 >> [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] conn(0x55ddb2be99f0 0x55ddb2ff26c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ada7bf7e897dd562 server_cookie=6c6ba666ed46863b in_seq=0 out_seq=0 2024-07-26T13:44:09.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.563+0000 7f8256ffd640 1 -- 172.21.15.100:0/2467609470 <== mds.2 v2:172.21.15.138:6834/3290105300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ddb2f48b00 con 0x55ddb2be99f0 2024-07-26T13:44:09.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.563+0000 7f8256ffd640 10 client.16524 handle_command_reply: tid=0 2024-07-26T13:44:09.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.563+0000 7f8284400740 1 client.16524 shutdown 2024-07-26T13:44:09.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.563+0000 7f8284400740 1 -- 172.21.15.100:0/2467609470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddb2c0a390 msgr2=0x55ddb2fe0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:09.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.563+0000 7f8284400740 1 --2- 172.21.15.100:0/2467609470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddb2c0a390 0x55ddb2fe0a10 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f826c0aafc0 tx=0x7f826c0a6f60 comp rx=0 tx=0).stop 2024-07-26T13:44:09.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.563+0000 7f8284400740 1 -- 172.21.15.100:0/2467609470 shutdown_connections 2024-07-26T13:44:09.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.563+0000 7f8284400740 1 --2- 172.21.15.100:0/2467609470 >> [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] conn(0x55ddb2be99f0 0x55ddb2ff26c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:09.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.563+0000 7f8284400740 1 --2- 172.21.15.100:0/2467609470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddb2c2b070 0x55ddb2fec3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:09.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.563+0000 7f8284400740 1 --2- 172.21.15.100:0/2467609470 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddb2c0a390 0x55ddb2fe0a10 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:09.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.563+0000 7f8284400740 1 --2- 172.21.15.100:0/2467609470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddb2c021b0 0x55ddb2fe04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:09.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.563+0000 7f8256ffd640 0 client.16524 ms_handle_reset on v2:172.21.15.138:6834/3290105300 2024-07-26T13:44:09.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.563+0000 7f8284400740 1 -- 172.21.15.100:0/2467609470 >> 172.21.15.100:0/2467609470 conn(0x55ddb2c1f460 msgr2=0x55ddb2fd3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:09.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.563+0000 7f8284400740 1 -- 172.21.15.100:0/2467609470 shutdown_connections 2024-07-26T13:44:09.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.563+0000 7f8284400740 1 -- 172.21.15.100:0/2467609470 wait complete. 2024-07-26T13:44:09.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.563+0000 7f8284400740 20 client.16524 trim_cache size 0 max 16384 2024-07-26T13:44:09.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.576+0000 7f8284400740 1 Processor -- start 2024-07-26T13:44:09.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.576+0000 7f8284400740 1 -- start start 2024-07-26T13:44:09.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.576+0000 7f8284400740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddb2c021b0 0x55ddb2fce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:09.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.576+0000 7f8284400740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddb2c0a390 0x55ddb2fec1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:09.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.576+0000 7f8284400740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddb2c2b070 0x55ddb2fd1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:09.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.576+0000 7f8284400740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f82781af8c0 con 0x55ddb2c0a390 2024-07-26T13:44:09.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.576+0000 7f8284400740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82781ad130 con 0x55ddb2c021b0 2024-07-26T13:44:09.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.576+0000 7f8284400740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f82781aaf80 con 0x55ddb2c2b070 2024-07-26T13:44:09.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.576+0000 7f8276ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddb2c0a390 0x55ddb2fec1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:09.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.576+0000 7f8276ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddb2c0a390 0x55ddb2fec1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37082/0 (socket says 172.21.15.100:37082) 2024-07-26T13:44:09.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.576+0000 7f8276ffd640 1 -- 172.21.15.100:0/2832974681 learned_addr learned my addr 172.21.15.100:0/2832974681 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:09.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.576+0000 7f82777fe640 1 --2- 172.21.15.100:0/2832974681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddb2c021b0 0x55ddb2fce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:09.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.576+0000 7f8276ffd640 1 -- 172.21.15.100:0/2832974681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddb2c2b070 msgr2=0x55ddb2fd1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:09.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.576+0000 7f8277fff640 1 --2- 172.21.15.100:0/2832974681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddb2c2b070 0x55ddb2fd1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:09.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.576+0000 7f8276ffd640 1 --2- 172.21.15.100:0/2832974681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddb2c2b070 0x55ddb2fd1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:09.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.576+0000 7f8276ffd640 1 -- 172.21.15.100:0/2832974681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddb2c021b0 msgr2=0x55ddb2fce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:09.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.576+0000 7f8276ffd640 1 --2- 172.21.15.100:0/2832974681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddb2c021b0 0x55ddb2fce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:09.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.576+0000 7f8276ffd640 1 -- 172.21.15.100:0/2832974681 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddb2fdaee0 con 0x55ddb2c0a390 2024-07-26T13:44:09.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.576+0000 7f82777fe640 1 --2- 172.21.15.100:0/2832974681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddb2c021b0 0x55ddb2fce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:44:09.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.576+0000 7f8277fff640 1 --2- 172.21.15.100:0/2832974681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddb2c2b070 0x55ddb2fd1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:44:09.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.577+0000 7f8276ffd640 1 --2- 172.21.15.100:0/2832974681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddb2c0a390 0x55ddb2fec1d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f826c0e2170 tx=0x7f826c0a6ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d281dd6d4defae4 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:09.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.577+0000 7f825d7fa640 1 -- 172.21.15.100:0/2832974681 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c0aae00 con 0x55ddb2c0a390 2024-07-26T13:44:09.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.577+0000 7f825d7fa640 1 -- 172.21.15.100:0/2832974681 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f826c0cc2f0 con 0x55ddb2c0a390 2024-07-26T13:44:09.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.577+0000 7f825d7fa640 1 -- 172.21.15.100:0/2832974681 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c0a6c80 con 0x55ddb2c0a390 2024-07-26T13:44:09.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.578+0000 7f8284400740 1 -- 172.21.15.100:0/2832974681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddb2c0a390 msgr2=0x55ddb2fec1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:09.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.578+0000 7f8284400740 1 --2- 172.21.15.100:0/2832974681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddb2c0a390 0x55ddb2fec1d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f826c0e2170 tx=0x7f826c0a6ef0 comp rx=0 tx=0).stop 2024-07-26T13:44:09.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.578+0000 7f8284400740 1 -- 172.21.15.100:0/2832974681 shutdown_connections 2024-07-26T13:44:09.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.578+0000 7f8284400740 1 --2- 172.21.15.100:0/2832974681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddb2c2b070 0x55ddb2fd1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:09.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.578+0000 7f8284400740 1 --2- 172.21.15.100:0/2832974681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddb2c0a390 0x55ddb2fec1d0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:09.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.578+0000 7f8284400740 1 --2- 172.21.15.100:0/2832974681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddb2c021b0 0x55ddb2fce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:09.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.578+0000 7f8284400740 1 -- 172.21.15.100:0/2832974681 >> 172.21.15.100:0/2832974681 conn(0x55ddb2c1f460 msgr2=0x55ddb2fd15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:09.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.578+0000 7f8284400740 1 -- 172.21.15.100:0/2832974681 shutdown_connections 2024-07-26T13:44:09.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.578+0000 7f8284400740 1 -- 172.21.15.100:0/2832974681 wait complete. 2024-07-26T13:44:09.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.578+0000 7f8284400740 1 -- start start 2024-07-26T13:44:09.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.578+0000 7f8284400740 1 Processor -- start 2024-07-26T13:44:09.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.579+0000 7f8284400740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddb2c021b0 0x55ddb2fe5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:09.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.579+0000 7f8284400740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddb2c0a390 0x55ddb2fe5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:09.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.579+0000 7f8284400740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddb2c2b070 0x55ddb3052ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:09.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.579+0000 7f8284400740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f82781ab5f0 con 0x55ddb2c021b0 2024-07-26T13:44:09.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.579+0000 7f8284400740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82781aca60 con 0x55ddb2c0a390 2024-07-26T13:44:09.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.579+0000 7f8284400740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f82781abcc0 con 0x55ddb2c2b070 2024-07-26T13:44:09.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.579+0000 7f8284400740 10 client.0 _ll_register_callbacks cb 0x7f826c00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:44:09.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.579+0000 7f82777fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddb2c021b0 0x55ddb2fe5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:09.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.579+0000 7f82777fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddb2c021b0 0x55ddb2fe5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37090/0 (socket says 172.21.15.100:37090) 2024-07-26T13:44:09.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.579+0000 7f82777fe640 1 -- 172.21.15.100:0/683198777 learned_addr learned my addr 172.21.15.100:0/683198777 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:09.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.579+0000 7f8277fff640 1 --2- 172.21.15.100:0/683198777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddb2c2b070 0x55ddb3052ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:09.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.579+0000 7f82777fe640 1 -- 172.21.15.100:0/683198777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddb2c2b070 msgr2=0x55ddb3052ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:09.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.579+0000 7f8276ffd640 1 --2- 172.21.15.100:0/683198777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddb2c0a390 0x55ddb2fe5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:09.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.579+0000 7f82777fe640 1 --2- 172.21.15.100:0/683198777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddb2c2b070 0x55ddb3052ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:09.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.579+0000 7f82777fe640 1 -- 172.21.15.100:0/683198777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddb2c0a390 msgr2=0x55ddb2fe5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:09.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.579+0000 7f82777fe640 1 --2- 172.21.15.100:0/683198777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddb2c0a390 0x55ddb2fe5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:09.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.579+0000 7f82777fe640 1 -- 172.21.15.100:0/683198777 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddb3053530 con 0x55ddb2c021b0 2024-07-26T13:44:09.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.579+0000 7f8276ffd640 1 --2- 172.21.15.100:0/683198777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddb2c0a390 0x55ddb2fe5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:44:09.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.579+0000 7f82777fe640 1 --2- 172.21.15.100:0/683198777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddb2c021b0 0x55ddb2fe5220 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f8268009900 tx=0x7f8268009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d37bd859818cac5 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:09.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.579+0000 7f8256ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:44:09.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.579+0000 7f8256ffd640 1 -- 172.21.15.100:0/683198777 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8268054020 con 0x55ddb2c021b0 2024-07-26T13:44:09.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.579+0000 7f8256ffd640 1 -- 172.21.15.100:0/683198777 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f8268005240 con 0x55ddb2c021b0 2024-07-26T13:44:09.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.579+0000 7f8284400740 1 -- 172.21.15.100:0/683198777 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ddb2f445a0 con 0x55ddb2c021b0 2024-07-26T13:44:09.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.580+0000 7f8256ffd640 1 -- 172.21.15.100:0/683198777 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f826805f050 con 0x55ddb2c021b0 2024-07-26T13:44:09.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.580+0000 7f8284400740 10 client.16530 fetch_fsmap learned FSMap version 37 2024-07-26T13:44:09.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.580+0000 7f8284400740 1 -- 172.21.15.100:0/683198777 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ddb305a830 con 0x55ddb2c021b0 2024-07-26T13:44:09.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.580+0000 7f8256ffd640 1 -- 172.21.15.100:0/683198777 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ddb2f445a0 con 0x55ddb2c021b0 2024-07-26T13:44:09.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.580+0000 7f8256ffd640 1 -- 172.21.15.100:0/683198777 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f8268003710 con 0x55ddb2c021b0 2024-07-26T13:44:09.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.580+0000 7f8284400740 10 client.16530 fetch_fsmap finished waiting for FSMap version 37 2024-07-26T13:44:09.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.580+0000 7f8284400740 10 client.16530 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-07-26T13:44:09.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.580+0000 7f8284400740 20 client.16530 populate_metadata read hostname 'smithi100' 2024-07-26T13:44:09.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.580+0000 7f8284400740 1 --2- 172.21.15.100:0/683198777 >> [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] conn(0x55ddb2bbeda0 0x55ddb305d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:09.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.580+0000 7f8284400740 4 client.16530 mds_command: new command op to 24557 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-26T13:44:09.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.580+0000 7f8284400740 1 -- 172.21.15.100:0/683198777 --> [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ddb2f41ce0 con 0x55ddb2bbeda0 2024-07-26T13:44:09.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.580+0000 7f8276ffd640 1 --2- 172.21.15.100:0/683198777 >> [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] conn(0x55ddb2bbeda0 0x55ddb305d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:09.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.581+0000 7f8276ffd640 1 --2- 172.21.15.100:0/683198777 >> [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] conn(0x55ddb2bbeda0 0x55ddb305d050 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b7ae6df35ab26e3 server_cookie=5c94b91e5db983e1 in_seq=0 out_seq=0 2024-07-26T13:44:09.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.581+0000 7f8256ffd640 10 client.16530 ms_handle_connect on v2:172.21.15.138:6834/3290105300 2024-07-26T13:44:09.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.581+0000 7f8256ffd640 1 -- 172.21.15.100:0/683198777 <== mds.2 v2:172.21.15.138:6834/3290105300 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ddb2f41ce0 con 0x55ddb2bbeda0 2024-07-26T13:44:09.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.581+0000 7f8256ffd640 10 client.16530 handle_command_reply: tid=0 2024-07-26T13:44:09.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.581+0000 7f8284400740 1 client.16530 shutdown 2024-07-26T13:44:09.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.581+0000 7f8284400740 1 -- 172.21.15.100:0/683198777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddb2c021b0 msgr2=0x55ddb2fe5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:09.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.581+0000 7f8284400740 1 --2- 172.21.15.100:0/683198777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddb2c021b0 0x55ddb2fe5220 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f8268009900 tx=0x7f8268009260 comp rx=0 tx=0).stop 2024-07-26T13:44:09.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.581+0000 7f8284400740 1 -- 172.21.15.100:0/683198777 shutdown_connections 2024-07-26T13:44:09.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.581+0000 7f8284400740 1 --2- 172.21.15.100:0/683198777 >> [v2:172.21.15.138:6834/3290105300,v1:172.21.15.138:6835/3290105300] conn(0x55ddb2bbeda0 0x55ddb305d050 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:09.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.581+0000 7f8284400740 1 --2- 172.21.15.100:0/683198777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ddb2c2b070 0x55ddb3052ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:09.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.581+0000 7f8284400740 1 --2- 172.21.15.100:0/683198777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ddb2c0a390 0x55ddb2fe5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:09.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.581+0000 7f8284400740 1 --2- 172.21.15.100:0/683198777 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ddb2c021b0 0x55ddb2fe5220 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:09.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.581+0000 7f8256ffd640 0 client.16530 ms_handle_reset on v2:172.21.15.138:6834/3290105300 2024-07-26T13:44:09.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.581+0000 7f8284400740 1 -- 172.21.15.100:0/683198777 >> 172.21.15.100:0/683198777 conn(0x55ddb2c1f460 msgr2=0x55ddb2fe1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:09.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.581+0000 7f8284400740 1 -- 172.21.15.100:0/683198777 shutdown_connections 2024-07-26T13:44:09.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.581+0000 7f8284400740 1 -- 172.21.15.100:0/683198777 wait complete. 2024-07-26T13:44:09.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.581+0000 7f8284400740 20 client.16530 trim_cache size 0 max 16384 2024-07-26T13:44:09.588 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-07-26T13:44:09.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.582+0000 7f827de7c640 1 -- 172.21.15.100:0/1028350314 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f82600b3d40 msgr2=0x7f82600b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:09.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.582+0000 7f827de7c640 1 --2- 172.21.15.100:0/1028350314 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f82600b3d40 0x7f82600b6250 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8268031440 tx=0x7f8268032ed0 comp rx=0 tx=0).stop 2024-07-26T13:44:09.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.582+0000 7f827de7c640 1 -- 172.21.15.100:0/1028350314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8278107dd0 msgr2=0x7f8278199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:09.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.582+0000 7f827de7c640 1 --2- 172.21.15.100:0/1028350314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8278107dd0 0x7f8278199310 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f826c00e9c0 tx=0x7f826c00ee80 comp rx=0 tx=0).stop 2024-07-26T13:44:09.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.582+0000 7f827de7c640 1 -- 172.21.15.100:0/1028350314 shutdown_connections 2024-07-26T13:44:09.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.582+0000 7f827de7c640 1 --2- 172.21.15.100:0/1028350314 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f82600b3d40 0x7f82600b6250 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:09.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.582+0000 7f827de7c640 1 --2- 172.21.15.100:0/1028350314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8278108720 0x7f827819e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:09.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.582+0000 7f827de7c640 1 --2- 172.21.15.100:0/1028350314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8278107dd0 0x7f8278199310 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:09.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.582+0000 7f827de7c640 1 --2- 172.21.15.100:0/1028350314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8278102490 0x7f8278198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:09.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.582+0000 7f827de7c640 1 -- 172.21.15.100:0/1028350314 >> 172.21.15.100:0/1028350314 conn(0x7f8278079160 msgr2=0x7f8278100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:09.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.582+0000 7f827de7c640 1 -- 172.21.15.100:0/1028350314 shutdown_connections 2024-07-26T13:44:09.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:09.582+0000 7f827de7c640 1 -- 172.21.15.100:0/1028350314 wait complete. 2024-07-26T13:44:09.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:09 smithi138 ceph-mon[36954]: pgmap v486: 97 pgs: 97 active+clean; 62 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 1.8 MiB/s wr, 92 op/s 2024-07-26T13:44:09.974 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:09 smithi100 ceph-mon[32599]: pgmap v486: 97 pgs: 97 active+clean; 62 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 1.8 MiB/s wr, 92 op/s 2024-07-26T13:44:10.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:09 smithi167 ceph-mon[37140]: pgmap v486: 97 pgs: 97 active+clean; 62 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 1.8 MiB/s wr, 92 op/s 2024-07-26T13:44:10.412 INFO:tasks.workunit.client.0.smithi100.stdout: 1 145991 0.00 MB/sec execute 24 sec latency 24311.194 ms 2024-07-26T13:44:10.501 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-26T13:44:10.758 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:44:11.412 INFO:tasks.workunit.client.0.smithi100.stdout: 1 145991 0.00 MB/sec execute 25 sec latency 25311.302 ms 2024-07-26T13:44:11.791 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:44:11.815 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:11 smithi100 ceph-mon[32599]: pgmap v487: 97 pgs: 97 active+clean; 62 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 1.8 MiB/s wr, 96 op/s 2024-07-26T13:44:11.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:11 smithi138 ceph-mon[36954]: pgmap v487: 97 pgs: 97 active+clean; 62 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 1.8 MiB/s wr, 96 op/s 2024-07-26T13:44:12.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:11 smithi167 ceph-mon[37140]: pgmap v487: 97 pgs: 97 active+clean; 62 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 1.8 MiB/s wr, 96 op/s 2024-07-26T13:44:12.182 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:44:12.412 INFO:tasks.workunit.client.0.smithi100.stdout: 1 145991 0.00 MB/sec execute 26 sec latency 26311.339 ms 2024-07-26T13:44:12.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.701+0000 7f8662901640 1 -- 172.21.15.100:0/1866096160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f865c102510 msgr2=0x7f865c102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:12.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.701+0000 7f8662901640 1 --2- 172.21.15.100:0/1866096160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f865c102510 0x7f865c102920 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f864c009900 tx=0x7f864c02a410 comp rx=0 tx=0).stop 2024-07-26T13:44:12.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.701+0000 7f8662901640 1 -- 172.21.15.100:0/1866096160 shutdown_connections 2024-07-26T13:44:12.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.701+0000 7f8662901640 1 --2- 172.21.15.100:0/1866096160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f865c1087a0 0x7f865c10efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:12.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.701+0000 7f8662901640 1 --2- 172.21.15.100:0/1866096160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f865c107e50 0x7f865c108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:12.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.701+0000 7f8662901640 1 --2- 172.21.15.100:0/1866096160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f865c102510 0x7f865c102920 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:12.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.701+0000 7f8662901640 1 -- 172.21.15.100:0/1866096160 >> 172.21.15.100:0/1866096160 conn(0x7f865c0791a0 msgr2=0x7f865c0795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:12.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.701+0000 7f8662901640 1 -- 172.21.15.100:0/1866096160 shutdown_connections 2024-07-26T13:44:12.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.701+0000 7f8662901640 1 -- 172.21.15.100:0/1866096160 wait complete. 2024-07-26T13:44:12.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.702+0000 7f8662901640 1 Processor -- start 2024-07-26T13:44:12.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.702+0000 7f8662901640 1 -- start start 2024-07-26T13:44:12.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.702+0000 7f8662901640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f865c102510 0x7f865c19ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:12.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.702+0000 7f8662901640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f865c107e50 0x7f865c19b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:12.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.702+0000 7f8662901640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f865c1087a0 0x7f865c19b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:12.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.702+0000 7f8662901640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f865c112470 con 0x7f865c1087a0 2024-07-26T13:44:12.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.702+0000 7f8662901640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f865c1122f0 con 0x7f865c102510 2024-07-26T13:44:12.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.702+0000 7f8662901640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f865c1125f0 con 0x7f865c107e50 2024-07-26T13:44:12.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.703+0000 7f8660e9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f865c1087a0 0x7f865c19b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:12.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.703+0000 7f8660e9e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f865c1087a0 0x7f865c19b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37108/0 (socket says 172.21.15.100:37108) 2024-07-26T13:44:12.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.703+0000 7f8660e9e640 1 -- 172.21.15.100:0/908957636 learned_addr learned my addr 172.21.15.100:0/908957636 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:12.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.703+0000 7f865bfff640 1 --2- 172.21.15.100:0/908957636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f865c102510 0x7f865c19ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:12.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.703+0000 7f865b7fe640 1 --2- 172.21.15.100:0/908957636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f865c107e50 0x7f865c19b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:12.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.703+0000 7f8660e9e640 1 -- 172.21.15.100:0/908957636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f865c107e50 msgr2=0x7f865c19b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:12.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.703+0000 7f8660e9e640 1 --2- 172.21.15.100:0/908957636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f865c107e50 0x7f865c19b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:12.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.703+0000 7f8660e9e640 1 -- 172.21.15.100:0/908957636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f865c102510 msgr2=0x7f865c19ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:12.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.703+0000 7f8660e9e640 1 --2- 172.21.15.100:0/908957636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f865c102510 0x7f865c19ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:12.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.703+0000 7f8660e9e640 1 -- 172.21.15.100:0/908957636 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f865c1a6010 con 0x7f865c1087a0 2024-07-26T13:44:12.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.703+0000 7f865b7fe640 1 --2- 172.21.15.100:0/908957636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f865c107e50 0x7f865c19b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:44:12.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.703+0000 7f8660e9e640 1 --2- 172.21.15.100:0/908957636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f865c1087a0 0x7f865c19b780 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f865400eaa0 tx=0x7f865400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:12.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.703+0000 7f86597fa640 1 -- 172.21.15.100:0/908957636 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865400cdb0 con 0x7f865c1087a0 2024-07-26T13:44:12.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.703+0000 7f8662901640 1 -- 172.21.15.100:0/908957636 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f865c1a6300 con 0x7f865c1087a0 2024-07-26T13:44:12.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.703+0000 7f86597fa640 1 -- 172.21.15.100:0/908957636 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8654005020 con 0x7f865c1087a0 2024-07-26T13:44:12.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.703+0000 7f86597fa640 1 -- 172.21.15.100:0/908957636 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86540116d0 con 0x7f865c1087a0 2024-07-26T13:44:12.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.703+0000 7f8662901640 1 -- 172.21.15.100:0/908957636 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f865c1a6840 con 0x7f865c1087a0 2024-07-26T13:44:12.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.706+0000 7f86597fa640 1 -- 172.21.15.100:0/908957636 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f86540051d0 con 0x7f865c1087a0 2024-07-26T13:44:12.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.706+0000 7f8668e85740 1 Processor -- start 2024-07-26T13:44:12.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.706+0000 7f8668e85740 1 -- start start 2024-07-26T13:44:12.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.706+0000 7f8668e85740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b6042d070 0x557b607d2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:12.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.706+0000 7f8668e85740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b604041b0 0x557b607d2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:12.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.706+0000 7f8668e85740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b6040c390 0x557b607d2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:12.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.706+0000 7f8668e85740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x557b60748bb0 con 0x557b6040c390 2024-07-26T13:44:12.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.706+0000 7f8668e85740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557b607380b0 con 0x557b6042d070 2024-07-26T13:44:12.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.706+0000 7f8668e85740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557b6073c0f0 con 0x557b604041b0 2024-07-26T13:44:12.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.707+0000 7f865b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b6040c390 0x557b607d2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:12.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.707+0000 7f865b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b6040c390 0x557b607d2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37124/0 (socket says 172.21.15.100:37124) 2024-07-26T13:44:12.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.707+0000 7f865b7fe640 1 -- 172.21.15.100:0/65540206 learned_addr learned my addr 172.21.15.100:0/65540206 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:12.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.707+0000 7f865b7fe640 1 --2- 172.21.15.100:0/65540206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b6042d070 0x557b607d2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:12.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.707+0000 7f865bfff640 1 --2- 172.21.15.100:0/65540206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b604041b0 0x557b607d2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:12.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.707+0000 7f86597fa640 1 --2- 172.21.15.100:0/908957636 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f86440b3d40 0x7f86440b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:12.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.707+0000 7f865b7fe640 1 -- 172.21.15.100:0/65540206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b604041b0 msgr2=0x557b607d2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:12.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.707+0000 7f865b7fe640 1 --2- 172.21.15.100:0/65540206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b604041b0 0x557b607d2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:12.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.707+0000 7f865b7fe640 1 -- 172.21.15.100:0/65540206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b6042d070 msgr2=0x557b607d2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:12.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.707+0000 7f865b7fe640 1 --2- 172.21.15.100:0/65540206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b6042d070 0x557b607d2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:12.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.707+0000 7f865b7fe640 1 -- 172.21.15.100:0/65540206 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b606deb90 con 0x557b6040c390 2024-07-26T13:44:12.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.707+0000 7f8660e9e640 1 --2- 172.21.15.100:0/908957636 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f86440b3d40 0x7f86440b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:12.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.707+0000 7f86597fa640 1 -- 172.21.15.100:0/908957636 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f86540e41f0 con 0x7f865c1087a0 2024-07-26T13:44:12.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.707+0000 7f865b7fe640 1 --2- 172.21.15.100:0/65540206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b6040c390 0x557b607d2ff0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f86500149f0 tx=0x7f865000c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32b10d59c103a150 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:12.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.707+0000 7f86427fc640 1 -- 172.21.15.100:0/65540206 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86500142d0 con 0x557b6040c390 2024-07-26T13:44:12.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.707+0000 7f86427fc640 1 -- 172.21.15.100:0/65540206 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86500054d0 con 0x557b6040c390 2024-07-26T13:44:12.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.707+0000 7f86427fc640 1 -- 172.21.15.100:0/65540206 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865000f760 con 0x557b6040c390 2024-07-26T13:44:12.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.708+0000 7f8668e85740 1 -- 172.21.15.100:0/65540206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b6040c390 msgr2=0x557b607d2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:12.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.708+0000 7f8660e9e640 1 --2- 172.21.15.100:0/908957636 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f86440b3d40 0x7f86440b6250 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f86540aad60 tx=0x7f8654092040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:12.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.708+0000 7f8668e85740 1 --2- 172.21.15.100:0/65540206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b6040c390 0x557b607d2ff0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f86500149f0 tx=0x7f865000c510 comp rx=0 tx=0).stop 2024-07-26T13:44:12.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.708+0000 7f8668e85740 1 -- 172.21.15.100:0/65540206 shutdown_connections 2024-07-26T13:44:12.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.708+0000 7f8668e85740 1 --2- 172.21.15.100:0/65540206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b6040c390 0x557b607d2ff0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:12.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.708+0000 7f8668e85740 1 --2- 172.21.15.100:0/65540206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b604041b0 0x557b607d2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:12.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.708+0000 7f8668e85740 1 --2- 172.21.15.100:0/65540206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b6042d070 0x557b607d2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:12.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.708+0000 7f8668e85740 1 -- 172.21.15.100:0/65540206 >> 172.21.15.100:0/65540206 conn(0x557b60421460 msgr2=0x557b607d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:12.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.708+0000 7f8668e85740 1 -- 172.21.15.100:0/65540206 shutdown_connections 2024-07-26T13:44:12.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.708+0000 7f8668e85740 1 -- 172.21.15.100:0/65540206 wait complete. 2024-07-26T13:44:12.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.709+0000 7f8668e85740 1 -- start start 2024-07-26T13:44:12.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.709+0000 7f8668e85740 1 Processor -- start 2024-07-26T13:44:12.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.709+0000 7f8668e85740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b604041b0 0x557b607de100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:12.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.709+0000 7f8668e85740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b6040c390 0x557b607de640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:12.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.709+0000 7f8668e85740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b6042d070 0x557b607e9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:12.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.709+0000 7f8668e85740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x557b6059b700 con 0x557b6042d070 2024-07-26T13:44:12.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.709+0000 7f8668e85740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557b606da9e0 con 0x557b604041b0 2024-07-26T13:44:12.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.709+0000 7f8668e85740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557b607e0cf0 con 0x557b6040c390 2024-07-26T13:44:12.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.709+0000 7f8668e85740 10 client.0 _ll_register_callbacks cb 0x557b606f54f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:44:12.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.710+0000 7f865b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b6042d070 0x557b607e9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:12.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.710+0000 7f865b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b6042d070 0x557b607e9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37140/0 (socket says 172.21.15.100:37140) 2024-07-26T13:44:12.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.710+0000 7f865b7fe640 1 -- 172.21.15.100:0/2013753233 learned_addr learned my addr 172.21.15.100:0/2013753233 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:12.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.710+0000 7f865bfff640 1 --2- 172.21.15.100:0/2013753233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b6040c390 0x557b607de640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:12.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.710+0000 7f865b7fe640 1 --2- 172.21.15.100:0/2013753233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b604041b0 0x557b607de100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:12.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.710+0000 7f865b7fe640 1 -- 172.21.15.100:0/2013753233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b6040c390 msgr2=0x557b607de640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:12.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.710+0000 7f865b7fe640 1 --2- 172.21.15.100:0/2013753233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b6040c390 0x557b607de640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:12.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.710+0000 7f865b7fe640 1 -- 172.21.15.100:0/2013753233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b604041b0 msgr2=0x557b607de100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:12.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.710+0000 7f865b7fe640 1 --2- 172.21.15.100:0/2013753233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b604041b0 0x557b607de100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:12.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.710+0000 7f865b7fe640 1 -- 172.21.15.100:0/2013753233 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b60498650 con 0x557b6042d070 2024-07-26T13:44:12.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.710+0000 7f865b7fe640 1 --2- 172.21.15.100:0/2013753233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b6042d070 0x557b607e9fe0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f865000c4b0 tx=0x7f8650004b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf41cf51f647b64 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:12.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.710+0000 7f863b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:44:12.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.710+0000 7f863b7fe640 1 -- 172.21.15.100:0/2013753233 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86500048c0 con 0x557b6042d070 2024-07-26T13:44:12.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.710+0000 7f863b7fe640 1 -- 172.21.15.100:0/2013753233 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f8650004e70 con 0x557b6042d070 2024-07-26T13:44:12.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.710+0000 7f8668e85740 1 -- 172.21.15.100:0/2013753233 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86500054d0 con 0x557b6042d070 2024-07-26T13:44:12.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.710+0000 7f863b7fe640 1 -- 172.21.15.100:0/2013753233 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8650005070 con 0x557b6042d070 2024-07-26T13:44:12.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.710+0000 7f8668e85740 10 client.16542 fetch_fsmap learned FSMap version 37 2024-07-26T13:44:12.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.711+0000 7f8668e85740 1 -- 172.21.15.100:0/2013753233 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557b607eccd0 con 0x557b6042d070 2024-07-26T13:44:12.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.711+0000 7f863b7fe640 1 -- 172.21.15.100:0/2013753233 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86500054d0 con 0x557b6042d070 2024-07-26T13:44:12.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.711+0000 7f863b7fe640 1 -- 172.21.15.100:0/2013753233 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f86500117e0 con 0x557b6042d070 2024-07-26T13:44:12.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.711+0000 7f8668e85740 10 client.16542 fetch_fsmap finished waiting for FSMap version 37 2024-07-26T13:44:12.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.711+0000 7f8668e85740 10 client.16542 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-26T13:44:12.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.711+0000 7f8668e85740 20 client.16542 populate_metadata read hostname 'smithi100' 2024-07-26T13:44:12.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.711+0000 7f8668e85740 1 --2- 172.21.15.100:0/2013753233 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x557b603eb9f0 0x557b607f0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:12.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.711+0000 7f8668e85740 4 client.16542 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:44:12.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.711+0000 7f8668e85740 1 -- 172.21.15.100:0/2013753233 --> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b6074ab00 con 0x557b603eb9f0 2024-07-26T13:44:12.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.711+0000 7f865bfff640 1 --2- 172.21.15.100:0/2013753233 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x557b603eb9f0 0x557b607f0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:12.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.712+0000 7f865bfff640 1 --2- 172.21.15.100:0/2013753233 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x557b603eb9f0 0x557b607f0030 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cc8ee4ec40765288 server_cookie=b8e09b71607ace23 in_seq=0 out_seq=0 2024-07-26T13:44:12.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.712+0000 7f863b7fe640 10 client.16542 ms_handle_connect on v2:172.21.15.138:6838/1580925471 2024-07-26T13:44:12.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.712+0000 7f863b7fe640 1 -- 172.21.15.100:0/2013753233 <== mds.1 v2:172.21.15.138:6838/1580925471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557b6074ab00 con 0x557b603eb9f0 2024-07-26T13:44:12.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.712+0000 7f863b7fe640 10 client.16542 handle_command_reply: tid=0 2024-07-26T13:44:12.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.712+0000 7f8668e85740 1 client.16542 shutdown 2024-07-26T13:44:12.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.713+0000 7f8668e85740 1 -- 172.21.15.100:0/2013753233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b6042d070 msgr2=0x557b607e9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:12.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.713+0000 7f8668e85740 1 --2- 172.21.15.100:0/2013753233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b6042d070 0x557b607e9fe0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f865000c4b0 tx=0x7f8650004b80 comp rx=0 tx=0).stop 2024-07-26T13:44:12.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.713+0000 7f8668e85740 1 -- 172.21.15.100:0/2013753233 shutdown_connections 2024-07-26T13:44:12.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.713+0000 7f8668e85740 1 --2- 172.21.15.100:0/2013753233 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x557b603eb9f0 0x557b607f0030 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:12.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.713+0000 7f8668e85740 1 --2- 172.21.15.100:0/2013753233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b6042d070 0x557b607e9fe0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:12.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.713+0000 7f8668e85740 1 --2- 172.21.15.100:0/2013753233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b6040c390 0x557b607de640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:12.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.713+0000 7f8668e85740 1 --2- 172.21.15.100:0/2013753233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b604041b0 0x557b607de100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:12.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.713+0000 7f8668e85740 1 -- 172.21.15.100:0/2013753233 >> 172.21.15.100:0/2013753233 conn(0x557b60421460 msgr2=0x557b607d5750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:12.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.713+0000 7f863b7fe640 0 client.16542 ms_handle_reset on v2:172.21.15.138:6838/1580925471 2024-07-26T13:44:12.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.713+0000 7f8668e85740 1 -- 172.21.15.100:0/2013753233 shutdown_connections 2024-07-26T13:44:12.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.713+0000 7f8668e85740 1 -- 172.21.15.100:0/2013753233 wait complete. 2024-07-26T13:44:12.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.713+0000 7f8668e85740 20 client.16542 trim_cache size 0 max 16384 2024-07-26T13:44:12.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.726+0000 7f8668e85740 1 Processor -- start 2024-07-26T13:44:12.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.726+0000 7f8668e85740 1 -- start start 2024-07-26T13:44:12.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.726+0000 7f8668e85740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b604041b0 0x557b607d0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:12.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.726+0000 7f8668e85740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b6040c390 0x557b607e9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:12.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.726+0000 7f8668e85740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b6042d070 0x557b607ea310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:12.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.726+0000 7f8668e85740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f865c1b3cc0 con 0x557b6042d070 2024-07-26T13:44:12.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.726+0000 7f8668e85740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f865c1b1530 con 0x557b6040c390 2024-07-26T13:44:12.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.726+0000 7f8668e85740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f865c1af380 con 0x557b604041b0 2024-07-26T13:44:12.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.726+0000 7f865b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b6042d070 0x557b607ea310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:12.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.726+0000 7f865b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b6042d070 0x557b607ea310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37152/0 (socket says 172.21.15.100:37152) 2024-07-26T13:44:12.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.726+0000 7f865b7fe640 1 -- 172.21.15.100:0/4002437747 learned_addr learned my addr 172.21.15.100:0/4002437747 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:12.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.726+0000 7f865bfff640 1 --2- 172.21.15.100:0/4002437747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b6040c390 0x557b607e9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:12.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.726+0000 7f865b7fe640 1 --2- 172.21.15.100:0/4002437747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b604041b0 0x557b607d0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:12.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.726+0000 7f865b7fe640 1 -- 172.21.15.100:0/4002437747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b604041b0 msgr2=0x557b607d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:12.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.726+0000 7f865b7fe640 1 --2- 172.21.15.100:0/4002437747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b604041b0 0x557b607d0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:12.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.726+0000 7f865b7fe640 1 -- 172.21.15.100:0/4002437747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b6040c390 msgr2=0x557b607e9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:12.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.726+0000 7f865b7fe640 1 --2- 172.21.15.100:0/4002437747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b6040c390 0x557b607e9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:12.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.726+0000 7f865b7fe640 1 -- 172.21.15.100:0/4002437747 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b603ebc20 con 0x557b6042d070 2024-07-26T13:44:12.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.727+0000 7f865b7fe640 1 --2- 172.21.15.100:0/4002437747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b6042d070 0x557b607ea310 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f86500053e0 tx=0x7f8650000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86df48343e7fd0eb server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:12.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.727+0000 7f8641ffb640 1 -- 172.21.15.100:0/4002437747 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865000c990 con 0x557b6042d070 2024-07-26T13:44:12.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.727+0000 7f8641ffb640 1 -- 172.21.15.100:0/4002437747 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8650005450 con 0x557b6042d070 2024-07-26T13:44:12.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.727+0000 7f8641ffb640 1 -- 172.21.15.100:0/4002437747 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865000f730 con 0x557b6042d070 2024-07-26T13:44:12.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.727+0000 7f8668e85740 1 -- 172.21.15.100:0/4002437747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b6042d070 msgr2=0x557b607ea310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:12.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.727+0000 7f8668e85740 1 --2- 172.21.15.100:0/4002437747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b6042d070 0x557b607ea310 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f86500053e0 tx=0x7f8650000bc0 comp rx=0 tx=0).stop 2024-07-26T13:44:12.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.728+0000 7f8668e85740 1 -- 172.21.15.100:0/4002437747 shutdown_connections 2024-07-26T13:44:12.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.728+0000 7f8668e85740 1 --2- 172.21.15.100:0/4002437747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b6042d070 0x557b607ea310 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:12.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.728+0000 7f8668e85740 1 --2- 172.21.15.100:0/4002437747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b6040c390 0x557b607e9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:12.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.728+0000 7f8668e85740 1 --2- 172.21.15.100:0/4002437747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b604041b0 0x557b607d0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:12.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.728+0000 7f8668e85740 1 -- 172.21.15.100:0/4002437747 >> 172.21.15.100:0/4002437747 conn(0x557b60421460 msgr2=0x557b607cfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:12.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.728+0000 7f8668e85740 1 -- 172.21.15.100:0/4002437747 shutdown_connections 2024-07-26T13:44:12.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.728+0000 7f8668e85740 1 -- 172.21.15.100:0/4002437747 wait complete. 2024-07-26T13:44:12.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.728+0000 7f8668e85740 1 -- start start 2024-07-26T13:44:12.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.728+0000 7f8668e85740 1 Processor -- start 2024-07-26T13:44:12.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.729+0000 7f8668e85740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b604041b0 0x557b607e62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:12.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.729+0000 7f8668e85740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b6040c390 0x557b607e17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:12.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.729+0000 7f8668e85740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b603c0da0 0x557b607e1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:12.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.729+0000 7f8668e85740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f865c1af9f0 con 0x557b6040c390 2024-07-26T13:44:12.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.729+0000 7f8668e85740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f865c1b0e60 con 0x557b604041b0 2024-07-26T13:44:12.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.729+0000 7f8668e85740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f865c1b00c0 con 0x557b603c0da0 2024-07-26T13:44:12.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.729+0000 7f8668e85740 10 client.0 _ll_register_callbacks cb 0x7f865000cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:44:12.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.729+0000 7f865bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b6040c390 0x557b607e17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:12.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.729+0000 7f865bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b6040c390 0x557b607e17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37162/0 (socket says 172.21.15.100:37162) 2024-07-26T13:44:12.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.729+0000 7f865bfff640 1 -- 172.21.15.100:0/639547976 learned_addr learned my addr 172.21.15.100:0/639547976 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:12.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.729+0000 7f865b7fe640 1 --2- 172.21.15.100:0/639547976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b603c0da0 0x557b607e1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:12.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.729+0000 7f865b7fe640 1 --2- 172.21.15.100:0/639547976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b604041b0 0x557b607e62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:12.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.729+0000 7f865bfff640 1 -- 172.21.15.100:0/639547976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b603c0da0 msgr2=0x557b607e1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:12.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.729+0000 7f865bfff640 1 --2- 172.21.15.100:0/639547976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b603c0da0 0x557b607e1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:12.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.729+0000 7f865bfff640 1 -- 172.21.15.100:0/639547976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b604041b0 msgr2=0x557b607e62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:12.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.729+0000 7f865bfff640 1 --2- 172.21.15.100:0/639547976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b604041b0 0x557b607e62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:12.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.729+0000 7f865bfff640 1 -- 172.21.15.100:0/639547976 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b607e2270 con 0x557b6040c390 2024-07-26T13:44:12.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.729+0000 7f865b7fe640 1 --2- 172.21.15.100:0/639547976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b604041b0 0x557b607e62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:44:12.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.729+0000 7f865bfff640 1 --2- 172.21.15.100:0/639547976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b6040c390 0x557b607e17f0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f864c000c00 tx=0x7f864c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8efd721906e1541 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:12.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.729+0000 7f863b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:44:12.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.729+0000 7f863b7fe640 1 -- 172.21.15.100:0/639547976 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864c035040 con 0x557b6040c390 2024-07-26T13:44:12.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.729+0000 7f863b7fe640 1 -- 172.21.15.100:0/639547976 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f864c005040 con 0x557b6040c390 2024-07-26T13:44:12.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.729+0000 7f8668e85740 1 -- 172.21.15.100:0/639547976 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8650005450 con 0x557b6040c390 2024-07-26T13:44:12.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.730+0000 7f863b7fe640 1 -- 172.21.15.100:0/639547976 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f864c055050 con 0x557b6040c390 2024-07-26T13:44:12.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.730+0000 7f8668e85740 10 client.16548 fetch_fsmap learned FSMap version 37 2024-07-26T13:44:12.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.730+0000 7f8668e85740 1 -- 172.21.15.100:0/639547976 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557b60858280 con 0x557b6040c390 2024-07-26T13:44:12.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.730+0000 7f863b7fe640 1 -- 172.21.15.100:0/639547976 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8650005450 con 0x557b6040c390 2024-07-26T13:44:12.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.730+0000 7f863b7fe640 1 -- 172.21.15.100:0/639547976 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f864c0035f0 con 0x557b6040c390 2024-07-26T13:44:12.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.730+0000 7f8668e85740 10 client.16548 fetch_fsmap finished waiting for FSMap version 37 2024-07-26T13:44:12.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.730+0000 7f8668e85740 10 client.16548 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-26T13:44:12.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.730+0000 7f8668e85740 20 client.16548 populate_metadata read hostname 'smithi100' 2024-07-26T13:44:12.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.730+0000 7f8668e85740 1 --2- 172.21.15.100:0/639547976 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x557b60858a30 0x557b6085ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:12.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.730+0000 7f8668e85740 4 client.16548 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-26T13:44:12.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.730+0000 7f8668e85740 1 -- 172.21.15.100:0/639547976 --> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557b607465a0 con 0x557b60858a30 2024-07-26T13:44:12.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.730+0000 7f865b7fe640 1 --2- 172.21.15.100:0/639547976 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x557b60858a30 0x557b6085ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:12.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.731+0000 7f865b7fe640 1 --2- 172.21.15.100:0/639547976 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x557b60858a30 0x557b6085ae50 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a2e18a6d7f0ac06b server_cookie=5b86f7f585e7b1f2 in_seq=0 out_seq=0 2024-07-26T13:44:12.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.731+0000 7f863b7fe640 10 client.16548 ms_handle_connect on v2:172.21.15.138:6838/1580925471 2024-07-26T13:44:12.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.731+0000 7f863b7fe640 1 -- 172.21.15.100:0/639547976 <== mds.1 v2:172.21.15.138:6838/1580925471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557b607465a0 con 0x557b60858a30 2024-07-26T13:44:12.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.731+0000 7f863b7fe640 10 client.16548 handle_command_reply: tid=0 2024-07-26T13:44:12.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.731+0000 7f8668e85740 1 client.16548 shutdown 2024-07-26T13:44:12.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.731+0000 7f8668e85740 1 -- 172.21.15.100:0/639547976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b6040c390 msgr2=0x557b607e17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:12.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.731+0000 7f8668e85740 1 --2- 172.21.15.100:0/639547976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b6040c390 0x557b607e17f0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f864c000c00 tx=0x7f864c004040 comp rx=0 tx=0).stop 2024-07-26T13:44:12.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.731+0000 7f8668e85740 1 -- 172.21.15.100:0/639547976 shutdown_connections 2024-07-26T13:44:12.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.731+0000 7f8668e85740 1 --2- 172.21.15.100:0/639547976 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x557b60858a30 0x557b6085ae50 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:12.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.731+0000 7f8668e85740 1 --2- 172.21.15.100:0/639547976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557b603c0da0 0x557b607e1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:12.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.731+0000 7f8668e85740 1 --2- 172.21.15.100:0/639547976 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557b6040c390 0x557b607e17f0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:12.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.731+0000 7f8668e85740 1 --2- 172.21.15.100:0/639547976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b604041b0 0x557b607e62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:12.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.731+0000 7f8668e85740 1 -- 172.21.15.100:0/639547976 >> 172.21.15.100:0/639547976 conn(0x557b60421460 msgr2=0x557b607d55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:12.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.731+0000 7f863b7fe640 0 client.16548 ms_handle_reset on v2:172.21.15.138:6838/1580925471 2024-07-26T13:44:12.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.731+0000 7f8668e85740 1 -- 172.21.15.100:0/639547976 shutdown_connections 2024-07-26T13:44:12.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.731+0000 7f8668e85740 1 -- 172.21.15.100:0/639547976 wait complete. 2024-07-26T13:44:12.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.731+0000 7f8668e85740 20 client.16548 trim_cache size 0 max 16384 2024-07-26T13:44:12.738 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-07-26T13:44:12.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.732+0000 7f8662901640 1 -- 172.21.15.100:0/908957636 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f86440b3d40 msgr2=0x7f86440b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:12.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.732+0000 7f8662901640 1 --2- 172.21.15.100:0/908957636 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f86440b3d40 0x7f86440b6250 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f86540aad60 tx=0x7f8654092040 comp rx=0 tx=0).stop 2024-07-26T13:44:12.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.732+0000 7f8662901640 1 -- 172.21.15.100:0/908957636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f865c1087a0 msgr2=0x7f865c19b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:12.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.732+0000 7f8662901640 1 --2- 172.21.15.100:0/908957636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f865c1087a0 0x7f865c19b780 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f865400eaa0 tx=0x7f865400ef60 comp rx=0 tx=0).stop 2024-07-26T13:44:12.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.732+0000 7f8662901640 1 -- 172.21.15.100:0/908957636 shutdown_connections 2024-07-26T13:44:12.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.732+0000 7f8662901640 1 --2- 172.21.15.100:0/908957636 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f86440b3d40 0x7f86440b6250 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:12.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.732+0000 7f8662901640 1 --2- 172.21.15.100:0/908957636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f865c1087a0 0x7f865c19b780 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:12.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.732+0000 7f8662901640 1 --2- 172.21.15.100:0/908957636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f865c107e50 0x7f865c19b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:12.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.732+0000 7f8662901640 1 --2- 172.21.15.100:0/908957636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f865c102510 0x7f865c19ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:12.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.732+0000 7f8662901640 1 -- 172.21.15.100:0/908957636 >> 172.21.15.100:0/908957636 conn(0x7f865c0791a0 msgr2=0x7f865c105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:12.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.732+0000 7f8662901640 1 -- 172.21.15.100:0/908957636 shutdown_connections 2024-07-26T13:44:12.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:12.732+0000 7f8662901640 1 -- 172.21.15.100:0/908957636 wait complete. 2024-07-26T13:44:13.412 INFO:tasks.workunit.client.0.smithi100.stdout: 1 145991 0.00 MB/sec execute 27 sec latency 27311.395 ms 2024-07-26T13:44:13.713 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-26T13:44:13.879 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:13 smithi100 ceph-mon[32599]: pgmap v488: 97 pgs: 97 active+clean; 62 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 1.6 MiB/s wr, 89 op/s 2024-07-26T13:44:13.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:13 smithi138 ceph-mon[36954]: pgmap v488: 97 pgs: 97 active+clean; 62 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 1.6 MiB/s wr, 89 op/s 2024-07-26T13:44:14.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:13 smithi167 ceph-mon[37140]: pgmap v488: 97 pgs: 97 active+clean; 62 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 1.6 MiB/s wr, 89 op/s 2024-07-26T13:44:14.412 INFO:tasks.workunit.client.0.smithi100.stdout: 1 145991 0.00 MB/sec execute 28 sec latency 28311.418 ms 2024-07-26T13:44:15.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.288+0000 7f5636744640 1 -- 172.21.15.100:0/2830146422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5630075ab0 msgr2=0x7f563010f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:15.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.288+0000 7f5636744640 1 --2- 172.21.15.100:0/2830146422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5630075ab0 0x7f563010f690 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f5624009900 tx=0x7f562402a410 comp rx=0 tx=0).stop 2024-07-26T13:44:15.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.288+0000 7f5636744640 1 -- 172.21.15.100:0/2830146422 shutdown_connections 2024-07-26T13:44:15.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.288+0000 7f5636744640 1 --2- 172.21.15.100:0/2830146422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f563010fbd0 0x7f5630111ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:15.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.288+0000 7f5636744640 1 --2- 172.21.15.100:0/2830146422 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5630075ab0 0x7f563010f690 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:15.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.288+0000 7f5636744640 1 --2- 172.21.15.100:0/2830146422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5630075160 0x7f5630075570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:15.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.288+0000 7f5636744640 1 -- 172.21.15.100:0/2830146422 >> 172.21.15.100:0/2830146422 conn(0x7f563007b340 msgr2=0x7f563007b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:15.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.288+0000 7f5636744640 1 -- 172.21.15.100:0/2830146422 shutdown_connections 2024-07-26T13:44:15.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.288+0000 7f5636744640 1 -- 172.21.15.100:0/2830146422 wait complete. 2024-07-26T13:44:15.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.289+0000 7f5636744640 1 Processor -- start 2024-07-26T13:44:15.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.289+0000 7f5636744640 1 -- start start 2024-07-26T13:44:15.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.289+0000 7f5636744640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5630075160 0x7f563019c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:15.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.289+0000 7f5636744640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5630075ab0 0x7f563019c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:15.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.289+0000 7f5636744640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f563010fbd0 0x7f56301a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:15.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.289+0000 7f5636744640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5630115700 con 0x7f5630075ab0 2024-07-26T13:44:15.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.289+0000 7f5636744640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5630115580 con 0x7f563010fbd0 2024-07-26T13:44:15.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.289+0000 7f5636744640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5630115880 con 0x7f5630075160 2024-07-26T13:44:15.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.290+0000 7f562f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5630075ab0 0x7f563019c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:15.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.290+0000 7f562f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5630075ab0 0x7f563019c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44384/0 (socket says 172.21.15.100:44384) 2024-07-26T13:44:15.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.290+0000 7f562f7fe640 1 -- 172.21.15.100:0/3169591306 learned_addr learned my addr 172.21.15.100:0/3169591306 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:15.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.290+0000 7f562ffff640 1 --2- 172.21.15.100:0/3169591306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5630075160 0x7f563019c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:15.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.290+0000 7f5634ce1640 1 --2- 172.21.15.100:0/3169591306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f563010fbd0 0x7f56301a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:15.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.290+0000 7f562f7fe640 1 -- 172.21.15.100:0/3169591306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5630075160 msgr2=0x7f563019c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:15.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.290+0000 7f562f7fe640 1 --2- 172.21.15.100:0/3169591306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5630075160 0x7f563019c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:15.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.290+0000 7f562f7fe640 1 -- 172.21.15.100:0/3169591306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f563010fbd0 msgr2=0x7f56301a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:15.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.290+0000 7f562f7fe640 1 --2- 172.21.15.100:0/3169591306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f563010fbd0 0x7f56301a1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:15.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.290+0000 7f562f7fe640 1 -- 172.21.15.100:0/3169591306 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56301a2140 con 0x7f5630075ab0 2024-07-26T13:44:15.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.290+0000 7f562f7fe640 1 --2- 172.21.15.100:0/3169591306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5630075ab0 0x7f563019c770 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f562402a930 tx=0x7f5624004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:15.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.290+0000 7f562d7fa640 1 -- 172.21.15.100:0/3169591306 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562403f030 con 0x7f5630075ab0 2024-07-26T13:44:15.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.290+0000 7f562d7fa640 1 -- 172.21.15.100:0/3169591306 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5624003aa0 con 0x7f5630075ab0 2024-07-26T13:44:15.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.290+0000 7f5636744640 1 -- 172.21.15.100:0/3169591306 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56301a23d0 con 0x7f5630075ab0 2024-07-26T13:44:15.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.290+0000 7f562d7fa640 1 -- 172.21.15.100:0/3169591306 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5624035070 con 0x7f5630075ab0 2024-07-26T13:44:15.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.290+0000 7f5636744640 1 -- 172.21.15.100:0/3169591306 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56301a8b90 con 0x7f5630075ab0 2024-07-26T13:44:15.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.291+0000 7f5636744640 1 -- 172.21.15.100:0/3169591306 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5630076620 con 0x7f5630075ab0 2024-07-26T13:44:15.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.293+0000 7f562d7fa640 1 -- 172.21.15.100:0/3169591306 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f5624042030 con 0x7f5630075ab0 2024-07-26T13:44:15.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.294+0000 7f562d7fa640 1 --2- 172.21.15.100:0/3169591306 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f56180b3d40 0x7f56180b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:15.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.294+0000 7f562ffff640 1 --2- 172.21.15.100:0/3169591306 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f56180b3d40 0x7f56180b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:15.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.294+0000 7f562d7fa640 1 -- 172.21.15.100:0/3169591306 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f5624104bc0 con 0x7f5630075ab0 2024-07-26T13:44:15.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.294+0000 7f562ffff640 1 --2- 172.21.15.100:0/3169591306 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f56180b3d40 0x7f56180b6250 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5620009990 tx=0x7f5620009110 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:15.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.297+0000 7f562d7fa640 1 -- 172.21.15.100:0/3169591306 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f56240cbea0 con 0x7f5630075ab0 2024-07-26T13:44:15.394 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:44:15.412 INFO:tasks.workunit.client.0.smithi100.stdout: 1 145991 0.00 MB/sec execute 29 sec latency 29311.442 ms 2024-07-26T13:44:15.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.511+0000 7f5636744640 1 -- 172.21.15.100:0/3169591306 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56301a8d60 con 0x7f5630075ab0 2024-07-26T13:44:15.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.511+0000 7f562d7fa640 1 -- 172.21.15.100:0/3169591306 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11933 (secure 0 0 0) 0x7f56240d4170 con 0x7f5630075ab0 2024-07-26T13:44:15.513 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:44:15.513 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":37,"btime":"2024-07-26T13:43:49:549366+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14790,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/313501409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":313501409},{"type":"v1","addr":"172.21.15.100:6839","nonce":313501409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/1402975506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":1402975506},{"type":"v1","addr":"172.21.15.100:6841","nonce":1402975506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-26T13:37:01.281843+0000","modified":"2024-07-26T13:43:49.175512+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24607,"mds_1":24605,"mds_2":24557},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24557":{"gid":24557,"name":"e","rank":2,"incarnation":30,"state":"up:active","state_seq":107,"addr":"172.21.15.138:6835/3290105300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3290105300},{"type":"v1","addr":"172.21.15.138:6835","nonce":3290105300}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/3393225483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3393225483},{"type":"v1","addr":"172.21.15.138:6837","nonce":3393225483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.138:6839/1580925471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1580925471},{"type":"v1","addr":"172.21.15.138:6839","nonce":1580925471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/398150514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":398150514},{"type":"v1","addr":"172.21.15.167:6839","nonce":398150514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25762":{"gid":25762,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2476068386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2476068386},{"type":"v1","addr":"172.21.15.167:6837","nonce":2476068386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24607,"qdb_cluster":[24557,24605,24607]},"id":1}]} 2024-07-26T13:44:15.513 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 37 2024-07-26T13:44:15.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.513+0000 7f5636744640 1 -- 172.21.15.100:0/3169591306 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f56180b3d40 msgr2=0x7f56180b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:15.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.513+0000 7f5636744640 1 --2- 172.21.15.100:0/3169591306 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f56180b3d40 0x7f56180b6250 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5620009990 tx=0x7f5620009110 comp rx=0 tx=0).stop 2024-07-26T13:44:15.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.513+0000 7f5636744640 1 -- 172.21.15.100:0/3169591306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5630075ab0 msgr2=0x7f563019c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:15.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.513+0000 7f5636744640 1 --2- 172.21.15.100:0/3169591306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5630075ab0 0x7f563019c770 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f562402a930 tx=0x7f5624004270 comp rx=0 tx=0).stop 2024-07-26T13:44:15.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.514+0000 7f5636744640 1 -- 172.21.15.100:0/3169591306 shutdown_connections 2024-07-26T13:44:15.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.514+0000 7f5636744640 1 --2- 172.21.15.100:0/3169591306 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f56180b3d40 0x7f56180b6250 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:15.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.514+0000 7f5636744640 1 --2- 172.21.15.100:0/3169591306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f563010fbd0 0x7f56301a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:15.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.514+0000 7f5636744640 1 --2- 172.21.15.100:0/3169591306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5630075ab0 0x7f563019c770 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:15.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.514+0000 7f5636744640 1 --2- 172.21.15.100:0/3169591306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5630075160 0x7f563019c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:15.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.514+0000 7f5636744640 1 -- 172.21.15.100:0/3169591306 >> 172.21.15.100:0/3169591306 conn(0x7f563007b340 msgr2=0x7f5630109770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:15.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.514+0000 7f5636744640 1 -- 172.21.15.100:0/3169591306 shutdown_connections 2024-07-26T13:44:15.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:15.514+0000 7f5636744640 1 -- 172.21.15.100:0/3169591306 wait complete. 2024-07-26T13:44:15.785 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:15 smithi100 ceph-mon[32599]: pgmap v489: 97 pgs: 97 active+clean; 61 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 1.7 MiB/s wr, 93 op/s 2024-07-26T13:44:15.785 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:15 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/3169591306' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:44:15.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:15 smithi138 ceph-mon[36954]: pgmap v489: 97 pgs: 97 active+clean; 61 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 1.7 MiB/s wr, 93 op/s 2024-07-26T13:44:15.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:15 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/3169591306' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:44:16.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:15 smithi167 ceph-mon[37140]: pgmap v489: 97 pgs: 97 active+clean; 61 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 1.7 MiB/s wr, 93 op/s 2024-07-26T13:44:16.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:15 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/3169591306' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:44:16.412 INFO:tasks.workunit.client.0.smithi100.stdout: 1 145991 0.00 MB/sec execute 30 sec latency 30311.474 ms 2024-07-26T13:44:17.308 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24607 quiesce db --set-id 47f95d77 --release --await' 2024-07-26T13:44:17.309 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- timeout 105 ceph tell mds.24607 quiesce db --set-id 47f95d77 --release --await 2024-07-26T13:44:17.412 INFO:tasks.workunit.client.0.smithi100.stdout: 1 145991 0.00 MB/sec execute 31 sec latency 31311.527 ms 2024-07-26T13:44:17.835 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:17 smithi100 ceph-mon[32599]: pgmap v490: 97 pgs: 97 active+clean; 61 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 1.7 MiB/s wr, 74 op/s 2024-07-26T13:44:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.842+0000 7f6e0e96c640 1 -- 172.21.15.100:0/3194208457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e08101a90 msgr2=0x7f6e08101ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.842+0000 7f6e0e96c640 1 --2- 172.21.15.100:0/3194208457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e08101a90 0x7f6e08101ea0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f6df8009900 tx=0x7f6df802a410 comp rx=0 tx=0).stop 2024-07-26T13:44:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.842+0000 7f6e0e96c640 1 -- 172.21.15.100:0/3194208457 shutdown_connections 2024-07-26T13:44:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.842+0000 7f6e0e96c640 1 --2- 172.21.15.100:0/3194208457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e081097b0 0x7f6e0810ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.842+0000 7f6e0e96c640 1 --2- 172.21.15.100:0/3194208457 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e08108e60 0x7f6e08109270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.842+0000 7f6e0e96c640 1 --2- 172.21.15.100:0/3194208457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e08101a90 0x7f6e08101ea0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.843+0000 7f6e0e96c640 1 -- 172.21.15.100:0/3194208457 >> 172.21.15.100:0/3194208457 conn(0x7f6e080798c0 msgr2=0x7f6e08079cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.843+0000 7f6e0e96c640 1 -- 172.21.15.100:0/3194208457 shutdown_connections 2024-07-26T13:44:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.843+0000 7f6e0e96c640 1 -- 172.21.15.100:0/3194208457 wait complete. 2024-07-26T13:44:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.843+0000 7f6e0e96c640 1 Processor -- start 2024-07-26T13:44:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.843+0000 7f6e0e96c640 1 -- start start 2024-07-26T13:44:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.844+0000 7f6e0e96c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e08101a90 0x7f6e08199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.844+0000 7f6e0e96c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e08108e60 0x7f6e0819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.844+0000 7f6e0e96c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e081097b0 0x7f6e0819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.844+0000 7f6e0e96c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6e08113480 con 0x7f6e08108e60 2024-07-26T13:44:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.844+0000 7f6e0e96c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6e08113300 con 0x7f6e081097b0 2024-07-26T13:44:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.844+0000 7f6e0e96c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6e08113600 con 0x7f6e08101a90 2024-07-26T13:44:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.844+0000 7f6e077fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e08108e60 0x7f6e0819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.844+0000 7f6e077fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e08108e60 0x7f6e0819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44396/0 (socket says 172.21.15.100:44396) 2024-07-26T13:44:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.844+0000 7f6e077fe640 1 -- 172.21.15.100:0/2239329323 learned_addr learned my addr 172.21.15.100:0/2239329323 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.844+0000 7f6e0cf09640 1 --2- 172.21.15.100:0/2239329323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e081097b0 0x7f6e0819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.844+0000 7f6e07fff640 1 --2- 172.21.15.100:0/2239329323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e08101a90 0x7f6e08199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.844+0000 7f6e077fe640 1 -- 172.21.15.100:0/2239329323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e08101a90 msgr2=0x7f6e08199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.844+0000 7f6e077fe640 1 --2- 172.21.15.100:0/2239329323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e08101a90 0x7f6e08199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.844+0000 7f6e077fe640 1 -- 172.21.15.100:0/2239329323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e081097b0 msgr2=0x7f6e0819f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.844+0000 7f6e077fe640 1 --2- 172.21.15.100:0/2239329323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e081097b0 0x7f6e0819f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.844+0000 7f6e077fe640 1 -- 172.21.15.100:0/2239329323 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e0819fe30 con 0x7f6e08108e60 2024-07-26T13:44:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.845+0000 7f6e077fe640 1 --2- 172.21.15.100:0/2239329323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e08108e60 0x7f6e0819a4a0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc00d750 tx=0x7f6dfc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.845+0000 7f6e057fa640 1 -- 172.21.15.100:0/2239329323 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc017070 con 0x7f6e08108e60 2024-07-26T13:44:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.845+0000 7f6e0e96c640 1 -- 172.21.15.100:0/2239329323 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e081a0120 con 0x7f6e08108e60 2024-07-26T13:44:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.845+0000 7f6e057fa640 1 -- 172.21.15.100:0/2239329323 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dfc0054d0 con 0x7f6e08108e60 2024-07-26T13:44:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.845+0000 7f6e057fa640 1 -- 172.21.15.100:0/2239329323 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc004c30 con 0x7f6e08108e60 2024-07-26T13:44:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.845+0000 7f6e0e96c640 1 -- 172.21.15.100:0/2239329323 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e081a68b0 con 0x7f6e08108e60 2024-07-26T13:44:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.848+0000 7f6e057fa640 1 -- 172.21.15.100:0/2239329323 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f6dfc004dd0 con 0x7f6e08108e60 2024-07-26T13:44:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.848+0000 7f6e15882740 1 Processor -- start 2024-07-26T13:44:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.848+0000 7f6e15882740 1 -- start start 2024-07-26T13:44:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.848+0000 7f6e15882740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5d401b0 0x55f3b60e2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.848+0000 7f6e15882740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3b5d172f0 0x55f3b60e3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.848+0000 7f6e15882740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b5d1f4d0 0x55f3b60e3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.848+0000 7f6e15882740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55f3b60583b0 con 0x55f3b5d1f4d0 2024-07-26T13:44:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.848+0000 7f6e15882740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f3b60478b0 con 0x55f3b5d172f0 2024-07-26T13:44:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.848+0000 7f6e15882740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f3b604b8f0 con 0x55f3b5d401b0 2024-07-26T13:44:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.848+0000 7f6e077fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b5d1f4d0 0x55f3b60e3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.848+0000 7f6e0cf09640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5d401b0 0x55f3b60e2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.848+0000 7f6e0cf09640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5d401b0 0x55f3b60e2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:43384/0 (socket says 172.21.15.100:43384) 2024-07-26T13:44:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.848+0000 7f6e0cf09640 1 -- 172.21.15.100:0/248719402 learned_addr learned my addr 172.21.15.100:0/248719402 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.849+0000 7f6e07fff640 1 --2- 172.21.15.100:0/248719402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3b5d172f0 0x55f3b60e3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.849+0000 7f6e057fa640 1 --2- 172.21.15.100:0/2239329323 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6df00b3d40 0x7f6df00b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.849+0000 7f6e0cf09640 1 -- 172.21.15.100:0/248719402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3b5d172f0 msgr2=0x55f3b60e3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.849+0000 7f6e0cf09640 1 --2- 172.21.15.100:0/248719402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3b5d172f0 0x55f3b60e3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.849+0000 7f6e0cf09640 1 -- 172.21.15.100:0/248719402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b5d1f4d0 msgr2=0x55f3b60e3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.849+0000 7f6e0cf09640 1 --2- 172.21.15.100:0/248719402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b5d1f4d0 0x55f3b60e3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.849+0000 7f6e0cf09640 1 -- 172.21.15.100:0/248719402 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3b5fee9a0 con 0x55f3b5d401b0 2024-07-26T13:44:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.849+0000 7f6e077fe640 1 --2- 172.21.15.100:0/248719402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b5d1f4d0 0x55f3b60e3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:44:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.849+0000 7f6e057fa640 1 -- 172.21.15.100:0/2239329323 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f6dfc0e39d0 con 0x7f6e08108e60 2024-07-26T13:44:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.849+0000 7f6e0cf09640 1 --2- 172.21.15.100:0/2239329323 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6df00b3d40 0x7f6df00b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.849+0000 7f6e0cf09640 1 --2- 172.21.15.100:0/248719402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5d401b0 0x55f3b60e2ce0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6e00003b40 tx=0x7f6e00009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46a8cd734dee8b51 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.849+0000 7f6dee7fc640 1 -- 172.21.15.100:0/248719402 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e00024070 con 0x55f3b5d401b0 2024-07-26T13:44:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.849+0000 7f6e0cf09640 1 --2- 172.21.15.100:0/2239329323 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6df00b3d40 0x7f6df00b6250 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6e00021000 tx=0x7f6e0001e000 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.850+0000 7f6dee7fc640 1 -- 172.21.15.100:0/248719402 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e00010dc0 con 0x55f3b5d401b0 2024-07-26T13:44:17.856 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.850+0000 7f6dee7fc640 1 -- 172.21.15.100:0/248719402 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0001edb0 con 0x55f3b5d401b0 2024-07-26T13:44:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.850+0000 7f6e15882740 1 -- 172.21.15.100:0/248719402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5d401b0 msgr2=0x55f3b60e2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.850+0000 7f6e15882740 1 --2- 172.21.15.100:0/248719402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5d401b0 0x55f3b60e2ce0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6e00003b40 tx=0x7f6e00009f90 comp rx=0 tx=0).stop 2024-07-26T13:44:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.850+0000 7f6e15882740 1 -- 172.21.15.100:0/248719402 shutdown_connections 2024-07-26T13:44:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.850+0000 7f6e15882740 1 --2- 172.21.15.100:0/248719402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b5d1f4d0 0x55f3b60e3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.850+0000 7f6e15882740 1 --2- 172.21.15.100:0/248719402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3b5d172f0 0x55f3b60e3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.850+0000 7f6e15882740 1 --2- 172.21.15.100:0/248719402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5d401b0 0x55f3b60e2ce0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.850+0000 7f6e15882740 1 -- 172.21.15.100:0/248719402 >> 172.21.15.100:0/248719402 conn(0x55f3b5d34300 msgr2=0x55f3b60e0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.851+0000 7f6e15882740 1 -- 172.21.15.100:0/248719402 shutdown_connections 2024-07-26T13:44:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.851+0000 7f6e15882740 1 -- 172.21.15.100:0/248719402 wait complete. 2024-07-26T13:44:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.851+0000 7f6e15882740 1 -- start start 2024-07-26T13:44:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.851+0000 7f6e15882740 1 Processor -- start 2024-07-26T13:44:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.852+0000 7f6e15882740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3b5d172f0 0x55f3b60ee850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.852+0000 7f6e15882740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b5d1f4d0 0x55f3b60eed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.852+0000 7f6e15882740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5d401b0 0x55f3b60fa740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.852+0000 7f6e15882740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55f3b5f81ab0 con 0x55f3b5d1f4d0 2024-07-26T13:44:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.852+0000 7f6e15882740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f3b5fcfcf0 con 0x55f3b5d172f0 2024-07-26T13:44:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.852+0000 7f6e15882740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f3b60f1460 con 0x55f3b5d401b0 2024-07-26T13:44:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.852+0000 7f6e15882740 10 client.0 _ll_register_callbacks cb 0x55f3b60419f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:44:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.852+0000 7f6e077fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b5d1f4d0 0x55f3b60eed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.852+0000 7f6e077fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b5d1f4d0 0x55f3b60eed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44414/0 (socket says 172.21.15.100:44414) 2024-07-26T13:44:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.852+0000 7f6e077fe640 1 -- 172.21.15.100:0/2852529965 learned_addr learned my addr 172.21.15.100:0/2852529965 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.852+0000 7f6e07fff640 1 --2- 172.21.15.100:0/2852529965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3b5d172f0 0x55f3b60ee850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.852+0000 7f6e0cf09640 1 --2- 172.21.15.100:0/2852529965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5d401b0 0x55f3b60fa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.852+0000 7f6e077fe640 1 -- 172.21.15.100:0/2852529965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5d401b0 msgr2=0x55f3b60fa740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.852+0000 7f6e077fe640 1 --2- 172.21.15.100:0/2852529965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5d401b0 0x55f3b60fa740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.852+0000 7f6e077fe640 1 -- 172.21.15.100:0/2852529965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3b5d172f0 msgr2=0x55f3b60ee850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.852+0000 7f6e077fe640 1 --2- 172.21.15.100:0/2852529965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3b5d172f0 0x55f3b60ee850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.852+0000 7f6e077fe640 1 -- 172.21.15.100:0/2852529965 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3b5dab790 con 0x55f3b5d1f4d0 2024-07-26T13:44:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.853+0000 7f6e077fe640 1 --2- 172.21.15.100:0/2852529965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b5d1f4d0 0x55f3b60eed90 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc0e1150 tx=0x7f6dfc0e7030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a607df2e0580d8a server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.853+0000 7f6de77fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:44:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.853+0000 7f6de77fe640 1 -- 172.21.15.100:0/2852529965 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc020070 con 0x55f3b5d1f4d0 2024-07-26T13:44:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.853+0000 7f6de77fe640 1 -- 172.21.15.100:0/2852529965 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f6dfc0a68f0 con 0x55f3b5d1f4d0 2024-07-26T13:44:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.853+0000 7f6e15882740 1 -- 172.21.15.100:0/2852529965 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e00010dc0 con 0x55f3b5d1f4d0 2024-07-26T13:44:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.853+0000 7f6de77fe640 1 -- 172.21.15.100:0/2852529965 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6dfc0a9fb0 con 0x55f3b5d1f4d0 2024-07-26T13:44:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.853+0000 7f6e15882740 10 client.16566 fetch_fsmap learned FSMap version 37 2024-07-26T13:44:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.853+0000 7f6e15882740 1 -- 172.21.15.100:0/2852529965 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f3b5e24620 con 0x55f3b5d1f4d0 2024-07-26T13:44:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.853+0000 7f6de77fe640 1 -- 172.21.15.100:0/2852529965 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e00010dc0 con 0x55f3b5d1f4d0 2024-07-26T13:44:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.853+0000 7f6de77fe640 1 -- 172.21.15.100:0/2852529965 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f6dfc0a5fd0 con 0x55f3b5d1f4d0 2024-07-26T13:44:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.853+0000 7f6e15882740 10 client.16566 fetch_fsmap finished waiting for FSMap version 37 2024-07-26T13:44:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.854+0000 7f6e15882740 10 client.16566 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:44:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.854+0000 7f6e15882740 20 client.16566 populate_metadata read hostname 'smithi100' 2024-07-26T13:44:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.854+0000 7f6e15882740 1 --2- 172.21.15.100:0/2852529965 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f3b5cfed80 0x55f3b60fe150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.854+0000 7f6e15882740 4 client.16566 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:44:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.854+0000 7f6e15882740 1 -- 172.21.15.100:0/2852529965 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3b605a300 con 0x55f3b5cfed80 2024-07-26T13:44:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.854+0000 7f6e07fff640 1 --2- 172.21.15.100:0/2852529965 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f3b5cfed80 0x55f3b60fe150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.854+0000 7f6de77fe640 10 client.16566 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.854+0000 7f6e07fff640 1 --2- 172.21.15.100:0/2852529965 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f3b5cfed80 0x55f3b60fe150 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19ea1e5a4b3571e5 server_cookie=a6313a083aef95f9 in_seq=0 out_seq=0 2024-07-26T13:44:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.855+0000 7f6de77fe640 1 -- 172.21.15.100:0/2852529965 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f3b605a300 con 0x55f3b5cfed80 2024-07-26T13:44:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.855+0000 7f6de77fe640 10 client.16566 handle_command_reply: tid=0 2024-07-26T13:44:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.855+0000 7f6e15882740 1 client.16566 shutdown 2024-07-26T13:44:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.855+0000 7f6e15882740 1 -- 172.21.15.100:0/2852529965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b5d1f4d0 msgr2=0x55f3b60eed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.855+0000 7f6e15882740 1 --2- 172.21.15.100:0/2852529965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b5d1f4d0 0x55f3b60eed90 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc0e1150 tx=0x7f6dfc0e7030 comp rx=0 tx=0).stop 2024-07-26T13:44:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.855+0000 7f6e15882740 1 -- 172.21.15.100:0/2852529965 shutdown_connections 2024-07-26T13:44:17.871 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.856+0000 7f6e15882740 1 --2- 172.21.15.100:0/2852529965 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f3b5cfed80 0x55f3b60fe150 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.856+0000 7f6de77fe640 0 client.16566 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.856+0000 7f6e15882740 1 --2- 172.21.15.100:0/2852529965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5d401b0 0x55f3b60fa740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.856+0000 7f6e15882740 1 --2- 172.21.15.100:0/2852529965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b5d1f4d0 0x55f3b60eed90 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.856+0000 7f6e15882740 1 --2- 172.21.15.100:0/2852529965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3b5d172f0 0x55f3b60ee850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.856+0000 7f6e15882740 1 -- 172.21.15.100:0/2852529965 >> 172.21.15.100:0/2852529965 conn(0x55f3b5d34300 msgr2=0x55f3b60e3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.856+0000 7f6e15882740 1 -- 172.21.15.100:0/2852529965 shutdown_connections 2024-07-26T13:44:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.856+0000 7f6e15882740 1 -- 172.21.15.100:0/2852529965 wait complete. 2024-07-26T13:44:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.856+0000 7f6e15882740 20 client.16566 trim_cache size 0 max 16384 2024-07-26T13:44:17.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.869+0000 7f24f2c35640 1 -- 172.21.15.100:0/161145104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24ec109860 msgr2=0x7f24ec110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:17.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.869+0000 7f24f2c35640 1 --2- 172.21.15.100:0/161145104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24ec109860 0x7f24ec110090 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f24e4009ca0 tx=0x7f24e402a8c0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.869+0000 7f24f2c35640 1 -- 172.21.15.100:0/161145104 shutdown_connections 2024-07-26T13:44:17.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.869+0000 7f24f2c35640 1 --2- 172.21.15.100:0/161145104 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24ec109860 0x7f24ec110090 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.869+0000 7f24f2c35640 1 --2- 172.21.15.100:0/161145104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24ec108f10 0x7f24ec109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.869+0000 7f24f2c35640 1 --2- 172.21.15.100:0/161145104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24ec1035d0 0x7f24ec1039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.869+0000 7f24f2c35640 1 -- 172.21.15.100:0/161145104 >> 172.21.15.100:0/161145104 conn(0x7f24ec0ff060 msgr2=0x7f24ec1014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:17.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.869+0000 7f24f2c35640 1 -- 172.21.15.100:0/161145104 shutdown_connections 2024-07-26T13:44:17.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.869+0000 7f24f2c35640 1 -- 172.21.15.100:0/161145104 wait complete. 2024-07-26T13:44:17.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.870+0000 7f24f2c35640 1 Processor -- start 2024-07-26T13:44:17.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.870+0000 7f24f2c35640 1 -- start start 2024-07-26T13:44:17.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.870+0000 7f24f2c35640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24ec1035d0 0x7f24ec199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:17.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.870+0000 7f24f2c35640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24ec108f10 0x7f24ec19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:17.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.870+0000 7f24f2c35640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24ec109860 0x7f24ec19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:17.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.870+0000 7f24f2c35640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f24ec113530 con 0x7f24ec108f10 2024-07-26T13:44:17.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.870+0000 7f24f2c35640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f24ec1133b0 con 0x7f24ec1035d0 2024-07-26T13:44:17.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.870+0000 7f24f2c35640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f24ec1136b0 con 0x7f24ec109860 2024-07-26T13:44:17.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.870+0000 7f24ebfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24ec108f10 0x7f24ec19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:17.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.870+0000 7f24ebfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24ec108f10 0x7f24ec19a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44432/0 (socket says 172.21.15.100:44432) 2024-07-26T13:44:17.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.870+0000 7f24ebfff640 1 -- 172.21.15.100:0/465446718 learned_addr learned my addr 172.21.15.100:0/465446718 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:17.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.870+0000 7f24f09d1640 1 --2- 172.21.15.100:0/465446718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24ec1035d0 0x7f24ec199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:17.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.870+0000 7f24f11d2640 1 --2- 172.21.15.100:0/465446718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24ec109860 0x7f24ec19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:17.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.870+0000 7f24ebfff640 1 -- 172.21.15.100:0/465446718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24ec109860 msgr2=0x7f24ec19f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:17.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.870+0000 7f24ebfff640 1 --2- 172.21.15.100:0/465446718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24ec109860 0x7f24ec19f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.870+0000 7f24ebfff640 1 -- 172.21.15.100:0/465446718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24ec1035d0 msgr2=0x7f24ec199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:17.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.870+0000 7f24ebfff640 1 --2- 172.21.15.100:0/465446718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24ec1035d0 0x7f24ec199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.870+0000 7f24ebfff640 1 -- 172.21.15.100:0/465446718 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24ec19fe50 con 0x7f24ec108f10 2024-07-26T13:44:17.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.871+0000 7f24f09d1640 1 --2- 172.21.15.100:0/465446718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24ec1035d0 0x7f24ec199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:44:17.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.871+0000 7f24ebfff640 1 --2- 172.21.15.100:0/465446718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24ec108f10 0x7f24ec19a480 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f24d800e970 tx=0x7f24d800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:17.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.871+0000 7f24e9ffb640 1 -- 172.21.15.100:0/465446718 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d800cd80 con 0x7f24ec108f10 2024-07-26T13:44:17.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.871+0000 7f24e9ffb640 1 -- 172.21.15.100:0/465446718 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24d80035f0 con 0x7f24ec108f10 2024-07-26T13:44:17.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.871+0000 7f24f2c35640 1 -- 172.21.15.100:0/465446718 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24ec1a0140 con 0x7f24ec108f10 2024-07-26T13:44:17.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.871+0000 7f24e9ffb640 1 -- 172.21.15.100:0/465446718 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d8010690 con 0x7f24ec108f10 2024-07-26T13:44:17.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.871+0000 7f24f2c35640 1 -- 172.21.15.100:0/465446718 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24ec075020 con 0x7f24ec108f10 2024-07-26T13:44:17.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.874+0000 7f24f91b9740 1 Processor -- start 2024-07-26T13:44:17.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.874+0000 7f24f91b9740 1 -- start start 2024-07-26T13:44:17.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.874+0000 7f24f91b9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635407a2070 0x563540b4fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:17.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.874+0000 7f24f91b9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635407791b0 0x563540b502c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:17.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.874+0000 7f24f91b9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563540781390 0x563540b50800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:17.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.874+0000 7f24f91b9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x563540abdbb0 con 0x5635407a2070 2024-07-26T13:44:17.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.874+0000 7f24f91b9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563540aad0b0 con 0x5635407791b0 2024-07-26T13:44:17.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.874+0000 7f24f91b9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563540ab10f0 con 0x563540781390 2024-07-26T13:44:17.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.874+0000 7f24f11d2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635407a2070 0x563540b4fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:17.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.874+0000 7f24f11d2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635407a2070 0x563540b4fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44448/0 (socket says 172.21.15.100:44448) 2024-07-26T13:44:17.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.874+0000 7f24f11d2640 1 -- 172.21.15.100:0/868084317 learned_addr learned my addr 172.21.15.100:0/868084317 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:17.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.874+0000 7f24e9ffb640 1 -- 172.21.15.100:0/465446718 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f24d8003aa0 con 0x7f24ec108f10 2024-07-26T13:44:17.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.875+0000 7f24f09d1640 1 --2- 172.21.15.100:0/868084317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635407791b0 0x563540b502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:17.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.875+0000 7f24ebfff640 1 --2- 172.21.15.100:0/868084317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563540781390 0x563540b50800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:17.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.875+0000 7f24f09d1640 1 -- 172.21.15.100:0/868084317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563540781390 msgr2=0x563540b50800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:17.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.875+0000 7f24f09d1640 1 --2- 172.21.15.100:0/868084317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563540781390 0x563540b50800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.875+0000 7f24f09d1640 1 -- 172.21.15.100:0/868084317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635407a2070 msgr2=0x563540b4fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:17.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.875+0000 7f24f09d1640 1 --2- 172.21.15.100:0/868084317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635407a2070 0x563540b4fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.875+0000 7f24f09d1640 1 -- 172.21.15.100:0/868084317 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563540a53b90 con 0x5635407791b0 2024-07-26T13:44:17.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.875+0000 7f24ebfff640 1 --2- 172.21.15.100:0/868084317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563540781390 0x563540b50800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:44:17.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.875+0000 7f24f09d1640 1 --2- 172.21.15.100:0/868084317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635407791b0 0x563540b502c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f24e0009fd0 tx=0x7f24e00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=194e9f7a8c58b51 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:17.880 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.875+0000 7f6e15882740 1 Processor -- start 2024-07-26T13:44:17.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.875+0000 7f24d6ffd640 1 -- 172.21.15.100:0/868084317 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e0004310 con 0x5635407791b0 2024-07-26T13:44:17.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.875+0000 7f24d6ffd640 1 -- 172.21.15.100:0/868084317 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24e00044b0 con 0x5635407791b0 2024-07-26T13:44:17.881 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.876+0000 7f6e15882740 1 -- start start 2024-07-26T13:44:17.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.876+0000 7f24d6ffd640 1 -- 172.21.15.100:0/868084317 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e00035f0 con 0x5635407791b0 2024-07-26T13:44:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.876+0000 7f6e15882740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5cfed80 0x55f3b60e3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.876+0000 7f6e15882740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3b5d172f0 0x55f3b60e4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:17.882 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.876+0000 7f6e15882740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b5d1f4d0 0x55f3b60e7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:17.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.876+0000 7f24e9ffb640 1 --2- 172.21.15.100:0/465446718 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f24dc0b3dc0 0x7f24dc0b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:17.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.876+0000 7f24f11d2640 1 --2- 172.21.15.100:0/465446718 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f24dc0b3dc0 0x7f24dc0b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:17.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.876+0000 7f24e9ffb640 1 -- 172.21.15.100:0/465446718 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f24d80e4310 con 0x7f24ec108f10 2024-07-26T13:44:17.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.877+0000 7f6e0cf09640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b5d1f4d0 0x55f3b60e7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:17.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.877+0000 7f6e0cf09640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b5d1f4d0 0x55f3b60e7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44462/0 (socket says 172.21.15.100:44462) 2024-07-26T13:44:17.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.877+0000 7f6e07fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5cfed80 0x55f3b60e3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:17.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.877+0000 7f6e0cf09640 1 -- 172.21.15.100:0/3642856722 learned_addr learned my addr 172.21.15.100:0/3642856722 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:17.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.877+0000 7f6e15882740 1 -- 172.21.15.100:0/3642856722 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6e081af950 con 0x55f3b5d1f4d0 2024-07-26T13:44:17.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.877+0000 7f6e15882740 1 -- 172.21.15.100:0/3642856722 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6e081ad1c0 con 0x55f3b5d172f0 2024-07-26T13:44:17.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.877+0000 7f6e15882740 1 -- 172.21.15.100:0/3642856722 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6e081ab010 con 0x55f3b5cfed80 2024-07-26T13:44:17.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.877+0000 7f24f11d2640 1 --2- 172.21.15.100:0/465446718 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f24dc0b3dc0 0x7f24dc0b62d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f24e4005e30 tx=0x7f24e4005c30 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:17.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.877+0000 7f24f91b9740 1 -- 172.21.15.100:0/868084317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635407791b0 msgr2=0x563540b502c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:17.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.877+0000 7f24f91b9740 1 --2- 172.21.15.100:0/868084317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635407791b0 0x563540b502c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f24e0009fd0 tx=0x7f24e00118d0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.884 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.877+0000 7f6e077fe640 1 --2- 172.21.15.100:0/3642856722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3b5d172f0 0x55f3b60e4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:17.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.877+0000 7f24f91b9740 1 -- 172.21.15.100:0/868084317 shutdown_connections 2024-07-26T13:44:17.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.877+0000 7f24f91b9740 1 --2- 172.21.15.100:0/868084317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563540781390 0x563540b50800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.877+0000 7f24f91b9740 1 --2- 172.21.15.100:0/868084317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635407791b0 0x563540b502c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.877+0000 7f24f91b9740 1 --2- 172.21.15.100:0/868084317 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635407a2070 0x563540b4fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.877+0000 7f24f91b9740 1 -- 172.21.15.100:0/868084317 >> 172.21.15.100:0/868084317 conn(0x563540796460 msgr2=0x563540b4dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:17.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.877+0000 7f24f91b9740 1 -- 172.21.15.100:0/868084317 shutdown_connections 2024-07-26T13:44:17.885 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:44:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.877+0000 7f6e07fff640 1 -- 172.21.15.100:0/3642856722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3b5d172f0 msgr2=0x55f3b60e4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.877+0000 7f6e07fff640 1 --2- 172.21.15.100:0/3642856722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3b5d172f0 0x55f3b60e4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.877+0000 7f6e07fff640 1 -- 172.21.15.100:0/3642856722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b5d1f4d0 msgr2=0x55f3b60e7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.877+0000 7f6e07fff640 1 --2- 172.21.15.100:0/3642856722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b5d1f4d0 0x55f3b60e7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.877+0000 7f6e07fff640 1 -- 172.21.15.100:0/3642856722 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3b60e54f0 con 0x55f3b5cfed80 2024-07-26T13:44:17.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.877+0000 7f6e0cf09640 1 --2- 172.21.15.100:0/3642856722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b5d1f4d0 0x55f3b60e7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:44:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.878+0000 7f6e07fff640 1 --2- 172.21.15.100:0/3642856722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5cfed80 0x55f3b60e3d10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6df8009230 tx=0x7f6df8005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c8fcefe387f248d server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:17.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.877+0000 7f24f91b9740 1 -- 172.21.15.100:0/868084317 wait complete. 2024-07-26T13:44:17.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.878+0000 7f24f91b9740 1 -- start start 2024-07-26T13:44:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.878+0000 7f6dedffb640 1 -- 172.21.15.100:0/3642856722 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df8035040 con 0x55f3b5cfed80 2024-07-26T13:44:17.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.878+0000 7f24f91b9740 1 Processor -- start 2024-07-26T13:44:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.878+0000 7f6dedffb640 1 -- 172.21.15.100:0/3642856722 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6df804b070 con 0x55f3b5cfed80 2024-07-26T13:44:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.878+0000 7f6dedffb640 1 -- 172.21.15.100:0/3642856722 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df8004750 con 0x55f3b5cfed80 2024-07-26T13:44:17.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.878+0000 7f24f91b9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635407791b0 0x563540b5b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:17.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.878+0000 7f24f91b9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563540781390 0x563540b5bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:17.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.878+0000 7f24f91b9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635407a2070 0x563540b67950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:17.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.878+0000 7f24f91b9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x563540910700 con 0x5635407a2070 2024-07-26T13:44:17.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.878+0000 7f24f91b9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563540a4f9e0 con 0x563540781390 2024-07-26T13:44:17.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.878+0000 7f24f91b9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563540b5e600 con 0x5635407791b0 2024-07-26T13:44:17.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.878+0000 7f24f91b9740 10 client.0 _ll_register_callbacks cb 0x563540a6a4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:44:17.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.879+0000 7f24f11d2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635407a2070 0x563540b67950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:17.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.879+0000 7f24f09d1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635407791b0 0x563540b5b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:17.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.879+0000 7f24f11d2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635407a2070 0x563540b67950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44472/0 (socket says 172.21.15.100:44472) 2024-07-26T13:44:17.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.879+0000 7f24f11d2640 1 -- 172.21.15.100:0/1610619404 learned_addr learned my addr 172.21.15.100:0/1610619404 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:17.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.879+0000 7f24f09d1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635407791b0 0x563540b5b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:43448/0 (socket says 172.21.15.100:43448) 2024-07-26T13:44:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.879+0000 7f6e15882740 1 -- 172.21.15.100:0/3642856722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5cfed80 msgr2=0x55f3b60e3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:17.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.879+0000 7f6e15882740 1 --2- 172.21.15.100:0/3642856722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5cfed80 0x55f3b60e3d10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6df8009230 tx=0x7f6df8005e80 comp rx=0 tx=0).stop 2024-07-26T13:44:17.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.879+0000 7f24ebfff640 1 --2- 172.21.15.100:0/1610619404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563540781390 0x563540b5bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:17.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.879+0000 7f24f09d1640 1 -- 172.21.15.100:0/1610619404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563540781390 msgr2=0x563540b5bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:17.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.879+0000 7f24f09d1640 1 --2- 172.21.15.100:0/1610619404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563540781390 0x563540b5bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.879+0000 7f24f09d1640 1 -- 172.21.15.100:0/1610619404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635407a2070 msgr2=0x563540b67950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:17.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.879+0000 7f24f09d1640 1 --2- 172.21.15.100:0/1610619404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635407a2070 0x563540b67950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.879+0000 7f6e15882740 1 -- 172.21.15.100:0/3642856722 shutdown_connections 2024-07-26T13:44:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.879+0000 7f6e15882740 1 --2- 172.21.15.100:0/3642856722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b5d1f4d0 0x55f3b60e7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.879+0000 7f6e15882740 1 --2- 172.21.15.100:0/3642856722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3b5d172f0 0x55f3b60e4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.879+0000 7f6e15882740 1 --2- 172.21.15.100:0/3642856722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5cfed80 0x55f3b60e3d10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.879+0000 7f6e15882740 1 -- 172.21.15.100:0/3642856722 >> 172.21.15.100:0/3642856722 conn(0x55f3b5d34300 msgr2=0x55f3b60ecea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:17.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.879+0000 7f24f09d1640 1 -- 172.21.15.100:0/1610619404 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56354080d650 con 0x5635407791b0 2024-07-26T13:44:17.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.879+0000 7f24f11d2640 1 --2- 172.21.15.100:0/1610619404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635407a2070 0x563540b67950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:44:17.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.879+0000 7f24ebfff640 1 --2- 172.21.15.100:0/1610619404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563540781390 0x563540b5bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:44:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.879+0000 7f6e15882740 1 -- 172.21.15.100:0/3642856722 shutdown_connections 2024-07-26T13:44:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.879+0000 7f6e15882740 1 -- 172.21.15.100:0/3642856722 wait complete. 2024-07-26T13:44:17.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.879+0000 7f24f09d1640 1 --2- 172.21.15.100:0/1610619404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635407791b0 0x563540b5b6b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f24e0005890 tx=0x7f24e0004960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7bca29603bd8133 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:17.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.879+0000 7f24cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-07-26T13:44:17.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.880+0000 7f24cbfff640 1 -- 172.21.15.100:0/1610619404 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e0029070 con 0x5635407791b0 2024-07-26T13:44:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.880+0000 7f6e15882740 1 -- start start 2024-07-26T13:44:17.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.880+0000 7f24f91b9740 1 -- 172.21.15.100:0/1610619404 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24e00044b0 con 0x5635407791b0 2024-07-26T13:44:17.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.880+0000 7f24cbfff640 1 -- 172.21.15.100:0/1610619404 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f24e0003b40 con 0x5635407791b0 2024-07-26T13:44:17.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.880+0000 7f24cbfff640 1 -- 172.21.15.100:0/1610619404 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f24e0035050 con 0x5635407791b0 2024-07-26T13:44:17.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.880+0000 7f24f91b9740 10 client.26047 fetch_fsmap learned FSMap version 37 2024-07-26T13:44:17.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.880+0000 7f24f91b9740 1 -- 172.21.15.100:0/1610619404 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563540b6a470 con 0x5635407791b0 2024-07-26T13:44:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.880+0000 7f6e15882740 1 Processor -- start 2024-07-26T13:44:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.880+0000 7f6e15882740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3b5cfed80 0x55f3b60f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.880+0000 7f6e15882740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b5d172f0 0x55f3b60f5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:17.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.880+0000 7f24cbfff640 1 -- 172.21.15.100:0/1610619404 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24e00044b0 con 0x5635407791b0 2024-07-26T13:44:17.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.880+0000 7f24cbfff640 1 -- 172.21.15.100:0/1610619404 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f24e0004c70 con 0x5635407791b0 2024-07-26T13:44:17.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.880+0000 7f24f91b9740 10 client.26047 fetch_fsmap finished waiting for FSMap version 37 2024-07-26T13:44:17.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.880+0000 7f24f91b9740 10 client.26047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:44:17.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.881+0000 7f24f91b9740 20 client.26047 populate_metadata read hostname 'smithi100' 2024-07-26T13:44:17.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.881+0000 7f24f91b9740 1 --2- 172.21.15.100:0/1610619404 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5635407609f0 0x563540b6d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.881+0000 7f6e15882740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5d1f4d0 0x55f3b61690b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:17.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.881+0000 7f24f91b9740 4 client.26047 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:44:17.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.881+0000 7f24f91b9740 1 -- 172.21.15.100:0/1610619404 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563540abfb00 con 0x5635407609f0 2024-07-26T13:44:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.881+0000 7f6e15882740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6e081ab680 con 0x55f3b5d172f0 2024-07-26T13:44:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.881+0000 7f6e15882740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6e081acaf0 con 0x55f3b5cfed80 2024-07-26T13:44:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.881+0000 7f6e07fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3b5cfed80 0x55f3b60f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.881+0000 7f6e07fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3b5cfed80 0x55f3b60f51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:36168/0 (socket says 172.21.15.100:36168) 2024-07-26T13:44:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.881+0000 7f6e07fff640 1 -- 172.21.15.100:0/2249419186 learned_addr learned my addr 172.21.15.100:0/2249419186 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.881+0000 7f6e15882740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6e081abd50 con 0x55f3b5d1f4d0 2024-07-26T13:44:17.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.881+0000 7f24ebfff640 1 --2- 172.21.15.100:0/1610619404 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5635407609f0 0x563540b6d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.881+0000 7f6e0cf09640 1 --2- 172.21.15.100:0/2249419186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5d1f4d0 0x55f3b61690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.881+0000 7f6e15882740 10 client.0 _ll_register_callbacks cb 0x7f6dfc00d6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:44:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.881+0000 7f6e077fe640 1 --2- 172.21.15.100:0/2249419186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b5d172f0 0x55f3b60f5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.881+0000 7f6e07fff640 1 -- 172.21.15.100:0/2249419186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5d1f4d0 msgr2=0x55f3b61690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.881+0000 7f6e07fff640 1 --2- 172.21.15.100:0/2249419186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5d1f4d0 0x55f3b61690b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.881+0000 7f6e07fff640 1 -- 172.21.15.100:0/2249419186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b5d172f0 msgr2=0x55f3b60f5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.881+0000 7f6e07fff640 1 --2- 172.21.15.100:0/2249419186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b5d172f0 0x55f3b60f5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.881+0000 7f6e07fff640 1 -- 172.21.15.100:0/2249419186 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3b6169720 con 0x55f3b5cfed80 2024-07-26T13:44:17.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.881+0000 7f24cbfff640 10 client.26047 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.881+0000 7f6e07fff640 1 --2- 172.21.15.100:0/2249419186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3b5cfed80 0x55f3b60f51b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6df8005060 tx=0x7f6df802a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3f0004fb38c816c server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.881+0000 7f6de77fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-07-26T13:44:17.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.881+0000 7f24ebfff640 1 --2- 172.21.15.100:0/1610619404 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5635407609f0 0x563540b6d830 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3a909164e85976c server_cookie=ba54c204bbcb70c4 in_seq=0 out_seq=0 2024-07-26T13:44:17.900 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.882+0000 7f6de77fe640 1 -- 172.21.15.100:0/2249419186 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df8053070 con 0x55f3b5cfed80 2024-07-26T13:44:17.900 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.882+0000 7f6e15882740 1 -- 172.21.15.100:0/2249419186 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6df804b070 con 0x55f3b5cfed80 2024-07-26T13:44:17.900 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.882+0000 7f6de77fe640 1 -- 172.21.15.100:0/2249419186 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f6df80055b0 con 0x55f3b5cfed80 2024-07-26T13:44:17.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.882+0000 7f24cbfff640 1 -- 172.21.15.100:0/1610619404 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563540abfb00 con 0x5635407609f0 2024-07-26T13:44:17.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.882+0000 7f24cbfff640 10 client.26047 handle_command_reply: tid=0 2024-07-26T13:44:17.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.882+0000 7f24f91b9740 1 client.26047 shutdown 2024-07-26T13:44:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.882+0000 7f6de77fe640 1 -- 172.21.15.100:0/2249419186 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6df805f050 con 0x55f3b5cfed80 2024-07-26T13:44:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.882+0000 7f6e15882740 10 client.25901 fetch_fsmap learned FSMap version 37 2024-07-26T13:44:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.882+0000 7f6e15882740 1 -- 172.21.15.100:0/2249419186 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f3b6170b20 con 0x55f3b5cfed80 2024-07-26T13:44:17.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.882+0000 7f24f91b9740 1 -- 172.21.15.100:0/1610619404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635407791b0 msgr2=0x563540b5b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:17.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.882+0000 7f24f91b9740 1 --2- 172.21.15.100:0/1610619404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635407791b0 0x563540b5b6b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f24e0005890 tx=0x7f24e0004960 comp rx=0 tx=0).stop 2024-07-26T13:44:17.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.882+0000 7f24f91b9740 1 -- 172.21.15.100:0/1610619404 shutdown_connections 2024-07-26T13:44:17.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.882+0000 7f24f91b9740 1 --2- 172.21.15.100:0/1610619404 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5635407609f0 0x563540b6d830 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.883+0000 7f6de77fe640 1 -- 172.21.15.100:0/2249419186 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6df804b070 con 0x55f3b5cfed80 2024-07-26T13:44:17.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.882+0000 7f24f91b9740 1 --2- 172.21.15.100:0/1610619404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635407a2070 0x563540b67950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.882+0000 7f24f91b9740 1 --2- 172.21.15.100:0/1610619404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563540781390 0x563540b5bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.882+0000 7f24f91b9740 1 --2- 172.21.15.100:0/1610619404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635407791b0 0x563540b5b6b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.882+0000 7f24f91b9740 1 -- 172.21.15.100:0/1610619404 >> 172.21.15.100:0/1610619404 conn(0x563540796460 msgr2=0x563540b52f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:17.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.882+0000 7f24cbfff640 0 client.26047 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:17.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.883+0000 7f24f91b9740 1 -- 172.21.15.100:0/1610619404 shutdown_connections 2024-07-26T13:44:17.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.883+0000 7f24f91b9740 1 -- 172.21.15.100:0/1610619404 wait complete. 2024-07-26T13:44:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.883+0000 7f6de77fe640 1 -- 172.21.15.100:0/2249419186 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f6df80035f0 con 0x55f3b5cfed80 2024-07-26T13:44:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.883+0000 7f6e15882740 10 client.25901 fetch_fsmap finished waiting for FSMap version 37 2024-07-26T13:44:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.883+0000 7f6e15882740 10 client.25901 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:44:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.883+0000 7f6e15882740 20 client.25901 populate_metadata read hostname 'smithi100' 2024-07-26T13:44:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.883+0000 7f6e15882740 1 --2- 172.21.15.100:0/2249419186 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f3b5d401b0 0x55f3b6173240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.883+0000 7f6e15882740 4 client.25901 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-26T13:44:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.883+0000 7f6e15882740 1 -- 172.21.15.100:0/2249419186 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f3b6055da0 con 0x55f3b5d401b0 2024-07-26T13:44:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.883+0000 7f6e077fe640 1 --2- 172.21.15.100:0/2249419186 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f3b5d401b0 0x55f3b6173240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:17.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.883+0000 7f24f91b9740 20 client.26047 trim_cache size 0 max 16384 2024-07-26T13:44:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.884+0000 7f6de77fe640 10 client.25901 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:17.906 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.884+0000 7f6e077fe640 1 --2- 172.21.15.100:0/2249419186 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f3b5d401b0 0x55f3b6173240 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e9e6b08ea91f341 server_cookie=a0bd5e375df612f7 in_seq=0 out_seq=0 2024-07-26T13:44:17.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.901+0000 7f24f91b9740 1 Processor -- start 2024-07-26T13:44:17.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.901+0000 7f24f91b9740 1 -- start start 2024-07-26T13:44:17.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.901+0000 7f24f91b9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635407791b0 0x563540b5d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:17.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.901+0000 7f24f91b9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563540781390 0x563540b67740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:17.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.901+0000 7f24f91b9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635407a2070 0x563540b67c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:17.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.901+0000 7f24f91b9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f24ec1af970 con 0x5635407a2070 2024-07-26T13:44:17.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.901+0000 7f24f91b9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f24ec1ad1e0 con 0x563540781390 2024-07-26T13:44:17.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.901+0000 7f24ebfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563540781390 0x563540b67740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:17.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.901+0000 7f24ebfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563540781390 0x563540b67740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:36178/0 (socket says 172.21.15.100:36178) 2024-07-26T13:44:17.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.901+0000 7f24ebfff640 1 -- 172.21.15.100:0/2537856972 learned_addr learned my addr 172.21.15.100:0/2537856972 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:17.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.901+0000 7f24f09d1640 1 --2- 172.21.15.100:0/2537856972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635407791b0 0x563540b5d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:17.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.901+0000 7f24f91b9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f24ec1ab030 con 0x5635407791b0 2024-07-26T13:44:17.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.902+0000 7f24f11d2640 1 --2- 172.21.15.100:0/2537856972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635407a2070 0x563540b67c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:17.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.902+0000 7f24ebfff640 1 -- 172.21.15.100:0/2537856972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635407791b0 msgr2=0x563540b5d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:17.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.902+0000 7f24ebfff640 1 --2- 172.21.15.100:0/2537856972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635407791b0 0x563540b5d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.902+0000 7f24ebfff640 1 -- 172.21.15.100:0/2537856972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635407a2070 msgr2=0x563540b67c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:17.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.902+0000 7f24ebfff640 1 --2- 172.21.15.100:0/2537856972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635407a2070 0x563540b67c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.902+0000 7f24ebfff640 1 -- 172.21.15.100:0/2537856972 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563540b5c550 con 0x563540781390 2024-07-26T13:44:17.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.902+0000 7f24f11d2640 1 --2- 172.21.15.100:0/2537856972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635407a2070 0x563540b67c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:44:17.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.902+0000 7f24ebfff640 1 --2- 172.21.15.100:0/2537856972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563540781390 0x563540b67740 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f24d80e1a30 tx=0x7f24d8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c040fdefd4be4b6c server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:17.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.903+0000 7f24d67fc640 1 -- 172.21.15.100:0/2537856972 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d80aae90 con 0x563540781390 2024-07-26T13:44:17.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.903+0000 7f24d67fc640 1 -- 172.21.15.100:0/2537856972 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24d80af490 con 0x563540781390 2024-07-26T13:44:17.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.903+0000 7f24d67fc640 1 -- 172.21.15.100:0/2537856972 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d80e1c50 con 0x563540781390 2024-07-26T13:44:17.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.904+0000 7f24f91b9740 1 -- 172.21.15.100:0/2537856972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563540781390 msgr2=0x563540b67740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:17.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.904+0000 7f24f91b9740 1 --2- 172.21.15.100:0/2537856972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563540781390 0x563540b67740 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f24d80e1a30 tx=0x7f24d8000bc0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.904+0000 7f24f91b9740 1 -- 172.21.15.100:0/2537856972 shutdown_connections 2024-07-26T13:44:17.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.904+0000 7f24f91b9740 1 --2- 172.21.15.100:0/2537856972 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5635407a2070 0x563540b67c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.904+0000 7f24f91b9740 1 --2- 172.21.15.100:0/2537856972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563540781390 0x563540b67740 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.904+0000 7f24f91b9740 1 --2- 172.21.15.100:0/2537856972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635407791b0 0x563540b5d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.904+0000 7f24f91b9740 1 -- 172.21.15.100:0/2537856972 >> 172.21.15.100:0/2537856972 conn(0x563540796460 msgr2=0x563540b5cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:17.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.904+0000 7f24f91b9740 1 -- 172.21.15.100:0/2537856972 shutdown_connections 2024-07-26T13:44:17.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.904+0000 7f24f91b9740 1 -- 172.21.15.100:0/2537856972 wait complete. 2024-07-26T13:44:17.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.904+0000 7f24f91b9740 1 -- start start 2024-07-26T13:44:17.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.905+0000 7f24f91b9740 1 Processor -- start 2024-07-26T13:44:17.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.905+0000 7f24f91b9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635407791b0 0x563540b63f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:17.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.905+0000 7f24f91b9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563540781390 0x563540b5f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:17.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.905+0000 7f24f91b9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635407a2070 0x563540b5f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:17.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.905+0000 7f24f91b9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f24ec1ab6a0 con 0x563540781390 2024-07-26T13:44:17.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.905+0000 7f24f91b9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f24ec1acb10 con 0x5635407a2070 2024-07-26T13:44:17.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.905+0000 7f24f91b9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f24ec1abd70 con 0x5635407791b0 2024-07-26T13:44:17.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.905+0000 7f24f91b9740 10 client.0 _ll_register_callbacks cb 0x7f24d8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:44:17.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.906+0000 7f24f09d1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635407791b0 0x563540b63f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:17.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.906+0000 7f24ebfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563540781390 0x563540b5f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:17.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.906+0000 7f24ebfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563540781390 0x563540b5f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44496/0 (socket says 172.21.15.100:44496) 2024-07-26T13:44:17.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.906+0000 7f24ebfff640 1 -- 172.21.15.100:0/1602356643 learned_addr learned my addr 172.21.15.100:0/1602356643 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:17.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.906+0000 7f24f11d2640 1 --2- 172.21.15.100:0/1602356643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635407a2070 0x563540b5f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:17.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.906+0000 7f24f09d1640 1 -- 172.21.15.100:0/1602356643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635407a2070 msgr2=0x563540b5f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:17.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.906+0000 7f24f09d1640 1 --2- 172.21.15.100:0/1602356643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635407a2070 0x563540b5f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.906+0000 7f24f09d1640 1 -- 172.21.15.100:0/1602356643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563540781390 msgr2=0x563540b5f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:17.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.906+0000 7f24f09d1640 1 --2- 172.21.15.100:0/1602356643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563540781390 0x563540b5f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.906+0000 7f24f09d1640 1 -- 172.21.15.100:0/1602356643 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563540b681c0 con 0x5635407791b0 2024-07-26T13:44:17.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.906+0000 7f24ebfff640 1 --2- 172.21.15.100:0/1602356643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563540781390 0x563540b5f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:44:17.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.907+0000 7f24f09d1640 1 --2- 172.21.15.100:0/1602356643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635407791b0 0x563540b63f50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f24e0011910 tx=0x7f24e0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc3e9e1bf504d660 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:17.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.907+0000 7f24cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-07-26T13:44:17.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.907+0000 7f24cbfff640 1 -- 172.21.15.100:0/1602356643 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e0029040 con 0x5635407791b0 2024-07-26T13:44:17.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.907+0000 7f24f91b9740 1 -- 172.21.15.100:0/1602356643 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24d80af490 con 0x5635407791b0 2024-07-26T13:44:17.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.907+0000 7f24cbfff640 1 -- 172.21.15.100:0/1602356643 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f24e0005460 con 0x5635407791b0 2024-07-26T13:44:17.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.907+0000 7f24cbfff640 1 -- 172.21.15.100:0/1602356643 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f24e0004be0 con 0x5635407791b0 2024-07-26T13:44:17.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.907+0000 7f24f91b9740 10 client.26056 fetch_fsmap learned FSMap version 37 2024-07-26T13:44:17.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.907+0000 7f24f91b9740 1 -- 172.21.15.100:0/1602356643 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563540bd5bd0 con 0x5635407791b0 2024-07-26T13:44:17.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.908+0000 7f24cbfff640 1 -- 172.21.15.100:0/1602356643 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24d80af490 con 0x5635407791b0 2024-07-26T13:44:17.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.908+0000 7f24cbfff640 1 -- 172.21.15.100:0/1602356643 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f24e000f670 con 0x5635407791b0 2024-07-26T13:44:17.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.908+0000 7f24f91b9740 10 client.26056 fetch_fsmap finished waiting for FSMap version 37 2024-07-26T13:44:17.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.908+0000 7f24f91b9740 10 client.26056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:44:17.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.908+0000 7f24f91b9740 20 client.26056 populate_metadata read hostname 'smithi100' 2024-07-26T13:44:17.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.908+0000 7f24f91b9740 1 --2- 172.21.15.100:0/1602356643 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x563540735da0 0x563540bd8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:17.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.908+0000 7f24f91b9740 4 client.26056 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-26T13:44:17.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.908+0000 7f24f91b9740 1 -- 172.21.15.100:0/1602356643 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563540abb5a0 con 0x563540735da0 2024-07-26T13:44:17.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.908+0000 7f24ebfff640 1 --2- 172.21.15.100:0/1602356643 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x563540735da0 0x563540bd8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:17.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.909+0000 7f24cbfff640 10 client.26056 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:17.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.909+0000 7f24ebfff640 1 --2- 172.21.15.100:0/1602356643 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x563540735da0 0x563540bd8340 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26dc1f37df9b17b3 server_cookie=be9b47b50e976bac in_seq=0 out_seq=0 2024-07-26T13:44:17.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.910+0000 7f24cbfff640 1 -- 172.21.15.100:0/1602356643 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563540abb5a0 con 0x563540735da0 2024-07-26T13:44:17.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.910+0000 7f24cbfff640 10 client.26056 handle_command_reply: tid=0 2024-07-26T13:44:17.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.910+0000 7f24f91b9740 1 client.26056 shutdown 2024-07-26T13:44:17.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.910+0000 7f24f91b9740 1 -- 172.21.15.100:0/1602356643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635407791b0 msgr2=0x563540b63f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:17.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.910+0000 7f24f91b9740 1 --2- 172.21.15.100:0/1602356643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635407791b0 0x563540b63f50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f24e0011910 tx=0x7f24e0004a40 comp rx=0 tx=0).stop 2024-07-26T13:44:17.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.910+0000 7f24f91b9740 1 -- 172.21.15.100:0/1602356643 shutdown_connections 2024-07-26T13:44:17.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.910+0000 7f24f91b9740 1 --2- 172.21.15.100:0/1602356643 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x563540735da0 0x563540bd8340 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.910+0000 7f24f91b9740 1 --2- 172.21.15.100:0/1602356643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635407a2070 0x563540b5f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.910+0000 7f24f91b9740 1 --2- 172.21.15.100:0/1602356643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563540781390 0x563540b5f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.910+0000 7f24f91b9740 1 --2- 172.21.15.100:0/1602356643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5635407791b0 0x563540b63f50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.910+0000 7f24cbfff640 0 client.26056 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:17.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.910+0000 7f24f91b9740 1 -- 172.21.15.100:0/1602356643 >> 172.21.15.100:0/1602356643 conn(0x563540796460 msgr2=0x563540b4dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:17.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.910+0000 7f24f91b9740 1 -- 172.21.15.100:0/1602356643 shutdown_connections 2024-07-26T13:44:17.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.910+0000 7f24f91b9740 1 -- 172.21.15.100:0/1602356643 wait complete. 2024-07-26T13:44:17.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.910+0000 7f24f91b9740 20 client.26056 trim_cache size 0 max 16384 2024-07-26T13:44:17.917 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-07-26T13:44:17.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.911+0000 7f24f2c35640 1 -- 172.21.15.100:0/465446718 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f24dc0b3dc0 msgr2=0x7f24dc0b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:17.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.911+0000 7f24f2c35640 1 --2- 172.21.15.100:0/465446718 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f24dc0b3dc0 0x7f24dc0b62d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f24e4005e30 tx=0x7f24e4005c30 comp rx=0 tx=0).stop 2024-07-26T13:44:17.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.911+0000 7f24f2c35640 1 -- 172.21.15.100:0/465446718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24ec108f10 msgr2=0x7f24ec19a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:17.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.911+0000 7f24f2c35640 1 --2- 172.21.15.100:0/465446718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24ec108f10 0x7f24ec19a480 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f24d800e970 tx=0x7f24d800ee30 comp rx=0 tx=0).stop 2024-07-26T13:44:17.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.911+0000 7f24f2c35640 1 -- 172.21.15.100:0/465446718 shutdown_connections 2024-07-26T13:44:17.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.911+0000 7f24f2c35640 1 --2- 172.21.15.100:0/465446718 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f24dc0b3dc0 0x7f24dc0b62d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.911+0000 7f24f2c35640 1 --2- 172.21.15.100:0/465446718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24ec109860 0x7f24ec19f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.911+0000 7f24f2c35640 1 --2- 172.21.15.100:0/465446718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24ec108f10 0x7f24ec19a480 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.911+0000 7f24f2c35640 1 --2- 172.21.15.100:0/465446718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24ec1035d0 0x7f24ec199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.911+0000 7f24f2c35640 1 -- 172.21.15.100:0/465446718 >> 172.21.15.100:0/465446718 conn(0x7f24ec0ff060 msgr2=0x7f24ec1064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:17.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.911+0000 7f24f2c35640 1 -- 172.21.15.100:0/465446718 shutdown_connections 2024-07-26T13:44:17.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:17.911+0000 7f24f2c35640 1 -- 172.21.15.100:0/465446718 wait complete. 2024-07-26T13:44:17.920 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.912+0000 7f6de77fe640 1 -- 172.21.15.100:0/2249419186 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f3b6055da0 con 0x55f3b5d401b0 2024-07-26T13:44:17.920 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.912+0000 7f6de77fe640 10 client.25901 handle_command_reply: tid=0 2024-07-26T13:44:17.920 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.912+0000 7f6e15882740 1 client.25901 shutdown 2024-07-26T13:44:17.920 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.912+0000 7f6e15882740 1 -- 172.21.15.100:0/2249419186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3b5cfed80 msgr2=0x55f3b60f51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:17.920 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.912+0000 7f6e15882740 1 --2- 172.21.15.100:0/2249419186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3b5cfed80 0x55f3b60f51b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6df8005060 tx=0x7f6df802a410 comp rx=0 tx=0).stop 2024-07-26T13:44:17.920 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.912+0000 7f6e15882740 1 -- 172.21.15.100:0/2249419186 shutdown_connections 2024-07-26T13:44:17.920 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.912+0000 7f6e15882740 1 --2- 172.21.15.100:0/2249419186 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f3b5d401b0 0x55f3b6173240 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.920 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.912+0000 7f6e15882740 1 --2- 172.21.15.100:0/2249419186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5d1f4d0 0x55f3b61690b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.920 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.912+0000 7f6e15882740 1 --2- 172.21.15.100:0/2249419186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f3b5d172f0 0x55f3b60f5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.912+0000 7f6e15882740 1 --2- 172.21.15.100:0/2249419186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f3b5cfed80 0x55f3b60f51b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.912+0000 7f6e15882740 1 -- 172.21.15.100:0/2249419186 >> 172.21.15.100:0/2249419186 conn(0x55f3b5d34300 msgr2=0x55f3b60e7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.912+0000 7f6de77fe640 0 client.25901 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.912+0000 7f6e15882740 1 -- 172.21.15.100:0/2249419186 shutdown_connections 2024-07-26T13:44:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.912+0000 7f6e15882740 1 -- 172.21.15.100:0/2249419186 wait complete. 2024-07-26T13:44:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.913+0000 7f6e15882740 20 client.25901 trim_cache size 0 max 16384 2024-07-26T13:44:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-07-26T13:44:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-07-26T13:44:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-07-26T13:44:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-07-26T13:44:17.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.913+0000 7f6e0e96c640 1 -- 172.21.15.100:0/2239329323 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6df00b3d40 msgr2=0x7f6df00b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:17.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.913+0000 7f6e0e96c640 1 --2- 172.21.15.100:0/2239329323 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6df00b3d40 0x7f6df00b6250 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6e00021000 tx=0x7f6e0001e000 comp rx=0 tx=0).stop 2024-07-26T13:44:17.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.913+0000 7f6e0e96c640 1 -- 172.21.15.100:0/2239329323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e08108e60 msgr2=0x7f6e0819a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:17.922 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.913+0000 7f6e0e96c640 1 --2- 172.21.15.100:0/2239329323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e08108e60 0x7f6e0819a4a0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc00d750 tx=0x7f6dfc00dc10 comp rx=0 tx=0).stop 2024-07-26T13:44:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.913+0000 7f6e0e96c640 1 -- 172.21.15.100:0/2239329323 shutdown_connections 2024-07-26T13:44:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.913+0000 7f6e0e96c640 1 --2- 172.21.15.100:0/2239329323 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6df00b3d40 0x7f6df00b6250 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.913+0000 7f6e0e96c640 1 --2- 172.21.15.100:0/2239329323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e081097b0 0x7f6e0819f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.913+0000 7f6e0e96c640 1 --2- 172.21.15.100:0/2239329323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e08108e60 0x7f6e0819a4a0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.913+0000 7f6e0e96c640 1 --2- 172.21.15.100:0/2239329323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e08101a90 0x7f6e08199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.913+0000 7f6e0e96c640 1 -- 172.21.15.100:0/2239329323 >> 172.21.15.100:0/2239329323 conn(0x7f6e080798c0 msgr2=0x7f6e08106fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.913+0000 7f6e0e96c640 1 -- 172.21.15.100:0/2239329323 shutdown_connections 2024-07-26T13:44:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:17.913+0000 7f6e0e96c640 1 -- 172.21.15.100:0/2239329323 wait complete. 2024-07-26T13:44:17.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:17 smithi138 ceph-mon[36954]: pgmap v490: 97 pgs: 97 active+clean; 61 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 1.7 MiB/s wr, 74 op/s 2024-07-26T13:44:18.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:17 smithi167 ceph-mon[37140]: pgmap v490: 97 pgs: 97 active+clean; 61 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 1.7 MiB/s wr, 74 op/s 2024-07-26T13:44:18.412 INFO:tasks.workunit.client.0.smithi100.stdout: 1 145991 0.00 MB/sec execute 32 sec latency 32311.597 ms 2024-07-26T13:44:19.412 INFO:tasks.workunit.client.0.smithi100.stdout: 1 145991 0.00 MB/sec execute 33 sec latency 33311.682 ms 2024-07-26T13:44:19.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:19 smithi167 ceph-mon[37140]: pgmap v491: 97 pgs: 97 active+clean; 61 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 1.9 MiB/s wr, 80 op/s 2024-07-26T13:44:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:19 smithi138 ceph-mon[36954]: pgmap v491: 97 pgs: 97 active+clean; 61 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 1.9 MiB/s wr, 80 op/s 2024-07-26T13:44:19.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:19 smithi100 ceph-mon[32599]: pgmap v491: 97 pgs: 97 active+clean; 61 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 1.9 MiB/s wr, 80 op/s 2024-07-26T13:44:20.412 INFO:tasks.workunit.client.0.smithi100.stdout: 1 145991 0.00 MB/sec execute 34 sec latency 34311.753 ms 2024-07-26T13:44:20.566 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-26T13:44:20.566 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-26T13:44:20.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.575+0000 7f94c23f1640 1 -- 172.21.15.100:0/2531840996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94bc10d820 msgr2=0x7f94bc10fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:20.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.575+0000 7f94c23f1640 1 --2- 172.21.15.100:0/2531840996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94bc10d820 0x7f94bc10fd30 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f94b4009ca0 tx=0x7f94b402a8c0 comp rx=0 tx=0).stop 2024-07-26T13:44:20.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.575+0000 7f94c23f1640 1 -- 172.21.15.100:0/2531840996 shutdown_connections 2024-07-26T13:44:20.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.575+0000 7f94c23f1640 1 --2- 172.21.15.100:0/2531840996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94bc10d820 0x7f94bc10fd30 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:20.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.575+0000 7f94c23f1640 1 --2- 172.21.15.100:0/2531840996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94bc104ca0 0x7f94bc10d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:20.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.575+0000 7f94c23f1640 1 --2- 172.21.15.100:0/2531840996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94bc104270 0x7f94bc104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:20.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.575+0000 7f94c23f1640 1 -- 172.21.15.100:0/2531840996 >> 172.21.15.100:0/2531840996 conn(0x7f94bc079160 msgr2=0x7f94bc079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:20.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.575+0000 7f94c23f1640 1 -- 172.21.15.100:0/2531840996 shutdown_connections 2024-07-26T13:44:20.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.575+0000 7f94c23f1640 1 -- 172.21.15.100:0/2531840996 wait complete. 2024-07-26T13:44:20.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.576+0000 7f94c23f1640 1 Processor -- start 2024-07-26T13:44:20.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.576+0000 7f94c23f1640 1 -- start start 2024-07-26T13:44:20.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.576+0000 7f94c23f1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94bc104270 0x7f94bc1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:20.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.576+0000 7f94c23f1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94bc104ca0 0x7f94bc198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:20.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.576+0000 7f94c23f1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94bc10d820 0x7f94bc19fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:20.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.576+0000 7f94c23f1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f94bc1134f0 con 0x7f94bc104270 2024-07-26T13:44:20.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.576+0000 7f94c23f1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f94bc113370 con 0x7f94bc104ca0 2024-07-26T13:44:20.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.576+0000 7f94c23f1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f94bc113670 con 0x7f94bc10d820 2024-07-26T13:44:20.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.576+0000 7f94bbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94bc104270 0x7f94bc1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:20.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.577+0000 7f94bbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94bc104270 0x7f94bc1989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44508/0 (socket says 172.21.15.100:44508) 2024-07-26T13:44:20.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.577+0000 7f94bbfff640 1 -- 172.21.15.100:0/1737725639 learned_addr learned my addr 172.21.15.100:0/1737725639 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:20.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.577+0000 7f94c098e640 1 --2- 172.21.15.100:0/1737725639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94bc10d820 0x7f94bc19fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:20.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.577+0000 7f94bb7fe640 1 --2- 172.21.15.100:0/1737725639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94bc104ca0 0x7f94bc198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:20.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.577+0000 7f94bbfff640 1 -- 172.21.15.100:0/1737725639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94bc10d820 msgr2=0x7f94bc19fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:20.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.577+0000 7f94bbfff640 1 --2- 172.21.15.100:0/1737725639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94bc10d820 0x7f94bc19fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:20.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.577+0000 7f94bbfff640 1 -- 172.21.15.100:0/1737725639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94bc104ca0 msgr2=0x7f94bc198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:20.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.577+0000 7f94bbfff640 1 --2- 172.21.15.100:0/1737725639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94bc104ca0 0x7f94bc198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:20.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.577+0000 7f94bbfff640 1 -- 172.21.15.100:0/1737725639 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94bc1997a0 con 0x7f94bc104270 2024-07-26T13:44:20.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.577+0000 7f94bbfff640 1 --2- 172.21.15.100:0/1737725639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94bc104270 0x7f94bc1989c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f94ac00e9c0 tx=0x7f94ac00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:20.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.577+0000 7f94b97fa640 1 -- 172.21.15.100:0/1737725639 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac00cd80 con 0x7f94bc104270 2024-07-26T13:44:20.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.577+0000 7f94c23f1640 1 -- 172.21.15.100:0/1737725639 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94bc1a0020 con 0x7f94bc104270 2024-07-26T13:44:20.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.577+0000 7f94b97fa640 1 -- 172.21.15.100:0/1737725639 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94ac0035f0 con 0x7f94bc104270 2024-07-26T13:44:20.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.577+0000 7f94b97fa640 1 -- 172.21.15.100:0/1737725639 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac010690 con 0x7f94bc104270 2024-07-26T13:44:20.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.577+0000 7f94c23f1640 1 -- 172.21.15.100:0/1737725639 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94bc1a0560 con 0x7f94bc104270 2024-07-26T13:44:20.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.580+0000 7f94b97fa640 1 -- 172.21.15.100:0/1737725639 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f94ac0037a0 con 0x7f94bc104270 2024-07-26T13:44:20.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.580+0000 7f94c8975740 1 Processor -- start 2024-07-26T13:44:20.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.580+0000 7f94c8975740 1 -- start start 2024-07-26T13:44:20.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.581+0000 7f94c8975740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cec91af520 0x55cec9555070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:20.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.581+0000 7f94c8975740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cec9186660 0x55cec95555b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:20.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.581+0000 7f94c8975740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cec918e840 0x55cec9555af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:20.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.581+0000 7f94c8975740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55cec94cae70 con 0x55cec918e840 2024-07-26T13:44:20.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.581+0000 7f94c8975740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cec94ba370 con 0x55cec9186660 2024-07-26T13:44:20.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.581+0000 7f94c8975740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cec94be3b0 con 0x55cec91af520 2024-07-26T13:44:20.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.581+0000 7f94bb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cec918e840 0x55cec9555af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:20.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.581+0000 7f94bb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cec918e840 0x55cec9555af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44514/0 (socket says 172.21.15.100:44514) 2024-07-26T13:44:20.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.581+0000 7f94bb7fe640 1 -- 172.21.15.100:0/2562108974 learned_addr learned my addr 172.21.15.100:0/2562108974 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:20.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.581+0000 7f94c098e640 1 --2- 172.21.15.100:0/2562108974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cec91af520 0x55cec9555070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:20.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.581+0000 7f94bbfff640 1 --2- 172.21.15.100:0/2562108974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cec9186660 0x55cec95555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:20.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.581+0000 7f94bb7fe640 1 -- 172.21.15.100:0/2562108974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cec91af520 msgr2=0x55cec9555070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:20.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.581+0000 7f94bb7fe640 1 --2- 172.21.15.100:0/2562108974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cec91af520 0x55cec9555070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:20.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.581+0000 7f94bb7fe640 1 -- 172.21.15.100:0/2562108974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cec9186660 msgr2=0x55cec95555b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:20.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.581+0000 7f94bb7fe640 1 --2- 172.21.15.100:0/2562108974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cec9186660 0x55cec95555b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:20.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.581+0000 7f94bb7fe640 1 -- 172.21.15.100:0/2562108974 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cec9487040 con 0x55cec918e840 2024-07-26T13:44:20.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.581+0000 7f94b97fa640 1 --2- 172.21.15.100:0/1737725639 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f94a40b3d40 0x7f94a40b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:20.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.582+0000 7f94c098e640 1 --2- 172.21.15.100:0/1737725639 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f94a40b3d40 0x7f94a40b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:20.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.582+0000 7f94bb7fe640 1 --2- 172.21.15.100:0/2562108974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cec918e840 0x55cec9555af0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f94b000dc10 tx=0x7f94b000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922383e0eac2a37c server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:20.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.582+0000 7f94b97fa640 1 -- 172.21.15.100:0/1737725639 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f94ac0e4480 con 0x7f94bc104270 2024-07-26T13:44:20.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.582+0000 7f94a27fc640 1 -- 172.21.15.100:0/2562108974 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b0005020 con 0x55cec918e840 2024-07-26T13:44:20.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.582+0000 7f94a27fc640 1 -- 172.21.15.100:0/2562108974 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94b00047a0 con 0x55cec918e840 2024-07-26T13:44:20.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.582+0000 7f94a27fc640 1 -- 172.21.15.100:0/2562108974 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b00116a0 con 0x55cec918e840 2024-07-26T13:44:20.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.583+0000 7f94c8975740 1 -- 172.21.15.100:0/2562108974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cec918e840 msgr2=0x55cec9555af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:20.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.583+0000 7f94c098e640 1 --2- 172.21.15.100:0/1737725639 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f94a40b3d40 0x7f94a40b6250 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f94b4006010 tx=0x7f94b4005d80 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:20.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.583+0000 7f94c8975740 1 --2- 172.21.15.100:0/2562108974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cec918e840 0x55cec9555af0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f94b000dc10 tx=0x7f94b000f730 comp rx=0 tx=0).stop 2024-07-26T13:44:20.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.583+0000 7f94c8975740 1 -- 172.21.15.100:0/2562108974 shutdown_connections 2024-07-26T13:44:20.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.583+0000 7f94c8975740 1 --2- 172.21.15.100:0/2562108974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cec918e840 0x55cec9555af0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:20.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.583+0000 7f94c8975740 1 --2- 172.21.15.100:0/2562108974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cec9186660 0x55cec95555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:20.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.583+0000 7f94c8975740 1 --2- 172.21.15.100:0/2562108974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cec91af520 0x55cec9555070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:20.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.583+0000 7f94c8975740 1 -- 172.21.15.100:0/2562108974 >> 172.21.15.100:0/2562108974 conn(0x55cec91a3910 msgr2=0x55cec9552f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:20.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.583+0000 7f94c8975740 1 -- 172.21.15.100:0/2562108974 shutdown_connections 2024-07-26T13:44:20.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.583+0000 7f94c8975740 1 -- 172.21.15.100:0/2562108974 wait complete. 2024-07-26T13:44:20.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.583+0000 7f94c8975740 1 -- start start 2024-07-26T13:44:20.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.584+0000 7f94c8975740 1 Processor -- start 2024-07-26T13:44:20.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.584+0000 7f94c8975740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cec9186660 0x55cec9560be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:20.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.584+0000 7f94c8975740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cec918e840 0x55cec9561120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:20.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.584+0000 7f94c8975740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cec91af520 0x55cec956cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:20.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.584+0000 7f94c8975740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55cec931dbb0 con 0x55cec91af520 2024-07-26T13:44:20.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.584+0000 7f94c8975740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cec945cac0 con 0x55cec918e840 2024-07-26T13:44:20.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.584+0000 7f94c8975740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cec9563850 con 0x55cec9186660 2024-07-26T13:44:20.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.584+0000 7f94c8975740 10 client.0 _ll_register_callbacks cb 0x55cec94653b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:44:20.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.584+0000 7f94c098e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cec91af520 0x55cec956cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:20.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.584+0000 7f94c098e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cec91af520 0x55cec956cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44516/0 (socket says 172.21.15.100:44516) 2024-07-26T13:44:20.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.584+0000 7f94c098e640 1 -- 172.21.15.100:0/615995745 learned_addr learned my addr 172.21.15.100:0/615995745 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:20.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.584+0000 7f94bbfff640 1 --2- 172.21.15.100:0/615995745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cec9186660 0x55cec9560be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:20.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.584+0000 7f94bb7fe640 1 --2- 172.21.15.100:0/615995745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cec918e840 0x55cec9561120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:20.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.585+0000 7f94c098e640 1 -- 172.21.15.100:0/615995745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cec9186660 msgr2=0x55cec9560be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:20.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.585+0000 7f94c098e640 1 --2- 172.21.15.100:0/615995745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cec9186660 0x55cec9560be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:20.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.585+0000 7f94c098e640 1 -- 172.21.15.100:0/615995745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cec918e840 msgr2=0x55cec9561120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:20.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.585+0000 7f94c098e640 1 --2- 172.21.15.100:0/615995745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cec918e840 0x55cec9561120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:20.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.585+0000 7f94c098e640 1 -- 172.21.15.100:0/615995745 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cec921ab00 con 0x55cec91af520 2024-07-26T13:44:20.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.585+0000 7f94bb7fe640 1 --2- 172.21.15.100:0/615995745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cec918e840 0x55cec9561120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:44:20.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.585+0000 7f94c098e640 1 --2- 172.21.15.100:0/615995745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cec91af520 0x55cec956cb40 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f94b402f540 tx=0x7f94b402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a846b6c6d3fefb7 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:20.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.585+0000 7f949b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:44:20.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.585+0000 7f949b7fe640 1 -- 172.21.15.100:0/615995745 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b402f610 con 0x55cec91af520 2024-07-26T13:44:20.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.585+0000 7f94c8975740 1 -- 172.21.15.100:0/615995745 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94b00047a0 con 0x55cec91af520 2024-07-26T13:44:20.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.585+0000 7f949b7fe640 1 -- 172.21.15.100:0/615995745 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f94b402f7b0 con 0x55cec91af520 2024-07-26T13:44:20.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.585+0000 7f949b7fe640 1 -- 172.21.15.100:0/615995745 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f94b402f9b0 con 0x55cec91af520 2024-07-26T13:44:20.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.585+0000 7f94c8975740 10 client.16602 fetch_fsmap learned FSMap version 37 2024-07-26T13:44:20.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.585+0000 7f94c8975740 1 -- 172.21.15.100:0/615995745 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55cec9291960 con 0x55cec91af520 2024-07-26T13:44:20.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.586+0000 7f949b7fe640 1 -- 172.21.15.100:0/615995745 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94b00047a0 con 0x55cec91af520 2024-07-26T13:44:20.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.586+0000 7f949b7fe640 1 -- 172.21.15.100:0/615995745 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f94b4051070 con 0x55cec91af520 2024-07-26T13:44:20.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.586+0000 7f94c8975740 10 client.16602 fetch_fsmap finished waiting for FSMap version 37 2024-07-26T13:44:20.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.586+0000 7f94c8975740 10 client.16602 resolve_mds: validated gid 24607 aka daemon mds.i 2024-07-26T13:44:20.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.586+0000 7f94c8975740 20 client.16602 populate_metadata read hostname 'smithi100' 2024-07-26T13:44:20.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.586+0000 7f94c8975740 1 --2- 172.21.15.100:0/615995745 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55cec916dea0 0x55cec9572b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:20.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.586+0000 7f94c8975740 4 client.16602 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:44:20.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.586+0000 7f94c8975740 1 -- 172.21.15.100:0/615995745 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cec94ccdc0 con 0x55cec916dea0 2024-07-26T13:44:20.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.586+0000 7f94bbfff640 1 --2- 172.21.15.100:0/615995745 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55cec916dea0 0x55cec9572b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:20.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.587+0000 7f949b7fe640 10 client.16602 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:20.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.587+0000 7f94bbfff640 1 --2- 172.21.15.100:0/615995745 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55cec916dea0 0x55cec9572b30 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cdc67a6e0c23398 server_cookie=3176327a47fffcf2 in_seq=0 out_seq=0 2024-07-26T13:44:20.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.587+0000 7f949b7fe640 1 -- 172.21.15.100:0/615995745 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cec94ccdc0 con 0x55cec916dea0 2024-07-26T13:44:20.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.587+0000 7f949b7fe640 10 client.16602 handle_command_reply: tid=0 2024-07-26T13:44:20.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.587+0000 7f94c8975740 1 client.16602 shutdown 2024-07-26T13:44:20.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.588+0000 7f94c8975740 1 -- 172.21.15.100:0/615995745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cec91af520 msgr2=0x55cec956cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:20.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.588+0000 7f94c8975740 1 --2- 172.21.15.100:0/615995745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cec91af520 0x55cec956cb40 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f94b402f540 tx=0x7f94b402a8c0 comp rx=0 tx=0).stop 2024-07-26T13:44:20.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.588+0000 7f94c8975740 1 -- 172.21.15.100:0/615995745 shutdown_connections 2024-07-26T13:44:20.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.588+0000 7f94c8975740 1 --2- 172.21.15.100:0/615995745 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55cec916dea0 0x55cec9572b30 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:20.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.588+0000 7f94c8975740 1 --2- 172.21.15.100:0/615995745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cec91af520 0x55cec956cb40 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:20.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.588+0000 7f949b7fe640 0 client.16602 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:20.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.588+0000 7f94c8975740 1 --2- 172.21.15.100:0/615995745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cec918e840 0x55cec9561120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:20.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.588+0000 7f94c8975740 1 --2- 172.21.15.100:0/615995745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cec9186660 0x55cec9560be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:20.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.588+0000 7f94c8975740 1 -- 172.21.15.100:0/615995745 >> 172.21.15.100:0/615995745 conn(0x55cec91a3910 msgr2=0x55cec9558250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:20.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.588+0000 7f94c8975740 1 -- 172.21.15.100:0/615995745 shutdown_connections 2024-07-26T13:44:20.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.588+0000 7f94c8975740 1 -- 172.21.15.100:0/615995745 wait complete. 2024-07-26T13:44:20.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.588+0000 7f94c8975740 20 client.16602 trim_cache size 0 max 16384 2024-07-26T13:44:20.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.606+0000 7f94c8975740 1 Processor -- start 2024-07-26T13:44:20.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.606+0000 7f94c8975740 1 -- start start 2024-07-26T13:44:20.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.606+0000 7f94c8975740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cec916dea0 0x55cec9558b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:20.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.606+0000 7f94c8975740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cec9186660 0x55cec956c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:20.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.606+0000 7f94c8975740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cec918e840 0x55cec9552880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:20.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.606+0000 7f94c8975740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f94bc1c0950 con 0x55cec918e840 2024-07-26T13:44:20.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.606+0000 7f94c8975740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f94bc1be1c0 con 0x55cec9186660 2024-07-26T13:44:20.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.606+0000 7f94c8975740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f94bc1bc010 con 0x55cec916dea0 2024-07-26T13:44:20.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.606+0000 7f94c098e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cec918e840 0x55cec9552880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:20.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.606+0000 7f94bbfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cec916dea0 0x55cec9558b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:20.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.606+0000 7f94c098e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cec918e840 0x55cec9552880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44520/0 (socket says 172.21.15.100:44520) 2024-07-26T13:44:20.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.606+0000 7f94c098e640 1 -- 172.21.15.100:0/1702997533 learned_addr learned my addr 172.21.15.100:0/1702997533 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:20.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.606+0000 7f94bb7fe640 1 --2- 172.21.15.100:0/1702997533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cec9186660 0x55cec956c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:20.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.607+0000 7f94c098e640 1 -- 172.21.15.100:0/1702997533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cec916dea0 msgr2=0x55cec9558b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:20.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.607+0000 7f94c098e640 1 --2- 172.21.15.100:0/1702997533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cec916dea0 0x55cec9558b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:20.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.607+0000 7f94c098e640 1 -- 172.21.15.100:0/1702997533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cec9186660 msgr2=0x55cec956c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:20.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.607+0000 7f94c098e640 1 --2- 172.21.15.100:0/1702997533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cec9186660 0x55cec956c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:20.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.607+0000 7f94c098e640 1 -- 172.21.15.100:0/1702997533 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cec9561910 con 0x55cec918e840 2024-07-26T13:44:20.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.607+0000 7f94bbfff640 1 --2- 172.21.15.100:0/1702997533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cec916dea0 0x55cec9558b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:44:20.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.607+0000 7f94c098e640 1 --2- 172.21.15.100:0/1702997533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cec918e840 0x55cec9552880 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f94b402ade0 tx=0x7f94b402f430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14b2228b23662d9 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:20.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.607+0000 7f94a1ffb640 1 -- 172.21.15.100:0/1702997533 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b402f750 con 0x55cec918e840 2024-07-26T13:44:20.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.607+0000 7f94a1ffb640 1 -- 172.21.15.100:0/1702997533 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94b4036050 con 0x55cec918e840 2024-07-26T13:44:20.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.607+0000 7f94a1ffb640 1 -- 172.21.15.100:0/1702997533 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b4030ca0 con 0x55cec918e840 2024-07-26T13:44:20.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.608+0000 7f94c8975740 1 -- 172.21.15.100:0/1702997533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cec918e840 msgr2=0x55cec9552880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:20.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.608+0000 7f94c8975740 1 --2- 172.21.15.100:0/1702997533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cec918e840 0x55cec9552880 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f94b402ade0 tx=0x7f94b402f430 comp rx=0 tx=0).stop 2024-07-26T13:44:20.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.608+0000 7f94c8975740 1 -- 172.21.15.100:0/1702997533 shutdown_connections 2024-07-26T13:44:20.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.608+0000 7f94c8975740 1 --2- 172.21.15.100:0/1702997533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cec918e840 0x55cec9552880 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:20.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.608+0000 7f94c8975740 1 --2- 172.21.15.100:0/1702997533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cec9186660 0x55cec956c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:20.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.608+0000 7f94c8975740 1 --2- 172.21.15.100:0/1702997533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cec916dea0 0x55cec9558b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:20.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.608+0000 7f94c8975740 1 -- 172.21.15.100:0/1702997533 >> 172.21.15.100:0/1702997533 conn(0x55cec91a3910 msgr2=0x55cec955f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:20.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.608+0000 7f94c8975740 1 -- 172.21.15.100:0/1702997533 shutdown_connections 2024-07-26T13:44:20.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.608+0000 7f94c8975740 1 -- 172.21.15.100:0/1702997533 wait complete. 2024-07-26T13:44:20.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.608+0000 7f94c8975740 1 -- start start 2024-07-26T13:44:20.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.609+0000 7f94c8975740 1 Processor -- start 2024-07-26T13:44:20.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.609+0000 7f94c8975740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cec916dea0 0x55cec9564d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:20.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.609+0000 7f94c8975740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cec9186660 0x55cec9565260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:20.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.609+0000 7f94c8975740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cec918e840 0x55cec95da2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:20.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.609+0000 7f94c8975740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f94bc1bc680 con 0x55cec916dea0 2024-07-26T13:44:20.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.609+0000 7f94c8975740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f94bc1bdaf0 con 0x55cec9186660 2024-07-26T13:44:20.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.609+0000 7f94c8975740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f94bc1bcd50 con 0x55cec918e840 2024-07-26T13:44:20.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.609+0000 7f94c8975740 10 client.0 _ll_register_callbacks cb 0x7f94b4009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:44:20.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.609+0000 7f94bbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cec916dea0 0x55cec9564d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:20.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.609+0000 7f94bbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cec916dea0 0x55cec9564d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44536/0 (socket says 172.21.15.100:44536) 2024-07-26T13:44:20.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.609+0000 7f94bbfff640 1 -- 172.21.15.100:0/88771017 learned_addr learned my addr 172.21.15.100:0/88771017 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:20.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.609+0000 7f94c098e640 1 --2- 172.21.15.100:0/88771017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cec918e840 0x55cec95da2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:20.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.609+0000 7f94bb7fe640 1 --2- 172.21.15.100:0/88771017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cec9186660 0x55cec9565260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:20.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.610+0000 7f94bbfff640 1 -- 172.21.15.100:0/88771017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cec918e840 msgr2=0x55cec95da2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:20.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.610+0000 7f94bbfff640 1 --2- 172.21.15.100:0/88771017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cec918e840 0x55cec95da2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:20.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.610+0000 7f94bbfff640 1 -- 172.21.15.100:0/88771017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cec9186660 msgr2=0x55cec9565260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:20.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.610+0000 7f94bbfff640 1 --2- 172.21.15.100:0/88771017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cec9186660 0x55cec9565260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:20.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.610+0000 7f94bbfff640 1 -- 172.21.15.100:0/88771017 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cec9565b50 con 0x55cec916dea0 2024-07-26T13:44:20.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.610+0000 7f94c098e640 1 --2- 172.21.15.100:0/88771017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cec918e840 0x55cec95da2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:44:20.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.610+0000 7f94bbfff640 1 --2- 172.21.15.100:0/88771017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cec916dea0 0x55cec9564d20 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f94ac0e1c00 tx=0x7f94ac003cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906d0e058db53dc7 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:20.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.610+0000 7f949b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:44:20.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.610+0000 7f949b7fe640 1 -- 172.21.15.100:0/88771017 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac0aa7d0 con 0x55cec916dea0 2024-07-26T13:44:20.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.610+0000 7f949b7fe640 1 -- 172.21.15.100:0/88771017 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f94ac0a72f0 con 0x55cec916dea0 2024-07-26T13:44:20.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.610+0000 7f94c8975740 1 -- 172.21.15.100:0/88771017 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94b4036050 con 0x55cec916dea0 2024-07-26T13:44:20.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.610+0000 7f949b7fe640 1 -- 172.21.15.100:0/88771017 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f94ac0aab70 con 0x55cec916dea0 2024-07-26T13:44:20.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.610+0000 7f94c8975740 10 client.16608 fetch_fsmap learned FSMap version 37 2024-07-26T13:44:20.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.610+0000 7f94c8975740 1 -- 172.21.15.100:0/88771017 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55cec95db950 con 0x55cec916dea0 2024-07-26T13:44:20.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.610+0000 7f949b7fe640 1 -- 172.21.15.100:0/88771017 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94b4036050 con 0x55cec916dea0 2024-07-26T13:44:20.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.611+0000 7f949b7fe640 1 -- 172.21.15.100:0/88771017 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f94ac0e3040 con 0x55cec916dea0 2024-07-26T13:44:20.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.611+0000 7f94c8975740 10 client.16608 fetch_fsmap finished waiting for FSMap version 37 2024-07-26T13:44:20.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.611+0000 7f94c8975740 10 client.16608 resolve_mds: validated gid 24607 aka daemon mds.i 2024-07-26T13:44:20.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.611+0000 7f94c8975740 20 client.16608 populate_metadata read hostname 'smithi100' 2024-07-26T13:44:20.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.611+0000 7f94c8975740 1 --2- 172.21.15.100:0/88771017 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55cec91af520 0x55cec95de240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:20.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.611+0000 7f94c8975740 4 client.16608 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "47f95d77", "release": true, "await": true}] 2024-07-26T13:44:20.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.611+0000 7f94c8975740 1 -- 172.21.15.100:0/88771017 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "quiesce db", "set_id": "47f95d77", "release": true, "await": true}) -- 0x55cec94c8860 con 0x55cec91af520 2024-07-26T13:44:20.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.611+0000 7f94bb7fe640 1 --2- 172.21.15.100:0/88771017 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55cec91af520 0x55cec95de240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:20.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.611+0000 7f94bb7fe640 1 --2- 172.21.15.100:0/88771017 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55cec91af520 0x55cec95de240 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=813884bfc646adf2 server_cookie=22c3e4524a769ca9 in_seq=0 out_seq=0 2024-07-26T13:44:20.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.611+0000 7f949b7fe640 10 client.16608 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:20.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.616+0000 7f949b7fe640 1 -- 172.21.15.100:0/88771017 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55cec94c8860 con 0x55cec91af520 2024-07-26T13:44:20.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.616+0000 7f949b7fe640 10 client.16608 handle_command_reply: tid=0 2024-07-26T13:44:20.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.616+0000 7f94c8975740 1 client.16608 shutdown 2024-07-26T13:44:20.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.616+0000 7f94c8975740 1 -- 172.21.15.100:0/88771017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cec916dea0 msgr2=0x55cec9564d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:20.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.616+0000 7f94c8975740 1 --2- 172.21.15.100:0/88771017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cec916dea0 0x55cec9564d20 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f94ac0e1c00 tx=0x7f94ac003cf0 comp rx=0 tx=0).stop 2024-07-26T13:44:20.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.616+0000 7f94c8975740 1 -- 172.21.15.100:0/88771017 shutdown_connections 2024-07-26T13:44:20.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.616+0000 7f94c8975740 1 --2- 172.21.15.100:0/88771017 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55cec91af520 0x55cec95de240 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:20.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.616+0000 7f94c8975740 1 --2- 172.21.15.100:0/88771017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cec918e840 0x55cec95da2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:20.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.616+0000 7f949b7fe640 0 client.16608 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:20.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.617+0000 7f94c8975740 1 --2- 172.21.15.100:0/88771017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cec9186660 0x55cec9565260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:20.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.617+0000 7f94c8975740 1 --2- 172.21.15.100:0/88771017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cec916dea0 0x55cec9564d20 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:20.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.617+0000 7f94c8975740 1 -- 172.21.15.100:0/88771017 >> 172.21.15.100:0/88771017 conn(0x55cec91a3910 msgr2=0x55cec955f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:20.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.617+0000 7f94c8975740 1 -- 172.21.15.100:0/88771017 shutdown_connections 2024-07-26T13:44:20.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.617+0000 7f94c8975740 1 -- 172.21.15.100:0/88771017 wait complete. 2024-07-26T13:44:20.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.617+0000 7f94c8975740 20 client.16608 trim_cache size 0 max 16384 2024-07-26T13:44:20.619 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:44:20.619 INFO:teuthology.orchestra.run.smithi100.stdout: "epoch": 37, 2024-07-26T13:44:20.619 INFO:teuthology.orchestra.run.smithi100.stdout: "leader": 24607, 2024-07-26T13:44:20.619 INFO:teuthology.orchestra.run.smithi100.stdout: "set_version": 12, 2024-07-26T13:44:20.619 INFO:teuthology.orchestra.run.smithi100.stdout: "sets": { 2024-07-26T13:44:20.619 INFO:teuthology.orchestra.run.smithi100.stdout: "47f95d77": { 2024-07-26T13:44:20.619 INFO:teuthology.orchestra.run.smithi100.stdout: "version": 12, 2024-07-26T13:44:20.620 INFO:teuthology.orchestra.run.smithi100.stdout: "age_ref": 0.0, 2024-07-26T13:44:20.620 INFO:teuthology.orchestra.run.smithi100.stdout: "state": { 2024-07-26T13:44:20.620 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RELEASED", 2024-07-26T13:44:20.620 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 0.0 2024-07-26T13:44:20.620 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-07-26T13:44:20.620 INFO:teuthology.orchestra.run.smithi100.stdout: "timeout": 90.0, 2024-07-26T13:44:20.620 INFO:teuthology.orchestra.run.smithi100.stdout: "expiration": 151.1, 2024-07-26T13:44:20.620 INFO:teuthology.orchestra.run.smithi100.stdout: "members": { 2024-07-26T13:44:20.620 INFO:teuthology.orchestra.run.smithi100.stdout: "file:/": { 2024-07-26T13:44:20.620 INFO:teuthology.orchestra.run.smithi100.stdout: "excluded": false, 2024-07-26T13:44:20.620 INFO:teuthology.orchestra.run.smithi100.stdout: "state": { 2024-07-26T13:44:20.620 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RELEASED", 2024-07-26T13:44:20.620 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 0.0 2024-07-26T13:44:20.620 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:44:20.621 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:44:20.621 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:44:20.621 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:44:20.621 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:44:20.621 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:44:20.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.617+0000 7f94c23f1640 1 -- 172.21.15.100:0/1737725639 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f94a40b3d40 msgr2=0x7f94a40b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:20.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.617+0000 7f94c23f1640 1 --2- 172.21.15.100:0/1737725639 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f94a40b3d40 0x7f94a40b6250 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f94b4006010 tx=0x7f94b4005d80 comp rx=0 tx=0).stop 2024-07-26T13:44:20.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.617+0000 7f94c23f1640 1 -- 172.21.15.100:0/1737725639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94bc104270 msgr2=0x7f94bc1989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:20.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.617+0000 7f94c23f1640 1 --2- 172.21.15.100:0/1737725639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94bc104270 0x7f94bc1989c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f94ac00e9c0 tx=0x7f94ac00ee80 comp rx=0 tx=0).stop 2024-07-26T13:44:20.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.617+0000 7f94c23f1640 1 -- 172.21.15.100:0/1737725639 shutdown_connections 2024-07-26T13:44:20.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.617+0000 7f94c23f1640 1 --2- 172.21.15.100:0/1737725639 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f94a40b3d40 0x7f94a40b6250 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:20.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.617+0000 7f94c23f1640 1 --2- 172.21.15.100:0/1737725639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94bc10d820 0x7f94bc19fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:20.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.617+0000 7f94c23f1640 1 --2- 172.21.15.100:0/1737725639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94bc104ca0 0x7f94bc198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:20.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.617+0000 7f94c23f1640 1 --2- 172.21.15.100:0/1737725639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94bc104270 0x7f94bc1989c0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:20.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.617+0000 7f94c23f1640 1 -- 172.21.15.100:0/1737725639 >> 172.21.15.100:0/1737725639 conn(0x7f94bc079160 msgr2=0x7f94bc1030f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:20.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.618+0000 7f94c23f1640 1 -- 172.21.15.100:0/1737725639 shutdown_connections 2024-07-26T13:44:20.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:20.618+0000 7f94c23f1640 1 -- 172.21.15.100:0/1737725639 wait complete. 2024-07-26T13:44:21.412 INFO:tasks.workunit.client.0.smithi100.stdout: 1 147891 0.20 MB/sec execute 35 sec latency 34519.118 ms 2024-07-26T13:44:21.483 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '47f95d77', total seconds elapsed: 40.8 2024-07-26T13:44:21.486 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:44:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:21 smithi100 ceph-mon[32599]: pgmap v492: 97 pgs: 97 active+clean; 63 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 1.7 MiB/s wr, 77 op/s 2024-07-26T13:44:21.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:21 smithi138 ceph-mon[36954]: pgmap v492: 97 pgs: 97 active+clean; 63 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 1.7 MiB/s wr, 77 op/s 2024-07-26T13:44:22.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:21 smithi167 ceph-mon[37140]: pgmap v492: 97 pgs: 97 active+clean; 63 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 1.7 MiB/s wr, 77 op/s 2024-07-26T13:44:22.412 INFO:tasks.workunit.client.0.smithi100.stdout: 1 149459 0.53 MB/sec execute 36 sec latency 10.317 ms 2024-07-26T13:44:23.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.314+0000 7f7406fb9640 1 -- 172.21.15.100:0/4256076954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74001035b0 msgr2=0x7f74001039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:23.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.314+0000 7f7406fb9640 1 --2- 172.21.15.100:0/4256076954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74001035b0 0x7f74001039c0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f73f8009ca0 tx=0x7f73f802a8c0 comp rx=0 tx=0).stop 2024-07-26T13:44:23.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.314+0000 7f7406fb9640 1 -- 172.21.15.100:0/4256076954 shutdown_connections 2024-07-26T13:44:23.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.314+0000 7f7406fb9640 1 --2- 172.21.15.100:0/4256076954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7400109840 0x7f7400110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:23.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.314+0000 7f7406fb9640 1 --2- 172.21.15.100:0/4256076954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7400108ef0 0x7f7400109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:23.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.314+0000 7f7406fb9640 1 --2- 172.21.15.100:0/4256076954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f74001035b0 0x7f74001039c0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:23.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.314+0000 7f7406fb9640 1 -- 172.21.15.100:0/4256076954 >> 172.21.15.100:0/4256076954 conn(0x7f74000fefe0 msgr2=0x7f7400101430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:23.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.314+0000 7f7406fb9640 1 -- 172.21.15.100:0/4256076954 shutdown_connections 2024-07-26T13:44:23.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.314+0000 7f7406fb9640 1 -- 172.21.15.100:0/4256076954 wait complete. 2024-07-26T13:44:23.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.315+0000 7f7406fb9640 1 Processor -- start 2024-07-26T13:44:23.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.315+0000 7f7406fb9640 1 -- start start 2024-07-26T13:44:23.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.315+0000 7f7406fb9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74001035b0 0x7f7400199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:23.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.315+0000 7f7406fb9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7400108ef0 0x7f740019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:23.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.315+0000 7f7406fb9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7400109840 0x7f740019f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:23.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.315+0000 7f7406fb9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7400113510 con 0x7f7400109840 2024-07-26T13:44:23.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.315+0000 7f7406fb9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7400113390 con 0x7f74001035b0 2024-07-26T13:44:23.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.315+0000 7f7406fb9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7400113690 con 0x7f7400108ef0 2024-07-26T13:44:23.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.315+0000 7f7405556640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7400109840 0x7f740019f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:23.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.315+0000 7f7404d55640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74001035b0 0x7f7400199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:23.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.315+0000 7f73fffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7400108ef0 0x7f740019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:23.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.315+0000 7f7404d55640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74001035b0 0x7f7400199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:36270/0 (socket says 172.21.15.100:36270) 2024-07-26T13:44:23.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.315+0000 7f73fffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7400108ef0 0x7f740019a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:43562/0 (socket says 172.21.15.100:43562) 2024-07-26T13:44:23.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.315+0000 7f7405556640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7400109840 0x7f740019f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44554/0 (socket says 172.21.15.100:44554) 2024-07-26T13:44:23.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.315+0000 7f7404d55640 1 -- 172.21.15.100:0/3870553811 learned_addr learned my addr 172.21.15.100:0/3870553811 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:23.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.315+0000 7f73fffff640 1 -- 172.21.15.100:0/3870553811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74001035b0 msgr2=0x7f7400199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:23.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.315+0000 7f73fffff640 1 --2- 172.21.15.100:0/3870553811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74001035b0 0x7f7400199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:23.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.315+0000 7f73fffff640 1 -- 172.21.15.100:0/3870553811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7400109840 msgr2=0x7f740019f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:23.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.315+0000 7f73fffff640 1 --2- 172.21.15.100:0/3870553811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7400109840 0x7f740019f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:23.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.315+0000 7f73fffff640 1 -- 172.21.15.100:0/3870553811 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f740019ff30 con 0x7f7400108ef0 2024-07-26T13:44:23.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.315+0000 7f7405556640 1 --2- 172.21.15.100:0/3870553811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7400109840 0x7f740019f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:44:23.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.316+0000 7f73fffff640 1 --2- 172.21.15.100:0/3870553811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7400108ef0 0x7f740019a560 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f740006c3f0 tx=0x7f73f400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:23.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.316+0000 7f73fdffb640 1 -- 172.21.15.100:0/3870553811 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f400cc40 con 0x7f7400108ef0 2024-07-26T13:44:23.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.316+0000 7f7406fb9640 1 -- 172.21.15.100:0/3870553811 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74001a0220 con 0x7f7400108ef0 2024-07-26T13:44:23.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.316+0000 7f73fdffb640 1 -- 172.21.15.100:0/3870553811 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73f400cde0 con 0x7f7400108ef0 2024-07-26T13:44:23.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.316+0000 7f73fdffb640 1 -- 172.21.15.100:0/3870553811 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f4018760 con 0x7f7400108ef0 2024-07-26T13:44:23.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.316+0000 7f7406fb9640 1 -- 172.21.15.100:0/3870553811 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74001a6980 con 0x7f7400108ef0 2024-07-26T13:44:23.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.319+0000 7f73fdffb640 1 -- 172.21.15.100:0/3870553811 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f73f4010a90 con 0x7f7400108ef0 2024-07-26T13:44:23.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.319+0000 7f740d53d740 1 Processor -- start 2024-07-26T13:44:23.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.319+0000 7f740d53d740 1 -- start start 2024-07-26T13:44:23.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.319+0000 7f740d53d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f09ace350 0x558f09e733b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:23.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.319+0000 7f740d53d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f09aa5490 0x558f09e738f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:23.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.319+0000 7f740d53d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f09aad670 0x558f09e73e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:23.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.319+0000 7f740d53d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x558f09d5b2f0 con 0x558f09aa5490 2024-07-26T13:44:23.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.319+0000 7f740d53d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558f09de9a80 con 0x558f09ace350 2024-07-26T13:44:23.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.319+0000 7f740d53d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558f09dd8f80 con 0x558f09aad670 2024-07-26T13:44:23.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.319+0000 7f7404d55640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f09aa5490 0x558f09e738f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:23.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.319+0000 7f7405556640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f09ace350 0x558f09e733b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:23.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.319+0000 7f7404d55640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f09aa5490 0x558f09e738f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44556/0 (socket says 172.21.15.100:44556) 2024-07-26T13:44:23.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.319+0000 7f7404d55640 1 -- 172.21.15.100:0/3220994968 learned_addr learned my addr 172.21.15.100:0/3220994968 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:23.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.319+0000 7f73fffff640 1 --2- 172.21.15.100:0/3220994968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f09aad670 0x558f09e73e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:23.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.320+0000 7f73fdffb640 1 --2- 172.21.15.100:0/3870553811 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f73e80b3d40 0x7f73e80b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:23.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.320+0000 7f7405556640 1 -- 172.21.15.100:0/3220994968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f09aad670 msgr2=0x558f09e73e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:23.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.320+0000 7f7405556640 1 --2- 172.21.15.100:0/3220994968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f09aad670 0x558f09e73e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:23.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.320+0000 7f7405556640 1 -- 172.21.15.100:0/3220994968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f09aa5490 msgr2=0x558f09e738f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:23.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.320+0000 7f7405556640 1 --2- 172.21.15.100:0/3220994968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f09aa5490 0x558f09e738f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:23.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.320+0000 7f7405556640 1 -- 172.21.15.100:0/3220994968 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f09b39930 con 0x558f09ace350 2024-07-26T13:44:23.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.320+0000 7f7404d55640 1 --2- 172.21.15.100:0/3220994968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f09aa5490 0x558f09e738f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:44:23.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.320+0000 7f73fdffb640 1 -- 172.21.15.100:0/3870553811 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f73f40e42e0 con 0x7f7400108ef0 2024-07-26T13:44:23.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.320+0000 7f7405556640 1 --2- 172.21.15.100:0/3870553811 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f73e80b3d40 0x7f73e80b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:23.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.320+0000 7f7405556640 1 --2- 172.21.15.100:0/3220994968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f09ace350 0x558f09e733b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f73f0009f30 tx=0x7f73f00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d76603daf7799bbd server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:23.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.320+0000 7f73e6ffd640 1 -- 172.21.15.100:0/3220994968 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f00037f0 con 0x558f09ace350 2024-07-26T13:44:23.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.320+0000 7f73e6ffd640 1 -- 172.21.15.100:0/3220994968 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73f0003f60 con 0x558f09ace350 2024-07-26T13:44:23.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.320+0000 7f73e6ffd640 1 -- 172.21.15.100:0/3220994968 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f00199e0 con 0x558f09ace350 2024-07-26T13:44:23.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.322+0000 7f740d53d740 1 -- 172.21.15.100:0/3220994968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f09ace350 msgr2=0x558f09e733b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:23.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.322+0000 7f7405556640 1 --2- 172.21.15.100:0/3870553811 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f73e80b3d40 0x7f73e80b6250 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f73f0020000 tx=0x7f73f0021000 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:23.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.322+0000 7f740d53d740 1 --2- 172.21.15.100:0/3220994968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f09ace350 0x558f09e733b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f73f0009f30 tx=0x7f73f00118d0 comp rx=0 tx=0).stop 2024-07-26T13:44:23.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.322+0000 7f740d53d740 1 -- 172.21.15.100:0/3220994968 shutdown_connections 2024-07-26T13:44:23.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.322+0000 7f740d53d740 1 --2- 172.21.15.100:0/3220994968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f09aad670 0x558f09e73e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:23.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.322+0000 7f740d53d740 1 --2- 172.21.15.100:0/3220994968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f09aa5490 0x558f09e738f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:23.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.322+0000 7f740d53d740 1 --2- 172.21.15.100:0/3220994968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f09ace350 0x558f09e733b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:23.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.322+0000 7f740d53d740 1 -- 172.21.15.100:0/3220994968 >> 172.21.15.100:0/3220994968 conn(0x558f09ac2740 msgr2=0x558f09e712a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:23.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.322+0000 7f740d53d740 1 -- 172.21.15.100:0/3220994968 shutdown_connections 2024-07-26T13:44:23.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.322+0000 7f740d53d740 1 -- 172.21.15.100:0/3220994968 wait complete. 2024-07-26T13:44:23.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.322+0000 7f740d53d740 1 -- start start 2024-07-26T13:44:23.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.322+0000 7f740d53d740 1 Processor -- start 2024-07-26T13:44:23.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.323+0000 7f740d53d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f09aa5490 0x558f09e7ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:23.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.323+0000 7f740d53d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f09aad670 0x558f09e7f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:23.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.323+0000 7f740d53d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f09ace350 0x558f09e8ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:23.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.323+0000 7f740d53d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x558f09d7b4f0 con 0x558f09aad670 2024-07-26T13:44:23.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.323+0000 7f740d53d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558f09ddcfc0 con 0x558f09ace350 2024-07-26T13:44:23.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.323+0000 7f740d53d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558f09c3c890 con 0x558f09aa5490 2024-07-26T13:44:23.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.323+0000 7f740d53d740 10 client.0 _ll_register_callbacks cb 0x558f09d85500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:44:23.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.323+0000 7f73fffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f09aad670 0x558f09e7f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:23.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.323+0000 7f73fffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f09aad670 0x558f09e7f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44570/0 (socket says 172.21.15.100:44570) 2024-07-26T13:44:23.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.323+0000 7f73fffff640 1 -- 172.21.15.100:0/2780955867 learned_addr learned my addr 172.21.15.100:0/2780955867 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:23.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.323+0000 7f7404d55640 1 --2- 172.21.15.100:0/2780955867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f09aa5490 0x558f09e7ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:23.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.323+0000 7f7405556640 1 --2- 172.21.15.100:0/2780955867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f09ace350 0x558f09e8ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:23.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.323+0000 7f73fffff640 1 -- 172.21.15.100:0/2780955867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f09aa5490 msgr2=0x558f09e7ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:23.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.323+0000 7f73fffff640 1 --2- 172.21.15.100:0/2780955867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f09aa5490 0x558f09e7ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:23.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.323+0000 7f73fffff640 1 -- 172.21.15.100:0/2780955867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f09ace350 msgr2=0x558f09e8ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:23.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.323+0000 7f73fffff640 1 --2- 172.21.15.100:0/2780955867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f09ace350 0x558f09e8ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:23.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.323+0000 7f73fffff640 1 -- 172.21.15.100:0/2780955867 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f09d8ac50 con 0x558f09aad670 2024-07-26T13:44:23.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.323+0000 7f73fffff640 1 --2- 172.21.15.100:0/2780955867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f09aad670 0x558f09e7f480 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f73f400c6a0 tx=0x7f73f4009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a207fa8cec3525 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:23.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.323+0000 7f73dffff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:44:23.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.324+0000 7f73dffff640 1 -- 172.21.15.100:0/2780955867 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f40aad40 con 0x558f09aad670 2024-07-26T13:44:23.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.324+0000 7f73dffff640 1 -- 172.21.15.100:0/2780955867 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f73f40a6af0 con 0x558f09aad670 2024-07-26T13:44:23.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.324+0000 7f740d53d740 1 -- 172.21.15.100:0/2780955867 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73f0003f60 con 0x558f09aad670 2024-07-26T13:44:23.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.324+0000 7f73dffff640 1 -- 172.21.15.100:0/2780955867 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f73f40aa6e0 con 0x558f09aad670 2024-07-26T13:44:23.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.324+0000 7f740d53d740 10 client.16620 fetch_fsmap learned FSMap version 37 2024-07-26T13:44:23.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.324+0000 7f740d53d740 1 -- 172.21.15.100:0/2780955867 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558f09e8da30 con 0x558f09aad670 2024-07-26T13:44:23.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.324+0000 7f73dffff640 1 -- 172.21.15.100:0/2780955867 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73f0003f60 con 0x558f09aad670 2024-07-26T13:44:23.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.324+0000 7f73dffff640 1 -- 172.21.15.100:0/2780955867 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f73f40b0650 con 0x558f09aad670 2024-07-26T13:44:23.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.324+0000 7f740d53d740 10 client.16620 fetch_fsmap finished waiting for FSMap version 37 2024-07-26T13:44:23.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.324+0000 7f740d53d740 10 client.16620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:44:23.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.324+0000 7f740d53d740 20 client.16620 populate_metadata read hostname 'smithi100' 2024-07-26T13:44:23.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.324+0000 7f740d53d740 1 --2- 172.21.15.100:0/2780955867 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x558f09a8ccd0 0x558f09e90ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:23.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.324+0000 7f740d53d740 4 client.16620 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:44:23.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.324+0000 7f740d53d740 1 -- 172.21.15.100:0/2780955867 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f09deb9d0 con 0x558f09a8ccd0 2024-07-26T13:44:23.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.325+0000 7f7404d55640 1 --2- 172.21.15.100:0/2780955867 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x558f09a8ccd0 0x558f09e90ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:23.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.325+0000 7f73dffff640 10 client.16620 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:23.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.325+0000 7f7404d55640 1 --2- 172.21.15.100:0/2780955867 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x558f09a8ccd0 0x558f09e90ef0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a00e357d89a188a server_cookie=c70e1c95c2b82062 in_seq=0 out_seq=0 2024-07-26T13:44:23.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.326+0000 7f73dffff640 1 -- 172.21.15.100:0/2780955867 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558f09deb9d0 con 0x558f09a8ccd0 2024-07-26T13:44:23.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.326+0000 7f73dffff640 10 client.16620 handle_command_reply: tid=0 2024-07-26T13:44:23.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.327+0000 7f740d53d740 1 client.16620 shutdown 2024-07-26T13:44:23.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.327+0000 7f740d53d740 1 -- 172.21.15.100:0/2780955867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f09aad670 msgr2=0x558f09e7f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:23.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.327+0000 7f740d53d740 1 --2- 172.21.15.100:0/2780955867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f09aad670 0x558f09e7f480 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f73f400c6a0 tx=0x7f73f4009700 comp rx=0 tx=0).stop 2024-07-26T13:44:23.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.327+0000 7f740d53d740 1 -- 172.21.15.100:0/2780955867 shutdown_connections 2024-07-26T13:44:23.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.327+0000 7f740d53d740 1 --2- 172.21.15.100:0/2780955867 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x558f09a8ccd0 0x558f09e90ef0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:23.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.327+0000 7f740d53d740 1 --2- 172.21.15.100:0/2780955867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f09ace350 0x558f09e8ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:23.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.327+0000 7f740d53d740 1 --2- 172.21.15.100:0/2780955867 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f09aad670 0x558f09e7f480 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:23.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.327+0000 7f73dffff640 0 client.16620 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:23.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.327+0000 7f740d53d740 1 --2- 172.21.15.100:0/2780955867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f09aa5490 0x558f09e7ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:23.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.327+0000 7f740d53d740 1 -- 172.21.15.100:0/2780955867 >> 172.21.15.100:0/2780955867 conn(0x558f09ac2740 msgr2=0x558f09e76540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:23.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.327+0000 7f740d53d740 1 -- 172.21.15.100:0/2780955867 shutdown_connections 2024-07-26T13:44:23.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.327+0000 7f740d53d740 1 -- 172.21.15.100:0/2780955867 wait complete. 2024-07-26T13:44:23.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.327+0000 7f740d53d740 20 client.16620 trim_cache size 0 max 16384 2024-07-26T13:44:23.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.346+0000 7f740d53d740 1 Processor -- start 2024-07-26T13:44:23.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.346+0000 7f740d53d740 1 -- start start 2024-07-26T13:44:23.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.346+0000 7f740d53d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f09a8ccd0 0x558f09e794d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:23.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.346+0000 7f740d53d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f09aa5490 0x558f09e6ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:23.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.346+0000 7f740d53d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f09aad670 0x558f09e8ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:23.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.346+0000 7f740d53d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f74001b1bf0 con 0x558f09aad670 2024-07-26T13:44:23.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.346+0000 7f740d53d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f74001af460 con 0x558f09a8ccd0 2024-07-26T13:44:23.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.346+0000 7f740d53d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f74001ad2b0 con 0x558f09aa5490 2024-07-26T13:44:23.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.347+0000 7f7405556640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f09aad670 0x558f09e8ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:23.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.347+0000 7f73fffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f09aa5490 0x558f09e6ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:23.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.347+0000 7f7405556640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f09aad670 0x558f09e8ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44584/0 (socket says 172.21.15.100:44584) 2024-07-26T13:44:23.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.347+0000 7f73fffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f09aa5490 0x558f09e6ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:43578/0 (socket says 172.21.15.100:43578) 2024-07-26T13:44:23.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.347+0000 7f7405556640 1 -- 172.21.15.100:0/3924395597 learned_addr learned my addr 172.21.15.100:0/3924395597 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:23.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.347+0000 7f7404d55640 1 --2- 172.21.15.100:0/3924395597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f09a8ccd0 0x558f09e794d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:23.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.347+0000 7f73fffff640 1 -- 172.21.15.100:0/3924395597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f09a8ccd0 msgr2=0x558f09e794d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:23.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.347+0000 7f73fffff640 1 --2- 172.21.15.100:0/3924395597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f09a8ccd0 0x558f09e794d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:23.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.347+0000 7f73fffff640 1 -- 172.21.15.100:0/3924395597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f09aad670 msgr2=0x558f09e8ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:23.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.347+0000 7f73fffff640 1 --2- 172.21.15.100:0/3924395597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f09aad670 0x558f09e8ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:23.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.347+0000 7f73fffff640 1 -- 172.21.15.100:0/3924395597 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f09e756a0 con 0x558f09aa5490 2024-07-26T13:44:23.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.347+0000 7f7405556640 1 --2- 172.21.15.100:0/3924395597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f09aad670 0x558f09e8ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:44:23.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.347+0000 7f73fffff640 1 --2- 172.21.15.100:0/3924395597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f09aa5490 0x558f09e6ffd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f73f40e2000 tx=0x7f73f4010e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f264ba351c6abd17 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:23.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.347+0000 7f73e67fc640 1 -- 172.21.15.100:0/3924395597 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f40aa8c0 con 0x558f09aa5490 2024-07-26T13:44:23.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.347+0000 7f73e67fc640 1 -- 172.21.15.100:0/3924395597 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73f40aaa80 con 0x558f09aa5490 2024-07-26T13:44:23.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.347+0000 7f73e67fc640 1 -- 172.21.15.100:0/3924395597 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f40b0650 con 0x558f09aa5490 2024-07-26T13:44:23.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.349+0000 7f740d53d740 1 -- 172.21.15.100:0/3924395597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f09aa5490 msgr2=0x558f09e6ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:23.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.349+0000 7f740d53d740 1 --2- 172.21.15.100:0/3924395597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f09aa5490 0x558f09e6ffd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f73f40e2000 tx=0x7f73f4010e30 comp rx=0 tx=0).stop 2024-07-26T13:44:23.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.349+0000 7f740d53d740 1 -- 172.21.15.100:0/3924395597 shutdown_connections 2024-07-26T13:44:23.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.349+0000 7f740d53d740 1 --2- 172.21.15.100:0/3924395597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f09aad670 0x558f09e8ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:23.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.349+0000 7f740d53d740 1 --2- 172.21.15.100:0/3924395597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f09aa5490 0x558f09e6ffd0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:23.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.349+0000 7f740d53d740 1 --2- 172.21.15.100:0/3924395597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f09a8ccd0 0x558f09e794d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:23.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.349+0000 7f740d53d740 1 -- 172.21.15.100:0/3924395597 >> 172.21.15.100:0/3924395597 conn(0x558f09ac2740 msgr2=0x558f09e6f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:23.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.349+0000 7f740d53d740 1 -- 172.21.15.100:0/3924395597 shutdown_connections 2024-07-26T13:44:23.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.349+0000 7f740d53d740 1 -- 172.21.15.100:0/3924395597 wait complete. 2024-07-26T13:44:23.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.349+0000 7f740d53d740 1 -- start start 2024-07-26T13:44:23.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.349+0000 7f740d53d740 1 Processor -- start 2024-07-26T13:44:23.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.350+0000 7f740d53d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f09a8ccd0 0x558f09e870f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:23.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.350+0000 7f740d53d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f09aa5490 0x558f09e825f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:23.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.350+0000 7f740d53d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f09aad670 0x558f09e82b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:23.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.350+0000 7f740d53d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f74001ad920 con 0x558f09a8ccd0 2024-07-26T13:44:23.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.350+0000 7f740d53d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f74001aed90 con 0x558f09aad670 2024-07-26T13:44:23.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.350+0000 7f740d53d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f74001adff0 con 0x558f09aa5490 2024-07-26T13:44:23.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.350+0000 7f7404d55640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f09a8ccd0 0x558f09e870f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:23.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.350+0000 7f740d53d740 10 client.0 _ll_register_callbacks cb 0x7f73f40a7430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:44:23.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.350+0000 7f7404d55640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f09a8ccd0 0x558f09e870f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44600/0 (socket says 172.21.15.100:44600) 2024-07-26T13:44:23.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.350+0000 7f7404d55640 1 -- 172.21.15.100:0/2630965565 learned_addr learned my addr 172.21.15.100:0/2630965565 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:23.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.350+0000 7f73fffff640 1 --2- 172.21.15.100:0/2630965565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f09aa5490 0x558f09e825f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:23.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.350+0000 7f7405556640 1 --2- 172.21.15.100:0/2630965565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f09aad670 0x558f09e82b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:23.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.350+0000 7f7404d55640 1 -- 172.21.15.100:0/2630965565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f09aa5490 msgr2=0x558f09e825f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:23.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.350+0000 7f7404d55640 1 --2- 172.21.15.100:0/2630965565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f09aa5490 0x558f09e825f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:23.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.350+0000 7f7404d55640 1 -- 172.21.15.100:0/2630965565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f09aad670 msgr2=0x558f09e82b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:23.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.350+0000 7f7404d55640 1 --2- 172.21.15.100:0/2630965565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f09aad670 0x558f09e82b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:23.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.350+0000 7f7404d55640 1 -- 172.21.15.100:0/2630965565 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f09e83070 con 0x558f09a8ccd0 2024-07-26T13:44:23.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.350+0000 7f7405556640 1 --2- 172.21.15.100:0/2630965565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f09aad670 0x558f09e82b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:44:23.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.350+0000 7f7404d55640 1 --2- 172.21.15.100:0/2630965565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f09a8ccd0 0x558f09e870f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f73f8004af0 tx=0x7f73f8031640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=366359f58fc15c8d server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:23.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.350+0000 7f73dffff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:44:23.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.351+0000 7f73dffff640 1 -- 172.21.15.100:0/2630965565 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f8056060 con 0x558f09a8ccd0 2024-07-26T13:44:23.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.351+0000 7f73dffff640 1 -- 172.21.15.100:0/2630965565 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f73f8003b70 con 0x558f09a8ccd0 2024-07-26T13:44:23.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.351+0000 7f740d53d740 1 -- 172.21.15.100:0/2630965565 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73f40aaa80 con 0x558f09a8ccd0 2024-07-26T13:44:23.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.351+0000 7f73dffff640 1 -- 172.21.15.100:0/2630965565 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f73f8003d70 con 0x558f09a8ccd0 2024-07-26T13:44:23.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.351+0000 7f740d53d740 10 client.16626 fetch_fsmap learned FSMap version 37 2024-07-26T13:44:23.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.351+0000 7f740d53d740 1 -- 172.21.15.100:0/2630965565 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558f09ef9440 con 0x558f09a8ccd0 2024-07-26T13:44:23.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.351+0000 7f73dffff640 1 -- 172.21.15.100:0/2630965565 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73f40aaa80 con 0x558f09a8ccd0 2024-07-26T13:44:23.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.351+0000 7f73dffff640 1 -- 172.21.15.100:0/2630965565 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f73f802fa00 con 0x558f09a8ccd0 2024-07-26T13:44:23.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.351+0000 7f740d53d740 10 client.16626 fetch_fsmap finished waiting for FSMap version 37 2024-07-26T13:44:23.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.351+0000 7f740d53d740 10 client.16626 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:44:23.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.351+0000 7f740d53d740 20 client.16626 populate_metadata read hostname 'smithi100' 2024-07-26T13:44:23.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.351+0000 7f740d53d740 1 --2- 172.21.15.100:0/2630965565 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x558f09ace350 0x558f09efbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:23.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.351+0000 7f740d53d740 4 client.16626 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-26T13:44:23.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.351+0000 7f740d53d740 1 -- 172.21.15.100:0/2630965565 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558f09de7470 con 0x558f09ace350 2024-07-26T13:44:23.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.352+0000 7f73fffff640 1 --2- 172.21.15.100:0/2630965565 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x558f09ace350 0x558f09efbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:23.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.352+0000 7f73dffff640 10 client.16626 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:23.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.352+0000 7f73fffff640 1 --2- 172.21.15.100:0/2630965565 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x558f09ace350 0x558f09efbc40 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec5ebcc5cdff662e server_cookie=2f07c5639241accd in_seq=0 out_seq=0 2024-07-26T13:44:23.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.353+0000 7f73dffff640 1 -- 172.21.15.100:0/2630965565 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558f09de7470 con 0x558f09ace350 2024-07-26T13:44:23.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.353+0000 7f73dffff640 10 client.16626 handle_command_reply: tid=0 2024-07-26T13:44:23.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.353+0000 7f740d53d740 1 client.16626 shutdown 2024-07-26T13:44:23.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.353+0000 7f740d53d740 1 -- 172.21.15.100:0/2630965565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f09a8ccd0 msgr2=0x558f09e870f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:23.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.353+0000 7f740d53d740 1 --2- 172.21.15.100:0/2630965565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f09a8ccd0 0x558f09e870f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f73f8004af0 tx=0x7f73f8031640 comp rx=0 tx=0).stop 2024-07-26T13:44:23.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.353+0000 7f740d53d740 1 -- 172.21.15.100:0/2630965565 shutdown_connections 2024-07-26T13:44:23.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.353+0000 7f740d53d740 1 --2- 172.21.15.100:0/2630965565 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x558f09ace350 0x558f09efbc40 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:23.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.353+0000 7f740d53d740 1 --2- 172.21.15.100:0/2630965565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f09aad670 0x558f09e82b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:23.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.353+0000 7f740d53d740 1 --2- 172.21.15.100:0/2630965565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558f09aa5490 0x558f09e825f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:23.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.353+0000 7f73dffff640 0 client.16626 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:23.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.353+0000 7f740d53d740 1 --2- 172.21.15.100:0/2630965565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558f09a8ccd0 0x558f09e870f0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:23.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.353+0000 7f740d53d740 1 -- 172.21.15.100:0/2630965565 >> 172.21.15.100:0/2630965565 conn(0x558f09ac2740 msgr2=0x558f09e801f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:23.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.354+0000 7f740d53d740 1 -- 172.21.15.100:0/2630965565 shutdown_connections 2024-07-26T13:44:23.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.354+0000 7f740d53d740 1 -- 172.21.15.100:0/2630965565 wait complete. 2024-07-26T13:44:23.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.354+0000 7f740d53d740 20 client.16626 trim_cache size 0 max 16384 2024-07-26T13:44:23.360 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:44:23.360 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-07-26T13:44:23.360 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "c66b943d-9809-430e-96fa-842951f15197", 2024-07-26T13:44:23.360 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-07-26T13:44:23.360 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:44:23.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.354+0000 7f7406fb9640 1 -- 172.21.15.100:0/3870553811 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f73e80b3d40 msgr2=0x7f73e80b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:23.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.354+0000 7f7406fb9640 1 --2- 172.21.15.100:0/3870553811 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f73e80b3d40 0x7f73e80b6250 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f73f0020000 tx=0x7f73f0021000 comp rx=0 tx=0).stop 2024-07-26T13:44:23.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.354+0000 7f7406fb9640 1 -- 172.21.15.100:0/3870553811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7400108ef0 msgr2=0x7f740019a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:23.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.354+0000 7f7406fb9640 1 --2- 172.21.15.100:0/3870553811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7400108ef0 0x7f740019a560 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f740006c3f0 tx=0x7f73f400ee50 comp rx=0 tx=0).stop 2024-07-26T13:44:23.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.354+0000 7f7406fb9640 1 -- 172.21.15.100:0/3870553811 shutdown_connections 2024-07-26T13:44:23.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.354+0000 7f7406fb9640 1 --2- 172.21.15.100:0/3870553811 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f73e80b3d40 0x7f73e80b6250 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:23.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.354+0000 7f7406fb9640 1 --2- 172.21.15.100:0/3870553811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7400109840 0x7f740019f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:23.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.354+0000 7f7406fb9640 1 --2- 172.21.15.100:0/3870553811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7400108ef0 0x7f740019a560 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:23.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.354+0000 7f7406fb9640 1 --2- 172.21.15.100:0/3870553811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74001035b0 0x7f7400199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:23.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.354+0000 7f7406fb9640 1 -- 172.21.15.100:0/3870553811 >> 172.21.15.100:0/3870553811 conn(0x7f74000fefe0 msgr2=0x7f74001085b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:23.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.354+0000 7f7406fb9640 1 -- 172.21.15.100:0/3870553811 shutdown_connections 2024-07-26T13:44:23.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:23.355+0000 7f7406fb9640 1 -- 172.21.15.100:0/3870553811 wait complete. 2024-07-26T13:44:23.412 INFO:tasks.workunit.client.0.smithi100.stdout: 1 151571 0.73 MB/sec execute 37 sec latency 62.969 ms 2024-07-26T13:44:23.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:23 smithi138 ceph-mon[36954]: pgmap v493: 97 pgs: 97 active+clean; 62 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 1.5 MiB/s wr, 70 op/s 2024-07-26T13:44:23.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:23 smithi100 ceph-mon[32599]: pgmap v493: 97 pgs: 97 active+clean; 62 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 1.5 MiB/s wr, 70 op/s 2024-07-26T13:44:24.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:23 smithi167 ceph-mon[37140]: pgmap v493: 97 pgs: 97 active+clean; 62 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 1.5 MiB/s wr, 70 op/s 2024-07-26T13:44:24.397 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-26T13:44:24.415 INFO:tasks.workunit.client.0.smithi100.stdout: 1 153217 1.01 MB/sec execute 38 sec latency 32.162 ms 2024-07-26T13:44:24.696 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:44:24.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:24 smithi100 ceph-mon[32599]: scrub queued for path: / 2024-07-26T13:44:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:24 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:44:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:24 smithi100 ceph-mon[32599]: scrub summary: active paths [/] 2024-07-26T13:44:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:24 smithi100 ceph-mon[32599]: scrub summary: active paths [/] 2024-07-26T13:44:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:24 smithi100 ceph-mon[32599]: scrub summary: active paths [/] 2024-07-26T13:44:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:24 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:44:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:24 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:44:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:24 smithi100 ceph-mon[32599]: scrub summary: active paths [/] 2024-07-26T13:44:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:24 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:44:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:24 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:44:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:24 smithi100 ceph-mon[32599]: pgmap v494: 97 pgs: 97 active+clean; 60 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 111 op/s 2024-07-26T13:44:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:24 smithi138 ceph-mon[36954]: scrub queued for path: / 2024-07-26T13:44:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:24 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:44:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:24 smithi138 ceph-mon[36954]: scrub summary: active paths [/] 2024-07-26T13:44:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:24 smithi138 ceph-mon[36954]: scrub summary: active paths [/] 2024-07-26T13:44:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:24 smithi138 ceph-mon[36954]: scrub summary: active paths [/] 2024-07-26T13:44:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:24 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:44:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:24 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:44:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:24 smithi138 ceph-mon[36954]: scrub summary: active paths [/] 2024-07-26T13:44:24.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:24 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:44:24.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:24 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:44:24.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:24 smithi138 ceph-mon[36954]: pgmap v494: 97 pgs: 97 active+clean; 60 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 111 op/s 2024-07-26T13:44:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:24 smithi167 ceph-mon[37140]: scrub queued for path: / 2024-07-26T13:44:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:24 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:44:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:24 smithi167 ceph-mon[37140]: scrub summary: active paths [/] 2024-07-26T13:44:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:24 smithi167 ceph-mon[37140]: scrub summary: active paths [/] 2024-07-26T13:44:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:24 smithi167 ceph-mon[37140]: scrub summary: active paths [/] 2024-07-26T13:44:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:24 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:44:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:24 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:44:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:24 smithi167 ceph-mon[37140]: scrub summary: active paths [/] 2024-07-26T13:44:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:24 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:44:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:24 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:44:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:24 smithi167 ceph-mon[37140]: pgmap v494: 97 pgs: 97 active+clean; 60 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 111 op/s 2024-07-26T13:44:25.412 INFO:tasks.workunit.client.0.smithi100.stdout: 1 155296 1.26 MB/sec execute 39 sec latency 137.271 ms 2024-07-26T13:44:26.412 INFO:tasks.workunit.client.0.smithi100.stdout: 1 155296 1.23 MB/sec execute 40 sec latency 1137.394 ms 2024-07-26T13:44:26.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.506+0000 7f1adccd7640 1 -- 172.21.15.100:0/1827694437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ad8108720 msgr2=0x7f1ad810ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:26.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.506+0000 7f1adccd7640 1 --2- 172.21.15.100:0/1827694437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ad8108720 0x7f1ad810ef50 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0009900 tx=0x7f1ad002a410 comp rx=0 tx=0).stop 2024-07-26T13:44:26.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.506+0000 7f1adccd7640 1 -- 172.21.15.100:0/1827694437 shutdown_connections 2024-07-26T13:44:26.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.506+0000 7f1adccd7640 1 --2- 172.21.15.100:0/1827694437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ad8108720 0x7f1ad810ef50 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:26.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.506+0000 7f1adccd7640 1 --2- 172.21.15.100:0/1827694437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ad8107dd0 0x7f1ad81081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:26.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.506+0000 7f1adccd7640 1 --2- 172.21.15.100:0/1827694437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ad8102490 0x7f1ad81028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:26.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.506+0000 7f1adccd7640 1 -- 172.21.15.100:0/1827694437 >> 172.21.15.100:0/1827694437 conn(0x7f1ad8079160 msgr2=0x7f1ad8079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:26.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.506+0000 7f1adccd7640 1 -- 172.21.15.100:0/1827694437 shutdown_connections 2024-07-26T13:44:26.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.506+0000 7f1adccd7640 1 -- 172.21.15.100:0/1827694437 wait complete. 2024-07-26T13:44:26.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.507+0000 7f1adccd7640 1 Processor -- start 2024-07-26T13:44:26.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.507+0000 7f1adccd7640 1 -- start start 2024-07-26T13:44:26.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.507+0000 7f1adccd7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ad8102490 0x7f1ad8199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:26.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.507+0000 7f1adccd7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ad8107dd0 0x7f1ad819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:26.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.507+0000 7f1adccd7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ad8108720 0x7f1ad819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:26.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.507+0000 7f1adccd7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad81134f0 con 0x7f1ad8107dd0 2024-07-26T13:44:26.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.507+0000 7f1adccd7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad8113370 con 0x7f1ad8108720 2024-07-26T13:44:26.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.507+0000 7f1adccd7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad8113670 con 0x7f1ad8102490 2024-07-26T13:44:26.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.507+0000 7f1ad5d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ad8107dd0 0x7f1ad819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:26.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.507+0000 7f1ad5d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ad8107dd0 0x7f1ad819a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55166/0 (socket says 172.21.15.100:55166) 2024-07-26T13:44:26.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.507+0000 7f1ad5d9b640 1 -- 172.21.15.100:0/2931272986 learned_addr learned my addr 172.21.15.100:0/2931272986 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:26.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.508+0000 7f1ad6d9d640 1 --2- 172.21.15.100:0/2931272986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ad8108720 0x7f1ad819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:26.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.508+0000 7f1ad659c640 1 --2- 172.21.15.100:0/2931272986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ad8102490 0x7f1ad8199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:26.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.508+0000 7f1ad5d9b640 1 -- 172.21.15.100:0/2931272986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ad8102490 msgr2=0x7f1ad8199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:26.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.508+0000 7f1ad5d9b640 1 --2- 172.21.15.100:0/2931272986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ad8102490 0x7f1ad8199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:26.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.508+0000 7f1ad5d9b640 1 -- 172.21.15.100:0/2931272986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ad8108720 msgr2=0x7f1ad819f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:26.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.508+0000 7f1ad5d9b640 1 --2- 172.21.15.100:0/2931272986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ad8108720 0x7f1ad819f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:26.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.508+0000 7f1ad5d9b640 1 -- 172.21.15.100:0/2931272986 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad819fdf0 con 0x7f1ad8107dd0 2024-07-26T13:44:26.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.508+0000 7f1ad6d9d640 1 --2- 172.21.15.100:0/2931272986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ad8108720 0x7f1ad819f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:44:26.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.508+0000 7f1ad5d9b640 1 --2- 172.21.15.100:0/2931272986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ad8107dd0 0x7f1ad819a4e0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000c970 tx=0x7f1ac000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:26.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.508+0000 7f1acf7fe640 1 -- 172.21.15.100:0/2931272986 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0014070 con 0x7f1ad8107dd0 2024-07-26T13:44:26.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.508+0000 7f1acf7fe640 1 -- 172.21.15.100:0/2931272986 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ac0004440 con 0x7f1ad8107dd0 2024-07-26T13:44:26.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.508+0000 7f1acf7fe640 1 -- 172.21.15.100:0/2931272986 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0003e50 con 0x7f1ad8107dd0 2024-07-26T13:44:26.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.508+0000 7f1adccd7640 1 -- 172.21.15.100:0/2931272986 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ad81a00e0 con 0x7f1ad8107dd0 2024-07-26T13:44:26.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.508+0000 7f1adccd7640 1 -- 172.21.15.100:0/2931272986 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ad81a6900 con 0x7f1ad8107dd0 2024-07-26T13:44:26.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.511+0000 7f1ae325b740 1 Processor -- start 2024-07-26T13:44:26.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.511+0000 7f1ae325b740 1 -- start start 2024-07-26T13:44:26.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.511+0000 7f1ae325b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ec9d62070 0x555eca107640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:26.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.511+0000 7f1ae325b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec9d391b0 0x555eca107b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:26.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.511+0000 7f1ae325b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ec9d41390 0x555eca1080c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:26.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.511+0000 7f1ae325b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x555eca07db60 con 0x555ec9d391b0 2024-07-26T13:44:26.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.511+0000 7f1acf7fe640 1 -- 172.21.15.100:0/2931272986 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f1ac0017070 con 0x7f1ad8107dd0 2024-07-26T13:44:26.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.511+0000 7f1ae325b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555eca06d060 con 0x555ec9d62070 2024-07-26T13:44:26.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.511+0000 7f1ae325b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555eca0710a0 con 0x555ec9d41390 2024-07-26T13:44:26.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.511+0000 7f1ad659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec9d391b0 0x555eca107b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:26.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.511+0000 7f1ad659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec9d391b0 0x555eca107b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55178/0 (socket says 172.21.15.100:55178) 2024-07-26T13:44:26.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.511+0000 7f1ad659c640 1 -- 172.21.15.100:0/1012919137 learned_addr learned my addr 172.21.15.100:0/1012919137 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:26.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.511+0000 7f1ad6d9d640 1 --2- 172.21.15.100:0/1012919137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ec9d62070 0x555eca107640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:26.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.511+0000 7f1ad5d9b640 1 --2- 172.21.15.100:0/1012919137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ec9d41390 0x555eca1080c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:26.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.512+0000 7f1ad659c640 1 -- 172.21.15.100:0/1012919137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ec9d41390 msgr2=0x555eca1080c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:26.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.512+0000 7f1ad659c640 1 --2- 172.21.15.100:0/1012919137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ec9d41390 0x555eca1080c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:26.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.512+0000 7f1ad659c640 1 -- 172.21.15.100:0/1012919137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ec9d62070 msgr2=0x555eca107640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:26.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.512+0000 7f1ad659c640 1 --2- 172.21.15.100:0/1012919137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ec9d62070 0x555eca107640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:26.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.512+0000 7f1ad659c640 1 -- 172.21.15.100:0/1012919137 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eca013b40 con 0x555ec9d391b0 2024-07-26T13:44:26.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.512+0000 7f1ad659c640 1 --2- 172.21.15.100:0/1012919137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec9d391b0 0x555eca107b80 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8009fa0 tx=0x7f1ac800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab3e7253f490c520 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:26.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.512+0000 7f1accff9640 1 -- 172.21.15.100:0/1012919137 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac801a070 con 0x555ec9d391b0 2024-07-26T13:44:26.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.512+0000 7f1accff9640 1 -- 172.21.15.100:0/1012919137 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ac80158c0 con 0x555ec9d391b0 2024-07-26T13:44:26.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.512+0000 7f1accff9640 1 -- 172.21.15.100:0/1012919137 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac800a8c0 con 0x555ec9d391b0 2024-07-26T13:44:26.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.512+0000 7f1acf7fe640 1 --2- 172.21.15.100:0/2931272986 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1ac40b3f90 0x7f1ac40b64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:26.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.512+0000 7f1ad6d9d640 1 --2- 172.21.15.100:0/2931272986 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1ac40b3f90 0x7f1ac40b64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:26.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.512+0000 7f1acf7fe640 1 -- 172.21.15.100:0/2931272986 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f1ac00e3b90 con 0x7f1ad8107dd0 2024-07-26T13:44:26.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.513+0000 7f1ae325b740 1 -- 172.21.15.100:0/1012919137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec9d391b0 msgr2=0x555eca107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:26.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.513+0000 7f1ad6d9d640 1 --2- 172.21.15.100:0/2931272986 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1ac40b3f90 0x7f1ac40b64a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0005940 tx=0x7f1ad0009660 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:26.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.513+0000 7f1ae325b740 1 --2- 172.21.15.100:0/1012919137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec9d391b0 0x555eca107b80 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8009fa0 tx=0x7f1ac800c8b0 comp rx=0 tx=0).stop 2024-07-26T13:44:26.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.513+0000 7f1ae325b740 1 -- 172.21.15.100:0/1012919137 shutdown_connections 2024-07-26T13:44:26.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.513+0000 7f1ae325b740 1 --2- 172.21.15.100:0/1012919137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ec9d41390 0x555eca1080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:26.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.513+0000 7f1ae325b740 1 --2- 172.21.15.100:0/1012919137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec9d391b0 0x555eca107b80 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:26.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.513+0000 7f1ae325b740 1 --2- 172.21.15.100:0/1012919137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ec9d62070 0x555eca107640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:26.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.513+0000 7f1ae325b740 1 -- 172.21.15.100:0/1012919137 >> 172.21.15.100:0/1012919137 conn(0x555ec9d56460 msgr2=0x555eca105530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:26.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.513+0000 7f1ae325b740 1 -- 172.21.15.100:0/1012919137 shutdown_connections 2024-07-26T13:44:26.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.513+0000 7f1ae325b740 1 -- 172.21.15.100:0/1012919137 wait complete. 2024-07-26T13:44:26.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.513+0000 7f1ae325b740 1 -- start start 2024-07-26T13:44:26.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.514+0000 7f1ae325b740 1 Processor -- start 2024-07-26T13:44:26.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.514+0000 7f1ae325b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec9d391b0 0x555eca113220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:26.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.514+0000 7f1ae325b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ec9d41390 0x555eca113760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:26.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.514+0000 7f1ae325b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ec9d62070 0x555eca11f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:26.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.514+0000 7f1ae325b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x555ec9ed0700 con 0x555ec9d391b0 2024-07-26T13:44:26.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.514+0000 7f1ae325b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555eca00f990 con 0x555ec9d62070 2024-07-26T13:44:26.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.514+0000 7f1ae325b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555eca115e20 con 0x555ec9d41390 2024-07-26T13:44:26.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.514+0000 7f1ae325b740 10 client.0 _ll_register_callbacks cb 0x555eca02a4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:44:26.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.514+0000 7f1ad659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec9d391b0 0x555eca113220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:26.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.514+0000 7f1ad659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec9d391b0 0x555eca113220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55190/0 (socket says 172.21.15.100:55190) 2024-07-26T13:44:26.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.514+0000 7f1ad659c640 1 -- 172.21.15.100:0/2394003356 learned_addr learned my addr 172.21.15.100:0/2394003356 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:26.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.514+0000 7f1ad6d9d640 1 --2- 172.21.15.100:0/2394003356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ec9d62070 0x555eca11f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:26.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.514+0000 7f1ad659c640 1 -- 172.21.15.100:0/2394003356 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ec9d41390 msgr2=0x555eca113760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:26.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.514+0000 7f1ad5d9b640 1 --2- 172.21.15.100:0/2394003356 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ec9d41390 0x555eca113760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:26.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.514+0000 7f1ad659c640 1 --2- 172.21.15.100:0/2394003356 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ec9d41390 0x555eca113760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:26.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.514+0000 7f1ad659c640 1 -- 172.21.15.100:0/2394003356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ec9d62070 msgr2=0x555eca11f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:26.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.514+0000 7f1ad659c640 1 --2- 172.21.15.100:0/2394003356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ec9d62070 0x555eca11f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:26.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.514+0000 7f1ad659c640 1 -- 172.21.15.100:0/2394003356 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ec9dcd650 con 0x555ec9d391b0 2024-07-26T13:44:26.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.514+0000 7f1ad6d9d640 1 --2- 172.21.15.100:0/2394003356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ec9d62070 0x555eca11f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:44:26.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.514+0000 7f1ad5d9b640 1 --2- 172.21.15.100:0/2394003356 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ec9d41390 0x555eca113760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:44:26.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.515+0000 7f1ad659c640 1 --2- 172.21.15.100:0/2394003356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec9d391b0 0x555eca113220 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f1ac800cfd0 tx=0x7f1ac8015ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440140884c638baa server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:26.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.515+0000 7f1abdffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:44:26.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.515+0000 7f1abdffb640 1 -- 172.21.15.100:0/2394003356 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac801a040 con 0x555ec9d391b0 2024-07-26T13:44:26.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.515+0000 7f1abdffb640 1 -- 172.21.15.100:0/2394003356 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f1ac800a640 con 0x555ec9d391b0 2024-07-26T13:44:26.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.515+0000 7f1ae325b740 1 -- 172.21.15.100:0/2394003356 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ac80158c0 con 0x555ec9d391b0 2024-07-26T13:44:26.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.515+0000 7f1abdffb640 1 -- 172.21.15.100:0/2394003356 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1ac802e050 con 0x555ec9d391b0 2024-07-26T13:44:26.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.515+0000 7f1ae325b740 10 client.16638 fetch_fsmap learned FSMap version 37 2024-07-26T13:44:26.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.515+0000 7f1ae325b740 1 -- 172.21.15.100:0/2394003356 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555eca121e60 con 0x555ec9d391b0 2024-07-26T13:44:26.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.515+0000 7f1abdffb640 1 -- 172.21.15.100:0/2394003356 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ac80158c0 con 0x555ec9d391b0 2024-07-26T13:44:26.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.515+0000 7f1abdffb640 1 -- 172.21.15.100:0/2394003356 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f1ac8020450 con 0x555ec9d391b0 2024-07-26T13:44:26.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.515+0000 7f1ae325b740 10 client.16638 fetch_fsmap finished waiting for FSMap version 37 2024-07-26T13:44:26.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.515+0000 7f1ae325b740 10 client.16638 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:44:26.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.515+0000 7f1ae325b740 20 client.16638 populate_metadata read hostname 'smithi100' 2024-07-26T13:44:26.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.515+0000 7f1ae325b740 1 --2- 172.21.15.100:0/2394003356 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555ec9d209f0 0x555eca125270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:26.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.515+0000 7f1ae325b740 4 client.16638 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:44:26.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.515+0000 7f1ae325b740 1 -- 172.21.15.100:0/2394003356 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555eca07fab0 con 0x555ec9d209f0 2024-07-26T13:44:26.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.516+0000 7f1ad5d9b640 1 --2- 172.21.15.100:0/2394003356 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555ec9d209f0 0x555eca125270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:26.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.516+0000 7f1abdffb640 10 client.16638 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:26.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.516+0000 7f1ad5d9b640 1 --2- 172.21.15.100:0/2394003356 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555ec9d209f0 0x555eca125270 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f99c82323de14c46 server_cookie=4496a8d86bf4a363 in_seq=0 out_seq=0 2024-07-26T13:44:26.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.517+0000 7f1abdffb640 1 -- 172.21.15.100:0/2394003356 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555eca07fab0 con 0x555ec9d209f0 2024-07-26T13:44:26.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.517+0000 7f1abdffb640 10 client.16638 handle_command_reply: tid=0 2024-07-26T13:44:26.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.517+0000 7f1ae325b740 1 client.16638 shutdown 2024-07-26T13:44:26.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.517+0000 7f1ae325b740 1 -- 172.21.15.100:0/2394003356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec9d391b0 msgr2=0x555eca113220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:26.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.517+0000 7f1ae325b740 1 --2- 172.21.15.100:0/2394003356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec9d391b0 0x555eca113220 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f1ac800cfd0 tx=0x7f1ac8015ca0 comp rx=0 tx=0).stop 2024-07-26T13:44:26.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.517+0000 7f1ae325b740 1 -- 172.21.15.100:0/2394003356 shutdown_connections 2024-07-26T13:44:26.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.517+0000 7f1ae325b740 1 --2- 172.21.15.100:0/2394003356 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555ec9d209f0 0x555eca125270 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:26.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.517+0000 7f1ae325b740 1 --2- 172.21.15.100:0/2394003356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ec9d62070 0x555eca11f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:26.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.517+0000 7f1ae325b740 1 --2- 172.21.15.100:0/2394003356 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ec9d41390 0x555eca113760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:26.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.517+0000 7f1abdffb640 0 client.16638 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:26.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.517+0000 7f1ae325b740 1 --2- 172.21.15.100:0/2394003356 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec9d391b0 0x555eca113220 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:26.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.517+0000 7f1ae325b740 1 -- 172.21.15.100:0/2394003356 >> 172.21.15.100:0/2394003356 conn(0x555ec9d56460 msgr2=0x555eca10a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:26.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.517+0000 7f1ae325b740 1 -- 172.21.15.100:0/2394003356 shutdown_connections 2024-07-26T13:44:26.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.517+0000 7f1ae325b740 1 -- 172.21.15.100:0/2394003356 wait complete. 2024-07-26T13:44:26.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.517+0000 7f1ae325b740 20 client.16638 trim_cache size 0 max 16384 2024-07-26T13:44:26.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.535+0000 7f1ae325b740 1 Processor -- start 2024-07-26T13:44:26.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.535+0000 7f1ae325b740 1 -- start start 2024-07-26T13:44:26.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.535+0000 7f1ae325b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec9d391b0 0x555eca108d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:26.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.535+0000 7f1ae325b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ec9d41390 0x555eca12a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:26.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.535+0000 7f1ae325b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ec9d62070 0x555eca12aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:26.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.535+0000 7f1ae325b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad81b1b40 con 0x555ec9d391b0 2024-07-26T13:44:26.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.535+0000 7f1ae325b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad81af3b0 con 0x555ec9d62070 2024-07-26T13:44:26.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.535+0000 7f1ae325b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad81ad200 con 0x555ec9d41390 2024-07-26T13:44:26.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.535+0000 7f1ad659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec9d391b0 0x555eca108d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:26.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.535+0000 7f1ad659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec9d391b0 0x555eca108d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55194/0 (socket says 172.21.15.100:55194) 2024-07-26T13:44:26.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.535+0000 7f1ad659c640 1 -- 172.21.15.100:0/3396720042 learned_addr learned my addr 172.21.15.100:0/3396720042 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:26.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.535+0000 7f1ad6d9d640 1 --2- 172.21.15.100:0/3396720042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ec9d62070 0x555eca12aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:26.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.535+0000 7f1ad5d9b640 1 --2- 172.21.15.100:0/3396720042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ec9d41390 0x555eca12a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:26.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.536+0000 7f1ad659c640 1 -- 172.21.15.100:0/3396720042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ec9d41390 msgr2=0x555eca12a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:26.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.536+0000 7f1ad659c640 1 --2- 172.21.15.100:0/3396720042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ec9d41390 0x555eca12a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:26.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.536+0000 7f1ad659c640 1 -- 172.21.15.100:0/3396720042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ec9d62070 msgr2=0x555eca12aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:26.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.536+0000 7f1ad659c640 1 --2- 172.21.15.100:0/3396720042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ec9d62070 0x555eca12aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:26.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.536+0000 7f1ad659c640 1 -- 172.21.15.100:0/3396720042 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eca109930 con 0x555ec9d391b0 2024-07-26T13:44:26.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.536+0000 7f1ad659c640 1 --2- 172.21.15.100:0/3396720042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec9d391b0 0x555eca108d80 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8004b90 tx=0x7f1ac8009ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0fda36b9409dee server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:26.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.536+0000 7f1abffff640 1 -- 172.21.15.100:0/3396720042 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac801a040 con 0x555ec9d391b0 2024-07-26T13:44:26.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.536+0000 7f1abffff640 1 -- 172.21.15.100:0/3396720042 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ac8025070 con 0x555ec9d391b0 2024-07-26T13:44:26.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.536+0000 7f1abffff640 1 -- 172.21.15.100:0/3396720042 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac8020a90 con 0x555ec9d391b0 2024-07-26T13:44:26.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.537+0000 7f1ae325b740 1 -- 172.21.15.100:0/3396720042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec9d391b0 msgr2=0x555eca108d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:26.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.537+0000 7f1ae325b740 1 --2- 172.21.15.100:0/3396720042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec9d391b0 0x555eca108d80 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8004b90 tx=0x7f1ac8009ce0 comp rx=0 tx=0).stop 2024-07-26T13:44:26.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.537+0000 7f1ae325b740 1 -- 172.21.15.100:0/3396720042 shutdown_connections 2024-07-26T13:44:26.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.537+0000 7f1ae325b740 1 --2- 172.21.15.100:0/3396720042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ec9d62070 0x555eca12aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:26.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.537+0000 7f1ae325b740 1 --2- 172.21.15.100:0/3396720042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ec9d41390 0x555eca12a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:26.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.537+0000 7f1ae325b740 1 --2- 172.21.15.100:0/3396720042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec9d391b0 0x555eca108d80 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:26.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.537+0000 7f1ae325b740 1 -- 172.21.15.100:0/3396720042 >> 172.21.15.100:0/3396720042 conn(0x555ec9d56460 msgr2=0x555eca108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:26.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.537+0000 7f1ae325b740 1 -- 172.21.15.100:0/3396720042 shutdown_connections 2024-07-26T13:44:26.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.537+0000 7f1ae325b740 1 -- 172.21.15.100:0/3396720042 wait complete. 2024-07-26T13:44:26.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.537+0000 7f1ae325b740 1 -- start start 2024-07-26T13:44:26.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.537+0000 7f1ae325b740 1 Processor -- start 2024-07-26T13:44:26.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.538+0000 7f1ae325b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ec9d391b0 0x555eca185930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:26.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.538+0000 7f1ae325b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ec9d41390 0x555eca185e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:26.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.538+0000 7f1ae325b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec9d62070 0x555eca1863b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:26.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.538+0000 7f1ae325b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad81ad870 con 0x555ec9d62070 2024-07-26T13:44:26.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.538+0000 7f1ae325b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad81aece0 con 0x555ec9d391b0 2024-07-26T13:44:26.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.538+0000 7f1ae325b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad81adf40 con 0x555ec9d41390 2024-07-26T13:44:26.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.538+0000 7f1ae325b740 10 client.0 _ll_register_callbacks cb 0x7f1ac8018070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:44:26.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.538+0000 7f1ad6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec9d62070 0x555eca1863b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:26.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.538+0000 7f1ad6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec9d62070 0x555eca1863b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55200/0 (socket says 172.21.15.100:55200) 2024-07-26T13:44:26.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.538+0000 7f1ad6d9d640 1 -- 172.21.15.100:0/383213730 learned_addr learned my addr 172.21.15.100:0/383213730 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:26.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.538+0000 7f1ad5d9b640 1 --2- 172.21.15.100:0/383213730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ec9d41390 0x555eca185e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:26.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.538+0000 7f1ad659c640 1 --2- 172.21.15.100:0/383213730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ec9d391b0 0x555eca185930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:26.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.538+0000 7f1ad6d9d640 1 -- 172.21.15.100:0/383213730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ec9d41390 msgr2=0x555eca185e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:26.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.538+0000 7f1ad6d9d640 1 --2- 172.21.15.100:0/383213730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ec9d41390 0x555eca185e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:26.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.538+0000 7f1ad6d9d640 1 -- 172.21.15.100:0/383213730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ec9d391b0 msgr2=0x555eca185930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:26.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.538+0000 7f1ad6d9d640 1 --2- 172.21.15.100:0/383213730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ec9d391b0 0x555eca185930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:26.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.538+0000 7f1ad6d9d640 1 -- 172.21.15.100:0/383213730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eca1909b0 con 0x555ec9d62070 2024-07-26T13:44:26.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.538+0000 7f1ad6d9d640 1 --2- 172.21.15.100:0/383213730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec9d62070 0x555eca1863b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0009900 tx=0x7f1ad0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9184ae58f86425 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:26.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.538+0000 7f1abdffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:44:26.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.539+0000 7f1abdffb640 1 -- 172.21.15.100:0/383213730 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0035040 con 0x555ec9d62070 2024-07-26T13:44:26.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.539+0000 7f1abdffb640 1 -- 172.21.15.100:0/383213730 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7f1ad0003a00 con 0x555ec9d62070 2024-07-26T13:44:26.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.539+0000 7f1ae325b740 1 -- 172.21.15.100:0/383213730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ac8025070 con 0x555ec9d62070 2024-07-26T13:44:26.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.539+0000 7f1abdffb640 1 -- 172.21.15.100:0/383213730 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1ad0003c00 con 0x555ec9d62070 2024-07-26T13:44:26.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.539+0000 7f1ae325b740 10 client.16644 fetch_fsmap learned FSMap version 37 2024-07-26T13:44:26.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.539+0000 7f1ae325b740 1 -- 172.21.15.100:0/383213730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555eca191c40 con 0x555ec9d62070 2024-07-26T13:44:26.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.539+0000 7f1abdffb640 1 -- 172.21.15.100:0/383213730 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ac8025070 con 0x555ec9d62070 2024-07-26T13:44:26.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.539+0000 7f1abdffb640 1 -- 172.21.15.100:0/383213730 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f1ad005a070 con 0x555ec9d62070 2024-07-26T13:44:26.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.539+0000 7f1ae325b740 10 client.16644 fetch_fsmap finished waiting for FSMap version 37 2024-07-26T13:44:26.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.539+0000 7f1ae325b740 10 client.16644 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:44:26.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.539+0000 7f1ae325b740 20 client.16644 populate_metadata read hostname 'smithi100' 2024-07-26T13:44:26.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.539+0000 7f1ae325b740 1 --2- 172.21.15.100:0/383213730 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555ec9cf5da0 0x555eca1944c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:26.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.539+0000 7f1ae325b740 4 client.16644 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-26T13:44:26.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.539+0000 7f1ae325b740 1 -- 172.21.15.100:0/383213730 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555eca07b550 con 0x555ec9cf5da0 2024-07-26T13:44:26.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.539+0000 7f1ad659c640 1 --2- 172.21.15.100:0/383213730 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555ec9cf5da0 0x555eca1944c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:26.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.540+0000 7f1abdffb640 10 client.16644 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:26.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.540+0000 7f1ad659c640 1 --2- 172.21.15.100:0/383213730 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555ec9cf5da0 0x555eca1944c0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9b56c56c4e545f4 server_cookie=4d10371d92e59e6 in_seq=0 out_seq=0 2024-07-26T13:44:26.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.540+0000 7f1abdffb640 1 -- 172.21.15.100:0/383213730 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555eca07b550 con 0x555ec9cf5da0 2024-07-26T13:44:26.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.540+0000 7f1abdffb640 10 client.16644 handle_command_reply: tid=0 2024-07-26T13:44:26.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.540+0000 7f1ae325b740 1 client.16644 shutdown 2024-07-26T13:44:26.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.540+0000 7f1ae325b740 1 -- 172.21.15.100:0/383213730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec9d62070 msgr2=0x555eca1863b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:26.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.540+0000 7f1ae325b740 1 --2- 172.21.15.100:0/383213730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec9d62070 0x555eca1863b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0009900 tx=0x7f1ad0009260 comp rx=0 tx=0).stop 2024-07-26T13:44:26.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.540+0000 7f1ae325b740 1 -- 172.21.15.100:0/383213730 shutdown_connections 2024-07-26T13:44:26.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.540+0000 7f1ae325b740 1 --2- 172.21.15.100:0/383213730 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555ec9cf5da0 0x555eca1944c0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:26.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.540+0000 7f1abdffb640 0 client.16644 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:26.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.540+0000 7f1ae325b740 1 --2- 172.21.15.100:0/383213730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555ec9d62070 0x555eca1863b0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:26.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.540+0000 7f1ae325b740 1 --2- 172.21.15.100:0/383213730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ec9d41390 0x555eca185e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:26.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.540+0000 7f1ae325b740 1 --2- 172.21.15.100:0/383213730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ec9d391b0 0x555eca185930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:26.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.540+0000 7f1ae325b740 1 -- 172.21.15.100:0/383213730 >> 172.21.15.100:0/383213730 conn(0x555ec9d56460 msgr2=0x555eca1144d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:26.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.540+0000 7f1ae325b740 1 -- 172.21.15.100:0/383213730 shutdown_connections 2024-07-26T13:44:26.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.540+0000 7f1ae325b740 1 -- 172.21.15.100:0/383213730 wait complete. 2024-07-26T13:44:26.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.541+0000 7f1ae325b740 20 client.16644 trim_cache size 0 max 16384 2024-07-26T13:44:26.549 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:44:26.549 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-26T13:44:26.549 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-07-26T13:44:26.549 INFO:teuthology.orchestra.run.smithi100.stdout: "c66b943d-9809-430e-96fa-842951f15197": { 2024-07-26T13:44:26.549 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-07-26T13:44:26.549 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "c66b943d-9809-430e-96fa-842951f15197", 2024-07-26T13:44:26.549 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-07-26T13:44:26.549 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:44:26.549 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:44:26.550 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:44:26.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.541+0000 7f1adccd7640 1 -- 172.21.15.100:0/2931272986 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1ac40b3f90 msgr2=0x7f1ac40b64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:26.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.541+0000 7f1adccd7640 1 --2- 172.21.15.100:0/2931272986 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1ac40b3f90 0x7f1ac40b64a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0005940 tx=0x7f1ad0009660 comp rx=0 tx=0).stop 2024-07-26T13:44:26.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.541+0000 7f1adccd7640 1 -- 172.21.15.100:0/2931272986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ad8107dd0 msgr2=0x7f1ad819a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:26.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.541+0000 7f1adccd7640 1 --2- 172.21.15.100:0/2931272986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ad8107dd0 0x7f1ad819a4e0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000c970 tx=0x7f1ac000ce30 comp rx=0 tx=0).stop 2024-07-26T13:44:26.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.541+0000 7f1adccd7640 1 -- 172.21.15.100:0/2931272986 shutdown_connections 2024-07-26T13:44:26.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.541+0000 7f1adccd7640 1 --2- 172.21.15.100:0/2931272986 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1ac40b3f90 0x7f1ac40b64a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:26.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.541+0000 7f1adccd7640 1 --2- 172.21.15.100:0/2931272986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ad8108720 0x7f1ad819f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:26.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.541+0000 7f1adccd7640 1 --2- 172.21.15.100:0/2931272986 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1ad8107dd0 0x7f1ad819a4e0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:26.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.541+0000 7f1adccd7640 1 --2- 172.21.15.100:0/2931272986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ad8102490 0x7f1ad8199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:26.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.541+0000 7f1adccd7640 1 -- 172.21.15.100:0/2931272986 >> 172.21.15.100:0/2931272986 conn(0x7f1ad8079160 msgr2=0x7f1ad8104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:26.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.541+0000 7f1adccd7640 1 -- 172.21.15.100:0/2931272986 shutdown_connections 2024-07-26T13:44:26.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:26.541+0000 7f1adccd7640 1 -- 172.21.15.100:0/2931272986 wait complete. 2024-07-26T13:44:27.292 INFO:tasks.cephfs.filesystem:scrub status for tag:c66b943d-9809-430e-96fa-842951f15197 - {'path': '/', 'tag': 'c66b943d-9809-430e-96fa-842951f15197', 'options': 'recursive,force'} 2024-07-26T13:44:27.413 INFO:tasks.workunit.client.0.smithi100.stdout: 1 155296 1.20 MB/sec execute 41 sec latency 2137.455 ms 2024-07-26T13:44:27.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:27 smithi138 ceph-mon[36954]: pgmap v495: 97 pgs: 97 active+clean; 63 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 130 op/s 2024-07-26T13:44:27.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:27 smithi100 ceph-mon[32599]: pgmap v495: 97 pgs: 97 active+clean; 63 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 130 op/s 2024-07-26T13:44:28.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:27 smithi167 ceph-mon[37140]: pgmap v495: 97 pgs: 97 active+clean; 63 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 130 op/s 2024-07-26T13:44:28.413 INFO:tasks.workunit.client.0.smithi100.stdout: 1 155296 1.17 MB/sec execute 42 sec latency 3137.569 ms 2024-07-26T13:44:29.413 INFO:tasks.workunit.client.0.smithi100.stdout: 1 155296 1.14 MB/sec execute 43 sec latency 4137.680 ms 2024-07-26T13:44:29.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:29 smithi138 ceph-mon[36954]: pgmap v496: 97 pgs: 97 active+clean; 64 MiB data, 802 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-07-26T13:44:29.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:29 smithi100 ceph-mon[32599]: pgmap v496: 97 pgs: 97 active+clean; 64 MiB data, 802 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-07-26T13:44:30.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:29 smithi167 ceph-mon[37140]: pgmap v496: 97 pgs: 97 active+clean; 64 MiB data, 802 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-07-26T13:44:30.413 INFO:tasks.workunit.client.0.smithi100.stdout: 1 155296 1.12 MB/sec execute 44 sec latency 5137.756 ms 2024-07-26T13:44:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:30 smithi138 ceph-mon[36954]: scrub summary: idle 2024-07-26T13:44:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:30 smithi138 ceph-mon[36954]: scrub summary: idle 2024-07-26T13:44:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:30 smithi138 ceph-mon[36954]: scrub summary: idle 2024-07-26T13:44:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:30 smithi100 ceph-mon[32599]: scrub summary: idle 2024-07-26T13:44:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:30 smithi100 ceph-mon[32599]: scrub summary: idle 2024-07-26T13:44:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:30 smithi100 ceph-mon[32599]: scrub summary: idle 2024-07-26T13:44:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:30 smithi167 ceph-mon[37140]: scrub summary: idle 2024-07-26T13:44:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:30 smithi167 ceph-mon[37140]: scrub summary: idle 2024-07-26T13:44:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:30 smithi167 ceph-mon[37140]: scrub summary: idle 2024-07-26T13:44:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.027+0000 7efdad577640 1 -- 172.21.15.100:0/3954591853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efda8107d10 msgr2=0x7efda8108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.027+0000 7efdad577640 1 --2- 172.21.15.100:0/3954591853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efda8107d10 0x7efda8108120 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7efd9c009900 tx=0x7efd9c02a410 comp rx=0 tx=0).stop 2024-07-26T13:44:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.027+0000 7efdad577640 1 -- 172.21.15.100:0/3954591853 shutdown_connections 2024-07-26T13:44:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.027+0000 7efdad577640 1 --2- 172.21.15.100:0/3954591853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efda8108660 0x7efda810ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:31.028 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.027+0000 7efdad577640 1 --2- 172.21.15.100:0/3954591853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efda8107d10 0x7efda8108120 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.027+0000 7efdad577640 1 --2- 172.21.15.100:0/3954591853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efda81023e0 0x7efda81027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.027+0000 7efdad577640 1 -- 172.21.15.100:0/3954591853 >> 172.21.15.100:0/3954591853 conn(0x7efda807a170 msgr2=0x7efda807a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.027+0000 7efdad577640 1 -- 172.21.15.100:0/3954591853 shutdown_connections 2024-07-26T13:44:31.029 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.027+0000 7efdad577640 1 -- 172.21.15.100:0/3954591853 wait complete. 2024-07-26T13:44:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.028+0000 7efdad577640 1 Processor -- start 2024-07-26T13:44:31.031 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.028+0000 7efdad577640 1 -- start start 2024-07-26T13:44:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.028+0000 7efdad577640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efda81023e0 0x7efda8198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.028+0000 7efdad577640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efda8107d10 0x7efda8199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.028+0000 7efdad577640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efda8108660 0x7efda81995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.028+0000 7efdad577640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efda8112340 con 0x7efda8107d10 2024-07-26T13:44:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.028+0000 7efdad577640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efda81121c0 con 0x7efda8108660 2024-07-26T13:44:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.028+0000 7efdad577640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efda81124c0 con 0x7efda81023e0 2024-07-26T13:44:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.028+0000 7efda67fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efda8107d10 0x7efda8199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.028+0000 7efda67fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efda8107d10 0x7efda8199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55216/0 (socket says 172.21.15.100:55216) 2024-07-26T13:44:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.029+0000 7efda67fc640 1 -- 172.21.15.100:0/477330550 learned_addr learned my addr 172.21.15.100:0/477330550 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.029+0000 7efda6ffd640 1 --2- 172.21.15.100:0/477330550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efda81023e0 0x7efda8198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.029+0000 7efda77fe640 1 --2- 172.21.15.100:0/477330550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efda8108660 0x7efda81995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.029+0000 7efda67fc640 1 -- 172.21.15.100:0/477330550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efda81023e0 msgr2=0x7efda8198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:31.032 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.029+0000 7efda67fc640 1 --2- 172.21.15.100:0/477330550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efda81023e0 0x7efda8198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:31.033 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.029+0000 7efda67fc640 1 -- 172.21.15.100:0/477330550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efda8108660 msgr2=0x7efda81995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:31.033 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.029+0000 7efda67fc640 1 --2- 172.21.15.100:0/477330550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efda8108660 0x7efda81995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:31.033 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.029+0000 7efda67fc640 1 -- 172.21.15.100:0/477330550 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda81a81c0 con 0x7efda8107d10 2024-07-26T13:44:31.033 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.029+0000 7efda67fc640 1 --2- 172.21.15.100:0/477330550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efda8107d10 0x7efda8199080 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7efd9c005ab0 tx=0x7efd9c003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:31.033 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.029+0000 7efd8ffff640 1 -- 172.21.15.100:0/477330550 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c03f070 con 0x7efda8107d10 2024-07-26T13:44:31.033 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.029+0000 7efd8ffff640 1 -- 172.21.15.100:0/477330550 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd9c031d60 con 0x7efda8107d10 2024-07-26T13:44:31.033 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.029+0000 7efdad577640 1 -- 172.21.15.100:0/477330550 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efda81a83f0 con 0x7efda8107d10 2024-07-26T13:44:31.033 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.029+0000 7efd8ffff640 1 -- 172.21.15.100:0/477330550 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c035030 con 0x7efda8107d10 2024-07-26T13:44:31.033 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.029+0000 7efdad577640 1 -- 172.21.15.100:0/477330550 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efda81a8940 con 0x7efda8107d10 2024-07-26T13:44:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.032+0000 7efdb4569740 1 Processor -- start 2024-07-26T13:44:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.032+0000 7efdb4569740 1 -- start start 2024-07-26T13:44:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.032+0000 7efd8ffff640 1 -- 172.21.15.100:0/477330550 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7efd9c042070 con 0x7efda8107d10 2024-07-26T13:44:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.032+0000 7efdb4569740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e296ca1b0 0x562e29a6c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.032+0000 7efdb4569740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e296a12f0 0x562e29a6cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.032+0000 7efdb4569740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e296a94d0 0x562e29a6d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.032+0000 7efdb4569740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x562e299e23b0 con 0x562e296a94d0 2024-07-26T13:44:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.032+0000 7efdb4569740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562e299d18b0 con 0x562e296ca1b0 2024-07-26T13:44:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.032+0000 7efdb4569740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562e299d58f0 con 0x562e296a12f0 2024-07-26T13:44:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.032+0000 7efda67fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e296a94d0 0x562e29a6d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.032+0000 7efda67fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e296a94d0 0x562e29a6d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55220/0 (socket says 172.21.15.100:55220) 2024-07-26T13:44:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.032+0000 7efda67fc640 1 -- 172.21.15.100:0/3836203082 learned_addr learned my addr 172.21.15.100:0/3836203082 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.032+0000 7efda6ffd640 1 --2- 172.21.15.100:0/3836203082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e296a12f0 0x562e29a6cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.033+0000 7efda77fe640 1 --2- 172.21.15.100:0/3836203082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e296ca1b0 0x562e29a6c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.033+0000 7efda67fc640 1 -- 172.21.15.100:0/3836203082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e296a12f0 msgr2=0x562e29a6cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.033+0000 7efda67fc640 1 --2- 172.21.15.100:0/3836203082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e296a12f0 0x562e29a6cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.033+0000 7efda67fc640 1 -- 172.21.15.100:0/3836203082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e296ca1b0 msgr2=0x562e29a6c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.033+0000 7efda67fc640 1 --2- 172.21.15.100:0/3836203082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e296ca1b0 0x562e29a6c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.033+0000 7efda67fc640 1 -- 172.21.15.100:0/3836203082 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e299789a0 con 0x562e296a94d0 2024-07-26T13:44:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.033+0000 7efda6ffd640 1 --2- 172.21.15.100:0/3836203082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e296a12f0 0x562e29a6cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:44:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.033+0000 7efda77fe640 1 --2- 172.21.15.100:0/3836203082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e296ca1b0 0x562e29a6c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:44:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.033+0000 7efd8ffff640 1 --2- 172.21.15.100:0/477330550 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7efd900b3d40 0x7efd900b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:31.044 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.033+0000 7efda67fc640 1 --2- 172.21.15.100:0/3836203082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e296a94d0 0x562e29a6d130 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7efd9c101ed0 tx=0x7efd9c10acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625d7a6544a8bd56 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.033+0000 7efda77fe640 1 --2- 172.21.15.100:0/477330550 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7efd900b3d40 0x7efd900b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.033+0000 7efd8ffff640 1 -- 172.21.15.100:0/477330550 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7efd9c104840 con 0x7efda8107d10 2024-07-26T13:44:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.033+0000 7efd8d7fa640 1 -- 172.21.15.100:0/3836203082 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c0c72e0 con 0x562e296a94d0 2024-07-26T13:44:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.033+0000 7efd8d7fa640 1 -- 172.21.15.100:0/3836203082 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd9c0caf30 con 0x562e296a94d0 2024-07-26T13:44:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.033+0000 7efd8d7fa640 1 -- 172.21.15.100:0/3836203082 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c0cab90 con 0x562e296a94d0 2024-07-26T13:44:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.034+0000 7efdb4569740 1 -- 172.21.15.100:0/3836203082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e296a94d0 msgr2=0x562e29a6d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.034+0000 7efda77fe640 1 --2- 172.21.15.100:0/477330550 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7efd900b3d40 0x7efd900b6250 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7efda0005fd0 tx=0x7efda0005f60 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.034+0000 7efdb4569740 1 --2- 172.21.15.100:0/3836203082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e296a94d0 0x562e29a6d130 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7efd9c101ed0 tx=0x7efd9c10acd0 comp rx=0 tx=0).stop 2024-07-26T13:44:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.034+0000 7efdb4569740 1 -- 172.21.15.100:0/3836203082 shutdown_connections 2024-07-26T13:44:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.034+0000 7efdb4569740 1 --2- 172.21.15.100:0/3836203082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e296a94d0 0x562e29a6d130 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.034+0000 7efdb4569740 1 --2- 172.21.15.100:0/3836203082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e296a12f0 0x562e29a6cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.034+0000 7efdb4569740 1 --2- 172.21.15.100:0/3836203082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e296ca1b0 0x562e29a6c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:31.045 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.034+0000 7efdb4569740 1 -- 172.21.15.100:0/3836203082 >> 172.21.15.100:0/3836203082 conn(0x562e296be300 msgr2=0x562e29a6a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.034+0000 7efdb4569740 1 -- 172.21.15.100:0/3836203082 shutdown_connections 2024-07-26T13:44:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.034+0000 7efdb4569740 1 -- 172.21.15.100:0/3836203082 wait complete. 2024-07-26T13:44:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.035+0000 7efdb4569740 1 -- start start 2024-07-26T13:44:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.035+0000 7efdb4569740 1 Processor -- start 2024-07-26T13:44:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.035+0000 7efdb4569740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e296a12f0 0x562e29a78180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.035+0000 7efdb4569740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e296a94d0 0x562e29a786c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.035+0000 7efdb4569740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e296ca1b0 0x562e29a83e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.035+0000 7efdb4569740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x562e2990bab0 con 0x562e296a94d0 2024-07-26T13:44:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.035+0000 7efdb4569740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562e29959cf0 con 0x562e296ca1b0 2024-07-26T13:44:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.035+0000 7efdb4569740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562e29a7abf0 con 0x562e296a12f0 2024-07-26T13:44:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.035+0000 7efdb4569740 10 client.0 _ll_register_callbacks cb 0x562e299cb3e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:44:31.046 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.035+0000 7efda67fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e296a94d0 0x562e29a786c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.035+0000 7efda67fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e296a94d0 0x562e29a786c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55234/0 (socket says 172.21.15.100:55234) 2024-07-26T13:44:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.035+0000 7efda67fc640 1 -- 172.21.15.100:0/1627294793 learned_addr learned my addr 172.21.15.100:0/1627294793 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.035+0000 7efda77fe640 1 --2- 172.21.15.100:0/1627294793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e296ca1b0 0x562e29a83e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.035+0000 7efda6ffd640 1 --2- 172.21.15.100:0/1627294793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e296a12f0 0x562e29a78180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.035+0000 7efda67fc640 1 -- 172.21.15.100:0/1627294793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e296a12f0 msgr2=0x562e29a78180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.035+0000 7efda67fc640 1 --2- 172.21.15.100:0/1627294793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e296a12f0 0x562e29a78180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.035+0000 7efda67fc640 1 -- 172.21.15.100:0/1627294793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e296ca1b0 msgr2=0x562e29a83e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.035+0000 7efda67fc640 1 --2- 172.21.15.100:0/1627294793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e296ca1b0 0x562e29a83e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.035+0000 7efda67fc640 1 -- 172.21.15.100:0/1627294793 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e29735790 con 0x562e296a94d0 2024-07-26T13:44:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.035+0000 7efda77fe640 1 --2- 172.21.15.100:0/1627294793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e296ca1b0 0x562e29a83e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:44:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.036+0000 7efda67fc640 1 --2- 172.21.15.100:0/1627294793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e296a94d0 0x562e29a786c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7efd9c006010 tx=0x7efd9c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b93a7b1ff7ebc270 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.036+0000 7efd8aff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:44:31.047 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.036+0000 7efd8aff5640 1 -- 172.21.15.100:0/1627294793 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c049070 con 0x562e296a94d0 2024-07-26T13:44:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.036+0000 7efd8aff5640 1 -- 172.21.15.100:0/1627294793 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7efd9c03b4f0 con 0x562e296a94d0 2024-07-26T13:44:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.036+0000 7efdb4569740 1 -- 172.21.15.100:0/1627294793 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd9c0caf30 con 0x562e296a94d0 2024-07-26T13:44:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.036+0000 7efd8aff5640 1 -- 172.21.15.100:0/1627294793 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7efd9c0cad70 con 0x562e296a94d0 2024-07-26T13:44:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.036+0000 7efdb4569740 10 client.16656 fetch_fsmap learned FSMap version 37 2024-07-26T13:44:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.036+0000 7efdb4569740 1 -- 172.21.15.100:0/1627294793 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x562e297ae620 con 0x562e296a94d0 2024-07-26T13:44:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.036+0000 7efd8aff5640 1 -- 172.21.15.100:0/1627294793 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd9c0caf30 con 0x562e296a94d0 2024-07-26T13:44:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.036+0000 7efd8aff5640 1 -- 172.21.15.100:0/1627294793 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7efd9c0a2030 con 0x562e296a94d0 2024-07-26T13:44:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.036+0000 7efdb4569740 10 client.16656 fetch_fsmap finished waiting for FSMap version 37 2024-07-26T13:44:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.036+0000 7efdb4569740 10 client.16656 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:44:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.036+0000 7efdb4569740 20 client.16656 populate_metadata read hostname 'smithi100' 2024-07-26T13:44:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.036+0000 7efdb4569740 1 --2- 172.21.15.100:0/1627294793 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x562e29688d80 0x562e29a89c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.036+0000 7efdb4569740 4 client.16656 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:44:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.036+0000 7efdb4569740 1 -- 172.21.15.100:0/1627294793 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e299e4300 con 0x562e29688d80 2024-07-26T13:44:31.048 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.037+0000 7efda6ffd640 1 --2- 172.21.15.100:0/1627294793 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x562e29688d80 0x562e29a89c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.037+0000 7efd8aff5640 10 client.16656 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.037+0000 7efda6ffd640 1 --2- 172.21.15.100:0/1627294793 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x562e29688d80 0x562e29a89c10 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f996a0ceb9d83afd server_cookie=bd9e2f85c20e1e6d in_seq=0 out_seq=0 2024-07-26T13:44:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.038+0000 7efd8aff5640 1 -- 172.21.15.100:0/1627294793 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562e299e4300 con 0x562e29688d80 2024-07-26T13:44:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.038+0000 7efd8aff5640 10 client.16656 handle_command_reply: tid=0 2024-07-26T13:44:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.038+0000 7efdb4569740 1 client.16656 shutdown 2024-07-26T13:44:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.038+0000 7efdb4569740 1 -- 172.21.15.100:0/1627294793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e296a94d0 msgr2=0x562e29a786c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.038+0000 7efdb4569740 1 --2- 172.21.15.100:0/1627294793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e296a94d0 0x562e29a786c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7efd9c006010 tx=0x7efd9c0045e0 comp rx=0 tx=0).stop 2024-07-26T13:44:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.038+0000 7efdb4569740 1 -- 172.21.15.100:0/1627294793 shutdown_connections 2024-07-26T13:44:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.038+0000 7efdb4569740 1 --2- 172.21.15.100:0/1627294793 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x562e29688d80 0x562e29a89c10 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.038+0000 7efdb4569740 1 --2- 172.21.15.100:0/1627294793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e296ca1b0 0x562e29a83e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.038+0000 7efdb4569740 1 --2- 172.21.15.100:0/1627294793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e296a94d0 0x562e29a786c0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.038+0000 7efd8aff5640 0 client.16656 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.038+0000 7efdb4569740 1 --2- 172.21.15.100:0/1627294793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e296a12f0 0x562e29a78180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.038+0000 7efdb4569740 1 -- 172.21.15.100:0/1627294793 >> 172.21.15.100:0/1627294793 conn(0x562e296be300 msgr2=0x562e29a6d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.038+0000 7efdb4569740 1 -- 172.21.15.100:0/1627294793 shutdown_connections 2024-07-26T13:44:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.038+0000 7efdb4569740 1 -- 172.21.15.100:0/1627294793 wait complete. 2024-07-26T13:44:31.050 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.038+0000 7efdb4569740 20 client.16656 trim_cache size 0 max 16384 2024-07-26T13:44:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.053+0000 7efdb4569740 1 Processor -- start 2024-07-26T13:44:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.053+0000 7efdb4569740 1 -- start start 2024-07-26T13:44:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.054+0000 7efdb4569740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e296a12f0 0x562e29a72780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.054+0000 7efdb4569740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e296a94d0 0x562e29a6ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:31.058 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.054+0000 7efdb4569740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e296ca1b0 0x562e29a83c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:31.058 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.054+0000 7efdb4569740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efda81b3ca0 con 0x562e296a12f0 2024-07-26T13:44:31.058 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.054+0000 7efdb4569740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efda81b1510 con 0x562e296ca1b0 2024-07-26T13:44:31.058 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.054+0000 7efdb4569740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efda81af360 con 0x562e296a94d0 2024-07-26T13:44:31.058 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.054+0000 7efda6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e296a12f0 0x562e29a72780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:31.058 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.054+0000 7efda6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e296a12f0 0x562e29a72780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55242/0 (socket says 172.21.15.100:55242) 2024-07-26T13:44:31.058 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.054+0000 7efda6ffd640 1 -- 172.21.15.100:0/3324724987 learned_addr learned my addr 172.21.15.100:0/3324724987 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:31.058 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.054+0000 7efda67fc640 1 --2- 172.21.15.100:0/3324724987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e296a94d0 0x562e29a6ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:31.058 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.054+0000 7efda77fe640 1 --2- 172.21.15.100:0/3324724987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e296ca1b0 0x562e29a83c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:31.058 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.054+0000 7efda6ffd640 1 -- 172.21.15.100:0/3324724987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e296a94d0 msgr2=0x562e29a6ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:31.058 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.054+0000 7efda6ffd640 1 --2- 172.21.15.100:0/3324724987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e296a94d0 0x562e29a6ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:31.058 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.054+0000 7efda6ffd640 1 -- 172.21.15.100:0/3324724987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e296ca1b0 msgr2=0x562e29a83c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:31.058 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.054+0000 7efda6ffd640 1 --2- 172.21.15.100:0/3324724987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e296ca1b0 0x562e29a83c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.054+0000 7efda6ffd640 1 -- 172.21.15.100:0/3324724987 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e29a78a30 con 0x562e296a12f0 2024-07-26T13:44:31.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.054+0000 7efda67fc640 1 --2- 172.21.15.100:0/3324724987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e296a94d0 0x562e29a6ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:44:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.054+0000 7efda6ffd640 1 --2- 172.21.15.100:0/3324724987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e296a12f0 0x562e29a72780 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7efd98013570 tx=0x7efd98014a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc43ebd52d54411d server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.054+0000 7efd8cff9640 1 -- 172.21.15.100:0/3324724987 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd98024030 con 0x562e296a12f0 2024-07-26T13:44:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.054+0000 7efd8cff9640 1 -- 172.21.15.100:0/3324724987 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd98003e50 con 0x562e296a12f0 2024-07-26T13:44:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.054+0000 7efd8cff9640 1 -- 172.21.15.100:0/3324724987 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd980204a0 con 0x562e296a12f0 2024-07-26T13:44:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.055+0000 7efdb4569740 1 -- 172.21.15.100:0/3324724987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e296a12f0 msgr2=0x562e29a72780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.055+0000 7efdb4569740 1 --2- 172.21.15.100:0/3324724987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e296a12f0 0x562e29a72780 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7efd98013570 tx=0x7efd98014a60 comp rx=0 tx=0).stop 2024-07-26T13:44:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.055+0000 7efdb4569740 1 -- 172.21.15.100:0/3324724987 shutdown_connections 2024-07-26T13:44:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.055+0000 7efdb4569740 1 --2- 172.21.15.100:0/3324724987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e296ca1b0 0x562e29a83c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.055+0000 7efdb4569740 1 --2- 172.21.15.100:0/3324724987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e296a94d0 0x562e29a6ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:31.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.055+0000 7efdb4569740 1 --2- 172.21.15.100:0/3324724987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e296a12f0 0x562e29a72780 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.055+0000 7efdb4569740 1 -- 172.21.15.100:0/3324724987 >> 172.21.15.100:0/3324724987 conn(0x562e296be300 msgr2=0x562e29a6d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.055+0000 7efdb4569740 1 -- 172.21.15.100:0/3324724987 shutdown_connections 2024-07-26T13:44:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.055+0000 7efdb4569740 1 -- 172.21.15.100:0/3324724987 wait complete. 2024-07-26T13:44:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.055+0000 7efdb4569740 1 -- start start 2024-07-26T13:44:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.056+0000 7efdb4569740 1 Processor -- start 2024-07-26T13:44:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.056+0000 7efdb4569740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e296a12f0 0x562e29a7cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.056+0000 7efdb4569740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e296a94d0 0x562e29a7d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.056+0000 7efdb4569740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e296ca1b0 0x562e29a7d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.056+0000 7efdb4569740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efda81af9d0 con 0x562e296a94d0 2024-07-26T13:44:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.056+0000 7efdb4569740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efda81b0e40 con 0x562e296ca1b0 2024-07-26T13:44:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.056+0000 7efdb4569740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efda81b00a0 con 0x562e296a12f0 2024-07-26T13:44:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.056+0000 7efdb4569740 10 client.0 _ll_register_callbacks cb 0x7efd98009a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:44:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.056+0000 7efda67fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e296a94d0 0x562e29a7d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.056+0000 7efda67fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e296a94d0 0x562e29a7d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:55250/0 (socket says 172.21.15.100:55250) 2024-07-26T13:44:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.056+0000 7efda67fc640 1 -- 172.21.15.100:0/4176316235 learned_addr learned my addr 172.21.15.100:0/4176316235 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.056+0000 7efda77fe640 1 --2- 172.21.15.100:0/4176316235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e296ca1b0 0x562e29a7d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.056+0000 7efda67fc640 1 -- 172.21.15.100:0/4176316235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e296a12f0 msgr2=0x562e29a7cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.056+0000 7efda6ffd640 1 --2- 172.21.15.100:0/4176316235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e296a12f0 0x562e29a7cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.056+0000 7efda67fc640 1 --2- 172.21.15.100:0/4176316235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e296a12f0 0x562e29a7cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:31.070 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.056+0000 7efda67fc640 1 -- 172.21.15.100:0/4176316235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e296ca1b0 msgr2=0x562e29a7d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.056+0000 7efda67fc640 1 --2- 172.21.15.100:0/4176316235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e296ca1b0 0x562e29a7d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.056+0000 7efda67fc640 1 -- 172.21.15.100:0/4176316235 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e29af04a0 con 0x562e296a94d0 2024-07-26T13:44:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.056+0000 7efda77fe640 1 --2- 172.21.15.100:0/4176316235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e296ca1b0 0x562e29a7d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:44:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.056+0000 7efda6ffd640 1 --2- 172.21.15.100:0/4176316235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e296a12f0 0x562e29a7cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:44:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.056+0000 7efda67fc640 1 --2- 172.21.15.100:0/4176316235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e296a94d0 0x562e29a7d2c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0cb5e0 tx=0x7efd9c03b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf392610c5304b8 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:31.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.056+0000 7efd8aff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:44:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.057+0000 7efd8aff5640 1 -- 172.21.15.100:0/4176316235 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c049070 con 0x562e296a94d0 2024-07-26T13:44:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.057+0000 7efd8aff5640 1 -- 172.21.15.100:0/4176316235 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7efd9c10ae00 con 0x562e296a94d0 2024-07-26T13:44:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.057+0000 7efdb4569740 1 -- 172.21.15.100:0/4176316235 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd98003e50 con 0x562e296a94d0 2024-07-26T13:44:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.057+0000 7efd8aff5640 1 -- 172.21.15.100:0/4176316235 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7efd9c0cb1f0 con 0x562e296a94d0 2024-07-26T13:44:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.057+0000 7efdb4569740 10 client.16662 fetch_fsmap learned FSMap version 37 2024-07-26T13:44:31.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.057+0000 7efdb4569740 1 -- 172.21.15.100:0/4176316235 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x562e29af04a0 con 0x562e296a94d0 2024-07-26T13:44:31.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.057+0000 7efd8aff5640 1 -- 172.21.15.100:0/4176316235 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd98003e50 con 0x562e296a94d0 2024-07-26T13:44:31.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.057+0000 7efd8aff5640 1 -- 172.21.15.100:0/4176316235 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7efd9c0a2030 con 0x562e296a94d0 2024-07-26T13:44:31.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.057+0000 7efdb4569740 10 client.16662 fetch_fsmap finished waiting for FSMap version 37 2024-07-26T13:44:31.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.057+0000 7efdb4569740 10 client.16662 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:44:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.057+0000 7efdb4569740 20 client.16662 populate_metadata read hostname 'smithi100' 2024-07-26T13:44:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.057+0000 7efdb4569740 1 --2- 172.21.15.100:0/4176316235 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x562e2965e1c0 0x562e29af3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.057+0000 7efdb4569740 4 client.16662 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-26T13:44:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.057+0000 7efdb4569740 1 -- 172.21.15.100:0/4176316235 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562e299dfda0 con 0x562e2965e1c0 2024-07-26T13:44:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.058+0000 7efda6ffd640 1 --2- 172.21.15.100:0/4176316235 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x562e2965e1c0 0x562e29af3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:31.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.058+0000 7efda6ffd640 1 --2- 172.21.15.100:0/4176316235 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x562e2965e1c0 0x562e29af3a70 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b033ff458aab1c85 server_cookie=4df3bfd643a14c6a in_seq=0 out_seq=0 2024-07-26T13:44:31.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.058+0000 7efd8aff5640 10 client.16662 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:31.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.084+0000 7efd8aff5640 1 -- 172.21.15.100:0/4176316235 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562e299dfda0 con 0x562e2965e1c0 2024-07-26T13:44:31.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.084+0000 7efd8aff5640 10 client.16662 handle_command_reply: tid=0 2024-07-26T13:44:31.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.085+0000 7efdb4569740 1 client.16662 shutdown 2024-07-26T13:44:31.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.085+0000 7efdb4569740 1 -- 172.21.15.100:0/4176316235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e296a94d0 msgr2=0x562e29a7d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.085+0000 7efdb4569740 1 --2- 172.21.15.100:0/4176316235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e296a94d0 0x562e29a7d2c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0cb5e0 tx=0x7efd9c03b460 comp rx=0 tx=0).stop 2024-07-26T13:44:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.085+0000 7efdb4569740 1 -- 172.21.15.100:0/4176316235 shutdown_connections 2024-07-26T13:44:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.085+0000 7efdb4569740 1 --2- 172.21.15.100:0/4176316235 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x562e2965e1c0 0x562e29af3a70 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.085+0000 7efdb4569740 1 --2- 172.21.15.100:0/4176316235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562e296ca1b0 0x562e29a7d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.085+0000 7efdb4569740 1 --2- 172.21.15.100:0/4176316235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562e296a94d0 0x562e29a7d2c0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.085+0000 7efd8aff5640 0 client.16662 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.085+0000 7efdb4569740 1 --2- 172.21.15.100:0/4176316235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562e296a12f0 0x562e29a7cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.085+0000 7efdb4569740 1 -- 172.21.15.100:0/4176316235 >> 172.21.15.100:0/4176316235 conn(0x562e296be300 msgr2=0x562e29a68c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.085+0000 7efdb4569740 1 -- 172.21.15.100:0/4176316235 shutdown_connections 2024-07-26T13:44:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.085+0000 7efdb4569740 1 -- 172.21.15.100:0/4176316235 wait complete. 2024-07-26T13:44:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.085+0000 7efdb4569740 20 client.16662 trim_cache size 0 max 16384 2024-07-26T13:44:31.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-07-26T13:44:31.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-07-26T13:44:31.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-07-26T13:44:31.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-07-26T13:44:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.086+0000 7efdad577640 1 -- 172.21.15.100:0/477330550 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7efd900b3d40 msgr2=0x7efd900b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.086+0000 7efdad577640 1 --2- 172.21.15.100:0/477330550 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7efd900b3d40 0x7efd900b6250 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7efda0005fd0 tx=0x7efda0005f60 comp rx=0 tx=0).stop 2024-07-26T13:44:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.086+0000 7efdad577640 1 -- 172.21.15.100:0/477330550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efda8107d10 msgr2=0x7efda8199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.086+0000 7efdad577640 1 --2- 172.21.15.100:0/477330550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efda8107d10 0x7efda8199080 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7efd9c005ab0 tx=0x7efd9c003cd0 comp rx=0 tx=0).stop 2024-07-26T13:44:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.086+0000 7efdad577640 1 -- 172.21.15.100:0/477330550 shutdown_connections 2024-07-26T13:44:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.086+0000 7efdad577640 1 --2- 172.21.15.100:0/477330550 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7efd900b3d40 0x7efd900b6250 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.086+0000 7efdad577640 1 --2- 172.21.15.100:0/477330550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efda8108660 0x7efda81995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.086+0000 7efdad577640 1 --2- 172.21.15.100:0/477330550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efda8107d10 0x7efda8199080 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.086+0000 7efdad577640 1 --2- 172.21.15.100:0/477330550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efda81023e0 0x7efda8198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.086+0000 7efdad577640 1 -- 172.21.15.100:0/477330550 >> 172.21.15.100:0/477330550 conn(0x7efda807a170 msgr2=0x7efda8103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:31.089 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.086+0000 7efdad577640 1 -- 172.21.15.100:0/477330550 shutdown_connections 2024-07-26T13:44:31.090 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:31.086+0000 7efdad577640 1 -- 172.21.15.100:0/477330550 wait complete. 2024-07-26T13:44:31.413 INFO:tasks.workunit.client.0.smithi100.stdout: 1 155296 1.09 MB/sec execute 45 sec latency 6137.852 ms 2024-07-26T13:44:31.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:31 smithi138 ceph-mon[36954]: pgmap v497: 97 pgs: 97 active+clean; 64 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.1 MiB/s wr, 144 op/s 2024-07-26T13:44:31.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:31 smithi100 ceph-mon[32599]: pgmap v497: 97 pgs: 97 active+clean; 64 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.1 MiB/s wr, 144 op/s 2024-07-26T13:44:32.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:31 smithi167 ceph-mon[37140]: pgmap v497: 97 pgs: 97 active+clean; 64 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.1 MiB/s wr, 144 op/s 2024-07-26T13:44:32.413 INFO:tasks.workunit.client.0.smithi100.stdout: 1 155296 1.07 MB/sec execute 46 sec latency 7137.908 ms 2024-07-26T13:44:33.413 INFO:tasks.workunit.client.0.smithi100.stdout: 1 155296 1.05 MB/sec execute 47 sec latency 8137.990 ms 2024-07-26T13:44:33.814 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:33 smithi100 ceph-mon[32599]: pgmap v498: 97 pgs: 97 active+clean; 64 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 760 KiB/s wr, 129 op/s 2024-07-26T13:44:33.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:33 smithi138 ceph-mon[36954]: pgmap v498: 97 pgs: 97 active+clean; 64 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 760 KiB/s wr, 129 op/s 2024-07-26T13:44:34.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:33 smithi167 ceph-mon[37140]: pgmap v498: 97 pgs: 97 active+clean; 64 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 760 KiB/s wr, 129 op/s 2024-07-26T13:44:34.413 INFO:tasks.workunit.client.0.smithi100.stdout: 1 155296 1.03 MB/sec execute 48 sec latency 9138.058 ms 2024-07-26T13:44:35.413 INFO:tasks.workunit.client.0.smithi100.stdout: 1 155296 1.00 MB/sec execute 49 sec latency 10138.139 ms 2024-07-26T13:44:35.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:35 smithi138 ceph-mon[36954]: pgmap v499: 97 pgs: 97 active+clean; 60 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 705 KiB/s wr, 128 op/s 2024-07-26T13:44:35.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:35 smithi100 ceph-mon[32599]: pgmap v499: 97 pgs: 97 active+clean; 60 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 705 KiB/s wr, 128 op/s 2024-07-26T13:44:36.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:35 smithi167 ceph-mon[37140]: pgmap v499: 97 pgs: 97 active+clean; 60 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 705 KiB/s wr, 128 op/s 2024-07-26T13:44:36.413 INFO:tasks.workunit.client.0.smithi100.stdout: 1 155296 0.98 MB/sec execute 50 sec latency 11138.214 ms 2024-07-26T13:44:37.413 INFO:tasks.workunit.client.0.smithi100.stdout: 1 155296 0.97 MB/sec execute 51 sec latency 12138.271 ms 2024-07-26T13:44:37.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:37 smithi138 ceph-mon[36954]: pgmap v500: 97 pgs: 97 active+clean; 61 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 473 KiB/s wr, 85 op/s 2024-07-26T13:44:37.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:37 smithi100 ceph-mon[32599]: pgmap v500: 97 pgs: 97 active+clean; 61 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 473 KiB/s wr, 85 op/s 2024-07-26T13:44:38.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:37 smithi167 ceph-mon[37140]: pgmap v500: 97 pgs: 97 active+clean; 61 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 473 KiB/s wr, 85 op/s 2024-07-26T13:44:38.413 INFO:tasks.workunit.client.0.smithi100.stdout: 1 155296 0.95 MB/sec execute 52 sec latency 13138.360 ms 2024-07-26T13:44:39.413 INFO:tasks.workunit.client.0.smithi100.stdout: 1 155842 0.99 MB/sec execute 53 sec latency 13906.700 ms 2024-07-26T13:44:39.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:39 smithi138 ceph-mon[36954]: pgmap v501: 97 pgs: 97 active+clean; 61 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 302 KiB/s wr, 55 op/s 2024-07-26T13:44:39.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:39 smithi100 ceph-mon[32599]: pgmap v501: 97 pgs: 97 active+clean; 61 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 302 KiB/s wr, 55 op/s 2024-07-26T13:44:40.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:39 smithi167 ceph-mon[37140]: pgmap v501: 97 pgs: 97 active+clean; 61 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 302 KiB/s wr, 55 op/s 2024-07-26T13:44:40.413 INFO:tasks.workunit.client.0.smithi100.stdout: 1 157185 1.10 MB/sec execute 54 sec latency 79.981 ms 2024-07-26T13:44:40.690 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: ceph-mds: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-5320-ge8f88bb1/rpm/el9/BUILD/ceph-19.0.0-5320-ge8f88bb1/redhat-linux-build/boost/include/boost/smart_ptr/intrusive_ptr.hpp:201: T* boost::intrusive_ptr::operator->() const [with T = const Message]: Assertion `px != 0' failed. 2024-07-26T13:44:40.691 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: *** Caught signal (Aborted) ** 2024-07-26T13:44:40.691 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: in thread 7f1205344640 thread_name:ms_dispatch 2024-07-26T13:44:40.691 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: ceph version 19.0.0-5320-ge8f88bb1 (e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528) squid (dev) 2024-07-26T13:44:40.691 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f120a71b6f0] 2024-07-26T13:44:40.691 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 2: /lib64/libc.so.6(+0x8b94c) [0x7f120a76894c] 2024-07-26T13:44:40.691 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 3: raise() 2024-07-26T13:44:40.691 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 4: abort() 2024-07-26T13:44:40.691 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 5: /lib64/libc.so.6(+0x2871b) [0x7f120a70571b] 2024-07-26T13:44:40.691 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 6: /lib64/libc.so.6(+0x37386) [0x7f120a714386] 2024-07-26T13:44:40.691 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 7: (MDSRank::get_session(boost::intrusive_ptr const&)+0x89) [0x563deaceb6b9] 2024-07-26T13:44:40.691 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 8: (MDCache::path_traverse(boost::intrusive_ptr const&, ContextFactory&, filepath const&, int, std::vector >*, CInode**, CDir**)+0x58e) [0x563deaea980c] 2024-07-26T13:44:40.691 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 9: (Server::handle_peer_rename_prep(boost::intrusive_ptr const&)+0x4fe) [0x563deadf7cde] 2024-07-26T13:44:40.692 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 10: (Server::dispatch_peer_request(boost::intrusive_ptr const&)+0x933) [0x563deadb9763] 2024-07-26T13:44:40.692 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 11: (Server::handle_peer_request(boost::intrusive_ptr const&)+0x10c3) [0x563deadbaa1b] 2024-07-26T13:44:40.692 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x9ba) [0x563dead9e752] 2024-07-26T13:44:40.692 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 13: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x81a) [0x563deaceca56] 2024-07-26T13:44:40.692 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 14: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x563deacf0637] 2024-07-26T13:44:40.692 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 15: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x563deacf0f5e] 2024-07-26T13:44:40.692 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 16: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x563deacd403a] 2024-07-26T13:44:40.692 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 17: /usr/lib64/ceph/libceph-common.so.2(+0x39b407) [0x7f120afe8407] 2024-07-26T13:44:40.692 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 18: (DispatchQueue::entry()+0x60f) [0x7f120afe8cf9] 2024-07-26T13:44:40.692 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 19: /usr/lib64/ceph/libceph-common.so.2(+0x4785a1) [0x7f120b0c55a1] 2024-07-26T13:44:40.693 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 20: (Thread::entry_wrapper()+0x43) [0x7f120ae72cdd] 2024-07-26T13:44:40.693 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 21: (Thread::_entry_func(void*)+0xd) [0x7f120ae72cf9] 2024-07-26T13:44:40.693 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 22: /lib64/libc.so.6(+0x89c02) [0x7f120a766c02] 2024-07-26T13:44:40.693 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 23: /lib64/libc.so.6(+0x10ec40) [0x7f120a7ebc40] 2024-07-26T13:44:40.693 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 2024-07-26T13:44:40.338+0000 7f1205344640 -1 *** Caught signal (Aborted) ** 2024-07-26T13:44:40.693 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: in thread 7f1205344640 thread_name:ms_dispatch 2024-07-26T13:44:40.694 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 2024-07-26T13:44:40.694 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: ceph version 19.0.0-5320-ge8f88bb1 (e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528) squid (dev) 2024-07-26T13:44:40.694 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f120a71b6f0] 2024-07-26T13:44:40.694 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 2: /lib64/libc.so.6(+0x8b94c) [0x7f120a76894c] 2024-07-26T13:44:40.694 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 3: raise() 2024-07-26T13:44:40.694 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 4: abort() 2024-07-26T13:44:40.694 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 5: /lib64/libc.so.6(+0x2871b) [0x7f120a70571b] 2024-07-26T13:44:40.694 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 6: /lib64/libc.so.6(+0x37386) [0x7f120a714386] 2024-07-26T13:44:40.694 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 7: (MDSRank::get_session(boost::intrusive_ptr const&)+0x89) [0x563deaceb6b9] 2024-07-26T13:44:40.694 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 8: (MDCache::path_traverse(boost::intrusive_ptr const&, ContextFactory&, filepath const&, int, std::vector >*, CInode**, CDir**)+0x58e) [0x563deaea980c] 2024-07-26T13:44:40.694 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 9: (Server::handle_peer_rename_prep(boost::intrusive_ptr const&)+0x4fe) [0x563deadf7cde] 2024-07-26T13:44:40.695 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 10: (Server::dispatch_peer_request(boost::intrusive_ptr const&)+0x933) [0x563deadb9763] 2024-07-26T13:44:40.695 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 11: (Server::handle_peer_request(boost::intrusive_ptr const&)+0x10c3) [0x563deadbaa1b] 2024-07-26T13:44:40.695 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x9ba) [0x563dead9e752] 2024-07-26T13:44:40.695 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 13: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x81a) [0x563deaceca56] 2024-07-26T13:44:40.695 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 14: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x563deacf0637] 2024-07-26T13:44:40.695 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 15: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x563deacf0f5e] 2024-07-26T13:44:40.695 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 16: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x563deacd403a] 2024-07-26T13:44:40.695 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 17: /usr/lib64/ceph/libceph-common.so.2(+0x39b407) [0x7f120afe8407] 2024-07-26T13:44:40.695 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 18: (DispatchQueue::entry()+0x60f) [0x7f120afe8cf9] 2024-07-26T13:44:40.695 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 19: /usr/lib64/ceph/libceph-common.so.2(+0x4785a1) [0x7f120b0c55a1] 2024-07-26T13:44:40.695 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 20: (Thread::entry_wrapper()+0x43) [0x7f120ae72cdd] 2024-07-26T13:44:40.695 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 21: (Thread::_entry_func(void*)+0xd) [0x7f120ae72cf9] 2024-07-26T13:44:40.695 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 22: /lib64/libc.so.6(+0x89c02) [0x7f120a766c02] 2024-07-26T13:44:40.696 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 23: /lib64/libc.so.6(+0x10ec40) [0x7f120a7ebc40] 2024-07-26T13:44:40.696 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-07-26T13:44:40.696 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 2024-07-26T13:44:40.696 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 0> 2024-07-26T13:44:40.338+0000 7f1205344640 -1 *** Caught signal (Aborted) ** 2024-07-26T13:44:40.696 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: in thread 7f1205344640 thread_name:ms_dispatch 2024-07-26T13:44:40.696 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 2024-07-26T13:44:40.696 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: ceph version 19.0.0-5320-ge8f88bb1 (e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528) squid (dev) 2024-07-26T13:44:40.696 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f120a71b6f0] 2024-07-26T13:44:40.696 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 2: /lib64/libc.so.6(+0x8b94c) [0x7f120a76894c] 2024-07-26T13:44:40.696 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 3: raise() 2024-07-26T13:44:40.696 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 4: abort() 2024-07-26T13:44:40.696 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 5: /lib64/libc.so.6(+0x2871b) [0x7f120a70571b] 2024-07-26T13:44:40.696 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 6: /lib64/libc.so.6(+0x37386) [0x7f120a714386] 2024-07-26T13:44:40.696 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 7: (MDSRank::get_session(boost::intrusive_ptr const&)+0x89) [0x563deaceb6b9] 2024-07-26T13:44:40.697 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 8: (MDCache::path_traverse(boost::intrusive_ptr const&, ContextFactory&, filepath const&, int, std::vector >*, CInode**, CDir**)+0x58e) [0x563deaea980c] 2024-07-26T13:44:40.697 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 9: (Server::handle_peer_rename_prep(boost::intrusive_ptr const&)+0x4fe) [0x563deadf7cde] 2024-07-26T13:44:40.697 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 10: (Server::dispatch_peer_request(boost::intrusive_ptr const&)+0x933) [0x563deadb9763] 2024-07-26T13:44:40.697 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 11: (Server::handle_peer_request(boost::intrusive_ptr const&)+0x10c3) [0x563deadbaa1b] 2024-07-26T13:44:40.697 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x9ba) [0x563dead9e752] 2024-07-26T13:44:40.697 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 13: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x81a) [0x563deaceca56] 2024-07-26T13:44:40.697 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 14: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x563deacf0637] 2024-07-26T13:44:40.697 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 15: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x563deacf0f5e] 2024-07-26T13:44:40.697 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 16: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x563deacd403a] 2024-07-26T13:44:40.697 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 17: /usr/lib64/ceph/libceph-common.so.2(+0x39b407) [0x7f120afe8407] 2024-07-26T13:44:40.697 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 18: (DispatchQueue::entry()+0x60f) [0x7f120afe8cf9] 2024-07-26T13:44:40.697 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 19: /usr/lib64/ceph/libceph-common.so.2(+0x4785a1) [0x7f120b0c55a1] 2024-07-26T13:44:40.697 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 20: (Thread::entry_wrapper()+0x43) [0x7f120ae72cdd] 2024-07-26T13:44:40.698 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 21: (Thread::_entry_func(void*)+0xd) [0x7f120ae72cf9] 2024-07-26T13:44:40.698 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 22: /lib64/libc.so.6(+0x89c02) [0x7f120a766c02] 2024-07-26T13:44:40.698 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 23: /lib64/libc.so.6(+0x10ec40) [0x7f120a7ebc40] 2024-07-26T13:44:40.698 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-07-26T13:44:40.698 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 2024-07-26T13:44:40.698 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 0> 2024-07-26T13:44:40.338+0000 7f1205344640 -1 *** Caught signal (Aborted) ** 2024-07-26T13:44:40.698 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: in thread 7f1205344640 thread_name:ms_dispatch 2024-07-26T13:44:40.698 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 2024-07-26T13:44:40.698 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: ceph version 19.0.0-5320-ge8f88bb1 (e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528) squid (dev) 2024-07-26T13:44:40.698 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f120a71b6f0] 2024-07-26T13:44:40.698 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 2: /lib64/libc.so.6(+0x8b94c) [0x7f120a76894c] 2024-07-26T13:44:40.698 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 3: raise() 2024-07-26T13:44:40.698 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 4: abort() 2024-07-26T13:44:40.699 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 5: /lib64/libc.so.6(+0x2871b) [0x7f120a70571b] 2024-07-26T13:44:40.699 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 6: /lib64/libc.so.6(+0x37386) [0x7f120a714386] 2024-07-26T13:44:40.699 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 7: (MDSRank::get_session(boost::intrusive_ptr const&)+0x89) [0x563deaceb6b9] 2024-07-26T13:44:40.699 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 8: (MDCache::path_traverse(boost::intrusive_ptr const&, ContextFactory&, filepath const&, int, std::vector >*, CInode**, CDir**)+0x58e) [0x563deaea980c] 2024-07-26T13:44:40.699 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 9: (Server::handle_peer_rename_prep(boost::intrusive_ptr const&)+0x4fe) [0x563deadf7cde] 2024-07-26T13:44:40.699 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 10: (Server::dispatch_peer_request(boost::intrusive_ptr const&)+0x933) [0x563deadb9763] 2024-07-26T13:44:40.699 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 11: (Server::handle_peer_request(boost::intrusive_ptr const&)+0x10c3) [0x563deadbaa1b] 2024-07-26T13:44:40.699 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x9ba) [0x563dead9e752] 2024-07-26T13:44:40.699 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 13: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x81a) [0x563deaceca56] 2024-07-26T13:44:40.699 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 14: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x563deacf0637] 2024-07-26T13:44:40.699 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 15: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x563deacf0f5e] 2024-07-26T13:44:40.699 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 16: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x563deacd403a] 2024-07-26T13:44:40.699 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 17: /usr/lib64/ceph/libceph-common.so.2(+0x39b407) [0x7f120afe8407] 2024-07-26T13:44:40.699 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 18: (DispatchQueue::entry()+0x60f) [0x7f120afe8cf9] 2024-07-26T13:44:40.700 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 19: /usr/lib64/ceph/libceph-common.so.2(+0x4785a1) [0x7f120b0c55a1] 2024-07-26T13:44:40.700 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 20: (Thread::entry_wrapper()+0x43) [0x7f120ae72cdd] 2024-07-26T13:44:40.700 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 21: (Thread::_entry_func(void*)+0xd) [0x7f120ae72cf9] 2024-07-26T13:44:40.700 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 22: /lib64/libc.so.6(+0x89c02) [0x7f120a766c02] 2024-07-26T13:44:40.700 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 23: /lib64/libc.so.6(+0x10ec40) [0x7f120a7ebc40] 2024-07-26T13:44:40.700 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-07-26T13:44:40.700 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[67887]: 2024-07-26T13:44:40.700 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:40 smithi138 podman[71144]: 2024-07-26 13:44:40.392904663 +0000 UTC m=+0.019864818 container died b5b2b94fbed0867785f37664d33cbe2a315f35ff07bc26dd513dde8838e24ef9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-26T13:44:41.414 INFO:tasks.workunit.client.0.smithi100.stdout: 1 157185 1.08 MB/sec execute 55 sec latency 1080.059 ms 2024-07-26T13:44:41.482 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:41 smithi138 podman[71144]: 2024-07-26 13:44:41.227000449 +0000 UTC m=+0.853960596 container remove b5b2b94fbed0867785f37664d33cbe2a315f35ff07bc26dd513dde8838e24ef9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb) 2024-07-26T13:44:41.482 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:41 smithi138 systemd[1]: ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.e.service: Main process exited, code=exited, status=134/n/a 2024-07-26T13:44:41.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:41 smithi138 ceph-mon[36954]: pgmap v502: 97 pgs: 97 active+clean; 62 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 242 KiB/s wr, 43 op/s 2024-07-26T13:44:41.940 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:41 smithi138 systemd[1]: ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.e.service: Failed with result 'exit-code'. 2024-07-26T13:44:41.940 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:41 smithi138 systemd[1]: ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.e.service: Consumed 16.505s CPU time. 2024-07-26T13:44:41.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:41 smithi100 ceph-mon[32599]: pgmap v502: 97 pgs: 97 active+clean; 62 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 242 KiB/s wr, 43 op/s 2024-07-26T13:44:42.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:41 smithi167 ceph-mon[37140]: pgmap v502: 97 pgs: 97 active+clean; 62 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 242 KiB/s wr, 43 op/s 2024-07-26T13:44:42.414 INFO:tasks.workunit.client.0.smithi100.stdout: 1 157185 1.06 MB/sec execute 56 sec latency 2080.161 ms 2024-07-26T13:44:43.414 INFO:tasks.workunit.client.0.smithi100.stdout: 1 157185 1.04 MB/sec execute 57 sec latency 3080.263 ms 2024-07-26T13:44:43.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:43 smithi100 ceph-mon[32599]: pgmap v503: 97 pgs: 97 active+clean; 63 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 225 KiB/s wr, 38 op/s 2024-07-26T13:44:43.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:43 smithi138 ceph-mon[36954]: pgmap v503: 97 pgs: 97 active+clean; 63 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 225 KiB/s wr, 38 op/s 2024-07-26T13:44:44.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:43 smithi167 ceph-mon[37140]: pgmap v503: 97 pgs: 97 active+clean; 63 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 225 KiB/s wr, 38 op/s 2024-07-26T13:44:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.193+0000 7ff4d54d3640 1 -- 172.21.15.100:0/368586519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4d0105c30 msgr2=0x7ff4d0106040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.193+0000 7ff4d54d3640 1 --2- 172.21.15.100:0/368586519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4d0105c30 0x7ff4d0106040 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8009900 tx=0x7ff4b802a410 comp rx=0 tx=0).stop 2024-07-26T13:44:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.193+0000 7ff4d54d3640 1 -- 172.21.15.100:0/368586519 shutdown_connections 2024-07-26T13:44:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.193+0000 7ff4d54d3640 1 --2- 172.21.15.100:0/368586519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4d0106580 0x7ff4d010fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.193+0000 7ff4d54d3640 1 --2- 172.21.15.100:0/368586519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4d0105c30 0x7ff4d0106040 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.193+0000 7ff4d54d3640 1 --2- 172.21.15.100:0/368586519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4d0101ac0 0x7ff4d0101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.193+0000 7ff4d54d3640 1 -- 172.21.15.100:0/368586519 >> 172.21.15.100:0/368586519 conn(0x7ff4d00798f0 msgr2=0x7ff4d0079d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.193+0000 7ff4d54d3640 1 -- 172.21.15.100:0/368586519 shutdown_connections 2024-07-26T13:44:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.193+0000 7ff4d54d3640 1 -- 172.21.15.100:0/368586519 wait complete. 2024-07-26T13:44:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.194+0000 7ff4d54d3640 1 Processor -- start 2024-07-26T13:44:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.194+0000 7ff4d54d3640 1 -- start start 2024-07-26T13:44:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.194+0000 7ff4d54d3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4d0101ac0 0x7ff4d0199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.194+0000 7ff4d54d3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4d0105c30 0x7ff4d019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.194+0000 7ff4d54d3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4d0106580 0x7ff4d019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.194+0000 7ff4d54d3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d01133f0 con 0x7ff4d0101ac0 2024-07-26T13:44:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.194+0000 7ff4d54d3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d0113270 con 0x7ff4d0106580 2024-07-26T13:44:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.194+0000 7ff4d54d3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d0113570 con 0x7ff4d0105c30 2024-07-26T13:44:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.194+0000 7ff4ceffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4d0101ac0 0x7ff4d0199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.194+0000 7ff4ceffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4d0101ac0 0x7ff4d0199e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35710/0 (socket says 172.21.15.100:35710) 2024-07-26T13:44:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.194+0000 7ff4ceffd640 1 -- 172.21.15.100:0/879316271 learned_addr learned my addr 172.21.15.100:0/879316271 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.194+0000 7ff4ce7fc640 1 --2- 172.21.15.100:0/879316271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4d0105c30 0x7ff4d019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.194+0000 7ff4cf7fe640 1 --2- 172.21.15.100:0/879316271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4d0106580 0x7ff4d019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.195+0000 7ff4ceffd640 1 -- 172.21.15.100:0/879316271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4d0105c30 msgr2=0x7ff4d019a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.195+0000 7ff4ceffd640 1 --2- 172.21.15.100:0/879316271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4d0105c30 0x7ff4d019a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.195+0000 7ff4ceffd640 1 -- 172.21.15.100:0/879316271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4d0106580 msgr2=0x7ff4d019f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.195+0000 7ff4ceffd640 1 --2- 172.21.15.100:0/879316271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4d0106580 0x7ff4d019f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.195+0000 7ff4ceffd640 1 -- 172.21.15.100:0/879316271 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d019fda0 con 0x7ff4d0101ac0 2024-07-26T13:44:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.195+0000 7ff4ceffd640 1 --2- 172.21.15.100:0/879316271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4d0101ac0 0x7ff4d0199e70 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff4c000d770 tx=0x7ff4c000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.195+0000 7ff4c7fff640 1 -- 172.21.15.100:0/879316271 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c0017070 con 0x7ff4d0101ac0 2024-07-26T13:44:44.200 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.195+0000 7ff4c7fff640 1 -- 172.21.15.100:0/879316271 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4c0009c30 con 0x7ff4d0101ac0 2024-07-26T13:44:44.200 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.195+0000 7ff4d54d3640 1 -- 172.21.15.100:0/879316271 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4d01a0090 con 0x7ff4d0101ac0 2024-07-26T13:44:44.200 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.195+0000 7ff4c7fff640 1 -- 172.21.15.100:0/879316271 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c0003cc0 con 0x7ff4d0101ac0 2024-07-26T13:44:44.200 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.195+0000 7ff4d54d3640 1 -- 172.21.15.100:0/879316271 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4d01a6820 con 0x7ff4d0101ac0 2024-07-26T13:44:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.198+0000 7ff4dc3eb740 1 Processor -- start 2024-07-26T13:44:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.198+0000 7ff4dc3eb740 1 -- start start 2024-07-26T13:44:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.198+0000 7ff4c7fff640 1 -- 172.21.15.100:0/879316271 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7ff4c0003e60 con 0x7ff4d0101ac0 2024-07-26T13:44:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.198+0000 7ff4dc3eb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56161234f1b0 0x5616126f1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.198+0000 7ff4dc3eb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616123262f0 0x5616126f2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.198+0000 7ff4dc3eb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56161232e4d0 0x5616126f2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.198+0000 7ff4dc3eb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5616126673b0 con 0x5616123262f0 2024-07-26T13:44:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.198+0000 7ff4dc3eb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5616126568b0 con 0x56161232e4d0 2024-07-26T13:44:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.198+0000 7ff4dc3eb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56161265a8f0 con 0x56161234f1b0 2024-07-26T13:44:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.199+0000 7ff4ceffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616123262f0 0x5616126f2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.199+0000 7ff4ceffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616123262f0 0x5616126f2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35712/0 (socket says 172.21.15.100:35712) 2024-07-26T13:44:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.199+0000 7ff4ceffd640 1 -- 172.21.15.100:0/2885260414 learned_addr learned my addr 172.21.15.100:0/2885260414 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.199+0000 7ff4cf7fe640 1 --2- 172.21.15.100:0/2885260414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56161234f1b0 0x5616126f1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.199+0000 7ff4ce7fc640 1 --2- 172.21.15.100:0/2885260414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56161232e4d0 0x5616126f2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.199+0000 7ff4ceffd640 1 -- 172.21.15.100:0/2885260414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56161234f1b0 msgr2=0x5616126f1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.199+0000 7ff4ceffd640 1 --2- 172.21.15.100:0/2885260414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56161234f1b0 0x5616126f1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.199+0000 7ff4ceffd640 1 -- 172.21.15.100:0/2885260414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56161232e4d0 msgr2=0x5616126f2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.199+0000 7ff4ceffd640 1 --2- 172.21.15.100:0/2885260414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56161232e4d0 0x5616126f2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.199+0000 7ff4ceffd640 1 -- 172.21.15.100:0/2885260414 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616125fd9a0 con 0x5616123262f0 2024-07-26T13:44:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.199+0000 7ff4ce7fc640 1 --2- 172.21.15.100:0/2885260414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56161232e4d0 0x5616126f2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:44:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.199+0000 7ff4c7fff640 1 --2- 172.21.15.100:0/879316271 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff4bc0b3dc0 0x7ff4bc0b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.199+0000 7ff4cf7fe640 1 --2- 172.21.15.100:0/879316271 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff4bc0b3dc0 0x7ff4bc0b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.199+0000 7ff4ceffd640 1 --2- 172.21.15.100:0/2885260414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616123262f0 0x5616126f2220 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ff4c00e0f30 tx=0x7ff4c00eb690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46a3d075bf07b185 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.199+0000 7ff4c7fff640 1 -- 172.21.15.100:0/879316271 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7ff4c00e38a0 con 0x7ff4d0101ac0 2024-07-26T13:44:44.209 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.199+0000 7ff4c57fa640 1 -- 172.21.15.100:0/2885260414 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c0020030 con 0x5616123262f0 2024-07-26T13:44:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.200+0000 7ff4c57fa640 1 -- 172.21.15.100:0/2885260414 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4c00aa040 con 0x5616123262f0 2024-07-26T13:44:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.200+0000 7ff4c57fa640 1 -- 172.21.15.100:0/2885260414 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c00a63f0 con 0x5616123262f0 2024-07-26T13:44:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.200+0000 7ff4dc3eb740 1 -- 172.21.15.100:0/2885260414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616123262f0 msgr2=0x5616126f2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.200+0000 7ff4cf7fe640 1 --2- 172.21.15.100:0/879316271 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff4bc0b3dc0 0x7ff4bc0b62d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8005fd0 tx=0x7ff4c8009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.200+0000 7ff4dc3eb740 1 --2- 172.21.15.100:0/2885260414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616123262f0 0x5616126f2220 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ff4c00e0f30 tx=0x7ff4c00eb690 comp rx=0 tx=0).stop 2024-07-26T13:44:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.201+0000 7ff4dc3eb740 1 -- 172.21.15.100:0/2885260414 shutdown_connections 2024-07-26T13:44:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.201+0000 7ff4dc3eb740 1 --2- 172.21.15.100:0/2885260414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56161232e4d0 0x5616126f2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.201+0000 7ff4dc3eb740 1 --2- 172.21.15.100:0/2885260414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616123262f0 0x5616126f2220 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.201+0000 7ff4dc3eb740 1 --2- 172.21.15.100:0/2885260414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56161234f1b0 0x5616126f1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:44.210 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.201+0000 7ff4dc3eb740 1 -- 172.21.15.100:0/2885260414 >> 172.21.15.100:0/2885260414 conn(0x561612343300 msgr2=0x5616126efbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.201+0000 7ff4dc3eb740 1 -- 172.21.15.100:0/2885260414 shutdown_connections 2024-07-26T13:44:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.201+0000 7ff4dc3eb740 1 -- 172.21.15.100:0/2885260414 wait complete. 2024-07-26T13:44:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.201+0000 7ff4dc3eb740 1 -- start start 2024-07-26T13:44:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.201+0000 7ff4dc3eb740 1 Processor -- start 2024-07-26T13:44:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.201+0000 7ff4dc3eb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616123262f0 0x5616126fd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.201+0000 7ff4dc3eb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56161232e4d0 0x5616126fdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.201+0000 7ff4dc3eb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56161234f1b0 0x561612709780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.201+0000 7ff4dc3eb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x561612590ab0 con 0x5616123262f0 2024-07-26T13:44:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.201+0000 7ff4dc3eb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5616125decf0 con 0x56161234f1b0 2024-07-26T13:44:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.201+0000 7ff4dc3eb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561612700460 con 0x56161232e4d0 2024-07-26T13:44:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.201+0000 7ff4dc3eb740 10 client.0 _ll_register_callbacks cb 0x5616126509f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:44:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.201+0000 7ff4ceffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616123262f0 0x5616126fd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.202+0000 7ff4ceffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616123262f0 0x5616126fd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35714/0 (socket says 172.21.15.100:35714) 2024-07-26T13:44:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.202+0000 7ff4ceffd640 1 -- 172.21.15.100:0/822851190 learned_addr learned my addr 172.21.15.100:0/822851190 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.202+0000 7ff4ce7fc640 1 --2- 172.21.15.100:0/822851190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56161232e4d0 0x5616126fdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.202+0000 7ff4cf7fe640 1 --2- 172.21.15.100:0/822851190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56161234f1b0 0x561612709780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.202+0000 7ff4ceffd640 1 -- 172.21.15.100:0/822851190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56161232e4d0 msgr2=0x5616126fdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.202+0000 7ff4ceffd640 1 --2- 172.21.15.100:0/822851190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56161232e4d0 0x5616126fdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.202+0000 7ff4ceffd640 1 -- 172.21.15.100:0/822851190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56161234f1b0 msgr2=0x561612709780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.202+0000 7ff4ceffd640 1 --2- 172.21.15.100:0/822851190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56161234f1b0 0x561612709780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.202+0000 7ff4ceffd640 1 -- 172.21.15.100:0/822851190 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616123ba790 con 0x5616123262f0 2024-07-26T13:44:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.202+0000 7ff4ceffd640 1 --2- 172.21.15.100:0/822851190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616123262f0 0x5616126fd7f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7ff4c00aa4c0 tx=0x7ff4c00cd110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c53e5fe5efe027 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.202+0000 7ff4b67fc640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:44:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.202+0000 7ff4b67fc640 1 -- 172.21.15.100:0/822851190 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c0020070 con 0x5616123262f0 2024-07-26T13:44:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.202+0000 7ff4b67fc640 1 -- 172.21.15.100:0/822851190 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7ff4c00b1e00 con 0x5616123262f0 2024-07-26T13:44:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.202+0000 7ff4dc3eb740 1 -- 172.21.15.100:0/822851190 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4c00aa040 con 0x5616123262f0 2024-07-26T13:44:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.202+0000 7ff4b67fc640 1 -- 172.21.15.100:0/822851190 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff4c00a9ca0 con 0x5616123262f0 2024-07-26T13:44:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.202+0000 7ff4dc3eb740 10 client.16674 fetch_fsmap learned FSMap version 37 2024-07-26T13:44:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.202+0000 7ff4dc3eb740 1 -- 172.21.15.100:0/822851190 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561612433620 con 0x5616123262f0 2024-07-26T13:44:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.203+0000 7ff4b67fc640 1 -- 172.21.15.100:0/822851190 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4c00aa040 con 0x5616123262f0 2024-07-26T13:44:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.203+0000 7ff4b67fc640 1 -- 172.21.15.100:0/822851190 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7ff4c00a67c0 con 0x5616123262f0 2024-07-26T13:44:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.203+0000 7ff4dc3eb740 10 client.16674 fetch_fsmap finished waiting for FSMap version 37 2024-07-26T13:44:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.203+0000 7ff4dc3eb740 10 client.16674 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:44:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.203+0000 7ff4dc3eb740 20 client.16674 populate_metadata read hostname 'smithi100' 2024-07-26T13:44:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.203+0000 7ff4dc3eb740 1 --2- 172.21.15.100:0/822851190 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56161230dd80 0x56161270f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.203+0000 7ff4dc3eb740 4 client.16674 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:44:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.203+0000 7ff4dc3eb740 1 -- 172.21.15.100:0/822851190 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561612669300 con 0x56161230dd80 2024-07-26T13:44:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.203+0000 7ff4ce7fc640 1 --2- 172.21.15.100:0/822851190 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56161230dd80 0x56161270f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.204+0000 7ff4b67fc640 10 client.16674 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.204+0000 7ff4ce7fc640 1 --2- 172.21.15.100:0/822851190 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56161230dd80 0x56161270f590 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb69e8d67ec672a5 server_cookie=36d169b901a2f601 in_seq=0 out_seq=0 2024-07-26T13:44:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.204+0000 7ff4b67fc640 1 -- 172.21.15.100:0/822851190 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561612669300 con 0x56161230dd80 2024-07-26T13:44:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.204+0000 7ff4b67fc640 10 client.16674 handle_command_reply: tid=0 2024-07-26T13:44:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.205+0000 7ff4dc3eb740 1 client.16674 shutdown 2024-07-26T13:44:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.205+0000 7ff4dc3eb740 1 -- 172.21.15.100:0/822851190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616123262f0 msgr2=0x5616126fd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.205+0000 7ff4dc3eb740 1 --2- 172.21.15.100:0/822851190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616123262f0 0x5616126fd7f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7ff4c00aa4c0 tx=0x7ff4c00cd110 comp rx=0 tx=0).stop 2024-07-26T13:44:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.205+0000 7ff4dc3eb740 1 -- 172.21.15.100:0/822851190 shutdown_connections 2024-07-26T13:44:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.205+0000 7ff4dc3eb740 1 --2- 172.21.15.100:0/822851190 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56161230dd80 0x56161270f590 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.205+0000 7ff4dc3eb740 1 --2- 172.21.15.100:0/822851190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56161234f1b0 0x561612709780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.205+0000 7ff4dc3eb740 1 --2- 172.21.15.100:0/822851190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56161232e4d0 0x5616126fdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.205+0000 7ff4b67fc640 0 client.16674 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.205+0000 7ff4dc3eb740 1 --2- 172.21.15.100:0/822851190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616123262f0 0x5616126fd7f0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.205+0000 7ff4dc3eb740 1 -- 172.21.15.100:0/822851190 >> 172.21.15.100:0/822851190 conn(0x561612343300 msgr2=0x5616126ee2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.205+0000 7ff4dc3eb740 1 -- 172.21.15.100:0/822851190 shutdown_connections 2024-07-26T13:44:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.205+0000 7ff4dc3eb740 1 -- 172.21.15.100:0/822851190 wait complete. 2024-07-26T13:44:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.205+0000 7ff4dc3eb740 20 client.16674 trim_cache size 0 max 16384 2024-07-26T13:44:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.221+0000 7ff4dc3eb740 1 Processor -- start 2024-07-26T13:44:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.221+0000 7ff4dc3eb740 1 -- start start 2024-07-26T13:44:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.222+0000 7ff4dc3eb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616123262f0 0x5616126ff4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.222+0000 7ff4dc3eb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56161232e4d0 0x5616126f2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.222+0000 7ff4dc3eb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56161234f1b0 0x5616126fdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.222+0000 7ff4dc3eb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d01b1ab0 con 0x56161234f1b0 2024-07-26T13:44:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.222+0000 7ff4dc3eb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d01af320 con 0x5616123262f0 2024-07-26T13:44:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.222+0000 7ff4dc3eb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d01ad170 con 0x56161232e4d0 2024-07-26T13:44:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.222+0000 7ff4cf7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56161234f1b0 0x5616126fdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.222+0000 7ff4cf7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56161234f1b0 0x5616126fdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35724/0 (socket says 172.21.15.100:35724) 2024-07-26T13:44:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.222+0000 7ff4cf7fe640 1 -- 172.21.15.100:0/10489536 learned_addr learned my addr 172.21.15.100:0/10489536 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.222+0000 7ff4ceffd640 1 --2- 172.21.15.100:0/10489536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616123262f0 0x5616126ff4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.222+0000 7ff4ce7fc640 1 --2- 172.21.15.100:0/10489536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56161232e4d0 0x5616126f2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.222+0000 7ff4cf7fe640 1 -- 172.21.15.100:0/10489536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56161232e4d0 msgr2=0x5616126f2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.222+0000 7ff4cf7fe640 1 --2- 172.21.15.100:0/10489536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56161232e4d0 0x5616126f2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.222+0000 7ff4cf7fe640 1 -- 172.21.15.100:0/10489536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616123262f0 msgr2=0x5616126ff4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.222+0000 7ff4cf7fe640 1 --2- 172.21.15.100:0/10489536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616123262f0 0x5616126ff4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.222+0000 7ff4cf7fe640 1 -- 172.21.15.100:0/10489536 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616126fc520 con 0x56161234f1b0 2024-07-26T13:44:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.222+0000 7ff4cf7fe640 1 --2- 172.21.15.100:0/10489536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56161234f1b0 0x5616126fdac0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8003f10 tx=0x7ff4c8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38967402d8dbcf36 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.223+0000 7ff4c4ff9640 1 -- 172.21.15.100:0/10489536 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c800f4e0 con 0x56161234f1b0 2024-07-26T13:44:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.223+0000 7ff4c4ff9640 1 -- 172.21.15.100:0/10489536 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4c800fb30 con 0x56161234f1b0 2024-07-26T13:44:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.223+0000 7ff4c4ff9640 1 -- 172.21.15.100:0/10489536 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c801daf0 con 0x56161234f1b0 2024-07-26T13:44:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.223+0000 7ff4dc3eb740 1 -- 172.21.15.100:0/10489536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56161234f1b0 msgr2=0x5616126fdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.223+0000 7ff4dc3eb740 1 --2- 172.21.15.100:0/10489536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56161234f1b0 0x5616126fdac0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8003f10 tx=0x7ff4c8016000 comp rx=0 tx=0).stop 2024-07-26T13:44:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.224+0000 7ff4dc3eb740 1 -- 172.21.15.100:0/10489536 shutdown_connections 2024-07-26T13:44:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.224+0000 7ff4dc3eb740 1 --2- 172.21.15.100:0/10489536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56161234f1b0 0x5616126fdac0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.224+0000 7ff4dc3eb740 1 --2- 172.21.15.100:0/10489536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56161232e4d0 0x5616126f2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.224+0000 7ff4dc3eb740 1 --2- 172.21.15.100:0/10489536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616123262f0 0x5616126ff4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.224+0000 7ff4dc3eb740 1 -- 172.21.15.100:0/10489536 >> 172.21.15.100:0/10489536 conn(0x561612343300 msgr2=0x5616126f6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.224+0000 7ff4dc3eb740 1 -- 172.21.15.100:0/10489536 shutdown_connections 2024-07-26T13:44:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.224+0000 7ff4dc3eb740 1 -- 172.21.15.100:0/10489536 wait complete. 2024-07-26T13:44:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.224+0000 7ff4dc3eb740 1 -- start start 2024-07-26T13:44:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.224+0000 7ff4dc3eb740 1 Processor -- start 2024-07-26T13:44:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.224+0000 7ff4dc3eb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616123262f0 0x561612712da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.224+0000 7ff4dc3eb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56161232e4d0 0x561612711270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.224+0000 7ff4dc3eb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56161234f1b0 0x5616127118c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.224+0000 7ff4dc3eb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d01ad7e0 con 0x5616123262f0 2024-07-26T13:44:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.224+0000 7ff4dc3eb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d01aec50 con 0x56161232e4d0 2024-07-26T13:44:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.224+0000 7ff4dc3eb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d01adeb0 con 0x56161234f1b0 2024-07-26T13:44:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.224+0000 7ff4dc3eb740 10 client.0 _ll_register_callbacks cb 0x7ff4c8009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:44:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.225+0000 7ff4ceffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616123262f0 0x561612712da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.225+0000 7ff4ceffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616123262f0 0x561612712da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35726/0 (socket says 172.21.15.100:35726) 2024-07-26T13:44:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.225+0000 7ff4ceffd640 1 -- 172.21.15.100:0/109352066 learned_addr learned my addr 172.21.15.100:0/109352066 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.225+0000 7ff4cf7fe640 1 --2- 172.21.15.100:0/109352066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56161234f1b0 0x5616127118c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.225+0000 7ff4ce7fc640 1 --2- 172.21.15.100:0/109352066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56161232e4d0 0x561612711270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.225+0000 7ff4ceffd640 1 -- 172.21.15.100:0/109352066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56161234f1b0 msgr2=0x5616127118c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.225+0000 7ff4ceffd640 1 --2- 172.21.15.100:0/109352066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56161234f1b0 0x5616127118c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.225+0000 7ff4ceffd640 1 -- 172.21.15.100:0/109352066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56161232e4d0 msgr2=0x561612711270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.225+0000 7ff4ceffd640 1 --2- 172.21.15.100:0/109352066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56161232e4d0 0x561612711270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.225+0000 7ff4ceffd640 1 -- 172.21.15.100:0/109352066 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561612712050 con 0x5616123262f0 2024-07-26T13:44:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.225+0000 7ff4b67fc640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:44:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.225+0000 7ff4ceffd640 1 --2- 172.21.15.100:0/109352066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616123262f0 0x561612712da0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff4c00e1020 tx=0x7ff4c00aa670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725103db0065ebcf server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.225+0000 7ff4b67fc640 1 -- 172.21.15.100:0/109352066 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c0020070 con 0x5616123262f0 2024-07-26T13:44:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.225+0000 7ff4b67fc640 1 -- 172.21.15.100:0/109352066 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8570+0+0 (secure 0 0 0) 0x7ff4c00a6b20 con 0x5616123262f0 2024-07-26T13:44:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.225+0000 7ff4dc3eb740 1 -- 172.21.15.100:0/109352066 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4c800fb30 con 0x5616123262f0 2024-07-26T13:44:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.225+0000 7ff4b67fc640 1 -- 172.21.15.100:0/109352066 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff4c00a9e80 con 0x5616123262f0 2024-07-26T13:44:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.225+0000 7ff4dc3eb740 10 client.16680 fetch_fsmap learned FSMap version 37 2024-07-26T13:44:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.225+0000 7ff4dc3eb740 1 -- 172.21.15.100:0/109352066 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561612712050 con 0x5616123262f0 2024-07-26T13:44:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.226+0000 7ff4b67fc640 1 -- 172.21.15.100:0/109352066 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4c800fb30 con 0x5616123262f0 2024-07-26T13:44:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.226+0000 7ff4b67fc640 1 -- 172.21.15.100:0/109352066 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7ff4c00aa790 con 0x5616123262f0 2024-07-26T13:44:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.226+0000 7ff4dc3eb740 10 client.16680 fetch_fsmap finished waiting for FSMap version 37 2024-07-26T13:44:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.226+0000 7ff4dc3eb740 10 client.16680 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:44:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.226+0000 7ff4dc3eb740 20 client.16680 populate_metadata read hostname 'smithi100' 2024-07-26T13:44:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.226+0000 7ff4dc3eb740 1 --2- 172.21.15.100:0/109352066 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5616122e31c0 0x561612779040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.226+0000 7ff4dc3eb740 4 client.16680 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-26T13:44:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.226+0000 7ff4dc3eb740 1 -- 172.21.15.100:0/109352066 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561612664da0 con 0x5616122e31c0 2024-07-26T13:44:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.226+0000 7ff4ce7fc640 1 --2- 172.21.15.100:0/109352066 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5616122e31c0 0x561612779040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.227+0000 7ff4ce7fc640 1 --2- 172.21.15.100:0/109352066 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5616122e31c0 0x561612779040 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d840a2a595f722f server_cookie=67ed4f5dfa37be05 in_seq=0 out_seq=0 2024-07-26T13:44:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:44.227+0000 7ff4b67fc640 10 client.16680 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:44.414 INFO:tasks.workunit.client.0.smithi100.stdout: 1 157185 1.02 MB/sec execute 58 sec latency 4080.363 ms 2024-07-26T13:44:45.414 INFO:tasks.workunit.client.0.smithi100.stdout: 1 157185 1.01 MB/sec execute 59 sec latency 5080.445 ms 2024-07-26T13:44:45.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:45 smithi138 ceph-mon[36954]: pgmap v504: 97 pgs: 97 active+clean; 63 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 271 KiB/s wr, 44 op/s 2024-07-26T13:44:45.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:45 smithi100 ceph-mon[32599]: pgmap v504: 97 pgs: 97 active+clean; 63 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 271 KiB/s wr, 44 op/s 2024-07-26T13:44:46.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:45 smithi167 ceph-mon[37140]: pgmap v504: 97 pgs: 97 active+clean; 63 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 271 KiB/s wr, 44 op/s 2024-07-26T13:44:46.414 INFO:tasks.workunit.client.0.smithi100.stdout: 1 157185 0.99 MB/sec execute 60 sec latency 6080.542 ms 2024-07-26T13:44:47.415 INFO:tasks.workunit.client.0.smithi100.stdout: 1 157185 0.97 MB/sec execute 61 sec latency 7080.640 ms 2024-07-26T13:44:47.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:47 smithi138 ceph-mon[36954]: pgmap v505: 97 pgs: 97 active+clean; 63 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 280 KiB/s wr, 38 op/s 2024-07-26T13:44:47.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:47 smithi100 ceph-mon[32599]: pgmap v505: 97 pgs: 97 active+clean; 63 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 280 KiB/s wr, 38 op/s 2024-07-26T13:44:48.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:47 smithi167 ceph-mon[37140]: pgmap v505: 97 pgs: 97 active+clean; 63 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 280 KiB/s wr, 38 op/s 2024-07-26T13:44:48.419 INFO:tasks.workunit.client.0.smithi100.stdout: 1 157185 0.96 MB/sec execute 62 sec latency 8080.738 ms 2024-07-26T13:44:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:48 smithi138 ceph-mon[36954]: pgmap v506: 97 pgs: 97 active+clean; 63 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 261 KiB/s wr, 38 op/s 2024-07-26T13:44:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:48 smithi100 ceph-mon[32599]: pgmap v506: 97 pgs: 97 active+clean; 63 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 261 KiB/s wr, 38 op/s 2024-07-26T13:44:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:48 smithi167 ceph-mon[37140]: pgmap v506: 97 pgs: 97 active+clean; 63 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 261 KiB/s wr, 38 op/s 2024-07-26T13:44:49.414 INFO:tasks.workunit.client.0.smithi100.stdout: 1 157185 0.94 MB/sec execute 63 sec latency 9080.837 ms 2024-07-26T13:44:50.415 INFO:tasks.workunit.client.0.smithi100.stdout: 1 157185 0.93 MB/sec execute 64 sec latency 10080.914 ms 2024-07-26T13:44:51.415 INFO:tasks.workunit.client.0.smithi100.stdout: 1 157185 0.91 MB/sec execute 65 sec latency 11081.023 ms 2024-07-26T13:44:51.831 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:51 smithi100 ceph-mon[32599]: pgmap v507: 97 pgs: 97 active+clean; 63 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 222 KiB/s wr, 43 op/s 2024-07-26T13:44:51.920 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:51 smithi138 ceph-mon[36954]: pgmap v507: 97 pgs: 97 active+clean; 63 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 222 KiB/s wr, 43 op/s 2024-07-26T13:44:51.920 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:51 smithi138 systemd[1]: ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.e.service: Scheduled restart job, restart counter is at 1. 2024-07-26T13:44:51.920 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:51 smithi138 systemd[1]: Stopped Ceph mds.e for bf83e2e6-4b52-11ef-bcb7-c7b262605968. 2024-07-26T13:44:51.921 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:51 smithi138 systemd[1]: ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.e.service: Consumed 16.505s CPU time. 2024-07-26T13:44:51.921 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:51 smithi138 systemd[1]: Starting Ceph mds.e for bf83e2e6-4b52-11ef-bcb7-c7b262605968... 2024-07-26T13:44:52.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:51 smithi167 ceph-mon[37140]: pgmap v507: 97 pgs: 97 active+clean; 63 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 222 KiB/s wr, 43 op/s 2024-07-26T13:44:52.190 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:51 smithi138 podman[71240]: 2024-07-26 13:44:51.822849405 +0000 UTC m=+0.021537612 image pull ae51f432d4fa82622bc253ca9e2da7666c617e096e99d43d0a0e4fdaca8865f8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2 2024-07-26T13:44:52.190 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:51 smithi138 podman[71240]: 2024-07-26 13:44:51.957091693 +0000 UTC m=+0.155779936 container create cad2ed376b2212f867d847b358b7f49d6b8038e6b3ee8eaaf93645e44403c599 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-07-26T13:44:52.415 INFO:tasks.workunit.client.0.smithi100.stdout: 1 157185 0.90 MB/sec execute 66 sec latency 12081.102 ms 2024-07-26T13:44:52.547 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:52 smithi138 podman[71240]: 2024-07-26 13:44:52.199141037 +0000 UTC m=+0.397829242 container init cad2ed376b2212f867d847b358b7f49d6b8038e6b3ee8eaaf93645e44403c599 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240724, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb) 2024-07-26T13:44:52.547 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:52 smithi138 podman[71240]: 2024-07-26 13:44:52.201681967 +0000 UTC m=+0.400370178 container start cad2ed376b2212f867d847b358b7f49d6b8038e6b3ee8eaaf93645e44403c599 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-07-26T13:44:52.547 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:52 smithi138 ceph-mds[71260]: -- 172.21.15.138:0/2611533173 <== mon.2 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5591707db860 con 0x55917497d180 2024-07-26T13:44:52.547 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:52 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: starting mds.e at 2024-07-26T13:44:52.547 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:52 smithi138 bash[71240]: cad2ed376b2212f867d847b358b7f49d6b8038e6b3ee8eaaf93645e44403c599 2024-07-26T13:44:52.547 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:44:52 smithi138 systemd[1]: Started Ceph mds.e for bf83e2e6-4b52-11ef-bcb7-c7b262605968. 2024-07-26T13:44:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:52 smithi138 ceph-mon[36954]: daemon mds.e restarted 2024-07-26T13:44:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:52 smithi138 ceph-mon[36954]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-26T13:44:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:52 smithi138 ceph-mon[36954]: osdmap e83: 12 total, 12 up, 12 in 2024-07-26T13:44:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:52 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.138:6834/2831339503,v1:172.21.15.138:6835/2831339503] up:boot 2024-07-26T13:44:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:52 smithi138 ceph-mon[36954]: Standby daemon mds.c assigned to filesystem cephfs as rank 2 2024-07-26T13:44:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:52 smithi138 ceph-mon[36954]: fsmap cephfs:2/3 {0=i=up:active,1=b=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-07-26T13:44:52.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-26T13:44:52.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:52 smithi138 ceph-mon[36954]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=c=up:replay} 2 up:standby-replay 7 up:standby 2024-07-26T13:44:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:52 smithi100 ceph-mon[32599]: daemon mds.e restarted 2024-07-26T13:44:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:52 smithi100 ceph-mon[32599]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-26T13:44:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:52 smithi100 ceph-mon[32599]: osdmap e83: 12 total, 12 up, 12 in 2024-07-26T13:44:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:52 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.138:6834/2831339503,v1:172.21.15.138:6835/2831339503] up:boot 2024-07-26T13:44:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:52 smithi100 ceph-mon[32599]: Standby daemon mds.c assigned to filesystem cephfs as rank 2 2024-07-26T13:44:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:52 smithi100 ceph-mon[32599]: fsmap cephfs:2/3 {0=i=up:active,1=b=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-07-26T13:44:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-26T13:44:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:52 smithi100 ceph-mon[32599]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=c=up:replay} 2 up:standby-replay 7 up:standby 2024-07-26T13:44:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:52 smithi167 ceph-mon[37140]: daemon mds.e restarted 2024-07-26T13:44:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:52 smithi167 ceph-mon[37140]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-26T13:44:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:52 smithi167 ceph-mon[37140]: osdmap e83: 12 total, 12 up, 12 in 2024-07-26T13:44:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:52 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.138:6834/2831339503,v1:172.21.15.138:6835/2831339503] up:boot 2024-07-26T13:44:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:52 smithi167 ceph-mon[37140]: Standby daemon mds.c assigned to filesystem cephfs as rank 2 2024-07-26T13:44:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:52 smithi167 ceph-mon[37140]: fsmap cephfs:2/3 {0=i=up:active,1=b=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-07-26T13:44:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-26T13:44:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:52 smithi167 ceph-mon[37140]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=c=up:replay} 2 up:standby-replay 7 up:standby 2024-07-26T13:44:53.415 INFO:tasks.workunit.client.0.smithi100.stdout: 1 157185 0.89 MB/sec execute 67 sec latency 13081.160 ms 2024-07-26T13:44:53.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:53 smithi138 ceph-mon[36954]: pgmap v509: 97 pgs: 97 active+clean; 65 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 280 KiB/s wr, 31 op/s 2024-07-26T13:44:53.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:53 smithi100 ceph-mon[32599]: pgmap v509: 97 pgs: 97 active+clean; 65 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 280 KiB/s wr, 31 op/s 2024-07-26T13:44:54.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:53 smithi167 ceph-mon[37140]: pgmap v509: 97 pgs: 97 active+clean; 65 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 280 KiB/s wr, 31 op/s 2024-07-26T13:44:54.415 INFO:tasks.workunit.client.0.smithi100.stdout: 1 157185 0.87 MB/sec execute 68 sec latency 14081.255 ms 2024-07-26T13:44:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:54 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.167:6836/2476068386,v1:172.21.15.167:6837/2476068386] up:resolve 2024-07-26T13:44:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:54 smithi138 ceph-mon[36954]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=c=up:resolve} 2 up:standby-replay 7 up:standby 2024-07-26T13:44:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:54 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.167:6836/2476068386,v1:172.21.15.167:6837/2476068386] up:resolve 2024-07-26T13:44:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:54 smithi100 ceph-mon[32599]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=c=up:resolve} 2 up:standby-replay 7 up:standby 2024-07-26T13:44:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:54 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.167:6836/2476068386,v1:172.21.15.167:6837/2476068386] up:resolve 2024-07-26T13:44:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:54 smithi167 ceph-mon[37140]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=c=up:resolve} 2 up:standby-replay 7 up:standby 2024-07-26T13:44:55.415 INFO:tasks.workunit.client.0.smithi100.stdout: 1 157185 0.86 MB/sec execute 69 sec latency 15081.352 ms 2024-07-26T13:44:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:55.596+0000 7ff4b67fc640 1 -- 172.21.15.100:0/109352066 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561612664da0 con 0x5616122e31c0 2024-07-26T13:44:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:55.596+0000 7ff4b67fc640 10 client.16680 handle_command_reply: tid=0 2024-07-26T13:44:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:55.596+0000 7ff4dc3eb740 1 client.16680 shutdown 2024-07-26T13:44:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:55.596+0000 7ff4dc3eb740 1 -- 172.21.15.100:0/109352066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616123262f0 msgr2=0x561612712da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:55.596+0000 7ff4dc3eb740 1 --2- 172.21.15.100:0/109352066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616123262f0 0x561612712da0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff4c00e1020 tx=0x7ff4c00aa670 comp rx=0 tx=0).stop 2024-07-26T13:44:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:55.596+0000 7ff4dc3eb740 1 -- 172.21.15.100:0/109352066 shutdown_connections 2024-07-26T13:44:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:55.596+0000 7ff4dc3eb740 1 --2- 172.21.15.100:0/109352066 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5616122e31c0 0x561612779040 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:55.596+0000 7ff4dc3eb740 1 --2- 172.21.15.100:0/109352066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56161234f1b0 0x5616127118c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:55.596+0000 7ff4b67fc640 0 client.16680 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:55.596+0000 7ff4dc3eb740 1 --2- 172.21.15.100:0/109352066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56161232e4d0 0x561612711270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:55.596+0000 7ff4dc3eb740 1 --2- 172.21.15.100:0/109352066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5616123262f0 0x561612712da0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:55.596+0000 7ff4dc3eb740 1 -- 172.21.15.100:0/109352066 >> 172.21.15.100:0/109352066 conn(0x561612343300 msgr2=0x5616126ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:55.596+0000 7ff4dc3eb740 1 -- 172.21.15.100:0/109352066 shutdown_connections 2024-07-26T13:44:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:55.596+0000 7ff4dc3eb740 1 -- 172.21.15.100:0/109352066 wait complete. 2024-07-26T13:44:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:55.596+0000 7ff4dc3eb740 20 client.16680 trim_cache size 0 max 16384 2024-07-26T13:44:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-07-26T13:44:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-07-26T13:44:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-07-26T13:44:55.600 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-07-26T13:44:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:55.597+0000 7ff4d54d3640 1 -- 172.21.15.100:0/879316271 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff4bc0b3dc0 msgr2=0x7ff4bc0b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:55.597+0000 7ff4d54d3640 1 --2- 172.21.15.100:0/879316271 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff4bc0b3dc0 0x7ff4bc0b62d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8005fd0 tx=0x7ff4c8009500 comp rx=0 tx=0).stop 2024-07-26T13:44:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:55.597+0000 7ff4d54d3640 1 -- 172.21.15.100:0/879316271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4d0101ac0 msgr2=0x7ff4d0199e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:55.597+0000 7ff4d54d3640 1 --2- 172.21.15.100:0/879316271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4d0101ac0 0x7ff4d0199e70 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff4c000d770 tx=0x7ff4c000dc30 comp rx=0 tx=0).stop 2024-07-26T13:44:55.601 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:55.597+0000 7ff4d54d3640 1 -- 172.21.15.100:0/879316271 shutdown_connections 2024-07-26T13:44:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:55.597+0000 7ff4d54d3640 1 --2- 172.21.15.100:0/879316271 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff4bc0b3dc0 0x7ff4bc0b62d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:55.597+0000 7ff4d54d3640 1 --2- 172.21.15.100:0/879316271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4d0106580 0x7ff4d019f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:55.597+0000 7ff4d54d3640 1 --2- 172.21.15.100:0/879316271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4d0105c30 0x7ff4d019a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:55.597+0000 7ff4d54d3640 1 --2- 172.21.15.100:0/879316271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4d0101ac0 0x7ff4d0199e70 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:55.597+0000 7ff4d54d3640 1 -- 172.21.15.100:0/879316271 >> 172.21.15.100:0/879316271 conn(0x7ff4d00798f0 msgr2=0x7ff4d0104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:55.597+0000 7ff4d54d3640 1 -- 172.21.15.100:0/879316271 shutdown_connections 2024-07-26T13:44:55.602 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:44:55.597+0000 7ff4d54d3640 1 -- 172.21.15.100:0/879316271 wait complete. 2024-07-26T13:44:55.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:55 smithi138 ceph-mon[36954]: pgmap v510: 97 pgs: 97 active+clean; 67 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 656 KiB/s wr, 35 op/s 2024-07-26T13:44:55.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:55 smithi138 ceph-mon[36954]: reconnect by client.15333 v1:192.168.0.1:0/1052122651 after 0 2024-07-26T13:44:55.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:55 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.167:6836/2476068386,v1:172.21.15.167:6837/2476068386] up:reconnect 2024-07-26T13:44:55.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:55 smithi138 ceph-mon[36954]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=c=up:reconnect} 2 up:standby-replay 7 up:standby 2024-07-26T13:44:55.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:55 smithi100 ceph-mon[32599]: pgmap v510: 97 pgs: 97 active+clean; 67 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 656 KiB/s wr, 35 op/s 2024-07-26T13:44:55.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:55 smithi100 ceph-mon[32599]: reconnect by client.15333 v1:192.168.0.1:0/1052122651 after 0 2024-07-26T13:44:55.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:55 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.167:6836/2476068386,v1:172.21.15.167:6837/2476068386] up:reconnect 2024-07-26T13:44:55.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:55 smithi100 ceph-mon[32599]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=c=up:reconnect} 2 up:standby-replay 7 up:standby 2024-07-26T13:44:56.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:55 smithi167 ceph-mon[37140]: pgmap v510: 97 pgs: 97 active+clean; 67 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 656 KiB/s wr, 35 op/s 2024-07-26T13:44:56.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:55 smithi167 ceph-mon[37140]: reconnect by client.15333 v1:192.168.0.1:0/1052122651 after 0 2024-07-26T13:44:56.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:55 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.167:6836/2476068386,v1:172.21.15.167:6837/2476068386] up:reconnect 2024-07-26T13:44:56.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:55 smithi167 ceph-mon[37140]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=c=up:reconnect} 2 up:standby-replay 7 up:standby 2024-07-26T13:44:56.415 INFO:tasks.workunit.client.0.smithi100.stdout: 1 157185 0.85 MB/sec execute 70 sec latency 16081.436 ms 2024-07-26T13:44:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:56 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.167:6836/2476068386,v1:172.21.15.167:6837/2476068386] up:rejoin 2024-07-26T13:44:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:56 smithi138 ceph-mon[36954]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=c=up:rejoin} 2 up:standby-replay 7 up:standby 2024-07-26T13:44:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:56 smithi138 ceph-mon[36954]: pgmap v511: 97 pgs: 97 active+clean; 68 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 976 KiB/s wr, 40 op/s 2024-07-26T13:44:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:56 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.167:6836/2476068386,v1:172.21.15.167:6837/2476068386] up:rejoin 2024-07-26T13:44:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:56 smithi100 ceph-mon[32599]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=c=up:rejoin} 2 up:standby-replay 7 up:standby 2024-07-26T13:44:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:56 smithi100 ceph-mon[32599]: pgmap v511: 97 pgs: 97 active+clean; 68 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 976 KiB/s wr, 40 op/s 2024-07-26T13:44:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:56 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.167:6836/2476068386,v1:172.21.15.167:6837/2476068386] up:rejoin 2024-07-26T13:44:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:56 smithi167 ceph-mon[37140]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=c=up:rejoin} 2 up:standby-replay 7 up:standby 2024-07-26T13:44:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:56 smithi167 ceph-mon[37140]: pgmap v511: 97 pgs: 97 active+clean; 68 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 976 KiB/s wr, 40 op/s 2024-07-26T13:44:57.294 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-26T13:44:57.415 INFO:tasks.workunit.client.0.smithi100.stdout: 1 157478 0.88 MB/sec execute 71 sec latency 16252.350 ms 2024-07-26T13:44:57.567 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:44:57.834 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:57 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.167:6836/2476068386,v1:172.21.15.167:6837/2476068386] up:clientreplay 2024-07-26T13:44:57.834 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:57 smithi100 ceph-mon[32599]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=c=up:clientreplay} 2 up:standby-replay 7 up:standby 2024-07-26T13:44:57.834 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:57 smithi100 ceph-mon[32599]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-07-26T13:44:57.841 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:57 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.167:6836/2476068386,v1:172.21.15.167:6837/2476068386] up:clientreplay 2024-07-26T13:44:57.841 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:57 smithi138 ceph-mon[36954]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=c=up:clientreplay} 2 up:standby-replay 7 up:standby 2024-07-26T13:44:57.841 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:57 smithi138 ceph-mon[36954]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-07-26T13:44:58.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:57 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.167:6836/2476068386,v1:172.21.15.167:6837/2476068386] up:clientreplay 2024-07-26T13:44:58.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:57 smithi167 ceph-mon[37140]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=c=up:clientreplay} 2 up:standby-replay 7 up:standby 2024-07-26T13:44:58.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:57 smithi167 ceph-mon[37140]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-07-26T13:44:58.415 INFO:tasks.workunit.client.0.smithi100.stdout: 1 159455 1.00 MB/sec execute 72 sec latency 894.263 ms 2024-07-26T13:44:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:58 smithi138 ceph-mon[36954]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-26T13:44:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:58 smithi138 ceph-mon[36954]: Cluster is now healthy 2024-07-26T13:44:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:58 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.167:6836/2476068386,v1:172.21.15.167:6837/2476068386] up:active 2024-07-26T13:44:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:58 smithi138 ceph-mon[36954]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:44:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:58 smithi138 ceph-mon[36954]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:44:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:44:58 smithi138 ceph-mon[36954]: pgmap v512: 97 pgs: 97 active+clean; 64 MiB data, 791 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 972 KiB/s wr, 48 op/s 2024-07-26T13:44:58.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:58 smithi100 ceph-mon[32599]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-26T13:44:58.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:58 smithi100 ceph-mon[32599]: Cluster is now healthy 2024-07-26T13:44:58.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:58 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.167:6836/2476068386,v1:172.21.15.167:6837/2476068386] up:active 2024-07-26T13:44:58.950 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:58 smithi100 ceph-mon[32599]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:44:58.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:58 smithi100 ceph-mon[32599]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:44:58.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:44:58 smithi100 ceph-mon[32599]: pgmap v512: 97 pgs: 97 active+clean; 64 MiB data, 791 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 972 KiB/s wr, 48 op/s 2024-07-26T13:44:59.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:58 smithi167 ceph-mon[37140]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-26T13:44:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:58 smithi167 ceph-mon[37140]: Cluster is now healthy 2024-07-26T13:44:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:58 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.167:6836/2476068386,v1:172.21.15.167:6837/2476068386] up:active 2024-07-26T13:44:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:58 smithi167 ceph-mon[37140]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:44:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:58 smithi167 ceph-mon[37140]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:44:59.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:44:58 smithi167 ceph-mon[37140]: pgmap v512: 97 pgs: 97 active+clean; 64 MiB data, 791 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 972 KiB/s wr, 48 op/s 2024-07-26T13:44:59.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.360+0000 7f431f48f640 1 -- 172.21.15.100:0/2886619820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4318107e00 msgr2=0x7f4318108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:59.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.360+0000 7f431f48f640 1 --2- 172.21.15.100:0/2886619820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4318107e00 0x7f4318108210 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f430c009900 tx=0x7f430c02a410 comp rx=0 tx=0).stop 2024-07-26T13:44:59.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.360+0000 7f431f48f640 1 -- 172.21.15.100:0/2886619820 shutdown_connections 2024-07-26T13:44:59.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.360+0000 7f431f48f640 1 --2- 172.21.15.100:0/2886619820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4318108750 0x7f431810ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:59.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.360+0000 7f431f48f640 1 --2- 172.21.15.100:0/2886619820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4318107e00 0x7f4318108210 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:59.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.360+0000 7f431f48f640 1 --2- 172.21.15.100:0/2886619820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4318102510 0x7f4318102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:59.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.360+0000 7f431f48f640 1 -- 172.21.15.100:0/2886619820 >> 172.21.15.100:0/2886619820 conn(0x7f43180791c0 msgr2=0x7f43180795d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:59.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.360+0000 7f431f48f640 1 -- 172.21.15.100:0/2886619820 shutdown_connections 2024-07-26T13:44:59.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.360+0000 7f431f48f640 1 -- 172.21.15.100:0/2886619820 wait complete. 2024-07-26T13:44:59.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.361+0000 7f431f48f640 1 Processor -- start 2024-07-26T13:44:59.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.361+0000 7f431f48f640 1 -- start start 2024-07-26T13:44:59.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.361+0000 7f431f48f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4318102510 0x7f4318198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:59.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.361+0000 7f431f48f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4318107e00 0x7f4318199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:59.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.361+0000 7f431f48f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4318108750 0x7f431819e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:59.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.361+0000 7f431f48f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f43181123e0 con 0x7f4318102510 2024-07-26T13:44:59.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.361+0000 7f431f48f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4318112260 con 0x7f4318108750 2024-07-26T13:44:59.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.361+0000 7f431f48f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4318112560 con 0x7f4318107e00 2024-07-26T13:44:59.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.361+0000 7f431d22b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4318102510 0x7f4318198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:59.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.361+0000 7f431d22b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4318102510 0x7f4318198df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37330/0 (socket says 172.21.15.100:37330) 2024-07-26T13:44:59.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.361+0000 7f431ca2a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4318107e00 0x7f4318199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:59.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.361+0000 7f431d22b640 1 -- 172.21.15.100:0/2705221035 learned_addr learned my addr 172.21.15.100:0/2705221035 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:59.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.361+0000 7f431da2c640 1 --2- 172.21.15.100:0/2705221035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4318108750 0x7f431819e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:59.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.361+0000 7f431d22b640 1 -- 172.21.15.100:0/2705221035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4318107e00 msgr2=0x7f4318199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:59.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.361+0000 7f431d22b640 1 --2- 172.21.15.100:0/2705221035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4318107e00 0x7f4318199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:59.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.361+0000 7f431d22b640 1 -- 172.21.15.100:0/2705221035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4318108750 msgr2=0x7f431819e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:59.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.361+0000 7f431d22b640 1 --2- 172.21.15.100:0/2705221035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4318108750 0x7f431819e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:59.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.361+0000 7f431d22b640 1 -- 172.21.15.100:0/2705221035 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431819ed60 con 0x7f4318102510 2024-07-26T13:44:59.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.361+0000 7f431ca2a640 1 --2- 172.21.15.100:0/2705221035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4318107e00 0x7f4318199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:44:59.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.361+0000 7f431da2c640 1 --2- 172.21.15.100:0/2705221035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4318108750 0x7f431819e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:44:59.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.362+0000 7f431d22b640 1 --2- 172.21.15.100:0/2705221035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4318102510 0x7f4318198df0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f430800eab0 tx=0x7f430800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:59.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.362+0000 7f43167fc640 1 -- 172.21.15.100:0/2705221035 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430800ce40 con 0x7f4318102510 2024-07-26T13:44:59.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.362+0000 7f431f48f640 1 -- 172.21.15.100:0/2705221035 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f431819f050 con 0x7f4318102510 2024-07-26T13:44:59.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.362+0000 7f43167fc640 1 -- 172.21.15.100:0/2705221035 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4308011920 con 0x7f4318102510 2024-07-26T13:44:59.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.362+0000 7f43167fc640 1 -- 172.21.15.100:0/2705221035 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4308010a30 con 0x7f4318102510 2024-07-26T13:44:59.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.362+0000 7f431f48f640 1 -- 172.21.15.100:0/2705221035 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43181a57b0 con 0x7f4318102510 2024-07-26T13:44:59.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.365+0000 7f4325a13740 1 Processor -- start 2024-07-26T13:44:59.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.365+0000 7f43167fc640 1 -- 172.21.15.100:0/2705221035 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f4308010bd0 con 0x7f4318102510 2024-07-26T13:44:59.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.365+0000 7f4325a13740 1 -- start start 2024-07-26T13:44:59.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.365+0000 7f4325a13740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642597f2070 0x564259b99730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:59.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.365+0000 7f4325a13740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642597c91b0 0x564259b99c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:59.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.365+0000 7f4325a13740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642597d1390 0x564259b9a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:59.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.365+0000 7f4325a13740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x564259b0db60 con 0x5642597f2070 2024-07-26T13:44:59.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.365+0000 7f4325a13740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564259afd060 con 0x5642597c91b0 2024-07-26T13:44:59.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.365+0000 7f4325a13740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564259b010a0 con 0x5642597d1390 2024-07-26T13:44:59.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.365+0000 7f431da2c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642597f2070 0x564259b99730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:59.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.365+0000 7f431da2c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642597f2070 0x564259b99730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37338/0 (socket says 172.21.15.100:37338) 2024-07-26T13:44:59.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.365+0000 7f431da2c640 1 -- 172.21.15.100:0/712385288 learned_addr learned my addr 172.21.15.100:0/712385288 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:59.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.365+0000 7f431ca2a640 1 --2- 172.21.15.100:0/712385288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642597d1390 0x564259b9a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:59.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.365+0000 7f431d22b640 1 --2- 172.21.15.100:0/712385288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642597c91b0 0x564259b99c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:59.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.366+0000 7f431ca2a640 1 -- 172.21.15.100:0/712385288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642597c91b0 msgr2=0x564259b99c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:59.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.366+0000 7f431ca2a640 1 --2- 172.21.15.100:0/712385288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642597c91b0 0x564259b99c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:59.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.366+0000 7f431ca2a640 1 -- 172.21.15.100:0/712385288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642597f2070 msgr2=0x564259b99730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:59.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.366+0000 7f431ca2a640 1 --2- 172.21.15.100:0/712385288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642597f2070 0x564259b99730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:59.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.366+0000 7f43167fc640 1 --2- 172.21.15.100:0/2705221035 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f43000b3d40 0x7f43000b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:59.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.366+0000 7f431ca2a640 1 -- 172.21.15.100:0/712385288 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564259aa3b40 con 0x5642597d1390 2024-07-26T13:44:59.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.366+0000 7f431da2c640 1 --2- 172.21.15.100:0/712385288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642597f2070 0x564259b99730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:44:59.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.366+0000 7f431d22b640 1 --2- 172.21.15.100:0/2705221035 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f43000b3d40 0x7f43000b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:59.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.366+0000 7f431ca2a640 1 --2- 172.21.15.100:0/712385288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642597d1390 0x564259b9a1b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f430c02a930 tx=0x7f430c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=748b690c870f82e7 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:59.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.366+0000 7f43167fc640 1 -- 172.21.15.100:0/2705221035 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7f43080e46a0 con 0x7f4318102510 2024-07-26T13:44:59.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.366+0000 7f42ff7fe640 1 -- 172.21.15.100:0/712385288 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c040070 con 0x5642597d1390 2024-07-26T13:44:59.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.366+0000 7f42ff7fe640 1 -- 172.21.15.100:0/712385288 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f430c0317a0 con 0x5642597d1390 2024-07-26T13:44:59.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.366+0000 7f42ff7fe640 1 -- 172.21.15.100:0/712385288 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c035040 con 0x5642597d1390 2024-07-26T13:44:59.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.367+0000 7f4325a13740 1 -- 172.21.15.100:0/712385288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642597d1390 msgr2=0x564259b9a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:59.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.367+0000 7f431d22b640 1 --2- 172.21.15.100:0/2705221035 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f43000b3d40 0x7f43000b6250 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x564259b983b0 tx=0x7f43080ea530 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:59.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.367+0000 7f4325a13740 1 --2- 172.21.15.100:0/712385288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642597d1390 0x564259b9a1b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f430c02a930 tx=0x7f430c000bc0 comp rx=0 tx=0).stop 2024-07-26T13:44:59.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.367+0000 7f4325a13740 1 -- 172.21.15.100:0/712385288 shutdown_connections 2024-07-26T13:44:59.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.367+0000 7f4325a13740 1 --2- 172.21.15.100:0/712385288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642597d1390 0x564259b9a1b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:59.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.367+0000 7f4325a13740 1 --2- 172.21.15.100:0/712385288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642597c91b0 0x564259b99c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:59.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.367+0000 7f4325a13740 1 --2- 172.21.15.100:0/712385288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642597f2070 0x564259b99730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:59.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.367+0000 7f4325a13740 1 -- 172.21.15.100:0/712385288 >> 172.21.15.100:0/712385288 conn(0x5642597e6460 msgr2=0x564259b97620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:59.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.367+0000 7f4325a13740 1 -- 172.21.15.100:0/712385288 shutdown_connections 2024-07-26T13:44:59.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.367+0000 7f4325a13740 1 -- 172.21.15.100:0/712385288 wait complete. 2024-07-26T13:44:59.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.368+0000 7f4325a13740 1 -- start start 2024-07-26T13:44:59.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.368+0000 7f4325a13740 1 Processor -- start 2024-07-26T13:44:59.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.368+0000 7f4325a13740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642597c91b0 0x564259ba5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:59.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.368+0000 7f4325a13740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642597d1390 0x564259ba58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:59.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.368+0000 7f4325a13740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642597f2070 0x564259bb11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:59.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.368+0000 7f4325a13740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x564259960700 con 0x5642597d1390 2024-07-26T13:44:59.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.368+0000 7f4325a13740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564259a9f990 con 0x5642597f2070 2024-07-26T13:44:59.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.368+0000 7f4325a13740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564259ba7eb0 con 0x5642597c91b0 2024-07-26T13:44:59.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.368+0000 7f431ca2a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642597d1390 0x564259ba58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:59.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.368+0000 7f4325a13740 10 client.0 _ll_register_callbacks cb 0x564259aba4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:44:59.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.368+0000 7f431da2c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642597c91b0 0x564259ba5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:59.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.368+0000 7f431ca2a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642597d1390 0x564259ba58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37350/0 (socket says 172.21.15.100:37350) 2024-07-26T13:44:59.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.368+0000 7f431ca2a640 1 -- 172.21.15.100:0/1426421177 learned_addr learned my addr 172.21.15.100:0/1426421177 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:59.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.369+0000 7f431da2c640 1 --2- 172.21.15.100:0/1426421177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642597f2070 0x564259bb11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:59.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.369+0000 7f431da2c640 1 -- 172.21.15.100:0/1426421177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642597f2070 msgr2=0x564259bb11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:59.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.369+0000 7f431da2c640 1 --2- 172.21.15.100:0/1426421177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642597f2070 0x564259bb11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:59.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.369+0000 7f431da2c640 1 -- 172.21.15.100:0/1426421177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642597d1390 msgr2=0x564259ba58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:59.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.369+0000 7f431da2c640 1 --2- 172.21.15.100:0/1426421177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642597d1390 0x564259ba58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:59.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.369+0000 7f431da2c640 1 -- 172.21.15.100:0/1426421177 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56425985d650 con 0x5642597c91b0 2024-07-26T13:44:59.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.369+0000 7f431ca2a640 1 --2- 172.21.15.100:0/1426421177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642597d1390 0x564259ba58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:44:59.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.369+0000 7f431da2c640 1 --2- 172.21.15.100:0/1426421177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642597c91b0 0x564259ba5340 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4310014dd0 tx=0x7f43100095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=259370ef4f699bed server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:59.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.369+0000 7f42fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-07-26T13:44:59.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.369+0000 7f42fcff9640 1 -- 172.21.15.100:0/1426421177 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431000fe60 con 0x5642597c91b0 2024-07-26T13:44:59.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.369+0000 7f4325a13740 1 -- 172.21.15.100:0/1426421177 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f430c0317a0 con 0x5642597c91b0 2024-07-26T13:44:59.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.369+0000 7f42fcff9640 1 -- 172.21.15.100:0/1426421177 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7f43100140d0 con 0x5642597c91b0 2024-07-26T13:44:59.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.369+0000 7f42fcff9640 1 -- 172.21.15.100:0/1426421177 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f4310028050 con 0x5642597c91b0 2024-07-26T13:44:59.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.369+0000 7f4325a13740 10 client.26155 fetch_fsmap learned FSMap version 45 2024-07-26T13:44:59.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.369+0000 7f4325a13740 1 -- 172.21.15.100:0/1426421177 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x564259bb3f60 con 0x5642597c91b0 2024-07-26T13:44:59.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.370+0000 7f42fcff9640 1 -- 172.21.15.100:0/1426421177 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f430c0317a0 con 0x5642597c91b0 2024-07-26T13:44:59.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.370+0000 7f42fcff9640 1 -- 172.21.15.100:0/1426421177 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f4310020020 con 0x5642597c91b0 2024-07-26T13:44:59.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.370+0000 7f4325a13740 10 client.26155 fetch_fsmap finished waiting for FSMap version 45 2024-07-26T13:44:59.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.370+0000 7f4325a13740 10 client.26155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:44:59.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.370+0000 7f4325a13740 20 client.26155 populate_metadata read hostname 'smithi100' 2024-07-26T13:44:59.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.370+0000 7f4325a13740 1 --2- 172.21.15.100:0/1426421177 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5642597b09f0 0x564259bb74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:59.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.370+0000 7f4325a13740 4 client.26155 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:44:59.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.370+0000 7f4325a13740 1 -- 172.21.15.100:0/1426421177 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564259b0fab0 con 0x5642597b09f0 2024-07-26T13:44:59.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.370+0000 7f431ca2a640 1 --2- 172.21.15.100:0/1426421177 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5642597b09f0 0x564259bb74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:59.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.371+0000 7f42fcff9640 10 client.26155 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:59.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.371+0000 7f431ca2a640 1 --2- 172.21.15.100:0/1426421177 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5642597b09f0 0x564259bb74a0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f83f96684c5cc47c server_cookie=e81f0e39fd458b00 in_seq=0 out_seq=0 2024-07-26T13:44:59.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.371+0000 7f42fcff9640 1 -- 172.21.15.100:0/1426421177 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564259b0fab0 con 0x5642597b09f0 2024-07-26T13:44:59.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.371+0000 7f42fcff9640 10 client.26155 handle_command_reply: tid=0 2024-07-26T13:44:59.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.371+0000 7f4325a13740 1 client.26155 shutdown 2024-07-26T13:44:59.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.372+0000 7f4325a13740 1 -- 172.21.15.100:0/1426421177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642597c91b0 msgr2=0x564259ba5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:59.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.372+0000 7f4325a13740 1 --2- 172.21.15.100:0/1426421177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642597c91b0 0x564259ba5340 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4310014dd0 tx=0x7f43100095c0 comp rx=0 tx=0).stop 2024-07-26T13:44:59.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.372+0000 7f4325a13740 1 -- 172.21.15.100:0/1426421177 shutdown_connections 2024-07-26T13:44:59.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.372+0000 7f4325a13740 1 --2- 172.21.15.100:0/1426421177 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5642597b09f0 0x564259bb74a0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:59.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.372+0000 7f4325a13740 1 --2- 172.21.15.100:0/1426421177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642597f2070 0x564259bb11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:59.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.372+0000 7f4325a13740 1 --2- 172.21.15.100:0/1426421177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642597d1390 0x564259ba58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:59.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.372+0000 7f42fcff9640 0 client.26155 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:59.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.372+0000 7f4325a13740 1 --2- 172.21.15.100:0/1426421177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642597c91b0 0x564259ba5340 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:59.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.372+0000 7f4325a13740 1 -- 172.21.15.100:0/1426421177 >> 172.21.15.100:0/1426421177 conn(0x5642597e6460 msgr2=0x564259b93bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:59.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.372+0000 7f4325a13740 1 -- 172.21.15.100:0/1426421177 shutdown_connections 2024-07-26T13:44:59.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.372+0000 7f4325a13740 1 -- 172.21.15.100:0/1426421177 wait complete. 2024-07-26T13:44:59.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.372+0000 7f4325a13740 20 client.26155 trim_cache size 0 max 16384 2024-07-26T13:44:59.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.390+0000 7f4325a13740 1 Processor -- start 2024-07-26T13:44:59.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.390+0000 7f4325a13740 1 -- start start 2024-07-26T13:44:59.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.390+0000 7f4325a13740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642597c91b0 0x564259b975a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:59.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.390+0000 7f4325a13740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642597d1390 0x564259bb0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:59.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.390+0000 7f4325a13740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642597f2070 0x564259bb14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:59.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.390+0000 7f4325a13740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f43181af950 con 0x5642597f2070 2024-07-26T13:44:59.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.390+0000 7f4325a13740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f43181ad1c0 con 0x5642597d1390 2024-07-26T13:44:59.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.390+0000 7f4325a13740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f43181aaff0 con 0x5642597c91b0 2024-07-26T13:44:59.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.390+0000 7f431da2c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642597f2070 0x564259bb14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:59.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.390+0000 7f431da2c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642597f2070 0x564259bb14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37358/0 (socket says 172.21.15.100:37358) 2024-07-26T13:44:59.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.390+0000 7f431da2c640 1 -- 172.21.15.100:0/1645160494 learned_addr learned my addr 172.21.15.100:0/1645160494 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:59.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.390+0000 7f431da2c640 1 --2- 172.21.15.100:0/1645160494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642597c91b0 0x564259b975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:59.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.391+0000 7f431ca2a640 1 --2- 172.21.15.100:0/1645160494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642597d1390 0x564259bb0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:59.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.391+0000 7f431da2c640 1 -- 172.21.15.100:0/1645160494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642597c91b0 msgr2=0x564259b975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:59.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.391+0000 7f431da2c640 1 --2- 172.21.15.100:0/1645160494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642597c91b0 0x564259b975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:59.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.391+0000 7f431da2c640 1 -- 172.21.15.100:0/1645160494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642597d1390 msgr2=0x564259bb0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:59.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.391+0000 7f431da2c640 1 --2- 172.21.15.100:0/1645160494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642597d1390 0x564259bb0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:59.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.391+0000 7f431da2c640 1 -- 172.21.15.100:0/1645160494 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564259ba6400 con 0x5642597f2070 2024-07-26T13:44:59.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.391+0000 7f431ca2a640 1 --2- 172.21.15.100:0/1645160494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642597d1390 0x564259bb0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:44:59.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.391+0000 7f431da2c640 1 --2- 172.21.15.100:0/1645160494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642597f2070 0x564259bb14d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f43100059f0 tx=0x7f4310005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda2a1d8a9d7f580 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:59.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.391+0000 7f42feffd640 1 -- 172.21.15.100:0/1645160494 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431000fe60 con 0x5642597f2070 2024-07-26T13:44:59.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.391+0000 7f42feffd640 1 -- 172.21.15.100:0/1645160494 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4310009c80 con 0x5642597f2070 2024-07-26T13:44:59.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.391+0000 7f42feffd640 1 -- 172.21.15.100:0/1645160494 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4310024ab0 con 0x5642597f2070 2024-07-26T13:44:59.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.392+0000 7f4325a13740 1 -- 172.21.15.100:0/1645160494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642597f2070 msgr2=0x564259bb14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:59.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.392+0000 7f4325a13740 1 --2- 172.21.15.100:0/1645160494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642597f2070 0x564259bb14d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f43100059f0 tx=0x7f4310005f90 comp rx=0 tx=0).stop 2024-07-26T13:44:59.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.392+0000 7f4325a13740 1 -- 172.21.15.100:0/1645160494 shutdown_connections 2024-07-26T13:44:59.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.392+0000 7f4325a13740 1 --2- 172.21.15.100:0/1645160494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642597f2070 0x564259bb14d0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:59.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.392+0000 7f4325a13740 1 --2- 172.21.15.100:0/1645160494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642597d1390 0x564259bb0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:59.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.392+0000 7f4325a13740 1 --2- 172.21.15.100:0/1645160494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642597c91b0 0x564259b975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:59.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.392+0000 7f4325a13740 1 -- 172.21.15.100:0/1645160494 >> 172.21.15.100:0/1645160494 conn(0x5642597e6460 msgr2=0x564259b96e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:59.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.392+0000 7f4325a13740 1 -- 172.21.15.100:0/1645160494 shutdown_connections 2024-07-26T13:44:59.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.392+0000 7f4325a13740 1 -- 172.21.15.100:0/1645160494 wait complete. 2024-07-26T13:44:59.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.392+0000 7f4325a13740 1 -- start start 2024-07-26T13:44:59.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.393+0000 7f4325a13740 1 Processor -- start 2024-07-26T13:44:59.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.393+0000 7f4325a13740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642597c91b0 0x564259c17e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:59.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.393+0000 7f4325a13740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642597d1390 0x564259c18420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:59.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.393+0000 7f4325a13740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642597f2070 0x564259bace20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:59.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.393+0000 7f4325a13740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f43181ab680 con 0x5642597f2070 2024-07-26T13:44:59.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.393+0000 7f4325a13740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f43181acaf0 con 0x5642597d1390 2024-07-26T13:44:59.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.393+0000 7f4325a13740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f43181abd50 con 0x5642597c91b0 2024-07-26T13:44:59.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.393+0000 7f4325a13740 10 client.0 _ll_register_callbacks cb 0x7f4310009b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:44:59.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.393+0000 7f431da2c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642597c91b0 0x564259c17e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:59.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.393+0000 7f431da2c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642597c91b0 0x564259c17e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:39008/0 (socket says 172.21.15.100:39008) 2024-07-26T13:44:59.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.393+0000 7f431da2c640 1 -- 172.21.15.100:0/3479351690 learned_addr learned my addr 172.21.15.100:0/3479351690 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:44:59.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.393+0000 7f431da2c640 1 --2- 172.21.15.100:0/3479351690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642597f2070 0x564259bace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:59.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.393+0000 7f431ca2a640 1 --2- 172.21.15.100:0/3479351690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642597d1390 0x564259c18420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:59.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.394+0000 7f431da2c640 1 -- 172.21.15.100:0/3479351690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642597d1390 msgr2=0x564259c18420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:59.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.394+0000 7f431da2c640 1 --2- 172.21.15.100:0/3479351690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642597d1390 0x564259c18420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:59.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.394+0000 7f431da2c640 1 -- 172.21.15.100:0/3479351690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642597f2070 msgr2=0x564259bace20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:44:59.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.394+0000 7f431da2c640 1 --2- 172.21.15.100:0/3479351690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642597f2070 0x564259bace20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:59.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.394+0000 7f431da2c640 1 -- 172.21.15.100:0/3479351690 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564259bad490 con 0x5642597c91b0 2024-07-26T13:44:59.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.394+0000 7f431da2c640 1 --2- 172.21.15.100:0/3479351690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642597c91b0 0x564259c17e40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4310014dd0 tx=0x7f4310014d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46112e920bcd7aac server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:44:59.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.394+0000 7f42fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-07-26T13:44:59.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.394+0000 7f42fcff9640 1 -- 172.21.15.100:0/3479351690 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43100240d0 con 0x5642597c91b0 2024-07-26T13:44:59.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.394+0000 7f42fcff9640 1 -- 172.21.15.100:0/3479351690 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7f4310035040 con 0x5642597c91b0 2024-07-26T13:44:59.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.394+0000 7f4325a13740 1 -- 172.21.15.100:0/3479351690 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4310009c80 con 0x5642597c91b0 2024-07-26T13:44:59.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.394+0000 7f42fcff9640 1 -- 172.21.15.100:0/3479351690 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f4310024270 con 0x5642597c91b0 2024-07-26T13:44:59.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.394+0000 7f4325a13740 10 client.26161 fetch_fsmap learned FSMap version 45 2024-07-26T13:44:59.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.394+0000 7f4325a13740 1 -- 172.21.15.100:0/3479351690 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x564259c1f690 con 0x5642597c91b0 2024-07-26T13:44:59.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.394+0000 7f42fcff9640 1 -- 172.21.15.100:0/3479351690 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4310009c80 con 0x5642597c91b0 2024-07-26T13:44:59.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.395+0000 7f42fcff9640 1 -- 172.21.15.100:0/3479351690 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f4310038020 con 0x5642597c91b0 2024-07-26T13:44:59.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.395+0000 7f4325a13740 10 client.26161 fetch_fsmap finished waiting for FSMap version 45 2024-07-26T13:44:59.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.395+0000 7f4325a13740 10 client.26161 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:44:59.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.395+0000 7f4325a13740 20 client.26161 populate_metadata read hostname 'smithi100' 2024-07-26T13:44:59.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.395+0000 7f4325a13740 1 --2- 172.21.15.100:0/3479351690 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x564259785da0 0x564259c21ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:44:59.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.395+0000 7f4325a13740 4 client.26161 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-26T13:44:59.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.395+0000 7f4325a13740 1 -- 172.21.15.100:0/3479351690 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564259b0b550 con 0x564259785da0 2024-07-26T13:44:59.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.395+0000 7f431ca2a640 1 --2- 172.21.15.100:0/3479351690 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x564259785da0 0x564259c21ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:44:59.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.395+0000 7f431ca2a640 1 --2- 172.21.15.100:0/3479351690 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x564259785da0 0x564259c21ed0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cb66622a2fdce39 server_cookie=4eae9e5b9983f041 in_seq=0 out_seq=0 2024-07-26T13:44:59.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.395+0000 7f42fcff9640 10 client.26161 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:59.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.396+0000 7f42fcff9640 1 -- 172.21.15.100:0/3479351690 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564259b0b550 con 0x564259785da0 2024-07-26T13:44:59.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.396+0000 7f42fcff9640 10 client.26161 handle_command_reply: tid=0 2024-07-26T13:44:59.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.396+0000 7f4325a13740 1 client.26161 shutdown 2024-07-26T13:44:59.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.396+0000 7f4325a13740 1 -- 172.21.15.100:0/3479351690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642597c91b0 msgr2=0x564259c17e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:59.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.396+0000 7f4325a13740 1 --2- 172.21.15.100:0/3479351690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642597c91b0 0x564259c17e40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4310014dd0 tx=0x7f4310014d20 comp rx=0 tx=0).stop 2024-07-26T13:44:59.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.396+0000 7f4325a13740 1 -- 172.21.15.100:0/3479351690 shutdown_connections 2024-07-26T13:44:59.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.396+0000 7f4325a13740 1 --2- 172.21.15.100:0/3479351690 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x564259785da0 0x564259c21ed0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:59.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.396+0000 7f4325a13740 1 --2- 172.21.15.100:0/3479351690 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5642597f2070 0x564259bace20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:59.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.396+0000 7f4325a13740 1 --2- 172.21.15.100:0/3479351690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642597d1390 0x564259c18420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:59.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.396+0000 7f4325a13740 1 --2- 172.21.15.100:0/3479351690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642597c91b0 0x564259c17e40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:59.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.396+0000 7f4325a13740 1 -- 172.21.15.100:0/3479351690 >> 172.21.15.100:0/3479351690 conn(0x5642597e6460 msgr2=0x564259ba65d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:59.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.396+0000 7f42fcff9640 0 client.26161 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:44:59.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.396+0000 7f4325a13740 1 -- 172.21.15.100:0/3479351690 shutdown_connections 2024-07-26T13:44:59.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.396+0000 7f4325a13740 1 -- 172.21.15.100:0/3479351690 wait complete. 2024-07-26T13:44:59.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.396+0000 7f4325a13740 20 client.26161 trim_cache size 0 max 16384 2024-07-26T13:44:59.403 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:44:59.404 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-07-26T13:44:59.404 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-07-26T13:44:59.404 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:44:59.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.397+0000 7f431f48f640 1 -- 172.21.15.100:0/2705221035 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f43000b3d40 msgr2=0x7f43000b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:59.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.397+0000 7f431f48f640 1 --2- 172.21.15.100:0/2705221035 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f43000b3d40 0x7f43000b6250 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x564259b983b0 tx=0x7f43080ea530 comp rx=0 tx=0).stop 2024-07-26T13:44:59.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.397+0000 7f431f48f640 1 -- 172.21.15.100:0/2705221035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4318102510 msgr2=0x7f4318198df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:44:59.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.397+0000 7f431f48f640 1 --2- 172.21.15.100:0/2705221035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4318102510 0x7f4318198df0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f430800eab0 tx=0x7f430800ef70 comp rx=0 tx=0).stop 2024-07-26T13:44:59.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.397+0000 7f431f48f640 1 -- 172.21.15.100:0/2705221035 shutdown_connections 2024-07-26T13:44:59.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.397+0000 7f431f48f640 1 --2- 172.21.15.100:0/2705221035 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f43000b3d40 0x7f43000b6250 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:59.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.397+0000 7f431f48f640 1 --2- 172.21.15.100:0/2705221035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4318108750 0x7f431819e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:59.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.397+0000 7f431f48f640 1 --2- 172.21.15.100:0/2705221035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4318107e00 0x7f4318199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:59.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.397+0000 7f431f48f640 1 --2- 172.21.15.100:0/2705221035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4318102510 0x7f4318198df0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:44:59.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.397+0000 7f431f48f640 1 -- 172.21.15.100:0/2705221035 >> 172.21.15.100:0/2705221035 conn(0x7f43180791c0 msgr2=0x7f4318105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:44:59.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.397+0000 7f431f48f640 1 -- 172.21.15.100:0/2705221035 shutdown_connections 2024-07-26T13:44:59.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:44:59.397+0000 7f431f48f640 1 -- 172.21.15.100:0/2705221035 wait complete. 2024-07-26T13:44:59.415 INFO:tasks.workunit.client.0.smithi100.stdout: 1 161074 1.12 MB/sec execute 73 sec latency 15.675 ms 2024-07-26T13:45:00.168 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-26T13:45:00.168 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:45:00.415 INFO:tasks.workunit.client.0.smithi100.stdout: 1 163451 1.29 MB/sec execute 74 sec latency 11.410 ms 2024-07-26T13:45:00.445 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:45:00.772 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:45:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 2024-07-26T13:45:00.273+0000 7f740ace1640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-07-26T13:45:01.415 INFO:tasks.workunit.client.0.smithi100.stdout: 1 164635 1.34 MB/sec execute 75 sec latency 23.024 ms 2024-07-26T13:45:01.823 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:01 smithi100 ceph-mon[32599]: pgmap v513: 97 pgs: 97 active+clean; 65 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.3 MiB/s wr, 101 op/s 2024-07-26T13:45:01.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:01 smithi138 ceph-mon[36954]: pgmap v513: 97 pgs: 97 active+clean; 65 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.3 MiB/s wr, 101 op/s 2024-07-26T13:45:02.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:01 smithi167 ceph-mon[37140]: pgmap v513: 97 pgs: 97 active+clean; 65 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.3 MiB/s wr, 101 op/s 2024-07-26T13:45:02.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.183+0000 7f6ee0b47640 1 -- 172.21.15.100:0/269137785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6edc1087a0 msgr2=0x7f6edc10efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:02.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.183+0000 7f6ee0b47640 1 --2- 172.21.15.100:0/269137785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6edc1087a0 0x7f6edc10efd0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4009ca0 tx=0x7f6ed402a8c0 comp rx=0 tx=0).stop 2024-07-26T13:45:02.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.183+0000 7f6ee0b47640 1 -- 172.21.15.100:0/269137785 shutdown_connections 2024-07-26T13:45:02.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.183+0000 7f6ee0b47640 1 --2- 172.21.15.100:0/269137785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6edc1087a0 0x7f6edc10efd0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:02.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.183+0000 7f6ee0b47640 1 --2- 172.21.15.100:0/269137785 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6edc107e50 0x7f6edc108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:02.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.183+0000 7f6ee0b47640 1 --2- 172.21.15.100:0/269137785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6edc102510 0x7f6edc102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:02.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.183+0000 7f6ee0b47640 1 -- 172.21.15.100:0/269137785 >> 172.21.15.100:0/269137785 conn(0x7f6edc0791a0 msgr2=0x7f6edc0795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:02.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.183+0000 7f6ee0b47640 1 -- 172.21.15.100:0/269137785 shutdown_connections 2024-07-26T13:45:02.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.184+0000 7f6ee0b47640 1 -- 172.21.15.100:0/269137785 wait complete. 2024-07-26T13:45:02.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.184+0000 7f6ee0b47640 1 Processor -- start 2024-07-26T13:45:02.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.185+0000 7f6ee0b47640 1 -- start start 2024-07-26T13:45:02.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.185+0000 7f6ee0b47640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6edc102510 0x7f6edc19c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:02.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.185+0000 7f6ee0b47640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6edc107e50 0x7f6edc19c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:02.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.185+0000 7f6ee0b47640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6edc1087a0 0x7f6edc1a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:02.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.185+0000 7f6ee0b47640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6edc112470 con 0x7f6edc107e50 2024-07-26T13:45:02.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.185+0000 7f6ee0b47640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6edc1122f0 con 0x7f6edc1087a0 2024-07-26T13:45:02.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.185+0000 7f6ee0b47640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6edc1125f0 con 0x7f6edc102510 2024-07-26T13:45:02.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.185+0000 7f6ed9d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6edc107e50 0x7f6edc19c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:02.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.185+0000 7f6ed9d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6edc107e50 0x7f6edc19c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37390/0 (socket says 172.21.15.100:37390) 2024-07-26T13:45:02.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.185+0000 7f6ed9d9b640 1 -- 172.21.15.100:0/2611663638 learned_addr learned my addr 172.21.15.100:0/2611663638 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:02.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.185+0000 7f6eda59c640 1 --2- 172.21.15.100:0/2611663638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6edc102510 0x7f6edc19c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:02.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.185+0000 7f6edad9d640 1 --2- 172.21.15.100:0/2611663638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6edc1087a0 0x7f6edc1a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:02.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.185+0000 7f6ed9d9b640 1 -- 172.21.15.100:0/2611663638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6edc102510 msgr2=0x7f6edc19c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:02.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.185+0000 7f6ed9d9b640 1 --2- 172.21.15.100:0/2611663638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6edc102510 0x7f6edc19c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:02.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.185+0000 7f6ed9d9b640 1 -- 172.21.15.100:0/2611663638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6edc1087a0 msgr2=0x7f6edc1a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:02.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.185+0000 7f6ed9d9b640 1 --2- 172.21.15.100:0/2611663638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6edc1087a0 0x7f6edc1a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:02.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.185+0000 7f6ed9d9b640 1 -- 172.21.15.100:0/2611663638 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6edc1a2040 con 0x7f6edc107e50 2024-07-26T13:45:02.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.186+0000 7f6ed9d9b640 1 --2- 172.21.15.100:0/2611663638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6edc107e50 0x7f6edc19c670 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f6ec400e9c0 tx=0x7f6ec400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:02.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.186+0000 7f6ed37fe640 1 -- 172.21.15.100:0/2611663638 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec400ccc0 con 0x7f6edc107e50 2024-07-26T13:45:02.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.186+0000 7f6ed37fe640 1 -- 172.21.15.100:0/2611663638 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ec40054d0 con 0x7f6edc107e50 2024-07-26T13:45:02.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.186+0000 7f6ee0b47640 1 -- 172.21.15.100:0/2611663638 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6edc1a2330 con 0x7f6edc107e50 2024-07-26T13:45:02.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.186+0000 7f6ed37fe640 1 -- 172.21.15.100:0/2611663638 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4010430 con 0x7f6edc107e50 2024-07-26T13:45:02.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.186+0000 7f6ee0b47640 1 -- 172.21.15.100:0/2611663638 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6edc1a8a90 con 0x7f6edc107e50 2024-07-26T13:45:02.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.187+0000 7f6ee0b47640 1 -- 172.21.15.100:0/2611663638 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6edc103790 con 0x7f6edc107e50 2024-07-26T13:45:02.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.190+0000 7f6ed37fe640 1 -- 172.21.15.100:0/2611663638 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f6ec4005000 con 0x7f6edc107e50 2024-07-26T13:45:02.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.190+0000 7f6ed37fe640 1 --2- 172.21.15.100:0/2611663638 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6ec80b3d40 0x7f6ec80b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:02.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.191+0000 7f6eda59c640 1 --2- 172.21.15.100:0/2611663638 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6ec80b3d40 0x7f6ec80b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:02.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.191+0000 7f6ed37fe640 1 -- 172.21.15.100:0/2611663638 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7f6ec40e45a0 con 0x7f6edc107e50 2024-07-26T13:45:02.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.191+0000 7f6eda59c640 1 --2- 172.21.15.100:0/2611663638 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6ec80b3d40 0x7f6ec80b6250 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc009bd0 tx=0x7f6ecc005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:02.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.193+0000 7f6ed37fe640 1 -- 172.21.15.100:0/2611663638 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f6ec4055600 con 0x7f6edc107e50 2024-07-26T13:45:02.415 INFO:tasks.workunit.client.0.smithi100.stdout: 1 166972 1.51 MB/sec execute 76 sec latency 14.611 ms 2024-07-26T13:45:02.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.424+0000 7f6ee0b47640 1 -- 172.21.15.100:0/2611663638 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6edc102920 con 0x7f6edc107e50 2024-07-26T13:45:02.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.425+0000 7f6ed37fe640 1 -- 172.21.15.100:0/2611663638 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11931 (secure 0 0 0) 0x7f6edc102920 con 0x7f6edc107e50 2024-07-26T13:45:02.426 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:45:02.426 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":45,"btime":"2024-07-26T13:44:57:595027+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14790,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/313501409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":313501409},{"type":"v1","addr":"172.21.15.100:6839","nonce":313501409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/1402975506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":1402975506},{"type":"v1","addr":"172.21.15.100:6841","nonce":1402975506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-26T13:37:01.281843+0000","modified":"2024-07-26T13:44:57.595007+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24607,"mds_1":24605,"mds_2":25762},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24575":{"gid":24575,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/3393225483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3393225483},{"type":"v1","addr":"172.21.15.138:6837","nonce":3393225483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.138:6839/1580925471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1580925471},{"type":"v1","addr":"172.21.15.138:6839","nonce":1580925471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/398150514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":398150514},{"type":"v1","addr":"172.21.15.167:6839","nonce":398150514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25762":{"gid":25762,"name":"c","rank":2,"incarnation":39,"state":"up:active","state_seq":32,"addr":"172.21.15.167:6837/2476068386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2476068386},{"type":"v1","addr":"172.21.15.167:6837","nonce":2476068386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25997":{"gid":25997,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/2831339503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2831339503},{"type":"v1","addr":"172.21.15.138:6835","nonce":2831339503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24607,"qdb_cluster":[24607,25762,24605]},"id":1}]} 2024-07-26T13:45:02.426 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 45 2024-07-26T13:45:02.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.427+0000 7f6ee0b47640 1 -- 172.21.15.100:0/2611663638 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6ec80b3d40 msgr2=0x7f6ec80b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:02.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.427+0000 7f6ee0b47640 1 --2- 172.21.15.100:0/2611663638 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6ec80b3d40 0x7f6ec80b6250 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc009bd0 tx=0x7f6ecc005f90 comp rx=0 tx=0).stop 2024-07-26T13:45:02.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.427+0000 7f6ee0b47640 1 -- 172.21.15.100:0/2611663638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6edc107e50 msgr2=0x7f6edc19c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:02.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.427+0000 7f6ee0b47640 1 --2- 172.21.15.100:0/2611663638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6edc107e50 0x7f6edc19c670 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f6ec400e9c0 tx=0x7f6ec400ee80 comp rx=0 tx=0).stop 2024-07-26T13:45:02.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.427+0000 7f6ee0b47640 1 -- 172.21.15.100:0/2611663638 shutdown_connections 2024-07-26T13:45:02.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.427+0000 7f6ee0b47640 1 --2- 172.21.15.100:0/2611663638 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6ec80b3d40 0x7f6ec80b6250 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:02.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.427+0000 7f6ee0b47640 1 --2- 172.21.15.100:0/2611663638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6edc1087a0 0x7f6edc1a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:02.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.427+0000 7f6ee0b47640 1 --2- 172.21.15.100:0/2611663638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6edc107e50 0x7f6edc19c670 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:02.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.427+0000 7f6ee0b47640 1 --2- 172.21.15.100:0/2611663638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6edc102510 0x7f6edc19c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:02.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.427+0000 7f6ee0b47640 1 -- 172.21.15.100:0/2611663638 >> 172.21.15.100:0/2611663638 conn(0x7f6edc0791a0 msgr2=0x7f6edc105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:02.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.427+0000 7f6ee0b47640 1 -- 172.21.15.100:0/2611663638 shutdown_connections 2024-07-26T13:45:02.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:02.427+0000 7f6ee0b47640 1 -- 172.21.15.100:0/2611663638 wait complete. 2024-07-26T13:45:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:45:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:45:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:45:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:45:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:02 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2611663638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:45:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:45:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:45:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:45:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:45:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:02 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/2611663638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:45:03.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:45:03.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:45:03.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:45:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:45:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:02 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/2611663638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:45:03.163 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-26T13:45:03.415 INFO:tasks.workunit.client.0.smithi100.stdout: 1 168689 1.56 MB/sec execute 77 sec latency 19.731 ms 2024-07-26T13:45:03.435 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:45:03.830 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:03 smithi100 ceph-mon[32599]: pgmap v514: 97 pgs: 97 active+clean; 66 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-07-26T13:45:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:03 smithi138 ceph-mon[36954]: pgmap v514: 97 pgs: 97 active+clean; 66 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-07-26T13:45:04.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:03 smithi167 ceph-mon[37140]: pgmap v514: 97 pgs: 97 active+clean; 66 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-07-26T13:45:04.415 INFO:tasks.workunit.client.0.smithi100.stdout: 1 170684 1.72 MB/sec execute 78 sec latency 15.245 ms 2024-07-26T13:45:05.416 INFO:tasks.workunit.client.0.smithi100.stdout: 1 172869 1.80 MB/sec execute 79 sec latency 18.173 ms 2024-07-26T13:45:05.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.506+0000 7ff9cf32b640 1 -- 172.21.15.100:0/945791781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9c81024b0 msgr2=0x7ff9c81028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:05.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.506+0000 7ff9cf32b640 1 --2- 172.21.15.100:0/945791781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9c81024b0 0x7ff9c81028c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4009900 tx=0x7ff9c402a410 comp rx=0 tx=0).stop 2024-07-26T13:45:05.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.506+0000 7ff9cf32b640 1 -- 172.21.15.100:0/945791781 shutdown_connections 2024-07-26T13:45:05.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.506+0000 7ff9cf32b640 1 --2- 172.21.15.100:0/945791781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9c8108740 0x7ff9c810ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:05.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.506+0000 7ff9cf32b640 1 --2- 172.21.15.100:0/945791781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9c8107df0 0x7ff9c8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:05.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.506+0000 7ff9cf32b640 1 --2- 172.21.15.100:0/945791781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9c81024b0 0x7ff9c81028c0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:05.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.506+0000 7ff9cf32b640 1 -- 172.21.15.100:0/945791781 >> 172.21.15.100:0/945791781 conn(0x7ff9c8079170 msgr2=0x7ff9c8079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:05.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.506+0000 7ff9cf32b640 1 -- 172.21.15.100:0/945791781 shutdown_connections 2024-07-26T13:45:05.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.506+0000 7ff9cf32b640 1 -- 172.21.15.100:0/945791781 wait complete. 2024-07-26T13:45:05.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.507+0000 7ff9cf32b640 1 Processor -- start 2024-07-26T13:45:05.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.507+0000 7ff9cf32b640 1 -- start start 2024-07-26T13:45:05.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.507+0000 7ff9cf32b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9c81024b0 0x7ff9c8198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:05.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.507+0000 7ff9cf32b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9c8107df0 0x7ff9c8199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:05.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.507+0000 7ff9cf32b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9c8108740 0x7ff9c819e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:05.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.507+0000 7ff9cf32b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c81123c0 con 0x7ff9c8108740 2024-07-26T13:45:05.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.507+0000 7ff9cf32b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c8112240 con 0x7ff9c81024b0 2024-07-26T13:45:05.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.507+0000 7ff9cf32b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c8112540 con 0x7ff9c8107df0 2024-07-26T13:45:05.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.508+0000 7ff9cc8c6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9c8107df0 0x7ff9c8199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:05.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.508+0000 7ff9cc8c6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9c8107df0 0x7ff9c8199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:40680/0 (socket says 172.21.15.100:40680) 2024-07-26T13:45:05.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.508+0000 7ff9cc8c6640 1 -- 172.21.15.100:0/4081740779 learned_addr learned my addr 172.21.15.100:0/4081740779 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:05.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.508+0000 7ff9cd0c7640 1 --2- 172.21.15.100:0/4081740779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9c81024b0 0x7ff9c8198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:05.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.508+0000 7ff9cd8c8640 1 --2- 172.21.15.100:0/4081740779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9c8108740 0x7ff9c819e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:05.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.508+0000 7ff9cc8c6640 1 -- 172.21.15.100:0/4081740779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9c81024b0 msgr2=0x7ff9c8198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:05.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.508+0000 7ff9cc8c6640 1 --2- 172.21.15.100:0/4081740779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9c81024b0 0x7ff9c8198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:05.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.508+0000 7ff9cc8c6640 1 -- 172.21.15.100:0/4081740779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9c8108740 msgr2=0x7ff9c819e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:05.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.508+0000 7ff9cc8c6640 1 --2- 172.21.15.100:0/4081740779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9c8108740 0x7ff9c819e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:05.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.508+0000 7ff9cc8c6640 1 -- 172.21.15.100:0/4081740779 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c819ed00 con 0x7ff9c8107df0 2024-07-26T13:45:05.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.508+0000 7ff9cc8c6640 1 --2- 172.21.15.100:0/4081740779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9c8107df0 0x7ff9c8199330 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff9b800e9f0 tx=0x7ff9b800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:05.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.509+0000 7ff9be7fc640 1 -- 172.21.15.100:0/4081740779 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b800ccc0 con 0x7ff9c8107df0 2024-07-26T13:45:05.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.509+0000 7ff9cf32b640 1 -- 172.21.15.100:0/4081740779 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9c819eff0 con 0x7ff9c8107df0 2024-07-26T13:45:05.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.509+0000 7ff9be7fc640 1 -- 172.21.15.100:0/4081740779 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9b80054d0 con 0x7ff9c8107df0 2024-07-26T13:45:05.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.509+0000 7ff9be7fc640 1 -- 172.21.15.100:0/4081740779 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8010430 con 0x7ff9c8107df0 2024-07-26T13:45:05.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.509+0000 7ff9cf32b640 1 -- 172.21.15.100:0/4081740779 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9c806b250 con 0x7ff9c8107df0 2024-07-26T13:45:05.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.512+0000 7ff9d58af740 1 Processor -- start 2024-07-26T13:45:05.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.512+0000 7ff9d58af740 1 -- start start 2024-07-26T13:45:05.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.512+0000 7ff9d58af740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9c8f6070 0x558e9cc9b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:05.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.512+0000 7ff9d58af740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e9c8cd1b0 0x558e9cc9bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:05.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.512+0000 7ff9d58af740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e9c8d5390 0x558e9cc9bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:05.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.512+0000 7ff9d58af740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x558e9cc11bb0 con 0x558e9c8f6070 2024-07-26T13:45:05.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.512+0000 7ff9d58af740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558e9cc010b0 con 0x558e9c8cd1b0 2024-07-26T13:45:05.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.512+0000 7ff9d58af740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558e9cc050f0 con 0x558e9c8d5390 2024-07-26T13:45:05.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.512+0000 7ff9cd0c7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e9c8cd1b0 0x558e9cc9bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:05.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.512+0000 7ff9cd0c7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e9c8cd1b0 0x558e9cc9bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:42904/0 (socket says 172.21.15.100:42904) 2024-07-26T13:45:05.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.512+0000 7ff9cd0c7640 1 -- 172.21.15.100:0/1811068614 learned_addr learned my addr 172.21.15.100:0/1811068614 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:05.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.512+0000 7ff9cd8c8640 1 --2- 172.21.15.100:0/1811068614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9c8f6070 0x558e9cc9b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:05.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.513+0000 7ff9cd0c7640 1 -- 172.21.15.100:0/1811068614 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e9c8d5390 msgr2=0x558e9cc9bff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-26T13:45:05.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.513+0000 7ff9cd0c7640 1 --2- 172.21.15.100:0/1811068614 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e9c8d5390 0x558e9cc9bff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:05.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.513+0000 7ff9cd0c7640 1 -- 172.21.15.100:0/1811068614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9c8f6070 msgr2=0x558e9cc9b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:05.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.513+0000 7ff9cd0c7640 1 --2- 172.21.15.100:0/1811068614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9c8f6070 0x558e9cc9b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:05.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.513+0000 7ff9cd0c7640 1 -- 172.21.15.100:0/1811068614 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e9cba7b90 con 0x558e9c8cd1b0 2024-07-26T13:45:05.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.513+0000 7ff9be7fc640 1 -- 172.21.15.100:0/4081740779 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7ff9b80105d0 con 0x7ff9c8107df0 2024-07-26T13:45:05.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.513+0000 7ff9cd0c7640 1 --2- 172.21.15.100:0/1811068614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e9c8cd1b0 0x558e9cc9bab0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4009900 tx=0x7ff9c4032b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6a0148734ca0e30 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:05.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.513+0000 7ff9b37fe640 1 -- 172.21.15.100:0/1811068614 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4040070 con 0x558e9c8cd1b0 2024-07-26T13:45:05.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.513+0000 7ff9b37fe640 1 -- 172.21.15.100:0/1811068614 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9c4004030 con 0x558e9c8cd1b0 2024-07-26T13:45:05.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.513+0000 7ff9b37fe640 1 -- 172.21.15.100:0/1811068614 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4035040 con 0x558e9c8cd1b0 2024-07-26T13:45:05.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.514+0000 7ff9be7fc640 1 --2- 172.21.15.100:0/4081740779 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff9b40b3dc0 0x7ff9b40b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:05.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.514+0000 7ff9be7fc640 1 -- 172.21.15.100:0/4081740779 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7ff9b80e4360 con 0x7ff9c8107df0 2024-07-26T13:45:05.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.514+0000 7ff9cd8c8640 1 --2- 172.21.15.100:0/4081740779 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff9b40b3dc0 0x7ff9b40b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:05.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.514+0000 7ff9d58af740 1 -- 172.21.15.100:0/1811068614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e9c8cd1b0 msgr2=0x558e9cc9bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:05.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.514+0000 7ff9d58af740 1 --2- 172.21.15.100:0/1811068614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e9c8cd1b0 0x558e9cc9bab0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4009900 tx=0x7ff9c4032b90 comp rx=0 tx=0).stop 2024-07-26T13:45:05.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.514+0000 7ff9d58af740 1 -- 172.21.15.100:0/1811068614 shutdown_connections 2024-07-26T13:45:05.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.514+0000 7ff9d58af740 1 --2- 172.21.15.100:0/1811068614 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e9c8d5390 0x558e9cc9bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:05.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.514+0000 7ff9d58af740 1 --2- 172.21.15.100:0/1811068614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e9c8cd1b0 0x558e9cc9bab0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:05.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.514+0000 7ff9d58af740 1 --2- 172.21.15.100:0/1811068614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9c8f6070 0x558e9cc9b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:05.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.514+0000 7ff9d58af740 1 -- 172.21.15.100:0/1811068614 >> 172.21.15.100:0/1811068614 conn(0x558e9c8ea460 msgr2=0x558e9cc99460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:05.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.515+0000 7ff9d58af740 1 -- 172.21.15.100:0/1811068614 shutdown_connections 2024-07-26T13:45:05.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.515+0000 7ff9cd8c8640 1 --2- 172.21.15.100:0/4081740779 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff9b40b3dc0 0x7ff9b40b62d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0005fd0 tx=0x7ff9c0009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:05.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.515+0000 7ff9d58af740 1 -- 172.21.15.100:0/1811068614 wait complete. 2024-07-26T13:45:05.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.515+0000 7ff9d58af740 1 -- start start 2024-07-26T13:45:05.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.515+0000 7ff9d58af740 1 Processor -- start 2024-07-26T13:45:05.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.515+0000 7ff9d58af740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9c8cd1b0 0x558e9cca7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:05.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.515+0000 7ff9d58af740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e9c8d5390 0x558e9cca7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:05.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.515+0000 7ff9d58af740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e9c8f6070 0x558e9ccb2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:05.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.515+0000 7ff9d58af740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x558e9ca64700 con 0x558e9c8cd1b0 2024-07-26T13:45:05.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.515+0000 7ff9d58af740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558e9cba39e0 con 0x558e9c8f6070 2024-07-26T13:45:05.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.515+0000 7ff9d58af740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558e9cca9cf0 con 0x558e9c8d5390 2024-07-26T13:45:05.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.516+0000 7ff9d58af740 10 client.0 _ll_register_callbacks cb 0x558e9cbbe4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:45:05.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.516+0000 7ff9cd0c7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9c8cd1b0 0x558e9cca7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:05.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.516+0000 7ff9cd0c7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9c8cd1b0 0x558e9cca7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35184/0 (socket says 172.21.15.100:35184) 2024-07-26T13:45:05.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.516+0000 7ff9cd0c7640 1 -- 172.21.15.100:0/2239243582 learned_addr learned my addr 172.21.15.100:0/2239243582 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:05.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.516+0000 7ff9cd8c8640 1 --2- 172.21.15.100:0/2239243582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e9c8f6070 0x558e9ccb2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:05.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.516+0000 7ff9cc8c6640 1 --2- 172.21.15.100:0/2239243582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e9c8d5390 0x558e9cca7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:05.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.516+0000 7ff9cd0c7640 1 -- 172.21.15.100:0/2239243582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e9c8d5390 msgr2=0x558e9cca7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:05.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.516+0000 7ff9cd0c7640 1 --2- 172.21.15.100:0/2239243582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e9c8d5390 0x558e9cca7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:05.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.516+0000 7ff9cd0c7640 1 -- 172.21.15.100:0/2239243582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e9c8f6070 msgr2=0x558e9ccb2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:05.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.516+0000 7ff9cd0c7640 1 --2- 172.21.15.100:0/2239243582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e9c8f6070 0x558e9ccb2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:05.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.516+0000 7ff9cd0c7640 1 -- 172.21.15.100:0/2239243582 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e9c961650 con 0x558e9c8cd1b0 2024-07-26T13:45:05.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.516+0000 7ff9cd0c7640 1 --2- 172.21.15.100:0/2239243582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9c8cd1b0 0x558e9cca7240 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4032f30 tx=0x7ff9c4032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d0a4209e4cbaee server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:05.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.516+0000 7ff9b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:45:05.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.517+0000 7ff9b0ff9640 1 -- 172.21.15.100:0/2239243582 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4035070 con 0x558e9c8cd1b0 2024-07-26T13:45:05.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.517+0000 7ff9b0ff9640 1 -- 172.21.15.100:0/2239243582 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7ff9c4005460 con 0x558e9c8cd1b0 2024-07-26T13:45:05.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.517+0000 7ff9d58af740 1 -- 172.21.15.100:0/2239243582 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9c4004030 con 0x558e9c8cd1b0 2024-07-26T13:45:05.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.517+0000 7ff9b0ff9640 1 -- 172.21.15.100:0/2239243582 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ff9c4040040 con 0x558e9c8cd1b0 2024-07-26T13:45:05.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.517+0000 7ff9d58af740 10 client.16722 fetch_fsmap learned FSMap version 45 2024-07-26T13:45:05.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.517+0000 7ff9d58af740 1 -- 172.21.15.100:0/2239243582 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x558e9ccb5db0 con 0x558e9c8cd1b0 2024-07-26T13:45:05.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.517+0000 7ff9b0ff9640 1 -- 172.21.15.100:0/2239243582 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9c4004030 con 0x558e9c8cd1b0 2024-07-26T13:45:05.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.517+0000 7ff9b0ff9640 1 -- 172.21.15.100:0/2239243582 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7ff9c4004b50 con 0x558e9c8cd1b0 2024-07-26T13:45:05.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.517+0000 7ff9d58af740 10 client.16722 fetch_fsmap finished waiting for FSMap version 45 2024-07-26T13:45:05.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.517+0000 7ff9d58af740 10 client.16722 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-26T13:45:05.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.517+0000 7ff9d58af740 20 client.16722 populate_metadata read hostname 'smithi100' 2024-07-26T13:45:05.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.517+0000 7ff9d58af740 1 --2- 172.21.15.100:0/2239243582 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x558e9c8b49f0 0x558e9ccb9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:05.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.517+0000 7ff9d58af740 4 client.16722 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:45:05.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.517+0000 7ff9d58af740 1 -- 172.21.15.100:0/2239243582 --> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e9cc13b00 con 0x558e9c8b49f0 2024-07-26T13:45:05.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.518+0000 7ff9cc8c6640 1 --2- 172.21.15.100:0/2239243582 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x558e9c8b49f0 0x558e9ccb9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:05.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.518+0000 7ff9b0ff9640 10 client.16722 ms_handle_connect on v2:172.21.15.138:6838/1580925471 2024-07-26T13:45:05.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.518+0000 7ff9cc8c6640 1 --2- 172.21.15.100:0/2239243582 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x558e9c8b49f0 0x558e9ccb9110 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8a6d138dd48ae5b server_cookie=1b178b53e11a9fb9 in_seq=0 out_seq=0 2024-07-26T13:45:05.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.519+0000 7ff9b0ff9640 1 -- 172.21.15.100:0/2239243582 <== mds.1 v2:172.21.15.138:6838/1580925471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558e9cc13b00 con 0x558e9c8b49f0 2024-07-26T13:45:05.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.519+0000 7ff9b0ff9640 10 client.16722 handle_command_reply: tid=0 2024-07-26T13:45:05.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.519+0000 7ff9d58af740 1 client.16722 shutdown 2024-07-26T13:45:05.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.519+0000 7ff9d58af740 1 -- 172.21.15.100:0/2239243582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9c8cd1b0 msgr2=0x558e9cca7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:05.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.519+0000 7ff9d58af740 1 --2- 172.21.15.100:0/2239243582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9c8cd1b0 0x558e9cca7240 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4032f30 tx=0x7ff9c4032f60 comp rx=0 tx=0).stop 2024-07-26T13:45:05.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.519+0000 7ff9d58af740 1 -- 172.21.15.100:0/2239243582 shutdown_connections 2024-07-26T13:45:05.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.519+0000 7ff9d58af740 1 --2- 172.21.15.100:0/2239243582 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x558e9c8b49f0 0x558e9ccb9110 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:05.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.519+0000 7ff9d58af740 1 --2- 172.21.15.100:0/2239243582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e9c8f6070 0x558e9ccb2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:05.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.519+0000 7ff9d58af740 1 --2- 172.21.15.100:0/2239243582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e9c8d5390 0x558e9cca7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:05.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.519+0000 7ff9d58af740 1 --2- 172.21.15.100:0/2239243582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9c8cd1b0 0x558e9cca7240 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:05.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.519+0000 7ff9d58af740 1 -- 172.21.15.100:0/2239243582 >> 172.21.15.100:0/2239243582 conn(0x558e9c8ea460 msgr2=0x558e9cc97c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:05.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.519+0000 7ff9b0ff9640 0 client.16722 ms_handle_reset on v2:172.21.15.138:6838/1580925471 2024-07-26T13:45:05.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.519+0000 7ff9d58af740 1 -- 172.21.15.100:0/2239243582 shutdown_connections 2024-07-26T13:45:05.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.519+0000 7ff9d58af740 1 -- 172.21.15.100:0/2239243582 wait complete. 2024-07-26T13:45:05.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.520+0000 7ff9d58af740 20 client.16722 trim_cache size 0 max 16384 2024-07-26T13:45:05.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.534+0000 7ff9d58af740 1 Processor -- start 2024-07-26T13:45:05.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.534+0000 7ff9d58af740 1 -- start start 2024-07-26T13:45:05.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.534+0000 7ff9d58af740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e9c8cd1b0 0x558e9cc9ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:05.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.534+0000 7ff9d58af740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9c8d5390 0x558e9cca07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:05.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.534+0000 7ff9d58af740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e9c8f6070 0x558e9cca0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:05.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.534+0000 7ff9d58af740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c81b3d00 con 0x558e9c8d5390 2024-07-26T13:45:05.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.534+0000 7ff9d58af740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c81b1570 con 0x558e9c8f6070 2024-07-26T13:45:05.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.534+0000 7ff9d58af740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c81af3a0 con 0x558e9c8cd1b0 2024-07-26T13:45:05.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.534+0000 7ff9cc8c6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9c8d5390 0x558e9cca07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:05.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.535+0000 7ff9cc8c6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9c8d5390 0x558e9cca07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35194/0 (socket says 172.21.15.100:35194) 2024-07-26T13:45:05.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.535+0000 7ff9cc8c6640 1 -- 172.21.15.100:0/2326930937 learned_addr learned my addr 172.21.15.100:0/2326930937 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:05.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.535+0000 7ff9cd8c8640 1 --2- 172.21.15.100:0/2326930937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e9c8f6070 0x558e9cca0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:05.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.535+0000 7ff9cd0c7640 1 --2- 172.21.15.100:0/2326930937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e9c8cd1b0 0x558e9cc9ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:05.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.535+0000 7ff9cc8c6640 1 -- 172.21.15.100:0/2326930937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e9c8cd1b0 msgr2=0x558e9cc9ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:05.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.535+0000 7ff9cc8c6640 1 --2- 172.21.15.100:0/2326930937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e9c8cd1b0 0x558e9cc9ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:05.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.535+0000 7ff9cc8c6640 1 -- 172.21.15.100:0/2326930937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e9c8f6070 msgr2=0x558e9cca0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:05.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.535+0000 7ff9cc8c6640 1 --2- 172.21.15.100:0/2326930937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e9c8f6070 0x558e9cca0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:05.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.535+0000 7ff9cc8c6640 1 -- 172.21.15.100:0/2326930937 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e9cc9d860 con 0x558e9c8d5390 2024-07-26T13:45:05.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.535+0000 7ff9cd8c8640 1 --2- 172.21.15.100:0/2326930937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e9c8f6070 0x558e9cca0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:45:05.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.535+0000 7ff9cc8c6640 1 --2- 172.21.15.100:0/2326930937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9c8d5390 0x558e9cca07f0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff9b80097b0 tx=0x7ff9b8004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b8f2e42bc2e48d server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:05.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.535+0000 7ff9b2ffd640 1 -- 172.21.15.100:0/2326930937 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b80c5070 con 0x558e9c8d5390 2024-07-26T13:45:05.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.535+0000 7ff9b2ffd640 1 -- 172.21.15.100:0/2326930937 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9b80e3040 con 0x558e9c8d5390 2024-07-26T13:45:05.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.535+0000 7ff9b2ffd640 1 -- 172.21.15.100:0/2326930937 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b80a7180 con 0x558e9c8d5390 2024-07-26T13:45:05.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.536+0000 7ff9d58af740 1 -- 172.21.15.100:0/2326930937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9c8d5390 msgr2=0x558e9cca07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:05.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.536+0000 7ff9d58af740 1 --2- 172.21.15.100:0/2326930937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9c8d5390 0x558e9cca07f0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff9b80097b0 tx=0x7ff9b8004a60 comp rx=0 tx=0).stop 2024-07-26T13:45:05.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.536+0000 7ff9d58af740 1 -- 172.21.15.100:0/2326930937 shutdown_connections 2024-07-26T13:45:05.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.536+0000 7ff9d58af740 1 --2- 172.21.15.100:0/2326930937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e9c8f6070 0x558e9cca0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:05.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.536+0000 7ff9d58af740 1 --2- 172.21.15.100:0/2326930937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9c8d5390 0x558e9cca07f0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:05.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.536+0000 7ff9d58af740 1 --2- 172.21.15.100:0/2326930937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e9c8cd1b0 0x558e9cc9ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:05.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.536+0000 7ff9d58af740 1 -- 172.21.15.100:0/2326930937 >> 172.21.15.100:0/2326930937 conn(0x558e9c8ea460 msgr2=0x558e9cc9c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:05.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.536+0000 7ff9d58af740 1 -- 172.21.15.100:0/2326930937 shutdown_connections 2024-07-26T13:45:05.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.536+0000 7ff9d58af740 1 -- 172.21.15.100:0/2326930937 wait complete. 2024-07-26T13:45:05.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.537+0000 7ff9d58af740 1 -- start start 2024-07-26T13:45:05.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.537+0000 7ff9d58af740 1 Processor -- start 2024-07-26T13:45:05.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.537+0000 7ff9d58af740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e9c8cd1b0 0x558e9ccbc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:05.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.537+0000 7ff9d58af740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9c8d5390 0x558e9ccb7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:05.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.537+0000 7ff9d58af740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e9c8f6070 0x558e9ccb80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:05.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.537+0000 7ff9d58af740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c81afa30 con 0x558e9c8d5390 2024-07-26T13:45:05.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.537+0000 7ff9d58af740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c81b0ea0 con 0x558e9c8f6070 2024-07-26T13:45:05.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.537+0000 7ff9d58af740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c81b0100 con 0x558e9c8cd1b0 2024-07-26T13:45:05.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.538+0000 7ff9d58af740 10 client.0 _ll_register_callbacks cb 0x7ff9c0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:45:05.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.538+0000 7ff9cc8c6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9c8d5390 0x558e9ccb7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:05.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.538+0000 7ff9cc8c6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9c8d5390 0x558e9ccb7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35204/0 (socket says 172.21.15.100:35204) 2024-07-26T13:45:05.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.538+0000 7ff9cc8c6640 1 -- 172.21.15.100:0/237639682 learned_addr learned my addr 172.21.15.100:0/237639682 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:05.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.538+0000 7ff9cd8c8640 1 --2- 172.21.15.100:0/237639682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e9c8f6070 0x558e9ccb80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:05.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.538+0000 7ff9cd0c7640 1 --2- 172.21.15.100:0/237639682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e9c8cd1b0 0x558e9ccbc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:05.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.538+0000 7ff9cc8c6640 1 -- 172.21.15.100:0/237639682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e9c8cd1b0 msgr2=0x558e9ccbc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:05.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.538+0000 7ff9cc8c6640 1 --2- 172.21.15.100:0/237639682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e9c8cd1b0 0x558e9ccbc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:05.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.538+0000 7ff9cc8c6640 1 -- 172.21.15.100:0/237639682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e9c8f6070 msgr2=0x558e9ccb80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:05.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.538+0000 7ff9cc8c6640 1 --2- 172.21.15.100:0/237639682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e9c8f6070 0x558e9ccb80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:05.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.538+0000 7ff9cc8c6640 1 -- 172.21.15.100:0/237639682 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e9ccb8760 con 0x558e9c8d5390 2024-07-26T13:45:05.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.538+0000 7ff9cd8c8640 1 --2- 172.21.15.100:0/237639682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e9c8f6070 0x558e9ccb80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:45:05.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.538+0000 7ff9cc8c6640 1 --2- 172.21.15.100:0/237639682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9c8d5390 0x558e9ccb7bb0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7ff9b80aabf0 tx=0x7ff9b80e1f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46873d7a978ec3da server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:05.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.538+0000 7ff9b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:45:05.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.538+0000 7ff9b0ff9640 1 -- 172.21.15.100:0/237639682 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b80c9020 con 0x558e9c8d5390 2024-07-26T13:45:05.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.538+0000 7ff9b0ff9640 1 -- 172.21.15.100:0/237639682 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7ff9b80046c0 con 0x558e9c8d5390 2024-07-26T13:45:05.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.538+0000 7ff9d58af740 1 -- 172.21.15.100:0/237639682 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9b80e3040 con 0x558e9c8d5390 2024-07-26T13:45:05.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.539+0000 7ff9b0ff9640 1 -- 172.21.15.100:0/237639682 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ff9b80c5040 con 0x558e9c8d5390 2024-07-26T13:45:05.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.539+0000 7ff9d58af740 10 client.16728 fetch_fsmap learned FSMap version 45 2024-07-26T13:45:05.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.539+0000 7ff9d58af740 1 -- 172.21.15.100:0/237639682 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x558e9cd21480 con 0x558e9c8d5390 2024-07-26T13:45:05.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.539+0000 7ff9b0ff9640 1 -- 172.21.15.100:0/237639682 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9b80e3040 con 0x558e9c8d5390 2024-07-26T13:45:05.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.539+0000 7ff9b0ff9640 1 -- 172.21.15.100:0/237639682 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7ff9b80b3ba0 con 0x558e9c8d5390 2024-07-26T13:45:05.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.539+0000 7ff9d58af740 10 client.16728 fetch_fsmap finished waiting for FSMap version 45 2024-07-26T13:45:05.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.539+0000 7ff9d58af740 10 client.16728 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-26T13:45:05.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.539+0000 7ff9d58af740 20 client.16728 populate_metadata read hostname 'smithi100' 2024-07-26T13:45:05.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.539+0000 7ff9d58af740 1 --2- 172.21.15.100:0/237639682 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x558e9c889da0 0x558e9cd23bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:05.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.539+0000 7ff9d58af740 4 client.16728 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-26T13:45:05.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.539+0000 7ff9d58af740 1 -- 172.21.15.100:0/237639682 --> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558e9cc0f5a0 con 0x558e9c889da0 2024-07-26T13:45:05.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.539+0000 7ff9cd0c7640 1 --2- 172.21.15.100:0/237639682 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x558e9c889da0 0x558e9cd23bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:05.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.540+0000 7ff9b0ff9640 10 client.16728 ms_handle_connect on v2:172.21.15.138:6838/1580925471 2024-07-26T13:45:05.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.540+0000 7ff9cd0c7640 1 --2- 172.21.15.100:0/237639682 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x558e9c889da0 0x558e9cd23bb0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b7728941a35b3c71 server_cookie=a671e9596758f84b in_seq=0 out_seq=0 2024-07-26T13:45:05.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.540+0000 7ff9b0ff9640 1 -- 172.21.15.100:0/237639682 <== mds.1 v2:172.21.15.138:6838/1580925471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558e9cc0f5a0 con 0x558e9c889da0 2024-07-26T13:45:05.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.540+0000 7ff9b0ff9640 10 client.16728 handle_command_reply: tid=0 2024-07-26T13:45:05.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.540+0000 7ff9d58af740 1 client.16728 shutdown 2024-07-26T13:45:05.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.540+0000 7ff9d58af740 1 -- 172.21.15.100:0/237639682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9c8d5390 msgr2=0x558e9ccb7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:05.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.540+0000 7ff9d58af740 1 --2- 172.21.15.100:0/237639682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9c8d5390 0x558e9ccb7bb0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7ff9b80aabf0 tx=0x7ff9b80e1f50 comp rx=0 tx=0).stop 2024-07-26T13:45:05.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.540+0000 7ff9d58af740 1 -- 172.21.15.100:0/237639682 shutdown_connections 2024-07-26T13:45:05.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.540+0000 7ff9d58af740 1 --2- 172.21.15.100:0/237639682 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x558e9c889da0 0x558e9cd23bb0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:05.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.540+0000 7ff9d58af740 1 --2- 172.21.15.100:0/237639682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e9c8f6070 0x558e9ccb80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:05.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.540+0000 7ff9d58af740 1 --2- 172.21.15.100:0/237639682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558e9c8d5390 0x558e9ccb7bb0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:05.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.540+0000 7ff9b0ff9640 0 client.16728 ms_handle_reset on v2:172.21.15.138:6838/1580925471 2024-07-26T13:45:05.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.540+0000 7ff9d58af740 1 --2- 172.21.15.100:0/237639682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e9c8cd1b0 0x558e9ccbc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:05.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.540+0000 7ff9d58af740 1 -- 172.21.15.100:0/237639682 >> 172.21.15.100:0/237639682 conn(0x558e9c8ea460 msgr2=0x558e9cc9e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:05.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.540+0000 7ff9d58af740 1 -- 172.21.15.100:0/237639682 shutdown_connections 2024-07-26T13:45:05.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.540+0000 7ff9d58af740 1 -- 172.21.15.100:0/237639682 wait complete. 2024-07-26T13:45:05.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.540+0000 7ff9d58af740 20 client.16728 trim_cache size 0 max 16384 2024-07-26T13:45:05.548 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-07-26T13:45:05.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.541+0000 7ff9cf32b640 1 -- 172.21.15.100:0/4081740779 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff9b40b3dc0 msgr2=0x7ff9b40b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:05.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.541+0000 7ff9cf32b640 1 --2- 172.21.15.100:0/4081740779 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff9b40b3dc0 0x7ff9b40b62d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0005fd0 tx=0x7ff9c0009500 comp rx=0 tx=0).stop 2024-07-26T13:45:05.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.541+0000 7ff9cf32b640 1 -- 172.21.15.100:0/4081740779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9c8107df0 msgr2=0x7ff9c8199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:05.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.541+0000 7ff9cf32b640 1 --2- 172.21.15.100:0/4081740779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9c8107df0 0x7ff9c8199330 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff9b800e9f0 tx=0x7ff9b800eeb0 comp rx=0 tx=0).stop 2024-07-26T13:45:05.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.541+0000 7ff9cf32b640 1 -- 172.21.15.100:0/4081740779 shutdown_connections 2024-07-26T13:45:05.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.541+0000 7ff9cf32b640 1 --2- 172.21.15.100:0/4081740779 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff9b40b3dc0 0x7ff9b40b62d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:05.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.541+0000 7ff9cf32b640 1 --2- 172.21.15.100:0/4081740779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9c8108740 0x7ff9c819e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:05.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.541+0000 7ff9cf32b640 1 --2- 172.21.15.100:0/4081740779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9c8107df0 0x7ff9c8199330 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:05.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.541+0000 7ff9cf32b640 1 --2- 172.21.15.100:0/4081740779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9c81024b0 0x7ff9c8198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:05.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.541+0000 7ff9cf32b640 1 -- 172.21.15.100:0/4081740779 >> 172.21.15.100:0/4081740779 conn(0x7ff9c8079170 msgr2=0x7ff9c8100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:05.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.541+0000 7ff9cf32b640 1 -- 172.21.15.100:0/4081740779 shutdown_connections 2024-07-26T13:45:05.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:05.541+0000 7ff9cf32b640 1 -- 172.21.15.100:0/4081740779 wait complete. 2024-07-26T13:45:05.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:05 smithi100 ceph-mon[32599]: pgmap v515: 97 pgs: 97 active+clean; 67 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-07-26T13:45:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:05 smithi138 ceph-mon[36954]: pgmap v515: 97 pgs: 97 active+clean; 67 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-07-26T13:45:06.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:05 smithi167 ceph-mon[37140]: pgmap v515: 97 pgs: 97 active+clean; 67 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-07-26T13:45:06.345 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-26T13:45:06.416 INFO:tasks.workunit.client.0.smithi100.stdout: 1 174625 1.92 MB/sec execute 80 sec latency 11.332 ms 2024-07-26T13:45:06.622 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:45:07.416 INFO:tasks.workunit.client.0.smithi100.stdout: 1 175223 1.95 MB/sec execute 81 sec latency 827.987 ms 2024-07-26T13:45:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:07 smithi100 ceph-mon[32599]: pgmap v516: 97 pgs: 97 active+clean; 71 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.3 MiB/s wr, 200 op/s 2024-07-26T13:45:07.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:07 smithi138 ceph-mon[36954]: pgmap v516: 97 pgs: 97 active+clean; 71 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.3 MiB/s wr, 200 op/s 2024-07-26T13:45:08.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:07 smithi167 ceph-mon[37140]: pgmap v516: 97 pgs: 97 active+clean; 71 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.3 MiB/s wr, 200 op/s 2024-07-26T13:45:08.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.362+0000 7f357a08a640 1 -- 172.21.15.100:0/2756405992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3574102490 msgr2=0x7f35741028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:08.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.362+0000 7f357a08a640 1 --2- 172.21.15.100:0/2756405992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3574102490 0x7f35741028a0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f355c009900 tx=0x7f355c02a410 comp rx=0 tx=0).stop 2024-07-26T13:45:08.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.362+0000 7f357a08a640 1 -- 172.21.15.100:0/2756405992 shutdown_connections 2024-07-26T13:45:08.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.362+0000 7f357a08a640 1 --2- 172.21.15.100:0/2756405992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3574108720 0x7f357410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.362+0000 7f357a08a640 1 --2- 172.21.15.100:0/2756405992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3574107dd0 0x7f35741081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.362+0000 7f357a08a640 1 --2- 172.21.15.100:0/2756405992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3574102490 0x7f35741028a0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.362+0000 7f357a08a640 1 -- 172.21.15.100:0/2756405992 >> 172.21.15.100:0/2756405992 conn(0x7f3574079160 msgr2=0x7f3574079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:08.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.362+0000 7f357a08a640 1 -- 172.21.15.100:0/2756405992 shutdown_connections 2024-07-26T13:45:08.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.362+0000 7f357a08a640 1 -- 172.21.15.100:0/2756405992 wait complete. 2024-07-26T13:45:08.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.362+0000 7f357a08a640 1 Processor -- start 2024-07-26T13:45:08.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.363+0000 7f357a08a640 1 -- start start 2024-07-26T13:45:08.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.363+0000 7f357a08a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3574102490 0x7f357419ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:08.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.363+0000 7f357a08a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3574107dd0 0x7f357419b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:08.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.363+0000 7f357a08a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3574108720 0x7f357419b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:08.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.363+0000 7f357a08a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3574112420 con 0x7f3574102490 2024-07-26T13:45:08.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.363+0000 7f357a08a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f35741122a0 con 0x7f3574107dd0 2024-07-26T13:45:08.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.363+0000 7f357a08a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f35741125a0 con 0x7f3574108720 2024-07-26T13:45:08.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.363+0000 7f35737fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3574102490 0x7f357419ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:08.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.363+0000 7f35737fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3574102490 0x7f357419ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35220/0 (socket says 172.21.15.100:35220) 2024-07-26T13:45:08.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.363+0000 7f35737fe640 1 -- 172.21.15.100:0/3897294457 learned_addr learned my addr 172.21.15.100:0/3897294457 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:08.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.363+0000 7f3572ffd640 1 --2- 172.21.15.100:0/3897294457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3574107dd0 0x7f357419b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:08.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.363+0000 7f3573fff640 1 --2- 172.21.15.100:0/3897294457 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3574108720 0x7f357419b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:08.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.363+0000 7f35737fe640 1 -- 172.21.15.100:0/3897294457 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3574108720 msgr2=0x7f357419b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:08.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.363+0000 7f35737fe640 1 --2- 172.21.15.100:0/3897294457 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3574108720 0x7f357419b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.363+0000 7f35737fe640 1 -- 172.21.15.100:0/3897294457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3574107dd0 msgr2=0x7f357419b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:08.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.363+0000 7f35737fe640 1 --2- 172.21.15.100:0/3897294457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3574107dd0 0x7f357419b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.364+0000 7f35737fe640 1 -- 172.21.15.100:0/3897294457 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35741a5fe0 con 0x7f3574102490 2024-07-26T13:45:08.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.364+0000 7f35737fe640 1 --2- 172.21.15.100:0/3897294457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3574102490 0x7f357419ad50 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f355c005940 tx=0x7f355c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:08.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.364+0000 7f3570ff9640 1 -- 172.21.15.100:0/3897294457 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c03f030 con 0x7f3574102490 2024-07-26T13:45:08.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.364+0000 7f357a08a640 1 -- 172.21.15.100:0/3897294457 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35741a6270 con 0x7f3574102490 2024-07-26T13:45:08.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.364+0000 7f3570ff9640 1 -- 172.21.15.100:0/3897294457 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f355c003cb0 con 0x7f3574102490 2024-07-26T13:45:08.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.364+0000 7f3570ff9640 1 -- 172.21.15.100:0/3897294457 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c035070 con 0x7f3574102490 2024-07-26T13:45:08.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.364+0000 7f357a08a640 1 -- 172.21.15.100:0/3897294457 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35741a6820 con 0x7f3574102490 2024-07-26T13:45:08.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.366+0000 7f3570ff9640 1 -- 172.21.15.100:0/3897294457 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f355c042030 con 0x7f3574102490 2024-07-26T13:45:08.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.367+0000 7f358060e740 1 Processor -- start 2024-07-26T13:45:08.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.367+0000 7f358060e740 1 -- start start 2024-07-26T13:45:08.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.367+0000 7f358060e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559bae33e070 0x559bae6e3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:08.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.367+0000 7f358060e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bae3151b0 0x559bae6e3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:08.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.367+0000 7f358060e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bae31d390 0x559bae6e3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:08.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.367+0000 7f358060e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x559bae659bb0 con 0x559bae33e070 2024-07-26T13:45:08.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.367+0000 7f358060e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559bae6490b0 con 0x559bae31d390 2024-07-26T13:45:08.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.367+0000 7f358060e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559bae64d0f0 con 0x559bae3151b0 2024-07-26T13:45:08.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.367+0000 7f3573fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559bae33e070 0x559bae6e3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:08.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.367+0000 7f3573fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559bae33e070 0x559bae6e3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35222/0 (socket says 172.21.15.100:35222) 2024-07-26T13:45:08.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.367+0000 7f3573fff640 1 -- 172.21.15.100:0/3101267286 learned_addr learned my addr 172.21.15.100:0/3101267286 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:08.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.367+0000 7f3572ffd640 1 --2- 172.21.15.100:0/3101267286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bae31d390 0x559bae6e3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:08.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.367+0000 7f35737fe640 1 --2- 172.21.15.100:0/3101267286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bae3151b0 0x559bae6e3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:08.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.367+0000 7f3573fff640 1 -- 172.21.15.100:0/3101267286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bae3151b0 msgr2=0x559bae6e3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:08.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.367+0000 7f3573fff640 1 --2- 172.21.15.100:0/3101267286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bae3151b0 0x559bae6e3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.367+0000 7f3573fff640 1 -- 172.21.15.100:0/3101267286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bae31d390 msgr2=0x559bae6e3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:08.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.367+0000 7f3573fff640 1 --2- 172.21.15.100:0/3101267286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bae31d390 0x559bae6e3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.367+0000 7f3573fff640 1 -- 172.21.15.100:0/3101267286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bae5efb90 con 0x559bae33e070 2024-07-26T13:45:08.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.367+0000 7f35737fe640 1 --2- 172.21.15.100:0/3101267286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bae3151b0 0x559bae6e3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:45:08.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.367+0000 7f3570ff9640 1 --2- 172.21.15.100:0/3897294457 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f35600b3d70 0x7f35600b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:08.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.368+0000 7f35737fe640 1 --2- 172.21.15.100:0/3897294457 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f35600b3d70 0x7f35600b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:08.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.368+0000 7f3573fff640 1 --2- 172.21.15.100:0/3101267286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559bae33e070 0x559bae6e3570 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f356c009f20 tx=0x7f356c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4dc5f628c86c746 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:08.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.368+0000 7f3570ff9640 1 -- 172.21.15.100:0/3897294457 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7f355c104670 con 0x7f3574102490 2024-07-26T13:45:08.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.368+0000 7f3569ffb640 1 -- 172.21.15.100:0/3101267286 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356c01a070 con 0x559bae33e070 2024-07-26T13:45:08.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.368+0000 7f3569ffb640 1 -- 172.21.15.100:0/3101267286 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f356c0158c0 con 0x559bae33e070 2024-07-26T13:45:08.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.368+0000 7f3569ffb640 1 -- 172.21.15.100:0/3101267286 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356c00a820 con 0x559bae33e070 2024-07-26T13:45:08.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.369+0000 7f358060e740 1 -- 172.21.15.100:0/3101267286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559bae33e070 msgr2=0x559bae6e3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:08.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.369+0000 7f358060e740 1 --2- 172.21.15.100:0/3101267286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559bae33e070 0x559bae6e3570 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f356c009f20 tx=0x7f356c00c8b0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.369+0000 7f35737fe640 1 --2- 172.21.15.100:0/3897294457 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f35600b3d70 0x7f35600b6280 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x559bae6e21f0 tx=0x7f355c040000 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:08.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.369+0000 7f358060e740 1 -- 172.21.15.100:0/3101267286 shutdown_connections 2024-07-26T13:45:08.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.369+0000 7f358060e740 1 --2- 172.21.15.100:0/3101267286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bae31d390 0x559bae6e3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.369+0000 7f358060e740 1 --2- 172.21.15.100:0/3101267286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bae3151b0 0x559bae6e3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.369+0000 7f358060e740 1 --2- 172.21.15.100:0/3101267286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559bae33e070 0x559bae6e3570 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.369+0000 7f358060e740 1 -- 172.21.15.100:0/3101267286 >> 172.21.15.100:0/3101267286 conn(0x559bae332460 msgr2=0x559bae6e1460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:08.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.369+0000 7f358060e740 1 -- 172.21.15.100:0/3101267286 shutdown_connections 2024-07-26T13:45:08.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.369+0000 7f358060e740 1 -- 172.21.15.100:0/3101267286 wait complete. 2024-07-26T13:45:08.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.369+0000 7f358060e740 1 -- start start 2024-07-26T13:45:08.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.369+0000 7f358060e740 1 Processor -- start 2024-07-26T13:45:08.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.370+0000 7f358060e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559bae3151b0 0x559bae6ef160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:08.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.370+0000 7f358060e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bae31d390 0x559bae6ef6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:08.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.370+0000 7f358060e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bae33e070 0x559bae6f5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:08.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.370+0000 7f358060e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x559bae4ac700 con 0x559bae3151b0 2024-07-26T13:45:08.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.370+0000 7f358060e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559bae5eb9e0 con 0x559bae31d390 2024-07-26T13:45:08.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.370+0000 7f358060e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559bae6f1d50 con 0x559bae33e070 2024-07-26T13:45:08.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.370+0000 7f358060e740 10 client.0 _ll_register_callbacks cb 0x559bae6064f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:45:08.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.370+0000 7f3573fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559bae3151b0 0x559bae6ef160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:08.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.370+0000 7f3573fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559bae3151b0 0x559bae6ef160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35228/0 (socket says 172.21.15.100:35228) 2024-07-26T13:45:08.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.370+0000 7f3573fff640 1 -- 172.21.15.100:0/1109962207 learned_addr learned my addr 172.21.15.100:0/1109962207 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:08.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.370+0000 7f3572ffd640 1 --2- 172.21.15.100:0/1109962207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bae31d390 0x559bae6ef6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:08.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.370+0000 7f3573fff640 1 --2- 172.21.15.100:0/1109962207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bae33e070 0x559bae6f5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:08.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.370+0000 7f3573fff640 1 -- 172.21.15.100:0/1109962207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bae33e070 msgr2=0x559bae6f5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:08.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.370+0000 7f3573fff640 1 --2- 172.21.15.100:0/1109962207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bae33e070 0x559bae6f5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.370+0000 7f3573fff640 1 -- 172.21.15.100:0/1109962207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bae31d390 msgr2=0x559bae6ef6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:08.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.370+0000 7f3573fff640 1 --2- 172.21.15.100:0/1109962207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bae31d390 0x559bae6ef6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.370+0000 7f3573fff640 1 -- 172.21.15.100:0/1109962207 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bae3a9650 con 0x559bae3151b0 2024-07-26T13:45:08.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.370+0000 7f3573fff640 1 --2- 172.21.15.100:0/1109962207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559bae3151b0 0x559bae6ef160 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f356c018040 tx=0x7f356c00a400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7766af940f8a048 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:08.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.370+0000 7f3552ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:45:08.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.370+0000 7f3552ffd640 1 -- 172.21.15.100:0/1109962207 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356c01a050 con 0x559bae3151b0 2024-07-26T13:45:08.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.371+0000 7f3552ffd640 1 -- 172.21.15.100:0/1109962207 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7f356c015d80 con 0x559bae3151b0 2024-07-26T13:45:08.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.371+0000 7f358060e740 1 -- 172.21.15.100:0/1109962207 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f356c0158c0 con 0x559bae3151b0 2024-07-26T13:45:08.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.371+0000 7f3552ffd640 1 -- 172.21.15.100:0/1109962207 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f356c030050 con 0x559bae3151b0 2024-07-26T13:45:08.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.371+0000 7f358060e740 10 client.16740 fetch_fsmap learned FSMap version 45 2024-07-26T13:45:08.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.371+0000 7f358060e740 1 -- 172.21.15.100:0/1109962207 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x559bae7065d0 con 0x559bae3151b0 2024-07-26T13:45:08.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.371+0000 7f3552ffd640 1 -- 172.21.15.100:0/1109962207 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f356c0158c0 con 0x559bae3151b0 2024-07-26T13:45:08.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.371+0000 7f3552ffd640 1 -- 172.21.15.100:0/1109962207 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f356c022880 con 0x559bae3151b0 2024-07-26T13:45:08.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.371+0000 7f358060e740 10 client.16740 fetch_fsmap finished waiting for FSMap version 45 2024-07-26T13:45:08.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.371+0000 7f358060e740 10 client.16740 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:45:08.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.371+0000 7f358060e740 20 client.16740 populate_metadata read hostname 'smithi100' 2024-07-26T13:45:08.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.371+0000 7f358060e740 1 --2- 172.21.15.100:0/1109962207 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559bae2fc9f0 0x559bae709a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:08.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.371+0000 7f358060e740 4 client.16740 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:45:08.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.371+0000 7f358060e740 1 -- 172.21.15.100:0/1109962207 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559bae65bb00 con 0x559bae2fc9f0 2024-07-26T13:45:08.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.371+0000 7f3572ffd640 1 --2- 172.21.15.100:0/1109962207 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559bae2fc9f0 0x559bae709a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:08.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.372+0000 7f3572ffd640 1 --2- 172.21.15.100:0/1109962207 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559bae2fc9f0 0x559bae709a40 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1c1110d6d8c3e0c server_cookie=c4b56de833b2d753 in_seq=0 out_seq=0 2024-07-26T13:45:08.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.372+0000 7f3552ffd640 10 client.16740 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:08.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.373+0000 7f3552ffd640 1 -- 172.21.15.100:0/1109962207 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559bae65bb00 con 0x559bae2fc9f0 2024-07-26T13:45:08.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.373+0000 7f3552ffd640 10 client.16740 handle_command_reply: tid=0 2024-07-26T13:45:08.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.373+0000 7f358060e740 1 client.16740 shutdown 2024-07-26T13:45:08.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.373+0000 7f358060e740 1 -- 172.21.15.100:0/1109962207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559bae3151b0 msgr2=0x559bae6ef160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:08.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.373+0000 7f358060e740 1 --2- 172.21.15.100:0/1109962207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559bae3151b0 0x559bae6ef160 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f356c018040 tx=0x7f356c00a400 comp rx=0 tx=0).stop 2024-07-26T13:45:08.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.373+0000 7f358060e740 1 -- 172.21.15.100:0/1109962207 shutdown_connections 2024-07-26T13:45:08.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.373+0000 7f358060e740 1 --2- 172.21.15.100:0/1109962207 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559bae2fc9f0 0x559bae709a40 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.373+0000 7f358060e740 1 --2- 172.21.15.100:0/1109962207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bae33e070 0x559bae6f5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.373+0000 7f358060e740 1 --2- 172.21.15.100:0/1109962207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bae31d390 0x559bae6ef6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.373+0000 7f3552ffd640 0 client.16740 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:08.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.373+0000 7f358060e740 1 --2- 172.21.15.100:0/1109962207 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559bae3151b0 0x559bae6ef160 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.373+0000 7f358060e740 1 -- 172.21.15.100:0/1109962207 >> 172.21.15.100:0/1109962207 conn(0x559bae332460 msgr2=0x559bae6dfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:08.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.373+0000 7f358060e740 1 -- 172.21.15.100:0/1109962207 shutdown_connections 2024-07-26T13:45:08.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.373+0000 7f358060e740 1 -- 172.21.15.100:0/1109962207 wait complete. 2024-07-26T13:45:08.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.373+0000 7f358060e740 20 client.16740 trim_cache size 0 max 16384 2024-07-26T13:45:08.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.386+0000 7f358060e740 1 Processor -- start 2024-07-26T13:45:08.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.386+0000 7f358060e740 1 -- start start 2024-07-26T13:45:08.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.386+0000 7f358060e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559bae2fc9f0 0x559bae6f4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:08.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.386+0000 7f358060e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bae3151b0 0x559bae6f0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:08.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.386+0000 7f358060e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bae31d390 0x559bae70f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:08.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.386+0000 7f358060e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f35741afa70 con 0x559bae2fc9f0 2024-07-26T13:45:08.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.386+0000 7f358060e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f35741ad2e0 con 0x559bae31d390 2024-07-26T13:45:08.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.386+0000 7f358060e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f35741ab110 con 0x559bae3151b0 2024-07-26T13:45:08.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.386+0000 7f3573fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559bae2fc9f0 0x559bae6f4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:08.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.386+0000 7f3573fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559bae2fc9f0 0x559bae6f4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35230/0 (socket says 172.21.15.100:35230) 2024-07-26T13:45:08.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.386+0000 7f3573fff640 1 -- 172.21.15.100:0/2671678544 learned_addr learned my addr 172.21.15.100:0/2671678544 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:08.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.387+0000 7f3572ffd640 1 --2- 172.21.15.100:0/2671678544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bae3151b0 0x559bae6f0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:08.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.387+0000 7f3573fff640 1 -- 172.21.15.100:0/2671678544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bae3151b0 msgr2=0x559bae6f0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:08.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.387+0000 7f3573fff640 1 --2- 172.21.15.100:0/2671678544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bae3151b0 0x559bae6f0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.387+0000 7f3573fff640 1 -- 172.21.15.100:0/2671678544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bae31d390 msgr2=0x559bae70f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:08.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.387+0000 7f3573fff640 1 --2- 172.21.15.100:0/2671678544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bae31d390 0x559bae70f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.387+0000 7f3573fff640 1 -- 172.21.15.100:0/2671678544 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bae6e63d0 con 0x559bae2fc9f0 2024-07-26T13:45:08.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.387+0000 7f3573fff640 1 --2- 172.21.15.100:0/2671678544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559bae2fc9f0 0x559bae6f4f80 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f356c0098f0 tx=0x7f356c009c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8815356430ac913 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:08.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.387+0000 7f35697fa640 1 -- 172.21.15.100:0/2671678544 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356c01a030 con 0x559bae2fc9f0 2024-07-26T13:45:08.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.387+0000 7f35697fa640 1 -- 172.21.15.100:0/2671678544 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f356c027070 con 0x559bae2fc9f0 2024-07-26T13:45:08.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.387+0000 7f35697fa640 1 -- 172.21.15.100:0/2671678544 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356c00a7b0 con 0x559bae2fc9f0 2024-07-26T13:45:08.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.388+0000 7f358060e740 1 -- 172.21.15.100:0/2671678544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559bae2fc9f0 msgr2=0x559bae6f4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:08.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.388+0000 7f358060e740 1 --2- 172.21.15.100:0/2671678544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559bae2fc9f0 0x559bae6f4f80 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f356c0098f0 tx=0x7f356c009c70 comp rx=0 tx=0).stop 2024-07-26T13:45:08.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.388+0000 7f358060e740 1 -- 172.21.15.100:0/2671678544 shutdown_connections 2024-07-26T13:45:08.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.388+0000 7f358060e740 1 --2- 172.21.15.100:0/2671678544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bae31d390 0x559bae70f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.388+0000 7f358060e740 1 --2- 172.21.15.100:0/2671678544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bae3151b0 0x559bae6f0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.388+0000 7f358060e740 1 --2- 172.21.15.100:0/2671678544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559bae2fc9f0 0x559bae6f4f80 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.388+0000 7f358060e740 1 -- 172.21.15.100:0/2671678544 >> 172.21.15.100:0/2671678544 conn(0x559bae332460 msgr2=0x559bae6f0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:08.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.388+0000 7f358060e740 1 -- 172.21.15.100:0/2671678544 shutdown_connections 2024-07-26T13:45:08.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.388+0000 7f358060e740 1 -- 172.21.15.100:0/2671678544 wait complete. 2024-07-26T13:45:08.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.388+0000 7f358060e740 1 -- start start 2024-07-26T13:45:08.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.388+0000 7f358060e740 1 Processor -- start 2024-07-26T13:45:08.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.389+0000 7f358060e740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559bae2fc9f0 0x559bae6ff7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:08.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.389+0000 7f358060e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bae3151b0 0x559bae6ffdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:08.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.389+0000 7f358060e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bae31d390 0x559bae76a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:08.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.389+0000 7f358060e740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f35741ab7a0 con 0x559bae2fc9f0 2024-07-26T13:45:08.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.389+0000 7f358060e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f35741acc10 con 0x559bae3151b0 2024-07-26T13:45:08.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.389+0000 7f358060e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f35741abe70 con 0x559bae31d390 2024-07-26T13:45:08.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.389+0000 7f358060e740 10 client.0 _ll_register_callbacks cb 0x7f356c018070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:45:08.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.389+0000 7f3573fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559bae2fc9f0 0x559bae6ff7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:08.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.389+0000 7f3573fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559bae2fc9f0 0x559bae6ff7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35234/0 (socket says 172.21.15.100:35234) 2024-07-26T13:45:08.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.389+0000 7f3573fff640 1 -- 172.21.15.100:0/3713039425 learned_addr learned my addr 172.21.15.100:0/3713039425 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:08.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.389+0000 7f3572ffd640 1 --2- 172.21.15.100:0/3713039425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bae3151b0 0x559bae6ffdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:08.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.389+0000 7f3573fff640 1 -- 172.21.15.100:0/3713039425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bae31d390 msgr2=0x559bae76a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:08.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.389+0000 7f3573fff640 1 --2- 172.21.15.100:0/3713039425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bae31d390 0x559bae76a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.389+0000 7f3573fff640 1 -- 172.21.15.100:0/3713039425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bae3151b0 msgr2=0x559bae6ffdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:08.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.389+0000 7f3573fff640 1 --2- 172.21.15.100:0/3713039425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bae3151b0 0x559bae6ffdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.389+0000 7f3573fff640 1 -- 172.21.15.100:0/3713039425 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bae76a710 con 0x559bae2fc9f0 2024-07-26T13:45:08.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.389+0000 7f3573fff640 1 --2- 172.21.15.100:0/3713039425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559bae2fc9f0 0x559bae6ff7d0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f356c022dc0 tx=0x7f356c00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab91ecd1c460663c server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:08.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.389+0000 7f3552ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:45:08.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.390+0000 7f3552ffd640 1 -- 172.21.15.100:0/3713039425 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356c01a040 con 0x559bae2fc9f0 2024-07-26T13:45:08.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.390+0000 7f3552ffd640 1 -- 172.21.15.100:0/3713039425 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7f356c015c80 con 0x559bae2fc9f0 2024-07-26T13:45:08.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.390+0000 7f358060e740 1 -- 172.21.15.100:0/3713039425 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f356c027070 con 0x559bae2fc9f0 2024-07-26T13:45:08.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.390+0000 7f3552ffd640 1 -- 172.21.15.100:0/3713039425 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f356c022440 con 0x559bae2fc9f0 2024-07-26T13:45:08.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.390+0000 7f358060e740 10 client.16746 fetch_fsmap learned FSMap version 45 2024-07-26T13:45:08.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.390+0000 7f358060e740 1 -- 172.21.15.100:0/3713039425 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x559bae771b00 con 0x559bae2fc9f0 2024-07-26T13:45:08.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.390+0000 7f3552ffd640 1 -- 172.21.15.100:0/3713039425 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f356c027070 con 0x559bae2fc9f0 2024-07-26T13:45:08.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.390+0000 7f3552ffd640 1 -- 172.21.15.100:0/3713039425 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f356c00ac00 con 0x559bae2fc9f0 2024-07-26T13:45:08.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.390+0000 7f358060e740 10 client.16746 fetch_fsmap finished waiting for FSMap version 45 2024-07-26T13:45:08.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.390+0000 7f358060e740 10 client.16746 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:45:08.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.390+0000 7f358060e740 20 client.16746 populate_metadata read hostname 'smithi100' 2024-07-26T13:45:08.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.390+0000 7f358060e740 1 --2- 172.21.15.100:0/3713039425 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559bae33e070 0x559bae774250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:08.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.390+0000 7f358060e740 4 client.16746 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-26T13:45:08.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.390+0000 7f358060e740 1 -- 172.21.15.100:0/3713039425 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559bae6575a0 con 0x559bae33e070 2024-07-26T13:45:08.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.391+0000 7f3572ffd640 1 --2- 172.21.15.100:0/3713039425 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559bae33e070 0x559bae774250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:08.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.391+0000 7f3552ffd640 10 client.16746 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:08.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.391+0000 7f3572ffd640 1 --2- 172.21.15.100:0/3713039425 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559bae33e070 0x559bae774250 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b21a22175b02a52c server_cookie=6cb2c158e49e2678 in_seq=0 out_seq=0 2024-07-26T13:45:08.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.391+0000 7f3552ffd640 1 -- 172.21.15.100:0/3713039425 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559bae6575a0 con 0x559bae33e070 2024-07-26T13:45:08.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.391+0000 7f3552ffd640 10 client.16746 handle_command_reply: tid=0 2024-07-26T13:45:08.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.391+0000 7f358060e740 1 client.16746 shutdown 2024-07-26T13:45:08.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.391+0000 7f358060e740 1 -- 172.21.15.100:0/3713039425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559bae2fc9f0 msgr2=0x559bae6ff7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:08.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.391+0000 7f358060e740 1 --2- 172.21.15.100:0/3713039425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559bae2fc9f0 0x559bae6ff7d0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f356c022dc0 tx=0x7f356c00cdb0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.391+0000 7f358060e740 1 -- 172.21.15.100:0/3713039425 shutdown_connections 2024-07-26T13:45:08.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.391+0000 7f358060e740 1 --2- 172.21.15.100:0/3713039425 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x559bae33e070 0x559bae774250 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.391+0000 7f358060e740 1 --2- 172.21.15.100:0/3713039425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559bae31d390 0x559bae76a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.391+0000 7f358060e740 1 --2- 172.21.15.100:0/3713039425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559bae3151b0 0x559bae6ffdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.391+0000 7f3552ffd640 0 client.16746 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:08.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.391+0000 7f358060e740 1 --2- 172.21.15.100:0/3713039425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x559bae2fc9f0 0x559bae6ff7d0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.391+0000 7f358060e740 1 -- 172.21.15.100:0/3713039425 >> 172.21.15.100:0/3713039425 conn(0x559bae332460 msgr2=0x559bae6e4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:08.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.392+0000 7f358060e740 1 -- 172.21.15.100:0/3713039425 shutdown_connections 2024-07-26T13:45:08.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.392+0000 7f358060e740 1 -- 172.21.15.100:0/3713039425 wait complete. 2024-07-26T13:45:08.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.392+0000 7f358060e740 20 client.16746 trim_cache size 0 max 16384 2024-07-26T13:45:08.415 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-07-26T13:45:08.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.392+0000 7f357a08a640 1 -- 172.21.15.100:0/3897294457 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f35600b3d70 msgr2=0x7f35600b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:08.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.392+0000 7f357a08a640 1 --2- 172.21.15.100:0/3897294457 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f35600b3d70 0x7f35600b6280 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x559bae6e21f0 tx=0x7f355c040000 comp rx=0 tx=0).stop 2024-07-26T13:45:08.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.392+0000 7f357a08a640 1 -- 172.21.15.100:0/3897294457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3574102490 msgr2=0x7f357419ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:08.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.392+0000 7f357a08a640 1 --2- 172.21.15.100:0/3897294457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3574102490 0x7f357419ad50 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f355c005940 tx=0x7f355c0042f0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.392+0000 7f357a08a640 1 -- 172.21.15.100:0/3897294457 shutdown_connections 2024-07-26T13:45:08.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.392+0000 7f357a08a640 1 --2- 172.21.15.100:0/3897294457 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f35600b3d70 0x7f35600b6280 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.392+0000 7f357a08a640 1 --2- 172.21.15.100:0/3897294457 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3574108720 0x7f357419b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.392+0000 7f357a08a640 1 --2- 172.21.15.100:0/3897294457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3574107dd0 0x7f357419b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.392+0000 7f357a08a640 1 --2- 172.21.15.100:0/3897294457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3574102490 0x7f357419ad50 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.392+0000 7f357a08a640 1 -- 172.21.15.100:0/3897294457 >> 172.21.15.100:0/3897294457 conn(0x7f3574079160 msgr2=0x7f3574100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:08.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.393+0000 7f357a08a640 1 -- 172.21.15.100:0/3897294457 shutdown_connections 2024-07-26T13:45:08.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:08.393+0000 7f357a08a640 1 -- 172.21.15.100:0/3897294457 wait complete. 2024-07-26T13:45:08.417 INFO:tasks.workunit.client.0.smithi100.stdout: 1 175223 1.93 MB/sec execute 82 sec latency 1828.013 ms 2024-07-26T13:45:08.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.699+0000 7f43e35c8640 1 -- 172.21.15.100:0/2784590183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43dc105c30 msgr2=0x7f43dc106040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:08.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.699+0000 7f43e35c8640 1 --2- 172.21.15.100:0/2784590183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43dc105c30 0x7f43dc106040 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009900 tx=0x7f43cc02a410 comp rx=0 tx=0).stop 2024-07-26T13:45:08.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.699+0000 7f43e35c8640 1 -- 172.21.15.100:0/2784590183 shutdown_connections 2024-07-26T13:45:08.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.699+0000 7f43e35c8640 1 --2- 172.21.15.100:0/2784590183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43dc106580 0x7f43dc10ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.699+0000 7f43e35c8640 1 --2- 172.21.15.100:0/2784590183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43dc105c30 0x7f43dc106040 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.699+0000 7f43e35c8640 1 --2- 172.21.15.100:0/2784590183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43dc101ac0 0x7f43dc101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.699+0000 7f43e35c8640 1 -- 172.21.15.100:0/2784590183 >> 172.21.15.100:0/2784590183 conn(0x7f43dc0798f0 msgr2=0x7f43dc079d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:08.703 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.699+0000 7f43e35c8640 1 -- 172.21.15.100:0/2784590183 shutdown_connections 2024-07-26T13:45:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.699+0000 7f43e35c8640 1 -- 172.21.15.100:0/2784590183 wait complete. 2024-07-26T13:45:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.700+0000 7f43e35c8640 1 Processor -- start 2024-07-26T13:45:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.700+0000 7f43e35c8640 1 -- start start 2024-07-26T13:45:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.700+0000 7f43e35c8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43dc101ac0 0x7f43dc199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.700+0000 7f43e35c8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43dc105c30 0x7f43dc19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.700+0000 7f43e35c8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43dc106580 0x7f43dc19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.700+0000 7f43e35c8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc113440 con 0x7f43dc105c30 2024-07-26T13:45:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.700+0000 7f43e35c8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc1132c0 con 0x7f43dc101ac0 2024-07-26T13:45:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.700+0000 7f43e35c8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc1135c0 con 0x7f43dc106580 2024-07-26T13:45:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.700+0000 7f43e0b63640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43dc105c30 0x7f43dc19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:08.704 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.700+0000 7f43e0b63640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43dc105c30 0x7f43dc19a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35240/0 (socket says 172.21.15.100:35240) 2024-07-26T13:45:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.700+0000 7f43e0b63640 1 -- 172.21.15.100:0/156254577 learned_addr learned my addr 172.21.15.100:0/156254577 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.701+0000 7f43e1b65640 1 --2- 172.21.15.100:0/156254577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43dc106580 0x7f43dc19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.701+0000 7f43e1364640 1 --2- 172.21.15.100:0/156254577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43dc101ac0 0x7f43dc199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.701+0000 7f43e0b63640 1 -- 172.21.15.100:0/156254577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43dc106580 msgr2=0x7f43dc19f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.701+0000 7f43e0b63640 1 --2- 172.21.15.100:0/156254577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43dc106580 0x7f43dc19f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.701+0000 7f43e0b63640 1 -- 172.21.15.100:0/156254577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43dc101ac0 msgr2=0x7f43dc199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.701+0000 7f43e0b63640 1 --2- 172.21.15.100:0/156254577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43dc101ac0 0x7f43dc199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.701+0000 7f43e0b63640 1 -- 172.21.15.100:0/156254577 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43dc19fdb0 con 0x7f43dc105c30 2024-07-26T13:45:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.701+0000 7f43e1364640 1 --2- 172.21.15.100:0/156254577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43dc101ac0 0x7f43dc199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:45:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.701+0000 7f43e0b63640 1 --2- 172.21.15.100:0/156254577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43dc105c30 0x7f43dc19a420 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f43cc02a930 tx=0x7f43cc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.701+0000 7f43d27fc640 1 -- 172.21.15.100:0/156254577 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc03f030 con 0x7f43dc105c30 2024-07-26T13:45:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.701+0000 7f43e35c8640 1 -- 172.21.15.100:0/156254577 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43dc1a0040 con 0x7f43dc105c30 2024-07-26T13:45:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.701+0000 7f43d27fc640 1 -- 172.21.15.100:0/156254577 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43cc003aa0 con 0x7f43dc105c30 2024-07-26T13:45:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.701+0000 7f43d27fc640 1 -- 172.21.15.100:0/156254577 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc035070 con 0x7f43dc105c30 2024-07-26T13:45:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.701+0000 7f43e35c8640 1 -- 172.21.15.100:0/156254577 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43dc1a6830 con 0x7f43dc105c30 2024-07-26T13:45:08.713 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.704+0000 7f43ea5b7740 1 Processor -- start 2024-07-26T13:45:08.713 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.704+0000 7f43ea5b7740 1 -- start start 2024-07-26T13:45:08.713 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.704+0000 7f43d27fc640 1 -- 172.21.15.100:0/156254577 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f43cc042030 con 0x7f43dc105c30 2024-07-26T13:45:08.713 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.704+0000 7f43ea5b7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aceaf6e1b0 0x55aceb3106b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:08.713 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.704+0000 7f43ea5b7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aceaf452f0 0x55aceb310bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:08.713 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.704+0000 7f43ea5b7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aceaf4d4d0 0x55aceb311130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.704+0000 7f43ea5b7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55aceb2863b0 con 0x55aceaf4d4d0 2024-07-26T13:45:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.704+0000 7f43ea5b7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55aceb2758b0 con 0x55aceaf6e1b0 2024-07-26T13:45:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.704+0000 7f43ea5b7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55aceb2798f0 con 0x55aceaf452f0 2024-07-26T13:45:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.704+0000 7f43e0b63640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aceaf4d4d0 0x55aceb311130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.704+0000 7f43e0b63640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aceaf4d4d0 0x55aceb311130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35246/0 (socket says 172.21.15.100:35246) 2024-07-26T13:45:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.704+0000 7f43e0b63640 1 -- 172.21.15.100:0/240637968 learned_addr learned my addr 172.21.15.100:0/240637968 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.704+0000 7f43e1364640 1 --2- 172.21.15.100:0/240637968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aceaf452f0 0x55aceb310bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.704+0000 7f43e1b65640 1 --2- 172.21.15.100:0/240637968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aceaf6e1b0 0x55aceb3106b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.705+0000 7f43e0b63640 1 -- 172.21.15.100:0/240637968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aceaf452f0 msgr2=0x55aceb310bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.705+0000 7f43e0b63640 1 --2- 172.21.15.100:0/240637968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aceaf452f0 0x55aceb310bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.705+0000 7f43e0b63640 1 -- 172.21.15.100:0/240637968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aceaf6e1b0 msgr2=0x55aceb3106b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.705+0000 7f43e0b63640 1 --2- 172.21.15.100:0/240637968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aceaf6e1b0 0x55aceb3106b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.705+0000 7f43e0b63640 1 -- 172.21.15.100:0/240637968 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aceb21c9a0 con 0x55aceaf4d4d0 2024-07-26T13:45:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.705+0000 7f43d27fc640 1 --2- 172.21.15.100:0/156254577 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f43c80b3d40 0x7f43c80b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.705+0000 7f43e1b65640 1 --2- 172.21.15.100:0/156254577 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f43c80b3d40 0x7f43c80b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.705+0000 7f43e0b63640 1 --2- 172.21.15.100:0/240637968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aceaf4d4d0 0x55aceb311130 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f43cc101d90 tx=0x7f43cc10acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d9dfb6fc75612f1 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.705+0000 7f43d27fc640 1 -- 172.21.15.100:0/156254577 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7f43cc104780 con 0x7f43dc105c30 2024-07-26T13:45:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.705+0000 7f43c77fe640 1 -- 172.21.15.100:0/240637968 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc049030 con 0x55aceaf4d4d0 2024-07-26T13:45:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.705+0000 7f43c77fe640 1 -- 172.21.15.100:0/240637968 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43cc0cae20 con 0x55aceaf4d4d0 2024-07-26T13:45:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.705+0000 7f43c77fe640 1 -- 172.21.15.100:0/240637968 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc0cb0e0 con 0x55aceaf4d4d0 2024-07-26T13:45:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.706+0000 7f43ea5b7740 1 -- 172.21.15.100:0/240637968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aceaf4d4d0 msgr2=0x55aceb311130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.706+0000 7f43e1b65640 1 --2- 172.21.15.100:0/156254577 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f43c80b3d40 0x7f43c80b6250 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f43d400af10 tx=0x7f43d400a4d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.706+0000 7f43ea5b7740 1 --2- 172.21.15.100:0/240637968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aceaf4d4d0 0x55aceb311130 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f43cc101d90 tx=0x7f43cc10acb0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.706+0000 7f43ea5b7740 1 -- 172.21.15.100:0/240637968 shutdown_connections 2024-07-26T13:45:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.706+0000 7f43ea5b7740 1 --2- 172.21.15.100:0/240637968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aceaf4d4d0 0x55aceb311130 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.706+0000 7f43ea5b7740 1 --2- 172.21.15.100:0/240637968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aceaf452f0 0x55aceb310bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.706+0000 7f43ea5b7740 1 --2- 172.21.15.100:0/240637968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aceaf6e1b0 0x55aceb3106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.706+0000 7f43ea5b7740 1 -- 172.21.15.100:0/240637968 >> 172.21.15.100:0/240637968 conn(0x55aceaf62300 msgr2=0x55aceb30e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:08.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.706+0000 7f43ea5b7740 1 -- 172.21.15.100:0/240637968 shutdown_connections 2024-07-26T13:45:08.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.706+0000 7f43ea5b7740 1 -- 172.21.15.100:0/240637968 wait complete. 2024-07-26T13:45:08.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.706+0000 7f43ea5b7740 1 -- start start 2024-07-26T13:45:08.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.707+0000 7f43ea5b7740 1 Processor -- start 2024-07-26T13:45:08.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.707+0000 7f43ea5b7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aceaf452f0 0x55aceb320590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:08.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.707+0000 7f43ea5b7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aceaf4d4d0 0x55aceb320ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:08.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.707+0000 7f43ea5b7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aceaf6e1b0 0x55aceb32c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:08.716 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.707+0000 7f43ea5b7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55aceb1afab0 con 0x55aceaf4d4d0 2024-07-26T13:45:08.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.707+0000 7f43ea5b7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55aceb1fdcf0 con 0x55aceaf452f0 2024-07-26T13:45:08.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.707+0000 7f43ea5b7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55aceb322fd0 con 0x55aceaf6e1b0 2024-07-26T13:45:08.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.707+0000 7f43ea5b7740 10 client.0 _ll_register_callbacks cb 0x55aceb26f3e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:45:08.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.707+0000 7f43e0b63640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aceaf4d4d0 0x55aceb320ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:08.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.707+0000 7f43e0b63640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aceaf4d4d0 0x55aceb320ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35252/0 (socket says 172.21.15.100:35252) 2024-07-26T13:45:08.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.707+0000 7f43e0b63640 1 -- 172.21.15.100:0/3789285551 learned_addr learned my addr 172.21.15.100:0/3789285551 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:08.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.707+0000 7f43e1364640 1 --2- 172.21.15.100:0/3789285551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aceaf452f0 0x55aceb320590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:08.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.707+0000 7f43e1b65640 1 --2- 172.21.15.100:0/3789285551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aceaf6e1b0 0x55aceb32c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:08.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.707+0000 7f43e0b63640 1 -- 172.21.15.100:0/3789285551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aceaf6e1b0 msgr2=0x55aceb32c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:08.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.707+0000 7f43e0b63640 1 --2- 172.21.15.100:0/3789285551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aceaf6e1b0 0x55aceb32c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.708+0000 7f43e0b63640 1 -- 172.21.15.100:0/3789285551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aceaf452f0 msgr2=0x55aceb320590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:08.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.708+0000 7f43e0b63640 1 --2- 172.21.15.100:0/3789285551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aceaf452f0 0x55aceb320590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.708+0000 7f43e0b63640 1 -- 172.21.15.100:0/3789285551 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aceafd9790 con 0x55aceaf4d4d0 2024-07-26T13:45:08.717 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.708+0000 7f43e0b63640 1 --2- 172.21.15.100:0/3789285551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aceaf4d4d0 0x55aceb320ad0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f43cc03d040 tx=0x7f43cc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b32cebebc7b8bb4 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.708+0000 7f43c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:45:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.708+0000 7f43c4ff9640 1 -- 172.21.15.100:0/3789285551 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc0caa80 con 0x55aceaf4d4d0 2024-07-26T13:45:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.708+0000 7f43c4ff9640 1 -- 172.21.15.100:0/3789285551 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7f43cc003c50 con 0x55aceaf4d4d0 2024-07-26T13:45:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.708+0000 7f43ea5b7740 1 -- 172.21.15.100:0/3789285551 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43cc0cae20 con 0x55aceaf4d4d0 2024-07-26T13:45:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.708+0000 7f43c4ff9640 1 -- 172.21.15.100:0/3789285551 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f43cc0cac60 con 0x55aceaf4d4d0 2024-07-26T13:45:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.708+0000 7f43ea5b7740 10 client.16758 fetch_fsmap learned FSMap version 45 2024-07-26T13:45:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.708+0000 7f43ea5b7740 1 -- 172.21.15.100:0/3789285551 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55aceb052620 con 0x55aceaf4d4d0 2024-07-26T13:45:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.708+0000 7f43c4ff9640 1 -- 172.21.15.100:0/3789285551 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43cc0e5e50 con 0x55aceaf4d4d0 2024-07-26T13:45:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.709+0000 7f43c4ff9640 1 -- 172.21.15.100:0/3789285551 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f43cc1020d0 con 0x55aceaf4d4d0 2024-07-26T13:45:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.709+0000 7f43ea5b7740 10 client.16758 fetch_fsmap finished waiting for FSMap version 45 2024-07-26T13:45:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.709+0000 7f43ea5b7740 10 client.16758 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:45:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.709+0000 7f43ea5b7740 20 client.16758 populate_metadata read hostname 'smithi100' 2024-07-26T13:45:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.709+0000 7f43ea5b7740 1 --2- 172.21.15.100:0/3789285551 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55aceaf2cd80 0x55aceb332040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.709+0000 7f43ea5b7740 4 client.16758 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:45:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.709+0000 7f43ea5b7740 1 -- 172.21.15.100:0/3789285551 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aceb288300 con 0x55aceaf2cd80 2024-07-26T13:45:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.710+0000 7f43e1364640 1 --2- 172.21.15.100:0/3789285551 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55aceaf2cd80 0x55aceb332040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.710+0000 7f43e1364640 1 --2- 172.21.15.100:0/3789285551 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55aceaf2cd80 0x55aceb332040 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=859d71e730ce765f server_cookie=456b47f176d4ae1d in_seq=0 out_seq=0 2024-07-26T13:45:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.710+0000 7f43c4ff9640 10 client.16758 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.711+0000 7f43c4ff9640 1 -- 172.21.15.100:0/3789285551 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aceb288300 con 0x55aceaf2cd80 2024-07-26T13:45:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.711+0000 7f43c4ff9640 10 client.16758 handle_command_reply: tid=0 2024-07-26T13:45:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.711+0000 7f43ea5b7740 1 client.16758 shutdown 2024-07-26T13:45:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.711+0000 7f43ea5b7740 1 -- 172.21.15.100:0/3789285551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aceaf4d4d0 msgr2=0x55aceb320ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.711+0000 7f43ea5b7740 1 --2- 172.21.15.100:0/3789285551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aceaf4d4d0 0x55aceb320ad0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f43cc03d040 tx=0x7f43cc02a410 comp rx=0 tx=0).stop 2024-07-26T13:45:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.711+0000 7f43ea5b7740 1 -- 172.21.15.100:0/3789285551 shutdown_connections 2024-07-26T13:45:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.711+0000 7f43ea5b7740 1 --2- 172.21.15.100:0/3789285551 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55aceaf2cd80 0x55aceb332040 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.711+0000 7f43ea5b7740 1 --2- 172.21.15.100:0/3789285551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aceaf6e1b0 0x55aceb32c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.711+0000 7f43ea5b7740 1 --2- 172.21.15.100:0/3789285551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aceaf4d4d0 0x55aceb320ad0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.711+0000 7f43ea5b7740 1 --2- 172.21.15.100:0/3789285551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aceaf452f0 0x55aceb320590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.711+0000 7f43c4ff9640 0 client.16758 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.711+0000 7f43ea5b7740 1 -- 172.21.15.100:0/3789285551 >> 172.21.15.100:0/3789285551 conn(0x55aceaf62300 msgr2=0x55aceb3120a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.711+0000 7f43ea5b7740 1 -- 172.21.15.100:0/3789285551 shutdown_connections 2024-07-26T13:45:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.711+0000 7f43ea5b7740 1 -- 172.21.15.100:0/3789285551 wait complete. 2024-07-26T13:45:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.711+0000 7f43ea5b7740 20 client.16758 trim_cache size 0 max 16384 2024-07-26T13:45:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.726+0000 7f43ea5b7740 1 Processor -- start 2024-07-26T13:45:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.726+0000 7f43ea5b7740 1 -- start start 2024-07-26T13:45:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.726+0000 7f43ea5b7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aceaf452f0 0x55aceb32c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.727+0000 7f43ea5b7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aceaf4d4d0 0x55aceb32c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.727+0000 7f43ea5b7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aceaf6e1b0 0x55aceb30de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.727+0000 7f43ea5b7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc1af900 con 0x55aceaf4d4d0 2024-07-26T13:45:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.727+0000 7f43ea5b7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc1ad170 con 0x55aceaf452f0 2024-07-26T13:45:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.727+0000 7f43ea5b7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc1aafa0 con 0x55aceaf6e1b0 2024-07-26T13:45:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.727+0000 7f43e0b63640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aceaf4d4d0 0x55aceb32c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.727+0000 7f43e0b63640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aceaf4d4d0 0x55aceb32c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35256/0 (socket says 172.21.15.100:35256) 2024-07-26T13:45:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.727+0000 7f43e0b63640 1 -- 172.21.15.100:0/1304588664 learned_addr learned my addr 172.21.15.100:0/1304588664 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.727+0000 7f43e0b63640 1 -- 172.21.15.100:0/1304588664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aceaf6e1b0 msgr2=0x55aceb30de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.727+0000 7f43e0b63640 1 --2- 172.21.15.100:0/1304588664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aceaf6e1b0 0x55aceb30de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.727+0000 7f43e0b63640 1 -- 172.21.15.100:0/1304588664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aceaf452f0 msgr2=0x55aceb32c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.727+0000 7f43e0b63640 1 --2- 172.21.15.100:0/1304588664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aceaf452f0 0x55aceb32c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.727+0000 7f43e0b63640 1 -- 172.21.15.100:0/1304588664 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aceb3114a0 con 0x55aceaf4d4d0 2024-07-26T13:45:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.727+0000 7f43e0b63640 1 --2- 172.21.15.100:0/1304588664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aceaf4d4d0 0x55aceb32c5f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f43cc032af0 tx=0x7f43cc031a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ade1f540e4fad0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.727+0000 7f43c6ffd640 1 -- 172.21.15.100:0/1304588664 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc0caa80 con 0x55aceaf4d4d0 2024-07-26T13:45:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.727+0000 7f43c6ffd640 1 -- 172.21.15.100:0/1304588664 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43cc0e5e50 con 0x55aceaf4d4d0 2024-07-26T13:45:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.727+0000 7f43c6ffd640 1 -- 172.21.15.100:0/1304588664 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc0cb0e0 con 0x55aceaf4d4d0 2024-07-26T13:45:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.728+0000 7f43ea5b7740 1 -- 172.21.15.100:0/1304588664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aceaf4d4d0 msgr2=0x55aceb32c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.728+0000 7f43ea5b7740 1 --2- 172.21.15.100:0/1304588664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aceaf4d4d0 0x55aceb32c5f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f43cc032af0 tx=0x7f43cc031a10 comp rx=0 tx=0).stop 2024-07-26T13:45:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.728+0000 7f43ea5b7740 1 -- 172.21.15.100:0/1304588664 shutdown_connections 2024-07-26T13:45:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.728+0000 7f43ea5b7740 1 --2- 172.21.15.100:0/1304588664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aceaf6e1b0 0x55aceb30de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.728+0000 7f43ea5b7740 1 --2- 172.21.15.100:0/1304588664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aceaf4d4d0 0x55aceb32c5f0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.728+0000 7f43ea5b7740 1 --2- 172.21.15.100:0/1304588664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aceaf452f0 0x55aceb32c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.728+0000 7f43ea5b7740 1 -- 172.21.15.100:0/1304588664 >> 172.21.15.100:0/1304588664 conn(0x55aceaf62300 msgr2=0x55aceb31ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.728+0000 7f43ea5b7740 1 -- 172.21.15.100:0/1304588664 shutdown_connections 2024-07-26T13:45:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.728+0000 7f43ea5b7740 1 -- 172.21.15.100:0/1304588664 wait complete. 2024-07-26T13:45:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.729+0000 7f43ea5b7740 1 -- start start 2024-07-26T13:45:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.729+0000 7f43ea5b7740 1 Processor -- start 2024-07-26T13:45:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.729+0000 7f43ea5b7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aceaf452f0 0x55aceb325630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.729+0000 7f43ea5b7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aceaf4d4d0 0x55aceb325b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.729+0000 7f43ea5b7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aceaf6e1b0 0x55aceb3987a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.729+0000 7f43ea5b7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc1ab630 con 0x55aceaf452f0 2024-07-26T13:45:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.729+0000 7f43ea5b7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc1acaa0 con 0x55aceaf4d4d0 2024-07-26T13:45:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.729+0000 7f43ea5b7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc1abd00 con 0x55aceaf6e1b0 2024-07-26T13:45:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.729+0000 7f43ea5b7740 10 client.0 _ll_register_callbacks cb 0x7f43d400a8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:45:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.729+0000 7f43e1364640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aceaf452f0 0x55aceb325630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.729+0000 7f43e1364640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aceaf452f0 0x55aceb325630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35264/0 (socket says 172.21.15.100:35264) 2024-07-26T13:45:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.729+0000 7f43e1364640 1 -- 172.21.15.100:0/605395481 learned_addr learned my addr 172.21.15.100:0/605395481 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.729+0000 7f43e1b65640 1 --2- 172.21.15.100:0/605395481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aceaf6e1b0 0x55aceb3987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.730+0000 7f43e1364640 1 -- 172.21.15.100:0/605395481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aceaf6e1b0 msgr2=0x55aceb3987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.730+0000 7f43e1364640 1 --2- 172.21.15.100:0/605395481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aceaf6e1b0 0x55aceb3987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.730+0000 7f43e1364640 1 -- 172.21.15.100:0/605395481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aceaf4d4d0 msgr2=0x55aceb325b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.730+0000 7f43e0b63640 1 --2- 172.21.15.100:0/605395481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aceaf4d4d0 0x55aceb325b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.730+0000 7f43e1364640 1 --2- 172.21.15.100:0/605395481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aceaf4d4d0 0x55aceb325b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.730+0000 7f43e1364640 1 -- 172.21.15.100:0/605395481 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aceb398ce0 con 0x55aceaf452f0 2024-07-26T13:45:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.730+0000 7f43e1b65640 1 --2- 172.21.15.100:0/605395481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aceaf6e1b0 0x55aceb3987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:45:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.730+0000 7f43e1364640 1 --2- 172.21.15.100:0/605395481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aceaf452f0 0x55aceb325630 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x55aceb30f330 tx=0x7f43d8011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24c348e08ebe9b67 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.730+0000 7f43c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:45:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.730+0000 7f43c4ff9640 1 -- 172.21.15.100:0/605395481 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d801d8f0 con 0x55aceaf452f0 2024-07-26T13:45:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.730+0000 7f43c4ff9640 1 -- 172.21.15.100:0/605395481 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7f43d801da90 con 0x55aceaf452f0 2024-07-26T13:45:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.730+0000 7f43ea5b7740 1 -- 172.21.15.100:0/605395481 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43cc0e5e50 con 0x55aceaf452f0 2024-07-26T13:45:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.730+0000 7f43c4ff9640 1 -- 172.21.15.100:0/605395481 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f43d802b050 con 0x55aceaf452f0 2024-07-26T13:45:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.730+0000 7f43ea5b7740 10 client.16764 fetch_fsmap learned FSMap version 45 2024-07-26T13:45:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.730+0000 7f43ea5b7740 1 -- 172.21.15.100:0/605395481 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55aceb399e20 con 0x55aceaf452f0 2024-07-26T13:45:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.730+0000 7f43c4ff9640 1 -- 172.21.15.100:0/605395481 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43cc0e5e50 con 0x55aceaf452f0 2024-07-26T13:45:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.730+0000 7f43c4ff9640 1 -- 172.21.15.100:0/605395481 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f43d8004bb0 con 0x55aceaf452f0 2024-07-26T13:45:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.730+0000 7f43ea5b7740 10 client.16764 fetch_fsmap finished waiting for FSMap version 45 2024-07-26T13:45:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.731+0000 7f43ea5b7740 10 client.16764 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:45:08.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.731+0000 7f43ea5b7740 20 client.16764 populate_metadata read hostname 'smithi100' 2024-07-26T13:45:08.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.731+0000 7f43ea5b7740 1 --2- 172.21.15.100:0/605395481 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55aceaf021c0 0x55aceb39c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:08.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.731+0000 7f43ea5b7740 4 client.16764 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-26T13:45:08.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.731+0000 7f43ea5b7740 1 -- 172.21.15.100:0/605395481 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aceb3167c0 con 0x55aceaf021c0 2024-07-26T13:45:08.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.731+0000 7f43e0b63640 1 --2- 172.21.15.100:0/605395481 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55aceaf021c0 0x55aceb39c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:08.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.731+0000 7f43c4ff9640 10 client.16764 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:08.738 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.731+0000 7f43e0b63640 1 --2- 172.21.15.100:0/605395481 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55aceaf021c0 0x55aceb39c610 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3329967b12072855 server_cookie=5fcc2a7a2ac25fc6 in_seq=0 out_seq=0 2024-07-26T13:45:08.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.774+0000 7f43c4ff9640 1 -- 172.21.15.100:0/605395481 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aceb3167c0 con 0x55aceaf021c0 2024-07-26T13:45:08.776 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.774+0000 7f43c4ff9640 10 client.16764 handle_command_reply: tid=0 2024-07-26T13:45:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.774+0000 7f43ea5b7740 1 client.16764 shutdown 2024-07-26T13:45:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.775+0000 7f43ea5b7740 1 -- 172.21.15.100:0/605395481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aceaf452f0 msgr2=0x55aceb325630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.775+0000 7f43ea5b7740 1 --2- 172.21.15.100:0/605395481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aceaf452f0 0x55aceb325630 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x55aceb30f330 tx=0x7f43d8011a10 comp rx=0 tx=0).stop 2024-07-26T13:45:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.775+0000 7f43ea5b7740 1 -- 172.21.15.100:0/605395481 shutdown_connections 2024-07-26T13:45:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.775+0000 7f43ea5b7740 1 --2- 172.21.15.100:0/605395481 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55aceaf021c0 0x55aceb39c610 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.775+0000 7f43ea5b7740 1 --2- 172.21.15.100:0/605395481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aceaf6e1b0 0x55aceb3987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.775+0000 7f43c4ff9640 0 client.16764 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.775+0000 7f43ea5b7740 1 --2- 172.21.15.100:0/605395481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aceaf4d4d0 0x55aceb325b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.775+0000 7f43ea5b7740 1 --2- 172.21.15.100:0/605395481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55aceaf452f0 0x55aceb325630 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.775+0000 7f43ea5b7740 1 -- 172.21.15.100:0/605395481 >> 172.21.15.100:0/605395481 conn(0x55aceaf62300 msgr2=0x55aceb31eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.775+0000 7f43ea5b7740 1 -- 172.21.15.100:0/605395481 shutdown_connections 2024-07-26T13:45:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.775+0000 7f43ea5b7740 1 -- 172.21.15.100:0/605395481 wait complete. 2024-07-26T13:45:08.777 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.775+0000 7f43ea5b7740 20 client.16764 trim_cache size 0 max 16384 2024-07-26T13:45:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-07-26T13:45:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-07-26T13:45:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-07-26T13:45:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-07-26T13:45:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.776+0000 7f43e35c8640 1 -- 172.21.15.100:0/156254577 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f43c80b3d40 msgr2=0x7f43c80b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.776+0000 7f43e35c8640 1 --2- 172.21.15.100:0/156254577 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f43c80b3d40 0x7f43c80b6250 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f43d400af10 tx=0x7f43d400a4d0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.776+0000 7f43e35c8640 1 -- 172.21.15.100:0/156254577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43dc105c30 msgr2=0x7f43dc19a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.776+0000 7f43e35c8640 1 --2- 172.21.15.100:0/156254577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43dc105c30 0x7f43dc19a420 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f43cc02a930 tx=0x7f43cc004270 comp rx=0 tx=0).stop 2024-07-26T13:45:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.776+0000 7f43e35c8640 1 -- 172.21.15.100:0/156254577 shutdown_connections 2024-07-26T13:45:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.776+0000 7f43e35c8640 1 --2- 172.21.15.100:0/156254577 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f43c80b3d40 0x7f43c80b6250 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.776+0000 7f43e35c8640 1 --2- 172.21.15.100:0/156254577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43dc106580 0x7f43dc19f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.776+0000 7f43e35c8640 1 --2- 172.21.15.100:0/156254577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43dc105c30 0x7f43dc19a420 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.776+0000 7f43e35c8640 1 --2- 172.21.15.100:0/156254577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43dc101ac0 0x7f43dc199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.776+0000 7f43e35c8640 1 -- 172.21.15.100:0/156254577 >> 172.21.15.100:0/156254577 conn(0x7f43dc0798f0 msgr2=0x7f43dc107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:08.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.776+0000 7f43e35c8640 1 -- 172.21.15.100:0/156254577 shutdown_connections 2024-07-26T13:45:08.780 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:08.776+0000 7f43e35c8640 1 -- 172.21.15.100:0/156254577 wait complete. 2024-07-26T13:45:09.266 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-26T13:45:09.416 INFO:tasks.workunit.client.0.smithi100.stdout: 1 175223 1.90 MB/sec execute 83 sec latency 2828.067 ms 2024-07-26T13:45:09.523 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:45:09.891 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:09 smithi100 ceph-mon[32599]: pgmap v517: 97 pgs: 97 active+clean; 74 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.4 MiB/s wr, 238 op/s 2024-07-26T13:45:09.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:09 smithi138 ceph-mon[36954]: pgmap v517: 97 pgs: 97 active+clean; 74 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.4 MiB/s wr, 238 op/s 2024-07-26T13:45:10.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:09 smithi167 ceph-mon[37140]: pgmap v517: 97 pgs: 97 active+clean; 74 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.4 MiB/s wr, 238 op/s 2024-07-26T13:45:10.416 INFO:tasks.workunit.client.0.smithi100.stdout: 1 175223 1.88 MB/sec execute 84 sec latency 3828.162 ms 2024-07-26T13:45:11.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.150+0000 7f41c502c640 1 -- 172.21.15.100:0/1727915691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41c0108730 msgr2=0x7f41c010ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:11.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.150+0000 7f41c502c640 1 --2- 172.21.15.100:0/1727915691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41c0108730 0x7f41c010ef60 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f41b8009ca0 tx=0x7f41b802a8c0 comp rx=0 tx=0).stop 2024-07-26T13:45:11.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.150+0000 7f41c502c640 1 -- 172.21.15.100:0/1727915691 shutdown_connections 2024-07-26T13:45:11.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.150+0000 7f41c502c640 1 --2- 172.21.15.100:0/1727915691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41c0108730 0x7f41c010ef60 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:11.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.150+0000 7f41c502c640 1 --2- 172.21.15.100:0/1727915691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41c0107de0 0x7f41c01081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:11.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.150+0000 7f41c502c640 1 --2- 172.21.15.100:0/1727915691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41c01024a0 0x7f41c01028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:11.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.150+0000 7f41c502c640 1 -- 172.21.15.100:0/1727915691 >> 172.21.15.100:0/1727915691 conn(0x7f41c0079140 msgr2=0x7f41c0079550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:11.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.150+0000 7f41c502c640 1 -- 172.21.15.100:0/1727915691 shutdown_connections 2024-07-26T13:45:11.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.150+0000 7f41c502c640 1 -- 172.21.15.100:0/1727915691 wait complete. 2024-07-26T13:45:11.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.151+0000 7f41c502c640 1 Processor -- start 2024-07-26T13:45:11.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.151+0000 7f41c502c640 1 -- start start 2024-07-26T13:45:11.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.151+0000 7f41c502c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41c01024a0 0x7f41c0199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:11.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.151+0000 7f41c502c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41c0107de0 0x7f41c019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:11.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.151+0000 7f41c502c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41c0108730 0x7f41c019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:11.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.151+0000 7f41c502c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f41c01123e0 con 0x7f41c0107de0 2024-07-26T13:45:11.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.151+0000 7f41c502c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f41c0112260 con 0x7f41c01024a0 2024-07-26T13:45:11.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.151+0000 7f41c502c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f41c0112560 con 0x7f41c0108730 2024-07-26T13:45:11.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.151+0000 7f41be59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41c0107de0 0x7f41c019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:11.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.151+0000 7f41be59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41c0107de0 0x7f41c019a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35290/0 (socket says 172.21.15.100:35290) 2024-07-26T13:45:11.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.151+0000 7f41be59c640 1 -- 172.21.15.100:0/337353607 learned_addr learned my addr 172.21.15.100:0/337353607 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:11.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.151+0000 7f41bed9d640 1 --2- 172.21.15.100:0/337353607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41c01024a0 0x7f41c0199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:11.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.152+0000 7f41bf59e640 1 --2- 172.21.15.100:0/337353607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41c0108730 0x7f41c019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:11.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.152+0000 7f41be59c640 1 -- 172.21.15.100:0/337353607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41c0108730 msgr2=0x7f41c019f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:11.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.152+0000 7f41be59c640 1 --2- 172.21.15.100:0/337353607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41c0108730 0x7f41c019f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:11.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.152+0000 7f41be59c640 1 -- 172.21.15.100:0/337353607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41c01024a0 msgr2=0x7f41c0199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:11.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.152+0000 7f41be59c640 1 --2- 172.21.15.100:0/337353607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41c01024a0 0x7f41c0199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:11.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.152+0000 7f41be59c640 1 -- 172.21.15.100:0/337353607 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c019fdd0 con 0x7f41c0107de0 2024-07-26T13:45:11.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.152+0000 7f41bf59e640 1 --2- 172.21.15.100:0/337353607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41c0108730 0x7f41c019f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:45:11.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.152+0000 7f41be59c640 1 --2- 172.21.15.100:0/337353607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41c0107de0 0x7f41c019a400 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f41b400c970 tx=0x7f41b400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:11.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.152+0000 7f41a7fff640 1 -- 172.21.15.100:0/337353607 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b4007bb0 con 0x7f41c0107de0 2024-07-26T13:45:11.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.152+0000 7f41c502c640 1 -- 172.21.15.100:0/337353607 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41c01a00c0 con 0x7f41c0107de0 2024-07-26T13:45:11.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.152+0000 7f41a7fff640 1 -- 172.21.15.100:0/337353607 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41b4007d50 con 0x7f41c0107de0 2024-07-26T13:45:11.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.152+0000 7f41a7fff640 1 -- 172.21.15.100:0/337353607 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b400f450 con 0x7f41c0107de0 2024-07-26T13:45:11.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.152+0000 7f41c502c640 1 -- 172.21.15.100:0/337353607 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41c00fef00 con 0x7f41c0107de0 2024-07-26T13:45:11.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.155+0000 7f41cb5b0740 1 Processor -- start 2024-07-26T13:45:11.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.155+0000 7f41cb5b0740 1 -- start start 2024-07-26T13:45:11.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.156+0000 7f41a7fff640 1 -- 172.21.15.100:0/337353607 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f41b400f5f0 con 0x7f41c0107de0 2024-07-26T13:45:11.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.156+0000 7f41cb5b0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb9ac09070 0x55eb9afae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:11.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.156+0000 7f41cb5b0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb9abe01b0 0x55eb9afaebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:11.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.156+0000 7f41cb5b0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb9abe8390 0x55eb9afaf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:11.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.156+0000 7f41cb5b0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55eb9af24bb0 con 0x55eb9ac09070 2024-07-26T13:45:11.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.156+0000 7f41cb5b0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55eb9af140b0 con 0x55eb9abe01b0 2024-07-26T13:45:11.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.156+0000 7f41cb5b0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55eb9af180f0 con 0x55eb9abe8390 2024-07-26T13:45:11.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.156+0000 7f41bf59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb9ac09070 0x55eb9afae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:11.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.156+0000 7f41bf59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb9ac09070 0x55eb9afae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35306/0 (socket says 172.21.15.100:35306) 2024-07-26T13:45:11.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.156+0000 7f41bf59e640 1 -- 172.21.15.100:0/3389667881 learned_addr learned my addr 172.21.15.100:0/3389667881 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:11.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.156+0000 7f41bed9d640 1 --2- 172.21.15.100:0/3389667881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb9abe01b0 0x55eb9afaebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:11.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.156+0000 7f41be59c640 1 --2- 172.21.15.100:0/3389667881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb9abe8390 0x55eb9afaf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:11.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.156+0000 7f41bf59e640 1 -- 172.21.15.100:0/3389667881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb9abe8390 msgr2=0x55eb9afaf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:11.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.156+0000 7f41bf59e640 1 --2- 172.21.15.100:0/3389667881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb9abe8390 0x55eb9afaf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:11.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.156+0000 7f41bf59e640 1 -- 172.21.15.100:0/3389667881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb9abe01b0 msgr2=0x55eb9afaebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:11.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.156+0000 7f41bf59e640 1 --2- 172.21.15.100:0/3389667881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb9abe01b0 0x55eb9afaebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:11.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.156+0000 7f41bf59e640 1 -- 172.21.15.100:0/3389667881 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb9aebab90 con 0x55eb9ac09070 2024-07-26T13:45:11.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.156+0000 7f41be59c640 1 --2- 172.21.15.100:0/3389667881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb9abe8390 0x55eb9afaf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:45:11.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.156+0000 7f41bed9d640 1 --2- 172.21.15.100:0/3389667881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb9abe01b0 0x55eb9afaebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:45:11.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.156+0000 7f41a7fff640 1 --2- 172.21.15.100:0/337353607 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f41a80b3d40 0x7f41a80b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:11.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.156+0000 7f41bf59e640 1 --2- 172.21.15.100:0/3389667881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb9ac09070 0x55eb9afae690 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f41b802f500 tx=0x7f41b80099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de836be012348521 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:11.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.157+0000 7f41bed9d640 1 --2- 172.21.15.100:0/337353607 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f41a80b3d40 0x7f41a80b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:11.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.157+0000 7f41a7fff640 1 -- 172.21.15.100:0/337353607 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7f41b40e3e10 con 0x7f41c0107de0 2024-07-26T13:45:11.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.157+0000 7f41a57fa640 1 -- 172.21.15.100:0/3389667881 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b8031450 con 0x55eb9ac09070 2024-07-26T13:45:11.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.157+0000 7f41a57fa640 1 -- 172.21.15.100:0/3389667881 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41b8036040 con 0x55eb9ac09070 2024-07-26T13:45:11.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.157+0000 7f41a57fa640 1 -- 172.21.15.100:0/3389667881 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b8004440 con 0x55eb9ac09070 2024-07-26T13:45:11.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.158+0000 7f41cb5b0740 1 -- 172.21.15.100:0/3389667881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb9ac09070 msgr2=0x55eb9afae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:11.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.158+0000 7f41cb5b0740 1 --2- 172.21.15.100:0/3389667881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb9ac09070 0x55eb9afae690 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f41b802f500 tx=0x7f41b80099c0 comp rx=0 tx=0).stop 2024-07-26T13:45:11.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.158+0000 7f41bed9d640 1 --2- 172.21.15.100:0/337353607 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f41a80b3d40 0x7f41a80b6250 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x55eb9afad310 tx=0x7f41b000b390 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:11.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.158+0000 7f41cb5b0740 1 -- 172.21.15.100:0/3389667881 shutdown_connections 2024-07-26T13:45:11.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.158+0000 7f41cb5b0740 1 --2- 172.21.15.100:0/3389667881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb9abe8390 0x55eb9afaf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:11.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.158+0000 7f41cb5b0740 1 --2- 172.21.15.100:0/3389667881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb9abe01b0 0x55eb9afaebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:11.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.158+0000 7f41cb5b0740 1 --2- 172.21.15.100:0/3389667881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb9ac09070 0x55eb9afae690 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:11.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.158+0000 7f41cb5b0740 1 -- 172.21.15.100:0/3389667881 >> 172.21.15.100:0/3389667881 conn(0x55eb9abfd460 msgr2=0x55eb9afac580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:11.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.158+0000 7f41cb5b0740 1 -- 172.21.15.100:0/3389667881 shutdown_connections 2024-07-26T13:45:11.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.158+0000 7f41cb5b0740 1 -- 172.21.15.100:0/3389667881 wait complete. 2024-07-26T13:45:11.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.158+0000 7f41cb5b0740 1 -- start start 2024-07-26T13:45:11.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.158+0000 7f41cb5b0740 1 Processor -- start 2024-07-26T13:45:11.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.159+0000 7f41cb5b0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb9abe01b0 0x55eb9afba360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:11.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.159+0000 7f41cb5b0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb9abe8390 0x55eb9afba940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:11.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.159+0000 7f41cb5b0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb9ac09070 0x55eb9afc6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:11.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.159+0000 7f41cb5b0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55eb9ad77700 con 0x55eb9ac09070 2024-07-26T13:45:11.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.159+0000 7f41cb5b0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55eb9aeb69e0 con 0x55eb9abe01b0 2024-07-26T13:45:11.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.159+0000 7f41cb5b0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55eb9afbce10 con 0x55eb9abe8390 2024-07-26T13:45:11.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.159+0000 7f41cb5b0740 10 client.0 _ll_register_callbacks cb 0x55eb9aed14f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:45:11.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.159+0000 7f41bf59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb9ac09070 0x55eb9afc6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:11.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.159+0000 7f41bf59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb9ac09070 0x55eb9afc6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35312/0 (socket says 172.21.15.100:35312) 2024-07-26T13:45:11.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.159+0000 7f41bf59e640 1 -- 172.21.15.100:0/1084943114 learned_addr learned my addr 172.21.15.100:0/1084943114 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:11.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.159+0000 7f41bed9d640 1 --2- 172.21.15.100:0/1084943114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb9abe01b0 0x55eb9afba360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:11.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.159+0000 7f41be59c640 1 --2- 172.21.15.100:0/1084943114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb9abe8390 0x55eb9afba940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:11.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.159+0000 7f41bf59e640 1 -- 172.21.15.100:0/1084943114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb9abe8390 msgr2=0x55eb9afba940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:11.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.159+0000 7f41bf59e640 1 --2- 172.21.15.100:0/1084943114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb9abe8390 0x55eb9afba940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:11.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.159+0000 7f41bf59e640 1 -- 172.21.15.100:0/1084943114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb9abe01b0 msgr2=0x55eb9afba360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:11.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.159+0000 7f41bf59e640 1 --2- 172.21.15.100:0/1084943114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb9abe01b0 0x55eb9afba360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:11.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.159+0000 7f41bf59e640 1 -- 172.21.15.100:0/1084943114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb9ac74650 con 0x55eb9ac09070 2024-07-26T13:45:11.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.159+0000 7f41bf59e640 1 --2- 172.21.15.100:0/1084943114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb9ac09070 0x55eb9afc6100 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f41b8006010 tx=0x7f41b802f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a764628cc1c0c2 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:11.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.159+0000 7f41a2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:45:11.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.159+0000 7f41a2ff5640 1 -- 172.21.15.100:0/1084943114 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b804a040 con 0x55eb9ac09070 2024-07-26T13:45:11.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.159+0000 7f41a2ff5640 1 -- 172.21.15.100:0/1084943114 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7f41b8031450 con 0x55eb9ac09070 2024-07-26T13:45:11.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.159+0000 7f41cb5b0740 1 -- 172.21.15.100:0/1084943114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41b8036040 con 0x55eb9ac09070 2024-07-26T13:45:11.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.160+0000 7f41a2ff5640 1 -- 172.21.15.100:0/1084943114 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f41b805c050 con 0x55eb9ac09070 2024-07-26T13:45:11.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.160+0000 7f41cb5b0740 10 client.16776 fetch_fsmap learned FSMap version 45 2024-07-26T13:45:11.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.160+0000 7f41cb5b0740 1 -- 172.21.15.100:0/1084943114 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55eb9afc8e40 con 0x55eb9ac09070 2024-07-26T13:45:11.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.160+0000 7f41a2ff5640 1 -- 172.21.15.100:0/1084943114 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41b8036040 con 0x55eb9ac09070 2024-07-26T13:45:11.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.160+0000 7f41a2ff5640 1 -- 172.21.15.100:0/1084943114 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f41b804d070 con 0x55eb9ac09070 2024-07-26T13:45:11.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.160+0000 7f41cb5b0740 10 client.16776 fetch_fsmap finished waiting for FSMap version 45 2024-07-26T13:45:11.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.160+0000 7f41cb5b0740 10 client.16776 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-26T13:45:11.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.160+0000 7f41cb5b0740 20 client.16776 populate_metadata read hostname 'smithi100' 2024-07-26T13:45:11.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.160+0000 7f41cb5b0740 1 --2- 172.21.15.100:0/1084943114 >> [v2:172.21.15.167:6836/2476068386,v1:172.21.15.167:6837/2476068386] conn(0x55eb9abc79f0 0x55eb9afcc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:11.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.160+0000 7f41cb5b0740 4 client.16776 mds_command: new command op to 25762 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:45:11.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.160+0000 7f41cb5b0740 1 -- 172.21.15.100:0/1084943114 --> [v2:172.21.15.167:6836/2476068386,v1:172.21.15.167:6837/2476068386] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb9af26b00 con 0x55eb9abc79f0 2024-07-26T13:45:11.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.160+0000 7f41bed9d640 1 --2- 172.21.15.100:0/1084943114 >> [v2:172.21.15.167:6836/2476068386,v1:172.21.15.167:6837/2476068386] conn(0x55eb9abc79f0 0x55eb9afcc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:11.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.161+0000 7f41a2ff5640 10 client.16776 ms_handle_connect on v2:172.21.15.167:6836/2476068386 2024-07-26T13:45:11.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.161+0000 7f41bed9d640 1 --2- 172.21.15.100:0/1084943114 >> [v2:172.21.15.167:6836/2476068386,v1:172.21.15.167:6837/2476068386] conn(0x55eb9abc79f0 0x55eb9afcc1a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6c88a4c7cbd36fbd server_cookie=f7a1782eb5309abb in_seq=0 out_seq=0 2024-07-26T13:45:11.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.162+0000 7f41a2ff5640 1 -- 172.21.15.100:0/1084943114 <== mds.2 v2:172.21.15.167:6836/2476068386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eb9af26b00 con 0x55eb9abc79f0 2024-07-26T13:45:11.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.162+0000 7f41a2ff5640 10 client.16776 handle_command_reply: tid=0 2024-07-26T13:45:11.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.162+0000 7f41cb5b0740 1 client.16776 shutdown 2024-07-26T13:45:11.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.162+0000 7f41cb5b0740 1 -- 172.21.15.100:0/1084943114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb9ac09070 msgr2=0x55eb9afc6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:11.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.162+0000 7f41cb5b0740 1 --2- 172.21.15.100:0/1084943114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb9ac09070 0x55eb9afc6100 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f41b8006010 tx=0x7f41b802f470 comp rx=0 tx=0).stop 2024-07-26T13:45:11.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.162+0000 7f41cb5b0740 1 -- 172.21.15.100:0/1084943114 shutdown_connections 2024-07-26T13:45:11.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.162+0000 7f41cb5b0740 1 --2- 172.21.15.100:0/1084943114 >> [v2:172.21.15.167:6836/2476068386,v1:172.21.15.167:6837/2476068386] conn(0x55eb9abc79f0 0x55eb9afcc1a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:11.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.162+0000 7f41cb5b0740 1 --2- 172.21.15.100:0/1084943114 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb9ac09070 0x55eb9afc6100 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:11.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.162+0000 7f41cb5b0740 1 --2- 172.21.15.100:0/1084943114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb9abe8390 0x55eb9afba940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:11.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.162+0000 7f41cb5b0740 1 --2- 172.21.15.100:0/1084943114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb9abe01b0 0x55eb9afba360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:11.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.162+0000 7f41a2ff5640 0 client.16776 ms_handle_reset on v2:172.21.15.167:6836/2476068386 2024-07-26T13:45:11.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.162+0000 7f41cb5b0740 1 -- 172.21.15.100:0/1084943114 >> 172.21.15.100:0/1084943114 conn(0x55eb9abfd460 msgr2=0x55eb9afaad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:11.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.162+0000 7f41cb5b0740 1 -- 172.21.15.100:0/1084943114 shutdown_connections 2024-07-26T13:45:11.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.162+0000 7f41cb5b0740 1 -- 172.21.15.100:0/1084943114 wait complete. 2024-07-26T13:45:11.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.162+0000 7f41cb5b0740 20 client.16776 trim_cache size 0 max 16384 2024-07-26T13:45:11.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.175+0000 7f41cb5b0740 1 Processor -- start 2024-07-26T13:45:11.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.175+0000 7f41cb5b0740 1 -- start start 2024-07-26T13:45:11.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.175+0000 7f41cb5b0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb9abe01b0 0x55eb9afac680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:11.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.175+0000 7f41cb5b0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb9abe8390 0x55eb9afb3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:11.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.175+0000 7f41cb5b0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb9ac09070 0x55eb9afb3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:11.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.175+0000 7f41cb5b0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f41c01b3e30 con 0x55eb9ac09070 2024-07-26T13:45:11.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.175+0000 7f41cb5b0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f41c01b16a0 con 0x55eb9abe8390 2024-07-26T13:45:11.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.175+0000 7f41cb5b0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f41c01af4d0 con 0x55eb9abe01b0 2024-07-26T13:45:11.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.175+0000 7f41bf59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb9ac09070 0x55eb9afb3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:11.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.175+0000 7f41bf59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb9ac09070 0x55eb9afb3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35328/0 (socket says 172.21.15.100:35328) 2024-07-26T13:45:11.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.175+0000 7f41bf59e640 1 -- 172.21.15.100:0/1516213703 learned_addr learned my addr 172.21.15.100:0/1516213703 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:11.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.175+0000 7f41bed9d640 1 --2- 172.21.15.100:0/1516213703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb9abe01b0 0x55eb9afac680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:11.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.176+0000 7f41be59c640 1 --2- 172.21.15.100:0/1516213703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb9abe8390 0x55eb9afb3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:11.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.176+0000 7f41bf59e640 1 -- 172.21.15.100:0/1516213703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb9abe01b0 msgr2=0x55eb9afac680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:11.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.176+0000 7f41bf59e640 1 --2- 172.21.15.100:0/1516213703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb9abe01b0 0x55eb9afac680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:11.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.176+0000 7f41bf59e640 1 -- 172.21.15.100:0/1516213703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb9abe8390 msgr2=0x55eb9afb3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:11.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.176+0000 7f41bf59e640 1 --2- 172.21.15.100:0/1516213703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb9abe8390 0x55eb9afb3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:11.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.176+0000 7f41bf59e640 1 -- 172.21.15.100:0/1516213703 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb9afb5cf0 con 0x55eb9ac09070 2024-07-26T13:45:11.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.176+0000 7f41be59c640 1 --2- 172.21.15.100:0/1516213703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb9abe8390 0x55eb9afb3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:45:11.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.176+0000 7f41bf59e640 1 --2- 172.21.15.100:0/1516213703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb9ac09070 0x55eb9afb3e50 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f41b8009ca0 tx=0x7f41b8031b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0aa5901fccd271b server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:11.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.176+0000 7f41a4ff9640 1 -- 172.21.15.100:0/1516213703 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b8065070 con 0x55eb9ac09070 2024-07-26T13:45:11.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.176+0000 7f41a4ff9640 1 -- 172.21.15.100:0/1516213703 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41b8036040 con 0x55eb9ac09070 2024-07-26T13:45:11.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.176+0000 7f41a4ff9640 1 -- 172.21.15.100:0/1516213703 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b8003bd0 con 0x55eb9ac09070 2024-07-26T13:45:11.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.177+0000 7f41cb5b0740 1 -- 172.21.15.100:0/1516213703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb9ac09070 msgr2=0x55eb9afb3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:11.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.177+0000 7f41cb5b0740 1 --2- 172.21.15.100:0/1516213703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb9ac09070 0x55eb9afb3e50 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f41b8009ca0 tx=0x7f41b8031b70 comp rx=0 tx=0).stop 2024-07-26T13:45:11.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.177+0000 7f41cb5b0740 1 -- 172.21.15.100:0/1516213703 shutdown_connections 2024-07-26T13:45:11.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.177+0000 7f41cb5b0740 1 --2- 172.21.15.100:0/1516213703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb9ac09070 0x55eb9afb3e50 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:11.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.177+0000 7f41cb5b0740 1 --2- 172.21.15.100:0/1516213703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb9abe8390 0x55eb9afb3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:11.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.177+0000 7f41cb5b0740 1 --2- 172.21.15.100:0/1516213703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb9abe01b0 0x55eb9afac680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:11.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.177+0000 7f41cb5b0740 1 -- 172.21.15.100:0/1516213703 >> 172.21.15.100:0/1516213703 conn(0x55eb9abfd460 msgr2=0x55eb9afabf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:11.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.177+0000 7f41cb5b0740 1 -- 172.21.15.100:0/1516213703 shutdown_connections 2024-07-26T13:45:11.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.177+0000 7f41cb5b0740 1 -- 172.21.15.100:0/1516213703 wait complete. 2024-07-26T13:45:11.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.177+0000 7f41cb5b0740 1 -- start start 2024-07-26T13:45:11.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.177+0000 7f41cb5b0740 1 Processor -- start 2024-07-26T13:45:11.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.178+0000 7f41cb5b0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb9abe01b0 0x55eb9afcf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:11.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.178+0000 7f41cb5b0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb9abe8390 0x55eb9afcaa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:11.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.178+0000 7f41cb5b0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb9ac09070 0x55eb9afcafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:11.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.178+0000 7f41cb5b0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f41c01afb60 con 0x55eb9ac09070 2024-07-26T13:45:11.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.178+0000 7f41cb5b0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f41c01b0fd0 con 0x55eb9abe8390 2024-07-26T13:45:11.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.178+0000 7f41cb5b0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f41c01b0230 con 0x55eb9abe01b0 2024-07-26T13:45:11.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.178+0000 7f41cb5b0740 10 client.0 _ll_register_callbacks cb 0x7f41b8009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:45:11.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.178+0000 7f41bf59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb9ac09070 0x55eb9afcafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:11.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.178+0000 7f41bf59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb9ac09070 0x55eb9afcafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35338/0 (socket says 172.21.15.100:35338) 2024-07-26T13:45:11.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.178+0000 7f41bf59e640 1 -- 172.21.15.100:0/2542775231 learned_addr learned my addr 172.21.15.100:0/2542775231 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:11.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.178+0000 7f41be59c640 1 --2- 172.21.15.100:0/2542775231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb9abe8390 0x55eb9afcaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:11.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.178+0000 7f41bed9d640 1 --2- 172.21.15.100:0/2542775231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb9abe01b0 0x55eb9afcf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:11.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.178+0000 7f41bf59e640 1 -- 172.21.15.100:0/2542775231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb9abe01b0 msgr2=0x55eb9afcf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:11.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.178+0000 7f41bf59e640 1 --2- 172.21.15.100:0/2542775231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb9abe01b0 0x55eb9afcf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:11.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.178+0000 7f41bf59e640 1 -- 172.21.15.100:0/2542775231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb9abe8390 msgr2=0x55eb9afcaa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:11.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.178+0000 7f41bf59e640 1 --2- 172.21.15.100:0/2542775231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb9abe8390 0x55eb9afcaa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:11.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.178+0000 7f41bf59e640 1 -- 172.21.15.100:0/2542775231 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb9afcb4e0 con 0x55eb9ac09070 2024-07-26T13:45:11.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.178+0000 7f41bf59e640 1 --2- 172.21.15.100:0/2542775231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb9ac09070 0x55eb9afcafa0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f41b8000f80 tx=0x7f41b802af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3926a3244aa793f9 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:11.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.178+0000 7f41a2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:45:11.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.178+0000 7f41a2ff5640 1 -- 172.21.15.100:0/2542775231 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b8065020 con 0x55eb9ac09070 2024-07-26T13:45:11.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.178+0000 7f41a2ff5640 1 -- 172.21.15.100:0/2542775231 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7f41b8003d80 con 0x55eb9ac09070 2024-07-26T13:45:11.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.178+0000 7f41cb5b0740 1 -- 172.21.15.100:0/2542775231 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41b8036040 con 0x55eb9ac09070 2024-07-26T13:45:11.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.179+0000 7f41a2ff5640 1 -- 172.21.15.100:0/2542775231 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f41b8070050 con 0x55eb9ac09070 2024-07-26T13:45:11.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.179+0000 7f41cb5b0740 10 client.16782 fetch_fsmap learned FSMap version 45 2024-07-26T13:45:11.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.179+0000 7f41cb5b0740 1 -- 172.21.15.100:0/2542775231 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55eb9b034320 con 0x55eb9ac09070 2024-07-26T13:45:11.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.179+0000 7f41a2ff5640 1 -- 172.21.15.100:0/2542775231 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41b8036040 con 0x55eb9ac09070 2024-07-26T13:45:11.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.179+0000 7f41a2ff5640 1 -- 172.21.15.100:0/2542775231 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f41b804d040 con 0x55eb9ac09070 2024-07-26T13:45:11.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.179+0000 7f41cb5b0740 10 client.16782 fetch_fsmap finished waiting for FSMap version 45 2024-07-26T13:45:11.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.179+0000 7f41cb5b0740 10 client.16782 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-26T13:45:11.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.179+0000 7f41cb5b0740 20 client.16782 populate_metadata read hostname 'smithi100' 2024-07-26T13:45:11.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.179+0000 7f41cb5b0740 1 --2- 172.21.15.100:0/2542775231 >> [v2:172.21.15.167:6836/2476068386,v1:172.21.15.167:6837/2476068386] conn(0x55eb9ab9cda0 0x55eb9b036960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:11.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.179+0000 7f41cb5b0740 4 client.16782 mds_command: new command op to 25762 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-26T13:45:11.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.179+0000 7f41cb5b0740 1 -- 172.21.15.100:0/2542775231 --> [v2:172.21.15.167:6836/2476068386,v1:172.21.15.167:6837/2476068386] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eb9af225a0 con 0x55eb9ab9cda0 2024-07-26T13:45:11.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.179+0000 7f41bed9d640 1 --2- 172.21.15.100:0/2542775231 >> [v2:172.21.15.167:6836/2476068386,v1:172.21.15.167:6837/2476068386] conn(0x55eb9ab9cda0 0x55eb9b036960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:11.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.180+0000 7f41bed9d640 1 --2- 172.21.15.100:0/2542775231 >> [v2:172.21.15.167:6836/2476068386,v1:172.21.15.167:6837/2476068386] conn(0x55eb9ab9cda0 0x55eb9b036960 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8ff53fe09c96b79a server_cookie=7d461be4168bc64c in_seq=0 out_seq=0 2024-07-26T13:45:11.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.180+0000 7f41a2ff5640 10 client.16782 ms_handle_connect on v2:172.21.15.167:6836/2476068386 2024-07-26T13:45:11.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.180+0000 7f41a2ff5640 1 -- 172.21.15.100:0/2542775231 <== mds.2 v2:172.21.15.167:6836/2476068386 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eb9af225a0 con 0x55eb9ab9cda0 2024-07-26T13:45:11.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.180+0000 7f41a2ff5640 10 client.16782 handle_command_reply: tid=0 2024-07-26T13:45:11.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.180+0000 7f41cb5b0740 1 client.16782 shutdown 2024-07-26T13:45:11.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.180+0000 7f41cb5b0740 1 -- 172.21.15.100:0/2542775231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb9ac09070 msgr2=0x55eb9afcafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:11.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.180+0000 7f41cb5b0740 1 --2- 172.21.15.100:0/2542775231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb9ac09070 0x55eb9afcafa0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f41b8000f80 tx=0x7f41b802af40 comp rx=0 tx=0).stop 2024-07-26T13:45:11.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.180+0000 7f41cb5b0740 1 -- 172.21.15.100:0/2542775231 shutdown_connections 2024-07-26T13:45:11.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.180+0000 7f41cb5b0740 1 --2- 172.21.15.100:0/2542775231 >> [v2:172.21.15.167:6836/2476068386,v1:172.21.15.167:6837/2476068386] conn(0x55eb9ab9cda0 0x55eb9b036960 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:11.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.180+0000 7f41cb5b0740 1 --2- 172.21.15.100:0/2542775231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55eb9ac09070 0x55eb9afcafa0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:11.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.180+0000 7f41cb5b0740 1 --2- 172.21.15.100:0/2542775231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb9abe8390 0x55eb9afcaa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:11.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.180+0000 7f41a2ff5640 0 client.16782 ms_handle_reset on v2:172.21.15.167:6836/2476068386 2024-07-26T13:45:11.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.180+0000 7f41cb5b0740 1 --2- 172.21.15.100:0/2542775231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb9abe01b0 0x55eb9afcf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:11.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.180+0000 7f41cb5b0740 1 -- 172.21.15.100:0/2542775231 >> 172.21.15.100:0/2542775231 conn(0x55eb9abfd460 msgr2=0x55eb9afb1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:11.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.180+0000 7f41cb5b0740 1 -- 172.21.15.100:0/2542775231 shutdown_connections 2024-07-26T13:45:11.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.180+0000 7f41cb5b0740 1 -- 172.21.15.100:0/2542775231 wait complete. 2024-07-26T13:45:11.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.180+0000 7f41cb5b0740 20 client.16782 trim_cache size 0 max 16384 2024-07-26T13:45:11.188 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-07-26T13:45:11.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.181+0000 7f41c502c640 1 -- 172.21.15.100:0/337353607 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f41a80b3d40 msgr2=0x7f41a80b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:11.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.181+0000 7f41c502c640 1 --2- 172.21.15.100:0/337353607 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f41a80b3d40 0x7f41a80b6250 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x55eb9afad310 tx=0x7f41b000b390 comp rx=0 tx=0).stop 2024-07-26T13:45:11.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.181+0000 7f41c502c640 1 -- 172.21.15.100:0/337353607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41c0107de0 msgr2=0x7f41c019a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:11.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.181+0000 7f41c502c640 1 --2- 172.21.15.100:0/337353607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41c0107de0 0x7f41c019a400 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f41b400c970 tx=0x7f41b400ce30 comp rx=0 tx=0).stop 2024-07-26T13:45:11.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.181+0000 7f41c502c640 1 -- 172.21.15.100:0/337353607 shutdown_connections 2024-07-26T13:45:11.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.181+0000 7f41c502c640 1 --2- 172.21.15.100:0/337353607 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f41a80b3d40 0x7f41a80b6250 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:11.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.181+0000 7f41c502c640 1 --2- 172.21.15.100:0/337353607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41c0108730 0x7f41c019f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:11.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.181+0000 7f41c502c640 1 --2- 172.21.15.100:0/337353607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41c0107de0 0x7f41c019a400 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:11.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.181+0000 7f41c502c640 1 --2- 172.21.15.100:0/337353607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41c01024a0 0x7f41c0199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:11.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.181+0000 7f41c502c640 1 -- 172.21.15.100:0/337353607 >> 172.21.15.100:0/337353607 conn(0x7f41c0079140 msgr2=0x7f41c01012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:11.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.181+0000 7f41c502c640 1 -- 172.21.15.100:0/337353607 shutdown_connections 2024-07-26T13:45:11.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:11.181+0000 7f41c502c640 1 -- 172.21.15.100:0/337353607 wait complete. 2024-07-26T13:45:11.416 INFO:tasks.workunit.client.0.smithi100.stdout: 1 175223 1.86 MB/sec execute 85 sec latency 4828.262 ms 2024-07-26T13:45:11.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:11 smithi138 ceph-mon[36954]: pgmap v518: 97 pgs: 97 active+clean; 67 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.4 MiB/s wr, 247 op/s 2024-07-26T13:45:11.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:11 smithi100 ceph-mon[32599]: pgmap v518: 97 pgs: 97 active+clean; 67 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.4 MiB/s wr, 247 op/s 2024-07-26T13:45:12.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:11 smithi167 ceph-mon[37140]: pgmap v518: 97 pgs: 97 active+clean; 67 MiB data, 871 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.4 MiB/s wr, 247 op/s 2024-07-26T13:45:12.416 INFO:tasks.workunit.client.0.smithi100.stdout: 1 175223 1.84 MB/sec execute 86 sec latency 5828.350 ms 2024-07-26T13:45:13.064 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-26T13:45:13.064 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-26T13:45:13.332 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:45:13.416 INFO:tasks.workunit.client.0.smithi100.stdout: 1 175223 1.82 MB/sec execute 87 sec latency 6828.444 ms 2024-07-26T13:45:13.719 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:13 smithi100 ceph-mon[32599]: pgmap v519: 97 pgs: 97 active+clean; 67 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.2 MiB/s wr, 204 op/s 2024-07-26T13:45:13.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:13 smithi138 ceph-mon[36954]: pgmap v519: 97 pgs: 97 active+clean; 67 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.2 MiB/s wr, 204 op/s 2024-07-26T13:45:14.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:13 smithi167 ceph-mon[37140]: pgmap v519: 97 pgs: 97 active+clean; 67 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.2 MiB/s wr, 204 op/s 2024-07-26T13:45:14.416 INFO:tasks.workunit.client.0.smithi100.stdout: 1 175223 1.80 MB/sec execute 88 sec latency 7828.552 ms 2024-07-26T13:45:15.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.205+0000 7fb429569640 1 -- 172.21.15.100:0/2345038998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb424075b30 msgr2=0x7fb42410f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:15.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.205+0000 7fb429569640 1 --2- 172.21.15.100:0/2345038998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb424075b30 0x7fb42410f5b0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009900 tx=0x7fb40c02a410 comp rx=0 tx=0).stop 2024-07-26T13:45:15.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.205+0000 7fb429569640 1 -- 172.21.15.100:0/2345038998 shutdown_connections 2024-07-26T13:45:15.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.205+0000 7fb429569640 1 --2- 172.21.15.100:0/2345038998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb42410faf0 0x7fb424111f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:15.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.205+0000 7fb429569640 1 --2- 172.21.15.100:0/2345038998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb424075b30 0x7fb42410f5b0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:15.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.205+0000 7fb429569640 1 --2- 172.21.15.100:0/2345038998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4240751e0 0x7fb4240755f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:15.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.205+0000 7fb429569640 1 -- 172.21.15.100:0/2345038998 >> 172.21.15.100:0/2345038998 conn(0x7fb4240ff000 msgr2=0x7fb424101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:15.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.205+0000 7fb429569640 1 -- 172.21.15.100:0/2345038998 shutdown_connections 2024-07-26T13:45:15.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.205+0000 7fb429569640 1 -- 172.21.15.100:0/2345038998 wait complete. 2024-07-26T13:45:15.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.206+0000 7fb429569640 1 Processor -- start 2024-07-26T13:45:15.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.206+0000 7fb429569640 1 -- start start 2024-07-26T13:45:15.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.206+0000 7fb429569640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4240751e0 0x7fb42419c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:15.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.206+0000 7fb429569640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb424075b30 0x7fb42419c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:15.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.206+0000 7fb429569640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb42410faf0 0x7fb4241a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:15.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.206+0000 7fb429569640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb424115690 con 0x7fb4240751e0 2024-07-26T13:45:15.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.206+0000 7fb429569640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb424115510 con 0x7fb424075b30 2024-07-26T13:45:15.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.206+0000 7fb429569640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb424115810 con 0x7fb42410faf0 2024-07-26T13:45:15.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.206+0000 7fb422ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4240751e0 0x7fb42419c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:15.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.206+0000 7fb422ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4240751e0 0x7fb42419c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53856/0 (socket says 172.21.15.100:53856) 2024-07-26T13:45:15.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.206+0000 7fb422ffd640 1 -- 172.21.15.100:0/713070523 learned_addr learned my addr 172.21.15.100:0/713070523 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:15.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.206+0000 7fb4237fe640 1 --2- 172.21.15.100:0/713070523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb42410faf0 0x7fb4241a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:15.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.206+0000 7fb4227fc640 1 --2- 172.21.15.100:0/713070523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb424075b30 0x7fb42419c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:15.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.207+0000 7fb422ffd640 1 -- 172.21.15.100:0/713070523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb42410faf0 msgr2=0x7fb4241a19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:15.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.207+0000 7fb422ffd640 1 --2- 172.21.15.100:0/713070523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb42410faf0 0x7fb4241a19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:15.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.207+0000 7fb422ffd640 1 -- 172.21.15.100:0/713070523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb424075b30 msgr2=0x7fb42419c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:15.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.207+0000 7fb422ffd640 1 --2- 172.21.15.100:0/713070523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb424075b30 0x7fb42419c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:15.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.207+0000 7fb422ffd640 1 -- 172.21.15.100:0/713070523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4241a20f0 con 0x7fb4240751e0 2024-07-26T13:45:15.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.207+0000 7fb422ffd640 1 --2- 172.21.15.100:0/713070523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4240751e0 0x7fb42419c180 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fb41400cce0 tx=0x7fb41400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:15.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.207+0000 7fb41bfff640 1 -- 172.21.15.100:0/713070523 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41400f450 con 0x7fb4240751e0 2024-07-26T13:45:15.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.207+0000 7fb41bfff640 1 -- 172.21.15.100:0/713070523 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb41400faa0 con 0x7fb4240751e0 2024-07-26T13:45:15.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.207+0000 7fb429569640 1 -- 172.21.15.100:0/713070523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4241a23e0 con 0x7fb4240751e0 2024-07-26T13:45:15.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.207+0000 7fb41bfff640 1 -- 172.21.15.100:0/713070523 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb414018bf0 con 0x7fb4240751e0 2024-07-26T13:45:15.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.207+0000 7fb429569640 1 -- 172.21.15.100:0/713070523 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4241a8b40 con 0x7fb4240751e0 2024-07-26T13:45:15.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.209+0000 7fb42faed740 1 Processor -- start 2024-07-26T13:45:15.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.209+0000 7fb42faed740 1 -- start start 2024-07-26T13:45:15.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.210+0000 7fb41bfff640 1 -- 172.21.15.100:0/713070523 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fb4140213f0 con 0x7fb4240751e0 2024-07-26T13:45:15.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.210+0000 7fb42faed740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564128255350 0x5641285fa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:15.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.210+0000 7fb42faed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412822c490 0x5641285fa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:15.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.210+0000 7fb42faed740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564128234670 0x5641285fae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:15.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.210+0000 7fb42faed740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5641284e22f0 con 0x564128255350 2024-07-26T13:45:15.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.210+0000 7fb42faed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564128570a80 con 0x56412822c490 2024-07-26T13:45:15.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.210+0000 7fb42faed740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56412855ff80 con 0x564128234670 2024-07-26T13:45:15.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.210+0000 7fb4237fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564128255350 0x5641285fa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:15.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.210+0000 7fb4237fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564128255350 0x5641285fa3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53868/0 (socket says 172.21.15.100:53868) 2024-07-26T13:45:15.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.210+0000 7fb4237fe640 1 -- 172.21.15.100:0/1204646425 learned_addr learned my addr 172.21.15.100:0/1204646425 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:15.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.210+0000 7fb422ffd640 1 --2- 172.21.15.100:0/1204646425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412822c490 0x5641285fa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:15.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.210+0000 7fb4227fc640 1 --2- 172.21.15.100:0/1204646425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564128234670 0x5641285fae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:15.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.210+0000 7fb4237fe640 1 -- 172.21.15.100:0/1204646425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564128234670 msgr2=0x5641285fae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:15.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.210+0000 7fb4237fe640 1 --2- 172.21.15.100:0/1204646425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564128234670 0x5641285fae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:15.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.210+0000 7fb4237fe640 1 -- 172.21.15.100:0/1204646425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412822c490 msgr2=0x5641285fa8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:15.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.210+0000 7fb4237fe640 1 --2- 172.21.15.100:0/1204646425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412822c490 0x5641285fa8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:15.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.210+0000 7fb4237fe640 1 -- 172.21.15.100:0/1204646425 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641282c0930 con 0x564128255350 2024-07-26T13:45:15.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.211+0000 7fb4237fe640 1 --2- 172.21.15.100:0/1204646425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564128255350 0x5641285fa3b0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fb41c00b6b0 tx=0x7fb41c004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a25c90d0ccdf4180 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:15.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.211+0000 7fb4197fa640 1 -- 172.21.15.100:0/1204646425 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c003b20 con 0x564128255350 2024-07-26T13:45:15.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.211+0000 7fb41bfff640 1 --2- 172.21.15.100:0/713070523 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb4100b3d40 0x7fb4100b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:15.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.211+0000 7fb4197fa640 1 -- 172.21.15.100:0/1204646425 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb41c015070 con 0x564128255350 2024-07-26T13:45:15.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.211+0000 7fb4197fa640 1 -- 172.21.15.100:0/1204646425 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c0116a0 con 0x564128255350 2024-07-26T13:45:15.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.211+0000 7fb422ffd640 1 --2- 172.21.15.100:0/713070523 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb4100b3d40 0x7fb4100b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:15.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.211+0000 7fb41bfff640 1 -- 172.21.15.100:0/713070523 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7fb4140e46c0 con 0x7fb4240751e0 2024-07-26T13:45:15.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.212+0000 7fb42faed740 1 -- 172.21.15.100:0/1204646425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564128255350 msgr2=0x5641285fa3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:15.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.212+0000 7fb422ffd640 1 --2- 172.21.15.100:0/713070523 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb4100b3d40 0x7fb4100b6250 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x5641285f9030 tx=0x7fb41400ff50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:15.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.212+0000 7fb42faed740 1 --2- 172.21.15.100:0/1204646425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564128255350 0x5641285fa3b0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fb41c00b6b0 tx=0x7fb41c004610 comp rx=0 tx=0).stop 2024-07-26T13:45:15.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.212+0000 7fb42faed740 1 -- 172.21.15.100:0/1204646425 shutdown_connections 2024-07-26T13:45:15.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.212+0000 7fb42faed740 1 --2- 172.21.15.100:0/1204646425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564128234670 0x5641285fae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:15.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.212+0000 7fb42faed740 1 --2- 172.21.15.100:0/1204646425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412822c490 0x5641285fa8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:15.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.212+0000 7fb42faed740 1 --2- 172.21.15.100:0/1204646425 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564128255350 0x5641285fa3b0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:15.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.212+0000 7fb42faed740 1 -- 172.21.15.100:0/1204646425 >> 172.21.15.100:0/1204646425 conn(0x564128249740 msgr2=0x5641285f82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:15.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.212+0000 7fb42faed740 1 -- 172.21.15.100:0/1204646425 shutdown_connections 2024-07-26T13:45:15.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.212+0000 7fb42faed740 1 -- 172.21.15.100:0/1204646425 wait complete. 2024-07-26T13:45:15.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.212+0000 7fb42faed740 1 -- start start 2024-07-26T13:45:15.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.213+0000 7fb42faed740 1 Processor -- start 2024-07-26T13:45:15.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.213+0000 7fb42faed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412822c490 0x564128605fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:15.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.213+0000 7fb42faed740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564128234670 0x564128606520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:15.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.213+0000 7fb42faed740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564128255350 0x564128611d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:15.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.213+0000 7fb42faed740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5641285024f0 con 0x564128255350 2024-07-26T13:45:15.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.213+0000 7fb42faed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564128563fc0 con 0x56412822c490 2024-07-26T13:45:15.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.213+0000 7fb42faed740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5641283c3890 con 0x564128234670 2024-07-26T13:45:15.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.213+0000 7fb42faed740 10 client.0 _ll_register_callbacks cb 0x56412850c500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:45:15.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.213+0000 7fb4237fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564128255350 0x564128611d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:15.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.213+0000 7fb4237fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564128255350 0x564128611d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53880/0 (socket says 172.21.15.100:53880) 2024-07-26T13:45:15.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.213+0000 7fb4237fe640 1 -- 172.21.15.100:0/3349189730 learned_addr learned my addr 172.21.15.100:0/3349189730 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:15.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.213+0000 7fb4237fe640 1 --2- 172.21.15.100:0/3349189730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412822c490 0x564128605fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:15.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.213+0000 7fb4227fc640 1 --2- 172.21.15.100:0/3349189730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564128234670 0x564128606520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:15.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.213+0000 7fb4237fe640 1 -- 172.21.15.100:0/3349189730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564128234670 msgr2=0x564128606520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:15.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.213+0000 7fb4237fe640 1 --2- 172.21.15.100:0/3349189730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564128234670 0x564128606520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:15.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.213+0000 7fb4237fe640 1 -- 172.21.15.100:0/3349189730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412822c490 msgr2=0x564128605fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:15.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.213+0000 7fb4237fe640 1 --2- 172.21.15.100:0/3349189730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412822c490 0x564128605fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:15.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.213+0000 7fb4237fe640 1 -- 172.21.15.100:0/3349189730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564128511c50 con 0x564128255350 2024-07-26T13:45:15.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.214+0000 7fb4237fe640 1 --2- 172.21.15.100:0/3349189730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564128255350 0x564128611d60 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fb41c000c00 tx=0x7fb41c01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9fb430dd5f55e9a server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:15.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.214+0000 7fb40a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:45:15.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.214+0000 7fb40a7fc640 1 -- 172.21.15.100:0/3349189730 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c023040 con 0x564128255350 2024-07-26T13:45:15.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.214+0000 7fb40a7fc640 1 -- 172.21.15.100:0/3349189730 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7fb41c01a670 con 0x564128255350 2024-07-26T13:45:15.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.214+0000 7fb42faed740 1 -- 172.21.15.100:0/3349189730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb41c015070 con 0x564128255350 2024-07-26T13:45:15.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.214+0000 7fb40a7fc640 1 -- 172.21.15.100:0/3349189730 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fb41c01a870 con 0x564128255350 2024-07-26T13:45:15.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.214+0000 7fb42faed740 10 client.16794 fetch_fsmap learned FSMap version 45 2024-07-26T13:45:15.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.214+0000 7fb42faed740 1 -- 172.21.15.100:0/3349189730 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x564128614a90 con 0x564128255350 2024-07-26T13:45:15.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.214+0000 7fb40a7fc640 1 -- 172.21.15.100:0/3349189730 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb41c015070 con 0x564128255350 2024-07-26T13:45:15.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.214+0000 7fb40a7fc640 1 -- 172.21.15.100:0/3349189730 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7fb41c030070 con 0x564128255350 2024-07-26T13:45:15.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.214+0000 7fb42faed740 10 client.16794 fetch_fsmap finished waiting for FSMap version 45 2024-07-26T13:45:15.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.214+0000 7fb42faed740 10 client.16794 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:45:15.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.214+0000 7fb42faed740 20 client.16794 populate_metadata read hostname 'smithi100' 2024-07-26T13:45:15.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.214+0000 7fb42faed740 1 --2- 172.21.15.100:0/3349189730 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x564128213cd0 0x564128617df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:15.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.214+0000 7fb42faed740 4 client.16794 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:45:15.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.214+0000 7fb42faed740 1 -- 172.21.15.100:0/3349189730 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641285729d0 con 0x564128213cd0 2024-07-26T13:45:15.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.215+0000 7fb4227fc640 1 --2- 172.21.15.100:0/3349189730 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x564128213cd0 0x564128617df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:15.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.215+0000 7fb4227fc640 1 --2- 172.21.15.100:0/3349189730 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x564128213cd0 0x564128617df0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5780c2142eb1b75 server_cookie=37fdc0a688efa05c in_seq=0 out_seq=0 2024-07-26T13:45:15.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.215+0000 7fb40a7fc640 10 client.16794 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:15.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.216+0000 7fb40a7fc640 1 -- 172.21.15.100:0/3349189730 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5641285729d0 con 0x564128213cd0 2024-07-26T13:45:15.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.216+0000 7fb40a7fc640 10 client.16794 handle_command_reply: tid=0 2024-07-26T13:45:15.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.216+0000 7fb42faed740 1 client.16794 shutdown 2024-07-26T13:45:15.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.216+0000 7fb42faed740 1 -- 172.21.15.100:0/3349189730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564128255350 msgr2=0x564128611d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:15.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.216+0000 7fb42faed740 1 --2- 172.21.15.100:0/3349189730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564128255350 0x564128611d60 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fb41c000c00 tx=0x7fb41c01acb0 comp rx=0 tx=0).stop 2024-07-26T13:45:15.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.216+0000 7fb42faed740 1 -- 172.21.15.100:0/3349189730 shutdown_connections 2024-07-26T13:45:15.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.216+0000 7fb42faed740 1 --2- 172.21.15.100:0/3349189730 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x564128213cd0 0x564128617df0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:15.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.216+0000 7fb42faed740 1 --2- 172.21.15.100:0/3349189730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564128255350 0x564128611d60 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:15.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.216+0000 7fb40a7fc640 0 client.16794 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:15.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.216+0000 7fb42faed740 1 --2- 172.21.15.100:0/3349189730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564128234670 0x564128606520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:15.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.216+0000 7fb42faed740 1 --2- 172.21.15.100:0/3349189730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412822c490 0x564128605fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:15.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.216+0000 7fb42faed740 1 -- 172.21.15.100:0/3349189730 >> 172.21.15.100:0/3349189730 conn(0x564128249740 msgr2=0x5641285fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:15.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.216+0000 7fb42faed740 1 -- 172.21.15.100:0/3349189730 shutdown_connections 2024-07-26T13:45:15.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.216+0000 7fb42faed740 1 -- 172.21.15.100:0/3349189730 wait complete. 2024-07-26T13:45:15.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.216+0000 7fb42faed740 20 client.16794 trim_cache size 0 max 16384 2024-07-26T13:45:15.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.233+0000 7fb42faed740 1 Processor -- start 2024-07-26T13:45:15.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.233+0000 7fb42faed740 1 -- start start 2024-07-26T13:45:15.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.234+0000 7fb42faed740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564128213cd0 0x564128607a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:15.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.234+0000 7fb42faed740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56412822c490 0x5641285f78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:15.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.234+0000 7fb42faed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564128234670 0x5641285f7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:15.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.234+0000 7fb42faed740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb4241b3e00 con 0x56412822c490 2024-07-26T13:45:15.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.234+0000 7fb42faed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb4241b1670 con 0x564128234670 2024-07-26T13:45:15.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.234+0000 7fb42faed740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb4241af4a0 con 0x564128213cd0 2024-07-26T13:45:15.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.234+0000 7fb4227fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56412822c490 0x5641285f78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:15.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.234+0000 7fb4227fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56412822c490 0x5641285f78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53890/0 (socket says 172.21.15.100:53890) 2024-07-26T13:45:15.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.234+0000 7fb4227fc640 1 -- 172.21.15.100:0/3687847469 learned_addr learned my addr 172.21.15.100:0/3687847469 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:15.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.234+0000 7fb4237fe640 1 --2- 172.21.15.100:0/3687847469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564128213cd0 0x564128607a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:15.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.234+0000 7fb4237fe640 1 --2- 172.21.15.100:0/3687847469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564128234670 0x5641285f7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:15.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.234+0000 7fb4227fc640 1 -- 172.21.15.100:0/3687847469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564128213cd0 msgr2=0x564128607a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:15.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.234+0000 7fb4227fc640 1 --2- 172.21.15.100:0/3687847469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564128213cd0 0x564128607a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:15.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.234+0000 7fb4227fc640 1 -- 172.21.15.100:0/3687847469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564128234670 msgr2=0x5641285f7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:15.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.234+0000 7fb4227fc640 1 --2- 172.21.15.100:0/3687847469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564128234670 0x5641285f7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:15.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.234+0000 7fb4227fc640 1 -- 172.21.15.100:0/3687847469 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641285fd210 con 0x56412822c490 2024-07-26T13:45:15.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.234+0000 7fb4237fe640 1 --2- 172.21.15.100:0/3687847469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564128234670 0x5641285f7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:45:15.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.235+0000 7fb4227fc640 1 --2- 172.21.15.100:0/3687847469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56412822c490 0x5641285f78d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009340 tx=0x7fb40c0092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=793a1e004828b23d server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:15.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.235+0000 7fb418ff9640 1 -- 172.21.15.100:0/3687847469 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c035040 con 0x56412822c490 2024-07-26T13:45:15.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.235+0000 7fb418ff9640 1 -- 172.21.15.100:0/3687847469 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb40c049070 con 0x56412822c490 2024-07-26T13:45:15.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.235+0000 7fb418ff9640 1 -- 172.21.15.100:0/3687847469 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c004a40 con 0x56412822c490 2024-07-26T13:45:15.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.236+0000 7fb42faed740 1 -- 172.21.15.100:0/3687847469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56412822c490 msgr2=0x5641285f78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:15.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.236+0000 7fb42faed740 1 --2- 172.21.15.100:0/3687847469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56412822c490 0x5641285f78d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009340 tx=0x7fb40c0092d0 comp rx=0 tx=0).stop 2024-07-26T13:45:15.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.236+0000 7fb42faed740 1 -- 172.21.15.100:0/3687847469 shutdown_connections 2024-07-26T13:45:15.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.236+0000 7fb42faed740 1 --2- 172.21.15.100:0/3687847469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564128234670 0x5641285f7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:15.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.236+0000 7fb42faed740 1 --2- 172.21.15.100:0/3687847469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56412822c490 0x5641285f78d0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:15.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.236+0000 7fb42faed740 1 --2- 172.21.15.100:0/3687847469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564128213cd0 0x564128607a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:15.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.236+0000 7fb42faed740 1 -- 172.21.15.100:0/3687847469 >> 172.21.15.100:0/3687847469 conn(0x564128249740 msgr2=0x564128607300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:15.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.236+0000 7fb42faed740 1 -- 172.21.15.100:0/3687847469 shutdown_connections 2024-07-26T13:45:15.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.236+0000 7fb42faed740 1 -- 172.21.15.100:0/3687847469 wait complete. 2024-07-26T13:45:15.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.236+0000 7fb42faed740 1 -- start start 2024-07-26T13:45:15.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.236+0000 7fb42faed740 1 Processor -- start 2024-07-26T13:45:15.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.237+0000 7fb42faed740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564128213cd0 0x564128609f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:15.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.237+0000 7fb42faed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412822c490 0x56412860a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:15.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.237+0000 7fb42faed740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564128234670 0x56412867f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:15.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.237+0000 7fb42faed740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb4241afb30 con 0x564128213cd0 2024-07-26T13:45:15.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.237+0000 7fb42faed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb4241b0fa0 con 0x56412822c490 2024-07-26T13:45:15.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.237+0000 7fb42faed740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb4241b0200 con 0x564128234670 2024-07-26T13:45:15.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.237+0000 7fb42faed740 10 client.0 _ll_register_callbacks cb 0x7fb41c011ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:45:15.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.237+0000 7fb4237fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564128213cd0 0x564128609f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:15.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.237+0000 7fb4237fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564128213cd0 0x564128609f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53898/0 (socket says 172.21.15.100:53898) 2024-07-26T13:45:15.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.237+0000 7fb4237fe640 1 -- 172.21.15.100:0/1569049385 learned_addr learned my addr 172.21.15.100:0/1569049385 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:15.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.237+0000 7fb4227fc640 1 --2- 172.21.15.100:0/1569049385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412822c490 0x56412860a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:15.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.237+0000 7fb4237fe640 1 -- 172.21.15.100:0/1569049385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564128234670 msgr2=0x56412867f460 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-07-26T13:45:15.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.237+0000 7fb4237fe640 1 --2- 172.21.15.100:0/1569049385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564128234670 0x56412867f460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:15.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.237+0000 7fb4237fe640 1 -- 172.21.15.100:0/1569049385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412822c490 msgr2=0x56412860a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:15.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.237+0000 7fb4237fe640 1 --2- 172.21.15.100:0/1569049385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412822c490 0x56412860a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:15.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.237+0000 7fb4237fe640 1 -- 172.21.15.100:0/1569049385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56412867f9a0 con 0x564128213cd0 2024-07-26T13:45:15.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.237+0000 7fb4227fc640 1 --2- 172.21.15.100:0/1569049385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412822c490 0x56412860a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:45:15.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.237+0000 7fb4237fe640 1 --2- 172.21.15.100:0/1569049385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564128213cd0 0x564128609f20 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fb41c004160 tx=0x7fb41c02c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9c5bf2fc41a6f2 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:15.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.237+0000 7fb40a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:45:15.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.237+0000 7fb40a7fc640 1 -- 172.21.15.100:0/1569049385 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c036020 con 0x564128213cd0 2024-07-26T13:45:15.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.237+0000 7fb40a7fc640 1 -- 172.21.15.100:0/1569049385 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7fb41c0117d0 con 0x564128213cd0 2024-07-26T13:45:15.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.237+0000 7fb42faed740 1 -- 172.21.15.100:0/1569049385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb40c049070 con 0x564128213cd0 2024-07-26T13:45:15.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.238+0000 7fb40a7fc640 1 -- 172.21.15.100:0/1569049385 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fb41c041050 con 0x564128213cd0 2024-07-26T13:45:15.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.238+0000 7fb42faed740 10 client.16800 fetch_fsmap learned FSMap version 45 2024-07-26T13:45:15.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.238+0000 7fb42faed740 1 -- 172.21.15.100:0/1569049385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x564128680a20 con 0x564128213cd0 2024-07-26T13:45:15.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.238+0000 7fb40a7fc640 1 -- 172.21.15.100:0/1569049385 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb40c049070 con 0x564128213cd0 2024-07-26T13:45:15.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.238+0000 7fb40a7fc640 1 -- 172.21.15.100:0/1569049385 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7fb41c030040 con 0x564128213cd0 2024-07-26T13:45:15.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.238+0000 7fb42faed740 10 client.16800 fetch_fsmap finished waiting for FSMap version 45 2024-07-26T13:45:15.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.238+0000 7fb42faed740 10 client.16800 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:45:15.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.238+0000 7fb42faed740 20 client.16800 populate_metadata read hostname 'smithi100' 2024-07-26T13:45:15.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.238+0000 7fb42faed740 1 --2- 172.21.15.100:0/1569049385 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x564128255350 0x5641286831e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:15.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.238+0000 7fb42faed740 4 client.16800 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-26T13:45:15.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.238+0000 7fb42faed740 1 -- 172.21.15.100:0/1569049385 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56412856e470 con 0x564128255350 2024-07-26T13:45:15.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.238+0000 7fb4227fc640 1 --2- 172.21.15.100:0/1569049385 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x564128255350 0x5641286831e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:15.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.239+0000 7fb40a7fc640 10 client.16800 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:15.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.239+0000 7fb4227fc640 1 --2- 172.21.15.100:0/1569049385 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x564128255350 0x5641286831e0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be2bffd2fca3bc40 server_cookie=4f779f4f2f1ad1f2 in_seq=0 out_seq=0 2024-07-26T13:45:15.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.240+0000 7fb40a7fc640 1 -- 172.21.15.100:0/1569049385 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56412856e470 con 0x564128255350 2024-07-26T13:45:15.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.240+0000 7fb40a7fc640 10 client.16800 handle_command_reply: tid=0 2024-07-26T13:45:15.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.240+0000 7fb42faed740 1 client.16800 shutdown 2024-07-26T13:45:15.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.240+0000 7fb42faed740 1 -- 172.21.15.100:0/1569049385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564128213cd0 msgr2=0x564128609f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:15.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.240+0000 7fb42faed740 1 --2- 172.21.15.100:0/1569049385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564128213cd0 0x564128609f20 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fb41c004160 tx=0x7fb41c02c760 comp rx=0 tx=0).stop 2024-07-26T13:45:15.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.240+0000 7fb42faed740 1 -- 172.21.15.100:0/1569049385 shutdown_connections 2024-07-26T13:45:15.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.240+0000 7fb42faed740 1 --2- 172.21.15.100:0/1569049385 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x564128255350 0x5641286831e0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:15.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.240+0000 7fb40a7fc640 0 client.16800 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:15.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.240+0000 7fb42faed740 1 --2- 172.21.15.100:0/1569049385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564128234670 0x56412867f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:15.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.240+0000 7fb42faed740 1 --2- 172.21.15.100:0/1569049385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56412822c490 0x56412860a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:15.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.240+0000 7fb42faed740 1 --2- 172.21.15.100:0/1569049385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564128213cd0 0x564128609f20 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:15.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.240+0000 7fb42faed740 1 -- 172.21.15.100:0/1569049385 >> 172.21.15.100:0/1569049385 conn(0x564128249740 msgr2=0x564128607a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:15.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.241+0000 7fb42faed740 1 -- 172.21.15.100:0/1569049385 shutdown_connections 2024-07-26T13:45:15.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.241+0000 7fb42faed740 1 -- 172.21.15.100:0/1569049385 wait complete. 2024-07-26T13:45:15.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.241+0000 7fb42faed740 20 client.16800 trim_cache size 0 max 16384 2024-07-26T13:45:15.338 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:45:15.338 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-07-26T13:45:15.338 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "3345a0cd-8286-41b3-b75f-339b48f00230", 2024-07-26T13:45:15.338 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-07-26T13:45:15.338 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:45:15.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.241+0000 7fb429569640 1 -- 172.21.15.100:0/713070523 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb4100b3d40 msgr2=0x7fb4100b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:15.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.241+0000 7fb429569640 1 --2- 172.21.15.100:0/713070523 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb4100b3d40 0x7fb4100b6250 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x5641285f9030 tx=0x7fb41400ff50 comp rx=0 tx=0).stop 2024-07-26T13:45:15.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.241+0000 7fb429569640 1 -- 172.21.15.100:0/713070523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4240751e0 msgr2=0x7fb42419c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:15.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.241+0000 7fb429569640 1 --2- 172.21.15.100:0/713070523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4240751e0 0x7fb42419c180 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fb41400cce0 tx=0x7fb41400a620 comp rx=0 tx=0).stop 2024-07-26T13:45:15.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.241+0000 7fb429569640 1 -- 172.21.15.100:0/713070523 shutdown_connections 2024-07-26T13:45:15.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.241+0000 7fb429569640 1 --2- 172.21.15.100:0/713070523 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb4100b3d40 0x7fb4100b6250 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:15.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.241+0000 7fb429569640 1 --2- 172.21.15.100:0/713070523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb42410faf0 0x7fb4241a19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:15.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.241+0000 7fb429569640 1 --2- 172.21.15.100:0/713070523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb424075b30 0x7fb42419c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:15.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.241+0000 7fb429569640 1 --2- 172.21.15.100:0/713070523 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4240751e0 0x7fb42419c180 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:15.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.241+0000 7fb429569640 1 -- 172.21.15.100:0/713070523 >> 172.21.15.100:0/713070523 conn(0x7fb4240ff000 msgr2=0x7fb424108580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:15.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.241+0000 7fb429569640 1 -- 172.21.15.100:0/713070523 shutdown_connections 2024-07-26T13:45:15.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:15.241+0000 7fb429569640 1 -- 172.21.15.100:0/713070523 wait complete. 2024-07-26T13:45:15.416 INFO:tasks.workunit.client.0.smithi100.stdout: 1 175405 1.78 MB/sec execute 89 sec latency 8704.078 ms 2024-07-26T13:45:15.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:15 smithi138 ceph-mon[36954]: pgmap v520: 97 pgs: 97 active+clean; 67 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.1 MiB/s wr, 194 op/s 2024-07-26T13:45:15.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:15 smithi138 ceph-mon[36954]: scrub summary: active paths [/] 2024-07-26T13:45:15.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:15 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:45:15.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:15 smithi138 ceph-mon[36954]: scrub summary: active paths [/] 2024-07-26T13:45:15.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:15 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:45:15.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:15 smithi100 ceph-mon[32599]: pgmap v520: 97 pgs: 97 active+clean; 67 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.1 MiB/s wr, 194 op/s 2024-07-26T13:45:15.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:15 smithi100 ceph-mon[32599]: scrub summary: active paths [/] 2024-07-26T13:45:15.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:15 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:45:15.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:15 smithi100 ceph-mon[32599]: scrub summary: active paths [/] 2024-07-26T13:45:15.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:15 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:45:16.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:15 smithi167 ceph-mon[37140]: pgmap v520: 97 pgs: 97 active+clean; 67 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.1 MiB/s wr, 194 op/s 2024-07-26T13:45:16.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:15 smithi167 ceph-mon[37140]: scrub summary: active paths [/] 2024-07-26T13:45:16.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:15 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:45:16.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:15 smithi167 ceph-mon[37140]: scrub summary: active paths [/] 2024-07-26T13:45:16.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:15 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:45:16.116 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-26T13:45:16.387 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:45:16.416 INFO:tasks.workunit.client.0.smithi100.stdout: 1 177663 1.91 MB/sec execute 90 sec latency 14.900 ms 2024-07-26T13:45:16.841 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:16 smithi100 ceph-mon[32599]: scrub queued for path: / 2024-07-26T13:45:16.842 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:16 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:45:16.842 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:16 smithi100 ceph-mon[32599]: scrub summary: active paths [/] 2024-07-26T13:45:16.842 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:16 smithi100 ceph-mon[32599]: scrub summary: active paths [/] 2024-07-26T13:45:16.842 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:16 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:45:16.842 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:16 smithi100 ceph-mon[32599]: scrub summary: active paths [/] 2024-07-26T13:45:16.842 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:16 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:45:16.842 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:16 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:45:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:16 smithi138 ceph-mon[36954]: scrub queued for path: / 2024-07-26T13:45:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:16 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:45:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:16 smithi138 ceph-mon[36954]: scrub summary: active paths [/] 2024-07-26T13:45:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:16 smithi138 ceph-mon[36954]: scrub summary: active paths [/] 2024-07-26T13:45:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:16 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:45:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:16 smithi138 ceph-mon[36954]: scrub summary: active paths [/] 2024-07-26T13:45:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:16 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:45:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:16 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:45:17.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:16 smithi167 ceph-mon[37140]: scrub queued for path: / 2024-07-26T13:45:17.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:16 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:45:17.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:16 smithi167 ceph-mon[37140]: scrub summary: active paths [/] 2024-07-26T13:45:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:16 smithi167 ceph-mon[37140]: scrub summary: active paths [/] 2024-07-26T13:45:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:16 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:45:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:16 smithi167 ceph-mon[37140]: scrub summary: active paths [/] 2024-07-26T13:45:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:16 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:45:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:16 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:45:17.416 INFO:tasks.workunit.client.0.smithi100.stdout: 1 179662 1.97 MB/sec execute 91 sec latency 23.004 ms 2024-07-26T13:45:17.775 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:17 smithi100 ceph-mon[32599]: pgmap v521: 97 pgs: 97 active+clean; 67 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 893 KiB/s wr, 139 op/s 2024-07-26T13:45:17.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:17 smithi138 ceph-mon[36954]: pgmap v521: 97 pgs: 97 active+clean; 67 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 893 KiB/s wr, 139 op/s 2024-07-26T13:45:18.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:17 smithi167 ceph-mon[37140]: pgmap v521: 97 pgs: 97 active+clean; 67 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 893 KiB/s wr, 139 op/s 2024-07-26T13:45:18.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.170+0000 7fd924ae1640 1 -- 172.21.15.100:0/2400455136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd920107dd0 msgr2=0x7fd9201081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:18.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.170+0000 7fd924ae1640 1 --2- 172.21.15.100:0/2400455136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd920107dd0 0x7fd9201081e0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fd914009900 tx=0x7fd91402a410 comp rx=0 tx=0).stop 2024-07-26T13:45:18.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.170+0000 7fd924ae1640 1 -- 172.21.15.100:0/2400455136 shutdown_connections 2024-07-26T13:45:18.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.170+0000 7fd924ae1640 1 --2- 172.21.15.100:0/2400455136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd920108720 0x7fd92010ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:18.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.170+0000 7fd924ae1640 1 --2- 172.21.15.100:0/2400455136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd920107dd0 0x7fd9201081e0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:18.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.170+0000 7fd924ae1640 1 --2- 172.21.15.100:0/2400455136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd920102490 0x7fd9201028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:18.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.170+0000 7fd924ae1640 1 -- 172.21.15.100:0/2400455136 >> 172.21.15.100:0/2400455136 conn(0x7fd920079160 msgr2=0x7fd920079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:18.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.170+0000 7fd924ae1640 1 -- 172.21.15.100:0/2400455136 shutdown_connections 2024-07-26T13:45:18.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.170+0000 7fd924ae1640 1 -- 172.21.15.100:0/2400455136 wait complete. 2024-07-26T13:45:18.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.171+0000 7fd924ae1640 1 Processor -- start 2024-07-26T13:45:18.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.171+0000 7fd924ae1640 1 -- start start 2024-07-26T13:45:18.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.171+0000 7fd924ae1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd920102490 0x7fd920104b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:18.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.171+0000 7fd924ae1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd920107dd0 0x7fd920105050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:18.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.171+0000 7fd924ae1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd920108720 0x7fd920105590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:18.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.171+0000 7fd924ae1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd920112370 con 0x7fd920102490 2024-07-26T13:45:18.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.171+0000 7fd924ae1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd9201121f0 con 0x7fd920108720 2024-07-26T13:45:18.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.171+0000 7fd924ae1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd9201124f0 con 0x7fd920107dd0 2024-07-26T13:45:18.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.171+0000 7fd91e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd920102490 0x7fd920104b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:18.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.171+0000 7fd91e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd920102490 0x7fd920104b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53924/0 (socket says 172.21.15.100:53924) 2024-07-26T13:45:18.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.171+0000 7fd91e59c640 1 -- 172.21.15.100:0/253236591 learned_addr learned my addr 172.21.15.100:0/253236591 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:18.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.171+0000 7fd91ed9d640 1 --2- 172.21.15.100:0/253236591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd920108720 0x7fd920105590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:18.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.171+0000 7fd91dd9b640 1 --2- 172.21.15.100:0/253236591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd920107dd0 0x7fd920105050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:18.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.172+0000 7fd91e59c640 1 -- 172.21.15.100:0/253236591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd920107dd0 msgr2=0x7fd920105050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:18.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.172+0000 7fd91e59c640 1 --2- 172.21.15.100:0/253236591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd920107dd0 0x7fd920105050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:18.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.172+0000 7fd91e59c640 1 -- 172.21.15.100:0/253236591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd920108720 msgr2=0x7fd920105590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:18.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.172+0000 7fd91e59c640 1 --2- 172.21.15.100:0/253236591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd920108720 0x7fd920105590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:18.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.172+0000 7fd91e59c640 1 -- 172.21.15.100:0/253236591 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9201a8140 con 0x7fd920102490 2024-07-26T13:45:18.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.172+0000 7fd91ed9d640 1 --2- 172.21.15.100:0/253236591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd920108720 0x7fd920105590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:45:18.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.172+0000 7fd91dd9b640 1 --2- 172.21.15.100:0/253236591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd920107dd0 0x7fd920105050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:45:18.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.172+0000 7fd91e59c640 1 --2- 172.21.15.100:0/253236591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd920102490 0x7fd920104b10 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fd91000e9f0 tx=0x7fd91000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:18.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.172+0000 7fd90f7fe640 1 -- 172.21.15.100:0/253236591 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91000cd80 con 0x7fd920102490 2024-07-26T13:45:18.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.172+0000 7fd924ae1640 1 -- 172.21.15.100:0/253236591 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9201a8430 con 0x7fd920102490 2024-07-26T13:45:18.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.172+0000 7fd90f7fe640 1 -- 172.21.15.100:0/253236591 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9100108e0 con 0x7fd920102490 2024-07-26T13:45:18.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.172+0000 7fd90f7fe640 1 -- 172.21.15.100:0/253236591 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd910018980 con 0x7fd920102490 2024-07-26T13:45:18.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.172+0000 7fd924ae1640 1 -- 172.21.15.100:0/253236591 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9201a8990 con 0x7fd920102490 2024-07-26T13:45:18.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.175+0000 7fd92b065740 1 Processor -- start 2024-07-26T13:45:18.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.175+0000 7fd92b065740 1 -- start start 2024-07-26T13:45:18.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.175+0000 7fd90f7fe640 1 -- 172.21.15.100:0/253236591 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fd910018b20 con 0x7fd920102490 2024-07-26T13:45:18.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.175+0000 7fd92b065740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56292ea75070 0x56292ee1a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:18.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.175+0000 7fd92b065740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56292ea4c1b0 0x56292ee1aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:18.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.175+0000 7fd92b065740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56292ea54390 0x56292ee1afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:18.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.175+0000 7fd92b065740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56292ed90b90 con 0x56292ea54390 2024-07-26T13:45:18.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.175+0000 7fd92b065740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56292ed80090 con 0x56292ea75070 2024-07-26T13:45:18.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.175+0000 7fd92b065740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56292ed840d0 con 0x56292ea4c1b0 2024-07-26T13:45:18.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.175+0000 7fd91dd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56292ea54390 0x56292ee1afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:18.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.175+0000 7fd91dd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56292ea54390 0x56292ee1afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53928/0 (socket says 172.21.15.100:53928) 2024-07-26T13:45:18.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.175+0000 7fd91e59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56292ea4c1b0 0x56292ee1aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:18.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.175+0000 7fd91dd9b640 1 -- 172.21.15.100:0/3893738357 learned_addr learned my addr 172.21.15.100:0/3893738357 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:18.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.175+0000 7fd91ed9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56292ea75070 0x56292ee1a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:18.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.176+0000 7fd91ed9d640 1 -- 172.21.15.100:0/3893738357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56292ea4c1b0 msgr2=0x56292ee1aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:18.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.176+0000 7fd91ed9d640 1 --2- 172.21.15.100:0/3893738357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56292ea4c1b0 0x56292ee1aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:18.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.176+0000 7fd91ed9d640 1 -- 172.21.15.100:0/3893738357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56292ea54390 msgr2=0x56292ee1afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:18.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.176+0000 7fd91ed9d640 1 --2- 172.21.15.100:0/3893738357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56292ea54390 0x56292ee1afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:18.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.176+0000 7fd91ed9d640 1 -- 172.21.15.100:0/3893738357 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56292ed26b70 con 0x56292ea75070 2024-07-26T13:45:18.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.176+0000 7fd91e59c640 1 --2- 172.21.15.100:0/3893738357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56292ea4c1b0 0x56292ee1aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:45:18.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.176+0000 7fd91dd9b640 1 --2- 172.21.15.100:0/3893738357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56292ea54390 0x56292ee1afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:45:18.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.176+0000 7fd91ed9d640 1 --2- 172.21.15.100:0/3893738357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56292ea75070 0x56292ee1a550 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd918005fd0 tx=0x7fd918011bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=337510ec4780a1f2 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:18.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.176+0000 7fd90cff9640 1 -- 172.21.15.100:0/3893738357 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd918004d20 con 0x56292ea75070 2024-07-26T13:45:18.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.176+0000 7fd90cff9640 1 -- 172.21.15.100:0/3893738357 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd91800f9a0 con 0x56292ea75070 2024-07-26T13:45:18.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.176+0000 7fd90cff9640 1 -- 172.21.15.100:0/3893738357 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd918004410 con 0x56292ea75070 2024-07-26T13:45:18.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.176+0000 7fd90f7fe640 1 --2- 172.21.15.100:0/253236591 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd9080b3d40 0x7fd9080b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:18.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.176+0000 7fd91e59c640 1 --2- 172.21.15.100:0/253236591 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd9080b3d40 0x7fd9080b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:18.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.177+0000 7fd90f7fe640 1 -- 172.21.15.100:0/253236591 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7fd9100e4400 con 0x7fd920102490 2024-07-26T13:45:18.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.177+0000 7fd92b065740 1 -- 172.21.15.100:0/3893738357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56292ea75070 msgr2=0x56292ee1a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:18.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.177+0000 7fd91e59c640 1 --2- 172.21.15.100:0/253236591 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd9080b3d40 0x7fd9080b6250 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x56292ee191d0 tx=0x7fd9100e9040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:18.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.177+0000 7fd92b065740 1 --2- 172.21.15.100:0/3893738357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56292ea75070 0x56292ee1a550 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd918005fd0 tx=0x7fd918011bf0 comp rx=0 tx=0).stop 2024-07-26T13:45:18.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.177+0000 7fd92b065740 1 -- 172.21.15.100:0/3893738357 shutdown_connections 2024-07-26T13:45:18.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.177+0000 7fd92b065740 1 --2- 172.21.15.100:0/3893738357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56292ea54390 0x56292ee1afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:18.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.177+0000 7fd92b065740 1 --2- 172.21.15.100:0/3893738357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56292ea4c1b0 0x56292ee1aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:18.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.177+0000 7fd92b065740 1 --2- 172.21.15.100:0/3893738357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56292ea75070 0x56292ee1a550 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:18.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.177+0000 7fd92b065740 1 -- 172.21.15.100:0/3893738357 >> 172.21.15.100:0/3893738357 conn(0x56292ea69460 msgr2=0x56292ee18440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:18.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.177+0000 7fd92b065740 1 -- 172.21.15.100:0/3893738357 shutdown_connections 2024-07-26T13:45:18.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.177+0000 7fd92b065740 1 -- 172.21.15.100:0/3893738357 wait complete. 2024-07-26T13:45:18.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.178+0000 7fd92b065740 1 -- start start 2024-07-26T13:45:18.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.178+0000 7fd92b065740 1 Processor -- start 2024-07-26T13:45:18.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.178+0000 7fd92b065740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56292ea4c1b0 0x56292ee260e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:18.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.178+0000 7fd92b065740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56292ea54390 0x56292ee26620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:18.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.178+0000 7fd92b065740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56292ea75070 0x56292ee32050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:18.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.178+0000 7fd92b065740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56292ebe3700 con 0x56292ea4c1b0 2024-07-26T13:45:18.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.178+0000 7fd92b065740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56292ed229c0 con 0x56292ea75070 2024-07-26T13:45:18.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.178+0000 7fd92b065740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56292ee28d30 con 0x56292ea54390 2024-07-26T13:45:18.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.178+0000 7fd92b065740 10 client.0 _ll_register_callbacks cb 0x56292ed3d4d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:45:18.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.178+0000 7fd91ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56292ea4c1b0 0x56292ee260e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:18.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.178+0000 7fd91ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56292ea4c1b0 0x56292ee260e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53940/0 (socket says 172.21.15.100:53940) 2024-07-26T13:45:18.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.178+0000 7fd91ed9d640 1 -- 172.21.15.100:0/1635787748 learned_addr learned my addr 172.21.15.100:0/1635787748 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:18.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.178+0000 7fd91dd9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56292ea54390 0x56292ee26620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:18.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.179+0000 7fd91ed9d640 1 --2- 172.21.15.100:0/1635787748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56292ea75070 0x56292ee32050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:18.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.179+0000 7fd91dd9b640 1 -- 172.21.15.100:0/1635787748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56292ea75070 msgr2=0x56292ee32050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:18.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.179+0000 7fd91dd9b640 1 --2- 172.21.15.100:0/1635787748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56292ea75070 0x56292ee32050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:18.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.179+0000 7fd91dd9b640 1 -- 172.21.15.100:0/1635787748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56292ea4c1b0 msgr2=0x56292ee260e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:18.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.179+0000 7fd91dd9b640 1 --2- 172.21.15.100:0/1635787748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56292ea4c1b0 0x56292ee260e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:18.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.179+0000 7fd91dd9b640 1 -- 172.21.15.100:0/1635787748 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56292eae0650 con 0x56292ea54390 2024-07-26T13:45:18.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.179+0000 7fd91dd9b640 1 --2- 172.21.15.100:0/1635787748 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56292ea54390 0x56292ee26620 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd91402a930 tx=0x7fd9140316d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec1eb24df62138e2 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:18.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.179+0000 7fd905ffb640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-07-26T13:45:18.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.179+0000 7fd905ffb640 1 -- 172.21.15.100:0/1635787748 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd914035040 con 0x56292ea54390 2024-07-26T13:45:18.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.179+0000 7fd905ffb640 1 -- 172.21.15.100:0/1635787748 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7fd914004240 con 0x56292ea54390 2024-07-26T13:45:18.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.179+0000 7fd92b065740 1 -- 172.21.15.100:0/1635787748 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd91800f9a0 con 0x56292ea54390 2024-07-26T13:45:18.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.179+0000 7fd905ffb640 1 -- 172.21.15.100:0/1635787748 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fd914031900 con 0x56292ea54390 2024-07-26T13:45:18.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.179+0000 7fd92b065740 10 client.26251 fetch_fsmap learned FSMap version 45 2024-07-26T13:45:18.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.179+0000 7fd92b065740 1 -- 172.21.15.100:0/1635787748 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56292ee34d40 con 0x56292ea54390 2024-07-26T13:45:18.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.180+0000 7fd905ffb640 1 -- 172.21.15.100:0/1635787748 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd91800f9a0 con 0x56292ea54390 2024-07-26T13:45:18.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.180+0000 7fd905ffb640 1 -- 172.21.15.100:0/1635787748 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7fd91404b070 con 0x56292ea54390 2024-07-26T13:45:18.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.180+0000 7fd92b065740 10 client.26251 fetch_fsmap finished waiting for FSMap version 45 2024-07-26T13:45:18.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.180+0000 7fd92b065740 10 client.26251 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:45:18.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.180+0000 7fd92b065740 20 client.26251 populate_metadata read hostname 'smithi100' 2024-07-26T13:45:18.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.180+0000 7fd92b065740 1 --2- 172.21.15.100:0/1635787748 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56292ea339f0 0x56292ee38280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:18.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.180+0000 7fd92b065740 4 client.26251 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:45:18.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.180+0000 7fd92b065740 1 -- 172.21.15.100:0/1635787748 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56292ed92ae0 con 0x56292ea339f0 2024-07-26T13:45:18.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.180+0000 7fd91ed9d640 1 --2- 172.21.15.100:0/1635787748 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56292ea339f0 0x56292ee38280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:18.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.180+0000 7fd905ffb640 10 client.26251 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:18.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.180+0000 7fd91ed9d640 1 --2- 172.21.15.100:0/1635787748 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56292ea339f0 0x56292ee38280 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c215ff7f5c266aa server_cookie=fa633773a6123681 in_seq=0 out_seq=0 2024-07-26T13:45:18.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.181+0000 7fd905ffb640 1 -- 172.21.15.100:0/1635787748 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56292ed92ae0 con 0x56292ea339f0 2024-07-26T13:45:18.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.181+0000 7fd905ffb640 10 client.26251 handle_command_reply: tid=0 2024-07-26T13:45:18.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.181+0000 7fd92b065740 1 client.26251 shutdown 2024-07-26T13:45:18.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.181+0000 7fd92b065740 1 -- 172.21.15.100:0/1635787748 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56292ea54390 msgr2=0x56292ee26620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:18.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.181+0000 7fd92b065740 1 --2- 172.21.15.100:0/1635787748 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56292ea54390 0x56292ee26620 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd91402a930 tx=0x7fd9140316d0 comp rx=0 tx=0).stop 2024-07-26T13:45:18.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.181+0000 7fd92b065740 1 -- 172.21.15.100:0/1635787748 shutdown_connections 2024-07-26T13:45:18.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.181+0000 7fd92b065740 1 --2- 172.21.15.100:0/1635787748 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56292ea339f0 0x56292ee38280 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:18.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.181+0000 7fd92b065740 1 --2- 172.21.15.100:0/1635787748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56292ea75070 0x56292ee32050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:18.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.181+0000 7fd92b065740 1 --2- 172.21.15.100:0/1635787748 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56292ea54390 0x56292ee26620 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:18.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.181+0000 7fd905ffb640 0 client.26251 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:18.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.181+0000 7fd92b065740 1 --2- 172.21.15.100:0/1635787748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56292ea4c1b0 0x56292ee260e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:18.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.181+0000 7fd92b065740 1 -- 172.21.15.100:0/1635787748 >> 172.21.15.100:0/1635787748 conn(0x56292ea69460 msgr2=0x56292ee16c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:18.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.181+0000 7fd92b065740 1 -- 172.21.15.100:0/1635787748 shutdown_connections 2024-07-26T13:45:18.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.181+0000 7fd92b065740 1 -- 172.21.15.100:0/1635787748 wait complete. 2024-07-26T13:45:18.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.182+0000 7fd92b065740 20 client.26251 trim_cache size 0 max 16384 2024-07-26T13:45:18.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.199+0000 7fd92b065740 1 Processor -- start 2024-07-26T13:45:18.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.199+0000 7fd92b065740 1 -- start start 2024-07-26T13:45:18.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.199+0000 7fd92b065740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56292ea339f0 0x56292ee20670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:18.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.199+0000 7fd92b065740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56292ea4c1b0 0x56292ee31e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:18.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.199+0000 7fd92b065740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56292ea54390 0x56292ee3da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:18.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.199+0000 7fd92b065740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd9201b3d50 con 0x56292ea4c1b0 2024-07-26T13:45:18.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.199+0000 7fd92b065740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd9201b15c0 con 0x56292ea339f0 2024-07-26T13:45:18.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.199+0000 7fd92b065740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd9201af3f0 con 0x56292ea54390 2024-07-26T13:45:18.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.199+0000 7fd91dd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56292ea4c1b0 0x56292ee31e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:18.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.199+0000 7fd91dd9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56292ea4c1b0 0x56292ee31e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53952/0 (socket says 172.21.15.100:53952) 2024-07-26T13:45:18.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.199+0000 7fd91dd9b640 1 -- 172.21.15.100:0/1221187091 learned_addr learned my addr 172.21.15.100:0/1221187091 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:18.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.199+0000 7fd91ed9d640 1 --2- 172.21.15.100:0/1221187091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56292ea54390 0x56292ee3da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:18.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.199+0000 7fd91ed9d640 1 --2- 172.21.15.100:0/1221187091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56292ea339f0 0x56292ee20670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:18.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.200+0000 7fd91dd9b640 1 -- 172.21.15.100:0/1221187091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56292ea54390 msgr2=0x56292ee3da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:18.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.200+0000 7fd91dd9b640 1 --2- 172.21.15.100:0/1221187091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56292ea54390 0x56292ee3da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:18.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.200+0000 7fd91dd9b640 1 -- 172.21.15.100:0/1221187091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56292ea339f0 msgr2=0x56292ee20670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:18.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.200+0000 7fd91dd9b640 1 --2- 172.21.15.100:0/1221187091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56292ea339f0 0x56292ee20670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:18.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.200+0000 7fd91dd9b640 1 -- 172.21.15.100:0/1221187091 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56292ee21770 con 0x56292ea4c1b0 2024-07-26T13:45:18.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.200+0000 7fd91dd9b640 1 --2- 172.21.15.100:0/1221187091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56292ea4c1b0 0x56292ee31e40 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fd914003690 tx=0x7fd914005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df7ec83d7b20d004 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:18.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.200+0000 7fd907fff640 1 -- 172.21.15.100:0/1221187091 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd914035030 con 0x56292ea4c1b0 2024-07-26T13:45:18.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.200+0000 7fd907fff640 1 -- 172.21.15.100:0/1221187091 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd914005450 con 0x56292ea4c1b0 2024-07-26T13:45:18.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.200+0000 7fd907fff640 1 -- 172.21.15.100:0/1221187091 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd914051070 con 0x56292ea4c1b0 2024-07-26T13:45:18.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.201+0000 7fd92b065740 1 -- 172.21.15.100:0/1221187091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56292ea4c1b0 msgr2=0x56292ee31e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:18.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.201+0000 7fd92b065740 1 --2- 172.21.15.100:0/1221187091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56292ea4c1b0 0x56292ee31e40 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fd914003690 tx=0x7fd914005fd0 comp rx=0 tx=0).stop 2024-07-26T13:45:18.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.201+0000 7fd92b065740 1 -- 172.21.15.100:0/1221187091 shutdown_connections 2024-07-26T13:45:18.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.201+0000 7fd92b065740 1 --2- 172.21.15.100:0/1221187091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56292ea54390 0x56292ee3da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:18.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.201+0000 7fd92b065740 1 --2- 172.21.15.100:0/1221187091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56292ea4c1b0 0x56292ee31e40 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:18.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.201+0000 7fd92b065740 1 --2- 172.21.15.100:0/1221187091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56292ea339f0 0x56292ee20670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:18.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.201+0000 7fd92b065740 1 -- 172.21.15.100:0/1221187091 >> 172.21.15.100:0/1221187091 conn(0x56292ea69460 msgr2=0x56292ee24760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:18.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.201+0000 7fd92b065740 1 -- 172.21.15.100:0/1221187091 shutdown_connections 2024-07-26T13:45:18.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.201+0000 7fd92b065740 1 -- 172.21.15.100:0/1221187091 wait complete. 2024-07-26T13:45:18.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.201+0000 7fd92b065740 1 -- start start 2024-07-26T13:45:18.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.201+0000 7fd92b065740 1 Processor -- start 2024-07-26T13:45:18.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.202+0000 7fd92b065740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56292ea339f0 0x56292ee2e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:18.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.202+0000 7fd92b065740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56292ea4c1b0 0x56292ee29a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:18.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.202+0000 7fd92b065740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56292ea54390 0x56292ee29f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:18.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.202+0000 7fd92b065740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd9201afa80 con 0x56292ea54390 2024-07-26T13:45:18.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.202+0000 7fd92b065740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd9201b0ef0 con 0x56292ea339f0 2024-07-26T13:45:18.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.202+0000 7fd92b065740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd9201b0150 con 0x56292ea4c1b0 2024-07-26T13:45:18.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.202+0000 7fd92b065740 10 client.0 _ll_register_callbacks cb 0x7fd914009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:45:18.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.202+0000 7fd91ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56292ea54390 0x56292ee29f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:18.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.202+0000 7fd91ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56292ea54390 0x56292ee29f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53960/0 (socket says 172.21.15.100:53960) 2024-07-26T13:45:18.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.202+0000 7fd91ed9d640 1 -- 172.21.15.100:0/368587896 learned_addr learned my addr 172.21.15.100:0/368587896 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:18.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.202+0000 7fd91dd9b640 1 --2- 172.21.15.100:0/368587896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56292ea4c1b0 0x56292ee29a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:18.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.202+0000 7fd91ed9d640 1 --2- 172.21.15.100:0/368587896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56292ea339f0 0x56292ee2e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:18.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.202+0000 7fd91ed9d640 1 -- 172.21.15.100:0/368587896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56292ea4c1b0 msgr2=0x56292ee29a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:18.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.202+0000 7fd91ed9d640 1 --2- 172.21.15.100:0/368587896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56292ea4c1b0 0x56292ee29a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:18.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.202+0000 7fd91ed9d640 1 -- 172.21.15.100:0/368587896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56292ea339f0 msgr2=0x56292ee2e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:18.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.202+0000 7fd91ed9d640 1 --2- 172.21.15.100:0/368587896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56292ea339f0 0x56292ee2e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:18.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.202+0000 7fd91ed9d640 1 -- 172.21.15.100:0/368587896 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56292ee2a5d0 con 0x56292ea54390 2024-07-26T13:45:18.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.202+0000 7fd91ed9d640 1 --2- 172.21.15.100:0/368587896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56292ea54390 0x56292ee29f60 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fd918010010 tx=0x7fd918009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff74b6641115feb9 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:18.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.202+0000 7fd905ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:45:18.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.202+0000 7fd905ffb640 1 -- 172.21.15.100:0/368587896 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd918003e30 con 0x56292ea54390 2024-07-26T13:45:18.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.202+0000 7fd905ffb640 1 -- 172.21.15.100:0/368587896 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7fd918004fd0 con 0x56292ea54390 2024-07-26T13:45:18.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.202+0000 7fd92b065740 1 -- 172.21.15.100:0/368587896 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd914005450 con 0x56292ea54390 2024-07-26T13:45:18.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.203+0000 7fd905ffb640 1 -- 172.21.15.100:0/368587896 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fd9180051d0 con 0x56292ea54390 2024-07-26T13:45:18.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.203+0000 7fd92b065740 10 client.16818 fetch_fsmap learned FSMap version 45 2024-07-26T13:45:18.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.203+0000 7fd92b065740 1 -- 172.21.15.100:0/368587896 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56292eea0890 con 0x56292ea54390 2024-07-26T13:45:18.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.203+0000 7fd905ffb640 1 -- 172.21.15.100:0/368587896 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd914005450 con 0x56292ea54390 2024-07-26T13:45:18.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.203+0000 7fd905ffb640 1 -- 172.21.15.100:0/368587896 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7fd918018070 con 0x56292ea54390 2024-07-26T13:45:18.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.203+0000 7fd92b065740 10 client.16818 fetch_fsmap finished waiting for FSMap version 45 2024-07-26T13:45:18.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.203+0000 7fd92b065740 10 client.16818 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:45:18.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.203+0000 7fd92b065740 20 client.16818 populate_metadata read hostname 'smithi100' 2024-07-26T13:45:18.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.203+0000 7fd92b065740 1 --2- 172.21.15.100:0/368587896 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56292ea75070 0x56292eea2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:18.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.203+0000 7fd92b065740 4 client.16818 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-26T13:45:18.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.203+0000 7fd92b065740 1 -- 172.21.15.100:0/368587896 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56292ed8e580 con 0x56292ea75070 2024-07-26T13:45:18.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.203+0000 7fd91dd9b640 1 --2- 172.21.15.100:0/368587896 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56292ea75070 0x56292eea2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:18.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.204+0000 7fd91dd9b640 1 --2- 172.21.15.100:0/368587896 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56292ea75070 0x56292eea2d70 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a13c8b1e96cdeeb server_cookie=1c6bdd90698f5c13 in_seq=0 out_seq=0 2024-07-26T13:45:18.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.204+0000 7fd905ffb640 10 client.16818 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:18.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.204+0000 7fd905ffb640 1 -- 172.21.15.100:0/368587896 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56292ed8e580 con 0x56292ea75070 2024-07-26T13:45:18.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.204+0000 7fd905ffb640 10 client.16818 handle_command_reply: tid=0 2024-07-26T13:45:18.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.204+0000 7fd92b065740 1 client.16818 shutdown 2024-07-26T13:45:18.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.204+0000 7fd92b065740 1 -- 172.21.15.100:0/368587896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56292ea54390 msgr2=0x56292ee29f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:18.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.204+0000 7fd92b065740 1 --2- 172.21.15.100:0/368587896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56292ea54390 0x56292ee29f60 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fd918010010 tx=0x7fd918009500 comp rx=0 tx=0).stop 2024-07-26T13:45:18.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.204+0000 7fd92b065740 1 -- 172.21.15.100:0/368587896 shutdown_connections 2024-07-26T13:45:18.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.204+0000 7fd92b065740 1 --2- 172.21.15.100:0/368587896 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56292ea75070 0x56292eea2d70 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:18.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.204+0000 7fd92b065740 1 --2- 172.21.15.100:0/368587896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56292ea54390 0x56292ee29f60 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:18.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.204+0000 7fd92b065740 1 --2- 172.21.15.100:0/368587896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56292ea4c1b0 0x56292ee29a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:18.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.204+0000 7fd92b065740 1 --2- 172.21.15.100:0/368587896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56292ea339f0 0x56292ee2e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:18.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.204+0000 7fd92b065740 1 -- 172.21.15.100:0/368587896 >> 172.21.15.100:0/368587896 conn(0x56292ea69460 msgr2=0x56292ee17b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:18.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.204+0000 7fd905ffb640 0 client.16818 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:18.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.204+0000 7fd92b065740 1 -- 172.21.15.100:0/368587896 shutdown_connections 2024-07-26T13:45:18.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.204+0000 7fd92b065740 1 -- 172.21.15.100:0/368587896 wait complete. 2024-07-26T13:45:18.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.204+0000 7fd92b065740 20 client.16818 trim_cache size 0 max 16384 2024-07-26T13:45:18.213 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:45:18.213 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-26T13:45:18.213 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-07-26T13:45:18.214 INFO:teuthology.orchestra.run.smithi100.stdout: "3345a0cd-8286-41b3-b75f-339b48f00230": { 2024-07-26T13:45:18.214 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-07-26T13:45:18.214 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "3345a0cd-8286-41b3-b75f-339b48f00230", 2024-07-26T13:45:18.214 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-07-26T13:45:18.214 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:45:18.214 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:45:18.214 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:45:18.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.205+0000 7fd924ae1640 1 -- 172.21.15.100:0/253236591 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd9080b3d40 msgr2=0x7fd9080b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:18.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.205+0000 7fd924ae1640 1 --2- 172.21.15.100:0/253236591 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd9080b3d40 0x7fd9080b6250 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x56292ee191d0 tx=0x7fd9100e9040 comp rx=0 tx=0).stop 2024-07-26T13:45:18.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.205+0000 7fd924ae1640 1 -- 172.21.15.100:0/253236591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd920102490 msgr2=0x7fd920104b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:18.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.205+0000 7fd924ae1640 1 --2- 172.21.15.100:0/253236591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd920102490 0x7fd920104b10 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fd91000e9f0 tx=0x7fd91000eeb0 comp rx=0 tx=0).stop 2024-07-26T13:45:18.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.205+0000 7fd924ae1640 1 -- 172.21.15.100:0/253236591 shutdown_connections 2024-07-26T13:45:18.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.205+0000 7fd924ae1640 1 --2- 172.21.15.100:0/253236591 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd9080b3d40 0x7fd9080b6250 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:18.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.205+0000 7fd924ae1640 1 --2- 172.21.15.100:0/253236591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd920108720 0x7fd920105590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:18.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.205+0000 7fd924ae1640 1 --2- 172.21.15.100:0/253236591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd920107dd0 0x7fd920105050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:18.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.205+0000 7fd924ae1640 1 --2- 172.21.15.100:0/253236591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd920102490 0x7fd920104b10 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:18.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.205+0000 7fd924ae1640 1 -- 172.21.15.100:0/253236591 >> 172.21.15.100:0/253236591 conn(0x7fd920079160 msgr2=0x7fd92006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:18.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.205+0000 7fd924ae1640 1 -- 172.21.15.100:0/253236591 shutdown_connections 2024-07-26T13:45:18.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:18.205+0000 7fd924ae1640 1 -- 172.21.15.100:0/253236591 wait complete. 2024-07-26T13:45:18.416 INFO:tasks.workunit.client.0.smithi100.stdout: 1 181359 2.08 MB/sec execute 92 sec latency 12.609 ms 2024-07-26T13:45:19.133 INFO:tasks.cephfs.filesystem:scrub status for tag:3345a0cd-8286-41b3-b75f-339b48f00230 - {'path': '/', 'tag': '3345a0cd-8286-41b3-b75f-339b48f00230', 'options': 'recursive,force'} 2024-07-26T13:45:19.241 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-07-26T13:45:19.241 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:45:19.416 INFO:tasks.workunit.client.0.smithi100.stdout: 1 183494 2.14 MB/sec execute 93 sec latency 16.363 ms 2024-07-26T13:45:19.503 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:45:19.882 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:19 smithi100 ceph-mon[32599]: pgmap v522: 97 pgs: 97 active+clean; 70 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 761 KiB/s wr, 129 op/s 2024-07-26T13:45:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:19 smithi138 ceph-mon[36954]: pgmap v522: 97 pgs: 97 active+clean; 70 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 761 KiB/s wr, 129 op/s 2024-07-26T13:45:20.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:19 smithi167 ceph-mon[37140]: pgmap v522: 97 pgs: 97 active+clean; 70 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 761 KiB/s wr, 129 op/s 2024-07-26T13:45:20.416 INFO:tasks.workunit.client.0.smithi100.stdout: 1 185232 2.24 MB/sec execute 94 sec latency 9.703 ms 2024-07-26T13:45:21.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.306+0000 7fc1d782f640 1 -- 172.21.15.100:0/2215563756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1d0102490 msgr2=0x7fc1d01028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:21.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.306+0000 7fc1d782f640 1 --2- 172.21.15.100:0/2215563756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1d0102490 0x7fc1d01028a0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc009900 tx=0x7fc1cc02a410 comp rx=0 tx=0).stop 2024-07-26T13:45:21.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.306+0000 7fc1d782f640 1 -- 172.21.15.100:0/2215563756 shutdown_connections 2024-07-26T13:45:21.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.306+0000 7fc1d782f640 1 --2- 172.21.15.100:0/2215563756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1d0108720 0x7fc1d010ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.306+0000 7fc1d782f640 1 --2- 172.21.15.100:0/2215563756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1d0107dd0 0x7fc1d01081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.306+0000 7fc1d782f640 1 --2- 172.21.15.100:0/2215563756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1d0102490 0x7fc1d01028a0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.306+0000 7fc1d782f640 1 -- 172.21.15.100:0/2215563756 >> 172.21.15.100:0/2215563756 conn(0x7fc1d0079160 msgr2=0x7fc1d0079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:21.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.306+0000 7fc1d782f640 1 -- 172.21.15.100:0/2215563756 shutdown_connections 2024-07-26T13:45:21.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.306+0000 7fc1d782f640 1 -- 172.21.15.100:0/2215563756 wait complete. 2024-07-26T13:45:21.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.307+0000 7fc1d782f640 1 Processor -- start 2024-07-26T13:45:21.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.307+0000 7fc1d782f640 1 -- start start 2024-07-26T13:45:21.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.307+0000 7fc1d782f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1d0102490 0x7fc1d019acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:21.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.307+0000 7fc1d782f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1d0107dd0 0x7fc1d019b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:21.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.307+0000 7fc1d782f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1d0108720 0x7fc1d019b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:21.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.307+0000 7fc1d782f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d01123a0 con 0x7fc1d0102490 2024-07-26T13:45:21.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.307+0000 7fc1d782f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d0112220 con 0x7fc1d0108720 2024-07-26T13:45:21.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.307+0000 7fc1d782f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d0112520 con 0x7fc1d0107dd0 2024-07-26T13:45:21.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.307+0000 7fc1d55cb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1d0102490 0x7fc1d019acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:21.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.307+0000 7fc1d5dcc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1d0108720 0x7fc1d019b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:21.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.307+0000 7fc1d55cb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1d0102490 0x7fc1d019acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53980/0 (socket says 172.21.15.100:53980) 2024-07-26T13:45:21.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.307+0000 7fc1d55cb640 1 -- 172.21.15.100:0/252422687 learned_addr learned my addr 172.21.15.100:0/252422687 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:21.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.307+0000 7fc1d4dca640 1 --2- 172.21.15.100:0/252422687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1d0107dd0 0x7fc1d019b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:21.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.307+0000 7fc1d5dcc640 1 -- 172.21.15.100:0/252422687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1d0107dd0 msgr2=0x7fc1d019b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:21.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.307+0000 7fc1d5dcc640 1 --2- 172.21.15.100:0/252422687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1d0107dd0 0x7fc1d019b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.307+0000 7fc1d5dcc640 1 -- 172.21.15.100:0/252422687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1d0102490 msgr2=0x7fc1d019acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:21.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.307+0000 7fc1d5dcc640 1 --2- 172.21.15.100:0/252422687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1d0102490 0x7fc1d019acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.307+0000 7fc1d5dcc640 1 -- 172.21.15.100:0/252422687 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d01a5f70 con 0x7fc1d0108720 2024-07-26T13:45:21.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.307+0000 7fc1d55cb640 1 --2- 172.21.15.100:0/252422687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1d0102490 0x7fc1d019acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:45:21.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.308+0000 7fc1d5dcc640 1 --2- 172.21.15.100:0/252422687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1d0108720 0x7fc1d019b730 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8009fd0 tx=0x7fc1c800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:21.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.308+0000 7fc1c67fc640 1 -- 172.21.15.100:0/252422687 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c800ce20 con 0x7fc1d0108720 2024-07-26T13:45:21.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.308+0000 7fc1d782f640 1 -- 172.21.15.100:0/252422687 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1d01a6260 con 0x7fc1d0108720 2024-07-26T13:45:21.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.308+0000 7fc1c67fc640 1 -- 172.21.15.100:0/252422687 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1c8005020 con 0x7fc1d0108720 2024-07-26T13:45:21.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.308+0000 7fc1c67fc640 1 -- 172.21.15.100:0/252422687 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c8010690 con 0x7fc1d0108720 2024-07-26T13:45:21.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.308+0000 7fc1d782f640 1 -- 172.21.15.100:0/252422687 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1d01a67a0 con 0x7fc1d0108720 2024-07-26T13:45:21.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.309+0000 7fc1d782f640 1 -- 172.21.15.100:0/252422687 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1d01028a0 con 0x7fc1d0108720 2024-07-26T13:45:21.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.311+0000 7fc1c67fc640 1 -- 172.21.15.100:0/252422687 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fc1c80054d0 con 0x7fc1d0108720 2024-07-26T13:45:21.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.311+0000 7fc1c67fc640 1 --2- 172.21.15.100:0/252422687 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc1bc0b3d40 0x7fc1bc0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:21.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.311+0000 7fc1d55cb640 1 --2- 172.21.15.100:0/252422687 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc1bc0b3d40 0x7fc1bc0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:21.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.311+0000 7fc1c67fc640 1 -- 172.21.15.100:0/252422687 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7fc1c80e4790 con 0x7fc1d0108720 2024-07-26T13:45:21.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.312+0000 7fc1d55cb640 1 --2- 172.21.15.100:0/252422687 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc1bc0b3d40 0x7fc1bc0b6250 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc009230 tx=0x7fc1cc0058d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:21.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.316+0000 7fc1c67fc640 1 -- 172.21.15.100:0/252422687 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fc1c80aba20 con 0x7fc1d0108720 2024-07-26T13:45:21.417 INFO:tasks.workunit.client.0.smithi100.stdout: 1 187650 2.35 MB/sec execute 95 sec latency 19.857 ms 2024-07-26T13:45:21.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.537+0000 7fc1d782f640 1 -- 172.21.15.100:0/252422687 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1d0109f60 con 0x7fc1d0108720 2024-07-26T13:45:21.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.538+0000 7fc1c67fc640 1 -- 172.21.15.100:0/252422687 <== mon.2 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11931 (secure 0 0 0) 0x7fc1c80ab320 con 0x7fc1d0108720 2024-07-26T13:45:21.539 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:45:21.539 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":45,"btime":"2024-07-26T13:44:57:595027+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14790,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/313501409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":313501409},{"type":"v1","addr":"172.21.15.100:6839","nonce":313501409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/1402975506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":1402975506},{"type":"v1","addr":"172.21.15.100:6841","nonce":1402975506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-26T13:37:01.281843+0000","modified":"2024-07-26T13:44:57.595007+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24607,"mds_1":24605,"mds_2":25762},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24575":{"gid":24575,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/3393225483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3393225483},{"type":"v1","addr":"172.21.15.138:6837","nonce":3393225483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.138:6839/1580925471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1580925471},{"type":"v1","addr":"172.21.15.138:6839","nonce":1580925471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/398150514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":398150514},{"type":"v1","addr":"172.21.15.167:6839","nonce":398150514}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25762":{"gid":25762,"name":"c","rank":2,"incarnation":39,"state":"up:active","state_seq":32,"addr":"172.21.15.167:6837/2476068386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2476068386},{"type":"v1","addr":"172.21.15.167:6837","nonce":2476068386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25997":{"gid":25997,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/2831339503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2831339503},{"type":"v1","addr":"172.21.15.138:6835","nonce":2831339503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24607,"qdb_cluster":[24607,25762,24605]},"id":1}]} 2024-07-26T13:45:21.539 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 45 2024-07-26T13:45:21.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.540+0000 7fc1d782f640 1 -- 172.21.15.100:0/252422687 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc1bc0b3d40 msgr2=0x7fc1bc0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:21.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.540+0000 7fc1d782f640 1 --2- 172.21.15.100:0/252422687 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc1bc0b3d40 0x7fc1bc0b6250 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc009230 tx=0x7fc1cc0058d0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.540+0000 7fc1d782f640 1 -- 172.21.15.100:0/252422687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1d0108720 msgr2=0x7fc1d019b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:21.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.540+0000 7fc1d782f640 1 --2- 172.21.15.100:0/252422687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1d0108720 0x7fc1d019b730 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8009fd0 tx=0x7fc1c800ef10 comp rx=0 tx=0).stop 2024-07-26T13:45:21.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.540+0000 7fc1d782f640 1 -- 172.21.15.100:0/252422687 shutdown_connections 2024-07-26T13:45:21.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.540+0000 7fc1d782f640 1 --2- 172.21.15.100:0/252422687 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc1bc0b3d40 0x7fc1bc0b6250 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.540+0000 7fc1d782f640 1 --2- 172.21.15.100:0/252422687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1d0108720 0x7fc1d019b730 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.540+0000 7fc1d782f640 1 --2- 172.21.15.100:0/252422687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1d0107dd0 0x7fc1d019b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.540+0000 7fc1d782f640 1 --2- 172.21.15.100:0/252422687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1d0102490 0x7fc1d019acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.540+0000 7fc1d782f640 1 -- 172.21.15.100:0/252422687 >> 172.21.15.100:0/252422687 conn(0x7fc1d0079160 msgr2=0x7fc1d0100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:21.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.540+0000 7fc1d782f640 1 -- 172.21.15.100:0/252422687 shutdown_connections 2024-07-26T13:45:21.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:21.540+0000 7fc1d782f640 1 -- 172.21.15.100:0/252422687 wait complete. 2024-07-26T13:45:21.841 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:21 smithi100 ceph-mon[32599]: pgmap v523: 97 pgs: 97 active+clean; 72 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 628 KiB/s wr, 112 op/s 2024-07-26T13:45:21.842 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:21 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/252422687' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:45:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.881+0000 7f40198c4640 1 -- 172.21.15.100:0/1398941874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4014101f40 msgr2=0x7f4014102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.881+0000 7f40198c4640 1 --2- 172.21.15.100:0/1398941874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4014101f40 0x7f4014102350 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc009900 tx=0x7f3ffc02a480 comp rx=0 tx=0).stop 2024-07-26T13:45:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.881+0000 7f40198c4640 1 -- 172.21.15.100:0/1398941874 shutdown_connections 2024-07-26T13:45:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.881+0000 7f40198c4640 1 --2- 172.21.15.100:0/1398941874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f401410faf0 0x7f4014111f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.881+0000 7f40198c4640 1 --2- 172.21.15.100:0/1398941874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4014102890 0x7f401410f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.881+0000 7f40198c4640 1 --2- 172.21.15.100:0/1398941874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4014101f40 0x7f4014102350 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.881+0000 7f40198c4640 1 -- 172.21.15.100:0/1398941874 >> 172.21.15.100:0/1398941874 conn(0x7f401407a170 msgr2=0x7f401407a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.881+0000 7f40198c4640 1 -- 172.21.15.100:0/1398941874 shutdown_connections 2024-07-26T13:45:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.881+0000 7f40198c4640 1 -- 172.21.15.100:0/1398941874 wait complete. 2024-07-26T13:45:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.882+0000 7f40198c4640 1 Processor -- start 2024-07-26T13:45:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.882+0000 7f40198c4640 1 -- start start 2024-07-26T13:45:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.882+0000 7f40198c4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4014101f40 0x7f401419c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.882+0000 7f40198c4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4014102890 0x7f401419c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.882+0000 7f40198c4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f401410faf0 0x7f40141a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.882+0000 7f40198c4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f40141155c0 con 0x7f401410faf0 2024-07-26T13:45:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.882+0000 7f40198c4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4014115440 con 0x7f4014102890 2024-07-26T13:45:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.882+0000 7f40198c4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4014115740 con 0x7f4014101f40 2024-07-26T13:45:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.883+0000 7f40137fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f401410faf0 0x7f40141a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.883+0000 7f40137fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f401410faf0 0x7f40141a1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54006/0 (socket says 172.21.15.100:54006) 2024-07-26T13:45:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.883+0000 7f40137fe640 1 -- 172.21.15.100:0/1937592135 learned_addr learned my addr 172.21.15.100:0/1937592135 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.883+0000 7f40127fc640 1 --2- 172.21.15.100:0/1937592135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4014102890 0x7f401419c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.883+0000 7f4012ffd640 1 --2- 172.21.15.100:0/1937592135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4014101f40 0x7f401419c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.883+0000 7f40137fe640 1 -- 172.21.15.100:0/1937592135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4014101f40 msgr2=0x7f401419c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.883+0000 7f40137fe640 1 --2- 172.21.15.100:0/1937592135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4014101f40 0x7f401419c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.883+0000 7f40137fe640 1 -- 172.21.15.100:0/1937592135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4014102890 msgr2=0x7f401419c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.883+0000 7f40137fe640 1 --2- 172.21.15.100:0/1937592135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4014102890 0x7f401419c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.883+0000 7f40137fe640 1 -- 172.21.15.100:0/1937592135 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40141a1f90 con 0x7f401410faf0 2024-07-26T13:45:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.883+0000 7f40137fe640 1 --2- 172.21.15.100:0/1937592135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f401410faf0 0x7f40141a1890 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f400c00eb10 tx=0x7f400c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.883+0000 7f40188c2640 1 -- 172.21.15.100:0/1937592135 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c004b20 con 0x7f401410faf0 2024-07-26T13:45:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.883+0000 7f40188c2640 1 -- 172.21.15.100:0/1937592135 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f400c004cc0 con 0x7f401410faf0 2024-07-26T13:45:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.883+0000 7f40188c2640 1 -- 172.21.15.100:0/1937592135 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c010670 con 0x7f401410faf0 2024-07-26T13:45:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.883+0000 7f40198c4640 1 -- 172.21.15.100:0/1937592135 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40141a2280 con 0x7f401410faf0 2024-07-26T13:45:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.883+0000 7f40198c4640 1 -- 172.21.15.100:0/1937592135 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40141a8a10 con 0x7f401410faf0 2024-07-26T13:45:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.886+0000 7f40207d6740 1 Processor -- start 2024-07-26T13:45:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.887+0000 7f40188c2640 1 -- 172.21.15.100:0/1937592135 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f400c005020 con 0x7f401410faf0 2024-07-26T13:45:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.887+0000 7f40207d6740 1 -- start start 2024-07-26T13:45:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.887+0000 7f40207d6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633cb2381b0 0x5633cb5da6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.887+0000 7f40207d6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633cb20f2f0 0x5633cb5dabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:21.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.887+0000 7f40207d6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633cb2174d0 0x5633cb5db130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:21.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.887+0000 7f40207d6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5633cb5503b0 con 0x5633cb2174d0 2024-07-26T13:45:21.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.887+0000 7f40207d6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5633cb53f8b0 con 0x5633cb2381b0 2024-07-26T13:45:21.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.887+0000 7f40207d6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5633cb5438f0 con 0x5633cb20f2f0 2024-07-26T13:45:21.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.887+0000 7f40127fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633cb2174d0 0x5633cb5db130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:21.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.887+0000 7f4012ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633cb20f2f0 0x5633cb5dabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:21.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.887+0000 7f40127fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633cb2174d0 0x5633cb5db130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54018/0 (socket says 172.21.15.100:54018) 2024-07-26T13:45:21.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.887+0000 7f40127fc640 1 -- 172.21.15.100:0/2808111442 learned_addr learned my addr 172.21.15.100:0/2808111442 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:21.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.887+0000 7f40127fc640 1 --2- 172.21.15.100:0/2808111442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633cb2381b0 0x5633cb5da6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:21.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.887+0000 7f4012ffd640 1 -- 172.21.15.100:0/2808111442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633cb2381b0 msgr2=0x5633cb5da6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:21.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.887+0000 7f4012ffd640 1 --2- 172.21.15.100:0/2808111442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633cb2381b0 0x5633cb5da6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.887+0000 7f4012ffd640 1 -- 172.21.15.100:0/2808111442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633cb2174d0 msgr2=0x5633cb5db130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:21.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.887+0000 7f4012ffd640 1 --2- 172.21.15.100:0/2808111442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633cb2174d0 0x5633cb5db130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.890 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.887+0000 7f40188c2640 1 --2- 172.21.15.100:0/1937592135 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f40000b3dc0 0x7f40000b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:21.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.887+0000 7f4012ffd640 1 -- 172.21.15.100:0/2808111442 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633cb4e69a0 con 0x5633cb20f2f0 2024-07-26T13:45:21.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.888+0000 7f40137fe640 1 --2- 172.21.15.100:0/1937592135 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f40000b3dc0 0x7f40000b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:21.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.888+0000 7f4012ffd640 1 --2- 172.21.15.100:0/2808111442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633cb20f2f0 0x5633cb5dabf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc006010 tx=0x7f3ffc032b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18832946b4099e52 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:21.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.888+0000 7f40188c2640 1 -- 172.21.15.100:0/1937592135 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7f400c0e45a0 con 0x7f401410faf0 2024-07-26T13:45:21.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.888+0000 7f4009ffb640 1 -- 172.21.15.100:0/2808111442 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc035040 con 0x5633cb20f2f0 2024-07-26T13:45:21.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.888+0000 7f4009ffb640 1 -- 172.21.15.100:0/2808111442 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ffc032e00 con 0x5633cb20f2f0 2024-07-26T13:45:21.891 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.888+0000 7f4009ffb640 1 -- 172.21.15.100:0/2808111442 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc005330 con 0x5633cb20f2f0 2024-07-26T13:45:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.889+0000 7f40207d6740 1 -- 172.21.15.100:0/2808111442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633cb20f2f0 msgr2=0x5633cb5dabf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.889+0000 7f40137fe640 1 --2- 172.21.15.100:0/1937592135 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f40000b3dc0 0x7f40000b62d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f400c0aae30 tx=0x7f400c00cf70 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.889+0000 7f40207d6740 1 --2- 172.21.15.100:0/2808111442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633cb20f2f0 0x5633cb5dabf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc006010 tx=0x7f3ffc032b90 comp rx=0 tx=0).stop 2024-07-26T13:45:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.889+0000 7f40207d6740 1 -- 172.21.15.100:0/2808111442 shutdown_connections 2024-07-26T13:45:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.889+0000 7f40207d6740 1 --2- 172.21.15.100:0/2808111442 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633cb2174d0 0x5633cb5db130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.889+0000 7f40207d6740 1 --2- 172.21.15.100:0/2808111442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633cb20f2f0 0x5633cb5dabf0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.889+0000 7f40207d6740 1 --2- 172.21.15.100:0/2808111442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633cb2381b0 0x5633cb5da6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.889+0000 7f40207d6740 1 -- 172.21.15.100:0/2808111442 >> 172.21.15.100:0/2808111442 conn(0x5633cb22c300 msgr2=0x5633cb5d85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.889+0000 7f40207d6740 1 -- 172.21.15.100:0/2808111442 shutdown_connections 2024-07-26T13:45:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.889+0000 7f40207d6740 1 -- 172.21.15.100:0/2808111442 wait complete. 2024-07-26T13:45:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.889+0000 7f40207d6740 1 -- start start 2024-07-26T13:45:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.890+0000 7f40207d6740 1 Processor -- start 2024-07-26T13:45:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.890+0000 7f40207d6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633cb20f2f0 0x5633cb5e61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.890+0000 7f40207d6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633cb2174d0 0x5633cb5e66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.890+0000 7f40207d6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633cb2381b0 0x5633cb5f20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.890+0000 7f40207d6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5633cb479ab0 con 0x5633cb20f2f0 2024-07-26T13:45:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.890+0000 7f40207d6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5633cb4c7cf0 con 0x5633cb2381b0 2024-07-26T13:45:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.890+0000 7f40207d6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5633cb5e8dd0 con 0x5633cb2174d0 2024-07-26T13:45:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.890+0000 7f40207d6740 10 client.0 _ll_register_callbacks cb 0x5633cb5393e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:45:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.890+0000 7f4012ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633cb2174d0 0x5633cb5e66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.890+0000 7f40127fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633cb20f2f0 0x5633cb5e61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.890+0000 7f4012ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633cb2174d0 0x5633cb5e66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:42016/0 (socket says 172.21.15.100:42016) 2024-07-26T13:45:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.890+0000 7f4012ffd640 1 -- 172.21.15.100:0/3918525221 learned_addr learned my addr 172.21.15.100:0/3918525221 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.890+0000 7f40127fc640 1 --2- 172.21.15.100:0/3918525221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633cb2381b0 0x5633cb5f20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.890+0000 7f40127fc640 1 -- 172.21.15.100:0/3918525221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633cb2174d0 msgr2=0x5633cb5e66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.890+0000 7f40127fc640 1 --2- 172.21.15.100:0/3918525221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633cb2174d0 0x5633cb5e66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.890+0000 7f40127fc640 1 -- 172.21.15.100:0/3918525221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633cb2381b0 msgr2=0x5633cb5f20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.890+0000 7f40127fc640 1 --2- 172.21.15.100:0/3918525221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633cb2381b0 0x5633cb5f20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.890+0000 7f40127fc640 1 -- 172.21.15.100:0/3918525221 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633cb2a3790 con 0x5633cb20f2f0 2024-07-26T13:45:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.890+0000 7f4012ffd640 1 --2- 172.21.15.100:0/3918525221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633cb2174d0 0x5633cb5e66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:45:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.891+0000 7f40127fc640 1 --2- 172.21.15.100:0/3918525221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633cb20f2f0 0x5633cb5e61b0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f40040044f0 tx=0x7f4004004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f63bda39cf2960 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.891+0000 7f3ff2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:45:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.891+0000 7f3ff2ffd640 1 -- 172.21.15.100:0/3918525221 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4004003e90 con 0x5633cb20f2f0 2024-07-26T13:45:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.891+0000 7f3ff2ffd640 1 -- 172.21.15.100:0/3918525221 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7f4004005020 con 0x5633cb20f2f0 2024-07-26T13:45:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.891+0000 7f40207d6740 1 -- 172.21.15.100:0/3918525221 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ffc032e00 con 0x5633cb20f2f0 2024-07-26T13:45:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.891+0000 7f3ff2ffd640 1 -- 172.21.15.100:0/3918525221 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f4004005220 con 0x5633cb20f2f0 2024-07-26T13:45:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.891+0000 7f40207d6740 10 client.16836 fetch_fsmap learned FSMap version 45 2024-07-26T13:45:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.891+0000 7f40207d6740 1 -- 172.21.15.100:0/3918525221 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5633cb31c620 con 0x5633cb20f2f0 2024-07-26T13:45:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.891+0000 7f3ff2ffd640 1 -- 172.21.15.100:0/3918525221 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ffc032e00 con 0x5633cb20f2f0 2024-07-26T13:45:21.899 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.891+0000 7f3ff2ffd640 1 -- 172.21.15.100:0/3918525221 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f400401d030 con 0x5633cb20f2f0 2024-07-26T13:45:21.899 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.891+0000 7f40207d6740 10 client.16836 fetch_fsmap finished waiting for FSMap version 45 2024-07-26T13:45:21.899 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.891+0000 7f40207d6740 10 client.16836 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:45:21.899 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.891+0000 7f40207d6740 20 client.16836 populate_metadata read hostname 'smithi100' 2024-07-26T13:45:21.899 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.891+0000 7f40207d6740 1 --2- 172.21.15.100:0/3918525221 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5633cb1f6d80 0x5633cb5f8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:21.899 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.891+0000 7f40207d6740 4 client.16836 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:45:21.899 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.891+0000 7f40207d6740 1 -- 172.21.15.100:0/3918525221 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633cb552300 con 0x5633cb1f6d80 2024-07-26T13:45:21.899 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.892+0000 7f4012ffd640 1 --2- 172.21.15.100:0/3918525221 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5633cb1f6d80 0x5633cb5f8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:21.899 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.892+0000 7f3ff2ffd640 10 client.16836 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:21.899 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.892+0000 7f4012ffd640 1 --2- 172.21.15.100:0/3918525221 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5633cb1f6d80 0x5633cb5f8150 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6aa23e83d91d7cd server_cookie=60e8260ace95942 in_seq=0 out_seq=0 2024-07-26T13:45:21.899 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.893+0000 7f3ff2ffd640 1 -- 172.21.15.100:0/3918525221 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5633cb552300 con 0x5633cb1f6d80 2024-07-26T13:45:21.899 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.893+0000 7f3ff2ffd640 10 client.16836 handle_command_reply: tid=0 2024-07-26T13:45:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.893+0000 7f40207d6740 1 client.16836 shutdown 2024-07-26T13:45:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.893+0000 7f40207d6740 1 -- 172.21.15.100:0/3918525221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633cb20f2f0 msgr2=0x5633cb5e61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.893+0000 7f40207d6740 1 --2- 172.21.15.100:0/3918525221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633cb20f2f0 0x5633cb5e61b0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f40040044f0 tx=0x7f4004004730 comp rx=0 tx=0).stop 2024-07-26T13:45:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.893+0000 7f40207d6740 1 -- 172.21.15.100:0/3918525221 shutdown_connections 2024-07-26T13:45:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.893+0000 7f40207d6740 1 --2- 172.21.15.100:0/3918525221 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5633cb1f6d80 0x5633cb5f8150 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.893+0000 7f40207d6740 1 --2- 172.21.15.100:0/3918525221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633cb2381b0 0x5633cb5f20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.893+0000 7f3ff2ffd640 0 client.16836 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.893+0000 7f40207d6740 1 --2- 172.21.15.100:0/3918525221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633cb2174d0 0x5633cb5e66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.893+0000 7f40207d6740 1 --2- 172.21.15.100:0/3918525221 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633cb20f2f0 0x5633cb5e61b0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.893+0000 7f40207d6740 1 -- 172.21.15.100:0/3918525221 >> 172.21.15.100:0/3918525221 conn(0x5633cb22c300 msgr2=0x5633cb5db6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.893+0000 7f40207d6740 1 -- 172.21.15.100:0/3918525221 shutdown_connections 2024-07-26T13:45:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.893+0000 7f40207d6740 1 -- 172.21.15.100:0/3918525221 wait complete. 2024-07-26T13:45:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.893+0000 7f40207d6740 20 client.16836 trim_cache size 0 max 16384 2024-07-26T13:45:21.911 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.908+0000 7f40207d6740 1 Processor -- start 2024-07-26T13:45:21.911 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.908+0000 7f40207d6740 1 -- start start 2024-07-26T13:45:21.911 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.908+0000 7f40207d6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633cb20f2f0 0x5633cb5e7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:21.911 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.908+0000 7f40207d6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633cb2174d0 0x5633cb5f1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:21.911 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.908+0000 7f40207d6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633cb2381b0 0x5633cb5f23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:21.911 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.908+0000 7f40207d6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f40141b3d10 con 0x5633cb2174d0 2024-07-26T13:45:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.908+0000 7f40207d6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f40141b1580 con 0x5633cb20f2f0 2024-07-26T13:45:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.908+0000 7f40207d6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f40141af3b0 con 0x5633cb2381b0 2024-07-26T13:45:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.908+0000 7f4012ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633cb2174d0 0x5633cb5f1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.908+0000 7f4012ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633cb2174d0 0x5633cb5f1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54026/0 (socket says 172.21.15.100:54026) 2024-07-26T13:45:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.908+0000 7f4012ffd640 1 -- 172.21.15.100:0/1785439900 learned_addr learned my addr 172.21.15.100:0/1785439900 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.908+0000 7f40127fc640 1 --2- 172.21.15.100:0/1785439900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633cb20f2f0 0x5633cb5e7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.908+0000 7f40127fc640 1 --2- 172.21.15.100:0/1785439900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633cb2381b0 0x5633cb5f23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.908+0000 7f4012ffd640 1 -- 172.21.15.100:0/1785439900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633cb2381b0 msgr2=0x5633cb5f23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.909+0000 7f4012ffd640 1 --2- 172.21.15.100:0/1785439900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633cb2381b0 0x5633cb5f23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.909+0000 7f4012ffd640 1 -- 172.21.15.100:0/1785439900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633cb20f2f0 msgr2=0x5633cb5e7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.909+0000 7f4012ffd640 1 --2- 172.21.15.100:0/1785439900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633cb20f2f0 0x5633cb5e7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.909+0000 7f4012ffd640 1 -- 172.21.15.100:0/1785439900 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633cb5dd4c0 con 0x5633cb2174d0 2024-07-26T13:45:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.909+0000 7f40127fc640 1 --2- 172.21.15.100:0/1785439900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633cb2381b0 0x5633cb5f23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:45:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.909+0000 7f4012ffd640 1 --2- 172.21.15.100:0/1785439900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633cb2174d0 0x5633cb5f1eb0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc009900 tx=0x7f3ffc005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611d8dacb20be539 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.909+0000 7f40097fa640 1 -- 172.21.15.100:0/1785439900 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc035040 con 0x5633cb2174d0 2024-07-26T13:45:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.909+0000 7f40097fa640 1 -- 172.21.15.100:0/1785439900 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ffc005570 con 0x5633cb2174d0 2024-07-26T13:45:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.909+0000 7f40097fa640 1 -- 172.21.15.100:0/1785439900 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc031870 con 0x5633cb2174d0 2024-07-26T13:45:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.910+0000 7f40207d6740 1 -- 172.21.15.100:0/1785439900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633cb2174d0 msgr2=0x5633cb5f1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.910+0000 7f40207d6740 1 --2- 172.21.15.100:0/1785439900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633cb2174d0 0x5633cb5f1eb0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc009900 tx=0x7f3ffc005050 comp rx=0 tx=0).stop 2024-07-26T13:45:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.910+0000 7f40207d6740 1 -- 172.21.15.100:0/1785439900 shutdown_connections 2024-07-26T13:45:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.910+0000 7f40207d6740 1 --2- 172.21.15.100:0/1785439900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633cb2381b0 0x5633cb5f23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.910+0000 7f40207d6740 1 --2- 172.21.15.100:0/1785439900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633cb2174d0 0x5633cb5f1eb0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.910+0000 7f40207d6740 1 --2- 172.21.15.100:0/1785439900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633cb20f2f0 0x5633cb5e7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.910+0000 7f40207d6740 1 -- 172.21.15.100:0/1785439900 >> 172.21.15.100:0/1785439900 conn(0x5633cb22c300 msgr2=0x5633cb5df7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.910+0000 7f40207d6740 1 -- 172.21.15.100:0/1785439900 shutdown_connections 2024-07-26T13:45:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.910+0000 7f40207d6740 1 -- 172.21.15.100:0/1785439900 wait complete. 2024-07-26T13:45:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.910+0000 7f40207d6740 1 -- start start 2024-07-26T13:45:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.910+0000 7f40207d6740 1 Processor -- start 2024-07-26T13:45:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.911+0000 7f40207d6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633cb20f2f0 0x5633cb5ecf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.911+0000 7f40207d6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633cb2174d0 0x5633cb5ed450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.911+0000 7f40207d6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633cb2381b0 0x5633cb5ed990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.911+0000 7f40207d6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f40141afa40 con 0x5633cb2174d0 2024-07-26T13:45:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.911+0000 7f40207d6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f40141b0eb0 con 0x5633cb2381b0 2024-07-26T13:45:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.911+0000 7f40207d6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f40141b0110 con 0x5633cb20f2f0 2024-07-26T13:45:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.911+0000 7f40207d6740 10 client.0 _ll_register_callbacks cb 0x7f4004009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:45:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.911+0000 7f4012ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633cb2174d0 0x5633cb5ed450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.911+0000 7f4012ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633cb2174d0 0x5633cb5ed450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54038/0 (socket says 172.21.15.100:54038) 2024-07-26T13:45:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.911+0000 7f4012ffd640 1 -- 172.21.15.100:0/3409579897 learned_addr learned my addr 172.21.15.100:0/3409579897 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.911+0000 7f40127fc640 1 --2- 172.21.15.100:0/3409579897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633cb20f2f0 0x5633cb5ecf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.911+0000 7f40127fc640 1 --2- 172.21.15.100:0/3409579897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633cb2381b0 0x5633cb5ed990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.911+0000 7f4012ffd640 1 -- 172.21.15.100:0/3409579897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633cb20f2f0 msgr2=0x5633cb5ecf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.911+0000 7f4012ffd640 1 --2- 172.21.15.100:0/3409579897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633cb20f2f0 0x5633cb5ecf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.911+0000 7f4012ffd640 1 -- 172.21.15.100:0/3409579897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633cb2381b0 msgr2=0x5633cb5ed990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.911+0000 7f4012ffd640 1 --2- 172.21.15.100:0/3409579897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633cb2381b0 0x5633cb5ed990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.911+0000 7f4012ffd640 1 -- 172.21.15.100:0/3409579897 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633cb65e8f0 con 0x5633cb2174d0 2024-07-26T13:45:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.911+0000 7f4012ffd640 1 --2- 172.21.15.100:0/3409579897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633cb2174d0 0x5633cb5ed450 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc005930 tx=0x7f3ffc032de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3fc4477fdf7e816 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.911+0000 7f3ff2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:45:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.911+0000 7f3ff2ffd640 1 -- 172.21.15.100:0/3409579897 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc05c020 con 0x5633cb2174d0 2024-07-26T13:45:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.912+0000 7f3ff2ffd640 1 -- 172.21.15.100:0/3409579897 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7f3ffc004240 con 0x5633cb2174d0 2024-07-26T13:45:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.912+0000 7f40207d6740 1 -- 172.21.15.100:0/3409579897 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ffc005570 con 0x5633cb2174d0 2024-07-26T13:45:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.912+0000 7f3ff2ffd640 1 -- 172.21.15.100:0/3409579897 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f3ffc067050 con 0x5633cb2174d0 2024-07-26T13:45:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.912+0000 7f40207d6740 10 client.16842 fetch_fsmap learned FSMap version 45 2024-07-26T13:45:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.912+0000 7f40207d6740 1 -- 172.21.15.100:0/3409579897 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5633cb65fb80 con 0x5633cb2174d0 2024-07-26T13:45:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.912+0000 7f3ff2ffd640 1 -- 172.21.15.100:0/3409579897 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ffc005570 con 0x5633cb2174d0 2024-07-26T13:45:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.912+0000 7f3ff2ffd640 1 -- 172.21.15.100:0/3409579897 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7f3ffc04d070 con 0x5633cb2174d0 2024-07-26T13:45:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.912+0000 7f40207d6740 10 client.16842 fetch_fsmap finished waiting for FSMap version 45 2024-07-26T13:45:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.912+0000 7f40207d6740 10 client.16842 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:45:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.912+0000 7f40207d6740 20 client.16842 populate_metadata read hostname 'smithi100' 2024-07-26T13:45:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.912+0000 7f40207d6740 1 --2- 172.21.15.100:0/3409579897 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5633cb1cc1c0 0x5633cb661e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.912+0000 7f40207d6740 4 client.16842 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-26T13:45:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.912+0000 7f40207d6740 1 -- 172.21.15.100:0/3409579897 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5633cb54dda0 con 0x5633cb1cc1c0 2024-07-26T13:45:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.912+0000 7f40127fc640 1 --2- 172.21.15.100:0/3409579897 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5633cb1cc1c0 0x5633cb661e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.913+0000 7f3ff2ffd640 10 client.16842 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.913+0000 7f40127fc640 1 --2- 172.21.15.100:0/3409579897 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5633cb1cc1c0 0x5633cb661e70 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3661ebea5407ec5e server_cookie=39d042ec48731c77 in_seq=0 out_seq=0 2024-07-26T13:45:21.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:21 smithi138 ceph-mon[36954]: pgmap v523: 97 pgs: 97 active+clean; 72 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 628 KiB/s wr, 112 op/s 2024-07-26T13:45:21.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:21 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/252422687' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:45:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.950+0000 7f3ff2ffd640 1 -- 172.21.15.100:0/3409579897 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5633cb54dda0 con 0x5633cb1cc1c0 2024-07-26T13:45:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.950+0000 7f3ff2ffd640 10 client.16842 handle_command_reply: tid=0 2024-07-26T13:45:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.950+0000 7f40207d6740 1 client.16842 shutdown 2024-07-26T13:45:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.950+0000 7f40207d6740 1 -- 172.21.15.100:0/3409579897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633cb2174d0 msgr2=0x5633cb5ed450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.950+0000 7f40207d6740 1 --2- 172.21.15.100:0/3409579897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633cb2174d0 0x5633cb5ed450 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc005930 tx=0x7f3ffc032de0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.950+0000 7f40207d6740 1 -- 172.21.15.100:0/3409579897 shutdown_connections 2024-07-26T13:45:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.950+0000 7f40207d6740 1 --2- 172.21.15.100:0/3409579897 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5633cb1cc1c0 0x5633cb661e70 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.950+0000 7f40207d6740 1 --2- 172.21.15.100:0/3409579897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633cb2381b0 0x5633cb5ed990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.950+0000 7f40207d6740 1 --2- 172.21.15.100:0/3409579897 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5633cb2174d0 0x5633cb5ed450 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.950+0000 7f3ff2ffd640 0 client.16842 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.950+0000 7f40207d6740 1 --2- 172.21.15.100:0/3409579897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633cb20f2f0 0x5633cb5ecf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.950+0000 7f40207d6740 1 -- 172.21.15.100:0/3409579897 >> 172.21.15.100:0/3409579897 conn(0x5633cb22c300 msgr2=0x5633cb5e7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.950+0000 7f40207d6740 1 -- 172.21.15.100:0/3409579897 shutdown_connections 2024-07-26T13:45:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.950+0000 7f40207d6740 1 -- 172.21.15.100:0/3409579897 wait complete. 2024-07-26T13:45:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.950+0000 7f40207d6740 20 client.16842 trim_cache size 0 max 16384 2024-07-26T13:45:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-07-26T13:45:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-07-26T13:45:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-07-26T13:45:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-07-26T13:45:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.951+0000 7f40198c4640 1 -- 172.21.15.100:0/1937592135 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f40000b3dc0 msgr2=0x7f40000b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.951+0000 7f40198c4640 1 --2- 172.21.15.100:0/1937592135 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f40000b3dc0 0x7f40000b62d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f400c0aae30 tx=0x7f400c00cf70 comp rx=0 tx=0).stop 2024-07-26T13:45:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.951+0000 7f40198c4640 1 -- 172.21.15.100:0/1937592135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f401410faf0 msgr2=0x7f40141a1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.951+0000 7f40198c4640 1 --2- 172.21.15.100:0/1937592135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f401410faf0 0x7f40141a1890 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f400c00eb10 tx=0x7f400c00c2d0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.951+0000 7f40198c4640 1 -- 172.21.15.100:0/1937592135 shutdown_connections 2024-07-26T13:45:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.951+0000 7f40198c4640 1 --2- 172.21.15.100:0/1937592135 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f40000b3dc0 0x7f40000b62d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.951+0000 7f40198c4640 1 --2- 172.21.15.100:0/1937592135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f401410faf0 0x7f40141a1890 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.951+0000 7f40198c4640 1 --2- 172.21.15.100:0/1937592135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4014102890 0x7f401419c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.951+0000 7f40198c4640 1 --2- 172.21.15.100:0/1937592135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4014101f40 0x7f401419c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.951+0000 7f40198c4640 1 -- 172.21.15.100:0/1937592135 >> 172.21.15.100:0/1937592135 conn(0x7f401407a170 msgr2=0x7f4014111070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.951+0000 7f40198c4640 1 -- 172.21.15.100:0/1937592135 shutdown_connections 2024-07-26T13:45:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:21.951+0000 7f40198c4640 1 -- 172.21.15.100:0/1937592135 wait complete. 2024-07-26T13:45:22.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:21 smithi167 ceph-mon[37140]: pgmap v523: 97 pgs: 97 active+clean; 72 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 628 KiB/s wr, 112 op/s 2024-07-26T13:45:22.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:21 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/252422687' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:45:22.246 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24607 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-07-26T13:45:22.247 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- timeout 105 ceph tell mds.24607 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-07-26T13:45:22.417 INFO:tasks.workunit.client.0.smithi100.stdout: 1 189390 2.44 MB/sec execute 96 sec latency 34.969 ms 2024-07-26T13:45:22.512 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:45:22.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:22 smithi100 ceph-mon[32599]: pgmap v524: 97 pgs: 97 active+clean; 76 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 950 KiB/s wr, 149 op/s 2024-07-26T13:45:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:22 smithi138 ceph-mon[36954]: pgmap v524: 97 pgs: 97 active+clean; 76 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 950 KiB/s wr, 149 op/s 2024-07-26T13:45:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:22 smithi167 ceph-mon[37140]: pgmap v524: 97 pgs: 97 active+clean; 76 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 950 KiB/s wr, 149 op/s 2024-07-26T13:45:23.417 INFO:tasks.workunit.client.0.smithi100.stdout: 1 191681 2.55 MB/sec execute 97 sec latency 24.779 ms 2024-07-26T13:45:24.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.232+0000 7fb2f6a4a640 1 -- 172.21.15.100:0/1892463092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2f0109860 msgr2=0x7fb2f0110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:24.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.232+0000 7fb2f6a4a640 1 --2- 172.21.15.100:0/1892463092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2f0109860 0x7fb2f0110090 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8009ca0 tx=0x7fb2e802a8c0 comp rx=0 tx=0).stop 2024-07-26T13:45:24.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.232+0000 7fb2f6a4a640 1 -- 172.21.15.100:0/1892463092 shutdown_connections 2024-07-26T13:45:24.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.232+0000 7fb2f6a4a640 1 --2- 172.21.15.100:0/1892463092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2f0109860 0x7fb2f0110090 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:24.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.232+0000 7fb2f6a4a640 1 --2- 172.21.15.100:0/1892463092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2f0108f10 0x7fb2f0109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:24.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.232+0000 7fb2f6a4a640 1 --2- 172.21.15.100:0/1892463092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2f01035d0 0x7fb2f01039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:24.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.232+0000 7fb2f6a4a640 1 -- 172.21.15.100:0/1892463092 >> 172.21.15.100:0/1892463092 conn(0x7fb2f00ff020 msgr2=0x7fb2f0101470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:24.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.232+0000 7fb2f6a4a640 1 -- 172.21.15.100:0/1892463092 shutdown_connections 2024-07-26T13:45:24.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.232+0000 7fb2f6a4a640 1 -- 172.21.15.100:0/1892463092 wait complete. 2024-07-26T13:45:24.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.233+0000 7fb2f6a4a640 1 Processor -- start 2024-07-26T13:45:24.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.233+0000 7fb2f6a4a640 1 -- start start 2024-07-26T13:45:24.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.234+0000 7fb2f6a4a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2f01035d0 0x7fb2f0199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:24.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.234+0000 7fb2f6a4a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2f0108f10 0x7fb2f019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:24.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.234+0000 7fb2f6a4a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2f0109860 0x7fb2f019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:24.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.234+0000 7fb2f6a4a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f0113510 con 0x7fb2f0109860 2024-07-26T13:45:24.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.234+0000 7fb2f6a4a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f0113390 con 0x7fb2f01035d0 2024-07-26T13:45:24.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.234+0000 7fb2f6a4a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f0113690 con 0x7fb2f0108f10 2024-07-26T13:45:24.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.234+0000 7fb2f4fe7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2f0109860 0x7fb2f019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:24.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.234+0000 7fb2ef7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2f0108f10 0x7fb2f019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:24.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.234+0000 7fb2f4fe7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2f0109860 0x7fb2f019f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54048/0 (socket says 172.21.15.100:54048) 2024-07-26T13:45:24.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.234+0000 7fb2f4fe7640 1 -- 172.21.15.100:0/3949731116 learned_addr learned my addr 172.21.15.100:0/3949731116 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:24.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.234+0000 7fb2effff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2f01035d0 0x7fb2f0199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:24.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.234+0000 7fb2f4fe7640 1 -- 172.21.15.100:0/3949731116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2f0108f10 msgr2=0x7fb2f019a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:24.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.234+0000 7fb2f4fe7640 1 --2- 172.21.15.100:0/3949731116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2f0108f10 0x7fb2f019a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:24.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.234+0000 7fb2f4fe7640 1 -- 172.21.15.100:0/3949731116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2f01035d0 msgr2=0x7fb2f0199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:24.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.234+0000 7fb2f4fe7640 1 --2- 172.21.15.100:0/3949731116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2f01035d0 0x7fb2f0199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:24.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.234+0000 7fb2f4fe7640 1 -- 172.21.15.100:0/3949731116 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f019fdd0 con 0x7fb2f0109860 2024-07-26T13:45:24.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.234+0000 7fb2effff640 1 --2- 172.21.15.100:0/3949731116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2f01035d0 0x7fb2f0199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:45:24.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.234+0000 7fb2f4fe7640 1 --2- 172.21.15.100:0/3949731116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2f0109860 0x7fb2f019f6a0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8005cc0 tx=0x7fb2e8031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:24.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.234+0000 7fb2ed7fa640 1 -- 172.21.15.100:0/3949731116 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8003b70 con 0x7fb2f0109860 2024-07-26T13:45:24.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.235+0000 7fb2f6a4a640 1 -- 172.21.15.100:0/3949731116 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2f01a0060 con 0x7fb2f0109860 2024-07-26T13:45:24.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.235+0000 7fb2ed7fa640 1 -- 172.21.15.100:0/3949731116 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2e80318f0 con 0x7fb2f0109860 2024-07-26T13:45:24.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.235+0000 7fb2ed7fa640 1 -- 172.21.15.100:0/3949731116 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e802f950 con 0x7fb2f0109860 2024-07-26T13:45:24.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.235+0000 7fb2f6a4a640 1 -- 172.21.15.100:0/3949731116 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2f0075020 con 0x7fb2f0109860 2024-07-26T13:45:24.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.237+0000 7fb2ed7fa640 1 -- 172.21.15.100:0/3949731116 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fb2e802fb10 con 0x7fb2f0109860 2024-07-26T13:45:24.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.237+0000 7fb2fcfce740 1 Processor -- start 2024-07-26T13:45:24.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.238+0000 7fb2fcfce740 1 -- start start 2024-07-26T13:45:24.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.238+0000 7fb2fcfce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d5bc67840 0x563d5c00d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:24.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.238+0000 7fb2fcfce740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d5bc3e980 0x563d5c00dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:24.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.238+0000 7fb2fcfce740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d5bc46b60 0x563d5c00e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:24.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.238+0000 7fb2fcfce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x563d5bf82780 con 0x563d5bc67840 2024-07-26T13:45:24.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.238+0000 7fb2fcfce740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563d5bf71c80 con 0x563d5bc3e980 2024-07-26T13:45:24.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.238+0000 7fb2fcfce740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563d5bf75cc0 con 0x563d5bc46b60 2024-07-26T13:45:24.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.238+0000 7fb2ef7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d5bc67840 0x563d5c00d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:24.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.238+0000 7fb2effff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d5bc3e980 0x563d5c00dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:24.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.238+0000 7fb2effff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d5bc3e980 0x563d5c00dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:34546/0 (socket says 172.21.15.100:34546) 2024-07-26T13:45:24.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.238+0000 7fb2ef7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d5bc67840 0x563d5c00d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54052/0 (socket says 172.21.15.100:54052) 2024-07-26T13:45:24.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.238+0000 7fb2effff640 1 -- 172.21.15.100:0/50730314 learned_addr learned my addr 172.21.15.100:0/50730314 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:24.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.238+0000 7fb2ef7fe640 1 --2- 172.21.15.100:0/50730314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d5bc46b60 0x563d5c00e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:24.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.238+0000 7fb2effff640 1 -- 172.21.15.100:0/50730314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d5bc46b60 msgr2=0x563d5c00e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:24.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.238+0000 7fb2effff640 1 --2- 172.21.15.100:0/50730314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d5bc46b60 0x563d5c00e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:24.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.238+0000 7fb2effff640 1 -- 172.21.15.100:0/50730314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d5bc67840 msgr2=0x563d5c00d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:24.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.238+0000 7fb2effff640 1 --2- 172.21.15.100:0/50730314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d5bc67840 0x563d5c00d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:24.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.238+0000 7fb2effff640 1 -- 172.21.15.100:0/50730314 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d5bf07080 con 0x563d5bc3e980 2024-07-26T13:45:24.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.239+0000 7fb2ed7fa640 1 --2- 172.21.15.100:0/3949731116 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb2d80b3d40 0x7fb2d80b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:24.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.239+0000 7fb2ef7fe640 1 --2- 172.21.15.100:0/3949731116 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb2d80b3d40 0x7fb2d80b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:24.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.239+0000 7fb2effff640 1 --2- 172.21.15.100:0/50730314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d5bc3e980 0x563d5c00dd80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb2e00115f0 tx=0x7fb2e0011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b6cd67f5f39ff9c server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:24.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.239+0000 7fb2ed7fa640 1 -- 172.21.15.100:0/3949731116 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7fb2e81053e0 con 0x7fb2f0109860 2024-07-26T13:45:24.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.239+0000 7fb2d67fc640 1 -- 172.21.15.100:0/50730314 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e001e070 con 0x563d5bc3e980 2024-07-26T13:45:24.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.239+0000 7fb2d67fc640 1 -- 172.21.15.100:0/50730314 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2e000fa70 con 0x563d5bc3e980 2024-07-26T13:45:24.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.239+0000 7fb2d67fc640 1 -- 172.21.15.100:0/50730314 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0019a90 con 0x563d5bc3e980 2024-07-26T13:45:24.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.240+0000 7fb2fcfce740 1 -- 172.21.15.100:0/50730314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d5bc3e980 msgr2=0x563d5c00dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:24.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.240+0000 7fb2fcfce740 1 --2- 172.21.15.100:0/50730314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d5bc3e980 0x563d5c00dd80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb2e00115f0 tx=0x7fb2e0011ab0 comp rx=0 tx=0).stop 2024-07-26T13:45:24.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.240+0000 7fb2ef7fe640 1 --2- 172.21.15.100:0/3949731116 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb2d80b3d40 0x7fb2d80b6250 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4010fd0 tx=0x7fb2e4005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:24.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.240+0000 7fb2fcfce740 1 -- 172.21.15.100:0/50730314 shutdown_connections 2024-07-26T13:45:24.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.240+0000 7fb2fcfce740 1 --2- 172.21.15.100:0/50730314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d5bc46b60 0x563d5c00e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:24.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.240+0000 7fb2fcfce740 1 --2- 172.21.15.100:0/50730314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d5bc3e980 0x563d5c00dd80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:24.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.240+0000 7fb2fcfce740 1 --2- 172.21.15.100:0/50730314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d5bc67840 0x563d5c00d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:24.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.240+0000 7fb2fcfce740 1 -- 172.21.15.100:0/50730314 >> 172.21.15.100:0/50730314 conn(0x563d5bc5b990 msgr2=0x563d5c00b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:24.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.240+0000 7fb2fcfce740 1 -- 172.21.15.100:0/50730314 shutdown_connections 2024-07-26T13:45:24.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.240+0000 7fb2fcfce740 1 -- 172.21.15.100:0/50730314 wait complete. 2024-07-26T13:45:24.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.241+0000 7fb2fcfce740 1 -- start start 2024-07-26T13:45:24.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.241+0000 7fb2fcfce740 1 Processor -- start 2024-07-26T13:45:24.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.241+0000 7fb2fcfce740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d5bc3e980 0x563d5c018e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:24.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.241+0000 7fb2fcfce740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d5bc46b60 0x563d5c019380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:24.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.241+0000 7fb2fcfce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d5bc67840 0x563d5c0198c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:24.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.241+0000 7fb2fcfce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x563d5bdd57a0 con 0x563d5bc67840 2024-07-26T13:45:24.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.241+0000 7fb2fcfce740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563d5bf142f0 con 0x563d5bc46b60 2024-07-26T13:45:24.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.241+0000 7fb2fcfce740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563d5c01c020 con 0x563d5bc3e980 2024-07-26T13:45:24.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.241+0000 7fb2fcfce740 10 client.0 _ll_register_callbacks cb 0x563d5bf1e110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:45:24.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.241+0000 7fb2effff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d5bc3e980 0x563d5c018e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:24.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.241+0000 7fb2f4fe7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d5bc67840 0x563d5c0198c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:24.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.241+0000 7fb2effff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d5bc3e980 0x563d5c018e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:42058/0 (socket says 172.21.15.100:42058) 2024-07-26T13:45:24.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.241+0000 7fb2effff640 1 -- 172.21.15.100:0/1072496033 learned_addr learned my addr 172.21.15.100:0/1072496033 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:24.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.241+0000 7fb2f4fe7640 1 --2- 172.21.15.100:0/1072496033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d5bc67840 0x563d5c0198c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54056/0 (socket says 172.21.15.100:54056) 2024-07-26T13:45:24.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.242+0000 7fb2ef7fe640 1 --2- 172.21.15.100:0/1072496033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d5bc46b60 0x563d5c019380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:24.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.242+0000 7fb2effff640 1 -- 172.21.15.100:0/1072496033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d5bc46b60 msgr2=0x563d5c019380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:24.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.242+0000 7fb2effff640 1 --2- 172.21.15.100:0/1072496033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d5bc46b60 0x563d5c019380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:24.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.242+0000 7fb2effff640 1 -- 172.21.15.100:0/1072496033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d5bc67840 msgr2=0x563d5c0198c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:24.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.242+0000 7fb2effff640 1 --2- 172.21.15.100:0/1072496033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d5bc67840 0x563d5c0198c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:24.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.242+0000 7fb2effff640 1 -- 172.21.15.100:0/1072496033 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d5bf18480 con 0x563d5bc3e980 2024-07-26T13:45:24.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.242+0000 7fb2effff640 1 --2- 172.21.15.100:0/1072496033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d5bc3e980 0x563d5c018e40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0014040 tx=0x7fb2e0005750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a325d75f06289300 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:24.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.242+0000 7fb2cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-07-26T13:45:24.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.242+0000 7fb2cf7fe640 1 -- 172.21.15.100:0/1072496033 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e001e040 con 0x563d5bc3e980 2024-07-26T13:45:24.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.242+0000 7fb2fcfce740 1 -- 172.21.15.100:0/1072496033 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2e000fa70 con 0x563d5bc3e980 2024-07-26T13:45:24.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.242+0000 7fb2cf7fe640 1 -- 172.21.15.100:0/1072496033 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7fb2e000f700 con 0x563d5bc3e980 2024-07-26T13:45:24.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.242+0000 7fb2cf7fe640 1 -- 172.21.15.100:0/1072496033 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fb2e0030050 con 0x563d5bc3e980 2024-07-26T13:45:24.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.242+0000 7fb2fcfce740 10 client.26293 fetch_fsmap learned FSMap version 45 2024-07-26T13:45:24.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.242+0000 7fb2fcfce740 1 -- 172.21.15.100:0/1072496033 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x563d5bce1b90 con 0x563d5bc3e980 2024-07-26T13:45:24.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.243+0000 7fb2cf7fe640 1 -- 172.21.15.100:0/1072496033 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2e000fa70 con 0x563d5bc3e980 2024-07-26T13:45:24.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.243+0000 7fb2cf7fe640 1 -- 172.21.15.100:0/1072496033 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7fb2e0021070 con 0x563d5bc3e980 2024-07-26T13:45:24.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.243+0000 7fb2fcfce740 10 client.26293 fetch_fsmap finished waiting for FSMap version 45 2024-07-26T13:45:24.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.243+0000 7fb2fcfce740 10 client.26293 resolve_mds: validated gid 24607 aka daemon mds.i 2024-07-26T13:45:24.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.243+0000 7fb2fcfce740 20 client.26293 populate_metadata read hostname 'smithi100' 2024-07-26T13:45:24.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.243+0000 7fb2fcfce740 1 --2- 172.21.15.100:0/1072496033 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x563d5bc261c0 0x563d5c02ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:24.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.243+0000 7fb2fcfce740 4 client.26293 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:45:24.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.243+0000 7fb2fcfce740 1 -- 172.21.15.100:0/1072496033 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d5bf846d0 con 0x563d5bc261c0 2024-07-26T13:45:24.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.243+0000 7fb2ef7fe640 1 --2- 172.21.15.100:0/1072496033 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x563d5bc261c0 0x563d5c02ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:24.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.244+0000 7fb2cf7fe640 10 client.26293 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:24.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.244+0000 7fb2ef7fe640 1 --2- 172.21.15.100:0/1072496033 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x563d5bc261c0 0x563d5c02ae00 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dae1da1657e5ff03 server_cookie=c1f5f54f6da3ad47 in_seq=0 out_seq=0 2024-07-26T13:45:24.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.244+0000 7fb2cf7fe640 1 -- 172.21.15.100:0/1072496033 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563d5bf846d0 con 0x563d5bc261c0 2024-07-26T13:45:24.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.244+0000 7fb2cf7fe640 10 client.26293 handle_command_reply: tid=0 2024-07-26T13:45:24.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.244+0000 7fb2fcfce740 1 client.26293 shutdown 2024-07-26T13:45:24.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.244+0000 7fb2fcfce740 1 -- 172.21.15.100:0/1072496033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d5bc3e980 msgr2=0x563d5c018e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:24.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.244+0000 7fb2fcfce740 1 --2- 172.21.15.100:0/1072496033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d5bc3e980 0x563d5c018e40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0014040 tx=0x7fb2e0005750 comp rx=0 tx=0).stop 2024-07-26T13:45:24.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.245+0000 7fb2fcfce740 1 -- 172.21.15.100:0/1072496033 shutdown_connections 2024-07-26T13:45:24.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.245+0000 7fb2fcfce740 1 --2- 172.21.15.100:0/1072496033 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x563d5bc261c0 0x563d5c02ae00 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:24.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.245+0000 7fb2fcfce740 1 --2- 172.21.15.100:0/1072496033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d5bc67840 0x563d5c0198c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:24.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.245+0000 7fb2fcfce740 1 --2- 172.21.15.100:0/1072496033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d5bc46b60 0x563d5c019380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:24.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.245+0000 7fb2cf7fe640 0 client.26293 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:24.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.245+0000 7fb2fcfce740 1 --2- 172.21.15.100:0/1072496033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d5bc3e980 0x563d5c018e40 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:24.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.245+0000 7fb2fcfce740 1 -- 172.21.15.100:0/1072496033 >> 172.21.15.100:0/1072496033 conn(0x563d5bc5b990 msgr2=0x563d5c010a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:24.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.245+0000 7fb2fcfce740 1 -- 172.21.15.100:0/1072496033 shutdown_connections 2024-07-26T13:45:24.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.245+0000 7fb2fcfce740 1 -- 172.21.15.100:0/1072496033 wait complete. 2024-07-26T13:45:24.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.245+0000 7fb2fcfce740 20 client.26293 trim_cache size 0 max 16384 2024-07-26T13:45:24.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.264+0000 7fb2fcfce740 1 Processor -- start 2024-07-26T13:45:24.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.264+0000 7fb2fcfce740 1 -- start start 2024-07-26T13:45:24.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.264+0000 7fb2fcfce740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d5bc3e980 0x563d5c01a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:24.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.264+0000 7fb2fcfce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d5bc46b60 0x563d5c012920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:24.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.264+0000 7fb2fcfce740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d5bc67840 0x563d5c012e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:24.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.264+0000 7fb2fcfce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f01af9b0 con 0x563d5bc46b60 2024-07-26T13:45:24.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.264+0000 7fb2fcfce740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f01ad220 con 0x563d5bc3e980 2024-07-26T13:45:24.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.264+0000 7fb2fcfce740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f01ab050 con 0x563d5bc67840 2024-07-26T13:45:24.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.264+0000 7fb2effff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d5bc3e980 0x563d5c01a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:24.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.264+0000 7fb2ef7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d5bc46b60 0x563d5c012920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:24.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.264+0000 7fb2effff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d5bc3e980 0x563d5c01a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:34568/0 (socket says 172.21.15.100:34568) 2024-07-26T13:45:24.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.264+0000 7fb2effff640 1 -- 172.21.15.100:0/2269137201 learned_addr learned my addr 172.21.15.100:0/2269137201 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:24.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.264+0000 7fb2ef7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d5bc46b60 0x563d5c012920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54066/0 (socket says 172.21.15.100:54066) 2024-07-26T13:45:24.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.264+0000 7fb2f4fe7640 1 --2- 172.21.15.100:0/2269137201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d5bc67840 0x563d5c012e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:24.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.265+0000 7fb2effff640 1 -- 172.21.15.100:0/2269137201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d5bc67840 msgr2=0x563d5c012e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:24.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.265+0000 7fb2effff640 1 --2- 172.21.15.100:0/2269137201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d5bc67840 0x563d5c012e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:24.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.265+0000 7fb2effff640 1 -- 172.21.15.100:0/2269137201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d5bc46b60 msgr2=0x563d5c012920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:24.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.265+0000 7fb2effff640 1 --2- 172.21.15.100:0/2269137201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d5bc46b60 0x563d5c012920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:24.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.265+0000 7fb2effff640 1 -- 172.21.15.100:0/2269137201 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d5c00fb30 con 0x563d5bc3e980 2024-07-26T13:45:24.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.265+0000 7fb2ef7fe640 1 --2- 172.21.15.100:0/2269137201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d5bc46b60 0x563d5c012920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:45:24.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.265+0000 7fb2effff640 1 --2- 172.21.15.100:0/2269137201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d5bc3e980 0x563d5c01a230 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0011af0 tx=0x7fb2e00096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a990fd0a35a484 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:24.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.265+0000 7fb2d5ffb640 1 -- 172.21.15.100:0/2269137201 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e001e040 con 0x563d5bc3e980 2024-07-26T13:45:24.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.265+0000 7fb2d5ffb640 1 -- 172.21.15.100:0/2269137201 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2e0027070 con 0x563d5bc3e980 2024-07-26T13:45:24.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.265+0000 7fb2d5ffb640 1 -- 172.21.15.100:0/2269137201 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0005310 con 0x563d5bc3e980 2024-07-26T13:45:24.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.266+0000 7fb2fcfce740 1 -- 172.21.15.100:0/2269137201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d5bc3e980 msgr2=0x563d5c01a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:24.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.266+0000 7fb2fcfce740 1 --2- 172.21.15.100:0/2269137201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d5bc3e980 0x563d5c01a230 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0011af0 tx=0x7fb2e00096c0 comp rx=0 tx=0).stop 2024-07-26T13:45:24.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.266+0000 7fb2fcfce740 1 -- 172.21.15.100:0/2269137201 shutdown_connections 2024-07-26T13:45:24.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.266+0000 7fb2fcfce740 1 --2- 172.21.15.100:0/2269137201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d5bc67840 0x563d5c012e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:24.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.266+0000 7fb2fcfce740 1 --2- 172.21.15.100:0/2269137201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d5bc46b60 0x563d5c012920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:24.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.266+0000 7fb2fcfce740 1 --2- 172.21.15.100:0/2269137201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d5bc3e980 0x563d5c01a230 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:24.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.266+0000 7fb2fcfce740 1 -- 172.21.15.100:0/2269137201 >> 172.21.15.100:0/2269137201 conn(0x563d5bc5b990 msgr2=0x563d5c0195b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:24.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.266+0000 7fb2fcfce740 1 -- 172.21.15.100:0/2269137201 shutdown_connections 2024-07-26T13:45:24.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.266+0000 7fb2fcfce740 1 -- 172.21.15.100:0/2269137201 wait complete. 2024-07-26T13:45:24.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.267+0000 7fb2fcfce740 1 -- start start 2024-07-26T13:45:24.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.267+0000 7fb2fcfce740 1 Processor -- start 2024-07-26T13:45:24.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.267+0000 7fb2fcfce740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d5bc3e980 0x563d5c01d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:24.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.267+0000 7fb2fcfce740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d5bc46b60 0x563d5c01d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:24.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.267+0000 7fb2fcfce740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d5bc67840 0x563d5c092510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:24.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.267+0000 7fb2fcfce740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f01ab6e0 con 0x563d5bc46b60 2024-07-26T13:45:24.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.267+0000 7fb2fcfce740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f01acb50 con 0x563d5bc67840 2024-07-26T13:45:24.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.267+0000 7fb2fcfce740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f01abdb0 con 0x563d5bc3e980 2024-07-26T13:45:24.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.267+0000 7fb2fcfce740 10 client.0 _ll_register_callbacks cb 0x7fb2e4010f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:45:24.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.267+0000 7fb2ef7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d5bc46b60 0x563d5c01d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:24.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.267+0000 7fb2effff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d5bc3e980 0x563d5c01d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:24.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.267+0000 7fb2effff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d5bc3e980 0x563d5c01d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:42072/0 (socket says 172.21.15.100:42072) 2024-07-26T13:45:24.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.267+0000 7fb2ef7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d5bc46b60 0x563d5c01d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54072/0 (socket says 172.21.15.100:54072) 2024-07-26T13:45:24.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.267+0000 7fb2effff640 1 -- 172.21.15.100:0/2765671959 learned_addr learned my addr 172.21.15.100:0/2765671959 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:24.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.267+0000 7fb2f4fe7640 1 --2- 172.21.15.100:0/2765671959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d5bc67840 0x563d5c092510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:24.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.268+0000 7fb2ef7fe640 1 -- 172.21.15.100:0/2765671959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d5bc3e980 msgr2=0x563d5c01d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:24.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.268+0000 7fb2ef7fe640 1 --2- 172.21.15.100:0/2765671959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d5bc3e980 0x563d5c01d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:24.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.268+0000 7fb2ef7fe640 1 -- 172.21.15.100:0/2765671959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d5bc67840 msgr2=0x563d5c092510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:24.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.268+0000 7fb2ef7fe640 1 --2- 172.21.15.100:0/2765671959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d5bc67840 0x563d5c092510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:24.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.268+0000 7fb2ef7fe640 1 -- 172.21.15.100:0/2765671959 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d5c01e270 con 0x563d5bc46b60 2024-07-26T13:45:24.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.268+0000 7fb2effff640 1 --2- 172.21.15.100:0/2765671959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d5bc3e980 0x563d5c01d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:45:24.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.268+0000 7fb2ef7fe640 1 --2- 172.21.15.100:0/2765671959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d5bc46b60 0x563d5c01d9d0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x563d5c02b480 tx=0x7fb2e4012040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f139d2f0520c41 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:24.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.268+0000 7fb2cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:45:24.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.268+0000 7fb2cf7fe640 1 -- 172.21.15.100:0/2765671959 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4030ae0 con 0x563d5bc46b60 2024-07-26T13:45:24.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.268+0000 7fb2cf7fe640 1 -- 172.21.15.100:0/2765671959 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7fb2e4019040 con 0x563d5bc46b60 2024-07-26T13:45:24.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.268+0000 7fb2fcfce740 1 -- 172.21.15.100:0/2765671959 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2e0027070 con 0x563d5bc46b60 2024-07-26T13:45:24.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.268+0000 7fb2cf7fe640 1 -- 172.21.15.100:0/2765671959 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fb2e4030c80 con 0x563d5bc46b60 2024-07-26T13:45:24.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.268+0000 7fb2fcfce740 10 client.16860 fetch_fsmap learned FSMap version 45 2024-07-26T13:45:24.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.268+0000 7fb2fcfce740 1 -- 172.21.15.100:0/2765671959 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x563d5c093ae0 con 0x563d5bc46b60 2024-07-26T13:45:24.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.268+0000 7fb2cf7fe640 1 -- 172.21.15.100:0/2765671959 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2e0027070 con 0x563d5bc46b60 2024-07-26T13:45:24.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.269+0000 7fb2cf7fe640 1 -- 172.21.15.100:0/2765671959 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 45) ==== 8354+0+0 (secure 0 0 0) 0x7fb2e401f040 con 0x563d5bc46b60 2024-07-26T13:45:24.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.269+0000 7fb2fcfce740 10 client.16860 fetch_fsmap finished waiting for FSMap version 45 2024-07-26T13:45:24.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.269+0000 7fb2fcfce740 10 client.16860 resolve_mds: validated gid 24607 aka daemon mds.i 2024-07-26T13:45:24.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.269+0000 7fb2fcfce740 20 client.16860 populate_metadata read hostname 'smithi100' 2024-07-26T13:45:24.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.269+0000 7fb2fcfce740 1 --2- 172.21.15.100:0/2765671959 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x563d5bbfb570 0x563d5c0962f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:24.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.269+0000 7fb2fcfce740 4 client.16860 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-07-26T13:45:24.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.269+0000 7fb2fcfce740 1 -- 172.21.15.100:0/2765671959 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x563d5bf80170 con 0x563d5bbfb570 2024-07-26T13:45:24.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.269+0000 7fb2effff640 1 --2- 172.21.15.100:0/2765671959 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x563d5bbfb570 0x563d5c0962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:24.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.269+0000 7fb2cf7fe640 10 client.16860 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:24.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.269+0000 7fb2effff640 1 --2- 172.21.15.100:0/2765671959 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x563d5bbfb570 0x563d5c0962f0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77e8f5fe5f3a1a4f server_cookie=fdafa13034443fe0 in_seq=0 out_seq=0 2024-07-26T13:45:24.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.339+0000 7fb2cf7fe640 1 -- 172.21.15.100:0/2765671959 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563d5bf80170 con 0x563d5bbfb570 2024-07-26T13:45:24.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.339+0000 7fb2cf7fe640 10 client.16860 handle_command_reply: tid=0 2024-07-26T13:45:24.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.339+0000 7fb2fcfce740 1 client.16860 shutdown 2024-07-26T13:45:24.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.339+0000 7fb2fcfce740 1 -- 172.21.15.100:0/2765671959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d5bc46b60 msgr2=0x563d5c01d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:24.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.339+0000 7fb2fcfce740 1 --2- 172.21.15.100:0/2765671959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d5bc46b60 0x563d5c01d9d0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x563d5c02b480 tx=0x7fb2e4012040 comp rx=0 tx=0).stop 2024-07-26T13:45:24.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.340+0000 7fb2fcfce740 1 -- 172.21.15.100:0/2765671959 shutdown_connections 2024-07-26T13:45:24.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.340+0000 7fb2fcfce740 1 --2- 172.21.15.100:0/2765671959 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x563d5bbfb570 0x563d5c0962f0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:24.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.340+0000 7fb2fcfce740 1 --2- 172.21.15.100:0/2765671959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d5bc67840 0x563d5c092510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:24.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.340+0000 7fb2fcfce740 1 --2- 172.21.15.100:0/2765671959 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563d5bc46b60 0x563d5c01d9d0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:24.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.340+0000 7fb2cf7fe640 0 client.16860 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:24.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.340+0000 7fb2fcfce740 1 --2- 172.21.15.100:0/2765671959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563d5bc3e980 0x563d5c01d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:24.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.340+0000 7fb2fcfce740 1 -- 172.21.15.100:0/2765671959 >> 172.21.15.100:0/2765671959 conn(0x563d5bc5b990 msgr2=0x563d5c0195b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:24.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.340+0000 7fb2fcfce740 1 -- 172.21.15.100:0/2765671959 shutdown_connections 2024-07-26T13:45:24.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.340+0000 7fb2fcfce740 1 -- 172.21.15.100:0/2765671959 wait complete. 2024-07-26T13:45:24.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.340+0000 7fb2fcfce740 20 client.16860 trim_cache size 0 max 16384 2024-07-26T13:45:24.343 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:45:24.343 INFO:teuthology.orchestra.run.smithi100.stdout: "epoch": 45, 2024-07-26T13:45:24.343 INFO:teuthology.orchestra.run.smithi100.stdout: "leader": 24607, 2024-07-26T13:45:24.343 INFO:teuthology.orchestra.run.smithi100.stdout: "set_version": 14, 2024-07-26T13:45:24.343 INFO:teuthology.orchestra.run.smithi100.stdout: "sets": { 2024-07-26T13:45:24.343 INFO:teuthology.orchestra.run.smithi100.stdout: "b3b1b2d8": { 2024-07-26T13:45:24.343 INFO:teuthology.orchestra.run.smithi100.stdout: "version": 14, 2024-07-26T13:45:24.343 INFO:teuthology.orchestra.run.smithi100.stdout: "age_ref": 0.0, 2024-07-26T13:45:24.343 INFO:teuthology.orchestra.run.smithi100.stdout: "state": { 2024-07-26T13:45:24.343 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "QUIESCED", 2024-07-26T13:45:24.344 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 0.0 2024-07-26T13:45:24.344 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-07-26T13:45:24.344 INFO:teuthology.orchestra.run.smithi100.stdout: "timeout": 90.0, 2024-07-26T13:45:24.344 INFO:teuthology.orchestra.run.smithi100.stdout: "expiration": 130.0, 2024-07-26T13:45:24.344 INFO:teuthology.orchestra.run.smithi100.stdout: "members": { 2024-07-26T13:45:24.344 INFO:teuthology.orchestra.run.smithi100.stdout: "file:/": { 2024-07-26T13:45:24.344 INFO:teuthology.orchestra.run.smithi100.stdout: "excluded": false, 2024-07-26T13:45:24.344 INFO:teuthology.orchestra.run.smithi100.stdout: "state": { 2024-07-26T13:45:24.344 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "QUIESCED", 2024-07-26T13:45:24.344 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 0.0 2024-07-26T13:45:24.344 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:45:24.344 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:45:24.344 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:45:24.344 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:45:24.345 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:45:24.345 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:45:24.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.340+0000 7fb2f6a4a640 1 -- 172.21.15.100:0/3949731116 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb2d80b3d40 msgr2=0x7fb2d80b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:24.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.340+0000 7fb2f6a4a640 1 --2- 172.21.15.100:0/3949731116 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb2d80b3d40 0x7fb2d80b6250 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4010fd0 tx=0x7fb2e4005f90 comp rx=0 tx=0).stop 2024-07-26T13:45:24.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.340+0000 7fb2f6a4a640 1 -- 172.21.15.100:0/3949731116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2f0109860 msgr2=0x7fb2f019f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:24.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.340+0000 7fb2f6a4a640 1 --2- 172.21.15.100:0/3949731116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2f0109860 0x7fb2f019f6a0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8005cc0 tx=0x7fb2e8031700 comp rx=0 tx=0).stop 2024-07-26T13:45:24.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.341+0000 7fb2f6a4a640 1 -- 172.21.15.100:0/3949731116 shutdown_connections 2024-07-26T13:45:24.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.341+0000 7fb2f6a4a640 1 --2- 172.21.15.100:0/3949731116 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb2d80b3d40 0x7fb2d80b6250 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:24.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.341+0000 7fb2f6a4a640 1 --2- 172.21.15.100:0/3949731116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2f0109860 0x7fb2f019f6a0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:24.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.341+0000 7fb2f6a4a640 1 --2- 172.21.15.100:0/3949731116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2f0108f10 0x7fb2f019a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:24.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.341+0000 7fb2f6a4a640 1 --2- 172.21.15.100:0/3949731116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2f01035d0 0x7fb2f0199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:24.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.341+0000 7fb2f6a4a640 1 -- 172.21.15.100:0/3949731116 >> 172.21.15.100:0/3949731116 conn(0x7fb2f00ff020 msgr2=0x7fb2f0107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:24.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.341+0000 7fb2f6a4a640 1 -- 172.21.15.100:0/3949731116 shutdown_connections 2024-07-26T13:45:24.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:24.341+0000 7fb2f6a4a640 1 -- 172.21.15.100:0/3949731116 wait complete. 2024-07-26T13:45:24.417 INFO:tasks.workunit.client.0.smithi100.stdout: 1 192995 2.59 MB/sec execute 98 sec latency 78.266 ms 2024-07-26T13:45:25.031 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b3b1b2d8', quiesce took 5.8 seconds. Will release after: 4.2 2024-07-26T13:45:25.417 INFO:tasks.workunit.client.0.smithi100.stdout: 1 192995 2.56 MB/sec execute 99 sec latency 1078.293 ms 2024-07-26T13:45:25.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:25 smithi138 ceph-mon[36954]: pgmap v525: 97 pgs: 97 active+clean; 75 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.2 MiB/s wr, 191 op/s 2024-07-26T13:45:25.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:25 smithi138 ceph-mon[36954]: scrub summary: idle 2024-07-26T13:45:25.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:25 smithi138 ceph-mon[36954]: scrub summary: idle 2024-07-26T13:45:25.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:25 smithi100 ceph-mon[32599]: pgmap v525: 97 pgs: 97 active+clean; 75 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.2 MiB/s wr, 191 op/s 2024-07-26T13:45:25.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:25 smithi100 ceph-mon[32599]: scrub summary: idle 2024-07-26T13:45:25.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:25 smithi100 ceph-mon[32599]: scrub summary: idle 2024-07-26T13:45:26.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:25 smithi167 ceph-mon[37140]: pgmap v525: 97 pgs: 97 active+clean; 75 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.2 MiB/s wr, 191 op/s 2024-07-26T13:45:26.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:25 smithi167 ceph-mon[37140]: scrub summary: idle 2024-07-26T13:45:26.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:25 smithi167 ceph-mon[37140]: scrub summary: idle 2024-07-26T13:45:26.417 INFO:tasks.workunit.client.0.smithi100.stdout: 1 192995 2.54 MB/sec execute 100 sec latency 2078.370 ms 2024-07-26T13:45:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:26 smithi138 ceph-mon[36954]: scrub summary: idle 2024-07-26T13:45:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:26 smithi138 ceph-mon[36954]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:45:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:26 smithi100 ceph-mon[32599]: scrub summary: idle 2024-07-26T13:45:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:26 smithi100 ceph-mon[32599]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:45:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:26 smithi167 ceph-mon[37140]: scrub summary: idle 2024-07-26T13:45:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:26 smithi167 ceph-mon[37140]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:45:27.417 INFO:tasks.workunit.client.0.smithi100.stdout: 1 192995 2.51 MB/sec execute 101 sec latency 3078.461 ms 2024-07-26T13:45:27.841 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:27 smithi138 ceph-mon[36954]: pgmap v526: 97 pgs: 97 active+clean; 76 MiB data, 920 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 193 op/s 2024-07-26T13:45:27.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:27 smithi100 ceph-mon[32599]: pgmap v526: 97 pgs: 97 active+clean; 76 MiB data, 920 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 193 op/s 2024-07-26T13:45:28.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:27 smithi167 ceph-mon[37140]: pgmap v526: 97 pgs: 97 active+clean; 76 MiB data, 920 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 193 op/s 2024-07-26T13:45:28.417 INFO:tasks.workunit.client.0.smithi100.stdout: 1 192995 2.49 MB/sec execute 102 sec latency 4078.593 ms 2024-07-26T13:45:29.232 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:45:29.417 INFO:tasks.workunit.client.0.smithi100.stdout: 1 192995 2.46 MB/sec execute 103 sec latency 5078.636 ms 2024-07-26T13:45:29.501 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:45:29.857 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:29 smithi100 ceph-mon[32599]: pgmap v527: 97 pgs: 97 active+clean; 78 MiB data, 920 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 223 op/s 2024-07-26T13:45:29.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:29 smithi138 ceph-mon[36954]: pgmap v527: 97 pgs: 97 active+clean; 78 MiB data, 920 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 223 op/s 2024-07-26T13:45:30.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:29 smithi167 ceph-mon[37140]: pgmap v527: 97 pgs: 97 active+clean; 78 MiB data, 920 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 223 op/s 2024-07-26T13:45:30.417 INFO:tasks.workunit.client.0.smithi100.stdout: 1 192995 2.44 MB/sec execute 104 sec latency 6078.702 ms 2024-07-26T13:45:31.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.303+0000 7eff47241640 1 -- 172.21.15.100:0/1798849338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff40101ac0 msgr2=0x7eff40101ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:31.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.303+0000 7eff47241640 1 --2- 172.21.15.100:0/1798849338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff40101ac0 0x7eff40101ed0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7eff30009900 tx=0x7eff3002a410 comp rx=0 tx=0).stop 2024-07-26T13:45:31.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.303+0000 7eff47241640 1 -- 172.21.15.100:0/1798849338 shutdown_connections 2024-07-26T13:45:31.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.303+0000 7eff47241640 1 --2- 172.21.15.100:0/1798849338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff40106600 0x7eff40110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:31.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.303+0000 7eff47241640 1 --2- 172.21.15.100:0/1798849338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff40105cb0 0x7eff401060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:31.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.303+0000 7eff47241640 1 --2- 172.21.15.100:0/1798849338 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff40101ac0 0x7eff40101ed0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:31.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.303+0000 7eff47241640 1 -- 172.21.15.100:0/1798849338 >> 172.21.15.100:0/1798849338 conn(0x7eff40077690 msgr2=0x7eff40077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:31.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.303+0000 7eff47241640 1 -- 172.21.15.100:0/1798849338 shutdown_connections 2024-07-26T13:45:31.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.303+0000 7eff47241640 1 -- 172.21.15.100:0/1798849338 wait complete. 2024-07-26T13:45:31.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.304+0000 7eff47241640 1 Processor -- start 2024-07-26T13:45:31.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.304+0000 7eff47241640 1 -- start start 2024-07-26T13:45:31.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.304+0000 7eff47241640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff40101ac0 0x7eff40199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:31.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.304+0000 7eff47241640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff40105cb0 0x7eff4019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:31.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.304+0000 7eff47241640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff40106600 0x7eff4019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:31.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.304+0000 7eff47241640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7eff40113500 con 0x7eff40101ac0 2024-07-26T13:45:31.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.304+0000 7eff47241640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7eff40113380 con 0x7eff40105cb0 2024-07-26T13:45:31.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.304+0000 7eff47241640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7eff40113680 con 0x7eff40106600 2024-07-26T13:45:31.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.304+0000 7eff44fdd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff40101ac0 0x7eff40199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:31.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.304+0000 7eff44fdd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff40101ac0 0x7eff40199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34324/0 (socket says 172.21.15.100:34324) 2024-07-26T13:45:31.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.304+0000 7eff44fdd640 1 -- 172.21.15.100:0/185070779 learned_addr learned my addr 172.21.15.100:0/185070779 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:31.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.305+0000 7eff457de640 1 --2- 172.21.15.100:0/185070779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff40106600 0x7eff4019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:31.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.305+0000 7eff3ffff640 1 --2- 172.21.15.100:0/185070779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff40105cb0 0x7eff4019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:31.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.305+0000 7eff44fdd640 1 -- 172.21.15.100:0/185070779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff40106600 msgr2=0x7eff4019f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:31.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.305+0000 7eff44fdd640 1 --2- 172.21.15.100:0/185070779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff40106600 0x7eff4019f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:31.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.305+0000 7eff44fdd640 1 -- 172.21.15.100:0/185070779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff40105cb0 msgr2=0x7eff4019a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:31.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.305+0000 7eff44fdd640 1 --2- 172.21.15.100:0/185070779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff40105cb0 0x7eff4019a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:31.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.305+0000 7eff44fdd640 1 -- 172.21.15.100:0/185070779 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff4019ff40 con 0x7eff40101ac0 2024-07-26T13:45:31.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.305+0000 7eff44fdd640 1 --2- 172.21.15.100:0/185070779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff40101ac0 0x7eff40199fd0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7eff30005940 tx=0x7eff300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:31.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.305+0000 7eff3dffb640 1 -- 172.21.15.100:0/185070779 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3003f030 con 0x7eff40101ac0 2024-07-26T13:45:31.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.305+0000 7eff3dffb640 1 -- 172.21.15.100:0/185070779 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff30003cb0 con 0x7eff40101ac0 2024-07-26T13:45:31.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.305+0000 7eff47241640 1 -- 172.21.15.100:0/185070779 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff401a01d0 con 0x7eff40101ac0 2024-07-26T13:45:31.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.305+0000 7eff3dffb640 1 -- 172.21.15.100:0/185070779 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff30035070 con 0x7eff40101ac0 2024-07-26T13:45:31.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.305+0000 7eff47241640 1 -- 172.21.15.100:0/185070779 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff401a6990 con 0x7eff40101ac0 2024-07-26T13:45:31.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.306+0000 7eff47241640 1 -- 172.21.15.100:0/185070779 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff40101ed0 con 0x7eff40101ac0 2024-07-26T13:45:31.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.308+0000 7eff3dffb640 1 -- 172.21.15.100:0/185070779 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7eff30042030 con 0x7eff40101ac0 2024-07-26T13:45:31.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.309+0000 7eff3dffb640 1 --2- 172.21.15.100:0/185070779 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7eff280b3d40 0x7eff280b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:31.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.309+0000 7eff3ffff640 1 --2- 172.21.15.100:0/185070779 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7eff280b3d40 0x7eff280b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:31.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.309+0000 7eff3dffb640 1 -- 172.21.15.100:0/185070779 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7eff30104a40 con 0x7eff40101ac0 2024-07-26T13:45:31.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.309+0000 7eff3ffff640 1 --2- 172.21.15.100:0/185070779 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7eff280b3d40 0x7eff280b6250 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7eff34005fd0 tx=0x7eff34005850 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:31.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.312+0000 7eff3dffb640 1 -- 172.21.15.100:0/185070779 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7eff300cbcd0 con 0x7eff40101ac0 2024-07-26T13:45:31.417 INFO:tasks.workunit.client.0.smithi100.stdout: 1 192995 2.42 MB/sec execute 105 sec latency 7078.754 ms 2024-07-26T13:45:31.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.523+0000 7eff47241640 1 -- 172.21.15.100:0/185070779 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff4019b100 con 0x7eff40101ac0 2024-07-26T13:45:31.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.523+0000 7eff3dffb640 1 -- 172.21.15.100:0/185070779 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11928 (secure 0 0 0) 0x7eff300cb5d0 con 0x7eff40101ac0 2024-07-26T13:45:31.525 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:45:31.525 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":46,"btime":"2024-07-26T13:45:25:551591+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14790,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/313501409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":313501409},{"type":"v1","addr":"172.21.15.100:6839","nonce":313501409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/1402975506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":1402975506},{"type":"v1","addr":"172.21.15.100:6841","nonce":1402975506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-26T13:37:01.281843+0000","modified":"2024-07-26T13:45:24.716007+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24607,"mds_1":24605,"mds_2":25762},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24575":{"gid":24575,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/3393225483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3393225483},{"type":"v1","addr":"172.21.15.138:6837","nonce":3393225483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.138:6839/1580925471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1580925471},{"type":"v1","addr":"172.21.15.138:6839","nonce":1580925471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/398150514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":398150514},{"type":"v1","addr":"172.21.15.167:6839","nonce":398150514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25762":{"gid":25762,"name":"c","rank":2,"incarnation":39,"state":"up:active","state_seq":32,"addr":"172.21.15.167:6837/2476068386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2476068386},{"type":"v1","addr":"172.21.15.167:6837","nonce":2476068386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25997":{"gid":25997,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/2831339503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2831339503},{"type":"v1","addr":"172.21.15.138:6835","nonce":2831339503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24607,"qdb_cluster":[24605,25762,24607]},"id":1}]} 2024-07-26T13:45:31.525 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 46 2024-07-26T13:45:31.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.526+0000 7eff47241640 1 -- 172.21.15.100:0/185070779 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7eff280b3d40 msgr2=0x7eff280b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:31.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.526+0000 7eff47241640 1 --2- 172.21.15.100:0/185070779 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7eff280b3d40 0x7eff280b6250 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7eff34005fd0 tx=0x7eff34005850 comp rx=0 tx=0).stop 2024-07-26T13:45:31.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.526+0000 7eff47241640 1 -- 172.21.15.100:0/185070779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff40101ac0 msgr2=0x7eff40199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:31.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.526+0000 7eff47241640 1 --2- 172.21.15.100:0/185070779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff40101ac0 0x7eff40199fd0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7eff30005940 tx=0x7eff300042f0 comp rx=0 tx=0).stop 2024-07-26T13:45:31.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.526+0000 7eff47241640 1 -- 172.21.15.100:0/185070779 shutdown_connections 2024-07-26T13:45:31.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.526+0000 7eff47241640 1 --2- 172.21.15.100:0/185070779 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7eff280b3d40 0x7eff280b6250 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:31.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.526+0000 7eff47241640 1 --2- 172.21.15.100:0/185070779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff40106600 0x7eff4019f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:31.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.526+0000 7eff47241640 1 --2- 172.21.15.100:0/185070779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff40105cb0 0x7eff4019a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:31.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.526+0000 7eff47241640 1 --2- 172.21.15.100:0/185070779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7eff40101ac0 0x7eff40199fd0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:31.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.526+0000 7eff47241640 1 -- 172.21.15.100:0/185070779 >> 172.21.15.100:0/185070779 conn(0x7eff40077690 msgr2=0x7eff40106f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:31.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.526+0000 7eff47241640 1 -- 172.21.15.100:0/185070779 shutdown_connections 2024-07-26T13:45:31.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:31.526+0000 7eff47241640 1 -- 172.21.15.100:0/185070779 wait complete. 2024-07-26T13:45:31.879 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:31 smithi100 ceph-mon[32599]: pgmap v528: 97 pgs: 97 active+clean; 78 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 189 op/s 2024-07-26T13:45:31.879 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:31 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/185070779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:45:31.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:31 smithi138 ceph-mon[36954]: pgmap v528: 97 pgs: 97 active+clean; 78 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 189 op/s 2024-07-26T13:45:31.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:31 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/185070779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:45:32.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:31 smithi167 ceph-mon[37140]: pgmap v528: 97 pgs: 97 active+clean; 78 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 189 op/s 2024-07-26T13:45:32.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:31 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/185070779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:45:32.301 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24607 quiesce db --set-id b3b1b2d8 --release --await' 2024-07-26T13:45:32.301 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- timeout 105 ceph tell mds.24607 quiesce db --set-id b3b1b2d8 --release --await 2024-07-26T13:45:32.417 INFO:tasks.workunit.client.0.smithi100.stdout: 1 192995 2.39 MB/sec execute 106 sec latency 8078.806 ms 2024-07-26T13:45:32.557 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:45:33.417 INFO:tasks.workunit.client.0.smithi100.stdout: 1 192995 2.37 MB/sec execute 107 sec latency 9078.884 ms 2024-07-26T13:45:33.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:33 smithi138 ceph-mon[36954]: pgmap v529: 97 pgs: 97 active+clean; 78 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.0 MiB/s wr, 165 op/s 2024-07-26T13:45:33.986 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:33 smithi100 ceph-mon[32599]: pgmap v529: 97 pgs: 97 active+clean; 78 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.0 MiB/s wr, 165 op/s 2024-07-26T13:45:34.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:33 smithi167 ceph-mon[37140]: pgmap v529: 97 pgs: 97 active+clean; 78 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.0 MiB/s wr, 165 op/s 2024-07-26T13:45:34.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.287+0000 7ff91a36b640 1 -- 172.21.15.100:0/2910809650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff914108720 msgr2=0x7ff91410ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:34.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.287+0000 7ff91a36b640 1 --2- 172.21.15.100:0/2910809650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff914108720 0x7ff91410ef50 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7ff90c009ca0 tx=0x7ff90c02a8c0 comp rx=0 tx=0).stop 2024-07-26T13:45:34.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.287+0000 7ff91a36b640 1 -- 172.21.15.100:0/2910809650 shutdown_connections 2024-07-26T13:45:34.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.287+0000 7ff91a36b640 1 --2- 172.21.15.100:0/2910809650 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff914108720 0x7ff91410ef50 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:34.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.287+0000 7ff91a36b640 1 --2- 172.21.15.100:0/2910809650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff914107dd0 0x7ff9141081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:34.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.287+0000 7ff91a36b640 1 --2- 172.21.15.100:0/2910809650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff914102490 0x7ff9141028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:34.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.287+0000 7ff91a36b640 1 -- 172.21.15.100:0/2910809650 >> 172.21.15.100:0/2910809650 conn(0x7ff914079160 msgr2=0x7ff914079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:34.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.287+0000 7ff91a36b640 1 -- 172.21.15.100:0/2910809650 shutdown_connections 2024-07-26T13:45:34.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.287+0000 7ff91a36b640 1 -- 172.21.15.100:0/2910809650 wait complete. 2024-07-26T13:45:34.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.288+0000 7ff91a36b640 1 Processor -- start 2024-07-26T13:45:34.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.288+0000 7ff91a36b640 1 -- start start 2024-07-26T13:45:34.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.289+0000 7ff91a36b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff914102490 0x7ff914198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:34.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.289+0000 7ff91a36b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff914107dd0 0x7ff914199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:34.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.289+0000 7ff91a36b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff914108720 0x7ff91419e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:34.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.289+0000 7ff91a36b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff9141123a0 con 0x7ff914107dd0 2024-07-26T13:45:34.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.289+0000 7ff91a36b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff914112220 con 0x7ff914108720 2024-07-26T13:45:34.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.289+0000 7ff91a36b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff914112520 con 0x7ff914102490 2024-07-26T13:45:34.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.289+0000 7ff90bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff914107dd0 0x7ff914199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:34.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.289+0000 7ff90bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff914107dd0 0x7ff914199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34332/0 (socket says 172.21.15.100:34332) 2024-07-26T13:45:34.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.289+0000 7ff90bfff640 1 -- 172.21.15.100:0/2512225017 learned_addr learned my addr 172.21.15.100:0/2512225017 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:34.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.289+0000 7ff913fff640 1 --2- 172.21.15.100:0/2512225017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff914102490 0x7ff914198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:34.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.289+0000 7ff918908640 1 --2- 172.21.15.100:0/2512225017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff914108720 0x7ff91419e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:34.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.289+0000 7ff90bfff640 1 -- 172.21.15.100:0/2512225017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff914102490 msgr2=0x7ff914198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:34.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.289+0000 7ff90bfff640 1 --2- 172.21.15.100:0/2512225017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff914102490 0x7ff914198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:34.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.289+0000 7ff90bfff640 1 -- 172.21.15.100:0/2512225017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff914108720 msgr2=0x7ff91419e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:34.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.289+0000 7ff90bfff640 1 --2- 172.21.15.100:0/2512225017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff914108720 0x7ff91419e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:34.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.289+0000 7ff90bfff640 1 -- 172.21.15.100:0/2512225017 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff91419ed40 con 0x7ff914107dd0 2024-07-26T13:45:34.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.289+0000 7ff90bfff640 1 --2- 172.21.15.100:0/2512225017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff914107dd0 0x7ff914199370 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc00d770 tx=0x7ff8fc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:34.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.290+0000 7ff911ffb640 1 -- 172.21.15.100:0/2512225017 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc017070 con 0x7ff914107dd0 2024-07-26T13:45:34.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.290+0000 7ff911ffb640 1 -- 172.21.15.100:0/2512225017 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8fc009c30 con 0x7ff914107dd0 2024-07-26T13:45:34.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.290+0000 7ff911ffb640 1 -- 172.21.15.100:0/2512225017 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc00f620 con 0x7ff914107dd0 2024-07-26T13:45:34.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.290+0000 7ff91a36b640 1 -- 172.21.15.100:0/2512225017 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff91419f030 con 0x7ff914107dd0 2024-07-26T13:45:34.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.290+0000 7ff91a36b640 1 -- 172.21.15.100:0/2512225017 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff91406a890 con 0x7ff914107dd0 2024-07-26T13:45:34.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.292+0000 7ff9208ef740 1 Processor -- start 2024-07-26T13:45:34.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.292+0000 7ff9208ef740 1 -- start start 2024-07-26T13:45:34.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.293+0000 7ff9208ef740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555db10e2520 0x555db1488070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:34.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.293+0000 7ff9208ef740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555db10b9660 0x555db14885b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:34.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.293+0000 7ff9208ef740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555db10c1840 0x555db1488af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:34.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.293+0000 7ff9208ef740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x555db13fde70 con 0x555db10b9660 2024-07-26T13:45:34.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.293+0000 7ff9208ef740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555db13ed370 con 0x555db10e2520 2024-07-26T13:45:34.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.293+0000 7ff9208ef740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555db13f13b0 con 0x555db10c1840 2024-07-26T13:45:34.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.293+0000 7ff911ffb640 1 -- 172.21.15.100:0/2512225017 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7ff8fc01a070 con 0x7ff914107dd0 2024-07-26T13:45:34.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.293+0000 7ff913fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555db10b9660 0x555db14885b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:34.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.293+0000 7ff913fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555db10b9660 0x555db14885b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34336/0 (socket says 172.21.15.100:34336) 2024-07-26T13:45:34.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.293+0000 7ff913fff640 1 -- 172.21.15.100:0/498156136 learned_addr learned my addr 172.21.15.100:0/498156136 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:34.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.293+0000 7ff918908640 1 --2- 172.21.15.100:0/498156136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555db10e2520 0x555db1488070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:34.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.293+0000 7ff913fff640 1 -- 172.21.15.100:0/498156136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555db10c1840 msgr2=0x555db1488af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:34.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.293+0000 7ff913fff640 1 --2- 172.21.15.100:0/498156136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555db10c1840 0x555db1488af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:34.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.293+0000 7ff913fff640 1 -- 172.21.15.100:0/498156136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555db10e2520 msgr2=0x555db1488070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:34.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.293+0000 7ff913fff640 1 --2- 172.21.15.100:0/498156136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555db10e2520 0x555db1488070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:34.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.293+0000 7ff913fff640 1 -- 172.21.15.100:0/498156136 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555db13ba040 con 0x555db10b9660 2024-07-26T13:45:34.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.293+0000 7ff913fff640 1 --2- 172.21.15.100:0/498156136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555db10b9660 0x555db14885b0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7ff904009ca0 tx=0x7ff904005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ecddd1c3fa9ee2d server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:34.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.293+0000 7ff90a7fc640 1 -- 172.21.15.100:0/498156136 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff904018600 con 0x555db10b9660 2024-07-26T13:45:34.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.294+0000 7ff90a7fc640 1 -- 172.21.15.100:0/498156136 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff904009e50 con 0x555db10b9660 2024-07-26T13:45:34.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.294+0000 7ff90a7fc640 1 -- 172.21.15.100:0/498156136 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff904017b80 con 0x555db10b9660 2024-07-26T13:45:34.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.294+0000 7ff911ffb640 1 --2- 172.21.15.100:0/2512225017 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff9000b3dc0 0x7ff9000b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:34.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.294+0000 7ff918908640 1 --2- 172.21.15.100:0/2512225017 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff9000b3dc0 0x7ff9000b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:34.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.294+0000 7ff911ffb640 1 -- 172.21.15.100:0/2512225017 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7ff8fc0e3a20 con 0x7ff914107dd0 2024-07-26T13:45:34.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.295+0000 7ff9208ef740 1 -- 172.21.15.100:0/498156136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555db10b9660 msgr2=0x555db14885b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:34.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.295+0000 7ff918908640 1 --2- 172.21.15.100:0/2512225017 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff9000b3dc0 0x7ff9000b62d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff90c006010 tx=0x7ff90c009440 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:34.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.295+0000 7ff9208ef740 1 --2- 172.21.15.100:0/498156136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555db10b9660 0x555db14885b0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7ff904009ca0 tx=0x7ff904005f90 comp rx=0 tx=0).stop 2024-07-26T13:45:34.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.295+0000 7ff9208ef740 1 -- 172.21.15.100:0/498156136 shutdown_connections 2024-07-26T13:45:34.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.295+0000 7ff9208ef740 1 --2- 172.21.15.100:0/498156136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555db10c1840 0x555db1488af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:34.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.295+0000 7ff9208ef740 1 --2- 172.21.15.100:0/498156136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555db10b9660 0x555db14885b0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:34.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.295+0000 7ff9208ef740 1 --2- 172.21.15.100:0/498156136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555db10e2520 0x555db1488070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:34.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.295+0000 7ff9208ef740 1 -- 172.21.15.100:0/498156136 >> 172.21.15.100:0/498156136 conn(0x555db10d6910 msgr2=0x555db1485f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:34.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.295+0000 7ff9208ef740 1 -- 172.21.15.100:0/498156136 shutdown_connections 2024-07-26T13:45:34.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.295+0000 7ff9208ef740 1 -- 172.21.15.100:0/498156136 wait complete. 2024-07-26T13:45:34.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.295+0000 7ff9208ef740 1 -- start start 2024-07-26T13:45:34.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.295+0000 7ff9208ef740 1 Processor -- start 2024-07-26T13:45:34.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.296+0000 7ff9208ef740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555db10b9660 0x555db1493cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:34.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.296+0000 7ff9208ef740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555db10c1840 0x555db1494210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:34.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.296+0000 7ff9208ef740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555db10e2520 0x555db149fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:34.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.296+0000 7ff9208ef740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x555db1250bb0 con 0x555db10c1840 2024-07-26T13:45:34.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.296+0000 7ff9208ef740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555db138fac0 con 0x555db10b9660 2024-07-26T13:45:34.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.296+0000 7ff9208ef740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555db1496850 con 0x555db10e2520 2024-07-26T13:45:34.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.296+0000 7ff9208ef740 10 client.0 _ll_register_callbacks cb 0x555db13983b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:45:34.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.296+0000 7ff90bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555db10c1840 0x555db1494210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:34.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.296+0000 7ff90bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555db10c1840 0x555db1494210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34338/0 (socket says 172.21.15.100:34338) 2024-07-26T13:45:34.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.296+0000 7ff90bfff640 1 -- 172.21.15.100:0/2388175459 learned_addr learned my addr 172.21.15.100:0/2388175459 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:34.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.296+0000 7ff913fff640 1 --2- 172.21.15.100:0/2388175459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555db10b9660 0x555db1493cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:34.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.296+0000 7ff90bfff640 1 -- 172.21.15.100:0/2388175459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555db10e2520 msgr2=0x555db149fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:34.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.296+0000 7ff90bfff640 1 --2- 172.21.15.100:0/2388175459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555db10e2520 0x555db149fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:34.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.296+0000 7ff90bfff640 1 -- 172.21.15.100:0/2388175459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555db10b9660 msgr2=0x555db1493cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:34.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.296+0000 7ff90bfff640 1 --2- 172.21.15.100:0/2388175459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555db10b9660 0x555db1493cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:34.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.296+0000 7ff90bfff640 1 -- 172.21.15.100:0/2388175459 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555db114db00 con 0x555db10c1840 2024-07-26T13:45:34.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.296+0000 7ff90bfff640 1 --2- 172.21.15.100:0/2388175459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555db10c1840 0x555db1494210 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc0aa5f0 tx=0x7ff8fc0aa0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd6210828cda7a9 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:34.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.296+0000 7ff8f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:45:34.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.297+0000 7ff8f37fe640 1 -- 172.21.15.100:0/2388175459 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc021050 con 0x555db10c1840 2024-07-26T13:45:34.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.297+0000 7ff8f37fe640 1 -- 172.21.15.100:0/2388175459 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7ff8fc0e1370 con 0x555db10c1840 2024-07-26T13:45:34.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.297+0000 7ff9208ef740 1 -- 172.21.15.100:0/2388175459 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff904009e50 con 0x555db10c1840 2024-07-26T13:45:34.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.297+0000 7ff8f37fe640 1 -- 172.21.15.100:0/2388175459 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7ff8fc0e1570 con 0x555db10c1840 2024-07-26T13:45:34.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.297+0000 7ff9208ef740 10 client.16878 fetch_fsmap learned FSMap version 46 2024-07-26T13:45:34.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.297+0000 7ff9208ef740 1 -- 172.21.15.100:0/2388175459 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x555db11c4960 con 0x555db10c1840 2024-07-26T13:45:34.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.297+0000 7ff8f37fe640 1 -- 172.21.15.100:0/2388175459 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff904009e50 con 0x555db10c1840 2024-07-26T13:45:34.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.297+0000 7ff8f37fe640 1 -- 172.21.15.100:0/2388175459 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 46) ==== 8346+0+0 (secure 0 0 0) 0x7ff8fc0cbab0 con 0x555db10c1840 2024-07-26T13:45:34.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.297+0000 7ff9208ef740 10 client.16878 fetch_fsmap finished waiting for FSMap version 46 2024-07-26T13:45:34.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.297+0000 7ff9208ef740 10 client.16878 resolve_mds: validated gid 24607 aka daemon mds.i 2024-07-26T13:45:34.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.297+0000 7ff9208ef740 20 client.16878 populate_metadata read hostname 'smithi100' 2024-07-26T13:45:34.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.297+0000 7ff9208ef740 1 --2- 172.21.15.100:0/2388175459 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555db10a0ea0 0x555db14a5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:34.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.297+0000 7ff9208ef740 4 client.16878 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:45:34.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.297+0000 7ff9208ef740 1 -- 172.21.15.100:0/2388175459 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555db13ffdc0 con 0x555db10a0ea0 2024-07-26T13:45:34.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.298+0000 7ff913fff640 1 --2- 172.21.15.100:0/2388175459 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555db10a0ea0 0x555db14a5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:34.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.298+0000 7ff913fff640 1 --2- 172.21.15.100:0/2388175459 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555db10a0ea0 0x555db14a5bc0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=675ff053f9b8eb9 server_cookie=8779af55ede8075e in_seq=0 out_seq=0 2024-07-26T13:45:34.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.298+0000 7ff8f37fe640 10 client.16878 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:34.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.299+0000 7ff8f37fe640 1 -- 172.21.15.100:0/2388175459 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555db13ffdc0 con 0x555db10a0ea0 2024-07-26T13:45:34.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.299+0000 7ff8f37fe640 10 client.16878 handle_command_reply: tid=0 2024-07-26T13:45:34.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.299+0000 7ff9208ef740 1 client.16878 shutdown 2024-07-26T13:45:34.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.299+0000 7ff9208ef740 1 -- 172.21.15.100:0/2388175459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555db10c1840 msgr2=0x555db1494210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:34.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.299+0000 7ff9208ef740 1 --2- 172.21.15.100:0/2388175459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555db10c1840 0x555db1494210 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc0aa5f0 tx=0x7ff8fc0aa0e0 comp rx=0 tx=0).stop 2024-07-26T13:45:34.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.299+0000 7ff9208ef740 1 -- 172.21.15.100:0/2388175459 shutdown_connections 2024-07-26T13:45:34.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.299+0000 7ff9208ef740 1 --2- 172.21.15.100:0/2388175459 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555db10a0ea0 0x555db14a5bc0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:34.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.299+0000 7ff9208ef740 1 --2- 172.21.15.100:0/2388175459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555db10e2520 0x555db149fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:34.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.299+0000 7ff9208ef740 1 --2- 172.21.15.100:0/2388175459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555db10c1840 0x555db1494210 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:34.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.299+0000 7ff8f37fe640 0 client.16878 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:34.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.299+0000 7ff9208ef740 1 --2- 172.21.15.100:0/2388175459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555db10b9660 0x555db1493cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:34.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.299+0000 7ff9208ef740 1 -- 172.21.15.100:0/2388175459 >> 172.21.15.100:0/2388175459 conn(0x555db10d6910 msgr2=0x555db148b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:34.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.299+0000 7ff9208ef740 1 -- 172.21.15.100:0/2388175459 shutdown_connections 2024-07-26T13:45:34.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.299+0000 7ff9208ef740 1 -- 172.21.15.100:0/2388175459 wait complete. 2024-07-26T13:45:34.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.299+0000 7ff9208ef740 20 client.16878 trim_cache size 0 max 16384 2024-07-26T13:45:34.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.316+0000 7ff9208ef740 1 Processor -- start 2024-07-26T13:45:34.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.316+0000 7ff9208ef740 1 -- start start 2024-07-26T13:45:34.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.316+0000 7ff9208ef740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555db10b9660 0x555db148f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:34.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.316+0000 7ff9208ef740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555db10c1840 0x555db14956e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:34.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.316+0000 7ff9208ef740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555db10e2520 0x555db14ab380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:34.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.316+0000 7ff9208ef740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff9141af9d0 con 0x555db10e2520 2024-07-26T13:45:34.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.316+0000 7ff9208ef740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff9141ad2a0 con 0x555db10b9660 2024-07-26T13:45:34.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.316+0000 7ff9208ef740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff9141ab0d0 con 0x555db10c1840 2024-07-26T13:45:34.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.316+0000 7ff918908640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555db10e2520 0x555db14ab380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:34.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.316+0000 7ff918908640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555db10e2520 0x555db14ab380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34340/0 (socket says 172.21.15.100:34340) 2024-07-26T13:45:34.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.316+0000 7ff918908640 1 -- 172.21.15.100:0/1909435950 learned_addr learned my addr 172.21.15.100:0/1909435950 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:34.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.316+0000 7ff90bfff640 1 --2- 172.21.15.100:0/1909435950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555db10c1840 0x555db14956e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:34.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.317+0000 7ff913fff640 1 --2- 172.21.15.100:0/1909435950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555db10b9660 0x555db148f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:34.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.317+0000 7ff90bfff640 1 -- 172.21.15.100:0/1909435950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555db10b9660 msgr2=0x555db148f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:34.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.317+0000 7ff90bfff640 1 --2- 172.21.15.100:0/1909435950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555db10b9660 0x555db148f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:34.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.317+0000 7ff90bfff640 1 -- 172.21.15.100:0/1909435950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555db10e2520 msgr2=0x555db14ab380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:34.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.317+0000 7ff90bfff640 1 --2- 172.21.15.100:0/1909435950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555db10e2520 0x555db14ab380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:34.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.317+0000 7ff90bfff640 1 -- 172.21.15.100:0/1909435950 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555db148a360 con 0x555db10c1840 2024-07-26T13:45:34.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.317+0000 7ff90bfff640 1 --2- 172.21.15.100:0/1909435950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555db10c1840 0x555db14956e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc0cbfd0 tx=0x7ff8fc0a9fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcf6ffa4043aa705 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:34.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.317+0000 7ff909ffb640 1 -- 172.21.15.100:0/1909435950 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc014070 con 0x555db10c1840 2024-07-26T13:45:34.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.317+0000 7ff909ffb640 1 -- 172.21.15.100:0/1909435950 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8fc0e3e50 con 0x555db10c1840 2024-07-26T13:45:34.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.317+0000 7ff909ffb640 1 -- 172.21.15.100:0/1909435950 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc0e1370 con 0x555db10c1840 2024-07-26T13:45:34.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.318+0000 7ff9208ef740 1 -- 172.21.15.100:0/1909435950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555db10c1840 msgr2=0x555db14956e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:34.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.318+0000 7ff9208ef740 1 --2- 172.21.15.100:0/1909435950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555db10c1840 0x555db14956e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc0cbfd0 tx=0x7ff8fc0a9fb0 comp rx=0 tx=0).stop 2024-07-26T13:45:34.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.318+0000 7ff9208ef740 1 -- 172.21.15.100:0/1909435950 shutdown_connections 2024-07-26T13:45:34.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.318+0000 7ff9208ef740 1 --2- 172.21.15.100:0/1909435950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555db10e2520 0x555db14ab380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:34.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.318+0000 7ff9208ef740 1 --2- 172.21.15.100:0/1909435950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555db10c1840 0x555db14956e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:34.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.318+0000 7ff9208ef740 1 --2- 172.21.15.100:0/1909435950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555db10b9660 0x555db148f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:34.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.318+0000 7ff9208ef740 1 -- 172.21.15.100:0/1909435950 >> 172.21.15.100:0/1909435950 conn(0x555db10d6910 msgr2=0x555db1494fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:34.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.318+0000 7ff9208ef740 1 -- 172.21.15.100:0/1909435950 shutdown_connections 2024-07-26T13:45:34.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.318+0000 7ff9208ef740 1 -- 172.21.15.100:0/1909435950 wait complete. 2024-07-26T13:45:34.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.319+0000 7ff9208ef740 1 -- start start 2024-07-26T13:45:34.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.319+0000 7ff9208ef740 1 Processor -- start 2024-07-26T13:45:34.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.319+0000 7ff9208ef740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555db10b9660 0x555db1498050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:34.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.319+0000 7ff9208ef740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555db10c1840 0x555db1498590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:34.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.319+0000 7ff9208ef740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555db10e2520 0x555db150d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:34.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.319+0000 7ff9208ef740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff9141ab760 con 0x555db10c1840 2024-07-26T13:45:34.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.319+0000 7ff9208ef740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff9141acbd0 con 0x555db10e2520 2024-07-26T13:45:34.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.319+0000 7ff9208ef740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff9141abe30 con 0x555db10b9660 2024-07-26T13:45:34.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.319+0000 7ff9208ef740 10 client.0 _ll_register_callbacks cb 0x7ff8fc010b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:45:34.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.319+0000 7ff90bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555db10c1840 0x555db1498590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:34.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.319+0000 7ff90bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555db10c1840 0x555db1498590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34346/0 (socket says 172.21.15.100:34346) 2024-07-26T13:45:34.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.319+0000 7ff90bfff640 1 -- 172.21.15.100:0/1405476800 learned_addr learned my addr 172.21.15.100:0/1405476800 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:34.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.319+0000 7ff913fff640 1 --2- 172.21.15.100:0/1405476800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555db10b9660 0x555db1498050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:34.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.319+0000 7ff918908640 1 --2- 172.21.15.100:0/1405476800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555db10e2520 0x555db150d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:34.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.320+0000 7ff90bfff640 1 -- 172.21.15.100:0/1405476800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555db10b9660 msgr2=0x555db1498050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:34.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.320+0000 7ff90bfff640 1 --2- 172.21.15.100:0/1405476800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555db10b9660 0x555db1498050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:34.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.320+0000 7ff90bfff640 1 -- 172.21.15.100:0/1405476800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555db10e2520 msgr2=0x555db150d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:34.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.320+0000 7ff90bfff640 1 --2- 172.21.15.100:0/1405476800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555db10e2520 0x555db150d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:34.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.320+0000 7ff90bfff640 1 -- 172.21.15.100:0/1405476800 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555db150d890 con 0x555db10c1840 2024-07-26T13:45:34.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.320+0000 7ff90bfff640 1 --2- 172.21.15.100:0/1405476800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555db10c1840 0x555db1498590 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc0aa360 tx=0x7ff8fc00de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=360aa74c12b0fab3 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:34.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.320+0000 7ff8f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:45:34.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.320+0000 7ff8f37fe640 1 -- 172.21.15.100:0/1405476800 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc014050 con 0x555db10c1840 2024-07-26T13:45:34.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.320+0000 7ff8f37fe640 1 -- 172.21.15.100:0/1405476800 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7ff8fc0a9ad0 con 0x555db10c1840 2024-07-26T13:45:34.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.320+0000 7ff9208ef740 1 -- 172.21.15.100:0/1405476800 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8fc0e3e50 con 0x555db10c1840 2024-07-26T13:45:34.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.321+0000 7ff8f37fe640 1 -- 172.21.15.100:0/1405476800 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7ff8fc0aa660 con 0x555db10c1840 2024-07-26T13:45:34.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.321+0000 7ff9208ef740 10 client.16884 fetch_fsmap learned FSMap version 46 2024-07-26T13:45:34.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.321+0000 7ff9208ef740 1 -- 172.21.15.100:0/1405476800 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x555db150d890 con 0x555db10c1840 2024-07-26T13:45:34.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.321+0000 7ff8f37fe640 1 -- 172.21.15.100:0/1405476800 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8fc0e3e50 con 0x555db10c1840 2024-07-26T13:45:34.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.321+0000 7ff8f37fe640 1 -- 172.21.15.100:0/1405476800 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 46) ==== 8346+0+0 (secure 0 0 0) 0x7ff8fc0afd40 con 0x555db10c1840 2024-07-26T13:45:34.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.321+0000 7ff9208ef740 10 client.16884 fetch_fsmap finished waiting for FSMap version 46 2024-07-26T13:45:34.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.321+0000 7ff9208ef740 10 client.16884 resolve_mds: validated gid 24607 aka daemon mds.i 2024-07-26T13:45:34.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.321+0000 7ff9208ef740 20 client.16884 populate_metadata read hostname 'smithi100' 2024-07-26T13:45:34.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.321+0000 7ff9208ef740 1 --2- 172.21.15.100:0/1405476800 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555db1076250 0x555db1510d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:34.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.321+0000 7ff9208ef740 4 client.16884 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b3b1b2d8", "release": true, "await": true}] 2024-07-26T13:45:34.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.321+0000 7ff9208ef740 1 -- 172.21.15.100:0/1405476800 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b3b1b2d8", "release": true, "await": true}) -- 0x555db13fb860 con 0x555db1076250 2024-07-26T13:45:34.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.321+0000 7ff913fff640 1 --2- 172.21.15.100:0/1405476800 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555db1076250 0x555db1510d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:34.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.322+0000 7ff8f37fe640 10 client.16884 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:34.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.322+0000 7ff913fff640 1 --2- 172.21.15.100:0/1405476800 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555db1076250 0x555db1510d80 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dce513a11346c442 server_cookie=a74def463dfab509 in_seq=0 out_seq=0 2024-07-26T13:45:34.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.328+0000 7ff8f37fe640 1 -- 172.21.15.100:0/1405476800 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555db13fb860 con 0x555db1076250 2024-07-26T13:45:34.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.328+0000 7ff8f37fe640 10 client.16884 handle_command_reply: tid=0 2024-07-26T13:45:34.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.329+0000 7ff9208ef740 1 client.16884 shutdown 2024-07-26T13:45:34.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.329+0000 7ff9208ef740 1 -- 172.21.15.100:0/1405476800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555db10c1840 msgr2=0x555db1498590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:34.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.329+0000 7ff9208ef740 1 --2- 172.21.15.100:0/1405476800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555db10c1840 0x555db1498590 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc0aa360 tx=0x7ff8fc00de80 comp rx=0 tx=0).stop 2024-07-26T13:45:34.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.329+0000 7ff9208ef740 1 -- 172.21.15.100:0/1405476800 shutdown_connections 2024-07-26T13:45:34.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.329+0000 7ff9208ef740 1 --2- 172.21.15.100:0/1405476800 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555db1076250 0x555db1510d80 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:34.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.329+0000 7ff9208ef740 1 --2- 172.21.15.100:0/1405476800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555db10e2520 0x555db150d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:34.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.329+0000 7ff9208ef740 1 --2- 172.21.15.100:0/1405476800 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555db10c1840 0x555db1498590 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:34.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.329+0000 7ff8f37fe640 0 client.16884 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:34.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.329+0000 7ff9208ef740 1 --2- 172.21.15.100:0/1405476800 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555db10b9660 0x555db1498050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:34.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.329+0000 7ff9208ef740 1 -- 172.21.15.100:0/1405476800 >> 172.21.15.100:0/1405476800 conn(0x555db10d6910 msgr2=0x555db1491f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:34.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.329+0000 7ff9208ef740 1 -- 172.21.15.100:0/1405476800 shutdown_connections 2024-07-26T13:45:34.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.329+0000 7ff9208ef740 1 -- 172.21.15.100:0/1405476800 wait complete. 2024-07-26T13:45:34.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.329+0000 7ff9208ef740 20 client.16884 trim_cache size 0 max 16384 2024-07-26T13:45:34.363 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:45:34.364 INFO:teuthology.orchestra.run.smithi100.stdout: "epoch": 46, 2024-07-26T13:45:34.364 INFO:teuthology.orchestra.run.smithi100.stdout: "leader": 24607, 2024-07-26T13:45:34.364 INFO:teuthology.orchestra.run.smithi100.stdout: "set_version": 16, 2024-07-26T13:45:34.364 INFO:teuthology.orchestra.run.smithi100.stdout: "sets": { 2024-07-26T13:45:34.364 INFO:teuthology.orchestra.run.smithi100.stdout: "b3b1b2d8": { 2024-07-26T13:45:34.364 INFO:teuthology.orchestra.run.smithi100.stdout: "version": 16, 2024-07-26T13:45:34.364 INFO:teuthology.orchestra.run.smithi100.stdout: "age_ref": 0.0, 2024-07-26T13:45:34.365 INFO:teuthology.orchestra.run.smithi100.stdout: "state": { 2024-07-26T13:45:34.365 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RELEASED", 2024-07-26T13:45:34.365 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 0.0 2024-07-26T13:45:34.365 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-07-26T13:45:34.365 INFO:teuthology.orchestra.run.smithi100.stdout: "timeout": 90.0, 2024-07-26T13:45:34.365 INFO:teuthology.orchestra.run.smithi100.stdout: "expiration": 130.0, 2024-07-26T13:45:34.365 INFO:teuthology.orchestra.run.smithi100.stdout: "members": { 2024-07-26T13:45:34.366 INFO:teuthology.orchestra.run.smithi100.stdout: "file:/": { 2024-07-26T13:45:34.366 INFO:teuthology.orchestra.run.smithi100.stdout: "excluded": false, 2024-07-26T13:45:34.366 INFO:teuthology.orchestra.run.smithi100.stdout: "state": { 2024-07-26T13:45:34.366 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RELEASED", 2024-07-26T13:45:34.366 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 0.0 2024-07-26T13:45:34.366 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:45:34.367 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:45:34.367 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:45:34.367 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:45:34.367 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:45:34.367 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:45:34.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.330+0000 7ff91a36b640 1 -- 172.21.15.100:0/2512225017 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff9000b3dc0 msgr2=0x7ff9000b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:34.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.330+0000 7ff91a36b640 1 --2- 172.21.15.100:0/2512225017 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff9000b3dc0 0x7ff9000b62d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff90c006010 tx=0x7ff90c009440 comp rx=0 tx=0).stop 2024-07-26T13:45:34.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.330+0000 7ff91a36b640 1 -- 172.21.15.100:0/2512225017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff914107dd0 msgr2=0x7ff914199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:34.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.330+0000 7ff91a36b640 1 --2- 172.21.15.100:0/2512225017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff914107dd0 0x7ff914199370 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc00d770 tx=0x7ff8fc00dc30 comp rx=0 tx=0).stop 2024-07-26T13:45:34.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.330+0000 7ff91a36b640 1 -- 172.21.15.100:0/2512225017 shutdown_connections 2024-07-26T13:45:34.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.330+0000 7ff91a36b640 1 --2- 172.21.15.100:0/2512225017 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff9000b3dc0 0x7ff9000b62d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:34.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.330+0000 7ff91a36b640 1 --2- 172.21.15.100:0/2512225017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff914108720 0x7ff91419e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:34.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.330+0000 7ff91a36b640 1 --2- 172.21.15.100:0/2512225017 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff914107dd0 0x7ff914199370 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:34.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.330+0000 7ff91a36b640 1 --2- 172.21.15.100:0/2512225017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff914102490 0x7ff914198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:34.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.330+0000 7ff91a36b640 1 -- 172.21.15.100:0/2512225017 >> 172.21.15.100:0/2512225017 conn(0x7ff914079160 msgr2=0x7ff914103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:34.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.330+0000 7ff91a36b640 1 -- 172.21.15.100:0/2512225017 shutdown_connections 2024-07-26T13:45:34.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:34.330+0000 7ff91a36b640 1 -- 172.21.15.100:0/2512225017 wait complete. 2024-07-26T13:45:34.417 INFO:tasks.workunit.client.0.smithi100.stdout: 1 193106 2.35 MB/sec execute 108 sec latency 9994.389 ms 2024-07-26T13:45:35.058 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.053+0000 7f12303a2640 1 -- 172.21.15.100:0/673393591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1228108e60 msgr2=0x7f1228109270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:35.058 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.053+0000 7f12303a2640 1 --2- 172.21.15.100:0/673393591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1228108e60 0x7f1228109270 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f1224009900 tx=0x7f122402a410 comp rx=0 tx=0).stop 2024-07-26T13:45:35.058 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.053+0000 7f12303a2640 1 -- 172.21.15.100:0/673393591 shutdown_connections 2024-07-26T13:45:35.058 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.053+0000 7f12303a2640 1 --2- 172.21.15.100:0/673393591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12281097b0 0x7f122810ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:35.058 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.053+0000 7f12303a2640 1 --2- 172.21.15.100:0/673393591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1228108e60 0x7f1228109270 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:35.058 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.053+0000 7f12303a2640 1 --2- 172.21.15.100:0/673393591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1228101a90 0x7f1228101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:35.058 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.053+0000 7f12303a2640 1 -- 172.21.15.100:0/673393591 >> 172.21.15.100:0/673393591 conn(0x7f12280798c0 msgr2=0x7f1228079cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:35.058 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.053+0000 7f12303a2640 1 -- 172.21.15.100:0/673393591 shutdown_connections 2024-07-26T13:45:35.058 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.053+0000 7f12303a2640 1 -- 172.21.15.100:0/673393591 wait complete. 2024-07-26T13:45:35.058 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.054+0000 7f12303a2640 1 Processor -- start 2024-07-26T13:45:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.054+0000 7f12303a2640 1 -- start start 2024-07-26T13:45:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.054+0000 7f12303a2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1228101a90 0x7f1228102a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.054+0000 7f12303a2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1228108e60 0x7f1228102f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.054+0000 7f12303a2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12281097b0 0x7f12281034a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.054+0000 7f12303a2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1228113440 con 0x7f12281097b0 2024-07-26T13:45:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.054+0000 7f12303a2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f12281132c0 con 0x7f1228101a90 2024-07-26T13:45:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.054+0000 7f12303a2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f12281135c0 con 0x7f1228108e60 2024-07-26T13:45:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.054+0000 7f122e93f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12281097b0 0x7f12281034a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.054+0000 7f122d93d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1228108e60 0x7f1228102f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.054+0000 7f122d93d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1228108e60 0x7f1228102f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:51576/0 (socket says 172.21.15.100:51576) 2024-07-26T13:45:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.054+0000 7f122d93d640 1 -- 172.21.15.100:0/3511915177 learned_addr learned my addr 172.21.15.100:0/3511915177 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.054+0000 7f122e93f640 1 --2- 172.21.15.100:0/3511915177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12281097b0 0x7f12281034a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59680/0 (socket says 172.21.15.100:59680) 2024-07-26T13:45:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.054+0000 7f122e13e640 1 --2- 172.21.15.100:0/3511915177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1228101a90 0x7f1228102a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.055+0000 7f122e93f640 1 -- 172.21.15.100:0/3511915177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1228108e60 msgr2=0x7f1228102f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.055+0000 7f122e93f640 1 --2- 172.21.15.100:0/3511915177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1228108e60 0x7f1228102f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.055+0000 7f122e93f640 1 -- 172.21.15.100:0/3511915177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1228101a90 msgr2=0x7f1228102a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.055+0000 7f122e93f640 1 --2- 172.21.15.100:0/3511915177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1228101a90 0x7f1228102a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.055+0000 7f122e93f640 1 -- 172.21.15.100:0/3511915177 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12281a8190 con 0x7f12281097b0 2024-07-26T13:45:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.055+0000 7f122e13e640 1 --2- 172.21.15.100:0/3511915177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1228101a90 0x7f1228102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:45:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.055+0000 7f122e93f640 1 --2- 172.21.15.100:0/3511915177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12281097b0 0x7f12281034a0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f1220009fd0 tx=0x7f122000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.055+0000 7f121f7fe640 1 -- 172.21.15.100:0/3511915177 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122000ce20 con 0x7f12281097b0 2024-07-26T13:45:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.055+0000 7f121f7fe640 1 -- 172.21.15.100:0/3511915177 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1220005020 con 0x7f12281097b0 2024-07-26T13:45:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.055+0000 7f12303a2640 1 -- 172.21.15.100:0/3511915177 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12281a8480 con 0x7f12281097b0 2024-07-26T13:45:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.055+0000 7f121f7fe640 1 -- 172.21.15.100:0/3511915177 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1220010690 con 0x7f12281097b0 2024-07-26T13:45:35.061 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.055+0000 7f12303a2640 1 -- 172.21.15.100:0/3511915177 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12281a89d0 con 0x7f12281097b0 2024-07-26T13:45:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.058+0000 7f121f7fe640 1 -- 172.21.15.100:0/3511915177 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f12200054d0 con 0x7f12281097b0 2024-07-26T13:45:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.059+0000 7f121f7fe640 1 --2- 172.21.15.100:0/3511915177 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f12140b3d70 0x7f12140b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.059+0000 7f1237392740 1 Processor -- start 2024-07-26T13:45:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.059+0000 7f122e13e640 1 --2- 172.21.15.100:0/3511915177 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f12140b3d70 0x7f12140b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.059+0000 7f121f7fe640 1 -- 172.21.15.100:0/3511915177 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7f12200e46b0 con 0x7f12281097b0 2024-07-26T13:45:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.059+0000 7f1237392740 1 -- start start 2024-07-26T13:45:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.059+0000 7f1237392740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b99cd1e1b0 0x55b99d0c06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.059+0000 7f1237392740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b99ccf52f0 0x55b99d0c0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.059+0000 7f1237392740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b99ccfd4d0 0x55b99d0c1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.059+0000 7f1237392740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55b99d0363b0 con 0x55b99ccf52f0 2024-07-26T13:45:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.059+0000 7f1237392740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b99d0258b0 con 0x55b99cd1e1b0 2024-07-26T13:45:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.060+0000 7f1237392740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b99d0298f0 con 0x55b99ccfd4d0 2024-07-26T13:45:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.060+0000 7f122e13e640 1 --2- 172.21.15.100:0/3511915177 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f12140b3d70 0x7f12140b6280 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1228104470 tx=0x7f1218005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.060+0000 7f122d93d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b99ccf52f0 0x55b99d0c0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.060+0000 7f122d93d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b99ccf52f0 0x55b99d0c0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59690/0 (socket says 172.21.15.100:59690) 2024-07-26T13:45:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.060+0000 7f122d93d640 1 -- 172.21.15.100:0/3980470145 learned_addr learned my addr 172.21.15.100:0/3980470145 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.060+0000 7f122e13e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b99cd1e1b0 0x55b99d0c06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.060+0000 7f122e93f640 1 --2- 172.21.15.100:0/3980470145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b99ccfd4d0 0x55b99d0c1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.060+0000 7f122d93d640 1 -- 172.21.15.100:0/3980470145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b99ccfd4d0 msgr2=0x55b99d0c1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.060+0000 7f122d93d640 1 --2- 172.21.15.100:0/3980470145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b99ccfd4d0 0x55b99d0c1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.060+0000 7f122d93d640 1 -- 172.21.15.100:0/3980470145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b99cd1e1b0 msgr2=0x55b99d0c06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.060+0000 7f122d93d640 1 --2- 172.21.15.100:0/3980470145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b99cd1e1b0 0x55b99d0c06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.060+0000 7f122d93d640 1 -- 172.21.15.100:0/3980470145 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b99cfcc9a0 con 0x55b99ccf52f0 2024-07-26T13:45:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.060+0000 7f122e13e640 1 --2- 172.21.15.100:0/3980470145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b99cd1e1b0 0x55b99d0c06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:45:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.060+0000 7f122d93d640 1 --2- 172.21.15.100:0/3980470145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b99ccf52f0 0x55b99d0c0bf0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f1224009230 tx=0x7f1224031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f7c8b6d95821bc server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.060+0000 7f121cbfe640 1 -- 172.21.15.100:0/3980470145 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1224040070 con 0x55b99ccf52f0 2024-07-26T13:45:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.060+0000 7f121cbfe640 1 -- 172.21.15.100:0/3980470145 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1224004030 con 0x55b99ccf52f0 2024-07-26T13:45:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.060+0000 7f121cbfe640 1 -- 172.21.15.100:0/3980470145 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1224035040 con 0x55b99ccf52f0 2024-07-26T13:45:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.062+0000 7f1237392740 1 -- 172.21.15.100:0/3980470145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b99ccf52f0 msgr2=0x55b99d0c0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.062+0000 7f1237392740 1 --2- 172.21.15.100:0/3980470145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b99ccf52f0 0x55b99d0c0bf0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f1224009230 tx=0x7f1224031b10 comp rx=0 tx=0).stop 2024-07-26T13:45:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.062+0000 7f1237392740 1 -- 172.21.15.100:0/3980470145 shutdown_connections 2024-07-26T13:45:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.062+0000 7f1237392740 1 --2- 172.21.15.100:0/3980470145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b99ccfd4d0 0x55b99d0c1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.062+0000 7f1237392740 1 --2- 172.21.15.100:0/3980470145 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b99ccf52f0 0x55b99d0c0bf0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.062+0000 7f1237392740 1 --2- 172.21.15.100:0/3980470145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b99cd1e1b0 0x55b99d0c06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.062+0000 7f1237392740 1 -- 172.21.15.100:0/3980470145 >> 172.21.15.100:0/3980470145 conn(0x55b99cd12300 msgr2=0x55b99d0be5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.062+0000 7f1237392740 1 -- 172.21.15.100:0/3980470145 shutdown_connections 2024-07-26T13:45:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.062+0000 7f1237392740 1 -- 172.21.15.100:0/3980470145 wait complete. 2024-07-26T13:45:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.062+0000 7f1237392740 1 -- start start 2024-07-26T13:45:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.063+0000 7f1237392740 1 Processor -- start 2024-07-26T13:45:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.063+0000 7f1237392740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b99ccf52f0 0x55b99d0cc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.063+0000 7f1237392740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b99ccfd4d0 0x55b99d0cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.063+0000 7f1237392740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b99cd1e1b0 0x55b99d0d80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.063+0000 7f1237392740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55b99cf5fab0 con 0x55b99cd1e1b0 2024-07-26T13:45:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.063+0000 7f1237392740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b99cfadcf0 con 0x55b99ccf52f0 2024-07-26T13:45:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.063+0000 7f1237392740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b99d0cedd0 con 0x55b99ccfd4d0 2024-07-26T13:45:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.063+0000 7f1237392740 10 client.0 _ll_register_callbacks cb 0x55b99d01f3e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:45:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.063+0000 7f122e93f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b99cd1e1b0 0x55b99d0d80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.063+0000 7f122d93d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b99ccfd4d0 0x55b99d0cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.063+0000 7f122e93f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b99cd1e1b0 0x55b99d0d80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59706/0 (socket says 172.21.15.100:59706) 2024-07-26T13:45:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.063+0000 7f122e93f640 1 -- 172.21.15.100:0/1813297424 learned_addr learned my addr 172.21.15.100:0/1813297424 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.063+0000 7f122e13e640 1 --2- 172.21.15.100:0/1813297424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b99ccf52f0 0x55b99d0cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.063+0000 7f122e93f640 1 -- 172.21.15.100:0/1813297424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b99ccfd4d0 msgr2=0x55b99d0cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.063+0000 7f122e93f640 1 --2- 172.21.15.100:0/1813297424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b99ccfd4d0 0x55b99d0cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.064+0000 7f122e93f640 1 -- 172.21.15.100:0/1813297424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b99ccf52f0 msgr2=0x55b99d0cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.064+0000 7f122e93f640 1 --2- 172.21.15.100:0/1813297424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b99ccf52f0 0x55b99d0cc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.064+0000 7f122e93f640 1 -- 172.21.15.100:0/1813297424 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b99cd89790 con 0x55b99cd1e1b0 2024-07-26T13:45:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.064+0000 7f122e13e640 1 --2- 172.21.15.100:0/1813297424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b99ccf52f0 0x55b99d0cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:45:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.064+0000 7f122e93f640 1 --2- 172.21.15.100:0/1813297424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b99cd1e1b0 0x55b99d0d80f0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f1220005820 tx=0x7f12200a77a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5817a0740df11e7d server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.064+0000 7f1211ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:45:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.064+0000 7f1211ffb640 1 -- 172.21.15.100:0/1813297424 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12200af840 con 0x55b99cd1e1b0 2024-07-26T13:45:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.064+0000 7f1211ffb640 1 -- 172.21.15.100:0/1813297424 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7f12200a7830 con 0x55b99cd1e1b0 2024-07-26T13:45:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.064+0000 7f1237392740 1 -- 172.21.15.100:0/1813297424 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1224004030 con 0x55b99cd1e1b0 2024-07-26T13:45:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.064+0000 7f1211ffb640 1 -- 172.21.15.100:0/1813297424 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f12200cc050 con 0x55b99cd1e1b0 2024-07-26T13:45:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.064+0000 7f1237392740 10 client.16896 fetch_fsmap learned FSMap version 46 2024-07-26T13:45:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.064+0000 7f1237392740 1 -- 172.21.15.100:0/1813297424 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55b99ce02620 con 0x55b99cd1e1b0 2024-07-26T13:45:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.064+0000 7f1211ffb640 1 -- 172.21.15.100:0/1813297424 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1224004030 con 0x55b99cd1e1b0 2024-07-26T13:45:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.065+0000 7f1211ffb640 1 -- 172.21.15.100:0/1813297424 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 46) ==== 8346+0+0 (secure 0 0 0) 0x7f12200aa530 con 0x55b99cd1e1b0 2024-07-26T13:45:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.065+0000 7f1237392740 10 client.16896 fetch_fsmap finished waiting for FSMap version 46 2024-07-26T13:45:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.065+0000 7f1237392740 10 client.16896 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:45:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.065+0000 7f1237392740 20 client.16896 populate_metadata read hostname 'smithi100' 2024-07-26T13:45:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.065+0000 7f1237392740 1 --2- 172.21.15.100:0/1813297424 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55b99ccdcd80 0x55b99d0de050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.065+0000 7f1237392740 4 client.16896 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:45:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.065+0000 7f1237392740 1 -- 172.21.15.100:0/1813297424 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b99d038300 con 0x55b99ccdcd80 2024-07-26T13:45:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.065+0000 7f122e13e640 1 --2- 172.21.15.100:0/1813297424 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55b99ccdcd80 0x55b99d0de050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.065+0000 7f122e13e640 1 --2- 172.21.15.100:0/1813297424 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55b99ccdcd80 0x55b99d0de050 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec18b0c9619dfb0f server_cookie=fa2614834b850b3d in_seq=0 out_seq=0 2024-07-26T13:45:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.065+0000 7f1211ffb640 10 client.16896 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.066+0000 7f1211ffb640 1 -- 172.21.15.100:0/1813297424 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b99d038300 con 0x55b99ccdcd80 2024-07-26T13:45:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.066+0000 7f1211ffb640 10 client.16896 handle_command_reply: tid=0 2024-07-26T13:45:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.066+0000 7f1237392740 1 client.16896 shutdown 2024-07-26T13:45:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.066+0000 7f1237392740 1 -- 172.21.15.100:0/1813297424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b99cd1e1b0 msgr2=0x55b99d0d80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.066+0000 7f1237392740 1 --2- 172.21.15.100:0/1813297424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b99cd1e1b0 0x55b99d0d80f0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f1220005820 tx=0x7f12200a77a0 comp rx=0 tx=0).stop 2024-07-26T13:45:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.066+0000 7f1237392740 1 -- 172.21.15.100:0/1813297424 shutdown_connections 2024-07-26T13:45:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.066+0000 7f1237392740 1 --2- 172.21.15.100:0/1813297424 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55b99ccdcd80 0x55b99d0de050 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.066+0000 7f1237392740 1 --2- 172.21.15.100:0/1813297424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b99cd1e1b0 0x55b99d0d80f0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.066+0000 7f1237392740 1 --2- 172.21.15.100:0/1813297424 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b99ccfd4d0 0x55b99d0cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.066+0000 7f1237392740 1 --2- 172.21.15.100:0/1813297424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b99ccf52f0 0x55b99d0cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.066+0000 7f1237392740 1 -- 172.21.15.100:0/1813297424 >> 172.21.15.100:0/1813297424 conn(0x55b99cd12300 msgr2=0x55b99d0c16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.066+0000 7f1211ffb640 0 client.16896 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.066+0000 7f1237392740 1 -- 172.21.15.100:0/1813297424 shutdown_connections 2024-07-26T13:45:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.067+0000 7f1237392740 1 -- 172.21.15.100:0/1813297424 wait complete. 2024-07-26T13:45:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.067+0000 7f1237392740 20 client.16896 trim_cache size 0 max 16384 2024-07-26T13:45:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.082+0000 7f1237392740 1 Processor -- start 2024-07-26T13:45:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.082+0000 7f1237392740 1 -- start start 2024-07-26T13:45:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.083+0000 7f1237392740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b99ccf52f0 0x55b99d0cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.083+0000 7f1237392740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b99ccfd4d0 0x55b99d0d7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.083+0000 7f1237392740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b99cd1e1b0 0x55b99d0d8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.083+0000 7f1237392740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f12281b3d80 con 0x55b99ccf52f0 2024-07-26T13:45:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.083+0000 7f1237392740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f12281b1650 con 0x55b99cd1e1b0 2024-07-26T13:45:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.083+0000 7f1237392740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f12281af480 con 0x55b99ccfd4d0 2024-07-26T13:45:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.083+0000 7f122e13e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b99ccf52f0 0x55b99d0cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.083+0000 7f122e13e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b99ccf52f0 0x55b99d0cbb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59714/0 (socket says 172.21.15.100:59714) 2024-07-26T13:45:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.083+0000 7f122e13e640 1 -- 172.21.15.100:0/4215218232 learned_addr learned my addr 172.21.15.100:0/4215218232 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.083+0000 7f122e93f640 1 --2- 172.21.15.100:0/4215218232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b99cd1e1b0 0x55b99d0d8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.083+0000 7f122d93d640 1 --2- 172.21.15.100:0/4215218232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b99ccfd4d0 0x55b99d0d7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.083+0000 7f122e13e640 1 -- 172.21.15.100:0/4215218232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b99ccfd4d0 msgr2=0x55b99d0d7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.083+0000 7f122e13e640 1 --2- 172.21.15.100:0/4215218232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b99ccfd4d0 0x55b99d0d7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.083+0000 7f122e13e640 1 -- 172.21.15.100:0/4215218232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b99cd1e1b0 msgr2=0x55b99d0d8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.083+0000 7f122e13e640 1 --2- 172.21.15.100:0/4215218232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b99cd1e1b0 0x55b99d0d8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.083+0000 7f122e13e640 1 -- 172.21.15.100:0/4215218232 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b99d0c2be0 con 0x55b99ccf52f0 2024-07-26T13:45:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.083+0000 7f122e93f640 1 --2- 172.21.15.100:0/4215218232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b99cd1e1b0 0x55b99d0d8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:45:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.083+0000 7f122e13e640 1 --2- 172.21.15.100:0/4215218232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b99ccf52f0 0x55b99d0cbb20 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f1218013010 tx=0x7f1218014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a891b18f4804f28 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.084+0000 7f1213fff640 1 -- 172.21.15.100:0/4215218232 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1218032670 con 0x55b99ccf52f0 2024-07-26T13:45:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.084+0000 7f1213fff640 1 -- 172.21.15.100:0/4215218232 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1218032cc0 con 0x55b99ccf52f0 2024-07-26T13:45:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.084+0000 7f1213fff640 1 -- 172.21.15.100:0/4215218232 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121800fba0 con 0x55b99ccf52f0 2024-07-26T13:45:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.084+0000 7f1237392740 1 -- 172.21.15.100:0/4215218232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b99ccf52f0 msgr2=0x55b99d0cbb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.084+0000 7f1237392740 1 --2- 172.21.15.100:0/4215218232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b99ccf52f0 0x55b99d0cbb20 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f1218013010 tx=0x7f1218014000 comp rx=0 tx=0).stop 2024-07-26T13:45:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.084+0000 7f1237392740 1 -- 172.21.15.100:0/4215218232 shutdown_connections 2024-07-26T13:45:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.084+0000 7f1237392740 1 --2- 172.21.15.100:0/4215218232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b99cd1e1b0 0x55b99d0d8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.084+0000 7f1237392740 1 --2- 172.21.15.100:0/4215218232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b99ccfd4d0 0x55b99d0d7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.084+0000 7f1237392740 1 --2- 172.21.15.100:0/4215218232 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b99ccf52f0 0x55b99d0cbb20 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.084+0000 7f1237392740 1 -- 172.21.15.100:0/4215218232 >> 172.21.15.100:0/4215218232 conn(0x55b99cd12300 msgr2=0x55b99d0cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.085+0000 7f1237392740 1 -- 172.21.15.100:0/4215218232 shutdown_connections 2024-07-26T13:45:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.085+0000 7f1237392740 1 -- 172.21.15.100:0/4215218232 wait complete. 2024-07-26T13:45:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.085+0000 7f1237392740 1 -- start start 2024-07-26T13:45:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.085+0000 7f1237392740 1 Processor -- start 2024-07-26T13:45:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.085+0000 7f1237392740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b99ccf52f0 0x55b99d0d4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.085+0000 7f1237392740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b99ccfd4d0 0x55b99d0d1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.085+0000 7f1237392740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b99cd1e1b0 0x55b99d0d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.085+0000 7f1237392740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f12281afb10 con 0x55b99ccfd4d0 2024-07-26T13:45:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.085+0000 7f1237392740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f12281b0f80 con 0x55b99ccf52f0 2024-07-26T13:45:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.085+0000 7f1237392740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f12281b01e0 con 0x55b99cd1e1b0 2024-07-26T13:45:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.085+0000 7f1237392740 10 client.0 _ll_register_callbacks cb 0x7f1220005400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:45:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.085+0000 7f122d93d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b99ccfd4d0 0x55b99d0d1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.085+0000 7f122d93d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b99ccfd4d0 0x55b99d0d1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59726/0 (socket says 172.21.15.100:59726) 2024-07-26T13:45:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.085+0000 7f122d93d640 1 -- 172.21.15.100:0/2488592856 learned_addr learned my addr 172.21.15.100:0/2488592856 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.085+0000 7f122e93f640 1 --2- 172.21.15.100:0/2488592856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b99cd1e1b0 0x55b99d0d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.086+0000 7f122e13e640 1 --2- 172.21.15.100:0/2488592856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b99ccf52f0 0x55b99d0d4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.086+0000 7f122d93d640 1 -- 172.21.15.100:0/2488592856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b99cd1e1b0 msgr2=0x55b99d0d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.086+0000 7f122d93d640 1 --2- 172.21.15.100:0/2488592856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b99cd1e1b0 0x55b99d0d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.086+0000 7f122d93d640 1 -- 172.21.15.100:0/2488592856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b99ccf52f0 msgr2=0x55b99d0d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.086+0000 7f122d93d640 1 --2- 172.21.15.100:0/2488592856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b99ccf52f0 0x55b99d0d4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.086+0000 7f122d93d640 1 -- 172.21.15.100:0/2488592856 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b99d0d1d10 con 0x55b99ccfd4d0 2024-07-26T13:45:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.086+0000 7f122d93d640 1 --2- 172.21.15.100:0/2488592856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b99ccfd4d0 0x55b99d0d1020 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f1224005130 tx=0x7f122402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a4cb4167a5440b1 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.086+0000 7f1211ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:45:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.086+0000 7f1211ffb640 1 -- 172.21.15.100:0/2488592856 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1224054060 con 0x55b99ccfd4d0 2024-07-26T13:45:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.086+0000 7f1211ffb640 1 -- 172.21.15.100:0/2488592856 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7f12240316a0 con 0x55b99ccfd4d0 2024-07-26T13:45:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.086+0000 7f1237392740 1 -- 172.21.15.100:0/2488592856 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1218032cc0 con 0x55b99ccfd4d0 2024-07-26T13:45:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.086+0000 7f1211ffb640 1 -- 172.21.15.100:0/2488592856 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f1224040040 con 0x55b99ccfd4d0 2024-07-26T13:45:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.086+0000 7f1237392740 10 client.16902 fetch_fsmap learned FSMap version 46 2024-07-26T13:45:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.086+0000 7f1237392740 1 -- 172.21.15.100:0/2488592856 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55b99d145740 con 0x55b99ccfd4d0 2024-07-26T13:45:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.087+0000 7f1211ffb640 1 -- 172.21.15.100:0/2488592856 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1218032cc0 con 0x55b99ccfd4d0 2024-07-26T13:45:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.087+0000 7f1211ffb640 1 -- 172.21.15.100:0/2488592856 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 46) ==== 8346+0+0 (secure 0 0 0) 0x7f122404e070 con 0x55b99ccfd4d0 2024-07-26T13:45:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.087+0000 7f1237392740 10 client.16902 fetch_fsmap finished waiting for FSMap version 46 2024-07-26T13:45:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.087+0000 7f1237392740 10 client.16902 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:45:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.087+0000 7f1237392740 20 client.16902 populate_metadata read hostname 'smithi100' 2024-07-26T13:45:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.087+0000 7f1237392740 1 --2- 172.21.15.100:0/2488592856 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55b99ccb21c0 0x55b99d147a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.087+0000 7f1237392740 4 client.16902 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-26T13:45:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.087+0000 7f1237392740 1 -- 172.21.15.100:0/2488592856 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b99d033da0 con 0x55b99ccb21c0 2024-07-26T13:45:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.087+0000 7f122e13e640 1 --2- 172.21.15.100:0/2488592856 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55b99ccb21c0 0x55b99d147a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.088+0000 7f1211ffb640 10 client.16902 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.088+0000 7f122e13e640 1 --2- 172.21.15.100:0/2488592856 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55b99ccb21c0 0x55b99d147a30 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d955eabfcca96 server_cookie=5782c0ad3465db04 in_seq=0 out_seq=0 2024-07-26T13:45:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.123+0000 7f1211ffb640 1 -- 172.21.15.100:0/2488592856 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b99d033da0 con 0x55b99ccb21c0 2024-07-26T13:45:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.123+0000 7f1211ffb640 10 client.16902 handle_command_reply: tid=0 2024-07-26T13:45:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.123+0000 7f1237392740 1 client.16902 shutdown 2024-07-26T13:45:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.123+0000 7f1237392740 1 -- 172.21.15.100:0/2488592856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b99ccfd4d0 msgr2=0x55b99d0d1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.123+0000 7f1237392740 1 --2- 172.21.15.100:0/2488592856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b99ccfd4d0 0x55b99d0d1020 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f1224005130 tx=0x7f122402a410 comp rx=0 tx=0).stop 2024-07-26T13:45:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.123+0000 7f1237392740 1 -- 172.21.15.100:0/2488592856 shutdown_connections 2024-07-26T13:45:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.123+0000 7f1237392740 1 --2- 172.21.15.100:0/2488592856 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55b99ccb21c0 0x55b99d147a30 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.123+0000 7f1237392740 1 --2- 172.21.15.100:0/2488592856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b99cd1e1b0 0x55b99d0d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.123+0000 7f1237392740 1 --2- 172.21.15.100:0/2488592856 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b99ccfd4d0 0x55b99d0d1020 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.123+0000 7f1211ffb640 0 client.16902 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.123+0000 7f1237392740 1 --2- 172.21.15.100:0/2488592856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b99ccf52f0 0x55b99d0d4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.123+0000 7f1237392740 1 -- 172.21.15.100:0/2488592856 >> 172.21.15.100:0/2488592856 conn(0x55b99cd12300 msgr2=0x55b99d0cd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.124+0000 7f1237392740 1 -- 172.21.15.100:0/2488592856 shutdown_connections 2024-07-26T13:45:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.124+0000 7f1237392740 1 -- 172.21.15.100:0/2488592856 wait complete. 2024-07-26T13:45:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.124+0000 7f1237392740 20 client.16902 trim_cache size 0 max 16384 2024-07-26T13:45:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-07-26T13:45:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-07-26T13:45:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-07-26T13:45:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-07-26T13:45:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.124+0000 7f12303a2640 1 -- 172.21.15.100:0/3511915177 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f12140b3d70 msgr2=0x7f12140b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.124+0000 7f12303a2640 1 --2- 172.21.15.100:0/3511915177 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f12140b3d70 0x7f12140b6280 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1228104470 tx=0x7f1218005f90 comp rx=0 tx=0).stop 2024-07-26T13:45:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.124+0000 7f12303a2640 1 -- 172.21.15.100:0/3511915177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12281097b0 msgr2=0x7f12281034a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.124+0000 7f12303a2640 1 --2- 172.21.15.100:0/3511915177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12281097b0 0x7f12281034a0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f1220009fd0 tx=0x7f122000ef10 comp rx=0 tx=0).stop 2024-07-26T13:45:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.124+0000 7f12303a2640 1 -- 172.21.15.100:0/3511915177 shutdown_connections 2024-07-26T13:45:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.124+0000 7f12303a2640 1 --2- 172.21.15.100:0/3511915177 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f12140b3d70 0x7f12140b6280 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.124+0000 7f12303a2640 1 --2- 172.21.15.100:0/3511915177 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12281097b0 0x7f12281034a0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.124+0000 7f12303a2640 1 --2- 172.21.15.100:0/3511915177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1228108e60 0x7f1228102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.124+0000 7f12303a2640 1 --2- 172.21.15.100:0/3511915177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1228101a90 0x7f1228102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.124+0000 7f12303a2640 1 -- 172.21.15.100:0/3511915177 >> 172.21.15.100:0/3511915177 conn(0x7f12280798c0 msgr2=0x7f1228106f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.124+0000 7f12303a2640 1 -- 172.21.15.100:0/3511915177 shutdown_connections 2024-07-26T13:45:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:35.124+0000 7f12303a2640 1 -- 172.21.15.100:0/3511915177 wait complete. 2024-07-26T13:45:35.156 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b3b1b2d8', total seconds elapsed: 15.9 2024-07-26T13:45:35.417 INFO:tasks.workunit.client.0.smithi100.stdout: 1 195397 2.45 MB/sec execute 109 sec latency 15.577 ms 2024-07-26T13:45:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:35 smithi100 ceph-mon[32599]: pgmap v530: 97 pgs: 97 active+clean; 81 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 878 KiB/s wr, 114 op/s 2024-07-26T13:45:35.945 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:35 smithi138 ceph-mon[36954]: pgmap v530: 97 pgs: 97 active+clean; 81 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 878 KiB/s wr, 114 op/s 2024-07-26T13:45:36.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:35 smithi167 ceph-mon[37140]: pgmap v530: 97 pgs: 97 active+clean; 81 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 878 KiB/s wr, 114 op/s 2024-07-26T13:45:36.417 INFO:tasks.workunit.client.0.smithi100.stdout: 1 197280 2.49 MB/sec execute 110 sec latency 19.052 ms 2024-07-26T13:45:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:36 smithi138 ceph-mon[36954]: pgmap v531: 97 pgs: 97 active+clean; 82 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 963 KiB/s wr, 89 op/s 2024-07-26T13:45:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:36 smithi100 ceph-mon[32599]: pgmap v531: 97 pgs: 97 active+clean; 82 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 963 KiB/s wr, 89 op/s 2024-07-26T13:45:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:36 smithi167 ceph-mon[37140]: pgmap v531: 97 pgs: 97 active+clean; 82 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 963 KiB/s wr, 89 op/s 2024-07-26T13:45:37.417 INFO:tasks.workunit.client.0.smithi100.stdout: 1 199095 2.58 MB/sec execute 111 sec latency 10.233 ms 2024-07-26T13:45:38.418 INFO:tasks.workunit.client.0.smithi100.stdout: 1 201080 2.62 MB/sec execute 112 sec latency 34.102 ms 2024-07-26T13:45:39.418 INFO:tasks.workunit.client.0.smithi100.stdout: 1 202778 2.71 MB/sec execute 113 sec latency 14.130 ms 2024-07-26T13:45:39.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:39 smithi138 ceph-mon[36954]: pgmap v532: 97 pgs: 97 active+clean; 78 MiB data, 929 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 101 op/s 2024-07-26T13:45:39.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:39 smithi100 ceph-mon[32599]: pgmap v532: 97 pgs: 97 active+clean; 78 MiB data, 929 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 101 op/s 2024-07-26T13:45:40.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:39 smithi167 ceph-mon[37140]: pgmap v532: 97 pgs: 97 active+clean; 78 MiB data, 929 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 101 op/s 2024-07-26T13:45:40.418 INFO:tasks.workunit.client.0.smithi100.stdout: 1 205174 2.79 MB/sec execute 114 sec latency 23.167 ms 2024-07-26T13:45:41.418 INFO:tasks.workunit.client.0.smithi100.stdout: 1 206893 2.84 MB/sec execute 115 sec latency 11.788 ms 2024-07-26T13:45:41.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:41 smithi138 ceph-mon[36954]: pgmap v533: 97 pgs: 97 active+clean; 78 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 112 op/s 2024-07-26T13:45:41.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:41 smithi100 ceph-mon[32599]: pgmap v533: 97 pgs: 97 active+clean; 78 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 112 op/s 2024-07-26T13:45:42.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:41 smithi167 ceph-mon[37140]: pgmap v533: 97 pgs: 97 active+clean; 78 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 112 op/s 2024-07-26T13:45:42.418 INFO:tasks.workunit.client.0.smithi100.stdout: 1 209077 2.93 MB/sec execute 116 sec latency 22.088 ms 2024-07-26T13:45:43.418 INFO:tasks.workunit.client.0.smithi100.stdout: 1 210688 2.98 MB/sec execute 117 sec latency 18.068 ms 2024-07-26T13:45:43.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:43 smithi138 ceph-mon[36954]: pgmap v534: 97 pgs: 97 active+clean; 84 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 190 op/s 2024-07-26T13:45:43.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:43 smithi100 ceph-mon[32599]: pgmap v534: 97 pgs: 97 active+clean; 84 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 190 op/s 2024-07-26T13:45:44.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:43 smithi167 ceph-mon[37140]: pgmap v534: 97 pgs: 97 active+clean; 84 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 190 op/s 2024-07-26T13:45:44.418 INFO:tasks.workunit.client.0.smithi100.stdout: 1 212287 3.02 MB/sec execute 118 sec latency 15.275 ms 2024-07-26T13:45:44.825 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:44 smithi100 ceph-mon[32599]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:45:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:44 smithi138 ceph-mon[36954]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:45:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:44 smithi167 ceph-mon[37140]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:45:45.418 INFO:tasks.workunit.client.0.smithi100.stdout: 1 213999 3.07 MB/sec execute 119 sec latency 10.754 ms 2024-07-26T13:45:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:45 smithi100 ceph-mon[32599]: pgmap v535: 97 pgs: 97 active+clean; 84 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 220 op/s 2024-07-26T13:45:45.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:45 smithi138 ceph-mon[36954]: pgmap v535: 97 pgs: 97 active+clean; 84 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 220 op/s 2024-07-26T13:45:46.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:45 smithi167 ceph-mon[37140]: pgmap v535: 97 pgs: 97 active+clean; 84 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 220 op/s 2024-07-26T13:45:46.418 INFO:tasks.workunit.client.0.smithi100.stdout: 1 216406 3.17 MB/sec execute 120 sec latency 22.512 ms 2024-07-26T13:45:47.418 INFO:tasks.workunit.client.0.smithi100.stdout: 1 217706 3.18 MB/sec execute 121 sec latency 71.098 ms 2024-07-26T13:45:47.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:47 smithi138 ceph-mon[36954]: pgmap v536: 97 pgs: 97 active+clean; 86 MiB data, 968 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.4 MiB/s wr, 241 op/s 2024-07-26T13:45:47.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:47 smithi100 ceph-mon[32599]: pgmap v536: 97 pgs: 97 active+clean; 86 MiB data, 968 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.4 MiB/s wr, 241 op/s 2024-07-26T13:45:48.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:47 smithi167 ceph-mon[37140]: pgmap v536: 97 pgs: 97 active+clean; 86 MiB data, 968 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.4 MiB/s wr, 241 op/s 2024-07-26T13:45:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.238+0000 7fc9d8e39640 1 -- 172.21.15.100:0/2064681527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9d41086a0 msgr2=0x7fc9d410eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.238+0000 7fc9d8e39640 1 --2- 172.21.15.100:0/2064681527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9d41086a0 0x7fc9d410eed0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc009ca0 tx=0x7fc9cc02a8c0 comp rx=0 tx=0).stop 2024-07-26T13:45:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.238+0000 7fc9d8e39640 1 -- 172.21.15.100:0/2064681527 shutdown_connections 2024-07-26T13:45:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.238+0000 7fc9d8e39640 1 --2- 172.21.15.100:0/2064681527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9d41086a0 0x7fc9d410eed0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.238+0000 7fc9d8e39640 1 --2- 172.21.15.100:0/2064681527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc9d4107d50 0x7fc9d4108160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.238+0000 7fc9d8e39640 1 --2- 172.21.15.100:0/2064681527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9d4102b90 0x7fc9d4102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.238+0000 7fc9d8e39640 1 -- 172.21.15.100:0/2064681527 >> 172.21.15.100:0/2064681527 conn(0x7fc9d40798b0 msgr2=0x7fc9d4079cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.238+0000 7fc9d8e39640 1 -- 172.21.15.100:0/2064681527 shutdown_connections 2024-07-26T13:45:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.238+0000 7fc9d8e39640 1 -- 172.21.15.100:0/2064681527 wait complete. 2024-07-26T13:45:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.238+0000 7fc9d8e39640 1 Processor -- start 2024-07-26T13:45:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.239+0000 7fc9d8e39640 1 -- start start 2024-07-26T13:45:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.239+0000 7fc9d8e39640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc9d4102b90 0x7fc9d4198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.239+0000 7fc9d8e39640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9d4107d50 0x7fc9d41993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.239+0000 7fc9d8e39640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9d41086a0 0x7fc9d419e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.239+0000 7fc9d8e39640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d4112370 con 0x7fc9d41086a0 2024-07-26T13:45:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.239+0000 7fc9d8e39640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d41121f0 con 0x7fc9d4107d50 2024-07-26T13:45:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.239+0000 7fc9d8e39640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d41124f0 con 0x7fc9d4102b90 2024-07-26T13:45:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.239+0000 7fc9d2d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9d41086a0 0x7fc9d419e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.239+0000 7fc9d2d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9d41086a0 0x7fc9d419e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47388/0 (socket says 172.21.15.100:47388) 2024-07-26T13:45:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.239+0000 7fc9d2d9d640 1 -- 172.21.15.100:0/1944194821 learned_addr learned my addr 172.21.15.100:0/1944194821 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.239+0000 7fc9d259c640 1 --2- 172.21.15.100:0/1944194821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc9d4102b90 0x7fc9d4198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.239+0000 7fc9c9d9b640 1 --2- 172.21.15.100:0/1944194821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9d4107d50 0x7fc9d41993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.239+0000 7fc9d2d9d640 1 -- 172.21.15.100:0/1944194821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc9d4102b90 msgr2=0x7fc9d4198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.239+0000 7fc9d2d9d640 1 --2- 172.21.15.100:0/1944194821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc9d4102b90 0x7fc9d4198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.239+0000 7fc9d2d9d640 1 -- 172.21.15.100:0/1944194821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9d4107d50 msgr2=0x7fc9d41993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.239+0000 7fc9d2d9d640 1 --2- 172.21.15.100:0/1944194821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9d4107d50 0x7fc9d41993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.239+0000 7fc9d2d9d640 1 -- 172.21.15.100:0/1944194821 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d419ed80 con 0x7fc9d41086a0 2024-07-26T13:45:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.240+0000 7fc9d2d9d640 1 --2- 172.21.15.100:0/1944194821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9d41086a0 0x7fc9d419e680 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc005cc0 tx=0x7fc9cc031680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.240+0000 7fc9cbfff640 1 -- 172.21.15.100:0/1944194821 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc04a070 con 0x7fc9d41086a0 2024-07-26T13:45:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.240+0000 7fc9cbfff640 1 -- 172.21.15.100:0/1944194821 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9cc003e20 con 0x7fc9d41086a0 2024-07-26T13:45:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.240+0000 7fc9d8e39640 1 -- 172.21.15.100:0/1944194821 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9d419f010 con 0x7fc9d41086a0 2024-07-26T13:45:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.240+0000 7fc9cbfff640 1 -- 172.21.15.100:0/1944194821 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc004860 con 0x7fc9d41086a0 2024-07-26T13:45:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.240+0000 7fc9d8e39640 1 -- 172.21.15.100:0/1944194821 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9d41017e0 con 0x7fc9d41086a0 2024-07-26T13:45:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.243+0000 7fc9dfe2a740 1 Processor -- start 2024-07-26T13:45:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.243+0000 7fc9cbfff640 1 -- 172.21.15.100:0/1944194821 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fc9cc04d070 con 0x7fc9d41086a0 2024-07-26T13:45:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.243+0000 7fc9dfe2a740 1 -- start start 2024-07-26T13:45:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.243+0000 7fc9dfe2a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f711a21b0 0x556f71544ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.243+0000 7fc9dfe2a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f711792f0 0x556f71545220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.243+0000 7fc9dfe2a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f711814d0 0x556f71545760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.243+0000 7fc9dfe2a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x556f714ba3b0 con 0x556f711a21b0 2024-07-26T13:45:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.243+0000 7fc9dfe2a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556f714a98b0 con 0x556f711792f0 2024-07-26T13:45:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.243+0000 7fc9dfe2a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556f714ad8f0 con 0x556f711814d0 2024-07-26T13:45:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.243+0000 7fc9c9d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f711a21b0 0x556f71544ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.243+0000 7fc9d259c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f711792f0 0x556f71545220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.243+0000 7fc9c9d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f711a21b0 0x556f71544ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47394/0 (socket says 172.21.15.100:47394) 2024-07-26T13:45:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.243+0000 7fc9c9d9b640 1 -- 172.21.15.100:0/982434044 learned_addr learned my addr 172.21.15.100:0/982434044 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.243+0000 7fc9c9d9b640 1 --2- 172.21.15.100:0/982434044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f711814d0 0x556f71545760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.244+0000 7fc9cbfff640 1 --2- 172.21.15.100:0/1944194821 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc9c00b3d40 0x7fc9c00b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.244+0000 7fc9d259c640 1 -- 172.21.15.100:0/982434044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f711814d0 msgr2=0x556f71545760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.244+0000 7fc9d259c640 1 --2- 172.21.15.100:0/982434044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f711814d0 0x556f71545760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.244+0000 7fc9d259c640 1 -- 172.21.15.100:0/982434044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f711a21b0 msgr2=0x556f71544ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.244+0000 7fc9d259c640 1 --2- 172.21.15.100:0/982434044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f711a21b0 0x556f71544ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.244+0000 7fc9d259c640 1 -- 172.21.15.100:0/982434044 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f714509a0 con 0x556f711792f0 2024-07-26T13:45:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.244+0000 7fc9d2d9d640 1 --2- 172.21.15.100:0/1944194821 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc9c00b3d40 0x7fc9c00b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.244+0000 7fc9d259c640 1 --2- 172.21.15.100:0/982434044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f711792f0 0x556f71545220 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4009c50 tx=0x7fc9c4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f31d7c57dd2ac4f3 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.244+0000 7fc9cbfff640 1 -- 172.21.15.100:0/1944194821 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7fc9cc106470 con 0x7fc9d41086a0 2024-07-26T13:45:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.244+0000 7fc9c8d99640 1 -- 172.21.15.100:0/982434044 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4004050 con 0x556f711792f0 2024-07-26T13:45:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.244+0000 7fc9c8d99640 1 -- 172.21.15.100:0/982434044 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9c40054d0 con 0x556f711792f0 2024-07-26T13:45:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.244+0000 7fc9c8d99640 1 -- 172.21.15.100:0/982434044 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4004cd0 con 0x556f711792f0 2024-07-26T13:45:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.245+0000 7fc9dfe2a740 1 -- 172.21.15.100:0/982434044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f711792f0 msgr2=0x556f71545220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.245+0000 7fc9d2d9d640 1 --2- 172.21.15.100:0/1944194821 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc9c00b3d40 0x7fc9c00b6250 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc0ca430 tx=0x7fc9cc0cd070 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.245+0000 7fc9dfe2a740 1 --2- 172.21.15.100:0/982434044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f711792f0 0x556f71545220 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4009c50 tx=0x7fc9c4011810 comp rx=0 tx=0).stop 2024-07-26T13:45:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.245+0000 7fc9dfe2a740 1 -- 172.21.15.100:0/982434044 shutdown_connections 2024-07-26T13:45:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.245+0000 7fc9dfe2a740 1 --2- 172.21.15.100:0/982434044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f711814d0 0x556f71545760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.245+0000 7fc9dfe2a740 1 --2- 172.21.15.100:0/982434044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f711792f0 0x556f71545220 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.245+0000 7fc9dfe2a740 1 --2- 172.21.15.100:0/982434044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f711a21b0 0x556f71544ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.245+0000 7fc9dfe2a740 1 -- 172.21.15.100:0/982434044 >> 172.21.15.100:0/982434044 conn(0x556f71196300 msgr2=0x556f71542bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.245+0000 7fc9dfe2a740 1 -- 172.21.15.100:0/982434044 shutdown_connections 2024-07-26T13:45:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.245+0000 7fc9dfe2a740 1 -- 172.21.15.100:0/982434044 wait complete. 2024-07-26T13:45:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.246+0000 7fc9dfe2a740 1 -- start start 2024-07-26T13:45:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.246+0000 7fc9dfe2a740 1 Processor -- start 2024-07-26T13:45:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.246+0000 7fc9dfe2a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f711792f0 0x556f715508a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.246+0000 7fc9dfe2a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f711814d0 0x556f71550de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.246+0000 7fc9dfe2a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f711a21b0 0x556f7155c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.246+0000 7fc9dfe2a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x556f713e3ab0 con 0x556f711814d0 2024-07-26T13:45:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.246+0000 7fc9dfe2a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556f71431cf0 con 0x556f711a21b0 2024-07-26T13:45:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.246+0000 7fc9dfe2a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556f71553460 con 0x556f711792f0 2024-07-26T13:45:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.246+0000 7fc9dfe2a740 10 client.0 _ll_register_callbacks cb 0x556f714a39f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:45:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.246+0000 7fc9d259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f711814d0 0x556f71550de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.246+0000 7fc9c9d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f711792f0 0x556f715508a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.246+0000 7fc9d259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f711814d0 0x556f71550de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47410/0 (socket says 172.21.15.100:47410) 2024-07-26T13:45:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.246+0000 7fc9d259c640 1 -- 172.21.15.100:0/134759326 learned_addr learned my addr 172.21.15.100:0/134759326 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.246+0000 7fc9c9d9b640 1 --2- 172.21.15.100:0/134759326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f711a21b0 0x556f7155c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.246+0000 7fc9d259c640 1 -- 172.21.15.100:0/134759326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f711792f0 msgr2=0x556f715508a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.246+0000 7fc9d259c640 1 --2- 172.21.15.100:0/134759326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f711792f0 0x556f715508a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.246+0000 7fc9d259c640 1 -- 172.21.15.100:0/134759326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f711a21b0 msgr2=0x556f7155c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.246+0000 7fc9d259c640 1 --2- 172.21.15.100:0/134759326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f711a21b0 0x556f7155c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.246+0000 7fc9d259c640 1 -- 172.21.15.100:0/134759326 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f7120d790 con 0x556f711814d0 2024-07-26T13:45:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.247+0000 7fc9d259c640 1 --2- 172.21.15.100:0/134759326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f711814d0 0x556f71550de0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4000c00 tx=0x7fc9c4005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4f038b91a9c657 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.247+0000 7fc9b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:45:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.247+0000 7fc9b9ffb640 1 -- 172.21.15.100:0/134759326 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4027070 con 0x556f711814d0 2024-07-26T13:45:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.247+0000 7fc9b9ffb640 1 -- 172.21.15.100:0/134759326 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7fc9c4003a00 con 0x556f711814d0 2024-07-26T13:45:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.247+0000 7fc9dfe2a740 1 -- 172.21.15.100:0/134759326 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9c40054d0 con 0x556f711814d0 2024-07-26T13:45:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.247+0000 7fc9b9ffb640 1 -- 172.21.15.100:0/134759326 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fc9c4033050 con 0x556f711814d0 2024-07-26T13:45:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.247+0000 7fc9dfe2a740 10 client.16914 fetch_fsmap learned FSMap version 47 2024-07-26T13:45:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.247+0000 7fc9dfe2a740 1 -- 172.21.15.100:0/134759326 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x556f71286620 con 0x556f711814d0 2024-07-26T13:45:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.247+0000 7fc9b9ffb640 1 -- 172.21.15.100:0/134759326 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9c40054d0 con 0x556f711814d0 2024-07-26T13:45:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.247+0000 7fc9b9ffb640 1 -- 172.21.15.100:0/134759326 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 47) ==== 8350+0+0 (secure 0 0 0) 0x7fc9c40051b0 con 0x556f711814d0 2024-07-26T13:45:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.247+0000 7fc9dfe2a740 10 client.16914 fetch_fsmap finished waiting for FSMap version 47 2024-07-26T13:45:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.247+0000 7fc9dfe2a740 10 client.16914 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:45:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.247+0000 7fc9dfe2a740 20 client.16914 populate_metadata read hostname 'smithi100' 2024-07-26T13:45:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.247+0000 7fc9dfe2a740 1 --2- 172.21.15.100:0/134759326 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x556f71160d80 0x556f71562650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.247+0000 7fc9dfe2a740 4 client.16914 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:45:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.247+0000 7fc9dfe2a740 1 -- 172.21.15.100:0/134759326 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f714bc300 con 0x556f71160d80 2024-07-26T13:45:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.247+0000 7fc9c9d9b640 1 --2- 172.21.15.100:0/134759326 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x556f71160d80 0x556f71562650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.248+0000 7fc9b9ffb640 10 client.16914 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.248+0000 7fc9c9d9b640 1 --2- 172.21.15.100:0/134759326 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x556f71160d80 0x556f71562650 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5780fde04d34a4c0 server_cookie=56df62606beb311a in_seq=0 out_seq=0 2024-07-26T13:45:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.249+0000 7fc9b9ffb640 1 -- 172.21.15.100:0/134759326 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556f714bc300 con 0x556f71160d80 2024-07-26T13:45:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.249+0000 7fc9b9ffb640 10 client.16914 handle_command_reply: tid=0 2024-07-26T13:45:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.249+0000 7fc9dfe2a740 1 client.16914 shutdown 2024-07-26T13:45:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.249+0000 7fc9dfe2a740 1 -- 172.21.15.100:0/134759326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f711814d0 msgr2=0x556f71550de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.249+0000 7fc9dfe2a740 1 --2- 172.21.15.100:0/134759326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f711814d0 0x556f71550de0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4000c00 tx=0x7fc9c4005760 comp rx=0 tx=0).stop 2024-07-26T13:45:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.249+0000 7fc9dfe2a740 1 -- 172.21.15.100:0/134759326 shutdown_connections 2024-07-26T13:45:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.249+0000 7fc9dfe2a740 1 --2- 172.21.15.100:0/134759326 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x556f71160d80 0x556f71562650 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.249+0000 7fc9dfe2a740 1 --2- 172.21.15.100:0/134759326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f711a21b0 0x556f7155c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.249+0000 7fc9dfe2a740 1 --2- 172.21.15.100:0/134759326 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f711814d0 0x556f71550de0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.249+0000 7fc9b9ffb640 0 client.16914 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.249+0000 7fc9dfe2a740 1 --2- 172.21.15.100:0/134759326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f711792f0 0x556f715508a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.249+0000 7fc9dfe2a740 1 -- 172.21.15.100:0/134759326 >> 172.21.15.100:0/134759326 conn(0x556f71196300 msgr2=0x556f71545d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.249+0000 7fc9dfe2a740 1 -- 172.21.15.100:0/134759326 shutdown_connections 2024-07-26T13:45:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.249+0000 7fc9dfe2a740 1 -- 172.21.15.100:0/134759326 wait complete. 2024-07-26T13:45:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.249+0000 7fc9dfe2a740 20 client.16914 trim_cache size 0 max 16384 2024-07-26T13:45:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.264+0000 7fc9dfe2a740 1 Processor -- start 2024-07-26T13:45:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.264+0000 7fc9dfe2a740 1 -- start start 2024-07-26T13:45:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.265+0000 7fc9dfe2a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f711792f0 0x556f71546380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.265+0000 7fc9dfe2a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f711814d0 0x556f71542c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.265+0000 7fc9dfe2a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f711a21b0 0x556f7154ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.265+0000 7fc9dfe2a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d41af950 con 0x556f711792f0 2024-07-26T13:45:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.265+0000 7fc9dfe2a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d41ad1f0 con 0x556f711a21b0 2024-07-26T13:45:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.265+0000 7fc9dfe2a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d41ab020 con 0x556f711814d0 2024-07-26T13:45:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.265+0000 7fc9c9d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f711792f0 0x556f71546380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.265+0000 7fc9c9d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f711792f0 0x556f71546380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47426/0 (socket says 172.21.15.100:47426) 2024-07-26T13:45:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.265+0000 7fc9c9d9b640 1 -- 172.21.15.100:0/2393087478 learned_addr learned my addr 172.21.15.100:0/2393087478 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.265+0000 7fc9d259c640 1 --2- 172.21.15.100:0/2393087478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f711814d0 0x556f71542c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.265+0000 7fc9c9d9b640 1 --2- 172.21.15.100:0/2393087478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f711a21b0 0x556f7154ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.265+0000 7fc9c9d9b640 1 -- 172.21.15.100:0/2393087478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f711814d0 msgr2=0x556f71542c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.265+0000 7fc9c9d9b640 1 --2- 172.21.15.100:0/2393087478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f711814d0 0x556f71542c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.265+0000 7fc9c9d9b640 1 -- 172.21.15.100:0/2393087478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f711a21b0 msgr2=0x556f7154ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.265+0000 7fc9c9d9b640 1 --2- 172.21.15.100:0/2393087478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f711a21b0 0x556f7154ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.265+0000 7fc9c9d9b640 1 -- 172.21.15.100:0/2393087478 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f71543940 con 0x556f711792f0 2024-07-26T13:45:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.265+0000 7fc9c9d9b640 1 --2- 172.21.15.100:0/2393087478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f711792f0 0x556f71546380 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc005f60 tx=0x7fc9bc0096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41712117cea3e104 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.265+0000 7fc9bbfff640 1 -- 172.21.15.100:0/2393087478 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc01c040 con 0x556f711792f0 2024-07-26T13:45:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.265+0000 7fc9bbfff640 1 -- 172.21.15.100:0/2393087478 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9bc0248c0 con 0x556f711792f0 2024-07-26T13:45:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.265+0000 7fc9bbfff640 1 -- 172.21.15.100:0/2393087478 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc02d7f0 con 0x556f711792f0 2024-07-26T13:45:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.266+0000 7fc9dfe2a740 1 -- 172.21.15.100:0/2393087478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f711792f0 msgr2=0x556f71546380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.266+0000 7fc9dfe2a740 1 --2- 172.21.15.100:0/2393087478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f711792f0 0x556f71546380 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc005f60 tx=0x7fc9bc0096a0 comp rx=0 tx=0).stop 2024-07-26T13:45:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.266+0000 7fc9dfe2a740 1 -- 172.21.15.100:0/2393087478 shutdown_connections 2024-07-26T13:45:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.266+0000 7fc9dfe2a740 1 --2- 172.21.15.100:0/2393087478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f711a21b0 0x556f7154ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.266+0000 7fc9dfe2a740 1 --2- 172.21.15.100:0/2393087478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f711814d0 0x556f71542c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.266+0000 7fc9dfe2a740 1 --2- 172.21.15.100:0/2393087478 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f711792f0 0x556f71546380 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.266+0000 7fc9dfe2a740 1 -- 172.21.15.100:0/2393087478 >> 172.21.15.100:0/2393087478 conn(0x556f71196300 msgr2=0x556f715424f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.266+0000 7fc9dfe2a740 1 -- 172.21.15.100:0/2393087478 shutdown_connections 2024-07-26T13:45:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.266+0000 7fc9dfe2a740 1 -- 172.21.15.100:0/2393087478 wait complete. 2024-07-26T13:45:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.267+0000 7fc9dfe2a740 1 -- start start 2024-07-26T13:45:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.267+0000 7fc9dfe2a740 1 Processor -- start 2024-07-26T13:45:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.267+0000 7fc9dfe2a740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f711792f0 0x556f71558a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.267+0000 7fc9dfe2a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f711814d0 0x556f71558f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.267+0000 7fc9dfe2a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f711a21b0 0x556f715557e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.267+0000 7fc9dfe2a740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d41ab6b0 con 0x556f711792f0 2024-07-26T13:45:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.267+0000 7fc9dfe2a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d41acb20 con 0x556f711814d0 2024-07-26T13:45:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.267+0000 7fc9dfe2a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d41abd80 con 0x556f711a21b0 2024-07-26T13:45:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.267+0000 7fc9dfe2a740 10 client.0 _ll_register_callbacks cb 0x7fc9bc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:45:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.267+0000 7fc9c9d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f711792f0 0x556f71558a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.267+0000 7fc9c9d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f711792f0 0x556f71558a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47442/0 (socket says 172.21.15.100:47442) 2024-07-26T13:45:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.267+0000 7fc9c9d9b640 1 -- 172.21.15.100:0/3410791426 learned_addr learned my addr 172.21.15.100:0/3410791426 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.267+0000 7fc9d259c640 1 --2- 172.21.15.100:0/3410791426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f711814d0 0x556f71558f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.267+0000 7fc9c9d9b640 1 --2- 172.21.15.100:0/3410791426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f711a21b0 0x556f715557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.267+0000 7fc9c9d9b640 1 -- 172.21.15.100:0/3410791426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f711a21b0 msgr2=0x556f715557e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.267+0000 7fc9c9d9b640 1 --2- 172.21.15.100:0/3410791426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f711a21b0 0x556f715557e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.267+0000 7fc9c9d9b640 1 -- 172.21.15.100:0/3410791426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f711814d0 msgr2=0x556f71558f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.267+0000 7fc9c9d9b640 1 --2- 172.21.15.100:0/3410791426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f711814d0 0x556f71558f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.267+0000 7fc9c9d9b640 1 -- 172.21.15.100:0/3410791426 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f71555f70 con 0x556f711792f0 2024-07-26T13:45:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.268+0000 7fc9c9d9b640 1 --2- 172.21.15.100:0/3410791426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f711792f0 0x556f71558a00 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc009670 tx=0x7fc9bc009d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557ed34f9ccc21c6 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.268+0000 7fc9b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:45:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.268+0000 7fc9b9ffb640 1 -- 172.21.15.100:0/3410791426 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc024590 con 0x556f711792f0 2024-07-26T13:45:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.268+0000 7fc9b9ffb640 1 -- 172.21.15.100:0/3410791426 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7fc9bc024a70 con 0x556f711792f0 2024-07-26T13:45:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.268+0000 7fc9dfe2a740 1 -- 172.21.15.100:0/3410791426 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9bc0248c0 con 0x556f711792f0 2024-07-26T13:45:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.268+0000 7fc9b9ffb640 1 -- 172.21.15.100:0/3410791426 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fc9bc024c70 con 0x556f711792f0 2024-07-26T13:45:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.268+0000 7fc9dfe2a740 10 client.16920 fetch_fsmap learned FSMap version 47 2024-07-26T13:45:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.268+0000 7fc9dfe2a740 1 -- 172.21.15.100:0/3410791426 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x556f715c9840 con 0x556f711792f0 2024-07-26T13:45:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.268+0000 7fc9b9ffb640 1 -- 172.21.15.100:0/3410791426 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9bc0248c0 con 0x556f711792f0 2024-07-26T13:45:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.269+0000 7fc9b9ffb640 1 -- 172.21.15.100:0/3410791426 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 47) ==== 8350+0+0 (secure 0 0 0) 0x7fc9bc003890 con 0x556f711792f0 2024-07-26T13:45:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.269+0000 7fc9dfe2a740 10 client.16920 fetch_fsmap finished waiting for FSMap version 47 2024-07-26T13:45:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.269+0000 7fc9dfe2a740 10 client.16920 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:45:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.269+0000 7fc9dfe2a740 20 client.16920 populate_metadata read hostname 'smithi100' 2024-07-26T13:45:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.269+0000 7fc9dfe2a740 1 --2- 172.21.15.100:0/3410791426 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x556f711361c0 0x556f715cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.269+0000 7fc9dfe2a740 4 client.16920 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-26T13:45:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.269+0000 7fc9dfe2a740 1 -- 172.21.15.100:0/3410791426 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556f714b7da0 con 0x556f711361c0 2024-07-26T13:45:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.269+0000 7fc9d259c640 1 --2- 172.21.15.100:0/3410791426 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x556f711361c0 0x556f715cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.269+0000 7fc9b9ffb640 10 client.16920 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.269+0000 7fc9d259c640 1 --2- 172.21.15.100:0/3410791426 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x556f711361c0 0x556f715cbef0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ce4637e18ef91e4 server_cookie=7f8245f305759ded in_seq=0 out_seq=0 2024-07-26T13:45:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.314+0000 7fc9b9ffb640 1 -- 172.21.15.100:0/3410791426 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556f714b7da0 con 0x556f711361c0 2024-07-26T13:45:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.314+0000 7fc9b9ffb640 10 client.16920 handle_command_reply: tid=0 2024-07-26T13:45:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.314+0000 7fc9dfe2a740 1 client.16920 shutdown 2024-07-26T13:45:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.314+0000 7fc9dfe2a740 1 -- 172.21.15.100:0/3410791426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f711792f0 msgr2=0x556f71558a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.314+0000 7fc9dfe2a740 1 --2- 172.21.15.100:0/3410791426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f711792f0 0x556f71558a00 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc009670 tx=0x7fc9bc009d70 comp rx=0 tx=0).stop 2024-07-26T13:45:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.314+0000 7fc9dfe2a740 1 -- 172.21.15.100:0/3410791426 shutdown_connections 2024-07-26T13:45:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.314+0000 7fc9dfe2a740 1 --2- 172.21.15.100:0/3410791426 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x556f711361c0 0x556f715cbef0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.314+0000 7fc9dfe2a740 1 --2- 172.21.15.100:0/3410791426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556f711a21b0 0x556f715557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.314+0000 7fc9dfe2a740 1 --2- 172.21.15.100:0/3410791426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f711814d0 0x556f71558f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.314+0000 7fc9b9ffb640 0 client.16920 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.314+0000 7fc9dfe2a740 1 --2- 172.21.15.100:0/3410791426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f711792f0 0x556f71558a00 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.314+0000 7fc9dfe2a740 1 -- 172.21.15.100:0/3410791426 >> 172.21.15.100:0/3410791426 conn(0x556f71196300 msgr2=0x556f71551af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:48.317 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.314+0000 7fc9dfe2a740 1 -- 172.21.15.100:0/3410791426 shutdown_connections 2024-07-26T13:45:48.317 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.314+0000 7fc9dfe2a740 1 -- 172.21.15.100:0/3410791426 wait complete. 2024-07-26T13:45:48.317 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.314+0000 7fc9dfe2a740 20 client.16920 trim_cache size 0 max 16384 2024-07-26T13:45:48.317 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-07-26T13:45:48.317 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-07-26T13:45:48.317 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-07-26T13:45:48.317 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-07-26T13:45:48.318 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.315+0000 7fc9d8e39640 1 -- 172.21.15.100:0/1944194821 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc9c00b3d40 msgr2=0x7fc9c00b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:48.318 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.315+0000 7fc9d8e39640 1 --2- 172.21.15.100:0/1944194821 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc9c00b3d40 0x7fc9c00b6250 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc0ca430 tx=0x7fc9cc0cd070 comp rx=0 tx=0).stop 2024-07-26T13:45:48.318 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.315+0000 7fc9d8e39640 1 -- 172.21.15.100:0/1944194821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9d41086a0 msgr2=0x7fc9d419e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:48.318 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.315+0000 7fc9d8e39640 1 --2- 172.21.15.100:0/1944194821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9d41086a0 0x7fc9d419e680 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc005cc0 tx=0x7fc9cc031680 comp rx=0 tx=0).stop 2024-07-26T13:45:48.318 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.315+0000 7fc9d8e39640 1 -- 172.21.15.100:0/1944194821 shutdown_connections 2024-07-26T13:45:48.318 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.315+0000 7fc9d8e39640 1 --2- 172.21.15.100:0/1944194821 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc9c00b3d40 0x7fc9c00b6250 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:48.319 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.315+0000 7fc9d8e39640 1 --2- 172.21.15.100:0/1944194821 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc9d41086a0 0x7fc9d419e680 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:48.319 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.315+0000 7fc9d8e39640 1 --2- 172.21.15.100:0/1944194821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9d4107d50 0x7fc9d41993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:48.319 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.315+0000 7fc9d8e39640 1 --2- 172.21.15.100:0/1944194821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc9d4102b90 0x7fc9d4198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:48.319 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.315+0000 7fc9d8e39640 1 -- 172.21.15.100:0/1944194821 >> 172.21.15.100:0/1944194821 conn(0x7fc9d40798b0 msgr2=0x7fc9d4105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:48.319 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.316+0000 7fc9d8e39640 1 -- 172.21.15.100:0/1944194821 shutdown_connections 2024-07-26T13:45:48.319 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:45:48.316+0000 7fc9d8e39640 1 -- 172.21.15.100:0/1944194821 wait complete. 2024-07-26T13:45:48.418 INFO:tasks.workunit.client.0.smithi100.stdout: 1 219200 3.22 MB/sec execute 122 sec latency 33.523 ms 2024-07-26T13:45:49.134 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-26T13:45:49.408 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:45:49.418 INFO:tasks.workunit.client.0.smithi100.stdout: 1 220771 3.27 MB/sec execute 123 sec latency 21.029 ms 2024-07-26T13:45:49.801 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:49 smithi100 ceph-mon[32599]: pgmap v537: 97 pgs: 97 active+clean; 89 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 246 op/s 2024-07-26T13:45:49.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:49 smithi138 ceph-mon[36954]: pgmap v537: 97 pgs: 97 active+clean; 89 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 246 op/s 2024-07-26T13:45:50.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:49 smithi167 ceph-mon[37140]: pgmap v537: 97 pgs: 97 active+clean; 89 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 246 op/s 2024-07-26T13:45:50.418 INFO:tasks.workunit.client.0.smithi100.stdout: 1 222724 3.33 MB/sec execute 124 sec latency 154.975 ms 2024-07-26T13:45:51.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.212+0000 7f4612e3e640 1 -- 172.21.15.100:0/1921468233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f460c101c70 msgr2=0x7f460c102080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:51.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.212+0000 7f4612e3e640 1 --2- 172.21.15.100:0/1921468233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f460c101c70 0x7f460c102080 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f4600009900 tx=0x7f460002a410 comp rx=0 tx=0).stop 2024-07-26T13:45:51.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.212+0000 7f4612e3e640 1 -- 172.21.15.100:0/1921468233 shutdown_connections 2024-07-26T13:45:51.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.212+0000 7f4612e3e640 1 --2- 172.21.15.100:0/1921468233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f460c107f00 0x7f460c10e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:51.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.212+0000 7f4612e3e640 1 --2- 172.21.15.100:0/1921468233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f460c1075b0 0x7f460c1079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:51.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.212+0000 7f4612e3e640 1 --2- 172.21.15.100:0/1921468233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f460c101c70 0x7f460c102080 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:51.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.212+0000 7f4612e3e640 1 -- 172.21.15.100:0/1921468233 >> 172.21.15.100:0/1921468233 conn(0x7f460c0799c0 msgr2=0x7f460c079dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:51.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.212+0000 7f4612e3e640 1 -- 172.21.15.100:0/1921468233 shutdown_connections 2024-07-26T13:45:51.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.212+0000 7f4612e3e640 1 -- 172.21.15.100:0/1921468233 wait complete. 2024-07-26T13:45:51.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.213+0000 7f4612e3e640 1 Processor -- start 2024-07-26T13:45:51.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.213+0000 7f4612e3e640 1 -- start start 2024-07-26T13:45:51.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.213+0000 7f4612e3e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f460c101c70 0x7f460c197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:51.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.214+0000 7f4612e3e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f460c1075b0 0x7f460c198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:51.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.214+0000 7f4612e3e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f460c107f00 0x7f460c19d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:51.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.214+0000 7f4612e3e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f460c111340 con 0x7f460c101c70 2024-07-26T13:45:51.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.214+0000 7f4612e3e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f460c1111c0 con 0x7f460c1075b0 2024-07-26T13:45:51.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.214+0000 7f4612e3e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f460c1114c0 con 0x7f460c107f00 2024-07-26T13:45:51.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.214+0000 7f4610bda640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f460c101c70 0x7f460c197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:51.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.214+0000 7f4610bda640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f460c101c70 0x7f460c197cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47462/0 (socket says 172.21.15.100:47462) 2024-07-26T13:45:51.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.214+0000 7f4610bda640 1 -- 172.21.15.100:0/3732079735 learned_addr learned my addr 172.21.15.100:0/3732079735 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:51.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.214+0000 7f46113db640 1 --2- 172.21.15.100:0/3732079735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f460c107f00 0x7f460c19d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:51.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.214+0000 7f460bfff640 1 --2- 172.21.15.100:0/3732079735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f460c1075b0 0x7f460c198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:51.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.214+0000 7f4610bda640 1 -- 172.21.15.100:0/3732079735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f460c107f00 msgr2=0x7f460c19d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:51.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.214+0000 7f4610bda640 1 --2- 172.21.15.100:0/3732079735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f460c107f00 0x7f460c19d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:51.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.214+0000 7f4610bda640 1 -- 172.21.15.100:0/3732079735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f460c1075b0 msgr2=0x7f460c198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:51.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.214+0000 7f4610bda640 1 --2- 172.21.15.100:0/3732079735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f460c1075b0 0x7f460c198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:51.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.214+0000 7f4610bda640 1 -- 172.21.15.100:0/3732079735 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f460c19dc60 con 0x7f460c101c70 2024-07-26T13:45:51.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.214+0000 7f460bfff640 1 --2- 172.21.15.100:0/3732079735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f460c1075b0 0x7f460c198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:45:51.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.214+0000 7f4610bda640 1 --2- 172.21.15.100:0/3732079735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f460c101c70 0x7f460c197cf0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f4600005940 tx=0x7f4600004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:51.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.215+0000 7f4609ffb640 1 -- 172.21.15.100:0/3732079735 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460003f030 con 0x7f460c101c70 2024-07-26T13:45:51.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.215+0000 7f4612e3e640 1 -- 172.21.15.100:0/3732079735 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f460c19def0 con 0x7f460c101c70 2024-07-26T13:45:51.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.215+0000 7f4609ffb640 1 -- 172.21.15.100:0/3732079735 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4600003aa0 con 0x7f460c101c70 2024-07-26T13:45:51.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.215+0000 7f4609ffb640 1 -- 172.21.15.100:0/3732079735 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4600035070 con 0x7f460c101c70 2024-07-26T13:45:51.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.215+0000 7f4612e3e640 1 -- 172.21.15.100:0/3732079735 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f460c1a46b0 con 0x7f460c101c70 2024-07-26T13:45:51.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.217+0000 7f46193c2740 1 Processor -- start 2024-07-26T13:45:51.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.217+0000 7f46193c2740 1 -- start start 2024-07-26T13:45:51.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.218+0000 7f46193c2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cd5042070 0x557cd53e7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:51.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.218+0000 7f46193c2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557cd50191b0 0x557cd53e7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:51.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.218+0000 7f46193c2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cd5021390 0x557cd53e7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:51.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.218+0000 7f46193c2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x557cd535db60 con 0x557cd5021390 2024-07-26T13:45:51.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.218+0000 7f46193c2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557cd534d060 con 0x557cd50191b0 2024-07-26T13:45:51.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.218+0000 7f46193c2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557cd53510a0 con 0x557cd5042070 2024-07-26T13:45:51.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.218+0000 7f460bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cd5021390 0x557cd53e7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:51.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.218+0000 7f460bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cd5021390 0x557cd53e7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47470/0 (socket says 172.21.15.100:47470) 2024-07-26T13:45:51.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.218+0000 7f460bfff640 1 -- 172.21.15.100:0/2616817790 learned_addr learned my addr 172.21.15.100:0/2616817790 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:51.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.218+0000 7f46113db640 1 --2- 172.21.15.100:0/2616817790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cd5042070 0x557cd53e7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:51.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.218+0000 7f460bfff640 1 -- 172.21.15.100:0/2616817790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cd5042070 msgr2=0x557cd53e7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:51.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.218+0000 7f460bfff640 1 --2- 172.21.15.100:0/2616817790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cd5042070 0x557cd53e7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:51.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.218+0000 7f460bfff640 1 -- 172.21.15.100:0/2616817790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557cd50191b0 msgr2=0x557cd53e7a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-26T13:45:51.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.218+0000 7f460bfff640 1 --2- 172.21.15.100:0/2616817790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557cd50191b0 0x557cd53e7a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:51.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.218+0000 7f460bfff640 1 -- 172.21.15.100:0/2616817790 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cd52f3b40 con 0x557cd5021390 2024-07-26T13:45:51.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.218+0000 7f460bfff640 1 --2- 172.21.15.100:0/2616817790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cd5021390 0x557cd53e7fa0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f45f800ac60 tx=0x7f45f800e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb071f5cb5544f2 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:51.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.219+0000 7f45f6ffd640 1 -- 172.21.15.100:0/2616817790 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f801c070 con 0x557cd5021390 2024-07-26T13:45:51.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.219+0000 7f4609ffb640 1 -- 172.21.15.100:0/3732079735 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f4600042030 con 0x7f460c101c70 2024-07-26T13:45:51.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.219+0000 7f45f6ffd640 1 -- 172.21.15.100:0/2616817790 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45f800ae10 con 0x557cd5021390 2024-07-26T13:45:51.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.219+0000 7f45f6ffd640 1 -- 172.21.15.100:0/2616817790 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f8017940 con 0x557cd5021390 2024-07-26T13:45:51.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.219+0000 7f4609ffb640 1 --2- 172.21.15.100:0/3732079735 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f45fc0b3d40 0x7f45fc0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:51.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.220+0000 7f46113db640 1 --2- 172.21.15.100:0/3732079735 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f45fc0b3d40 0x7f45fc0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:51.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.220+0000 7f4609ffb640 1 -- 172.21.15.100:0/3732079735 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7f46001045a0 con 0x7f460c101c70 2024-07-26T13:45:51.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.220+0000 7f46193c2740 1 -- 172.21.15.100:0/2616817790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cd5021390 msgr2=0x557cd53e7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:51.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.220+0000 7f46193c2740 1 --2- 172.21.15.100:0/2616817790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cd5021390 0x557cd53e7fa0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f45f800ac60 tx=0x7f45f800e820 comp rx=0 tx=0).stop 2024-07-26T13:45:51.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.220+0000 7f46193c2740 1 -- 172.21.15.100:0/2616817790 shutdown_connections 2024-07-26T13:45:51.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.220+0000 7f46193c2740 1 --2- 172.21.15.100:0/2616817790 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cd5021390 0x557cd53e7fa0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:51.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.220+0000 7f46193c2740 1 --2- 172.21.15.100:0/2616817790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557cd50191b0 0x557cd53e7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:51.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.220+0000 7f46193c2740 1 --2- 172.21.15.100:0/2616817790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cd5042070 0x557cd53e7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:51.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.220+0000 7f46193c2740 1 -- 172.21.15.100:0/2616817790 >> 172.21.15.100:0/2616817790 conn(0x557cd5036460 msgr2=0x557cd53e5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:51.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.220+0000 7f46193c2740 1 -- 172.21.15.100:0/2616817790 shutdown_connections 2024-07-26T13:45:51.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.220+0000 7f46193c2740 1 -- 172.21.15.100:0/2616817790 wait complete. 2024-07-26T13:45:51.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.220+0000 7f46113db640 1 --2- 172.21.15.100:0/3732079735 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f45fc0b3d40 0x7f45fc0b6250 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4604005fd0 tx=0x7f4604009600 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:51.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.220+0000 7f46193c2740 1 -- start start 2024-07-26T13:45:51.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.221+0000 7f46193c2740 1 Processor -- start 2024-07-26T13:45:51.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.221+0000 7f46193c2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557cd50191b0 0x557cd53f30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:51.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.221+0000 7f46193c2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cd5021390 0x557cd53f35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:51.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.221+0000 7f46193c2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cd5042070 0x557cd53ff020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:51.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.221+0000 7f46193c2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x557cd51b0700 con 0x557cd5042070 2024-07-26T13:45:51.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.221+0000 7f46193c2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557cd52ef990 con 0x557cd50191b0 2024-07-26T13:45:51.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.221+0000 7f46193c2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557cd53f5d00 con 0x557cd5021390 2024-07-26T13:45:51.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.221+0000 7f46193c2740 10 client.0 _ll_register_callbacks cb 0x557cd530a4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:45:51.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.221+0000 7f46113db640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cd5042070 0x557cd53ff020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:51.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.221+0000 7f46113db640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cd5042070 0x557cd53ff020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47486/0 (socket says 172.21.15.100:47486) 2024-07-26T13:45:51.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.221+0000 7f46113db640 1 -- 172.21.15.100:0/106101900 learned_addr learned my addr 172.21.15.100:0/106101900 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:51.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.221+0000 7f460bfff640 1 --2- 172.21.15.100:0/106101900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cd5021390 0x557cd53f35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:51.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.221+0000 7f4610bda640 1 --2- 172.21.15.100:0/106101900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557cd50191b0 0x557cd53f30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:51.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.222+0000 7f46113db640 1 -- 172.21.15.100:0/106101900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cd5021390 msgr2=0x557cd53f35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:51.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.222+0000 7f46113db640 1 --2- 172.21.15.100:0/106101900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cd5021390 0x557cd53f35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:51.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.222+0000 7f46113db640 1 -- 172.21.15.100:0/106101900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557cd50191b0 msgr2=0x557cd53f30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:51.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.222+0000 7f46113db640 1 --2- 172.21.15.100:0/106101900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557cd50191b0 0x557cd53f30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:51.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.222+0000 7f46113db640 1 -- 172.21.15.100:0/106101900 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cd50ad650 con 0x557cd5042070 2024-07-26T13:45:51.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.222+0000 7f46113db640 1 --2- 172.21.15.100:0/106101900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cd5042070 0x557cd53ff020 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f46040042b0 tx=0x7f4604004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0774f04afef7ab6 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:51.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.222+0000 7f45ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:45:51.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.222+0000 7f45ebfff640 1 -- 172.21.15.100:0/106101900 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4604025070 con 0x557cd5042070 2024-07-26T13:45:51.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.222+0000 7f45ebfff640 1 -- 172.21.15.100:0/106101900 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7f4604011d60 con 0x557cd5042070 2024-07-26T13:45:51.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.222+0000 7f46193c2740 1 -- 172.21.15.100:0/106101900 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45f800ae10 con 0x557cd5042070 2024-07-26T13:45:51.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.222+0000 7f45ebfff640 1 -- 172.21.15.100:0/106101900 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f4604031050 con 0x557cd5042070 2024-07-26T13:45:51.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.222+0000 7f46193c2740 10 client.16932 fetch_fsmap learned FSMap version 47 2024-07-26T13:45:51.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.222+0000 7f46193c2740 1 -- 172.21.15.100:0/106101900 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x557cd5401d70 con 0x557cd5042070 2024-07-26T13:45:51.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.223+0000 7f45ebfff640 1 -- 172.21.15.100:0/106101900 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45f800ae10 con 0x557cd5042070 2024-07-26T13:45:51.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.223+0000 7f45ebfff640 1 -- 172.21.15.100:0/106101900 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 47) ==== 8350+0+0 (secure 0 0 0) 0x7f460400f040 con 0x557cd5042070 2024-07-26T13:45:51.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.223+0000 7f46193c2740 10 client.16932 fetch_fsmap finished waiting for FSMap version 47 2024-07-26T13:45:51.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.223+0000 7f46193c2740 10 client.16932 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:45:51.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.223+0000 7f46193c2740 20 client.16932 populate_metadata read hostname 'smithi100' 2024-07-26T13:45:51.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.223+0000 7f46193c2740 1 --2- 172.21.15.100:0/106101900 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x557cd50009f0 0x557cd5405260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:51.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.223+0000 7f46193c2740 4 client.16932 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:45:51.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.223+0000 7f46193c2740 1 -- 172.21.15.100:0/106101900 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557cd535fab0 con 0x557cd50009f0 2024-07-26T13:45:51.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.223+0000 7f4610bda640 1 --2- 172.21.15.100:0/106101900 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x557cd50009f0 0x557cd5405260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:51.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.223+0000 7f45ebfff640 10 client.16932 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:51.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.224+0000 7f4610bda640 1 --2- 172.21.15.100:0/106101900 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x557cd50009f0 0x557cd5405260 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b64ee81813962721 server_cookie=ed1040c1d7d9c362 in_seq=0 out_seq=0 2024-07-26T13:45:51.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.224+0000 7f45ebfff640 1 -- 172.21.15.100:0/106101900 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557cd535fab0 con 0x557cd50009f0 2024-07-26T13:45:51.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.224+0000 7f45ebfff640 10 client.16932 handle_command_reply: tid=0 2024-07-26T13:45:51.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.224+0000 7f46193c2740 1 client.16932 shutdown 2024-07-26T13:45:51.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.224+0000 7f46193c2740 1 -- 172.21.15.100:0/106101900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cd5042070 msgr2=0x557cd53ff020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:51.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.224+0000 7f46193c2740 1 --2- 172.21.15.100:0/106101900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cd5042070 0x557cd53ff020 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f46040042b0 tx=0x7f4604004a40 comp rx=0 tx=0).stop 2024-07-26T13:45:51.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.225+0000 7f46193c2740 1 -- 172.21.15.100:0/106101900 shutdown_connections 2024-07-26T13:45:51.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.225+0000 7f46193c2740 1 --2- 172.21.15.100:0/106101900 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x557cd50009f0 0x557cd5405260 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:51.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.225+0000 7f46193c2740 1 --2- 172.21.15.100:0/106101900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cd5042070 0x557cd53ff020 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:51.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.225+0000 7f46193c2740 1 --2- 172.21.15.100:0/106101900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cd5021390 0x557cd53f35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:51.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.225+0000 7f46193c2740 1 --2- 172.21.15.100:0/106101900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557cd50191b0 0x557cd53f30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:51.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.225+0000 7f46193c2740 1 -- 172.21.15.100:0/106101900 >> 172.21.15.100:0/106101900 conn(0x557cd5036460 msgr2=0x557cd53e3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:51.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.225+0000 7f45ebfff640 0 client.16932 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:51.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.225+0000 7f46193c2740 1 -- 172.21.15.100:0/106101900 shutdown_connections 2024-07-26T13:45:51.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.225+0000 7f46193c2740 1 -- 172.21.15.100:0/106101900 wait complete. 2024-07-26T13:45:51.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.225+0000 7f46193c2740 20 client.16932 trim_cache size 0 max 16384 2024-07-26T13:45:51.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.254+0000 7f46193c2740 1 Processor -- start 2024-07-26T13:45:51.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.254+0000 7f46193c2740 1 -- start start 2024-07-26T13:45:51.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.254+0000 7f46193c2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cd50191b0 0x557cd53e5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:51.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.254+0000 7f46193c2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557cd5021390 0x557cd53fee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:51.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.254+0000 7f46193c2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cd5042070 0x557cd540aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:51.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.254+0000 7f46193c2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f460c1af960 con 0x557cd5042070 2024-07-26T13:45:51.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.254+0000 7f46193c2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f460c1ad200 con 0x557cd5021390 2024-07-26T13:45:51.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.254+0000 7f46193c2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f460c1ab030 con 0x557cd50191b0 2024-07-26T13:45:51.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.254+0000 7f46113db640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cd5042070 0x557cd540aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:51.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.254+0000 7f46113db640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cd5042070 0x557cd540aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47488/0 (socket says 172.21.15.100:47488) 2024-07-26T13:45:51.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.254+0000 7f46113db640 1 -- 172.21.15.100:0/1147953066 learned_addr learned my addr 172.21.15.100:0/1147953066 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:51.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.254+0000 7f4610bda640 1 --2- 172.21.15.100:0/1147953066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cd50191b0 0x557cd53e5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:51.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.254+0000 7f460bfff640 1 --2- 172.21.15.100:0/1147953066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557cd5021390 0x557cd53fee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:51.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.255+0000 7f46113db640 1 -- 172.21.15.100:0/1147953066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cd50191b0 msgr2=0x557cd53e5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:51.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.255+0000 7f46113db640 1 --2- 172.21.15.100:0/1147953066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cd50191b0 0x557cd53e5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:51.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.255+0000 7f46113db640 1 -- 172.21.15.100:0/1147953066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557cd5021390 msgr2=0x557cd53fee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:51.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.255+0000 7f46113db640 1 --2- 172.21.15.100:0/1147953066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557cd5021390 0x557cd53fee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:51.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.255+0000 7f46113db640 1 -- 172.21.15.100:0/1147953066 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cd53f13b0 con 0x557cd5042070 2024-07-26T13:45:51.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.255+0000 7f46113db640 1 --2- 172.21.15.100:0/1147953066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cd5042070 0x557cd540aa20 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f46040294f0 tx=0x7f4604003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4fce7d43be4753 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:51.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.255+0000 7f45f67fc640 1 -- 172.21.15.100:0/1147953066 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4604039020 con 0x557cd5042070 2024-07-26T13:45:51.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.255+0000 7f45f67fc640 1 -- 172.21.15.100:0/1147953066 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4604005bc0 con 0x557cd5042070 2024-07-26T13:45:51.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.255+0000 7f45f67fc640 1 -- 172.21.15.100:0/1147953066 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4604025040 con 0x557cd5042070 2024-07-26T13:45:51.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.256+0000 7f46193c2740 1 -- 172.21.15.100:0/1147953066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cd5042070 msgr2=0x557cd540aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:51.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.256+0000 7f46193c2740 1 --2- 172.21.15.100:0/1147953066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cd5042070 0x557cd540aa20 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f46040294f0 tx=0x7f4604003e00 comp rx=0 tx=0).stop 2024-07-26T13:45:51.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.256+0000 7f46193c2740 1 -- 172.21.15.100:0/1147953066 shutdown_connections 2024-07-26T13:45:51.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.256+0000 7f46193c2740 1 --2- 172.21.15.100:0/1147953066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cd5042070 0x557cd540aa20 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:51.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.256+0000 7f46193c2740 1 --2- 172.21.15.100:0/1147953066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557cd5021390 0x557cd53fee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:51.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.256+0000 7f46193c2740 1 --2- 172.21.15.100:0/1147953066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cd50191b0 0x557cd53e5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:51.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.256+0000 7f46193c2740 1 -- 172.21.15.100:0/1147953066 >> 172.21.15.100:0/1147953066 conn(0x557cd5036460 msgr2=0x557cd53e4e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:51.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.256+0000 7f46193c2740 1 -- 172.21.15.100:0/1147953066 shutdown_connections 2024-07-26T13:45:51.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.256+0000 7f46193c2740 1 -- 172.21.15.100:0/1147953066 wait complete. 2024-07-26T13:45:51.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.256+0000 7f46193c2740 1 -- start start 2024-07-26T13:45:51.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.256+0000 7f46193c2740 1 Processor -- start 2024-07-26T13:45:51.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.257+0000 7f46193c2740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cd50191b0 0x557cd53f6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:51.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.257+0000 7f46193c2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cd5021390 0x557cd53f72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:51.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.257+0000 7f46193c2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557cd5042070 0x557cd53f7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:51.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.257+0000 7f46193c2740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f460c1ab6c0 con 0x557cd50191b0 2024-07-26T13:45:51.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.257+0000 7f46193c2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f460c1acb30 con 0x557cd5042070 2024-07-26T13:45:51.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.257+0000 7f46193c2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f460c1abd90 con 0x557cd5021390 2024-07-26T13:45:51.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.257+0000 7f46193c2740 10 client.0 _ll_register_callbacks cb 0x7f4604009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:45:51.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.257+0000 7f4610bda640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cd50191b0 0x557cd53f6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:51.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.257+0000 7f4610bda640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cd50191b0 0x557cd53f6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47498/0 (socket says 172.21.15.100:47498) 2024-07-26T13:45:51.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.257+0000 7f4610bda640 1 -- 172.21.15.100:0/3696317004 learned_addr learned my addr 172.21.15.100:0/3696317004 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:51.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.257+0000 7f460bfff640 1 --2- 172.21.15.100:0/3696317004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cd5021390 0x557cd53f72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:51.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.257+0000 7f46113db640 1 --2- 172.21.15.100:0/3696317004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557cd5042070 0x557cd53f7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:51.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.258+0000 7f4610bda640 1 -- 172.21.15.100:0/3696317004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cd5021390 msgr2=0x557cd53f72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:51.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.258+0000 7f4610bda640 1 --2- 172.21.15.100:0/3696317004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cd5021390 0x557cd53f72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:51.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.258+0000 7f4610bda640 1 -- 172.21.15.100:0/3696317004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557cd5042070 msgr2=0x557cd53f7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:51.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.258+0000 7f4610bda640 1 --2- 172.21.15.100:0/3696317004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557cd5042070 0x557cd53f7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:51.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.258+0000 7f4610bda640 1 -- 172.21.15.100:0/3696317004 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cd546c9f0 con 0x557cd50191b0 2024-07-26T13:45:51.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.258+0000 7f4610bda640 1 --2- 172.21.15.100:0/3696317004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cd50191b0 0x557cd53f6da0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f4600006010 tx=0x7f4600005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecfcd459e1c3353e server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:51.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.258+0000 7f45ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:45:51.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.258+0000 7f45ebfff640 1 -- 172.21.15.100:0/3696317004 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4600049040 con 0x557cd50191b0 2024-07-26T13:45:51.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.258+0000 7f45ebfff640 1 -- 172.21.15.100:0/3696317004 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7f46000c7720 con 0x557cd50191b0 2024-07-26T13:45:51.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.258+0000 7f46193c2740 1 -- 172.21.15.100:0/3696317004 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4604005bc0 con 0x557cd50191b0 2024-07-26T13:45:51.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.258+0000 7f45ebfff640 1 -- 172.21.15.100:0/3696317004 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f46000cb130 con 0x557cd50191b0 2024-07-26T13:45:51.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.258+0000 7f46193c2740 10 client.16938 fetch_fsmap learned FSMap version 47 2024-07-26T13:45:51.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.258+0000 7f46193c2740 1 -- 172.21.15.100:0/3696317004 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x557cd546dbe0 con 0x557cd50191b0 2024-07-26T13:45:51.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.258+0000 7f45ebfff640 1 -- 172.21.15.100:0/3696317004 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4604005bc0 con 0x557cd50191b0 2024-07-26T13:45:51.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.258+0000 7f45ebfff640 1 -- 172.21.15.100:0/3696317004 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 47) ==== 8350+0+0 (secure 0 0 0) 0x7f46000cb390 con 0x557cd50191b0 2024-07-26T13:45:51.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.259+0000 7f46193c2740 10 client.16938 fetch_fsmap finished waiting for FSMap version 47 2024-07-26T13:45:51.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.259+0000 7f46193c2740 10 client.16938 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:45:51.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.259+0000 7f46193c2740 20 client.16938 populate_metadata read hostname 'smithi100' 2024-07-26T13:45:51.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.259+0000 7f46193c2740 1 --2- 172.21.15.100:0/3696317004 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x557cd4fd5da0 0x557cd5470350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:51.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.259+0000 7f46193c2740 4 client.16938 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-26T13:45:51.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.259+0000 7f46193c2740 1 -- 172.21.15.100:0/3696317004 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557cd535b550 con 0x557cd4fd5da0 2024-07-26T13:45:51.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.259+0000 7f460bfff640 1 --2- 172.21.15.100:0/3696317004 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x557cd4fd5da0 0x557cd5470350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:51.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.259+0000 7f45ebfff640 10 client.16938 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:51.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.259+0000 7f460bfff640 1 --2- 172.21.15.100:0/3696317004 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x557cd4fd5da0 0x557cd5470350 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9fce1930a8a290c server_cookie=7009d10b68d10afe in_seq=0 out_seq=0 2024-07-26T13:45:51.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.260+0000 7f45ebfff640 1 -- 172.21.15.100:0/3696317004 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557cd535b550 con 0x557cd4fd5da0 2024-07-26T13:45:51.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.260+0000 7f45ebfff640 10 client.16938 handle_command_reply: tid=0 2024-07-26T13:45:51.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.260+0000 7f46193c2740 1 client.16938 shutdown 2024-07-26T13:45:51.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.260+0000 7f46193c2740 1 -- 172.21.15.100:0/3696317004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cd50191b0 msgr2=0x557cd53f6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:51.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.260+0000 7f46193c2740 1 --2- 172.21.15.100:0/3696317004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cd50191b0 0x557cd53f6da0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f4600006010 tx=0x7f4600005e70 comp rx=0 tx=0).stop 2024-07-26T13:45:51.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.260+0000 7f46193c2740 1 -- 172.21.15.100:0/3696317004 shutdown_connections 2024-07-26T13:45:51.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.260+0000 7f46193c2740 1 --2- 172.21.15.100:0/3696317004 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x557cd4fd5da0 0x557cd5470350 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:51.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.260+0000 7f46193c2740 1 --2- 172.21.15.100:0/3696317004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557cd5042070 0x557cd53f7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:51.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.260+0000 7f46193c2740 1 --2- 172.21.15.100:0/3696317004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557cd5021390 0x557cd53f72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:51.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.260+0000 7f45ebfff640 0 client.16938 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:45:51.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.260+0000 7f46193c2740 1 --2- 172.21.15.100:0/3696317004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557cd50191b0 0x557cd53f6da0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:51.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.260+0000 7f46193c2740 1 -- 172.21.15.100:0/3696317004 >> 172.21.15.100:0/3696317004 conn(0x557cd5036460 msgr2=0x557cd53e8550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:51.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.260+0000 7f46193c2740 1 -- 172.21.15.100:0/3696317004 shutdown_connections 2024-07-26T13:45:51.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.260+0000 7f46193c2740 1 -- 172.21.15.100:0/3696317004 wait complete. 2024-07-26T13:45:51.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.260+0000 7f46193c2740 20 client.16938 trim_cache size 0 max 16384 2024-07-26T13:45:51.268 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:45:51.268 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-07-26T13:45:51.269 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-07-26T13:45:51.269 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:45:51.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.261+0000 7f4612e3e640 1 -- 172.21.15.100:0/3732079735 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f45fc0b3d40 msgr2=0x7f45fc0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:51.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.261+0000 7f4612e3e640 1 --2- 172.21.15.100:0/3732079735 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f45fc0b3d40 0x7f45fc0b6250 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4604005fd0 tx=0x7f4604009600 comp rx=0 tx=0).stop 2024-07-26T13:45:51.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.261+0000 7f4612e3e640 1 -- 172.21.15.100:0/3732079735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f460c101c70 msgr2=0x7f460c197cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:51.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.261+0000 7f4612e3e640 1 --2- 172.21.15.100:0/3732079735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f460c101c70 0x7f460c197cf0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f4600005940 tx=0x7f4600004270 comp rx=0 tx=0).stop 2024-07-26T13:45:51.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.261+0000 7f4612e3e640 1 -- 172.21.15.100:0/3732079735 shutdown_connections 2024-07-26T13:45:51.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.261+0000 7f4612e3e640 1 --2- 172.21.15.100:0/3732079735 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f45fc0b3d40 0x7f45fc0b6250 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:51.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.261+0000 7f4612e3e640 1 --2- 172.21.15.100:0/3732079735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f460c107f00 0x7f460c19d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:51.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.261+0000 7f4612e3e640 1 --2- 172.21.15.100:0/3732079735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f460c1075b0 0x7f460c198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:51.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.261+0000 7f4612e3e640 1 --2- 172.21.15.100:0/3732079735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f460c101c70 0x7f460c197cf0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:51.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.261+0000 7f4612e3e640 1 -- 172.21.15.100:0/3732079735 >> 172.21.15.100:0/3732079735 conn(0x7f460c0799c0 msgr2=0x7f460c105600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:51.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.261+0000 7f4612e3e640 1 -- 172.21.15.100:0/3732079735 shutdown_connections 2024-07-26T13:45:51.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:51.261+0000 7f4612e3e640 1 -- 172.21.15.100:0/3732079735 wait complete. 2024-07-26T13:45:51.418 INFO:tasks.workunit.client.0.smithi100.stdout: 1 222724 3.30 MB/sec execute 125 sec latency 1155.016 ms 2024-07-26T13:45:51.715 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:51 smithi100 ceph-mon[32599]: pgmap v538: 97 pgs: 97 active+clean; 96 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.8 MiB/s wr, 317 op/s 2024-07-26T13:45:51.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:51 smithi138 ceph-mon[36954]: pgmap v538: 97 pgs: 97 active+clean; 96 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.8 MiB/s wr, 317 op/s 2024-07-26T13:45:52.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:51 smithi167 ceph-mon[37140]: pgmap v538: 97 pgs: 97 active+clean; 96 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.8 MiB/s wr, 317 op/s 2024-07-26T13:45:52.078 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-26T13:45:52.079 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:45:52.341 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:45:52.418 INFO:tasks.workunit.client.0.smithi100.stdout: 1 222724 3.27 MB/sec execute 126 sec latency 2155.093 ms 2024-07-26T13:45:52.628 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:52 smithi100 ceph-mon[32599]: pgmap v539: 97 pgs: 97 active+clean; 88 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.6 MiB/s wr, 291 op/s 2024-07-26T13:45:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:52 smithi138 ceph-mon[36954]: pgmap v539: 97 pgs: 97 active+clean; 88 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.6 MiB/s wr, 291 op/s 2024-07-26T13:45:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:52 smithi167 ceph-mon[37140]: pgmap v539: 97 pgs: 97 active+clean; 88 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.6 MiB/s wr, 291 op/s 2024-07-26T13:45:53.418 INFO:tasks.workunit.client.0.smithi100.stdout: 1 222724 3.25 MB/sec execute 127 sec latency 3155.210 ms 2024-07-26T13:45:53.728 INFO:tasks.quiescer.fs.[cephfs]:Total duration (43.9) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-07-26T13:45:53.728 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.95 2024-07-26T13:45:53.729 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:45:53.949 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: ceph-mds: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-5320-ge8f88bb1/rpm/el9/BUILD/ceph-19.0.0-5320-ge8f88bb1/redhat-linux-build/boost/include/boost/smart_ptr/intrusive_ptr.hpp:201: T* boost::intrusive_ptr::operator->() const [with T = const Message]: Assertion `px != 0' failed. 2024-07-26T13:45:53.949 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: *** Caught signal (Aborted) ** 2024-07-26T13:45:53.949 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: in thread 7f471654c640 thread_name:ms_dispatch 2024-07-26T13:45:53.949 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: ceph version 19.0.0-5320-ge8f88bb1 (e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528) squid (dev) 2024-07-26T13:45:53.949 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f471b9236f0] 2024-07-26T13:45:53.949 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 2: /lib64/libc.so.6(+0x8b94c) [0x7f471b97094c] 2024-07-26T13:45:53.949 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 3: raise() 2024-07-26T13:45:53.950 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 4: abort() 2024-07-26T13:45:53.950 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 5: /lib64/libc.so.6(+0x2871b) [0x7f471b90d71b] 2024-07-26T13:45:53.950 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 6: /lib64/libc.so.6(+0x37386) [0x7f471b91c386] 2024-07-26T13:45:53.950 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 7: (MDSRank::get_session(boost::intrusive_ptr const&)+0x89) [0x555fe9b786b9] 2024-07-26T13:45:53.950 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 8: (MDCache::path_traverse(boost::intrusive_ptr const&, ContextFactory&, filepath const&, int, std::vector >*, CInode**, CDir**)+0x58e) [0x555fe9d3680c] 2024-07-26T13:45:53.950 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 9: (Server::handle_peer_rename_prep(boost::intrusive_ptr const&)+0x4fe) [0x555fe9c84cde] 2024-07-26T13:45:53.950 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 10: (Server::dispatch_peer_request(boost::intrusive_ptr const&)+0x933) [0x555fe9c46763] 2024-07-26T13:45:53.950 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 11: (Server::handle_peer_request(boost::intrusive_ptr const&)+0x10c3) [0x555fe9c47a1b] 2024-07-26T13:45:53.950 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x9ba) [0x555fe9c2b752] 2024-07-26T13:45:53.950 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 13: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x81a) [0x555fe9b79a56] 2024-07-26T13:45:53.950 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 14: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x555fe9b7d637] 2024-07-26T13:45:53.950 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 15: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x555fe9b7df5e] 2024-07-26T13:45:53.950 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 16: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x555fe9b6103a] 2024-07-26T13:45:53.951 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 17: /usr/lib64/ceph/libceph-common.so.2(+0x39b407) [0x7f471c1f0407] 2024-07-26T13:45:53.951 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 18: (DispatchQueue::entry()+0x60f) [0x7f471c1f0cf9] 2024-07-26T13:45:53.951 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 19: /usr/lib64/ceph/libceph-common.so.2(+0x4785a1) [0x7f471c2cd5a1] 2024-07-26T13:45:53.951 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 20: (Thread::entry_wrapper()+0x43) [0x7f471c07acdd] 2024-07-26T13:45:53.952 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 21: (Thread::_entry_func(void*)+0xd) [0x7f471c07acf9] 2024-07-26T13:45:53.952 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 22: /lib64/libc.so.6(+0x89c02) [0x7f471b96ec02] 2024-07-26T13:45:53.952 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 23: /lib64/libc.so.6(+0x10ec40) [0x7f471b9f3c40] 2024-07-26T13:45:53.952 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 2024-07-26T13:45:53.580+0000 7f471654c640 -1 *** Caught signal (Aborted) ** 2024-07-26T13:45:53.952 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: in thread 7f471654c640 thread_name:ms_dispatch 2024-07-26T13:45:53.952 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 2024-07-26T13:45:53.952 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: ceph version 19.0.0-5320-ge8f88bb1 (e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528) squid (dev) 2024-07-26T13:45:53.952 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f471b9236f0] 2024-07-26T13:45:53.952 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 2: /lib64/libc.so.6(+0x8b94c) [0x7f471b97094c] 2024-07-26T13:45:53.952 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 3: raise() 2024-07-26T13:45:53.953 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 4: abort() 2024-07-26T13:45:53.953 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 5: /lib64/libc.so.6(+0x2871b) [0x7f471b90d71b] 2024-07-26T13:45:53.953 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 6: /lib64/libc.so.6(+0x37386) [0x7f471b91c386] 2024-07-26T13:45:53.953 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 7: (MDSRank::get_session(boost::intrusive_ptr const&)+0x89) [0x555fe9b786b9] 2024-07-26T13:45:53.953 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 8: (MDCache::path_traverse(boost::intrusive_ptr const&, ContextFactory&, filepath const&, int, std::vector >*, CInode**, CDir**)+0x58e) [0x555fe9d3680c] 2024-07-26T13:45:53.953 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 9: (Server::handle_peer_rename_prep(boost::intrusive_ptr const&)+0x4fe) [0x555fe9c84cde] 2024-07-26T13:45:53.953 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 10: (Server::dispatch_peer_request(boost::intrusive_ptr const&)+0x933) [0x555fe9c46763] 2024-07-26T13:45:53.953 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 11: (Server::handle_peer_request(boost::intrusive_ptr const&)+0x10c3) [0x555fe9c47a1b] 2024-07-26T13:45:53.953 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x9ba) [0x555fe9c2b752] 2024-07-26T13:45:53.953 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 13: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x81a) [0x555fe9b79a56] 2024-07-26T13:45:53.953 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 14: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x555fe9b7d637] 2024-07-26T13:45:53.953 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 15: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x555fe9b7df5e] 2024-07-26T13:45:53.953 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 16: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x555fe9b6103a] 2024-07-26T13:45:53.954 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 17: /usr/lib64/ceph/libceph-common.so.2(+0x39b407) [0x7f471c1f0407] 2024-07-26T13:45:53.954 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 18: (DispatchQueue::entry()+0x60f) [0x7f471c1f0cf9] 2024-07-26T13:45:53.954 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 19: /usr/lib64/ceph/libceph-common.so.2(+0x4785a1) [0x7f471c2cd5a1] 2024-07-26T13:45:53.954 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 20: (Thread::entry_wrapper()+0x43) [0x7f471c07acdd] 2024-07-26T13:45:53.954 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 21: (Thread::_entry_func(void*)+0xd) [0x7f471c07acf9] 2024-07-26T13:45:53.954 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 22: /lib64/libc.so.6(+0x89c02) [0x7f471b96ec02] 2024-07-26T13:45:53.954 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 23: /lib64/libc.so.6(+0x10ec40) [0x7f471b9f3c40] 2024-07-26T13:45:53.954 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-07-26T13:45:53.954 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 2024-07-26T13:45:53.954 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 0> 2024-07-26T13:45:53.580+0000 7f471654c640 -1 *** Caught signal (Aborted) ** 2024-07-26T13:45:53.954 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: in thread 7f471654c640 thread_name:ms_dispatch 2024-07-26T13:45:53.954 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 2024-07-26T13:45:53.954 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: ceph version 19.0.0-5320-ge8f88bb1 (e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528) squid (dev) 2024-07-26T13:45:53.954 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f471b9236f0] 2024-07-26T13:45:53.955 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 2: /lib64/libc.so.6(+0x8b94c) [0x7f471b97094c] 2024-07-26T13:45:53.955 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 3: raise() 2024-07-26T13:45:53.955 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 4: abort() 2024-07-26T13:45:53.955 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 5: /lib64/libc.so.6(+0x2871b) [0x7f471b90d71b] 2024-07-26T13:45:53.955 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 6: /lib64/libc.so.6(+0x37386) [0x7f471b91c386] 2024-07-26T13:45:53.955 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 7: (MDSRank::get_session(boost::intrusive_ptr const&)+0x89) [0x555fe9b786b9] 2024-07-26T13:45:53.955 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 8: (MDCache::path_traverse(boost::intrusive_ptr const&, ContextFactory&, filepath const&, int, std::vector >*, CInode**, CDir**)+0x58e) [0x555fe9d3680c] 2024-07-26T13:45:53.955 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 9: (Server::handle_peer_rename_prep(boost::intrusive_ptr const&)+0x4fe) [0x555fe9c84cde] 2024-07-26T13:45:53.955 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 10: (Server::dispatch_peer_request(boost::intrusive_ptr const&)+0x933) [0x555fe9c46763] 2024-07-26T13:45:53.955 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 11: (Server::handle_peer_request(boost::intrusive_ptr const&)+0x10c3) [0x555fe9c47a1b] 2024-07-26T13:45:53.955 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x9ba) [0x555fe9c2b752] 2024-07-26T13:45:53.955 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 13: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x81a) [0x555fe9b79a56] 2024-07-26T13:45:53.955 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 14: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x555fe9b7d637] 2024-07-26T13:45:53.955 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 15: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x555fe9b7df5e] 2024-07-26T13:45:53.956 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 16: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x555fe9b6103a] 2024-07-26T13:45:53.956 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 17: /usr/lib64/ceph/libceph-common.so.2(+0x39b407) [0x7f471c1f0407] 2024-07-26T13:45:53.956 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 18: (DispatchQueue::entry()+0x60f) [0x7f471c1f0cf9] 2024-07-26T13:45:53.956 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 19: /usr/lib64/ceph/libceph-common.so.2(+0x4785a1) [0x7f471c2cd5a1] 2024-07-26T13:45:53.956 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 20: (Thread::entry_wrapper()+0x43) [0x7f471c07acdd] 2024-07-26T13:45:53.956 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 21: (Thread::_entry_func(void*)+0xd) [0x7f471c07acf9] 2024-07-26T13:45:53.956 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 22: /lib64/libc.so.6(+0x89c02) [0x7f471b96ec02] 2024-07-26T13:45:53.956 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 23: /lib64/libc.so.6(+0x10ec40) [0x7f471b9f3c40] 2024-07-26T13:45:53.956 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-07-26T13:45:53.956 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 2024-07-26T13:45:53.956 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: -5> 2024-07-26T13:45:53.580+0000 7f471654c640 -1 *** Caught signal (Aborted) ** 2024-07-26T13:45:53.956 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: in thread 7f471654c640 thread_name:ms_dispatch 2024-07-26T13:45:53.956 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 2024-07-26T13:45:53.957 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: ceph version 19.0.0-5320-ge8f88bb1 (e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528) squid (dev) 2024-07-26T13:45:53.957 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f471b9236f0] 2024-07-26T13:45:53.957 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 2: /lib64/libc.so.6(+0x8b94c) [0x7f471b97094c] 2024-07-26T13:45:53.957 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 3: raise() 2024-07-26T13:45:53.957 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 4: abort() 2024-07-26T13:45:53.957 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 5: /lib64/libc.so.6(+0x2871b) [0x7f471b90d71b] 2024-07-26T13:45:53.957 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 6: /lib64/libc.so.6(+0x37386) [0x7f471b91c386] 2024-07-26T13:45:53.957 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 7: (MDSRank::get_session(boost::intrusive_ptr const&)+0x89) [0x555fe9b786b9] 2024-07-26T13:45:53.957 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 8: (MDCache::path_traverse(boost::intrusive_ptr const&, ContextFactory&, filepath const&, int, std::vector >*, CInode**, CDir**)+0x58e) [0x555fe9d3680c] 2024-07-26T13:45:53.957 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 9: (Server::handle_peer_rename_prep(boost::intrusive_ptr const&)+0x4fe) [0x555fe9c84cde] 2024-07-26T13:45:53.957 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 10: (Server::dispatch_peer_request(boost::intrusive_ptr const&)+0x933) [0x555fe9c46763] 2024-07-26T13:45:53.957 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 11: (Server::handle_peer_request(boost::intrusive_ptr const&)+0x10c3) [0x555fe9c47a1b] 2024-07-26T13:45:53.957 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x9ba) [0x555fe9c2b752] 2024-07-26T13:45:53.957 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 13: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x81a) [0x555fe9b79a56] 2024-07-26T13:45:53.957 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 14: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x555fe9b7d637] 2024-07-26T13:45:53.958 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 15: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x555fe9b7df5e] 2024-07-26T13:45:53.958 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 16: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x555fe9b6103a] 2024-07-26T13:45:53.958 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 17: /usr/lib64/ceph/libceph-common.so.2(+0x39b407) [0x7f471c1f0407] 2024-07-26T13:45:53.958 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 18: (DispatchQueue::entry()+0x60f) [0x7f471c1f0cf9] 2024-07-26T13:45:53.958 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 19: /usr/lib64/ceph/libceph-common.so.2(+0x4785a1) [0x7f471c2cd5a1] 2024-07-26T13:45:53.958 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 20: (Thread::entry_wrapper()+0x43) [0x7f471c07acdd] 2024-07-26T13:45:53.958 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 21: (Thread::_entry_func(void*)+0xd) [0x7f471c07acf9] 2024-07-26T13:45:53.958 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 22: /lib64/libc.so.6(+0x89c02) [0x7f471b96ec02] 2024-07-26T13:45:53.958 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 23: /lib64/libc.so.6(+0x10ec40) [0x7f471b9f3c40] 2024-07-26T13:45:53.958 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-07-26T13:45:53.958 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[71915]: 2024-07-26T13:45:53.958 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:53 smithi167 podman[72375]: 2024-07-26 13:45:53.637300134 +0000 UTC m=+0.019953694 container died e0d4ef191c8d9901c511ebe9fad8969ad51ab96a95f1bcb28c96cc04af6ba9a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, org.label-schema.vendor=CentOS) 2024-07-26T13:45:54.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.127+0000 7fa072b89640 1 -- 172.21.15.100:0/1960642994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa06c1024b0 msgr2=0x7fa06c1028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:54.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.127+0000 7fa072b89640 1 --2- 172.21.15.100:0/1960642994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa06c1024b0 0x7fa06c1028c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009900 tx=0x7fa05c02a410 comp rx=0 tx=0).stop 2024-07-26T13:45:54.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.127+0000 7fa072b89640 1 -- 172.21.15.100:0/1960642994 shutdown_connections 2024-07-26T13:45:54.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.127+0000 7fa072b89640 1 --2- 172.21.15.100:0/1960642994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa06c108740 0x7fa06c10ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:54.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.127+0000 7fa072b89640 1 --2- 172.21.15.100:0/1960642994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa06c107df0 0x7fa06c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:54.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.127+0000 7fa072b89640 1 --2- 172.21.15.100:0/1960642994 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa06c1024b0 0x7fa06c1028c0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:54.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.127+0000 7fa072b89640 1 -- 172.21.15.100:0/1960642994 >> 172.21.15.100:0/1960642994 conn(0x7fa06c079170 msgr2=0x7fa06c079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:54.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.127+0000 7fa072b89640 1 -- 172.21.15.100:0/1960642994 shutdown_connections 2024-07-26T13:45:54.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.127+0000 7fa072b89640 1 -- 172.21.15.100:0/1960642994 wait complete. 2024-07-26T13:45:54.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.128+0000 7fa072b89640 1 Processor -- start 2024-07-26T13:45:54.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.128+0000 7fa072b89640 1 -- start start 2024-07-26T13:45:54.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.128+0000 7fa072b89640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa06c1024b0 0x7fa06c198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:54.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.128+0000 7fa072b89640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa06c107df0 0x7fa06c199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:54.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.128+0000 7fa072b89640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa06c108740 0x7fa06c19e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:54.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.128+0000 7fa072b89640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c1123c0 con 0x7fa06c108740 2024-07-26T13:45:54.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.128+0000 7fa072b89640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c112240 con 0x7fa06c107df0 2024-07-26T13:45:54.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.128+0000 7fa072b89640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c112540 con 0x7fa06c1024b0 2024-07-26T13:45:54.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.128+0000 7fa071126640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa06c108740 0x7fa06c19e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:54.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.128+0000 7fa070925640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa06c1024b0 0x7fa06c198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:54.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.128+0000 7fa070925640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa06c1024b0 0x7fa06c198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:49850/0 (socket says 172.21.15.100:49850) 2024-07-26T13:45:54.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.128+0000 7fa071126640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa06c108740 0x7fa06c19e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47520/0 (socket says 172.21.15.100:47520) 2024-07-26T13:45:54.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.128+0000 7fa070925640 1 -- 172.21.15.100:0/4035435433 learned_addr learned my addr 172.21.15.100:0/4035435433 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:54.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.128+0000 7fa06bfff640 1 --2- 172.21.15.100:0/4035435433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa06c107df0 0x7fa06c199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:54.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.129+0000 7fa071126640 1 -- 172.21.15.100:0/4035435433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa06c1024b0 msgr2=0x7fa06c198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:54.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.129+0000 7fa071126640 1 --2- 172.21.15.100:0/4035435433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa06c1024b0 0x7fa06c198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:54.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.129+0000 7fa071126640 1 -- 172.21.15.100:0/4035435433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa06c107df0 msgr2=0x7fa06c199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:54.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.129+0000 7fa071126640 1 --2- 172.21.15.100:0/4035435433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa06c107df0 0x7fa06c199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:54.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.129+0000 7fa071126640 1 -- 172.21.15.100:0/4035435433 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa06c19ec60 con 0x7fa06c108740 2024-07-26T13:45:54.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.129+0000 7fa071126640 1 --2- 172.21.15.100:0/4035435433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa06c108740 0x7fa06c19e5f0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fa06400cab0 tx=0x7fa06400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:54.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.129+0000 7fa069ffb640 1 -- 172.21.15.100:0/4035435433 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa06400ae20 con 0x7fa06c108740 2024-07-26T13:45:54.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.129+0000 7fa069ffb640 1 -- 172.21.15.100:0/4035435433 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa06400f900 con 0x7fa06c108740 2024-07-26T13:45:54.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.129+0000 7fa072b89640 1 -- 172.21.15.100:0/4035435433 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa06c19ef50 con 0x7fa06c108740 2024-07-26T13:45:54.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.129+0000 7fa069ffb640 1 -- 172.21.15.100:0/4035435433 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0640189f0 con 0x7fa06c108740 2024-07-26T13:45:54.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.129+0000 7fa072b89640 1 -- 172.21.15.100:0/4035435433 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa06c06b570 con 0x7fa06c108740 2024-07-26T13:45:54.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.130+0000 7fa072b89640 1 -- 172.21.15.100:0/4035435433 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa06c103730 con 0x7fa06c108740 2024-07-26T13:45:54.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.132+0000 7fa069ffb640 1 -- 172.21.15.100:0/4035435433 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fa06400f450 con 0x7fa06c108740 2024-07-26T13:45:54.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.132+0000 7fa069ffb640 1 --2- 172.21.15.100:0/4035435433 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fa0540b3d70 0x7fa0540b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:54.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.132+0000 7fa070925640 1 --2- 172.21.15.100:0/4035435433 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fa0540b3d70 0x7fa0540b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:54.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.133+0000 7fa069ffb640 1 -- 172.21.15.100:0/4035435433 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7fa0640e4770 con 0x7fa06c108740 2024-07-26T13:45:54.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.133+0000 7fa070925640 1 --2- 172.21.15.100:0/4035435433 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fa0540b3d70 0x7fa0540b6280 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa05c031440 tx=0x7fa05c03a040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:54.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.137+0000 7fa069ffb640 1 -- 172.21.15.100:0/4035435433 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fa064014030 con 0x7fa06c108740 2024-07-26T13:45:54.237 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:45:54.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.362+0000 7fa072b89640 1 -- 172.21.15.100:0/4035435433 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa06c06bff0 con 0x7fa06c108740 2024-07-26T13:45:54.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.363+0000 7fa069ffb640 1 -- 172.21.15.100:0/4035435433 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11929 (secure 0 0 0) 0x7fa0640aba00 con 0x7fa06c108740 2024-07-26T13:45:54.364 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:45:54.364 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":47,"btime":"2024-07-26T13:45:43:559244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14790,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/313501409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":313501409},{"type":"v1","addr":"172.21.15.100:6839","nonce":313501409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/1402975506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":1402975506},{"type":"v1","addr":"172.21.15.100:6841","nonce":1402975506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-26T13:37:01.281843+0000","modified":"2024-07-26T13:45:43.558267+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24607,"mds_1":24605,"mds_2":25762},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24575":{"gid":24575,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/3393225483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3393225483},{"type":"v1","addr":"172.21.15.138:6837","nonce":3393225483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.138:6839/1580925471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1580925471},{"type":"v1","addr":"172.21.15.138:6839","nonce":1580925471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/398150514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":398150514},{"type":"v1","addr":"172.21.15.167:6839","nonce":398150514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25762":{"gid":25762,"name":"c","rank":2,"incarnation":39,"state":"up:active","state_seq":32,"addr":"172.21.15.167:6837/2476068386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2476068386},{"type":"v1","addr":"172.21.15.167:6837","nonce":2476068386}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25997":{"gid":25997,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/2831339503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2831339503},{"type":"v1","addr":"172.21.15.138:6835","nonce":2831339503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24607,"qdb_cluster":[24607,25762,24605]},"id":1}]} 2024-07-26T13:45:54.365 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 47 2024-07-26T13:45:54.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.365+0000 7fa072b89640 1 -- 172.21.15.100:0/4035435433 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fa0540b3d70 msgr2=0x7fa0540b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:54.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.365+0000 7fa072b89640 1 --2- 172.21.15.100:0/4035435433 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fa0540b3d70 0x7fa0540b6280 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa05c031440 tx=0x7fa05c03a040 comp rx=0 tx=0).stop 2024-07-26T13:45:54.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.365+0000 7fa072b89640 1 -- 172.21.15.100:0/4035435433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa06c108740 msgr2=0x7fa06c19e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:54.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.365+0000 7fa072b89640 1 --2- 172.21.15.100:0/4035435433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa06c108740 0x7fa06c19e5f0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fa06400cab0 tx=0x7fa06400cf70 comp rx=0 tx=0).stop 2024-07-26T13:45:54.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.365+0000 7fa072b89640 1 -- 172.21.15.100:0/4035435433 shutdown_connections 2024-07-26T13:45:54.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.365+0000 7fa072b89640 1 --2- 172.21.15.100:0/4035435433 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fa0540b3d70 0x7fa0540b6280 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:54.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.365+0000 7fa072b89640 1 --2- 172.21.15.100:0/4035435433 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa06c108740 0x7fa06c19e5f0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:54.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.365+0000 7fa072b89640 1 --2- 172.21.15.100:0/4035435433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa06c107df0 0x7fa06c199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:54.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.365+0000 7fa072b89640 1 --2- 172.21.15.100:0/4035435433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa06c1024b0 0x7fa06c198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:54.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.365+0000 7fa072b89640 1 -- 172.21.15.100:0/4035435433 >> 172.21.15.100:0/4035435433 conn(0x7fa06c079170 msgr2=0x7fa06c100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:54.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.365+0000 7fa072b89640 1 -- 172.21.15.100:0/4035435433 shutdown_connections 2024-07-26T13:45:54.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:54.365+0000 7fa072b89640 1 -- 172.21.15.100:0/4035435433 wait complete. 2024-07-26T13:45:54.418 INFO:tasks.workunit.client.0.smithi100.stdout: 1 222738 3.22 MB/sec execute 128 sec latency 3302.218 ms 2024-07-26T13:45:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:54 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/4035435433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:45:54.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:54 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/4035435433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:45:54.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:54 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/4035435433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:45:54.772 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:54 smithi167 podman[72375]: 2024-07-26 13:45:54.298225018 +0000 UTC m=+0.680878598 container remove e0d4ef191c8d9901c511ebe9fad8969ad51ab96a95f1bcb28c96cc04af6ba9a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240724, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, ceph=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-26T13:45:54.772 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:54 smithi167 systemd[1]: ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.c.service: Main process exited, code=exited, status=134/n/a 2024-07-26T13:45:54.773 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:54 smithi167 systemd[1]: ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-07-26T13:45:54.773 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:45:54 smithi167 systemd[1]: ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.c.service: Consumed 15.482s CPU time. 2024-07-26T13:45:55.418 INFO:tasks.workunit.client.0.smithi100.stdout: 1 222738 3.20 MB/sec execute 129 sec latency 1841.172 ms 2024-07-26T13:45:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:55 smithi100 ceph-mon[32599]: pgmap v540: 97 pgs: 97 active+clean; 89 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 233 op/s 2024-07-26T13:45:55.663 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-26T13:45:55.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:55 smithi138 ceph-mon[36954]: pgmap v540: 97 pgs: 97 active+clean; 89 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 233 op/s 2024-07-26T13:45:55.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:55 smithi167 ceph-mon[37140]: pgmap v540: 97 pgs: 97 active+clean; 89 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 233 op/s 2024-07-26T13:45:56.419 INFO:tasks.workunit.client.0.smithi100.stdout: 1 222738 3.17 MB/sec execute 130 sec latency 2841.219 ms 2024-07-26T13:45:56.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.831+0000 7f1214e91640 1 -- 172.21.15.100:0/1856992438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1210108720 msgr2=0x7f121010ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:56.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.831+0000 7f1214e91640 1 --2- 172.21.15.100:0/1856992438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1210108720 0x7f121010ef50 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f1208009ca0 tx=0x7f120802a8c0 comp rx=0 tx=0).stop 2024-07-26T13:45:56.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.831+0000 7f1214e91640 1 -- 172.21.15.100:0/1856992438 shutdown_connections 2024-07-26T13:45:56.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.831+0000 7f1214e91640 1 --2- 172.21.15.100:0/1856992438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1210108720 0x7f121010ef50 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:56.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.831+0000 7f1214e91640 1 --2- 172.21.15.100:0/1856992438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1210107dd0 0x7f12101081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:56.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.831+0000 7f1214e91640 1 --2- 172.21.15.100:0/1856992438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1210102490 0x7f12101028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:56.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.831+0000 7f1214e91640 1 -- 172.21.15.100:0/1856992438 >> 172.21.15.100:0/1856992438 conn(0x7f1210079160 msgr2=0x7f1210079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:56.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.831+0000 7f1214e91640 1 -- 172.21.15.100:0/1856992438 shutdown_connections 2024-07-26T13:45:56.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.831+0000 7f1214e91640 1 -- 172.21.15.100:0/1856992438 wait complete. 2024-07-26T13:45:56.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.832+0000 7f1214e91640 1 Processor -- start 2024-07-26T13:45:56.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.832+0000 7f1214e91640 1 -- start start 2024-07-26T13:45:56.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.832+0000 7f1214e91640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1210102490 0x7f1210112c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:56.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.832+0000 7f1214e91640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1210107dd0 0x7f12101131c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:56.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.832+0000 7f1214e91640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1210108720 0x7f1210113700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:56.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.832+0000 7f1214e91640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f12101123a0 con 0x7f1210102490 2024-07-26T13:45:56.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.832+0000 7f1214e91640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1210112220 con 0x7f1210108720 2024-07-26T13:45:56.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.832+0000 7f1214e91640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1210112520 con 0x7f1210107dd0 2024-07-26T13:45:56.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.832+0000 7f120e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1210102490 0x7f1210112c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:56.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.832+0000 7f120e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1210102490 0x7f1210112c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37468/0 (socket says 172.21.15.100:37468) 2024-07-26T13:45:56.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.832+0000 7f120e59c640 1 -- 172.21.15.100:0/299558968 learned_addr learned my addr 172.21.15.100:0/299558968 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:56.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.832+0000 7f120dd9b640 1 --2- 172.21.15.100:0/299558968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1210107dd0 0x7f12101131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:56.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.832+0000 7f120ed9d640 1 --2- 172.21.15.100:0/299558968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1210108720 0x7f1210113700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:56.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.833+0000 7f120e59c640 1 -- 172.21.15.100:0/299558968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1210107dd0 msgr2=0x7f12101131c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:56.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.833+0000 7f120e59c640 1 --2- 172.21.15.100:0/299558968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1210107dd0 0x7f12101131c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:56.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.833+0000 7f120e59c640 1 -- 172.21.15.100:0/299558968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1210108720 msgr2=0x7f1210113700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:56.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.833+0000 7f120e59c640 1 --2- 172.21.15.100:0/299558968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1210108720 0x7f1210113700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:56.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.833+0000 7f120e59c640 1 -- 172.21.15.100:0/299558968 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12101a8110 con 0x7f1210102490 2024-07-26T13:45:56.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.833+0000 7f120e59c640 1 --2- 172.21.15.100:0/299558968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1210102490 0x7f1210112c80 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f120000d770 tx=0x7f120000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:56.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.833+0000 7f11ff7fe640 1 -- 172.21.15.100:0/299558968 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1200017070 con 0x7f1210102490 2024-07-26T13:45:56.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.833+0000 7f11ff7fe640 1 -- 172.21.15.100:0/299558968 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12000054d0 con 0x7f1210102490 2024-07-26T13:45:56.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.833+0000 7f1214e91640 1 -- 172.21.15.100:0/299558968 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12101a8400 con 0x7f1210102490 2024-07-26T13:45:56.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.833+0000 7f11ff7fe640 1 -- 172.21.15.100:0/299558968 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1200004c30 con 0x7f1210102490 2024-07-26T13:45:56.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.833+0000 7f1214e91640 1 -- 172.21.15.100:0/299558968 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12101a8940 con 0x7f1210102490 2024-07-26T13:45:56.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.834+0000 7f1214e91640 1 -- 172.21.15.100:0/299558968 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12101028a0 con 0x7f1210102490 2024-07-26T13:45:56.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.836+0000 7f11ff7fe640 1 -- 172.21.15.100:0/299558968 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f1200004dd0 con 0x7f1210102490 2024-07-26T13:45:56.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.836+0000 7f11ff7fe640 1 --2- 172.21.15.100:0/299558968 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f11f80b3dc0 0x7f11f80b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:56.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.836+0000 7f120dd9b640 1 --2- 172.21.15.100:0/299558968 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f11f80b3dc0 0x7f11f80b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:56.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.836+0000 7f11ff7fe640 1 -- 172.21.15.100:0/299558968 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7f12000e3e50 con 0x7f1210102490 2024-07-26T13:45:56.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.837+0000 7f120dd9b640 1 --2- 172.21.15.100:0/299558968 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f11f80b3dc0 0x7f11f80b62d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1210079700 tx=0x7f120400b410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:56.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:56.839+0000 7f11ff7fe640 1 -- 172.21.15.100:0/299558968 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f12000aab10 con 0x7f1210102490 2024-07-26T13:45:56.881 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:45:57.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:57.060+0000 7f1214e91640 1 -- 172.21.15.100:0/299558968 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12101a8d00 con 0x7f1210102490 2024-07-26T13:45:57.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:57.060+0000 7f11ff7fe640 1 -- 172.21.15.100:0/299558968 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11929 (secure 0 0 0) 0x7f12000af060 con 0x7f1210102490 2024-07-26T13:45:57.061 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:45:57.062 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":47,"btime":"2024-07-26T13:45:43:559244+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14790,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/313501409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":313501409},{"type":"v1","addr":"172.21.15.100:6839","nonce":313501409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/1402975506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":1402975506},{"type":"v1","addr":"172.21.15.100:6841","nonce":1402975506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-26T13:37:01.281843+0000","modified":"2024-07-26T13:45:43.558267+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24607,"mds_1":24605,"mds_2":25762},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24575":{"gid":24575,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/3393225483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3393225483},{"type":"v1","addr":"172.21.15.138:6837","nonce":3393225483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.138:6839/1580925471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1580925471},{"type":"v1","addr":"172.21.15.138:6839","nonce":1580925471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/398150514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":398150514},{"type":"v1","addr":"172.21.15.167:6839","nonce":398150514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25762":{"gid":25762,"name":"c","rank":2,"incarnation":39,"state":"up:active","state_seq":32,"addr":"172.21.15.167:6837/2476068386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2476068386},{"type":"v1","addr":"172.21.15.167:6837","nonce":2476068386}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25997":{"gid":25997,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/2831339503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2831339503},{"type":"v1","addr":"172.21.15.138:6835","nonce":2831339503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24607,"qdb_cluster":[24607,25762,24605]},"id":1}]} 2024-07-26T13:45:57.062 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 47 2024-07-26T13:45:57.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:57.062+0000 7f1214e91640 1 -- 172.21.15.100:0/299558968 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f11f80b3dc0 msgr2=0x7f11f80b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:57.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:57.062+0000 7f1214e91640 1 --2- 172.21.15.100:0/299558968 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f11f80b3dc0 0x7f11f80b62d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1210079700 tx=0x7f120400b410 comp rx=0 tx=0).stop 2024-07-26T13:45:57.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:57.062+0000 7f1214e91640 1 -- 172.21.15.100:0/299558968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1210102490 msgr2=0x7f1210112c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:57.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:57.062+0000 7f1214e91640 1 --2- 172.21.15.100:0/299558968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1210102490 0x7f1210112c80 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f120000d770 tx=0x7f120000dc30 comp rx=0 tx=0).stop 2024-07-26T13:45:57.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:57.062+0000 7f1214e91640 1 -- 172.21.15.100:0/299558968 shutdown_connections 2024-07-26T13:45:57.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:57.062+0000 7f1214e91640 1 --2- 172.21.15.100:0/299558968 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f11f80b3dc0 0x7f11f80b62d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:57.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:57.062+0000 7f1214e91640 1 --2- 172.21.15.100:0/299558968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1210108720 0x7f1210113700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:57.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:57.062+0000 7f1214e91640 1 --2- 172.21.15.100:0/299558968 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1210107dd0 0x7f12101131c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:57.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:57.062+0000 7f1214e91640 1 --2- 172.21.15.100:0/299558968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1210102490 0x7f1210112c80 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:57.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:57.062+0000 7f1214e91640 1 -- 172.21.15.100:0/299558968 >> 172.21.15.100:0/299558968 conn(0x7f1210079160 msgr2=0x7f1210103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:57.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:57.063+0000 7f1214e91640 1 -- 172.21.15.100:0/299558968 shutdown_connections 2024-07-26T13:45:57.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:57.063+0000 7f1214e91640 1 -- 172.21.15.100:0/299558968 wait complete. 2024-07-26T13:45:57.419 INFO:tasks.workunit.client.0.smithi100.stdout: 1 222738 3.15 MB/sec execute 131 sec latency 3841.302 ms 2024-07-26T13:45:57.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:57 smithi138 ceph-mon[36954]: pgmap v541: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 209 op/s 2024-07-26T13:45:57.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:57 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/299558968' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:45:57.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:57 smithi100 ceph-mon[32599]: pgmap v541: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 209 op/s 2024-07-26T13:45:57.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:57 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/299558968' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:45:57.981 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24607 quiesce db / --timeout 90 --expiration 141.95 --await-for 45.0' 2024-07-26T13:45:57.981 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- timeout 105 ceph tell mds.24607 quiesce db / --timeout 90 --expiration 141.95 --await-for 45.0 2024-07-26T13:45:58.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:57 smithi167 ceph-mon[37140]: pgmap v541: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 209 op/s 2024-07-26T13:45:58.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:57 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/299558968' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:45:58.419 INFO:tasks.workunit.client.0.smithi100.stdout: 1 222738 3.13 MB/sec execute 132 sec latency 4841.780 ms 2024-07-26T13:45:59.419 INFO:tasks.workunit.client.0.smithi100.stdout: 1 222738 3.10 MB/sec execute 133 sec latency 5841.832 ms 2024-07-26T13:45:59.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.443+0000 7fb295c2b640 1 -- 172.21.15.100:0/1851195453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb290075160 msgr2=0x7fb290075570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:59.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.443+0000 7fb295c2b640 1 --2- 172.21.15.100:0/1851195453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb290075160 0x7fb290075570 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fb280009900 tx=0x7fb28002a410 comp rx=0 tx=0).stop 2024-07-26T13:45:59.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.443+0000 7fb295c2b640 1 -- 172.21.15.100:0/1851195453 shutdown_connections 2024-07-26T13:45:59.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.443+0000 7fb295c2b640 1 --2- 172.21.15.100:0/1851195453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb29010fb70 0x7fb290111f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:59.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.443+0000 7fb295c2b640 1 --2- 172.21.15.100:0/1851195453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb290075ab0 0x7fb29010f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:59.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.443+0000 7fb295c2b640 1 --2- 172.21.15.100:0/1851195453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb290075160 0x7fb290075570 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:59.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.443+0000 7fb295c2b640 1 -- 172.21.15.100:0/1851195453 >> 172.21.15.100:0/1851195453 conn(0x7fb29007b280 msgr2=0x7fb29007b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:59.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.443+0000 7fb295c2b640 1 -- 172.21.15.100:0/1851195453 shutdown_connections 2024-07-26T13:45:59.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.443+0000 7fb295c2b640 1 -- 172.21.15.100:0/1851195453 wait complete. 2024-07-26T13:45:59.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.444+0000 7fb295c2b640 1 Processor -- start 2024-07-26T13:45:59.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.444+0000 7fb295c2b640 1 -- start start 2024-07-26T13:45:59.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.445+0000 7fb295c2b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb290075160 0x7fb29019bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:59.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.445+0000 7fb295c2b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb290075ab0 0x7fb29019c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:59.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.445+0000 7fb295c2b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb29010fb70 0x7fb2901a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:59.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.445+0000 7fb295c2b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb290115730 con 0x7fb290075160 2024-07-26T13:45:59.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.445+0000 7fb295c2b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb2901155b0 con 0x7fb290075ab0 2024-07-26T13:45:59.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.445+0000 7fb295c2b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb2901158b0 con 0x7fb29010fb70 2024-07-26T13:45:59.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.445+0000 7fb28f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb290075160 0x7fb29019bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:59.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.445+0000 7fb28f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb290075160 0x7fb29019bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37482/0 (socket says 172.21.15.100:37482) 2024-07-26T13:45:59.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.445+0000 7fb28f7fe640 1 -- 172.21.15.100:0/3003013112 learned_addr learned my addr 172.21.15.100:0/3003013112 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:59.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.445+0000 7fb28ffff640 1 --2- 172.21.15.100:0/3003013112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb29010fb70 0x7fb2901a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:59.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.445+0000 7fb28f7fe640 1 -- 172.21.15.100:0/3003013112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb29010fb70 msgr2=0x7fb2901a17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:59.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.445+0000 7fb28f7fe640 1 --2- 172.21.15.100:0/3003013112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb29010fb70 0x7fb2901a17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:59.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.445+0000 7fb28f7fe640 1 -- 172.21.15.100:0/3003013112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb290075ab0 msgr2=0x7fb29019c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:59.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.445+0000 7fb28effd640 1 --2- 172.21.15.100:0/3003013112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb290075ab0 0x7fb29019c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:59.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.445+0000 7fb28f7fe640 1 --2- 172.21.15.100:0/3003013112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb290075ab0 0x7fb29019c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:59.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.445+0000 7fb28f7fe640 1 -- 172.21.15.100:0/3003013112 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2901a1e30 con 0x7fb290075160 2024-07-26T13:45:59.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.445+0000 7fb28f7fe640 1 --2- 172.21.15.100:0/3003013112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb290075160 0x7fb29019bf80 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fb280030940 tx=0x7fb280030970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:59.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.445+0000 7fb28cff9640 1 -- 172.21.15.100:0/3003013112 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28003f070 con 0x7fb290075160 2024-07-26T13:45:59.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.445+0000 7fb28cff9640 1 -- 172.21.15.100:0/3003013112 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb280030da0 con 0x7fb290075160 2024-07-26T13:45:59.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.445+0000 7fb295c2b640 1 -- 172.21.15.100:0/3003013112 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2901a20c0 con 0x7fb290075160 2024-07-26T13:45:59.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.445+0000 7fb28cff9640 1 -- 172.21.15.100:0/3003013112 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb280035030 con 0x7fb290075160 2024-07-26T13:45:59.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.445+0000 7fb295c2b640 1 -- 172.21.15.100:0/3003013112 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2901a25a0 con 0x7fb290075160 2024-07-26T13:45:59.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.448+0000 7fb29c1af740 1 Processor -- start 2024-07-26T13:45:59.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.448+0000 7fb29c1af740 1 -- start start 2024-07-26T13:45:59.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.448+0000 7fb29c1af740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a4f0ecf070 0x55a4f1274570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:59.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.448+0000 7fb29c1af740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4f0ea61b0 0x55a4f1274ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:59.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.448+0000 7fb29c1af740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a4f0eae390 0x55a4f1274ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:59.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.448+0000 7fb29c1af740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55a4f11eabb0 con 0x55a4f0eae390 2024-07-26T13:45:59.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.448+0000 7fb28cff9640 1 -- 172.21.15.100:0/3003013112 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fb280042070 con 0x7fb290075160 2024-07-26T13:45:59.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.448+0000 7fb29c1af740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a4f11da0b0 con 0x55a4f0ea61b0 2024-07-26T13:45:59.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.448+0000 7fb29c1af740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a4f11de0f0 con 0x55a4f0ecf070 2024-07-26T13:45:59.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.448+0000 7fb28effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a4f0eae390 0x55a4f1274ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:59.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.448+0000 7fb28effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a4f0eae390 0x55a4f1274ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37486/0 (socket says 172.21.15.100:37486) 2024-07-26T13:45:59.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.448+0000 7fb28ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a4f0ecf070 0x55a4f1274570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:59.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.448+0000 7fb28effd640 1 -- 172.21.15.100:0/1031713577 learned_addr learned my addr 172.21.15.100:0/1031713577 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:59.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.449+0000 7fb28f7fe640 1 --2- 172.21.15.100:0/1031713577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4f0ea61b0 0x55a4f1274ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:59.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.449+0000 7fb28effd640 1 -- 172.21.15.100:0/1031713577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a4f0ecf070 msgr2=0x55a4f1274570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:59.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.449+0000 7fb28effd640 1 --2- 172.21.15.100:0/1031713577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a4f0ecf070 0x55a4f1274570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:59.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.449+0000 7fb28effd640 1 -- 172.21.15.100:0/1031713577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4f0ea61b0 msgr2=0x55a4f1274ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:59.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.449+0000 7fb28effd640 1 --2- 172.21.15.100:0/1031713577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4f0ea61b0 0x55a4f1274ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:59.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.449+0000 7fb28effd640 1 -- 172.21.15.100:0/1031713577 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4f1180b90 con 0x55a4f0eae390 2024-07-26T13:45:59.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.449+0000 7fb28effd640 1 --2- 172.21.15.100:0/1031713577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a4f0eae390 0x55a4f1274ff0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fb28400baa0 tx=0x7fb28400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49506dabd95fc7df server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:59.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.449+0000 7fb275ffb640 1 -- 172.21.15.100:0/1031713577 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb284017420 con 0x55a4f0eae390 2024-07-26T13:45:59.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.449+0000 7fb275ffb640 1 -- 172.21.15.100:0/1031713577 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb284017a70 con 0x55a4f0eae390 2024-07-26T13:45:59.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.449+0000 7fb275ffb640 1 -- 172.21.15.100:0/1031713577 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28400fa50 con 0x55a4f0eae390 2024-07-26T13:45:59.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.449+0000 7fb28cff9640 1 --2- 172.21.15.100:0/3003013112 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb2780b3d70 0x7fb2780b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:59.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.449+0000 7fb28ffff640 1 --2- 172.21.15.100:0/3003013112 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb2780b3d70 0x7fb2780b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:59.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.449+0000 7fb28cff9640 1 -- 172.21.15.100:0/3003013112 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7fb280104c90 con 0x7fb290075160 2024-07-26T13:45:59.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.450+0000 7fb28ffff640 1 --2- 172.21.15.100:0/3003013112 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb2780b3d70 0x7fb2780b6280 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb288007ed0 tx=0x7fb2880075c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:59.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.450+0000 7fb29c1af740 1 -- 172.21.15.100:0/1031713577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a4f0eae390 msgr2=0x55a4f1274ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:59.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.450+0000 7fb29c1af740 1 --2- 172.21.15.100:0/1031713577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a4f0eae390 0x55a4f1274ff0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fb28400baa0 tx=0x7fb28400bf60 comp rx=0 tx=0).stop 2024-07-26T13:45:59.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.450+0000 7fb29c1af740 1 -- 172.21.15.100:0/1031713577 shutdown_connections 2024-07-26T13:45:59.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.450+0000 7fb29c1af740 1 --2- 172.21.15.100:0/1031713577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a4f0eae390 0x55a4f1274ff0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:59.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.450+0000 7fb29c1af740 1 --2- 172.21.15.100:0/1031713577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4f0ea61b0 0x55a4f1274ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:59.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.450+0000 7fb29c1af740 1 --2- 172.21.15.100:0/1031713577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a4f0ecf070 0x55a4f1274570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:59.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.450+0000 7fb29c1af740 1 -- 172.21.15.100:0/1031713577 >> 172.21.15.100:0/1031713577 conn(0x55a4f0ec3460 msgr2=0x55a4f1272460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:59.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.450+0000 7fb29c1af740 1 -- 172.21.15.100:0/1031713577 shutdown_connections 2024-07-26T13:45:59.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.450+0000 7fb29c1af740 1 -- 172.21.15.100:0/1031713577 wait complete. 2024-07-26T13:45:59.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.450+0000 7fb29c1af740 1 -- start start 2024-07-26T13:45:59.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.451+0000 7fb29c1af740 1 Processor -- start 2024-07-26T13:45:59.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.451+0000 7fb29c1af740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a4f0ea61b0 0x55a4f1280140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:59.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.451+0000 7fb29c1af740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4f0eae390 0x55a4f1280680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:59.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.451+0000 7fb29c1af740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a4f0ecf070 0x55a4f128c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:59.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.451+0000 7fb29c1af740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55a4f103d700 con 0x55a4f0ecf070 2024-07-26T13:45:59.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.451+0000 7fb29c1af740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a4f117c9e0 con 0x55a4f0eae390 2024-07-26T13:45:59.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.451+0000 7fb29c1af740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a4f1282d50 con 0x55a4f0ea61b0 2024-07-26T13:45:59.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.451+0000 7fb29c1af740 10 client.0 _ll_register_callbacks cb 0x55a4f11974f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:45:59.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.451+0000 7fb28ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a4f0ecf070 0x55a4f128c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:59.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.451+0000 7fb28ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a4f0ecf070 0x55a4f128c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37498/0 (socket says 172.21.15.100:37498) 2024-07-26T13:45:59.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.451+0000 7fb28ffff640 1 -- 172.21.15.100:0/13894015 learned_addr learned my addr 172.21.15.100:0/13894015 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:59.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.451+0000 7fb28f7fe640 1 --2- 172.21.15.100:0/13894015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a4f0ea61b0 0x55a4f1280140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:59.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.451+0000 7fb28effd640 1 --2- 172.21.15.100:0/13894015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4f0eae390 0x55a4f1280680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:59.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.451+0000 7fb28ffff640 1 -- 172.21.15.100:0/13894015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a4f0ea61b0 msgr2=0x55a4f1280140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:59.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.451+0000 7fb28ffff640 1 --2- 172.21.15.100:0/13894015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a4f0ea61b0 0x55a4f1280140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:59.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.451+0000 7fb28ffff640 1 -- 172.21.15.100:0/13894015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4f0eae390 msgr2=0x55a4f1280680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:59.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.451+0000 7fb28ffff640 1 --2- 172.21.15.100:0/13894015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4f0eae390 0x55a4f1280680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:59.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.451+0000 7fb28ffff640 1 -- 172.21.15.100:0/13894015 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4f0f3a650 con 0x55a4f0ecf070 2024-07-26T13:45:59.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.451+0000 7fb28effd640 1 --2- 172.21.15.100:0/13894015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4f0eae390 0x55a4f1280680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:45:59.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.452+0000 7fb28ffff640 1 --2- 172.21.15.100:0/13894015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a4f0ecf070 0x55a4f128c040 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fb288012e00 tx=0x7fb288004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886216589dcc1685 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:59.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.452+0000 7fb26effd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:45:59.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.452+0000 7fb26effd640 1 -- 172.21.15.100:0/13894015 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288023020 con 0x55a4f0ecf070 2024-07-26T13:45:59.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.452+0000 7fb26effd640 1 -- 172.21.15.100:0/13894015 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7fb28801f440 con 0x55a4f0ecf070 2024-07-26T13:45:59.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.452+0000 7fb29c1af740 1 -- 172.21.15.100:0/13894015 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb284017a70 con 0x55a4f0ecf070 2024-07-26T13:45:59.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.452+0000 7fb26effd640 1 -- 172.21.15.100:0/13894015 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb28802e050 con 0x55a4f0ecf070 2024-07-26T13:45:59.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.452+0000 7fb29c1af740 10 client.16962 fetch_fsmap learned FSMap version 47 2024-07-26T13:45:59.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.452+0000 7fb29c1af740 1 -- 172.21.15.100:0/13894015 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55a4f128edd0 con 0x55a4f0ecf070 2024-07-26T13:45:59.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.452+0000 7fb26effd640 1 -- 172.21.15.100:0/13894015 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb284017a70 con 0x55a4f0ecf070 2024-07-26T13:45:59.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.452+0000 7fb26effd640 1 -- 172.21.15.100:0/13894015 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 47) ==== 8350+0+0 (secure 0 0 0) 0x7fb288011040 con 0x55a4f0ecf070 2024-07-26T13:45:59.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.452+0000 7fb29c1af740 10 client.16962 fetch_fsmap finished waiting for FSMap version 47 2024-07-26T13:45:59.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.453+0000 7fb29c1af740 10 client.16962 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-26T13:45:59.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.453+0000 7fb29c1af740 20 client.16962 populate_metadata read hostname 'smithi100' 2024-07-26T13:45:59.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.453+0000 7fb29c1af740 1 --2- 172.21.15.100:0/13894015 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55a4f0e8d9f0 0x55a4f1292610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:59.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.453+0000 7fb29c1af740 4 client.16962 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:45:59.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.453+0000 7fb29c1af740 1 -- 172.21.15.100:0/13894015 --> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4f11ecb00 con 0x55a4f0e8d9f0 2024-07-26T13:45:59.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.453+0000 7fb28f7fe640 1 --2- 172.21.15.100:0/13894015 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55a4f0e8d9f0 0x55a4f1292610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:59.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.454+0000 7fb28f7fe640 1 --2- 172.21.15.100:0/13894015 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55a4f0e8d9f0 0x55a4f1292610 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=302658110a26dc9a server_cookie=3ef468e5c454c85c in_seq=0 out_seq=0 2024-07-26T13:45:59.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.454+0000 7fb26effd640 10 client.16962 ms_handle_connect on v2:172.21.15.138:6838/1580925471 2024-07-26T13:45:59.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.454+0000 7fb26effd640 1 -- 172.21.15.100:0/13894015 <== mds.1 v2:172.21.15.138:6838/1580925471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a4f11ecb00 con 0x55a4f0e8d9f0 2024-07-26T13:45:59.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.454+0000 7fb26effd640 10 client.16962 handle_command_reply: tid=0 2024-07-26T13:45:59.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.454+0000 7fb29c1af740 1 client.16962 shutdown 2024-07-26T13:45:59.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.455+0000 7fb29c1af740 1 -- 172.21.15.100:0/13894015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a4f0ecf070 msgr2=0x55a4f128c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:59.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.455+0000 7fb29c1af740 1 --2- 172.21.15.100:0/13894015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a4f0ecf070 0x55a4f128c040 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fb288012e00 tx=0x7fb288004a40 comp rx=0 tx=0).stop 2024-07-26T13:45:59.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.455+0000 7fb29c1af740 1 -- 172.21.15.100:0/13894015 shutdown_connections 2024-07-26T13:45:59.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.455+0000 7fb29c1af740 1 --2- 172.21.15.100:0/13894015 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55a4f0e8d9f0 0x55a4f1292610 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:59.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.455+0000 7fb29c1af740 1 --2- 172.21.15.100:0/13894015 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a4f0ecf070 0x55a4f128c040 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:59.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.455+0000 7fb29c1af740 1 --2- 172.21.15.100:0/13894015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4f0eae390 0x55a4f1280680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:59.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.455+0000 7fb29c1af740 1 --2- 172.21.15.100:0/13894015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a4f0ea61b0 0x55a4f1280140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:59.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.455+0000 7fb29c1af740 1 -- 172.21.15.100:0/13894015 >> 172.21.15.100:0/13894015 conn(0x55a4f0ec3460 msgr2=0x55a4f1270c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:59.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.455+0000 7fb26effd640 0 client.16962 ms_handle_reset on v2:172.21.15.138:6838/1580925471 2024-07-26T13:45:59.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.455+0000 7fb29c1af740 1 -- 172.21.15.100:0/13894015 shutdown_connections 2024-07-26T13:45:59.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.455+0000 7fb29c1af740 1 -- 172.21.15.100:0/13894015 wait complete. 2024-07-26T13:45:59.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.455+0000 7fb29c1af740 20 client.16962 trim_cache size 0 max 16384 2024-07-26T13:45:59.465 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:45:59.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.468+0000 7fb29c1af740 1 Processor -- start 2024-07-26T13:45:59.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.468+0000 7fb29c1af740 1 -- start start 2024-07-26T13:45:59.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.468+0000 7fb29c1af740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a4f0ea61b0 0x55a4f1271350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:59.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.468+0000 7fb29c1af740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4f0eae390 0x55a4f128bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:59.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.468+0000 7fb29c1af740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a4f0ecf070 0x55a4f128c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:59.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.468+0000 7fb29c1af740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb2901b3d60 con 0x55a4f0ecf070 2024-07-26T13:45:59.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.468+0000 7fb29c1af740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb2901b1600 con 0x55a4f0eae390 2024-07-26T13:45:59.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.468+0000 7fb29c1af740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb2901af430 con 0x55a4f0ea61b0 2024-07-26T13:45:59.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.469+0000 7fb28ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a4f0ecf070 0x55a4f128c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:59.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.469+0000 7fb28ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a4f0ecf070 0x55a4f128c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37510/0 (socket says 172.21.15.100:37510) 2024-07-26T13:45:59.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.469+0000 7fb28ffff640 1 -- 172.21.15.100:0/2415069566 learned_addr learned my addr 172.21.15.100:0/2415069566 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:59.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.469+0000 7fb28f7fe640 1 --2- 172.21.15.100:0/2415069566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a4f0ea61b0 0x55a4f1271350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:59.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.469+0000 7fb28effd640 1 --2- 172.21.15.100:0/2415069566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4f0eae390 0x55a4f128bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:59.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.469+0000 7fb28ffff640 1 -- 172.21.15.100:0/2415069566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a4f0ea61b0 msgr2=0x55a4f1271350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:59.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.469+0000 7fb28ffff640 1 --2- 172.21.15.100:0/2415069566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a4f0ea61b0 0x55a4f1271350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:59.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.469+0000 7fb28ffff640 1 -- 172.21.15.100:0/2415069566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4f0eae390 msgr2=0x55a4f128bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:59.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.469+0000 7fb28ffff640 1 --2- 172.21.15.100:0/2415069566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4f0eae390 0x55a4f128bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:59.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.469+0000 7fb28ffff640 1 -- 172.21.15.100:0/2415069566 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4f127ac50 con 0x55a4f0ecf070 2024-07-26T13:45:59.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.469+0000 7fb28ffff640 1 --2- 172.21.15.100:0/2415069566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a4f0ecf070 0x55a4f128c4f0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fb288012820 tx=0x7fb28801e3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87b0927513fe8b8c server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:59.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.470+0000 7fb2757fa640 1 -- 172.21.15.100:0/2415069566 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288036070 con 0x55a4f0ecf070 2024-07-26T13:45:59.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.470+0000 7fb2757fa640 1 -- 172.21.15.100:0/2415069566 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb288018030 con 0x55a4f0ecf070 2024-07-26T13:45:59.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.470+0000 7fb2757fa640 1 -- 172.21.15.100:0/2415069566 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288023050 con 0x55a4f0ecf070 2024-07-26T13:45:59.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.471+0000 7fb29c1af740 1 -- 172.21.15.100:0/2415069566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a4f0ecf070 msgr2=0x55a4f128c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:59.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.471+0000 7fb29c1af740 1 --2- 172.21.15.100:0/2415069566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a4f0ecf070 0x55a4f128c4f0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fb288012820 tx=0x7fb28801e3e0 comp rx=0 tx=0).stop 2024-07-26T13:45:59.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.471+0000 7fb29c1af740 1 -- 172.21.15.100:0/2415069566 shutdown_connections 2024-07-26T13:45:59.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.471+0000 7fb29c1af740 1 --2- 172.21.15.100:0/2415069566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a4f0ecf070 0x55a4f128c4f0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:59.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.471+0000 7fb29c1af740 1 --2- 172.21.15.100:0/2415069566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4f0eae390 0x55a4f128bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:59.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.471+0000 7fb29c1af740 1 --2- 172.21.15.100:0/2415069566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a4f0ea61b0 0x55a4f1271350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:59.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.471+0000 7fb29c1af740 1 -- 172.21.15.100:0/2415069566 >> 172.21.15.100:0/2415069566 conn(0x55a4f0ec3460 msgr2=0x55a4f1270c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:59.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.471+0000 7fb29c1af740 1 -- 172.21.15.100:0/2415069566 shutdown_connections 2024-07-26T13:45:59.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.471+0000 7fb29c1af740 1 -- 172.21.15.100:0/2415069566 wait complete. 2024-07-26T13:45:59.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.472+0000 7fb29c1af740 1 -- start start 2024-07-26T13:45:59.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.472+0000 7fb29c1af740 1 Processor -- start 2024-07-26T13:45:59.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.472+0000 7fb29c1af740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a4f0ea61b0 0x55a4f1292d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:59.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.472+0000 7fb29c1af740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4f0eae390 0x55a4f1293310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:59.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.472+0000 7fb29c1af740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a4f0ecf070 0x55a4f12f2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:59.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.472+0000 7fb29c1af740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb2901afac0 con 0x55a4f0ecf070 2024-07-26T13:45:59.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.472+0000 7fb29c1af740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb2901b0f30 con 0x55a4f0eae390 2024-07-26T13:45:59.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.472+0000 7fb29c1af740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb2901b0190 con 0x55a4f0ea61b0 2024-07-26T13:45:59.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.472+0000 7fb29c1af740 10 client.0 _ll_register_callbacks cb 0x7fb288007b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:45:59.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.473+0000 7fb28f7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a4f0ea61b0 0x55a4f1292d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:59.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.473+0000 7fb28f7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a4f0ea61b0 0x55a4f1292d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:60730/0 (socket says 172.21.15.100:60730) 2024-07-26T13:45:59.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.473+0000 7fb28f7fe640 1 -- 172.21.15.100:0/2211131678 learned_addr learned my addr 172.21.15.100:0/2211131678 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:45:59.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.473+0000 7fb28ffff640 1 --2- 172.21.15.100:0/2211131678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a4f0ecf070 0x55a4f12f2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:59.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.473+0000 7fb28effd640 1 --2- 172.21.15.100:0/2211131678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4f0eae390 0x55a4f1293310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:59.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.473+0000 7fb28f7fe640 1 -- 172.21.15.100:0/2211131678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4f0eae390 msgr2=0x55a4f1293310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:59.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.473+0000 7fb28f7fe640 1 --2- 172.21.15.100:0/2211131678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4f0eae390 0x55a4f1293310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:59.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.473+0000 7fb28f7fe640 1 -- 172.21.15.100:0/2211131678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a4f0ecf070 msgr2=0x55a4f12f2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:45:59.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.473+0000 7fb28f7fe640 1 --2- 172.21.15.100:0/2211131678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a4f0ecf070 0x55a4f12f2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:59.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.473+0000 7fb28f7fe640 1 -- 172.21.15.100:0/2211131678 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4f128ca30 con 0x55a4f0ea61b0 2024-07-26T13:45:59.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.473+0000 7fb28f7fe640 1 --2- 172.21.15.100:0/2211131678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a4f0ea61b0 0x55a4f1292d30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb2800c8a90 tx=0x7fb280003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69ff98716ee5da48 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:45:59.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.473+0000 7fb26effd640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-07-26T13:45:59.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.474+0000 7fb26effd640 1 -- 172.21.15.100:0/2211131678 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb280049040 con 0x55a4f0ea61b0 2024-07-26T13:45:59.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.474+0000 7fb26effd640 1 -- 172.21.15.100:0/2211131678 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7fb2801025e0 con 0x55a4f0ea61b0 2024-07-26T13:45:59.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.474+0000 7fb29c1af740 1 -- 172.21.15.100:0/2211131678 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb288018030 con 0x55a4f0ea61b0 2024-07-26T13:45:59.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.474+0000 7fb26effd640 1 -- 172.21.15.100:0/2211131678 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb2801027e0 con 0x55a4f0ea61b0 2024-07-26T13:45:59.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.474+0000 7fb29c1af740 10 client.26395 fetch_fsmap learned FSMap version 47 2024-07-26T13:45:59.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.474+0000 7fb29c1af740 1 -- 172.21.15.100:0/2211131678 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55a4f12fa370 con 0x55a4f0ea61b0 2024-07-26T13:45:59.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.474+0000 7fb26effd640 1 -- 172.21.15.100:0/2211131678 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb288018030 con 0x55a4f0ea61b0 2024-07-26T13:45:59.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.475+0000 7fb26effd640 1 -- 172.21.15.100:0/2211131678 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 47) ==== 8350+0+0 (secure 0 0 0) 0x7fb2800cba80 con 0x55a4f0ea61b0 2024-07-26T13:45:59.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.475+0000 7fb29c1af740 10 client.26395 fetch_fsmap finished waiting for FSMap version 47 2024-07-26T13:45:59.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.475+0000 7fb29c1af740 10 client.26395 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-26T13:45:59.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.475+0000 7fb29c1af740 20 client.26395 populate_metadata read hostname 'smithi100' 2024-07-26T13:45:59.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.475+0000 7fb29c1af740 1 --2- 172.21.15.100:0/2211131678 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55a4f0e62da0 0x55a4f12fcae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:45:59.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.475+0000 7fb29c1af740 4 client.26395 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-26T13:45:59.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.475+0000 7fb29c1af740 1 -- 172.21.15.100:0/2211131678 --> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a4f11e85a0 con 0x55a4f0e62da0 2024-07-26T13:45:59.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.475+0000 7fb28effd640 1 --2- 172.21.15.100:0/2211131678 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55a4f0e62da0 0x55a4f12fcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:45:59.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.476+0000 7fb26effd640 10 client.26395 ms_handle_connect on v2:172.21.15.138:6838/1580925471 2024-07-26T13:45:59.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.476+0000 7fb28effd640 1 --2- 172.21.15.100:0/2211131678 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55a4f0e62da0 0x55a4f12fcae0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5d49904e1c1d487b server_cookie=605ac4eacd02965d in_seq=0 out_seq=0 2024-07-26T13:45:59.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.476+0000 7fb26effd640 1 -- 172.21.15.100:0/2211131678 <== mds.1 v2:172.21.15.138:6838/1580925471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a4f11e85a0 con 0x55a4f0e62da0 2024-07-26T13:45:59.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.476+0000 7fb26effd640 10 client.26395 handle_command_reply: tid=0 2024-07-26T13:45:59.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.476+0000 7fb29c1af740 1 client.26395 shutdown 2024-07-26T13:45:59.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.476+0000 7fb29c1af740 1 -- 172.21.15.100:0/2211131678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a4f0ea61b0 msgr2=0x55a4f1292d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:59.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.476+0000 7fb29c1af740 1 --2- 172.21.15.100:0/2211131678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a4f0ea61b0 0x55a4f1292d30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb2800c8a90 tx=0x7fb280003820 comp rx=0 tx=0).stop 2024-07-26T13:45:59.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.476+0000 7fb29c1af740 1 -- 172.21.15.100:0/2211131678 shutdown_connections 2024-07-26T13:45:59.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.476+0000 7fb29c1af740 1 --2- 172.21.15.100:0/2211131678 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55a4f0e62da0 0x55a4f12fcae0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:59.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.476+0000 7fb29c1af740 1 --2- 172.21.15.100:0/2211131678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a4f0ecf070 0x55a4f12f2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:59.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.476+0000 7fb29c1af740 1 --2- 172.21.15.100:0/2211131678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4f0eae390 0x55a4f1293310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:59.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.476+0000 7fb29c1af740 1 --2- 172.21.15.100:0/2211131678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a4f0ea61b0 0x55a4f1292d30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:59.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.476+0000 7fb29c1af740 1 -- 172.21.15.100:0/2211131678 >> 172.21.15.100:0/2211131678 conn(0x55a4f0ec3460 msgr2=0x55a4f1279680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:59.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.476+0000 7fb26effd640 0 client.26395 ms_handle_reset on v2:172.21.15.138:6838/1580925471 2024-07-26T13:45:59.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.477+0000 7fb29c1af740 1 -- 172.21.15.100:0/2211131678 shutdown_connections 2024-07-26T13:45:59.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.477+0000 7fb29c1af740 1 -- 172.21.15.100:0/2211131678 wait complete. 2024-07-26T13:45:59.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.477+0000 7fb29c1af740 20 client.26395 trim_cache size 0 max 16384 2024-07-26T13:45:59.481 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-07-26T13:45:59.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.478+0000 7fb295c2b640 1 -- 172.21.15.100:0/3003013112 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb2780b3d70 msgr2=0x7fb2780b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:59.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.478+0000 7fb295c2b640 1 --2- 172.21.15.100:0/3003013112 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb2780b3d70 0x7fb2780b6280 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb288007ed0 tx=0x7fb2880075c0 comp rx=0 tx=0).stop 2024-07-26T13:45:59.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.478+0000 7fb295c2b640 1 -- 172.21.15.100:0/3003013112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb290075160 msgr2=0x7fb29019bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:45:59.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.478+0000 7fb295c2b640 1 --2- 172.21.15.100:0/3003013112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb290075160 0x7fb29019bf80 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fb280030940 tx=0x7fb280030970 comp rx=0 tx=0).stop 2024-07-26T13:45:59.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.478+0000 7fb295c2b640 1 -- 172.21.15.100:0/3003013112 shutdown_connections 2024-07-26T13:45:59.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.478+0000 7fb295c2b640 1 --2- 172.21.15.100:0/3003013112 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb2780b3d70 0x7fb2780b6280 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:59.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.478+0000 7fb295c2b640 1 --2- 172.21.15.100:0/3003013112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb29010fb70 0x7fb2901a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:59.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.478+0000 7fb295c2b640 1 --2- 172.21.15.100:0/3003013112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb290075ab0 0x7fb29019c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:59.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.478+0000 7fb295c2b640 1 --2- 172.21.15.100:0/3003013112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb290075160 0x7fb29019bf80 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:45:59.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.478+0000 7fb295c2b640 1 -- 172.21.15.100:0/3003013112 >> 172.21.15.100:0/3003013112 conn(0x7fb29007b280 msgr2=0x7fb2901075d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:45:59.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.478+0000 7fb295c2b640 1 -- 172.21.15.100:0/3003013112 shutdown_connections 2024-07-26T13:45:59.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:45:59.478+0000 7fb295c2b640 1 -- 172.21.15.100:0/3003013112 wait complete. 2024-07-26T13:45:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:45:59 smithi100 ceph-mon[32599]: pgmap v542: 97 pgs: 97 active+clean; 92 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 192 op/s 2024-07-26T13:45:59.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:45:59 smithi138 ceph-mon[36954]: pgmap v542: 97 pgs: 97 active+clean; 92 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 192 op/s 2024-07-26T13:46:00.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:45:59 smithi167 ceph-mon[37140]: pgmap v542: 97 pgs: 97 active+clean; 92 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 192 op/s 2024-07-26T13:46:00.419 INFO:tasks.workunit.client.0.smithi100.stdout: 1 222738 3.08 MB/sec execute 134 sec latency 6841.909 ms 2024-07-26T13:46:00.863 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-26T13:46:01.419 INFO:tasks.workunit.client.0.smithi100.stdout: 1 222738 3.06 MB/sec execute 135 sec latency 7841.934 ms 2024-07-26T13:46:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.424+0000 7f186e49b640 1 -- 172.21.15.100:0/4208886532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18681023e0 msgr2=0x7f18681027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.424+0000 7f186e49b640 1 --2- 172.21.15.100:0/4208886532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18681023e0 0x7f18681027f0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f1858009900 tx=0x7f185802a410 comp rx=0 tx=0).stop 2024-07-26T13:46:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.424+0000 7f186e49b640 1 -- 172.21.15.100:0/4208886532 shutdown_connections 2024-07-26T13:46:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.424+0000 7f186e49b640 1 --2- 172.21.15.100:0/4208886532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1868108660 0x7f186810ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.424+0000 7f186e49b640 1 --2- 172.21.15.100:0/4208886532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1868107d10 0x7f1868108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.424+0000 7f186e49b640 1 --2- 172.21.15.100:0/4208886532 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18681023e0 0x7f18681027f0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.424+0000 7f186e49b640 1 -- 172.21.15.100:0/4208886532 >> 172.21.15.100:0/4208886532 conn(0x7f186807a170 msgr2=0x7f186807a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.424+0000 7f186e49b640 1 -- 172.21.15.100:0/4208886532 shutdown_connections 2024-07-26T13:46:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.425+0000 7f186e49b640 1 -- 172.21.15.100:0/4208886532 wait complete. 2024-07-26T13:46:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.426+0000 7f186e49b640 1 Processor -- start 2024-07-26T13:46:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.426+0000 7f186e49b640 1 -- start start 2024-07-26T13:46:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.426+0000 7f186e49b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18681023e0 0x7f1868199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.426+0000 7f186e49b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1868107d10 0x7f186819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.426+0000 7f186e49b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1868108660 0x7f186819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.426+0000 7f186e49b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1868113430 con 0x7f18681023e0 2024-07-26T13:46:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.426+0000 7f186e49b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f18681132b0 con 0x7f1868108660 2024-07-26T13:46:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.426+0000 7f186e49b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f18681135b0 con 0x7f1868107d10 2024-07-26T13:46:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.426+0000 7f1867fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18681023e0 0x7f1868199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.426+0000 7f1867fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18681023e0 0x7f1868199e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37550/0 (socket says 172.21.15.100:37550) 2024-07-26T13:46:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.427+0000 7f1867fff640 1 -- 172.21.15.100:0/3779820759 learned_addr learned my addr 172.21.15.100:0/3779820759 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.427+0000 7f18677fe640 1 --2- 172.21.15.100:0/3779820759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1868107d10 0x7f186819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.427+0000 7f186ca38640 1 --2- 172.21.15.100:0/3779820759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1868108660 0x7f186819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.427+0000 7f1867fff640 1 -- 172.21.15.100:0/3779820759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1868107d10 msgr2=0x7f186819a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.427+0000 7f1867fff640 1 --2- 172.21.15.100:0/3779820759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1868107d10 0x7f186819a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.427+0000 7f1867fff640 1 -- 172.21.15.100:0/3779820759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1868108660 msgr2=0x7f186819f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.427+0000 7f1867fff640 1 --2- 172.21.15.100:0/3779820759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1868108660 0x7f186819f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.427+0000 7f1867fff640 1 -- 172.21.15.100:0/3779820759 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f186819fdc0 con 0x7f18681023e0 2024-07-26T13:46:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.427+0000 7f1867fff640 1 --2- 172.21.15.100:0/3779820759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18681023e0 0x7f1868199e90 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f1858005940 tx=0x7f18580040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.427+0000 7f18657fa640 1 -- 172.21.15.100:0/3779820759 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185803f030 con 0x7f18681023e0 2024-07-26T13:46:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.428+0000 7f186e49b640 1 -- 172.21.15.100:0/3779820759 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18681a0050 con 0x7f18681023e0 2024-07-26T13:46:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.428+0000 7f18657fa640 1 -- 172.21.15.100:0/3779820759 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18580044c0 con 0x7f18681023e0 2024-07-26T13:46:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.428+0000 7f186e49b640 1 -- 172.21.15.100:0/3779820759 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18681a6840 con 0x7f18681023e0 2024-07-26T13:46:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.428+0000 7f18657fa640 1 -- 172.21.15.100:0/3779820759 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1858035070 con 0x7f18681023e0 2024-07-26T13:46:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.431+0000 7f18657fa640 1 -- 172.21.15.100:0/3779820759 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f1858042030 con 0x7f18681023e0 2024-07-26T13:46:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.431+0000 7f18753b8740 1 Processor -- start 2024-07-26T13:46:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.432+0000 7f18753b8740 1 -- start start 2024-07-26T13:46:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.432+0000 7f18657fa640 1 --2- 172.21.15.100:0/3779820759 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f18500b3d40 0x7f18500b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.432+0000 7f18753b8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617978cb1b0 0x561797c6dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.432+0000 7f18753b8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617978a22f0 0x561797c6e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.432+0000 7f18753b8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617978aa4d0 0x561797c6e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.432+0000 7f18753b8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x561797be33b0 con 0x5617978cb1b0 2024-07-26T13:46:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.432+0000 7f18753b8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561797bd28b0 con 0x5617978a22f0 2024-07-26T13:46:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.432+0000 7f18753b8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561797bd68f0 con 0x5617978aa4d0 2024-07-26T13:46:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.432+0000 7f18657fa640 1 -- 172.21.15.100:0/3779820759 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7f1858104040 con 0x7f18681023e0 2024-07-26T13:46:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.432+0000 7f186ca38640 1 --2- 172.21.15.100:0/3779820759 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f18500b3d40 0x7f18500b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.432+0000 7f1867fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617978cb1b0 0x561797c6dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.432+0000 7f1867fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617978cb1b0 0x561797c6dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37560/0 (socket says 172.21.15.100:37560) 2024-07-26T13:46:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.432+0000 7f1867fff640 1 -- 172.21.15.100:0/1204086304 learned_addr learned my addr 172.21.15.100:0/1204086304 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.433+0000 7f1867fff640 1 -- 172.21.15.100:0/1204086304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617978aa4d0 msgr2=0x561797c6e760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-26T13:46:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.433+0000 7f1867fff640 1 --2- 172.21.15.100:0/1204086304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617978aa4d0 0x561797c6e760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.433+0000 7f1867fff640 1 -- 172.21.15.100:0/1204086304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617978a22f0 msgr2=0x561797c6e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.433+0000 7f18677fe640 1 --2- 172.21.15.100:0/1204086304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617978a22f0 0x561797c6e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.433+0000 7f1867fff640 1 --2- 172.21.15.100:0/1204086304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617978a22f0 0x561797c6e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.433+0000 7f1867fff640 1 -- 172.21.15.100:0/1204086304 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561797b799a0 con 0x5617978cb1b0 2024-07-26T13:46:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.433+0000 7f1867fff640 1 --2- 172.21.15.100:0/1204086304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617978cb1b0 0x561797c6dce0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f1858032af0 tx=0x7f18580ca790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6982e2e079f02ade server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.433+0000 7f186ca38640 1 --2- 172.21.15.100:0/3779820759 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f18500b3d40 0x7f18500b6250 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1860005fd0 tx=0x7f1860009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.433+0000 7f184e7fc640 1 -- 172.21.15.100:0/1204086304 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1858101e60 con 0x5617978cb1b0 2024-07-26T13:46:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.433+0000 7f184e7fc640 1 -- 172.21.15.100:0/1204086304 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18580d3c80 con 0x5617978cb1b0 2024-07-26T13:46:01.436 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.433+0000 7f184e7fc640 1 -- 172.21.15.100:0/1204086304 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1858049070 con 0x5617978cb1b0 2024-07-26T13:46:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.434+0000 7f18753b8740 1 -- 172.21.15.100:0/1204086304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617978cb1b0 msgr2=0x561797c6dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.434+0000 7f18753b8740 1 --2- 172.21.15.100:0/1204086304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617978cb1b0 0x561797c6dce0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f1858032af0 tx=0x7f18580ca790 comp rx=0 tx=0).stop 2024-07-26T13:46:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.434+0000 7f18753b8740 1 -- 172.21.15.100:0/1204086304 shutdown_connections 2024-07-26T13:46:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.434+0000 7f18753b8740 1 --2- 172.21.15.100:0/1204086304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617978aa4d0 0x561797c6e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.434+0000 7f18753b8740 1 --2- 172.21.15.100:0/1204086304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617978a22f0 0x561797c6e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.434+0000 7f18753b8740 1 --2- 172.21.15.100:0/1204086304 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617978cb1b0 0x561797c6dce0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.434+0000 7f18753b8740 1 -- 172.21.15.100:0/1204086304 >> 172.21.15.100:0/1204086304 conn(0x5617978bf300 msgr2=0x561797c6bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.434+0000 7f18753b8740 1 -- 172.21.15.100:0/1204086304 shutdown_connections 2024-07-26T13:46:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.434+0000 7f18753b8740 1 -- 172.21.15.100:0/1204086304 wait complete. 2024-07-26T13:46:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.435+0000 7f18753b8740 1 -- start start 2024-07-26T13:46:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.435+0000 7f18753b8740 1 Processor -- start 2024-07-26T13:46:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.436+0000 7f18753b8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617978a22f0 0x561797c79960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.436+0000 7f18753b8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617978aa4d0 0x561797c79f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.436+0000 7f18753b8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617978cb1b0 0x561797c85400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.436+0000 7f18753b8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x561797b0cab0 con 0x5617978a22f0 2024-07-26T13:46:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.436+0000 7f18753b8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561797b5acf0 con 0x5617978cb1b0 2024-07-26T13:46:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.436+0000 7f18753b8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561797c7c220 con 0x5617978aa4d0 2024-07-26T13:46:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.436+0000 7f18753b8740 10 client.0 _ll_register_callbacks cb 0x561797bcc9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:46:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.436+0000 7f1867fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617978a22f0 0x561797c79960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.436+0000 7f1867fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617978a22f0 0x561797c79960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37562/0 (socket says 172.21.15.100:37562) 2024-07-26T13:46:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.436+0000 7f1867fff640 1 -- 172.21.15.100:0/2471570704 learned_addr learned my addr 172.21.15.100:0/2471570704 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.436+0000 7f186ca38640 1 --2- 172.21.15.100:0/2471570704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617978cb1b0 0x561797c85400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.436+0000 7f18677fe640 1 --2- 172.21.15.100:0/2471570704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617978aa4d0 0x561797c79f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.436+0000 7f1867fff640 1 -- 172.21.15.100:0/2471570704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617978aa4d0 msgr2=0x561797c79f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.436+0000 7f1867fff640 1 --2- 172.21.15.100:0/2471570704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617978aa4d0 0x561797c79f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.436+0000 7f1867fff640 1 -- 172.21.15.100:0/2471570704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617978cb1b0 msgr2=0x561797c85400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.436+0000 7f1867fff640 1 --2- 172.21.15.100:0/2471570704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617978cb1b0 0x561797c85400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.436+0000 7f1867fff640 1 -- 172.21.15.100:0/2471570704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561797936790 con 0x5617978a22f0 2024-07-26T13:46:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.436+0000 7f1867fff640 1 --2- 172.21.15.100:0/2471570704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617978a22f0 0x561797c79960 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f185802a450 tx=0x7f1858101bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe2d7714bd2c443 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.436+0000 7f18477fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:46:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.437+0000 7f18477fe640 1 -- 172.21.15.100:0/2471570704 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1858101e60 con 0x5617978a22f0 2024-07-26T13:46:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.437+0000 7f18477fe640 1 -- 172.21.15.100:0/2471570704 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7f18580cb0f0 con 0x5617978a22f0 2024-07-26T13:46:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.437+0000 7f18753b8740 1 -- 172.21.15.100:0/2471570704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18580d3c80 con 0x5617978a22f0 2024-07-26T13:46:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.437+0000 7f18477fe640 1 -- 172.21.15.100:0/2471570704 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f1858049040 con 0x5617978a22f0 2024-07-26T13:46:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.437+0000 7f18753b8740 10 client.16980 fetch_fsmap learned FSMap version 47 2024-07-26T13:46:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.437+0000 7f18753b8740 1 -- 172.21.15.100:0/2471570704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5617979af620 con 0x5617978a22f0 2024-07-26T13:46:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.437+0000 7f18477fe640 1 -- 172.21.15.100:0/2471570704 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18580d3c80 con 0x5617978a22f0 2024-07-26T13:46:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.437+0000 7f18477fe640 1 -- 172.21.15.100:0/2471570704 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 47) ==== 8350+0+0 (secure 0 0 0) 0x7f185804c060 con 0x5617978a22f0 2024-07-26T13:46:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.437+0000 7f18753b8740 10 client.16980 fetch_fsmap finished waiting for FSMap version 47 2024-07-26T13:46:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.437+0000 7f18753b8740 10 client.16980 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:46:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.437+0000 7f18753b8740 20 client.16980 populate_metadata read hostname 'smithi100' 2024-07-26T13:46:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.437+0000 7f18753b8740 1 --2- 172.21.15.100:0/2471570704 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x561797889d80 0x561797c8b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.438+0000 7f18753b8740 4 client.16980 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:46:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.438+0000 7f18753b8740 1 -- 172.21.15.100:0/2471570704 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561797be5300 con 0x561797889d80 2024-07-26T13:46:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.438+0000 7f18677fe640 1 --2- 172.21.15.100:0/2471570704 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x561797889d80 0x561797c8b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.438+0000 7f18477fe640 10 client.16980 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.438+0000 7f18677fe640 1 --2- 172.21.15.100:0/2471570704 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x561797889d80 0x561797c8b370 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc3d2f21b8d1dab3 server_cookie=9671bef8319cc332 in_seq=0 out_seq=0 2024-07-26T13:46:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.439+0000 7f18477fe640 1 -- 172.21.15.100:0/2471570704 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561797be5300 con 0x561797889d80 2024-07-26T13:46:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.439+0000 7f18477fe640 10 client.16980 handle_command_reply: tid=0 2024-07-26T13:46:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.439+0000 7f18753b8740 1 client.16980 shutdown 2024-07-26T13:46:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.439+0000 7f18753b8740 1 -- 172.21.15.100:0/2471570704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617978a22f0 msgr2=0x561797c79960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.439+0000 7f18753b8740 1 --2- 172.21.15.100:0/2471570704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617978a22f0 0x561797c79960 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f185802a450 tx=0x7f1858101bc0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.439+0000 7f18753b8740 1 -- 172.21.15.100:0/2471570704 shutdown_connections 2024-07-26T13:46:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.440+0000 7f18753b8740 1 --2- 172.21.15.100:0/2471570704 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x561797889d80 0x561797c8b370 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.440+0000 7f18753b8740 1 --2- 172.21.15.100:0/2471570704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617978cb1b0 0x561797c85400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.440+0000 7f18753b8740 1 --2- 172.21.15.100:0/2471570704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617978aa4d0 0x561797c79f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.440+0000 7f18753b8740 1 --2- 172.21.15.100:0/2471570704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617978a22f0 0x561797c79960 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.440+0000 7f18477fe640 0 client.16980 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.440+0000 7f18753b8740 1 -- 172.21.15.100:0/2471570704 >> 172.21.15.100:0/2471570704 conn(0x5617978bf300 msgr2=0x561797c6a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.440+0000 7f18753b8740 1 -- 172.21.15.100:0/2471570704 shutdown_connections 2024-07-26T13:46:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.440+0000 7f18753b8740 1 -- 172.21.15.100:0/2471570704 wait complete. 2024-07-26T13:46:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.440+0000 7f18753b8740 20 client.16980 trim_cache size 0 max 16384 2024-07-26T13:46:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.455+0000 7f18753b8740 1 Processor -- start 2024-07-26T13:46:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.455+0000 7f18753b8740 1 -- start start 2024-07-26T13:46:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.456+0000 7f18753b8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617978a22f0 0x561797c734b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.456+0000 7f18753b8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617978aa4d0 0x561797c85270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.456+0000 7f18753b8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617978cb1b0 0x561797c6b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.456+0000 7f18753b8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f18681b1b90 con 0x5617978aa4d0 2024-07-26T13:46:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.456+0000 7f18753b8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f18681af430 con 0x5617978a22f0 2024-07-26T13:46:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.456+0000 7f18753b8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f18681ad260 con 0x5617978cb1b0 2024-07-26T13:46:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.456+0000 7f18677fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617978aa4d0 0x561797c85270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.456+0000 7f18677fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617978aa4d0 0x561797c85270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37570/0 (socket says 172.21.15.100:37570) 2024-07-26T13:46:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.456+0000 7f18677fe640 1 -- 172.21.15.100:0/2316882586 learned_addr learned my addr 172.21.15.100:0/2316882586 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.456+0000 7f186ca38640 1 --2- 172.21.15.100:0/2316882586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617978cb1b0 0x561797c6b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.456+0000 7f1867fff640 1 --2- 172.21.15.100:0/2316882586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617978a22f0 0x561797c734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.456+0000 7f18677fe640 1 -- 172.21.15.100:0/2316882586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617978cb1b0 msgr2=0x561797c6b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.456+0000 7f18677fe640 1 --2- 172.21.15.100:0/2316882586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617978cb1b0 0x561797c6b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.456+0000 7f18677fe640 1 -- 172.21.15.100:0/2316882586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617978a22f0 msgr2=0x561797c734b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.456+0000 7f18677fe640 1 --2- 172.21.15.100:0/2316882586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617978a22f0 0x561797c734b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.456+0000 7f18677fe640 1 -- 172.21.15.100:0/2316882586 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561797c74fa0 con 0x5617978aa4d0 2024-07-26T13:46:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.456+0000 7f18677fe640 1 --2- 172.21.15.100:0/2316882586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617978aa4d0 0x561797c85270 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f1868104230 tx=0x7f185c012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e1c5e291864902a server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.457+0000 7f184dffb640 1 -- 172.21.15.100:0/2316882586 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c004230 con 0x5617978aa4d0 2024-07-26T13:46:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.457+0000 7f184dffb640 1 -- 172.21.15.100:0/2316882586 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f185c0043d0 con 0x5617978aa4d0 2024-07-26T13:46:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.457+0000 7f184dffb640 1 -- 172.21.15.100:0/2316882586 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c01e8f0 con 0x5617978aa4d0 2024-07-26T13:46:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.457+0000 7f18753b8740 1 -- 172.21.15.100:0/2316882586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617978aa4d0 msgr2=0x561797c85270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.458+0000 7f18753b8740 1 --2- 172.21.15.100:0/2316882586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617978aa4d0 0x561797c85270 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f1868104230 tx=0x7f185c012820 comp rx=0 tx=0).stop 2024-07-26T13:46:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.458+0000 7f18753b8740 1 -- 172.21.15.100:0/2316882586 shutdown_connections 2024-07-26T13:46:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.458+0000 7f18753b8740 1 --2- 172.21.15.100:0/2316882586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617978cb1b0 0x561797c6b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.458+0000 7f18753b8740 1 --2- 172.21.15.100:0/2316882586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617978aa4d0 0x561797c85270 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.458+0000 7f18753b8740 1 --2- 172.21.15.100:0/2316882586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617978a22f0 0x561797c734b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.458+0000 7f18753b8740 1 -- 172.21.15.100:0/2316882586 >> 172.21.15.100:0/2316882586 conn(0x5617978bf300 msgr2=0x561797c72da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.458+0000 7f18753b8740 1 -- 172.21.15.100:0/2316882586 shutdown_connections 2024-07-26T13:46:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.458+0000 7f18753b8740 1 -- 172.21.15.100:0/2316882586 wait complete. 2024-07-26T13:46:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.458+0000 7f18753b8740 1 -- start start 2024-07-26T13:46:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.458+0000 7f18753b8740 1 Processor -- start 2024-07-26T13:46:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.458+0000 7f18753b8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617978a22f0 0x561797c7e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.458+0000 7f18753b8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617978aa4d0 0x561797c7e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.458+0000 7f18753b8740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617978cb1b0 0x561797c7edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.458+0000 7f18753b8740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f18681ad8f0 con 0x5617978cb1b0 2024-07-26T13:46:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.458+0000 7f18753b8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f18681aed60 con 0x5617978aa4d0 2024-07-26T13:46:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.458+0000 7f18753b8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f18681adfc0 con 0x5617978a22f0 2024-07-26T13:46:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.459+0000 7f18753b8740 10 client.0 _ll_register_callbacks cb 0x7f18580ce070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:46:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.459+0000 7f186ca38640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617978cb1b0 0x561797c7edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.459+0000 7f186ca38640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617978cb1b0 0x561797c7edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37586/0 (socket says 172.21.15.100:37586) 2024-07-26T13:46:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.459+0000 7f186ca38640 1 -- 172.21.15.100:0/3325404085 learned_addr learned my addr 172.21.15.100:0/3325404085 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.459+0000 7f18677fe640 1 --2- 172.21.15.100:0/3325404085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617978aa4d0 0x561797c7e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.459+0000 7f1867fff640 1 --2- 172.21.15.100:0/3325404085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617978a22f0 0x561797c7e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:01.467 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.459+0000 7f186ca38640 1 -- 172.21.15.100:0/3325404085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617978a22f0 msgr2=0x561797c7e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.459+0000 7f186ca38640 1 --2- 172.21.15.100:0/3325404085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617978a22f0 0x561797c7e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.459+0000 7f186ca38640 1 -- 172.21.15.100:0/3325404085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617978aa4d0 msgr2=0x561797c7e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.459+0000 7f186ca38640 1 --2- 172.21.15.100:0/3325404085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617978aa4d0 0x561797c7e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.459+0000 7f186ca38640 1 -- 172.21.15.100:0/3325404085 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561797cf1940 con 0x5617978cb1b0 2024-07-26T13:46:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.459+0000 7f18677fe640 1 --2- 172.21.15.100:0/3325404085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617978aa4d0 0x561797c7e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:46:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.459+0000 7f1867fff640 1 --2- 172.21.15.100:0/3325404085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617978a22f0 0x561797c7e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:46:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.459+0000 7f186ca38640 1 --2- 172.21.15.100:0/3325404085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617978cb1b0 0x561797c7edc0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f18600042c0 tx=0x7f1860004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da3b095fcae04617 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.459+0000 7f18477fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:46:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.459+0000 7f18477fe640 1 -- 172.21.15.100:0/3325404085 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1860004b20 con 0x5617978cb1b0 2024-07-26T13:46:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.459+0000 7f18477fe640 1 -- 172.21.15.100:0/3325404085 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7f1860009040 con 0x5617978cb1b0 2024-07-26T13:46:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.459+0000 7f18753b8740 1 -- 172.21.15.100:0/3325404085 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f185c0043d0 con 0x5617978cb1b0 2024-07-26T13:46:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.459+0000 7f18477fe640 1 -- 172.21.15.100:0/3325404085 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f1860034050 con 0x5617978cb1b0 2024-07-26T13:46:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.459+0000 7f18753b8740 10 client.16986 fetch_fsmap learned FSMap version 47 2024-07-26T13:46:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.459+0000 7f18753b8740 1 -- 172.21.15.100:0/3325404085 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x561797cf2c30 con 0x5617978cb1b0 2024-07-26T13:46:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.460+0000 7f18477fe640 1 -- 172.21.15.100:0/3325404085 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f185c0043d0 con 0x5617978cb1b0 2024-07-26T13:46:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.460+0000 7f18477fe640 1 -- 172.21.15.100:0/3325404085 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 47) ==== 8350+0+0 (secure 0 0 0) 0x7f186001d770 con 0x5617978cb1b0 2024-07-26T13:46:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.460+0000 7f18753b8740 10 client.16986 fetch_fsmap finished waiting for FSMap version 47 2024-07-26T13:46:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.460+0000 7f18753b8740 10 client.16986 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:46:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.460+0000 7f18753b8740 20 client.16986 populate_metadata read hostname 'smithi100' 2024-07-26T13:46:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.460+0000 7f18753b8740 1 --2- 172.21.15.100:0/3325404085 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56179785f1c0 0x561797cf54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.460+0000 7f18753b8740 4 client.16986 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-26T13:46:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.460+0000 7f18753b8740 1 -- 172.21.15.100:0/3325404085 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561797be0da0 con 0x56179785f1c0 2024-07-26T13:46:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.460+0000 7f1867fff640 1 --2- 172.21.15.100:0/3325404085 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56179785f1c0 0x561797cf54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.461+0000 7f1867fff640 1 --2- 172.21.15.100:0/3325404085 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56179785f1c0 0x561797cf54a0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0e744961af87c4 server_cookie=aa8d649ba75d6395 in_seq=0 out_seq=0 2024-07-26T13:46:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:01.461+0000 7f18477fe640 10 client.16986 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:01.683 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:01 smithi100 ceph-mon[32599]: pgmap v543: 97 pgs: 97 active+clean; 90 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 170 op/s 2024-07-26T13:46:01.930 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:46:01.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.934+0000 7fc812117640 1 -- 172.21.15.100:0/949169875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc80c107ec0 msgr2=0x7fc80c10e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:01.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.934+0000 7fc812117640 1 --2- 172.21.15.100:0/949169875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc80c107ec0 0x7fc80c10e6f0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fc80400adb0 tx=0x7fc80402b7a0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.934+0000 7fc812117640 1 -- 172.21.15.100:0/949169875 shutdown_connections 2024-07-26T13:46:01.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.934+0000 7fc812117640 1 --2- 172.21.15.100:0/949169875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc80c107ec0 0x7fc80c10e6f0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.934+0000 7fc812117640 1 --2- 172.21.15.100:0/949169875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc80c107570 0x7fc80c107980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.934+0000 7fc812117640 1 --2- 172.21.15.100:0/949169875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc80c101c30 0x7fc80c102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.934+0000 7fc812117640 1 -- 172.21.15.100:0/949169875 >> 172.21.15.100:0/949169875 conn(0x7fc80c079980 msgr2=0x7fc80c079d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:01.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.934+0000 7fc812117640 1 -- 172.21.15.100:0/949169875 shutdown_connections 2024-07-26T13:46:01.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.934+0000 7fc812117640 1 -- 172.21.15.100:0/949169875 wait complete. 2024-07-26T13:46:01.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.935+0000 7fc812117640 1 Processor -- start 2024-07-26T13:46:01.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.935+0000 7fc812117640 1 -- start start 2024-07-26T13:46:01.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.936+0000 7fc812117640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc80c101c30 0x7fc80c199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:01.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.936+0000 7fc812117640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc80c107570 0x7fc80c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:01.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.936+0000 7fc812117640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc80c107ec0 0x7fc80c19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:01.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.936+0000 7fc812117640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c111300 con 0x7fc80c107ec0 2024-07-26T13:46:01.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.936+0000 7fc812117640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c111180 con 0x7fc80c101c30 2024-07-26T13:46:01.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.936+0000 7fc812117640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c111480 con 0x7fc80c107570 2024-07-26T13:46:01.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.936+0000 7fc80bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc80c107ec0 0x7fc80c19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:01.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.936+0000 7fc80bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc80c107ec0 0x7fc80c19f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37596/0 (socket says 172.21.15.100:37596) 2024-07-26T13:46:01.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.936+0000 7fc80bfff640 1 -- 172.21.15.100:0/3519124204 learned_addr learned my addr 172.21.15.100:0/3519124204 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:01.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.936+0000 7fc80affd640 1 --2- 172.21.15.100:0/3519124204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc80c107570 0x7fc80c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:01.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.936+0000 7fc80b7fe640 1 --2- 172.21.15.100:0/3519124204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc80c101c30 0x7fc80c199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:01.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.936+0000 7fc80bfff640 1 -- 172.21.15.100:0/3519124204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc80c107570 msgr2=0x7fc80c19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:01.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.936+0000 7fc80bfff640 1 --2- 172.21.15.100:0/3519124204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc80c107570 0x7fc80c19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.936+0000 7fc80bfff640 1 -- 172.21.15.100:0/3519124204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc80c101c30 msgr2=0x7fc80c199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:01.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.936+0000 7fc80bfff640 1 --2- 172.21.15.100:0/3519124204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc80c101c30 0x7fc80c199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.936+0000 7fc80bfff640 1 -- 172.21.15.100:0/3519124204 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc80c19fe70 con 0x7fc80c107ec0 2024-07-26T13:46:01.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.936+0000 7fc80affd640 1 --2- 172.21.15.100:0/3519124204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc80c107570 0x7fc80c19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:46:01.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.937+0000 7fc80bfff640 1 --2- 172.21.15.100:0/3519124204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc80c107ec0 0x7fc80c19f740 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fc804031040 tx=0x7fc804003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:01.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.937+0000 7fc808ff9640 1 -- 172.21.15.100:0/3519124204 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804040070 con 0x7fc80c107ec0 2024-07-26T13:46:01.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.937+0000 7fc812117640 1 -- 172.21.15.100:0/3519124204 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc80c1a0100 con 0x7fc80c107ec0 2024-07-26T13:46:01.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.937+0000 7fc812117640 1 -- 172.21.15.100:0/3519124204 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc80c197a00 con 0x7fc80c107ec0 2024-07-26T13:46:01.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.937+0000 7fc808ff9640 1 -- 172.21.15.100:0/3519124204 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc804032810 con 0x7fc80c107ec0 2024-07-26T13:46:01.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.937+0000 7fc808ff9640 1 -- 172.21.15.100:0/3519124204 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804004750 con 0x7fc80c107ec0 2024-07-26T13:46:01.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:01 smithi138 ceph-mon[36954]: pgmap v543: 97 pgs: 97 active+clean; 90 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 170 op/s 2024-07-26T13:46:01.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.940+0000 7fc81869b740 1 Processor -- start 2024-07-26T13:46:01.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.941+0000 7fc808ff9640 1 -- 172.21.15.100:0/3519124204 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fc804030510 con 0x7fc80c107ec0 2024-07-26T13:46:01.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.941+0000 7fc81869b740 1 -- start start 2024-07-26T13:46:01.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.941+0000 7fc81869b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8386b3850 0x55d838a59540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:01.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.941+0000 7fc81869b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d83868a990 0x55d838a59a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:01.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.941+0000 7fc81869b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d838692b70 0x55d838a59fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:01.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.941+0000 7fc81869b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55d8389ce6c0 con 0x55d8386b3850 2024-07-26T13:46:01.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.941+0000 7fc81869b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d8389bdbc0 con 0x55d83868a990 2024-07-26T13:46:01.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.941+0000 7fc81869b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d8389c1c00 con 0x55d838692b70 2024-07-26T13:46:01.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.941+0000 7fc80affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8386b3850 0x55d838a59540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:01.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.942+0000 7fc80affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8386b3850 0x55d838a59540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37612/0 (socket says 172.21.15.100:37612) 2024-07-26T13:46:01.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.942+0000 7fc80affd640 1 -- 172.21.15.100:0/1513785883 learned_addr learned my addr 172.21.15.100:0/1513785883 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:01.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.942+0000 7fc80affd640 1 --2- 172.21.15.100:0/1513785883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d838692b70 0x55d838a59fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:01.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.942+0000 7fc80affd640 1 -- 172.21.15.100:0/1513785883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d838692b70 msgr2=0x55d838a59fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:01.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.942+0000 7fc80affd640 1 --2- 172.21.15.100:0/1513785883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d838692b70 0x55d838a59fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.942+0000 7fc80affd640 1 -- 172.21.15.100:0/1513785883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d83868a990 msgr2=0x55d838a59a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-26T13:46:01.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.942+0000 7fc808ff9640 1 --2- 172.21.15.100:0/3519124204 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc7f40b3dc0 0x7fc7f40b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:01.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.942+0000 7fc808ff9640 1 -- 172.21.15.100:0/3519124204 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7fc804105270 con 0x7fc80c107ec0 2024-07-26T13:46:01.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.942+0000 7fc80b7fe640 1 --2- 172.21.15.100:0/1513785883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d83868a990 0x55d838a59a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:01.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.942+0000 7fc80affd640 1 --2- 172.21.15.100:0/1513785883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d83868a990 0x55d838a59a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.942+0000 7fc80affd640 1 -- 172.21.15.100:0/1513785883 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d838949a40 con 0x55d8386b3850 2024-07-26T13:46:01.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.943+0000 7fc80b7fe640 1 --2- 172.21.15.100:0/1513785883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d83868a990 0x55d838a59a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:46:01.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.943+0000 7fc80affd640 1 --2- 172.21.15.100:0/3519124204 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc7f40b3dc0 0x7fc7f40b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:01.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.943+0000 7fc80affd640 1 --2- 172.21.15.100:0/1513785883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8386b3850 0x55d838a59540 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fc800012850 tx=0x7fc80000bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29706c3f01e43504 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:01.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.943+0000 7fc7f1ffb640 1 -- 172.21.15.100:0/1513785883 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80000ead0 con 0x55d8386b3850 2024-07-26T13:46:01.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.943+0000 7fc7f1ffb640 1 -- 172.21.15.100:0/1513785883 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc80000ee70 con 0x55d8386b3850 2024-07-26T13:46:01.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.943+0000 7fc7f1ffb640 1 -- 172.21.15.100:0/1513785883 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc800027070 con 0x55d8386b3850 2024-07-26T13:46:01.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.944+0000 7fc81869b740 1 -- 172.21.15.100:0/1513785883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8386b3850 msgr2=0x55d838a59540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:01.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.944+0000 7fc80affd640 1 --2- 172.21.15.100:0/3519124204 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc7f40b3dc0 0x7fc7f40b62d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc800020000 tx=0x7fc80001d000 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:01.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.944+0000 7fc81869b740 1 --2- 172.21.15.100:0/1513785883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8386b3850 0x55d838a59540 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fc800012850 tx=0x7fc80000bb10 comp rx=0 tx=0).stop 2024-07-26T13:46:01.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.944+0000 7fc81869b740 1 -- 172.21.15.100:0/1513785883 shutdown_connections 2024-07-26T13:46:01.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.944+0000 7fc81869b740 1 --2- 172.21.15.100:0/1513785883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d838692b70 0x55d838a59fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.944+0000 7fc81869b740 1 --2- 172.21.15.100:0/1513785883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d83868a990 0x55d838a59a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.944+0000 7fc81869b740 1 --2- 172.21.15.100:0/1513785883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8386b3850 0x55d838a59540 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.944+0000 7fc81869b740 1 -- 172.21.15.100:0/1513785883 >> 172.21.15.100:0/1513785883 conn(0x55d8386a7c40 msgr2=0x55d838a57430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:01.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.944+0000 7fc81869b740 1 -- 172.21.15.100:0/1513785883 shutdown_connections 2024-07-26T13:46:01.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.944+0000 7fc81869b740 1 -- 172.21.15.100:0/1513785883 wait complete. 2024-07-26T13:46:01.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.945+0000 7fc81869b740 1 -- start start 2024-07-26T13:46:01.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.945+0000 7fc81869b740 1 Processor -- start 2024-07-26T13:46:01.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.945+0000 7fc81869b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d83868a990 0x55d838a64bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:01.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.945+0000 7fc81869b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d838692b70 0x55d838a65100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:01.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.945+0000 7fc81869b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8386b3850 0x55d838a71060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:01.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.945+0000 7fc81869b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55d8388217b0 con 0x55d83868a990 2024-07-26T13:46:01.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.945+0000 7fc81869b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d838960240 con 0x55d838692b70 2024-07-26T13:46:01.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.945+0000 7fc81869b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d838a67ce0 con 0x55d8386b3850 2024-07-26T13:46:01.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.945+0000 7fc81869b740 10 client.0 _ll_register_callbacks cb 0x55d8389412f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:46:01.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.946+0000 7fc80b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d83868a990 0x55d838a64bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:01.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.946+0000 7fc80b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d83868a990 0x55d838a64bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37616/0 (socket says 172.21.15.100:37616) 2024-07-26T13:46:01.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.946+0000 7fc80b7fe640 1 -- 172.21.15.100:0/3273684292 learned_addr learned my addr 172.21.15.100:0/3273684292 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:01.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.946+0000 7fc80affd640 1 --2- 172.21.15.100:0/3273684292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d838692b70 0x55d838a65100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:01.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.946+0000 7fc80bfff640 1 --2- 172.21.15.100:0/3273684292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8386b3850 0x55d838a71060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:01.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.946+0000 7fc80b7fe640 1 -- 172.21.15.100:0/3273684292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8386b3850 msgr2=0x55d838a71060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:01.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.946+0000 7fc80b7fe640 1 --2- 172.21.15.100:0/3273684292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8386b3850 0x55d838a71060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.946+0000 7fc80b7fe640 1 -- 172.21.15.100:0/3273684292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d838692b70 msgr2=0x55d838a65100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:01.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.946+0000 7fc80b7fe640 1 --2- 172.21.15.100:0/3273684292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d838692b70 0x55d838a65100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.946+0000 7fc80b7fe640 1 -- 172.21.15.100:0/3273684292 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d838971740 con 0x55d83868a990 2024-07-26T13:46:01.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.946+0000 7fc80b7fe640 1 --2- 172.21.15.100:0/3273684292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d83868a990 0x55d838a64bc0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc00b910 tx=0x7fc7fc00bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b649f9930c1b25aa server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:01.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.946+0000 7fc7eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:46:01.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.947+0000 7fc7eaffd640 1 -- 172.21.15.100:0/3273684292 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc005220 con 0x55d83868a990 2024-07-26T13:46:01.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.947+0000 7fc7eaffd640 1 -- 172.21.15.100:0/3273684292 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7fc7fc0053c0 con 0x55d83868a990 2024-07-26T13:46:01.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.947+0000 7fc81869b740 1 -- 172.21.15.100:0/3273684292 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d8389d0610 con 0x55d83868a990 2024-07-26T13:46:01.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.947+0000 7fc7eaffd640 1 -- 172.21.15.100:0/3273684292 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fc7fc0055c0 con 0x55d83868a990 2024-07-26T13:46:01.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.947+0000 7fc81869b740 10 client.16998 fetch_fsmap learned FSMap version 47 2024-07-26T13:46:01.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.947+0000 7fc81869b740 1 -- 172.21.15.100:0/3273684292 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55d83872dba0 con 0x55d83868a990 2024-07-26T13:46:01.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.947+0000 7fc7eaffd640 1 -- 172.21.15.100:0/3273684292 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d8389d0610 con 0x55d83868a990 2024-07-26T13:46:01.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.948+0000 7fc7eaffd640 1 -- 172.21.15.100:0/3273684292 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 47) ==== 8350+0+0 (secure 0 0 0) 0x7fc7fc012810 con 0x55d83868a990 2024-07-26T13:46:01.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.948+0000 7fc81869b740 10 client.16998 fetch_fsmap finished waiting for FSMap version 47 2024-07-26T13:46:01.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.948+0000 7fc81869b740 10 client.16998 resolve_mds: validated gid 24607 aka daemon mds.i 2024-07-26T13:46:01.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.948+0000 7fc81869b740 20 client.16998 populate_metadata read hostname 'smithi100' 2024-07-26T13:46:01.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.948+0000 7fc81869b740 1 --2- 172.21.15.100:0/3273684292 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55d8386721d0 0x55d838a76e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:01.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.948+0000 7fc81869b740 4 client.16998 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:46:01.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.948+0000 7fc81869b740 1 -- 172.21.15.100:0/3273684292 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8389cc0b0 con 0x55d8386721d0 2024-07-26T13:46:01.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.948+0000 7fc80affd640 1 --2- 172.21.15.100:0/3273684292 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55d8386721d0 0x55d838a76e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:01.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.948+0000 7fc7eaffd640 10 client.16998 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:01.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.948+0000 7fc80affd640 1 --2- 172.21.15.100:0/3273684292 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55d8386721d0 0x55d838a76e40 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2bb5692a7775615 server_cookie=1058bd20a26e9f50 in_seq=0 out_seq=0 2024-07-26T13:46:01.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.949+0000 7fc7eaffd640 1 -- 172.21.15.100:0/3273684292 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d8389cc0b0 con 0x55d8386721d0 2024-07-26T13:46:01.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.949+0000 7fc7eaffd640 10 client.16998 handle_command_reply: tid=0 2024-07-26T13:46:01.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.949+0000 7fc81869b740 1 client.16998 shutdown 2024-07-26T13:46:01.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.949+0000 7fc81869b740 1 -- 172.21.15.100:0/3273684292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d83868a990 msgr2=0x55d838a64bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:01.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.950+0000 7fc81869b740 1 --2- 172.21.15.100:0/3273684292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d83868a990 0x55d838a64bc0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc00b910 tx=0x7fc7fc00bdd0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.950+0000 7fc81869b740 1 -- 172.21.15.100:0/3273684292 shutdown_connections 2024-07-26T13:46:01.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.950+0000 7fc81869b740 1 --2- 172.21.15.100:0/3273684292 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55d8386721d0 0x55d838a76e40 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.950+0000 7fc81869b740 1 --2- 172.21.15.100:0/3273684292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8386b3850 0x55d838a71060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.950+0000 7fc81869b740 1 --2- 172.21.15.100:0/3273684292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d838692b70 0x55d838a65100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.950+0000 7fc81869b740 1 --2- 172.21.15.100:0/3273684292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d83868a990 0x55d838a64bc0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.950+0000 7fc7eaffd640 0 client.16998 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:01.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.950+0000 7fc81869b740 1 -- 172.21.15.100:0/3273684292 >> 172.21.15.100:0/3273684292 conn(0x55d8386a7c40 msgr2=0x55d838a5c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:01.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.950+0000 7fc81869b740 1 -- 172.21.15.100:0/3273684292 shutdown_connections 2024-07-26T13:46:01.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.950+0000 7fc81869b740 1 -- 172.21.15.100:0/3273684292 wait complete. 2024-07-26T13:46:01.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.950+0000 7fc81869b740 20 client.16998 trim_cache size 0 max 16384 2024-07-26T13:46:01.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.968+0000 7fc81869b740 1 Processor -- start 2024-07-26T13:46:01.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.968+0000 7fc81869b740 1 -- start start 2024-07-26T13:46:01.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.969+0000 7fc81869b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8386a7c40 0x55d838a7c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:01.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.969+0000 7fc81869b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8386721d0 0x55d838a7cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:01.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.969+0000 7fc81869b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d83868a990 0x55d838a6cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:01.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.969+0000 7fc81869b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f80111a0 con 0x55d8386a7c40 2024-07-26T13:46:01.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.969+0000 7fc81869b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f800ea40 con 0x55d8386721d0 2024-07-26T13:46:01.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.969+0000 7fc81869b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f800c870 con 0x55d83868a990 2024-07-26T13:46:01.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.969+0000 7fc80b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8386a7c40 0x55d838a7c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:01.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.969+0000 7fc80b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8386a7c40 0x55d838a7c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37632/0 (socket says 172.21.15.100:37632) 2024-07-26T13:46:01.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.969+0000 7fc80b7fe640 1 -- 172.21.15.100:0/2001974420 learned_addr learned my addr 172.21.15.100:0/2001974420 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:01.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.969+0000 7fc80bfff640 1 --2- 172.21.15.100:0/2001974420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d83868a990 0x55d838a6cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:01.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.969+0000 7fc80affd640 1 --2- 172.21.15.100:0/2001974420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8386721d0 0x55d838a7cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:01.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.969+0000 7fc80b7fe640 1 -- 172.21.15.100:0/2001974420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d83868a990 msgr2=0x55d838a6cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:01.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.969+0000 7fc80b7fe640 1 --2- 172.21.15.100:0/2001974420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d83868a990 0x55d838a6cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.969+0000 7fc80b7fe640 1 -- 172.21.15.100:0/2001974420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8386721d0 msgr2=0x55d838a7cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:01.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.969+0000 7fc80b7fe640 1 --2- 172.21.15.100:0/2001974420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8386721d0 0x55d838a7cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.969+0000 7fc80b7fe640 1 -- 172.21.15.100:0/2001974420 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d838a65470 con 0x55d8386a7c40 2024-07-26T13:46:01.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.969+0000 7fc80affd640 1 --2- 172.21.15.100:0/2001974420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8386721d0 0x55d838a7cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:46:01.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.969+0000 7fc80b7fe640 1 --2- 172.21.15.100:0/2001974420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8386a7c40 0x55d838a7c640 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc004470 tx=0x7fc7fc0225c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30fed51690b4d18f server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:01.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.969+0000 7fc7f17fa640 1 -- 172.21.15.100:0/2001974420 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc003ea0 con 0x55d8386a7c40 2024-07-26T13:46:01.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.970+0000 7fc7f17fa640 1 -- 172.21.15.100:0/2001974420 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7fc016070 con 0x55d8386a7c40 2024-07-26T13:46:01.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.970+0000 7fc7f17fa640 1 -- 172.21.15.100:0/2001974420 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc004680 con 0x55d8386a7c40 2024-07-26T13:46:01.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.970+0000 7fc81869b740 1 -- 172.21.15.100:0/2001974420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8386a7c40 msgr2=0x55d838a7c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:01.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.970+0000 7fc81869b740 1 --2- 172.21.15.100:0/2001974420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8386a7c40 0x55d838a7c640 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc004470 tx=0x7fc7fc0225c0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.970+0000 7fc81869b740 1 -- 172.21.15.100:0/2001974420 shutdown_connections 2024-07-26T13:46:01.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.971+0000 7fc81869b740 1 --2- 172.21.15.100:0/2001974420 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d83868a990 0x55d838a6cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.971+0000 7fc81869b740 1 --2- 172.21.15.100:0/2001974420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d8386721d0 0x55d838a7cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.971+0000 7fc81869b740 1 --2- 172.21.15.100:0/2001974420 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8386a7c40 0x55d838a7c640 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.971+0000 7fc81869b740 1 -- 172.21.15.100:0/2001974420 >> 172.21.15.100:0/2001974420 conn(0x55d838a57730 msgr2=0x55d838a55a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:01.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.971+0000 7fc81869b740 1 -- 172.21.15.100:0/2001974420 shutdown_connections 2024-07-26T13:46:01.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.971+0000 7fc81869b740 1 -- 172.21.15.100:0/2001974420 wait complete. 2024-07-26T13:46:01.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.971+0000 7fc81869b740 1 -- start start 2024-07-26T13:46:01.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.971+0000 7fc81869b740 1 Processor -- start 2024-07-26T13:46:01.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.971+0000 7fc81869b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8386721d0 0x55d838a6b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:01.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.971+0000 7fc81869b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d83868a990 0x55d838a6b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:01.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.971+0000 7fc81869b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8386a7c40 0x55d838ade620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:01.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.972+0000 7fc81869b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f800cf00 con 0x55d8386721d0 2024-07-26T13:46:01.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.972+0000 7fc80b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8386721d0 0x55d838a6b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:01.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.972+0000 7fc80b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8386721d0 0x55d838a6b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37636/0 (socket says 172.21.15.100:37636) 2024-07-26T13:46:01.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.972+0000 7fc81869b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f800e370 con 0x55d83868a990 2024-07-26T13:46:01.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.972+0000 7fc80b7fe640 1 -- 172.21.15.100:0/727258713 learned_addr learned my addr 172.21.15.100:0/727258713 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:01.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.972+0000 7fc81869b740 1 -- 172.21.15.100:0/727258713 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f800d5d0 con 0x55d8386a7c40 2024-07-26T13:46:01.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.972+0000 7fc80bfff640 1 --2- 172.21.15.100:0/727258713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8386a7c40 0x55d838ade620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:01.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.972+0000 7fc81869b740 10 client.0 _ll_register_callbacks cb 0x7fc7fc0040a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:46:01.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.972+0000 7fc80affd640 1 --2- 172.21.15.100:0/727258713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d83868a990 0x55d838a6b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:01.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.972+0000 7fc80b7fe640 1 -- 172.21.15.100:0/727258713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8386a7c40 msgr2=0x55d838ade620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:01.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.972+0000 7fc80b7fe640 1 --2- 172.21.15.100:0/727258713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8386a7c40 0x55d838ade620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.972+0000 7fc80b7fe640 1 -- 172.21.15.100:0/727258713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d83868a990 msgr2=0x55d838a6b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:01.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.972+0000 7fc80b7fe640 1 --2- 172.21.15.100:0/727258713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d83868a990 0x55d838a6b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:01.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.972+0000 7fc80b7fe640 1 -- 172.21.15.100:0/727258713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d838a6c030 con 0x55d8386721d0 2024-07-26T13:46:01.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.972+0000 7fc80bfff640 1 --2- 172.21.15.100:0/727258713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8386a7c40 0x55d838ade620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:46:01.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.972+0000 7fc80affd640 1 --2- 172.21.15.100:0/727258713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d83868a990 0x55d838a6b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:46:01.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.972+0000 7fc80b7fe640 1 --2- 172.21.15.100:0/727258713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8386721d0 0x55d838a6b220 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc00c010 tx=0x7fc7fc012680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f69da3b1f459368e server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:01.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.972+0000 7fc7eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:46:01.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.972+0000 7fc7eaffd640 1 -- 172.21.15.100:0/727258713 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc012a20 con 0x55d8386721d0 2024-07-26T13:46:01.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.972+0000 7fc7eaffd640 1 -- 172.21.15.100:0/727258713 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7fc7fc003ea0 con 0x55d8386721d0 2024-07-26T13:46:01.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.972+0000 7fc81869b740 1 -- 172.21.15.100:0/727258713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7fc016070 con 0x55d8386721d0 2024-07-26T13:46:01.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.973+0000 7fc7eaffd640 1 -- 172.21.15.100:0/727258713 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fc7fc044050 con 0x55d8386721d0 2024-07-26T13:46:01.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.973+0000 7fc81869b740 10 client.17004 fetch_fsmap learned FSMap version 47 2024-07-26T13:46:01.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.973+0000 7fc81869b740 1 -- 172.21.15.100:0/727258713 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55d838adfaf0 con 0x55d8386721d0 2024-07-26T13:46:01.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.973+0000 7fc7eaffd640 1 -- 172.21.15.100:0/727258713 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7fc016070 con 0x55d8386721d0 2024-07-26T13:46:01.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.973+0000 7fc7eaffd640 1 -- 172.21.15.100:0/727258713 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 47) ==== 8350+0+0 (secure 0 0 0) 0x7fc7fc01a600 con 0x55d8386721d0 2024-07-26T13:46:01.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.973+0000 7fc81869b740 10 client.17004 fetch_fsmap finished waiting for FSMap version 47 2024-07-26T13:46:01.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.973+0000 7fc81869b740 10 client.17004 resolve_mds: validated gid 24607 aka daemon mds.i 2024-07-26T13:46:01.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.973+0000 7fc81869b740 20 client.17004 populate_metadata read hostname 'smithi100' 2024-07-26T13:46:01.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.973+0000 7fc81869b740 1 --2- 172.21.15.100:0/727258713 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55d838692b70 0x55d838ae22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:01.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.973+0000 7fc81869b740 4 client.17004 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.95, "await_for": 45.0}] 2024-07-26T13:46:01.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.973+0000 7fc81869b740 1 -- 172.21.15.100:0/727258713 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.95, "await_for": 45.0}) -- 0x55d8389c97f0 con 0x55d838692b70 2024-07-26T13:46:01.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.973+0000 7fc80affd640 1 --2- 172.21.15.100:0/727258713 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55d838692b70 0x55d838ae22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:01.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.974+0000 7fc7eaffd640 10 client.17004 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:01.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:01.974+0000 7fc80affd640 1 --2- 172.21.15.100:0/727258713 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55d838692b70 0x55d838ae22b0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86c6ee50c1766a8f server_cookie=d93b7fbc3d93d14e in_seq=0 out_seq=0 2024-07-26T13:46:02.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:01 smithi167 ceph-mon[37140]: pgmap v543: 97 pgs: 97 active+clean; 90 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 170 op/s 2024-07-26T13:46:02.419 INFO:tasks.workunit.client.0.smithi100.stdout: 1 222738 3.03 MB/sec execute 136 sec latency 8841.996 ms 2024-07-26T13:46:02.870 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:46:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:46:03.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:46:03.419 INFO:tasks.workunit.client.0.smithi100.stdout: 1 222738 3.01 MB/sec execute 137 sec latency 9842.032 ms 2024-07-26T13:46:03.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.779+0000 7f4e90a4c640 1 -- 172.21.15.100:0/3433144294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e8c1019f0 msgr2=0x7f4e8c101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:03.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.779+0000 7f4e90a4c640 1 --2- 172.21.15.100:0/3433144294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e8c1019f0 0x7f4e8c101e00 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c009900 tx=0x7f4e7c02a410 comp rx=0 tx=0).stop 2024-07-26T13:46:03.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.779+0000 7f4e90a4c640 1 -- 172.21.15.100:0/3433144294 shutdown_connections 2024-07-26T13:46:03.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.779+0000 7f4e90a4c640 1 --2- 172.21.15.100:0/3433144294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e8c109880 0x7f4e8c1100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:03.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.779+0000 7f4e90a4c640 1 --2- 172.21.15.100:0/3433144294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e8c108f30 0x7f4e8c109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:03.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.779+0000 7f4e90a4c640 1 --2- 172.21.15.100:0/3433144294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e8c1019f0 0x7f4e8c101e00 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:03.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.779+0000 7f4e90a4c640 1 -- 172.21.15.100:0/3433144294 >> 172.21.15.100:0/3433144294 conn(0x7f4e8c0799c0 msgr2=0x7f4e8c079dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:03.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.779+0000 7f4e90a4c640 1 -- 172.21.15.100:0/3433144294 shutdown_connections 2024-07-26T13:46:03.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.779+0000 7f4e90a4c640 1 -- 172.21.15.100:0/3433144294 wait complete. 2024-07-26T13:46:03.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.780+0000 7f4e90a4c640 1 Processor -- start 2024-07-26T13:46:03.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.780+0000 7f4e90a4c640 1 -- start start 2024-07-26T13:46:03.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.780+0000 7f4e90a4c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e8c1019f0 0x7f4e8c199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:03.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.780+0000 7f4e90a4c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e8c108f30 0x7f4e8c19a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:03.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.780+0000 7f4e90a4c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e8c109880 0x7f4e8c19f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:03.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.780+0000 7f4e90a4c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8c113550 con 0x7f4e8c108f30 2024-07-26T13:46:03.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.780+0000 7f4e90a4c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8c1133d0 con 0x7f4e8c109880 2024-07-26T13:46:03.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.780+0000 7f4e90a4c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8c1136d0 con 0x7f4e8c1019f0 2024-07-26T13:46:03.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.781+0000 7f4e89d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e8c108f30 0x7f4e8c19a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:03.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.781+0000 7f4e89d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e8c108f30 0x7f4e8c19a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37654/0 (socket says 172.21.15.100:37654) 2024-07-26T13:46:03.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.781+0000 7f4e89d9b640 1 -- 172.21.15.100:0/2310327437 learned_addr learned my addr 172.21.15.100:0/2310327437 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:03.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.781+0000 7f4e8a59c640 1 --2- 172.21.15.100:0/2310327437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e8c1019f0 0x7f4e8c199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:03.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.781+0000 7f4e8ad9d640 1 --2- 172.21.15.100:0/2310327437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e8c109880 0x7f4e8c19f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:03.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.781+0000 7f4e89d9b640 1 -- 172.21.15.100:0/2310327437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e8c1019f0 msgr2=0x7f4e8c199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:03.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.781+0000 7f4e89d9b640 1 --2- 172.21.15.100:0/2310327437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e8c1019f0 0x7f4e8c199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:03.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.781+0000 7f4e89d9b640 1 -- 172.21.15.100:0/2310327437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e8c109880 msgr2=0x7f4e8c19f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:03.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.781+0000 7f4e89d9b640 1 --2- 172.21.15.100:0/2310327437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e8c109880 0x7f4e8c19f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:03.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.781+0000 7f4e89d9b640 1 -- 172.21.15.100:0/2310327437 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e8c19fef0 con 0x7f4e8c108f30 2024-07-26T13:46:03.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.781+0000 7f4e89d9b640 1 --2- 172.21.15.100:0/2310327437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e8c108f30 0x7f4e8c19a520 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f4e8000c970 tx=0x7f4e8000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:03.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.781+0000 7f4e7b7fe640 1 -- 172.21.15.100:0/2310327437 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e80007bb0 con 0x7f4e8c108f30 2024-07-26T13:46:03.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.781+0000 7f4e7b7fe640 1 -- 172.21.15.100:0/2310327437 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e80007d50 con 0x7f4e8c108f30 2024-07-26T13:46:03.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.781+0000 7f4e90a4c640 1 -- 172.21.15.100:0/2310327437 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e8c1a01e0 con 0x7f4e8c108f30 2024-07-26T13:46:03.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.781+0000 7f4e7b7fe640 1 -- 172.21.15.100:0/2310327437 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8000f450 con 0x7f4e8c108f30 2024-07-26T13:46:03.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.781+0000 7f4e90a4c640 1 -- 172.21.15.100:0/2310327437 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e8c1a6940 con 0x7f4e8c108f30 2024-07-26T13:46:03.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.784+0000 7f4e7b7fe640 1 -- 172.21.15.100:0/2310327437 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f4e80004eb0 con 0x7f4e8c108f30 2024-07-26T13:46:03.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.784+0000 7f4e96fd0740 1 Processor -- start 2024-07-26T13:46:03.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.784+0000 7f4e96fd0740 1 -- start start 2024-07-26T13:46:03.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.784+0000 7f4e96fd0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624a9ca9070 0x5624aa04e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:03.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.784+0000 7f4e96fd0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624a9c801b0 0x5624aa04eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:03.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.784+0000 7f4e96fd0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a9c88390 0x5624aa04eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:03.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.784+0000 7f4e96fd0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5624a9fc4bb0 con 0x5624a9c801b0 2024-07-26T13:46:03.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.784+0000 7f4e96fd0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5624a9fb40b0 con 0x5624a9c88390 2024-07-26T13:46:03.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.784+0000 7f4e96fd0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5624a9fb80f0 con 0x5624a9ca9070 2024-07-26T13:46:03.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.785+0000 7f4e8a59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624a9c801b0 0x5624aa04eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:03.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.785+0000 7f4e8a59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624a9c801b0 0x5624aa04eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37658/0 (socket says 172.21.15.100:37658) 2024-07-26T13:46:03.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.785+0000 7f4e8a59c640 1 -- 172.21.15.100:0/2610581098 learned_addr learned my addr 172.21.15.100:0/2610581098 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:03.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.785+0000 7f4e8ad9d640 1 --2- 172.21.15.100:0/2610581098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624a9ca9070 0x5624aa04e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:03.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.785+0000 7f4e89d9b640 1 --2- 172.21.15.100:0/2610581098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a9c88390 0x5624aa04eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:03.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.785+0000 7f4e7b7fe640 1 --2- 172.21.15.100:0/2310327437 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f4e740b3d40 0x7f4e740b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:03.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.785+0000 7f4e8a59c640 1 -- 172.21.15.100:0/2610581098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624a9ca9070 msgr2=0x5624aa04e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:03.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.785+0000 7f4e8a59c640 1 --2- 172.21.15.100:0/2610581098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624a9ca9070 0x5624aa04e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:03.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.785+0000 7f4e8a59c640 1 -- 172.21.15.100:0/2610581098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a9c88390 msgr2=0x5624aa04eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:03.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.785+0000 7f4e8a59c640 1 --2- 172.21.15.100:0/2610581098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a9c88390 0x5624aa04eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:03.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.785+0000 7f4e8a59c640 1 -- 172.21.15.100:0/2610581098 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624a9f5ab90 con 0x5624a9c801b0 2024-07-26T13:46:03.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.785+0000 7f4e7b7fe640 1 -- 172.21.15.100:0/2310327437 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7f4e800e3ce0 con 0x7f4e8c108f30 2024-07-26T13:46:03.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.785+0000 7f4e8ad9d640 1 --2- 172.21.15.100:0/2310327437 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f4e740b3d40 0x7f4e740b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:03.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.785+0000 7f4e8a59c640 1 --2- 172.21.15.100:0/2610581098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624a9c801b0 0x5624aa04eab0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c009900 tx=0x7f4e7c032e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c23e9171aa86770b server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:03.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.785+0000 7f4e78ff9640 1 -- 172.21.15.100:0/2610581098 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c040070 con 0x5624a9c801b0 2024-07-26T13:46:03.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.786+0000 7f4e78ff9640 1 -- 172.21.15.100:0/2610581098 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e7c0316a0 con 0x5624a9c801b0 2024-07-26T13:46:03.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.786+0000 7f4e78ff9640 1 -- 172.21.15.100:0/2610581098 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c035040 con 0x5624a9c801b0 2024-07-26T13:46:03.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.786+0000 7f4e96fd0740 1 -- 172.21.15.100:0/2610581098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624a9c801b0 msgr2=0x5624aa04eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:03.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.786+0000 7f4e8ad9d640 1 --2- 172.21.15.100:0/2310327437 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f4e740b3d40 0x7f4e740b6250 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4e84005fd0 tx=0x7f4e84004a40 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:03.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.786+0000 7f4e96fd0740 1 --2- 172.21.15.100:0/2610581098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624a9c801b0 0x5624aa04eab0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c009900 tx=0x7f4e7c032e60 comp rx=0 tx=0).stop 2024-07-26T13:46:03.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.786+0000 7f4e96fd0740 1 -- 172.21.15.100:0/2610581098 shutdown_connections 2024-07-26T13:46:03.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.786+0000 7f4e96fd0740 1 --2- 172.21.15.100:0/2610581098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a9c88390 0x5624aa04eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:03.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.786+0000 7f4e96fd0740 1 --2- 172.21.15.100:0/2610581098 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624a9c801b0 0x5624aa04eab0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:03.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.786+0000 7f4e96fd0740 1 --2- 172.21.15.100:0/2610581098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624a9ca9070 0x5624aa04e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:03.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.786+0000 7f4e96fd0740 1 -- 172.21.15.100:0/2610581098 >> 172.21.15.100:0/2610581098 conn(0x5624a9c9d460 msgr2=0x5624aa04c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:03.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.787+0000 7f4e96fd0740 1 -- 172.21.15.100:0/2610581098 shutdown_connections 2024-07-26T13:46:03.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.787+0000 7f4e96fd0740 1 -- 172.21.15.100:0/2610581098 wait complete. 2024-07-26T13:46:03.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.787+0000 7f4e96fd0740 1 -- start start 2024-07-26T13:46:03.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.787+0000 7f4e96fd0740 1 Processor -- start 2024-07-26T13:46:03.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.787+0000 7f4e96fd0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a9c801b0 0x5624aa05a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:03.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.787+0000 7f4e96fd0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624a9c88390 0x5624aa05a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:03.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.787+0000 7f4e96fd0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624a9ca9070 0x5624aa066040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:03.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.787+0000 7f4e96fd0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5624a9e17700 con 0x5624a9c88390 2024-07-26T13:46:03.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.787+0000 7f4e96fd0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5624a9f569e0 con 0x5624a9c801b0 2024-07-26T13:46:03.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.787+0000 7f4e96fd0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5624aa05ccf0 con 0x5624a9ca9070 2024-07-26T13:46:03.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.787+0000 7f4e96fd0740 10 client.0 _ll_register_callbacks cb 0x5624a9f714f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:46:03.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.787+0000 7f4e89d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624a9c88390 0x5624aa05a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:03.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.787+0000 7f4e89d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624a9c88390 0x5624aa05a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37664/0 (socket says 172.21.15.100:37664) 2024-07-26T13:46:03.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.787+0000 7f4e89d9b640 1 -- 172.21.15.100:0/3394889176 learned_addr learned my addr 172.21.15.100:0/3394889176 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:03.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.788+0000 7f4e8ad9d640 1 --2- 172.21.15.100:0/3394889176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624a9ca9070 0x5624aa066040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:03.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.788+0000 7f4e8a59c640 1 --2- 172.21.15.100:0/3394889176 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a9c801b0 0x5624aa05a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:03.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.788+0000 7f4e89d9b640 1 -- 172.21.15.100:0/3394889176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624a9ca9070 msgr2=0x5624aa066040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:03.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.788+0000 7f4e89d9b640 1 --2- 172.21.15.100:0/3394889176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624a9ca9070 0x5624aa066040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:03.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.788+0000 7f4e89d9b640 1 -- 172.21.15.100:0/3394889176 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a9c801b0 msgr2=0x5624aa05a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:03.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.788+0000 7f4e89d9b640 1 --2- 172.21.15.100:0/3394889176 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a9c801b0 0x5624aa05a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:03.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.788+0000 7f4e89d9b640 1 -- 172.21.15.100:0/3394889176 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624a9d14650 con 0x5624a9c88390 2024-07-26T13:46:03.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.788+0000 7f4e8ad9d640 1 --2- 172.21.15.100:0/3394889176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624a9ca9070 0x5624aa066040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:46:03.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.788+0000 7f4e89d9b640 1 --2- 172.21.15.100:0/3394889176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624a9c88390 0x5624aa05a710 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f4e8000c940 tx=0x7f4e8000a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0fdcfac6add222 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:03.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.788+0000 7f4e71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:46:03.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.788+0000 7f4e71ffb640 1 -- 172.21.15.100:0/3394889176 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8001d070 con 0x5624a9c88390 2024-07-26T13:46:03.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.788+0000 7f4e71ffb640 1 -- 172.21.15.100:0/3394889176 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7f4e800e1630 con 0x5624a9c88390 2024-07-26T13:46:03.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.788+0000 7f4e96fd0740 1 -- 172.21.15.100:0/3394889176 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e7c0316a0 con 0x5624a9c88390 2024-07-26T13:46:03.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.788+0000 7f4e71ffb640 1 -- 172.21.15.100:0/3394889176 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f4e800053a0 con 0x5624a9c88390 2024-07-26T13:46:03.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.788+0000 7f4e96fd0740 10 client.17016 fetch_fsmap learned FSMap version 47 2024-07-26T13:46:03.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.788+0000 7f4e96fd0740 1 -- 172.21.15.100:0/3394889176 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5624aa068df0 con 0x5624a9c88390 2024-07-26T13:46:03.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.788+0000 7f4e71ffb640 1 -- 172.21.15.100:0/3394889176 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e7c0316a0 con 0x5624a9c88390 2024-07-26T13:46:03.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.789+0000 7f4e71ffb640 1 -- 172.21.15.100:0/3394889176 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 47) ==== 8350+0+0 (secure 0 0 0) 0x7f4e800abba0 con 0x5624a9c88390 2024-07-26T13:46:03.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.789+0000 7f4e96fd0740 10 client.17016 fetch_fsmap finished waiting for FSMap version 47 2024-07-26T13:46:03.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.789+0000 7f4e96fd0740 10 client.17016 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:46:03.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.789+0000 7f4e96fd0740 20 client.17016 populate_metadata read hostname 'smithi100' 2024-07-26T13:46:03.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.789+0000 7f4e96fd0740 1 --2- 172.21.15.100:0/3394889176 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5624a9c679f0 0x5624aa06c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:03.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.789+0000 7f4e96fd0740 4 client.17016 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:46:03.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.789+0000 7f4e96fd0740 1 -- 172.21.15.100:0/3394889176 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624a9fc6b00 con 0x5624a9c679f0 2024-07-26T13:46:03.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.789+0000 7f4e8a59c640 1 --2- 172.21.15.100:0/3394889176 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5624a9c679f0 0x5624aa06c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:03.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.789+0000 7f4e71ffb640 10 client.17016 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:03.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.789+0000 7f4e8a59c640 1 --2- 172.21.15.100:0/3394889176 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5624a9c679f0 0x5624aa06c360 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af40144872e9b769 server_cookie=6cf854bd1326eda7 in_seq=0 out_seq=0 2024-07-26T13:46:03.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.790+0000 7f4e71ffb640 1 -- 172.21.15.100:0/3394889176 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5624a9fc6b00 con 0x5624a9c679f0 2024-07-26T13:46:03.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.790+0000 7f4e71ffb640 10 client.17016 handle_command_reply: tid=0 2024-07-26T13:46:03.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.790+0000 7f4e96fd0740 1 client.17016 shutdown 2024-07-26T13:46:03.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.790+0000 7f4e96fd0740 1 -- 172.21.15.100:0/3394889176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624a9c88390 msgr2=0x5624aa05a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:03.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.790+0000 7f4e96fd0740 1 --2- 172.21.15.100:0/3394889176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624a9c88390 0x5624aa05a710 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f4e8000c940 tx=0x7f4e8000a560 comp rx=0 tx=0).stop 2024-07-26T13:46:03.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.791+0000 7f4e96fd0740 1 -- 172.21.15.100:0/3394889176 shutdown_connections 2024-07-26T13:46:03.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.791+0000 7f4e96fd0740 1 --2- 172.21.15.100:0/3394889176 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5624a9c679f0 0x5624aa06c360 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:03.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.791+0000 7f4e71ffb640 0 client.17016 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:03.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.791+0000 7f4e96fd0740 1 --2- 172.21.15.100:0/3394889176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624a9ca9070 0x5624aa066040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:03.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.791+0000 7f4e96fd0740 1 --2- 172.21.15.100:0/3394889176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624a9c88390 0x5624aa05a710 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:03.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.791+0000 7f4e96fd0740 1 --2- 172.21.15.100:0/3394889176 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a9c801b0 0x5624aa05a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:03.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.791+0000 7f4e96fd0740 1 -- 172.21.15.100:0/3394889176 >> 172.21.15.100:0/3394889176 conn(0x5624a9c9d460 msgr2=0x5624aa051700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:03.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.791+0000 7f4e96fd0740 1 -- 172.21.15.100:0/3394889176 shutdown_connections 2024-07-26T13:46:03.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.791+0000 7f4e96fd0740 1 -- 172.21.15.100:0/3394889176 wait complete. 2024-07-26T13:46:03.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.791+0000 7f4e96fd0740 20 client.17016 trim_cache size 0 max 16384 2024-07-26T13:46:03.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.804+0000 7f4e96fd0740 1 Processor -- start 2024-07-26T13:46:03.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.804+0000 7f4e96fd0740 1 -- start start 2024-07-26T13:46:03.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.804+0000 7f4e96fd0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624a9c679f0 0x5624aa051d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:03.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.804+0000 7f4e96fd0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a9c801b0 0x5624aa065e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:03.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.804+0000 7f4e96fd0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624a9c88390 0x5624aa0537d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:03.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.804+0000 7f4e96fd0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8c1b1c00 con 0x5624a9c88390 2024-07-26T13:46:03.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.804+0000 7f4e96fd0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8c1af4a0 con 0x5624a9c801b0 2024-07-26T13:46:03.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.804+0000 7f4e96fd0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8c1ad2d0 con 0x5624a9c679f0 2024-07-26T13:46:03.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.804+0000 7f4e8ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624a9c88390 0x5624aa0537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:03.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.804+0000 7f4e8ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624a9c88390 0x5624aa0537d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37668/0 (socket says 172.21.15.100:37668) 2024-07-26T13:46:03.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.804+0000 7f4e8ad9d640 1 -- 172.21.15.100:0/1198780595 learned_addr learned my addr 172.21.15.100:0/1198780595 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:03.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.804+0000 7f4e8a59c640 1 --2- 172.21.15.100:0/1198780595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624a9c679f0 0x5624aa051d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:03.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.804+0000 7f4e89d9b640 1 --2- 172.21.15.100:0/1198780595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a9c801b0 0x5624aa065e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:03.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.804+0000 7f4e8ad9d640 1 -- 172.21.15.100:0/1198780595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624a9c679f0 msgr2=0x5624aa051d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:03.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.804+0000 7f4e8ad9d640 1 --2- 172.21.15.100:0/1198780595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624a9c679f0 0x5624aa051d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:03.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.804+0000 7f4e8ad9d640 1 -- 172.21.15.100:0/1198780595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a9c801b0 msgr2=0x5624aa065e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:03.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.804+0000 7f4e8ad9d640 1 --2- 172.21.15.100:0/1198780595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a9c801b0 0x5624aa065e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:03.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.805+0000 7f4e8ad9d640 1 -- 172.21.15.100:0/1198780595 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624aa05b270 con 0x5624a9c88390 2024-07-26T13:46:03.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.805+0000 7f4e89d9b640 1 --2- 172.21.15.100:0/1198780595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a9c801b0 0x5624aa065e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:46:03.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.805+0000 7f4e8ad9d640 1 --2- 172.21.15.100:0/1198780595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624a9c88390 0x5624aa0537d0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f4e840049d0 tx=0x7f4e84016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e00e10c7fa8d3 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:03.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.805+0000 7f4e73fff640 1 -- 172.21.15.100:0/1198780595 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e840206c0 con 0x5624a9c88390 2024-07-26T13:46:03.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.805+0000 7f4e73fff640 1 -- 172.21.15.100:0/1198780595 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e84020d10 con 0x5624a9c88390 2024-07-26T13:46:03.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.805+0000 7f4e73fff640 1 -- 172.21.15.100:0/1198780595 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8401de60 con 0x5624a9c88390 2024-07-26T13:46:03.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.806+0000 7f4e96fd0740 1 -- 172.21.15.100:0/1198780595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624a9c88390 msgr2=0x5624aa0537d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:03.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.806+0000 7f4e96fd0740 1 --2- 172.21.15.100:0/1198780595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624a9c88390 0x5624aa0537d0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f4e840049d0 tx=0x7f4e84016000 comp rx=0 tx=0).stop 2024-07-26T13:46:03.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.806+0000 7f4e96fd0740 1 -- 172.21.15.100:0/1198780595 shutdown_connections 2024-07-26T13:46:03.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.806+0000 7f4e96fd0740 1 --2- 172.21.15.100:0/1198780595 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624a9c88390 0x5624aa0537d0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:03.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.806+0000 7f4e96fd0740 1 --2- 172.21.15.100:0/1198780595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a9c801b0 0x5624aa065e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:03.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.806+0000 7f4e96fd0740 1 --2- 172.21.15.100:0/1198780595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624a9c679f0 0x5624aa051d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:03.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.806+0000 7f4e96fd0740 1 -- 172.21.15.100:0/1198780595 >> 172.21.15.100:0/1198780595 conn(0x5624a9c9d460 msgr2=0x5624aa051670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:03.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.806+0000 7f4e96fd0740 1 -- 172.21.15.100:0/1198780595 shutdown_connections 2024-07-26T13:46:03.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.806+0000 7f4e96fd0740 1 -- 172.21.15.100:0/1198780595 wait complete. 2024-07-26T13:46:03.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.806+0000 7f4e96fd0740 1 -- start start 2024-07-26T13:46:03.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.806+0000 7f4e96fd0740 1 Processor -- start 2024-07-26T13:46:03.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.807+0000 7f4e96fd0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624a9c679f0 0x5624aa062810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:03.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.807+0000 7f4e96fd0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a9c801b0 0x5624aa04b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:03.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.807+0000 7f4e96fd0740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624a9c88390 0x5624aa04bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:03.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.807+0000 7f4e96fd0740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8c1ad960 con 0x5624a9c88390 2024-07-26T13:46:03.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.807+0000 7f4e96fd0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8c1aedd0 con 0x5624a9c801b0 2024-07-26T13:46:03.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.807+0000 7f4e96fd0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8c1ae030 con 0x5624a9c679f0 2024-07-26T13:46:03.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.807+0000 7f4e96fd0740 10 client.0 _ll_register_callbacks cb 0x7f4e800e7300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:46:03.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.807+0000 7f4e8ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624a9c88390 0x5624aa04bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:03.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.807+0000 7f4e8ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624a9c88390 0x5624aa04bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37684/0 (socket says 172.21.15.100:37684) 2024-07-26T13:46:03.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.807+0000 7f4e8ad9d640 1 -- 172.21.15.100:0/2276387932 learned_addr learned my addr 172.21.15.100:0/2276387932 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:03.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.807+0000 7f4e8a59c640 1 --2- 172.21.15.100:0/2276387932 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624a9c679f0 0x5624aa062810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:03.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.807+0000 7f4e89d9b640 1 --2- 172.21.15.100:0/2276387932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a9c801b0 0x5624aa04b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:03.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.807+0000 7f4e8ad9d640 1 -- 172.21.15.100:0/2276387932 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624a9c679f0 msgr2=0x5624aa062810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:03.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.807+0000 7f4e8ad9d640 1 --2- 172.21.15.100:0/2276387932 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624a9c679f0 0x5624aa062810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:03.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.807+0000 7f4e8ad9d640 1 -- 172.21.15.100:0/2276387932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a9c801b0 msgr2=0x5624aa04b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:03.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.807+0000 7f4e8ad9d640 1 --2- 172.21.15.100:0/2276387932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a9c801b0 0x5624aa04b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:03.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.807+0000 7f4e8ad9d640 1 -- 172.21.15.100:0/2276387932 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624aa05e240 con 0x5624a9c88390 2024-07-26T13:46:03.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.807+0000 7f4e89d9b640 1 --2- 172.21.15.100:0/2276387932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a9c801b0 0x5624aa04b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:46:03.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.807+0000 7f4e8ad9d640 1 --2- 172.21.15.100:0/2276387932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624a9c88390 0x5624aa04bfc0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f4e84004410 tx=0x7f4e8401d660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa4f1384d20759a server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:03.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.807+0000 7f4e71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:46:03.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.807+0000 7f4e71ffb640 1 -- 172.21.15.100:0/2276387932 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8401dbe0 con 0x5624a9c88390 2024-07-26T13:46:03.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.807+0000 7f4e71ffb640 1 -- 172.21.15.100:0/2276387932 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8656+0+0 (secure 0 0 0) 0x7f4e8401dd80 con 0x5624a9c88390 2024-07-26T13:46:03.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.807+0000 7f4e96fd0740 1 -- 172.21.15.100:0/2276387932 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e84020d10 con 0x5624a9c88390 2024-07-26T13:46:03.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.808+0000 7f4e71ffb640 1 -- 172.21.15.100:0/2276387932 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f4e840204c0 con 0x5624a9c88390 2024-07-26T13:46:03.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.808+0000 7f4e96fd0740 10 client.17022 fetch_fsmap learned FSMap version 47 2024-07-26T13:46:03.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.808+0000 7f4e96fd0740 1 -- 172.21.15.100:0/2276387932 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5624aa0d4bf0 con 0x5624a9c88390 2024-07-26T13:46:03.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.808+0000 7f4e71ffb640 1 -- 172.21.15.100:0/2276387932 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e84020d10 con 0x5624a9c88390 2024-07-26T13:46:03.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.808+0000 7f4e71ffb640 1 -- 172.21.15.100:0/2276387932 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 47) ==== 8350+0+0 (secure 0 0 0) 0x7f4e8403a070 con 0x5624a9c88390 2024-07-26T13:46:03.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.808+0000 7f4e96fd0740 10 client.17022 fetch_fsmap finished waiting for FSMap version 47 2024-07-26T13:46:03.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.808+0000 7f4e96fd0740 10 client.17022 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:46:03.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.808+0000 7f4e96fd0740 20 client.17022 populate_metadata read hostname 'smithi100' 2024-07-26T13:46:03.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.808+0000 7f4e96fd0740 1 --2- 172.21.15.100:0/2276387932 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5624a9ca9070 0x5624aa0d72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:03.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.808+0000 7f4e96fd0740 4 client.17022 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-26T13:46:03.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.808+0000 7f4e96fd0740 1 -- 172.21.15.100:0/2276387932 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5624a9fc25a0 con 0x5624a9ca9070 2024-07-26T13:46:03.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.808+0000 7f4e8a59c640 1 --2- 172.21.15.100:0/2276387932 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5624a9ca9070 0x5624aa0d72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:03.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.809+0000 7f4e8a59c640 1 --2- 172.21.15.100:0/2276387932 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5624a9ca9070 0x5624aa0d72c0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b98367bb5e13ba2e server_cookie=2b65ae0e137629dc in_seq=0 out_seq=0 2024-07-26T13:46:03.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.809+0000 7f4e71ffb640 10 client.17022 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:03.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.809+0000 7f4e71ffb640 1 -- 172.21.15.100:0/2276387932 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5624a9fc25a0 con 0x5624a9ca9070 2024-07-26T13:46:03.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.809+0000 7f4e71ffb640 10 client.17022 handle_command_reply: tid=0 2024-07-26T13:46:03.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.809+0000 7f4e96fd0740 1 client.17022 shutdown 2024-07-26T13:46:03.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.809+0000 7f4e96fd0740 1 -- 172.21.15.100:0/2276387932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624a9c88390 msgr2=0x5624aa04bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:03.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.809+0000 7f4e96fd0740 1 --2- 172.21.15.100:0/2276387932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624a9c88390 0x5624aa04bfc0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f4e84004410 tx=0x7f4e8401d660 comp rx=0 tx=0).stop 2024-07-26T13:46:03.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.809+0000 7f4e96fd0740 1 -- 172.21.15.100:0/2276387932 shutdown_connections 2024-07-26T13:46:03.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.809+0000 7f4e96fd0740 1 --2- 172.21.15.100:0/2276387932 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5624a9ca9070 0x5624aa0d72c0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:03.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.809+0000 7f4e96fd0740 1 --2- 172.21.15.100:0/2276387932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5624a9c88390 0x5624aa04bfc0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:03.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.809+0000 7f4e96fd0740 1 --2- 172.21.15.100:0/2276387932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5624a9c801b0 0x5624aa04b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:03.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.809+0000 7f4e96fd0740 1 --2- 172.21.15.100:0/2276387932 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5624a9c679f0 0x5624aa062810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:03.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.809+0000 7f4e96fd0740 1 -- 172.21.15.100:0/2276387932 >> 172.21.15.100:0/2276387932 conn(0x5624a9c9d460 msgr2=0x5624aa04f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:03.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.809+0000 7f4e71ffb640 0 client.17022 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:03.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.809+0000 7f4e96fd0740 1 -- 172.21.15.100:0/2276387932 shutdown_connections 2024-07-26T13:46:03.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.809+0000 7f4e96fd0740 1 -- 172.21.15.100:0/2276387932 wait complete. 2024-07-26T13:46:03.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.809+0000 7f4e96fd0740 20 client.17022 trim_cache size 0 max 16384 2024-07-26T13:46:03.818 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-07-26T13:46:03.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.810+0000 7f4e90a4c640 1 -- 172.21.15.100:0/2310327437 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f4e740b3d40 msgr2=0x7f4e740b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:03.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.810+0000 7f4e90a4c640 1 --2- 172.21.15.100:0/2310327437 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f4e740b3d40 0x7f4e740b6250 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4e84005fd0 tx=0x7f4e84004a40 comp rx=0 tx=0).stop 2024-07-26T13:46:03.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.810+0000 7f4e90a4c640 1 -- 172.21.15.100:0/2310327437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e8c108f30 msgr2=0x7f4e8c19a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:03.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.810+0000 7f4e90a4c640 1 --2- 172.21.15.100:0/2310327437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e8c108f30 0x7f4e8c19a520 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f4e8000c970 tx=0x7f4e8000ce30 comp rx=0 tx=0).stop 2024-07-26T13:46:03.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.810+0000 7f4e90a4c640 1 -- 172.21.15.100:0/2310327437 shutdown_connections 2024-07-26T13:46:03.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.810+0000 7f4e90a4c640 1 --2- 172.21.15.100:0/2310327437 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f4e740b3d40 0x7f4e740b6250 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:03.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.810+0000 7f4e90a4c640 1 --2- 172.21.15.100:0/2310327437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e8c109880 0x7f4e8c19f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:03.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.810+0000 7f4e90a4c640 1 --2- 172.21.15.100:0/2310327437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e8c108f30 0x7f4e8c19a520 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:03.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.810+0000 7f4e90a4c640 1 --2- 172.21.15.100:0/2310327437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e8c1019f0 0x7f4e8c199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:03.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.810+0000 7f4e90a4c640 1 -- 172.21.15.100:0/2310327437 >> 172.21.15.100:0/2310327437 conn(0x7f4e8c0799c0 msgr2=0x7f4e8c107070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:03.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.810+0000 7f4e90a4c640 1 -- 172.21.15.100:0/2310327437 shutdown_connections 2024-07-26T13:46:03.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:03.810+0000 7f4e90a4c640 1 -- 172.21.15.100:0/2310327437 wait complete. 2024-07-26T13:46:03.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:03 smithi100 ceph-mon[32599]: pgmap v544: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 1.1 MiB/s wr, 73 op/s 2024-07-26T13:46:03.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:46:03.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:46:03.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:46:03.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:46:03.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:46:03.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:46:03.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:46:03.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:46:03.821 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:46:03.822 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:46:03.822 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:46:03.822 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:46:03.822 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:46:03.822 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:46:03.822 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:46:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:03 smithi138 ceph-mon[36954]: pgmap v544: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 1.1 MiB/s wr, 73 op/s 2024-07-26T13:46:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:46:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:46:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:46:03.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:46:03.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:46:03.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:46:03.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:46:03.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:46:03.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:46:03.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:46:03.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:46:03.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:46:03.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:46:03.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:46:03.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:46:04.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:03 smithi167 ceph-mon[37140]: pgmap v544: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 1.1 MiB/s wr, 73 op/s 2024-07-26T13:46:04.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:46:04.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:46:04.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:46:04.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:46:04.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:46:04.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:46:04.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:46:04.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:46:04.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:46:04.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:46:04.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:46:04.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:46:04.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:46:04.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:46:04.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:46:04.420 INFO:tasks.workunit.client.0.smithi100.stdout: 1 222738 2.99 MB/sec execute 138 sec latency 10842.067 ms 2024-07-26T13:46:04.585 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-26T13:46:04.859 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:04 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T13:46:04.859 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:04 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T13:46:04.859 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:04 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T13:46:04.859 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:04 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T13:46:04.859 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:04 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T13:46:04.859 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:04 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T13:46:04.893 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:46:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:04 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T13:46:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:04 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T13:46:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:04 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T13:46:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:04 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T13:46:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:04 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T13:46:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:04 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T13:46:05.010 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:04 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T13:46:05.010 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:04 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T13:46:05.010 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:04 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T13:46:05.010 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:04 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T13:46:05.011 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:04 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T13:46:05.011 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:04 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T13:46:05.011 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:46:04 smithi167 systemd[1]: ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.c.service: Scheduled restart job, restart counter is at 2. 2024-07-26T13:46:05.011 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:46:04 smithi167 systemd[1]: Stopped Ceph mds.c for bf83e2e6-4b52-11ef-bcb7-c7b262605968. 2024-07-26T13:46:05.011 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:46:04 smithi167 systemd[1]: ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.c.service: Consumed 15.482s CPU time. 2024-07-26T13:46:05.011 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:46:04 smithi167 systemd[1]: Starting Ceph mds.c for bf83e2e6-4b52-11ef-bcb7-c7b262605968... 2024-07-26T13:46:05.272 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:46:05 smithi167 podman[72601]: 2024-07-26 13:46:04.912382378 +0000 UTC m=+0.017730598 image pull ae51f432d4fa82622bc253ca9e2da7666c617e096e99d43d0a0e4fdaca8865f8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2 2024-07-26T13:46:05.273 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:46:05 smithi167 podman[72601]: 2024-07-26 13:46:05.04166161 +0000 UTC m=+0.147009859 container create 5b999749f6bc19443b492e786711bd600b1ad817c7b9386e4d404a46d512f1a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb) 2024-07-26T13:46:05.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:05.347+0000 7fc7eaffd640 1 -- 172.21.15.100:0/727258713 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: -125 ) ==== 8+0+591 (crc 0 0 0) 0x55d8389c97f0 con 0x55d838692b70 2024-07-26T13:46:05.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:05.347+0000 7fc7eaffd640 10 client.17004 handle_command_reply: tid=0 2024-07-26T13:46:05.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:05.347+0000 7fc81869b740 1 client.17004 shutdown 2024-07-26T13:46:05.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:05.347+0000 7fc81869b740 1 -- 172.21.15.100:0/727258713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8386721d0 msgr2=0x55d838a6b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:05.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:05.347+0000 7fc81869b740 1 --2- 172.21.15.100:0/727258713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8386721d0 0x55d838a6b220 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc00c010 tx=0x7fc7fc012680 comp rx=0 tx=0).stop 2024-07-26T13:46:05.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:05.347+0000 7fc81869b740 1 -- 172.21.15.100:0/727258713 shutdown_connections 2024-07-26T13:46:05.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:05.347+0000 7fc81869b740 1 --2- 172.21.15.100:0/727258713 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55d838692b70 0x55d838ae22b0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:05.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:05.347+0000 7fc7eaffd640 0 client.17004 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:05.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:05.347+0000 7fc81869b740 1 --2- 172.21.15.100:0/727258713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8386a7c40 0x55d838ade620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:05.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:05.347+0000 7fc81869b740 1 --2- 172.21.15.100:0/727258713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d83868a990 0x55d838a6b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:05.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:05.347+0000 7fc81869b740 1 --2- 172.21.15.100:0/727258713 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d8386721d0 0x55d838a6b220 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:05.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:05.347+0000 7fc81869b740 1 -- 172.21.15.100:0/727258713 >> 172.21.15.100:0/727258713 conn(0x55d838a57730 msgr2=0x55d838a55a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:05.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:05.348+0000 7fc81869b740 1 -- 172.21.15.100:0/727258713 shutdown_connections 2024-07-26T13:46:05.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:05.349+0000 7fc81869b740 1 -- 172.21.15.100:0/727258713 wait complete. 2024-07-26T13:46:05.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:05.349+0000 7fc81869b740 20 client.17004 trim_cache size 0 max 16384 2024-07-26T13:46:05.351 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:46:05.351 INFO:teuthology.orchestra.run.smithi100.stdout: "epoch": 48, 2024-07-26T13:46:05.351 INFO:teuthology.orchestra.run.smithi100.stdout: "leader": 24607, 2024-07-26T13:46:05.351 INFO:teuthology.orchestra.run.smithi100.stdout: "set_version": 18, 2024-07-26T13:46:05.352 INFO:teuthology.orchestra.run.smithi100.stdout: "sets": { 2024-07-26T13:46:05.352 INFO:teuthology.orchestra.run.smithi100.stdout: "fc7b57fa": { 2024-07-26T13:46:05.352 INFO:teuthology.orchestra.run.smithi100.stdout: "version": 18, 2024-07-26T13:46:05.352 INFO:teuthology.orchestra.run.smithi100.stdout: "age_ref": 0.0, 2024-07-26T13:46:05.352 INFO:teuthology.orchestra.run.smithi100.stdout: "state": { 2024-07-26T13:46:05.352 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "CANCELED", 2024-07-26T13:46:05.352 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 0.0 2024-07-26T13:46:05.352 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-07-26T13:46:05.352 INFO:teuthology.orchestra.run.smithi100.stdout: "timeout": 90.0, 2024-07-26T13:46:05.352 INFO:teuthology.orchestra.run.smithi100.stdout: "expiration": 141.9, 2024-07-26T13:46:05.352 INFO:teuthology.orchestra.run.smithi100.stdout: "members": { 2024-07-26T13:46:05.352 INFO:teuthology.orchestra.run.smithi100.stdout: "file:/": { 2024-07-26T13:46:05.352 INFO:teuthology.orchestra.run.smithi100.stdout: "excluded": true, 2024-07-26T13:46:05.352 INFO:teuthology.orchestra.run.smithi100.stdout: "state": { 2024-07-26T13:46:05.353 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "QUIESCING", 2024-07-26T13:46:05.353 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 3.4 2024-07-26T13:46:05.353 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:46:05.353 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:46:05.353 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:46:05.353 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:46:05.353 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:46:05.353 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:46:05.353 INFO:teuthology.orchestra.run.smithi100.stderr:Error ECANCELED: 2024-07-26T13:46:05.419 INFO:tasks.workunit.client.0.smithi100.stdout: 1 222738 2.97 MB/sec execute 139 sec latency 11842.106 ms 2024-07-26T13:46:05.546 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:46:05 smithi167 podman[72601]: 2024-07-26 13:46:05.303470524 +0000 UTC m=+0.408818746 container init 5b999749f6bc19443b492e786711bd600b1ad817c7b9386e4d404a46d512f1a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, org.label-schema.build-date=20240724, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-07-26T13:46:05.546 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:46:05 smithi167 podman[72601]: 2024-07-26 13:46:05.306708479 +0000 UTC m=+0.412056695 container start 5b999749f6bc19443b492e786711bd600b1ad817c7b9386e4d404a46d512f1a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-07-26T13:46:05.546 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:46:05 smithi167 ceph-mds[72621]: -- 172.21.15.167:0/3173265752 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56112c45f860 con 0x561130601a80 2024-07-26T13:46:05.546 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:46:05 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-c[72616]: starting mds.c at 2024-07-26T13:46:05.546 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:46:05 smithi167 bash[72601]: 5b999749f6bc19443b492e786711bd600b1ad817c7b9386e4d404a46d512f1a3 2024-07-26T13:46:05.547 INFO:journalctl@ceph.mds.c.smithi167.stdout:Jul 26 13:46:05 smithi167 systemd[1]: Started Ceph mds.c for bf83e2e6-4b52-11ef-bcb7-c7b262605968. 2024-07-26T13:46:05.835 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:05 smithi100 ceph-mon[32599]: pgmap v545: 97 pgs: 97 active+clean; 90 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 981 KiB/s wr, 69 op/s 2024-07-26T13:46:05.836 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:05 smithi100 ceph-mon[32599]: daemon mds.c restarted 2024-07-26T13:46:05.836 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:05 smithi100 ceph-mon[32599]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-26T13:46:05.836 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:05 smithi100 ceph-mon[32599]: osdmap e84: 12 total, 12 up, 12 in 2024-07-26T13:46:05.836 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:05 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.167:6836/3519222898,v1:172.21.15.167:6837/3519222898] up:boot 2024-07-26T13:46:05.836 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:05 smithi100 ceph-mon[32599]: Standby daemon mds.e assigned to filesystem cephfs as rank 2 2024-07-26T13:46:05.836 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:05 smithi100 ceph-mon[32599]: fsmap cephfs:2/3 {0=i=up:active,1=b=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-07-26T13:46:05.836 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:05 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-26T13:46:05.836 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:05 smithi100 ceph-mon[32599]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=e=up:replay} 2 up:standby-replay 7 up:standby 2024-07-26T13:46:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:05 smithi138 ceph-mon[36954]: pgmap v545: 97 pgs: 97 active+clean; 90 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 981 KiB/s wr, 69 op/s 2024-07-26T13:46:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:05 smithi138 ceph-mon[36954]: daemon mds.c restarted 2024-07-26T13:46:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:05 smithi138 ceph-mon[36954]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-26T13:46:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:05 smithi138 ceph-mon[36954]: osdmap e84: 12 total, 12 up, 12 in 2024-07-26T13:46:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:05 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.167:6836/3519222898,v1:172.21.15.167:6837/3519222898] up:boot 2024-07-26T13:46:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:05 smithi138 ceph-mon[36954]: Standby daemon mds.e assigned to filesystem cephfs as rank 2 2024-07-26T13:46:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:05 smithi138 ceph-mon[36954]: fsmap cephfs:2/3 {0=i=up:active,1=b=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-07-26T13:46:05.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:05 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-26T13:46:05.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:05 smithi138 ceph-mon[36954]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=e=up:replay} 2 up:standby-replay 7 up:standby 2024-07-26T13:46:06.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:05 smithi167 ceph-mon[37140]: pgmap v545: 97 pgs: 97 active+clean; 90 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 981 KiB/s wr, 69 op/s 2024-07-26T13:46:06.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:05 smithi167 ceph-mon[37140]: daemon mds.c restarted 2024-07-26T13:46:06.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:05 smithi167 ceph-mon[37140]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-26T13:46:06.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:05 smithi167 ceph-mon[37140]: osdmap e84: 12 total, 12 up, 12 in 2024-07-26T13:46:06.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:05 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.167:6836/3519222898,v1:172.21.15.167:6837/3519222898] up:boot 2024-07-26T13:46:06.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:05 smithi167 ceph-mon[37140]: Standby daemon mds.e assigned to filesystem cephfs as rank 2 2024-07-26T13:46:06.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:05 smithi167 ceph-mon[37140]: fsmap cephfs:2/3 {0=i=up:active,1=b=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-07-26T13:46:06.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:05 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-26T13:46:06.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:05 smithi167 ceph-mon[37140]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=e=up:replay} 2 up:standby-replay 7 up:standby 2024-07-26T13:46:06.420 INFO:tasks.workunit.client.0.smithi100.stdout: 1 222738 2.95 MB/sec execute 140 sec latency 12842.141 ms 2024-07-26T13:46:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:06 smithi100 ceph-mon[32599]: pgmap v547: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 331 KiB/s wr, 45 op/s 2024-07-26T13:46:06.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:06 smithi167 ceph-mon[37140]: pgmap v547: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 331 KiB/s wr, 45 op/s 2024-07-26T13:46:06.848 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-07-26T13:46:06.848 WARNING:tasks.quiescer.fs.[cephfs]:Quiesce set got cancelled (cap = 10).Won't raise an error since this could be a failover, will wait for the next quiesce attempt 2024-07-26T13:46:06.849 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.95 2024-07-26T13:46:06.849 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:46:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:06 smithi138 ceph-mon[36954]: pgmap v547: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 331 KiB/s wr, 45 op/s 2024-07-26T13:46:07.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.365+0000 7fb5e699f640 1 -- 172.21.15.100:0/1997351455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5e0103590 msgr2=0x7fb5e01039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:07.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.365+0000 7fb5e699f640 1 --2- 172.21.15.100:0/1997351455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5e0103590 0x7fb5e01039a0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0009900 tx=0x7fb5d002a410 comp rx=0 tx=0).stop 2024-07-26T13:46:07.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.365+0000 7fb5e699f640 1 -- 172.21.15.100:0/1997351455 shutdown_connections 2024-07-26T13:46:07.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.365+0000 7fb5e699f640 1 --2- 172.21.15.100:0/1997351455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5e0109820 0x7fb5e0110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:07.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.365+0000 7fb5e699f640 1 --2- 172.21.15.100:0/1997351455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5e0108ed0 0x7fb5e01092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:07.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.365+0000 7fb5e699f640 1 --2- 172.21.15.100:0/1997351455 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5e0103590 0x7fb5e01039a0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:07.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.365+0000 7fb5e699f640 1 -- 172.21.15.100:0/1997351455 >> 172.21.15.100:0/1997351455 conn(0x7fb5e007b280 msgr2=0x7fb5e007b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:07.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.365+0000 7fb5e699f640 1 -- 172.21.15.100:0/1997351455 shutdown_connections 2024-07-26T13:46:07.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.365+0000 7fb5e699f640 1 -- 172.21.15.100:0/1997351455 wait complete. 2024-07-26T13:46:07.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.366+0000 7fb5e699f640 1 Processor -- start 2024-07-26T13:46:07.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.366+0000 7fb5e699f640 1 -- start start 2024-07-26T13:46:07.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.366+0000 7fb5e699f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5e0103590 0x7fb5e0199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:07.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.366+0000 7fb5e699f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5e0108ed0 0x7fb5e019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:07.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.366+0000 7fb5e699f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5e0109820 0x7fb5e019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:07.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.366+0000 7fb5e699f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e01134f0 con 0x7fb5e0108ed0 2024-07-26T13:46:07.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.366+0000 7fb5e699f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e0113370 con 0x7fb5e0109820 2024-07-26T13:46:07.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.366+0000 7fb5e699f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e0113670 con 0x7fb5e0103590 2024-07-26T13:46:07.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.366+0000 7fb5df7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5e0108ed0 0x7fb5e019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:07.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.366+0000 7fb5df7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5e0108ed0 0x7fb5e019a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48650/0 (socket says 172.21.15.100:48650) 2024-07-26T13:46:07.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.366+0000 7fb5df7fe640 1 -- 172.21.15.100:0/3060545256 learned_addr learned my addr 172.21.15.100:0/3060545256 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:07.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.366+0000 7fb5dffff640 1 --2- 172.21.15.100:0/3060545256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5e0103590 0x7fb5e0199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:07.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.366+0000 7fb5e4f3c640 1 --2- 172.21.15.100:0/3060545256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5e0109820 0x7fb5e019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:07.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.367+0000 7fb5df7fe640 1 -- 172.21.15.100:0/3060545256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5e0103590 msgr2=0x7fb5e0199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:07.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.367+0000 7fb5df7fe640 1 --2- 172.21.15.100:0/3060545256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5e0103590 0x7fb5e0199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:07.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.367+0000 7fb5df7fe640 1 -- 172.21.15.100:0/3060545256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5e0109820 msgr2=0x7fb5e019f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:07.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.367+0000 7fb5df7fe640 1 --2- 172.21.15.100:0/3060545256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5e0109820 0x7fb5e019f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:07.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.367+0000 7fb5df7fe640 1 -- 172.21.15.100:0/3060545256 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e019fe50 con 0x7fb5e0108ed0 2024-07-26T13:46:07.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.367+0000 7fb5dffff640 1 --2- 172.21.15.100:0/3060545256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5e0103590 0x7fb5e0199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:46:07.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.367+0000 7fb5df7fe640 1 --2- 172.21.15.100:0/3060545256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5e0108ed0 0x7fb5e019a480 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fb5d400e9c0 tx=0x7fb5d400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:07.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.367+0000 7fb5dd7fa640 1 -- 172.21.15.100:0/3060545256 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d400ccc0 con 0x7fb5e0108ed0 2024-07-26T13:46:07.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.367+0000 7fb5e699f640 1 -- 172.21.15.100:0/3060545256 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e01a0140 con 0x7fb5e0108ed0 2024-07-26T13:46:07.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.367+0000 7fb5dd7fa640 1 -- 172.21.15.100:0/3060545256 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5d40054d0 con 0x7fb5e0108ed0 2024-07-26T13:46:07.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.367+0000 7fb5dd7fa640 1 -- 172.21.15.100:0/3060545256 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d4010430 con 0x7fb5e0108ed0 2024-07-26T13:46:07.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.367+0000 7fb5e699f640 1 -- 172.21.15.100:0/3060545256 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e01a68a0 con 0x7fb5e0108ed0 2024-07-26T13:46:07.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.370+0000 7fb5ecf23740 1 Processor -- start 2024-07-26T13:46:07.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.370+0000 7fb5ecf23740 1 -- start start 2024-07-26T13:46:07.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.370+0000 7fb5ecf23740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56079e226070 0x56079e5cb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:07.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.370+0000 7fb5ecf23740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56079e1fd1b0 0x56079e5cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:07.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.370+0000 7fb5ecf23740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56079e205390 0x56079e5cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:07.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.370+0000 7fb5ecf23740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56079e541bb0 con 0x56079e1fd1b0 2024-07-26T13:46:07.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.370+0000 7fb5ecf23740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56079e5310b0 con 0x56079e226070 2024-07-26T13:46:07.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.370+0000 7fb5ecf23740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56079e5350f0 con 0x56079e205390 2024-07-26T13:46:07.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.370+0000 7fb5dffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56079e1fd1b0 0x56079e5cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:07.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.370+0000 7fb5dd7fa640 1 -- 172.21.15.100:0/3060545256 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fb5d4005000 con 0x7fb5e0108ed0 2024-07-26T13:46:07.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.370+0000 7fb5dffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56079e1fd1b0 0x56079e5cbab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48664/0 (socket says 172.21.15.100:48664) 2024-07-26T13:46:07.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.370+0000 7fb5dffff640 1 -- 172.21.15.100:0/3183500951 learned_addr learned my addr 172.21.15.100:0/3183500951 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:07.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.371+0000 7fb5df7fe640 1 --2- 172.21.15.100:0/3183500951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56079e205390 0x56079e5cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:07.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.371+0000 7fb5e4f3c640 1 --2- 172.21.15.100:0/3183500951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56079e226070 0x56079e5cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:07.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.371+0000 7fb5dffff640 1 -- 172.21.15.100:0/3183500951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56079e205390 msgr2=0x56079e5cbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:07.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.371+0000 7fb5dffff640 1 --2- 172.21.15.100:0/3183500951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56079e205390 0x56079e5cbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:07.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.371+0000 7fb5dffff640 1 -- 172.21.15.100:0/3183500951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56079e226070 msgr2=0x56079e5cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:07.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.371+0000 7fb5dffff640 1 --2- 172.21.15.100:0/3183500951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56079e226070 0x56079e5cb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:07.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.371+0000 7fb5dffff640 1 -- 172.21.15.100:0/3183500951 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56079e4d7b90 con 0x56079e1fd1b0 2024-07-26T13:46:07.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.371+0000 7fb5e4f3c640 1 --2- 172.21.15.100:0/3183500951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56079e226070 0x56079e5cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:46:07.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.371+0000 7fb5dffff640 1 --2- 172.21.15.100:0/3183500951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56079e1fd1b0 0x56079e5cbab0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0009900 tx=0x7fb5d0005ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f2bffc0e6990899 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:07.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.371+0000 7fb5c67fc640 1 -- 172.21.15.100:0/3183500951 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0040070 con 0x56079e1fd1b0 2024-07-26T13:46:07.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.371+0000 7fb5c67fc640 1 -- 172.21.15.100:0/3183500951 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5d0031820 con 0x56079e1fd1b0 2024-07-26T13:46:07.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.371+0000 7fb5c67fc640 1 -- 172.21.15.100:0/3183500951 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0035040 con 0x56079e1fd1b0 2024-07-26T13:46:07.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.371+0000 7fb5dd7fa640 1 --2- 172.21.15.100:0/3060545256 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb5c80b3dc0 0x7fb5c80b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:07.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.372+0000 7fb5e4f3c640 1 --2- 172.21.15.100:0/3060545256 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb5c80b3dc0 0x7fb5c80b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:07.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.372+0000 7fb5dd7fa640 1 -- 172.21.15.100:0/3060545256 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7fb5d40e4480 con 0x7fb5e0108ed0 2024-07-26T13:46:07.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.372+0000 7fb5ecf23740 1 -- 172.21.15.100:0/3183500951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56079e1fd1b0 msgr2=0x56079e5cbab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:07.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.372+0000 7fb5e4f3c640 1 --2- 172.21.15.100:0/3060545256 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb5c80b3dc0 0x7fb5c80b62d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8005fd0 tx=0x7fb5d8009600 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:07.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.372+0000 7fb5ecf23740 1 --2- 172.21.15.100:0/3183500951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56079e1fd1b0 0x56079e5cbab0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0009900 tx=0x7fb5d0005ed0 comp rx=0 tx=0).stop 2024-07-26T13:46:07.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.372+0000 7fb5ecf23740 1 -- 172.21.15.100:0/3183500951 shutdown_connections 2024-07-26T13:46:07.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.372+0000 7fb5ecf23740 1 --2- 172.21.15.100:0/3183500951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56079e205390 0x56079e5cbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:07.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.373+0000 7fb5ecf23740 1 --2- 172.21.15.100:0/3183500951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56079e1fd1b0 0x56079e5cbab0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:07.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.373+0000 7fb5ecf23740 1 --2- 172.21.15.100:0/3183500951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56079e226070 0x56079e5cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:07.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.373+0000 7fb5ecf23740 1 -- 172.21.15.100:0/3183500951 >> 172.21.15.100:0/3183500951 conn(0x56079e21a460 msgr2=0x56079e5c9460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:07.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.373+0000 7fb5ecf23740 1 -- 172.21.15.100:0/3183500951 shutdown_connections 2024-07-26T13:46:07.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.373+0000 7fb5ecf23740 1 -- 172.21.15.100:0/3183500951 wait complete. 2024-07-26T13:46:07.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.373+0000 7fb5ecf23740 1 -- start start 2024-07-26T13:46:07.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.373+0000 7fb5ecf23740 1 Processor -- start 2024-07-26T13:46:07.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.374+0000 7fb5ecf23740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56079e1fd1b0 0x56079e5db490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:07.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.374+0000 7fb5ecf23740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56079e205390 0x56079e5db9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:07.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.374+0000 7fb5ecf23740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56079e226070 0x56079e5e73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:07.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.374+0000 7fb5ecf23740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56079e394700 con 0x56079e226070 2024-07-26T13:46:07.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.374+0000 7fb5ecf23740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56079e4d39e0 con 0x56079e1fd1b0 2024-07-26T13:46:07.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.374+0000 7fb5ecf23740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56079e5de050 con 0x56079e205390 2024-07-26T13:46:07.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.374+0000 7fb5ecf23740 10 client.0 _ll_register_callbacks cb 0x56079e4ee4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:46:07.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.374+0000 7fb5dffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56079e1fd1b0 0x56079e5db490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:07.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.374+0000 7fb5e4f3c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56079e226070 0x56079e5e73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:07.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.374+0000 7fb5dffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56079e1fd1b0 0x56079e5db490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:49770/0 (socket says 172.21.15.100:49770) 2024-07-26T13:46:07.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.374+0000 7fb5dffff640 1 -- 172.21.15.100:0/3171599584 learned_addr learned my addr 172.21.15.100:0/3171599584 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:07.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.374+0000 7fb5df7fe640 1 --2- 172.21.15.100:0/3171599584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56079e205390 0x56079e5db9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:07.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.374+0000 7fb5e4f3c640 1 -- 172.21.15.100:0/3171599584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56079e205390 msgr2=0x56079e5db9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:07.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.374+0000 7fb5e4f3c640 1 --2- 172.21.15.100:0/3171599584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56079e205390 0x56079e5db9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:07.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.374+0000 7fb5e4f3c640 1 -- 172.21.15.100:0/3171599584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56079e1fd1b0 msgr2=0x56079e5db490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:07.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.374+0000 7fb5e4f3c640 1 --2- 172.21.15.100:0/3171599584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56079e1fd1b0 0x56079e5db490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:07.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.374+0000 7fb5e4f3c640 1 -- 172.21.15.100:0/3171599584 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56079e291650 con 0x56079e226070 2024-07-26T13:46:07.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.374+0000 7fb5dffff640 1 --2- 172.21.15.100:0/3171599584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56079e1fd1b0 0x56079e5db490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:46:07.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.375+0000 7fb5e4f3c640 1 --2- 172.21.15.100:0/3171599584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56079e226070 0x56079e5e73e0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8010700 tx=0x7fb5d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=700ca0e559fdae05 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:07.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.375+0000 7fb5bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:46:07.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.375+0000 7fb5bf7fe640 1 -- 172.21.15.100:0/3171599584 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8004d20 con 0x56079e226070 2024-07-26T13:46:07.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.375+0000 7fb5bf7fe640 1 -- 172.21.15.100:0/3171599584 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7fb5d801d590 con 0x56079e226070 2024-07-26T13:46:07.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.375+0000 7fb5ecf23740 1 -- 172.21.15.100:0/3171599584 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5d0031820 con 0x56079e226070 2024-07-26T13:46:07.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.375+0000 7fb5bf7fe640 1 -- 172.21.15.100:0/3171599584 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fb5d8032050 con 0x56079e226070 2024-07-26T13:46:07.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.375+0000 7fb5ecf23740 10 client.17040 fetch_fsmap learned FSMap version 50 2024-07-26T13:46:07.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.375+0000 7fb5ecf23740 1 -- 172.21.15.100:0/3171599584 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x56079e5ea090 con 0x56079e226070 2024-07-26T13:46:07.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.375+0000 7fb5bf7fe640 1 -- 172.21.15.100:0/3171599584 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5d0031820 con 0x56079e226070 2024-07-26T13:46:07.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.375+0000 7fb5bf7fe640 1 -- 172.21.15.100:0/3171599584 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 50) ==== 8350+0+0 (secure 0 0 0) 0x7fb5d800f040 con 0x56079e226070 2024-07-26T13:46:07.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.376+0000 7fb5ecf23740 10 client.17040 fetch_fsmap finished waiting for FSMap version 50 2024-07-26T13:46:07.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.376+0000 7fb5ecf23740 10 client.17040 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-07-26T13:46:07.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.376+0000 7fb5ecf23740 20 client.17040 populate_metadata read hostname 'smithi100' 2024-07-26T13:46:07.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.376+0000 7fb5ecf23740 1 --2- 172.21.15.100:0/3171599584 >> [v2:172.21.15.138:6834/2831339503,v1:172.21.15.138:6835/2831339503] conn(0x56079e1e49f0 0x56079e5ed750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:07.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.376+0000 7fb5ecf23740 4 client.17040 mds_command: new command op to 25997 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:46:07.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.376+0000 7fb5ecf23740 1 -- 172.21.15.100:0/3171599584 --> [v2:172.21.15.138:6834/2831339503,v1:172.21.15.138:6835/2831339503] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56079e543b00 con 0x56079e1e49f0 2024-07-26T13:46:07.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.376+0000 7fb5dffff640 1 --2- 172.21.15.100:0/3171599584 >> [v2:172.21.15.138:6834/2831339503,v1:172.21.15.138:6835/2831339503] conn(0x56079e1e49f0 0x56079e5ed750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:07.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.376+0000 7fb5bf7fe640 10 client.17040 ms_handle_connect on v2:172.21.15.138:6834/2831339503 2024-07-26T13:46:07.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.377+0000 7fb5dffff640 1 --2- 172.21.15.100:0/3171599584 >> [v2:172.21.15.138:6834/2831339503,v1:172.21.15.138:6835/2831339503] conn(0x56079e1e49f0 0x56079e5ed750 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7613bba9babdfe81 server_cookie=b17adb8dabdfa2a4 in_seq=0 out_seq=0 2024-07-26T13:46:07.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.377+0000 7fb5bf7fe640 1 -- 172.21.15.100:0/3171599584 <== mds.2 v2:172.21.15.138:6834/2831339503 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56079e543b00 con 0x56079e1e49f0 2024-07-26T13:46:07.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.377+0000 7fb5bf7fe640 10 client.17040 handle_command_reply: tid=0 2024-07-26T13:46:07.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.377+0000 7fb5ecf23740 1 client.17040 shutdown 2024-07-26T13:46:07.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.378+0000 7fb5ecf23740 1 -- 172.21.15.100:0/3171599584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56079e226070 msgr2=0x56079e5e73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:07.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.378+0000 7fb5ecf23740 1 --2- 172.21.15.100:0/3171599584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56079e226070 0x56079e5e73e0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8010700 tx=0x7fb5d8004a40 comp rx=0 tx=0).stop 2024-07-26T13:46:07.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.378+0000 7fb5ecf23740 1 -- 172.21.15.100:0/3171599584 shutdown_connections 2024-07-26T13:46:07.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.378+0000 7fb5ecf23740 1 --2- 172.21.15.100:0/3171599584 >> [v2:172.21.15.138:6834/2831339503,v1:172.21.15.138:6835/2831339503] conn(0x56079e1e49f0 0x56079e5ed750 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:07.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.378+0000 7fb5ecf23740 1 --2- 172.21.15.100:0/3171599584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56079e226070 0x56079e5e73e0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:07.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.378+0000 7fb5bf7fe640 0 client.17040 ms_handle_reset on v2:172.21.15.138:6834/2831339503 2024-07-26T13:46:07.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.378+0000 7fb5ecf23740 1 --2- 172.21.15.100:0/3171599584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56079e205390 0x56079e5db9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:07.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.378+0000 7fb5ecf23740 1 --2- 172.21.15.100:0/3171599584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56079e1fd1b0 0x56079e5db490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:07.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.378+0000 7fb5ecf23740 1 -- 172.21.15.100:0/3171599584 >> 172.21.15.100:0/3171599584 conn(0x56079e21a460 msgr2=0x56079e5ccf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:07.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.378+0000 7fb5ecf23740 1 -- 172.21.15.100:0/3171599584 shutdown_connections 2024-07-26T13:46:07.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.378+0000 7fb5ecf23740 1 -- 172.21.15.100:0/3171599584 wait complete. 2024-07-26T13:46:07.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.378+0000 7fb5ecf23740 20 client.17040 trim_cache size 0 max 16384 2024-07-26T13:46:07.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.392+0000 7fb5ecf23740 1 Processor -- start 2024-07-26T13:46:07.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.392+0000 7fb5ecf23740 1 -- start start 2024-07-26T13:46:07.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.393+0000 7fb5ecf23740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56079e1e49f0 0x56079e5d6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:07.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.393+0000 7fb5ecf23740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56079e1fd1b0 0x56079e5d9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:07.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.393+0000 7fb5ecf23740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56079e205390 0x56079e5da050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:07.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.393+0000 7fb5ecf23740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e01ad940 con 0x56079e1e49f0 2024-07-26T13:46:07.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.393+0000 7fb5ecf23740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e01ab0c0 con 0x56079e1fd1b0 2024-07-26T13:46:07.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.393+0000 7fb5ecf23740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e01ad270 con 0x56079e205390 2024-07-26T13:46:07.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.393+0000 7fb5dffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56079e1e49f0 0x56079e5d6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:07.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.393+0000 7fb5dffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56079e1e49f0 0x56079e5d6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48680/0 (socket says 172.21.15.100:48680) 2024-07-26T13:46:07.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.393+0000 7fb5dffff640 1 -- 172.21.15.100:0/3215020905 learned_addr learned my addr 172.21.15.100:0/3215020905 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:07.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.393+0000 7fb5df7fe640 1 --2- 172.21.15.100:0/3215020905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56079e1fd1b0 0x56079e5d9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:07.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.393+0000 7fb5e4f3c640 1 --2- 172.21.15.100:0/3215020905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56079e205390 0x56079e5da050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:07.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.394+0000 7fb5dffff640 1 -- 172.21.15.100:0/3215020905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56079e205390 msgr2=0x56079e5da050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:07.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.394+0000 7fb5dffff640 1 --2- 172.21.15.100:0/3215020905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56079e205390 0x56079e5da050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:07.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.394+0000 7fb5dffff640 1 -- 172.21.15.100:0/3215020905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56079e1fd1b0 msgr2=0x56079e5d9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:07.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.394+0000 7fb5dffff640 1 --2- 172.21.15.100:0/3215020905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56079e1fd1b0 0x56079e5d9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:07.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.394+0000 7fb5dffff640 1 -- 172.21.15.100:0/3215020905 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56079e5cde90 con 0x56079e1e49f0 2024-07-26T13:46:07.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.394+0000 7fb5dffff640 1 --2- 172.21.15.100:0/3215020905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56079e1e49f0 0x56079e5d6410 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0033000 tx=0x7fb5d0005500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b21278d6ed48d89 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:07.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.394+0000 7fb5c5ffb640 1 -- 172.21.15.100:0/3215020905 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0035040 con 0x56079e1e49f0 2024-07-26T13:46:07.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.394+0000 7fb5c5ffb640 1 -- 172.21.15.100:0/3215020905 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5d00035f0 con 0x56079e1e49f0 2024-07-26T13:46:07.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.394+0000 7fb5c5ffb640 1 -- 172.21.15.100:0/3215020905 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0040040 con 0x56079e1e49f0 2024-07-26T13:46:07.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.395+0000 7fb5ecf23740 1 -- 172.21.15.100:0/3215020905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56079e1e49f0 msgr2=0x56079e5d6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:07.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.395+0000 7fb5ecf23740 1 --2- 172.21.15.100:0/3215020905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56079e1e49f0 0x56079e5d6410 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0033000 tx=0x7fb5d0005500 comp rx=0 tx=0).stop 2024-07-26T13:46:07.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.395+0000 7fb5ecf23740 1 -- 172.21.15.100:0/3215020905 shutdown_connections 2024-07-26T13:46:07.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.395+0000 7fb5ecf23740 1 --2- 172.21.15.100:0/3215020905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56079e205390 0x56079e5da050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:07.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.395+0000 7fb5ecf23740 1 --2- 172.21.15.100:0/3215020905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56079e1fd1b0 0x56079e5d9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:07.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.395+0000 7fb5ecf23740 1 --2- 172.21.15.100:0/3215020905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56079e1e49f0 0x56079e5d6410 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:07.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.395+0000 7fb5ecf23740 1 -- 172.21.15.100:0/3215020905 >> 172.21.15.100:0/3215020905 conn(0x56079e21a460 msgr2=0x56079e5c8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:07.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.396+0000 7fb5ecf23740 1 -- 172.21.15.100:0/3215020905 shutdown_connections 2024-07-26T13:46:07.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.396+0000 7fb5ecf23740 1 -- 172.21.15.100:0/3215020905 wait complete. 2024-07-26T13:46:07.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.396+0000 7fb5ecf23740 1 -- start start 2024-07-26T13:46:07.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.396+0000 7fb5ecf23740 1 Processor -- start 2024-07-26T13:46:07.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.397+0000 7fb5ecf23740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56079e1e49f0 0x56079e64e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:07.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.397+0000 7fb5ecf23740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56079e1fd1b0 0x56079e64e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:07.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.397+0000 7fb5ecf23740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56079e205390 0x56079e5dea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:07.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.397+0000 7fb5ecf23740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e01acba0 con 0x56079e1e49f0 2024-07-26T13:46:07.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.397+0000 7fb5ecf23740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e01ab730 con 0x56079e1fd1b0 2024-07-26T13:46:07.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.397+0000 7fb5ecf23740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e01abe00 con 0x56079e205390 2024-07-26T13:46:07.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.397+0000 7fb5ecf23740 10 client.0 _ll_register_callbacks cb 0x7fb5d8009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:46:07.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.397+0000 7fb5dffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56079e1e49f0 0x56079e64e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:07.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.397+0000 7fb5dffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56079e1e49f0 0x56079e64e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48686/0 (socket says 172.21.15.100:48686) 2024-07-26T13:46:07.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.397+0000 7fb5dffff640 1 -- 172.21.15.100:0/3263050620 learned_addr learned my addr 172.21.15.100:0/3263050620 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:07.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.397+0000 7fb5df7fe640 1 --2- 172.21.15.100:0/3263050620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56079e1fd1b0 0x56079e64e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:07.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.397+0000 7fb5e4f3c640 1 --2- 172.21.15.100:0/3263050620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56079e205390 0x56079e5dea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:07.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.397+0000 7fb5dffff640 1 -- 172.21.15.100:0/3263050620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56079e205390 msgr2=0x56079e5dea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:07.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.397+0000 7fb5dffff640 1 --2- 172.21.15.100:0/3263050620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56079e205390 0x56079e5dea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:07.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.397+0000 7fb5dffff640 1 -- 172.21.15.100:0/3263050620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56079e1fd1b0 msgr2=0x56079e64e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:07.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.397+0000 7fb5dffff640 1 --2- 172.21.15.100:0/3263050620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56079e1fd1b0 0x56079e64e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:07.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.397+0000 7fb5dffff640 1 -- 172.21.15.100:0/3263050620 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56079e5df1a0 con 0x56079e1e49f0 2024-07-26T13:46:07.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.397+0000 7fb5dffff640 1 --2- 172.21.15.100:0/3263050620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56079e1e49f0 0x56079e64e250 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fb5d00095f0 tx=0x7fb5d002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d3491ca5800f0b server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:07.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.397+0000 7fb5bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:46:07.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.398+0000 7fb5bf7fe640 1 -- 172.21.15.100:0/3263050620 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0040030 con 0x56079e1e49f0 2024-07-26T13:46:07.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.398+0000 7fb5ecf23740 1 -- 172.21.15.100:0/3263050620 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5d00035f0 con 0x56079e1e49f0 2024-07-26T13:46:07.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.398+0000 7fb5bf7fe640 1 -- 172.21.15.100:0/3263050620 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7fb5d0003fc0 con 0x56079e1e49f0 2024-07-26T13:46:07.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.398+0000 7fb5bf7fe640 1 -- 172.21.15.100:0/3263050620 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fb5d0035070 con 0x56079e1e49f0 2024-07-26T13:46:07.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.398+0000 7fb5ecf23740 10 client.17046 fetch_fsmap learned FSMap version 50 2024-07-26T13:46:07.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.398+0000 7fb5ecf23740 1 -- 172.21.15.100:0/3263050620 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x56079e655a00 con 0x56079e1e49f0 2024-07-26T13:46:07.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.399+0000 7fb5bf7fe640 1 -- 172.21.15.100:0/3263050620 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5d00035f0 con 0x56079e1e49f0 2024-07-26T13:46:07.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.399+0000 7fb5bf7fe640 1 -- 172.21.15.100:0/3263050620 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 50) ==== 8350+0+0 (secure 0 0 0) 0x7fb5d0067030 con 0x56079e1e49f0 2024-07-26T13:46:07.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.399+0000 7fb5ecf23740 10 client.17046 fetch_fsmap finished waiting for FSMap version 50 2024-07-26T13:46:07.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.399+0000 7fb5ecf23740 10 client.17046 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-07-26T13:46:07.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.399+0000 7fb5ecf23740 20 client.17046 populate_metadata read hostname 'smithi100' 2024-07-26T13:46:07.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.399+0000 7fb5ecf23740 1 --2- 172.21.15.100:0/3263050620 >> [v2:172.21.15.138:6834/2831339503,v1:172.21.15.138:6835/2831339503] conn(0x56079e226070 0x56079e5d38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:07.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.399+0000 7fb5ecf23740 4 client.17046 mds_command: new command op to 25997 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-26T13:46:07.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.399+0000 7fb5ecf23740 1 -- 172.21.15.100:0/3263050620 --> [v2:172.21.15.138:6834/2831339503,v1:172.21.15.138:6835/2831339503] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56079e53f5a0 con 0x56079e226070 2024-07-26T13:46:07.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.400+0000 7fb5df7fe640 1 --2- 172.21.15.100:0/3263050620 >> [v2:172.21.15.138:6834/2831339503,v1:172.21.15.138:6835/2831339503] conn(0x56079e226070 0x56079e5d38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:07.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.400+0000 7fb5bf7fe640 10 client.17046 ms_handle_connect on v2:172.21.15.138:6834/2831339503 2024-07-26T13:46:07.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.400+0000 7fb5df7fe640 1 --2- 172.21.15.100:0/3263050620 >> [v2:172.21.15.138:6834/2831339503,v1:172.21.15.138:6835/2831339503] conn(0x56079e226070 0x56079e5d38d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f62c74bde89ac792 server_cookie=87c7324f2e554ea7 in_seq=0 out_seq=0 2024-07-26T13:46:07.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.400+0000 7fb5bf7fe640 1 -- 172.21.15.100:0/3263050620 <== mds.2 v2:172.21.15.138:6834/2831339503 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56079e53f5a0 con 0x56079e226070 2024-07-26T13:46:07.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.400+0000 7fb5bf7fe640 10 client.17046 handle_command_reply: tid=0 2024-07-26T13:46:07.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.400+0000 7fb5ecf23740 1 client.17046 shutdown 2024-07-26T13:46:07.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.400+0000 7fb5ecf23740 1 -- 172.21.15.100:0/3263050620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56079e1e49f0 msgr2=0x56079e64e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:07.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.400+0000 7fb5ecf23740 1 --2- 172.21.15.100:0/3263050620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56079e1e49f0 0x56079e64e250 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fb5d00095f0 tx=0x7fb5d002a410 comp rx=0 tx=0).stop 2024-07-26T13:46:07.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.401+0000 7fb5ecf23740 1 -- 172.21.15.100:0/3263050620 shutdown_connections 2024-07-26T13:46:07.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.401+0000 7fb5ecf23740 1 --2- 172.21.15.100:0/3263050620 >> [v2:172.21.15.138:6834/2831339503,v1:172.21.15.138:6835/2831339503] conn(0x56079e226070 0x56079e5d38d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:07.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.401+0000 7fb5bf7fe640 0 client.17046 ms_handle_reset on v2:172.21.15.138:6834/2831339503 2024-07-26T13:46:07.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.401+0000 7fb5ecf23740 1 --2- 172.21.15.100:0/3263050620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56079e205390 0x56079e5dea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:07.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.401+0000 7fb5ecf23740 1 --2- 172.21.15.100:0/3263050620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56079e1fd1b0 0x56079e64e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:07.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.401+0000 7fb5ecf23740 1 --2- 172.21.15.100:0/3263050620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56079e1e49f0 0x56079e64e250 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:07.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.401+0000 7fb5ecf23740 1 -- 172.21.15.100:0/3263050620 >> 172.21.15.100:0/3263050620 conn(0x56079e21a460 msgr2=0x56079e5c8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:07.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.401+0000 7fb5ecf23740 1 -- 172.21.15.100:0/3263050620 shutdown_connections 2024-07-26T13:46:07.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.401+0000 7fb5ecf23740 1 -- 172.21.15.100:0/3263050620 wait complete. 2024-07-26T13:46:07.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.401+0000 7fb5ecf23740 20 client.17046 trim_cache size 0 max 16384 2024-07-26T13:46:07.406 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-07-26T13:46:07.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.401+0000 7fb5e699f640 1 -- 172.21.15.100:0/3060545256 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb5c80b3dc0 msgr2=0x7fb5c80b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:07.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.401+0000 7fb5e699f640 1 --2- 172.21.15.100:0/3060545256 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb5c80b3dc0 0x7fb5c80b62d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8005fd0 tx=0x7fb5d8009600 comp rx=0 tx=0).stop 2024-07-26T13:46:07.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.401+0000 7fb5e699f640 1 -- 172.21.15.100:0/3060545256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5e0108ed0 msgr2=0x7fb5e019a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:07.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.401+0000 7fb5e699f640 1 --2- 172.21.15.100:0/3060545256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5e0108ed0 0x7fb5e019a480 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fb5d400e9c0 tx=0x7fb5d400ee80 comp rx=0 tx=0).stop 2024-07-26T13:46:07.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.402+0000 7fb5e699f640 1 -- 172.21.15.100:0/3060545256 shutdown_connections 2024-07-26T13:46:07.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.402+0000 7fb5e699f640 1 --2- 172.21.15.100:0/3060545256 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb5c80b3dc0 0x7fb5c80b62d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:07.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.402+0000 7fb5e699f640 1 --2- 172.21.15.100:0/3060545256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5e0109820 0x7fb5e019f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:07.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.402+0000 7fb5e699f640 1 --2- 172.21.15.100:0/3060545256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb5e0108ed0 0x7fb5e019a480 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:07.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.402+0000 7fb5e699f640 1 --2- 172.21.15.100:0/3060545256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5e0103590 0x7fb5e0199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:07.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.402+0000 7fb5e699f640 1 -- 172.21.15.100:0/3060545256 >> 172.21.15.100:0/3060545256 conn(0x7fb5e007b280 msgr2=0x7fb5e0106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:07.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.402+0000 7fb5e699f640 1 -- 172.21.15.100:0/3060545256 shutdown_connections 2024-07-26T13:46:07.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:07.402+0000 7fb5e699f640 1 -- 172.21.15.100:0/3060545256 wait complete. 2024-07-26T13:46:07.410 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:46:07.419 INFO:tasks.workunit.client.0.smithi100.stdout: 1 222738 2.93 MB/sec execute 141 sec latency 13842.180 ms 2024-07-26T13:46:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:07 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.138:6834/2831339503,v1:172.21.15.138:6835/2831339503] up:resolve 2024-07-26T13:46:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:07 smithi100 ceph-mon[32599]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=e=up:resolve} 2 up:standby-replay 7 up:standby 2024-07-26T13:46:07.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:07 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.138:6834/2831339503,v1:172.21.15.138:6835/2831339503] up:resolve 2024-07-26T13:46:07.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:07 smithi138 ceph-mon[36954]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=e=up:resolve} 2 up:standby-replay 7 up:standby 2024-07-26T13:46:08.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:07 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.138:6834/2831339503,v1:172.21.15.138:6835/2831339503] up:resolve 2024-07-26T13:46:08.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:07 smithi167 ceph-mon[37140]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=e=up:resolve} 2 up:standby-replay 7 up:standby 2024-07-26T13:46:08.420 INFO:tasks.workunit.client.0.smithi100.stdout: 1 222738 2.91 MB/sec execute 142 sec latency 14842.289 ms 2024-07-26T13:46:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:08.619+0000 7f18477fe640 1 -- 172.21.15.100:0/3325404085 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561797be0da0 con 0x56179785f1c0 2024-07-26T13:46:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:08.619+0000 7f18477fe640 10 client.16986 handle_command_reply: tid=0 2024-07-26T13:46:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:08.619+0000 7f18753b8740 1 client.16986 shutdown 2024-07-26T13:46:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:08.619+0000 7f18753b8740 1 -- 172.21.15.100:0/3325404085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617978cb1b0 msgr2=0x561797c7edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:08.619+0000 7f18753b8740 1 --2- 172.21.15.100:0/3325404085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617978cb1b0 0x561797c7edc0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f18600042c0 tx=0x7f1860004ab0 comp rx=0 tx=0).stop 2024-07-26T13:46:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:08.619+0000 7f18753b8740 1 -- 172.21.15.100:0/3325404085 shutdown_connections 2024-07-26T13:46:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:08.619+0000 7f18753b8740 1 --2- 172.21.15.100:0/3325404085 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x56179785f1c0 0x561797cf54a0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:08.619+0000 7f18753b8740 1 --2- 172.21.15.100:0/3325404085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5617978cb1b0 0x561797c7edc0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:08.619+0000 7f18753b8740 1 --2- 172.21.15.100:0/3325404085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617978aa4d0 0x561797c7e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:08.619+0000 7f18477fe640 0 client.16986 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:08.619+0000 7f18753b8740 1 --2- 172.21.15.100:0/3325404085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5617978a22f0 0x561797c7e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:08.619+0000 7f18753b8740 1 -- 172.21.15.100:0/3325404085 >> 172.21.15.100:0/3325404085 conn(0x5617978bf300 msgr2=0x561797c6b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:08.619+0000 7f18753b8740 1 -- 172.21.15.100:0/3325404085 shutdown_connections 2024-07-26T13:46:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:08.619+0000 7f18753b8740 1 -- 172.21.15.100:0/3325404085 wait complete. 2024-07-26T13:46:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:08.619+0000 7f18753b8740 20 client.16986 trim_cache size 0 max 16384 2024-07-26T13:46:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-07-26T13:46:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-07-26T13:46:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-07-26T13:46:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-07-26T13:46:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:08.620+0000 7f186e49b640 1 -- 172.21.15.100:0/3779820759 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f18500b3d40 msgr2=0x7f18500b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:08.620+0000 7f186e49b640 1 --2- 172.21.15.100:0/3779820759 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f18500b3d40 0x7f18500b6250 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1860005fd0 tx=0x7f1860009500 comp rx=0 tx=0).stop 2024-07-26T13:46:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:08.620+0000 7f186e49b640 1 -- 172.21.15.100:0/3779820759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18681023e0 msgr2=0x7f1868199e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:08.620+0000 7f186e49b640 1 --2- 172.21.15.100:0/3779820759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18681023e0 0x7f1868199e90 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f1858005940 tx=0x7f18580040c0 comp rx=0 tx=0).stop 2024-07-26T13:46:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:08.620+0000 7f186e49b640 1 -- 172.21.15.100:0/3779820759 shutdown_connections 2024-07-26T13:46:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:08.620+0000 7f186e49b640 1 --2- 172.21.15.100:0/3779820759 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f18500b3d40 0x7f18500b6250 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:08.620+0000 7f186e49b640 1 --2- 172.21.15.100:0/3779820759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1868108660 0x7f186819f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:08.620+0000 7f186e49b640 1 --2- 172.21.15.100:0/3779820759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1868107d10 0x7f186819a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:08.620+0000 7f186e49b640 1 --2- 172.21.15.100:0/3779820759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18681023e0 0x7f1868199e90 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:08.620+0000 7f186e49b640 1 -- 172.21.15.100:0/3779820759 >> 172.21.15.100:0/3779820759 conn(0x7f186807a170 msgr2=0x7f1868105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:08.620+0000 7f186e49b640 1 -- 172.21.15.100:0/3779820759 shutdown_connections 2024-07-26T13:46:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:08.620+0000 7f186e49b640 1 -- 172.21.15.100:0/3779820759 wait complete. 2024-07-26T13:46:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:08 smithi100 ceph-mon[32599]: reconnect by client.15333 v1:192.168.0.1:0/1052122651 after 0 2024-07-26T13:46:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:08 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.138:6834/2831339503,v1:172.21.15.138:6835/2831339503] up:reconnect 2024-07-26T13:46:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:08 smithi100 ceph-mon[32599]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=e=up:reconnect} 2 up:standby-replay 7 up:standby 2024-07-26T13:46:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:08 smithi100 ceph-mon[32599]: pgmap v548: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 146 KiB/s wr, 33 op/s 2024-07-26T13:46:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:08 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.138:6834/2831339503,v1:172.21.15.138:6835/2831339503] up:rejoin 2024-07-26T13:46:08.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:08 smithi100 ceph-mon[32599]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=e=up:rejoin} 2 up:standby-replay 7 up:standby 2024-07-26T13:46:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:08 smithi138 ceph-mon[36954]: reconnect by client.15333 v1:192.168.0.1:0/1052122651 after 0 2024-07-26T13:46:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:08 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.138:6834/2831339503,v1:172.21.15.138:6835/2831339503] up:reconnect 2024-07-26T13:46:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:08 smithi138 ceph-mon[36954]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=e=up:reconnect} 2 up:standby-replay 7 up:standby 2024-07-26T13:46:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:08 smithi138 ceph-mon[36954]: pgmap v548: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 146 KiB/s wr, 33 op/s 2024-07-26T13:46:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:08 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.138:6834/2831339503,v1:172.21.15.138:6835/2831339503] up:rejoin 2024-07-26T13:46:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:08 smithi138 ceph-mon[36954]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=e=up:rejoin} 2 up:standby-replay 7 up:standby 2024-07-26T13:46:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:08 smithi167 ceph-mon[37140]: reconnect by client.15333 v1:192.168.0.1:0/1052122651 after 0 2024-07-26T13:46:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:08 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.138:6834/2831339503,v1:172.21.15.138:6835/2831339503] up:reconnect 2024-07-26T13:46:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:08 smithi167 ceph-mon[37140]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=e=up:reconnect} 2 up:standby-replay 7 up:standby 2024-07-26T13:46:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:08 smithi167 ceph-mon[37140]: pgmap v548: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 146 KiB/s wr, 33 op/s 2024-07-26T13:46:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:08 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.138:6834/2831339503,v1:172.21.15.138:6835/2831339503] up:rejoin 2024-07-26T13:46:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:08 smithi167 ceph-mon[37140]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=e=up:rejoin} 2 up:standby-replay 7 up:standby 2024-07-26T13:46:09.420 INFO:tasks.workunit.client.0.smithi100.stdout: 1 223340 2.91 MB/sec execute 143 sec latency 15078.948 ms 2024-07-26T13:46:09.454 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-26T13:46:09.454 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-26T13:46:09.855 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:09 smithi100 ceph-mon[32599]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-07-26T13:46:09.855 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:09 smithi100 ceph-mon[32599]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-26T13:46:09.855 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:09 smithi100 ceph-mon[32599]: Cluster is now healthy 2024-07-26T13:46:09.855 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:09 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.138:6834/2831339503,v1:172.21.15.138:6835/2831339503] up:active 2024-07-26T13:46:09.855 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:09 smithi100 ceph-mon[32599]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:46:09.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.852+0000 7f02a6f68640 1 -- 172.21.15.100:0/2587113045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02a010d890 msgr2=0x7f02a010fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:09.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.852+0000 7f02a6f68640 1 --2- 172.21.15.100:0/2587113045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02a010d890 0x7f02a010fd60 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f0298009ca0 tx=0x7f029802a8c0 comp rx=0 tx=0).stop 2024-07-26T13:46:09.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.852+0000 7f02a6f68640 1 -- 172.21.15.100:0/2587113045 shutdown_connections 2024-07-26T13:46:09.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.852+0000 7f02a6f68640 1 --2- 172.21.15.100:0/2587113045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02a010d890 0x7f02a010fd60 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:09.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.852+0000 7f02a6f68640 1 --2- 172.21.15.100:0/2587113045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02a0104d10 0x7f02a010d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:09.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.852+0000 7f02a6f68640 1 --2- 172.21.15.100:0/2587113045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02a01042e0 0x7f02a01046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:09.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.852+0000 7f02a6f68640 1 -- 172.21.15.100:0/2587113045 >> 172.21.15.100:0/2587113045 conn(0x7f02a0079180 msgr2=0x7f02a0079590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:09.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.852+0000 7f02a6f68640 1 -- 172.21.15.100:0/2587113045 shutdown_connections 2024-07-26T13:46:09.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.852+0000 7f02a6f68640 1 -- 172.21.15.100:0/2587113045 wait complete. 2024-07-26T13:46:09.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.852+0000 7f02a6f68640 1 Processor -- start 2024-07-26T13:46:09.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.853+0000 7f02a6f68640 1 -- start start 2024-07-26T13:46:09.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.853+0000 7f02a6f68640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02a01042e0 0x7f02a0199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:09.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.853+0000 7f02a6f68640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02a0104d10 0x7f02a019a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:09.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.853+0000 7f02a6f68640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02a010d890 0x7f02a019f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:09.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.853+0000 7f02a6f68640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f02a01134d0 con 0x7f02a01042e0 2024-07-26T13:46:09.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.853+0000 7f02a6f68640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f02a0113350 con 0x7f02a010d890 2024-07-26T13:46:09.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.853+0000 7f02a6f68640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f02a0113650 con 0x7f02a0104d10 2024-07-26T13:46:09.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.853+0000 7f02a4d04640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02a01042e0 0x7f02a0199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:09.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.853+0000 7f02a4d04640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02a01042e0 0x7f02a0199e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48708/0 (socket says 172.21.15.100:48708) 2024-07-26T13:46:09.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.853+0000 7f02a4d04640 1 -- 172.21.15.100:0/3554906928 learned_addr learned my addr 172.21.15.100:0/3554906928 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:09.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.853+0000 7f029ffff640 1 --2- 172.21.15.100:0/3554906928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02a0104d10 0x7f02a019a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:09.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.853+0000 7f02a5505640 1 --2- 172.21.15.100:0/3554906928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02a010d890 0x7f02a019f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:09.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.853+0000 7f02a4d04640 1 -- 172.21.15.100:0/3554906928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02a0104d10 msgr2=0x7f02a019a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:09.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.853+0000 7f02a4d04640 1 --2- 172.21.15.100:0/3554906928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02a0104d10 0x7f02a019a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:09.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.853+0000 7f02a4d04640 1 -- 172.21.15.100:0/3554906928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02a010d890 msgr2=0x7f02a019f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:09.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.853+0000 7f02a4d04640 1 --2- 172.21.15.100:0/3554906928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02a010d890 0x7f02a019f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:09.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.853+0000 7f02a4d04640 1 -- 172.21.15.100:0/3554906928 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a019fd90 con 0x7f02a01042e0 2024-07-26T13:46:09.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.854+0000 7f02a4d04640 1 --2- 172.21.15.100:0/3554906928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02a01042e0 0x7f02a0199e20 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f029400e9c0 tx=0x7f029400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:09.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.854+0000 7f029dffb640 1 -- 172.21.15.100:0/3554906928 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029400cd80 con 0x7f02a01042e0 2024-07-26T13:46:09.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.854+0000 7f029dffb640 1 -- 172.21.15.100:0/3554906928 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02940035f0 con 0x7f02a01042e0 2024-07-26T13:46:09.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.854+0000 7f02a6f68640 1 -- 172.21.15.100:0/3554906928 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02a01a0080 con 0x7f02a01042e0 2024-07-26T13:46:09.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.854+0000 7f029dffb640 1 -- 172.21.15.100:0/3554906928 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0294010690 con 0x7f02a01042e0 2024-07-26T13:46:09.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.854+0000 7f02a6f68640 1 -- 172.21.15.100:0/3554906928 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02a01a67e0 con 0x7f02a01042e0 2024-07-26T13:46:09.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.855+0000 7f02a6f68640 1 -- 172.21.15.100:0/3554906928 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02a006bc10 con 0x7f02a01042e0 2024-07-26T13:46:09.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.857+0000 7f029dffb640 1 -- 172.21.15.100:0/3554906928 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f0294004000 con 0x7f02a01042e0 2024-07-26T13:46:09.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.857+0000 7f029dffb640 1 --2- 172.21.15.100:0/3554906928 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f02900b3d40 0x7f02900b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:09.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.857+0000 7f029ffff640 1 --2- 172.21.15.100:0/3554906928 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f02900b3d40 0x7f02900b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:09.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.857+0000 7f029dffb640 1 -- 172.21.15.100:0/3554906928 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7f02940e46b0 con 0x7f02a01042e0 2024-07-26T13:46:09.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.858+0000 7f029ffff640 1 --2- 172.21.15.100:0/3554906928 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f02900b3d40 0x7f02900b6250 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f02a0101d20 tx=0x7f028c005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:09.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:09.863+0000 7f029dffb640 1 -- 172.21.15.100:0/3554906928 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f02940ab8e0 con 0x7f02a01042e0 2024-07-26T13:46:09.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:09 smithi138 ceph-mon[36954]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-07-26T13:46:09.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:09 smithi138 ceph-mon[36954]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-26T13:46:09.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:09 smithi138 ceph-mon[36954]: Cluster is now healthy 2024-07-26T13:46:09.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:09 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.138:6834/2831339503,v1:172.21.15.138:6835/2831339503] up:active 2024-07-26T13:46:09.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:09 smithi138 ceph-mon[36954]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:46:09.977 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:46:10.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:09 smithi167 ceph-mon[37140]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-07-26T13:46:10.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:09 smithi167 ceph-mon[37140]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-26T13:46:10.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:09 smithi167 ceph-mon[37140]: Cluster is now healthy 2024-07-26T13:46:10.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:09 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.138:6834/2831339503,v1:172.21.15.138:6835/2831339503] up:active 2024-07-26T13:46:10.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:09 smithi167 ceph-mon[37140]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:46:10.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:10.116+0000 7f02a6f68640 1 -- 172.21.15.100:0/3554906928 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02a01a6a90 con 0x7f02a01042e0 2024-07-26T13:46:10.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:10.117+0000 7f029dffb640 1 -- 172.21.15.100:0/3554906928 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+11928 (secure 0 0 0) 0x7f02940ab1e0 con 0x7f02a01042e0 2024-07-26T13:46:10.118 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:46:10.118 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":53,"btime":"2024-07-26T13:46:08:649224+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14790,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/313501409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":313501409},{"type":"v1","addr":"172.21.15.100:6839","nonce":313501409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/1402975506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":1402975506},{"type":"v1","addr":"172.21.15.100:6841","nonce":1402975506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":53,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-26T13:37:01.281843+0000","modified":"2024-07-26T13:46:08.649219+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24607,"mds_1":24605,"mds_2":25997},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24575":{"gid":24575,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/3393225483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3393225483},{"type":"v1","addr":"172.21.15.138:6837","nonce":3393225483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.138:6839/1580925471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1580925471},{"type":"v1","addr":"172.21.15.138:6839","nonce":1580925471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/398150514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":398150514},{"type":"v1","addr":"172.21.15.167:6839","nonce":398150514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25997":{"gid":25997,"name":"e","rank":2,"incarnation":49,"state":"up:active","state_seq":23,"addr":"172.21.15.138:6835/2831339503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2831339503},{"type":"v1","addr":"172.21.15.138:6835","nonce":2831339503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26446":{"gid":26446,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/3519222898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3519222898},{"type":"v1","addr":"172.21.15.167:6837","nonce":3519222898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24607,"qdb_cluster":[24605,24607,25997]},"id":1}]} 2024-07-26T13:46:10.118 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 53 2024-07-26T13:46:10.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:10.119+0000 7f02a6f68640 1 -- 172.21.15.100:0/3554906928 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f02900b3d40 msgr2=0x7f02900b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:10.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:10.119+0000 7f02a6f68640 1 --2- 172.21.15.100:0/3554906928 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f02900b3d40 0x7f02900b6250 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f02a0101d20 tx=0x7f028c005f90 comp rx=0 tx=0).stop 2024-07-26T13:46:10.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:10.119+0000 7f02a6f68640 1 -- 172.21.15.100:0/3554906928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02a01042e0 msgr2=0x7f02a0199e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:10.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:10.119+0000 7f02a6f68640 1 --2- 172.21.15.100:0/3554906928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02a01042e0 0x7f02a0199e20 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f029400e9c0 tx=0x7f029400ee80 comp rx=0 tx=0).stop 2024-07-26T13:46:10.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:10.119+0000 7f02a6f68640 1 -- 172.21.15.100:0/3554906928 shutdown_connections 2024-07-26T13:46:10.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:10.119+0000 7f02a6f68640 1 --2- 172.21.15.100:0/3554906928 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f02900b3d40 0x7f02900b6250 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:10.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:10.119+0000 7f02a6f68640 1 --2- 172.21.15.100:0/3554906928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02a010d890 0x7f02a019f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:10.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:10.119+0000 7f02a6f68640 1 --2- 172.21.15.100:0/3554906928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02a0104d10 0x7f02a019a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:10.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:10.119+0000 7f02a6f68640 1 --2- 172.21.15.100:0/3554906928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02a01042e0 0x7f02a0199e20 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:10.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:10.119+0000 7f02a6f68640 1 -- 172.21.15.100:0/3554906928 >> 172.21.15.100:0/3554906928 conn(0x7f02a0079180 msgr2=0x7f02a0100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:10.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:10.119+0000 7f02a6f68640 1 -- 172.21.15.100:0/3554906928 shutdown_connections 2024-07-26T13:46:10.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:10.119+0000 7f02a6f68640 1 -- 172.21.15.100:0/3554906928 wait complete. 2024-07-26T13:46:10.420 INFO:tasks.workunit.client.0.smithi100.stdout: 1 224797 2.94 MB/sec execute 144 sec latency 21.614 ms 2024-07-26T13:46:10.616 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:46:10 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 2024-07-26T13:46:10.278+0000 7f740ace1640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-07-26T13:46:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:10 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/3554906928' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:46:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:10 smithi138 ceph-mon[36954]: pgmap v549: 97 pgs: 97 active+clean; 87 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 436 KiB/s wr, 50 op/s 2024-07-26T13:46:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:10 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/3554906928' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:46:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:10 smithi100 ceph-mon[32599]: pgmap v549: 97 pgs: 97 active+clean; 87 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 436 KiB/s wr, 50 op/s 2024-07-26T13:46:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:10 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/3554906928' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:46:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:10 smithi167 ceph-mon[37140]: pgmap v549: 97 pgs: 97 active+clean; 87 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 436 KiB/s wr, 50 op/s 2024-07-26T13:46:11.420 INFO:tasks.workunit.client.0.smithi100.stdout: 1 227196 3.02 MB/sec execute 145 sec latency 26.508 ms 2024-07-26T13:46:11.947 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24607 quiesce db / --timeout 90 --expiration 141.95 --await-for 45.0' 2024-07-26T13:46:11.947 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- timeout 105 ceph tell mds.24607 quiesce db / --timeout 90 --expiration 141.95 --await-for 45.0 2024-07-26T13:46:12.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.399+0000 7f19aa72a640 1 -- 172.21.15.100:0/1734845822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19a4108760 msgr2=0x7f19a410ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:12.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.399+0000 7f19aa72a640 1 --2- 172.21.15.100:0/1734845822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19a4108760 0x7f19a410ef90 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f199c009ca0 tx=0x7f199c02a8c0 comp rx=0 tx=0).stop 2024-07-26T13:46:12.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.399+0000 7f19aa72a640 1 -- 172.21.15.100:0/1734845822 shutdown_connections 2024-07-26T13:46:12.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.399+0000 7f19aa72a640 1 --2- 172.21.15.100:0/1734845822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19a4108760 0x7f19a410ef90 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:12.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.399+0000 7f19aa72a640 1 --2- 172.21.15.100:0/1734845822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19a4107e10 0x7f19a4108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:12.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.399+0000 7f19aa72a640 1 --2- 172.21.15.100:0/1734845822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19a41024d0 0x7f19a41028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:12.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.399+0000 7f19aa72a640 1 -- 172.21.15.100:0/1734845822 >> 172.21.15.100:0/1734845822 conn(0x7f19a4079210 msgr2=0x7f19a4079620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:12.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.399+0000 7f19aa72a640 1 -- 172.21.15.100:0/1734845822 shutdown_connections 2024-07-26T13:46:12.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.399+0000 7f19aa72a640 1 -- 172.21.15.100:0/1734845822 wait complete. 2024-07-26T13:46:12.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.400+0000 7f19aa72a640 1 Processor -- start 2024-07-26T13:46:12.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.400+0000 7f19aa72a640 1 -- start start 2024-07-26T13:46:12.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.401+0000 7f19aa72a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19a41024d0 0x7f19a419ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:12.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.401+0000 7f19aa72a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19a4107e10 0x7f19a419b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:12.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.401+0000 7f19aa72a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19a4108760 0x7f19a419b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:12.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.401+0000 7f19aa72a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f19a41123e0 con 0x7f19a4107e10 2024-07-26T13:46:12.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.401+0000 7f19aa72a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f19a4112260 con 0x7f19a4108760 2024-07-26T13:46:12.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.401+0000 7f19aa72a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f19a4112560 con 0x7f19a41024d0 2024-07-26T13:46:12.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.401+0000 7f19a37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19a4107e10 0x7f19a419b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:12.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.401+0000 7f19a37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19a4107e10 0x7f19a419b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48726/0 (socket says 172.21.15.100:48726) 2024-07-26T13:46:12.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.401+0000 7f19a37fe640 1 -- 172.21.15.100:0/1149788399 learned_addr learned my addr 172.21.15.100:0/1149788399 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:12.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.401+0000 7f19a3fff640 1 --2- 172.21.15.100:0/1149788399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19a41024d0 0x7f19a419ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:12.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.401+0000 7f19a8cc7640 1 --2- 172.21.15.100:0/1149788399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19a4108760 0x7f19a419b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:12.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.401+0000 7f19a37fe640 1 -- 172.21.15.100:0/1149788399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19a41024d0 msgr2=0x7f19a419ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:12.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.401+0000 7f19a37fe640 1 --2- 172.21.15.100:0/1149788399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19a41024d0 0x7f19a419ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:12.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.401+0000 7f19a37fe640 1 -- 172.21.15.100:0/1149788399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19a4108760 msgr2=0x7f19a419b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:12.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.401+0000 7f19a37fe640 1 --2- 172.21.15.100:0/1149788399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19a4108760 0x7f19a419b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:12.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.401+0000 7f19a37fe640 1 -- 172.21.15.100:0/1149788399 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a41a5f20 con 0x7f19a4107e10 2024-07-26T13:46:12.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.401+0000 7f19a37fe640 1 --2- 172.21.15.100:0/1149788399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19a4107e10 0x7f19a419b1d0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f199800e9f0 tx=0x7f199800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:12.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.402+0000 7f19a17fa640 1 -- 172.21.15.100:0/1149788399 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199800ccc0 con 0x7f19a4107e10 2024-07-26T13:46:12.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.402+0000 7f19aa72a640 1 -- 172.21.15.100:0/1149788399 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19a41a6210 con 0x7f19a4107e10 2024-07-26T13:46:12.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.402+0000 7f19a17fa640 1 -- 172.21.15.100:0/1149788399 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19980054d0 con 0x7f19a4107e10 2024-07-26T13:46:12.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.402+0000 7f19a17fa640 1 -- 172.21.15.100:0/1149788399 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1998010430 con 0x7f19a4107e10 2024-07-26T13:46:12.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.402+0000 7f19aa72a640 1 -- 172.21.15.100:0/1149788399 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19a41a6700 con 0x7f19a4107e10 2024-07-26T13:46:12.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.404+0000 7f19a17fa640 1 -- 172.21.15.100:0/1149788399 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f19980046c0 con 0x7f19a4107e10 2024-07-26T13:46:12.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.405+0000 7f19b0cae740 1 Processor -- start 2024-07-26T13:46:12.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.405+0000 7f19b0cae740 1 -- start start 2024-07-26T13:46:12.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.405+0000 7f19a17fa640 1 --2- 172.21.15.100:0/1149788399 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f198c0b3d70 0x7f198c0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:12.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.405+0000 7f19b0cae740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df8a3a1350 0x55df8a7463b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:12.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.405+0000 7f19b0cae740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df8a378490 0x55df8a7468f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:12.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.405+0000 7f19b0cae740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df8a380670 0x55df8a746e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:12.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.405+0000 7f19b0cae740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55df8a62e2f0 con 0x55df8a3a1350 2024-07-26T13:46:12.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.405+0000 7f19b0cae740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55df8a6bca80 con 0x55df8a378490 2024-07-26T13:46:12.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.405+0000 7f19b0cae740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55df8a6abf80 con 0x55df8a380670 2024-07-26T13:46:12.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.406+0000 7f19a3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df8a3a1350 0x55df8a7463b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:12.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.406+0000 7f19a8cc7640 1 --2- 172.21.15.100:0/1149788399 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f198c0b3d70 0x7f198c0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:12.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.406+0000 7f19a37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df8a378490 0x55df8a7468f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:12.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.406+0000 7f19a3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df8a3a1350 0x55df8a7463b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48742/0 (socket says 172.21.15.100:48742) 2024-07-26T13:46:12.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.406+0000 7f19a3fff640 1 -- 172.21.15.100:0/1115965862 learned_addr learned my addr 172.21.15.100:0/1115965862 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:12.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.406+0000 7f19a8cc7640 1 --2- 172.21.15.100:0/1115965862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df8a380670 0x55df8a746e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:12.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.406+0000 7f19a17fa640 1 -- 172.21.15.100:0/1149788399 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7f19980e4400 con 0x7f19a4107e10 2024-07-26T13:46:12.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.406+0000 7f19a8cc7640 1 -- 172.21.15.100:0/1115965862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df8a378490 msgr2=0x55df8a7468f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:12.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.406+0000 7f19a8cc7640 1 --2- 172.21.15.100:0/1115965862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df8a378490 0x55df8a7468f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:12.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.406+0000 7f19a8cc7640 1 -- 172.21.15.100:0/1115965862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df8a3a1350 msgr2=0x55df8a7463b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:12.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.406+0000 7f19a8cc7640 1 --2- 172.21.15.100:0/1115965862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df8a3a1350 0x55df8a7463b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:12.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.406+0000 7f19a8cc7640 1 -- 172.21.15.100:0/1115965862 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df8a40c930 con 0x55df8a380670 2024-07-26T13:46:12.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.406+0000 7f19a8cc7640 1 --2- 172.21.15.100:0/1115965862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df8a380670 0x55df8a746e30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f199c005e30 tx=0x7f199c032fe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc9b0649f90b11c1 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:12.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.406+0000 7f19a8cc7640 1 --2- 172.21.15.100:0/1149788399 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f198c0b3d70 0x7f198c0b6280 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f199c02ade0 tx=0x7f199c036000 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:12.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.407+0000 7f198a7fc640 1 -- 172.21.15.100:0/1115965862 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199c04f070 con 0x55df8a380670 2024-07-26T13:46:12.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.407+0000 7f198a7fc640 1 -- 172.21.15.100:0/1115965862 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f199c036070 con 0x55df8a380670 2024-07-26T13:46:12.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.407+0000 7f198a7fc640 1 -- 172.21.15.100:0/1115965862 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199c045040 con 0x55df8a380670 2024-07-26T13:46:12.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.407+0000 7f19b0cae740 1 -- 172.21.15.100:0/1115965862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df8a380670 msgr2=0x55df8a746e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:12.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.407+0000 7f19b0cae740 1 --2- 172.21.15.100:0/1115965862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df8a380670 0x55df8a746e30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f199c005e30 tx=0x7f199c032fe0 comp rx=0 tx=0).stop 2024-07-26T13:46:12.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.408+0000 7f19b0cae740 1 -- 172.21.15.100:0/1115965862 shutdown_connections 2024-07-26T13:46:12.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.408+0000 7f19b0cae740 1 --2- 172.21.15.100:0/1115965862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df8a380670 0x55df8a746e30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:12.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.408+0000 7f19b0cae740 1 --2- 172.21.15.100:0/1115965862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df8a378490 0x55df8a7468f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:12.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.408+0000 7f19b0cae740 1 --2- 172.21.15.100:0/1115965862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df8a3a1350 0x55df8a7463b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:12.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.408+0000 7f19b0cae740 1 -- 172.21.15.100:0/1115965862 >> 172.21.15.100:0/1115965862 conn(0x55df8a395740 msgr2=0x55df8a7442a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:12.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.408+0000 7f19b0cae740 1 -- 172.21.15.100:0/1115965862 shutdown_connections 2024-07-26T13:46:12.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.408+0000 7f19b0cae740 1 -- 172.21.15.100:0/1115965862 wait complete. 2024-07-26T13:46:12.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.408+0000 7f19b0cae740 1 -- start start 2024-07-26T13:46:12.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.408+0000 7f19b0cae740 1 Processor -- start 2024-07-26T13:46:12.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.409+0000 7f19b0cae740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df8a378490 0x55df8a7563a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:12.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.409+0000 7f19b0cae740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df8a380670 0x55df8a7568e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:12.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.409+0000 7f19b0cae740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df8a3a1350 0x55df8a762040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:12.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.409+0000 7f19b0cae740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55df8a64e4f0 con 0x55df8a3a1350 2024-07-26T13:46:12.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.409+0000 7f19b0cae740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55df8a6affc0 con 0x55df8a378490 2024-07-26T13:46:12.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.409+0000 7f19b0cae740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55df8a50f890 con 0x55df8a380670 2024-07-26T13:46:12.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.409+0000 7f19b0cae740 10 client.0 _ll_register_callbacks cb 0x55df8a658500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:46:12.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.409+0000 7f19a3fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df8a378490 0x55df8a7563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:12.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.409+0000 7f19a8cc7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df8a3a1350 0x55df8a762040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:12.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.409+0000 7f19a3fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df8a378490 0x55df8a7563a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:49854/0 (socket says 172.21.15.100:49854) 2024-07-26T13:46:12.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.409+0000 7f19a3fff640 1 -- 172.21.15.100:0/1916197094 learned_addr learned my addr 172.21.15.100:0/1916197094 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:12.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.409+0000 7f19a37fe640 1 --2- 172.21.15.100:0/1916197094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df8a380670 0x55df8a7568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:12.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.409+0000 7f19a8cc7640 1 -- 172.21.15.100:0/1916197094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df8a380670 msgr2=0x55df8a7568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:12.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.409+0000 7f19a8cc7640 1 --2- 172.21.15.100:0/1916197094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df8a380670 0x55df8a7568e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:12.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.409+0000 7f19a8cc7640 1 -- 172.21.15.100:0/1916197094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df8a378490 msgr2=0x55df8a7563a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:12.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.409+0000 7f19a8cc7640 1 --2- 172.21.15.100:0/1916197094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df8a378490 0x55df8a7563a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:12.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.409+0000 7f19a8cc7640 1 -- 172.21.15.100:0/1916197094 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df8a65dc50 con 0x55df8a3a1350 2024-07-26T13:46:12.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.410+0000 7f19a8cc7640 1 --2- 172.21.15.100:0/1916197094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df8a3a1350 0x55df8a762040 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f199c005e30 tx=0x7f199c0045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53231212fa644e5e server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:12.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.410+0000 7f19837fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:46:12.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.410+0000 7f19837fe640 1 -- 172.21.15.100:0/1916197094 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199c04f040 con 0x55df8a3a1350 2024-07-26T13:46:12.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.410+0000 7f19837fe640 1 -- 172.21.15.100:0/1916197094 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7f199c02f660 con 0x55df8a3a1350 2024-07-26T13:46:12.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.410+0000 7f19b0cae740 1 -- 172.21.15.100:0/1916197094 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f199c036070 con 0x55df8a3a1350 2024-07-26T13:46:12.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.410+0000 7f19837fe640 1 -- 172.21.15.100:0/1916197094 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f199c02f370 con 0x55df8a3a1350 2024-07-26T13:46:12.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.410+0000 7f19b0cae740 10 client.17064 fetch_fsmap learned FSMap version 53 2024-07-26T13:46:12.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.410+0000 7f19b0cae740 1 -- 172.21.15.100:0/1916197094 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55df8a764d60 con 0x55df8a3a1350 2024-07-26T13:46:12.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.410+0000 7f19837fe640 1 -- 172.21.15.100:0/1916197094 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f199c036070 con 0x55df8a3a1350 2024-07-26T13:46:12.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.410+0000 7f19837fe640 1 -- 172.21.15.100:0/1916197094 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7f199c061030 con 0x55df8a3a1350 2024-07-26T13:46:12.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.411+0000 7f19b0cae740 10 client.17064 fetch_fsmap finished waiting for FSMap version 53 2024-07-26T13:46:12.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.411+0000 7f19b0cae740 10 client.17064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:46:12.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.411+0000 7f19b0cae740 20 client.17064 populate_metadata read hostname 'smithi100' 2024-07-26T13:46:12.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.411+0000 7f19b0cae740 1 --2- 172.21.15.100:0/1916197094 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55df8a35fcd0 0x55df8a7680c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:12.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.411+0000 7f19b0cae740 4 client.17064 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:46:12.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.411+0000 7f19b0cae740 1 -- 172.21.15.100:0/1916197094 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df8a6be9d0 con 0x55df8a35fcd0 2024-07-26T13:46:12.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.411+0000 7f19a3fff640 1 --2- 172.21.15.100:0/1916197094 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55df8a35fcd0 0x55df8a7680c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:12.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.411+0000 7f19837fe640 10 client.17064 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:12.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.411+0000 7f19a3fff640 1 --2- 172.21.15.100:0/1916197094 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55df8a35fcd0 0x55df8a7680c0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e1eb831ed254230 server_cookie=2d77ed240304aa58 in_seq=0 out_seq=0 2024-07-26T13:46:12.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.412+0000 7f19837fe640 1 -- 172.21.15.100:0/1916197094 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55df8a6be9d0 con 0x55df8a35fcd0 2024-07-26T13:46:12.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.412+0000 7f19837fe640 10 client.17064 handle_command_reply: tid=0 2024-07-26T13:46:12.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.412+0000 7f19b0cae740 1 client.17064 shutdown 2024-07-26T13:46:12.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.412+0000 7f19b0cae740 1 -- 172.21.15.100:0/1916197094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df8a3a1350 msgr2=0x55df8a762040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:12.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.412+0000 7f19b0cae740 1 --2- 172.21.15.100:0/1916197094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df8a3a1350 0x55df8a762040 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f199c005e30 tx=0x7f199c0045c0 comp rx=0 tx=0).stop 2024-07-26T13:46:12.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.412+0000 7f19b0cae740 1 -- 172.21.15.100:0/1916197094 shutdown_connections 2024-07-26T13:46:12.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.412+0000 7f19b0cae740 1 --2- 172.21.15.100:0/1916197094 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55df8a35fcd0 0x55df8a7680c0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:12.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.412+0000 7f19b0cae740 1 --2- 172.21.15.100:0/1916197094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df8a3a1350 0x55df8a762040 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:12.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.412+0000 7f19b0cae740 1 --2- 172.21.15.100:0/1916197094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df8a380670 0x55df8a7568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:12.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.413+0000 7f19b0cae740 1 --2- 172.21.15.100:0/1916197094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df8a378490 0x55df8a7563a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:12.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.413+0000 7f19b0cae740 1 -- 172.21.15.100:0/1916197094 >> 172.21.15.100:0/1916197094 conn(0x55df8a395740 msgr2=0x55df8a7428c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:12.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.413+0000 7f19837fe640 0 client.17064 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:12.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.413+0000 7f19b0cae740 1 -- 172.21.15.100:0/1916197094 shutdown_connections 2024-07-26T13:46:12.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.413+0000 7f19b0cae740 1 -- 172.21.15.100:0/1916197094 wait complete. 2024-07-26T13:46:12.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.413+0000 7f19b0cae740 20 client.17064 trim_cache size 0 max 16384 2024-07-26T13:46:12.421 INFO:tasks.workunit.client.0.smithi100.stdout: 1 228678 3.05 MB/sec execute 146 sec latency 18.907 ms 2024-07-26T13:46:12.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.436+0000 7f19b0cae740 1 Processor -- start 2024-07-26T13:46:12.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.436+0000 7f19b0cae740 1 -- start start 2024-07-26T13:46:12.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.436+0000 7f19b0cae740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df8a378490 0x55df8a761fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:12.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.436+0000 7f19b0cae740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df8a380670 0x55df8a74bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:12.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.436+0000 7f19b0cae740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df8a3a1350 0x55df8a757dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:12.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.436+0000 7f19b0cae740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f19a41af930 con 0x55df8a380670 2024-07-26T13:46:12.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.436+0000 7f19b0cae740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f19a41ad200 con 0x55df8a3a1350 2024-07-26T13:46:12.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.436+0000 7f19b0cae740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f19a41ab050 con 0x55df8a378490 2024-07-26T13:46:12.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.436+0000 7f19a37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df8a380670 0x55df8a74bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:12.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.436+0000 7f19a37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df8a380670 0x55df8a74bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48760/0 (socket says 172.21.15.100:48760) 2024-07-26T13:46:12.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.436+0000 7f19a37fe640 1 -- 172.21.15.100:0/573327759 learned_addr learned my addr 172.21.15.100:0/573327759 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:12.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.436+0000 7f19a8cc7640 1 --2- 172.21.15.100:0/573327759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df8a3a1350 0x55df8a757dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:12.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.436+0000 7f19a3fff640 1 --2- 172.21.15.100:0/573327759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df8a378490 0x55df8a761fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:12.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.437+0000 7f19a37fe640 1 -- 172.21.15.100:0/573327759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df8a378490 msgr2=0x55df8a761fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:12.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.437+0000 7f19a37fe640 1 --2- 172.21.15.100:0/573327759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df8a378490 0x55df8a761fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:12.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.437+0000 7f19a37fe640 1 -- 172.21.15.100:0/573327759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df8a3a1350 msgr2=0x55df8a757dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:12.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.437+0000 7f19a37fe640 1 --2- 172.21.15.100:0/573327759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df8a3a1350 0x55df8a757dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:12.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.437+0000 7f19a37fe640 1 -- 172.21.15.100:0/573327759 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df8a73fa90 con 0x55df8a380670 2024-07-26T13:46:12.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.437+0000 7f19a37fe640 1 --2- 172.21.15.100:0/573327759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df8a380670 0x55df8a74bef0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f1998004aa0 tx=0x7f199800cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae7ca6be3972b10 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:12.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.437+0000 7f1989ffb640 1 -- 172.21.15.100:0/573327759 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19980aaa40 con 0x55df8a380670 2024-07-26T13:46:12.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.437+0000 7f1989ffb640 1 -- 172.21.15.100:0/573327759 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1998005680 con 0x55df8a380670 2024-07-26T13:46:12.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.437+0000 7f1989ffb640 1 -- 172.21.15.100:0/573327759 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19980b3fe0 con 0x55df8a380670 2024-07-26T13:46:12.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.438+0000 7f19b0cae740 1 -- 172.21.15.100:0/573327759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df8a380670 msgr2=0x55df8a74bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:12.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.438+0000 7f19b0cae740 1 --2- 172.21.15.100:0/573327759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df8a380670 0x55df8a74bef0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f1998004aa0 tx=0x7f199800cf50 comp rx=0 tx=0).stop 2024-07-26T13:46:12.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.438+0000 7f19b0cae740 1 -- 172.21.15.100:0/573327759 shutdown_connections 2024-07-26T13:46:12.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.438+0000 7f19b0cae740 1 --2- 172.21.15.100:0/573327759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df8a3a1350 0x55df8a757dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:12.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.438+0000 7f19b0cae740 1 --2- 172.21.15.100:0/573327759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df8a380670 0x55df8a74bef0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:12.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.438+0000 7f19b0cae740 1 --2- 172.21.15.100:0/573327759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df8a378490 0x55df8a761fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:12.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.438+0000 7f19b0cae740 1 -- 172.21.15.100:0/573327759 >> 172.21.15.100:0/573327759 conn(0x55df8a395740 msgr2=0x55df8a7576c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:12.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.438+0000 7f19b0cae740 1 -- 172.21.15.100:0/573327759 shutdown_connections 2024-07-26T13:46:12.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.438+0000 7f19b0cae740 1 -- 172.21.15.100:0/573327759 wait complete. 2024-07-26T13:46:12.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.439+0000 7f19b0cae740 1 -- start start 2024-07-26T13:46:12.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.439+0000 7f19b0cae740 1 Processor -- start 2024-07-26T13:46:12.449 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:46:12.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.439+0000 7f19b0cae740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df8a378490 0x55df8a76b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:12.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.439+0000 7f19b0cae740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df8a380670 0x55df8a76b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:12.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.439+0000 7f19b0cae740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df8a3a1350 0x55df8a766110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:12.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.439+0000 7f19b0cae740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f19a41ab6c0 con 0x55df8a378490 2024-07-26T13:46:12.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.439+0000 7f19b0cae740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f19a41acb30 con 0x55df8a380670 2024-07-26T13:46:12.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.439+0000 7f19b0cae740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f19a41abd90 con 0x55df8a3a1350 2024-07-26T13:46:12.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.439+0000 7f19b0cae740 10 client.0 _ll_register_callbacks cb 0x7f199c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:46:12.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.439+0000 7f19a3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df8a378490 0x55df8a76b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:12.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.439+0000 7f19a3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df8a378490 0x55df8a76b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48774/0 (socket says 172.21.15.100:48774) 2024-07-26T13:46:12.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.439+0000 7f19a3fff640 1 -- 172.21.15.100:0/2710346559 learned_addr learned my addr 172.21.15.100:0/2710346559 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:12.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.439+0000 7f19a37fe640 1 --2- 172.21.15.100:0/2710346559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df8a380670 0x55df8a76b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:12.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.439+0000 7f19a8cc7640 1 --2- 172.21.15.100:0/2710346559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df8a3a1350 0x55df8a766110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:12.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.440+0000 7f19a37fe640 1 -- 172.21.15.100:0/2710346559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df8a3a1350 msgr2=0x55df8a766110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:12.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.440+0000 7f19a37fe640 1 --2- 172.21.15.100:0/2710346559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df8a3a1350 0x55df8a766110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:12.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.440+0000 7f19a37fe640 1 -- 172.21.15.100:0/2710346559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df8a378490 msgr2=0x55df8a76b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:12.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.440+0000 7f19a37fe640 1 --2- 172.21.15.100:0/2710346559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df8a378490 0x55df8a76b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:12.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.440+0000 7f19a37fe640 1 -- 172.21.15.100:0/2710346559 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df8a7668a0 con 0x55df8a380670 2024-07-26T13:46:12.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.440+0000 7f19a37fe640 1 --2- 172.21.15.100:0/2710346559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df8a380670 0x55df8a76b8a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f19980aae70 tx=0x7f199800c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc51954b6efe3cce server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:12.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.440+0000 7f19837fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-07-26T13:46:12.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.440+0000 7f19837fe640 1 -- 172.21.15.100:0/2710346559 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19980aaa40 con 0x55df8a380670 2024-07-26T13:46:12.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.440+0000 7f19b0cae740 1 -- 172.21.15.100:0/2710346559 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1998005680 con 0x55df8a380670 2024-07-26T13:46:12.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.440+0000 7f19837fe640 1 -- 172.21.15.100:0/2710346559 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7f19980be5a0 con 0x55df8a380670 2024-07-26T13:46:12.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.440+0000 7f19837fe640 1 -- 172.21.15.100:0/2710346559 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f19980aaf30 con 0x55df8a380670 2024-07-26T13:46:12.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.440+0000 7f19b0cae740 10 client.26309 fetch_fsmap learned FSMap version 53 2024-07-26T13:46:12.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.440+0000 7f19b0cae740 1 -- 172.21.15.100:0/2710346559 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55df8a7d5890 con 0x55df8a380670 2024-07-26T13:46:12.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.441+0000 7f19837fe640 1 -- 172.21.15.100:0/2710346559 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1998005680 con 0x55df8a380670 2024-07-26T13:46:12.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.441+0000 7f19837fe640 1 -- 172.21.15.100:0/2710346559 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7f19980aa190 con 0x55df8a380670 2024-07-26T13:46:12.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.441+0000 7f19b0cae740 10 client.26309 fetch_fsmap finished waiting for FSMap version 53 2024-07-26T13:46:12.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.441+0000 7f19b0cae740 10 client.26309 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:46:12.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.441+0000 7f19b0cae740 20 client.26309 populate_metadata read hostname 'smithi100' 2024-07-26T13:46:12.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.441+0000 7f19b0cae740 1 --2- 172.21.15.100:0/2710346559 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55df8a335080 0x55df8a7d7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:12.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.441+0000 7f19b0cae740 4 client.26309 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-26T13:46:12.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.441+0000 7f19b0cae740 1 -- 172.21.15.100:0/2710346559 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55df8a751d50 con 0x55df8a335080 2024-07-26T13:46:12.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.441+0000 7f19a3fff640 1 --2- 172.21.15.100:0/2710346559 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55df8a335080 0x55df8a7d7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:12.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.441+0000 7f19837fe640 10 client.26309 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:12.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.442+0000 7f19a3fff640 1 --2- 172.21.15.100:0/2710346559 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55df8a335080 0x55df8a7d7f10 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6327b9a3fd7c435c server_cookie=25cb11135670528e in_seq=0 out_seq=0 2024-07-26T13:46:12.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.443+0000 7f19837fe640 1 -- 172.21.15.100:0/2710346559 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55df8a751d50 con 0x55df8a335080 2024-07-26T13:46:12.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.443+0000 7f19837fe640 10 client.26309 handle_command_reply: tid=0 2024-07-26T13:46:12.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.443+0000 7f19b0cae740 1 client.26309 shutdown 2024-07-26T13:46:12.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.443+0000 7f19b0cae740 1 -- 172.21.15.100:0/2710346559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df8a380670 msgr2=0x55df8a76b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:12.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.443+0000 7f19b0cae740 1 --2- 172.21.15.100:0/2710346559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df8a380670 0x55df8a76b8a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f19980aae70 tx=0x7f199800c7f0 comp rx=0 tx=0).stop 2024-07-26T13:46:12.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.443+0000 7f19b0cae740 1 -- 172.21.15.100:0/2710346559 shutdown_connections 2024-07-26T13:46:12.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.443+0000 7f19b0cae740 1 --2- 172.21.15.100:0/2710346559 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55df8a335080 0x55df8a7d7f10 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:12.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.443+0000 7f19b0cae740 1 --2- 172.21.15.100:0/2710346559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df8a3a1350 0x55df8a766110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:12.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.443+0000 7f19837fe640 0 client.26309 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:12.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.443+0000 7f19b0cae740 1 --2- 172.21.15.100:0/2710346559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df8a380670 0x55df8a76b8a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:12.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.443+0000 7f19b0cae740 1 --2- 172.21.15.100:0/2710346559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55df8a378490 0x55df8a76b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:12.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.443+0000 7f19b0cae740 1 -- 172.21.15.100:0/2710346559 >> 172.21.15.100:0/2710346559 conn(0x55df8a395740 msgr2=0x55df8a74bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:12.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.443+0000 7f19b0cae740 1 -- 172.21.15.100:0/2710346559 shutdown_connections 2024-07-26T13:46:12.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.443+0000 7f19b0cae740 1 -- 172.21.15.100:0/2710346559 wait complete. 2024-07-26T13:46:12.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.443+0000 7f19b0cae740 20 client.26309 trim_cache size 0 max 16384 2024-07-26T13:46:12.464 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:46:12.464 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-07-26T13:46:12.464 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "d6a5ba31-c00b-4228-bf3e-05684f27e70e", 2024-07-26T13:46:12.464 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-07-26T13:46:12.464 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:46:12.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.444+0000 7f19aa72a640 1 -- 172.21.15.100:0/1149788399 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f198c0b3d70 msgr2=0x7f198c0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:12.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.444+0000 7f19aa72a640 1 --2- 172.21.15.100:0/1149788399 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f198c0b3d70 0x7f198c0b6280 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f199c02ade0 tx=0x7f199c036000 comp rx=0 tx=0).stop 2024-07-26T13:46:12.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.444+0000 7f19aa72a640 1 -- 172.21.15.100:0/1149788399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19a4107e10 msgr2=0x7f19a419b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:12.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.444+0000 7f19aa72a640 1 --2- 172.21.15.100:0/1149788399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19a4107e10 0x7f19a419b1d0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f199800e9f0 tx=0x7f199800eeb0 comp rx=0 tx=0).stop 2024-07-26T13:46:12.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.444+0000 7f19aa72a640 1 -- 172.21.15.100:0/1149788399 shutdown_connections 2024-07-26T13:46:12.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.444+0000 7f19aa72a640 1 --2- 172.21.15.100:0/1149788399 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f198c0b3d70 0x7f198c0b6280 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:12.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.444+0000 7f19aa72a640 1 --2- 172.21.15.100:0/1149788399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19a4108760 0x7f19a419b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:12.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.444+0000 7f19aa72a640 1 --2- 172.21.15.100:0/1149788399 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f19a4107e10 0x7f19a419b1d0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:12.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.444+0000 7f19aa72a640 1 --2- 172.21.15.100:0/1149788399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19a41024d0 0x7f19a419ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:12.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.444+0000 7f19aa72a640 1 -- 172.21.15.100:0/1149788399 >> 172.21.15.100:0/1149788399 conn(0x7f19a4079210 msgr2=0x7f19a4104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:12.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.444+0000 7f19aa72a640 1 -- 172.21.15.100:0/1149788399 shutdown_connections 2024-07-26T13:46:12.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:12.444+0000 7f19aa72a640 1 -- 172.21.15.100:0/1149788399 wait complete. 2024-07-26T13:46:13.191 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-26T13:46:13.420 INFO:tasks.workunit.client.0.smithi100.stdout: 1 230964 3.12 MB/sec execute 147 sec latency 17.575 ms 2024-07-26T13:46:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:13 smithi100 ceph-mon[32599]: scrub queued for path: / 2024-07-26T13:46:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:13 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:46:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:13 smithi100 ceph-mon[32599]: scrub summary: active paths [/] 2024-07-26T13:46:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:13 smithi100 ceph-mon[32599]: scrub summary: active paths [/] 2024-07-26T13:46:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:13 smithi100 ceph-mon[32599]: scrub summary: active paths [/] 2024-07-26T13:46:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:13 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:46:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:13 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:46:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:13 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:46:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:13 smithi100 ceph-mon[32599]: pgmap v550: 97 pgs: 97 active+clean; 87 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.1 MiB/s wr, 97 op/s 2024-07-26T13:46:13.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:13 smithi138 ceph-mon[36954]: scrub queued for path: / 2024-07-26T13:46:13.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:13 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:46:13.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:13 smithi138 ceph-mon[36954]: scrub summary: active paths [/] 2024-07-26T13:46:13.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:13 smithi138 ceph-mon[36954]: scrub summary: active paths [/] 2024-07-26T13:46:13.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:13 smithi138 ceph-mon[36954]: scrub summary: active paths [/] 2024-07-26T13:46:13.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:13 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:46:13.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:13 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:46:13.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:13 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:46:13.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:13 smithi138 ceph-mon[36954]: pgmap v550: 97 pgs: 97 active+clean; 87 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.1 MiB/s wr, 97 op/s 2024-07-26T13:46:14.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:13 smithi167 ceph-mon[37140]: scrub queued for path: / 2024-07-26T13:46:14.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:13 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:46:14.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:13 smithi167 ceph-mon[37140]: scrub summary: active paths [/] 2024-07-26T13:46:14.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:13 smithi167 ceph-mon[37140]: scrub summary: active paths [/] 2024-07-26T13:46:14.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:13 smithi167 ceph-mon[37140]: scrub summary: active paths [/] 2024-07-26T13:46:14.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:13 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:46:14.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:13 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:46:14.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:13 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:46:14.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:13 smithi167 ceph-mon[37140]: pgmap v550: 97 pgs: 97 active+clean; 87 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.1 MiB/s wr, 97 op/s 2024-07-26T13:46:14.420 INFO:tasks.workunit.client.0.smithi100.stdout: 1 232746 3.14 MB/sec execute 148 sec latency 22.517 ms 2024-07-26T13:46:14.445 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:46:15.421 INFO:tasks.workunit.client.0.smithi100.stdout: 1 234619 3.21 MB/sec execute 149 sec latency 11.457 ms 2024-07-26T13:46:15.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.552+0000 7faedcf10640 1 -- 172.21.15.100:0/4156835520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faed8104bb0 msgr2=0x7faed8104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:15.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.552+0000 7faedcf10640 1 --2- 172.21.15.100:0/4156835520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faed8104bb0 0x7faed8104fc0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7faec0009900 tx=0x7faec002a410 comp rx=0 tx=0).stop 2024-07-26T13:46:15.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.552+0000 7faedcf10640 1 -- 172.21.15.100:0/4156835520 shutdown_connections 2024-07-26T13:46:15.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.552+0000 7faedcf10640 1 --2- 172.21.15.100:0/4156835520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faed8105500 0x7faed810ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:15.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.552+0000 7faedcf10640 1 --2- 172.21.15.100:0/4156835520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faed8104bb0 0x7faed8104fc0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:15.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.552+0000 7faedcf10640 1 --2- 172.21.15.100:0/4156835520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faed8102ba0 0x7faed8102fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:15.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.552+0000 7faedcf10640 1 -- 172.21.15.100:0/4156835520 >> 172.21.15.100:0/4156835520 conn(0x7faed8079980 msgr2=0x7faed8079d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:15.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.552+0000 7faedcf10640 1 -- 172.21.15.100:0/4156835520 shutdown_connections 2024-07-26T13:46:15.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.552+0000 7faedcf10640 1 -- 172.21.15.100:0/4156835520 wait complete. 2024-07-26T13:46:15.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.553+0000 7faedcf10640 1 Processor -- start 2024-07-26T13:46:15.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.553+0000 7faedcf10640 1 -- start start 2024-07-26T13:46:15.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.553+0000 7faedcf10640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faed8102ba0 0x7faed8199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:15.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.553+0000 7faedcf10640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faed8104bb0 0x7faed819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:15.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.553+0000 7faedcf10640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faed8105500 0x7faed819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:15.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.553+0000 7faedcf10640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7faed8112400 con 0x7faed8102ba0 2024-07-26T13:46:15.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.553+0000 7faedcf10640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faed8112280 con 0x7faed8104bb0 2024-07-26T13:46:15.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.553+0000 7faedcf10640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faed8112580 con 0x7faed8105500 2024-07-26T13:46:15.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.554+0000 7faed659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faed8102ba0 0x7faed8199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:15.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.554+0000 7faed6d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faed8105500 0x7faed819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:15.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.554+0000 7faed659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faed8102ba0 0x7faed8199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52090/0 (socket says 172.21.15.100:52090) 2024-07-26T13:46:15.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.554+0000 7faed659c640 1 -- 172.21.15.100:0/2954227415 learned_addr learned my addr 172.21.15.100:0/2954227415 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:15.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.554+0000 7faed5d9b640 1 --2- 172.21.15.100:0/2954227415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faed8104bb0 0x7faed819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:15.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.554+0000 7faed6d9d640 1 -- 172.21.15.100:0/2954227415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faed8104bb0 msgr2=0x7faed819a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:15.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.554+0000 7faed6d9d640 1 --2- 172.21.15.100:0/2954227415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faed8104bb0 0x7faed819a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:15.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.554+0000 7faed6d9d640 1 -- 172.21.15.100:0/2954227415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faed8102ba0 msgr2=0x7faed8199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:15.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.554+0000 7faed6d9d640 1 --2- 172.21.15.100:0/2954227415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faed8102ba0 0x7faed8199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:15.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.554+0000 7faed6d9d640 1 -- 172.21.15.100:0/2954227415 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed819fed0 con 0x7faed8105500 2024-07-26T13:46:15.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.554+0000 7faed6d9d640 1 --2- 172.21.15.100:0/2954227415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faed8105500 0x7faed819f7a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7faed000ec50 tx=0x7faed000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:15.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.554+0000 7faecf7fe640 1 -- 172.21.15.100:0/2954227415 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed0003b70 con 0x7faed8105500 2024-07-26T13:46:15.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.554+0000 7faedcf10640 1 -- 172.21.15.100:0/2954227415 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faed81a01c0 con 0x7faed8105500 2024-07-26T13:46:15.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.554+0000 7faecf7fe640 1 -- 172.21.15.100:0/2954227415 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faed0003d10 con 0x7faed8105500 2024-07-26T13:46:15.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.554+0000 7faecf7fe640 1 -- 172.21.15.100:0/2954227415 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed0010830 con 0x7faed8105500 2024-07-26T13:46:15.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.554+0000 7faedcf10640 1 -- 172.21.15.100:0/2954227415 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faed81a6920 con 0x7faed8105500 2024-07-26T13:46:15.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.557+0000 7faee3494740 1 Processor -- start 2024-07-26T13:46:15.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.557+0000 7faecf7fe640 1 -- 172.21.15.100:0/2954227415 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7faed0003fa0 con 0x7faed8105500 2024-07-26T13:46:15.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.557+0000 7faee3494740 1 -- start start 2024-07-26T13:46:15.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.558+0000 7faee3494740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a70c92850 0x557a71038510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:15.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.558+0000 7faee3494740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a70c69990 0x557a71038a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:15.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.558+0000 7faee3494740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a70c71b70 0x557a71038f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:15.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.558+0000 7faee3494740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x557a70fad690 con 0x557a70c71b70 2024-07-26T13:46:15.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.558+0000 7faee3494740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557a70f9cb90 con 0x557a70c92850 2024-07-26T13:46:15.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.558+0000 7faee3494740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557a70fa0bd0 con 0x557a70c69990 2024-07-26T13:46:15.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.558+0000 7faed5d9b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a70c92850 0x557a71038510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:15.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.558+0000 7faed5d9b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a70c92850 0x557a71038510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:54812/0 (socket says 172.21.15.100:54812) 2024-07-26T13:46:15.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.558+0000 7faed659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a70c69990 0x557a71038a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:15.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.558+0000 7faed5d9b640 1 -- 172.21.15.100:0/1807320612 learned_addr learned my addr 172.21.15.100:0/1807320612 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:15.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.558+0000 7faed5d9b640 1 --2- 172.21.15.100:0/1807320612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a70c71b70 0x557a71038f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:15.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.558+0000 7faecf7fe640 1 --2- 172.21.15.100:0/2954227415 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7faec40b3d40 0x7faec40b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:15.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.558+0000 7faed659c640 1 -- 172.21.15.100:0/1807320612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a70c92850 msgr2=0x557a71038510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:15.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.558+0000 7faed659c640 1 --2- 172.21.15.100:0/1807320612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a70c92850 0x557a71038510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:15.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.558+0000 7faed659c640 1 -- 172.21.15.100:0/1807320612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a70c71b70 msgr2=0x557a71038f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:15.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.558+0000 7faed659c640 1 --2- 172.21.15.100:0/1807320612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a70c71b70 0x557a71038f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:15.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.558+0000 7faed659c640 1 -- 172.21.15.100:0/1807320612 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a70f28a10 con 0x557a70c69990 2024-07-26T13:46:15.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.558+0000 7faed6d9d640 1 --2- 172.21.15.100:0/2954227415 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7faec40b3d40 0x7faec40b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:15.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.559+0000 7faecf7fe640 1 -- 172.21.15.100:0/2954227415 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7faed00e45f0 con 0x7faed8105500 2024-07-26T13:46:15.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.559+0000 7faed659c640 1 --2- 172.21.15.100:0/1807320612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a70c69990 0x557a71038a50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7faec80045c0 tx=0x7faec800bc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a23d43513f2a9c05 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:15.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.559+0000 7faeccff9640 1 -- 172.21.15.100:0/1807320612 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec800be60 con 0x557a70c69990 2024-07-26T13:46:15.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.559+0000 7faed6d9d640 1 --2- 172.21.15.100:0/2954227415 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7faec40b3d40 0x7faec40b6250 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7faed00a7140 tx=0x7faed0078810 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:15.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.559+0000 7faeccff9640 1 -- 172.21.15.100:0/1807320612 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faec8015070 con 0x557a70c69990 2024-07-26T13:46:15.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.559+0000 7faeccff9640 1 -- 172.21.15.100:0/1807320612 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec8004ca0 con 0x557a70c69990 2024-07-26T13:46:15.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.560+0000 7faee3494740 1 -- 172.21.15.100:0/1807320612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a70c69990 msgr2=0x557a71038a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:15.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.560+0000 7faee3494740 1 --2- 172.21.15.100:0/1807320612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a70c69990 0x557a71038a50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7faec80045c0 tx=0x7faec800bc70 comp rx=0 tx=0).stop 2024-07-26T13:46:15.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.560+0000 7faee3494740 1 -- 172.21.15.100:0/1807320612 shutdown_connections 2024-07-26T13:46:15.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.560+0000 7faee3494740 1 --2- 172.21.15.100:0/1807320612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a70c71b70 0x557a71038f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:15.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.560+0000 7faee3494740 1 --2- 172.21.15.100:0/1807320612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a70c69990 0x557a71038a50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:15.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.560+0000 7faee3494740 1 --2- 172.21.15.100:0/1807320612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a70c92850 0x557a71038510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:15.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.560+0000 7faee3494740 1 -- 172.21.15.100:0/1807320612 >> 172.21.15.100:0/1807320612 conn(0x557a70c86c40 msgr2=0x557a71036400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:15.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.560+0000 7faee3494740 1 -- 172.21.15.100:0/1807320612 shutdown_connections 2024-07-26T13:46:15.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.560+0000 7faee3494740 1 -- 172.21.15.100:0/1807320612 wait complete. 2024-07-26T13:46:15.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.561+0000 7faee3494740 1 -- start start 2024-07-26T13:46:15.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.561+0000 7faee3494740 1 Processor -- start 2024-07-26T13:46:15.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.561+0000 7faee3494740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a70c69990 0x557a71048810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:15.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.561+0000 7faee3494740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a70c71b70 0x557a71048df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:15.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.561+0000 7faee3494740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a70c92850 0x557a7104de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:15.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.561+0000 7faee3494740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x557a70e007b0 con 0x557a70c92850 2024-07-26T13:46:15.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.561+0000 7faee3494740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557a70f3f210 con 0x557a70c69990 2024-07-26T13:46:15.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.561+0000 7faee3494740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557a71046c60 con 0x557a70c71b70 2024-07-26T13:46:15.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.561+0000 7faee3494740 10 client.0 _ll_register_callbacks cb 0x557a70f202c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:46:15.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.561+0000 7faed659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a70c71b70 0x557a71048df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:15.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.561+0000 7faed5d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a70c92850 0x557a7104de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:15.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.561+0000 7faed5d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a70c92850 0x557a7104de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52108/0 (socket says 172.21.15.100:52108) 2024-07-26T13:46:15.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.561+0000 7faed659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a70c71b70 0x557a71048df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:60416/0 (socket says 172.21.15.100:60416) 2024-07-26T13:46:15.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.561+0000 7faed5d9b640 1 -- 172.21.15.100:0/3879112899 learned_addr learned my addr 172.21.15.100:0/3879112899 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:15.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.561+0000 7faed5d9b640 1 --2- 172.21.15.100:0/3879112899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a70c69990 0x557a71048810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:15.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.562+0000 7faed5d9b640 1 -- 172.21.15.100:0/3879112899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a70c71b70 msgr2=0x557a71048df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:15.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.562+0000 7faed5d9b640 1 --2- 172.21.15.100:0/3879112899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a70c71b70 0x557a71048df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:15.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.562+0000 7faed5d9b640 1 -- 172.21.15.100:0/3879112899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a70c69990 msgr2=0x557a71048810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:15.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.562+0000 7faed5d9b640 1 --2- 172.21.15.100:0/3879112899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a70c69990 0x557a71048810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:15.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.562+0000 7faed5d9b640 1 -- 172.21.15.100:0/3879112899 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a70f50710 con 0x557a70c92850 2024-07-26T13:46:15.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.562+0000 7faed659c640 1 --2- 172.21.15.100:0/3879112899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a70c71b70 0x557a71048df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:46:15.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.562+0000 7faed5d9b640 1 --2- 172.21.15.100:0/3879112899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a70c92850 0x557a7104de10 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7faec002aa00 tx=0x7faec0043040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42596dfd71238cc9 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:15.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.562+0000 7faebdffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:46:15.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.562+0000 7faebdffb640 1 -- 172.21.15.100:0/3879112899 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec0035070 con 0x557a70c92850 2024-07-26T13:46:15.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.562+0000 7faebdffb640 1 -- 172.21.15.100:0/3879112899 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7faec0003bd0 con 0x557a70c92850 2024-07-26T13:46:15.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.562+0000 7faee3494740 1 -- 172.21.15.100:0/3879112899 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557a70faf5e0 con 0x557a70c92850 2024-07-26T13:46:15.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.562+0000 7faebdffb640 1 -- 172.21.15.100:0/3879112899 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7faec0053050 con 0x557a70c92850 2024-07-26T13:46:15.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.562+0000 7faee3494740 10 client.17082 fetch_fsmap learned FSMap version 53 2024-07-26T13:46:15.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.562+0000 7faee3494740 1 -- 172.21.15.100:0/3879112899 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x557a70d0cba0 con 0x557a70c92850 2024-07-26T13:46:15.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.562+0000 7faebdffb640 1 -- 172.21.15.100:0/3879112899 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557a70faf5e0 con 0x557a70c92850 2024-07-26T13:46:15.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.563+0000 7faebdffb640 1 -- 172.21.15.100:0/3879112899 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7faec003d450 con 0x557a70c92850 2024-07-26T13:46:15.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.563+0000 7faee3494740 10 client.17082 fetch_fsmap finished waiting for FSMap version 53 2024-07-26T13:46:15.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.563+0000 7faee3494740 10 client.17082 resolve_mds: validated gid 24607 aka daemon mds.i 2024-07-26T13:46:15.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.563+0000 7faee3494740 20 client.17082 populate_metadata read hostname 'smithi100' 2024-07-26T13:46:15.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.563+0000 7faee3494740 1 --2- 172.21.15.100:0/3879112899 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x557a70c511d0 0x557a71059fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:15.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.563+0000 7faee3494740 4 client.17082 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:46:15.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.563+0000 7faee3494740 1 -- 172.21.15.100:0/3879112899 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a70fab080 con 0x557a70c511d0 2024-07-26T13:46:15.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.563+0000 7faed659c640 1 --2- 172.21.15.100:0/3879112899 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x557a70c511d0 0x557a71059fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:15.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.563+0000 7faed659c640 1 --2- 172.21.15.100:0/3879112899 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x557a70c511d0 0x557a71059fd0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be936d1707735ff8 server_cookie=8a75721a1f58a315 in_seq=0 out_seq=0 2024-07-26T13:46:15.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.563+0000 7faebdffb640 10 client.17082 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:15.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.564+0000 7faebdffb640 1 -- 172.21.15.100:0/3879112899 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557a70fab080 con 0x557a70c511d0 2024-07-26T13:46:15.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.564+0000 7faebdffb640 10 client.17082 handle_command_reply: tid=0 2024-07-26T13:46:15.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.564+0000 7faee3494740 1 client.17082 shutdown 2024-07-26T13:46:15.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.564+0000 7faee3494740 1 -- 172.21.15.100:0/3879112899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a70c92850 msgr2=0x557a7104de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:15.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.564+0000 7faee3494740 1 --2- 172.21.15.100:0/3879112899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a70c92850 0x557a7104de10 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7faec002aa00 tx=0x7faec0043040 comp rx=0 tx=0).stop 2024-07-26T13:46:15.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.564+0000 7faee3494740 1 -- 172.21.15.100:0/3879112899 shutdown_connections 2024-07-26T13:46:15.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.564+0000 7faee3494740 1 --2- 172.21.15.100:0/3879112899 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x557a70c511d0 0x557a71059fd0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:15.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.564+0000 7faebdffb640 0 client.17082 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:15.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.564+0000 7faee3494740 1 --2- 172.21.15.100:0/3879112899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a70c92850 0x557a7104de10 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:15.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.564+0000 7faee3494740 1 --2- 172.21.15.100:0/3879112899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a70c71b70 0x557a71048df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:15.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.564+0000 7faee3494740 1 --2- 172.21.15.100:0/3879112899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a70c69990 0x557a71048810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:15.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.564+0000 7faee3494740 1 -- 172.21.15.100:0/3879112899 >> 172.21.15.100:0/3879112899 conn(0x557a70c86c40 msgr2=0x557a7103b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:15.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.565+0000 7faee3494740 1 -- 172.21.15.100:0/3879112899 shutdown_connections 2024-07-26T13:46:15.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.565+0000 7faee3494740 1 -- 172.21.15.100:0/3879112899 wait complete. 2024-07-26T13:46:15.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.565+0000 7faee3494740 20 client.17082 trim_cache size 0 max 16384 2024-07-26T13:46:15.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.585+0000 7faee3494740 1 Processor -- start 2024-07-26T13:46:15.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.585+0000 7faee3494740 1 -- start start 2024-07-26T13:46:15.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.585+0000 7faee3494740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a70c511d0 0x557a7104c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:15.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.585+0000 7faee3494740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a70c69990 0x557a7104d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:15.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.585+0000 7faee3494740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a70c71b70 0x557a7104abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:15.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.585+0000 7faee3494740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7faed81afa50 con 0x557a70c69990 2024-07-26T13:46:15.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.585+0000 7faee3494740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faed81ad320 con 0x557a70c71b70 2024-07-26T13:46:15.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.585+0000 7faee3494740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faed81ab170 con 0x557a70c511d0 2024-07-26T13:46:15.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.586+0000 7faed659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a70c69990 0x557a7104d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:15.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.586+0000 7faed659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a70c69990 0x557a7104d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52116/0 (socket says 172.21.15.100:52116) 2024-07-26T13:46:15.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.586+0000 7faed659c640 1 -- 172.21.15.100:0/695250331 learned_addr learned my addr 172.21.15.100:0/695250331 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:15.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.586+0000 7faed5d9b640 1 --2- 172.21.15.100:0/695250331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a70c511d0 0x557a7104c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:15.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.586+0000 7faed5d9b640 1 --2- 172.21.15.100:0/695250331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a70c71b70 0x557a7104abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:15.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.586+0000 7faed659c640 1 -- 172.21.15.100:0/695250331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a70c511d0 msgr2=0x557a7104c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:15.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.586+0000 7faed659c640 1 --2- 172.21.15.100:0/695250331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a70c511d0 0x557a7104c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:15.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.586+0000 7faed659c640 1 -- 172.21.15.100:0/695250331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a70c71b70 msgr2=0x557a7104abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:15.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.586+0000 7faed659c640 1 --2- 172.21.15.100:0/695250331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a70c71b70 0x557a7104abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:15.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.586+0000 7faed659c640 1 -- 172.21.15.100:0/695250331 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a710392f0 con 0x557a70c69990 2024-07-26T13:46:15.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.586+0000 7faed659c640 1 --2- 172.21.15.100:0/695250331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a70c69990 0x557a7104d380 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7faec80039c0 tx=0x7faec8004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d15c8b50e0351a4 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:15.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.586+0000 7faebffff640 1 -- 172.21.15.100:0/695250331 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec8003e50 con 0x557a70c69990 2024-07-26T13:46:15.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.586+0000 7faebffff640 1 -- 172.21.15.100:0/695250331 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faec8015040 con 0x557a70c69990 2024-07-26T13:46:15.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.586+0000 7faebffff640 1 -- 172.21.15.100:0/695250331 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec802fe60 con 0x557a70c69990 2024-07-26T13:46:15.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.587+0000 7faee3494740 1 -- 172.21.15.100:0/695250331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a70c69990 msgr2=0x557a7104d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:15.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.587+0000 7faee3494740 1 --2- 172.21.15.100:0/695250331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a70c69990 0x557a7104d380 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7faec80039c0 tx=0x7faec8004920 comp rx=0 tx=0).stop 2024-07-26T13:46:15.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.587+0000 7faee3494740 1 -- 172.21.15.100:0/695250331 shutdown_connections 2024-07-26T13:46:15.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.587+0000 7faee3494740 1 --2- 172.21.15.100:0/695250331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a70c71b70 0x557a7104abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:15.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.587+0000 7faee3494740 1 --2- 172.21.15.100:0/695250331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a70c69990 0x557a7104d380 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:15.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.587+0000 7faee3494740 1 --2- 172.21.15.100:0/695250331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a70c511d0 0x557a7104c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:15.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.587+0000 7faee3494740 1 -- 172.21.15.100:0/695250331 >> 172.21.15.100:0/695250331 conn(0x557a70c86c40 msgr2=0x557a7104cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:15.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.587+0000 7faee3494740 1 -- 172.21.15.100:0/695250331 shutdown_connections 2024-07-26T13:46:15.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.587+0000 7faee3494740 1 -- 172.21.15.100:0/695250331 wait complete. 2024-07-26T13:46:15.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.588+0000 7faee3494740 1 -- start start 2024-07-26T13:46:15.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.588+0000 7faee3494740 1 Processor -- start 2024-07-26T13:46:15.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.588+0000 7faee3494740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a70c511d0 0x557a710545e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:15.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.588+0000 7faee3494740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a70c69990 0x557a71054b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:15.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.588+0000 7faee3494740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a70c71b70 0x557a71047050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:15.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.588+0000 7faee3494740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7faed81ab7e0 con 0x557a70c69990 2024-07-26T13:46:15.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.588+0000 7faee3494740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faed81acc50 con 0x557a70c71b70 2024-07-26T13:46:15.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.588+0000 7faee3494740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faed81abeb0 con 0x557a70c511d0 2024-07-26T13:46:15.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.588+0000 7faee3494740 10 client.0 _ll_register_callbacks cb 0x7faec0009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:46:15.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.588+0000 7faed659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a70c69990 0x557a71054b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:15.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.588+0000 7faed659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a70c69990 0x557a71054b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52130/0 (socket says 172.21.15.100:52130) 2024-07-26T13:46:15.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.588+0000 7faed659c640 1 -- 172.21.15.100:0/2343402365 learned_addr learned my addr 172.21.15.100:0/2343402365 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:15.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.589+0000 7faed5d9b640 1 --2- 172.21.15.100:0/2343402365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a70c511d0 0x557a710545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:15.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.589+0000 7faed5d9b640 1 --2- 172.21.15.100:0/2343402365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a70c71b70 0x557a71047050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:15.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.589+0000 7faed659c640 1 -- 172.21.15.100:0/2343402365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a70c511d0 msgr2=0x557a710545e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:15.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.589+0000 7faed659c640 1 --2- 172.21.15.100:0/2343402365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a70c511d0 0x557a710545e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:15.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.589+0000 7faed659c640 1 -- 172.21.15.100:0/2343402365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a70c71b70 msgr2=0x557a71047050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:15.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.589+0000 7faed659c640 1 --2- 172.21.15.100:0/2343402365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a70c71b70 0x557a71047050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:15.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.589+0000 7faed659c640 1 -- 172.21.15.100:0/2343402365 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a710477e0 con 0x557a70c69990 2024-07-26T13:46:15.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.589+0000 7faed659c640 1 --2- 172.21.15.100:0/2343402365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a70c69990 0x557a71054b80 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7faec8000c00 tx=0x7faec8003ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdcc609ff289a4b8 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:15.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.589+0000 7faebdffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:46:15.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.589+0000 7faebdffb640 1 -- 172.21.15.100:0/2343402365 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec8031e60 con 0x557a70c69990 2024-07-26T13:46:15.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.589+0000 7faebdffb640 1 -- 172.21.15.100:0/2343402365 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7faec8003e60 con 0x557a70c69990 2024-07-26T13:46:15.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.589+0000 7faee3494740 1 -- 172.21.15.100:0/2343402365 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faec8015040 con 0x557a70c69990 2024-07-26T13:46:15.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.590+0000 7faebdffb640 1 -- 172.21.15.100:0/2343402365 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7faec8040050 con 0x557a70c69990 2024-07-26T13:46:15.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.590+0000 7faee3494740 10 client.17088 fetch_fsmap learned FSMap version 53 2024-07-26T13:46:15.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.590+0000 7faee3494740 1 -- 172.21.15.100:0/2343402365 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x557a71050bb0 con 0x557a70c69990 2024-07-26T13:46:15.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.590+0000 7faebdffb640 1 -- 172.21.15.100:0/2343402365 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faec8015040 con 0x557a70c69990 2024-07-26T13:46:15.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.590+0000 7faebdffb640 1 -- 172.21.15.100:0/2343402365 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7faec80117e0 con 0x557a70c69990 2024-07-26T13:46:15.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.590+0000 7faee3494740 10 client.17088 fetch_fsmap finished waiting for FSMap version 53 2024-07-26T13:46:15.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.590+0000 7faee3494740 10 client.17088 resolve_mds: validated gid 24607 aka daemon mds.i 2024-07-26T13:46:15.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.590+0000 7faee3494740 20 client.17088 populate_metadata read hostname 'smithi100' 2024-07-26T13:46:15.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.590+0000 7faee3494740 1 --2- 172.21.15.100:0/2343402365 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x557a70c92850 0x557a710c47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:15.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.590+0000 7faee3494740 4 client.17088 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.95, "await_for": 45.0}] 2024-07-26T13:46:15.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.590+0000 7faee3494740 1 -- 172.21.15.100:0/2343402365 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.95, "await_for": 45.0}) -- 0x557a70fa87c0 con 0x557a70c92850 2024-07-26T13:46:15.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.590+0000 7faed5d9b640 1 --2- 172.21.15.100:0/2343402365 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x557a70c92850 0x557a710c47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:15.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.591+0000 7faebdffb640 10 client.17088 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:15.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.591+0000 7faed5d9b640 1 --2- 172.21.15.100:0/2343402365 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x557a70c92850 0x557a710c47e0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79eb78f93cfa69ed server_cookie=11eed179643a0dab in_seq=0 out_seq=0 2024-07-26T13:46:15.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.644+0000 7faebdffb640 1 -- 172.21.15.100:0/2343402365 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557a70fa87c0 con 0x557a70c92850 2024-07-26T13:46:15.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.644+0000 7faebdffb640 10 client.17088 handle_command_reply: tid=0 2024-07-26T13:46:15.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.644+0000 7faee3494740 1 client.17088 shutdown 2024-07-26T13:46:15.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.644+0000 7faee3494740 1 -- 172.21.15.100:0/2343402365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a70c69990 msgr2=0x557a71054b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:15.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.644+0000 7faee3494740 1 --2- 172.21.15.100:0/2343402365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a70c69990 0x557a71054b80 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7faec8000c00 tx=0x7faec8003ae0 comp rx=0 tx=0).stop 2024-07-26T13:46:15.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.644+0000 7faee3494740 1 -- 172.21.15.100:0/2343402365 shutdown_connections 2024-07-26T13:46:15.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.644+0000 7faee3494740 1 --2- 172.21.15.100:0/2343402365 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x557a70c92850 0x557a710c47e0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:15.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.644+0000 7faee3494740 1 --2- 172.21.15.100:0/2343402365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557a70c71b70 0x557a71047050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:15.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.644+0000 7faee3494740 1 --2- 172.21.15.100:0/2343402365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557a70c69990 0x557a71054b80 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:15.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.644+0000 7faee3494740 1 --2- 172.21.15.100:0/2343402365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557a70c511d0 0x557a710545e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:15.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.644+0000 7faee3494740 1 -- 172.21.15.100:0/2343402365 >> 172.21.15.100:0/2343402365 conn(0x557a70c86c40 msgr2=0x557a7104c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:15.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.644+0000 7faebdffb640 0 client.17088 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:15.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.644+0000 7faee3494740 1 -- 172.21.15.100:0/2343402365 shutdown_connections 2024-07-26T13:46:15.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.644+0000 7faee3494740 1 -- 172.21.15.100:0/2343402365 wait complete. 2024-07-26T13:46:15.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.644+0000 7faee3494740 20 client.17088 trim_cache size 0 max 16384 2024-07-26T13:46:15.648 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:46:15.648 INFO:teuthology.orchestra.run.smithi100.stdout: "epoch": 53, 2024-07-26T13:46:15.648 INFO:teuthology.orchestra.run.smithi100.stdout: "leader": 24607, 2024-07-26T13:46:15.648 INFO:teuthology.orchestra.run.smithi100.stdout: "set_version": 20, 2024-07-26T13:46:15.648 INFO:teuthology.orchestra.run.smithi100.stdout: "sets": { 2024-07-26T13:46:15.648 INFO:teuthology.orchestra.run.smithi100.stdout: "cd8d97d4": { 2024-07-26T13:46:15.648 INFO:teuthology.orchestra.run.smithi100.stdout: "version": 20, 2024-07-26T13:46:15.649 INFO:teuthology.orchestra.run.smithi100.stdout: "age_ref": 0.0, 2024-07-26T13:46:15.649 INFO:teuthology.orchestra.run.smithi100.stdout: "state": { 2024-07-26T13:46:15.649 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "QUIESCED", 2024-07-26T13:46:15.649 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 0.0 2024-07-26T13:46:15.649 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-07-26T13:46:15.649 INFO:teuthology.orchestra.run.smithi100.stdout: "timeout": 90.0, 2024-07-26T13:46:15.649 INFO:teuthology.orchestra.run.smithi100.stdout: "expiration": 141.9, 2024-07-26T13:46:15.649 INFO:teuthology.orchestra.run.smithi100.stdout: "members": { 2024-07-26T13:46:15.649 INFO:teuthology.orchestra.run.smithi100.stdout: "file:/": { 2024-07-26T13:46:15.649 INFO:teuthology.orchestra.run.smithi100.stdout: "excluded": false, 2024-07-26T13:46:15.649 INFO:teuthology.orchestra.run.smithi100.stdout: "state": { 2024-07-26T13:46:15.649 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "QUIESCED", 2024-07-26T13:46:15.649 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 0.0 2024-07-26T13:46:15.650 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:46:15.650 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:46:15.650 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:46:15.650 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:46:15.650 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:46:15.650 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:46:15.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.645+0000 7faedcf10640 1 -- 172.21.15.100:0/2954227415 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7faec40b3d40 msgr2=0x7faec40b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:15.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.645+0000 7faedcf10640 1 --2- 172.21.15.100:0/2954227415 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7faec40b3d40 0x7faec40b6250 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7faed00a7140 tx=0x7faed0078810 comp rx=0 tx=0).stop 2024-07-26T13:46:15.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.645+0000 7faedcf10640 1 -- 172.21.15.100:0/2954227415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faed8105500 msgr2=0x7faed819f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:15.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.645+0000 7faedcf10640 1 --2- 172.21.15.100:0/2954227415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faed8105500 0x7faed819f7a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7faed000ec50 tx=0x7faed000c6a0 comp rx=0 tx=0).stop 2024-07-26T13:46:15.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.645+0000 7faedcf10640 1 -- 172.21.15.100:0/2954227415 shutdown_connections 2024-07-26T13:46:15.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.645+0000 7faedcf10640 1 --2- 172.21.15.100:0/2954227415 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7faec40b3d40 0x7faec40b6250 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:15.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.645+0000 7faedcf10640 1 --2- 172.21.15.100:0/2954227415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faed8105500 0x7faed819f7a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:15.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.645+0000 7faedcf10640 1 --2- 172.21.15.100:0/2954227415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faed8104bb0 0x7faed819a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:15.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.645+0000 7faedcf10640 1 --2- 172.21.15.100:0/2954227415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faed8102ba0 0x7faed8199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:15.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.645+0000 7faedcf10640 1 -- 172.21.15.100:0/2954227415 >> 172.21.15.100:0/2954227415 conn(0x7faed8079980 msgr2=0x7faed8105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:15.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.645+0000 7faedcf10640 1 -- 172.21.15.100:0/2954227415 shutdown_connections 2024-07-26T13:46:15.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:15.645+0000 7faedcf10640 1 -- 172.21.15.100:0/2954227415 wait complete. 2024-07-26T13:46:15.886 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:15 smithi100 ceph-mon[32599]: pgmap v551: 97 pgs: 97 active+clean; 86 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.5 MiB/s wr, 146 op/s 2024-07-26T13:46:15.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:15 smithi138 ceph-mon[36954]: pgmap v551: 97 pgs: 97 active+clean; 86 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.5 MiB/s wr, 146 op/s 2024-07-26T13:46:15.973 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:15 smithi167 ceph-mon[37140]: pgmap v551: 97 pgs: 97 active+clean; 86 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.5 MiB/s wr, 146 op/s 2024-07-26T13:46:16.420 INFO:tasks.workunit.client.0.smithi100.stdout: 1 234890 3.19 MB/sec execute 150 sec latency 818.277 ms 2024-07-26T13:46:16.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.853+0000 7f1a60a11640 1 -- 172.21.15.100:0/830110402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a5c1087a0 msgr2=0x7f1a5c10efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:16.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.853+0000 7f1a60a11640 1 --2- 172.21.15.100:0/830110402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a5c1087a0 0x7f1a5c10efd0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f1a54009ca0 tx=0x7f1a5402a8c0 comp rx=0 tx=0).stop 2024-07-26T13:46:16.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.853+0000 7f1a60a11640 1 -- 172.21.15.100:0/830110402 shutdown_connections 2024-07-26T13:46:16.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.853+0000 7f1a60a11640 1 --2- 172.21.15.100:0/830110402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a5c1087a0 0x7f1a5c10efd0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:16.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.853+0000 7f1a60a11640 1 --2- 172.21.15.100:0/830110402 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a5c107e50 0x7f1a5c108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:16.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.853+0000 7f1a60a11640 1 --2- 172.21.15.100:0/830110402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a5c102510 0x7f1a5c102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:16.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.853+0000 7f1a60a11640 1 -- 172.21.15.100:0/830110402 >> 172.21.15.100:0/830110402 conn(0x7f1a5c0791a0 msgr2=0x7f1a5c0795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:16.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.853+0000 7f1a60a11640 1 -- 172.21.15.100:0/830110402 shutdown_connections 2024-07-26T13:46:16.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.853+0000 7f1a60a11640 1 -- 172.21.15.100:0/830110402 wait complete. 2024-07-26T13:46:16.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.853+0000 7f1a60a11640 1 Processor -- start 2024-07-26T13:46:16.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.854+0000 7f1a60a11640 1 -- start start 2024-07-26T13:46:16.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.854+0000 7f1a60a11640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a5c102510 0x7f1a5c199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:16.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.854+0000 7f1a60a11640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a5c107e50 0x7f1a5c19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:16.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.854+0000 7f1a60a11640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a5c1087a0 0x7f1a5c19f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:16.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.854+0000 7f1a60a11640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5c112450 con 0x7f1a5c107e50 2024-07-26T13:46:16.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.854+0000 7f1a60a11640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5c1122d0 con 0x7f1a5c102510 2024-07-26T13:46:16.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.854+0000 7f1a60a11640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5c1125d0 con 0x7f1a5c1087a0 2024-07-26T13:46:16.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.854+0000 7f1a59d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a5c107e50 0x7f1a5c19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:16.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.854+0000 7f1a59d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a5c107e50 0x7f1a5c19a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52144/0 (socket says 172.21.15.100:52144) 2024-07-26T13:46:16.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.854+0000 7f1a59d9b640 1 -- 172.21.15.100:0/4161370987 learned_addr learned my addr 172.21.15.100:0/4161370987 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:16.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.854+0000 7f1a5ad9d640 1 --2- 172.21.15.100:0/4161370987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a5c1087a0 0x7f1a5c19f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:16.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.854+0000 7f1a5a59c640 1 --2- 172.21.15.100:0/4161370987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a5c102510 0x7f1a5c199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:16.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.854+0000 7f1a59d9b640 1 -- 172.21.15.100:0/4161370987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a5c1087a0 msgr2=0x7f1a5c19f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:16.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.854+0000 7f1a59d9b640 1 --2- 172.21.15.100:0/4161370987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a5c1087a0 0x7f1a5c19f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:16.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.854+0000 7f1a59d9b640 1 -- 172.21.15.100:0/4161370987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a5c102510 msgr2=0x7f1a5c199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:16.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.854+0000 7f1a59d9b640 1 --2- 172.21.15.100:0/4161370987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a5c102510 0x7f1a5c199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:16.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.854+0000 7f1a59d9b640 1 -- 172.21.15.100:0/4161370987 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a5c19fd60 con 0x7f1a5c107e50 2024-07-26T13:46:16.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.854+0000 7f1a59d9b640 1 --2- 172.21.15.100:0/4161370987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a5c107e50 0x7f1a5c19a450 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000c8c0 tx=0x7f1a5000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:16.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.855+0000 7f1a4b7fe640 1 -- 172.21.15.100:0/4161370987 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a50007bb0 con 0x7f1a5c107e50 2024-07-26T13:46:16.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.855+0000 7f1a4b7fe640 1 -- 172.21.15.100:0/4161370987 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a50007d50 con 0x7f1a5c107e50 2024-07-26T13:46:16.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.855+0000 7f1a60a11640 1 -- 172.21.15.100:0/4161370987 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a5c1a0050 con 0x7f1a5c107e50 2024-07-26T13:46:16.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.855+0000 7f1a4b7fe640 1 -- 172.21.15.100:0/4161370987 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a500187c0 con 0x7f1a5c107e50 2024-07-26T13:46:16.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.855+0000 7f1a60a11640 1 -- 172.21.15.100:0/4161370987 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a5c1a6870 con 0x7f1a5c107e50 2024-07-26T13:46:16.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.857+0000 7f1a66f95740 1 Processor -- start 2024-07-26T13:46:16.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.857+0000 7f1a4b7fe640 1 -- 172.21.15.100:0/4161370987 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f1a5000fa20 con 0x7f1a5c107e50 2024-07-26T13:46:16.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.858+0000 7f1a66f95740 1 -- start start 2024-07-26T13:46:16.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.858+0000 7f1a66f95740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd76ce070 0x560bd7a73550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:16.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.858+0000 7f1a66f95740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd76a51b0 0x560bd7a73a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:16.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.858+0000 7f1a66f95740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd76ad390 0x560bd7a73fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:16.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.858+0000 7f1a66f95740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x560bd79e9b90 con 0x560bd76a51b0 2024-07-26T13:46:16.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.858+0000 7f1a66f95740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560bd79d9090 con 0x560bd76ce070 2024-07-26T13:46:16.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.858+0000 7f1a66f95740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560bd79dd0d0 con 0x560bd76ad390 2024-07-26T13:46:16.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.858+0000 7f1a5a59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd76a51b0 0x560bd7a73a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:16.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.858+0000 7f1a5a59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd76a51b0 0x560bd7a73a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52150/0 (socket says 172.21.15.100:52150) 2024-07-26T13:46:16.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.858+0000 7f1a5a59c640 1 -- 172.21.15.100:0/1481401566 learned_addr learned my addr 172.21.15.100:0/1481401566 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:16.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.858+0000 7f1a5ad9d640 1 --2- 172.21.15.100:0/1481401566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd76ce070 0x560bd7a73550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:16.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.858+0000 7f1a59d9b640 1 --2- 172.21.15.100:0/1481401566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd76ad390 0x560bd7a73fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:16.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.858+0000 7f1a5a59c640 1 -- 172.21.15.100:0/1481401566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd76ad390 msgr2=0x560bd7a73fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:16.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.858+0000 7f1a5a59c640 1 --2- 172.21.15.100:0/1481401566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd76ad390 0x560bd7a73fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:16.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.858+0000 7f1a5a59c640 1 -- 172.21.15.100:0/1481401566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd76ce070 msgr2=0x560bd7a73550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:16.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.858+0000 7f1a5a59c640 1 --2- 172.21.15.100:0/1481401566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd76ce070 0x560bd7a73550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:16.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.858+0000 7f1a5a59c640 1 -- 172.21.15.100:0/1481401566 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bd797fb70 con 0x560bd76a51b0 2024-07-26T13:46:16.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.858+0000 7f1a5ad9d640 1 --2- 172.21.15.100:0/1481401566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd76ce070 0x560bd7a73550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:46:16.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.859+0000 7f1a4b7fe640 1 --2- 172.21.15.100:0/4161370987 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1a440b3d70 0x7f1a440b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:16.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.859+0000 7f1a5a59c640 1 --2- 172.21.15.100:0/1481401566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd76a51b0 0x560bd7a73a90 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c00cbf0 tx=0x7f1a4c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b23df9413c795575 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:16.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.859+0000 7f1a5ad9d640 1 --2- 172.21.15.100:0/4161370987 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1a440b3d70 0x7f1a440b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:16.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.859+0000 7f1a48ff9640 1 -- 172.21.15.100:0/1481401566 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c010630 con 0x560bd76a51b0 2024-07-26T13:46:16.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.859+0000 7f1a4b7fe640 1 -- 172.21.15.100:0/4161370987 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7f1a500e41b0 con 0x7f1a5c107e50 2024-07-26T13:46:16.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.859+0000 7f1a48ff9640 1 -- 172.21.15.100:0/1481401566 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a4c010c80 con 0x560bd76a51b0 2024-07-26T13:46:16.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.859+0000 7f1a48ff9640 1 -- 172.21.15.100:0/1481401566 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c00fb40 con 0x560bd76a51b0 2024-07-26T13:46:16.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.860+0000 7f1a66f95740 1 -- 172.21.15.100:0/1481401566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd76a51b0 msgr2=0x560bd7a73a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:16.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.860+0000 7f1a5ad9d640 1 --2- 172.21.15.100:0/4161370987 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1a440b3d70 0x7f1a440b6280 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1a540098d0 tx=0x7f1a5402f470 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:16.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.860+0000 7f1a66f95740 1 --2- 172.21.15.100:0/1481401566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd76a51b0 0x560bd7a73a90 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c00cbf0 tx=0x7f1a4c00e720 comp rx=0 tx=0).stop 2024-07-26T13:46:16.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.860+0000 7f1a66f95740 1 -- 172.21.15.100:0/1481401566 shutdown_connections 2024-07-26T13:46:16.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.860+0000 7f1a66f95740 1 --2- 172.21.15.100:0/1481401566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd76ad390 0x560bd7a73fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:16.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.860+0000 7f1a66f95740 1 --2- 172.21.15.100:0/1481401566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd76a51b0 0x560bd7a73a90 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:16.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.860+0000 7f1a66f95740 1 --2- 172.21.15.100:0/1481401566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd76ce070 0x560bd7a73550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:16.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.860+0000 7f1a66f95740 1 -- 172.21.15.100:0/1481401566 >> 172.21.15.100:0/1481401566 conn(0x560bd76c2460 msgr2=0x560bd7a71440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:16.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.860+0000 7f1a66f95740 1 -- 172.21.15.100:0/1481401566 shutdown_connections 2024-07-26T13:46:16.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.860+0000 7f1a66f95740 1 -- 172.21.15.100:0/1481401566 wait complete. 2024-07-26T13:46:16.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.860+0000 7f1a66f95740 1 -- start start 2024-07-26T13:46:16.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.861+0000 7f1a66f95740 1 Processor -- start 2024-07-26T13:46:16.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.861+0000 7f1a66f95740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd76a51b0 0x560bd7a83530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:16.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.861+0000 7f1a66f95740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd76ad390 0x560bd7a83a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:16.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.861+0000 7f1a66f95740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd76ce070 0x560bd7a8f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:16.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.861+0000 7f1a66f95740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x560bd783c700 con 0x560bd76a51b0 2024-07-26T13:46:16.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.861+0000 7f1a66f95740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560bd797b9c0 con 0x560bd76ad390 2024-07-26T13:46:16.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.861+0000 7f1a66f95740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560bd7a86090 con 0x560bd76ce070 2024-07-26T13:46:16.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.861+0000 7f1a66f95740 10 client.0 _ll_register_callbacks cb 0x560bd79964d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:46:16.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.861+0000 7f1a5a59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd76a51b0 0x560bd7a83530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:16.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.861+0000 7f1a5a59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd76a51b0 0x560bd7a83530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52152/0 (socket says 172.21.15.100:52152) 2024-07-26T13:46:16.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.861+0000 7f1a5a59c640 1 -- 172.21.15.100:0/2145727587 learned_addr learned my addr 172.21.15.100:0/2145727587 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:16.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.861+0000 7f1a59d9b640 1 --2- 172.21.15.100:0/2145727587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd76ad390 0x560bd7a83a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:16.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.861+0000 7f1a5ad9d640 1 --2- 172.21.15.100:0/2145727587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd76ce070 0x560bd7a8f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:16.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.861+0000 7f1a5a59c640 1 -- 172.21.15.100:0/2145727587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd76ce070 msgr2=0x560bd7a8f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:16.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.861+0000 7f1a5a59c640 1 --2- 172.21.15.100:0/2145727587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd76ce070 0x560bd7a8f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:16.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.861+0000 7f1a5a59c640 1 -- 172.21.15.100:0/2145727587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd76ad390 msgr2=0x560bd7a83a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:16.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.861+0000 7f1a5a59c640 1 --2- 172.21.15.100:0/2145727587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd76ad390 0x560bd7a83a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:16.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.861+0000 7f1a5a59c640 1 -- 172.21.15.100:0/2145727587 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bd7739650 con 0x560bd76a51b0 2024-07-26T13:46:16.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.862+0000 7f1a5a59c640 1 --2- 172.21.15.100:0/2145727587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd76a51b0 0x560bd7a83530 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c0056d0 tx=0x7f1a4c005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f58075e1b39ebcf server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:16.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.862+0000 7f1a41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:46:16.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.862+0000 7f1a41ffb640 1 -- 172.21.15.100:0/2145727587 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c029060 con 0x560bd76a51b0 2024-07-26T13:46:16.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.862+0000 7f1a41ffb640 1 -- 172.21.15.100:0/2145727587 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7f1a4c00f880 con 0x560bd76a51b0 2024-07-26T13:46:16.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.862+0000 7f1a66f95740 1 -- 172.21.15.100:0/2145727587 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a4c010c80 con 0x560bd76a51b0 2024-07-26T13:46:16.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.862+0000 7f1a41ffb640 1 -- 172.21.15.100:0/2145727587 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f1a4c030050 con 0x560bd76a51b0 2024-07-26T13:46:16.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.862+0000 7f1a66f95740 10 client.17100 fetch_fsmap learned FSMap version 53 2024-07-26T13:46:16.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.862+0000 7f1a66f95740 1 -- 172.21.15.100:0/2145727587 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x560bd7a920e0 con 0x560bd76a51b0 2024-07-26T13:46:16.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.862+0000 7f1a41ffb640 1 -- 172.21.15.100:0/2145727587 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a4c010c80 con 0x560bd76a51b0 2024-07-26T13:46:16.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.862+0000 7f1a41ffb640 1 -- 172.21.15.100:0/2145727587 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7f1a4c010840 con 0x560bd76a51b0 2024-07-26T13:46:16.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.862+0000 7f1a66f95740 10 client.17100 fetch_fsmap finished waiting for FSMap version 53 2024-07-26T13:46:16.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.862+0000 7f1a66f95740 10 client.17100 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:46:16.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.862+0000 7f1a66f95740 20 client.17100 populate_metadata read hostname 'smithi100' 2024-07-26T13:46:16.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.862+0000 7f1a66f95740 1 --2- 172.21.15.100:0/2145727587 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560bd768c9f0 0x560bd7a954f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:16.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.862+0000 7f1a66f95740 4 client.17100 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:46:16.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.862+0000 7f1a66f95740 1 -- 172.21.15.100:0/2145727587 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bd79ebae0 con 0x560bd768c9f0 2024-07-26T13:46:16.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.863+0000 7f1a59d9b640 1 --2- 172.21.15.100:0/2145727587 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560bd768c9f0 0x560bd7a954f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:16.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.863+0000 7f1a41ffb640 10 client.17100 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:16.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.863+0000 7f1a59d9b640 1 --2- 172.21.15.100:0/2145727587 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560bd768c9f0 0x560bd7a954f0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11e53c697a2dc701 server_cookie=dcec6c57d7cbcc8a in_seq=0 out_seq=0 2024-07-26T13:46:16.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.864+0000 7f1a41ffb640 1 -- 172.21.15.100:0/2145727587 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560bd79ebae0 con 0x560bd768c9f0 2024-07-26T13:46:16.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.864+0000 7f1a41ffb640 10 client.17100 handle_command_reply: tid=0 2024-07-26T13:46:16.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.864+0000 7f1a66f95740 1 client.17100 shutdown 2024-07-26T13:46:16.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.864+0000 7f1a66f95740 1 -- 172.21.15.100:0/2145727587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd76a51b0 msgr2=0x560bd7a83530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:16.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.864+0000 7f1a66f95740 1 --2- 172.21.15.100:0/2145727587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd76a51b0 0x560bd7a83530 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c0056d0 tx=0x7f1a4c005700 comp rx=0 tx=0).stop 2024-07-26T13:46:16.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.864+0000 7f1a66f95740 1 -- 172.21.15.100:0/2145727587 shutdown_connections 2024-07-26T13:46:16.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.864+0000 7f1a66f95740 1 --2- 172.21.15.100:0/2145727587 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560bd768c9f0 0x560bd7a954f0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:16.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.864+0000 7f1a66f95740 1 --2- 172.21.15.100:0/2145727587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd76ce070 0x560bd7a8f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:16.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.864+0000 7f1a66f95740 1 --2- 172.21.15.100:0/2145727587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd76ad390 0x560bd7a83a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:16.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.864+0000 7f1a66f95740 1 --2- 172.21.15.100:0/2145727587 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd76a51b0 0x560bd7a83530 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:16.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.864+0000 7f1a66f95740 1 -- 172.21.15.100:0/2145727587 >> 172.21.15.100:0/2145727587 conn(0x560bd76c2460 msgr2=0x560bd7a74f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:16.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.864+0000 7f1a41ffb640 0 client.17100 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:16.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.864+0000 7f1a66f95740 1 -- 172.21.15.100:0/2145727587 shutdown_connections 2024-07-26T13:46:16.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.864+0000 7f1a66f95740 1 -- 172.21.15.100:0/2145727587 wait complete. 2024-07-26T13:46:16.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.864+0000 7f1a66f95740 20 client.17100 trim_cache size 0 max 16384 2024-07-26T13:46:16.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.881+0000 7f1a66f95740 1 Processor -- start 2024-07-26T13:46:16.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.881+0000 7f1a66f95740 1 -- start start 2024-07-26T13:46:16.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.881+0000 7f1a66f95740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd76a51b0 0x560bd7a78cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:16.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.881+0000 7f1a66f95740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd76ad390 0x560bd7a70c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:16.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.881+0000 7f1a66f95740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd76ce070 0x560bd7a8f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:16.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.881+0000 7f1a66f95740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5c1af8c0 con 0x560bd76ad390 2024-07-26T13:46:16.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.881+0000 7f1a66f95740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5c1ad190 con 0x560bd76a51b0 2024-07-26T13:46:16.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.881+0000 7f1a66f95740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5c1ab000 con 0x560bd76ce070 2024-07-26T13:46:16.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.882+0000 7f1a59d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd76ad390 0x560bd7a70c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:16.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.882+0000 7f1a59d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd76ad390 0x560bd7a70c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52162/0 (socket says 172.21.15.100:52162) 2024-07-26T13:46:16.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.882+0000 7f1a59d9b640 1 -- 172.21.15.100:0/968708893 learned_addr learned my addr 172.21.15.100:0/968708893 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:16.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.882+0000 7f1a5ad9d640 1 --2- 172.21.15.100:0/968708893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd76ce070 0x560bd7a8f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:16.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.882+0000 7f1a5a59c640 1 --2- 172.21.15.100:0/968708893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd76a51b0 0x560bd7a78cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:16.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.882+0000 7f1a59d9b640 1 -- 172.21.15.100:0/968708893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd76ce070 msgr2=0x560bd7a8f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:16.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.882+0000 7f1a59d9b640 1 --2- 172.21.15.100:0/968708893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd76ce070 0x560bd7a8f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:16.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.882+0000 7f1a59d9b640 1 -- 172.21.15.100:0/968708893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd76a51b0 msgr2=0x560bd7a78cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:16.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.882+0000 7f1a59d9b640 1 --2- 172.21.15.100:0/968708893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd76a51b0 0x560bd7a78cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:16.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.882+0000 7f1a59d9b640 1 -- 172.21.15.100:0/968708893 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bd7a79600 con 0x560bd76ad390 2024-07-26T13:46:16.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.882+0000 7f1a59d9b640 1 --2- 172.21.15.100:0/968708893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd76ad390 0x560bd7a70c30 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f1a500aa9e0 tx=0x7f1a5000fdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c389b5407d91bf35 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:16.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.882+0000 7f1a43fff640 1 -- 172.21.15.100:0/968708893 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a500aace0 con 0x560bd76ad390 2024-07-26T13:46:16.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.882+0000 7f1a43fff640 1 -- 172.21.15.100:0/968708893 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a500af2e0 con 0x560bd76ad390 2024-07-26T13:46:16.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.882+0000 7f1a43fff640 1 -- 172.21.15.100:0/968708893 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a500b3b60 con 0x560bd76ad390 2024-07-26T13:46:16.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.883+0000 7f1a66f95740 1 -- 172.21.15.100:0/968708893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd76ad390 msgr2=0x560bd7a70c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:16.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.883+0000 7f1a66f95740 1 --2- 172.21.15.100:0/968708893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd76ad390 0x560bd7a70c30 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f1a500aa9e0 tx=0x7f1a5000fdc0 comp rx=0 tx=0).stop 2024-07-26T13:46:16.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.883+0000 7f1a66f95740 1 -- 172.21.15.100:0/968708893 shutdown_connections 2024-07-26T13:46:16.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.883+0000 7f1a66f95740 1 --2- 172.21.15.100:0/968708893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd76ce070 0x560bd7a8f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:16.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.883+0000 7f1a66f95740 1 --2- 172.21.15.100:0/968708893 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd76ad390 0x560bd7a70c30 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:16.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.883+0000 7f1a66f95740 1 --2- 172.21.15.100:0/968708893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd76a51b0 0x560bd7a78cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:16.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.883+0000 7f1a66f95740 1 -- 172.21.15.100:0/968708893 >> 172.21.15.100:0/968708893 conn(0x560bd76c2460 msgr2=0x560bd7a785c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:16.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.883+0000 7f1a66f95740 1 -- 172.21.15.100:0/968708893 shutdown_connections 2024-07-26T13:46:16.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.883+0000 7f1a66f95740 1 -- 172.21.15.100:0/968708893 wait complete. 2024-07-26T13:46:16.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.883+0000 7f1a66f95740 1 -- start start 2024-07-26T13:46:16.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.883+0000 7f1a66f95740 1 Processor -- start 2024-07-26T13:46:16.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.884+0000 7f1a66f95740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd76a51b0 0x560bd7a98910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:16.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.884+0000 7f1a66f95740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd76ad390 0x560bd7a95de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:16.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.884+0000 7f1a66f95740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd76ce070 0x560bd7a96320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:16.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.884+0000 7f1a66f95740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5c1ab650 con 0x560bd76ad390 2024-07-26T13:46:16.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.884+0000 7f1a66f95740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5c1acac0 con 0x560bd76a51b0 2024-07-26T13:46:16.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.884+0000 7f1a66f95740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5c1abd20 con 0x560bd76ce070 2024-07-26T13:46:16.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.884+0000 7f1a66f95740 10 client.0 _ll_register_callbacks cb 0x7f1a4c014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:46:16.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.884+0000 7f1a59d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd76ad390 0x560bd7a95de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:16.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.884+0000 7f1a59d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd76ad390 0x560bd7a95de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52176/0 (socket says 172.21.15.100:52176) 2024-07-26T13:46:16.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.884+0000 7f1a59d9b640 1 -- 172.21.15.100:0/835533206 learned_addr learned my addr 172.21.15.100:0/835533206 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:16.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.884+0000 7f1a5a59c640 1 --2- 172.21.15.100:0/835533206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd76a51b0 0x560bd7a98910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:16.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.884+0000 7f1a5ad9d640 1 --2- 172.21.15.100:0/835533206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd76ce070 0x560bd7a96320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:16.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.884+0000 7f1a59d9b640 1 -- 172.21.15.100:0/835533206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd76ce070 msgr2=0x560bd7a96320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:16.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.884+0000 7f1a59d9b640 1 --2- 172.21.15.100:0/835533206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd76ce070 0x560bd7a96320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:16.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.884+0000 7f1a59d9b640 1 -- 172.21.15.100:0/835533206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd76a51b0 msgr2=0x560bd7a98910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:16.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.884+0000 7f1a59d9b640 1 --2- 172.21.15.100:0/835533206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd76a51b0 0x560bd7a98910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:16.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.884+0000 7f1a59d9b640 1 -- 172.21.15.100:0/835533206 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bd7a96860 con 0x560bd76ad390 2024-07-26T13:46:16.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.884+0000 7f1a59d9b640 1 --2- 172.21.15.100:0/835533206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd76ad390 0x560bd7a95de0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f1a500a7f50 tx=0x7f1a500076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c64659d75064cfc server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:16.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.884+0000 7f1a41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:46:16.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.885+0000 7f1a41ffb640 1 -- 172.21.15.100:0/835533206 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a500aace0 con 0x560bd76ad390 2024-07-26T13:46:16.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.885+0000 7f1a41ffb640 1 -- 172.21.15.100:0/835533206 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7f1a500a6f70 con 0x560bd76ad390 2024-07-26T13:46:16.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.885+0000 7f1a66f95740 1 -- 172.21.15.100:0/835533206 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a500af2e0 con 0x560bd76ad390 2024-07-26T13:46:16.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.885+0000 7f1a41ffb640 1 -- 172.21.15.100:0/835533206 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f1a500b3900 con 0x560bd76ad390 2024-07-26T13:46:16.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.885+0000 7f1a66f95740 10 client.17106 fetch_fsmap learned FSMap version 53 2024-07-26T13:46:16.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.885+0000 7f1a66f95740 1 -- 172.21.15.100:0/835533206 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x560bd7afd6a0 con 0x560bd76ad390 2024-07-26T13:46:16.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.885+0000 7f1a41ffb640 1 -- 172.21.15.100:0/835533206 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a500af2e0 con 0x560bd76ad390 2024-07-26T13:46:16.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.885+0000 7f1a41ffb640 1 -- 172.21.15.100:0/835533206 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7f1a50092080 con 0x560bd76ad390 2024-07-26T13:46:16.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.885+0000 7f1a66f95740 10 client.17106 fetch_fsmap finished waiting for FSMap version 53 2024-07-26T13:46:16.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.885+0000 7f1a66f95740 10 client.17106 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:46:16.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.885+0000 7f1a66f95740 20 client.17106 populate_metadata read hostname 'smithi100' 2024-07-26T13:46:16.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.885+0000 7f1a66f95740 1 --2- 172.21.15.100:0/835533206 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560bd7661da0 0x560bd7affd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:16.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.885+0000 7f1a66f95740 4 client.17106 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-26T13:46:16.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.885+0000 7f1a66f95740 1 -- 172.21.15.100:0/835533206 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560bd79e7580 con 0x560bd7661da0 2024-07-26T13:46:16.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.885+0000 7f1a5a59c640 1 --2- 172.21.15.100:0/835533206 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560bd7661da0 0x560bd7affd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:16.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.886+0000 7f1a5a59c640 1 --2- 172.21.15.100:0/835533206 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560bd7661da0 0x560bd7affd30 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7a8cd8af7fc54c2 server_cookie=5a9fac706766cd93 in_seq=0 out_seq=0 2024-07-26T13:46:16.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.886+0000 7f1a41ffb640 10 client.17106 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:16.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.886+0000 7f1a41ffb640 1 -- 172.21.15.100:0/835533206 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560bd79e7580 con 0x560bd7661da0 2024-07-26T13:46:16.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.886+0000 7f1a41ffb640 10 client.17106 handle_command_reply: tid=0 2024-07-26T13:46:16.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.886+0000 7f1a66f95740 1 client.17106 shutdown 2024-07-26T13:46:16.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.886+0000 7f1a66f95740 1 -- 172.21.15.100:0/835533206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd76ad390 msgr2=0x560bd7a95de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:16.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.886+0000 7f1a66f95740 1 --2- 172.21.15.100:0/835533206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd76ad390 0x560bd7a95de0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f1a500a7f50 tx=0x7f1a500076c0 comp rx=0 tx=0).stop 2024-07-26T13:46:16.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.886+0000 7f1a66f95740 1 -- 172.21.15.100:0/835533206 shutdown_connections 2024-07-26T13:46:16.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.886+0000 7f1a66f95740 1 --2- 172.21.15.100:0/835533206 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560bd7661da0 0x560bd7affd30 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:16.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.886+0000 7f1a66f95740 1 --2- 172.21.15.100:0/835533206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bd76ce070 0x560bd7a96320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:16.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.886+0000 7f1a66f95740 1 --2- 172.21.15.100:0/835533206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560bd76ad390 0x560bd7a95de0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:16.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.886+0000 7f1a66f95740 1 --2- 172.21.15.100:0/835533206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560bd76a51b0 0x560bd7a98910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:16.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.886+0000 7f1a66f95740 1 -- 172.21.15.100:0/835533206 >> 172.21.15.100:0/835533206 conn(0x560bd76c2460 msgr2=0x560bd7a74580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:16.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.886+0000 7f1a41ffb640 0 client.17106 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:16.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.886+0000 7f1a66f95740 1 -- 172.21.15.100:0/835533206 shutdown_connections 2024-07-26T13:46:16.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.886+0000 7f1a66f95740 1 -- 172.21.15.100:0/835533206 wait complete. 2024-07-26T13:46:16.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.886+0000 7f1a66f95740 20 client.17106 trim_cache size 0 max 16384 2024-07-26T13:46:16.893 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:46:16.893 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-26T13:46:16.893 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-07-26T13:46:16.893 INFO:teuthology.orchestra.run.smithi100.stdout: "d6a5ba31-c00b-4228-bf3e-05684f27e70e": { 2024-07-26T13:46:16.893 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-07-26T13:46:16.893 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "d6a5ba31-c00b-4228-bf3e-05684f27e70e", 2024-07-26T13:46:16.893 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-07-26T13:46:16.894 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:46:16.894 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:46:16.894 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:46:16.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.887+0000 7f1a60a11640 1 -- 172.21.15.100:0/4161370987 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1a440b3d70 msgr2=0x7f1a440b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:16.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.887+0000 7f1a60a11640 1 --2- 172.21.15.100:0/4161370987 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1a440b3d70 0x7f1a440b6280 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1a540098d0 tx=0x7f1a5402f470 comp rx=0 tx=0).stop 2024-07-26T13:46:16.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.887+0000 7f1a60a11640 1 -- 172.21.15.100:0/4161370987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a5c107e50 msgr2=0x7f1a5c19a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:16.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.887+0000 7f1a60a11640 1 --2- 172.21.15.100:0/4161370987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a5c107e50 0x7f1a5c19a450 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000c8c0 tx=0x7f1a5000cd80 comp rx=0 tx=0).stop 2024-07-26T13:46:16.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.887+0000 7f1a60a11640 1 -- 172.21.15.100:0/4161370987 shutdown_connections 2024-07-26T13:46:16.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.887+0000 7f1a60a11640 1 --2- 172.21.15.100:0/4161370987 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1a440b3d70 0x7f1a440b6280 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:16.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.887+0000 7f1a60a11640 1 --2- 172.21.15.100:0/4161370987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a5c1087a0 0x7f1a5c19f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:16.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.887+0000 7f1a60a11640 1 --2- 172.21.15.100:0/4161370987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a5c107e50 0x7f1a5c19a450 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:16.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.887+0000 7f1a60a11640 1 --2- 172.21.15.100:0/4161370987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a5c102510 0x7f1a5c199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:16.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.887+0000 7f1a60a11640 1 -- 172.21.15.100:0/4161370987 >> 172.21.15.100:0/4161370987 conn(0x7f1a5c0791a0 msgr2=0x7f1a5c101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:16.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.887+0000 7f1a60a11640 1 -- 172.21.15.100:0/4161370987 shutdown_connections 2024-07-26T13:46:16.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:16.887+0000 7f1a60a11640 1 -- 172.21.15.100:0/4161370987 wait complete. 2024-07-26T13:46:17.420 INFO:tasks.workunit.client.0.smithi100.stdout: 1 234890 3.17 MB/sec execute 151 sec latency 1818.408 ms 2024-07-26T13:46:17.819 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:17 smithi100 ceph-mon[32599]: pgmap v552: 97 pgs: 97 active+clean; 88 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.7 MiB/s wr, 162 op/s 2024-07-26T13:46:17.854 INFO:tasks.cephfs.filesystem:scrub status for tag:d6a5ba31-c00b-4228-bf3e-05684f27e70e - {'path': '/', 'tag': 'd6a5ba31-c00b-4228-bf3e-05684f27e70e', 'options': 'recursive,force'} 2024-07-26T13:46:17.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:17 smithi138 ceph-mon[36954]: pgmap v552: 97 pgs: 97 active+clean; 88 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.7 MiB/s wr, 162 op/s 2024-07-26T13:46:18.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:17 smithi167 ceph-mon[37140]: pgmap v552: 97 pgs: 97 active+clean; 88 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.7 MiB/s wr, 162 op/s 2024-07-26T13:46:18.339 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'cd8d97d4', quiesce took 11.5 seconds. Will release after: 10.45 2024-07-26T13:46:18.420 INFO:tasks.workunit.client.0.smithi100.stdout: 1 234890 3.15 MB/sec execute 152 sec latency 2818.515 ms 2024-07-26T13:46:19.420 INFO:tasks.workunit.client.0.smithi100.stdout: 1 234890 3.13 MB/sec execute 153 sec latency 3818.620 ms 2024-07-26T13:46:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:19 smithi138 ceph-mon[36954]: pgmap v553: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.2 MiB/s wr, 203 op/s 2024-07-26T13:46:19.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:19 smithi100 ceph-mon[32599]: pgmap v553: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.2 MiB/s wr, 203 op/s 2024-07-26T13:46:20.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:19 smithi167 ceph-mon[37140]: pgmap v553: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.2 MiB/s wr, 203 op/s 2024-07-26T13:46:20.421 INFO:tasks.workunit.client.0.smithi100.stdout: 1 234890 3.11 MB/sec execute 154 sec latency 4818.724 ms 2024-07-26T13:46:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:20 smithi138 ceph-mon[36954]: scrub summary: idle 2024-07-26T13:46:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:20 smithi138 ceph-mon[36954]: scrub summary: idle 2024-07-26T13:46:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:20 smithi138 ceph-mon[36954]: scrub summary: idle 2024-07-26T13:46:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:20 smithi100 ceph-mon[32599]: scrub summary: idle 2024-07-26T13:46:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:20 smithi100 ceph-mon[32599]: scrub summary: idle 2024-07-26T13:46:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:20 smithi100 ceph-mon[32599]: scrub summary: idle 2024-07-26T13:46:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:20 smithi167 ceph-mon[37140]: scrub summary: idle 2024-07-26T13:46:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:20 smithi167 ceph-mon[37140]: scrub summary: idle 2024-07-26T13:46:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:20 smithi167 ceph-mon[37140]: scrub summary: idle 2024-07-26T13:46:21.421 INFO:tasks.workunit.client.0.smithi100.stdout: 1 234890 3.09 MB/sec execute 155 sec latency 5818.787 ms 2024-07-26T13:46:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.729+0000 7fae9f4fa640 1 -- 172.21.15.100:0/2558666647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae98107d40 msgr2=0x7fae98108150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.729+0000 7fae9f4fa640 1 --2- 172.21.15.100:0/2558666647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae98107d40 0x7fae98108150 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fae94009900 tx=0x7fae9402a410 comp rx=0 tx=0).stop 2024-07-26T13:46:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.729+0000 7fae9f4fa640 1 -- 172.21.15.100:0/2558666647 shutdown_connections 2024-07-26T13:46:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.729+0000 7fae9f4fa640 1 --2- 172.21.15.100:0/2558666647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae98108690 0x7fae9810eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.729+0000 7fae9f4fa640 1 --2- 172.21.15.100:0/2558666647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae98107d40 0x7fae98108150 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.729+0000 7fae9f4fa640 1 --2- 172.21.15.100:0/2558666647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae98102410 0x7fae98102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.729+0000 7fae9f4fa640 1 -- 172.21.15.100:0/2558666647 >> 172.21.15.100:0/2558666647 conn(0x7fae980fddf0 msgr2=0x7fae98100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.729+0000 7fae9f4fa640 1 -- 172.21.15.100:0/2558666647 shutdown_connections 2024-07-26T13:46:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.729+0000 7fae9f4fa640 1 -- 172.21.15.100:0/2558666647 wait complete. 2024-07-26T13:46:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.730+0000 7fae9f4fa640 1 Processor -- start 2024-07-26T13:46:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.730+0000 7fae9f4fa640 1 -- start start 2024-07-26T13:46:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.730+0000 7fae9f4fa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae98102410 0x7fae9810f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.730+0000 7fae9f4fa640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae98107d40 0x7fae9810fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.730+0000 7fae9f4fa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae98108690 0x7fae98110420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.730+0000 7fae9f4fa640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fae98113470 con 0x7fae98102410 2024-07-26T13:46:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.730+0000 7fae9f4fa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fae981132f0 con 0x7fae98108690 2024-07-26T13:46:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.730+0000 7fae9f4fa640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fae981135f0 con 0x7fae98107d40 2024-07-26T13:46:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.730+0000 7fae9d296640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae98102410 0x7fae9810f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.730+0000 7fae9d296640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae98102410 0x7fae9810f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52198/0 (socket says 172.21.15.100:52198) 2024-07-26T13:46:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.730+0000 7fae9d296640 1 -- 172.21.15.100:0/1326384349 learned_addr learned my addr 172.21.15.100:0/1326384349 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.730+0000 7fae9da97640 1 --2- 172.21.15.100:0/1326384349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae98108690 0x7fae98110420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.730+0000 7fae9ca95640 1 --2- 172.21.15.100:0/1326384349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae98107d40 0x7fae9810fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.730+0000 7fae9d296640 1 -- 172.21.15.100:0/1326384349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae98107d40 msgr2=0x7fae9810fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.730+0000 7fae9d296640 1 --2- 172.21.15.100:0/1326384349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae98107d40 0x7fae9810fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.730+0000 7fae9d296640 1 -- 172.21.15.100:0/1326384349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae98108690 msgr2=0x7fae98110420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.730+0000 7fae9d296640 1 --2- 172.21.15.100:0/1326384349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae98108690 0x7fae98110420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.730+0000 7fae9d296640 1 -- 172.21.15.100:0/1326384349 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae981a8180 con 0x7fae98102410 2024-07-26T13:46:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.730+0000 7fae9d296640 1 --2- 172.21.15.100:0/1326384349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae98102410 0x7fae9810f9a0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fae8c00e9f0 tx=0x7fae8c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.731+0000 7fae8a7fc640 1 -- 172.21.15.100:0/1326384349 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c00cd80 con 0x7fae98102410 2024-07-26T13:46:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.731+0000 7fae9f4fa640 1 -- 172.21.15.100:0/1326384349 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae981a8470 con 0x7fae98102410 2024-07-26T13:46:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.731+0000 7fae8a7fc640 1 -- 172.21.15.100:0/1326384349 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae8c0054d0 con 0x7fae98102410 2024-07-26T13:46:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.731+0000 7fae8a7fc640 1 -- 172.21.15.100:0/1326384349 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c010690 con 0x7fae98102410 2024-07-26T13:46:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.731+0000 7fae9f4fa640 1 -- 172.21.15.100:0/1326384349 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae981a8970 con 0x7fae98102410 2024-07-26T13:46:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.734+0000 7faea6414740 1 Processor -- start 2024-07-26T13:46:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.734+0000 7faea6414740 1 -- start start 2024-07-26T13:46:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.734+0000 7faea6414740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605908951b0 0x560590c37ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.734+0000 7faea6414740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56059086c2f0 0x560590c38220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.734+0000 7faea6414740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605908744d0 0x560590c38760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.734+0000 7faea6414740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x560590bad3b0 con 0x5605908744d0 2024-07-26T13:46:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.734+0000 7faea6414740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560590b9c8b0 con 0x5605908951b0 2024-07-26T13:46:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.734+0000 7faea6414740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560590ba08f0 con 0x56059086c2f0 2024-07-26T13:46:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.734+0000 7fae8a7fc640 1 -- 172.21.15.100:0/1326384349 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fae8c004e50 con 0x7fae98102410 2024-07-26T13:46:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.735+0000 7fae9ca95640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605908744d0 0x560590c38760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.735+0000 7fae9ca95640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605908744d0 0x560590c38760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52212/0 (socket says 172.21.15.100:52212) 2024-07-26T13:46:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.735+0000 7fae9ca95640 1 -- 172.21.15.100:0/2716732948 learned_addr learned my addr 172.21.15.100:0/2716732948 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.735+0000 7fae9da97640 1 --2- 172.21.15.100:0/2716732948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605908951b0 0x560590c37ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.735+0000 7fae9d296640 1 --2- 172.21.15.100:0/2716732948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56059086c2f0 0x560590c38220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.735+0000 7fae9ca95640 1 -- 172.21.15.100:0/2716732948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56059086c2f0 msgr2=0x560590c38220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.735+0000 7fae9ca95640 1 --2- 172.21.15.100:0/2716732948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56059086c2f0 0x560590c38220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.735+0000 7fae9ca95640 1 -- 172.21.15.100:0/2716732948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605908951b0 msgr2=0x560590c37ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.735+0000 7fae9ca95640 1 --2- 172.21.15.100:0/2716732948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605908951b0 0x560590c37ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.735+0000 7fae9ca95640 1 -- 172.21.15.100:0/2716732948 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560590b439a0 con 0x5605908744d0 2024-07-26T13:46:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.735+0000 7fae9d296640 1 --2- 172.21.15.100:0/2716732948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56059086c2f0 0x560590c38220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:46:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.735+0000 7fae9ca95640 1 --2- 172.21.15.100:0/2716732948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605908744d0 0x560590c38760 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fae94006010 tx=0x7fae94032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a38e927b3c5474d server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.735+0000 7fae837fe640 1 -- 172.21.15.100:0/2716732948 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9403f070 con 0x5605908744d0 2024-07-26T13:46:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.735+0000 7fae8a7fc640 1 --2- 172.21.15.100:0/1326384349 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fae840b3d70 0x7fae840b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.735+0000 7fae837fe640 1 -- 172.21.15.100:0/2716732948 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae94004030 con 0x5605908744d0 2024-07-26T13:46:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.735+0000 7fae837fe640 1 -- 172.21.15.100:0/2716732948 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae94035070 con 0x5605908744d0 2024-07-26T13:46:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.736+0000 7fae9da97640 1 --2- 172.21.15.100:0/1326384349 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fae840b3d70 0x7fae840b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.736+0000 7fae8a7fc640 1 -- 172.21.15.100:0/1326384349 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7fae8c0e45b0 con 0x7fae98102410 2024-07-26T13:46:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.736+0000 7faea6414740 1 -- 172.21.15.100:0/2716732948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605908744d0 msgr2=0x560590c38760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.736+0000 7fae9da97640 1 --2- 172.21.15.100:0/1326384349 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fae840b3d70 0x7fae840b6280 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fae90005fd0 tx=0x7fae90009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.736+0000 7faea6414740 1 --2- 172.21.15.100:0/2716732948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605908744d0 0x560590c38760 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fae94006010 tx=0x7fae94032b90 comp rx=0 tx=0).stop 2024-07-26T13:46:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.736+0000 7faea6414740 1 -- 172.21.15.100:0/2716732948 shutdown_connections 2024-07-26T13:46:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.736+0000 7faea6414740 1 --2- 172.21.15.100:0/2716732948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605908744d0 0x560590c38760 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.736+0000 7faea6414740 1 --2- 172.21.15.100:0/2716732948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56059086c2f0 0x560590c38220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.736+0000 7faea6414740 1 --2- 172.21.15.100:0/2716732948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605908951b0 0x560590c37ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.736+0000 7faea6414740 1 -- 172.21.15.100:0/2716732948 >> 172.21.15.100:0/2716732948 conn(0x560590889300 msgr2=0x560590c35bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.737+0000 7faea6414740 1 -- 172.21.15.100:0/2716732948 shutdown_connections 2024-07-26T13:46:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.737+0000 7faea6414740 1 -- 172.21.15.100:0/2716732948 wait complete. 2024-07-26T13:46:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.737+0000 7faea6414740 1 -- start start 2024-07-26T13:46:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.737+0000 7faea6414740 1 Processor -- start 2024-07-26T13:46:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.737+0000 7faea6414740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56059086c2f0 0x560590c43780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.738+0000 7faea6414740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605908744d0 0x560590c43cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.738+0000 7faea6414740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605908951b0 0x560590c4f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.738+0000 7faea6414740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x560590ad6ab0 con 0x56059086c2f0 2024-07-26T13:46:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.738+0000 7faea6414740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560590b24cf0 con 0x5605908951b0 2024-07-26T13:46:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.738+0000 7faea6414740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560590c46400 con 0x5605908744d0 2024-07-26T13:46:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.738+0000 7faea6414740 10 client.0 _ll_register_callbacks cb 0x560590b969f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:46:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.738+0000 7fae9d296640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56059086c2f0 0x560590c43780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.738+0000 7fae9d296640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56059086c2f0 0x560590c43780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52216/0 (socket says 172.21.15.100:52216) 2024-07-26T13:46:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.738+0000 7fae9d296640 1 -- 172.21.15.100:0/3223522895 learned_addr learned my addr 172.21.15.100:0/3223522895 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.738+0000 7fae9ca95640 1 --2- 172.21.15.100:0/3223522895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605908744d0 0x560590c43cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.738+0000 7fae9d296640 1 -- 172.21.15.100:0/3223522895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605908744d0 msgr2=0x560590c43cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.738+0000 7fae9d296640 1 --2- 172.21.15.100:0/3223522895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605908744d0 0x560590c43cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.738+0000 7fae9d296640 1 -- 172.21.15.100:0/3223522895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605908951b0 msgr2=0x560590c4f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.738+0000 7fae9d296640 1 --2- 172.21.15.100:0/3223522895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605908951b0 0x560590c4f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.738+0000 7fae9d296640 1 -- 172.21.15.100:0/3223522895 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560590900790 con 0x56059086c2f0 2024-07-26T13:46:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.738+0000 7fae9d296640 1 --2- 172.21.15.100:0/3223522895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56059086c2f0 0x560590c43780 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fae8c0e1c20 tx=0x7fae8c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725bd7420db4bca0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.738+0000 7fae80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:46:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.738+0000 7fae80ff9640 1 -- 172.21.15.100:0/3223522895 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c005680 con 0x56059086c2f0 2024-07-26T13:46:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.738+0000 7fae80ff9640 1 -- 172.21.15.100:0/3223522895 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7fae8c0e1e40 con 0x56059086c2f0 2024-07-26T13:46:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.738+0000 7faea6414740 1 -- 172.21.15.100:0/3223522895 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae94004030 con 0x56059086c2f0 2024-07-26T13:46:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.738+0000 7fae80ff9640 1 -- 172.21.15.100:0/3223522895 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fae8c0e2040 con 0x56059086c2f0 2024-07-26T13:46:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.738+0000 7faea6414740 10 client.17118 fetch_fsmap learned FSMap version 53 2024-07-26T13:46:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.738+0000 7faea6414740 1 -- 172.21.15.100:0/3223522895 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x560590979620 con 0x56059086c2f0 2024-07-26T13:46:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.739+0000 7fae80ff9640 1 -- 172.21.15.100:0/3223522895 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae94004030 con 0x56059086c2f0 2024-07-26T13:46:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.739+0000 7fae80ff9640 1 -- 172.21.15.100:0/3223522895 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7fae8c0b3ca0 con 0x56059086c2f0 2024-07-26T13:46:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.739+0000 7faea6414740 10 client.17118 fetch_fsmap finished waiting for FSMap version 53 2024-07-26T13:46:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.739+0000 7faea6414740 10 client.17118 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:46:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.739+0000 7faea6414740 20 client.17118 populate_metadata read hostname 'smithi100' 2024-07-26T13:46:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.739+0000 7faea6414740 1 --2- 172.21.15.100:0/3223522895 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560590853d80 0x560590c556f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.739+0000 7faea6414740 4 client.17118 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:46:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.739+0000 7faea6414740 1 -- 172.21.15.100:0/3223522895 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560590baf300 con 0x560590853d80 2024-07-26T13:46:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.739+0000 7fae9ca95640 1 --2- 172.21.15.100:0/3223522895 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560590853d80 0x560590c556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.740+0000 7fae80ff9640 10 client.17118 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.740+0000 7fae9ca95640 1 --2- 172.21.15.100:0/3223522895 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560590853d80 0x560590c556f0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29875c5f4565fe5b server_cookie=5389670af573c677 in_seq=0 out_seq=0 2024-07-26T13:46:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.740+0000 7fae80ff9640 1 -- 172.21.15.100:0/3223522895 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560590baf300 con 0x560590853d80 2024-07-26T13:46:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.740+0000 7fae80ff9640 10 client.17118 handle_command_reply: tid=0 2024-07-26T13:46:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.740+0000 7faea6414740 1 client.17118 shutdown 2024-07-26T13:46:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.740+0000 7faea6414740 1 -- 172.21.15.100:0/3223522895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56059086c2f0 msgr2=0x560590c43780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.740+0000 7faea6414740 1 --2- 172.21.15.100:0/3223522895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56059086c2f0 0x560590c43780 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fae8c0e1c20 tx=0x7fae8c0048f0 comp rx=0 tx=0).stop 2024-07-26T13:46:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.740+0000 7faea6414740 1 -- 172.21.15.100:0/3223522895 shutdown_connections 2024-07-26T13:46:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.741+0000 7faea6414740 1 --2- 172.21.15.100:0/3223522895 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560590853d80 0x560590c556f0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.741+0000 7faea6414740 1 --2- 172.21.15.100:0/3223522895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605908951b0 0x560590c4f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.741+0000 7faea6414740 1 --2- 172.21.15.100:0/3223522895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605908744d0 0x560590c43cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.741+0000 7fae80ff9640 0 client.17118 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.741+0000 7faea6414740 1 --2- 172.21.15.100:0/3223522895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56059086c2f0 0x560590c43780 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.741+0000 7faea6414740 1 -- 172.21.15.100:0/3223522895 >> 172.21.15.100:0/3223522895 conn(0x560590889300 msgr2=0x560590c38d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.741+0000 7faea6414740 1 -- 172.21.15.100:0/3223522895 shutdown_connections 2024-07-26T13:46:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.741+0000 7faea6414740 1 -- 172.21.15.100:0/3223522895 wait complete. 2024-07-26T13:46:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.741+0000 7faea6414740 20 client.17118 trim_cache size 0 max 16384 2024-07-26T13:46:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.756+0000 7faea6414740 1 Processor -- start 2024-07-26T13:46:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.756+0000 7faea6414740 1 -- start start 2024-07-26T13:46:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.756+0000 7faea6414740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56059086c2f0 0x560590c43bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.756+0000 7faea6414740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605908744d0 0x560590c4f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.756+0000 7faea6414740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605908951b0 0x560590c4fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.756+0000 7faea6414740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fae981b3df0 con 0x5605908951b0 2024-07-26T13:46:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.756+0000 7faea6414740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fae981b16c0 con 0x56059086c2f0 2024-07-26T13:46:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.756+0000 7faea6414740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fae981af510 con 0x5605908744d0 2024-07-26T13:46:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.756+0000 7fae9da97640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605908951b0 0x560590c4fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.756+0000 7fae9da97640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605908951b0 0x560590c4fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52232/0 (socket says 172.21.15.100:52232) 2024-07-26T13:46:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.756+0000 7fae9da97640 1 -- 172.21.15.100:0/790022132 learned_addr learned my addr 172.21.15.100:0/790022132 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.756+0000 7fae9ca95640 1 --2- 172.21.15.100:0/790022132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605908744d0 0x560590c4f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.756+0000 7fae9d296640 1 --2- 172.21.15.100:0/790022132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56059086c2f0 0x560590c43bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.756+0000 7fae9da97640 1 -- 172.21.15.100:0/790022132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605908744d0 msgr2=0x560590c4f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.756+0000 7fae9da97640 1 --2- 172.21.15.100:0/790022132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605908744d0 0x560590c4f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.756+0000 7fae9da97640 1 -- 172.21.15.100:0/790022132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56059086c2f0 msgr2=0x560590c43bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.756+0000 7fae9da97640 1 --2- 172.21.15.100:0/790022132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56059086c2f0 0x560590c43bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.756+0000 7fae9da97640 1 -- 172.21.15.100:0/790022132 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560590c432f0 con 0x5605908951b0 2024-07-26T13:46:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.756+0000 7fae9d296640 1 --2- 172.21.15.100:0/790022132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56059086c2f0 0x560590c43bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:46:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.756+0000 7fae9ca95640 1 --2- 172.21.15.100:0/790022132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605908744d0 0x560590c4f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:46:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.757+0000 7fae9da97640 1 --2- 172.21.15.100:0/790022132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605908951b0 0x560590c4fa50 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fae90003d30 tx=0x7fae90004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3820e0988c6fb274 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.757+0000 7fae82ffd640 1 -- 172.21.15.100:0/790022132 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae90003e20 con 0x5605908951b0 2024-07-26T13:46:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.757+0000 7fae82ffd640 1 -- 172.21.15.100:0/790022132 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae9000f990 con 0x5605908951b0 2024-07-26T13:46:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.757+0000 7fae82ffd640 1 -- 172.21.15.100:0/790022132 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9001db40 con 0x5605908951b0 2024-07-26T13:46:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.757+0000 7faea6414740 1 -- 172.21.15.100:0/790022132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605908951b0 msgr2=0x560590c4fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.757+0000 7faea6414740 1 --2- 172.21.15.100:0/790022132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605908951b0 0x560590c4fa50 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fae90003d30 tx=0x7fae90004a40 comp rx=0 tx=0).stop 2024-07-26T13:46:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.757+0000 7faea6414740 1 -- 172.21.15.100:0/790022132 shutdown_connections 2024-07-26T13:46:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.757+0000 7faea6414740 1 --2- 172.21.15.100:0/790022132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5605908951b0 0x560590c4fa50 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.757+0000 7faea6414740 1 --2- 172.21.15.100:0/790022132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605908744d0 0x560590c4f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.757+0000 7faea6414740 1 --2- 172.21.15.100:0/790022132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56059086c2f0 0x560590c43bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.757+0000 7faea6414740 1 -- 172.21.15.100:0/790022132 >> 172.21.15.100:0/790022132 conn(0x560590889300 msgr2=0x560590c434e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.757+0000 7faea6414740 1 -- 172.21.15.100:0/790022132 shutdown_connections 2024-07-26T13:46:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.758+0000 7faea6414740 1 -- 172.21.15.100:0/790022132 wait complete. 2024-07-26T13:46:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.758+0000 7faea6414740 1 -- start start 2024-07-26T13:46:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.758+0000 7faea6414740 1 Processor -- start 2024-07-26T13:46:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.758+0000 7faea6414740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56059086c2f0 0x560590c48650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.758+0000 7faea6414740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605908744d0 0x560590c48b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:21.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.758+0000 7faea6414740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605908951b0 0x560590c490d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:21.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.758+0000 7faea6414740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fae981afb80 con 0x56059086c2f0 2024-07-26T13:46:21.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.758+0000 7faea6414740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fae981b0ff0 con 0x5605908951b0 2024-07-26T13:46:21.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.758+0000 7faea6414740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fae981b0250 con 0x5605908744d0 2024-07-26T13:46:21.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.758+0000 7faea6414740 10 client.0 _ll_register_callbacks cb 0x7fae8c009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:46:21.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.758+0000 7fae9d296640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56059086c2f0 0x560590c48650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:21.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.758+0000 7fae9d296640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56059086c2f0 0x560590c48650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52246/0 (socket says 172.21.15.100:52246) 2024-07-26T13:46:21.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.758+0000 7fae9d296640 1 -- 172.21.15.100:0/2072766156 learned_addr learned my addr 172.21.15.100:0/2072766156 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:21.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.759+0000 7fae9ca95640 1 --2- 172.21.15.100:0/2072766156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605908744d0 0x560590c48b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:21.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.759+0000 7fae9d296640 1 -- 172.21.15.100:0/2072766156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605908744d0 msgr2=0x560590c48b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:21.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.759+0000 7fae9d296640 1 --2- 172.21.15.100:0/2072766156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605908744d0 0x560590c48b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:21.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.759+0000 7fae9d296640 1 -- 172.21.15.100:0/2072766156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605908951b0 msgr2=0x560590c490d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:21.765 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.759+0000 7fae9d296640 1 --2- 172.21.15.100:0/2072766156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605908951b0 0x560590c490d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.759+0000 7fae9d296640 1 -- 172.21.15.100:0/2072766156 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560590cbc010 con 0x56059086c2f0 2024-07-26T13:46:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.759+0000 7fae9d296640 1 --2- 172.21.15.100:0/2072766156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56059086c2f0 0x560590c48650 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fae8c0a82f0 tx=0x7fae8c0e20f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb5b95ac863e6bb server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.759+0000 7fae80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:46:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.759+0000 7fae80ff9640 1 -- 172.21.15.100:0/2072766156 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c0b4130 con 0x56059086c2f0 2024-07-26T13:46:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.759+0000 7fae80ff9640 1 -- 172.21.15.100:0/2072766156 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7fae8c0aa4f0 con 0x56059086c2f0 2024-07-26T13:46:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.759+0000 7faea6414740 1 -- 172.21.15.100:0/2072766156 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae9000f990 con 0x56059086c2f0 2024-07-26T13:46:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.759+0000 7fae80ff9640 1 -- 172.21.15.100:0/2072766156 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fae8c0ab080 con 0x56059086c2f0 2024-07-26T13:46:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.759+0000 7faea6414740 10 client.17124 fetch_fsmap learned FSMap version 53 2024-07-26T13:46:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.759+0000 7faea6414740 1 -- 172.21.15.100:0/2072766156 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x560590cbd160 con 0x56059086c2f0 2024-07-26T13:46:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.759+0000 7fae80ff9640 1 -- 172.21.15.100:0/2072766156 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae9000f990 con 0x56059086c2f0 2024-07-26T13:46:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.759+0000 7fae80ff9640 1 -- 172.21.15.100:0/2072766156 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7fae8c0ca020 con 0x56059086c2f0 2024-07-26T13:46:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.760+0000 7faea6414740 10 client.17124 fetch_fsmap finished waiting for FSMap version 53 2024-07-26T13:46:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.760+0000 7faea6414740 10 client.17124 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:46:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.760+0000 7faea6414740 20 client.17124 populate_metadata read hostname 'smithi100' 2024-07-26T13:46:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.760+0000 7faea6414740 1 --2- 172.21.15.100:0/2072766156 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5605908291c0 0x560590cbf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.760+0000 7faea6414740 4 client.17124 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-26T13:46:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.760+0000 7faea6414740 1 -- 172.21.15.100:0/2072766156 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560590baada0 con 0x5605908291c0 2024-07-26T13:46:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.760+0000 7fae9ca95640 1 --2- 172.21.15.100:0/2072766156 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5605908291c0 0x560590cbf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.760+0000 7fae80ff9640 10 client.17124 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.760+0000 7fae9ca95640 1 --2- 172.21.15.100:0/2072766156 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5605908291c0 0x560590cbf770 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9c1c083b787abd4 server_cookie=2a1f028d8b45d3e8 in_seq=0 out_seq=0 2024-07-26T13:46:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.808+0000 7fae80ff9640 1 -- 172.21.15.100:0/2072766156 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560590baada0 con 0x5605908291c0 2024-07-26T13:46:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.808+0000 7fae80ff9640 10 client.17124 handle_command_reply: tid=0 2024-07-26T13:46:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.808+0000 7faea6414740 1 client.17124 shutdown 2024-07-26T13:46:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.808+0000 7faea6414740 1 -- 172.21.15.100:0/2072766156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56059086c2f0 msgr2=0x560590c48650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.808+0000 7faea6414740 1 --2- 172.21.15.100:0/2072766156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56059086c2f0 0x560590c48650 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fae8c0a82f0 tx=0x7fae8c0e20f0 comp rx=0 tx=0).stop 2024-07-26T13:46:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.808+0000 7faea6414740 1 -- 172.21.15.100:0/2072766156 shutdown_connections 2024-07-26T13:46:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.808+0000 7faea6414740 1 --2- 172.21.15.100:0/2072766156 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5605908291c0 0x560590cbf770 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.808+0000 7faea6414740 1 --2- 172.21.15.100:0/2072766156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5605908951b0 0x560590c490d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.808+0000 7faea6414740 1 --2- 172.21.15.100:0/2072766156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605908744d0 0x560590c48b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.808+0000 7faea6414740 1 --2- 172.21.15.100:0/2072766156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56059086c2f0 0x560590c48650 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.808+0000 7fae80ff9640 0 client.17124 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.808+0000 7faea6414740 1 -- 172.21.15.100:0/2072766156 >> 172.21.15.100:0/2072766156 conn(0x560590889300 msgr2=0x560590c448f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.808+0000 7faea6414740 1 -- 172.21.15.100:0/2072766156 shutdown_connections 2024-07-26T13:46:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.808+0000 7faea6414740 1 -- 172.21.15.100:0/2072766156 wait complete. 2024-07-26T13:46:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.808+0000 7faea6414740 20 client.17124 trim_cache size 0 max 16384 2024-07-26T13:46:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-07-26T13:46:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-07-26T13:46:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-07-26T13:46:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-07-26T13:46:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.809+0000 7fae9f4fa640 1 -- 172.21.15.100:0/1326384349 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fae840b3d70 msgr2=0x7fae840b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.809+0000 7fae9f4fa640 1 --2- 172.21.15.100:0/1326384349 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fae840b3d70 0x7fae840b6280 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fae90005fd0 tx=0x7fae90009500 comp rx=0 tx=0).stop 2024-07-26T13:46:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.809+0000 7fae9f4fa640 1 -- 172.21.15.100:0/1326384349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae98102410 msgr2=0x7fae9810f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.809+0000 7fae9f4fa640 1 --2- 172.21.15.100:0/1326384349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae98102410 0x7fae9810f9a0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fae8c00e9f0 tx=0x7fae8c00edf0 comp rx=0 tx=0).stop 2024-07-26T13:46:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.809+0000 7fae9f4fa640 1 -- 172.21.15.100:0/1326384349 shutdown_connections 2024-07-26T13:46:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.809+0000 7fae9f4fa640 1 --2- 172.21.15.100:0/1326384349 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fae840b3d70 0x7fae840b6280 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.809+0000 7fae9f4fa640 1 --2- 172.21.15.100:0/1326384349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae98108690 0x7fae98110420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.809+0000 7fae9f4fa640 1 --2- 172.21.15.100:0/1326384349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae98107d40 0x7fae9810fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.809+0000 7fae9f4fa640 1 --2- 172.21.15.100:0/1326384349 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae98102410 0x7fae9810f9a0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.809+0000 7fae9f4fa640 1 -- 172.21.15.100:0/1326384349 >> 172.21.15.100:0/1326384349 conn(0x7fae980fddf0 msgr2=0x7fae98106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.809+0000 7fae9f4fa640 1 -- 172.21.15.100:0/1326384349 shutdown_connections 2024-07-26T13:46:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:21.809+0000 7fae9f4fa640 1 -- 172.21.15.100:0/1326384349 wait complete. 2024-07-26T13:46:21.924 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:21 smithi167 ceph-mon[37140]: pgmap v554: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.6 MiB/s wr, 224 op/s 2024-07-26T13:46:21.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:21 smithi138 ceph-mon[36954]: pgmap v554: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.6 MiB/s wr, 224 op/s 2024-07-26T13:46:21.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:21 smithi100 ceph-mon[32599]: pgmap v554: 97 pgs: 97 active+clean; 92 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.6 MiB/s wr, 224 op/s 2024-07-26T13:46:22.421 INFO:tasks.workunit.client.0.smithi100.stdout: 1 234890 3.07 MB/sec execute 156 sec latency 6818.879 ms 2024-07-26T13:46:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:22 smithi138 ceph-mon[36954]: pgmap v555: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.5 MiB/s wr, 213 op/s 2024-07-26T13:46:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:22 smithi100 ceph-mon[32599]: pgmap v555: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.5 MiB/s wr, 213 op/s 2024-07-26T13:46:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:22 smithi167 ceph-mon[37140]: pgmap v555: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.5 MiB/s wr, 213 op/s 2024-07-26T13:46:23.421 INFO:tasks.workunit.client.0.smithi100.stdout: 1 234890 3.05 MB/sec execute 157 sec latency 7818.974 ms 2024-07-26T13:46:24.421 INFO:tasks.workunit.client.0.smithi100.stdout: 1 234890 3.03 MB/sec execute 158 sec latency 8819.057 ms 2024-07-26T13:46:25.421 INFO:tasks.workunit.client.0.smithi100.stdout: 1 234890 3.01 MB/sec execute 159 sec latency 9819.147 ms 2024-07-26T13:46:25.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:25 smithi138 ceph-mon[36954]: pgmap v556: 97 pgs: 97 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 196 op/s 2024-07-26T13:46:25.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:25 smithi100 ceph-mon[32599]: pgmap v556: 97 pgs: 97 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 196 op/s 2024-07-26T13:46:26.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:25 smithi167 ceph-mon[37140]: pgmap v556: 97 pgs: 97 active+clean; 89 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 196 op/s 2024-07-26T13:46:26.421 INFO:tasks.workunit.client.0.smithi100.stdout: 1 234890 2.99 MB/sec execute 160 sec latency 10819.211 ms 2024-07-26T13:46:27.421 INFO:tasks.workunit.client.0.smithi100.stdout: 1 234890 2.97 MB/sec execute 161 sec latency 11819.289 ms 2024-07-26T13:46:27.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:27 smithi138 ceph-mon[36954]: pgmap v557: 97 pgs: 97 active+clean; 87 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 152 op/s 2024-07-26T13:46:27.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:27 smithi100 ceph-mon[32599]: pgmap v557: 97 pgs: 97 active+clean; 87 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 152 op/s 2024-07-26T13:46:28.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:27 smithi167 ceph-mon[37140]: pgmap v557: 97 pgs: 97 active+clean; 87 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 152 op/s 2024-07-26T13:46:28.421 INFO:tasks.workunit.client.0.smithi100.stdout: 1 234890 2.95 MB/sec execute 162 sec latency 12819.389 ms 2024-07-26T13:46:28.791 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:46:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:28 smithi138 ceph-mon[36954]: pgmap v558: 97 pgs: 97 active+clean; 87 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 1.7 MiB/s wr, 123 op/s 2024-07-26T13:46:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:28 smithi100 ceph-mon[32599]: pgmap v558: 97 pgs: 97 active+clean; 87 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 1.7 MiB/s wr, 123 op/s 2024-07-26T13:46:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:28 smithi167 ceph-mon[37140]: pgmap v558: 97 pgs: 97 active+clean; 87 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 1.7 MiB/s wr, 123 op/s 2024-07-26T13:46:29.058 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:46:29.421 INFO:tasks.workunit.client.0.smithi100.stdout: 1 234890 2.93 MB/sec execute 163 sec latency 13819.483 ms 2024-07-26T13:46:30.421 INFO:tasks.workunit.client.0.smithi100.stdout: 1 234890 2.92 MB/sec execute 164 sec latency 14819.589 ms 2024-07-26T13:46:30.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.869+0000 7f235fbd7640 1 -- 172.21.15.100:0/2577571542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2358108730 msgr2=0x7f235810ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:30.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.869+0000 7f235fbd7640 1 --2- 172.21.15.100:0/2577571542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2358108730 0x7f235810ef60 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f2350009ca0 tx=0x7f235002a8c0 comp rx=0 tx=0).stop 2024-07-26T13:46:30.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.869+0000 7f235fbd7640 1 -- 172.21.15.100:0/2577571542 shutdown_connections 2024-07-26T13:46:30.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.869+0000 7f235fbd7640 1 --2- 172.21.15.100:0/2577571542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2358108730 0x7f235810ef60 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:30.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.869+0000 7f235fbd7640 1 --2- 172.21.15.100:0/2577571542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2358107de0 0x7f23581081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:30.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.869+0000 7f235fbd7640 1 --2- 172.21.15.100:0/2577571542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23581024f0 0x7f2358102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:30.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.869+0000 7f235fbd7640 1 -- 172.21.15.100:0/2577571542 >> 172.21.15.100:0/2577571542 conn(0x7f2358079150 msgr2=0x7f2358079560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:30.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.869+0000 7f235fbd7640 1 -- 172.21.15.100:0/2577571542 shutdown_connections 2024-07-26T13:46:30.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.869+0000 7f235fbd7640 1 -- 172.21.15.100:0/2577571542 wait complete. 2024-07-26T13:46:30.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.870+0000 7f235fbd7640 1 Processor -- start 2024-07-26T13:46:30.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.870+0000 7f235fbd7640 1 -- start start 2024-07-26T13:46:30.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.870+0000 7f235fbd7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23581024f0 0x7f2358198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:30.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.870+0000 7f235fbd7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2358107de0 0x7f23581993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:30.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.870+0000 7f235fbd7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2358108730 0x7f235819e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:30.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.870+0000 7f235fbd7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2358112400 con 0x7f23581024f0 2024-07-26T13:46:30.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.870+0000 7f235fbd7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2358112280 con 0x7f2358108730 2024-07-26T13:46:30.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.870+0000 7f235fbd7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2358112580 con 0x7f2358107de0 2024-07-26T13:46:30.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.870+0000 7f235d973640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23581024f0 0x7f2358198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:30.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.870+0000 7f235d973640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23581024f0 0x7f2358198e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57064/0 (socket says 172.21.15.100:57064) 2024-07-26T13:46:30.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.870+0000 7f235d973640 1 -- 172.21.15.100:0/2066841303 learned_addr learned my addr 172.21.15.100:0/2066841303 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:30.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.870+0000 7f235e174640 1 --2- 172.21.15.100:0/2066841303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2358108730 0x7f235819e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:30.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.870+0000 7f235d172640 1 --2- 172.21.15.100:0/2066841303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2358107de0 0x7f23581993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:30.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.870+0000 7f235d973640 1 -- 172.21.15.100:0/2066841303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2358107de0 msgr2=0x7f23581993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:30.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.870+0000 7f235d973640 1 --2- 172.21.15.100:0/2066841303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2358107de0 0x7f23581993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:30.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.870+0000 7f235d973640 1 -- 172.21.15.100:0/2066841303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2358108730 msgr2=0x7f235819e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:30.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.870+0000 7f235d973640 1 --2- 172.21.15.100:0/2066841303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2358108730 0x7f235819e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:30.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.871+0000 7f235d973640 1 -- 172.21.15.100:0/2066841303 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f235819edc0 con 0x7f23581024f0 2024-07-26T13:46:30.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.871+0000 7f235d973640 1 --2- 172.21.15.100:0/2066841303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23581024f0 0x7f2358198e50 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f234800e9f0 tx=0x7f234800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:30.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.871+0000 7f2356ffd640 1 -- 172.21.15.100:0/2066841303 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234800cd80 con 0x7f23581024f0 2024-07-26T13:46:30.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.871+0000 7f2356ffd640 1 -- 172.21.15.100:0/2066841303 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23480054d0 con 0x7f23581024f0 2024-07-26T13:46:30.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.871+0000 7f235fbd7640 1 -- 172.21.15.100:0/2066841303 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f235819f0b0 con 0x7f23581024f0 2024-07-26T13:46:30.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.871+0000 7f2356ffd640 1 -- 172.21.15.100:0/2066841303 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2348010690 con 0x7f23581024f0 2024-07-26T13:46:30.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.871+0000 7f235fbd7640 1 -- 172.21.15.100:0/2066841303 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23581a5810 con 0x7f23581024f0 2024-07-26T13:46:30.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.872+0000 7f235fbd7640 1 -- 172.21.15.100:0/2066841303 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2358102900 con 0x7f23581024f0 2024-07-26T13:46:30.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.873+0000 7f2356ffd640 1 -- 172.21.15.100:0/2066841303 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f2348004e50 con 0x7f23581024f0 2024-07-26T13:46:30.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.874+0000 7f2356ffd640 1 --2- 172.21.15.100:0/2066841303 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f23400b3dc0 0x7f23400b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:30.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.875+0000 7f235d172640 1 --2- 172.21.15.100:0/2066841303 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f23400b3dc0 0x7f23400b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:30.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.875+0000 7f2356ffd640 1 -- 172.21.15.100:0/2066841303 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7f23480e46f0 con 0x7f23581024f0 2024-07-26T13:46:30.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.875+0000 7f235d172640 1 --2- 172.21.15.100:0/2066841303 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f23400b3dc0 0x7f23400b62d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f234c005d90 tx=0x7f234c005d20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:30.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:30.878+0000 7f2356ffd640 1 -- 172.21.15.100:0/2066841303 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f2348055600 con 0x7f23581024f0 2024-07-26T13:46:31.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:31.085+0000 7f235fbd7640 1 -- 172.21.15.100:0/2066841303 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2358199f80 con 0x7f23581024f0 2024-07-26T13:46:31.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:31.086+0000 7f2356ffd640 1 -- 172.21.15.100:0/2066841303 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+11928 (secure 0 0 0) 0x7f23480ab920 con 0x7f23581024f0 2024-07-26T13:46:31.088 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:46:31.088 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":53,"btime":"2024-07-26T13:46:08:649224+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14790,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/313501409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":313501409},{"type":"v1","addr":"172.21.15.100:6839","nonce":313501409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/1402975506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":1402975506},{"type":"v1","addr":"172.21.15.100:6841","nonce":1402975506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":53,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-26T13:37:01.281843+0000","modified":"2024-07-26T13:46:08.649219+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24607,"mds_1":24605,"mds_2":25997},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24575":{"gid":24575,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/3393225483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3393225483},{"type":"v1","addr":"172.21.15.138:6837","nonce":3393225483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.138:6839/1580925471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1580925471},{"type":"v1","addr":"172.21.15.138:6839","nonce":1580925471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/398150514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":398150514},{"type":"v1","addr":"172.21.15.167:6839","nonce":398150514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25997":{"gid":25997,"name":"e","rank":2,"incarnation":49,"state":"up:active","state_seq":23,"addr":"172.21.15.138:6835/2831339503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2831339503},{"type":"v1","addr":"172.21.15.138:6835","nonce":2831339503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26446":{"gid":26446,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/3519222898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3519222898},{"type":"v1","addr":"172.21.15.167:6837","nonce":3519222898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24607,"qdb_cluster":[24605,24607,25997]},"id":1}]} 2024-07-26T13:46:31.088 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 53 2024-07-26T13:46:31.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:31.089+0000 7f235fbd7640 1 -- 172.21.15.100:0/2066841303 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f23400b3dc0 msgr2=0x7f23400b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:31.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:31.089+0000 7f235fbd7640 1 --2- 172.21.15.100:0/2066841303 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f23400b3dc0 0x7f23400b62d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f234c005d90 tx=0x7f234c005d20 comp rx=0 tx=0).stop 2024-07-26T13:46:31.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:31.089+0000 7f235fbd7640 1 -- 172.21.15.100:0/2066841303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23581024f0 msgr2=0x7f2358198e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:31.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:31.089+0000 7f235fbd7640 1 --2- 172.21.15.100:0/2066841303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23581024f0 0x7f2358198e50 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f234800e9f0 tx=0x7f234800edf0 comp rx=0 tx=0).stop 2024-07-26T13:46:31.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:31.089+0000 7f235fbd7640 1 -- 172.21.15.100:0/2066841303 shutdown_connections 2024-07-26T13:46:31.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:31.089+0000 7f235fbd7640 1 --2- 172.21.15.100:0/2066841303 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f23400b3dc0 0x7f23400b62d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:31.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:31.089+0000 7f235fbd7640 1 --2- 172.21.15.100:0/2066841303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2358108730 0x7f235819e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:31.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:31.089+0000 7f235fbd7640 1 --2- 172.21.15.100:0/2066841303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2358107de0 0x7f23581993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:31.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:31.089+0000 7f235fbd7640 1 --2- 172.21.15.100:0/2066841303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23581024f0 0x7f2358198e50 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:31.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:31.089+0000 7f235fbd7640 1 -- 172.21.15.100:0/2066841303 >> 172.21.15.100:0/2066841303 conn(0x7f2358079150 msgr2=0x7f2358103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:31.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:31.089+0000 7f235fbd7640 1 -- 172.21.15.100:0/2066841303 shutdown_connections 2024-07-26T13:46:31.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:31.089+0000 7f235fbd7640 1 -- 172.21.15.100:0/2066841303 wait complete. 2024-07-26T13:46:31.421 INFO:tasks.workunit.client.0.smithi100.stdout: 1 234890 2.90 MB/sec execute 165 sec latency 15819.619 ms 2024-07-26T13:46:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:31 smithi100 ceph-mon[32599]: pgmap v559: 97 pgs: 97 active+clean; 87 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.1 MiB/s wr, 76 op/s 2024-07-26T13:46:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:31 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2066841303' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:46:31.820 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24607 quiesce db --set-id cd8d97d4 --release --await' 2024-07-26T13:46:31.820 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- timeout 105 ceph tell mds.24607 quiesce db --set-id cd8d97d4 --release --await 2024-07-26T13:46:31.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:31 smithi138 ceph-mon[36954]: pgmap v559: 97 pgs: 97 active+clean; 87 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.1 MiB/s wr, 76 op/s 2024-07-26T13:46:31.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:31 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/2066841303' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:46:32.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:31 smithi167 ceph-mon[37140]: pgmap v559: 97 pgs: 97 active+clean; 87 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.1 MiB/s wr, 76 op/s 2024-07-26T13:46:32.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:31 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/2066841303' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:46:32.080 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:46:32.421 INFO:tasks.workunit.client.0.smithi100.stdout: 1 234890 2.88 MB/sec execute 166 sec latency 16819.677 ms 2024-07-26T13:46:33.422 INFO:tasks.workunit.client.0.smithi100.stdout: 1 234890 2.86 MB/sec execute 167 sec latency 17819.714 ms 2024-07-26T13:46:33.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.752+0000 7f9c6a5ee640 1 -- 172.21.15.100:0/3150665446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c64105d00 msgr2=0x7f9c64106110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:33.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.752+0000 7f9c6a5ee640 1 --2- 172.21.15.100:0/3150665446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c64105d00 0x7f9c64106110 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c009900 tx=0x7f9c4c02a410 comp rx=0 tx=0).stop 2024-07-26T13:46:33.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.752+0000 7f9c6a5ee640 1 -- 172.21.15.100:0/3150665446 shutdown_connections 2024-07-26T13:46:33.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.752+0000 7f9c6a5ee640 1 --2- 172.21.15.100:0/3150665446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c64106650 0x7f9c641100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:33.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.752+0000 7f9c6a5ee640 1 --2- 172.21.15.100:0/3150665446 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c64105d00 0x7f9c64106110 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:33.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.752+0000 7f9c6a5ee640 1 --2- 172.21.15.100:0/3150665446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c64101a20 0x7f9c64101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:33.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.752+0000 7f9c6a5ee640 1 -- 172.21.15.100:0/3150665446 >> 172.21.15.100:0/3150665446 conn(0x7f9c64079860 msgr2=0x7f9c64079c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:33.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.752+0000 7f9c6a5ee640 1 -- 172.21.15.100:0/3150665446 shutdown_connections 2024-07-26T13:46:33.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.752+0000 7f9c6a5ee640 1 -- 172.21.15.100:0/3150665446 wait complete. 2024-07-26T13:46:33.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.753+0000 7f9c6a5ee640 1 Processor -- start 2024-07-26T13:46:33.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.753+0000 7f9c6a5ee640 1 -- start start 2024-07-26T13:46:33.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.753+0000 7f9c6a5ee640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c64101a20 0x7f9c64199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:33.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.753+0000 7f9c6a5ee640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c64105d00 0x7f9c6419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:33.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.753+0000 7f9c6a5ee640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c64106650 0x7f9c6419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:33.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.753+0000 7f9c6a5ee640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9c641134e0 con 0x7f9c64101a20 2024-07-26T13:46:33.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.753+0000 7f9c6a5ee640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9c64113360 con 0x7f9c64105d00 2024-07-26T13:46:33.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.753+0000 7f9c6a5ee640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9c64113660 con 0x7f9c64106650 2024-07-26T13:46:33.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.754+0000 7f9c63fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c64101a20 0x7f9c64199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:33.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.754+0000 7f9c63fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c64101a20 0x7f9c64199ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57080/0 (socket says 172.21.15.100:57080) 2024-07-26T13:46:33.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.754+0000 7f9c63fff640 1 -- 172.21.15.100:0/2977108547 learned_addr learned my addr 172.21.15.100:0/2977108547 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:33.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.754+0000 7f9c5bfff640 1 --2- 172.21.15.100:0/2977108547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c64105d00 0x7f9c6419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:33.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.754+0000 7f9c68b8b640 1 --2- 172.21.15.100:0/2977108547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c64106650 0x7f9c6419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:33.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.754+0000 7f9c63fff640 1 -- 172.21.15.100:0/2977108547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c64106650 msgr2=0x7f9c6419f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:33.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.754+0000 7f9c63fff640 1 --2- 172.21.15.100:0/2977108547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c64106650 0x7f9c6419f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:33.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.754+0000 7f9c63fff640 1 -- 172.21.15.100:0/2977108547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c64105d00 msgr2=0x7f9c6419a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:33.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.754+0000 7f9c63fff640 1 --2- 172.21.15.100:0/2977108547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c64105d00 0x7f9c6419a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:33.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.754+0000 7f9c63fff640 1 -- 172.21.15.100:0/2977108547 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c6419fe60 con 0x7f9c64101a20 2024-07-26T13:46:33.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.754+0000 7f9c5bfff640 1 --2- 172.21.15.100:0/2977108547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c64105d00 0x7f9c6419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:46:33.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.754+0000 7f9c63fff640 1 --2- 172.21.15.100:0/2977108547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c64101a20 0x7f9c64199ef0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f9c5400d770 tx=0x7f9c5400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:33.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.754+0000 7f9c61ffb640 1 -- 172.21.15.100:0/2977108547 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c54017070 con 0x7f9c64101a20 2024-07-26T13:46:33.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.754+0000 7f9c6a5ee640 1 -- 172.21.15.100:0/2977108547 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c641a0150 con 0x7f9c64101a20 2024-07-26T13:46:33.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.754+0000 7f9c61ffb640 1 -- 172.21.15.100:0/2977108547 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c540054d0 con 0x7f9c64101a20 2024-07-26T13:46:33.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.754+0000 7f9c61ffb640 1 -- 172.21.15.100:0/2977108547 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c54004c30 con 0x7f9c64101a20 2024-07-26T13:46:33.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.754+0000 7f9c6a5ee640 1 -- 172.21.15.100:0/2977108547 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c641a68b0 con 0x7f9c64101a20 2024-07-26T13:46:33.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.757+0000 7f9c70b72740 1 Processor -- start 2024-07-26T13:46:33.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.757+0000 7f9c70b72740 1 -- start start 2024-07-26T13:46:33.759 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:33 smithi100 ceph-mon[32599]: pgmap v560: 97 pgs: 97 active+clean; 87 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 662 KiB/s wr, 56 op/s 2024-07-26T13:46:33.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.758+0000 7f9c70b72740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c2934f520 0x557c296f5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:33.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.758+0000 7f9c70b72740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c29326660 0x557c296f55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:33.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.758+0000 7f9c70b72740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c2932e840 0x557c296f5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:33.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.758+0000 7f9c70b72740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x557c2966ae70 con 0x557c29326660 2024-07-26T13:46:33.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.758+0000 7f9c70b72740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557c2965a370 con 0x557c2934f520 2024-07-26T13:46:33.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.758+0000 7f9c70b72740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557c2965e3b0 con 0x557c2932e840 2024-07-26T13:46:33.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.758+0000 7f9c61ffb640 1 -- 172.21.15.100:0/2977108547 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f9c54010460 con 0x7f9c64101a20 2024-07-26T13:46:33.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.758+0000 7f9c68b8b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c2934f520 0x557c296f5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:33.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.758+0000 7f9c68b8b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c2934f520 0x557c296f5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:60148/0 (socket says 172.21.15.100:60148) 2024-07-26T13:46:33.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.758+0000 7f9c68b8b640 1 -- 172.21.15.100:0/1039217311 learned_addr learned my addr 172.21.15.100:0/1039217311 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:33.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.758+0000 7f9c5bfff640 1 --2- 172.21.15.100:0/1039217311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c2932e840 0x557c296f5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:33.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.758+0000 7f9c63fff640 1 --2- 172.21.15.100:0/1039217311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c29326660 0x557c296f55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:33.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.758+0000 7f9c68b8b640 1 -- 172.21.15.100:0/1039217311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c2932e840 msgr2=0x557c296f5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:33.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.758+0000 7f9c68b8b640 1 --2- 172.21.15.100:0/1039217311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c2932e840 0x557c296f5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:33.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.759+0000 7f9c68b8b640 1 -- 172.21.15.100:0/1039217311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c29326660 msgr2=0x557c296f55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:33.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.759+0000 7f9c68b8b640 1 --2- 172.21.15.100:0/1039217311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c29326660 0x557c296f55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:33.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.759+0000 7f9c68b8b640 1 -- 172.21.15.100:0/1039217311 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c29627040 con 0x557c2934f520 2024-07-26T13:46:33.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.759+0000 7f9c5bfff640 1 --2- 172.21.15.100:0/1039217311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c2932e840 0x557c296f5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:46:33.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.759+0000 7f9c68b8b640 1 --2- 172.21.15.100:0/1039217311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c2934f520 0x557c296f5070 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c008dc0 tx=0x7f9c5c00e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=335117af54905f2b server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:33.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.759+0000 7f9c5a7fc640 1 -- 172.21.15.100:0/1039217311 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c014070 con 0x557c2934f520 2024-07-26T13:46:33.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.759+0000 7f9c5a7fc640 1 -- 172.21.15.100:0/1039217311 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c5c003b70 con 0x557c2934f520 2024-07-26T13:46:33.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.759+0000 7f9c5a7fc640 1 -- 172.21.15.100:0/1039217311 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c0107b0 con 0x557c2934f520 2024-07-26T13:46:33.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.759+0000 7f9c61ffb640 1 --2- 172.21.15.100:0/2977108547 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9c500b3d40 0x7f9c500b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:33.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.759+0000 7f9c63fff640 1 --2- 172.21.15.100:0/2977108547 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9c500b3d40 0x7f9c500b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:33.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.760+0000 7f9c61ffb640 1 -- 172.21.15.100:0/2977108547 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7f9c540e3d40 con 0x7f9c64101a20 2024-07-26T13:46:33.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.760+0000 7f9c70b72740 1 -- 172.21.15.100:0/1039217311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c2934f520 msgr2=0x557c296f5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:33.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.760+0000 7f9c63fff640 1 --2- 172.21.15.100:0/2977108547 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9c500b3d40 0x7f9c500b6250 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x557c296f3cf0 tx=0x7f9c54092040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:33.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.760+0000 7f9c70b72740 1 --2- 172.21.15.100:0/1039217311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c2934f520 0x557c296f5070 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c008dc0 tx=0x7f9c5c00e720 comp rx=0 tx=0).stop 2024-07-26T13:46:33.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.760+0000 7f9c70b72740 1 -- 172.21.15.100:0/1039217311 shutdown_connections 2024-07-26T13:46:33.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.760+0000 7f9c70b72740 1 --2- 172.21.15.100:0/1039217311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c2932e840 0x557c296f5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:33.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.760+0000 7f9c70b72740 1 --2- 172.21.15.100:0/1039217311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c29326660 0x557c296f55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:33.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.760+0000 7f9c70b72740 1 --2- 172.21.15.100:0/1039217311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c2934f520 0x557c296f5070 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:33.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.760+0000 7f9c70b72740 1 -- 172.21.15.100:0/1039217311 >> 172.21.15.100:0/1039217311 conn(0x557c29343910 msgr2=0x557c296f2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:33.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.760+0000 7f9c70b72740 1 -- 172.21.15.100:0/1039217311 shutdown_connections 2024-07-26T13:46:33.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.760+0000 7f9c70b72740 1 -- 172.21.15.100:0/1039217311 wait complete. 2024-07-26T13:46:33.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.761+0000 7f9c70b72740 1 -- start start 2024-07-26T13:46:33.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.761+0000 7f9c70b72740 1 Processor -- start 2024-07-26T13:46:33.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.761+0000 7f9c70b72740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c29326660 0x557c29700c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:33.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.761+0000 7f9c70b72740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c2932e840 0x557c29701190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:33.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.761+0000 7f9c70b72740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c2934f520 0x557c2970cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:33.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.761+0000 7f9c70b72740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x557c294bdbb0 con 0x557c29326660 2024-07-26T13:46:33.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.761+0000 7f9c70b72740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557c295fcac0 con 0x557c2934f520 2024-07-26T13:46:33.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.761+0000 7f9c70b72740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557c29703850 con 0x557c2932e840 2024-07-26T13:46:33.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.761+0000 7f9c70b72740 10 client.0 _ll_register_callbacks cb 0x557c296053b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:46:33.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.761+0000 7f9c68b8b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c29326660 0x557c29700c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:33.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.762+0000 7f9c68b8b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c29326660 0x557c29700c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57092/0 (socket says 172.21.15.100:57092) 2024-07-26T13:46:33.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.762+0000 7f9c68b8b640 1 -- 172.21.15.100:0/2761539128 learned_addr learned my addr 172.21.15.100:0/2761539128 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:33.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.762+0000 7f9c5bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c2932e840 0x557c29701190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:33.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.762+0000 7f9c68b8b640 1 --2- 172.21.15.100:0/2761539128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c2934f520 0x557c2970cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:33.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.762+0000 7f9c68b8b640 1 -- 172.21.15.100:0/2761539128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c2932e840 msgr2=0x557c29701190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:33.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.762+0000 7f9c68b8b640 1 --2- 172.21.15.100:0/2761539128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c2932e840 0x557c29701190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:33.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.762+0000 7f9c68b8b640 1 -- 172.21.15.100:0/2761539128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c2934f520 msgr2=0x557c2970cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:33.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.762+0000 7f9c68b8b640 1 --2- 172.21.15.100:0/2761539128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c2934f520 0x557c2970cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:33.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.762+0000 7f9c68b8b640 1 -- 172.21.15.100:0/2761539128 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c293bab00 con 0x557c29326660 2024-07-26T13:46:33.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.762+0000 7f9c68b8b640 1 --2- 172.21.15.100:0/2761539128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c29326660 0x557c29700c50 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x557c29703420 tx=0x7f9c5c005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ce64b93933fa89d server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:33.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.762+0000 7f9c437fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:46:33.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.762+0000 7f9c437fe640 1 -- 172.21.15.100:0/2761539128 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c014070 con 0x557c29326660 2024-07-26T13:46:33.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.762+0000 7f9c437fe640 1 -- 172.21.15.100:0/2761539128 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7f9c5c010cf0 con 0x557c29326660 2024-07-26T13:46:33.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.762+0000 7f9c70b72740 1 -- 172.21.15.100:0/2761539128 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c5c003b70 con 0x557c29326660 2024-07-26T13:46:33.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.762+0000 7f9c437fe640 1 -- 172.21.15.100:0/2761539128 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f9c5c004a40 con 0x557c29326660 2024-07-26T13:46:33.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.762+0000 7f9c70b72740 10 client.17142 fetch_fsmap learned FSMap version 53 2024-07-26T13:46:33.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.762+0000 7f9c70b72740 1 -- 172.21.15.100:0/2761539128 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x557c29431960 con 0x557c29326660 2024-07-26T13:46:33.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.763+0000 7f9c437fe640 1 -- 172.21.15.100:0/2761539128 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c5c003b70 con 0x557c29326660 2024-07-26T13:46:33.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.763+0000 7f9c437fe640 1 -- 172.21.15.100:0/2761539128 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7f9c5c02c020 con 0x557c29326660 2024-07-26T13:46:33.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.763+0000 7f9c70b72740 10 client.17142 fetch_fsmap finished waiting for FSMap version 53 2024-07-26T13:46:33.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.763+0000 7f9c70b72740 10 client.17142 resolve_mds: validated gid 24607 aka daemon mds.i 2024-07-26T13:46:33.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.763+0000 7f9c70b72740 20 client.17142 populate_metadata read hostname 'smithi100' 2024-07-26T13:46:33.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.763+0000 7f9c70b72740 1 --2- 172.21.15.100:0/2761539128 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x557c2930dea0 0x557c297129f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:33.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.763+0000 7f9c70b72740 4 client.17142 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:46:33.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.763+0000 7f9c70b72740 1 -- 172.21.15.100:0/2761539128 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c2966cdc0 con 0x557c2930dea0 2024-07-26T13:46:33.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.763+0000 7f9c5bfff640 1 --2- 172.21.15.100:0/2761539128 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x557c2930dea0 0x557c297129f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:33.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.764+0000 7f9c437fe640 10 client.17142 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:33.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.764+0000 7f9c5bfff640 1 --2- 172.21.15.100:0/2761539128 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x557c2930dea0 0x557c297129f0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f36944693a8aa33 server_cookie=3c4a7f798cc4c6c3 in_seq=0 out_seq=0 2024-07-26T13:46:33.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.765+0000 7f9c437fe640 1 -- 172.21.15.100:0/2761539128 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557c2966cdc0 con 0x557c2930dea0 2024-07-26T13:46:33.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.765+0000 7f9c437fe640 10 client.17142 handle_command_reply: tid=0 2024-07-26T13:46:33.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.765+0000 7f9c70b72740 1 client.17142 shutdown 2024-07-26T13:46:33.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.765+0000 7f9c70b72740 1 -- 172.21.15.100:0/2761539128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c29326660 msgr2=0x557c29700c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:33.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.765+0000 7f9c70b72740 1 --2- 172.21.15.100:0/2761539128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c29326660 0x557c29700c50 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x557c29703420 tx=0x7f9c5c005f60 comp rx=0 tx=0).stop 2024-07-26T13:46:33.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.765+0000 7f9c70b72740 1 -- 172.21.15.100:0/2761539128 shutdown_connections 2024-07-26T13:46:33.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.765+0000 7f9c70b72740 1 --2- 172.21.15.100:0/2761539128 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x557c2930dea0 0x557c297129f0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:33.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.765+0000 7f9c70b72740 1 --2- 172.21.15.100:0/2761539128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c2934f520 0x557c2970cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:33.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.765+0000 7f9c70b72740 1 --2- 172.21.15.100:0/2761539128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c2932e840 0x557c29701190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:33.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.765+0000 7f9c70b72740 1 --2- 172.21.15.100:0/2761539128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c29326660 0x557c29700c50 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:33.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.765+0000 7f9c70b72740 1 -- 172.21.15.100:0/2761539128 >> 172.21.15.100:0/2761539128 conn(0x557c29343910 msgr2=0x557c296f8200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:33.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.765+0000 7f9c437fe640 0 client.17142 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:33.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.765+0000 7f9c70b72740 1 -- 172.21.15.100:0/2761539128 shutdown_connections 2024-07-26T13:46:33.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.765+0000 7f9c70b72740 1 -- 172.21.15.100:0/2761539128 wait complete. 2024-07-26T13:46:33.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.765+0000 7f9c70b72740 20 client.17142 trim_cache size 0 max 16384 2024-07-26T13:46:33.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.782+0000 7f9c70b72740 1 Processor -- start 2024-07-26T13:46:33.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.782+0000 7f9c70b72740 1 -- start start 2024-07-26T13:46:33.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.782+0000 7f9c70b72740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c29326660 0x557c296f1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:33.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.782+0000 7f9c70b72740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c2932e840 0x557c2970c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:33.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.782+0000 7f9c70b72740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c2934f520 0x557c2970cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:33.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.782+0000 7f9c70b72740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9c641b1b70 con 0x557c29326660 2024-07-26T13:46:33.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.782+0000 7f9c70b72740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9c641af440 con 0x557c2932e840 2024-07-26T13:46:33.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.782+0000 7f9c70b72740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9c641ad290 con 0x557c2934f520 2024-07-26T13:46:33.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.782+0000 7f9c68b8b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c29326660 0x557c296f1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:33.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.782+0000 7f9c68b8b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c29326660 0x557c296f1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57094/0 (socket says 172.21.15.100:57094) 2024-07-26T13:46:33.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.783+0000 7f9c68b8b640 1 -- 172.21.15.100:0/2122050136 learned_addr learned my addr 172.21.15.100:0/2122050136 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:33.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.783+0000 7f9c5bfff640 1 --2- 172.21.15.100:0/2122050136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c2932e840 0x557c2970c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:33.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.783+0000 7f9c68b8b640 1 -- 172.21.15.100:0/2122050136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c2934f520 msgr2=0x557c2970cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:33.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.783+0000 7f9c68b8b640 1 --2- 172.21.15.100:0/2122050136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c2934f520 0x557c2970cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:33.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.783+0000 7f9c68b8b640 1 -- 172.21.15.100:0/2122050136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c2932e840 msgr2=0x557c2970c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:33.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.783+0000 7f9c68b8b640 1 --2- 172.21.15.100:0/2122050136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c2932e840 0x557c2970c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:33.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.783+0000 7f9c68b8b640 1 -- 172.21.15.100:0/2122050136 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c296f7590 con 0x557c29326660 2024-07-26T13:46:33.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.783+0000 7f9c5bfff640 1 --2- 172.21.15.100:0/2122050136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c2932e840 0x557c2970c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:46:33.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.783+0000 7f9c68b8b640 1 --2- 172.21.15.100:0/2122050136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c29326660 0x557c296f1c80 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c005e10 tx=0x7f9c5c0108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209fda8fbb0d40be server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:33.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.783+0000 7f9c59ffb640 1 -- 172.21.15.100:0/2122050136 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c014070 con 0x557c29326660 2024-07-26T13:46:33.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.783+0000 7f9c59ffb640 1 -- 172.21.15.100:0/2122050136 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c5c010a30 con 0x557c29326660 2024-07-26T13:46:33.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.783+0000 7f9c59ffb640 1 -- 172.21.15.100:0/2122050136 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c0047b0 con 0x557c29326660 2024-07-26T13:46:33.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.784+0000 7f9c70b72740 1 -- 172.21.15.100:0/2122050136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c29326660 msgr2=0x557c296f1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:33.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.784+0000 7f9c70b72740 1 --2- 172.21.15.100:0/2122050136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c29326660 0x557c296f1c80 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c005e10 tx=0x7f9c5c0108b0 comp rx=0 tx=0).stop 2024-07-26T13:46:33.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.784+0000 7f9c70b72740 1 -- 172.21.15.100:0/2122050136 shutdown_connections 2024-07-26T13:46:33.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.784+0000 7f9c70b72740 1 --2- 172.21.15.100:0/2122050136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c2934f520 0x557c2970cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:33.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.784+0000 7f9c70b72740 1 --2- 172.21.15.100:0/2122050136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c2932e840 0x557c2970c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:33.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.784+0000 7f9c70b72740 1 --2- 172.21.15.100:0/2122050136 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c29326660 0x557c296f1c80 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:33.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.784+0000 7f9c70b72740 1 -- 172.21.15.100:0/2122050136 >> 172.21.15.100:0/2122050136 conn(0x557c29343910 msgr2=0x557c296f1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:33.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.784+0000 7f9c70b72740 1 -- 172.21.15.100:0/2122050136 shutdown_connections 2024-07-26T13:46:33.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.784+0000 7f9c70b72740 1 -- 172.21.15.100:0/2122050136 wait complete. 2024-07-26T13:46:33.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.785+0000 7f9c70b72740 1 -- start start 2024-07-26T13:46:33.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.785+0000 7f9c70b72740 1 Processor -- start 2024-07-26T13:46:33.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.785+0000 7f9c70b72740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c29326660 0x557c29709130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:33.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.785+0000 7f9c70b72740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c2932e840 0x557c297045e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:33.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.785+0000 7f9c70b72740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c2934f520 0x557c29704b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:33.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.785+0000 7f9c70b72740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9c641ad900 con 0x557c2932e840 2024-07-26T13:46:33.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.785+0000 7f9c70b72740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9c641aed70 con 0x557c2934f520 2024-07-26T13:46:33.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.785+0000 7f9c70b72740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9c641adfd0 con 0x557c29326660 2024-07-26T13:46:33.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.785+0000 7f9c70b72740 10 client.0 _ll_register_callbacks cb 0x7f9c5c008c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:46:33.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.785+0000 7f9c5bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c2932e840 0x557c297045e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:33.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.785+0000 7f9c5bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c2932e840 0x557c297045e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57106/0 (socket says 172.21.15.100:57106) 2024-07-26T13:46:33.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.785+0000 7f9c5bfff640 1 -- 172.21.15.100:0/3037906680 learned_addr learned my addr 172.21.15.100:0/3037906680 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:33.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.785+0000 7f9c68b8b640 1 --2- 172.21.15.100:0/3037906680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c2934f520 0x557c29704b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:33.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.785+0000 7f9c68b8b640 1 --2- 172.21.15.100:0/3037906680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c29326660 0x557c29709130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:33.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.786+0000 7f9c5bfff640 1 -- 172.21.15.100:0/3037906680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c29326660 msgr2=0x557c29709130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:33.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.786+0000 7f9c5bfff640 1 --2- 172.21.15.100:0/3037906680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c29326660 0x557c29709130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:33.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.786+0000 7f9c5bfff640 1 -- 172.21.15.100:0/3037906680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c2934f520 msgr2=0x557c29704b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:33.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.786+0000 7f9c5bfff640 1 --2- 172.21.15.100:0/3037906680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c2934f520 0x557c29704b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:33.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.786+0000 7f9c5bfff640 1 -- 172.21.15.100:0/3037906680 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c29705230 con 0x557c2932e840 2024-07-26T13:46:33.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.786+0000 7f9c5bfff640 1 --2- 172.21.15.100:0/3037906680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c2932e840 0x557c297045e0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c009850 tx=0x7f9c4c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57006dfb5e68dd0a server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:33.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.786+0000 7f9c437fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:46:33.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.786+0000 7f9c437fe640 1 -- 172.21.15.100:0/3037906680 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c048070 con 0x557c2932e840 2024-07-26T13:46:33.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.786+0000 7f9c70b72740 1 -- 172.21.15.100:0/3037906680 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c5c010a30 con 0x557c2932e840 2024-07-26T13:46:33.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.786+0000 7f9c437fe640 1 -- 172.21.15.100:0/3037906680 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7f9c4c005040 con 0x557c2932e840 2024-07-26T13:46:33.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.786+0000 7f9c437fe640 1 -- 172.21.15.100:0/3037906680 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f9c4c035040 con 0x557c2932e840 2024-07-26T13:46:33.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.786+0000 7f9c70b72740 10 client.17148 fetch_fsmap learned FSMap version 53 2024-07-26T13:46:33.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.786+0000 7f9c70b72740 1 -- 172.21.15.100:0/3037906680 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x557c2977af50 con 0x557c2932e840 2024-07-26T13:46:33.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.786+0000 7f9c437fe640 1 -- 172.21.15.100:0/3037906680 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c5c010a30 con 0x557c2932e840 2024-07-26T13:46:33.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.787+0000 7f9c437fe640 1 -- 172.21.15.100:0/3037906680 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7f9c4c04b020 con 0x557c2932e840 2024-07-26T13:46:33.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.787+0000 7f9c70b72740 10 client.17148 fetch_fsmap finished waiting for FSMap version 53 2024-07-26T13:46:33.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.787+0000 7f9c70b72740 10 client.17148 resolve_mds: validated gid 24607 aka daemon mds.i 2024-07-26T13:46:33.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.787+0000 7f9c70b72740 20 client.17148 populate_metadata read hostname 'smithi100' 2024-07-26T13:46:33.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.787+0000 7f9c70b72740 1 --2- 172.21.15.100:0/3037906680 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x557c292e3250 0x557c2977d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:33.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.787+0000 7f9c70b72740 4 client.17148 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "cd8d97d4", "release": true, "await": true}] 2024-07-26T13:46:33.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.787+0000 7f9c70b72740 1 -- 172.21.15.100:0/3037906680 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "quiesce db", "set_id": "cd8d97d4", "release": true, "await": true}) -- 0x557c29668860 con 0x557c292e3250 2024-07-26T13:46:33.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.787+0000 7f9c68b8b640 1 --2- 172.21.15.100:0/3037906680 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x557c292e3250 0x557c2977d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:33.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.787+0000 7f9c68b8b640 1 --2- 172.21.15.100:0/3037906680 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x557c292e3250 0x557c2977d760 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=918984f070c9dfd9 server_cookie=a6ec405a253b606d in_seq=0 out_seq=0 2024-07-26T13:46:33.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.787+0000 7f9c437fe640 10 client.17148 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:33.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.798+0000 7f9c437fe640 1 -- 172.21.15.100:0/3037906680 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557c29668860 con 0x557c292e3250 2024-07-26T13:46:33.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.798+0000 7f9c437fe640 10 client.17148 handle_command_reply: tid=0 2024-07-26T13:46:33.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.798+0000 7f9c70b72740 1 client.17148 shutdown 2024-07-26T13:46:33.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.798+0000 7f9c70b72740 1 -- 172.21.15.100:0/3037906680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c2932e840 msgr2=0x557c297045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:33.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.798+0000 7f9c70b72740 1 --2- 172.21.15.100:0/3037906680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c2932e840 0x557c297045e0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c009850 tx=0x7f9c4c004040 comp rx=0 tx=0).stop 2024-07-26T13:46:33.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.799+0000 7f9c70b72740 1 -- 172.21.15.100:0/3037906680 shutdown_connections 2024-07-26T13:46:33.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.799+0000 7f9c70b72740 1 --2- 172.21.15.100:0/3037906680 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x557c292e3250 0x557c2977d760 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:33.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.799+0000 7f9c70b72740 1 --2- 172.21.15.100:0/3037906680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c2934f520 0x557c29704b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:33.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.799+0000 7f9c437fe640 0 client.17148 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:33.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.799+0000 7f9c70b72740 1 --2- 172.21.15.100:0/3037906680 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x557c2932e840 0x557c297045e0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:33.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.799+0000 7f9c70b72740 1 --2- 172.21.15.100:0/3037906680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c29326660 0x557c29709130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:33.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.799+0000 7f9c70b72740 1 -- 172.21.15.100:0/3037906680 >> 172.21.15.100:0/3037906680 conn(0x557c29343910 msgr2=0x557c296f1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:33.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.799+0000 7f9c70b72740 1 -- 172.21.15.100:0/3037906680 shutdown_connections 2024-07-26T13:46:33.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.799+0000 7f9c70b72740 1 -- 172.21.15.100:0/3037906680 wait complete. 2024-07-26T13:46:33.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.799+0000 7f9c70b72740 20 client.17148 trim_cache size 0 max 16384 2024-07-26T13:46:33.801 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:46:33.801 INFO:teuthology.orchestra.run.smithi100.stdout: "epoch": 53, 2024-07-26T13:46:33.801 INFO:teuthology.orchestra.run.smithi100.stdout: "leader": 24607, 2024-07-26T13:46:33.801 INFO:teuthology.orchestra.run.smithi100.stdout: "set_version": 22, 2024-07-26T13:46:33.801 INFO:teuthology.orchestra.run.smithi100.stdout: "sets": { 2024-07-26T13:46:33.802 INFO:teuthology.orchestra.run.smithi100.stdout: "cd8d97d4": { 2024-07-26T13:46:33.802 INFO:teuthology.orchestra.run.smithi100.stdout: "version": 22, 2024-07-26T13:46:33.802 INFO:teuthology.orchestra.run.smithi100.stdout: "age_ref": 0.0, 2024-07-26T13:46:33.802 INFO:teuthology.orchestra.run.smithi100.stdout: "state": { 2024-07-26T13:46:33.802 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RELEASED", 2024-07-26T13:46:33.802 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 0.0 2024-07-26T13:46:33.802 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-07-26T13:46:33.802 INFO:teuthology.orchestra.run.smithi100.stdout: "timeout": 90.0, 2024-07-26T13:46:33.802 INFO:teuthology.orchestra.run.smithi100.stdout: "expiration": 141.9, 2024-07-26T13:46:33.802 INFO:teuthology.orchestra.run.smithi100.stdout: "members": { 2024-07-26T13:46:33.802 INFO:teuthology.orchestra.run.smithi100.stdout: "file:/": { 2024-07-26T13:46:33.802 INFO:teuthology.orchestra.run.smithi100.stdout: "excluded": false, 2024-07-26T13:46:33.802 INFO:teuthology.orchestra.run.smithi100.stdout: "state": { 2024-07-26T13:46:33.803 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "RELEASED", 2024-07-26T13:46:33.803 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 0.0 2024-07-26T13:46:33.803 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:46:33.803 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:46:33.803 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:46:33.803 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:46:33.803 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:46:33.803 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:46:33.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.799+0000 7f9c6a5ee640 1 -- 172.21.15.100:0/2977108547 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9c500b3d40 msgr2=0x7f9c500b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:33.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.799+0000 7f9c6a5ee640 1 --2- 172.21.15.100:0/2977108547 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9c500b3d40 0x7f9c500b6250 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x557c296f3cf0 tx=0x7f9c54092040 comp rx=0 tx=0).stop 2024-07-26T13:46:33.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.799+0000 7f9c6a5ee640 1 -- 172.21.15.100:0/2977108547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c64101a20 msgr2=0x7f9c64199ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:33.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.799+0000 7f9c6a5ee640 1 --2- 172.21.15.100:0/2977108547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c64101a20 0x7f9c64199ef0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f9c5400d770 tx=0x7f9c5400dc30 comp rx=0 tx=0).stop 2024-07-26T13:46:33.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.799+0000 7f9c6a5ee640 1 -- 172.21.15.100:0/2977108547 shutdown_connections 2024-07-26T13:46:33.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.799+0000 7f9c6a5ee640 1 --2- 172.21.15.100:0/2977108547 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9c500b3d40 0x7f9c500b6250 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:33.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.799+0000 7f9c6a5ee640 1 --2- 172.21.15.100:0/2977108547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c64106650 0x7f9c6419f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:33.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.799+0000 7f9c6a5ee640 1 --2- 172.21.15.100:0/2977108547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c64105d00 0x7f9c6419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:33.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.799+0000 7f9c6a5ee640 1 --2- 172.21.15.100:0/2977108547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c64101a20 0x7f9c64199ef0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:33.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.799+0000 7f9c6a5ee640 1 -- 172.21.15.100:0/2977108547 >> 172.21.15.100:0/2977108547 conn(0x7f9c64079860 msgr2=0x7f9c64107010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:33.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.800+0000 7f9c6a5ee640 1 -- 172.21.15.100:0/2977108547 shutdown_connections 2024-07-26T13:46:33.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:33.800+0000 7f9c6a5ee640 1 -- 172.21.15.100:0/2977108547 wait complete. 2024-07-26T13:46:33.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:33 smithi138 ceph-mon[36954]: pgmap v560: 97 pgs: 97 active+clean; 87 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 662 KiB/s wr, 56 op/s 2024-07-26T13:46:34.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:33 smithi167 ceph-mon[37140]: pgmap v560: 97 pgs: 97 active+clean; 87 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 662 KiB/s wr, 56 op/s 2024-07-26T13:46:34.422 INFO:tasks.workunit.client.0.smithi100.stdout: 1 236351 2.88 MB/sec execute 168 sec latency 18195.576 ms 2024-07-26T13:46:34.550 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'cd8d97d4', total seconds elapsed: 27.7 2024-07-26T13:46:34.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.911+0000 7f123c806640 1 -- 172.21.15.100:0/1500169647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1238108660 msgr2=0x7f123810ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:34.912 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.911+0000 7f123c806640 1 --2- 172.21.15.100:0/1500169647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1238108660 0x7f123810ee90 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f1228009900 tx=0x7f122802a410 comp rx=0 tx=0).stop 2024-07-26T13:46:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.911+0000 7f123c806640 1 -- 172.21.15.100:0/1500169647 shutdown_connections 2024-07-26T13:46:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.911+0000 7f123c806640 1 --2- 172.21.15.100:0/1500169647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1238108660 0x7f123810ee90 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.911+0000 7f123c806640 1 --2- 172.21.15.100:0/1500169647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1238107d10 0x7f1238108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.911+0000 7f123c806640 1 --2- 172.21.15.100:0/1500169647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12381023e0 0x7f12381027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.911+0000 7f123c806640 1 -- 172.21.15.100:0/1500169647 >> 172.21.15.100:0/1500169647 conn(0x7f123807a170 msgr2=0x7f123807a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.911+0000 7f123c806640 1 -- 172.21.15.100:0/1500169647 shutdown_connections 2024-07-26T13:46:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.911+0000 7f123c806640 1 -- 172.21.15.100:0/1500169647 wait complete. 2024-07-26T13:46:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.912+0000 7f123c806640 1 Processor -- start 2024-07-26T13:46:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.912+0000 7f123c806640 1 -- start start 2024-07-26T13:46:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.912+0000 7f123c806640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12381023e0 0x7f1238198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.912+0000 7f123c806640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1238107d10 0x7f1238199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.912+0000 7f123c806640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1238108660 0x7f123819e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.912+0000 7f123c806640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1238112330 con 0x7f1238108660 2024-07-26T13:46:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.912+0000 7f123c806640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f12381121b0 con 0x7f1238107d10 2024-07-26T13:46:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.912+0000 7f123c806640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f12381124b0 con 0x7f12381023e0 2024-07-26T13:46:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.912+0000 7f1236d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1238108660 0x7f123819e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.912+0000 7f1236d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1238108660 0x7f123819e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57128/0 (socket says 172.21.15.100:57128) 2024-07-26T13:46:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.912+0000 7f123659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12381023e0 0x7f1238198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.912+0000 7f1236d9d640 1 -- 172.21.15.100:0/3200331603 learned_addr learned my addr 172.21.15.100:0/3200331603 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.912+0000 7f1235d9b640 1 --2- 172.21.15.100:0/3200331603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1238107d10 0x7f1238199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.913+0000 7f1236d9d640 1 -- 172.21.15.100:0/3200331603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12381023e0 msgr2=0x7f1238198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.913+0000 7f1236d9d640 1 --2- 172.21.15.100:0/3200331603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12381023e0 0x7f1238198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.913+0000 7f1236d9d640 1 -- 172.21.15.100:0/3200331603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1238107d10 msgr2=0x7f1238199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.913+0000 7f1236d9d640 1 --2- 172.21.15.100:0/3200331603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1238107d10 0x7f1238199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.913+0000 7f1236d9d640 1 -- 172.21.15.100:0/3200331603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f123819eca0 con 0x7f1238108660 2024-07-26T13:46:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.913+0000 7f1236d9d640 1 --2- 172.21.15.100:0/3200331603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1238108660 0x7f123819e5a0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f1228004770 tx=0x7f1228004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.913+0000 7f12277fe640 1 -- 172.21.15.100:0/3200331603 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122803f030 con 0x7f1238108660 2024-07-26T13:46:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.913+0000 7f12277fe640 1 -- 172.21.15.100:0/3200331603 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1228003aa0 con 0x7f1238108660 2024-07-26T13:46:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.913+0000 7f123c806640 1 -- 172.21.15.100:0/3200331603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f123819ef30 con 0x7f1238108660 2024-07-26T13:46:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.913+0000 7f12277fe640 1 -- 172.21.15.100:0/3200331603 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1228035070 con 0x7f1238108660 2024-07-26T13:46:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.913+0000 7f123c806640 1 -- 172.21.15.100:0/3200331603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12381a5720 con 0x7f1238108660 2024-07-26T13:46:34.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.915+0000 7f12437fc740 1 Processor -- start 2024-07-26T13:46:34.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.916+0000 7f12277fe640 1 -- 172.21.15.100:0/3200331603 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f1228042030 con 0x7f1238108660 2024-07-26T13:46:34.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.916+0000 7f12437fc740 1 -- start start 2024-07-26T13:46:34.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.916+0000 7f12437fc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558595ec1b0 0x55585998e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:34.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.916+0000 7f12437fc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558595c32f0 0x55585998ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:34.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.916+0000 7f12437fc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558595cb4d0 0x55585998f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:34.924 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.916+0000 7f12437fc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5558599043b0 con 0x5558595cb4d0 2024-07-26T13:46:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.916+0000 7f12437fc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5558598f38b0 con 0x5558595c32f0 2024-07-26T13:46:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.916+0000 7f12437fc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5558598f78f0 con 0x5558595ec1b0 2024-07-26T13:46:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.916+0000 7f1235d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558595ec1b0 0x55585998e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.916+0000 7f1235d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558595cb4d0 0x55585998f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.916+0000 7f123659c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558595c32f0 0x55585998ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.916+0000 7f1235d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558595cb4d0 0x55585998f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57134/0 (socket says 172.21.15.100:57134) 2024-07-26T13:46:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.916+0000 7f1235d9b640 1 -- 172.21.15.100:0/1981213931 learned_addr learned my addr 172.21.15.100:0/1981213931 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.917+0000 7f1235d9b640 1 -- 172.21.15.100:0/1981213931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558595ec1b0 msgr2=0x55585998e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.917+0000 7f1235d9b640 1 --2- 172.21.15.100:0/1981213931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558595ec1b0 0x55585998e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.917+0000 7f1235d9b640 1 -- 172.21.15.100:0/1981213931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558595c32f0 msgr2=0x55585998ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.917+0000 7f1235d9b640 1 --2- 172.21.15.100:0/1981213931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558595c32f0 0x55585998ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.917+0000 7f1235d9b640 1 -- 172.21.15.100:0/1981213931 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55585989a9a0 con 0x5558595cb4d0 2024-07-26T13:46:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.917+0000 7f123659c640 1 --2- 172.21.15.100:0/1981213931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558595c32f0 0x55585998ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:46:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.917+0000 7f12277fe640 1 --2- 172.21.15.100:0/3200331603 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f12200b3d40 0x7f12200b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.917+0000 7f123659c640 1 --2- 172.21.15.100:0/3200331603 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f12200b3d40 0x7f12200b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.917+0000 7f1235d9b640 1 --2- 172.21.15.100:0/1981213931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558595cb4d0 0x55585998f130 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f122c012a20 tx=0x7f122c009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4976d73c0c7f3d4c server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.917+0000 7f12277fe640 1 -- 172.21.15.100:0/3200331603 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7f12281046d0 con 0x7f1238108660 2024-07-26T13:46:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.917+0000 7f1224ff9640 1 -- 172.21.15.100:0/1981213931 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c012030 con 0x5558595cb4d0 2024-07-26T13:46:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.917+0000 7f1224ff9640 1 -- 172.21.15.100:0/1981213931 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f122c0121d0 con 0x5558595cb4d0 2024-07-26T13:46:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.917+0000 7f1224ff9640 1 -- 172.21.15.100:0/1981213931 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c005020 con 0x5558595cb4d0 2024-07-26T13:46:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.918+0000 7f12437fc740 1 -- 172.21.15.100:0/1981213931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558595cb4d0 msgr2=0x55585998f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.918+0000 7f12437fc740 1 --2- 172.21.15.100:0/1981213931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558595cb4d0 0x55585998f130 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f122c012a20 tx=0x7f122c009680 comp rx=0 tx=0).stop 2024-07-26T13:46:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.918+0000 7f123659c640 1 --2- 172.21.15.100:0/3200331603 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f12200b3d40 0x7f12200b6250 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x55585998d330 tx=0x7f1230005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.918+0000 7f12437fc740 1 -- 172.21.15.100:0/1981213931 shutdown_connections 2024-07-26T13:46:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.918+0000 7f12437fc740 1 --2- 172.21.15.100:0/1981213931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558595cb4d0 0x55585998f130 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.918+0000 7f12437fc740 1 --2- 172.21.15.100:0/1981213931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558595c32f0 0x55585998ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.918+0000 7f12437fc740 1 --2- 172.21.15.100:0/1981213931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558595ec1b0 0x55585998e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.918+0000 7f12437fc740 1 -- 172.21.15.100:0/1981213931 >> 172.21.15.100:0/1981213931 conn(0x5558595e0300 msgr2=0x55585998c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.918+0000 7f12437fc740 1 -- 172.21.15.100:0/1981213931 shutdown_connections 2024-07-26T13:46:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.918+0000 7f12437fc740 1 -- 172.21.15.100:0/1981213931 wait complete. 2024-07-26T13:46:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.919+0000 7f12437fc740 1 -- start start 2024-07-26T13:46:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.919+0000 7f12437fc740 1 Processor -- start 2024-07-26T13:46:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.919+0000 7f12437fc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558595c32f0 0x55585999a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.919+0000 7f12437fc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558595cb4d0 0x55585999a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.919+0000 7f12437fc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558595ec1b0 0x5558599a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.919+0000 7f12437fc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55585982dab0 con 0x5558595cb4d0 2024-07-26T13:46:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.919+0000 7f12437fc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55585987bcf0 con 0x5558595ec1b0 2024-07-26T13:46:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.919+0000 7f12437fc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55585999cdd0 con 0x5558595c32f0 2024-07-26T13:46:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.919+0000 7f12437fc740 10 client.0 _ll_register_callbacks cb 0x5558598ed3e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:46:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.919+0000 7f123659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558595c32f0 0x55585999a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.919+0000 7f123659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558595c32f0 0x55585999a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:42754/0 (socket says 172.21.15.100:42754) 2024-07-26T13:46:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.919+0000 7f123659c640 1 -- 172.21.15.100:0/1451354439 learned_addr learned my addr 172.21.15.100:0/1451354439 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.919+0000 7f1235d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558595cb4d0 0x55585999a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.919+0000 7f1236d9d640 1 --2- 172.21.15.100:0/1451354439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558595ec1b0 0x5558599a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.919+0000 7f123659c640 1 -- 172.21.15.100:0/1451354439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558595ec1b0 msgr2=0x5558599a60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.920+0000 7f123659c640 1 --2- 172.21.15.100:0/1451354439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558595ec1b0 0x5558599a60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.920+0000 7f123659c640 1 -- 172.21.15.100:0/1451354439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558595cb4d0 msgr2=0x55585999a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.920+0000 7f123659c640 1 --2- 172.21.15.100:0/1451354439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558595cb4d0 0x55585999a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.920+0000 7f123659c640 1 -- 172.21.15.100:0/1451354439 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555859657790 con 0x5558595c32f0 2024-07-26T13:46:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.920+0000 7f1235d9b640 1 --2- 172.21.15.100:0/1451354439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558595cb4d0 0x55585999a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:46:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.920+0000 7f1236d9d640 1 --2- 172.21.15.100:0/1451354439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558595ec1b0 0x5558599a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:46:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.920+0000 7f123659c640 1 --2- 172.21.15.100:0/1451354439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558595c32f0 0x55585999a250 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f12300045f0 tx=0x7f1230004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57855545872e96f6 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.920+0000 7f121dffb640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-07-26T13:46:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.920+0000 7f121dffb640 1 -- 172.21.15.100:0/1451354439 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123001f640 con 0x5558595c32f0 2024-07-26T13:46:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.920+0000 7f121dffb640 1 -- 172.21.15.100:0/1451354439 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7f123001f7e0 con 0x5558595c32f0 2024-07-26T13:46:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.920+0000 7f12437fc740 1 -- 172.21.15.100:0/1451354439 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f122c0121d0 con 0x5558595c32f0 2024-07-26T13:46:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.920+0000 7f121dffb640 1 -- 172.21.15.100:0/1451354439 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f123001f9e0 con 0x5558595c32f0 2024-07-26T13:46:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.920+0000 7f12437fc740 10 client.26563 fetch_fsmap learned FSMap version 53 2024-07-26T13:46:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.920+0000 7f12437fc740 1 -- 172.21.15.100:0/1451354439 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x5558596d0620 con 0x5558595c32f0 2024-07-26T13:46:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.920+0000 7f121dffb640 1 -- 172.21.15.100:0/1451354439 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f122c0121d0 con 0x5558595c32f0 2024-07-26T13:46:34.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.921+0000 7f121dffb640 1 -- 172.21.15.100:0/1451354439 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7f1230027070 con 0x5558595c32f0 2024-07-26T13:46:34.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.921+0000 7f12437fc740 10 client.26563 fetch_fsmap finished waiting for FSMap version 53 2024-07-26T13:46:34.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.921+0000 7f12437fc740 10 client.26563 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:46:34.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.921+0000 7f12437fc740 20 client.26563 populate_metadata read hostname 'smithi100' 2024-07-26T13:46:34.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.921+0000 7f12437fc740 1 --2- 172.21.15.100:0/1451354439 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5558595aad80 0x5558599ac190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:34.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.921+0000 7f12437fc740 4 client.26563 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:46:34.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.921+0000 7f12437fc740 1 -- 172.21.15.100:0/1451354439 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555859906300 con 0x5558595aad80 2024-07-26T13:46:34.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.921+0000 7f1235d9b640 1 --2- 172.21.15.100:0/1451354439 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5558595aad80 0x5558599ac190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:34.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.921+0000 7f121dffb640 10 client.26563 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:34.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.921+0000 7f1235d9b640 1 --2- 172.21.15.100:0/1451354439 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5558595aad80 0x5558599ac190 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bace2a537f5fb72 server_cookie=fad46f1a36468096 in_seq=0 out_seq=0 2024-07-26T13:46:34.930 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.922+0000 7f121dffb640 1 -- 172.21.15.100:0/1451354439 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555859906300 con 0x5558595aad80 2024-07-26T13:46:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.922+0000 7f121dffb640 10 client.26563 handle_command_reply: tid=0 2024-07-26T13:46:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.922+0000 7f12437fc740 1 client.26563 shutdown 2024-07-26T13:46:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.922+0000 7f12437fc740 1 -- 172.21.15.100:0/1451354439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558595c32f0 msgr2=0x55585999a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.922+0000 7f12437fc740 1 --2- 172.21.15.100:0/1451354439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558595c32f0 0x55585999a250 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f12300045f0 tx=0x7f1230004730 comp rx=0 tx=0).stop 2024-07-26T13:46:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.922+0000 7f12437fc740 1 -- 172.21.15.100:0/1451354439 shutdown_connections 2024-07-26T13:46:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.922+0000 7f12437fc740 1 --2- 172.21.15.100:0/1451354439 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5558595aad80 0x5558599ac190 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.922+0000 7f12437fc740 1 --2- 172.21.15.100:0/1451354439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558595ec1b0 0x5558599a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.922+0000 7f12437fc740 1 --2- 172.21.15.100:0/1451354439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558595cb4d0 0x55585999a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.922+0000 7f121dffb640 0 client.26563 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.922+0000 7f12437fc740 1 --2- 172.21.15.100:0/1451354439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558595c32f0 0x55585999a250 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.922+0000 7f12437fc740 1 -- 172.21.15.100:0/1451354439 >> 172.21.15.100:0/1451354439 conn(0x5558595e0300 msgr2=0x55585998f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.922+0000 7f12437fc740 1 -- 172.21.15.100:0/1451354439 shutdown_connections 2024-07-26T13:46:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.922+0000 7f12437fc740 1 -- 172.21.15.100:0/1451354439 wait complete. 2024-07-26T13:46:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.922+0000 7f12437fc740 20 client.26563 trim_cache size 0 max 16384 2024-07-26T13:46:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.937+0000 7f12437fc740 1 Processor -- start 2024-07-26T13:46:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.937+0000 7f12437fc740 1 -- start start 2024-07-26T13:46:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.938+0000 7f12437fc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558595c32f0 0x555859993e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.938+0000 7f12437fc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558595cb4d0 0x55585999a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.938+0000 7f12437fc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558595ec1b0 0x5558599a5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.938+0000 7f12437fc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f12381b0a40 con 0x5558595c32f0 2024-07-26T13:46:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.938+0000 7f12437fc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f12381ae310 con 0x5558595ec1b0 2024-07-26T13:46:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.938+0000 7f12437fc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f12381ac160 con 0x5558595cb4d0 2024-07-26T13:46:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.938+0000 7f123659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558595c32f0 0x555859993e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.938+0000 7f123659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558595c32f0 0x555859993e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57156/0 (socket says 172.21.15.100:57156) 2024-07-26T13:46:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.938+0000 7f123659c640 1 -- 172.21.15.100:0/3812460468 learned_addr learned my addr 172.21.15.100:0/3812460468 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.938+0000 7f1236d9d640 1 --2- 172.21.15.100:0/3812460468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558595ec1b0 0x5558599a5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.938+0000 7f1235d9b640 1 --2- 172.21.15.100:0/3812460468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558595cb4d0 0x55585999a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.938+0000 7f123659c640 1 -- 172.21.15.100:0/3812460468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558595cb4d0 msgr2=0x55585999a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.938+0000 7f123659c640 1 --2- 172.21.15.100:0/3812460468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558595cb4d0 0x55585999a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.938+0000 7f123659c640 1 -- 172.21.15.100:0/3812460468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558595ec1b0 msgr2=0x5558599a5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.938+0000 7f123659c640 1 --2- 172.21.15.100:0/3812460468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558595ec1b0 0x5558599a5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.938+0000 7f123659c640 1 -- 172.21.15.100:0/3812460468 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555859990be0 con 0x5558595c32f0 2024-07-26T13:46:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.938+0000 7f1235d9b640 1 --2- 172.21.15.100:0/3812460468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558595cb4d0 0x55585999a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:46:34.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.938+0000 7f123659c640 1 --2- 172.21.15.100:0/3812460468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558595c32f0 0x555859993e80 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f1230009d50 tx=0x7f1230004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa05d5b1420defc server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:34.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.938+0000 7f121ffff640 1 -- 172.21.15.100:0/3812460468 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123001f840 con 0x5558595c32f0 2024-07-26T13:46:34.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.938+0000 7f121ffff640 1 -- 172.21.15.100:0/3812460468 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f123001fb80 con 0x5558595c32f0 2024-07-26T13:46:34.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.938+0000 7f121ffff640 1 -- 172.21.15.100:0/3812460468 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123001d700 con 0x5558595c32f0 2024-07-26T13:46:34.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.939+0000 7f12437fc740 1 -- 172.21.15.100:0/3812460468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558595c32f0 msgr2=0x555859993e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:34.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.939+0000 7f12437fc740 1 --2- 172.21.15.100:0/3812460468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558595c32f0 0x555859993e80 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f1230009d50 tx=0x7f1230004620 comp rx=0 tx=0).stop 2024-07-26T13:46:34.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.939+0000 7f12437fc740 1 -- 172.21.15.100:0/3812460468 shutdown_connections 2024-07-26T13:46:34.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.939+0000 7f12437fc740 1 --2- 172.21.15.100:0/3812460468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558595ec1b0 0x5558599a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:34.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.939+0000 7f12437fc740 1 --2- 172.21.15.100:0/3812460468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558595cb4d0 0x55585999a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:34.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.939+0000 7f12437fc740 1 --2- 172.21.15.100:0/3812460468 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558595c32f0 0x555859993e80 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:34.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.939+0000 7f12437fc740 1 -- 172.21.15.100:0/3812460468 >> 172.21.15.100:0/3812460468 conn(0x5558595e0300 msgr2=0x555859993770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:34.942 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.939+0000 7f12437fc740 1 -- 172.21.15.100:0/3812460468 shutdown_connections 2024-07-26T13:46:34.943 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.939+0000 7f12437fc740 1 -- 172.21.15.100:0/3812460468 wait complete. 2024-07-26T13:46:34.943 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.939+0000 7f12437fc740 1 -- start start 2024-07-26T13:46:34.943 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.939+0000 7f12437fc740 1 Processor -- start 2024-07-26T13:46:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.940+0000 7f12437fc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558595c32f0 0x55585999e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.940+0000 7f12437fc740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558595cb4d0 0x55585999ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.940+0000 7f12437fc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558595ec1b0 0x555859a0ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.940+0000 7f12437fc740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f12381ac7d0 con 0x5558595cb4d0 2024-07-26T13:46:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.940+0000 7f12437fc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f12381adc40 con 0x5558595ec1b0 2024-07-26T13:46:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.940+0000 7f12437fc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f12381acea0 con 0x5558595c32f0 2024-07-26T13:46:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.940+0000 7f12437fc740 10 client.0 _ll_register_callbacks cb 0x7f1230009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:46:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.940+0000 7f1235d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558595cb4d0 0x55585999ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.940+0000 7f1235d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558595cb4d0 0x55585999ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57158/0 (socket says 172.21.15.100:57158) 2024-07-26T13:46:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.940+0000 7f1235d9b640 1 -- 172.21.15.100:0/502778724 learned_addr learned my addr 172.21.15.100:0/502778724 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.940+0000 7f123659c640 1 --2- 172.21.15.100:0/502778724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558595c32f0 0x55585999e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.940+0000 7f1236d9d640 1 --2- 172.21.15.100:0/502778724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558595ec1b0 0x555859a0ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.940+0000 7f1235d9b640 1 -- 172.21.15.100:0/502778724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558595c32f0 msgr2=0x55585999e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.940+0000 7f1235d9b640 1 --2- 172.21.15.100:0/502778724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558595c32f0 0x55585999e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.940+0000 7f1235d9b640 1 -- 172.21.15.100:0/502778724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558595ec1b0 msgr2=0x555859a0ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.940+0000 7f1235d9b640 1 --2- 172.21.15.100:0/502778724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558595ec1b0 0x555859a0ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.940+0000 7f1235d9b640 1 -- 172.21.15.100:0/502778724 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555859a0c0a0 con 0x5558595cb4d0 2024-07-26T13:46:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.940+0000 7f1235d9b640 1 --2- 172.21.15.100:0/502778724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558595cb4d0 0x55585999ef50 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f122c0129f0 tx=0x7f122c009a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e318944c39017d0c server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.940+0000 7f121dffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:46:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.940+0000 7f121dffb640 1 -- 172.21.15.100:0/502778724 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c02c070 con 0x5558595cb4d0 2024-07-26T13:46:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.940+0000 7f121dffb640 1 -- 172.21.15.100:0/502778724 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7f122c004a00 con 0x5558595cb4d0 2024-07-26T13:46:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.940+0000 7f12437fc740 1 -- 172.21.15.100:0/502778724 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f123001fb80 con 0x5558595cb4d0 2024-07-26T13:46:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.941+0000 7f121dffb640 1 -- 172.21.15.100:0/502778724 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f122c035050 con 0x5558595cb4d0 2024-07-26T13:46:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.941+0000 7f12437fc740 10 client.17166 fetch_fsmap learned FSMap version 53 2024-07-26T13:46:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.941+0000 7f12437fc740 1 -- 172.21.15.100:0/502778724 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x555859a13470 con 0x5558595cb4d0 2024-07-26T13:46:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.941+0000 7f121dffb640 1 -- 172.21.15.100:0/502778724 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f123001fb80 con 0x5558595cb4d0 2024-07-26T13:46:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.941+0000 7f121dffb640 1 -- 172.21.15.100:0/502778724 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7f122c02f020 con 0x5558595cb4d0 2024-07-26T13:46:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.941+0000 7f12437fc740 10 client.17166 fetch_fsmap finished waiting for FSMap version 53 2024-07-26T13:46:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.941+0000 7f12437fc740 10 client.17166 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:46:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.941+0000 7f12437fc740 20 client.17166 populate_metadata read hostname 'smithi100' 2024-07-26T13:46:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.941+0000 7f12437fc740 1 --2- 172.21.15.100:0/502778724 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5558595801c0 0x555859a159a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.941+0000 7f12437fc740 4 client.17166 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-26T13:46:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.941+0000 7f12437fc740 1 -- 172.21.15.100:0/502778724 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555859901da0 con 0x5558595801c0 2024-07-26T13:46:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.941+0000 7f123659c640 1 --2- 172.21.15.100:0/502778724 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5558595801c0 0x555859a159a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.942+0000 7f121dffb640 10 client.17166 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.942+0000 7f123659c640 1 --2- 172.21.15.100:0/502778724 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5558595801c0 0x555859a159a0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a59c5204dd35c0c server_cookie=622f3e6a8e9bc13 in_seq=0 out_seq=0 2024-07-26T13:46:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.971+0000 7f121dffb640 1 -- 172.21.15.100:0/502778724 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555859901da0 con 0x5558595801c0 2024-07-26T13:46:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.971+0000 7f121dffb640 10 client.17166 handle_command_reply: tid=0 2024-07-26T13:46:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.971+0000 7f12437fc740 1 client.17166 shutdown 2024-07-26T13:46:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.971+0000 7f12437fc740 1 -- 172.21.15.100:0/502778724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558595cb4d0 msgr2=0x55585999ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.971+0000 7f12437fc740 1 --2- 172.21.15.100:0/502778724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558595cb4d0 0x55585999ef50 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f122c0129f0 tx=0x7f122c009a80 comp rx=0 tx=0).stop 2024-07-26T13:46:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.971+0000 7f12437fc740 1 -- 172.21.15.100:0/502778724 shutdown_connections 2024-07-26T13:46:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.971+0000 7f12437fc740 1 --2- 172.21.15.100:0/502778724 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5558595801c0 0x555859a159a0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.971+0000 7f12437fc740 1 --2- 172.21.15.100:0/502778724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558595ec1b0 0x555859a0ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.971+0000 7f12437fc740 1 --2- 172.21.15.100:0/502778724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5558595cb4d0 0x55585999ef50 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.971+0000 7f121dffb640 0 client.17166 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.971+0000 7f12437fc740 1 --2- 172.21.15.100:0/502778724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5558595c32f0 0x55585999e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.971+0000 7f12437fc740 1 -- 172.21.15.100:0/502778724 >> 172.21.15.100:0/502778724 conn(0x5558595e0300 msgr2=0x55585999b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.971+0000 7f12437fc740 1 -- 172.21.15.100:0/502778724 shutdown_connections 2024-07-26T13:46:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.971+0000 7f12437fc740 1 -- 172.21.15.100:0/502778724 wait complete. 2024-07-26T13:46:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.972+0000 7f12437fc740 20 client.17166 trim_cache size 0 max 16384 2024-07-26T13:46:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-07-26T13:46:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-07-26T13:46:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-07-26T13:46:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-07-26T13:46:34.975 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.972+0000 7f123c806640 1 -- 172.21.15.100:0/3200331603 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f12200b3d40 msgr2=0x7f12200b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:34.975 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.972+0000 7f123c806640 1 --2- 172.21.15.100:0/3200331603 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f12200b3d40 0x7f12200b6250 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x55585998d330 tx=0x7f1230005f90 comp rx=0 tx=0).stop 2024-07-26T13:46:34.975 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.972+0000 7f123c806640 1 -- 172.21.15.100:0/3200331603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1238108660 msgr2=0x7f123819e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:34.975 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.972+0000 7f123c806640 1 --2- 172.21.15.100:0/3200331603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1238108660 0x7f123819e5a0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f1228004770 tx=0x7f1228004270 comp rx=0 tx=0).stop 2024-07-26T13:46:34.975 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.972+0000 7f123c806640 1 -- 172.21.15.100:0/3200331603 shutdown_connections 2024-07-26T13:46:34.976 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.972+0000 7f123c806640 1 --2- 172.21.15.100:0/3200331603 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f12200b3d40 0x7f12200b6250 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:34.976 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.972+0000 7f123c806640 1 --2- 172.21.15.100:0/3200331603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1238108660 0x7f123819e5a0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:34.976 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.972+0000 7f123c806640 1 --2- 172.21.15.100:0/3200331603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1238107d10 0x7f1238199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:34.976 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.972+0000 7f123c806640 1 --2- 172.21.15.100:0/3200331603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12381023e0 0x7f1238198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:34.976 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.972+0000 7f123c806640 1 -- 172.21.15.100:0/3200331603 >> 172.21.15.100:0/3200331603 conn(0x7f123807a170 msgr2=0x7f1238105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:34.976 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.973+0000 7f123c806640 1 -- 172.21.15.100:0/3200331603 shutdown_connections 2024-07-26T13:46:34.976 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:34.973+0000 7f123c806640 1 -- 172.21.15.100:0/3200331603 wait complete. 2024-07-26T13:46:35.422 INFO:tasks.workunit.client.0.smithi100.stdout: 1 238171 2.94 MB/sec execute 169 sec latency 27.003 ms 2024-07-26T13:46:35.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:35 smithi138 ceph-mon[36954]: pgmap v561: 97 pgs: 97 active+clean; 88 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 517 KiB/s wr, 49 op/s 2024-07-26T13:46:35.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:35 smithi100 ceph-mon[32599]: pgmap v561: 97 pgs: 97 active+clean; 88 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 517 KiB/s wr, 49 op/s 2024-07-26T13:46:36.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:35 smithi167 ceph-mon[37140]: pgmap v561: 97 pgs: 97 active+clean; 88 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 517 KiB/s wr, 49 op/s 2024-07-26T13:46:36.422 INFO:tasks.workunit.client.0.smithi100.stdout: 1 240266 2.97 MB/sec execute 170 sec latency 17.442 ms 2024-07-26T13:46:37.422 INFO:tasks.workunit.client.0.smithi100.stdout: 1 242046 3.02 MB/sec execute 171 sec latency 19.583 ms 2024-07-26T13:46:37.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:37 smithi138 ceph-mon[36954]: pgmap v562: 97 pgs: 97 active+clean; 88 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 175 KiB/s wr, 27 op/s 2024-07-26T13:46:37.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:37 smithi100 ceph-mon[32599]: pgmap v562: 97 pgs: 97 active+clean; 88 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 175 KiB/s wr, 27 op/s 2024-07-26T13:46:38.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:37 smithi167 ceph-mon[37140]: pgmap v562: 97 pgs: 97 active+clean; 88 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 175 KiB/s wr, 27 op/s 2024-07-26T13:46:38.422 INFO:tasks.workunit.client.0.smithi100.stdout: 1 243708 3.05 MB/sec execute 172 sec latency 17.508 ms 2024-07-26T13:46:39.422 INFO:tasks.workunit.client.0.smithi100.stdout: 1 245375 3.09 MB/sec execute 173 sec latency 11.141 ms 2024-07-26T13:46:39.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:39 smithi138 ceph-mon[36954]: pgmap v563: 97 pgs: 97 active+clean; 88 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 552 KiB/s wr, 68 op/s 2024-07-26T13:46:39.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:39 smithi100 ceph-mon[32599]: pgmap v563: 97 pgs: 97 active+clean; 88 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 552 KiB/s wr, 68 op/s 2024-07-26T13:46:40.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:39 smithi167 ceph-mon[37140]: pgmap v563: 97 pgs: 97 active+clean; 88 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 552 KiB/s wr, 68 op/s 2024-07-26T13:46:40.422 INFO:tasks.workunit.client.0.smithi100.stdout: 1 247683 3.15 MB/sec execute 174 sec latency 26.655 ms 2024-07-26T13:46:41.422 INFO:tasks.workunit.client.0.smithi100.stdout: 1 249397 3.18 MB/sec execute 175 sec latency 13.231 ms 2024-07-26T13:46:41.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:41 smithi138 ceph-mon[36954]: pgmap v564: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 864 KiB/s wr, 107 op/s 2024-07-26T13:46:41.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:41 smithi100 ceph-mon[32599]: pgmap v564: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 864 KiB/s wr, 107 op/s 2024-07-26T13:46:42.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:41 smithi167 ceph-mon[37140]: pgmap v564: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 864 KiB/s wr, 107 op/s 2024-07-26T13:46:42.422 INFO:tasks.workunit.client.0.smithi100.stdout: 1 251685 3.23 MB/sec execute 176 sec latency 19.201 ms 2024-07-26T13:46:43.422 INFO:tasks.workunit.client.0.smithi100.stdout: 1 253159 3.25 MB/sec execute 177 sec latency 17.172 ms 2024-07-26T13:46:43.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:43 smithi138 ceph-mon[36954]: pgmap v565: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 902 KiB/s wr, 106 op/s 2024-07-26T13:46:43.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:43 smithi100 ceph-mon[32599]: pgmap v565: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 902 KiB/s wr, 106 op/s 2024-07-26T13:46:44.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:43 smithi167 ceph-mon[37140]: pgmap v565: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 902 KiB/s wr, 106 op/s 2024-07-26T13:46:44.422 INFO:tasks.workunit.client.0.smithi100.stdout: 1 255066 3.30 MB/sec execute 178 sec latency 16.008 ms 2024-07-26T13:46:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:44 smithi138 ceph-mon[36954]: pgmap v566: 97 pgs: 97 active+clean; 99 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 192 op/s 2024-07-26T13:46:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:44 smithi100 ceph-mon[32599]: pgmap v566: 97 pgs: 97 active+clean; 99 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 192 op/s 2024-07-26T13:46:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:44 smithi167 ceph-mon[37140]: pgmap v566: 97 pgs: 97 active+clean; 99 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 192 op/s 2024-07-26T13:46:45.422 INFO:tasks.workunit.client.0.smithi100.stdout: 1 256584 3.32 MB/sec execute 179 sec latency 27.975 ms 2024-07-26T13:46:46.422 INFO:tasks.workunit.client.0.smithi100.stdout: 1 259050 3.39 MB/sec execute 180 sec latency 13.021 ms 2024-07-26T13:46:47.422 INFO:tasks.workunit.client.0.smithi100.stdout: 1 260457 3.41 MB/sec execute 181 sec latency 14.282 ms 2024-07-26T13:46:47.855 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-26T13:46:47.879 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:47 smithi100 ceph-mon[32599]: pgmap v567: 97 pgs: 97 active+clean; 98 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 203 op/s 2024-07-26T13:46:47.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:47 smithi138 ceph-mon[36954]: pgmap v567: 97 pgs: 97 active+clean; 98 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 203 op/s 2024-07-26T13:46:48.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:47 smithi167 ceph-mon[37140]: pgmap v567: 97 pgs: 97 active+clean; 98 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 203 op/s 2024-07-26T13:46:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.075+0000 7f3a750fa640 1 -- 172.21.15.100:0/401379316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a70108660 msgr2=0x7f3a7010ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.075+0000 7f3a750fa640 1 --2- 172.21.15.100:0/401379316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a70108660 0x7f3a7010ee90 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f3a68009ca0 tx=0x7f3a6802a8c0 comp rx=0 tx=0).stop 2024-07-26T13:46:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.075+0000 7f3a750fa640 1 -- 172.21.15.100:0/401379316 shutdown_connections 2024-07-26T13:46:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.075+0000 7f3a750fa640 1 --2- 172.21.15.100:0/401379316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a70108660 0x7f3a7010ee90 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.075+0000 7f3a750fa640 1 --2- 172.21.15.100:0/401379316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a70107d10 0x7f3a70108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.075+0000 7f3a750fa640 1 --2- 172.21.15.100:0/401379316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a701023e0 0x7f3a701027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.075+0000 7f3a750fa640 1 -- 172.21.15.100:0/401379316 >> 172.21.15.100:0/401379316 conn(0x7f3a7007a170 msgr2=0x7f3a7007a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:48.078 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.076+0000 7f3a750fa640 1 -- 172.21.15.100:0/401379316 shutdown_connections 2024-07-26T13:46:48.078 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.076+0000 7f3a750fa640 1 -- 172.21.15.100:0/401379316 wait complete. 2024-07-26T13:46:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.077+0000 7f3a750fa640 1 Processor -- start 2024-07-26T13:46:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.077+0000 7f3a750fa640 1 -- start start 2024-07-26T13:46:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.078+0000 7f3a750fa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a701023e0 0x7f3a7019bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.078+0000 7f3a750fa640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a70107d10 0x7f3a7019c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.078+0000 7f3a750fa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a70108660 0x7f3a701a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.078+0000 7f3a750fa640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3a70112310 con 0x7f3a701023e0 2024-07-26T13:46:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.078+0000 7f3a750fa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3a70112190 con 0x7f3a70108660 2024-07-26T13:46:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.078+0000 7f3a750fa640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3a70112490 con 0x7f3a70107d10 2024-07-26T13:46:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.078+0000 7f3a6ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a701023e0 0x7f3a7019bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.078+0000 7f3a6ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a701023e0 0x7f3a7019bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38994/0 (socket says 172.21.15.100:38994) 2024-07-26T13:46:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.078+0000 7f3a6ed9d640 1 -- 172.21.15.100:0/25842222 learned_addr learned my addr 172.21.15.100:0/25842222 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.078+0000 7f3a6f59e640 1 --2- 172.21.15.100:0/25842222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a70108660 0x7f3a701a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.078+0000 7f3a6e59c640 1 --2- 172.21.15.100:0/25842222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a70107d10 0x7f3a7019c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.078+0000 7f3a6f59e640 1 -- 172.21.15.100:0/25842222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a70107d10 msgr2=0x7f3a7019c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.078+0000 7f3a6f59e640 1 --2- 172.21.15.100:0/25842222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a70107d10 0x7f3a7019c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.078+0000 7f3a6f59e640 1 -- 172.21.15.100:0/25842222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a701023e0 msgr2=0x7f3a7019bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.078+0000 7f3a6f59e640 1 --2- 172.21.15.100:0/25842222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a701023e0 0x7f3a7019bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.078+0000 7f3a6f59e640 1 -- 172.21.15.100:0/25842222 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a701a1f10 con 0x7f3a70108660 2024-07-26T13:46:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.078+0000 7f3a6ed9d640 1 --2- 172.21.15.100:0/25842222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a701023e0 0x7f3a7019bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:46:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.078+0000 7f3a6e59c640 1 --2- 172.21.15.100:0/25842222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a70107d10 0x7f3a7019c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:46:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.079+0000 7f3a6f59e640 1 --2- 172.21.15.100:0/25842222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a70108660 0x7f3a701a1810 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3a6802ade0 tx=0x7f3a68031660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.079+0000 7f3a57fff640 1 -- 172.21.15.100:0/25842222 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a68003b70 con 0x7f3a70108660 2024-07-26T13:46:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.079+0000 7f3a57fff640 1 -- 172.21.15.100:0/25842222 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a68031910 con 0x7f3a70108660 2024-07-26T13:46:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.079+0000 7f3a750fa640 1 -- 172.21.15.100:0/25842222 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a701a21a0 con 0x7f3a70108660 2024-07-26T13:46:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.079+0000 7f3a57fff640 1 -- 172.21.15.100:0/25842222 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6802f950 con 0x7f3a70108660 2024-07-26T13:46:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.079+0000 7f3a750fa640 1 -- 172.21.15.100:0/25842222 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a701a8990 con 0x7f3a70108660 2024-07-26T13:46:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.082+0000 7f3a7c00d740 1 Processor -- start 2024-07-26T13:46:48.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.083+0000 7f3a57fff640 1 -- 172.21.15.100:0/25842222 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f3a68004040 con 0x7f3a70108660 2024-07-26T13:46:48.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.083+0000 7f3a7c00d740 1 -- start start 2024-07-26T13:46:48.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.083+0000 7f3a7c00d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba8f7031b0 0x55ba8faae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:48.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.083+0000 7f3a7c00d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba8f6da2f0 0x55ba8faaea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:48.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.083+0000 7f3a7c00d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8f6e24d0 0x55ba8faaef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:48.085 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.083+0000 7f3a7c00d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ba8fa1b3b0 con 0x55ba8f6e24d0 2024-07-26T13:46:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.083+0000 7f3a7c00d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ba8fa0a8b0 con 0x55ba8f7031b0 2024-07-26T13:46:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.083+0000 7f3a7c00d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ba8fa0e8f0 con 0x55ba8f6da2f0 2024-07-26T13:46:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.083+0000 7f3a6ed9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba8f6da2f0 0x55ba8faaea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.083+0000 7f3a6e59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba8f7031b0 0x55ba8faae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.083+0000 7f3a6ed9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba8f6da2f0 0x55ba8faaea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:48446/0 (socket says 172.21.15.100:48446) 2024-07-26T13:46:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.083+0000 7f3a6ed9d640 1 -- 172.21.15.100:0/4199366100 learned_addr learned my addr 172.21.15.100:0/4199366100 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.083+0000 7f3a6e59c640 1 --2- 172.21.15.100:0/4199366100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8f6e24d0 0x55ba8faaef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.083+0000 7f3a57fff640 1 --2- 172.21.15.100:0/25842222 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3a580b3d40 0x7f3a580b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.084+0000 7f3a6ed9d640 1 -- 172.21.15.100:0/4199366100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba8f7031b0 msgr2=0x55ba8faae4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.084+0000 7f3a6ed9d640 1 --2- 172.21.15.100:0/4199366100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba8f7031b0 0x55ba8faae4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.084+0000 7f3a6ed9d640 1 -- 172.21.15.100:0/4199366100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8f6e24d0 msgr2=0x55ba8faaef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:48.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.084+0000 7f3a6ed9d640 1 --2- 172.21.15.100:0/4199366100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8f6e24d0 0x55ba8faaef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:48.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.084+0000 7f3a6ed9d640 1 -- 172.21.15.100:0/4199366100 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba8f9b19a0 con 0x55ba8f6da2f0 2024-07-26T13:46:48.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.084+0000 7f3a57fff640 1 -- 172.21.15.100:0/25842222 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7f3a681052a0 con 0x7f3a70108660 2024-07-26T13:46:48.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.084+0000 7f3a6ed9d640 1 --2- 172.21.15.100:0/4199366100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba8f6da2f0 0x55ba8faaea30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3a60011680 tx=0x7f3a60009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32ef685f3b9dc1a7 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:48.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.084+0000 7f3a557fa640 1 -- 172.21.15.100:0/4199366100 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6001e070 con 0x55ba8f6da2f0 2024-07-26T13:46:48.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.084+0000 7f3a6f59e640 1 --2- 172.21.15.100:0/25842222 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3a580b3d40 0x7f3a580b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:48.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.084+0000 7f3a557fa640 1 -- 172.21.15.100:0/4199366100 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a6000fb10 con 0x55ba8f6da2f0 2024-07-26T13:46:48.087 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.084+0000 7f3a557fa640 1 -- 172.21.15.100:0/4199366100 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a60019b80 con 0x55ba8f6da2f0 2024-07-26T13:46:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.085+0000 7f3a7c00d740 1 -- 172.21.15.100:0/4199366100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba8f6da2f0 msgr2=0x55ba8faaea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.085+0000 7f3a6f59e640 1 --2- 172.21.15.100:0/25842222 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3a580b3d40 0x7f3a580b6250 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3a68102a80 tx=0x7f3a6802a8c0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.085+0000 7f3a7c00d740 1 --2- 172.21.15.100:0/4199366100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba8f6da2f0 0x55ba8faaea30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3a60011680 tx=0x7f3a60009f90 comp rx=0 tx=0).stop 2024-07-26T13:46:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.085+0000 7f3a7c00d740 1 -- 172.21.15.100:0/4199366100 shutdown_connections 2024-07-26T13:46:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.085+0000 7f3a7c00d740 1 --2- 172.21.15.100:0/4199366100 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8f6e24d0 0x55ba8faaef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.085+0000 7f3a7c00d740 1 --2- 172.21.15.100:0/4199366100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba8f6da2f0 0x55ba8faaea30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.085+0000 7f3a7c00d740 1 --2- 172.21.15.100:0/4199366100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba8f7031b0 0x55ba8faae4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.085+0000 7f3a7c00d740 1 -- 172.21.15.100:0/4199366100 >> 172.21.15.100:0/4199366100 conn(0x55ba8f6f7300 msgr2=0x55ba8faac3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.085+0000 7f3a7c00d740 1 -- 172.21.15.100:0/4199366100 shutdown_connections 2024-07-26T13:46:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.085+0000 7f3a7c00d740 1 -- 172.21.15.100:0/4199366100 wait complete. 2024-07-26T13:46:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.086+0000 7f3a7c00d740 1 -- start start 2024-07-26T13:46:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.086+0000 7f3a7c00d740 1 Processor -- start 2024-07-26T13:46:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.086+0000 7f3a7c00d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba8f6da2f0 0x55ba8faba080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.086+0000 7f3a7c00d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8f6e24d0 0x55ba8faba5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.086+0000 7f3a7c00d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba8f7031b0 0x55ba8fac5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.086+0000 7f3a7c00d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ba8f944ab0 con 0x55ba8f6e24d0 2024-07-26T13:46:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.086+0000 7f3a7c00d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ba8f992cf0 con 0x55ba8f6da2f0 2024-07-26T13:46:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.086+0000 7f3a7c00d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ba8fabcc70 con 0x55ba8f7031b0 2024-07-26T13:46:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.086+0000 7f3a7c00d740 10 client.0 _ll_register_callbacks cb 0x55ba8fa049f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:46:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.086+0000 7f3a6e59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba8f6da2f0 0x55ba8faba080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.086+0000 7f3a6ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8f6e24d0 0x55ba8faba5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.087+0000 7f3a6e59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba8f6da2f0 0x55ba8faba080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:35014/0 (socket says 172.21.15.100:35014) 2024-07-26T13:46:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.087+0000 7f3a6e59c640 1 -- 172.21.15.100:0/3589133007 learned_addr learned my addr 172.21.15.100:0/3589133007 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.087+0000 7f3a6e59c640 1 --2- 172.21.15.100:0/3589133007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba8f7031b0 0x55ba8fac5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.087+0000 7f3a6e59c640 1 -- 172.21.15.100:0/3589133007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba8f7031b0 msgr2=0x55ba8fac5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.087+0000 7f3a6e59c640 1 --2- 172.21.15.100:0/3589133007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba8f7031b0 0x55ba8fac5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.087+0000 7f3a6e59c640 1 -- 172.21.15.100:0/3589133007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8f6e24d0 msgr2=0x55ba8faba5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.087+0000 7f3a6e59c640 1 --2- 172.21.15.100:0/3589133007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8f6e24d0 0x55ba8faba5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.087+0000 7f3a6e59c640 1 -- 172.21.15.100:0/3589133007 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba8f76e790 con 0x55ba8f6da2f0 2024-07-26T13:46:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.087+0000 7f3a6ed9d640 1 --2- 172.21.15.100:0/3589133007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8f6e24d0 0x55ba8faba5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:46:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.087+0000 7f3a6e59c640 1 --2- 172.21.15.100:0/3589133007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba8f6da2f0 0x55ba8faba080 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3a64011d30 tx=0x7f3a64004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cfaae0b27889e64 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.087+0000 7f3a52ff5640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-07-26T13:46:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.087+0000 7f3a52ff5640 1 -- 172.21.15.100:0/3589133007 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a64011260 con 0x55ba8f6da2f0 2024-07-26T13:46:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.087+0000 7f3a52ff5640 1 -- 172.21.15.100:0/3589133007 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7f3a64005020 con 0x55ba8f6da2f0 2024-07-26T13:46:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.087+0000 7f3a7c00d740 1 -- 172.21.15.100:0/3589133007 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a6000fb10 con 0x55ba8f6da2f0 2024-07-26T13:46:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.087+0000 7f3a52ff5640 1 -- 172.21.15.100:0/3589133007 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f3a64005220 con 0x55ba8f6da2f0 2024-07-26T13:46:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.088+0000 7f3a7c00d740 10 client.26399 fetch_fsmap learned FSMap version 53 2024-07-26T13:46:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.088+0000 7f3a7c00d740 1 -- 172.21.15.100:0/3589133007 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55ba8f7e7620 con 0x55ba8f6da2f0 2024-07-26T13:46:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.088+0000 7f3a52ff5640 1 -- 172.21.15.100:0/3589133007 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a6000fb10 con 0x55ba8f6da2f0 2024-07-26T13:46:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.088+0000 7f3a52ff5640 1 -- 172.21.15.100:0/3589133007 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7f3a64020040 con 0x55ba8f6da2f0 2024-07-26T13:46:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.088+0000 7f3a7c00d740 10 client.26399 fetch_fsmap finished waiting for FSMap version 53 2024-07-26T13:46:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.088+0000 7f3a7c00d740 10 client.26399 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:46:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.088+0000 7f3a7c00d740 20 client.26399 populate_metadata read hostname 'smithi100' 2024-07-26T13:46:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.088+0000 7f3a7c00d740 1 --2- 172.21.15.100:0/3589133007 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55ba8f6c1d80 0x55ba8facc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.088+0000 7f3a7c00d740 4 client.26399 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:46:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.088+0000 7f3a7c00d740 1 -- 172.21.15.100:0/3589133007 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba8fa1d300 con 0x55ba8f6c1d80 2024-07-26T13:46:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.088+0000 7f3a6ed9d640 1 --2- 172.21.15.100:0/3589133007 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55ba8f6c1d80 0x55ba8facc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.089+0000 7f3a52ff5640 10 client.26399 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.089+0000 7f3a6ed9d640 1 --2- 172.21.15.100:0/3589133007 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55ba8f6c1d80 0x55ba8facc460 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=601b72a7d39c85be server_cookie=2e43f7fd278cd8e3 in_seq=0 out_seq=0 2024-07-26T13:46:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.090+0000 7f3a52ff5640 1 -- 172.21.15.100:0/3589133007 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ba8fa1d300 con 0x55ba8f6c1d80 2024-07-26T13:46:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.090+0000 7f3a52ff5640 10 client.26399 handle_command_reply: tid=0 2024-07-26T13:46:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.090+0000 7f3a7c00d740 1 client.26399 shutdown 2024-07-26T13:46:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.090+0000 7f3a7c00d740 1 -- 172.21.15.100:0/3589133007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba8f6da2f0 msgr2=0x55ba8faba080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.090+0000 7f3a7c00d740 1 --2- 172.21.15.100:0/3589133007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba8f6da2f0 0x55ba8faba080 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3a64011d30 tx=0x7f3a64004730 comp rx=0 tx=0).stop 2024-07-26T13:46:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.090+0000 7f3a7c00d740 1 -- 172.21.15.100:0/3589133007 shutdown_connections 2024-07-26T13:46:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.090+0000 7f3a7c00d740 1 --2- 172.21.15.100:0/3589133007 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55ba8f6c1d80 0x55ba8facc460 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.090+0000 7f3a7c00d740 1 --2- 172.21.15.100:0/3589133007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba8f7031b0 0x55ba8fac5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.090+0000 7f3a7c00d740 1 --2- 172.21.15.100:0/3589133007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8f6e24d0 0x55ba8faba5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.090+0000 7f3a52ff5640 0 client.26399 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.090+0000 7f3a7c00d740 1 --2- 172.21.15.100:0/3589133007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba8f6da2f0 0x55ba8faba080 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.090+0000 7f3a7c00d740 1 -- 172.21.15.100:0/3589133007 >> 172.21.15.100:0/3589133007 conn(0x55ba8f6f7300 msgr2=0x55ba8faa22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.090+0000 7f3a7c00d740 1 -- 172.21.15.100:0/3589133007 shutdown_connections 2024-07-26T13:46:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.090+0000 7f3a7c00d740 1 -- 172.21.15.100:0/3589133007 wait complete. 2024-07-26T13:46:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.090+0000 7f3a7c00d740 20 client.26399 trim_cache size 0 max 16384 2024-07-26T13:46:48.113 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.106+0000 7f3a7c00d740 1 Processor -- start 2024-07-26T13:46:48.113 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.106+0000 7f3a7c00d740 1 -- start start 2024-07-26T13:46:48.113 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.106+0000 7f3a7c00d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba8f6da2f0 0x55ba8faba350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:48.113 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.106+0000 7f3a7c00d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8f6e24d0 0x55ba8faafc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:48.113 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.106+0000 7f3a7c00d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba8f7031b0 0x55ba8fac5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:48.113 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.106+0000 7f3a7c00d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3a701af980 con 0x55ba8f6e24d0 2024-07-26T13:46:48.113 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.106+0000 7f3a7c00d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3a701ad250 con 0x55ba8f6da2f0 2024-07-26T13:46:48.113 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.106+0000 7f3a7c00d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3a701ab0c0 con 0x55ba8f7031b0 2024-07-26T13:46:48.113 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.106+0000 7f3a6ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8f6e24d0 0x55ba8faafc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:48.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.106+0000 7f3a6e59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba8f6da2f0 0x55ba8faba350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:48.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.106+0000 7f3a6e59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba8f6da2f0 0x55ba8faba350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:35030/0 (socket says 172.21.15.100:35030) 2024-07-26T13:46:48.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.106+0000 7f3a6ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8f6e24d0 0x55ba8faafc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39006/0 (socket says 172.21.15.100:39006) 2024-07-26T13:46:48.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.106+0000 7f3a6ed9d640 1 -- 172.21.15.100:0/2287882475 learned_addr learned my addr 172.21.15.100:0/2287882475 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:48.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.106+0000 7f3a6e59c640 1 --2- 172.21.15.100:0/2287882475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba8f7031b0 0x55ba8fac5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:48.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.106+0000 7f3a6e59c640 1 -- 172.21.15.100:0/2287882475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba8f7031b0 msgr2=0x55ba8fac5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:48.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.106+0000 7f3a6e59c640 1 --2- 172.21.15.100:0/2287882475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba8f7031b0 0x55ba8fac5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:48.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.106+0000 7f3a6e59c640 1 -- 172.21.15.100:0/2287882475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8f6e24d0 msgr2=0x55ba8faafc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:48.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.106+0000 7f3a6e59c640 1 --2- 172.21.15.100:0/2287882475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8f6e24d0 0x55ba8faafc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:48.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.106+0000 7f3a6e59c640 1 -- 172.21.15.100:0/2287882475 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba8fab07f0 con 0x55ba8f6da2f0 2024-07-26T13:46:48.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.106+0000 7f3a6ed9d640 1 --2- 172.21.15.100:0/2287882475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8f6e24d0 0x55ba8faafc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:46:48.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.107+0000 7f3a6e59c640 1 --2- 172.21.15.100:0/2287882475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba8f6da2f0 0x55ba8faba350 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3a640056b0 tx=0x7f3a64005b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcadedbd4af22421 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:48.114 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.107+0000 7f3a54ff9640 1 -- 172.21.15.100:0/2287882475 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a640053c0 con 0x55ba8f6da2f0 2024-07-26T13:46:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.107+0000 7f3a54ff9640 1 -- 172.21.15.100:0/2287882475 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a6400ec50 con 0x55ba8f6da2f0 2024-07-26T13:46:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.107+0000 7f3a54ff9640 1 -- 172.21.15.100:0/2287882475 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a64004d20 con 0x55ba8f6da2f0 2024-07-26T13:46:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.108+0000 7f3a7c00d740 1 -- 172.21.15.100:0/2287882475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba8f6da2f0 msgr2=0x55ba8faba350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.108+0000 7f3a7c00d740 1 --2- 172.21.15.100:0/2287882475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba8f6da2f0 0x55ba8faba350 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3a640056b0 tx=0x7f3a64005b90 comp rx=0 tx=0).stop 2024-07-26T13:46:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.108+0000 7f3a7c00d740 1 -- 172.21.15.100:0/2287882475 shutdown_connections 2024-07-26T13:46:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.108+0000 7f3a7c00d740 1 --2- 172.21.15.100:0/2287882475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba8f7031b0 0x55ba8fac5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.108+0000 7f3a7c00d740 1 --2- 172.21.15.100:0/2287882475 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8f6e24d0 0x55ba8faafc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.108+0000 7f3a7c00d740 1 --2- 172.21.15.100:0/2287882475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba8f6da2f0 0x55ba8faba350 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.108+0000 7f3a7c00d740 1 -- 172.21.15.100:0/2287882475 >> 172.21.15.100:0/2287882475 conn(0x55ba8f6f7300 msgr2=0x55ba8faaf520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.108+0000 7f3a7c00d740 1 -- 172.21.15.100:0/2287882475 shutdown_connections 2024-07-26T13:46:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.108+0000 7f3a7c00d740 1 -- 172.21.15.100:0/2287882475 wait complete. 2024-07-26T13:46:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.108+0000 7f3a7c00d740 1 -- start start 2024-07-26T13:46:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.108+0000 7f3a7c00d740 1 Processor -- start 2024-07-26T13:46:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.108+0000 7f3a7c00d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba8f6da2f0 0x55ba8fac2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.109+0000 7f3a7c00d740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8f6e24d0 0x55ba8fabee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.109+0000 7f3a7c00d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba8f7031b0 0x55ba8fabf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.109+0000 7f3a7c00d740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3a701ab710 con 0x55ba8f6e24d0 2024-07-26T13:46:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.109+0000 7f3a7c00d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3a701acb80 con 0x55ba8f6da2f0 2024-07-26T13:46:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.109+0000 7f3a7c00d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3a701abde0 con 0x55ba8f7031b0 2024-07-26T13:46:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.109+0000 7f3a7c00d740 10 client.0 _ll_register_callbacks cb 0x7f3a640118b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:46:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.109+0000 7f3a6ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8f6e24d0 0x55ba8fabee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.109+0000 7f3a6ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8f6e24d0 0x55ba8fabee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39008/0 (socket says 172.21.15.100:39008) 2024-07-26T13:46:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.109+0000 7f3a6ed9d640 1 -- 172.21.15.100:0/714393125 learned_addr learned my addr 172.21.15.100:0/714393125 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.109+0000 7f3a6e59c640 1 --2- 172.21.15.100:0/714393125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba8f7031b0 0x55ba8fabf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.109+0000 7f3a6e59c640 1 --2- 172.21.15.100:0/714393125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba8f6da2f0 0x55ba8fac2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.109+0000 7f3a6ed9d640 1 -- 172.21.15.100:0/714393125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba8f7031b0 msgr2=0x55ba8fabf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.109+0000 7f3a6ed9d640 1 --2- 172.21.15.100:0/714393125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba8f7031b0 0x55ba8fabf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.109+0000 7f3a6ed9d640 1 -- 172.21.15.100:0/714393125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba8f6da2f0 msgr2=0x55ba8fac2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.109+0000 7f3a6ed9d640 1 --2- 172.21.15.100:0/714393125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba8f6da2f0 0x55ba8fac2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.109+0000 7f3a6ed9d640 1 -- 172.21.15.100:0/714393125 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba8fabf910 con 0x55ba8f6e24d0 2024-07-26T13:46:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.109+0000 7f3a6ed9d640 1 --2- 172.21.15.100:0/714393125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8f6e24d0 0x55ba8fabee90 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f3a60011650 tx=0x7f3a60019410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5d87e18b91eba7b server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.109+0000 7f3a52ff5640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:46:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.109+0000 7f3a52ff5640 1 -- 172.21.15.100:0/714393125 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6001e040 con 0x55ba8f6e24d0 2024-07-26T13:46:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.109+0000 7f3a52ff5640 1 -- 172.21.15.100:0/714393125 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7f3a6000f760 con 0x55ba8f6e24d0 2024-07-26T13:46:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.109+0000 7f3a7c00d740 1 -- 172.21.15.100:0/714393125 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a6400ec50 con 0x55ba8f6e24d0 2024-07-26T13:46:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.109+0000 7f3a52ff5640 1 -- 172.21.15.100:0/714393125 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f3a60009c10 con 0x55ba8f6e24d0 2024-07-26T13:46:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.110+0000 7f3a7c00d740 10 client.17184 fetch_fsmap learned FSMap version 53 2024-07-26T13:46:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.110+0000 7f3a7c00d740 1 -- 172.21.15.100:0/714393125 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55ba8fb338d0 con 0x55ba8f6e24d0 2024-07-26T13:46:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.110+0000 7f3a52ff5640 1 -- 172.21.15.100:0/714393125 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a6400ec50 con 0x55ba8f6e24d0 2024-07-26T13:46:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.110+0000 7f3a52ff5640 1 -- 172.21.15.100:0/714393125 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7f3a60019a60 con 0x55ba8f6e24d0 2024-07-26T13:46:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.110+0000 7f3a7c00d740 10 client.17184 fetch_fsmap finished waiting for FSMap version 53 2024-07-26T13:46:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.110+0000 7f3a7c00d740 10 client.17184 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:46:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.110+0000 7f3a7c00d740 20 client.17184 populate_metadata read hostname 'smithi100' 2024-07-26T13:46:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.110+0000 7f3a7c00d740 1 --2- 172.21.15.100:0/714393125 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55ba8f6971c0 0x55ba8fb35ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.110+0000 7f3a7c00d740 4 client.17184 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-26T13:46:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.110+0000 7f3a7c00d740 1 -- 172.21.15.100:0/714393125 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ba8fa18da0 con 0x55ba8f6971c0 2024-07-26T13:46:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.110+0000 7f3a6e59c640 1 --2- 172.21.15.100:0/714393125 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55ba8f6971c0 0x55ba8fb35ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.111+0000 7f3a52ff5640 10 client.17184 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.111+0000 7f3a6e59c640 1 --2- 172.21.15.100:0/714393125 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55ba8f6971c0 0x55ba8fb35ed0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccb38ecd8dfdd25d server_cookie=cbc33becce1cb95c in_seq=0 out_seq=0 2024-07-26T13:46:48.133 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:46:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.177+0000 7f3a52ff5640 1 -- 172.21.15.100:0/714393125 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ba8fa18da0 con 0x55ba8f6971c0 2024-07-26T13:46:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.177+0000 7f3a52ff5640 10 client.17184 handle_command_reply: tid=0 2024-07-26T13:46:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.177+0000 7f3a7c00d740 1 client.17184 shutdown 2024-07-26T13:46:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.178+0000 7f3a7c00d740 1 -- 172.21.15.100:0/714393125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8f6e24d0 msgr2=0x55ba8fabee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:48.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.178+0000 7f3a7c00d740 1 --2- 172.21.15.100:0/714393125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8f6e24d0 0x55ba8fabee90 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f3a60011650 tx=0x7f3a60019410 comp rx=0 tx=0).stop 2024-07-26T13:46:48.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.178+0000 7f3a7c00d740 1 -- 172.21.15.100:0/714393125 shutdown_connections 2024-07-26T13:46:48.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.178+0000 7f3a7c00d740 1 --2- 172.21.15.100:0/714393125 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55ba8f6971c0 0x55ba8fb35ed0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:48.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.178+0000 7f3a7c00d740 1 --2- 172.21.15.100:0/714393125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba8f7031b0 0x55ba8fabf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:48.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.178+0000 7f3a7c00d740 1 --2- 172.21.15.100:0/714393125 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ba8f6e24d0 0x55ba8fabee90 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:48.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.178+0000 7f3a7c00d740 1 --2- 172.21.15.100:0/714393125 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba8f6da2f0 0x55ba8fac2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:48.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.178+0000 7f3a7c00d740 1 -- 172.21.15.100:0/714393125 >> 172.21.15.100:0/714393125 conn(0x55ba8f6f7300 msgr2=0x55ba8faac3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:48.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.178+0000 7f3a52ff5640 0 client.17184 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:48.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.178+0000 7f3a7c00d740 1 -- 172.21.15.100:0/714393125 shutdown_connections 2024-07-26T13:46:48.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.178+0000 7f3a7c00d740 1 -- 172.21.15.100:0/714393125 wait complete. 2024-07-26T13:46:48.180 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.178+0000 7f3a7c00d740 20 client.17184 trim_cache size 0 max 16384 2024-07-26T13:46:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-07-26T13:46:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-07-26T13:46:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-07-26T13:46:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-07-26T13:46:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.178+0000 7f3a750fa640 1 -- 172.21.15.100:0/25842222 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3a580b3d40 msgr2=0x7f3a580b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.178+0000 7f3a750fa640 1 --2- 172.21.15.100:0/25842222 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3a580b3d40 0x7f3a580b6250 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3a68102a80 tx=0x7f3a6802a8c0 comp rx=0 tx=0).stop 2024-07-26T13:46:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.178+0000 7f3a750fa640 1 -- 172.21.15.100:0/25842222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a70108660 msgr2=0x7f3a701a1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.178+0000 7f3a750fa640 1 --2- 172.21.15.100:0/25842222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a70108660 0x7f3a701a1810 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3a6802ade0 tx=0x7f3a68031660 comp rx=0 tx=0).stop 2024-07-26T13:46:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.179+0000 7f3a750fa640 1 -- 172.21.15.100:0/25842222 shutdown_connections 2024-07-26T13:46:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.179+0000 7f3a750fa640 1 --2- 172.21.15.100:0/25842222 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3a580b3d40 0x7f3a580b6250 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.179+0000 7f3a750fa640 1 --2- 172.21.15.100:0/25842222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a70108660 0x7f3a701a1810 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.179+0000 7f3a750fa640 1 --2- 172.21.15.100:0/25842222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a70107d10 0x7f3a7019c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.179+0000 7f3a750fa640 1 --2- 172.21.15.100:0/25842222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a701023e0 0x7f3a7019bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.179+0000 7f3a750fa640 1 -- 172.21.15.100:0/25842222 >> 172.21.15.100:0/25842222 conn(0x7f3a7007a170 msgr2=0x7f3a7006a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:48.183 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.179+0000 7f3a750fa640 1 -- 172.21.15.100:0/25842222 shutdown_connections 2024-07-26T13:46:48.183 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:46:48.179+0000 7f3a750fa640 1 -- 172.21.15.100:0/25842222 wait complete. 2024-07-26T13:46:48.422 INFO:tasks.workunit.client.0.smithi100.stdout: 1 262771 3.47 MB/sec execute 182 sec latency 25.926 ms 2024-07-26T13:46:49.423 INFO:tasks.workunit.client.0.smithi100.stdout: 1 264338 3.48 MB/sec execute 183 sec latency 23.227 ms 2024-07-26T13:46:49.735 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:49 smithi100 ceph-mon[32599]: pgmap v568: 97 pgs: 97 active+clean; 99 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.2 MiB/s wr, 219 op/s 2024-07-26T13:46:49.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:49 smithi138 ceph-mon[36954]: pgmap v568: 97 pgs: 97 active+clean; 99 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.2 MiB/s wr, 219 op/s 2024-07-26T13:46:50.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:49 smithi167 ceph-mon[37140]: pgmap v568: 97 pgs: 97 active+clean; 99 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.2 MiB/s wr, 219 op/s 2024-07-26T13:46:50.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.088+0000 7f74555f8640 1 -- 172.21.15.100:0/2388281321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7450108720 msgr2=0x7f745010ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:50.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.088+0000 7f74555f8640 1 --2- 172.21.15.100:0/2388281321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7450108720 0x7f745010ef50 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f7448009ca0 tx=0x7f744802a8c0 comp rx=0 tx=0).stop 2024-07-26T13:46:50.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.088+0000 7f74555f8640 1 -- 172.21.15.100:0/2388281321 shutdown_connections 2024-07-26T13:46:50.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.088+0000 7f74555f8640 1 --2- 172.21.15.100:0/2388281321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7450108720 0x7f745010ef50 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:50.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.088+0000 7f74555f8640 1 --2- 172.21.15.100:0/2388281321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7450107dd0 0x7f74501081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:50.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.088+0000 7f74555f8640 1 --2- 172.21.15.100:0/2388281321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7450102490 0x7f74501028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:50.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.088+0000 7f74555f8640 1 -- 172.21.15.100:0/2388281321 >> 172.21.15.100:0/2388281321 conn(0x7f7450079160 msgr2=0x7f7450079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:50.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.088+0000 7f74555f8640 1 -- 172.21.15.100:0/2388281321 shutdown_connections 2024-07-26T13:46:50.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.088+0000 7f74555f8640 1 -- 172.21.15.100:0/2388281321 wait complete. 2024-07-26T13:46:50.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.089+0000 7f74555f8640 1 Processor -- start 2024-07-26T13:46:50.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.089+0000 7f74555f8640 1 -- start start 2024-07-26T13:46:50.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.089+0000 7f74555f8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7450102490 0x7f7450199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:50.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.089+0000 7f74555f8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7450107dd0 0x7f745019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:50.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.089+0000 7f74555f8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7450108720 0x7f745019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:50.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.089+0000 7f74555f8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f74501123a0 con 0x7f7450107dd0 2024-07-26T13:46:50.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.089+0000 7f74555f8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7450112220 con 0x7f7450102490 2024-07-26T13:46:50.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.089+0000 7f74555f8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7450112520 con 0x7f7450108720 2024-07-26T13:46:50.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.090+0000 7f744e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7450107dd0 0x7f745019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:50.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.090+0000 7f744effd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7450102490 0x7f7450199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:50.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.090+0000 7f744e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7450107dd0 0x7f745019a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39022/0 (socket says 172.21.15.100:39022) 2024-07-26T13:46:50.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.090+0000 7f744e7fc640 1 -- 172.21.15.100:0/4234551195 learned_addr learned my addr 172.21.15.100:0/4234551195 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:50.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.090+0000 7f744f7fe640 1 --2- 172.21.15.100:0/4234551195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7450108720 0x7f745019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:50.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.090+0000 7f744e7fc640 1 -- 172.21.15.100:0/4234551195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7450108720 msgr2=0x7f745019f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:50.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.090+0000 7f744e7fc640 1 --2- 172.21.15.100:0/4234551195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7450108720 0x7f745019f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:50.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.090+0000 7f744e7fc640 1 -- 172.21.15.100:0/4234551195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7450102490 msgr2=0x7f7450199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:50.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.090+0000 7f744e7fc640 1 --2- 172.21.15.100:0/4234551195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7450102490 0x7f7450199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:50.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.090+0000 7f744e7fc640 1 -- 172.21.15.100:0/4234551195 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f745019fdf0 con 0x7f7450107dd0 2024-07-26T13:46:50.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.090+0000 7f744f7fe640 1 --2- 172.21.15.100:0/4234551195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7450108720 0x7f745019f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:46:50.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.090+0000 7f744e7fc640 1 --2- 172.21.15.100:0/4234551195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7450107dd0 0x7f745019a420 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f743800e9f0 tx=0x7f743800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:50.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.090+0000 7f7447fff640 1 -- 172.21.15.100:0/4234551195 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743800ccc0 con 0x7f7450107dd0 2024-07-26T13:46:50.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.090+0000 7f74555f8640 1 -- 172.21.15.100:0/4234551195 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74501a00e0 con 0x7f7450107dd0 2024-07-26T13:46:50.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.090+0000 7f7447fff640 1 -- 172.21.15.100:0/4234551195 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74380054d0 con 0x7f7450107dd0 2024-07-26T13:46:50.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.090+0000 7f7447fff640 1 -- 172.21.15.100:0/4234551195 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7438010430 con 0x7f7450107dd0 2024-07-26T13:46:50.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.090+0000 7f74555f8640 1 -- 172.21.15.100:0/4234551195 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74501a6840 con 0x7f7450107dd0 2024-07-26T13:46:50.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.093+0000 7f745bb7c740 1 Processor -- start 2024-07-26T13:46:50.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.093+0000 7f7447fff640 1 -- 172.21.15.100:0/4234551195 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f74380046c0 con 0x7f7450107dd0 2024-07-26T13:46:50.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.093+0000 7f745bb7c740 1 -- start start 2024-07-26T13:46:50.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.094+0000 7f745bb7c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d9ac7070 0x55a2d9e6c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:50.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.094+0000 7f745bb7c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2d9a9e1b0 0x55a2d9e6ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:50.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.094+0000 7f745bb7c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2d9aa6390 0x55a2d9e6cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:50.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.094+0000 7f745bb7c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55a2d9de2b90 con 0x55a2d9a9e1b0 2024-07-26T13:46:50.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.094+0000 7f745bb7c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a2d9dd2090 con 0x55a2d9ac7070 2024-07-26T13:46:50.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.094+0000 7f745bb7c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a2d9dd60d0 con 0x55a2d9aa6390 2024-07-26T13:46:50.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.094+0000 7f744e7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2d9aa6390 0x55a2d9e6cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:50.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.094+0000 7f744f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d9ac7070 0x55a2d9e6c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:50.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.094+0000 7f744f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d9ac7070 0x55a2d9e6c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:35076/0 (socket says 172.21.15.100:35076) 2024-07-26T13:46:50.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.094+0000 7f744f7fe640 1 -- 172.21.15.100:0/2812789435 learned_addr learned my addr 172.21.15.100:0/2812789435 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:50.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.094+0000 7f744effd640 1 --2- 172.21.15.100:0/2812789435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2d9a9e1b0 0x55a2d9e6ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:50.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.094+0000 7f744f7fe640 1 -- 172.21.15.100:0/2812789435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2d9aa6390 msgr2=0x55a2d9e6cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:50.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.094+0000 7f744f7fe640 1 --2- 172.21.15.100:0/2812789435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2d9aa6390 0x55a2d9e6cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:50.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.094+0000 7f744f7fe640 1 -- 172.21.15.100:0/2812789435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2d9a9e1b0 msgr2=0x55a2d9e6ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:50.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.094+0000 7f744f7fe640 1 --2- 172.21.15.100:0/2812789435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2d9a9e1b0 0x55a2d9e6ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:50.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.094+0000 7f744f7fe640 1 -- 172.21.15.100:0/2812789435 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2d9d78b70 con 0x55a2d9ac7070 2024-07-26T13:46:50.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.095+0000 7f7447fff640 1 --2- 172.21.15.100:0/4234551195 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f743c0b3d40 0x7f743c0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:50.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.095+0000 7f744e7fc640 1 --2- 172.21.15.100:0/2812789435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2d9aa6390 0x55a2d9e6cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:46:50.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.095+0000 7f744effd640 1 --2- 172.21.15.100:0/4234551195 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f743c0b3d40 0x7f743c0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:50.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.095+0000 7f744f7fe640 1 --2- 172.21.15.100:0/2812789435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d9ac7070 0x55a2d9e6c550 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7448033040 tx=0x7f744802f310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=944e7c5f885735c6 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:50.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.095+0000 7f7447fff640 1 -- 172.21.15.100:0/4234551195 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7f74380e4420 con 0x7f7450107dd0 2024-07-26T13:46:50.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.095+0000 7f74457fa640 1 -- 172.21.15.100:0/2812789435 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7448031650 con 0x55a2d9ac7070 2024-07-26T13:46:50.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.095+0000 7f74457fa640 1 -- 172.21.15.100:0/2812789435 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7448036040 con 0x55a2d9ac7070 2024-07-26T13:46:50.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.095+0000 7f74457fa640 1 -- 172.21.15.100:0/2812789435 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74480044c0 con 0x55a2d9ac7070 2024-07-26T13:46:50.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.096+0000 7f745bb7c740 1 -- 172.21.15.100:0/2812789435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d9ac7070 msgr2=0x55a2d9e6c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:50.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.096+0000 7f745bb7c740 1 --2- 172.21.15.100:0/2812789435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d9ac7070 0x55a2d9e6c550 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7448033040 tx=0x7f744802f310 comp rx=0 tx=0).stop 2024-07-26T13:46:50.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.096+0000 7f744effd640 1 --2- 172.21.15.100:0/4234551195 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f743c0b3d40 0x7f743c0b6250 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x55a2d9e6b1d0 tx=0x7f7440005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:50.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.096+0000 7f745bb7c740 1 -- 172.21.15.100:0/2812789435 shutdown_connections 2024-07-26T13:46:50.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.096+0000 7f745bb7c740 1 --2- 172.21.15.100:0/2812789435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2d9aa6390 0x55a2d9e6cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:50.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.096+0000 7f745bb7c740 1 --2- 172.21.15.100:0/2812789435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2d9a9e1b0 0x55a2d9e6ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:50.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.096+0000 7f745bb7c740 1 --2- 172.21.15.100:0/2812789435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d9ac7070 0x55a2d9e6c550 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:50.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.096+0000 7f745bb7c740 1 -- 172.21.15.100:0/2812789435 >> 172.21.15.100:0/2812789435 conn(0x55a2d9abb460 msgr2=0x55a2d9e6a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:50.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.096+0000 7f745bb7c740 1 -- 172.21.15.100:0/2812789435 shutdown_connections 2024-07-26T13:46:50.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.096+0000 7f745bb7c740 1 -- 172.21.15.100:0/2812789435 wait complete. 2024-07-26T13:46:50.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.097+0000 7f745bb7c740 1 -- start start 2024-07-26T13:46:50.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.097+0000 7f745bb7c740 1 Processor -- start 2024-07-26T13:46:50.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.097+0000 7f745bb7c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d9a9e1b0 0x55a2d9e781f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:50.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.097+0000 7f745bb7c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2d9aa6390 0x55a2d9e787b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:50.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.097+0000 7f745bb7c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2d9ac7070 0x55a2d9e83e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:50.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.097+0000 7f745bb7c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55a2d9c35700 con 0x55a2d9ac7070 2024-07-26T13:46:50.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.097+0000 7f745bb7c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a2d9d749c0 con 0x55a2d9a9e1b0 2024-07-26T13:46:50.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.097+0000 7f745bb7c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a2d9e7ab50 con 0x55a2d9aa6390 2024-07-26T13:46:50.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.097+0000 7f745bb7c740 10 client.0 _ll_register_callbacks cb 0x55a2d9d8f4d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:46:50.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.097+0000 7f744effd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d9a9e1b0 0x55a2d9e781f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:50.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.097+0000 7f744f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2d9ac7070 0x55a2d9e83e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:50.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.097+0000 7f744e7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2d9aa6390 0x55a2d9e787b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:50.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.097+0000 7f744effd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d9a9e1b0 0x55a2d9e781f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:35086/0 (socket says 172.21.15.100:35086) 2024-07-26T13:46:50.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.097+0000 7f744f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2d9ac7070 0x55a2d9e83e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39050/0 (socket says 172.21.15.100:39050) 2024-07-26T13:46:50.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.097+0000 7f744effd640 1 -- 172.21.15.100:0/1693827357 learned_addr learned my addr 172.21.15.100:0/1693827357 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:50.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.097+0000 7f744effd640 1 -- 172.21.15.100:0/1693827357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2d9aa6390 msgr2=0x55a2d9e787b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:50.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.097+0000 7f744effd640 1 --2- 172.21.15.100:0/1693827357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2d9aa6390 0x55a2d9e787b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:50.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.098+0000 7f744effd640 1 -- 172.21.15.100:0/1693827357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2d9ac7070 msgr2=0x55a2d9e83e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:50.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.098+0000 7f744effd640 1 --2- 172.21.15.100:0/1693827357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2d9ac7070 0x55a2d9e83e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:50.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.098+0000 7f744effd640 1 -- 172.21.15.100:0/1693827357 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2d9b32650 con 0x55a2d9a9e1b0 2024-07-26T13:46:50.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.098+0000 7f744f7fe640 1 --2- 172.21.15.100:0/1693827357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2d9ac7070 0x55a2d9e83e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:46:50.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.098+0000 7f744effd640 1 --2- 172.21.15.100:0/1693827357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d9a9e1b0 0x55a2d9e781f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f74400045f0 tx=0x7f7440004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11924306ec9448b3 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:50.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.098+0000 7f74367fc640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-07-26T13:46:50.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.098+0000 7f74367fc640 1 -- 172.21.15.100:0/1693827357 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744001f640 con 0x55a2d9a9e1b0 2024-07-26T13:46:50.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.098+0000 7f745bb7c740 1 -- 172.21.15.100:0/1693827357 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7448036040 con 0x55a2d9a9e1b0 2024-07-26T13:46:50.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.098+0000 7f74367fc640 1 -- 172.21.15.100:0/1693827357 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7f744001f7e0 con 0x55a2d9a9e1b0 2024-07-26T13:46:50.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.098+0000 7f74367fc640 1 -- 172.21.15.100:0/1693827357 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f744001f9e0 con 0x55a2d9a9e1b0 2024-07-26T13:46:50.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.098+0000 7f745bb7c740 10 client.26417 fetch_fsmap learned FSMap version 53 2024-07-26T13:46:50.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.098+0000 7f745bb7c740 1 -- 172.21.15.100:0/1693827357 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55a2d9e86ca0 con 0x55a2d9a9e1b0 2024-07-26T13:46:50.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.098+0000 7f74367fc640 1 -- 172.21.15.100:0/1693827357 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7448036040 con 0x55a2d9a9e1b0 2024-07-26T13:46:50.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.099+0000 7f74367fc640 1 -- 172.21.15.100:0/1693827357 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7f7440027070 con 0x55a2d9a9e1b0 2024-07-26T13:46:50.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.099+0000 7f745bb7c740 10 client.26417 fetch_fsmap finished waiting for FSMap version 53 2024-07-26T13:46:50.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.099+0000 7f745bb7c740 10 client.26417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:46:50.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.099+0000 7f745bb7c740 20 client.26417 populate_metadata read hostname 'smithi100' 2024-07-26T13:46:50.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.099+0000 7f745bb7c740 1 --2- 172.21.15.100:0/1693827357 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55a2d9a859f0 0x55a2d9e8a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:50.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.099+0000 7f745bb7c740 4 client.26417 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:46:50.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.099+0000 7f745bb7c740 1 -- 172.21.15.100:0/1693827357 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2d9de4ae0 con 0x55a2d9a859f0 2024-07-26T13:46:50.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.099+0000 7f744e7fc640 1 --2- 172.21.15.100:0/1693827357 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55a2d9a859f0 0x55a2d9e8a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:50.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.099+0000 7f744e7fc640 1 --2- 172.21.15.100:0/1693827357 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55a2d9a859f0 0x55a2d9e8a130 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62bc6d517669fb95 server_cookie=a85d62edf4a11cb0 in_seq=0 out_seq=0 2024-07-26T13:46:50.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.099+0000 7f74367fc640 10 client.26417 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:50.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.100+0000 7f74367fc640 1 -- 172.21.15.100:0/1693827357 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a2d9de4ae0 con 0x55a2d9a859f0 2024-07-26T13:46:50.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.100+0000 7f74367fc640 10 client.26417 handle_command_reply: tid=0 2024-07-26T13:46:50.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.100+0000 7f745bb7c740 1 client.26417 shutdown 2024-07-26T13:46:50.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.100+0000 7f745bb7c740 1 -- 172.21.15.100:0/1693827357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d9a9e1b0 msgr2=0x55a2d9e781f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:50.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.100+0000 7f745bb7c740 1 --2- 172.21.15.100:0/1693827357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d9a9e1b0 0x55a2d9e781f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f74400045f0 tx=0x7f7440004730 comp rx=0 tx=0).stop 2024-07-26T13:46:50.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.100+0000 7f745bb7c740 1 -- 172.21.15.100:0/1693827357 shutdown_connections 2024-07-26T13:46:50.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.100+0000 7f745bb7c740 1 --2- 172.21.15.100:0/1693827357 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55a2d9a859f0 0x55a2d9e8a130 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:50.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.100+0000 7f745bb7c740 1 --2- 172.21.15.100:0/1693827357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2d9ac7070 0x55a2d9e83e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:50.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.100+0000 7f745bb7c740 1 --2- 172.21.15.100:0/1693827357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2d9aa6390 0x55a2d9e787b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:50.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.100+0000 7f74367fc640 0 client.26417 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:50.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.100+0000 7f745bb7c740 1 --2- 172.21.15.100:0/1693827357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d9a9e1b0 0x55a2d9e781f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:50.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.100+0000 7f745bb7c740 1 -- 172.21.15.100:0/1693827357 >> 172.21.15.100:0/1693827357 conn(0x55a2d9abb460 msgr2=0x55a2d9e68c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:50.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.100+0000 7f745bb7c740 1 -- 172.21.15.100:0/1693827357 shutdown_connections 2024-07-26T13:46:50.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.100+0000 7f745bb7c740 1 -- 172.21.15.100:0/1693827357 wait complete. 2024-07-26T13:46:50.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.101+0000 7f745bb7c740 20 client.26417 trim_cache size 0 max 16384 2024-07-26T13:46:50.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.119+0000 7f745bb7c740 1 Processor -- start 2024-07-26T13:46:50.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.119+0000 7f745bb7c740 1 -- start start 2024-07-26T13:46:50.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.119+0000 7f745bb7c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2d9a9e1b0 0x55a2d9e69330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:50.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.119+0000 7f745bb7c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d9aa6390 0x55a2d9e83c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:50.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.119+0000 7f745bb7c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2d9ac7070 0x55a2d9e69e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:50.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.119+0000 7f745bb7c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f74501b1b70 con 0x55a2d9ac7070 2024-07-26T13:46:50.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.119+0000 7f745bb7c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f74501af440 con 0x55a2d9aa6390 2024-07-26T13:46:50.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.119+0000 7f745bb7c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f74501ad290 con 0x55a2d9a9e1b0 2024-07-26T13:46:50.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.119+0000 7f744e7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d9aa6390 0x55a2d9e83c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:50.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.119+0000 7f744effd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2d9a9e1b0 0x55a2d9e69330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:50.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.119+0000 7f744f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2d9ac7070 0x55a2d9e69e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:50.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.119+0000 7f744e7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d9aa6390 0x55a2d9e83c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:35088/0 (socket says 172.21.15.100:35088) 2024-07-26T13:46:50.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.119+0000 7f744e7fc640 1 -- 172.21.15.100:0/3354220434 learned_addr learned my addr 172.21.15.100:0/3354220434 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:50.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.119+0000 7f744effd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2d9a9e1b0 0x55a2d9e69330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:48520/0 (socket says 172.21.15.100:48520) 2024-07-26T13:46:50.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.120+0000 7f744e7fc640 1 -- 172.21.15.100:0/3354220434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2d9a9e1b0 msgr2=0x55a2d9e69330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:50.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.120+0000 7f744e7fc640 1 --2- 172.21.15.100:0/3354220434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2d9a9e1b0 0x55a2d9e69330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:50.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.120+0000 7f744e7fc640 1 -- 172.21.15.100:0/3354220434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2d9ac7070 msgr2=0x55a2d9e69e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:50.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.120+0000 7f744e7fc640 1 --2- 172.21.15.100:0/3354220434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2d9ac7070 0x55a2d9e69e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:50.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.120+0000 7f744e7fc640 1 -- 172.21.15.100:0/3354220434 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2d9e73880 con 0x55a2d9aa6390 2024-07-26T13:46:50.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.120+0000 7f744e7fc640 1 --2- 172.21.15.100:0/3354220434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d9aa6390 0x55a2d9e83c30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f74380aaee0 tx=0x7f74380e7f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2de57321ae3e1e8f server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:50.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.120+0000 7f7444ff9640 1 -- 172.21.15.100:0/3354220434 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74380aa6c0 con 0x55a2d9aa6390 2024-07-26T13:46:50.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.120+0000 7f7444ff9640 1 -- 172.21.15.100:0/3354220434 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74380af330 con 0x55a2d9aa6390 2024-07-26T13:46:50.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.120+0000 7f7444ff9640 1 -- 172.21.15.100:0/3354220434 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74380af550 con 0x55a2d9aa6390 2024-07-26T13:46:50.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.121+0000 7f745bb7c740 1 -- 172.21.15.100:0/3354220434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d9aa6390 msgr2=0x55a2d9e83c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:50.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.121+0000 7f745bb7c740 1 --2- 172.21.15.100:0/3354220434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d9aa6390 0x55a2d9e83c30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f74380aaee0 tx=0x7f74380e7f70 comp rx=0 tx=0).stop 2024-07-26T13:46:50.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.121+0000 7f745bb7c740 1 -- 172.21.15.100:0/3354220434 shutdown_connections 2024-07-26T13:46:50.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.121+0000 7f745bb7c740 1 --2- 172.21.15.100:0/3354220434 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2d9ac7070 0x55a2d9e69e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:50.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.121+0000 7f745bb7c740 1 --2- 172.21.15.100:0/3354220434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d9aa6390 0x55a2d9e83c30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:50.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.121+0000 7f745bb7c740 1 --2- 172.21.15.100:0/3354220434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2d9a9e1b0 0x55a2d9e69330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:50.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.121+0000 7f745bb7c740 1 -- 172.21.15.100:0/3354220434 >> 172.21.15.100:0/3354220434 conn(0x55a2d9abb460 msgr2=0x55a2d9e68c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:50.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.121+0000 7f745bb7c740 1 -- 172.21.15.100:0/3354220434 shutdown_connections 2024-07-26T13:46:50.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.121+0000 7f745bb7c740 1 -- 172.21.15.100:0/3354220434 wait complete. 2024-07-26T13:46:50.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.122+0000 7f745bb7c740 1 -- start start 2024-07-26T13:46:50.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.122+0000 7f745bb7c740 1 Processor -- start 2024-07-26T13:46:50.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.122+0000 7f745bb7c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2d9a9e1b0 0x55a2d9e80100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:50.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.122+0000 7f745bb7c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2d9aa6390 0x55a2d9e806c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:50.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.122+0000 7f745bb7c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d9ac7070 0x55a2d9e7b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:50.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.122+0000 7f745bb7c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f74501ad900 con 0x55a2d9aa6390 2024-07-26T13:46:50.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.122+0000 7f745bb7c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f74501aed70 con 0x55a2d9ac7070 2024-07-26T13:46:50.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.122+0000 7f745bb7c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f74501adfd0 con 0x55a2d9a9e1b0 2024-07-26T13:46:50.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.122+0000 7f745bb7c740 10 client.0 _ll_register_callbacks cb 0x7f7440009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:46:50.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.122+0000 7f744e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2d9aa6390 0x55a2d9e806c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:50.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.122+0000 7f744effd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2d9a9e1b0 0x55a2d9e80100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:50.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.122+0000 7f744f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d9ac7070 0x55a2d9e7b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:50.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.122+0000 7f744effd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2d9a9e1b0 0x55a2d9e80100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:48536/0 (socket says 172.21.15.100:48536) 2024-07-26T13:46:50.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.122+0000 7f744e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2d9aa6390 0x55a2d9e806c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39074/0 (socket says 172.21.15.100:39074) 2024-07-26T13:46:50.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.122+0000 7f744effd640 1 -- 172.21.15.100:0/455486121 learned_addr learned my addr 172.21.15.100:0/455486121 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:50.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.122+0000 7f744f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d9ac7070 0x55a2d9e7b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:35102/0 (socket says 172.21.15.100:35102) 2024-07-26T13:46:50.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.122+0000 7f744e7fc640 1 -- 172.21.15.100:0/455486121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2d9a9e1b0 msgr2=0x55a2d9e80100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:50.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.122+0000 7f744e7fc640 1 --2- 172.21.15.100:0/455486121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2d9a9e1b0 0x55a2d9e80100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:50.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.122+0000 7f744e7fc640 1 -- 172.21.15.100:0/455486121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d9ac7070 msgr2=0x55a2d9e7b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:50.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.122+0000 7f744e7fc640 1 --2- 172.21.15.100:0/455486121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d9ac7070 0x55a2d9e7b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:50.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.122+0000 7f744e7fc640 1 -- 172.21.15.100:0/455486121 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2d9e7bf60 con 0x55a2d9aa6390 2024-07-26T13:46:50.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.123+0000 7f744effd640 1 --2- 172.21.15.100:0/455486121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2d9a9e1b0 0x55a2d9e80100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:46:50.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.123+0000 7f744e7fc640 1 --2- 172.21.15.100:0/455486121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2d9aa6390 0x55a2d9e806c0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f74380a6000 tx=0x7f74380e4300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8228fa5317f3260c server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:50.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.123+0000 7f74367fc640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:46:50.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.123+0000 7f74367fc640 1 -- 172.21.15.100:0/455486121 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74380aaa60 con 0x55a2d9aa6390 2024-07-26T13:46:50.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.123+0000 7f74367fc640 1 -- 172.21.15.100:0/455486121 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7f74380a71e0 con 0x55a2d9aa6390 2024-07-26T13:46:50.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.123+0000 7f745bb7c740 1 -- 172.21.15.100:0/455486121 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a2d9de0580 con 0x55a2d9aa6390 2024-07-26T13:46:50.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.123+0000 7f74367fc640 1 -- 172.21.15.100:0/455486121 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f74380ab650 con 0x55a2d9aa6390 2024-07-26T13:46:50.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.123+0000 7f745bb7c740 10 client.17202 fetch_fsmap learned FSMap version 53 2024-07-26T13:46:50.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.123+0000 7f745bb7c740 1 -- 172.21.15.100:0/455486121 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55a2d9ef24f0 con 0x55a2d9aa6390 2024-07-26T13:46:50.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.123+0000 7f74367fc640 1 -- 172.21.15.100:0/455486121 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a2d9de0580 con 0x55a2d9aa6390 2024-07-26T13:46:50.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.123+0000 7f74367fc640 1 -- 172.21.15.100:0/455486121 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7f74380ca020 con 0x55a2d9aa6390 2024-07-26T13:46:50.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.123+0000 7f745bb7c740 10 client.17202 fetch_fsmap finished waiting for FSMap version 53 2024-07-26T13:46:50.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.123+0000 7f745bb7c740 10 client.17202 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:46:50.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.124+0000 7f745bb7c740 20 client.17202 populate_metadata read hostname 'smithi100' 2024-07-26T13:46:50.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.124+0000 7f745bb7c740 1 --2- 172.21.15.100:0/455486121 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55a2d9a5ada0 0x55a2d9ef4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:50.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.124+0000 7f745bb7c740 4 client.17202 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-26T13:46:50.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.124+0000 7f745bb7c740 1 -- 172.21.15.100:0/455486121 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a2d9dddcc0 con 0x55a2d9a5ada0 2024-07-26T13:46:50.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.124+0000 7f744effd640 1 --2- 172.21.15.100:0/455486121 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55a2d9a5ada0 0x55a2d9ef4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:50.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.124+0000 7f744effd640 1 --2- 172.21.15.100:0/455486121 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55a2d9a5ada0 0x55a2d9ef4bf0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4a412a28cf9408b server_cookie=6fd91a336d2ba85c in_seq=0 out_seq=0 2024-07-26T13:46:50.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.124+0000 7f74367fc640 10 client.17202 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:50.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.124+0000 7f74367fc640 1 -- 172.21.15.100:0/455486121 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a2d9dddcc0 con 0x55a2d9a5ada0 2024-07-26T13:46:50.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.124+0000 7f74367fc640 10 client.17202 handle_command_reply: tid=0 2024-07-26T13:46:50.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.124+0000 7f745bb7c740 1 client.17202 shutdown 2024-07-26T13:46:50.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.124+0000 7f745bb7c740 1 -- 172.21.15.100:0/455486121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2d9aa6390 msgr2=0x55a2d9e806c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:50.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.124+0000 7f745bb7c740 1 --2- 172.21.15.100:0/455486121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2d9aa6390 0x55a2d9e806c0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f74380a6000 tx=0x7f74380e4300 comp rx=0 tx=0).stop 2024-07-26T13:46:50.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.124+0000 7f745bb7c740 1 -- 172.21.15.100:0/455486121 shutdown_connections 2024-07-26T13:46:50.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.124+0000 7f745bb7c740 1 --2- 172.21.15.100:0/455486121 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55a2d9a5ada0 0x55a2d9ef4bf0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:50.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.124+0000 7f745bb7c740 1 --2- 172.21.15.100:0/455486121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a2d9ac7070 0x55a2d9e7b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:50.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.124+0000 7f745bb7c740 1 --2- 172.21.15.100:0/455486121 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55a2d9aa6390 0x55a2d9e806c0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:50.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.124+0000 7f745bb7c740 1 --2- 172.21.15.100:0/455486121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a2d9a9e1b0 0x55a2d9e80100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:50.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.124+0000 7f74367fc640 0 client.17202 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:50.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.124+0000 7f745bb7c740 1 -- 172.21.15.100:0/455486121 >> 172.21.15.100:0/455486121 conn(0x55a2d9abb460 msgr2=0x55a2d9e71660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:50.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.125+0000 7f745bb7c740 1 -- 172.21.15.100:0/455486121 shutdown_connections 2024-07-26T13:46:50.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.125+0000 7f745bb7c740 1 -- 172.21.15.100:0/455486121 wait complete. 2024-07-26T13:46:50.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.125+0000 7f745bb7c740 20 client.17202 trim_cache size 0 max 16384 2024-07-26T13:46:50.132 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:46:50.132 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-07-26T13:46:50.132 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-07-26T13:46:50.132 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:46:50.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.125+0000 7f74555f8640 1 -- 172.21.15.100:0/4234551195 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f743c0b3d40 msgr2=0x7f743c0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:50.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.125+0000 7f74555f8640 1 --2- 172.21.15.100:0/4234551195 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f743c0b3d40 0x7f743c0b6250 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x55a2d9e6b1d0 tx=0x7f7440005f90 comp rx=0 tx=0).stop 2024-07-26T13:46:50.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.125+0000 7f74555f8640 1 -- 172.21.15.100:0/4234551195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7450107dd0 msgr2=0x7f745019a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:50.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.125+0000 7f74555f8640 1 --2- 172.21.15.100:0/4234551195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7450107dd0 0x7f745019a420 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f743800e9f0 tx=0x7f743800eeb0 comp rx=0 tx=0).stop 2024-07-26T13:46:50.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.125+0000 7f74555f8640 1 -- 172.21.15.100:0/4234551195 shutdown_connections 2024-07-26T13:46:50.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.125+0000 7f74555f8640 1 --2- 172.21.15.100:0/4234551195 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f743c0b3d40 0x7f743c0b6250 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:50.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.125+0000 7f74555f8640 1 --2- 172.21.15.100:0/4234551195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7450108720 0x7f745019f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:50.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.125+0000 7f74555f8640 1 --2- 172.21.15.100:0/4234551195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7450107dd0 0x7f745019a420 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:50.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.125+0000 7f74555f8640 1 --2- 172.21.15.100:0/4234551195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7450102490 0x7f7450199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:50.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.125+0000 7f74555f8640 1 -- 172.21.15.100:0/4234551195 >> 172.21.15.100:0/4234551195 conn(0x7f7450079160 msgr2=0x7f745006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:50.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.125+0000 7f74555f8640 1 -- 172.21.15.100:0/4234551195 shutdown_connections 2024-07-26T13:46:50.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:50.125+0000 7f74555f8640 1 -- 172.21.15.100:0/4234551195 wait complete. 2024-07-26T13:46:50.423 INFO:tasks.workunit.client.0.smithi100.stdout: 1 266467 3.53 MB/sec execute 184 sec latency 39.739 ms 2024-07-26T13:46:51.013 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-26T13:46:51.013 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:46:51.331 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:46:51.423 INFO:tasks.workunit.client.0.smithi100.stdout: 1 268516 3.55 MB/sec execute 185 sec latency 34.261 ms 2024-07-26T13:46:51.720 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:51 smithi100 ceph-mon[32599]: pgmap v569: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.6 MiB/s wr, 269 op/s 2024-07-26T13:46:51.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:51 smithi138 ceph-mon[36954]: pgmap v569: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.6 MiB/s wr, 269 op/s 2024-07-26T13:46:52.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:51 smithi167 ceph-mon[37140]: pgmap v569: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.6 MiB/s wr, 269 op/s 2024-07-26T13:46:52.423 INFO:tasks.workunit.client.0.smithi100.stdout: 1 270324 3.60 MB/sec execute 186 sec latency 13.438 ms 2024-07-26T13:46:53.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.136+0000 7f5d45dc5640 1 -- 172.21.15.100:0/437397884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d40108720 msgr2=0x7f5d4010ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:53.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.136+0000 7f5d45dc5640 1 --2- 172.21.15.100:0/437397884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d40108720 0x7f5d4010ef50 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f5d38009ca0 tx=0x7f5d3802a8c0 comp rx=0 tx=0).stop 2024-07-26T13:46:53.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.136+0000 7f5d45dc5640 1 -- 172.21.15.100:0/437397884 shutdown_connections 2024-07-26T13:46:53.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.136+0000 7f5d45dc5640 1 --2- 172.21.15.100:0/437397884 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d40108720 0x7f5d4010ef50 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:53.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.136+0000 7f5d45dc5640 1 --2- 172.21.15.100:0/437397884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d40107dd0 0x7f5d401081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:53.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.136+0000 7f5d45dc5640 1 --2- 172.21.15.100:0/437397884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d40102490 0x7f5d401028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:53.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.136+0000 7f5d45dc5640 1 -- 172.21.15.100:0/437397884 >> 172.21.15.100:0/437397884 conn(0x7f5d40079160 msgr2=0x7f5d40079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:53.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.136+0000 7f5d45dc5640 1 -- 172.21.15.100:0/437397884 shutdown_connections 2024-07-26T13:46:53.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.136+0000 7f5d45dc5640 1 -- 172.21.15.100:0/437397884 wait complete. 2024-07-26T13:46:53.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.137+0000 7f5d45dc5640 1 Processor -- start 2024-07-26T13:46:53.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.138+0000 7f5d45dc5640 1 -- start start 2024-07-26T13:46:53.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.138+0000 7f5d45dc5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d40102490 0x7f5d40199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:53.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.138+0000 7f5d45dc5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d40107dd0 0x7f5d4019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:53.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.138+0000 7f5d45dc5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d40108720 0x7f5d4019f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:53.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.138+0000 7f5d45dc5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5d401134b0 con 0x7f5d40108720 2024-07-26T13:46:53.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.138+0000 7f5d45dc5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5d40113330 con 0x7f5d40107dd0 2024-07-26T13:46:53.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.138+0000 7f5d45dc5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5d40113630 con 0x7f5d40102490 2024-07-26T13:46:53.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.138+0000 7f5d3f7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d40102490 0x7f5d40199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:53.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.138+0000 7f5d3f7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d40102490 0x7f5d40199ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:48558/0 (socket says 172.21.15.100:48558) 2024-07-26T13:46:53.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.138+0000 7f5d3f7fe640 1 -- 172.21.15.100:0/1145926996 learned_addr learned my addr 172.21.15.100:0/1145926996 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:53.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.138+0000 7f5d3effd640 1 --2- 172.21.15.100:0/1145926996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d40107dd0 0x7f5d4019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:53.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.139+0000 7f5d3ffff640 1 --2- 172.21.15.100:0/1145926996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d40108720 0x7f5d4019f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:53.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.139+0000 7f5d3f7fe640 1 -- 172.21.15.100:0/1145926996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d40107dd0 msgr2=0x7f5d4019a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:53.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.139+0000 7f5d3f7fe640 1 --2- 172.21.15.100:0/1145926996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d40107dd0 0x7f5d4019a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:53.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.139+0000 7f5d3f7fe640 1 -- 172.21.15.100:0/1145926996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d40108720 msgr2=0x7f5d4019f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:53.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.139+0000 7f5d3f7fe640 1 --2- 172.21.15.100:0/1145926996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d40108720 0x7f5d4019f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:53.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.139+0000 7f5d3f7fe640 1 -- 172.21.15.100:0/1145926996 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d4019fe00 con 0x7f5d40102490 2024-07-26T13:46:53.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.139+0000 7f5d3ffff640 1 --2- 172.21.15.100:0/1145926996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d40108720 0x7f5d4019f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:46:53.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.139+0000 7f5d3f7fe640 1 --2- 172.21.15.100:0/1145926996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d40102490 0x7f5d40199ec0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5d3000ea20 tx=0x7f5d3000eee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:53.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.140+0000 7f5d3cff9640 1 -- 172.21.15.100:0/1145926996 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3000cd80 con 0x7f5d40102490 2024-07-26T13:46:53.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.140+0000 7f5d3cff9640 1 -- 172.21.15.100:0/1145926996 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d300108e0 con 0x7f5d40102490 2024-07-26T13:46:53.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.140+0000 7f5d45dc5640 1 -- 172.21.15.100:0/1145926996 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d401a00f0 con 0x7f5d40102490 2024-07-26T13:46:53.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.140+0000 7f5d3cff9640 1 -- 172.21.15.100:0/1145926996 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d30018840 con 0x7f5d40102490 2024-07-26T13:46:53.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.140+0000 7f5d45dc5640 1 -- 172.21.15.100:0/1145926996 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d4006c810 con 0x7f5d40102490 2024-07-26T13:46:53.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.141+0000 7f5d45dc5640 1 -- 172.21.15.100:0/1145926996 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d401028a0 con 0x7f5d40102490 2024-07-26T13:46:53.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.142+0000 7f5d3cff9640 1 -- 172.21.15.100:0/1145926996 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f5d30010430 con 0x7f5d40102490 2024-07-26T13:46:53.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.143+0000 7f5d3cff9640 1 --2- 172.21.15.100:0/1145926996 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5d280b3dc0 0x7f5d280b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:53.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.143+0000 7f5d3effd640 1 --2- 172.21.15.100:0/1145926996 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5d280b3dc0 0x7f5d280b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:53.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.143+0000 7f5d3cff9640 1 -- 172.21.15.100:0/1145926996 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7f5d300e4720 con 0x7f5d40102490 2024-07-26T13:46:53.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.143+0000 7f5d3effd640 1 --2- 172.21.15.100:0/1145926996 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5d280b3dc0 0x7f5d280b62d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5d40069e20 tx=0x7f5d34005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:53.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.146+0000 7f5d3cff9640 1 -- 172.21.15.100:0/1145926996 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f5d300ab950 con 0x7f5d40102490 2024-07-26T13:46:53.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.375+0000 7f5d45dc5640 1 -- 172.21.15.100:0/1145926996 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d4019aff0 con 0x7f5d40102490 2024-07-26T13:46:53.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.376+0000 7f5d3cff9640 1 -- 172.21.15.100:0/1145926996 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11929 (secure 0 0 0) 0x7f5d300ab250 con 0x7f5d40102490 2024-07-26T13:46:53.378 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:46:53.379 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":54,"btime":"2024-07-26T13:46:52:579523+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14790,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/313501409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":313501409},{"type":"v1","addr":"172.21.15.100:6839","nonce":313501409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/1402975506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":1402975506},{"type":"v1","addr":"172.21.15.100:6841","nonce":1402975506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":54,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-26T13:37:01.281843+0000","modified":"2024-07-26T13:46:52.241071+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24607,"mds_1":24605,"mds_2":25997},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24575":{"gid":24575,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/3393225483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3393225483},{"type":"v1","addr":"172.21.15.138:6837","nonce":3393225483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.138:6839/1580925471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1580925471},{"type":"v1","addr":"172.21.15.138:6839","nonce":1580925471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/398150514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":398150514},{"type":"v1","addr":"172.21.15.167:6839","nonce":398150514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25997":{"gid":25997,"name":"e","rank":2,"incarnation":49,"state":"up:active","state_seq":23,"addr":"172.21.15.138:6835/2831339503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2831339503},{"type":"v1","addr":"172.21.15.138:6835","nonce":2831339503}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26446":{"gid":26446,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/3519222898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3519222898},{"type":"v1","addr":"172.21.15.167:6837","nonce":3519222898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24607,"qdb_cluster":[25997,24607,24605]},"id":1}]} 2024-07-26T13:46:53.379 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 54 2024-07-26T13:46:53.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.378+0000 7f5d45dc5640 1 -- 172.21.15.100:0/1145926996 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5d280b3dc0 msgr2=0x7f5d280b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:53.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.378+0000 7f5d45dc5640 1 --2- 172.21.15.100:0/1145926996 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5d280b3dc0 0x7f5d280b62d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5d40069e20 tx=0x7f5d34005f90 comp rx=0 tx=0).stop 2024-07-26T13:46:53.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.378+0000 7f5d45dc5640 1 -- 172.21.15.100:0/1145926996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d40102490 msgr2=0x7f5d40199ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:53.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.378+0000 7f5d45dc5640 1 --2- 172.21.15.100:0/1145926996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d40102490 0x7f5d40199ec0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5d3000ea20 tx=0x7f5d3000eee0 comp rx=0 tx=0).stop 2024-07-26T13:46:53.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.378+0000 7f5d45dc5640 1 -- 172.21.15.100:0/1145926996 shutdown_connections 2024-07-26T13:46:53.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.378+0000 7f5d45dc5640 1 --2- 172.21.15.100:0/1145926996 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f5d280b3dc0 0x7f5d280b62d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:53.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.378+0000 7f5d45dc5640 1 --2- 172.21.15.100:0/1145926996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5d40108720 0x7f5d4019f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:53.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.378+0000 7f5d45dc5640 1 --2- 172.21.15.100:0/1145926996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d40107dd0 0x7f5d4019a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:53.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.378+0000 7f5d45dc5640 1 --2- 172.21.15.100:0/1145926996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d40102490 0x7f5d40199ec0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:53.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.378+0000 7f5d45dc5640 1 -- 172.21.15.100:0/1145926996 >> 172.21.15.100:0/1145926996 conn(0x7f5d40079160 msgr2=0x7f5d40103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:53.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.379+0000 7f5d45dc5640 1 -- 172.21.15.100:0/1145926996 shutdown_connections 2024-07-26T13:46:53.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:53.379+0000 7f5d45dc5640 1 -- 172.21.15.100:0/1145926996 wait complete. 2024-07-26T13:46:53.422 INFO:tasks.workunit.client.0.smithi100.stdout: 1 272011 3.62 MB/sec execute 187 sec latency 20.308 ms 2024-07-26T13:46:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:53 smithi100 ceph-mon[32599]: pgmap v570: 97 pgs: 97 active+clean; 98 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 250 op/s 2024-07-26T13:46:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:53 smithi100 ceph-mon[32599]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:46:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:53 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/1145926996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:46:53.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:53 smithi138 ceph-mon[36954]: pgmap v570: 97 pgs: 97 active+clean; 98 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 250 op/s 2024-07-26T13:46:53.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:53 smithi138 ceph-mon[36954]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:46:53.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:53 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/1145926996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:46:54.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:53 smithi167 ceph-mon[37140]: pgmap v570: 97 pgs: 97 active+clean; 98 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 250 op/s 2024-07-26T13:46:54.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:53 smithi167 ceph-mon[37140]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:46:54.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:53 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/1145926996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:46:54.317 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-26T13:46:54.422 INFO:tasks.workunit.client.0.smithi100.stdout: 1 273734 3.66 MB/sec execute 188 sec latency 11.446 ms 2024-07-26T13:46:54.585 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:46:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:54 smithi138 ceph-mon[36954]: pgmap v571: 97 pgs: 97 active+clean; 101 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 332 op/s 2024-07-26T13:46:54.951 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:54 smithi100 ceph-mon[32599]: pgmap v571: 97 pgs: 97 active+clean; 101 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 332 op/s 2024-07-26T13:46:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:54 smithi167 ceph-mon[37140]: pgmap v571: 97 pgs: 97 active+clean; 101 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 332 op/s 2024-07-26T13:46:55.423 INFO:tasks.workunit.client.0.smithi100.stdout: 1 275643 3.68 MB/sec execute 189 sec latency 23.008 ms 2024-07-26T13:46:56.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.382+0000 7fee6f233640 1 -- 172.21.15.100:0/2121574550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee68102490 msgr2=0x7fee681028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:56.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.382+0000 7fee6f233640 1 --2- 172.21.15.100:0/2121574550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee68102490 0x7fee681028a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009900 tx=0x7fee5c02a410 comp rx=0 tx=0).stop 2024-07-26T13:46:56.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.383+0000 7fee6f233640 1 -- 172.21.15.100:0/2121574550 shutdown_connections 2024-07-26T13:46:56.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.383+0000 7fee6f233640 1 --2- 172.21.15.100:0/2121574550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee68108720 0x7fee6810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:56.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.383+0000 7fee6f233640 1 --2- 172.21.15.100:0/2121574550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee68107dd0 0x7fee681081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:56.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.383+0000 7fee6f233640 1 --2- 172.21.15.100:0/2121574550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee68102490 0x7fee681028a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:56.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.383+0000 7fee6f233640 1 -- 172.21.15.100:0/2121574550 >> 172.21.15.100:0/2121574550 conn(0x7fee68079160 msgr2=0x7fee68079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:56.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.383+0000 7fee6f233640 1 -- 172.21.15.100:0/2121574550 shutdown_connections 2024-07-26T13:46:56.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.383+0000 7fee6f233640 1 -- 172.21.15.100:0/2121574550 wait complete. 2024-07-26T13:46:56.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.383+0000 7fee6f233640 1 Processor -- start 2024-07-26T13:46:56.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.383+0000 7fee6f233640 1 -- start start 2024-07-26T13:46:56.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.384+0000 7fee6f233640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee68102490 0x7fee68199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:56.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.384+0000 7fee6f233640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee68107dd0 0x7fee6819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:56.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.384+0000 7fee6f233640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee68108720 0x7fee6819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:56.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.384+0000 7fee6f233640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fee681134b0 con 0x7fee68108720 2024-07-26T13:46:56.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.384+0000 7fee6f233640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fee68113330 con 0x7fee68107dd0 2024-07-26T13:46:56.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.384+0000 7fee6f233640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fee68113630 con 0x7fee68102490 2024-07-26T13:46:56.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.384+0000 7fee6cfcf640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee68102490 0x7fee68199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:56.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.384+0000 7fee6d7d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee68108720 0x7fee6819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:56.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.384+0000 7fee67fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee68107dd0 0x7fee6819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:56.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.384+0000 7fee6cfcf640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee68102490 0x7fee68199e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:35096/0 (socket says 172.21.15.100:35096) 2024-07-26T13:46:56.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.384+0000 7fee6cfcf640 1 -- 172.21.15.100:0/1937721714 learned_addr learned my addr 172.21.15.100:0/1937721714 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:56.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.384+0000 7fee67fff640 1 -- 172.21.15.100:0/1937721714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee68102490 msgr2=0x7fee68199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:56.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.384+0000 7fee67fff640 1 --2- 172.21.15.100:0/1937721714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee68102490 0x7fee68199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:56.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.384+0000 7fee67fff640 1 -- 172.21.15.100:0/1937721714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee68108720 msgr2=0x7fee6819f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:56.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.384+0000 7fee67fff640 1 --2- 172.21.15.100:0/1937721714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee68108720 0x7fee6819f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:56.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.384+0000 7fee67fff640 1 -- 172.21.15.100:0/1937721714 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee6819fdd0 con 0x7fee68107dd0 2024-07-26T13:46:56.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.384+0000 7fee67fff640 1 --2- 172.21.15.100:0/1937721714 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee68107dd0 0x7fee6819a400 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fee5400e9c0 tx=0x7fee5400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:56.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.385+0000 7fee65ffb640 1 -- 172.21.15.100:0/1937721714 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5400ccc0 con 0x7fee68107dd0 2024-07-26T13:46:56.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.385+0000 7fee6f233640 1 -- 172.21.15.100:0/1937721714 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee681a00c0 con 0x7fee68107dd0 2024-07-26T13:46:56.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.385+0000 7fee65ffb640 1 -- 172.21.15.100:0/1937721714 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee540108e0 con 0x7fee68107dd0 2024-07-26T13:46:56.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.385+0000 7fee65ffb640 1 -- 172.21.15.100:0/1937721714 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee540187a0 con 0x7fee68107dd0 2024-07-26T13:46:56.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.385+0000 7fee6f233640 1 -- 172.21.15.100:0/1937721714 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee681a6820 con 0x7fee68107dd0 2024-07-26T13:46:56.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.388+0000 7fee757b7740 1 Processor -- start 2024-07-26T13:46:56.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.389+0000 7fee65ffb640 1 -- 172.21.15.100:0/1937721714 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fee54018940 con 0x7fee68107dd0 2024-07-26T13:46:56.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.389+0000 7fee757b7740 1 -- start start 2024-07-26T13:46:56.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.389+0000 7fee757b7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558edae24070 0x558edb1c9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:56.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.389+0000 7fee757b7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558edadfb1b0 0x558edb1c9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:56.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.389+0000 7fee757b7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558edae03390 0x558edb1c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:56.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.389+0000 7fee757b7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x558edb13fbb0 con 0x558edae24070 2024-07-26T13:46:56.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.389+0000 7fee757b7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558edb12f0b0 con 0x558edadfb1b0 2024-07-26T13:46:56.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.389+0000 7fee757b7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558edb1330f0 con 0x558edae03390 2024-07-26T13:46:56.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.389+0000 7fee6cfcf640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558edadfb1b0 0x558edb1c9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:56.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.389+0000 7fee67fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558edae03390 0x558edb1c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:56.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.389+0000 7fee6d7d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558edae24070 0x558edb1c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:56.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.389+0000 7fee6cfcf640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558edadfb1b0 0x558edb1c9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:42212/0 (socket says 172.21.15.100:42212) 2024-07-26T13:46:56.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.389+0000 7fee6cfcf640 1 -- 172.21.15.100:0/1163522748 learned_addr learned my addr 172.21.15.100:0/1163522748 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:56.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.390+0000 7fee6cfcf640 1 -- 172.21.15.100:0/1163522748 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558edae03390 msgr2=0x558edb1c9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:56.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.390+0000 7fee6cfcf640 1 --2- 172.21.15.100:0/1163522748 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558edae03390 0x558edb1c9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:56.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.390+0000 7fee6cfcf640 1 -- 172.21.15.100:0/1163522748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558edae24070 msgr2=0x558edb1c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:56.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.390+0000 7fee6cfcf640 1 --2- 172.21.15.100:0/1163522748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558edae24070 0x558edb1c9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:56.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.390+0000 7fee6cfcf640 1 -- 172.21.15.100:0/1163522748 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558edb0d5b90 con 0x558edadfb1b0 2024-07-26T13:46:56.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.390+0000 7fee6d7d0640 1 --2- 172.21.15.100:0/1163522748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558edae24070 0x558edb1c9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:46:56.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.390+0000 7fee6cfcf640 1 --2- 172.21.15.100:0/1163522748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558edadfb1b0 0x558edb1c9ab0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fee5c000c00 tx=0x7fee5c031b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eec2a2ec9da5e4e8 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:56.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.390+0000 7fee65ffb640 1 --2- 172.21.15.100:0/1937721714 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fee580b3d40 0x7fee580b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:56.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.390+0000 7fee6d7d0640 1 --2- 172.21.15.100:0/1937721714 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fee580b3d40 0x7fee580b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:56.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.390+0000 7fee52ffd640 1 -- 172.21.15.100:0/1163522748 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c040070 con 0x558edadfb1b0 2024-07-26T13:46:56.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.390+0000 7fee52ffd640 1 -- 172.21.15.100:0/1163522748 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee5c004030 con 0x558edadfb1b0 2024-07-26T13:46:56.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.390+0000 7fee52ffd640 1 -- 172.21.15.100:0/1163522748 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c035040 con 0x558edadfb1b0 2024-07-26T13:46:56.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.390+0000 7fee65ffb640 1 -- 172.21.15.100:0/1937721714 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7fee540e4400 con 0x7fee68107dd0 2024-07-26T13:46:56.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.391+0000 7fee757b7740 1 -- 172.21.15.100:0/1163522748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558edadfb1b0 msgr2=0x558edb1c9ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:56.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.391+0000 7fee6d7d0640 1 --2- 172.21.15.100:0/1937721714 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fee580b3d40 0x7fee580b6250 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fee60009fd0 tx=0x7fee60009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:56.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.391+0000 7fee757b7740 1 --2- 172.21.15.100:0/1163522748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558edadfb1b0 0x558edb1c9ab0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fee5c000c00 tx=0x7fee5c031b10 comp rx=0 tx=0).stop 2024-07-26T13:46:56.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.391+0000 7fee757b7740 1 -- 172.21.15.100:0/1163522748 shutdown_connections 2024-07-26T13:46:56.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.391+0000 7fee757b7740 1 --2- 172.21.15.100:0/1163522748 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558edae03390 0x558edb1c9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:56.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.391+0000 7fee757b7740 1 --2- 172.21.15.100:0/1163522748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558edadfb1b0 0x558edb1c9ab0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:56.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.391+0000 7fee757b7740 1 --2- 172.21.15.100:0/1163522748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558edae24070 0x558edb1c9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:56.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.391+0000 7fee757b7740 1 -- 172.21.15.100:0/1163522748 >> 172.21.15.100:0/1163522748 conn(0x558edae18460 msgr2=0x558edb1c7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:56.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.392+0000 7fee757b7740 1 -- 172.21.15.100:0/1163522748 shutdown_connections 2024-07-26T13:46:56.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.392+0000 7fee757b7740 1 -- 172.21.15.100:0/1163522748 wait complete. 2024-07-26T13:46:56.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.392+0000 7fee757b7740 1 -- start start 2024-07-26T13:46:56.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.392+0000 7fee757b7740 1 Processor -- start 2024-07-26T13:46:56.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.393+0000 7fee757b7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558edadfb1b0 0x558edb1d9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:56.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.393+0000 7fee757b7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558edae03390 0x558edb1d9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:56.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.393+0000 7fee757b7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558edae24070 0x558edb1e5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:56.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.393+0000 7fee757b7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x558edaf92700 con 0x558edae24070 2024-07-26T13:46:56.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.393+0000 7fee757b7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558edb0d19e0 con 0x558edae03390 2024-07-26T13:46:56.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.393+0000 7fee757b7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558edb1dbff0 con 0x558edadfb1b0 2024-07-26T13:46:56.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.393+0000 7fee757b7740 10 client.0 _ll_register_callbacks cb 0x558edb0ec4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:46:56.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.393+0000 7fee67fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558edae03390 0x558edb1d9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:56.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.393+0000 7fee6d7d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558edae24070 0x558edb1e5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:56.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.393+0000 7fee67fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558edae03390 0x558edb1d9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:42224/0 (socket says 172.21.15.100:42224) 2024-07-26T13:46:56.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.393+0000 7fee6cfcf640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558edadfb1b0 0x558edb1d9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:56.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.393+0000 7fee67fff640 1 -- 172.21.15.100:0/2947592703 learned_addr learned my addr 172.21.15.100:0/2947592703 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:56.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.393+0000 7fee67fff640 1 -- 172.21.15.100:0/2947592703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558edadfb1b0 msgr2=0x558edb1d9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:56.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.393+0000 7fee67fff640 1 --2- 172.21.15.100:0/2947592703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558edadfb1b0 0x558edb1d9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:56.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.393+0000 7fee67fff640 1 -- 172.21.15.100:0/2947592703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558edae24070 msgr2=0x558edb1e5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:56.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.393+0000 7fee67fff640 1 --2- 172.21.15.100:0/2947592703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558edae24070 0x558edb1e5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:56.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.393+0000 7fee67fff640 1 -- 172.21.15.100:0/2947592703 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558edae8f650 con 0x558edae03390 2024-07-26T13:46:56.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.393+0000 7fee6d7d0640 1 --2- 172.21.15.100:0/2947592703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558edae24070 0x558edb1e5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:46:56.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.393+0000 7fee67fff640 1 --2- 172.21.15.100:0/2947592703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558edae03390 0x558edb1d9a40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fee540a8250 tx=0x7fee54009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61c1397122411a2e server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:56.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.393+0000 7fee47fff640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-07-26T13:46:56.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.393+0000 7fee47fff640 1 -- 172.21.15.100:0/2947592703 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee540aa750 con 0x558edae03390 2024-07-26T13:46:56.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.393+0000 7fee47fff640 1 -- 172.21.15.100:0/2947592703 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7fee54010430 con 0x558edae03390 2024-07-26T13:46:56.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.394+0000 7fee757b7740 1 -- 172.21.15.100:0/2947592703 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee5c004030 con 0x558edae03390 2024-07-26T13:46:56.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.394+0000 7fee47fff640 1 -- 172.21.15.100:0/2947592703 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fee540aaaf0 con 0x558edae03390 2024-07-26T13:46:56.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.394+0000 7fee757b7740 10 client.26441 fetch_fsmap learned FSMap version 54 2024-07-26T13:46:56.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.394+0000 7fee757b7740 1 -- 172.21.15.100:0/2947592703 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x558edb1e7ff0 con 0x558edae03390 2024-07-26T13:46:56.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.394+0000 7fee47fff640 1 -- 172.21.15.100:0/2947592703 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee5c004030 con 0x558edae03390 2024-07-26T13:46:56.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.394+0000 7fee47fff640 1 -- 172.21.15.100:0/2947592703 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 54) ==== 8350+0+0 (secure 0 0 0) 0x7fee540ae030 con 0x558edae03390 2024-07-26T13:46:56.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.394+0000 7fee757b7740 10 client.26441 fetch_fsmap finished waiting for FSMap version 54 2024-07-26T13:46:56.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.394+0000 7fee757b7740 10 client.26441 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-26T13:46:56.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.394+0000 7fee757b7740 20 client.26441 populate_metadata read hostname 'smithi100' 2024-07-26T13:46:56.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.394+0000 7fee757b7740 1 --2- 172.21.15.100:0/2947592703 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x558edade29f0 0x558edb1eb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:56.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.394+0000 7fee757b7740 4 client.26441 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:46:56.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.394+0000 7fee757b7740 1 -- 172.21.15.100:0/2947592703 --> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558edb141b00 con 0x558edade29f0 2024-07-26T13:46:56.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.394+0000 7fee6cfcf640 1 --2- 172.21.15.100:0/2947592703 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x558edade29f0 0x558edb1eb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:56.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.395+0000 7fee47fff640 10 client.26441 ms_handle_connect on v2:172.21.15.138:6838/1580925471 2024-07-26T13:46:56.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.395+0000 7fee6cfcf640 1 --2- 172.21.15.100:0/2947592703 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x558edade29f0 0x558edb1eb580 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=44c4bdeb0e0f7411 server_cookie=bb67aaa6384d550d in_seq=0 out_seq=0 2024-07-26T13:46:56.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.396+0000 7fee47fff640 1 -- 172.21.15.100:0/2947592703 <== mds.1 v2:172.21.15.138:6838/1580925471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558edb141b00 con 0x558edade29f0 2024-07-26T13:46:56.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.396+0000 7fee47fff640 10 client.26441 handle_command_reply: tid=0 2024-07-26T13:46:56.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.396+0000 7fee757b7740 1 client.26441 shutdown 2024-07-26T13:46:56.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.396+0000 7fee757b7740 1 -- 172.21.15.100:0/2947592703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558edae03390 msgr2=0x558edb1d9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:56.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.396+0000 7fee757b7740 1 --2- 172.21.15.100:0/2947592703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558edae03390 0x558edb1d9a40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fee540a8250 tx=0x7fee54009700 comp rx=0 tx=0).stop 2024-07-26T13:46:56.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.396+0000 7fee757b7740 1 -- 172.21.15.100:0/2947592703 shutdown_connections 2024-07-26T13:46:56.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.396+0000 7fee757b7740 1 --2- 172.21.15.100:0/2947592703 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x558edade29f0 0x558edb1eb580 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:56.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.396+0000 7fee757b7740 1 --2- 172.21.15.100:0/2947592703 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558edae24070 0x558edb1e5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:56.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.396+0000 7fee757b7740 1 --2- 172.21.15.100:0/2947592703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558edae03390 0x558edb1d9a40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:56.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.396+0000 7fee757b7740 1 --2- 172.21.15.100:0/2947592703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558edadfb1b0 0x558edb1d9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:56.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.396+0000 7fee47fff640 0 client.26441 ms_handle_reset on v2:172.21.15.138:6838/1580925471 2024-07-26T13:46:56.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.396+0000 7fee757b7740 1 -- 172.21.15.100:0/2947592703 >> 172.21.15.100:0/2947592703 conn(0x558edae18460 msgr2=0x558edb1c5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:56.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.396+0000 7fee757b7740 1 -- 172.21.15.100:0/2947592703 shutdown_connections 2024-07-26T13:46:56.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.396+0000 7fee757b7740 1 -- 172.21.15.100:0/2947592703 wait complete. 2024-07-26T13:46:56.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.396+0000 7fee757b7740 20 client.26441 trim_cache size 0 max 16384 2024-07-26T13:46:56.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.410+0000 7fee757b7740 1 Processor -- start 2024-07-26T13:46:56.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.410+0000 7fee757b7740 1 -- start start 2024-07-26T13:46:56.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.410+0000 7fee757b7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558edadfb1b0 0x558edb1c74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:56.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.410+0000 7fee757b7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558edae03390 0x558edb1e5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:56.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.410+0000 7fee757b7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558edae24070 0x558edb1f0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:56.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.410+0000 7fee757b7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fee681af9b0 con 0x558edadfb1b0 2024-07-26T13:46:56.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.410+0000 7fee757b7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fee681ad250 con 0x558edae03390 2024-07-26T13:46:56.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.410+0000 7fee757b7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fee681ab0a0 con 0x558edae24070 2024-07-26T13:46:56.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.410+0000 7fee6cfcf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558edadfb1b0 0x558edb1c74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:56.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.410+0000 7fee6cfcf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558edadfb1b0 0x558edb1c74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34610/0 (socket says 172.21.15.100:34610) 2024-07-26T13:46:56.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.410+0000 7fee6cfcf640 1 -- 172.21.15.100:0/3733154091 learned_addr learned my addr 172.21.15.100:0/3733154091 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:56.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.410+0000 7fee6d7d0640 1 --2- 172.21.15.100:0/3733154091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558edae24070 0x558edb1f0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:56.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.410+0000 7fee67fff640 1 --2- 172.21.15.100:0/3733154091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558edae03390 0x558edb1e5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:56.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.410+0000 7fee67fff640 1 -- 172.21.15.100:0/3733154091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558edae24070 msgr2=0x558edb1f0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:56.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.410+0000 7fee67fff640 1 --2- 172.21.15.100:0/3733154091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558edae24070 0x558edb1f0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:56.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.410+0000 7fee67fff640 1 -- 172.21.15.100:0/3733154091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558edadfb1b0 msgr2=0x558edb1c74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:56.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.410+0000 7fee67fff640 1 --2- 172.21.15.100:0/3733154091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558edadfb1b0 0x558edb1c74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:56.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.410+0000 7fee67fff640 1 -- 172.21.15.100:0/3733154091 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558edb1d4b90 con 0x558edae03390 2024-07-26T13:46:56.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.410+0000 7fee6cfcf640 1 --2- 172.21.15.100:0/3733154091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558edadfb1b0 0x558edb1c74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:46:56.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.411+0000 7fee67fff640 1 --2- 172.21.15.100:0/3733154091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558edae03390 0x558edb1e5110 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fee540aaf70 tx=0x7fee5400efb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9fdbdec1b8bb7d6 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:56.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.411+0000 7fee527fc640 1 -- 172.21.15.100:0/3733154091 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee540aa750 con 0x558edae03390 2024-07-26T13:46:56.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.411+0000 7fee527fc640 1 -- 172.21.15.100:0/3733154091 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee540aafe0 con 0x558edae03390 2024-07-26T13:46:56.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.411+0000 7fee527fc640 1 -- 172.21.15.100:0/3733154091 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee540a9cb0 con 0x558edae03390 2024-07-26T13:46:56.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.412+0000 7fee757b7740 1 -- 172.21.15.100:0/3733154091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558edae03390 msgr2=0x558edb1e5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:56.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.412+0000 7fee757b7740 1 --2- 172.21.15.100:0/3733154091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558edae03390 0x558edb1e5110 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fee540aaf70 tx=0x7fee5400efb0 comp rx=0 tx=0).stop 2024-07-26T13:46:56.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.412+0000 7fee757b7740 1 -- 172.21.15.100:0/3733154091 shutdown_connections 2024-07-26T13:46:56.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.412+0000 7fee757b7740 1 --2- 172.21.15.100:0/3733154091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558edae24070 0x558edb1f0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:56.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.412+0000 7fee757b7740 1 --2- 172.21.15.100:0/3733154091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558edae03390 0x558edb1e5110 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:56.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.412+0000 7fee757b7740 1 --2- 172.21.15.100:0/3733154091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558edadfb1b0 0x558edb1c74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:56.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.412+0000 7fee757b7740 1 -- 172.21.15.100:0/3733154091 >> 172.21.15.100:0/3733154091 conn(0x558edae18460 msgr2=0x558edb1c6da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:56.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.412+0000 7fee757b7740 1 -- 172.21.15.100:0/3733154091 shutdown_connections 2024-07-26T13:46:56.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.412+0000 7fee757b7740 1 -- 172.21.15.100:0/3733154091 wait complete. 2024-07-26T13:46:56.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.412+0000 7fee757b7740 1 -- start start 2024-07-26T13:46:56.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.412+0000 7fee757b7740 1 Processor -- start 2024-07-26T13:46:56.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.413+0000 7fee757b7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558edadfb1b0 0x558edb1e17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:56.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.413+0000 7fee757b7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558edae03390 0x558edb1dccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:56.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.413+0000 7fee757b7740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558edae24070 0x558edb1dd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:56.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.413+0000 7fee757b7740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fee681ab710 con 0x558edae24070 2024-07-26T13:46:56.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.413+0000 7fee757b7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fee681acb80 con 0x558edadfb1b0 2024-07-26T13:46:56.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.413+0000 7fee757b7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fee681abde0 con 0x558edae03390 2024-07-26T13:46:56.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.413+0000 7fee757b7740 10 client.0 _ll_register_callbacks cb 0x7fee540e7c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:46:56.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.413+0000 7fee6cfcf640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558edadfb1b0 0x558edb1e17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:56.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.413+0000 7fee6cfcf640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558edadfb1b0 0x558edb1e17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:42242/0 (socket says 172.21.15.100:42242) 2024-07-26T13:46:56.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.413+0000 7fee6cfcf640 1 -- 172.21.15.100:0/3150241315 learned_addr learned my addr 172.21.15.100:0/3150241315 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:56.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.413+0000 7fee6d7d0640 1 --2- 172.21.15.100:0/3150241315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558edae24070 0x558edb1dd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:56.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.413+0000 7fee67fff640 1 --2- 172.21.15.100:0/3150241315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558edae03390 0x558edb1dccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:56.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.413+0000 7fee6cfcf640 1 -- 172.21.15.100:0/3150241315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558edae03390 msgr2=0x558edb1dccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:56.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.413+0000 7fee6cfcf640 1 --2- 172.21.15.100:0/3150241315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558edae03390 0x558edb1dccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:56.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.413+0000 7fee6cfcf640 1 -- 172.21.15.100:0/3150241315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558edae24070 msgr2=0x558edb1dd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:56.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.413+0000 7fee6cfcf640 1 --2- 172.21.15.100:0/3150241315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558edae24070 0x558edb1dd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:56.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.413+0000 7fee6cfcf640 1 -- 172.21.15.100:0/3150241315 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558edb1dd7e0 con 0x558edadfb1b0 2024-07-26T13:46:56.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.414+0000 7fee6cfcf640 1 --2- 172.21.15.100:0/3150241315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558edadfb1b0 0x558edb1e17b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fee5c0096c0 tx=0x7fee5c005280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=730a0e83e6577c67 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:56.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.414+0000 7fee47fff640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-07-26T13:46:56.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.414+0000 7fee47fff640 1 -- 172.21.15.100:0/3150241315 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c053070 con 0x558edadfb1b0 2024-07-26T13:46:56.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.414+0000 7fee47fff640 1 -- 172.21.15.100:0/3150241315 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7fee5c0316a0 con 0x558edadfb1b0 2024-07-26T13:46:56.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.414+0000 7fee757b7740 1 -- 172.21.15.100:0/3150241315 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee540aafe0 con 0x558edadfb1b0 2024-07-26T13:46:56.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.414+0000 7fee47fff640 1 -- 172.21.15.100:0/3150241315 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fee5c05f050 con 0x558edadfb1b0 2024-07-26T13:46:56.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.414+0000 7fee757b7740 10 client.26447 fetch_fsmap learned FSMap version 54 2024-07-26T13:46:56.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.414+0000 7fee757b7740 1 -- 172.21.15.100:0/3150241315 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x558edb253ae0 con 0x558edadfb1b0 2024-07-26T13:46:56.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.414+0000 7fee47fff640 1 -- 172.21.15.100:0/3150241315 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee540aafe0 con 0x558edadfb1b0 2024-07-26T13:46:56.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.414+0000 7fee47fff640 1 -- 172.21.15.100:0/3150241315 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 54) ==== 8350+0+0 (secure 0 0 0) 0x7fee5c04f3a0 con 0x558edadfb1b0 2024-07-26T13:46:56.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.414+0000 7fee757b7740 10 client.26447 fetch_fsmap finished waiting for FSMap version 54 2024-07-26T13:46:56.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.414+0000 7fee757b7740 10 client.26447 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-26T13:46:56.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.414+0000 7fee757b7740 20 client.26447 populate_metadata read hostname 'smithi100' 2024-07-26T13:46:56.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.414+0000 7fee757b7740 1 --2- 172.21.15.100:0/3150241315 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x558edadb7da0 0x558edb255e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:56.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.414+0000 7fee757b7740 4 client.26447 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-26T13:46:56.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.414+0000 7fee757b7740 1 -- 172.21.15.100:0/3150241315 --> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558edb13d5a0 con 0x558edadb7da0 2024-07-26T13:46:56.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.415+0000 7fee67fff640 1 --2- 172.21.15.100:0/3150241315 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x558edadb7da0 0x558edb255e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:56.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.415+0000 7fee47fff640 10 client.26447 ms_handle_connect on v2:172.21.15.138:6838/1580925471 2024-07-26T13:46:56.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.415+0000 7fee67fff640 1 --2- 172.21.15.100:0/3150241315 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x558edadb7da0 0x558edb255e90 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=35c0ea5041aba735 server_cookie=7f0663cafceb8438 in_seq=0 out_seq=0 2024-07-26T13:46:56.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.415+0000 7fee47fff640 1 -- 172.21.15.100:0/3150241315 <== mds.1 v2:172.21.15.138:6838/1580925471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558edb13d5a0 con 0x558edadb7da0 2024-07-26T13:46:56.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.415+0000 7fee47fff640 10 client.26447 handle_command_reply: tid=0 2024-07-26T13:46:56.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.415+0000 7fee757b7740 1 client.26447 shutdown 2024-07-26T13:46:56.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.415+0000 7fee757b7740 1 -- 172.21.15.100:0/3150241315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558edadfb1b0 msgr2=0x558edb1e17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:56.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.415+0000 7fee757b7740 1 --2- 172.21.15.100:0/3150241315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558edadfb1b0 0x558edb1e17b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fee5c0096c0 tx=0x7fee5c005280 comp rx=0 tx=0).stop 2024-07-26T13:46:56.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.416+0000 7fee757b7740 1 -- 172.21.15.100:0/3150241315 shutdown_connections 2024-07-26T13:46:56.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.416+0000 7fee757b7740 1 --2- 172.21.15.100:0/3150241315 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x558edadb7da0 0x558edb255e90 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:56.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.416+0000 7fee47fff640 0 client.26447 ms_handle_reset on v2:172.21.15.138:6838/1580925471 2024-07-26T13:46:56.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.416+0000 7fee757b7740 1 --2- 172.21.15.100:0/3150241315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x558edae24070 0x558edb1dd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:56.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.416+0000 7fee757b7740 1 --2- 172.21.15.100:0/3150241315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558edae03390 0x558edb1dccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:56.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.416+0000 7fee757b7740 1 --2- 172.21.15.100:0/3150241315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558edadfb1b0 0x558edb1e17b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:56.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.416+0000 7fee757b7740 1 -- 172.21.15.100:0/3150241315 >> 172.21.15.100:0/3150241315 conn(0x558edae18460 msgr2=0x558edb1ca5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:56.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.416+0000 7fee757b7740 1 -- 172.21.15.100:0/3150241315 shutdown_connections 2024-07-26T13:46:56.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.416+0000 7fee757b7740 1 -- 172.21.15.100:0/3150241315 wait complete. 2024-07-26T13:46:56.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.416+0000 7fee757b7740 20 client.26447 trim_cache size 0 max 16384 2024-07-26T13:46:56.423 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-07-26T13:46:56.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.416+0000 7fee6f233640 1 -- 172.21.15.100:0/1937721714 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fee580b3d40 msgr2=0x7fee580b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:56.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.416+0000 7fee6f233640 1 --2- 172.21.15.100:0/1937721714 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fee580b3d40 0x7fee580b6250 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fee60009fd0 tx=0x7fee60009500 comp rx=0 tx=0).stop 2024-07-26T13:46:56.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.417+0000 7fee6f233640 1 -- 172.21.15.100:0/1937721714 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee68107dd0 msgr2=0x7fee6819a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:56.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.417+0000 7fee6f233640 1 --2- 172.21.15.100:0/1937721714 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee68107dd0 0x7fee6819a400 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fee5400e9c0 tx=0x7fee5400ee80 comp rx=0 tx=0).stop 2024-07-26T13:46:56.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.417+0000 7fee6f233640 1 -- 172.21.15.100:0/1937721714 shutdown_connections 2024-07-26T13:46:56.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.417+0000 7fee6f233640 1 --2- 172.21.15.100:0/1937721714 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fee580b3d40 0x7fee580b6250 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:56.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.417+0000 7fee6f233640 1 --2- 172.21.15.100:0/1937721714 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee68108720 0x7fee6819f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:56.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.417+0000 7fee6f233640 1 --2- 172.21.15.100:0/1937721714 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee68107dd0 0x7fee6819a400 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:56.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.417+0000 7fee6f233640 1 --2- 172.21.15.100:0/1937721714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee68102490 0x7fee68199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:56.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.417+0000 7fee6f233640 1 -- 172.21.15.100:0/1937721714 >> 172.21.15.100:0/1937721714 conn(0x7fee68079160 msgr2=0x7fee68100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:56.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.417+0000 7fee6f233640 1 -- 172.21.15.100:0/1937721714 shutdown_connections 2024-07-26T13:46:56.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:56.417+0000 7fee6f233640 1 -- 172.21.15.100:0/1937721714 wait complete. 2024-07-26T13:46:56.425 INFO:tasks.workunit.client.0.smithi100.stdout: 1 277503 3.73 MB/sec execute 190 sec latency 16.876 ms 2024-07-26T13:46:57.355 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-26T13:46:57.422 INFO:tasks.workunit.client.0.smithi100.stdout: 1 280181 3.79 MB/sec execute 191 sec latency 18.547 ms 2024-07-26T13:46:57.619 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:46:57.890 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:57 smithi100 ceph-mon[32599]: pgmap v572: 97 pgs: 97 active+clean; 101 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 250 op/s 2024-07-26T13:46:57.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:57 smithi138 ceph-mon[36954]: pgmap v572: 97 pgs: 97 active+clean; 101 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 250 op/s 2024-07-26T13:46:58.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:57 smithi167 ceph-mon[37140]: pgmap v572: 97 pgs: 97 active+clean; 101 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 250 op/s 2024-07-26T13:46:58.423 INFO:tasks.workunit.client.0.smithi100.stdout: 1 281629 3.81 MB/sec execute 192 sec latency 16.578 ms 2024-07-26T13:46:59.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.354+0000 7f021bb50640 1 -- 172.21.15.100:0/1375219001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0214107dd0 msgr2=0x7f02141081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:59.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.354+0000 7f021bb50640 1 --2- 172.21.15.100:0/1375219001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0214107dd0 0x7f02141081e0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f0204009900 tx=0x7f020402a410 comp rx=0 tx=0).stop 2024-07-26T13:46:59.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.354+0000 7f021bb50640 1 -- 172.21.15.100:0/1375219001 shutdown_connections 2024-07-26T13:46:59.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.354+0000 7f021bb50640 1 --2- 172.21.15.100:0/1375219001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0214108720 0x7f021410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:59.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.354+0000 7f021bb50640 1 --2- 172.21.15.100:0/1375219001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0214107dd0 0x7f02141081e0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:59.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.354+0000 7f021bb50640 1 --2- 172.21.15.100:0/1375219001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0214102490 0x7f02141028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:59.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.354+0000 7f021bb50640 1 -- 172.21.15.100:0/1375219001 >> 172.21.15.100:0/1375219001 conn(0x7f0214079130 msgr2=0x7f0214079540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:59.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.354+0000 7f021bb50640 1 -- 172.21.15.100:0/1375219001 shutdown_connections 2024-07-26T13:46:59.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.354+0000 7f021bb50640 1 -- 172.21.15.100:0/1375219001 wait complete. 2024-07-26T13:46:59.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.355+0000 7f021bb50640 1 Processor -- start 2024-07-26T13:46:59.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.355+0000 7f021bb50640 1 -- start start 2024-07-26T13:46:59.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.355+0000 7f021bb50640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0214102490 0x7f021419acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:59.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.355+0000 7f021bb50640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0214107dd0 0x7f021419b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:59.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.355+0000 7f021bb50640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0214108720 0x7f021419b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:59.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.355+0000 7f021bb50640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f02141123b0 con 0x7f0214108720 2024-07-26T13:46:59.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.355+0000 7f021bb50640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0214112230 con 0x7f0214102490 2024-07-26T13:46:59.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.355+0000 7f021bb50640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0214112530 con 0x7f0214107dd0 2024-07-26T13:46:59.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.355+0000 7f02190eb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0214107dd0 0x7f021419b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:59.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.355+0000 7f021a0ed640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0214108720 0x7f021419b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:59.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.355+0000 7f02198ec640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0214102490 0x7f021419acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:59.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.355+0000 7f02190eb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0214107dd0 0x7f021419b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:35158/0 (socket says 172.21.15.100:35158) 2024-07-26T13:46:59.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.355+0000 7f021a0ed640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0214108720 0x7f021419b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34620/0 (socket says 172.21.15.100:34620) 2024-07-26T13:46:59.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.355+0000 7f02190eb640 1 -- 172.21.15.100:0/2206922096 learned_addr learned my addr 172.21.15.100:0/2206922096 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:59.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.355+0000 7f02198ec640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0214102490 0x7f021419acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:42256/0 (socket says 172.21.15.100:42256) 2024-07-26T13:46:59.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.355+0000 7f02198ec640 1 -- 172.21.15.100:0/2206922096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0214107dd0 msgr2=0x7f021419b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:59.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.355+0000 7f02198ec640 1 --2- 172.21.15.100:0/2206922096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0214107dd0 0x7f021419b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:59.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.355+0000 7f02198ec640 1 -- 172.21.15.100:0/2206922096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0214108720 msgr2=0x7f021419b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:59.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.355+0000 7f02198ec640 1 --2- 172.21.15.100:0/2206922096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0214108720 0x7f021419b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:59.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.355+0000 7f02198ec640 1 -- 172.21.15.100:0/2206922096 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02141a5f80 con 0x7f0214102490 2024-07-26T13:46:59.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.355+0000 7f021a0ed640 1 --2- 172.21.15.100:0/2206922096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0214108720 0x7f021419b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:46:59.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.356+0000 7f02190eb640 1 --2- 172.21.15.100:0/2206922096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0214107dd0 0x7f021419b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:46:59.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.356+0000 7f02198ec640 1 --2- 172.21.15.100:0/2206922096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0214102490 0x7f021419acc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f021000e9f0 tx=0x7f021000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:59.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.356+0000 7f020affd640 1 -- 172.21.15.100:0/2206922096 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021000ccc0 con 0x7f0214102490 2024-07-26T13:46:59.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.356+0000 7f021bb50640 1 -- 172.21.15.100:0/2206922096 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02141a6270 con 0x7f0214102490 2024-07-26T13:46:59.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.356+0000 7f020affd640 1 -- 172.21.15.100:0/2206922096 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0210005020 con 0x7f0214102490 2024-07-26T13:46:59.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.356+0000 7f020affd640 1 -- 172.21.15.100:0/2206922096 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0210004a00 con 0x7f0214102490 2024-07-26T13:46:59.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.356+0000 7f021bb50640 1 -- 172.21.15.100:0/2206922096 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02141a67b0 con 0x7f0214102490 2024-07-26T13:46:59.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.359+0000 7f02220d4740 1 Processor -- start 2024-07-26T13:46:59.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.359+0000 7f02220d4740 1 -- start start 2024-07-26T13:46:59.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.360+0000 7f02220d4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c61dd3070 0x560c62178570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:59.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.360+0000 7f02220d4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c61daa1b0 0x560c62178ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:59.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.360+0000 7f02220d4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c61db2390 0x560c62178ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:59.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.360+0000 7f02220d4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x560c620eebb0 con 0x560c61daa1b0 2024-07-26T13:46:59.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.360+0000 7f020affd640 1 -- 172.21.15.100:0/2206922096 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f0210004ba0 con 0x7f0214102490 2024-07-26T13:46:59.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.360+0000 7f02220d4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560c620de0b0 con 0x560c61db2390 2024-07-26T13:46:59.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.360+0000 7f02220d4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560c620e20f0 con 0x560c61dd3070 2024-07-26T13:46:59.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.360+0000 7f02190eb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c61db2390 0x560c62178ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:59.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.360+0000 7f021a0ed640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c61dd3070 0x560c62178570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:59.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.360+0000 7f02198ec640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c61daa1b0 0x560c62178ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:59.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.360+0000 7f02190eb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c61db2390 0x560c62178ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:42260/0 (socket says 172.21.15.100:42260) 2024-07-26T13:46:59.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.360+0000 7f021a0ed640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c61dd3070 0x560c62178570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:35166/0 (socket says 172.21.15.100:35166) 2024-07-26T13:46:59.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.360+0000 7f02190eb640 1 -- 172.21.15.100:0/2718492793 learned_addr learned my addr 172.21.15.100:0/2718492793 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:59.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.360+0000 7f021a0ed640 1 -- 172.21.15.100:0/2718492793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c61db2390 msgr2=0x560c62178ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:59.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.360+0000 7f021a0ed640 1 --2- 172.21.15.100:0/2718492793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c61db2390 0x560c62178ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:59.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.360+0000 7f021a0ed640 1 -- 172.21.15.100:0/2718492793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c61daa1b0 msgr2=0x560c62178ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:59.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.360+0000 7f021a0ed640 1 --2- 172.21.15.100:0/2718492793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c61daa1b0 0x560c62178ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:59.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.360+0000 7f021a0ed640 1 -- 172.21.15.100:0/2718492793 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c62084b90 con 0x560c61dd3070 2024-07-26T13:46:59.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.361+0000 7f021a0ed640 1 --2- 172.21.15.100:0/2718492793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c61dd3070 0x560c62178570 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f020c005fd0 tx=0x7f020c013f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=883788224ab91718 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:59.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.361+0000 7f01fffff640 1 -- 172.21.15.100:0/2718492793 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c00ebb0 con 0x560c61dd3070 2024-07-26T13:46:59.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.361+0000 7f01fffff640 1 -- 172.21.15.100:0/2718492793 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f020c003e20 con 0x560c61dd3070 2024-07-26T13:46:59.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.361+0000 7f01fffff640 1 -- 172.21.15.100:0/2718492793 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c004840 con 0x560c61dd3070 2024-07-26T13:46:59.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.361+0000 7f020affd640 1 --2- 172.21.15.100:0/2206922096 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f02000b3d40 0x7f02000b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:59.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.361+0000 7f02198ec640 1 --2- 172.21.15.100:0/2206922096 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f02000b3d40 0x7f02000b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:59.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.361+0000 7f020affd640 1 -- 172.21.15.100:0/2206922096 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7f02100e40d0 con 0x7f0214102490 2024-07-26T13:46:59.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.362+0000 7f02220d4740 1 -- 172.21.15.100:0/2718492793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c61dd3070 msgr2=0x560c62178570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:59.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.362+0000 7f02198ec640 1 --2- 172.21.15.100:0/2206922096 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f02000b3d40 0x7f02000b6250 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f02100e17f0 tx=0x7f02100e7c20 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:59.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.362+0000 7f02220d4740 1 --2- 172.21.15.100:0/2718492793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c61dd3070 0x560c62178570 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f020c005fd0 tx=0x7f020c013f80 comp rx=0 tx=0).stop 2024-07-26T13:46:59.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.362+0000 7f02220d4740 1 -- 172.21.15.100:0/2718492793 shutdown_connections 2024-07-26T13:46:59.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.362+0000 7f02220d4740 1 --2- 172.21.15.100:0/2718492793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c61db2390 0x560c62178ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:59.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.362+0000 7f02220d4740 1 --2- 172.21.15.100:0/2718492793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c61daa1b0 0x560c62178ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:59.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.362+0000 7f02220d4740 1 --2- 172.21.15.100:0/2718492793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c61dd3070 0x560c62178570 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:59.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.362+0000 7f02220d4740 1 -- 172.21.15.100:0/2718492793 >> 172.21.15.100:0/2718492793 conn(0x560c61dc7460 msgr2=0x560c62176460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:59.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.362+0000 7f02220d4740 1 -- 172.21.15.100:0/2718492793 shutdown_connections 2024-07-26T13:46:59.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.362+0000 7f02220d4740 1 -- 172.21.15.100:0/2718492793 wait complete. 2024-07-26T13:46:59.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.363+0000 7f02220d4740 1 -- start start 2024-07-26T13:46:59.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.363+0000 7f02220d4740 1 Processor -- start 2024-07-26T13:46:59.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.363+0000 7f02220d4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c61daa1b0 0x560c621840d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:59.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.363+0000 7f02220d4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c61db2390 0x560c62184610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:59.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.363+0000 7f02220d4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c61dd3070 0x560c62190040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:59.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.363+0000 7f02220d4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x560c61f41700 con 0x560c61daa1b0 2024-07-26T13:46:59.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.363+0000 7f02220d4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560c620809e0 con 0x560c61db2390 2024-07-26T13:46:59.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.363+0000 7f02220d4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560c62186d50 con 0x560c61dd3070 2024-07-26T13:46:59.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.363+0000 7f02220d4740 10 client.0 _ll_register_callbacks cb 0x560c6209b4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:46:59.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.364+0000 7f021a0ed640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c61daa1b0 0x560c621840d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:59.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.364+0000 7f02190eb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c61db2390 0x560c62184610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:59.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.364+0000 7f02190eb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c61db2390 0x560c62184610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:42276/0 (socket says 172.21.15.100:42276) 2024-07-26T13:46:59.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.364+0000 7f021a0ed640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c61daa1b0 0x560c621840d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34628/0 (socket says 172.21.15.100:34628) 2024-07-26T13:46:59.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.364+0000 7f02190eb640 1 -- 172.21.15.100:0/394251269 learned_addr learned my addr 172.21.15.100:0/394251269 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:59.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.364+0000 7f021a0ed640 1 --2- 172.21.15.100:0/394251269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c61dd3070 0x560c62190040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:59.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.364+0000 7f02190eb640 1 -- 172.21.15.100:0/394251269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c61dd3070 msgr2=0x560c62190040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:59.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.364+0000 7f02190eb640 1 --2- 172.21.15.100:0/394251269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c61dd3070 0x560c62190040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:59.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.364+0000 7f02190eb640 1 -- 172.21.15.100:0/394251269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c61daa1b0 msgr2=0x560c621840d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:59.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.364+0000 7f02190eb640 1 --2- 172.21.15.100:0/394251269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c61daa1b0 0x560c621840d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:59.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.364+0000 7f02190eb640 1 -- 172.21.15.100:0/394251269 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c61e3e650 con 0x560c61db2390 2024-07-26T13:46:59.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.364+0000 7f02190eb640 1 --2- 172.21.15.100:0/394251269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c61db2390 0x560c62184610 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0204031750 tx=0x7f0204031780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63b3065fee04649d server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:59.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.364+0000 7f01fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-07-26T13:46:59.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.364+0000 7f01fd7fa640 1 -- 172.21.15.100:0/394251269 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0204035040 con 0x560c61db2390 2024-07-26T13:46:59.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.364+0000 7f01fd7fa640 1 -- 172.21.15.100:0/394251269 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7f0204004450 con 0x560c61db2390 2024-07-26T13:46:59.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.364+0000 7f02220d4740 1 -- 172.21.15.100:0/394251269 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f020c003e20 con 0x560c61db2390 2024-07-26T13:46:59.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.365+0000 7f01fd7fa640 1 -- 172.21.15.100:0/394251269 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f02040035f0 con 0x560c61db2390 2024-07-26T13:46:59.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.365+0000 7f02220d4740 10 client.26459 fetch_fsmap learned FSMap version 54 2024-07-26T13:46:59.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.365+0000 7f02220d4740 1 -- 172.21.15.100:0/394251269 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x560c62192d10 con 0x560c61db2390 2024-07-26T13:46:59.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.365+0000 7f01fd7fa640 1 -- 172.21.15.100:0/394251269 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f020c003e20 con 0x560c61db2390 2024-07-26T13:46:59.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.365+0000 7f01fd7fa640 1 -- 172.21.15.100:0/394251269 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 54) ==== 8350+0+0 (secure 0 0 0) 0x7f020404d030 con 0x560c61db2390 2024-07-26T13:46:59.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.365+0000 7f02220d4740 10 client.26459 fetch_fsmap finished waiting for FSMap version 54 2024-07-26T13:46:59.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.365+0000 7f02220d4740 10 client.26459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:46:59.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.365+0000 7f02220d4740 20 client.26459 populate_metadata read hostname 'smithi100' 2024-07-26T13:46:59.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.365+0000 7f02220d4740 1 --2- 172.21.15.100:0/394251269 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560c61d919f0 0x560c62196290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:59.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.365+0000 7f02220d4740 4 client.26459 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:46:59.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.365+0000 7f02220d4740 1 -- 172.21.15.100:0/394251269 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c620f0b00 con 0x560c61d919f0 2024-07-26T13:46:59.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.365+0000 7f021a0ed640 1 --2- 172.21.15.100:0/394251269 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560c61d919f0 0x560c62196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:59.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.366+0000 7f021a0ed640 1 --2- 172.21.15.100:0/394251269 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560c61d919f0 0x560c62196290 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89a81f7e52b8ed2a server_cookie=7dc0d2d885a38551 in_seq=0 out_seq=0 2024-07-26T13:46:59.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.366+0000 7f01fd7fa640 10 client.26459 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:59.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.367+0000 7f01fd7fa640 1 -- 172.21.15.100:0/394251269 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560c620f0b00 con 0x560c61d919f0 2024-07-26T13:46:59.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.367+0000 7f01fd7fa640 10 client.26459 handle_command_reply: tid=0 2024-07-26T13:46:59.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.367+0000 7f02220d4740 1 client.26459 shutdown 2024-07-26T13:46:59.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.367+0000 7f02220d4740 1 -- 172.21.15.100:0/394251269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c61db2390 msgr2=0x560c62184610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:59.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.367+0000 7f02220d4740 1 --2- 172.21.15.100:0/394251269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c61db2390 0x560c62184610 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0204031750 tx=0x7f0204031780 comp rx=0 tx=0).stop 2024-07-26T13:46:59.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.367+0000 7f02220d4740 1 -- 172.21.15.100:0/394251269 shutdown_connections 2024-07-26T13:46:59.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.367+0000 7f02220d4740 1 --2- 172.21.15.100:0/394251269 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560c61d919f0 0x560c62196290 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:59.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.367+0000 7f02220d4740 1 --2- 172.21.15.100:0/394251269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c61dd3070 0x560c62190040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:59.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.367+0000 7f02220d4740 1 --2- 172.21.15.100:0/394251269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c61db2390 0x560c62184610 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:59.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.367+0000 7f02220d4740 1 --2- 172.21.15.100:0/394251269 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c61daa1b0 0x560c621840d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:59.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.367+0000 7f01fd7fa640 0 client.26459 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:59.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.367+0000 7f02220d4740 1 -- 172.21.15.100:0/394251269 >> 172.21.15.100:0/394251269 conn(0x560c61dc7460 msgr2=0x560c62174c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:59.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.367+0000 7f02220d4740 1 -- 172.21.15.100:0/394251269 shutdown_connections 2024-07-26T13:46:59.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.367+0000 7f02220d4740 1 -- 172.21.15.100:0/394251269 wait complete. 2024-07-26T13:46:59.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.367+0000 7f02220d4740 20 client.26459 trim_cache size 0 max 16384 2024-07-26T13:46:59.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.380+0000 7f02220d4740 1 Processor -- start 2024-07-26T13:46:59.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.380+0000 7f02220d4740 1 -- start start 2024-07-26T13:46:59.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.381+0000 7f02220d4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c61d919f0 0x560c6217f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:59.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.381+0000 7f02220d4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c61daa1b0 0x560c6217e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:59.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.381+0000 7f02220d4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c61db2390 0x560c62185b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:59.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.381+0000 7f02220d4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f02141af8e0 con 0x560c61daa1b0 2024-07-26T13:46:59.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.381+0000 7f02220d4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f02141ad180 con 0x560c61d919f0 2024-07-26T13:46:59.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.381+0000 7f02220d4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f02141aaff0 con 0x560c61db2390 2024-07-26T13:46:59.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.381+0000 7f02190eb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c61daa1b0 0x560c6217e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:59.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.381+0000 7f021a0ed640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c61d919f0 0x560c6217f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:59.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.381+0000 7f02190eb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c61daa1b0 0x560c6217e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34634/0 (socket says 172.21.15.100:34634) 2024-07-26T13:46:59.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.381+0000 7f02190eb640 1 -- 172.21.15.100:0/3092083173 learned_addr learned my addr 172.21.15.100:0/3092083173 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:59.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.381+0000 7f021a0ed640 1 --2- 172.21.15.100:0/3092083173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c61db2390 0x560c62185b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:59.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.381+0000 7f02190eb640 1 -- 172.21.15.100:0/3092083173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c61db2390 msgr2=0x560c62185b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:59.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.381+0000 7f02190eb640 1 --2- 172.21.15.100:0/3092083173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c61db2390 0x560c62185b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:59.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.381+0000 7f02190eb640 1 -- 172.21.15.100:0/3092083173 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c61d919f0 msgr2=0x560c6217f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:59.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.381+0000 7f02190eb640 1 --2- 172.21.15.100:0/3092083173 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c61d919f0 0x560c6217f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:59.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.381+0000 7f02190eb640 1 -- 172.21.15.100:0/3092083173 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c6217a860 con 0x560c61daa1b0 2024-07-26T13:46:59.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.381+0000 7f02190eb640 1 --2- 172.21.15.100:0/3092083173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c61daa1b0 0x560c6217e690 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f0204031440 tx=0x7f0204005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b6cf0c00cb12b77 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:59.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.381+0000 7f01ff7fe640 1 -- 172.21.15.100:0/3092083173 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0204035030 con 0x560c61daa1b0 2024-07-26T13:46:59.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.382+0000 7f01ff7fe640 1 -- 172.21.15.100:0/3092083173 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0204048d00 con 0x560c61daa1b0 2024-07-26T13:46:59.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.382+0000 7f01ff7fe640 1 -- 172.21.15.100:0/3092083173 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0204004250 con 0x560c61daa1b0 2024-07-26T13:46:59.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.382+0000 7f02220d4740 1 -- 172.21.15.100:0/3092083173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c61daa1b0 msgr2=0x560c6217e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:59.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.382+0000 7f02220d4740 1 --2- 172.21.15.100:0/3092083173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c61daa1b0 0x560c6217e690 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f0204031440 tx=0x7f0204005780 comp rx=0 tx=0).stop 2024-07-26T13:46:59.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.382+0000 7f02220d4740 1 -- 172.21.15.100:0/3092083173 shutdown_connections 2024-07-26T13:46:59.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.382+0000 7f02220d4740 1 --2- 172.21.15.100:0/3092083173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c61db2390 0x560c62185b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:59.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.382+0000 7f02220d4740 1 --2- 172.21.15.100:0/3092083173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c61daa1b0 0x560c6217e690 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:59.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.382+0000 7f02220d4740 1 --2- 172.21.15.100:0/3092083173 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c61d919f0 0x560c6217f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:59.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.382+0000 7f02220d4740 1 -- 172.21.15.100:0/3092083173 >> 172.21.15.100:0/3092083173 conn(0x560c61dc7460 msgr2=0x560c621853f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:59.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.383+0000 7f02220d4740 1 -- 172.21.15.100:0/3092083173 shutdown_connections 2024-07-26T13:46:59.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.383+0000 7f02220d4740 1 -- 172.21.15.100:0/3092083173 wait complete. 2024-07-26T13:46:59.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.383+0000 7f02220d4740 1 -- start start 2024-07-26T13:46:59.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.383+0000 7f02220d4740 1 Processor -- start 2024-07-26T13:46:59.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.384+0000 7f02220d4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c61d919f0 0x560c62188e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:59.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.384+0000 7f02220d4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c61daa1b0 0x560c62189460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:59.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.384+0000 7f02220d4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c61db2390 0x560c621f6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:59.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.384+0000 7f02220d4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f02141ab640 con 0x560c61db2390 2024-07-26T13:46:59.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.384+0000 7f02220d4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f02141acab0 con 0x560c61d919f0 2024-07-26T13:46:59.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.384+0000 7f02220d4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f02141abd10 con 0x560c61daa1b0 2024-07-26T13:46:59.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.384+0000 7f02220d4740 10 client.0 _ll_register_callbacks cb 0x7f0204009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:46:59.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.384+0000 7f021a0ed640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c61d919f0 0x560c62188e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:59.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.384+0000 7f02190eb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c61daa1b0 0x560c62189460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:59.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.384+0000 7f02190eb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c61daa1b0 0x560c62189460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:35198/0 (socket says 172.21.15.100:35198) 2024-07-26T13:46:59.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.384+0000 7f021a0ed640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c61d919f0 0x560c62188e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:42294/0 (socket says 172.21.15.100:42294) 2024-07-26T13:46:59.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.384+0000 7f02190eb640 1 -- 172.21.15.100:0/1979313568 learned_addr learned my addr 172.21.15.100:0/1979313568 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:46:59.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.384+0000 7f021a0ed640 1 --2- 172.21.15.100:0/1979313568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c61db2390 0x560c621f6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:59.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.384+0000 7f02190eb640 1 -- 172.21.15.100:0/1979313568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c61d919f0 msgr2=0x560c62188e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:59.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.384+0000 7f02190eb640 1 --2- 172.21.15.100:0/1979313568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c61d919f0 0x560c62188e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:59.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.384+0000 7f02190eb640 1 -- 172.21.15.100:0/1979313568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c61db2390 msgr2=0x560c621f6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:46:59.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.384+0000 7f02190eb640 1 --2- 172.21.15.100:0/1979313568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c61db2390 0x560c621f6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:59.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.384+0000 7f02190eb640 1 -- 172.21.15.100:0/1979313568 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c621f7130 con 0x560c61daa1b0 2024-07-26T13:46:59.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.384+0000 7f021a0ed640 1 --2- 172.21.15.100:0/1979313568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c61db2390 0x560c621f6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:46:59.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.384+0000 7f02190eb640 1 --2- 172.21.15.100:0/1979313568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c61daa1b0 0x560c62189460 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0204048910 tx=0x7f0204004480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df312a3a2cd599d7 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:46:59.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.384+0000 7f01fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-07-26T13:46:59.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.385+0000 7f01fd7fa640 1 -- 172.21.15.100:0/1979313568 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0204065060 con 0x560c61daa1b0 2024-07-26T13:46:59.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.385+0000 7f01fd7fa640 1 -- 172.21.15.100:0/1979313568 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7f0204031b70 con 0x560c61daa1b0 2024-07-26T13:46:59.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.385+0000 7f02220d4740 1 -- 172.21.15.100:0/1979313568 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0204048d00 con 0x560c61daa1b0 2024-07-26T13:46:59.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.385+0000 7f01fd7fa640 1 -- 172.21.15.100:0/1979313568 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f020406c050 con 0x560c61daa1b0 2024-07-26T13:46:59.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.385+0000 7f02220d4740 10 client.26647 fetch_fsmap learned FSMap version 54 2024-07-26T13:46:59.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.385+0000 7f02220d4740 1 -- 172.21.15.100:0/1979313568 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x560c621fe520 con 0x560c61daa1b0 2024-07-26T13:46:59.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.385+0000 7f01fd7fa640 1 -- 172.21.15.100:0/1979313568 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0204048d00 con 0x560c61daa1b0 2024-07-26T13:46:59.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.385+0000 7f01fd7fa640 1 -- 172.21.15.100:0/1979313568 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 54) ==== 8350+0+0 (secure 0 0 0) 0x7f0204032050 con 0x560c61daa1b0 2024-07-26T13:46:59.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.385+0000 7f02220d4740 10 client.26647 fetch_fsmap finished waiting for FSMap version 54 2024-07-26T13:46:59.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.385+0000 7f02220d4740 10 client.26647 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:46:59.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.385+0000 7f02220d4740 20 client.26647 populate_metadata read hostname 'smithi100' 2024-07-26T13:46:59.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.385+0000 7f02220d4740 1 --2- 172.21.15.100:0/1979313568 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560c61dd3070 0x560c62200b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:46:59.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.385+0000 7f02220d4740 4 client.26647 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-26T13:46:59.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.385+0000 7f02220d4740 1 -- 172.21.15.100:0/1979313568 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560c620ec5a0 con 0x560c61dd3070 2024-07-26T13:46:59.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.386+0000 7f021a0ed640 1 --2- 172.21.15.100:0/1979313568 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560c61dd3070 0x560c62200b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:46:59.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.386+0000 7f021a0ed640 1 --2- 172.21.15.100:0/1979313568 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560c61dd3070 0x560c62200b90 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0c3c42465f8749f server_cookie=5faff142b1526f1e in_seq=0 out_seq=0 2024-07-26T13:46:59.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.386+0000 7f01fd7fa640 10 client.26647 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:59.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.386+0000 7f01fd7fa640 1 -- 172.21.15.100:0/1979313568 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560c620ec5a0 con 0x560c61dd3070 2024-07-26T13:46:59.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.386+0000 7f01fd7fa640 10 client.26647 handle_command_reply: tid=0 2024-07-26T13:46:59.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.386+0000 7f02220d4740 1 client.26647 shutdown 2024-07-26T13:46:59.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.386+0000 7f02220d4740 1 -- 172.21.15.100:0/1979313568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c61daa1b0 msgr2=0x560c62189460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:59.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.386+0000 7f02220d4740 1 --2- 172.21.15.100:0/1979313568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c61daa1b0 0x560c62189460 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0204048910 tx=0x7f0204004480 comp rx=0 tx=0).stop 2024-07-26T13:46:59.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.387+0000 7f02220d4740 1 -- 172.21.15.100:0/1979313568 shutdown_connections 2024-07-26T13:46:59.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.387+0000 7f02220d4740 1 --2- 172.21.15.100:0/1979313568 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x560c61dd3070 0x560c62200b90 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:59.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.387+0000 7f02220d4740 1 --2- 172.21.15.100:0/1979313568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x560c61db2390 0x560c621f6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:59.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.387+0000 7f02220d4740 1 --2- 172.21.15.100:0/1979313568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560c61daa1b0 0x560c62189460 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:59.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.387+0000 7f01fd7fa640 0 client.26647 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:46:59.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.387+0000 7f02220d4740 1 --2- 172.21.15.100:0/1979313568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c61d919f0 0x560c62188e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:59.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.387+0000 7f02220d4740 1 -- 172.21.15.100:0/1979313568 >> 172.21.15.100:0/1979313568 conn(0x560c61dc7460 msgr2=0x560c62185380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:59.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.387+0000 7f02220d4740 1 -- 172.21.15.100:0/1979313568 shutdown_connections 2024-07-26T13:46:59.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.387+0000 7f02220d4740 1 -- 172.21.15.100:0/1979313568 wait complete. 2024-07-26T13:46:59.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.387+0000 7f02220d4740 20 client.26647 trim_cache size 0 max 16384 2024-07-26T13:46:59.394 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-07-26T13:46:59.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.387+0000 7f021bb50640 1 -- 172.21.15.100:0/2206922096 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f02000b3d40 msgr2=0x7f02000b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:59.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.387+0000 7f021bb50640 1 --2- 172.21.15.100:0/2206922096 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f02000b3d40 0x7f02000b6250 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f02100e17f0 tx=0x7f02100e7c20 comp rx=0 tx=0).stop 2024-07-26T13:46:59.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.387+0000 7f021bb50640 1 -- 172.21.15.100:0/2206922096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0214102490 msgr2=0x7f021419acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:46:59.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.387+0000 7f021bb50640 1 --2- 172.21.15.100:0/2206922096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0214102490 0x7f021419acc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f021000e9f0 tx=0x7f021000eeb0 comp rx=0 tx=0).stop 2024-07-26T13:46:59.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.387+0000 7f021bb50640 1 -- 172.21.15.100:0/2206922096 shutdown_connections 2024-07-26T13:46:59.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.387+0000 7f021bb50640 1 --2- 172.21.15.100:0/2206922096 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f02000b3d40 0x7f02000b6250 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:59.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.387+0000 7f021bb50640 1 --2- 172.21.15.100:0/2206922096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0214108720 0x7f021419b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:59.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.387+0000 7f021bb50640 1 --2- 172.21.15.100:0/2206922096 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0214107dd0 0x7f021419b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:59.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.387+0000 7f021bb50640 1 --2- 172.21.15.100:0/2206922096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0214102490 0x7f021419acc0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:46:59.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.387+0000 7f021bb50640 1 -- 172.21.15.100:0/2206922096 >> 172.21.15.100:0/2206922096 conn(0x7f0214079130 msgr2=0x7f0214103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:46:59.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.388+0000 7f021bb50640 1 -- 172.21.15.100:0/2206922096 shutdown_connections 2024-07-26T13:46:59.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:46:59.388+0000 7f021bb50640 1 -- 172.21.15.100:0/2206922096 wait complete. 2024-07-26T13:46:59.423 INFO:tasks.workunit.client.0.smithi100.stdout: 1 283976 3.86 MB/sec execute 193 sec latency 17.644 ms 2024-07-26T13:46:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:46:59 smithi100 ceph-mon[32599]: pgmap v573: 97 pgs: 97 active+clean; 103 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.2 MiB/s wr, 257 op/s 2024-07-26T13:46:59.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:46:59 smithi138 ceph-mon[36954]: pgmap v573: 97 pgs: 97 active+clean; 103 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.2 MiB/s wr, 257 op/s 2024-07-26T13:47:00.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:46:59 smithi167 ceph-mon[37140]: pgmap v573: 97 pgs: 97 active+clean; 103 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.2 MiB/s wr, 257 op/s 2024-07-26T13:47:00.267 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-26T13:47:00.423 INFO:tasks.workunit.client.0.smithi100.stdout: 1 285294 3.87 MB/sec execute 194 sec latency 25.346 ms 2024-07-26T13:47:00.553 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:47:01.281 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.280+0000 7fef38ec2640 1 -- 172.21.15.100:0/1234168552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef34101bb0 msgr2=0x7fef34101fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:01.281 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.280+0000 7fef38ec2640 1 --2- 172.21.15.100:0/1234168552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef34101bb0 0x7fef34101fc0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fef24009900 tx=0x7fef2402a410 comp rx=0 tx=0).stop 2024-07-26T13:47:01.281 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.280+0000 7fef38ec2640 1 -- 172.21.15.100:0/1234168552 shutdown_connections 2024-07-26T13:47:01.281 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.280+0000 7fef38ec2640 1 --2- 172.21.15.100:0/1234168552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef34107e30 0x7fef3410e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:01.281 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.280+0000 7fef38ec2640 1 --2- 172.21.15.100:0/1234168552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef341074e0 0x7fef341078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:01.281 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.280+0000 7fef38ec2640 1 --2- 172.21.15.100:0/1234168552 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef34101bb0 0x7fef34101fc0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:01.281 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.280+0000 7fef38ec2640 1 -- 172.21.15.100:0/1234168552 >> 172.21.15.100:0/1234168552 conn(0x7fef34077500 msgr2=0x7fef34077940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:01.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.280+0000 7fef38ec2640 1 -- 172.21.15.100:0/1234168552 shutdown_connections 2024-07-26T13:47:01.282 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.280+0000 7fef38ec2640 1 -- 172.21.15.100:0/1234168552 wait complete. 2024-07-26T13:47:01.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.281+0000 7fef38ec2640 1 Processor -- start 2024-07-26T13:47:01.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.281+0000 7fef38ec2640 1 -- start start 2024-07-26T13:47:01.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.281+0000 7fef38ec2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef34101bb0 0x7fef34199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:01.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.281+0000 7fef38ec2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef341074e0 0x7fef3419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:01.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.281+0000 7fef38ec2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef34107e30 0x7fef3419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:01.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.281+0000 7fef38ec2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fef34113460 con 0x7fef34107e30 2024-07-26T13:47:01.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.281+0000 7fef38ec2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fef341132e0 con 0x7fef341074e0 2024-07-26T13:47:01.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.281+0000 7fef38ec2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fef341135e0 con 0x7fef34101bb0 2024-07-26T13:47:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.281+0000 7fef32d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef34107e30 0x7fef3419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.281+0000 7fef3259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef34101bb0 0x7fef34199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.281+0000 7fef32d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef34107e30 0x7fef3419f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34660/0 (socket says 172.21.15.100:34660) 2024-07-26T13:47:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.281+0000 7fef32d9d640 1 -- 172.21.15.100:0/140185748 learned_addr learned my addr 172.21.15.100:0/140185748 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.281+0000 7fef31d9b640 1 --2- 172.21.15.100:0/140185748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef341074e0 0x7fef3419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.282+0000 7fef32d9d640 1 -- 172.21.15.100:0/140185748 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef34101bb0 msgr2=0x7fef34199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.282+0000 7fef32d9d640 1 --2- 172.21.15.100:0/140185748 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef34101bb0 0x7fef34199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.282+0000 7fef32d9d640 1 -- 172.21.15.100:0/140185748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef341074e0 msgr2=0x7fef3419a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.282+0000 7fef32d9d640 1 --2- 172.21.15.100:0/140185748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef341074e0 0x7fef3419a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.282+0000 7fef32d9d640 1 -- 172.21.15.100:0/140185748 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef3419fdb0 con 0x7fef34107e30 2024-07-26T13:47:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.282+0000 7fef31d9b640 1 --2- 172.21.15.100:0/140185748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef341074e0 0x7fef3419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:47:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.282+0000 7fef32d9d640 1 --2- 172.21.15.100:0/140185748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef34107e30 0x7fef3419f6b0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fef2c00eb10 tx=0x7fef2c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.282+0000 7fef237fe640 1 -- 172.21.15.100:0/140185748 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c004b20 con 0x7fef34107e30 2024-07-26T13:47:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.282+0000 7fef237fe640 1 -- 172.21.15.100:0/140185748 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef2c004cc0 con 0x7fef34107e30 2024-07-26T13:47:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.282+0000 7fef237fe640 1 -- 172.21.15.100:0/140185748 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c010670 con 0x7fef34107e30 2024-07-26T13:47:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.282+0000 7fef38ec2640 1 -- 172.21.15.100:0/140185748 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef341a00a0 con 0x7fef34107e30 2024-07-26T13:47:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.282+0000 7fef38ec2640 1 -- 172.21.15.100:0/140185748 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef341a6830 con 0x7fef34107e30 2024-07-26T13:47:01.293 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.284+0000 7fef3fde6740 1 Processor -- start 2024-07-26T13:47:01.293 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.285+0000 7fef237fe640 1 -- 172.21.15.100:0/140185748 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fef2c005020 con 0x7fef34107e30 2024-07-26T13:47:01.294 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.285+0000 7fef3fde6740 1 -- start start 2024-07-26T13:47:01.294 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.285+0000 7fef3fde6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d8d7121b0 0x562d8dab4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:01.294 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.285+0000 7fef3fde6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d8d6e92f0 0x562d8dab5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:01.294 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.285+0000 7fef3fde6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8d6f14d0 0x562d8dab5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:01.294 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.285+0000 7fef3fde6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x562d8da2a3b0 con 0x562d8d6f14d0 2024-07-26T13:47:01.294 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.285+0000 7fef3fde6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562d8da198b0 con 0x562d8d7121b0 2024-07-26T13:47:01.294 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.285+0000 7fef3fde6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562d8da1d8f0 con 0x562d8d6e92f0 2024-07-26T13:47:01.294 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.285+0000 7fef31d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8d6f14d0 0x562d8dab5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:01.294 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.285+0000 7fef3259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d8d6e92f0 0x562d8dab5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:01.294 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.285+0000 7fef31d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8d6f14d0 0x562d8dab5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34664/0 (socket says 172.21.15.100:34664) 2024-07-26T13:47:01.294 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.285+0000 7fef31d9b640 1 -- 172.21.15.100:0/1733246002 learned_addr learned my addr 172.21.15.100:0/1733246002 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:01.294 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.286+0000 7fef31d9b640 1 --2- 172.21.15.100:0/1733246002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d8d7121b0 0x562d8dab4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.286+0000 7fef237fe640 1 --2- 172.21.15.100:0/140185748 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fef1c0b3d40 0x7fef1c0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.286+0000 7fef3259c640 1 -- 172.21.15.100:0/1733246002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d8d7121b0 msgr2=0x562d8dab4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.286+0000 7fef3259c640 1 --2- 172.21.15.100:0/1733246002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d8d7121b0 0x562d8dab4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.286+0000 7fef3259c640 1 -- 172.21.15.100:0/1733246002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8d6f14d0 msgr2=0x562d8dab5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.286+0000 7fef3259c640 1 --2- 172.21.15.100:0/1733246002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8d6f14d0 0x562d8dab5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.286+0000 7fef3259c640 1 -- 172.21.15.100:0/1733246002 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d8d9c09a0 con 0x562d8d6e92f0 2024-07-26T13:47:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.286+0000 7fef31d9b640 1 --2- 172.21.15.100:0/1733246002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8d6f14d0 0x562d8dab5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:47:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.286+0000 7fef32d9d640 1 --2- 172.21.15.100:0/140185748 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fef1c0b3d40 0x7fef1c0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.286+0000 7fef3259c640 1 --2- 172.21.15.100:0/1733246002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d8d6e92f0 0x562d8dab5220 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fef24006010 tx=0x7fef24032b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ff0dc90bc7eb9cd server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.286+0000 7fef237fe640 1 -- 172.21.15.100:0/140185748 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7fef2c0e46b0 con 0x7fef34107e30 2024-07-26T13:47:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.286+0000 7fef20ff9640 1 -- 172.21.15.100:0/1733246002 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef24035040 con 0x562d8d6e92f0 2024-07-26T13:47:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.286+0000 7fef20ff9640 1 -- 172.21.15.100:0/1733246002 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef24004030 con 0x562d8d6e92f0 2024-07-26T13:47:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.286+0000 7fef20ff9640 1 -- 172.21.15.100:0/1733246002 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef24005330 con 0x562d8d6e92f0 2024-07-26T13:47:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.286+0000 7fef32d9d640 1 --2- 172.21.15.100:0/140185748 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fef1c0b3d40 0x7fef1c0b6250 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fef2c0a7260 tx=0x7fef2c0a2040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:01.296 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.287+0000 7fef3fde6740 1 -- 172.21.15.100:0/1733246002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d8d6e92f0 msgr2=0x562d8dab5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:01.296 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.287+0000 7fef3fde6740 1 --2- 172.21.15.100:0/1733246002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d8d6e92f0 0x562d8dab5220 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fef24006010 tx=0x7fef24032b90 comp rx=0 tx=0).stop 2024-07-26T13:47:01.296 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.287+0000 7fef3fde6740 1 -- 172.21.15.100:0/1733246002 shutdown_connections 2024-07-26T13:47:01.296 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.287+0000 7fef3fde6740 1 --2- 172.21.15.100:0/1733246002 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8d6f14d0 0x562d8dab5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:01.296 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.287+0000 7fef3fde6740 1 --2- 172.21.15.100:0/1733246002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d8d6e92f0 0x562d8dab5220 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:01.296 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.287+0000 7fef3fde6740 1 --2- 172.21.15.100:0/1733246002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d8d7121b0 0x562d8dab4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:01.296 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.287+0000 7fef3fde6740 1 -- 172.21.15.100:0/1733246002 >> 172.21.15.100:0/1733246002 conn(0x562d8d706300 msgr2=0x562d8dab2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:01.296 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.287+0000 7fef3fde6740 1 -- 172.21.15.100:0/1733246002 shutdown_connections 2024-07-26T13:47:01.296 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.287+0000 7fef3fde6740 1 -- 172.21.15.100:0/1733246002 wait complete. 2024-07-26T13:47:01.296 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.288+0000 7fef3fde6740 1 -- start start 2024-07-26T13:47:01.296 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.288+0000 7fef3fde6740 1 Processor -- start 2024-07-26T13:47:01.296 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.288+0000 7fef3fde6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d8d6e92f0 0x562d8dac0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:01.296 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.288+0000 7fef3fde6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8d6f14d0 0x562d8dac0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.288+0000 7fef3fde6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d8d7121b0 0x562d8dacc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.288+0000 7fef3fde6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x562d8d953ab0 con 0x562d8d6f14d0 2024-07-26T13:47:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.288+0000 7fef3fde6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562d8d9a1cf0 con 0x562d8d6e92f0 2024-07-26T13:47:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.288+0000 7fef3fde6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562d8dac3400 con 0x562d8d7121b0 2024-07-26T13:47:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.288+0000 7fef3fde6740 10 client.0 _ll_register_callbacks cb 0x562d8da139f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:47:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.288+0000 7fef3259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8d6f14d0 0x562d8dac0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.288+0000 7fef3259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8d6f14d0 0x562d8dac0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34676/0 (socket says 172.21.15.100:34676) 2024-07-26T13:47:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.288+0000 7fef31d9b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d8d6e92f0 0x562d8dac0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.288+0000 7fef3259c640 1 -- 172.21.15.100:0/3114674588 learned_addr learned my addr 172.21.15.100:0/3114674588 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.288+0000 7fef31d9b640 1 --2- 172.21.15.100:0/3114674588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d8d7121b0 0x562d8dacc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.289+0000 7fef31d9b640 1 -- 172.21.15.100:0/3114674588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d8d7121b0 msgr2=0x562d8dacc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.289+0000 7fef31d9b640 1 --2- 172.21.15.100:0/3114674588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d8d7121b0 0x562d8dacc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.289+0000 7fef31d9b640 1 -- 172.21.15.100:0/3114674588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8d6f14d0 msgr2=0x562d8dac0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.289+0000 7fef31d9b640 1 --2- 172.21.15.100:0/3114674588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8d6f14d0 0x562d8dac0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.289+0000 7fef31d9b640 1 -- 172.21.15.100:0/3114674588 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d8d77d790 con 0x562d8d6e92f0 2024-07-26T13:47:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.289+0000 7fef3259c640 1 --2- 172.21.15.100:0/3114674588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8d6f14d0 0x562d8dac0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:47:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.289+0000 7fef31d9b640 1 --2- 172.21.15.100:0/3114674588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d8d6e92f0 0x562d8dac0820 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fef280058c0 tx=0x7fef28004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c16bf018a5134779 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.289+0000 7fef19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-07-26T13:47:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.289+0000 7fef19ffb640 1 -- 172.21.15.100:0/3114674588 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2801a040 con 0x562d8d6e92f0 2024-07-26T13:47:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.289+0000 7fef3fde6740 1 -- 172.21.15.100:0/3114674588 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef24004030 con 0x562d8d6e92f0 2024-07-26T13:47:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.289+0000 7fef19ffb640 1 -- 172.21.15.100:0/3114674588 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7fef28004230 con 0x562d8d6e92f0 2024-07-26T13:47:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.289+0000 7fef19ffb640 1 -- 172.21.15.100:0/3114674588 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fef28004430 con 0x562d8d6e92f0 2024-07-26T13:47:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.289+0000 7fef3fde6740 10 client.26477 fetch_fsmap learned FSMap version 54 2024-07-26T13:47:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.289+0000 7fef3fde6740 1 -- 172.21.15.100:0/3114674588 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x562d8d7f6620 con 0x562d8d6e92f0 2024-07-26T13:47:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.289+0000 7fef19ffb640 1 -- 172.21.15.100:0/3114674588 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef24004030 con 0x562d8d6e92f0 2024-07-26T13:47:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.290+0000 7fef19ffb640 1 -- 172.21.15.100:0/3114674588 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 54) ==== 8350+0+0 (secure 0 0 0) 0x7fef2800e5f0 con 0x562d8d6e92f0 2024-07-26T13:47:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.290+0000 7fef3fde6740 10 client.26477 fetch_fsmap finished waiting for FSMap version 54 2024-07-26T13:47:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.290+0000 7fef3fde6740 10 client.26477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:47:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.290+0000 7fef3fde6740 20 client.26477 populate_metadata read hostname 'smithi100' 2024-07-26T13:47:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.290+0000 7fef3fde6740 1 --2- 172.21.15.100:0/3114674588 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x562d8d6d0d80 0x562d8dad2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.290+0000 7fef3fde6740 4 client.26477 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:47:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.290+0000 7fef3fde6740 1 -- 172.21.15.100:0/3114674588 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d8da2c300 con 0x562d8d6d0d80 2024-07-26T13:47:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.290+0000 7fef3259c640 1 --2- 172.21.15.100:0/3114674588 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x562d8d6d0d80 0x562d8dad2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.290+0000 7fef3259c640 1 --2- 172.21.15.100:0/3114674588 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x562d8d6d0d80 0x562d8dad2720 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16cc2e4112b4505 server_cookie=52007abd72f3cdfd in_seq=0 out_seq=0 2024-07-26T13:47:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.290+0000 7fef19ffb640 10 client.26477 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.291+0000 7fef19ffb640 1 -- 172.21.15.100:0/3114674588 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562d8da2c300 con 0x562d8d6d0d80 2024-07-26T13:47:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.291+0000 7fef19ffb640 10 client.26477 handle_command_reply: tid=0 2024-07-26T13:47:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.291+0000 7fef3fde6740 1 client.26477 shutdown 2024-07-26T13:47:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.291+0000 7fef3fde6740 1 -- 172.21.15.100:0/3114674588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d8d6e92f0 msgr2=0x562d8dac0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.291+0000 7fef3fde6740 1 --2- 172.21.15.100:0/3114674588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d8d6e92f0 0x562d8dac0820 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fef280058c0 tx=0x7fef28004730 comp rx=0 tx=0).stop 2024-07-26T13:47:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.291+0000 7fef3fde6740 1 -- 172.21.15.100:0/3114674588 shutdown_connections 2024-07-26T13:47:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.291+0000 7fef3fde6740 1 --2- 172.21.15.100:0/3114674588 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x562d8d6d0d80 0x562d8dad2720 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.291+0000 7fef3fde6740 1 --2- 172.21.15.100:0/3114674588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d8d7121b0 0x562d8dacc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.291+0000 7fef19ffb640 0 client.26477 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.291+0000 7fef3fde6740 1 --2- 172.21.15.100:0/3114674588 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8d6f14d0 0x562d8dac0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.291+0000 7fef3fde6740 1 --2- 172.21.15.100:0/3114674588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d8d6e92f0 0x562d8dac0820 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.291+0000 7fef3fde6740 1 -- 172.21.15.100:0/3114674588 >> 172.21.15.100:0/3114674588 conn(0x562d8d706300 msgr2=0x562d8dab5d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.291+0000 7fef3fde6740 1 -- 172.21.15.100:0/3114674588 shutdown_connections 2024-07-26T13:47:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.291+0000 7fef3fde6740 1 -- 172.21.15.100:0/3114674588 wait complete. 2024-07-26T13:47:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.291+0000 7fef3fde6740 20 client.26477 trim_cache size 0 max 16384 2024-07-26T13:47:01.313 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.307+0000 7fef3fde6740 1 Processor -- start 2024-07-26T13:47:01.313 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.307+0000 7fef3fde6740 1 -- start start 2024-07-26T13:47:01.313 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.307+0000 7fef3fde6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8d706300 0x562d8dabfe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:01.313 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.307+0000 7fef3fde6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d8d6d0d80 0x562d8dac2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:01.313 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.307+0000 7fef3fde6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d8d6e92f0 0x562d8dacc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:01.313 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.307+0000 7fef3fde6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fef341b1b20 con 0x562d8d706300 2024-07-26T13:47:01.314 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.307+0000 7fef3fde6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fef341af3c0 con 0x562d8d6d0d80 2024-07-26T13:47:01.314 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.307+0000 7fef3fde6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fef341ad210 con 0x562d8d6e92f0 2024-07-26T13:47:01.314 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.307+0000 7fef31d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8d706300 0x562d8dabfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:01.314 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.307+0000 7fef31d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8d706300 0x562d8dabfe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34684/0 (socket says 172.21.15.100:34684) 2024-07-26T13:47:01.314 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.308+0000 7fef31d9b640 1 -- 172.21.15.100:0/42837716 learned_addr learned my addr 172.21.15.100:0/42837716 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:01.314 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.308+0000 7fef3259c640 1 --2- 172.21.15.100:0/42837716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d8d6d0d80 0x562d8dac2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:01.314 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.308+0000 7fef31d9b640 1 --2- 172.21.15.100:0/42837716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d8d6e92f0 0x562d8dacc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:01.314 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.308+0000 7fef31d9b640 1 -- 172.21.15.100:0/42837716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d8d6e92f0 msgr2=0x562d8dacc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:01.314 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.308+0000 7fef31d9b640 1 --2- 172.21.15.100:0/42837716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d8d6e92f0 0x562d8dacc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:01.314 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.308+0000 7fef31d9b640 1 -- 172.21.15.100:0/42837716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d8d6d0d80 msgr2=0x562d8dac2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:01.314 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.308+0000 7fef31d9b640 1 --2- 172.21.15.100:0/42837716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d8d6d0d80 0x562d8dac2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:01.314 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.308+0000 7fef31d9b640 1 -- 172.21.15.100:0/42837716 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d8dab3960 con 0x562d8d706300 2024-07-26T13:47:01.314 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.308+0000 7fef31d9b640 1 --2- 172.21.15.100:0/42837716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8d706300 0x562d8dabfe80 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fef2800f4f0 tx=0x7fef28011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=705aea6b803e8111 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.308+0000 7fef1bfff640 1 -- 172.21.15.100:0/42837716 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2800ee80 con 0x562d8d706300 2024-07-26T13:47:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.308+0000 7fef1bfff640 1 -- 172.21.15.100:0/42837716 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef280054d0 con 0x562d8d706300 2024-07-26T13:47:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.308+0000 7fef1bfff640 1 -- 172.21.15.100:0/42837716 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef28004b90 con 0x562d8d706300 2024-07-26T13:47:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.309+0000 7fef3fde6740 1 -- 172.21.15.100:0/42837716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8d706300 msgr2=0x562d8dabfe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.309+0000 7fef3fde6740 1 --2- 172.21.15.100:0/42837716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8d706300 0x562d8dabfe80 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fef2800f4f0 tx=0x7fef28011440 comp rx=0 tx=0).stop 2024-07-26T13:47:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.309+0000 7fef3fde6740 1 -- 172.21.15.100:0/42837716 shutdown_connections 2024-07-26T13:47:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.309+0000 7fef3fde6740 1 --2- 172.21.15.100:0/42837716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d8d6e92f0 0x562d8dacc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.309+0000 7fef3fde6740 1 --2- 172.21.15.100:0/42837716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d8d6d0d80 0x562d8dac2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.309+0000 7fef3fde6740 1 --2- 172.21.15.100:0/42837716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8d706300 0x562d8dabfe80 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.309+0000 7fef3fde6740 1 -- 172.21.15.100:0/42837716 >> 172.21.15.100:0/42837716 conn(0x562d8dad2a10 msgr2=0x562d8dac1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.309+0000 7fef3fde6740 1 -- 172.21.15.100:0/42837716 shutdown_connections 2024-07-26T13:47:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.309+0000 7fef3fde6740 1 -- 172.21.15.100:0/42837716 wait complete. 2024-07-26T13:47:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.309+0000 7fef3fde6740 1 -- start start 2024-07-26T13:47:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.309+0000 7fef3fde6740 1 Processor -- start 2024-07-26T13:47:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.310+0000 7fef3fde6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d8d6d0d80 0x562d8dac8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.310+0000 7fef3fde6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8d6e92f0 0x562d8dac5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.310+0000 7fef3fde6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d8d706300 0x562d8dac5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.310+0000 7fef3fde6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fef341ad880 con 0x562d8d6e92f0 2024-07-26T13:47:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.310+0000 7fef3fde6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fef341aecf0 con 0x562d8d706300 2024-07-26T13:47:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.310+0000 7fef3fde6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fef341adf50 con 0x562d8d6d0d80 2024-07-26T13:47:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.310+0000 7fef3fde6740 10 client.0 _ll_register_callbacks cb 0x7fef2800f3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:47:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.310+0000 7fef3259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8d6e92f0 0x562d8dac5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.310+0000 7fef3259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8d6e92f0 0x562d8dac5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34700/0 (socket says 172.21.15.100:34700) 2024-07-26T13:47:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.310+0000 7fef3259c640 1 -- 172.21.15.100:0/3898674850 learned_addr learned my addr 172.21.15.100:0/3898674850 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.310+0000 7fef31d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d8d6d0d80 0x562d8dac8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.310+0000 7fef31d9b640 1 --2- 172.21.15.100:0/3898674850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d8d706300 0x562d8dac5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.310+0000 7fef3259c640 1 -- 172.21.15.100:0/3898674850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d8d6d0d80 msgr2=0x562d8dac8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.310+0000 7fef3259c640 1 --2- 172.21.15.100:0/3898674850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d8d6d0d80 0x562d8dac8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.310+0000 7fef3259c640 1 -- 172.21.15.100:0/3898674850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d8d706300 msgr2=0x562d8dac5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.310+0000 7fef3259c640 1 --2- 172.21.15.100:0/3898674850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d8d706300 0x562d8dac5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.310+0000 7fef3259c640 1 -- 172.21.15.100:0/3898674850 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d8dac6150 con 0x562d8d6e92f0 2024-07-26T13:47:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.310+0000 7fef31d9b640 1 --2- 172.21.15.100:0/3898674850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d8d6d0d80 0x562d8dac8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:47:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.310+0000 7fef3259c640 1 --2- 172.21.15.100:0/3898674850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8d6e92f0 0x562d8dac5620 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fef2402a930 tx=0x7fef24032fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18604094ccdf44ba server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.310+0000 7fef19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:47:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.311+0000 7fef19ffb640 1 -- 172.21.15.100:0/3898674850 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2405c060 con 0x562d8d6e92f0 2024-07-26T13:47:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.311+0000 7fef19ffb640 1 -- 172.21.15.100:0/3898674850 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7fef240050c0 con 0x562d8d6e92f0 2024-07-26T13:47:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.311+0000 7fef3fde6740 1 -- 172.21.15.100:0/3898674850 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef280054d0 con 0x562d8d6e92f0 2024-07-26T13:47:01.318 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.311+0000 7fef19ffb640 1 -- 172.21.15.100:0/3898674850 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fef24063050 con 0x562d8d6e92f0 2024-07-26T13:47:01.318 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.311+0000 7fef3fde6740 10 client.17259 fetch_fsmap learned FSMap version 54 2024-07-26T13:47:01.318 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.311+0000 7fef3fde6740 1 -- 172.21.15.100:0/3898674850 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x562d8db39d10 con 0x562d8d6e92f0 2024-07-26T13:47:01.318 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.311+0000 7fef19ffb640 1 -- 172.21.15.100:0/3898674850 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef280054d0 con 0x562d8d6e92f0 2024-07-26T13:47:01.318 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.311+0000 7fef19ffb640 1 -- 172.21.15.100:0/3898674850 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 54) ==== 8350+0+0 (secure 0 0 0) 0x7fef24032040 con 0x562d8d6e92f0 2024-07-26T13:47:01.318 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.311+0000 7fef3fde6740 10 client.17259 fetch_fsmap finished waiting for FSMap version 54 2024-07-26T13:47:01.318 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.311+0000 7fef3fde6740 10 client.17259 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:47:01.318 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.311+0000 7fef3fde6740 20 client.17259 populate_metadata read hostname 'smithi100' 2024-07-26T13:47:01.318 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.311+0000 7fef3fde6740 1 --2- 172.21.15.100:0/3898674850 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x562d8d6f14d0 0x562d8db3c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:01.318 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.311+0000 7fef3fde6740 4 client.17259 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-26T13:47:01.318 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.311+0000 7fef3fde6740 1 -- 172.21.15.100:0/3898674850 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562d8da27da0 con 0x562d8d6f14d0 2024-07-26T13:47:01.318 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.311+0000 7fef31d9b640 1 --2- 172.21.15.100:0/3898674850 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x562d8d6f14d0 0x562d8db3c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:01.318 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.312+0000 7fef19ffb640 10 client.17259 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:01.318 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.312+0000 7fef31d9b640 1 --2- 172.21.15.100:0/3898674850 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x562d8d6f14d0 0x562d8db3c060 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f92185ce2b5111b server_cookie=e1bb3dc790c0abd9 in_seq=0 out_seq=0 2024-07-26T13:47:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.366+0000 7fef19ffb640 1 -- 172.21.15.100:0/3898674850 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562d8da27da0 con 0x562d8d6f14d0 2024-07-26T13:47:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.366+0000 7fef19ffb640 10 client.17259 handle_command_reply: tid=0 2024-07-26T13:47:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.366+0000 7fef3fde6740 1 client.17259 shutdown 2024-07-26T13:47:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.366+0000 7fef3fde6740 1 -- 172.21.15.100:0/3898674850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8d6e92f0 msgr2=0x562d8dac5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.366+0000 7fef3fde6740 1 --2- 172.21.15.100:0/3898674850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8d6e92f0 0x562d8dac5620 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fef2402a930 tx=0x7fef24032fc0 comp rx=0 tx=0).stop 2024-07-26T13:47:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.366+0000 7fef3fde6740 1 -- 172.21.15.100:0/3898674850 shutdown_connections 2024-07-26T13:47:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.366+0000 7fef3fde6740 1 --2- 172.21.15.100:0/3898674850 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x562d8d6f14d0 0x562d8db3c060 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.366+0000 7fef3fde6740 1 --2- 172.21.15.100:0/3898674850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562d8d706300 0x562d8dac5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.366+0000 7fef3fde6740 1 --2- 172.21.15.100:0/3898674850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562d8d6e92f0 0x562d8dac5620 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.366+0000 7fef3fde6740 1 --2- 172.21.15.100:0/3898674850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562d8d6d0d80 0x562d8dac8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.366+0000 7fef3fde6740 1 -- 172.21.15.100:0/3898674850 >> 172.21.15.100:0/3898674850 conn(0x562d8dad2a10 msgr2=0x562d8dac18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.366+0000 7fef19ffb640 0 client.17259 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.367+0000 7fef3fde6740 1 -- 172.21.15.100:0/3898674850 shutdown_connections 2024-07-26T13:47:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.367+0000 7fef3fde6740 1 -- 172.21.15.100:0/3898674850 wait complete. 2024-07-26T13:47:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.367+0000 7fef3fde6740 20 client.17259 trim_cache size 0 max 16384 2024-07-26T13:47:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-07-26T13:47:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-07-26T13:47:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-07-26T13:47:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-07-26T13:47:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.367+0000 7fef38ec2640 1 -- 172.21.15.100:0/140185748 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fef1c0b3d40 msgr2=0x7fef1c0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.367+0000 7fef38ec2640 1 --2- 172.21.15.100:0/140185748 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fef1c0b3d40 0x7fef1c0b6250 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fef2c0a7260 tx=0x7fef2c0a2040 comp rx=0 tx=0).stop 2024-07-26T13:47:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.367+0000 7fef38ec2640 1 -- 172.21.15.100:0/140185748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef34107e30 msgr2=0x7fef3419f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.367+0000 7fef38ec2640 1 --2- 172.21.15.100:0/140185748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef34107e30 0x7fef3419f6b0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fef2c00eb10 tx=0x7fef2c00c2d0 comp rx=0 tx=0).stop 2024-07-26T13:47:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.368+0000 7fef38ec2640 1 -- 172.21.15.100:0/140185748 shutdown_connections 2024-07-26T13:47:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.368+0000 7fef38ec2640 1 --2- 172.21.15.100:0/140185748 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fef1c0b3d40 0x7fef1c0b6250 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.368+0000 7fef38ec2640 1 --2- 172.21.15.100:0/140185748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef34107e30 0x7fef3419f6b0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.368+0000 7fef38ec2640 1 --2- 172.21.15.100:0/140185748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef341074e0 0x7fef3419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.368+0000 7fef38ec2640 1 --2- 172.21.15.100:0/140185748 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef34101bb0 0x7fef34199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.368+0000 7fef38ec2640 1 -- 172.21.15.100:0/140185748 >> 172.21.15.100:0/140185748 conn(0x7fef34077500 msgr2=0x7fef341055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.368+0000 7fef38ec2640 1 -- 172.21.15.100:0/140185748 shutdown_connections 2024-07-26T13:47:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:01.368+0000 7fef38ec2640 1 -- 172.21.15.100:0/140185748 wait complete. 2024-07-26T13:47:01.423 INFO:tasks.workunit.client.0.smithi100.stdout: 1 286283 3.86 MB/sec execute 195 sec latency 35.716 ms 2024-07-26T13:47:01.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:01 smithi100 ceph-mon[32599]: pgmap v574: 97 pgs: 97 active+clean; 109 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.6 MiB/s wr, 311 op/s 2024-07-26T13:47:01.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:01 smithi138 ceph-mon[36954]: pgmap v574: 97 pgs: 97 active+clean; 109 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.6 MiB/s wr, 311 op/s 2024-07-26T13:47:02.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:01 smithi167 ceph-mon[37140]: pgmap v574: 97 pgs: 97 active+clean; 109 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.6 MiB/s wr, 311 op/s 2024-07-26T13:47:02.423 INFO:tasks.workunit.client.0.smithi100.stdout: 1 288055 3.90 MB/sec execute 196 sec latency 15.663 ms 2024-07-26T13:47:02.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.439+0000 7efc8ac9d640 1 -- 172.21.15.100:0/888722334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc84107dd0 msgr2=0x7efc841081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:02.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.439+0000 7efc8ac9d640 1 --2- 172.21.15.100:0/888722334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc84107dd0 0x7efc841081e0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7efc78009900 tx=0x7efc7802a410 comp rx=0 tx=0).stop 2024-07-26T13:47:02.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.440+0000 7efc8ac9d640 1 -- 172.21.15.100:0/888722334 shutdown_connections 2024-07-26T13:47:02.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.440+0000 7efc8ac9d640 1 --2- 172.21.15.100:0/888722334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc84108720 0x7efc8410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:02.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.440+0000 7efc8ac9d640 1 --2- 172.21.15.100:0/888722334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc84107dd0 0x7efc841081e0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:02.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.440+0000 7efc8ac9d640 1 --2- 172.21.15.100:0/888722334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc84102490 0x7efc841028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:02.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.440+0000 7efc8ac9d640 1 -- 172.21.15.100:0/888722334 >> 172.21.15.100:0/888722334 conn(0x7efc84079160 msgr2=0x7efc84079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:02.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.440+0000 7efc8ac9d640 1 -- 172.21.15.100:0/888722334 shutdown_connections 2024-07-26T13:47:02.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.440+0000 7efc8ac9d640 1 -- 172.21.15.100:0/888722334 wait complete. 2024-07-26T13:47:02.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.440+0000 7efc8ac9d640 1 Processor -- start 2024-07-26T13:47:02.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.440+0000 7efc8ac9d640 1 -- start start 2024-07-26T13:47:02.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.441+0000 7efc8ac9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc84102490 0x7efc8419c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:02.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.441+0000 7efc8ac9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc84107dd0 0x7efc8419c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:02.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.441+0000 7efc8ac9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc84108720 0x7efc841a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:02.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.441+0000 7efc8ac9d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efc84112370 con 0x7efc84107dd0 2024-07-26T13:47:02.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.441+0000 7efc8ac9d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efc841121f0 con 0x7efc84108720 2024-07-26T13:47:02.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.441+0000 7efc8ac9d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efc841124f0 con 0x7efc84102490 2024-07-26T13:47:02.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.441+0000 7efc83fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc84107dd0 0x7efc8419c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:02.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.441+0000 7efc83fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc84107dd0 0x7efc8419c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34718/0 (socket says 172.21.15.100:34718) 2024-07-26T13:47:02.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.441+0000 7efc83fff640 1 -- 172.21.15.100:0/1553798985 learned_addr learned my addr 172.21.15.100:0/1553798985 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:02.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.441+0000 7efc88a39640 1 --2- 172.21.15.100:0/1553798985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc84102490 0x7efc8419c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:02.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.441+0000 7efc8923a640 1 --2- 172.21.15.100:0/1553798985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc84108720 0x7efc841a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:02.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.441+0000 7efc83fff640 1 -- 172.21.15.100:0/1553798985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc84102490 msgr2=0x7efc8419c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:02.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.441+0000 7efc83fff640 1 --2- 172.21.15.100:0/1553798985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc84102490 0x7efc8419c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:02.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.441+0000 7efc83fff640 1 -- 172.21.15.100:0/1553798985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc84108720 msgr2=0x7efc841a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:02.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.441+0000 7efc83fff640 1 --2- 172.21.15.100:0/1553798985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc84108720 0x7efc841a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:02.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.441+0000 7efc83fff640 1 -- 172.21.15.100:0/1553798985 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc841a1fc0 con 0x7efc84107dd0 2024-07-26T13:47:02.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.441+0000 7efc83fff640 1 --2- 172.21.15.100:0/1553798985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc84107dd0 0x7efc8419c5f0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7efc78005ab0 tx=0x7efc78031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:02.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.441+0000 7efc81ffb640 1 -- 172.21.15.100:0/1553798985 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7803f030 con 0x7efc84107dd0 2024-07-26T13:47:02.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.442+0000 7efc8ac9d640 1 -- 172.21.15.100:0/1553798985 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc841a2250 con 0x7efc84107dd0 2024-07-26T13:47:02.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.442+0000 7efc81ffb640 1 -- 172.21.15.100:0/1553798985 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc780043d0 con 0x7efc84107dd0 2024-07-26T13:47:02.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.442+0000 7efc81ffb640 1 -- 172.21.15.100:0/1553798985 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc78035070 con 0x7efc84107dd0 2024-07-26T13:47:02.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.442+0000 7efc8ac9d640 1 -- 172.21.15.100:0/1553798985 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc841a8a10 con 0x7efc84107dd0 2024-07-26T13:47:02.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.445+0000 7efc91221740 1 Processor -- start 2024-07-26T13:47:02.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.445+0000 7efc91221740 1 -- start start 2024-07-26T13:47:02.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.445+0000 7efc81ffb640 1 -- 172.21.15.100:0/1553798985 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7efc78042030 con 0x7efc84107dd0 2024-07-26T13:47:02.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.445+0000 7efc91221740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564246384070 0x564246729570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:02.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.445+0000 7efc91221740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56424635b1b0 0x564246729ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:02.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.445+0000 7efc91221740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564246363390 0x564246729ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:02.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.445+0000 7efc91221740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56424669fbb0 con 0x56424635b1b0 2024-07-26T13:47:02.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.445+0000 7efc91221740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56424668f0b0 con 0x564246384070 2024-07-26T13:47:02.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.445+0000 7efc91221740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5642466930f0 con 0x564246363390 2024-07-26T13:47:02.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.446+0000 7efc88a39640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56424635b1b0 0x564246729ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:02.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.446+0000 7efc8923a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564246384070 0x564246729570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:02.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.446+0000 7efc83fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564246363390 0x564246729ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:02.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.446+0000 7efc8923a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564246384070 0x564246729570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:42392/0 (socket says 172.21.15.100:42392) 2024-07-26T13:47:02.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.446+0000 7efc88a39640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56424635b1b0 0x564246729ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34720/0 (socket says 172.21.15.100:34720) 2024-07-26T13:47:02.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.446+0000 7efc8923a640 1 -- 172.21.15.100:0/3694793006 learned_addr learned my addr 172.21.15.100:0/3694793006 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:02.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.446+0000 7efc8923a640 1 -- 172.21.15.100:0/3694793006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564246363390 msgr2=0x564246729ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:02.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.446+0000 7efc8923a640 1 --2- 172.21.15.100:0/3694793006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564246363390 0x564246729ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:02.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.446+0000 7efc8923a640 1 -- 172.21.15.100:0/3694793006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56424635b1b0 msgr2=0x564246729ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:02.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.446+0000 7efc8923a640 1 --2- 172.21.15.100:0/3694793006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56424635b1b0 0x564246729ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:02.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.446+0000 7efc8923a640 1 -- 172.21.15.100:0/3694793006 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564246635b90 con 0x564246384070 2024-07-26T13:47:02.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.446+0000 7efc8923a640 1 --2- 172.21.15.100:0/3694793006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564246384070 0x564246729570 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efc7c009fd0 tx=0x7efc7c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc1b88fbb711ff1c server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:02.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.446+0000 7efc6affd640 1 -- 172.21.15.100:0/3694793006 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c004d20 con 0x564246384070 2024-07-26T13:47:02.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.446+0000 7efc81ffb640 1 --2- 172.21.15.100:0/1553798985 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7efc6c0b3d40 0x7efc6c0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:02.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.446+0000 7efc6affd640 1 -- 172.21.15.100:0/3694793006 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc7c004050 con 0x564246384070 2024-07-26T13:47:02.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.446+0000 7efc6affd640 1 -- 172.21.15.100:0/3694793006 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c005280 con 0x564246384070 2024-07-26T13:47:02.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.446+0000 7efc88a39640 1 --2- 172.21.15.100:0/1553798985 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7efc6c0b3d40 0x7efc6c0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:02.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.446+0000 7efc81ffb640 1 -- 172.21.15.100:0/1553798985 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7efc78104700 con 0x7efc84107dd0 2024-07-26T13:47:02.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.447+0000 7efc91221740 1 -- 172.21.15.100:0/3694793006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564246384070 msgr2=0x564246729570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:02.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.447+0000 7efc91221740 1 --2- 172.21.15.100:0/3694793006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564246384070 0x564246729570 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efc7c009fd0 tx=0x7efc7c0118d0 comp rx=0 tx=0).stop 2024-07-26T13:47:02.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.447+0000 7efc88a39640 1 --2- 172.21.15.100:0/1553798985 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7efc6c0b3d40 0x7efc6c0b6250 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x5642467281f0 tx=0x7efc74005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:02.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.447+0000 7efc91221740 1 -- 172.21.15.100:0/3694793006 shutdown_connections 2024-07-26T13:47:02.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.447+0000 7efc91221740 1 --2- 172.21.15.100:0/3694793006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564246363390 0x564246729ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:02.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.447+0000 7efc91221740 1 --2- 172.21.15.100:0/3694793006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56424635b1b0 0x564246729ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:02.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.447+0000 7efc91221740 1 --2- 172.21.15.100:0/3694793006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564246384070 0x564246729570 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:02.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.447+0000 7efc91221740 1 -- 172.21.15.100:0/3694793006 >> 172.21.15.100:0/3694793006 conn(0x564246378460 msgr2=0x564246727460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:02.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.447+0000 7efc91221740 1 -- 172.21.15.100:0/3694793006 shutdown_connections 2024-07-26T13:47:02.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.447+0000 7efc91221740 1 -- 172.21.15.100:0/3694793006 wait complete. 2024-07-26T13:47:02.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.448+0000 7efc91221740 1 -- start start 2024-07-26T13:47:02.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.448+0000 7efc91221740 1 Processor -- start 2024-07-26T13:47:02.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.448+0000 7efc91221740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56424635b1b0 0x5642467351d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:02.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.448+0000 7efc91221740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564246363390 0x564246735750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:02.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.448+0000 7efc91221740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564246384070 0x564246741070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:02.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.448+0000 7efc91221740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5642464f2700 con 0x564246363390 2024-07-26T13:47:02.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.448+0000 7efc91221740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5642466319e0 con 0x564246384070 2024-07-26T13:47:02.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.449+0000 7efc91221740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564246737d50 con 0x56424635b1b0 2024-07-26T13:47:02.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.449+0000 7efc91221740 10 client.0 _ll_register_callbacks cb 0x56424664c4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:47:02.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.449+0000 7efc83fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564246363390 0x564246735750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:02.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.449+0000 7efc83fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564246363390 0x564246735750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34732/0 (socket says 172.21.15.100:34732) 2024-07-26T13:47:02.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.449+0000 7efc83fff640 1 -- 172.21.15.100:0/3597725461 learned_addr learned my addr 172.21.15.100:0/3597725461 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:02.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.449+0000 7efc88a39640 1 --2- 172.21.15.100:0/3597725461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56424635b1b0 0x5642467351d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:02.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.449+0000 7efc8923a640 1 --2- 172.21.15.100:0/3597725461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564246384070 0x564246741070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:02.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.449+0000 7efc83fff640 1 -- 172.21.15.100:0/3597725461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56424635b1b0 msgr2=0x5642467351d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:02.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.449+0000 7efc83fff640 1 --2- 172.21.15.100:0/3597725461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56424635b1b0 0x5642467351d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:02.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.449+0000 7efc83fff640 1 -- 172.21.15.100:0/3597725461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564246384070 msgr2=0x564246741070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:02.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.449+0000 7efc83fff640 1 --2- 172.21.15.100:0/3597725461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564246384070 0x564246741070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:02.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.449+0000 7efc83fff640 1 -- 172.21.15.100:0/3597725461 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642463ef650 con 0x564246363390 2024-07-26T13:47:02.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.449+0000 7efc88a39640 1 --2- 172.21.15.100:0/3597725461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56424635b1b0 0x5642467351d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:47:02.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.449+0000 7efc8923a640 1 --2- 172.21.15.100:0/3597725461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564246384070 0x564246741070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:47:02.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.449+0000 7efc83fff640 1 --2- 172.21.15.100:0/3597725461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564246363390 0x564246735750 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7efc78101dd0 tx=0x7efc78009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e97e9c9f48d499b server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:02.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.449+0000 7efc63fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:47:02.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.449+0000 7efc63fff640 1 -- 172.21.15.100:0/3597725461 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc780cab80 con 0x564246363390 2024-07-26T13:47:02.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.449+0000 7efc63fff640 1 -- 172.21.15.100:0/3597725461 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7efc780cb490 con 0x564246363390 2024-07-26T13:47:02.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.449+0000 7efc91221740 1 -- 172.21.15.100:0/3597725461 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc7c004050 con 0x564246363390 2024-07-26T13:47:02.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.450+0000 7efc63fff640 1 -- 172.21.15.100:0/3597725461 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7efc780ca9a0 con 0x564246363390 2024-07-26T13:47:02.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.450+0000 7efc91221740 10 client.17271 fetch_fsmap learned FSMap version 54 2024-07-26T13:47:02.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.450+0000 7efc91221740 1 -- 172.21.15.100:0/3597725461 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x564246743de0 con 0x564246363390 2024-07-26T13:47:02.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.450+0000 7efc63fff640 1 -- 172.21.15.100:0/3597725461 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc7c004050 con 0x564246363390 2024-07-26T13:47:02.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.450+0000 7efc63fff640 1 -- 172.21.15.100:0/3597725461 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 54) ==== 8350+0+0 (secure 0 0 0) 0x7efc780d3e50 con 0x564246363390 2024-07-26T13:47:02.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.450+0000 7efc91221740 10 client.17271 fetch_fsmap finished waiting for FSMap version 54 2024-07-26T13:47:02.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.450+0000 7efc91221740 10 client.17271 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-07-26T13:47:02.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.450+0000 7efc91221740 20 client.17271 populate_metadata read hostname 'smithi100' 2024-07-26T13:47:02.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.450+0000 7efc91221740 1 --2- 172.21.15.100:0/3597725461 >> [v2:172.21.15.138:6834/2831339503,v1:172.21.15.138:6835/2831339503] conn(0x5642463429f0 0x5642467472c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:02.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.450+0000 7efc91221740 4 client.17271 mds_command: new command op to 25997 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:47:02.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.450+0000 7efc91221740 1 -- 172.21.15.100:0/3597725461 --> [v2:172.21.15.138:6834/2831339503,v1:172.21.15.138:6835/2831339503] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642466a1b00 con 0x5642463429f0 2024-07-26T13:47:02.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.450+0000 7efc88a39640 1 --2- 172.21.15.100:0/3597725461 >> [v2:172.21.15.138:6834/2831339503,v1:172.21.15.138:6835/2831339503] conn(0x5642463429f0 0x5642467472c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:02.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.451+0000 7efc63fff640 10 client.17271 ms_handle_connect on v2:172.21.15.138:6834/2831339503 2024-07-26T13:47:02.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.451+0000 7efc88a39640 1 --2- 172.21.15.100:0/3597725461 >> [v2:172.21.15.138:6834/2831339503,v1:172.21.15.138:6835/2831339503] conn(0x5642463429f0 0x5642467472c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fe64339acf6312e2 server_cookie=b079619aa00b085e in_seq=0 out_seq=0 2024-07-26T13:47:02.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.452+0000 7efc63fff640 1 -- 172.21.15.100:0/3597725461 <== mds.2 v2:172.21.15.138:6834/2831339503 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5642466a1b00 con 0x5642463429f0 2024-07-26T13:47:02.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.452+0000 7efc63fff640 10 client.17271 handle_command_reply: tid=0 2024-07-26T13:47:02.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.452+0000 7efc91221740 1 client.17271 shutdown 2024-07-26T13:47:02.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.452+0000 7efc91221740 1 -- 172.21.15.100:0/3597725461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564246363390 msgr2=0x564246735750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:02.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.452+0000 7efc91221740 1 --2- 172.21.15.100:0/3597725461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564246363390 0x564246735750 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7efc78101dd0 tx=0x7efc78009230 comp rx=0 tx=0).stop 2024-07-26T13:47:02.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.452+0000 7efc91221740 1 -- 172.21.15.100:0/3597725461 shutdown_connections 2024-07-26T13:47:02.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.452+0000 7efc91221740 1 --2- 172.21.15.100:0/3597725461 >> [v2:172.21.15.138:6834/2831339503,v1:172.21.15.138:6835/2831339503] conn(0x5642463429f0 0x5642467472c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:02.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.452+0000 7efc91221740 1 --2- 172.21.15.100:0/3597725461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564246384070 0x564246741070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:02.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.452+0000 7efc91221740 1 --2- 172.21.15.100:0/3597725461 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564246363390 0x564246735750 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:02.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.452+0000 7efc63fff640 0 client.17271 ms_handle_reset on v2:172.21.15.138:6834/2831339503 2024-07-26T13:47:02.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.452+0000 7efc91221740 1 --2- 172.21.15.100:0/3597725461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56424635b1b0 0x5642467351d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:02.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.452+0000 7efc91221740 1 -- 172.21.15.100:0/3597725461 >> 172.21.15.100:0/3597725461 conn(0x564246378460 msgr2=0x564246725c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:02.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.452+0000 7efc91221740 1 -- 172.21.15.100:0/3597725461 shutdown_connections 2024-07-26T13:47:02.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.452+0000 7efc91221740 1 -- 172.21.15.100:0/3597725461 wait complete. 2024-07-26T13:47:02.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.452+0000 7efc91221740 20 client.17271 trim_cache size 0 max 16384 2024-07-26T13:47:02.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.465+0000 7efc91221740 1 Processor -- start 2024-07-26T13:47:02.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.465+0000 7efc91221740 1 -- start start 2024-07-26T13:47:02.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.465+0000 7efc91221740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56424635b1b0 0x564246726350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:02.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.465+0000 7efc91221740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564246363390 0x564246740e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:02.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.465+0000 7efc91221740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564246384070 0x564246726e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:02.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.465+0000 7efc91221740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efc841b3d10 con 0x564246384070 2024-07-26T13:47:02.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.465+0000 7efc91221740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efc841b15b0 con 0x56424635b1b0 2024-07-26T13:47:02.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.465+0000 7efc91221740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efc841af400 con 0x564246363390 2024-07-26T13:47:02.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.465+0000 7efc8923a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564246384070 0x564246726e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:02.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.465+0000 7efc8923a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564246384070 0x564246726e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34746/0 (socket says 172.21.15.100:34746) 2024-07-26T13:47:02.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.466+0000 7efc8923a640 1 -- 172.21.15.100:0/3777119837 learned_addr learned my addr 172.21.15.100:0/3777119837 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:02.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.466+0000 7efc88a39640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56424635b1b0 0x564246726350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:02.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.466+0000 7efc83fff640 1 --2- 172.21.15.100:0/3777119837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564246363390 0x564246740e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:02.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.466+0000 7efc8923a640 1 -- 172.21.15.100:0/3777119837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564246363390 msgr2=0x564246740e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:02.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.466+0000 7efc8923a640 1 --2- 172.21.15.100:0/3777119837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564246363390 0x564246740e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:02.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.466+0000 7efc8923a640 1 -- 172.21.15.100:0/3777119837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56424635b1b0 msgr2=0x564246726350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:02.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.466+0000 7efc8923a640 1 --2- 172.21.15.100:0/3777119837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56424635b1b0 0x564246726350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:02.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.466+0000 7efc8923a640 1 -- 172.21.15.100:0/3777119837 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642467334d0 con 0x564246384070 2024-07-26T13:47:02.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.466+0000 7efc8923a640 1 --2- 172.21.15.100:0/3777119837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564246384070 0x564246726e50 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7efc7c003fd0 tx=0x7efc7c003ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d3b7d95448db4a server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:02.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.466+0000 7efc6a7fc640 1 -- 172.21.15.100:0/3777119837 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c004a00 con 0x564246384070 2024-07-26T13:47:02.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.466+0000 7efc6a7fc640 1 -- 172.21.15.100:0/3777119837 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc7c004ba0 con 0x564246384070 2024-07-26T13:47:02.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.466+0000 7efc6a7fc640 1 -- 172.21.15.100:0/3777119837 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c005290 con 0x564246384070 2024-07-26T13:47:02.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.467+0000 7efc91221740 1 -- 172.21.15.100:0/3777119837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564246384070 msgr2=0x564246726e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:02.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.467+0000 7efc91221740 1 --2- 172.21.15.100:0/3777119837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564246384070 0x564246726e50 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7efc7c003fd0 tx=0x7efc7c003ed0 comp rx=0 tx=0).stop 2024-07-26T13:47:02.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.467+0000 7efc91221740 1 -- 172.21.15.100:0/3777119837 shutdown_connections 2024-07-26T13:47:02.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.467+0000 7efc91221740 1 --2- 172.21.15.100:0/3777119837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564246384070 0x564246726e50 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:02.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.467+0000 7efc91221740 1 --2- 172.21.15.100:0/3777119837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564246363390 0x564246740e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:02.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.467+0000 7efc91221740 1 --2- 172.21.15.100:0/3777119837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56424635b1b0 0x564246726350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:02.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.467+0000 7efc91221740 1 -- 172.21.15.100:0/3777119837 >> 172.21.15.100:0/3777119837 conn(0x564246378460 msgr2=0x564246725c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:02.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.467+0000 7efc91221740 1 -- 172.21.15.100:0/3777119837 shutdown_connections 2024-07-26T13:47:02.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.467+0000 7efc91221740 1 -- 172.21.15.100:0/3777119837 wait complete. 2024-07-26T13:47:02.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.467+0000 7efc91221740 1 -- start start 2024-07-26T13:47:02.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.468+0000 7efc91221740 1 Processor -- start 2024-07-26T13:47:02.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.468+0000 7efc91221740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56424635b1b0 0x56424673d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:02.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.468+0000 7efc91221740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564246363390 0x564246738ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:02.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.468+0000 7efc91221740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564246384070 0x564246739000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:02.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.468+0000 7efc91221740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efc841afa70 con 0x56424635b1b0 2024-07-26T13:47:02.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.468+0000 7efc91221740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efc841b0ee0 con 0x564246363390 2024-07-26T13:47:02.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.468+0000 7efc91221740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efc841b0140 con 0x564246384070 2024-07-26T13:47:02.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.468+0000 7efc91221740 10 client.0 _ll_register_callbacks cb 0x7efc78005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:47:02.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.468+0000 7efc88a39640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56424635b1b0 0x56424673d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:02.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.468+0000 7efc88a39640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56424635b1b0 0x56424673d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34760/0 (socket says 172.21.15.100:34760) 2024-07-26T13:47:02.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.468+0000 7efc88a39640 1 -- 172.21.15.100:0/3458085659 learned_addr learned my addr 172.21.15.100:0/3458085659 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:02.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.468+0000 7efc83fff640 1 --2- 172.21.15.100:0/3458085659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564246363390 0x564246738ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:02.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.468+0000 7efc8923a640 1 --2- 172.21.15.100:0/3458085659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564246384070 0x564246739000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:02.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.468+0000 7efc88a39640 1 -- 172.21.15.100:0/3458085659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564246384070 msgr2=0x564246739000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:02.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.468+0000 7efc88a39640 1 --2- 172.21.15.100:0/3458085659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564246384070 0x564246739000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:02.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.468+0000 7efc88a39640 1 -- 172.21.15.100:0/3458085659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564246363390 msgr2=0x564246738ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:02.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.468+0000 7efc88a39640 1 --2- 172.21.15.100:0/3458085659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564246363390 0x564246738ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:02.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.468+0000 7efc88a39640 1 -- 172.21.15.100:0/3458085659 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642467396f0 con 0x56424635b1b0 2024-07-26T13:47:02.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.469+0000 7efc88a39640 1 --2- 172.21.15.100:0/3458085659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56424635b1b0 0x56424673d610 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x564246747a20 tx=0x7efc740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4003787647dcd9e server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:02.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.469+0000 7efc63fff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:47:02.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.469+0000 7efc63fff640 1 -- 172.21.15.100:0/3458085659 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc74003e50 con 0x56424635b1b0 2024-07-26T13:47:02.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.469+0000 7efc63fff640 1 -- 172.21.15.100:0/3458085659 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8742+0+0 (secure 0 0 0) 0x7efc74005020 con 0x56424635b1b0 2024-07-26T13:47:02.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.469+0000 7efc91221740 1 -- 172.21.15.100:0/3458085659 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc7c004ba0 con 0x56424635b1b0 2024-07-26T13:47:02.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.469+0000 7efc63fff640 1 -- 172.21.15.100:0/3458085659 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7efc74005220 con 0x56424635b1b0 2024-07-26T13:47:02.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.469+0000 7efc91221740 10 client.17277 fetch_fsmap learned FSMap version 54 2024-07-26T13:47:02.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.469+0000 7efc91221740 1 -- 172.21.15.100:0/3458085659 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5642467af980 con 0x56424635b1b0 2024-07-26T13:47:02.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.469+0000 7efc63fff640 1 -- 172.21.15.100:0/3458085659 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc7c004ba0 con 0x56424635b1b0 2024-07-26T13:47:02.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.469+0000 7efc63fff640 1 -- 172.21.15.100:0/3458085659 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 54) ==== 8350+0+0 (secure 0 0 0) 0x7efc7402b6b0 con 0x56424635b1b0 2024-07-26T13:47:02.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.469+0000 7efc91221740 10 client.17277 fetch_fsmap finished waiting for FSMap version 54 2024-07-26T13:47:02.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.469+0000 7efc91221740 10 client.17277 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-07-26T13:47:02.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.469+0000 7efc91221740 20 client.17277 populate_metadata read hostname 'smithi100' 2024-07-26T13:47:02.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.469+0000 7efc91221740 1 --2- 172.21.15.100:0/3458085659 >> [v2:172.21.15.138:6834/2831339503,v1:172.21.15.138:6835/2831339503] conn(0x564246317da0 0x5642467b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:02.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.470+0000 7efc91221740 4 client.17277 mds_command: new command op to 25997 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-26T13:47:02.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.470+0000 7efc91221740 1 -- 172.21.15.100:0/3458085659 --> [v2:172.21.15.138:6834/2831339503,v1:172.21.15.138:6835/2831339503] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564246735ae0 con 0x564246317da0 2024-07-26T13:47:02.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.470+0000 7efc83fff640 1 --2- 172.21.15.100:0/3458085659 >> [v2:172.21.15.138:6834/2831339503,v1:172.21.15.138:6835/2831339503] conn(0x564246317da0 0x5642467b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:02.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.470+0000 7efc63fff640 10 client.17277 ms_handle_connect on v2:172.21.15.138:6834/2831339503 2024-07-26T13:47:02.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.470+0000 7efc83fff640 1 --2- 172.21.15.100:0/3458085659 >> [v2:172.21.15.138:6834/2831339503,v1:172.21.15.138:6835/2831339503] conn(0x564246317da0 0x5642467b21a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dfb606dd5024ca7d server_cookie=89c7674d5d82ddc6 in_seq=0 out_seq=0 2024-07-26T13:47:02.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.470+0000 7efc63fff640 1 -- 172.21.15.100:0/3458085659 <== mds.2 v2:172.21.15.138:6834/2831339503 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564246735ae0 con 0x564246317da0 2024-07-26T13:47:02.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.470+0000 7efc63fff640 10 client.17277 handle_command_reply: tid=0 2024-07-26T13:47:02.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.470+0000 7efc91221740 1 client.17277 shutdown 2024-07-26T13:47:02.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.471+0000 7efc91221740 1 -- 172.21.15.100:0/3458085659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56424635b1b0 msgr2=0x56424673d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:02.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.471+0000 7efc91221740 1 --2- 172.21.15.100:0/3458085659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56424635b1b0 0x56424673d610 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x564246747a20 tx=0x7efc740046c0 comp rx=0 tx=0).stop 2024-07-26T13:47:02.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.471+0000 7efc91221740 1 -- 172.21.15.100:0/3458085659 shutdown_connections 2024-07-26T13:47:02.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.471+0000 7efc91221740 1 --2- 172.21.15.100:0/3458085659 >> [v2:172.21.15.138:6834/2831339503,v1:172.21.15.138:6835/2831339503] conn(0x564246317da0 0x5642467b21a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:02.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.471+0000 7efc91221740 1 --2- 172.21.15.100:0/3458085659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564246384070 0x564246739000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:02.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.471+0000 7efc91221740 1 --2- 172.21.15.100:0/3458085659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564246363390 0x564246738ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:02.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.471+0000 7efc63fff640 0 client.17277 ms_handle_reset on v2:172.21.15.138:6834/2831339503 2024-07-26T13:47:02.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.471+0000 7efc91221740 1 --2- 172.21.15.100:0/3458085659 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56424635b1b0 0x56424673d610 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:02.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.471+0000 7efc91221740 1 -- 172.21.15.100:0/3458085659 >> 172.21.15.100:0/3458085659 conn(0x564246378460 msgr2=0x56424672e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:02.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.471+0000 7efc91221740 1 -- 172.21.15.100:0/3458085659 shutdown_connections 2024-07-26T13:47:02.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.471+0000 7efc91221740 1 -- 172.21.15.100:0/3458085659 wait complete. 2024-07-26T13:47:02.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.471+0000 7efc91221740 20 client.17277 trim_cache size 0 max 16384 2024-07-26T13:47:02.477 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-07-26T13:47:02.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.472+0000 7efc8ac9d640 1 -- 172.21.15.100:0/1553798985 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7efc6c0b3d40 msgr2=0x7efc6c0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:02.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.472+0000 7efc8ac9d640 1 --2- 172.21.15.100:0/1553798985 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7efc6c0b3d40 0x7efc6c0b6250 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x5642467281f0 tx=0x7efc74005f90 comp rx=0 tx=0).stop 2024-07-26T13:47:02.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.472+0000 7efc8ac9d640 1 -- 172.21.15.100:0/1553798985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc84107dd0 msgr2=0x7efc8419c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:02.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.472+0000 7efc8ac9d640 1 --2- 172.21.15.100:0/1553798985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc84107dd0 0x7efc8419c5f0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7efc78005ab0 tx=0x7efc78031a20 comp rx=0 tx=0).stop 2024-07-26T13:47:02.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.472+0000 7efc8ac9d640 1 -- 172.21.15.100:0/1553798985 shutdown_connections 2024-07-26T13:47:02.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.472+0000 7efc8ac9d640 1 --2- 172.21.15.100:0/1553798985 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7efc6c0b3d40 0x7efc6c0b6250 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:02.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.472+0000 7efc8ac9d640 1 --2- 172.21.15.100:0/1553798985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc84108720 0x7efc841a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:02.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.472+0000 7efc8ac9d640 1 --2- 172.21.15.100:0/1553798985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc84107dd0 0x7efc8419c5f0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:02.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.472+0000 7efc8ac9d640 1 --2- 172.21.15.100:0/1553798985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc84102490 0x7efc8419c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:02.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.472+0000 7efc8ac9d640 1 -- 172.21.15.100:0/1553798985 >> 172.21.15.100:0/1553798985 conn(0x7efc84079160 msgr2=0x7efc8406a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:02.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.472+0000 7efc8ac9d640 1 -- 172.21.15.100:0/1553798985 shutdown_connections 2024-07-26T13:47:02.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:02.472+0000 7efc8ac9d640 1 -- 172.21.15.100:0/1553798985 wait complete. 2024-07-26T13:47:03.423 INFO:tasks.workunit.client.0.smithi100.stdout: 1 290669 3.96 MB/sec execute 197 sec latency 17.150 ms 2024-07-26T13:47:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:03 smithi138 ceph-mon[36954]: pgmap v575: 97 pgs: 97 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 235 op/s 2024-07-26T13:47:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:47:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:47:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:47:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:47:03.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:03 smithi100 ceph-mon[32599]: pgmap v575: 97 pgs: 97 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 235 op/s 2024-07-26T13:47:03.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:47:03.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:47:03.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:47:03.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:47:04.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:03 smithi167 ceph-mon[37140]: pgmap v575: 97 pgs: 97 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 235 op/s 2024-07-26T13:47:04.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:47:04.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:47:04.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:47:04.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:47:04.285 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-26T13:47:04.285 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-26T13:47:04.423 INFO:tasks.workunit.client.0.smithi100.stdout: 1 292042 3.97 MB/sec execute 198 sec latency 17.638 ms 2024-07-26T13:47:04.556 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:47:05.423 INFO:tasks.workunit.client.0.smithi100.stdout: 1 292494 3.97 MB/sec execute 199 sec latency 661.633 ms 2024-07-26T13:47:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:05 smithi138 ceph-mon[36954]: pgmap v576: 97 pgs: 97 active+clean; 109 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.4 MiB/s wr, 329 op/s 2024-07-26T13:47:05.980 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:05 smithi100 ceph-mon[32599]: pgmap v576: 97 pgs: 97 active+clean; 109 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.4 MiB/s wr, 329 op/s 2024-07-26T13:47:06.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:05 smithi167 ceph-mon[37140]: pgmap v576: 97 pgs: 97 active+clean; 109 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.4 MiB/s wr, 329 op/s 2024-07-26T13:47:06.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.343+0000 7f70b2795640 1 -- 172.21.15.100:0/4276032592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70ac108750 msgr2=0x7f70ac10ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:06.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.343+0000 7f70b2795640 1 --2- 172.21.15.100:0/4276032592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70ac108750 0x7f70ac10ef80 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f70a4009ca0 tx=0x7f70a402a8c0 comp rx=0 tx=0).stop 2024-07-26T13:47:06.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.343+0000 7f70b2795640 1 -- 172.21.15.100:0/4276032592 shutdown_connections 2024-07-26T13:47:06.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.343+0000 7f70b2795640 1 --2- 172.21.15.100:0/4276032592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70ac108750 0x7f70ac10ef80 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:06.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.343+0000 7f70b2795640 1 --2- 172.21.15.100:0/4276032592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70ac107e00 0x7f70ac108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:06.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.343+0000 7f70b2795640 1 --2- 172.21.15.100:0/4276032592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70ac102510 0x7f70ac102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:06.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.343+0000 7f70b2795640 1 -- 172.21.15.100:0/4276032592 >> 172.21.15.100:0/4276032592 conn(0x7f70ac079190 msgr2=0x7f70ac0795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:06.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.343+0000 7f70b2795640 1 -- 172.21.15.100:0/4276032592 shutdown_connections 2024-07-26T13:47:06.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.343+0000 7f70b2795640 1 -- 172.21.15.100:0/4276032592 wait complete. 2024-07-26T13:47:06.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.344+0000 7f70b2795640 1 Processor -- start 2024-07-26T13:47:06.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.344+0000 7f70b2795640 1 -- start start 2024-07-26T13:47:06.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.344+0000 7f70b2795640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70ac102510 0x7f70ac19ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:06.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.344+0000 7f70b2795640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70ac107e00 0x7f70ac19b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:06.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.344+0000 7f70b2795640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70ac108750 0x7f70ac19b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:06.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.344+0000 7f70b2795640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac112420 con 0x7f70ac102510 2024-07-26T13:47:06.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.344+0000 7f70b2795640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac1122a0 con 0x7f70ac108750 2024-07-26T13:47:06.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.344+0000 7f70b2795640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac1125a0 con 0x7f70ac107e00 2024-07-26T13:47:06.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.344+0000 7f70abfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70ac102510 0x7f70ac19ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:06.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.344+0000 7f70abfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70ac102510 0x7f70ac19ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50086/0 (socket says 172.21.15.100:50086) 2024-07-26T13:47:06.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.344+0000 7f70abfff640 1 -- 172.21.15.100:0/130015599 learned_addr learned my addr 172.21.15.100:0/130015599 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:06.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.344+0000 7f70b0d32640 1 --2- 172.21.15.100:0/130015599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70ac108750 0x7f70ac19b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:06.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.344+0000 7f70ab7fe640 1 --2- 172.21.15.100:0/130015599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70ac107e00 0x7f70ac19b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:06.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.345+0000 7f70abfff640 1 -- 172.21.15.100:0/130015599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70ac107e00 msgr2=0x7f70ac19b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:06.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.345+0000 7f70abfff640 1 --2- 172.21.15.100:0/130015599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70ac107e00 0x7f70ac19b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:06.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.345+0000 7f70abfff640 1 -- 172.21.15.100:0/130015599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70ac108750 msgr2=0x7f70ac19b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:06.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.345+0000 7f70abfff640 1 --2- 172.21.15.100:0/130015599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70ac108750 0x7f70ac19b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:06.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.345+0000 7f70abfff640 1 -- 172.21.15.100:0/130015599 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70ac1a5ef0 con 0x7f70ac102510 2024-07-26T13:47:06.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.345+0000 7f70abfff640 1 --2- 172.21.15.100:0/130015599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70ac102510 0x7f70ac19ac30 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f709c00d950 tx=0x7f709c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:06.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.345+0000 7f70a97fa640 1 -- 172.21.15.100:0/130015599 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709c017030 con 0x7f70ac102510 2024-07-26T13:47:06.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.345+0000 7f70a97fa640 1 -- 172.21.15.100:0/130015599 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f709c004020 con 0x7f70ac102510 2024-07-26T13:47:06.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.345+0000 7f70b2795640 1 -- 172.21.15.100:0/130015599 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70ac1a61e0 con 0x7f70ac102510 2024-07-26T13:47:06.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.345+0000 7f70a97fa640 1 -- 172.21.15.100:0/130015599 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709c0047a0 con 0x7f70ac102510 2024-07-26T13:47:06.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.345+0000 7f70b2795640 1 -- 172.21.15.100:0/130015599 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70ac1a6720 con 0x7f70ac102510 2024-07-26T13:47:06.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.347+0000 7f70b8d19740 1 Processor -- start 2024-07-26T13:47:06.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.348+0000 7f70a97fa640 1 -- 172.21.15.100:0/130015599 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f709c009a30 con 0x7f70ac102510 2024-07-26T13:47:06.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.348+0000 7f70b8d19740 1 -- start start 2024-07-26T13:47:06.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.348+0000 7f70b8d19740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ca30c3350 0x562ca34683b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:06.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.348+0000 7f70b8d19740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ca309a490 0x562ca34688f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:06.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.348+0000 7f70b8d19740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca30a2670 0x562ca3468e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:06.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.348+0000 7f70b8d19740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x562ca33502f0 con 0x562ca309a490 2024-07-26T13:47:06.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.348+0000 7f70b8d19740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562ca33dea80 con 0x562ca30a2670 2024-07-26T13:47:06.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.348+0000 7f70b8d19740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562ca33cdf80 con 0x562ca30c3350 2024-07-26T13:47:06.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.348+0000 7f70abfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ca309a490 0x562ca34688f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:06.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.348+0000 7f70abfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ca309a490 0x562ca34688f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50090/0 (socket says 172.21.15.100:50090) 2024-07-26T13:47:06.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.348+0000 7f70abfff640 1 -- 172.21.15.100:0/138115756 learned_addr learned my addr 172.21.15.100:0/138115756 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:06.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.348+0000 7f70b0d32640 1 --2- 172.21.15.100:0/138115756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ca30c3350 0x562ca34683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:06.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.348+0000 7f70ab7fe640 1 --2- 172.21.15.100:0/138115756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca30a2670 0x562ca3468e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:06.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.348+0000 7f70abfff640 1 -- 172.21.15.100:0/138115756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ca30c3350 msgr2=0x562ca34683b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:06.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.348+0000 7f70abfff640 1 --2- 172.21.15.100:0/138115756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ca30c3350 0x562ca34683b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:06.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.348+0000 7f70abfff640 1 -- 172.21.15.100:0/138115756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca30a2670 msgr2=0x562ca3468e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:06.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.348+0000 7f70abfff640 1 --2- 172.21.15.100:0/138115756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca30a2670 0x562ca3468e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:06.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.348+0000 7f70abfff640 1 -- 172.21.15.100:0/138115756 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ca312e930 con 0x562ca309a490 2024-07-26T13:47:06.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.348+0000 7f70a97fa640 1 --2- 172.21.15.100:0/130015599 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f70940b3d40 0x7f70940b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:06.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.349+0000 7f70b0d32640 1 --2- 172.21.15.100:0/138115756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ca30c3350 0x562ca34683b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:47:06.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.349+0000 7f70b0d32640 1 --2- 172.21.15.100:0/130015599 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f70940b3d40 0x7f70940b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:06.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.349+0000 7f70abfff640 1 --2- 172.21.15.100:0/138115756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ca309a490 0x562ca34688f0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f709c0e1100 tx=0x7f709c0ec6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deff0314744dd6a7 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:06.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.349+0000 7f70a97fa640 1 -- 172.21.15.100:0/130015599 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8830+0+0 (secure 0 0 0) 0x7f709c0e3b70 con 0x7f70ac102510 2024-07-26T13:47:06.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.349+0000 7f70927fc640 1 -- 172.21.15.100:0/138115756 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709c0aa420 con 0x562ca309a490 2024-07-26T13:47:06.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.349+0000 7f70927fc640 1 -- 172.21.15.100:0/138115756 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f709c0aa160 con 0x562ca309a490 2024-07-26T13:47:06.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.349+0000 7f70927fc640 1 -- 172.21.15.100:0/138115756 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709c0a9dc0 con 0x562ca309a490 2024-07-26T13:47:06.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.350+0000 7f70b8d19740 1 -- 172.21.15.100:0/138115756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ca309a490 msgr2=0x562ca34688f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:06.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.350+0000 7f70b0d32640 1 --2- 172.21.15.100:0/130015599 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f70940b3d40 0x7f70940b6250 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f70a4006010 tx=0x7f70a4005d80 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:06.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.350+0000 7f70b8d19740 1 --2- 172.21.15.100:0/138115756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ca309a490 0x562ca34688f0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f709c0e1100 tx=0x7f709c0ec6b0 comp rx=0 tx=0).stop 2024-07-26T13:47:06.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.350+0000 7f70b8d19740 1 -- 172.21.15.100:0/138115756 shutdown_connections 2024-07-26T13:47:06.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.350+0000 7f70b8d19740 1 --2- 172.21.15.100:0/138115756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca30a2670 0x562ca3468e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:06.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.350+0000 7f70b8d19740 1 --2- 172.21.15.100:0/138115756 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ca309a490 0x562ca34688f0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:06.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.350+0000 7f70b8d19740 1 --2- 172.21.15.100:0/138115756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ca30c3350 0x562ca34683b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:06.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.350+0000 7f70b8d19740 1 -- 172.21.15.100:0/138115756 >> 172.21.15.100:0/138115756 conn(0x562ca30b7740 msgr2=0x562ca34662a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:06.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.350+0000 7f70b8d19740 1 -- 172.21.15.100:0/138115756 shutdown_connections 2024-07-26T13:47:06.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.350+0000 7f70b8d19740 1 -- 172.21.15.100:0/138115756 wait complete. 2024-07-26T13:47:06.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.350+0000 7f70b8d19740 1 -- start start 2024-07-26T13:47:06.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.351+0000 7f70b8d19740 1 Processor -- start 2024-07-26T13:47:06.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.351+0000 7f70b8d19740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ca309a490 0x562ca3473ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:06.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.351+0000 7f70b8d19740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ca30a2670 0x562ca3474420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:06.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.351+0000 7f70b8d19740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca30c3350 0x562ca347fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:06.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.351+0000 7f70b8d19740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x562ca33704f0 con 0x562ca30a2670 2024-07-26T13:47:06.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.351+0000 7f70b8d19740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562ca33d1fc0 con 0x562ca30c3350 2024-07-26T13:47:06.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.351+0000 7f70b8d19740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562ca3231890 con 0x562ca309a490 2024-07-26T13:47:06.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.351+0000 7f70b8d19740 10 client.0 _ll_register_callbacks cb 0x562ca337a500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:47:06.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.351+0000 7f70ab7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ca30a2670 0x562ca3474420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:06.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.351+0000 7f70ab7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ca30a2670 0x562ca3474420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50098/0 (socket says 172.21.15.100:50098) 2024-07-26T13:47:06.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.351+0000 7f70ab7fe640 1 -- 172.21.15.100:0/3723985946 learned_addr learned my addr 172.21.15.100:0/3723985946 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:06.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.351+0000 7f70abfff640 1 --2- 172.21.15.100:0/3723985946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ca309a490 0x562ca3473ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:06.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.351+0000 7f70b0d32640 1 --2- 172.21.15.100:0/3723985946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca30c3350 0x562ca347fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:06.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.351+0000 7f70ab7fe640 1 -- 172.21.15.100:0/3723985946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ca309a490 msgr2=0x562ca3473ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:06.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.351+0000 7f70ab7fe640 1 --2- 172.21.15.100:0/3723985946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ca309a490 0x562ca3473ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:06.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.351+0000 7f70ab7fe640 1 -- 172.21.15.100:0/3723985946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca30c3350 msgr2=0x562ca347fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:06.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.351+0000 7f70ab7fe640 1 --2- 172.21.15.100:0/3723985946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca30c3350 0x562ca347fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:06.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.351+0000 7f70ab7fe640 1 -- 172.21.15.100:0/3723985946 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ca337fc50 con 0x562ca30a2670 2024-07-26T13:47:06.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.351+0000 7f70b0d32640 1 --2- 172.21.15.100:0/3723985946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca30c3350 0x562ca347fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:47:06.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.352+0000 7f70ab7fe640 1 --2- 172.21.15.100:0/3723985946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ca30a2670 0x562ca3474420 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f70a000b990 tx=0x7f70a000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd0548df57d544c7 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:06.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.352+0000 7f708b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:47:06.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.352+0000 7f708b7fe640 1 -- 172.21.15.100:0/3723985946 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a0003e00 con 0x562ca30a2670 2024-07-26T13:47:06.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.352+0000 7f708b7fe640 1 -- 172.21.15.100:0/3723985946 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8830+0+0 (secure 0 0 0) 0x7f70a0005020 con 0x562ca30a2670 2024-07-26T13:47:06.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.352+0000 7f70b8d19740 1 -- 172.21.15.100:0/3723985946 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f709c0aa160 con 0x562ca30a2670 2024-07-26T13:47:06.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.352+0000 7f708b7fe640 1 -- 172.21.15.100:0/3723985946 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f70a0005220 con 0x562ca30a2670 2024-07-26T13:47:06.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.352+0000 7f70b8d19740 10 client.17289 fetch_fsmap learned FSMap version 54 2024-07-26T13:47:06.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.352+0000 7f70b8d19740 1 -- 172.21.15.100:0/3723985946 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x562ca3482a00 con 0x562ca30a2670 2024-07-26T13:47:06.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.352+0000 7f708b7fe640 1 -- 172.21.15.100:0/3723985946 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f709c0aa160 con 0x562ca30a2670 2024-07-26T13:47:06.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.352+0000 7f708b7fe640 1 -- 172.21.15.100:0/3723985946 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 54) ==== 8350+0+0 (secure 0 0 0) 0x7f70a001b470 con 0x562ca30a2670 2024-07-26T13:47:06.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.352+0000 7f70b8d19740 10 client.17289 fetch_fsmap finished waiting for FSMap version 54 2024-07-26T13:47:06.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.352+0000 7f70b8d19740 10 client.17289 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:47:06.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.352+0000 7f70b8d19740 20 client.17289 populate_metadata read hostname 'smithi100' 2024-07-26T13:47:06.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.352+0000 7f70b8d19740 1 --2- 172.21.15.100:0/3723985946 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x562ca3081cd0 0x562ca3485f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:06.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.352+0000 7f70b8d19740 4 client.17289 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:47:06.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.352+0000 7f70b8d19740 1 -- 172.21.15.100:0/3723985946 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ca33e09d0 con 0x562ca3081cd0 2024-07-26T13:47:06.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.353+0000 7f70abfff640 1 --2- 172.21.15.100:0/3723985946 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x562ca3081cd0 0x562ca3485f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:06.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.353+0000 7f708b7fe640 10 client.17289 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:06.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.353+0000 7f70abfff640 1 --2- 172.21.15.100:0/3723985946 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x562ca3081cd0 0x562ca3485f20 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad4adcaf6694582d server_cookie=1d5877a3c0cbe129 in_seq=0 out_seq=0 2024-07-26T13:47:06.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.354+0000 7f708b7fe640 1 -- 172.21.15.100:0/3723985946 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562ca33e09d0 con 0x562ca3081cd0 2024-07-26T13:47:06.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.354+0000 7f708b7fe640 10 client.17289 handle_command_reply: tid=0 2024-07-26T13:47:06.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.354+0000 7f70b8d19740 1 client.17289 shutdown 2024-07-26T13:47:06.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.354+0000 7f70b8d19740 1 -- 172.21.15.100:0/3723985946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ca30a2670 msgr2=0x562ca3474420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:06.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.354+0000 7f70b8d19740 1 --2- 172.21.15.100:0/3723985946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ca30a2670 0x562ca3474420 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f70a000b990 tx=0x7f70a000be50 comp rx=0 tx=0).stop 2024-07-26T13:47:06.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.354+0000 7f70b8d19740 1 -- 172.21.15.100:0/3723985946 shutdown_connections 2024-07-26T13:47:06.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.354+0000 7f70b8d19740 1 --2- 172.21.15.100:0/3723985946 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x562ca3081cd0 0x562ca3485f20 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:06.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.354+0000 7f70b8d19740 1 --2- 172.21.15.100:0/3723985946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca30c3350 0x562ca347fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:06.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.354+0000 7f70b8d19740 1 --2- 172.21.15.100:0/3723985946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ca30a2670 0x562ca3474420 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:06.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.354+0000 7f70b8d19740 1 --2- 172.21.15.100:0/3723985946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ca309a490 0x562ca3473ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:06.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.354+0000 7f708b7fe640 0 client.17289 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:06.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.354+0000 7f70b8d19740 1 -- 172.21.15.100:0/3723985946 >> 172.21.15.100:0/3723985946 conn(0x562ca30b7740 msgr2=0x562ca346b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:06.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.354+0000 7f70b8d19740 1 -- 172.21.15.100:0/3723985946 shutdown_connections 2024-07-26T13:47:06.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.354+0000 7f70b8d19740 1 -- 172.21.15.100:0/3723985946 wait complete. 2024-07-26T13:47:06.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.354+0000 7f70b8d19740 20 client.17289 trim_cache size 0 max 16384 2024-07-26T13:47:06.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.371+0000 7f70b8d19740 1 Processor -- start 2024-07-26T13:47:06.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.371+0000 7f70b8d19740 1 -- start start 2024-07-26T13:47:06.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.371+0000 7f70b8d19740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca309a490 0x562ca3465280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:06.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.372+0000 7f70b8d19740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ca30a2670 0x562ca3466230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:06.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.372+0000 7f70b8d19740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ca30c3350 0x562ca3475910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:06.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.372+0000 7f70b8d19740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac1b1d50 con 0x562ca30a2670 2024-07-26T13:47:06.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.372+0000 7f70b8d19740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac1af5f0 con 0x562ca309a490 2024-07-26T13:47:06.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.372+0000 7f70b8d19740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac1ad460 con 0x562ca30c3350 2024-07-26T13:47:06.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.372+0000 7f70ab7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ca30a2670 0x562ca3466230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:06.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.372+0000 7f70ab7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ca30a2670 0x562ca3466230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50100/0 (socket says 172.21.15.100:50100) 2024-07-26T13:47:06.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.372+0000 7f70ab7fe640 1 -- 172.21.15.100:0/2336221237 learned_addr learned my addr 172.21.15.100:0/2336221237 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:06.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.372+0000 7f70abfff640 1 --2- 172.21.15.100:0/2336221237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca309a490 0x562ca3465280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:06.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.372+0000 7f70b0d32640 1 --2- 172.21.15.100:0/2336221237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ca30c3350 0x562ca3475910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:06.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.372+0000 7f70ab7fe640 1 -- 172.21.15.100:0/2336221237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ca30c3350 msgr2=0x562ca3475910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:06.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.372+0000 7f70ab7fe640 1 --2- 172.21.15.100:0/2336221237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ca30c3350 0x562ca3475910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:06.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.372+0000 7f70ab7fe640 1 -- 172.21.15.100:0/2336221237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca309a490 msgr2=0x562ca3465280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:06.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.372+0000 7f70ab7fe640 1 --2- 172.21.15.100:0/2336221237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca309a490 0x562ca3465280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:06.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.372+0000 7f70ab7fe640 1 -- 172.21.15.100:0/2336221237 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ca3474fa0 con 0x562ca30a2670 2024-07-26T13:47:06.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.372+0000 7f70b0d32640 1 --2- 172.21.15.100:0/2336221237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ca30c3350 0x562ca3475910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:47:06.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.372+0000 7f70ab7fe640 1 --2- 172.21.15.100:0/2336221237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ca30a2670 0x562ca3466230 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f70a0000c00 tx=0x7f70a0003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f8a98eeea6e027 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:06.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.372+0000 7f7091ffb640 1 -- 172.21.15.100:0/2336221237 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a001b470 con 0x562ca30a2670 2024-07-26T13:47:06.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.372+0000 7f7091ffb640 1 -- 172.21.15.100:0/2336221237 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70a001b610 con 0x562ca30a2670 2024-07-26T13:47:06.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.372+0000 7f7091ffb640 1 -- 172.21.15.100:0/2336221237 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a00219a0 con 0x562ca30a2670 2024-07-26T13:47:06.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.373+0000 7f70b8d19740 1 -- 172.21.15.100:0/2336221237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ca30a2670 msgr2=0x562ca3466230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:06.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.373+0000 7f70b8d19740 1 --2- 172.21.15.100:0/2336221237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ca30a2670 0x562ca3466230 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f70a0000c00 tx=0x7f70a0003f30 comp rx=0 tx=0).stop 2024-07-26T13:47:06.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.373+0000 7f70b8d19740 1 -- 172.21.15.100:0/2336221237 shutdown_connections 2024-07-26T13:47:06.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.373+0000 7f70b8d19740 1 --2- 172.21.15.100:0/2336221237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ca30c3350 0x562ca3475910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:06.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.373+0000 7f70b8d19740 1 --2- 172.21.15.100:0/2336221237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ca30a2670 0x562ca3466230 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:06.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.373+0000 7f70b8d19740 1 --2- 172.21.15.100:0/2336221237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca309a490 0x562ca3465280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:06.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.373+0000 7f70b8d19740 1 -- 172.21.15.100:0/2336221237 >> 172.21.15.100:0/2336221237 conn(0x562ca30b7740 msgr2=0x562ca3475200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:06.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.373+0000 7f70b8d19740 1 -- 172.21.15.100:0/2336221237 shutdown_connections 2024-07-26T13:47:06.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.373+0000 7f70b8d19740 1 -- 172.21.15.100:0/2336221237 wait complete. 2024-07-26T13:47:06.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.374+0000 7f70b8d19740 1 -- start start 2024-07-26T13:47:06.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.374+0000 7f70b8d19740 1 Processor -- start 2024-07-26T13:47:06.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.374+0000 7f70b8d19740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ca309a490 0x562ca3477bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:06.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.374+0000 7f70b8d19740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca30a2670 0x562ca3478110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:06.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.374+0000 7f70b8d19740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ca30c3350 0x562ca3478650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:06.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.374+0000 7f70b8d19740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac1adab0 con 0x562ca309a490 2024-07-26T13:47:06.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.374+0000 7f70b8d19740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac1aef20 con 0x562ca30a2670 2024-07-26T13:47:06.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.374+0000 7f70b8d19740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac1ae180 con 0x562ca30c3350 2024-07-26T13:47:06.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.374+0000 7f70b8d19740 10 client.0 _ll_register_callbacks cb 0x7f70a0004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:47:06.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.375+0000 7f70abfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ca309a490 0x562ca3477bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:06.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.375+0000 7f70abfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ca309a490 0x562ca3477bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50116/0 (socket says 172.21.15.100:50116) 2024-07-26T13:47:06.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.375+0000 7f70abfff640 1 -- 172.21.15.100:0/2128271063 learned_addr learned my addr 172.21.15.100:0/2128271063 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:06.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.375+0000 7f70ab7fe640 1 --2- 172.21.15.100:0/2128271063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca30a2670 0x562ca3478110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:06.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.375+0000 7f70b0d32640 1 --2- 172.21.15.100:0/2128271063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ca30c3350 0x562ca3478650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:06.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.375+0000 7f70abfff640 1 -- 172.21.15.100:0/2128271063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ca30c3350 msgr2=0x562ca3478650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:06.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.375+0000 7f70abfff640 1 --2- 172.21.15.100:0/2128271063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ca30c3350 0x562ca3478650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:06.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.375+0000 7f70abfff640 1 -- 172.21.15.100:0/2128271063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca30a2670 msgr2=0x562ca3478110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:06.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.375+0000 7f70abfff640 1 --2- 172.21.15.100:0/2128271063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca30a2670 0x562ca3478110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:06.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.375+0000 7f70abfff640 1 -- 172.21.15.100:0/2128271063 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ca34ed760 con 0x562ca309a490 2024-07-26T13:47:06.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.375+0000 7f70abfff640 1 --2- 172.21.15.100:0/2128271063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ca309a490 0x562ca3477bd0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f709c0a78c0 tx=0x7f709c0a6da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a36bfde1802ae5c server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:06.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.375+0000 7f708b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:47:06.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.375+0000 7f708b7fe640 1 -- 172.21.15.100:0/2128271063 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709c0a9fa0 con 0x562ca309a490 2024-07-26T13:47:06.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.375+0000 7f708b7fe640 1 -- 172.21.15.100:0/2128271063 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8830+0+0 (secure 0 0 0) 0x7f709c0e7490 con 0x562ca309a490 2024-07-26T13:47:06.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.375+0000 7f70b8d19740 1 -- 172.21.15.100:0/2128271063 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70a001b610 con 0x562ca309a490 2024-07-26T13:47:06.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.375+0000 7f708b7fe640 1 -- 172.21.15.100:0/2128271063 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f709c0a9dc0 con 0x562ca309a490 2024-07-26T13:47:06.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.375+0000 7f70b8d19740 10 client.17295 fetch_fsmap learned FSMap version 54 2024-07-26T13:47:06.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.375+0000 7f70b8d19740 1 -- 172.21.15.100:0/2128271063 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x562ca34eea20 con 0x562ca309a490 2024-07-26T13:47:06.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.376+0000 7f708b7fe640 1 -- 172.21.15.100:0/2128271063 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70a001b610 con 0x562ca309a490 2024-07-26T13:47:06.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.376+0000 7f708b7fe640 1 -- 172.21.15.100:0/2128271063 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 54) ==== 8350+0+0 (secure 0 0 0) 0x7f709c0c3020 con 0x562ca309a490 2024-07-26T13:47:06.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.376+0000 7f70b8d19740 10 client.17295 fetch_fsmap finished waiting for FSMap version 54 2024-07-26T13:47:06.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.376+0000 7f70b8d19740 10 client.17295 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:47:06.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.376+0000 7f70b8d19740 20 client.17295 populate_metadata read hostname 'smithi100' 2024-07-26T13:47:06.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.376+0000 7f70b8d19740 1 --2- 172.21.15.100:0/2128271063 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x562ca3057080 0x562ca34f0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:06.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.376+0000 7f70b8d19740 4 client.17295 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-26T13:47:06.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.376+0000 7f70b8d19740 1 -- 172.21.15.100:0/2128271063 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562ca33dc470 con 0x562ca3057080 2024-07-26T13:47:06.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.376+0000 7f70ab7fe640 1 --2- 172.21.15.100:0/2128271063 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x562ca3057080 0x562ca34f0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:06.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.376+0000 7f708b7fe640 10 client.17295 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:06.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.376+0000 7f70ab7fe640 1 --2- 172.21.15.100:0/2128271063 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x562ca3057080 0x562ca34f0d60 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c2f2dfe4a2dec39 server_cookie=8c923e83f845f9e4 in_seq=0 out_seq=0 2024-07-26T13:47:06.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.377+0000 7f708b7fe640 1 -- 172.21.15.100:0/2128271063 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562ca33dc470 con 0x562ca3057080 2024-07-26T13:47:06.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.377+0000 7f708b7fe640 10 client.17295 handle_command_reply: tid=0 2024-07-26T13:47:06.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.377+0000 7f70b8d19740 1 client.17295 shutdown 2024-07-26T13:47:06.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.377+0000 7f70b8d19740 1 -- 172.21.15.100:0/2128271063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ca309a490 msgr2=0x562ca3477bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:06.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.377+0000 7f70b8d19740 1 --2- 172.21.15.100:0/2128271063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ca309a490 0x562ca3477bd0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f709c0a78c0 tx=0x7f709c0a6da0 comp rx=0 tx=0).stop 2024-07-26T13:47:06.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.378+0000 7f70b8d19740 1 -- 172.21.15.100:0/2128271063 shutdown_connections 2024-07-26T13:47:06.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.378+0000 7f70b8d19740 1 --2- 172.21.15.100:0/2128271063 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x562ca3057080 0x562ca34f0d60 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:06.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.378+0000 7f70b8d19740 1 --2- 172.21.15.100:0/2128271063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562ca30c3350 0x562ca3478650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:06.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.378+0000 7f70b8d19740 1 --2- 172.21.15.100:0/2128271063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ca30a2670 0x562ca3478110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:06.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.378+0000 7f70b8d19740 1 --2- 172.21.15.100:0/2128271063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ca309a490 0x562ca3477bd0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:06.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.378+0000 7f70b8d19740 1 -- 172.21.15.100:0/2128271063 >> 172.21.15.100:0/2128271063 conn(0x562ca30b7740 msgr2=0x562ca3465050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:06.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.378+0000 7f708b7fe640 0 client.17295 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:06.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.378+0000 7f70b8d19740 1 -- 172.21.15.100:0/2128271063 shutdown_connections 2024-07-26T13:47:06.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.378+0000 7f70b8d19740 1 -- 172.21.15.100:0/2128271063 wait complete. 2024-07-26T13:47:06.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.378+0000 7f70b8d19740 20 client.17295 trim_cache size 0 max 16384 2024-07-26T13:47:06.385 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:47:06.385 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-07-26T13:47:06.385 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "f200fd50-e326-4bfc-976e-d04881835bdc", 2024-07-26T13:47:06.385 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-07-26T13:47:06.385 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:47:06.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.378+0000 7f70b2795640 1 -- 172.21.15.100:0/130015599 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f70940b3d40 msgr2=0x7f70940b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:06.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.378+0000 7f70b2795640 1 --2- 172.21.15.100:0/130015599 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f70940b3d40 0x7f70940b6250 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f70a4006010 tx=0x7f70a4005d80 comp rx=0 tx=0).stop 2024-07-26T13:47:06.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.378+0000 7f70b2795640 1 -- 172.21.15.100:0/130015599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70ac102510 msgr2=0x7f70ac19ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:06.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.378+0000 7f70b2795640 1 --2- 172.21.15.100:0/130015599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70ac102510 0x7f70ac19ac30 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f709c00d950 tx=0x7f709c00de10 comp rx=0 tx=0).stop 2024-07-26T13:47:06.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.379+0000 7f70b2795640 1 -- 172.21.15.100:0/130015599 shutdown_connections 2024-07-26T13:47:06.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.379+0000 7f70b2795640 1 --2- 172.21.15.100:0/130015599 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f70940b3d40 0x7f70940b6250 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:06.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.379+0000 7f70b2795640 1 --2- 172.21.15.100:0/130015599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70ac108750 0x7f70ac19b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:06.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.379+0000 7f70b2795640 1 --2- 172.21.15.100:0/130015599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70ac107e00 0x7f70ac19b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:06.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.379+0000 7f70b2795640 1 --2- 172.21.15.100:0/130015599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70ac102510 0x7f70ac19ac30 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:06.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.379+0000 7f70b2795640 1 -- 172.21.15.100:0/130015599 >> 172.21.15.100:0/130015599 conn(0x7f70ac079190 msgr2=0x7f70ac104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:06.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.379+0000 7f70b2795640 1 -- 172.21.15.100:0/130015599 shutdown_connections 2024-07-26T13:47:06.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:06.379+0000 7f70b2795640 1 -- 172.21.15.100:0/130015599 wait complete. 2024-07-26T13:47:06.423 INFO:tasks.workunit.client.0.smithi100.stdout: 1 292494 3.95 MB/sec execute 200 sec latency 1661.717 ms 2024-07-26T13:47:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:06 smithi138 ceph-mon[36954]: osdmap e85: 12 total, 12 up, 12 in 2024-07-26T13:47:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:06 smithi138 ceph-mon[36954]: pgmap v578: 97 pgs: 97 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.2 MiB/s wr, 307 op/s 2024-07-26T13:47:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:06 smithi100 ceph-mon[32599]: osdmap e85: 12 total, 12 up, 12 in 2024-07-26T13:47:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:06 smithi100 ceph-mon[32599]: pgmap v578: 97 pgs: 97 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.2 MiB/s wr, 307 op/s 2024-07-26T13:47:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:06 smithi167 ceph-mon[37140]: osdmap e85: 12 total, 12 up, 12 in 2024-07-26T13:47:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:06 smithi167 ceph-mon[37140]: pgmap v578: 97 pgs: 97 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.2 MiB/s wr, 307 op/s 2024-07-26T13:47:07.175 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-26T13:47:07.423 INFO:tasks.workunit.client.0.smithi100.stdout: 1 292494 3.93 MB/sec execute 201 sec latency 2661.747 ms 2024-07-26T13:47:07.431 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:47:07.835 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:07 smithi100 ceph-mon[32599]: scrub queued for path: / 2024-07-26T13:47:07.835 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:07 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:47:07.835 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:07 smithi100 ceph-mon[32599]: scrub summary: active paths [/] 2024-07-26T13:47:07.835 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:07 smithi100 ceph-mon[32599]: scrub summary: active paths [/] 2024-07-26T13:47:07.835 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:07 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:47:07.836 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:07 smithi100 ceph-mon[32599]: scrub summary: active paths [/] 2024-07-26T13:47:07.836 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:07 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:47:07.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:07 smithi138 ceph-mon[36954]: scrub queued for path: / 2024-07-26T13:47:07.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:07 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:47:07.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:07 smithi138 ceph-mon[36954]: scrub summary: active paths [/] 2024-07-26T13:47:07.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:07 smithi138 ceph-mon[36954]: scrub summary: active paths [/] 2024-07-26T13:47:07.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:07 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:47:07.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:07 smithi138 ceph-mon[36954]: scrub summary: active paths [/] 2024-07-26T13:47:07.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:07 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:47:08.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:07 smithi167 ceph-mon[37140]: scrub queued for path: / 2024-07-26T13:47:08.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:07 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:47:08.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:07 smithi167 ceph-mon[37140]: scrub summary: active paths [/] 2024-07-26T13:47:08.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:07 smithi167 ceph-mon[37140]: scrub summary: active paths [/] 2024-07-26T13:47:08.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:07 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:47:08.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:07 smithi167 ceph-mon[37140]: scrub summary: active paths [/] 2024-07-26T13:47:08.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:07 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:47:08.423 INFO:tasks.workunit.client.0.smithi100.stdout: 1 292494 3.91 MB/sec execute 202 sec latency 3661.855 ms 2024-07-26T13:47:08.859 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:08 smithi100 ceph-mon[32599]: pgmap v579: 97 pgs: 97 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 303 op/s 2024-07-26T13:47:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:08 smithi138 ceph-mon[36954]: pgmap v579: 97 pgs: 97 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 303 op/s 2024-07-26T13:47:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:08 smithi167 ceph-mon[37140]: pgmap v579: 97 pgs: 97 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 303 op/s 2024-07-26T13:47:09.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.251+0000 7fe35580d640 1 -- 172.21.15.100:0/51197054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe350108720 msgr2=0x7fe35010ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:09.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.251+0000 7fe35580d640 1 --2- 172.21.15.100:0/51197054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe350108720 0x7fe35010ef50 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fe348009900 tx=0x7fe34802a410 comp rx=0 tx=0).stop 2024-07-26T13:47:09.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.251+0000 7fe35580d640 1 -- 172.21.15.100:0/51197054 shutdown_connections 2024-07-26T13:47:09.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.251+0000 7fe35580d640 1 --2- 172.21.15.100:0/51197054 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe350108720 0x7fe35010ef50 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:09.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.251+0000 7fe35580d640 1 --2- 172.21.15.100:0/51197054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe350107dd0 0x7fe3501081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:09.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.251+0000 7fe35580d640 1 --2- 172.21.15.100:0/51197054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe350102490 0x7fe3501028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:09.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.251+0000 7fe35580d640 1 -- 172.21.15.100:0/51197054 >> 172.21.15.100:0/51197054 conn(0x7fe350079160 msgr2=0x7fe350079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:09.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.251+0000 7fe35580d640 1 -- 172.21.15.100:0/51197054 shutdown_connections 2024-07-26T13:47:09.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.251+0000 7fe35580d640 1 -- 172.21.15.100:0/51197054 wait complete. 2024-07-26T13:47:09.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.252+0000 7fe35580d640 1 Processor -- start 2024-07-26T13:47:09.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.252+0000 7fe35580d640 1 -- start start 2024-07-26T13:47:09.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.252+0000 7fe35580d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe350102490 0x7fe35019ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:09.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.252+0000 7fe35580d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe350107dd0 0x7fe35019b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:09.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.252+0000 7fe35580d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe350108720 0x7fe35019b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:09.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.252+0000 7fe35580d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe3501123a0 con 0x7fe350102490 2024-07-26T13:47:09.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.252+0000 7fe35580d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe350112220 con 0x7fe350108720 2024-07-26T13:47:09.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.252+0000 7fe35580d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe350112520 con 0x7fe350107dd0 2024-07-26T13:47:09.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.252+0000 7fe34effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe350102490 0x7fe35019ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:09.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.252+0000 7fe34effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe350102490 0x7fe35019ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50132/0 (socket says 172.21.15.100:50132) 2024-07-26T13:47:09.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.252+0000 7fe34effd640 1 -- 172.21.15.100:0/1514147566 learned_addr learned my addr 172.21.15.100:0/1514147566 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:09.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.252+0000 7fe34e7fc640 1 --2- 172.21.15.100:0/1514147566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe350107dd0 0x7fe35019b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:09.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.253+0000 7fe34f7fe640 1 --2- 172.21.15.100:0/1514147566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe350108720 0x7fe35019b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:09.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.253+0000 7fe34effd640 1 -- 172.21.15.100:0/1514147566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe350107dd0 msgr2=0x7fe35019b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:09.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.253+0000 7fe34effd640 1 --2- 172.21.15.100:0/1514147566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe350107dd0 0x7fe35019b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:09.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.253+0000 7fe34effd640 1 -- 172.21.15.100:0/1514147566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe350108720 msgr2=0x7fe35019b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:09.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.253+0000 7fe34effd640 1 --2- 172.21.15.100:0/1514147566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe350108720 0x7fe35019b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:09.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.253+0000 7fe34effd640 1 -- 172.21.15.100:0/1514147566 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3501a5fc0 con 0x7fe350102490 2024-07-26T13:47:09.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.253+0000 7fe34effd640 1 --2- 172.21.15.100:0/1514147566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe350102490 0x7fe35019ad90 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fe34000cce0 tx=0x7fe34000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:09.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.253+0000 7fe35480b640 1 -- 172.21.15.100:0/1514147566 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe340004b20 con 0x7fe350102490 2024-07-26T13:47:09.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.253+0000 7fe35580d640 1 -- 172.21.15.100:0/1514147566 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3501a62b0 con 0x7fe350102490 2024-07-26T13:47:09.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.253+0000 7fe35480b640 1 -- 172.21.15.100:0/1514147566 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe340004cc0 con 0x7fe350102490 2024-07-26T13:47:09.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.253+0000 7fe35580d640 1 -- 172.21.15.100:0/1514147566 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3501a67f0 con 0x7fe350102490 2024-07-26T13:47:09.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.253+0000 7fe35480b640 1 -- 172.21.15.100:0/1514147566 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34001d060 con 0x7fe350102490 2024-07-26T13:47:09.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.255+0000 7fe35bd91740 1 Processor -- start 2024-07-26T13:47:09.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.256+0000 7fe35bd91740 1 -- start start 2024-07-26T13:47:09.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.256+0000 7fe35480b640 1 -- 172.21.15.100:0/1514147566 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fe340005000 con 0x7fe350102490 2024-07-26T13:47:09.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.256+0000 7fe35bd91740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f65f740070 0x55f65fae5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:09.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.256+0000 7fe35bd91740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f65f7171b0 0x55f65fae5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:09.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.256+0000 7fe35bd91740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f65f71f390 0x55f65fae5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:09.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.256+0000 7fe35bd91740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55f65fa5bb60 con 0x55f65f71f390 2024-07-26T13:47:09.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.256+0000 7fe35bd91740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f65fa4b060 con 0x55f65f740070 2024-07-26T13:47:09.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.256+0000 7fe35bd91740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f65fa4f0a0 con 0x55f65f7171b0 2024-07-26T13:47:09.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.256+0000 7fe34e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f65f71f390 0x55f65fae5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:09.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.256+0000 7fe34e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f65f71f390 0x55f65fae5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50146/0 (socket says 172.21.15.100:50146) 2024-07-26T13:47:09.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.256+0000 7fe34e7fc640 1 -- 172.21.15.100:0/3426456411 learned_addr learned my addr 172.21.15.100:0/3426456411 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:09.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.256+0000 7fe34f7fe640 1 --2- 172.21.15.100:0/3426456411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f65f740070 0x55f65fae5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:09.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.256+0000 7fe34effd640 1 --2- 172.21.15.100:0/3426456411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f65f7171b0 0x55f65fae5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:09.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.256+0000 7fe34e7fc640 1 -- 172.21.15.100:0/3426456411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f65f7171b0 msgr2=0x55f65fae5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:09.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.256+0000 7fe34e7fc640 1 --2- 172.21.15.100:0/3426456411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f65f7171b0 0x55f65fae5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:09.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.256+0000 7fe34e7fc640 1 -- 172.21.15.100:0/3426456411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f65f740070 msgr2=0x55f65fae5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:09.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.256+0000 7fe34e7fc640 1 --2- 172.21.15.100:0/3426456411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f65f740070 0x55f65fae5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:09.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.256+0000 7fe34e7fc640 1 -- 172.21.15.100:0/3426456411 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f65f9f1b40 con 0x55f65f71f390 2024-07-26T13:47:09.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.257+0000 7fe35480b640 1 --2- 172.21.15.100:0/1514147566 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe3380b3d40 0x7fe3380b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:09.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.257+0000 7fe34e7fc640 1 --2- 172.21.15.100:0/3426456411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f65f71f390 0x55f65fae5fa0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fe34400abf0 tx=0x7fe34400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415ad3ccdea4283f server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:09.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.257+0000 7fe34f7fe640 1 --2- 172.21.15.100:0/1514147566 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe3380b3d40 0x7fe3380b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:09.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.257+0000 7fe35480b640 1 -- 172.21.15.100:0/1514147566 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8830+0+0 (secure 0 0 0) 0x7fe3400e45a0 con 0x7fe350102490 2024-07-26T13:47:09.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.257+0000 7fe335ffb640 1 -- 172.21.15.100:0/3426456411 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34401d070 con 0x55f65f71f390 2024-07-26T13:47:09.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.257+0000 7fe335ffb640 1 -- 172.21.15.100:0/3426456411 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe344018af0 con 0x55f65f71f390 2024-07-26T13:47:09.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.257+0000 7fe335ffb640 1 -- 172.21.15.100:0/3426456411 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe344010b40 con 0x55f65f71f390 2024-07-26T13:47:09.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.258+0000 7fe35bd91740 1 -- 172.21.15.100:0/3426456411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f65f71f390 msgr2=0x55f65fae5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:09.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.258+0000 7fe34f7fe640 1 --2- 172.21.15.100:0/1514147566 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe3380b3d40 0x7fe3380b6250 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe3480096c0 tx=0x7fe348005e60 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:09.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.258+0000 7fe35bd91740 1 --2- 172.21.15.100:0/3426456411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f65f71f390 0x55f65fae5fa0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fe34400abf0 tx=0x7fe34400f790 comp rx=0 tx=0).stop 2024-07-26T13:47:09.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.258+0000 7fe35bd91740 1 -- 172.21.15.100:0/3426456411 shutdown_connections 2024-07-26T13:47:09.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.258+0000 7fe35bd91740 1 --2- 172.21.15.100:0/3426456411 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f65f71f390 0x55f65fae5fa0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:09.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.258+0000 7fe35bd91740 1 --2- 172.21.15.100:0/3426456411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f65f7171b0 0x55f65fae5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:09.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.258+0000 7fe35bd91740 1 --2- 172.21.15.100:0/3426456411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f65f740070 0x55f65fae5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:09.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.258+0000 7fe35bd91740 1 -- 172.21.15.100:0/3426456411 >> 172.21.15.100:0/3426456411 conn(0x55f65f734460 msgr2=0x55f65fae3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:09.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.258+0000 7fe35bd91740 1 -- 172.21.15.100:0/3426456411 shutdown_connections 2024-07-26T13:47:09.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.258+0000 7fe35bd91740 1 -- 172.21.15.100:0/3426456411 wait complete. 2024-07-26T13:47:09.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.258+0000 7fe35bd91740 1 -- start start 2024-07-26T13:47:09.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.258+0000 7fe35bd91740 1 Processor -- start 2024-07-26T13:47:09.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.259+0000 7fe35bd91740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f65f7171b0 0x55f65faf10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:09.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.259+0000 7fe35bd91740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f65f71f390 0x55f65faf15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:09.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.259+0000 7fe35bd91740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f65f740070 0x55f65fafcff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:09.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.259+0000 7fe35bd91740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55f65f8ae700 con 0x55f65f71f390 2024-07-26T13:47:09.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.259+0000 7fe35bd91740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f65f9ed990 con 0x55f65f740070 2024-07-26T13:47:09.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.259+0000 7fe35bd91740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f65faf3d00 con 0x55f65f7171b0 2024-07-26T13:47:09.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.259+0000 7fe35bd91740 10 client.0 _ll_register_callbacks cb 0x55f65fa084a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:47:09.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.259+0000 7fe34e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f65f71f390 0x55f65faf15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:09.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.259+0000 7fe34e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f65f71f390 0x55f65faf15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50152/0 (socket says 172.21.15.100:50152) 2024-07-26T13:47:09.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.259+0000 7fe34e7fc640 1 -- 172.21.15.100:0/186428135 learned_addr learned my addr 172.21.15.100:0/186428135 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:09.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.259+0000 7fe34effd640 1 --2- 172.21.15.100:0/186428135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f65f7171b0 0x55f65faf10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:09.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.259+0000 7fe34f7fe640 1 --2- 172.21.15.100:0/186428135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f65f740070 0x55f65fafcff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:09.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.259+0000 7fe34e7fc640 1 -- 172.21.15.100:0/186428135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f65f7171b0 msgr2=0x55f65faf10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:09.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.259+0000 7fe34e7fc640 1 --2- 172.21.15.100:0/186428135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f65f7171b0 0x55f65faf10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:09.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.259+0000 7fe34e7fc640 1 -- 172.21.15.100:0/186428135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f65f740070 msgr2=0x55f65fafcff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:09.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.259+0000 7fe34e7fc640 1 --2- 172.21.15.100:0/186428135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f65f740070 0x55f65fafcff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:09.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.259+0000 7fe34e7fc640 1 -- 172.21.15.100:0/186428135 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f65f7ab650 con 0x55f65f71f390 2024-07-26T13:47:09.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.259+0000 7fe34f7fe640 1 --2- 172.21.15.100:0/186428135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f65f740070 0x55f65fafcff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:47:09.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.259+0000 7fe34e7fc640 1 --2- 172.21.15.100:0/186428135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f65f71f390 0x55f65faf15e0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fe344010720 tx=0x7fe34400a910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8279b09b7c8ef0d4 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:09.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.259+0000 7fe32effd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:47:09.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.259+0000 7fe32effd640 1 -- 172.21.15.100:0/186428135 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe344027020 con 0x55f65f71f390 2024-07-26T13:47:09.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.260+0000 7fe32effd640 1 -- 172.21.15.100:0/186428135 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8830+0+0 (secure 0 0 0) 0x7fe344018740 con 0x55f65f71f390 2024-07-26T13:47:09.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.260+0000 7fe35bd91740 1 -- 172.21.15.100:0/186428135 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe344018af0 con 0x55f65f71f390 2024-07-26T13:47:09.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.260+0000 7fe32effd640 1 -- 172.21.15.100:0/186428135 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fe344032050 con 0x55f65f71f390 2024-07-26T13:47:09.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.260+0000 7fe35bd91740 10 client.17307 fetch_fsmap learned FSMap version 54 2024-07-26T13:47:09.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.260+0000 7fe35bd91740 1 -- 172.21.15.100:0/186428135 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55f65faffce0 con 0x55f65f71f390 2024-07-26T13:47:09.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.260+0000 7fe32effd640 1 -- 172.21.15.100:0/186428135 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe344018af0 con 0x55f65f71f390 2024-07-26T13:47:09.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.260+0000 7fe32effd640 1 -- 172.21.15.100:0/186428135 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 54) ==== 8350+0+0 (secure 0 0 0) 0x7fe3440046c0 con 0x55f65f71f390 2024-07-26T13:47:09.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.260+0000 7fe35bd91740 10 client.17307 fetch_fsmap finished waiting for FSMap version 54 2024-07-26T13:47:09.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.260+0000 7fe35bd91740 10 client.17307 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:47:09.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.260+0000 7fe35bd91740 20 client.17307 populate_metadata read hostname 'smithi100' 2024-07-26T13:47:09.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.260+0000 7fe35bd91740 1 --2- 172.21.15.100:0/186428135 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f65f6fe9f0 0x55f65fb030f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:09.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.260+0000 7fe35bd91740 4 client.17307 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:47:09.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.260+0000 7fe35bd91740 1 -- 172.21.15.100:0/186428135 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f65fa5dab0 con 0x55f65f6fe9f0 2024-07-26T13:47:09.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.261+0000 7fe34effd640 1 --2- 172.21.15.100:0/186428135 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f65f6fe9f0 0x55f65fb030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:09.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.261+0000 7fe34effd640 1 --2- 172.21.15.100:0/186428135 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f65f6fe9f0 0x55f65fb030f0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=388f3462a1348aef server_cookie=170df0a94c2ea6cc in_seq=0 out_seq=0 2024-07-26T13:47:09.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.261+0000 7fe32effd640 10 client.17307 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:09.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.262+0000 7fe32effd640 1 -- 172.21.15.100:0/186428135 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f65fa5dab0 con 0x55f65f6fe9f0 2024-07-26T13:47:09.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.262+0000 7fe32effd640 10 client.17307 handle_command_reply: tid=0 2024-07-26T13:47:09.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.262+0000 7fe35bd91740 1 client.17307 shutdown 2024-07-26T13:47:09.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.262+0000 7fe35bd91740 1 -- 172.21.15.100:0/186428135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f65f71f390 msgr2=0x55f65faf15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:09.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.262+0000 7fe35bd91740 1 --2- 172.21.15.100:0/186428135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f65f71f390 0x55f65faf15e0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fe344010720 tx=0x7fe34400a910 comp rx=0 tx=0).stop 2024-07-26T13:47:09.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.262+0000 7fe35bd91740 1 -- 172.21.15.100:0/186428135 shutdown_connections 2024-07-26T13:47:09.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.262+0000 7fe35bd91740 1 --2- 172.21.15.100:0/186428135 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f65f6fe9f0 0x55f65fb030f0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:09.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.262+0000 7fe35bd91740 1 --2- 172.21.15.100:0/186428135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f65f740070 0x55f65fafcff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:09.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.262+0000 7fe35bd91740 1 --2- 172.21.15.100:0/186428135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f65f71f390 0x55f65faf15e0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:09.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.262+0000 7fe32effd640 0 client.17307 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:09.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.262+0000 7fe35bd91740 1 --2- 172.21.15.100:0/186428135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f65f7171b0 0x55f65faf10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:09.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.262+0000 7fe35bd91740 1 -- 172.21.15.100:0/186428135 >> 172.21.15.100:0/186428135 conn(0x55f65f734460 msgr2=0x55f65fae1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:09.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.262+0000 7fe35bd91740 1 -- 172.21.15.100:0/186428135 shutdown_connections 2024-07-26T13:47:09.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.262+0000 7fe35bd91740 1 -- 172.21.15.100:0/186428135 wait complete. 2024-07-26T13:47:09.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.262+0000 7fe35bd91740 20 client.17307 trim_cache size 0 max 16384 2024-07-26T13:47:09.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.279+0000 7fe35bd91740 1 Processor -- start 2024-07-26T13:47:09.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.279+0000 7fe35bd91740 1 -- start start 2024-07-26T13:47:09.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.279+0000 7fe35bd91740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f65f7171b0 0x55f65faf2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:09.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.280+0000 7fe35bd91740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f65f71f390 0x55f65fae6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:09.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.280+0000 7fe35bd91740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f65f740070 0x55f65fafcf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:09.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.280+0000 7fe35bd91740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe3501b1d60 con 0x55f65f740070 2024-07-26T13:47:09.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.280+0000 7fe35bd91740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe3501af600 con 0x55f65f71f390 2024-07-26T13:47:09.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.280+0000 7fe35bd91740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe3501ad470 con 0x55f65f7171b0 2024-07-26T13:47:09.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.280+0000 7fe34f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f65f740070 0x55f65fafcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:09.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.280+0000 7fe34f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f65f740070 0x55f65fafcf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50162/0 (socket says 172.21.15.100:50162) 2024-07-26T13:47:09.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.280+0000 7fe34f7fe640 1 -- 172.21.15.100:0/981058426 learned_addr learned my addr 172.21.15.100:0/981058426 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:09.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.280+0000 7fe34effd640 1 --2- 172.21.15.100:0/981058426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f65f7171b0 0x55f65faf2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:09.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.280+0000 7fe34e7fc640 1 --2- 172.21.15.100:0/981058426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f65f71f390 0x55f65fae6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:09.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.280+0000 7fe34f7fe640 1 -- 172.21.15.100:0/981058426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f65f7171b0 msgr2=0x55f65faf2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:09.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.280+0000 7fe34f7fe640 1 --2- 172.21.15.100:0/981058426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f65f7171b0 0x55f65faf2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:09.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.280+0000 7fe34f7fe640 1 -- 172.21.15.100:0/981058426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f65f71f390 msgr2=0x55f65fae6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:09.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.280+0000 7fe34f7fe640 1 --2- 172.21.15.100:0/981058426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f65f71f390 0x55f65fae6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:09.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.280+0000 7fe34f7fe640 1 -- 172.21.15.100:0/981058426 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f65faef3a0 con 0x55f65f740070 2024-07-26T13:47:09.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.280+0000 7fe34e7fc640 1 --2- 172.21.15.100:0/981058426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f65f71f390 0x55f65fae6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:47:09.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.280+0000 7fe34f7fe640 1 --2- 172.21.15.100:0/981058426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f65f740070 0x55f65fafcf60 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fe348033000 tx=0x7fe34803d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c0355a2edcae128 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:09.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.280+0000 7fe3357fa640 1 -- 172.21.15.100:0/981058426 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34804c070 con 0x55f65f740070 2024-07-26T13:47:09.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.280+0000 7fe3357fa640 1 -- 172.21.15.100:0/981058426 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe348004300 con 0x55f65f740070 2024-07-26T13:47:09.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.280+0000 7fe3357fa640 1 -- 172.21.15.100:0/981058426 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe348035050 con 0x55f65f740070 2024-07-26T13:47:09.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.281+0000 7fe35bd91740 1 -- 172.21.15.100:0/981058426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f65f740070 msgr2=0x55f65fafcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:09.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.281+0000 7fe35bd91740 1 --2- 172.21.15.100:0/981058426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f65f740070 0x55f65fafcf60 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fe348033000 tx=0x7fe34803d000 comp rx=0 tx=0).stop 2024-07-26T13:47:09.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.281+0000 7fe35bd91740 1 -- 172.21.15.100:0/981058426 shutdown_connections 2024-07-26T13:47:09.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.281+0000 7fe35bd91740 1 --2- 172.21.15.100:0/981058426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f65f740070 0x55f65fafcf60 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:09.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.281+0000 7fe35bd91740 1 --2- 172.21.15.100:0/981058426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f65f71f390 0x55f65fae6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:09.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.281+0000 7fe35bd91740 1 --2- 172.21.15.100:0/981058426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f65f7171b0 0x55f65faf2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:09.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.281+0000 7fe35bd91740 1 -- 172.21.15.100:0/981058426 >> 172.21.15.100:0/981058426 conn(0x55f65f734460 msgr2=0x55f65fae6550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:09.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.281+0000 7fe35bd91740 1 -- 172.21.15.100:0/981058426 shutdown_connections 2024-07-26T13:47:09.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.281+0000 7fe35bd91740 1 -- 172.21.15.100:0/981058426 wait complete. 2024-07-26T13:47:09.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.282+0000 7fe35bd91740 1 -- start start 2024-07-26T13:47:09.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.282+0000 7fe35bd91740 1 Processor -- start 2024-07-26T13:47:09.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.282+0000 7fe35bd91740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f65f7171b0 0x55f65fb04330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:09.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.282+0000 7fe35bd91740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f65f71f390 0x55f65fb04870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:09.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.282+0000 7fe35bd91740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f65f740070 0x55f65fb6a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:09.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.282+0000 7fe35bd91740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe3501adac0 con 0x55f65f740070 2024-07-26T13:47:09.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.282+0000 7fe35bd91740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe3501aef30 con 0x55f65f71f390 2024-07-26T13:47:09.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.282+0000 7fe35bd91740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe3501ae190 con 0x55f65f7171b0 2024-07-26T13:47:09.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.282+0000 7fe35bd91740 10 client.0 _ll_register_callbacks cb 0x7fe34401b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:47:09.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.282+0000 7fe34f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f65f740070 0x55f65fb6a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:09.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.282+0000 7fe34f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f65f740070 0x55f65fb6a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50164/0 (socket says 172.21.15.100:50164) 2024-07-26T13:47:09.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.282+0000 7fe34f7fe640 1 -- 172.21.15.100:0/1199737961 learned_addr learned my addr 172.21.15.100:0/1199737961 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:09.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.282+0000 7fe34e7fc640 1 --2- 172.21.15.100:0/1199737961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f65f71f390 0x55f65fb04870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:09.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.283+0000 7fe34effd640 1 --2- 172.21.15.100:0/1199737961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f65f7171b0 0x55f65fb04330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:09.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.283+0000 7fe34f7fe640 1 -- 172.21.15.100:0/1199737961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f65f7171b0 msgr2=0x55f65fb04330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:09.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.283+0000 7fe34f7fe640 1 --2- 172.21.15.100:0/1199737961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f65f7171b0 0x55f65fb04330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:09.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.283+0000 7fe34f7fe640 1 -- 172.21.15.100:0/1199737961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f65f71f390 msgr2=0x55f65fb04870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:09.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.283+0000 7fe34f7fe640 1 --2- 172.21.15.100:0/1199737961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f65f71f390 0x55f65fb04870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:09.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.283+0000 7fe34f7fe640 1 -- 172.21.15.100:0/1199737961 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f65fb05140 con 0x55f65f740070 2024-07-26T13:47:09.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.283+0000 7fe34f7fe640 1 --2- 172.21.15.100:0/1199737961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f65f740070 0x55f65fb6a640 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fe34802a930 tx=0x7fe348003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d115bcb68ff6372 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:09.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.283+0000 7fe32effd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:47:09.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.283+0000 7fe32effd640 1 -- 172.21.15.100:0/1199737961 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34804c030 con 0x55f65f740070 2024-07-26T13:47:09.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.283+0000 7fe32effd640 1 -- 172.21.15.100:0/1199737961 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8830+0+0 (secure 0 0 0) 0x7fe348051660 con 0x55f65f740070 2024-07-26T13:47:09.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.283+0000 7fe35bd91740 1 -- 172.21.15.100:0/1199737961 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe348004300 con 0x55f65f740070 2024-07-26T13:47:09.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.283+0000 7fe32effd640 1 -- 172.21.15.100:0/1199737961 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fe348063050 con 0x55f65f740070 2024-07-26T13:47:09.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.283+0000 7fe35bd91740 10 client.17313 fetch_fsmap learned FSMap version 54 2024-07-26T13:47:09.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.283+0000 7fe35bd91740 1 -- 172.21.15.100:0/1199737961 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55f65fb6bb60 con 0x55f65f740070 2024-07-26T13:47:09.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.284+0000 7fe32effd640 1 -- 172.21.15.100:0/1199737961 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe348004300 con 0x55f65f740070 2024-07-26T13:47:09.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.284+0000 7fe32effd640 1 -- 172.21.15.100:0/1199737961 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 54) ==== 8350+0+0 (secure 0 0 0) 0x7fe348048830 con 0x55f65f740070 2024-07-26T13:47:09.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.284+0000 7fe35bd91740 10 client.17313 fetch_fsmap finished waiting for FSMap version 54 2024-07-26T13:47:09.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.284+0000 7fe35bd91740 10 client.17313 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:47:09.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.284+0000 7fe35bd91740 20 client.17313 populate_metadata read hostname 'smithi100' 2024-07-26T13:47:09.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.284+0000 7fe35bd91740 1 --2- 172.21.15.100:0/1199737961 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f65f6d3da0 0x55f65fb6e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:09.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.284+0000 7fe35bd91740 4 client.17313 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-26T13:47:09.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.284+0000 7fe35bd91740 1 -- 172.21.15.100:0/1199737961 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f65fae38b0 con 0x55f65f6d3da0 2024-07-26T13:47:09.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.284+0000 7fe34effd640 1 --2- 172.21.15.100:0/1199737961 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f65f6d3da0 0x55f65fb6e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:09.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.285+0000 7fe34effd640 1 --2- 172.21.15.100:0/1199737961 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f65f6d3da0 0x55f65fb6e260 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1170be1e3be0e3dc server_cookie=c374c1a33ed9bb18 in_seq=0 out_seq=0 2024-07-26T13:47:09.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.285+0000 7fe32effd640 10 client.17313 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:09.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.285+0000 7fe32effd640 1 -- 172.21.15.100:0/1199737961 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x55f65fae38b0 con 0x55f65f6d3da0 2024-07-26T13:47:09.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.285+0000 7fe32effd640 10 client.17313 handle_command_reply: tid=0 2024-07-26T13:47:09.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.285+0000 7fe35bd91740 1 client.17313 shutdown 2024-07-26T13:47:09.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.285+0000 7fe35bd91740 1 -- 172.21.15.100:0/1199737961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f65f740070 msgr2=0x55f65fb6a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:09.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.285+0000 7fe35bd91740 1 --2- 172.21.15.100:0/1199737961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f65f740070 0x55f65fb6a640 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fe34802a930 tx=0x7fe348003dc0 comp rx=0 tx=0).stop 2024-07-26T13:47:09.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.285+0000 7fe35bd91740 1 -- 172.21.15.100:0/1199737961 shutdown_connections 2024-07-26T13:47:09.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.285+0000 7fe35bd91740 1 --2- 172.21.15.100:0/1199737961 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f65f6d3da0 0x55f65fb6e260 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:09.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.285+0000 7fe35bd91740 1 --2- 172.21.15.100:0/1199737961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f65f740070 0x55f65fb6a640 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:09.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.285+0000 7fe35bd91740 1 --2- 172.21.15.100:0/1199737961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f65f71f390 0x55f65fb04870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:09.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.285+0000 7fe35bd91740 1 --2- 172.21.15.100:0/1199737961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f65f7171b0 0x55f65fb04330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:09.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.285+0000 7fe35bd91740 1 -- 172.21.15.100:0/1199737961 >> 172.21.15.100:0/1199737961 conn(0x55f65f734460 msgr2=0x55f65faf2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:09.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.285+0000 7fe32effd640 0 client.17313 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:09.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.285+0000 7fe35bd91740 1 -- 172.21.15.100:0/1199737961 shutdown_connections 2024-07-26T13:47:09.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.285+0000 7fe35bd91740 1 -- 172.21.15.100:0/1199737961 wait complete. 2024-07-26T13:47:09.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.285+0000 7fe35bd91740 20 client.17313 trim_cache size 0 max 16384 2024-07-26T13:47:09.291 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:47:09.291 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "scrub active (1 inodes in the stack)", 2024-07-26T13:47:09.291 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": { 2024-07-26T13:47:09.291 INFO:teuthology.orchestra.run.smithi100.stdout: "f200fd50-e326-4bfc-976e-d04881835bdc": { 2024-07-26T13:47:09.291 INFO:teuthology.orchestra.run.smithi100.stdout: "path": "/", 2024-07-26T13:47:09.291 INFO:teuthology.orchestra.run.smithi100.stdout: "tag": "f200fd50-e326-4bfc-976e-d04881835bdc", 2024-07-26T13:47:09.291 INFO:teuthology.orchestra.run.smithi100.stdout: "options": "recursive,force" 2024-07-26T13:47:09.291 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:47:09.292 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-07-26T13:47:09.292 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:47:09.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.286+0000 7fe35580d640 1 -- 172.21.15.100:0/1514147566 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe3380b3d40 msgr2=0x7fe3380b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:09.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.286+0000 7fe35580d640 1 --2- 172.21.15.100:0/1514147566 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe3380b3d40 0x7fe3380b6250 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe3480096c0 tx=0x7fe348005e60 comp rx=0 tx=0).stop 2024-07-26T13:47:09.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.286+0000 7fe35580d640 1 -- 172.21.15.100:0/1514147566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe350102490 msgr2=0x7fe35019ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:09.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.286+0000 7fe35580d640 1 --2- 172.21.15.100:0/1514147566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe350102490 0x7fe35019ad90 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fe34000cce0 tx=0x7fe34000a620 comp rx=0 tx=0).stop 2024-07-26T13:47:09.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.286+0000 7fe35580d640 1 -- 172.21.15.100:0/1514147566 shutdown_connections 2024-07-26T13:47:09.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.286+0000 7fe35580d640 1 --2- 172.21.15.100:0/1514147566 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe3380b3d40 0x7fe3380b6250 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:09.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.286+0000 7fe35580d640 1 --2- 172.21.15.100:0/1514147566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe350108720 0x7fe35019b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:09.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.286+0000 7fe35580d640 1 --2- 172.21.15.100:0/1514147566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe350107dd0 0x7fe35019b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:09.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.286+0000 7fe35580d640 1 --2- 172.21.15.100:0/1514147566 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe350102490 0x7fe35019ad90 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:09.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.286+0000 7fe35580d640 1 -- 172.21.15.100:0/1514147566 >> 172.21.15.100:0/1514147566 conn(0x7fe350079160 msgr2=0x7fe350103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:09.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.286+0000 7fe35580d640 1 -- 172.21.15.100:0/1514147566 shutdown_connections 2024-07-26T13:47:09.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:09.286+0000 7fe35580d640 1 -- 172.21.15.100:0/1514147566 wait complete. 2024-07-26T13:47:09.423 INFO:tasks.workunit.client.0.smithi100.stdout: 1 292494 3.89 MB/sec execute 203 sec latency 4661.946 ms 2024-07-26T13:47:10.104 INFO:tasks.cephfs.filesystem:scrub status for tag:f200fd50-e326-4bfc-976e-d04881835bdc - {'path': '/', 'tag': 'f200fd50-e326-4bfc-976e-d04881835bdc', 'options': 'recursive,force'} 2024-07-26T13:47:10.423 INFO:tasks.workunit.client.0.smithi100.stdout: 1 292494 3.87 MB/sec execute 204 sec latency 5662.039 ms 2024-07-26T13:47:11.424 INFO:tasks.workunit.client.0.smithi100.stdout: 1 292494 3.85 MB/sec execute 205 sec latency 6662.102 ms 2024-07-26T13:47:11.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:11 smithi138 ceph-mon[36954]: pgmap v580: 97 pgs: 97 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 246 op/s 2024-07-26T13:47:11.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:11 smithi100 ceph-mon[32599]: pgmap v580: 97 pgs: 97 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 246 op/s 2024-07-26T13:47:12.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:11 smithi167 ceph-mon[37140]: pgmap v580: 97 pgs: 97 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 246 op/s 2024-07-26T13:47:12.424 INFO:tasks.workunit.client.0.smithi100.stdout: 1 292494 3.83 MB/sec execute 206 sec latency 7662.211 ms 2024-07-26T13:47:13.424 INFO:tasks.workunit.client.0.smithi100.stdout: 1 292494 3.81 MB/sec execute 207 sec latency 8662.298 ms 2024-07-26T13:47:13.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:13 smithi138 ceph-mon[36954]: pgmap v581: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 225 op/s 2024-07-26T13:47:13.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:13 smithi100 ceph-mon[32599]: pgmap v581: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 225 op/s 2024-07-26T13:47:14.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:13 smithi167 ceph-mon[37140]: pgmap v581: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 225 op/s 2024-07-26T13:47:14.424 INFO:tasks.workunit.client.0.smithi100.stdout: 1 292494 3.79 MB/sec execute 208 sec latency 9662.359 ms 2024-07-26T13:47:14.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.483+0000 7fd3c8108640 1 -- 172.21.15.100:0/595193661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3c0108680 msgr2=0x7fd3c010eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:14.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.483+0000 7fd3c8108640 1 --2- 172.21.15.100:0/595193661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3c0108680 0x7fd3c010eeb0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8009ca0 tx=0x7fd3b802a8c0 comp rx=0 tx=0).stop 2024-07-26T13:47:14.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.483+0000 7fd3c8108640 1 -- 172.21.15.100:0/595193661 shutdown_connections 2024-07-26T13:47:14.484 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.483+0000 7fd3c8108640 1 --2- 172.21.15.100:0/595193661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3c0108680 0x7fd3c010eeb0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:14.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.483+0000 7fd3c8108640 1 --2- 172.21.15.100:0/595193661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3c0107d30 0x7fd3c0108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:14.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.483+0000 7fd3c8108640 1 --2- 172.21.15.100:0/595193661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3c0102400 0x7fd3c0102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:14.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.483+0000 7fd3c8108640 1 -- 172.21.15.100:0/595193661 >> 172.21.15.100:0/595193661 conn(0x7fd3c007a120 msgr2=0x7fd3c007a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:14.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.483+0000 7fd3c8108640 1 -- 172.21.15.100:0/595193661 shutdown_connections 2024-07-26T13:47:14.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.483+0000 7fd3c8108640 1 -- 172.21.15.100:0/595193661 wait complete. 2024-07-26T13:47:14.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.483+0000 7fd3c8108640 1 Processor -- start 2024-07-26T13:47:14.485 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.484+0000 7fd3c8108640 1 -- start start 2024-07-26T13:47:14.490 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.484+0000 7fd3c8108640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3c0102400 0x7fd3c0198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.484+0000 7fd3c8108640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3c0107d30 0x7fd3c01992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.484+0000 7fd3c8108640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3c0108680 0x7fd3c019e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.484+0000 7fd3c8108640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c0112350 con 0x7fd3c0102400 2024-07-26T13:47:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.484+0000 7fd3c8108640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c01121d0 con 0x7fd3c0108680 2024-07-26T13:47:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.484+0000 7fd3c8108640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c01124d0 con 0x7fd3c0107d30 2024-07-26T13:47:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.484+0000 7fd3c5ea4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3c0102400 0x7fd3c0198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.484+0000 7fd3c5ea4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3c0102400 0x7fd3c0198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50184/0 (socket says 172.21.15.100:50184) 2024-07-26T13:47:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.484+0000 7fd3c5ea4640 1 -- 172.21.15.100:0/451658271 learned_addr learned my addr 172.21.15.100:0/451658271 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.484+0000 7fd3c56a3640 1 --2- 172.21.15.100:0/451658271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3c0107d30 0x7fd3c01992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.484+0000 7fd3c66a5640 1 --2- 172.21.15.100:0/451658271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3c0108680 0x7fd3c019e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.484+0000 7fd3c5ea4640 1 -- 172.21.15.100:0/451658271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3c0107d30 msgr2=0x7fd3c01992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.484+0000 7fd3c5ea4640 1 --2- 172.21.15.100:0/451658271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3c0107d30 0x7fd3c01992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:14.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.484+0000 7fd3c5ea4640 1 -- 172.21.15.100:0/451658271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3c0108680 msgr2=0x7fd3c019e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:14.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.484+0000 7fd3c5ea4640 1 --2- 172.21.15.100:0/451658271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3c0108680 0x7fd3c019e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:14.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.484+0000 7fd3c5ea4640 1 -- 172.21.15.100:0/451658271 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c019ec40 con 0x7fd3c0102400 2024-07-26T13:47:14.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.484+0000 7fd3c5ea4640 1 --2- 172.21.15.100:0/451658271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3c0102400 0x7fd3c0198d10 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc00e9f0 tx=0x7fd3bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:14.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.485+0000 7fd3b6ffd640 1 -- 172.21.15.100:0/451658271 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc00cd80 con 0x7fd3c0102400 2024-07-26T13:47:14.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.485+0000 7fd3b6ffd640 1 -- 172.21.15.100:0/451658271 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3bc0108e0 con 0x7fd3c0102400 2024-07-26T13:47:14.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.485+0000 7fd3c8108640 1 -- 172.21.15.100:0/451658271 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3c019ef30 con 0x7fd3c0102400 2024-07-26T13:47:14.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.485+0000 7fd3b6ffd640 1 -- 172.21.15.100:0/451658271 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc018980 con 0x7fd3c0102400 2024-07-26T13:47:14.492 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.485+0000 7fd3c8108640 1 -- 172.21.15.100:0/451658271 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3c01a56c0 con 0x7fd3c0102400 2024-07-26T13:47:14.498 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.487+0000 7fd3cf019740 1 Processor -- start 2024-07-26T13:47:14.498 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.487+0000 7fd3b6ffd640 1 -- 172.21.15.100:0/451658271 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fd3bc010430 con 0x7fd3c0102400 2024-07-26T13:47:14.498 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.487+0000 7fd3cf019740 1 -- start start 2024-07-26T13:47:14.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.488+0000 7fd3cf019740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56474891b1b0 0x564748cbdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:14.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.488+0000 7fd3cf019740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647488f22f0 0x564748cbe220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:14.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.488+0000 7fd3cf019740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647488fa4d0 0x564748cbe760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:14.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.488+0000 7fd3cf019740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x564748c333b0 con 0x5647488f22f0 2024-07-26T13:47:14.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.488+0000 7fd3cf019740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564748c228b0 con 0x56474891b1b0 2024-07-26T13:47:14.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.488+0000 7fd3cf019740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564748c268f0 con 0x5647488fa4d0 2024-07-26T13:47:14.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.488+0000 7fd3c5ea4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647488f22f0 0x564748cbe220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:14.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.488+0000 7fd3c5ea4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647488f22f0 0x564748cbe220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50198/0 (socket says 172.21.15.100:50198) 2024-07-26T13:47:14.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.488+0000 7fd3c5ea4640 1 -- 172.21.15.100:0/1185779460 learned_addr learned my addr 172.21.15.100:0/1185779460 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:14.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.488+0000 7fd3c56a3640 1 --2- 172.21.15.100:0/1185779460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647488fa4d0 0x564748cbe760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:14.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.488+0000 7fd3c66a5640 1 --2- 172.21.15.100:0/1185779460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56474891b1b0 0x564748cbdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:14.499 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.488+0000 7fd3c5ea4640 1 -- 172.21.15.100:0/1185779460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647488fa4d0 msgr2=0x564748cbe760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.488+0000 7fd3c5ea4640 1 --2- 172.21.15.100:0/1185779460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647488fa4d0 0x564748cbe760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.488+0000 7fd3c5ea4640 1 -- 172.21.15.100:0/1185779460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56474891b1b0 msgr2=0x564748cbdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.488+0000 7fd3c5ea4640 1 --2- 172.21.15.100:0/1185779460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56474891b1b0 0x564748cbdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.488+0000 7fd3c5ea4640 1 -- 172.21.15.100:0/1185779460 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564748bc99a0 con 0x5647488f22f0 2024-07-26T13:47:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.488+0000 7fd3c66a5640 1 --2- 172.21.15.100:0/1185779460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56474891b1b0 0x564748cbdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:47:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.488+0000 7fd3b6ffd640 1 --2- 172.21.15.100:0/451658271 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd3ac0b3dc0 0x7fd3ac0b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.488+0000 7fd3c66a5640 1 --2- 172.21.15.100:0/451658271 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd3ac0b3dc0 0x7fd3ac0b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.488+0000 7fd3c5ea4640 1 --2- 172.21.15.100:0/1185779460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647488f22f0 0x564748cbe220 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc0e1ae0 tx=0x7fd3bc0eab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab60b71607dd424 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.489+0000 7fd3b6ffd640 1 -- 172.21.15.100:0/451658271 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8786+0+0 (secure 0 0 0) 0x7fd3bc0e4500 con 0x7fd3c0102400 2024-07-26T13:47:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.489+0000 7fd3abfff640 1 -- 172.21.15.100:0/1185779460 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc026020 con 0x5647488f22f0 2024-07-26T13:47:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.489+0000 7fd3abfff640 1 -- 172.21.15.100:0/1185779460 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3bc010720 con 0x5647488f22f0 2024-07-26T13:47:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.489+0000 7fd3abfff640 1 -- 172.21.15.100:0/1185779460 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc0aa770 con 0x5647488f22f0 2024-07-26T13:47:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.490+0000 7fd3cf019740 1 -- 172.21.15.100:0/1185779460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647488f22f0 msgr2=0x564748cbe220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:14.500 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.490+0000 7fd3c66a5640 1 --2- 172.21.15.100:0/451658271 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd3ac0b3dc0 0x7fd3ac0b62d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8005cc0 tx=0x7fd3b80098e0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.490+0000 7fd3cf019740 1 --2- 172.21.15.100:0/1185779460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647488f22f0 0x564748cbe220 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc0e1ae0 tx=0x7fd3bc0eab80 comp rx=0 tx=0).stop 2024-07-26T13:47:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.490+0000 7fd3cf019740 1 -- 172.21.15.100:0/1185779460 shutdown_connections 2024-07-26T13:47:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.490+0000 7fd3cf019740 1 --2- 172.21.15.100:0/1185779460 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647488fa4d0 0x564748cbe760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.490+0000 7fd3cf019740 1 --2- 172.21.15.100:0/1185779460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647488f22f0 0x564748cbe220 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.490+0000 7fd3cf019740 1 --2- 172.21.15.100:0/1185779460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56474891b1b0 0x564748cbdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.490+0000 7fd3cf019740 1 -- 172.21.15.100:0/1185779460 >> 172.21.15.100:0/1185779460 conn(0x56474890f300 msgr2=0x564748cbbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.490+0000 7fd3cf019740 1 -- 172.21.15.100:0/1185779460 shutdown_connections 2024-07-26T13:47:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.490+0000 7fd3cf019740 1 -- 172.21.15.100:0/1185779460 wait complete. 2024-07-26T13:47:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.490+0000 7fd3cf019740 1 -- start start 2024-07-26T13:47:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.490+0000 7fd3cf019740 1 Processor -- start 2024-07-26T13:47:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.491+0000 7fd3cf019740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647488f22f0 0x564748cc97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.491+0000 7fd3cf019740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647488fa4d0 0x564748cc9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:14.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.491+0000 7fd3cf019740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56474891b1b0 0x564748cd57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:14.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.491+0000 7fd3cf019740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x564748b5cab0 con 0x5647488f22f0 2024-07-26T13:47:14.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.491+0000 7fd3cf019740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564748baacf0 con 0x56474891b1b0 2024-07-26T13:47:14.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.491+0000 7fd3cf019740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564748ccc460 con 0x5647488fa4d0 2024-07-26T13:47:14.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.491+0000 7fd3cf019740 10 client.0 _ll_register_callbacks cb 0x564748c1c9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:47:14.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.491+0000 7fd3c5ea4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647488f22f0 0x564748cc97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:14.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.491+0000 7fd3c5ea4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647488f22f0 0x564748cc97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50214/0 (socket says 172.21.15.100:50214) 2024-07-26T13:47:14.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.491+0000 7fd3c5ea4640 1 -- 172.21.15.100:0/364958826 learned_addr learned my addr 172.21.15.100:0/364958826 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:14.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.491+0000 7fd3c66a5640 1 --2- 172.21.15.100:0/364958826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56474891b1b0 0x564748cd57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:14.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.491+0000 7fd3c56a3640 1 --2- 172.21.15.100:0/364958826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647488fa4d0 0x564748cc9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:14.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.491+0000 7fd3c5ea4640 1 -- 172.21.15.100:0/364958826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647488fa4d0 msgr2=0x564748cc9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:14.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.491+0000 7fd3c5ea4640 1 --2- 172.21.15.100:0/364958826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647488fa4d0 0x564748cc9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:14.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.491+0000 7fd3c5ea4640 1 -- 172.21.15.100:0/364958826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56474891b1b0 msgr2=0x564748cd57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:14.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.491+0000 7fd3c5ea4640 1 --2- 172.21.15.100:0/364958826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56474891b1b0 0x564748cd57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:14.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.491+0000 7fd3c5ea4640 1 -- 172.21.15.100:0/364958826 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564748986790 con 0x5647488f22f0 2024-07-26T13:47:14.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.491+0000 7fd3c56a3640 1 --2- 172.21.15.100:0/364958826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647488fa4d0 0x564748cc9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:47:14.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.491+0000 7fd3c66a5640 1 --2- 172.21.15.100:0/364958826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56474891b1b0 0x564748cd57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:47:14.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.491+0000 7fd3c5ea4640 1 --2- 172.21.15.100:0/364958826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647488f22f0 0x564748cc97c0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc0e2140 tx=0x7fd3bc018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b80b488af975d00 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:14.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.491+0000 7fd3a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:47:14.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.491+0000 7fd3a97fa640 1 -- 172.21.15.100:0/364958826 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc0aa770 con 0x5647488f22f0 2024-07-26T13:47:14.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.491+0000 7fd3a97fa640 1 -- 172.21.15.100:0/364958826 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8786+0+0 (secure 0 0 0) 0x7fd3bc0ab260 con 0x5647488f22f0 2024-07-26T13:47:14.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.491+0000 7fd3cf019740 1 -- 172.21.15.100:0/364958826 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564748c35300 con 0x5647488f22f0 2024-07-26T13:47:14.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.492+0000 7fd3a97fa640 1 -- 172.21.15.100:0/364958826 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fd3bc0aadd0 con 0x5647488f22f0 2024-07-26T13:47:14.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.492+0000 7fd3cf019740 10 client.17325 fetch_fsmap learned FSMap version 54 2024-07-26T13:47:14.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.492+0000 7fd3cf019740 1 -- 172.21.15.100:0/364958826 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5647489ff620 con 0x5647488f22f0 2024-07-26T13:47:14.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.492+0000 7fd3a97fa640 1 -- 172.21.15.100:0/364958826 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564748c35300 con 0x5647488f22f0 2024-07-26T13:47:14.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.492+0000 7fd3a97fa640 1 -- 172.21.15.100:0/364958826 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 54) ==== 8350+0+0 (secure 0 0 0) 0x7fd3bc0cc0c0 con 0x5647488f22f0 2024-07-26T13:47:14.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.492+0000 7fd3cf019740 10 client.17325 fetch_fsmap finished waiting for FSMap version 54 2024-07-26T13:47:14.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.492+0000 7fd3cf019740 10 client.17325 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:47:14.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.492+0000 7fd3cf019740 20 client.17325 populate_metadata read hostname 'smithi100' 2024-07-26T13:47:14.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.492+0000 7fd3cf019740 1 --2- 172.21.15.100:0/364958826 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5647488d9d80 0x564748cdb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:14.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.492+0000 7fd3cf019740 4 client.17325 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:47:14.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.492+0000 7fd3cf019740 1 -- 172.21.15.100:0/364958826 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564748c30da0 con 0x5647488d9d80 2024-07-26T13:47:14.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.492+0000 7fd3c56a3640 1 --2- 172.21.15.100:0/364958826 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5647488d9d80 0x564748cdb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:14.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.493+0000 7fd3a97fa640 10 client.17325 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:14.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.493+0000 7fd3c56a3640 1 --2- 172.21.15.100:0/364958826 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5647488d9d80 0x564748cdb500 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f607e405281dec2 server_cookie=e2dd117d33ddbff9 in_seq=0 out_seq=0 2024-07-26T13:47:14.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.494+0000 7fd3a97fa640 1 -- 172.21.15.100:0/364958826 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564748c30da0 con 0x5647488d9d80 2024-07-26T13:47:14.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.494+0000 7fd3a97fa640 10 client.17325 handle_command_reply: tid=0 2024-07-26T13:47:14.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.494+0000 7fd3cf019740 1 client.17325 shutdown 2024-07-26T13:47:14.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.494+0000 7fd3cf019740 1 -- 172.21.15.100:0/364958826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647488f22f0 msgr2=0x564748cc97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:14.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.494+0000 7fd3cf019740 1 --2- 172.21.15.100:0/364958826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647488f22f0 0x564748cc97c0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc0e2140 tx=0x7fd3bc018420 comp rx=0 tx=0).stop 2024-07-26T13:47:14.505 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.494+0000 7fd3cf019740 1 -- 172.21.15.100:0/364958826 shutdown_connections 2024-07-26T13:47:14.505 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.494+0000 7fd3cf019740 1 --2- 172.21.15.100:0/364958826 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5647488d9d80 0x564748cdb500 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:14.505 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.494+0000 7fd3cf019740 1 --2- 172.21.15.100:0/364958826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56474891b1b0 0x564748cd57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:14.505 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.494+0000 7fd3cf019740 1 --2- 172.21.15.100:0/364958826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647488fa4d0 0x564748cc9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:14.505 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.494+0000 7fd3cf019740 1 --2- 172.21.15.100:0/364958826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647488f22f0 0x564748cc97c0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:14.505 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.494+0000 7fd3a97fa640 0 client.17325 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:14.505 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.494+0000 7fd3cf019740 1 -- 172.21.15.100:0/364958826 >> 172.21.15.100:0/364958826 conn(0x56474890f300 msgr2=0x564748cbed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:14.505 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.494+0000 7fd3cf019740 1 -- 172.21.15.100:0/364958826 shutdown_connections 2024-07-26T13:47:14.505 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.494+0000 7fd3cf019740 1 -- 172.21.15.100:0/364958826 wait complete. 2024-07-26T13:47:14.505 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.494+0000 7fd3cf019740 20 client.17325 trim_cache size 0 max 16384 2024-07-26T13:47:14.511 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.509+0000 7fd3cf019740 1 Processor -- start 2024-07-26T13:47:14.511 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.509+0000 7fd3cf019740 1 -- start start 2024-07-26T13:47:14.511 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.509+0000 7fd3cf019740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647488f22f0 0x564748cc3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:14.511 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.509+0000 7fd3cf019740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647488fa4d0 0x564748ccb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:14.511 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.509+0000 7fd3cf019740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56474891b1b0 0x564748cc8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:14.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.509+0000 7fd3cf019740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c01afb50 con 0x5647488fa4d0 2024-07-26T13:47:14.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.509+0000 7fd3cf019740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c01ad3f0 con 0x56474891b1b0 2024-07-26T13:47:14.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.509+0000 7fd3cf019740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c01ab260 con 0x5647488f22f0 2024-07-26T13:47:14.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.509+0000 7fd3c56a3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647488fa4d0 0x564748ccb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:14.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.509+0000 7fd3c56a3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647488fa4d0 0x564748ccb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50230/0 (socket says 172.21.15.100:50230) 2024-07-26T13:47:14.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.509+0000 7fd3c56a3640 1 -- 172.21.15.100:0/3236694164 learned_addr learned my addr 172.21.15.100:0/3236694164 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:14.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.510+0000 7fd3c56a3640 1 -- 172.21.15.100:0/3236694164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647488f22f0 msgr2=0x564748cc3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:14.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.510+0000 7fd3c66a5640 1 --2- 172.21.15.100:0/3236694164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56474891b1b0 0x564748cc8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:14.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.510+0000 7fd3c5ea4640 1 --2- 172.21.15.100:0/3236694164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647488f22f0 0x564748cc3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:14.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.510+0000 7fd3c56a3640 1 --2- 172.21.15.100:0/3236694164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647488f22f0 0x564748cc3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:14.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.510+0000 7fd3c56a3640 1 -- 172.21.15.100:0/3236694164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56474891b1b0 msgr2=0x564748cc8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:14.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.510+0000 7fd3c56a3640 1 --2- 172.21.15.100:0/3236694164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56474891b1b0 0x564748cc8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:14.512 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.510+0000 7fd3c56a3640 1 -- 172.21.15.100:0/3236694164 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564748ce16e0 con 0x5647488fa4d0 2024-07-26T13:47:14.513 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.510+0000 7fd3c66a5640 1 --2- 172.21.15.100:0/3236694164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56474891b1b0 0x564748cc8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:47:14.513 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.510+0000 7fd3c5ea4640 1 --2- 172.21.15.100:0/3236694164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647488f22f0 0x564748cc3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:47:14.513 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.510+0000 7fd3c56a3640 1 --2- 172.21.15.100:0/3236694164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647488fa4d0 0x564748ccb5a0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x564748c33410 tx=0x7fd3b0014840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=340ac6bd0ed0e250 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:14.513 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.510+0000 7fd3ab7fe640 1 -- 172.21.15.100:0/3236694164 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0004050 con 0x5647488fa4d0 2024-07-26T13:47:14.513 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.510+0000 7fd3ab7fe640 1 -- 172.21.15.100:0/3236694164 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3b00041f0 con 0x5647488fa4d0 2024-07-26T13:47:14.513 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.510+0000 7fd3ab7fe640 1 -- 172.21.15.100:0/3236694164 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b00126b0 con 0x5647488fa4d0 2024-07-26T13:47:14.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.511+0000 7fd3cf019740 1 -- 172.21.15.100:0/3236694164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647488fa4d0 msgr2=0x564748ccb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:14.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.511+0000 7fd3cf019740 1 --2- 172.21.15.100:0/3236694164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647488fa4d0 0x564748ccb5a0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x564748c33410 tx=0x7fd3b0014840 comp rx=0 tx=0).stop 2024-07-26T13:47:14.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.511+0000 7fd3cf019740 1 -- 172.21.15.100:0/3236694164 shutdown_connections 2024-07-26T13:47:14.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.511+0000 7fd3cf019740 1 --2- 172.21.15.100:0/3236694164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56474891b1b0 0x564748cc8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:14.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.511+0000 7fd3cf019740 1 --2- 172.21.15.100:0/3236694164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647488fa4d0 0x564748ccb5a0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:14.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.511+0000 7fd3cf019740 1 --2- 172.21.15.100:0/3236694164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647488f22f0 0x564748cc3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:14.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.511+0000 7fd3cf019740 1 -- 172.21.15.100:0/3236694164 >> 172.21.15.100:0/3236694164 conn(0x56474890f300 msgr2=0x564748cc3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:14.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.511+0000 7fd3cf019740 1 -- 172.21.15.100:0/3236694164 shutdown_connections 2024-07-26T13:47:14.517 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.511+0000 7fd3cf019740 1 -- 172.21.15.100:0/3236694164 wait complete. 2024-07-26T13:47:14.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.511+0000 7fd3cf019740 1 -- start start 2024-07-26T13:47:14.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.511+0000 7fd3cf019740 1 Processor -- start 2024-07-26T13:47:14.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.512+0000 7fd3cf019740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647488f22f0 0x564748d3a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:14.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.512+0000 7fd3cf019740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647488fa4d0 0x564748d3ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:14.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.512+0000 7fd3cf019740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56474891b1b0 0x564748d3fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:14.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.512+0000 7fd3cf019740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c01ab8b0 con 0x5647488f22f0 2024-07-26T13:47:14.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.512+0000 7fd3cf019740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c01acd20 con 0x56474891b1b0 2024-07-26T13:47:14.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.512+0000 7fd3cf019740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c01abf80 con 0x5647488fa4d0 2024-07-26T13:47:14.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.512+0000 7fd3cf019740 10 client.0 _ll_register_callbacks cb 0x7fd3bc0ea570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:47:14.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.512+0000 7fd3c5ea4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647488f22f0 0x564748d3a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:14.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.512+0000 7fd3c5ea4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647488f22f0 0x564748d3a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50234/0 (socket says 172.21.15.100:50234) 2024-07-26T13:47:14.518 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.512+0000 7fd3c5ea4640 1 -- 172.21.15.100:0/3465141883 learned_addr learned my addr 172.21.15.100:0/3465141883 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:14.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.512+0000 7fd3c66a5640 1 --2- 172.21.15.100:0/3465141883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56474891b1b0 0x564748d3fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:14.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.512+0000 7fd3c56a3640 1 --2- 172.21.15.100:0/3465141883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647488fa4d0 0x564748d3ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:14.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.512+0000 7fd3c5ea4640 1 -- 172.21.15.100:0/3465141883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647488fa4d0 msgr2=0x564748d3ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:14.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.512+0000 7fd3c5ea4640 1 --2- 172.21.15.100:0/3465141883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647488fa4d0 0x564748d3ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:14.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.512+0000 7fd3c5ea4640 1 -- 172.21.15.100:0/3465141883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56474891b1b0 msgr2=0x564748d3fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:14.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.512+0000 7fd3c5ea4640 1 --2- 172.21.15.100:0/3465141883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56474891b1b0 0x564748d3fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:14.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.512+0000 7fd3c5ea4640 1 -- 172.21.15.100:0/3465141883 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564748d40540 con 0x5647488f22f0 2024-07-26T13:47:14.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.512+0000 7fd3c5ea4640 1 --2- 172.21.15.100:0/3465141883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647488f22f0 0x564748d3a8b0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc0ea640 tx=0x7fd3bc0cc370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48122fc5a3b5301d server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:14.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.512+0000 7fd3a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:47:14.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.513+0000 7fd3a97fa640 1 -- 172.21.15.100:0/3465141883 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc0a6ec0 con 0x5647488f22f0 2024-07-26T13:47:14.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.513+0000 7fd3a97fa640 1 -- 172.21.15.100:0/3465141883 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8786+0+0 (secure 0 0 0) 0x7fd3bc0b3e30 con 0x5647488f22f0 2024-07-26T13:47:14.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.513+0000 7fd3cf019740 1 -- 172.21.15.100:0/3465141883 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3b00041f0 con 0x5647488f22f0 2024-07-26T13:47:14.519 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.513+0000 7fd3a97fa640 1 -- 172.21.15.100:0/3465141883 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fd3bc0aa770 con 0x5647488f22f0 2024-07-26T13:47:14.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.513+0000 7fd3cf019740 10 client.17331 fetch_fsmap learned FSMap version 54 2024-07-26T13:47:14.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.513+0000 7fd3cf019740 1 -- 172.21.15.100:0/3465141883 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x564748d47810 con 0x5647488f22f0 2024-07-26T13:47:14.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.513+0000 7fd3a97fa640 1 -- 172.21.15.100:0/3465141883 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3b00041f0 con 0x5647488f22f0 2024-07-26T13:47:14.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.513+0000 7fd3a97fa640 1 -- 172.21.15.100:0/3465141883 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 54) ==== 8350+0+0 (secure 0 0 0) 0x7fd3bc0aa260 con 0x5647488f22f0 2024-07-26T13:47:14.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.513+0000 7fd3cf019740 10 client.17331 fetch_fsmap finished waiting for FSMap version 54 2024-07-26T13:47:14.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.513+0000 7fd3cf019740 10 client.17331 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:47:14.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.513+0000 7fd3cf019740 20 client.17331 populate_metadata read hostname 'smithi100' 2024-07-26T13:47:14.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.513+0000 7fd3cf019740 1 --2- 172.21.15.100:0/3465141883 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5647488af1c0 0x564748d49e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:14.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.513+0000 7fd3cf019740 4 client.17331 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-26T13:47:14.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.513+0000 7fd3cf019740 1 -- 172.21.15.100:0/3465141883 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564748c2e4e0 con 0x5647488af1c0 2024-07-26T13:47:14.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.513+0000 7fd3c56a3640 1 --2- 172.21.15.100:0/3465141883 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5647488af1c0 0x564748d49e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:14.520 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.514+0000 7fd3a97fa640 10 client.17331 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:14.521 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.514+0000 7fd3c56a3640 1 --2- 172.21.15.100:0/3465141883 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5647488af1c0 0x564748d49e10 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69389684df63459f server_cookie=74dc5d270b5fa263 in_seq=0 out_seq=0 2024-07-26T13:47:14.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.552+0000 7fd3a97fa640 1 -- 172.21.15.100:0/3465141883 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564748c2e4e0 con 0x5647488af1c0 2024-07-26T13:47:14.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.552+0000 7fd3a97fa640 10 client.17331 handle_command_reply: tid=0 2024-07-26T13:47:14.553 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.552+0000 7fd3cf019740 1 client.17331 shutdown 2024-07-26T13:47:14.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.552+0000 7fd3cf019740 1 -- 172.21.15.100:0/3465141883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647488f22f0 msgr2=0x564748d3a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:14.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.552+0000 7fd3cf019740 1 --2- 172.21.15.100:0/3465141883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647488f22f0 0x564748d3a8b0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc0ea640 tx=0x7fd3bc0cc370 comp rx=0 tx=0).stop 2024-07-26T13:47:14.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.552+0000 7fd3cf019740 1 -- 172.21.15.100:0/3465141883 shutdown_connections 2024-07-26T13:47:14.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.552+0000 7fd3cf019740 1 --2- 172.21.15.100:0/3465141883 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5647488af1c0 0x564748d49e10 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:14.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.552+0000 7fd3cf019740 1 --2- 172.21.15.100:0/3465141883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56474891b1b0 0x564748d3fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:14.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.552+0000 7fd3a97fa640 0 client.17331 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:14.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.552+0000 7fd3cf019740 1 --2- 172.21.15.100:0/3465141883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647488fa4d0 0x564748d3ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:14.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.552+0000 7fd3cf019740 1 --2- 172.21.15.100:0/3465141883 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5647488f22f0 0x564748d3a8b0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:14.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.552+0000 7fd3cf019740 1 -- 172.21.15.100:0/3465141883 >> 172.21.15.100:0/3465141883 conn(0x56474890f300 msgr2=0x564748cc2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:14.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.552+0000 7fd3cf019740 1 -- 172.21.15.100:0/3465141883 shutdown_connections 2024-07-26T13:47:14.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.552+0000 7fd3cf019740 1 -- 172.21.15.100:0/3465141883 wait complete. 2024-07-26T13:47:14.554 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.552+0000 7fd3cf019740 20 client.17331 trim_cache size 0 max 16384 2024-07-26T13:47:14.555 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-07-26T13:47:14.555 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-07-26T13:47:14.555 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-07-26T13:47:14.555 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-07-26T13:47:14.556 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.553+0000 7fd3c8108640 1 -- 172.21.15.100:0/451658271 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd3ac0b3dc0 msgr2=0x7fd3ac0b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:14.556 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.553+0000 7fd3c8108640 1 --2- 172.21.15.100:0/451658271 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd3ac0b3dc0 0x7fd3ac0b62d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8005cc0 tx=0x7fd3b80098e0 comp rx=0 tx=0).stop 2024-07-26T13:47:14.556 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.553+0000 7fd3c8108640 1 -- 172.21.15.100:0/451658271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3c0102400 msgr2=0x7fd3c0198d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:14.556 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.553+0000 7fd3c8108640 1 --2- 172.21.15.100:0/451658271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3c0102400 0x7fd3c0198d10 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc00e9f0 tx=0x7fd3bc00eeb0 comp rx=0 tx=0).stop 2024-07-26T13:47:14.556 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.553+0000 7fd3c8108640 1 -- 172.21.15.100:0/451658271 shutdown_connections 2024-07-26T13:47:14.556 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.553+0000 7fd3c8108640 1 --2- 172.21.15.100:0/451658271 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fd3ac0b3dc0 0x7fd3ac0b62d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:14.556 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.553+0000 7fd3c8108640 1 --2- 172.21.15.100:0/451658271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3c0108680 0x7fd3c019e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:14.556 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.553+0000 7fd3c8108640 1 --2- 172.21.15.100:0/451658271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3c0107d30 0x7fd3c01992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:14.556 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.553+0000 7fd3c8108640 1 --2- 172.21.15.100:0/451658271 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd3c0102400 0x7fd3c0198d10 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:14.556 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.553+0000 7fd3c8108640 1 -- 172.21.15.100:0/451658271 >> 172.21.15.100:0/451658271 conn(0x7fd3c007a120 msgr2=0x7fd3c0105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:14.556 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.553+0000 7fd3c8108640 1 -- 172.21.15.100:0/451658271 shutdown_connections 2024-07-26T13:47:14.557 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:14.553+0000 7fd3c8108640 1 -- 172.21.15.100:0/451658271 wait complete. 2024-07-26T13:47:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:14 smithi138 ceph-mon[36954]: osdmap e86: 12 total, 12 up, 12 in 2024-07-26T13:47:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:14 smithi138 ceph-mon[36954]: pgmap v583: 97 pgs: 97 active+clean; 116 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 1.1 MiB/s wr, 85 op/s 2024-07-26T13:47:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:14 smithi100 ceph-mon[32599]: osdmap e86: 12 total, 12 up, 12 in 2024-07-26T13:47:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:14 smithi100 ceph-mon[32599]: pgmap v583: 97 pgs: 97 active+clean; 116 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 1.1 MiB/s wr, 85 op/s 2024-07-26T13:47:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:14 smithi167 ceph-mon[37140]: osdmap e86: 12 total, 12 up, 12 in 2024-07-26T13:47:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:14 smithi167 ceph-mon[37140]: pgmap v583: 97 pgs: 97 active+clean; 116 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 1.1 MiB/s wr, 85 op/s 2024-07-26T13:47:15.424 INFO:tasks.workunit.client.0.smithi100.stdout: 1 292494 3.78 MB/sec execute 209 sec latency 10662.420 ms 2024-07-26T13:47:16.424 INFO:tasks.workunit.client.0.smithi100.stdout: 1 292494 3.76 MB/sec execute 210 sec latency 11662.494 ms 2024-07-26T13:47:17.424 INFO:tasks.workunit.client.0.smithi100.stdout: 1 292567 3.74 MB/sec execute 211 sec latency 12545.111 ms 2024-07-26T13:47:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-mon[36954]: pgmap v584: 97 pgs: 97 active+clean; 115 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 1.1 MiB/s wr, 88 op/s 2024-07-26T13:47:17.691 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: ceph-mds: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-5320-ge8f88bb1/rpm/el9/BUILD/ceph-19.0.0-5320-ge8f88bb1/redhat-linux-build/boost/include/boost/smart_ptr/intrusive_ptr.hpp:201: T* boost::intrusive_ptr::operator->() const [with T = const Message]: Assertion `px != 0' failed. 2024-07-26T13:47:17.691 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: *** Caught signal (Aborted) ** 2024-07-26T13:47:17.691 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: in thread 7f53bbd9b640 thread_name:ms_dispatch 2024-07-26T13:47:17.691 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: ceph version 19.0.0-5320-ge8f88bb1 (e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528) squid (dev) 2024-07-26T13:47:17.691 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f53c11726f0] 2024-07-26T13:47:17.691 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 2: /lib64/libc.so.6(+0x8b94c) [0x7f53c11bf94c] 2024-07-26T13:47:17.691 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 3: raise() 2024-07-26T13:47:17.691 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 4: abort() 2024-07-26T13:47:17.691 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 5: /lib64/libc.so.6(+0x2871b) [0x7f53c115c71b] 2024-07-26T13:47:17.692 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 6: /lib64/libc.so.6(+0x37386) [0x7f53c116b386] 2024-07-26T13:47:17.692 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 7: (MDSRank::get_session(boost::intrusive_ptr const&)+0x89) [0x55916e80c6b9] 2024-07-26T13:47:17.692 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 8: (MDCache::path_traverse(boost::intrusive_ptr const&, ContextFactory&, filepath const&, int, std::vector >*, CInode**, CDir**)+0x58e) [0x55916e9ca80c] 2024-07-26T13:47:17.692 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 9: (Server::handle_peer_rename_prep(boost::intrusive_ptr const&)+0x4fe) [0x55916e918cde] 2024-07-26T13:47:17.692 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 10: (Server::dispatch_peer_request(boost::intrusive_ptr const&)+0x933) [0x55916e8da763] 2024-07-26T13:47:17.692 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 11: (Server::handle_peer_request(boost::intrusive_ptr const&)+0x10c3) [0x55916e8dba1b] 2024-07-26T13:47:17.692 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x9ba) [0x55916e8bf752] 2024-07-26T13:47:17.692 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 13: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x81a) [0x55916e80da56] 2024-07-26T13:47:17.692 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 14: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55916e811637] 2024-07-26T13:47:17.692 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 15: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55916e811f5e] 2024-07-26T13:47:17.692 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 16: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55916e7f503a] 2024-07-26T13:47:17.692 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 17: /usr/lib64/ceph/libceph-common.so.2(+0x39b407) [0x7f53c1a3f407] 2024-07-26T13:47:17.693 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 18: (DispatchQueue::entry()+0x60f) [0x7f53c1a3fcf9] 2024-07-26T13:47:17.693 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 19: /usr/lib64/ceph/libceph-common.so.2(+0x4785a1) [0x7f53c1b1c5a1] 2024-07-26T13:47:17.693 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 20: (Thread::entry_wrapper()+0x43) [0x7f53c18c9cdd] 2024-07-26T13:47:17.693 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 21: (Thread::_entry_func(void*)+0xd) [0x7f53c18c9cf9] 2024-07-26T13:47:17.694 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 22: /lib64/libc.so.6(+0x89c02) [0x7f53c11bdc02] 2024-07-26T13:47:17.694 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 23: /lib64/libc.so.6(+0x10ec40) [0x7f53c1242c40] 2024-07-26T13:47:17.694 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 2024-07-26T13:47:17.366+0000 7f53bbd9b640 -1 *** Caught signal (Aborted) ** 2024-07-26T13:47:17.694 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: in thread 7f53bbd9b640 thread_name:ms_dispatch 2024-07-26T13:47:17.694 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 2024-07-26T13:47:17.694 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: ceph version 19.0.0-5320-ge8f88bb1 (e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528) squid (dev) 2024-07-26T13:47:17.694 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f53c11726f0] 2024-07-26T13:47:17.694 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 2: /lib64/libc.so.6(+0x8b94c) [0x7f53c11bf94c] 2024-07-26T13:47:17.694 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 3: raise() 2024-07-26T13:47:17.694 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 4: abort() 2024-07-26T13:47:17.694 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 5: /lib64/libc.so.6(+0x2871b) [0x7f53c115c71b] 2024-07-26T13:47:17.694 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 6: /lib64/libc.so.6(+0x37386) [0x7f53c116b386] 2024-07-26T13:47:17.695 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 7: (MDSRank::get_session(boost::intrusive_ptr const&)+0x89) [0x55916e80c6b9] 2024-07-26T13:47:17.695 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 8: (MDCache::path_traverse(boost::intrusive_ptr const&, ContextFactory&, filepath const&, int, std::vector >*, CInode**, CDir**)+0x58e) [0x55916e9ca80c] 2024-07-26T13:47:17.695 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 9: (Server::handle_peer_rename_prep(boost::intrusive_ptr const&)+0x4fe) [0x55916e918cde] 2024-07-26T13:47:17.695 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 10: (Server::dispatch_peer_request(boost::intrusive_ptr const&)+0x933) [0x55916e8da763] 2024-07-26T13:47:17.695 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 11: (Server::handle_peer_request(boost::intrusive_ptr const&)+0x10c3) [0x55916e8dba1b] 2024-07-26T13:47:17.695 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x9ba) [0x55916e8bf752] 2024-07-26T13:47:17.695 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 13: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x81a) [0x55916e80da56] 2024-07-26T13:47:17.695 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 14: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55916e811637] 2024-07-26T13:47:17.695 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 15: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55916e811f5e] 2024-07-26T13:47:17.695 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 16: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55916e7f503a] 2024-07-26T13:47:17.695 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 17: /usr/lib64/ceph/libceph-common.so.2(+0x39b407) [0x7f53c1a3f407] 2024-07-26T13:47:17.695 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 18: (DispatchQueue::entry()+0x60f) [0x7f53c1a3fcf9] 2024-07-26T13:47:17.695 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 19: /usr/lib64/ceph/libceph-common.so.2(+0x4785a1) [0x7f53c1b1c5a1] 2024-07-26T13:47:17.695 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 20: (Thread::entry_wrapper()+0x43) [0x7f53c18c9cdd] 2024-07-26T13:47:17.696 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 21: (Thread::_entry_func(void*)+0xd) [0x7f53c18c9cf9] 2024-07-26T13:47:17.696 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 22: /lib64/libc.so.6(+0x89c02) [0x7f53c11bdc02] 2024-07-26T13:47:17.696 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 23: /lib64/libc.so.6(+0x10ec40) [0x7f53c1242c40] 2024-07-26T13:47:17.696 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-07-26T13:47:17.696 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 2024-07-26T13:47:17.696 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 0> 2024-07-26T13:47:17.366+0000 7f53bbd9b640 -1 *** Caught signal (Aborted) ** 2024-07-26T13:47:17.696 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: in thread 7f53bbd9b640 thread_name:ms_dispatch 2024-07-26T13:47:17.696 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 2024-07-26T13:47:17.696 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: ceph version 19.0.0-5320-ge8f88bb1 (e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528) squid (dev) 2024-07-26T13:47:17.696 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f53c11726f0] 2024-07-26T13:47:17.696 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 2: /lib64/libc.so.6(+0x8b94c) [0x7f53c11bf94c] 2024-07-26T13:47:17.696 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 3: raise() 2024-07-26T13:47:17.696 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 4: abort() 2024-07-26T13:47:17.696 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 5: /lib64/libc.so.6(+0x2871b) [0x7f53c115c71b] 2024-07-26T13:47:17.696 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 6: /lib64/libc.so.6(+0x37386) [0x7f53c116b386] 2024-07-26T13:47:17.696 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 7: (MDSRank::get_session(boost::intrusive_ptr const&)+0x89) [0x55916e80c6b9] 2024-07-26T13:47:17.697 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 8: (MDCache::path_traverse(boost::intrusive_ptr const&, ContextFactory&, filepath const&, int, std::vector >*, CInode**, CDir**)+0x58e) [0x55916e9ca80c] 2024-07-26T13:47:17.697 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 9: (Server::handle_peer_rename_prep(boost::intrusive_ptr const&)+0x4fe) [0x55916e918cde] 2024-07-26T13:47:17.697 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 10: (Server::dispatch_peer_request(boost::intrusive_ptr const&)+0x933) [0x55916e8da763] 2024-07-26T13:47:17.697 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 11: (Server::handle_peer_request(boost::intrusive_ptr const&)+0x10c3) [0x55916e8dba1b] 2024-07-26T13:47:17.697 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x9ba) [0x55916e8bf752] 2024-07-26T13:47:17.697 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 13: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x81a) [0x55916e80da56] 2024-07-26T13:47:17.697 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 14: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55916e811637] 2024-07-26T13:47:17.697 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 15: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55916e811f5e] 2024-07-26T13:47:17.697 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 16: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55916e7f503a] 2024-07-26T13:47:17.697 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 17: /usr/lib64/ceph/libceph-common.so.2(+0x39b407) [0x7f53c1a3f407] 2024-07-26T13:47:17.697 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 18: (DispatchQueue::entry()+0x60f) [0x7f53c1a3fcf9] 2024-07-26T13:47:17.697 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 19: /usr/lib64/ceph/libceph-common.so.2(+0x4785a1) [0x7f53c1b1c5a1] 2024-07-26T13:47:17.697 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 20: (Thread::entry_wrapper()+0x43) [0x7f53c18c9cdd] 2024-07-26T13:47:17.697 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 21: (Thread::_entry_func(void*)+0xd) [0x7f53c18c9cf9] 2024-07-26T13:47:17.697 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 22: /lib64/libc.so.6(+0x89c02) [0x7f53c11bdc02] 2024-07-26T13:47:17.697 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 23: /lib64/libc.so.6(+0x10ec40) [0x7f53c1242c40] 2024-07-26T13:47:17.698 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-07-26T13:47:17.698 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 2024-07-26T13:47:17.698 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 0> 2024-07-26T13:47:17.366+0000 7f53bbd9b640 -1 *** Caught signal (Aborted) ** 2024-07-26T13:47:17.698 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: in thread 7f53bbd9b640 thread_name:ms_dispatch 2024-07-26T13:47:17.698 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 2024-07-26T13:47:17.698 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: ceph version 19.0.0-5320-ge8f88bb1 (e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528) squid (dev) 2024-07-26T13:47:17.698 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f53c11726f0] 2024-07-26T13:47:17.698 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 2: /lib64/libc.so.6(+0x8b94c) [0x7f53c11bf94c] 2024-07-26T13:47:17.698 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 3: raise() 2024-07-26T13:47:17.698 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 4: abort() 2024-07-26T13:47:17.698 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 5: /lib64/libc.so.6(+0x2871b) [0x7f53c115c71b] 2024-07-26T13:47:17.698 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 6: /lib64/libc.so.6(+0x37386) [0x7f53c116b386] 2024-07-26T13:47:17.698 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 7: (MDSRank::get_session(boost::intrusive_ptr const&)+0x89) [0x55916e80c6b9] 2024-07-26T13:47:17.699 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 8: (MDCache::path_traverse(boost::intrusive_ptr const&, ContextFactory&, filepath const&, int, std::vector >*, CInode**, CDir**)+0x58e) [0x55916e9ca80c] 2024-07-26T13:47:17.699 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 9: (Server::handle_peer_rename_prep(boost::intrusive_ptr const&)+0x4fe) [0x55916e918cde] 2024-07-26T13:47:17.699 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 10: (Server::dispatch_peer_request(boost::intrusive_ptr const&)+0x933) [0x55916e8da763] 2024-07-26T13:47:17.699 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 11: (Server::handle_peer_request(boost::intrusive_ptr const&)+0x10c3) [0x55916e8dba1b] 2024-07-26T13:47:17.699 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x9ba) [0x55916e8bf752] 2024-07-26T13:47:17.699 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 13: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x81a) [0x55916e80da56] 2024-07-26T13:47:17.699 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 14: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55916e811637] 2024-07-26T13:47:17.699 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 15: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55916e811f5e] 2024-07-26T13:47:17.699 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 16: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55916e7f503a] 2024-07-26T13:47:17.699 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 17: /usr/lib64/ceph/libceph-common.so.2(+0x39b407) [0x7f53c1a3f407] 2024-07-26T13:47:17.699 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 18: (DispatchQueue::entry()+0x60f) [0x7f53c1a3fcf9] 2024-07-26T13:47:17.699 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 19: /usr/lib64/ceph/libceph-common.so.2(+0x4785a1) [0x7f53c1b1c5a1] 2024-07-26T13:47:17.699 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 20: (Thread::entry_wrapper()+0x43) [0x7f53c18c9cdd] 2024-07-26T13:47:17.699 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 21: (Thread::_entry_func(void*)+0xd) [0x7f53c18c9cf9] 2024-07-26T13:47:17.699 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 22: /lib64/libc.so.6(+0x89c02) [0x7f53c11bdc02] 2024-07-26T13:47:17.700 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 23: /lib64/libc.so.6(+0x10ec40) [0x7f53c1242c40] 2024-07-26T13:47:17.700 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-07-26T13:47:17.700 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71255]: 2024-07-26T13:47:17.700 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:17 smithi138 podman[71816]: 2024-07-26 13:47:17.421302534 +0000 UTC m=+0.018873170 container died cad2ed376b2212f867d847b358b7f49d6b8038e6b3ee8eaaf93645e44403c599 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-07-26T13:47:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:17 smithi100 ceph-mon[32599]: pgmap v584: 97 pgs: 97 active+clean; 115 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 1.1 MiB/s wr, 88 op/s 2024-07-26T13:47:18.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:17 smithi167 ceph-mon[37140]: pgmap v584: 97 pgs: 97 active+clean; 115 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 1.1 MiB/s wr, 88 op/s 2024-07-26T13:47:18.424 INFO:tasks.workunit.client.0.smithi100.stdout: 1 292567 3.72 MB/sec execute 212 sec latency 1072.678 ms 2024-07-26T13:47:18.590 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:18 smithi138 podman[71816]: 2024-07-26 13:47:18.207864892 +0000 UTC m=+0.805435559 container remove cad2ed376b2212f867d847b358b7f49d6b8038e6b3ee8eaaf93645e44403c599 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb) 2024-07-26T13:47:18.590 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:18 smithi138 systemd[1]: ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.e.service: Main process exited, code=exited, status=134/n/a 2024-07-26T13:47:18.591 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:18 smithi138 systemd[1]: ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.e.service: Failed with result 'exit-code'. 2024-07-26T13:47:18.591 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:18 smithi138 systemd[1]: ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.e.service: Consumed 14.854s CPU time. 2024-07-26T13:47:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:18 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:47:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:18 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:47:19.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:18 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:47:19.424 INFO:tasks.workunit.client.0.smithi100.stdout: 1 292567 3.71 MB/sec execute 213 sec latency 2072.727 ms 2024-07-26T13:47:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:19 smithi138 ceph-mon[36954]: pgmap v585: 97 pgs: 97 active+clean; 115 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 1.4 MiB/s wr, 88 op/s 2024-07-26T13:47:19.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:19 smithi100 ceph-mon[32599]: pgmap v585: 97 pgs: 97 active+clean; 115 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 1.4 MiB/s wr, 88 op/s 2024-07-26T13:47:20.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:19 smithi167 ceph-mon[37140]: pgmap v585: 97 pgs: 97 active+clean; 115 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 1.4 MiB/s wr, 88 op/s 2024-07-26T13:47:20.424 INFO:tasks.workunit.client.0.smithi100.stdout: 1 292567 3.69 MB/sec execute 214 sec latency 3072.814 ms 2024-07-26T13:47:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:20 smithi100 ceph-mon[32599]: pgmap v586: 97 pgs: 97 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 1.6 MiB/s wr, 82 op/s 2024-07-26T13:47:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:20 smithi138 ceph-mon[36954]: pgmap v586: 97 pgs: 97 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 1.6 MiB/s wr, 82 op/s 2024-07-26T13:47:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:20 smithi167 ceph-mon[37140]: pgmap v586: 97 pgs: 97 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 1.6 MiB/s wr, 82 op/s 2024-07-26T13:47:21.424 INFO:tasks.workunit.client.0.smithi100.stdout: 1 292567 3.67 MB/sec execute 215 sec latency 4072.921 ms 2024-07-26T13:47:22.424 INFO:tasks.workunit.client.0.smithi100.stdout: 1 292567 3.65 MB/sec execute 216 sec latency 5072.964 ms 2024-07-26T13:47:23.424 INFO:tasks.workunit.client.0.smithi100.stdout: 1 292567 3.64 MB/sec execute 217 sec latency 6073.099 ms 2024-07-26T13:47:23.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:23 smithi138 ceph-mon[36954]: pgmap v587: 97 pgs: 97 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 1.5 MiB/s wr, 87 op/s 2024-07-26T13:47:23.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:23 smithi100 ceph-mon[32599]: pgmap v587: 97 pgs: 97 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 1.5 MiB/s wr, 87 op/s 2024-07-26T13:47:24.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:23 smithi167 ceph-mon[37140]: pgmap v587: 97 pgs: 97 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 1.5 MiB/s wr, 87 op/s 2024-07-26T13:47:24.425 INFO:tasks.workunit.client.0.smithi100.stdout: 1 292567 3.62 MB/sec execute 218 sec latency 7073.179 ms 2024-07-26T13:47:25.425 INFO:tasks.workunit.client.0.smithi100.stdout: 1 292567 3.60 MB/sec execute 219 sec latency 8073.301 ms 2024-07-26T13:47:25.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:25 smithi138 ceph-mon[36954]: pgmap v588: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.5 MiB/s wr, 80 op/s 2024-07-26T13:47:25.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:25 smithi100 ceph-mon[32599]: pgmap v588: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.5 MiB/s wr, 80 op/s 2024-07-26T13:47:26.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:25 smithi167 ceph-mon[37140]: pgmap v588: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.5 MiB/s wr, 80 op/s 2024-07-26T13:47:26.425 INFO:tasks.workunit.client.0.smithi100.stdout: 1 292567 3.59 MB/sec execute 220 sec latency 9073.375 ms 2024-07-26T13:47:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:26 smithi138 ceph-mon[36954]: pgmap v589: 97 pgs: 97 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 1.7 MiB/s wr, 80 op/s 2024-07-26T13:47:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:26 smithi100 ceph-mon[32599]: pgmap v589: 97 pgs: 97 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 1.7 MiB/s wr, 80 op/s 2024-07-26T13:47:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:26 smithi167 ceph-mon[37140]: pgmap v589: 97 pgs: 97 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 1.7 MiB/s wr, 80 op/s 2024-07-26T13:47:27.425 INFO:tasks.workunit.client.0.smithi100.stdout: 1 292567 3.57 MB/sec execute 221 sec latency 10073.461 ms 2024-07-26T13:47:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.661+0000 7fa27848c640 1 -- 172.21.15.100:0/1087985360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa27010d780 msgr2=0x7fa27010fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.661+0000 7fa27848c640 1 --2- 172.21.15.100:0/1087985360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa27010d780 0x7fa27010fc90 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fa268009900 tx=0x7fa26802a410 comp rx=0 tx=0).stop 2024-07-26T13:47:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.661+0000 7fa27848c640 1 -- 172.21.15.100:0/1087985360 shutdown_connections 2024-07-26T13:47:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.661+0000 7fa27848c640 1 --2- 172.21.15.100:0/1087985360 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa27010d780 0x7fa27010fc90 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.661+0000 7fa27848c640 1 --2- 172.21.15.100:0/1087985360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa270104c00 0x7fa27010d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.661+0000 7fa27848c640 1 --2- 172.21.15.100:0/1087985360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2701041d0 0x7fa2701045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.661+0000 7fa27848c640 1 -- 172.21.15.100:0/1087985360 >> 172.21.15.100:0/1087985360 conn(0x7fa27007a170 msgr2=0x7fa27007a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.661+0000 7fa27848c640 1 -- 172.21.15.100:0/1087985360 shutdown_connections 2024-07-26T13:47:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.661+0000 7fa27848c640 1 -- 172.21.15.100:0/1087985360 wait complete. 2024-07-26T13:47:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.662+0000 7fa27848c640 1 Processor -- start 2024-07-26T13:47:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.662+0000 7fa27848c640 1 -- start start 2024-07-26T13:47:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.662+0000 7fa27848c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2701041d0 0x7fa270199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.662+0000 7fa27848c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa270104c00 0x7fa27019a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.662+0000 7fa27848c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa27010d780 0x7fa27019f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.662+0000 7fa27848c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa270113420 con 0x7fa270104c00 2024-07-26T13:47:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.662+0000 7fa27848c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa2701132a0 con 0x7fa2701041d0 2024-07-26T13:47:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.662+0000 7fa27848c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa2701135a0 con 0x7fa27010d780 2024-07-26T13:47:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.662+0000 7fa275a27640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa270104c00 0x7fa27019a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.662+0000 7fa276228640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2701041d0 0x7fa270199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.662+0000 7fa275a27640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa270104c00 0x7fa27019a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47628/0 (socket says 172.21.15.100:47628) 2024-07-26T13:47:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.662+0000 7fa275a27640 1 -- 172.21.15.100:0/4096979414 learned_addr learned my addr 172.21.15.100:0/4096979414 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.662+0000 7fa276a29640 1 --2- 172.21.15.100:0/4096979414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa27010d780 0x7fa27019f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.662+0000 7fa275a27640 1 -- 172.21.15.100:0/4096979414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa27010d780 msgr2=0x7fa27019f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.662+0000 7fa275a27640 1 --2- 172.21.15.100:0/4096979414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa27010d780 0x7fa27019f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.662+0000 7fa275a27640 1 -- 172.21.15.100:0/4096979414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2701041d0 msgr2=0x7fa270199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.662+0000 7fa275a27640 1 --2- 172.21.15.100:0/4096979414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2701041d0 0x7fa270199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.662+0000 7fa275a27640 1 -- 172.21.15.100:0/4096979414 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa27019fd70 con 0x7fa270104c00 2024-07-26T13:47:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.663+0000 7fa276228640 1 --2- 172.21.15.100:0/4096979414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2701041d0 0x7fa270199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:47:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.663+0000 7fa275a27640 1 --2- 172.21.15.100:0/4096979414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa270104c00 0x7fa27019a3e0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fa26400c970 tx=0x7fa26400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.663+0000 7fa26f7fe640 1 -- 172.21.15.100:0/4096979414 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa264007bb0 con 0x7fa270104c00 2024-07-26T13:47:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.663+0000 7fa26f7fe640 1 -- 172.21.15.100:0/4096979414 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa264007d50 con 0x7fa270104c00 2024-07-26T13:47:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.663+0000 7fa27848c640 1 -- 172.21.15.100:0/4096979414 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2701a0060 con 0x7fa270104c00 2024-07-26T13:47:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.663+0000 7fa26f7fe640 1 -- 172.21.15.100:0/4096979414 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26400f450 con 0x7fa270104c00 2024-07-26T13:47:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.663+0000 7fa27848c640 1 -- 172.21.15.100:0/4096979414 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2701a67f0 con 0x7fa270104c00 2024-07-26T13:47:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.666+0000 7fa26f7fe640 1 -- 172.21.15.100:0/4096979414 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fa264004eb0 con 0x7fa270104c00 2024-07-26T13:47:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.666+0000 7fa27f39f740 1 Processor -- start 2024-07-26T13:47:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.666+0000 7fa27f39f740 1 -- start start 2024-07-26T13:47:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.666+0000 7fa27f39f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55846ac971b0 0x55846b039ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.666+0000 7fa27f39f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846ac6e2f0 0x55846b03a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.666+0000 7fa27f39f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55846ac764d0 0x55846b03a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.666+0000 7fa27f39f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55846afaf3b0 con 0x55846ac764d0 2024-07-26T13:47:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.666+0000 7fa27f39f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55846af9e8b0 con 0x55846ac6e2f0 2024-07-26T13:47:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.666+0000 7fa27f39f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55846afa28f0 con 0x55846ac971b0 2024-07-26T13:47:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.667+0000 7fa275a27640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55846ac764d0 0x55846b03a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.667+0000 7fa275a27640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55846ac764d0 0x55846b03a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47644/0 (socket says 172.21.15.100:47644) 2024-07-26T13:47:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.667+0000 7fa275a27640 1 -- 172.21.15.100:0/3985790832 learned_addr learned my addr 172.21.15.100:0/3985790832 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.667+0000 7fa276a29640 1 --2- 172.21.15.100:0/3985790832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55846ac971b0 0x55846b039ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.667+0000 7fa276228640 1 --2- 172.21.15.100:0/3985790832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846ac6e2f0 0x55846b03a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.667+0000 7fa26f7fe640 1 --2- 172.21.15.100:0/4096979414 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fa2580b3d40 0x7fa2580b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.667+0000 7fa276a29640 1 --2- 172.21.15.100:0/4096979414 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fa2580b3d40 0x7fa2580b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.667+0000 7fa275a27640 1 -- 172.21.15.100:0/3985790832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55846ac971b0 msgr2=0x55846b039ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.667+0000 7fa275a27640 1 --2- 172.21.15.100:0/3985790832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55846ac971b0 0x55846b039ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.667+0000 7fa275a27640 1 -- 172.21.15.100:0/3985790832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846ac6e2f0 msgr2=0x55846b03a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.667+0000 7fa275a27640 1 --2- 172.21.15.100:0/3985790832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846ac6e2f0 0x55846b03a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.667+0000 7fa275a27640 1 -- 172.21.15.100:0/3985790832 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55846af459a0 con 0x55846ac764d0 2024-07-26T13:47:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.667+0000 7fa26f7fe640 1 -- 172.21.15.100:0/4096979414 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8786+0+0 (secure 0 0 0) 0x7fa2640e3e40 con 0x7fa270104c00 2024-07-26T13:47:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.667+0000 7fa275a27640 1 --2- 172.21.15.100:0/3985790832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55846ac764d0 0x55846b03a760 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fa2640e13f0 tx=0x7fa264000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa27fddd267a349c server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.667+0000 7fa276a29640 1 --2- 172.21.15.100:0/4096979414 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fa2580b3d40 0x7fa2580b6250 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa268005ab0 tx=0x7fa26802a410 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.667+0000 7fa26cbfe640 1 -- 172.21.15.100:0/3985790832 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2640aa710 con 0x55846ac764d0 2024-07-26T13:47:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.667+0000 7fa26cbfe640 1 -- 172.21.15.100:0/3985790832 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2640aa450 con 0x55846ac764d0 2024-07-26T13:47:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.667+0000 7fa26cbfe640 1 -- 172.21.15.100:0/3985790832 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2640a60d0 con 0x55846ac764d0 2024-07-26T13:47:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.668+0000 7fa27f39f740 1 -- 172.21.15.100:0/3985790832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55846ac764d0 msgr2=0x55846b03a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.668+0000 7fa27f39f740 1 --2- 172.21.15.100:0/3985790832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55846ac764d0 0x55846b03a760 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fa2640e13f0 tx=0x7fa264000bc0 comp rx=0 tx=0).stop 2024-07-26T13:47:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.668+0000 7fa27f39f740 1 -- 172.21.15.100:0/3985790832 shutdown_connections 2024-07-26T13:47:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.668+0000 7fa27f39f740 1 --2- 172.21.15.100:0/3985790832 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55846ac764d0 0x55846b03a760 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.668+0000 7fa27f39f740 1 --2- 172.21.15.100:0/3985790832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846ac6e2f0 0x55846b03a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.668+0000 7fa27f39f740 1 --2- 172.21.15.100:0/3985790832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55846ac971b0 0x55846b039ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.668+0000 7fa27f39f740 1 -- 172.21.15.100:0/3985790832 >> 172.21.15.100:0/3985790832 conn(0x55846ac8b300 msgr2=0x55846b037bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.668+0000 7fa27f39f740 1 -- 172.21.15.100:0/3985790832 shutdown_connections 2024-07-26T13:47:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.668+0000 7fa27f39f740 1 -- 172.21.15.100:0/3985790832 wait complete. 2024-07-26T13:47:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.669+0000 7fa27f39f740 1 -- start start 2024-07-26T13:47:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.669+0000 7fa27f39f740 1 Processor -- start 2024-07-26T13:47:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.669+0000 7fa27f39f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55846ac6e2f0 0x55846b049c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.669+0000 7fa27f39f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846ac764d0 0x55846b04a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.669+0000 7fa27f39f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55846ac971b0 0x55846b055b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.669+0000 7fa27f39f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55846aed8ab0 con 0x55846ac971b0 2024-07-26T13:47:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.669+0000 7fa27f39f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55846af26cf0 con 0x55846ac764d0 2024-07-26T13:47:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.669+0000 7fa27f39f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55846b04c7e0 con 0x55846ac6e2f0 2024-07-26T13:47:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.669+0000 7fa27f39f740 10 client.0 _ll_register_callbacks cb 0x55846af989f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:47:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.669+0000 7fa276a29640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55846ac971b0 0x55846b055b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.669+0000 7fa276a29640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55846ac971b0 0x55846b055b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47646/0 (socket says 172.21.15.100:47646) 2024-07-26T13:47:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.669+0000 7fa276a29640 1 -- 172.21.15.100:0/1081711164 learned_addr learned my addr 172.21.15.100:0/1081711164 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.669+0000 7fa276228640 1 --2- 172.21.15.100:0/1081711164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55846ac6e2f0 0x55846b049c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.670+0000 7fa275a27640 1 --2- 172.21.15.100:0/1081711164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846ac764d0 0x55846b04a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.670+0000 7fa276228640 1 -- 172.21.15.100:0/1081711164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846ac764d0 msgr2=0x55846b04a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.670+0000 7fa276228640 1 --2- 172.21.15.100:0/1081711164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846ac764d0 0x55846b04a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.670+0000 7fa276228640 1 -- 172.21.15.100:0/1081711164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55846ac971b0 msgr2=0x55846b055b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.670+0000 7fa276228640 1 --2- 172.21.15.100:0/1081711164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55846ac971b0 0x55846b055b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.670+0000 7fa276228640 1 -- 172.21.15.100:0/1081711164 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55846ad02790 con 0x55846ac6e2f0 2024-07-26T13:47:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.670+0000 7fa276a29640 1 --2- 172.21.15.100:0/1081711164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55846ac971b0 0x55846b055b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:47:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.670+0000 7fa276228640 1 --2- 172.21.15.100:0/1081711164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55846ac6e2f0 0x55846b049c50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa260005fd0 tx=0x7fa260013830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c23db6233a0de6d9 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.670+0000 7fa255ffb640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-07-26T13:47:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.670+0000 7fa255ffb640 1 -- 172.21.15.100:0/1081711164 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2600107e0 con 0x55846ac6e2f0 2024-07-26T13:47:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.670+0000 7fa255ffb640 1 -- 172.21.15.100:0/1081711164 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8786+0+0 (secure 0 0 0) 0x7fa260010980 con 0x55846ac6e2f0 2024-07-26T13:47:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.670+0000 7fa27f39f740 1 -- 172.21.15.100:0/1081711164 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2640aa450 con 0x55846ac6e2f0 2024-07-26T13:47:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.670+0000 7fa255ffb640 1 -- 172.21.15.100:0/1081711164 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fa260010b80 con 0x55846ac6e2f0 2024-07-26T13:47:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.670+0000 7fa27f39f740 10 client.26740 fetch_fsmap learned FSMap version 54 2024-07-26T13:47:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.670+0000 7fa27f39f740 1 -- 172.21.15.100:0/1081711164 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55846ad7b620 con 0x55846ac6e2f0 2024-07-26T13:47:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.671+0000 7fa255ffb640 1 -- 172.21.15.100:0/1081711164 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2640aa450 con 0x55846ac6e2f0 2024-07-26T13:47:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.671+0000 7fa255ffb640 1 -- 172.21.15.100:0/1081711164 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 54) ==== 8350+0+0 (secure 0 0 0) 0x7fa26001a070 con 0x55846ac6e2f0 2024-07-26T13:47:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.671+0000 7fa27f39f740 10 client.26740 fetch_fsmap finished waiting for FSMap version 54 2024-07-26T13:47:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.671+0000 7fa27f39f740 10 client.26740 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:47:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.671+0000 7fa27f39f740 20 client.26740 populate_metadata read hostname 'smithi100' 2024-07-26T13:47:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.671+0000 7fa27f39f740 1 --2- 172.21.15.100:0/1081711164 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55846ac55d80 0x55846b05bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.671+0000 7fa27f39f740 4 client.26740 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:47:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.671+0000 7fa27f39f740 1 -- 172.21.15.100:0/1081711164 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55846afb1300 con 0x55846ac55d80 2024-07-26T13:47:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.671+0000 7fa275a27640 1 --2- 172.21.15.100:0/1081711164 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55846ac55d80 0x55846b05bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.672+0000 7fa275a27640 1 --2- 172.21.15.100:0/1081711164 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55846ac55d80 0x55846b05bbd0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6e2891049a1d90e server_cookie=25ea8765f914ffad in_seq=0 out_seq=0 2024-07-26T13:47:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.672+0000 7fa255ffb640 10 client.26740 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.672+0000 7fa255ffb640 1 -- 172.21.15.100:0/1081711164 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55846afb1300 con 0x55846ac55d80 2024-07-26T13:47:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.672+0000 7fa255ffb640 10 client.26740 handle_command_reply: tid=0 2024-07-26T13:47:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.672+0000 7fa27f39f740 1 client.26740 shutdown 2024-07-26T13:47:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.673+0000 7fa27f39f740 1 -- 172.21.15.100:0/1081711164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55846ac6e2f0 msgr2=0x55846b049c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.673+0000 7fa27f39f740 1 --2- 172.21.15.100:0/1081711164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55846ac6e2f0 0x55846b049c50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa260005fd0 tx=0x7fa260013830 comp rx=0 tx=0).stop 2024-07-26T13:47:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.673+0000 7fa27f39f740 1 -- 172.21.15.100:0/1081711164 shutdown_connections 2024-07-26T13:47:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.673+0000 7fa27f39f740 1 --2- 172.21.15.100:0/1081711164 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55846ac55d80 0x55846b05bbd0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.673+0000 7fa27f39f740 1 --2- 172.21.15.100:0/1081711164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55846ac971b0 0x55846b055b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.673+0000 7fa27f39f740 1 --2- 172.21.15.100:0/1081711164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846ac764d0 0x55846b04a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.673+0000 7fa27f39f740 1 --2- 172.21.15.100:0/1081711164 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55846ac6e2f0 0x55846b049c50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.673+0000 7fa255ffb640 0 client.26740 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.673+0000 7fa27f39f740 1 -- 172.21.15.100:0/1081711164 >> 172.21.15.100:0/1081711164 conn(0x55846ac8b300 msgr2=0x55846b03b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.673+0000 7fa27f39f740 1 -- 172.21.15.100:0/1081711164 shutdown_connections 2024-07-26T13:47:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.673+0000 7fa27f39f740 1 -- 172.21.15.100:0/1081711164 wait complete. 2024-07-26T13:47:27.681 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.673+0000 7fa27f39f740 20 client.26740 trim_cache size 0 max 16384 2024-07-26T13:47:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.688+0000 7fa27f39f740 1 Processor -- start 2024-07-26T13:47:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.688+0000 7fa27f39f740 1 -- start start 2024-07-26T13:47:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.688+0000 7fa27f39f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55846ac6e2f0 0x55846b049f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.688+0000 7fa27f39f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846ac764d0 0x55846b0482a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.688+0000 7fa27f39f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55846ac971b0 0x55846b0487e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.688+0000 7fa27f39f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa2701afac0 con 0x55846ac971b0 2024-07-26T13:47:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.688+0000 7fa27f39f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa2701ad360 con 0x55846ac764d0 2024-07-26T13:47:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.688+0000 7fa27f39f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa2701ab1d0 con 0x55846ac6e2f0 2024-07-26T13:47:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.688+0000 7fa276a29640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55846ac971b0 0x55846b0487e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.688+0000 7fa276a29640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55846ac971b0 0x55846b0487e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47650/0 (socket says 172.21.15.100:47650) 2024-07-26T13:47:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.688+0000 7fa276a29640 1 -- 172.21.15.100:0/203707943 learned_addr learned my addr 172.21.15.100:0/203707943 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.688+0000 7fa275a27640 1 --2- 172.21.15.100:0/203707943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846ac764d0 0x55846b0482a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.688+0000 7fa276228640 1 --2- 172.21.15.100:0/203707943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55846ac6e2f0 0x55846b049f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.688+0000 7fa276a29640 1 -- 172.21.15.100:0/203707943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55846ac6e2f0 msgr2=0x55846b049f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.688+0000 7fa276a29640 1 --2- 172.21.15.100:0/203707943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55846ac6e2f0 0x55846b049f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.689+0000 7fa276a29640 1 -- 172.21.15.100:0/203707943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846ac764d0 msgr2=0x55846b0482a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.689+0000 7fa276a29640 1 --2- 172.21.15.100:0/203707943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846ac764d0 0x55846b0482a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.689+0000 7fa276a29640 1 -- 172.21.15.100:0/203707943 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55846b04a460 con 0x55846ac971b0 2024-07-26T13:47:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.689+0000 7fa276a29640 1 --2- 172.21.15.100:0/203707943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55846ac971b0 0x55846b0487e0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fa268009230 tx=0x7fa268005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88a5d4a85bc3c845 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.689+0000 7fa257fff640 1 -- 172.21.15.100:0/203707943 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa268035070 con 0x55846ac971b0 2024-07-26T13:47:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.689+0000 7fa257fff640 1 -- 172.21.15.100:0/203707943 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2680055b0 con 0x55846ac971b0 2024-07-26T13:47:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.689+0000 7fa257fff640 1 -- 172.21.15.100:0/203707943 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa268004e60 con 0x55846ac971b0 2024-07-26T13:47:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.690+0000 7fa27f39f740 1 -- 172.21.15.100:0/203707943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55846ac971b0 msgr2=0x55846b0487e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.690+0000 7fa27f39f740 1 --2- 172.21.15.100:0/203707943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55846ac971b0 0x55846b0487e0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fa268009230 tx=0x7fa268005a40 comp rx=0 tx=0).stop 2024-07-26T13:47:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.690+0000 7fa27f39f740 1 -- 172.21.15.100:0/203707943 shutdown_connections 2024-07-26T13:47:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.690+0000 7fa27f39f740 1 --2- 172.21.15.100:0/203707943 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55846ac971b0 0x55846b0487e0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.690+0000 7fa27f39f740 1 --2- 172.21.15.100:0/203707943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846ac764d0 0x55846b0482a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.690+0000 7fa27f39f740 1 --2- 172.21.15.100:0/203707943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55846ac6e2f0 0x55846b049f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.690+0000 7fa27f39f740 1 -- 172.21.15.100:0/203707943 >> 172.21.15.100:0/203707943 conn(0x55846ac8b300 msgr2=0x55846b03ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.690+0000 7fa27f39f740 1 -- 172.21.15.100:0/203707943 shutdown_connections 2024-07-26T13:47:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.690+0000 7fa27f39f740 1 -- 172.21.15.100:0/203707943 wait complete. 2024-07-26T13:47:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.690+0000 7fa27f39f740 1 -- start start 2024-07-26T13:47:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.690+0000 7fa27f39f740 1 Processor -- start 2024-07-26T13:47:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.690+0000 7fa27f39f740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55846ac6e2f0 0x55846b051ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.690+0000 7fa27f39f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846ac764d0 0x55846b052280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.690+0000 7fa27f39f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55846ac971b0 0x55846b04eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.690+0000 7fa27f39f740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa2701ab820 con 0x55846ac6e2f0 2024-07-26T13:47:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.690+0000 7fa27f39f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa2701acc90 con 0x55846ac764d0 2024-07-26T13:47:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.690+0000 7fa27f39f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa2701abef0 con 0x55846ac971b0 2024-07-26T13:47:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.690+0000 7fa27f39f740 10 client.0 _ll_register_callbacks cb 0x7fa2600113e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:47:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.690+0000 7fa276228640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55846ac6e2f0 0x55846b051ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.691+0000 7fa276228640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55846ac6e2f0 0x55846b051ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47654/0 (socket says 172.21.15.100:47654) 2024-07-26T13:47:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.691+0000 7fa276228640 1 -- 172.21.15.100:0/222811240 learned_addr learned my addr 172.21.15.100:0/222811240 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.691+0000 7fa276a29640 1 --2- 172.21.15.100:0/222811240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55846ac971b0 0x55846b04eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.691+0000 7fa275a27640 1 --2- 172.21.15.100:0/222811240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846ac764d0 0x55846b052280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.691+0000 7fa276228640 1 -- 172.21.15.100:0/222811240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55846ac971b0 msgr2=0x55846b04eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.691+0000 7fa276228640 1 --2- 172.21.15.100:0/222811240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55846ac971b0 0x55846b04eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.691+0000 7fa276228640 1 -- 172.21.15.100:0/222811240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846ac764d0 msgr2=0x55846b052280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.691+0000 7fa276228640 1 --2- 172.21.15.100:0/222811240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846ac764d0 0x55846b052280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.691+0000 7fa276228640 1 -- 172.21.15.100:0/222811240 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55846b04f2e0 con 0x55846ac6e2f0 2024-07-26T13:47:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.691+0000 7fa275a27640 1 --2- 172.21.15.100:0/222811240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846ac764d0 0x55846b052280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:47:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.691+0000 7fa276228640 1 --2- 172.21.15.100:0/222811240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55846ac6e2f0 0x55846b051ca0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fa260000f80 tx=0x7fa26001b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2296aef99b18e766 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.691+0000 7fa255ffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:47:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.691+0000 7fa255ffb640 1 -- 172.21.15.100:0/222811240 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa260030060 con 0x55846ac6e2f0 2024-07-26T13:47:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.691+0000 7fa255ffb640 1 -- 172.21.15.100:0/222811240 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8786+0+0 (secure 0 0 0) 0x7fa2600049a0 con 0x55846ac6e2f0 2024-07-26T13:47:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.691+0000 7fa27f39f740 1 -- 172.21.15.100:0/222811240 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2680055b0 con 0x55846ac6e2f0 2024-07-26T13:47:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.691+0000 7fa255ffb640 1 -- 172.21.15.100:0/222811240 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fa260037050 con 0x55846ac6e2f0 2024-07-26T13:47:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.691+0000 7fa27f39f740 10 client.17349 fetch_fsmap learned FSMap version 54 2024-07-26T13:47:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.691+0000 7fa27f39f740 1 -- 172.21.15.100:0/222811240 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55846b0c2e70 con 0x55846ac6e2f0 2024-07-26T13:47:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.691+0000 7fa255ffb640 1 -- 172.21.15.100:0/222811240 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2680055b0 con 0x55846ac6e2f0 2024-07-26T13:47:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.692+0000 7fa255ffb640 1 -- 172.21.15.100:0/222811240 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 54) ==== 8350+0+0 (secure 0 0 0) 0x7fa26001a070 con 0x55846ac6e2f0 2024-07-26T13:47:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.692+0000 7fa27f39f740 10 client.17349 fetch_fsmap finished waiting for FSMap version 54 2024-07-26T13:47:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.692+0000 7fa27f39f740 10 client.17349 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:47:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.692+0000 7fa27f39f740 20 client.17349 populate_metadata read hostname 'smithi100' 2024-07-26T13:47:27.698 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.692+0000 7fa27f39f740 1 --2- 172.21.15.100:0/222811240 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55846ac2b1c0 0x55846b0c5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.692+0000 7fa27f39f740 4 client.17349 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-26T13:47:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.692+0000 7fa27f39f740 1 -- 172.21.15.100:0/222811240 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55846afacda0 con 0x55846ac2b1c0 2024-07-26T13:47:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.692+0000 7fa275a27640 1 --2- 172.21.15.100:0/222811240 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55846ac2b1c0 0x55846b0c5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.692+0000 7fa255ffb640 10 client.17349 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:27.699 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:27.692+0000 7fa275a27640 1 --2- 172.21.15.100:0/222811240 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55846ac2b1c0 0x55846b0c5660 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f6325a5ceed35d5 server_cookie=82cdb55c1346d24f in_seq=0 out_seq=0 2024-07-26T13:47:28.425 INFO:tasks.workunit.client.0.smithi100.stdout: 1 292567 3.55 MB/sec execute 222 sec latency 11073.517 ms 2024-07-26T13:47:28.943 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:28 smithi138 systemd[1]: ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.e.service: Scheduled restart job, restart counter is at 2. 2024-07-26T13:47:28.944 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:28 smithi138 systemd[1]: Stopped Ceph mds.e for bf83e2e6-4b52-11ef-bcb7-c7b262605968. 2024-07-26T13:47:28.944 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:28 smithi138 systemd[1]: ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.e.service: Consumed 14.854s CPU time. 2024-07-26T13:47:28.944 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:28 smithi138 systemd[1]: Starting Ceph mds.e for bf83e2e6-4b52-11ef-bcb7-c7b262605968... 2024-07-26T13:47:28.944 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:28 smithi138 podman[71909]: 2024-07-26 13:47:28.826996156 +0000 UTC m=+0.017523439 image pull ae51f432d4fa82622bc253ca9e2da7666c617e096e99d43d0a0e4fdaca8865f8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2 2024-07-26T13:47:29.226 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:28 smithi138 podman[71909]: 2024-07-26 13:47:28.943477123 +0000 UTC m=+0.134004441 container create 350d3320487c6730506955ebba5c990f370244c601da207d0f2ad2f7b84db05c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.build-date=20240724) 2024-07-26T13:47:29.226 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:29 smithi138 podman[71909]: 2024-07-26 13:47:29.189306374 +0000 UTC m=+0.379833650 container init 350d3320487c6730506955ebba5c990f370244c601da207d0f2ad2f7b84db05c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-07-26T13:47:29.226 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:29 smithi138 podman[71909]: 2024-07-26 13:47:29.192111056 +0000 UTC m=+0.382638335 container start 350d3320487c6730506955ebba5c990f370244c601da207d0f2ad2f7b84db05c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-07-26T13:47:29.226 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:29 smithi138 ceph-mds[71929]: -- 172.21.15.138:0/8862401 <== mon.2 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562d07f0d860 con 0x562d0c0af180 2024-07-26T13:47:29.425 INFO:tasks.workunit.client.0.smithi100.stdout: 1 292567 3.54 MB/sec execute 223 sec latency 12073.612 ms 2024-07-26T13:47:29.589 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:29 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-e[71925]: starting mds.e at 2024-07-26T13:47:29.589 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:29 smithi138 bash[71909]: 350d3320487c6730506955ebba5c990f370244c601da207d0f2ad2f7b84db05c 2024-07-26T13:47:29.589 INFO:journalctl@ceph.mds.e.smithi138.stdout:Jul 26 13:47:29 smithi138 systemd[1]: Started Ceph mds.e for bf83e2e6-4b52-11ef-bcb7-c7b262605968. 2024-07-26T13:47:29.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:29 smithi138 ceph-mon[36954]: pgmap v590: 97 pgs: 97 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 1.8 MiB/s wr, 79 op/s 2024-07-26T13:47:29.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:29 smithi138 ceph-mon[36954]: daemon mds.e restarted 2024-07-26T13:47:29.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:29 smithi138 ceph-mon[36954]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-26T13:47:29.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:29 smithi138 ceph-mon[36954]: osdmap e87: 12 total, 12 up, 12 in 2024-07-26T13:47:29.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:29 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.138:6834/1781111462,v1:172.21.15.138:6835/1781111462] up:boot 2024-07-26T13:47:29.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:29 smithi138 ceph-mon[36954]: Standby daemon mds.c assigned to filesystem cephfs as rank 2 2024-07-26T13:47:29.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:29 smithi138 ceph-mon[36954]: fsmap cephfs:2/3 {0=i=up:active,1=b=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-07-26T13:47:29.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-26T13:47:29.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:29 smithi138 ceph-mon[36954]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=c=up:replay} 2 up:standby-replay 7 up:standby 2024-07-26T13:47:29.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:29 smithi100 ceph-mon[32599]: pgmap v590: 97 pgs: 97 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 1.8 MiB/s wr, 79 op/s 2024-07-26T13:47:29.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:29 smithi100 ceph-mon[32599]: daemon mds.e restarted 2024-07-26T13:47:29.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:29 smithi100 ceph-mon[32599]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-26T13:47:29.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:29 smithi100 ceph-mon[32599]: osdmap e87: 12 total, 12 up, 12 in 2024-07-26T13:47:29.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:29 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.138:6834/1781111462,v1:172.21.15.138:6835/1781111462] up:boot 2024-07-26T13:47:29.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:29 smithi100 ceph-mon[32599]: Standby daemon mds.c assigned to filesystem cephfs as rank 2 2024-07-26T13:47:29.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:29 smithi100 ceph-mon[32599]: fsmap cephfs:2/3 {0=i=up:active,1=b=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-07-26T13:47:29.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-26T13:47:29.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:29 smithi100 ceph-mon[32599]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=c=up:replay} 2 up:standby-replay 7 up:standby 2024-07-26T13:47:30.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:29 smithi167 ceph-mon[37140]: pgmap v590: 97 pgs: 97 active+clean; 112 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 1.8 MiB/s wr, 79 op/s 2024-07-26T13:47:30.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:29 smithi167 ceph-mon[37140]: daemon mds.e restarted 2024-07-26T13:47:30.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:29 smithi167 ceph-mon[37140]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-26T13:47:30.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:29 smithi167 ceph-mon[37140]: osdmap e87: 12 total, 12 up, 12 in 2024-07-26T13:47:30.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:29 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.138:6834/1781111462,v1:172.21.15.138:6835/1781111462] up:boot 2024-07-26T13:47:30.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:29 smithi167 ceph-mon[37140]: Standby daemon mds.c assigned to filesystem cephfs as rank 2 2024-07-26T13:47:30.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:29 smithi167 ceph-mon[37140]: fsmap cephfs:2/3 {0=i=up:active,1=b=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-07-26T13:47:30.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-26T13:47:30.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:29 smithi167 ceph-mon[37140]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=c=up:replay} 2 up:standby-replay 7 up:standby 2024-07-26T13:47:30.425 INFO:tasks.workunit.client.0.smithi100.stdout: 1 292567 3.52 MB/sec execute 224 sec latency 13073.731 ms 2024-07-26T13:47:31.425 INFO:tasks.workunit.client.0.smithi100.stdout: 1 292567 3.51 MB/sec execute 225 sec latency 14073.829 ms 2024-07-26T13:47:31.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:31 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.167:6836/3519222898,v1:172.21.15.167:6837/3519222898] up:resolve 2024-07-26T13:47:31.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:31 smithi167 ceph-mon[37140]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=c=up:resolve} 2 up:standby-replay 7 up:standby 2024-07-26T13:47:31.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:31 smithi167 ceph-mon[37140]: pgmap v592: 97 pgs: 97 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 1.6 MiB/s wr, 68 op/s 2024-07-26T13:47:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:31 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.167:6836/3519222898,v1:172.21.15.167:6837/3519222898] up:resolve 2024-07-26T13:47:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:31 smithi138 ceph-mon[36954]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=c=up:resolve} 2 up:standby-replay 7 up:standby 2024-07-26T13:47:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:31 smithi138 ceph-mon[36954]: pgmap v592: 97 pgs: 97 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 1.6 MiB/s wr, 68 op/s 2024-07-26T13:47:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:31 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.167:6836/3519222898,v1:172.21.15.167:6837/3519222898] up:resolve 2024-07-26T13:47:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:31 smithi100 ceph-mon[32599]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=c=up:resolve} 2 up:standby-replay 7 up:standby 2024-07-26T13:47:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:31 smithi100 ceph-mon[32599]: pgmap v592: 97 pgs: 97 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 1.6 MiB/s wr, 68 op/s 2024-07-26T13:47:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:32.282+0000 7fa255ffb640 1 -- 172.21.15.100:0/222811240 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55846afacda0 con 0x55846ac2b1c0 2024-07-26T13:47:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:32.282+0000 7fa255ffb640 10 client.17349 handle_command_reply: tid=0 2024-07-26T13:47:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:32.282+0000 7fa27f39f740 1 client.17349 shutdown 2024-07-26T13:47:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:32.282+0000 7fa27f39f740 1 -- 172.21.15.100:0/222811240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55846ac6e2f0 msgr2=0x55846b051ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:32.282+0000 7fa27f39f740 1 --2- 172.21.15.100:0/222811240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55846ac6e2f0 0x55846b051ca0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fa260000f80 tx=0x7fa26001b760 comp rx=0 tx=0).stop 2024-07-26T13:47:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:32.282+0000 7fa27f39f740 1 -- 172.21.15.100:0/222811240 shutdown_connections 2024-07-26T13:47:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:32.282+0000 7fa27f39f740 1 --2- 172.21.15.100:0/222811240 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55846ac2b1c0 0x55846b0c5660 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:32.282+0000 7fa27f39f740 1 --2- 172.21.15.100:0/222811240 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55846ac971b0 0x55846b04eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:32.282+0000 7fa27f39f740 1 --2- 172.21.15.100:0/222811240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55846ac764d0 0x55846b052280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:32.282+0000 7fa27f39f740 1 --2- 172.21.15.100:0/222811240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55846ac6e2f0 0x55846b051ca0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:32.282+0000 7fa255ffb640 0 client.17349 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:32.282+0000 7fa27f39f740 1 -- 172.21.15.100:0/222811240 >> 172.21.15.100:0/222811240 conn(0x55846ac8b300 msgr2=0x55846b048260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:32.282+0000 7fa27f39f740 1 -- 172.21.15.100:0/222811240 shutdown_connections 2024-07-26T13:47:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:32.282+0000 7fa27f39f740 1 -- 172.21.15.100:0/222811240 wait complete. 2024-07-26T13:47:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:32.282+0000 7fa27f39f740 20 client.17349 trim_cache size 0 max 16384 2024-07-26T13:47:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-07-26T13:47:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-07-26T13:47:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-07-26T13:47:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-07-26T13:47:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:32.282+0000 7fa27848c640 1 -- 172.21.15.100:0/4096979414 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fa2580b3d40 msgr2=0x7fa2580b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:32.282+0000 7fa27848c640 1 --2- 172.21.15.100:0/4096979414 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fa2580b3d40 0x7fa2580b6250 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa268005ab0 tx=0x7fa26802a410 comp rx=0 tx=0).stop 2024-07-26T13:47:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:32.282+0000 7fa27848c640 1 -- 172.21.15.100:0/4096979414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa270104c00 msgr2=0x7fa27019a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:32.282+0000 7fa27848c640 1 --2- 172.21.15.100:0/4096979414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa270104c00 0x7fa27019a3e0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fa26400c970 tx=0x7fa26400ce30 comp rx=0 tx=0).stop 2024-07-26T13:47:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:32.282+0000 7fa27848c640 1 -- 172.21.15.100:0/4096979414 shutdown_connections 2024-07-26T13:47:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:32.282+0000 7fa27848c640 1 --2- 172.21.15.100:0/4096979414 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fa2580b3d40 0x7fa2580b6250 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:32.282+0000 7fa27848c640 1 --2- 172.21.15.100:0/4096979414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa27010d780 0x7fa27019f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:32.282+0000 7fa27848c640 1 --2- 172.21.15.100:0/4096979414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa270104c00 0x7fa27019a3e0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:32.282+0000 7fa27848c640 1 --2- 172.21.15.100:0/4096979414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2701041d0 0x7fa270199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:32.283+0000 7fa27848c640 1 -- 172.21.15.100:0/4096979414 >> 172.21.15.100:0/4096979414 conn(0x7fa27007a170 msgr2=0x7fa270107300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:32.283+0000 7fa27848c640 1 -- 172.21.15.100:0/4096979414 shutdown_connections 2024-07-26T13:47:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:32.283+0000 7fa27848c640 1 -- 172.21.15.100:0/4096979414 wait complete. 2024-07-26T13:47:32.425 INFO:tasks.workunit.client.0.smithi100.stdout: 1 292567 3.49 MB/sec execute 226 sec latency 15073.924 ms 2024-07-26T13:47:32.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:32 smithi167 ceph-mon[37140]: reconnect by client.15333 v1:192.168.0.1:0/1052122651 after 0.000999982 2024-07-26T13:47:32.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:32 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.167:6836/3519222898,v1:172.21.15.167:6837/3519222898] up:reconnect 2024-07-26T13:47:32.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:32 smithi167 ceph-mon[37140]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=c=up:reconnect} 2 up:standby-replay 7 up:standby 2024-07-26T13:47:32.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:32 smithi138 ceph-mon[36954]: reconnect by client.15333 v1:192.168.0.1:0/1052122651 after 0.000999982 2024-07-26T13:47:32.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:32 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.167:6836/3519222898,v1:172.21.15.167:6837/3519222898] up:reconnect 2024-07-26T13:47:32.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:32 smithi138 ceph-mon[36954]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=c=up:reconnect} 2 up:standby-replay 7 up:standby 2024-07-26T13:47:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:32 smithi100 ceph-mon[32599]: reconnect by client.15333 v1:192.168.0.1:0/1052122651 after 0.000999982 2024-07-26T13:47:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:32 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.167:6836/3519222898,v1:172.21.15.167:6837/3519222898] up:reconnect 2024-07-26T13:47:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:32 smithi100 ceph-mon[32599]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=c=up:reconnect} 2 up:standby-replay 7 up:standby 2024-07-26T13:47:33.425 INFO:tasks.workunit.client.0.smithi100.stdout: 1 292651 3.48 MB/sec execute 227 sec latency 15918.415 ms 2024-07-26T13:47:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:33 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.167:6836/3519222898,v1:172.21.15.167:6837/3519222898] up:rejoin 2024-07-26T13:47:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:33 smithi167 ceph-mon[37140]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=c=up:rejoin} 2 up:standby-replay 7 up:standby 2024-07-26T13:47:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:33 smithi167 ceph-mon[37140]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-07-26T13:47:33.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:33 smithi167 ceph-mon[37140]: pgmap v593: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 1.7 MiB/s wr, 65 op/s 2024-07-26T13:47:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:33 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.167:6836/3519222898,v1:172.21.15.167:6837/3519222898] up:rejoin 2024-07-26T13:47:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:33 smithi138 ceph-mon[36954]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=c=up:rejoin} 2 up:standby-replay 7 up:standby 2024-07-26T13:47:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:33 smithi138 ceph-mon[36954]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-07-26T13:47:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:33 smithi138 ceph-mon[36954]: pgmap v593: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 1.7 MiB/s wr, 65 op/s 2024-07-26T13:47:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:33 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.167:6836/3519222898,v1:172.21.15.167:6837/3519222898] up:rejoin 2024-07-26T13:47:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:33 smithi100 ceph-mon[32599]: fsmap cephfs:3/3 {0=i=up:active,1=b=up:active,2=c=up:rejoin} 2 up:standby-replay 7 up:standby 2024-07-26T13:47:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:33 smithi100 ceph-mon[32599]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-07-26T13:47:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:33 smithi100 ceph-mon[32599]: pgmap v593: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 1.7 MiB/s wr, 65 op/s 2024-07-26T13:47:34.425 INFO:tasks.workunit.client.0.smithi100.stdout: 1 294723 3.52 MB/sec execute 228 sec latency 22.548 ms 2024-07-26T13:47:34.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:34 smithi167 ceph-mon[37140]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-26T13:47:34.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:34 smithi167 ceph-mon[37140]: Cluster is now healthy 2024-07-26T13:47:34.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:34 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.167:6836/3519222898,v1:172.21.15.167:6837/3519222898] up:active 2024-07-26T13:47:34.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:34 smithi167 ceph-mon[37140]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:47:34.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:34 smithi167 ceph-mon[37140]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:47:34.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:34 smithi138 ceph-mon[36954]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-26T13:47:34.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:34 smithi138 ceph-mon[36954]: Cluster is now healthy 2024-07-26T13:47:34.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:34 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.167:6836/3519222898,v1:172.21.15.167:6837/3519222898] up:active 2024-07-26T13:47:34.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:34 smithi138 ceph-mon[36954]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:47:34.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:34 smithi138 ceph-mon[36954]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:47:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:34 smithi100 ceph-mon[32599]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-26T13:47:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:34 smithi100 ceph-mon[32599]: Cluster is now healthy 2024-07-26T13:47:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:34 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.167:6836/3519222898,v1:172.21.15.167:6837/3519222898] up:active 2024-07-26T13:47:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:34 smithi100 ceph-mon[32599]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:47:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:34 smithi100 ceph-mon[32599]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:47:35.425 INFO:tasks.workunit.client.0.smithi100.stdout: 1 296693 3.53 MB/sec execute 229 sec latency 22.603 ms 2024-07-26T13:47:35.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:35 smithi138 ceph-mon[36954]: pgmap v594: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 77 op/s 2024-07-26T13:47:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:35 smithi100 ceph-mon[32599]: pgmap v594: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 77 op/s 2024-07-26T13:47:35.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:35 smithi167 ceph-mon[37140]: pgmap v594: 97 pgs: 97 active+clean; 110 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 77 op/s 2024-07-26T13:47:35.772 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:47:35 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 2024-07-26T13:47:35.283+0000 7f740ace1640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-07-26T13:47:36.425 INFO:tasks.workunit.client.0.smithi100.stdout: 1 298435 3.57 MB/sec execute 230 sec latency 25.260 ms 2024-07-26T13:47:37.426 INFO:tasks.workunit.client.0.smithi100.stdout: 1 300890 3.61 MB/sec execute 231 sec latency 18.375 ms 2024-07-26T13:47:37.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:37 smithi138 ceph-mon[36954]: pgmap v595: 97 pgs: 97 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.5 MiB/s wr, 96 op/s 2024-07-26T13:47:37.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:37 smithi100 ceph-mon[32599]: pgmap v595: 97 pgs: 97 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.5 MiB/s wr, 96 op/s 2024-07-26T13:47:38.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:37 smithi167 ceph-mon[37140]: pgmap v595: 97 pgs: 97 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.5 MiB/s wr, 96 op/s 2024-07-26T13:47:38.426 INFO:tasks.workunit.client.0.smithi100.stdout: 1 302594 3.63 MB/sec execute 232 sec latency 11.689 ms 2024-07-26T13:47:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:38 smithi138 ceph-mon[36954]: pgmap v596: 97 pgs: 97 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.1 MiB/s wr, 178 op/s 2024-07-26T13:47:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:38 smithi100 ceph-mon[32599]: pgmap v596: 97 pgs: 97 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.1 MiB/s wr, 178 op/s 2024-07-26T13:47:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:38 smithi167 ceph-mon[37140]: pgmap v596: 97 pgs: 97 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.1 MiB/s wr, 178 op/s 2024-07-26T13:47:39.426 INFO:tasks.workunit.client.0.smithi100.stdout: 1 305225 3.69 MB/sec execute 233 sec latency 15.373 ms 2024-07-26T13:47:40.105 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-26T13:47:40.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:39 smithi138 ceph-mon[36954]: scrub summary: idle 2024-07-26T13:47:40.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:39 smithi100 ceph-mon[32599]: scrub summary: idle 2024-07-26T13:47:40.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:39 smithi167 ceph-mon[37140]: scrub summary: idle 2024-07-26T13:47:40.372 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:47:40.426 INFO:tasks.workunit.client.0.smithi100.stdout: 1 306760 3.70 MB/sec execute 234 sec latency 18.399 ms 2024-07-26T13:47:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:40 smithi138 ceph-mon[36954]: scrub summary: idle 2024-07-26T13:47:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:40 smithi138 ceph-mon[36954]: pgmap v597: 97 pgs: 97 active+clean; 116 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.5 MiB/s wr, 222 op/s 2024-07-26T13:47:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:40 smithi100 ceph-mon[32599]: scrub summary: idle 2024-07-26T13:47:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:40 smithi100 ceph-mon[32599]: pgmap v597: 97 pgs: 97 active+clean; 116 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.5 MiB/s wr, 222 op/s 2024-07-26T13:47:41.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:40 smithi167 ceph-mon[37140]: scrub summary: idle 2024-07-26T13:47:41.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:40 smithi167 ceph-mon[37140]: pgmap v597: 97 pgs: 97 active+clean; 116 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.5 MiB/s wr, 222 op/s 2024-07-26T13:47:41.426 INFO:tasks.workunit.client.0.smithi100.stdout: 1 309048 3.74 MB/sec execute 235 sec latency 13.198 ms 2024-07-26T13:47:42.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.250+0000 7fc73a63a640 1 -- 172.21.15.100:0/3692561829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7341042d0 msgr2=0x7fc7341046e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:42.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.250+0000 7fc73a63a640 1 --2- 172.21.15.100:0/3692561829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7341042d0 0x7fc7341046e0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fc72c009ca0 tx=0x7fc72c02a8c0 comp rx=0 tx=0).stop 2024-07-26T13:47:42.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.250+0000 7fc73a63a640 1 -- 172.21.15.100:0/3692561829 shutdown_connections 2024-07-26T13:47:42.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.250+0000 7fc73a63a640 1 --2- 172.21.15.100:0/3692561829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc73410d880 0x7fc73410fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:42.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.250+0000 7fc73a63a640 1 --2- 172.21.15.100:0/3692561829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc734104d00 0x7fc73410d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:42.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.250+0000 7fc73a63a640 1 --2- 172.21.15.100:0/3692561829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc7341042d0 0x7fc7341046e0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:42.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.250+0000 7fc73a63a640 1 -- 172.21.15.100:0/3692561829 >> 172.21.15.100:0/3692561829 conn(0x7fc734079160 msgr2=0x7fc734079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:42.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.250+0000 7fc73a63a640 1 -- 172.21.15.100:0/3692561829 shutdown_connections 2024-07-26T13:47:42.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.250+0000 7fc73a63a640 1 -- 172.21.15.100:0/3692561829 wait complete. 2024-07-26T13:47:42.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.251+0000 7fc73a63a640 1 Processor -- start 2024-07-26T13:47:42.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.251+0000 7fc73a63a640 1 -- start start 2024-07-26T13:47:42.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.251+0000 7fc73a63a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7341042d0 0x7fc734199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:42.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.251+0000 7fc73a63a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc734104d00 0x7fc73419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:42.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.251+0000 7fc73a63a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc73410d880 0x7fc73419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:42.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.251+0000 7fc73a63a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc7341134d0 con 0x7fc73410d880 2024-07-26T13:47:42.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.251+0000 7fc73a63a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc734113350 con 0x7fc7341042d0 2024-07-26T13:47:42.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.251+0000 7fc73a63a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc734113650 con 0x7fc734104d00 2024-07-26T13:47:42.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.252+0000 7fc7337fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc734104d00 0x7fc73419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:42.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.252+0000 7fc738bd7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc73410d880 0x7fc73419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:42.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.252+0000 7fc7337fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc734104d00 0x7fc73419a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:34514/0 (socket says 172.21.15.100:34514) 2024-07-26T13:47:42.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.252+0000 7fc7337fe640 1 -- 172.21.15.100:0/986473088 learned_addr learned my addr 172.21.15.100:0/986473088 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:42.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.252+0000 7fc733fff640 1 --2- 172.21.15.100:0/986473088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7341042d0 0x7fc734199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:42.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.252+0000 7fc7337fe640 1 -- 172.21.15.100:0/986473088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7341042d0 msgr2=0x7fc734199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:42.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.252+0000 7fc7337fe640 1 --2- 172.21.15.100:0/986473088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7341042d0 0x7fc734199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:42.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.252+0000 7fc7337fe640 1 -- 172.21.15.100:0/986473088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc73410d880 msgr2=0x7fc73419f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:42.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.252+0000 7fc7337fe640 1 --2- 172.21.15.100:0/986473088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc73410d880 0x7fc73419f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:42.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.252+0000 7fc7337fe640 1 -- 172.21.15.100:0/986473088 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc73419fe10 con 0x7fc734104d00 2024-07-26T13:47:42.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.252+0000 7fc738bd7640 1 --2- 172.21.15.100:0/986473088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc73410d880 0x7fc73419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:47:42.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.252+0000 7fc7337fe640 1 --2- 172.21.15.100:0/986473088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc734104d00 0x7fc73419a440 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc72800e9c0 tx=0x7fc72800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:42.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.252+0000 7fc7317fa640 1 -- 172.21.15.100:0/986473088 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72800cd80 con 0x7fc734104d00 2024-07-26T13:47:42.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.253+0000 7fc73a63a640 1 -- 172.21.15.100:0/986473088 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7341a0100 con 0x7fc734104d00 2024-07-26T13:47:42.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.253+0000 7fc7317fa640 1 -- 172.21.15.100:0/986473088 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7280054d0 con 0x7fc734104d00 2024-07-26T13:47:42.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.253+0000 7fc7317fa640 1 -- 172.21.15.100:0/986473088 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc728010430 con 0x7fc734104d00 2024-07-26T13:47:42.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.253+0000 7fc73a63a640 1 -- 172.21.15.100:0/986473088 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7341a6860 con 0x7fc734104d00 2024-07-26T13:47:42.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.255+0000 7fc740bbe740 1 Processor -- start 2024-07-26T13:47:42.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.255+0000 7fc740bbe740 1 -- start start 2024-07-26T13:47:42.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.255+0000 7fc7317fa640 1 -- 172.21.15.100:0/986473088 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fc7280105d0 con 0x7fc734104d00 2024-07-26T13:47:42.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.255+0000 7fc740bbe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564546923070 0x564546cc8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:42.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.255+0000 7fc740bbe740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645468fa1b0 0x564546cc8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:42.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.255+0000 7fc740bbe740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564546902390 0x564546cc8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:42.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.255+0000 7fc740bbe740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x564546c3eb90 con 0x5645468fa1b0 2024-07-26T13:47:42.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.255+0000 7fc740bbe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564546c2e090 con 0x564546923070 2024-07-26T13:47:42.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.255+0000 7fc740bbe740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564546c320d0 con 0x564546902390 2024-07-26T13:47:42.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.256+0000 7fc733fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645468fa1b0 0x564546cc8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:42.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.256+0000 7fc733fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645468fa1b0 0x564546cc8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38524/0 (socket says 172.21.15.100:38524) 2024-07-26T13:47:42.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.256+0000 7fc733fff640 1 -- 172.21.15.100:0/344129350 learned_addr learned my addr 172.21.15.100:0/344129350 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:42.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.256+0000 7fc738bd7640 1 --2- 172.21.15.100:0/344129350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564546923070 0x564546cc8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:42.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.256+0000 7fc7337fe640 1 --2- 172.21.15.100:0/344129350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564546902390 0x564546cc8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:42.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.256+0000 7fc733fff640 1 -- 172.21.15.100:0/344129350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564546902390 msgr2=0x564546cc8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:42.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.256+0000 7fc733fff640 1 --2- 172.21.15.100:0/344129350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564546902390 0x564546cc8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:42.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.256+0000 7fc733fff640 1 -- 172.21.15.100:0/344129350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564546923070 msgr2=0x564546cc8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:42.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.256+0000 7fc733fff640 1 --2- 172.21.15.100:0/344129350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564546923070 0x564546cc8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:42.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.256+0000 7fc733fff640 1 -- 172.21.15.100:0/344129350 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564546bd4b70 con 0x5645468fa1b0 2024-07-26T13:47:42.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.256+0000 7fc7337fe640 1 --2- 172.21.15.100:0/344129350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564546902390 0x564546cc8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:47:42.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.256+0000 7fc733fff640 1 --2- 172.21.15.100:0/344129350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645468fa1b0 0x564546cc8a90 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fc73406b5d0 tx=0x7fc72c009a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602edba8c7f4631c server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:42.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.256+0000 7fc71a7fc640 1 -- 172.21.15.100:0/344129350 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c031cf0 con 0x5645468fa1b0 2024-07-26T13:47:42.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.256+0000 7fc7317fa640 1 --2- 172.21.15.100:0/986473088 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc71c0b3d40 0x7fc71c0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:42.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.256+0000 7fc71a7fc640 1 -- 172.21.15.100:0/344129350 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc72c036030 con 0x5645468fa1b0 2024-07-26T13:47:42.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.256+0000 7fc71a7fc640 1 -- 172.21.15.100:0/344129350 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c0042a0 con 0x5645468fa1b0 2024-07-26T13:47:42.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.257+0000 7fc738bd7640 1 --2- 172.21.15.100:0/986473088 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc71c0b3d40 0x7fc71c0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:42.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.257+0000 7fc7317fa640 1 -- 172.21.15.100:0/986473088 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8828+0+0 (secure 0 0 0) 0x7fc7280e4570 con 0x7fc734104d00 2024-07-26T13:47:42.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.257+0000 7fc740bbe740 1 -- 172.21.15.100:0/344129350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645468fa1b0 msgr2=0x564546cc8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:42.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.257+0000 7fc738bd7640 1 --2- 172.21.15.100:0/986473088 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc71c0b3d40 0x7fc71c0b6250 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc724012640 tx=0x7fc724005f90 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:42.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.257+0000 7fc740bbe740 1 --2- 172.21.15.100:0/344129350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645468fa1b0 0x564546cc8a90 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fc73406b5d0 tx=0x7fc72c009a10 comp rx=0 tx=0).stop 2024-07-26T13:47:42.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.257+0000 7fc740bbe740 1 -- 172.21.15.100:0/344129350 shutdown_connections 2024-07-26T13:47:42.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.257+0000 7fc740bbe740 1 --2- 172.21.15.100:0/344129350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564546902390 0x564546cc8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:42.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.257+0000 7fc740bbe740 1 --2- 172.21.15.100:0/344129350 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645468fa1b0 0x564546cc8a90 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:42.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.257+0000 7fc740bbe740 1 --2- 172.21.15.100:0/344129350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564546923070 0x564546cc8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:42.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.257+0000 7fc740bbe740 1 -- 172.21.15.100:0/344129350 >> 172.21.15.100:0/344129350 conn(0x564546917460 msgr2=0x564546cc6440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:42.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.258+0000 7fc740bbe740 1 -- 172.21.15.100:0/344129350 shutdown_connections 2024-07-26T13:47:42.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.258+0000 7fc740bbe740 1 -- 172.21.15.100:0/344129350 wait complete. 2024-07-26T13:47:42.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.258+0000 7fc740bbe740 1 -- start start 2024-07-26T13:47:42.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.258+0000 7fc740bbe740 1 Processor -- start 2024-07-26T13:47:42.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.258+0000 7fc740bbe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645468fa1b0 0x564546cd4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:42.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.258+0000 7fc740bbe740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564546902390 0x564546cd45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:42.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.258+0000 7fc740bbe740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564546923070 0x564546cdfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:42.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.258+0000 7fc740bbe740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x564546a91700 con 0x564546902390 2024-07-26T13:47:42.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.258+0000 7fc740bbe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564546bd09c0 con 0x5645468fa1b0 2024-07-26T13:47:42.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.258+0000 7fc740bbe740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564546cd6b50 con 0x564546923070 2024-07-26T13:47:42.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.259+0000 7fc740bbe740 10 client.0 _ll_register_callbacks cb 0x564546beb4d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:47:42.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.259+0000 7fc7337fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564546902390 0x564546cd45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:42.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.259+0000 7fc7337fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564546902390 0x564546cd45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38526/0 (socket says 172.21.15.100:38526) 2024-07-26T13:47:42.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.259+0000 7fc7337fe640 1 -- 172.21.15.100:0/1728816355 learned_addr learned my addr 172.21.15.100:0/1728816355 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:42.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.259+0000 7fc738bd7640 1 --2- 172.21.15.100:0/1728816355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564546923070 0x564546cdfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:42.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.259+0000 7fc733fff640 1 --2- 172.21.15.100:0/1728816355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645468fa1b0 0x564546cd4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:42.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.259+0000 7fc7337fe640 1 -- 172.21.15.100:0/1728816355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564546923070 msgr2=0x564546cdfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:42.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.259+0000 7fc7337fe640 1 --2- 172.21.15.100:0/1728816355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564546923070 0x564546cdfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:42.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.259+0000 7fc7337fe640 1 -- 172.21.15.100:0/1728816355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645468fa1b0 msgr2=0x564546cd4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:42.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.259+0000 7fc7337fe640 1 --2- 172.21.15.100:0/1728816355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645468fa1b0 0x564546cd4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:42.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.259+0000 7fc7337fe640 1 -- 172.21.15.100:0/1728816355 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56454698e650 con 0x564546902390 2024-07-26T13:47:42.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.259+0000 7fc738bd7640 1 --2- 172.21.15.100:0/1728816355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564546923070 0x564546cdfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:47:42.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.259+0000 7fc7337fe640 1 --2- 172.21.15.100:0/1728816355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564546902390 0x564546cd45b0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fc7280aafe0 tx=0x7fc7280a6fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb5391d1c9e02c6 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:42.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.259+0000 7fc7137fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:47:42.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.259+0000 7fc7137fe640 1 -- 172.21.15.100:0/1728816355 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7280046c0 con 0x564546902390 2024-07-26T13:47:42.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.259+0000 7fc7137fe640 1 -- 172.21.15.100:0/1728816355 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8828+0+0 (secure 0 0 0) 0x7fc7280b3e80 con 0x564546902390 2024-07-26T13:47:42.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.259+0000 7fc740bbe740 1 -- 172.21.15.100:0/1728816355 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc72c036030 con 0x564546902390 2024-07-26T13:47:42.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.259+0000 7fc7137fe640 1 -- 172.21.15.100:0/1728816355 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fc7280aab60 con 0x564546902390 2024-07-26T13:47:42.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.259+0000 7fc740bbe740 10 client.17364 fetch_fsmap learned FSMap version 61 2024-07-26T13:47:42.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.259+0000 7fc740bbe740 1 -- 172.21.15.100:0/1728816355 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x564546ce2b30 con 0x564546902390 2024-07-26T13:47:42.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.260+0000 7fc7137fe640 1 -- 172.21.15.100:0/1728816355 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc72c036030 con 0x564546902390 2024-07-26T13:47:42.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.260+0000 7fc7137fe640 1 -- 172.21.15.100:0/1728816355 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 61) ==== 8346+0+0 (secure 0 0 0) 0x7fc7280c4030 con 0x564546902390 2024-07-26T13:47:42.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.260+0000 7fc740bbe740 10 client.17364 fetch_fsmap finished waiting for FSMap version 61 2024-07-26T13:47:42.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.260+0000 7fc740bbe740 10 client.17364 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:47:42.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.260+0000 7fc740bbe740 20 client.17364 populate_metadata read hostname 'smithi100' 2024-07-26T13:47:42.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.260+0000 7fc740bbe740 1 --2- 172.21.15.100:0/1728816355 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5645468e19f0 0x564546ce60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:42.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.260+0000 7fc740bbe740 4 client.17364 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:47:42.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.260+0000 7fc740bbe740 1 -- 172.21.15.100:0/1728816355 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564546c40ae0 con 0x5645468e19f0 2024-07-26T13:47:42.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.260+0000 7fc733fff640 1 --2- 172.21.15.100:0/1728816355 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5645468e19f0 0x564546ce60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:42.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.261+0000 7fc7137fe640 10 client.17364 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:42.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.261+0000 7fc733fff640 1 --2- 172.21.15.100:0/1728816355 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5645468e19f0 0x564546ce60a0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3abca23c818c9a89 server_cookie=6c0a067cdc398e33 in_seq=0 out_seq=0 2024-07-26T13:47:42.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.262+0000 7fc7137fe640 1 -- 172.21.15.100:0/1728816355 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564546c40ae0 con 0x5645468e19f0 2024-07-26T13:47:42.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.262+0000 7fc7137fe640 10 client.17364 handle_command_reply: tid=0 2024-07-26T13:47:42.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.262+0000 7fc740bbe740 1 client.17364 shutdown 2024-07-26T13:47:42.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.262+0000 7fc740bbe740 1 -- 172.21.15.100:0/1728816355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564546902390 msgr2=0x564546cd45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:42.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.262+0000 7fc740bbe740 1 --2- 172.21.15.100:0/1728816355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564546902390 0x564546cd45b0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fc7280aafe0 tx=0x7fc7280a6fd0 comp rx=0 tx=0).stop 2024-07-26T13:47:42.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.262+0000 7fc740bbe740 1 -- 172.21.15.100:0/1728816355 shutdown_connections 2024-07-26T13:47:42.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.262+0000 7fc740bbe740 1 --2- 172.21.15.100:0/1728816355 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5645468e19f0 0x564546ce60a0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:42.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.262+0000 7fc740bbe740 1 --2- 172.21.15.100:0/1728816355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564546923070 0x564546cdfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:42.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.262+0000 7fc740bbe740 1 --2- 172.21.15.100:0/1728816355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564546902390 0x564546cd45b0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:42.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.262+0000 7fc740bbe740 1 --2- 172.21.15.100:0/1728816355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5645468fa1b0 0x564546cd4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:42.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.262+0000 7fc740bbe740 1 -- 172.21.15.100:0/1728816355 >> 172.21.15.100:0/1728816355 conn(0x564546917460 msgr2=0x564546cc4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:42.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.262+0000 7fc7137fe640 0 client.17364 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:42.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.263+0000 7fc740bbe740 1 -- 172.21.15.100:0/1728816355 shutdown_connections 2024-07-26T13:47:42.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.263+0000 7fc740bbe740 1 -- 172.21.15.100:0/1728816355 wait complete. 2024-07-26T13:47:42.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.263+0000 7fc740bbe740 20 client.17364 trim_cache size 0 max 16384 2024-07-26T13:47:42.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.280+0000 7fc740bbe740 1 Processor -- start 2024-07-26T13:47:42.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.280+0000 7fc740bbe740 1 -- start start 2024-07-26T13:47:42.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.280+0000 7fc740bbe740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645468fa1b0 0x564546cce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:42.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.280+0000 7fc740bbe740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564546902390 0x564546cd5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:42.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.280+0000 7fc740bbe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564546923070 0x564546cdfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:42.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.280+0000 7fc740bbe740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc7341b1c30 con 0x564546902390 2024-07-26T13:47:42.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.280+0000 7fc740bbe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc7341af500 con 0x564546923070 2024-07-26T13:47:42.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.280+0000 7fc740bbe740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc7341ad350 con 0x5645468fa1b0 2024-07-26T13:47:42.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.281+0000 7fc7337fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564546902390 0x564546cd5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:42.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.281+0000 7fc7337fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564546902390 0x564546cd5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38536/0 (socket says 172.21.15.100:38536) 2024-07-26T13:47:42.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.281+0000 7fc733fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645468fa1b0 0x564546cce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:42.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.281+0000 7fc7337fe640 1 -- 172.21.15.100:0/863687885 learned_addr learned my addr 172.21.15.100:0/863687885 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:42.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.281+0000 7fc738bd7640 1 --2- 172.21.15.100:0/863687885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564546923070 0x564546cdfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:42.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.281+0000 7fc7337fe640 1 -- 172.21.15.100:0/863687885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645468fa1b0 msgr2=0x564546cce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:42.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.281+0000 7fc7337fe640 1 --2- 172.21.15.100:0/863687885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645468fa1b0 0x564546cce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:42.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.281+0000 7fc7337fe640 1 -- 172.21.15.100:0/863687885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564546923070 msgr2=0x564546cdfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:42.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.281+0000 7fc7337fe640 1 --2- 172.21.15.100:0/863687885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564546923070 0x564546cdfc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:42.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.281+0000 7fc7337fe640 1 -- 172.21.15.100:0/863687885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564546cd4920 con 0x564546902390 2024-07-26T13:47:42.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.281+0000 7fc738bd7640 1 --2- 172.21.15.100:0/863687885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564546923070 0x564546cdfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:47:42.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.281+0000 7fc7337fe640 1 --2- 172.21.15.100:0/863687885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564546902390 0x564546cd5aa0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fc7280aad00 tx=0x7fc7280048b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69a82f039ad6175 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:42.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.281+0000 7fc719ffb640 1 -- 172.21.15.100:0/863687885 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7280aa9a0 con 0x564546902390 2024-07-26T13:47:42.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.281+0000 7fc719ffb640 1 -- 172.21.15.100:0/863687885 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7280046c0 con 0x564546902390 2024-07-26T13:47:42.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.281+0000 7fc719ffb640 1 -- 172.21.15.100:0/863687885 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7280af650 con 0x564546902390 2024-07-26T13:47:42.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.282+0000 7fc740bbe740 1 -- 172.21.15.100:0/863687885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564546902390 msgr2=0x564546cd5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:42.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.282+0000 7fc740bbe740 1 --2- 172.21.15.100:0/863687885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564546902390 0x564546cd5aa0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fc7280aad00 tx=0x7fc7280048b0 comp rx=0 tx=0).stop 2024-07-26T13:47:42.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.282+0000 7fc740bbe740 1 -- 172.21.15.100:0/863687885 shutdown_connections 2024-07-26T13:47:42.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.282+0000 7fc740bbe740 1 --2- 172.21.15.100:0/863687885 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564546923070 0x564546cdfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:42.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.282+0000 7fc740bbe740 1 --2- 172.21.15.100:0/863687885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x564546902390 0x564546cd5aa0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:42.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.282+0000 7fc740bbe740 1 --2- 172.21.15.100:0/863687885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5645468fa1b0 0x564546cce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:42.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.282+0000 7fc740bbe740 1 -- 172.21.15.100:0/863687885 >> 172.21.15.100:0/863687885 conn(0x564546917460 msgr2=0x564546cd5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:42.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.282+0000 7fc740bbe740 1 -- 172.21.15.100:0/863687885 shutdown_connections 2024-07-26T13:47:42.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.282+0000 7fc740bbe740 1 -- 172.21.15.100:0/863687885 wait complete. 2024-07-26T13:47:42.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.283+0000 7fc740bbe740 1 -- start start 2024-07-26T13:47:42.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.283+0000 7fc740bbe740 1 Processor -- start 2024-07-26T13:47:42.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.283+0000 7fc740bbe740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645468fa1b0 0x564546cd75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:42.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.283+0000 7fc740bbe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564546902390 0x564546cd7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:42.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.283+0000 7fc740bbe740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564546923070 0x564546d4c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:42.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.283+0000 7fc740bbe740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc7341ad9c0 con 0x5645468fa1b0 2024-07-26T13:47:42.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.283+0000 7fc740bbe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc7341aee30 con 0x564546902390 2024-07-26T13:47:42.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.283+0000 7fc740bbe740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc7341ae090 con 0x564546923070 2024-07-26T13:47:42.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.283+0000 7fc740bbe740 10 client.0 _ll_register_callbacks cb 0x7fc72800cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:47:42.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.283+0000 7fc733fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645468fa1b0 0x564546cd75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:42.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.283+0000 7fc733fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645468fa1b0 0x564546cd75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38542/0 (socket says 172.21.15.100:38542) 2024-07-26T13:47:42.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.283+0000 7fc733fff640 1 -- 172.21.15.100:0/2335903603 learned_addr learned my addr 172.21.15.100:0/2335903603 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:42.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.283+0000 7fc7337fe640 1 --2- 172.21.15.100:0/2335903603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564546902390 0x564546cd7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:42.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.283+0000 7fc738bd7640 1 --2- 172.21.15.100:0/2335903603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564546923070 0x564546d4c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:42.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.284+0000 7fc733fff640 1 -- 172.21.15.100:0/2335903603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564546923070 msgr2=0x564546d4c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:42.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.284+0000 7fc733fff640 1 --2- 172.21.15.100:0/2335903603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564546923070 0x564546d4c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:42.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.284+0000 7fc733fff640 1 -- 172.21.15.100:0/2335903603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564546902390 msgr2=0x564546cd7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:42.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.284+0000 7fc733fff640 1 --2- 172.21.15.100:0/2335903603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564546902390 0x564546cd7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:42.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.284+0000 7fc733fff640 1 -- 172.21.15.100:0/2335903603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564546d4cdc0 con 0x5645468fa1b0 2024-07-26T13:47:42.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.284+0000 7fc7337fe640 1 --2- 172.21.15.100:0/2335903603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564546902390 0x564546cd7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:47:42.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.284+0000 7fc733fff640 1 --2- 172.21.15.100:0/2335903603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645468fa1b0 0x564546cd75d0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fc72c009950 tx=0x7fc72c0098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926e95616a6a746e server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:42.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.284+0000 7fc7137fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:47:42.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.284+0000 7fc7137fe640 1 -- 172.21.15.100:0/2335903603 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c060060 con 0x5645468fa1b0 2024-07-26T13:47:42.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.284+0000 7fc7137fe640 1 -- 172.21.15.100:0/2335903603 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8828+0+0 (secure 0 0 0) 0x7fc72c031cf0 con 0x5645468fa1b0 2024-07-26T13:47:42.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.284+0000 7fc740bbe740 1 -- 172.21.15.100:0/2335903603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564546c3c580 con 0x5645468fa1b0 2024-07-26T13:47:42.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.284+0000 7fc7137fe640 1 -- 172.21.15.100:0/2335903603 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fc72c04a040 con 0x5645468fa1b0 2024-07-26T13:47:42.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.284+0000 7fc740bbe740 10 client.17370 fetch_fsmap learned FSMap version 61 2024-07-26T13:47:42.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.284+0000 7fc740bbe740 1 -- 172.21.15.100:0/2335903603 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x564546d4e050 con 0x5645468fa1b0 2024-07-26T13:47:42.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.284+0000 7fc7137fe640 1 -- 172.21.15.100:0/2335903603 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564546c3c580 con 0x5645468fa1b0 2024-07-26T13:47:42.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.285+0000 7fc7137fe640 1 -- 172.21.15.100:0/2335903603 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 61) ==== 8346+0+0 (secure 0 0 0) 0x7fc72c043070 con 0x5645468fa1b0 2024-07-26T13:47:42.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.285+0000 7fc740bbe740 10 client.17370 fetch_fsmap finished waiting for FSMap version 61 2024-07-26T13:47:42.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.285+0000 7fc740bbe740 10 client.17370 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:47:42.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.285+0000 7fc740bbe740 20 client.17370 populate_metadata read hostname 'smithi100' 2024-07-26T13:47:42.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.285+0000 7fc740bbe740 1 --2- 172.21.15.100:0/2335903603 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5645468b6da0 0x564546d50560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:42.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.285+0000 7fc740bbe740 4 client.17370 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-26T13:47:42.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.285+0000 7fc740bbe740 1 -- 172.21.15.100:0/2335903603 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564546c39cc0 con 0x5645468b6da0 2024-07-26T13:47:42.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.285+0000 7fc7337fe640 1 --2- 172.21.15.100:0/2335903603 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5645468b6da0 0x564546d50560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:42.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.285+0000 7fc7137fe640 10 client.17370 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:42.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.285+0000 7fc7337fe640 1 --2- 172.21.15.100:0/2335903603 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5645468b6da0 0x564546d50560 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=226ccf46218562d8 server_cookie=d7a43926debae7f5 in_seq=0 out_seq=0 2024-07-26T13:47:42.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.286+0000 7fc7137fe640 1 -- 172.21.15.100:0/2335903603 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564546c39cc0 con 0x5645468b6da0 2024-07-26T13:47:42.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.286+0000 7fc7137fe640 10 client.17370 handle_command_reply: tid=0 2024-07-26T13:47:42.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.286+0000 7fc740bbe740 1 client.17370 shutdown 2024-07-26T13:47:42.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.286+0000 7fc740bbe740 1 -- 172.21.15.100:0/2335903603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645468fa1b0 msgr2=0x564546cd75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:42.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.286+0000 7fc740bbe740 1 --2- 172.21.15.100:0/2335903603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645468fa1b0 0x564546cd75d0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fc72c009950 tx=0x7fc72c0098e0 comp rx=0 tx=0).stop 2024-07-26T13:47:42.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.286+0000 7fc740bbe740 1 -- 172.21.15.100:0/2335903603 shutdown_connections 2024-07-26T13:47:42.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.286+0000 7fc740bbe740 1 --2- 172.21.15.100:0/2335903603 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5645468b6da0 0x564546d50560 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:42.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.286+0000 7fc740bbe740 1 --2- 172.21.15.100:0/2335903603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564546923070 0x564546d4c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:42.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.286+0000 7fc7137fe640 0 client.17370 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:42.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.286+0000 7fc740bbe740 1 --2- 172.21.15.100:0/2335903603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564546902390 0x564546cd7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:42.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.286+0000 7fc740bbe740 1 --2- 172.21.15.100:0/2335903603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5645468fa1b0 0x564546cd75d0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:42.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.286+0000 7fc740bbe740 1 -- 172.21.15.100:0/2335903603 >> 172.21.15.100:0/2335903603 conn(0x564546917460 msgr2=0x564546cd5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:42.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.286+0000 7fc740bbe740 1 -- 172.21.15.100:0/2335903603 shutdown_connections 2024-07-26T13:47:42.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.286+0000 7fc740bbe740 1 -- 172.21.15.100:0/2335903603 wait complete. 2024-07-26T13:47:42.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.286+0000 7fc740bbe740 20 client.17370 trim_cache size 0 max 16384 2024-07-26T13:47:42.296 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:47:42.296 INFO:teuthology.orchestra.run.smithi100.stdout: "status": "no active scrubs running", 2024-07-26T13:47:42.296 INFO:teuthology.orchestra.run.smithi100.stdout: "scrubs": {} 2024-07-26T13:47:42.296 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:47:42.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.287+0000 7fc73a63a640 1 -- 172.21.15.100:0/986473088 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc71c0b3d40 msgr2=0x7fc71c0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:42.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.287+0000 7fc73a63a640 1 --2- 172.21.15.100:0/986473088 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc71c0b3d40 0x7fc71c0b6250 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc724012640 tx=0x7fc724005f90 comp rx=0 tx=0).stop 2024-07-26T13:47:42.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.287+0000 7fc73a63a640 1 -- 172.21.15.100:0/986473088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc734104d00 msgr2=0x7fc73419a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:42.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.287+0000 7fc73a63a640 1 --2- 172.21.15.100:0/986473088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc734104d00 0x7fc73419a440 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc72800e9c0 tx=0x7fc72800ee80 comp rx=0 tx=0).stop 2024-07-26T13:47:42.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.287+0000 7fc73a63a640 1 -- 172.21.15.100:0/986473088 shutdown_connections 2024-07-26T13:47:42.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.287+0000 7fc73a63a640 1 --2- 172.21.15.100:0/986473088 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fc71c0b3d40 0x7fc71c0b6250 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:42.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.287+0000 7fc73a63a640 1 --2- 172.21.15.100:0/986473088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc73410d880 0x7fc73419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:42.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.287+0000 7fc73a63a640 1 --2- 172.21.15.100:0/986473088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc734104d00 0x7fc73419a440 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:42.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.287+0000 7fc73a63a640 1 --2- 172.21.15.100:0/986473088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7341042d0 0x7fc734199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:42.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.287+0000 7fc73a63a640 1 -- 172.21.15.100:0/986473088 >> 172.21.15.100:0/986473088 conn(0x7fc734079160 msgr2=0x7fc7341002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:42.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.287+0000 7fc73a63a640 1 -- 172.21.15.100:0/986473088 shutdown_connections 2024-07-26T13:47:42.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:42.287+0000 7fc73a63a640 1 -- 172.21.15.100:0/986473088 wait complete. 2024-07-26T13:47:42.426 INFO:tasks.workunit.client.0.smithi100.stdout: 1 311068 3.76 MB/sec execute 236 sec latency 20.013 ms 2024-07-26T13:47:43.324 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-26T13:47:43.324 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:47:43.426 INFO:tasks.workunit.client.0.smithi100.stdout: 1 312663 3.79 MB/sec execute 237 sec latency 56.264 ms 2024-07-26T13:47:43.607 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:47:43.874 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:43 smithi100 ceph-mon[32599]: pgmap v598: 97 pgs: 97 active+clean; 116 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.4 MiB/s wr, 214 op/s 2024-07-26T13:47:43.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:43 smithi138 ceph-mon[36954]: pgmap v598: 97 pgs: 97 active+clean; 116 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.4 MiB/s wr, 214 op/s 2024-07-26T13:47:44.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:43 smithi167 ceph-mon[37140]: pgmap v598: 97 pgs: 97 active+clean; 116 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.4 MiB/s wr, 214 op/s 2024-07-26T13:47:44.426 INFO:tasks.workunit.client.0.smithi100.stdout: 1 314763 3.81 MB/sec execute 238 sec latency 17.922 ms 2024-07-26T13:47:44.939 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:44 smithi100 ceph-mon[32599]: pgmap v599: 97 pgs: 97 active+clean; 119 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.5 MiB/s wr, 243 op/s 2024-07-26T13:47:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:44 smithi138 ceph-mon[36954]: pgmap v599: 97 pgs: 97 active+clean; 119 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.5 MiB/s wr, 243 op/s 2024-07-26T13:47:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:44 smithi167 ceph-mon[37140]: pgmap v599: 97 pgs: 97 active+clean; 119 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.5 MiB/s wr, 243 op/s 2024-07-26T13:47:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.386+0000 7f401506f640 1 -- 172.21.15.100:0/895135508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4010102410 msgr2=0x7f4010102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.386+0000 7f401506f640 1 --2- 172.21.15.100:0/895135508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4010102410 0x7f4010102820 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8009900 tx=0x7f3ff802a410 comp rx=0 tx=0).stop 2024-07-26T13:47:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.386+0000 7f401506f640 1 -- 172.21.15.100:0/895135508 shutdown_connections 2024-07-26T13:47:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.386+0000 7f401506f640 1 --2- 172.21.15.100:0/895135508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4010108690 0x7f401010eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.386+0000 7f401506f640 1 --2- 172.21.15.100:0/895135508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4010107d40 0x7f4010108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.386+0000 7f401506f640 1 --2- 172.21.15.100:0/895135508 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4010102410 0x7f4010102820 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.386+0000 7f401506f640 1 -- 172.21.15.100:0/895135508 >> 172.21.15.100:0/895135508 conn(0x7f40100fddf0 msgr2=0x7f4010100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.386+0000 7f401506f640 1 -- 172.21.15.100:0/895135508 shutdown_connections 2024-07-26T13:47:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.386+0000 7f401506f640 1 -- 172.21.15.100:0/895135508 wait complete. 2024-07-26T13:47:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.387+0000 7f401506f640 1 Processor -- start 2024-07-26T13:47:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.387+0000 7f401506f640 1 -- start start 2024-07-26T13:47:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.388+0000 7f401506f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4010102410 0x7f4010198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.388+0000 7f401506f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4010107d40 0x7f4010199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.388+0000 7f401506f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4010108690 0x7f401019e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.388+0000 7f401506f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4010112360 con 0x7f4010102410 2024-07-26T13:47:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.388+0000 7f401506f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f40101121e0 con 0x7f4010108690 2024-07-26T13:47:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.388+0000 7f401506f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f40101124e0 con 0x7f4010107d40 2024-07-26T13:47:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.388+0000 7f400ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4010102410 0x7f4010198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.388+0000 7f400ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4010102410 0x7f4010198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51706/0 (socket says 172.21.15.100:51706) 2024-07-26T13:47:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.388+0000 7f400ed9d640 1 -- 172.21.15.100:0/1817327561 learned_addr learned my addr 172.21.15.100:0/1817327561 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.388+0000 7f400e59c640 1 --2- 172.21.15.100:0/1817327561 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4010107d40 0x7f4010199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.388+0000 7f400f59e640 1 --2- 172.21.15.100:0/1817327561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4010108690 0x7f401019e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.388+0000 7f400ed9d640 1 -- 172.21.15.100:0/1817327561 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4010107d40 msgr2=0x7f4010199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.388+0000 7f400ed9d640 1 --2- 172.21.15.100:0/1817327561 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4010107d40 0x7f4010199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.388+0000 7f400ed9d640 1 -- 172.21.15.100:0/1817327561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4010108690 msgr2=0x7f401019e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.388+0000 7f400ed9d640 1 --2- 172.21.15.100:0/1817327561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4010108690 0x7f401019e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.388+0000 7f400ed9d640 1 -- 172.21.15.100:0/1817327561 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f401019ecf0 con 0x7f4010102410 2024-07-26T13:47:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.388+0000 7f400ed9d640 1 --2- 172.21.15.100:0/1817327561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4010102410 0x7f4010198dc0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8005940 tx=0x7f3ff8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.388+0000 7f4007fff640 1 -- 172.21.15.100:0/1817327561 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff803f030 con 0x7f4010102410 2024-07-26T13:47:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.389+0000 7f401506f640 1 -- 172.21.15.100:0/1817327561 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f401019ef80 con 0x7f4010102410 2024-07-26T13:47:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.389+0000 7f4007fff640 1 -- 172.21.15.100:0/1817327561 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ff8003aa0 con 0x7f4010102410 2024-07-26T13:47:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.389+0000 7f4007fff640 1 -- 172.21.15.100:0/1817327561 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8035070 con 0x7f4010102410 2024-07-26T13:47:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.389+0000 7f401506f640 1 -- 172.21.15.100:0/1817327561 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40101a5770 con 0x7f4010102410 2024-07-26T13:47:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.391+0000 7f401bf88740 1 Processor -- start 2024-07-26T13:47:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.391+0000 7f401bf88740 1 -- start start 2024-07-26T13:47:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.392+0000 7f401bf88740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc6d18a1b0 0x55cc6d52c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.392+0000 7f401bf88740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc6d1612f0 0x55cc6d52cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.392+0000 7f401bf88740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc6d1694d0 0x55cc6d52d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.392+0000 7f401bf88740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55cc6d4a23b0 con 0x55cc6d1612f0 2024-07-26T13:47:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.392+0000 7f401bf88740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cc6d4918b0 con 0x55cc6d18a1b0 2024-07-26T13:47:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.392+0000 7f401bf88740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cc6d4958f0 con 0x55cc6d1694d0 2024-07-26T13:47:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.392+0000 7f400e59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc6d1694d0 0x55cc6d52d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.392+0000 7f400f59e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc6d18a1b0 0x55cc6d52c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.392+0000 7f4007fff640 1 -- 172.21.15.100:0/1817327561 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f3ff8042030 con 0x7f4010102410 2024-07-26T13:47:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.392+0000 7f400e59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc6d1694d0 0x55cc6d52d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:40780/0 (socket says 172.21.15.100:40780) 2024-07-26T13:47:45.401 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.392+0000 7f400e59c640 1 -- 172.21.15.100:0/640175310 learned_addr learned my addr 172.21.15.100:0/640175310 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.392+0000 7f400ed9d640 1 --2- 172.21.15.100:0/640175310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc6d1612f0 0x55cc6d52cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.392+0000 7f400e59c640 1 -- 172.21.15.100:0/640175310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc6d18a1b0 msgr2=0x55cc6d52c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.392+0000 7f400e59c640 1 --2- 172.21.15.100:0/640175310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc6d18a1b0 0x55cc6d52c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.392+0000 7f400e59c640 1 -- 172.21.15.100:0/640175310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc6d1612f0 msgr2=0x55cc6d52cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.392+0000 7f400e59c640 1 --2- 172.21.15.100:0/640175310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc6d1612f0 0x55cc6d52cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.392+0000 7f400e59c640 1 -- 172.21.15.100:0/640175310 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc6d4389a0 con 0x55cc6d1694d0 2024-07-26T13:47:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.392+0000 7f400e59c640 1 --2- 172.21.15.100:0/640175310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc6d1694d0 0x55cc6d52d130 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f400000bca0 tx=0x7f40000107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34f629c4c6889b92 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.393+0000 7f40057fa640 1 -- 172.21.15.100:0/640175310 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400001c680 con 0x55cc6d1694d0 2024-07-26T13:47:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.393+0000 7f40057fa640 1 -- 172.21.15.100:0/640175310 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f400000be50 con 0x55cc6d1694d0 2024-07-26T13:47:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.393+0000 7f40057fa640 1 -- 172.21.15.100:0/640175310 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4000011bc0 con 0x55cc6d1694d0 2024-07-26T13:47:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.393+0000 7f4007fff640 1 --2- 172.21.15.100:0/1817327561 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3ffc0b3dc0 0x7f3ffc0b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.393+0000 7f400f59e640 1 --2- 172.21.15.100:0/1817327561 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3ffc0b3dc0 0x7f3ffc0b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.393+0000 7f4007fff640 1 -- 172.21.15.100:0/1817327561 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8828+0+0 (secure 0 0 0) 0x7f3ff81045b0 con 0x7f4010102410 2024-07-26T13:47:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.394+0000 7f401bf88740 1 -- 172.21.15.100:0/640175310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc6d1694d0 msgr2=0x55cc6d52d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.394+0000 7f401bf88740 1 --2- 172.21.15.100:0/640175310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc6d1694d0 0x55cc6d52d130 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f400000bca0 tx=0x7f40000107a0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.394+0000 7f400f59e640 1 --2- 172.21.15.100:0/1817327561 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3ffc0b3dc0 0x7f3ffc0b62d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4008005fd0 tx=0x7f4008009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.394+0000 7f401bf88740 1 -- 172.21.15.100:0/640175310 shutdown_connections 2024-07-26T13:47:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.394+0000 7f401bf88740 1 --2- 172.21.15.100:0/640175310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc6d1694d0 0x55cc6d52d130 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.394+0000 7f401bf88740 1 --2- 172.21.15.100:0/640175310 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc6d1612f0 0x55cc6d52cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.394+0000 7f401bf88740 1 --2- 172.21.15.100:0/640175310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc6d18a1b0 0x55cc6d52c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.394+0000 7f401bf88740 1 -- 172.21.15.100:0/640175310 >> 172.21.15.100:0/640175310 conn(0x55cc6d17e300 msgr2=0x55cc6d52a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.394+0000 7f401bf88740 1 -- 172.21.15.100:0/640175310 shutdown_connections 2024-07-26T13:47:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.394+0000 7f401bf88740 1 -- 172.21.15.100:0/640175310 wait complete. 2024-07-26T13:47:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.394+0000 7f401bf88740 1 -- start start 2024-07-26T13:47:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.394+0000 7f401bf88740 1 Processor -- start 2024-07-26T13:47:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.395+0000 7f401bf88740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc6d1612f0 0x55cc6d53a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.395+0000 7f401bf88740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc6d1694d0 0x55cc6d53a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.395+0000 7f401bf88740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc6d18a1b0 0x55cc6d5463d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.395+0000 7f401bf88740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55cc6d3cbab0 con 0x55cc6d1694d0 2024-07-26T13:47:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.395+0000 7f401bf88740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cc6d419cf0 con 0x55cc6d1612f0 2024-07-26T13:47:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.395+0000 7f401bf88740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cc6d53d010 con 0x55cc6d18a1b0 2024-07-26T13:47:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.395+0000 7f401bf88740 10 client.0 _ll_register_callbacks cb 0x55cc6d48b3e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:47:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.395+0000 7f400e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc6d1694d0 0x55cc6d53a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.395+0000 7f400e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc6d1694d0 0x55cc6d53a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51728/0 (socket says 172.21.15.100:51728) 2024-07-26T13:47:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.395+0000 7f400e59c640 1 -- 172.21.15.100:0/3660048396 learned_addr learned my addr 172.21.15.100:0/3660048396 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.395+0000 7f400f59e640 1 --2- 172.21.15.100:0/3660048396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc6d18a1b0 0x55cc6d5463d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.395+0000 7f400ed9d640 1 --2- 172.21.15.100:0/3660048396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc6d1612f0 0x55cc6d53a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.395+0000 7f400e59c640 1 -- 172.21.15.100:0/3660048396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc6d18a1b0 msgr2=0x55cc6d5463d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.395+0000 7f400e59c640 1 --2- 172.21.15.100:0/3660048396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc6d18a1b0 0x55cc6d5463d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.395+0000 7f400e59c640 1 -- 172.21.15.100:0/3660048396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc6d1612f0 msgr2=0x55cc6d53a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.395+0000 7f400e59c640 1 --2- 172.21.15.100:0/3660048396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc6d1612f0 0x55cc6d53a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.395+0000 7f400e59c640 1 -- 172.21.15.100:0/3660048396 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc6d1f5790 con 0x55cc6d1694d0 2024-07-26T13:47:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.395+0000 7f400e59c640 1 --2- 172.21.15.100:0/3660048396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc6d1694d0 0x55cc6d53a640 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f4000005780 tx=0x7f400000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850a608d446a63c4 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.395+0000 7f3ff67fc640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:47:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.395+0000 7f3ff67fc640 1 -- 172.21.15.100:0/3660048396 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400002b060 con 0x55cc6d1694d0 2024-07-26T13:47:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.395+0000 7f3ff67fc640 1 -- 172.21.15.100:0/3660048396 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8828+0+0 (secure 0 0 0) 0x7f4000011cd0 con 0x55cc6d1694d0 2024-07-26T13:47:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.395+0000 7f401bf88740 1 -- 172.21.15.100:0/3660048396 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f400000be50 con 0x55cc6d1694d0 2024-07-26T13:47:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.396+0000 7f3ff67fc640 1 -- 172.21.15.100:0/3660048396 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f4000032050 con 0x55cc6d1694d0 2024-07-26T13:47:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.396+0000 7f401bf88740 10 client.17382 fetch_fsmap learned FSMap version 61 2024-07-26T13:47:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.396+0000 7f401bf88740 1 -- 172.21.15.100:0/3660048396 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55cc6d26e620 con 0x55cc6d1694d0 2024-07-26T13:47:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.396+0000 7f3ff67fc640 1 -- 172.21.15.100:0/3660048396 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f400000be50 con 0x55cc6d1694d0 2024-07-26T13:47:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.396+0000 7f3ff67fc640 1 -- 172.21.15.100:0/3660048396 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 61) ==== 8346+0+0 (secure 0 0 0) 0x7f400001a480 con 0x55cc6d1694d0 2024-07-26T13:47:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.396+0000 7f401bf88740 10 client.17382 fetch_fsmap finished waiting for FSMap version 61 2024-07-26T13:47:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.396+0000 7f401bf88740 10 client.17382 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:47:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.396+0000 7f401bf88740 20 client.17382 populate_metadata read hostname 'smithi100' 2024-07-26T13:47:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.396+0000 7f401bf88740 1 --2- 172.21.15.100:0/3660048396 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55cc6d148d80 0x55cc6d54c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.396+0000 7f401bf88740 4 client.17382 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:47:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.396+0000 7f401bf88740 1 -- 172.21.15.100:0/3660048396 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc6d4a4300 con 0x55cc6d148d80 2024-07-26T13:47:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.396+0000 7f400ed9d640 1 --2- 172.21.15.100:0/3660048396 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55cc6d148d80 0x55cc6d54c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.397+0000 7f3ff67fc640 10 client.17382 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.397+0000 7f400ed9d640 1 --2- 172.21.15.100:0/3660048396 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55cc6d148d80 0x55cc6d54c180 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c03138eebe6954 server_cookie=a202a1252d3b3711 in_seq=0 out_seq=0 2024-07-26T13:47:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.398+0000 7f3ff67fc640 1 -- 172.21.15.100:0/3660048396 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cc6d4a4300 con 0x55cc6d148d80 2024-07-26T13:47:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.398+0000 7f3ff67fc640 10 client.17382 handle_command_reply: tid=0 2024-07-26T13:47:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.398+0000 7f401bf88740 1 client.17382 shutdown 2024-07-26T13:47:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.398+0000 7f401bf88740 1 -- 172.21.15.100:0/3660048396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc6d1694d0 msgr2=0x55cc6d53a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.398+0000 7f401bf88740 1 --2- 172.21.15.100:0/3660048396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc6d1694d0 0x55cc6d53a640 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f4000005780 tx=0x7f400000b900 comp rx=0 tx=0).stop 2024-07-26T13:47:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.398+0000 7f401bf88740 1 -- 172.21.15.100:0/3660048396 shutdown_connections 2024-07-26T13:47:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.398+0000 7f401bf88740 1 --2- 172.21.15.100:0/3660048396 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55cc6d148d80 0x55cc6d54c180 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.398+0000 7f401bf88740 1 --2- 172.21.15.100:0/3660048396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc6d18a1b0 0x55cc6d5463d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.398+0000 7f401bf88740 1 --2- 172.21.15.100:0/3660048396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc6d1694d0 0x55cc6d53a640 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.398+0000 7f3ff67fc640 0 client.17382 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.398+0000 7f401bf88740 1 --2- 172.21.15.100:0/3660048396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc6d1612f0 0x55cc6d53a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.398+0000 7f401bf88740 1 -- 172.21.15.100:0/3660048396 >> 172.21.15.100:0/3660048396 conn(0x55cc6d17e300 msgr2=0x55cc6d528c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.398+0000 7f401bf88740 1 -- 172.21.15.100:0/3660048396 shutdown_connections 2024-07-26T13:47:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.398+0000 7f401bf88740 1 -- 172.21.15.100:0/3660048396 wait complete. 2024-07-26T13:47:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.398+0000 7f401bf88740 20 client.17382 trim_cache size 0 max 16384 2024-07-26T13:47:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.415+0000 7f401bf88740 1 Processor -- start 2024-07-26T13:47:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.415+0000 7f401bf88740 1 -- start start 2024-07-26T13:47:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.416+0000 7f401bf88740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc6d148d80 0x55cc6d53a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.416+0000 7f401bf88740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc6d1612f0 0x55cc6d546340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.416+0000 7f401bf88740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc6d1694d0 0x55cc6d546880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.416+0000 7f401bf88740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f40101afa30 con 0x55cc6d148d80 2024-07-26T13:47:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.416+0000 7f401bf88740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f40101ad300 con 0x55cc6d1694d0 2024-07-26T13:47:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.416+0000 7f401bf88740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f40101ab150 con 0x55cc6d1612f0 2024-07-26T13:47:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.416+0000 7f400ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc6d148d80 0x55cc6d53a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.416+0000 7f400ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc6d148d80 0x55cc6d53a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51740/0 (socket says 172.21.15.100:51740) 2024-07-26T13:47:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.416+0000 7f400ed9d640 1 -- 172.21.15.100:0/1589680336 learned_addr learned my addr 172.21.15.100:0/1589680336 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.416+0000 7f400f59e640 1 --2- 172.21.15.100:0/1589680336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc6d1694d0 0x55cc6d546880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.416+0000 7f400e59c640 1 --2- 172.21.15.100:0/1589680336 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc6d1612f0 0x55cc6d546340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.416+0000 7f400ed9d640 1 -- 172.21.15.100:0/1589680336 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc6d1612f0 msgr2=0x55cc6d546340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.416+0000 7f400ed9d640 1 --2- 172.21.15.100:0/1589680336 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc6d1612f0 0x55cc6d546340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.416+0000 7f400ed9d640 1 -- 172.21.15.100:0/1589680336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc6d1694d0 msgr2=0x55cc6d546880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.416+0000 7f400ed9d640 1 --2- 172.21.15.100:0/1589680336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc6d1694d0 0x55cc6d546880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.416+0000 7f400ed9d640 1 -- 172.21.15.100:0/1589680336 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc6d53b380 con 0x55cc6d148d80 2024-07-26T13:47:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.416+0000 7f400ed9d640 1 --2- 172.21.15.100:0/1589680336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc6d148d80 0x55cc6d53a5a0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f3ff803b580 tx=0x7f3ff80caa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc393cb2329db15c server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.416+0000 7f4004ff9640 1 -- 172.21.15.100:0/1589680336 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff80ae070 con 0x55cc6d148d80 2024-07-26T13:47:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.417+0000 7f4004ff9640 1 -- 172.21.15.100:0/1589680336 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ff80cab40 con 0x55cc6d148d80 2024-07-26T13:47:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.417+0000 7f4004ff9640 1 -- 172.21.15.100:0/1589680336 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8049040 con 0x55cc6d148d80 2024-07-26T13:47:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.417+0000 7f401bf88740 1 -- 172.21.15.100:0/1589680336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc6d148d80 msgr2=0x55cc6d53a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.417+0000 7f401bf88740 1 --2- 172.21.15.100:0/1589680336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc6d148d80 0x55cc6d53a5a0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f3ff803b580 tx=0x7f3ff80caa40 comp rx=0 tx=0).stop 2024-07-26T13:47:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.418+0000 7f401bf88740 1 -- 172.21.15.100:0/1589680336 shutdown_connections 2024-07-26T13:47:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.418+0000 7f401bf88740 1 --2- 172.21.15.100:0/1589680336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc6d1694d0 0x55cc6d546880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.418+0000 7f401bf88740 1 --2- 172.21.15.100:0/1589680336 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc6d1612f0 0x55cc6d546340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.418+0000 7f401bf88740 1 --2- 172.21.15.100:0/1589680336 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc6d148d80 0x55cc6d53a5a0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.418+0000 7f401bf88740 1 -- 172.21.15.100:0/1589680336 >> 172.21.15.100:0/1589680336 conn(0x55cc6d17e300 msgr2=0x55cc6d529ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.418+0000 7f401bf88740 1 -- 172.21.15.100:0/1589680336 shutdown_connections 2024-07-26T13:47:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.418+0000 7f401bf88740 1 -- 172.21.15.100:0/1589680336 wait complete. 2024-07-26T13:47:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.418+0000 7f401bf88740 1 -- start start 2024-07-26T13:47:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.418+0000 7f401bf88740 1 Processor -- start 2024-07-26T13:47:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.418+0000 7f401bf88740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc6d148d80 0x55cc6d54dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.418+0000 7f401bf88740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc6d1612f0 0x55cc6d54e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.418+0000 7f401bf88740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc6d1694d0 0x55cc6d5ab700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.418+0000 7f401bf88740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f40101ab7c0 con 0x55cc6d1694d0 2024-07-26T13:47:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.418+0000 7f401bf88740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f40101acc30 con 0x55cc6d148d80 2024-07-26T13:47:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.418+0000 7f401bf88740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f40101abe90 con 0x55cc6d1612f0 2024-07-26T13:47:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.419+0000 7f401bf88740 10 client.0 _ll_register_callbacks cb 0x7f40080098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:47:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.419+0000 7f400f59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc6d1694d0 0x55cc6d5ab700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.419+0000 7f400e59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc6d1612f0 0x55cc6d54e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.419+0000 7f400f59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc6d1694d0 0x55cc6d5ab700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51750/0 (socket says 172.21.15.100:51750) 2024-07-26T13:47:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.419+0000 7f400e59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc6d1612f0 0x55cc6d54e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:40806/0 (socket says 172.21.15.100:40806) 2024-07-26T13:47:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.419+0000 7f400f59e640 1 -- 172.21.15.100:0/1117609799 learned_addr learned my addr 172.21.15.100:0/1117609799 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.419+0000 7f400ed9d640 1 --2- 172.21.15.100:0/1117609799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc6d148d80 0x55cc6d54dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.419+0000 7f400f59e640 1 -- 172.21.15.100:0/1117609799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc6d1612f0 msgr2=0x55cc6d54e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.419+0000 7f400f59e640 1 --2- 172.21.15.100:0/1117609799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc6d1612f0 0x55cc6d54e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.419+0000 7f400f59e640 1 -- 172.21.15.100:0/1117609799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc6d148d80 msgr2=0x55cc6d54dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.419+0000 7f400f59e640 1 --2- 172.21.15.100:0/1117609799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc6d148d80 0x55cc6d54dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.419+0000 7f400f59e640 1 -- 172.21.15.100:0/1117609799 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc6d5abda0 con 0x55cc6d1694d0 2024-07-26T13:47:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.419+0000 7f400e59c640 1 --2- 172.21.15.100:0/1117609799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc6d1612f0 0x55cc6d54e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:47:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.419+0000 7f400f59e640 1 --2- 172.21.15.100:0/1117609799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc6d1694d0 0x55cc6d5ab700 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f40080127a0 tx=0x7f4008004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c3b2124cb14be7 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.419+0000 7f3ff67fc640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:47:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.419+0000 7f3ff67fc640 1 -- 172.21.15.100:0/1117609799 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400802c060 con 0x55cc6d1694d0 2024-07-26T13:47:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.419+0000 7f3ff67fc640 1 -- 172.21.15.100:0/1117609799 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8828+0+0 (secure 0 0 0) 0x7f400801ca40 con 0x55cc6d1694d0 2024-07-26T13:47:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.419+0000 7f401bf88740 1 -- 172.21.15.100:0/1117609799 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ff80cab40 con 0x55cc6d1694d0 2024-07-26T13:47:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.419+0000 7f3ff67fc640 1 -- 172.21.15.100:0/1117609799 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f400801cc80 con 0x55cc6d1694d0 2024-07-26T13:47:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.419+0000 7f401bf88740 10 client.17388 fetch_fsmap learned FSMap version 61 2024-07-26T13:47:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.419+0000 7f401bf88740 1 -- 172.21.15.100:0/1117609799 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55cc6d5b31a0 con 0x55cc6d1694d0 2024-07-26T13:47:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.420+0000 7f3ff67fc640 1 -- 172.21.15.100:0/1117609799 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ff80cab40 con 0x55cc6d1694d0 2024-07-26T13:47:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.420+0000 7f3ff67fc640 1 -- 172.21.15.100:0/1117609799 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 61) ==== 8346+0+0 (secure 0 0 0) 0x7f4008019040 con 0x55cc6d1694d0 2024-07-26T13:47:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.420+0000 7f401bf88740 10 client.17388 fetch_fsmap finished waiting for FSMap version 61 2024-07-26T13:47:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.420+0000 7f401bf88740 10 client.17388 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:47:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.420+0000 7f401bf88740 20 client.17388 populate_metadata read hostname 'smithi100' 2024-07-26T13:47:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.420+0000 7f401bf88740 1 --2- 172.21.15.100:0/1117609799 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55cc6d18a1b0 0x55cc6d5b59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.420+0000 7f401bf88740 4 client.17388 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-26T13:47:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.420+0000 7f401bf88740 1 -- 172.21.15.100:0/1117609799 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cc6d49fda0 con 0x55cc6d18a1b0 2024-07-26T13:47:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.420+0000 7f400ed9d640 1 --2- 172.21.15.100:0/1117609799 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55cc6d18a1b0 0x55cc6d5b59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.420+0000 7f400ed9d640 1 --2- 172.21.15.100:0/1117609799 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55cc6d18a1b0 0x55cc6d5b59c0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61f46973e29e429b server_cookie=f1e7a399f5b0e2c9 in_seq=0 out_seq=0 2024-07-26T13:47:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.420+0000 7f3ff67fc640 10 client.17388 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:45.427 INFO:tasks.workunit.client.0.smithi100.stdout: 1 316426 3.84 MB/sec execute 239 sec latency 23.301 ms 2024-07-26T13:47:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.499+0000 7f3ff67fc640 1 -- 172.21.15.100:0/1117609799 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cc6d49fda0 con 0x55cc6d18a1b0 2024-07-26T13:47:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.499+0000 7f3ff67fc640 10 client.17388 handle_command_reply: tid=0 2024-07-26T13:47:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.499+0000 7f401bf88740 1 client.17388 shutdown 2024-07-26T13:47:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.500+0000 7f401bf88740 1 -- 172.21.15.100:0/1117609799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc6d1694d0 msgr2=0x55cc6d5ab700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.500+0000 7f401bf88740 1 --2- 172.21.15.100:0/1117609799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc6d1694d0 0x55cc6d5ab700 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f40080127a0 tx=0x7f4008004a40 comp rx=0 tx=0).stop 2024-07-26T13:47:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.500+0000 7f401bf88740 1 -- 172.21.15.100:0/1117609799 shutdown_connections 2024-07-26T13:47:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.500+0000 7f401bf88740 1 --2- 172.21.15.100:0/1117609799 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55cc6d18a1b0 0x55cc6d5b59c0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.500+0000 7f401bf88740 1 --2- 172.21.15.100:0/1117609799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55cc6d1694d0 0x55cc6d5ab700 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.500+0000 7f3ff67fc640 0 client.17388 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.500+0000 7f401bf88740 1 --2- 172.21.15.100:0/1117609799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cc6d1612f0 0x55cc6d54e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.500+0000 7f401bf88740 1 --2- 172.21.15.100:0/1117609799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cc6d148d80 0x55cc6d54dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.500+0000 7f401bf88740 1 -- 172.21.15.100:0/1117609799 >> 172.21.15.100:0/1117609799 conn(0x55cc6d17e300 msgr2=0x55cc6d52d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.500+0000 7f401bf88740 1 -- 172.21.15.100:0/1117609799 shutdown_connections 2024-07-26T13:47:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.500+0000 7f401bf88740 1 -- 172.21.15.100:0/1117609799 wait complete. 2024-07-26T13:47:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.500+0000 7f401bf88740 20 client.17388 trim_cache size 0 max 16384 2024-07-26T13:47:45.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-07-26T13:47:45.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-07-26T13:47:45.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-07-26T13:47:45.503 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-07-26T13:47:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.501+0000 7f401506f640 1 -- 172.21.15.100:0/1817327561 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3ffc0b3dc0 msgr2=0x7f3ffc0b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.501+0000 7f401506f640 1 --2- 172.21.15.100:0/1817327561 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3ffc0b3dc0 0x7f3ffc0b62d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4008005fd0 tx=0x7f4008009500 comp rx=0 tx=0).stop 2024-07-26T13:47:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.501+0000 7f401506f640 1 -- 172.21.15.100:0/1817327561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4010102410 msgr2=0x7f4010198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.501+0000 7f401506f640 1 --2- 172.21.15.100:0/1817327561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4010102410 0x7f4010198dc0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8005940 tx=0x7f3ff8004270 comp rx=0 tx=0).stop 2024-07-26T13:47:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.501+0000 7f401506f640 1 -- 172.21.15.100:0/1817327561 shutdown_connections 2024-07-26T13:47:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.501+0000 7f401506f640 1 --2- 172.21.15.100:0/1817327561 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f3ffc0b3dc0 0x7f3ffc0b62d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.501+0000 7f401506f640 1 --2- 172.21.15.100:0/1817327561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4010108690 0x7f401019e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.501+0000 7f401506f640 1 --2- 172.21.15.100:0/1817327561 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4010107d40 0x7f4010199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.501+0000 7f401506f640 1 --2- 172.21.15.100:0/1817327561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4010102410 0x7f4010198dc0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.501+0000 7f401506f640 1 -- 172.21.15.100:0/1817327561 >> 172.21.15.100:0/1817327561 conn(0x7f40100fddf0 msgr2=0x7f4010106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.502+0000 7f401506f640 1 -- 172.21.15.100:0/1817327561 shutdown_connections 2024-07-26T13:47:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:45.502+0000 7f401506f640 1 -- 172.21.15.100:0/1817327561 wait complete. 2024-07-26T13:47:45.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.545+0000 7f1b4431b640 1 -- 172.21.15.100:0/2407206546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b3c107e00 msgr2=0x7f1b3c108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:45.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.545+0000 7f1b4431b640 1 --2- 172.21.15.100:0/2407206546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b3c107e00 0x7f1b3c108210 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f1b38009900 tx=0x7f1b3802a410 comp rx=0 tx=0).stop 2024-07-26T13:47:45.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.546+0000 7f1b4431b640 1 -- 172.21.15.100:0/2407206546 shutdown_connections 2024-07-26T13:47:45.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.546+0000 7f1b4431b640 1 --2- 172.21.15.100:0/2407206546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b3c108750 0x7f1b3c10ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.546+0000 7f1b4431b640 1 --2- 172.21.15.100:0/2407206546 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b3c107e00 0x7f1b3c108210 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.546+0000 7f1b4431b640 1 --2- 172.21.15.100:0/2407206546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b3c1024c0 0x7f1b3c1028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.546+0000 7f1b4431b640 1 -- 172.21.15.100:0/2407206546 >> 172.21.15.100:0/2407206546 conn(0x7f1b3c079180 msgr2=0x7f1b3c079590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:45.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.546+0000 7f1b4431b640 1 -- 172.21.15.100:0/2407206546 shutdown_connections 2024-07-26T13:47:45.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.546+0000 7f1b4431b640 1 -- 172.21.15.100:0/2407206546 wait complete. 2024-07-26T13:47:45.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.546+0000 7f1b4431b640 1 Processor -- start 2024-07-26T13:47:45.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.546+0000 7f1b4431b640 1 -- start start 2024-07-26T13:47:45.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.547+0000 7f1b4431b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b3c1024c0 0x7f1b3c19ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:45.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.547+0000 7f1b4431b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b3c107e00 0x7f1b3c19b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:45.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.547+0000 7f1b4431b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b3c108750 0x7f1b3c19b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:45.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.547+0000 7f1b4431b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c112380 con 0x7f1b3c107e00 2024-07-26T13:47:45.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.547+0000 7f1b4431b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c112200 con 0x7f1b3c1024c0 2024-07-26T13:47:45.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.547+0000 7f1b4431b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c112500 con 0x7f1b3c108750 2024-07-26T13:47:45.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.547+0000 7f1b420b7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b3c1024c0 0x7f1b3c19ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:45.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.547+0000 7f1b428b8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b3c108750 0x7f1b3c19b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:45.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.547+0000 7f1b420b7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b3c1024c0 0x7f1b3c19ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:35836/0 (socket says 172.21.15.100:35836) 2024-07-26T13:47:45.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.547+0000 7f1b420b7640 1 -- 172.21.15.100:0/958300067 learned_addr learned my addr 172.21.15.100:0/958300067 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:45.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.547+0000 7f1b418b6640 1 --2- 172.21.15.100:0/958300067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b3c107e00 0x7f1b3c19b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:45.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.547+0000 7f1b420b7640 1 -- 172.21.15.100:0/958300067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b3c108750 msgr2=0x7f1b3c19b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:45.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.547+0000 7f1b420b7640 1 --2- 172.21.15.100:0/958300067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b3c108750 0x7f1b3c19b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.547+0000 7f1b420b7640 1 -- 172.21.15.100:0/958300067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b3c107e00 msgr2=0x7f1b3c19b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:45.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.547+0000 7f1b420b7640 1 --2- 172.21.15.100:0/958300067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b3c107e00 0x7f1b3c19b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.547+0000 7f1b420b7640 1 -- 172.21.15.100:0/958300067 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b3c1a5f20 con 0x7f1b3c1024c0 2024-07-26T13:47:45.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.547+0000 7f1b428b8640 1 --2- 172.21.15.100:0/958300067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b3c108750 0x7f1b3c19b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:47:45.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.547+0000 7f1b418b6640 1 --2- 172.21.15.100:0/958300067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b3c107e00 0x7f1b3c19b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:47:45.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.548+0000 7f1b420b7640 1 --2- 172.21.15.100:0/958300067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b3c1024c0 0x7f1b3c19ac10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00e9f0 tx=0x7f1b2c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:45.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.548+0000 7f1b337fe640 1 -- 172.21.15.100:0/958300067 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c00cd80 con 0x7f1b3c1024c0 2024-07-26T13:47:45.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.548+0000 7f1b4431b640 1 -- 172.21.15.100:0/958300067 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b3c1a6210 con 0x7f1b3c1024c0 2024-07-26T13:47:45.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.548+0000 7f1b337fe640 1 -- 172.21.15.100:0/958300067 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b2c0108e0 con 0x7f1b3c1024c0 2024-07-26T13:47:45.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.548+0000 7f1b4431b640 1 -- 172.21.15.100:0/958300067 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b3c1a6770 con 0x7f1b3c1024c0 2024-07-26T13:47:45.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.548+0000 7f1b337fe640 1 -- 172.21.15.100:0/958300067 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c018980 con 0x7f1b3c1024c0 2024-07-26T13:47:45.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.549+0000 7f1b4431b640 1 -- 172.21.15.100:0/958300067 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b200052c0 con 0x7f1b3c1024c0 2024-07-26T13:47:45.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.551+0000 7f1b337fe640 1 -- 172.21.15.100:0/958300067 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f1b2c010430 con 0x7f1b3c1024c0 2024-07-26T13:47:45.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.551+0000 7f1b337fe640 1 --2- 172.21.15.100:0/958300067 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1b280b3dc0 0x7f1b280b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:45.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.552+0000 7f1b418b6640 1 --2- 172.21.15.100:0/958300067 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1b280b3dc0 0x7f1b280b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:45.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.552+0000 7f1b337fe640 1 -- 172.21.15.100:0/958300067 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8828+0+0 (secure 0 0 0) 0x7f1b2c0e4320 con 0x7f1b3c1024c0 2024-07-26T13:47:45.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.552+0000 7f1b418b6640 1 --2- 172.21.15.100:0/958300067 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1b280b3dc0 0x7f1b280b62d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1b38000c00 tx=0x7f1b38031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:45.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.559+0000 7f1b337fe640 1 -- 172.21.15.100:0/958300067 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f1b2c0aaeb0 con 0x7f1b3c1024c0 2024-07-26T13:47:45.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.780+0000 7f1b4431b640 1 -- 172.21.15.100:0/958300067 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b200055b0 con 0x7f1b3c1024c0 2024-07-26T13:47:45.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.781+0000 7f1b337fe640 1 -- 172.21.15.100:0/958300067 <== mon.2 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+11928 (secure 0 0 0) 0x7f1b2c0af400 con 0x7f1b3c1024c0 2024-07-26T13:47:45.783 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:47:45.783 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":61,"btime":"2024-07-26T13:47:33:271455+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14790,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/313501409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":313501409},{"type":"v1","addr":"172.21.15.100:6839","nonce":313501409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/1402975506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":1402975506},{"type":"v1","addr":"172.21.15.100:6841","nonce":1402975506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":61,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-26T13:37:01.281843+0000","modified":"2024-07-26T13:47:33.271423+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":87,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24607,"mds_1":24605,"mds_2":26446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24575":{"gid":24575,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/3393225483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3393225483},{"type":"v1","addr":"172.21.15.138:6837","nonce":3393225483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.138:6839/1580925471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1580925471},{"type":"v1","addr":"172.21.15.138:6839","nonce":1580925471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/398150514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":398150514},{"type":"v1","addr":"172.21.15.167:6839","nonce":398150514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26446":{"gid":26446,"name":"c","rank":2,"incarnation":56,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/3519222898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3519222898},{"type":"v1","addr":"172.21.15.167:6837","nonce":3519222898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26558":{"gid":26558,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/1781111462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1781111462},{"type":"v1","addr":"172.21.15.138:6835","nonce":1781111462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24607,"qdb_cluster":[26446,24607,24605]},"id":1}]} 2024-07-26T13:47:45.783 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 61 2024-07-26T13:47:45.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.783+0000 7f1b4431b640 1 -- 172.21.15.100:0/958300067 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1b280b3dc0 msgr2=0x7f1b280b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:45.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.783+0000 7f1b4431b640 1 --2- 172.21.15.100:0/958300067 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1b280b3dc0 0x7f1b280b62d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1b38000c00 tx=0x7f1b38031400 comp rx=0 tx=0).stop 2024-07-26T13:47:45.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.783+0000 7f1b4431b640 1 -- 172.21.15.100:0/958300067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b3c1024c0 msgr2=0x7f1b3c19ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:45.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.783+0000 7f1b4431b640 1 --2- 172.21.15.100:0/958300067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b3c1024c0 0x7f1b3c19ac10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00e9f0 tx=0x7f1b2c00eeb0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.783+0000 7f1b4431b640 1 -- 172.21.15.100:0/958300067 shutdown_connections 2024-07-26T13:47:45.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.783+0000 7f1b4431b640 1 --2- 172.21.15.100:0/958300067 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f1b280b3dc0 0x7f1b280b62d0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.783+0000 7f1b4431b640 1 --2- 172.21.15.100:0/958300067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b3c108750 0x7f1b3c19b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.783+0000 7f1b4431b640 1 --2- 172.21.15.100:0/958300067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1b3c107e00 0x7f1b3c19b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.783+0000 7f1b4431b640 1 --2- 172.21.15.100:0/958300067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b3c1024c0 0x7f1b3c19ac10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:45.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.783+0000 7f1b4431b640 1 -- 172.21.15.100:0/958300067 >> 172.21.15.100:0/958300067 conn(0x7f1b3c079180 msgr2=0x7f1b3c104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:45.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.783+0000 7f1b4431b640 1 -- 172.21.15.100:0/958300067 shutdown_connections 2024-07-26T13:47:45.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:45.783+0000 7f1b4431b640 1 -- 172.21.15.100:0/958300067 wait complete. 2024-07-26T13:47:46.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:45 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/958300067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:47:46.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:45 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/958300067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:47:46.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:45 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/958300067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:47:46.426 INFO:tasks.workunit.client.0.smithi100.stdout: 1 318713 3.88 MB/sec execute 240 sec latency 38.989 ms 2024-07-26T13:47:46.575 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-26T13:47:46.847 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:47:47.091 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:46 smithi100 ceph-mon[32599]: pgmap v600: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.4 MiB/s wr, 246 op/s 2024-07-26T13:47:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:46 smithi138 ceph-mon[36954]: pgmap v600: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.4 MiB/s wr, 246 op/s 2024-07-26T13:47:47.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:46 smithi167 ceph-mon[37140]: pgmap v600: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.4 MiB/s wr, 246 op/s 2024-07-26T13:47:47.426 INFO:tasks.workunit.client.0.smithi100.stdout: 1 320173 3.89 MB/sec execute 241 sec latency 26.326 ms 2024-07-26T13:47:48.426 INFO:tasks.workunit.client.0.smithi100.stdout: 1 322603 3.93 MB/sec execute 242 sec latency 18.600 ms 2024-07-26T13:47:48.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.528+0000 7f98f2547640 1 -- 172.21.15.100:0/2265254712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ec102490 msgr2=0x7f98ec1028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:48.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.528+0000 7f98f2547640 1 --2- 172.21.15.100:0/2265254712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ec102490 0x7f98ec1028a0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f98dc009900 tx=0x7f98dc02a410 comp rx=0 tx=0).stop 2024-07-26T13:47:48.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.528+0000 7f98f2547640 1 -- 172.21.15.100:0/2265254712 shutdown_connections 2024-07-26T13:47:48.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.528+0000 7f98f2547640 1 --2- 172.21.15.100:0/2265254712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98ec108720 0x7f98ec10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:48.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.528+0000 7f98f2547640 1 --2- 172.21.15.100:0/2265254712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98ec107dd0 0x7f98ec1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:48.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.528+0000 7f98f2547640 1 --2- 172.21.15.100:0/2265254712 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ec102490 0x7f98ec1028a0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:48.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.528+0000 7f98f2547640 1 -- 172.21.15.100:0/2265254712 >> 172.21.15.100:0/2265254712 conn(0x7f98ec079160 msgr2=0x7f98ec079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:48.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.529+0000 7f98f2547640 1 -- 172.21.15.100:0/2265254712 shutdown_connections 2024-07-26T13:47:48.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.529+0000 7f98f2547640 1 -- 172.21.15.100:0/2265254712 wait complete. 2024-07-26T13:47:48.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.529+0000 7f98f2547640 1 Processor -- start 2024-07-26T13:47:48.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.529+0000 7f98f2547640 1 -- start start 2024-07-26T13:47:48.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.530+0000 7f98f2547640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98ec102490 0x7f98ec199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:48.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.530+0000 7f98f2547640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ec107dd0 0x7f98ec19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:48.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.530+0000 7f98f2547640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98ec108720 0x7f98ec19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:48.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.530+0000 7f98f2547640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec1134b0 con 0x7f98ec107dd0 2024-07-26T13:47:48.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.530+0000 7f98f2547640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec113330 con 0x7f98ec102490 2024-07-26T13:47:48.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.530+0000 7f98f2547640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec113630 con 0x7f98ec108720 2024-07-26T13:47:48.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.530+0000 7f98eb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ec107dd0 0x7f98ec19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:48.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.530+0000 7f98ebfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98ec102490 0x7f98ec199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:48.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.530+0000 7f98eb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ec107dd0 0x7f98ec19a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51792/0 (socket says 172.21.15.100:51792) 2024-07-26T13:47:48.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.530+0000 7f98eb7fe640 1 -- 172.21.15.100:0/2679546853 learned_addr learned my addr 172.21.15.100:0/2679546853 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:48.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.530+0000 7f98f0ae4640 1 --2- 172.21.15.100:0/2679546853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98ec108720 0x7f98ec19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:48.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.530+0000 7f98eb7fe640 1 -- 172.21.15.100:0/2679546853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98ec108720 msgr2=0x7f98ec19f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:48.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.530+0000 7f98eb7fe640 1 --2- 172.21.15.100:0/2679546853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98ec108720 0x7f98ec19f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:48.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.530+0000 7f98eb7fe640 1 -- 172.21.15.100:0/2679546853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98ec102490 msgr2=0x7f98ec199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:48.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.530+0000 7f98eb7fe640 1 --2- 172.21.15.100:0/2679546853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98ec102490 0x7f98ec199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:48.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.530+0000 7f98eb7fe640 1 -- 172.21.15.100:0/2679546853 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98ec19feb0 con 0x7f98ec107dd0 2024-07-26T13:47:48.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.530+0000 7f98ebfff640 1 --2- 172.21.15.100:0/2679546853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98ec102490 0x7f98ec199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:47:48.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.530+0000 7f98eb7fe640 1 --2- 172.21.15.100:0/2679546853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ec107dd0 0x7f98ec19a4e0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f98e000e9c0 tx=0x7f98e000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:48.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.531+0000 7f98e97fa640 1 -- 172.21.15.100:0/2679546853 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e000cd80 con 0x7f98ec107dd0 2024-07-26T13:47:48.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.531+0000 7f98f2547640 1 -- 172.21.15.100:0/2679546853 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98ec1a01a0 con 0x7f98ec107dd0 2024-07-26T13:47:48.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.531+0000 7f98e97fa640 1 -- 172.21.15.100:0/2679546853 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98e00054d0 con 0x7f98ec107dd0 2024-07-26T13:47:48.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.531+0000 7f98e97fa640 1 -- 172.21.15.100:0/2679546853 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e0010430 con 0x7f98ec107dd0 2024-07-26T13:47:48.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.531+0000 7f98f2547640 1 -- 172.21.15.100:0/2679546853 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98ec1a6900 con 0x7f98ec107dd0 2024-07-26T13:47:48.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.533+0000 7f98f8acb740 1 Processor -- start 2024-07-26T13:47:48.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.533+0000 7f98f8acb740 1 -- start start 2024-07-26T13:47:48.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.533+0000 7f98e97fa640 1 -- 172.21.15.100:0/2679546853 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f98e00105d0 con 0x7f98ec107dd0 2024-07-26T13:47:48.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.533+0000 7f98f8acb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83d694070 0x55c83da39570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:48.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.533+0000 7f98f8acb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c83d66b1b0 0x55c83da39ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:48.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.533+0000 7f98f8acb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c83d673390 0x55c83da39ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:48.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.534+0000 7f98f8acb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c83d9afbb0 con 0x55c83d694070 2024-07-26T13:47:48.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.534+0000 7f98f8acb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c83d99f0b0 con 0x55c83d66b1b0 2024-07-26T13:47:48.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.534+0000 7f98f8acb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c83d9a30f0 con 0x55c83d673390 2024-07-26T13:47:48.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.534+0000 7f98f0ae4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83d694070 0x55c83da39570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:48.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.534+0000 7f98f0ae4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83d694070 0x55c83da39570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51802/0 (socket says 172.21.15.100:51802) 2024-07-26T13:47:48.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.534+0000 7f98f0ae4640 1 -- 172.21.15.100:0/4135525233 learned_addr learned my addr 172.21.15.100:0/4135525233 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:48.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.534+0000 7f98ebfff640 1 --2- 172.21.15.100:0/4135525233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c83d66b1b0 0x55c83da39ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:48.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.534+0000 7f98eb7fe640 1 --2- 172.21.15.100:0/4135525233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c83d673390 0x55c83da39ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:48.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.534+0000 7f98ebfff640 1 -- 172.21.15.100:0/4135525233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c83d673390 msgr2=0x55c83da39ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:48.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.534+0000 7f98ebfff640 1 --2- 172.21.15.100:0/4135525233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c83d673390 0x55c83da39ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:48.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.534+0000 7f98ebfff640 1 -- 172.21.15.100:0/4135525233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83d694070 msgr2=0x55c83da39570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:48.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.534+0000 7f98ebfff640 1 --2- 172.21.15.100:0/4135525233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83d694070 0x55c83da39570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:48.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.534+0000 7f98ebfff640 1 -- 172.21.15.100:0/4135525233 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c83d945b90 con 0x55c83d66b1b0 2024-07-26T13:47:48.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.534+0000 7f98f0ae4640 1 --2- 172.21.15.100:0/4135525233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83d694070 0x55c83da39570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:47:48.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.535+0000 7f98ebfff640 1 --2- 172.21.15.100:0/4135525233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c83d66b1b0 0x55c83da39ab0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f98dc009230 tx=0x7f98dc005e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d45ada0fe2e9d server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:48.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.535+0000 7f98e97fa640 1 --2- 172.21.15.100:0/2679546853 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f98d40b3d40 0x7f98d40b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:48.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.535+0000 7f98f0ae4640 1 --2- 172.21.15.100:0/2679546853 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f98d40b3d40 0x7f98d40b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:48.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.535+0000 7f98d27fc640 1 -- 172.21.15.100:0/4135525233 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc035040 con 0x55c83d66b1b0 2024-07-26T13:47:48.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.535+0000 7f98d27fc640 1 -- 172.21.15.100:0/4135525233 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98dc031720 con 0x55c83d66b1b0 2024-07-26T13:47:48.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.535+0000 7f98e97fa640 1 -- 172.21.15.100:0/2679546853 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8828+0+0 (secure 0 0 0) 0x7f98e00e4610 con 0x7f98ec107dd0 2024-07-26T13:47:48.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.535+0000 7f98d27fc640 1 -- 172.21.15.100:0/4135525233 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc03fa40 con 0x55c83d66b1b0 2024-07-26T13:47:48.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.536+0000 7f98f8acb740 1 -- 172.21.15.100:0/4135525233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c83d66b1b0 msgr2=0x55c83da39ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:48.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.536+0000 7f98f0ae4640 1 --2- 172.21.15.100:0/2679546853 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f98d40b3d40 0x7f98d40b6250 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f98e40117b0 tx=0x7f98e4009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:48.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.536+0000 7f98f8acb740 1 --2- 172.21.15.100:0/4135525233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c83d66b1b0 0x55c83da39ab0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f98dc009230 tx=0x7f98dc005e20 comp rx=0 tx=0).stop 2024-07-26T13:47:48.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.536+0000 7f98f8acb740 1 -- 172.21.15.100:0/4135525233 shutdown_connections 2024-07-26T13:47:48.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.536+0000 7f98f8acb740 1 --2- 172.21.15.100:0/4135525233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c83d673390 0x55c83da39ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:48.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.536+0000 7f98f8acb740 1 --2- 172.21.15.100:0/4135525233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c83d66b1b0 0x55c83da39ab0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:48.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.536+0000 7f98f8acb740 1 --2- 172.21.15.100:0/4135525233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83d694070 0x55c83da39570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:48.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.536+0000 7f98f8acb740 1 -- 172.21.15.100:0/4135525233 >> 172.21.15.100:0/4135525233 conn(0x55c83d688460 msgr2=0x55c83da37460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:48.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.536+0000 7f98f8acb740 1 -- 172.21.15.100:0/4135525233 shutdown_connections 2024-07-26T13:47:48.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.536+0000 7f98f8acb740 1 -- 172.21.15.100:0/4135525233 wait complete. 2024-07-26T13:47:48.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.536+0000 7f98f8acb740 1 -- start start 2024-07-26T13:47:48.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.537+0000 7f98f8acb740 1 Processor -- start 2024-07-26T13:47:48.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.537+0000 7f98f8acb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c83d66b1b0 0x55c83da451d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:48.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.537+0000 7f98f8acb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c83d673390 0x55c83da45750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:48.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.537+0000 7f98f8acb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83d694070 0x55c83da50fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:48.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.537+0000 7f98f8acb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c83d802700 con 0x55c83d694070 2024-07-26T13:47:48.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.537+0000 7f98f8acb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c83d9419e0 con 0x55c83d673390 2024-07-26T13:47:48.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.537+0000 7f98f8acb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c83da47cf0 con 0x55c83d66b1b0 2024-07-26T13:47:48.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.537+0000 7f98f8acb740 10 client.0 _ll_register_callbacks cb 0x55c83d95c4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:47:48.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.537+0000 7f98ebfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c83d66b1b0 0x55c83da451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:48.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.537+0000 7f98ebfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c83d66b1b0 0x55c83da451d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:40856/0 (socket says 172.21.15.100:40856) 2024-07-26T13:47:48.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.537+0000 7f98ebfff640 1 -- 172.21.15.100:0/3324691982 learned_addr learned my addr 172.21.15.100:0/3324691982 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:48.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.537+0000 7f98f0ae4640 1 --2- 172.21.15.100:0/3324691982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83d694070 0x55c83da50fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:48.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.538+0000 7f98eb7fe640 1 --2- 172.21.15.100:0/3324691982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c83d673390 0x55c83da45750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:48.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.538+0000 7f98ebfff640 1 -- 172.21.15.100:0/3324691982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c83d673390 msgr2=0x55c83da45750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:48.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.538+0000 7f98ebfff640 1 --2- 172.21.15.100:0/3324691982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c83d673390 0x55c83da45750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:48.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.538+0000 7f98ebfff640 1 -- 172.21.15.100:0/3324691982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83d694070 msgr2=0x55c83da50fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:48.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.538+0000 7f98ebfff640 1 --2- 172.21.15.100:0/3324691982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83d694070 0x55c83da50fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:48.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.538+0000 7f98ebfff640 1 -- 172.21.15.100:0/3324691982 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c83d6ff650 con 0x55c83d66b1b0 2024-07-26T13:47:48.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.538+0000 7f98ebfff640 1 --2- 172.21.15.100:0/3324691982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c83d66b1b0 0x55c83da451d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f98dc005510 tx=0x7f98dc0319e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3d98749cd1fc5a8 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:48.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.538+0000 7f98cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-07-26T13:47:48.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.538+0000 7f98cb7fe640 1 -- 172.21.15.100:0/3324691982 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc050070 con 0x55c83d66b1b0 2024-07-26T13:47:48.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.538+0000 7f98cb7fe640 1 -- 172.21.15.100:0/3324691982 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8828+0+0 (secure 0 0 0) 0x7f98dc004240 con 0x55c83d66b1b0 2024-07-26T13:47:48.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.538+0000 7f98f8acb740 1 -- 172.21.15.100:0/3324691982 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98dc031720 con 0x55c83d66b1b0 2024-07-26T13:47:48.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.538+0000 7f98cb7fe640 1 -- 172.21.15.100:0/3324691982 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f98dc05c050 con 0x55c83d66b1b0 2024-07-26T13:47:48.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.539+0000 7f98f8acb740 10 client.26800 fetch_fsmap learned FSMap version 61 2024-07-26T13:47:48.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.539+0000 7f98f8acb740 1 -- 172.21.15.100:0/3324691982 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55c83da53d50 con 0x55c83d66b1b0 2024-07-26T13:47:48.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.539+0000 7f98cb7fe640 1 -- 172.21.15.100:0/3324691982 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98dc031720 con 0x55c83d66b1b0 2024-07-26T13:47:48.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.539+0000 7f98cb7fe640 1 -- 172.21.15.100:0/3324691982 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 61) ==== 8346+0+0 (secure 0 0 0) 0x7f98dc003820 con 0x55c83d66b1b0 2024-07-26T13:47:48.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.539+0000 7f98f8acb740 10 client.26800 fetch_fsmap finished waiting for FSMap version 61 2024-07-26T13:47:48.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.539+0000 7f98f8acb740 10 client.26800 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-26T13:47:48.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.539+0000 7f98f8acb740 20 client.26800 populate_metadata read hostname 'smithi100' 2024-07-26T13:47:48.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.539+0000 7f98f8acb740 1 --2- 172.21.15.100:0/3324691982 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55c83d6529f0 0x55c83da570b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:48.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.539+0000 7f98f8acb740 4 client.26800 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:47:48.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.539+0000 7f98f8acb740 1 -- 172.21.15.100:0/3324691982 --> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c83d9b1b00 con 0x55c83d6529f0 2024-07-26T13:47:48.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.539+0000 7f98eb7fe640 1 --2- 172.21.15.100:0/3324691982 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55c83d6529f0 0x55c83da570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:48.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.540+0000 7f98cb7fe640 10 client.26800 ms_handle_connect on v2:172.21.15.138:6838/1580925471 2024-07-26T13:47:48.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.540+0000 7f98eb7fe640 1 --2- 172.21.15.100:0/3324691982 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55c83d6529f0 0x55c83da570b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f43ff45054859d72 server_cookie=f142af26dceb5fe2 in_seq=0 out_seq=0 2024-07-26T13:47:48.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.541+0000 7f98cb7fe640 1 -- 172.21.15.100:0/3324691982 <== mds.1 v2:172.21.15.138:6838/1580925471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c83d9b1b00 con 0x55c83d6529f0 2024-07-26T13:47:48.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.541+0000 7f98cb7fe640 10 client.26800 handle_command_reply: tid=0 2024-07-26T13:47:48.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.541+0000 7f98f8acb740 1 client.26800 shutdown 2024-07-26T13:47:48.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.541+0000 7f98f8acb740 1 -- 172.21.15.100:0/3324691982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c83d66b1b0 msgr2=0x55c83da451d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:48.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.541+0000 7f98f8acb740 1 --2- 172.21.15.100:0/3324691982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c83d66b1b0 0x55c83da451d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f98dc005510 tx=0x7f98dc0319e0 comp rx=0 tx=0).stop 2024-07-26T13:47:48.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.541+0000 7f98f8acb740 1 -- 172.21.15.100:0/3324691982 shutdown_connections 2024-07-26T13:47:48.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.541+0000 7f98f8acb740 1 --2- 172.21.15.100:0/3324691982 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55c83d6529f0 0x55c83da570b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:48.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.541+0000 7f98f8acb740 1 --2- 172.21.15.100:0/3324691982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83d694070 0x55c83da50fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:48.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.541+0000 7f98cb7fe640 0 client.26800 ms_handle_reset on v2:172.21.15.138:6838/1580925471 2024-07-26T13:47:48.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.541+0000 7f98f8acb740 1 --2- 172.21.15.100:0/3324691982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c83d673390 0x55c83da45750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:48.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.541+0000 7f98f8acb740 1 --2- 172.21.15.100:0/3324691982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c83d66b1b0 0x55c83da451d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:48.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.541+0000 7f98f8acb740 1 -- 172.21.15.100:0/3324691982 >> 172.21.15.100:0/3324691982 conn(0x55c83d688460 msgr2=0x55c83da35c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:48.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.541+0000 7f98f8acb740 1 -- 172.21.15.100:0/3324691982 shutdown_connections 2024-07-26T13:47:48.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.541+0000 7f98f8acb740 1 -- 172.21.15.100:0/3324691982 wait complete. 2024-07-26T13:47:48.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.541+0000 7f98f8acb740 20 client.26800 trim_cache size 0 max 16384 2024-07-26T13:47:48.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.555+0000 7f98f8acb740 1 Processor -- start 2024-07-26T13:47:48.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.555+0000 7f98f8acb740 1 -- start start 2024-07-26T13:47:48.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.555+0000 7f98f8acb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83d6529f0 0x55c83da362b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:48.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.555+0000 7f98f8acb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c83d66b1b0 0x55c83da50f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:48.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.555+0000 7f98f8acb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c83d673390 0x55c83da3ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:48.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.555+0000 7f98f8acb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec1afb80 con 0x55c83d6529f0 2024-07-26T13:47:48.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.555+0000 7f98f8acb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec1ad450 con 0x55c83d673390 2024-07-26T13:47:48.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.555+0000 7f98f8acb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec1ab2a0 con 0x55c83d66b1b0 2024-07-26T13:47:48.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.555+0000 7f98ebfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83d6529f0 0x55c83da362b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:48.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.555+0000 7f98ebfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83d6529f0 0x55c83da362b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51820/0 (socket says 172.21.15.100:51820) 2024-07-26T13:47:48.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.555+0000 7f98ebfff640 1 -- 172.21.15.100:0/2129310891 learned_addr learned my addr 172.21.15.100:0/2129310891 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:48.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.555+0000 7f98eb7fe640 1 --2- 172.21.15.100:0/2129310891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c83d66b1b0 0x55c83da50f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:48.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.555+0000 7f98f0ae4640 1 --2- 172.21.15.100:0/2129310891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c83d673390 0x55c83da3ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:48.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.555+0000 7f98ebfff640 1 -- 172.21.15.100:0/2129310891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c83d66b1b0 msgr2=0x55c83da50f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:48.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.555+0000 7f98ebfff640 1 --2- 172.21.15.100:0/2129310891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c83d66b1b0 0x55c83da50f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:48.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.555+0000 7f98ebfff640 1 -- 172.21.15.100:0/2129310891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c83d673390 msgr2=0x55c83da3ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:48.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.555+0000 7f98ebfff640 1 --2- 172.21.15.100:0/2129310891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c83d673390 0x55c83da3ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:48.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.555+0000 7f98ebfff640 1 -- 172.21.15.100:0/2129310891 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c83da46290 con 0x55c83d6529f0 2024-07-26T13:47:48.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.556+0000 7f98ebfff640 1 --2- 172.21.15.100:0/2129310891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83d6529f0 0x55c83da362b0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f98dc0057c0 tx=0x7f98dc032ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926d0daf2188e353 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:48.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.556+0000 7f98d1ffb640 1 -- 172.21.15.100:0/2129310891 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc050030 con 0x55c83d6529f0 2024-07-26T13:47:48.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.556+0000 7f98d1ffb640 1 -- 172.21.15.100:0/2129310891 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98dc004040 con 0x55c83d6529f0 2024-07-26T13:47:48.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.556+0000 7f98d1ffb640 1 -- 172.21.15.100:0/2129310891 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc035050 con 0x55c83d6529f0 2024-07-26T13:47:48.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.557+0000 7f98f8acb740 1 -- 172.21.15.100:0/2129310891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83d6529f0 msgr2=0x55c83da362b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:48.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.557+0000 7f98f8acb740 1 --2- 172.21.15.100:0/2129310891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83d6529f0 0x55c83da362b0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f98dc0057c0 tx=0x7f98dc032ed0 comp rx=0 tx=0).stop 2024-07-26T13:47:48.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.557+0000 7f98f8acb740 1 -- 172.21.15.100:0/2129310891 shutdown_connections 2024-07-26T13:47:48.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.557+0000 7f98f8acb740 1 --2- 172.21.15.100:0/2129310891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c83d673390 0x55c83da3ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:48.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.557+0000 7f98f8acb740 1 --2- 172.21.15.100:0/2129310891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c83d66b1b0 0x55c83da50f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:48.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.557+0000 7f98f8acb740 1 --2- 172.21.15.100:0/2129310891 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83d6529f0 0x55c83da362b0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:48.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.557+0000 7f98f8acb740 1 -- 172.21.15.100:0/2129310891 >> 172.21.15.100:0/2129310891 conn(0x55c83d688460 msgr2=0x55c83da3e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:48.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.557+0000 7f98f8acb740 1 -- 172.21.15.100:0/2129310891 shutdown_connections 2024-07-26T13:47:48.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.557+0000 7f98f8acb740 1 -- 172.21.15.100:0/2129310891 wait complete. 2024-07-26T13:47:48.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.557+0000 7f98f8acb740 1 -- start start 2024-07-26T13:47:48.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.557+0000 7f98f8acb740 1 Processor -- start 2024-07-26T13:47:48.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.558+0000 7f98f8acb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c83d6529f0 0x55c83da5a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:48.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.558+0000 7f98f8acb740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83d66b1b0 0x55c83da57980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:48.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.558+0000 7f98f8acb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c83d673390 0x55c83da57ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:48.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.558+0000 7f98f8acb740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec1ab910 con 0x55c83d66b1b0 2024-07-26T13:47:48.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.558+0000 7f98f8acb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec1acd80 con 0x55c83d6529f0 2024-07-26T13:47:48.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.558+0000 7f98f8acb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f98ec1abfe0 con 0x55c83d673390 2024-07-26T13:47:48.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.558+0000 7f98f8acb740 10 client.0 _ll_register_callbacks cb 0x7f98e40099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:47:48.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.558+0000 7f98eb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83d66b1b0 0x55c83da57980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:48.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.558+0000 7f98eb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83d66b1b0 0x55c83da57980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51836/0 (socket says 172.21.15.100:51836) 2024-07-26T13:47:48.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.558+0000 7f98ebfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c83d6529f0 0x55c83da5a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:48.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.558+0000 7f98eb7fe640 1 -- 172.21.15.100:0/3249933995 learned_addr learned my addr 172.21.15.100:0/3249933995 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:48.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.558+0000 7f98f0ae4640 1 --2- 172.21.15.100:0/3249933995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c83d673390 0x55c83da57ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:48.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.558+0000 7f98eb7fe640 1 -- 172.21.15.100:0/3249933995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c83d673390 msgr2=0x55c83da57ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:48.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.558+0000 7f98eb7fe640 1 --2- 172.21.15.100:0/3249933995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c83d673390 0x55c83da57ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:48.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.558+0000 7f98eb7fe640 1 -- 172.21.15.100:0/3249933995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c83d6529f0 msgr2=0x55c83da5a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:48.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.558+0000 7f98eb7fe640 1 --2- 172.21.15.100:0/3249933995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c83d6529f0 0x55c83da5a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:48.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.558+0000 7f98eb7fe640 1 -- 172.21.15.100:0/3249933995 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c83da58400 con 0x55c83d66b1b0 2024-07-26T13:47:48.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.558+0000 7f98eb7fe640 1 --2- 172.21.15.100:0/3249933995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83d66b1b0 0x55c83da57980 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f98e00a8360 tx=0x7f98e0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e225c03f304d8951 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:48.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.558+0000 7f98cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:47:48.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.559+0000 7f98cb7fe640 1 -- 172.21.15.100:0/3249933995 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e00a6fb0 con 0x55c83d66b1b0 2024-07-26T13:47:48.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.559+0000 7f98cb7fe640 1 -- 172.21.15.100:0/3249933995 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8828+0+0 (secure 0 0 0) 0x7f98e00b3d10 con 0x55c83d66b1b0 2024-07-26T13:47:48.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.559+0000 7f98f8acb740 1 -- 172.21.15.100:0/3249933995 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98dc004040 con 0x55c83d66b1b0 2024-07-26T13:47:48.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.559+0000 7f98cb7fe640 1 -- 172.21.15.100:0/3249933995 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f98e00aa860 con 0x55c83d66b1b0 2024-07-26T13:47:48.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.559+0000 7f98f8acb740 10 client.17412 fetch_fsmap learned FSMap version 61 2024-07-26T13:47:48.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.559+0000 7f98f8acb740 1 -- 172.21.15.100:0/3249933995 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55c83dabf2a0 con 0x55c83d66b1b0 2024-07-26T13:47:48.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.559+0000 7f98cb7fe640 1 -- 172.21.15.100:0/3249933995 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98dc004040 con 0x55c83d66b1b0 2024-07-26T13:47:48.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.559+0000 7f98cb7fe640 1 -- 172.21.15.100:0/3249933995 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 61) ==== 8346+0+0 (secure 0 0 0) 0x7f98e00108c0 con 0x55c83d66b1b0 2024-07-26T13:47:48.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.559+0000 7f98f8acb740 10 client.17412 fetch_fsmap finished waiting for FSMap version 61 2024-07-26T13:47:48.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.559+0000 7f98f8acb740 10 client.17412 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-07-26T13:47:48.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.559+0000 7f98f8acb740 20 client.17412 populate_metadata read hostname 'smithi100' 2024-07-26T13:47:48.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.559+0000 7f98f8acb740 1 --2- 172.21.15.100:0/3249933995 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55c83d694070 0x55c83dac1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:48.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.559+0000 7f98f8acb740 4 client.17412 mds_command: new command op to 24605 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-26T13:47:48.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.559+0000 7f98f8acb740 1 -- 172.21.15.100:0/3249933995 --> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c83d9ad5a0 con 0x55c83d694070 2024-07-26T13:47:48.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.559+0000 7f98ebfff640 1 --2- 172.21.15.100:0/3249933995 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55c83d694070 0x55c83dac1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:48.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.560+0000 7f98cb7fe640 10 client.17412 ms_handle_connect on v2:172.21.15.138:6838/1580925471 2024-07-26T13:47:48.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.560+0000 7f98ebfff640 1 --2- 172.21.15.100:0/3249933995 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55c83d694070 0x55c83dac1a60 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=38c2a55702e00dab server_cookie=6e541938ed392ee4 in_seq=0 out_seq=0 2024-07-26T13:47:48.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.560+0000 7f98cb7fe640 1 -- 172.21.15.100:0/3249933995 <== mds.1 v2:172.21.15.138:6838/1580925471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c83d9ad5a0 con 0x55c83d694070 2024-07-26T13:47:48.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.560+0000 7f98cb7fe640 10 client.17412 handle_command_reply: tid=0 2024-07-26T13:47:48.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.560+0000 7f98f8acb740 1 client.17412 shutdown 2024-07-26T13:47:48.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.560+0000 7f98f8acb740 1 -- 172.21.15.100:0/3249933995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83d66b1b0 msgr2=0x55c83da57980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:48.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.560+0000 7f98f8acb740 1 --2- 172.21.15.100:0/3249933995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83d66b1b0 0x55c83da57980 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f98e00a8360 tx=0x7f98e0009310 comp rx=0 tx=0).stop 2024-07-26T13:47:48.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.560+0000 7f98f8acb740 1 -- 172.21.15.100:0/3249933995 shutdown_connections 2024-07-26T13:47:48.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.560+0000 7f98f8acb740 1 --2- 172.21.15.100:0/3249933995 >> [v2:172.21.15.138:6838/1580925471,v1:172.21.15.138:6839/1580925471] conn(0x55c83d694070 0x55c83dac1a60 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:48.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.560+0000 7f98f8acb740 1 --2- 172.21.15.100:0/3249933995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c83d673390 0x55c83da57ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:48.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.560+0000 7f98f8acb740 1 --2- 172.21.15.100:0/3249933995 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c83d66b1b0 0x55c83da57980 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:48.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.560+0000 7f98cb7fe640 0 client.17412 ms_handle_reset on v2:172.21.15.138:6838/1580925471 2024-07-26T13:47:48.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.560+0000 7f98f8acb740 1 --2- 172.21.15.100:0/3249933995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c83d6529f0 0x55c83da5a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:48.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.560+0000 7f98f8acb740 1 -- 172.21.15.100:0/3249933995 >> 172.21.15.100:0/3249933995 conn(0x55c83d688460 msgr2=0x55c83da36b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:48.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.560+0000 7f98f8acb740 1 -- 172.21.15.100:0/3249933995 shutdown_connections 2024-07-26T13:47:48.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.560+0000 7f98f8acb740 1 -- 172.21.15.100:0/3249933995 wait complete. 2024-07-26T13:47:48.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.561+0000 7f98f8acb740 20 client.17412 trim_cache size 0 max 16384 2024-07-26T13:47:48.567 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-07-26T13:47:48.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.561+0000 7f98f2547640 1 -- 172.21.15.100:0/2679546853 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f98d40b3d40 msgr2=0x7f98d40b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:48.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.561+0000 7f98f2547640 1 --2- 172.21.15.100:0/2679546853 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f98d40b3d40 0x7f98d40b6250 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f98e40117b0 tx=0x7f98e4009500 comp rx=0 tx=0).stop 2024-07-26T13:47:48.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.561+0000 7f98f2547640 1 -- 172.21.15.100:0/2679546853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ec107dd0 msgr2=0x7f98ec19a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:48.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.561+0000 7f98f2547640 1 --2- 172.21.15.100:0/2679546853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ec107dd0 0x7f98ec19a4e0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f98e000e9c0 tx=0x7f98e000ee80 comp rx=0 tx=0).stop 2024-07-26T13:47:48.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.561+0000 7f98f2547640 1 -- 172.21.15.100:0/2679546853 shutdown_connections 2024-07-26T13:47:48.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.561+0000 7f98f2547640 1 --2- 172.21.15.100:0/2679546853 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f98d40b3d40 0x7f98d40b6250 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:48.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.561+0000 7f98f2547640 1 --2- 172.21.15.100:0/2679546853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98ec108720 0x7f98ec19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:48.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.561+0000 7f98f2547640 1 --2- 172.21.15.100:0/2679546853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f98ec107dd0 0x7f98ec19a4e0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:48.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.561+0000 7f98f2547640 1 --2- 172.21.15.100:0/2679546853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98ec102490 0x7f98ec199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:48.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.561+0000 7f98f2547640 1 -- 172.21.15.100:0/2679546853 >> 172.21.15.100:0/2679546853 conn(0x7f98ec079160 msgr2=0x7f98ec100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:48.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.561+0000 7f98f2547640 1 -- 172.21.15.100:0/2679546853 shutdown_connections 2024-07-26T13:47:48.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:48.561+0000 7f98f2547640 1 -- 172.21.15.100:0/2679546853 wait complete. 2024-07-26T13:47:49.252 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-26T13:47:49.426 INFO:tasks.workunit.client.0.smithi100.stdout: 1 324053 3.94 MB/sec execute 243 sec latency 14.309 ms 2024-07-26T13:47:49.518 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:47:49.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:49 smithi138 ceph-mon[36954]: pgmap v601: 97 pgs: 97 active+clean; 116 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.5 MiB/s wr, 273 op/s 2024-07-26T13:47:49.945 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:49 smithi100 ceph-mon[32599]: pgmap v601: 97 pgs: 97 active+clean; 116 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.5 MiB/s wr, 273 op/s 2024-07-26T13:47:50.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:49 smithi167 ceph-mon[37140]: pgmap v601: 97 pgs: 97 active+clean; 116 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.5 MiB/s wr, 273 op/s 2024-07-26T13:47:50.426 INFO:tasks.workunit.client.0.smithi100.stdout: 1 326445 3.99 MB/sec execute 244 sec latency 14.405 ms 2024-07-26T13:47:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:50 smithi100 ceph-mon[32599]: pgmap v602: 97 pgs: 97 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.2 MiB/s wr, 278 op/s 2024-07-26T13:47:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:50 smithi138 ceph-mon[36954]: pgmap v602: 97 pgs: 97 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.2 MiB/s wr, 278 op/s 2024-07-26T13:47:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:50 smithi167 ceph-mon[37140]: pgmap v602: 97 pgs: 97 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.2 MiB/s wr, 278 op/s 2024-07-26T13:47:51.426 INFO:tasks.workunit.client.0.smithi100.stdout: 1 327843 4.00 MB/sec execute 245 sec latency 47.733 ms 2024-07-26T13:47:51.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.439+0000 7f025d9d2640 1 -- 172.21.15.100:0/339808570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0258107dd0 msgr2=0x7f02581081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:51.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.439+0000 7f025d9d2640 1 --2- 172.21.15.100:0/339808570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0258107dd0 0x7f02581081e0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f024c009900 tx=0x7f024c02a410 comp rx=0 tx=0).stop 2024-07-26T13:47:51.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.439+0000 7f025d9d2640 1 -- 172.21.15.100:0/339808570 shutdown_connections 2024-07-26T13:47:51.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.439+0000 7f025d9d2640 1 --2- 172.21.15.100:0/339808570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0258108720 0x7f025810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:51.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.439+0000 7f025d9d2640 1 --2- 172.21.15.100:0/339808570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0258107dd0 0x7f02581081e0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:51.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.439+0000 7f025d9d2640 1 --2- 172.21.15.100:0/339808570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0258102490 0x7f02581028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:51.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.439+0000 7f025d9d2640 1 -- 172.21.15.100:0/339808570 >> 172.21.15.100:0/339808570 conn(0x7f0258079160 msgr2=0x7f0258079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:51.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.439+0000 7f025d9d2640 1 -- 172.21.15.100:0/339808570 shutdown_connections 2024-07-26T13:47:51.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.439+0000 7f025d9d2640 1 -- 172.21.15.100:0/339808570 wait complete. 2024-07-26T13:47:51.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.440+0000 7f025d9d2640 1 Processor -- start 2024-07-26T13:47:51.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.440+0000 7f025d9d2640 1 -- start start 2024-07-26T13:47:51.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.440+0000 7f025d9d2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0258102490 0x7f025810f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:51.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.440+0000 7f025d9d2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0258107dd0 0x7f025810ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:51.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.440+0000 7f025d9d2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0258108720 0x7f0258110460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:51.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.440+0000 7f025d9d2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f02581134b0 con 0x7f0258102490 2024-07-26T13:47:51.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.440+0000 7f025d9d2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0258113330 con 0x7f0258107dd0 2024-07-26T13:47:51.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.440+0000 7f025d9d2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0258113630 con 0x7f0258108720 2024-07-26T13:47:51.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.441+0000 7f0256ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0258102490 0x7f025810f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:51.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.441+0000 7f0256ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0258102490 0x7f025810f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51848/0 (socket says 172.21.15.100:51848) 2024-07-26T13:47:51.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.441+0000 7f0256ffd640 1 -- 172.21.15.100:0/995208953 learned_addr learned my addr 172.21.15.100:0/995208953 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:51.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.441+0000 7f02577fe640 1 --2- 172.21.15.100:0/995208953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0258108720 0x7f0258110460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:51.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.441+0000 7f02567fc640 1 --2- 172.21.15.100:0/995208953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0258107dd0 0x7f025810ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:51.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.441+0000 7f0256ffd640 1 -- 172.21.15.100:0/995208953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0258108720 msgr2=0x7f0258110460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:51.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.441+0000 7f0256ffd640 1 --2- 172.21.15.100:0/995208953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0258108720 0x7f0258110460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:51.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.441+0000 7f0256ffd640 1 -- 172.21.15.100:0/995208953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0258107dd0 msgr2=0x7f025810ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:51.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.441+0000 7f0256ffd640 1 --2- 172.21.15.100:0/995208953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0258107dd0 0x7f025810ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:51.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.441+0000 7f0256ffd640 1 -- 172.21.15.100:0/995208953 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02581a81a0 con 0x7f0258102490 2024-07-26T13:47:51.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.441+0000 7f0256ffd640 1 --2- 172.21.15.100:0/995208953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0258102490 0x7f025810f9e0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f024800e9f0 tx=0x7f024800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:51.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.441+0000 7f025c9d0640 1 -- 172.21.15.100:0/995208953 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024800cd80 con 0x7f0258102490 2024-07-26T13:47:51.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.441+0000 7f025c9d0640 1 -- 172.21.15.100:0/995208953 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02480108e0 con 0x7f0258102490 2024-07-26T13:47:51.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.441+0000 7f025c9d0640 1 -- 172.21.15.100:0/995208953 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0248018980 con 0x7f0258102490 2024-07-26T13:47:51.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.441+0000 7f025d9d2640 1 -- 172.21.15.100:0/995208953 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02581a8490 con 0x7f0258102490 2024-07-26T13:47:51.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.441+0000 7f025d9d2640 1 -- 172.21.15.100:0/995208953 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02581a8950 con 0x7f0258102490 2024-07-26T13:47:51.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.444+0000 7f0263f56740 1 Processor -- start 2024-07-26T13:47:51.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.444+0000 7f0263f56740 1 -- start start 2024-07-26T13:47:51.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.444+0000 7f025c9d0640 1 -- 172.21.15.100:0/995208953 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f0248010430 con 0x7f0258102490 2024-07-26T13:47:51.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.444+0000 7f0263f56740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae23fd8070 0x55ae2437d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:51.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.444+0000 7f0263f56740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae23faf1b0 0x55ae2437dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:51.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.444+0000 7f0263f56740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae23fb7390 0x55ae2437dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:51.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.444+0000 7f0263f56740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ae242f3bb0 con 0x55ae23fd8070 2024-07-26T13:47:51.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.444+0000 7f0263f56740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ae242e30b0 con 0x55ae23faf1b0 2024-07-26T13:47:51.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.444+0000 7f0263f56740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ae242e70f0 con 0x55ae23fb7390 2024-07-26T13:47:51.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.444+0000 7f02577fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae23fd8070 0x55ae2437d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:51.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.445+0000 7f02567fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae23fb7390 0x55ae2437dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:51.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.445+0000 7f02577fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae23fd8070 0x55ae2437d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51856/0 (socket says 172.21.15.100:51856) 2024-07-26T13:47:51.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.445+0000 7f0256ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae23faf1b0 0x55ae2437dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:51.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.445+0000 7f02577fe640 1 -- 172.21.15.100:0/4184874731 learned_addr learned my addr 172.21.15.100:0/4184874731 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:51.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.445+0000 7f02567fc640 1 -- 172.21.15.100:0/4184874731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae23faf1b0 msgr2=0x55ae2437dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:51.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.445+0000 7f02567fc640 1 --2- 172.21.15.100:0/4184874731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae23faf1b0 0x55ae2437dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:51.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.445+0000 7f02567fc640 1 -- 172.21.15.100:0/4184874731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae23fd8070 msgr2=0x55ae2437d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:51.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.445+0000 7f02567fc640 1 --2- 172.21.15.100:0/4184874731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae23fd8070 0x55ae2437d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:51.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.445+0000 7f02567fc640 1 -- 172.21.15.100:0/4184874731 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae24289b90 con 0x55ae23fb7390 2024-07-26T13:47:51.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.445+0000 7f02577fe640 1 --2- 172.21.15.100:0/4184874731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae23fd8070 0x55ae2437d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:47:51.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.445+0000 7f025c9d0640 1 --2- 172.21.15.100:0/995208953 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f02400b3d40 0x7f02400b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:51.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.445+0000 7f02567fc640 1 --2- 172.21.15.100:0/4184874731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae23fb7390 0x55ae2437dff0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f024c006010 tx=0x7f024c032b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3f28eb2632600fe server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:51.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.445+0000 7f02577fe640 1 --2- 172.21.15.100:0/995208953 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f02400b3d40 0x7f02400b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:51.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.445+0000 7f025c9d0640 1 -- 172.21.15.100:0/995208953 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8828+0+0 (secure 0 0 0) 0x7f02480e4440 con 0x7f0258102490 2024-07-26T13:47:51.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.445+0000 7f023dffb640 1 -- 172.21.15.100:0/4184874731 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c040070 con 0x55ae23fb7390 2024-07-26T13:47:51.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.446+0000 7f023dffb640 1 -- 172.21.15.100:0/4184874731 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f024c004030 con 0x55ae23fb7390 2024-07-26T13:47:51.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.446+0000 7f023dffb640 1 -- 172.21.15.100:0/4184874731 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c035040 con 0x55ae23fb7390 2024-07-26T13:47:51.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.446+0000 7f0263f56740 1 -- 172.21.15.100:0/4184874731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae23fb7390 msgr2=0x55ae2437dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:51.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.446+0000 7f0263f56740 1 --2- 172.21.15.100:0/4184874731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae23fb7390 0x55ae2437dff0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f024c006010 tx=0x7f024c032b20 comp rx=0 tx=0).stop 2024-07-26T13:47:51.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.446+0000 7f02577fe640 1 --2- 172.21.15.100:0/995208953 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f02400b3d40 0x7f02400b6250 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0250009fd0 tx=0x7f0250009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:51.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.446+0000 7f0263f56740 1 -- 172.21.15.100:0/4184874731 shutdown_connections 2024-07-26T13:47:51.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.446+0000 7f0263f56740 1 --2- 172.21.15.100:0/4184874731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae23fb7390 0x55ae2437dff0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:51.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.446+0000 7f0263f56740 1 --2- 172.21.15.100:0/4184874731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae23faf1b0 0x55ae2437dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:51.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.446+0000 7f0263f56740 1 --2- 172.21.15.100:0/4184874731 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae23fd8070 0x55ae2437d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:51.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.446+0000 7f0263f56740 1 -- 172.21.15.100:0/4184874731 >> 172.21.15.100:0/4184874731 conn(0x55ae23fcc460 msgr2=0x55ae2437b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:51.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.447+0000 7f0263f56740 1 -- 172.21.15.100:0/4184874731 shutdown_connections 2024-07-26T13:47:51.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.447+0000 7f0263f56740 1 -- 172.21.15.100:0/4184874731 wait complete. 2024-07-26T13:47:51.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.447+0000 7f0263f56740 1 -- start start 2024-07-26T13:47:51.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.447+0000 7f0263f56740 1 Processor -- start 2024-07-26T13:47:51.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.447+0000 7f0263f56740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae23faf1b0 0x55ae243891a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:51.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.447+0000 7f0263f56740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae23fb7390 0x55ae24389740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:51.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.447+0000 7f0263f56740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae23fd8070 0x55ae24394fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:51.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.448+0000 7f0263f56740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55ae24146700 con 0x55ae23fd8070 2024-07-26T13:47:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.448+0000 7f0263f56740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ae242859e0 con 0x55ae23faf1b0 2024-07-26T13:47:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.448+0000 7f0263f56740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ae2438bcf0 con 0x55ae23fb7390 2024-07-26T13:47:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.448+0000 7f0263f56740 10 client.0 _ll_register_callbacks cb 0x55ae242a04f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:47:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.448+0000 7f02577fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae23fd8070 0x55ae24394fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.448+0000 7f0256ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae23faf1b0 0x55ae243891a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.448+0000 7f02577fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae23fd8070 0x55ae24394fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51860/0 (socket says 172.21.15.100:51860) 2024-07-26T13:47:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.448+0000 7f02577fe640 1 -- 172.21.15.100:0/277085895 learned_addr learned my addr 172.21.15.100:0/277085895 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.448+0000 7f02567fc640 1 --2- 172.21.15.100:0/277085895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae23fb7390 0x55ae24389740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.448+0000 7f0256ffd640 1 -- 172.21.15.100:0/277085895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae23fb7390 msgr2=0x55ae24389740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.448+0000 7f0256ffd640 1 --2- 172.21.15.100:0/277085895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae23fb7390 0x55ae24389740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.448+0000 7f0256ffd640 1 -- 172.21.15.100:0/277085895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae23fd8070 msgr2=0x55ae24394fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.448+0000 7f0256ffd640 1 --2- 172.21.15.100:0/277085895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae23fd8070 0x55ae24394fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.448+0000 7f0256ffd640 1 -- 172.21.15.100:0/277085895 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae24043650 con 0x55ae23faf1b0 2024-07-26T13:47:51.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.448+0000 7f0256ffd640 1 --2- 172.21.15.100:0/277085895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae23faf1b0 0x55ae243891a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f02480e1a30 tx=0x7f0248010cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95c28a1256e5c602 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:51.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.448+0000 7f0236ffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-07-26T13:47:51.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.448+0000 7f0236ffd640 1 -- 172.21.15.100:0/277085895 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02480aa920 con 0x55ae23faf1b0 2024-07-26T13:47:51.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.448+0000 7f0263f56740 1 -- 172.21.15.100:0/277085895 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f024c004030 con 0x55ae23faf1b0 2024-07-26T13:47:51.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.448+0000 7f0236ffd640 1 -- 172.21.15.100:0/277085895 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8828+0+0 (secure 0 0 0) 0x7f02480aa230 con 0x55ae23faf1b0 2024-07-26T13:47:51.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.449+0000 7f0236ffd640 1 -- 172.21.15.100:0/277085895 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f02480aa740 con 0x55ae23faf1b0 2024-07-26T13:47:51.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.449+0000 7f0263f56740 10 client.26627 fetch_fsmap learned FSMap version 61 2024-07-26T13:47:51.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.449+0000 7f0263f56740 1 -- 172.21.15.100:0/277085895 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55ae24397de0 con 0x55ae23faf1b0 2024-07-26T13:47:51.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.449+0000 7f0236ffd640 1 -- 172.21.15.100:0/277085895 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f024c004030 con 0x55ae23faf1b0 2024-07-26T13:47:51.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.449+0000 7f0236ffd640 1 -- 172.21.15.100:0/277085895 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 61) ==== 8346+0+0 (secure 0 0 0) 0x7f02480a6940 con 0x55ae23faf1b0 2024-07-26T13:47:51.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.449+0000 7f0263f56740 10 client.26627 fetch_fsmap finished waiting for FSMap version 61 2024-07-26T13:47:51.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.449+0000 7f0263f56740 10 client.26627 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:47:51.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.449+0000 7f0263f56740 20 client.26627 populate_metadata read hostname 'smithi100' 2024-07-26T13:47:51.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.449+0000 7f0263f56740 1 --2- 172.21.15.100:0/277085895 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55ae23f969f0 0x55ae2439b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:51.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.449+0000 7f0263f56740 4 client.26627 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:47:51.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.449+0000 7f0263f56740 1 -- 172.21.15.100:0/277085895 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae242f5b00 con 0x55ae23f969f0 2024-07-26T13:47:51.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.449+0000 7f02567fc640 1 --2- 172.21.15.100:0/277085895 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55ae23f969f0 0x55ae2439b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:51.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.450+0000 7f0236ffd640 10 client.26627 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:51.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.450+0000 7f02567fc640 1 --2- 172.21.15.100:0/277085895 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55ae23f969f0 0x55ae2439b2c0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bbd2b3591cbe030 server_cookie=6d1636be667106b5 in_seq=0 out_seq=0 2024-07-26T13:47:51.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.451+0000 7f0236ffd640 1 -- 172.21.15.100:0/277085895 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ae242f5b00 con 0x55ae23f969f0 2024-07-26T13:47:51.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.451+0000 7f0236ffd640 10 client.26627 handle_command_reply: tid=0 2024-07-26T13:47:51.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.451+0000 7f0263f56740 1 client.26627 shutdown 2024-07-26T13:47:51.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.451+0000 7f0263f56740 1 -- 172.21.15.100:0/277085895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae23faf1b0 msgr2=0x55ae243891a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:51.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.451+0000 7f0263f56740 1 --2- 172.21.15.100:0/277085895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae23faf1b0 0x55ae243891a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f02480e1a30 tx=0x7f0248010cc0 comp rx=0 tx=0).stop 2024-07-26T13:47:51.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.451+0000 7f0263f56740 1 -- 172.21.15.100:0/277085895 shutdown_connections 2024-07-26T13:47:51.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.451+0000 7f0263f56740 1 --2- 172.21.15.100:0/277085895 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55ae23f969f0 0x55ae2439b2c0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:51.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.451+0000 7f0263f56740 1 --2- 172.21.15.100:0/277085895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae23fd8070 0x55ae24394fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:51.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.451+0000 7f0263f56740 1 --2- 172.21.15.100:0/277085895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae23fb7390 0x55ae24389740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:51.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.451+0000 7f0263f56740 1 --2- 172.21.15.100:0/277085895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae23faf1b0 0x55ae243891a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:51.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.451+0000 7f0263f56740 1 -- 172.21.15.100:0/277085895 >> 172.21.15.100:0/277085895 conn(0x55ae23fcc460 msgr2=0x55ae24379c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:51.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.451+0000 7f0236ffd640 0 client.26627 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:51.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.451+0000 7f0263f56740 1 -- 172.21.15.100:0/277085895 shutdown_connections 2024-07-26T13:47:51.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.451+0000 7f0263f56740 1 -- 172.21.15.100:0/277085895 wait complete. 2024-07-26T13:47:51.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.451+0000 7f0263f56740 20 client.26627 trim_cache size 0 max 16384 2024-07-26T13:47:51.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.464+0000 7f0263f56740 1 Processor -- start 2024-07-26T13:47:51.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.464+0000 7f0263f56740 1 -- start start 2024-07-26T13:47:51.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.465+0000 7f0263f56740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae23faf1b0 0x55ae2437a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:51.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.465+0000 7f0263f56740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae23fb7390 0x55ae24394dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:51.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.465+0000 7f0263f56740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae23fd8070 0x55ae24382680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:51.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.465+0000 7f0263f56740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f02581b3e60 con 0x55ae23faf1b0 2024-07-26T13:47:51.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.465+0000 7f0263f56740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f02581b1730 con 0x55ae23fb7390 2024-07-26T13:47:51.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.465+0000 7f0263f56740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f02581af580 con 0x55ae23fd8070 2024-07-26T13:47:51.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.465+0000 7f0256ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae23faf1b0 0x55ae2437a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:51.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.465+0000 7f0256ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae23faf1b0 0x55ae2437a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51862/0 (socket says 172.21.15.100:51862) 2024-07-26T13:47:51.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.465+0000 7f0256ffd640 1 -- 172.21.15.100:0/3651491401 learned_addr learned my addr 172.21.15.100:0/3651491401 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:51.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.465+0000 7f02577fe640 1 --2- 172.21.15.100:0/3651491401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae23fd8070 0x55ae24382680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:51.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.465+0000 7f02567fc640 1 --2- 172.21.15.100:0/3651491401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae23fb7390 0x55ae24394dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:51.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.465+0000 7f0256ffd640 1 -- 172.21.15.100:0/3651491401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae23fd8070 msgr2=0x55ae24382680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:51.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.465+0000 7f0256ffd640 1 --2- 172.21.15.100:0/3651491401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae23fd8070 0x55ae24382680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:51.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.465+0000 7f0256ffd640 1 -- 172.21.15.100:0/3651491401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae23fb7390 msgr2=0x55ae24394dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:51.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.465+0000 7f0256ffd640 1 --2- 172.21.15.100:0/3651491401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae23fb7390 0x55ae24394dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:51.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.465+0000 7f0256ffd640 1 -- 172.21.15.100:0/3651491401 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae24384830 con 0x55ae23faf1b0 2024-07-26T13:47:51.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.465+0000 7f02567fc640 1 --2- 172.21.15.100:0/3651491401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae23fb7390 0x55ae24394dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:47:51.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.465+0000 7f0256ffd640 1 --2- 172.21.15.100:0/3651491401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae23faf1b0 0x55ae2437a350 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f02480e2060 tx=0x7f02480e7b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d38765596e0f8f5 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:51.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.466+0000 7f023d7fa640 1 -- 172.21.15.100:0/3651491401 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02480aa920 con 0x55ae23faf1b0 2024-07-26T13:47:51.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.466+0000 7f023d7fa640 1 -- 172.21.15.100:0/3651491401 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0248010720 con 0x55ae23faf1b0 2024-07-26T13:47:51.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.466+0000 7f023d7fa640 1 -- 172.21.15.100:0/3651491401 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0248018b20 con 0x55ae23faf1b0 2024-07-26T13:47:51.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.467+0000 7f0263f56740 1 -- 172.21.15.100:0/3651491401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae23faf1b0 msgr2=0x55ae2437a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:51.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.467+0000 7f0263f56740 1 --2- 172.21.15.100:0/3651491401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae23faf1b0 0x55ae2437a350 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f02480e2060 tx=0x7f02480e7b60 comp rx=0 tx=0).stop 2024-07-26T13:47:51.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.467+0000 7f0263f56740 1 -- 172.21.15.100:0/3651491401 shutdown_connections 2024-07-26T13:47:51.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.467+0000 7f0263f56740 1 --2- 172.21.15.100:0/3651491401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae23fd8070 0x55ae24382680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:51.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.467+0000 7f0263f56740 1 --2- 172.21.15.100:0/3651491401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae23fb7390 0x55ae24394dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:51.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.467+0000 7f0263f56740 1 --2- 172.21.15.100:0/3651491401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae23faf1b0 0x55ae2437a350 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:51.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.467+0000 7f0263f56740 1 -- 172.21.15.100:0/3651491401 >> 172.21.15.100:0/3651491401 conn(0x55ae23fcc460 msgr2=0x55ae24379c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:51.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.467+0000 7f0263f56740 1 -- 172.21.15.100:0/3651491401 shutdown_connections 2024-07-26T13:47:51.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.467+0000 7f0263f56740 1 -- 172.21.15.100:0/3651491401 wait complete. 2024-07-26T13:47:51.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.467+0000 7f0263f56740 1 -- start start 2024-07-26T13:47:51.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.467+0000 7f0263f56740 1 Processor -- start 2024-07-26T13:47:51.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.468+0000 7f0263f56740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae23faf1b0 0x55ae24391560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:51.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.468+0000 7f0263f56740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae23fb7390 0x55ae2438ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:51.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.468+0000 7f0263f56740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae23fd8070 0x55ae2438cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:51.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.468+0000 7f0263f56740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f02581afbf0 con 0x55ae23fd8070 2024-07-26T13:47:51.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.468+0000 7f0263f56740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f02581b1060 con 0x55ae23fb7390 2024-07-26T13:47:51.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.468+0000 7f0263f56740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f02581b02c0 con 0x55ae23faf1b0 2024-07-26T13:47:51.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.468+0000 7f0263f56740 10 client.0 _ll_register_callbacks cb 0x7f024800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:47:51.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.468+0000 7f02577fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae23fd8070 0x55ae2438cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:51.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.468+0000 7f0256ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae23faf1b0 0x55ae24391560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:51.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.468+0000 7f02577fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae23fd8070 0x55ae2438cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51874/0 (socket says 172.21.15.100:51874) 2024-07-26T13:47:51.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.468+0000 7f02577fe640 1 -- 172.21.15.100:0/3125261127 learned_addr learned my addr 172.21.15.100:0/3125261127 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:51.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.468+0000 7f02567fc640 1 --2- 172.21.15.100:0/3125261127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae23fb7390 0x55ae2438ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:51.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.468+0000 7f0256ffd640 1 -- 172.21.15.100:0/3125261127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae23fb7390 msgr2=0x55ae2438ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:51.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.468+0000 7f0256ffd640 1 --2- 172.21.15.100:0/3125261127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae23fb7390 0x55ae2438ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:51.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.468+0000 7f0256ffd640 1 -- 172.21.15.100:0/3125261127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae23fd8070 msgr2=0x55ae2438cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:51.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.468+0000 7f0256ffd640 1 --2- 172.21.15.100:0/3125261127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae23fd8070 0x55ae2438cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:51.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.468+0000 7f0256ffd640 1 -- 172.21.15.100:0/3125261127 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae2438d640 con 0x55ae23faf1b0 2024-07-26T13:47:51.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.468+0000 7f02577fe640 1 --2- 172.21.15.100:0/3125261127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae23fd8070 0x55ae2438cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:47:51.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.468+0000 7f0256ffd640 1 --2- 172.21.15.100:0/3125261127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae23faf1b0 0x55ae24391560 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f02480e2060 tx=0x7f024800cf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40542b222a682669 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:51.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.468+0000 7f0236ffd640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-07-26T13:47:51.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.469+0000 7f0236ffd640 1 -- 172.21.15.100:0/3125261127 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02480aa920 con 0x55ae23faf1b0 2024-07-26T13:47:51.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.469+0000 7f0263f56740 1 -- 172.21.15.100:0/3125261127 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ae242f15a0 con 0x55ae23faf1b0 2024-07-26T13:47:51.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.469+0000 7f0236ffd640 1 -- 172.21.15.100:0/3125261127 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8828+0+0 (secure 0 0 0) 0x7f0248018d30 con 0x55ae23faf1b0 2024-07-26T13:47:51.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.469+0000 7f0236ffd640 1 -- 172.21.15.100:0/3125261127 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f02480bb050 con 0x55ae23faf1b0 2024-07-26T13:47:51.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.469+0000 7f0263f56740 10 client.26824 fetch_fsmap learned FSMap version 61 2024-07-26T13:47:51.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.469+0000 7f0263f56740 1 -- 172.21.15.100:0/3125261127 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55ae244038e0 con 0x55ae23faf1b0 2024-07-26T13:47:51.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.469+0000 7f0236ffd640 1 -- 172.21.15.100:0/3125261127 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ae242f15a0 con 0x55ae23faf1b0 2024-07-26T13:47:51.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.469+0000 7f0236ffd640 1 -- 172.21.15.100:0/3125261127 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 61) ==== 8346+0+0 (secure 0 0 0) 0x7f02480b06b0 con 0x55ae23faf1b0 2024-07-26T13:47:51.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.469+0000 7f0263f56740 10 client.26824 fetch_fsmap finished waiting for FSMap version 61 2024-07-26T13:47:51.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.469+0000 7f0263f56740 10 client.26824 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:47:51.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.469+0000 7f0263f56740 20 client.26824 populate_metadata read hostname 'smithi100' 2024-07-26T13:47:51.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.469+0000 7f0263f56740 1 --2- 172.21.15.100:0/3125261127 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55ae23f6bda0 0x55ae24405d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:51.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.469+0000 7f0263f56740 4 client.26824 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-26T13:47:51.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.469+0000 7f0263f56740 1 -- 172.21.15.100:0/3125261127 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ae242eece0 con 0x55ae23f6bda0 2024-07-26T13:47:51.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.469+0000 7f02567fc640 1 --2- 172.21.15.100:0/3125261127 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55ae23f6bda0 0x55ae24405d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:51.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.470+0000 7f0236ffd640 10 client.26824 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:51.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.470+0000 7f02567fc640 1 --2- 172.21.15.100:0/3125261127 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55ae23f6bda0 0x55ae24405d10 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56f52d63191a2769 server_cookie=bb0247b5b95da9a0 in_seq=0 out_seq=0 2024-07-26T13:47:51.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.470+0000 7f0236ffd640 1 -- 172.21.15.100:0/3125261127 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ae242eece0 con 0x55ae23f6bda0 2024-07-26T13:47:51.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.470+0000 7f0236ffd640 10 client.26824 handle_command_reply: tid=0 2024-07-26T13:47:51.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.471+0000 7f0263f56740 1 client.26824 shutdown 2024-07-26T13:47:51.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.471+0000 7f0263f56740 1 -- 172.21.15.100:0/3125261127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae23faf1b0 msgr2=0x55ae24391560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:51.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.471+0000 7f0263f56740 1 --2- 172.21.15.100:0/3125261127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae23faf1b0 0x55ae24391560 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f02480e2060 tx=0x7f024800cf20 comp rx=0 tx=0).stop 2024-07-26T13:47:51.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.471+0000 7f0263f56740 1 -- 172.21.15.100:0/3125261127 shutdown_connections 2024-07-26T13:47:51.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.471+0000 7f0263f56740 1 --2- 172.21.15.100:0/3125261127 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55ae23f6bda0 0x55ae24405d10 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:51.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.471+0000 7f0236ffd640 0 client.26824 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:51.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.471+0000 7f0263f56740 1 --2- 172.21.15.100:0/3125261127 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55ae23fd8070 0x55ae2438cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:51.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.471+0000 7f0263f56740 1 --2- 172.21.15.100:0/3125261127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae23fb7390 0x55ae2438ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:51.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.471+0000 7f0263f56740 1 --2- 172.21.15.100:0/3125261127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae23faf1b0 0x55ae24391560 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:51.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.471+0000 7f0263f56740 1 -- 172.21.15.100:0/3125261127 >> 172.21.15.100:0/3125261127 conn(0x55ae23fcc460 msgr2=0x55ae2437ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:51.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.471+0000 7f0263f56740 1 -- 172.21.15.100:0/3125261127 shutdown_connections 2024-07-26T13:47:51.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.471+0000 7f0263f56740 1 -- 172.21.15.100:0/3125261127 wait complete. 2024-07-26T13:47:51.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.471+0000 7f0263f56740 20 client.26824 trim_cache size 0 max 16384 2024-07-26T13:47:51.479 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-07-26T13:47:51.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.472+0000 7f025d9d2640 1 -- 172.21.15.100:0/995208953 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f02400b3d40 msgr2=0x7f02400b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:51.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.472+0000 7f025d9d2640 1 --2- 172.21.15.100:0/995208953 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f02400b3d40 0x7f02400b6250 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0250009fd0 tx=0x7f0250009500 comp rx=0 tx=0).stop 2024-07-26T13:47:51.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.472+0000 7f025d9d2640 1 -- 172.21.15.100:0/995208953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0258102490 msgr2=0x7f025810f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:51.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.472+0000 7f025d9d2640 1 --2- 172.21.15.100:0/995208953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0258102490 0x7f025810f9e0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f024800e9f0 tx=0x7f024800eeb0 comp rx=0 tx=0).stop 2024-07-26T13:47:51.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.472+0000 7f025d9d2640 1 -- 172.21.15.100:0/995208953 shutdown_connections 2024-07-26T13:47:51.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.472+0000 7f025d9d2640 1 --2- 172.21.15.100:0/995208953 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f02400b3d40 0x7f02400b6250 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:51.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.472+0000 7f025d9d2640 1 --2- 172.21.15.100:0/995208953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0258108720 0x7f0258110460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:51.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.472+0000 7f025d9d2640 1 --2- 172.21.15.100:0/995208953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0258107dd0 0x7f025810ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:51.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.472+0000 7f025d9d2640 1 --2- 172.21.15.100:0/995208953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0258102490 0x7f025810f9e0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:51.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.472+0000 7f025d9d2640 1 -- 172.21.15.100:0/995208953 >> 172.21.15.100:0/995208953 conn(0x7f0258079160 msgr2=0x7f0258104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:51.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.472+0000 7f025d9d2640 1 -- 172.21.15.100:0/995208953 shutdown_connections 2024-07-26T13:47:51.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:51.472+0000 7f025d9d2640 1 -- 172.21.15.100:0/995208953 wait complete. 2024-07-26T13:47:52.172 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-26T13:47:52.426 INFO:tasks.workunit.client.0.smithi100.stdout: 1 330138 4.04 MB/sec execute 246 sec latency 13.660 ms 2024-07-26T13:47:52.431 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:47:52.831 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:52 smithi100 ceph-mon[32599]: pgmap v603: 97 pgs: 97 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.5 MiB/s wr, 211 op/s 2024-07-26T13:47:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:52 smithi138 ceph-mon[36954]: pgmap v603: 97 pgs: 97 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.5 MiB/s wr, 211 op/s 2024-07-26T13:47:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:52 smithi167 ceph-mon[37140]: pgmap v603: 97 pgs: 97 active+clean; 114 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.5 MiB/s wr, 211 op/s 2024-07-26T13:47:53.426 INFO:tasks.workunit.client.0.smithi100.stdout: 1 331661 4.05 MB/sec execute 247 sec latency 19.933 ms 2024-07-26T13:47:54.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.210+0000 7f10cbb7c640 1 -- 172.21.15.100:0/3089269742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10c4107dd0 msgr2=0x7f10c41081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:54.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.210+0000 7f10cbb7c640 1 --2- 172.21.15.100:0/3089269742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10c4107dd0 0x7f10c41081e0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f10b8009ca0 tx=0x7f10b802a8c0 comp rx=0 tx=0).stop 2024-07-26T13:47:54.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.210+0000 7f10cbb7c640 1 -- 172.21.15.100:0/3089269742 shutdown_connections 2024-07-26T13:47:54.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.210+0000 7f10cbb7c640 1 --2- 172.21.15.100:0/3089269742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10c4108720 0x7f10c410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:54.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.210+0000 7f10cbb7c640 1 --2- 172.21.15.100:0/3089269742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10c4107dd0 0x7f10c41081e0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:54.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.210+0000 7f10cbb7c640 1 --2- 172.21.15.100:0/3089269742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10c4102490 0x7f10c41028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:54.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.210+0000 7f10cbb7c640 1 -- 172.21.15.100:0/3089269742 >> 172.21.15.100:0/3089269742 conn(0x7f10c4079160 msgr2=0x7f10c4079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:54.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.210+0000 7f10cbb7c640 1 -- 172.21.15.100:0/3089269742 shutdown_connections 2024-07-26T13:47:54.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.210+0000 7f10cbb7c640 1 -- 172.21.15.100:0/3089269742 wait complete. 2024-07-26T13:47:54.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.211+0000 7f10cbb7c640 1 Processor -- start 2024-07-26T13:47:54.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.211+0000 7f10cbb7c640 1 -- start start 2024-07-26T13:47:54.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.211+0000 7f10cbb7c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10c4102490 0x7f10c419aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:54.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.211+0000 7f10cbb7c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10c4107dd0 0x7f10c419b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:54.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.211+0000 7f10cbb7c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10c4108720 0x7f10c419b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:54.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.211+0000 7f10cbb7c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f10c4112350 con 0x7f10c4108720 2024-07-26T13:47:54.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.211+0000 7f10cbb7c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f10c41121d0 con 0x7f10c4102490 2024-07-26T13:47:54.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.211+0000 7f10cbb7c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f10c41124d0 con 0x7f10c4107dd0 2024-07-26T13:47:54.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.211+0000 7f10ca119640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10c4108720 0x7f10c419b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:54.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.211+0000 7f10ca119640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10c4108720 0x7f10c419b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51904/0 (socket says 172.21.15.100:51904) 2024-07-26T13:47:54.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.211+0000 7f10c9918640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10c4102490 0x7f10c419aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:54.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.211+0000 7f10ca119640 1 -- 172.21.15.100:0/2566723902 learned_addr learned my addr 172.21.15.100:0/2566723902 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:54.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.211+0000 7f10c9117640 1 --2- 172.21.15.100:0/2566723902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10c4107dd0 0x7f10c419b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:54.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.211+0000 7f10c9918640 1 -- 172.21.15.100:0/2566723902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10c4107dd0 msgr2=0x7f10c419b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:54.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.211+0000 7f10c9918640 1 --2- 172.21.15.100:0/2566723902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10c4107dd0 0x7f10c419b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:54.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.211+0000 7f10c9918640 1 -- 172.21.15.100:0/2566723902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10c4108720 msgr2=0x7f10c419b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:54.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.211+0000 7f10c9918640 1 --2- 172.21.15.100:0/2566723902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10c4108720 0x7f10c419b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:54.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.211+0000 7f10c9918640 1 -- 172.21.15.100:0/2566723902 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c41a5f60 con 0x7f10c4102490 2024-07-26T13:47:54.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.211+0000 7f10c9117640 1 --2- 172.21.15.100:0/2566723902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10c4107dd0 0x7f10c419b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:47:54.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.211+0000 7f10ca119640 1 --2- 172.21.15.100:0/2566723902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10c4108720 0x7f10c419b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:47:54.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.212+0000 7f10c9918640 1 --2- 172.21.15.100:0/2566723902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10c4102490 0x7f10c419aca0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f10b400e9f0 tx=0x7f10b400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:54.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.212+0000 7f10c2ffd640 1 -- 172.21.15.100:0/2566723902 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b400ccc0 con 0x7f10c4102490 2024-07-26T13:47:54.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.212+0000 7f10cbb7c640 1 -- 172.21.15.100:0/2566723902 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10c41a6250 con 0x7f10c4102490 2024-07-26T13:47:54.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.212+0000 7f10c2ffd640 1 -- 172.21.15.100:0/2566723902 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10b40108e0 con 0x7f10c4102490 2024-07-26T13:47:54.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.212+0000 7f10c2ffd640 1 -- 172.21.15.100:0/2566723902 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b4018880 con 0x7f10c4102490 2024-07-26T13:47:54.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.212+0000 7f10cbb7c640 1 -- 172.21.15.100:0/2566723902 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10c41a67e0 con 0x7f10c4102490 2024-07-26T13:47:54.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.214+0000 7f10d2100740 1 Processor -- start 2024-07-26T13:47:54.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.215+0000 7f10d2100740 1 -- start start 2024-07-26T13:47:54.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.215+0000 7f10d2100740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f452a6070 0x562f4564b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:54.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.215+0000 7f10d2100740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f4527d1b0 0x562f4564bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:54.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.215+0000 7f10d2100740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f45285390 0x562f4564bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:54.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.215+0000 7f10d2100740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x562f455c1bb0 con 0x562f452a6070 2024-07-26T13:47:54.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.215+0000 7f10d2100740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562f455b10b0 con 0x562f45285390 2024-07-26T13:47:54.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.215+0000 7f10d2100740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562f455b50f0 con 0x562f4527d1b0 2024-07-26T13:47:54.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.215+0000 7f10ca119640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f452a6070 0x562f4564b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:54.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.215+0000 7f10ca119640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f452a6070 0x562f4564b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51920/0 (socket says 172.21.15.100:51920) 2024-07-26T13:47:54.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.215+0000 7f10ca119640 1 -- 172.21.15.100:0/1609060602 learned_addr learned my addr 172.21.15.100:0/1609060602 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:54.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.215+0000 7f10c9117640 1 --2- 172.21.15.100:0/1609060602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f45285390 0x562f4564bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:54.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.215+0000 7f10c2ffd640 1 -- 172.21.15.100:0/2566723902 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f10b4010430 con 0x7f10c4102490 2024-07-26T13:47:54.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.215+0000 7f10ca119640 1 -- 172.21.15.100:0/1609060602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f4527d1b0 msgr2=0x562f4564bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:54.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.215+0000 7f10ca119640 1 --2- 172.21.15.100:0/1609060602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f4527d1b0 0x562f4564bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:54.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.215+0000 7f10ca119640 1 -- 172.21.15.100:0/1609060602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f45285390 msgr2=0x562f4564bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:54.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.215+0000 7f10ca119640 1 --2- 172.21.15.100:0/1609060602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f45285390 0x562f4564bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:54.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.215+0000 7f10ca119640 1 -- 172.21.15.100:0/1609060602 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f45557b90 con 0x562f452a6070 2024-07-26T13:47:54.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.215+0000 7f10c9117640 1 --2- 172.21.15.100:0/1609060602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f45285390 0x562f4564bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:47:54.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.216+0000 7f10ca119640 1 --2- 172.21.15.100:0/1609060602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f452a6070 0x562f4564b570 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f10bc0115b0 tx=0x7f10bc011a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1547e44df0c3ff9 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:54.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.216+0000 7f10c2ffd640 1 --2- 172.21.15.100:0/2566723902 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f10ac0b3d40 0x7f10ac0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:54.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.216+0000 7f10c9918640 1 --2- 172.21.15.100:0/2566723902 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f10ac0b3d40 0x7f10ac0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:54.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.216+0000 7f10abfff640 1 -- 172.21.15.100:0/1609060602 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc01e070 con 0x562f452a6070 2024-07-26T13:47:54.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.216+0000 7f10abfff640 1 -- 172.21.15.100:0/1609060602 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10bc00fb10 con 0x562f452a6070 2024-07-26T13:47:54.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.216+0000 7f10abfff640 1 -- 172.21.15.100:0/1609060602 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc019c00 con 0x562f452a6070 2024-07-26T13:47:54.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.217+0000 7f10c2ffd640 1 -- 172.21.15.100:0/2566723902 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8828+0+0 (secure 0 0 0) 0x7f10b40e45c0 con 0x7f10c4102490 2024-07-26T13:47:54.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.218+0000 7f10d2100740 1 -- 172.21.15.100:0/1609060602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f452a6070 msgr2=0x562f4564b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:54.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.218+0000 7f10d2100740 1 --2- 172.21.15.100:0/1609060602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f452a6070 0x562f4564b570 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f10bc0115b0 tx=0x7f10bc011a70 comp rx=0 tx=0).stop 2024-07-26T13:47:54.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.218+0000 7f10c9918640 1 --2- 172.21.15.100:0/2566723902 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f10ac0b3d40 0x7f10ac0b6250 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x562f4564a1f0 tx=0x7f10b4078790 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:54.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.218+0000 7f10d2100740 1 -- 172.21.15.100:0/1609060602 shutdown_connections 2024-07-26T13:47:54.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.218+0000 7f10d2100740 1 --2- 172.21.15.100:0/1609060602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f45285390 0x562f4564bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:54.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.218+0000 7f10d2100740 1 --2- 172.21.15.100:0/1609060602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f4527d1b0 0x562f4564bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:54.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.218+0000 7f10d2100740 1 --2- 172.21.15.100:0/1609060602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f452a6070 0x562f4564b570 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:54.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.218+0000 7f10d2100740 1 -- 172.21.15.100:0/1609060602 >> 172.21.15.100:0/1609060602 conn(0x562f4529a460 msgr2=0x562f45649460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:54.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.218+0000 7f10d2100740 1 -- 172.21.15.100:0/1609060602 shutdown_connections 2024-07-26T13:47:54.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.218+0000 7f10d2100740 1 -- 172.21.15.100:0/1609060602 wait complete. 2024-07-26T13:47:54.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.218+0000 7f10d2100740 1 -- start start 2024-07-26T13:47:54.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.218+0000 7f10d2100740 1 Processor -- start 2024-07-26T13:47:54.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.219+0000 7f10d2100740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f4527d1b0 0x562f45657200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:54.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.219+0000 7f10d2100740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f45285390 0x562f456577a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:54.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.219+0000 7f10d2100740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f452a6070 0x562f4565d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:54.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.219+0000 7f10d2100740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x562f45414700 con 0x562f4527d1b0 2024-07-26T13:47:54.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.219+0000 7f10d2100740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562f455539e0 con 0x562f452a6070 2024-07-26T13:47:54.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.219+0000 7f10d2100740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562f45659d50 con 0x562f45285390 2024-07-26T13:47:54.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.219+0000 7f10d2100740 10 client.0 _ll_register_callbacks cb 0x562f4556e4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:47:54.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.219+0000 7f10ca119640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f4527d1b0 0x562f45657200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:54.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.219+0000 7f10ca119640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f4527d1b0 0x562f45657200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51934/0 (socket says 172.21.15.100:51934) 2024-07-26T13:47:54.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.219+0000 7f10ca119640 1 -- 172.21.15.100:0/2826747486 learned_addr learned my addr 172.21.15.100:0/2826747486 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:54.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.219+0000 7f10c9117640 1 --2- 172.21.15.100:0/2826747486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f45285390 0x562f456577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:54.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.219+0000 7f10ca119640 1 --2- 172.21.15.100:0/2826747486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f452a6070 0x562f4565d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:54.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.219+0000 7f10ca119640 1 -- 172.21.15.100:0/2826747486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f45285390 msgr2=0x562f456577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:54.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.219+0000 7f10ca119640 1 --2- 172.21.15.100:0/2826747486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f45285390 0x562f456577a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:54.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.219+0000 7f10ca119640 1 -- 172.21.15.100:0/2826747486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f452a6070 msgr2=0x562f4565d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:54.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.219+0000 7f10ca119640 1 --2- 172.21.15.100:0/2826747486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f452a6070 0x562f4565d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:54.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.219+0000 7f10ca119640 1 -- 172.21.15.100:0/2826747486 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f45311650 con 0x562f4527d1b0 2024-07-26T13:47:54.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.220+0000 7f10ca119640 1 --2- 172.21.15.100:0/2826747486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f4527d1b0 0x562f45657200 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f10bc000c00 tx=0x7f10bc019490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=449284f7cc6662fa server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:54.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.220+0000 7f10a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:47:54.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.220+0000 7f10a97fa640 1 -- 172.21.15.100:0/2826747486 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc01e040 con 0x562f4527d1b0 2024-07-26T13:47:54.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.220+0000 7f10a97fa640 1 -- 172.21.15.100:0/2826747486 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8828+0+0 (secure 0 0 0) 0x7f10bc019b50 con 0x562f4527d1b0 2024-07-26T13:47:54.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.220+0000 7f10d2100740 1 -- 172.21.15.100:0/2826747486 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10bc00fb10 con 0x562f4527d1b0 2024-07-26T13:47:54.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.220+0000 7f10a97fa640 1 -- 172.21.15.100:0/2826747486 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f10bc019d50 con 0x562f4527d1b0 2024-07-26T13:47:54.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.220+0000 7f10d2100740 10 client.17442 fetch_fsmap learned FSMap version 61 2024-07-26T13:47:54.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.220+0000 7f10d2100740 1 -- 172.21.15.100:0/2826747486 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x562f45665e50 con 0x562f4527d1b0 2024-07-26T13:47:54.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.220+0000 7f10a97fa640 1 -- 172.21.15.100:0/2826747486 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10bc0054d0 con 0x562f4527d1b0 2024-07-26T13:47:54.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.221+0000 7f10a97fa640 1 -- 172.21.15.100:0/2826747486 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 61) ==== 8346+0+0 (secure 0 0 0) 0x7f10bc004b80 con 0x562f4527d1b0 2024-07-26T13:47:54.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.221+0000 7f10d2100740 10 client.17442 fetch_fsmap finished waiting for FSMap version 61 2024-07-26T13:47:54.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.221+0000 7f10d2100740 10 client.17442 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-26T13:47:54.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.221+0000 7f10d2100740 20 client.17442 populate_metadata read hostname 'smithi100' 2024-07-26T13:47:54.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.221+0000 7f10d2100740 1 --2- 172.21.15.100:0/2826747486 >> [v2:172.21.15.167:6836/3519222898,v1:172.21.15.167:6837/3519222898] conn(0x562f452649f0 0x562f456691b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:54.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.221+0000 7f10d2100740 4 client.17442 mds_command: new command op to 26446 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:47:54.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.221+0000 7f10d2100740 1 -- 172.21.15.100:0/2826747486 --> [v2:172.21.15.167:6836/3519222898,v1:172.21.15.167:6837/3519222898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f10bc00fb10 con 0x562f452649f0 2024-07-26T13:47:54.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.221+0000 7f10c9117640 1 --2- 172.21.15.100:0/2826747486 >> [v2:172.21.15.167:6836/3519222898,v1:172.21.15.167:6837/3519222898] conn(0x562f452649f0 0x562f456691b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:54.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.221+0000 7f10c9117640 1 --2- 172.21.15.100:0/2826747486 >> [v2:172.21.15.167:6836/3519222898,v1:172.21.15.167:6837/3519222898] conn(0x562f452649f0 0x562f456691b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a513cb8058de0984 server_cookie=bffc757aaf1ab8ad in_seq=0 out_seq=0 2024-07-26T13:47:54.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.221+0000 7f10a97fa640 10 client.17442 ms_handle_connect on v2:172.21.15.167:6836/3519222898 2024-07-26T13:47:54.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.222+0000 7f10a97fa640 1 -- 172.21.15.100:0/2826747486 <== mds.2 v2:172.21.15.167:6836/3519222898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f10bc00fb10 con 0x562f452649f0 2024-07-26T13:47:54.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.222+0000 7f10a97fa640 10 client.17442 handle_command_reply: tid=0 2024-07-26T13:47:54.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.222+0000 7f10d2100740 1 client.17442 shutdown 2024-07-26T13:47:54.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.223+0000 7f10d2100740 1 -- 172.21.15.100:0/2826747486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f4527d1b0 msgr2=0x562f45657200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:54.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.223+0000 7f10d2100740 1 --2- 172.21.15.100:0/2826747486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f4527d1b0 0x562f45657200 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f10bc000c00 tx=0x7f10bc019490 comp rx=0 tx=0).stop 2024-07-26T13:47:54.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.223+0000 7f10d2100740 1 -- 172.21.15.100:0/2826747486 shutdown_connections 2024-07-26T13:47:54.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.223+0000 7f10d2100740 1 --2- 172.21.15.100:0/2826747486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f452a6070 0x562f4565d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:54.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.223+0000 7f10d2100740 1 --2- 172.21.15.100:0/2826747486 >> [v2:172.21.15.167:6836/3519222898,v1:172.21.15.167:6837/3519222898] conn(0x562f452649f0 0x562f456691b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:54.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.223+0000 7f10d2100740 1 --2- 172.21.15.100:0/2826747486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f45285390 0x562f456577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:54.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.223+0000 7f10d2100740 1 --2- 172.21.15.100:0/2826747486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f4527d1b0 0x562f45657200 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:54.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.223+0000 7f10a97fa640 0 client.17442 ms_handle_reset on v2:172.21.15.167:6836/3519222898 2024-07-26T13:47:54.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.223+0000 7f10d2100740 1 -- 172.21.15.100:0/2826747486 >> 172.21.15.100:0/2826747486 conn(0x562f4529a460 msgr2=0x562f45647c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:54.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.223+0000 7f10d2100740 1 -- 172.21.15.100:0/2826747486 shutdown_connections 2024-07-26T13:47:54.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.223+0000 7f10d2100740 1 -- 172.21.15.100:0/2826747486 wait complete. 2024-07-26T13:47:54.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.223+0000 7f10d2100740 20 client.17442 trim_cache size 0 max 16384 2024-07-26T13:47:54.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.236+0000 7f10d2100740 1 Processor -- start 2024-07-26T13:47:54.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.236+0000 7f10d2100740 1 -- start start 2024-07-26T13:47:54.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.237+0000 7f10d2100740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f4527d1b0 0x562f45648350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:54.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.237+0000 7f10d2100740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f45285390 0x562f4565cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:54.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.237+0000 7f10d2100740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f452a6070 0x562f45648e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:54.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.237+0000 7f10d2100740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f10c41b1d80 con 0x562f45285390 2024-07-26T13:47:54.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.237+0000 7f10d2100740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f10c41af650 con 0x562f4527d1b0 2024-07-26T13:47:54.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.237+0000 7f10d2100740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f10c41ad4a0 con 0x562f452a6070 2024-07-26T13:47:54.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.237+0000 7f10ca119640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f452a6070 0x562f45648e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:54.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.237+0000 7f10ca119640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f452a6070 0x562f45648e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:40982/0 (socket says 172.21.15.100:40982) 2024-07-26T13:47:54.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.237+0000 7f10ca119640 1 -- 172.21.15.100:0/3508543592 learned_addr learned my addr 172.21.15.100:0/3508543592 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:54.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.237+0000 7f10ca119640 1 --2- 172.21.15.100:0/3508543592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f4527d1b0 0x562f45648350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:54.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.237+0000 7f10c9117640 1 --2- 172.21.15.100:0/3508543592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f45285390 0x562f4565cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:54.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.237+0000 7f10ca119640 1 -- 172.21.15.100:0/3508543592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f4527d1b0 msgr2=0x562f45648350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:54.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.237+0000 7f10ca119640 1 --2- 172.21.15.100:0/3508543592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f4527d1b0 0x562f45648350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:54.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.237+0000 7f10ca119640 1 -- 172.21.15.100:0/3508543592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f45285390 msgr2=0x562f4565cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:54.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.237+0000 7f10ca119640 1 --2- 172.21.15.100:0/3508543592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f45285390 0x562f4565cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:54.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.237+0000 7f10ca119640 1 -- 172.21.15.100:0/3508543592 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f45652890 con 0x562f452a6070 2024-07-26T13:47:54.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.237+0000 7f10c9117640 1 --2- 172.21.15.100:0/3508543592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f45285390 0x562f4565cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:47:54.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.237+0000 7f10ca119640 1 --2- 172.21.15.100:0/3508543592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f452a6070 0x562f45648e50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f10bc0059f0 tx=0x7f10bc011a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=912bd499e478a2f8 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:54.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.238+0000 7f10ab7fe640 1 -- 172.21.15.100:0/3508543592 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc01e070 con 0x562f452a6070 2024-07-26T13:47:54.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.238+0000 7f10ab7fe640 1 -- 172.21.15.100:0/3508543592 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10bc00fa70 con 0x562f452a6070 2024-07-26T13:47:54.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.238+0000 7f10ab7fe640 1 -- 172.21.15.100:0/3508543592 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc0052d0 con 0x562f452a6070 2024-07-26T13:47:54.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.238+0000 7f10d2100740 1 -- 172.21.15.100:0/3508543592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f452a6070 msgr2=0x562f45648e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:54.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.238+0000 7f10d2100740 1 --2- 172.21.15.100:0/3508543592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f452a6070 0x562f45648e50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f10bc0059f0 tx=0x7f10bc011a70 comp rx=0 tx=0).stop 2024-07-26T13:47:54.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.239+0000 7f10d2100740 1 -- 172.21.15.100:0/3508543592 shutdown_connections 2024-07-26T13:47:54.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.239+0000 7f10d2100740 1 --2- 172.21.15.100:0/3508543592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f452a6070 0x562f45648e50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:54.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.239+0000 7f10d2100740 1 --2- 172.21.15.100:0/3508543592 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f45285390 0x562f4565cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:54.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.239+0000 7f10d2100740 1 --2- 172.21.15.100:0/3508543592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f4527d1b0 0x562f45648350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:54.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.239+0000 7f10d2100740 1 -- 172.21.15.100:0/3508543592 >> 172.21.15.100:0/3508543592 conn(0x562f4529a460 msgr2=0x562f45647c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:54.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.239+0000 7f10d2100740 1 -- 172.21.15.100:0/3508543592 shutdown_connections 2024-07-26T13:47:54.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.239+0000 7f10d2100740 1 -- 172.21.15.100:0/3508543592 wait complete. 2024-07-26T13:47:54.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.239+0000 7f10d2100740 1 -- start start 2024-07-26T13:47:54.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.239+0000 7f10d2100740 1 Processor -- start 2024-07-26T13:47:54.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.240+0000 7f10d2100740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f4527d1b0 0x562f45663910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:54.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.240+0000 7f10d2100740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f45285390 0x562f4565edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:54.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.240+0000 7f10d2100740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f452a6070 0x562f4565f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:54.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.240+0000 7f10d2100740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f10c41adb10 con 0x562f4527d1b0 2024-07-26T13:47:54.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.240+0000 7f10d2100740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f10c41aef80 con 0x562f452a6070 2024-07-26T13:47:54.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.240+0000 7f10d2100740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f10c41ae1e0 con 0x562f45285390 2024-07-26T13:47:54.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.240+0000 7f10d2100740 10 client.0 _ll_register_callbacks cb 0x7f10bc01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:47:54.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.240+0000 7f10ca119640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f4527d1b0 0x562f45663910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:54.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.240+0000 7f10ca119640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f4527d1b0 0x562f45663910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51954/0 (socket says 172.21.15.100:51954) 2024-07-26T13:47:54.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.240+0000 7f10ca119640 1 -- 172.21.15.100:0/2054756726 learned_addr learned my addr 172.21.15.100:0/2054756726 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:54.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.240+0000 7f10c9117640 1 --2- 172.21.15.100:0/2054756726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f45285390 0x562f4565edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:54.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.240+0000 7f10ca119640 1 --2- 172.21.15.100:0/2054756726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f452a6070 0x562f4565f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:54.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.240+0000 7f10c9117640 1 -- 172.21.15.100:0/2054756726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f452a6070 msgr2=0x562f4565f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:54.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.240+0000 7f10c9117640 1 --2- 172.21.15.100:0/2054756726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f452a6070 0x562f4565f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:54.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.240+0000 7f10c9117640 1 -- 172.21.15.100:0/2054756726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f4527d1b0 msgr2=0x562f45663910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:54.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.240+0000 7f10c9117640 1 --2- 172.21.15.100:0/2054756726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f4527d1b0 0x562f45663910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:54.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.240+0000 7f10c9117640 1 -- 172.21.15.100:0/2054756726 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f4565f840 con 0x562f45285390 2024-07-26T13:47:54.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.240+0000 7f10c9117640 1 --2- 172.21.15.100:0/2054756726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f45285390 0x562f4565edc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f10b8006010 tx=0x7f10b8009480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c0292b709d3c4ef server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:54.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.240+0000 7f10a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-07-26T13:47:54.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.240+0000 7f10a97fa640 1 -- 172.21.15.100:0/2054756726 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b8054060 con 0x562f45285390 2024-07-26T13:47:54.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.241+0000 7f10a97fa640 1 -- 172.21.15.100:0/2054756726 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8828+0+0 (secure 0 0 0) 0x7f10b8031650 con 0x562f45285390 2024-07-26T13:47:54.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.241+0000 7f10d2100740 1 -- 172.21.15.100:0/2054756726 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10bc00fa70 con 0x562f45285390 2024-07-26T13:47:54.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.241+0000 7f10a97fa640 1 -- 172.21.15.100:0/2054756726 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f10b805b050 con 0x562f45285390 2024-07-26T13:47:54.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.241+0000 7f10d2100740 10 client.26836 fetch_fsmap learned FSMap version 61 2024-07-26T13:47:54.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.241+0000 7f10d2100740 1 -- 172.21.15.100:0/2054756726 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x562f456d14d0 con 0x562f45285390 2024-07-26T13:47:54.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.241+0000 7f10a97fa640 1 -- 172.21.15.100:0/2054756726 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10bc00fa70 con 0x562f45285390 2024-07-26T13:47:54.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.241+0000 7f10a97fa640 1 -- 172.21.15.100:0/2054756726 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 61) ==== 8346+0+0 (secure 0 0 0) 0x7f10b804e030 con 0x562f45285390 2024-07-26T13:47:54.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.241+0000 7f10d2100740 10 client.26836 fetch_fsmap finished waiting for FSMap version 61 2024-07-26T13:47:54.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.241+0000 7f10d2100740 10 client.26836 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-07-26T13:47:54.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.241+0000 7f10d2100740 20 client.26836 populate_metadata read hostname 'smithi100' 2024-07-26T13:47:54.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.241+0000 7f10d2100740 1 --2- 172.21.15.100:0/2054756726 >> [v2:172.21.15.167:6836/3519222898,v1:172.21.15.167:6837/3519222898] conn(0x562f45239da0 0x562f456d3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:54.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.241+0000 7f10d2100740 4 client.26836 mds_command: new command op to 26446 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-26T13:47:54.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.241+0000 7f10d2100740 1 -- 172.21.15.100:0/2054756726 --> [v2:172.21.15.167:6836/3519222898,v1:172.21.15.167:6837/3519222898] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562f455c3b00 con 0x562f45239da0 2024-07-26T13:47:54.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.241+0000 7f10ca119640 1 --2- 172.21.15.100:0/2054756726 >> [v2:172.21.15.167:6836/3519222898,v1:172.21.15.167:6837/3519222898] conn(0x562f45239da0 0x562f456d3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:54.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.242+0000 7f10ca119640 1 --2- 172.21.15.100:0/2054756726 >> [v2:172.21.15.167:6836/3519222898,v1:172.21.15.167:6837/3519222898] conn(0x562f45239da0 0x562f456d3cc0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1af92980cc435f9c server_cookie=cacac2f160614ed0 in_seq=0 out_seq=0 2024-07-26T13:47:54.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.242+0000 7f10a97fa640 10 client.26836 ms_handle_connect on v2:172.21.15.167:6836/3519222898 2024-07-26T13:47:54.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.242+0000 7f10a97fa640 1 -- 172.21.15.100:0/2054756726 <== mds.2 v2:172.21.15.167:6836/3519222898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562f455c3b00 con 0x562f45239da0 2024-07-26T13:47:54.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.242+0000 7f10a97fa640 10 client.26836 handle_command_reply: tid=0 2024-07-26T13:47:54.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.242+0000 7f10d2100740 1 client.26836 shutdown 2024-07-26T13:47:54.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.242+0000 7f10d2100740 1 -- 172.21.15.100:0/2054756726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f45285390 msgr2=0x562f4565edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:54.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.242+0000 7f10d2100740 1 --2- 172.21.15.100:0/2054756726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f45285390 0x562f4565edc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f10b8006010 tx=0x7f10b8009480 comp rx=0 tx=0).stop 2024-07-26T13:47:54.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.242+0000 7f10d2100740 1 -- 172.21.15.100:0/2054756726 shutdown_connections 2024-07-26T13:47:54.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.242+0000 7f10d2100740 1 --2- 172.21.15.100:0/2054756726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f452a6070 0x562f4565f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:54.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.242+0000 7f10d2100740 1 --2- 172.21.15.100:0/2054756726 >> [v2:172.21.15.167:6836/3519222898,v1:172.21.15.167:6837/3519222898] conn(0x562f45239da0 0x562f456d3cc0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:54.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.242+0000 7f10d2100740 1 --2- 172.21.15.100:0/2054756726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562f45285390 0x562f4565edc0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:54.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.242+0000 7f10d2100740 1 --2- 172.21.15.100:0/2054756726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562f4527d1b0 0x562f45663910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:54.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.242+0000 7f10a97fa640 0 client.26836 ms_handle_reset on v2:172.21.15.167:6836/3519222898 2024-07-26T13:47:54.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.242+0000 7f10d2100740 1 -- 172.21.15.100:0/2054756726 >> 172.21.15.100:0/2054756726 conn(0x562f4529a460 msgr2=0x562f45648d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:54.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.242+0000 7f10d2100740 1 -- 172.21.15.100:0/2054756726 shutdown_connections 2024-07-26T13:47:54.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.242+0000 7f10d2100740 1 -- 172.21.15.100:0/2054756726 wait complete. 2024-07-26T13:47:54.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.242+0000 7f10d2100740 20 client.26836 trim_cache size 0 max 16384 2024-07-26T13:47:54.248 INFO:teuthology.orchestra.run.smithi100.stdout:[] 2024-07-26T13:47:54.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.243+0000 7f10cbb7c640 1 -- 172.21.15.100:0/2566723902 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f10ac0b3d40 msgr2=0x7f10ac0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:54.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.243+0000 7f10cbb7c640 1 --2- 172.21.15.100:0/2566723902 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f10ac0b3d40 0x7f10ac0b6250 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x562f4564a1f0 tx=0x7f10b4078790 comp rx=0 tx=0).stop 2024-07-26T13:47:54.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.243+0000 7f10cbb7c640 1 -- 172.21.15.100:0/2566723902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10c4102490 msgr2=0x7f10c419aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:54.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.243+0000 7f10cbb7c640 1 --2- 172.21.15.100:0/2566723902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10c4102490 0x7f10c419aca0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f10b400e9f0 tx=0x7f10b400eeb0 comp rx=0 tx=0).stop 2024-07-26T13:47:54.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.243+0000 7f10cbb7c640 1 -- 172.21.15.100:0/2566723902 shutdown_connections 2024-07-26T13:47:54.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.243+0000 7f10cbb7c640 1 --2- 172.21.15.100:0/2566723902 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f10ac0b3d40 0x7f10ac0b6250 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:54.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.243+0000 7f10cbb7c640 1 --2- 172.21.15.100:0/2566723902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f10c4108720 0x7f10c419b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:54.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.243+0000 7f10cbb7c640 1 --2- 172.21.15.100:0/2566723902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10c4107dd0 0x7f10c419b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:54.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.243+0000 7f10cbb7c640 1 --2- 172.21.15.100:0/2566723902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10c4102490 0x7f10c419aca0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:54.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.243+0000 7f10cbb7c640 1 -- 172.21.15.100:0/2566723902 >> 172.21.15.100:0/2566723902 conn(0x7f10c4079160 msgr2=0x7f10c4104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:54.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.243+0000 7f10cbb7c640 1 -- 172.21.15.100:0/2566723902 shutdown_connections 2024-07-26T13:47:54.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:54.243+0000 7f10cbb7c640 1 -- 172.21.15.100:0/2566723902 wait complete. 2024-07-26T13:47:54.426 INFO:tasks.workunit.client.0.smithi100.stdout: 1 333826 4.09 MB/sec execute 248 sec latency 13.176 ms 2024-07-26T13:47:54.922 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:54 smithi100 ceph-mon[32599]: pgmap v604: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.2 MiB/s wr, 283 op/s 2024-07-26T13:47:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:54 smithi138 ceph-mon[36954]: pgmap v604: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.2 MiB/s wr, 283 op/s 2024-07-26T13:47:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:54 smithi167 ceph-mon[37140]: pgmap v604: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.2 MiB/s wr, 283 op/s 2024-07-26T13:47:55.426 INFO:tasks.workunit.client.0.smithi100.stdout: 1 335812 4.10 MB/sec execute 249 sec latency 14.594 ms 2024-07-26T13:47:55.951 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-26T13:47:55.951 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-26T13:47:56.079 INFO:tasks.quiescer.fs.[cephfs]:Total duration (44.4) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-07-26T13:47:56.080 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.2 2024-07-26T13:47:56.080 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:47:56.218 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:47:56.426 INFO:tasks.workunit.client.0.smithi100.stdout: 1 337551 4.13 MB/sec execute 250 sec latency 12.028 ms 2024-07-26T13:47:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:56 smithi138 ceph-mon[36954]: pgmap v605: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 252 op/s 2024-07-26T13:47:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:56 smithi100 ceph-mon[32599]: pgmap v605: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 252 op/s 2024-07-26T13:47:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:56 smithi167 ceph-mon[37140]: pgmap v605: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 252 op/s 2024-07-26T13:47:57.399 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:47:57.426 INFO:tasks.workunit.client.0.smithi100.stdout: 1 339989 4.16 MB/sec execute 251 sec latency 16.163 ms 2024-07-26T13:47:57.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.873+0000 7ff119fe2640 1 -- 172.21.15.100:0/2481006448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff114107dd0 msgr2=0x7ff1141081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:57.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.873+0000 7ff119fe2640 1 --2- 172.21.15.100:0/2481006448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff114107dd0 0x7ff1141081e0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc009900 tx=0x7ff0fc02a410 comp rx=0 tx=0).stop 2024-07-26T13:47:57.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.873+0000 7ff119fe2640 1 -- 172.21.15.100:0/2481006448 shutdown_connections 2024-07-26T13:47:57.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.873+0000 7ff119fe2640 1 --2- 172.21.15.100:0/2481006448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff114108720 0x7ff11410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:57.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.873+0000 7ff119fe2640 1 --2- 172.21.15.100:0/2481006448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff114107dd0 0x7ff1141081e0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:57.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.873+0000 7ff119fe2640 1 --2- 172.21.15.100:0/2481006448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff114102490 0x7ff1141028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:57.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.873+0000 7ff119fe2640 1 -- 172.21.15.100:0/2481006448 >> 172.21.15.100:0/2481006448 conn(0x7ff114079160 msgr2=0x7ff114079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:57.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.873+0000 7ff119fe2640 1 -- 172.21.15.100:0/2481006448 shutdown_connections 2024-07-26T13:47:57.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.873+0000 7ff119fe2640 1 -- 172.21.15.100:0/2481006448 wait complete. 2024-07-26T13:47:57.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.874+0000 7ff119fe2640 1 Processor -- start 2024-07-26T13:47:57.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.874+0000 7ff119fe2640 1 -- start start 2024-07-26T13:47:57.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.874+0000 7ff119fe2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff114102490 0x7ff11419ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:57.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.874+0000 7ff119fe2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff114107dd0 0x7ff11419b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:57.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.874+0000 7ff119fe2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff114108720 0x7ff11419b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:57.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.874+0000 7ff119fe2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff114112350 con 0x7ff114107dd0 2024-07-26T13:47:57.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.874+0000 7ff119fe2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff1141121d0 con 0x7ff114102490 2024-07-26T13:47:57.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.874+0000 7ff119fe2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff1141124d0 con 0x7ff114108720 2024-07-26T13:47:57.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.874+0000 7ff10bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff114107dd0 0x7ff11419b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:57.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.874+0000 7ff10bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff114107dd0 0x7ff11419b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36822/0 (socket says 172.21.15.100:36822) 2024-07-26T13:47:57.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.874+0000 7ff10bfff640 1 -- 172.21.15.100:0/375627364 learned_addr learned my addr 172.21.15.100:0/375627364 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:57.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.874+0000 7ff113fff640 1 --2- 172.21.15.100:0/375627364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff114108720 0x7ff11419b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:57.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.874+0000 7ff1137fe640 1 --2- 172.21.15.100:0/375627364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff114102490 0x7ff11419ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:57.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.874+0000 7ff10bfff640 1 -- 172.21.15.100:0/375627364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff114108720 msgr2=0x7ff11419b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:57.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.874+0000 7ff10bfff640 1 --2- 172.21.15.100:0/375627364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff114108720 0x7ff11419b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:57.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.874+0000 7ff10bfff640 1 -- 172.21.15.100:0/375627364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff114102490 msgr2=0x7ff11419ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:57.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.874+0000 7ff10bfff640 1 --2- 172.21.15.100:0/375627364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff114102490 0x7ff11419ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:57.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.874+0000 7ff10bfff640 1 -- 172.21.15.100:0/375627364 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1141a5fd0 con 0x7ff114107dd0 2024-07-26T13:47:57.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.875+0000 7ff10bfff640 1 --2- 172.21.15.100:0/375627364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff114107dd0 0x7ff11419b280 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc032fd0 tx=0x7ff0fc031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:57.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.875+0000 7ff1117fa640 1 -- 172.21.15.100:0/375627364 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc03f030 con 0x7ff114107dd0 2024-07-26T13:47:57.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.875+0000 7ff1117fa640 1 -- 172.21.15.100:0/375627364 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0fc004490 con 0x7ff114107dd0 2024-07-26T13:47:57.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.875+0000 7ff119fe2640 1 -- 172.21.15.100:0/375627364 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1141a6260 con 0x7ff114107dd0 2024-07-26T13:47:57.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.875+0000 7ff1117fa640 1 -- 172.21.15.100:0/375627364 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc035070 con 0x7ff114107dd0 2024-07-26T13:47:57.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.875+0000 7ff119fe2640 1 -- 172.21.15.100:0/375627364 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1141a6740 con 0x7ff114107dd0 2024-07-26T13:47:57.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.877+0000 7ff120566740 1 Processor -- start 2024-07-26T13:47:57.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.878+0000 7ff120566740 1 -- start start 2024-07-26T13:47:57.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.878+0000 7ff120566740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b6e9ba350 0x555b6ed5f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:57.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.878+0000 7ff120566740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e991490 0x555b6ed5f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:57.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.878+0000 7ff120566740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555b6e999670 0x555b6ed5fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:57.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.878+0000 7ff120566740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x555b6ec472f0 con 0x555b6e999670 2024-07-26T13:47:57.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.878+0000 7ff120566740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555b6ecd5a80 con 0x555b6e9ba350 2024-07-26T13:47:57.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.878+0000 7ff120566740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555b6ecc4f80 con 0x555b6e991490 2024-07-26T13:47:57.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.878+0000 7ff1137fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e991490 0x555b6ed5f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:57.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.878+0000 7ff113fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b6e9ba350 0x555b6ed5f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:57.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.878+0000 7ff1137fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e991490 0x555b6ed5f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:50724/0 (socket says 172.21.15.100:50724) 2024-07-26T13:47:57.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.878+0000 7ff1137fe640 1 -- 172.21.15.100:0/1677522005 learned_addr learned my addr 172.21.15.100:0/1677522005 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:57.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.878+0000 7ff113fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b6e9ba350 0x555b6ed5f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:55708/0 (socket says 172.21.15.100:55708) 2024-07-26T13:47:57.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.878+0000 7ff1117fa640 1 -- 172.21.15.100:0/375627364 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7ff0fc042030 con 0x7ff114107dd0 2024-07-26T13:47:57.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.878+0000 7ff1137fe640 1 -- 172.21.15.100:0/1677522005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b6e9ba350 msgr2=0x555b6ed5f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:57.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.878+0000 7ff1137fe640 1 --2- 172.21.15.100:0/1677522005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b6e9ba350 0x555b6ed5f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:57.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.878+0000 7ff1137fe640 1 -- 172.21.15.100:0/1677522005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555b6e999670 msgr2=0x555b6ed5fe30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-26T13:47:57.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.878+0000 7ff1137fe640 1 --2- 172.21.15.100:0/1677522005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555b6e999670 0x555b6ed5fe30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:57.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.878+0000 7ff1137fe640 1 -- 172.21.15.100:0/1677522005 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b6ea25930 con 0x555b6e991490 2024-07-26T13:47:57.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.878+0000 7ff113fff640 1 --2- 172.21.15.100:0/1677522005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b6e9ba350 0x555b6ed5f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:47:57.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.879+0000 7ff1137fe640 1 --2- 172.21.15.100:0/1677522005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e991490 0x555b6ed5f8f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff104009c90 tx=0x7ff104011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa376abe8fa7125b server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:57.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.879+0000 7ff109ffb640 1 -- 172.21.15.100:0/1677522005 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff104005220 con 0x555b6e991490 2024-07-26T13:47:57.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.879+0000 7ff109ffb640 1 -- 172.21.15.100:0/1677522005 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff104004c50 con 0x555b6e991490 2024-07-26T13:47:57.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.879+0000 7ff109ffb640 1 -- 172.21.15.100:0/1677522005 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff104019670 con 0x555b6e991490 2024-07-26T13:47:57.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.879+0000 7ff1117fa640 1 --2- 172.21.15.100:0/375627364 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff1000b3d70 0x7ff1000b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:57.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.879+0000 7ff113fff640 1 --2- 172.21.15.100:0/375627364 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff1000b3d70 0x7ff1000b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:57.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.879+0000 7ff1117fa640 1 -- 172.21.15.100:0/375627364 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8828+0+0 (secure 0 0 0) 0x7ff0fc104410 con 0x7ff114107dd0 2024-07-26T13:47:57.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.880+0000 7ff120566740 1 -- 172.21.15.100:0/1677522005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e991490 msgr2=0x555b6ed5f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:57.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.880+0000 7ff113fff640 1 --2- 172.21.15.100:0/375627364 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff1000b3d70 0x7ff1000b6280 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff10c009fd0 tx=0x7ff10c009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:57.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.880+0000 7ff120566740 1 --2- 172.21.15.100:0/1677522005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e991490 0x555b6ed5f8f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff104009c90 tx=0x7ff104011810 comp rx=0 tx=0).stop 2024-07-26T13:47:57.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.880+0000 7ff120566740 1 -- 172.21.15.100:0/1677522005 shutdown_connections 2024-07-26T13:47:57.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.880+0000 7ff120566740 1 --2- 172.21.15.100:0/1677522005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555b6e999670 0x555b6ed5fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:57.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.880+0000 7ff120566740 1 --2- 172.21.15.100:0/1677522005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e991490 0x555b6ed5f8f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:57.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.880+0000 7ff120566740 1 --2- 172.21.15.100:0/1677522005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b6e9ba350 0x555b6ed5f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:57.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.880+0000 7ff120566740 1 -- 172.21.15.100:0/1677522005 >> 172.21.15.100:0/1677522005 conn(0x555b6e9ae740 msgr2=0x555b6ed5d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:57.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.880+0000 7ff120566740 1 -- 172.21.15.100:0/1677522005 shutdown_connections 2024-07-26T13:47:57.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.880+0000 7ff120566740 1 -- 172.21.15.100:0/1677522005 wait complete. 2024-07-26T13:47:57.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.880+0000 7ff120566740 1 -- start start 2024-07-26T13:47:57.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.881+0000 7ff120566740 1 Processor -- start 2024-07-26T13:47:57.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.881+0000 7ff120566740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555b6e991490 0x555b6ed6aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:57.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.881+0000 7ff120566740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b6e999670 0x555b6ed6b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:57.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.881+0000 7ff120566740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e9ba350 0x555b6ed76d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:57.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.881+0000 7ff120566740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x555b6ec674f0 con 0x555b6e991490 2024-07-26T13:47:57.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.881+0000 7ff120566740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555b6ecc8fc0 con 0x555b6e999670 2024-07-26T13:47:57.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.881+0000 7ff120566740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555b6eb28890 con 0x555b6e9ba350 2024-07-26T13:47:57.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.881+0000 7ff1137fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555b6e991490 0x555b6ed6aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:57.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.881+0000 7ff120566740 10 client.0 _ll_register_callbacks cb 0x555b6ec71500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:47:57.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.881+0000 7ff10bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b6e999670 0x555b6ed6b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:57.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.881+0000 7ff1137fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555b6e991490 0x555b6ed6aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36830/0 (socket says 172.21.15.100:36830) 2024-07-26T13:47:57.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.881+0000 7ff1137fe640 1 -- 172.21.15.100:0/3218667025 learned_addr learned my addr 172.21.15.100:0/3218667025 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:57.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.881+0000 7ff113fff640 1 --2- 172.21.15.100:0/3218667025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e9ba350 0x555b6ed76d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:57.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.882+0000 7ff1137fe640 1 -- 172.21.15.100:0/3218667025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e9ba350 msgr2=0x555b6ed76d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:57.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.882+0000 7ff1137fe640 1 --2- 172.21.15.100:0/3218667025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e9ba350 0x555b6ed76d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:57.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.882+0000 7ff1137fe640 1 -- 172.21.15.100:0/3218667025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b6e999670 msgr2=0x555b6ed6b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:57.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.882+0000 7ff1137fe640 1 --2- 172.21.15.100:0/3218667025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b6e999670 0x555b6ed6b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:57.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.882+0000 7ff1137fe640 1 -- 172.21.15.100:0/3218667025 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b6ec76c50 con 0x555b6e991490 2024-07-26T13:47:57.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.882+0000 7ff113fff640 1 --2- 172.21.15.100:0/3218667025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e9ba350 0x555b6ed76d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:47:57.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.882+0000 7ff10bfff640 1 --2- 172.21.15.100:0/3218667025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b6e999670 0x555b6ed6b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:47:57.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.882+0000 7ff1137fe640 1 --2- 172.21.15.100:0/3218667025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555b6e991490 0x555b6ed6aed0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7ff1040050b0 tx=0x7ff104005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=180945d7316bd11a server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:57.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.882+0000 7ff0f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:47:57.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.882+0000 7ff0f2ffd640 1 -- 172.21.15.100:0/3218667025 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff104029070 con 0x555b6e991490 2024-07-26T13:47:57.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.882+0000 7ff0f2ffd640 1 -- 172.21.15.100:0/3218667025 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8828+0+0 (secure 0 0 0) 0x7ff104023630 con 0x555b6e991490 2024-07-26T13:47:57.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.882+0000 7ff120566740 1 -- 172.21.15.100:0/3218667025 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff104004c50 con 0x555b6e991490 2024-07-26T13:47:57.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.882+0000 7ff0f2ffd640 1 -- 172.21.15.100:0/3218667025 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7ff104035050 con 0x555b6e991490 2024-07-26T13:47:57.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.882+0000 7ff120566740 10 client.17457 fetch_fsmap learned FSMap version 61 2024-07-26T13:47:57.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.882+0000 7ff120566740 1 -- 172.21.15.100:0/3218667025 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x555b6ed799b0 con 0x555b6e991490 2024-07-26T13:47:57.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.882+0000 7ff0f2ffd640 1 -- 172.21.15.100:0/3218667025 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff104004c50 con 0x555b6e991490 2024-07-26T13:47:57.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.883+0000 7ff0f2ffd640 1 -- 172.21.15.100:0/3218667025 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 61) ==== 8346+0+0 (secure 0 0 0) 0x7ff1040194c0 con 0x555b6e991490 2024-07-26T13:47:57.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.883+0000 7ff120566740 10 client.17457 fetch_fsmap finished waiting for FSMap version 61 2024-07-26T13:47:57.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.883+0000 7ff120566740 10 client.17457 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:47:57.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.883+0000 7ff120566740 20 client.17457 populate_metadata read hostname 'smithi100' 2024-07-26T13:47:57.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.883+0000 7ff120566740 1 --2- 172.21.15.100:0/3218667025 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555b6e978cd0 0x555b6ed7cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:57.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.883+0000 7ff120566740 4 client.17457 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:47:57.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.883+0000 7ff120566740 1 -- 172.21.15.100:0/3218667025 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b6ecd79d0 con 0x555b6e978cd0 2024-07-26T13:47:57.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.883+0000 7ff10bfff640 1 --2- 172.21.15.100:0/3218667025 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555b6e978cd0 0x555b6ed7cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:57.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.883+0000 7ff0f2ffd640 10 client.17457 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:57.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.883+0000 7ff10bfff640 1 --2- 172.21.15.100:0/3218667025 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555b6e978cd0 0x555b6ed7cd10 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=140a261c7c6feff2 server_cookie=2d5abf3f8c1978ad in_seq=0 out_seq=0 2024-07-26T13:47:57.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.884+0000 7ff0f2ffd640 1 -- 172.21.15.100:0/3218667025 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555b6ecd79d0 con 0x555b6e978cd0 2024-07-26T13:47:57.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.884+0000 7ff0f2ffd640 10 client.17457 handle_command_reply: tid=0 2024-07-26T13:47:57.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.884+0000 7ff120566740 1 client.17457 shutdown 2024-07-26T13:47:57.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.884+0000 7ff120566740 1 -- 172.21.15.100:0/3218667025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555b6e991490 msgr2=0x555b6ed6aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:57.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.884+0000 7ff120566740 1 --2- 172.21.15.100:0/3218667025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555b6e991490 0x555b6ed6aed0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7ff1040050b0 tx=0x7ff104005850 comp rx=0 tx=0).stop 2024-07-26T13:47:57.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.884+0000 7ff120566740 1 -- 172.21.15.100:0/3218667025 shutdown_connections 2024-07-26T13:47:57.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.884+0000 7ff120566740 1 --2- 172.21.15.100:0/3218667025 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555b6e978cd0 0x555b6ed7cd10 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:57.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.884+0000 7ff120566740 1 --2- 172.21.15.100:0/3218667025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e9ba350 0x555b6ed76d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:57.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.884+0000 7ff120566740 1 --2- 172.21.15.100:0/3218667025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b6e999670 0x555b6ed6b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:57.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.884+0000 7ff120566740 1 --2- 172.21.15.100:0/3218667025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555b6e991490 0x555b6ed6aed0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:57.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.884+0000 7ff120566740 1 -- 172.21.15.100:0/3218667025 >> 172.21.15.100:0/3218667025 conn(0x555b6e9ae740 msgr2=0x555b6ed62540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:57.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.884+0000 7ff0f2ffd640 0 client.17457 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:57.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.885+0000 7ff120566740 1 -- 172.21.15.100:0/3218667025 shutdown_connections 2024-07-26T13:47:57.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.885+0000 7ff120566740 1 -- 172.21.15.100:0/3218667025 wait complete. 2024-07-26T13:47:57.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.885+0000 7ff120566740 20 client.17457 trim_cache size 0 max 16384 2024-07-26T13:47:57.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.903+0000 7ff120566740 1 Processor -- start 2024-07-26T13:47:57.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.903+0000 7ff120566740 1 -- start start 2024-07-26T13:47:57.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.903+0000 7ff120566740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555b6e999670 0x555b6ed654d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:57.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.903+0000 7ff120566740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b6e9ba350 0x555b6ed69f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:57.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.903+0000 7ff120566740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e94e080 0x555b6ed60af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:57.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.903+0000 7ff120566740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff1141b1d10 con 0x555b6e999670 2024-07-26T13:47:57.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.903+0000 7ff120566740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff1141af5e0 con 0x555b6e9ba350 2024-07-26T13:47:57.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.903+0000 7ff120566740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff1141ad430 con 0x555b6e94e080 2024-07-26T13:47:57.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.903+0000 7ff1137fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555b6e999670 0x555b6ed654d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:57.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.903+0000 7ff10bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b6e9ba350 0x555b6ed69f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:57.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.903+0000 7ff113fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e94e080 0x555b6ed60af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:57.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.903+0000 7ff10bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b6e9ba350 0x555b6ed69f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:55724/0 (socket says 172.21.15.100:55724) 2024-07-26T13:47:57.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.903+0000 7ff10bfff640 1 -- 172.21.15.100:0/2570190260 learned_addr learned my addr 172.21.15.100:0/2570190260 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:57.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.904+0000 7ff10bfff640 1 -- 172.21.15.100:0/2570190260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e94e080 msgr2=0x555b6ed60af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:57.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.904+0000 7ff10bfff640 1 --2- 172.21.15.100:0/2570190260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e94e080 0x555b6ed60af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:57.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.904+0000 7ff10bfff640 1 -- 172.21.15.100:0/2570190260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555b6e999670 msgr2=0x555b6ed654d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:57.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.904+0000 7ff10bfff640 1 --2- 172.21.15.100:0/2570190260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555b6e999670 0x555b6ed654d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:57.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.904+0000 7ff10bfff640 1 -- 172.21.15.100:0/2570190260 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b6ed6bf90 con 0x555b6e9ba350 2024-07-26T13:47:57.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.904+0000 7ff10bfff640 1 --2- 172.21.15.100:0/2570190260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b6e9ba350 0x555b6ed69f10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc0c7020 tx=0x7ff0fc0b2040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67f507d1efd66b95 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:57.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.904+0000 7ff1097fa640 1 -- 172.21.15.100:0/2570190260 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc049040 con 0x555b6e9ba350 2024-07-26T13:47:57.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.904+0000 7ff1097fa640 1 -- 172.21.15.100:0/2570190260 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0fc103070 con 0x555b6e9ba350 2024-07-26T13:47:57.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.904+0000 7ff1097fa640 1 -- 172.21.15.100:0/2570190260 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc0c7230 con 0x555b6e9ba350 2024-07-26T13:47:57.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.905+0000 7ff120566740 1 -- 172.21.15.100:0/2570190260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b6e9ba350 msgr2=0x555b6ed69f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:57.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.905+0000 7ff120566740 1 --2- 172.21.15.100:0/2570190260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b6e9ba350 0x555b6ed69f10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc0c7020 tx=0x7ff0fc0b2040 comp rx=0 tx=0).stop 2024-07-26T13:47:57.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.905+0000 7ff120566740 1 -- 172.21.15.100:0/2570190260 shutdown_connections 2024-07-26T13:47:57.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.905+0000 7ff120566740 1 --2- 172.21.15.100:0/2570190260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e94e080 0x555b6ed60af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:57.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.905+0000 7ff120566740 1 --2- 172.21.15.100:0/2570190260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b6e9ba350 0x555b6ed69f10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:57.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.905+0000 7ff120566740 1 --2- 172.21.15.100:0/2570190260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555b6e999670 0x555b6ed654d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:57.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.905+0000 7ff120566740 1 -- 172.21.15.100:0/2570190260 >> 172.21.15.100:0/2570190260 conn(0x555b6e9ae740 msgr2=0x555b6ed603e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:57.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.905+0000 7ff120566740 1 -- 172.21.15.100:0/2570190260 shutdown_connections 2024-07-26T13:47:57.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.905+0000 7ff120566740 1 -- 172.21.15.100:0/2570190260 wait complete. 2024-07-26T13:47:57.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.906+0000 7ff120566740 1 -- start start 2024-07-26T13:47:57.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.906+0000 7ff120566740 1 Processor -- start 2024-07-26T13:47:57.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.906+0000 7ff120566740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e94e080 0x555b6ed72eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:57.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.906+0000 7ff120566740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b6e999670 0x555b6ed73470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:57.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.906+0000 7ff120566740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555b6e9ba350 0x555b6eddded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:57.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.906+0000 7ff120566740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff1141adaa0 con 0x555b6e9ba350 2024-07-26T13:47:57.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.906+0000 7ff120566740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff1141aef10 con 0x555b6e999670 2024-07-26T13:47:57.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.906+0000 7ff120566740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff1141ae170 con 0x555b6e94e080 2024-07-26T13:47:57.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.906+0000 7ff120566740 10 client.0 _ll_register_callbacks cb 0x7ff1040049f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:47:57.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.906+0000 7ff10bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b6e999670 0x555b6ed73470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:57.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.906+0000 7ff1137fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e94e080 0x555b6ed72eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:57.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.906+0000 7ff113fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555b6e9ba350 0x555b6eddded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:57.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.906+0000 7ff10bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b6e999670 0x555b6ed73470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:55736/0 (socket says 172.21.15.100:55736) 2024-07-26T13:47:57.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.906+0000 7ff1137fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e94e080 0x555b6ed72eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:50750/0 (socket says 172.21.15.100:50750) 2024-07-26T13:47:57.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.906+0000 7ff10bfff640 1 -- 172.21.15.100:0/425583333 learned_addr learned my addr 172.21.15.100:0/425583333 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:57.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.907+0000 7ff10bfff640 1 -- 172.21.15.100:0/425583333 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e94e080 msgr2=0x555b6ed72eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:57.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.907+0000 7ff10bfff640 1 --2- 172.21.15.100:0/425583333 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e94e080 0x555b6ed72eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:57.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.907+0000 7ff10bfff640 1 -- 172.21.15.100:0/425583333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555b6e9ba350 msgr2=0x555b6eddded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:57.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.907+0000 7ff10bfff640 1 --2- 172.21.15.100:0/425583333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555b6e9ba350 0x555b6eddded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:57.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.907+0000 7ff10bfff640 1 -- 172.21.15.100:0/425583333 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b6edde660 con 0x555b6e999670 2024-07-26T13:47:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.907+0000 7ff10bfff640 1 --2- 172.21.15.100:0/425583333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b6e999670 0x555b6ed73470 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc102030 tx=0x7ff0fc03b560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3d73de180e4fcef server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.907+0000 7ff0f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-07-26T13:47:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.907+0000 7ff0f2ffd640 1 -- 172.21.15.100:0/425583333 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc049040 con 0x555b6e999670 2024-07-26T13:47:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.907+0000 7ff0f2ffd640 1 -- 172.21.15.100:0/425583333 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8828+0+0 (secure 0 0 0) 0x7ff0fc101cb0 con 0x555b6e999670 2024-07-26T13:47:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.907+0000 7ff120566740 1 -- 172.21.15.100:0/425583333 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0fc103070 con 0x555b6e999670 2024-07-26T13:47:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.907+0000 7ff0f2ffd640 1 -- 172.21.15.100:0/425583333 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7ff0fc0d3fa0 con 0x555b6e999670 2024-07-26T13:47:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.907+0000 7ff120566740 10 client.26669 fetch_fsmap learned FSMap version 61 2024-07-26T13:47:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.907+0000 7ff120566740 1 -- 172.21.15.100:0/425583333 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x555b6ed70aa0 con 0x555b6e999670 2024-07-26T13:47:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.907+0000 7ff0f2ffd640 1 -- 172.21.15.100:0/425583333 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0fc103070 con 0x555b6e999670 2024-07-26T13:47:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.908+0000 7ff0f2ffd640 1 -- 172.21.15.100:0/425583333 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 61) ==== 8346+0+0 (secure 0 0 0) 0x7ff0fc0cb200 con 0x555b6e999670 2024-07-26T13:47:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.908+0000 7ff120566740 10 client.26669 fetch_fsmap finished waiting for FSMap version 61 2024-07-26T13:47:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.908+0000 7ff120566740 10 client.26669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:47:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.908+0000 7ff120566740 20 client.26669 populate_metadata read hostname 'smithi100' 2024-07-26T13:47:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.908+0000 7ff120566740 1 --2- 172.21.15.100:0/425583333 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555b6ede99a0 0x555b6edebdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.908+0000 7ff120566740 4 client.26669 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-26T13:47:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.908+0000 7ff120566740 1 -- 172.21.15.100:0/425583333 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555b6ecd3470 con 0x555b6ede99a0 2024-07-26T13:47:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.908+0000 7ff1137fe640 1 --2- 172.21.15.100:0/425583333 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555b6ede99a0 0x555b6edebdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.908+0000 7ff0f2ffd640 10 client.26669 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.908+0000 7ff1137fe640 1 --2- 172.21.15.100:0/425583333 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555b6ede99a0 0x555b6edebdc0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a9bd4ee8f024772 server_cookie=f3cc1811e953f76a in_seq=0 out_seq=0 2024-07-26T13:47:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.909+0000 7ff0f2ffd640 1 -- 172.21.15.100:0/425583333 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555b6ecd3470 con 0x555b6ede99a0 2024-07-26T13:47:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.909+0000 7ff0f2ffd640 10 client.26669 handle_command_reply: tid=0 2024-07-26T13:47:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.909+0000 7ff120566740 1 client.26669 shutdown 2024-07-26T13:47:57.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.909+0000 7ff120566740 1 -- 172.21.15.100:0/425583333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b6e999670 msgr2=0x555b6ed73470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:57.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.909+0000 7ff120566740 1 --2- 172.21.15.100:0/425583333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b6e999670 0x555b6ed73470 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc102030 tx=0x7ff0fc03b560 comp rx=0 tx=0).stop 2024-07-26T13:47:57.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.909+0000 7ff120566740 1 -- 172.21.15.100:0/425583333 shutdown_connections 2024-07-26T13:47:57.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.909+0000 7ff120566740 1 --2- 172.21.15.100:0/425583333 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x555b6ede99a0 0x555b6edebdc0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:57.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.909+0000 7ff120566740 1 --2- 172.21.15.100:0/425583333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x555b6e9ba350 0x555b6eddded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:57.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.909+0000 7ff120566740 1 --2- 172.21.15.100:0/425583333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b6e999670 0x555b6ed73470 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:57.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.909+0000 7ff0f2ffd640 0 client.26669 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:57.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.910+0000 7ff120566740 1 --2- 172.21.15.100:0/425583333 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b6e94e080 0x555b6ed72eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:57.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.910+0000 7ff120566740 1 -- 172.21.15.100:0/425583333 >> 172.21.15.100:0/425583333 conn(0x555b6e9ae740 msgr2=0x555b6ed69ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:57.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.910+0000 7ff120566740 1 -- 172.21.15.100:0/425583333 shutdown_connections 2024-07-26T13:47:57.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.910+0000 7ff120566740 1 -- 172.21.15.100:0/425583333 wait complete. 2024-07-26T13:47:57.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.910+0000 7ff120566740 20 client.26669 trim_cache size 0 max 16384 2024-07-26T13:47:57.915 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:47:57.915 INFO:teuthology.orchestra.run.smithi100.stdout: "return_code": 0, 2024-07-26T13:47:57.916 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_tag": "016474ca-9cc6-47e4-bc6c-e21286f834e1", 2024-07-26T13:47:57.916 INFO:teuthology.orchestra.run.smithi100.stdout: "mode": "asynchronous" 2024-07-26T13:47:57.916 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:47:57.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.910+0000 7ff119fe2640 1 -- 172.21.15.100:0/375627364 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff1000b3d70 msgr2=0x7ff1000b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:57.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.910+0000 7ff119fe2640 1 --2- 172.21.15.100:0/375627364 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff1000b3d70 0x7ff1000b6280 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff10c009fd0 tx=0x7ff10c009500 comp rx=0 tx=0).stop 2024-07-26T13:47:57.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.910+0000 7ff119fe2640 1 -- 172.21.15.100:0/375627364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff114107dd0 msgr2=0x7ff11419b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:57.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.910+0000 7ff119fe2640 1 --2- 172.21.15.100:0/375627364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff114107dd0 0x7ff11419b280 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc032fd0 tx=0x7ff0fc031a20 comp rx=0 tx=0).stop 2024-07-26T13:47:57.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.911+0000 7ff119fe2640 1 -- 172.21.15.100:0/375627364 shutdown_connections 2024-07-26T13:47:57.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.911+0000 7ff119fe2640 1 --2- 172.21.15.100:0/375627364 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff1000b3d70 0x7ff1000b6280 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:57.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.911+0000 7ff119fe2640 1 --2- 172.21.15.100:0/375627364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff114108720 0x7ff11419b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:57.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.911+0000 7ff119fe2640 1 --2- 172.21.15.100:0/375627364 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff114107dd0 0x7ff11419b280 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:57.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.911+0000 7ff119fe2640 1 --2- 172.21.15.100:0/375627364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff114102490 0x7ff11419ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:57.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.911+0000 7ff119fe2640 1 -- 172.21.15.100:0/375627364 >> 172.21.15.100:0/375627364 conn(0x7ff114079160 msgr2=0x7ff114104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:57.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.911+0000 7ff119fe2640 1 -- 172.21.15.100:0/375627364 shutdown_connections 2024-07-26T13:47:57.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:57.911+0000 7ff119fe2640 1 -- 172.21.15.100:0/375627364 wait complete. 2024-07-26T13:47:58.426 INFO:tasks.workunit.client.0.smithi100.stdout: 1 341586 4.18 MB/sec execute 252 sec latency 21.615 ms 2024-07-26T13:47:58.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:58 smithi100 ceph-mon[32599]: scrub summary: active paths [/] 2024-07-26T13:47:58.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:58 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:47:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.606+0000 7f6f975eb640 1 -- 172.21.15.100:0/103037079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f90107d30 msgr2=0x7f6f90108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.606+0000 7f6f975eb640 1 --2- 172.21.15.100:0/103037079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f90107d30 0x7f6f90108140 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f6f80009900 tx=0x7f6f8002a410 comp rx=0 tx=0).stop 2024-07-26T13:47:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.606+0000 7f6f975eb640 1 -- 172.21.15.100:0/103037079 shutdown_connections 2024-07-26T13:47:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.606+0000 7f6f975eb640 1 --2- 172.21.15.100:0/103037079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f90108680 0x7f6f9010eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.606+0000 7f6f975eb640 1 --2- 172.21.15.100:0/103037079 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f90107d30 0x7f6f90108140 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.606+0000 7f6f975eb640 1 --2- 172.21.15.100:0/103037079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f90102400 0x7f6f90102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.606+0000 7f6f975eb640 1 -- 172.21.15.100:0/103037079 >> 172.21.15.100:0/103037079 conn(0x7f6f900fde40 msgr2=0x7f6f90100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.607+0000 7f6f975eb640 1 -- 172.21.15.100:0/103037079 shutdown_connections 2024-07-26T13:47:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.607+0000 7f6f975eb640 1 -- 172.21.15.100:0/103037079 wait complete. 2024-07-26T13:47:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.607+0000 7f6f975eb640 1 Processor -- start 2024-07-26T13:47:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.607+0000 7f6f975eb640 1 -- start start 2024-07-26T13:47:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.607+0000 7f6f975eb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f90102400 0x7f6f90198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.607+0000 7f6f975eb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f90107d30 0x7f6f90199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.607+0000 7f6f975eb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f90108680 0x7f6f9019e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.607+0000 7f6f975eb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6f90112310 con 0x7f6f90108680 2024-07-26T13:47:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.607+0000 7f6f975eb640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6f90112190 con 0x7f6f90107d30 2024-07-26T13:47:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.607+0000 7f6f975eb640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6f90112490 con 0x7f6f90102400 2024-07-26T13:47:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.608+0000 7f6f95b88640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f90108680 0x7f6f9019e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.608+0000 7f6f95387640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f90102400 0x7f6f90198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.608+0000 7f6f95b88640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f90108680 0x7f6f9019e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36862/0 (socket says 172.21.15.100:36862) 2024-07-26T13:47:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.608+0000 7f6f95b88640 1 -- 172.21.15.100:0/3789491418 learned_addr learned my addr 172.21.15.100:0/3789491418 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.608+0000 7f6f94b86640 1 --2- 172.21.15.100:0/3789491418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f90107d30 0x7f6f90199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.608+0000 7f6f95b88640 1 -- 172.21.15.100:0/3789491418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f90102400 msgr2=0x7f6f90198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.608+0000 7f6f95b88640 1 --2- 172.21.15.100:0/3789491418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f90102400 0x7f6f90198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.608+0000 7f6f95b88640 1 -- 172.21.15.100:0/3789491418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f90107d30 msgr2=0x7f6f90199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.608+0000 7f6f95b88640 1 --2- 172.21.15.100:0/3789491418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f90107d30 0x7f6f90199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.608+0000 7f6f95b88640 1 -- 172.21.15.100:0/3789491418 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f9019ec10 con 0x7f6f90108680 2024-07-26T13:47:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.608+0000 7f6f95b88640 1 --2- 172.21.15.100:0/3789491418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f90108680 0x7f6f9019e5a0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f6f8800eb10 tx=0x7f6f8800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.608+0000 7f6f867fc640 1 -- 172.21.15.100:0/3789491418 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8800ce20 con 0x7f6f90108680 2024-07-26T13:47:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.608+0000 7f6f867fc640 1 -- 172.21.15.100:0/3789491418 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f88005020 con 0x7f6f90108680 2024-07-26T13:47:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.608+0000 7f6f975eb640 1 -- 172.21.15.100:0/3789491418 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f9019ef00 con 0x7f6f90108680 2024-07-26T13:47:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.608+0000 7f6f867fc640 1 -- 172.21.15.100:0/3789491418 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f88010690 con 0x7f6f90108680 2024-07-26T13:47:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.608+0000 7f6f975eb640 1 -- 172.21.15.100:0/3789491418 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f901a5720 con 0x7f6f90108680 2024-07-26T13:47:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.611+0000 7f6f9e5e9740 1 Processor -- start 2024-07-26T13:47:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.611+0000 7f6f867fc640 1 -- 172.21.15.100:0/3789491418 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f6f88005510 con 0x7f6f90108680 2024-07-26T13:47:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.611+0000 7f6f9e5e9740 1 -- start start 2024-07-26T13:47:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.611+0000 7f6f9e5e9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f58bf121b0 0x55f58c2b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.611+0000 7f6f9e5e9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f58bee92f0 0x55f58c2b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.611+0000 7f6f9e5e9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f58bef14d0 0x55f58c2b5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.611+0000 7f6f9e5e9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55f58c22a3b0 con 0x55f58bef14d0 2024-07-26T13:47:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.611+0000 7f6f9e5e9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f58c2198b0 con 0x55f58bee92f0 2024-07-26T13:47:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.611+0000 7f6f9e5e9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f58c21d8f0 con 0x55f58bf121b0 2024-07-26T13:47:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.612+0000 7f6f94b86640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f58bf121b0 0x55f58c2b46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:58.620 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.612+0000 7f6f94b86640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f58bf121b0 0x55f58c2b46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:50780/0 (socket says 172.21.15.100:50780) 2024-07-26T13:47:58.620 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.612+0000 7f6f94b86640 1 -- 172.21.15.100:0/3219838951 learned_addr learned my addr 172.21.15.100:0/3219838951 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:58.620 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.612+0000 7f6f94b86640 1 --2- 172.21.15.100:0/3219838951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f58bef14d0 0x55f58c2b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:58.620 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.612+0000 7f6f95387640 1 --2- 172.21.15.100:0/3219838951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f58bee92f0 0x55f58c2b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:58.620 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.612+0000 7f6f94b86640 1 -- 172.21.15.100:0/3219838951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f58bee92f0 msgr2=0x55f58c2b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:58.620 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.612+0000 7f6f867fc640 1 --2- 172.21.15.100:0/3789491418 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6f7c0b3d40 0x7f6f7c0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:58.620 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.612+0000 7f6f94b86640 1 --2- 172.21.15.100:0/3219838951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f58bee92f0 0x55f58c2b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:58.620 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.612+0000 7f6f94b86640 1 -- 172.21.15.100:0/3219838951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f58bef14d0 msgr2=0x55f58c2b5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:58.620 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.612+0000 7f6f94b86640 1 --2- 172.21.15.100:0/3219838951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f58bef14d0 0x55f58c2b5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:58.620 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.612+0000 7f6f94b86640 1 -- 172.21.15.100:0/3219838951 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f58c1c09a0 con 0x55f58bf121b0 2024-07-26T13:47:58.620 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.612+0000 7f6f95b88640 1 --2- 172.21.15.100:0/3789491418 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6f7c0b3d40 0x7f6f7c0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:58.620 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.612+0000 7f6f94b86640 1 --2- 172.21.15.100:0/3219838951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f58bf121b0 0x55f58c2b46b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6f800096c0 tx=0x7f6f80009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56850b27972bcf43 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:58.620 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.612+0000 7f6f867fc640 1 -- 172.21.15.100:0/3789491418 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8828+0+0 (secure 0 0 0) 0x7f6f880e48e0 con 0x7f6f90108680 2024-07-26T13:47:58.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.612+0000 7f6f7b7fe640 1 -- 172.21.15.100:0/3219838951 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f80035040 con 0x55f58bf121b0 2024-07-26T13:47:58.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.612+0000 7f6f7b7fe640 1 -- 172.21.15.100:0/3219838951 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f800035f0 con 0x55f58bf121b0 2024-07-26T13:47:58.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.612+0000 7f6f7b7fe640 1 -- 172.21.15.100:0/3219838951 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f800044e0 con 0x55f58bf121b0 2024-07-26T13:47:58.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.613+0000 7f6f9e5e9740 1 -- 172.21.15.100:0/3219838951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f58bf121b0 msgr2=0x55f58c2b46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:58.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.613+0000 7f6f95b88640 1 --2- 172.21.15.100:0/3789491418 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6f7c0b3d40 0x7f6f7c0b6250 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6f880ab350 tx=0x7f6f88078b10 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:58.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.613+0000 7f6f9e5e9740 1 --2- 172.21.15.100:0/3219838951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f58bf121b0 0x55f58c2b46b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6f800096c0 tx=0x7f6f80009260 comp rx=0 tx=0).stop 2024-07-26T13:47:58.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.613+0000 7f6f9e5e9740 1 -- 172.21.15.100:0/3219838951 shutdown_connections 2024-07-26T13:47:58.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.613+0000 7f6f9e5e9740 1 --2- 172.21.15.100:0/3219838951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f58bef14d0 0x55f58c2b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:58.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.613+0000 7f6f9e5e9740 1 --2- 172.21.15.100:0/3219838951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f58bee92f0 0x55f58c2b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:58.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.613+0000 7f6f9e5e9740 1 --2- 172.21.15.100:0/3219838951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f58bf121b0 0x55f58c2b46b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:58.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.613+0000 7f6f9e5e9740 1 -- 172.21.15.100:0/3219838951 >> 172.21.15.100:0/3219838951 conn(0x55f58bf06300 msgr2=0x55f58c2b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:58.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.613+0000 7f6f9e5e9740 1 -- 172.21.15.100:0/3219838951 shutdown_connections 2024-07-26T13:47:58.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.613+0000 7f6f9e5e9740 1 -- 172.21.15.100:0/3219838951 wait complete. 2024-07-26T13:47:58.621 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.614+0000 7f6f9e5e9740 1 -- start start 2024-07-26T13:47:58.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.614+0000 7f6f9e5e9740 1 Processor -- start 2024-07-26T13:47:58.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.614+0000 7f6f9e5e9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f58bee92f0 0x55f58c2c20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:58.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.614+0000 7f6f9e5e9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f58bef14d0 0x55f58c2c2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:58.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.614+0000 7f6f9e5e9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f58bf121b0 0x55f58c2ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:58.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.614+0000 7f6f9e5e9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55f58c153ab0 con 0x55f58bf121b0 2024-07-26T13:47:58.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.614+0000 7f6f9e5e9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f58c1a1cf0 con 0x55f58bef14d0 2024-07-26T13:47:58.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.614+0000 7f6f9e5e9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f58c2c4fe0 con 0x55f58bee92f0 2024-07-26T13:47:58.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.614+0000 7f6f9e5e9740 10 client.0 _ll_register_callbacks cb 0x55f58c2133e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:47:58.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.615+0000 7f6f94b86640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f58bf121b0 0x55f58c2ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:58.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.615+0000 7f6f94b86640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f58bee92f0 0x55f58c2c20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:58.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.615+0000 7f6f94b86640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f58bee92f0 0x55f58c2c20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.100:50796/0 (socket says 172.21.15.100:50796) 2024-07-26T13:47:58.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.615+0000 7f6f94b86640 1 -- 172.21.15.100:0/3119952013 learned_addr learned my addr 172.21.15.100:0/3119952013 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:58.622 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.615+0000 7f6f95387640 1 --2- 172.21.15.100:0/3119952013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f58bef14d0 0x55f58c2c2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:58.623 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.615+0000 7f6f94b86640 1 -- 172.21.15.100:0/3119952013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f58bef14d0 msgr2=0x55f58c2c2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:58.623 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.615+0000 7f6f94b86640 1 --2- 172.21.15.100:0/3119952013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f58bef14d0 0x55f58c2c2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:58.623 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.615+0000 7f6f94b86640 1 -- 172.21.15.100:0/3119952013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f58bf121b0 msgr2=0x55f58c2ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:58.623 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.615+0000 7f6f94b86640 1 --2- 172.21.15.100:0/3119952013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f58bf121b0 0x55f58c2ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:58.623 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.615+0000 7f6f94b86640 1 -- 172.21.15.100:0/3119952013 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f58bf7d790 con 0x55f58bee92f0 2024-07-26T13:47:58.623 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.615+0000 7f6f95387640 1 --2- 172.21.15.100:0/3119952013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f58bef14d0 0x55f58c2c2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:47:58.623 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.615+0000 7f6f94b86640 1 --2- 172.21.15.100:0/3119952013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f58bee92f0 0x55f58c2c20e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6f80006010 tx=0x7f6f80000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d608ce5b6027e4fc server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:58.623 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.615+0000 7f6f78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-07-26T13:47:58.623 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.615+0000 7f6f78ff9640 1 -- 172.21.15.100:0/3119952013 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8004f020 con 0x55f58bee92f0 2024-07-26T13:47:58.624 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.615+0000 7f6f9e5e9740 1 -- 172.21.15.100:0/3119952013 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f800035f0 con 0x55f58bee92f0 2024-07-26T13:47:58.624 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.615+0000 7f6f78ff9640 1 -- 172.21.15.100:0/3119952013 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8828+0+0 (secure 0 0 0) 0x7f6f80004430 con 0x55f58bee92f0 2024-07-26T13:47:58.624 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.615+0000 7f6f78ff9640 1 -- 172.21.15.100:0/3119952013 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f6f80035070 con 0x55f58bee92f0 2024-07-26T13:47:58.624 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.615+0000 7f6f9e5e9740 10 client.26866 fetch_fsmap learned FSMap version 61 2024-07-26T13:47:58.624 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.615+0000 7f6f9e5e9740 1 -- 172.21.15.100:0/3119952013 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55f58bff6620 con 0x55f58bee92f0 2024-07-26T13:47:58.624 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.616+0000 7f6f78ff9640 1 -- 172.21.15.100:0/3119952013 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f800035f0 con 0x55f58bee92f0 2024-07-26T13:47:58.624 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.616+0000 7f6f78ff9640 1 -- 172.21.15.100:0/3119952013 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 61) ==== 8346+0+0 (secure 0 0 0) 0x7f6f8003a040 con 0x55f58bee92f0 2024-07-26T13:47:58.624 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.616+0000 7f6f9e5e9740 10 client.26866 fetch_fsmap finished waiting for FSMap version 61 2024-07-26T13:47:58.624 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.616+0000 7f6f9e5e9740 10 client.26866 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:47:58.624 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.616+0000 7f6f9e5e9740 20 client.26866 populate_metadata read hostname 'smithi100' 2024-07-26T13:47:58.624 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.616+0000 7f6f9e5e9740 1 --2- 172.21.15.100:0/3119952013 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f58bed0d80 0x55f58c2d4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:58.624 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.616+0000 7f6f9e5e9740 4 client.26866 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:47:58.624 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.616+0000 7f6f9e5e9740 1 -- 172.21.15.100:0/3119952013 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f58c22c300 con 0x55f58bed0d80 2024-07-26T13:47:58.624 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.616+0000 7f6f95387640 1 --2- 172.21.15.100:0/3119952013 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f58bed0d80 0x55f58c2d4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.617+0000 7f6f78ff9640 10 client.26866 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.617+0000 7f6f95387640 1 --2- 172.21.15.100:0/3119952013 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f58bed0d80 0x55f58c2d4050 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b3b8ca1f82e3c79 server_cookie=8f3fa6de7acc764c in_seq=0 out_seq=0 2024-07-26T13:47:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.617+0000 7f6f78ff9640 1 -- 172.21.15.100:0/3119952013 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f58c22c300 con 0x55f58bed0d80 2024-07-26T13:47:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.617+0000 7f6f78ff9640 10 client.26866 handle_command_reply: tid=0 2024-07-26T13:47:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.618+0000 7f6f9e5e9740 1 client.26866 shutdown 2024-07-26T13:47:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.618+0000 7f6f9e5e9740 1 -- 172.21.15.100:0/3119952013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f58bee92f0 msgr2=0x55f58c2c20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.618+0000 7f6f9e5e9740 1 --2- 172.21.15.100:0/3119952013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f58bee92f0 0x55f58c2c20e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6f80006010 tx=0x7f6f80000bc0 comp rx=0 tx=0).stop 2024-07-26T13:47:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.618+0000 7f6f9e5e9740 1 -- 172.21.15.100:0/3119952013 shutdown_connections 2024-07-26T13:47:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.618+0000 7f6f9e5e9740 1 --2- 172.21.15.100:0/3119952013 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f58bed0d80 0x55f58c2d4050 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.618+0000 7f6f9e5e9740 1 --2- 172.21.15.100:0/3119952013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f58bf121b0 0x55f58c2ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.618+0000 7f6f9e5e9740 1 --2- 172.21.15.100:0/3119952013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f58bef14d0 0x55f58c2c2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.618+0000 7f6f9e5e9740 1 --2- 172.21.15.100:0/3119952013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f58bee92f0 0x55f58c2c20e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.618+0000 7f6f78ff9640 0 client.26866 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.618+0000 7f6f9e5e9740 1 -- 172.21.15.100:0/3119952013 >> 172.21.15.100:0/3119952013 conn(0x55f58bf06300 msgr2=0x55f58c2b60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.618+0000 7f6f9e5e9740 1 -- 172.21.15.100:0/3119952013 shutdown_connections 2024-07-26T13:47:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.618+0000 7f6f9e5e9740 1 -- 172.21.15.100:0/3119952013 wait complete. 2024-07-26T13:47:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.618+0000 7f6f9e5e9740 20 client.26866 trim_cache size 0 max 16384 2024-07-26T13:47:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.634+0000 7f6f9e5e9740 1 Processor -- start 2024-07-26T13:47:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.634+0000 7f6f9e5e9740 1 -- start start 2024-07-26T13:47:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.634+0000 7f6f9e5e9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f58bee92f0 0x55f58c2b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.634+0000 7f6f9e5e9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f58bef14d0 0x55f58c2d7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.634+0000 7f6f9e5e9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f58bf121b0 0x55f58c2d75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.634+0000 7f6f9e5e9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6f901af950 con 0x55f58bee92f0 2024-07-26T13:47:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.634+0000 7f6f9e5e9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6f901ad220 con 0x55f58bef14d0 2024-07-26T13:47:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.634+0000 7f6f9e5e9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6f901ab090 con 0x55f58bf121b0 2024-07-26T13:47:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.634+0000 7f6f94b86640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f58bee92f0 0x55f58c2b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.635+0000 7f6f94b86640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f58bee92f0 0x55f58c2b5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36898/0 (socket says 172.21.15.100:36898) 2024-07-26T13:47:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.635+0000 7f6f94b86640 1 -- 172.21.15.100:0/1083828245 learned_addr learned my addr 172.21.15.100:0/1083828245 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.635+0000 7f6f94b86640 1 --2- 172.21.15.100:0/1083828245 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f58bf121b0 0x55f58c2d75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.635+0000 7f6f94b86640 1 -- 172.21.15.100:0/1083828245 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f58bf121b0 msgr2=0x55f58c2d75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.635+0000 7f6f94b86640 1 --2- 172.21.15.100:0/1083828245 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f58bf121b0 0x55f58c2d75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.635+0000 7f6f94b86640 1 -- 172.21.15.100:0/1083828245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f58bef14d0 msgr2=0x55f58c2d7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.635+0000 7f6f94b86640 1 --2- 172.21.15.100:0/1083828245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f58bef14d0 0x55f58c2d7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.635+0000 7f6f94b86640 1 -- 172.21.15.100:0/1083828245 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f58c2ac730 con 0x55f58bee92f0 2024-07-26T13:47:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.635+0000 7f6f94b86640 1 --2- 172.21.15.100:0/1083828245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f58bee92f0 0x55f58c2b5df0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f6f8002a930 tx=0x7f6f80005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49bfcbc5f6b88b7b server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.635+0000 7f6f7affd640 1 -- 172.21.15.100:0/1083828245 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8004f070 con 0x55f58bee92f0 2024-07-26T13:47:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.635+0000 7f6f7affd640 1 -- 172.21.15.100:0/1083828245 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f800316c0 con 0x55f58bee92f0 2024-07-26T13:47:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.635+0000 7f6f7affd640 1 -- 172.21.15.100:0/1083828245 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f800035f0 con 0x55f58bee92f0 2024-07-26T13:47:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.636+0000 7f6f9e5e9740 1 -- 172.21.15.100:0/1083828245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f58bee92f0 msgr2=0x55f58c2b5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.636+0000 7f6f9e5e9740 1 --2- 172.21.15.100:0/1083828245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f58bee92f0 0x55f58c2b5df0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f6f8002a930 tx=0x7f6f80005a20 comp rx=0 tx=0).stop 2024-07-26T13:47:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.636+0000 7f6f9e5e9740 1 -- 172.21.15.100:0/1083828245 shutdown_connections 2024-07-26T13:47:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.636+0000 7f6f9e5e9740 1 --2- 172.21.15.100:0/1083828245 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f58bf121b0 0x55f58c2d75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.636+0000 7f6f9e5e9740 1 --2- 172.21.15.100:0/1083828245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f58bef14d0 0x55f58c2d7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.636+0000 7f6f9e5e9740 1 --2- 172.21.15.100:0/1083828245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f58bee92f0 0x55f58c2b5df0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.636+0000 7f6f9e5e9740 1 -- 172.21.15.100:0/1083828245 >> 172.21.15.100:0/1083828245 conn(0x55f58bf06300 msgr2=0x55f58c2b56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.636+0000 7f6f9e5e9740 1 -- 172.21.15.100:0/1083828245 shutdown_connections 2024-07-26T13:47:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.636+0000 7f6f9e5e9740 1 -- 172.21.15.100:0/1083828245 wait complete. 2024-07-26T13:47:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.636+0000 7f6f9e5e9740 1 -- start start 2024-07-26T13:47:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.637+0000 7f6f9e5e9740 1 Processor -- start 2024-07-26T13:47:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.637+0000 7f6f9e5e9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f58bee92f0 0x55f58c32e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.637+0000 7f6f9e5e9740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f58bef14d0 0x55f58c32e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.637+0000 7f6f9e5e9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f58bf121b0 0x55f58c333620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.637+0000 7f6f9e5e9740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6f901ab6e0 con 0x55f58bef14d0 2024-07-26T13:47:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.637+0000 7f6f9e5e9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6f901acb50 con 0x55f58bf121b0 2024-07-26T13:47:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.637+0000 7f6f9e5e9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6f901abdb0 con 0x55f58bee92f0 2024-07-26T13:47:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.637+0000 7f6f9e5e9740 10 client.0 _ll_register_callbacks cb 0x7f6f80009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:47:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.637+0000 7f6f95387640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f58bef14d0 0x55f58c32e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.637+0000 7f6f95387640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f58bef14d0 0x55f58c32e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36902/0 (socket says 172.21.15.100:36902) 2024-07-26T13:47:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.637+0000 7f6f95387640 1 -- 172.21.15.100:0/1527396007 learned_addr learned my addr 172.21.15.100:0/1527396007 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.637+0000 7f6f94b86640 1 --2- 172.21.15.100:0/1527396007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f58bee92f0 0x55f58c32e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.637+0000 7f6f94b86640 1 --2- 172.21.15.100:0/1527396007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f58bf121b0 0x55f58c333620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.637+0000 7f6f95387640 1 -- 172.21.15.100:0/1527396007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f58bee92f0 msgr2=0x55f58c32e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.637+0000 7f6f95387640 1 --2- 172.21.15.100:0/1527396007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f58bee92f0 0x55f58c32e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.637+0000 7f6f95387640 1 -- 172.21.15.100:0/1527396007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f58bf121b0 msgr2=0x55f58c333620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:47:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.637+0000 7f6f95387640 1 --2- 172.21.15.100:0/1527396007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f58bf121b0 0x55f58c333620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.637+0000 7f6f95387640 1 -- 172.21.15.100:0/1527396007 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f58c333cc0 con 0x55f58bef14d0 2024-07-26T13:47:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.637+0000 7f6f94b86640 1 --2- 172.21.15.100:0/1527396007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f58bf121b0 0x55f58c333620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:47:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.638+0000 7f6f95387640 1 --2- 172.21.15.100:0/1527396007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f58bef14d0 0x55f58c32e610 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x55f58c1a1dd0 tx=0x7f6f8c010f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2fa31e123f35d6 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.638+0000 7f6f78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:47:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.638+0000 7f6f78ff9640 1 -- 172.21.15.100:0/1527396007 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c027410 con 0x55f58bef14d0 2024-07-26T13:47:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.638+0000 7f6f9e5e9740 1 -- 172.21.15.100:0/1527396007 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f800316c0 con 0x55f58bef14d0 2024-07-26T13:47:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.638+0000 7f6f78ff9640 1 -- 172.21.15.100:0/1527396007 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8828+0+0 (secure 0 0 0) 0x7f6f8c01d3f0 con 0x55f58bef14d0 2024-07-26T13:47:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.638+0000 7f6f78ff9640 1 -- 172.21.15.100:0/1527396007 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f6f8c01d5f0 con 0x55f58bef14d0 2024-07-26T13:47:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.638+0000 7f6f9e5e9740 10 client.17481 fetch_fsmap learned FSMap version 61 2024-07-26T13:47:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.638+0000 7f6f9e5e9740 1 -- 172.21.15.100:0/1527396007 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55f58c33b140 con 0x55f58bef14d0 2024-07-26T13:47:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.638+0000 7f6f78ff9640 1 -- 172.21.15.100:0/1527396007 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f800316c0 con 0x55f58bef14d0 2024-07-26T13:47:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.638+0000 7f6f78ff9640 1 -- 172.21.15.100:0/1527396007 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 61) ==== 8346+0+0 (secure 0 0 0) 0x7f6f8c027ae0 con 0x55f58bef14d0 2024-07-26T13:47:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.638+0000 7f6f9e5e9740 10 client.17481 fetch_fsmap finished waiting for FSMap version 61 2024-07-26T13:47:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.638+0000 7f6f9e5e9740 10 client.17481 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:47:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.638+0000 7f6f9e5e9740 20 client.17481 populate_metadata read hostname 'smithi100' 2024-07-26T13:47:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.638+0000 7f6f9e5e9740 1 --2- 172.21.15.100:0/1527396007 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f58bea61c0 0x55f58c33d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.638+0000 7f6f9e5e9740 4 client.17481 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-07-26T13:47:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.638+0000 7f6f9e5e9740 1 -- 172.21.15.100:0/1527396007 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f58c227da0 con 0x55f58bea61c0 2024-07-26T13:47:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.639+0000 7f6f94b86640 1 --2- 172.21.15.100:0/1527396007 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f58bea61c0 0x55f58c33d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.639+0000 7f6f94b86640 1 --2- 172.21.15.100:0/1527396007 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f58bea61c0 0x55f58c33d930 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68356bdbf5a474bb server_cookie=c431c122ac39d535 in_seq=0 out_seq=0 2024-07-26T13:47:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.639+0000 7f6f78ff9640 10 client.17481 ms_handle_connect on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:58.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:58 smithi138 ceph-mon[36954]: scrub summary: active paths [/] 2024-07-26T13:47:58.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:58 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:47:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.719+0000 7f6f78ff9640 1 -- 172.21.15.100:0/1527396007 <== mds.0 v2:172.21.15.167:6838/398150514 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f58c227da0 con 0x55f58bea61c0 2024-07-26T13:47:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.719+0000 7f6f78ff9640 10 client.17481 handle_command_reply: tid=0 2024-07-26T13:47:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.719+0000 7f6f9e5e9740 1 client.17481 shutdown 2024-07-26T13:47:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.719+0000 7f6f9e5e9740 1 -- 172.21.15.100:0/1527396007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f58bef14d0 msgr2=0x55f58c32e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.719+0000 7f6f9e5e9740 1 --2- 172.21.15.100:0/1527396007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f58bef14d0 0x55f58c32e610 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x55f58c1a1dd0 tx=0x7f6f8c010f00 comp rx=0 tx=0).stop 2024-07-26T13:47:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.719+0000 7f6f9e5e9740 1 -- 172.21.15.100:0/1527396007 shutdown_connections 2024-07-26T13:47:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.719+0000 7f6f9e5e9740 1 --2- 172.21.15.100:0/1527396007 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55f58bea61c0 0x55f58c33d930 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.719+0000 7f6f9e5e9740 1 --2- 172.21.15.100:0/1527396007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f58bf121b0 0x55f58c333620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.719+0000 7f6f9e5e9740 1 --2- 172.21.15.100:0/1527396007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f58bef14d0 0x55f58c32e610 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.719+0000 7f6f78ff9640 0 client.17481 ms_handle_reset on v2:172.21.15.167:6838/398150514 2024-07-26T13:47:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.719+0000 7f6f9e5e9740 1 --2- 172.21.15.100:0/1527396007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f58bee92f0 0x55f58c32e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.719+0000 7f6f9e5e9740 1 -- 172.21.15.100:0/1527396007 >> 172.21.15.100:0/1527396007 conn(0x55f58bf06300 msgr2=0x55f58c2c3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.719+0000 7f6f9e5e9740 1 -- 172.21.15.100:0/1527396007 shutdown_connections 2024-07-26T13:47:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.719+0000 7f6f9e5e9740 1 -- 172.21.15.100:0/1527396007 wait complete. 2024-07-26T13:47:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.719+0000 7f6f9e5e9740 20 client.17481 trim_cache size 0 max 16384 2024-07-26T13:47:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:{ 2024-07-26T13:47:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "message": "", 2024-07-26T13:47:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout: "return_code": 0 2024-07-26T13:47:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stdout:} 2024-07-26T13:47:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.720+0000 7f6f975eb640 1 -- 172.21.15.100:0/3789491418 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6f7c0b3d40 msgr2=0x7f6f7c0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.720+0000 7f6f975eb640 1 --2- 172.21.15.100:0/3789491418 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6f7c0b3d40 0x7f6f7c0b6250 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6f880ab350 tx=0x7f6f88078b10 comp rx=0 tx=0).stop 2024-07-26T13:47:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.720+0000 7f6f975eb640 1 -- 172.21.15.100:0/3789491418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f90108680 msgr2=0x7f6f9019e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.720+0000 7f6f975eb640 1 --2- 172.21.15.100:0/3789491418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f90108680 0x7f6f9019e5a0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f6f8800eb10 tx=0x7f6f8800ef10 comp rx=0 tx=0).stop 2024-07-26T13:47:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.720+0000 7f6f975eb640 1 -- 172.21.15.100:0/3789491418 shutdown_connections 2024-07-26T13:47:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.720+0000 7f6f975eb640 1 --2- 172.21.15.100:0/3789491418 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f6f7c0b3d40 0x7f6f7c0b6250 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.720+0000 7f6f975eb640 1 --2- 172.21.15.100:0/3789491418 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6f90108680 0x7f6f9019e5a0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.720+0000 7f6f975eb640 1 --2- 172.21.15.100:0/3789491418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f90107d30 0x7f6f90199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.720+0000 7f6f975eb640 1 --2- 172.21.15.100:0/3789491418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f90102400 0x7f6f90198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.720+0000 7f6f975eb640 1 -- 172.21.15.100:0/3789491418 >> 172.21.15.100:0/3789491418 conn(0x7f6f900fde40 msgr2=0x7f6f90100260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.720+0000 7f6f975eb640 1 -- 172.21.15.100:0/3789491418 shutdown_connections 2024-07-26T13:47:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:47:58.720+0000 7f6f975eb640 1 -- 172.21.15.100:0/3789491418 wait complete. 2024-07-26T13:47:58.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:58 smithi167 ceph-mon[37140]: scrub summary: active paths [/] 2024-07-26T13:47:58.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:58 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:47:59.426 INFO:tasks.workunit.client.0.smithi100.stdout: 1 343895 4.21 MB/sec execute 253 sec latency 38.907 ms 2024-07-26T13:47:59.491 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:59 smithi100 ceph-mon[32599]: scrub queued for path: / 2024-07-26T13:47:59.491 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:59 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:47:59.491 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:59 smithi100 ceph-mon[32599]: scrub summary: active paths [/] 2024-07-26T13:47:59.491 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:59 smithi100 ceph-mon[32599]: scrub summary: active paths [/] 2024-07-26T13:47:59.491 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:59 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:47:59.492 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:59 smithi100 ceph-mon[32599]: scrub summary: idle+waiting paths [/] 2024-07-26T13:47:59.492 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:47:59 smithi100 ceph-mon[32599]: pgmap v606: 97 pgs: 97 active+clean; 124 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.9 MiB/s wr, 252 op/s 2024-07-26T13:47:59.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:59 smithi138 ceph-mon[36954]: scrub queued for path: / 2024-07-26T13:47:59.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:59 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:47:59.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:59 smithi138 ceph-mon[36954]: scrub summary: active paths [/] 2024-07-26T13:47:59.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:59 smithi138 ceph-mon[36954]: scrub summary: active paths [/] 2024-07-26T13:47:59.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:59 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:47:59.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:59 smithi138 ceph-mon[36954]: scrub summary: idle+waiting paths [/] 2024-07-26T13:47:59.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:47:59 smithi138 ceph-mon[36954]: pgmap v606: 97 pgs: 97 active+clean; 124 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.9 MiB/s wr, 252 op/s 2024-07-26T13:47:59.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.716+0000 7fb1ac2c4640 1 -- 172.21.15.100:0/475136862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1a41029c0 msgr2=0x7fb1a410f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:59.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.716+0000 7fb1ac2c4640 1 --2- 172.21.15.100:0/475136862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1a41029c0 0x7fb1a410f6e0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fb194009900 tx=0x7fb19402a480 comp rx=0 tx=0).stop 2024-07-26T13:47:59.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.716+0000 7fb1ac2c4640 1 -- 172.21.15.100:0/475136862 shutdown_connections 2024-07-26T13:47:59.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.716+0000 7fb1ac2c4640 1 --2- 172.21.15.100:0/475136862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1a410fc20 0x7fb1a4112040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:59.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.716+0000 7fb1ac2c4640 1 --2- 172.21.15.100:0/475136862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1a41029c0 0x7fb1a410f6e0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:59.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.716+0000 7fb1ac2c4640 1 --2- 172.21.15.100:0/475136862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1a4102070 0x7fb1a4102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:59.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.716+0000 7fb1ac2c4640 1 -- 172.21.15.100:0/475136862 >> 172.21.15.100:0/475136862 conn(0x7fb1a4079160 msgr2=0x7fb1a4079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:59.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.716+0000 7fb1ac2c4640 1 -- 172.21.15.100:0/475136862 shutdown_connections 2024-07-26T13:47:59.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.716+0000 7fb1ac2c4640 1 -- 172.21.15.100:0/475136862 wait complete. 2024-07-26T13:47:59.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.717+0000 7fb1ac2c4640 1 Processor -- start 2024-07-26T13:47:59.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.717+0000 7fb1ac2c4640 1 -- start start 2024-07-26T13:47:59.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.717+0000 7fb1ac2c4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1a4102070 0x7fb1a419c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:59.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.717+0000 7fb1ac2c4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1a41029c0 0x7fb1a419c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:59.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.717+0000 7fb1ac2c4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1a410fc20 0x7fb1a41a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:59.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.717+0000 7fb1ac2c4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a4115690 con 0x7fb1a410fc20 2024-07-26T13:47:59.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.717+0000 7fb1ac2c4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a4115510 con 0x7fb1a41029c0 2024-07-26T13:47:59.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.717+0000 7fb1ac2c4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a4115810 con 0x7fb1a4102070 2024-07-26T13:47:59.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.717+0000 7fb1aa861640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1a410fc20 0x7fb1a41a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:59.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.717+0000 7fb1aa861640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1a410fc20 0x7fb1a41a1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36914/0 (socket says 172.21.15.100:36914) 2024-07-26T13:47:59.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.717+0000 7fb1a985f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1a41029c0 0x7fb1a419c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:59.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.717+0000 7fb1aa861640 1 -- 172.21.15.100:0/2977676135 learned_addr learned my addr 172.21.15.100:0/2977676135 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:47:59.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.717+0000 7fb1aa060640 1 --2- 172.21.15.100:0/2977676135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1a4102070 0x7fb1a419c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:59.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.718+0000 7fb1a985f640 1 -- 172.21.15.100:0/2977676135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1a4102070 msgr2=0x7fb1a419c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:59.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.718+0000 7fb1a985f640 1 --2- 172.21.15.100:0/2977676135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1a4102070 0x7fb1a419c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:59.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.718+0000 7fb1a985f640 1 -- 172.21.15.100:0/2977676135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1a410fc20 msgr2=0x7fb1a41a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:59.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.718+0000 7fb1a985f640 1 --2- 172.21.15.100:0/2977676135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1a410fc20 0x7fb1a41a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:59.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.718+0000 7fb1a985f640 1 -- 172.21.15.100:0/2977676135 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a41a2070 con 0x7fb1a41029c0 2024-07-26T13:47:59.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.718+0000 7fb1aa861640 1 --2- 172.21.15.100:0/2977676135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1a410fc20 0x7fb1a41a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:47:59.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.718+0000 7fb1a985f640 1 --2- 172.21.15.100:0/2977676135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1a41029c0 0x7fb1a419c6a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb194006010 tx=0x7fb194004060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:59.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.718+0000 7fb19b7fe640 1 -- 172.21.15.100:0/2977676135 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19403f070 con 0x7fb1a41029c0 2024-07-26T13:47:59.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.718+0000 7fb19b7fe640 1 -- 172.21.15.100:0/2977676135 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb194003ca0 con 0x7fb1a41029c0 2024-07-26T13:47:59.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.718+0000 7fb19b7fe640 1 -- 172.21.15.100:0/2977676135 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb194035070 con 0x7fb1a41029c0 2024-07-26T13:47:59.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.718+0000 7fb1ac2c4640 1 -- 172.21.15.100:0/2977676135 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1a41a2300 con 0x7fb1a41029c0 2024-07-26T13:47:59.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.718+0000 7fb1ac2c4640 1 -- 172.21.15.100:0/2977676135 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1a41a8ac0 con 0x7fb1a41029c0 2024-07-26T13:47:59.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.720+0000 7fb1ac2c4640 1 -- 172.21.15.100:0/2977676135 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1a406bc10 con 0x7fb1a41029c0 2024-07-26T13:47:59.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.721+0000 7fb19b7fe640 1 -- 172.21.15.100:0/2977676135 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fb19403b460 con 0x7fb1a41029c0 2024-07-26T13:47:59.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.722+0000 7fb19b7fe640 1 --2- 172.21.15.100:0/2977676135 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb1900b3dc0 0x7fb1900b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:47:59.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.722+0000 7fb1aa060640 1 --2- 172.21.15.100:0/2977676135 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb1900b3dc0 0x7fb1900b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:47:59.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.722+0000 7fb19b7fe640 1 -- 172.21.15.100:0/2977676135 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8828+0+0 (secure 0 0 0) 0x7fb1941047d0 con 0x7fb1a41029c0 2024-07-26T13:47:59.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.722+0000 7fb1aa060640 1 --2- 172.21.15.100:0/2977676135 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb1900b3dc0 0x7fb1900b62d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0005fd0 tx=0x7fb1a0009500 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:47:59.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.727+0000 7fb19b7fe640 1 -- 172.21.15.100:0/2977676135 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fb1940cb300 con 0x7fb1a41029c0 2024-07-26T13:47:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:59 smithi167 ceph-mon[37140]: scrub queued for path: / 2024-07-26T13:47:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:59 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:47:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:59 smithi167 ceph-mon[37140]: scrub summary: active paths [/] 2024-07-26T13:47:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:59 smithi167 ceph-mon[37140]: scrub summary: active paths [/] 2024-07-26T13:47:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:59 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:47:59.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:59 smithi167 ceph-mon[37140]: scrub summary: idle+waiting paths [/] 2024-07-26T13:47:59.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:47:59 smithi167 ceph-mon[37140]: pgmap v606: 97 pgs: 97 active+clean; 124 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.9 MiB/s wr, 252 op/s 2024-07-26T13:47:59.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.961+0000 7fb1ac2c4640 1 -- 172.21.15.100:0/2977676135 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1a419d230 con 0x7fb1a41029c0 2024-07-26T13:47:59.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.962+0000 7fb19b7fe640 1 -- 172.21.15.100:0/2977676135 <== mon.2 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+11928 (secure 0 0 0) 0x7fb1940cf850 con 0x7fb1a41029c0 2024-07-26T13:47:59.963 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:47:59.964 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":61,"btime":"2024-07-26T13:47:33:271455+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14790,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/313501409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":313501409},{"type":"v1","addr":"172.21.15.100:6839","nonce":313501409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/1402975506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":1402975506},{"type":"v1","addr":"172.21.15.100:6841","nonce":1402975506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":61,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-26T13:37:01.281843+0000","modified":"2024-07-26T13:47:33.271423+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":87,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24607,"mds_1":24605,"mds_2":26446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24575":{"gid":24575,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6837/3393225483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3393225483},{"type":"v1","addr":"172.21.15.138:6837","nonce":3393225483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24605":{"gid":24605,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.138:6839/1580925471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1580925471},{"type":"v1","addr":"172.21.15.138:6839","nonce":1580925471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24607":{"gid":24607,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.167:6839/398150514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":398150514},{"type":"v1","addr":"172.21.15.167:6839","nonce":398150514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26446":{"gid":26446,"name":"c","rank":2,"incarnation":56,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/3519222898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3519222898},{"type":"v1","addr":"172.21.15.167:6837","nonce":3519222898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26558":{"gid":26558,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/1781111462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1781111462},{"type":"v1","addr":"172.21.15.138:6835","nonce":1781111462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24607,"qdb_cluster":[26446,24607,24605]},"id":1}]} 2024-07-26T13:47:59.964 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 61 2024-07-26T13:47:59.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.964+0000 7fb1ac2c4640 1 -- 172.21.15.100:0/2977676135 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb1900b3dc0 msgr2=0x7fb1900b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:59.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.964+0000 7fb1ac2c4640 1 --2- 172.21.15.100:0/2977676135 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb1900b3dc0 0x7fb1900b62d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0005fd0 tx=0x7fb1a0009500 comp rx=0 tx=0).stop 2024-07-26T13:47:59.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.964+0000 7fb1ac2c4640 1 -- 172.21.15.100:0/2977676135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1a41029c0 msgr2=0x7fb1a419c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:47:59.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.964+0000 7fb1ac2c4640 1 --2- 172.21.15.100:0/2977676135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1a41029c0 0x7fb1a419c6a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb194006010 tx=0x7fb194004060 comp rx=0 tx=0).stop 2024-07-26T13:47:59.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.964+0000 7fb1ac2c4640 1 -- 172.21.15.100:0/2977676135 shutdown_connections 2024-07-26T13:47:59.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.964+0000 7fb1ac2c4640 1 --2- 172.21.15.100:0/2977676135 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fb1900b3dc0 0x7fb1900b62d0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:59.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.964+0000 7fb1ac2c4640 1 --2- 172.21.15.100:0/2977676135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1a410fc20 0x7fb1a41a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:59.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.964+0000 7fb1ac2c4640 1 --2- 172.21.15.100:0/2977676135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1a41029c0 0x7fb1a419c6a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:59.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.964+0000 7fb1ac2c4640 1 --2- 172.21.15.100:0/2977676135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1a4102070 0x7fb1a419c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:47:59.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.964+0000 7fb1ac2c4640 1 -- 172.21.15.100:0/2977676135 >> 172.21.15.100:0/2977676135 conn(0x7fb1a4079160 msgr2=0x7fb1a41097c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:47:59.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.965+0000 7fb1ac2c4640 1 -- 172.21.15.100:0/2977676135 shutdown_connections 2024-07-26T13:47:59.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:47:59.965+0000 7fb1ac2c4640 1 -- 172.21.15.100:0/2977676135 wait complete. 2024-07-26T13:48:00.382 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-26T13:48:00.426 INFO:tasks.workunit.client.0.smithi100.stdout: 1 345274 4.22 MB/sec execute 254 sec latency 29.917 ms 2024-07-26T13:48:00.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:00 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/2977676135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:48:00.719 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:00 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/2977676135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:48:00.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/2977676135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:48:00.773 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: ceph-mds: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-5320-ge8f88bb1/rpm/el9/BUILD/ceph-19.0.0-5320-ge8f88bb1/redhat-linux-build/boost/include/boost/smart_ptr/intrusive_ptr.hpp:201: T* boost::intrusive_ptr::operator->() const [with T = const Message]: Assertion `px != 0' failed. 2024-07-26T13:48:00.773 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: *** Caught signal (Aborted) ** 2024-07-26T13:48:00.773 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: in thread 7f740d4e6640 thread_name:ms_dispatch 2024-07-26T13:48:00.773 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: ceph version 19.0.0-5320-ge8f88bb1 (e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528) squid (dev) 2024-07-26T13:48:00.773 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f74128bd6f0] 2024-07-26T13:48:00.773 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 2: /lib64/libc.so.6(+0x8b94c) [0x7f741290a94c] 2024-07-26T13:48:00.773 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 3: raise() 2024-07-26T13:48:00.773 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 4: abort() 2024-07-26T13:48:00.773 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 5: /lib64/libc.so.6(+0x2871b) [0x7f74128a771b] 2024-07-26T13:48:00.774 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 6: /lib64/libc.so.6(+0x37386) [0x7f74128b6386] 2024-07-26T13:48:00.774 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 7: (MDSRank::get_session(boost::intrusive_ptr const&)+0x89) [0x558925c6d6b9] 2024-07-26T13:48:00.774 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 8: (MDCache::path_traverse(boost::intrusive_ptr const&, ContextFactory&, filepath const&, int, std::vector >*, CInode**, CDir**)+0x58e) [0x558925e2b80c] 2024-07-26T13:48:00.774 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 9: (Server::handle_peer_rename_prep(boost::intrusive_ptr const&)+0x4fe) [0x558925d79cde] 2024-07-26T13:48:00.774 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 10: (Server::dispatch_peer_request(boost::intrusive_ptr const&)+0x933) [0x558925d3b763] 2024-07-26T13:48:00.774 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 11: (Server::handle_peer_request(boost::intrusive_ptr const&)+0x10c3) [0x558925d3ca1b] 2024-07-26T13:48:00.774 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x9ba) [0x558925d20752] 2024-07-26T13:48:00.774 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 13: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x81a) [0x558925c6ea56] 2024-07-26T13:48:00.774 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 14: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x558925c72637] 2024-07-26T13:48:00.774 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 15: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x558925c72f5e] 2024-07-26T13:48:00.774 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 16: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x558925c5603a] 2024-07-26T13:48:00.774 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 17: /usr/lib64/ceph/libceph-common.so.2(+0x39b407) [0x7f741318a407] 2024-07-26T13:48:00.775 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 18: (DispatchQueue::entry()+0x60f) [0x7f741318acf9] 2024-07-26T13:48:00.775 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 19: /usr/lib64/ceph/libceph-common.so.2(+0x4785a1) [0x7f74132675a1] 2024-07-26T13:48:00.775 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 20: (Thread::entry_wrapper()+0x43) [0x7f7413014cdd] 2024-07-26T13:48:00.776 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 21: (Thread::_entry_func(void*)+0xd) [0x7f7413014cf9] 2024-07-26T13:48:00.776 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 22: /lib64/libc.so.6(+0x89c02) [0x7f7412908c02] 2024-07-26T13:48:00.776 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 23: /lib64/libc.so.6(+0x10ec40) [0x7f741298dc40] 2024-07-26T13:48:00.776 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 2024-07-26T13:48:00.426+0000 7f740d4e6640 -1 *** Caught signal (Aborted) ** 2024-07-26T13:48:00.776 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: in thread 7f740d4e6640 thread_name:ms_dispatch 2024-07-26T13:48:00.776 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 2024-07-26T13:48:00.776 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: ceph version 19.0.0-5320-ge8f88bb1 (e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528) squid (dev) 2024-07-26T13:48:00.776 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f74128bd6f0] 2024-07-26T13:48:00.776 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 2: /lib64/libc.so.6(+0x8b94c) [0x7f741290a94c] 2024-07-26T13:48:00.776 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 3: raise() 2024-07-26T13:48:00.776 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 4: abort() 2024-07-26T13:48:00.776 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 5: /lib64/libc.so.6(+0x2871b) [0x7f74128a771b] 2024-07-26T13:48:00.776 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 6: /lib64/libc.so.6(+0x37386) [0x7f74128b6386] 2024-07-26T13:48:00.777 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 7: (MDSRank::get_session(boost::intrusive_ptr const&)+0x89) [0x558925c6d6b9] 2024-07-26T13:48:00.777 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 8: (MDCache::path_traverse(boost::intrusive_ptr const&, ContextFactory&, filepath const&, int, std::vector >*, CInode**, CDir**)+0x58e) [0x558925e2b80c] 2024-07-26T13:48:00.777 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 9: (Server::handle_peer_rename_prep(boost::intrusive_ptr const&)+0x4fe) [0x558925d79cde] 2024-07-26T13:48:00.777 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 10: (Server::dispatch_peer_request(boost::intrusive_ptr const&)+0x933) [0x558925d3b763] 2024-07-26T13:48:00.777 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 11: (Server::handle_peer_request(boost::intrusive_ptr const&)+0x10c3) [0x558925d3ca1b] 2024-07-26T13:48:00.777 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x9ba) [0x558925d20752] 2024-07-26T13:48:00.777 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 13: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x81a) [0x558925c6ea56] 2024-07-26T13:48:00.777 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 14: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x558925c72637] 2024-07-26T13:48:00.777 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 15: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x558925c72f5e] 2024-07-26T13:48:00.777 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 16: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x558925c5603a] 2024-07-26T13:48:00.777 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 17: /usr/lib64/ceph/libceph-common.so.2(+0x39b407) [0x7f741318a407] 2024-07-26T13:48:00.777 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 18: (DispatchQueue::entry()+0x60f) [0x7f741318acf9] 2024-07-26T13:48:00.777 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 19: /usr/lib64/ceph/libceph-common.so.2(+0x4785a1) [0x7f74132675a1] 2024-07-26T13:48:00.777 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 20: (Thread::entry_wrapper()+0x43) [0x7f7413014cdd] 2024-07-26T13:48:00.778 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 21: (Thread::_entry_func(void*)+0xd) [0x7f7413014cf9] 2024-07-26T13:48:00.778 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 22: /lib64/libc.so.6(+0x89c02) [0x7f7412908c02] 2024-07-26T13:48:00.778 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 23: /lib64/libc.so.6(+0x10ec40) [0x7f741298dc40] 2024-07-26T13:48:00.778 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-07-26T13:48:00.778 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 2024-07-26T13:48:00.778 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 0> 2024-07-26T13:48:00.426+0000 7f740d4e6640 -1 *** Caught signal (Aborted) ** 2024-07-26T13:48:00.778 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: in thread 7f740d4e6640 thread_name:ms_dispatch 2024-07-26T13:48:00.778 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 2024-07-26T13:48:00.778 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: ceph version 19.0.0-5320-ge8f88bb1 (e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528) squid (dev) 2024-07-26T13:48:00.778 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f74128bd6f0] 2024-07-26T13:48:00.778 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 2: /lib64/libc.so.6(+0x8b94c) [0x7f741290a94c] 2024-07-26T13:48:00.778 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 3: raise() 2024-07-26T13:48:00.778 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 4: abort() 2024-07-26T13:48:00.778 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 5: /lib64/libc.so.6(+0x2871b) [0x7f74128a771b] 2024-07-26T13:48:00.779 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 6: /lib64/libc.so.6(+0x37386) [0x7f74128b6386] 2024-07-26T13:48:00.779 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 7: (MDSRank::get_session(boost::intrusive_ptr const&)+0x89) [0x558925c6d6b9] 2024-07-26T13:48:00.779 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 8: (MDCache::path_traverse(boost::intrusive_ptr const&, ContextFactory&, filepath const&, int, std::vector >*, CInode**, CDir**)+0x58e) [0x558925e2b80c] 2024-07-26T13:48:00.779 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 9: (Server::handle_peer_rename_prep(boost::intrusive_ptr const&)+0x4fe) [0x558925d79cde] 2024-07-26T13:48:00.779 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 10: (Server::dispatch_peer_request(boost::intrusive_ptr const&)+0x933) [0x558925d3b763] 2024-07-26T13:48:00.779 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 11: (Server::handle_peer_request(boost::intrusive_ptr const&)+0x10c3) [0x558925d3ca1b] 2024-07-26T13:48:00.779 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x9ba) [0x558925d20752] 2024-07-26T13:48:00.779 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 13: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x81a) [0x558925c6ea56] 2024-07-26T13:48:00.779 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 14: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x558925c72637] 2024-07-26T13:48:00.779 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 15: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x558925c72f5e] 2024-07-26T13:48:00.779 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 16: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x558925c5603a] 2024-07-26T13:48:00.779 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 17: /usr/lib64/ceph/libceph-common.so.2(+0x39b407) [0x7f741318a407] 2024-07-26T13:48:00.779 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 18: (DispatchQueue::entry()+0x60f) [0x7f741318acf9] 2024-07-26T13:48:00.779 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 19: /usr/lib64/ceph/libceph-common.so.2(+0x4785a1) [0x7f74132675a1] 2024-07-26T13:48:00.780 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 20: (Thread::entry_wrapper()+0x43) [0x7f7413014cdd] 2024-07-26T13:48:00.780 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 21: (Thread::_entry_func(void*)+0xd) [0x7f7413014cf9] 2024-07-26T13:48:00.780 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 22: /lib64/libc.so.6(+0x89c02) [0x7f7412908c02] 2024-07-26T13:48:00.780 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 23: /lib64/libc.so.6(+0x10ec40) [0x7f741298dc40] 2024-07-26T13:48:00.780 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-07-26T13:48:00.780 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 2024-07-26T13:48:00.780 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 0> 2024-07-26T13:48:00.426+0000 7f740d4e6640 -1 *** Caught signal (Aborted) ** 2024-07-26T13:48:00.780 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: in thread 7f740d4e6640 thread_name:ms_dispatch 2024-07-26T13:48:00.780 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 2024-07-26T13:48:00.780 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: ceph version 19.0.0-5320-ge8f88bb1 (e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528) squid (dev) 2024-07-26T13:48:00.780 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f74128bd6f0] 2024-07-26T13:48:00.780 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 2: /lib64/libc.so.6(+0x8b94c) [0x7f741290a94c] 2024-07-26T13:48:00.780 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 3: raise() 2024-07-26T13:48:00.780 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 4: abort() 2024-07-26T13:48:00.781 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 5: /lib64/libc.so.6(+0x2871b) [0x7f74128a771b] 2024-07-26T13:48:00.781 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 6: /lib64/libc.so.6(+0x37386) [0x7f74128b6386] 2024-07-26T13:48:00.781 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 7: (MDSRank::get_session(boost::intrusive_ptr const&)+0x89) [0x558925c6d6b9] 2024-07-26T13:48:00.781 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 8: (MDCache::path_traverse(boost::intrusive_ptr const&, ContextFactory&, filepath const&, int, std::vector >*, CInode**, CDir**)+0x58e) [0x558925e2b80c] 2024-07-26T13:48:00.781 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 9: (Server::handle_peer_rename_prep(boost::intrusive_ptr const&)+0x4fe) [0x558925d79cde] 2024-07-26T13:48:00.781 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 10: (Server::dispatch_peer_request(boost::intrusive_ptr const&)+0x933) [0x558925d3b763] 2024-07-26T13:48:00.781 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 11: (Server::handle_peer_request(boost::intrusive_ptr const&)+0x10c3) [0x558925d3ca1b] 2024-07-26T13:48:00.781 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x9ba) [0x558925d20752] 2024-07-26T13:48:00.781 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 13: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x81a) [0x558925c6ea56] 2024-07-26T13:48:00.781 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 14: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x558925c72637] 2024-07-26T13:48:00.781 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 15: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x558925c72f5e] 2024-07-26T13:48:00.781 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 16: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x558925c5603a] 2024-07-26T13:48:00.781 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 17: /usr/lib64/ceph/libceph-common.so.2(+0x39b407) [0x7f741318a407] 2024-07-26T13:48:00.782 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 18: (DispatchQueue::entry()+0x60f) [0x7f741318acf9] 2024-07-26T13:48:00.782 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 19: /usr/lib64/ceph/libceph-common.so.2(+0x4785a1) [0x7f74132675a1] 2024-07-26T13:48:00.782 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 20: (Thread::entry_wrapper()+0x43) [0x7f7413014cdd] 2024-07-26T13:48:00.782 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 21: (Thread::_entry_func(void*)+0xd) [0x7f7413014cf9] 2024-07-26T13:48:00.782 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 22: /lib64/libc.so.6(+0x89c02) [0x7f7412908c02] 2024-07-26T13:48:00.782 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 23: /lib64/libc.so.6(+0x10ec40) [0x7f741298dc40] 2024-07-26T13:48:00.782 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-07-26T13:48:00.782 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[69332]: 2024-07-26T13:48:00.782 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:00 smithi167 podman[72923]: 2024-07-26 13:48:00.485323954 +0000 UTC m=+0.019182537 container died dd8ee84abc2d1afc61ce4713e9267a89e11913afbf6df3bcd49ffb48163c4f1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240724, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-07-26T13:48:01.237 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24607 quiesce db / --timeout 90 --expiration 142.2 --await-for 45.0' 2024-07-26T13:48:01.237 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- timeout 105 ceph tell mds.24607 quiesce db / --timeout 90 --expiration 142.2 --await-for 45.0 2024-07-26T13:48:01.323 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:48:01.427 INFO:tasks.workunit.client.0.smithi100.stdout: 1 345274 4.21 MB/sec execute 255 sec latency 1003.284 ms 2024-07-26T13:48:01.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:01 smithi167 ceph-mon[37140]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:48:01.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:01 smithi167 ceph-mon[37140]: pgmap v607: 97 pgs: 97 active+clean; 129 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 274 op/s 2024-07-26T13:48:01.523 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:01 smithi167 podman[72923]: 2024-07-26 13:48:01.197846732 +0000 UTC m=+0.731705316 container remove dd8ee84abc2d1afc61ce4713e9267a89e11913afbf6df3bcd49ffb48163c4f1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, ceph=True, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-26T13:48:01.523 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:01 smithi167 systemd[1]: ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.i.service: Main process exited, code=exited, status=134/n/a 2024-07-26T13:48:01.523 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:01 smithi167 systemd[1]: ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.i.service: Failed with result 'exit-code'. 2024-07-26T13:48:01.523 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:01 smithi167 systemd[1]: ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.i.service: Consumed 1min 49.357s CPU time. 2024-07-26T13:48:01.588 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:01 smithi100 ceph-mon[32599]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:48:01.588 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:01 smithi100 ceph-mon[32599]: pgmap v607: 97 pgs: 97 active+clean; 129 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 274 op/s 2024-07-26T13:48:01.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:01 smithi138 ceph-mon[36954]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:48:01.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:01 smithi138 ceph-mon[36954]: pgmap v607: 97 pgs: 97 active+clean; 129 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 274 op/s 2024-07-26T13:48:02.427 INFO:tasks.workunit.client.0.smithi100.stdout: 1 345274 4.19 MB/sec execute 256 sec latency 2003.360 ms 2024-07-26T13:48:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:02 smithi100 ceph-mon[32599]: pgmap v608: 97 pgs: 97 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.9 MiB/s wr, 239 op/s 2024-07-26T13:48:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:02 smithi138 ceph-mon[36954]: pgmap v608: 97 pgs: 97 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.9 MiB/s wr, 239 op/s 2024-07-26T13:48:03.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.019+0000 7fe5ac807640 1 -- 172.21.15.100:0/1249718640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5a8102510 msgr2=0x7fe5a8102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:48:03.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.019+0000 7fe5ac807640 1 --2- 172.21.15.100:0/1249718640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5a8102510 0x7fe5a8102920 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fe598009900 tx=0x7fe59802a410 comp rx=0 tx=0).stop 2024-07-26T13:48:03.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.019+0000 7fe5ac807640 1 -- 172.21.15.100:0/1249718640 shutdown_connections 2024-07-26T13:48:03.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.019+0000 7fe5ac807640 1 --2- 172.21.15.100:0/1249718640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5a81087a0 0x7fe5a810efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:48:03.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.019+0000 7fe5ac807640 1 --2- 172.21.15.100:0/1249718640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5a8107e50 0x7fe5a8108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:48:03.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.019+0000 7fe5ac807640 1 --2- 172.21.15.100:0/1249718640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5a8102510 0x7fe5a8102920 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:48:03.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.019+0000 7fe5ac807640 1 -- 172.21.15.100:0/1249718640 >> 172.21.15.100:0/1249718640 conn(0x7fe5a80791a0 msgr2=0x7fe5a80795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:48:03.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.019+0000 7fe5ac807640 1 -- 172.21.15.100:0/1249718640 shutdown_connections 2024-07-26T13:48:03.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.019+0000 7fe5ac807640 1 -- 172.21.15.100:0/1249718640 wait complete. 2024-07-26T13:48:03.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.020+0000 7fe5ac807640 1 Processor -- start 2024-07-26T13:48:03.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.020+0000 7fe5ac807640 1 -- start start 2024-07-26T13:48:03.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.020+0000 7fe5ac807640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5a8102510 0x7fe5a8199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:48:03.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.020+0000 7fe5ac807640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5a8107e50 0x7fe5a819a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:48:03.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.020+0000 7fe5ac807640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5a81087a0 0x7fe5a819f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:48:03.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.020+0000 7fe5ac807640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a8113580 con 0x7fe5a81087a0 2024-07-26T13:48:03.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.020+0000 7fe5ac807640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a8113400 con 0x7fe5a8107e50 2024-07-26T13:48:03.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.020+0000 7fe5ac807640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a8113700 con 0x7fe5a8102510 2024-07-26T13:48:03.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.020+0000 7fe5a6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5a81087a0 0x7fe5a819f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:03.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.020+0000 7fe5a6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5a81087a0 0x7fe5a819f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36938/0 (socket says 172.21.15.100:36938) 2024-07-26T13:48:03.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.020+0000 7fe5a6d9d640 1 -- 172.21.15.100:0/2403309286 learned_addr learned my addr 172.21.15.100:0/2403309286 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:48:03.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.020+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5a8107e50 0x7fe5a819a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:03.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.020+0000 7fe5a659c640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5a8102510 0x7fe5a8199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:03.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.021+0000 7fe5a6d9d640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5a8102510 msgr2=0x7fe5a8199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:48:03.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.021+0000 7fe5a6d9d640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5a8102510 0x7fe5a8199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:48:03.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.021+0000 7fe5a6d9d640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5a8107e50 msgr2=0x7fe5a819a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:48:03.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.021+0000 7fe5a6d9d640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5a8107e50 0x7fe5a819a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:48:03.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.021+0000 7fe5a6d9d640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a819fee0 con 0x7fe5a81087a0 2024-07-26T13:48:03.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.021+0000 7fe5a659c640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5a8102510 0x7fe5a8199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:48:03.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.021+0000 7fe5a6d9d640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe5a81087a0 0x7fe5a819f7b0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fe5a000eaa0 tx=0x7fe5a000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:48:03.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.021+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a000cdb0 con 0x7fe5a81087a0 2024-07-26T13:48:03.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.021+0000 7fe5ac807640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5a81a01d0 con 0x7fe5a81087a0 2024-07-26T13:48:03.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.021+0000 7fe5ac807640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5a810f990 con 0x7fe5a81087a0 2024-07-26T13:48:03.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.021+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5a0011470 con 0x7fe5a81087a0 2024-07-26T13:48:03.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.021+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a001f060 con 0x7fe5a81087a0 2024-07-26T13:48:03.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:02 smithi167 ceph-mon[37140]: pgmap v608: 97 pgs: 97 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.9 MiB/s wr, 239 op/s 2024-07-26T13:48:03.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.024+0000 7fe5b2d8b740 1 Processor -- start 2024-07-26T13:48:03.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.024+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fe5a0005280 con 0x7fe5a81087a0 2024-07-26T13:48:03.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.025+0000 7fe5b2d8b740 1 -- start start 2024-07-26T13:48:03.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.025+0000 7fe5b2d8b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0ec8a8070 0x55c0ecc4d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:48:03.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.025+0000 7fe5b2d8b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0ec87f1b0 0x55c0ecc4da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:48:03.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.025+0000 7fe5b2d8b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0ec887390 0x55c0ecc4dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:48:03.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.025+0000 7fe5b2d8b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c0ecbc3b90 con 0x55c0ec887390 2024-07-26T13:48:03.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.025+0000 7fe5b2d8b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c0ecbb3090 con 0x55c0ec87f1b0 2024-07-26T13:48:03.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.025+0000 7fe5b2d8b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c0ecbb70d0 con 0x55c0ec8a8070 2024-07-26T13:48:03.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.025+0000 7fe59ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0ec887390 0x55c0ecc4dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:03.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.025+0000 7fe59ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0ec887390 0x55c0ecc4dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36946/0 (socket says 172.21.15.100:36946) 2024-07-26T13:48:03.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.025+0000 7fe59ffff640 1 -- 172.21.15.100:0/3962012168 learned_addr learned my addr 172.21.15.100:0/3962012168 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:48:03.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.025+0000 7fe5a659c640 1 --2- 172.21.15.100:0/3962012168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0ec87f1b0 0x55c0ecc4da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:03.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.025+0000 7fe59ffff640 1 --2- 172.21.15.100:0/3962012168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0ec8a8070 0x55c0ecc4d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:03.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.026+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b3d40 0x7fe5940b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:48:03.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.026+0000 7fe59ffff640 1 -- 172.21.15.100:0/3962012168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0ec8a8070 msgr2=0x55c0ecc4d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:48:03.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.026+0000 7fe59ffff640 1 --2- 172.21.15.100:0/3962012168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0ec8a8070 0x55c0ecc4d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:48:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.026+0000 7fe59ffff640 1 -- 172.21.15.100:0/3962012168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0ec87f1b0 msgr2=0x55c0ecc4da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:48:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.026+0000 7fe59ffff640 1 --2- 172.21.15.100:0/3962012168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0ec87f1b0 0x55c0ecc4da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:48:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.026+0000 7fe59ffff640 1 -- 172.21.15.100:0/3962012168 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0ecb59b70 con 0x55c0ec887390 2024-07-26T13:48:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.026+0000 7fe5a6d9d640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b3d40 0x7fe5940b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.026+0000 7fe59ffff640 1 --2- 172.21.15.100:0/3962012168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0ec887390 0x55c0ecc4dfd0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fe5900124c0 tx=0x7fe590005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=694092aa77b958fe server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:48:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.026+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8828+0+0 (secure 0 0 0) 0x7fe5a00e4230 con 0x7fe5a81087a0 2024-07-26T13:48:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.026+0000 7fe59cff9640 1 -- 172.21.15.100:0/3962012168 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe590004230 con 0x55c0ec887390 2024-07-26T13:48:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.026+0000 7fe59cff9640 1 -- 172.21.15.100:0/3962012168 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5900043d0 con 0x55c0ec887390 2024-07-26T13:48:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.026+0000 7fe59cff9640 1 -- 172.21.15.100:0/3962012168 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5900035f0 con 0x55c0ec887390 2024-07-26T13:48:03.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.027+0000 7fe5b2d8b740 1 -- 172.21.15.100:0/3962012168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0ec887390 msgr2=0x55c0ecc4dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:48:03.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.027+0000 7fe5b2d8b740 1 --2- 172.21.15.100:0/3962012168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0ec887390 0x55c0ecc4dfd0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fe5900124c0 tx=0x7fe590005c90 comp rx=0 tx=0).stop 2024-07-26T13:48:03.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.027+0000 7fe5a6d9d640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b3d40 0x7fe5940b6250 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe5a00aaca0 tx=0x7fe5a0078460 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:48:03.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.027+0000 7fe5b2d8b740 1 -- 172.21.15.100:0/3962012168 shutdown_connections 2024-07-26T13:48:03.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.027+0000 7fe5b2d8b740 1 --2- 172.21.15.100:0/3962012168 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0ec887390 0x55c0ecc4dfd0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:48:03.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.027+0000 7fe5b2d8b740 1 --2- 172.21.15.100:0/3962012168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0ec87f1b0 0x55c0ecc4da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:48:03.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.027+0000 7fe5b2d8b740 1 --2- 172.21.15.100:0/3962012168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0ec8a8070 0x55c0ecc4d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:48:03.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.027+0000 7fe5b2d8b740 1 -- 172.21.15.100:0/3962012168 >> 172.21.15.100:0/3962012168 conn(0x55c0ec89c460 msgr2=0x55c0ecc4b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:48:03.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.027+0000 7fe5b2d8b740 1 -- 172.21.15.100:0/3962012168 shutdown_connections 2024-07-26T13:48:03.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.027+0000 7fe5b2d8b740 1 -- 172.21.15.100:0/3962012168 wait complete. 2024-07-26T13:48:03.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.028+0000 7fe5b2d8b740 1 -- start start 2024-07-26T13:48:03.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.028+0000 7fe5b2d8b740 1 Processor -- start 2024-07-26T13:48:03.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.028+0000 7fe5b2d8b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0ec87f1b0 0x55c0ecc59160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:48:03.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.028+0000 7fe5b2d8b740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0ec887390 0x55c0ecc596a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:48:03.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.028+0000 7fe5b2d8b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0ec8a8070 0x55c0ecc65080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:48:03.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.028+0000 7fe5b2d8b740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c0eca16700 con 0x55c0ec887390 2024-07-26T13:48:03.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.028+0000 7fe5b2d8b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c0ecb559c0 con 0x55c0ec8a8070 2024-07-26T13:48:03.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.028+0000 7fe5b2d8b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c0ecc5bd30 con 0x55c0ec87f1b0 2024-07-26T13:48:03.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.028+0000 7fe5b2d8b740 10 client.0 _ll_register_callbacks cb 0x55c0ecb704d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:48:03.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.029+0000 7fe5a659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0ec887390 0x55c0ecc596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:03.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.029+0000 7fe59ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0ec87f1b0 0x55c0ecc59160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:03.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.029+0000 7fe5a659c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0ec887390 0x55c0ecc596a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36950/0 (socket says 172.21.15.100:36950) 2024-07-26T13:48:03.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.029+0000 7fe5a659c640 1 -- 172.21.15.100:0/972906438 learned_addr learned my addr 172.21.15.100:0/972906438 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:48:03.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.029+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0ec8a8070 0x55c0ecc65080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:03.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.029+0000 7fe5a659c640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0ec87f1b0 msgr2=0x55c0ecc59160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:48:03.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.029+0000 7fe5a659c640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c0ec87f1b0 0x55c0ecc59160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:48:03.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.029+0000 7fe5a659c640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0ec8a8070 msgr2=0x55c0ecc65080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:48:03.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.029+0000 7fe5a659c640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0ec8a8070 0x55c0ecc65080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:48:03.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.029+0000 7fe5a659c640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0ec913650 con 0x55c0ec887390 2024-07-26T13:48:03.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.029+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0ec8a8070 0x55c0ecc65080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:48:03.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.029+0000 7fe5a659c640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c0ec887390 0x55c0ecc596a0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fe598033000 tx=0x7fe5980316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8627f7163b000383 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:48:03.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.029+0000 7fe58dffb640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:48:03.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.029+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe598035040 con 0x55c0ec887390 2024-07-26T13:48:03.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.030+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8828+0+0 (secure 0 0 0) 0x7fe598004460 con 0x55c0ec887390 2024-07-26T13:48:03.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.030+0000 7fe5b2d8b740 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5900043d0 con 0x55c0ec887390 2024-07-26T13:48:03.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.030+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fe598050050 con 0x55c0ec887390 2024-07-26T13:48:03.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.030+0000 7fe5b2d8b740 10 client.17499 fetch_fsmap learned FSMap version 62 2024-07-26T13:48:03.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.030+0000 7fe5b2d8b740 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55c0ecc67dd0 con 0x55c0ec887390 2024-07-26T13:48:03.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.030+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5900043d0 con 0x55c0ec887390 2024-07-26T13:48:03.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.030+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 62) ==== 8350+0+0 (secure 0 0 0) 0x7fe59804b030 con 0x55c0ec887390 2024-07-26T13:48:03.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.030+0000 7fe5b2d8b740 10 client.17499 fetch_fsmap finished waiting for FSMap version 62 2024-07-26T13:48:03.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.030+0000 7fe5b2d8b740 10 client.17499 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-07-26T13:48:03.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.030+0000 7fe5b2d8b740 20 client.17499 populate_metadata read hostname 'smithi100' 2024-07-26T13:48:03.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.030+0000 7fe5b2d8b740 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:48:03.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.030+0000 7fe5b2d8b740 4 client.17499 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:48:03.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.030+0000 7fe5b2d8b740 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0ecbc5ae0 con 0x55c0ec8669f0 2024-07-26T13:48:03.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.031+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.167:6838/398150514 2024-07-26T13:48:03.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.031+0000 7fe58dffb640 1 client.17499 ms_handle_refused on v2:172.21.15.167:6838/398150514 2024-07-26T13:48:03.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.031+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-26T13:48:03.059 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:48:03.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.232+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.167:6838/398150514 2024-07-26T13:48:03.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.232+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-26T13:48:03.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.232+0000 7fe58dffb640 1 client.17499 ms_handle_refused on v2:172.21.15.167:6838/398150514 2024-07-26T13:48:03.427 INFO:tasks.workunit.client.0.smithi100.stdout: 1 345274 4.17 MB/sec execute 257 sec latency 3003.493 ms 2024-07-26T13:48:03.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.633+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.167:6838/398150514 2024-07-26T13:48:03.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.633+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-07-26T13:48:03.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:03.633+0000 7fe58dffb640 1 client.17499 ms_handle_refused on v2:172.21.15.167:6838/398150514 2024-07-26T13:48:03.937 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:48:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:48:04.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:48:04.427 INFO:tasks.workunit.client.0.smithi100.stdout: 1 345274 4.16 MB/sec execute 258 sec latency 4003.535 ms 2024-07-26T13:48:04.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.434+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.167:6838/398150514 2024-07-26T13:48:04.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.434+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-07-26T13:48:04.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.434+0000 7fe58dffb640 1 client.17499 ms_handle_refused on v2:172.21.15.167:6838/398150514 2024-07-26T13:48:04.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.775+0000 7f64116d0640 1 -- 172.21.15.100:0/1365792222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f640c1035b0 msgr2=0x7f640c1039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:48:04.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.775+0000 7f64116d0640 1 --2- 172.21.15.100:0/1365792222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f640c1035b0 0x7f640c1039c0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f63fc009900 tx=0x7f63fc02a410 comp rx=0 tx=0).stop 2024-07-26T13:48:04.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.776+0000 7f64116d0640 1 -- 172.21.15.100:0/1365792222 shutdown_connections 2024-07-26T13:48:04.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.776+0000 7f64116d0640 1 --2- 172.21.15.100:0/1365792222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f640c109840 0x7f640c110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:48:04.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.776+0000 7f64116d0640 1 --2- 172.21.15.100:0/1365792222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f640c108ef0 0x7f640c109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:48:04.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.776+0000 7f64116d0640 1 --2- 172.21.15.100:0/1365792222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f640c1035b0 0x7f640c1039c0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:48:04.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.776+0000 7f64116d0640 1 -- 172.21.15.100:0/1365792222 >> 172.21.15.100:0/1365792222 conn(0x7f640c0fefe0 msgr2=0x7f640c101430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:48:04.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.776+0000 7f64116d0640 1 -- 172.21.15.100:0/1365792222 shutdown_connections 2024-07-26T13:48:04.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.776+0000 7f64116d0640 1 -- 172.21.15.100:0/1365792222 wait complete. 2024-07-26T13:48:04.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.777+0000 7f64116d0640 1 Processor -- start 2024-07-26T13:48:04.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.777+0000 7f64116d0640 1 -- start start 2024-07-26T13:48:04.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.777+0000 7f64116d0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f640c1035b0 0x7f640c199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:48:04.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.777+0000 7f64116d0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f640c108ef0 0x7f640c19a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:48:04.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.777+0000 7f64116d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f640c109840 0x7f640c19f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:48:04.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.777+0000 7f64116d0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f640c113560 con 0x7f640c109840 2024-07-26T13:48:04.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.777+0000 7f64116d0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f640c1133e0 con 0x7f640c1035b0 2024-07-26T13:48:04.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.777+0000 7f64116d0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f640c1136e0 con 0x7f640c108ef0 2024-07-26T13:48:04.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.777+0000 7f640b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f640c109840 0x7f640c19f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:04.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.777+0000 7f640b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f640c109840 0x7f640c19f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36980/0 (socket says 172.21.15.100:36980) 2024-07-26T13:48:04.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.777+0000 7f640b7fe640 1 -- 172.21.15.100:0/146016097 learned_addr learned my addr 172.21.15.100:0/146016097 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:48:04.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.777+0000 7f640affd640 1 --2- 172.21.15.100:0/146016097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f640c1035b0 0x7f640c199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:04.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.777+0000 7f64027fc640 1 --2- 172.21.15.100:0/146016097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f640c108ef0 0x7f640c19a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:04.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.778+0000 7f640b7fe640 1 -- 172.21.15.100:0/146016097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f640c108ef0 msgr2=0x7f640c19a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:48:04.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.778+0000 7f640b7fe640 1 --2- 172.21.15.100:0/146016097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f640c108ef0 0x7f640c19a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:48:04.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.778+0000 7f640b7fe640 1 -- 172.21.15.100:0/146016097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f640c1035b0 msgr2=0x7f640c199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:48:04.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.778+0000 7f640b7fe640 1 --2- 172.21.15.100:0/146016097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f640c1035b0 0x7f640c199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:48:04.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.778+0000 7f640b7fe640 1 -- 172.21.15.100:0/146016097 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f640c19ff20 con 0x7f640c109840 2024-07-26T13:48:04.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.778+0000 7f640b7fe640 1 --2- 172.21.15.100:0/146016097 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f640c109840 0x7f640c19f7f0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f640400b970 tx=0x7f640400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:48:04.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.778+0000 7f6408ff9640 1 -- 172.21.15.100:0/146016097 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6404019070 con 0x7f640c109840 2024-07-26T13:48:04.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.778+0000 7f64116d0640 1 -- 172.21.15.100:0/146016097 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f640c1a0210 con 0x7f640c109840 2024-07-26T13:48:04.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.778+0000 7f6408ff9640 1 -- 172.21.15.100:0/146016097 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6404014440 con 0x7f640c109840 2024-07-26T13:48:04.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.778+0000 7f6408ff9640 1 -- 172.21.15.100:0/146016097 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64040075d0 con 0x7f640c109840 2024-07-26T13:48:04.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.778+0000 7f64116d0640 1 -- 172.21.15.100:0/146016097 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f640c1a6970 con 0x7f640c109840 2024-07-26T13:48:04.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.780+0000 7f6417c54740 1 Processor -- start 2024-07-26T13:48:04.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.781+0000 7f6408ff9640 1 -- 172.21.15.100:0/146016097 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f640401c070 con 0x7f640c109840 2024-07-26T13:48:04.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.781+0000 7f6417c54740 1 -- start start 2024-07-26T13:48:04.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.781+0000 7f6417c54740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5581facf5840 0x5581fb09b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:48:04.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.781+0000 7f6417c54740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581faccc980 0x5581fb09bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:48:04.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.781+0000 7f6417c54740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581facd4b60 0x5581fb09c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:48:04.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.781+0000 7f6417c54740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5581fb0107b0 con 0x5581faccc980 2024-07-26T13:48:04.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.781+0000 7f6417c54740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5581fafffcb0 con 0x5581facd4b60 2024-07-26T13:48:04.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.781+0000 7f6417c54740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5581fb003cf0 con 0x5581facf5840 2024-07-26T13:48:04.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.781+0000 7f640affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581faccc980 0x5581fb09bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:04.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.781+0000 7f640affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581faccc980 0x5581fb09bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36984/0 (socket says 172.21.15.100:36984) 2024-07-26T13:48:04.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.781+0000 7f640affd640 1 -- 172.21.15.100:0/423748259 learned_addr learned my addr 172.21.15.100:0/423748259 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:48:04.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.781+0000 7f64027fc640 1 --2- 172.21.15.100:0/423748259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5581facf5840 0x5581fb09b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:04.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.781+0000 7f64027fc640 1 --2- 172.21.15.100:0/423748259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581facd4b60 0x5581fb09c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:04.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.781+0000 7f6408ff9640 1 --2- 172.21.15.100:0/146016097 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f63ec0b3d40 0x7f63ec0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:48:04.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.782+0000 7f640affd640 1 -- 172.21.15.100:0/423748259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5581facf5840 msgr2=0x5581fb09b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:48:04.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.782+0000 7f640affd640 1 --2- 172.21.15.100:0/423748259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5581facf5840 0x5581fb09b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:48:04.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.782+0000 7f640affd640 1 -- 172.21.15.100:0/423748259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581facd4b60 msgr2=0x5581fb09c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:48:04.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.782+0000 7f640affd640 1 --2- 172.21.15.100:0/423748259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581facd4b60 0x5581fb09c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:48:04.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.782+0000 7f640affd640 1 -- 172.21.15.100:0/423748259 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581faf950b0 con 0x5581faccc980 2024-07-26T13:48:04.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.782+0000 7f640b7fe640 1 --2- 172.21.15.100:0/146016097 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f63ec0b3d40 0x7f63ec0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:04.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.782+0000 7f64027fc640 1 --2- 172.21.15.100:0/423748259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581facd4b60 0x5581fb09c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:48:04.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.782+0000 7f6408ff9640 1 -- 172.21.15.100:0/146016097 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8828+0+0 (secure 0 0 0) 0x7f64040e3590 con 0x7f640c109840 2024-07-26T13:48:04.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.782+0000 7f640affd640 1 --2- 172.21.15.100:0/423748259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581faccc980 0x5581fb09bdb0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f63fc009900 tx=0x7f63fc032e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d1919ea81853f41 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:48:04.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.782+0000 7f64017fa640 1 -- 172.21.15.100:0/423748259 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63fc040070 con 0x5581faccc980 2024-07-26T13:48:04.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.782+0000 7f64017fa640 1 -- 172.21.15.100:0/423748259 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63fc0316a0 con 0x5581faccc980 2024-07-26T13:48:04.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.782+0000 7f64017fa640 1 -- 172.21.15.100:0/423748259 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63fc035040 con 0x5581faccc980 2024-07-26T13:48:04.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.783+0000 7f6417c54740 1 -- 172.21.15.100:0/423748259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581faccc980 msgr2=0x5581fb09bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:48:04.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.783+0000 7f6417c54740 1 --2- 172.21.15.100:0/423748259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581faccc980 0x5581fb09bdb0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f63fc009900 tx=0x7f63fc032e60 comp rx=0 tx=0).stop 2024-07-26T13:48:04.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.783+0000 7f640b7fe640 1 --2- 172.21.15.100:0/146016097 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f63ec0b3d40 0x7f63ec0b6250 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f64040a74a0 tx=0x7f6404004b50 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:48:04.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.783+0000 7f6417c54740 1 -- 172.21.15.100:0/423748259 shutdown_connections 2024-07-26T13:48:04.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.783+0000 7f6417c54740 1 --2- 172.21.15.100:0/423748259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581facd4b60 0x5581fb09c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:48:04.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.783+0000 7f6417c54740 1 --2- 172.21.15.100:0/423748259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581faccc980 0x5581fb09bdb0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:48:04.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.783+0000 7f6417c54740 1 --2- 172.21.15.100:0/423748259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5581facf5840 0x5581fb09b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:48:04.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.783+0000 7f6417c54740 1 -- 172.21.15.100:0/423748259 >> 172.21.15.100:0/423748259 conn(0x5581face9990 msgr2=0x5581fb099760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:48:04.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.783+0000 7f6417c54740 1 -- 172.21.15.100:0/423748259 shutdown_connections 2024-07-26T13:48:04.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.783+0000 7f6417c54740 1 -- 172.21.15.100:0/423748259 wait complete. 2024-07-26T13:48:04.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.784+0000 7f6417c54740 1 -- start start 2024-07-26T13:48:04.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.784+0000 7f6417c54740 1 Processor -- start 2024-07-26T13:48:04.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.784+0000 7f6417c54740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581faccc980 0x5581fb0a6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:48:04.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.784+0000 7f6417c54740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5581facd4b60 0x5581fb0a7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:48:04.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.784+0000 7f6417c54740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581facf5840 0x5581fb0a7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:48:04.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.784+0000 7f6417c54740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5581fae637a0 con 0x5581facf5840 2024-07-26T13:48:04.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.784+0000 7f6417c54740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5581fafa2320 con 0x5581faccc980 2024-07-26T13:48:04.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.784+0000 7f6417c54740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5581fb0a9ff0 con 0x5581facd4b60 2024-07-26T13:48:04.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.784+0000 7f6417c54740 10 client.0 _ll_register_callbacks cb 0x5581fafac140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:48:04.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.784+0000 7f64027fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581facf5840 0x5581fb0a7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:04.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.784+0000 7f64027fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581facf5840 0x5581fb0a7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36986/0 (socket says 172.21.15.100:36986) 2024-07-26T13:48:04.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.784+0000 7f64027fc640 1 -- 172.21.15.100:0/3807228955 learned_addr learned my addr 172.21.15.100:0/3807228955 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:48:04.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.784+0000 7f64027fc640 1 --2- 172.21.15.100:0/3807228955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581faccc980 0x5581fb0a6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:04.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.784+0000 7f64027fc640 1 -- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5581facd4b60 msgr2=0x5581fb0a7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:48:04.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.784+0000 7f640affd640 1 --2- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5581facd4b60 0x5581fb0a7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:04.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.784+0000 7f64027fc640 1 --2- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5581facd4b60 0x5581fb0a7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:48:04.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.784+0000 7f64027fc640 1 -- 172.21.15.100:0/3807228955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581faccc980 msgr2=0x5581fb0a6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:48:04.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.784+0000 7f64027fc640 1 --2- 172.21.15.100:0/3807228955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5581faccc980 0x5581fb0a6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:48:04.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.784+0000 7f64027fc640 1 -- 172.21.15.100:0/3807228955 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581fafa64b0 con 0x5581facf5840 2024-07-26T13:48:04.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.785+0000 7f64027fc640 1 --2- 172.21.15.100:0/3807228955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5581facf5840 0x5581fb0a7950 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f63f400c760 tx=0x7f63f400cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88cc426e114795c server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:48:04.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.785+0000 7f63fa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:48:04.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.785+0000 7f63fa7fc640 1 -- 172.21.15.100:0/3807228955 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f4004020 con 0x5581facf5840 2024-07-26T13:48:04.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.785+0000 7f63fa7fc640 1 -- 172.21.15.100:0/3807228955 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8828+0+0 (secure 0 0 0) 0x7f63f40041c0 con 0x5581facf5840 2024-07-26T13:48:04.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.785+0000 7f6417c54740 1 -- 172.21.15.100:0/3807228955 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63fc0316a0 con 0x5581facf5840 2024-07-26T13:48:04.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.785+0000 7f63fa7fc640 1 -- 172.21.15.100:0/3807228955 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f63f4028050 con 0x5581facf5840 2024-07-26T13:48:04.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.785+0000 7f6417c54740 10 client.17511 fetch_fsmap learned FSMap version 62 2024-07-26T13:48:04.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.785+0000 7f6417c54740 1 -- 172.21.15.100:0/3807228955 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x5581fad6fb90 con 0x5581facf5840 2024-07-26T13:48:04.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.785+0000 7f63fa7fc640 1 -- 172.21.15.100:0/3807228955 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63fc0316a0 con 0x5581facf5840 2024-07-26T13:48:04.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.785+0000 7f63fa7fc640 1 -- 172.21.15.100:0/3807228955 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 62) ==== 8350+0+0 (secure 0 0 0) 0x7f63f4003c40 con 0x5581facf5840 2024-07-26T13:48:04.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.785+0000 7f6417c54740 10 client.17511 fetch_fsmap finished waiting for FSMap version 62 2024-07-26T13:48:04.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.785+0000 7f6417c54740 10 client.17511 resolve_mds: validated gid 24607 aka daemon mds.i 2024-07-26T13:48:04.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.785+0000 7f6417c54740 20 client.17511 populate_metadata read hostname 'smithi100' 2024-07-26T13:48:04.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.786+0000 7f6417c54740 1 --2- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 0x5581fb0b9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:48:04.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.786+0000 7f6417c54740 4 client.17511 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:48:04.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.786+0000 7f6417c54740 1 -- 172.21.15.100:0/3807228955 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581fb012700 con 0x5581facb41c0 2024-07-26T13:48:04.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.786+0000 7f640affd640 1 -- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 msgr2=0x5581fb0b9000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.167:6838/398150514 2024-07-26T13:48:04.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.786+0000 7f63fa7fc640 1 client.17511 ms_handle_refused on v2:172.21.15.167:6838/398150514 2024-07-26T13:48:04.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.786+0000 7f640affd640 1 --2- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 0x5581fb0b9000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-26T13:48:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:04 smithi138 ceph-mon[36954]: pgmap v609: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.0 MiB/s wr, 291 op/s 2024-07-26T13:48:04.970 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:04 smithi100 ceph-mon[32599]: pgmap v609: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.0 MiB/s wr, 291 op/s 2024-07-26T13:48:04.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.987+0000 7f640affd640 1 -- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 msgr2=0x5581fb0b9000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.167:6838/398150514 2024-07-26T13:48:04.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.987+0000 7f63fa7fc640 1 client.17511 ms_handle_refused on v2:172.21.15.167:6838/398150514 2024-07-26T13:48:04.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:04.987+0000 7f640affd640 1 --2- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 0x5581fb0b9000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-26T13:48:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:04 smithi167 ceph-mon[37140]: pgmap v609: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.0 MiB/s wr, 291 op/s 2024-07-26T13:48:05.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:05.388+0000 7f640affd640 1 -- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 msgr2=0x5581fb0b9000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.167:6838/398150514 2024-07-26T13:48:05.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:05.388+0000 7f640affd640 1 --2- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 0x5581fb0b9000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-07-26T13:48:05.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:05.388+0000 7f63fa7fc640 1 client.17511 ms_handle_refused on v2:172.21.15.167:6838/398150514 2024-07-26T13:48:05.427 INFO:tasks.workunit.client.0.smithi100.stdout: 1 345274 4.14 MB/sec execute 259 sec latency 5003.591 ms 2024-07-26T13:48:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:05 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:48:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:05 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:48:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:05 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:48:05.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:05 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:48:05.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:05 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:48:05.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:05 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:48:06.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:05 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:48:06.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:05 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:48:06.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:05 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:48:06.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:06.036+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.167:6838/398150514 2024-07-26T13:48:06.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:06.036+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-07-26T13:48:06.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:06.036+0000 7fe58dffb640 1 client.17499 ms_handle_refused on v2:172.21.15.167:6838/398150514 2024-07-26T13:48:06.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:06.190+0000 7f640affd640 1 -- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 msgr2=0x5581fb0b9000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.167:6838/398150514 2024-07-26T13:48:06.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:06.190+0000 7f640affd640 1 --2- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 0x5581fb0b9000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-07-26T13:48:06.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:06.190+0000 7f63fa7fc640 1 client.17511 ms_handle_refused on v2:172.21.15.167:6838/398150514 2024-07-26T13:48:06.427 INFO:tasks.workunit.client.0.smithi100.stdout: 1 345274 4.13 MB/sec execute 260 sec latency 6003.666 ms 2024-07-26T13:48:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:06 smithi138 ceph-mon[36954]: pgmap v610: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 211 op/s 2024-07-26T13:48:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:06 smithi100 ceph-mon[32599]: pgmap v610: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 211 op/s 2024-07-26T13:48:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:06 smithi167 ceph-mon[37140]: pgmap v610: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 211 op/s 2024-07-26T13:48:07.427 INFO:tasks.workunit.client.0.smithi100.stdout: 1 345274 4.11 MB/sec execute 261 sec latency 7003.783 ms 2024-07-26T13:48:07.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:07.791+0000 7f640affd640 1 -- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 msgr2=0x5581fb0b9000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.167:6838/398150514 2024-07-26T13:48:07.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:07.791+0000 7f640affd640 1 --2- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 0x5581fb0b9000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-07-26T13:48:07.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:07.791+0000 7f63fa7fc640 1 client.17511 ms_handle_refused on v2:172.21.15.167:6838/398150514 2024-07-26T13:48:08.427 INFO:tasks.workunit.client.0.smithi100.stdout: 1 345274 4.09 MB/sec execute 262 sec latency 8003.885 ms 2024-07-26T13:48:09.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:09.240+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.167:6838/398150514 2024-07-26T13:48:09.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:09.240+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-07-26T13:48:09.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:09.240+0000 7fe58dffb640 1 client.17499 ms_handle_refused on v2:172.21.15.167:6838/398150514 2024-07-26T13:48:09.427 INFO:tasks.workunit.client.0.smithi100.stdout: 1 345274 4.08 MB/sec execute 263 sec latency 9003.933 ms 2024-07-26T13:48:09.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:09 smithi138 ceph-mon[36954]: pgmap v611: 97 pgs: 97 active+clean; 123 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 212 op/s 2024-07-26T13:48:09.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:09 smithi100 ceph-mon[32599]: pgmap v611: 97 pgs: 97 active+clean; 123 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 212 op/s 2024-07-26T13:48:10.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:09 smithi167 ceph-mon[37140]: pgmap v611: 97 pgs: 97 active+clean; 123 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 212 op/s 2024-07-26T13:48:10.427 INFO:tasks.workunit.client.0.smithi100.stdout: 1 345274 4.06 MB/sec execute 264 sec latency 10004.015 ms 2024-07-26T13:48:10.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:10.994+0000 7f640affd640 1 -- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 msgr2=0x5581fb0b9000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.167:6838/398150514 2024-07-26T13:48:10.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:10.994+0000 7f63fa7fc640 1 client.17511 ms_handle_refused on v2:172.21.15.167:6838/398150514 2024-07-26T13:48:10.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:10.994+0000 7f640affd640 1 --2- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 0x5581fb0b9000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-07-26T13:48:11.427 INFO:tasks.workunit.client.0.smithi100.stdout: 1 345274 4.05 MB/sec execute 265 sec latency 11004.130 ms 2024-07-26T13:48:11.797 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:11 smithi167 ceph-mon[37140]: pgmap v612: 97 pgs: 97 active+clean; 125 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 192 op/s 2024-07-26T13:48:11.797 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:11 smithi167 systemd[1]: ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.i.service: Scheduled restart job, restart counter is at 1. 2024-07-26T13:48:11.797 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:11 smithi167 systemd[1]: Stopped Ceph mds.i for bf83e2e6-4b52-11ef-bcb7-c7b262605968. 2024-07-26T13:48:11.797 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:11 smithi167 systemd[1]: ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.i.service: Consumed 1min 49.357s CPU time. 2024-07-26T13:48:11.798 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:11 smithi167 systemd[1]: Starting Ceph mds.i for bf83e2e6-4b52-11ef-bcb7-c7b262605968... 2024-07-26T13:48:11.821 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.818+0000 7f88af80d640 1 -- 172.21.15.100:0/2730646235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88a810d7c0 msgr2=0x7f88a810fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:48:11.821 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.818+0000 7f88af80d640 1 --2- 172.21.15.100:0/2730646235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88a810d7c0 0x7f88a810fcb0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f88a0009900 tx=0x7f88a002a410 comp rx=0 tx=0).stop 2024-07-26T13:48:11.821 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.818+0000 7f88af80d640 1 -- 172.21.15.100:0/2730646235 shutdown_connections 2024-07-26T13:48:11.821 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.818+0000 7f88af80d640 1 --2- 172.21.15.100:0/2730646235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88a810d7c0 0x7f88a810fcb0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:48:11.821 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.818+0000 7f88af80d640 1 --2- 172.21.15.100:0/2730646235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88a8104cb0 0x7f88a810d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:48:11.821 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.818+0000 7f88af80d640 1 --2- 172.21.15.100:0/2730646235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88a8104360 0x7f88a8104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:48:11.821 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.818+0000 7f88af80d640 1 -- 172.21.15.100:0/2730646235 >> 172.21.15.100:0/2730646235 conn(0x7f88a80fde10 msgr2=0x7f88a8100260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:48:11.822 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.819+0000 7f88af80d640 1 -- 172.21.15.100:0/2730646235 shutdown_connections 2024-07-26T13:48:11.822 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.819+0000 7f88af80d640 1 -- 172.21.15.100:0/2730646235 wait complete. 2024-07-26T13:48:11.822 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.819+0000 7f88af80d640 1 Processor -- start 2024-07-26T13:48:11.822 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.819+0000 7f88af80d640 1 -- start start 2024-07-26T13:48:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.819+0000 7f88af80d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88a8104360 0x7f88a8199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:48:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.819+0000 7f88af80d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88a8104cb0 0x7f88a819a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:48:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.820+0000 7f88af80d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88a810d7c0 0x7f88a819f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:48:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.820+0000 7f88af80d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f88a8113430 con 0x7f88a8104cb0 2024-07-26T13:48:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.820+0000 7f88af80d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f88a81132b0 con 0x7f88a810d7c0 2024-07-26T13:48:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.820+0000 7f88af80d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f88a81135b0 con 0x7f88a8104360 2024-07-26T13:48:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.820+0000 7f88acda8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88a8104cb0 0x7f88a819a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.820+0000 7f88acda8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88a8104cb0 0x7f88a819a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39682/0 (socket says 172.21.15.100:39682) 2024-07-26T13:48:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.820+0000 7f88acda8640 1 -- 172.21.15.100:0/3621986241 learned_addr learned my addr 172.21.15.100:0/3621986241 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:48:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.820+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/3621986241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88a8104360 0x7f88a8199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.820+0000 7f88addaa640 1 --2- 172.21.15.100:0/3621986241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88a810d7c0 0x7f88a819f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.820+0000 7f88acda8640 1 -- 172.21.15.100:0/3621986241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88a8104360 msgr2=0x7f88a8199e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:48:11.825 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.820+0000 7f88acda8640 1 --2- 172.21.15.100:0/3621986241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88a8104360 0x7f88a8199e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:48:11.825 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.820+0000 7f88acda8640 1 -- 172.21.15.100:0/3621986241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88a810d7c0 msgr2=0x7f88a819f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:48:11.825 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.820+0000 7f88acda8640 1 --2- 172.21.15.100:0/3621986241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88a810d7c0 0x7f88a819f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:48:11.825 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.820+0000 7f88acda8640 1 -- 172.21.15.100:0/3621986241 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a819fcb0 con 0x7f88a8104cb0 2024-07-26T13:48:11.825 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.820+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/3621986241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88a8104360 0x7f88a8199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:48:11.825 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.820+0000 7f88acda8640 1 --2- 172.21.15.100:0/3621986241 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f88a8104cb0 0x7f88a819a3b0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f889c00db20 tx=0x7f889c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:48:11.825 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.820+0000 7f88a67fc640 1 -- 172.21.15.100:0/3621986241 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c017070 con 0x7f88a8104cb0 2024-07-26T13:48:11.825 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.820+0000 7f88af80d640 1 -- 172.21.15.100:0/3621986241 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88a819ffa0 con 0x7f88a8104cb0 2024-07-26T13:48:11.825 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.820+0000 7f88a67fc640 1 -- 172.21.15.100:0/3621986241 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f889c003e20 con 0x7f88a8104cb0 2024-07-26T13:48:11.825 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.820+0000 7f88a67fc640 1 -- 172.21.15.100:0/3621986241 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c0106c0 con 0x7f88a8104cb0 2024-07-26T13:48:11.825 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.820+0000 7f88af80d640 1 -- 172.21.15.100:0/3621986241 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88a81a67c0 con 0x7f88a8104cb0 2024-07-26T13:48:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.823+0000 7f88b67fe740 1 Processor -- start 2024-07-26T13:48:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.823+0000 7f88b67fe740 1 -- start start 2024-07-26T13:48:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.823+0000 7f88a67fc640 1 -- 172.21.15.100:0/3621986241 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f889c004140 con 0x7f88a8104cb0 2024-07-26T13:48:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.823+0000 7f88b67fe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7065df1b0 0x55c706981ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:48:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.823+0000 7f88b67fe740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7065b62f0 0x55c706982220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:48:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.823+0000 7f88b67fe740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c7065be4d0 0x55c706982760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:48:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.823+0000 7f88b67fe740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c7068f73b0 con 0x55c7065be4d0 2024-07-26T13:48:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.823+0000 7f88b67fe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c7068e68b0 con 0x55c7065df1b0 2024-07-26T13:48:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.823+0000 7f88b67fe740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c7068ea8f0 con 0x55c7065b62f0 2024-07-26T13:48:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.823+0000 7f88acda8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c7065be4d0 0x55c706982760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.823+0000 7f88acda8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c7065be4d0 0x55c706982760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39698/0 (socket says 172.21.15.100:39698) 2024-07-26T13:48:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.823+0000 7f88acda8640 1 -- 172.21.15.100:0/3670510495 learned_addr learned my addr 172.21.15.100:0/3670510495 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:48:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.823+0000 7f88addaa640 1 --2- 172.21.15.100:0/3670510495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7065df1b0 0x55c706981ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.824+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/3670510495 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7065b62f0 0x55c706982220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.824+0000 7f88acda8640 1 -- 172.21.15.100:0/3670510495 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7065b62f0 msgr2=0x55c706982220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:48:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.824+0000 7f88acda8640 1 --2- 172.21.15.100:0/3670510495 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7065b62f0 0x55c706982220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:48:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.824+0000 7f88acda8640 1 -- 172.21.15.100:0/3670510495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7065df1b0 msgr2=0x55c706981ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:48:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.824+0000 7f88acda8640 1 --2- 172.21.15.100:0/3670510495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7065df1b0 0x55c706981ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:48:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.824+0000 7f88acda8640 1 -- 172.21.15.100:0/3670510495 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c70688d9a0 con 0x55c7065be4d0 2024-07-26T13:48:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.824+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/3670510495 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7065b62f0 0x55c706982220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:48:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.824+0000 7f88a67fc640 1 --2- 172.21.15.100:0/3621986241 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f88900b3d40 0x7f88900b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:48:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.824+0000 7f88addaa640 1 --2- 172.21.15.100:0/3621986241 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f88900b3d40 0x7f88900b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.824+0000 7f88acda8640 1 --2- 172.21.15.100:0/3670510495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c7065be4d0 0x55c706982760 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f889c0e14a0 tx=0x7f889c0ec6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb544052234db3e server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:48:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.824+0000 7f88a67fc640 1 -- 172.21.15.100:0/3621986241 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8828+0+0 (secure 0 0 0) 0x7f889c0e3f10 con 0x7f88a8104cb0 2024-07-26T13:48:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.824+0000 7f888f7fe640 1 -- 172.21.15.100:0/3670510495 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c020030 con 0x55c7065be4d0 2024-07-26T13:48:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.824+0000 7f888f7fe640 1 -- 172.21.15.100:0/3670510495 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f889c0b33f0 con 0x55c7065be4d0 2024-07-26T13:48:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.824+0000 7f888f7fe640 1 -- 172.21.15.100:0/3670510495 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c0a68b0 con 0x55c7065be4d0 2024-07-26T13:48:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.825+0000 7f88b67fe740 1 -- 172.21.15.100:0/3670510495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c7065be4d0 msgr2=0x55c706982760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:48:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.825+0000 7f88addaa640 1 --2- 172.21.15.100:0/3621986241 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f88900b3d40 0x7f88900b6250 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f88a0006010 tx=0x7f88a00058d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:48:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.825+0000 7f88b67fe740 1 --2- 172.21.15.100:0/3670510495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c7065be4d0 0x55c706982760 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f889c0e14a0 tx=0x7f889c0ec6f0 comp rx=0 tx=0).stop 2024-07-26T13:48:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.825+0000 7f88b67fe740 1 -- 172.21.15.100:0/3670510495 shutdown_connections 2024-07-26T13:48:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.825+0000 7f88b67fe740 1 --2- 172.21.15.100:0/3670510495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c7065be4d0 0x55c706982760 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:48:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.825+0000 7f88b67fe740 1 --2- 172.21.15.100:0/3670510495 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7065b62f0 0x55c706982220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:48:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.825+0000 7f88b67fe740 1 --2- 172.21.15.100:0/3670510495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7065df1b0 0x55c706981ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:48:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.825+0000 7f88b67fe740 1 -- 172.21.15.100:0/3670510495 >> 172.21.15.100:0/3670510495 conn(0x55c7065d3300 msgr2=0x55c70697fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:48:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.825+0000 7f88b67fe740 1 -- 172.21.15.100:0/3670510495 shutdown_connections 2024-07-26T13:48:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.825+0000 7f88b67fe740 1 -- 172.21.15.100:0/3670510495 wait complete. 2024-07-26T13:48:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.826+0000 7f88b67fe740 1 -- start start 2024-07-26T13:48:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.826+0000 7f88b67fe740 1 Processor -- start 2024-07-26T13:48:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.826+0000 7f88b67fe740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7065b62f0 0x55c70698d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:48:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.826+0000 7f88b67fe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7065be4d0 0x55c70698db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:48:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.826+0000 7f88b67fe740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c7065df1b0 0x55c706999780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:48:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.826+0000 7f88b67fe740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55c706820ab0 con 0x55c7065df1b0 2024-07-26T13:48:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.826+0000 7f88b67fe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c70686ecf0 con 0x55c7065be4d0 2024-07-26T13:48:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.826+0000 7f88b67fe740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c706990460 con 0x55c7065b62f0 2024-07-26T13:48:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.826+0000 7f88b67fe740 10 client.0 _ll_register_callbacks cb 0x55c7068e09f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:48:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.826+0000 7f88addaa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c7065df1b0 0x55c706999780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.826+0000 7f88addaa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c7065df1b0 0x55c706999780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39712/0 (socket says 172.21.15.100:39712) 2024-07-26T13:48:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.826+0000 7f88addaa640 1 -- 172.21.15.100:0/1992671409 learned_addr learned my addr 172.21.15.100:0/1992671409 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:48:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.827+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7065b62f0 0x55c70698d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.827+0000 7f88addaa640 1 -- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7065b62f0 msgr2=0x55c70698d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:48:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.827+0000 7f88addaa640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7065b62f0 0x55c70698d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:48:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.827+0000 7f88addaa640 1 -- 172.21.15.100:0/1992671409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7065be4d0 msgr2=0x55c70698db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:48:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.827+0000 7f88acda8640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7065be4d0 0x55c70698db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.827+0000 7f88addaa640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7065be4d0 0x55c70698db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:48:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.827+0000 7f88addaa640 1 -- 172.21.15.100:0/1992671409 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c70664a790 con 0x55c7065df1b0 2024-07-26T13:48:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.827+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7065b62f0 0x55c70698d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:48:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.827+0000 7f88addaa640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55c7065df1b0 0x55c706999780 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f88a0009900 tx=0x7f88a002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df1da67faac9165a server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:48:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.827+0000 7f888cff9640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:48:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.827+0000 7f888cff9640 1 -- 172.21.15.100:0/1992671409 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a004c070 con 0x55c7065df1b0 2024-07-26T13:48:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.827+0000 7f888cff9640 1 -- 172.21.15.100:0/1992671409 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8828+0+0 (secure 0 0 0) 0x7f88a0004250 con 0x55c7065df1b0 2024-07-26T13:48:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.827+0000 7f88b67fe740 1 -- 172.21.15.100:0/1992671409 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f889c0b33f0 con 0x55c7065df1b0 2024-07-26T13:48:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.827+0000 7f888cff9640 1 -- 172.21.15.100:0/1992671409 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f88a0058050 con 0x55c7065df1b0 2024-07-26T13:48:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.827+0000 7f88b67fe740 10 client.17523 fetch_fsmap learned FSMap version 62 2024-07-26T13:48:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.827+0000 7f88b67fe740 1 -- 172.21.15.100:0/1992671409 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55c7066c3620 con 0x55c7065df1b0 2024-07-26T13:48:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.827+0000 7f888cff9640 1 -- 172.21.15.100:0/1992671409 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f889c0b33f0 con 0x55c7065df1b0 2024-07-26T13:48:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.827+0000 7f888cff9640 1 -- 172.21.15.100:0/1992671409 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 62) ==== 8350+0+0 (secure 0 0 0) 0x7f88a003c4d0 con 0x55c7065df1b0 2024-07-26T13:48:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.828+0000 7f88b67fe740 10 client.17523 fetch_fsmap finished waiting for FSMap version 62 2024-07-26T13:48:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.828+0000 7f88b67fe740 10 client.17523 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-07-26T13:48:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.828+0000 7f88b67fe740 20 client.17523 populate_metadata read hostname 'smithi100' 2024-07-26T13:48:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.828+0000 7f88b67fe740 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:48:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.828+0000 7f88b67fe740 4 client.17523 mds_command: new command op to 24607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:48:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.828+0000 7f88b67fe740 1 -- 172.21.15.100:0/1992671409 --> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7068f9300 con 0x55c70659dd80 2024-07-26T13:48:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.828+0000 7f88ad5a9640 1 -- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 msgr2=0x55c70699f7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.167:6838/398150514 2024-07-26T13:48:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.828+0000 7f888cff9640 1 client.17523 ms_handle_refused on v2:172.21.15.167:6838/398150514 2024-07-26T13:48:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:11.828+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-07-26T13:48:11.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:11 smithi138 ceph-mon[36954]: pgmap v612: 97 pgs: 97 active+clean; 125 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 192 op/s 2024-07-26T13:48:11.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:11 smithi100 ceph-mon[32599]: pgmap v612: 97 pgs: 97 active+clean; 125 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 192 op/s 2024-07-26T13:48:12.030 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:12.029+0000 7f88ad5a9640 1 -- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 msgr2=0x55c70699f7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.167:6838/398150514 2024-07-26T13:48:12.031 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:12.029+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-07-26T13:48:12.031 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:12.029+0000 7f888cff9640 1 client.17523 ms_handle_refused on v2:172.21.15.167:6838/398150514 2024-07-26T13:48:12.077 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:11 smithi167 podman[73143]: 2024-07-26 13:48:11.699372025 +0000 UTC m=+0.017546296 image pull ae51f432d4fa82622bc253ca9e2da7666c617e096e99d43d0a0e4fdaca8865f8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2 2024-07-26T13:48:12.078 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:11 smithi167 podman[73143]: 2024-07-26 13:48:11.832711599 +0000 UTC m=+0.150885866 container create 46b8d9340e10bf84e186bbd7f44767cd63f325eb5887921d4282d63769901e5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i, GIT_CLEAN=True, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb) 2024-07-26T13:48:12.352 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:12 smithi167 podman[73143]: 2024-07-26 13:48:12.077530867 +0000 UTC m=+0.395705128 container init 46b8d9340e10bf84e186bbd7f44767cd63f325eb5887921d4282d63769901e5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-26T13:48:12.353 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:12 smithi167 podman[73143]: 2024-07-26 13:48:12.080726571 +0000 UTC m=+0.398900841 container start 46b8d9340e10bf84e186bbd7f44767cd63f325eb5887921d4282d63769901e5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_CLEAN=True) 2024-07-26T13:48:12.353 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:12 smithi167 ceph-mds[73162]: -- 172.21.15.167:0/3343865894 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56235ecc7c20 con 0x562362e69600 2024-07-26T13:48:12.353 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:12 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-i[73158]: starting mds.i at 2024-07-26T13:48:12.353 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:12 smithi167 bash[73143]: 46b8d9340e10bf84e186bbd7f44767cd63f325eb5887921d4282d63769901e5a 2024-07-26T13:48:12.353 INFO:journalctl@ceph.mds.i.smithi167.stdout:Jul 26 13:48:12 smithi167 systemd[1]: Started Ceph mds.i for bf83e2e6-4b52-11ef-bcb7-c7b262605968. 2024-07-26T13:48:12.428 INFO:tasks.workunit.client.0.smithi100.stdout: 1 345274 4.03 MB/sec execute 266 sec latency 12004.213 ms 2024-07-26T13:48:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:12.430+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:12.430+0000 7f88ad5a9640 1 -- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 msgr2=0x55c70699f7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-07-26T13:48:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:12.430+0000 7f88ad5a9640 1 -- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 msgr2=0x55c70699f7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:48:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:12.430+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:48:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:12.430+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-07-26T13:48:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:12 smithi138 ceph-mon[36954]: daemon mds.i restarted 2024-07-26T13:48:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:12 smithi138 ceph-mon[36954]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-26T13:48:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:12 smithi138 ceph-mon[36954]: osdmap e88: 12 total, 12 up, 12 in 2024-07-26T13:48:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:12 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.167:6838/4179082337,v1:172.21.15.167:6839/4179082337] up:boot 2024-07-26T13:48:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:12 smithi138 ceph-mon[36954]: Standby daemon mds.f assigned to filesystem cephfs as rank 0 2024-07-26T13:48:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:12 smithi138 ceph-mon[36954]: fsmap cephfs:2/3 {1=b=up:active,2=c=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-07-26T13:48:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:12 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-26T13:48:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:12 smithi138 ceph-mon[36954]: fsmap cephfs:3/3 {0=f=up:replay,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:48:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:12 smithi138 ceph-mon[36954]: pgmap v614: 97 pgs: 97 active+clean; 128 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 1.0 MiB/s wr, 92 op/s 2024-07-26T13:48:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:12 smithi100 ceph-mon[32599]: daemon mds.i restarted 2024-07-26T13:48:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:12 smithi100 ceph-mon[32599]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-26T13:48:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:12 smithi100 ceph-mon[32599]: osdmap e88: 12 total, 12 up, 12 in 2024-07-26T13:48:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:12 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.167:6838/4179082337,v1:172.21.15.167:6839/4179082337] up:boot 2024-07-26T13:48:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:12 smithi100 ceph-mon[32599]: Standby daemon mds.f assigned to filesystem cephfs as rank 0 2024-07-26T13:48:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:12 smithi100 ceph-mon[32599]: fsmap cephfs:2/3 {1=b=up:active,2=c=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-07-26T13:48:12.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:12 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-26T13:48:12.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:12 smithi100 ceph-mon[32599]: fsmap cephfs:3/3 {0=f=up:replay,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:48:12.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:12 smithi100 ceph-mon[32599]: pgmap v614: 97 pgs: 97 active+clean; 128 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 1.0 MiB/s wr, 92 op/s 2024-07-26T13:48:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:12 smithi167 ceph-mon[37140]: daemon mds.i restarted 2024-07-26T13:48:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:12 smithi167 ceph-mon[37140]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-26T13:48:13.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:12 smithi167 ceph-mon[37140]: osdmap e88: 12 total, 12 up, 12 in 2024-07-26T13:48:13.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:12 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.167:6838/4179082337,v1:172.21.15.167:6839/4179082337] up:boot 2024-07-26T13:48:13.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:12 smithi167 ceph-mon[37140]: Standby daemon mds.f assigned to filesystem cephfs as rank 0 2024-07-26T13:48:13.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:12 smithi167 ceph-mon[37140]: fsmap cephfs:2/3 {1=b=up:active,2=c=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-07-26T13:48:13.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:12 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-26T13:48:13.027 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:12 smithi167 ceph-mon[37140]: fsmap cephfs:3/3 {0=f=up:replay,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:48:13.027 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:12 smithi167 ceph-mon[37140]: pgmap v614: 97 pgs: 97 active+clean; 128 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 1.0 MiB/s wr, 92 op/s 2024-07-26T13:48:13.234 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:13.232+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:13.235 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:13.232+0000 7f88ad5a9640 1 -- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 msgr2=0x55c70699f7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-07-26T13:48:13.235 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:13.232+0000 7f88ad5a9640 1 -- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 msgr2=0x55c70699f7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:48:13.235 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:13.232+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:48:13.235 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:13.232+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-07-26T13:48:13.427 INFO:tasks.workunit.client.0.smithi100.stdout: 1 345274 4.02 MB/sec execute 267 sec latency 13004.273 ms 2024-07-26T13:48:14.428 INFO:tasks.workunit.client.0.smithi100.stdout: 1 345274 4.00 MB/sec execute 268 sec latency 14004.350 ms 2024-07-26T13:48:14.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:14 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.167:6834/641259145,v1:172.21.15.167:6835/641259145] up:resolve 2024-07-26T13:48:14.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:14 smithi138 ceph-mon[36954]: fsmap cephfs:3/3 {0=f=up:resolve,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:48:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:14 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.167:6834/641259145,v1:172.21.15.167:6835/641259145] up:resolve 2024-07-26T13:48:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:14 smithi100 ceph-mon[32599]: fsmap cephfs:3/3 {0=f=up:resolve,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:48:14.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:14 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.167:6834/641259145,v1:172.21.15.167:6835/641259145] up:resolve 2024-07-26T13:48:14.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:14 smithi167 ceph-mon[37140]: fsmap cephfs:3/3 {0=f=up:resolve,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:48:14.835 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:14.834+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:14.835+0000 7f88ad5a9640 1 -- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 msgr2=0x55c70699f7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-07-26T13:48:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:14.835+0000 7f88ad5a9640 1 -- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 msgr2=0x55c70699f7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:48:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:14.835+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:48:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:14.835+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-07-26T13:48:15.428 INFO:tasks.workunit.client.0.smithi100.stdout: 1 345274 3.99 MB/sec execute 269 sec latency 15004.385 ms 2024-07-26T13:48:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:15 smithi138 ceph-mon[36954]: reconnect by client.15600 172.21.15.100:0/4280938035 after 0 2024-07-26T13:48:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:15 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.167:6834/641259145,v1:172.21.15.167:6835/641259145] up:reconnect 2024-07-26T13:48:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:15 smithi138 ceph-mon[36954]: fsmap cephfs:3/3 {0=f=up:reconnect,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:48:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:15 smithi138 ceph-mon[36954]: reconnect by client.15333 v1:192.168.0.1:0/1052122651 after 0.00299995 2024-07-26T13:48:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:15 smithi138 ceph-mon[36954]: pgmap v615: 97 pgs: 97 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 1.4 MiB/s wr, 39 op/s 2024-07-26T13:48:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:15 smithi100 ceph-mon[32599]: reconnect by client.15600 172.21.15.100:0/4280938035 after 0 2024-07-26T13:48:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:15 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.167:6834/641259145,v1:172.21.15.167:6835/641259145] up:reconnect 2024-07-26T13:48:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:15 smithi100 ceph-mon[32599]: fsmap cephfs:3/3 {0=f=up:reconnect,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:48:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:15 smithi100 ceph-mon[32599]: reconnect by client.15333 v1:192.168.0.1:0/1052122651 after 0.00299995 2024-07-26T13:48:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:15 smithi100 ceph-mon[32599]: pgmap v615: 97 pgs: 97 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 1.4 MiB/s wr, 39 op/s 2024-07-26T13:48:15.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:15 smithi167 ceph-mon[37140]: reconnect by client.15600 172.21.15.100:0/4280938035 after 0 2024-07-26T13:48:15.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:15 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.167:6834/641259145,v1:172.21.15.167:6835/641259145] up:reconnect 2024-07-26T13:48:15.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:15 smithi167 ceph-mon[37140]: fsmap cephfs:3/3 {0=f=up:reconnect,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:48:15.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:15 smithi167 ceph-mon[37140]: reconnect by client.15333 v1:192.168.0.1:0/1052122651 after 0.00299995 2024-07-26T13:48:15.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:15 smithi167 ceph-mon[37140]: pgmap v615: 97 pgs: 97 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 1.4 MiB/s wr, 39 op/s 2024-07-26T13:48:15.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:15.646+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:15.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:15.647+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:48:15.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:15.647+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:48:15.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:15.647+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:48:15.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:15.647+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 12.800000 2024-07-26T13:48:16.428 INFO:tasks.workunit.client.0.smithi100.stdout: 1 345480 3.98 MB/sec execute 270 sec latency 15735.101 ms 2024-07-26T13:48:16.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:16 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.167:6834/641259145,v1:172.21.15.167:6835/641259145] up:rejoin 2024-07-26T13:48:16.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:16 smithi138 ceph-mon[36954]: fsmap cephfs:3/3 {0=f=up:rejoin,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:48:16.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:16 smithi138 ceph-mon[36954]: daemon mds.f is now active in filesystem cephfs as rank 0 2024-07-26T13:48:16.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:16 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.167:6834/641259145,v1:172.21.15.167:6835/641259145] up:rejoin 2024-07-26T13:48:16.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:16 smithi100 ceph-mon[32599]: fsmap cephfs:3/3 {0=f=up:rejoin,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:48:16.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:16 smithi100 ceph-mon[32599]: daemon mds.f is now active in filesystem cephfs as rank 0 2024-07-26T13:48:16.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:16 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.167:6834/641259145,v1:172.21.15.167:6835/641259145] up:rejoin 2024-07-26T13:48:16.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:16 smithi167 ceph-mon[37140]: fsmap cephfs:3/3 {0=f=up:rejoin,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:48:16.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:16 smithi167 ceph-mon[37140]: daemon mds.f is now active in filesystem cephfs as rank 0 2024-07-26T13:48:17.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:17.401+0000 7f640affd640 1 --2- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 0x5581fb0b9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:17.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:17.401+0000 7f640affd640 1 -- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 msgr2=0x5581fb0b9000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:48:17.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:17.401+0000 7f640affd640 1 -- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 msgr2=0x5581fb0b9000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:48:17.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:17.401+0000 7f640affd640 1 --2- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 0x5581fb0b9000 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:48:17.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:17.401+0000 7f640affd640 1 --2- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 0x5581fb0b9000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 12.800000 2024-07-26T13:48:17.428 INFO:tasks.workunit.client.0.smithi100.stdout: 1 347392 4.00 MB/sec execute 271 sec latency 30.162 ms 2024-07-26T13:48:17.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:17 smithi138 ceph-mon[36954]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-26T13:48:17.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:17 smithi138 ceph-mon[36954]: Cluster is now healthy 2024-07-26T13:48:17.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:17 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.167:6834/641259145,v1:172.21.15.167:6835/641259145] up:active 2024-07-26T13:48:17.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:17 smithi138 ceph-mon[36954]: fsmap cephfs:3 {0=f=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:48:17.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:17 smithi138 ceph-mon[36954]: fsmap cephfs:3 {0=f=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:48:17.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:17 smithi138 ceph-mon[36954]: pgmap v616: 97 pgs: 97 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 1.5 MiB/s wr, 46 op/s 2024-07-26T13:48:17.459 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:17 smithi167 ceph-mon[37140]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-26T13:48:17.459 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:17 smithi167 ceph-mon[37140]: Cluster is now healthy 2024-07-26T13:48:17.459 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:17 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.167:6834/641259145,v1:172.21.15.167:6835/641259145] up:active 2024-07-26T13:48:17.459 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:17 smithi167 ceph-mon[37140]: fsmap cephfs:3 {0=f=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:48:17.460 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:17 smithi167 ceph-mon[37140]: fsmap cephfs:3 {0=f=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:48:17.460 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:17 smithi167 ceph-mon[37140]: pgmap v616: 97 pgs: 97 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 1.5 MiB/s wr, 46 op/s 2024-07-26T13:48:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:17 smithi100 ceph-mon[32599]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-26T13:48:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:17 smithi100 ceph-mon[32599]: Cluster is now healthy 2024-07-26T13:48:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:17 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.167:6834/641259145,v1:172.21.15.167:6835/641259145] up:active 2024-07-26T13:48:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:17 smithi100 ceph-mon[32599]: fsmap cephfs:3 {0=f=up:active,1=b=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:48:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:17 smithi100 ceph-mon[32599]: fsmap cephfs:3 {0=f=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:48:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:17 smithi100 ceph-mon[32599]: pgmap v616: 97 pgs: 97 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 1.5 MiB/s wr, 46 op/s 2024-07-26T13:48:17.773 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jul 26 13:48:17 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-f[68732]: 2024-07-26T13:48:17.457+0000 7f6a1c36f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-26T13:48:17.773 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jul 26 13:48:17 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-f[68732]: 2024-07-26T13:48:17.457+0000 7f6a1c36f640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-07-26T13:48:17.773 INFO:journalctl@ceph.mds.f.smithi167.stdout:Jul 26 13:48:17 smithi167 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-f[68732]: 2024-07-26T13:48:17.457+0000 7f6a1c36f640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-07-26T13:48:18.038 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:18.037+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:18.106 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:18.037+0000 7f88ad5a9640 1 -- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 msgr2=0x55c70699f7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-07-26T13:48:18.107 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:18.037+0000 7f88ad5a9640 1 -- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 msgr2=0x55c70699f7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:48:18.107 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:18.038+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:48:18.107 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:18.038+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-07-26T13:48:18.428 INFO:tasks.workunit.client.0.smithi100.stdout: 1 348718 4.01 MB/sec execute 272 sec latency 22.218 ms 2024-07-26T13:48:18.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:18 smithi138 ceph-mon[36954]: osdmap e89: 12 total, 12 up, 12 in 2024-07-26T13:48:18.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:18 smithi100 ceph-mon[32599]: osdmap e89: 12 total, 12 up, 12 in 2024-07-26T13:48:18.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:18 smithi167 ceph-mon[37140]: osdmap e89: 12 total, 12 up, 12 in 2024-07-26T13:48:19.428 INFO:tasks.workunit.client.0.smithi100.stdout: 1 351353 4.06 MB/sec execute 273 sec latency 16.361 ms 2024-07-26T13:48:19.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:19 smithi138 ceph-mon[36954]: pgmap v618: 97 pgs: 97 active+clean; 132 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 78 op/s 2024-07-26T13:48:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:19 smithi100 ceph-mon[32599]: pgmap v618: 97 pgs: 97 active+clean; 132 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 78 op/s 2024-07-26T13:48:19.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:19 smithi167 ceph-mon[37140]: pgmap v618: 97 pgs: 97 active+clean; 132 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 78 op/s 2024-07-26T13:48:20.428 INFO:tasks.workunit.client.0.smithi100.stdout: 1 352677 4.07 MB/sec execute 274 sec latency 19.521 ms 2024-07-26T13:48:21.428 INFO:tasks.workunit.client.0.smithi100.stdout: 1 355028 4.10 MB/sec execute 275 sec latency 12.006 ms 2024-07-26T13:48:21.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:21 smithi138 ceph-mon[36954]: fsmap cephfs:3 {0=f=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:48:21.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:21 smithi138 ceph-mon[36954]: pgmap v619: 97 pgs: 97 active+clean; 136 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 148 op/s 2024-07-26T13:48:21.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:21 smithi100 ceph-mon[32599]: fsmap cephfs:3 {0=f=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:48:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:21 smithi100 ceph-mon[32599]: pgmap v619: 97 pgs: 97 active+clean; 136 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 148 op/s 2024-07-26T13:48:21.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:21 smithi167 ceph-mon[37140]: fsmap cephfs:3 {0=f=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:48:21.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:21 smithi167 ceph-mon[37140]: pgmap v619: 97 pgs: 97 active+clean; 136 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 148 op/s 2024-07-26T13:48:22.428 INFO:tasks.workunit.client.0.smithi100.stdout: 1 355232 4.09 MB/sec execute 276 sec latency 722.156 ms 2024-07-26T13:48:23.428 INFO:tasks.workunit.client.0.smithi100.stdout: 1 355232 4.07 MB/sec execute 277 sec latency 1722.243 ms 2024-07-26T13:48:23.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:23 smithi167 ceph-mon[37140]: pgmap v620: 97 pgs: 97 active+clean; 137 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 161 op/s 2024-07-26T13:48:23.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:23 smithi138 ceph-mon[36954]: pgmap v620: 97 pgs: 97 active+clean; 137 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 161 op/s 2024-07-26T13:48:23.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:23 smithi100 ceph-mon[32599]: pgmap v620: 97 pgs: 97 active+clean; 137 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 161 op/s 2024-07-26T13:48:24.428 INFO:tasks.workunit.client.0.smithi100.stdout: 1 355232 4.06 MB/sec execute 278 sec latency 2722.364 ms 2024-07-26T13:48:24.445 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:24.444+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:24.445+0000 7f88ad5a9640 1 -- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 msgr2=0x55c70699f7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-07-26T13:48:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:24.445+0000 7f88ad5a9640 1 -- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 msgr2=0x55c70699f7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:48:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:24.445+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:48:24.446 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:24.445+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 12.800000 2024-07-26T13:48:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:24 smithi100 ceph-mon[32599]: osdmap e90: 12 total, 12 up, 12 in 2024-07-26T13:48:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:24 smithi100 ceph-mon[32599]: pgmap v622: 97 pgs: 97 active+clean; 130 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.5 MiB/s wr, 217 op/s 2024-07-26T13:48:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:24 smithi167 ceph-mon[37140]: osdmap e90: 12 total, 12 up, 12 in 2024-07-26T13:48:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:24 smithi167 ceph-mon[37140]: pgmap v622: 97 pgs: 97 active+clean; 130 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.5 MiB/s wr, 217 op/s 2024-07-26T13:48:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:24 smithi138 ceph-mon[36954]: osdmap e90: 12 total, 12 up, 12 in 2024-07-26T13:48:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:24 smithi138 ceph-mon[36954]: pgmap v622: 97 pgs: 97 active+clean; 130 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.5 MiB/s wr, 217 op/s 2024-07-26T13:48:25.428 INFO:tasks.workunit.client.0.smithi100.stdout: 1 355232 4.04 MB/sec execute 279 sec latency 3722.450 ms 2024-07-26T13:48:26.428 INFO:tasks.workunit.client.0.smithi100.stdout: 1 355232 4.03 MB/sec execute 280 sec latency 4722.548 ms 2024-07-26T13:48:27.428 INFO:tasks.workunit.client.0.smithi100.stdout: 1 355232 4.01 MB/sec execute 281 sec latency 5722.671 ms 2024-07-26T13:48:27.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:27 smithi138 ceph-mon[36954]: pgmap v623: 97 pgs: 97 active+clean; 133 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 220 op/s 2024-07-26T13:48:27.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:27 smithi100 ceph-mon[32599]: pgmap v623: 97 pgs: 97 active+clean; 133 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 220 op/s 2024-07-26T13:48:28.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:27 smithi167 ceph-mon[37140]: pgmap v623: 97 pgs: 97 active+clean; 133 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 220 op/s 2024-07-26T13:48:28.429 INFO:tasks.workunit.client.0.smithi100.stdout: 1 355232 4.00 MB/sec execute 282 sec latency 6722.747 ms 2024-07-26T13:48:28.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:28.460+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:28.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:28.461+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:48:28.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:28.461+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:48:28.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:28.461+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:48:28.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:28.461+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:48:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:28 smithi100 ceph-mon[32599]: pgmap v624: 97 pgs: 97 active+clean; 135 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 198 op/s 2024-07-26T13:48:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:28 smithi138 ceph-mon[36954]: pgmap v624: 97 pgs: 97 active+clean; 135 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 198 op/s 2024-07-26T13:48:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:28 smithi167 ceph-mon[37140]: pgmap v624: 97 pgs: 97 active+clean; 135 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 198 op/s 2024-07-26T13:48:29.429 INFO:tasks.workunit.client.0.smithi100.stdout: 1 355232 3.99 MB/sec execute 283 sec latency 7722.837 ms 2024-07-26T13:48:30.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:30.215+0000 7f640affd640 1 --2- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 0x5581fb0b9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:30.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:30.215+0000 7f640affd640 1 -- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 msgr2=0x5581fb0b9000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:48:30.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:30.215+0000 7f640affd640 1 -- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 msgr2=0x5581fb0b9000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:48:30.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:30.215+0000 7f640affd640 1 --2- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 0x5581fb0b9000 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:48:30.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:30.215+0000 7f640affd640 1 --2- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 0x5581fb0b9000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:48:30.429 INFO:tasks.workunit.client.0.smithi100.stdout: 1 355232 3.97 MB/sec execute 284 sec latency 8722.937 ms 2024-07-26T13:48:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:30 smithi138 ceph-mon[36954]: pgmap v625: 97 pgs: 97 active+clean; 138 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 147 op/s 2024-07-26T13:48:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:30 smithi100 ceph-mon[32599]: pgmap v625: 97 pgs: 97 active+clean; 138 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 147 op/s 2024-07-26T13:48:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:30 smithi167 ceph-mon[37140]: pgmap v625: 97 pgs: 97 active+clean; 138 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 147 op/s 2024-07-26T13:48:31.429 INFO:tasks.workunit.client.0.smithi100.stdout: 1 355232 3.96 MB/sec execute 285 sec latency 9723.042 ms 2024-07-26T13:48:32.429 INFO:tasks.workunit.client.0.smithi100.stdout: 1 355232 3.94 MB/sec execute 286 sec latency 10723.121 ms 2024-07-26T13:48:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:32 smithi138 ceph-mon[36954]: pgmap v626: 97 pgs: 97 active+clean; 137 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 120 op/s 2024-07-26T13:48:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:32 smithi100 ceph-mon[32599]: pgmap v626: 97 pgs: 97 active+clean; 137 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 120 op/s 2024-07-26T13:48:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:32 smithi167 ceph-mon[37140]: pgmap v626: 97 pgs: 97 active+clean; 137 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 120 op/s 2024-07-26T13:48:33.429 INFO:tasks.workunit.client.0.smithi100.stdout: 1 355232 3.93 MB/sec execute 287 sec latency 11723.191 ms 2024-07-26T13:48:34.429 INFO:tasks.workunit.client.0.smithi100.stdout: 1 355232 3.92 MB/sec execute 288 sec latency 12723.279 ms 2024-07-26T13:48:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:34 smithi138 ceph-mon[36954]: pgmap v627: 97 pgs: 97 active+clean; 133 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 1.8 MiB/s wr, 96 op/s 2024-07-26T13:48:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:34 smithi100 ceph-mon[32599]: pgmap v627: 97 pgs: 97 active+clean; 133 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 1.8 MiB/s wr, 96 op/s 2024-07-26T13:48:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:34 smithi167 ceph-mon[37140]: pgmap v627: 97 pgs: 97 active+clean; 133 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 1.8 MiB/s wr, 96 op/s 2024-07-26T13:48:35.429 INFO:tasks.workunit.client.0.smithi100.stdout: 1 355306 3.90 MB/sec execute 289 sec latency 13648.782 ms 2024-07-26T13:48:36.429 INFO:tasks.workunit.client.0.smithi100.stdout: 1 357895 3.94 MB/sec execute 290 sec latency 18.989 ms 2024-07-26T13:48:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:36 smithi138 ceph-mon[36954]: pgmap v628: 97 pgs: 97 active+clean; 131 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 1.7 MiB/s wr, 91 op/s 2024-07-26T13:48:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:36 smithi100 ceph-mon[32599]: pgmap v628: 97 pgs: 97 active+clean; 131 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 1.7 MiB/s wr, 91 op/s 2024-07-26T13:48:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:36 smithi167 ceph-mon[37140]: pgmap v628: 97 pgs: 97 active+clean; 131 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 1.7 MiB/s wr, 91 op/s 2024-07-26T13:48:37.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:37.258+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:37.259 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:37.258+0000 7f88ad5a9640 1 -- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 msgr2=0x55c70699f7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-07-26T13:48:37.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:37.258+0000 7f88ad5a9640 1 -- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 msgr2=0x55c70699f7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:48:37.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:37.258+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:48:37.260 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:37.258+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:48:37.429 INFO:tasks.workunit.client.0.smithi100.stdout: 1 359634 3.96 MB/sec execute 291 sec latency 20.594 ms 2024-07-26T13:48:38.429 INFO:tasks.workunit.client.0.smithi100.stdout: 1 362003 3.99 MB/sec execute 292 sec latency 13.745 ms 2024-07-26T13:48:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:38 smithi138 ceph-mon[36954]: pgmap v629: 97 pgs: 97 active+clean; 132 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 1.7 MiB/s wr, 99 op/s 2024-07-26T13:48:38.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:38 smithi100 ceph-mon[32599]: pgmap v629: 97 pgs: 97 active+clean; 132 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 1.7 MiB/s wr, 99 op/s 2024-07-26T13:48:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:38 smithi167 ceph-mon[37140]: pgmap v629: 97 pgs: 97 active+clean; 132 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 1.7 MiB/s wr, 99 op/s 2024-07-26T13:48:39.429 INFO:tasks.workunit.client.0.smithi100.stdout: 1 363350 4.00 MB/sec execute 293 sec latency 24.026 ms 2024-07-26T13:48:40.429 INFO:tasks.workunit.client.0.smithi100.stdout: 1 365677 4.03 MB/sec execute 294 sec latency 14.338 ms 2024-07-26T13:48:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:40 smithi138 ceph-mon[36954]: pgmap v630: 97 pgs: 97 active+clean; 133 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 119 op/s 2024-07-26T13:48:40.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:40 smithi100 ceph-mon[32599]: pgmap v630: 97 pgs: 97 active+clean; 133 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 119 op/s 2024-07-26T13:48:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:40 smithi167 ceph-mon[37140]: pgmap v630: 97 pgs: 97 active+clean; 133 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 119 op/s 2024-07-26T13:48:41.429 INFO:tasks.workunit.client.0.smithi100.stdout: 1 367480 4.04 MB/sec execute 295 sec latency 14.235 ms 2024-07-26T13:48:42.430 INFO:tasks.workunit.client.0.smithi100.stdout: 1 369313 4.07 MB/sec execute 296 sec latency 11.896 ms 2024-07-26T13:48:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:42 smithi138 ceph-mon[36954]: pgmap v631: 97 pgs: 97 active+clean; 135 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 138 op/s 2024-07-26T13:48:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:42 smithi100 ceph-mon[32599]: pgmap v631: 97 pgs: 97 active+clean; 135 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 138 op/s 2024-07-26T13:48:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:42 smithi167 ceph-mon[37140]: pgmap v631: 97 pgs: 97 active+clean; 135 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 138 op/s 2024-07-26T13:48:43.429 INFO:tasks.workunit.client.0.smithi100.stdout: 1 371220 4.08 MB/sec execute 297 sec latency 20.844 ms 2024-07-26T13:48:43.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:43.476+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:43.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:43.477+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:48:43.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:43.477+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:48:43.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:43.477+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:48:43.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:43.477+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:48:44.430 INFO:tasks.workunit.client.0.smithi100.stdout: 1 372901 4.11 MB/sec execute 298 sec latency 9.844 ms 2024-07-26T13:48:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:44 smithi138 ceph-mon[36954]: pgmap v632: 97 pgs: 97 active+clean; 138 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 158 op/s 2024-07-26T13:48:44.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:44 smithi100 ceph-mon[32599]: pgmap v632: 97 pgs: 97 active+clean; 138 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 158 op/s 2024-07-26T13:48:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:44 smithi167 ceph-mon[37140]: pgmap v632: 97 pgs: 97 active+clean; 138 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 158 op/s 2024-07-26T13:48:45.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:45.231+0000 7f640affd640 1 --2- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 0x5581fb0b9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:45.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:45.231+0000 7f640affd640 1 -- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 msgr2=0x5581fb0b9000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:48:45.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:45.231+0000 7f640affd640 1 -- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 msgr2=0x5581fb0b9000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:48:45.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:45.231+0000 7f640affd640 1 --2- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 0x5581fb0b9000 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:48:45.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:45.231+0000 7f640affd640 1 --2- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 0x5581fb0b9000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:48:45.430 INFO:tasks.workunit.client.0.smithi100.stdout: 1 374885 4.12 MB/sec execute 299 sec latency 20.611 ms 2024-07-26T13:48:46.430 INFO:tasks.workunit.client.0.smithi100.stdout: 1 376712 4.14 MB/sec execute 300 sec latency 12.736 ms 2024-07-26T13:48:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:46 smithi138 ceph-mon[36954]: pgmap v633: 97 pgs: 97 active+clean; 142 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.7 MiB/s wr, 183 op/s 2024-07-26T13:48:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:46 smithi100 ceph-mon[32599]: pgmap v633: 97 pgs: 97 active+clean; 142 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.7 MiB/s wr, 183 op/s 2024-07-26T13:48:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:46 smithi167 ceph-mon[37140]: pgmap v633: 97 pgs: 97 active+clean; 142 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.7 MiB/s wr, 183 op/s 2024-07-26T13:48:47.430 INFO:tasks.workunit.client.0.smithi100.stdout: 1 379363 4.18 MB/sec execute 301 sec latency 14.846 ms 2024-07-26T13:48:48.430 INFO:tasks.workunit.client.0.smithi100.stdout: 1 380802 4.19 MB/sec execute 302 sec latency 20.502 ms 2024-07-26T13:48:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:48 smithi138 ceph-mon[36954]: pgmap v634: 97 pgs: 97 active+clean; 148 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.2 MiB/s wr, 225 op/s 2024-07-26T13:48:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:48 smithi100 ceph-mon[32599]: pgmap v634: 97 pgs: 97 active+clean; 148 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.2 MiB/s wr, 225 op/s 2024-07-26T13:48:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:48 smithi167 ceph-mon[37140]: pgmap v634: 97 pgs: 97 active+clean; 148 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.2 MiB/s wr, 225 op/s 2024-07-26T13:48:49.430 INFO:tasks.workunit.client.0.smithi100.stdout: 1 383316 4.23 MB/sec execute 303 sec latency 14.757 ms 2024-07-26T13:48:50.430 INFO:tasks.workunit.client.0.smithi100.stdout: 1 384634 4.23 MB/sec execute 304 sec latency 20.017 ms 2024-07-26T13:48:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:50 smithi138 ceph-mon[36954]: pgmap v635: 97 pgs: 97 active+clean; 152 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.3 MiB/s wr, 234 op/s 2024-07-26T13:48:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:50 smithi100 ceph-mon[32599]: pgmap v635: 97 pgs: 97 active+clean; 152 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.3 MiB/s wr, 234 op/s 2024-07-26T13:48:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:50 smithi167 ceph-mon[37140]: pgmap v635: 97 pgs: 97 active+clean; 152 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.3 MiB/s wr, 234 op/s 2024-07-26T13:48:51.430 INFO:tasks.workunit.client.0.smithi100.stdout: 1 386965 4.26 MB/sec execute 305 sec latency 12.051 ms 2024-07-26T13:48:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:52.270+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:52.270+0000 7f88ad5a9640 1 -- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 msgr2=0x55c70699f7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-07-26T13:48:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:52.270+0000 7f88ad5a9640 1 -- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 msgr2=0x55c70699f7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:48:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:52.270+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:48:52.273 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:48:52.270+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:48:52.430 INFO:tasks.workunit.client.0.smithi100.stdout: 1 388502 4.27 MB/sec execute 306 sec latency 74.303 ms 2024-07-26T13:48:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:52 smithi138 ceph-mon[36954]: pgmap v636: 97 pgs: 97 active+clean; 154 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.1 MiB/s wr, 220 op/s 2024-07-26T13:48:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:52 smithi100 ceph-mon[32599]: pgmap v636: 97 pgs: 97 active+clean; 154 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.1 MiB/s wr, 220 op/s 2024-07-26T13:48:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:52 smithi167 ceph-mon[37140]: pgmap v636: 97 pgs: 97 active+clean; 154 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.1 MiB/s wr, 220 op/s 2024-07-26T13:48:53.430 INFO:tasks.workunit.client.0.smithi100.stdout: 1 390645 4.30 MB/sec execute 307 sec latency 46.353 ms 2024-07-26T13:48:54.430 INFO:tasks.workunit.client.0.smithi100.stdout: 1 392778 4.31 MB/sec execute 308 sec latency 18.729 ms 2024-07-26T13:48:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:54 smithi138 ceph-mon[36954]: pgmap v637: 97 pgs: 97 active+clean; 160 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.8 MiB/s wr, 251 op/s 2024-07-26T13:48:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:54 smithi100 ceph-mon[32599]: pgmap v637: 97 pgs: 97 active+clean; 160 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.8 MiB/s wr, 251 op/s 2024-07-26T13:48:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:54 smithi167 ceph-mon[37140]: pgmap v637: 97 pgs: 97 active+clean; 160 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.8 MiB/s wr, 251 op/s 2024-07-26T13:48:55.430 INFO:tasks.workunit.client.0.smithi100.stdout: 1 394567 4.33 MB/sec execute 309 sec latency 22.779 ms 2024-07-26T13:48:56.430 INFO:tasks.workunit.client.0.smithi100.stdout: 1 396974 4.36 MB/sec execute 310 sec latency 15.302 ms 2024-07-26T13:48:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:56 smithi138 ceph-mon[36954]: pgmap v638: 97 pgs: 97 active+clean; 161 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.8 MiB/s wr, 242 op/s 2024-07-26T13:48:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:56 smithi100 ceph-mon[32599]: pgmap v638: 97 pgs: 97 active+clean; 161 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.8 MiB/s wr, 242 op/s 2024-07-26T13:48:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:56 smithi167 ceph-mon[37140]: pgmap v638: 97 pgs: 97 active+clean; 161 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.8 MiB/s wr, 242 op/s 2024-07-26T13:48:57.430 INFO:tasks.workunit.client.0.smithi100.stdout: 1 398473 4.37 MB/sec execute 311 sec latency 14.995 ms 2024-07-26T13:48:58.430 INFO:tasks.workunit.client.0.smithi100.stdout: 1 401013 4.41 MB/sec execute 312 sec latency 14.424 ms 2024-07-26T13:48:58.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:58.492+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:48:58.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:58.492+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:48:58.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:58.492+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:48:58.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:58.492+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:48:58.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:48:58.492+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:48:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:48:58 smithi100 ceph-mon[32599]: pgmap v639: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.2 MiB/s wr, 289 op/s 2024-07-26T13:48:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:48:58 smithi138 ceph-mon[36954]: pgmap v639: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.2 MiB/s wr, 289 op/s 2024-07-26T13:48:59.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:48:58 smithi167 ceph-mon[37140]: pgmap v639: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.2 MiB/s wr, 289 op/s 2024-07-26T13:48:59.431 INFO:tasks.workunit.client.0.smithi100.stdout: 1 402327 4.41 MB/sec execute 313 sec latency 21.677 ms 2024-07-26T13:49:00.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:00.246+0000 7f640affd640 1 --2- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 0x5581fb0b9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:49:00.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:00.246+0000 7f640affd640 1 -- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 msgr2=0x5581fb0b9000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:49:00.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:00.246+0000 7f640affd640 1 -- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 msgr2=0x5581fb0b9000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:49:00.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:00.246+0000 7f640affd640 1 --2- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 0x5581fb0b9000 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:49:00.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:00.246+0000 7f640affd640 1 --2- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 0x5581fb0b9000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:49:00.431 INFO:tasks.workunit.client.0.smithi100.stdout: 1 404696 4.44 MB/sec execute 314 sec latency 13.320 ms 2024-07-26T13:49:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:00 smithi138 ceph-mon[36954]: pgmap v640: 97 pgs: 97 active+clean; 168 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.7 MiB/s wr, 253 op/s 2024-07-26T13:49:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:00 smithi100 ceph-mon[32599]: pgmap v640: 97 pgs: 97 active+clean; 168 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.7 MiB/s wr, 253 op/s 2024-07-26T13:49:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:00 smithi167 ceph-mon[37140]: pgmap v640: 97 pgs: 97 active+clean; 168 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.7 MiB/s wr, 253 op/s 2024-07-26T13:49:01.431 INFO:tasks.workunit.client.0.smithi100.stdout: 1 404795 4.43 MB/sec execute 315 sec latency 879.904 ms 2024-07-26T13:49:02.431 INFO:tasks.workunit.client.0.smithi100.stdout: 1 404795 4.42 MB/sec execute 316 sec latency 1879.986 ms 2024-07-26T13:49:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:02 smithi138 ceph-mon[36954]: pgmap v641: 97 pgs: 97 active+clean; 171 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.5 MiB/s wr, 232 op/s 2024-07-26T13:49:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:02 smithi100 ceph-mon[32599]: pgmap v641: 97 pgs: 97 active+clean; 171 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.5 MiB/s wr, 232 op/s 2024-07-26T13:49:03.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:02 smithi167 ceph-mon[37140]: pgmap v641: 97 pgs: 97 active+clean; 171 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.5 MiB/s wr, 232 op/s 2024-07-26T13:49:03.431 INFO:tasks.workunit.client.0.smithi100.stdout: 1 404795 4.40 MB/sec execute 317 sec latency 2880.036 ms 2024-07-26T13:49:04.431 INFO:tasks.workunit.client.0.smithi100.stdout: 1 404988 4.39 MB/sec execute 318 sec latency 3685.226 ms 2024-07-26T13:49:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:04 smithi138 ceph-mon[36954]: pgmap v642: 97 pgs: 97 active+clean; 174 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.6 MiB/s wr, 260 op/s 2024-07-26T13:49:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:04 smithi100 ceph-mon[32599]: pgmap v642: 97 pgs: 97 active+clean; 174 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.6 MiB/s wr, 260 op/s 2024-07-26T13:49:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:04 smithi167 ceph-mon[37140]: pgmap v642: 97 pgs: 97 active+clean; 174 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.6 MiB/s wr, 260 op/s 2024-07-26T13:49:05.431 INFO:tasks.workunit.client.0.smithi100.stdout: 1 406436 4.39 MB/sec execute 319 sec latency 30.881 ms 2024-07-26T13:49:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:05 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:49:05.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:05 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:49:06.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:05 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:49:06.431 INFO:tasks.workunit.client.0.smithi100.stdout: 1 408387 4.42 MB/sec execute 320 sec latency 13.049 ms 2024-07-26T13:49:07.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:49:07.286+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:49:07.287 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:49:07.286+0000 7f88ad5a9640 1 -- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 msgr2=0x55c70699f7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-07-26T13:49:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:49:07.286+0000 7f88ad5a9640 1 -- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 msgr2=0x55c70699f7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:49:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:49:07.286+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:49:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:49:07.286+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:49:07.431 INFO:tasks.workunit.client.0.smithi100.stdout: 1 410149 4.43 MB/sec execute 321 sec latency 15.846 ms 2024-07-26T13:49:07.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:49:07.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:49:07.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:07 smithi138 ceph-mon[36954]: pgmap v643: 97 pgs: 97 active+clean; 174 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.8 MiB/s wr, 203 op/s 2024-07-26T13:49:07.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:49:07.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:49:07.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:49:07.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:49:07.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:49:07.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:49:07.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:49:07.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:49:07.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:49:07.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:07 smithi100 ceph-mon[32599]: pgmap v643: 97 pgs: 97 active+clean; 174 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.8 MiB/s wr, 203 op/s 2024-07-26T13:49:07.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:49:07.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:49:07.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:49:07.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:49:07.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:49:07.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:49:07.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:49:08.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:49:08.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:49:08.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:07 smithi167 ceph-mon[37140]: pgmap v643: 97 pgs: 97 active+clean; 174 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.8 MiB/s wr, 203 op/s 2024-07-26T13:49:08.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:49:08.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:49:08.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:49:08.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:49:08.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:49:08.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:49:08.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:49:08.431 INFO:tasks.workunit.client.0.smithi100.stdout: 1 411869 4.45 MB/sec execute 322 sec latency 25.169 ms 2024-07-26T13:49:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:08 smithi138 ceph-mon[36954]: osdmap e91: 12 total, 12 up, 12 in 2024-07-26T13:49:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:08 smithi138 ceph-mon[36954]: pgmap v645: 97 pgs: 97 active+clean; 179 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-07-26T13:49:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:08 smithi100 ceph-mon[32599]: osdmap e91: 12 total, 12 up, 12 in 2024-07-26T13:49:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:08 smithi100 ceph-mon[32599]: pgmap v645: 97 pgs: 97 active+clean; 179 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-07-26T13:49:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:08 smithi167 ceph-mon[37140]: osdmap e91: 12 total, 12 up, 12 in 2024-07-26T13:49:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:08 smithi167 ceph-mon[37140]: pgmap v645: 97 pgs: 97 active+clean; 179 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-07-26T13:49:09.431 INFO:tasks.workunit.client.0.smithi100.stdout: 1 413883 4.46 MB/sec execute 323 sec latency 18.873 ms 2024-07-26T13:49:10.431 INFO:tasks.workunit.client.0.smithi100.stdout: 1 415641 4.49 MB/sec execute 324 sec latency 16.695 ms 2024-07-26T13:49:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:10 smithi138 ceph-mon[36954]: pgmap v646: 97 pgs: 97 active+clean; 182 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 210 op/s 2024-07-26T13:49:10.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:10 smithi100 ceph-mon[32599]: pgmap v646: 97 pgs: 97 active+clean; 182 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 210 op/s 2024-07-26T13:49:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:10 smithi167 ceph-mon[37140]: pgmap v646: 97 pgs: 97 active+clean; 182 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 210 op/s 2024-07-26T13:49:11.431 INFO:tasks.workunit.client.0.smithi100.stdout: 1 417759 4.51 MB/sec execute 325 sec latency 12.795 ms 2024-07-26T13:49:12.431 INFO:tasks.workunit.client.0.smithi100.stdout: 1 419450 4.52 MB/sec execute 326 sec latency 16.929 ms 2024-07-26T13:49:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:12 smithi138 ceph-mon[36954]: pgmap v647: 97 pgs: 97 active+clean; 183 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 201 op/s 2024-07-26T13:49:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:12 smithi100 ceph-mon[32599]: pgmap v647: 97 pgs: 97 active+clean; 183 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 201 op/s 2024-07-26T13:49:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:12 smithi167 ceph-mon[37140]: pgmap v647: 97 pgs: 97 active+clean; 183 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 201 op/s 2024-07-26T13:49:13.431 INFO:tasks.workunit.client.0.smithi100.stdout: 1 421184 4.53 MB/sec execute 327 sec latency 53.211 ms 2024-07-26T13:49:13.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:13.507+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:49:13.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:13.507+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:49:13.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:13.507+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:49:13.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:13.507+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:49:13.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:13.507+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:49:14.431 INFO:tasks.workunit.client.0.smithi100.stdout: 1 423068 4.55 MB/sec execute 328 sec latency 23.055 ms 2024-07-26T13:49:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:14 smithi100 ceph-mon[32599]: osdmap e92: 12 total, 12 up, 12 in 2024-07-26T13:49:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:14 smithi100 ceph-mon[32599]: pgmap v649: 97 pgs: 97 active+clean; 190 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.9 MiB/s wr, 273 op/s 2024-07-26T13:49:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:14 smithi167 ceph-mon[37140]: osdmap e92: 12 total, 12 up, 12 in 2024-07-26T13:49:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:14 smithi167 ceph-mon[37140]: pgmap v649: 97 pgs: 97 active+clean; 190 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.9 MiB/s wr, 273 op/s 2024-07-26T13:49:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:14 smithi138 ceph-mon[36954]: osdmap e92: 12 total, 12 up, 12 in 2024-07-26T13:49:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:14 smithi138 ceph-mon[36954]: pgmap v649: 97 pgs: 97 active+clean; 190 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.9 MiB/s wr, 273 op/s 2024-07-26T13:49:15.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:15.261+0000 7f640affd640 1 --2- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 0x5581fb0b9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:49:15.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:15.261+0000 7f640affd640 1 -- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 msgr2=0x5581fb0b9000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:49:15.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:15.261+0000 7f640affd640 1 -- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 msgr2=0x5581fb0b9000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:49:15.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:15.261+0000 7f640affd640 1 --2- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 0x5581fb0b9000 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:49:15.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:15.261+0000 7f640affd640 1 --2- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 0x5581fb0b9000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:49:15.431 INFO:tasks.workunit.client.0.smithi100.stdout: 1 425675 4.59 MB/sec execute 329 sec latency 13.794 ms 2024-07-26T13:49:16.431 INFO:tasks.workunit.client.0.smithi100.stdout: 1 427083 4.59 MB/sec execute 330 sec latency 21.443 ms 2024-07-26T13:49:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:16 smithi138 ceph-mon[36954]: pgmap v650: 97 pgs: 97 active+clean; 194 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.4 MiB/s wr, 269 op/s 2024-07-26T13:49:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:16 smithi100 ceph-mon[32599]: pgmap v650: 97 pgs: 97 active+clean; 194 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.4 MiB/s wr, 269 op/s 2024-07-26T13:49:17.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:16 smithi167 ceph-mon[37140]: pgmap v650: 97 pgs: 97 active+clean; 194 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.4 MiB/s wr, 269 op/s 2024-07-26T13:49:17.431 INFO:tasks.workunit.client.0.smithi100.stdout: 1 429439 4.62 MB/sec execute 331 sec latency 23.240 ms 2024-07-26T13:49:18.431 INFO:tasks.workunit.client.0.smithi100.stdout: 1 430781 4.62 MB/sec execute 332 sec latency 41.418 ms 2024-07-26T13:49:18.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:18 smithi138 ceph-mon[36954]: pgmap v651: 97 pgs: 97 active+clean; 195 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 246 op/s 2024-07-26T13:49:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:18 smithi100 ceph-mon[32599]: pgmap v651: 97 pgs: 97 active+clean; 195 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 246 op/s 2024-07-26T13:49:19.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:18 smithi167 ceph-mon[37140]: pgmap v651: 97 pgs: 97 active+clean; 195 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 246 op/s 2024-07-26T13:49:19.431 INFO:tasks.workunit.client.0.smithi100.stdout: 1 433076 4.65 MB/sec execute 333 sec latency 18.437 ms 2024-07-26T13:49:20.431 INFO:tasks.workunit.client.0.smithi100.stdout: 1 434360 4.65 MB/sec execute 334 sec latency 21.598 ms 2024-07-26T13:49:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:20 smithi138 ceph-mon[36954]: pgmap v652: 97 pgs: 97 active+clean; 207 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.2 MiB/s wr, 306 op/s 2024-07-26T13:49:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:20 smithi100 ceph-mon[32599]: pgmap v652: 97 pgs: 97 active+clean; 207 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.2 MiB/s wr, 306 op/s 2024-07-26T13:49:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:20 smithi167 ceph-mon[37140]: pgmap v652: 97 pgs: 97 active+clean; 207 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.2 MiB/s wr, 306 op/s 2024-07-26T13:49:21.431 INFO:tasks.workunit.client.0.smithi100.stdout: 1 436527 4.68 MB/sec execute 335 sec latency 15.122 ms 2024-07-26T13:49:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:49:22.301+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:49:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:49:22.301+0000 7f88ad5a9640 1 -- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 msgr2=0x55c70699f7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-07-26T13:49:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:49:22.301+0000 7f88ad5a9640 1 -- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 msgr2=0x55c70699f7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:49:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:49:22.301+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:49:22.303 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:49:22.301+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:49:22.431 INFO:tasks.workunit.client.0.smithi100.stdout: 1 437843 4.68 MB/sec execute 336 sec latency 23.880 ms 2024-07-26T13:49:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:22 smithi138 ceph-mon[36954]: pgmap v653: 97 pgs: 97 active+clean; 209 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.5 MiB/s wr, 305 op/s 2024-07-26T13:49:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:22 smithi100 ceph-mon[32599]: pgmap v653: 97 pgs: 97 active+clean; 209 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.5 MiB/s wr, 305 op/s 2024-07-26T13:49:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:22 smithi167 ceph-mon[37140]: pgmap v653: 97 pgs: 97 active+clean; 209 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.5 MiB/s wr, 305 op/s 2024-07-26T13:49:23.431 INFO:tasks.workunit.client.0.smithi100.stdout: 1 440080 4.71 MB/sec execute 337 sec latency 51.188 ms 2024-07-26T13:49:24.432 INFO:tasks.workunit.client.0.smithi100.stdout: 1 441495 4.71 MB/sec execute 338 sec latency 17.796 ms 2024-07-26T13:49:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:24 smithi138 ceph-mon[36954]: pgmap v654: 97 pgs: 97 active+clean; 216 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.5 MiB/s wr, 296 op/s 2024-07-26T13:49:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:24 smithi100 ceph-mon[32599]: pgmap v654: 97 pgs: 97 active+clean; 216 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.5 MiB/s wr, 296 op/s 2024-07-26T13:49:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:24 smithi167 ceph-mon[37140]: pgmap v654: 97 pgs: 97 active+clean; 216 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.5 MiB/s wr, 296 op/s 2024-07-26T13:49:25.431 INFO:tasks.workunit.client.0.smithi100.stdout: 1 443764 4.74 MB/sec execute 339 sec latency 14.168 ms 2024-07-26T13:49:26.432 INFO:tasks.workunit.client.0.smithi100.stdout: 1 445361 4.74 MB/sec execute 340 sec latency 33.511 ms 2024-07-26T13:49:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:26 smithi138 ceph-mon[36954]: pgmap v655: 97 pgs: 97 active+clean; 215 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.2 MiB/s wr, 270 op/s 2024-07-26T13:49:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:26 smithi100 ceph-mon[32599]: pgmap v655: 97 pgs: 97 active+clean; 215 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.2 MiB/s wr, 270 op/s 2024-07-26T13:49:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:26 smithi167 ceph-mon[37140]: pgmap v655: 97 pgs: 97 active+clean; 215 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.2 MiB/s wr, 270 op/s 2024-07-26T13:49:27.432 INFO:tasks.workunit.client.0.smithi100.stdout: 1 447317 4.77 MB/sec execute 341 sec latency 60.087 ms 2024-07-26T13:49:28.432 INFO:tasks.workunit.client.0.smithi100.stdout: 1 448595 4.77 MB/sec execute 342 sec latency 25.554 ms 2024-07-26T13:49:28.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:28.522+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:49:28.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:28.522+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:49:28.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:28.522+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:49:28.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:28.522+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:49:28.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:28.522+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:49:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:28 smithi138 ceph-mon[36954]: pgmap v656: 97 pgs: 97 active+clean; 220 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.4 MiB/s wr, 294 op/s 2024-07-26T13:49:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:28 smithi100 ceph-mon[32599]: pgmap v656: 97 pgs: 97 active+clean; 220 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.4 MiB/s wr, 294 op/s 2024-07-26T13:49:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:28 smithi167 ceph-mon[37140]: pgmap v656: 97 pgs: 97 active+clean; 220 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.4 MiB/s wr, 294 op/s 2024-07-26T13:49:29.432 INFO:tasks.workunit.client.0.smithi100.stdout: 1 450795 4.79 MB/sec execute 343 sec latency 35.694 ms 2024-07-26T13:49:30.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:30.262+0000 7f640affd640 1 --2- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 0x5581fb0b9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:49:30.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:30.262+0000 7f640affd640 1 -- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 msgr2=0x5581fb0b9000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:49:30.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:30.262+0000 7f640affd640 1 -- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 msgr2=0x5581fb0b9000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:49:30.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:30.262+0000 7f640affd640 1 --2- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 0x5581fb0b9000 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:49:30.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:30.262+0000 7f640affd640 1 --2- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 0x5581fb0b9000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:49:30.432 INFO:tasks.workunit.client.0.smithi100.stdout: 1 452074 4.80 MB/sec execute 344 sec latency 14.293 ms 2024-07-26T13:49:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:30 smithi138 ceph-mon[36954]: pgmap v657: 97 pgs: 97 active+clean; 226 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 330 op/s 2024-07-26T13:49:30.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:30 smithi100 ceph-mon[32599]: pgmap v657: 97 pgs: 97 active+clean; 226 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 330 op/s 2024-07-26T13:49:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:30 smithi167 ceph-mon[37140]: pgmap v657: 97 pgs: 97 active+clean; 226 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 330 op/s 2024-07-26T13:49:31.432 INFO:tasks.workunit.client.0.smithi100.stdout: 1 454348 4.82 MB/sec execute 345 sec latency 15.598 ms 2024-07-26T13:49:32.432 INFO:tasks.workunit.client.0.smithi100.stdout: 1 455697 4.82 MB/sec execute 346 sec latency 20.790 ms 2024-07-26T13:49:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:32 smithi138 ceph-mon[36954]: pgmap v658: 97 pgs: 97 active+clean; 226 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.9 MiB/s wr, 265 op/s 2024-07-26T13:49:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:32 smithi100 ceph-mon[32599]: pgmap v658: 97 pgs: 97 active+clean; 226 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.9 MiB/s wr, 265 op/s 2024-07-26T13:49:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:32 smithi167 ceph-mon[37140]: pgmap v658: 97 pgs: 97 active+clean; 226 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.9 MiB/s wr, 265 op/s 2024-07-26T13:49:33.432 INFO:tasks.workunit.client.0.smithi100.stdout: 1 457906 4.85 MB/sec execute 347 sec latency 15.487 ms 2024-07-26T13:49:34.432 INFO:tasks.workunit.client.0.smithi100.stdout: 1 458797 4.88 MB/sec execute 348 sec latency 46.626 ms 2024-07-26T13:49:35.432 INFO:tasks.workunit.client.0.smithi100.stdout: 1 461123 4.91 MB/sec execute 349 sec latency 15.573 ms 2024-07-26T13:49:35.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:35 smithi138 ceph-mon[36954]: fsmap cephfs:3 {0=f=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:49:35.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:35 smithi138 ceph-mon[36954]: pgmap v659: 97 pgs: 97 active+clean; 227 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 279 op/s 2024-07-26T13:49:35.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:35 smithi100 ceph-mon[32599]: fsmap cephfs:3 {0=f=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:49:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:35 smithi100 ceph-mon[32599]: pgmap v659: 97 pgs: 97 active+clean; 227 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 279 op/s 2024-07-26T13:49:35.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:35 smithi167 ceph-mon[37140]: fsmap cephfs:3 {0=f=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:49:35.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:35 smithi167 ceph-mon[37140]: pgmap v659: 97 pgs: 97 active+clean; 227 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 279 op/s 2024-07-26T13:49:36.432 INFO:tasks.workunit.client.0.smithi100.stdout: 1 461877 4.92 MB/sec execute 350 sec latency 827.843 ms 2024-07-26T13:49:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:36 smithi138 ceph-mon[36954]: pgmap v660: 97 pgs: 97 active+clean; 234 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 267 op/s 2024-07-26T13:49:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:36 smithi100 ceph-mon[32599]: pgmap v660: 97 pgs: 97 active+clean; 234 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 267 op/s 2024-07-26T13:49:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:36 smithi167 ceph-mon[37140]: pgmap v660: 97 pgs: 97 active+clean; 234 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.5 MiB/s wr, 267 op/s 2024-07-26T13:49:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:49:37.317+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:49:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:49:37.318+0000 7f88ad5a9640 1 -- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 msgr2=0x55c70699f7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-07-26T13:49:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:49:37.318+0000 7f88ad5a9640 1 -- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 msgr2=0x55c70699f7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:49:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:49:37.318+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:49:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:49:37.318+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:49:37.432 INFO:tasks.workunit.client.0.smithi100.stdout: 1 461902 4.90 MB/sec execute 351 sec latency 1176.369 ms 2024-07-26T13:49:38.432 INFO:tasks.workunit.client.0.smithi100.stdout: 1 461902 4.89 MB/sec execute 352 sec latency 1634.494 ms 2024-07-26T13:49:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:38 smithi138 ceph-mon[36954]: pgmap v661: 97 pgs: 97 active+clean; 236 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 272 op/s 2024-07-26T13:49:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:38 smithi100 ceph-mon[32599]: pgmap v661: 97 pgs: 97 active+clean; 236 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 272 op/s 2024-07-26T13:49:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:38 smithi167 ceph-mon[37140]: pgmap v661: 97 pgs: 97 active+clean; 236 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 272 op/s 2024-07-26T13:49:39.432 INFO:tasks.workunit.client.0.smithi100.stdout: 1 461902 4.88 MB/sec execute 353 sec latency 2634.628 ms 2024-07-26T13:49:40.433 INFO:tasks.workunit.client.0.smithi100.stdout: 1 461970 4.86 MB/sec execute 354 sec latency 3562.071 ms 2024-07-26T13:49:40.676 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: ceph-mds: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-5320-ge8f88bb1/rpm/el9/BUILD/ceph-19.0.0-5320-ge8f88bb1/redhat-linux-build/boost/include/boost/smart_ptr/intrusive_ptr.hpp:201: T* boost::intrusive_ptr::operator->() const [with T = const Message]: Assertion `px != 0' failed. 2024-07-26T13:49:40.676 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: *** Caught signal (Aborted) ** 2024-07-26T13:49:40.676 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: in thread 7f1ac8511640 thread_name:ms_dispatch 2024-07-26T13:49:40.676 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: ceph version 19.0.0-5320-ge8f88bb1 (e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528) squid (dev) 2024-07-26T13:49:40.676 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f1acd8e86f0] 2024-07-26T13:49:40.676 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 2: /lib64/libc.so.6(+0x8b94c) [0x7f1acd93594c] 2024-07-26T13:49:40.676 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 3: raise() 2024-07-26T13:49:40.676 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 4: abort() 2024-07-26T13:49:40.676 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 5: /lib64/libc.so.6(+0x2871b) [0x7f1acd8d271b] 2024-07-26T13:49:40.676 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 6: /lib64/libc.so.6(+0x37386) [0x7f1acd8e1386] 2024-07-26T13:49:40.677 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 7: (MDSRank::get_session(boost::intrusive_ptr const&)+0x89) [0x55bb91f986b9] 2024-07-26T13:49:40.677 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 8: (MDCache::path_traverse(boost::intrusive_ptr const&, ContextFactory&, filepath const&, int, std::vector >*, CInode**, CDir**)+0x58e) [0x55bb9215680c] 2024-07-26T13:49:40.677 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 9: (Server::handle_peer_rename_prep(boost::intrusive_ptr const&)+0x4fe) [0x55bb920a4cde] 2024-07-26T13:49:40.677 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 10: (Server::dispatch_peer_request(boost::intrusive_ptr const&)+0x933) [0x55bb92066763] 2024-07-26T13:49:40.677 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 11: (Server::handle_peer_request(boost::intrusive_ptr const&)+0x10c3) [0x55bb92067a1b] 2024-07-26T13:49:40.677 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x9ba) [0x55bb9204b752] 2024-07-26T13:49:40.677 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 13: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x81a) [0x55bb91f99a56] 2024-07-26T13:49:40.677 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 14: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55bb91f9d637] 2024-07-26T13:49:40.677 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 15: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55bb91f9df5e] 2024-07-26T13:49:40.677 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 16: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55bb91f8103a] 2024-07-26T13:49:40.677 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 17: /usr/lib64/ceph/libceph-common.so.2(+0x39b407) [0x7f1ace1b5407] 2024-07-26T13:49:40.677 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 18: (DispatchQueue::entry()+0x60f) [0x7f1ace1b5cf9] 2024-07-26T13:49:40.677 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 19: /usr/lib64/ceph/libceph-common.so.2(+0x4785a1) [0x7f1ace2925a1] 2024-07-26T13:49:40.678 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 20: (Thread::entry_wrapper()+0x43) [0x7f1ace03fcdd] 2024-07-26T13:49:40.678 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 21: (Thread::_entry_func(void*)+0xd) [0x7f1ace03fcf9] 2024-07-26T13:49:40.678 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 22: /lib64/libc.so.6(+0x89c02) [0x7f1acd933c02] 2024-07-26T13:49:40.678 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 23: /lib64/libc.so.6(+0x10ec40) [0x7f1acd9b8c40] 2024-07-26T13:49:40.678 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 2024-07-26T13:49:40.377+0000 7f1ac8511640 -1 *** Caught signal (Aborted) ** 2024-07-26T13:49:40.678 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: in thread 7f1ac8511640 thread_name:ms_dispatch 2024-07-26T13:49:40.678 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 2024-07-26T13:49:40.678 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: ceph version 19.0.0-5320-ge8f88bb1 (e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528) squid (dev) 2024-07-26T13:49:40.678 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f1acd8e86f0] 2024-07-26T13:49:40.678 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 2: /lib64/libc.so.6(+0x8b94c) [0x7f1acd93594c] 2024-07-26T13:49:40.678 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 3: raise() 2024-07-26T13:49:40.678 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 4: abort() 2024-07-26T13:49:40.678 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 5: /lib64/libc.so.6(+0x2871b) [0x7f1acd8d271b] 2024-07-26T13:49:40.678 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 6: /lib64/libc.so.6(+0x37386) [0x7f1acd8e1386] 2024-07-26T13:49:40.679 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 7: (MDSRank::get_session(boost::intrusive_ptr const&)+0x89) [0x55bb91f986b9] 2024-07-26T13:49:40.679 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 8: (MDCache::path_traverse(boost::intrusive_ptr const&, ContextFactory&, filepath const&, int, std::vector >*, CInode**, CDir**)+0x58e) [0x55bb9215680c] 2024-07-26T13:49:40.679 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 9: (Server::handle_peer_rename_prep(boost::intrusive_ptr const&)+0x4fe) [0x55bb920a4cde] 2024-07-26T13:49:40.679 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 10: (Server::dispatch_peer_request(boost::intrusive_ptr const&)+0x933) [0x55bb92066763] 2024-07-26T13:49:40.679 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 11: (Server::handle_peer_request(boost::intrusive_ptr const&)+0x10c3) [0x55bb92067a1b] 2024-07-26T13:49:40.679 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x9ba) [0x55bb9204b752] 2024-07-26T13:49:40.679 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 13: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x81a) [0x55bb91f99a56] 2024-07-26T13:49:40.679 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 14: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55bb91f9d637] 2024-07-26T13:49:40.679 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 15: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55bb91f9df5e] 2024-07-26T13:49:40.679 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 16: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55bb91f8103a] 2024-07-26T13:49:40.679 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 17: /usr/lib64/ceph/libceph-common.so.2(+0x39b407) [0x7f1ace1b5407] 2024-07-26T13:49:40.679 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 18: (DispatchQueue::entry()+0x60f) [0x7f1ace1b5cf9] 2024-07-26T13:49:40.679 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 19: /usr/lib64/ceph/libceph-common.so.2(+0x4785a1) [0x7f1ace2925a1] 2024-07-26T13:49:40.680 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 20: (Thread::entry_wrapper()+0x43) [0x7f1ace03fcdd] 2024-07-26T13:49:40.680 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 21: (Thread::_entry_func(void*)+0xd) [0x7f1ace03fcf9] 2024-07-26T13:49:40.680 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 22: /lib64/libc.so.6(+0x89c02) [0x7f1acd933c02] 2024-07-26T13:49:40.680 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 23: /lib64/libc.so.6(+0x10ec40) [0x7f1acd9b8c40] 2024-07-26T13:49:40.680 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-07-26T13:49:40.681 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 2024-07-26T13:49:40.681 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 0> 2024-07-26T13:49:40.377+0000 7f1ac8511640 -1 *** Caught signal (Aborted) ** 2024-07-26T13:49:40.681 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: in thread 7f1ac8511640 thread_name:ms_dispatch 2024-07-26T13:49:40.681 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 2024-07-26T13:49:40.681 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: ceph version 19.0.0-5320-ge8f88bb1 (e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528) squid (dev) 2024-07-26T13:49:40.681 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f1acd8e86f0] 2024-07-26T13:49:40.681 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 2: /lib64/libc.so.6(+0x8b94c) [0x7f1acd93594c] 2024-07-26T13:49:40.681 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 3: raise() 2024-07-26T13:49:40.681 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 4: abort() 2024-07-26T13:49:40.681 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 5: /lib64/libc.so.6(+0x2871b) [0x7f1acd8d271b] 2024-07-26T13:49:40.681 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 6: /lib64/libc.so.6(+0x37386) [0x7f1acd8e1386] 2024-07-26T13:49:40.681 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 7: (MDSRank::get_session(boost::intrusive_ptr const&)+0x89) [0x55bb91f986b9] 2024-07-26T13:49:40.681 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 8: (MDCache::path_traverse(boost::intrusive_ptr const&, ContextFactory&, filepath const&, int, std::vector >*, CInode**, CDir**)+0x58e) [0x55bb9215680c] 2024-07-26T13:49:40.682 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 9: (Server::handle_peer_rename_prep(boost::intrusive_ptr const&)+0x4fe) [0x55bb920a4cde] 2024-07-26T13:49:40.682 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 10: (Server::dispatch_peer_request(boost::intrusive_ptr const&)+0x933) [0x55bb92066763] 2024-07-26T13:49:40.682 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 11: (Server::handle_peer_request(boost::intrusive_ptr const&)+0x10c3) [0x55bb92067a1b] 2024-07-26T13:49:40.682 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x9ba) [0x55bb9204b752] 2024-07-26T13:49:40.682 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 13: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x81a) [0x55bb91f99a56] 2024-07-26T13:49:40.682 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 14: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55bb91f9d637] 2024-07-26T13:49:40.682 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 15: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55bb91f9df5e] 2024-07-26T13:49:40.682 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 16: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55bb91f8103a] 2024-07-26T13:49:40.682 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 17: /usr/lib64/ceph/libceph-common.so.2(+0x39b407) [0x7f1ace1b5407] 2024-07-26T13:49:40.682 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 18: (DispatchQueue::entry()+0x60f) [0x7f1ace1b5cf9] 2024-07-26T13:49:40.682 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 19: /usr/lib64/ceph/libceph-common.so.2(+0x4785a1) [0x7f1ace2925a1] 2024-07-26T13:49:40.682 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 20: (Thread::entry_wrapper()+0x43) [0x7f1ace03fcdd] 2024-07-26T13:49:40.682 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 21: (Thread::_entry_func(void*)+0xd) [0x7f1ace03fcf9] 2024-07-26T13:49:40.683 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 22: /lib64/libc.so.6(+0x89c02) [0x7f1acd933c02] 2024-07-26T13:49:40.683 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 23: /lib64/libc.so.6(+0x10ec40) [0x7f1acd9b8c40] 2024-07-26T13:49:40.683 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-07-26T13:49:40.683 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 2024-07-26T13:49:40.683 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 0> 2024-07-26T13:49:40.377+0000 7f1ac8511640 -1 *** Caught signal (Aborted) ** 2024-07-26T13:49:40.683 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: in thread 7f1ac8511640 thread_name:ms_dispatch 2024-07-26T13:49:40.683 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 2024-07-26T13:49:40.683 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: ceph version 19.0.0-5320-ge8f88bb1 (e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528) squid (dev) 2024-07-26T13:49:40.683 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f1acd8e86f0] 2024-07-26T13:49:40.683 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 2: /lib64/libc.so.6(+0x8b94c) [0x7f1acd93594c] 2024-07-26T13:49:40.683 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 3: raise() 2024-07-26T13:49:40.683 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 4: abort() 2024-07-26T13:49:40.683 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 5: /lib64/libc.so.6(+0x2871b) [0x7f1acd8d271b] 2024-07-26T13:49:40.684 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 6: /lib64/libc.so.6(+0x37386) [0x7f1acd8e1386] 2024-07-26T13:49:40.684 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 7: (MDSRank::get_session(boost::intrusive_ptr const&)+0x89) [0x55bb91f986b9] 2024-07-26T13:49:40.684 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 8: (MDCache::path_traverse(boost::intrusive_ptr const&, ContextFactory&, filepath const&, int, std::vector >*, CInode**, CDir**)+0x58e) [0x55bb9215680c] 2024-07-26T13:49:40.684 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 9: (Server::handle_peer_rename_prep(boost::intrusive_ptr const&)+0x4fe) [0x55bb920a4cde] 2024-07-26T13:49:40.684 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 10: (Server::dispatch_peer_request(boost::intrusive_ptr const&)+0x933) [0x55bb92066763] 2024-07-26T13:49:40.684 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 11: (Server::handle_peer_request(boost::intrusive_ptr const&)+0x10c3) [0x55bb92067a1b] 2024-07-26T13:49:40.684 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x9ba) [0x55bb9204b752] 2024-07-26T13:49:40.684 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 13: (MDSRank::handle_message(boost::intrusive_ptr const&)+0x81a) [0x55bb91f99a56] 2024-07-26T13:49:40.684 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 14: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x28f) [0x55bb91f9d637] 2024-07-26T13:49:40.684 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 15: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x94) [0x55bb91f9df5e] 2024-07-26T13:49:40.684 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 16: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x3f0) [0x55bb91f8103a] 2024-07-26T13:49:40.684 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 17: /usr/lib64/ceph/libceph-common.so.2(+0x39b407) [0x7f1ace1b5407] 2024-07-26T13:49:40.685 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 18: (DispatchQueue::entry()+0x60f) [0x7f1ace1b5cf9] 2024-07-26T13:49:40.685 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 19: /usr/lib64/ceph/libceph-common.so.2(+0x4785a1) [0x7f1ace2925a1] 2024-07-26T13:49:40.685 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 20: (Thread::entry_wrapper()+0x43) [0x7f1ace03fcdd] 2024-07-26T13:49:40.685 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 21: (Thread::_entry_func(void*)+0xd) [0x7f1ace03fcf9] 2024-07-26T13:49:40.685 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 22: /lib64/libc.so.6(+0x89c02) [0x7f1acd933c02] 2024-07-26T13:49:40.685 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 23: /lib64/libc.so.6(+0x10ec40) [0x7f1acd9b8c40] 2024-07-26T13:49:40.685 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-07-26T13:49:40.685 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[68500]: 2024-07-26T13:49:40.685 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:40 smithi138 podman[72753]: 2024-07-26 13:49:40.438912636 +0000 UTC m=+0.021568893 container died e336832bdc01fd5a1dea2d988067e30ce239fbac9099bf3fbc87406a15235104 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-07-26T13:49:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-mon[36954]: fsmap cephfs:3 {0=f=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:49:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:40 smithi138 ceph-mon[36954]: pgmap v662: 97 pgs: 97 active+clean; 237 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 226 op/s 2024-07-26T13:49:40.969 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:40 smithi100 ceph-mon[32599]: fsmap cephfs:3 {0=f=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:49:40.969 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:40 smithi100 ceph-mon[32599]: pgmap v662: 97 pgs: 97 active+clean; 237 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 226 op/s 2024-07-26T13:49:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:40 smithi167 ceph-mon[37140]: fsmap cephfs:3 {0=f=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:49:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:40 smithi167 ceph-mon[37140]: pgmap v662: 97 pgs: 97 active+clean; 237 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 226 op/s 2024-07-26T13:49:41.415 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:41 smithi138 podman[72753]: 2024-07-26 13:49:41.173172591 +0000 UTC m=+0.755828839 container remove e336832bdc01fd5a1dea2d988067e30ce239fbac9099bf3fbc87406a15235104 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-07-26T13:49:41.415 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:41 smithi138 systemd[1]: ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.b.service: Main process exited, code=exited, status=134/n/a 2024-07-26T13:49:41.432 INFO:tasks.workunit.client.0.smithi100.stdout: 1 461970 4.85 MB/sec execute 355 sec latency 1058.734 ms 2024-07-26T13:49:41.690 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:41 smithi138 systemd[1]: ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.b.service: Failed with result 'exit-code'. 2024-07-26T13:49:41.690 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:41 smithi138 systemd[1]: ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.b.service: Consumed 1min 38.211s CPU time. 2024-07-26T13:49:42.433 INFO:tasks.workunit.client.0.smithi100.stdout: 1 461970 4.84 MB/sec execute 356 sec latency 2058.779 ms 2024-07-26T13:49:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:42 smithi138 ceph-mon[36954]: pgmap v663: 97 pgs: 97 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 165 op/s 2024-07-26T13:49:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:42 smithi100 ceph-mon[32599]: pgmap v663: 97 pgs: 97 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 165 op/s 2024-07-26T13:49:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:42 smithi167 ceph-mon[37140]: pgmap v663: 97 pgs: 97 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 165 op/s 2024-07-26T13:49:43.433 INFO:tasks.workunit.client.0.smithi100.stdout: 1 461970 4.82 MB/sec execute 357 sec latency 3058.867 ms 2024-07-26T13:49:43.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:43.538+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:49:43.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:43.538+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:49:43.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:43.539+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:49:43.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:43.539+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:49:43.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:43.539+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:49:44.433 INFO:tasks.workunit.client.0.smithi100.stdout: 1 461970 4.81 MB/sec execute 358 sec latency 4058.957 ms 2024-07-26T13:49:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:44 smithi138 ceph-mon[36954]: pgmap v664: 97 pgs: 97 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 146 op/s 2024-07-26T13:49:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:44 smithi100 ceph-mon[32599]: pgmap v664: 97 pgs: 97 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 146 op/s 2024-07-26T13:49:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:44 smithi167 ceph-mon[37140]: pgmap v664: 97 pgs: 97 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 146 op/s 2024-07-26T13:49:45.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:45.279+0000 7f640affd640 1 --2- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 0x5581fb0b9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:49:45.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:45.280+0000 7f640affd640 1 -- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 msgr2=0x5581fb0b9000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:49:45.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:45.280+0000 7f640affd640 1 -- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 msgr2=0x5581fb0b9000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:49:45.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:45.280+0000 7f640affd640 1 --2- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 0x5581fb0b9000 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:49:45.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:45.280+0000 7f640affd640 1 --2- 172.21.15.100:0/3807228955 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x5581facb41c0 0x5581fb0b9000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:49:45.433 INFO:tasks.workunit.client.0.smithi100.stdout: 1 461970 4.80 MB/sec execute 359 sec latency 5059.052 ms 2024-07-26T13:49:46.433 INFO:tasks.workunit.client.0.smithi100.stdout: 1 461970 4.78 MB/sec execute 360 sec latency 6059.165 ms 2024-07-26T13:49:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:46 smithi138 ceph-mon[36954]: pgmap v665: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 116 op/s 2024-07-26T13:49:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:46 smithi100 ceph-mon[32599]: pgmap v665: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 116 op/s 2024-07-26T13:49:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:46 smithi167 ceph-mon[37140]: pgmap v665: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 116 op/s 2024-07-26T13:49:47.433 INFO:tasks.workunit.client.0.smithi100.stdout: 1 461970 4.77 MB/sec execute 361 sec latency 7059.211 ms 2024-07-26T13:49:48.433 INFO:tasks.workunit.client.0.smithi100.stdout: 1 461970 4.76 MB/sec execute 362 sec latency 8059.293 ms 2024-07-26T13:49:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:48 smithi138 ceph-mon[36954]: pgmap v666: 97 pgs: 97 active+clean; 240 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 1.4 MiB/s wr, 77 op/s 2024-07-26T13:49:48.968 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:48 smithi100 ceph-mon[32599]: pgmap v666: 97 pgs: 97 active+clean; 240 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 1.4 MiB/s wr, 77 op/s 2024-07-26T13:49:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:48 smithi167 ceph-mon[37140]: pgmap v666: 97 pgs: 97 active+clean; 240 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 1.4 MiB/s wr, 77 op/s 2024-07-26T13:49:49.433 INFO:tasks.workunit.client.0.smithi100.stdout: 1 461970 4.74 MB/sec execute 363 sec latency 9059.388 ms 2024-07-26T13:49:50.433 INFO:tasks.workunit.client.0.smithi100.stdout: 1 461970 4.73 MB/sec execute 364 sec latency 10059.459 ms 2024-07-26T13:49:50.440 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-07-26T13:49:50.441 ERROR:tasks.quiescer.fs.[cephfs]:Couldn't parse response with error Expecting value: line 1 column 1 (char 0); rc: 124 (EMEDIUMTYPE); stdout: 2024-07-26T13:49:50.441 ERROR:tasks.quiescer.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528/qa/tasks/quiescer.py", line 294, in do_quiesce response = json.loads(stdout) File "/usr/lib/python3.10/json/__init__.py", line 346, in loads return _default_decoder.decode(s) File "/usr/lib/python3.10/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib/python3.10/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528/qa/tasks/quiescer.py", line 357, in _run self.do_quiesce(d) File "/home/teuthworker/src/git.ceph.com_ceph-c_e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528/qa/tasks/quiescer.py", line 298, in do_quiesce raise RuntimeError(f"Error parsing quiesce response: {e}") RuntimeError: Error parsing quiesce response: Expecting value: line 1 column 1 (char 0) 2024-07-26T13:49:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:50 smithi138 ceph-mon[36954]: pgmap v667: 97 pgs: 97 active+clean; 240 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 1.4 MiB/s wr, 64 op/s 2024-07-26T13:49:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:50 smithi100 ceph-mon[32599]: pgmap v667: 97 pgs: 97 active+clean; 240 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 1.4 MiB/s wr, 64 op/s 2024-07-26T13:49:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:50 smithi167 ceph-mon[37140]: pgmap v667: 97 pgs: 97 active+clean; 240 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 1.4 MiB/s wr, 64 op/s 2024-07-26T13:49:51.433 INFO:tasks.workunit.client.0.smithi100.stdout: 1 461970 4.72 MB/sec execute 365 sec latency 11059.574 ms 2024-07-26T13:49:51.741 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:51 smithi138 systemd[1]: ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.b.service: Scheduled restart job, restart counter is at 1. 2024-07-26T13:49:51.741 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:51 smithi138 systemd[1]: Stopped Ceph mds.b for bf83e2e6-4b52-11ef-bcb7-c7b262605968. 2024-07-26T13:49:51.741 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:51 smithi138 systemd[1]: ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968@mds.b.service: Consumed 1min 38.211s CPU time. 2024-07-26T13:49:51.741 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:51 smithi138 systemd[1]: Starting Ceph mds.b for bf83e2e6-4b52-11ef-bcb7-c7b262605968... 2024-07-26T13:49:51.741 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:51 smithi138 podman[72846]: 2024-07-26 13:49:51.582928378 +0000 UTC m=+0.017619227 image pull ae51f432d4fa82622bc253ca9e2da7666c617e096e99d43d0a0e4fdaca8865f8 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2 2024-07-26T13:49:52.021 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:51 smithi138 podman[72846]: 2024-07-26 13:49:51.741027177 +0000 UTC m=+0.175718016 container create 9e079b328c8c4fcfceb895814bf4da1daaa9d3724301795a6beb248f903709ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, ceph=True) 2024-07-26T13:49:52.021 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:51 smithi138 podman[72846]: 2024-07-26 13:49:51.986560313 +0000 UTC m=+0.421251152 container init 9e079b328c8c4fcfceb895814bf4da1daaa9d3724301795a6beb248f903709ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, ceph=True, org.label-schema.build-date=20240724, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-26T13:49:52.021 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:51 smithi138 podman[72846]: 2024-07-26 13:49:51.989317744 +0000 UTC m=+0.424008598 container start 9e079b328c8c4fcfceb895814bf4da1daaa9d3724301795a6beb248f903709ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:54b7054dea190fddefae31e8872b94a8bc32a70716ff596bdd9c7ee47e62e2d2, name=ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, RELEASE=wip-vshankar-testing-20240725.121331-debug-e8f88bb, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.license=GPLv2) 2024-07-26T13:49:52.334 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:49:52.333+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:49:52.335 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:49:52.334+0000 7f88ad5a9640 1 -- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 msgr2=0x55c70699f7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-07-26T13:49:52.335 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:49:52.334+0000 7f88ad5a9640 1 -- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 msgr2=0x55c70699f7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:49:52.335 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:49:52.334+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:49:52.335 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:49:52.334+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:49:52.434 INFO:tasks.workunit.client.0.smithi100.stdout: 1 461970 4.70 MB/sec execute 366 sec latency 12059.666 ms 2024-07-26T13:49:52.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:52 smithi138 ceph-mon[36954]: daemon mds.b restarted 2024-07-26T13:49:52.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:52 smithi138 ceph-mon[36954]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-26T13:49:52.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:52 smithi138 ceph-mon[36954]: osdmap e93: 12 total, 12 up, 12 in 2024-07-26T13:49:52.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:52 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.138:6838/133335730,v1:172.21.15.138:6839/133335730] up:boot 2024-07-26T13:49:52.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:52 smithi138 ceph-mon[36954]: Standby daemon mds.h assigned to filesystem cephfs as rank 1 2024-07-26T13:49:52.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:52 smithi138 ceph-mon[36954]: fsmap cephfs:2/3 {0=f=up:active,2=c=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-07-26T13:49:52.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-26T13:49:52.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:52 smithi138 ceph-mon[36954]: fsmap cephfs:3/3 {0=f=up:active,1=h=up:replay,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:49:52.441 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:52 smithi138 ceph-mds[72866]: -- 172.21.15.138:0/1658221012 <== mon.2 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56421af35860 con 0x56421f0d7600 2024-07-26T13:49:52.441 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:52 smithi138 ceph-bf83e2e6-4b52-11ef-bcb7-c7b262605968-mds-b[72862]: starting mds.b at 2024-07-26T13:49:52.441 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:52 smithi138 bash[72846]: 9e079b328c8c4fcfceb895814bf4da1daaa9d3724301795a6beb248f903709ea 2024-07-26T13:49:52.441 INFO:journalctl@ceph.mds.b.smithi138.stdout:Jul 26 13:49:52 smithi138 systemd[1]: Started Ceph mds.b for bf83e2e6-4b52-11ef-bcb7-c7b262605968. 2024-07-26T13:49:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:52 smithi100 ceph-mon[32599]: daemon mds.b restarted 2024-07-26T13:49:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:52 smithi100 ceph-mon[32599]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-26T13:49:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:52 smithi100 ceph-mon[32599]: osdmap e93: 12 total, 12 up, 12 in 2024-07-26T13:49:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:52 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.138:6838/133335730,v1:172.21.15.138:6839/133335730] up:boot 2024-07-26T13:49:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:52 smithi100 ceph-mon[32599]: Standby daemon mds.h assigned to filesystem cephfs as rank 1 2024-07-26T13:49:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:52 smithi100 ceph-mon[32599]: fsmap cephfs:2/3 {0=f=up:active,2=c=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-07-26T13:49:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-26T13:49:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:52 smithi100 ceph-mon[32599]: fsmap cephfs:3/3 {0=f=up:active,1=h=up:replay,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:49:52.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:52 smithi167 ceph-mon[37140]: daemon mds.b restarted 2024-07-26T13:49:52.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:52 smithi167 ceph-mon[37140]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-26T13:49:52.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:52 smithi167 ceph-mon[37140]: osdmap e93: 12 total, 12 up, 12 in 2024-07-26T13:49:52.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:52 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.138:6838/133335730,v1:172.21.15.138:6839/133335730] up:boot 2024-07-26T13:49:52.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:52 smithi167 ceph-mon[37140]: Standby daemon mds.h assigned to filesystem cephfs as rank 1 2024-07-26T13:49:52.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:52 smithi167 ceph-mon[37140]: fsmap cephfs:2/3 {0=f=up:active,2=c=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-07-26T13:49:52.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-26T13:49:52.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:52 smithi167 ceph-mon[37140]: fsmap cephfs:3/3 {0=f=up:active,1=h=up:replay,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:49:53.434 INFO:tasks.workunit.client.0.smithi100.stdout: 1 461970 4.69 MB/sec execute 367 sec latency 13059.742 ms 2024-07-26T13:49:53.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:53 smithi138 ceph-mon[36954]: pgmap v669: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 1.4 MiB/s wr, 59 op/s 2024-07-26T13:49:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:53 smithi100 ceph-mon[32599]: pgmap v669: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 1.4 MiB/s wr, 59 op/s 2024-07-26T13:49:53.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:53 smithi167 ceph-mon[37140]: pgmap v669: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 1.4 MiB/s wr, 59 op/s 2024-07-26T13:49:54.434 INFO:tasks.workunit.client.0.smithi100.stdout: 1 461970 4.68 MB/sec execute 368 sec latency 14059.858 ms 2024-07-26T13:49:54.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:54 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.138:6836/3393225483,v1:172.21.15.138:6837/3393225483] up:resolve 2024-07-26T13:49:54.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:54 smithi138 ceph-mon[36954]: fsmap cephfs:3/3 {0=f=up:active,1=h=up:resolve,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:49:54.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:54 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.138:6836/3393225483,v1:172.21.15.138:6837/3393225483] up:resolve 2024-07-26T13:49:54.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:54 smithi100 ceph-mon[32599]: fsmap cephfs:3/3 {0=f=up:active,1=h=up:resolve,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:49:54.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:54 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.138:6836/3393225483,v1:172.21.15.138:6837/3393225483] up:resolve 2024-07-26T13:49:54.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:54 smithi167 ceph-mon[37140]: fsmap cephfs:3/3 {0=f=up:active,1=h=up:resolve,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:49:55.434 INFO:tasks.workunit.client.0.smithi100.stdout: 1 461970 4.67 MB/sec execute 369 sec latency 15059.923 ms 2024-07-26T13:49:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:55 smithi138 ceph-mon[36954]: reconnect by client.15333 v1:192.168.0.1:0/1052122651 after 0.000999979 2024-07-26T13:49:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:55 smithi138 ceph-mon[36954]: reconnect by client.15600 172.21.15.100:0/4280938035 after 0.00199996 2024-07-26T13:49:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:55 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.138:6836/3393225483,v1:172.21.15.138:6837/3393225483] up:reconnect 2024-07-26T13:49:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:55 smithi138 ceph-mon[36954]: fsmap cephfs:3/3 {0=f=up:active,1=h=up:reconnect,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:49:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:55 smithi138 ceph-mon[36954]: pgmap v670: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 1.5 MiB/s wr, 62 op/s 2024-07-26T13:49:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:55 smithi100 ceph-mon[32599]: reconnect by client.15333 v1:192.168.0.1:0/1052122651 after 0.000999979 2024-07-26T13:49:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:55 smithi100 ceph-mon[32599]: reconnect by client.15600 172.21.15.100:0/4280938035 after 0.00199996 2024-07-26T13:49:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:55 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.138:6836/3393225483,v1:172.21.15.138:6837/3393225483] up:reconnect 2024-07-26T13:49:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:55 smithi100 ceph-mon[32599]: fsmap cephfs:3/3 {0=f=up:active,1=h=up:reconnect,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:49:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:55 smithi100 ceph-mon[32599]: pgmap v670: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 1.5 MiB/s wr, 62 op/s 2024-07-26T13:49:55.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:55 smithi167 ceph-mon[37140]: reconnect by client.15333 v1:192.168.0.1:0/1052122651 after 0.000999979 2024-07-26T13:49:55.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:55 smithi167 ceph-mon[37140]: reconnect by client.15600 172.21.15.100:0/4280938035 after 0.00199996 2024-07-26T13:49:55.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:55 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.138:6836/3393225483,v1:172.21.15.138:6837/3393225483] up:reconnect 2024-07-26T13:49:55.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:55 smithi167 ceph-mon[37140]: fsmap cephfs:3/3 {0=f=up:active,1=h=up:reconnect,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:49:55.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:55 smithi167 ceph-mon[37140]: pgmap v670: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 1.5 MiB/s wr, 62 op/s 2024-07-26T13:49:56.424 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:56 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.138:6836/3393225483,v1:172.21.15.138:6837/3393225483] up:rejoin 2024-07-26T13:49:56.424 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:56 smithi167 ceph-mon[37140]: fsmap cephfs:3/3 {0=f=up:active,1=h=up:rejoin,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:49:56.424 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:56 smithi167 ceph-mon[37140]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-07-26T13:49:56.434 INFO:tasks.workunit.client.0.smithi100.stdout: 1 462241 4.66 MB/sec execute 370 sec latency 15736.051 ms 2024-07-26T13:49:56.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:56 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.138:6836/3393225483,v1:172.21.15.138:6837/3393225483] up:rejoin 2024-07-26T13:49:56.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:56 smithi138 ceph-mon[36954]: fsmap cephfs:3/3 {0=f=up:active,1=h=up:rejoin,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:49:56.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:56 smithi138 ceph-mon[36954]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-07-26T13:49:56.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:56 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.138:6836/3393225483,v1:172.21.15.138:6837/3393225483] up:rejoin 2024-07-26T13:49:56.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:56 smithi100 ceph-mon[32599]: fsmap cephfs:3/3 {0=f=up:active,1=h=up:rejoin,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:49:56.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:56 smithi100 ceph-mon[32599]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-07-26T13:49:57.434 INFO:tasks.workunit.client.0.smithi100.stdout: 1 464238 4.66 MB/sec execute 371 sec latency 16.610 ms 2024-07-26T13:49:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:57 smithi138 ceph-mon[36954]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-26T13:49:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:57 smithi138 ceph-mon[36954]: Cluster is now healthy 2024-07-26T13:49:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:57 smithi138 ceph-mon[36954]: mds.? [v2:172.21.15.138:6836/3393225483,v1:172.21.15.138:6837/3393225483] up:active 2024-07-26T13:49:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:57 smithi138 ceph-mon[36954]: fsmap cephfs:3 {0=f=up:active,1=h=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:49:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:57 smithi138 ceph-mon[36954]: fsmap cephfs:3 {0=f=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:49:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:57 smithi138 ceph-mon[36954]: pgmap v671: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 1.5 MiB/s wr, 59 op/s 2024-07-26T13:49:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:57 smithi100 ceph-mon[32599]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-26T13:49:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:57 smithi100 ceph-mon[32599]: Cluster is now healthy 2024-07-26T13:49:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:57 smithi100 ceph-mon[32599]: mds.? [v2:172.21.15.138:6836/3393225483,v1:172.21.15.138:6837/3393225483] up:active 2024-07-26T13:49:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:57 smithi100 ceph-mon[32599]: fsmap cephfs:3 {0=f=up:active,1=h=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:49:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:57 smithi100 ceph-mon[32599]: fsmap cephfs:3 {0=f=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:49:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:57 smithi100 ceph-mon[32599]: pgmap v671: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 1.5 MiB/s wr, 59 op/s 2024-07-26T13:49:57.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:57 smithi167 ceph-mon[37140]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-26T13:49:57.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:57 smithi167 ceph-mon[37140]: Cluster is now healthy 2024-07-26T13:49:57.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:57 smithi167 ceph-mon[37140]: mds.? [v2:172.21.15.138:6836/3393225483,v1:172.21.15.138:6837/3393225483] up:active 2024-07-26T13:49:57.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:57 smithi167 ceph-mon[37140]: fsmap cephfs:3 {0=f=up:active,1=h=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-07-26T13:49:57.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:57 smithi167 ceph-mon[37140]: fsmap cephfs:3 {0=f=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:49:57.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:57 smithi167 ceph-mon[37140]: pgmap v671: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 1.5 MiB/s wr, 59 op/s 2024-07-26T13:49:58.434 INFO:tasks.workunit.client.0.smithi100.stdout: 1 466320 4.68 MB/sec execute 372 sec latency 21.922 ms 2024-07-26T13:49:58.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:58.554+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:49:58.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:58.554+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:49:58.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:58.554+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:49:58.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:58.554+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:49:58.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:49:58.554+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:49:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:49:58 smithi138 ceph-mon[36954]: pgmap v672: 97 pgs: 97 active+clean; 244 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.8 MiB/s wr, 88 op/s 2024-07-26T13:49:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:49:58 smithi100 ceph-mon[32599]: pgmap v672: 97 pgs: 97 active+clean; 244 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.8 MiB/s wr, 88 op/s 2024-07-26T13:49:59.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:49:58 smithi167 ceph-mon[37140]: pgmap v672: 97 pgs: 97 active+clean; 244 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.8 MiB/s wr, 88 op/s 2024-07-26T13:49:59.434 INFO:tasks.workunit.client.0.smithi100.stdout: 1 468802 4.71 MB/sec execute 373 sec latency 31.375 ms 2024-07-26T13:50:00.434 INFO:tasks.workunit.client.0.smithi100.stdout: 1 470652 4.72 MB/sec execute 374 sec latency 40.192 ms 2024-07-26T13:50:00.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:00 smithi138 ceph-mon[36954]: overall HEALTH_OK 2024-07-26T13:50:00.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:00 smithi100 ceph-mon[32599]: overall HEALTH_OK 2024-07-26T13:50:00.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:00 smithi167 ceph-mon[37140]: overall HEALTH_OK 2024-07-26T13:50:01.434 INFO:tasks.workunit.client.0.smithi100.stdout: 1 472612 4.74 MB/sec execute 375 sec latency 28.921 ms 2024-07-26T13:50:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:01 smithi138 ceph-mon[36954]: pgmap v673: 97 pgs: 97 active+clean; 240 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.1 MiB/s wr, 105 op/s 2024-07-26T13:50:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:01 smithi100 ceph-mon[32599]: pgmap v673: 97 pgs: 97 active+clean; 240 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.1 MiB/s wr, 105 op/s 2024-07-26T13:50:01.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:01 smithi167 ceph-mon[37140]: pgmap v673: 97 pgs: 97 active+clean; 240 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.1 MiB/s wr, 105 op/s 2024-07-26T13:50:02.434 INFO:tasks.workunit.client.0.smithi100.stdout: 1 474027 4.75 MB/sec execute 376 sec latency 23.982 ms 2024-07-26T13:50:02.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-26T13:50:02.478 INFO:tasks.workunit.client.0.smithi100.stdout:[474064] write failed on handle 13210 (Resource temporarily unavailable) 2024-07-26T13:50:02.478 INFO:tasks.workunit.client.0.smithi100.stdout:Child failed with status 1 2024-07-26T13:50:02.479 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-07-26T13:50:02.479 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-07-26T13:50:02.760 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi100 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-07-26T13:50:02.989 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=5e31ef85d8ef46e6a5e499c5b046a824 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi100 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-07-26T13:50:02.992 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-07-26T13:50:03.002 INFO:tasks.quiescer:joining Quiescers 2024-07-26T13:50:03.002 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-07-26T13:50:03.003 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- ceph fs dump --format=json 2024-07-26T13:50:03.266 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:50:03.388 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:03 smithi100 ceph-mon[32599]: osdmap e94: 12 total, 12 up, 12 in 2024-07-26T13:50:03.388 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:03 smithi100 ceph-mon[32599]: pgmap v675: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.9 MiB/s wr, 169 op/s 2024-07-26T13:50:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:03 smithi138 ceph-mon[36954]: osdmap e94: 12 total, 12 up, 12 in 2024-07-26T13:50:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:03 smithi138 ceph-mon[36954]: pgmap v675: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.9 MiB/s wr, 169 op/s 2024-07-26T13:50:03.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:03 smithi167 ceph-mon[37140]: osdmap e94: 12 total, 12 up, 12 in 2024-07-26T13:50:03.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:03 smithi167 ceph-mon[37140]: pgmap v675: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.9 MiB/s wr, 169 op/s 2024-07-26T13:50:04.719 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:04 smithi100 ceph-mon[32599]: pgmap v676: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 243 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.2 MiB/s wr, 225 op/s 2024-07-26T13:50:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:04 smithi138 ceph-mon[36954]: pgmap v676: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 243 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.2 MiB/s wr, 225 op/s 2024-07-26T13:50:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:04 smithi167 ceph-mon[37140]: pgmap v676: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 243 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.2 MiB/s wr, 225 op/s 2024-07-26T13:50:05.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.078+0000 7f3058d7a640 1 -- 172.21.15.100:0/3435032426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3054104290 msgr2=0x7f30541046a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:50:05.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.078+0000 7f3058d7a640 1 --2- 172.21.15.100:0/3435032426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3054104290 0x7f30541046a0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f304c009900 tx=0x7f304c02a410 comp rx=0 tx=0).stop 2024-07-26T13:50:05.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.078+0000 7f3058d7a640 1 -- 172.21.15.100:0/3435032426 shutdown_connections 2024-07-26T13:50:05.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.078+0000 7f3058d7a640 1 --2- 172.21.15.100:0/3435032426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f305410d840 0x7f305410fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:05.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.078+0000 7f3058d7a640 1 --2- 172.21.15.100:0/3435032426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3054104cc0 0x7f305410d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:05.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.078+0000 7f3058d7a640 1 --2- 172.21.15.100:0/3435032426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3054104290 0x7f30541046a0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:05.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.078+0000 7f3058d7a640 1 -- 172.21.15.100:0/3435032426 >> 172.21.15.100:0/3435032426 conn(0x7f3054079160 msgr2=0x7f3054079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:50:05.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.078+0000 7f3058d7a640 1 -- 172.21.15.100:0/3435032426 shutdown_connections 2024-07-26T13:50:05.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.078+0000 7f3058d7a640 1 -- 172.21.15.100:0/3435032426 wait complete. 2024-07-26T13:50:05.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.079+0000 7f3058d7a640 1 Processor -- start 2024-07-26T13:50:05.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.079+0000 7f3058d7a640 1 -- start start 2024-07-26T13:50:05.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.079+0000 7f3058d7a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3054104290 0x7f30541021a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:50:05.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.079+0000 7f3058d7a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3054104cc0 0x7f30541026e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:50:05.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.079+0000 7f3058d7a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f305410d840 0x7f3054102c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:50:05.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.079+0000 7f3058d7a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3054113490 con 0x7f305410d840 2024-07-26T13:50:05.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.079+0000 7f3058d7a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3054113310 con 0x7f3054104290 2024-07-26T13:50:05.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.079+0000 7f3058d7a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3054113610 con 0x7f3054104cc0 2024-07-26T13:50:05.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.079+0000 7f3052d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f305410d840 0x7f3054102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:50:05.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.080+0000 7f3052d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f305410d840 0x7f3054102c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43480/0 (socket says 172.21.15.100:43480) 2024-07-26T13:50:05.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.080+0000 7f3052d9d640 1 -- 172.21.15.100:0/949323952 learned_addr learned my addr 172.21.15.100:0/949323952 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:50:05.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.080+0000 7f3051d9b640 1 --2- 172.21.15.100:0/949323952 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3054104cc0 0x7f30541026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:50:05.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.080+0000 7f305259c640 1 --2- 172.21.15.100:0/949323952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3054104290 0x7f30541021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:50:05.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.080+0000 7f3052d9d640 1 -- 172.21.15.100:0/949323952 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3054104cc0 msgr2=0x7f30541026e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:50:05.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.080+0000 7f3052d9d640 1 --2- 172.21.15.100:0/949323952 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3054104cc0 0x7f30541026e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:05.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.080+0000 7f3052d9d640 1 -- 172.21.15.100:0/949323952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3054104290 msgr2=0x7f30541021a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:50:05.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.080+0000 7f3052d9d640 1 --2- 172.21.15.100:0/949323952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3054104290 0x7f30541021a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:05.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.080+0000 7f3052d9d640 1 -- 172.21.15.100:0/949323952 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30541a8160 con 0x7f305410d840 2024-07-26T13:50:05.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.080+0000 7f3051d9b640 1 --2- 172.21.15.100:0/949323952 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3054104cc0 0x7f30541026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:50:05.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.080+0000 7f3052d9d640 1 --2- 172.21.15.100:0/949323952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f305410d840 0x7f3054102c20 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f304400d750 tx=0x7f304400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:50:05.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.080+0000 7f30437fe640 1 -- 172.21.15.100:0/949323952 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3044017070 con 0x7f305410d840 2024-07-26T13:50:05.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.080+0000 7f3058d7a640 1 -- 172.21.15.100:0/949323952 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30541a8450 con 0x7f305410d840 2024-07-26T13:50:05.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.080+0000 7f30437fe640 1 -- 172.21.15.100:0/949323952 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30440054d0 con 0x7f305410d840 2024-07-26T13:50:05.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.080+0000 7f30437fe640 1 -- 172.21.15.100:0/949323952 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3044004c60 con 0x7f305410d840 2024-07-26T13:50:05.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.080+0000 7f3058d7a640 1 -- 172.21.15.100:0/949323952 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30541a8940 con 0x7f305410d840 2024-07-26T13:50:05.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.081+0000 7f3058d7a640 1 -- 172.21.15.100:0/949323952 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30541046a0 con 0x7f305410d840 2024-07-26T13:50:05.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.083+0000 7f30437fe640 1 -- 172.21.15.100:0/949323952 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f3044009bd0 con 0x7f305410d840 2024-07-26T13:50:05.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.084+0000 7f30437fe640 1 --2- 172.21.15.100:0/949323952 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f303c0b3d40 0x7f303c0b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:50:05.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.084+0000 7f305259c640 1 --2- 172.21.15.100:0/949323952 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f303c0b3d40 0x7f303c0b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:50:05.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.084+0000 7f30437fe640 1 -- 172.21.15.100:0/949323952 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9088+0+0 (secure 0 0 0) 0x7f30440e4080 con 0x7f305410d840 2024-07-26T13:50:05.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.084+0000 7f305259c640 1 --2- 172.21.15.100:0/949323952 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f303c0b3d40 0x7f303c0b6250 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f304c000c00 tx=0x7f304c031400 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:50:05.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.087+0000 7f30437fe640 1 -- 172.21.15.100:0/949323952 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f30440ab160 con 0x7f305410d840 2024-07-26T13:50:05.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.296+0000 7f3058d7a640 1 -- 172.21.15.100:0/949323952 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f305402e0c0 con 0x7f305410d840 2024-07-26T13:50:05.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.297+0000 7f30437fe640 1 -- 172.21.15.100:0/949323952 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+11933 (secure 0 0 0) 0x7f30440aaa60 con 0x7f305410d840 2024-07-26T13:50:05.298 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-07-26T13:50:05.299 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":79,"btime":"2024-07-26T13:49:56:111332+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14727,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2380716258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2380716258},{"type":"v1","addr":"172.21.15.100:6835","nonce":2380716258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/3079467433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":3079467433},{"type":"v1","addr":"172.21.15.100:6837","nonce":3079467433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14790,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/313501409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":313501409},{"type":"v1","addr":"172.21.15.100:6839","nonce":313501409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6841/1402975506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6840","nonce":1402975506},{"type":"v1","addr":"172.21.15.100:6841","nonce":1402975506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24542,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2139594730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2139594730},{"type":"v1","addr":"172.21.15.138:6833","nonce":2139594730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24547,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/174302302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":174302302},{"type":"v1","addr":"172.21.15.167:6833","nonce":174302302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":79,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-26T13:37:01.281843+0000","modified":"2024-07-26T13:49:56.111313+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24586,"mds_1":24575,"mds_2":26446},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24575":{"gid":24575,"name":"h","rank":1,"incarnation":74,"state":"up:active","state_seq":205,"addr":"172.21.15.138:6837/3393225483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3393225483},{"type":"v1","addr":"172.21.15.138:6837","nonce":3393225483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"f","rank":0,"incarnation":64,"state":"up:active","state_seq":181,"addr":"172.21.15.167:6835/641259145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":641259145},{"type":"v1","addr":"172.21.15.167:6835","nonce":641259145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26446":{"gid":26446,"name":"c","rank":2,"incarnation":56,"state":"up:active","state_seq":25,"addr":"172.21.15.167:6837/3519222898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":3519222898},{"type":"v1","addr":"172.21.15.167:6837","nonce":3519222898}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26558":{"gid":26558,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/1781111462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1781111462},{"type":"v1","addr":"172.21.15.138:6835","nonce":1781111462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26726":{"gid":26726,"name":"b","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6839/133335730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":133335730},{"type":"v1","addr":"172.21.15.138:6839","nonce":133335730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26905":{"gid":26905,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/4179082337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":4179082337},{"type":"v1","addr":"172.21.15.167:6839","nonce":4179082337}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24586,"qdb_cluster":[26446,24586,24575]},"id":1}]} 2024-07-26T13:50:05.299 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 79 2024-07-26T13:50:05.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.299+0000 7f3058d7a640 1 -- 172.21.15.100:0/949323952 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f303c0b3d40 msgr2=0x7f303c0b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:50:05.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.299+0000 7f3058d7a640 1 --2- 172.21.15.100:0/949323952 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f303c0b3d40 0x7f303c0b6250 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f304c000c00 tx=0x7f304c031400 comp rx=0 tx=0).stop 2024-07-26T13:50:05.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.299+0000 7f3058d7a640 1 -- 172.21.15.100:0/949323952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f305410d840 msgr2=0x7f3054102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:50:05.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.299+0000 7f3058d7a640 1 --2- 172.21.15.100:0/949323952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f305410d840 0x7f3054102c20 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f304400d750 tx=0x7f304400dc10 comp rx=0 tx=0).stop 2024-07-26T13:50:05.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.299+0000 7f3058d7a640 1 -- 172.21.15.100:0/949323952 shutdown_connections 2024-07-26T13:50:05.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.299+0000 7f3058d7a640 1 --2- 172.21.15.100:0/949323952 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f303c0b3d40 0x7f303c0b6250 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:05.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.299+0000 7f3058d7a640 1 --2- 172.21.15.100:0/949323952 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f305410d840 0x7f3054102c20 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:05.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.299+0000 7f3058d7a640 1 --2- 172.21.15.100:0/949323952 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3054104cc0 0x7f30541026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:05.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.299+0000 7f3058d7a640 1 --2- 172.21.15.100:0/949323952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3054104290 0x7f30541021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:05.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.299+0000 7f3058d7a640 1 -- 172.21.15.100:0/949323952 >> 172.21.15.100:0/949323952 conn(0x7f3054079160 msgr2=0x7f30541002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:50:05.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.299+0000 7f3058d7a640 1 -- 172.21.15.100:0/949323952 shutdown_connections 2024-07-26T13:50:05.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:05.299+0000 7f3058d7a640 1 -- 172.21.15.100:0/949323952 wait complete. 2024-07-26T13:50:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:05 smithi138 ceph-mon[36954]: from='client.? 172.21.15.100:0/949323952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:50:05.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:05 smithi100 ceph-mon[32599]: from='client.? 172.21.15.100:0/949323952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:50:06.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:05 smithi167 ceph-mon[37140]: from='client.? 172.21.15.100:0/949323952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-26T13:50:06.354 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24586 quiesce db --cancel --all' 2024-07-26T13:50:06.354 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528 shell --fsid bf83e2e6-4b52-11ef-bcb7-c7b262605968 -- timeout 105 ceph tell mds.24586 quiesce db --cancel --all 2024-07-26T13:50:06.610 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/bf83e2e6-4b52-11ef-bcb7-c7b262605968/mon.a/config 2024-07-26T13:50:06.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:06 smithi138 ceph-mon[36954]: pgmap v677: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 248 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.8 MiB/s wr, 250 op/s 2024-07-26T13:50:06.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:06 smithi100 ceph-mon[32599]: pgmap v677: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 248 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.8 MiB/s wr, 250 op/s 2024-07-26T13:50:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:06 smithi167 ceph-mon[37140]: pgmap v677: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 248 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.8 MiB/s wr, 250 op/s 2024-07-26T13:50:07.343 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:50:07.342+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:50:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:50:07.343+0000 7f88ad5a9640 1 -- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 msgr2=0x55c70699f7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-07-26T13:50:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:50:07.343+0000 7f88ad5a9640 1 -- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 msgr2=0x55c70699f7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:50:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:50:07.343+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:50:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi100.front.sepia.ceph.com.smithi100.stderr:2024-07-26T13:50:07.343+0000 7f88ad5a9640 1 --2- 172.21.15.100:0/1992671409 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c70659dd80 0x55c70699f7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:50:07.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:50:07.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:50:07.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:50:07.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:50:07.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:50:07.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:50:07.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:50:07.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:50:07.942 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:50:07.943 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:50:08.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:50:08.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:50:08.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:50:08.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:50:08.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:50:08.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.321+0000 7f92da2c0640 1 -- 172.21.15.100:0/375713110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92d4102490 msgr2=0x7f92d41028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:50:08.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.321+0000 7f92da2c0640 1 --2- 172.21.15.100:0/375713110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92d4102490 0x7f92d41028a0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f92c4009900 tx=0x7f92c402a410 comp rx=0 tx=0).stop 2024-07-26T13:50:08.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.321+0000 7f92da2c0640 1 -- 172.21.15.100:0/375713110 shutdown_connections 2024-07-26T13:50:08.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.321+0000 7f92da2c0640 1 --2- 172.21.15.100:0/375713110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92d4108720 0x7f92d410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:08.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.321+0000 7f92da2c0640 1 --2- 172.21.15.100:0/375713110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92d4107dd0 0x7f92d41081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:08.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.321+0000 7f92da2c0640 1 --2- 172.21.15.100:0/375713110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92d4102490 0x7f92d41028a0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:08.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.321+0000 7f92da2c0640 1 -- 172.21.15.100:0/375713110 >> 172.21.15.100:0/375713110 conn(0x7f92d4079160 msgr2=0x7f92d4079570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:50:08.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.321+0000 7f92da2c0640 1 -- 172.21.15.100:0/375713110 shutdown_connections 2024-07-26T13:50:08.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.321+0000 7f92da2c0640 1 -- 172.21.15.100:0/375713110 wait complete. 2024-07-26T13:50:08.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.322+0000 7f92da2c0640 1 Processor -- start 2024-07-26T13:50:08.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.322+0000 7f92da2c0640 1 -- start start 2024-07-26T13:50:08.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.323+0000 7f92da2c0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92d4102490 0x7f92d419ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:50:08.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.323+0000 7f92da2c0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92d4107dd0 0x7f92d419b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:50:08.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.323+0000 7f92da2c0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92d4108720 0x7f92d419b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:50:08.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.323+0000 7f92da2c0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f92d41123a0 con 0x7f92d4102490 2024-07-26T13:50:08.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.323+0000 7f92da2c0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f92d4112220 con 0x7f92d4108720 2024-07-26T13:50:08.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.323+0000 7f92da2c0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f92d4112520 con 0x7f92d4107dd0 2024-07-26T13:50:08.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.323+0000 7f92d3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92d4102490 0x7f92d419ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:50:08.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.323+0000 7f92d3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92d4102490 0x7f92d419ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43498/0 (socket says 172.21.15.100:43498) 2024-07-26T13:50:08.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.323+0000 7f92d37fe640 1 --2- 172.21.15.100:0/4200421012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92d4107dd0 0x7f92d419b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:50:08.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.323+0000 7f92d3fff640 1 -- 172.21.15.100:0/4200421012 learned_addr learned my addr 172.21.15.100:0/4200421012 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:50:08.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.323+0000 7f92d885d640 1 --2- 172.21.15.100:0/4200421012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92d4108720 0x7f92d419b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:50:08.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.323+0000 7f92d37fe640 1 -- 172.21.15.100:0/4200421012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92d4108720 msgr2=0x7f92d419b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:50:08.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.323+0000 7f92d37fe640 1 --2- 172.21.15.100:0/4200421012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92d4108720 0x7f92d419b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:08.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.323+0000 7f92d37fe640 1 -- 172.21.15.100:0/4200421012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92d4102490 msgr2=0x7f92d419ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:50:08.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.323+0000 7f92d37fe640 1 --2- 172.21.15.100:0/4200421012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92d4102490 0x7f92d419ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:08.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.323+0000 7f92d37fe640 1 -- 172.21.15.100:0/4200421012 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92d41a5f70 con 0x7f92d4107dd0 2024-07-26T13:50:08.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.324+0000 7f92d37fe640 1 --2- 172.21.15.100:0/4200421012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92d4107dd0 0x7f92d419b2b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f92d406a4d0 tx=0x7f92c800dbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:50:08.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.324+0000 7f92d17fa640 1 -- 172.21.15.100:0/4200421012 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c8017070 con 0x7f92d4107dd0 2024-07-26T13:50:08.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.324+0000 7f92d17fa640 1 -- 172.21.15.100:0/4200421012 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92c80099d0 con 0x7f92d4107dd0 2024-07-26T13:50:08.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.324+0000 7f92da2c0640 1 -- 172.21.15.100:0/4200421012 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92d41a6260 con 0x7f92d4107dd0 2024-07-26T13:50:08.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.324+0000 7f92d17fa640 1 -- 172.21.15.100:0/4200421012 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c80047a0 con 0x7f92d4107dd0 2024-07-26T13:50:08.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.324+0000 7f92da2c0640 1 -- 172.21.15.100:0/4200421012 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92d41a6750 con 0x7f92d4107dd0 2024-07-26T13:50:08.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.326+0000 7f92d17fa640 1 -- 172.21.15.100:0/4200421012 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f92c8004940 con 0x7f92d4107dd0 2024-07-26T13:50:08.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.327+0000 7f92e0844740 1 Processor -- start 2024-07-26T13:50:08.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.327+0000 7f92e0844740 1 -- start start 2024-07-26T13:50:08.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.327+0000 7f92e0844740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dfea3002d0 0x55dfea6a5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:50:08.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.327+0000 7f92d17fa640 1 --2- 172.21.15.100:0/4200421012 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f92bc0b3d70 0x7f92bc0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:50:08.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.327+0000 7f92e0844740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfea2d7410 0x55dfea6a5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:50:08.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.327+0000 7f92e0844740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfea2df5f0 0x55dfea6a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:50:08.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.327+0000 7f92e0844740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55dfea61ba80 con 0x55dfea2d7410 2024-07-26T13:50:08.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.327+0000 7f92e0844740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dfea60af80 con 0x55dfea2df5f0 2024-07-26T13:50:08.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.327+0000 7f92e0844740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55dfea60efc0 con 0x55dfea3002d0 2024-07-26T13:50:08.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.327+0000 7f92d37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfea2d7410 0x55dfea6a5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:50:08.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.327+0000 7f92d3fff640 1 --2- 172.21.15.100:0/4200421012 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f92bc0b3d70 0x7f92bc0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:50:08.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.327+0000 7f92d37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfea2d7410 0x55dfea6a5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43510/0 (socket says 172.21.15.100:43510) 2024-07-26T13:50:08.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.327+0000 7f92d37fe640 1 -- 172.21.15.100:0/1872449526 learned_addr learned my addr 172.21.15.100:0/1872449526 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:50:08.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.327+0000 7f92d885d640 1 --2- 172.21.15.100:0/1872449526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfea2df5f0 0x55dfea6a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:50:08.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.327+0000 7f92d885d640 1 --2- 172.21.15.100:0/1872449526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dfea3002d0 0x55dfea6a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:50:08.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.328+0000 7f92d17fa640 1 -- 172.21.15.100:0/4200421012 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9088+0+0 (secure 0 0 0) 0x7f92c80e3e10 con 0x7f92d4107dd0 2024-07-26T13:50:08.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.328+0000 7f92d885d640 1 -- 172.21.15.100:0/1872449526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dfea3002d0 msgr2=0x55dfea6a5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:50:08.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.328+0000 7f92d885d640 1 --2- 172.21.15.100:0/1872449526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dfea3002d0 0x55dfea6a5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:08.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.328+0000 7f92d885d640 1 -- 172.21.15.100:0/1872449526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfea2d7410 msgr2=0x55dfea6a5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:50:08.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.328+0000 7f92d885d640 1 --2- 172.21.15.100:0/1872449526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfea2d7410 0x55dfea6a5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:08.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.328+0000 7f92d885d640 1 -- 172.21.15.100:0/1872449526 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfea645400 con 0x55dfea2df5f0 2024-07-26T13:50:08.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.328+0000 7f92d37fe640 1 --2- 172.21.15.100:0/1872449526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfea2d7410 0x55dfea6a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:50:08.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.328+0000 7f92d3fff640 1 --2- 172.21.15.100:0/4200421012 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f92bc0b3d70 0x7f92bc0b6280 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f92c4009230 tx=0x7f92c40058d0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:50:08.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.328+0000 7f92d885d640 1 --2- 172.21.15.100:0/1872449526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfea2df5f0 0x55dfea6a63f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f92cc014b00 tx=0x7f92cc0078b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73a59edc2e2c3e76 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:50:08.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.328+0000 7f92ba7fc640 1 -- 172.21.15.100:0/1872449526 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc0142d0 con 0x55dfea2df5f0 2024-07-26T13:50:08.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.328+0000 7f92ba7fc640 1 -- 172.21.15.100:0/1872449526 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92cc00faf0 con 0x55dfea2df5f0 2024-07-26T13:50:08.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.328+0000 7f92ba7fc640 1 -- 172.21.15.100:0/1872449526 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc004030 con 0x55dfea2df5f0 2024-07-26T13:50:08.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.329+0000 7f92e0844740 1 -- 172.21.15.100:0/1872449526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfea2df5f0 msgr2=0x55dfea6a63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:50:08.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.329+0000 7f92e0844740 1 --2- 172.21.15.100:0/1872449526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfea2df5f0 0x55dfea6a63f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f92cc014b00 tx=0x7f92cc0078b0 comp rx=0 tx=0).stop 2024-07-26T13:50:08.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.329+0000 7f92e0844740 1 -- 172.21.15.100:0/1872449526 shutdown_connections 2024-07-26T13:50:08.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.329+0000 7f92e0844740 1 --2- 172.21.15.100:0/1872449526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfea2df5f0 0x55dfea6a63f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:08.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.329+0000 7f92e0844740 1 --2- 172.21.15.100:0/1872449526 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfea2d7410 0x55dfea6a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:08.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.329+0000 7f92e0844740 1 --2- 172.21.15.100:0/1872449526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dfea3002d0 0x55dfea6a5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:08.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.329+0000 7f92e0844740 1 -- 172.21.15.100:0/1872449526 >> 172.21.15.100:0/1872449526 conn(0x55dfea2f46c0 msgr2=0x55dfea6a3860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:50:08.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.329+0000 7f92e0844740 1 -- 172.21.15.100:0/1872449526 shutdown_connections 2024-07-26T13:50:08.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.329+0000 7f92e0844740 1 -- 172.21.15.100:0/1872449526 wait complete. 2024-07-26T13:50:08.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.330+0000 7f92e0844740 1 -- start start 2024-07-26T13:50:08.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.330+0000 7f92e0844740 1 Processor -- start 2024-07-26T13:50:08.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.330+0000 7f92e0844740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfea2d7410 0x55dfea6b1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:50:08.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.330+0000 7f92e0844740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dfea2df5f0 0x55dfea6b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:50:08.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.330+0000 7f92e0844740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfea3002d0 0x55dfea6bd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:50:08.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.330+0000 7f92e0844740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55dfea46e810 con 0x55dfea3002d0 2024-07-26T13:50:08.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.330+0000 7f92e0844740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dfea5ad960 con 0x55dfea2d7410 2024-07-26T13:50:08.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.330+0000 7f92e0844740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55dfea6b40f0 con 0x55dfea2df5f0 2024-07-26T13:50:08.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.330+0000 7f92d885d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfea3002d0 0x55dfea6bd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:50:08.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.330+0000 7f92d3fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfea2d7410 0x55dfea6b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:50:08.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.330+0000 7f92d885d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfea3002d0 0x55dfea6bd3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43516/0 (socket says 172.21.15.100:43516) 2024-07-26T13:50:08.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.330+0000 7f92d3fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfea2d7410 0x55dfea6b1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.100:59420/0 (socket says 172.21.15.100:59420) 2024-07-26T13:50:08.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.330+0000 7f92d885d640 1 -- 172.21.15.100:0/3980436784 learned_addr learned my addr 172.21.15.100:0/3980436784 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:50:08.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.330+0000 7f92d37fe640 1 --2- 172.21.15.100:0/3980436784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dfea2df5f0 0x55dfea6b1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:50:08.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.330+0000 7f92e0844740 10 client.0 _ll_register_callbacks cb 0x55dfea5b5500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:50:08.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.331+0000 7f92d885d640 1 -- 172.21.15.100:0/3980436784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dfea2df5f0 msgr2=0x55dfea6b1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:50:08.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.331+0000 7f92d885d640 1 --2- 172.21.15.100:0/3980436784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dfea2df5f0 0x55dfea6b1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:08.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.331+0000 7f92d885d640 1 -- 172.21.15.100:0/3980436784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfea2d7410 msgr2=0x55dfea6b1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:50:08.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.331+0000 7f92d885d640 1 --2- 172.21.15.100:0/3980436784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfea2d7410 0x55dfea6b1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:08.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.331+0000 7f92d885d640 1 -- 172.21.15.100:0/3980436784 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfea5b19b0 con 0x55dfea3002d0 2024-07-26T13:50:08.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.331+0000 7f92d37fe640 1 --2- 172.21.15.100:0/3980436784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dfea2df5f0 0x55dfea6b1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:50:08.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.331+0000 7f92d885d640 1 --2- 172.21.15.100:0/3980436784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfea3002d0 0x55dfea6bd3e0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f92cc014740 tx=0x7f92cc0075c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b3702fe64717369 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:50:08.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.331+0000 7f92b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:50:08.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.331+0000 7f92b37fe640 1 -- 172.21.15.100:0/3980436784 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc004c40 con 0x55dfea3002d0 2024-07-26T13:50:08.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.331+0000 7f92b37fe640 1 -- 172.21.15.100:0/3980436784 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9088+0+0 (secure 0 0 0) 0x7f92cc005000 con 0x55dfea3002d0 2024-07-26T13:50:08.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.331+0000 7f92e0844740 1 -- 172.21.15.100:0/3980436784 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92cc00faf0 con 0x55dfea3002d0 2024-07-26T13:50:08.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.331+0000 7f92b37fe640 1 -- 172.21.15.100:0/3980436784 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f92cc004de0 con 0x55dfea3002d0 2024-07-26T13:50:08.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.331+0000 7f92e0844740 10 client.17547 fetch_fsmap learned FSMap version 80 2024-07-26T13:50:08.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.332+0000 7f92e0844740 1 -- 172.21.15.100:0/3980436784 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x55dfea36b8b0 con 0x55dfea3002d0 2024-07-26T13:50:08.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.332+0000 7f92b37fe640 1 -- 172.21.15.100:0/3980436784 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92cc026020 con 0x55dfea3002d0 2024-07-26T13:50:08.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.332+0000 7f92b37fe640 1 -- 172.21.15.100:0/3980436784 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 80) ==== 8354+0+0 (secure 0 0 0) 0x7f92cc02c7c0 con 0x55dfea3002d0 2024-07-26T13:50:08.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.332+0000 7f92e0844740 10 client.17547 fetch_fsmap finished waiting for FSMap version 80 2024-07-26T13:50:08.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.332+0000 7f92e0844740 10 client.17547 resolve_mds: validated gid 24586 aka daemon mds.f 2024-07-26T13:50:08.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.332+0000 7f92e0844740 20 client.17547 populate_metadata read hostname 'smithi100' 2024-07-26T13:50:08.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.332+0000 7f92e0844740 1 --2- 172.21.15.100:0/3980436784 >> [v2:172.21.15.167:6834/641259145,v1:172.21.15.167:6835/641259145] conn(0x55dfea2bec50 0x55dfea6c36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:50:08.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.332+0000 7f92e0844740 4 client.17547 mds_command: new command op to 24586 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-26T13:50:08.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.332+0000 7f92e0844740 1 -- 172.21.15.100:0/3980436784 --> [v2:172.21.15.167:6834/641259145,v1:172.21.15.167:6835/641259145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfea61d9d0 con 0x55dfea2bec50 2024-07-26T13:50:08.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.332+0000 7f92d3fff640 1 --2- 172.21.15.100:0/3980436784 >> [v2:172.21.15.167:6834/641259145,v1:172.21.15.167:6835/641259145] conn(0x55dfea2bec50 0x55dfea6c36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:50:08.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.333+0000 7f92b37fe640 10 client.17547 ms_handle_connect on v2:172.21.15.167:6834/641259145 2024-07-26T13:50:08.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.333+0000 7f92d3fff640 1 --2- 172.21.15.100:0/3980436784 >> [v2:172.21.15.167:6834/641259145,v1:172.21.15.167:6835/641259145] conn(0x55dfea2bec50 0x55dfea6c36c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f3cb2e7171a9c4d server_cookie=82e2779db8ae7dd3 in_seq=0 out_seq=0 2024-07-26T13:50:08.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.334+0000 7f92b37fe640 1 -- 172.21.15.100:0/3980436784 <== mds.0 v2:172.21.15.167:6834/641259145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dfea61d9d0 con 0x55dfea2bec50 2024-07-26T13:50:08.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.334+0000 7f92b37fe640 10 client.17547 handle_command_reply: tid=0 2024-07-26T13:50:08.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.334+0000 7f92e0844740 1 client.17547 shutdown 2024-07-26T13:50:08.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.334+0000 7f92e0844740 1 -- 172.21.15.100:0/3980436784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfea3002d0 msgr2=0x55dfea6bd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:50:08.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.334+0000 7f92e0844740 1 --2- 172.21.15.100:0/3980436784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfea3002d0 0x55dfea6bd3e0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f92cc014740 tx=0x7f92cc0075c0 comp rx=0 tx=0).stop 2024-07-26T13:50:08.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.334+0000 7f92e0844740 1 -- 172.21.15.100:0/3980436784 shutdown_connections 2024-07-26T13:50:08.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.334+0000 7f92e0844740 1 --2- 172.21.15.100:0/3980436784 >> [v2:172.21.15.167:6834/641259145,v1:172.21.15.167:6835/641259145] conn(0x55dfea2bec50 0x55dfea6c36c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:08.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.334+0000 7f92e0844740 1 --2- 172.21.15.100:0/3980436784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfea3002d0 0x55dfea6bd3e0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:08.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.334+0000 7f92e0844740 1 --2- 172.21.15.100:0/3980436784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dfea2df5f0 0x55dfea6b1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:08.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.334+0000 7f92e0844740 1 --2- 172.21.15.100:0/3980436784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfea2d7410 0x55dfea6b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:08.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.334+0000 7f92b37fe640 0 client.17547 ms_handle_reset on v2:172.21.15.167:6834/641259145 2024-07-26T13:50:08.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.334+0000 7f92e0844740 1 -- 172.21.15.100:0/3980436784 >> 172.21.15.100:0/3980436784 conn(0x55dfea2f46c0 msgr2=0x55dfea6a8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:50:08.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.334+0000 7f92e0844740 1 -- 172.21.15.100:0/3980436784 shutdown_connections 2024-07-26T13:50:08.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.334+0000 7f92e0844740 1 -- 172.21.15.100:0/3980436784 wait complete. 2024-07-26T13:50:08.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.334+0000 7f92e0844740 20 client.17547 trim_cache size 0 max 16384 2024-07-26T13:50:08.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.351+0000 7f92e0844740 1 Processor -- start 2024-07-26T13:50:08.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.351+0000 7f92e0844740 1 -- start start 2024-07-26T13:50:08.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.351+0000 7f92e0844740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfea2f46c0 0x55dfea6bd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:50:08.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.351+0000 7f92e0844740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dfea2d7410 0x55dfea6a9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:50:08.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.351+0000 7f92e0844740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfea2df5f0 0x55dfea6a31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:50:08.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.351+0000 7f92e0844740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f92d41b1ee0 con 0x55dfea2f46c0 2024-07-26T13:50:08.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.351+0000 7f92e0844740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f92d41af780 con 0x55dfea2df5f0 2024-07-26T13:50:08.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.351+0000 7f92e0844740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f92d41ad5d0 con 0x55dfea2d7410 2024-07-26T13:50:08.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.351+0000 7f92d3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfea2f46c0 0x55dfea6bd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:50:08.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.351+0000 7f92d3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfea2f46c0 0x55dfea6bd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43518/0 (socket says 172.21.15.100:43518) 2024-07-26T13:50:08.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.351+0000 7f92d3fff640 1 -- 172.21.15.100:0/2673659510 learned_addr learned my addr 172.21.15.100:0/2673659510 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:50:08.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.351+0000 7f92d37fe640 1 --2- 172.21.15.100:0/2673659510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dfea2d7410 0x55dfea6a9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:50:08.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.351+0000 7f92d885d640 1 --2- 172.21.15.100:0/2673659510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfea2df5f0 0x55dfea6a31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:50:08.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.352+0000 7f92d3fff640 1 -- 172.21.15.100:0/2673659510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dfea2d7410 msgr2=0x55dfea6a9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:50:08.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.352+0000 7f92d3fff640 1 --2- 172.21.15.100:0/2673659510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dfea2d7410 0x55dfea6a9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:08.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.352+0000 7f92d3fff640 1 -- 172.21.15.100:0/2673659510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfea2df5f0 msgr2=0x55dfea6a31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:50:08.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.352+0000 7f92d3fff640 1 --2- 172.21.15.100:0/2673659510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfea2df5f0 0x55dfea6a31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:08.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.352+0000 7f92d3fff640 1 -- 172.21.15.100:0/2673659510 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfea6a7080 con 0x55dfea2f46c0 2024-07-26T13:50:08.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.352+0000 7f92d3fff640 1 --2- 172.21.15.100:0/2673659510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfea2f46c0 0x55dfea6bd350 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f92c4031440 tx=0x7f92c4040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86ff6253b526000 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:50:08.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.352+0000 7f92b9ffb640 1 -- 172.21.15.100:0/2673659510 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c4031e60 con 0x55dfea2f46c0 2024-07-26T13:50:08.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.352+0000 7f92b9ffb640 1 -- 172.21.15.100:0/2673659510 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92c4056070 con 0x55dfea2f46c0 2024-07-26T13:50:08.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.352+0000 7f92b9ffb640 1 -- 172.21.15.100:0/2673659510 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c4035040 con 0x55dfea2f46c0 2024-07-26T13:50:08.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.353+0000 7f92e0844740 1 -- 172.21.15.100:0/2673659510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfea2f46c0 msgr2=0x55dfea6bd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:50:08.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.353+0000 7f92e0844740 1 --2- 172.21.15.100:0/2673659510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfea2f46c0 0x55dfea6bd350 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f92c4031440 tx=0x7f92c4040000 comp rx=0 tx=0).stop 2024-07-26T13:50:08.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.353+0000 7f92e0844740 1 -- 172.21.15.100:0/2673659510 shutdown_connections 2024-07-26T13:50:08.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.353+0000 7f92e0844740 1 --2- 172.21.15.100:0/2673659510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfea2df5f0 0x55dfea6a31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:08.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.353+0000 7f92e0844740 1 --2- 172.21.15.100:0/2673659510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dfea2d7410 0x55dfea6a9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:08.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.353+0000 7f92e0844740 1 --2- 172.21.15.100:0/2673659510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfea2f46c0 0x55dfea6bd350 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:08.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.353+0000 7f92e0844740 1 -- 172.21.15.100:0/2673659510 >> 172.21.15.100:0/2673659510 conn(0x55dfea6a38a0 msgr2=0x55dfea6ac470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:50:08.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.353+0000 7f92e0844740 1 -- 172.21.15.100:0/2673659510 shutdown_connections 2024-07-26T13:50:08.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.353+0000 7f92e0844740 1 -- 172.21.15.100:0/2673659510 wait complete. 2024-07-26T13:50:08.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.353+0000 7f92e0844740 1 -- start start 2024-07-26T13:50:08.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.354+0000 7f92e0844740 1 Processor -- start 2024-07-26T13:50:08.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.354+0000 7f92e0844740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfea2d7410 0x55dfea6c45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:50:08.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.354+0000 7f92e0844740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfea2df5f0 0x55dfea6c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:50:08.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.354+0000 7f92e0844740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dfea2f46c0 0x55dfea6c5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:50:08.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.354+0000 7f92e0844740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f92d41adc40 con 0x55dfea2df5f0 2024-07-26T13:50:08.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.354+0000 7f92e0844740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f92d41af0b0 con 0x55dfea2d7410 2024-07-26T13:50:08.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.354+0000 7f92e0844740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f92d41ae310 con 0x55dfea2f46c0 2024-07-26T13:50:08.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.354+0000 7f92e0844740 10 client.0 _ll_register_callbacks cb 0x7f92c806d710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-26T13:50:08.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.354+0000 7f92d37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfea2df5f0 0x55dfea6c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:50:08.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.354+0000 7f92d37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfea2df5f0 0x55dfea6c4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43522/0 (socket says 172.21.15.100:43522) 2024-07-26T13:50:08.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.354+0000 7f92d37fe640 1 -- 172.21.15.100:0/998005881 learned_addr learned my addr 172.21.15.100:0/998005881 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:50:08.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.354+0000 7f92d3fff640 1 --2- 172.21.15.100:0/998005881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfea2d7410 0x55dfea6c45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:50:08.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.354+0000 7f92d885d640 1 --2- 172.21.15.100:0/998005881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dfea2f46c0 0x55dfea6c5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:50:08.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.354+0000 7f92d37fe640 1 -- 172.21.15.100:0/998005881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dfea2f46c0 msgr2=0x55dfea6c5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:50:08.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.354+0000 7f92d37fe640 1 --2- 172.21.15.100:0/998005881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dfea2f46c0 0x55dfea6c5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:08.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.354+0000 7f92d37fe640 1 -- 172.21.15.100:0/998005881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfea2d7410 msgr2=0x55dfea6c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-26T13:50:08.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.354+0000 7f92d37fe640 1 --2- 172.21.15.100:0/998005881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfea2d7410 0x55dfea6c45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:08.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.354+0000 7f92d37fe640 1 -- 172.21.15.100:0/998005881 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfea72ac10 con 0x55dfea2df5f0 2024-07-26T13:50:08.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.354+0000 7f92d3fff640 1 --2- 172.21.15.100:0/998005881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfea2d7410 0x55dfea6c45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-26T13:50:08.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.355+0000 7f92d37fe640 1 --2- 172.21.15.100:0/998005881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfea2df5f0 0x55dfea6c4af0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f92c8005020 tx=0x7f92c80a7040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4223aea960dcd3f server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:50:08.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.355+0000 7f92b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-07-26T13:50:08.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.355+0000 7f92b37fe640 1 -- 172.21.15.100:0/998005881 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c80a72c0 con 0x55dfea2df5f0 2024-07-26T13:50:08.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.355+0000 7f92b37fe640 1 -- 172.21.15.100:0/998005881 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9088+0+0 (secure 0 0 0) 0x7f92c80abd10 con 0x55dfea2df5f0 2024-07-26T13:50:08.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.355+0000 7f92e0844740 1 -- 172.21.15.100:0/998005881 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92c4056070 con 0x55dfea2df5f0 2024-07-26T13:50:08.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.355+0000 7f92b37fe640 1 -- 172.21.15.100:0/998005881 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f92c80abf10 con 0x55dfea2df5f0 2024-07-26T13:50:08.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.355+0000 7f92e0844740 10 client.17553 fetch_fsmap learned FSMap version 80 2024-07-26T13:50:08.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.355+0000 7f92e0844740 1 -- 172.21.15.100:0/998005881 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x55dfea72bf10 con 0x55dfea2df5f0 2024-07-26T13:50:08.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.355+0000 7f92b37fe640 1 -- 172.21.15.100:0/998005881 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92c4056070 con 0x55dfea2df5f0 2024-07-26T13:50:08.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.355+0000 7f92b37fe640 1 -- 172.21.15.100:0/998005881 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 80) ==== 8354+0+0 (secure 0 0 0) 0x7f92c80cd140 con 0x55dfea2df5f0 2024-07-26T13:50:08.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.355+0000 7f92e0844740 10 client.17553 fetch_fsmap finished waiting for FSMap version 80 2024-07-26T13:50:08.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.355+0000 7f92e0844740 10 client.17553 resolve_mds: validated gid 24586 aka daemon mds.f 2024-07-26T13:50:08.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.355+0000 7f92e0844740 20 client.17553 populate_metadata read hostname 'smithi100' 2024-07-26T13:50:08.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.355+0000 7f92e0844740 1 --2- 172.21.15.100:0/998005881 >> [v2:172.21.15.167:6834/641259145,v1:172.21.15.167:6835/641259145] conn(0x55dfea3002d0 0x55dfea72e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:50:08.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.355+0000 7f92e0844740 4 client.17553 mds_command: new command op to 24586 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-07-26T13:50:08.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.355+0000 7f92e0844740 1 -- 172.21.15.100:0/998005881 --> [v2:172.21.15.167:6834/641259145,v1:172.21.15.167:6835/641259145] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55dfea619470 con 0x55dfea3002d0 2024-07-26T13:50:08.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.355+0000 7f92d3fff640 1 --2- 172.21.15.100:0/998005881 >> [v2:172.21.15.167:6834/641259145,v1:172.21.15.167:6835/641259145] conn(0x55dfea3002d0 0x55dfea72e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:50:08.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.356+0000 7f92b37fe640 10 client.17553 ms_handle_connect on v2:172.21.15.167:6834/641259145 2024-07-26T13:50:08.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.356+0000 7f92d3fff640 1 --2- 172.21.15.100:0/998005881 >> [v2:172.21.15.167:6834/641259145,v1:172.21.15.167:6835/641259145] conn(0x55dfea3002d0 0x55dfea72e700 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfce2dc9130a7cbf server_cookie=e1543db0bdeedfc9 in_seq=0 out_seq=0 2024-07-26T13:50:08.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.356+0000 7f92b37fe640 1 -- 172.21.15.100:0/998005881 <== mds.0 v2:172.21.15.167:6834/641259145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55dfea619470 con 0x55dfea3002d0 2024-07-26T13:50:08.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.356+0000 7f92b37fe640 10 client.17553 handle_command_reply: tid=0 2024-07-26T13:50:08.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.356+0000 7f92e0844740 1 client.17553 shutdown 2024-07-26T13:50:08.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.356+0000 7f92e0844740 1 -- 172.21.15.100:0/998005881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfea2df5f0 msgr2=0x55dfea6c4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:50:08.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.356+0000 7f92e0844740 1 --2- 172.21.15.100:0/998005881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfea2df5f0 0x55dfea6c4af0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f92c8005020 tx=0x7f92c80a7040 comp rx=0 tx=0).stop 2024-07-26T13:50:08.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.356+0000 7f92e0844740 1 -- 172.21.15.100:0/998005881 shutdown_connections 2024-07-26T13:50:08.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.356+0000 7f92e0844740 1 --2- 172.21.15.100:0/998005881 >> [v2:172.21.15.167:6834/641259145,v1:172.21.15.167:6835/641259145] conn(0x55dfea3002d0 0x55dfea72e700 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:08.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.356+0000 7f92e0844740 1 --2- 172.21.15.100:0/998005881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dfea2f46c0 0x55dfea6c5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:08.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.357+0000 7f92e0844740 1 --2- 172.21.15.100:0/998005881 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55dfea2df5f0 0x55dfea6c4af0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:08.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.357+0000 7f92e0844740 1 --2- 172.21.15.100:0/998005881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dfea2d7410 0x55dfea6c45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:08.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.357+0000 7f92b37fe640 0 client.17553 ms_handle_reset on v2:172.21.15.167:6834/641259145 2024-07-26T13:50:08.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.357+0000 7f92e0844740 1 -- 172.21.15.100:0/998005881 >> 172.21.15.100:0/998005881 conn(0x55dfea6a38a0 msgr2=0x55dfea6ac470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:50:08.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.357+0000 7f92e0844740 1 -- 172.21.15.100:0/998005881 shutdown_connections 2024-07-26T13:50:08.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.357+0000 7f92e0844740 1 -- 172.21.15.100:0/998005881 wait complete. 2024-07-26T13:50:08.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.357+0000 7f92e0844740 20 client.17553 trim_cache size 0 max 16384 2024-07-26T13:50:08.368 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-07-26T13:50:08.368 INFO:teuthology.orchestra.run.smithi100.stdout: "epoch": 80, 2024-07-26T13:50:08.369 INFO:teuthology.orchestra.run.smithi100.stdout: "leader": 24586, 2024-07-26T13:50:08.369 INFO:teuthology.orchestra.run.smithi100.stdout: "set_version": 23, 2024-07-26T13:50:08.369 INFO:teuthology.orchestra.run.smithi100.stdout: "sets": {} 2024-07-26T13:50:08.369 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-07-26T13:50:08.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.357+0000 7f92da2c0640 1 -- 172.21.15.100:0/4200421012 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f92bc0b3d70 msgr2=0x7f92bc0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:50:08.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.357+0000 7f92da2c0640 1 --2- 172.21.15.100:0/4200421012 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f92bc0b3d70 0x7f92bc0b6280 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f92c4009230 tx=0x7f92c40058d0 comp rx=0 tx=0).stop 2024-07-26T13:50:08.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.357+0000 7f92da2c0640 1 -- 172.21.15.100:0/4200421012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92d4107dd0 msgr2=0x7f92d419b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:50:08.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.357+0000 7f92da2c0640 1 --2- 172.21.15.100:0/4200421012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92d4107dd0 0x7f92d419b2b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f92d406a4d0 tx=0x7f92c800dbb0 comp rx=0 tx=0).stop 2024-07-26T13:50:08.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.357+0000 7f92da2c0640 1 -- 172.21.15.100:0/4200421012 shutdown_connections 2024-07-26T13:50:08.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.357+0000 7f92da2c0640 1 --2- 172.21.15.100:0/4200421012 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f92bc0b3d70 0x7f92bc0b6280 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:08.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.357+0000 7f92da2c0640 1 --2- 172.21.15.100:0/4200421012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92d4108720 0x7f92d419b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:08.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.357+0000 7f92da2c0640 1 --2- 172.21.15.100:0/4200421012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92d4107dd0 0x7f92d419b2b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:08.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.357+0000 7f92da2c0640 1 --2- 172.21.15.100:0/4200421012 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92d4102490 0x7f92d419ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:08.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.357+0000 7f92da2c0640 1 -- 172.21.15.100:0/4200421012 >> 172.21.15.100:0/4200421012 conn(0x7f92d4079160 msgr2=0x7f92d4100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:50:08.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.357+0000 7f92da2c0640 1 -- 172.21.15.100:0/4200421012 shutdown_connections 2024-07-26T13:50:08.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:08.357+0000 7f92da2c0640 1 -- 172.21.15.100:0/4200421012 wait complete. 2024-07-26T13:50:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:08 smithi138 ceph-mon[36954]: fsmap cephfs:3 {0=f=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:50:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:50:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:50:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:50:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:50:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:50:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:08 smithi138 ceph-mon[36954]: pgmap v678: 97 pgs: 97 active+clean; 255 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.9 MiB/s wr, 225 op/s 2024-07-26T13:50:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:08 smithi100 ceph-mon[32599]: fsmap cephfs:3 {0=f=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:50:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:50:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:50:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:50:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:50:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:50:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:08 smithi100 ceph-mon[32599]: pgmap v678: 97 pgs: 97 active+clean; 255 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.9 MiB/s wr, 225 op/s 2024-07-26T13:50:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:08 smithi167 ceph-mon[37140]: fsmap cephfs:3 {0=f=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:50:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:50:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:50:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:50:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:50:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:50:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:08 smithi167 ceph-mon[37140]: pgmap v678: 97 pgs: 97 active+clean; 255 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.9 MiB/s wr, 225 op/s 2024-07-26T13:50:09.150 ERROR:teuthology.run_tasks:Manager failed: quiescer Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528/qa/tasks/quiescer.py", line 422, in task stop_all_quiescers(ctx.ceph[cluster_name].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_e8f88bb1fe0f3b5172ca5b6b08c2be739c07e528/qa/tasks/quiescer.py", line 381, in stop_all_quiescers raise RuntimeError(f"error during quiesce thrashing: {thrasher.exception}") RuntimeError: error during quiesce thrashing: Error parsing quiesce response: Expecting value: line 1 column 1 (char 0) 2024-07-26T13:50:09.155 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-07-26T13:50:09.165 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi100.front.sepia.ceph.com 2024-07-26T13:50:09.174 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-07-26T13:50:09.184 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-07-26T13:50:09.184 INFO:teuthology.task.exec:Executing custom commands... 2024-07-26T13:50:09.184 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi100.front.sepia.ceph.com 2024-07-26T13:50:09.185 DEBUG:teuthology.orchestra.run.smithi100:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-07-26T13:50:09.242 DEBUG:teuthology.orchestra.run.smithi100:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-07-26T13:50:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:10 smithi138 ceph-mon[36954]: pgmap v679: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 213 op/s 2024-07-26T13:50:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:10 smithi100 ceph-mon[32599]: pgmap v679: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 213 op/s 2024-07-26T13:50:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:10 smithi167 ceph-mon[37140]: pgmap v679: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.1 MiB/s wr, 213 op/s 2024-07-26T13:50:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:12 smithi138 ceph-mon[36954]: fsmap cephfs:3 {0=f=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:50:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:12 smithi138 ceph-mon[36954]: pgmap v680: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 187 op/s 2024-07-26T13:50:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:12 smithi100 ceph-mon[32599]: fsmap cephfs:3 {0=f=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:50:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:12 smithi100 ceph-mon[32599]: pgmap v680: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 187 op/s 2024-07-26T13:50:13.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:12 smithi167 ceph-mon[37140]: fsmap cephfs:3 {0=f=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:50:13.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:12 smithi167 ceph-mon[37140]: pgmap v680: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 187 op/s 2024-07-26T13:50:13.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:13.569+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:50:13.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:13.570+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:50:13.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:13.570+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:50:13.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:13.570+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:50:13.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:13.570+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:50:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:14 smithi100 ceph-mon[32599]: osdmap e95: 12 total, 12 up, 12 in 2024-07-26T13:50:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:14 smithi100 ceph-mon[32599]: pgmap v682: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 3.1 MiB/s wr, 86 op/s 2024-07-26T13:50:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:14 smithi167 ceph-mon[37140]: osdmap e95: 12 total, 12 up, 12 in 2024-07-26T13:50:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:14 smithi167 ceph-mon[37140]: pgmap v682: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 3.1 MiB/s wr, 86 op/s 2024-07-26T13:50:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:14 smithi138 ceph-mon[36954]: osdmap e95: 12 total, 12 up, 12 in 2024-07-26T13:50:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:14 smithi138 ceph-mon[36954]: pgmap v682: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 3.1 MiB/s wr, 86 op/s 2024-07-26T13:50:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:16 smithi138 ceph-mon[36954]: pgmap v683: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 2.4 MiB/s wr, 55 op/s 2024-07-26T13:50:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:16 smithi100 ceph-mon[32599]: pgmap v683: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 2.4 MiB/s wr, 55 op/s 2024-07-26T13:50:17.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:16 smithi167 ceph-mon[37140]: pgmap v683: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 2.4 MiB/s wr, 55 op/s 2024-07-26T13:50:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:18 smithi138 ceph-mon[36954]: pgmap v684: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 1.5 MiB/s wr, 40 op/s 2024-07-26T13:50:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:18 smithi100 ceph-mon[32599]: pgmap v684: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 1.5 MiB/s wr, 40 op/s 2024-07-26T13:50:19.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:18 smithi167 ceph-mon[37140]: pgmap v684: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 1.5 MiB/s wr, 40 op/s 2024-07-26T13:50:21.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:21 smithi138 ceph-mon[36954]: fsmap cephfs:3 {0=f=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:50:21.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:21 smithi138 ceph-mon[36954]: pgmap v685: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 939 KiB/s wr, 31 op/s 2024-07-26T13:50:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:21 smithi100 ceph-mon[32599]: fsmap cephfs:3 {0=f=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:50:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:21 smithi100 ceph-mon[32599]: pgmap v685: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 939 KiB/s wr, 31 op/s 2024-07-26T13:50:21.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:21 smithi167 ceph-mon[37140]: fsmap cephfs:3 {0=f=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:50:21.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:21 smithi167 ceph-mon[37140]: pgmap v685: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 939 KiB/s wr, 31 op/s 2024-07-26T13:50:22.312 DEBUG:teuthology.orchestra.run.smithi100:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-07-26T13:50:22.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.436+0000 7ff1fe1c4640 1 -- 172.21.15.100:0/2197382328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1f810d790 msgr2=0x7ff1f810fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:50:22.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.436+0000 7ff1fe1c4640 1 --2- 172.21.15.100:0/2197382328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1f810d790 0x7ff1f810fc80 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0009900 tx=0x7ff1f002a410 comp rx=0 tx=0).stop 2024-07-26T13:50:22.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.436+0000 7ff1fe1c4640 1 -- 172.21.15.100:0/2197382328 shutdown_connections 2024-07-26T13:50:22.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.436+0000 7ff1fe1c4640 1 --2- 172.21.15.100:0/2197382328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1f810d790 0x7ff1f810fc80 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:22.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.436+0000 7ff1fe1c4640 1 --2- 172.21.15.100:0/2197382328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1f8104c80 0x7ff1f810d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:22.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.436+0000 7ff1fe1c4640 1 --2- 172.21.15.100:0/2197382328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1f8104330 0x7ff1f8104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:22.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.436+0000 7ff1fe1c4640 1 -- 172.21.15.100:0/2197382328 >> 172.21.15.100:0/2197382328 conn(0x7ff1f807a170 msgr2=0x7ff1f807a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:50:22.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.436+0000 7ff1fe1c4640 1 -- 172.21.15.100:0/2197382328 shutdown_connections 2024-07-26T13:50:22.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.436+0000 7ff1fe1c4640 1 -- 172.21.15.100:0/2197382328 wait complete. 2024-07-26T13:50:22.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.437+0000 7ff1fe1c4640 1 Processor -- start 2024-07-26T13:50:22.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.437+0000 7ff1fe1c4640 1 -- start start 2024-07-26T13:50:22.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.437+0000 7ff1fe1c4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1f8104330 0x7ff1f8102060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:50:22.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.437+0000 7ff1fe1c4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1f8104c80 0x7ff1f81025a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:50:22.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.437+0000 7ff1fe1c4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1f810d790 0x7ff1f8102ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:50:22.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.437+0000 7ff1fe1c4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f8113400 con 0x7ff1f810d790 2024-07-26T13:50:22.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.437+0000 7ff1fe1c4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f8113280 con 0x7ff1f8104330 2024-07-26T13:50:22.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.437+0000 7ff1fe1c4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f8113580 con 0x7ff1f8104c80 2024-07-26T13:50:22.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.438+0000 7ff1f7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1f810d790 0x7ff1f8102ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:50:22.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.438+0000 7ff1f7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1f810d790 0x7ff1f8102ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56390/0 (socket says 172.21.15.100:56390) 2024-07-26T13:50:22.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.438+0000 7ff1f7fff640 1 -- 172.21.15.100:0/3007508449 learned_addr learned my addr 172.21.15.100:0/3007508449 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:50:22.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.438+0000 7ff1f6ffd640 1 --2- 172.21.15.100:0/3007508449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1f8104c80 0x7ff1f81025a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:50:22.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.438+0000 7ff1f77fe640 1 --2- 172.21.15.100:0/3007508449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1f8104330 0x7ff1f8102060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:50:22.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.438+0000 7ff1f7fff640 1 -- 172.21.15.100:0/3007508449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1f8104c80 msgr2=0x7ff1f81025a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:50:22.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.438+0000 7ff1f7fff640 1 --2- 172.21.15.100:0/3007508449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1f8104c80 0x7ff1f81025a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:22.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.438+0000 7ff1f7fff640 1 -- 172.21.15.100:0/3007508449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1f8104330 msgr2=0x7ff1f8102060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:50:22.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.438+0000 7ff1f7fff640 1 --2- 172.21.15.100:0/3007508449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1f8104330 0x7ff1f8102060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:22.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.438+0000 7ff1f7fff640 1 -- 172.21.15.100:0/3007508449 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f81a8070 con 0x7ff1f810d790 2024-07-26T13:50:22.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.438+0000 7ff1f77fe640 1 --2- 172.21.15.100:0/3007508449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1f8104330 0x7ff1f8102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-26T13:50:22.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.438+0000 7ff1f7fff640 1 --2- 172.21.15.100:0/3007508449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1f810d790 0x7ff1f8102ae0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0004770 tx=0x7ff1f0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:50:22.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.438+0000 7ff1f4ff9640 1 -- 172.21.15.100:0/3007508449 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f003f030 con 0x7ff1f810d790 2024-07-26T13:50:22.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.438+0000 7ff1f4ff9640 1 -- 172.21.15.100:0/3007508449 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1f0003aa0 con 0x7ff1f810d790 2024-07-26T13:50:22.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.438+0000 7ff1fe1c4640 1 -- 172.21.15.100:0/3007508449 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f81a8300 con 0x7ff1f810d790 2024-07-26T13:50:22.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.438+0000 7ff1f4ff9640 1 -- 172.21.15.100:0/3007508449 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0035070 con 0x7ff1f810d790 2024-07-26T13:50:22.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.438+0000 7ff1fe1c4640 1 -- 172.21.15.100:0/3007508449 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f81a8840 con 0x7ff1f810d790 2024-07-26T13:50:22.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.439+0000 7ff1fe1c4640 1 -- 172.21.15.100:0/3007508449 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1f8104740 con 0x7ff1f810d790 2024-07-26T13:50:22.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.442+0000 7ff1f4ff9640 1 -- 172.21.15.100:0/3007508449 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7ff1f0042030 con 0x7ff1f810d790 2024-07-26T13:50:22.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.442+0000 7ff1f4ff9640 1 --2- 172.21.15.100:0/3007508449 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff1e00b3df0 0x7ff1e00b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:50:22.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.443+0000 7ff1f77fe640 1 --2- 172.21.15.100:0/3007508449 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff1e00b3df0 0x7ff1e00b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:50:22.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.443+0000 7ff1f4ff9640 1 -- 172.21.15.100:0/3007508449 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9044+0+0 (secure 0 0 0) 0x7ff1f0104bd0 con 0x7ff1f810d790 2024-07-26T13:50:22.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.443+0000 7ff1f77fe640 1 --2- 172.21.15.100:0/3007508449 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff1e00b3df0 0x7ff1e00b6300 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8005d00 tx=0x7ff1e8005ef0 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:50:22.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.446+0000 7ff1f4ff9640 1 -- 172.21.15.100:0/3007508449 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7ff1f00cbcd0 con 0x7ff1f810d790 2024-07-26T13:50:22.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.652+0000 7ff1fe1c4640 1 -- 172.21.15.100:0/3007508449 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7ff1f81062d0 con 0x7ff1e00b3df0 2024-07-26T13:50:22.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.661+0000 7ff1f4ff9640 1 -- 172.21.15.100:0/3007508449 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7ff1f81062d0 con 0x7ff1e00b3df0 2024-07-26T13:50:22.662 INFO:teuthology.orchestra.run.smithi100.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-07-26T00:00:00", "created": "2024-07-26T13:40:21", "first": "2024-07-26T13:41:00", "last": "2024-07-26T13:50:00", "last_pruned": "2024-07-26T13:50:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-07-26T13:50:22.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.663+0000 7ff1fe1c4640 1 -- 172.21.15.100:0/3007508449 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff1e00b3df0 msgr2=0x7ff1e00b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:50:22.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.663+0000 7ff1fe1c4640 1 --2- 172.21.15.100:0/3007508449 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff1e00b3df0 0x7ff1e00b6300 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8005d00 tx=0x7ff1e8005ef0 comp rx=0 tx=0).stop 2024-07-26T13:50:22.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.663+0000 7ff1fe1c4640 1 -- 172.21.15.100:0/3007508449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1f810d790 msgr2=0x7ff1f8102ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:50:22.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.663+0000 7ff1fe1c4640 1 --2- 172.21.15.100:0/3007508449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1f810d790 0x7ff1f8102ae0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0004770 tx=0x7ff1f0004270 comp rx=0 tx=0).stop 2024-07-26T13:50:22.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.663+0000 7ff1fe1c4640 1 -- 172.21.15.100:0/3007508449 shutdown_connections 2024-07-26T13:50:22.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.663+0000 7ff1fe1c4640 1 --2- 172.21.15.100:0/3007508449 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7ff1e00b3df0 0x7ff1e00b6300 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:22.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.663+0000 7ff1fe1c4640 1 --2- 172.21.15.100:0/3007508449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff1f810d790 0x7ff1f8102ae0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:22.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.663+0000 7ff1fe1c4640 1 --2- 172.21.15.100:0/3007508449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1f8104c80 0x7ff1f81025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:22.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.663+0000 7ff1fe1c4640 1 --2- 172.21.15.100:0/3007508449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1f8104330 0x7ff1f8102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:22.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.663+0000 7ff1fe1c4640 1 -- 172.21.15.100:0/3007508449 >> 172.21.15.100:0/3007508449 conn(0x7ff1f807a170 msgr2=0x7ff1f8105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:50:22.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.663+0000 7ff1fe1c4640 1 -- 172.21.15.100:0/3007508449 shutdown_connections 2024-07-26T13:50:22.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.663+0000 7ff1fe1c4640 1 -- 172.21.15.100:0/3007508449 wait complete. 2024-07-26T13:50:22.676 DEBUG:teuthology.orchestra.run.smithi100:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-07-26T13:50:22.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.789+0000 7f9f9dc44640 1 -- 172.21.15.100:0/3349729669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f981097b0 msgr2=0x7f9f9810ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:50:22.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.789+0000 7f9f9dc44640 1 --2- 172.21.15.100:0/3349729669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f981097b0 0x7f9f9810ffe0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f9f90009ca0 tx=0x7f9f9002a8c0 comp rx=0 tx=0).stop 2024-07-26T13:50:22.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.789+0000 7f9f9dc44640 1 -- 172.21.15.100:0/3349729669 shutdown_connections 2024-07-26T13:50:22.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.789+0000 7f9f9dc44640 1 --2- 172.21.15.100:0/3349729669 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f981097b0 0x7f9f9810ffe0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:22.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.789+0000 7f9f9dc44640 1 --2- 172.21.15.100:0/3349729669 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f98108e60 0x7f9f98109270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:22.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.789+0000 7f9f9dc44640 1 --2- 172.21.15.100:0/3349729669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f98103530 0x7f9f98103940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:22.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.789+0000 7f9f9dc44640 1 -- 172.21.15.100:0/3349729669 >> 172.21.15.100:0/3349729669 conn(0x7f9f980fef30 msgr2=0x7f9f98101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:50:22.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.789+0000 7f9f9dc44640 1 -- 172.21.15.100:0/3349729669 shutdown_connections 2024-07-26T13:50:22.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.789+0000 7f9f9dc44640 1 -- 172.21.15.100:0/3349729669 wait complete. 2024-07-26T13:50:22.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.789+0000 7f9f9dc44640 1 Processor -- start 2024-07-26T13:50:22.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.790+0000 7f9f9dc44640 1 -- start start 2024-07-26T13:50:22.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.790+0000 7f9f9dc44640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f98103530 0x7f9f98199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:50:22.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.790+0000 7f9f9dc44640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f98108e60 0x7f9f9819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:50:22.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.790+0000 7f9f9dc44640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f981097b0 0x7f9f9819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:50:22.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.790+0000 7f9f9dc44640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9f98113480 con 0x7f9f98103530 2024-07-26T13:50:22.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.790+0000 7f9f9dc44640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9f98113300 con 0x7f9f981097b0 2024-07-26T13:50:22.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.790+0000 7f9f9dc44640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9f98113600 con 0x7f9f98108e60 2024-07-26T13:50:22.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.790+0000 7f9f977fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f98103530 0x7f9f98199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:50:22.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.790+0000 7f9f977fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f98103530 0x7f9f98199ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56406/0 (socket says 172.21.15.100:56406) 2024-07-26T13:50:22.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.790+0000 7f9f977fe640 1 -- 172.21.15.100:0/2646843381 learned_addr learned my addr 172.21.15.100:0/2646843381 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-07-26T13:50:22.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.790+0000 7f9f97fff640 1 --2- 172.21.15.100:0/2646843381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f981097b0 0x7f9f9819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:50:22.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.790+0000 7f9f96ffd640 1 --2- 172.21.15.100:0/2646843381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f98108e60 0x7f9f9819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:50:22.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.790+0000 7f9f977fe640 1 -- 172.21.15.100:0/2646843381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f98108e60 msgr2=0x7f9f9819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:50:22.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.790+0000 7f9f977fe640 1 --2- 172.21.15.100:0/2646843381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f98108e60 0x7f9f9819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:22.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.790+0000 7f9f977fe640 1 -- 172.21.15.100:0/2646843381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f981097b0 msgr2=0x7f9f9819f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:50:22.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.790+0000 7f9f977fe640 1 --2- 172.21.15.100:0/2646843381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f981097b0 0x7f9f9819f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:22.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.790+0000 7f9f977fe640 1 -- 172.21.15.100:0/2646843381 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f9819fe10 con 0x7f9f98103530 2024-07-26T13:50:22.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.790+0000 7f9f97fff640 1 --2- 172.21.15.100:0/2646843381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f981097b0 0x7f9f9819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-26T13:50:22.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.790+0000 7f9f977fe640 1 --2- 172.21.15.100:0/2646843381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f98103530 0x7f9f98199ee0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f9f8800c890 tx=0x7f9f8800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:50:22.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.791+0000 7f9f94ff9640 1 -- 172.21.15.100:0/2646843381 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f88004220 con 0x7f9f98103530 2024-07-26T13:50:22.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.791+0000 7f9f94ff9640 1 -- 172.21.15.100:0/2646843381 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f880043c0 con 0x7f9f98103530 2024-07-26T13:50:22.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.791+0000 7f9f9dc44640 1 -- 172.21.15.100:0/2646843381 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f981a0100 con 0x7f9f98103530 2024-07-26T13:50:22.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.791+0000 7f9f94ff9640 1 -- 172.21.15.100:0/2646843381 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f88004a00 con 0x7f9f98103530 2024-07-26T13:50:22.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.791+0000 7f9f9dc44640 1 -- 172.21.15.100:0/2646843381 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f981a6890 con 0x7f9f98103530 2024-07-26T13:50:22.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.792+0000 7f9f9dc44640 1 -- 172.21.15.100:0/2646843381 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f98103940 con 0x7f9f98103530 2024-07-26T13:50:22.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.793+0000 7f9f94ff9640 1 -- 172.21.15.100:0/2646843381 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f9f880079b0 con 0x7f9f98103530 2024-07-26T13:50:22.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.794+0000 7f9f94ff9640 1 --2- 172.21.15.100:0/2646843381 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9f800b3d40 0x7f9f800b6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T13:50:22.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.794+0000 7f9f96ffd640 1 --2- 172.21.15.100:0/2646843381 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9f800b3d40 0x7f9f800b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:50:22.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.795+0000 7f9f94ff9640 1 -- 172.21.15.100:0/2646843381 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9044+0+0 (secure 0 0 0) 0x7f9f880e4450 con 0x7f9f98103530 2024-07-26T13:50:22.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.795+0000 7f9f96ffd640 1 --2- 172.21.15.100:0/2646843381 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9f800b3d40 0x7f9f800b6250 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c005d70 tx=0x7f9f8c005d00 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T13:50:22.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:22.798+0000 7f9f94ff9640 1 -- 172.21.15.100:0/2646843381 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f9f88014030 con 0x7f9f98103530 2024-07-26T13:50:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:22 smithi138 ceph-mon[36954]: pgmap v686: 97 pgs: 97 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 572 KiB/s wr, 25 op/s 2024-07-26T13:50:22.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:22 smithi100 ceph-mon[32599]: pgmap v686: 97 pgs: 97 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 572 KiB/s wr, 25 op/s 2024-07-26T13:50:23.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:23.008+0000 7f9f9dc44640 1 -- 172.21.15.100:0/2646843381 --> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f9f98063230 con 0x7f9f800b3d40 2024-07-26T13:50:23.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:23.017+0000 7f9f94ff9640 1 -- 172.21.15.100:0/2646843381 <== mgr.14214 v2:172.21.15.100:6800/3937009534 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f9f98063230 con 0x7f9f800b3d40 2024-07-26T13:50:23.018 INFO:teuthology.orchestra.run.smithi100.stdout:/ 1m 6m3h 2024-07-26T13:50:23.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:23.019+0000 7f9f9dc44640 1 -- 172.21.15.100:0/2646843381 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9f800b3d40 msgr2=0x7f9f800b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:50:23.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:23.019+0000 7f9f9dc44640 1 --2- 172.21.15.100:0/2646843381 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9f800b3d40 0x7f9f800b6250 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c005d70 tx=0x7f9f8c005d00 comp rx=0 tx=0).stop 2024-07-26T13:50:23.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:23.019+0000 7f9f9dc44640 1 -- 172.21.15.100:0/2646843381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f98103530 msgr2=0x7f9f98199ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-26T13:50:23.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:23.019+0000 7f9f9dc44640 1 --2- 172.21.15.100:0/2646843381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f98103530 0x7f9f98199ee0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f9f8800c890 tx=0x7f9f8800cd50 comp rx=0 tx=0).stop 2024-07-26T13:50:23.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:23.019+0000 7f9f9dc44640 1 -- 172.21.15.100:0/2646843381 shutdown_connections 2024-07-26T13:50:23.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:23.019+0000 7f9f9dc44640 1 --2- 172.21.15.100:0/2646843381 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7f9f800b3d40 0x7f9f800b6250 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:23.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:23.019+0000 7f9f9dc44640 1 --2- 172.21.15.100:0/2646843381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f981097b0 0x7f9f9819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:23.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:23.019+0000 7f9f9dc44640 1 --2- 172.21.15.100:0/2646843381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f98108e60 0x7f9f9819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:23.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:23.019+0000 7f9f9dc44640 1 --2- 172.21.15.100:0/2646843381 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f98103530 0x7f9f98199ee0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T13:50:23.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:23.019+0000 7f9f9dc44640 1 -- 172.21.15.100:0/2646843381 >> 172.21.15.100:0/2646843381 conn(0x7f9f980fef30 msgr2=0x7f9f98108430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-26T13:50:23.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:23.019+0000 7f9f9dc44640 1 -- 172.21.15.100:0/2646843381 shutdown_connections 2024-07-26T13:50:23.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:23.019+0000 7f9f9dc44640 1 -- 172.21.15.100:0/2646843381 wait complete. 2024-07-26T13:50:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:22 smithi167 ceph-mon[37140]: pgmap v686: 97 pgs: 97 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 572 KiB/s wr, 25 op/s 2024-07-26T13:50:23.031 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-07-26T13:50:23.042 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-07-26T13:50:23.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:23 smithi138 ceph-mon[36954]: from='client.17559 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:50:23.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:23 smithi138 ceph-mon[36954]: from='client.17565 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:50:23.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:23 smithi100 ceph-mon[32599]: from='client.17559 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:50:23.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:23 smithi100 ceph-mon[32599]: from='client.17565 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:50:24.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:23 smithi167 ceph-mon[37140]: from='client.17559 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:50:24.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:23 smithi167 ceph-mon[37140]: from='client.17565 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-26T13:50:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:24 smithi100 ceph-mon[32599]: pgmap v687: 97 pgs: 97 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 167 KiB/s wr, 19 op/s 2024-07-26T13:50:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:24 smithi167 ceph-mon[37140]: pgmap v687: 97 pgs: 97 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 167 KiB/s wr, 19 op/s 2024-07-26T13:50:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:24 smithi138 ceph-mon[36954]: pgmap v687: 97 pgs: 97 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 167 KiB/s wr, 19 op/s 2024-07-26T13:50:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:26 smithi138 ceph-mon[36954]: pgmap v688: 97 pgs: 97 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 108 KiB/s wr, 17 op/s 2024-07-26T13:50:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:26 smithi100 ceph-mon[32599]: pgmap v688: 97 pgs: 97 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 108 KiB/s wr, 17 op/s 2024-07-26T13:50:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:26 smithi167 ceph-mon[37140]: pgmap v688: 97 pgs: 97 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 108 KiB/s wr, 17 op/s 2024-07-26T13:50:28.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:28.585+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:50:28.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:28.586+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:50:28.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:28.586+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:50:28.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:28.586+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:50:28.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:28.586+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:50:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:28 smithi138 ceph-mon[36954]: pgmap v689: 97 pgs: 97 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 110 KiB/s wr, 17 op/s 2024-07-26T13:50:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:28 smithi100 ceph-mon[32599]: pgmap v689: 97 pgs: 97 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 110 KiB/s wr, 17 op/s 2024-07-26T13:50:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:28 smithi167 ceph-mon[37140]: pgmap v689: 97 pgs: 97 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 110 KiB/s wr, 17 op/s 2024-07-26T13:50:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:30 smithi138 ceph-mon[36954]: pgmap v690: 97 pgs: 97 active+clean; 265 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 292 KiB/s wr, 17 op/s 2024-07-26T13:50:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:30 smithi100 ceph-mon[32599]: pgmap v690: 97 pgs: 97 active+clean; 265 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 292 KiB/s wr, 17 op/s 2024-07-26T13:50:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:30 smithi167 ceph-mon[37140]: pgmap v690: 97 pgs: 97 active+clean; 265 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 292 KiB/s wr, 17 op/s 2024-07-26T13:50:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:32 smithi138 ceph-mon[36954]: pgmap v691: 97 pgs: 97 active+clean; 267 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 430 KiB/s wr, 20 op/s 2024-07-26T13:50:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:32 smithi100 ceph-mon[32599]: pgmap v691: 97 pgs: 97 active+clean; 267 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 430 KiB/s wr, 20 op/s 2024-07-26T13:50:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:32 smithi167 ceph-mon[37140]: pgmap v691: 97 pgs: 97 active+clean; 267 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 430 KiB/s wr, 20 op/s 2024-07-26T13:50:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:34 smithi138 ceph-mon[36954]: pgmap v692: 97 pgs: 97 active+clean; 267 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 642 KiB/s wr, 26 op/s 2024-07-26T13:50:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:34 smithi100 ceph-mon[32599]: pgmap v692: 97 pgs: 97 active+clean; 267 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 642 KiB/s wr, 26 op/s 2024-07-26T13:50:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:34 smithi167 ceph-mon[37140]: pgmap v692: 97 pgs: 97 active+clean; 267 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 642 KiB/s wr, 26 op/s 2024-07-26T13:50:36.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:36 smithi138 ceph-mon[36954]: pgmap v693: 97 pgs: 97 active+clean; 264 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 688 KiB/s wr, 27 op/s 2024-07-26T13:50:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:36 smithi100 ceph-mon[32599]: pgmap v693: 97 pgs: 97 active+clean; 264 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 688 KiB/s wr, 27 op/s 2024-07-26T13:50:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:36 smithi167 ceph-mon[37140]: pgmap v693: 97 pgs: 97 active+clean; 264 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 688 KiB/s wr, 27 op/s 2024-07-26T13:50:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:38 smithi138 ceph-mon[36954]: pgmap v694: 97 pgs: 97 active+clean; 265 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 891 KiB/s wr, 36 op/s 2024-07-26T13:50:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:38 smithi100 ceph-mon[32599]: pgmap v694: 97 pgs: 97 active+clean; 265 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 891 KiB/s wr, 36 op/s 2024-07-26T13:50:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:38 smithi167 ceph-mon[37140]: pgmap v694: 97 pgs: 97 active+clean; 265 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 891 KiB/s wr, 36 op/s 2024-07-26T13:50:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:40 smithi138 ceph-mon[36954]: pgmap v695: 97 pgs: 97 active+clean; 264 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 906 KiB/s wr, 37 op/s 2024-07-26T13:50:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:40 smithi100 ceph-mon[32599]: pgmap v695: 97 pgs: 97 active+clean; 264 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 906 KiB/s wr, 37 op/s 2024-07-26T13:50:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:40 smithi167 ceph-mon[37140]: pgmap v695: 97 pgs: 97 active+clean; 264 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 906 KiB/s wr, 37 op/s 2024-07-26T13:50:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:42 smithi138 ceph-mon[36954]: pgmap v696: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 712 KiB/s wr, 37 op/s 2024-07-26T13:50:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:42 smithi100 ceph-mon[32599]: pgmap v696: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 712 KiB/s wr, 37 op/s 2024-07-26T13:50:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:42 smithi167 ceph-mon[37140]: pgmap v696: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 712 KiB/s wr, 37 op/s 2024-07-26T13:50:43.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:43.602+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:50:43.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:43.602+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:50:43.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:43.602+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:50:43.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:43.602+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:50:43.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:43.602+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:50:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:44 smithi138 ceph-mon[36954]: pgmap v697: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 536 KiB/s wr, 34 op/s 2024-07-26T13:50:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:44 smithi100 ceph-mon[32599]: pgmap v697: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 536 KiB/s wr, 34 op/s 2024-07-26T13:50:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:44 smithi167 ceph-mon[37140]: pgmap v697: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 536 KiB/s wr, 34 op/s 2024-07-26T13:50:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:46 smithi138 ceph-mon[36954]: pgmap v698: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 317 KiB/s wr, 25 op/s 2024-07-26T13:50:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:46 smithi100 ceph-mon[32599]: pgmap v698: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 317 KiB/s wr, 25 op/s 2024-07-26T13:50:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:46 smithi167 ceph-mon[37140]: pgmap v698: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 317 KiB/s wr, 25 op/s 2024-07-26T13:50:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:48 smithi138 ceph-mon[36954]: pgmap v699: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 264 KiB/s wr, 23 op/s 2024-07-26T13:50:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:48 smithi100 ceph-mon[32599]: pgmap v699: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 264 KiB/s wr, 23 op/s 2024-07-26T13:50:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:48 smithi167 ceph-mon[37140]: pgmap v699: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 264 KiB/s wr, 23 op/s 2024-07-26T13:50:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:50 smithi138 ceph-mon[36954]: pgmap v700: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 60 KiB/s wr, 12 op/s 2024-07-26T13:50:50.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:50 smithi100 ceph-mon[32599]: pgmap v700: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 60 KiB/s wr, 12 op/s 2024-07-26T13:50:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:50 smithi167 ceph-mon[37140]: pgmap v700: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 60 KiB/s wr, 12 op/s 2024-07-26T13:50:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:52 smithi138 ceph-mon[36954]: pgmap v701: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 11 op/s 2024-07-26T13:50:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:52 smithi100 ceph-mon[32599]: pgmap v701: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 11 op/s 2024-07-26T13:50:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:52 smithi167 ceph-mon[37140]: pgmap v701: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 11 op/s 2024-07-26T13:50:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:54 smithi138 ceph-mon[36954]: pgmap v702: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 9 op/s 2024-07-26T13:50:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:54 smithi100 ceph-mon[32599]: pgmap v702: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 9 op/s 2024-07-26T13:50:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:54 smithi167 ceph-mon[37140]: pgmap v702: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 9 op/s 2024-07-26T13:50:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:56 smithi138 ceph-mon[36954]: pgmap v703: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-07-26T13:50:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:56 smithi100 ceph-mon[32599]: pgmap v703: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-07-26T13:50:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:56 smithi167 ceph-mon[37140]: pgmap v703: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-07-26T13:50:58.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:58.615+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:50:58.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:58.616+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:50:58.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:58.616+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:50:58.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:58.616+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:50:58.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:50:58.616+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:50:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:50:58 smithi138 ceph-mon[36954]: pgmap v704: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-07-26T13:50:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:50:58 smithi100 ceph-mon[32599]: pgmap v704: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-07-26T13:50:59.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:50:58 smithi167 ceph-mon[37140]: pgmap v704: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-07-26T13:51:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:51:00 smithi138 ceph-mon[36954]: pgmap v705: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.2 KiB/s wr, 5 op/s 2024-07-26T13:51:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:51:00 smithi100 ceph-mon[32599]: pgmap v705: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.2 KiB/s wr, 5 op/s 2024-07-26T13:51:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:51:00 smithi167 ceph-mon[37140]: pgmap v705: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.2 KiB/s wr, 5 op/s 2024-07-26T13:51:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:51:02 smithi138 ceph-mon[36954]: pgmap v706: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 29 KiB/s wr, 9 op/s 2024-07-26T13:51:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:51:02 smithi100 ceph-mon[32599]: pgmap v706: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 29 KiB/s wr, 9 op/s 2024-07-26T13:51:03.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:51:02 smithi167 ceph-mon[37140]: pgmap v706: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 29 KiB/s wr, 9 op/s 2024-07-26T13:51:03.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:51:03.042+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c0034c0 con 0x7fe5a81087a0 2024-07-26T13:51:03.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:51:03.042+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 6 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c0034c0 con 0x7fe5a81087a0 2024-07-26T13:51:03.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:51:03.048+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe5800034c0 con 0x55c0ec887390 2024-07-26T13:51:03.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:51:03.048+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 6 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe5800034c0 con 0x55c0ec887390 2024-07-26T13:51:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:51:04 smithi138 ceph-mon[36954]: pgmap v707: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 56 KiB/s wr, 24 op/s 2024-07-26T13:51:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:51:04 smithi100 ceph-mon[32599]: pgmap v707: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 56 KiB/s wr, 24 op/s 2024-07-26T13:51:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:51:04 smithi167 ceph-mon[37140]: pgmap v707: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 56 KiB/s wr, 24 op/s 2024-07-26T13:51:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:51:06 smithi138 ceph-mon[36954]: pgmap v708: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 50 KiB/s wr, 22 op/s 2024-07-26T13:51:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:51:06 smithi100 ceph-mon[32599]: pgmap v708: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 50 KiB/s wr, 22 op/s 2024-07-26T13:51:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:51:06 smithi167 ceph-mon[37140]: pgmap v708: 97 pgs: 97 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 50 KiB/s wr, 22 op/s 2024-07-26T13:51:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:51:08 smithi138 ceph-mon[36954]: osdmap e96: 12 total, 12 up, 12 in 2024-07-26T13:51:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:51:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:51:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:51:08 smithi138 ceph-mon[36954]: pgmap v710: 97 pgs: 97 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 61 KiB/s wr, 27 op/s 2024-07-26T13:51:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:51:08 smithi100 ceph-mon[32599]: osdmap e96: 12 total, 12 up, 12 in 2024-07-26T13:51:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:51:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:51:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:51:08 smithi100 ceph-mon[32599]: pgmap v710: 97 pgs: 97 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 61 KiB/s wr, 27 op/s 2024-07-26T13:51:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:51:08 smithi167 ceph-mon[37140]: osdmap e96: 12 total, 12 up, 12 in 2024-07-26T13:51:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:51:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:51:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:51:08 smithi167 ceph-mon[37140]: pgmap v710: 97 pgs: 97 active+clean; 263 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 61 KiB/s wr, 27 op/s 2024-07-26T13:51:09.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:51:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:51:09.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:51:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:51:09.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:51:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:51:09.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:51:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:51:09.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:51:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:51:09.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:51:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:51:10.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:51:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:51:10.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:51:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:51:10.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:51:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:51:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:51:10 smithi100 ceph-mon[32599]: pgmap v711: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 61 KiB/s wr, 27 op/s 2024-07-26T13:51:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:51:10 smithi167 ceph-mon[37140]: pgmap v711: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 61 KiB/s wr, 27 op/s 2024-07-26T13:51:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:51:10 smithi138 ceph-mon[36954]: pgmap v711: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 61 KiB/s wr, 27 op/s 2024-07-26T13:51:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:51:12 smithi138 ceph-mon[36954]: pgmap v712: 97 pgs: 97 active+clean; 262 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 34 KiB/s wr, 24 op/s 2024-07-26T13:51:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:51:12 smithi100 ceph-mon[32599]: pgmap v712: 97 pgs: 97 active+clean; 262 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 34 KiB/s wr, 24 op/s 2024-07-26T13:51:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:51:12 smithi167 ceph-mon[37140]: pgmap v712: 97 pgs: 97 active+clean; 262 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 34 KiB/s wr, 24 op/s 2024-07-26T13:51:13.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:51:13.617+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:51:13.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:51:13.618+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:51:13.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:51:13.618+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:51:13.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:51:13.618+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:51:13.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:51:13.618+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:51:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:51:14 smithi100 ceph-mon[32599]: osdmap e97: 12 total, 12 up, 12 in 2024-07-26T13:51:14.997 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:51:14 smithi100 ceph-mon[32599]: pgmap v714: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-07-26T13:51:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:51:14 smithi167 ceph-mon[37140]: osdmap e97: 12 total, 12 up, 12 in 2024-07-26T13:51:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:51:14 smithi167 ceph-mon[37140]: pgmap v714: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-07-26T13:51:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:51:14 smithi138 ceph-mon[36954]: osdmap e97: 12 total, 12 up, 12 in 2024-07-26T13:51:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:51:14 smithi138 ceph-mon[36954]: pgmap v714: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-07-26T13:51:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:51:16 smithi138 ceph-mon[36954]: pgmap v715: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-07-26T13:51:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:51:16 smithi100 ceph-mon[32599]: pgmap v715: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-07-26T13:51:17.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:51:16 smithi167 ceph-mon[37140]: pgmap v715: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-07-26T13:51:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:51:18 smithi138 ceph-mon[36954]: pgmap v716: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-07-26T13:51:18.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:51:18 smithi100 ceph-mon[32599]: pgmap v716: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-07-26T13:51:19.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:51:18 smithi167 ceph-mon[37140]: pgmap v716: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-07-26T13:51:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:51:20 smithi138 ceph-mon[36954]: pgmap v717: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-07-26T13:51:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:51:20 smithi100 ceph-mon[32599]: pgmap v717: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-07-26T13:51:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:51:20 smithi167 ceph-mon[37140]: pgmap v717: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-07-26T13:51:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:51:22 smithi138 ceph-mon[36954]: pgmap v718: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-07-26T13:51:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:51:22 smithi100 ceph-mon[32599]: pgmap v718: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-07-26T13:51:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:51:22 smithi167 ceph-mon[37140]: pgmap v718: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 7 op/s 2024-07-26T13:51:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:51:24 smithi138 ceph-mon[36954]: pgmap v719: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-07-26T13:51:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:51:24 smithi100 ceph-mon[32599]: pgmap v719: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-07-26T13:51:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:51:24 smithi167 ceph-mon[37140]: pgmap v719: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-07-26T13:51:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:51:26 smithi138 ceph-mon[36954]: pgmap v720: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-07-26T13:51:26.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:51:26 smithi100 ceph-mon[32599]: pgmap v720: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-07-26T13:51:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:51:26 smithi167 ceph-mon[37140]: pgmap v720: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-07-26T13:51:28.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:51:28.627+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:51:28.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:51:28.628+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:51:28.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:51:28.628+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:51:28.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:51:28.628+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:51:28.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:51:28.628+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:51:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:51:28 smithi138 ceph-mon[36954]: pgmap v721: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-07-26T13:51:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:51:28 smithi100 ceph-mon[32599]: pgmap v721: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-07-26T13:51:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:51:28 smithi167 ceph-mon[37140]: pgmap v721: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-07-26T13:51:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:51:30 smithi138 ceph-mon[36954]: pgmap v722: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-07-26T13:51:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:51:30 smithi100 ceph-mon[32599]: pgmap v722: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-07-26T13:51:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:51:30 smithi167 ceph-mon[37140]: pgmap v722: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-07-26T13:51:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:51:32 smithi138 ceph-mon[36954]: pgmap v723: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-07-26T13:51:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:51:32 smithi100 ceph-mon[32599]: pgmap v723: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-07-26T13:51:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:51:32 smithi167 ceph-mon[37140]: pgmap v723: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-07-26T13:51:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:51:34 smithi138 ceph-mon[36954]: pgmap v724: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-07-26T13:51:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:51:34 smithi100 ceph-mon[32599]: pgmap v724: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-07-26T13:51:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:51:34 smithi167 ceph-mon[37140]: pgmap v724: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-07-26T13:51:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:51:36 smithi138 ceph-mon[36954]: pgmap v725: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:51:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:51:36 smithi100 ceph-mon[32599]: pgmap v725: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:51:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:51:36 smithi167 ceph-mon[37140]: pgmap v725: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:51:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:51:38 smithi138 ceph-mon[36954]: pgmap v726: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T13:51:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:51:38 smithi100 ceph-mon[32599]: pgmap v726: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T13:51:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:51:38 smithi167 ceph-mon[37140]: pgmap v726: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T13:51:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:51:40 smithi138 ceph-mon[36954]: pgmap v727: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T13:51:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:51:40 smithi100 ceph-mon[32599]: pgmap v727: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T13:51:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:51:40 smithi167 ceph-mon[37140]: pgmap v727: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T13:51:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:51:42 smithi138 ceph-mon[36954]: pgmap v728: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T13:51:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:51:42 smithi100 ceph-mon[32599]: pgmap v728: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T13:51:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:51:42 smithi167 ceph-mon[37140]: pgmap v728: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T13:51:43.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:51:43.644+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:51:43.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:51:43.645+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:51:43.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:51:43.645+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:51:43.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:51:43.645+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:51:43.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:51:43.645+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:51:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:51:44 smithi138 ceph-mon[36954]: pgmap v729: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T13:51:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:51:44 smithi100 ceph-mon[32599]: pgmap v729: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T13:51:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:51:44 smithi167 ceph-mon[37140]: pgmap v729: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T13:51:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:51:46 smithi138 ceph-mon[36954]: pgmap v730: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T13:51:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:51:46 smithi100 ceph-mon[32599]: pgmap v730: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T13:51:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:51:46 smithi167 ceph-mon[37140]: pgmap v730: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T13:51:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:51:48 smithi138 ceph-mon[36954]: pgmap v731: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-07-26T13:51:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:51:48 smithi100 ceph-mon[32599]: pgmap v731: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-07-26T13:51:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:51:48 smithi167 ceph-mon[37140]: pgmap v731: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-07-26T13:51:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:51:50 smithi138 ceph-mon[36954]: pgmap v732: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T13:51:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:51:50 smithi100 ceph-mon[32599]: pgmap v732: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T13:51:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:51:50 smithi167 ceph-mon[37140]: pgmap v732: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T13:51:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:51:52 smithi138 ceph-mon[36954]: pgmap v733: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T13:51:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:51:52 smithi100 ceph-mon[32599]: pgmap v733: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T13:51:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:51:52 smithi167 ceph-mon[37140]: pgmap v733: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T13:51:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:51:54 smithi138 ceph-mon[36954]: pgmap v734: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T13:51:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:51:54 smithi100 ceph-mon[32599]: pgmap v734: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T13:51:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:51:54 smithi167 ceph-mon[37140]: pgmap v734: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T13:51:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:51:56 smithi138 ceph-mon[36954]: pgmap v735: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T13:51:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:51:56 smithi100 ceph-mon[32599]: pgmap v735: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T13:51:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:51:56 smithi167 ceph-mon[37140]: pgmap v735: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T13:51:58.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:51:58.661+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:51:58.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:51:58.661+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:51:58.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:51:58.661+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:51:58.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:51:58.661+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:51:58.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:51:58.661+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:51:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:51:58 smithi138 ceph-mon[36954]: pgmap v736: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T13:51:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:51:58 smithi100 ceph-mon[32599]: pgmap v736: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T13:51:59.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:51:58 smithi167 ceph-mon[37140]: pgmap v736: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T13:52:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:52:00 smithi138 ceph-mon[36954]: pgmap v737: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:52:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:52:00 smithi100 ceph-mon[32599]: pgmap v737: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:52:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:52:00 smithi167 ceph-mon[37140]: pgmap v737: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:52:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:52:02 smithi138 ceph-mon[36954]: pgmap v738: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T13:52:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:52:02 smithi100 ceph-mon[32599]: pgmap v738: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T13:52:03.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:52:02 smithi167 ceph-mon[37140]: pgmap v738: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T13:52:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:52:04 smithi138 ceph-mon[36954]: pgmap v739: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-07-26T13:52:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:52:04 smithi100 ceph-mon[32599]: pgmap v739: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-07-26T13:52:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:52:04 smithi167 ceph-mon[37140]: pgmap v739: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-07-26T13:52:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:52:06 smithi138 ceph-mon[36954]: pgmap v740: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-07-26T13:52:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:52:06 smithi100 ceph-mon[32599]: pgmap v740: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-07-26T13:52:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:52:06 smithi167 ceph-mon[37140]: pgmap v740: 97 pgs: 97 active+clean; 262 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-07-26T13:52:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:52:08 smithi100 ceph-mon[32599]: osdmap e98: 12 total, 12 up, 12 in 2024-07-26T13:52:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:52:08 smithi100 ceph-mon[32599]: pgmap v742: 97 pgs: 97 active+clean; 262 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-07-26T13:52:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:52:08 smithi167 ceph-mon[37140]: osdmap e98: 12 total, 12 up, 12 in 2024-07-26T13:52:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:52:08 smithi167 ceph-mon[37140]: pgmap v742: 97 pgs: 97 active+clean; 262 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-07-26T13:52:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:52:08 smithi138 ceph-mon[36954]: osdmap e98: 12 total, 12 up, 12 in 2024-07-26T13:52:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:52:08 smithi138 ceph-mon[36954]: pgmap v742: 97 pgs: 97 active+clean; 262 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-07-26T13:52:09.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:52:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:52:09.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:52:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:52:09.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:52:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:52:09.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:52:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:52:10.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:52:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:52:10.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:52:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:52:10.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:52:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:52:10.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:52:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:52:10.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:52:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:52:10.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:52:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:52:10.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:52:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:52:10.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:52:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:52:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:52:10 smithi100 ceph-mon[32599]: pgmap v743: 97 pgs: 97 active+clean; 261 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-07-26T13:52:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:52:10 smithi167 ceph-mon[37140]: pgmap v743: 97 pgs: 97 active+clean; 261 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-07-26T13:52:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:52:10 smithi138 ceph-mon[36954]: pgmap v743: 97 pgs: 97 active+clean; 261 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-07-26T13:52:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:52:12 smithi138 ceph-mon[36954]: pgmap v744: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 24 op/s 2024-07-26T13:52:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:52:12 smithi100 ceph-mon[32599]: pgmap v744: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 24 op/s 2024-07-26T13:52:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:52:12 smithi167 ceph-mon[37140]: pgmap v744: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 24 op/s 2024-07-26T13:52:13.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:52:13.677+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:52:13.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:52:13.677+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:52:13.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:52:13.677+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:52:13.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:52:13.677+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:52:13.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:52:13.677+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:52:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:52:14 smithi167 ceph-mon[37140]: osdmap e99: 12 total, 12 up, 12 in 2024-07-26T13:52:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:52:14 smithi167 ceph-mon[37140]: pgmap v746: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T13:52:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:52:14 smithi138 ceph-mon[36954]: osdmap e99: 12 total, 12 up, 12 in 2024-07-26T13:52:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:52:14 smithi138 ceph-mon[36954]: pgmap v746: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T13:52:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:52:14 smithi100 ceph-mon[32599]: osdmap e99: 12 total, 12 up, 12 in 2024-07-26T13:52:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:52:14 smithi100 ceph-mon[32599]: pgmap v746: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T13:52:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:52:16 smithi138 ceph-mon[36954]: pgmap v747: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T13:52:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:52:16 smithi100 ceph-mon[32599]: pgmap v747: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T13:52:17.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:52:16 smithi167 ceph-mon[37140]: pgmap v747: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T13:52:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:52:18 smithi138 ceph-mon[36954]: pgmap v748: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:52:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:52:18 smithi100 ceph-mon[32599]: pgmap v748: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:52:19.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:52:18 smithi167 ceph-mon[37140]: pgmap v748: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:52:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:52:20 smithi138 ceph-mon[36954]: pgmap v749: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:52:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:52:20 smithi100 ceph-mon[32599]: pgmap v749: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:52:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:52:20 smithi167 ceph-mon[37140]: pgmap v749: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:52:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:52:22 smithi100 ceph-mon[32599]: pgmap v750: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:52:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:52:22 smithi167 ceph-mon[37140]: pgmap v750: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:52:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:52:22 smithi138 ceph-mon[36954]: pgmap v750: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:52:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:52:24 smithi100 ceph-mon[32599]: pgmap v751: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T13:52:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:52:24 smithi167 ceph-mon[37140]: pgmap v751: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T13:52:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:52:24 smithi138 ceph-mon[36954]: pgmap v751: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T13:52:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:52:26 smithi100 ceph-mon[32599]: pgmap v752: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:52:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:52:26 smithi167 ceph-mon[37140]: pgmap v752: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:52:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:52:26 smithi138 ceph-mon[36954]: pgmap v752: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:52:28.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:52:28.693+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:52:28.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:52:28.693+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:52:28.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:52:28.694+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:52:28.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:52:28.694+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:52:28.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:52:28.694+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:52:28.992 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:52:28 smithi138 ceph-mon[36954]: pgmap v753: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:52:28.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:52:28 smithi100 ceph-mon[32599]: pgmap v753: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:52:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:52:28 smithi167 ceph-mon[37140]: pgmap v753: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:52:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:52:30 smithi100 ceph-mon[32599]: pgmap v754: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:52:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:52:30 smithi167 ceph-mon[37140]: pgmap v754: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:52:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:52:30 smithi138 ceph-mon[36954]: pgmap v754: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:52:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:52:32 smithi100 ceph-mon[32599]: pgmap v755: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:52:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:52:32 smithi167 ceph-mon[37140]: pgmap v755: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:52:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:52:32 smithi138 ceph-mon[36954]: pgmap v755: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:52:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:52:34 smithi100 ceph-mon[32599]: pgmap v756: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:52:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:52:34 smithi167 ceph-mon[37140]: pgmap v756: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:52:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:52:34 smithi138 ceph-mon[36954]: pgmap v756: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:52:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:52:36 smithi100 ceph-mon[32599]: pgmap v757: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:52:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:52:36 smithi167 ceph-mon[37140]: pgmap v757: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:52:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:52:36 smithi138 ceph-mon[36954]: pgmap v757: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:52:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:52:38 smithi100 ceph-mon[32599]: pgmap v758: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:52:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:52:38 smithi167 ceph-mon[37140]: pgmap v758: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:52:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:52:38 smithi138 ceph-mon[36954]: pgmap v758: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:52:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:52:40 smithi100 ceph-mon[32599]: pgmap v759: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:52:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:52:40 smithi167 ceph-mon[37140]: pgmap v759: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:52:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:52:40 smithi138 ceph-mon[36954]: pgmap v759: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:52:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:52:42 smithi138 ceph-mon[36954]: fsmap cephfs:3 {0=f=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:52:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:52:42 smithi138 ceph-mon[36954]: pgmap v760: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:52:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:52:42 smithi100 ceph-mon[32599]: fsmap cephfs:3 {0=f=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:52:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:52:42 smithi100 ceph-mon[32599]: pgmap v760: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:52:43.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:52:42 smithi167 ceph-mon[37140]: fsmap cephfs:3 {0=f=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-07-26T13:52:43.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:52:42 smithi167 ceph-mon[37140]: pgmap v760: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:52:43.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:52:43.695+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:52:43.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:52:43.696+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:52:43.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:52:43.696+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:52:43.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:52:43.696+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:52:43.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:52:43.696+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:52:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:52:44 smithi100 ceph-mon[32599]: pgmap v761: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:52:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:52:44 smithi167 ceph-mon[37140]: pgmap v761: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:52:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:52:44 smithi138 ceph-mon[36954]: pgmap v761: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:52:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:52:46 smithi100 ceph-mon[32599]: pgmap v762: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:52:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:52:46 smithi167 ceph-mon[37140]: pgmap v762: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:52:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:52:46 smithi138 ceph-mon[36954]: pgmap v762: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:52:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:52:48 smithi100 ceph-mon[32599]: pgmap v763: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:52:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:52:48 smithi167 ceph-mon[37140]: pgmap v763: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:52:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:52:48 smithi138 ceph-mon[36954]: pgmap v763: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:52:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:52:50 smithi100 ceph-mon[32599]: pgmap v764: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:52:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:52:50 smithi167 ceph-mon[37140]: pgmap v764: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:52:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:52:50 smithi138 ceph-mon[36954]: pgmap v764: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:52:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:52:52 smithi100 ceph-mon[32599]: pgmap v765: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:52:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:52:52 smithi167 ceph-mon[37140]: pgmap v765: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:52:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:52:52 smithi138 ceph-mon[36954]: pgmap v765: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:52:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:52:54 smithi100 ceph-mon[32599]: pgmap v766: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:52:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:52:54 smithi167 ceph-mon[37140]: pgmap v766: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:52:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:52:54 smithi138 ceph-mon[36954]: pgmap v766: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:52:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:52:56 smithi100 ceph-mon[32599]: pgmap v767: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:52:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:52:56 smithi167 ceph-mon[37140]: pgmap v767: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:52:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:52:56 smithi138 ceph-mon[36954]: pgmap v767: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:52:58.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:52:58.711+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:52:58.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:52:58.712+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:52:58.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:52:58.712+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:52:58.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:52:58.712+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:52:58.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:52:58.712+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:52:58.992 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:52:58 smithi138 ceph-mon[36954]: pgmap v768: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:52:58.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:52:58 smithi100 ceph-mon[32599]: pgmap v768: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:52:59.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:52:58 smithi167 ceph-mon[37140]: pgmap v768: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:53:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:53:00 smithi100 ceph-mon[32599]: pgmap v769: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:53:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:53:00 smithi167 ceph-mon[37140]: pgmap v769: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:53:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:53:00 smithi138 ceph-mon[36954]: pgmap v769: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:53:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:53:02 smithi100 ceph-mon[32599]: pgmap v770: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T13:53:03.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:53:02 smithi167 ceph-mon[37140]: pgmap v770: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T13:53:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:53:02 smithi138 ceph-mon[36954]: pgmap v770: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T13:53:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:53:04 smithi100 ceph-mon[32599]: pgmap v771: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-07-26T13:53:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:53:04 smithi167 ceph-mon[37140]: pgmap v771: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-07-26T13:53:05.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:53:04 smithi138 ceph-mon[36954]: pgmap v771: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-07-26T13:53:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:53:06 smithi100 ceph-mon[32599]: pgmap v772: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-07-26T13:53:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:53:06 smithi167 ceph-mon[37140]: pgmap v772: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-07-26T13:53:07.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:53:06 smithi138 ceph-mon[36954]: pgmap v772: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-07-26T13:53:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:53:08 smithi100 ceph-mon[32599]: osdmap e100: 12 total, 12 up, 12 in 2024-07-26T13:53:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:53:08 smithi100 ceph-mon[32599]: pgmap v774: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-07-26T13:53:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:53:08 smithi167 ceph-mon[37140]: osdmap e100: 12 total, 12 up, 12 in 2024-07-26T13:53:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:53:08 smithi167 ceph-mon[37140]: pgmap v774: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-07-26T13:53:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:53:08 smithi138 ceph-mon[36954]: osdmap e100: 12 total, 12 up, 12 in 2024-07-26T13:53:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:53:08 smithi138 ceph-mon[36954]: pgmap v774: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-07-26T13:53:09.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:53:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:53:10.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:53:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:53:10.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:53:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:53:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:53:10 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:53:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:53:10 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:53:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:53:10 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:53:10.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:53:10 smithi100 ceph-mon[32599]: pgmap v775: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-07-26T13:53:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:53:10 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:53:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:53:10 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:53:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:53:10 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:53:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:53:10 smithi167 ceph-mon[37140]: pgmap v775: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-07-26T13:53:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:53:10 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:53:11.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:53:10 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:53:11.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:53:10 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:53:11.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:53:10 smithi138 ceph-mon[36954]: pgmap v775: 97 pgs: 97 active+clean; 260 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-07-26T13:53:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:53:12 smithi100 ceph-mon[32599]: pgmap v776: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 24 op/s 2024-07-26T13:53:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:53:12 smithi167 ceph-mon[37140]: pgmap v776: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 24 op/s 2024-07-26T13:53:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:53:12 smithi138 ceph-mon[36954]: pgmap v776: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 24 op/s 2024-07-26T13:53:13.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:53:13.726+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:53:13.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:53:13.726+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:53:13.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:53:13.726+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:53:13.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:53:13.726+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:53:13.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:53:13.726+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:53:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:53:14 smithi167 ceph-mon[37140]: osdmap e101: 12 total, 12 up, 12 in 2024-07-26T13:53:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:53:14 smithi167 ceph-mon[37140]: pgmap v778: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T13:53:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:53:14 smithi138 ceph-mon[36954]: osdmap e101: 12 total, 12 up, 12 in 2024-07-26T13:53:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:53:14 smithi138 ceph-mon[36954]: pgmap v778: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T13:53:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:53:14 smithi100 ceph-mon[32599]: osdmap e101: 12 total, 12 up, 12 in 2024-07-26T13:53:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:53:14 smithi100 ceph-mon[32599]: pgmap v778: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T13:53:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:53:16 smithi100 ceph-mon[32599]: pgmap v779: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T13:53:17.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:53:16 smithi167 ceph-mon[37140]: pgmap v779: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T13:53:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:53:16 smithi138 ceph-mon[36954]: pgmap v779: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T13:53:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:53:18 smithi100 ceph-mon[32599]: pgmap v780: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:53:19.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:53:18 smithi167 ceph-mon[37140]: pgmap v780: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:53:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:53:18 smithi138 ceph-mon[36954]: pgmap v780: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:53:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:53:20 smithi100 ceph-mon[32599]: pgmap v781: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:53:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:53:20 smithi167 ceph-mon[37140]: pgmap v781: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:53:21.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:53:20 smithi138 ceph-mon[36954]: pgmap v781: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:53:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:53:22 smithi100 ceph-mon[32599]: pgmap v782: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:53:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:53:22 smithi167 ceph-mon[37140]: pgmap v782: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:53:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:53:22 smithi138 ceph-mon[36954]: pgmap v782: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:53:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:53:24 smithi100 ceph-mon[32599]: pgmap v783: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T13:53:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:53:24 smithi167 ceph-mon[37140]: pgmap v783: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T13:53:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:53:24 smithi138 ceph-mon[36954]: pgmap v783: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T13:53:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:53:26 smithi100 ceph-mon[32599]: pgmap v784: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:53:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:53:26 smithi167 ceph-mon[37140]: pgmap v784: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:53:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:53:26 smithi138 ceph-mon[36954]: pgmap v784: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:53:28.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:53:28.735+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:53:28.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:53:28.735+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:53:28.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:53:28.735+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:53:28.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:53:28.735+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:53:28.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:53:28.735+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:53:28.992 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:53:28 smithi138 ceph-mon[36954]: pgmap v785: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:53:28.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:53:28 smithi100 ceph-mon[32599]: pgmap v785: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:53:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:53:28 smithi167 ceph-mon[37140]: pgmap v785: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:53:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:53:30 smithi100 ceph-mon[32599]: pgmap v786: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:53:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:53:30 smithi167 ceph-mon[37140]: pgmap v786: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:53:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:53:30 smithi138 ceph-mon[36954]: pgmap v786: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:53:33.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:53:33 smithi100 ceph-mon[32599]: pgmap v787: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:53:34.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:53:33 smithi167 ceph-mon[37140]: pgmap v787: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:53:34.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:53:33 smithi138 ceph-mon[36954]: pgmap v787: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:53:35.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:53:35 smithi100 ceph-mon[32599]: pgmap v788: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:53:36.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:53:35 smithi167 ceph-mon[37140]: pgmap v788: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:53:36.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:53:35 smithi138 ceph-mon[36954]: pgmap v788: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:53:37.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:53:37 smithi100 ceph-mon[32599]: pgmap v789: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:53:38.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:53:37 smithi167 ceph-mon[37140]: pgmap v789: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:53:38.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:53:37 smithi138 ceph-mon[36954]: pgmap v789: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:53:39.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:53:39 smithi100 ceph-mon[32599]: pgmap v790: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:53:40.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:53:39 smithi167 ceph-mon[37140]: pgmap v790: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:53:40.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:53:39 smithi138 ceph-mon[36954]: pgmap v790: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:53:41.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:53:41 smithi100 ceph-mon[32599]: pgmap v791: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:53:42.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:53:41 smithi167 ceph-mon[37140]: pgmap v791: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:53:42.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:53:41 smithi138 ceph-mon[36954]: pgmap v791: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:53:43.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:53:43.751+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:53:43.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:53:43.752+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:53:43.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:53:43.752+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:53:43.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:53:43.752+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:53:43.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:53:43.752+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:53:43.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:53:43 smithi100 ceph-mon[32599]: pgmap v792: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:53:44.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:53:43 smithi167 ceph-mon[37140]: pgmap v792: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:53:44.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:53:43 smithi138 ceph-mon[36954]: pgmap v792: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:53:45.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:53:45 smithi100 ceph-mon[32599]: pgmap v793: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:53:46.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:53:45 smithi167 ceph-mon[37140]: pgmap v793: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:53:46.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:53:45 smithi138 ceph-mon[36954]: pgmap v793: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:53:47.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:53:47 smithi100 ceph-mon[32599]: pgmap v794: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:53:48.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:53:47 smithi167 ceph-mon[37140]: pgmap v794: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:53:48.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:53:47 smithi138 ceph-mon[36954]: pgmap v794: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:53:49.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:53:49 smithi100 ceph-mon[32599]: pgmap v795: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:53:50.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:53:49 smithi167 ceph-mon[37140]: pgmap v795: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:53:50.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:53:49 smithi138 ceph-mon[36954]: pgmap v795: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:53:51.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:53:51 smithi100 ceph-mon[32599]: pgmap v796: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:53:52.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:53:51 smithi167 ceph-mon[37140]: pgmap v796: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:53:52.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:53:51 smithi138 ceph-mon[36954]: pgmap v796: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:53:53.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:53:53 smithi100 ceph-mon[32599]: pgmap v797: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:53:54.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:53:53 smithi167 ceph-mon[37140]: pgmap v797: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:53:54.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:53:53 smithi138 ceph-mon[36954]: pgmap v797: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:53:55.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:53:55 smithi100 ceph-mon[32599]: pgmap v798: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:53:56.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:53:55 smithi167 ceph-mon[37140]: pgmap v798: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:53:56.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:53:55 smithi138 ceph-mon[36954]: pgmap v798: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:53:57.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:53:57 smithi100 ceph-mon[32599]: pgmap v799: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:53:58.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:53:57 smithi167 ceph-mon[37140]: pgmap v799: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:53:58.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:53:57 smithi138 ceph-mon[36954]: pgmap v799: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:53:58.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:53:58.768+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:53:58.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:53:58.768+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:53:58.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:53:58.768+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:53:58.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:53:58.768+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:53:58.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:53:58.768+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:54:00.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:53:59 smithi167 ceph-mon[37140]: pgmap v800: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:54:00.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:53:59 smithi138 ceph-mon[36954]: pgmap v800: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:54:00.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:53:59 smithi100 ceph-mon[32599]: pgmap v800: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:54:02.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:54:01 smithi167 ceph-mon[37140]: pgmap v801: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:54:02.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:54:01 smithi138 ceph-mon[36954]: pgmap v801: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:54:02.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:54:01 smithi100 ceph-mon[32599]: pgmap v801: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:54:03.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:54:03.066+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c005850 con 0x7fe5a81087a0 2024-07-26T13:54:03.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:54:03.067+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 7 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c005850 con 0x7fe5a81087a0 2024-07-26T13:54:03.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:54:03.071+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580005850 con 0x55c0ec887390 2024-07-26T13:54:03.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:54:03.072+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 7 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580005850 con 0x55c0ec887390 2024-07-26T13:54:04.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:54:03 smithi167 ceph-mon[37140]: pgmap v802: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T13:54:04.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:54:03 smithi138 ceph-mon[36954]: pgmap v802: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T13:54:04.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:54:03 smithi100 ceph-mon[32599]: pgmap v802: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T13:54:06.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:54:05 smithi167 ceph-mon[37140]: pgmap v803: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-07-26T13:54:06.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:54:05 smithi138 ceph-mon[36954]: pgmap v803: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-07-26T13:54:06.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:54:05 smithi100 ceph-mon[32599]: pgmap v803: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-07-26T13:54:08.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:54:07 smithi138 ceph-mon[36954]: pgmap v804: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-07-26T13:54:08.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:54:07 smithi100 ceph-mon[32599]: pgmap v804: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-07-26T13:54:08.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:54:07 smithi167 ceph-mon[37140]: pgmap v804: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-07-26T13:54:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:54:08 smithi138 ceph-mon[36954]: osdmap e102: 12 total, 12 up, 12 in 2024-07-26T13:54:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:54:08 smithi100 ceph-mon[32599]: osdmap e102: 12 total, 12 up, 12 in 2024-07-26T13:54:09.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:54:08 smithi167 ceph-mon[37140]: osdmap e102: 12 total, 12 up, 12 in 2024-07-26T13:54:10.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:54:09 smithi138 ceph-mon[36954]: pgmap v806: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-07-26T13:54:10.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:54:09 smithi100 ceph-mon[32599]: pgmap v806: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-07-26T13:54:10.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:54:09 smithi167 ceph-mon[37140]: pgmap v806: 97 pgs: 97 active+clean; 259 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-07-26T13:54:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:54:10 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:54:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:54:10 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:54:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:54:10 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:54:11.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:54:10 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:54:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:54:10 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:54:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:54:10 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:54:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:54:10 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:54:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:54:10 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:54:11.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:54:10 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:54:11.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:54:10 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:54:11.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:54:10 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:54:11.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:54:10 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:54:12.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:54:12 smithi167 ceph-mon[37140]: pgmap v807: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-07-26T13:54:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:54:12 smithi138 ceph-mon[36954]: pgmap v807: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-07-26T13:54:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:54:12 smithi100 ceph-mon[32599]: pgmap v807: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-07-26T13:54:13.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:54:13.783+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:54:13.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:54:13.784+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:54:13.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:54:13.784+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:54:13.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:54:13.784+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:54:13.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:54:13.784+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:54:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:54:14 smithi167 ceph-mon[37140]: pgmap v808: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 24 op/s 2024-07-26T13:54:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:54:14 smithi167 ceph-mon[37140]: osdmap e103: 12 total, 12 up, 12 in 2024-07-26T13:54:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:54:14 smithi138 ceph-mon[36954]: pgmap v808: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 24 op/s 2024-07-26T13:54:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:54:14 smithi138 ceph-mon[36954]: osdmap e103: 12 total, 12 up, 12 in 2024-07-26T13:54:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:54:14 smithi100 ceph-mon[32599]: pgmap v808: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 27 KiB/s wr, 24 op/s 2024-07-26T13:54:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:54:14 smithi100 ceph-mon[32599]: osdmap e103: 12 total, 12 up, 12 in 2024-07-26T13:54:16.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:54:16 smithi167 ceph-mon[37140]: pgmap v810: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T13:54:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:54:16 smithi138 ceph-mon[36954]: pgmap v810: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T13:54:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:54:16 smithi100 ceph-mon[32599]: pgmap v810: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T13:54:18.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:54:18 smithi167 ceph-mon[37140]: pgmap v811: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T13:54:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:54:18 smithi138 ceph-mon[36954]: pgmap v811: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T13:54:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:54:18 smithi100 ceph-mon[32599]: pgmap v811: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T13:54:20.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:54:20 smithi167 ceph-mon[37140]: pgmap v812: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:54:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:54:20 smithi138 ceph-mon[36954]: pgmap v812: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:54:20.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:54:20 smithi100 ceph-mon[32599]: pgmap v812: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:54:22.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:54:22 smithi167 ceph-mon[37140]: pgmap v813: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:54:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:54:22 smithi138 ceph-mon[36954]: pgmap v813: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:54:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:54:22 smithi100 ceph-mon[32599]: pgmap v813: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:54:24.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:54:24 smithi167 ceph-mon[37140]: pgmap v814: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:54:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:54:24 smithi138 ceph-mon[36954]: pgmap v814: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:54:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:54:24 smithi100 ceph-mon[32599]: pgmap v814: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:54:26.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:54:26 smithi167 ceph-mon[37140]: pgmap v815: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T13:54:26.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:54:26 smithi100 ceph-mon[32599]: pgmap v815: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T13:54:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:54:26 smithi138 ceph-mon[36954]: pgmap v815: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T13:54:28.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:54:28.791+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:54:28.793 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:54:28 smithi100 ceph-mon[32599]: pgmap v816: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:54:28.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:54:28.792+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:54:28.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:54:28.792+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:54:28.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:54:28.792+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:54:28.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:54:28.792+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:54:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:54:28 smithi138 ceph-mon[36954]: pgmap v816: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:54:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:54:28 smithi167 ceph-mon[37140]: pgmap v816: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:54:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:54:30 smithi138 ceph-mon[36954]: pgmap v817: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:54:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:54:30 smithi100 ceph-mon[32599]: pgmap v817: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:54:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:54:30 smithi167 ceph-mon[37140]: pgmap v817: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:54:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:54:32 smithi138 ceph-mon[36954]: pgmap v818: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:54:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:54:32 smithi100 ceph-mon[32599]: pgmap v818: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:54:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:54:32 smithi167 ceph-mon[37140]: pgmap v818: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:54:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:54:34 smithi138 ceph-mon[36954]: pgmap v819: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:54:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:54:34 smithi100 ceph-mon[32599]: pgmap v819: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:54:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:54:34 smithi167 ceph-mon[37140]: pgmap v819: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:54:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:54:36 smithi138 ceph-mon[36954]: pgmap v820: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:54:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:54:36 smithi100 ceph-mon[32599]: pgmap v820: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:54:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:54:36 smithi167 ceph-mon[37140]: pgmap v820: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:54:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:54:38 smithi138 ceph-mon[36954]: pgmap v821: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:54:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:54:38 smithi100 ceph-mon[32599]: pgmap v821: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:54:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:54:38 smithi167 ceph-mon[37140]: pgmap v821: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:54:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:54:40 smithi138 ceph-mon[36954]: pgmap v822: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:54:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:54:40 smithi100 ceph-mon[32599]: pgmap v822: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:54:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:54:40 smithi167 ceph-mon[37140]: pgmap v822: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:54:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:54:42 smithi138 ceph-mon[36954]: pgmap v823: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:54:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:54:42 smithi100 ceph-mon[32599]: pgmap v823: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:54:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:54:42 smithi167 ceph-mon[37140]: pgmap v823: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:54:43.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:54:43.808+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:54:43.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:54:43.808+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:54:43.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:54:43.808+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:54:43.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:54:43.808+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:54:43.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:54:43.808+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:54:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:54:44 smithi138 ceph-mon[36954]: pgmap v824: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:54:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:54:44 smithi100 ceph-mon[32599]: pgmap v824: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:54:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:54:44 smithi167 ceph-mon[37140]: pgmap v824: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:54:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:54:46 smithi138 ceph-mon[36954]: pgmap v825: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:54:46.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:54:46 smithi100 ceph-mon[32599]: pgmap v825: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:54:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:54:46 smithi167 ceph-mon[37140]: pgmap v825: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:54:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:54:48 smithi138 ceph-mon[36954]: pgmap v826: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:54:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:54:48 smithi100 ceph-mon[32599]: pgmap v826: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:54:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:54:48 smithi167 ceph-mon[37140]: pgmap v826: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:54:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:54:50 smithi138 ceph-mon[36954]: pgmap v827: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:54:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:54:50 smithi100 ceph-mon[32599]: pgmap v827: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:54:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:54:50 smithi167 ceph-mon[37140]: pgmap v827: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:54:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:54:52 smithi138 ceph-mon[36954]: pgmap v828: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:54:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:54:52 smithi100 ceph-mon[32599]: pgmap v828: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:54:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:54:52 smithi167 ceph-mon[37140]: pgmap v828: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:54:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:54:54 smithi138 ceph-mon[36954]: pgmap v829: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:54:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:54:54 smithi100 ceph-mon[32599]: pgmap v829: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:54:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:54:54 smithi167 ceph-mon[37140]: pgmap v829: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:54:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:54:56 smithi138 ceph-mon[36954]: pgmap v830: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:54:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:54:56 smithi100 ceph-mon[32599]: pgmap v830: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:54:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:54:56 smithi167 ceph-mon[37140]: pgmap v830: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:54:58.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:54:58.822+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:54:58.823 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:54:58 smithi100 ceph-mon[32599]: pgmap v831: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:54:58.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:54:58.822+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:54:58.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:54:58.822+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:54:58.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:54:58.822+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:54:58.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:54:58.822+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:54:58.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:54:58 smithi167 ceph-mon[37140]: pgmap v831: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:54:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:54:58 smithi138 ceph-mon[36954]: pgmap v831: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:55:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:55:00 smithi138 ceph-mon[36954]: pgmap v832: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:55:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:55:00 smithi100 ceph-mon[32599]: pgmap v832: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:55:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:55:00 smithi167 ceph-mon[37140]: pgmap v832: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:55:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:55:02 smithi138 ceph-mon[36954]: pgmap v833: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:55:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:55:02 smithi100 ceph-mon[32599]: pgmap v833: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:55:03.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:55:02 smithi167 ceph-mon[37140]: pgmap v833: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:55:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:55:04 smithi138 ceph-mon[36954]: pgmap v834: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T13:55:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:55:04 smithi100 ceph-mon[32599]: pgmap v834: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T13:55:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:55:04 smithi167 ceph-mon[37140]: pgmap v834: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T13:55:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:55:06 smithi138 ceph-mon[36954]: pgmap v835: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T13:55:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:55:06 smithi100 ceph-mon[32599]: pgmap v835: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T13:55:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:55:06 smithi167 ceph-mon[37140]: pgmap v835: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T13:55:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:55:08 smithi138 ceph-mon[36954]: pgmap v836: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T13:55:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:55:08 smithi138 ceph-mon[36954]: osdmap e104: 12 total, 12 up, 12 in 2024-07-26T13:55:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:55:08 smithi100 ceph-mon[32599]: pgmap v836: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T13:55:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:55:08 smithi100 ceph-mon[32599]: osdmap e104: 12 total, 12 up, 12 in 2024-07-26T13:55:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:55:08 smithi167 ceph-mon[37140]: pgmap v836: 97 pgs: 97 active+clean; 258 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T13:55:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:55:08 smithi167 ceph-mon[37140]: osdmap e104: 12 total, 12 up, 12 in 2024-07-26T13:55:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:55:10 smithi138 ceph-mon[36954]: pgmap v838: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 250 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T13:55:10.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:55:10 smithi100 ceph-mon[32599]: pgmap v838: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 250 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T13:55:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:55:10 smithi167 ceph-mon[37140]: pgmap v838: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 250 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T13:55:11.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:55:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:55:11.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:55:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:55:11.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:55:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:55:11.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:55:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:55:11.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:55:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:55:11.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:55:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:55:11.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:55:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:55:11.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:55:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:55:12.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:55:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:55:12.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:55:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:55:12.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:55:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:55:12.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:55:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:55:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:55:12 smithi138 ceph-mon[36954]: pgmap v839: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 241 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T13:55:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:55:12 smithi100 ceph-mon[32599]: pgmap v839: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 241 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T13:55:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:55:12 smithi167 ceph-mon[37140]: pgmap v839: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 241 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T13:55:13.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:55:13.835+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:55:13.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:55:13.836+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:55:13.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:55:13.836+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:55:13.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:55:13.836+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:55:13.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:55:13.836+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:55:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:55:14 smithi138 ceph-mon[36954]: pgmap v840: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 27 KiB/s wr, 24 op/s 2024-07-26T13:55:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:55:14 smithi138 ceph-mon[36954]: osdmap e105: 12 total, 12 up, 12 in 2024-07-26T13:55:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:55:14 smithi100 ceph-mon[32599]: pgmap v840: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 27 KiB/s wr, 24 op/s 2024-07-26T13:55:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:55:14 smithi100 ceph-mon[32599]: osdmap e105: 12 total, 12 up, 12 in 2024-07-26T13:55:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:55:14 smithi167 ceph-mon[37140]: pgmap v840: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 27 KiB/s wr, 24 op/s 2024-07-26T13:55:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:55:14 smithi167 ceph-mon[37140]: osdmap e105: 12 total, 12 up, 12 in 2024-07-26T13:55:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:55:16 smithi138 ceph-mon[36954]: pgmap v842: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T13:55:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:55:16 smithi100 ceph-mon[32599]: pgmap v842: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T13:55:17.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:55:16 smithi167 ceph-mon[37140]: pgmap v842: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T13:55:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:55:18 smithi138 ceph-mon[36954]: pgmap v843: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 225 B/s wr, 6 op/s 2024-07-26T13:55:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:55:18 smithi100 ceph-mon[32599]: pgmap v843: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 225 B/s wr, 6 op/s 2024-07-26T13:55:19.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:55:18 smithi167 ceph-mon[37140]: pgmap v843: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 225 B/s wr, 6 op/s 2024-07-26T13:55:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:55:20 smithi138 ceph-mon[36954]: pgmap v844: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:55:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:55:20 smithi100 ceph-mon[32599]: pgmap v844: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:55:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:55:20 smithi167 ceph-mon[37140]: pgmap v844: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:55:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:55:22 smithi138 ceph-mon[36954]: pgmap v845: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:55:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:55:22 smithi100 ceph-mon[32599]: pgmap v845: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:55:23.043 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:55:22 smithi167 ceph-mon[37140]: pgmap v845: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:55:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:55:24 smithi138 ceph-mon[36954]: pgmap v846: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:55:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:55:24 smithi100 ceph-mon[32599]: pgmap v846: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:55:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:55:24 smithi167 ceph-mon[37140]: pgmap v846: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:55:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:55:26 smithi138 ceph-mon[36954]: pgmap v847: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T13:55:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:55:26 smithi100 ceph-mon[32599]: pgmap v847: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T13:55:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:55:26 smithi167 ceph-mon[37140]: pgmap v847: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T13:55:28.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:55:28.839+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:55:28.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:55:28.840+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:55:28.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:55:28.840+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:55:28.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:55:28.840+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:55:28.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:55:28.840+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:55:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:55:28 smithi138 ceph-mon[36954]: pgmap v848: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:55:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:55:28 smithi100 ceph-mon[32599]: pgmap v848: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:55:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:55:28 smithi167 ceph-mon[37140]: pgmap v848: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:55:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:55:30 smithi138 ceph-mon[36954]: pgmap v849: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:55:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:55:30 smithi100 ceph-mon[32599]: pgmap v849: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:55:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:55:30 smithi167 ceph-mon[37140]: pgmap v849: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:55:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:55:32 smithi138 ceph-mon[36954]: pgmap v850: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:55:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:55:32 smithi100 ceph-mon[32599]: pgmap v850: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:55:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:55:32 smithi167 ceph-mon[37140]: pgmap v850: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:55:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:55:34 smithi138 ceph-mon[36954]: pgmap v851: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:55:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:55:34 smithi100 ceph-mon[32599]: pgmap v851: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:55:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:55:34 smithi167 ceph-mon[37140]: pgmap v851: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:55:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:55:36 smithi138 ceph-mon[36954]: pgmap v852: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:55:36.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:55:36 smithi100 ceph-mon[32599]: pgmap v852: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:55:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:55:36 smithi167 ceph-mon[37140]: pgmap v852: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:55:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:55:38 smithi138 ceph-mon[36954]: pgmap v853: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:55:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:55:38 smithi100 ceph-mon[32599]: pgmap v853: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:55:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:55:38 smithi167 ceph-mon[37140]: pgmap v853: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:55:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:55:40 smithi138 ceph-mon[36954]: pgmap v854: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:55:40.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:55:40 smithi100 ceph-mon[32599]: pgmap v854: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:55:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:55:40 smithi167 ceph-mon[37140]: pgmap v854: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:55:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:55:42 smithi138 ceph-mon[36954]: pgmap v855: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:55:42.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:55:42 smithi100 ceph-mon[32599]: pgmap v855: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:55:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:55:42 smithi167 ceph-mon[37140]: pgmap v855: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:55:43.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:55:43.855+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:55:43.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:55:43.856+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:55:43.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:55:43.856+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:55:43.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:55:43.856+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:55:43.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:55:43.856+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:55:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:55:44 smithi138 ceph-mon[36954]: pgmap v856: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:55:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:55:44 smithi100 ceph-mon[32599]: pgmap v856: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:55:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:55:44 smithi167 ceph-mon[37140]: pgmap v856: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:55:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:55:46 smithi138 ceph-mon[36954]: pgmap v857: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:55:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:55:46 smithi100 ceph-mon[32599]: pgmap v857: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:55:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:55:46 smithi167 ceph-mon[37140]: pgmap v857: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:55:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:55:48 smithi138 ceph-mon[36954]: pgmap v858: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:55:48.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:55:48 smithi100 ceph-mon[32599]: pgmap v858: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:55:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:55:48 smithi167 ceph-mon[37140]: pgmap v858: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:55:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:55:50 smithi138 ceph-mon[36954]: pgmap v859: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:55:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:55:50 smithi100 ceph-mon[32599]: pgmap v859: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:55:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:55:50 smithi167 ceph-mon[37140]: pgmap v859: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:55:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:55:52 smithi138 ceph-mon[36954]: pgmap v860: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:55:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:55:52 smithi100 ceph-mon[32599]: pgmap v860: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:55:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:55:52 smithi167 ceph-mon[37140]: pgmap v860: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:55:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:55:54 smithi138 ceph-mon[36954]: pgmap v861: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:55:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:55:54 smithi100 ceph-mon[32599]: pgmap v861: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:55:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:55:54 smithi167 ceph-mon[37140]: pgmap v861: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:55:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:55:56 smithi138 ceph-mon[36954]: pgmap v862: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:55:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:55:56 smithi100 ceph-mon[32599]: pgmap v862: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:55:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:55:56 smithi167 ceph-mon[37140]: pgmap v862: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:55:57.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:55:57 smithi100 ceph-mon[32599]: pgmap v863: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:55:58.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:55:57 smithi167 ceph-mon[37140]: pgmap v863: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:55:58.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:55:57 smithi138 ceph-mon[36954]: pgmap v863: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:55:58.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:55:58.870+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:55:58.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:55:58.870+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:55:58.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:55:58.870+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:55:58.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:55:58.870+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:55:58.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:55:58.870+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:55:59.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:55:59 smithi100 ceph-mon[32599]: pgmap v864: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:56:00.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:55:59 smithi167 ceph-mon[37140]: pgmap v864: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:56:00.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:55:59 smithi138 ceph-mon[36954]: pgmap v864: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:56:01.992 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:01 smithi138 ceph-mon[36954]: pgmap v865: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:56:01.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:01 smithi100 ceph-mon[32599]: pgmap v865: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:56:02.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:01 smithi167 ceph-mon[37140]: pgmap v865: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:56:03.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:03 smithi100 ceph-mon[32599]: pgmap v866: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T13:56:04.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:03 smithi167 ceph-mon[37140]: pgmap v866: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T13:56:04.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:03 smithi138 ceph-mon[36954]: pgmap v866: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T13:56:05.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:05 smithi100 ceph-mon[32599]: pgmap v867: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T13:56:06.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:05 smithi167 ceph-mon[37140]: pgmap v867: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T13:56:06.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:05 smithi138 ceph-mon[36954]: pgmap v867: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T13:56:07.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:07 smithi100 ceph-mon[32599]: pgmap v868: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T13:56:08.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:07 smithi167 ceph-mon[37140]: pgmap v868: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T13:56:08.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:07 smithi138 ceph-mon[36954]: pgmap v868: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T13:56:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:08 smithi167 ceph-mon[37140]: osdmap e106: 12 total, 12 up, 12 in 2024-07-26T13:56:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:08 smithi138 ceph-mon[36954]: osdmap e106: 12 total, 12 up, 12 in 2024-07-26T13:56:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:08 smithi100 ceph-mon[32599]: osdmap e106: 12 total, 12 up, 12 in 2024-07-26T13:56:10.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:09 smithi167 ceph-mon[37140]: pgmap v870: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T13:56:10.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:09 smithi138 ceph-mon[36954]: pgmap v870: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T13:56:10.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:09 smithi100 ceph-mon[32599]: pgmap v870: 97 pgs: 97 active+clean; 235 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T13:56:12.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:11 smithi167 ceph-mon[37140]: pgmap v871: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T13:56:12.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:56:12.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:56:12.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:56:12.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:56:12.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:56:12.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:56:12.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:56:12.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:56:12.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:56:12.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:56:12.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:56:12.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:56:12.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:56:12.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:11 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T13:56:12.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:11 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T13:56:12.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:11 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T13:56:12.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:11 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T13:56:12.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:11 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T13:56:12.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:11 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T13:56:12.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:56:12.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:56:12.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:56:12.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:11 smithi138 ceph-mon[36954]: pgmap v871: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T13:56:12.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:56:12.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:56:12.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:56:12.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:56:12.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:56:12.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:56:12.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:56:12.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:56:12.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:56:12.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:56:12.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:56:12.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:56:12.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:56:12.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:11 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T13:56:12.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:11 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T13:56:12.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:11 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T13:56:12.193 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:11 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T13:56:12.193 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:11 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T13:56:12.193 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:11 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T13:56:12.193 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:56:12.193 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:56:12.193 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:56:12.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:11 smithi100 ceph-mon[32599]: pgmap v871: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T13:56:12.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:56:12.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:56:12.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:56:12.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:56:12.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:56:12.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:56:12.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:56:12.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:56:12.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:56:12.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:56:12.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:56:12.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:56:12.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T13:56:12.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:11 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T13:56:12.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:11 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T13:56:12.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:11 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T13:56:12.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:11 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T13:56:12.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:11 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T13:56:12.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:11 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T13:56:12.247 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:56:12.247 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:56:12.247 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:56:13.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:56:13.880+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:56:13.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:56:13.880+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:56:13.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:56:13.880+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:56:13.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:56:13.880+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:56:13.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:56:13.881+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:56:14.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:13 smithi167 ceph-mon[37140]: pgmap v872: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 27 KiB/s wr, 24 op/s 2024-07-26T13:56:14.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:13 smithi138 ceph-mon[36954]: pgmap v872: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 27 KiB/s wr, 24 op/s 2024-07-26T13:56:14.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:13 smithi100 ceph-mon[32599]: pgmap v872: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 27 KiB/s wr, 24 op/s 2024-07-26T13:56:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:14 smithi138 ceph-mon[36954]: osdmap e107: 12 total, 12 up, 12 in 2024-07-26T13:56:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:14 smithi100 ceph-mon[32599]: osdmap e107: 12 total, 12 up, 12 in 2024-07-26T13:56:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:14 smithi167 ceph-mon[37140]: osdmap e107: 12 total, 12 up, 12 in 2024-07-26T13:56:16.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:15 smithi138 ceph-mon[36954]: pgmap v874: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T13:56:16.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:15 smithi100 ceph-mon[32599]: pgmap v874: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T13:56:16.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:15 smithi167 ceph-mon[37140]: pgmap v874: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T13:56:18.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:56:18.084+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c005b50 con 0x7fe5a81087a0 2024-07-26T13:56:18.087 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:17 smithi100 ceph-mon[32599]: pgmap v875: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 228 B/s wr, 6 op/s 2024-07-26T13:56:18.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:56:18.085+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 8 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe57c005b50 con 0x7fe5a81087a0 2024-07-26T13:56:18.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:56:18.089+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580005b50 con 0x55c0ec887390 2024-07-26T13:56:18.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:56:18.089+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 8 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe580005b50 con 0x55c0ec887390 2024-07-26T13:56:18.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:17 smithi138 ceph-mon[36954]: pgmap v875: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 228 B/s wr, 6 op/s 2024-07-26T13:56:18.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:17 smithi167 ceph-mon[37140]: pgmap v875: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 228 B/s wr, 6 op/s 2024-07-26T13:56:20.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:19 smithi138 ceph-mon[36954]: pgmap v876: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:56:20.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:19 smithi100 ceph-mon[32599]: pgmap v876: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:56:20.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:19 smithi167 ceph-mon[37140]: pgmap v876: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:56:22.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:21 smithi138 ceph-mon[36954]: pgmap v877: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:56:22.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:21 smithi100 ceph-mon[32599]: pgmap v877: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:56:22.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:21 smithi167 ceph-mon[37140]: pgmap v877: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:56:24.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:23 smithi138 ceph-mon[36954]: pgmap v878: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:56:24.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:23 smithi100 ceph-mon[32599]: pgmap v878: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:56:24.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:23 smithi167 ceph-mon[37140]: pgmap v878: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:56:26.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:25 smithi138 ceph-mon[36954]: pgmap v879: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T13:56:26.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:25 smithi100 ceph-mon[32599]: pgmap v879: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T13:56:26.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:25 smithi167 ceph-mon[37140]: pgmap v879: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T13:56:28.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:27 smithi138 ceph-mon[36954]: pgmap v880: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:56:28.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:27 smithi100 ceph-mon[32599]: pgmap v880: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:56:28.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:27 smithi167 ceph-mon[37140]: pgmap v880: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:56:28.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:56:28.896+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:56:28.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:56:28.897+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:56:28.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:56:28.897+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:56:28.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:56:28.897+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:56:28.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:56:28.897+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:56:30.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:29 smithi138 ceph-mon[36954]: pgmap v881: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:56:30.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:29 smithi100 ceph-mon[32599]: pgmap v881: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:56:30.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:29 smithi167 ceph-mon[37140]: pgmap v881: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:56:32.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:31 smithi138 ceph-mon[36954]: pgmap v882: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:56:32.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:31 smithi100 ceph-mon[32599]: pgmap v882: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:56:32.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:31 smithi167 ceph-mon[37140]: pgmap v882: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:56:34.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:33 smithi138 ceph-mon[36954]: pgmap v883: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:56:34.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:33 smithi100 ceph-mon[32599]: pgmap v883: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:56:34.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:33 smithi167 ceph-mon[37140]: pgmap v883: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:56:36.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:35 smithi138 ceph-mon[36954]: pgmap v884: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:56:36.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:35 smithi100 ceph-mon[32599]: pgmap v884: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:56:36.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:35 smithi167 ceph-mon[37140]: pgmap v884: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:56:38.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:37 smithi138 ceph-mon[36954]: pgmap v885: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:56:38.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:37 smithi100 ceph-mon[32599]: pgmap v885: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:56:38.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:37 smithi167 ceph-mon[37140]: pgmap v885: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:56:40.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:39 smithi138 ceph-mon[36954]: pgmap v886: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:56:40.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:39 smithi100 ceph-mon[32599]: pgmap v886: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:56:40.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:39 smithi167 ceph-mon[37140]: pgmap v886: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:56:42.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:41 smithi138 ceph-mon[36954]: pgmap v887: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:56:42.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:41 smithi100 ceph-mon[32599]: pgmap v887: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:56:42.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:41 smithi167 ceph-mon[37140]: pgmap v887: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:56:43.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:56:43.912+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:56:43.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:56:43.912+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:56:43.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:56:43.912+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:56:43.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:56:43.912+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:56:43.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:56:43.912+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:56:44.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:43 smithi138 ceph-mon[36954]: pgmap v888: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:56:44.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:43 smithi100 ceph-mon[32599]: pgmap v888: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:56:44.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:43 smithi167 ceph-mon[37140]: pgmap v888: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:56:46.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:45 smithi138 ceph-mon[36954]: pgmap v889: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:56:46.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:45 smithi100 ceph-mon[32599]: pgmap v889: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:56:46.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:45 smithi167 ceph-mon[37140]: pgmap v889: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:56:48.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:47 smithi138 ceph-mon[36954]: pgmap v890: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:56:48.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:47 smithi100 ceph-mon[32599]: pgmap v890: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:56:48.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:47 smithi167 ceph-mon[37140]: pgmap v890: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:56:50.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:49 smithi138 ceph-mon[36954]: pgmap v891: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:56:50.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:49 smithi100 ceph-mon[32599]: pgmap v891: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:56:50.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:49 smithi167 ceph-mon[37140]: pgmap v891: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:56:52.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:51 smithi138 ceph-mon[36954]: pgmap v892: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:56:52.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:51 smithi100 ceph-mon[32599]: pgmap v892: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:56:52.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:51 smithi167 ceph-mon[37140]: pgmap v892: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:56:54.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:53 smithi138 ceph-mon[36954]: pgmap v893: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:56:54.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:53 smithi100 ceph-mon[32599]: pgmap v893: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:56:54.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:53 smithi167 ceph-mon[37140]: pgmap v893: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:56:56.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:55 smithi138 ceph-mon[36954]: pgmap v894: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:56:56.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:55 smithi100 ceph-mon[32599]: pgmap v894: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:56:56.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:55 smithi167 ceph-mon[37140]: pgmap v894: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:56:58.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:57 smithi138 ceph-mon[36954]: pgmap v895: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:56:58.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:57 smithi100 ceph-mon[32599]: pgmap v895: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:56:58.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:57 smithi167 ceph-mon[37140]: pgmap v895: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:56:58.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:56:58.928+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:56:58.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:56:58.929+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:56:58.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:56:58.929+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:56:58.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:56:58.929+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:56:58.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:56:58.929+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:57:00.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:56:59 smithi138 ceph-mon[36954]: pgmap v896: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:57:00.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:56:59 smithi100 ceph-mon[32599]: pgmap v896: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:57:00.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:56:59 smithi167 ceph-mon[37140]: pgmap v896: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:57:02.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:57:01 smithi138 ceph-mon[36954]: pgmap v897: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:57:02.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:57:01 smithi100 ceph-mon[32599]: pgmap v897: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:57:02.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:57:01 smithi167 ceph-mon[37140]: pgmap v897: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:57:03.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:57:03.091+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c005eb0 con 0x7fe5a81087a0 2024-07-26T13:57:03.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:57:03.091+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 9 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c005eb0 con 0x7fe5a81087a0 2024-07-26T13:57:03.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:57:03.095+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580005e30 con 0x55c0ec887390 2024-07-26T13:57:03.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:57:03.095+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 9 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580005e30 con 0x55c0ec887390 2024-07-26T13:57:04.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:57:03 smithi138 ceph-mon[36954]: pgmap v898: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T13:57:04.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:57:03 smithi100 ceph-mon[32599]: pgmap v898: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T13:57:04.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:57:03 smithi167 ceph-mon[37140]: pgmap v898: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T13:57:06.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:57:05 smithi138 ceph-mon[36954]: pgmap v899: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T13:57:06.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:57:05 smithi100 ceph-mon[32599]: pgmap v899: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T13:57:06.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:57:05 smithi167 ceph-mon[37140]: pgmap v899: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T13:57:08.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:57:07 smithi138 ceph-mon[36954]: pgmap v900: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T13:57:08.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:57:07 smithi138 ceph-mon[36954]: osdmap e108: 12 total, 12 up, 12 in 2024-07-26T13:57:08.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:57:07 smithi100 ceph-mon[32599]: pgmap v900: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T13:57:08.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:57:07 smithi100 ceph-mon[32599]: osdmap e108: 12 total, 12 up, 12 in 2024-07-26T13:57:08.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:57:07 smithi167 ceph-mon[37140]: pgmap v900: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T13:57:08.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:57:07 smithi167 ceph-mon[37140]: osdmap e108: 12 total, 12 up, 12 in 2024-07-26T13:57:10.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:57:09 smithi138 ceph-mon[36954]: pgmap v902: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T13:57:10.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:57:09 smithi100 ceph-mon[32599]: pgmap v902: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T13:57:10.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:57:09 smithi167 ceph-mon[37140]: pgmap v902: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T13:57:12.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:57:11 smithi138 ceph-mon[36954]: pgmap v903: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T13:57:12.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:57:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:57:12.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:57:11 smithi100 ceph-mon[32599]: pgmap v903: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T13:57:12.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:57:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:57:12.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:57:11 smithi167 ceph-mon[37140]: pgmap v903: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T13:57:12.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:57:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:57:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:57:12 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:57:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:57:12 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:57:13.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:57:12 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:57:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:57:12 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:57:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:57:12 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:57:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:57:12 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:57:13.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:57:12 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:57:13.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:57:12 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:57:13.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:57:12 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:57:13.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:57:13.931+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:57:13.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:57:13.932+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:57:13.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:57:13.932+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:57:13.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:57:13.932+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:57:13.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:57:13.932+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:57:14.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:57:14 smithi100 ceph-mon[32599]: pgmap v904: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T13:57:14.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:57:14 smithi100 ceph-mon[32599]: osdmap e109: 12 total, 12 up, 12 in 2024-07-26T13:57:14.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:57:14 smithi138 ceph-mon[36954]: pgmap v904: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T13:57:14.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:57:14 smithi138 ceph-mon[36954]: osdmap e109: 12 total, 12 up, 12 in 2024-07-26T13:57:14.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:57:14 smithi167 ceph-mon[37140]: pgmap v904: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T13:57:14.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:57:14 smithi167 ceph-mon[37140]: osdmap e109: 12 total, 12 up, 12 in 2024-07-26T13:57:16.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:57:16 smithi138 ceph-mon[36954]: pgmap v906: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T13:57:16.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:57:16 smithi100 ceph-mon[32599]: pgmap v906: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T13:57:16.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:57:16 smithi167 ceph-mon[37140]: pgmap v906: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T13:57:18.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:57:18 smithi138 ceph-mon[36954]: pgmap v907: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 208 B/s wr, 6 op/s 2024-07-26T13:57:18.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:57:18 smithi100 ceph-mon[32599]: pgmap v907: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 208 B/s wr, 6 op/s 2024-07-26T13:57:18.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:57:18 smithi167 ceph-mon[37140]: pgmap v907: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 208 B/s wr, 6 op/s 2024-07-26T13:57:20.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:57:20 smithi138 ceph-mon[36954]: pgmap v908: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:57:20.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:57:20 smithi100 ceph-mon[32599]: pgmap v908: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:57:20.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:57:20 smithi167 ceph-mon[37140]: pgmap v908: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:57:22.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:57:22 smithi138 ceph-mon[36954]: pgmap v909: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:57:22.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:57:22 smithi100 ceph-mon[32599]: pgmap v909: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:57:22.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:57:22 smithi167 ceph-mon[37140]: pgmap v909: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:57:24.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:57:24 smithi138 ceph-mon[36954]: pgmap v910: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:57:24.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:57:24 smithi100 ceph-mon[32599]: pgmap v910: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:57:24.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:57:24 smithi167 ceph-mon[37140]: pgmap v910: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:57:26.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:57:26 smithi138 ceph-mon[36954]: pgmap v911: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T13:57:26.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:57:26 smithi100 ceph-mon[32599]: pgmap v911: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T13:57:26.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:57:26 smithi167 ceph-mon[37140]: pgmap v911: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T13:57:28.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:57:28 smithi138 ceph-mon[36954]: pgmap v912: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:57:28.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:57:28 smithi100 ceph-mon[32599]: pgmap v912: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:57:28.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:57:28 smithi167 ceph-mon[37140]: pgmap v912: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:57:28.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:57:28.947+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:57:28.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:57:28.948+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:57:28.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:57:28.948+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:57:28.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:57:28.948+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:57:28.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:57:28.948+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:57:30.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:57:30 smithi138 ceph-mon[36954]: pgmap v913: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:57:30.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:57:30 smithi100 ceph-mon[32599]: pgmap v913: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:57:30.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:57:30 smithi167 ceph-mon[37140]: pgmap v913: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:57:32.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:57:32 smithi138 ceph-mon[36954]: pgmap v914: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:57:32.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:57:32 smithi100 ceph-mon[32599]: pgmap v914: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:57:32.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:57:32 smithi167 ceph-mon[37140]: pgmap v914: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:57:34.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:57:34 smithi138 ceph-mon[36954]: pgmap v915: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:57:34.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:57:34 smithi100 ceph-mon[32599]: pgmap v915: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:57:34.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:57:34 smithi167 ceph-mon[37140]: pgmap v915: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:57:36.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:57:36 smithi100 ceph-mon[32599]: pgmap v916: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:57:36.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:57:36 smithi167 ceph-mon[37140]: pgmap v916: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:57:36.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:57:36 smithi138 ceph-mon[36954]: pgmap v916: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:57:38.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:57:38 smithi100 ceph-mon[32599]: pgmap v917: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:57:38.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:57:38 smithi167 ceph-mon[37140]: pgmap v917: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:57:38.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:57:38 smithi138 ceph-mon[36954]: pgmap v917: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:57:40.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:57:40 smithi100 ceph-mon[32599]: pgmap v918: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:57:40.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:57:40 smithi167 ceph-mon[37140]: pgmap v918: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:57:40.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:57:40 smithi138 ceph-mon[36954]: pgmap v918: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:57:42.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:57:42 smithi100 ceph-mon[32599]: pgmap v919: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:57:42.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:57:42 smithi167 ceph-mon[37140]: pgmap v919: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:57:42.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:57:42 smithi138 ceph-mon[36954]: pgmap v919: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:57:43.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:57:43.963+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:57:43.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:57:43.964+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:57:43.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:57:43.964+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:57:43.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:57:43.964+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:57:43.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:57:43.964+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:57:44.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:57:44 smithi100 ceph-mon[32599]: pgmap v920: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:57:44.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:57:44 smithi167 ceph-mon[37140]: pgmap v920: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:57:44.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:57:44 smithi138 ceph-mon[36954]: pgmap v920: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:57:46.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:57:46 smithi100 ceph-mon[32599]: pgmap v921: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:57:46.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:57:46 smithi167 ceph-mon[37140]: pgmap v921: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:57:46.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:57:46 smithi138 ceph-mon[36954]: pgmap v921: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:57:48.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:57:48 smithi100 ceph-mon[32599]: pgmap v922: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:57:48.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:57:48 smithi167 ceph-mon[37140]: pgmap v922: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:57:48.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:57:48 smithi138 ceph-mon[36954]: pgmap v922: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:57:50.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:57:50 smithi100 ceph-mon[32599]: pgmap v923: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:57:50.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:57:50 smithi167 ceph-mon[37140]: pgmap v923: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:57:50.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:57:50 smithi138 ceph-mon[36954]: pgmap v923: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:57:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:57:52 smithi100 ceph-mon[32599]: pgmap v924: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:57:52.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:57:52 smithi167 ceph-mon[37140]: pgmap v924: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:57:52.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:57:52 smithi138 ceph-mon[36954]: pgmap v924: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:57:54.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:57:54 smithi100 ceph-mon[32599]: pgmap v925: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:57:54.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:57:54 smithi167 ceph-mon[37140]: pgmap v925: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:57:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:57:54 smithi138 ceph-mon[36954]: pgmap v925: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:57:56.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:57:56 smithi100 ceph-mon[32599]: pgmap v926: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:57:56.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:57:56 smithi167 ceph-mon[37140]: pgmap v926: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:57:56.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:57:56 smithi138 ceph-mon[36954]: pgmap v926: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:57:58.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:57:58 smithi100 ceph-mon[32599]: pgmap v927: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:57:58.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:57:58 smithi167 ceph-mon[37140]: pgmap v927: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:57:58.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:57:58 smithi138 ceph-mon[36954]: pgmap v927: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:57:58.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:57:58.980+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:57:58.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:57:58.980+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:57:58.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:57:58.980+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:57:58.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:57:58.980+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:57:58.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:57:58.980+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:58:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:58:00 smithi100 ceph-mon[32599]: pgmap v928: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:58:00.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:58:00 smithi167 ceph-mon[37140]: pgmap v928: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:58:00.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:58:00 smithi138 ceph-mon[36954]: pgmap v928: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:58:02.493 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:58:02 smithi138 ceph-mon[36954]: pgmap v929: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:58:02.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:58:02 smithi100 ceph-mon[32599]: pgmap v929: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:58:02.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:58:02 smithi167 ceph-mon[37140]: pgmap v929: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:58:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:58:04 smithi100 ceph-mon[32599]: pgmap v930: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T13:58:04.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:58:04 smithi167 ceph-mon[37140]: pgmap v930: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T13:58:04.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:58:04 smithi138 ceph-mon[36954]: pgmap v930: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T13:58:06.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:58:06 smithi100 ceph-mon[32599]: pgmap v931: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T13:58:06.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:58:06 smithi167 ceph-mon[37140]: pgmap v931: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T13:58:06.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:58:06 smithi138 ceph-mon[36954]: pgmap v931: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T13:58:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:58:08 smithi167 ceph-mon[37140]: pgmap v932: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T13:58:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:58:08 smithi167 ceph-mon[37140]: osdmap e110: 12 total, 12 up, 12 in 2024-07-26T13:58:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:58:08 smithi138 ceph-mon[36954]: pgmap v932: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T13:58:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:58:08 smithi138 ceph-mon[36954]: osdmap e110: 12 total, 12 up, 12 in 2024-07-26T13:58:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:58:08 smithi100 ceph-mon[32599]: pgmap v932: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T13:58:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:58:08 smithi100 ceph-mon[32599]: osdmap e110: 12 total, 12 up, 12 in 2024-07-26T13:58:10.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:58:10 smithi167 ceph-mon[37140]: pgmap v934: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T13:58:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:58:10 smithi138 ceph-mon[36954]: pgmap v934: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T13:58:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:58:10 smithi100 ceph-mon[32599]: pgmap v934: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T13:58:12.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:58:12 smithi167 ceph-mon[37140]: pgmap v935: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T13:58:12.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:58:12 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:58:12.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:58:12 smithi138 ceph-mon[36954]: pgmap v935: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T13:58:12.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:58:12 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:58:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:58:12 smithi100 ceph-mon[32599]: pgmap v935: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T13:58:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:58:12 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:58:13.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:58:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:58:13.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:58:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:58:13.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:58:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:58:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:58:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:58:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:58:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:58:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:58:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:58:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:58:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:58:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:58:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:58:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:58:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:58:13.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:58:13.996+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:58:13.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:58:13.996+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:58:13.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:58:13.996+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:58:13.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:58:13.996+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:58:13.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:58:13.996+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:58:14.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:58:14 smithi167 ceph-mon[37140]: pgmap v936: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T13:58:14.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:58:14 smithi167 ceph-mon[37140]: osdmap e111: 12 total, 12 up, 12 in 2024-07-26T13:58:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:58:14 smithi138 ceph-mon[36954]: pgmap v936: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T13:58:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:58:14 smithi138 ceph-mon[36954]: osdmap e111: 12 total, 12 up, 12 in 2024-07-26T13:58:14.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:58:14 smithi100 ceph-mon[32599]: pgmap v936: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T13:58:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:58:14 smithi100 ceph-mon[32599]: osdmap e111: 12 total, 12 up, 12 in 2024-07-26T13:58:16.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:58:16 smithi167 ceph-mon[37140]: pgmap v938: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T13:58:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:58:16 smithi138 ceph-mon[36954]: pgmap v938: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T13:58:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:58:16 smithi100 ceph-mon[32599]: pgmap v938: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T13:58:18.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:58:18 smithi167 ceph-mon[37140]: pgmap v939: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 215 B/s wr, 6 op/s 2024-07-26T13:58:18.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:58:18 smithi138 ceph-mon[36954]: pgmap v939: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 215 B/s wr, 6 op/s 2024-07-26T13:58:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:58:18 smithi100 ceph-mon[32599]: pgmap v939: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 215 B/s wr, 6 op/s 2024-07-26T13:58:20.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:58:20 smithi167 ceph-mon[37140]: pgmap v940: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:58:20.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:58:20 smithi138 ceph-mon[36954]: pgmap v940: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:58:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:58:20 smithi100 ceph-mon[32599]: pgmap v940: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:58:22.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:58:22 smithi167 ceph-mon[37140]: pgmap v941: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:58:22.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:58:22 smithi138 ceph-mon[36954]: pgmap v941: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:58:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:58:22 smithi100 ceph-mon[32599]: pgmap v941: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:58:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:58:24 smithi138 ceph-mon[36954]: pgmap v942: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:58:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:58:24 smithi100 ceph-mon[32599]: pgmap v942: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:58:24.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:58:24 smithi167 ceph-mon[37140]: pgmap v942: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:58:26.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:58:26 smithi100 ceph-mon[32599]: pgmap v943: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T13:58:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:58:26 smithi138 ceph-mon[36954]: pgmap v943: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T13:58:26.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:58:26 smithi167 ceph-mon[37140]: pgmap v943: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T13:58:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:58:28 smithi138 ceph-mon[36954]: pgmap v944: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:58:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:58:28 smithi100 ceph-mon[32599]: pgmap v944: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:58:28.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:58:28 smithi167 ceph-mon[37140]: pgmap v944: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:58:29.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:58:29.012+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:58:29.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:58:29.013+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:58:29.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:58:29.013+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:58:29.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:58:29.013+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:58:29.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:58:29.013+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:58:30.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:58:30 smithi167 ceph-mon[37140]: pgmap v945: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T13:58:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:58:30 smithi138 ceph-mon[36954]: pgmap v945: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T13:58:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:58:30 smithi100 ceph-mon[32599]: pgmap v945: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T13:58:32.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:58:32 smithi138 ceph-mon[36954]: pgmap v946: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:58:32.755 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:58:32 smithi100 ceph-mon[32599]: pgmap v946: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:58:32.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:58:32 smithi167 ceph-mon[37140]: pgmap v946: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:58:34.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:58:34 smithi138 ceph-mon[36954]: pgmap v947: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:58:34.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:58:34 smithi100 ceph-mon[32599]: pgmap v947: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:58:34.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:58:34 smithi167 ceph-mon[37140]: pgmap v947: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:58:36.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:58:36 smithi138 ceph-mon[36954]: pgmap v948: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:58:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:58:36 smithi100 ceph-mon[32599]: pgmap v948: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:58:36.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:58:36 smithi167 ceph-mon[37140]: pgmap v948: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:58:38.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:58:38 smithi138 ceph-mon[36954]: pgmap v949: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:58:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:58:38 smithi100 ceph-mon[32599]: pgmap v949: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:58:38.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:58:38 smithi167 ceph-mon[37140]: pgmap v949: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:58:40.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:58:40 smithi138 ceph-mon[36954]: pgmap v950: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:58:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:58:40 smithi100 ceph-mon[32599]: pgmap v950: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:58:40.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:58:40 smithi167 ceph-mon[37140]: pgmap v950: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:58:42.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:58:42 smithi138 ceph-mon[36954]: pgmap v951: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:58:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:58:42 smithi100 ceph-mon[32599]: pgmap v951: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:58:42.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:58:42 smithi167 ceph-mon[37140]: pgmap v951: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:58:44.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:58:44.028+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:58:44.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:58:44.029+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:58:44.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:58:44.029+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:58:44.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:58:44.029+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:58:44.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:58:44.029+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:58:44.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:58:44 smithi138 ceph-mon[36954]: pgmap v952: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:58:44.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:58:44 smithi100 ceph-mon[32599]: pgmap v952: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:58:44.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:58:44 smithi167 ceph-mon[37140]: pgmap v952: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:58:46.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:58:46 smithi138 ceph-mon[36954]: pgmap v953: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:58:46.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:58:46 smithi100 ceph-mon[32599]: pgmap v953: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:58:46.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:58:46 smithi167 ceph-mon[37140]: pgmap v953: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:58:48.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:58:48 smithi138 ceph-mon[36954]: pgmap v954: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:58:48.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:58:48 smithi100 ceph-mon[32599]: pgmap v954: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:58:48.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:58:48 smithi167 ceph-mon[37140]: pgmap v954: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:58:50.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:58:50 smithi138 ceph-mon[36954]: pgmap v955: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:58:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:58:50 smithi100 ceph-mon[32599]: pgmap v955: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:58:50.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:58:50 smithi167 ceph-mon[37140]: pgmap v955: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:58:52.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:58:52 smithi138 ceph-mon[36954]: pgmap v956: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:58:52.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:58:52 smithi100 ceph-mon[32599]: pgmap v956: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:58:52.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:58:52 smithi167 ceph-mon[37140]: pgmap v956: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:58:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:58:54 smithi138 ceph-mon[36954]: pgmap v957: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:58:54.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:58:54 smithi100 ceph-mon[32599]: pgmap v957: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:58:54.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:58:54 smithi167 ceph-mon[37140]: pgmap v957: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:58:56.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:58:56 smithi138 ceph-mon[36954]: pgmap v958: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:58:56.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:58:56 smithi100 ceph-mon[32599]: pgmap v958: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:58:56.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:58:56 smithi167 ceph-mon[37140]: pgmap v958: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:58:58.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:58:58 smithi138 ceph-mon[36954]: pgmap v959: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:58:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:58:58 smithi100 ceph-mon[32599]: pgmap v959: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:58:58.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:58:58 smithi167 ceph-mon[37140]: pgmap v959: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:58:59.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:58:59.036+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:58:59.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:58:59.037+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:58:59.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:58:59.037+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:58:59.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:58:59.037+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:58:59.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:58:59.037+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:59:00.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:00 smithi138 ceph-mon[36954]: pgmap v960: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:59:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:00 smithi100 ceph-mon[32599]: pgmap v960: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:59:00.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:00 smithi167 ceph-mon[37140]: pgmap v960: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:59:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:02 smithi138 ceph-mon[36954]: pgmap v961: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:59:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:02 smithi100 ceph-mon[32599]: pgmap v961: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:59:02.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:02 smithi167 ceph-mon[37140]: pgmap v961: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:59:04.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:04 smithi138 ceph-mon[36954]: pgmap v962: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T13:59:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:04 smithi100 ceph-mon[32599]: pgmap v962: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T13:59:04.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:04 smithi167 ceph-mon[37140]: pgmap v962: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T13:59:06.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:06 smithi138 ceph-mon[36954]: pgmap v963: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T13:59:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:06 smithi100 ceph-mon[32599]: pgmap v963: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T13:59:06.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:06 smithi167 ceph-mon[37140]: pgmap v963: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T13:59:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:08 smithi138 ceph-mon[36954]: pgmap v964: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T13:59:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:08 smithi138 ceph-mon[36954]: osdmap e112: 12 total, 12 up, 12 in 2024-07-26T13:59:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:08 smithi100 ceph-mon[32599]: pgmap v964: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T13:59:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:08 smithi100 ceph-mon[32599]: osdmap e112: 12 total, 12 up, 12 in 2024-07-26T13:59:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:08 smithi167 ceph-mon[37140]: pgmap v964: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T13:59:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:08 smithi167 ceph-mon[37140]: osdmap e112: 12 total, 12 up, 12 in 2024-07-26T13:59:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:10 smithi138 ceph-mon[36954]: pgmap v966: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T13:59:10.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:10 smithi100 ceph-mon[32599]: pgmap v966: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T13:59:10.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:10 smithi167 ceph-mon[37140]: pgmap v966: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T13:59:12.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:12 smithi138 ceph-mon[36954]: pgmap v967: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T13:59:12.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:12 smithi100 ceph-mon[32599]: pgmap v967: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T13:59:12.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:12 smithi167 ceph-mon[37140]: pgmap v967: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T13:59:13.621 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:59:13.629 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:59:13.662 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T13:59:14.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:59:14.052+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:59:14.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:59:14.052+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:59:14.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:59:14.052+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:59:14.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:59:14.052+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:59:14.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:59:14.052+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:59:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:14 smithi138 ceph-mon[36954]: pgmap v968: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T13:59:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:14 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:59:14.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:14 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:59:14.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:14 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:59:14.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:14 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:59:14.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:14 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:59:14.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:14 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:59:14.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:14 smithi100 ceph-mon[32599]: pgmap v968: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T13:59:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:14 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:59:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:14 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:59:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:14 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:59:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:14 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:59:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:14 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:59:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:14 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:59:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:14 smithi167 ceph-mon[37140]: pgmap v968: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T13:59:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:14 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:59:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:14 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:59:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:14 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:59:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:14 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:59:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:14 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:59:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:14 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:59:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:15 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:59:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:15 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:59:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:15 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:59:15.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:15 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:59:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:15 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:59:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:15 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:59:15.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:15 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T13:59:15.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:15 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T13:59:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:15 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T13:59:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:16 smithi100 ceph-mon[32599]: pgmap v969: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T13:59:17.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:16 smithi167 ceph-mon[37140]: pgmap v969: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T13:59:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:16 smithi138 ceph-mon[36954]: pgmap v969: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T13:59:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:18 smithi100 ceph-mon[32599]: pgmap v970: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T13:59:19.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:18 smithi167 ceph-mon[37140]: pgmap v970: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T13:59:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:18 smithi138 ceph-mon[36954]: pgmap v970: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T13:59:20.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:19 smithi138 ceph-mon[36954]: pgmap v971: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 179 B/s wr, 7 op/s 2024-07-26T13:59:20.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:19 smithi138 ceph-mon[36954]: osdmap e113: 12 total, 12 up, 12 in 2024-07-26T13:59:20.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:19 smithi100 ceph-mon[32599]: pgmap v971: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 179 B/s wr, 7 op/s 2024-07-26T13:59:20.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:19 smithi100 ceph-mon[32599]: osdmap e113: 12 total, 12 up, 12 in 2024-07-26T13:59:20.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:19 smithi167 ceph-mon[37140]: pgmap v971: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 179 B/s wr, 7 op/s 2024-07-26T13:59:20.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:19 smithi167 ceph-mon[37140]: osdmap e113: 12 total, 12 up, 12 in 2024-07-26T13:59:22.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:21 smithi138 ceph-mon[36954]: pgmap v973: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:59:22.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:21 smithi100 ceph-mon[32599]: pgmap v973: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:59:22.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:21 smithi167 ceph-mon[37140]: pgmap v973: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:59:24.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:23 smithi138 ceph-mon[36954]: pgmap v974: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:59:24.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:23 smithi100 ceph-mon[32599]: pgmap v974: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:59:24.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:23 smithi167 ceph-mon[37140]: pgmap v974: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T13:59:26.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:25 smithi138 ceph-mon[36954]: pgmap v975: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T13:59:26.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:25 smithi100 ceph-mon[32599]: pgmap v975: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T13:59:26.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:25 smithi167 ceph-mon[37140]: pgmap v975: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T13:59:28.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:27 smithi138 ceph-mon[36954]: pgmap v976: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T13:59:28.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:27 smithi100 ceph-mon[32599]: pgmap v976: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T13:59:28.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:27 smithi167 ceph-mon[37140]: pgmap v976: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T13:59:29.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:59:29.068+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:59:29.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:59:29.068+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:59:29.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:59:29.068+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:59:29.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:59:29.068+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:59:29.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:59:29.069+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:59:30.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:29 smithi138 ceph-mon[36954]: pgmap v977: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T13:59:30.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:29 smithi100 ceph-mon[32599]: pgmap v977: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T13:59:30.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:29 smithi167 ceph-mon[37140]: pgmap v977: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T13:59:32.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:31 smithi138 ceph-mon[36954]: pgmap v978: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:59:32.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:31 smithi100 ceph-mon[32599]: pgmap v978: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:59:32.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:31 smithi167 ceph-mon[37140]: pgmap v978: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:59:34.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:33 smithi138 ceph-mon[36954]: pgmap v979: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:59:34.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:33 smithi100 ceph-mon[32599]: pgmap v979: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:59:34.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:33 smithi167 ceph-mon[37140]: pgmap v979: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:59:36.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:35 smithi138 ceph-mon[36954]: pgmap v980: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:59:36.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:35 smithi100 ceph-mon[32599]: pgmap v980: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:59:36.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:35 smithi167 ceph-mon[37140]: pgmap v980: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:59:38.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:37 smithi138 ceph-mon[36954]: pgmap v981: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:59:38.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:37 smithi100 ceph-mon[32599]: pgmap v981: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:59:38.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:37 smithi167 ceph-mon[37140]: pgmap v981: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:59:40.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:39 smithi138 ceph-mon[36954]: pgmap v982: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:59:40.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:39 smithi100 ceph-mon[32599]: pgmap v982: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:59:40.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:39 smithi167 ceph-mon[37140]: pgmap v982: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:59:42.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:41 smithi138 ceph-mon[36954]: pgmap v983: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:59:42.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:41 smithi100 ceph-mon[32599]: pgmap v983: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:59:42.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:41 smithi167 ceph-mon[37140]: pgmap v983: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:59:44.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:59:44.084+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:59:44.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:59:44.085+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:59:44.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:59:44.085+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:59:44.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:59:44.085+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:59:44.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:59:44.085+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T13:59:44.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:43 smithi138 ceph-mon[36954]: pgmap v984: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:59:44.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:43 smithi100 ceph-mon[32599]: pgmap v984: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:59:44.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:43 smithi167 ceph-mon[37140]: pgmap v984: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:59:46.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:45 smithi138 ceph-mon[36954]: pgmap v985: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:59:46.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:45 smithi100 ceph-mon[32599]: pgmap v985: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:59:46.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:45 smithi167 ceph-mon[37140]: pgmap v985: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:59:48.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:47 smithi138 ceph-mon[36954]: pgmap v986: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:59:48.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:47 smithi100 ceph-mon[32599]: pgmap v986: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:59:48.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:47 smithi167 ceph-mon[37140]: pgmap v986: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:59:50.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:49 smithi138 ceph-mon[36954]: pgmap v987: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:59:50.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:49 smithi100 ceph-mon[32599]: pgmap v987: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:59:50.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:49 smithi167 ceph-mon[37140]: pgmap v987: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T13:59:52.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:51 smithi138 ceph-mon[36954]: pgmap v988: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:59:52.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:51 smithi100 ceph-mon[32599]: pgmap v988: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:59:52.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:51 smithi167 ceph-mon[37140]: pgmap v988: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:59:54.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:53 smithi138 ceph-mon[36954]: pgmap v989: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:59:54.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:53 smithi100 ceph-mon[32599]: pgmap v989: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:59:54.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:53 smithi167 ceph-mon[37140]: pgmap v989: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T13:59:56.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:55 smithi138 ceph-mon[36954]: pgmap v990: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:59:56.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:55 smithi100 ceph-mon[32599]: pgmap v990: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:59:56.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:55 smithi167 ceph-mon[37140]: pgmap v990: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T13:59:58.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:57 smithi138 ceph-mon[36954]: pgmap v991: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:59:58.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:57 smithi100 ceph-mon[32599]: pgmap v991: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:59:58.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:57 smithi167 ceph-mon[37140]: pgmap v991: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T13:59:59.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:59:59.100+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T13:59:59.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:59:59.101+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T13:59:59.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:59:59.101+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T13:59:59.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:59:59.101+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T13:59:59.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T13:59:59.101+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:00:00.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 13:59:59 smithi138 ceph-mon[36954]: pgmap v992: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:00:00.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 13:59:59 smithi100 ceph-mon[32599]: pgmap v992: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:00:00.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 13:59:59 smithi167 ceph-mon[37140]: pgmap v992: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:00:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:00 smithi138 ceph-mon[36954]: overall HEALTH_OK 2024-07-26T14:00:01.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:00 smithi100 ceph-mon[32599]: overall HEALTH_OK 2024-07-26T14:00:01.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:00 smithi167 ceph-mon[37140]: overall HEALTH_OK 2024-07-26T14:00:02.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:01 smithi138 ceph-mon[36954]: pgmap v993: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:00:02.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:01 smithi100 ceph-mon[32599]: pgmap v993: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:00:02.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:01 smithi167 ceph-mon[37140]: pgmap v993: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:00:03.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:00:03.115+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c006110 con 0x7fe5a81087a0 2024-07-26T14:00:03.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:00:03.115+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 10 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c006110 con 0x7fe5a81087a0 2024-07-26T14:00:03.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:00:03.118+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006050 con 0x55c0ec887390 2024-07-26T14:00:03.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:00:03.119+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 10 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006050 con 0x55c0ec887390 2024-07-26T14:00:04.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:03 smithi138 ceph-mon[36954]: pgmap v994: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:00:04.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:03 smithi100 ceph-mon[32599]: pgmap v994: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:00:04.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:03 smithi167 ceph-mon[37140]: pgmap v994: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:00:06.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:05 smithi100 ceph-mon[32599]: pgmap v995: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:00:06.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:05 smithi167 ceph-mon[37140]: pgmap v995: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:00:06.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:05 smithi138 ceph-mon[36954]: pgmap v995: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:00:08.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:07 smithi100 ceph-mon[32599]: pgmap v996: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:00:08.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:07 smithi100 ceph-mon[32599]: osdmap e114: 12 total, 12 up, 12 in 2024-07-26T14:00:08.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:07 smithi167 ceph-mon[37140]: pgmap v996: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:00:08.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:07 smithi167 ceph-mon[37140]: osdmap e114: 12 total, 12 up, 12 in 2024-07-26T14:00:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:07 smithi138 ceph-mon[36954]: pgmap v996: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:00:08.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:07 smithi138 ceph-mon[36954]: osdmap e114: 12 total, 12 up, 12 in 2024-07-26T14:00:10.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:09 smithi100 ceph-mon[32599]: pgmap v998: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:00:10.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:09 smithi167 ceph-mon[37140]: pgmap v998: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:00:10.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:09 smithi138 ceph-mon[36954]: pgmap v998: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:00:12.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:11 smithi100 ceph-mon[32599]: pgmap v999: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:00:12.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:11 smithi167 ceph-mon[37140]: pgmap v999: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:00:12.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:11 smithi138 ceph-mon[36954]: pgmap v999: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:00:14.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:00:14.103+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:00:14.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:00:14.104+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T14:00:14.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:00:14.104+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:00:14.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:00:14.104+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:00:14.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:00:14.104+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:00:14.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:13 smithi100 ceph-mon[32599]: pgmap v1000: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:00:14.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:13 smithi100 ceph-mon[32599]: osdmap e115: 12 total, 12 up, 12 in 2024-07-26T14:00:14.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:13 smithi167 ceph-mon[37140]: pgmap v1000: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:00:14.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:13 smithi167 ceph-mon[37140]: osdmap e115: 12 total, 12 up, 12 in 2024-07-26T14:00:14.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:13 smithi138 ceph-mon[36954]: pgmap v1000: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:00:14.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:13 smithi138 ceph-mon[36954]: osdmap e115: 12 total, 12 up, 12 in 2024-07-26T14:00:15.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:14 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:00:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:14 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:00:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:14 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:00:16.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:16 smithi138 ceph-mon[36954]: pgmap v1002: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T14:00:16.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:00:16.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:00:16.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:00:16.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:00:16.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:00:16.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:00:16.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:00:16.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:00:16.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:00:16.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:16 smithi100 ceph-mon[32599]: pgmap v1002: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T14:00:16.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:00:16.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:00:16.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:00:16.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:00:16.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:00:16.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:00:16.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:00:16.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:00:16.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:00:16.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:16 smithi167 ceph-mon[37140]: pgmap v1002: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T14:00:16.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:00:16.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:00:16.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:00:16.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:00:16.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:00:16.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:00:16.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:00:16.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:00:16.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:00:18.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:18 smithi138 ceph-mon[36954]: pgmap v1003: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 208 B/s wr, 5 op/s 2024-07-26T14:00:18.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:18 smithi100 ceph-mon[32599]: pgmap v1003: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 208 B/s wr, 5 op/s 2024-07-26T14:00:18.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:18 smithi167 ceph-mon[37140]: pgmap v1003: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 208 B/s wr, 5 op/s 2024-07-26T14:00:20.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:20 smithi138 ceph-mon[36954]: pgmap v1004: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:00:20.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:20 smithi100 ceph-mon[32599]: pgmap v1004: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:00:20.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:20 smithi167 ceph-mon[37140]: pgmap v1004: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:00:22.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:22 smithi138 ceph-mon[36954]: pgmap v1005: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:00:22.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:22 smithi100 ceph-mon[32599]: pgmap v1005: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:00:22.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:22 smithi167 ceph-mon[37140]: pgmap v1005: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:00:24.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:24 smithi138 ceph-mon[36954]: pgmap v1006: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:00:24.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:24 smithi100 ceph-mon[32599]: pgmap v1006: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:00:24.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:24 smithi167 ceph-mon[37140]: pgmap v1006: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:00:26.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:26 smithi138 ceph-mon[36954]: pgmap v1007: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:00:26.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:26 smithi100 ceph-mon[32599]: pgmap v1007: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:00:26.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:26 smithi167 ceph-mon[37140]: pgmap v1007: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:00:28.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:28 smithi138 ceph-mon[36954]: pgmap v1008: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:00:28.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:28 smithi100 ceph-mon[32599]: pgmap v1008: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:00:28.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:28 smithi167 ceph-mon[37140]: pgmap v1008: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:00:29.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:00:29.119+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:00:29.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:00:29.120+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T14:00:29.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:00:29.120+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:00:29.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:00:29.120+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:00:29.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:00:29.120+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:00:30.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:30 smithi138 ceph-mon[36954]: pgmap v1009: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:00:30.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:30 smithi100 ceph-mon[32599]: pgmap v1009: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:00:30.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:30 smithi167 ceph-mon[37140]: pgmap v1009: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:00:32.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:32 smithi138 ceph-mon[36954]: pgmap v1010: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:00:32.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:32 smithi100 ceph-mon[32599]: pgmap v1010: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:00:32.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:32 smithi167 ceph-mon[37140]: pgmap v1010: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:00:34.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:34 smithi138 ceph-mon[36954]: pgmap v1011: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:00:34.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:34 smithi100 ceph-mon[32599]: pgmap v1011: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:00:34.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:34 smithi167 ceph-mon[37140]: pgmap v1011: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:00:36.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:36 smithi138 ceph-mon[36954]: pgmap v1012: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:00:36.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:36 smithi100 ceph-mon[32599]: pgmap v1012: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:00:36.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:36 smithi167 ceph-mon[37140]: pgmap v1012: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:00:38.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:38 smithi138 ceph-mon[36954]: pgmap v1013: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:00:38.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:38 smithi100 ceph-mon[32599]: pgmap v1013: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:00:38.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:38 smithi167 ceph-mon[37140]: pgmap v1013: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:00:40.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:40 smithi138 ceph-mon[36954]: pgmap v1014: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T14:00:40.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:40 smithi100 ceph-mon[32599]: pgmap v1014: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T14:00:40.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:40 smithi167 ceph-mon[37140]: pgmap v1014: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T14:00:42.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:42 smithi138 ceph-mon[36954]: pgmap v1015: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:00:42.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:42 smithi100 ceph-mon[32599]: pgmap v1015: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:00:42.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:42 smithi167 ceph-mon[37140]: pgmap v1015: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:00:44.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:00:44.135+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:00:44.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:00:44.136+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T14:00:44.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:00:44.136+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:00:44.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:00:44.136+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:00:44.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:00:44.136+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:00:44.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:44 smithi138 ceph-mon[36954]: pgmap v1016: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:00:44.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:44 smithi100 ceph-mon[32599]: pgmap v1016: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:00:44.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:44 smithi167 ceph-mon[37140]: pgmap v1016: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:00:46.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:46 smithi138 ceph-mon[36954]: pgmap v1017: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:00:46.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:46 smithi100 ceph-mon[32599]: pgmap v1017: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:00:46.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:46 smithi167 ceph-mon[37140]: pgmap v1017: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:00:48.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:48 smithi138 ceph-mon[36954]: pgmap v1018: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:00:48.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:48 smithi100 ceph-mon[32599]: pgmap v1018: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:00:48.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:48 smithi167 ceph-mon[37140]: pgmap v1018: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:00:50.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:50 smithi138 ceph-mon[36954]: pgmap v1019: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T14:00:50.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:50 smithi100 ceph-mon[32599]: pgmap v1019: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T14:00:50.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:50 smithi167 ceph-mon[37140]: pgmap v1019: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T14:00:52.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:52 smithi138 ceph-mon[36954]: pgmap v1020: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:00:52.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:52 smithi100 ceph-mon[32599]: pgmap v1020: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:00:52.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:52 smithi167 ceph-mon[37140]: pgmap v1020: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:00:54.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:54 smithi138 ceph-mon[36954]: pgmap v1021: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:00:54.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:54 smithi100 ceph-mon[32599]: pgmap v1021: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:00:54.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:54 smithi167 ceph-mon[37140]: pgmap v1021: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:00:56.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:56 smithi100 ceph-mon[32599]: pgmap v1022: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:00:56.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:56 smithi167 ceph-mon[37140]: pgmap v1022: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:00:56.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:56 smithi138 ceph-mon[36954]: pgmap v1022: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:00:58.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:00:58 smithi100 ceph-mon[32599]: pgmap v1023: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:00:58.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:00:58 smithi167 ceph-mon[37140]: pgmap v1023: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:00:58.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:00:58 smithi138 ceph-mon[36954]: pgmap v1023: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:00:59.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:00:59.151+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:00:59.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:00:59.152+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T14:00:59.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:00:59.152+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:00:59.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:00:59.152+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:00:59.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:00:59.152+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:01:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:00 smithi100 ceph-mon[32599]: pgmap v1024: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:01:00.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:00 smithi167 ceph-mon[37140]: pgmap v1024: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:01:00.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:00 smithi138 ceph-mon[36954]: pgmap v1024: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:01:02.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:02 smithi100 ceph-mon[32599]: pgmap v1025: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:01:02.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:02 smithi167 ceph-mon[37140]: pgmap v1025: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:01:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:02 smithi138 ceph-mon[36954]: pgmap v1025: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:01:04.493 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:04 smithi138 ceph-mon[36954]: pgmap v1026: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:01:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:04 smithi100 ceph-mon[32599]: pgmap v1026: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:01:04.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:04 smithi167 ceph-mon[37140]: pgmap v1026: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:01:06.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:06 smithi100 ceph-mon[32599]: pgmap v1027: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:01:06.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:06 smithi167 ceph-mon[37140]: pgmap v1027: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:01:06.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:06 smithi138 ceph-mon[36954]: pgmap v1027: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:01:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:08 smithi100 ceph-mon[32599]: pgmap v1028: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:01:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:08 smithi100 ceph-mon[32599]: osdmap e116: 12 total, 12 up, 12 in 2024-07-26T14:01:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:08 smithi167 ceph-mon[37140]: pgmap v1028: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:01:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:08 smithi167 ceph-mon[37140]: osdmap e116: 12 total, 12 up, 12 in 2024-07-26T14:01:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:08 smithi138 ceph-mon[36954]: pgmap v1028: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:01:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:08 smithi138 ceph-mon[36954]: osdmap e116: 12 total, 12 up, 12 in 2024-07-26T14:01:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:10 smithi100 ceph-mon[32599]: pgmap v1030: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:01:10.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:10 smithi167 ceph-mon[37140]: pgmap v1030: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:01:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:10 smithi138 ceph-mon[36954]: pgmap v1030: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:01:12.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:12 smithi100 ceph-mon[32599]: pgmap v1031: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:01:12.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:12 smithi167 ceph-mon[37140]: pgmap v1031: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:01:12.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:12 smithi138 ceph-mon[36954]: pgmap v1031: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:01:14.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:01:14.168+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:01:14.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:01:14.168+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T14:01:14.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:01:14.168+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:01:14.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:01:14.168+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:01:14.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:01:14.168+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:01:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:14 smithi100 ceph-mon[32599]: pgmap v1032: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:01:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:14 smithi100 ceph-mon[32599]: osdmap e117: 12 total, 12 up, 12 in 2024-07-26T14:01:14.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:14 smithi167 ceph-mon[37140]: pgmap v1032: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:01:14.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:14 smithi167 ceph-mon[37140]: osdmap e117: 12 total, 12 up, 12 in 2024-07-26T14:01:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:14 smithi138 ceph-mon[36954]: pgmap v1032: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:01:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:14 smithi138 ceph-mon[36954]: osdmap e117: 12 total, 12 up, 12 in 2024-07-26T14:01:16.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:16 smithi100 ceph-mon[32599]: pgmap v1034: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T14:01:16.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:01:16.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:16 smithi167 ceph-mon[37140]: pgmap v1034: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T14:01:16.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:01:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:16 smithi138 ceph-mon[36954]: pgmap v1034: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T14:01:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:01:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:01:17.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:01:17.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:01:17.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:01:17.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:01:17.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:01:17.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:01:17.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:01:17.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:01:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:01:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:01:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:01:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:01:17.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:01:17.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:01:17.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:01:17.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:01:17.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:01:17.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:01:17.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:01:17.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:01:17.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:01:17.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:01:17.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:01:17.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:01:17.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:01:17.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:01:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:18 smithi100 ceph-mon[32599]: pgmap v1035: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 213 B/s wr, 5 op/s 2024-07-26T14:01:18.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:18 smithi167 ceph-mon[37140]: pgmap v1035: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 213 B/s wr, 5 op/s 2024-07-26T14:01:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:18 smithi138 ceph-mon[36954]: pgmap v1035: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 213 B/s wr, 5 op/s 2024-07-26T14:01:20.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:20 smithi100 ceph-mon[32599]: pgmap v1036: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T14:01:20.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:20 smithi167 ceph-mon[37140]: pgmap v1036: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T14:01:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:20 smithi138 ceph-mon[36954]: pgmap v1036: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T14:01:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:22 smithi100 ceph-mon[32599]: pgmap v1037: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T14:01:22.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:22 smithi167 ceph-mon[37140]: pgmap v1037: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T14:01:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:22 smithi138 ceph-mon[36954]: pgmap v1037: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T14:01:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:24 smithi100 ceph-mon[32599]: pgmap v1038: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T14:01:24.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:24 smithi167 ceph-mon[37140]: pgmap v1038: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T14:01:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:24 smithi138 ceph-mon[36954]: pgmap v1038: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T14:01:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:26 smithi100 ceph-mon[32599]: pgmap v1039: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:01:26.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:26 smithi167 ceph-mon[37140]: pgmap v1039: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:01:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:26 smithi138 ceph-mon[36954]: pgmap v1039: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:01:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:28 smithi100 ceph-mon[32599]: pgmap v1040: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:01:28.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:28 smithi167 ceph-mon[37140]: pgmap v1040: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:01:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:28 smithi138 ceph-mon[36954]: pgmap v1040: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:01:29.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:01:29.184+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:01:29.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:01:29.184+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T14:01:29.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:01:29.184+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:01:29.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:01:29.184+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:01:29.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:01:29.184+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:01:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:30 smithi100 ceph-mon[32599]: pgmap v1041: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:01:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:30 smithi167 ceph-mon[37140]: pgmap v1041: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:01:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:30 smithi138 ceph-mon[36954]: pgmap v1041: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:01:32.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:32 smithi100 ceph-mon[32599]: pgmap v1042: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:01:32.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:32 smithi167 ceph-mon[37140]: pgmap v1042: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:01:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:32 smithi138 ceph-mon[36954]: pgmap v1042: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:01:34.742 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:34 smithi138 ceph-mon[36954]: pgmap v1043: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:01:34.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:34 smithi100 ceph-mon[32599]: pgmap v1043: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:01:34.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:34 smithi167 ceph-mon[37140]: pgmap v1043: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:01:36.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:36 smithi100 ceph-mon[32599]: pgmap v1044: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:01:36.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:36 smithi167 ceph-mon[37140]: pgmap v1044: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:01:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:36 smithi138 ceph-mon[36954]: pgmap v1044: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:01:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:38 smithi100 ceph-mon[32599]: pgmap v1045: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:01:38.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:38 smithi167 ceph-mon[37140]: pgmap v1045: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:01:38.944 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:38 smithi138 ceph-mon[36954]: pgmap v1045: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:01:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:40 smithi100 ceph-mon[32599]: pgmap v1046: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:01:40.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:40 smithi167 ceph-mon[37140]: pgmap v1046: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:01:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:40 smithi138 ceph-mon[36954]: pgmap v1046: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:01:42.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:42 smithi100 ceph-mon[32599]: pgmap v1047: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:01:42.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:42 smithi167 ceph-mon[37140]: pgmap v1047: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:01:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:42 smithi138 ceph-mon[36954]: pgmap v1047: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:01:44.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:01:44.200+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:01:44.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:01:44.200+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T14:01:44.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:01:44.200+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:01:44.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:01:44.200+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:01:44.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:01:44.200+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:01:44.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:44 smithi100 ceph-mon[32599]: pgmap v1048: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:01:44.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:44 smithi167 ceph-mon[37140]: pgmap v1048: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:01:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:44 smithi138 ceph-mon[36954]: pgmap v1048: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:01:46.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:46 smithi100 ceph-mon[32599]: pgmap v1049: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:01:46.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:46 smithi167 ceph-mon[37140]: pgmap v1049: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:01:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:46 smithi138 ceph-mon[36954]: pgmap v1049: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:01:48.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:48 smithi167 ceph-mon[37140]: pgmap v1050: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:01:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:48 smithi138 ceph-mon[36954]: pgmap v1050: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:01:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:48 smithi100 ceph-mon[32599]: pgmap v1050: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:01:50.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:50 smithi167 ceph-mon[37140]: pgmap v1051: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:01:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:50 smithi138 ceph-mon[36954]: pgmap v1051: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:01:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:50 smithi100 ceph-mon[32599]: pgmap v1051: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:01:52.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:52 smithi167 ceph-mon[37140]: pgmap v1052: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:01:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:52 smithi138 ceph-mon[36954]: pgmap v1052: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:01:52.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:52 smithi100 ceph-mon[32599]: pgmap v1052: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:01:54.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:54 smithi167 ceph-mon[37140]: pgmap v1053: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:01:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:54 smithi138 ceph-mon[36954]: pgmap v1053: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:01:54.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:54 smithi100 ceph-mon[32599]: pgmap v1053: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:01:56.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:56 smithi167 ceph-mon[37140]: pgmap v1054: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:01:56.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:56 smithi100 ceph-mon[32599]: pgmap v1054: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:01:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:56 smithi138 ceph-mon[36954]: pgmap v1054: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:01:58.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:01:58 smithi167 ceph-mon[37140]: pgmap v1055: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:01:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:01:58 smithi138 ceph-mon[36954]: pgmap v1055: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:01:58.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:01:58 smithi100 ceph-mon[32599]: pgmap v1055: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:01:59.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:01:59.210+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:01:59.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:01:59.211+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T14:01:59.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:01:59.211+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:01:59.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:01:59.211+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:01:59.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:01:59.211+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:02:00.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:02:00 smithi167 ceph-mon[37140]: pgmap v1056: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:02:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:02:00 smithi138 ceph-mon[36954]: pgmap v1056: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:02:00.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:02:00 smithi100 ceph-mon[32599]: pgmap v1056: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:02:02.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:02:02 smithi167 ceph-mon[37140]: pgmap v1057: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:02:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:02:02 smithi138 ceph-mon[36954]: pgmap v1057: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:02:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:02:02 smithi100 ceph-mon[32599]: pgmap v1057: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:02:04.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:02:04 smithi167 ceph-mon[37140]: pgmap v1058: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:02:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:02:04 smithi138 ceph-mon[36954]: pgmap v1058: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:02:04.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:02:04 smithi100 ceph-mon[32599]: pgmap v1058: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:02:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:02:06 smithi138 ceph-mon[36954]: pgmap v1059: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:02:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:02:06 smithi100 ceph-mon[32599]: pgmap v1059: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:02:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:02:06 smithi167 ceph-mon[37140]: pgmap v1059: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:02:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:02:08 smithi138 ceph-mon[36954]: pgmap v1060: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:02:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:02:08 smithi138 ceph-mon[36954]: osdmap e118: 12 total, 12 up, 12 in 2024-07-26T14:02:08.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:02:08 smithi100 ceph-mon[32599]: pgmap v1060: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:02:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:02:08 smithi100 ceph-mon[32599]: osdmap e118: 12 total, 12 up, 12 in 2024-07-26T14:02:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:02:08 smithi167 ceph-mon[37140]: pgmap v1060: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:02:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:02:08 smithi167 ceph-mon[37140]: osdmap e118: 12 total, 12 up, 12 in 2024-07-26T14:02:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:02:10 smithi138 ceph-mon[36954]: pgmap v1062: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T14:02:10.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:02:10 smithi100 ceph-mon[32599]: pgmap v1062: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T14:02:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:02:10 smithi167 ceph-mon[37140]: pgmap v1062: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T14:02:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:02:12 smithi138 ceph-mon[36954]: pgmap v1063: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T14:02:12.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:02:12 smithi100 ceph-mon[32599]: pgmap v1063: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T14:02:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:02:12 smithi167 ceph-mon[37140]: pgmap v1063: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T14:02:14.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:02:14.215+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:02:14.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:02:14.216+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T14:02:14.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:02:14.216+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:02:14.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:02:14.216+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:02:14.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:02:14.216+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:02:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:02:14 smithi138 ceph-mon[36954]: pgmap v1064: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:02:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:02:14 smithi138 ceph-mon[36954]: osdmap e119: 12 total, 12 up, 12 in 2024-07-26T14:02:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:02:14 smithi100 ceph-mon[32599]: pgmap v1064: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:02:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:02:14 smithi100 ceph-mon[32599]: osdmap e119: 12 total, 12 up, 12 in 2024-07-26T14:02:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:02:14 smithi167 ceph-mon[37140]: pgmap v1064: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:02:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:02:14 smithi167 ceph-mon[37140]: osdmap e119: 12 total, 12 up, 12 in 2024-07-26T14:02:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:02:16 smithi138 ceph-mon[36954]: pgmap v1066: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T14:02:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:02:16 smithi100 ceph-mon[32599]: pgmap v1066: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T14:02:17.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:02:16 smithi167 ceph-mon[37140]: pgmap v1066: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T14:02:17.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:02:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:02:17.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:02:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:02:17.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:02:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:02:17.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:02:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:02:17.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:02:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:02:17.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:02:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:02:17.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:02:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:02:17.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:02:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:02:18.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:02:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:02:18.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:02:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:02:18.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:02:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:02:18.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:02:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:02:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:02:18 smithi138 ceph-mon[36954]: pgmap v1067: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 220 B/s wr, 6 op/s 2024-07-26T14:02:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:02:18 smithi100 ceph-mon[32599]: pgmap v1067: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 220 B/s wr, 6 op/s 2024-07-26T14:02:19.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:02:18 smithi167 ceph-mon[37140]: pgmap v1067: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 220 B/s wr, 6 op/s 2024-07-26T14:02:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:02:20 smithi138 ceph-mon[36954]: pgmap v1068: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:02:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:02:20 smithi100 ceph-mon[32599]: pgmap v1068: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:02:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:02:20 smithi167 ceph-mon[37140]: pgmap v1068: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:02:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:02:22 smithi138 ceph-mon[36954]: pgmap v1069: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:02:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:02:22 smithi100 ceph-mon[32599]: pgmap v1069: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:02:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:02:22 smithi167 ceph-mon[37140]: pgmap v1069: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:02:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:02:24 smithi138 ceph-mon[36954]: pgmap v1070: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:02:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:02:24 smithi100 ceph-mon[32599]: pgmap v1070: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:02:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:02:24 smithi167 ceph-mon[37140]: pgmap v1070: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:02:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:02:26 smithi138 ceph-mon[36954]: pgmap v1071: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:02:26.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:02:26 smithi100 ceph-mon[32599]: pgmap v1071: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:02:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:02:26 smithi167 ceph-mon[37140]: pgmap v1071: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:02:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:02:28 smithi138 ceph-mon[36954]: pgmap v1072: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:02:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:02:28 smithi100 ceph-mon[32599]: pgmap v1072: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:02:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:02:28 smithi167 ceph-mon[37140]: pgmap v1072: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:02:29.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:02:29.231+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:02:29.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:02:29.231+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T14:02:29.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:02:29.231+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:02:29.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:02:29.231+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:02:29.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:02:29.231+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:02:30.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:02:30 smithi167 ceph-mon[37140]: pgmap v1073: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:02:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:02:30 smithi138 ceph-mon[36954]: pgmap v1073: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:02:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:02:30 smithi100 ceph-mon[32599]: pgmap v1073: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:02:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:02:32 smithi138 ceph-mon[36954]: pgmap v1074: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:02:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:02:32 smithi100 ceph-mon[32599]: pgmap v1074: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:02:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:02:32 smithi167 ceph-mon[37140]: pgmap v1074: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:02:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:02:34 smithi138 ceph-mon[36954]: pgmap v1075: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:02:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:02:34 smithi100 ceph-mon[32599]: pgmap v1075: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:02:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:02:34 smithi167 ceph-mon[37140]: pgmap v1075: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:02:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:02:36 smithi138 ceph-mon[36954]: pgmap v1076: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:02:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:02:36 smithi100 ceph-mon[32599]: pgmap v1076: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:02:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:02:36 smithi167 ceph-mon[37140]: pgmap v1076: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:02:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:02:38 smithi138 ceph-mon[36954]: pgmap v1077: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:02:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:02:38 smithi100 ceph-mon[32599]: pgmap v1077: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:02:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:02:38 smithi167 ceph-mon[37140]: pgmap v1077: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:02:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:02:40 smithi138 ceph-mon[36954]: pgmap v1078: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:02:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:02:40 smithi100 ceph-mon[32599]: pgmap v1078: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:02:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:02:40 smithi167 ceph-mon[37140]: pgmap v1078: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:02:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:02:42 smithi138 ceph-mon[36954]: pgmap v1079: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:02:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:02:42 smithi100 ceph-mon[32599]: pgmap v1079: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:02:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:02:42 smithi167 ceph-mon[37140]: pgmap v1079: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:02:44.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:02:44.237+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:02:44.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:02:44.237+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T14:02:44.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:02:44.237+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:02:44.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:02:44.237+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:02:44.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:02:44.237+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:02:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:02:44 smithi138 ceph-mon[36954]: pgmap v1080: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:02:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:02:44 smithi100 ceph-mon[32599]: pgmap v1080: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:02:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:02:44 smithi167 ceph-mon[37140]: pgmap v1080: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:02:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:02:46 smithi138 ceph-mon[36954]: pgmap v1081: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:02:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:02:46 smithi100 ceph-mon[32599]: pgmap v1081: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:02:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:02:46 smithi167 ceph-mon[37140]: pgmap v1081: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:02:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:02:48 smithi138 ceph-mon[36954]: pgmap v1082: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:02:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:02:48 smithi100 ceph-mon[32599]: pgmap v1082: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:02:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:02:48 smithi167 ceph-mon[37140]: pgmap v1082: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:02:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:02:50 smithi138 ceph-mon[36954]: pgmap v1083: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:02:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:02:50 smithi100 ceph-mon[32599]: pgmap v1083: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:02:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:02:50 smithi167 ceph-mon[37140]: pgmap v1083: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:02:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:02:52 smithi138 ceph-mon[36954]: pgmap v1084: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:02:52.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:02:52 smithi100 ceph-mon[32599]: pgmap v1084: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:02:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:02:52 smithi167 ceph-mon[37140]: pgmap v1084: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:02:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:02:54 smithi138 ceph-mon[36954]: pgmap v1085: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:02:54.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:02:54 smithi100 ceph-mon[32599]: pgmap v1085: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:02:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:02:54 smithi167 ceph-mon[37140]: pgmap v1085: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:02:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:02:56 smithi138 ceph-mon[36954]: pgmap v1086: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:02:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:02:56 smithi100 ceph-mon[32599]: pgmap v1086: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:02:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:02:56 smithi167 ceph-mon[37140]: pgmap v1086: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:02:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:02:58 smithi138 ceph-mon[36954]: pgmap v1087: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:02:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:02:58 smithi100 ceph-mon[32599]: pgmap v1087: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:02:59.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:02:58 smithi167 ceph-mon[37140]: pgmap v1087: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:02:59.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:02:59.244+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:02:59.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:02:59.245+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-07-26T14:02:59.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:02:59.245+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:02:59.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:02:59.245+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:02:59.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:02:59.245+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:03:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:03:00 smithi138 ceph-mon[36954]: pgmap v1088: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:03:00.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:03:00 smithi100 ceph-mon[32599]: pgmap v1088: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:03:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:03:00 smithi167 ceph-mon[37140]: pgmap v1088: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:03:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:03:02 smithi138 ceph-mon[36954]: pgmap v1089: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:03:02.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:03:02 smithi100 ceph-mon[32599]: pgmap v1089: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:03:03.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:03:02 smithi167 ceph-mon[37140]: pgmap v1089: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:03:03.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:03:03.027+0000 7fe5a6d9d640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b3d40 msgr2=0x7fe5940b6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).tick idle (900000852) for more than 900000000 us, fault. 2024-07-26T14:03:03.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:03:03.027+0000 7fe5a6d9d640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b3d40 0x7fe5940b6250 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe5a00aaca0 tx=0x7fe5a0078460 comp rx=0 tx=0).stop 2024-07-26T14:03:03.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:03:03.027+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b3d40 msgr2=0x7fe5940b6250 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-07-26T14:03:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:03:03.027+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b3d40 0x7fe5940b6250 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T14:03:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:03:03.027+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T14:03:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:03:03.028+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:03:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:03:03.028+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe5900049f0 tx=0x7fe590046040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T14:03:03.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:03:03.140+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c0063f0 con 0x7fe5a81087a0 2024-07-26T14:03:03.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:03:03.140+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 11 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c0063f0 con 0x7fe5a81087a0 2024-07-26T14:03:03.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:03:03.141+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe5800062c0 con 0x55c0ec887390 2024-07-26T14:03:03.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:03:03.141+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 11 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe5800062c0 con 0x55c0ec887390 2024-07-26T14:03:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:03:04 smithi138 ceph-mon[36954]: pgmap v1090: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:03:04.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:03:04 smithi100 ceph-mon[32599]: pgmap v1090: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:03:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:03:04 smithi167 ceph-mon[37140]: pgmap v1090: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:03:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:03:06 smithi138 ceph-mon[36954]: pgmap v1091: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:03:06.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:03:06 smithi100 ceph-mon[32599]: pgmap v1091: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:03:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:03:06 smithi167 ceph-mon[37140]: pgmap v1091: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:03:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:03:08 smithi138 ceph-mon[36954]: pgmap v1092: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:03:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:03:08 smithi138 ceph-mon[36954]: osdmap e120: 12 total, 12 up, 12 in 2024-07-26T14:03:08.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:03:08 smithi100 ceph-mon[32599]: pgmap v1092: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:03:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:03:08 smithi100 ceph-mon[32599]: osdmap e120: 12 total, 12 up, 12 in 2024-07-26T14:03:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:03:08 smithi167 ceph-mon[37140]: pgmap v1092: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:03:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:03:08 smithi167 ceph-mon[37140]: osdmap e120: 12 total, 12 up, 12 in 2024-07-26T14:03:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:03:10 smithi138 ceph-mon[36954]: pgmap v1094: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T14:03:10.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:03:10 smithi100 ceph-mon[32599]: pgmap v1094: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T14:03:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:03:10 smithi167 ceph-mon[37140]: pgmap v1094: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T14:03:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:03:12 smithi138 ceph-mon[36954]: pgmap v1095: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T14:03:12.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:03:12 smithi100 ceph-mon[32599]: pgmap v1095: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T14:03:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:03:12 smithi167 ceph-mon[37140]: pgmap v1095: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T14:03:14.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:03:14.247+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:03:14.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:03:14.248+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:03:14.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:03:14.248+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:03:14.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:03:14.248+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:03:14.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:03:14.248+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:03:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:03:14 smithi138 ceph-mon[36954]: pgmap v1096: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:03:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:03:14 smithi138 ceph-mon[36954]: osdmap e121: 12 total, 12 up, 12 in 2024-07-26T14:03:14.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:03:14 smithi100 ceph-mon[32599]: pgmap v1096: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:03:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:03:14 smithi100 ceph-mon[32599]: osdmap e121: 12 total, 12 up, 12 in 2024-07-26T14:03:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:03:14 smithi167 ceph-mon[37140]: pgmap v1096: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:03:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:03:14 smithi167 ceph-mon[37140]: osdmap e121: 12 total, 12 up, 12 in 2024-07-26T14:03:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:03:16 smithi138 ceph-mon[36954]: pgmap v1098: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T14:03:16.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:03:16 smithi100 ceph-mon[32599]: pgmap v1098: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T14:03:17.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:03:16 smithi167 ceph-mon[37140]: pgmap v1098: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T14:03:17.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:03:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:03:17.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:03:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:03:17.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:03:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:03:17.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:03:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:03:17.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:03:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:03:17.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:03:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:03:17.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:03:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:03:17.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:03:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:03:18.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:03:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:03:18.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:03:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:03:18.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:03:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:03:18.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:03:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:03:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:03:18 smithi138 ceph-mon[36954]: pgmap v1099: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 222 B/s wr, 6 op/s 2024-07-26T14:03:18.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:03:18 smithi100 ceph-mon[32599]: pgmap v1099: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 222 B/s wr, 6 op/s 2024-07-26T14:03:19.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:03:18 smithi167 ceph-mon[37140]: pgmap v1099: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 222 B/s wr, 6 op/s 2024-07-26T14:03:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:03:20 smithi138 ceph-mon[36954]: pgmap v1100: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:03:20.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:03:20 smithi100 ceph-mon[32599]: pgmap v1100: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:03:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:03:20 smithi167 ceph-mon[37140]: pgmap v1100: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:03:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:03:22 smithi138 ceph-mon[36954]: pgmap v1101: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:03:22.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:03:22 smithi100 ceph-mon[32599]: pgmap v1101: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:03:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:03:22 smithi167 ceph-mon[37140]: pgmap v1101: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:03:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:03:24 smithi138 ceph-mon[36954]: pgmap v1102: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:03:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:03:24 smithi100 ceph-mon[32599]: pgmap v1102: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:03:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:03:24 smithi167 ceph-mon[37140]: pgmap v1102: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:03:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:03:26 smithi138 ceph-mon[36954]: pgmap v1103: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:03:26.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:03:26 smithi100 ceph-mon[32599]: pgmap v1103: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:03:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:03:26 smithi167 ceph-mon[37140]: pgmap v1103: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:03:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:03:28 smithi138 ceph-mon[36954]: pgmap v1104: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:03:28.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:03:28 smithi100 ceph-mon[32599]: pgmap v1104: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:03:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:03:28 smithi167 ceph-mon[37140]: pgmap v1104: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:03:29.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:03:29.261+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:03:29.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:03:29.262+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:03:29.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:03:29.262+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:03:29.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:03:29.262+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:03:29.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:03:29.262+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:03:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:03:30 smithi138 ceph-mon[36954]: pgmap v1105: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:03:30.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:03:30 smithi100 ceph-mon[32599]: pgmap v1105: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:03:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:03:30 smithi167 ceph-mon[37140]: pgmap v1105: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:03:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:03:32 smithi138 ceph-mon[36954]: pgmap v1106: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:03:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:03:32 smithi100 ceph-mon[32599]: pgmap v1106: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:03:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:03:32 smithi167 ceph-mon[37140]: pgmap v1106: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:03:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:03:34 smithi138 ceph-mon[36954]: pgmap v1107: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:03:34.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:03:34 smithi100 ceph-mon[32599]: pgmap v1107: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:03:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:03:34 smithi167 ceph-mon[37140]: pgmap v1107: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:03:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:03:36 smithi138 ceph-mon[36954]: pgmap v1108: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:03:36.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:03:36 smithi100 ceph-mon[32599]: pgmap v1108: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:03:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:03:36 smithi167 ceph-mon[37140]: pgmap v1108: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:03:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:03:38 smithi100 ceph-mon[32599]: pgmap v1109: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:03:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:03:38 smithi167 ceph-mon[37140]: pgmap v1109: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:03:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:03:38 smithi138 ceph-mon[36954]: pgmap v1109: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:03:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:03:40 smithi100 ceph-mon[32599]: pgmap v1110: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:03:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:03:40 smithi167 ceph-mon[37140]: pgmap v1110: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:03:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:03:40 smithi138 ceph-mon[36954]: pgmap v1110: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:03:42.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:03:42 smithi100 ceph-mon[32599]: pgmap v1111: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:03:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:03:42 smithi167 ceph-mon[37140]: pgmap v1111: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:03:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:03:42 smithi138 ceph-mon[36954]: pgmap v1111: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:03:44.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:03:44.277+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:03:44.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:03:44.277+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:03:44.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:03:44.277+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:03:44.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:03:44.277+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:03:44.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:03:44.277+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:03:44.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:03:44 smithi100 ceph-mon[32599]: pgmap v1112: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:03:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:03:44 smithi167 ceph-mon[37140]: pgmap v1112: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:03:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:03:44 smithi138 ceph-mon[36954]: pgmap v1112: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:03:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:03:46 smithi100 ceph-mon[32599]: pgmap v1113: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:03:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:03:46 smithi167 ceph-mon[37140]: pgmap v1113: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:03:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:03:46 smithi138 ceph-mon[36954]: pgmap v1113: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:03:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:03:48 smithi100 ceph-mon[32599]: pgmap v1114: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:03:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:03:48 smithi167 ceph-mon[37140]: pgmap v1114: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:03:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:03:48 smithi138 ceph-mon[36954]: pgmap v1114: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:03:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:03:50 smithi100 ceph-mon[32599]: pgmap v1115: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:03:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:03:50 smithi167 ceph-mon[37140]: pgmap v1115: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:03:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:03:50 smithi138 ceph-mon[36954]: pgmap v1115: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:03:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:03:52 smithi100 ceph-mon[32599]: pgmap v1116: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:03:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:03:52 smithi167 ceph-mon[37140]: pgmap v1116: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:03:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:03:52 smithi138 ceph-mon[36954]: pgmap v1116: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:03:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:03:54 smithi100 ceph-mon[32599]: pgmap v1117: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:03:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:03:54 smithi167 ceph-mon[37140]: pgmap v1117: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:03:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:03:54 smithi138 ceph-mon[36954]: pgmap v1117: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:03:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:03:56 smithi100 ceph-mon[32599]: pgmap v1118: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:03:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:03:56 smithi167 ceph-mon[37140]: pgmap v1118: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:03:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:03:56 smithi138 ceph-mon[36954]: pgmap v1118: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:03:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:03:58 smithi100 ceph-mon[32599]: pgmap v1119: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:03:59.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:03:58 smithi167 ceph-mon[37140]: pgmap v1119: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:03:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:03:58 smithi138 ceph-mon[36954]: pgmap v1119: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:03:59.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:03:59.282+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:03:59.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:03:59.283+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:03:59.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:03:59.283+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:03:59.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:03:59.283+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:03:59.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:03:59.283+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:04:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:04:00 smithi100 ceph-mon[32599]: pgmap v1120: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:04:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:04:00 smithi167 ceph-mon[37140]: pgmap v1120: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:04:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:04:00 smithi138 ceph-mon[36954]: pgmap v1120: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:04:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:04:02 smithi100 ceph-mon[32599]: pgmap v1121: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:04:03.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:04:02 smithi167 ceph-mon[37140]: pgmap v1121: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:04:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:04:02 smithi138 ceph-mon[36954]: pgmap v1121: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:04:04.992 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:04:04 smithi138 ceph-mon[36954]: pgmap v1122: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:04:04.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:04:04 smithi100 ceph-mon[32599]: pgmap v1122: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:04:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:04:04 smithi167 ceph-mon[37140]: pgmap v1122: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:04:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:04:06 smithi100 ceph-mon[32599]: pgmap v1123: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:04:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:04:06 smithi167 ceph-mon[37140]: pgmap v1123: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:04:07.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:04:06 smithi138 ceph-mon[36954]: pgmap v1123: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:04:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:04:08 smithi167 ceph-mon[37140]: pgmap v1124: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:04:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:04:08 smithi167 ceph-mon[37140]: osdmap e122: 12 total, 12 up, 12 in 2024-07-26T14:04:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:04:08 smithi138 ceph-mon[36954]: pgmap v1124: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:04:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:04:08 smithi138 ceph-mon[36954]: osdmap e122: 12 total, 12 up, 12 in 2024-07-26T14:04:09.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:04:08 smithi100 ceph-mon[32599]: pgmap v1124: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:04:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:04:08 smithi100 ceph-mon[32599]: osdmap e122: 12 total, 12 up, 12 in 2024-07-26T14:04:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:04:10 smithi167 ceph-mon[37140]: pgmap v1126: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:04:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:04:10 smithi138 ceph-mon[36954]: pgmap v1126: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:04:11.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:04:10 smithi100 ceph-mon[32599]: pgmap v1126: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:04:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:04:12 smithi167 ceph-mon[37140]: pgmap v1127: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:04:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:04:12 smithi138 ceph-mon[36954]: pgmap v1127: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:04:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:04:12 smithi100 ceph-mon[32599]: pgmap v1127: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:04:14.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:04:14.298+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:04:14.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:04:14.298+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:04:14.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:04:14.298+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:04:14.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:04:14.298+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:04:14.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:04:14.299+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:04:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:04:14 smithi167 ceph-mon[37140]: pgmap v1128: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:04:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:04:14 smithi138 ceph-mon[36954]: pgmap v1128: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:04:15.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:04:14 smithi100 ceph-mon[32599]: pgmap v1128: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:04:16.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:04:15 smithi167 ceph-mon[37140]: pgmap v1129: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:04:16.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:04:15 smithi138 ceph-mon[36954]: pgmap v1129: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:04:16.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:04:15 smithi100 ceph-mon[32599]: pgmap v1129: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:04:18.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:04:17 smithi138 ceph-mon[36954]: pgmap v1130: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:04:18.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:04:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:04:18.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:04:17 smithi100 ceph-mon[32599]: pgmap v1130: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:04:18.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:04:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:04:18.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:04:17 smithi167 ceph-mon[37140]: pgmap v1130: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:04:18.359 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:04:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:04:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:04:18 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:04:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:04:18 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:04:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:04:18 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:04:19.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:04:18 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:04:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:04:18 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:04:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:04:18 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:04:19.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:04:18 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:04:19.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:04:18 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:04:19.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:04:18 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:04:20.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:04:19 smithi100 ceph-mon[32599]: pgmap v1131: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 184 B/s wr, 7 op/s 2024-07-26T14:04:20.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:04:19 smithi100 ceph-mon[32599]: osdmap e123: 12 total, 12 up, 12 in 2024-07-26T14:04:20.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:04:19 smithi167 ceph-mon[37140]: pgmap v1131: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 184 B/s wr, 7 op/s 2024-07-26T14:04:20.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:04:19 smithi167 ceph-mon[37140]: osdmap e123: 12 total, 12 up, 12 in 2024-07-26T14:04:20.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:04:19 smithi138 ceph-mon[36954]: pgmap v1131: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 184 B/s wr, 7 op/s 2024-07-26T14:04:20.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:04:19 smithi138 ceph-mon[36954]: osdmap e123: 12 total, 12 up, 12 in 2024-07-26T14:04:22.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:04:21 smithi100 ceph-mon[32599]: pgmap v1133: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:04:22.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:04:21 smithi167 ceph-mon[37140]: pgmap v1133: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:04:22.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:04:21 smithi138 ceph-mon[36954]: pgmap v1133: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:04:24.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:04:23 smithi100 ceph-mon[32599]: pgmap v1134: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:04:24.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:04:23 smithi167 ceph-mon[37140]: pgmap v1134: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:04:24.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:04:23 smithi138 ceph-mon[36954]: pgmap v1134: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:04:26.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:04:25 smithi100 ceph-mon[32599]: pgmap v1135: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:04:26.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:04:25 smithi167 ceph-mon[37140]: pgmap v1135: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:04:26.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:04:25 smithi138 ceph-mon[36954]: pgmap v1135: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:04:28.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:04:27 smithi100 ceph-mon[32599]: pgmap v1136: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:04:28.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:04:27 smithi167 ceph-mon[37140]: pgmap v1136: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:04:28.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:04:27 smithi138 ceph-mon[36954]: pgmap v1136: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:04:29.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:04:29.309+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:04:29.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:04:29.310+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:04:29.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:04:29.310+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:04:29.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:04:29.310+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:04:29.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:04:29.310+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:04:30.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:04:29 smithi167 ceph-mon[37140]: pgmap v1137: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:04:30.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:04:29 smithi100 ceph-mon[32599]: pgmap v1137: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:04:30.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:04:29 smithi138 ceph-mon[36954]: pgmap v1137: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:04:32.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:04:31 smithi100 ceph-mon[32599]: pgmap v1138: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:04:32.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:04:31 smithi167 ceph-mon[37140]: pgmap v1138: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:04:32.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:04:31 smithi138 ceph-mon[36954]: pgmap v1138: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:04:33.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:04:33.152+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c006610 con 0x7fe5a81087a0 2024-07-26T14:04:33.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:04:33.153+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 12 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe57c006610 con 0x7fe5a81087a0 2024-07-26T14:04:33.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:04:33.153+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T14:04:33.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:04:33.153+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 12 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T14:04:34.242 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:04:33 smithi138 ceph-mon[36954]: pgmap v1139: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:04:34.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:04:33 smithi100 ceph-mon[32599]: pgmap v1139: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:04:34.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:04:33 smithi167 ceph-mon[37140]: pgmap v1139: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:04:36.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:04:35 smithi100 ceph-mon[32599]: pgmap v1140: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:04:36.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:04:35 smithi167 ceph-mon[37140]: pgmap v1140: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:04:36.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:04:35 smithi138 ceph-mon[36954]: pgmap v1140: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:04:38.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:04:37 smithi100 ceph-mon[32599]: pgmap v1141: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:04:38.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:04:37 smithi167 ceph-mon[37140]: pgmap v1141: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:04:38.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:04:37 smithi138 ceph-mon[36954]: pgmap v1141: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:04:40.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:04:39 smithi100 ceph-mon[32599]: pgmap v1142: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:04:40.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:04:39 smithi167 ceph-mon[37140]: pgmap v1142: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:04:40.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:04:39 smithi138 ceph-mon[36954]: pgmap v1142: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:04:42.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:04:41 smithi100 ceph-mon[32599]: pgmap v1143: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:04:42.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:04:41 smithi167 ceph-mon[37140]: pgmap v1143: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:04:42.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:04:41 smithi138 ceph-mon[36954]: pgmap v1143: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:04:44.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:04:43 smithi100 ceph-mon[32599]: pgmap v1144: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:04:44.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:04:43 smithi167 ceph-mon[37140]: pgmap v1144: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:04:44.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:04:44.312+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:04:44.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:04:44.313+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:04:44.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:04:44.313+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:04:44.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:04:44.313+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:04:44.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:04:44.313+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:04:44.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:04:43 smithi138 ceph-mon[36954]: pgmap v1144: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:04:46.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:04:45 smithi167 ceph-mon[37140]: pgmap v1145: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:04:46.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:04:45 smithi138 ceph-mon[36954]: pgmap v1145: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:04:46.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:04:45 smithi100 ceph-mon[32599]: pgmap v1145: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:04:48.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:04:47 smithi167 ceph-mon[37140]: pgmap v1146: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:04:48.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:04:47 smithi138 ceph-mon[36954]: pgmap v1146: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:04:48.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:04:47 smithi100 ceph-mon[32599]: pgmap v1146: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:04:50.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:04:50 smithi167 ceph-mon[37140]: pgmap v1147: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:04:50.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:04:50 smithi138 ceph-mon[36954]: pgmap v1147: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:04:50.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:04:50 smithi100 ceph-mon[32599]: pgmap v1147: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:04:52.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:04:52 smithi167 ceph-mon[37140]: pgmap v1148: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:04:52.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:04:52 smithi138 ceph-mon[36954]: pgmap v1148: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:04:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:04:52 smithi100 ceph-mon[32599]: pgmap v1148: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:04:54.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:04:54 smithi167 ceph-mon[37140]: pgmap v1149: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:04:54.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:04:54 smithi138 ceph-mon[36954]: pgmap v1149: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:04:54.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:04:54 smithi100 ceph-mon[32599]: pgmap v1149: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:04:56.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:04:56 smithi167 ceph-mon[37140]: pgmap v1150: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:04:56.296 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:04:56 smithi100 ceph-mon[32599]: pgmap v1150: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:04:56.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:04:56 smithi138 ceph-mon[36954]: pgmap v1150: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:04:58.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:04:58 smithi167 ceph-mon[37140]: pgmap v1151: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:04:58.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:04:58 smithi138 ceph-mon[36954]: pgmap v1151: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:04:58.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:04:58 smithi100 ceph-mon[32599]: pgmap v1151: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:04:59.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:04:59.328+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:04:59.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:04:59.328+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:04:59.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:04:59.328+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:04:59.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:04:59.328+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:04:59.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:04:59.328+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:05:00.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:05:00 smithi167 ceph-mon[37140]: pgmap v1152: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:05:00.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:05:00 smithi138 ceph-mon[36954]: pgmap v1152: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:05:00.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:05:00 smithi100 ceph-mon[32599]: pgmap v1152: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:05:02.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:05:02 smithi167 ceph-mon[37140]: pgmap v1153: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:05:02.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:05:02 smithi138 ceph-mon[36954]: pgmap v1153: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:05:02.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:05:02 smithi100 ceph-mon[32599]: pgmap v1153: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:05:04.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:05:04 smithi138 ceph-mon[36954]: pgmap v1154: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-07-26T14:05:04.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:05:04 smithi100 ceph-mon[32599]: pgmap v1154: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-07-26T14:05:04.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:05:04 smithi167 ceph-mon[37140]: pgmap v1154: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-07-26T14:05:06.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:05:06 smithi138 ceph-mon[36954]: pgmap v1155: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-07-26T14:05:06.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:05:06 smithi100 ceph-mon[32599]: pgmap v1155: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-07-26T14:05:06.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:05:06 smithi167 ceph-mon[37140]: pgmap v1155: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-07-26T14:05:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:05:08 smithi138 ceph-mon[36954]: pgmap v1156: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-07-26T14:05:08.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:05:08 smithi100 ceph-mon[32599]: pgmap v1156: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-07-26T14:05:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:05:08 smithi167 ceph-mon[37140]: pgmap v1156: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-07-26T14:05:10.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:05:10 smithi138 ceph-mon[36954]: pgmap v1157: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-07-26T14:05:10.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:05:10 smithi100 ceph-mon[32599]: pgmap v1157: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-07-26T14:05:10.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:05:10 smithi167 ceph-mon[37140]: pgmap v1157: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-07-26T14:05:12.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:05:12 smithi138 ceph-mon[36954]: pgmap v1158: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-07-26T14:05:12.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:05:12 smithi100 ceph-mon[32599]: pgmap v1158: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-07-26T14:05:12.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:05:12 smithi167 ceph-mon[37140]: pgmap v1158: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-07-26T14:05:14.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:05:14.335+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:05:14.337 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:05:14 smithi100 ceph-mon[32599]: pgmap v1159: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-07-26T14:05:14.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:05:14.336+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:05:14.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:05:14.336+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:05:14.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:05:14.336+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:05:14.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:05:14.336+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:05:14.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:05:14 smithi138 ceph-mon[36954]: pgmap v1159: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-07-26T14:05:14.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:05:14 smithi167 ceph-mon[37140]: pgmap v1159: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-07-26T14:05:16.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:05:16 smithi138 ceph-mon[36954]: pgmap v1160: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-07-26T14:05:16.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:05:16 smithi100 ceph-mon[32599]: pgmap v1160: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-07-26T14:05:16.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:05:16 smithi167 ceph-mon[37140]: pgmap v1160: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-07-26T14:05:18.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:05:18 smithi138 ceph-mon[36954]: pgmap v1161: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-07-26T14:05:18.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:05:18 smithi100 ceph-mon[32599]: pgmap v1161: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-07-26T14:05:18.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:05:18 smithi167 ceph-mon[37140]: pgmap v1161: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-07-26T14:05:19.333 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:05:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:05:19.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:05:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:05:19.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:05:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:05:20.359 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:05:20 smithi100 ceph-mon[32599]: pgmap v1162: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-07-26T14:05:20.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:05:20 smithi138 ceph-mon[36954]: pgmap v1162: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-07-26T14:05:20.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:05:20 smithi167 ceph-mon[37140]: pgmap v1162: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-07-26T14:05:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:05:22 smithi100 ceph-mon[32599]: pgmap v1163: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:05:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:05:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:05:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:05:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:05:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:05:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:05:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:05:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:05:22.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:05:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:05:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:05:22 smithi167 ceph-mon[37140]: pgmap v1163: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:05:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:05:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:05:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:05:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:05:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:05:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:05:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:05:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:05:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:05:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:05:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:05:22 smithi138 ceph-mon[36954]: pgmap v1163: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:05:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:05:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:05:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:05:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:05:23.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:05:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:05:23.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:05:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:05:23.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:05:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:05:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:05:24 smithi100 ceph-mon[32599]: pgmap v1164: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:05:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:05:24 smithi167 ceph-mon[37140]: pgmap v1164: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:05:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:05:24 smithi138 ceph-mon[36954]: pgmap v1164: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:05:26.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:05:26 smithi100 ceph-mon[32599]: pgmap v1165: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:05:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:05:26 smithi167 ceph-mon[37140]: pgmap v1165: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:05:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:05:26 smithi138 ceph-mon[36954]: pgmap v1165: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:05:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:05:28 smithi100 ceph-mon[32599]: pgmap v1166: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:05:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:05:28 smithi167 ceph-mon[37140]: pgmap v1166: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:05:29.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:05:28 smithi138 ceph-mon[36954]: pgmap v1166: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:05:29.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:05:29.344+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:05:29.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:05:29.344+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:05:29.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:05:29.344+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:05:29.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:05:29.344+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:05:29.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:05:29.344+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:05:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:05:30 smithi100 ceph-mon[32599]: pgmap v1167: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:05:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:05:30 smithi167 ceph-mon[37140]: pgmap v1167: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:05:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:05:30 smithi138 ceph-mon[36954]: pgmap v1167: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:05:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:05:32 smithi100 ceph-mon[32599]: pgmap v1168: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:05:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:05:32 smithi167 ceph-mon[37140]: pgmap v1168: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:05:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:05:32 smithi138 ceph-mon[36954]: pgmap v1168: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:05:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:05:34 smithi167 ceph-mon[37140]: pgmap v1169: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:05:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:05:34 smithi138 ceph-mon[36954]: pgmap v1169: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:05:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:05:34 smithi100 ceph-mon[32599]: pgmap v1169: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:05:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:05:36 smithi167 ceph-mon[37140]: pgmap v1170: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:05:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:05:36 smithi138 ceph-mon[36954]: pgmap v1170: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:05:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:05:36 smithi100 ceph-mon[32599]: pgmap v1170: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:05:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:05:38 smithi167 ceph-mon[37140]: pgmap v1171: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:05:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:05:38 smithi138 ceph-mon[36954]: pgmap v1171: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:05:39.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:05:38 smithi100 ceph-mon[32599]: pgmap v1171: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:05:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:05:40 smithi167 ceph-mon[37140]: pgmap v1172: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:05:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:05:40 smithi138 ceph-mon[36954]: pgmap v1172: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:05:41.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:05:40 smithi100 ceph-mon[32599]: pgmap v1172: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:05:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:05:42 smithi167 ceph-mon[37140]: pgmap v1173: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:05:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:05:42 smithi138 ceph-mon[36954]: pgmap v1173: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:05:43.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:05:42 smithi100 ceph-mon[32599]: pgmap v1173: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:05:44.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:05:44.360+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:05:44.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:05:44.360+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:05:44.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:05:44.360+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:05:44.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:05:44.360+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:05:44.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:05:44.360+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:05:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:05:44 smithi167 ceph-mon[37140]: pgmap v1174: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:05:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:05:44 smithi138 ceph-mon[36954]: pgmap v1174: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:05:45.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:05:44 smithi100 ceph-mon[32599]: pgmap v1174: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:05:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:05:46 smithi167 ceph-mon[37140]: pgmap v1175: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:05:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:05:46 smithi138 ceph-mon[36954]: pgmap v1175: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:05:47.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:05:46 smithi100 ceph-mon[32599]: pgmap v1175: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:05:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:05:48 smithi167 ceph-mon[37140]: pgmap v1176: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:05:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:05:48 smithi138 ceph-mon[36954]: pgmap v1176: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:05:49.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:05:48 smithi100 ceph-mon[32599]: pgmap v1176: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:05:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:05:50 smithi167 ceph-mon[37140]: pgmap v1177: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:05:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:05:50 smithi138 ceph-mon[36954]: pgmap v1177: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:05:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:05:50 smithi100 ceph-mon[32599]: pgmap v1177: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:05:52.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:05:51 smithi138 ceph-mon[36954]: pgmap v1178: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:05:52.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:05:51 smithi100 ceph-mon[32599]: pgmap v1178: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:05:52.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:05:51 smithi167 ceph-mon[37140]: pgmap v1178: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:05:54.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:05:53 smithi100 ceph-mon[32599]: pgmap v1179: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:05:54.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:05:53 smithi167 ceph-mon[37140]: pgmap v1179: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:05:54.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:05:53 smithi138 ceph-mon[36954]: pgmap v1179: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:05:56.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:05:55 smithi100 ceph-mon[32599]: pgmap v1180: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:05:56.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:05:55 smithi167 ceph-mon[37140]: pgmap v1180: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:05:56.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:05:55 smithi138 ceph-mon[36954]: pgmap v1180: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:05:58.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:05:57 smithi100 ceph-mon[32599]: pgmap v1181: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:05:58.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:05:57 smithi167 ceph-mon[37140]: pgmap v1181: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:05:58.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:05:57 smithi138 ceph-mon[36954]: pgmap v1181: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:05:59.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:05:59.376+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:05:59.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:05:59.376+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:05:59.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:05:59.376+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:05:59.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:05:59.376+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:05:59.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:05:59.376+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:06:00.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:05:59 smithi100 ceph-mon[32599]: pgmap v1182: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:06:00.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:05:59 smithi167 ceph-mon[37140]: pgmap v1182: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:06:00.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:05:59 smithi138 ceph-mon[36954]: pgmap v1182: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:06:02.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:01 smithi100 ceph-mon[32599]: pgmap v1183: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:06:02.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:01 smithi167 ceph-mon[37140]: pgmap v1183: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:06:02.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:01 smithi138 ceph-mon[36954]: pgmap v1183: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:06:03.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:06:03.165+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580003690 con 0x55c0ec887390 2024-07-26T14:06:03.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:06:03.165+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c003690 con 0x7fe5a81087a0 2024-07-26T14:06:03.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:06:03.165+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 13 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c003690 con 0x7fe5a81087a0 2024-07-26T14:06:03.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:06:03.165+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 13 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580003690 con 0x55c0ec887390 2024-07-26T14:06:04.243 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:03 smithi138 ceph-mon[36954]: pgmap v1184: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:06:04.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:03 smithi100 ceph-mon[32599]: pgmap v1184: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:06:04.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:03 smithi167 ceph-mon[37140]: pgmap v1184: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:06:06.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:05 smithi100 ceph-mon[32599]: pgmap v1185: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:06:06.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:05 smithi167 ceph-mon[37140]: pgmap v1185: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:06:06.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:05 smithi138 ceph-mon[36954]: pgmap v1185: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:06:08.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:07 smithi100 ceph-mon[32599]: pgmap v1186: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:06:08.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:07 smithi100 ceph-mon[32599]: osdmap e124: 12 total, 12 up, 12 in 2024-07-26T14:06:08.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:07 smithi167 ceph-mon[37140]: pgmap v1186: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:06:08.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:07 smithi167 ceph-mon[37140]: osdmap e124: 12 total, 12 up, 12 in 2024-07-26T14:06:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:07 smithi138 ceph-mon[36954]: pgmap v1186: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:06:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:07 smithi138 ceph-mon[36954]: osdmap e124: 12 total, 12 up, 12 in 2024-07-26T14:06:10.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:09 smithi100 ceph-mon[32599]: pgmap v1188: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:06:10.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:09 smithi167 ceph-mon[37140]: pgmap v1188: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:06:10.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:09 smithi138 ceph-mon[36954]: pgmap v1188: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:06:12.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:11 smithi100 ceph-mon[32599]: pgmap v1189: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:06:12.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:11 smithi167 ceph-mon[37140]: pgmap v1189: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:06:12.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:11 smithi138 ceph-mon[36954]: pgmap v1189: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:06:14.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:13 smithi100 ceph-mon[32599]: pgmap v1190: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:06:14.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:13 smithi100 ceph-mon[32599]: osdmap e125: 12 total, 12 up, 12 in 2024-07-26T14:06:14.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:13 smithi167 ceph-mon[37140]: pgmap v1190: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:06:14.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:13 smithi167 ceph-mon[37140]: osdmap e125: 12 total, 12 up, 12 in 2024-07-26T14:06:14.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:06:14.392+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:06:14.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:06:14.393+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:06:14.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:06:14.393+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:06:14.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:06:14.393+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:06:14.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:06:14.393+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:06:14.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:13 smithi138 ceph-mon[36954]: pgmap v1190: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:06:14.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:13 smithi138 ceph-mon[36954]: osdmap e125: 12 total, 12 up, 12 in 2024-07-26T14:06:16.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:15 smithi100 ceph-mon[32599]: pgmap v1192: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:06:16.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:15 smithi167 ceph-mon[37140]: pgmap v1192: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:06:16.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:15 smithi138 ceph-mon[36954]: pgmap v1192: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:06:18.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:17 smithi167 ceph-mon[37140]: pgmap v1193: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T14:06:18.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:17 smithi138 ceph-mon[36954]: pgmap v1193: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T14:06:18.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:17 smithi100 ceph-mon[32599]: pgmap v1193: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T14:06:20.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:19 smithi167 ceph-mon[37140]: pgmap v1194: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:06:20.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:19 smithi138 ceph-mon[36954]: pgmap v1194: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:06:20.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:19 smithi100 ceph-mon[32599]: pgmap v1194: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:06:22.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:22 smithi167 ceph-mon[37140]: pgmap v1195: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:06:22.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:06:22.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:22 smithi138 ceph-mon[36954]: pgmap v1195: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:06:22.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:06:22.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:22 smithi100 ceph-mon[32599]: pgmap v1195: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:06:22.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:06:23.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:06:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:06:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:06:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:06:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:23 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T14:06:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:23 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T14:06:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:06:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:06:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:06:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:06:23.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:23 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T14:06:23.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:23 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T14:06:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:06:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:06:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:06:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:06:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:23 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T14:06:23.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:23 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T14:06:24.010 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:24 smithi138 ceph-mon[36954]: pgmap v1196: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:06:24.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:24 smithi167 ceph-mon[37140]: pgmap v1196: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:06:24.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:24 smithi100 ceph-mon[32599]: pgmap v1196: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:06:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:26 smithi138 ceph-mon[36954]: pgmap v1197: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:06:26.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:06:26.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:06:26.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:06:26.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:06:26.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:06:26.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:06:26.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:26 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T14:06:26.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:26 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T14:06:26.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:06:26.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:06:26.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:06:26.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:06:26.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:06:26.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:06:26.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:26 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T14:06:26.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:26 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T14:06:26.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:06:26.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:06:26.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:06:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:26 smithi100 ceph-mon[32599]: pgmap v1197: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:06:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:06:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:06:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:06:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:06:26.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:06:26.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:06:26.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:26 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T14:06:26.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:26 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T14:06:26.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:06:26.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:06:26.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:06:26.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:06:26.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:06:26.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:06:26.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:26 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T14:06:26.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:26 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T14:06:26.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:06:26.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:06:26.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:06:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:26 smithi167 ceph-mon[37140]: pgmap v1197: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:06:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:06:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:06:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:06:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:06:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:06:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:06:27.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:26 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T14:06:27.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:26 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T14:06:27.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:06:27.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:06:27.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:06:27.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:06:27.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:06:27.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:06:27.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:26 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T14:06:27.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:26 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T14:06:27.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:06:27.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:06:27.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:06:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:28 smithi138 ceph-mon[36954]: pgmap v1198: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:06:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:28 smithi100 ceph-mon[32599]: pgmap v1198: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:06:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:28 smithi167 ceph-mon[37140]: pgmap v1198: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:06:29.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:06:29.403+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:06:29.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:06:29.404+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:06:29.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:06:29.404+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:06:29.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:06:29.404+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:06:29.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:06:29.404+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:06:30.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:30 smithi167 ceph-mon[37140]: pgmap v1199: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:06:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:30 smithi138 ceph-mon[36954]: pgmap v1199: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:06:30.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:30 smithi100 ceph-mon[32599]: pgmap v1199: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:06:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:32 smithi138 ceph-mon[36954]: pgmap v1200: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:06:32.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:32 smithi100 ceph-mon[32599]: pgmap v1200: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:06:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:32 smithi167 ceph-mon[37140]: pgmap v1200: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:06:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:34 smithi138 ceph-mon[36954]: pgmap v1201: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:06:34.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:34 smithi100 ceph-mon[32599]: pgmap v1201: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:06:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:34 smithi167 ceph-mon[37140]: pgmap v1201: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:06:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:36 smithi138 ceph-mon[36954]: pgmap v1202: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:06:36.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:36 smithi100 ceph-mon[32599]: pgmap v1202: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:06:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:36 smithi167 ceph-mon[37140]: pgmap v1202: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:06:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:38 smithi138 ceph-mon[36954]: pgmap v1203: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:06:38.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:38 smithi100 ceph-mon[32599]: pgmap v1203: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:06:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:38 smithi167 ceph-mon[37140]: pgmap v1203: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:06:40.875 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:40 smithi167 ceph-mon[37140]: pgmap v1204: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:06:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:40 smithi138 ceph-mon[36954]: pgmap v1204: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:06:40.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:40 smithi100 ceph-mon[32599]: pgmap v1204: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:06:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:42 smithi138 ceph-mon[36954]: pgmap v1205: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:06:42.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:42 smithi100 ceph-mon[32599]: pgmap v1205: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:06:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:42 smithi167 ceph-mon[37140]: pgmap v1205: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:06:44.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:06:44.420+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:06:44.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:06:44.420+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:06:44.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:06:44.420+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:06:44.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:06:44.420+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:06:44.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:06:44.420+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:06:44.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:44 smithi100 ceph-mon[32599]: pgmap v1206: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:06:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:44 smithi138 ceph-mon[36954]: pgmap v1206: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:06:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:44 smithi167 ceph-mon[37140]: pgmap v1206: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:06:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:46 smithi138 ceph-mon[36954]: pgmap v1207: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:06:46.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:46 smithi100 ceph-mon[32599]: pgmap v1207: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:06:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:46 smithi167 ceph-mon[37140]: pgmap v1207: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:06:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:48 smithi138 ceph-mon[36954]: pgmap v1208: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:06:48.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:48 smithi100 ceph-mon[32599]: pgmap v1208: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:06:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:48 smithi167 ceph-mon[37140]: pgmap v1208: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:06:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:50 smithi138 ceph-mon[36954]: pgmap v1209: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:06:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:50 smithi100 ceph-mon[32599]: pgmap v1209: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:06:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:50 smithi167 ceph-mon[37140]: pgmap v1209: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:06:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:52 smithi138 ceph-mon[36954]: pgmap v1210: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:06:52.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:52 smithi100 ceph-mon[32599]: pgmap v1210: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:06:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:52 smithi167 ceph-mon[37140]: pgmap v1210: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:06:54.604 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:54 smithi138 ceph-mon[36954]: pgmap v1211: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:06:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:54 smithi100 ceph-mon[32599]: pgmap v1211: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:06:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:54 smithi167 ceph-mon[37140]: pgmap v1211: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:06:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:56 smithi138 ceph-mon[36954]: pgmap v1212: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:06:56.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:56 smithi100 ceph-mon[32599]: pgmap v1212: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:06:57.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:56 smithi167 ceph-mon[37140]: pgmap v1212: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:06:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:06:58 smithi138 ceph-mon[36954]: pgmap v1213: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:06:58.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:06:58 smithi100 ceph-mon[32599]: pgmap v1213: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:06:59.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:06:58 smithi167 ceph-mon[37140]: pgmap v1213: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:06:59.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:06:59.436+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:06:59.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:06:59.437+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:06:59.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:06:59.437+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:06:59.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:06:59.437+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:06:59.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:06:59.437+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:07:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:07:00 smithi138 ceph-mon[36954]: pgmap v1214: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:07:00.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:07:00 smithi100 ceph-mon[32599]: pgmap v1214: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:07:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:07:00 smithi167 ceph-mon[37140]: pgmap v1214: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:07:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:07:02 smithi138 ceph-mon[36954]: pgmap v1215: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:07:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:07:02 smithi100 ceph-mon[32599]: pgmap v1215: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:07:03.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:07:02 smithi167 ceph-mon[37140]: pgmap v1215: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:07:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:07:04 smithi138 ceph-mon[36954]: pgmap v1216: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:07:04.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:07:04 smithi100 ceph-mon[32599]: pgmap v1216: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:07:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:07:04 smithi167 ceph-mon[37140]: pgmap v1216: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:07:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:07:06 smithi138 ceph-mon[36954]: pgmap v1217: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:07:06.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:07:06 smithi100 ceph-mon[32599]: pgmap v1217: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:07:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:07:06 smithi167 ceph-mon[37140]: pgmap v1217: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:07:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:07:08 smithi138 ceph-mon[36954]: pgmap v1218: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:07:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:07:08 smithi138 ceph-mon[36954]: osdmap e126: 12 total, 12 up, 12 in 2024-07-26T14:07:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:07:08 smithi100 ceph-mon[32599]: pgmap v1218: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:07:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:07:08 smithi100 ceph-mon[32599]: osdmap e126: 12 total, 12 up, 12 in 2024-07-26T14:07:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:07:08 smithi167 ceph-mon[37140]: pgmap v1218: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:07:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:07:08 smithi167 ceph-mon[37140]: osdmap e126: 12 total, 12 up, 12 in 2024-07-26T14:07:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:07:10 smithi138 ceph-mon[36954]: pgmap v1220: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:07:10.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:07:10 smithi100 ceph-mon[32599]: pgmap v1220: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:07:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:07:10 smithi167 ceph-mon[37140]: pgmap v1220: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:07:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:07:12 smithi138 ceph-mon[36954]: pgmap v1221: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:07:12.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:07:12 smithi100 ceph-mon[32599]: pgmap v1221: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:07:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:07:12 smithi167 ceph-mon[37140]: pgmap v1221: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:07:14.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:07:14.447+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:07:14.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:07:14.448+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:07:14.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:07:14.448+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:07:14.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:07:14.448+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:07:14.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:07:14.448+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:07:14.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:07:14 smithi100 ceph-mon[32599]: pgmap v1222: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:07:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:07:14 smithi100 ceph-mon[32599]: osdmap e127: 12 total, 12 up, 12 in 2024-07-26T14:07:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:07:14 smithi138 ceph-mon[36954]: pgmap v1222: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:07:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:07:14 smithi138 ceph-mon[36954]: osdmap e127: 12 total, 12 up, 12 in 2024-07-26T14:07:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:07:14 smithi167 ceph-mon[37140]: pgmap v1222: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:07:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:07:14 smithi167 ceph-mon[37140]: osdmap e127: 12 total, 12 up, 12 in 2024-07-26T14:07:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:07:16 smithi138 ceph-mon[36954]: pgmap v1224: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:07:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:07:16 smithi100 ceph-mon[32599]: pgmap v1224: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:07:17.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:07:16 smithi167 ceph-mon[37140]: pgmap v1224: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:07:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:07:18 smithi138 ceph-mon[36954]: pgmap v1225: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 220 B/s wr, 6 op/s 2024-07-26T14:07:18.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:07:18 smithi100 ceph-mon[32599]: pgmap v1225: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 220 B/s wr, 6 op/s 2024-07-26T14:07:19.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:07:18 smithi167 ceph-mon[37140]: pgmap v1225: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 220 B/s wr, 6 op/s 2024-07-26T14:07:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:07:20 smithi138 ceph-mon[36954]: pgmap v1226: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:07:20.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:07:20 smithi100 ceph-mon[32599]: pgmap v1226: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:07:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:07:20 smithi167 ceph-mon[37140]: pgmap v1226: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:07:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:07:22 smithi138 ceph-mon[36954]: pgmap v1227: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:07:22.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:07:22 smithi100 ceph-mon[32599]: pgmap v1227: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:07:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:07:22 smithi167 ceph-mon[37140]: pgmap v1227: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:07:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:07:24 smithi138 ceph-mon[36954]: pgmap v1228: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:07:24.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:07:24 smithi100 ceph-mon[32599]: pgmap v1228: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:07:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:07:24 smithi167 ceph-mon[37140]: pgmap v1228: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:07:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:07:26 smithi138 ceph-mon[36954]: pgmap v1229: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:07:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:07:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:07:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:07:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:07:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:07:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:07:26.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:07:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:07:26.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:07:26 smithi100 ceph-mon[32599]: pgmap v1229: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:07:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:07:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:07:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:07:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:07:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:07:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:07:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:07:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:07:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:07:26 smithi167 ceph-mon[37140]: pgmap v1229: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:07:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:07:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:07:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:07:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:07:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:07:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:07:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:07:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:07:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:07:28 smithi138 ceph-mon[36954]: pgmap v1230: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:07:28.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:07:28 smithi100 ceph-mon[32599]: pgmap v1230: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:07:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:07:28 smithi167 ceph-mon[37140]: pgmap v1230: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:07:29.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:07:29.463+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:07:29.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:07:29.464+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:07:29.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:07:29.464+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:07:29.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:07:29.464+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:07:29.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:07:29.464+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:07:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:07:30 smithi138 ceph-mon[36954]: pgmap v1231: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:07:30.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:07:30 smithi100 ceph-mon[32599]: pgmap v1231: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:07:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:07:30 smithi167 ceph-mon[37140]: pgmap v1231: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:07:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:07:32 smithi138 ceph-mon[36954]: pgmap v1232: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:07:32.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:07:32 smithi100 ceph-mon[32599]: pgmap v1232: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:07:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:07:32 smithi167 ceph-mon[37140]: pgmap v1232: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:07:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:07:34 smithi138 ceph-mon[36954]: pgmap v1233: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:07:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:07:34 smithi100 ceph-mon[32599]: pgmap v1233: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:07:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:07:34 smithi167 ceph-mon[37140]: pgmap v1233: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:07:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:07:36 smithi138 ceph-mon[36954]: pgmap v1234: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:07:36.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:07:36 smithi100 ceph-mon[32599]: pgmap v1234: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:07:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:07:36 smithi167 ceph-mon[37140]: pgmap v1234: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:07:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:07:38 smithi138 ceph-mon[36954]: pgmap v1235: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:07:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:07:38 smithi100 ceph-mon[32599]: pgmap v1235: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:07:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:07:38 smithi167 ceph-mon[37140]: pgmap v1235: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:07:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:07:40 smithi138 ceph-mon[36954]: pgmap v1236: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:07:40.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:07:40 smithi100 ceph-mon[32599]: pgmap v1236: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:07:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:07:40 smithi167 ceph-mon[37140]: pgmap v1236: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:07:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:07:42 smithi138 ceph-mon[36954]: pgmap v1237: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:07:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:07:42 smithi100 ceph-mon[32599]: pgmap v1237: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:07:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:07:42 smithi167 ceph-mon[37140]: pgmap v1237: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:07:44.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:07:44.479+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:07:44.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:07:44.480+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:07:44.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:07:44.480+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:07:44.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:07:44.480+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:07:44.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:07:44.480+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:07:44.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:07:44 smithi100 ceph-mon[32599]: pgmap v1238: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:07:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:07:44 smithi138 ceph-mon[36954]: pgmap v1238: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:07:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:07:44 smithi167 ceph-mon[37140]: pgmap v1238: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:07:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:07:46 smithi100 ceph-mon[32599]: pgmap v1239: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:07:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:07:46 smithi167 ceph-mon[37140]: pgmap v1239: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:07:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:07:46 smithi138 ceph-mon[36954]: pgmap v1239: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:07:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:07:48 smithi100 ceph-mon[32599]: pgmap v1240: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:07:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:07:48 smithi167 ceph-mon[37140]: pgmap v1240: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:07:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:07:48 smithi138 ceph-mon[36954]: pgmap v1240: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:07:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:07:50 smithi100 ceph-mon[32599]: pgmap v1241: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:07:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:07:50 smithi167 ceph-mon[37140]: pgmap v1241: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:07:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:07:50 smithi138 ceph-mon[36954]: pgmap v1241: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:07:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:07:52 smithi100 ceph-mon[32599]: pgmap v1242: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:07:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:07:52 smithi167 ceph-mon[37140]: pgmap v1242: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:07:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:07:52 smithi138 ceph-mon[36954]: pgmap v1242: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:07:54.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:07:54 smithi100 ceph-mon[32599]: pgmap v1243: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:07:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:07:54 smithi167 ceph-mon[37140]: pgmap v1243: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:07:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:07:54 smithi138 ceph-mon[36954]: pgmap v1243: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:07:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:07:56 smithi100 ceph-mon[32599]: pgmap v1244: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:07:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:07:56 smithi167 ceph-mon[37140]: pgmap v1244: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:07:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:07:56 smithi138 ceph-mon[36954]: pgmap v1244: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:07:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:07:58 smithi100 ceph-mon[32599]: pgmap v1245: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:07:59.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:07:58 smithi167 ceph-mon[37140]: pgmap v1245: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:07:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:07:58 smithi138 ceph-mon[36954]: pgmap v1245: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:07:59.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:07:59.493+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:07:59.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:07:59.493+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:07:59.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:07:59.493+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:07:59.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:07:59.493+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:07:59.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:07:59.493+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:08:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:08:00 smithi100 ceph-mon[32599]: pgmap v1246: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:08:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:08:00 smithi167 ceph-mon[37140]: pgmap v1246: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:08:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:08:00 smithi138 ceph-mon[36954]: pgmap v1246: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:08:02.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:08:02 smithi100 ceph-mon[32599]: pgmap v1247: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:08:03.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:08:02 smithi167 ceph-mon[37140]: pgmap v1247: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:08:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:08:02 smithi138 ceph-mon[36954]: pgmap v1247: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:08:04.992 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:08:04 smithi138 ceph-mon[36954]: pgmap v1248: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:08:04.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:08:04 smithi100 ceph-mon[32599]: pgmap v1248: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:08:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:08:04 smithi167 ceph-mon[37140]: pgmap v1248: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:08:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:08:06 smithi100 ceph-mon[32599]: pgmap v1249: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:08:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:08:06 smithi167 ceph-mon[37140]: pgmap v1249: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:08:07.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:08:06 smithi138 ceph-mon[36954]: pgmap v1249: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:08:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:08:08 smithi100 ceph-mon[32599]: pgmap v1250: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:08:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:08:08 smithi100 ceph-mon[32599]: osdmap e128: 12 total, 12 up, 12 in 2024-07-26T14:08:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:08:08 smithi167 ceph-mon[37140]: pgmap v1250: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:08:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:08:08 smithi167 ceph-mon[37140]: osdmap e128: 12 total, 12 up, 12 in 2024-07-26T14:08:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:08:08 smithi138 ceph-mon[36954]: pgmap v1250: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:08:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:08:08 smithi138 ceph-mon[36954]: osdmap e128: 12 total, 12 up, 12 in 2024-07-26T14:08:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:08:10 smithi100 ceph-mon[32599]: pgmap v1252: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:08:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:08:10 smithi167 ceph-mon[37140]: pgmap v1252: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:08:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:08:10 smithi138 ceph-mon[36954]: pgmap v1252: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:08:12.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:08:12 smithi100 ceph-mon[32599]: pgmap v1253: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:08:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:08:12 smithi167 ceph-mon[37140]: pgmap v1253: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:08:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:08:12 smithi138 ceph-mon[36954]: pgmap v1253: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:08:14.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:08:14.509+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:08:14.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:08:14.509+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:08:14.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:08:14.509+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:08:14.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:08:14.509+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:08:14.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:08:14.509+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:08:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:08:14 smithi100 ceph-mon[32599]: pgmap v1254: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:08:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:08:14 smithi100 ceph-mon[32599]: osdmap e129: 12 total, 12 up, 12 in 2024-07-26T14:08:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:08:14 smithi167 ceph-mon[37140]: pgmap v1254: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:08:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:08:14 smithi167 ceph-mon[37140]: osdmap e129: 12 total, 12 up, 12 in 2024-07-26T14:08:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:08:14 smithi138 ceph-mon[36954]: pgmap v1254: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:08:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:08:14 smithi138 ceph-mon[36954]: osdmap e129: 12 total, 12 up, 12 in 2024-07-26T14:08:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:08:16 smithi100 ceph-mon[32599]: pgmap v1256: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:08:17.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:08:16 smithi167 ceph-mon[37140]: pgmap v1256: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:08:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:08:16 smithi138 ceph-mon[36954]: pgmap v1256: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:08:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:08:18 smithi100 ceph-mon[32599]: pgmap v1257: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 222 B/s wr, 6 op/s 2024-07-26T14:08:19.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:08:18 smithi167 ceph-mon[37140]: pgmap v1257: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 222 B/s wr, 6 op/s 2024-07-26T14:08:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:08:18 smithi138 ceph-mon[36954]: pgmap v1257: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 222 B/s wr, 6 op/s 2024-07-26T14:08:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:08:20 smithi167 ceph-mon[37140]: pgmap v1258: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:08:21.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:08:20 smithi138 ceph-mon[36954]: pgmap v1258: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:08:21.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:08:20 smithi100 ceph-mon[32599]: pgmap v1258: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:08:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:08:22 smithi167 ceph-mon[37140]: pgmap v1259: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:08:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:08:22 smithi138 ceph-mon[36954]: pgmap v1259: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:08:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:08:22 smithi100 ceph-mon[32599]: pgmap v1259: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:08:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:08:24 smithi167 ceph-mon[37140]: pgmap v1260: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:08:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:08:24 smithi138 ceph-mon[36954]: pgmap v1260: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:08:25.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:08:24 smithi100 ceph-mon[32599]: pgmap v1260: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:08:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:08:26 smithi167 ceph-mon[37140]: pgmap v1261: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:08:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:08:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:08:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:08:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:08:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:08:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:08:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:08:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:08:27.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:08:26 smithi100 ceph-mon[32599]: pgmap v1261: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:08:27.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:08:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:08:27.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:08:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:08:27.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:08:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:08:27.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:08:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:08:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:08:26 smithi138 ceph-mon[36954]: pgmap v1261: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:08:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:08:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:08:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:08:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:08:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:08:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:08:27.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:08:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:08:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:08:28 smithi167 ceph-mon[37140]: pgmap v1262: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:08:29.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:08:28 smithi138 ceph-mon[36954]: pgmap v1262: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:08:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:08:28 smithi100 ceph-mon[32599]: pgmap v1262: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:08:29.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:08:29.525+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:08:29.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:08:29.525+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:08:29.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:08:29.525+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:08:29.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:08:29.525+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:08:29.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:08:29.525+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:08:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:08:30 smithi167 ceph-mon[37140]: pgmap v1263: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:08:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:08:30 smithi138 ceph-mon[36954]: pgmap v1263: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:08:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:08:30 smithi100 ceph-mon[32599]: pgmap v1263: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:08:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:08:32 smithi167 ceph-mon[37140]: pgmap v1264: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:08:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:08:32 smithi138 ceph-mon[36954]: pgmap v1264: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:08:33.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:08:32 smithi100 ceph-mon[32599]: pgmap v1264: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:08:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:08:34 smithi138 ceph-mon[36954]: pgmap v1265: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:08:35.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:08:34 smithi100 ceph-mon[32599]: pgmap v1265: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:08:35.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:08:34 smithi167 ceph-mon[37140]: pgmap v1265: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:08:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:08:36 smithi138 ceph-mon[36954]: pgmap v1266: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:08:37.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:08:36 smithi100 ceph-mon[32599]: pgmap v1266: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:08:37.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:08:36 smithi167 ceph-mon[37140]: pgmap v1266: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:08:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:08:38 smithi138 ceph-mon[36954]: pgmap v1267: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:08:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:08:38 smithi100 ceph-mon[32599]: pgmap v1267: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:08:39.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:08:38 smithi167 ceph-mon[37140]: pgmap v1267: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:08:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:08:40 smithi138 ceph-mon[36954]: pgmap v1268: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:08:41.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:08:40 smithi100 ceph-mon[32599]: pgmap v1268: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:08:41.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:08:40 smithi167 ceph-mon[37140]: pgmap v1268: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:08:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:08:42 smithi138 ceph-mon[36954]: pgmap v1269: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:08:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:08:42 smithi100 ceph-mon[32599]: pgmap v1269: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:08:43.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:08:42 smithi167 ceph-mon[37140]: pgmap v1269: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:08:44.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:08:44.529+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:08:44.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:08:44.529+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:08:44.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:08:44.529+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:08:44.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:08:44.529+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:08:44.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:08:44.530+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:08:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:08:44 smithi138 ceph-mon[36954]: pgmap v1270: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:08:45.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:08:44 smithi100 ceph-mon[32599]: pgmap v1270: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:08:45.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:08:44 smithi167 ceph-mon[37140]: pgmap v1270: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:08:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:08:46 smithi138 ceph-mon[36954]: pgmap v1271: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:08:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:08:46 smithi100 ceph-mon[32599]: pgmap v1271: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:08:47.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:08:46 smithi167 ceph-mon[37140]: pgmap v1271: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:08:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:08:48 smithi138 ceph-mon[36954]: pgmap v1272: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:08:49.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:08:48 smithi100 ceph-mon[32599]: pgmap v1272: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:08:49.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:08:48 smithi167 ceph-mon[37140]: pgmap v1272: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:08:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:08:50 smithi138 ceph-mon[36954]: pgmap v1273: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:08:51.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:08:50 smithi100 ceph-mon[32599]: pgmap v1273: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:08:51.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:08:50 smithi167 ceph-mon[37140]: pgmap v1273: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:08:52.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:08:51 smithi138 ceph-mon[36954]: pgmap v1274: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:08:52.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:08:51 smithi100 ceph-mon[32599]: pgmap v1274: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:08:52.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:08:51 smithi167 ceph-mon[37140]: pgmap v1274: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:08:54.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:08:54 smithi167 ceph-mon[37140]: pgmap v1275: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:08:54.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:08:54 smithi138 ceph-mon[36954]: pgmap v1275: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:08:54.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:08:54 smithi100 ceph-mon[32599]: pgmap v1275: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:08:56.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:08:56 smithi167 ceph-mon[37140]: pgmap v1276: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:08:56.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:08:56 smithi138 ceph-mon[36954]: pgmap v1276: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:08:56.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:08:56 smithi100 ceph-mon[32599]: pgmap v1276: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:08:58.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:08:58 smithi167 ceph-mon[37140]: pgmap v1277: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:08:58.296 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:08:58 smithi100 ceph-mon[32599]: pgmap v1277: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:08:58.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:08:58 smithi138 ceph-mon[36954]: pgmap v1277: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:08:59.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:08:59.545+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:08:59.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:08:59.545+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:08:59.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:08:59.545+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:08:59.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:08:59.545+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:08:59.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:08:59.546+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:09:00.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:00 smithi167 ceph-mon[37140]: pgmap v1278: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:09:00.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:00 smithi138 ceph-mon[36954]: pgmap v1278: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:09:00.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:00 smithi100 ceph-mon[32599]: pgmap v1278: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:09:02.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:02 smithi138 ceph-mon[36954]: pgmap v1279: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:09:02.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:02 smithi100 ceph-mon[32599]: pgmap v1279: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:09:02.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:02 smithi167 ceph-mon[37140]: pgmap v1279: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:09:03.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:09:03.190+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c003690 con 0x7fe5a81087a0 2024-07-26T14:09:03.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:09:03.190+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580003690 con 0x55c0ec887390 2024-07-26T14:09:03.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:09:03.191+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 14 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c003690 con 0x7fe5a81087a0 2024-07-26T14:09:03.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:09:03.191+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 14 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580003690 con 0x55c0ec887390 2024-07-26T14:09:04.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:04 smithi138 ceph-mon[36954]: pgmap v1280: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:09:04.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:04 smithi100 ceph-mon[32599]: pgmap v1280: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:09:04.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:04 smithi167 ceph-mon[37140]: pgmap v1280: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:09:06.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:06 smithi138 ceph-mon[36954]: pgmap v1281: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:09:06.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:06 smithi100 ceph-mon[32599]: pgmap v1281: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:09:06.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:06 smithi167 ceph-mon[37140]: pgmap v1281: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:09:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:08 smithi138 ceph-mon[36954]: pgmap v1282: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:09:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:08 smithi138 ceph-mon[36954]: osdmap e130: 12 total, 12 up, 12 in 2024-07-26T14:09:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:08 smithi100 ceph-mon[32599]: pgmap v1282: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:09:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:08 smithi100 ceph-mon[32599]: osdmap e130: 12 total, 12 up, 12 in 2024-07-26T14:09:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:08 smithi167 ceph-mon[37140]: pgmap v1282: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:09:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:08 smithi167 ceph-mon[37140]: osdmap e130: 12 total, 12 up, 12 in 2024-07-26T14:09:10.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:10 smithi138 ceph-mon[36954]: pgmap v1284: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:09:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:10 smithi100 ceph-mon[32599]: pgmap v1284: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:09:10.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:10 smithi167 ceph-mon[37140]: pgmap v1284: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:09:12.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:12 smithi138 ceph-mon[36954]: pgmap v1285: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:09:12.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:12 smithi100 ceph-mon[32599]: pgmap v1285: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:09:12.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:12 smithi167 ceph-mon[37140]: pgmap v1285: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:09:14.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:14 smithi138 ceph-mon[36954]: pgmap v1286: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:09:14.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:14 smithi138 ceph-mon[36954]: osdmap e131: 12 total, 12 up, 12 in 2024-07-26T14:09:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:14 smithi100 ceph-mon[32599]: pgmap v1286: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:09:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:14 smithi100 ceph-mon[32599]: osdmap e131: 12 total, 12 up, 12 in 2024-07-26T14:09:14.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:14 smithi167 ceph-mon[37140]: pgmap v1286: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:09:14.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:14 smithi167 ceph-mon[37140]: osdmap e131: 12 total, 12 up, 12 in 2024-07-26T14:09:14.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:09:14.561+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:09:14.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:09:14.561+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:09:14.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:09:14.562+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:09:14.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:09:14.562+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:09:14.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:09:14.562+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:09:16.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:16 smithi138 ceph-mon[36954]: pgmap v1288: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:09:16.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:16 smithi100 ceph-mon[32599]: pgmap v1288: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:09:16.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:16 smithi167 ceph-mon[37140]: pgmap v1288: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:09:18.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:18 smithi138 ceph-mon[36954]: pgmap v1289: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T14:09:18.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:18 smithi100 ceph-mon[32599]: pgmap v1289: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T14:09:18.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:18 smithi167 ceph-mon[37140]: pgmap v1289: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T14:09:20.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:20 smithi138 ceph-mon[36954]: pgmap v1290: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:09:20.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:20 smithi100 ceph-mon[32599]: pgmap v1290: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:09:20.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:20 smithi167 ceph-mon[37140]: pgmap v1290: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:09:22.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:22 smithi138 ceph-mon[36954]: pgmap v1291: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:09:22.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:22 smithi100 ceph-mon[32599]: pgmap v1291: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:09:22.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:22 smithi167 ceph-mon[37140]: pgmap v1291: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:09:24.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:24 smithi138 ceph-mon[36954]: pgmap v1292: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:09:24.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:24 smithi100 ceph-mon[32599]: pgmap v1292: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:09:24.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:24 smithi167 ceph-mon[37140]: pgmap v1292: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:09:26.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:26 smithi138 ceph-mon[36954]: pgmap v1293: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:09:26.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:26 smithi100 ceph-mon[32599]: pgmap v1293: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:09:26.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:26 smithi167 ceph-mon[37140]: pgmap v1293: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:09:27.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:09:27.471 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:09:27.486 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:09:28.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:28 smithi138 ceph-mon[36954]: pgmap v1294: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:09:28.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:28 smithi100 ceph-mon[32599]: pgmap v1294: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:09:28.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:28 smithi167 ceph-mon[37140]: pgmap v1294: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:09:29.577 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:09:29.577 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:09:29.577 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:09:29.578 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:09:29.578 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:09:29.578 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:09:29.578 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:09:29.578 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:09:29.578 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:09:29.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:09:29.575+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:09:29.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:09:29.576+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:09:29.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:09:29.576+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:09:29.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:09:29.576+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:09:29.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:09:29.576+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:09:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:09:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:09:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:09:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:09:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:09:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:09:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:09:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:09:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:09:29.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:09:29.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:09:29.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:09:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:09:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:09:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:09:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:09:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:09:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:09:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:30 smithi167 ceph-mon[37140]: pgmap v1295: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:09:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:30 smithi138 ceph-mon[36954]: pgmap v1295: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:09:31.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:30 smithi100 ceph-mon[32599]: pgmap v1295: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:09:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:32 smithi167 ceph-mon[37140]: pgmap v1296: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:09:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:32 smithi138 ceph-mon[36954]: pgmap v1296: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:09:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:32 smithi100 ceph-mon[32599]: pgmap v1296: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:09:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:34 smithi167 ceph-mon[37140]: pgmap v1297: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:09:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:34 smithi138 ceph-mon[36954]: pgmap v1297: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:09:35.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:34 smithi100 ceph-mon[32599]: pgmap v1297: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:09:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:36 smithi167 ceph-mon[37140]: pgmap v1298: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:09:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:36 smithi138 ceph-mon[36954]: pgmap v1298: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:09:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:36 smithi100 ceph-mon[32599]: pgmap v1298: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:09:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:38 smithi167 ceph-mon[37140]: pgmap v1299: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:09:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:38 smithi138 ceph-mon[36954]: pgmap v1299: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:09:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:38 smithi100 ceph-mon[32599]: pgmap v1299: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:09:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:40 smithi167 ceph-mon[37140]: pgmap v1300: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:09:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:40 smithi138 ceph-mon[36954]: pgmap v1300: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:09:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:40 smithi100 ceph-mon[32599]: pgmap v1300: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:09:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:42 smithi167 ceph-mon[37140]: pgmap v1301: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:09:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:42 smithi138 ceph-mon[36954]: pgmap v1301: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:09:43.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:42 smithi100 ceph-mon[32599]: pgmap v1301: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:09:44.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:09:44.578+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:09:44.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:09:44.578+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:09:44.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:09:44.578+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:09:44.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:09:44.578+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:09:44.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:09:44.578+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:09:44.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:44 smithi100 ceph-mon[32599]: pgmap v1302: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:09:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:44 smithi167 ceph-mon[37140]: pgmap v1302: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:09:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:44 smithi138 ceph-mon[36954]: pgmap v1302: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:09:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:46 smithi138 ceph-mon[36954]: pgmap v1303: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:09:47.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:46 smithi100 ceph-mon[32599]: pgmap v1303: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:09:47.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:46 smithi167 ceph-mon[37140]: pgmap v1303: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:09:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:48 smithi138 ceph-mon[36954]: pgmap v1304: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:09:49.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:48 smithi100 ceph-mon[32599]: pgmap v1304: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:09:49.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:48 smithi167 ceph-mon[37140]: pgmap v1304: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:09:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:50 smithi138 ceph-mon[36954]: pgmap v1305: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:09:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:50 smithi100 ceph-mon[32599]: pgmap v1305: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:09:51.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:50 smithi167 ceph-mon[37140]: pgmap v1305: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:09:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:52 smithi138 ceph-mon[36954]: pgmap v1306: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:09:53.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:52 smithi100 ceph-mon[32599]: pgmap v1306: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:09:53.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:52 smithi167 ceph-mon[37140]: pgmap v1306: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:09:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:54 smithi138 ceph-mon[36954]: pgmap v1307: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:09:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:54 smithi100 ceph-mon[32599]: pgmap v1307: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:09:55.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:54 smithi167 ceph-mon[37140]: pgmap v1307: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:09:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:56 smithi138 ceph-mon[36954]: pgmap v1308: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:09:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:56 smithi100 ceph-mon[32599]: pgmap v1308: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:09:57.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:56 smithi167 ceph-mon[37140]: pgmap v1308: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:09:59.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:09:58 smithi100 ceph-mon[32599]: pgmap v1309: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:09:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:09:58 smithi138 ceph-mon[36954]: pgmap v1309: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:09:59.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:09:58 smithi167 ceph-mon[37140]: pgmap v1309: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:09:59.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:09:59.588+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:09:59.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:09:59.589+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:09:59.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:09:59.589+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:09:59.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:09:59.589+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:09:59.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:09:59.589+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:10:01.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:00 smithi167 ceph-mon[37140]: pgmap v1310: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:10:01.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:00 smithi167 ceph-mon[37140]: overall HEALTH_OK 2024-07-26T14:10:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:00 smithi138 ceph-mon[36954]: pgmap v1310: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:10:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:00 smithi138 ceph-mon[36954]: overall HEALTH_OK 2024-07-26T14:10:01.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:00 smithi100 ceph-mon[32599]: pgmap v1310: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:10:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:00 smithi100 ceph-mon[32599]: overall HEALTH_OK 2024-07-26T14:10:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:02 smithi138 ceph-mon[36954]: pgmap v1311: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:10:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:02 smithi100 ceph-mon[32599]: pgmap v1311: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:10:03.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:02 smithi167 ceph-mon[37140]: pgmap v1311: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:10:05.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:04 smithi138 ceph-mon[36954]: pgmap v1312: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:10:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:04 smithi100 ceph-mon[32599]: pgmap v1312: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:10:05.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:04 smithi167 ceph-mon[37140]: pgmap v1312: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:10:07.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:06 smithi138 ceph-mon[36954]: pgmap v1313: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:10:07.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:06 smithi100 ceph-mon[32599]: pgmap v1313: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:10:07.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:06 smithi167 ceph-mon[37140]: pgmap v1313: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:10:08.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:07 smithi138 ceph-mon[36954]: osdmap e132: 12 total, 12 up, 12 in 2024-07-26T14:10:08.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:07 smithi138 ceph-mon[36954]: pgmap v1315: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:10:08.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:07 smithi100 ceph-mon[32599]: osdmap e132: 12 total, 12 up, 12 in 2024-07-26T14:10:08.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:07 smithi100 ceph-mon[32599]: pgmap v1315: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:10:08.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:07 smithi167 ceph-mon[37140]: osdmap e132: 12 total, 12 up, 12 in 2024-07-26T14:10:08.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:07 smithi167 ceph-mon[37140]: pgmap v1315: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:10:10.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:10 smithi138 ceph-mon[36954]: pgmap v1316: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:10:10.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:10 smithi100 ceph-mon[32599]: pgmap v1316: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:10:10.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:10 smithi167 ceph-mon[37140]: pgmap v1316: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:10:12.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:12 smithi138 ceph-mon[36954]: pgmap v1317: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:10:12.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:12 smithi100 ceph-mon[32599]: pgmap v1317: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:10:12.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:12 smithi167 ceph-mon[37140]: pgmap v1317: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:10:14.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:14 smithi138 ceph-mon[36954]: pgmap v1318: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:10:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:14 smithi100 ceph-mon[32599]: pgmap v1318: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:10:14.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:14 smithi167 ceph-mon[37140]: pgmap v1318: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:10:14.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:10:14.604+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:10:14.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:10:14.605+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:10:14.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:10:14.605+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:10:14.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:10:14.605+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:10:14.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:10:14.605+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:10:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:15 smithi138 ceph-mon[36954]: osdmap e133: 12 total, 12 up, 12 in 2024-07-26T14:10:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:15 smithi100 ceph-mon[32599]: osdmap e133: 12 total, 12 up, 12 in 2024-07-26T14:10:15.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:15 smithi167 ceph-mon[37140]: osdmap e133: 12 total, 12 up, 12 in 2024-07-26T14:10:16.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:16 smithi138 ceph-mon[36954]: pgmap v1320: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 250 B/s wr, 7 op/s 2024-07-26T14:10:16.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:16 smithi100 ceph-mon[32599]: pgmap v1320: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 250 B/s wr, 7 op/s 2024-07-26T14:10:16.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:16 smithi167 ceph-mon[37140]: pgmap v1320: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 250 B/s wr, 7 op/s 2024-07-26T14:10:18.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:18 smithi138 ceph-mon[36954]: pgmap v1321: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:10:18.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:18 smithi100 ceph-mon[32599]: pgmap v1321: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:10:18.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:18 smithi167 ceph-mon[37140]: pgmap v1321: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:10:20.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:20 smithi138 ceph-mon[36954]: pgmap v1322: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:10:20.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:20 smithi100 ceph-mon[32599]: pgmap v1322: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:10:20.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:20 smithi167 ceph-mon[37140]: pgmap v1322: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:10:22.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:22 smithi138 ceph-mon[36954]: pgmap v1323: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:10:22.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:22 smithi100 ceph-mon[32599]: pgmap v1323: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:10:22.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:22 smithi167 ceph-mon[37140]: pgmap v1323: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:10:24.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:24 smithi138 ceph-mon[36954]: pgmap v1324: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:10:24.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:24 smithi100 ceph-mon[32599]: pgmap v1324: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:10:24.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:24 smithi167 ceph-mon[37140]: pgmap v1324: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:10:26.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:26 smithi138 ceph-mon[36954]: pgmap v1325: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:10:26.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:26 smithi100 ceph-mon[32599]: pgmap v1325: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:10:26.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:26 smithi167 ceph-mon[37140]: pgmap v1325: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:10:28.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:28 smithi138 ceph-mon[36954]: pgmap v1326: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:10:28.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:28 smithi100 ceph-mon[32599]: pgmap v1326: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:10:28.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:28 smithi167 ceph-mon[37140]: pgmap v1326: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:10:29.395 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:10:29.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:10:29.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:10:29.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:10:29.620+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:10:29.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:10:29.620+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:10:29.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:10:29.620+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:10:29.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:10:29.620+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:10:29.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:10:29.620+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:10:30.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:30 smithi138 ceph-mon[36954]: pgmap v1327: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:10:30.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:10:30.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:10:30.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:10:30.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:10:30.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:10:30.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:10:30.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:10:30.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:10:30.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:10:30.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:30 smithi100 ceph-mon[32599]: pgmap v1327: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:10:30.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:10:30.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:10:30.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:10:30.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:10:30.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:10:30.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:10:30.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:10:30.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:10:30.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:10:30.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:30 smithi167 ceph-mon[37140]: pgmap v1327: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:10:30.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:10:30.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:10:30.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:10:30.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:10:30.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:10:30.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:10:30.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:10:30.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:10:30.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:10:32.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:32 smithi138 ceph-mon[36954]: pgmap v1328: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:10:32.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:32 smithi100 ceph-mon[32599]: pgmap v1328: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:10:32.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:32 smithi167 ceph-mon[37140]: pgmap v1328: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:10:34.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:34 smithi138 ceph-mon[36954]: pgmap v1329: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:10:34.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:34 smithi100 ceph-mon[32599]: pgmap v1329: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:10:34.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:34 smithi167 ceph-mon[37140]: pgmap v1329: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:10:36.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:36 smithi138 ceph-mon[36954]: pgmap v1330: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:10:36.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:36 smithi100 ceph-mon[32599]: pgmap v1330: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:10:36.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:36 smithi167 ceph-mon[37140]: pgmap v1330: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:10:38.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:38 smithi138 ceph-mon[36954]: pgmap v1331: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:10:38.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:38 smithi100 ceph-mon[32599]: pgmap v1331: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:10:38.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:38 smithi167 ceph-mon[37140]: pgmap v1331: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:10:40.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:40 smithi138 ceph-mon[36954]: pgmap v1332: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:10:40.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:40 smithi100 ceph-mon[32599]: pgmap v1332: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:10:40.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:40 smithi167 ceph-mon[37140]: pgmap v1332: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:10:42.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:42 smithi138 ceph-mon[36954]: pgmap v1333: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:10:42.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:42 smithi100 ceph-mon[32599]: pgmap v1333: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:10:42.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:42 smithi167 ceph-mon[37140]: pgmap v1333: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:10:44.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:44 smithi138 ceph-mon[36954]: pgmap v1334: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:10:44.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:44 smithi100 ceph-mon[32599]: pgmap v1334: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:10:44.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:44 smithi167 ceph-mon[37140]: pgmap v1334: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:10:44.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:10:44.625+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:10:44.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:10:44.625+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:10:44.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:10:44.625+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:10:44.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:10:44.625+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:10:44.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:10:44.625+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:10:46.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:46 smithi138 ceph-mon[36954]: pgmap v1335: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:10:46.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:46 smithi100 ceph-mon[32599]: pgmap v1335: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:10:46.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:46 smithi167 ceph-mon[37140]: pgmap v1335: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:10:48.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:48 smithi100 ceph-mon[32599]: pgmap v1336: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:10:48.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:48 smithi167 ceph-mon[37140]: pgmap v1336: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:10:48.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:48 smithi138 ceph-mon[36954]: pgmap v1336: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:10:50.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:50 smithi100 ceph-mon[32599]: pgmap v1337: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:10:50.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:50 smithi167 ceph-mon[37140]: pgmap v1337: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:10:50.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:50 smithi138 ceph-mon[36954]: pgmap v1337: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:10:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:52 smithi100 ceph-mon[32599]: pgmap v1338: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:10:52.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:52 smithi167 ceph-mon[37140]: pgmap v1338: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:10:52.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:52 smithi138 ceph-mon[36954]: pgmap v1338: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:10:54.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:54 smithi100 ceph-mon[32599]: pgmap v1339: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:10:54.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:54 smithi167 ceph-mon[37140]: pgmap v1339: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:10:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:54 smithi138 ceph-mon[36954]: pgmap v1339: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:10:56.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:56 smithi100 ceph-mon[32599]: pgmap v1340: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:10:56.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:56 smithi167 ceph-mon[37140]: pgmap v1340: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:10:56.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:56 smithi138 ceph-mon[36954]: pgmap v1340: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:10:58.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:10:58 smithi100 ceph-mon[32599]: pgmap v1341: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:10:58.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:10:58 smithi167 ceph-mon[37140]: pgmap v1341: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:10:58.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:10:58 smithi138 ceph-mon[36954]: pgmap v1341: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:10:59.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:10:59.641+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:10:59.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:10:59.641+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:10:59.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:10:59.641+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:10:59.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:10:59.641+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:10:59.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:10:59.641+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:11:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:11:00 smithi100 ceph-mon[32599]: pgmap v1342: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:11:00.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:11:00 smithi167 ceph-mon[37140]: pgmap v1342: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:11:00.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:11:00 smithi138 ceph-mon[36954]: pgmap v1342: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:11:02.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:11:02 smithi100 ceph-mon[32599]: pgmap v1343: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:11:02.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:11:02 smithi167 ceph-mon[37140]: pgmap v1343: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:11:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:11:02 smithi138 ceph-mon[36954]: pgmap v1343: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:11:04.492 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:11:04 smithi138 ceph-mon[36954]: pgmap v1344: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:11:04.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:11:04 smithi100 ceph-mon[32599]: pgmap v1344: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:11:04.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:11:04 smithi167 ceph-mon[37140]: pgmap v1344: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:11:06.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:11:06 smithi100 ceph-mon[32599]: pgmap v1345: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:11:06.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:11:06 smithi167 ceph-mon[37140]: pgmap v1345: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:11:06.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:11:06 smithi138 ceph-mon[36954]: pgmap v1345: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:11:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:11:08 smithi100 ceph-mon[32599]: pgmap v1346: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:11:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:11:08 smithi100 ceph-mon[32599]: osdmap e134: 12 total, 12 up, 12 in 2024-07-26T14:11:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:11:08 smithi167 ceph-mon[37140]: pgmap v1346: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:11:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:11:08 smithi167 ceph-mon[37140]: osdmap e134: 12 total, 12 up, 12 in 2024-07-26T14:11:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:11:08 smithi138 ceph-mon[36954]: pgmap v1346: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:11:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:11:08 smithi138 ceph-mon[36954]: osdmap e134: 12 total, 12 up, 12 in 2024-07-26T14:11:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:11:10 smithi100 ceph-mon[32599]: pgmap v1348: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:11:10.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:11:10 smithi167 ceph-mon[37140]: pgmap v1348: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:11:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:11:10 smithi138 ceph-mon[36954]: pgmap v1348: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:11:12.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:11:12 smithi100 ceph-mon[32599]: pgmap v1349: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:11:12.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:11:12 smithi167 ceph-mon[37140]: pgmap v1349: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:11:12.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:11:12 smithi138 ceph-mon[36954]: pgmap v1349: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:11:14.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:11:14 smithi100 ceph-mon[32599]: pgmap v1350: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:11:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:11:14 smithi100 ceph-mon[32599]: osdmap e135: 12 total, 12 up, 12 in 2024-07-26T14:11:14.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:11:14 smithi167 ceph-mon[37140]: pgmap v1350: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:11:14.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:11:14 smithi167 ceph-mon[37140]: osdmap e135: 12 total, 12 up, 12 in 2024-07-26T14:11:14.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:11:14.657+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:11:14.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:11:14.657+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:11:14.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:11:14.657+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:11:14.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:11:14.657+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:11:14.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:11:14.657+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:11:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:11:14 smithi138 ceph-mon[36954]: pgmap v1350: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:11:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:11:14 smithi138 ceph-mon[36954]: osdmap e135: 12 total, 12 up, 12 in 2024-07-26T14:11:16.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:11:16 smithi167 ceph-mon[37140]: pgmap v1352: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:11:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:11:16 smithi138 ceph-mon[36954]: pgmap v1352: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:11:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:11:16 smithi100 ceph-mon[32599]: pgmap v1352: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:11:18.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:11:18 smithi167 ceph-mon[37140]: pgmap v1353: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 209 B/s wr, 6 op/s 2024-07-26T14:11:18.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:11:18 smithi138 ceph-mon[36954]: pgmap v1353: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 209 B/s wr, 6 op/s 2024-07-26T14:11:18.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:11:18 smithi100 ceph-mon[32599]: pgmap v1353: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 209 B/s wr, 6 op/s 2024-07-26T14:11:20.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:11:20 smithi167 ceph-mon[37140]: pgmap v1354: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:11:20.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:11:20 smithi138 ceph-mon[36954]: pgmap v1354: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:11:20.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:11:20 smithi100 ceph-mon[32599]: pgmap v1354: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:11:22.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:11:22 smithi167 ceph-mon[37140]: pgmap v1355: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:11:22.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:11:22 smithi138 ceph-mon[36954]: pgmap v1355: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:11:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:11:22 smithi100 ceph-mon[32599]: pgmap v1355: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:11:24.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:11:24 smithi167 ceph-mon[37140]: pgmap v1356: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:11:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:11:24 smithi138 ceph-mon[36954]: pgmap v1356: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:11:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:11:24 smithi100 ceph-mon[32599]: pgmap v1356: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:11:26.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:11:26 smithi167 ceph-mon[37140]: pgmap v1357: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:11:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:11:26 smithi138 ceph-mon[36954]: pgmap v1357: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:11:26.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:11:26 smithi100 ceph-mon[32599]: pgmap v1357: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:11:28.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:11:28 smithi167 ceph-mon[37140]: pgmap v1358: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:11:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:11:28 smithi138 ceph-mon[36954]: pgmap v1358: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:11:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:11:28 smithi100 ceph-mon[32599]: pgmap v1358: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:11:29.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:11:29.671+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:11:29.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:11:29.672+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:11:29.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:11:29.672+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:11:29.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:11:29.672+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:11:29.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:11:29.672+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:11:30.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:11:30 smithi167 ceph-mon[37140]: pgmap v1359: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:11:30.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:11:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:11:30.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:11:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:11:30.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:11:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:11:30.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:11:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:11:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:11:30 smithi138 ceph-mon[36954]: pgmap v1359: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:11:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:11:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:11:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:11:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:11:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:11:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:11:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:11:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:11:30.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:11:30 smithi100 ceph-mon[32599]: pgmap v1359: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:11:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:11:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:11:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:11:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:11:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:11:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:11:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:11:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:11:32.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:11:32 smithi167 ceph-mon[37140]: pgmap v1360: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:11:32.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:11:32 smithi138 ceph-mon[36954]: pgmap v1360: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:11:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:11:32 smithi100 ceph-mon[32599]: pgmap v1360: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:11:34.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:11:34 smithi138 ceph-mon[36954]: pgmap v1361: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:11:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:11:34 smithi100 ceph-mon[32599]: pgmap v1361: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:11:34.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:11:34 smithi167 ceph-mon[37140]: pgmap v1361: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:11:36.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:11:36 smithi138 ceph-mon[36954]: pgmap v1362: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:11:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:11:36 smithi100 ceph-mon[32599]: pgmap v1362: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:11:36.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:11:36 smithi167 ceph-mon[37140]: pgmap v1362: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:11:38.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:11:38 smithi138 ceph-mon[36954]: pgmap v1363: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:11:38.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:11:38 smithi100 ceph-mon[32599]: pgmap v1363: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:11:38.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:11:38 smithi167 ceph-mon[37140]: pgmap v1363: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:11:40.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:11:40 smithi138 ceph-mon[36954]: pgmap v1364: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:11:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:11:40 smithi100 ceph-mon[32599]: pgmap v1364: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:11:40.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:11:40 smithi167 ceph-mon[37140]: pgmap v1364: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:11:42.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:11:42 smithi138 ceph-mon[36954]: pgmap v1365: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:11:42.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:11:42 smithi100 ceph-mon[32599]: pgmap v1365: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:11:42.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:11:42 smithi167 ceph-mon[37140]: pgmap v1365: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:11:44.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:11:44.687+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:11:44.689 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:11:44 smithi100 ceph-mon[32599]: pgmap v1366: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:11:44.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:11:44.688+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:11:44.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:11:44.688+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:11:44.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:11:44.688+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:11:44.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:11:44.688+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:11:44.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:11:44 smithi138 ceph-mon[36954]: pgmap v1366: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:11:44.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:11:44 smithi167 ceph-mon[37140]: pgmap v1366: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:11:46.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:11:46 smithi138 ceph-mon[36954]: pgmap v1367: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:11:46.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:11:46 smithi100 ceph-mon[32599]: pgmap v1367: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:11:46.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:11:46 smithi167 ceph-mon[37140]: pgmap v1367: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:11:48.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:11:48 smithi138 ceph-mon[36954]: pgmap v1368: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:11:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:11:48 smithi100 ceph-mon[32599]: pgmap v1368: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:11:48.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:11:48 smithi167 ceph-mon[37140]: pgmap v1368: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:11:50.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:11:50 smithi138 ceph-mon[36954]: pgmap v1369: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:11:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:11:50 smithi100 ceph-mon[32599]: pgmap v1369: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:11:50.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:11:50 smithi167 ceph-mon[37140]: pgmap v1369: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:11:52.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:11:52 smithi138 ceph-mon[36954]: pgmap v1370: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:11:52.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:11:52 smithi100 ceph-mon[32599]: pgmap v1370: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:11:52.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:11:52 smithi167 ceph-mon[37140]: pgmap v1370: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:11:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:11:54 smithi138 ceph-mon[36954]: pgmap v1371: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:11:54.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:11:54 smithi100 ceph-mon[32599]: pgmap v1371: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:11:54.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:11:54 smithi167 ceph-mon[37140]: pgmap v1371: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:11:56.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:11:56 smithi138 ceph-mon[36954]: pgmap v1372: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:11:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:11:56 smithi100 ceph-mon[32599]: pgmap v1372: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:11:56.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:11:56 smithi167 ceph-mon[37140]: pgmap v1372: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:11:58.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:11:58 smithi138 ceph-mon[36954]: pgmap v1373: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:11:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:11:58 smithi100 ceph-mon[32599]: pgmap v1373: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:11:58.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:11:58 smithi167 ceph-mon[37140]: pgmap v1373: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:11:59.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:11:59.704+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:11:59.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:11:59.704+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:11:59.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:11:59.704+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:11:59.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:11:59.704+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:11:59.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:11:59.704+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:12:00.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:12:00 smithi138 ceph-mon[36954]: pgmap v1374: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:12:00.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:12:00 smithi100 ceph-mon[32599]: pgmap v1374: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:12:00.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:12:00 smithi167 ceph-mon[37140]: pgmap v1374: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:12:02.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:12:02 smithi167 ceph-mon[37140]: pgmap v1375: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:12:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:12:02 smithi138 ceph-mon[36954]: pgmap v1375: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:12:02.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:12:02 smithi100 ceph-mon[32599]: pgmap v1375: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:12:03.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:12:03.216+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580003690 con 0x55c0ec887390 2024-07-26T14:12:03.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:12:03.216+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c003690 con 0x7fe5a81087a0 2024-07-26T14:12:03.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:12:03.216+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 15 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580003690 con 0x55c0ec887390 2024-07-26T14:12:03.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:12:03.216+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 15 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c003690 con 0x7fe5a81087a0 2024-07-26T14:12:04.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:12:04 smithi138 ceph-mon[36954]: pgmap v1376: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:12:04.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:12:04 smithi100 ceph-mon[32599]: pgmap v1376: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:12:04.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:12:04 smithi167 ceph-mon[37140]: pgmap v1376: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:12:06.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:12:06 smithi138 ceph-mon[36954]: pgmap v1377: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:12:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:12:06 smithi100 ceph-mon[32599]: pgmap v1377: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:12:06.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:12:06 smithi167 ceph-mon[37140]: pgmap v1377: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:12:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:12:08 smithi138 ceph-mon[36954]: pgmap v1378: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:12:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:12:08 smithi138 ceph-mon[36954]: osdmap e136: 12 total, 12 up, 12 in 2024-07-26T14:12:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:12:08 smithi100 ceph-mon[32599]: pgmap v1378: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:12:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:12:08 smithi100 ceph-mon[32599]: osdmap e136: 12 total, 12 up, 12 in 2024-07-26T14:12:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:12:08 smithi167 ceph-mon[37140]: pgmap v1378: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:12:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:12:08 smithi167 ceph-mon[37140]: osdmap e136: 12 total, 12 up, 12 in 2024-07-26T14:12:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:12:10 smithi138 ceph-mon[36954]: pgmap v1380: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:12:10.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:12:10 smithi100 ceph-mon[32599]: pgmap v1380: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:12:10.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:12:10 smithi167 ceph-mon[37140]: pgmap v1380: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:12:12.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:12:12 smithi138 ceph-mon[36954]: pgmap v1381: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:12:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:12:12 smithi100 ceph-mon[32599]: pgmap v1381: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:12:12.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:12:12 smithi167 ceph-mon[37140]: pgmap v1381: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:12:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:12:14 smithi138 ceph-mon[36954]: pgmap v1382: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:12:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:12:14 smithi138 ceph-mon[36954]: osdmap e137: 12 total, 12 up, 12 in 2024-07-26T14:12:14.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:12:14.720+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:12:14.721 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:12:14 smithi100 ceph-mon[32599]: pgmap v1382: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:12:14.721 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:12:14 smithi100 ceph-mon[32599]: osdmap e137: 12 total, 12 up, 12 in 2024-07-26T14:12:14.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:12:14.720+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:12:14.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:12:14.720+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:12:14.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:12:14.720+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:12:14.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:12:14.720+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:12:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:12:14 smithi167 ceph-mon[37140]: pgmap v1382: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:12:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:12:14 smithi167 ceph-mon[37140]: osdmap e137: 12 total, 12 up, 12 in 2024-07-26T14:12:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:12:16 smithi138 ceph-mon[36954]: pgmap v1384: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:12:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:12:16 smithi100 ceph-mon[32599]: pgmap v1384: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:12:16.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:12:16 smithi167 ceph-mon[37140]: pgmap v1384: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:12:18.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:12:18 smithi138 ceph-mon[36954]: pgmap v1385: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 211 B/s wr, 6 op/s 2024-07-26T14:12:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:12:18 smithi100 ceph-mon[32599]: pgmap v1385: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 211 B/s wr, 6 op/s 2024-07-26T14:12:18.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:12:18 smithi167 ceph-mon[37140]: pgmap v1385: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 211 B/s wr, 6 op/s 2024-07-26T14:12:20.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:12:20 smithi138 ceph-mon[36954]: pgmap v1386: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:12:20.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:12:20 smithi100 ceph-mon[32599]: pgmap v1386: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:12:20.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:12:20 smithi167 ceph-mon[37140]: pgmap v1386: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:12:22.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:12:22 smithi138 ceph-mon[36954]: pgmap v1387: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:12:22.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:12:22 smithi100 ceph-mon[32599]: pgmap v1387: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:12:22.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:12:22 smithi167 ceph-mon[37140]: pgmap v1387: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:12:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:12:24 smithi138 ceph-mon[36954]: pgmap v1388: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:12:24.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:12:24 smithi100 ceph-mon[32599]: pgmap v1388: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:12:24.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:12:24 smithi167 ceph-mon[37140]: pgmap v1388: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:12:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:12:26 smithi138 ceph-mon[36954]: pgmap v1389: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:12:26.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:12:26 smithi100 ceph-mon[32599]: pgmap v1389: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:12:26.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:12:26 smithi167 ceph-mon[37140]: pgmap v1389: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:12:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:12:28 smithi138 ceph-mon[36954]: pgmap v1390: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:12:28.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:12:28 smithi100 ceph-mon[32599]: pgmap v1390: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:12:28.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:12:28 smithi167 ceph-mon[37140]: pgmap v1390: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:12:29.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:12:29.724+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:12:29.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:12:29.725+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:12:29.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:12:29.725+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:12:29.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:12:29.725+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:12:29.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:12:29.725+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:12:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:12:30 smithi138 ceph-mon[36954]: pgmap v1391: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:12:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:12:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:12:30.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:12:30 smithi100 ceph-mon[32599]: pgmap v1391: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:12:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:12:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:12:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:12:30 smithi167 ceph-mon[37140]: pgmap v1391: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:12:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:12:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:12:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:12:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:12:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:12:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:12:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:12:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:12:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:12:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:12:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:12:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:12:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:12:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:12:31.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:12:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:12:31.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:12:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:12:31.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:12:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:12:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:12:32 smithi100 ceph-mon[32599]: pgmap v1392: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:12:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:12:32 smithi167 ceph-mon[37140]: pgmap v1392: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:12:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:12:32 smithi138 ceph-mon[36954]: pgmap v1392: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:12:34.992 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:12:34 smithi138 ceph-mon[36954]: pgmap v1393: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:12:34.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:12:34 smithi100 ceph-mon[32599]: pgmap v1393: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:12:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:12:34 smithi167 ceph-mon[37140]: pgmap v1393: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:12:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:12:36 smithi100 ceph-mon[32599]: pgmap v1394: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:12:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:12:36 smithi167 ceph-mon[37140]: pgmap v1394: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:12:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:12:36 smithi138 ceph-mon[36954]: pgmap v1394: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:12:38.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:12:38 smithi100 ceph-mon[32599]: pgmap v1395: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:12:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:12:38 smithi167 ceph-mon[37140]: pgmap v1395: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:12:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:12:38 smithi138 ceph-mon[36954]: pgmap v1395: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:12:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:12:40 smithi100 ceph-mon[32599]: pgmap v1396: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:12:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:12:40 smithi167 ceph-mon[37140]: pgmap v1396: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:12:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:12:40 smithi138 ceph-mon[36954]: pgmap v1396: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:12:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:12:42 smithi100 ceph-mon[32599]: pgmap v1397: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:12:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:12:42 smithi167 ceph-mon[37140]: pgmap v1397: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:12:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:12:42 smithi138 ceph-mon[36954]: pgmap v1397: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:12:44.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:12:44.735+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:12:44.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:12:44.736+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:12:44.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:12:44.736+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:12:44.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:12:44.736+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:12:44.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:12:44.736+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:12:44.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:12:44 smithi100 ceph-mon[32599]: pgmap v1398: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:12:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:12:44 smithi167 ceph-mon[37140]: pgmap v1398: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:12:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:12:44 smithi138 ceph-mon[36954]: pgmap v1398: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:12:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:12:46 smithi100 ceph-mon[32599]: pgmap v1399: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:12:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:12:46 smithi167 ceph-mon[37140]: pgmap v1399: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:12:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:12:46 smithi138 ceph-mon[36954]: pgmap v1399: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:12:48.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:12:48.222+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c003690 con 0x7fe5a81087a0 2024-07-26T14:12:48.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:12:48.222+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580003690 con 0x55c0ec887390 2024-07-26T14:12:48.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:12:48.223+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 16 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe57c003690 con 0x7fe5a81087a0 2024-07-26T14:12:48.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:12:48.223+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 16 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe580003690 con 0x55c0ec887390 2024-07-26T14:12:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:12:48 smithi100 ceph-mon[32599]: pgmap v1400: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:12:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:12:48 smithi167 ceph-mon[37140]: pgmap v1400: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:12:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:12:48 smithi138 ceph-mon[36954]: pgmap v1400: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:12:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:12:50 smithi100 ceph-mon[32599]: pgmap v1401: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:12:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:12:50 smithi167 ceph-mon[37140]: pgmap v1401: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:12:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:12:50 smithi138 ceph-mon[36954]: pgmap v1401: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:12:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:12:52 smithi100 ceph-mon[32599]: pgmap v1402: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:12:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:12:52 smithi167 ceph-mon[37140]: pgmap v1402: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:12:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:12:52 smithi138 ceph-mon[36954]: pgmap v1402: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:12:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:12:54 smithi167 ceph-mon[37140]: pgmap v1403: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:12:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:12:54 smithi138 ceph-mon[36954]: pgmap v1403: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:12:55.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:12:54 smithi100 ceph-mon[32599]: pgmap v1403: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:12:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:12:56 smithi167 ceph-mon[37140]: pgmap v1404: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:12:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:12:56 smithi138 ceph-mon[36954]: pgmap v1404: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:12:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:12:56 smithi100 ceph-mon[32599]: pgmap v1404: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:12:59.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:12:58 smithi167 ceph-mon[37140]: pgmap v1405: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:12:59.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:12:58 smithi100 ceph-mon[32599]: pgmap v1405: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:12:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:12:58 smithi138 ceph-mon[36954]: pgmap v1405: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:12:59.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:12:59.741+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:12:59.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:12:59.742+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:12:59.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:12:59.742+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:12:59.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:12:59.742+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:12:59.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:12:59.742+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:13:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:13:00 smithi167 ceph-mon[37140]: pgmap v1406: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:13:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:13:00 smithi138 ceph-mon[36954]: pgmap v1406: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:13:01.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:13:00 smithi100 ceph-mon[32599]: pgmap v1406: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:13:03.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:13:02 smithi167 ceph-mon[37140]: pgmap v1407: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:13:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:13:02 smithi138 ceph-mon[36954]: pgmap v1407: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:13:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:13:02 smithi100 ceph-mon[32599]: pgmap v1407: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:13:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:13:04 smithi167 ceph-mon[37140]: pgmap v1408: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:13:05.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:13:04 smithi138 ceph-mon[36954]: pgmap v1408: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:13:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:13:04 smithi100 ceph-mon[32599]: pgmap v1408: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:13:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:13:06 smithi167 ceph-mon[37140]: pgmap v1409: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:13:07.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:13:06 smithi138 ceph-mon[36954]: pgmap v1409: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:13:07.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:13:06 smithi100 ceph-mon[32599]: pgmap v1409: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:13:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:13:08 smithi167 ceph-mon[37140]: pgmap v1410: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:13:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:13:08 smithi167 ceph-mon[37140]: osdmap e138: 12 total, 12 up, 12 in 2024-07-26T14:13:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:13:08 smithi138 ceph-mon[36954]: pgmap v1410: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:13:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:13:08 smithi138 ceph-mon[36954]: osdmap e138: 12 total, 12 up, 12 in 2024-07-26T14:13:09.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:13:08 smithi100 ceph-mon[32599]: pgmap v1410: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:13:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:13:08 smithi100 ceph-mon[32599]: osdmap e138: 12 total, 12 up, 12 in 2024-07-26T14:13:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:13:10 smithi138 ceph-mon[36954]: pgmap v1412: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:13:11.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:13:10 smithi100 ceph-mon[32599]: pgmap v1412: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:13:11.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:13:10 smithi167 ceph-mon[37140]: pgmap v1412: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:13:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:13:12 smithi138 ceph-mon[36954]: pgmap v1413: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:13:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:13:12 smithi100 ceph-mon[32599]: pgmap v1413: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:13:13.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:13:12 smithi167 ceph-mon[37140]: pgmap v1413: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:13:14.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:13:14.757+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:13:14.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:13:14.757+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:13:14.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:13:14.757+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:13:14.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:13:14.757+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:13:14.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:13:14.757+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:13:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:13:14 smithi138 ceph-mon[36954]: pgmap v1414: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:13:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:13:14 smithi138 ceph-mon[36954]: osdmap e139: 12 total, 12 up, 12 in 2024-07-26T14:13:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:13:14 smithi100 ceph-mon[32599]: pgmap v1414: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:13:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:13:14 smithi100 ceph-mon[32599]: osdmap e139: 12 total, 12 up, 12 in 2024-07-26T14:13:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:13:14 smithi167 ceph-mon[37140]: pgmap v1414: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:13:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:13:14 smithi167 ceph-mon[37140]: osdmap e139: 12 total, 12 up, 12 in 2024-07-26T14:13:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:13:16 smithi138 ceph-mon[36954]: pgmap v1416: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:13:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:13:16 smithi100 ceph-mon[32599]: pgmap v1416: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:13:17.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:13:16 smithi167 ceph-mon[37140]: pgmap v1416: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:13:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:13:18 smithi138 ceph-mon[36954]: pgmap v1417: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 220 B/s wr, 6 op/s 2024-07-26T14:13:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:13:18 smithi100 ceph-mon[32599]: pgmap v1417: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 220 B/s wr, 6 op/s 2024-07-26T14:13:19.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:13:18 smithi167 ceph-mon[37140]: pgmap v1417: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 220 B/s wr, 6 op/s 2024-07-26T14:13:21.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:13:20 smithi138 ceph-mon[36954]: pgmap v1418: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:13:21.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:13:20 smithi100 ceph-mon[32599]: pgmap v1418: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:13:21.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:13:20 smithi167 ceph-mon[37140]: pgmap v1418: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:13:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:13:22 smithi138 ceph-mon[36954]: pgmap v1419: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:13:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:13:22 smithi100 ceph-mon[32599]: pgmap v1419: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:13:23.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:13:22 smithi167 ceph-mon[37140]: pgmap v1419: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:13:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:13:24 smithi138 ceph-mon[36954]: pgmap v1420: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:13:25.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:13:24 smithi100 ceph-mon[32599]: pgmap v1420: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:13:25.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:13:24 smithi167 ceph-mon[37140]: pgmap v1420: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:13:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:13:26 smithi138 ceph-mon[36954]: pgmap v1421: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:13:27.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:13:26 smithi100 ceph-mon[32599]: pgmap v1421: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:13:27.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:13:26 smithi167 ceph-mon[37140]: pgmap v1421: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:13:29.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:13:28 smithi138 ceph-mon[36954]: pgmap v1422: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:13:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:13:28 smithi100 ceph-mon[32599]: pgmap v1422: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:13:29.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:13:28 smithi167 ceph-mon[37140]: pgmap v1422: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:13:29.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:13:29.772+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:13:29.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:13:29.773+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:13:29.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:13:29.773+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:13:29.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:13:29.773+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:13:29.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:13:29.773+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:13:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:13:30 smithi138 ceph-mon[36954]: pgmap v1423: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:13:31.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:13:30 smithi100 ceph-mon[32599]: pgmap v1423: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:13:31.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:13:30 smithi167 ceph-mon[37140]: pgmap v1423: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:13:32.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:13:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:13:32.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:13:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:13:32.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:13:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:13:32.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:13:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:13:32.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:13:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:13:32.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:13:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:13:32.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:13:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:13:32.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:13:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:13:32.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:13:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:13:32.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:13:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:13:32.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:13:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:13:32.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:13:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:13:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:13:32 smithi138 ceph-mon[36954]: pgmap v1424: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:13:33.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:13:32 smithi100 ceph-mon[32599]: pgmap v1424: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:13:33.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:13:32 smithi167 ceph-mon[37140]: pgmap v1424: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:13:35.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:13:34 smithi167 ceph-mon[37140]: pgmap v1425: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:13:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:13:34 smithi138 ceph-mon[36954]: pgmap v1425: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:13:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:13:34 smithi100 ceph-mon[32599]: pgmap v1425: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:13:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:13:36 smithi138 ceph-mon[36954]: pgmap v1426: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:13:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:13:36 smithi100 ceph-mon[32599]: pgmap v1426: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:13:37.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:13:36 smithi167 ceph-mon[37140]: pgmap v1426: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:13:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:13:38 smithi138 ceph-mon[36954]: pgmap v1427: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:13:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:13:38 smithi100 ceph-mon[32599]: pgmap v1427: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:13:39.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:13:38 smithi167 ceph-mon[37140]: pgmap v1427: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:13:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:13:40 smithi138 ceph-mon[36954]: pgmap v1428: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:13:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:13:40 smithi100 ceph-mon[32599]: pgmap v1428: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:13:41.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:13:40 smithi167 ceph-mon[37140]: pgmap v1428: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:13:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:13:42 smithi138 ceph-mon[36954]: pgmap v1429: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:13:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:13:42 smithi100 ceph-mon[32599]: pgmap v1429: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:13:43.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:13:42 smithi167 ceph-mon[37140]: pgmap v1429: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:13:44.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:13:44.788+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:13:44.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:13:44.789+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:13:44.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:13:44.789+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:13:44.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:13:44.789+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:13:44.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:13:44.789+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:13:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:13:44 smithi138 ceph-mon[36954]: pgmap v1430: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:13:45.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:13:44 smithi100 ceph-mon[32599]: pgmap v1430: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:13:45.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:13:44 smithi167 ceph-mon[37140]: pgmap v1430: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:13:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:13:46 smithi138 ceph-mon[36954]: pgmap v1431: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:13:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:13:46 smithi100 ceph-mon[32599]: pgmap v1431: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:13:47.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:13:46 smithi167 ceph-mon[37140]: pgmap v1431: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:13:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:13:48 smithi138 ceph-mon[36954]: pgmap v1432: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:13:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:13:48 smithi100 ceph-mon[32599]: pgmap v1432: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:13:49.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:13:48 smithi167 ceph-mon[37140]: pgmap v1432: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:13:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:13:50 smithi138 ceph-mon[36954]: pgmap v1433: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:13:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:13:50 smithi100 ceph-mon[32599]: pgmap v1433: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:13:51.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:13:50 smithi167 ceph-mon[37140]: pgmap v1433: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:13:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:13:52 smithi138 ceph-mon[36954]: pgmap v1434: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:13:53.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:13:52 smithi100 ceph-mon[32599]: pgmap v1434: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:13:53.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:13:52 smithi167 ceph-mon[37140]: pgmap v1434: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:13:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:13:54 smithi138 ceph-mon[36954]: pgmap v1435: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:13:55.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:13:54 smithi100 ceph-mon[32599]: pgmap v1435: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:13:55.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:13:54 smithi167 ceph-mon[37140]: pgmap v1435: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:13:56.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:13:55 smithi138 ceph-mon[36954]: pgmap v1436: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:13:56.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:13:55 smithi100 ceph-mon[32599]: pgmap v1436: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:13:56.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:13:55 smithi167 ceph-mon[37140]: pgmap v1436: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:13:58.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:13:58 smithi138 ceph-mon[36954]: pgmap v1437: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:13:58.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:13:58 smithi100 ceph-mon[32599]: pgmap v1437: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:13:58.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:13:58 smithi167 ceph-mon[37140]: pgmap v1437: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:13:59.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:13:59.799+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:13:59.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:13:59.800+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:13:59.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:13:59.800+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:13:59.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:13:59.800+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:13:59.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:13:59.800+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:14:00.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:14:00 smithi138 ceph-mon[36954]: pgmap v1438: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:14:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:14:00 smithi100 ceph-mon[32599]: pgmap v1438: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:14:00.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:14:00 smithi167 ceph-mon[37140]: pgmap v1438: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:14:02.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:14:02 smithi138 ceph-mon[36954]: pgmap v1439: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:14:02.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:14:02 smithi100 ceph-mon[32599]: pgmap v1439: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:14:02.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:14:02 smithi167 ceph-mon[37140]: pgmap v1439: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:14:04.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:14:04 smithi138 ceph-mon[36954]: pgmap v1440: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:14:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:14:04 smithi100 ceph-mon[32599]: pgmap v1440: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:14:04.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:14:04 smithi167 ceph-mon[37140]: pgmap v1440: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:14:06.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:14:06 smithi138 ceph-mon[36954]: pgmap v1441: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:14:06.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:14:06 smithi100 ceph-mon[32599]: pgmap v1441: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:14:06.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:14:06 smithi167 ceph-mon[37140]: pgmap v1441: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:14:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:14:08 smithi138 ceph-mon[36954]: pgmap v1442: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:14:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:14:08 smithi138 ceph-mon[36954]: osdmap e140: 12 total, 12 up, 12 in 2024-07-26T14:14:08.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:14:08 smithi100 ceph-mon[32599]: pgmap v1442: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:14:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:14:08 smithi100 ceph-mon[32599]: osdmap e140: 12 total, 12 up, 12 in 2024-07-26T14:14:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:14:08 smithi167 ceph-mon[37140]: pgmap v1442: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:14:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:14:08 smithi167 ceph-mon[37140]: osdmap e140: 12 total, 12 up, 12 in 2024-07-26T14:14:10.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:14:10 smithi138 ceph-mon[36954]: pgmap v1444: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:14:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:14:10 smithi100 ceph-mon[32599]: pgmap v1444: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:14:10.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:14:10 smithi167 ceph-mon[37140]: pgmap v1444: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:14:12.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:14:12 smithi138 ceph-mon[36954]: pgmap v1445: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:14:12.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:14:12 smithi100 ceph-mon[32599]: pgmap v1445: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:14:12.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:14:12 smithi167 ceph-mon[37140]: pgmap v1445: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:14:14.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:14:14 smithi138 ceph-mon[36954]: pgmap v1446: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:14:14.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:14:14 smithi138 ceph-mon[36954]: osdmap e141: 12 total, 12 up, 12 in 2024-07-26T14:14:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:14:14 smithi100 ceph-mon[32599]: pgmap v1446: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:14:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:14:14 smithi100 ceph-mon[32599]: osdmap e141: 12 total, 12 up, 12 in 2024-07-26T14:14:14.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:14:14 smithi167 ceph-mon[37140]: pgmap v1446: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:14:14.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:14:14 smithi167 ceph-mon[37140]: osdmap e141: 12 total, 12 up, 12 in 2024-07-26T14:14:14.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:14:14.815+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:14:14.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:14:14.816+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:14:14.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:14:14.816+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:14:14.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:14:14.816+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:14:14.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:14:14.816+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:14:16.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:14:16 smithi138 ceph-mon[36954]: pgmap v1448: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:14:16.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:14:16 smithi100 ceph-mon[32599]: pgmap v1448: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:14:16.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:14:16 smithi167 ceph-mon[37140]: pgmap v1448: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:14:18.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:14:18 smithi138 ceph-mon[36954]: pgmap v1449: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T14:14:18.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:14:18 smithi100 ceph-mon[32599]: pgmap v1449: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T14:14:18.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:14:18 smithi167 ceph-mon[37140]: pgmap v1449: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T14:14:20.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:14:20 smithi138 ceph-mon[36954]: pgmap v1450: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:14:20.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:14:20 smithi100 ceph-mon[32599]: pgmap v1450: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:14:20.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:14:20 smithi167 ceph-mon[37140]: pgmap v1450: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:14:22.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:14:22 smithi138 ceph-mon[36954]: pgmap v1451: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:14:22.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:14:22 smithi100 ceph-mon[32599]: pgmap v1451: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:14:22.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:14:22 smithi167 ceph-mon[37140]: pgmap v1451: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:14:24.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:14:24 smithi138 ceph-mon[36954]: pgmap v1452: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:14:24.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:14:24 smithi100 ceph-mon[32599]: pgmap v1452: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:14:24.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:14:24 smithi167 ceph-mon[37140]: pgmap v1452: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:14:26.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:14:26 smithi138 ceph-mon[36954]: pgmap v1453: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:14:26.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:14:26 smithi100 ceph-mon[32599]: pgmap v1453: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:14:26.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:14:26 smithi167 ceph-mon[37140]: pgmap v1453: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:14:28.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:14:28 smithi138 ceph-mon[36954]: pgmap v1454: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:14:28.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:14:28 smithi100 ceph-mon[32599]: pgmap v1454: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:14:28.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:14:28 smithi167 ceph-mon[37140]: pgmap v1454: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:14:29.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:14:29.830+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:14:29.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:14:29.831+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:14:29.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:14:29.831+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:14:29.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:14:29.831+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:14:29.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:14:29.831+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:14:30.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:14:30 smithi138 ceph-mon[36954]: pgmap v1455: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:14:30.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:14:30 smithi100 ceph-mon[32599]: pgmap v1455: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:14:30.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:14:30 smithi167 ceph-mon[37140]: pgmap v1455: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:14:32.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:14:32 smithi138 ceph-mon[36954]: pgmap v1456: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:14:32.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:14:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:14:32.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:14:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:14:32.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:14:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:14:32.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:14:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:14:32.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:14:32 smithi100 ceph-mon[32599]: pgmap v1456: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:14:32.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:14:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:14:32.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:14:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:14:32.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:14:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:14:32.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:14:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:14:32.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:14:32 smithi167 ceph-mon[37140]: pgmap v1456: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:14:32.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:14:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:14:32.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:14:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:14:32.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:14:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:14:32.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:14:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:14:34.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:14:34 smithi138 ceph-mon[36954]: pgmap v1457: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:14:34.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:14:34 smithi100 ceph-mon[32599]: pgmap v1457: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:14:34.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:14:34 smithi167 ceph-mon[37140]: pgmap v1457: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:14:36.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:14:36 smithi138 ceph-mon[36954]: pgmap v1458: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:14:36.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:14:36 smithi100 ceph-mon[32599]: pgmap v1458: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:14:36.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:14:36 smithi167 ceph-mon[37140]: pgmap v1458: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:14:38.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:14:38 smithi138 ceph-mon[36954]: pgmap v1459: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:14:38.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:14:38 smithi100 ceph-mon[32599]: pgmap v1459: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:14:38.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:14:38 smithi167 ceph-mon[37140]: pgmap v1459: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:14:40.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:14:40 smithi138 ceph-mon[36954]: pgmap v1460: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:14:40.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:14:40 smithi100 ceph-mon[32599]: pgmap v1460: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:14:40.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:14:40 smithi167 ceph-mon[37140]: pgmap v1460: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:14:42.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:14:42 smithi138 ceph-mon[36954]: pgmap v1461: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:14:42.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:14:42 smithi100 ceph-mon[32599]: pgmap v1461: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:14:42.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:14:42 smithi167 ceph-mon[37140]: pgmap v1461: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:14:44.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:14:44 smithi138 ceph-mon[36954]: pgmap v1462: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:14:44.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:14:44 smithi100 ceph-mon[32599]: pgmap v1462: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:14:44.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:14:44 smithi167 ceph-mon[37140]: pgmap v1462: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:14:44.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:14:44.841+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:14:44.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:14:44.841+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:14:44.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:14:44.841+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:14:44.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:14:44.841+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:14:44.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:14:44.841+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:14:46.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:14:46 smithi100 ceph-mon[32599]: pgmap v1463: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:14:46.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:14:46 smithi167 ceph-mon[37140]: pgmap v1463: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:14:46.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:14:46 smithi138 ceph-mon[36954]: pgmap v1463: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:14:48.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:14:48 smithi100 ceph-mon[32599]: pgmap v1464: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:14:48.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:14:48 smithi167 ceph-mon[37140]: pgmap v1464: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:14:48.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:14:48 smithi138 ceph-mon[36954]: pgmap v1464: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:14:50.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:14:50 smithi100 ceph-mon[32599]: pgmap v1465: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:14:50.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:14:50 smithi167 ceph-mon[37140]: pgmap v1465: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:14:50.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:14:50 smithi138 ceph-mon[36954]: pgmap v1465: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:14:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:14:52 smithi100 ceph-mon[32599]: pgmap v1466: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:14:52.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:14:52 smithi167 ceph-mon[37140]: pgmap v1466: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:14:52.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:14:52 smithi138 ceph-mon[36954]: pgmap v1466: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:14:54.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:14:54 smithi100 ceph-mon[32599]: pgmap v1467: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:14:54.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:14:54 smithi167 ceph-mon[37140]: pgmap v1467: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:14:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:14:54 smithi138 ceph-mon[36954]: pgmap v1467: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:14:56.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:14:56 smithi100 ceph-mon[32599]: pgmap v1468: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:14:56.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:14:56 smithi167 ceph-mon[37140]: pgmap v1468: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:14:56.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:14:56 smithi138 ceph-mon[36954]: pgmap v1468: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:14:58.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:14:58 smithi100 ceph-mon[32599]: pgmap v1469: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:14:58.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:14:58 smithi167 ceph-mon[37140]: pgmap v1469: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:14:58.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:14:58 smithi138 ceph-mon[36954]: pgmap v1469: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:14:59.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:14:59.844+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:14:59.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:14:59.844+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:14:59.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:14:59.844+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:14:59.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:14:59.844+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:14:59.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:14:59.844+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:15:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:15:00 smithi100 ceph-mon[32599]: pgmap v1470: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:15:00.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:15:00 smithi167 ceph-mon[37140]: pgmap v1470: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:15:00.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:15:00 smithi138 ceph-mon[36954]: pgmap v1470: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:15:02.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:15:02 smithi100 ceph-mon[32599]: pgmap v1471: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:15:02.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:15:02 smithi167 ceph-mon[37140]: pgmap v1471: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:15:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:15:02 smithi138 ceph-mon[36954]: pgmap v1471: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:15:03.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:15:03.241+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580003690 con 0x55c0ec887390 2024-07-26T14:15:03.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:15:03.241+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c003690 con 0x7fe5a81087a0 2024-07-26T14:15:03.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:15:03.242+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 17 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c003690 con 0x7fe5a81087a0 2024-07-26T14:15:03.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:15:03.242+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 17 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580003690 con 0x55c0ec887390 2024-07-26T14:15:04.492 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:15:04 smithi138 ceph-mon[36954]: pgmap v1472: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:15:04.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:15:04 smithi100 ceph-mon[32599]: pgmap v1472: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:15:04.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:15:04 smithi167 ceph-mon[37140]: pgmap v1472: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:15:06.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:15:06 smithi100 ceph-mon[32599]: pgmap v1473: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:15:06.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:15:06 smithi167 ceph-mon[37140]: pgmap v1473: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:15:06.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:15:06 smithi138 ceph-mon[36954]: pgmap v1473: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:15:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:15:08 smithi100 ceph-mon[32599]: pgmap v1474: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:15:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:15:08 smithi100 ceph-mon[32599]: osdmap e142: 12 total, 12 up, 12 in 2024-07-26T14:15:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:15:08 smithi167 ceph-mon[37140]: pgmap v1474: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:15:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:15:08 smithi167 ceph-mon[37140]: osdmap e142: 12 total, 12 up, 12 in 2024-07-26T14:15:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:15:08 smithi138 ceph-mon[36954]: pgmap v1474: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:15:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:15:08 smithi138 ceph-mon[36954]: osdmap e142: 12 total, 12 up, 12 in 2024-07-26T14:15:10.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:15:10 smithi100 ceph-mon[32599]: pgmap v1476: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:15:10.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:15:10 smithi167 ceph-mon[37140]: pgmap v1476: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:15:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:15:10 smithi138 ceph-mon[36954]: pgmap v1476: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:15:12.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:15:12 smithi100 ceph-mon[32599]: pgmap v1477: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:15:12.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:15:12 smithi167 ceph-mon[37140]: pgmap v1477: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:15:12.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:15:12 smithi138 ceph-mon[36954]: pgmap v1477: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:15:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:15:14 smithi100 ceph-mon[32599]: pgmap v1478: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:15:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:15:14 smithi100 ceph-mon[32599]: osdmap e143: 12 total, 12 up, 12 in 2024-07-26T14:15:14.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:15:14 smithi167 ceph-mon[37140]: pgmap v1478: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:15:14.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:15:14 smithi167 ceph-mon[37140]: osdmap e143: 12 total, 12 up, 12 in 2024-07-26T14:15:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:15:14 smithi138 ceph-mon[36954]: pgmap v1478: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:15:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:15:14 smithi138 ceph-mon[36954]: osdmap e143: 12 total, 12 up, 12 in 2024-07-26T14:15:14.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:15:14.857+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:15:14.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:15:14.858+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:15:14.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:15:14.858+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:15:14.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:15:14.858+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:15:14.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:15:14.858+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:15:16.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:15:16 smithi167 ceph-mon[37140]: pgmap v1480: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:15:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:15:16 smithi138 ceph-mon[36954]: pgmap v1480: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:15:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:15:16 smithi100 ceph-mon[32599]: pgmap v1480: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:15:18.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:15:18 smithi167 ceph-mon[37140]: pgmap v1481: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T14:15:18.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:15:18 smithi138 ceph-mon[36954]: pgmap v1481: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T14:15:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:15:18 smithi100 ceph-mon[32599]: pgmap v1481: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T14:15:20.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:15:20 smithi167 ceph-mon[37140]: pgmap v1482: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:15:20.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:15:20 smithi138 ceph-mon[36954]: pgmap v1482: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:15:20.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:15:20 smithi100 ceph-mon[32599]: pgmap v1482: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:15:22.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:15:22 smithi167 ceph-mon[37140]: pgmap v1483: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:15:22.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:15:22 smithi138 ceph-mon[36954]: pgmap v1483: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:15:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:15:22 smithi100 ceph-mon[32599]: pgmap v1483: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:15:24.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:15:24 smithi167 ceph-mon[37140]: pgmap v1484: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:15:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:15:24 smithi138 ceph-mon[36954]: pgmap v1484: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:15:24.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:15:24 smithi100 ceph-mon[32599]: pgmap v1484: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:15:26.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:15:26 smithi167 ceph-mon[37140]: pgmap v1485: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:15:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:15:26 smithi138 ceph-mon[36954]: pgmap v1485: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:15:26.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:15:26 smithi100 ceph-mon[32599]: pgmap v1485: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:15:28.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:15:28 smithi167 ceph-mon[37140]: pgmap v1486: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:15:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:15:28 smithi138 ceph-mon[36954]: pgmap v1486: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:15:28.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:15:28 smithi100 ceph-mon[32599]: pgmap v1486: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:15:29.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:15:29.862+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:15:29.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:15:29.862+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:15:29.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:15:29.862+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:15:29.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:15:29.862+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:15:29.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:15:29.862+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:15:30.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:15:30 smithi167 ceph-mon[37140]: pgmap v1487: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:15:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:15:30 smithi138 ceph-mon[36954]: pgmap v1487: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:15:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:15:30 smithi100 ceph-mon[32599]: pgmap v1487: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:15:32.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:15:32 smithi167 ceph-mon[37140]: pgmap v1488: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:15:32.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:15:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:15:32.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:15:32 smithi138 ceph-mon[36954]: pgmap v1488: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:15:32.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:15:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:15:32.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:15:32 smithi100 ceph-mon[32599]: pgmap v1488: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:15:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:15:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:15:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:15:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:15:33.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:15:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:15:33.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:15:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:15:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:15:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:15:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:15:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:15:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:15:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:15:33.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:15:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:15:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:15:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:15:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:15:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:15:34.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:15:34 smithi138 ceph-mon[36954]: pgmap v1489: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:15:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:15:34 smithi100 ceph-mon[32599]: pgmap v1489: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:15:34.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:15:34 smithi167 ceph-mon[37140]: pgmap v1489: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:15:36.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:15:36 smithi138 ceph-mon[36954]: pgmap v1490: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:15:36.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:15:36 smithi100 ceph-mon[32599]: pgmap v1490: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:15:36.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:15:36 smithi167 ceph-mon[37140]: pgmap v1490: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:15:38.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:15:38 smithi138 ceph-mon[36954]: pgmap v1491: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:15:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:15:38 smithi100 ceph-mon[32599]: pgmap v1491: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:15:38.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:15:38 smithi167 ceph-mon[37140]: pgmap v1491: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:15:40.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:15:40 smithi138 ceph-mon[36954]: pgmap v1492: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:15:40.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:15:40 smithi100 ceph-mon[32599]: pgmap v1492: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:15:40.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:15:40 smithi167 ceph-mon[37140]: pgmap v1492: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:15:42.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:15:42 smithi138 ceph-mon[36954]: pgmap v1493: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:15:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:15:42 smithi100 ceph-mon[32599]: pgmap v1493: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:15:42.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:15:42 smithi167 ceph-mon[37140]: pgmap v1493: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:15:44.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:15:44 smithi138 ceph-mon[36954]: pgmap v1494: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:15:44.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:15:44 smithi100 ceph-mon[32599]: pgmap v1494: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:15:44.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:15:44 smithi167 ceph-mon[37140]: pgmap v1494: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:15:44.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:15:44.878+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:15:44.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:15:44.878+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:15:44.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:15:44.878+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:15:44.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:15:44.879+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:15:44.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:15:44.879+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:15:46.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:15:46 smithi138 ceph-mon[36954]: pgmap v1495: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:15:46.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:15:46 smithi100 ceph-mon[32599]: pgmap v1495: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:15:46.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:15:46 smithi167 ceph-mon[37140]: pgmap v1495: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:15:48.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:15:48 smithi138 ceph-mon[36954]: pgmap v1496: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:15:48.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:15:48 smithi100 ceph-mon[32599]: pgmap v1496: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:15:48.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:15:48 smithi167 ceph-mon[37140]: pgmap v1496: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:15:50.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:15:50 smithi138 ceph-mon[36954]: pgmap v1497: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:15:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:15:50 smithi100 ceph-mon[32599]: pgmap v1497: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:15:50.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:15:50 smithi167 ceph-mon[37140]: pgmap v1497: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:15:52.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:15:52 smithi138 ceph-mon[36954]: pgmap v1498: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:15:52.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:15:52 smithi100 ceph-mon[32599]: pgmap v1498: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:15:52.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:15:52 smithi167 ceph-mon[37140]: pgmap v1498: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:15:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:15:54 smithi138 ceph-mon[36954]: pgmap v1499: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:15:54.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:15:54 smithi100 ceph-mon[32599]: pgmap v1499: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:15:54.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:15:54 smithi167 ceph-mon[37140]: pgmap v1499: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:15:56.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:15:56 smithi138 ceph-mon[36954]: pgmap v1500: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:15:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:15:56 smithi100 ceph-mon[32599]: pgmap v1500: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:15:56.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:15:56 smithi167 ceph-mon[37140]: pgmap v1500: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:15:58.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:15:58 smithi138 ceph-mon[36954]: pgmap v1501: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:15:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:15:58 smithi100 ceph-mon[32599]: pgmap v1501: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:15:58.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:15:58 smithi167 ceph-mon[37140]: pgmap v1501: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:15:59.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:15:59.894+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:15:59.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:15:59.895+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:15:59.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:15:59.895+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:15:59.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:15:59.895+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:15:59.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:15:59.895+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:16:00.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:00 smithi138 ceph-mon[36954]: pgmap v1502: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:16:00.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:00 smithi100 ceph-mon[32599]: pgmap v1502: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:16:00.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:00 smithi167 ceph-mon[37140]: pgmap v1502: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:16:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:02 smithi138 ceph-mon[36954]: pgmap v1503: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:16:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:02 smithi100 ceph-mon[32599]: pgmap v1503: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:16:02.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:02 smithi167 ceph-mon[37140]: pgmap v1503: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:16:04.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:04 smithi138 ceph-mon[36954]: pgmap v1504: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:16:04.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:04 smithi100 ceph-mon[32599]: pgmap v1504: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:16:04.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:04 smithi167 ceph-mon[37140]: pgmap v1504: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:16:06.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:06 smithi138 ceph-mon[36954]: pgmap v1505: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:16:06.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:06 smithi100 ceph-mon[32599]: pgmap v1505: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:16:06.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:06 smithi167 ceph-mon[37140]: pgmap v1505: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:16:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:08 smithi138 ceph-mon[36954]: pgmap v1506: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:16:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:08 smithi138 ceph-mon[36954]: osdmap e144: 12 total, 12 up, 12 in 2024-07-26T14:16:08.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:08 smithi100 ceph-mon[32599]: pgmap v1506: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:16:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:08 smithi100 ceph-mon[32599]: osdmap e144: 12 total, 12 up, 12 in 2024-07-26T14:16:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:08 smithi167 ceph-mon[37140]: pgmap v1506: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:16:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:08 smithi167 ceph-mon[37140]: osdmap e144: 12 total, 12 up, 12 in 2024-07-26T14:16:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:10 smithi138 ceph-mon[36954]: pgmap v1508: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:16:10.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:10 smithi100 ceph-mon[32599]: pgmap v1508: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:16:10.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:10 smithi167 ceph-mon[37140]: pgmap v1508: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:16:12.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:12 smithi138 ceph-mon[36954]: pgmap v1509: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:16:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:12 smithi100 ceph-mon[32599]: pgmap v1509: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:16:12.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:12 smithi167 ceph-mon[37140]: pgmap v1509: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:16:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:14 smithi138 ceph-mon[36954]: pgmap v1510: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:16:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:14 smithi138 ceph-mon[36954]: osdmap e145: 12 total, 12 up, 12 in 2024-07-26T14:16:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:14 smithi100 ceph-mon[32599]: pgmap v1510: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:16:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:14 smithi100 ceph-mon[32599]: osdmap e145: 12 total, 12 up, 12 in 2024-07-26T14:16:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:14 smithi167 ceph-mon[37140]: pgmap v1510: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:16:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:14 smithi167 ceph-mon[37140]: osdmap e145: 12 total, 12 up, 12 in 2024-07-26T14:16:14.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:16:14.910+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:16:14.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:16:14.911+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:16:14.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:16:14.911+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:16:14.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:16:14.911+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:16:14.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:16:14.911+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:16:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:16 smithi138 ceph-mon[36954]: pgmap v1512: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:16:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:16 smithi100 ceph-mon[32599]: pgmap v1512: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:16:16.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:16 smithi167 ceph-mon[37140]: pgmap v1512: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:16:18.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:18 smithi138 ceph-mon[36954]: pgmap v1513: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 211 B/s wr, 6 op/s 2024-07-26T14:16:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:18 smithi100 ceph-mon[32599]: pgmap v1513: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 211 B/s wr, 6 op/s 2024-07-26T14:16:18.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:18 smithi167 ceph-mon[37140]: pgmap v1513: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 211 B/s wr, 6 op/s 2024-07-26T14:16:20.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:20 smithi138 ceph-mon[36954]: pgmap v1514: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:16:20.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:20 smithi100 ceph-mon[32599]: pgmap v1514: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:16:20.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:20 smithi167 ceph-mon[37140]: pgmap v1514: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:16:22.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:22 smithi100 ceph-mon[32599]: pgmap v1515: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:16:22.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:22 smithi167 ceph-mon[37140]: pgmap v1515: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:16:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:22 smithi138 ceph-mon[36954]: pgmap v1515: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:16:24.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:24 smithi100 ceph-mon[32599]: pgmap v1516: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:16:24.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:24 smithi167 ceph-mon[37140]: pgmap v1516: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:16:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:24 smithi138 ceph-mon[36954]: pgmap v1516: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:16:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:26 smithi100 ceph-mon[32599]: pgmap v1517: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:16:26.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:26 smithi167 ceph-mon[37140]: pgmap v1517: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:16:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:26 smithi138 ceph-mon[36954]: pgmap v1517: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:16:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:28 smithi100 ceph-mon[32599]: pgmap v1518: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:16:28.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:28 smithi167 ceph-mon[37140]: pgmap v1518: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:16:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:28 smithi138 ceph-mon[36954]: pgmap v1518: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:16:29.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:16:29.926+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:16:29.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:16:29.927+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:16:29.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:16:29.927+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:16:29.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:16:29.927+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:16:29.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:16:29.927+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:16:30.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:30 smithi100 ceph-mon[32599]: pgmap v1519: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:16:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:30 smithi167 ceph-mon[37140]: pgmap v1519: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:16:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:30 smithi138 ceph-mon[36954]: pgmap v1519: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:16:32.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:32 smithi100 ceph-mon[32599]: pgmap v1520: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:16:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:16:32.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:32 smithi167 ceph-mon[37140]: pgmap v1520: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:16:32.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:16:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:32 smithi138 ceph-mon[36954]: pgmap v1520: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:16:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:16:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:16:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:16:33.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:16:33.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:16:33.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:16:33.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:16:33.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:16:33.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:16:33.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:16:33.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:16:33.747 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:16:33.747 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:16:33.747 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:16:33.747 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:16:33.747 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:16:33.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:16:33.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:16:33.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:16:33.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:16:33.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:16:33.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:16:33.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:16:33.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:16:33.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:16:33.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:16:33.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:16:33.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:16:33.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:16:33.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:16:33.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:16:33.818 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:16:33.818 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:16:33.818 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:16:33.819 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:16:33.819 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:16:33.819 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:16:33.819 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:16:33.820 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:16:33.820 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:16:33.820 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:16:33.820 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:16:33.820 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:16:33.820 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:16:33.820 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:16:33.820 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:16:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:34 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T14:16:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:34 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T14:16:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:34 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T14:16:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:34 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T14:16:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:34 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T14:16:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:34 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T14:16:34.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:34 smithi100 ceph-mon[32599]: pgmap v1521: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:16:34.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:34 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T14:16:34.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:34 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T14:16:34.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:34 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T14:16:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:34 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T14:16:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:34 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T14:16:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:34 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T14:16:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:34 smithi167 ceph-mon[37140]: pgmap v1521: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:16:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:34 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T14:16:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:34 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T14:16:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:34 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T14:16:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:34 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T14:16:34.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:34 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T14:16:34.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:34 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T14:16:34.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:34 smithi138 ceph-mon[36954]: pgmap v1521: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:16:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:36 smithi100 ceph-mon[32599]: pgmap v1522: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:16:36.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:36 smithi167 ceph-mon[37140]: pgmap v1522: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:16:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:36 smithi138 ceph-mon[36954]: pgmap v1522: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:16:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:38 smithi100 ceph-mon[32599]: pgmap v1523: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:16:38.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:38 smithi167 ceph-mon[37140]: pgmap v1523: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:16:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:38 smithi138 ceph-mon[36954]: pgmap v1523: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:16:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:40 smithi100 ceph-mon[32599]: pgmap v1524: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:16:40.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:40 smithi167 ceph-mon[37140]: pgmap v1524: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:16:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:40 smithi138 ceph-mon[36954]: pgmap v1524: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:16:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:42 smithi100 ceph-mon[32599]: pgmap v1525: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:16:42.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:42 smithi167 ceph-mon[37140]: pgmap v1525: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:16:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:42 smithi138 ceph-mon[36954]: pgmap v1525: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:16:44.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:44 smithi100 ceph-mon[32599]: pgmap v1526: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:16:44.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:44 smithi167 ceph-mon[37140]: pgmap v1526: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:16:44.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:16:44.937+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:16:44.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:16:44.938+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:16:44.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:16:44.938+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:16:44.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:16:44.938+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:16:44.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:16:44.938+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:16:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:44 smithi138 ceph-mon[36954]: pgmap v1526: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:16:46.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:46 smithi100 ceph-mon[32599]: pgmap v1527: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:16:46.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:46 smithi167 ceph-mon[37140]: pgmap v1527: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:16:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:46 smithi138 ceph-mon[36954]: pgmap v1527: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:16:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:48 smithi100 ceph-mon[32599]: pgmap v1528: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:16:48.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:48 smithi167 ceph-mon[37140]: pgmap v1528: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:16:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:48 smithi138 ceph-mon[36954]: pgmap v1528: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:16:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:50 smithi100 ceph-mon[32599]: pgmap v1529: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:16:50.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:50 smithi167 ceph-mon[37140]: pgmap v1529: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:16:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:50 smithi138 ceph-mon[36954]: pgmap v1529: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:16:52.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:52 smithi100 ceph-mon[32599]: pgmap v1530: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:16:52.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:52 smithi167 ceph-mon[37140]: pgmap v1530: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:16:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:52 smithi138 ceph-mon[36954]: pgmap v1530: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:16:54.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:54 smithi100 ceph-mon[32599]: pgmap v1531: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:16:54.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:54 smithi167 ceph-mon[37140]: pgmap v1531: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:16:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:54 smithi138 ceph-mon[36954]: pgmap v1531: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:16:56.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:56 smithi167 ceph-mon[37140]: pgmap v1532: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:16:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:56 smithi138 ceph-mon[36954]: pgmap v1532: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:16:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:56 smithi100 ceph-mon[32599]: pgmap v1532: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:16:58.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:16:58 smithi167 ceph-mon[37140]: pgmap v1533: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:16:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:16:58 smithi138 ceph-mon[36954]: pgmap v1533: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:16:58.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:16:58 smithi100 ceph-mon[32599]: pgmap v1533: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:16:59.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:16:59.953+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:16:59.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:16:59.954+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:16:59.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:16:59.954+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:16:59.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:16:59.954+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:16:59.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:16:59.954+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:17:00.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:17:00 smithi167 ceph-mon[37140]: pgmap v1534: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:17:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:17:00 smithi138 ceph-mon[36954]: pgmap v1534: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:17:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:17:00 smithi100 ceph-mon[32599]: pgmap v1534: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:17:02.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:17:02 smithi167 ceph-mon[37140]: pgmap v1535: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:17:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:17:02 smithi138 ceph-mon[36954]: pgmap v1535: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:17:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:17:02 smithi100 ceph-mon[32599]: pgmap v1535: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:17:04.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:17:04 smithi167 ceph-mon[37140]: pgmap v1536: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:17:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:17:04 smithi138 ceph-mon[36954]: pgmap v1536: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:17:04.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:17:04 smithi100 ceph-mon[32599]: pgmap v1536: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:17:06.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:17:06 smithi167 ceph-mon[37140]: pgmap v1537: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:17:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:17:06 smithi138 ceph-mon[36954]: pgmap v1537: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:17:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:17:06 smithi100 ceph-mon[32599]: pgmap v1537: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:17:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:17:08 smithi167 ceph-mon[37140]: pgmap v1538: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:17:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:17:08 smithi167 ceph-mon[37140]: osdmap e146: 12 total, 12 up, 12 in 2024-07-26T14:17:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:17:08 smithi138 ceph-mon[36954]: pgmap v1538: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:17:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:17:08 smithi138 ceph-mon[36954]: osdmap e146: 12 total, 12 up, 12 in 2024-07-26T14:17:08.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:17:08 smithi100 ceph-mon[32599]: pgmap v1538: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:17:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:17:08 smithi100 ceph-mon[32599]: osdmap e146: 12 total, 12 up, 12 in 2024-07-26T14:17:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:17:10 smithi138 ceph-mon[36954]: pgmap v1540: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:17:10.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:17:10 smithi100 ceph-mon[32599]: pgmap v1540: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:17:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:17:10 smithi167 ceph-mon[37140]: pgmap v1540: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:17:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:17:12 smithi138 ceph-mon[36954]: pgmap v1541: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:17:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:17:12 smithi100 ceph-mon[32599]: pgmap v1541: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:17:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:17:12 smithi167 ceph-mon[37140]: pgmap v1541: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:17:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:17:14 smithi138 ceph-mon[36954]: pgmap v1542: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:17:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:17:14 smithi138 ceph-mon[36954]: osdmap e147: 12 total, 12 up, 12 in 2024-07-26T14:17:14.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:17:14.963+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:17:14.964 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:17:14 smithi100 ceph-mon[32599]: pgmap v1542: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:17:14.965 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:17:14 smithi100 ceph-mon[32599]: osdmap e147: 12 total, 12 up, 12 in 2024-07-26T14:17:14.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:17:14.964+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:17:14.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:17:14.964+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:17:14.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:17:14.964+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:17:14.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:17:14.964+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:17:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:17:14 smithi167 ceph-mon[37140]: pgmap v1542: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:17:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:17:14 smithi167 ceph-mon[37140]: osdmap e147: 12 total, 12 up, 12 in 2024-07-26T14:17:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:17:16 smithi138 ceph-mon[36954]: pgmap v1544: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:17:16.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:17:16 smithi100 ceph-mon[32599]: pgmap v1544: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:17:17.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:17:16 smithi167 ceph-mon[37140]: pgmap v1544: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:17:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:17:18 smithi138 ceph-mon[36954]: pgmap v1545: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 213 B/s wr, 6 op/s 2024-07-26T14:17:18.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:17:18 smithi100 ceph-mon[32599]: pgmap v1545: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 213 B/s wr, 6 op/s 2024-07-26T14:17:19.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:17:18 smithi167 ceph-mon[37140]: pgmap v1545: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 213 B/s wr, 6 op/s 2024-07-26T14:17:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:17:20 smithi138 ceph-mon[36954]: pgmap v1546: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:17:20.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:17:20 smithi100 ceph-mon[32599]: pgmap v1546: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:17:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:17:20 smithi167 ceph-mon[37140]: pgmap v1546: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:17:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:17:22 smithi138 ceph-mon[36954]: pgmap v1547: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:17:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:17:22 smithi100 ceph-mon[32599]: pgmap v1547: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:17:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:17:22 smithi167 ceph-mon[37140]: pgmap v1547: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:17:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:17:24 smithi138 ceph-mon[36954]: pgmap v1548: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:17:24.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:17:24 smithi100 ceph-mon[32599]: pgmap v1548: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:17:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:17:24 smithi167 ceph-mon[37140]: pgmap v1548: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:17:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:17:26 smithi138 ceph-mon[36954]: pgmap v1549: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:17:26.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:17:26 smithi100 ceph-mon[32599]: pgmap v1549: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:17:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:17:26 smithi167 ceph-mon[37140]: pgmap v1549: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:17:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:17:28 smithi138 ceph-mon[36954]: pgmap v1550: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:17:28.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:17:28 smithi100 ceph-mon[32599]: pgmap v1550: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:17:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:17:28 smithi167 ceph-mon[37140]: pgmap v1550: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:17:29.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:17:29.979+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:17:29.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:17:29.980+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:17:29.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:17:29.980+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:17:29.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:17:29.980+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:17:29.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:17:29.980+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:17:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:17:30 smithi138 ceph-mon[36954]: pgmap v1551: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:17:30.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:17:30 smithi100 ceph-mon[32599]: pgmap v1551: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:17:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:17:30 smithi167 ceph-mon[37140]: pgmap v1551: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:17:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:17:32 smithi138 ceph-mon[36954]: pgmap v1552: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:17:32.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:17:32 smithi100 ceph-mon[32599]: pgmap v1552: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:17:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:17:32 smithi167 ceph-mon[37140]: pgmap v1552: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:17:33.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:17:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:17:33.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:17:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:17:33.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:17:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:17:33.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:17:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:17:33.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:17:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:17:33.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:17:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:17:33.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:17:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:17:33.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:17:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:17:34.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:17:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:17:34.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:17:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:17:34.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:17:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:17:34.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:17:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:17:34.841 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:17:34 smithi138 ceph-mon[36954]: pgmap v1553: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:17:34.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:17:34 smithi100 ceph-mon[32599]: pgmap v1553: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:17:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:17:34 smithi167 ceph-mon[37140]: pgmap v1553: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:17:36.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:17:36 smithi167 ceph-mon[37140]: pgmap v1554: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:17:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:17:36 smithi138 ceph-mon[36954]: pgmap v1554: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:17:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:17:36 smithi100 ceph-mon[32599]: pgmap v1554: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:17:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:17:38 smithi138 ceph-mon[36954]: pgmap v1555: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:17:38.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:17:38 smithi100 ceph-mon[32599]: pgmap v1555: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:17:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:17:38 smithi167 ceph-mon[37140]: pgmap v1555: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:17:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:17:40 smithi138 ceph-mon[36954]: pgmap v1556: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:17:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:17:40 smithi100 ceph-mon[32599]: pgmap v1556: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:17:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:17:40 smithi167 ceph-mon[37140]: pgmap v1556: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:17:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:17:42 smithi138 ceph-mon[36954]: pgmap v1557: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:17:42.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:17:42 smithi100 ceph-mon[32599]: pgmap v1557: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:17:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:17:42 smithi167 ceph-mon[37140]: pgmap v1557: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:17:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:17:44 smithi138 ceph-mon[36954]: pgmap v1558: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:17:44.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:17:44 smithi100 ceph-mon[32599]: pgmap v1558: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:17:44.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:17:44.995+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:17:44.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:17:44.995+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:17:44.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:17:44.995+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:17:44.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:17:44.995+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:17:44.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:17:44.996+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:17:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:17:44 smithi167 ceph-mon[37140]: pgmap v1558: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:17:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:17:46 smithi138 ceph-mon[36954]: pgmap v1559: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:17:46.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:17:46 smithi100 ceph-mon[32599]: pgmap v1559: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:17:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:17:46 smithi167 ceph-mon[37140]: pgmap v1559: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:17:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:17:48 smithi138 ceph-mon[36954]: pgmap v1560: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:17:48.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:17:48 smithi100 ceph-mon[32599]: pgmap v1560: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:17:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:17:48 smithi167 ceph-mon[37140]: pgmap v1560: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:17:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:17:50 smithi138 ceph-mon[36954]: pgmap v1561: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:17:50.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:17:50 smithi100 ceph-mon[32599]: pgmap v1561: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:17:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:17:50 smithi167 ceph-mon[37140]: pgmap v1561: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:17:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:17:52 smithi138 ceph-mon[36954]: pgmap v1562: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:17:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:17:52 smithi100 ceph-mon[32599]: pgmap v1562: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:17:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:17:52 smithi167 ceph-mon[37140]: pgmap v1562: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:17:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:17:54 smithi138 ceph-mon[36954]: pgmap v1563: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:17:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:17:54 smithi100 ceph-mon[32599]: pgmap v1563: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:17:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:17:54 smithi167 ceph-mon[37140]: pgmap v1563: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:17:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:17:56 smithi138 ceph-mon[36954]: pgmap v1564: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:17:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:17:56 smithi100 ceph-mon[32599]: pgmap v1564: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:17:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:17:56 smithi167 ceph-mon[37140]: pgmap v1564: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:17:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:17:58 smithi138 ceph-mon[36954]: pgmap v1565: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:17:58.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:17:58 smithi100 ceph-mon[32599]: pgmap v1565: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:17:59.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:17:58 smithi167 ceph-mon[37140]: pgmap v1565: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:18:00.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:18:00.011+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:18:00.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:18:00.012+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:18:00.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:18:00.012+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:18:00.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:18:00.012+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:18:00.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:18:00.012+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:18:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:18:00 smithi138 ceph-mon[36954]: pgmap v1566: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:18:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:18:00 smithi100 ceph-mon[32599]: pgmap v1566: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:18:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:18:00 smithi167 ceph-mon[37140]: pgmap v1566: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:18:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:18:02 smithi138 ceph-mon[36954]: pgmap v1567: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:18:02.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:18:02 smithi100 ceph-mon[32599]: pgmap v1567: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:18:03.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:18:02 smithi167 ceph-mon[37140]: pgmap v1567: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:18:03.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:18:03.028+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 msgr2=0x7fe5940c0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-07-26T14:18:03.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:18:03.028+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 msgr2=0x7fe5940c0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-07-26T14:18:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:18:03.028+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe5900049f0 tx=0x7fe590046040 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:18:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:18:03.028+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe5900049f0 tx=0x7fe590046040 comp rx=0 tx=0).stop 2024-07-26T14:18:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:18:03.028+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 msgr2=0x7fe5940c0480 unknown :-1 s=STATE_CLOSED l=1).tick see no progress in more than 10000000 us during connecting to v2:172.21.15.100:6800/3937009534, fault. 2024-07-26T14:18:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:18:03.028+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 msgr2=0x7fe5940c0480 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-07-26T14:18:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:18:03.028+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T14:18:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:18:03.028+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b3d40 0x7fe5940c09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T14:18:03.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:18:03.028+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b3d40 0x7fe5940c09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:18:03.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:18:03.028+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b3d40 0x7fe5940c09c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe590005d00 tx=0x7fe59005a040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T14:18:03.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:18:03.267+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c003690 con 0x7fe5a81087a0 2024-07-26T14:18:03.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:18:03.267+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T14:18:03.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:18:03.267+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 18 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c003690 con 0x7fe5a81087a0 2024-07-26T14:18:03.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:18:03.267+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 18 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T14:18:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:18:04 smithi138 ceph-mon[36954]: pgmap v1568: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:18:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:18:04 smithi100 ceph-mon[32599]: pgmap v1568: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:18:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:18:04 smithi167 ceph-mon[37140]: pgmap v1568: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:18:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:18:06 smithi138 ceph-mon[36954]: pgmap v1569: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:18:06.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:18:06 smithi100 ceph-mon[32599]: pgmap v1569: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:18:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:18:06 smithi167 ceph-mon[37140]: pgmap v1569: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:18:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:18:08 smithi138 ceph-mon[36954]: pgmap v1570: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:18:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:18:08 smithi138 ceph-mon[36954]: osdmap e148: 12 total, 12 up, 12 in 2024-07-26T14:18:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:18:08 smithi100 ceph-mon[32599]: pgmap v1570: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:18:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:18:08 smithi100 ceph-mon[32599]: osdmap e148: 12 total, 12 up, 12 in 2024-07-26T14:18:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:18:08 smithi167 ceph-mon[37140]: pgmap v1570: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:18:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:18:08 smithi167 ceph-mon[37140]: osdmap e148: 12 total, 12 up, 12 in 2024-07-26T14:18:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:18:10 smithi138 ceph-mon[36954]: pgmap v1572: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:18:10.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:18:10 smithi100 ceph-mon[32599]: pgmap v1572: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:18:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:18:10 smithi167 ceph-mon[37140]: pgmap v1572: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:18:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:18:12 smithi138 ceph-mon[36954]: pgmap v1573: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:18:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:18:12 smithi100 ceph-mon[32599]: pgmap v1573: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:18:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:18:12 smithi167 ceph-mon[37140]: pgmap v1573: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:18:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:18:14 smithi138 ceph-mon[36954]: pgmap v1574: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:18:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:18:14 smithi138 ceph-mon[36954]: osdmap e149: 12 total, 12 up, 12 in 2024-07-26T14:18:14.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:18:14 smithi100 ceph-mon[32599]: pgmap v1574: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:18:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:18:14 smithi100 ceph-mon[32599]: osdmap e149: 12 total, 12 up, 12 in 2024-07-26T14:18:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:18:14 smithi167 ceph-mon[37140]: pgmap v1574: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:18:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:18:14 smithi167 ceph-mon[37140]: osdmap e149: 12 total, 12 up, 12 in 2024-07-26T14:18:15.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:18:15.025+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:18:15.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:18:15.025+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:18:15.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:18:15.025+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:18:15.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:18:15.025+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:18:15.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:18:15.025+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:18:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:18:16 smithi138 ceph-mon[36954]: pgmap v1576: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:18:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:18:16 smithi100 ceph-mon[32599]: pgmap v1576: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:18:17.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:18:16 smithi167 ceph-mon[37140]: pgmap v1576: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:18:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:18:18 smithi138 ceph-mon[36954]: pgmap v1577: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 215 B/s wr, 6 op/s 2024-07-26T14:18:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:18:18 smithi100 ceph-mon[32599]: pgmap v1577: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 215 B/s wr, 6 op/s 2024-07-26T14:18:19.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:18:18 smithi167 ceph-mon[37140]: pgmap v1577: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 215 B/s wr, 6 op/s 2024-07-26T14:18:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:18:20 smithi138 ceph-mon[36954]: pgmap v1578: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:18:20.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:18:20 smithi100 ceph-mon[32599]: pgmap v1578: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:18:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:18:20 smithi167 ceph-mon[37140]: pgmap v1578: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:18:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:18:22 smithi138 ceph-mon[36954]: pgmap v1579: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:18:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:18:22 smithi100 ceph-mon[32599]: pgmap v1579: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:18:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:18:22 smithi167 ceph-mon[37140]: pgmap v1579: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:18:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:18:24 smithi138 ceph-mon[36954]: pgmap v1580: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:18:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:18:24 smithi100 ceph-mon[32599]: pgmap v1580: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:18:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:18:24 smithi167 ceph-mon[37140]: pgmap v1580: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:18:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:18:26 smithi138 ceph-mon[36954]: pgmap v1581: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:18:26.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:18:26 smithi100 ceph-mon[32599]: pgmap v1581: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:18:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:18:26 smithi167 ceph-mon[37140]: pgmap v1581: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:18:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:18:28 smithi138 ceph-mon[36954]: pgmap v1582: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:18:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:18:28 smithi100 ceph-mon[32599]: pgmap v1582: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:18:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:18:28 smithi167 ceph-mon[37140]: pgmap v1582: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:18:30.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:18:30.031+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:18:30.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:18:30.032+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:18:30.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:18:30.032+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:18:30.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:18:30.032+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:18:30.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:18:30.032+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:18:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:18:30 smithi138 ceph-mon[36954]: pgmap v1583: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:18:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:18:30 smithi100 ceph-mon[32599]: pgmap v1583: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:18:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:18:30 smithi167 ceph-mon[37140]: pgmap v1583: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:18:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:18:32 smithi138 ceph-mon[36954]: pgmap v1584: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:18:32.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:18:32 smithi100 ceph-mon[32599]: pgmap v1584: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:18:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:18:32 smithi167 ceph-mon[37140]: pgmap v1584: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:18:33.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:18:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:18:33.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:18:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:18:34.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:18:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:18:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:18:34 smithi138 ceph-mon[36954]: pgmap v1585: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:18:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:18:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:18:34.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:18:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:18:34.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:18:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:18:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:18:34 smithi100 ceph-mon[32599]: pgmap v1585: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:18:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:18:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:18:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:18:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:18:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:18:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:18:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:18:34 smithi167 ceph-mon[37140]: pgmap v1585: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:18:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:18:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:18:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:18:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:18:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:18:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:18:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:18:36 smithi138 ceph-mon[36954]: pgmap v1586: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:18:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:18:36 smithi100 ceph-mon[32599]: pgmap v1586: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:18:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:18:36 smithi167 ceph-mon[37140]: pgmap v1586: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:18:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:18:38 smithi138 ceph-mon[36954]: pgmap v1587: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:18:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:18:38 smithi100 ceph-mon[32599]: pgmap v1587: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:18:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:18:38 smithi167 ceph-mon[37140]: pgmap v1587: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:18:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:18:40 smithi138 ceph-mon[36954]: pgmap v1588: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:18:40.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:18:40 smithi100 ceph-mon[32599]: pgmap v1588: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:18:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:18:40 smithi167 ceph-mon[37140]: pgmap v1588: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:18:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:18:42 smithi100 ceph-mon[32599]: pgmap v1589: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:18:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:18:42 smithi167 ceph-mon[37140]: pgmap v1589: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:18:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:18:42 smithi138 ceph-mon[36954]: pgmap v1589: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:18:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:18:44 smithi100 ceph-mon[32599]: pgmap v1590: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:18:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:18:44 smithi167 ceph-mon[37140]: pgmap v1590: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:18:45.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:18:45.047+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:18:45.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:18:45.048+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:18:45.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:18:45.048+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:18:45.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:18:45.048+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:18:45.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:18:45.048+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:18:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:18:44 smithi138 ceph-mon[36954]: pgmap v1590: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:18:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:18:46 smithi100 ceph-mon[32599]: pgmap v1591: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:18:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:18:46 smithi167 ceph-mon[37140]: pgmap v1591: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:18:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:18:46 smithi138 ceph-mon[36954]: pgmap v1591: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:18:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:18:48 smithi100 ceph-mon[32599]: pgmap v1592: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:18:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:18:48 smithi167 ceph-mon[37140]: pgmap v1592: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:18:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:18:48 smithi138 ceph-mon[36954]: pgmap v1592: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:18:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:18:50 smithi100 ceph-mon[32599]: pgmap v1593: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:18:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:18:50 smithi167 ceph-mon[37140]: pgmap v1593: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:18:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:18:50 smithi138 ceph-mon[36954]: pgmap v1593: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:18:52.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:18:52 smithi100 ceph-mon[32599]: pgmap v1594: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:18:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:18:52 smithi167 ceph-mon[37140]: pgmap v1594: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:18:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:18:52 smithi138 ceph-mon[36954]: pgmap v1594: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:18:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:18:54 smithi100 ceph-mon[32599]: pgmap v1595: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:18:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:18:54 smithi167 ceph-mon[37140]: pgmap v1595: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:18:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:18:54 smithi138 ceph-mon[36954]: pgmap v1595: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:18:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:18:56 smithi100 ceph-mon[32599]: pgmap v1596: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:18:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:18:56 smithi167 ceph-mon[37140]: pgmap v1596: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:18:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:18:56 smithi138 ceph-mon[36954]: pgmap v1596: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:18:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:18:58 smithi100 ceph-mon[32599]: pgmap v1597: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:18:59.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:18:58 smithi167 ceph-mon[37140]: pgmap v1597: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:18:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:18:58 smithi138 ceph-mon[36954]: pgmap v1597: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:19:00.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:19:00.062+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:19:00.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:19:00.063+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:19:00.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:19:00.063+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:19:00.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:19:00.063+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:19:00.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:19:00.063+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:19:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:00 smithi100 ceph-mon[32599]: pgmap v1598: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:19:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:00 smithi167 ceph-mon[37140]: pgmap v1598: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:19:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:00 smithi138 ceph-mon[36954]: pgmap v1598: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:19:02.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:02 smithi100 ceph-mon[32599]: pgmap v1599: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:19:03.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:02 smithi167 ceph-mon[37140]: pgmap v1599: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:19:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:02 smithi138 ceph-mon[36954]: pgmap v1599: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:19:04.992 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:04 smithi138 ceph-mon[36954]: pgmap v1600: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:19:04.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:04 smithi100 ceph-mon[32599]: pgmap v1600: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:19:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:04 smithi167 ceph-mon[37140]: pgmap v1600: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:19:06.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:06 smithi100 ceph-mon[32599]: pgmap v1601: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:19:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:06 smithi167 ceph-mon[37140]: pgmap v1601: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:19:07.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:06 smithi138 ceph-mon[36954]: pgmap v1601: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:19:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:08 smithi100 ceph-mon[32599]: pgmap v1602: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:19:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:08 smithi100 ceph-mon[32599]: osdmap e150: 12 total, 12 up, 12 in 2024-07-26T14:19:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:08 smithi167 ceph-mon[37140]: pgmap v1602: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:19:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:08 smithi167 ceph-mon[37140]: osdmap e150: 12 total, 12 up, 12 in 2024-07-26T14:19:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:08 smithi138 ceph-mon[36954]: pgmap v1602: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:19:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:08 smithi138 ceph-mon[36954]: osdmap e150: 12 total, 12 up, 12 in 2024-07-26T14:19:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:10 smithi100 ceph-mon[32599]: pgmap v1604: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:19:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:10 smithi167 ceph-mon[37140]: pgmap v1604: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:19:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:10 smithi138 ceph-mon[36954]: pgmap v1604: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:19:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:12 smithi167 ceph-mon[37140]: pgmap v1605: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:19:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:12 smithi138 ceph-mon[36954]: pgmap v1605: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:19:13.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:12 smithi100 ceph-mon[32599]: pgmap v1605: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:19:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:14 smithi167 ceph-mon[37140]: pgmap v1606: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:19:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:14 smithi167 ceph-mon[37140]: osdmap e151: 12 total, 12 up, 12 in 2024-07-26T14:19:15.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:19:15.079+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:19:15.081 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:14 smithi100 ceph-mon[32599]: pgmap v1606: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:19:15.081 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:14 smithi100 ceph-mon[32599]: osdmap e151: 12 total, 12 up, 12 in 2024-07-26T14:19:15.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:19:15.080+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:19:15.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:19:15.080+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:19:15.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:19:15.080+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:19:15.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:19:15.080+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:19:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:14 smithi138 ceph-mon[36954]: pgmap v1606: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:19:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:14 smithi138 ceph-mon[36954]: osdmap e151: 12 total, 12 up, 12 in 2024-07-26T14:19:17.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:16 smithi167 ceph-mon[37140]: pgmap v1608: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:19:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:16 smithi138 ceph-mon[36954]: pgmap v1608: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:19:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:16 smithi100 ceph-mon[32599]: pgmap v1608: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:19:19.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:18 smithi167 ceph-mon[37140]: pgmap v1609: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 217 B/s wr, 6 op/s 2024-07-26T14:19:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:18 smithi138 ceph-mon[36954]: pgmap v1609: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 217 B/s wr, 6 op/s 2024-07-26T14:19:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:18 smithi100 ceph-mon[32599]: pgmap v1609: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 217 B/s wr, 6 op/s 2024-07-26T14:19:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:20 smithi167 ceph-mon[37140]: pgmap v1610: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:19:21.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:20 smithi138 ceph-mon[36954]: pgmap v1610: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:19:21.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:20 smithi100 ceph-mon[32599]: pgmap v1610: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:19:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:22 smithi167 ceph-mon[37140]: pgmap v1611: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:19:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:22 smithi138 ceph-mon[36954]: pgmap v1611: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:19:23.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:22 smithi100 ceph-mon[32599]: pgmap v1611: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:19:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:24 smithi167 ceph-mon[37140]: pgmap v1612: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:19:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:24 smithi138 ceph-mon[36954]: pgmap v1612: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:19:25.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:24 smithi100 ceph-mon[32599]: pgmap v1612: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:19:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:26 smithi167 ceph-mon[37140]: pgmap v1613: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:19:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:26 smithi138 ceph-mon[36954]: pgmap v1613: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:19:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:26 smithi100 ceph-mon[32599]: pgmap v1613: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:19:29.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:28 smithi138 ceph-mon[36954]: pgmap v1614: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:19:29.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:28 smithi100 ceph-mon[32599]: pgmap v1614: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:19:29.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:28 smithi167 ceph-mon[37140]: pgmap v1614: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:19:30.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:19:30.090+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:19:30.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:19:30.090+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:19:30.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:19:30.090+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:19:30.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:19:30.090+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:19:30.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:19:30.090+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:19:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:30 smithi138 ceph-mon[36954]: pgmap v1615: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:19:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:30 smithi100 ceph-mon[32599]: pgmap v1615: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:19:31.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:30 smithi167 ceph-mon[37140]: pgmap v1615: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:19:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:32 smithi138 ceph-mon[36954]: pgmap v1616: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:19:33.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:32 smithi100 ceph-mon[32599]: pgmap v1616: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:19:33.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:32 smithi167 ceph-mon[37140]: pgmap v1616: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:19:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:34 smithi138 ceph-mon[36954]: pgmap v1617: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:19:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:19:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:34 smithi100 ceph-mon[32599]: pgmap v1617: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:19:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:19:35.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:34 smithi167 ceph-mon[37140]: pgmap v1617: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:19:35.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:19:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:36 smithi138 ceph-mon[36954]: pgmap v1618: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:19:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:19:36.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:19:36.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:19:36.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:19:36.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:19:36.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:19:36.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:19:36.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:19:36.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:19:36.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:36 smithi100 ceph-mon[32599]: pgmap v1618: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:19:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:19:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:19:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:19:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:19:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:19:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:19:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:19:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:19:36.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:19:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:36 smithi167 ceph-mon[37140]: pgmap v1618: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:19:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:19:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:19:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:19:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:19:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:19:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:19:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:19:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:19:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:19:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:38 smithi138 ceph-mon[36954]: pgmap v1619: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:19:38.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:38 smithi100 ceph-mon[32599]: pgmap v1619: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:19:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:38 smithi167 ceph-mon[37140]: pgmap v1619: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:19:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:40 smithi138 ceph-mon[36954]: pgmap v1620: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:19:40.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:40 smithi100 ceph-mon[32599]: pgmap v1620: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:19:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:40 smithi167 ceph-mon[37140]: pgmap v1620: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:19:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:42 smithi138 ceph-mon[36954]: pgmap v1621: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:19:42.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:42 smithi100 ceph-mon[32599]: pgmap v1621: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:19:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:42 smithi167 ceph-mon[37140]: pgmap v1621: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:19:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:44 smithi138 ceph-mon[36954]: pgmap v1622: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:19:44.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:44 smithi100 ceph-mon[32599]: pgmap v1622: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:19:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:44 smithi167 ceph-mon[37140]: pgmap v1622: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:19:45.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:19:45.105+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:19:45.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:19:45.106+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:19:45.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:19:45.106+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:19:45.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:19:45.106+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:19:45.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:19:45.106+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:19:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:46 smithi138 ceph-mon[36954]: pgmap v1623: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:19:46.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:46 smithi100 ceph-mon[32599]: pgmap v1623: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:19:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:46 smithi167 ceph-mon[37140]: pgmap v1623: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:19:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:48 smithi138 ceph-mon[36954]: pgmap v1624: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:19:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:48 smithi100 ceph-mon[32599]: pgmap v1624: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:19:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:48 smithi167 ceph-mon[37140]: pgmap v1624: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:19:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:50 smithi138 ceph-mon[36954]: pgmap v1625: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:19:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:50 smithi100 ceph-mon[32599]: pgmap v1625: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:19:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:50 smithi167 ceph-mon[37140]: pgmap v1625: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:19:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:52 smithi138 ceph-mon[36954]: pgmap v1626: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:19:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:52 smithi100 ceph-mon[32599]: pgmap v1626: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:19:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:52 smithi167 ceph-mon[37140]: pgmap v1626: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:19:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:54 smithi138 ceph-mon[36954]: pgmap v1627: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:19:54.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:54 smithi100 ceph-mon[32599]: pgmap v1627: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:19:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:54 smithi167 ceph-mon[37140]: pgmap v1627: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:19:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:56 smithi138 ceph-mon[36954]: pgmap v1628: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:19:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:56 smithi100 ceph-mon[32599]: pgmap v1628: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:19:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:56 smithi167 ceph-mon[37140]: pgmap v1628: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:19:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:19:58 smithi138 ceph-mon[36954]: pgmap v1629: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:19:58.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:19:58 smithi100 ceph-mon[32599]: pgmap v1629: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:19:59.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:19:58 smithi167 ceph-mon[37140]: pgmap v1629: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:20:00.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:20:00.122+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:20:00.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:20:00.122+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:20:00.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:20:00.122+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:20:00.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:20:00.122+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:20:00.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:20:00.122+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:20:00.762 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:00 smithi100 ceph-mon[32599]: pgmap v1630: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:20:00.762 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:00 smithi100 ceph-mon[32599]: overall HEALTH_OK 2024-07-26T14:20:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:00 smithi138 ceph-mon[36954]: pgmap v1630: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:20:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:00 smithi138 ceph-mon[36954]: overall HEALTH_OK 2024-07-26T14:20:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:00 smithi167 ceph-mon[37140]: pgmap v1630: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:20:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:00 smithi167 ceph-mon[37140]: overall HEALTH_OK 2024-07-26T14:20:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:02 smithi138 ceph-mon[36954]: pgmap v1631: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:20:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:02 smithi100 ceph-mon[32599]: pgmap v1631: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:20:03.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:02 smithi167 ceph-mon[37140]: pgmap v1631: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:20:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:04 smithi138 ceph-mon[36954]: pgmap v1632: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:20:04.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:04 smithi100 ceph-mon[32599]: pgmap v1632: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:20:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:04 smithi167 ceph-mon[37140]: pgmap v1632: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:20:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:06 smithi138 ceph-mon[36954]: pgmap v1633: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:20:06.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:06 smithi100 ceph-mon[32599]: pgmap v1633: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:20:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:06 smithi167 ceph-mon[37140]: pgmap v1633: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:20:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:08 smithi138 ceph-mon[36954]: pgmap v1634: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:20:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:08 smithi138 ceph-mon[36954]: osdmap e152: 12 total, 12 up, 12 in 2024-07-26T14:20:08.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:08 smithi100 ceph-mon[32599]: pgmap v1634: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:20:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:08 smithi100 ceph-mon[32599]: osdmap e152: 12 total, 12 up, 12 in 2024-07-26T14:20:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:08 smithi167 ceph-mon[37140]: pgmap v1634: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:20:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:08 smithi167 ceph-mon[37140]: osdmap e152: 12 total, 12 up, 12 in 2024-07-26T14:20:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:10 smithi138 ceph-mon[36954]: pgmap v1636: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:20:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:10 smithi100 ceph-mon[32599]: pgmap v1636: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:20:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:10 smithi167 ceph-mon[37140]: pgmap v1636: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:20:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:12 smithi138 ceph-mon[36954]: pgmap v1637: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:20:12.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:12 smithi100 ceph-mon[32599]: pgmap v1637: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:20:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:12 smithi167 ceph-mon[37140]: pgmap v1637: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:20:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:14 smithi138 ceph-mon[36954]: pgmap v1638: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:20:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:14 smithi138 ceph-mon[36954]: osdmap e153: 12 total, 12 up, 12 in 2024-07-26T14:20:14.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:14 smithi100 ceph-mon[32599]: pgmap v1638: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:20:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:14 smithi100 ceph-mon[32599]: osdmap e153: 12 total, 12 up, 12 in 2024-07-26T14:20:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:14 smithi167 ceph-mon[37140]: pgmap v1638: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:20:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:14 smithi167 ceph-mon[37140]: osdmap e153: 12 total, 12 up, 12 in 2024-07-26T14:20:15.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:20:15.127+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:20:15.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:20:15.127+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:20:15.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:20:15.127+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:20:15.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:20:15.127+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:20:15.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:20:15.127+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:20:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:16 smithi138 ceph-mon[36954]: pgmap v1640: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:20:16.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:16 smithi100 ceph-mon[32599]: pgmap v1640: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:20:17.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:16 smithi167 ceph-mon[37140]: pgmap v1640: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:20:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:18 smithi138 ceph-mon[36954]: pgmap v1641: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 214 B/s wr, 6 op/s 2024-07-26T14:20:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:18 smithi100 ceph-mon[32599]: pgmap v1641: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 214 B/s wr, 6 op/s 2024-07-26T14:20:19.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:18 smithi167 ceph-mon[37140]: pgmap v1641: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 214 B/s wr, 6 op/s 2024-07-26T14:20:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:20 smithi138 ceph-mon[36954]: pgmap v1642: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:20:20.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:20 smithi100 ceph-mon[32599]: pgmap v1642: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:20:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:20 smithi167 ceph-mon[37140]: pgmap v1642: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:20:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:22 smithi138 ceph-mon[36954]: pgmap v1643: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:20:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:22 smithi100 ceph-mon[32599]: pgmap v1643: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:20:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:22 smithi167 ceph-mon[37140]: pgmap v1643: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:20:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:24 smithi138 ceph-mon[36954]: pgmap v1644: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:20:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:24 smithi100 ceph-mon[32599]: pgmap v1644: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:20:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:24 smithi167 ceph-mon[37140]: pgmap v1644: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:20:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:26 smithi138 ceph-mon[36954]: pgmap v1645: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:20:26.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:26 smithi100 ceph-mon[32599]: pgmap v1645: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:20:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:26 smithi167 ceph-mon[37140]: pgmap v1645: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:20:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:28 smithi138 ceph-mon[36954]: pgmap v1646: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:20:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:28 smithi100 ceph-mon[32599]: pgmap v1646: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:20:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:28 smithi167 ceph-mon[37140]: pgmap v1646: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:20:30.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:20:30.143+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:20:30.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:20:30.144+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:20:30.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:20:30.144+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:20:30.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:20:30.144+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:20:30.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:20:30.144+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:20:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:30 smithi138 ceph-mon[36954]: pgmap v1647: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:20:30.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:30 smithi100 ceph-mon[32599]: pgmap v1647: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:20:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:30 smithi167 ceph-mon[37140]: pgmap v1647: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:20:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:32 smithi100 ceph-mon[32599]: pgmap v1648: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:20:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:32 smithi167 ceph-mon[37140]: pgmap v1648: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:20:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:32 smithi138 ceph-mon[36954]: pgmap v1648: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:20:34.992 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:34 smithi138 ceph-mon[36954]: pgmap v1649: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:20:34.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:34 smithi100 ceph-mon[32599]: pgmap v1649: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:20:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:34 smithi167 ceph-mon[37140]: pgmap v1649: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:20:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:36 smithi100 ceph-mon[32599]: pgmap v1650: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:20:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:20:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:20:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:20:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:20:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:20:36.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:20:36.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:20:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:36 smithi167 ceph-mon[37140]: pgmap v1650: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:20:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:20:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:20:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:20:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:20:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:20:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:20:37.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:20:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:36 smithi138 ceph-mon[36954]: pgmap v1650: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:20:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:20:37.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:20:37.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:20:37.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:20:37.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:20:37.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:20:37.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:20:37.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:20:37.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:20:37.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:20:38.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:20:38.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:20:38.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:20:38.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:20:38.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:20:38.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:20:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:38 smithi138 ceph-mon[36954]: pgmap v1651: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:20:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:38 smithi100 ceph-mon[32599]: pgmap v1651: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:20:39.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:38 smithi167 ceph-mon[37140]: pgmap v1651: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:20:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:40 smithi138 ceph-mon[36954]: pgmap v1652: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:20:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:40 smithi100 ceph-mon[32599]: pgmap v1652: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:20:41.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:40 smithi167 ceph-mon[37140]: pgmap v1652: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:20:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:42 smithi138 ceph-mon[36954]: pgmap v1653: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:20:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:42 smithi100 ceph-mon[32599]: pgmap v1653: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:20:43.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:42 smithi167 ceph-mon[37140]: pgmap v1653: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:20:45.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:20:45.159+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:20:45.162 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:44 smithi100 ceph-mon[32599]: pgmap v1654: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:20:45.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:20:45.160+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:20:45.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:20:45.160+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:20:45.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:20:45.160+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:20:45.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:20:45.160+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:20:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:44 smithi138 ceph-mon[36954]: pgmap v1654: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:20:45.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:44 smithi167 ceph-mon[37140]: pgmap v1654: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:20:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:46 smithi138 ceph-mon[36954]: pgmap v1655: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:20:47.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:46 smithi100 ceph-mon[32599]: pgmap v1655: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:20:47.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:46 smithi167 ceph-mon[37140]: pgmap v1655: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:20:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:48 smithi138 ceph-mon[36954]: pgmap v1656: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:20:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:48 smithi100 ceph-mon[32599]: pgmap v1656: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:20:49.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:48 smithi167 ceph-mon[37140]: pgmap v1656: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:20:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:50 smithi138 ceph-mon[36954]: pgmap v1657: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:20:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:50 smithi100 ceph-mon[32599]: pgmap v1657: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:20:51.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:50 smithi167 ceph-mon[37140]: pgmap v1657: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:20:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:52 smithi138 ceph-mon[36954]: pgmap v1658: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:20:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:52 smithi100 ceph-mon[32599]: pgmap v1658: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:20:53.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:52 smithi167 ceph-mon[37140]: pgmap v1658: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:20:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:54 smithi138 ceph-mon[36954]: pgmap v1659: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:20:55.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:54 smithi100 ceph-mon[32599]: pgmap v1659: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:20:55.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:54 smithi167 ceph-mon[37140]: pgmap v1659: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:20:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:56 smithi138 ceph-mon[36954]: pgmap v1660: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:20:57.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:56 smithi100 ceph-mon[32599]: pgmap v1660: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:20:57.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:56 smithi167 ceph-mon[37140]: pgmap v1660: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:20:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:20:58 smithi138 ceph-mon[36954]: pgmap v1661: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:20:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:20:58 smithi100 ceph-mon[32599]: pgmap v1661: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:20:59.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:20:58 smithi167 ceph-mon[37140]: pgmap v1661: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:21:00.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:21:00.175+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:21:00.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:21:00.176+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:21:00.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:21:00.176+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:21:00.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:21:00.176+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:21:00.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:21:00.176+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:21:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:21:00 smithi138 ceph-mon[36954]: pgmap v1662: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:21:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:21:00 smithi100 ceph-mon[32599]: pgmap v1662: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:21:01.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:21:00 smithi167 ceph-mon[37140]: pgmap v1662: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:21:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:21:02 smithi138 ceph-mon[36954]: pgmap v1663: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:21:03.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:21:02 smithi100 ceph-mon[32599]: pgmap v1663: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:21:03.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:21:02 smithi167 ceph-mon[37140]: pgmap v1663: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:21:03.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:21:03.292+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c003690 con 0x7fe5a81087a0 2024-07-26T14:21:03.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:21:03.292+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T14:21:03.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:21:03.292+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 19 ==== auth_reply(proto 2 0 (0) Success) ==== 783+0+0 (secure 0 0 0) 0x7fe57c003690 con 0x7fe5a81087a0 2024-07-26T14:21:03.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:21:03.292+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c006610 con 0x7fe5a81087a0 2024-07-26T14:21:03.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:21:03.292+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 19 ==== auth_reply(proto 2 0 (0) Success) ==== 612+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T14:21:03.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:21:03.292+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T14:21:03.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:21:03.292+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 20 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c006610 con 0x7fe5a81087a0 2024-07-26T14:21:03.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:21:03.292+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 20 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T14:21:05.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:21:04 smithi138 ceph-mon[36954]: pgmap v1664: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:21:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:21:04 smithi100 ceph-mon[32599]: pgmap v1664: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:21:05.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:21:04 smithi167 ceph-mon[37140]: pgmap v1664: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:21:06.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:21:05 smithi100 ceph-mon[32599]: pgmap v1665: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:21:06.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:21:05 smithi167 ceph-mon[37140]: pgmap v1665: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:21:06.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:21:05 smithi138 ceph-mon[36954]: pgmap v1665: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:21:08.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:21:07 smithi100 ceph-mon[32599]: osdmap e154: 12 total, 12 up, 12 in 2024-07-26T14:21:08.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:21:07 smithi100 ceph-mon[32599]: pgmap v1667: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:21:08.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:21:07 smithi167 ceph-mon[37140]: osdmap e154: 12 total, 12 up, 12 in 2024-07-26T14:21:08.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:21:07 smithi167 ceph-mon[37140]: pgmap v1667: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:21:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:21:07 smithi138 ceph-mon[36954]: osdmap e154: 12 total, 12 up, 12 in 2024-07-26T14:21:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:21:07 smithi138 ceph-mon[36954]: pgmap v1667: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:21:10.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:21:10 smithi167 ceph-mon[37140]: pgmap v1668: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:21:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:21:10 smithi138 ceph-mon[36954]: pgmap v1668: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:21:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:21:10 smithi100 ceph-mon[32599]: pgmap v1668: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:21:12.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:21:12 smithi167 ceph-mon[37140]: pgmap v1669: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:21:12.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:21:12 smithi138 ceph-mon[36954]: pgmap v1669: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:21:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:21:12 smithi100 ceph-mon[32599]: pgmap v1669: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:21:14.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:21:14 smithi167 ceph-mon[37140]: pgmap v1670: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:21:14.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:21:14 smithi167 ceph-mon[37140]: osdmap e155: 12 total, 12 up, 12 in 2024-07-26T14:21:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:21:14 smithi138 ceph-mon[36954]: pgmap v1670: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:21:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:21:14 smithi138 ceph-mon[36954]: osdmap e155: 12 total, 12 up, 12 in 2024-07-26T14:21:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:21:14 smithi100 ceph-mon[32599]: pgmap v1670: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:21:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:21:14 smithi100 ceph-mon[32599]: osdmap e155: 12 total, 12 up, 12 in 2024-07-26T14:21:15.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:21:15.189+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:21:15.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:21:15.190+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:21:15.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:21:15.190+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:21:15.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:21:15.190+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:21:15.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:21:15.190+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:21:16.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:21:16 smithi167 ceph-mon[37140]: pgmap v1672: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 248 B/s wr, 6 op/s 2024-07-26T14:21:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:21:16 smithi138 ceph-mon[36954]: pgmap v1672: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 248 B/s wr, 6 op/s 2024-07-26T14:21:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:21:16 smithi100 ceph-mon[32599]: pgmap v1672: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 248 B/s wr, 6 op/s 2024-07-26T14:21:18.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:21:18 smithi167 ceph-mon[37140]: pgmap v1673: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:21:18.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:21:18 smithi138 ceph-mon[36954]: pgmap v1673: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:21:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:21:18 smithi100 ceph-mon[32599]: pgmap v1673: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:21:20.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:21:20 smithi167 ceph-mon[37140]: pgmap v1674: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:21:20.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:21:20 smithi138 ceph-mon[36954]: pgmap v1674: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:21:20.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:21:20 smithi100 ceph-mon[32599]: pgmap v1674: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:21:22.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:21:22 smithi138 ceph-mon[36954]: pgmap v1675: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:21:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:21:22 smithi100 ceph-mon[32599]: pgmap v1675: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:21:22.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:21:22 smithi167 ceph-mon[37140]: pgmap v1675: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:21:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:21:24 smithi138 ceph-mon[36954]: pgmap v1676: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:21:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:21:24 smithi100 ceph-mon[32599]: pgmap v1676: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:21:24.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:21:24 smithi167 ceph-mon[37140]: pgmap v1676: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:21:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:21:26 smithi138 ceph-mon[36954]: pgmap v1677: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:21:26.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:21:26 smithi100 ceph-mon[32599]: pgmap v1677: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:21:26.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:21:26 smithi167 ceph-mon[37140]: pgmap v1677: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:21:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:21:28 smithi138 ceph-mon[36954]: pgmap v1678: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:21:28.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:21:28 smithi100 ceph-mon[32599]: pgmap v1678: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:21:28.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:21:28 smithi167 ceph-mon[37140]: pgmap v1678: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:21:30.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:21:30.194+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:21:30.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:21:30.195+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:21:30.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:21:30.195+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:21:30.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:21:30.195+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:21:30.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:21:30.195+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:21:30.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:21:30 smithi100 ceph-mon[32599]: pgmap v1679: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:21:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:21:30 smithi138 ceph-mon[36954]: pgmap v1679: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:21:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:21:30 smithi167 ceph-mon[37140]: pgmap v1679: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:21:32.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:21:32 smithi138 ceph-mon[36954]: pgmap v1680: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:21:32.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:21:32 smithi100 ceph-mon[32599]: pgmap v1680: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:21:32.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:21:32 smithi167 ceph-mon[37140]: pgmap v1680: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:21:34.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:21:34 smithi138 ceph-mon[36954]: pgmap v1681: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:21:34.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:21:34 smithi100 ceph-mon[32599]: pgmap v1681: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:21:34.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:21:34 smithi167 ceph-mon[37140]: pgmap v1681: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:21:36.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:21:36 smithi138 ceph-mon[36954]: pgmap v1682: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:21:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:21:36 smithi100 ceph-mon[32599]: pgmap v1682: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:21:36.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:21:36 smithi167 ceph-mon[37140]: pgmap v1682: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:21:37.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:21:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:21:37.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:21:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:21:37.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:21:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:21:38.674 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:21:38 smithi167 ceph-mon[37140]: pgmap v1683: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:21:38.674 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:21:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:21:38.674 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:21:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:21:38.674 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:21:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:21:38.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:21:38 smithi138 ceph-mon[36954]: pgmap v1683: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:21:38.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:21:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:21:38.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:21:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:21:38.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:21:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:21:38.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:21:38 smithi100 ceph-mon[32599]: pgmap v1683: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:21:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:21:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:21:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:21:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:21:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:21:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:21:40.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:21:40 smithi138 ceph-mon[36954]: pgmap v1684: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:21:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:21:40 smithi100 ceph-mon[32599]: pgmap v1684: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:21:40.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:21:40 smithi167 ceph-mon[37140]: pgmap v1684: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:21:42.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:21:42 smithi138 ceph-mon[36954]: pgmap v1685: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:21:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:21:42 smithi100 ceph-mon[32599]: pgmap v1685: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:21:42.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:21:42 smithi167 ceph-mon[37140]: pgmap v1685: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:21:44.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:21:44 smithi138 ceph-mon[36954]: pgmap v1686: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:21:44.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:21:44 smithi100 ceph-mon[32599]: pgmap v1686: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:21:44.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:21:44 smithi167 ceph-mon[37140]: pgmap v1686: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:21:45.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:21:45.206+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:21:45.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:21:45.207+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:21:45.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:21:45.207+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:21:45.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:21:45.207+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:21:45.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:21:45.207+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:21:46.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:21:46 smithi138 ceph-mon[36954]: pgmap v1687: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:21:46.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:21:46 smithi100 ceph-mon[32599]: pgmap v1687: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:21:46.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:21:46 smithi167 ceph-mon[37140]: pgmap v1687: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:21:48.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:21:48 smithi138 ceph-mon[36954]: pgmap v1688: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:21:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:21:48 smithi100 ceph-mon[32599]: pgmap v1688: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:21:48.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:21:48 smithi167 ceph-mon[37140]: pgmap v1688: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:21:50.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:21:50 smithi138 ceph-mon[36954]: pgmap v1689: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:21:50.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:21:50 smithi100 ceph-mon[32599]: pgmap v1689: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:21:50.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:21:50 smithi167 ceph-mon[37140]: pgmap v1689: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:21:52.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:21:52 smithi138 ceph-mon[36954]: pgmap v1690: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:21:52.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:21:52 smithi100 ceph-mon[32599]: pgmap v1690: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:21:52.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:21:52 smithi167 ceph-mon[37140]: pgmap v1690: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:21:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:21:54 smithi138 ceph-mon[36954]: pgmap v1691: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:21:54.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:21:54 smithi100 ceph-mon[32599]: pgmap v1691: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:21:54.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:21:54 smithi167 ceph-mon[37140]: pgmap v1691: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:21:56.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:21:56 smithi138 ceph-mon[36954]: pgmap v1692: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:21:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:21:56 smithi100 ceph-mon[32599]: pgmap v1692: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:21:56.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:21:56 smithi167 ceph-mon[37140]: pgmap v1692: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:21:58.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:21:58 smithi138 ceph-mon[36954]: pgmap v1693: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:21:58.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:21:58 smithi100 ceph-mon[32599]: pgmap v1693: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:21:58.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:21:58 smithi167 ceph-mon[37140]: pgmap v1693: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:22:00.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:22:00.223+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:22:00.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:22:00.223+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:22:00.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:22:00.223+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:22:00.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:22:00.223+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:22:00.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:22:00.223+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:22:00.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:22:00 smithi100 ceph-mon[32599]: pgmap v1694: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:22:00.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:22:00 smithi138 ceph-mon[36954]: pgmap v1694: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:22:00.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:22:00 smithi167 ceph-mon[37140]: pgmap v1694: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:22:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:22:02 smithi138 ceph-mon[36954]: pgmap v1695: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:22:02.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:22:02 smithi100 ceph-mon[32599]: pgmap v1695: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:22:02.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:22:02 smithi167 ceph-mon[37140]: pgmap v1695: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:22:04.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:22:04 smithi138 ceph-mon[36954]: pgmap v1696: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:22:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:22:04 smithi100 ceph-mon[32599]: pgmap v1696: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:22:04.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:22:04 smithi167 ceph-mon[37140]: pgmap v1696: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:22:06.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:22:06 smithi138 ceph-mon[36954]: pgmap v1697: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:22:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:22:06 smithi100 ceph-mon[32599]: pgmap v1697: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:22:06.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:22:06 smithi167 ceph-mon[37140]: pgmap v1697: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:22:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:22:08 smithi138 ceph-mon[36954]: pgmap v1698: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:22:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:22:08 smithi138 ceph-mon[36954]: osdmap e156: 12 total, 12 up, 12 in 2024-07-26T14:22:08.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:22:08 smithi100 ceph-mon[32599]: pgmap v1698: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:22:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:22:08 smithi100 ceph-mon[32599]: osdmap e156: 12 total, 12 up, 12 in 2024-07-26T14:22:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:22:08 smithi167 ceph-mon[37140]: pgmap v1698: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:22:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:22:08 smithi167 ceph-mon[37140]: osdmap e156: 12 total, 12 up, 12 in 2024-07-26T14:22:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:22:10 smithi138 ceph-mon[36954]: pgmap v1700: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:22:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:22:10 smithi100 ceph-mon[32599]: pgmap v1700: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:22:10.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:22:10 smithi167 ceph-mon[37140]: pgmap v1700: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:22:12.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:22:12 smithi138 ceph-mon[36954]: pgmap v1701: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:22:12.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:22:12 smithi100 ceph-mon[32599]: pgmap v1701: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:22:12.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:22:12 smithi167 ceph-mon[37140]: pgmap v1701: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:22:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:22:14 smithi138 ceph-mon[36954]: pgmap v1702: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:22:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:22:14 smithi138 ceph-mon[36954]: osdmap e157: 12 total, 12 up, 12 in 2024-07-26T14:22:14.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:22:14 smithi100 ceph-mon[32599]: pgmap v1702: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:22:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:22:14 smithi100 ceph-mon[32599]: osdmap e157: 12 total, 12 up, 12 in 2024-07-26T14:22:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:22:14 smithi167 ceph-mon[37140]: pgmap v1702: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:22:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:22:14 smithi167 ceph-mon[37140]: osdmap e157: 12 total, 12 up, 12 in 2024-07-26T14:22:15.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:22:15.237+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:22:15.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:22:15.237+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:22:15.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:22:15.237+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:22:15.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:22:15.237+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:22:15.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:22:15.237+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:22:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:22:16 smithi138 ceph-mon[36954]: pgmap v1704: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:22:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:22:16 smithi100 ceph-mon[32599]: pgmap v1704: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:22:16.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:22:16 smithi167 ceph-mon[37140]: pgmap v1704: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:22:18.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:22:18 smithi138 ceph-mon[36954]: pgmap v1705: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T14:22:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:22:18 smithi100 ceph-mon[32599]: pgmap v1705: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T14:22:18.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:22:18 smithi167 ceph-mon[37140]: pgmap v1705: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T14:22:20.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:22:20 smithi138 ceph-mon[36954]: pgmap v1706: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:22:20.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:22:20 smithi100 ceph-mon[32599]: pgmap v1706: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:22:20.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:22:20 smithi167 ceph-mon[37140]: pgmap v1706: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:22:22.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:22:22 smithi138 ceph-mon[36954]: pgmap v1707: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:22:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:22:22 smithi100 ceph-mon[32599]: pgmap v1707: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:22:22.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:22:22 smithi167 ceph-mon[37140]: pgmap v1707: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:22:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:22:24 smithi138 ceph-mon[36954]: pgmap v1708: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:22:24.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:22:24 smithi100 ceph-mon[32599]: pgmap v1708: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:22:24.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:22:24 smithi167 ceph-mon[37140]: pgmap v1708: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:22:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:22:26 smithi138 ceph-mon[36954]: pgmap v1709: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:22:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:22:26 smithi100 ceph-mon[32599]: pgmap v1709: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:22:26.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:22:26 smithi167 ceph-mon[37140]: pgmap v1709: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:22:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:22:28 smithi138 ceph-mon[36954]: pgmap v1710: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:22:28.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:22:28 smithi100 ceph-mon[32599]: pgmap v1710: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:22:28.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:22:28 smithi167 ceph-mon[37140]: pgmap v1710: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:22:30.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:22:30.253+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:22:30.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:22:30.253+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:22:30.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:22:30.253+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:22:30.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:22:30.253+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:22:30.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:22:30.253+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:22:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:22:30 smithi138 ceph-mon[36954]: pgmap v1711: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:22:30.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:22:30 smithi100 ceph-mon[32599]: pgmap v1711: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:22:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:22:30 smithi167 ceph-mon[37140]: pgmap v1711: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:22:32.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:22:32 smithi138 ceph-mon[36954]: pgmap v1712: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:22:32.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:22:32 smithi100 ceph-mon[32599]: pgmap v1712: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:22:32.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:22:32 smithi167 ceph-mon[37140]: pgmap v1712: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:22:34.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:22:34 smithi138 ceph-mon[36954]: pgmap v1713: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:22:34.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:22:34 smithi100 ceph-mon[32599]: pgmap v1713: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:22:34.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:22:34 smithi167 ceph-mon[37140]: pgmap v1713: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:22:36.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:22:36 smithi138 ceph-mon[36954]: pgmap v1714: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:22:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:22:36 smithi100 ceph-mon[32599]: pgmap v1714: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:22:36.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:22:36 smithi167 ceph-mon[37140]: pgmap v1714: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:22:38.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:22:38 smithi138 ceph-mon[36954]: pgmap v1715: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:22:38.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:22:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:22:38.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:22:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:22:38.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:22:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:22:38.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:22:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:22:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:22:38 smithi100 ceph-mon[32599]: pgmap v1715: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:22:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:22:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:22:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:22:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:22:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:22:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:22:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:22:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:22:38.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:22:38 smithi167 ceph-mon[37140]: pgmap v1715: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:22:38.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:22:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:22:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:22:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:22:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:22:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:22:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:22:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:22:40.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:22:40 smithi138 ceph-mon[36954]: pgmap v1716: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:22:40.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:22:40 smithi100 ceph-mon[32599]: pgmap v1716: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:22:40.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:22:40 smithi167 ceph-mon[37140]: pgmap v1716: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:22:42.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:22:42 smithi138 ceph-mon[36954]: pgmap v1717: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:22:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:22:42 smithi100 ceph-mon[32599]: pgmap v1717: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:22:42.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:22:42 smithi167 ceph-mon[37140]: pgmap v1717: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:22:44.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:22:44 smithi138 ceph-mon[36954]: pgmap v1718: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:22:44.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:22:44 smithi100 ceph-mon[32599]: pgmap v1718: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:22:44.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:22:44 smithi167 ceph-mon[37140]: pgmap v1718: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:22:45.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:22:45.266+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:22:45.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:22:45.266+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:22:45.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:22:45.266+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:22:45.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:22:45.266+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:22:45.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:22:45.266+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:22:46.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:22:46 smithi138 ceph-mon[36954]: pgmap v1719: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:22:46.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:22:46 smithi100 ceph-mon[32599]: pgmap v1719: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:22:46.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:22:46 smithi167 ceph-mon[37140]: pgmap v1719: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:22:48.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:22:48 smithi138 ceph-mon[36954]: pgmap v1720: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:22:48.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:22:48 smithi100 ceph-mon[32599]: pgmap v1720: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:22:48.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:22:48 smithi167 ceph-mon[37140]: pgmap v1720: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:22:50.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:22:50 smithi138 ceph-mon[36954]: pgmap v1721: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:22:50.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:22:50 smithi100 ceph-mon[32599]: pgmap v1721: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:22:50.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:22:50 smithi167 ceph-mon[37140]: pgmap v1721: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:22:52.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:22:52 smithi138 ceph-mon[36954]: pgmap v1722: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:22:52.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:22:52 smithi100 ceph-mon[32599]: pgmap v1722: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:22:52.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:22:52 smithi167 ceph-mon[37140]: pgmap v1722: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:22:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:22:54 smithi138 ceph-mon[36954]: pgmap v1723: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:22:54.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:22:54 smithi100 ceph-mon[32599]: pgmap v1723: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:22:54.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:22:54 smithi167 ceph-mon[37140]: pgmap v1723: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:22:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:22:56 smithi100 ceph-mon[32599]: pgmap v1724: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:22:56.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:22:56 smithi167 ceph-mon[37140]: pgmap v1724: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:22:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:22:56 smithi138 ceph-mon[36954]: pgmap v1724: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:22:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:22:58 smithi100 ceph-mon[32599]: pgmap v1725: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:22:58.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:22:58 smithi167 ceph-mon[37140]: pgmap v1725: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:22:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:22:58 smithi138 ceph-mon[36954]: pgmap v1725: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:23:00.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:23:00.269+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:23:00.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:23:00.269+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:23:00.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:23:00.269+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:23:00.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:23:00.269+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:23:00.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:23:00.269+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:23:00.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:23:00 smithi100 ceph-mon[32599]: pgmap v1726: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:23:00.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:23:00 smithi167 ceph-mon[37140]: pgmap v1726: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:23:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:23:00 smithi138 ceph-mon[36954]: pgmap v1726: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:23:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:23:02 smithi100 ceph-mon[32599]: pgmap v1727: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:23:02.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:23:02 smithi167 ceph-mon[37140]: pgmap v1727: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:23:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:23:02 smithi138 ceph-mon[36954]: pgmap v1727: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:23:04.742 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:23:04 smithi138 ceph-mon[36954]: pgmap v1728: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:23:04.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:23:04 smithi100 ceph-mon[32599]: pgmap v1728: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:23:04.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:23:04 smithi167 ceph-mon[37140]: pgmap v1728: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:23:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:23:06 smithi100 ceph-mon[32599]: pgmap v1729: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:23:06.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:23:06 smithi167 ceph-mon[37140]: pgmap v1729: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:23:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:23:06 smithi138 ceph-mon[36954]: pgmap v1729: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:23:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:23:08 smithi100 ceph-mon[32599]: pgmap v1730: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:23:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:23:08 smithi100 ceph-mon[32599]: osdmap e158: 12 total, 12 up, 12 in 2024-07-26T14:23:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:23:08 smithi167 ceph-mon[37140]: pgmap v1730: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:23:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:23:08 smithi167 ceph-mon[37140]: osdmap e158: 12 total, 12 up, 12 in 2024-07-26T14:23:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:23:08 smithi138 ceph-mon[36954]: pgmap v1730: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:23:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:23:08 smithi138 ceph-mon[36954]: osdmap e158: 12 total, 12 up, 12 in 2024-07-26T14:23:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:23:10 smithi100 ceph-mon[32599]: pgmap v1732: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:23:10.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:23:10 smithi167 ceph-mon[37140]: pgmap v1732: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:23:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:23:10 smithi138 ceph-mon[36954]: pgmap v1732: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:23:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:23:12 smithi100 ceph-mon[32599]: pgmap v1733: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:23:12.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:23:12 smithi167 ceph-mon[37140]: pgmap v1733: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:23:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:23:12 smithi138 ceph-mon[36954]: pgmap v1733: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:23:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:23:14 smithi100 ceph-mon[32599]: pgmap v1734: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:23:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:23:14 smithi100 ceph-mon[32599]: osdmap e159: 12 total, 12 up, 12 in 2024-07-26T14:23:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:23:14 smithi167 ceph-mon[37140]: pgmap v1734: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:23:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:23:14 smithi167 ceph-mon[37140]: osdmap e159: 12 total, 12 up, 12 in 2024-07-26T14:23:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:23:14 smithi138 ceph-mon[36954]: pgmap v1734: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:23:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:23:14 smithi138 ceph-mon[36954]: osdmap e159: 12 total, 12 up, 12 in 2024-07-26T14:23:15.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:23:15.280+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:23:15.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:23:15.280+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:23:15.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:23:15.280+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:23:15.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:23:15.280+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:23:15.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:23:15.280+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:23:16.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:23:16 smithi100 ceph-mon[32599]: pgmap v1736: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:23:16.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:23:16 smithi167 ceph-mon[37140]: pgmap v1736: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:23:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:23:16 smithi138 ceph-mon[36954]: pgmap v1736: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:23:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:23:18 smithi100 ceph-mon[32599]: pgmap v1737: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 209 B/s wr, 6 op/s 2024-07-26T14:23:18.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:23:18 smithi167 ceph-mon[37140]: pgmap v1737: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 209 B/s wr, 6 op/s 2024-07-26T14:23:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:23:18 smithi138 ceph-mon[36954]: pgmap v1737: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 209 B/s wr, 6 op/s 2024-07-26T14:23:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:23:20 smithi100 ceph-mon[32599]: pgmap v1738: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:23:20.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:23:20 smithi167 ceph-mon[37140]: pgmap v1738: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:23:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:23:20 smithi138 ceph-mon[36954]: pgmap v1738: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:23:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:23:22 smithi100 ceph-mon[32599]: pgmap v1739: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:23:22.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:23:22 smithi167 ceph-mon[37140]: pgmap v1739: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:23:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:23:22 smithi138 ceph-mon[36954]: pgmap v1739: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:23:24.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:23:24 smithi167 ceph-mon[37140]: pgmap v1740: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:23:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:23:24 smithi138 ceph-mon[36954]: pgmap v1740: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:23:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:23:24 smithi100 ceph-mon[32599]: pgmap v1740: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:23:26.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:23:26 smithi167 ceph-mon[37140]: pgmap v1741: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:23:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:23:26 smithi138 ceph-mon[36954]: pgmap v1741: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:23:26.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:23:26 smithi100 ceph-mon[32599]: pgmap v1741: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:23:28.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:23:28 smithi167 ceph-mon[37140]: pgmap v1742: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:23:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:23:28 smithi138 ceph-mon[36954]: pgmap v1742: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:23:28.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:23:28 smithi100 ceph-mon[32599]: pgmap v1742: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:23:30.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:23:30.296+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:23:30.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:23:30.296+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:23:30.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:23:30.296+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:23:30.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:23:30.296+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:23:30.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:23:30.297+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:23:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:23:30 smithi100 ceph-mon[32599]: pgmap v1743: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:23:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:23:30 smithi167 ceph-mon[37140]: pgmap v1743: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:23:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:23:30 smithi138 ceph-mon[36954]: pgmap v1743: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:23:32.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:23:32 smithi167 ceph-mon[37140]: pgmap v1744: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:23:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:23:32 smithi138 ceph-mon[36954]: pgmap v1744: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:23:32.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:23:32 smithi100 ceph-mon[32599]: pgmap v1744: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:23:34.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:23:34 smithi167 ceph-mon[37140]: pgmap v1745: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:23:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:23:34 smithi138 ceph-mon[36954]: pgmap v1745: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:23:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:23:34 smithi100 ceph-mon[32599]: pgmap v1745: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:23:36.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:23:36 smithi167 ceph-mon[37140]: pgmap v1746: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:23:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:23:36 smithi138 ceph-mon[36954]: pgmap v1746: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:23:36.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:23:36 smithi100 ceph-mon[32599]: pgmap v1746: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:23:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:23:38 smithi138 ceph-mon[36954]: pgmap v1747: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:23:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:23:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:23:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:23:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:23:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:23:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:23:38.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:23:38 smithi100 ceph-mon[32599]: pgmap v1747: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:23:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:23:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:23:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:23:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:23:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:23:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:23:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:23:38 smithi167 ceph-mon[37140]: pgmap v1747: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:23:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:23:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:23:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:23:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:23:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:23:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:23:39.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:23:39 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:23:39.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:23:39 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:23:40.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:23:39 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:23:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:23:40 smithi138 ceph-mon[36954]: pgmap v1748: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:23:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:23:40 smithi100 ceph-mon[32599]: pgmap v1748: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:23:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:23:40 smithi167 ceph-mon[37140]: pgmap v1748: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:23:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:23:42 smithi138 ceph-mon[36954]: pgmap v1749: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:23:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:23:42 smithi100 ceph-mon[32599]: pgmap v1749: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:23:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:23:42 smithi167 ceph-mon[37140]: pgmap v1749: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:23:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:23:44 smithi138 ceph-mon[36954]: pgmap v1750: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:23:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:23:44 smithi100 ceph-mon[32599]: pgmap v1750: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:23:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:23:44 smithi167 ceph-mon[37140]: pgmap v1750: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:23:45.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:23:45.312+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:23:45.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:23:45.313+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:23:45.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:23:45.313+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:23:45.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:23:45.313+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:23:45.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:23:45.313+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:23:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:23:46 smithi138 ceph-mon[36954]: pgmap v1751: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:23:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:23:46 smithi100 ceph-mon[32599]: pgmap v1751: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:23:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:23:46 smithi167 ceph-mon[37140]: pgmap v1751: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:23:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:23:48 smithi138 ceph-mon[36954]: pgmap v1752: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:23:48.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:23:48 smithi100 ceph-mon[32599]: pgmap v1752: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:23:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:23:48 smithi167 ceph-mon[37140]: pgmap v1752: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:23:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:23:50 smithi138 ceph-mon[36954]: pgmap v1753: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:23:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:23:50 smithi100 ceph-mon[32599]: pgmap v1753: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:23:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:23:50 smithi167 ceph-mon[37140]: pgmap v1753: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:23:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:23:52 smithi138 ceph-mon[36954]: pgmap v1754: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:23:52.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:23:52 smithi100 ceph-mon[32599]: pgmap v1754: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:23:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:23:52 smithi167 ceph-mon[37140]: pgmap v1754: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:23:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:23:54 smithi138 ceph-mon[36954]: pgmap v1755: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:23:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:23:54 smithi100 ceph-mon[32599]: pgmap v1755: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:23:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:23:54 smithi167 ceph-mon[37140]: pgmap v1755: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:23:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:23:56 smithi138 ceph-mon[36954]: pgmap v1756: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:23:56.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:23:56 smithi100 ceph-mon[32599]: pgmap v1756: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:23:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:23:56 smithi167 ceph-mon[37140]: pgmap v1756: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:23:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:23:58 smithi138 ceph-mon[36954]: pgmap v1757: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:23:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:23:58 smithi100 ceph-mon[32599]: pgmap v1757: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:23:59.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:23:58 smithi167 ceph-mon[37140]: pgmap v1757: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:24:00.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:24:00.326+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:24:00.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:24:00.326+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:24:00.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:24:00.326+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:24:00.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:24:00.326+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:24:00.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:24:00.326+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:24:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:24:00 smithi138 ceph-mon[36954]: pgmap v1758: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:24:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:24:00 smithi100 ceph-mon[32599]: pgmap v1758: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:24:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:24:00 smithi167 ceph-mon[37140]: pgmap v1758: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:24:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:24:02 smithi138 ceph-mon[36954]: pgmap v1759: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:24:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:24:02 smithi100 ceph-mon[32599]: pgmap v1759: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:24:03.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:24:02 smithi167 ceph-mon[37140]: pgmap v1759: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:24:03.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:24:03.318+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c006610 con 0x7fe5a81087a0 2024-07-26T14:24:03.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:24:03.318+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T14:24:03.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:24:03.318+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 21 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c006610 con 0x7fe5a81087a0 2024-07-26T14:24:03.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:24:03.318+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 21 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T14:24:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:24:04 smithi138 ceph-mon[36954]: pgmap v1760: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:24:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:24:04 smithi100 ceph-mon[32599]: pgmap v1760: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:24:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:24:04 smithi167 ceph-mon[37140]: pgmap v1760: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:24:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:24:06 smithi138 ceph-mon[36954]: pgmap v1761: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:24:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:24:06 smithi100 ceph-mon[32599]: pgmap v1761: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:24:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:24:06 smithi167 ceph-mon[37140]: pgmap v1761: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:24:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:24:08 smithi138 ceph-mon[36954]: pgmap v1762: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:24:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:24:08 smithi138 ceph-mon[36954]: osdmap e160: 12 total, 12 up, 12 in 2024-07-26T14:24:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:24:08 smithi100 ceph-mon[32599]: pgmap v1762: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:24:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:24:08 smithi100 ceph-mon[32599]: osdmap e160: 12 total, 12 up, 12 in 2024-07-26T14:24:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:24:08 smithi167 ceph-mon[37140]: pgmap v1762: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:24:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:24:08 smithi167 ceph-mon[37140]: osdmap e160: 12 total, 12 up, 12 in 2024-07-26T14:24:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:24:10 smithi138 ceph-mon[36954]: pgmap v1764: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:24:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:24:10 smithi100 ceph-mon[32599]: pgmap v1764: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:24:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:24:10 smithi167 ceph-mon[37140]: pgmap v1764: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:24:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:24:12 smithi138 ceph-mon[36954]: pgmap v1765: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:24:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:24:12 smithi100 ceph-mon[32599]: pgmap v1765: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:24:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:24:12 smithi167 ceph-mon[37140]: pgmap v1765: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:24:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:24:14 smithi138 ceph-mon[36954]: pgmap v1766: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:24:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:24:14 smithi138 ceph-mon[36954]: osdmap e161: 12 total, 12 up, 12 in 2024-07-26T14:24:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:24:14 smithi100 ceph-mon[32599]: pgmap v1766: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:24:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:24:14 smithi100 ceph-mon[32599]: osdmap e161: 12 total, 12 up, 12 in 2024-07-26T14:24:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:24:14 smithi167 ceph-mon[37140]: pgmap v1766: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:24:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:24:14 smithi167 ceph-mon[37140]: osdmap e161: 12 total, 12 up, 12 in 2024-07-26T14:24:15.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:24:15.343+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:24:15.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:24:15.344+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:24:15.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:24:15.344+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:24:15.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:24:15.344+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:24:15.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:24:15.344+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:24:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:24:16 smithi138 ceph-mon[36954]: pgmap v1768: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:24:16.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:24:16 smithi100 ceph-mon[32599]: pgmap v1768: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:24:17.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:24:16 smithi167 ceph-mon[37140]: pgmap v1768: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:24:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:24:18 smithi138 ceph-mon[36954]: pgmap v1769: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 211 B/s wr, 6 op/s 2024-07-26T14:24:18.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:24:18 smithi100 ceph-mon[32599]: pgmap v1769: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 211 B/s wr, 6 op/s 2024-07-26T14:24:19.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:24:18 smithi167 ceph-mon[37140]: pgmap v1769: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 211 B/s wr, 6 op/s 2024-07-26T14:24:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:24:20 smithi138 ceph-mon[36954]: pgmap v1770: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:24:20.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:24:20 smithi100 ceph-mon[32599]: pgmap v1770: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:24:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:24:20 smithi167 ceph-mon[37140]: pgmap v1770: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:24:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:24:22 smithi138 ceph-mon[36954]: pgmap v1771: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:24:22.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:24:22 smithi100 ceph-mon[32599]: pgmap v1771: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:24:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:24:22 smithi167 ceph-mon[37140]: pgmap v1771: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:24:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:24:24 smithi138 ceph-mon[36954]: pgmap v1772: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:24:24.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:24:24 smithi100 ceph-mon[32599]: pgmap v1772: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:24:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:24:24 smithi167 ceph-mon[37140]: pgmap v1772: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:24:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:24:26 smithi138 ceph-mon[36954]: pgmap v1773: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:24:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:24:26 smithi100 ceph-mon[32599]: pgmap v1773: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:24:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:24:26 smithi167 ceph-mon[37140]: pgmap v1773: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:24:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:24:28 smithi138 ceph-mon[36954]: pgmap v1774: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:24:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:24:28 smithi100 ceph-mon[32599]: pgmap v1774: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:24:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:24:28 smithi167 ceph-mon[37140]: pgmap v1774: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:24:30.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:24:30.359+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:24:30.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:24:30.360+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:24:30.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:24:30.360+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:24:30.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:24:30.360+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:24:30.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:24:30.360+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:24:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:24:30 smithi138 ceph-mon[36954]: pgmap v1775: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:24:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:24:30 smithi100 ceph-mon[32599]: pgmap v1775: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:24:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:24:30 smithi167 ceph-mon[37140]: pgmap v1775: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:24:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:24:32 smithi138 ceph-mon[36954]: pgmap v1776: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:24:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:24:32 smithi100 ceph-mon[32599]: pgmap v1776: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:24:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:24:32 smithi167 ceph-mon[37140]: pgmap v1776: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:24:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:24:34 smithi138 ceph-mon[36954]: pgmap v1777: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:24:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:24:34 smithi100 ceph-mon[32599]: pgmap v1777: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:24:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:24:34 smithi167 ceph-mon[37140]: pgmap v1777: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:24:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:24:36 smithi138 ceph-mon[36954]: pgmap v1778: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:24:36.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:24:36 smithi100 ceph-mon[32599]: pgmap v1778: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:24:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:24:36 smithi167 ceph-mon[37140]: pgmap v1778: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:24:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:24:38 smithi138 ceph-mon[36954]: pgmap v1779: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:24:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:24:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:24:38.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:24:38 smithi100 ceph-mon[32599]: pgmap v1779: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:24:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:24:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:24:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:24:38 smithi167 ceph-mon[37140]: pgmap v1779: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:24:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:24:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:24:39.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:24:39 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:24:39.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:24:39 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:24:39.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:24:39 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:24:39.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:24:39 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:24:39.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:24:39 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:24:39.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:24:39 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:24:40.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:24:39 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:24:40.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:24:39 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:24:40.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:24:39 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:24:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:24:40 smithi138 ceph-mon[36954]: pgmap v1780: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:24:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:24:40 smithi100 ceph-mon[32599]: pgmap v1780: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:24:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:24:40 smithi167 ceph-mon[37140]: pgmap v1780: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:24:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:24:42 smithi138 ceph-mon[36954]: pgmap v1781: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:24:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:24:42 smithi100 ceph-mon[32599]: pgmap v1781: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:24:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:24:42 smithi167 ceph-mon[37140]: pgmap v1781: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:24:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:24:44 smithi138 ceph-mon[36954]: pgmap v1782: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:24:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:24:44 smithi100 ceph-mon[32599]: pgmap v1782: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:24:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:24:44 smithi167 ceph-mon[37140]: pgmap v1782: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:24:45.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:24:45.375+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:24:45.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:24:45.376+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:24:45.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:24:45.376+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:24:45.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:24:45.376+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:24:45.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:24:45.376+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:24:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:24:46 smithi138 ceph-mon[36954]: pgmap v1783: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:24:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:24:46 smithi100 ceph-mon[32599]: pgmap v1783: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:24:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:24:46 smithi167 ceph-mon[37140]: pgmap v1783: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:24:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:24:48 smithi138 ceph-mon[36954]: pgmap v1784: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:24:48.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:24:48 smithi100 ceph-mon[32599]: pgmap v1784: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:24:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:24:48 smithi167 ceph-mon[37140]: pgmap v1784: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:24:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:24:50 smithi138 ceph-mon[36954]: pgmap v1785: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:24:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:24:50 smithi100 ceph-mon[32599]: pgmap v1785: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:24:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:24:50 smithi167 ceph-mon[37140]: pgmap v1785: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:24:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:24:52 smithi138 ceph-mon[36954]: pgmap v1786: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:24:52.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:24:52 smithi100 ceph-mon[32599]: pgmap v1786: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:24:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:24:52 smithi167 ceph-mon[37140]: pgmap v1786: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:24:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:24:54 smithi138 ceph-mon[36954]: pgmap v1787: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:24:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:24:54 smithi100 ceph-mon[32599]: pgmap v1787: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:24:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:24:54 smithi167 ceph-mon[37140]: pgmap v1787: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:24:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:24:56 smithi138 ceph-mon[36954]: pgmap v1788: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:24:56.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:24:56 smithi100 ceph-mon[32599]: pgmap v1788: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:24:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:24:56 smithi167 ceph-mon[37140]: pgmap v1788: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:24:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:24:58 smithi138 ceph-mon[36954]: pgmap v1789: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:24:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:24:58 smithi100 ceph-mon[32599]: pgmap v1789: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:24:59.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:24:58 smithi167 ceph-mon[37140]: pgmap v1789: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:25:00.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:25:00.392+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:25:00.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:25:00.393+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:25:00.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:25:00.393+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:25:00.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:25:00.393+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:25:00.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:25:00.393+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:25:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:25:00 smithi138 ceph-mon[36954]: pgmap v1790: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:25:00.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:25:00 smithi100 ceph-mon[32599]: pgmap v1790: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:25:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:25:00 smithi167 ceph-mon[37140]: pgmap v1790: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:25:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:25:02 smithi100 ceph-mon[32599]: pgmap v1791: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:25:03.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:25:02 smithi167 ceph-mon[37140]: pgmap v1791: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:25:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:25:02 smithi138 ceph-mon[36954]: pgmap v1791: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:25:04.992 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:25:04 smithi138 ceph-mon[36954]: pgmap v1792: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:25:04.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:25:04 smithi100 ceph-mon[32599]: pgmap v1792: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:25:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:25:04 smithi167 ceph-mon[37140]: pgmap v1792: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:25:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:25:06 smithi100 ceph-mon[32599]: pgmap v1793: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:25:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:25:06 smithi167 ceph-mon[37140]: pgmap v1793: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:25:07.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:25:06 smithi138 ceph-mon[36954]: pgmap v1793: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:25:08.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:25:08 smithi100 ceph-mon[32599]: pgmap v1794: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:25:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:25:08 smithi100 ceph-mon[32599]: osdmap e162: 12 total, 12 up, 12 in 2024-07-26T14:25:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:25:08 smithi167 ceph-mon[37140]: pgmap v1794: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:25:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:25:08 smithi167 ceph-mon[37140]: osdmap e162: 12 total, 12 up, 12 in 2024-07-26T14:25:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:25:08 smithi138 ceph-mon[36954]: pgmap v1794: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:25:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:25:08 smithi138 ceph-mon[36954]: osdmap e162: 12 total, 12 up, 12 in 2024-07-26T14:25:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:25:10 smithi100 ceph-mon[32599]: pgmap v1796: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:25:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:25:10 smithi167 ceph-mon[37140]: pgmap v1796: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:25:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:25:10 smithi138 ceph-mon[36954]: pgmap v1796: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:25:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:25:12 smithi100 ceph-mon[32599]: pgmap v1797: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:25:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:25:12 smithi167 ceph-mon[37140]: pgmap v1797: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:25:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:25:12 smithi138 ceph-mon[36954]: pgmap v1797: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:25:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:25:14 smithi100 ceph-mon[32599]: pgmap v1798: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:25:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:25:14 smithi100 ceph-mon[32599]: osdmap e163: 12 total, 12 up, 12 in 2024-07-26T14:25:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:25:14 smithi167 ceph-mon[37140]: pgmap v1798: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:25:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:25:14 smithi167 ceph-mon[37140]: osdmap e163: 12 total, 12 up, 12 in 2024-07-26T14:25:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:25:14 smithi138 ceph-mon[36954]: pgmap v1798: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:25:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:25:14 smithi138 ceph-mon[36954]: osdmap e163: 12 total, 12 up, 12 in 2024-07-26T14:25:15.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:25:15.408+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:25:15.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:25:15.408+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:25:15.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:25:15.408+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:25:15.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:25:15.408+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:25:15.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:25:15.408+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:25:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:25:16 smithi100 ceph-mon[32599]: pgmap v1800: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:25:17.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:25:16 smithi167 ceph-mon[37140]: pgmap v1800: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:25:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:25:16 smithi138 ceph-mon[36954]: pgmap v1800: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:25:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:25:18 smithi100 ceph-mon[32599]: pgmap v1801: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 213 B/s wr, 6 op/s 2024-07-26T14:25:19.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:25:18 smithi167 ceph-mon[37140]: pgmap v1801: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 213 B/s wr, 6 op/s 2024-07-26T14:25:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:25:18 smithi138 ceph-mon[36954]: pgmap v1801: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 213 B/s wr, 6 op/s 2024-07-26T14:25:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:25:20 smithi100 ceph-mon[32599]: pgmap v1802: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:25:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:25:20 smithi167 ceph-mon[37140]: pgmap v1802: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:25:21.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:25:20 smithi138 ceph-mon[36954]: pgmap v1802: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:25:22.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:25:22 smithi100 ceph-mon[32599]: pgmap v1803: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:25:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:25:22 smithi167 ceph-mon[37140]: pgmap v1803: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:25:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:25:22 smithi138 ceph-mon[36954]: pgmap v1803: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:25:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:25:24 smithi100 ceph-mon[32599]: pgmap v1804: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:25:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:25:24 smithi167 ceph-mon[37140]: pgmap v1804: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:25:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:25:24 smithi138 ceph-mon[36954]: pgmap v1804: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:25:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:25:26 smithi100 ceph-mon[32599]: pgmap v1805: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:25:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:25:26 smithi167 ceph-mon[37140]: pgmap v1805: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:25:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:25:26 smithi138 ceph-mon[36954]: pgmap v1805: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:25:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:25:28 smithi100 ceph-mon[32599]: pgmap v1806: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:25:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:25:28 smithi167 ceph-mon[37140]: pgmap v1806: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:25:29.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:25:28 smithi138 ceph-mon[36954]: pgmap v1806: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:25:30.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:25:30.415+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:25:30.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:25:30.416+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:25:30.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:25:30.416+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:25:30.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:25:30.416+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:25:30.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:25:30.416+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:25:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:25:30 smithi100 ceph-mon[32599]: pgmap v1807: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:25:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:25:30 smithi167 ceph-mon[37140]: pgmap v1807: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:25:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:25:30 smithi138 ceph-mon[36954]: pgmap v1807: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:25:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:25:32 smithi167 ceph-mon[37140]: pgmap v1808: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:25:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:25:32 smithi138 ceph-mon[36954]: pgmap v1808: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:25:33.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:25:32 smithi100 ceph-mon[32599]: pgmap v1808: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:25:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:25:34 smithi167 ceph-mon[37140]: pgmap v1809: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:25:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:25:34 smithi138 ceph-mon[36954]: pgmap v1809: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:25:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:25:34 smithi100 ceph-mon[32599]: pgmap v1809: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:25:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:25:36 smithi167 ceph-mon[37140]: pgmap v1810: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:25:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:25:36 smithi138 ceph-mon[36954]: pgmap v1810: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:25:37.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:25:36 smithi100 ceph-mon[32599]: pgmap v1810: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:25:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:25:38 smithi167 ceph-mon[37140]: pgmap v1811: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:25:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:25:38 smithi138 ceph-mon[36954]: pgmap v1811: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:25:39.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:25:38 smithi100 ceph-mon[32599]: pgmap v1811: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:25:40.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:25:39 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:25:40.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:25:39 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:25:40.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:25:39 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:25:40.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:25:39 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:25:40.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:25:39 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:25:40.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:25:39 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:25:40.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:25:39 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:25:40.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:25:39 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:25:40.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:25:39 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:25:40.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:25:39 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:25:40.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:25:39 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:25:40.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:25:39 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:25:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:25:40 smithi167 ceph-mon[37140]: pgmap v1812: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:25:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:25:40 smithi138 ceph-mon[36954]: pgmap v1812: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:25:41.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:25:40 smithi100 ceph-mon[32599]: pgmap v1812: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:25:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:25:42 smithi167 ceph-mon[37140]: pgmap v1813: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:25:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:25:42 smithi138 ceph-mon[36954]: pgmap v1813: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:25:43.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:25:42 smithi100 ceph-mon[32599]: pgmap v1813: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:25:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:25:44 smithi167 ceph-mon[37140]: pgmap v1814: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:25:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:25:44 smithi138 ceph-mon[36954]: pgmap v1814: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:25:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:25:44 smithi100 ceph-mon[32599]: pgmap v1814: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:25:45.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:25:45.429+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:25:45.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:25:45.429+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:25:45.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:25:45.429+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:25:45.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:25:45.429+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:25:45.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:25:45.429+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:25:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:25:46 smithi138 ceph-mon[36954]: pgmap v1815: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:25:47.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:25:46 smithi100 ceph-mon[32599]: pgmap v1815: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:25:47.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:25:46 smithi167 ceph-mon[37140]: pgmap v1815: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:25:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:25:48 smithi138 ceph-mon[36954]: pgmap v1816: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:25:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:25:48 smithi100 ceph-mon[32599]: pgmap v1816: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:25:49.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:25:48 smithi167 ceph-mon[37140]: pgmap v1816: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:25:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:25:50 smithi138 ceph-mon[36954]: pgmap v1817: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:25:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:25:50 smithi100 ceph-mon[32599]: pgmap v1817: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:25:51.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:25:50 smithi167 ceph-mon[37140]: pgmap v1817: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:25:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:25:52 smithi138 ceph-mon[36954]: pgmap v1818: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:25:53.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:25:52 smithi100 ceph-mon[32599]: pgmap v1818: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:25:53.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:25:52 smithi167 ceph-mon[37140]: pgmap v1818: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:25:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:25:54 smithi138 ceph-mon[36954]: pgmap v1819: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:25:55.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:25:54 smithi100 ceph-mon[32599]: pgmap v1819: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:25:55.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:25:54 smithi167 ceph-mon[37140]: pgmap v1819: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:25:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:25:56 smithi138 ceph-mon[36954]: pgmap v1820: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:25:57.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:25:56 smithi100 ceph-mon[32599]: pgmap v1820: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:25:57.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:25:56 smithi167 ceph-mon[37140]: pgmap v1820: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:25:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:25:58 smithi138 ceph-mon[36954]: pgmap v1821: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:25:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:25:58 smithi100 ceph-mon[32599]: pgmap v1821: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:25:59.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:25:58 smithi167 ceph-mon[37140]: pgmap v1821: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:26:00.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:26:00.439+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:26:00.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:26:00.440+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:26:00.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:26:00.440+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:26:00.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:26:00.440+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:26:00.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:26:00.440+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:26:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:00 smithi138 ceph-mon[36954]: pgmap v1822: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:26:01.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:00 smithi100 ceph-mon[32599]: pgmap v1822: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:26:01.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:00 smithi167 ceph-mon[37140]: pgmap v1822: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:26:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:02 smithi138 ceph-mon[36954]: pgmap v1823: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:26:03.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:02 smithi100 ceph-mon[32599]: pgmap v1823: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:26:03.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:02 smithi167 ceph-mon[37140]: pgmap v1823: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:26:05.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:04 smithi138 ceph-mon[36954]: pgmap v1824: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:26:05.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:04 smithi100 ceph-mon[32599]: pgmap v1824: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:26:05.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:04 smithi167 ceph-mon[37140]: pgmap v1824: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:26:07.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:06 smithi138 ceph-mon[36954]: pgmap v1825: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:26:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:06 smithi100 ceph-mon[32599]: pgmap v1825: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:26:07.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:06 smithi167 ceph-mon[37140]: pgmap v1825: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:26:09.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:08 smithi167 ceph-mon[37140]: pgmap v1826: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:26:09.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:08 smithi167 ceph-mon[37140]: osdmap e164: 12 total, 12 up, 12 in 2024-07-26T14:26:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:08 smithi138 ceph-mon[36954]: pgmap v1826: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:26:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:08 smithi138 ceph-mon[36954]: osdmap e164: 12 total, 12 up, 12 in 2024-07-26T14:26:09.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:08 smithi100 ceph-mon[32599]: pgmap v1826: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:26:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:08 smithi100 ceph-mon[32599]: osdmap e164: 12 total, 12 up, 12 in 2024-07-26T14:26:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:10 smithi138 ceph-mon[36954]: pgmap v1828: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:26:11.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:10 smithi100 ceph-mon[32599]: pgmap v1828: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:26:11.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:10 smithi167 ceph-mon[37140]: pgmap v1828: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:26:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:12 smithi138 ceph-mon[36954]: pgmap v1829: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:26:13.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:12 smithi100 ceph-mon[32599]: pgmap v1829: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:26:13.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:12 smithi167 ceph-mon[37140]: pgmap v1829: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:26:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:14 smithi138 ceph-mon[36954]: pgmap v1830: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:26:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:14 smithi138 ceph-mon[36954]: osdmap e165: 12 total, 12 up, 12 in 2024-07-26T14:26:15.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:14 smithi100 ceph-mon[32599]: pgmap v1830: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:26:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:14 smithi100 ceph-mon[32599]: osdmap e165: 12 total, 12 up, 12 in 2024-07-26T14:26:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:14 smithi167 ceph-mon[37140]: pgmap v1830: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:26:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:14 smithi167 ceph-mon[37140]: osdmap e165: 12 total, 12 up, 12 in 2024-07-26T14:26:15.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:26:15.455+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:26:15.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:26:15.456+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:26:15.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:26:15.456+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:26:15.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:26:15.456+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:26:15.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:26:15.456+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:26:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:16 smithi138 ceph-mon[36954]: pgmap v1832: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:26:17.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:16 smithi100 ceph-mon[32599]: pgmap v1832: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:26:17.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:16 smithi167 ceph-mon[37140]: pgmap v1832: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:26:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:18 smithi138 ceph-mon[36954]: pgmap v1833: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 215 B/s wr, 6 op/s 2024-07-26T14:26:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:18 smithi100 ceph-mon[32599]: pgmap v1833: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 215 B/s wr, 6 op/s 2024-07-26T14:26:19.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:18 smithi167 ceph-mon[37140]: pgmap v1833: 97 pgs: 97 active+clean; 234 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 215 B/s wr, 6 op/s 2024-07-26T14:26:21.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:20 smithi138 ceph-mon[36954]: pgmap v1834: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:26:21.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:20 smithi100 ceph-mon[32599]: pgmap v1834: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:26:21.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:20 smithi167 ceph-mon[37140]: pgmap v1834: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:26:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:22 smithi138 ceph-mon[36954]: pgmap v1835: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:26:23.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:22 smithi100 ceph-mon[32599]: pgmap v1835: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:26:23.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:22 smithi167 ceph-mon[37140]: pgmap v1835: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:26:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:24 smithi138 ceph-mon[36954]: pgmap v1836: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:26:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:24 smithi100 ceph-mon[32599]: pgmap v1836: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:26:25.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:24 smithi167 ceph-mon[37140]: pgmap v1836: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:26:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:26 smithi138 ceph-mon[36954]: pgmap v1837: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:26:27.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:26 smithi100 ceph-mon[32599]: pgmap v1837: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:26:27.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:26 smithi167 ceph-mon[37140]: pgmap v1837: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:26:29.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:28 smithi138 ceph-mon[36954]: pgmap v1838: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:26:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:28 smithi100 ceph-mon[32599]: pgmap v1838: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:26:29.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:28 smithi167 ceph-mon[37140]: pgmap v1838: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:26:30.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:26:30.468+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:26:30.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:26:30.468+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:26:30.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:26:30.468+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:26:30.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:26:30.468+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:26:30.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:26:30.468+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:26:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:30 smithi138 ceph-mon[36954]: pgmap v1839: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:26:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:30 smithi100 ceph-mon[32599]: pgmap v1839: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:26:31.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:30 smithi167 ceph-mon[37140]: pgmap v1839: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:26:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:32 smithi138 ceph-mon[36954]: pgmap v1840: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:26:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:32 smithi100 ceph-mon[32599]: pgmap v1840: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:26:33.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:32 smithi167 ceph-mon[37140]: pgmap v1840: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:26:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:34 smithi138 ceph-mon[36954]: pgmap v1841: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:26:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:34 smithi100 ceph-mon[32599]: pgmap v1841: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:26:35.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:34 smithi167 ceph-mon[37140]: pgmap v1841: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:26:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:36 smithi138 ceph-mon[36954]: pgmap v1842: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:26:37.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:36 smithi100 ceph-mon[32599]: pgmap v1842: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:26:37.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:36 smithi167 ceph-mon[37140]: pgmap v1842: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:26:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:38 smithi138 ceph-mon[36954]: pgmap v1843: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:26:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:38 smithi100 ceph-mon[32599]: pgmap v1843: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:26:39.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:38 smithi167 ceph-mon[37140]: pgmap v1843: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:26:40.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:39 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:26:40.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:39 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:26:40.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:39 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:26:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:40 smithi138 ceph-mon[36954]: pgmap v1844: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:26:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:26:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:26:41.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:26:41.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:26:41.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:26:41.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:26:41.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:26:41.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:26:41.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:26:41.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:26:41.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:26:41.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:26:41.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:26:41.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:26:41.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:26:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:40 smithi100 ceph-mon[32599]: pgmap v1844: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:26:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:26:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:26:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:26:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:26:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:26:41.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:26:41.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:26:41.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:26:41.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:26:41.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:26:41.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:26:41.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:26:41.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:26:41.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:26:41.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:26:41.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:40 smithi167 ceph-mon[37140]: pgmap v1844: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:26:41.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:26:41.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:26:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:26:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:26:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:26:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:26:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:26:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:26:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:26:41.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:26:41.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:26:41.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:26:41.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:26:41.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:26:41.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:26:42.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:41 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T14:26:42.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:41 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T14:26:42.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:41 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T14:26:42.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:41 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T14:26:42.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:41 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T14:26:42.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:41 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T14:26:42.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:41 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T14:26:42.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:41 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T14:26:42.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:41 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T14:26:42.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:41 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T14:26:42.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:41 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T14:26:42.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:41 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T14:26:42.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:41 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T14:26:42.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:41 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T14:26:42.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:41 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T14:26:42.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:41 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T14:26:42.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:41 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T14:26:42.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:41 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T14:26:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:42 smithi138 ceph-mon[36954]: pgmap v1845: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:26:43.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:42 smithi100 ceph-mon[32599]: pgmap v1845: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:26:43.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:42 smithi167 ceph-mon[37140]: pgmap v1845: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:26:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:44 smithi138 ceph-mon[36954]: pgmap v1846: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:26:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:44 smithi100 ceph-mon[32599]: pgmap v1846: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:26:45.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:44 smithi167 ceph-mon[37140]: pgmap v1846: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:26:45.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:26:45.471+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:26:45.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:26:45.472+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:26:45.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:26:45.472+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:26:45.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:26:45.472+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:26:45.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:26:45.472+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:26:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:46 smithi138 ceph-mon[36954]: pgmap v1847: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:26:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:46 smithi100 ceph-mon[32599]: pgmap v1847: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:26:47.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:46 smithi167 ceph-mon[37140]: pgmap v1847: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:26:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:48 smithi138 ceph-mon[36954]: pgmap v1848: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:26:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:48 smithi100 ceph-mon[32599]: pgmap v1848: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:26:49.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:48 smithi167 ceph-mon[37140]: pgmap v1848: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:26:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:50 smithi138 ceph-mon[36954]: pgmap v1849: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:26:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:50 smithi100 ceph-mon[32599]: pgmap v1849: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:26:51.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:50 smithi167 ceph-mon[37140]: pgmap v1849: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:26:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:52 smithi138 ceph-mon[36954]: pgmap v1850: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:26:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:52 smithi100 ceph-mon[32599]: pgmap v1850: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:26:53.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:52 smithi167 ceph-mon[37140]: pgmap v1850: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:26:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:54 smithi138 ceph-mon[36954]: pgmap v1851: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:26:55.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:54 smithi100 ceph-mon[32599]: pgmap v1851: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:26:55.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:54 smithi167 ceph-mon[37140]: pgmap v1851: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:26:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:56 smithi138 ceph-mon[36954]: pgmap v1852: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:26:57.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:56 smithi100 ceph-mon[32599]: pgmap v1852: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:26:57.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:56 smithi167 ceph-mon[37140]: pgmap v1852: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:26:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:26:58 smithi138 ceph-mon[36954]: pgmap v1853: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:26:59.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:26:58 smithi100 ceph-mon[32599]: pgmap v1853: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:26:59.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:26:58 smithi167 ceph-mon[37140]: pgmap v1853: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:27:00.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:27:00.487+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:27:00.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:27:00.488+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:27:00.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:27:00.488+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:27:00.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:27:00.488+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:27:00.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:27:00.488+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:27:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:27:00 smithi138 ceph-mon[36954]: pgmap v1854: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:27:01.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:27:00 smithi100 ceph-mon[32599]: pgmap v1854: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:27:01.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:27:00 smithi167 ceph-mon[37140]: pgmap v1854: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:27:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:27:02 smithi138 ceph-mon[36954]: pgmap v1855: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:27:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:27:02 smithi100 ceph-mon[32599]: pgmap v1855: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:27:03.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:27:02 smithi167 ceph-mon[37140]: pgmap v1855: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:27:03.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:27:03.343+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c006610 con 0x7fe5a81087a0 2024-07-26T14:27:03.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:27:03.343+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T14:27:03.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:27:03.343+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 22 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c100 con 0x7fe5a81087a0 2024-07-26T14:27:03.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:27:03.343+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 22 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T14:27:05.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:27:04 smithi138 ceph-mon[36954]: pgmap v1856: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:27:05.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:27:04 smithi100 ceph-mon[32599]: pgmap v1856: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:27:05.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:27:04 smithi167 ceph-mon[37140]: pgmap v1856: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:27:07.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:27:06 smithi138 ceph-mon[36954]: pgmap v1857: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:27:07.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:27:06 smithi100 ceph-mon[32599]: pgmap v1857: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:27:07.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:27:06 smithi167 ceph-mon[37140]: pgmap v1857: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:27:08.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:27:07 smithi138 ceph-mon[36954]: osdmap e166: 12 total, 12 up, 12 in 2024-07-26T14:27:08.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:27:07 smithi100 ceph-mon[32599]: osdmap e166: 12 total, 12 up, 12 in 2024-07-26T14:27:08.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:27:07 smithi167 ceph-mon[37140]: osdmap e166: 12 total, 12 up, 12 in 2024-07-26T14:27:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:27:08 smithi138 ceph-mon[36954]: pgmap v1859: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:27:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:27:08 smithi100 ceph-mon[32599]: pgmap v1859: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:27:09.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:27:08 smithi167 ceph-mon[37140]: pgmap v1859: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:27:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:27:10 smithi100 ceph-mon[32599]: pgmap v1860: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:27:11.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:27:10 smithi167 ceph-mon[37140]: pgmap v1860: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:27:11.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:27:10 smithi138 ceph-mon[36954]: pgmap v1860: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:27:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:27:12 smithi100 ceph-mon[32599]: pgmap v1861: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:27:13.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:27:12 smithi167 ceph-mon[37140]: pgmap v1861: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:27:13.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:27:12 smithi138 ceph-mon[36954]: pgmap v1861: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:27:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:27:14 smithi100 ceph-mon[32599]: pgmap v1862: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:27:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:27:14 smithi100 ceph-mon[32599]: osdmap e167: 12 total, 12 up, 12 in 2024-07-26T14:27:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:27:14 smithi167 ceph-mon[37140]: pgmap v1862: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:27:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:27:14 smithi167 ceph-mon[37140]: osdmap e167: 12 total, 12 up, 12 in 2024-07-26T14:27:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:27:14 smithi138 ceph-mon[36954]: pgmap v1862: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:27:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:27:14 smithi138 ceph-mon[36954]: osdmap e167: 12 total, 12 up, 12 in 2024-07-26T14:27:15.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:27:15.503+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:27:15.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:27:15.504+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:27:15.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:27:15.504+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:27:15.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:27:15.504+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:27:15.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:27:15.504+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:27:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:27:16 smithi100 ceph-mon[32599]: pgmap v1864: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 244 B/s wr, 6 op/s 2024-07-26T14:27:17.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:27:16 smithi167 ceph-mon[37140]: pgmap v1864: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 244 B/s wr, 6 op/s 2024-07-26T14:27:17.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:27:16 smithi138 ceph-mon[36954]: pgmap v1864: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 244 B/s wr, 6 op/s 2024-07-26T14:27:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:27:18 smithi100 ceph-mon[32599]: pgmap v1865: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:27:19.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:27:18 smithi167 ceph-mon[37140]: pgmap v1865: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:27:19.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:27:18 smithi138 ceph-mon[36954]: pgmap v1865: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:27:21.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:27:20 smithi100 ceph-mon[32599]: pgmap v1866: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:27:21.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:27:20 smithi167 ceph-mon[37140]: pgmap v1866: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:27:21.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:27:20 smithi138 ceph-mon[36954]: pgmap v1866: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:27:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:27:22 smithi100 ceph-mon[32599]: pgmap v1867: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:27:23.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:27:22 smithi167 ceph-mon[37140]: pgmap v1867: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:27:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:27:22 smithi138 ceph-mon[36954]: pgmap v1867: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:27:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:27:24 smithi100 ceph-mon[32599]: pgmap v1868: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:27:25.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:27:24 smithi167 ceph-mon[37140]: pgmap v1868: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:27:25.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:27:24 smithi138 ceph-mon[36954]: pgmap v1868: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:27:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:27:26 smithi100 ceph-mon[32599]: pgmap v1869: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:27:27.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:27:26 smithi167 ceph-mon[37140]: pgmap v1869: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:27:27.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:27:26 smithi138 ceph-mon[36954]: pgmap v1869: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:27:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:27:28 smithi100 ceph-mon[32599]: pgmap v1870: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:27:29.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:27:28 smithi167 ceph-mon[37140]: pgmap v1870: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:27:29.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:27:28 smithi138 ceph-mon[36954]: pgmap v1870: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:27:30.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:27:30.507+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:27:30.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:27:30.507+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:27:30.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:27:30.507+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:27:30.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:27:30.507+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:27:30.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:27:30.507+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:27:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:27:30 smithi100 ceph-mon[32599]: pgmap v1871: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:27:31.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:27:30 smithi167 ceph-mon[37140]: pgmap v1871: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:27:31.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:27:30 smithi138 ceph-mon[36954]: pgmap v1871: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:27:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:27:32 smithi100 ceph-mon[32599]: pgmap v1872: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:27:33.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:27:32 smithi167 ceph-mon[37140]: pgmap v1872: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:27:33.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:27:32 smithi138 ceph-mon[36954]: pgmap v1872: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:27:35.242 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:27:34 smithi138 ceph-mon[36954]: pgmap v1873: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:27:35.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:27:34 smithi100 ceph-mon[32599]: pgmap v1873: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:27:35.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:27:34 smithi167 ceph-mon[37140]: pgmap v1873: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:27:36.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:27:36 smithi138 ceph-mon[36954]: pgmap v1874: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:27:36.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:27:36 smithi100 ceph-mon[32599]: pgmap v1874: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:27:36.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:27:36 smithi167 ceph-mon[37140]: pgmap v1874: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:27:38.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:27:38 smithi138 ceph-mon[36954]: pgmap v1875: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:27:38.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:27:38 smithi100 ceph-mon[32599]: pgmap v1875: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:27:38.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:27:38 smithi167 ceph-mon[37140]: pgmap v1875: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:27:40.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:27:40 smithi138 ceph-mon[36954]: pgmap v1876: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:27:40.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:27:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:27:40.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:27:40 smithi100 ceph-mon[32599]: pgmap v1876: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:27:40.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:27:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:27:40.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:27:40 smithi167 ceph-mon[37140]: pgmap v1876: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:27:40.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:27:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:27:41.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:27:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:27:41.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:27:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:27:41.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:27:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:27:41.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:27:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:27:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:27:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:27:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:27:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:27:41.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:27:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:27:41.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:27:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:27:41.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:27:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:27:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:27:42 smithi100 ceph-mon[32599]: pgmap v1877: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:27:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:27:42 smithi167 ceph-mon[37140]: pgmap v1877: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:27:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:27:42 smithi138 ceph-mon[36954]: pgmap v1877: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:27:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:27:44 smithi100 ceph-mon[32599]: pgmap v1878: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:27:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:27:44 smithi167 ceph-mon[37140]: pgmap v1878: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:27:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:27:44 smithi138 ceph-mon[36954]: pgmap v1878: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:27:45.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:27:45.518+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:27:45.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:27:45.518+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:27:45.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:27:45.518+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:27:45.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:27:45.518+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:27:45.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:27:45.518+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:27:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:27:46 smithi100 ceph-mon[32599]: pgmap v1879: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:27:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:27:46 smithi167 ceph-mon[37140]: pgmap v1879: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:27:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:27:46 smithi138 ceph-mon[36954]: pgmap v1879: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:27:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:27:48 smithi100 ceph-mon[32599]: pgmap v1880: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:27:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:27:48 smithi167 ceph-mon[37140]: pgmap v1880: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:27:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:27:48 smithi138 ceph-mon[36954]: pgmap v1880: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:27:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:27:50 smithi167 ceph-mon[37140]: pgmap v1881: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:27:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:27:50 smithi138 ceph-mon[36954]: pgmap v1881: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:27:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:27:50 smithi100 ceph-mon[32599]: pgmap v1881: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:27:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:27:52 smithi167 ceph-mon[37140]: pgmap v1882: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:27:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:27:52 smithi138 ceph-mon[36954]: pgmap v1882: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:27:53.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:27:52 smithi100 ceph-mon[32599]: pgmap v1882: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:27:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:27:54 smithi167 ceph-mon[37140]: pgmap v1883: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:27:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:27:54 smithi138 ceph-mon[36954]: pgmap v1883: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:27:55.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:27:54 smithi100 ceph-mon[32599]: pgmap v1883: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:27:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:27:56 smithi167 ceph-mon[37140]: pgmap v1884: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:27:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:27:56 smithi138 ceph-mon[36954]: pgmap v1884: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:27:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:27:56 smithi100 ceph-mon[32599]: pgmap v1884: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:27:59.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:27:58 smithi167 ceph-mon[37140]: pgmap v1885: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:27:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:27:58 smithi138 ceph-mon[36954]: pgmap v1885: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:27:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:27:58 smithi100 ceph-mon[32599]: pgmap v1885: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:28:00.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:28:00.533+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:28:00.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:28:00.534+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:28:00.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:28:00.534+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:28:00.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:28:00.534+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:28:00.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:28:00.534+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:28:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:28:00 smithi100 ceph-mon[32599]: pgmap v1886: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:28:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:28:00 smithi167 ceph-mon[37140]: pgmap v1886: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:28:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:28:00 smithi138 ceph-mon[36954]: pgmap v1886: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:28:03.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:28:02 smithi167 ceph-mon[37140]: pgmap v1887: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:28:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:28:02 smithi138 ceph-mon[36954]: pgmap v1887: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:28:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:28:02 smithi100 ceph-mon[32599]: pgmap v1887: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:28:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:28:04 smithi167 ceph-mon[37140]: pgmap v1888: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:28:05.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:28:04 smithi138 ceph-mon[36954]: pgmap v1888: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:28:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:28:04 smithi100 ceph-mon[32599]: pgmap v1888: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:28:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:28:06 smithi167 ceph-mon[37140]: pgmap v1889: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:28:07.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:28:06 smithi138 ceph-mon[36954]: pgmap v1889: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:28:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:28:06 smithi100 ceph-mon[32599]: pgmap v1889: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:28:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:28:08 smithi138 ceph-mon[36954]: pgmap v1890: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:28:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:28:08 smithi138 ceph-mon[36954]: osdmap e168: 12 total, 12 up, 12 in 2024-07-26T14:28:09.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:28:08 smithi100 ceph-mon[32599]: pgmap v1890: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:28:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:28:08 smithi100 ceph-mon[32599]: osdmap e168: 12 total, 12 up, 12 in 2024-07-26T14:28:09.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:28:08 smithi167 ceph-mon[37140]: pgmap v1890: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:28:09.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:28:08 smithi167 ceph-mon[37140]: osdmap e168: 12 total, 12 up, 12 in 2024-07-26T14:28:11.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:28:10 smithi167 ceph-mon[37140]: pgmap v1892: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:28:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:28:10 smithi138 ceph-mon[36954]: pgmap v1892: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:28:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:28:10 smithi100 ceph-mon[32599]: pgmap v1892: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:28:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:28:12 smithi138 ceph-mon[36954]: pgmap v1893: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:28:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:28:12 smithi100 ceph-mon[32599]: pgmap v1893: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:28:13.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:28:12 smithi167 ceph-mon[37140]: pgmap v1893: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:28:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:28:14 smithi138 ceph-mon[36954]: pgmap v1894: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:28:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:28:14 smithi138 ceph-mon[36954]: osdmap e169: 12 total, 12 up, 12 in 2024-07-26T14:28:15.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:28:14 smithi100 ceph-mon[32599]: pgmap v1894: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:28:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:28:14 smithi100 ceph-mon[32599]: osdmap e169: 12 total, 12 up, 12 in 2024-07-26T14:28:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:28:14 smithi167 ceph-mon[37140]: pgmap v1894: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:28:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:28:14 smithi167 ceph-mon[37140]: osdmap e169: 12 total, 12 up, 12 in 2024-07-26T14:28:15.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:28:15.548+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:28:15.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:28:15.549+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:28:15.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:28:15.549+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:28:15.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:28:15.549+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:28:15.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:28:15.549+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:28:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:28:16 smithi138 ceph-mon[36954]: pgmap v1896: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:28:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:28:16 smithi100 ceph-mon[32599]: pgmap v1896: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:28:17.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:28:16 smithi167 ceph-mon[37140]: pgmap v1896: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:28:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:28:18 smithi138 ceph-mon[36954]: pgmap v1897: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 214 B/s wr, 6 op/s 2024-07-26T14:28:19.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:28:18 smithi100 ceph-mon[32599]: pgmap v1897: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 214 B/s wr, 6 op/s 2024-07-26T14:28:19.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:28:18 smithi167 ceph-mon[37140]: pgmap v1897: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 214 B/s wr, 6 op/s 2024-07-26T14:28:21.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:28:20 smithi138 ceph-mon[36954]: pgmap v1898: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:28:21.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:28:20 smithi100 ceph-mon[32599]: pgmap v1898: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:28:21.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:28:20 smithi167 ceph-mon[37140]: pgmap v1898: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:28:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:28:22 smithi138 ceph-mon[36954]: pgmap v1899: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:28:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:28:22 smithi100 ceph-mon[32599]: pgmap v1899: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:28:23.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:28:22 smithi167 ceph-mon[37140]: pgmap v1899: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:28:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:28:24 smithi138 ceph-mon[36954]: pgmap v1900: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:28:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:28:24 smithi100 ceph-mon[32599]: pgmap v1900: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:28:25.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:28:24 smithi167 ceph-mon[37140]: pgmap v1900: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:28:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:28:26 smithi138 ceph-mon[36954]: pgmap v1901: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:28:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:28:26 smithi100 ceph-mon[32599]: pgmap v1901: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:28:27.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:28:26 smithi167 ceph-mon[37140]: pgmap v1901: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:28:29.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:28:28 smithi138 ceph-mon[36954]: pgmap v1902: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:28:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:28:28 smithi100 ceph-mon[32599]: pgmap v1902: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:28:29.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:28:28 smithi167 ceph-mon[37140]: pgmap v1902: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:28:30.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:28:30.563+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:28:30.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:28:30.564+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:28:30.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:28:30.564+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:28:30.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:28:30.564+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:28:30.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:28:30.564+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:28:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:28:30 smithi138 ceph-mon[36954]: pgmap v1903: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:28:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:28:30 smithi100 ceph-mon[32599]: pgmap v1903: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:28:31.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:28:30 smithi167 ceph-mon[37140]: pgmap v1903: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:28:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:28:32 smithi138 ceph-mon[36954]: pgmap v1904: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:28:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:28:32 smithi100 ceph-mon[32599]: pgmap v1904: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:28:33.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:28:32 smithi167 ceph-mon[37140]: pgmap v1904: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:28:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:28:34 smithi138 ceph-mon[36954]: pgmap v1905: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:28:35.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:28:34 smithi100 ceph-mon[32599]: pgmap v1905: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:28:35.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:28:34 smithi167 ceph-mon[37140]: pgmap v1905: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:28:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:28:36 smithi138 ceph-mon[36954]: pgmap v1906: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:28:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:28:36 smithi100 ceph-mon[32599]: pgmap v1906: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:28:37.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:28:36 smithi167 ceph-mon[37140]: pgmap v1906: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:28:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:28:38 smithi138 ceph-mon[36954]: pgmap v1907: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:28:39.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:28:38 smithi100 ceph-mon[32599]: pgmap v1907: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:28:39.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:28:38 smithi167 ceph-mon[37140]: pgmap v1907: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:28:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:28:40 smithi138 ceph-mon[36954]: pgmap v1908: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:28:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:28:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:28:41.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:28:40 smithi100 ceph-mon[32599]: pgmap v1908: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:28:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:28:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:28:41.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:28:40 smithi167 ceph-mon[37140]: pgmap v1908: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:28:41.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:28:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:28:42.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:28:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:28:42.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:28:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:28:42.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:28:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:28:42.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:28:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:28:42.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:28:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:28:42.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:28:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:28:42.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:28:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:28:42.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:28:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:28:42.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:28:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:28:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:28:42 smithi138 ceph-mon[36954]: pgmap v1909: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:28:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:28:42 smithi100 ceph-mon[32599]: pgmap v1909: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:28:43.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:28:42 smithi167 ceph-mon[37140]: pgmap v1909: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:28:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:28:44 smithi138 ceph-mon[36954]: pgmap v1910: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:28:45.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:28:44 smithi100 ceph-mon[32599]: pgmap v1910: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:28:45.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:28:44 smithi167 ceph-mon[37140]: pgmap v1910: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:28:45.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:28:45.576+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:28:45.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:28:45.576+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:28:45.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:28:45.576+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:28:45.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:28:45.576+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:28:45.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:28:45.576+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:28:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:28:46 smithi138 ceph-mon[36954]: pgmap v1911: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:28:47.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:28:46 smithi100 ceph-mon[32599]: pgmap v1911: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:28:47.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:28:46 smithi167 ceph-mon[37140]: pgmap v1911: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:28:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:28:48 smithi138 ceph-mon[36954]: pgmap v1912: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:28:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:28:48 smithi100 ceph-mon[32599]: pgmap v1912: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:28:49.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:28:48 smithi167 ceph-mon[37140]: pgmap v1912: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:28:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:28:50 smithi138 ceph-mon[36954]: pgmap v1913: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:28:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:28:50 smithi100 ceph-mon[32599]: pgmap v1913: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:28:51.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:28:50 smithi167 ceph-mon[37140]: pgmap v1913: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:28:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:28:52 smithi138 ceph-mon[36954]: pgmap v1914: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:28:53.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:28:52 smithi100 ceph-mon[32599]: pgmap v1914: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:28:53.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:28:52 smithi167 ceph-mon[37140]: pgmap v1914: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:28:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:28:54 smithi138 ceph-mon[36954]: pgmap v1915: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:28:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:28:54 smithi100 ceph-mon[32599]: pgmap v1915: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:28:55.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:28:54 smithi167 ceph-mon[37140]: pgmap v1915: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:28:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:28:56 smithi138 ceph-mon[36954]: pgmap v1916: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:28:57.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:28:56 smithi100 ceph-mon[32599]: pgmap v1916: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:28:57.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:28:56 smithi167 ceph-mon[37140]: pgmap v1916: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:28:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:28:58 smithi138 ceph-mon[36954]: pgmap v1917: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:28:59.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:28:58 smithi100 ceph-mon[32599]: pgmap v1917: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:28:59.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:28:58 smithi167 ceph-mon[37140]: pgmap v1917: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:29:00.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:29:00.592+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:29:00.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:29:00.592+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:29:00.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:29:00.592+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:29:00.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:29:00.592+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:29:00.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:29:00.592+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:29:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:00 smithi138 ceph-mon[36954]: pgmap v1918: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:29:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:00 smithi100 ceph-mon[32599]: pgmap v1918: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:29:01.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:00 smithi167 ceph-mon[37140]: pgmap v1918: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:29:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:02 smithi138 ceph-mon[36954]: pgmap v1919: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:29:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:02 smithi100 ceph-mon[32599]: pgmap v1919: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:29:03.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:02 smithi167 ceph-mon[37140]: pgmap v1919: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:29:05.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:04 smithi138 ceph-mon[36954]: pgmap v1920: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:29:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:04 smithi100 ceph-mon[32599]: pgmap v1920: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:29:05.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:04 smithi167 ceph-mon[37140]: pgmap v1920: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:29:07.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:06 smithi138 ceph-mon[36954]: pgmap v1921: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:29:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:06 smithi100 ceph-mon[32599]: pgmap v1921: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:29:07.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:06 smithi167 ceph-mon[37140]: pgmap v1921: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:29:08.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:07 smithi138 ceph-mon[36954]: osdmap e170: 12 total, 12 up, 12 in 2024-07-26T14:29:08.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:07 smithi100 ceph-mon[32599]: osdmap e170: 12 total, 12 up, 12 in 2024-07-26T14:29:08.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:07 smithi167 ceph-mon[37140]: osdmap e170: 12 total, 12 up, 12 in 2024-07-26T14:29:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:08 smithi138 ceph-mon[36954]: pgmap v1923: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:29:09.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:08 smithi100 ceph-mon[32599]: pgmap v1923: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:29:09.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:08 smithi167 ceph-mon[37140]: pgmap v1923: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:29:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:10 smithi138 ceph-mon[36954]: pgmap v1924: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:29:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:10 smithi100 ceph-mon[32599]: pgmap v1924: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:29:11.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:10 smithi167 ceph-mon[37140]: pgmap v1924: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:29:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:12 smithi138 ceph-mon[36954]: pgmap v1925: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:29:13.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:12 smithi100 ceph-mon[32599]: pgmap v1925: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:29:13.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:12 smithi167 ceph-mon[37140]: pgmap v1925: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:29:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:14 smithi138 ceph-mon[36954]: pgmap v1926: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:29:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:14 smithi138 ceph-mon[36954]: osdmap e171: 12 total, 12 up, 12 in 2024-07-26T14:29:15.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:14 smithi100 ceph-mon[32599]: pgmap v1926: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:29:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:14 smithi100 ceph-mon[32599]: osdmap e171: 12 total, 12 up, 12 in 2024-07-26T14:29:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:14 smithi167 ceph-mon[37140]: pgmap v1926: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:29:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:14 smithi167 ceph-mon[37140]: osdmap e171: 12 total, 12 up, 12 in 2024-07-26T14:29:15.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:29:15.607+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:29:15.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:29:15.608+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:29:15.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:29:15.608+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:29:15.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:29:15.608+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:29:15.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:29:15.608+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:29:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:16 smithi138 ceph-mon[36954]: pgmap v1928: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 241 B/s wr, 6 op/s 2024-07-26T14:29:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:16 smithi100 ceph-mon[32599]: pgmap v1928: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 241 B/s wr, 6 op/s 2024-07-26T14:29:17.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:16 smithi167 ceph-mon[37140]: pgmap v1928: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 241 B/s wr, 6 op/s 2024-07-26T14:29:18.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:29:18.361+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c006610 con 0x7fe5a81087a0 2024-07-26T14:29:18.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:29:18.361+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T14:29:18.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:29:18.362+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 23 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe5a008c070 con 0x7fe5a81087a0 2024-07-26T14:29:18.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:29:18.362+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 23 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T14:29:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:18 smithi138 ceph-mon[36954]: pgmap v1929: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:29:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:18 smithi100 ceph-mon[32599]: pgmap v1929: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:29:19.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:18 smithi167 ceph-mon[37140]: pgmap v1929: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:29:21.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:20 smithi138 ceph-mon[36954]: pgmap v1930: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:29:21.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:20 smithi100 ceph-mon[32599]: pgmap v1930: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:29:21.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:20 smithi167 ceph-mon[37140]: pgmap v1930: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:29:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:22 smithi138 ceph-mon[36954]: pgmap v1931: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:29:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:22 smithi100 ceph-mon[32599]: pgmap v1931: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:29:23.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:22 smithi167 ceph-mon[37140]: pgmap v1931: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:29:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:24 smithi138 ceph-mon[36954]: pgmap v1932: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:29:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:24 smithi100 ceph-mon[32599]: pgmap v1932: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:29:25.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:24 smithi167 ceph-mon[37140]: pgmap v1932: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:29:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:26 smithi138 ceph-mon[36954]: pgmap v1933: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:29:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:26 smithi100 ceph-mon[32599]: pgmap v1933: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:29:27.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:26 smithi167 ceph-mon[37140]: pgmap v1933: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:29:29.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:28 smithi138 ceph-mon[36954]: pgmap v1934: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:29:29.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:28 smithi100 ceph-mon[32599]: pgmap v1934: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:29:29.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:28 smithi167 ceph-mon[37140]: pgmap v1934: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:29:30.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:29:30.623+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:29:30.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:29:30.624+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:29:30.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:29:30.624+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:29:30.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:29:30.624+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:29:30.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:29:30.624+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:29:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:30 smithi138 ceph-mon[36954]: pgmap v1935: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:29:31.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:30 smithi100 ceph-mon[32599]: pgmap v1935: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:29:31.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:30 smithi167 ceph-mon[37140]: pgmap v1935: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:29:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:32 smithi138 ceph-mon[36954]: pgmap v1936: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:29:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:32 smithi100 ceph-mon[32599]: pgmap v1936: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:29:33.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:32 smithi167 ceph-mon[37140]: pgmap v1936: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:29:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:34 smithi138 ceph-mon[36954]: pgmap v1937: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:29:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:34 smithi100 ceph-mon[32599]: pgmap v1937: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:29:35.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:34 smithi167 ceph-mon[37140]: pgmap v1937: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:29:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:36 smithi100 ceph-mon[32599]: pgmap v1938: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:29:37.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:36 smithi167 ceph-mon[37140]: pgmap v1938: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:29:37.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:36 smithi138 ceph-mon[36954]: pgmap v1938: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:29:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:38 smithi100 ceph-mon[32599]: pgmap v1939: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:29:39.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:38 smithi167 ceph-mon[37140]: pgmap v1939: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:29:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:38 smithi138 ceph-mon[36954]: pgmap v1939: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:29:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:40 smithi100 ceph-mon[32599]: pgmap v1940: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:29:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:40 smithi167 ceph-mon[37140]: pgmap v1940: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:29:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:40 smithi138 ceph-mon[36954]: pgmap v1940: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:29:42.227 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:29:42.250 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:29:42.259 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:29:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:42 smithi100 ceph-mon[32599]: pgmap v1941: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:29:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:42 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:29:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:42 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:29:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:42 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:29:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:42 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:29:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:42 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:29:43.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:42 smithi167 ceph-mon[37140]: pgmap v1941: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:29:43.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:42 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:29:43.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:42 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:29:43.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:42 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:29:43.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:42 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:29:43.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:42 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:29:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:42 smithi138 ceph-mon[36954]: pgmap v1941: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:29:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:42 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:29:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:42 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:29:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:42 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:29:43.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:42 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:29:43.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:42 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:29:44.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:43 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:29:44.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:43 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:29:44.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:43 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:29:44.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:43 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:29:44.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:43 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:29:44.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:43 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:29:44.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:43 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:29:44.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:43 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:29:44.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:43 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:29:44.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:43 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:29:44.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:43 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:29:44.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:43 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:29:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:44 smithi100 ceph-mon[32599]: pgmap v1942: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:29:45.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:44 smithi167 ceph-mon[37140]: pgmap v1942: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:29:45.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:44 smithi138 ceph-mon[36954]: pgmap v1942: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:29:45.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:29:45.639+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:29:45.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:29:45.640+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:29:45.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:29:45.640+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:29:45.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:29:45.640+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:29:45.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:29:45.640+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:29:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:46 smithi100 ceph-mon[32599]: pgmap v1943: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:29:47.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:46 smithi167 ceph-mon[37140]: pgmap v1943: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:29:47.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:46 smithi138 ceph-mon[36954]: pgmap v1943: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:29:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:48 smithi100 ceph-mon[32599]: pgmap v1944: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:29:49.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:48 smithi167 ceph-mon[37140]: pgmap v1944: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:29:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:48 smithi138 ceph-mon[36954]: pgmap v1944: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:29:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:50 smithi100 ceph-mon[32599]: pgmap v1945: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:29:51.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:50 smithi167 ceph-mon[37140]: pgmap v1945: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:29:51.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:50 smithi138 ceph-mon[36954]: pgmap v1945: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:29:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:52 smithi100 ceph-mon[32599]: pgmap v1946: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:29:53.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:52 smithi167 ceph-mon[37140]: pgmap v1946: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:29:53.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:52 smithi138 ceph-mon[36954]: pgmap v1946: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:29:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:54 smithi100 ceph-mon[32599]: pgmap v1947: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:29:55.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:54 smithi167 ceph-mon[37140]: pgmap v1947: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:29:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:54 smithi138 ceph-mon[36954]: pgmap v1947: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:29:56.986 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:56 smithi100 ceph-mon[32599]: pgmap v1948: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:29:57.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:56 smithi167 ceph-mon[37140]: pgmap v1948: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:29:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:56 smithi138 ceph-mon[36954]: pgmap v1948: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:29:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:29:58 smithi100 ceph-mon[32599]: pgmap v1949: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:29:59.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:29:58 smithi167 ceph-mon[37140]: pgmap v1949: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:29:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:29:58 smithi138 ceph-mon[36954]: pgmap v1949: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:30:00.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:30:00.655+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:30:00.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:30:00.655+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:30:00.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:30:00.655+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:30:00.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:30:00.655+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:30:00.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:30:00.655+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:30:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:00 smithi100 ceph-mon[32599]: pgmap v1950: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:30:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:00 smithi100 ceph-mon[32599]: overall HEALTH_OK 2024-07-26T14:30:01.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:00 smithi167 ceph-mon[37140]: pgmap v1950: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:30:01.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:00 smithi167 ceph-mon[37140]: overall HEALTH_OK 2024-07-26T14:30:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:00 smithi138 ceph-mon[36954]: pgmap v1950: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:30:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:00 smithi138 ceph-mon[36954]: overall HEALTH_OK 2024-07-26T14:30:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:02 smithi100 ceph-mon[32599]: pgmap v1951: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:30:03.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:02 smithi167 ceph-mon[37140]: pgmap v1951: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:30:03.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:30:03.368+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c006610 con 0x7fe5a81087a0 2024-07-26T14:30:03.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:30:03.368+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T14:30:03.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:30:03.368+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 24 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c070 con 0x7fe5a81087a0 2024-07-26T14:30:03.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:30:03.368+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 24 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T14:30:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:02 smithi138 ceph-mon[36954]: pgmap v1951: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:30:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:04 smithi100 ceph-mon[32599]: pgmap v1952: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:30:05.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:04 smithi167 ceph-mon[37140]: pgmap v1952: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:30:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:04 smithi138 ceph-mon[36954]: pgmap v1952: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:30:07.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:06 smithi167 ceph-mon[37140]: pgmap v1953: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:30:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:06 smithi138 ceph-mon[36954]: pgmap v1953: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:30:07.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:06 smithi100 ceph-mon[32599]: pgmap v1953: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:30:08.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:08 smithi167 ceph-mon[37140]: osdmap e172: 12 total, 12 up, 12 in 2024-07-26T14:30:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:08 smithi138 ceph-mon[36954]: osdmap e172: 12 total, 12 up, 12 in 2024-07-26T14:30:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:08 smithi100 ceph-mon[32599]: osdmap e172: 12 total, 12 up, 12 in 2024-07-26T14:30:09.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:09 smithi167 ceph-mon[37140]: pgmap v1955: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:30:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:09 smithi138 ceph-mon[36954]: pgmap v1955: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:30:09.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:09 smithi100 ceph-mon[32599]: pgmap v1955: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:30:10.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:10 smithi167 ceph-mon[37140]: pgmap v1956: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:30:10.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:10 smithi138 ceph-mon[36954]: pgmap v1956: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:30:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:10 smithi100 ceph-mon[32599]: pgmap v1956: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:30:12.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:12 smithi138 ceph-mon[36954]: pgmap v1957: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:30:12.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:12 smithi100 ceph-mon[32599]: pgmap v1957: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:30:12.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:12 smithi167 ceph-mon[37140]: pgmap v1957: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:30:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:14 smithi138 ceph-mon[36954]: pgmap v1958: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:30:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:14 smithi138 ceph-mon[36954]: osdmap e173: 12 total, 12 up, 12 in 2024-07-26T14:30:14.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:14 smithi100 ceph-mon[32599]: pgmap v1958: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:30:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:14 smithi100 ceph-mon[32599]: osdmap e173: 12 total, 12 up, 12 in 2024-07-26T14:30:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:14 smithi167 ceph-mon[37140]: pgmap v1958: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:30:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:14 smithi167 ceph-mon[37140]: osdmap e173: 12 total, 12 up, 12 in 2024-07-26T14:30:15.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:30:15.661+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:30:15.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:30:15.661+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:30:15.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:30:15.661+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:30:15.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:30:15.661+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:30:15.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:30:15.661+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:30:16.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:16 smithi100 ceph-mon[32599]: pgmap v1960: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 244 B/s wr, 6 op/s 2024-07-26T14:30:16.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:16 smithi167 ceph-mon[37140]: pgmap v1960: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 244 B/s wr, 6 op/s 2024-07-26T14:30:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:16 smithi138 ceph-mon[36954]: pgmap v1960: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 244 B/s wr, 6 op/s 2024-07-26T14:30:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:18 smithi100 ceph-mon[32599]: pgmap v1961: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:30:18.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:18 smithi167 ceph-mon[37140]: pgmap v1961: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:30:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:18 smithi138 ceph-mon[36954]: pgmap v1961: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:30:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:20 smithi100 ceph-mon[32599]: pgmap v1962: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T14:30:20.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:20 smithi167 ceph-mon[37140]: pgmap v1962: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T14:30:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:20 smithi138 ceph-mon[36954]: pgmap v1962: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T14:30:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:22 smithi100 ceph-mon[32599]: pgmap v1963: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T14:30:22.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:22 smithi167 ceph-mon[37140]: pgmap v1963: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T14:30:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:22 smithi138 ceph-mon[36954]: pgmap v1963: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T14:30:24.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:24 smithi100 ceph-mon[32599]: pgmap v1964: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T14:30:24.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:24 smithi167 ceph-mon[37140]: pgmap v1964: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T14:30:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:24 smithi138 ceph-mon[36954]: pgmap v1964: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T14:30:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:26 smithi100 ceph-mon[32599]: pgmap v1965: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:30:26.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:26 smithi167 ceph-mon[37140]: pgmap v1965: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:30:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:26 smithi138 ceph-mon[36954]: pgmap v1965: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:30:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:28 smithi100 ceph-mon[32599]: pgmap v1966: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:30:28.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:28 smithi167 ceph-mon[37140]: pgmap v1966: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:30:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:28 smithi138 ceph-mon[36954]: pgmap v1966: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:30:30.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:30:30.676+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:30:30.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:30:30.677+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:30:30.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:30:30.677+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:30:30.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:30:30.677+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:30:30.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:30:30.677+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:30:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:30 smithi100 ceph-mon[32599]: pgmap v1967: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:30:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:30 smithi167 ceph-mon[37140]: pgmap v1967: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:30:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:30 smithi138 ceph-mon[36954]: pgmap v1967: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:30:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:32 smithi100 ceph-mon[32599]: pgmap v1968: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:30:32.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:32 smithi167 ceph-mon[37140]: pgmap v1968: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:30:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:32 smithi138 ceph-mon[36954]: pgmap v1968: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:30:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:34 smithi100 ceph-mon[32599]: pgmap v1969: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:30:34.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:34 smithi167 ceph-mon[37140]: pgmap v1969: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:30:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:34 smithi138 ceph-mon[36954]: pgmap v1969: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:30:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:36 smithi100 ceph-mon[32599]: pgmap v1970: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:30:36.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:36 smithi167 ceph-mon[37140]: pgmap v1970: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:30:36.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:36 smithi138 ceph-mon[36954]: pgmap v1970: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:30:38.742 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:38 smithi138 ceph-mon[36954]: pgmap v1971: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:30:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:38 smithi100 ceph-mon[32599]: pgmap v1971: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:30:38.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:38 smithi167 ceph-mon[37140]: pgmap v1971: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:30:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:40 smithi100 ceph-mon[32599]: pgmap v1972: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:30:40.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:40 smithi167 ceph-mon[37140]: pgmap v1972: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:30:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:40 smithi138 ceph-mon[36954]: pgmap v1972: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:30:42.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:42 smithi100 ceph-mon[32599]: pgmap v1973: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:30:42.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:42 smithi167 ceph-mon[37140]: pgmap v1973: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:30:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:42 smithi138 ceph-mon[36954]: pgmap v1973: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:30:43.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:43 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:30:43.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:43 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:30:43.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:43 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:30:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:44 smithi100 ceph-mon[32599]: pgmap v1974: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:30:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:30:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:30:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:30:44.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:30:44.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:30:44.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:30:44.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:30:44.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:30:44.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:30:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:44 smithi167 ceph-mon[37140]: pgmap v1974: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:30:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:30:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:30:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:30:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:30:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:30:45.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:30:45.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:30:45.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:30:45.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:30:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:44 smithi138 ceph-mon[36954]: pgmap v1974: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:30:45.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:30:45.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:30:45.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:30:45.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:30:45.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:30:45.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:30:45.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:30:45.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:30:45.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:30:45.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:30:45.692+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:30:45.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:30:45.693+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:30:45.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:30:45.693+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:30:45.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:30:45.693+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:30:45.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:30:45.693+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:30:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:46 smithi100 ceph-mon[32599]: pgmap v1975: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:30:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:46 smithi167 ceph-mon[37140]: pgmap v1975: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:30:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:46 smithi138 ceph-mon[36954]: pgmap v1975: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:30:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:48 smithi100 ceph-mon[32599]: pgmap v1976: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:30:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:48 smithi167 ceph-mon[37140]: pgmap v1976: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:30:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:48 smithi138 ceph-mon[36954]: pgmap v1976: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:30:50.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:50 smithi100 ceph-mon[32599]: pgmap v1977: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:30:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:50 smithi167 ceph-mon[37140]: pgmap v1977: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:30:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:50 smithi138 ceph-mon[36954]: pgmap v1977: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:30:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:52 smithi100 ceph-mon[32599]: pgmap v1978: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:30:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:52 smithi167 ceph-mon[37140]: pgmap v1978: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:30:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:52 smithi138 ceph-mon[36954]: pgmap v1978: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:30:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:54 smithi167 ceph-mon[37140]: pgmap v1979: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:30:55.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:54 smithi138 ceph-mon[36954]: pgmap v1979: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:30:55.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:54 smithi100 ceph-mon[32599]: pgmap v1979: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:30:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:56 smithi167 ceph-mon[37140]: pgmap v1980: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:30:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:56 smithi138 ceph-mon[36954]: pgmap v1980: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:30:57.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:56 smithi100 ceph-mon[32599]: pgmap v1980: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:30:59.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:30:58 smithi167 ceph-mon[37140]: pgmap v1981: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:30:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:30:58 smithi138 ceph-mon[36954]: pgmap v1981: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:30:59.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:30:58 smithi100 ceph-mon[32599]: pgmap v1981: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:31:00.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:31:00.708+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:31:00.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:31:00.709+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:31:00.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:31:00.709+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:31:00.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:31:00.709+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:31:00.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:31:00.709+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:31:00.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:00 smithi100 ceph-mon[32599]: pgmap v1982: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:31:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:00 smithi167 ceph-mon[37140]: pgmap v1982: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:31:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:00 smithi138 ceph-mon[36954]: pgmap v1982: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:31:03.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:02 smithi167 ceph-mon[37140]: pgmap v1983: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:31:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:02 smithi138 ceph-mon[36954]: pgmap v1983: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:31:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:02 smithi100 ceph-mon[32599]: pgmap v1983: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:31:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:04 smithi167 ceph-mon[37140]: pgmap v1984: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:31:05.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:04 smithi138 ceph-mon[36954]: pgmap v1984: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:31:05.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:04 smithi100 ceph-mon[32599]: pgmap v1984: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:31:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:06 smithi167 ceph-mon[37140]: pgmap v1985: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:31:07.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:06 smithi138 ceph-mon[36954]: pgmap v1985: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:31:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:06 smithi100 ceph-mon[32599]: pgmap v1985: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:31:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:08 smithi138 ceph-mon[36954]: pgmap v1986: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:31:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:08 smithi138 ceph-mon[36954]: osdmap e174: 12 total, 12 up, 12 in 2024-07-26T14:31:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:08 smithi100 ceph-mon[32599]: pgmap v1986: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:31:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:08 smithi100 ceph-mon[32599]: osdmap e174: 12 total, 12 up, 12 in 2024-07-26T14:31:09.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:08 smithi167 ceph-mon[37140]: pgmap v1986: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:31:09.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:08 smithi167 ceph-mon[37140]: osdmap e174: 12 total, 12 up, 12 in 2024-07-26T14:31:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:10 smithi138 ceph-mon[36954]: pgmap v1988: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:31:11.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:10 smithi100 ceph-mon[32599]: pgmap v1988: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:31:11.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:10 smithi167 ceph-mon[37140]: pgmap v1988: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:31:13.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:12 smithi167 ceph-mon[37140]: pgmap v1989: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:31:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:12 smithi138 ceph-mon[36954]: pgmap v1989: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:31:13.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:12 smithi100 ceph-mon[32599]: pgmap v1989: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:31:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:14 smithi138 ceph-mon[36954]: pgmap v1990: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:31:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:14 smithi138 ceph-mon[36954]: osdmap e175: 12 total, 12 up, 12 in 2024-07-26T14:31:15.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:14 smithi100 ceph-mon[32599]: pgmap v1990: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:31:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:14 smithi100 ceph-mon[32599]: osdmap e175: 12 total, 12 up, 12 in 2024-07-26T14:31:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:14 smithi167 ceph-mon[37140]: pgmap v1990: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:31:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:14 smithi167 ceph-mon[37140]: osdmap e175: 12 total, 12 up, 12 in 2024-07-26T14:31:15.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:31:15.724+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:31:15.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:31:15.725+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:31:15.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:31:15.725+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:31:15.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:31:15.725+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:31:15.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:31:15.725+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:31:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:16 smithi138 ceph-mon[36954]: pgmap v1992: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:31:17.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:16 smithi100 ceph-mon[32599]: pgmap v1992: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:31:17.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:16 smithi167 ceph-mon[37140]: pgmap v1992: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:31:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:18 smithi138 ceph-mon[36954]: pgmap v1993: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 212 B/s wr, 6 op/s 2024-07-26T14:31:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:18 smithi100 ceph-mon[32599]: pgmap v1993: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 212 B/s wr, 6 op/s 2024-07-26T14:31:19.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:18 smithi167 ceph-mon[37140]: pgmap v1993: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 212 B/s wr, 6 op/s 2024-07-26T14:31:21.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:20 smithi138 ceph-mon[36954]: pgmap v1994: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:31:21.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:20 smithi100 ceph-mon[32599]: pgmap v1994: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:31:21.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:20 smithi167 ceph-mon[37140]: pgmap v1994: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:31:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:22 smithi138 ceph-mon[36954]: pgmap v1995: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:31:23.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:22 smithi100 ceph-mon[32599]: pgmap v1995: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:31:23.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:22 smithi167 ceph-mon[37140]: pgmap v1995: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:31:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:24 smithi138 ceph-mon[36954]: pgmap v1996: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:31:25.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:24 smithi100 ceph-mon[32599]: pgmap v1996: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:31:25.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:24 smithi167 ceph-mon[37140]: pgmap v1996: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:31:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:26 smithi138 ceph-mon[36954]: pgmap v1997: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:31:27.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:26 smithi100 ceph-mon[32599]: pgmap v1997: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:31:27.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:26 smithi167 ceph-mon[37140]: pgmap v1997: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:31:29.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:28 smithi138 ceph-mon[36954]: pgmap v1998: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:31:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:28 smithi100 ceph-mon[32599]: pgmap v1998: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:31:29.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:28 smithi167 ceph-mon[37140]: pgmap v1998: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:31:30.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:31:30.734+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:31:30.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:31:30.734+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:31:30.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:31:30.734+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:31:30.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:31:30.734+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:31:30.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:31:30.734+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:31:30.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:30 smithi100 ceph-mon[32599]: pgmap v1999: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:31:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:30 smithi138 ceph-mon[36954]: pgmap v1999: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:31:31.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:30 smithi167 ceph-mon[37140]: pgmap v1999: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:31:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:32 smithi138 ceph-mon[36954]: pgmap v2000: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:31:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:32 smithi100 ceph-mon[32599]: pgmap v2000: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:31:33.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:32 smithi167 ceph-mon[37140]: pgmap v2000: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:31:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:34 smithi138 ceph-mon[36954]: pgmap v2001: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:31:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:34 smithi100 ceph-mon[32599]: pgmap v2001: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:31:35.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:34 smithi167 ceph-mon[37140]: pgmap v2001: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:31:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:36 smithi138 ceph-mon[36954]: pgmap v2002: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:31:37.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:36 smithi100 ceph-mon[32599]: pgmap v2002: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:31:37.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:36 smithi167 ceph-mon[37140]: pgmap v2002: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:31:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:38 smithi138 ceph-mon[36954]: pgmap v2003: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:31:39.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:38 smithi100 ceph-mon[32599]: pgmap v2003: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:31:39.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:38 smithi167 ceph-mon[37140]: pgmap v2003: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:31:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:40 smithi138 ceph-mon[36954]: pgmap v2004: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:31:41.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:40 smithi100 ceph-mon[32599]: pgmap v2004: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:31:41.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:40 smithi167 ceph-mon[37140]: pgmap v2004: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:31:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:42 smithi138 ceph-mon[36954]: pgmap v2005: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:31:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:42 smithi100 ceph-mon[32599]: pgmap v2005: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:31:43.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:42 smithi167 ceph-mon[37140]: pgmap v2005: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:31:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:44 smithi138 ceph-mon[36954]: pgmap v2006: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:31:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:31:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:44 smithi100 ceph-mon[32599]: pgmap v2006: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:31:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:31:45.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:44 smithi167 ceph-mon[37140]: pgmap v2006: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:31:45.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:31:45.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:31:45.746+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:31:45.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:31:45.746+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:31:45.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:31:45.746+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:31:45.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:31:45.746+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:31:45.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:31:45.747+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:31:46.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:31:46.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:31:46.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:31:46.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:31:46.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:31:46.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:31:46.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:31:46.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:31:46.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:31:46.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:31:46.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:31:46.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:31:46.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:31:46.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:31:46.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:31:46.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:31:46.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:31:46.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:31:46.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:31:46.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:31:46.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:31:46.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:31:46.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:31:46.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:31:46.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:31:46.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:31:46.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:31:47.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:47 smithi167 ceph-mon[37140]: pgmap v2007: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:31:47.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:47 smithi138 ceph-mon[36954]: pgmap v2007: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:31:47.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:47 smithi100 ceph-mon[32599]: pgmap v2007: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:31:49.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:49 smithi167 ceph-mon[37140]: pgmap v2008: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:31:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:49 smithi138 ceph-mon[36954]: pgmap v2008: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:31:49.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:49 smithi100 ceph-mon[32599]: pgmap v2008: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:31:51.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:51 smithi167 ceph-mon[37140]: pgmap v2009: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:31:51.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:51 smithi138 ceph-mon[36954]: pgmap v2009: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:31:51.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:51 smithi100 ceph-mon[32599]: pgmap v2009: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:31:53.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:53 smithi167 ceph-mon[37140]: pgmap v2010: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:31:53.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:53 smithi138 ceph-mon[36954]: pgmap v2010: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:31:53.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:53 smithi100 ceph-mon[32599]: pgmap v2010: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:31:55.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:55 smithi167 ceph-mon[37140]: pgmap v2011: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:31:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:55 smithi138 ceph-mon[36954]: pgmap v2011: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:31:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:55 smithi100 ceph-mon[32599]: pgmap v2011: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:31:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:57 smithi138 ceph-mon[36954]: pgmap v2012: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:31:57.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:57 smithi100 ceph-mon[32599]: pgmap v2012: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:31:57.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:57 smithi167 ceph-mon[37140]: pgmap v2012: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:31:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:31:59 smithi138 ceph-mon[36954]: pgmap v2013: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:31:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:31:59 smithi100 ceph-mon[32599]: pgmap v2013: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:31:59.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:31:59 smithi167 ceph-mon[37140]: pgmap v2013: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:32:00.395 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:32:00 smithi100 ceph-mon[32599]: pgmap v2014: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:32:00.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:32:00 smithi138 ceph-mon[36954]: pgmap v2014: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:32:00.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:32:00 smithi167 ceph-mon[37140]: pgmap v2014: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:32:00.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:32:00.762+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:32:00.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:32:00.762+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:32:00.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:32:00.762+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:32:00.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:32:00.762+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:32:00.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:32:00.762+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:32:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:32:02 smithi100 ceph-mon[32599]: pgmap v2015: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:32:02.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:32:02 smithi167 ceph-mon[37140]: pgmap v2015: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:32:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:32:02 smithi138 ceph-mon[36954]: pgmap v2015: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:32:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:32:04 smithi100 ceph-mon[32599]: pgmap v2016: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:32:04.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:32:04 smithi167 ceph-mon[37140]: pgmap v2016: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:32:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:32:04 smithi138 ceph-mon[36954]: pgmap v2016: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:32:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:32:06 smithi100 ceph-mon[32599]: pgmap v2017: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:32:06.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:32:06 smithi167 ceph-mon[37140]: pgmap v2017: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:32:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:32:06 smithi138 ceph-mon[36954]: pgmap v2017: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:32:08.742 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:32:08 smithi138 ceph-mon[36954]: pgmap v2018: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:32:08.743 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:32:08 smithi138 ceph-mon[36954]: osdmap e176: 12 total, 12 up, 12 in 2024-07-26T14:32:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:32:08 smithi100 ceph-mon[32599]: pgmap v2018: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:32:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:32:08 smithi100 ceph-mon[32599]: osdmap e176: 12 total, 12 up, 12 in 2024-07-26T14:32:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:32:08 smithi167 ceph-mon[37140]: pgmap v2018: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:32:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:32:08 smithi167 ceph-mon[37140]: osdmap e176: 12 total, 12 up, 12 in 2024-07-26T14:32:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:32:10 smithi100 ceph-mon[32599]: pgmap v2020: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:32:10.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:32:10 smithi167 ceph-mon[37140]: pgmap v2020: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:32:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:32:10 smithi138 ceph-mon[36954]: pgmap v2020: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:32:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:32:12 smithi100 ceph-mon[32599]: pgmap v2021: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:32:12.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:32:12 smithi167 ceph-mon[37140]: pgmap v2021: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:32:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:32:12 smithi138 ceph-mon[36954]: pgmap v2021: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:32:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:32:14 smithi100 ceph-mon[32599]: pgmap v2022: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:32:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:32:14 smithi100 ceph-mon[32599]: osdmap e177: 12 total, 12 up, 12 in 2024-07-26T14:32:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:32:14 smithi167 ceph-mon[37140]: pgmap v2022: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:32:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:32:14 smithi167 ceph-mon[37140]: osdmap e177: 12 total, 12 up, 12 in 2024-07-26T14:32:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:32:14 smithi138 ceph-mon[36954]: pgmap v2022: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:32:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:32:14 smithi138 ceph-mon[36954]: osdmap e177: 12 total, 12 up, 12 in 2024-07-26T14:32:15.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:32:15.778+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:32:15.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:32:15.778+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:32:15.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:32:15.778+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:32:15.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:32:15.778+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:32:15.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:32:15.778+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:32:16.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:32:16 smithi167 ceph-mon[37140]: pgmap v2024: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:32:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:32:16 smithi138 ceph-mon[36954]: pgmap v2024: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:32:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:32:16 smithi100 ceph-mon[32599]: pgmap v2024: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:32:18.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:32:18 smithi167 ceph-mon[37140]: pgmap v2025: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T14:32:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:32:18 smithi138 ceph-mon[36954]: pgmap v2025: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T14:32:18.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:32:18 smithi100 ceph-mon[32599]: pgmap v2025: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T14:32:20.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:32:20 smithi167 ceph-mon[37140]: pgmap v2026: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:32:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:32:20 smithi138 ceph-mon[36954]: pgmap v2026: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:32:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:32:20 smithi100 ceph-mon[32599]: pgmap v2026: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:32:22.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:32:22 smithi167 ceph-mon[37140]: pgmap v2027: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:32:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:32:22 smithi138 ceph-mon[36954]: pgmap v2027: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:32:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:32:22 smithi100 ceph-mon[32599]: pgmap v2027: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:32:24.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:32:24 smithi167 ceph-mon[37140]: pgmap v2028: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:32:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:32:24 smithi138 ceph-mon[36954]: pgmap v2028: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:32:24.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:32:24 smithi100 ceph-mon[32599]: pgmap v2028: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:32:26.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:32:26 smithi167 ceph-mon[37140]: pgmap v2029: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T14:32:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:32:26 smithi138 ceph-mon[36954]: pgmap v2029: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T14:32:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:32:26 smithi100 ceph-mon[32599]: pgmap v2029: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T14:32:28.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:32:28 smithi167 ceph-mon[37140]: pgmap v2030: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:32:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:32:28 smithi138 ceph-mon[36954]: pgmap v2030: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:32:28.998 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:32:28 smithi100 ceph-mon[32599]: pgmap v2030: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:32:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:32:30 smithi138 ceph-mon[36954]: pgmap v2031: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:32:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:32:30 smithi100 ceph-mon[32599]: pgmap v2031: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:32:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:32:30 smithi167 ceph-mon[37140]: pgmap v2031: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:32:30.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:32:30.794+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:32:30.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:32:30.794+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:32:30.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:32:30.794+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:32:30.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:32:30.794+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:32:30.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:32:30.794+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:32:32.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:32:32 smithi100 ceph-mon[32599]: pgmap v2032: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:32:32.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:32:32 smithi167 ceph-mon[37140]: pgmap v2032: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:32:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:32:32 smithi138 ceph-mon[36954]: pgmap v2032: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:32:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:32:34 smithi100 ceph-mon[32599]: pgmap v2033: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:32:34.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:32:34 smithi167 ceph-mon[37140]: pgmap v2033: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:32:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:32:34 smithi138 ceph-mon[36954]: pgmap v2033: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:32:36.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:32:36 smithi100 ceph-mon[32599]: pgmap v2034: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:32:36.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:32:36 smithi167 ceph-mon[37140]: pgmap v2034: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:32:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:32:36 smithi138 ceph-mon[36954]: pgmap v2034: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:32:38.742 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:32:38 smithi138 ceph-mon[36954]: pgmap v2035: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:32:38.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:32:38 smithi100 ceph-mon[32599]: pgmap v2035: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:32:38.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:32:38 smithi167 ceph-mon[37140]: pgmap v2035: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:32:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:32:40 smithi100 ceph-mon[32599]: pgmap v2036: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:32:40.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:32:40 smithi167 ceph-mon[37140]: pgmap v2036: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:32:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:32:40 smithi138 ceph-mon[36954]: pgmap v2036: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:32:42.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:32:42 smithi167 ceph-mon[37140]: pgmap v2037: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:32:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:32:42 smithi138 ceph-mon[36954]: pgmap v2037: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:32:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:32:42 smithi100 ceph-mon[32599]: pgmap v2037: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:32:44.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:32:44 smithi167 ceph-mon[37140]: pgmap v2038: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:32:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:32:44 smithi138 ceph-mon[36954]: pgmap v2038: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:32:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:32:44 smithi100 ceph-mon[32599]: pgmap v2038: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:32:45.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:32:45 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:32:45.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:32:45.810+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:32:45.811 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:32:45 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:32:45.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:32:45.810+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:32:45.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:32:45.810+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:32:45.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:32:45.810+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:32:45.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:32:45.810+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:32:45.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:32:45 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:32:46.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:32:46 smithi167 ceph-mon[37140]: pgmap v2039: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:32:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:32:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:32:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:32:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:32:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:32:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:32:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:32:46 smithi138 ceph-mon[36954]: pgmap v2039: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:32:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:32:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:32:46.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:32:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:32:46.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:32:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:32:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:32:46 smithi100 ceph-mon[32599]: pgmap v2039: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:32:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:32:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:32:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:32:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:32:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:32:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:32:48.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:32:48 smithi167 ceph-mon[37140]: pgmap v2040: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:32:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:32:48 smithi138 ceph-mon[36954]: pgmap v2040: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:32:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:32:48 smithi100 ceph-mon[32599]: pgmap v2040: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:32:50.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:32:50 smithi167 ceph-mon[37140]: pgmap v2041: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:32:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:32:50 smithi138 ceph-mon[36954]: pgmap v2041: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:32:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:32:50 smithi100 ceph-mon[32599]: pgmap v2041: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:32:52.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:32:52 smithi167 ceph-mon[37140]: pgmap v2042: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:32:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:32:52 smithi138 ceph-mon[36954]: pgmap v2042: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:32:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:32:52 smithi100 ceph-mon[32599]: pgmap v2042: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:32:54.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:32:54 smithi167 ceph-mon[37140]: pgmap v2043: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:32:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:32:54 smithi138 ceph-mon[36954]: pgmap v2043: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:32:54.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:32:54 smithi100 ceph-mon[32599]: pgmap v2043: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:32:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:32:56 smithi138 ceph-mon[36954]: pgmap v2044: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:32:56.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:32:56 smithi100 ceph-mon[32599]: pgmap v2044: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:32:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:32:56 smithi167 ceph-mon[37140]: pgmap v2044: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:32:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:32:58 smithi138 ceph-mon[36954]: pgmap v2045: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:32:58.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:32:58 smithi100 ceph-mon[32599]: pgmap v2045: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:32:59.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:32:58 smithi167 ceph-mon[37140]: pgmap v2045: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:33:00.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:33:00 smithi100 ceph-mon[32599]: pgmap v2046: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:33:00.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:33:00.826+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:33:00.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:33:00.826+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:33:00.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:33:00.826+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:33:00.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:33:00.826+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:33:00.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:33:00.826+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:33:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:33:00 smithi138 ceph-mon[36954]: pgmap v2046: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:33:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:33:00 smithi167 ceph-mon[37140]: pgmap v2046: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:33:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:33:02 smithi138 ceph-mon[36954]: pgmap v2047: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:33:02.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:33:02 smithi100 ceph-mon[32599]: pgmap v2047: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:33:03.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:33:02 smithi167 ceph-mon[37140]: pgmap v2047: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:33:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:33:03.028+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b3d40 msgr2=0x7fe5940c09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-07-26T14:33:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:33:03.028+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b3d40 msgr2=0x7fe5940c09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-07-26T14:33:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:33:03.028+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b3d40 0x7fe5940c09c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe590005d00 tx=0x7fe59005a040 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:33:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:33:03.028+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b3d40 0x7fe5940c09c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe590005d00 tx=0x7fe59005a040 comp rx=0 tx=0).stop 2024-07-26T14:33:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:33:03.028+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b3d40 msgr2=0x7fe5940c09c0 unknown :-1 s=STATE_CLOSED l=1).tick see no progress in more than 10000000 us during connecting to v2:172.21.15.100:6800/3937009534, fault. 2024-07-26T14:33:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:33:03.028+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b3d40 msgr2=0x7fe5940c09c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-07-26T14:33:03.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:33:03.028+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b3d40 0x7fe5940c09c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T14:33:03.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:33:03.028+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T14:33:03.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:33:03.028+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:33:03.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:33:03.028+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe59000b6b0 tx=0x7fe59002f040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T14:33:03.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:33:03.393+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c006610 con 0x7fe5a81087a0 2024-07-26T14:33:03.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:33:03.393+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T14:33:03.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:33:03.394+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 25 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T14:33:03.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:33:03.394+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 25 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c020 con 0x7fe5a81087a0 2024-07-26T14:33:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:33:04 smithi138 ceph-mon[36954]: pgmap v2048: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:33:04.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:33:04 smithi100 ceph-mon[32599]: pgmap v2048: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:33:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:33:04 smithi167 ceph-mon[37140]: pgmap v2048: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:33:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:33:06 smithi138 ceph-mon[36954]: pgmap v2049: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:33:06.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:33:06 smithi100 ceph-mon[32599]: pgmap v2049: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:33:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:33:06 smithi167 ceph-mon[37140]: pgmap v2049: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:33:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:33:08 smithi138 ceph-mon[36954]: pgmap v2050: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:33:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:33:08 smithi138 ceph-mon[36954]: osdmap e178: 12 total, 12 up, 12 in 2024-07-26T14:33:08.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:33:08 smithi100 ceph-mon[32599]: pgmap v2050: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:33:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:33:08 smithi100 ceph-mon[32599]: osdmap e178: 12 total, 12 up, 12 in 2024-07-26T14:33:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:33:08 smithi167 ceph-mon[37140]: pgmap v2050: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:33:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:33:08 smithi167 ceph-mon[37140]: osdmap e178: 12 total, 12 up, 12 in 2024-07-26T14:33:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:33:10 smithi138 ceph-mon[36954]: pgmap v2052: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:33:10.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:33:10 smithi100 ceph-mon[32599]: pgmap v2052: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:33:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:33:10 smithi167 ceph-mon[37140]: pgmap v2052: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:33:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:33:12 smithi138 ceph-mon[36954]: pgmap v2053: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:33:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:33:12 smithi100 ceph-mon[32599]: pgmap v2053: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:33:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:33:12 smithi167 ceph-mon[37140]: pgmap v2053: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:33:14.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:33:14 smithi167 ceph-mon[37140]: pgmap v2054: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:33:14.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:33:14 smithi167 ceph-mon[37140]: osdmap e179: 12 total, 12 up, 12 in 2024-07-26T14:33:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:33:14 smithi138 ceph-mon[36954]: pgmap v2054: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:33:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:33:14 smithi138 ceph-mon[36954]: osdmap e179: 12 total, 12 up, 12 in 2024-07-26T14:33:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:33:14 smithi100 ceph-mon[32599]: pgmap v2054: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:33:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:33:14 smithi100 ceph-mon[32599]: osdmap e179: 12 total, 12 up, 12 in 2024-07-26T14:33:15.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:33:15.830+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:33:15.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:33:15.831+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:33:15.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:33:15.831+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:33:15.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:33:15.831+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:33:15.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:33:15.831+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:33:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:33:16 smithi138 ceph-mon[36954]: pgmap v2056: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:33:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:33:16 smithi100 ceph-mon[32599]: pgmap v2056: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:33:17.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:33:16 smithi167 ceph-mon[37140]: pgmap v2056: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:33:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:33:18 smithi138 ceph-mon[36954]: pgmap v2057: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T14:33:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:33:18 smithi100 ceph-mon[32599]: pgmap v2057: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T14:33:19.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:33:18 smithi167 ceph-mon[37140]: pgmap v2057: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T14:33:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:33:20 smithi138 ceph-mon[36954]: pgmap v2058: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:33:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:33:20 smithi100 ceph-mon[32599]: pgmap v2058: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:33:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:33:20 smithi167 ceph-mon[37140]: pgmap v2058: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:33:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:33:22 smithi138 ceph-mon[36954]: pgmap v2059: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:33:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:33:22 smithi100 ceph-mon[32599]: pgmap v2059: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:33:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:33:22 smithi167 ceph-mon[37140]: pgmap v2059: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:33:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:33:24 smithi138 ceph-mon[36954]: pgmap v2060: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:33:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:33:24 smithi100 ceph-mon[32599]: pgmap v2060: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:33:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:33:24 smithi167 ceph-mon[37140]: pgmap v2060: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:33:26.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:33:26 smithi138 ceph-mon[36954]: pgmap v2061: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:33:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:33:26 smithi100 ceph-mon[32599]: pgmap v2061: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:33:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:33:26 smithi167 ceph-mon[37140]: pgmap v2061: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:33:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:33:28 smithi138 ceph-mon[36954]: pgmap v2062: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:33:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:33:28 smithi100 ceph-mon[32599]: pgmap v2062: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:33:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:33:28 smithi167 ceph-mon[37140]: pgmap v2062: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:33:30.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:33:30.846+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:33:30.848 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:33:30 smithi100 ceph-mon[32599]: pgmap v2063: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:33:30.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:33:30.846+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:33:30.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:33:30.846+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:33:30.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:33:30.846+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:33:30.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:33:30.846+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:33:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:33:30 smithi138 ceph-mon[36954]: pgmap v2063: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:33:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:33:30 smithi167 ceph-mon[37140]: pgmap v2063: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:33:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:33:32 smithi138 ceph-mon[36954]: pgmap v2064: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:33:32.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:33:32 smithi100 ceph-mon[32599]: pgmap v2064: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:33:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:33:32 smithi167 ceph-mon[37140]: pgmap v2064: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:33:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:33:34 smithi138 ceph-mon[36954]: pgmap v2065: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:33:34.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:33:34 smithi100 ceph-mon[32599]: pgmap v2065: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:33:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:33:34 smithi167 ceph-mon[37140]: pgmap v2065: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:33:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:33:36 smithi138 ceph-mon[36954]: pgmap v2066: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:33:36.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:33:36 smithi100 ceph-mon[32599]: pgmap v2066: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:33:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:33:36 smithi167 ceph-mon[37140]: pgmap v2066: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:33:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:33:38 smithi138 ceph-mon[36954]: pgmap v2067: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:33:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:33:38 smithi100 ceph-mon[32599]: pgmap v2067: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:33:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:33:38 smithi167 ceph-mon[37140]: pgmap v2067: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:33:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:33:40 smithi138 ceph-mon[36954]: pgmap v2068: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:33:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:33:40 smithi100 ceph-mon[32599]: pgmap v2068: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:33:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:33:40 smithi167 ceph-mon[37140]: pgmap v2068: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:33:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:33:42 smithi138 ceph-mon[36954]: pgmap v2069: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:33:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:33:42 smithi100 ceph-mon[32599]: pgmap v2069: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:33:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:33:42 smithi167 ceph-mon[37140]: pgmap v2069: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:33:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:33:44 smithi138 ceph-mon[36954]: pgmap v2070: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:33:44.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:33:44 smithi100 ceph-mon[32599]: pgmap v2070: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:33:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:33:44 smithi167 ceph-mon[37140]: pgmap v2070: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:33:45.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:33:45.855+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:33:45.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:33:45.855+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:33:45.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:33:45.855+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:33:45.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:33:45.855+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:33:45.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:33:45.855+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:33:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:33:46 smithi138 ceph-mon[36954]: pgmap v2071: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:33:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:33:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:33:46.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:33:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:33:46.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:33:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:33:46.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:33:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:33:46.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:33:46 smithi100 ceph-mon[32599]: pgmap v2071: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:33:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:33:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:33:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:33:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:33:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:33:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:33:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:33:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:33:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:33:46 smithi167 ceph-mon[37140]: pgmap v2071: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:33:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:33:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:33:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:33:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:33:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:33:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:33:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:33:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:33:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:33:48 smithi138 ceph-mon[36954]: pgmap v2072: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:33:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:33:48 smithi100 ceph-mon[32599]: pgmap v2072: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:33:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:33:48 smithi167 ceph-mon[37140]: pgmap v2072: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:33:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:33:50 smithi138 ceph-mon[36954]: pgmap v2073: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:33:50.999 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:33:50 smithi100 ceph-mon[32599]: pgmap v2073: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:33:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:33:50 smithi167 ceph-mon[37140]: pgmap v2073: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:33:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:33:52 smithi138 ceph-mon[36954]: pgmap v2074: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:33:52.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:33:52 smithi100 ceph-mon[32599]: pgmap v2074: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:33:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:33:52 smithi167 ceph-mon[37140]: pgmap v2074: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:33:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:33:54 smithi138 ceph-mon[36954]: pgmap v2075: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:33:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:33:54 smithi100 ceph-mon[32599]: pgmap v2075: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:33:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:33:54 smithi167 ceph-mon[37140]: pgmap v2075: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:33:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:33:56 smithi138 ceph-mon[36954]: pgmap v2076: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:33:56.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:33:56 smithi100 ceph-mon[32599]: pgmap v2076: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:33:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:33:56 smithi167 ceph-mon[37140]: pgmap v2076: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:33:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:33:58 smithi138 ceph-mon[36954]: pgmap v2077: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:33:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:33:58 smithi100 ceph-mon[32599]: pgmap v2077: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:33:59.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:33:58 smithi167 ceph-mon[37140]: pgmap v2077: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:34:00.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:34:00.869+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:34:00.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:34:00.870+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:34:00.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:34:00.870+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:34:00.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:34:00.870+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:34:00.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:34:00.870+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:34:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:34:00 smithi138 ceph-mon[36954]: pgmap v2078: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:34:00.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:34:00 smithi100 ceph-mon[32599]: pgmap v2078: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:34:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:34:00 smithi167 ceph-mon[37140]: pgmap v2078: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:34:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:34:02 smithi138 ceph-mon[36954]: pgmap v2079: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:34:02.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:34:02 smithi100 ceph-mon[32599]: pgmap v2079: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:34:03.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:34:02 smithi167 ceph-mon[37140]: pgmap v2079: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:34:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:34:04 smithi138 ceph-mon[36954]: pgmap v2080: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:34:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:34:04 smithi100 ceph-mon[32599]: pgmap v2080: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:34:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:34:04 smithi167 ceph-mon[37140]: pgmap v2080: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:34:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:34:06 smithi138 ceph-mon[36954]: pgmap v2081: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:34:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:34:06 smithi100 ceph-mon[32599]: pgmap v2081: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:34:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:34:06 smithi167 ceph-mon[37140]: pgmap v2081: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:34:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:34:08 smithi138 ceph-mon[36954]: pgmap v2082: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:34:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:34:08 smithi138 ceph-mon[36954]: osdmap e180: 12 total, 12 up, 12 in 2024-07-26T14:34:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:34:08 smithi100 ceph-mon[32599]: pgmap v2082: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:34:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:34:08 smithi100 ceph-mon[32599]: osdmap e180: 12 total, 12 up, 12 in 2024-07-26T14:34:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:34:08 smithi167 ceph-mon[37140]: pgmap v2082: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:34:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:34:08 smithi167 ceph-mon[37140]: osdmap e180: 12 total, 12 up, 12 in 2024-07-26T14:34:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:34:10 smithi138 ceph-mon[36954]: pgmap v2084: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:34:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:34:10 smithi100 ceph-mon[32599]: pgmap v2084: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:34:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:34:10 smithi167 ceph-mon[37140]: pgmap v2084: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:34:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:34:12 smithi138 ceph-mon[36954]: pgmap v2085: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:34:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:34:12 smithi100 ceph-mon[32599]: pgmap v2085: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:34:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:34:12 smithi167 ceph-mon[37140]: pgmap v2085: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:34:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:34:14 smithi138 ceph-mon[36954]: pgmap v2086: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:34:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:34:14 smithi138 ceph-mon[36954]: osdmap e181: 12 total, 12 up, 12 in 2024-07-26T14:34:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:34:14 smithi100 ceph-mon[32599]: pgmap v2086: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:34:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:34:14 smithi100 ceph-mon[32599]: osdmap e181: 12 total, 12 up, 12 in 2024-07-26T14:34:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:34:14 smithi167 ceph-mon[37140]: pgmap v2086: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:34:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:34:14 smithi167 ceph-mon[37140]: osdmap e181: 12 total, 12 up, 12 in 2024-07-26T14:34:15.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:34:15.885+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:34:15.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:34:15.886+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:34:15.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:34:15.886+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:34:15.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:34:15.886+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:34:15.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:34:15.886+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:34:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:34:16 smithi138 ceph-mon[36954]: pgmap v2088: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:34:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:34:16 smithi100 ceph-mon[32599]: pgmap v2088: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:34:17.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:34:16 smithi167 ceph-mon[37140]: pgmap v2088: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:34:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:34:18 smithi138 ceph-mon[36954]: pgmap v2089: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 208 B/s wr, 6 op/s 2024-07-26T14:34:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:34:18 smithi100 ceph-mon[32599]: pgmap v2089: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 208 B/s wr, 6 op/s 2024-07-26T14:34:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:34:18 smithi167 ceph-mon[37140]: pgmap v2089: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 208 B/s wr, 6 op/s 2024-07-26T14:34:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:34:20 smithi138 ceph-mon[36954]: pgmap v2090: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:34:20.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:34:20 smithi100 ceph-mon[32599]: pgmap v2090: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:34:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:34:20 smithi167 ceph-mon[37140]: pgmap v2090: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:34:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:34:22 smithi138 ceph-mon[36954]: pgmap v2091: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:34:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:34:22 smithi100 ceph-mon[32599]: pgmap v2091: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:34:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:34:22 smithi167 ceph-mon[37140]: pgmap v2091: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:34:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:34:24 smithi138 ceph-mon[36954]: pgmap v2092: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:34:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:34:24 smithi100 ceph-mon[32599]: pgmap v2092: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:34:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:34:24 smithi167 ceph-mon[37140]: pgmap v2092: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:34:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:34:26 smithi100 ceph-mon[32599]: pgmap v2093: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:34:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:34:26 smithi167 ceph-mon[37140]: pgmap v2093: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:34:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:34:26 smithi138 ceph-mon[36954]: pgmap v2093: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:34:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:34:28 smithi100 ceph-mon[32599]: pgmap v2094: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:34:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:34:28 smithi167 ceph-mon[37140]: pgmap v2094: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:34:29.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:34:28 smithi138 ceph-mon[36954]: pgmap v2094: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:34:30.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:34:30.901+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:34:30.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:34:30.902+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:34:30.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:34:30.902+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:34:30.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:34:30.902+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:34:30.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:34:30.902+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:34:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:34:30 smithi100 ceph-mon[32599]: pgmap v2095: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:34:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:34:30 smithi167 ceph-mon[37140]: pgmap v2095: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:34:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:34:30 smithi138 ceph-mon[36954]: pgmap v2095: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:34:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:34:32 smithi100 ceph-mon[32599]: pgmap v2096: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:34:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:34:32 smithi167 ceph-mon[37140]: pgmap v2096: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:34:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:34:32 smithi138 ceph-mon[36954]: pgmap v2096: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:34:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:34:34 smithi100 ceph-mon[32599]: pgmap v2097: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:34:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:34:34 smithi167 ceph-mon[37140]: pgmap v2097: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:34:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:34:34 smithi138 ceph-mon[36954]: pgmap v2097: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:34:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:34:36 smithi100 ceph-mon[32599]: pgmap v2098: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:34:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:34:36 smithi167 ceph-mon[37140]: pgmap v2098: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:34:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:34:36 smithi138 ceph-mon[36954]: pgmap v2098: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:34:38.992 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:34:38 smithi138 ceph-mon[36954]: pgmap v2099: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:34:38.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:34:38 smithi100 ceph-mon[32599]: pgmap v2099: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:34:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:34:38 smithi167 ceph-mon[37140]: pgmap v2099: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:34:40.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:34:40 smithi100 ceph-mon[32599]: pgmap v2100: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:34:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:34:40 smithi167 ceph-mon[37140]: pgmap v2100: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:34:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:34:40 smithi138 ceph-mon[36954]: pgmap v2100: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:34:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:34:42 smithi100 ceph-mon[32599]: pgmap v2101: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:34:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:34:42 smithi167 ceph-mon[37140]: pgmap v2101: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:34:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:34:42 smithi138 ceph-mon[36954]: pgmap v2101: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:34:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:34:44 smithi100 ceph-mon[32599]: pgmap v2102: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:34:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:34:44 smithi167 ceph-mon[37140]: pgmap v2102: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:34:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:34:44 smithi138 ceph-mon[36954]: pgmap v2102: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:34:45.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:34:45.910+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:34:45.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:34:45.911+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:34:45.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:34:45.911+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:34:45.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:34:45.911+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:34:45.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:34:45.911+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:34:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:34:46 smithi100 ceph-mon[32599]: pgmap v2103: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:34:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:34:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:34:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:34:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:34:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:34:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:34:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:34:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:34:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:34:46 smithi167 ceph-mon[37140]: pgmap v2103: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:34:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:34:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:34:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:34:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:34:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:34:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:34:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:34:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:34:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:34:46 smithi138 ceph-mon[36954]: pgmap v2103: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:34:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:34:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:34:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:34:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:34:47.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:34:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:34:47.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:34:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:34:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:34:48 smithi100 ceph-mon[32599]: pgmap v2104: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:34:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:34:48 smithi167 ceph-mon[37140]: pgmap v2104: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:34:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:34:48 smithi138 ceph-mon[36954]: pgmap v2104: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:34:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:34:50 smithi100 ceph-mon[32599]: pgmap v2105: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:34:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:34:50 smithi167 ceph-mon[37140]: pgmap v2105: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:34:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:34:50 smithi138 ceph-mon[36954]: pgmap v2105: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:34:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:34:52 smithi100 ceph-mon[32599]: pgmap v2106: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:34:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:34:52 smithi167 ceph-mon[37140]: pgmap v2106: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:34:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:34:52 smithi138 ceph-mon[36954]: pgmap v2106: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:34:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:34:54 smithi100 ceph-mon[32599]: pgmap v2107: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:34:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:34:54 smithi167 ceph-mon[37140]: pgmap v2107: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:34:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:34:54 smithi138 ceph-mon[36954]: pgmap v2107: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:34:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:34:56 smithi100 ceph-mon[32599]: pgmap v2108: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:34:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:34:56 smithi167 ceph-mon[37140]: pgmap v2108: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:34:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:34:56 smithi138 ceph-mon[36954]: pgmap v2108: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:34:59.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:34:58 smithi167 ceph-mon[37140]: pgmap v2109: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:34:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:34:58 smithi138 ceph-mon[36954]: pgmap v2109: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:34:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:34:58 smithi100 ceph-mon[32599]: pgmap v2109: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:35:00.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:35:00.927+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:35:00.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:35:00.928+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:35:00.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:35:00.928+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:35:00.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:35:00.928+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:35:00.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:35:00.928+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:35:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:35:00 smithi167 ceph-mon[37140]: pgmap v2110: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:35:01.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:35:00 smithi100 ceph-mon[32599]: pgmap v2110: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:35:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:35:00 smithi138 ceph-mon[36954]: pgmap v2110: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:35:03.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:35:02 smithi167 ceph-mon[37140]: pgmap v2111: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:35:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:35:02 smithi138 ceph-mon[36954]: pgmap v2111: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:35:03.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:35:02 smithi100 ceph-mon[32599]: pgmap v2111: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:35:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:35:04 smithi167 ceph-mon[37140]: pgmap v2112: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:35:05.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:35:04 smithi138 ceph-mon[36954]: pgmap v2112: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:35:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:35:04 smithi100 ceph-mon[32599]: pgmap v2112: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:35:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:35:06 smithi167 ceph-mon[37140]: pgmap v2113: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:35:07.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:35:06 smithi138 ceph-mon[36954]: pgmap v2113: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:35:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:35:06 smithi100 ceph-mon[32599]: pgmap v2113: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:35:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:35:08 smithi138 ceph-mon[36954]: pgmap v2114: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:35:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:35:08 smithi138 ceph-mon[36954]: osdmap e182: 12 total, 12 up, 12 in 2024-07-26T14:35:09.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:35:08 smithi100 ceph-mon[32599]: pgmap v2114: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:35:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:35:08 smithi100 ceph-mon[32599]: osdmap e182: 12 total, 12 up, 12 in 2024-07-26T14:35:09.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:35:08 smithi167 ceph-mon[37140]: pgmap v2114: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:35:09.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:35:08 smithi167 ceph-mon[37140]: osdmap e182: 12 total, 12 up, 12 in 2024-07-26T14:35:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:35:10 smithi138 ceph-mon[36954]: pgmap v2116: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:35:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:35:10 smithi100 ceph-mon[32599]: pgmap v2116: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:35:11.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:35:10 smithi167 ceph-mon[37140]: pgmap v2116: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:35:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:35:12 smithi138 ceph-mon[36954]: pgmap v2117: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:35:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:35:12 smithi100 ceph-mon[32599]: pgmap v2117: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:35:13.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:35:12 smithi167 ceph-mon[37140]: pgmap v2117: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:35:15.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:35:14 smithi167 ceph-mon[37140]: pgmap v2118: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:35:15.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:35:14 smithi167 ceph-mon[37140]: osdmap e183: 12 total, 12 up, 12 in 2024-07-26T14:35:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:35:14 smithi138 ceph-mon[36954]: pgmap v2118: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:35:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:35:14 smithi138 ceph-mon[36954]: osdmap e183: 12 total, 12 up, 12 in 2024-07-26T14:35:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:35:14 smithi100 ceph-mon[32599]: pgmap v2118: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:35:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:35:14 smithi100 ceph-mon[32599]: osdmap e183: 12 total, 12 up, 12 in 2024-07-26T14:35:15.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:35:15.943+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:35:15.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:35:15.944+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:35:15.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:35:15.944+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:35:15.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:35:15.944+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:35:15.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:35:15.944+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:35:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:35:16 smithi138 ceph-mon[36954]: pgmap v2120: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:35:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:35:16 smithi100 ceph-mon[32599]: pgmap v2120: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:35:17.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:35:16 smithi167 ceph-mon[37140]: pgmap v2120: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:35:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:35:18 smithi138 ceph-mon[36954]: pgmap v2121: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 211 B/s wr, 6 op/s 2024-07-26T14:35:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:35:18 smithi100 ceph-mon[32599]: pgmap v2121: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 211 B/s wr, 6 op/s 2024-07-26T14:35:19.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:35:18 smithi167 ceph-mon[37140]: pgmap v2121: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 211 B/s wr, 6 op/s 2024-07-26T14:35:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:35:20 smithi100 ceph-mon[32599]: pgmap v2122: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:35:20.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:35:20 smithi167 ceph-mon[37140]: pgmap v2122: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:35:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:35:20 smithi138 ceph-mon[36954]: pgmap v2122: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:35:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:35:22 smithi138 ceph-mon[36954]: pgmap v2123: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:35:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:35:22 smithi100 ceph-mon[32599]: pgmap v2123: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:35:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:35:22 smithi167 ceph-mon[37140]: pgmap v2123: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:35:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:35:24 smithi138 ceph-mon[36954]: pgmap v2124: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:35:24.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:35:24 smithi100 ceph-mon[32599]: pgmap v2124: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:35:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:35:24 smithi167 ceph-mon[37140]: pgmap v2124: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:35:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:35:26 smithi138 ceph-mon[36954]: pgmap v2125: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:35:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:35:26 smithi100 ceph-mon[32599]: pgmap v2125: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:35:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:35:26 smithi167 ceph-mon[37140]: pgmap v2125: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:35:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:35:28 smithi138 ceph-mon[36954]: pgmap v2126: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:35:28.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:35:28 smithi100 ceph-mon[32599]: pgmap v2126: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:35:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:35:28 smithi167 ceph-mon[37140]: pgmap v2126: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:35:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:35:30 smithi138 ceph-mon[36954]: pgmap v2127: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:35:30.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:35:30.959+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:35:30.960 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:35:30 smithi100 ceph-mon[32599]: pgmap v2127: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:35:30.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:35:30.960+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:35:30.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:35:30.960+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:35:30.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:35:30.960+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:35:30.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:35:30.960+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:35:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:35:30 smithi167 ceph-mon[37140]: pgmap v2127: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:35:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:35:32 smithi138 ceph-mon[36954]: pgmap v2128: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:35:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:35:32 smithi100 ceph-mon[32599]: pgmap v2128: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:35:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:35:32 smithi167 ceph-mon[37140]: pgmap v2128: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:35:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:35:34 smithi138 ceph-mon[36954]: pgmap v2129: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:35:34.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:35:34 smithi100 ceph-mon[32599]: pgmap v2129: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:35:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:35:34 smithi167 ceph-mon[37140]: pgmap v2129: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:35:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:35:36 smithi138 ceph-mon[36954]: pgmap v2130: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:35:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:35:36 smithi100 ceph-mon[32599]: pgmap v2130: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:35:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:35:36 smithi167 ceph-mon[37140]: pgmap v2130: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:35:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:35:38 smithi138 ceph-mon[36954]: pgmap v2131: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:35:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:35:38 smithi100 ceph-mon[32599]: pgmap v2131: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:35:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:35:38 smithi167 ceph-mon[37140]: pgmap v2131: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:35:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:35:40 smithi138 ceph-mon[36954]: pgmap v2132: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:35:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:35:40 smithi100 ceph-mon[32599]: pgmap v2132: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:35:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:35:40 smithi167 ceph-mon[37140]: pgmap v2132: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:35:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:35:42 smithi138 ceph-mon[36954]: pgmap v2133: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:35:42.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:35:42 smithi100 ceph-mon[32599]: pgmap v2133: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:35:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:35:42 smithi167 ceph-mon[37140]: pgmap v2133: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:35:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:35:44 smithi138 ceph-mon[36954]: pgmap v2134: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:35:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:35:44 smithi100 ceph-mon[32599]: pgmap v2134: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:35:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:35:44 smithi167 ceph-mon[37140]: pgmap v2134: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:35:45.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:35:45.975+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:35:45.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:35:45.976+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:35:45.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:35:45.976+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:35:45.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:35:45.976+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:35:45.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:35:45.976+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:35:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:35:46 smithi138 ceph-mon[36954]: pgmap v2135: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:35:46.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:35:46 smithi100 ceph-mon[32599]: pgmap v2135: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:35:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:35:46 smithi167 ceph-mon[37140]: pgmap v2135: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:35:47.869 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:35:47 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:35:47.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:35:47 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:35:48.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:35:47 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:35:48.896 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:35:48 smithi100 ceph-mon[32599]: pgmap v2136: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:35:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:35:48 smithi138 ceph-mon[36954]: pgmap v2136: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:35:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:35:48 smithi167 ceph-mon[37140]: pgmap v2136: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:35:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:35:50 smithi100 ceph-mon[32599]: pgmap v2137: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:35:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:35:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:35:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:35:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:35:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:35:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:35:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:35:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:35:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:35:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:35:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:35:50 smithi138 ceph-mon[36954]: pgmap v2137: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:35:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:35:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:35:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:35:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:35:50.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:35:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:35:50.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:35:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:35:50.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:35:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:35:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:35:50 smithi167 ceph-mon[37140]: pgmap v2137: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:35:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:35:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:35:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:35:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:35:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:35:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:35:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:35:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:35:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:35:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:35:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:35:52 smithi138 ceph-mon[36954]: pgmap v2138: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:35:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:35:52 smithi100 ceph-mon[32599]: pgmap v2138: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:35:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:35:52 smithi167 ceph-mon[37140]: pgmap v2138: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:35:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:35:54 smithi138 ceph-mon[36954]: pgmap v2139: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:35:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:35:54 smithi100 ceph-mon[32599]: pgmap v2139: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:35:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:35:54 smithi167 ceph-mon[37140]: pgmap v2139: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:35:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:35:56 smithi138 ceph-mon[36954]: pgmap v2140: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:35:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:35:56 smithi100 ceph-mon[32599]: pgmap v2140: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:35:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:35:56 smithi167 ceph-mon[37140]: pgmap v2140: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:35:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:35:58 smithi138 ceph-mon[36954]: pgmap v2141: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T14:35:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:35:58 smithi100 ceph-mon[32599]: pgmap v2141: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T14:35:59.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:35:58 smithi167 ceph-mon[37140]: pgmap v2141: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T14:36:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:00 smithi138 ceph-mon[36954]: pgmap v2142: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:36:00.986 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:00 smithi100 ceph-mon[32599]: pgmap v2142: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:36:00.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:36:00.984+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:36:00.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:36:00.985+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:36:00.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:36:00.985+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:36:00.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:36:00.985+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:36:00.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:36:00.985+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:36:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:00 smithi167 ceph-mon[37140]: pgmap v2142: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:36:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:02 smithi138 ceph-mon[36954]: pgmap v2143: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:36:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:02 smithi100 ceph-mon[32599]: pgmap v2143: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:36:03.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:02 smithi167 ceph-mon[37140]: pgmap v2143: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:36:03.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:36:03.418+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c006610 con 0x7fe5a81087a0 2024-07-26T14:36:03.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:36:03.418+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T14:36:03.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:36:03.419+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 26 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c070 con 0x7fe5a81087a0 2024-07-26T14:36:03.421 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:36:03.419+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 26 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T14:36:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:04 smithi138 ceph-mon[36954]: pgmap v2144: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:36:04.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:04 smithi100 ceph-mon[32599]: pgmap v2144: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:36:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:04 smithi167 ceph-mon[37140]: pgmap v2144: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T14:36:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:06 smithi138 ceph-mon[36954]: pgmap v2145: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:36:06.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:06 smithi100 ceph-mon[32599]: pgmap v2145: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:36:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:06 smithi167 ceph-mon[37140]: pgmap v2145: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:36:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:08 smithi138 ceph-mon[36954]: pgmap v2146: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:36:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:08 smithi138 ceph-mon[36954]: osdmap e184: 12 total, 12 up, 12 in 2024-07-26T14:36:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:08 smithi100 ceph-mon[32599]: pgmap v2146: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:36:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:08 smithi100 ceph-mon[32599]: osdmap e184: 12 total, 12 up, 12 in 2024-07-26T14:36:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:08 smithi167 ceph-mon[37140]: pgmap v2146: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:36:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:08 smithi167 ceph-mon[37140]: osdmap e184: 12 total, 12 up, 12 in 2024-07-26T14:36:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:10 smithi138 ceph-mon[36954]: pgmap v2148: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:36:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:10 smithi100 ceph-mon[32599]: pgmap v2148: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:36:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:10 smithi167 ceph-mon[37140]: pgmap v2148: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:36:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:12 smithi138 ceph-mon[36954]: pgmap v2149: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:36:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:12 smithi100 ceph-mon[32599]: pgmap v2149: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:36:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:12 smithi167 ceph-mon[37140]: pgmap v2149: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:36:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:14 smithi138 ceph-mon[36954]: pgmap v2150: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:36:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:14 smithi138 ceph-mon[36954]: osdmap e185: 12 total, 12 up, 12 in 2024-07-26T14:36:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:14 smithi100 ceph-mon[32599]: pgmap v2150: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:36:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:14 smithi100 ceph-mon[32599]: osdmap e185: 12 total, 12 up, 12 in 2024-07-26T14:36:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:14 smithi167 ceph-mon[37140]: pgmap v2150: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T14:36:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:14 smithi167 ceph-mon[37140]: osdmap e185: 12 total, 12 up, 12 in 2024-07-26T14:36:16.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:36:16.001+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:36:16.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:36:16.002+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:36:16.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:36:16.002+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:36:16.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:36:16.002+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:36:16.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:36:16.002+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:36:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:16 smithi138 ceph-mon[36954]: pgmap v2152: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:36:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:16 smithi100 ceph-mon[32599]: pgmap v2152: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:36:17.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:16 smithi167 ceph-mon[37140]: pgmap v2152: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:36:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:18 smithi138 ceph-mon[36954]: pgmap v2153: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T14:36:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:18 smithi100 ceph-mon[32599]: pgmap v2153: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T14:36:19.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:18 smithi167 ceph-mon[37140]: pgmap v2153: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T14:36:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:20 smithi138 ceph-mon[36954]: pgmap v2154: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:36:20.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:20 smithi100 ceph-mon[32599]: pgmap v2154: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:36:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:20 smithi167 ceph-mon[37140]: pgmap v2154: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:36:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:22 smithi138 ceph-mon[36954]: pgmap v2155: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:36:22.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:22 smithi100 ceph-mon[32599]: pgmap v2155: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:36:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:22 smithi167 ceph-mon[37140]: pgmap v2155: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:36:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:24 smithi138 ceph-mon[36954]: pgmap v2156: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:36:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:24 smithi100 ceph-mon[32599]: pgmap v2156: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:36:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:24 smithi167 ceph-mon[37140]: pgmap v2156: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:36:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:26 smithi138 ceph-mon[36954]: pgmap v2157: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:36:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:26 smithi100 ceph-mon[32599]: pgmap v2157: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:36:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:26 smithi167 ceph-mon[37140]: pgmap v2157: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:36:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:28 smithi138 ceph-mon[36954]: pgmap v2158: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T14:36:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:28 smithi100 ceph-mon[32599]: pgmap v2158: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T14:36:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:28 smithi167 ceph-mon[37140]: pgmap v2158: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T14:36:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:30 smithi100 ceph-mon[32599]: pgmap v2159: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:36:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:30 smithi167 ceph-mon[37140]: pgmap v2159: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:36:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:30 smithi138 ceph-mon[36954]: pgmap v2159: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:36:31.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:36:31.015+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:36:31.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:36:31.016+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:36:31.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:36:31.016+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:36:31.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:36:31.016+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:36:31.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:36:31.016+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:36:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:32 smithi138 ceph-mon[36954]: pgmap v2160: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:36:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:32 smithi100 ceph-mon[32599]: pgmap v2160: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:36:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:32 smithi167 ceph-mon[37140]: pgmap v2160: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:36:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:34 smithi138 ceph-mon[36954]: pgmap v2161: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:36:34.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:34 smithi100 ceph-mon[32599]: pgmap v2161: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:36:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:34 smithi167 ceph-mon[37140]: pgmap v2161: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:36:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:36 smithi138 ceph-mon[36954]: pgmap v2162: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:36:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:36 smithi100 ceph-mon[32599]: pgmap v2162: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:36:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:36 smithi167 ceph-mon[37140]: pgmap v2162: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:36:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:38 smithi138 ceph-mon[36954]: pgmap v2163: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T14:36:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:38 smithi100 ceph-mon[32599]: pgmap v2163: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T14:36:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:38 smithi167 ceph-mon[37140]: pgmap v2163: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T14:36:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:40 smithi138 ceph-mon[36954]: pgmap v2164: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:36:40.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:40 smithi100 ceph-mon[32599]: pgmap v2164: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:36:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:40 smithi167 ceph-mon[37140]: pgmap v2164: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:36:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:42 smithi138 ceph-mon[36954]: pgmap v2165: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:36:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:42 smithi100 ceph-mon[32599]: pgmap v2165: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:36:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:42 smithi167 ceph-mon[37140]: pgmap v2165: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:36:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:44 smithi138 ceph-mon[36954]: pgmap v2166: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:36:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:44 smithi100 ceph-mon[32599]: pgmap v2166: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:36:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:44 smithi167 ceph-mon[37140]: pgmap v2166: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:36:46.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:36:46.031+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:36:46.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:36:46.032+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:36:46.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:36:46.032+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:36:46.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:36:46.032+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:36:46.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:36:46.032+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:36:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:46 smithi138 ceph-mon[36954]: pgmap v2167: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:36:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:46 smithi100 ceph-mon[32599]: pgmap v2167: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:36:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:46 smithi167 ceph-mon[37140]: pgmap v2167: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:36:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:48 smithi138 ceph-mon[36954]: pgmap v2168: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T14:36:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:48 smithi100 ceph-mon[32599]: pgmap v2168: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T14:36:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:48 smithi167 ceph-mon[37140]: pgmap v2168: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T14:36:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:50 smithi138 ceph-mon[36954]: pgmap v2169: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:36:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:36:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:50 smithi100 ceph-mon[32599]: pgmap v2169: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:36:50.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:36:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:50 smithi167 ceph-mon[37140]: pgmap v2169: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:36:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:36:51.686 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:51 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:36:51.687 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:51 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:36:51.687 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:51 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:36:51.687 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:51 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:36:51.687 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:51 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T14:36:51.687 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:51 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T14:36:51.722 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:51 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:36:51.723 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:51 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:36:51.723 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:51 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:36:51.723 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:51 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:36:51.723 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:51 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T14:36:51.723 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:51 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T14:36:51.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:51 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:36:51.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:51 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:36:51.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:51 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:36:51.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:51 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:36:51.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:51 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T14:36:51.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:51 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T14:36:52.727 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:52 smithi138 ceph-mon[36954]: pgmap v2170: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:36:52.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:52 smithi167 ceph-mon[37140]: pgmap v2170: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:36:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:52 smithi100 ceph-mon[32599]: pgmap v2170: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:36:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:54 smithi138 ceph-mon[36954]: pgmap v2171: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:36:54.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:36:54.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:36:54.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:36:54.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:36:54.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:36:54.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:36:54.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:36:54.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:36:54.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:36:54.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:36:54.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:36:54.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:36:54.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:36:54.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:36:54.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:36:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:54 smithi100 ceph-mon[32599]: pgmap v2171: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:36:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:36:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:36:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:36:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:36:54.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:36:54.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:36:54.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:36:54.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:36:54.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:36:54.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:36:54.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:36:54.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:36:54.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:36:54.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:36:54.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:36:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:54 smithi167 ceph-mon[37140]: pgmap v2171: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:36:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:36:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:36:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:36:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:36:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:36:55.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:36:55.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:36:55.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:36:55.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:36:55.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:36:55.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:36:55.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:36:55.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:36:55.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:36:55.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:36:55.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:55 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T14:36:55.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:55 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T14:36:55.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:55 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T14:36:55.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:55 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T14:36:55.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:55 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T14:36:55.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:55 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T14:36:55.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:55 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T14:36:55.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:55 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T14:36:56.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:55 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T14:36:56.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:55 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T14:36:56.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:55 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T14:36:56.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:55 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T14:36:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:56 smithi138 ceph-mon[36954]: pgmap v2172: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T14:36:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:56 smithi100 ceph-mon[32599]: pgmap v2172: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T14:36:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:56 smithi167 ceph-mon[37140]: pgmap v2172: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-07-26T14:36:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:36:58 smithi138 ceph-mon[36954]: pgmap v2173: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T14:36:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:36:58 smithi100 ceph-mon[32599]: pgmap v2173: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T14:36:59.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:36:58 smithi167 ceph-mon[37140]: pgmap v2173: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T14:37:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:37:00 smithi138 ceph-mon[36954]: pgmap v2174: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:37:00.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:37:00 smithi100 ceph-mon[32599]: pgmap v2174: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:37:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:37:00 smithi167 ceph-mon[37140]: pgmap v2174: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:37:01.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:37:01.047+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:37:01.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:37:01.047+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:37:01.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:37:01.047+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:37:01.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:37:01.047+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:37:01.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:37:01.047+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:37:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:37:02 smithi138 ceph-mon[36954]: pgmap v2175: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:37:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:37:02 smithi100 ceph-mon[32599]: pgmap v2175: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:37:03.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:37:02 smithi167 ceph-mon[37140]: pgmap v2175: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:37:04.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:37:04 smithi167 ceph-mon[37140]: pgmap v2176: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T14:37:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:37:04 smithi138 ceph-mon[36954]: pgmap v2176: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T14:37:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:37:04 smithi100 ceph-mon[32599]: pgmap v2176: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T14:37:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:37:06 smithi138 ceph-mon[36954]: pgmap v2177: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:37:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:37:06 smithi100 ceph-mon[32599]: pgmap v2177: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:37:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:37:06 smithi167 ceph-mon[37140]: pgmap v2177: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:37:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:37:08 smithi138 ceph-mon[36954]: pgmap v2178: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:37:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:37:08 smithi138 ceph-mon[36954]: osdmap e186: 12 total, 12 up, 12 in 2024-07-26T14:37:08.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:37:08 smithi100 ceph-mon[32599]: pgmap v2178: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:37:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:37:08 smithi100 ceph-mon[32599]: osdmap e186: 12 total, 12 up, 12 in 2024-07-26T14:37:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:37:08 smithi167 ceph-mon[37140]: pgmap v2178: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:37:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:37:08 smithi167 ceph-mon[37140]: osdmap e186: 12 total, 12 up, 12 in 2024-07-26T14:37:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:37:10 smithi138 ceph-mon[36954]: pgmap v2180: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:37:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:37:10 smithi100 ceph-mon[32599]: pgmap v2180: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:37:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:37:10 smithi167 ceph-mon[37140]: pgmap v2180: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:37:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:37:12 smithi138 ceph-mon[36954]: pgmap v2181: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:37:12.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:37:12 smithi100 ceph-mon[32599]: pgmap v2181: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:37:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:37:12 smithi167 ceph-mon[37140]: pgmap v2181: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:37:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:37:14 smithi138 ceph-mon[36954]: pgmap v2182: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:37:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:37:14 smithi138 ceph-mon[36954]: osdmap e187: 12 total, 12 up, 12 in 2024-07-26T14:37:14.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:37:14 smithi100 ceph-mon[32599]: pgmap v2182: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:37:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:37:14 smithi100 ceph-mon[32599]: osdmap e187: 12 total, 12 up, 12 in 2024-07-26T14:37:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:37:14 smithi167 ceph-mon[37140]: pgmap v2182: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:37:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:37:14 smithi167 ceph-mon[37140]: osdmap e187: 12 total, 12 up, 12 in 2024-07-26T14:37:16.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:37:16.062+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:37:16.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:37:16.062+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:37:16.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:37:16.062+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:37:16.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:37:16.062+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:37:16.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:37:16.062+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:37:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:37:16 smithi138 ceph-mon[36954]: pgmap v2184: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:37:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:37:16 smithi100 ceph-mon[32599]: pgmap v2184: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:37:17.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:37:16 smithi167 ceph-mon[37140]: pgmap v2184: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:37:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:37:18 smithi138 ceph-mon[36954]: pgmap v2185: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T14:37:18.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:37:18 smithi100 ceph-mon[32599]: pgmap v2185: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T14:37:19.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:37:18 smithi167 ceph-mon[37140]: pgmap v2185: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T14:37:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:37:20 smithi138 ceph-mon[36954]: pgmap v2186: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:37:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:37:20 smithi100 ceph-mon[32599]: pgmap v2186: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:37:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:37:20 smithi167 ceph-mon[37140]: pgmap v2186: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:37:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:37:22 smithi138 ceph-mon[36954]: pgmap v2187: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:37:22.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:37:22 smithi100 ceph-mon[32599]: pgmap v2187: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:37:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:37:22 smithi167 ceph-mon[37140]: pgmap v2187: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:37:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:37:24 smithi138 ceph-mon[36954]: pgmap v2188: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:37:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:37:24 smithi100 ceph-mon[32599]: pgmap v2188: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:37:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:37:24 smithi167 ceph-mon[37140]: pgmap v2188: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:37:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:37:26 smithi138 ceph-mon[36954]: pgmap v2189: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:37:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:37:26 smithi100 ceph-mon[32599]: pgmap v2189: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:37:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:37:26 smithi167 ceph-mon[37140]: pgmap v2189: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:37:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:37:28 smithi138 ceph-mon[36954]: pgmap v2190: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T14:37:28.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:37:28 smithi100 ceph-mon[32599]: pgmap v2190: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T14:37:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:37:28 smithi167 ceph-mon[37140]: pgmap v2190: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T14:37:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:37:30 smithi138 ceph-mon[36954]: pgmap v2191: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:37:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:37:30 smithi100 ceph-mon[32599]: pgmap v2191: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:37:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:37:30 smithi167 ceph-mon[37140]: pgmap v2191: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:37:31.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:37:31.066+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:37:31.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:37:31.067+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:37:31.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:37:31.067+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:37:31.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:37:31.067+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:37:31.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:37:31.067+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:37:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:37:32 smithi138 ceph-mon[36954]: pgmap v2192: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:37:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:37:32 smithi100 ceph-mon[32599]: pgmap v2192: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:37:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:37:32 smithi167 ceph-mon[37140]: pgmap v2192: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:37:33.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:37:33.432+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c006610 con 0x7fe5a81087a0 2024-07-26T14:37:33.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:37:33.432+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T14:37:33.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:37:33.432+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 27 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T14:37:33.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:37:33.432+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 27 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe5a008c070 con 0x7fe5a81087a0 2024-07-26T14:37:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:37:34 smithi138 ceph-mon[36954]: pgmap v2193: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:37:34.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:37:34 smithi100 ceph-mon[32599]: pgmap v2193: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:37:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:37:34 smithi167 ceph-mon[37140]: pgmap v2193: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:37:36.949 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:37:36 smithi138 ceph-mon[36954]: pgmap v2194: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:37:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:37:36 smithi100 ceph-mon[32599]: pgmap v2194: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:37:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:37:36 smithi167 ceph-mon[37140]: pgmap v2194: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:37:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:37:38 smithi138 ceph-mon[36954]: pgmap v2195: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:37:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:37:38 smithi100 ceph-mon[32599]: pgmap v2195: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:37:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:37:38 smithi167 ceph-mon[37140]: pgmap v2195: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:37:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:37:40 smithi138 ceph-mon[36954]: pgmap v2196: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:37:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:37:40 smithi100 ceph-mon[32599]: pgmap v2196: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:37:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:37:40 smithi167 ceph-mon[37140]: pgmap v2196: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:37:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:37:42 smithi138 ceph-mon[36954]: pgmap v2197: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:37:42.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:37:42 smithi100 ceph-mon[32599]: pgmap v2197: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:37:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:37:42 smithi167 ceph-mon[37140]: pgmap v2197: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:37:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:37:44 smithi138 ceph-mon[36954]: pgmap v2198: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:37:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:37:44 smithi100 ceph-mon[32599]: pgmap v2198: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:37:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:37:44 smithi167 ceph-mon[37140]: pgmap v2198: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:37:46.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:37:46.071+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:37:46.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:37:46.072+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:37:46.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:37:46.072+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:37:46.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:37:46.072+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:37:46.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:37:46.072+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:37:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:37:46 smithi100 ceph-mon[32599]: pgmap v2199: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:37:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:37:46 smithi167 ceph-mon[37140]: pgmap v2199: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:37:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:37:46 smithi138 ceph-mon[36954]: pgmap v2199: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:37:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:37:48 smithi100 ceph-mon[32599]: pgmap v2200: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:37:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:37:48 smithi167 ceph-mon[37140]: pgmap v2200: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:37:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:37:48 smithi138 ceph-mon[36954]: pgmap v2200: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:37:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:37:50 smithi100 ceph-mon[32599]: pgmap v2201: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:37:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:37:50 smithi167 ceph-mon[37140]: pgmap v2201: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:37:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:37:50 smithi138 ceph-mon[36954]: pgmap v2201: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:37:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:37:52 smithi100 ceph-mon[32599]: pgmap v2202: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:37:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:37:52 smithi167 ceph-mon[37140]: pgmap v2202: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:37:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:37:52 smithi138 ceph-mon[36954]: pgmap v2202: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:37:54.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:37:54 smithi100 ceph-mon[32599]: pgmap v2203: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:37:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:37:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:37:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:37:54 smithi167 ceph-mon[37140]: pgmap v2203: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:37:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:37:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:37:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:37:54 smithi138 ceph-mon[36954]: pgmap v2203: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:37:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:37:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:37:55.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:37:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:37:55.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:37:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:37:55.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:37:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:37:56.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:37:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:37:56.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:37:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:37:56.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:37:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:37:56.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:37:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:37:56.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:37:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:37:56.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:37:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:37:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:37:56 smithi100 ceph-mon[32599]: pgmap v2204: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:37:57.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:37:56 smithi167 ceph-mon[37140]: pgmap v2204: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:37:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:37:56 smithi138 ceph-mon[36954]: pgmap v2204: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:37:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:37:58 smithi100 ceph-mon[32599]: pgmap v2205: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:37:59.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:37:58 smithi167 ceph-mon[37140]: pgmap v2205: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:37:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:37:58 smithi138 ceph-mon[36954]: pgmap v2205: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:38:01.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:38:01.087+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:38:01.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:38:01.088+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:38:01.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:38:01.088+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:38:01.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:38:01.088+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:38:01.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:38:01.088+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:38:01.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:38:00 smithi167 ceph-mon[37140]: pgmap v2206: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:38:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:38:00 smithi138 ceph-mon[36954]: pgmap v2206: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:38:01.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:38:00 smithi100 ceph-mon[32599]: pgmap v2206: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:38:03.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:38:02 smithi167 ceph-mon[37140]: pgmap v2207: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:38:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:38:02 smithi138 ceph-mon[36954]: pgmap v2207: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:38:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:38:02 smithi100 ceph-mon[32599]: pgmap v2207: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:38:05.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:38:05 smithi167 ceph-mon[37140]: pgmap v2208: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-07-26T14:38:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:38:05 smithi138 ceph-mon[36954]: pgmap v2208: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-07-26T14:38:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:38:05 smithi100 ceph-mon[32599]: pgmap v2208: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-07-26T14:38:07.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:38:07 smithi167 ceph-mon[37140]: pgmap v2209: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:38:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:38:07 smithi138 ceph-mon[36954]: pgmap v2209: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:38:07.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:38:07 smithi100 ceph-mon[32599]: pgmap v2209: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:38:08.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:38:08 smithi167 ceph-mon[37140]: osdmap e188: 12 total, 12 up, 12 in 2024-07-26T14:38:08.341 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:38:08 smithi138 ceph-mon[36954]: osdmap e188: 12 total, 12 up, 12 in 2024-07-26T14:38:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:38:08 smithi100 ceph-mon[32599]: osdmap e188: 12 total, 12 up, 12 in 2024-07-26T14:38:09.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:38:09 smithi167 ceph-mon[37140]: pgmap v2211: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:38:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:38:09 smithi138 ceph-mon[36954]: pgmap v2211: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:38:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:38:09 smithi100 ceph-mon[32599]: pgmap v2211: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:38:11.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:38:11 smithi138 ceph-mon[36954]: pgmap v2212: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:38:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:38:11 smithi100 ceph-mon[32599]: pgmap v2212: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:38:11.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:38:11 smithi167 ceph-mon[37140]: pgmap v2212: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:38:13.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:38:13 smithi138 ceph-mon[36954]: pgmap v2213: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:38:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:38:13 smithi100 ceph-mon[32599]: pgmap v2213: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:38:13.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:38:13 smithi167 ceph-mon[37140]: pgmap v2213: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:38:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:38:15 smithi138 ceph-mon[36954]: pgmap v2214: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:38:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:38:15 smithi138 ceph-mon[36954]: osdmap e189: 12 total, 12 up, 12 in 2024-07-26T14:38:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:38:15 smithi100 ceph-mon[32599]: pgmap v2214: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:38:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:38:15 smithi100 ceph-mon[32599]: osdmap e189: 12 total, 12 up, 12 in 2024-07-26T14:38:15.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:38:15 smithi167 ceph-mon[37140]: pgmap v2214: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:38:15.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:38:15 smithi167 ceph-mon[37140]: osdmap e189: 12 total, 12 up, 12 in 2024-07-26T14:38:16.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:38:16.103+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:38:16.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:38:16.104+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:38:16.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:38:16.104+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:38:16.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:38:16.104+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:38:16.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:38:16.104+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:38:17.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:38:17 smithi167 ceph-mon[37140]: pgmap v2216: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 240 B/s wr, 6 op/s 2024-07-26T14:38:17.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:38:17 smithi138 ceph-mon[36954]: pgmap v2216: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 240 B/s wr, 6 op/s 2024-07-26T14:38:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:38:17 smithi100 ceph-mon[32599]: pgmap v2216: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 240 B/s wr, 6 op/s 2024-07-26T14:38:19.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:38:19 smithi138 ceph-mon[36954]: pgmap v2217: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:38:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:38:19 smithi100 ceph-mon[32599]: pgmap v2217: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:38:19.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:38:19 smithi167 ceph-mon[37140]: pgmap v2217: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:38:21.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:38:21 smithi138 ceph-mon[36954]: pgmap v2218: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:38:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:38:21 smithi100 ceph-mon[32599]: pgmap v2218: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:38:21.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:38:21 smithi167 ceph-mon[37140]: pgmap v2218: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:38:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:38:23 smithi138 ceph-mon[36954]: pgmap v2219: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:38:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:38:23 smithi100 ceph-mon[32599]: pgmap v2219: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:38:23.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:38:23 smithi167 ceph-mon[37140]: pgmap v2219: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:38:25.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:38:25 smithi138 ceph-mon[36954]: pgmap v2220: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:38:25.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:38:25 smithi100 ceph-mon[32599]: pgmap v2220: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:38:25.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:38:25 smithi167 ceph-mon[37140]: pgmap v2220: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:38:27.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:38:27 smithi138 ceph-mon[36954]: pgmap v2221: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:38:27.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:38:27 smithi100 ceph-mon[32599]: pgmap v2221: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:38:27.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:38:27 smithi167 ceph-mon[37140]: pgmap v2221: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:38:29.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:38:29 smithi138 ceph-mon[36954]: pgmap v2222: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:38:29.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:38:29 smithi100 ceph-mon[32599]: pgmap v2222: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:38:29.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:38:29 smithi167 ceph-mon[37140]: pgmap v2222: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:38:31.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:38:31.111+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:38:31.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:38:31.112+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:38:31.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:38:31.112+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:38:31.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:38:31.112+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:38:31.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:38:31.112+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:38:31.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:38:31 smithi138 ceph-mon[36954]: pgmap v2223: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:38:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:38:31 smithi100 ceph-mon[32599]: pgmap v2223: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:38:31.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:38:31 smithi167 ceph-mon[37140]: pgmap v2223: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:38:33.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:38:33 smithi138 ceph-mon[36954]: pgmap v2224: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:38:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:38:33 smithi100 ceph-mon[32599]: pgmap v2224: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:38:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:38:33 smithi167 ceph-mon[37140]: pgmap v2224: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:38:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:38:35 smithi138 ceph-mon[36954]: pgmap v2225: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:38:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:38:35 smithi100 ceph-mon[32599]: pgmap v2225: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:38:35.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:38:35 smithi167 ceph-mon[37140]: pgmap v2225: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:38:37.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:38:37 smithi138 ceph-mon[36954]: pgmap v2226: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:38:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:38:37 smithi100 ceph-mon[32599]: pgmap v2226: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:38:37.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:38:37 smithi167 ceph-mon[37140]: pgmap v2226: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:38:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:38:39 smithi138 ceph-mon[36954]: pgmap v2227: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:38:39.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:38:39 smithi100 ceph-mon[32599]: pgmap v2227: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:38:39.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:38:39 smithi167 ceph-mon[37140]: pgmap v2227: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:38:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:38:41 smithi138 ceph-mon[36954]: pgmap v2228: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:38:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:38:41 smithi100 ceph-mon[32599]: pgmap v2228: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:38:41.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:38:41 smithi167 ceph-mon[37140]: pgmap v2228: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:38:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:38:43 smithi138 ceph-mon[36954]: pgmap v2229: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:38:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:38:43 smithi100 ceph-mon[32599]: pgmap v2229: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:38:43.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:38:43 smithi167 ceph-mon[37140]: pgmap v2229: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:38:45.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:38:45 smithi138 ceph-mon[36954]: pgmap v2230: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:38:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:38:45 smithi100 ceph-mon[32599]: pgmap v2230: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:38:45.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:38:45 smithi167 ceph-mon[37140]: pgmap v2230: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:38:46.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:38:46.127+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:38:46.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:38:46.128+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:38:46.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:38:46.128+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:38:46.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:38:46.128+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:38:46.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:38:46.128+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:38:47.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:38:47 smithi138 ceph-mon[36954]: pgmap v2231: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:38:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:38:47 smithi100 ceph-mon[32599]: pgmap v2231: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:38:47.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:38:47 smithi167 ceph-mon[37140]: pgmap v2231: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:38:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:38:49 smithi138 ceph-mon[36954]: pgmap v2232: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:38:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:38:49 smithi100 ceph-mon[32599]: pgmap v2232: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:38:49.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:38:49 smithi167 ceph-mon[37140]: pgmap v2232: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:38:50.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:38:50 smithi138 ceph-mon[36954]: pgmap v2233: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:38:50.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:38:50 smithi100 ceph-mon[32599]: pgmap v2233: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:38:50.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:38:50 smithi167 ceph-mon[37140]: pgmap v2233: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:38:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:38:52 smithi138 ceph-mon[36954]: pgmap v2234: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:38:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:38:52 smithi100 ceph-mon[32599]: pgmap v2234: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:38:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:38:52 smithi167 ceph-mon[37140]: pgmap v2234: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:38:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:38:54 smithi138 ceph-mon[36954]: pgmap v2235: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:38:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:38:54 smithi100 ceph-mon[32599]: pgmap v2235: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:38:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:38:54 smithi167 ceph-mon[37140]: pgmap v2235: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:38:55.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:38:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:38:55.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:38:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:38:55.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:38:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:38:55.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:38:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:38:55.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:38:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:38:55.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:38:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:38:55.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:38:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:38:55.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:38:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:38:56.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:38:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:38:56.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:38:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:38:56.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:38:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:38:56.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:38:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:38:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:38:56 smithi138 ceph-mon[36954]: pgmap v2236: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:38:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:38:56 smithi100 ceph-mon[32599]: pgmap v2236: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:38:57.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:38:56 smithi167 ceph-mon[37140]: pgmap v2236: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:38:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:38:58 smithi138 ceph-mon[36954]: pgmap v2237: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:38:58.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:38:58 smithi100 ceph-mon[32599]: pgmap v2237: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:38:59.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:38:58 smithi167 ceph-mon[37140]: pgmap v2237: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:39:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:00 smithi138 ceph-mon[36954]: pgmap v2238: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:39:00.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:00 smithi100 ceph-mon[32599]: pgmap v2238: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:39:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:00 smithi167 ceph-mon[37140]: pgmap v2238: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:39:01.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:39:01.143+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:39:01.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:39:01.144+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:39:01.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:39:01.144+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:39:01.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:39:01.144+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:39:01.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:39:01.144+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:39:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:02 smithi138 ceph-mon[36954]: pgmap v2239: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:39:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:02 smithi100 ceph-mon[32599]: pgmap v2239: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:39:03.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:02 smithi167 ceph-mon[37140]: pgmap v2239: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:39:03.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:39:03.445+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c006610 con 0x7fe5a81087a0 2024-07-26T14:39:03.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:39:03.445+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T14:39:03.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:39:03.446+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 28 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c070 con 0x7fe5a81087a0 2024-07-26T14:39:03.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:39:03.446+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 28 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T14:39:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:04 smithi138 ceph-mon[36954]: pgmap v2240: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T14:39:04.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:04 smithi100 ceph-mon[32599]: pgmap v2240: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T14:39:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:04 smithi167 ceph-mon[37140]: pgmap v2240: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T14:39:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:06 smithi138 ceph-mon[36954]: pgmap v2241: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:39:06.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:06 smithi100 ceph-mon[32599]: pgmap v2241: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:39:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:06 smithi167 ceph-mon[37140]: pgmap v2241: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:39:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:08 smithi138 ceph-mon[36954]: pgmap v2242: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:39:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:08 smithi138 ceph-mon[36954]: osdmap e190: 12 total, 12 up, 12 in 2024-07-26T14:39:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:08 smithi100 ceph-mon[32599]: pgmap v2242: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:39:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:08 smithi100 ceph-mon[32599]: osdmap e190: 12 total, 12 up, 12 in 2024-07-26T14:39:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:08 smithi167 ceph-mon[37140]: pgmap v2242: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:39:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:08 smithi167 ceph-mon[37140]: osdmap e190: 12 total, 12 up, 12 in 2024-07-26T14:39:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:10 smithi138 ceph-mon[36954]: pgmap v2244: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:39:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:10 smithi100 ceph-mon[32599]: pgmap v2244: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:39:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:10 smithi167 ceph-mon[37140]: pgmap v2244: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:39:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:12 smithi138 ceph-mon[36954]: pgmap v2245: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:39:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:12 smithi100 ceph-mon[32599]: pgmap v2245: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:39:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:12 smithi167 ceph-mon[37140]: pgmap v2245: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:39:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:14 smithi138 ceph-mon[36954]: pgmap v2246: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:39:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:14 smithi138 ceph-mon[36954]: osdmap e191: 12 total, 12 up, 12 in 2024-07-26T14:39:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:14 smithi100 ceph-mon[32599]: pgmap v2246: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:39:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:14 smithi100 ceph-mon[32599]: osdmap e191: 12 total, 12 up, 12 in 2024-07-26T14:39:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:14 smithi167 ceph-mon[37140]: pgmap v2246: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:39:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:14 smithi167 ceph-mon[37140]: osdmap e191: 12 total, 12 up, 12 in 2024-07-26T14:39:16.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:39:16.159+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:39:16.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:39:16.160+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:39:16.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:39:16.160+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:39:16.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:39:16.160+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:39:16.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:39:16.160+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:39:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:16 smithi138 ceph-mon[36954]: pgmap v2248: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:39:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:16 smithi100 ceph-mon[32599]: pgmap v2248: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:39:17.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:16 smithi167 ceph-mon[37140]: pgmap v2248: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:39:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:18 smithi138 ceph-mon[36954]: pgmap v2249: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T14:39:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:18 smithi100 ceph-mon[32599]: pgmap v2249: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T14:39:19.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:18 smithi167 ceph-mon[37140]: pgmap v2249: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T14:39:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:20 smithi138 ceph-mon[36954]: pgmap v2250: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:39:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:20 smithi100 ceph-mon[32599]: pgmap v2250: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:39:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:20 smithi167 ceph-mon[37140]: pgmap v2250: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:39:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:22 smithi138 ceph-mon[36954]: pgmap v2251: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:39:22.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:22 smithi100 ceph-mon[32599]: pgmap v2251: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:39:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:22 smithi167 ceph-mon[37140]: pgmap v2251: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:39:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:24 smithi138 ceph-mon[36954]: pgmap v2252: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:39:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:24 smithi100 ceph-mon[32599]: pgmap v2252: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:39:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:24 smithi167 ceph-mon[37140]: pgmap v2252: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:39:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:26 smithi138 ceph-mon[36954]: pgmap v2253: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:39:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:26 smithi100 ceph-mon[32599]: pgmap v2253: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:39:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:26 smithi167 ceph-mon[37140]: pgmap v2253: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:39:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:28 smithi138 ceph-mon[36954]: pgmap v2254: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:39:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:28 smithi100 ceph-mon[32599]: pgmap v2254: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:39:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:28 smithi167 ceph-mon[37140]: pgmap v2254: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:39:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:30 smithi138 ceph-mon[36954]: pgmap v2255: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:39:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:30 smithi100 ceph-mon[32599]: pgmap v2255: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:39:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:30 smithi167 ceph-mon[37140]: pgmap v2255: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:39:31.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:39:31.175+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:39:31.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:39:31.176+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:39:31.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:39:31.176+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:39:31.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:39:31.176+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:39:31.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:39:31.176+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:39:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:32 smithi138 ceph-mon[36954]: pgmap v2256: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:39:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:32 smithi100 ceph-mon[32599]: pgmap v2256: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:39:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:32 smithi167 ceph-mon[37140]: pgmap v2256: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:39:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:34 smithi138 ceph-mon[36954]: pgmap v2257: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:39:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:34 smithi100 ceph-mon[32599]: pgmap v2257: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:39:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:34 smithi167 ceph-mon[37140]: pgmap v2257: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:39:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:36 smithi138 ceph-mon[36954]: pgmap v2258: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:39:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:36 smithi100 ceph-mon[32599]: pgmap v2258: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:39:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:36 smithi167 ceph-mon[37140]: pgmap v2258: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:39:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:38 smithi138 ceph-mon[36954]: pgmap v2259: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:39:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:38 smithi100 ceph-mon[32599]: pgmap v2259: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:39:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:38 smithi167 ceph-mon[37140]: pgmap v2259: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:39:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:40 smithi138 ceph-mon[36954]: pgmap v2260: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:39:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:40 smithi100 ceph-mon[32599]: pgmap v2260: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:39:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:40 smithi167 ceph-mon[37140]: pgmap v2260: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:39:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:42 smithi100 ceph-mon[32599]: pgmap v2261: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:39:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:42 smithi167 ceph-mon[37140]: pgmap v2261: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:39:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:42 smithi138 ceph-mon[36954]: pgmap v2261: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:39:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:44 smithi100 ceph-mon[32599]: pgmap v2262: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:39:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:44 smithi167 ceph-mon[37140]: pgmap v2262: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:39:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:44 smithi138 ceph-mon[36954]: pgmap v2262: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:39:46.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:39:46.192+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:39:46.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:39:46.192+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:39:46.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:39:46.192+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:39:46.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:39:46.192+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:39:46.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:39:46.192+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:39:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:46 smithi100 ceph-mon[32599]: pgmap v2263: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:39:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:46 smithi167 ceph-mon[37140]: pgmap v2263: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:39:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:46 smithi138 ceph-mon[36954]: pgmap v2263: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:39:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:48 smithi100 ceph-mon[32599]: pgmap v2264: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:39:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:48 smithi167 ceph-mon[37140]: pgmap v2264: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:39:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:48 smithi138 ceph-mon[36954]: pgmap v2264: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:39:50.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:50 smithi100 ceph-mon[32599]: pgmap v2265: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:39:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:50 smithi167 ceph-mon[37140]: pgmap v2265: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:39:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:50 smithi138 ceph-mon[36954]: pgmap v2265: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:39:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:52 smithi100 ceph-mon[32599]: pgmap v2266: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:39:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:52 smithi167 ceph-mon[37140]: pgmap v2266: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:39:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:52 smithi138 ceph-mon[36954]: pgmap v2266: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:39:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:54 smithi100 ceph-mon[32599]: pgmap v2267: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:39:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:54 smithi167 ceph-mon[37140]: pgmap v2267: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:39:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:54 smithi138 ceph-mon[36954]: pgmap v2267: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:39:55.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:39:56.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:39:56.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:39:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:56 smithi100 ceph-mon[32599]: pgmap v2268: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:39:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:56 smithi167 ceph-mon[37140]: pgmap v2268: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:39:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:56 smithi138 ceph-mon[36954]: pgmap v2268: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:39:58.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:39:58.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:39:58.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:39:58.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:39:58.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:39:58.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:39:58.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:39:58.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:39:58.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:39:58.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:39:58 smithi138 ceph-mon[36954]: pgmap v2269: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:39:58.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:39:58.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:39:58.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:39:58.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:39:58.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:39:58.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:39:58.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:39:58.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:39:58.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:39:58.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:39:58 smithi100 ceph-mon[32599]: pgmap v2269: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:39:58.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:39:58.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:39:58.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:39:58.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:39:58.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:39:58.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:39:58.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:39:58.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:39:58.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:39:58.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:39:58 smithi167 ceph-mon[37140]: pgmap v2269: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:40:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:00 smithi138 ceph-mon[36954]: pgmap v2270: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:40:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:00 smithi138 ceph-mon[36954]: overall HEALTH_OK 2024-07-26T14:40:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:00 smithi100 ceph-mon[32599]: pgmap v2270: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:40:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:00 smithi100 ceph-mon[32599]: overall HEALTH_OK 2024-07-26T14:40:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:00 smithi167 ceph-mon[37140]: pgmap v2270: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:40:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:00 smithi167 ceph-mon[37140]: overall HEALTH_OK 2024-07-26T14:40:01.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:40:01.202+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:40:01.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:40:01.203+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:40:01.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:40:01.203+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:40:01.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:40:01.203+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:40:01.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:40:01.203+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:40:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:02 smithi138 ceph-mon[36954]: pgmap v2271: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:40:02.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:02 smithi100 ceph-mon[32599]: pgmap v2271: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:40:03.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:02 smithi167 ceph-mon[37140]: pgmap v2271: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:40:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:04 smithi138 ceph-mon[36954]: pgmap v2272: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T14:40:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:04 smithi100 ceph-mon[32599]: pgmap v2272: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T14:40:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:04 smithi167 ceph-mon[37140]: pgmap v2272: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T14:40:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:06 smithi138 ceph-mon[36954]: pgmap v2273: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:40:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:06 smithi100 ceph-mon[32599]: pgmap v2273: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:40:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:06 smithi167 ceph-mon[37140]: pgmap v2273: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:40:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:08 smithi138 ceph-mon[36954]: pgmap v2274: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:40:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:08 smithi138 ceph-mon[36954]: osdmap e192: 12 total, 12 up, 12 in 2024-07-26T14:40:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:08 smithi100 ceph-mon[32599]: pgmap v2274: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:40:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:08 smithi100 ceph-mon[32599]: osdmap e192: 12 total, 12 up, 12 in 2024-07-26T14:40:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:08 smithi167 ceph-mon[37140]: pgmap v2274: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:40:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:08 smithi167 ceph-mon[37140]: osdmap e192: 12 total, 12 up, 12 in 2024-07-26T14:40:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:10 smithi138 ceph-mon[36954]: pgmap v2276: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:40:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:10 smithi100 ceph-mon[32599]: pgmap v2276: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:40:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:10 smithi167 ceph-mon[37140]: pgmap v2276: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:40:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:12 smithi138 ceph-mon[36954]: pgmap v2277: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:40:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:12 smithi100 ceph-mon[32599]: pgmap v2277: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:40:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:12 smithi167 ceph-mon[37140]: pgmap v2277: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:40:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:14 smithi138 ceph-mon[36954]: pgmap v2278: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:40:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:14 smithi138 ceph-mon[36954]: osdmap e193: 12 total, 12 up, 12 in 2024-07-26T14:40:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:14 smithi100 ceph-mon[32599]: pgmap v2278: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:40:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:14 smithi100 ceph-mon[32599]: osdmap e193: 12 total, 12 up, 12 in 2024-07-26T14:40:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:14 smithi167 ceph-mon[37140]: pgmap v2278: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:40:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:14 smithi167 ceph-mon[37140]: osdmap e193: 12 total, 12 up, 12 in 2024-07-26T14:40:16.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:40:16.214+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:40:16.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:40:16.214+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:40:16.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:40:16.214+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:40:16.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:40:16.214+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:40:16.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:40:16.215+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:40:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:16 smithi138 ceph-mon[36954]: pgmap v2280: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:40:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:16 smithi100 ceph-mon[32599]: pgmap v2280: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:40:17.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:16 smithi167 ceph-mon[37140]: pgmap v2280: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:40:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:18 smithi138 ceph-mon[36954]: pgmap v2281: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T14:40:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:18 smithi100 ceph-mon[32599]: pgmap v2281: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T14:40:19.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:18 smithi167 ceph-mon[37140]: pgmap v2281: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T14:40:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:20 smithi138 ceph-mon[36954]: pgmap v2282: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:40:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:20 smithi100 ceph-mon[32599]: pgmap v2282: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:40:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:20 smithi167 ceph-mon[37140]: pgmap v2282: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:40:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:22 smithi138 ceph-mon[36954]: pgmap v2283: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:40:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:22 smithi100 ceph-mon[32599]: pgmap v2283: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:40:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:22 smithi167 ceph-mon[37140]: pgmap v2283: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:40:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:24 smithi138 ceph-mon[36954]: pgmap v2284: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:40:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:24 smithi100 ceph-mon[32599]: pgmap v2284: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:40:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:24 smithi167 ceph-mon[37140]: pgmap v2284: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:40:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:26 smithi138 ceph-mon[36954]: pgmap v2285: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:40:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:26 smithi100 ceph-mon[32599]: pgmap v2285: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:40:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:26 smithi167 ceph-mon[37140]: pgmap v2285: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:40:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:28 smithi138 ceph-mon[36954]: pgmap v2286: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:40:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:28 smithi100 ceph-mon[32599]: pgmap v2286: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:40:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:28 smithi167 ceph-mon[37140]: pgmap v2286: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:40:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:30 smithi138 ceph-mon[36954]: pgmap v2287: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:40:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:30 smithi100 ceph-mon[32599]: pgmap v2287: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:40:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:30 smithi167 ceph-mon[37140]: pgmap v2287: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:40:31.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:40:31.223+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:40:31.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:40:31.224+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:40:31.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:40:31.224+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:40:31.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:40:31.224+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:40:31.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:40:31.224+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:40:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:32 smithi138 ceph-mon[36954]: pgmap v2288: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:40:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:32 smithi100 ceph-mon[32599]: pgmap v2288: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:40:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:32 smithi167 ceph-mon[37140]: pgmap v2288: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:40:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:34 smithi138 ceph-mon[36954]: pgmap v2289: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:40:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:34 smithi100 ceph-mon[32599]: pgmap v2289: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:40:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:34 smithi167 ceph-mon[37140]: pgmap v2289: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:40:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:36 smithi138 ceph-mon[36954]: pgmap v2290: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:40:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:36 smithi100 ceph-mon[32599]: pgmap v2290: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:40:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:36 smithi167 ceph-mon[37140]: pgmap v2290: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:40:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:38 smithi138 ceph-mon[36954]: pgmap v2291: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:40:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:38 smithi100 ceph-mon[32599]: pgmap v2291: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:40:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:38 smithi167 ceph-mon[37140]: pgmap v2291: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:40:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:40 smithi138 ceph-mon[36954]: pgmap v2292: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:40:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:40 smithi100 ceph-mon[32599]: pgmap v2292: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:40:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:40 smithi167 ceph-mon[37140]: pgmap v2292: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:40:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:42 smithi138 ceph-mon[36954]: pgmap v2293: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:40:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:42 smithi100 ceph-mon[32599]: pgmap v2293: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:40:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:42 smithi167 ceph-mon[37140]: pgmap v2293: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:40:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:44 smithi138 ceph-mon[36954]: pgmap v2294: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:40:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:44 smithi100 ceph-mon[32599]: pgmap v2294: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:40:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:44 smithi167 ceph-mon[37140]: pgmap v2294: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:40:46.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:40:46.225+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:40:46.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:40:46.226+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:40:46.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:40:46.226+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:40:46.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:40:46.226+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:40:46.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:40:46.226+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:40:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:46 smithi138 ceph-mon[36954]: pgmap v2295: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:40:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:46 smithi100 ceph-mon[32599]: pgmap v2295: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:40:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:46 smithi167 ceph-mon[37140]: pgmap v2295: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:40:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:48 smithi138 ceph-mon[36954]: pgmap v2296: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:40:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:48 smithi100 ceph-mon[32599]: pgmap v2296: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:40:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:48 smithi167 ceph-mon[37140]: pgmap v2296: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:40:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:50 smithi138 ceph-mon[36954]: pgmap v2297: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:40:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:50 smithi100 ceph-mon[32599]: pgmap v2297: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:40:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:50 smithi167 ceph-mon[37140]: pgmap v2297: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:40:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:52 smithi138 ceph-mon[36954]: pgmap v2298: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:40:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:52 smithi100 ceph-mon[32599]: pgmap v2298: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:40:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:52 smithi167 ceph-mon[37140]: pgmap v2298: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:40:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:54 smithi138 ceph-mon[36954]: pgmap v2299: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T14:40:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:54 smithi100 ceph-mon[32599]: pgmap v2299: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T14:40:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:54 smithi167 ceph-mon[37140]: pgmap v2299: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T14:40:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:56 smithi138 ceph-mon[36954]: pgmap v2300: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:40:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:56 smithi100 ceph-mon[32599]: pgmap v2300: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:40:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:56 smithi167 ceph-mon[37140]: pgmap v2300: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:40:57.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:40:57.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:40:58.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:40:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:58 smithi100 ceph-mon[32599]: pgmap v2301: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:40:59.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:40:59.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:40:59.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:40:59.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:40:59.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:40:59.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:40:59.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:40:59.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:40:59.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:40:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:40:59.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:58 smithi167 ceph-mon[37140]: pgmap v2301: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:40:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:40:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:40:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:40:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:40:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:40:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:40:59.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:40:59.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:40:59.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:40:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:40:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:58 smithi138 ceph-mon[36954]: pgmap v2301: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:40:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:40:59.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:40:59.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:40:59.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:40:59.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:40:59.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:40:59.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:40:59.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:40:59.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:40:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:41:01.234 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:41:00 smithi100 ceph-mon[32599]: pgmap v2302: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:41:01.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:41:01.231+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:41:01.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:41:01.231+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:41:01.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:41:01.231+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:41:01.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:41:01.231+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:41:01.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:41:01.232+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:41:01.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:41:00 smithi167 ceph-mon[37140]: pgmap v2302: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:41:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:41:00 smithi138 ceph-mon[36954]: pgmap v2302: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:41:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:41:02 smithi100 ceph-mon[32599]: pgmap v2303: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:41:03.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:41:02 smithi167 ceph-mon[37140]: pgmap v2303: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:41:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:41:02 smithi138 ceph-mon[36954]: pgmap v2303: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:41:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:41:04 smithi100 ceph-mon[32599]: pgmap v2304: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-07-26T14:41:05.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:41:04 smithi167 ceph-mon[37140]: pgmap v2304: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-07-26T14:41:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:41:04 smithi138 ceph-mon[36954]: pgmap v2304: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-07-26T14:41:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:41:06 smithi100 ceph-mon[32599]: pgmap v2305: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:41:07.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:41:06 smithi167 ceph-mon[37140]: pgmap v2305: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:41:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:41:06 smithi138 ceph-mon[36954]: pgmap v2305: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:41:08.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:41:07 smithi167 ceph-mon[37140]: osdmap e194: 12 total, 12 up, 12 in 2024-07-26T14:41:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:41:08 smithi138 ceph-mon[36954]: osdmap e194: 12 total, 12 up, 12 in 2024-07-26T14:41:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:41:07 smithi100 ceph-mon[32599]: osdmap e194: 12 total, 12 up, 12 in 2024-07-26T14:41:09.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:41:09 smithi167 ceph-mon[37140]: pgmap v2307: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:41:09.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:41:09 smithi138 ceph-mon[36954]: pgmap v2307: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:41:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:41:09 smithi100 ceph-mon[32599]: pgmap v2307: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:41:11.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:41:11 smithi167 ceph-mon[37140]: pgmap v2308: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:41:11.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:41:11 smithi138 ceph-mon[36954]: pgmap v2308: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:41:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:41:11 smithi100 ceph-mon[32599]: pgmap v2308: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:41:13.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:41:13 smithi167 ceph-mon[37140]: pgmap v2309: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:41:13.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:41:13 smithi138 ceph-mon[36954]: pgmap v2309: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:41:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:41:13 smithi100 ceph-mon[32599]: pgmap v2309: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:41:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:41:15 smithi167 ceph-mon[37140]: pgmap v2310: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:41:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:41:15 smithi167 ceph-mon[37140]: osdmap e195: 12 total, 12 up, 12 in 2024-07-26T14:41:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:41:15 smithi138 ceph-mon[36954]: pgmap v2310: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:41:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:41:15 smithi138 ceph-mon[36954]: osdmap e195: 12 total, 12 up, 12 in 2024-07-26T14:41:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:41:15 smithi100 ceph-mon[32599]: pgmap v2310: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:41:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:41:15 smithi100 ceph-mon[32599]: osdmap e195: 12 total, 12 up, 12 in 2024-07-26T14:41:16.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:41:16.247+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:41:16.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:41:16.248+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:41:16.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:41:16.248+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:41:16.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:41:16.248+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:41:16.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:41:16.248+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:41:17.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:41:17 smithi167 ceph-mon[37140]: pgmap v2312: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 237 B/s wr, 6 op/s 2024-07-26T14:41:17.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:41:17 smithi138 ceph-mon[36954]: pgmap v2312: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 237 B/s wr, 6 op/s 2024-07-26T14:41:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:41:17 smithi100 ceph-mon[32599]: pgmap v2312: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 237 B/s wr, 6 op/s 2024-07-26T14:41:19.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:41:19 smithi167 ceph-mon[37140]: pgmap v2313: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:41:19.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:41:19 smithi138 ceph-mon[36954]: pgmap v2313: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:41:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:41:19 smithi100 ceph-mon[32599]: pgmap v2313: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:41:21.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:41:21 smithi138 ceph-mon[36954]: pgmap v2314: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:41:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:41:21 smithi100 ceph-mon[32599]: pgmap v2314: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:41:21.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:41:21 smithi167 ceph-mon[37140]: pgmap v2314: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:41:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:41:23 smithi138 ceph-mon[36954]: pgmap v2315: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:41:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:41:23 smithi100 ceph-mon[32599]: pgmap v2315: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:41:23.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:41:23 smithi167 ceph-mon[37140]: pgmap v2315: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:41:25.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:41:25 smithi138 ceph-mon[36954]: pgmap v2316: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:41:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:41:25 smithi100 ceph-mon[32599]: pgmap v2316: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:41:25.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:41:25 smithi167 ceph-mon[37140]: pgmap v2316: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:41:27.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:41:27 smithi138 ceph-mon[36954]: pgmap v2317: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:41:27.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:41:27 smithi100 ceph-mon[32599]: pgmap v2317: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:41:27.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:41:27 smithi167 ceph-mon[37140]: pgmap v2317: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:41:29.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:41:29 smithi138 ceph-mon[36954]: pgmap v2318: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:41:29.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:41:29 smithi100 ceph-mon[32599]: pgmap v2318: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:41:29.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:41:29 smithi167 ceph-mon[37140]: pgmap v2318: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:41:31.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:41:31.263+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:41:31.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:41:31.263+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:41:31.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:41:31.263+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:41:31.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:41:31.263+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:41:31.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:41:31.264+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:41:31.296 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:41:31 smithi100 ceph-mon[32599]: pgmap v2319: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:41:31.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:41:31 smithi138 ceph-mon[36954]: pgmap v2319: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:41:31.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:41:31 smithi167 ceph-mon[37140]: pgmap v2319: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:41:33.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:41:33 smithi138 ceph-mon[36954]: pgmap v2320: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:41:33.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:41:33 smithi100 ceph-mon[32599]: pgmap v2320: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:41:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:41:33 smithi167 ceph-mon[37140]: pgmap v2320: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:41:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:41:35 smithi138 ceph-mon[36954]: pgmap v2321: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:41:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:41:35 smithi100 ceph-mon[32599]: pgmap v2321: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:41:35.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:41:35 smithi167 ceph-mon[37140]: pgmap v2321: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:41:37.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:41:37 smithi138 ceph-mon[36954]: pgmap v2322: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:41:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:41:37 smithi100 ceph-mon[32599]: pgmap v2322: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:41:37.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:41:37 smithi167 ceph-mon[37140]: pgmap v2322: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:41:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:41:39 smithi138 ceph-mon[36954]: pgmap v2323: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:41:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:41:39 smithi100 ceph-mon[32599]: pgmap v2323: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:41:39.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:41:39 smithi167 ceph-mon[37140]: pgmap v2323: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:41:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:41:41 smithi138 ceph-mon[36954]: pgmap v2324: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:41:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:41:41 smithi100 ceph-mon[32599]: pgmap v2324: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:41:41.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:41:41 smithi167 ceph-mon[37140]: pgmap v2324: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:41:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:41:43 smithi138 ceph-mon[36954]: pgmap v2325: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:41:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:41:43 smithi100 ceph-mon[32599]: pgmap v2325: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:41:43.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:41:43 smithi167 ceph-mon[37140]: pgmap v2325: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:41:45.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:41:45 smithi138 ceph-mon[36954]: pgmap v2326: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:41:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:41:45 smithi100 ceph-mon[32599]: pgmap v2326: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:41:45.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:41:45 smithi167 ceph-mon[37140]: pgmap v2326: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:41:46.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:41:46.279+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:41:46.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:41:46.280+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:41:46.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:41:46.280+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:41:46.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:41:46.280+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:41:46.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:41:46.280+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:41:47.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:41:47 smithi138 ceph-mon[36954]: pgmap v2327: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:41:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:41:47 smithi100 ceph-mon[32599]: pgmap v2327: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:41:47.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:41:47 smithi167 ceph-mon[37140]: pgmap v2327: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:41:49.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:41:49 smithi167 ceph-mon[37140]: pgmap v2328: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:41:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:41:49 smithi138 ceph-mon[36954]: pgmap v2328: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:41:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:41:49 smithi100 ceph-mon[32599]: pgmap v2328: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:41:51.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:41:51 smithi138 ceph-mon[36954]: pgmap v2329: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:41:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:41:51 smithi100 ceph-mon[32599]: pgmap v2329: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:41:51.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:41:51 smithi167 ceph-mon[37140]: pgmap v2329: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:41:53.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:41:53 smithi138 ceph-mon[36954]: pgmap v2330: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:41:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:41:53 smithi100 ceph-mon[32599]: pgmap v2330: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:41:53.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:41:53 smithi167 ceph-mon[37140]: pgmap v2330: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:41:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:41:55 smithi138 ceph-mon[36954]: pgmap v2331: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:41:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:41:55 smithi100 ceph-mon[32599]: pgmap v2331: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:41:55.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:41:55 smithi167 ceph-mon[37140]: pgmap v2331: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:41:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:41:57 smithi138 ceph-mon[36954]: pgmap v2332: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:41:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:41:57 smithi100 ceph-mon[32599]: pgmap v2332: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:41:57.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:41:57 smithi167 ceph-mon[37140]: pgmap v2332: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:41:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:41:59 smithi138 ceph-mon[36954]: pgmap v2333: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:41:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:41:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:41:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:41:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:41:59.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:41:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:41:59.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:41:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:41:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:41:59 smithi100 ceph-mon[32599]: pgmap v2333: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:41:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:41:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:41:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:41:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:41:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:41:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:41:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:41:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:41:59.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:41:59 smithi167 ceph-mon[37140]: pgmap v2333: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:41:59.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:41:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:41:59.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:41:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:41:59.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:41:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:41:59.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:41:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:42:01.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:42:01.295+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:42:01.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:42:01.296+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:42:01.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:42:01.296+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:42:01.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:42:01.296+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:42:01.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:42:01.296+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:42:01.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:42:01 smithi138 ceph-mon[36954]: pgmap v2334: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:42:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:42:01 smithi100 ceph-mon[32599]: pgmap v2334: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:42:01.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:42:01 smithi167 ceph-mon[37140]: pgmap v2334: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:42:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:42:03 smithi138 ceph-mon[36954]: pgmap v2335: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:42:03.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:42:03.471+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T14:42:03.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:42:03.471+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c006610 con 0x7fe5a81087a0 2024-07-26T14:42:03.472 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:42:03 smithi100 ceph-mon[32599]: pgmap v2335: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:42:03.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:42:03.471+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 29 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T14:42:03.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:42:03.471+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 29 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c020 con 0x7fe5a81087a0 2024-07-26T14:42:03.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:42:03 smithi167 ceph-mon[37140]: pgmap v2335: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:42:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:42:05 smithi138 ceph-mon[36954]: pgmap v2336: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T14:42:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:42:05 smithi100 ceph-mon[32599]: pgmap v2336: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T14:42:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:42:05 smithi167 ceph-mon[37140]: pgmap v2336: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T14:42:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:42:07 smithi138 ceph-mon[36954]: pgmap v2337: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:42:07.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:42:07 smithi100 ceph-mon[32599]: pgmap v2337: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:42:07.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:42:07 smithi167 ceph-mon[37140]: pgmap v2337: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:42:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:42:08 smithi138 ceph-mon[36954]: osdmap e196: 12 total, 12 up, 12 in 2024-07-26T14:42:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:42:08 smithi100 ceph-mon[32599]: osdmap e196: 12 total, 12 up, 12 in 2024-07-26T14:42:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:42:08 smithi167 ceph-mon[37140]: osdmap e196: 12 total, 12 up, 12 in 2024-07-26T14:42:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:42:09 smithi138 ceph-mon[36954]: pgmap v2339: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:42:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:42:09 smithi100 ceph-mon[32599]: pgmap v2339: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:42:09.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:42:09 smithi167 ceph-mon[37140]: pgmap v2339: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:42:10.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:42:10 smithi138 ceph-mon[36954]: pgmap v2340: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:42:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:42:10 smithi100 ceph-mon[32599]: pgmap v2340: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:42:10.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:42:10 smithi167 ceph-mon[37140]: pgmap v2340: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:42:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:42:12 smithi138 ceph-mon[36954]: pgmap v2341: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:42:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:42:12 smithi100 ceph-mon[32599]: pgmap v2341: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:42:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:42:12 smithi167 ceph-mon[37140]: pgmap v2341: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:42:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:42:14 smithi138 ceph-mon[36954]: pgmap v2342: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:42:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:42:14 smithi138 ceph-mon[36954]: osdmap e197: 12 total, 12 up, 12 in 2024-07-26T14:42:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:42:14 smithi100 ceph-mon[32599]: pgmap v2342: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:42:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:42:14 smithi100 ceph-mon[32599]: osdmap e197: 12 total, 12 up, 12 in 2024-07-26T14:42:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:42:14 smithi167 ceph-mon[37140]: pgmap v2342: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:42:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:42:14 smithi167 ceph-mon[37140]: osdmap e197: 12 total, 12 up, 12 in 2024-07-26T14:42:16.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:42:16.298+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:42:16.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:42:16.299+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:42:16.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:42:16.299+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:42:16.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:42:16.299+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:42:16.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:42:16.299+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:42:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:42:16 smithi138 ceph-mon[36954]: pgmap v2344: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 240 B/s wr, 6 op/s 2024-07-26T14:42:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:42:16 smithi100 ceph-mon[32599]: pgmap v2344: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 240 B/s wr, 6 op/s 2024-07-26T14:42:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:42:16 smithi167 ceph-mon[37140]: pgmap v2344: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 240 B/s wr, 6 op/s 2024-07-26T14:42:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:42:18 smithi138 ceph-mon[36954]: pgmap v2345: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:42:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:42:18 smithi100 ceph-mon[32599]: pgmap v2345: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:42:19.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:42:18 smithi167 ceph-mon[37140]: pgmap v2345: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:42:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:42:20 smithi100 ceph-mon[32599]: pgmap v2346: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:42:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:42:20 smithi167 ceph-mon[37140]: pgmap v2346: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:42:21.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:42:20 smithi138 ceph-mon[36954]: pgmap v2346: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:42:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:42:22 smithi100 ceph-mon[32599]: pgmap v2347: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:42:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:42:22 smithi167 ceph-mon[37140]: pgmap v2347: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:42:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:42:22 smithi138 ceph-mon[36954]: pgmap v2347: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:42:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:42:24 smithi100 ceph-mon[32599]: pgmap v2348: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:42:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:42:24 smithi167 ceph-mon[37140]: pgmap v2348: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:42:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:42:24 smithi138 ceph-mon[36954]: pgmap v2348: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:42:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:42:26 smithi100 ceph-mon[32599]: pgmap v2349: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:42:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:42:26 smithi167 ceph-mon[37140]: pgmap v2349: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:42:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:42:26 smithi138 ceph-mon[36954]: pgmap v2349: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:42:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:42:28 smithi100 ceph-mon[32599]: pgmap v2350: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:42:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:42:28 smithi167 ceph-mon[37140]: pgmap v2350: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:42:29.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:42:28 smithi138 ceph-mon[36954]: pgmap v2350: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:42:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:42:30 smithi100 ceph-mon[32599]: pgmap v2351: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:42:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:42:30 smithi167 ceph-mon[37140]: pgmap v2351: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:42:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:42:30 smithi138 ceph-mon[36954]: pgmap v2351: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:42:31.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:42:31.314+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:42:31.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:42:31.315+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:42:31.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:42:31.315+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:42:31.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:42:31.315+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:42:31.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:42:31.315+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:42:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:42:32 smithi100 ceph-mon[32599]: pgmap v2352: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:42:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:42:32 smithi167 ceph-mon[37140]: pgmap v2352: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:42:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:42:32 smithi138 ceph-mon[36954]: pgmap v2352: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:42:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:42:34 smithi100 ceph-mon[32599]: pgmap v2353: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:42:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:42:34 smithi167 ceph-mon[37140]: pgmap v2353: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:42:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:42:34 smithi138 ceph-mon[36954]: pgmap v2353: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:42:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:42:36 smithi100 ceph-mon[32599]: pgmap v2354: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:42:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:42:36 smithi167 ceph-mon[37140]: pgmap v2354: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:42:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:42:36 smithi138 ceph-mon[36954]: pgmap v2354: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:42:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:42:38 smithi100 ceph-mon[32599]: pgmap v2355: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:42:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:42:38 smithi167 ceph-mon[37140]: pgmap v2355: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:42:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:42:38 smithi138 ceph-mon[36954]: pgmap v2355: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:42:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:42:40 smithi100 ceph-mon[32599]: pgmap v2356: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:42:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:42:40 smithi167 ceph-mon[37140]: pgmap v2356: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:42:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:42:40 smithi138 ceph-mon[36954]: pgmap v2356: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:42:42.992 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:42:42 smithi138 ceph-mon[36954]: pgmap v2357: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:42:42.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:42:42 smithi100 ceph-mon[32599]: pgmap v2357: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:42:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:42:42 smithi167 ceph-mon[37140]: pgmap v2357: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:42:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:42:44 smithi100 ceph-mon[32599]: pgmap v2358: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:42:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:42:44 smithi167 ceph-mon[37140]: pgmap v2358: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:42:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:42:44 smithi138 ceph-mon[36954]: pgmap v2358: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:42:46.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:42:46.324+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:42:46.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:42:46.325+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:42:46.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:42:46.325+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:42:46.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:42:46.325+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:42:46.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:42:46.325+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:42:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:42:46 smithi100 ceph-mon[32599]: pgmap v2359: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:42:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:42:46 smithi167 ceph-mon[37140]: pgmap v2359: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:42:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:42:46 smithi138 ceph-mon[36954]: pgmap v2359: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:42:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:42:48 smithi167 ceph-mon[37140]: pgmap v2360: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:42:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:42:48 smithi138 ceph-mon[36954]: pgmap v2360: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:42:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:42:48 smithi100 ceph-mon[32599]: pgmap v2360: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:42:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:42:50 smithi167 ceph-mon[37140]: pgmap v2361: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:42:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:42:50 smithi138 ceph-mon[36954]: pgmap v2361: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:42:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:42:50 smithi100 ceph-mon[32599]: pgmap v2361: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:42:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:42:52 smithi167 ceph-mon[37140]: pgmap v2362: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:42:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:42:52 smithi138 ceph-mon[36954]: pgmap v2362: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:42:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:42:52 smithi100 ceph-mon[32599]: pgmap v2362: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:42:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:42:54 smithi167 ceph-mon[37140]: pgmap v2363: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:42:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:42:54 smithi138 ceph-mon[36954]: pgmap v2363: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:42:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:42:54 smithi100 ceph-mon[32599]: pgmap v2363: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:42:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:42:56 smithi167 ceph-mon[37140]: pgmap v2364: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:42:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:42:56 smithi138 ceph-mon[36954]: pgmap v2364: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:42:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:42:56 smithi100 ceph-mon[32599]: pgmap v2364: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:42:59.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:42:58 smithi167 ceph-mon[37140]: pgmap v2365: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:42:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:42:58 smithi138 ceph-mon[36954]: pgmap v2365: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:42:59.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:42:58 smithi100 ceph-mon[32599]: pgmap v2365: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:43:00.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:42:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:43:00.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:42:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:43:00.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:42:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:43:00.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:42:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:43:00.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:42:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:43:00.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:42:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:43:00.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:42:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:43:00.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:42:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:43:00.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:42:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:43:00.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:42:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:43:00.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:42:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:43:00.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:42:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:43:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:43:00 smithi167 ceph-mon[37140]: pgmap v2366: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:43:01.145 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:43:00 smithi100 ceph-mon[32599]: pgmap v2366: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:43:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:43:00 smithi138 ceph-mon[36954]: pgmap v2366: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:43:01.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:43:01.338+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:43:01.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:43:01.338+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:43:01.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:43:01.338+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:43:01.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:43:01.338+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:43:01.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:43:01.338+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:43:03.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:43:02 smithi167 ceph-mon[37140]: pgmap v2367: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:43:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:43:02 smithi138 ceph-mon[36954]: pgmap v2367: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:43:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:43:02 smithi100 ceph-mon[32599]: pgmap v2367: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:43:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:43:04 smithi167 ceph-mon[37140]: pgmap v2368: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T14:43:05.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:43:04 smithi138 ceph-mon[36954]: pgmap v2368: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T14:43:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:43:04 smithi100 ceph-mon[32599]: pgmap v2368: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T14:43:07.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:43:06 smithi138 ceph-mon[36954]: pgmap v2369: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:43:07.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:43:06 smithi100 ceph-mon[32599]: pgmap v2369: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:43:07.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:43:06 smithi167 ceph-mon[37140]: pgmap v2369: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:43:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:43:08 smithi138 ceph-mon[36954]: pgmap v2370: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:43:09.193 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:43:08 smithi138 ceph-mon[36954]: osdmap e198: 12 total, 12 up, 12 in 2024-07-26T14:43:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:43:08 smithi100 ceph-mon[32599]: pgmap v2370: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:43:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:43:08 smithi100 ceph-mon[32599]: osdmap e198: 12 total, 12 up, 12 in 2024-07-26T14:43:09.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:43:08 smithi167 ceph-mon[37140]: pgmap v2370: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:43:09.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:43:08 smithi167 ceph-mon[37140]: osdmap e198: 12 total, 12 up, 12 in 2024-07-26T14:43:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:43:10 smithi138 ceph-mon[36954]: pgmap v2372: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:43:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:43:10 smithi100 ceph-mon[32599]: pgmap v2372: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:43:11.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:43:10 smithi167 ceph-mon[37140]: pgmap v2372: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:43:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:43:12 smithi138 ceph-mon[36954]: pgmap v2373: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:43:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:43:12 smithi100 ceph-mon[32599]: pgmap v2373: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:43:13.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:43:12 smithi167 ceph-mon[37140]: pgmap v2373: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:43:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:43:14 smithi138 ceph-mon[36954]: pgmap v2374: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:43:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:43:14 smithi138 ceph-mon[36954]: osdmap e199: 12 total, 12 up, 12 in 2024-07-26T14:43:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:43:14 smithi100 ceph-mon[32599]: pgmap v2374: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:43:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:43:14 smithi100 ceph-mon[32599]: osdmap e199: 12 total, 12 up, 12 in 2024-07-26T14:43:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:43:14 smithi167 ceph-mon[37140]: pgmap v2374: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:43:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:43:14 smithi167 ceph-mon[37140]: osdmap e199: 12 total, 12 up, 12 in 2024-07-26T14:43:16.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:43:16.343+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:43:16.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:43:16.344+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:43:16.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:43:16.344+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:43:16.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:43:16.344+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:43:16.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:43:16.344+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:43:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:43:16 smithi138 ceph-mon[36954]: pgmap v2376: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:43:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:43:16 smithi100 ceph-mon[32599]: pgmap v2376: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:43:17.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:43:16 smithi167 ceph-mon[37140]: pgmap v2376: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:43:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:43:18 smithi138 ceph-mon[36954]: pgmap v2377: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T14:43:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:43:18 smithi100 ceph-mon[32599]: pgmap v2377: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T14:43:19.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:43:18 smithi167 ceph-mon[37140]: pgmap v2377: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T14:43:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:43:20 smithi138 ceph-mon[36954]: pgmap v2378: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:43:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:43:20 smithi100 ceph-mon[32599]: pgmap v2378: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:43:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:43:20 smithi167 ceph-mon[37140]: pgmap v2378: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:43:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:43:22 smithi100 ceph-mon[32599]: pgmap v2379: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:43:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:43:22 smithi167 ceph-mon[37140]: pgmap v2379: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:43:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:43:22 smithi138 ceph-mon[36954]: pgmap v2379: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:43:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:43:24 smithi100 ceph-mon[32599]: pgmap v2380: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:43:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:43:24 smithi167 ceph-mon[37140]: pgmap v2380: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:43:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:43:24 smithi138 ceph-mon[36954]: pgmap v2380: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:43:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:43:26 smithi100 ceph-mon[32599]: pgmap v2381: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:43:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:43:26 smithi167 ceph-mon[37140]: pgmap v2381: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:43:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:43:26 smithi138 ceph-mon[36954]: pgmap v2381: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:43:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:43:28 smithi100 ceph-mon[32599]: pgmap v2382: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:43:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:43:28 smithi167 ceph-mon[37140]: pgmap v2382: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:43:29.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:43:28 smithi138 ceph-mon[36954]: pgmap v2382: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:43:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:43:30 smithi100 ceph-mon[32599]: pgmap v2383: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:43:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:43:30 smithi167 ceph-mon[37140]: pgmap v2383: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:43:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:43:30 smithi138 ceph-mon[36954]: pgmap v2383: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:43:31.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:43:31.359+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:43:31.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:43:31.360+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:43:31.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:43:31.360+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:43:31.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:43:31.360+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:43:31.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:43:31.360+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:43:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:43:32 smithi100 ceph-mon[32599]: pgmap v2384: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:43:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:43:32 smithi167 ceph-mon[37140]: pgmap v2384: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:43:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:43:32 smithi138 ceph-mon[36954]: pgmap v2384: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:43:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:43:34 smithi100 ceph-mon[32599]: pgmap v2385: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:43:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:43:34 smithi167 ceph-mon[37140]: pgmap v2385: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:43:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:43:34 smithi138 ceph-mon[36954]: pgmap v2385: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:43:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:43:36 smithi100 ceph-mon[32599]: pgmap v2386: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:43:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:43:36 smithi167 ceph-mon[37140]: pgmap v2386: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:43:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:43:36 smithi138 ceph-mon[36954]: pgmap v2386: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:43:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:43:38 smithi100 ceph-mon[32599]: pgmap v2387: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:43:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:43:38 smithi167 ceph-mon[37140]: pgmap v2387: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:43:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:43:38 smithi138 ceph-mon[36954]: pgmap v2387: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:43:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:43:40 smithi100 ceph-mon[32599]: pgmap v2388: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:43:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:43:40 smithi167 ceph-mon[37140]: pgmap v2388: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:43:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:43:40 smithi138 ceph-mon[36954]: pgmap v2388: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:43:42.992 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:43:42 smithi138 ceph-mon[36954]: pgmap v2389: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:43:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:43:42 smithi100 ceph-mon[32599]: pgmap v2389: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:43:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:43:42 smithi167 ceph-mon[37140]: pgmap v2389: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:43:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:43:44 smithi100 ceph-mon[32599]: pgmap v2390: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:43:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:43:44 smithi167 ceph-mon[37140]: pgmap v2390: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:43:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:43:44 smithi138 ceph-mon[36954]: pgmap v2390: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:43:46.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:43:46.376+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:43:46.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:43:46.376+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:43:46.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:43:46.376+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:43:46.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:43:46.376+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:43:46.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:43:46.376+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:43:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:43:46 smithi100 ceph-mon[32599]: pgmap v2391: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:43:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:43:46 smithi167 ceph-mon[37140]: pgmap v2391: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:43:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:43:46 smithi138 ceph-mon[36954]: pgmap v2391: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:43:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:43:48 smithi167 ceph-mon[37140]: pgmap v2392: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:43:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:43:48 smithi138 ceph-mon[36954]: pgmap v2392: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:43:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:43:48 smithi100 ceph-mon[32599]: pgmap v2392: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:43:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:43:50 smithi167 ceph-mon[37140]: pgmap v2393: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:43:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:43:50 smithi138 ceph-mon[36954]: pgmap v2393: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:43:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:43:50 smithi100 ceph-mon[32599]: pgmap v2393: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:43:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:43:52 smithi167 ceph-mon[37140]: pgmap v2394: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:43:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:43:52 smithi138 ceph-mon[36954]: pgmap v2394: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:43:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:43:52 smithi100 ceph-mon[32599]: pgmap v2394: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:43:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:43:54 smithi167 ceph-mon[37140]: pgmap v2395: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:43:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:43:54 smithi138 ceph-mon[36954]: pgmap v2395: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:43:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:43:54 smithi100 ceph-mon[32599]: pgmap v2395: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:43:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:43:56 smithi167 ceph-mon[37140]: pgmap v2396: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:43:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:43:56 smithi138 ceph-mon[36954]: pgmap v2396: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:43:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:43:56 smithi100 ceph-mon[32599]: pgmap v2396: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:43:59.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:43:58 smithi167 ceph-mon[37140]: pgmap v2397: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:43:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:43:58 smithi138 ceph-mon[36954]: pgmap v2397: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:43:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:43:58 smithi100 ceph-mon[32599]: pgmap v2397: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:44:00.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:43:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:44:00.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:43:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:44:00.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:43:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:44:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:44:00 smithi167 ceph-mon[37140]: pgmap v2398: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:44:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:44:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:44:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:44:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:44:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:44:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:44:01.145 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:44:00 smithi100 ceph-mon[32599]: pgmap v2398: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:44:01.145 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:44:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:44:01.145 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:44:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:44:01.146 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:44:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:44:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:44:00 smithi138 ceph-mon[36954]: pgmap v2398: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:44:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:44:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:44:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:44:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:44:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:44:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:44:01.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:44:01.385+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:44:01.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:44:01.385+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:44:01.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:44:01.385+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:44:01.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:44:01.385+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:44:01.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:44:01.385+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:44:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:44:02 smithi138 ceph-mon[36954]: pgmap v2399: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:44:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:44:02 smithi100 ceph-mon[32599]: pgmap v2399: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:44:03.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:44:02 smithi167 ceph-mon[37140]: pgmap v2399: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:44:05.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:44:04 smithi138 ceph-mon[36954]: pgmap v2400: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T14:44:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:44:04 smithi100 ceph-mon[32599]: pgmap v2400: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T14:44:05.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:44:04 smithi167 ceph-mon[37140]: pgmap v2400: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T14:44:07.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:44:06 smithi138 ceph-mon[36954]: pgmap v2401: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:44:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:44:06 smithi100 ceph-mon[32599]: pgmap v2401: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:44:07.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:44:06 smithi167 ceph-mon[37140]: pgmap v2401: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:44:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:44:08 smithi138 ceph-mon[36954]: pgmap v2402: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:44:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:44:08 smithi138 ceph-mon[36954]: osdmap e200: 12 total, 12 up, 12 in 2024-07-26T14:44:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:44:08 smithi100 ceph-mon[32599]: pgmap v2402: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:44:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:44:08 smithi100 ceph-mon[32599]: osdmap e200: 12 total, 12 up, 12 in 2024-07-26T14:44:09.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:44:08 smithi167 ceph-mon[37140]: pgmap v2402: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:44:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:44:08 smithi167 ceph-mon[37140]: osdmap e200: 12 total, 12 up, 12 in 2024-07-26T14:44:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:44:10 smithi138 ceph-mon[36954]: pgmap v2404: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:44:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:44:10 smithi100 ceph-mon[32599]: pgmap v2404: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:44:11.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:44:10 smithi167 ceph-mon[37140]: pgmap v2404: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:44:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:44:12 smithi138 ceph-mon[36954]: pgmap v2405: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:44:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:44:12 smithi100 ceph-mon[32599]: pgmap v2405: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:44:13.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:44:12 smithi167 ceph-mon[37140]: pgmap v2405: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:44:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:44:14 smithi138 ceph-mon[36954]: pgmap v2406: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:44:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:44:14 smithi138 ceph-mon[36954]: osdmap e201: 12 total, 12 up, 12 in 2024-07-26T14:44:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:44:14 smithi100 ceph-mon[32599]: pgmap v2406: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:44:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:44:14 smithi100 ceph-mon[32599]: osdmap e201: 12 total, 12 up, 12 in 2024-07-26T14:44:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:44:14 smithi167 ceph-mon[37140]: pgmap v2406: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:44:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:44:14 smithi167 ceph-mon[37140]: osdmap e201: 12 total, 12 up, 12 in 2024-07-26T14:44:16.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:44:16.397+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:44:16.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:44:16.397+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:44:16.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:44:16.397+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:44:16.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:44:16.397+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:44:16.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:44:16.397+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:44:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:44:16 smithi138 ceph-mon[36954]: pgmap v2408: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:44:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:44:16 smithi100 ceph-mon[32599]: pgmap v2408: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:44:17.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:44:16 smithi167 ceph-mon[37140]: pgmap v2408: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:44:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:44:18 smithi138 ceph-mon[36954]: pgmap v2409: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T14:44:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:44:18 smithi100 ceph-mon[32599]: pgmap v2409: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T14:44:19.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:44:18 smithi167 ceph-mon[37140]: pgmap v2409: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T14:44:21.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:44:20 smithi138 ceph-mon[36954]: pgmap v2410: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:44:21.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:44:20 smithi100 ceph-mon[32599]: pgmap v2410: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:44:21.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:44:20 smithi167 ceph-mon[37140]: pgmap v2410: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:44:23.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:44:22 smithi167 ceph-mon[37140]: pgmap v2411: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:44:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:44:22 smithi138 ceph-mon[36954]: pgmap v2411: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:44:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:44:22 smithi100 ceph-mon[32599]: pgmap v2411: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:44:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:44:24 smithi138 ceph-mon[36954]: pgmap v2412: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:44:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:44:24 smithi100 ceph-mon[32599]: pgmap v2412: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:44:25.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:44:24 smithi167 ceph-mon[37140]: pgmap v2412: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:44:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:44:26 smithi138 ceph-mon[36954]: pgmap v2413: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:44:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:44:26 smithi100 ceph-mon[32599]: pgmap v2413: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:44:27.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:44:26 smithi167 ceph-mon[37140]: pgmap v2413: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:44:29.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:44:28 smithi138 ceph-mon[36954]: pgmap v2414: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:44:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:44:28 smithi100 ceph-mon[32599]: pgmap v2414: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:44:29.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:44:28 smithi167 ceph-mon[37140]: pgmap v2414: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:44:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:44:30 smithi138 ceph-mon[36954]: pgmap v2415: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:44:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:44:30 smithi100 ceph-mon[32599]: pgmap v2415: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:44:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:44:30 smithi167 ceph-mon[37140]: pgmap v2415: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:44:31.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:44:31.414+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:44:31.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:44:31.414+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:44:31.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:44:31.414+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:44:31.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:44:31.414+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:44:31.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:44:31.414+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:44:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:44:32 smithi100 ceph-mon[32599]: pgmap v2416: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:44:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:44:32 smithi167 ceph-mon[37140]: pgmap v2416: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:44:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:44:32 smithi138 ceph-mon[36954]: pgmap v2416: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:44:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:44:34 smithi100 ceph-mon[32599]: pgmap v2417: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:44:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:44:34 smithi167 ceph-mon[37140]: pgmap v2417: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:44:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:44:34 smithi138 ceph-mon[36954]: pgmap v2417: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:44:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:44:36 smithi100 ceph-mon[32599]: pgmap v2418: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:44:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:44:36 smithi167 ceph-mon[37140]: pgmap v2418: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:44:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:44:36 smithi138 ceph-mon[36954]: pgmap v2418: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:44:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:44:38 smithi100 ceph-mon[32599]: pgmap v2419: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:44:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:44:38 smithi167 ceph-mon[37140]: pgmap v2419: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:44:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:44:38 smithi138 ceph-mon[36954]: pgmap v2419: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:44:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:44:40 smithi100 ceph-mon[32599]: pgmap v2420: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:44:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:44:40 smithi167 ceph-mon[37140]: pgmap v2420: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:44:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:44:40 smithi138 ceph-mon[36954]: pgmap v2420: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:44:42.992 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:44:42 smithi138 ceph-mon[36954]: pgmap v2421: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:44:42.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:44:42 smithi100 ceph-mon[32599]: pgmap v2421: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:44:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:44:42 smithi167 ceph-mon[37140]: pgmap v2421: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:44:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:44:44 smithi100 ceph-mon[32599]: pgmap v2422: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:44:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:44:44 smithi167 ceph-mon[37140]: pgmap v2422: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:44:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:44:44 smithi138 ceph-mon[36954]: pgmap v2422: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:44:46.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:44:46.430+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:44:46.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:44:46.431+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:44:46.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:44:46.431+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:44:46.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:44:46.431+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:44:46.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:44:46.431+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:44:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:44:46 smithi167 ceph-mon[37140]: pgmap v2423: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:44:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:44:46 smithi138 ceph-mon[36954]: pgmap v2423: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:44:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:44:46 smithi100 ceph-mon[32599]: pgmap v2423: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:44:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:44:48 smithi167 ceph-mon[37140]: pgmap v2424: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:44:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:44:48 smithi138 ceph-mon[36954]: pgmap v2424: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:44:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:44:48 smithi100 ceph-mon[32599]: pgmap v2424: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:44:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:44:50 smithi167 ceph-mon[37140]: pgmap v2425: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:44:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:44:50 smithi138 ceph-mon[36954]: pgmap v2425: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:44:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:44:50 smithi100 ceph-mon[32599]: pgmap v2425: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:44:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:44:52 smithi167 ceph-mon[37140]: pgmap v2426: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:44:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:44:52 smithi138 ceph-mon[36954]: pgmap v2426: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:44:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:44:52 smithi100 ceph-mon[32599]: pgmap v2426: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:44:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:44:54 smithi167 ceph-mon[37140]: pgmap v2427: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:44:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:44:54 smithi138 ceph-mon[36954]: pgmap v2427: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:44:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:44:54 smithi100 ceph-mon[32599]: pgmap v2427: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:44:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:44:56 smithi167 ceph-mon[37140]: pgmap v2428: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:44:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:44:56 smithi138 ceph-mon[36954]: pgmap v2428: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:44:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:44:56 smithi100 ceph-mon[32599]: pgmap v2428: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:44:59.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:44:58 smithi167 ceph-mon[37140]: pgmap v2429: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:44:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:44:58 smithi138 ceph-mon[36954]: pgmap v2429: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:44:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:44:58 smithi100 ceph-mon[32599]: pgmap v2429: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:45:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:45:00 smithi167 ceph-mon[37140]: pgmap v2430: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:45:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:45:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:45:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:45:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:45:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:45:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:45:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:45:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:45:01.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:45:00 smithi100 ceph-mon[32599]: pgmap v2430: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:45:01.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:45:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:45:01.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:45:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:45:01.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:45:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:45:01.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:45:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:45:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:45:00 smithi138 ceph-mon[36954]: pgmap v2430: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:45:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:45:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:45:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:45:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:45:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:45:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:45:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:45:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:45:01.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:45:01.446+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:45:01.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:45:01.446+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:45:01.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:45:01.446+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:45:01.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:45:01.446+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:45:01.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:45:01.446+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:45:03.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:45:02 smithi167 ceph-mon[37140]: pgmap v2431: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:45:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:45:02 smithi138 ceph-mon[36954]: pgmap v2431: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:45:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:45:02 smithi100 ceph-mon[32599]: pgmap v2431: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:45:03.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:45:03.497+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T14:45:03.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:45:03.497+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c006610 con 0x7fe5a81087a0 2024-07-26T14:45:03.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:45:03.497+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 30 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T14:45:03.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:45:03.497+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 30 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c100 con 0x7fe5a81087a0 2024-07-26T14:45:05.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:45:04 smithi138 ceph-mon[36954]: pgmap v2432: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T14:45:05.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:45:04 smithi100 ceph-mon[32599]: pgmap v2432: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T14:45:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:45:04 smithi167 ceph-mon[37140]: pgmap v2432: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T14:45:07.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:45:06 smithi138 ceph-mon[36954]: pgmap v2433: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:45:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:45:06 smithi100 ceph-mon[32599]: pgmap v2433: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:45:07.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:45:06 smithi167 ceph-mon[37140]: pgmap v2433: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:45:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:45:08 smithi138 ceph-mon[36954]: pgmap v2434: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:45:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:45:08 smithi138 ceph-mon[36954]: osdmap e202: 12 total, 12 up, 12 in 2024-07-26T14:45:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:45:08 smithi100 ceph-mon[32599]: pgmap v2434: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:45:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:45:08 smithi100 ceph-mon[32599]: osdmap e202: 12 total, 12 up, 12 in 2024-07-26T14:45:09.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:45:08 smithi167 ceph-mon[37140]: pgmap v2434: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:45:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:45:08 smithi167 ceph-mon[37140]: osdmap e202: 12 total, 12 up, 12 in 2024-07-26T14:45:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:45:10 smithi138 ceph-mon[36954]: pgmap v2436: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:45:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:45:10 smithi100 ceph-mon[32599]: pgmap v2436: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:45:11.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:45:10 smithi167 ceph-mon[37140]: pgmap v2436: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:45:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:45:12 smithi138 ceph-mon[36954]: pgmap v2437: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:45:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:45:12 smithi100 ceph-mon[32599]: pgmap v2437: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:45:13.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:45:12 smithi167 ceph-mon[37140]: pgmap v2437: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:45:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:45:14 smithi138 ceph-mon[36954]: pgmap v2438: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:45:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:45:14 smithi138 ceph-mon[36954]: osdmap e203: 12 total, 12 up, 12 in 2024-07-26T14:45:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:45:14 smithi100 ceph-mon[32599]: pgmap v2438: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:45:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:45:14 smithi100 ceph-mon[32599]: osdmap e203: 12 total, 12 up, 12 in 2024-07-26T14:45:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:45:14 smithi167 ceph-mon[37140]: pgmap v2438: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:45:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:45:14 smithi167 ceph-mon[37140]: osdmap e203: 12 total, 12 up, 12 in 2024-07-26T14:45:16.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:45:16.462+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:45:16.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:45:16.462+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:45:16.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:45:16.462+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:45:16.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:45:16.462+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:45:16.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:45:16.463+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:45:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:45:16 smithi138 ceph-mon[36954]: pgmap v2440: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:45:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:45:16 smithi100 ceph-mon[32599]: pgmap v2440: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:45:17.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:45:16 smithi167 ceph-mon[37140]: pgmap v2440: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T14:45:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:45:18 smithi138 ceph-mon[36954]: pgmap v2441: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T14:45:19.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:45:18 smithi100 ceph-mon[32599]: pgmap v2441: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T14:45:19.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:45:18 smithi167 ceph-mon[37140]: pgmap v2441: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T14:45:21.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:45:20 smithi138 ceph-mon[36954]: pgmap v2442: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:45:21.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:45:20 smithi100 ceph-mon[32599]: pgmap v2442: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:45:21.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:45:20 smithi167 ceph-mon[37140]: pgmap v2442: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:45:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:45:22 smithi138 ceph-mon[36954]: pgmap v2443: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:45:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:45:22 smithi100 ceph-mon[32599]: pgmap v2443: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:45:23.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:45:22 smithi167 ceph-mon[37140]: pgmap v2443: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:45:25.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:45:24 smithi167 ceph-mon[37140]: pgmap v2444: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:45:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:45:24 smithi138 ceph-mon[36954]: pgmap v2444: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:45:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:45:24 smithi100 ceph-mon[32599]: pgmap v2444: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:45:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:45:26 smithi138 ceph-mon[36954]: pgmap v2445: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:45:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:45:26 smithi100 ceph-mon[32599]: pgmap v2445: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:45:27.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:45:26 smithi167 ceph-mon[37140]: pgmap v2445: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:45:29.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:45:28 smithi138 ceph-mon[36954]: pgmap v2446: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:45:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:45:28 smithi100 ceph-mon[32599]: pgmap v2446: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:45:29.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:45:28 smithi167 ceph-mon[37140]: pgmap v2446: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:45:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:45:30 smithi138 ceph-mon[36954]: pgmap v2447: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:45:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:45:30 smithi100 ceph-mon[32599]: pgmap v2447: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:45:31.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:45:30 smithi167 ceph-mon[37140]: pgmap v2447: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:45:31.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:45:31.471+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:45:31.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:45:31.471+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:45:31.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:45:31.471+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:45:31.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:45:31.471+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:45:31.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:45:31.471+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:45:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:45:32 smithi138 ceph-mon[36954]: pgmap v2448: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:45:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:45:32 smithi100 ceph-mon[32599]: pgmap v2448: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:45:33.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:45:32 smithi167 ceph-mon[37140]: pgmap v2448: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:45:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:45:34 smithi138 ceph-mon[36954]: pgmap v2449: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:45:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:45:34 smithi100 ceph-mon[32599]: pgmap v2449: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:45:35.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:45:34 smithi167 ceph-mon[37140]: pgmap v2449: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:45:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:45:36 smithi138 ceph-mon[36954]: pgmap v2450: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:45:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:45:36 smithi100 ceph-mon[32599]: pgmap v2450: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:45:37.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:45:36 smithi167 ceph-mon[37140]: pgmap v2450: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:45:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:45:38 smithi138 ceph-mon[36954]: pgmap v2451: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:45:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:45:38 smithi100 ceph-mon[32599]: pgmap v2451: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:45:39.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:45:38 smithi167 ceph-mon[37140]: pgmap v2451: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:45:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:45:40 smithi138 ceph-mon[36954]: pgmap v2452: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:45:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:45:40 smithi100 ceph-mon[32599]: pgmap v2452: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:45:41.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:45:40 smithi167 ceph-mon[37140]: pgmap v2452: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:45:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:45:42 smithi138 ceph-mon[36954]: pgmap v2453: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:45:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:45:42 smithi100 ceph-mon[32599]: pgmap v2453: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:45:43.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:45:42 smithi167 ceph-mon[37140]: pgmap v2453: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:45:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:45:44 smithi138 ceph-mon[36954]: pgmap v2454: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:45:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:45:44 smithi100 ceph-mon[32599]: pgmap v2454: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:45:45.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:45:44 smithi167 ceph-mon[37140]: pgmap v2454: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:45:46.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:45:46.487+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:45:46.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:45:46.488+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:45:46.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:45:46.488+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:45:46.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:45:46.488+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:45:46.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:45:46.488+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:45:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:45:46 smithi138 ceph-mon[36954]: pgmap v2455: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:45:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:45:46 smithi100 ceph-mon[32599]: pgmap v2455: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:45:47.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:45:46 smithi167 ceph-mon[37140]: pgmap v2455: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:45:48.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:45:48.503+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T14:45:48.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:45:48.503+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T14:45:48.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:45:48.504+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 31 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe5a008c0e0 con 0x7fe5a81087a0 2024-07-26T14:45:48.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:45:48.504+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 31 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T14:45:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:45:48 smithi138 ceph-mon[36954]: pgmap v2456: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:45:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:45:48 smithi100 ceph-mon[32599]: pgmap v2456: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:45:49.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:45:48 smithi167 ceph-mon[37140]: pgmap v2456: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:45:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:45:50 smithi138 ceph-mon[36954]: pgmap v2457: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:45:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:45:50 smithi100 ceph-mon[32599]: pgmap v2457: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:45:51.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:45:50 smithi167 ceph-mon[37140]: pgmap v2457: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:45:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:45:52 smithi138 ceph-mon[36954]: pgmap v2458: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:45:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:45:52 smithi100 ceph-mon[32599]: pgmap v2458: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:45:53.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:45:52 smithi167 ceph-mon[37140]: pgmap v2458: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:45:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:45:54 smithi138 ceph-mon[36954]: pgmap v2459: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:45:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:45:54 smithi100 ceph-mon[32599]: pgmap v2459: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:45:55.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:45:54 smithi167 ceph-mon[37140]: pgmap v2459: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:45:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:45:56 smithi138 ceph-mon[36954]: pgmap v2460: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:45:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:45:56 smithi100 ceph-mon[32599]: pgmap v2460: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:45:57.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:45:56 smithi167 ceph-mon[37140]: pgmap v2460: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:45:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:45:58 smithi138 ceph-mon[36954]: pgmap v2461: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:45:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:45:58 smithi100 ceph-mon[32599]: pgmap v2461: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:45:59.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:45:58 smithi167 ceph-mon[37140]: pgmap v2461: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:46:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:46:00 smithi138 ceph-mon[36954]: pgmap v2462: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:46:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:46:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:46:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:46:00 smithi100 ceph-mon[32599]: pgmap v2462: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:46:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:46:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:46:01.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:46:00 smithi167 ceph-mon[37140]: pgmap v2462: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:46:01.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:46:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:46:01.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:46:01.503+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:46:01.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:46:01.504+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:46:01.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:46:01.504+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:46:01.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:46:01.504+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:46:01.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:46:01.504+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:46:02.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:46:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:46:02.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:46:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:46:02.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:46:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:46:02.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:46:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:46:02.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:46:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:46:02.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:46:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:46:02.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:46:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:46:02.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:46:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:46:02.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:46:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:46:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:46:02 smithi138 ceph-mon[36954]: pgmap v2463: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:46:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:46:02 smithi100 ceph-mon[32599]: pgmap v2463: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:46:03.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:46:02 smithi167 ceph-mon[37140]: pgmap v2463: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:46:05.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:46:04 smithi138 ceph-mon[36954]: pgmap v2464: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T14:46:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:46:04 smithi100 ceph-mon[32599]: pgmap v2464: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T14:46:05.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:46:04 smithi167 ceph-mon[37140]: pgmap v2464: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T14:46:07.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:46:06 smithi138 ceph-mon[36954]: pgmap v2465: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:46:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:46:06 smithi100 ceph-mon[32599]: pgmap v2465: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:46:07.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:46:06 smithi167 ceph-mon[37140]: pgmap v2465: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:46:08.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:46:07 smithi138 ceph-mon[36954]: osdmap e204: 12 total, 12 up, 12 in 2024-07-26T14:46:08.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:46:07 smithi100 ceph-mon[32599]: osdmap e204: 12 total, 12 up, 12 in 2024-07-26T14:46:08.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:46:07 smithi167 ceph-mon[37140]: osdmap e204: 12 total, 12 up, 12 in 2024-07-26T14:46:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:46:08 smithi138 ceph-mon[36954]: pgmap v2467: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:46:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:46:08 smithi100 ceph-mon[32599]: pgmap v2467: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:46:09.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:46:08 smithi167 ceph-mon[37140]: pgmap v2467: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:46:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:46:10 smithi138 ceph-mon[36954]: pgmap v2468: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:46:11.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:46:10 smithi100 ceph-mon[32599]: pgmap v2468: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:46:11.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:46:10 smithi167 ceph-mon[37140]: pgmap v2468: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:46:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:46:12 smithi138 ceph-mon[36954]: pgmap v2469: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:46:13.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:46:12 smithi100 ceph-mon[32599]: pgmap v2469: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:46:13.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:46:12 smithi167 ceph-mon[37140]: pgmap v2469: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:46:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:46:14 smithi138 ceph-mon[36954]: pgmap v2470: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:46:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:46:14 smithi138 ceph-mon[36954]: osdmap e205: 12 total, 12 up, 12 in 2024-07-26T14:46:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:46:14 smithi100 ceph-mon[32599]: pgmap v2470: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:46:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:46:14 smithi100 ceph-mon[32599]: osdmap e205: 12 total, 12 up, 12 in 2024-07-26T14:46:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:46:14 smithi167 ceph-mon[37140]: pgmap v2470: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:46:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:46:14 smithi167 ceph-mon[37140]: osdmap e205: 12 total, 12 up, 12 in 2024-07-26T14:46:16.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:46:16.508+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:46:16.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:46:16.508+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:46:16.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:46:16.508+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:46:16.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:46:16.508+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:46:16.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:46:16.508+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:46:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:46:16 smithi138 ceph-mon[36954]: pgmap v2472: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 232 B/s wr, 7 op/s 2024-07-26T14:46:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:46:16 smithi100 ceph-mon[32599]: pgmap v2472: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 232 B/s wr, 7 op/s 2024-07-26T14:46:17.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:46:16 smithi167 ceph-mon[37140]: pgmap v2472: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 232 B/s wr, 7 op/s 2024-07-26T14:46:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:46:18 smithi138 ceph-mon[36954]: pgmap v2473: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:46:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:46:18 smithi100 ceph-mon[32599]: pgmap v2473: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:46:19.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:46:18 smithi167 ceph-mon[37140]: pgmap v2473: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:46:21.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:46:20 smithi138 ceph-mon[36954]: pgmap v2474: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:46:21.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:46:20 smithi100 ceph-mon[32599]: pgmap v2474: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:46:21.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:46:20 smithi167 ceph-mon[37140]: pgmap v2474: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:46:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:46:22 smithi138 ceph-mon[36954]: pgmap v2475: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:46:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:46:22 smithi100 ceph-mon[32599]: pgmap v2475: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:46:23.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:46:22 smithi167 ceph-mon[37140]: pgmap v2475: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:46:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:46:24 smithi138 ceph-mon[36954]: pgmap v2476: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:46:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:46:24 smithi100 ceph-mon[32599]: pgmap v2476: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:46:25.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:46:24 smithi167 ceph-mon[37140]: pgmap v2476: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:46:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:46:26 smithi138 ceph-mon[36954]: pgmap v2477: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:46:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:46:26 smithi100 ceph-mon[32599]: pgmap v2477: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:46:27.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:46:26 smithi167 ceph-mon[37140]: pgmap v2477: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:46:29.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:46:28 smithi138 ceph-mon[36954]: pgmap v2478: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:46:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:46:28 smithi100 ceph-mon[32599]: pgmap v2478: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:46:29.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:46:28 smithi167 ceph-mon[37140]: pgmap v2478: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:46:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:46:30 smithi138 ceph-mon[36954]: pgmap v2479: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:46:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:46:30 smithi100 ceph-mon[32599]: pgmap v2479: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:46:31.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:46:30 smithi167 ceph-mon[37140]: pgmap v2479: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:46:31.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:46:31.510+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:46:31.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:46:31.510+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:46:31.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:46:31.510+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:46:31.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:46:31.510+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:46:31.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:46:31.510+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:46:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:46:32 smithi138 ceph-mon[36954]: pgmap v2480: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:46:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:46:32 smithi100 ceph-mon[32599]: pgmap v2480: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:46:33.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:46:32 smithi167 ceph-mon[37140]: pgmap v2480: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:46:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:46:34 smithi100 ceph-mon[32599]: pgmap v2481: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:46:35.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:46:34 smithi167 ceph-mon[37140]: pgmap v2481: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:46:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:46:34 smithi138 ceph-mon[36954]: pgmap v2481: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:46:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:46:36 smithi100 ceph-mon[32599]: pgmap v2482: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:46:37.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:46:36 smithi167 ceph-mon[37140]: pgmap v2482: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:46:37.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:46:36 smithi138 ceph-mon[36954]: pgmap v2482: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:46:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:46:38 smithi100 ceph-mon[32599]: pgmap v2483: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:46:39.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:46:38 smithi167 ceph-mon[37140]: pgmap v2483: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:46:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:46:38 smithi138 ceph-mon[36954]: pgmap v2483: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:46:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:46:40 smithi100 ceph-mon[32599]: pgmap v2484: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:46:41.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:46:40 smithi167 ceph-mon[37140]: pgmap v2484: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:46:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:46:40 smithi138 ceph-mon[36954]: pgmap v2484: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:46:43.242 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:46:42 smithi138 ceph-mon[36954]: pgmap v2485: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:46:43.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:46:42 smithi100 ceph-mon[32599]: pgmap v2485: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:46:43.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:46:42 smithi167 ceph-mon[37140]: pgmap v2485: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:46:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:46:44 smithi100 ceph-mon[32599]: pgmap v2486: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T14:46:45.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:46:44 smithi167 ceph-mon[37140]: pgmap v2486: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T14:46:45.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:46:44 smithi138 ceph-mon[36954]: pgmap v2486: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T14:46:46.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:46:46.525+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:46:46.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:46:46.525+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:46:46.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:46:46.525+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:46:46.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:46:46.525+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:46:46.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:46:46.526+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:46:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:46:46 smithi100 ceph-mon[32599]: pgmap v2487: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:46:47.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:46:46 smithi167 ceph-mon[37140]: pgmap v2487: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:46:47.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:46:46 smithi138 ceph-mon[36954]: pgmap v2487: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:46:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:46:48 smithi100 ceph-mon[32599]: pgmap v2488: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:46:49.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:46:48 smithi167 ceph-mon[37140]: pgmap v2488: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:46:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:46:48 smithi138 ceph-mon[36954]: pgmap v2488: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:46:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:46:50 smithi100 ceph-mon[32599]: pgmap v2489: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:46:51.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:46:50 smithi167 ceph-mon[37140]: pgmap v2489: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:46:51.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:46:50 smithi138 ceph-mon[36954]: pgmap v2489: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:46:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:46:52 smithi100 ceph-mon[32599]: pgmap v2490: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:46:53.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:46:52 smithi167 ceph-mon[37140]: pgmap v2490: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:46:53.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:46:52 smithi138 ceph-mon[36954]: pgmap v2490: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:46:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:46:54 smithi100 ceph-mon[32599]: pgmap v2491: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:46:55.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:46:54 smithi167 ceph-mon[37140]: pgmap v2491: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:46:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:46:54 smithi138 ceph-mon[36954]: pgmap v2491: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:46:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:46:56 smithi100 ceph-mon[32599]: pgmap v2492: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:46:57.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:46:56 smithi167 ceph-mon[37140]: pgmap v2492: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:46:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:46:56 smithi138 ceph-mon[36954]: pgmap v2492: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:46:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:46:58 smithi100 ceph-mon[32599]: pgmap v2493: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:46:59.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:46:58 smithi167 ceph-mon[37140]: pgmap v2493: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:46:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:46:58 smithi138 ceph-mon[36954]: pgmap v2493: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:47:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:00 smithi100 ceph-mon[32599]: pgmap v2494: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:47:01.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:00 smithi167 ceph-mon[37140]: pgmap v2494: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:47:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:00 smithi138 ceph-mon[36954]: pgmap v2494: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:47:01.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:47:01.534+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:47:01.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:47:01.534+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:47:01.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:47:01.534+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:47:01.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:47:01.534+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:47:01.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:47:01.534+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:47:02.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:47:02.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:47:02.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:47:02.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:47:02.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:47:02.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:47:02.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:47:02.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:47:02.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:47:02.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:47:02.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:47:02.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:47:02.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:47:02.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:47:02.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:47:02.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:47:02.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:47:02.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:47:02.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:47:02.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:47:02.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:47:02.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:47:02.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:47:02.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:47:02.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:47:02.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:47:02.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:47:02.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:47:02.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:47:02.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:47:02.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:47:02.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:47:02.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:47:02.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:47:02.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:47:02.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:47:02.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:47:02.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:47:02.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:47:02.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:47:02.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:47:02.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:47:02.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:47:02.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:47:02.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:47:02.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:47:02.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:47:02.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:47:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:02 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T14:47:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:02 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T14:47:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:02 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T14:47:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:02 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T14:47:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:02 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T14:47:03.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:02 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T14:47:03.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:02 smithi100 ceph-mon[32599]: pgmap v2495: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:47:03.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:02 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T14:47:03.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:02 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T14:47:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:02 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T14:47:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:02 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T14:47:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:02 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T14:47:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:02 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T14:47:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:02 smithi167 ceph-mon[37140]: pgmap v2495: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:47:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:02 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T14:47:03.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:02 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T14:47:03.456 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:02 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T14:47:03.458 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:02 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T14:47:03.459 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:02 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T14:47:03.459 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:02 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T14:47:03.459 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:02 smithi138 ceph-mon[36954]: pgmap v2495: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:47:05.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:04 smithi167 ceph-mon[37140]: pgmap v2496: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T14:47:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:04 smithi138 ceph-mon[36954]: pgmap v2496: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T14:47:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:04 smithi100 ceph-mon[32599]: pgmap v2496: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T14:47:07.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:06 smithi167 ceph-mon[37140]: pgmap v2497: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:47:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:06 smithi138 ceph-mon[36954]: pgmap v2497: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:47:07.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:06 smithi100 ceph-mon[32599]: pgmap v2497: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:47:08.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:08 smithi167 ceph-mon[37140]: osdmap e206: 12 total, 12 up, 12 in 2024-07-26T14:47:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:08 smithi138 ceph-mon[36954]: osdmap e206: 12 total, 12 up, 12 in 2024-07-26T14:47:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:08 smithi100 ceph-mon[32599]: osdmap e206: 12 total, 12 up, 12 in 2024-07-26T14:47:09.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:09 smithi167 ceph-mon[37140]: pgmap v2499: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:47:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:09 smithi138 ceph-mon[36954]: pgmap v2499: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:47:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:09 smithi100 ceph-mon[32599]: pgmap v2499: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:47:11.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:11 smithi167 ceph-mon[37140]: pgmap v2500: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:47:11.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:11 smithi138 ceph-mon[36954]: pgmap v2500: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:47:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:11 smithi100 ceph-mon[32599]: pgmap v2500: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:47:13.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:13 smithi167 ceph-mon[37140]: pgmap v2501: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:47:13.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:13 smithi138 ceph-mon[36954]: pgmap v2501: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:47:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:13 smithi100 ceph-mon[32599]: pgmap v2501: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:47:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:15 smithi138 ceph-mon[36954]: pgmap v2502: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:47:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:15 smithi138 ceph-mon[36954]: osdmap e207: 12 total, 12 up, 12 in 2024-07-26T14:47:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:15 smithi100 ceph-mon[32599]: pgmap v2502: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:47:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:15 smithi100 ceph-mon[32599]: osdmap e207: 12 total, 12 up, 12 in 2024-07-26T14:47:15.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:15 smithi167 ceph-mon[37140]: pgmap v2502: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:47:15.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:15 smithi167 ceph-mon[37140]: osdmap e207: 12 total, 12 up, 12 in 2024-07-26T14:47:16.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:47:16.550+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:47:16.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:47:16.551+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:47:16.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:47:16.551+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:47:16.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:47:16.551+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:47:16.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:47:16.551+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:47:17.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:17 smithi138 ceph-mon[36954]: pgmap v2504: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 234 B/s wr, 6 op/s 2024-07-26T14:47:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:17 smithi100 ceph-mon[32599]: pgmap v2504: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 234 B/s wr, 6 op/s 2024-07-26T14:47:17.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:17 smithi167 ceph-mon[37140]: pgmap v2504: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 234 B/s wr, 6 op/s 2024-07-26T14:47:19.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:19 smithi138 ceph-mon[36954]: pgmap v2505: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:47:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:19 smithi100 ceph-mon[32599]: pgmap v2505: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:47:19.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:19 smithi167 ceph-mon[37140]: pgmap v2505: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:47:21.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:21 smithi138 ceph-mon[36954]: pgmap v2506: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:47:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:21 smithi100 ceph-mon[32599]: pgmap v2506: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:47:21.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:21 smithi167 ceph-mon[37140]: pgmap v2506: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:47:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:23 smithi138 ceph-mon[36954]: pgmap v2507: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:47:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:23 smithi100 ceph-mon[32599]: pgmap v2507: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:47:23.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:23 smithi167 ceph-mon[37140]: pgmap v2507: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:47:25.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:25 smithi167 ceph-mon[37140]: pgmap v2508: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:47:25.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:25 smithi138 ceph-mon[36954]: pgmap v2508: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:47:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:25 smithi100 ceph-mon[32599]: pgmap v2508: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:47:27.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:27 smithi138 ceph-mon[36954]: pgmap v2509: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:47:27.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:27 smithi100 ceph-mon[32599]: pgmap v2509: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:47:27.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:27 smithi167 ceph-mon[37140]: pgmap v2509: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:47:29.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:29 smithi138 ceph-mon[36954]: pgmap v2510: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:47:29.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:29 smithi100 ceph-mon[32599]: pgmap v2510: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:47:29.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:29 smithi167 ceph-mon[37140]: pgmap v2510: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:47:31.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:31 smithi138 ceph-mon[36954]: pgmap v2511: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:47:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:31 smithi100 ceph-mon[32599]: pgmap v2511: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:47:31.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:31 smithi167 ceph-mon[37140]: pgmap v2511: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:47:31.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:47:31.566+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:47:31.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:47:31.567+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:47:31.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:47:31.567+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:47:31.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:47:31.567+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:47:31.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:47:31.567+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:47:33.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:33 smithi138 ceph-mon[36954]: pgmap v2512: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:47:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:33 smithi100 ceph-mon[32599]: pgmap v2512: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:47:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:33 smithi167 ceph-mon[37140]: pgmap v2512: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:47:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:35 smithi138 ceph-mon[36954]: pgmap v2513: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:47:35.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:35 smithi100 ceph-mon[32599]: pgmap v2513: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:47:35.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:35 smithi167 ceph-mon[37140]: pgmap v2513: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:47:37.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:37 smithi138 ceph-mon[36954]: pgmap v2514: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:47:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:37 smithi100 ceph-mon[32599]: pgmap v2514: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:47:37.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:37 smithi167 ceph-mon[37140]: pgmap v2514: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:47:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:39 smithi138 ceph-mon[36954]: pgmap v2515: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:47:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:39 smithi100 ceph-mon[32599]: pgmap v2515: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:47:39.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:39 smithi167 ceph-mon[37140]: pgmap v2515: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:47:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:41 smithi138 ceph-mon[36954]: pgmap v2516: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:47:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:41 smithi100 ceph-mon[32599]: pgmap v2516: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:47:41.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:41 smithi167 ceph-mon[37140]: pgmap v2516: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:47:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:43 smithi138 ceph-mon[36954]: pgmap v2517: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:47:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:43 smithi100 ceph-mon[32599]: pgmap v2517: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:47:43.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:43 smithi167 ceph-mon[37140]: pgmap v2517: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:47:45.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:45 smithi138 ceph-mon[36954]: pgmap v2518: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:47:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:45 smithi100 ceph-mon[32599]: pgmap v2518: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:47:45.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:45 smithi167 ceph-mon[37140]: pgmap v2518: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:47:46.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:47:46.577+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:47:46.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:47:46.577+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:47:46.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:47:46.577+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:47:46.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:47:46.577+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:47:46.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:47:46.577+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:47:47.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:47 smithi138 ceph-mon[36954]: pgmap v2519: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:47:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:47 smithi100 ceph-mon[32599]: pgmap v2519: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:47:47.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:47 smithi167 ceph-mon[37140]: pgmap v2519: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:47:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:49 smithi138 ceph-mon[36954]: pgmap v2520: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:47:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:49 smithi100 ceph-mon[32599]: pgmap v2520: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:47:49.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:49 smithi167 ceph-mon[37140]: pgmap v2520: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:47:51.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:51 smithi138 ceph-mon[36954]: pgmap v2521: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:47:51.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:51 smithi100 ceph-mon[32599]: pgmap v2521: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:47:51.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:51 smithi167 ceph-mon[37140]: pgmap v2521: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:47:53.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:53 smithi138 ceph-mon[36954]: pgmap v2522: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:47:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:53 smithi100 ceph-mon[32599]: pgmap v2522: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:47:53.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:53 smithi167 ceph-mon[37140]: pgmap v2522: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:47:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:55 smithi138 ceph-mon[36954]: pgmap v2523: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T14:47:55.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:55 smithi100 ceph-mon[32599]: pgmap v2523: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T14:47:55.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:55 smithi167 ceph-mon[37140]: pgmap v2523: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T14:47:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:57 smithi138 ceph-mon[36954]: pgmap v2524: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:47:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:57 smithi100 ceph-mon[32599]: pgmap v2524: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:47:57.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:57 smithi167 ceph-mon[37140]: pgmap v2524: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:47:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:47:59 smithi138 ceph-mon[36954]: pgmap v2525: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:47:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:47:59 smithi100 ceph-mon[32599]: pgmap v2525: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:47:59.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:47:59 smithi167 ceph-mon[37140]: pgmap v2525: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:48:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:48:01 smithi138 ceph-mon[36954]: pgmap v2526: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:48:01.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:48:01 smithi100 ceph-mon[32599]: pgmap v2526: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:48:01.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:48:01 smithi167 ceph-mon[37140]: pgmap v2526: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:48:01.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:48:01.589+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:48:01.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:48:01.589+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:48:01.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:48:01.589+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:48:01.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:48:01.589+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:48:01.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:48:01.589+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:48:02.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:48:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:48:02.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:48:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:48:02.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:48:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:48:03.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:48:03.029+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 msgr2=0x7fe5940c0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).tick idle (900001268) for more than 900000000 us, fault. 2024-07-26T14:48:03.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:48:03.029+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe59000b6b0 tx=0x7fe59002f040 comp rx=0 tx=0).stop 2024-07-26T14:48:03.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:48:03.029+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 msgr2=0x7fe5940c0480 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-07-26T14:48:03.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:48:03.029+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T14:48:03.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:48:03.029+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b3d40 0x7fe5940b4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T14:48:03.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:48:03.029+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b3d40 0x7fe5940b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:48:03.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:48:03.030+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b3d40 0x7fe5940b4730 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe5900051a0 tx=0x7fe590074040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T14:48:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:48:03 smithi138 ceph-mon[36954]: pgmap v2527: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:48:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:48:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:48:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:48:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:48:03.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:48:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:48:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:48:03 smithi100 ceph-mon[32599]: pgmap v2527: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:48:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:48:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:48:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:48:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:48:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:48:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:48:03.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:48:03 smithi167 ceph-mon[37140]: pgmap v2527: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:48:03.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:48:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:48:03.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:48:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:48:03.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:48:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:48:03.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:48:03.522+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T14:48:03.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:48:03.522+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T14:48:03.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:48:03.522+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 32 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T14:48:03.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:48:03.522+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 32 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c100 con 0x7fe5a81087a0 2024-07-26T14:48:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:48:05 smithi138 ceph-mon[36954]: pgmap v2528: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-07-26T14:48:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:48:05 smithi100 ceph-mon[32599]: pgmap v2528: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-07-26T14:48:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:48:05 smithi167 ceph-mon[37140]: pgmap v2528: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-07-26T14:48:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:48:07 smithi138 ceph-mon[36954]: pgmap v2529: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:48:07.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:48:07 smithi100 ceph-mon[32599]: pgmap v2529: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:48:07.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:48:07 smithi167 ceph-mon[37140]: pgmap v2529: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:48:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:48:08 smithi138 ceph-mon[36954]: osdmap e208: 12 total, 12 up, 12 in 2024-07-26T14:48:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:48:08 smithi100 ceph-mon[32599]: osdmap e208: 12 total, 12 up, 12 in 2024-07-26T14:48:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:48:08 smithi167 ceph-mon[37140]: osdmap e208: 12 total, 12 up, 12 in 2024-07-26T14:48:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:48:09 smithi138 ceph-mon[36954]: pgmap v2531: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:48:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:48:09 smithi100 ceph-mon[32599]: pgmap v2531: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:48:09.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:48:09 smithi167 ceph-mon[37140]: pgmap v2531: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:48:11.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:48:11 smithi138 ceph-mon[36954]: pgmap v2532: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:48:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:48:11 smithi100 ceph-mon[32599]: pgmap v2532: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:48:11.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:48:11 smithi167 ceph-mon[37140]: pgmap v2532: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:48:13.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:48:13 smithi138 ceph-mon[36954]: pgmap v2533: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:48:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:48:13 smithi100 ceph-mon[32599]: pgmap v2533: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:48:13.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:48:13 smithi167 ceph-mon[37140]: pgmap v2533: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:48:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:48:15 smithi138 ceph-mon[36954]: pgmap v2534: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:48:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:48:15 smithi138 ceph-mon[36954]: osdmap e209: 12 total, 12 up, 12 in 2024-07-26T14:48:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:48:15 smithi100 ceph-mon[32599]: pgmap v2534: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:48:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:48:15 smithi100 ceph-mon[32599]: osdmap e209: 12 total, 12 up, 12 in 2024-07-26T14:48:15.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:48:15 smithi167 ceph-mon[37140]: pgmap v2534: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:48:15.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:48:15 smithi167 ceph-mon[37140]: osdmap e209: 12 total, 12 up, 12 in 2024-07-26T14:48:16.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:48:16.604+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:48:16.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:48:16.604+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:48:16.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:48:16.604+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:48:16.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:48:16.604+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:48:16.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:48:16.604+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:48:17.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:48:17 smithi138 ceph-mon[36954]: pgmap v2536: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 237 B/s wr, 6 op/s 2024-07-26T14:48:17.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:48:17 smithi100 ceph-mon[32599]: pgmap v2536: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 237 B/s wr, 6 op/s 2024-07-26T14:48:17.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:48:17 smithi167 ceph-mon[37140]: pgmap v2536: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 237 B/s wr, 6 op/s 2024-07-26T14:48:19.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:48:19 smithi138 ceph-mon[36954]: pgmap v2537: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:48:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:48:19 smithi100 ceph-mon[32599]: pgmap v2537: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:48:19.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:48:19 smithi167 ceph-mon[37140]: pgmap v2537: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:48:21.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:48:21 smithi138 ceph-mon[36954]: pgmap v2538: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:48:21.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:48:21 smithi100 ceph-mon[32599]: pgmap v2538: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:48:21.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:48:21 smithi167 ceph-mon[37140]: pgmap v2538: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:48:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:48:23 smithi138 ceph-mon[36954]: pgmap v2539: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:48:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:48:23 smithi100 ceph-mon[32599]: pgmap v2539: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:48:23.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:48:23 smithi167 ceph-mon[37140]: pgmap v2539: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:48:25.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:48:25 smithi138 ceph-mon[36954]: pgmap v2540: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:48:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:48:25 smithi100 ceph-mon[32599]: pgmap v2540: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:48:25.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:48:25 smithi167 ceph-mon[37140]: pgmap v2540: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:48:27.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:48:27 smithi138 ceph-mon[36954]: pgmap v2541: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:48:27.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:48:27 smithi100 ceph-mon[32599]: pgmap v2541: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:48:27.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:48:27 smithi167 ceph-mon[37140]: pgmap v2541: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:48:29.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:48:29 smithi138 ceph-mon[36954]: pgmap v2542: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:48:29.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:48:29 smithi100 ceph-mon[32599]: pgmap v2542: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:48:29.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:48:29 smithi167 ceph-mon[37140]: pgmap v2542: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:48:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:48:30 smithi138 ceph-mon[36954]: pgmap v2543: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:48:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:48:30 smithi100 ceph-mon[32599]: pgmap v2543: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:48:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:48:30 smithi167 ceph-mon[37140]: pgmap v2543: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:48:31.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:48:31.621+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:48:31.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:48:31.622+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:48:31.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:48:31.622+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:48:31.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:48:31.622+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:48:31.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:48:31.622+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:48:33.107 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:48:32 smithi100 ceph-mon[32599]: pgmap v2544: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:48:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:48:32 smithi138 ceph-mon[36954]: pgmap v2544: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:48:33.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:48:32 smithi167 ceph-mon[37140]: pgmap v2544: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:48:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:48:34 smithi138 ceph-mon[36954]: pgmap v2545: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:48:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:48:34 smithi100 ceph-mon[32599]: pgmap v2545: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:48:35.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:48:34 smithi167 ceph-mon[37140]: pgmap v2545: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:48:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:48:36 smithi138 ceph-mon[36954]: pgmap v2546: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:48:37.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:48:36 smithi100 ceph-mon[32599]: pgmap v2546: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:48:37.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:48:36 smithi167 ceph-mon[37140]: pgmap v2546: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:48:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:48:38 smithi138 ceph-mon[36954]: pgmap v2547: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:48:39.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:48:38 smithi100 ceph-mon[32599]: pgmap v2547: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:48:39.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:48:38 smithi167 ceph-mon[37140]: pgmap v2547: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:48:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:48:40 smithi138 ceph-mon[36954]: pgmap v2548: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:48:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:48:40 smithi100 ceph-mon[32599]: pgmap v2548: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:48:41.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:48:40 smithi167 ceph-mon[37140]: pgmap v2548: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:48:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:48:42 smithi100 ceph-mon[32599]: pgmap v2549: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:48:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:48:42 smithi138 ceph-mon[36954]: pgmap v2549: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:48:43.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:48:42 smithi167 ceph-mon[37140]: pgmap v2549: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:48:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:48:44 smithi138 ceph-mon[36954]: pgmap v2550: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:48:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:48:44 smithi100 ceph-mon[32599]: pgmap v2550: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:48:45.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:48:44 smithi167 ceph-mon[37140]: pgmap v2550: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:48:46.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:48:46.638+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:48:46.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:48:46.638+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:48:46.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:48:46.638+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:48:46.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:48:46.638+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:48:46.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:48:46.638+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:48:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:48:46 smithi100 ceph-mon[32599]: pgmap v2551: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:48:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:48:46 smithi138 ceph-mon[36954]: pgmap v2551: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:48:47.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:48:46 smithi167 ceph-mon[37140]: pgmap v2551: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:48:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:48:48 smithi138 ceph-mon[36954]: pgmap v2552: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:48:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:48:48 smithi100 ceph-mon[32599]: pgmap v2552: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:48:49.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:48:48 smithi167 ceph-mon[37140]: pgmap v2552: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:48:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:48:50 smithi138 ceph-mon[36954]: pgmap v2553: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:48:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:48:50 smithi100 ceph-mon[32599]: pgmap v2553: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:48:51.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:48:50 smithi167 ceph-mon[37140]: pgmap v2553: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:48:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:48:52 smithi138 ceph-mon[36954]: pgmap v2554: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:48:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:48:52 smithi100 ceph-mon[32599]: pgmap v2554: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:48:53.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:48:52 smithi167 ceph-mon[37140]: pgmap v2554: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:48:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:48:54 smithi138 ceph-mon[36954]: pgmap v2555: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:48:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:48:54 smithi100 ceph-mon[32599]: pgmap v2555: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:48:55.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:48:54 smithi167 ceph-mon[37140]: pgmap v2555: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:48:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:48:56 smithi138 ceph-mon[36954]: pgmap v2556: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:48:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:48:56 smithi100 ceph-mon[32599]: pgmap v2556: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:48:57.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:48:56 smithi167 ceph-mon[37140]: pgmap v2556: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:48:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:48:58 smithi138 ceph-mon[36954]: pgmap v2557: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:48:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:48:58 smithi100 ceph-mon[32599]: pgmap v2557: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:48:59.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:48:58 smithi167 ceph-mon[37140]: pgmap v2557: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:49:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:49:00 smithi138 ceph-mon[36954]: pgmap v2558: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:49:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:49:00 smithi100 ceph-mon[32599]: pgmap v2558: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:49:01.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:49:00 smithi167 ceph-mon[37140]: pgmap v2558: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:49:01.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:49:01.643+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:49:01.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:49:01.644+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:49:01.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:49:01.644+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:49:01.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:49:01.644+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:49:01.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:49:01.644+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:49:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:49:02 smithi138 ceph-mon[36954]: pgmap v2559: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:49:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:49:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:49:03.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:49:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:49:03.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:49:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:49:03.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:49:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:49:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:49:02 smithi100 ceph-mon[32599]: pgmap v2559: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:49:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:49:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:49:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:49:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:49:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:49:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:49:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:49:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:49:03.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:49:02 smithi167 ceph-mon[37140]: pgmap v2559: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:49:03.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:49:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:49:03.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:49:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:49:03.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:49:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:49:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:49:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:49:05.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:49:04 smithi138 ceph-mon[36954]: pgmap v2560: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-07-26T14:49:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:49:04 smithi100 ceph-mon[32599]: pgmap v2560: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-07-26T14:49:05.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:49:04 smithi167 ceph-mon[37140]: pgmap v2560: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-07-26T14:49:07.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:49:06 smithi138 ceph-mon[36954]: pgmap v2561: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:49:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:49:06 smithi100 ceph-mon[32599]: pgmap v2561: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:49:07.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:49:06 smithi167 ceph-mon[37140]: pgmap v2561: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:49:08.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:49:07 smithi138 ceph-mon[36954]: osdmap e210: 12 total, 12 up, 12 in 2024-07-26T14:49:08.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:49:07 smithi100 ceph-mon[32599]: osdmap e210: 12 total, 12 up, 12 in 2024-07-26T14:49:08.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:49:07 smithi167 ceph-mon[37140]: osdmap e210: 12 total, 12 up, 12 in 2024-07-26T14:49:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:49:08 smithi138 ceph-mon[36954]: pgmap v2563: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:49:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:49:08 smithi100 ceph-mon[32599]: pgmap v2563: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:49:09.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:49:08 smithi167 ceph-mon[37140]: pgmap v2563: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:49:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:49:10 smithi138 ceph-mon[36954]: pgmap v2564: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:49:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:49:10 smithi100 ceph-mon[32599]: pgmap v2564: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:49:11.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:49:10 smithi167 ceph-mon[37140]: pgmap v2564: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:49:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:49:12 smithi138 ceph-mon[36954]: pgmap v2565: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:49:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:49:12 smithi100 ceph-mon[32599]: pgmap v2565: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:49:13.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:49:12 smithi167 ceph-mon[37140]: pgmap v2565: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:49:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:49:14 smithi138 ceph-mon[36954]: pgmap v2566: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:49:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:49:14 smithi138 ceph-mon[36954]: osdmap e211: 12 total, 12 up, 12 in 2024-07-26T14:49:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:49:14 smithi100 ceph-mon[32599]: pgmap v2566: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:49:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:49:14 smithi100 ceph-mon[32599]: osdmap e211: 12 total, 12 up, 12 in 2024-07-26T14:49:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:49:14 smithi167 ceph-mon[37140]: pgmap v2566: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:49:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:49:14 smithi167 ceph-mon[37140]: osdmap e211: 12 total, 12 up, 12 in 2024-07-26T14:49:16.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:49:16.650+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:49:16.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:49:16.651+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:49:16.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:49:16.651+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:49:16.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:49:16.651+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:49:16.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:49:16.651+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:49:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:49:16 smithi100 ceph-mon[32599]: pgmap v2568: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T14:49:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:49:16 smithi138 ceph-mon[36954]: pgmap v2568: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T14:49:17.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:49:16 smithi167 ceph-mon[37140]: pgmap v2568: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T14:49:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:49:18 smithi138 ceph-mon[36954]: pgmap v2569: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:49:19.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:49:18 smithi100 ceph-mon[32599]: pgmap v2569: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:49:19.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:49:18 smithi167 ceph-mon[37140]: pgmap v2569: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:49:21.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:49:20 smithi138 ceph-mon[36954]: pgmap v2570: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:49:21.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:49:20 smithi100 ceph-mon[32599]: pgmap v2570: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:49:21.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:49:20 smithi167 ceph-mon[37140]: pgmap v2570: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:49:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:49:22 smithi138 ceph-mon[36954]: pgmap v2571: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:49:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:49:22 smithi100 ceph-mon[32599]: pgmap v2571: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:49:23.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:49:22 smithi167 ceph-mon[37140]: pgmap v2571: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:49:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:49:24 smithi138 ceph-mon[36954]: pgmap v2572: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:49:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:49:24 smithi100 ceph-mon[32599]: pgmap v2572: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:49:25.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:49:24 smithi167 ceph-mon[37140]: pgmap v2572: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:49:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:49:26 smithi138 ceph-mon[36954]: pgmap v2573: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:49:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:49:26 smithi100 ceph-mon[32599]: pgmap v2573: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:49:27.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:49:26 smithi167 ceph-mon[37140]: pgmap v2573: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:49:29.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:49:28 smithi138 ceph-mon[36954]: pgmap v2574: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:49:29.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:49:28 smithi100 ceph-mon[32599]: pgmap v2574: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:49:29.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:49:28 smithi167 ceph-mon[37140]: pgmap v2574: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:49:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:49:30 smithi138 ceph-mon[36954]: pgmap v2575: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:49:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:49:30 smithi100 ceph-mon[32599]: pgmap v2575: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:49:31.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:49:30 smithi167 ceph-mon[37140]: pgmap v2575: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:49:31.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:49:31.656+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:49:31.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:49:31.657+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:49:31.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:49:31.657+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:49:31.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:49:31.657+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:49:31.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:49:31.657+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:49:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:49:32 smithi138 ceph-mon[36954]: pgmap v2576: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:49:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:49:32 smithi100 ceph-mon[32599]: pgmap v2576: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:49:33.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:49:32 smithi167 ceph-mon[37140]: pgmap v2576: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:49:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:49:34 smithi138 ceph-mon[36954]: pgmap v2577: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:49:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:49:34 smithi100 ceph-mon[32599]: pgmap v2577: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:49:35.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:49:34 smithi167 ceph-mon[37140]: pgmap v2577: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:49:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:49:36 smithi138 ceph-mon[36954]: pgmap v2578: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:49:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:49:36 smithi100 ceph-mon[32599]: pgmap v2578: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:49:37.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:49:36 smithi167 ceph-mon[37140]: pgmap v2578: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:49:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:49:38 smithi138 ceph-mon[36954]: pgmap v2579: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:49:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:49:38 smithi100 ceph-mon[32599]: pgmap v2579: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:49:39.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:49:38 smithi167 ceph-mon[37140]: pgmap v2579: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:49:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:49:40 smithi138 ceph-mon[36954]: pgmap v2580: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:49:41.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:49:40 smithi100 ceph-mon[32599]: pgmap v2580: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:49:41.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:49:40 smithi167 ceph-mon[37140]: pgmap v2580: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:49:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:49:42 smithi138 ceph-mon[36954]: pgmap v2581: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:49:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:49:42 smithi100 ceph-mon[32599]: pgmap v2581: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:49:43.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:49:42 smithi167 ceph-mon[37140]: pgmap v2581: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:49:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:49:44 smithi100 ceph-mon[32599]: pgmap v2582: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:49:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:49:44 smithi167 ceph-mon[37140]: pgmap v2582: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:49:45.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:49:44 smithi138 ceph-mon[36954]: pgmap v2582: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:49:46.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:49:46.661+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:49:46.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:49:46.661+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:49:46.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:49:46.661+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:49:46.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:49:46.661+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:49:46.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:49:46.661+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:49:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:49:46 smithi100 ceph-mon[32599]: pgmap v2583: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:49:47.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:49:46 smithi167 ceph-mon[37140]: pgmap v2583: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:49:47.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:49:46 smithi138 ceph-mon[36954]: pgmap v2583: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:49:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:49:48 smithi100 ceph-mon[32599]: pgmap v2584: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:49:49.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:49:48 smithi167 ceph-mon[37140]: pgmap v2584: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:49:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:49:48 smithi138 ceph-mon[36954]: pgmap v2584: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:49:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:49:50 smithi100 ceph-mon[32599]: pgmap v2585: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:49:51.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:49:50 smithi167 ceph-mon[37140]: pgmap v2585: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:49:51.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:49:50 smithi138 ceph-mon[36954]: pgmap v2585: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:49:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:49:52 smithi100 ceph-mon[32599]: pgmap v2586: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:49:53.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:49:52 smithi167 ceph-mon[37140]: pgmap v2586: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:49:53.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:49:52 smithi138 ceph-mon[36954]: pgmap v2586: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:49:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:49:54 smithi100 ceph-mon[32599]: pgmap v2587: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:49:55.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:49:54 smithi167 ceph-mon[37140]: pgmap v2587: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:49:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:49:54 smithi138 ceph-mon[36954]: pgmap v2587: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:49:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:49:56 smithi100 ceph-mon[32599]: pgmap v2588: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:49:57.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:49:56 smithi167 ceph-mon[37140]: pgmap v2588: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:49:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:49:56 smithi138 ceph-mon[36954]: pgmap v2588: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:49:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:49:58 smithi100 ceph-mon[32599]: pgmap v2589: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:49:59.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:49:58 smithi167 ceph-mon[37140]: pgmap v2589: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:49:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:49:58 smithi138 ceph-mon[36954]: pgmap v2589: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:50:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:00 smithi100 ceph-mon[32599]: pgmap v2590: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:50:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:00 smithi100 ceph-mon[32599]: overall HEALTH_OK 2024-07-26T14:50:01.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:00 smithi167 ceph-mon[37140]: pgmap v2590: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:50:01.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:00 smithi167 ceph-mon[37140]: overall HEALTH_OK 2024-07-26T14:50:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:00 smithi138 ceph-mon[36954]: pgmap v2590: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:50:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:00 smithi138 ceph-mon[36954]: overall HEALTH_OK 2024-07-26T14:50:01.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:50:01.677+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:50:01.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:50:01.677+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:50:01.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:50:01.677+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:50:01.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:50:01.677+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:50:01.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:50:01.677+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:50:03.223 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:02 smithi138 ceph-mon[36954]: pgmap v2591: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:50:03.223 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:50:03.234 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:02 smithi167 ceph-mon[37140]: pgmap v2591: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:50:03.234 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:50:03.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:02 smithi100 ceph-mon[32599]: pgmap v2591: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:50:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:50:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:04 smithi100 ceph-mon[32599]: pgmap v2592: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T14:50:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:50:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:50:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:50:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:50:05.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:50:05.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:50:05.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:50:05.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:50:05.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:50:05.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:04 smithi167 ceph-mon[37140]: pgmap v2592: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T14:50:05.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:50:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:50:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:50:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:50:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:50:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:50:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:50:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:50:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:50:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:04 smithi138 ceph-mon[36954]: pgmap v2592: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T14:50:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:50:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:50:05.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:50:05.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:50:05.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:50:05.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:50:05.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:50:05.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:50:05.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:50:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:06 smithi100 ceph-mon[32599]: pgmap v2593: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:50:07.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:06 smithi167 ceph-mon[37140]: pgmap v2593: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:50:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:06 smithi138 ceph-mon[36954]: pgmap v2593: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:50:08.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:07 smithi167 ceph-mon[37140]: osdmap e212: 12 total, 12 up, 12 in 2024-07-26T14:50:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:07 smithi138 ceph-mon[36954]: osdmap e212: 12 total, 12 up, 12 in 2024-07-26T14:50:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:07 smithi100 ceph-mon[32599]: osdmap e212: 12 total, 12 up, 12 in 2024-07-26T14:50:09.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:08 smithi167 ceph-mon[37140]: pgmap v2595: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:50:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:08 smithi138 ceph-mon[36954]: pgmap v2595: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:50:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:08 smithi100 ceph-mon[32599]: pgmap v2595: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:50:11.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:11 smithi167 ceph-mon[37140]: pgmap v2596: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:50:11.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:11 smithi138 ceph-mon[36954]: pgmap v2596: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:50:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:11 smithi100 ceph-mon[32599]: pgmap v2596: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:50:13.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:13 smithi167 ceph-mon[37140]: pgmap v2597: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:50:13.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:13 smithi138 ceph-mon[36954]: pgmap v2597: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:50:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:13 smithi100 ceph-mon[32599]: pgmap v2597: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:50:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:15 smithi167 ceph-mon[37140]: pgmap v2598: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:50:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:15 smithi167 ceph-mon[37140]: osdmap e213: 12 total, 12 up, 12 in 2024-07-26T14:50:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:15 smithi138 ceph-mon[36954]: pgmap v2598: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:50:15.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:15 smithi138 ceph-mon[36954]: osdmap e213: 12 total, 12 up, 12 in 2024-07-26T14:50:15.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:15 smithi100 ceph-mon[32599]: pgmap v2598: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T14:50:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:15 smithi100 ceph-mon[32599]: osdmap e213: 12 total, 12 up, 12 in 2024-07-26T14:50:16.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:50:16.692+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:50:16.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:50:16.692+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:50:16.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:50:16.692+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:50:16.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:50:16.692+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:50:16.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:50:16.693+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:50:17.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:17 smithi167 ceph-mon[37140]: pgmap v2600: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 232 B/s wr, 6 op/s 2024-07-26T14:50:17.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:17 smithi138 ceph-mon[36954]: pgmap v2600: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 232 B/s wr, 6 op/s 2024-07-26T14:50:17.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:17 smithi100 ceph-mon[32599]: pgmap v2600: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 232 B/s wr, 6 op/s 2024-07-26T14:50:19.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:19 smithi167 ceph-mon[37140]: pgmap v2601: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:50:19.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:19 smithi138 ceph-mon[36954]: pgmap v2601: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:50:19.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:19 smithi100 ceph-mon[32599]: pgmap v2601: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:50:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:21 smithi167 ceph-mon[37140]: pgmap v2602: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:50:21.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:21 smithi138 ceph-mon[36954]: pgmap v2602: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:50:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:21 smithi100 ceph-mon[32599]: pgmap v2602: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:50:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:23 smithi138 ceph-mon[36954]: pgmap v2603: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:50:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:23 smithi100 ceph-mon[32599]: pgmap v2603: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:50:23.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:23 smithi167 ceph-mon[37140]: pgmap v2603: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:50:25.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:25 smithi167 ceph-mon[37140]: pgmap v2604: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:50:25.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:25 smithi138 ceph-mon[36954]: pgmap v2604: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:50:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:25 smithi100 ceph-mon[32599]: pgmap v2604: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:50:27.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:27 smithi138 ceph-mon[36954]: pgmap v2605: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:50:27.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:27 smithi100 ceph-mon[32599]: pgmap v2605: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:50:27.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:27 smithi167 ceph-mon[37140]: pgmap v2605: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:50:29.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:29 smithi138 ceph-mon[36954]: pgmap v2606: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:50:29.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:29 smithi100 ceph-mon[32599]: pgmap v2606: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:50:29.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:29 smithi167 ceph-mon[37140]: pgmap v2606: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:50:31.298 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:31 smithi100 ceph-mon[32599]: pgmap v2607: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:50:31.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:31 smithi138 ceph-mon[36954]: pgmap v2607: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:50:31.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:31 smithi167 ceph-mon[37140]: pgmap v2607: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:50:31.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:50:31.708+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:50:31.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:50:31.709+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:50:31.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:50:31.709+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:50:31.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:50:31.709+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:50:31.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:50:31.709+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:50:33.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:33 smithi138 ceph-mon[36954]: pgmap v2608: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:50:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:33 smithi100 ceph-mon[32599]: pgmap v2608: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:50:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:33 smithi167 ceph-mon[37140]: pgmap v2608: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:50:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:35 smithi138 ceph-mon[36954]: pgmap v2609: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T14:50:35.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:35 smithi100 ceph-mon[32599]: pgmap v2609: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T14:50:35.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:35 smithi167 ceph-mon[37140]: pgmap v2609: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T14:50:37.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:37 smithi138 ceph-mon[36954]: pgmap v2610: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:50:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:37 smithi100 ceph-mon[32599]: pgmap v2610: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:50:37.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:37 smithi167 ceph-mon[37140]: pgmap v2610: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:50:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:39 smithi138 ceph-mon[36954]: pgmap v2611: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:50:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:39 smithi100 ceph-mon[32599]: pgmap v2611: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:50:39.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:39 smithi167 ceph-mon[37140]: pgmap v2611: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:50:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:41 smithi138 ceph-mon[36954]: pgmap v2612: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:50:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:41 smithi100 ceph-mon[32599]: pgmap v2612: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:50:41.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:41 smithi167 ceph-mon[37140]: pgmap v2612: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:50:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:43 smithi138 ceph-mon[36954]: pgmap v2613: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:50:43.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:43 smithi100 ceph-mon[32599]: pgmap v2613: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:50:43.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:43 smithi167 ceph-mon[37140]: pgmap v2613: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:50:45.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:45 smithi138 ceph-mon[36954]: pgmap v2614: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T14:50:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:45 smithi100 ceph-mon[32599]: pgmap v2614: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T14:50:45.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:45 smithi167 ceph-mon[37140]: pgmap v2614: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T14:50:46.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:50:46.724+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:50:46.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:50:46.725+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:50:46.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:50:46.725+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:50:46.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:50:46.725+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:50:46.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:50:46.725+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:50:47.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:47 smithi138 ceph-mon[36954]: pgmap v2615: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:50:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:47 smithi100 ceph-mon[32599]: pgmap v2615: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:50:47.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:47 smithi167 ceph-mon[37140]: pgmap v2615: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:50:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:49 smithi138 ceph-mon[36954]: pgmap v2616: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:50:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:49 smithi100 ceph-mon[32599]: pgmap v2616: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:50:49.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:49 smithi167 ceph-mon[37140]: pgmap v2616: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:50:51.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:51 smithi138 ceph-mon[36954]: pgmap v2617: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:50:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:51 smithi100 ceph-mon[32599]: pgmap v2617: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:50:51.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:51 smithi167 ceph-mon[37140]: pgmap v2617: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:50:53.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:53 smithi138 ceph-mon[36954]: pgmap v2618: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:50:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:53 smithi100 ceph-mon[32599]: pgmap v2618: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:50:53.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:53 smithi167 ceph-mon[37140]: pgmap v2618: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:50:55.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:55 smithi167 ceph-mon[37140]: pgmap v2619: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:50:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:55 smithi138 ceph-mon[36954]: pgmap v2619: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:50:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:55 smithi100 ceph-mon[32599]: pgmap v2619: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:50:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:57 smithi138 ceph-mon[36954]: pgmap v2620: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:50:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:57 smithi100 ceph-mon[32599]: pgmap v2620: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:50:57.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:57 smithi167 ceph-mon[37140]: pgmap v2620: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:50:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:50:59 smithi138 ceph-mon[36954]: pgmap v2621: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:50:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:50:59 smithi100 ceph-mon[32599]: pgmap v2621: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:50:59.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:50:59 smithi167 ceph-mon[37140]: pgmap v2621: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:51:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:01 smithi138 ceph-mon[36954]: pgmap v2622: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:51:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:01 smithi100 ceph-mon[32599]: pgmap v2622: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:51:01.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:01 smithi167 ceph-mon[37140]: pgmap v2622: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:51:01.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:51:01.740+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:51:01.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:51:01.741+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:51:01.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:51:01.741+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:51:01.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:51:01.741+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:51:01.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:51:01.741+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:51:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:03 smithi138 ceph-mon[36954]: pgmap v2623: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:51:03.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:03 smithi100 ceph-mon[32599]: pgmap v2623: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:51:03.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:03 smithi167 ceph-mon[37140]: pgmap v2623: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:51:03.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:51:03.547+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T14:51:03.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:51:03.547+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T14:51:03.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:51:03.548+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 33 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c020 con 0x7fe5a81087a0 2024-07-26T14:51:03.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:51:03.548+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 33 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T14:51:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:05 smithi138 ceph-mon[36954]: pgmap v2624: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T14:51:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:05 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:51:05.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:05 smithi100 ceph-mon[32599]: pgmap v2624: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T14:51:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:05 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:51:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:05 smithi167 ceph-mon[37140]: pgmap v2624: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T14:51:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:05 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:51:06.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:51:06.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:51:06.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:51:06.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:51:06.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:51:06.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:51:06.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:51:06.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:51:06.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:51:06.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:06 smithi167 ceph-mon[37140]: pgmap v2625: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:51:06.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:51:06.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:51:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:51:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:51:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:51:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:51:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:51:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:51:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:51:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:06 smithi138 ceph-mon[36954]: pgmap v2625: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:51:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:51:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:51:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:51:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:51:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:51:06.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:51:06.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:51:06.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:51:06.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:51:06.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:06 smithi100 ceph-mon[32599]: pgmap v2625: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:51:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:08 smithi100 ceph-mon[32599]: osdmap e214: 12 total, 12 up, 12 in 2024-07-26T14:51:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:08 smithi100 ceph-mon[32599]: pgmap v2627: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:51:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:08 smithi167 ceph-mon[37140]: osdmap e214: 12 total, 12 up, 12 in 2024-07-26T14:51:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:08 smithi167 ceph-mon[37140]: pgmap v2627: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:51:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:08 smithi138 ceph-mon[36954]: osdmap e214: 12 total, 12 up, 12 in 2024-07-26T14:51:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:08 smithi138 ceph-mon[36954]: pgmap v2627: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:51:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:10 smithi138 ceph-mon[36954]: pgmap v2628: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:51:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:10 smithi100 ceph-mon[32599]: pgmap v2628: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:51:11.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:10 smithi167 ceph-mon[37140]: pgmap v2628: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:51:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:12 smithi138 ceph-mon[36954]: pgmap v2629: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:51:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:12 smithi100 ceph-mon[32599]: pgmap v2629: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:51:13.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:12 smithi167 ceph-mon[37140]: pgmap v2629: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:51:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:14 smithi138 ceph-mon[36954]: pgmap v2630: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:51:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:14 smithi138 ceph-mon[36954]: osdmap e215: 12 total, 12 up, 12 in 2024-07-26T14:51:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:14 smithi100 ceph-mon[32599]: pgmap v2630: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:51:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:14 smithi100 ceph-mon[32599]: osdmap e215: 12 total, 12 up, 12 in 2024-07-26T14:51:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:14 smithi167 ceph-mon[37140]: pgmap v2630: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:51:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:14 smithi167 ceph-mon[37140]: osdmap e215: 12 total, 12 up, 12 in 2024-07-26T14:51:16.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:51:16.743+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:51:16.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:51:16.743+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:51:16.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:51:16.743+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:51:16.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:51:16.743+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:51:16.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:51:16.743+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:51:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:16 smithi138 ceph-mon[36954]: pgmap v2632: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 252 B/s wr, 6 op/s 2024-07-26T14:51:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:16 smithi100 ceph-mon[32599]: pgmap v2632: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 252 B/s wr, 6 op/s 2024-07-26T14:51:17.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:16 smithi167 ceph-mon[37140]: pgmap v2632: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 252 B/s wr, 6 op/s 2024-07-26T14:51:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:18 smithi138 ceph-mon[36954]: pgmap v2633: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:51:19.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:18 smithi100 ceph-mon[32599]: pgmap v2633: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:51:19.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:18 smithi167 ceph-mon[37140]: pgmap v2633: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:51:21.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:20 smithi138 ceph-mon[36954]: pgmap v2634: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:51:21.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:20 smithi100 ceph-mon[32599]: pgmap v2634: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:51:21.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:20 smithi167 ceph-mon[37140]: pgmap v2634: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:51:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:22 smithi138 ceph-mon[36954]: pgmap v2635: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:51:23.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:22 smithi100 ceph-mon[32599]: pgmap v2635: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:51:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:22 smithi167 ceph-mon[37140]: pgmap v2635: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:51:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:24 smithi138 ceph-mon[36954]: pgmap v2636: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:51:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:24 smithi100 ceph-mon[32599]: pgmap v2636: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:51:25.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:24 smithi167 ceph-mon[37140]: pgmap v2636: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:51:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:26 smithi138 ceph-mon[36954]: pgmap v2637: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:51:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:26 smithi100 ceph-mon[32599]: pgmap v2637: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:51:27.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:26 smithi167 ceph-mon[37140]: pgmap v2637: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:51:29.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:28 smithi138 ceph-mon[36954]: pgmap v2638: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:51:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:28 smithi100 ceph-mon[32599]: pgmap v2638: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:51:29.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:28 smithi167 ceph-mon[37140]: pgmap v2638: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:51:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:30 smithi138 ceph-mon[36954]: pgmap v2639: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:51:31.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:30 smithi100 ceph-mon[32599]: pgmap v2639: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:51:31.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:30 smithi167 ceph-mon[37140]: pgmap v2639: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:51:31.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:51:31.760+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:51:31.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:51:31.761+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:51:31.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:51:31.761+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:51:31.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:51:31.761+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:51:31.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:51:31.761+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:51:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:32 smithi138 ceph-mon[36954]: pgmap v2640: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:51:33.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:32 smithi100 ceph-mon[32599]: pgmap v2640: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:51:33.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:32 smithi167 ceph-mon[37140]: pgmap v2640: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:51:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:34 smithi138 ceph-mon[36954]: pgmap v2641: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:51:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:34 smithi100 ceph-mon[32599]: pgmap v2641: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:51:35.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:34 smithi167 ceph-mon[37140]: pgmap v2641: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:51:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:36 smithi138 ceph-mon[36954]: pgmap v2642: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:51:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:36 smithi100 ceph-mon[32599]: pgmap v2642: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:51:37.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:36 smithi167 ceph-mon[37140]: pgmap v2642: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:51:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:38 smithi138 ceph-mon[36954]: pgmap v2643: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:51:39.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:38 smithi100 ceph-mon[32599]: pgmap v2643: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:51:39.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:38 smithi167 ceph-mon[37140]: pgmap v2643: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:51:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:40 smithi138 ceph-mon[36954]: pgmap v2644: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:51:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:40 smithi100 ceph-mon[32599]: pgmap v2644: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:51:41.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:40 smithi167 ceph-mon[37140]: pgmap v2644: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:51:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:42 smithi138 ceph-mon[36954]: pgmap v2645: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:51:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:42 smithi100 ceph-mon[32599]: pgmap v2645: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:51:43.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:42 smithi167 ceph-mon[37140]: pgmap v2645: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:51:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:44 smithi138 ceph-mon[36954]: pgmap v2646: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:51:45.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:44 smithi100 ceph-mon[32599]: pgmap v2646: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:51:45.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:44 smithi167 ceph-mon[37140]: pgmap v2646: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:51:46.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:51:46.777+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:51:46.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:51:46.777+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:51:46.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:51:46.777+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:51:46.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:51:46.777+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:51:46.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:51:46.777+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:51:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:46 smithi138 ceph-mon[36954]: pgmap v2647: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:51:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:46 smithi100 ceph-mon[32599]: pgmap v2647: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:51:47.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:46 smithi167 ceph-mon[37140]: pgmap v2647: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:51:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:48 smithi138 ceph-mon[36954]: pgmap v2648: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:51:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:48 smithi100 ceph-mon[32599]: pgmap v2648: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:51:49.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:48 smithi167 ceph-mon[37140]: pgmap v2648: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:51:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:50 smithi138 ceph-mon[36954]: pgmap v2649: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:51:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:50 smithi100 ceph-mon[32599]: pgmap v2649: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:51:51.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:50 smithi167 ceph-mon[37140]: pgmap v2649: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:51:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:52 smithi138 ceph-mon[36954]: pgmap v2650: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:51:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:52 smithi100 ceph-mon[32599]: pgmap v2650: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:51:53.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:52 smithi167 ceph-mon[37140]: pgmap v2650: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:51:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:54 smithi138 ceph-mon[36954]: pgmap v2651: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:51:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:54 smithi100 ceph-mon[32599]: pgmap v2651: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:51:55.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:54 smithi167 ceph-mon[37140]: pgmap v2651: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:51:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:56 smithi138 ceph-mon[36954]: pgmap v2652: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:51:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:56 smithi100 ceph-mon[32599]: pgmap v2652: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:51:57.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:56 smithi167 ceph-mon[37140]: pgmap v2652: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:51:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:51:58 smithi138 ceph-mon[36954]: pgmap v2653: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:51:59.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:51:58 smithi100 ceph-mon[32599]: pgmap v2653: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:51:59.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:51:58 smithi167 ceph-mon[37140]: pgmap v2653: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:52:01.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:52:00 smithi100 ceph-mon[32599]: pgmap v2654: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:52:01.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:52:00 smithi167 ceph-mon[37140]: pgmap v2654: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:52:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:52:00 smithi138 ceph-mon[36954]: pgmap v2654: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:52:01.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:52:01.788+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:52:01.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:52:01.789+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:52:01.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:52:01.789+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:52:01.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:52:01.789+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:52:01.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:52:01.789+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:52:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:52:02 smithi100 ceph-mon[32599]: pgmap v2655: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:52:03.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:52:02 smithi167 ceph-mon[37140]: pgmap v2655: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:52:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:52:02 smithi138 ceph-mon[36954]: pgmap v2655: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:52:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:52:04 smithi100 ceph-mon[32599]: pgmap v2656: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T14:52:05.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:52:04 smithi167 ceph-mon[37140]: pgmap v2656: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T14:52:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:52:04 smithi138 ceph-mon[36954]: pgmap v2656: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T14:52:06.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:52:05 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:52:06.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:52:05 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:52:06.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:52:05 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:52:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:52:06 smithi100 ceph-mon[32599]: pgmap v2657: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:52:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:52:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:52:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:52:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:52:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:52:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:52:07.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:52:06 smithi167 ceph-mon[37140]: pgmap v2657: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:52:07.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:52:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:52:07.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:52:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:52:07.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:52:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:52:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:52:06 smithi138 ceph-mon[36954]: pgmap v2657: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:52:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:52:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:52:07.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:52:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:52:07.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:52:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:52:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:52:08 smithi138 ceph-mon[36954]: osdmap e216: 12 total, 12 up, 12 in 2024-07-26T14:52:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:52:08 smithi138 ceph-mon[36954]: pgmap v2659: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:52:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:52:08 smithi100 ceph-mon[32599]: osdmap e216: 12 total, 12 up, 12 in 2024-07-26T14:52:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:52:08 smithi100 ceph-mon[32599]: pgmap v2659: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:52:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:52:08 smithi167 ceph-mon[37140]: osdmap e216: 12 total, 12 up, 12 in 2024-07-26T14:52:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:52:08 smithi167 ceph-mon[37140]: pgmap v2659: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:52:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:52:10 smithi167 ceph-mon[37140]: pgmap v2660: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:52:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:52:10 smithi138 ceph-mon[36954]: pgmap v2660: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:52:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:52:10 smithi100 ceph-mon[32599]: pgmap v2660: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:52:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:52:12 smithi138 ceph-mon[36954]: pgmap v2661: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:52:13.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:52:12 smithi100 ceph-mon[32599]: pgmap v2661: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:52:13.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:52:12 smithi167 ceph-mon[37140]: pgmap v2661: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:52:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:52:14 smithi138 ceph-mon[36954]: pgmap v2662: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:52:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:52:14 smithi138 ceph-mon[36954]: osdmap e217: 12 total, 12 up, 12 in 2024-07-26T14:52:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:52:14 smithi100 ceph-mon[32599]: pgmap v2662: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:52:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:52:14 smithi100 ceph-mon[32599]: osdmap e217: 12 total, 12 up, 12 in 2024-07-26T14:52:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:52:14 smithi167 ceph-mon[37140]: pgmap v2662: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:52:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:52:14 smithi167 ceph-mon[37140]: osdmap e217: 12 total, 12 up, 12 in 2024-07-26T14:52:16.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:52:16.804+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:52:16.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:52:16.805+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:52:16.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:52:16.805+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:52:16.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:52:16.805+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:52:16.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:52:16.805+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:52:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:52:16 smithi138 ceph-mon[36954]: pgmap v2664: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 239 B/s wr, 6 op/s 2024-07-26T14:52:17.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:52:16 smithi100 ceph-mon[32599]: pgmap v2664: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 239 B/s wr, 6 op/s 2024-07-26T14:52:17.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:52:16 smithi167 ceph-mon[37140]: pgmap v2664: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 239 B/s wr, 6 op/s 2024-07-26T14:52:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:52:18 smithi138 ceph-mon[36954]: pgmap v2665: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:52:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:52:18 smithi100 ceph-mon[32599]: pgmap v2665: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:52:19.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:52:18 smithi167 ceph-mon[37140]: pgmap v2665: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:52:21.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:52:20 smithi138 ceph-mon[36954]: pgmap v2666: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:52:21.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:52:20 smithi100 ceph-mon[32599]: pgmap v2666: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:52:21.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:52:20 smithi167 ceph-mon[37140]: pgmap v2666: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:52:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:52:22 smithi138 ceph-mon[36954]: pgmap v2667: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:52:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:52:22 smithi100 ceph-mon[32599]: pgmap v2667: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:52:23.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:52:22 smithi167 ceph-mon[37140]: pgmap v2667: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:52:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:52:24 smithi138 ceph-mon[36954]: pgmap v2668: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:52:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:52:24 smithi100 ceph-mon[32599]: pgmap v2668: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:52:25.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:52:24 smithi167 ceph-mon[37140]: pgmap v2668: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:52:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:52:26 smithi138 ceph-mon[36954]: pgmap v2669: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:52:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:52:26 smithi100 ceph-mon[32599]: pgmap v2669: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:52:27.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:52:26 smithi167 ceph-mon[37140]: pgmap v2669: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:52:29.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:52:28 smithi138 ceph-mon[36954]: pgmap v2670: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:52:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:52:28 smithi100 ceph-mon[32599]: pgmap v2670: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:52:29.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:52:28 smithi167 ceph-mon[37140]: pgmap v2670: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:52:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:52:30 smithi167 ceph-mon[37140]: pgmap v2671: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:52:31.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:52:30 smithi100 ceph-mon[32599]: pgmap v2671: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:52:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:52:30 smithi138 ceph-mon[36954]: pgmap v2671: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:52:31.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:52:31.820+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:52:31.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:52:31.821+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:52:31.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:52:31.821+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:52:31.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:52:31.821+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:52:31.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:52:31.821+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:52:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:52:32 smithi138 ceph-mon[36954]: pgmap v2672: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:52:33.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:52:32 smithi100 ceph-mon[32599]: pgmap v2672: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:52:33.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:52:32 smithi167 ceph-mon[37140]: pgmap v2672: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:52:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:52:34 smithi138 ceph-mon[36954]: pgmap v2673: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:52:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:52:34 smithi100 ceph-mon[32599]: pgmap v2673: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:52:35.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:52:34 smithi167 ceph-mon[37140]: pgmap v2673: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:52:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:52:36 smithi138 ceph-mon[36954]: pgmap v2674: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:52:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:52:36 smithi100 ceph-mon[32599]: pgmap v2674: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:52:37.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:52:36 smithi167 ceph-mon[37140]: pgmap v2674: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:52:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:52:38 smithi138 ceph-mon[36954]: pgmap v2675: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:52:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:52:38 smithi100 ceph-mon[32599]: pgmap v2675: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:52:39.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:52:38 smithi167 ceph-mon[37140]: pgmap v2675: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:52:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:52:40 smithi138 ceph-mon[36954]: pgmap v2676: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:52:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:52:40 smithi100 ceph-mon[32599]: pgmap v2676: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:52:41.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:52:40 smithi167 ceph-mon[37140]: pgmap v2676: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:52:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:52:42 smithi138 ceph-mon[36954]: pgmap v2677: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:52:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:52:42 smithi100 ceph-mon[32599]: pgmap v2677: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:52:43.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:52:42 smithi167 ceph-mon[37140]: pgmap v2677: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:52:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:52:44 smithi138 ceph-mon[36954]: pgmap v2678: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:52:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:52:44 smithi100 ceph-mon[32599]: pgmap v2678: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:52:45.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:52:44 smithi167 ceph-mon[37140]: pgmap v2678: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:52:46.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:52:46.836+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:52:46.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:52:46.837+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:52:46.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:52:46.837+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:52:46.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:52:46.837+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:52:46.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:52:46.837+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:52:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:52:46 smithi138 ceph-mon[36954]: pgmap v2679: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:52:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:52:46 smithi100 ceph-mon[32599]: pgmap v2679: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:52:47.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:52:46 smithi167 ceph-mon[37140]: pgmap v2679: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:52:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:52:48 smithi138 ceph-mon[36954]: pgmap v2680: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:52:49.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:52:48 smithi100 ceph-mon[32599]: pgmap v2680: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:52:49.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:52:48 smithi167 ceph-mon[37140]: pgmap v2680: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:52:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:52:50 smithi138 ceph-mon[36954]: pgmap v2681: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:52:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:52:50 smithi100 ceph-mon[32599]: pgmap v2681: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:52:51.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:52:50 smithi167 ceph-mon[37140]: pgmap v2681: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:52:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:52:52 smithi138 ceph-mon[36954]: pgmap v2682: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:52:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:52:52 smithi100 ceph-mon[32599]: pgmap v2682: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:52:53.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:52:52 smithi167 ceph-mon[37140]: pgmap v2682: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:52:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:52:54 smithi138 ceph-mon[36954]: pgmap v2683: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:52:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:52:54 smithi100 ceph-mon[32599]: pgmap v2683: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:52:55.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:52:54 smithi167 ceph-mon[37140]: pgmap v2683: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:52:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:52:56 smithi138 ceph-mon[36954]: pgmap v2684: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:52:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:52:56 smithi100 ceph-mon[32599]: pgmap v2684: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:52:57.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:52:56 smithi167 ceph-mon[37140]: pgmap v2684: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:52:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:52:58 smithi138 ceph-mon[36954]: pgmap v2685: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:52:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:52:58 smithi100 ceph-mon[32599]: pgmap v2685: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:52:59.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:52:58 smithi167 ceph-mon[37140]: pgmap v2685: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:53:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:53:00 smithi138 ceph-mon[36954]: pgmap v2686: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:53:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:53:00 smithi100 ceph-mon[32599]: pgmap v2686: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:53:01.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:53:00 smithi167 ceph-mon[37140]: pgmap v2686: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:53:01.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:53:01.853+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:53:01.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:53:01.853+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:53:01.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:53:01.853+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:53:01.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:53:01.853+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:53:01.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:53:01.853+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:53:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:53:02 smithi138 ceph-mon[36954]: pgmap v2687: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:53:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:53:02 smithi100 ceph-mon[32599]: pgmap v2687: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:53:03.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:53:02 smithi167 ceph-mon[37140]: pgmap v2687: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:53:05.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:53:04 smithi138 ceph-mon[36954]: pgmap v2688: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T14:53:05.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:53:04 smithi100 ceph-mon[32599]: pgmap v2688: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T14:53:05.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:53:04 smithi167 ceph-mon[37140]: pgmap v2688: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T14:53:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:53:06 smithi100 ceph-mon[32599]: pgmap v2689: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:53:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:53:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:53:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:53:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:53:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:53:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:53:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:53:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:53:07.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:53:06 smithi167 ceph-mon[37140]: pgmap v2689: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:53:07.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:53:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:53:07.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:53:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:53:07.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:53:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:53:07.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:53:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:53:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:53:06 smithi138 ceph-mon[36954]: pgmap v2689: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:53:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:53:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:53:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:53:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:53:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:53:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:53:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:53:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:53:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:53:08 smithi138 ceph-mon[36954]: osdmap e218: 12 total, 12 up, 12 in 2024-07-26T14:53:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:53:08 smithi138 ceph-mon[36954]: pgmap v2691: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:53:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:53:08 smithi100 ceph-mon[32599]: osdmap e218: 12 total, 12 up, 12 in 2024-07-26T14:53:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:53:08 smithi100 ceph-mon[32599]: pgmap v2691: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:53:09.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:53:08 smithi167 ceph-mon[37140]: osdmap e218: 12 total, 12 up, 12 in 2024-07-26T14:53:09.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:53:08 smithi167 ceph-mon[37140]: pgmap v2691: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:53:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:53:10 smithi138 ceph-mon[36954]: pgmap v2692: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:53:11.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:53:10 smithi100 ceph-mon[32599]: pgmap v2692: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:53:11.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:53:10 smithi167 ceph-mon[37140]: pgmap v2692: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:53:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:53:12 smithi138 ceph-mon[36954]: pgmap v2693: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:53:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:53:12 smithi100 ceph-mon[32599]: pgmap v2693: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:53:13.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:53:12 smithi167 ceph-mon[37140]: pgmap v2693: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:53:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:53:14 smithi138 ceph-mon[36954]: pgmap v2694: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:53:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:53:14 smithi138 ceph-mon[36954]: osdmap e219: 12 total, 12 up, 12 in 2024-07-26T14:53:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:53:14 smithi100 ceph-mon[32599]: pgmap v2694: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:53:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:53:14 smithi100 ceph-mon[32599]: osdmap e219: 12 total, 12 up, 12 in 2024-07-26T14:53:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:53:14 smithi167 ceph-mon[37140]: pgmap v2694: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:53:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:53:14 smithi167 ceph-mon[37140]: osdmap e219: 12 total, 12 up, 12 in 2024-07-26T14:53:16.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:53:16.868+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:53:16.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:53:16.869+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:53:16.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:53:16.869+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:53:16.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:53:16.869+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:53:16.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:53:16.869+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:53:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:53:16 smithi138 ceph-mon[36954]: pgmap v2696: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 254 B/s wr, 6 op/s 2024-07-26T14:53:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:53:16 smithi100 ceph-mon[32599]: pgmap v2696: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 254 B/s wr, 6 op/s 2024-07-26T14:53:17.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:53:16 smithi167 ceph-mon[37140]: pgmap v2696: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 254 B/s wr, 6 op/s 2024-07-26T14:53:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:53:18 smithi138 ceph-mon[36954]: pgmap v2697: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:53:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:53:18 smithi100 ceph-mon[32599]: pgmap v2697: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:53:19.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:53:18 smithi167 ceph-mon[37140]: pgmap v2697: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:53:21.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:53:20 smithi138 ceph-mon[36954]: pgmap v2698: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:53:21.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:53:20 smithi100 ceph-mon[32599]: pgmap v2698: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:53:21.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:53:20 smithi167 ceph-mon[37140]: pgmap v2698: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:53:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:53:22 smithi138 ceph-mon[36954]: pgmap v2699: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:53:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:53:22 smithi100 ceph-mon[32599]: pgmap v2699: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:53:23.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:53:22 smithi167 ceph-mon[37140]: pgmap v2699: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:53:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:53:24 smithi138 ceph-mon[36954]: pgmap v2700: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:53:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:53:24 smithi100 ceph-mon[32599]: pgmap v2700: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:53:25.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:53:24 smithi167 ceph-mon[37140]: pgmap v2700: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:53:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:53:26 smithi138 ceph-mon[36954]: pgmap v2701: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:53:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:53:26 smithi100 ceph-mon[32599]: pgmap v2701: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:53:27.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:53:26 smithi167 ceph-mon[37140]: pgmap v2701: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:53:29.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:53:28 smithi138 ceph-mon[36954]: pgmap v2702: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:53:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:53:28 smithi100 ceph-mon[32599]: pgmap v2702: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:53:29.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:53:28 smithi167 ceph-mon[37140]: pgmap v2702: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:53:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:53:30 smithi138 ceph-mon[36954]: pgmap v2703: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:53:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:53:30 smithi100 ceph-mon[32599]: pgmap v2703: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:53:31.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:53:30 smithi167 ceph-mon[37140]: pgmap v2703: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:53:31.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:53:31.885+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:53:31.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:53:31.885+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:53:31.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:53:31.885+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:53:31.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:53:31.885+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:53:31.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:53:31.885+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:53:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:53:32 smithi138 ceph-mon[36954]: pgmap v2704: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:53:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:53:32 smithi100 ceph-mon[32599]: pgmap v2704: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:53:33.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:53:32 smithi167 ceph-mon[37140]: pgmap v2704: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:53:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:53:34 smithi100 ceph-mon[32599]: pgmap v2705: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:53:35.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:53:34 smithi167 ceph-mon[37140]: pgmap v2705: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:53:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:53:34 smithi138 ceph-mon[36954]: pgmap v2705: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:53:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:53:36 smithi100 ceph-mon[32599]: pgmap v2706: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:53:37.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:53:36 smithi167 ceph-mon[37140]: pgmap v2706: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:53:37.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:53:36 smithi138 ceph-mon[36954]: pgmap v2706: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:53:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:53:38 smithi100 ceph-mon[32599]: pgmap v2707: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:53:39.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:53:38 smithi167 ceph-mon[37140]: pgmap v2707: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:53:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:53:38 smithi138 ceph-mon[36954]: pgmap v2707: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:53:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:53:40 smithi100 ceph-mon[32599]: pgmap v2708: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:53:41.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:53:40 smithi167 ceph-mon[37140]: pgmap v2708: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:53:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:53:40 smithi138 ceph-mon[36954]: pgmap v2708: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:53:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:53:42 smithi100 ceph-mon[32599]: pgmap v2709: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:53:43.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:53:42 smithi167 ceph-mon[37140]: pgmap v2709: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:53:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:53:42 smithi138 ceph-mon[36954]: pgmap v2709: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:53:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:53:44 smithi100 ceph-mon[32599]: pgmap v2710: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:53:45.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:53:44 smithi167 ceph-mon[37140]: pgmap v2710: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:53:45.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:53:44 smithi138 ceph-mon[36954]: pgmap v2710: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:53:46.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:53:46.895+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:53:46.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:53:46.896+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:53:46.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:53:46.896+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:53:46.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:53:46.896+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:53:46.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:53:46.896+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:53:47.242 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:53:46 smithi138 ceph-mon[36954]: pgmap v2711: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:53:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:53:46 smithi100 ceph-mon[32599]: pgmap v2711: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:53:47.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:53:46 smithi167 ceph-mon[37140]: pgmap v2711: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:53:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:53:48 smithi100 ceph-mon[32599]: pgmap v2712: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:53:49.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:53:48 smithi167 ceph-mon[37140]: pgmap v2712: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:53:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:53:48 smithi138 ceph-mon[36954]: pgmap v2712: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:53:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:53:50 smithi100 ceph-mon[32599]: pgmap v2713: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:53:51.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:53:50 smithi167 ceph-mon[37140]: pgmap v2713: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:53:51.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:53:50 smithi138 ceph-mon[36954]: pgmap v2713: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:53:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:53:52 smithi100 ceph-mon[32599]: pgmap v2714: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:53:53.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:53:52 smithi167 ceph-mon[37140]: pgmap v2714: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:53:53.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:53:52 smithi138 ceph-mon[36954]: pgmap v2714: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:53:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:53:54 smithi100 ceph-mon[32599]: pgmap v2715: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:53:55.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:53:54 smithi167 ceph-mon[37140]: pgmap v2715: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:53:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:53:54 smithi138 ceph-mon[36954]: pgmap v2715: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T14:53:57.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:53:56 smithi100 ceph-mon[32599]: pgmap v2716: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:53:57.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:53:56 smithi167 ceph-mon[37140]: pgmap v2716: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:53:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:53:56 smithi138 ceph-mon[36954]: pgmap v2716: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:53:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:53:58 smithi100 ceph-mon[32599]: pgmap v2717: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:53:59.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:53:58 smithi167 ceph-mon[37140]: pgmap v2717: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:53:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:53:58 smithi138 ceph-mon[36954]: pgmap v2717: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:54:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:54:00 smithi100 ceph-mon[32599]: pgmap v2718: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:54:01.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:54:00 smithi167 ceph-mon[37140]: pgmap v2718: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:54:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:54:00 smithi138 ceph-mon[36954]: pgmap v2718: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:54:01.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:54:01.898+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:54:01.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:54:01.899+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:54:01.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:54:01.899+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:54:01.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:54:01.899+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:54:01.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:54:01.899+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:54:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:54:02 smithi100 ceph-mon[32599]: pgmap v2719: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:54:03.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:54:02 smithi167 ceph-mon[37140]: pgmap v2719: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:54:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:54:02 smithi138 ceph-mon[36954]: pgmap v2719: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:54:03.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:54:03.573+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T14:54:03.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:54:03.573+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T14:54:03.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:54:03.573+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 34 ==== auth_reply(proto 2 0 (0) Success) ==== 783+0+0 (secure 0 0 0) 0x7fe5a008c070 con 0x7fe5a81087a0 2024-07-26T14:54:03.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:54:03.574+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c003690 con 0x7fe5a81087a0 2024-07-26T14:54:03.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:54:03.574+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 34 ==== auth_reply(proto 2 0 (0) Success) ==== 612+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T14:54:03.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:54:03.574+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T14:54:03.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:54:03.574+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 35 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c003690 con 0x7fe5a81087a0 2024-07-26T14:54:03.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:54:03.574+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 35 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T14:54:05.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:54:04 smithi167 ceph-mon[37140]: pgmap v2720: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T14:54:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:54:04 smithi138 ceph-mon[36954]: pgmap v2720: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T14:54:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:54:04 smithi100 ceph-mon[32599]: pgmap v2720: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T14:54:07.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:54:07 smithi167 ceph-mon[37140]: pgmap v2721: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:54:07.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:54:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:54:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:54:07 smithi138 ceph-mon[36954]: pgmap v2721: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:54:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:54:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:54:07.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:54:07 smithi100 ceph-mon[32599]: pgmap v2721: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:54:07.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:54:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:54:08.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:54:08 smithi167 ceph-mon[37140]: osdmap e220: 12 total, 12 up, 12 in 2024-07-26T14:54:08.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:54:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:54:08.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:54:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:54:08.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:54:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:54:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:54:08 smithi138 ceph-mon[36954]: osdmap e220: 12 total, 12 up, 12 in 2024-07-26T14:54:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:54:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:54:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:54:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:54:08.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:54:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:54:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:54:08 smithi100 ceph-mon[32599]: osdmap e220: 12 total, 12 up, 12 in 2024-07-26T14:54:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:54:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:54:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:54:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:54:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:54:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:54:09.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:54:09 smithi167 ceph-mon[37140]: pgmap v2723: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:54:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:54:09 smithi138 ceph-mon[36954]: pgmap v2723: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:54:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:54:09 smithi100 ceph-mon[32599]: pgmap v2723: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:54:11.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:54:11 smithi167 ceph-mon[37140]: pgmap v2724: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:54:11.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:54:11 smithi138 ceph-mon[36954]: pgmap v2724: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:54:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:54:11 smithi100 ceph-mon[32599]: pgmap v2724: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:54:13.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:54:13 smithi167 ceph-mon[37140]: pgmap v2725: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:54:13.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:54:13 smithi138 ceph-mon[36954]: pgmap v2725: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:54:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:54:13 smithi100 ceph-mon[32599]: pgmap v2725: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:54:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:54:15 smithi167 ceph-mon[37140]: pgmap v2726: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:54:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:54:15 smithi167 ceph-mon[37140]: osdmap e221: 12 total, 12 up, 12 in 2024-07-26T14:54:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:54:15 smithi138 ceph-mon[36954]: pgmap v2726: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:54:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:54:15 smithi138 ceph-mon[36954]: osdmap e221: 12 total, 12 up, 12 in 2024-07-26T14:54:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:54:15 smithi100 ceph-mon[32599]: pgmap v2726: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:54:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:54:15 smithi100 ceph-mon[32599]: osdmap e221: 12 total, 12 up, 12 in 2024-07-26T14:54:16.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:54:16.903+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:54:16.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:54:16.904+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:54:16.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:54:16.904+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:54:16.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:54:16.904+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:54:16.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:54:16.904+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:54:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:54:17 smithi100 ceph-mon[32599]: pgmap v2728: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T14:54:17.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:54:17 smithi167 ceph-mon[37140]: pgmap v2728: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T14:54:17.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:54:17 smithi138 ceph-mon[36954]: pgmap v2728: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T14:54:19.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:54:19 smithi138 ceph-mon[36954]: pgmap v2729: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:54:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:54:19 smithi100 ceph-mon[32599]: pgmap v2729: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:54:19.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:54:19 smithi167 ceph-mon[37140]: pgmap v2729: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:54:21.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:54:21 smithi138 ceph-mon[36954]: pgmap v2730: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:54:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:54:21 smithi100 ceph-mon[32599]: pgmap v2730: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:54:21.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:54:21 smithi167 ceph-mon[37140]: pgmap v2730: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:54:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:54:23 smithi138 ceph-mon[36954]: pgmap v2731: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:54:23.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:54:23 smithi100 ceph-mon[32599]: pgmap v2731: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:54:23.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:54:23 smithi167 ceph-mon[37140]: pgmap v2731: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:54:25.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:54:25 smithi167 ceph-mon[37140]: pgmap v2732: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:54:25.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:54:25 smithi138 ceph-mon[36954]: pgmap v2732: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:54:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:54:25 smithi100 ceph-mon[32599]: pgmap v2732: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:54:27.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:54:27 smithi138 ceph-mon[36954]: pgmap v2733: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:54:27.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:54:27 smithi100 ceph-mon[32599]: pgmap v2733: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:54:27.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:54:27 smithi167 ceph-mon[37140]: pgmap v2733: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:54:29.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:54:29 smithi138 ceph-mon[36954]: pgmap v2734: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:54:29.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:54:29 smithi100 ceph-mon[32599]: pgmap v2734: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:54:29.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:54:29 smithi167 ceph-mon[37140]: pgmap v2734: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:54:31.296 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:54:31 smithi100 ceph-mon[32599]: pgmap v2735: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:54:31.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:54:31 smithi138 ceph-mon[36954]: pgmap v2735: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:54:31.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:54:31 smithi167 ceph-mon[37140]: pgmap v2735: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:54:31.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:54:31.910+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:54:31.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:54:31.910+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:54:31.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:54:31.910+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:54:31.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:54:31.910+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:54:31.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:54:31.910+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:54:33.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:54:33 smithi138 ceph-mon[36954]: pgmap v2736: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:54:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:54:33 smithi100 ceph-mon[32599]: pgmap v2736: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:54:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:54:33 smithi167 ceph-mon[37140]: pgmap v2736: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:54:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:54:35 smithi138 ceph-mon[36954]: pgmap v2737: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:54:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:54:35 smithi100 ceph-mon[32599]: pgmap v2737: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:54:35.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:54:35 smithi167 ceph-mon[37140]: pgmap v2737: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:54:37.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:54:37 smithi138 ceph-mon[36954]: pgmap v2738: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:54:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:54:37 smithi100 ceph-mon[32599]: pgmap v2738: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:54:37.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:54:37 smithi167 ceph-mon[37140]: pgmap v2738: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:54:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:54:39 smithi138 ceph-mon[36954]: pgmap v2739: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:54:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:54:39 smithi100 ceph-mon[32599]: pgmap v2739: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:54:39.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:54:39 smithi167 ceph-mon[37140]: pgmap v2739: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:54:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:54:41 smithi138 ceph-mon[36954]: pgmap v2740: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:54:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:54:41 smithi100 ceph-mon[32599]: pgmap v2740: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:54:41.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:54:41 smithi167 ceph-mon[37140]: pgmap v2740: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:54:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:54:43 smithi138 ceph-mon[36954]: pgmap v2741: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:54:43.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:54:43 smithi100 ceph-mon[32599]: pgmap v2741: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:54:43.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:54:43 smithi167 ceph-mon[37140]: pgmap v2741: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:54:45.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:54:45 smithi138 ceph-mon[36954]: pgmap v2742: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:54:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:54:45 smithi100 ceph-mon[32599]: pgmap v2742: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:54:45.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:54:45 smithi167 ceph-mon[37140]: pgmap v2742: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:54:46.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:54:46.926+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:54:46.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:54:46.928+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:54:46.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:54:46.928+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:54:46.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:54:46.928+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:54:46.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:54:46.928+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:54:47.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:54:47 smithi100 ceph-mon[32599]: pgmap v2743: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:54:47.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:54:47 smithi138 ceph-mon[36954]: pgmap v2743: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:54:47.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:54:47 smithi167 ceph-mon[37140]: pgmap v2743: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:54:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:54:49 smithi138 ceph-mon[36954]: pgmap v2744: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:54:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:54:49 smithi100 ceph-mon[32599]: pgmap v2744: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:54:49.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:54:49 smithi167 ceph-mon[37140]: pgmap v2744: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:54:51.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:54:51 smithi138 ceph-mon[36954]: pgmap v2745: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:54:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:54:51 smithi100 ceph-mon[32599]: pgmap v2745: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:54:51.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:54:51 smithi167 ceph-mon[37140]: pgmap v2745: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:54:53.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:54:53 smithi138 ceph-mon[36954]: pgmap v2746: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:54:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:54:53 smithi100 ceph-mon[32599]: pgmap v2746: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:54:53.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:54:53 smithi167 ceph-mon[37140]: pgmap v2746: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:54:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:54:55 smithi138 ceph-mon[36954]: pgmap v2747: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:54:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:54:55 smithi100 ceph-mon[32599]: pgmap v2747: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:54:55.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:54:55 smithi167 ceph-mon[37140]: pgmap v2747: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:54:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:54:57 smithi138 ceph-mon[36954]: pgmap v2748: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:54:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:54:57 smithi100 ceph-mon[32599]: pgmap v2748: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:54:57.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:54:57 smithi167 ceph-mon[37140]: pgmap v2748: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:54:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:54:59 smithi138 ceph-mon[36954]: pgmap v2749: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:54:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:54:59 smithi100 ceph-mon[32599]: pgmap v2749: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:54:59.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:54:59 smithi167 ceph-mon[37140]: pgmap v2749: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:55:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:55:01 smithi138 ceph-mon[36954]: pgmap v2750: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:55:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:55:01 smithi100 ceph-mon[32599]: pgmap v2750: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:55:01.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:55:01 smithi167 ceph-mon[37140]: pgmap v2750: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:55:01.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:55:01.943+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:55:01.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:55:01.943+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:55:01.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:55:01.943+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:55:01.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:55:01.943+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:55:01.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:55:01.943+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:55:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:55:03 smithi138 ceph-mon[36954]: pgmap v2751: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:55:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:55:03 smithi100 ceph-mon[32599]: pgmap v2751: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:55:03.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:55:03 smithi167 ceph-mon[37140]: pgmap v2751: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:55:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:55:05 smithi138 ceph-mon[36954]: pgmap v2752: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:55:05.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:55:05 smithi100 ceph-mon[32599]: pgmap v2752: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:55:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:55:05 smithi167 ceph-mon[37140]: pgmap v2752: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:55:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:55:07 smithi138 ceph-mon[36954]: pgmap v2753: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:55:07.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:55:07 smithi100 ceph-mon[32599]: pgmap v2753: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:55:07.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:55:07 smithi167 ceph-mon[37140]: pgmap v2753: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:55:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:55:08 smithi138 ceph-mon[36954]: osdmap e222: 12 total, 12 up, 12 in 2024-07-26T14:55:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:55:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:55:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:55:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:55:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:55:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:55:08.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:55:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:55:08.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:55:08 smithi100 ceph-mon[32599]: osdmap e222: 12 total, 12 up, 12 in 2024-07-26T14:55:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:55:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:55:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:55:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:55:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:55:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:55:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:55:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:55:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:55:08 smithi167 ceph-mon[37140]: osdmap e222: 12 total, 12 up, 12 in 2024-07-26T14:55:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:55:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:55:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:55:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:55:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:55:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:55:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:55:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:55:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:55:09 smithi138 ceph-mon[36954]: pgmap v2755: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:55:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:55:09 smithi100 ceph-mon[32599]: pgmap v2755: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:55:09.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:55:09 smithi167 ceph-mon[37140]: pgmap v2755: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:55:11.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:55:11 smithi138 ceph-mon[36954]: pgmap v2756: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:55:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:55:11 smithi100 ceph-mon[32599]: pgmap v2756: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:55:11.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:55:11 smithi167 ceph-mon[37140]: pgmap v2756: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:55:13.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:55:13 smithi138 ceph-mon[36954]: pgmap v2757: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:55:13.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:55:13 smithi100 ceph-mon[32599]: pgmap v2757: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:55:13.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:55:13 smithi167 ceph-mon[37140]: pgmap v2757: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:55:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:55:15 smithi138 ceph-mon[36954]: pgmap v2758: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:55:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:55:15 smithi138 ceph-mon[36954]: osdmap e223: 12 total, 12 up, 12 in 2024-07-26T14:55:15.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:55:15 smithi100 ceph-mon[32599]: pgmap v2758: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:55:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:55:15 smithi100 ceph-mon[32599]: osdmap e223: 12 total, 12 up, 12 in 2024-07-26T14:55:15.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:55:15 smithi167 ceph-mon[37140]: pgmap v2758: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:55:15.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:55:15 smithi167 ceph-mon[37140]: osdmap e223: 12 total, 12 up, 12 in 2024-07-26T14:55:16.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:55:16.960+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:55:16.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:55:16.960+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:55:16.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:55:16.960+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:55:16.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:55:16.960+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:55:16.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:55:16.960+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:55:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:55:17 smithi100 ceph-mon[32599]: pgmap v2760: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 233 B/s wr, 6 op/s 2024-07-26T14:55:17.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:55:17 smithi138 ceph-mon[36954]: pgmap v2760: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 233 B/s wr, 6 op/s 2024-07-26T14:55:17.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:55:17 smithi167 ceph-mon[37140]: pgmap v2760: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 233 B/s wr, 6 op/s 2024-07-26T14:55:19.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:55:19 smithi138 ceph-mon[36954]: pgmap v2761: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:55:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:55:19 smithi100 ceph-mon[32599]: pgmap v2761: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:55:19.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:55:19 smithi167 ceph-mon[37140]: pgmap v2761: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:55:21.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:55:21 smithi138 ceph-mon[36954]: pgmap v2762: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:55:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:55:21 smithi100 ceph-mon[32599]: pgmap v2762: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:55:21.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:55:21 smithi167 ceph-mon[37140]: pgmap v2762: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:55:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:55:23 smithi138 ceph-mon[36954]: pgmap v2763: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:55:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:55:23 smithi100 ceph-mon[32599]: pgmap v2763: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:55:23.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:55:23 smithi167 ceph-mon[37140]: pgmap v2763: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:55:25.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:55:25 smithi138 ceph-mon[36954]: pgmap v2764: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:55:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:55:25 smithi100 ceph-mon[32599]: pgmap v2764: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:55:25.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:55:25 smithi167 ceph-mon[37140]: pgmap v2764: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:55:27.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:55:27 smithi138 ceph-mon[36954]: pgmap v2765: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:55:27.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:55:27 smithi100 ceph-mon[32599]: pgmap v2765: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:55:27.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:55:27 smithi167 ceph-mon[37140]: pgmap v2765: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:55:29.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:55:29 smithi138 ceph-mon[36954]: pgmap v2766: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:55:29.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:55:29 smithi100 ceph-mon[32599]: pgmap v2766: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:55:29.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:55:29 smithi167 ceph-mon[37140]: pgmap v2766: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:55:31.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:55:31 smithi138 ceph-mon[36954]: pgmap v2767: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:55:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:55:31 smithi100 ceph-mon[32599]: pgmap v2767: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:55:31.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:55:31 smithi167 ceph-mon[37140]: pgmap v2767: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:55:31.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:55:31.975+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:55:31.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:55:31.976+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:55:31.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:55:31.976+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:55:31.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:55:31.976+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:55:31.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:55:31.976+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:55:33.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:55:33 smithi138 ceph-mon[36954]: pgmap v2768: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:55:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:55:33 smithi100 ceph-mon[32599]: pgmap v2768: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:55:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:55:33 smithi167 ceph-mon[37140]: pgmap v2768: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:55:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:55:35 smithi138 ceph-mon[36954]: pgmap v2769: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:55:35.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:55:35 smithi100 ceph-mon[32599]: pgmap v2769: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:55:35.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:55:35 smithi167 ceph-mon[37140]: pgmap v2769: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:55:37.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:55:37 smithi138 ceph-mon[36954]: pgmap v2770: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:55:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:55:37 smithi100 ceph-mon[32599]: pgmap v2770: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:55:37.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:55:37 smithi167 ceph-mon[37140]: pgmap v2770: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:55:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:55:39 smithi138 ceph-mon[36954]: pgmap v2771: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:55:39.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:55:39 smithi100 ceph-mon[32599]: pgmap v2771: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:55:39.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:55:39 smithi167 ceph-mon[37140]: pgmap v2771: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:55:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:55:41 smithi138 ceph-mon[36954]: pgmap v2772: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:55:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:55:41 smithi100 ceph-mon[32599]: pgmap v2772: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:55:41.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:55:41 smithi167 ceph-mon[37140]: pgmap v2772: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:55:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:55:43 smithi138 ceph-mon[36954]: pgmap v2773: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:55:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:55:43 smithi100 ceph-mon[32599]: pgmap v2773: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:55:43.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:55:43 smithi167 ceph-mon[37140]: pgmap v2773: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:55:45.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:55:45 smithi138 ceph-mon[36954]: pgmap v2774: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:55:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:55:45 smithi100 ceph-mon[32599]: pgmap v2774: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:55:45.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:55:45 smithi167 ceph-mon[37140]: pgmap v2774: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:55:46.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:55:46.991+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:55:46.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:55:46.992+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:55:46.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:55:46.992+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:55:46.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:55:46.992+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:55:46.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:55:46.992+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:55:47.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:55:47 smithi100 ceph-mon[32599]: pgmap v2775: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:55:47.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:55:47 smithi138 ceph-mon[36954]: pgmap v2775: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:55:47.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:55:47 smithi167 ceph-mon[37140]: pgmap v2775: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:55:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:55:49 smithi138 ceph-mon[36954]: pgmap v2776: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:55:49.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:55:49 smithi100 ceph-mon[32599]: pgmap v2776: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:55:49.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:55:49 smithi167 ceph-mon[37140]: pgmap v2776: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:55:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:55:51 smithi100 ceph-mon[32599]: pgmap v2777: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:55:51.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:55:51 smithi167 ceph-mon[37140]: pgmap v2777: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:55:51.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:55:51 smithi138 ceph-mon[36954]: pgmap v2777: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:55:53.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:55:53 smithi100 ceph-mon[32599]: pgmap v2778: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:55:53.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:55:53 smithi167 ceph-mon[37140]: pgmap v2778: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:55:53.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:55:53 smithi138 ceph-mon[36954]: pgmap v2778: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:55:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:55:55 smithi100 ceph-mon[32599]: pgmap v2779: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:55:55.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:55:55 smithi167 ceph-mon[37140]: pgmap v2779: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:55:55.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:55:55 smithi138 ceph-mon[36954]: pgmap v2779: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:55:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:55:57 smithi100 ceph-mon[32599]: pgmap v2780: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:55:57.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:55:57 smithi167 ceph-mon[37140]: pgmap v2780: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:55:57.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:55:57 smithi138 ceph-mon[36954]: pgmap v2780: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:55:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:55:59 smithi100 ceph-mon[32599]: pgmap v2781: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:55:59.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:55:59 smithi167 ceph-mon[37140]: pgmap v2781: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:55:59.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:55:59 smithi138 ceph-mon[36954]: pgmap v2781: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:56:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:56:01 smithi100 ceph-mon[32599]: pgmap v2782: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:56:01.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:56:01 smithi167 ceph-mon[37140]: pgmap v2782: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:56:01.658 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:56:01 smithi138 ceph-mon[36954]: pgmap v2782: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:56:02.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:56:02.007+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:56:02.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:56:02.008+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:56:02.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:56:02.008+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:56:02.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:56:02.008+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:56:02.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:56:02.008+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:56:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:56:03 smithi100 ceph-mon[32599]: pgmap v2783: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:56:03.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:56:03 smithi167 ceph-mon[37140]: pgmap v2783: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:56:03.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:56:03 smithi138 ceph-mon[36954]: pgmap v2783: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:56:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:56:05 smithi100 ceph-mon[32599]: pgmap v2784: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:56:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:56:05 smithi167 ceph-mon[37140]: pgmap v2784: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:56:05.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:56:05 smithi138 ceph-mon[36954]: pgmap v2784: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:56:07.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:56:07 smithi100 ceph-mon[32599]: pgmap v2785: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:56:07.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:56:07 smithi167 ceph-mon[37140]: pgmap v2785: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:56:07.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:56:07 smithi138 ceph-mon[36954]: pgmap v2785: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:56:08.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:56:08 smithi100 ceph-mon[32599]: osdmap e224: 12 total, 12 up, 12 in 2024-07-26T14:56:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:56:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:56:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:56:08 smithi167 ceph-mon[37140]: osdmap e224: 12 total, 12 up, 12 in 2024-07-26T14:56:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:56:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:56:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:56:08 smithi138 ceph-mon[36954]: osdmap e224: 12 total, 12 up, 12 in 2024-07-26T14:56:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:56:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:56:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:56:09 smithi100 ceph-mon[32599]: pgmap v2787: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:56:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:56:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:56:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:56:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:56:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:56:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:56:09.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:56:09 smithi167 ceph-mon[37140]: pgmap v2787: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:56:09.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:56:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:56:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:56:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:56:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:56:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:56:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:56:09 smithi138 ceph-mon[36954]: pgmap v2787: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:56:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:56:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:56:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:56:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:56:09.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:56:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:56:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:56:11 smithi100 ceph-mon[32599]: pgmap v2788: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:56:11.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:56:11 smithi167 ceph-mon[37140]: pgmap v2788: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:56:11.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:56:11 smithi138 ceph-mon[36954]: pgmap v2788: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:56:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:56:13 smithi100 ceph-mon[32599]: pgmap v2789: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:56:13.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:56:13 smithi167 ceph-mon[37140]: pgmap v2789: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:56:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:56:13 smithi138 ceph-mon[36954]: pgmap v2789: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:56:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:56:15 smithi100 ceph-mon[32599]: pgmap v2790: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:56:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:56:15 smithi100 ceph-mon[32599]: osdmap e225: 12 total, 12 up, 12 in 2024-07-26T14:56:15.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:56:15 smithi167 ceph-mon[37140]: pgmap v2790: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:56:15.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:56:15 smithi167 ceph-mon[37140]: osdmap e225: 12 total, 12 up, 12 in 2024-07-26T14:56:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:56:15 smithi138 ceph-mon[36954]: pgmap v2790: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:56:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:56:15 smithi138 ceph-mon[36954]: osdmap e225: 12 total, 12 up, 12 in 2024-07-26T14:56:17.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:56:17.022+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:56:17.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:56:17.023+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:56:17.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:56:17.023+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:56:17.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:56:17.023+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:56:17.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:56:17.023+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:56:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:56:17 smithi100 ceph-mon[32599]: pgmap v2792: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 235 B/s wr, 6 op/s 2024-07-26T14:56:17.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:56:17 smithi167 ceph-mon[37140]: pgmap v2792: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 235 B/s wr, 6 op/s 2024-07-26T14:56:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:56:17 smithi138 ceph-mon[36954]: pgmap v2792: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 235 B/s wr, 6 op/s 2024-07-26T14:56:19.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:56:19 smithi167 ceph-mon[37140]: pgmap v2793: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:56:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:56:19 smithi138 ceph-mon[36954]: pgmap v2793: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:56:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:56:19 smithi100 ceph-mon[32599]: pgmap v2793: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:56:21.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:56:21 smithi167 ceph-mon[37140]: pgmap v2794: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:56:21.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:56:21 smithi138 ceph-mon[36954]: pgmap v2794: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:56:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:56:21 smithi100 ceph-mon[32599]: pgmap v2794: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:56:23.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:56:23 smithi167 ceph-mon[37140]: pgmap v2795: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:56:23.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:56:23 smithi138 ceph-mon[36954]: pgmap v2795: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:56:23.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:56:23 smithi100 ceph-mon[32599]: pgmap v2795: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:56:25.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:56:25 smithi167 ceph-mon[37140]: pgmap v2796: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:56:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:56:25 smithi138 ceph-mon[36954]: pgmap v2796: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:56:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:56:25 smithi100 ceph-mon[32599]: pgmap v2796: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:56:27.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:56:27 smithi167 ceph-mon[37140]: pgmap v2797: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T14:56:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:56:27 smithi138 ceph-mon[36954]: pgmap v2797: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T14:56:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:56:27 smithi100 ceph-mon[32599]: pgmap v2797: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T14:56:29.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:56:29 smithi167 ceph-mon[37140]: pgmap v2798: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:56:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:56:29 smithi138 ceph-mon[36954]: pgmap v2798: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:56:29.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:56:29 smithi100 ceph-mon[32599]: pgmap v2798: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:56:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:56:30 smithi138 ceph-mon[36954]: pgmap v2799: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:56:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:56:30 smithi100 ceph-mon[32599]: pgmap v2799: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:56:31.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:56:30 smithi167 ceph-mon[37140]: pgmap v2799: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:56:32.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:56:32.031+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:56:32.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:56:32.032+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:56:32.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:56:32.032+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:56:32.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:56:32.032+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:56:32.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:56:32.032+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:56:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:56:32 smithi100 ceph-mon[32599]: pgmap v2800: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:56:33.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:56:32 smithi167 ceph-mon[37140]: pgmap v2800: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:56:33.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:56:32 smithi138 ceph-mon[36954]: pgmap v2800: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:56:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:56:34 smithi100 ceph-mon[32599]: pgmap v2801: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:56:35.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:56:34 smithi167 ceph-mon[37140]: pgmap v2801: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:56:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:56:34 smithi138 ceph-mon[36954]: pgmap v2801: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:56:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:56:36 smithi100 ceph-mon[32599]: pgmap v2802: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:56:37.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:56:36 smithi167 ceph-mon[37140]: pgmap v2802: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:56:37.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:56:36 smithi138 ceph-mon[36954]: pgmap v2802: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:56:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:56:38 smithi100 ceph-mon[32599]: pgmap v2803: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:56:39.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:56:38 smithi167 ceph-mon[37140]: pgmap v2803: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:56:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:56:38 smithi138 ceph-mon[36954]: pgmap v2803: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:56:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:56:40 smithi100 ceph-mon[32599]: pgmap v2804: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:56:41.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:56:40 smithi167 ceph-mon[37140]: pgmap v2804: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:56:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:56:40 smithi138 ceph-mon[36954]: pgmap v2804: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:56:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:56:42 smithi100 ceph-mon[32599]: pgmap v2805: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:56:43.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:56:42 smithi167 ceph-mon[37140]: pgmap v2805: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:56:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:56:42 smithi138 ceph-mon[36954]: pgmap v2805: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:56:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:56:44 smithi100 ceph-mon[32599]: pgmap v2806: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:56:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:56:44 smithi167 ceph-mon[37140]: pgmap v2806: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:56:45.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:56:44 smithi138 ceph-mon[36954]: pgmap v2806: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:56:47.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:56:47.048+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:56:47.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:56:47.048+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:56:47.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:56:47.048+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:56:47.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:56:47.048+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:56:47.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:56:47.048+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:56:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:56:46 smithi100 ceph-mon[32599]: pgmap v2807: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:56:47.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:56:46 smithi167 ceph-mon[37140]: pgmap v2807: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:56:47.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:56:46 smithi138 ceph-mon[36954]: pgmap v2807: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:56:49.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:56:48 smithi167 ceph-mon[37140]: pgmap v2808: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:56:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:56:48 smithi138 ceph-mon[36954]: pgmap v2808: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:56:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:56:48 smithi100 ceph-mon[32599]: pgmap v2808: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:56:51.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:56:51 smithi167 ceph-mon[37140]: pgmap v2809: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:56:51.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:56:51 smithi138 ceph-mon[36954]: pgmap v2809: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:56:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:56:51 smithi100 ceph-mon[32599]: pgmap v2809: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:56:53.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:56:53 smithi167 ceph-mon[37140]: pgmap v2810: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:56:53.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:56:53 smithi138 ceph-mon[36954]: pgmap v2810: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:56:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:56:53 smithi100 ceph-mon[32599]: pgmap v2810: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:56:55.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:56:55 smithi167 ceph-mon[37140]: pgmap v2811: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:56:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:56:55 smithi138 ceph-mon[36954]: pgmap v2811: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:56:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:56:55 smithi100 ceph-mon[32599]: pgmap v2811: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:56:57.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:56:57 smithi167 ceph-mon[37140]: pgmap v2812: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:56:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:56:57 smithi138 ceph-mon[36954]: pgmap v2812: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:56:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:56:57 smithi100 ceph-mon[32599]: pgmap v2812: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:56:59.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:56:59 smithi167 ceph-mon[37140]: pgmap v2813: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:56:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:56:59 smithi138 ceph-mon[36954]: pgmap v2813: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:56:59.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:56:59 smithi100 ceph-mon[32599]: pgmap v2813: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:57:01.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:01 smithi167 ceph-mon[37140]: pgmap v2814: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:57:01.296 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:01 smithi100 ceph-mon[32599]: pgmap v2814: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:57:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:01 smithi138 ceph-mon[36954]: pgmap v2814: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:57:02.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:57:02.054+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:57:02.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:57:02.055+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:57:02.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:57:02.055+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:57:02.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:57:02.055+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:57:02.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:57:02.055+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:57:03.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:03 smithi167 ceph-mon[37140]: pgmap v2815: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:57:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:03 smithi138 ceph-mon[36954]: pgmap v2815: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:57:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:03 smithi100 ceph-mon[32599]: pgmap v2815: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:57:03.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:57:03.598+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T14:57:03.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:57:03.598+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c003690 con 0x7fe5a81087a0 2024-07-26T14:57:03.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:57:03.599+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 36 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T14:57:03.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:57:03.599+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 36 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c003690 con 0x7fe5a81087a0 2024-07-26T14:57:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:05 smithi138 ceph-mon[36954]: pgmap v2816: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T14:57:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:05 smithi100 ceph-mon[32599]: pgmap v2816: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T14:57:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:05 smithi167 ceph-mon[37140]: pgmap v2816: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T14:57:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:07 smithi138 ceph-mon[36954]: pgmap v2817: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:57:07.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:07 smithi100 ceph-mon[32599]: pgmap v2817: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:57:07.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:07 smithi167 ceph-mon[37140]: pgmap v2817: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:57:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:08 smithi138 ceph-mon[36954]: osdmap e226: 12 total, 12 up, 12 in 2024-07-26T14:57:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:08 smithi100 ceph-mon[32599]: osdmap e226: 12 total, 12 up, 12 in 2024-07-26T14:57:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:08 smithi167 ceph-mon[37140]: osdmap e226: 12 total, 12 up, 12 in 2024-07-26T14:57:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:09 smithi138 ceph-mon[36954]: pgmap v2819: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:57:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:57:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:57:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:57:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:57:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:57:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:57:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:57:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:57:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:57:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:57:09.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:57:09.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:57:09.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:57:09.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:57:09.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:57:09.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:57:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:09 smithi100 ceph-mon[32599]: pgmap v2819: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:57:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:57:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:57:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:57:09.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:57:09.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:57:09.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:57:09.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:57:09.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:57:09.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:57:09.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:57:09.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:57:09.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:57:09.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:57:09.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:57:09.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:57:09.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:57:09.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:09 smithi167 ceph-mon[37140]: pgmap v2819: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:57:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:57:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:57:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:57:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:57:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:57:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:57:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:57:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:57:09.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:57:09.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:57:09.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:57:09.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:57:09.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T14:57:09.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:57:09.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:57:09.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:57:10.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:10 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T14:57:10.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:10 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T14:57:10.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:10 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T14:57:10.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:10 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T14:57:10.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:10 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T14:57:10.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:10 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T14:57:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:10 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T14:57:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:10 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T14:57:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:10 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T14:57:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:10 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T14:57:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:10 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T14:57:10.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:10 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T14:57:10.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:10 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T14:57:10.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:10 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T14:57:10.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:10 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T14:57:10.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:10 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T14:57:10.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:10 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T14:57:10.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:10 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T14:57:11.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:11 smithi138 ceph-mon[36954]: pgmap v2820: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T14:57:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:11 smithi100 ceph-mon[32599]: pgmap v2820: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T14:57:11.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:11 smithi167 ceph-mon[37140]: pgmap v2820: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T14:57:13.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:13 smithi138 ceph-mon[36954]: pgmap v2821: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T14:57:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:13 smithi100 ceph-mon[32599]: pgmap v2821: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T14:57:13.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:13 smithi167 ceph-mon[37140]: pgmap v2821: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T14:57:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:15 smithi138 ceph-mon[36954]: pgmap v2822: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:57:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:15 smithi138 ceph-mon[36954]: osdmap e227: 12 total, 12 up, 12 in 2024-07-26T14:57:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:15 smithi100 ceph-mon[32599]: pgmap v2822: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:57:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:15 smithi100 ceph-mon[32599]: osdmap e227: 12 total, 12 up, 12 in 2024-07-26T14:57:15.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:15 smithi167 ceph-mon[37140]: pgmap v2822: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:57:15.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:15 smithi167 ceph-mon[37140]: osdmap e227: 12 total, 12 up, 12 in 2024-07-26T14:57:17.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:57:17.071+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:57:17.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:57:17.071+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:57:17.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:57:17.071+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:57:17.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:57:17.071+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:57:17.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:57:17.071+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:57:17.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:17 smithi138 ceph-mon[36954]: pgmap v2824: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T14:57:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:17 smithi100 ceph-mon[32599]: pgmap v2824: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T14:57:17.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:17 smithi167 ceph-mon[37140]: pgmap v2824: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T14:57:19.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:19 smithi138 ceph-mon[36954]: pgmap v2825: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:57:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:19 smithi100 ceph-mon[32599]: pgmap v2825: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:57:19.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:19 smithi167 ceph-mon[37140]: pgmap v2825: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:57:21.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:21 smithi138 ceph-mon[36954]: pgmap v2826: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:57:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:21 smithi100 ceph-mon[32599]: pgmap v2826: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:57:21.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:21 smithi167 ceph-mon[37140]: pgmap v2826: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:57:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:23 smithi138 ceph-mon[36954]: pgmap v2827: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:57:23.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:23 smithi100 ceph-mon[32599]: pgmap v2827: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:57:23.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:23 smithi167 ceph-mon[37140]: pgmap v2827: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:57:25.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:25 smithi167 ceph-mon[37140]: pgmap v2828: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:57:25.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:25 smithi138 ceph-mon[36954]: pgmap v2828: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:57:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:25 smithi100 ceph-mon[32599]: pgmap v2828: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:57:27.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:27 smithi138 ceph-mon[36954]: pgmap v2829: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:57:27.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:27 smithi100 ceph-mon[32599]: pgmap v2829: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:57:27.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:27 smithi167 ceph-mon[37140]: pgmap v2829: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:57:29.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:29 smithi138 ceph-mon[36954]: pgmap v2830: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:57:29.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:29 smithi100 ceph-mon[32599]: pgmap v2830: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:57:29.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:29 smithi167 ceph-mon[37140]: pgmap v2830: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:57:31.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:31 smithi138 ceph-mon[36954]: pgmap v2831: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:57:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:31 smithi100 ceph-mon[32599]: pgmap v2831: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:57:31.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:31 smithi167 ceph-mon[37140]: pgmap v2831: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:57:32.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:57:32.085+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:57:32.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:57:32.085+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:57:32.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:57:32.085+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:57:32.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:57:32.085+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:57:32.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:57:32.085+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:57:33.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:33 smithi138 ceph-mon[36954]: pgmap v2832: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:57:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:33 smithi100 ceph-mon[32599]: pgmap v2832: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:57:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:33 smithi167 ceph-mon[37140]: pgmap v2832: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:57:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:35 smithi138 ceph-mon[36954]: pgmap v2833: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:57:35.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:35 smithi100 ceph-mon[32599]: pgmap v2833: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:57:35.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:35 smithi167 ceph-mon[37140]: pgmap v2833: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:57:37.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:37 smithi138 ceph-mon[36954]: pgmap v2834: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:57:37.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:37 smithi100 ceph-mon[32599]: pgmap v2834: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:57:37.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:37 smithi167 ceph-mon[37140]: pgmap v2834: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T14:57:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:39 smithi138 ceph-mon[36954]: pgmap v2835: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:57:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:39 smithi100 ceph-mon[32599]: pgmap v2835: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:57:39.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:39 smithi167 ceph-mon[37140]: pgmap v2835: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:57:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:41 smithi138 ceph-mon[36954]: pgmap v2836: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:57:41.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:41 smithi100 ceph-mon[32599]: pgmap v2836: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:57:41.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:41 smithi167 ceph-mon[37140]: pgmap v2836: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T14:57:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:43 smithi138 ceph-mon[36954]: pgmap v2837: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:57:43.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:43 smithi100 ceph-mon[32599]: pgmap v2837: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:57:43.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:43 smithi167 ceph-mon[37140]: pgmap v2837: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T14:57:45.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:45 smithi138 ceph-mon[36954]: pgmap v2838: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:57:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:45 smithi100 ceph-mon[32599]: pgmap v2838: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:57:45.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:45 smithi167 ceph-mon[37140]: pgmap v2838: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:57:47.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:57:47.099+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:57:47.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:57:47.099+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:57:47.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:57:47.099+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:57:47.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:57:47.100+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:57:47.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:57:47.100+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:57:47.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:47 smithi138 ceph-mon[36954]: pgmap v2839: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:57:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:47 smithi100 ceph-mon[32599]: pgmap v2839: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:57:47.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:47 smithi167 ceph-mon[37140]: pgmap v2839: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:57:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:49 smithi138 ceph-mon[36954]: pgmap v2840: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:57:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:49 smithi100 ceph-mon[32599]: pgmap v2840: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:57:49.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:49 smithi167 ceph-mon[37140]: pgmap v2840: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:57:51.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:51 smithi138 ceph-mon[36954]: pgmap v2841: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:57:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:51 smithi100 ceph-mon[32599]: pgmap v2841: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:57:51.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:51 smithi167 ceph-mon[37140]: pgmap v2841: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:57:53.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:53 smithi138 ceph-mon[36954]: pgmap v2842: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:57:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:53 smithi100 ceph-mon[32599]: pgmap v2842: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:57:53.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:53 smithi167 ceph-mon[37140]: pgmap v2842: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:57:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:55 smithi138 ceph-mon[36954]: pgmap v2843: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:57:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:55 smithi100 ceph-mon[32599]: pgmap v2843: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:57:55.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:55 smithi167 ceph-mon[37140]: pgmap v2843: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:57:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:57 smithi138 ceph-mon[36954]: pgmap v2844: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:57:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:57 smithi100 ceph-mon[32599]: pgmap v2844: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:57:57.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:57 smithi167 ceph-mon[37140]: pgmap v2844: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:57:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:57:59 smithi138 ceph-mon[36954]: pgmap v2845: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:57:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:57:59 smithi100 ceph-mon[32599]: pgmap v2845: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:57:59.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:57:59 smithi167 ceph-mon[37140]: pgmap v2845: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:58:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:58:01 smithi138 ceph-mon[36954]: pgmap v2846: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:58:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:58:01 smithi100 ceph-mon[32599]: pgmap v2846: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:58:01.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:58:01 smithi167 ceph-mon[37140]: pgmap v2846: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:58:02.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:58:02.115+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:58:02.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:58:02.116+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:58:02.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:58:02.116+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:58:02.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:58:02.116+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:58:02.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:58:02.116+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:58:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:58:03 smithi138 ceph-mon[36954]: pgmap v2847: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:58:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:58:03 smithi100 ceph-mon[32599]: pgmap v2847: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:58:03.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:58:03 smithi167 ceph-mon[37140]: pgmap v2847: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:58:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:58:05 smithi138 ceph-mon[36954]: pgmap v2848: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:58:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:58:05 smithi100 ceph-mon[32599]: pgmap v2848: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:58:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:58:05 smithi167 ceph-mon[37140]: pgmap v2848: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:58:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:58:07 smithi138 ceph-mon[36954]: pgmap v2849: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:58:07.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:58:07 smithi100 ceph-mon[32599]: pgmap v2849: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:58:07.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:58:07 smithi167 ceph-mon[37140]: pgmap v2849: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:58:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:58:08 smithi138 ceph-mon[36954]: osdmap e228: 12 total, 12 up, 12 in 2024-07-26T14:58:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:58:08 smithi100 ceph-mon[32599]: osdmap e228: 12 total, 12 up, 12 in 2024-07-26T14:58:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:58:08 smithi167 ceph-mon[37140]: osdmap e228: 12 total, 12 up, 12 in 2024-07-26T14:58:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:58:09 smithi138 ceph-mon[36954]: pgmap v2851: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:58:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:58:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:58:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:58:09 smithi100 ceph-mon[32599]: pgmap v2851: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:58:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:58:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:58:09.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:58:09 smithi167 ceph-mon[37140]: pgmap v2851: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:58:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:58:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:58:10.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:58:10 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:58:10.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:58:10 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:58:10.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:58:10 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:58:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:58:10 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:58:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:58:10 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:58:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:58:10 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:58:10.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:58:10 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:58:10.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:58:10 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:58:10.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:58:10 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:58:11.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:58:11 smithi138 ceph-mon[36954]: pgmap v2852: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:58:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:58:11 smithi100 ceph-mon[32599]: pgmap v2852: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:58:11.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:58:11 smithi167 ceph-mon[37140]: pgmap v2852: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:58:13.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:58:13 smithi138 ceph-mon[36954]: pgmap v2853: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:58:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:58:13 smithi100 ceph-mon[32599]: pgmap v2853: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:58:13.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:58:13 smithi167 ceph-mon[37140]: pgmap v2853: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:58:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:58:15 smithi138 ceph-mon[36954]: pgmap v2854: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:58:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:58:15 smithi138 ceph-mon[36954]: osdmap e229: 12 total, 12 up, 12 in 2024-07-26T14:58:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:58:15 smithi100 ceph-mon[32599]: pgmap v2854: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:58:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:58:15 smithi100 ceph-mon[32599]: osdmap e229: 12 total, 12 up, 12 in 2024-07-26T14:58:15.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:58:15 smithi167 ceph-mon[37140]: pgmap v2854: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:58:15.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:58:15 smithi167 ceph-mon[37140]: osdmap e229: 12 total, 12 up, 12 in 2024-07-26T14:58:17.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:58:17.127+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:58:17.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:58:17.127+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:58:17.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:58:17.127+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:58:17.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:58:17.127+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:58:17.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:58:17.128+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:58:17.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:58:17 smithi138 ceph-mon[36954]: pgmap v2856: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 233 B/s wr, 6 op/s 2024-07-26T14:58:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:58:17 smithi100 ceph-mon[32599]: pgmap v2856: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 233 B/s wr, 6 op/s 2024-07-26T14:58:17.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:58:17 smithi167 ceph-mon[37140]: pgmap v2856: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 233 B/s wr, 6 op/s 2024-07-26T14:58:19.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:58:19 smithi138 ceph-mon[36954]: pgmap v2857: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:58:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:58:19 smithi100 ceph-mon[32599]: pgmap v2857: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:58:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:58:19 smithi167 ceph-mon[37140]: pgmap v2857: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:58:21.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:58:21 smithi138 ceph-mon[36954]: pgmap v2858: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:58:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:58:21 smithi100 ceph-mon[32599]: pgmap v2858: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:58:21.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:58:21 smithi167 ceph-mon[37140]: pgmap v2858: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:58:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:58:23 smithi100 ceph-mon[32599]: pgmap v2859: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:58:23.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:58:23 smithi167 ceph-mon[37140]: pgmap v2859: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:58:23.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:58:23 smithi138 ceph-mon[36954]: pgmap v2859: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:58:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:58:25 smithi100 ceph-mon[32599]: pgmap v2860: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:58:25.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:58:25 smithi167 ceph-mon[37140]: pgmap v2860: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:58:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:58:25 smithi138 ceph-mon[36954]: pgmap v2860: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:58:27.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:58:27 smithi100 ceph-mon[32599]: pgmap v2861: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:58:27.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:58:27 smithi167 ceph-mon[37140]: pgmap v2861: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:58:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:58:27 smithi138 ceph-mon[36954]: pgmap v2861: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:58:29.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:58:29 smithi100 ceph-mon[32599]: pgmap v2862: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:58:29.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:58:29 smithi167 ceph-mon[37140]: pgmap v2862: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:58:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:58:29 smithi138 ceph-mon[36954]: pgmap v2862: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:58:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:58:31 smithi100 ceph-mon[32599]: pgmap v2863: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:58:31.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:58:31 smithi167 ceph-mon[37140]: pgmap v2863: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:58:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:58:31 smithi138 ceph-mon[36954]: pgmap v2863: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:58:32.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:58:32.143+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:58:32.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:58:32.144+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:58:32.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:58:32.144+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:58:32.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:58:32.144+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:58:32.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:58:32.144+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:58:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:58:33 smithi100 ceph-mon[32599]: pgmap v2864: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:58:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:58:33 smithi167 ceph-mon[37140]: pgmap v2864: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:58:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:58:33 smithi138 ceph-mon[36954]: pgmap v2864: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:58:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:58:35 smithi100 ceph-mon[32599]: pgmap v2865: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:58:35.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:58:35 smithi167 ceph-mon[37140]: pgmap v2865: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:58:35.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:58:35 smithi138 ceph-mon[36954]: pgmap v2865: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:58:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:58:37 smithi100 ceph-mon[32599]: pgmap v2866: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:58:37.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:58:37 smithi167 ceph-mon[37140]: pgmap v2866: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:58:37.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:58:37 smithi138 ceph-mon[36954]: pgmap v2866: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:58:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:58:39 smithi100 ceph-mon[32599]: pgmap v2867: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:58:39.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:58:39 smithi167 ceph-mon[37140]: pgmap v2867: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:58:39.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:58:39 smithi138 ceph-mon[36954]: pgmap v2867: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:58:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:58:41 smithi100 ceph-mon[32599]: pgmap v2868: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:58:41.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:58:41 smithi167 ceph-mon[37140]: pgmap v2868: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:58:41.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:58:41 smithi138 ceph-mon[36954]: pgmap v2868: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:58:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:58:43 smithi100 ceph-mon[32599]: pgmap v2869: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:58:43.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:58:43 smithi167 ceph-mon[37140]: pgmap v2869: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:58:43.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:58:43 smithi138 ceph-mon[36954]: pgmap v2869: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:58:45.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:58:45 smithi100 ceph-mon[32599]: pgmap v2870: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:58:45.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:58:45 smithi167 ceph-mon[37140]: pgmap v2870: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:58:45.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:58:45 smithi138 ceph-mon[36954]: pgmap v2870: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:58:47.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:58:47.147+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:58:47.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:58:47.148+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:58:47.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:58:47.148+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:58:47.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:58:47.148+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:58:47.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:58:47.148+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:58:47.492 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:58:47 smithi138 ceph-mon[36954]: pgmap v2871: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:58:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:58:47 smithi100 ceph-mon[32599]: pgmap v2871: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:58:47.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:58:47 smithi167 ceph-mon[37140]: pgmap v2871: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:58:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:58:49 smithi100 ceph-mon[32599]: pgmap v2872: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:58:49.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:58:49 smithi167 ceph-mon[37140]: pgmap v2872: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:58:49.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:58:49 smithi138 ceph-mon[36954]: pgmap v2872: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:58:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:58:51 smithi100 ceph-mon[32599]: pgmap v2873: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:58:51.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:58:51 smithi167 ceph-mon[37140]: pgmap v2873: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:58:51.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:58:51 smithi138 ceph-mon[36954]: pgmap v2873: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:58:53.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:58:53 smithi167 ceph-mon[37140]: pgmap v2874: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:58:53.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:58:53 smithi138 ceph-mon[36954]: pgmap v2874: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:58:53.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:58:53 smithi100 ceph-mon[32599]: pgmap v2874: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:58:55.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:58:55 smithi167 ceph-mon[37140]: pgmap v2875: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:58:55.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:58:55 smithi138 ceph-mon[36954]: pgmap v2875: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:58:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:58:55 smithi100 ceph-mon[32599]: pgmap v2875: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:58:57.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:58:57 smithi167 ceph-mon[37140]: pgmap v2876: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:58:57.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:58:57 smithi138 ceph-mon[36954]: pgmap v2876: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:58:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:58:57 smithi100 ceph-mon[32599]: pgmap v2876: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:58:59.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:58:59 smithi167 ceph-mon[37140]: pgmap v2877: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:58:59.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:58:59 smithi138 ceph-mon[36954]: pgmap v2877: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:58:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:58:59 smithi100 ceph-mon[32599]: pgmap v2877: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:59:01.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:59:01 smithi167 ceph-mon[37140]: pgmap v2878: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:59:01.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:59:01 smithi100 ceph-mon[32599]: pgmap v2878: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:59:01.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:59:01 smithi138 ceph-mon[36954]: pgmap v2878: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:59:02.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:59:02.161+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:59:02.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:59:02.162+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:59:02.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:59:02.162+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:59:02.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:59:02.162+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:59:02.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:59:02.162+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:59:03.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:59:03 smithi167 ceph-mon[37140]: pgmap v2879: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:59:03.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:59:03 smithi138 ceph-mon[36954]: pgmap v2879: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:59:03.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:59:03 smithi100 ceph-mon[32599]: pgmap v2879: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:59:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:59:05 smithi167 ceph-mon[37140]: pgmap v2880: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:59:05.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:59:05 smithi138 ceph-mon[36954]: pgmap v2880: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:59:05.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:59:05 smithi100 ceph-mon[32599]: pgmap v2880: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T14:59:07.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:59:07 smithi138 ceph-mon[36954]: pgmap v2881: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:59:07.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:59:07 smithi100 ceph-mon[32599]: pgmap v2881: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:59:07.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:59:07 smithi167 ceph-mon[37140]: pgmap v2881: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T14:59:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:59:08 smithi138 ceph-mon[36954]: osdmap e230: 12 total, 12 up, 12 in 2024-07-26T14:59:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:59:08 smithi100 ceph-mon[32599]: osdmap e230: 12 total, 12 up, 12 in 2024-07-26T14:59:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:59:08 smithi167 ceph-mon[37140]: osdmap e230: 12 total, 12 up, 12 in 2024-07-26T14:59:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:59:09 smithi138 ceph-mon[36954]: pgmap v2883: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:59:09.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:59:09 smithi100 ceph-mon[32599]: pgmap v2883: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:59:09.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:59:09 smithi167 ceph-mon[37140]: pgmap v2883: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:59:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:59:10 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:59:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:59:10 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:59:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:59:10 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:59:10.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:59:10 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:59:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:59:10 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:59:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:59:10 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:59:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:59:10 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:59:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:59:10 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:59:10.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:59:10 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T14:59:10.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:59:10 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T14:59:10.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:59:10 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T14:59:10.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:59:10 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T14:59:11.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:59:11 smithi138 ceph-mon[36954]: pgmap v2884: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:59:11.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:59:11 smithi100 ceph-mon[32599]: pgmap v2884: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:59:11.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:59:11 smithi167 ceph-mon[37140]: pgmap v2884: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:59:12.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:59:12 smithi138 ceph-mon[36954]: pgmap v2885: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:59:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:59:12 smithi100 ceph-mon[32599]: pgmap v2885: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:59:12.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:59:12 smithi167 ceph-mon[37140]: pgmap v2885: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T14:59:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:59:15 smithi138 ceph-mon[36954]: pgmap v2886: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:59:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:59:15 smithi138 ceph-mon[36954]: osdmap e231: 12 total, 12 up, 12 in 2024-07-26T14:59:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:59:15 smithi100 ceph-mon[32599]: pgmap v2886: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:59:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:59:15 smithi100 ceph-mon[32599]: osdmap e231: 12 total, 12 up, 12 in 2024-07-26T14:59:15.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:59:15 smithi167 ceph-mon[37140]: pgmap v2886: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:59:15.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:59:15 smithi167 ceph-mon[37140]: osdmap e231: 12 total, 12 up, 12 in 2024-07-26T14:59:17.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:59:17.177+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:59:17.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:59:17.178+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:59:17.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:59:17.178+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:59:17.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:59:17.178+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:59:17.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:59:17.178+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:59:17.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:59:17 smithi138 ceph-mon[36954]: pgmap v2888: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 235 B/s wr, 6 op/s 2024-07-26T14:59:17.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:59:17 smithi100 ceph-mon[32599]: pgmap v2888: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 235 B/s wr, 6 op/s 2024-07-26T14:59:17.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:59:17 smithi167 ceph-mon[37140]: pgmap v2888: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 235 B/s wr, 6 op/s 2024-07-26T14:59:19.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:59:19 smithi138 ceph-mon[36954]: pgmap v2889: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:59:19.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:59:19 smithi100 ceph-mon[32599]: pgmap v2889: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:59:19.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:59:19 smithi167 ceph-mon[37140]: pgmap v2889: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T14:59:21.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:59:21 smithi138 ceph-mon[36954]: pgmap v2890: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:59:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:59:21 smithi100 ceph-mon[32599]: pgmap v2890: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:59:21.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:59:21 smithi167 ceph-mon[37140]: pgmap v2890: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:59:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:59:23 smithi138 ceph-mon[36954]: pgmap v2891: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:59:23.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:59:23 smithi100 ceph-mon[32599]: pgmap v2891: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:59:23.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:59:23 smithi167 ceph-mon[37140]: pgmap v2891: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T14:59:25.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:59:25 smithi167 ceph-mon[37140]: pgmap v2892: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:59:25.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:59:25 smithi138 ceph-mon[36954]: pgmap v2892: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:59:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:59:25 smithi100 ceph-mon[32599]: pgmap v2892: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T14:59:27.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:59:27 smithi138 ceph-mon[36954]: pgmap v2893: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:59:27.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:59:27 smithi100 ceph-mon[32599]: pgmap v2893: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:59:27.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:59:27 smithi167 ceph-mon[37140]: pgmap v2893: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T14:59:29.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:59:29 smithi138 ceph-mon[36954]: pgmap v2894: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:59:29.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:59:29 smithi100 ceph-mon[32599]: pgmap v2894: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:59:29.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:59:29 smithi167 ceph-mon[37140]: pgmap v2894: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:59:31.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:59:31 smithi138 ceph-mon[36954]: pgmap v2895: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:59:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:59:31 smithi100 ceph-mon[32599]: pgmap v2895: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:59:31.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:59:31 smithi167 ceph-mon[37140]: pgmap v2895: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:59:32.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:59:32.193+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:59:32.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:59:32.194+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:59:32.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:59:32.194+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:59:32.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:59:32.194+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:59:32.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:59:32.194+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:59:33.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:59:33 smithi138 ceph-mon[36954]: pgmap v2896: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:59:33.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:59:33 smithi100 ceph-mon[32599]: pgmap v2896: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:59:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:59:33 smithi167 ceph-mon[37140]: pgmap v2896: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:59:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:59:35 smithi138 ceph-mon[36954]: pgmap v2897: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:59:35.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:59:35 smithi100 ceph-mon[32599]: pgmap v2897: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:59:35.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:59:35 smithi167 ceph-mon[37140]: pgmap v2897: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:59:37.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:59:37 smithi138 ceph-mon[36954]: pgmap v2898: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:59:37.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:59:37 smithi100 ceph-mon[32599]: pgmap v2898: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:59:37.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:59:37 smithi167 ceph-mon[37140]: pgmap v2898: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:59:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:59:39 smithi138 ceph-mon[36954]: pgmap v2899: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:59:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:59:39 smithi100 ceph-mon[32599]: pgmap v2899: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:59:39.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:59:39 smithi167 ceph-mon[37140]: pgmap v2899: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:59:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:59:41 smithi138 ceph-mon[36954]: pgmap v2900: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:59:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:59:41 smithi100 ceph-mon[32599]: pgmap v2900: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:59:41.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:59:41 smithi167 ceph-mon[37140]: pgmap v2900: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:59:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:59:43 smithi138 ceph-mon[36954]: pgmap v2901: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:59:43.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:59:43 smithi100 ceph-mon[32599]: pgmap v2901: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:59:43.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:59:43 smithi167 ceph-mon[37140]: pgmap v2901: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:59:45.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:59:45 smithi138 ceph-mon[36954]: pgmap v2902: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:59:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:59:45 smithi100 ceph-mon[32599]: pgmap v2902: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:59:45.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:59:45 smithi167 ceph-mon[37140]: pgmap v2902: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:59:47.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:59:47.209+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T14:59:47.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:59:47.210+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T14:59:47.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:59:47.210+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T14:59:47.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:59:47.210+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T14:59:47.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T14:59:47.210+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T14:59:47.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:59:47 smithi138 ceph-mon[36954]: pgmap v2903: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:59:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:59:47 smithi100 ceph-mon[32599]: pgmap v2903: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:59:47.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:59:47 smithi167 ceph-mon[37140]: pgmap v2903: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:59:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:59:49 smithi138 ceph-mon[36954]: pgmap v2904: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:59:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:59:49 smithi100 ceph-mon[32599]: pgmap v2904: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:59:49.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:59:49 smithi167 ceph-mon[37140]: pgmap v2904: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:59:51.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:59:51 smithi138 ceph-mon[36954]: pgmap v2905: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:59:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:59:51 smithi100 ceph-mon[32599]: pgmap v2905: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:59:51.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:59:51 smithi167 ceph-mon[37140]: pgmap v2905: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:59:53.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:59:53 smithi138 ceph-mon[36954]: pgmap v2906: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:59:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:59:53 smithi100 ceph-mon[32599]: pgmap v2906: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:59:53.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:59:53 smithi167 ceph-mon[37140]: pgmap v2906: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:59:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:59:55 smithi138 ceph-mon[36954]: pgmap v2907: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:59:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:59:55 smithi100 ceph-mon[32599]: pgmap v2907: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:59:55.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:59:55 smithi167 ceph-mon[37140]: pgmap v2907: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T14:59:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:59:57 smithi138 ceph-mon[36954]: pgmap v2908: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:59:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:59:57 smithi100 ceph-mon[32599]: pgmap v2908: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:59:57.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:59:57 smithi167 ceph-mon[37140]: pgmap v2908: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T14:59:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 14:59:59 smithi138 ceph-mon[36954]: pgmap v2909: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:59:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 14:59:59 smithi100 ceph-mon[32599]: pgmap v2909: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T14:59:59.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 14:59:59 smithi167 ceph-mon[37140]: pgmap v2909: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:00:00.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:00 smithi138 ceph-mon[36954]: overall HEALTH_OK 2024-07-26T15:00:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:00 smithi100 ceph-mon[32599]: overall HEALTH_OK 2024-07-26T15:00:00.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:00 smithi167 ceph-mon[37140]: overall HEALTH_OK 2024-07-26T15:00:01.395 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:01 smithi100 ceph-mon[32599]: pgmap v2910: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:00:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:01 smithi138 ceph-mon[36954]: pgmap v2910: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:00:01.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:01 smithi167 ceph-mon[37140]: pgmap v2910: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:00:02.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:00:02.226+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:00:02.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:00:02.226+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:00:02.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:00:02.226+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:00:02.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:00:02.226+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:00:02.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:00:02.226+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:00:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:03 smithi138 ceph-mon[36954]: pgmap v2911: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:00:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:03 smithi100 ceph-mon[32599]: pgmap v2911: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:00:03.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:03 smithi167 ceph-mon[37140]: pgmap v2911: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:00:03.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:00:03.625+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T15:00:03.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:00:03.625+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T15:00:03.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:00:03.625+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 37 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T15:00:03.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:00:03.625+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 37 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T15:00:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:05 smithi138 ceph-mon[36954]: pgmap v2912: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:00:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:05 smithi100 ceph-mon[32599]: pgmap v2912: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:00:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:05 smithi167 ceph-mon[37140]: pgmap v2912: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:00:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:07 smithi138 ceph-mon[36954]: pgmap v2913: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:00:07.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:07 smithi100 ceph-mon[32599]: pgmap v2913: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:00:07.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:07 smithi167 ceph-mon[37140]: pgmap v2913: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:00:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:08 smithi138 ceph-mon[36954]: osdmap e232: 12 total, 12 up, 12 in 2024-07-26T15:00:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:08 smithi100 ceph-mon[32599]: osdmap e232: 12 total, 12 up, 12 in 2024-07-26T15:00:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:08 smithi167 ceph-mon[37140]: osdmap e232: 12 total, 12 up, 12 in 2024-07-26T15:00:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:09 smithi138 ceph-mon[36954]: pgmap v2915: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:00:09.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:09 smithi100 ceph-mon[32599]: pgmap v2915: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:00:09.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:09 smithi167 ceph-mon[37140]: pgmap v2915: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:00:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:11 smithi100 ceph-mon[32599]: pgmap v2916: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:00:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:00:11.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:11 smithi138 ceph-mon[36954]: pgmap v2916: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:00:11.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:00:11.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:11 smithi167 ceph-mon[37140]: pgmap v2916: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:00:11.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:00:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:12 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:00:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:12 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:00:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:12 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:00:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:12 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:00:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:12 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:00:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:12 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:00:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:12 smithi138 ceph-mon[36954]: pgmap v2917: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:00:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:12 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:00:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:12 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:00:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:12 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:00:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:12 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:00:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:12 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:00:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:12 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:00:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:12 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:00:12.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:12 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:00:12.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:12 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:00:12.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:12 smithi100 ceph-mon[32599]: pgmap v2917: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:00:12.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:12 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:00:12.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:12 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:00:12.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:12 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:00:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:12 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:00:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:12 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:00:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:12 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:00:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:12 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:00:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:12 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:00:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:12 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:00:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:12 smithi167 ceph-mon[37140]: pgmap v2917: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:00:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:12 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:00:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:12 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:00:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:12 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:00:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:15 smithi138 ceph-mon[36954]: pgmap v2918: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:00:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:15 smithi138 ceph-mon[36954]: osdmap e233: 12 total, 12 up, 12 in 2024-07-26T15:00:15.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:15 smithi100 ceph-mon[32599]: pgmap v2918: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:00:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:15 smithi100 ceph-mon[32599]: osdmap e233: 12 total, 12 up, 12 in 2024-07-26T15:00:15.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:15 smithi167 ceph-mon[37140]: pgmap v2918: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:00:15.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:15 smithi167 ceph-mon[37140]: osdmap e233: 12 total, 12 up, 12 in 2024-07-26T15:00:17.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:00:17.242+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:00:17.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:00:17.242+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:00:17.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:00:17.242+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:00:17.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:00:17.242+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:00:17.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:00:17.242+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:00:17.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:17 smithi138 ceph-mon[36954]: pgmap v2920: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T15:00:17.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:17 smithi100 ceph-mon[32599]: pgmap v2920: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T15:00:17.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:17 smithi167 ceph-mon[37140]: pgmap v2920: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T15:00:19.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:19 smithi138 ceph-mon[36954]: pgmap v2921: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:00:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:19 smithi100 ceph-mon[32599]: pgmap v2921: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:00:19.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:19 smithi167 ceph-mon[37140]: pgmap v2921: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:00:21.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:21 smithi138 ceph-mon[36954]: pgmap v2922: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:00:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:21 smithi100 ceph-mon[32599]: pgmap v2922: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:00:21.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:21 smithi167 ceph-mon[37140]: pgmap v2922: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:00:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:23 smithi138 ceph-mon[36954]: pgmap v2923: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:00:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:23 smithi100 ceph-mon[32599]: pgmap v2923: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:00:23.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:23 smithi167 ceph-mon[37140]: pgmap v2923: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:00:25.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:25 smithi167 ceph-mon[37140]: pgmap v2924: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:00:25.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:25 smithi138 ceph-mon[36954]: pgmap v2924: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:00:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:25 smithi100 ceph-mon[32599]: pgmap v2924: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:00:27.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:27 smithi138 ceph-mon[36954]: pgmap v2925: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:00:27.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:27 smithi100 ceph-mon[32599]: pgmap v2925: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:00:27.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:27 smithi167 ceph-mon[37140]: pgmap v2925: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:00:29.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:29 smithi138 ceph-mon[36954]: pgmap v2926: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:00:29.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:29 smithi100 ceph-mon[32599]: pgmap v2926: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:00:29.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:29 smithi167 ceph-mon[37140]: pgmap v2926: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:00:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:30 smithi138 ceph-mon[36954]: pgmap v2927: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:00:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:30 smithi100 ceph-mon[32599]: pgmap v2927: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:00:31.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:30 smithi167 ceph-mon[37140]: pgmap v2927: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:00:32.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:00:32.258+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:00:32.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:00:32.259+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:00:32.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:00:32.259+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:00:32.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:00:32.259+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:00:32.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:00:32.259+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:00:33.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:33 smithi138 ceph-mon[36954]: pgmap v2928: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:00:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:33 smithi100 ceph-mon[32599]: pgmap v2928: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:00:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:33 smithi167 ceph-mon[37140]: pgmap v2928: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:00:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:35 smithi138 ceph-mon[36954]: pgmap v2929: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:00:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:35 smithi100 ceph-mon[32599]: pgmap v2929: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:00:35.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:35 smithi167 ceph-mon[37140]: pgmap v2929: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:00:37.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:37 smithi138 ceph-mon[36954]: pgmap v2930: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:00:37.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:37 smithi100 ceph-mon[32599]: pgmap v2930: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:00:37.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:37 smithi167 ceph-mon[37140]: pgmap v2930: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:00:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:39 smithi138 ceph-mon[36954]: pgmap v2931: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:00:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:39 smithi100 ceph-mon[32599]: pgmap v2931: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:00:39.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:39 smithi167 ceph-mon[37140]: pgmap v2931: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:00:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:41 smithi138 ceph-mon[36954]: pgmap v2932: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:00:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:41 smithi100 ceph-mon[32599]: pgmap v2932: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:00:41.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:41 smithi167 ceph-mon[37140]: pgmap v2932: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:00:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:43 smithi138 ceph-mon[36954]: pgmap v2933: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:00:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:43 smithi100 ceph-mon[32599]: pgmap v2933: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:00:43.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:43 smithi167 ceph-mon[37140]: pgmap v2933: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:00:45.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:45 smithi138 ceph-mon[36954]: pgmap v2934: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:00:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:45 smithi100 ceph-mon[32599]: pgmap v2934: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:00:45.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:45 smithi167 ceph-mon[37140]: pgmap v2934: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:00:47.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:00:47.274+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:00:47.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:00:47.275+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:00:47.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:00:47.275+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:00:47.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:00:47.275+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:00:47.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:00:47.275+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:00:47.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:47 smithi138 ceph-mon[36954]: pgmap v2935: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:00:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:47 smithi100 ceph-mon[32599]: pgmap v2935: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:00:47.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:47 smithi167 ceph-mon[37140]: pgmap v2935: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:00:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:49 smithi138 ceph-mon[36954]: pgmap v2936: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:00:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:49 smithi100 ceph-mon[32599]: pgmap v2936: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:00:49.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:49 smithi167 ceph-mon[37140]: pgmap v2936: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:00:51.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:51 smithi138 ceph-mon[36954]: pgmap v2937: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:00:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:51 smithi100 ceph-mon[32599]: pgmap v2937: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:00:51.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:51 smithi167 ceph-mon[37140]: pgmap v2937: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:00:53.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:53 smithi138 ceph-mon[36954]: pgmap v2938: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:00:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:53 smithi100 ceph-mon[32599]: pgmap v2938: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:00:53.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:53 smithi167 ceph-mon[37140]: pgmap v2938: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:00:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:55 smithi138 ceph-mon[36954]: pgmap v2939: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:00:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:55 smithi100 ceph-mon[32599]: pgmap v2939: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:00:55.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:55 smithi167 ceph-mon[37140]: pgmap v2939: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:00:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:57 smithi138 ceph-mon[36954]: pgmap v2940: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:00:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:57 smithi100 ceph-mon[32599]: pgmap v2940: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:00:57.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:57 smithi167 ceph-mon[37140]: pgmap v2940: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:00:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:00:59 smithi138 ceph-mon[36954]: pgmap v2941: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:00:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:00:59 smithi100 ceph-mon[32599]: pgmap v2941: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:00:59.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:00:59 smithi167 ceph-mon[37140]: pgmap v2941: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:01:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:01 smithi138 ceph-mon[36954]: pgmap v2942: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:01:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:01 smithi100 ceph-mon[32599]: pgmap v2942: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:01:01.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:01 smithi167 ceph-mon[37140]: pgmap v2942: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:01:02.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:01:02.290+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:01:02.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:01:02.290+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:01:02.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:01:02.290+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:01:02.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:01:02.290+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:01:02.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:01:02.290+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:01:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:03 smithi138 ceph-mon[36954]: pgmap v2943: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:01:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:03 smithi100 ceph-mon[32599]: pgmap v2943: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:01:03.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:03 smithi167 ceph-mon[37140]: pgmap v2943: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:01:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:05 smithi138 ceph-mon[36954]: pgmap v2944: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:01:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:05 smithi100 ceph-mon[32599]: pgmap v2944: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:01:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:05 smithi167 ceph-mon[37140]: pgmap v2944: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:01:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:07 smithi138 ceph-mon[36954]: pgmap v2945: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:01:07.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:07 smithi100 ceph-mon[32599]: pgmap v2945: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:01:07.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:07 smithi167 ceph-mon[37140]: pgmap v2945: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:01:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:08 smithi138 ceph-mon[36954]: osdmap e234: 12 total, 12 up, 12 in 2024-07-26T15:01:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:08 smithi100 ceph-mon[32599]: osdmap e234: 12 total, 12 up, 12 in 2024-07-26T15:01:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:08 smithi167 ceph-mon[37140]: osdmap e234: 12 total, 12 up, 12 in 2024-07-26T15:01:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:09 smithi138 ceph-mon[36954]: pgmap v2947: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:01:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:09 smithi100 ceph-mon[32599]: pgmap v2947: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:01:09.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:09 smithi167 ceph-mon[37140]: pgmap v2947: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:01:11.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:11 smithi138 ceph-mon[36954]: pgmap v2948: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:01:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:11 smithi100 ceph-mon[32599]: pgmap v2948: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:01:11.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:11 smithi167 ceph-mon[37140]: pgmap v2948: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:01:13.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:13 smithi138 ceph-mon[36954]: pgmap v2949: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:01:13.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:01:13.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:01:13.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:01:13.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:01:13.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:01:13.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:01:13.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:01:13.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:01:13.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:01:13.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:01:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:13 smithi100 ceph-mon[32599]: pgmap v2949: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:01:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:01:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:01:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:01:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:01:13.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:01:13.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:01:13.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:01:13.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:01:13.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:01:13.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:01:13.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:13 smithi167 ceph-mon[37140]: pgmap v2949: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:01:13.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:01:13.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:01:13.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:01:13.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:01:13.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:01:13.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:01:13.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:01:13.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:01:13.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:01:13.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:01:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:15 smithi138 ceph-mon[36954]: pgmap v2950: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:01:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:15 smithi138 ceph-mon[36954]: osdmap e235: 12 total, 12 up, 12 in 2024-07-26T15:01:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:15 smithi100 ceph-mon[32599]: pgmap v2950: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:01:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:15 smithi100 ceph-mon[32599]: osdmap e235: 12 total, 12 up, 12 in 2024-07-26T15:01:15.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:15 smithi167 ceph-mon[37140]: pgmap v2950: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:01:15.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:15 smithi167 ceph-mon[37140]: osdmap e235: 12 total, 12 up, 12 in 2024-07-26T15:01:17.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:01:17.296+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:01:17.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:01:17.296+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:01:17.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:01:17.296+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:01:17.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:01:17.296+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:01:17.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:01:17.296+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:01:17.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:17 smithi138 ceph-mon[36954]: pgmap v2952: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:01:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:17 smithi100 ceph-mon[32599]: pgmap v2952: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:01:17.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:17 smithi167 ceph-mon[37140]: pgmap v2952: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:01:19.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:19 smithi138 ceph-mon[36954]: pgmap v2953: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:01:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:19 smithi100 ceph-mon[32599]: pgmap v2953: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:01:19.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:19 smithi167 ceph-mon[37140]: pgmap v2953: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:01:21.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:21 smithi138 ceph-mon[36954]: pgmap v2954: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:01:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:21 smithi100 ceph-mon[32599]: pgmap v2954: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:01:21.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:21 smithi167 ceph-mon[37140]: pgmap v2954: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:01:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:23 smithi138 ceph-mon[36954]: pgmap v2955: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:01:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:23 smithi100 ceph-mon[32599]: pgmap v2955: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:01:23.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:23 smithi167 ceph-mon[37140]: pgmap v2955: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:01:25.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:25 smithi138 ceph-mon[36954]: pgmap v2956: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:01:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:25 smithi100 ceph-mon[32599]: pgmap v2956: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:01:25.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:25 smithi167 ceph-mon[37140]: pgmap v2956: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:01:27.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:27 smithi138 ceph-mon[36954]: pgmap v2957: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:01:27.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:27 smithi100 ceph-mon[32599]: pgmap v2957: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:01:27.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:27 smithi167 ceph-mon[37140]: pgmap v2957: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:01:29.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:29 smithi138 ceph-mon[36954]: pgmap v2958: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:01:29.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:29 smithi100 ceph-mon[32599]: pgmap v2958: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:01:29.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:29 smithi167 ceph-mon[37140]: pgmap v2958: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:01:31.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:31 smithi138 ceph-mon[36954]: pgmap v2959: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:01:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:31 smithi100 ceph-mon[32599]: pgmap v2959: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:01:31.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:31 smithi167 ceph-mon[37140]: pgmap v2959: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:01:32.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:01:32.311+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:01:32.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:01:32.312+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:01:32.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:01:32.312+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:01:32.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:01:32.312+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:01:32.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:01:32.312+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:01:33.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:33 smithi138 ceph-mon[36954]: pgmap v2960: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:01:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:33 smithi100 ceph-mon[32599]: pgmap v2960: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:01:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:33 smithi167 ceph-mon[37140]: pgmap v2960: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:01:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:35 smithi138 ceph-mon[36954]: pgmap v2961: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:01:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:35 smithi100 ceph-mon[32599]: pgmap v2961: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:01:35.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:35 smithi167 ceph-mon[37140]: pgmap v2961: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:01:37.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:37 smithi138 ceph-mon[36954]: pgmap v2962: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:01:37.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:37 smithi100 ceph-mon[32599]: pgmap v2962: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:01:37.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:37 smithi167 ceph-mon[37140]: pgmap v2962: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:01:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:39 smithi138 ceph-mon[36954]: pgmap v2963: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:01:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:39 smithi100 ceph-mon[32599]: pgmap v2963: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:01:39.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:39 smithi167 ceph-mon[37140]: pgmap v2963: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:01:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:41 smithi138 ceph-mon[36954]: pgmap v2964: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:01:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:41 smithi100 ceph-mon[32599]: pgmap v2964: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:01:41.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:41 smithi167 ceph-mon[37140]: pgmap v2964: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:01:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:43 smithi138 ceph-mon[36954]: pgmap v2965: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:01:43.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:43 smithi100 ceph-mon[32599]: pgmap v2965: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:01:43.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:43 smithi167 ceph-mon[37140]: pgmap v2965: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:01:45.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:45 smithi138 ceph-mon[36954]: pgmap v2966: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:01:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:45 smithi100 ceph-mon[32599]: pgmap v2966: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:01:45.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:45 smithi167 ceph-mon[37140]: pgmap v2966: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:01:47.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:01:47.327+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:01:47.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:01:47.328+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:01:47.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:01:47.328+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:01:47.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:01:47.328+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:01:47.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:01:47.328+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:01:47.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:47 smithi138 ceph-mon[36954]: pgmap v2967: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:01:47.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:47 smithi100 ceph-mon[32599]: pgmap v2967: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:01:47.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:47 smithi167 ceph-mon[37140]: pgmap v2967: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:01:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:49 smithi138 ceph-mon[36954]: pgmap v2968: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:01:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:49 smithi100 ceph-mon[32599]: pgmap v2968: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:01:49.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:49 smithi167 ceph-mon[37140]: pgmap v2968: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:01:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:51 smithi100 ceph-mon[32599]: pgmap v2969: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:01:51.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:51 smithi167 ceph-mon[37140]: pgmap v2969: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:01:51.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:51 smithi138 ceph-mon[36954]: pgmap v2969: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:01:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:53 smithi100 ceph-mon[32599]: pgmap v2970: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:01:53.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:53 smithi167 ceph-mon[37140]: pgmap v2970: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:01:53.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:53 smithi138 ceph-mon[36954]: pgmap v2970: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:01:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:55 smithi100 ceph-mon[32599]: pgmap v2971: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:01:55.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:55 smithi167 ceph-mon[37140]: pgmap v2971: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:01:55.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:55 smithi138 ceph-mon[36954]: pgmap v2971: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:01:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:57 smithi100 ceph-mon[32599]: pgmap v2972: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:01:57.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:57 smithi167 ceph-mon[37140]: pgmap v2972: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:01:57.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:57 smithi138 ceph-mon[36954]: pgmap v2972: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:01:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:01:59 smithi100 ceph-mon[32599]: pgmap v2973: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:01:59.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:01:59 smithi167 ceph-mon[37140]: pgmap v2973: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:01:59.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:01:59 smithi138 ceph-mon[36954]: pgmap v2973: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:02:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:01 smithi100 ceph-mon[32599]: pgmap v2974: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:02:01.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:01 smithi167 ceph-mon[37140]: pgmap v2974: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:02:01.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:01 smithi138 ceph-mon[36954]: pgmap v2974: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:02:02.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:02:02.332+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:02:02.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:02:02.332+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:02:02.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:02:02.332+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:02:02.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:02:02.332+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:02:02.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:02:02.332+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:02:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:03 smithi100 ceph-mon[32599]: pgmap v2975: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:02:03.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:03 smithi167 ceph-mon[37140]: pgmap v2975: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:02:03.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:03 smithi138 ceph-mon[36954]: pgmap v2975: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:02:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:05 smithi100 ceph-mon[32599]: pgmap v2976: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:02:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:05 smithi167 ceph-mon[37140]: pgmap v2976: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:02:05.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:05 smithi138 ceph-mon[36954]: pgmap v2976: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:02:07.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:07 smithi100 ceph-mon[32599]: pgmap v2977: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:02:07.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:07 smithi167 ceph-mon[37140]: pgmap v2977: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:02:07.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:07 smithi138 ceph-mon[36954]: pgmap v2977: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:02:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:08 smithi100 ceph-mon[32599]: osdmap e236: 12 total, 12 up, 12 in 2024-07-26T15:02:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:08 smithi167 ceph-mon[37140]: osdmap e236: 12 total, 12 up, 12 in 2024-07-26T15:02:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:08 smithi138 ceph-mon[36954]: osdmap e236: 12 total, 12 up, 12 in 2024-07-26T15:02:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:09 smithi100 ceph-mon[32599]: pgmap v2979: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:02:09.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:09 smithi167 ceph-mon[37140]: pgmap v2979: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:02:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:09 smithi138 ceph-mon[36954]: pgmap v2979: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:02:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:11 smithi100 ceph-mon[32599]: pgmap v2980: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:02:11.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:11 smithi167 ceph-mon[37140]: pgmap v2980: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:02:11.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:11 smithi138 ceph-mon[36954]: pgmap v2980: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:02:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:13 smithi100 ceph-mon[32599]: pgmap v2981: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:02:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:02:13.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:13 smithi167 ceph-mon[37140]: pgmap v2981: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:02:13.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:02:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:13 smithi138 ceph-mon[36954]: pgmap v2981: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:02:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:02:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:14 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:02:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:14 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:02:15.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:14 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:02:15.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:14 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:02:15.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:14 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:02:15.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:14 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:02:15.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:14 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:02:15.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:14 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:02:15.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:14 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:02:15.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:14 smithi138 ceph-mon[36954]: pgmap v2982: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:02:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:14 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:02:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:14 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:02:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:14 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:02:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:14 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:02:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:14 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:02:15.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:14 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:02:15.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:14 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:02:15.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:14 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:02:15.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:14 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:02:15.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:14 smithi100 ceph-mon[32599]: pgmap v2982: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:02:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:14 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:02:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:14 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:02:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:14 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:02:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:14 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:02:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:14 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:02:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:14 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:02:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:14 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:02:15.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:14 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:02:15.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:14 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:02:15.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:14 smithi167 ceph-mon[37140]: pgmap v2982: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:02:16.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:15 smithi138 ceph-mon[36954]: osdmap e237: 12 total, 12 up, 12 in 2024-07-26T15:02:16.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:15 smithi100 ceph-mon[32599]: osdmap e237: 12 total, 12 up, 12 in 2024-07-26T15:02:16.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:15 smithi167 ceph-mon[37140]: osdmap e237: 12 total, 12 up, 12 in 2024-07-26T15:02:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:16 smithi138 ceph-mon[36954]: pgmap v2984: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 232 B/s wr, 6 op/s 2024-07-26T15:02:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:16 smithi100 ceph-mon[32599]: pgmap v2984: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 232 B/s wr, 6 op/s 2024-07-26T15:02:17.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:16 smithi167 ceph-mon[37140]: pgmap v2984: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 232 B/s wr, 6 op/s 2024-07-26T15:02:17.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:02:17.348+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:02:17.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:02:17.349+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:02:17.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:02:17.349+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:02:17.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:02:17.349+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:02:17.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:02:17.349+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:02:18.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:02:18.644+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T15:02:18.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:02:18.644+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T15:02:18.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:02:18.644+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 38 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T15:02:18.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:02:18.645+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 38 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T15:02:19.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:19 smithi138 ceph-mon[36954]: pgmap v2985: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:02:19.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:19 smithi100 ceph-mon[32599]: pgmap v2985: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:02:19.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:19 smithi167 ceph-mon[37140]: pgmap v2985: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:02:21.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:21 smithi138 ceph-mon[36954]: pgmap v2986: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:02:21.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:21 smithi100 ceph-mon[32599]: pgmap v2986: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:02:21.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:21 smithi167 ceph-mon[37140]: pgmap v2986: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:02:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:23 smithi138 ceph-mon[36954]: pgmap v2987: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:02:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:23 smithi100 ceph-mon[32599]: pgmap v2987: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:02:23.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:23 smithi167 ceph-mon[37140]: pgmap v2987: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:02:25.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:25 smithi138 ceph-mon[36954]: pgmap v2988: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:02:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:25 smithi100 ceph-mon[32599]: pgmap v2988: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:02:25.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:25 smithi167 ceph-mon[37140]: pgmap v2988: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:02:27.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:27 smithi138 ceph-mon[36954]: pgmap v2989: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:02:27.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:27 smithi100 ceph-mon[32599]: pgmap v2989: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:02:27.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:27 smithi167 ceph-mon[37140]: pgmap v2989: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:02:29.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:29 smithi138 ceph-mon[36954]: pgmap v2990: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:02:29.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:29 smithi100 ceph-mon[32599]: pgmap v2990: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:02:29.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:29 smithi167 ceph-mon[37140]: pgmap v2990: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:02:31.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:31 smithi138 ceph-mon[36954]: pgmap v2991: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:02:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:31 smithi100 ceph-mon[32599]: pgmap v2991: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:02:31.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:31 smithi167 ceph-mon[37140]: pgmap v2991: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:02:32.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:02:32.365+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:02:32.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:02:32.365+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:02:32.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:02:32.365+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:02:32.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:02:32.365+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:02:32.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:02:32.365+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:02:33.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:33 smithi138 ceph-mon[36954]: pgmap v2992: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:02:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:33 smithi100 ceph-mon[32599]: pgmap v2992: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:02:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:33 smithi167 ceph-mon[37140]: pgmap v2992: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:02:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:35 smithi138 ceph-mon[36954]: pgmap v2993: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:02:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:35 smithi100 ceph-mon[32599]: pgmap v2993: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:02:35.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:35 smithi167 ceph-mon[37140]: pgmap v2993: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:02:37.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:37 smithi138 ceph-mon[36954]: pgmap v2994: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:02:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:37 smithi100 ceph-mon[32599]: pgmap v2994: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:02:37.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:37 smithi167 ceph-mon[37140]: pgmap v2994: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:02:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:39 smithi138 ceph-mon[36954]: pgmap v2995: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:02:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:39 smithi100 ceph-mon[32599]: pgmap v2995: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:02:39.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:39 smithi167 ceph-mon[37140]: pgmap v2995: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:02:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:41 smithi138 ceph-mon[36954]: pgmap v2996: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:02:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:41 smithi100 ceph-mon[32599]: pgmap v2996: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:02:41.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:41 smithi167 ceph-mon[37140]: pgmap v2996: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:02:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:43 smithi138 ceph-mon[36954]: pgmap v2997: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:02:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:43 smithi100 ceph-mon[32599]: pgmap v2997: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:02:43.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:43 smithi167 ceph-mon[37140]: pgmap v2997: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:02:45.443 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:45 smithi138 ceph-mon[36954]: pgmap v2998: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:02:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:45 smithi100 ceph-mon[32599]: pgmap v2998: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:02:45.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:45 smithi167 ceph-mon[37140]: pgmap v2998: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:02:47.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:02:47.381+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:02:47.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:02:47.382+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:02:47.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:02:47.382+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:02:47.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:02:47.382+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:02:47.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:02:47.382+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:02:47.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:47 smithi138 ceph-mon[36954]: pgmap v2999: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:02:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:47 smithi100 ceph-mon[32599]: pgmap v2999: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:02:47.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:47 smithi167 ceph-mon[37140]: pgmap v2999: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:02:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:49 smithi138 ceph-mon[36954]: pgmap v3000: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:02:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:49 smithi100 ceph-mon[32599]: pgmap v3000: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:02:49.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:49 smithi167 ceph-mon[37140]: pgmap v3000: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:02:51.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:51 smithi138 ceph-mon[36954]: pgmap v3001: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:02:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:51 smithi100 ceph-mon[32599]: pgmap v3001: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:02:51.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:51 smithi167 ceph-mon[37140]: pgmap v3001: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:02:53.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:53 smithi138 ceph-mon[36954]: pgmap v3002: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:02:53.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:53 smithi100 ceph-mon[32599]: pgmap v3002: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:02:53.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:53 smithi167 ceph-mon[37140]: pgmap v3002: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:02:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:55 smithi138 ceph-mon[36954]: pgmap v3003: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:02:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:55 smithi100 ceph-mon[32599]: pgmap v3003: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:02:55.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:55 smithi167 ceph-mon[37140]: pgmap v3003: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:02:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:57 smithi138 ceph-mon[36954]: pgmap v3004: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:02:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:57 smithi100 ceph-mon[32599]: pgmap v3004: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:02:57.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:57 smithi167 ceph-mon[37140]: pgmap v3004: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:02:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:02:59 smithi138 ceph-mon[36954]: pgmap v3005: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:02:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:02:59 smithi100 ceph-mon[32599]: pgmap v3005: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:02:59.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:02:59 smithi167 ceph-mon[37140]: pgmap v3005: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:03:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:03:01 smithi138 ceph-mon[36954]: pgmap v3006: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:03:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:03:01 smithi100 ceph-mon[32599]: pgmap v3006: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:03:01.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:03:01 smithi167 ceph-mon[37140]: pgmap v3006: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:03:02.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:03:02.397+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:03:02.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:03:02.397+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:03:02.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:03:02.397+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:03:02.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:03:02.397+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:03:02.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:03:02.397+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:03:03.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:03:03.031+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b3d40 msgr2=0x7fe5940b4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-07-26T15:03:03.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:03:03.031+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b3d40 msgr2=0x7fe5940b4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-07-26T15:03:03.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:03:03.031+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b3d40 0x7fe5940b4730 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe5900051a0 tx=0x7fe590074040 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:03:03.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:03:03.031+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b3d40 0x7fe5940b4730 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe5900051a0 tx=0x7fe590074040 comp rx=0 tx=0).stop 2024-07-26T15:03:03.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:03:03.031+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b3d40 msgr2=0x7fe5940b4730 unknown :-1 s=STATE_CLOSED l=1).tick see no progress in more than 10000000 us during connecting to v2:172.21.15.100:6800/3937009534, fault. 2024-07-26T15:03:03.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:03:03.031+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b3d40 msgr2=0x7fe5940b4730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-07-26T15:03:03.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:03:03.031+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b3d40 0x7fe5940b4730 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T15:03:03.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:03:03.031+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T15:03:03.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:03:03.031+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:03:03.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:03:03.031+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe59000b6b0 tx=0x7fe59005a040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T15:03:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:03:03 smithi138 ceph-mon[36954]: pgmap v3007: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:03:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:03:03 smithi100 ceph-mon[32599]: pgmap v3007: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:03:03.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:03:03 smithi167 ceph-mon[37140]: pgmap v3007: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:03:03.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:03:03.650+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T15:03:03.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:03:03.650+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T15:03:03.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:03:03.650+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 39 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T15:03:03.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:03:03.650+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 39 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T15:03:05.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:03:05 smithi138 ceph-mon[36954]: pgmap v3008: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:03:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:03:05 smithi100 ceph-mon[32599]: pgmap v3008: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:03:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:03:05 smithi167 ceph-mon[37140]: pgmap v3008: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:03:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:03:07 smithi138 ceph-mon[36954]: pgmap v3009: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:03:07.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:03:07 smithi100 ceph-mon[32599]: pgmap v3009: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:03:07.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:03:07 smithi167 ceph-mon[37140]: pgmap v3009: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:03:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:03:08 smithi138 ceph-mon[36954]: osdmap e238: 12 total, 12 up, 12 in 2024-07-26T15:03:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:03:08 smithi100 ceph-mon[32599]: osdmap e238: 12 total, 12 up, 12 in 2024-07-26T15:03:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:03:08 smithi167 ceph-mon[37140]: osdmap e238: 12 total, 12 up, 12 in 2024-07-26T15:03:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:03:09 smithi138 ceph-mon[36954]: pgmap v3011: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:03:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:03:09 smithi100 ceph-mon[32599]: pgmap v3011: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:03:09.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:03:09 smithi167 ceph-mon[37140]: pgmap v3011: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:03:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:03:11 smithi100 ceph-mon[32599]: pgmap v3012: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:03:11.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:03:11 smithi167 ceph-mon[37140]: pgmap v3012: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:03:11.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:03:11 smithi138 ceph-mon[36954]: pgmap v3012: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:03:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:03:13 smithi100 ceph-mon[32599]: pgmap v3013: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:03:13.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:03:13 smithi167 ceph-mon[37140]: pgmap v3013: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:03:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:03:13 smithi138 ceph-mon[36954]: pgmap v3013: 97 pgs: 97 active+clean; 234 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:03:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:03:14 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:03:14.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:03:14 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:03:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:03:14 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:03:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:03:15 smithi100 ceph-mon[32599]: pgmap v3014: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:03:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:03:15 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:03:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:03:15 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:03:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:03:15 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:03:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:03:15 smithi100 ceph-mon[32599]: osdmap e239: 12 total, 12 up, 12 in 2024-07-26T15:03:15.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:03:15 smithi167 ceph-mon[37140]: pgmap v3014: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:03:15.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:03:15 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:03:15.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:03:15 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:03:15.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:03:15 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:03:15.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:03:15 smithi167 ceph-mon[37140]: osdmap e239: 12 total, 12 up, 12 in 2024-07-26T15:03:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:03:15 smithi138 ceph-mon[36954]: pgmap v3014: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:03:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:03:15 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:03:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:03:15 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:03:15.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:03:15 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:03:15.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:03:15 smithi138 ceph-mon[36954]: osdmap e239: 12 total, 12 up, 12 in 2024-07-26T15:03:17.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:03:17.405+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:03:17.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:03:17.406+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:03:17.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:03:17.406+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:03:17.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:03:17.406+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:03:17.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:03:17.406+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:03:17.493 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:03:17 smithi138 ceph-mon[36954]: pgmap v3016: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T15:03:17.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:03:17 smithi100 ceph-mon[32599]: pgmap v3016: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T15:03:17.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:03:17 smithi167 ceph-mon[37140]: pgmap v3016: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T15:03:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:03:19 smithi100 ceph-mon[32599]: pgmap v3017: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:03:19.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:03:19 smithi167 ceph-mon[37140]: pgmap v3017: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:03:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:03:19 smithi138 ceph-mon[36954]: pgmap v3017: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:03:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:03:21 smithi100 ceph-mon[32599]: pgmap v3018: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:03:21.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:03:21 smithi167 ceph-mon[37140]: pgmap v3018: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:03:21.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:03:21 smithi138 ceph-mon[36954]: pgmap v3018: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:03:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:03:23 smithi100 ceph-mon[32599]: pgmap v3019: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:03:23.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:03:23 smithi167 ceph-mon[37140]: pgmap v3019: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:03:23.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:03:23 smithi138 ceph-mon[36954]: pgmap v3019: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:03:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:03:25 smithi100 ceph-mon[32599]: pgmap v3020: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:03:25.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:03:25 smithi167 ceph-mon[37140]: pgmap v3020: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:03:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:03:25 smithi138 ceph-mon[36954]: pgmap v3020: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:03:27.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:03:27 smithi100 ceph-mon[32599]: pgmap v3021: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:03:27.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:03:27 smithi167 ceph-mon[37140]: pgmap v3021: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:03:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:03:27 smithi138 ceph-mon[36954]: pgmap v3021: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:03:29.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:03:29 smithi100 ceph-mon[32599]: pgmap v3022: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:03:29.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:03:29 smithi167 ceph-mon[37140]: pgmap v3022: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:03:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:03:29 smithi138 ceph-mon[36954]: pgmap v3022: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:03:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:03:31 smithi100 ceph-mon[32599]: pgmap v3023: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:03:31.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:03:31 smithi167 ceph-mon[37140]: pgmap v3023: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:03:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:03:31 smithi138 ceph-mon[36954]: pgmap v3023: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:03:32.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:03:32.421+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:03:32.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:03:32.422+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:03:32.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:03:32.422+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:03:32.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:03:32.422+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:03:32.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:03:32.422+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:03:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:03:33 smithi100 ceph-mon[32599]: pgmap v3024: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:03:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:03:33 smithi167 ceph-mon[37140]: pgmap v3024: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:03:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:03:33 smithi138 ceph-mon[36954]: pgmap v3024: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:03:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:03:35 smithi100 ceph-mon[32599]: pgmap v3025: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:03:35.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:03:35 smithi167 ceph-mon[37140]: pgmap v3025: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:03:35.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:03:35 smithi138 ceph-mon[36954]: pgmap v3025: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:03:37.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:03:37 smithi167 ceph-mon[37140]: pgmap v3026: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:03:37.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:03:37 smithi138 ceph-mon[36954]: pgmap v3026: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:03:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:03:37 smithi100 ceph-mon[32599]: pgmap v3026: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:03:39.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:03:39 smithi167 ceph-mon[37140]: pgmap v3027: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:03:39.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:03:39 smithi138 ceph-mon[36954]: pgmap v3027: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:03:39.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:03:39 smithi100 ceph-mon[32599]: pgmap v3027: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:03:41.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:03:41 smithi167 ceph-mon[37140]: pgmap v3028: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:03:41.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:03:41 smithi138 ceph-mon[36954]: pgmap v3028: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:03:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:03:41 smithi100 ceph-mon[32599]: pgmap v3028: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:03:43.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:03:43 smithi167 ceph-mon[37140]: pgmap v3029: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:03:43.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:03:43 smithi138 ceph-mon[36954]: pgmap v3029: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:03:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:03:43 smithi100 ceph-mon[32599]: pgmap v3029: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:03:45.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:03:45 smithi167 ceph-mon[37140]: pgmap v3030: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:03:45.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:03:45 smithi138 ceph-mon[36954]: pgmap v3030: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:03:45.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:03:45 smithi100 ceph-mon[32599]: pgmap v3030: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:03:47.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:03:47.432+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:03:47.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:03:47.432+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:03:47.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:03:47.432+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:03:47.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:03:47.432+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:03:47.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:03:47.432+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:03:47.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:03:47 smithi167 ceph-mon[37140]: pgmap v3031: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:03:47.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:03:47 smithi138 ceph-mon[36954]: pgmap v3031: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:03:47.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:03:47 smithi100 ceph-mon[32599]: pgmap v3031: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:03:49.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:03:49 smithi167 ceph-mon[37140]: pgmap v3032: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:03:49.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:03:49 smithi138 ceph-mon[36954]: pgmap v3032: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:03:49.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:03:49 smithi100 ceph-mon[32599]: pgmap v3032: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:03:51.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:03:51 smithi167 ceph-mon[37140]: pgmap v3033: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:03:51.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:03:51 smithi138 ceph-mon[36954]: pgmap v3033: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:03:51.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:03:51 smithi100 ceph-mon[32599]: pgmap v3033: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:03:53.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:03:53 smithi167 ceph-mon[37140]: pgmap v3034: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:03:53.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:03:53 smithi138 ceph-mon[36954]: pgmap v3034: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:03:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:03:53 smithi100 ceph-mon[32599]: pgmap v3034: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:03:55.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:03:55 smithi138 ceph-mon[36954]: pgmap v3035: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:03:55.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:03:55 smithi100 ceph-mon[32599]: pgmap v3035: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:03:55.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:03:55 smithi167 ceph-mon[37140]: pgmap v3035: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:03:57.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:03:57 smithi138 ceph-mon[36954]: pgmap v3036: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:03:57.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:03:57 smithi100 ceph-mon[32599]: pgmap v3036: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:03:57.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:03:57 smithi167 ceph-mon[37140]: pgmap v3036: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:03:59.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:03:59 smithi138 ceph-mon[36954]: pgmap v3037: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:03:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:03:59 smithi100 ceph-mon[32599]: pgmap v3037: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:03:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:03:59 smithi167 ceph-mon[37140]: pgmap v3037: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:04:01.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:04:01 smithi100 ceph-mon[32599]: pgmap v3038: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:04:01.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:04:01 smithi138 ceph-mon[36954]: pgmap v3038: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:04:01.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:04:01 smithi167 ceph-mon[37140]: pgmap v3038: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:04:02.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:04:02.445+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:04:02.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:04:02.446+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:04:02.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:04:02.446+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:04:02.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:04:02.446+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:04:02.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:04:02.446+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:04:03.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:04:03 smithi138 ceph-mon[36954]: pgmap v3039: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:04:03.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:04:03 smithi100 ceph-mon[32599]: pgmap v3039: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:04:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:04:03 smithi167 ceph-mon[37140]: pgmap v3039: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:04:05.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:04:05 smithi138 ceph-mon[36954]: pgmap v3040: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:04:05.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:04:05 smithi100 ceph-mon[32599]: pgmap v3040: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:04:05.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:04:05 smithi167 ceph-mon[37140]: pgmap v3040: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:04:07.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:04:07 smithi138 ceph-mon[36954]: pgmap v3041: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:04:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:04:07 smithi100 ceph-mon[32599]: pgmap v3041: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:04:07.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:04:07 smithi167 ceph-mon[37140]: pgmap v3041: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:04:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:04:08 smithi138 ceph-mon[36954]: osdmap e240: 12 total, 12 up, 12 in 2024-07-26T15:04:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:04:08 smithi100 ceph-mon[32599]: osdmap e240: 12 total, 12 up, 12 in 2024-07-26T15:04:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:04:08 smithi167 ceph-mon[37140]: osdmap e240: 12 total, 12 up, 12 in 2024-07-26T15:04:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:04:09 smithi138 ceph-mon[36954]: pgmap v3043: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:04:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:04:09 smithi100 ceph-mon[32599]: pgmap v3043: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:04:09.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:04:09 smithi167 ceph-mon[37140]: pgmap v3043: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:04:11.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:04:11 smithi138 ceph-mon[36954]: pgmap v3044: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:04:11.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:04:11 smithi100 ceph-mon[32599]: pgmap v3044: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:04:11.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:04:11 smithi167 ceph-mon[37140]: pgmap v3044: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:04:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:04:13 smithi138 ceph-mon[36954]: pgmap v3045: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:04:13.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:04:13 smithi100 ceph-mon[32599]: pgmap v3045: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:04:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:04:13 smithi167 ceph-mon[37140]: pgmap v3045: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:04:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:04:15 smithi138 ceph-mon[36954]: pgmap v3046: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:04:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:04:15 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:04:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:04:15 smithi138 ceph-mon[36954]: osdmap e241: 12 total, 12 up, 12 in 2024-07-26T15:04:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:04:15 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:04:15.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:04:15 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:04:15.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:04:15 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:04:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:04:15 smithi100 ceph-mon[32599]: pgmap v3046: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:04:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:04:15 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:04:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:04:15 smithi100 ceph-mon[32599]: osdmap e241: 12 total, 12 up, 12 in 2024-07-26T15:04:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:04:15 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:04:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:04:15 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:04:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:04:15 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:04:15.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:04:15 smithi167 ceph-mon[37140]: pgmap v3046: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:04:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:04:15 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:04:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:04:15 smithi167 ceph-mon[37140]: osdmap e241: 12 total, 12 up, 12 in 2024-07-26T15:04:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:04:15 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:04:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:04:15 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:04:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:04:15 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:04:17.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:04:17.462+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:04:17.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:04:17.462+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:04:17.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:04:17.462+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:04:17.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:04:17.462+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:04:17.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:04:17.462+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:04:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:04:17 smithi138 ceph-mon[36954]: pgmap v3048: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 233 B/s wr, 6 op/s 2024-07-26T15:04:17.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:04:17 smithi100 ceph-mon[32599]: pgmap v3048: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 233 B/s wr, 6 op/s 2024-07-26T15:04:17.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:04:17 smithi167 ceph-mon[37140]: pgmap v3048: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 233 B/s wr, 6 op/s 2024-07-26T15:04:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:04:19 smithi138 ceph-mon[36954]: pgmap v3049: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:04:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:04:19 smithi100 ceph-mon[32599]: pgmap v3049: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:04:19.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:04:19 smithi167 ceph-mon[37140]: pgmap v3049: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:04:21.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:04:21 smithi138 ceph-mon[36954]: pgmap v3050: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:04:21.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:04:21 smithi100 ceph-mon[32599]: pgmap v3050: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:04:21.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:04:21 smithi167 ceph-mon[37140]: pgmap v3050: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:04:23.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:04:23 smithi138 ceph-mon[36954]: pgmap v3051: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:04:23.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:04:23 smithi100 ceph-mon[32599]: pgmap v3051: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:04:23.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:04:23 smithi167 ceph-mon[37140]: pgmap v3051: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:04:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:04:25 smithi138 ceph-mon[36954]: pgmap v3052: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:04:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:04:25 smithi100 ceph-mon[32599]: pgmap v3052: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:04:25.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:04:25 smithi167 ceph-mon[37140]: pgmap v3052: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:04:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:04:27 smithi138 ceph-mon[36954]: pgmap v3053: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:04:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:04:27 smithi100 ceph-mon[32599]: pgmap v3053: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:04:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:04:27 smithi167 ceph-mon[37140]: pgmap v3053: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:04:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:04:28 smithi138 ceph-mon[36954]: pgmap v3054: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:04:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:04:28 smithi100 ceph-mon[32599]: pgmap v3054: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:04:28.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:04:28 smithi167 ceph-mon[37140]: pgmap v3054: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:04:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:04:30 smithi100 ceph-mon[32599]: pgmap v3055: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:04:31.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:04:30 smithi167 ceph-mon[37140]: pgmap v3055: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:04:31.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:04:30 smithi138 ceph-mon[36954]: pgmap v3055: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:04:32.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:04:32.477+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:04:32.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:04:32.477+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:04:32.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:04:32.477+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:04:32.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:04:32.477+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:04:32.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:04:32.477+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:04:33.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:04:33 smithi138 ceph-mon[36954]: pgmap v3056: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:04:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:04:33 smithi100 ceph-mon[32599]: pgmap v3056: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:04:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:04:33 smithi167 ceph-mon[37140]: pgmap v3056: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:04:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:04:35 smithi138 ceph-mon[36954]: pgmap v3057: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:04:35.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:04:35 smithi100 ceph-mon[32599]: pgmap v3057: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:04:35.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:04:35 smithi167 ceph-mon[37140]: pgmap v3057: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:04:37.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:04:37 smithi138 ceph-mon[36954]: pgmap v3058: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:04:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:04:37 smithi100 ceph-mon[32599]: pgmap v3058: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:04:37.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:04:37 smithi167 ceph-mon[37140]: pgmap v3058: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:04:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:04:39 smithi138 ceph-mon[36954]: pgmap v3059: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:04:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:04:39 smithi100 ceph-mon[32599]: pgmap v3059: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:04:39.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:04:39 smithi167 ceph-mon[37140]: pgmap v3059: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:04:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:04:41 smithi138 ceph-mon[36954]: pgmap v3060: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:04:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:04:41 smithi100 ceph-mon[32599]: pgmap v3060: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:04:41.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:04:41 smithi167 ceph-mon[37140]: pgmap v3060: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:04:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:04:43 smithi138 ceph-mon[36954]: pgmap v3061: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:04:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:04:43 smithi100 ceph-mon[32599]: pgmap v3061: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:04:43.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:04:43 smithi167 ceph-mon[37140]: pgmap v3061: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:04:45.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:04:45 smithi138 ceph-mon[36954]: pgmap v3062: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:04:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:04:45 smithi100 ceph-mon[32599]: pgmap v3062: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:04:45.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:04:45 smithi167 ceph-mon[37140]: pgmap v3062: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:04:47.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:04:47 smithi138 ceph-mon[36954]: pgmap v3063: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:04:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:04:47 smithi100 ceph-mon[32599]: pgmap v3063: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:04:47.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:04:47.493+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:04:47.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:04:47.494+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:04:47.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:04:47.494+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:04:47.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:04:47.494+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:04:47.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:04:47.494+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:04:47.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:04:47 smithi167 ceph-mon[37140]: pgmap v3063: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:04:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:04:49 smithi138 ceph-mon[36954]: pgmap v3064: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:04:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:04:49 smithi100 ceph-mon[32599]: pgmap v3064: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:04:49.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:04:49 smithi167 ceph-mon[37140]: pgmap v3064: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:04:51.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:04:51 smithi138 ceph-mon[36954]: pgmap v3065: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:04:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:04:51 smithi100 ceph-mon[32599]: pgmap v3065: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:04:51.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:04:51 smithi167 ceph-mon[37140]: pgmap v3065: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:04:53.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:04:53 smithi138 ceph-mon[36954]: pgmap v3066: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:04:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:04:53 smithi100 ceph-mon[32599]: pgmap v3066: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:04:53.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:04:53 smithi167 ceph-mon[37140]: pgmap v3066: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:04:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:04:55 smithi138 ceph-mon[36954]: pgmap v3067: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:04:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:04:55 smithi100 ceph-mon[32599]: pgmap v3067: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:04:55.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:04:55 smithi167 ceph-mon[37140]: pgmap v3067: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:04:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:04:57 smithi138 ceph-mon[36954]: pgmap v3068: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:04:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:04:57 smithi100 ceph-mon[32599]: pgmap v3068: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:04:57.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:04:57 smithi167 ceph-mon[37140]: pgmap v3068: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:04:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:04:59 smithi138 ceph-mon[36954]: pgmap v3069: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:04:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:04:59 smithi100 ceph-mon[32599]: pgmap v3069: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:04:59.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:04:59 smithi167 ceph-mon[37140]: pgmap v3069: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:05:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:05:01 smithi138 ceph-mon[36954]: pgmap v3070: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:05:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:05:01 smithi100 ceph-mon[32599]: pgmap v3070: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:05:01.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:05:01 smithi167 ceph-mon[37140]: pgmap v3070: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:05:02.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:05:02.500+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:05:02.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:05:02.501+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:05:02.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:05:02.501+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:05:02.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:05:02.501+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:05:02.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:05:02.501+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:05:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:05:03 smithi138 ceph-mon[36954]: pgmap v3071: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-07-26T15:05:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:05:03 smithi100 ceph-mon[32599]: pgmap v3071: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-07-26T15:05:03.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:05:03 smithi167 ceph-mon[37140]: pgmap v3071: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-07-26T15:05:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:05:05 smithi138 ceph-mon[36954]: pgmap v3072: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-07-26T15:05:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:05:05 smithi100 ceph-mon[32599]: pgmap v3072: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-07-26T15:05:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:05:05 smithi167 ceph-mon[37140]: pgmap v3072: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-07-26T15:05:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:05:07 smithi138 ceph-mon[36954]: pgmap v3073: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-07-26T15:05:07.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:05:07 smithi100 ceph-mon[32599]: pgmap v3073: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-07-26T15:05:07.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:05:07 smithi167 ceph-mon[37140]: pgmap v3073: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-07-26T15:05:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:05:09 smithi100 ceph-mon[32599]: pgmap v3074: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-07-26T15:05:09.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:05:09 smithi167 ceph-mon[37140]: pgmap v3074: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-07-26T15:05:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:05:09 smithi138 ceph-mon[36954]: pgmap v3074: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-07-26T15:05:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:05:11 smithi100 ceph-mon[32599]: pgmap v3075: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-07-26T15:05:11.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:05:11 smithi167 ceph-mon[37140]: pgmap v3075: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-07-26T15:05:11.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:05:11 smithi138 ceph-mon[36954]: pgmap v3075: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-07-26T15:05:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:05:13 smithi100 ceph-mon[32599]: pgmap v3076: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-07-26T15:05:13.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:05:13 smithi167 ceph-mon[37140]: pgmap v3076: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-07-26T15:05:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:05:13 smithi138 ceph-mon[36954]: pgmap v3076: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-07-26T15:05:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:05:15 smithi100 ceph-mon[32599]: pgmap v3077: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-07-26T15:05:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:05:15 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:05:15.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:05:15 smithi167 ceph-mon[37140]: pgmap v3077: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-07-26T15:05:15.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:05:15 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:05:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:05:15 smithi138 ceph-mon[36954]: pgmap v3077: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-07-26T15:05:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:05:15 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:05:16.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:05:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:05:16.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:05:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:05:16.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:05:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:05:16.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:05:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:05:16.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:05:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:05:16.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:05:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:05:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:05:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:05:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:05:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:05:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:05:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:05:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:05:17 smithi100 ceph-mon[32599]: pgmap v3078: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-07-26T15:05:17.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:05:17.509+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:05:17.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:05:17.510+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:05:17.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:05:17.510+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:05:17.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:05:17.510+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:05:17.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:05:17.510+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:05:17.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:05:17 smithi167 ceph-mon[37140]: pgmap v3078: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-07-26T15:05:17.591 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:05:17 smithi138 ceph-mon[36954]: pgmap v3078: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-07-26T15:05:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:05:19 smithi100 ceph-mon[32599]: pgmap v3079: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-07-26T15:05:19.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:05:19 smithi167 ceph-mon[37140]: pgmap v3079: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-07-26T15:05:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:05:19 smithi138 ceph-mon[36954]: pgmap v3079: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-07-26T15:05:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:05:21 smithi100 ceph-mon[32599]: pgmap v3080: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-07-26T15:05:21.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:05:21 smithi167 ceph-mon[37140]: pgmap v3080: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-07-26T15:05:21.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:05:21 smithi138 ceph-mon[36954]: pgmap v3080: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-07-26T15:05:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:05:23 smithi100 ceph-mon[32599]: pgmap v3081: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:05:23.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:05:23 smithi167 ceph-mon[37140]: pgmap v3081: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:05:23.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:05:23 smithi138 ceph-mon[36954]: pgmap v3081: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:05:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:05:25 smithi100 ceph-mon[32599]: pgmap v3082: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:05:25.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:05:25 smithi167 ceph-mon[37140]: pgmap v3082: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:05:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:05:25 smithi138 ceph-mon[36954]: pgmap v3082: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:05:27.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:05:27 smithi100 ceph-mon[32599]: pgmap v3083: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:05:27.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:05:27 smithi167 ceph-mon[37140]: pgmap v3083: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:05:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:05:27 smithi138 ceph-mon[36954]: pgmap v3083: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:05:29.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:05:29 smithi100 ceph-mon[32599]: pgmap v3084: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:05:29.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:05:29 smithi167 ceph-mon[37140]: pgmap v3084: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:05:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:05:29 smithi138 ceph-mon[36954]: pgmap v3084: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:05:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:05:31 smithi100 ceph-mon[32599]: pgmap v3085: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:05:31.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:05:31 smithi167 ceph-mon[37140]: pgmap v3085: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:05:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:05:31 smithi138 ceph-mon[36954]: pgmap v3085: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:05:32.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:05:32.520+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:05:32.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:05:32.521+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:05:32.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:05:32.521+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:05:32.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:05:32.521+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:05:32.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:05:32.521+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:05:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:05:33 smithi100 ceph-mon[32599]: pgmap v3086: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:05:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:05:33 smithi167 ceph-mon[37140]: pgmap v3086: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:05:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:05:33 smithi138 ceph-mon[36954]: pgmap v3086: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:05:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:05:35 smithi100 ceph-mon[32599]: pgmap v3087: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:05:35.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:05:35 smithi167 ceph-mon[37140]: pgmap v3087: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:05:35.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:05:35 smithi138 ceph-mon[36954]: pgmap v3087: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:05:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:05:37 smithi100 ceph-mon[32599]: pgmap v3088: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:05:37.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:05:37 smithi167 ceph-mon[37140]: pgmap v3088: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:05:37.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:05:37 smithi138 ceph-mon[36954]: pgmap v3088: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:05:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:05:39 smithi100 ceph-mon[32599]: pgmap v3089: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:05:39.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:05:39 smithi167 ceph-mon[37140]: pgmap v3089: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:05:39.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:05:39 smithi138 ceph-mon[36954]: pgmap v3089: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:05:41.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:05:41 smithi167 ceph-mon[37140]: pgmap v3090: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:05:41.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:05:41 smithi138 ceph-mon[36954]: pgmap v3090: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:05:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:05:41 smithi100 ceph-mon[32599]: pgmap v3090: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:05:43.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:05:43 smithi167 ceph-mon[37140]: pgmap v3091: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:05:43.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:05:43 smithi138 ceph-mon[36954]: pgmap v3091: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:05:43.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:05:43 smithi100 ceph-mon[32599]: pgmap v3091: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:05:45.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:05:45 smithi167 ceph-mon[37140]: pgmap v3092: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:05:45.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:05:45 smithi138 ceph-mon[36954]: pgmap v3092: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:05:45.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:05:45 smithi100 ceph-mon[32599]: pgmap v3092: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:05:47.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:05:47 smithi167 ceph-mon[37140]: pgmap v3093: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:05:47.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:05:47.524+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:05:47.526 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:05:47 smithi100 ceph-mon[32599]: pgmap v3093: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:05:47.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:05:47.525+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:05:47.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:05:47.525+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:05:47.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:05:47.525+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:05:47.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:05:47.525+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:05:47.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:05:47 smithi138 ceph-mon[36954]: pgmap v3093: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:05:49.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:05:49 smithi167 ceph-mon[37140]: pgmap v3094: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:05:49.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:05:49 smithi138 ceph-mon[36954]: pgmap v3094: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:05:49.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:05:49 smithi100 ceph-mon[32599]: pgmap v3094: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:05:51.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:05:51 smithi167 ceph-mon[37140]: pgmap v3095: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:05:51.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:05:51 smithi138 ceph-mon[36954]: pgmap v3095: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:05:51.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:05:51 smithi100 ceph-mon[32599]: pgmap v3095: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:05:53.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:05:53 smithi167 ceph-mon[37140]: pgmap v3096: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:05:53.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:05:53 smithi138 ceph-mon[36954]: pgmap v3096: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:05:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:05:53 smithi100 ceph-mon[32599]: pgmap v3096: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:05:55.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:05:55 smithi167 ceph-mon[37140]: pgmap v3097: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:05:55.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:05:55 smithi138 ceph-mon[36954]: pgmap v3097: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:05:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:05:55 smithi100 ceph-mon[32599]: pgmap v3097: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:05:57.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:05:57 smithi167 ceph-mon[37140]: pgmap v3098: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:05:57.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:05:57 smithi138 ceph-mon[36954]: pgmap v3098: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:05:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:05:57 smithi100 ceph-mon[32599]: pgmap v3098: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:05:59.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:05:59 smithi138 ceph-mon[36954]: pgmap v3099: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:05:59.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:05:59 smithi100 ceph-mon[32599]: pgmap v3099: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:05:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:05:59 smithi167 ceph-mon[37140]: pgmap v3099: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:06:01.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:06:01 smithi100 ceph-mon[32599]: pgmap v3100: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:06:01.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:06:01 smithi138 ceph-mon[36954]: pgmap v3100: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:06:01.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:06:01 smithi167 ceph-mon[37140]: pgmap v3100: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:06:02.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:06:02.535+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:06:02.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:06:02.536+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:06:02.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:06:02.536+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:06:02.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:06:02.536+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:06:02.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:06:02.536+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:06:03.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:06:03.674+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T15:06:03.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:06:03.674+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T15:06:03.676 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:06:03 smithi100 ceph-mon[32599]: pgmap v3101: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:06:03.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:06:03.675+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 40 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T15:06:03.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:06:03.675+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 40 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T15:06:03.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:06:03 smithi138 ceph-mon[36954]: pgmap v3101: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:06:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:06:03 smithi167 ceph-mon[37140]: pgmap v3101: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:06:05.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:06:05 smithi138 ceph-mon[36954]: pgmap v3102: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:06:05.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:06:05 smithi100 ceph-mon[32599]: pgmap v3102: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:06:05.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:06:05 smithi167 ceph-mon[37140]: pgmap v3102: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:06:07.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:06:07 smithi138 ceph-mon[36954]: pgmap v3103: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:06:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:06:07 smithi100 ceph-mon[32599]: pgmap v3103: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:06:07.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:06:07 smithi167 ceph-mon[37140]: pgmap v3103: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:06:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:06:08 smithi138 ceph-mon[36954]: osdmap e242: 12 total, 12 up, 12 in 2024-07-26T15:06:08.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:06:08 smithi100 ceph-mon[32599]: osdmap e242: 12 total, 12 up, 12 in 2024-07-26T15:06:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:06:08 smithi167 ceph-mon[37140]: osdmap e242: 12 total, 12 up, 12 in 2024-07-26T15:06:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:06:09 smithi138 ceph-mon[36954]: pgmap v3105: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:06:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:06:09 smithi100 ceph-mon[32599]: pgmap v3105: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:06:09.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:06:09 smithi167 ceph-mon[37140]: pgmap v3105: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:06:11.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:06:11 smithi138 ceph-mon[36954]: pgmap v3106: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:06:11.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:06:11 smithi100 ceph-mon[32599]: pgmap v3106: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:06:11.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:06:11 smithi167 ceph-mon[37140]: pgmap v3106: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:06:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:06:13 smithi138 ceph-mon[36954]: pgmap v3107: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:06:13.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:06:13 smithi100 ceph-mon[32599]: pgmap v3107: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:06:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:06:13 smithi167 ceph-mon[37140]: pgmap v3107: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:06:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:06:15 smithi138 ceph-mon[36954]: pgmap v3108: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:06:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:06:15 smithi138 ceph-mon[36954]: osdmap e243: 12 total, 12 up, 12 in 2024-07-26T15:06:15.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:06:15 smithi100 ceph-mon[32599]: pgmap v3108: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:06:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:06:15 smithi100 ceph-mon[32599]: osdmap e243: 12 total, 12 up, 12 in 2024-07-26T15:06:15.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:06:15 smithi167 ceph-mon[37140]: pgmap v3108: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:06:15.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:06:15 smithi167 ceph-mon[37140]: osdmap e243: 12 total, 12 up, 12 in 2024-07-26T15:06:16.647 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:06:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:06:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:06:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:06:16.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:06:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:06:17.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:06:17.551+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:06:17.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:06:17.551+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:06:17.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:06:17.551+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:06:17.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:06:17.551+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:06:17.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:06:17.552+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:06:17.591 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:06:17 smithi138 ceph-mon[36954]: pgmap v3110: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 232 B/s wr, 6 op/s 2024-07-26T15:06:17.645 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:06:17 smithi100 ceph-mon[32599]: pgmap v3110: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 232 B/s wr, 6 op/s 2024-07-26T15:06:17.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:06:17 smithi167 ceph-mon[37140]: pgmap v3110: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 232 B/s wr, 6 op/s 2024-07-26T15:06:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:06:19 smithi100 ceph-mon[32599]: pgmap v3111: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:06:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:06:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:06:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:06:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:06:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:06:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:06:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:06:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:06:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:06:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:06:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:06:19 smithi138 ceph-mon[36954]: pgmap v3111: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:06:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:06:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:06:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:06:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:06:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:06:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:06:19.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:06:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:06:19.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:06:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:06:19.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:06:19 smithi167 ceph-mon[37140]: pgmap v3111: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:06:19.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:06:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:06:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:06:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:06:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:06:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:06:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:06:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:06:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:06:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:06:21.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:06:21 smithi138 ceph-mon[36954]: pgmap v3112: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:06:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:06:21 smithi100 ceph-mon[32599]: pgmap v3112: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:06:21.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:06:21 smithi167 ceph-mon[37140]: pgmap v3112: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:06:23.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:06:23 smithi138 ceph-mon[36954]: pgmap v3113: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:06:23.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:06:23 smithi100 ceph-mon[32599]: pgmap v3113: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:06:23.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:06:23 smithi167 ceph-mon[37140]: pgmap v3113: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:06:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:06:25 smithi138 ceph-mon[36954]: pgmap v3114: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:06:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:06:25 smithi100 ceph-mon[32599]: pgmap v3114: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:06:25.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:06:25 smithi167 ceph-mon[37140]: pgmap v3114: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:06:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:06:27 smithi138 ceph-mon[36954]: pgmap v3115: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:06:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:06:27 smithi100 ceph-mon[32599]: pgmap v3115: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:06:27.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:06:27 smithi167 ceph-mon[37140]: pgmap v3115: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:06:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:06:29 smithi138 ceph-mon[36954]: pgmap v3116: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:06:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:06:29 smithi100 ceph-mon[32599]: pgmap v3116: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:06:29.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:06:29 smithi167 ceph-mon[37140]: pgmap v3116: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:06:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:06:31 smithi138 ceph-mon[36954]: pgmap v3117: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:06:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:06:31 smithi100 ceph-mon[32599]: pgmap v3117: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:06:31.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:06:31 smithi167 ceph-mon[37140]: pgmap v3117: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:06:32.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:06:32.562+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:06:32.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:06:32.562+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:06:32.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:06:32.562+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:06:32.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:06:32.562+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:06:32.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:06:32.563+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:06:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:06:33 smithi138 ceph-mon[36954]: pgmap v3118: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:06:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:06:33 smithi100 ceph-mon[32599]: pgmap v3118: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:06:33.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:06:33 smithi167 ceph-mon[37140]: pgmap v3118: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:06:35.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:06:35 smithi138 ceph-mon[36954]: pgmap v3119: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:06:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:06:35 smithi100 ceph-mon[32599]: pgmap v3119: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:06:35.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:06:35 smithi167 ceph-mon[37140]: pgmap v3119: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:06:36.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:06:36 smithi138 ceph-mon[36954]: pgmap v3120: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:06:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:06:36 smithi100 ceph-mon[32599]: pgmap v3120: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:06:36.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:06:36 smithi167 ceph-mon[37140]: pgmap v3120: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:06:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:06:39 smithi138 ceph-mon[36954]: pgmap v3121: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:06:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:06:39 smithi100 ceph-mon[32599]: pgmap v3121: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:06:39.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:06:39 smithi167 ceph-mon[37140]: pgmap v3121: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:06:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:06:41 smithi138 ceph-mon[36954]: pgmap v3122: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:06:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:06:41 smithi100 ceph-mon[32599]: pgmap v3122: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:06:41.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:06:41 smithi167 ceph-mon[37140]: pgmap v3122: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:06:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:06:43 smithi138 ceph-mon[36954]: pgmap v3123: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:06:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:06:43 smithi100 ceph-mon[32599]: pgmap v3123: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:06:43.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:06:43 smithi167 ceph-mon[37140]: pgmap v3123: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:06:45.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:06:45 smithi138 ceph-mon[36954]: pgmap v3124: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:06:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:06:45 smithi100 ceph-mon[32599]: pgmap v3124: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:06:45.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:06:45 smithi167 ceph-mon[37140]: pgmap v3124: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:06:47.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:06:47 smithi138 ceph-mon[36954]: pgmap v3125: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:06:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:06:47 smithi100 ceph-mon[32599]: pgmap v3125: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:06:47.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:06:47 smithi167 ceph-mon[37140]: pgmap v3125: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:06:47.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:06:47.570+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:06:47.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:06:47.570+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:06:47.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:06:47.570+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:06:47.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:06:47.570+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:06:47.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:06:47.570+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:06:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:06:49 smithi100 ceph-mon[32599]: pgmap v3126: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:06:49.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:06:49 smithi167 ceph-mon[37140]: pgmap v3126: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:06:49.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:06:49 smithi138 ceph-mon[36954]: pgmap v3126: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:06:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:06:51 smithi100 ceph-mon[32599]: pgmap v3127: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:06:51.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:06:51 smithi167 ceph-mon[37140]: pgmap v3127: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:06:51.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:06:51 smithi138 ceph-mon[36954]: pgmap v3127: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:06:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:06:53 smithi100 ceph-mon[32599]: pgmap v3128: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:06:53.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:06:53 smithi167 ceph-mon[37140]: pgmap v3128: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:06:53.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:06:53 smithi138 ceph-mon[36954]: pgmap v3128: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:06:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:06:55 smithi100 ceph-mon[32599]: pgmap v3129: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:06:55.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:06:55 smithi167 ceph-mon[37140]: pgmap v3129: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:06:55.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:06:55 smithi138 ceph-mon[36954]: pgmap v3129: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:06:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:06:57 smithi100 ceph-mon[32599]: pgmap v3130: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:06:57.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:06:57 smithi167 ceph-mon[37140]: pgmap v3130: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:06:57.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:06:57 smithi138 ceph-mon[36954]: pgmap v3130: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:06:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:06:59 smithi100 ceph-mon[32599]: pgmap v3131: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:06:59.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:06:59 smithi167 ceph-mon[37140]: pgmap v3131: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:06:59.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:06:59 smithi138 ceph-mon[36954]: pgmap v3131: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:07:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:01 smithi100 ceph-mon[32599]: pgmap v3132: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:07:01.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:01 smithi167 ceph-mon[37140]: pgmap v3132: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:07:01.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:01 smithi138 ceph-mon[36954]: pgmap v3132: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:07:02.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:07:02.579+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:07:02.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:07:02.579+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:07:02.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:07:02.579+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:07:02.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:07:02.579+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:07:02.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:07:02.579+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:07:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:03 smithi100 ceph-mon[32599]: pgmap v3133: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:07:03.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:03 smithi167 ceph-mon[37140]: pgmap v3133: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:07:03.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:03 smithi138 ceph-mon[36954]: pgmap v3133: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:07:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:05 smithi100 ceph-mon[32599]: pgmap v3134: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:07:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:05 smithi167 ceph-mon[37140]: pgmap v3134: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:07:05.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:05 smithi138 ceph-mon[36954]: pgmap v3134: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:07:07.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:07 smithi100 ceph-mon[32599]: pgmap v3135: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:07:07.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:07 smithi167 ceph-mon[37140]: pgmap v3135: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:07:07.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:07 smithi138 ceph-mon[36954]: pgmap v3135: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:07:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:08 smithi100 ceph-mon[32599]: osdmap e244: 12 total, 12 up, 12 in 2024-07-26T15:07:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:08 smithi167 ceph-mon[37140]: osdmap e244: 12 total, 12 up, 12 in 2024-07-26T15:07:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:08 smithi138 ceph-mon[36954]: osdmap e244: 12 total, 12 up, 12 in 2024-07-26T15:07:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:09 smithi100 ceph-mon[32599]: pgmap v3137: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:07:09.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:09 smithi167 ceph-mon[37140]: pgmap v3137: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:07:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:09 smithi138 ceph-mon[36954]: pgmap v3137: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:07:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:11 smithi100 ceph-mon[32599]: pgmap v3138: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:07:11.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:11 smithi167 ceph-mon[37140]: pgmap v3138: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:07:11.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:11 smithi138 ceph-mon[36954]: pgmap v3138: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:07:13.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:13 smithi167 ceph-mon[37140]: pgmap v3139: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:07:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:13 smithi138 ceph-mon[36954]: pgmap v3139: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:07:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:13 smithi100 ceph-mon[32599]: pgmap v3139: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:07:15.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:15 smithi167 ceph-mon[37140]: pgmap v3140: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:07:15.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:15 smithi167 ceph-mon[37140]: osdmap e245: 12 total, 12 up, 12 in 2024-07-26T15:07:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:15 smithi138 ceph-mon[36954]: pgmap v3140: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:07:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:15 smithi138 ceph-mon[36954]: osdmap e245: 12 total, 12 up, 12 in 2024-07-26T15:07:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:15 smithi100 ceph-mon[32599]: pgmap v3140: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:07:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:15 smithi100 ceph-mon[32599]: osdmap e245: 12 total, 12 up, 12 in 2024-07-26T15:07:17.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:17 smithi167 ceph-mon[37140]: pgmap v3142: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T15:07:17.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:07:17.596+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:07:17.598 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:17 smithi100 ceph-mon[32599]: pgmap v3142: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T15:07:17.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:07:17.597+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:07:17.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:07:17.597+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:07:17.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:07:17.597+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:07:17.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:07:17.597+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:07:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:17 smithi138 ceph-mon[36954]: pgmap v3142: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T15:07:19.526 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:19 smithi167 ceph-mon[37140]: pgmap v3143: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:07:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:19 smithi138 ceph-mon[36954]: pgmap v3143: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:07:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:19 smithi100 ceph-mon[32599]: pgmap v3143: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:07:20.410 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:20 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:07:20.410 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:20 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:07:20.410 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:20 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:07:20.411 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:20 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:07:20.411 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:20 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:07:20.416 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:20 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:07:20.417 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:20 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:07:20.417 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:20 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:07:20.417 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:20 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:07:20.417 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:20 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:07:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:20 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:07:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:20 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:07:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:20 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:07:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:20 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:07:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:20 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:07:21.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:21 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T15:07:21.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:21 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T15:07:21.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:21 smithi138 ceph-mon[36954]: pgmap v3144: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:07:21.445 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:21 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T15:07:21.446 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:21 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T15:07:21.446 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:21 smithi167 ceph-mon[37140]: pgmap v3144: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:07:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:21 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T15:07:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:21 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T15:07:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:21 smithi100 ceph-mon[32599]: pgmap v3144: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:07:23.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:23 smithi167 ceph-mon[37140]: pgmap v3145: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:07:23.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:07:23.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:07:23.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:07:23.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:07:23.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:07:23.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:07:23.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:07:23.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:07:23.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:07:23.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:07:23.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:07:23.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:07:23.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:07:23.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:07:23.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:07:23.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:23 smithi138 ceph-mon[36954]: pgmap v3145: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:07:23.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:07:23.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:07:23.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:07:23.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:07:23.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:07:23.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:07:23.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:07:23.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:07:23.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:07:23.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:07:23.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:07:23.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:07:23.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:07:23.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:07:23.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:07:23.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:23 smithi100 ceph-mon[32599]: pgmap v3145: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:07:23.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:07:23.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:07:23.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:07:23.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:07:23.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:07:23.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:07:23.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:07:23.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:07:23.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:07:23.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:07:23.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:07:23.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:07:23.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:07:23.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:07:23.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:07:24.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:24 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T15:07:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:24 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T15:07:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:24 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T15:07:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:24 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T15:07:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:24 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T15:07:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:24 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T15:07:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:24 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T15:07:24.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:24 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T15:07:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:24 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T15:07:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:24 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T15:07:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:24 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T15:07:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:24 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T15:07:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:25 smithi138 ceph-mon[36954]: pgmap v3146: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:07:25.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:25 smithi100 ceph-mon[32599]: pgmap v3146: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:07:25.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:25 smithi167 ceph-mon[37140]: pgmap v3146: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:07:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:27 smithi138 ceph-mon[36954]: pgmap v3147: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:07:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:27 smithi100 ceph-mon[32599]: pgmap v3147: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:07:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:27 smithi167 ceph-mon[37140]: pgmap v3147: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:07:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:29 smithi138 ceph-mon[36954]: pgmap v3148: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:07:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:29 smithi100 ceph-mon[32599]: pgmap v3148: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:07:29.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:29 smithi167 ceph-mon[37140]: pgmap v3148: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:07:31.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:31 smithi100 ceph-mon[32599]: pgmap v3149: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:07:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:31 smithi138 ceph-mon[36954]: pgmap v3149: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:07:31.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:31 smithi167 ceph-mon[37140]: pgmap v3149: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:07:32.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:07:32.612+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:07:32.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:07:32.613+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:07:32.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:07:32.613+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:07:32.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:07:32.613+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:07:32.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:07:32.613+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:07:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:33 smithi138 ceph-mon[36954]: pgmap v3150: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:07:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:33 smithi100 ceph-mon[32599]: pgmap v3150: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:07:33.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:33 smithi167 ceph-mon[37140]: pgmap v3150: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:07:35.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:35 smithi138 ceph-mon[36954]: pgmap v3151: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:07:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:35 smithi100 ceph-mon[32599]: pgmap v3151: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:07:35.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:35 smithi167 ceph-mon[37140]: pgmap v3151: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:07:37.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:37 smithi138 ceph-mon[36954]: pgmap v3152: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:07:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:37 smithi100 ceph-mon[32599]: pgmap v3152: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:07:37.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:37 smithi167 ceph-mon[37140]: pgmap v3152: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:07:39.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:39 smithi138 ceph-mon[36954]: pgmap v3153: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:07:39.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:39 smithi100 ceph-mon[32599]: pgmap v3153: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:07:39.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:39 smithi167 ceph-mon[37140]: pgmap v3153: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:07:41.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:41 smithi138 ceph-mon[36954]: pgmap v3154: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:07:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:41 smithi100 ceph-mon[32599]: pgmap v3154: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:07:41.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:41 smithi167 ceph-mon[37140]: pgmap v3154: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:07:43.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:43 smithi138 ceph-mon[36954]: pgmap v3155: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:07:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:43 smithi100 ceph-mon[32599]: pgmap v3155: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:07:43.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:43 smithi167 ceph-mon[37140]: pgmap v3155: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:07:45.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:45 smithi138 ceph-mon[36954]: pgmap v3156: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:07:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:45 smithi100 ceph-mon[32599]: pgmap v3156: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:07:45.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:45 smithi167 ceph-mon[37140]: pgmap v3156: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:07:47.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:07:47.628+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:07:47.630 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:47 smithi100 ceph-mon[32599]: pgmap v3157: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:07:47.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:07:47.629+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:07:47.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:07:47.629+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:07:47.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:07:47.629+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:07:47.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:07:47.629+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:07:47.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:47 smithi138 ceph-mon[36954]: pgmap v3157: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:07:47.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:47 smithi167 ceph-mon[37140]: pgmap v3157: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:07:49.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:49 smithi138 ceph-mon[36954]: pgmap v3158: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:07:49.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:49 smithi100 ceph-mon[32599]: pgmap v3158: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:07:49.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:49 smithi167 ceph-mon[37140]: pgmap v3158: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:07:51.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:51 smithi138 ceph-mon[36954]: pgmap v3159: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:07:51.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:51 smithi100 ceph-mon[32599]: pgmap v3159: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:07:51.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:51 smithi167 ceph-mon[37140]: pgmap v3159: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:07:53.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:53 smithi138 ceph-mon[36954]: pgmap v3160: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:07:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:53 smithi100 ceph-mon[32599]: pgmap v3160: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:07:53.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:53 smithi167 ceph-mon[37140]: pgmap v3160: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:07:55.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:55 smithi138 ceph-mon[36954]: pgmap v3161: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:07:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:55 smithi100 ceph-mon[32599]: pgmap v3161: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:07:55.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:55 smithi167 ceph-mon[37140]: pgmap v3161: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:07:57.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:57 smithi138 ceph-mon[36954]: pgmap v3162: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:07:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:57 smithi100 ceph-mon[32599]: pgmap v3162: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:07:57.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:57 smithi167 ceph-mon[37140]: pgmap v3162: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:07:59.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:07:59 smithi138 ceph-mon[36954]: pgmap v3163: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:07:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:07:59 smithi100 ceph-mon[32599]: pgmap v3163: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:07:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:07:59 smithi167 ceph-mon[37140]: pgmap v3163: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:08:01.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:08:01 smithi138 ceph-mon[36954]: pgmap v3164: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:08:01.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:08:01 smithi100 ceph-mon[32599]: pgmap v3164: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:08:01.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:08:01 smithi167 ceph-mon[37140]: pgmap v3164: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:08:02.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:08:02.633+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:08:02.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:08:02.634+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:08:02.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:08:02.634+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:08:02.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:08:02.634+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:08:02.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:08:02.634+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:08:03.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:08:03 smithi138 ceph-mon[36954]: pgmap v3165: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:08:03.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:08:03 smithi100 ceph-mon[32599]: pgmap v3165: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:08:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:08:03 smithi167 ceph-mon[37140]: pgmap v3165: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:08:05.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:08:05 smithi138 ceph-mon[36954]: pgmap v3166: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:08:05.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:08:05 smithi100 ceph-mon[32599]: pgmap v3166: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:08:05.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:08:05 smithi167 ceph-mon[37140]: pgmap v3166: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:08:07.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:08:07 smithi138 ceph-mon[36954]: pgmap v3167: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:08:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:08:07 smithi100 ceph-mon[32599]: pgmap v3167: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:08:07.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:08:07 smithi167 ceph-mon[37140]: pgmap v3167: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:08:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:08:08 smithi138 ceph-mon[36954]: osdmap e246: 12 total, 12 up, 12 in 2024-07-26T15:08:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:08:08 smithi100 ceph-mon[32599]: osdmap e246: 12 total, 12 up, 12 in 2024-07-26T15:08:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:08:08 smithi167 ceph-mon[37140]: osdmap e246: 12 total, 12 up, 12 in 2024-07-26T15:08:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:08:09 smithi138 ceph-mon[36954]: pgmap v3169: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:08:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:08:09 smithi100 ceph-mon[32599]: pgmap v3169: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:08:09.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:08:09 smithi167 ceph-mon[37140]: pgmap v3169: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:08:11.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:08:11 smithi138 ceph-mon[36954]: pgmap v3170: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:08:11.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:08:11 smithi100 ceph-mon[32599]: pgmap v3170: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:08:11.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:08:11 smithi167 ceph-mon[37140]: pgmap v3170: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:08:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:08:13 smithi138 ceph-mon[36954]: pgmap v3171: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:08:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:08:13 smithi100 ceph-mon[32599]: pgmap v3171: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:08:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:08:13 smithi167 ceph-mon[37140]: pgmap v3171: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:08:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:08:15 smithi138 ceph-mon[36954]: pgmap v3172: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:08:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:08:15 smithi138 ceph-mon[36954]: osdmap e247: 12 total, 12 up, 12 in 2024-07-26T15:08:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:08:15 smithi100 ceph-mon[32599]: pgmap v3172: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:08:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:08:15 smithi100 ceph-mon[32599]: osdmap e247: 12 total, 12 up, 12 in 2024-07-26T15:08:15.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:08:15 smithi167 ceph-mon[37140]: pgmap v3172: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:08:15.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:08:15 smithi167 ceph-mon[37140]: osdmap e247: 12 total, 12 up, 12 in 2024-07-26T15:08:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:08:16 smithi138 ceph-mon[36954]: pgmap v3174: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 232 B/s wr, 6 op/s 2024-07-26T15:08:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:08:16 smithi100 ceph-mon[32599]: pgmap v3174: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 232 B/s wr, 6 op/s 2024-07-26T15:08:16.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:08:16 smithi167 ceph-mon[37140]: pgmap v3174: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 232 B/s wr, 6 op/s 2024-07-26T15:08:17.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:08:17.651+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:08:17.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:08:17.654+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:08:17.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:08:17.654+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:08:17.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:08:17.654+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:08:17.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:08:17.654+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:08:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:08:19 smithi100 ceph-mon[32599]: pgmap v3175: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:08:19.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:08:19 smithi167 ceph-mon[37140]: pgmap v3175: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:08:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:08:19 smithi138 ceph-mon[36954]: pgmap v3175: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:08:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:08:21 smithi100 ceph-mon[32599]: pgmap v3176: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:08:21.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:08:21 smithi167 ceph-mon[37140]: pgmap v3176: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:08:21.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:08:21 smithi138 ceph-mon[36954]: pgmap v3176: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:08:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:08:23 smithi100 ceph-mon[32599]: pgmap v3177: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:08:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:08:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:08:23.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:08:23 smithi167 ceph-mon[37140]: pgmap v3177: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:08:23.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:08:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:08:23.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:08:23 smithi138 ceph-mon[36954]: pgmap v3177: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:08:23.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:08:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:08:24.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:08:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:08:24.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:08:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:08:24.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:08:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:08:24.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:08:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:08:24.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:08:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:08:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:08:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:08:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:08:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:08:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:08:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:08:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:08:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:08:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:08:25 smithi100 ceph-mon[32599]: pgmap v3178: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:08:25.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:08:25 smithi167 ceph-mon[37140]: pgmap v3178: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:08:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:08:25 smithi138 ceph-mon[36954]: pgmap v3178: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:08:27.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:08:27 smithi100 ceph-mon[32599]: pgmap v3179: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:08:27.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:08:27 smithi167 ceph-mon[37140]: pgmap v3179: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:08:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:08:27 smithi138 ceph-mon[36954]: pgmap v3179: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:08:29.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:08:29 smithi100 ceph-mon[32599]: pgmap v3180: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:08:29.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:08:29 smithi167 ceph-mon[37140]: pgmap v3180: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:08:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:08:29 smithi138 ceph-mon[36954]: pgmap v3180: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:08:31.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:08:31 smithi167 ceph-mon[37140]: pgmap v3181: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:08:31.296 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:08:31 smithi100 ceph-mon[32599]: pgmap v3181: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:08:31.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:08:31 smithi138 ceph-mon[36954]: pgmap v3181: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:08:32.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:08:32.670+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:08:32.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:08:32.671+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:08:32.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:08:32.671+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:08:32.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:08:32.671+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:08:32.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:08:32.671+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:08:33.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:08:33 smithi100 ceph-mon[32599]: pgmap v3182: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:08:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:08:33 smithi167 ceph-mon[37140]: pgmap v3182: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:08:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:08:33 smithi138 ceph-mon[36954]: pgmap v3182: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:08:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:08:35 smithi100 ceph-mon[32599]: pgmap v3183: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:08:35.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:08:35 smithi167 ceph-mon[37140]: pgmap v3183: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:08:35.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:08:35 smithi138 ceph-mon[36954]: pgmap v3183: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:08:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:08:37 smithi100 ceph-mon[32599]: pgmap v3184: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:08:37.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:08:37 smithi167 ceph-mon[37140]: pgmap v3184: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:08:37.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:08:37 smithi138 ceph-mon[36954]: pgmap v3184: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:08:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:08:39 smithi100 ceph-mon[32599]: pgmap v3185: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:08:39.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:08:39 smithi167 ceph-mon[37140]: pgmap v3185: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:08:39.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:08:39 smithi138 ceph-mon[36954]: pgmap v3185: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:08:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:08:41 smithi100 ceph-mon[32599]: pgmap v3186: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:08:41.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:08:41 smithi167 ceph-mon[37140]: pgmap v3186: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:08:41.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:08:41 smithi138 ceph-mon[36954]: pgmap v3186: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:08:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:08:43 smithi100 ceph-mon[32599]: pgmap v3187: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:08:43.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:08:43 smithi167 ceph-mon[37140]: pgmap v3187: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:08:43.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:08:43 smithi138 ceph-mon[36954]: pgmap v3187: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:08:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:08:45 smithi100 ceph-mon[32599]: pgmap v3188: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:08:45.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:08:45 smithi167 ceph-mon[37140]: pgmap v3188: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:08:45.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:08:45 smithi138 ceph-mon[36954]: pgmap v3188: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:08:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:08:47 smithi100 ceph-mon[32599]: pgmap v3189: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:08:47.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:08:47 smithi167 ceph-mon[37140]: pgmap v3189: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:08:47.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:08:47.687+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:08:47.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:08:47.687+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:08:47.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:08:47.687+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:08:47.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:08:47.687+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:08:47.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:08:47.687+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:08:47.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:08:47 smithi138 ceph-mon[36954]: pgmap v3189: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:08:49.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:08:49 smithi167 ceph-mon[37140]: pgmap v3190: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:08:49.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:08:49 smithi138 ceph-mon[36954]: pgmap v3190: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:08:49.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:08:49 smithi100 ceph-mon[32599]: pgmap v3190: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:08:51.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:08:51 smithi167 ceph-mon[37140]: pgmap v3191: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:08:51.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:08:51 smithi138 ceph-mon[36954]: pgmap v3191: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:08:51.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:08:51 smithi100 ceph-mon[32599]: pgmap v3191: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:08:53.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:08:53 smithi167 ceph-mon[37140]: pgmap v3192: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:08:53.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:08:53 smithi138 ceph-mon[36954]: pgmap v3192: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:08:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:08:53 smithi100 ceph-mon[32599]: pgmap v3192: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:08:55.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:08:55 smithi167 ceph-mon[37140]: pgmap v3193: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:08:55.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:08:55 smithi138 ceph-mon[36954]: pgmap v3193: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:08:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:08:55 smithi100 ceph-mon[32599]: pgmap v3193: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:08:57.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:08:57 smithi167 ceph-mon[37140]: pgmap v3194: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:08:57.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:08:57 smithi138 ceph-mon[36954]: pgmap v3194: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:08:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:08:57 smithi100 ceph-mon[32599]: pgmap v3194: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:08:59.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:08:59 smithi167 ceph-mon[37140]: pgmap v3195: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:08:59.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:08:59 smithi138 ceph-mon[36954]: pgmap v3195: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:08:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:08:59 smithi100 ceph-mon[32599]: pgmap v3195: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:09:01.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:09:01 smithi167 ceph-mon[37140]: pgmap v3196: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:09:01.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:09:01 smithi100 ceph-mon[32599]: pgmap v3196: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:09:01.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:09:01 smithi138 ceph-mon[36954]: pgmap v3196: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:09:02.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:09:02.703+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:09:02.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:09:02.703+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:09:02.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:09:02.703+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:09:02.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:09:02.703+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:09:02.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:09:02.703+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:09:03.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:09:03 smithi167 ceph-mon[37140]: pgmap v3197: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:09:03.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:09:03 smithi138 ceph-mon[36954]: pgmap v3197: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:09:03.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:09:03.700+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T15:09:03.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:09:03.700+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T15:09:03.701 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:09:03 smithi100 ceph-mon[32599]: pgmap v3197: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:09:03.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:09:03.701+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 41 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T15:09:03.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:09:03.701+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 41 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T15:09:05.694 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:09:05 smithi138 ceph-mon[36954]: pgmap v3198: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:09:05.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:09:05 smithi100 ceph-mon[32599]: pgmap v3198: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:09:05.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:09:05 smithi167 ceph-mon[37140]: pgmap v3198: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:09:07.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:09:07 smithi138 ceph-mon[36954]: pgmap v3199: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:09:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:09:07 smithi100 ceph-mon[32599]: pgmap v3199: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:09:07.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:09:07 smithi167 ceph-mon[37140]: pgmap v3199: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:09:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:09:08 smithi138 ceph-mon[36954]: osdmap e248: 12 total, 12 up, 12 in 2024-07-26T15:09:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:09:08 smithi100 ceph-mon[32599]: osdmap e248: 12 total, 12 up, 12 in 2024-07-26T15:09:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:09:08 smithi167 ceph-mon[37140]: osdmap e248: 12 total, 12 up, 12 in 2024-07-26T15:09:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:09:09 smithi138 ceph-mon[36954]: pgmap v3201: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:09:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:09:09 smithi100 ceph-mon[32599]: pgmap v3201: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:09:09.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:09:09 smithi167 ceph-mon[37140]: pgmap v3201: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:09:11.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:09:11 smithi138 ceph-mon[36954]: pgmap v3202: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:09:11.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:09:11 smithi100 ceph-mon[32599]: pgmap v3202: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:09:11.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:09:11 smithi167 ceph-mon[37140]: pgmap v3202: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:09:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:09:13 smithi138 ceph-mon[36954]: pgmap v3203: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:09:13.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:09:13 smithi100 ceph-mon[32599]: pgmap v3203: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:09:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:09:13 smithi167 ceph-mon[37140]: pgmap v3203: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:09:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:09:15 smithi138 ceph-mon[36954]: pgmap v3204: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:09:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:09:15 smithi138 ceph-mon[36954]: osdmap e249: 12 total, 12 up, 12 in 2024-07-26T15:09:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:09:15 smithi100 ceph-mon[32599]: pgmap v3204: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:09:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:09:15 smithi100 ceph-mon[32599]: osdmap e249: 12 total, 12 up, 12 in 2024-07-26T15:09:15.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:09:15 smithi167 ceph-mon[37140]: pgmap v3204: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:09:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:09:15 smithi167 ceph-mon[37140]: osdmap e249: 12 total, 12 up, 12 in 2024-07-26T15:09:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:09:17 smithi138 ceph-mon[36954]: pgmap v3206: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T15:09:17.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:09:17.719+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:09:17.721 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:09:17 smithi100 ceph-mon[32599]: pgmap v3206: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T15:09:17.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:09:17.719+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:09:17.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:09:17.719+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:09:17.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:09:17.719+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:09:17.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:09:17.719+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:09:17.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:09:17 smithi167 ceph-mon[37140]: pgmap v3206: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T15:09:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:09:19 smithi138 ceph-mon[36954]: pgmap v3207: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:09:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:09:19 smithi100 ceph-mon[32599]: pgmap v3207: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:09:19.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:09:19 smithi167 ceph-mon[37140]: pgmap v3207: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:09:21.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:09:21 smithi138 ceph-mon[36954]: pgmap v3208: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:09:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:09:21 smithi100 ceph-mon[32599]: pgmap v3208: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:09:21.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:09:21 smithi167 ceph-mon[37140]: pgmap v3208: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:09:23.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:09:23 smithi138 ceph-mon[36954]: pgmap v3209: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:09:23.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:09:23 smithi100 ceph-mon[32599]: pgmap v3209: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:09:23.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:09:23 smithi167 ceph-mon[37140]: pgmap v3209: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:09:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:09:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:09:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:09:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:09:24.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:09:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:09:24.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:09:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:09:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:09:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:09:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:09:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:09:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:09:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:09:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:09:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:09:24.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:09:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:09:24.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:09:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:09:24.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:09:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:09:24.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:09:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:09:25.674 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:09:25 smithi167 ceph-mon[37140]: pgmap v3210: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:09:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:09:25 smithi138 ceph-mon[36954]: pgmap v3210: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:09:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:09:25 smithi100 ceph-mon[32599]: pgmap v3210: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:09:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:09:27 smithi138 ceph-mon[36954]: pgmap v3211: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:09:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:09:27 smithi100 ceph-mon[32599]: pgmap v3211: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:09:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:09:27 smithi167 ceph-mon[37140]: pgmap v3211: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:09:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:09:29 smithi138 ceph-mon[36954]: pgmap v3212: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:09:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:09:29 smithi100 ceph-mon[32599]: pgmap v3212: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:09:29.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:09:29 smithi167 ceph-mon[37140]: pgmap v3212: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:09:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:09:31 smithi138 ceph-mon[36954]: pgmap v3213: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:09:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:09:31 smithi100 ceph-mon[32599]: pgmap v3213: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:09:31.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:09:31 smithi167 ceph-mon[37140]: pgmap v3213: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:09:32.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:09:32.734+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:09:32.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:09:32.734+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:09:32.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:09:32.734+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:09:32.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:09:32.734+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:09:32.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:09:32.734+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:09:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:09:33 smithi138 ceph-mon[36954]: pgmap v3214: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:09:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:09:33 smithi100 ceph-mon[32599]: pgmap v3214: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:09:33.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:09:33 smithi167 ceph-mon[37140]: pgmap v3214: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:09:35.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:09:35 smithi138 ceph-mon[36954]: pgmap v3215: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:09:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:09:35 smithi100 ceph-mon[32599]: pgmap v3215: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:09:35.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:09:35 smithi167 ceph-mon[37140]: pgmap v3215: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:09:37.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:09:37 smithi138 ceph-mon[36954]: pgmap v3216: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:09:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:09:37 smithi100 ceph-mon[32599]: pgmap v3216: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:09:37.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:09:37 smithi167 ceph-mon[37140]: pgmap v3216: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:09:39.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:09:39 smithi138 ceph-mon[36954]: pgmap v3217: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:09:39.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:09:39 smithi100 ceph-mon[32599]: pgmap v3217: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:09:39.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:09:39 smithi167 ceph-mon[37140]: pgmap v3217: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:09:41.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:09:41 smithi138 ceph-mon[36954]: pgmap v3218: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:09:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:09:41 smithi100 ceph-mon[32599]: pgmap v3218: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:09:41.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:09:41 smithi167 ceph-mon[37140]: pgmap v3218: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:09:43.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:09:43 smithi138 ceph-mon[36954]: pgmap v3219: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:09:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:09:43 smithi100 ceph-mon[32599]: pgmap v3219: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:09:43.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:09:43 smithi167 ceph-mon[37140]: pgmap v3219: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:09:45.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:09:45 smithi138 ceph-mon[36954]: pgmap v3220: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:09:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:09:45 smithi100 ceph-mon[32599]: pgmap v3220: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:09:45.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:09:45 smithi167 ceph-mon[37140]: pgmap v3220: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:09:47.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:09:47 smithi138 ceph-mon[36954]: pgmap v3221: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:09:47.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:09:47 smithi100 ceph-mon[32599]: pgmap v3221: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:09:47.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:09:47.750+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:09:47.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:09:47.751+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:09:47.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:09:47.751+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:09:47.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:09:47.751+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:09:47.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:09:47.751+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:09:47.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:09:47 smithi167 ceph-mon[37140]: pgmap v3221: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:09:49.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:09:49 smithi138 ceph-mon[36954]: pgmap v3222: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:09:49.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:09:49 smithi100 ceph-mon[32599]: pgmap v3222: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:09:49.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:09:49 smithi167 ceph-mon[37140]: pgmap v3222: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:09:51.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:09:51 smithi138 ceph-mon[36954]: pgmap v3223: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:09:51.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:09:51 smithi100 ceph-mon[32599]: pgmap v3223: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:09:51.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:09:51 smithi167 ceph-mon[37140]: pgmap v3223: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:09:53.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:09:53 smithi138 ceph-mon[36954]: pgmap v3224: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:09:53.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:09:53 smithi100 ceph-mon[32599]: pgmap v3224: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:09:53.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:09:53 smithi167 ceph-mon[37140]: pgmap v3224: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:09:55.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:09:55 smithi138 ceph-mon[36954]: pgmap v3225: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:09:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:09:55 smithi100 ceph-mon[32599]: pgmap v3225: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:09:55.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:09:55 smithi167 ceph-mon[37140]: pgmap v3225: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:09:57.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:09:57 smithi138 ceph-mon[36954]: pgmap v3226: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:09:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:09:57 smithi100 ceph-mon[32599]: pgmap v3226: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:09:57.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:09:57 smithi167 ceph-mon[37140]: pgmap v3226: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:09:59.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:09:59 smithi138 ceph-mon[36954]: pgmap v3227: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:09:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:09:59 smithi100 ceph-mon[32599]: pgmap v3227: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:09:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:09:59 smithi167 ceph-mon[37140]: pgmap v3227: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:10:00.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:00 smithi138 ceph-mon[36954]: overall HEALTH_OK 2024-07-26T15:10:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:00 smithi100 ceph-mon[32599]: overall HEALTH_OK 2024-07-26T15:10:00.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:00 smithi167 ceph-mon[37140]: overall HEALTH_OK 2024-07-26T15:10:01.645 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:01 smithi100 ceph-mon[32599]: pgmap v3228: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:10:01.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:01 smithi138 ceph-mon[36954]: pgmap v3228: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:10:01.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:01 smithi167 ceph-mon[37140]: pgmap v3228: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:10:02.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:10:02.759+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:10:02.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:10:02.760+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:10:02.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:10:02.760+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:10:02.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:10:02.760+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:10:02.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:10:02.760+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:10:03.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:03 smithi138 ceph-mon[36954]: pgmap v3229: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:10:03.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:03 smithi100 ceph-mon[32599]: pgmap v3229: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:10:03.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:03 smithi167 ceph-mon[37140]: pgmap v3229: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:10:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:04 smithi100 ceph-mon[32599]: pgmap v3230: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:10:04.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:04 smithi167 ceph-mon[37140]: pgmap v3230: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:10:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:04 smithi138 ceph-mon[36954]: pgmap v3230: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:10:07.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:07 smithi100 ceph-mon[32599]: pgmap v3231: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:10:07.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:07 smithi167 ceph-mon[37140]: pgmap v3231: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:10:07.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:07 smithi138 ceph-mon[36954]: pgmap v3231: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:10:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:08 smithi167 ceph-mon[37140]: osdmap e250: 12 total, 12 up, 12 in 2024-07-26T15:10:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:08 smithi138 ceph-mon[36954]: osdmap e250: 12 total, 12 up, 12 in 2024-07-26T15:10:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:08 smithi100 ceph-mon[32599]: osdmap e250: 12 total, 12 up, 12 in 2024-07-26T15:10:09.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:09 smithi167 ceph-mon[37140]: pgmap v3233: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:10:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:09 smithi138 ceph-mon[36954]: pgmap v3233: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:10:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:09 smithi100 ceph-mon[32599]: pgmap v3233: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:10:11.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:11 smithi167 ceph-mon[37140]: pgmap v3234: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:10:11.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:11 smithi138 ceph-mon[36954]: pgmap v3234: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:10:11.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:11 smithi100 ceph-mon[32599]: pgmap v3234: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:10:13.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:13 smithi167 ceph-mon[37140]: pgmap v3235: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:10:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:13 smithi138 ceph-mon[36954]: pgmap v3235: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:10:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:13 smithi100 ceph-mon[32599]: pgmap v3235: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:10:15.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:15 smithi167 ceph-mon[37140]: pgmap v3236: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:10:15.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:15 smithi167 ceph-mon[37140]: osdmap e251: 12 total, 12 up, 12 in 2024-07-26T15:10:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:15 smithi138 ceph-mon[36954]: pgmap v3236: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:10:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:15 smithi138 ceph-mon[36954]: osdmap e251: 12 total, 12 up, 12 in 2024-07-26T15:10:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:15 smithi100 ceph-mon[32599]: pgmap v3236: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:10:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:15 smithi100 ceph-mon[32599]: osdmap e251: 12 total, 12 up, 12 in 2024-07-26T15:10:17.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:17 smithi167 ceph-mon[37140]: pgmap v3238: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:10:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:17 smithi138 ceph-mon[36954]: pgmap v3238: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:10:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:17 smithi100 ceph-mon[32599]: pgmap v3238: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:10:17.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:10:17.775+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:10:17.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:10:17.776+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:10:17.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:10:17.776+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:10:17.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:10:17.776+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:10:17.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:10:17.776+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:10:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:19 smithi138 ceph-mon[36954]: pgmap v3239: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:10:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:19 smithi100 ceph-mon[32599]: pgmap v3239: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:10:19.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:19 smithi167 ceph-mon[37140]: pgmap v3239: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:10:21.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:21 smithi138 ceph-mon[36954]: pgmap v3240: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:10:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:21 smithi100 ceph-mon[32599]: pgmap v3240: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:10:21.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:21 smithi167 ceph-mon[37140]: pgmap v3240: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:10:23.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:23 smithi138 ceph-mon[36954]: pgmap v3241: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:10:23.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:23 smithi100 ceph-mon[32599]: pgmap v3241: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:10:23.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:23 smithi167 ceph-mon[37140]: pgmap v3241: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:10:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:10:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:10:24.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:10:25.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:25 smithi138 ceph-mon[36954]: pgmap v3242: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:10:25.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:25 smithi100 ceph-mon[32599]: pgmap v3242: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:10:25.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:25 smithi167 ceph-mon[37140]: pgmap v3242: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:10:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:10:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:10:26.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:10:26.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:10:26.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:10:26.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:10:26.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:10:26.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:10:26.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:10:26.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:26 smithi138 ceph-mon[36954]: pgmap v3243: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:10:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:10:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:10:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:10:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:10:26.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:10:26.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:10:26.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:10:26.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:10:26.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:10:26.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:26 smithi100 ceph-mon[32599]: pgmap v3243: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:10:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:10:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:10:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:10:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:10:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:10:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:10:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:10:27.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:10:27.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:10:27.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:26 smithi167 ceph-mon[37140]: pgmap v3243: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:10:29.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:29 smithi167 ceph-mon[37140]: pgmap v3244: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:10:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:29 smithi138 ceph-mon[36954]: pgmap v3244: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:10:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:29 smithi100 ceph-mon[32599]: pgmap v3244: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:10:31.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:31 smithi167 ceph-mon[37140]: pgmap v3245: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:10:31.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:31 smithi100 ceph-mon[32599]: pgmap v3245: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:10:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:31 smithi138 ceph-mon[36954]: pgmap v3245: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:10:32.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:10:32.791+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:10:32.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:10:32.792+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:10:32.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:10:32.792+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:10:32.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:10:32.792+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:10:32.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:10:32.792+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:10:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:33 smithi167 ceph-mon[37140]: pgmap v3246: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:10:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:33 smithi138 ceph-mon[36954]: pgmap v3246: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:10:33.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:10:33.713+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T15:10:33.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:10:33.713+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T15:10:33.714 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:33 smithi100 ceph-mon[32599]: pgmap v3246: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:10:33.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:10:33.713+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 42 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T15:10:33.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:10:33.713+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 42 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T15:10:35.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:35 smithi167 ceph-mon[37140]: pgmap v3247: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:10:35.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:35 smithi138 ceph-mon[36954]: pgmap v3247: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:10:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:35 smithi100 ceph-mon[32599]: pgmap v3247: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:10:37.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:37 smithi167 ceph-mon[37140]: pgmap v3248: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:10:37.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:37 smithi138 ceph-mon[36954]: pgmap v3248: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:10:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:37 smithi100 ceph-mon[32599]: pgmap v3248: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:10:39.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:39 smithi167 ceph-mon[37140]: pgmap v3249: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:10:39.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:39 smithi138 ceph-mon[36954]: pgmap v3249: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:10:39.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:39 smithi100 ceph-mon[32599]: pgmap v3249: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:10:41.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:41 smithi167 ceph-mon[37140]: pgmap v3250: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:10:41.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:41 smithi138 ceph-mon[36954]: pgmap v3250: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:10:41.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:41 smithi100 ceph-mon[32599]: pgmap v3250: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:10:43.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:43 smithi167 ceph-mon[37140]: pgmap v3251: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:10:43.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:43 smithi138 ceph-mon[36954]: pgmap v3251: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:10:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:43 smithi100 ceph-mon[32599]: pgmap v3251: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:10:45.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:45 smithi138 ceph-mon[36954]: pgmap v3252: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:10:45.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:45 smithi100 ceph-mon[32599]: pgmap v3252: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:10:45.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:45 smithi167 ceph-mon[37140]: pgmap v3252: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:10:47.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:47 smithi138 ceph-mon[36954]: pgmap v3253: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:10:47.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:47 smithi100 ceph-mon[32599]: pgmap v3253: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:10:47.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:47 smithi167 ceph-mon[37140]: pgmap v3253: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:10:47.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:10:47.799+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:10:47.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:10:47.800+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:10:47.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:10:47.800+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:10:47.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:10:47.800+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:10:47.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:10:47.800+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:10:49.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:49 smithi138 ceph-mon[36954]: pgmap v3254: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:10:49.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:49 smithi100 ceph-mon[32599]: pgmap v3254: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:10:49.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:49 smithi167 ceph-mon[37140]: pgmap v3254: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:10:51.689 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:51 smithi138 ceph-mon[36954]: pgmap v3255: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:10:51.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:51 smithi100 ceph-mon[32599]: pgmap v3255: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:10:51.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:51 smithi167 ceph-mon[37140]: pgmap v3255: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:10:53.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:53 smithi138 ceph-mon[36954]: pgmap v3256: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:10:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:53 smithi100 ceph-mon[32599]: pgmap v3256: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:10:53.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:53 smithi167 ceph-mon[37140]: pgmap v3256: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:10:55.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:55 smithi167 ceph-mon[37140]: pgmap v3257: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:10:55.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:55 smithi138 ceph-mon[36954]: pgmap v3257: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:10:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:55 smithi100 ceph-mon[32599]: pgmap v3257: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:10:57.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:57 smithi138 ceph-mon[36954]: pgmap v3258: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:10:57.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:57 smithi100 ceph-mon[32599]: pgmap v3258: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:10:57.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:57 smithi167 ceph-mon[37140]: pgmap v3258: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:10:59.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:10:59 smithi138 ceph-mon[36954]: pgmap v3259: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:10:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:10:59 smithi100 ceph-mon[32599]: pgmap v3259: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:10:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:10:59 smithi167 ceph-mon[37140]: pgmap v3259: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:11:01.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:01 smithi138 ceph-mon[36954]: pgmap v3260: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:11:01.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:01 smithi100 ceph-mon[32599]: pgmap v3260: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:11:01.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:01 smithi167 ceph-mon[37140]: pgmap v3260: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:11:02.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:11:02.815+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:11:02.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:11:02.816+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:11:02.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:11:02.816+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:11:02.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:11:02.816+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:11:02.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:11:02.816+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:11:03.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:03 smithi138 ceph-mon[36954]: pgmap v3261: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:11:03.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:03 smithi100 ceph-mon[32599]: pgmap v3261: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:11:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:03 smithi167 ceph-mon[37140]: pgmap v3261: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:11:05.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:05 smithi138 ceph-mon[36954]: pgmap v3262: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:11:05.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:05 smithi100 ceph-mon[32599]: pgmap v3262: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:11:05.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:05 smithi167 ceph-mon[37140]: pgmap v3262: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:11:07.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:07 smithi138 ceph-mon[36954]: pgmap v3263: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:11:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:07 smithi100 ceph-mon[32599]: pgmap v3263: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:11:07.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:07 smithi167 ceph-mon[37140]: pgmap v3263: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:11:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:08 smithi138 ceph-mon[36954]: osdmap e252: 12 total, 12 up, 12 in 2024-07-26T15:11:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:08 smithi100 ceph-mon[32599]: osdmap e252: 12 total, 12 up, 12 in 2024-07-26T15:11:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:08 smithi167 ceph-mon[37140]: osdmap e252: 12 total, 12 up, 12 in 2024-07-26T15:11:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:09 smithi138 ceph-mon[36954]: pgmap v3265: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:11:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:09 smithi100 ceph-mon[32599]: pgmap v3265: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:11:09.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:09 smithi167 ceph-mon[37140]: pgmap v3265: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:11:11.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:11 smithi138 ceph-mon[36954]: pgmap v3266: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:11:11.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:11 smithi100 ceph-mon[32599]: pgmap v3266: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:11:11.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:11 smithi167 ceph-mon[37140]: pgmap v3266: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:11:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:13 smithi138 ceph-mon[36954]: pgmap v3267: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:11:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:13 smithi100 ceph-mon[32599]: pgmap v3267: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:11:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:13 smithi167 ceph-mon[37140]: pgmap v3267: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:11:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:15 smithi138 ceph-mon[36954]: pgmap v3268: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:11:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:15 smithi138 ceph-mon[36954]: osdmap e253: 12 total, 12 up, 12 in 2024-07-26T15:11:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:15 smithi100 ceph-mon[32599]: pgmap v3268: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:11:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:15 smithi100 ceph-mon[32599]: osdmap e253: 12 total, 12 up, 12 in 2024-07-26T15:11:15.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:15 smithi167 ceph-mon[37140]: pgmap v3268: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:11:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:15 smithi167 ceph-mon[37140]: osdmap e253: 12 total, 12 up, 12 in 2024-07-26T15:11:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:17 smithi138 ceph-mon[36954]: pgmap v3270: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T15:11:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:17 smithi100 ceph-mon[32599]: pgmap v3270: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T15:11:17.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:17 smithi167 ceph-mon[37140]: pgmap v3270: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T15:11:17.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:11:17.831+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:11:17.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:11:17.832+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:11:17.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:11:17.832+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:11:17.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:11:17.832+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:11:17.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:11:17.832+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:11:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:19 smithi138 ceph-mon[36954]: pgmap v3271: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:11:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:19 smithi100 ceph-mon[32599]: pgmap v3271: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:11:19.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:19 smithi167 ceph-mon[37140]: pgmap v3271: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:11:21.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:21 smithi138 ceph-mon[36954]: pgmap v3272: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:11:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:21 smithi100 ceph-mon[32599]: pgmap v3272: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:11:21.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:21 smithi167 ceph-mon[37140]: pgmap v3272: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:11:23.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:23 smithi138 ceph-mon[36954]: pgmap v3273: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:11:23.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:23 smithi100 ceph-mon[32599]: pgmap v3273: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:11:23.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:23 smithi167 ceph-mon[37140]: pgmap v3273: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:11:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:25 smithi138 ceph-mon[36954]: pgmap v3274: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:11:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:25 smithi100 ceph-mon[32599]: pgmap v3274: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:11:25.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:25 smithi167 ceph-mon[37140]: pgmap v3274: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:11:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:11:26.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:11:26.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:11:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:27 smithi138 ceph-mon[36954]: pgmap v3275: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:11:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:11:27.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:11:27.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:11:27.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:11:27.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:11:27.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:11:27.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:11:27.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:11:27.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:11:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:27 smithi100 ceph-mon[32599]: pgmap v3275: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:11:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:11:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:11:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:11:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:11:27.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:11:27.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:11:27.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:11:27.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:11:27.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:11:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:27 smithi167 ceph-mon[37140]: pgmap v3275: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:11:27.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:11:27.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:11:27.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:11:27.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:11:27.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:11:27.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:11:27.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:11:27.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:11:27.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:11:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:29 smithi138 ceph-mon[36954]: pgmap v3276: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:11:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:29 smithi100 ceph-mon[32599]: pgmap v3276: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:11:29.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:29 smithi167 ceph-mon[37140]: pgmap v3276: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:11:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:31 smithi138 ceph-mon[36954]: pgmap v3277: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:11:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:31 smithi100 ceph-mon[32599]: pgmap v3277: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:11:31.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:31 smithi167 ceph-mon[37140]: pgmap v3277: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:11:32.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:11:32.847+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:11:32.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:11:32.848+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:11:32.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:11:32.848+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:11:32.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:11:32.848+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:11:32.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:11:32.848+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:11:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:33 smithi138 ceph-mon[36954]: pgmap v3278: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:11:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:33 smithi100 ceph-mon[32599]: pgmap v3278: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:11:33.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:33 smithi167 ceph-mon[37140]: pgmap v3278: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:11:35.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:35 smithi138 ceph-mon[36954]: pgmap v3279: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:11:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:35 smithi100 ceph-mon[32599]: pgmap v3279: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:11:35.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:35 smithi167 ceph-mon[37140]: pgmap v3279: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:11:37.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:37 smithi138 ceph-mon[36954]: pgmap v3280: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:11:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:37 smithi100 ceph-mon[32599]: pgmap v3280: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:11:37.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:37 smithi167 ceph-mon[37140]: pgmap v3280: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:11:39.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:39 smithi138 ceph-mon[36954]: pgmap v3281: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:11:39.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:39 smithi100 ceph-mon[32599]: pgmap v3281: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:11:39.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:39 smithi167 ceph-mon[37140]: pgmap v3281: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:11:41.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:41 smithi138 ceph-mon[36954]: pgmap v3282: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:11:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:41 smithi100 ceph-mon[32599]: pgmap v3282: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:11:41.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:41 smithi167 ceph-mon[37140]: pgmap v3282: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:11:43.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:43 smithi138 ceph-mon[36954]: pgmap v3283: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:11:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:43 smithi100 ceph-mon[32599]: pgmap v3283: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:11:43.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:43 smithi167 ceph-mon[37140]: pgmap v3283: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:11:45.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:45 smithi138 ceph-mon[36954]: pgmap v3284: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:11:45.760 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:45 smithi100 ceph-mon[32599]: pgmap v3284: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:11:45.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:45 smithi167 ceph-mon[37140]: pgmap v3284: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:11:47.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:47 smithi138 ceph-mon[36954]: pgmap v3285: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:11:47.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:47 smithi100 ceph-mon[32599]: pgmap v3285: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:11:47.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:47 smithi167 ceph-mon[37140]: pgmap v3285: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:11:47.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:11:47.863+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:11:47.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:11:47.864+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:11:47.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:11:47.864+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:11:47.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:11:47.864+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:11:47.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:11:47.864+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:11:49.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:49 smithi138 ceph-mon[36954]: pgmap v3286: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:11:49.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:49 smithi100 ceph-mon[32599]: pgmap v3286: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:11:49.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:49 smithi167 ceph-mon[37140]: pgmap v3286: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:11:51.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:51 smithi138 ceph-mon[36954]: pgmap v3287: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:11:51.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:51 smithi100 ceph-mon[32599]: pgmap v3287: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:11:51.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:51 smithi167 ceph-mon[37140]: pgmap v3287: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:11:53.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:53 smithi138 ceph-mon[36954]: pgmap v3288: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:11:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:53 smithi100 ceph-mon[32599]: pgmap v3288: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:11:53.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:53 smithi167 ceph-mon[37140]: pgmap v3288: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:11:55.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:55 smithi138 ceph-mon[36954]: pgmap v3289: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:11:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:55 smithi100 ceph-mon[32599]: pgmap v3289: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:11:55.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:55 smithi167 ceph-mon[37140]: pgmap v3289: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:11:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:56 smithi100 ceph-mon[32599]: pgmap v3290: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:11:56.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:56 smithi167 ceph-mon[37140]: pgmap v3290: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:11:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:56 smithi138 ceph-mon[36954]: pgmap v3290: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:11:59.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:11:59 smithi138 ceph-mon[36954]: pgmap v3291: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:11:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:11:59 smithi100 ceph-mon[32599]: pgmap v3291: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:11:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:11:59 smithi167 ceph-mon[37140]: pgmap v3291: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:12:01.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:12:01 smithi100 ceph-mon[32599]: pgmap v3292: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:12:01.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:12:01 smithi138 ceph-mon[36954]: pgmap v3292: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:12:01.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:12:01 smithi167 ceph-mon[37140]: pgmap v3292: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:12:02.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:12:02.880+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:12:02.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:12:02.880+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:12:02.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:12:02.880+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:12:02.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:12:02.880+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:12:02.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:12:02.880+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:12:03.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:12:03 smithi138 ceph-mon[36954]: pgmap v3293: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-07-26T15:12:03.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:12:03.725+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T15:12:03.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:12:03.725+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T15:12:03.727 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:12:03 smithi100 ceph-mon[32599]: pgmap v3293: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-07-26T15:12:03.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:12:03.725+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 43 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T15:12:03.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:12:03.726+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 43 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T15:12:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:12:03 smithi167 ceph-mon[37140]: pgmap v3293: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-07-26T15:12:05.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:12:05 smithi138 ceph-mon[36954]: pgmap v3294: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:12:05.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:12:05 smithi100 ceph-mon[32599]: pgmap v3294: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:12:05.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:12:05 smithi167 ceph-mon[37140]: pgmap v3294: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:12:07.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:12:07 smithi138 ceph-mon[36954]: pgmap v3295: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:12:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:12:07 smithi100 ceph-mon[32599]: pgmap v3295: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:12:07.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:12:07 smithi167 ceph-mon[37140]: pgmap v3295: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:12:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:12:08 smithi138 ceph-mon[36954]: osdmap e254: 12 total, 12 up, 12 in 2024-07-26T15:12:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:12:08 smithi100 ceph-mon[32599]: osdmap e254: 12 total, 12 up, 12 in 2024-07-26T15:12:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:12:08 smithi167 ceph-mon[37140]: osdmap e254: 12 total, 12 up, 12 in 2024-07-26T15:12:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:12:09 smithi138 ceph-mon[36954]: pgmap v3297: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:12:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:12:09 smithi100 ceph-mon[32599]: pgmap v3297: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:12:09.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:12:09 smithi167 ceph-mon[37140]: pgmap v3297: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:12:11.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:12:11 smithi138 ceph-mon[36954]: pgmap v3298: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:12:11.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:12:11 smithi100 ceph-mon[32599]: pgmap v3298: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:12:11.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:12:11 smithi167 ceph-mon[37140]: pgmap v3298: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:12:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:12:13 smithi138 ceph-mon[36954]: pgmap v3299: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:12:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:12:13 smithi100 ceph-mon[32599]: pgmap v3299: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:12:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:12:13 smithi167 ceph-mon[37140]: pgmap v3299: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:12:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:12:15 smithi138 ceph-mon[36954]: pgmap v3300: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:12:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:12:15 smithi138 ceph-mon[36954]: osdmap e255: 12 total, 12 up, 12 in 2024-07-26T15:12:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:12:15 smithi100 ceph-mon[32599]: pgmap v3300: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:12:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:12:15 smithi100 ceph-mon[32599]: osdmap e255: 12 total, 12 up, 12 in 2024-07-26T15:12:15.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:12:15 smithi167 ceph-mon[37140]: pgmap v3300: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:12:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:12:15 smithi167 ceph-mon[37140]: osdmap e255: 12 total, 12 up, 12 in 2024-07-26T15:12:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:12:17 smithi138 ceph-mon[36954]: pgmap v3302: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:12:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:12:17 smithi100 ceph-mon[32599]: pgmap v3302: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:12:17.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:12:17 smithi167 ceph-mon[37140]: pgmap v3302: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:12:17.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:12:17.896+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:12:17.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:12:17.897+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:12:17.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:12:17.897+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:12:17.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:12:17.897+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:12:17.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:12:17.897+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:12:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:12:19 smithi138 ceph-mon[36954]: pgmap v3303: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:12:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:12:19 smithi100 ceph-mon[32599]: pgmap v3303: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:12:19.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:12:19 smithi167 ceph-mon[37140]: pgmap v3303: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:12:21.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:12:21 smithi138 ceph-mon[36954]: pgmap v3304: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:12:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:12:21 smithi100 ceph-mon[32599]: pgmap v3304: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:12:21.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:12:21 smithi167 ceph-mon[37140]: pgmap v3304: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:12:23.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:12:23 smithi138 ceph-mon[36954]: pgmap v3305: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:12:23.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:12:23 smithi100 ceph-mon[32599]: pgmap v3305: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:12:23.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:12:23 smithi167 ceph-mon[37140]: pgmap v3305: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:12:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:12:25 smithi138 ceph-mon[36954]: pgmap v3306: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:12:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:12:25 smithi100 ceph-mon[32599]: pgmap v3306: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:12:25.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:12:25 smithi167 ceph-mon[37140]: pgmap v3306: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:12:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:12:27 smithi138 ceph-mon[36954]: pgmap v3307: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:12:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:12:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:12:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:12:27 smithi100 ceph-mon[32599]: pgmap v3307: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:12:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:12:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:12:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:12:27 smithi167 ceph-mon[37140]: pgmap v3307: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:12:27.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:12:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:12:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:12:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:12:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:12:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:12:28.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:12:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:12:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:12:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:12:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:12:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:12:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:12:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:12:28.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:12:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:12:28.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:12:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:12:28.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:12:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:12:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:12:29 smithi138 ceph-mon[36954]: pgmap v3308: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:12:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:12:29 smithi100 ceph-mon[32599]: pgmap v3308: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:12:29.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:12:29 smithi167 ceph-mon[37140]: pgmap v3308: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:12:31.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:12:31 smithi138 ceph-mon[36954]: pgmap v3309: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:12:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:12:31 smithi100 ceph-mon[32599]: pgmap v3309: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:12:31.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:12:31 smithi167 ceph-mon[37140]: pgmap v3309: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:12:32.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:12:32.912+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:12:32.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:12:32.913+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:12:32.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:12:32.913+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:12:32.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:12:32.913+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:12:32.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:12:32.913+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:12:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:12:33 smithi138 ceph-mon[36954]: pgmap v3310: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:12:33.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:12:33 smithi100 ceph-mon[32599]: pgmap v3310: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:12:33.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:12:33 smithi167 ceph-mon[37140]: pgmap v3310: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:12:35.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:12:35 smithi138 ceph-mon[36954]: pgmap v3311: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:12:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:12:35 smithi100 ceph-mon[32599]: pgmap v3311: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:12:35.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:12:35 smithi167 ceph-mon[37140]: pgmap v3311: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:12:37.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:12:37 smithi138 ceph-mon[36954]: pgmap v3312: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:12:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:12:37 smithi100 ceph-mon[32599]: pgmap v3312: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:12:37.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:12:37 smithi167 ceph-mon[37140]: pgmap v3312: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:12:39.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:12:39 smithi138 ceph-mon[36954]: pgmap v3313: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:12:39.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:12:39 smithi100 ceph-mon[32599]: pgmap v3313: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:12:39.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:12:39 smithi167 ceph-mon[37140]: pgmap v3313: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:12:41.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:12:41 smithi138 ceph-mon[36954]: pgmap v3314: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:12:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:12:41 smithi100 ceph-mon[32599]: pgmap v3314: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:12:41.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:12:41 smithi167 ceph-mon[37140]: pgmap v3314: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:12:43.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:12:43 smithi138 ceph-mon[36954]: pgmap v3315: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:12:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:12:43 smithi100 ceph-mon[32599]: pgmap v3315: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:12:43.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:12:43 smithi167 ceph-mon[37140]: pgmap v3315: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:12:45.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:12:45 smithi138 ceph-mon[36954]: pgmap v3316: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:12:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:12:45 smithi100 ceph-mon[32599]: pgmap v3316: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:12:45.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:12:45 smithi167 ceph-mon[37140]: pgmap v3316: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:12:47.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:12:47 smithi138 ceph-mon[36954]: pgmap v3317: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:12:47.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:12:47 smithi100 ceph-mon[32599]: pgmap v3317: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:12:47.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:12:47 smithi167 ceph-mon[37140]: pgmap v3317: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:12:47.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:12:47.919+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:12:47.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:12:47.920+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:12:47.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:12:47.920+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:12:47.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:12:47.920+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:12:47.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:12:47.920+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:12:49.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:12:49 smithi138 ceph-mon[36954]: pgmap v3318: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:12:49.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:12:49 smithi100 ceph-mon[32599]: pgmap v3318: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:12:49.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:12:49 smithi167 ceph-mon[37140]: pgmap v3318: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:12:51.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:12:51 smithi138 ceph-mon[36954]: pgmap v3319: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:12:51.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:12:51 smithi100 ceph-mon[32599]: pgmap v3319: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:12:51.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:12:51 smithi167 ceph-mon[37140]: pgmap v3319: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:12:53.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:12:53 smithi138 ceph-mon[36954]: pgmap v3320: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:12:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:12:53 smithi100 ceph-mon[32599]: pgmap v3320: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:12:53.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:12:53 smithi167 ceph-mon[37140]: pgmap v3320: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:12:55.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:12:55 smithi138 ceph-mon[36954]: pgmap v3321: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:12:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:12:55 smithi100 ceph-mon[32599]: pgmap v3321: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:12:55.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:12:55 smithi167 ceph-mon[37140]: pgmap v3321: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:12:57.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:12:57 smithi138 ceph-mon[36954]: pgmap v3322: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:12:57.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:12:57 smithi100 ceph-mon[32599]: pgmap v3322: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:12:57.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:12:57 smithi167 ceph-mon[37140]: pgmap v3322: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:12:59.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:12:59 smithi138 ceph-mon[36954]: pgmap v3323: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:12:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:12:59 smithi100 ceph-mon[32599]: pgmap v3323: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:12:59.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:12:59 smithi167 ceph-mon[37140]: pgmap v3323: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:13:01.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:13:01 smithi138 ceph-mon[36954]: pgmap v3324: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:13:01.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:13:01 smithi100 ceph-mon[32599]: pgmap v3324: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:13:01.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:13:01 smithi167 ceph-mon[37140]: pgmap v3324: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:13:02.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:13:02.935+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:13:02.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:13:02.935+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:13:02.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:13:02.935+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:13:02.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:13:02.935+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:13:02.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:13:02.935+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:13:03.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:13:03 smithi138 ceph-mon[36954]: pgmap v3325: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:13:03.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:13:03 smithi100 ceph-mon[32599]: pgmap v3325: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:13:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:13:03 smithi167 ceph-mon[37140]: pgmap v3325: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:13:05.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:13:05 smithi138 ceph-mon[36954]: pgmap v3326: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:13:05.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:13:05 smithi100 ceph-mon[32599]: pgmap v3326: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:13:05.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:13:05 smithi167 ceph-mon[37140]: pgmap v3326: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:13:07.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:13:07 smithi138 ceph-mon[36954]: pgmap v3327: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:13:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:13:07 smithi100 ceph-mon[32599]: pgmap v3327: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:13:07.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:13:07 smithi167 ceph-mon[37140]: pgmap v3327: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:13:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:13:08 smithi138 ceph-mon[36954]: osdmap e256: 12 total, 12 up, 12 in 2024-07-26T15:13:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:13:08 smithi100 ceph-mon[32599]: osdmap e256: 12 total, 12 up, 12 in 2024-07-26T15:13:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:13:08 smithi167 ceph-mon[37140]: osdmap e256: 12 total, 12 up, 12 in 2024-07-26T15:13:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:13:09 smithi138 ceph-mon[36954]: pgmap v3329: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:13:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:13:09 smithi100 ceph-mon[32599]: pgmap v3329: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:13:09.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:13:09 smithi167 ceph-mon[37140]: pgmap v3329: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:13:11.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:13:11 smithi138 ceph-mon[36954]: pgmap v3330: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:13:11.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:13:11 smithi100 ceph-mon[32599]: pgmap v3330: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:13:11.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:13:11 smithi167 ceph-mon[37140]: pgmap v3330: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:13:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:13:13 smithi138 ceph-mon[36954]: pgmap v3331: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:13:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:13:13 smithi100 ceph-mon[32599]: pgmap v3331: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:13:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:13:13 smithi167 ceph-mon[37140]: pgmap v3331: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:13:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:13:15 smithi138 ceph-mon[36954]: pgmap v3332: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:13:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:13:15 smithi138 ceph-mon[36954]: osdmap e257: 12 total, 12 up, 12 in 2024-07-26T15:13:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:13:15 smithi100 ceph-mon[32599]: pgmap v3332: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:13:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:13:15 smithi100 ceph-mon[32599]: osdmap e257: 12 total, 12 up, 12 in 2024-07-26T15:13:15.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:13:15 smithi167 ceph-mon[37140]: pgmap v3332: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:13:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:13:15 smithi167 ceph-mon[37140]: osdmap e257: 12 total, 12 up, 12 in 2024-07-26T15:13:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:13:17 smithi138 ceph-mon[36954]: pgmap v3334: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T15:13:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:13:17 smithi100 ceph-mon[32599]: pgmap v3334: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T15:13:17.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:13:17 smithi167 ceph-mon[37140]: pgmap v3334: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T15:13:17.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:13:17.951+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:13:17.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:13:17.952+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:13:17.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:13:17.952+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:13:17.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:13:17.952+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:13:17.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:13:17.952+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:13:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:13:19 smithi138 ceph-mon[36954]: pgmap v3335: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:13:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:13:19 smithi100 ceph-mon[32599]: pgmap v3335: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:13:19.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:13:19 smithi167 ceph-mon[37140]: pgmap v3335: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:13:21.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:13:21 smithi138 ceph-mon[36954]: pgmap v3336: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:13:21.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:13:21 smithi100 ceph-mon[32599]: pgmap v3336: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:13:21.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:13:21 smithi167 ceph-mon[37140]: pgmap v3336: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:13:23.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:13:23 smithi138 ceph-mon[36954]: pgmap v3337: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:13:23.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:13:23 smithi100 ceph-mon[32599]: pgmap v3337: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:13:23.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:13:23 smithi167 ceph-mon[37140]: pgmap v3337: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:13:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:13:25 smithi138 ceph-mon[36954]: pgmap v3338: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:13:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:13:25 smithi100 ceph-mon[32599]: pgmap v3338: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:13:25.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:13:25 smithi167 ceph-mon[37140]: pgmap v3338: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:13:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:13:27 smithi138 ceph-mon[36954]: pgmap v3339: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:13:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:13:27 smithi100 ceph-mon[32599]: pgmap v3339: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:13:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:13:27 smithi167 ceph-mon[37140]: pgmap v3339: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:13:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:13:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:13:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:13:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:13:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:13:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:13:28.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:13:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:13:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:13:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:13:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:13:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:13:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:13:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:13:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:13:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:13:28.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:13:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:13:28.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:13:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:13:28.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:13:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:13:28.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:13:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:13:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:13:29 smithi138 ceph-mon[36954]: pgmap v3340: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:13:29.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:13:29 smithi100 ceph-mon[32599]: pgmap v3340: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:13:29.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:13:29 smithi167 ceph-mon[37140]: pgmap v3340: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:13:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:13:31 smithi138 ceph-mon[36954]: pgmap v3341: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:13:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:13:31 smithi100 ceph-mon[32599]: pgmap v3341: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:13:31.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:13:31 smithi167 ceph-mon[37140]: pgmap v3341: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:13:32.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:13:32.963+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:13:32.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:13:32.964+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:13:32.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:13:32.964+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:13:32.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:13:32.964+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:13:32.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:13:32.964+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:13:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:13:33 smithi138 ceph-mon[36954]: pgmap v3342: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:13:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:13:33 smithi100 ceph-mon[32599]: pgmap v3342: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:13:33.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:13:33 smithi167 ceph-mon[37140]: pgmap v3342: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:13:35.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:13:35 smithi138 ceph-mon[36954]: pgmap v3343: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:13:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:13:35 smithi100 ceph-mon[32599]: pgmap v3343: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:13:35.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:13:35 smithi167 ceph-mon[37140]: pgmap v3343: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:13:37.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:13:37 smithi138 ceph-mon[36954]: pgmap v3344: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:13:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:13:37 smithi100 ceph-mon[32599]: pgmap v3344: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:13:37.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:13:37 smithi167 ceph-mon[37140]: pgmap v3344: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:13:39.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:13:39 smithi138 ceph-mon[36954]: pgmap v3345: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:13:39.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:13:39 smithi100 ceph-mon[32599]: pgmap v3345: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:13:39.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:13:39 smithi167 ceph-mon[37140]: pgmap v3345: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:13:41.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:13:41 smithi138 ceph-mon[36954]: pgmap v3346: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:13:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:13:41 smithi100 ceph-mon[32599]: pgmap v3346: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:13:41.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:13:41 smithi167 ceph-mon[37140]: pgmap v3346: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:13:43.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:13:43 smithi138 ceph-mon[36954]: pgmap v3347: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:13:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:13:43 smithi100 ceph-mon[32599]: pgmap v3347: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:13:43.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:13:43 smithi167 ceph-mon[37140]: pgmap v3347: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:13:45.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:13:45 smithi138 ceph-mon[36954]: pgmap v3348: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:13:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:13:45 smithi100 ceph-mon[32599]: pgmap v3348: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:13:45.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:13:45 smithi167 ceph-mon[37140]: pgmap v3348: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:13:46.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:13:46 smithi100 ceph-mon[32599]: pgmap v3349: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:13:46.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:13:46 smithi167 ceph-mon[37140]: pgmap v3349: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:13:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:13:46 smithi138 ceph-mon[36954]: pgmap v3349: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:13:47.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:13:47.980+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:13:47.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:13:47.980+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:13:47.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:13:47.980+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:13:47.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:13:47.980+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:13:47.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:13:47.980+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:13:49.643 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:13:49 smithi138 ceph-mon[36954]: pgmap v3350: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:13:49.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:13:49 smithi100 ceph-mon[32599]: pgmap v3350: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:13:49.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:13:49 smithi167 ceph-mon[37140]: pgmap v3350: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:13:51.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:13:51 smithi138 ceph-mon[36954]: pgmap v3351: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:13:51.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:13:51 smithi100 ceph-mon[32599]: pgmap v3351: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:13:51.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:13:51 smithi167 ceph-mon[37140]: pgmap v3351: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:13:53.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:13:53 smithi138 ceph-mon[36954]: pgmap v3352: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:13:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:13:53 smithi100 ceph-mon[32599]: pgmap v3352: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:13:53.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:13:53 smithi167 ceph-mon[37140]: pgmap v3352: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:13:55.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:13:55 smithi138 ceph-mon[36954]: pgmap v3353: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:13:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:13:55 smithi100 ceph-mon[32599]: pgmap v3353: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:13:55.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:13:55 smithi167 ceph-mon[37140]: pgmap v3353: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:13:57.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:13:57 smithi138 ceph-mon[36954]: pgmap v3354: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:13:57.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:13:57 smithi100 ceph-mon[32599]: pgmap v3354: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:13:57.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:13:57 smithi167 ceph-mon[37140]: pgmap v3354: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:13:59.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:13:59 smithi138 ceph-mon[36954]: pgmap v3355: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:13:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:13:59 smithi100 ceph-mon[32599]: pgmap v3355: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:13:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:13:59 smithi167 ceph-mon[37140]: pgmap v3355: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:14:01.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:14:01 smithi138 ceph-mon[36954]: pgmap v3356: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:14:01.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:14:01 smithi100 ceph-mon[32599]: pgmap v3356: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:14:01.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:14:01 smithi167 ceph-mon[37140]: pgmap v3356: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:14:02.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:14:02.993+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:14:02.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:14:02.994+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:14:02.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:14:02.994+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:14:02.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:14:02.994+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:14:02.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:14:02.994+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:14:03.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:14:03 smithi138 ceph-mon[36954]: pgmap v3357: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:14:03.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:14:03 smithi100 ceph-mon[32599]: pgmap v3357: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:14:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:14:03 smithi167 ceph-mon[37140]: pgmap v3357: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:14:05.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:14:05 smithi138 ceph-mon[36954]: pgmap v3358: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:14:05.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:14:05 smithi100 ceph-mon[32599]: pgmap v3358: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:14:05.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:14:05 smithi167 ceph-mon[37140]: pgmap v3358: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:14:07.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:14:07 smithi138 ceph-mon[36954]: pgmap v3359: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:14:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:14:07 smithi100 ceph-mon[32599]: pgmap v3359: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:14:07.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:14:07 smithi167 ceph-mon[37140]: pgmap v3359: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:14:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:14:08 smithi138 ceph-mon[36954]: osdmap e258: 12 total, 12 up, 12 in 2024-07-26T15:14:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:14:08 smithi100 ceph-mon[32599]: osdmap e258: 12 total, 12 up, 12 in 2024-07-26T15:14:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:14:08 smithi167 ceph-mon[37140]: osdmap e258: 12 total, 12 up, 12 in 2024-07-26T15:14:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:14:09 smithi138 ceph-mon[36954]: pgmap v3361: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:14:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:14:09 smithi100 ceph-mon[32599]: pgmap v3361: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:14:09.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:14:09 smithi167 ceph-mon[37140]: pgmap v3361: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:14:11.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:14:11 smithi138 ceph-mon[36954]: pgmap v3362: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:14:11.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:14:11 smithi100 ceph-mon[32599]: pgmap v3362: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:14:11.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:14:11 smithi167 ceph-mon[37140]: pgmap v3362: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:14:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:14:13 smithi138 ceph-mon[36954]: pgmap v3363: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:14:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:14:13 smithi100 ceph-mon[32599]: pgmap v3363: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:14:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:14:13 smithi167 ceph-mon[37140]: pgmap v3363: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:14:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:14:15 smithi138 ceph-mon[36954]: pgmap v3364: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:14:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:14:15 smithi138 ceph-mon[36954]: osdmap e259: 12 total, 12 up, 12 in 2024-07-26T15:14:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:14:15 smithi100 ceph-mon[32599]: pgmap v3364: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:14:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:14:15 smithi100 ceph-mon[32599]: osdmap e259: 12 total, 12 up, 12 in 2024-07-26T15:14:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:14:15 smithi167 ceph-mon[37140]: pgmap v3364: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:14:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:14:15 smithi167 ceph-mon[37140]: osdmap e259: 12 total, 12 up, 12 in 2024-07-26T15:14:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:14:17 smithi138 ceph-mon[36954]: pgmap v3366: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:14:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:14:17 smithi100 ceph-mon[32599]: pgmap v3366: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:14:17.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:14:17 smithi167 ceph-mon[37140]: pgmap v3366: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:14:18.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:14:18.009+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:14:18.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:14:18.010+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:14:18.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:14:18.010+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:14:18.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:14:18.010+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:14:18.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:14:18.010+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:14:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:14:19 smithi138 ceph-mon[36954]: pgmap v3367: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:14:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:14:19 smithi100 ceph-mon[32599]: pgmap v3367: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:14:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:14:19 smithi167 ceph-mon[37140]: pgmap v3367: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:14:21.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:14:21 smithi138 ceph-mon[36954]: pgmap v3368: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:14:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:14:21 smithi100 ceph-mon[32599]: pgmap v3368: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:14:21.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:14:21 smithi167 ceph-mon[37140]: pgmap v3368: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:14:23.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:14:23 smithi138 ceph-mon[36954]: pgmap v3369: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:14:23.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:14:23 smithi100 ceph-mon[32599]: pgmap v3369: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:14:23.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:14:23 smithi167 ceph-mon[37140]: pgmap v3369: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:14:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:14:25 smithi138 ceph-mon[36954]: pgmap v3370: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:14:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:14:25 smithi100 ceph-mon[32599]: pgmap v3370: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:14:25.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:14:25 smithi167 ceph-mon[37140]: pgmap v3370: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:14:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:14:27 smithi138 ceph-mon[36954]: pgmap v3371: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:14:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:14:27 smithi100 ceph-mon[32599]: pgmap v3371: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:14:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:14:27 smithi167 ceph-mon[37140]: pgmap v3371: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:14:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:14:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:14:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:14:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:14:28.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:14:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:14:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:14:29 smithi138 ceph-mon[36954]: pgmap v3372: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:14:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:14:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:14:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:14:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:14:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:14:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:14:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:14:29 smithi100 ceph-mon[32599]: pgmap v3372: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:14:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:14:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:14:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:14:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:14:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:14:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:14:29.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:14:29 smithi167 ceph-mon[37140]: pgmap v3372: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:14:29.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:14:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:14:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:14:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:14:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:14:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:14:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:14:31 smithi138 ceph-mon[36954]: pgmap v3373: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:14:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:14:31 smithi100 ceph-mon[32599]: pgmap v3373: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:14:31.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:14:31 smithi167 ceph-mon[37140]: pgmap v3373: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:14:33.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:14:33.023+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:14:33.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:14:33.024+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:14:33.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:14:33.024+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:14:33.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:14:33.024+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:14:33.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:14:33.024+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:14:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:14:33 smithi138 ceph-mon[36954]: pgmap v3374: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:14:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:14:33 smithi100 ceph-mon[32599]: pgmap v3374: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:14:33.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:14:33 smithi167 ceph-mon[37140]: pgmap v3374: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:14:35.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:14:35 smithi138 ceph-mon[36954]: pgmap v3375: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:14:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:14:35 smithi100 ceph-mon[32599]: pgmap v3375: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:14:35.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:14:35 smithi167 ceph-mon[37140]: pgmap v3375: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:14:37.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:14:37 smithi138 ceph-mon[36954]: pgmap v3376: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:14:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:14:37 smithi100 ceph-mon[32599]: pgmap v3376: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:14:37.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:14:37 smithi167 ceph-mon[37140]: pgmap v3376: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:14:39.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:14:39 smithi138 ceph-mon[36954]: pgmap v3377: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:14:39.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:14:39 smithi100 ceph-mon[32599]: pgmap v3377: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:14:39.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:14:39 smithi167 ceph-mon[37140]: pgmap v3377: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:14:41.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:14:41 smithi138 ceph-mon[36954]: pgmap v3378: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-07-26T15:14:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:14:41 smithi100 ceph-mon[32599]: pgmap v3378: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-07-26T15:14:41.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:14:41 smithi167 ceph-mon[37140]: pgmap v3378: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-07-26T15:14:43.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:14:43 smithi138 ceph-mon[36954]: pgmap v3379: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:14:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:14:43 smithi100 ceph-mon[32599]: pgmap v3379: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:14:43.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:14:43 smithi167 ceph-mon[37140]: pgmap v3379: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:14:45.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:14:45 smithi138 ceph-mon[36954]: pgmap v3380: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:14:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:14:45 smithi100 ceph-mon[32599]: pgmap v3380: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:14:45.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:14:45 smithi167 ceph-mon[37140]: pgmap v3380: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:14:47.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:14:47 smithi138 ceph-mon[36954]: pgmap v3381: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:14:47.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:14:47 smithi100 ceph-mon[32599]: pgmap v3381: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:14:47.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:14:47 smithi167 ceph-mon[37140]: pgmap v3381: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:14:48.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:14:48.031+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:14:48.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:14:48.032+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:14:48.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:14:48.032+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:14:48.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:14:48.032+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:14:48.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:14:48.032+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:14:49.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:14:49 smithi138 ceph-mon[36954]: pgmap v3382: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:14:49.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:14:49 smithi100 ceph-mon[32599]: pgmap v3382: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:14:49.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:14:49 smithi167 ceph-mon[37140]: pgmap v3382: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:14:51.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:14:51 smithi138 ceph-mon[36954]: pgmap v3383: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:14:51.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:14:51 smithi100 ceph-mon[32599]: pgmap v3383: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:14:51.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:14:51 smithi167 ceph-mon[37140]: pgmap v3383: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:14:52.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:14:52 smithi100 ceph-mon[32599]: pgmap v3384: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:14:52.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:14:52 smithi167 ceph-mon[37140]: pgmap v3384: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:14:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:14:52 smithi138 ceph-mon[36954]: pgmap v3384: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:14:55.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:14:55 smithi138 ceph-mon[36954]: pgmap v3385: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:14:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:14:55 smithi100 ceph-mon[32599]: pgmap v3385: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:14:55.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:14:55 smithi167 ceph-mon[37140]: pgmap v3385: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:14:57.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:14:57 smithi138 ceph-mon[36954]: pgmap v3386: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:14:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:14:57 smithi100 ceph-mon[32599]: pgmap v3386: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:14:57.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:14:57 smithi167 ceph-mon[37140]: pgmap v3386: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:14:59.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:14:59 smithi138 ceph-mon[36954]: pgmap v3387: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:14:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:14:59 smithi100 ceph-mon[32599]: pgmap v3387: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:14:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:14:59 smithi167 ceph-mon[37140]: pgmap v3387: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:15:01.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:15:01 smithi138 ceph-mon[36954]: pgmap v3388: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:15:01.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:15:01 smithi100 ceph-mon[32599]: pgmap v3388: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:15:01.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:15:01 smithi167 ceph-mon[37140]: pgmap v3388: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:15:03.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:15:03.047+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:15:03.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:15:03.048+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:15:03.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:15:03.048+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:15:03.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:15:03.048+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:15:03.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:15:03.048+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:15:03.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:15:03 smithi138 ceph-mon[36954]: pgmap v3389: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:15:03.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:15:03 smithi100 ceph-mon[32599]: pgmap v3389: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:15:03.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:15:03.749+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T15:15:03.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:15:03.749+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T15:15:03.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:15:03.750+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 44 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T15:15:03.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:15:03.750+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 44 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T15:15:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:15:03 smithi167 ceph-mon[37140]: pgmap v3389: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:15:05.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:15:05 smithi138 ceph-mon[36954]: pgmap v3390: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:15:05.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:15:05 smithi100 ceph-mon[32599]: pgmap v3390: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:15:05.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:15:05 smithi167 ceph-mon[37140]: pgmap v3390: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:15:07.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:15:07 smithi138 ceph-mon[36954]: pgmap v3391: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:15:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:15:07 smithi100 ceph-mon[32599]: pgmap v3391: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:15:07.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:15:07 smithi167 ceph-mon[37140]: pgmap v3391: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:15:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:15:08 smithi138 ceph-mon[36954]: osdmap e260: 12 total, 12 up, 12 in 2024-07-26T15:15:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:15:08 smithi100 ceph-mon[32599]: osdmap e260: 12 total, 12 up, 12 in 2024-07-26T15:15:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:15:08 smithi167 ceph-mon[37140]: osdmap e260: 12 total, 12 up, 12 in 2024-07-26T15:15:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:15:09 smithi138 ceph-mon[36954]: pgmap v3393: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:15:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:15:09 smithi100 ceph-mon[32599]: pgmap v3393: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:15:09.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:15:09 smithi167 ceph-mon[37140]: pgmap v3393: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:15:11.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:15:11 smithi138 ceph-mon[36954]: pgmap v3394: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:15:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:15:11 smithi100 ceph-mon[32599]: pgmap v3394: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:15:11.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:15:11 smithi167 ceph-mon[37140]: pgmap v3394: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:15:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:15:13 smithi138 ceph-mon[36954]: pgmap v3395: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:15:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:15:13 smithi100 ceph-mon[32599]: pgmap v3395: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:15:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:15:13 smithi167 ceph-mon[37140]: pgmap v3395: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:15:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:15:15 smithi138 ceph-mon[36954]: pgmap v3396: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:15:15.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:15:15 smithi138 ceph-mon[36954]: osdmap e261: 12 total, 12 up, 12 in 2024-07-26T15:15:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:15:15 smithi100 ceph-mon[32599]: pgmap v3396: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:15:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:15:15 smithi100 ceph-mon[32599]: osdmap e261: 12 total, 12 up, 12 in 2024-07-26T15:15:15.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:15:15 smithi167 ceph-mon[37140]: pgmap v3396: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:15:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:15:15 smithi167 ceph-mon[37140]: osdmap e261: 12 total, 12 up, 12 in 2024-07-26T15:15:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:15:17 smithi138 ceph-mon[36954]: pgmap v3398: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:15:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:15:17 smithi100 ceph-mon[32599]: pgmap v3398: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:15:17.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:15:17 smithi167 ceph-mon[37140]: pgmap v3398: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:15:18.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:15:18.055+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:15:18.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:15:18.055+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:15:18.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:15:18.055+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:15:18.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:15:18.055+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:15:18.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:15:18.055+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:15:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:15:19 smithi138 ceph-mon[36954]: pgmap v3399: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:15:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:15:19 smithi100 ceph-mon[32599]: pgmap v3399: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:15:19.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:15:19 smithi167 ceph-mon[37140]: pgmap v3399: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:15:21.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:15:21 smithi138 ceph-mon[36954]: pgmap v3400: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:15:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:15:21 smithi100 ceph-mon[32599]: pgmap v3400: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:15:21.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:15:21 smithi167 ceph-mon[37140]: pgmap v3400: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:15:23.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:15:23 smithi138 ceph-mon[36954]: pgmap v3401: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:15:23.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:15:23 smithi100 ceph-mon[32599]: pgmap v3401: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:15:23.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:15:23 smithi167 ceph-mon[37140]: pgmap v3401: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:15:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:15:25 smithi138 ceph-mon[36954]: pgmap v3402: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:15:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:15:25 smithi100 ceph-mon[32599]: pgmap v3402: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:15:25.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:15:25 smithi167 ceph-mon[37140]: pgmap v3402: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:15:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:15:27 smithi138 ceph-mon[36954]: pgmap v3403: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:15:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:15:27 smithi100 ceph-mon[32599]: pgmap v3403: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:15:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:15:27 smithi167 ceph-mon[37140]: pgmap v3403: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:15:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:15:29 smithi138 ceph-mon[36954]: pgmap v3404: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:15:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:15:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:15:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:15:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:15:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:15:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:15:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:15:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:15:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:15:29 smithi100 ceph-mon[32599]: pgmap v3404: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:15:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:15:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:15:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:15:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:15:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:15:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:15:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:15:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:15:29.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:15:29 smithi167 ceph-mon[37140]: pgmap v3404: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:15:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:15:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:15:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:15:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:15:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:15:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:15:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:15:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:15:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:15:31 smithi138 ceph-mon[36954]: pgmap v3405: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:15:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:15:31 smithi100 ceph-mon[32599]: pgmap v3405: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:15:31.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:15:31 smithi167 ceph-mon[37140]: pgmap v3405: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:15:33.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:15:33.071+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:15:33.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:15:33.072+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:15:33.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:15:33.072+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:15:33.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:15:33.072+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:15:33.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:15:33.072+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:15:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:15:33 smithi138 ceph-mon[36954]: pgmap v3406: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:15:33.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:15:33 smithi100 ceph-mon[32599]: pgmap v3406: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:15:33.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:15:33 smithi167 ceph-mon[37140]: pgmap v3406: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:15:35.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:15:35 smithi138 ceph-mon[36954]: pgmap v3407: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:15:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:15:35 smithi100 ceph-mon[32599]: pgmap v3407: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:15:35.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:15:35 smithi167 ceph-mon[37140]: pgmap v3407: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:15:37.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:15:37 smithi138 ceph-mon[36954]: pgmap v3408: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:15:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:15:37 smithi100 ceph-mon[32599]: pgmap v3408: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:15:37.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:15:37 smithi167 ceph-mon[37140]: pgmap v3408: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:15:39.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:15:39 smithi138 ceph-mon[36954]: pgmap v3409: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:15:39.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:15:39 smithi100 ceph-mon[32599]: pgmap v3409: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:15:39.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:15:39 smithi167 ceph-mon[37140]: pgmap v3409: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:15:41.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:15:41 smithi138 ceph-mon[36954]: pgmap v3410: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:15:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:15:41 smithi100 ceph-mon[32599]: pgmap v3410: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:15:41.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:15:41 smithi167 ceph-mon[37140]: pgmap v3410: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:15:43.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:15:43 smithi138 ceph-mon[36954]: pgmap v3411: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:15:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:15:43 smithi100 ceph-mon[32599]: pgmap v3411: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:15:43.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:15:43 smithi167 ceph-mon[37140]: pgmap v3411: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:15:45.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:15:45 smithi138 ceph-mon[36954]: pgmap v3412: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:15:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:15:45 smithi100 ceph-mon[32599]: pgmap v3412: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:15:45.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:15:45 smithi167 ceph-mon[37140]: pgmap v3412: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:15:47.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:15:47 smithi138 ceph-mon[36954]: pgmap v3413: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:15:47.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:15:47 smithi100 ceph-mon[32599]: pgmap v3413: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:15:47.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:15:47 smithi167 ceph-mon[37140]: pgmap v3413: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:15:48.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:15:48.088+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:15:48.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:15:48.088+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:15:48.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:15:48.088+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:15:48.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:15:48.088+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:15:48.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:15:48.088+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:15:49.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:15:49 smithi138 ceph-mon[36954]: pgmap v3414: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:15:49.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:15:49 smithi100 ceph-mon[32599]: pgmap v3414: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:15:49.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:15:49 smithi167 ceph-mon[37140]: pgmap v3414: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:15:51.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:15:51 smithi138 ceph-mon[36954]: pgmap v3415: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:15:51.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:15:51 smithi100 ceph-mon[32599]: pgmap v3415: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:15:51.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:15:51 smithi167 ceph-mon[37140]: pgmap v3415: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:15:53.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:15:53 smithi138 ceph-mon[36954]: pgmap v3416: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:15:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:15:53 smithi100 ceph-mon[32599]: pgmap v3416: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:15:53.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:15:53 smithi167 ceph-mon[37140]: pgmap v3416: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:15:55.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:15:55 smithi138 ceph-mon[36954]: pgmap v3417: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:15:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:15:55 smithi100 ceph-mon[32599]: pgmap v3417: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:15:55.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:15:55 smithi167 ceph-mon[37140]: pgmap v3417: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:15:57.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:15:57 smithi138 ceph-mon[36954]: pgmap v3418: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:15:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:15:57 smithi100 ceph-mon[32599]: pgmap v3418: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:15:57.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:15:57 smithi167 ceph-mon[37140]: pgmap v3418: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:15:59.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:15:59 smithi138 ceph-mon[36954]: pgmap v3419: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:15:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:15:59 smithi100 ceph-mon[32599]: pgmap v3419: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:15:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:15:59 smithi167 ceph-mon[37140]: pgmap v3419: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:16:01.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:16:01 smithi138 ceph-mon[36954]: pgmap v3420: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:16:01.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:16:01 smithi100 ceph-mon[32599]: pgmap v3420: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:16:01.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:16:01 smithi167 ceph-mon[37140]: pgmap v3420: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:16:03.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:16:03.089+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:16:03.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:16:03.090+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:16:03.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:16:03.090+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:16:03.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:16:03.090+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:16:03.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:16:03.090+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:16:03.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:16:03 smithi138 ceph-mon[36954]: pgmap v3421: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:16:03.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:16:03 smithi100 ceph-mon[32599]: pgmap v3421: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:16:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:16:03 smithi167 ceph-mon[37140]: pgmap v3421: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:16:05.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:16:05 smithi138 ceph-mon[36954]: pgmap v3422: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:16:05.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:16:05 smithi100 ceph-mon[32599]: pgmap v3422: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:16:05.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:16:05 smithi167 ceph-mon[37140]: pgmap v3422: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:16:07.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:16:07 smithi138 ceph-mon[36954]: pgmap v3423: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:16:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:16:07 smithi100 ceph-mon[32599]: pgmap v3423: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:16:07.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:16:07 smithi167 ceph-mon[37140]: pgmap v3423: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:16:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:16:08 smithi100 ceph-mon[32599]: osdmap e262: 12 total, 12 up, 12 in 2024-07-26T15:16:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:16:08 smithi167 ceph-mon[37140]: osdmap e262: 12 total, 12 up, 12 in 2024-07-26T15:16:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:16:08 smithi138 ceph-mon[36954]: osdmap e262: 12 total, 12 up, 12 in 2024-07-26T15:16:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:16:09 smithi100 ceph-mon[32599]: pgmap v3425: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:16:09.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:16:09 smithi167 ceph-mon[37140]: pgmap v3425: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:16:09.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:16:09 smithi138 ceph-mon[36954]: pgmap v3425: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:16:10.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:16:10 smithi167 ceph-mon[37140]: pgmap v3426: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:16:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:16:10 smithi138 ceph-mon[36954]: pgmap v3426: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:16:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:16:10 smithi100 ceph-mon[32599]: pgmap v3426: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:16:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:16:13 smithi138 ceph-mon[36954]: pgmap v3427: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:16:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:16:13 smithi100 ceph-mon[32599]: pgmap v3427: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:16:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:16:13 smithi167 ceph-mon[37140]: pgmap v3427: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:16:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:16:15 smithi138 ceph-mon[36954]: pgmap v3428: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:16:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:16:15 smithi138 ceph-mon[36954]: osdmap e263: 12 total, 12 up, 12 in 2024-07-26T15:16:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:16:15 smithi100 ceph-mon[32599]: pgmap v3428: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:16:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:16:15 smithi100 ceph-mon[32599]: osdmap e263: 12 total, 12 up, 12 in 2024-07-26T15:16:15.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:16:15 smithi167 ceph-mon[37140]: pgmap v3428: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:16:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:16:15 smithi167 ceph-mon[37140]: osdmap e263: 12 total, 12 up, 12 in 2024-07-26T15:16:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:16:17 smithi138 ceph-mon[36954]: pgmap v3430: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T15:16:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:16:17 smithi100 ceph-mon[32599]: pgmap v3430: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T15:16:17.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:16:17 smithi167 ceph-mon[37140]: pgmap v3430: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T15:16:18.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:16:18.105+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:16:18.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:16:18.106+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:16:18.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:16:18.106+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:16:18.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:16:18.106+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:16:18.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:16:18.106+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:16:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:16:19 smithi138 ceph-mon[36954]: pgmap v3431: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:16:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:16:19 smithi100 ceph-mon[32599]: pgmap v3431: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:16:19.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:16:19 smithi167 ceph-mon[37140]: pgmap v3431: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:16:21.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:16:21 smithi138 ceph-mon[36954]: pgmap v3432: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:16:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:16:21 smithi100 ceph-mon[32599]: pgmap v3432: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:16:21.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:16:21 smithi167 ceph-mon[37140]: pgmap v3432: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:16:23.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:16:23 smithi138 ceph-mon[36954]: pgmap v3433: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:16:23.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:16:23 smithi100 ceph-mon[32599]: pgmap v3433: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:16:23.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:16:23 smithi167 ceph-mon[37140]: pgmap v3433: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:16:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:16:25 smithi138 ceph-mon[36954]: pgmap v3434: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:16:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:16:25 smithi100 ceph-mon[32599]: pgmap v3434: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:16:25.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:16:25 smithi167 ceph-mon[37140]: pgmap v3434: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:16:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:16:27 smithi138 ceph-mon[36954]: pgmap v3435: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:16:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:16:27 smithi100 ceph-mon[32599]: pgmap v3435: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:16:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:16:27 smithi167 ceph-mon[37140]: pgmap v3435: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:16:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:16:29 smithi138 ceph-mon[36954]: pgmap v3436: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:16:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:16:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:16:29.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:16:29 smithi100 ceph-mon[32599]: pgmap v3436: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:16:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:16:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:16:29.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:16:29 smithi167 ceph-mon[37140]: pgmap v3436: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:16:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:16:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:16:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:16:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:16:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:16:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:16:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:16:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:16:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:16:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:16:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:16:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:16:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:16:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:16:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:16:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:16:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:16:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:16:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:16:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:16:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:16:31 smithi138 ceph-mon[36954]: pgmap v3437: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:16:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:16:31 smithi100 ceph-mon[32599]: pgmap v3437: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:16:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:16:31 smithi167 ceph-mon[37140]: pgmap v3437: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:16:33.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:16:33.121+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:16:33.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:16:33.121+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:16:33.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:16:33.121+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:16:33.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:16:33.121+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:16:33.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:16:33.121+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:16:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:16:33 smithi138 ceph-mon[36954]: pgmap v3438: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:16:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:16:33 smithi100 ceph-mon[32599]: pgmap v3438: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:16:33.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:16:33 smithi167 ceph-mon[37140]: pgmap v3438: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:16:35.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:16:35 smithi138 ceph-mon[36954]: pgmap v3439: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:16:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:16:35 smithi100 ceph-mon[32599]: pgmap v3439: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:16:35.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:16:35 smithi167 ceph-mon[37140]: pgmap v3439: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:16:37.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:16:37 smithi138 ceph-mon[36954]: pgmap v3440: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:16:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:16:37 smithi100 ceph-mon[32599]: pgmap v3440: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:16:37.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:16:37 smithi167 ceph-mon[37140]: pgmap v3440: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:16:39.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:16:39 smithi138 ceph-mon[36954]: pgmap v3441: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:16:39.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:16:39 smithi100 ceph-mon[32599]: pgmap v3441: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:16:39.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:16:39 smithi167 ceph-mon[37140]: pgmap v3441: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:16:41.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:16:41 smithi138 ceph-mon[36954]: pgmap v3442: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:16:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:16:41 smithi100 ceph-mon[32599]: pgmap v3442: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:16:41.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:16:41 smithi167 ceph-mon[37140]: pgmap v3442: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:16:43.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:16:43 smithi138 ceph-mon[36954]: pgmap v3443: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:16:43.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:16:43 smithi100 ceph-mon[32599]: pgmap v3443: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:16:43.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:16:43 smithi167 ceph-mon[37140]: pgmap v3443: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:16:45.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:16:45 smithi138 ceph-mon[36954]: pgmap v3444: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:16:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:16:45 smithi100 ceph-mon[32599]: pgmap v3444: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:16:45.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:16:45 smithi167 ceph-mon[37140]: pgmap v3444: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:16:47.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:16:47 smithi138 ceph-mon[36954]: pgmap v3445: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:16:47.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:16:47 smithi100 ceph-mon[32599]: pgmap v3445: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:16:47.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:16:47 smithi167 ceph-mon[37140]: pgmap v3445: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:16:48.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:16:48.132+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:16:48.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:16:48.132+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:16:48.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:16:48.132+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:16:48.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:16:48.132+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:16:48.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:16:48.132+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:16:49.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:16:49 smithi138 ceph-mon[36954]: pgmap v3446: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:16:49.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:16:49 smithi100 ceph-mon[32599]: pgmap v3446: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:16:49.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:16:49 smithi167 ceph-mon[37140]: pgmap v3446: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:16:51.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:16:51 smithi138 ceph-mon[36954]: pgmap v3447: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:16:51.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:16:51 smithi100 ceph-mon[32599]: pgmap v3447: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:16:51.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:16:51 smithi167 ceph-mon[37140]: pgmap v3447: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:16:53.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:16:53 smithi138 ceph-mon[36954]: pgmap v3448: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:16:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:16:53 smithi100 ceph-mon[32599]: pgmap v3448: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:16:53.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:16:53 smithi167 ceph-mon[37140]: pgmap v3448: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:16:55.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:16:55 smithi138 ceph-mon[36954]: pgmap v3449: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:16:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:16:55 smithi100 ceph-mon[32599]: pgmap v3449: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:16:55.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:16:55 smithi167 ceph-mon[37140]: pgmap v3449: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:16:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:16:57 smithi100 ceph-mon[32599]: pgmap v3450: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:16:57.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:16:57 smithi167 ceph-mon[37140]: pgmap v3450: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:16:57.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:16:57 smithi138 ceph-mon[36954]: pgmap v3450: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:16:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:16:59 smithi100 ceph-mon[32599]: pgmap v3451: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:16:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:16:59 smithi167 ceph-mon[37140]: pgmap v3451: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:16:59.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:16:59 smithi138 ceph-mon[36954]: pgmap v3451: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:17:01.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:01 smithi100 ceph-mon[32599]: pgmap v3452: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:17:01.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:01 smithi167 ceph-mon[37140]: pgmap v3452: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:17:01.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:01 smithi138 ceph-mon[36954]: pgmap v3452: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:17:03.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:17:03.143+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:17:03.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:17:03.144+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:17:03.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:17:03.144+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:17:03.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:17:03.144+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:17:03.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:17:03.144+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:17:03.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:03 smithi100 ceph-mon[32599]: pgmap v3453: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:17:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:03 smithi167 ceph-mon[37140]: pgmap v3453: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:17:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:03 smithi138 ceph-mon[36954]: pgmap v3453: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:17:05.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:05 smithi100 ceph-mon[32599]: pgmap v3454: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:17:05.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:05 smithi167 ceph-mon[37140]: pgmap v3454: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:17:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:05 smithi138 ceph-mon[36954]: pgmap v3454: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:17:06.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:06 smithi100 ceph-mon[32599]: pgmap v3455: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:17:06.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:06 smithi167 ceph-mon[37140]: pgmap v3455: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:17:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:06 smithi138 ceph-mon[36954]: pgmap v3455: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:17:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:08 smithi100 ceph-mon[32599]: osdmap e264: 12 total, 12 up, 12 in 2024-07-26T15:17:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:08 smithi100 ceph-mon[32599]: pgmap v3457: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:17:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:08 smithi167 ceph-mon[37140]: osdmap e264: 12 total, 12 up, 12 in 2024-07-26T15:17:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:08 smithi167 ceph-mon[37140]: pgmap v3457: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:17:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:08 smithi138 ceph-mon[36954]: osdmap e264: 12 total, 12 up, 12 in 2024-07-26T15:17:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:08 smithi138 ceph-mon[36954]: pgmap v3457: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:17:11.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:11 smithi138 ceph-mon[36954]: pgmap v3458: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:17:11.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:11 smithi100 ceph-mon[32599]: pgmap v3458: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:17:11.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:11 smithi167 ceph-mon[37140]: pgmap v3458: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:17:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:13 smithi138 ceph-mon[36954]: pgmap v3459: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:17:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:13 smithi100 ceph-mon[32599]: pgmap v3459: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:17:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:13 smithi167 ceph-mon[37140]: pgmap v3459: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:17:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:15 smithi138 ceph-mon[36954]: pgmap v3460: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:17:15.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:15 smithi138 ceph-mon[36954]: osdmap e265: 12 total, 12 up, 12 in 2024-07-26T15:17:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:15 smithi100 ceph-mon[32599]: pgmap v3460: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:17:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:15 smithi100 ceph-mon[32599]: osdmap e265: 12 total, 12 up, 12 in 2024-07-26T15:17:15.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:15 smithi167 ceph-mon[37140]: pgmap v3460: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:17:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:15 smithi167 ceph-mon[37140]: osdmap e265: 12 total, 12 up, 12 in 2024-07-26T15:17:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:17 smithi138 ceph-mon[36954]: pgmap v3462: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T15:17:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:17 smithi100 ceph-mon[32599]: pgmap v3462: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T15:17:17.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:17 smithi167 ceph-mon[37140]: pgmap v3462: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T15:17:18.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:17:18.159+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:17:18.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:17:18.159+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:17:18.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:17:18.159+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:17:18.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:17:18.159+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:17:18.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:17:18.159+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:17:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:19 smithi138 ceph-mon[36954]: pgmap v3463: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:17:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:19 smithi100 ceph-mon[32599]: pgmap v3463: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:17:19.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:19 smithi167 ceph-mon[37140]: pgmap v3463: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:17:21.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:21 smithi138 ceph-mon[36954]: pgmap v3464: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:17:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:21 smithi100 ceph-mon[32599]: pgmap v3464: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:17:21.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:21 smithi167 ceph-mon[37140]: pgmap v3464: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:17:23.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:23 smithi138 ceph-mon[36954]: pgmap v3465: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:17:23.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:23 smithi100 ceph-mon[32599]: pgmap v3465: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:17:23.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:23 smithi167 ceph-mon[37140]: pgmap v3465: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:17:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:25 smithi138 ceph-mon[36954]: pgmap v3466: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:17:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:25 smithi100 ceph-mon[32599]: pgmap v3466: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:17:25.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:25 smithi167 ceph-mon[37140]: pgmap v3466: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:17:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:27 smithi138 ceph-mon[36954]: pgmap v3467: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:17:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:27 smithi100 ceph-mon[32599]: pgmap v3467: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:17:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:27 smithi167 ceph-mon[37140]: pgmap v3467: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:17:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:29 smithi138 ceph-mon[36954]: pgmap v3468: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:17:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:29 smithi100 ceph-mon[32599]: pgmap v3468: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:17:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:29 smithi167 ceph-mon[37140]: pgmap v3468: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:17:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:17:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:17:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:17:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:17:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:17:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:17:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:17:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:17:30.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:17:30.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:17:30.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:17:30.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:17:30.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:17:30.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:17:30.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:17:30.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:17:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:17:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:17:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:17:30.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:17:30.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:17:30.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:17:30.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:17:30.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:17:30.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:17:30.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:17:30.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:17:30.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:17:30.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:17:30.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:17:30.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:17:30.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:17:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:17:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:17:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:17:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:17:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:17:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:17:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:17:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:17:30.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:17:30.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:17:30.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:17:30.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:17:30.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:17:30.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:17:30.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:17:30.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:17:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:31 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T15:17:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:31 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T15:17:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:31 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T15:17:31.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:31 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T15:17:31.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:31 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T15:17:31.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:31 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T15:17:31.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:31 smithi138 ceph-mon[36954]: pgmap v3469: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:17:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:31 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T15:17:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:31 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T15:17:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:31 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T15:17:31.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:31 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T15:17:31.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:31 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T15:17:31.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:31 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T15:17:31.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:31 smithi100 ceph-mon[32599]: pgmap v3469: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:17:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:31 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T15:17:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:31 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T15:17:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:31 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T15:17:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:31 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T15:17:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:31 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T15:17:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:31 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T15:17:31.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:31 smithi167 ceph-mon[37140]: pgmap v3469: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:17:33.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:17:33.172+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:17:33.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:17:33.173+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:17:33.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:17:33.173+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:17:33.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:17:33.173+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:17:33.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:17:33.173+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:17:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:33 smithi100 ceph-mon[32599]: pgmap v3470: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:17:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:33 smithi138 ceph-mon[36954]: pgmap v3470: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:17:33.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:33 smithi167 ceph-mon[37140]: pgmap v3470: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:17:35.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:35 smithi138 ceph-mon[36954]: pgmap v3471: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:17:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:35 smithi100 ceph-mon[32599]: pgmap v3471: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:17:35.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:35 smithi167 ceph-mon[37140]: pgmap v3471: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:17:37.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:37 smithi138 ceph-mon[36954]: pgmap v3472: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:17:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:37 smithi100 ceph-mon[32599]: pgmap v3472: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:17:37.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:37 smithi167 ceph-mon[37140]: pgmap v3472: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:17:39.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:39 smithi138 ceph-mon[36954]: pgmap v3473: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:17:39.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:39 smithi100 ceph-mon[32599]: pgmap v3473: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:17:39.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:39 smithi167 ceph-mon[37140]: pgmap v3473: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:17:41.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:41 smithi138 ceph-mon[36954]: pgmap v3474: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:17:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:41 smithi100 ceph-mon[32599]: pgmap v3474: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:17:41.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:41 smithi167 ceph-mon[37140]: pgmap v3474: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:17:43.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:43 smithi138 ceph-mon[36954]: pgmap v3475: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:17:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:43 smithi100 ceph-mon[32599]: pgmap v3475: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:17:43.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:43 smithi167 ceph-mon[37140]: pgmap v3475: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:17:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:45 smithi100 ceph-mon[32599]: pgmap v3476: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:17:45.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:45 smithi167 ceph-mon[37140]: pgmap v3476: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:17:45.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:45 smithi138 ceph-mon[36954]: pgmap v3476: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:17:47.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:47 smithi100 ceph-mon[32599]: pgmap v3477: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:17:47.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:47 smithi167 ceph-mon[37140]: pgmap v3477: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:17:47.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:47 smithi138 ceph-mon[36954]: pgmap v3477: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:17:48.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:17:48.188+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:17:48.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:17:48.188+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:17:48.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:17:48.188+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:17:48.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:17:48.188+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:17:48.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:17:48.188+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:17:49.743 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:49 smithi138 ceph-mon[36954]: pgmap v3478: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:17:49.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:49 smithi100 ceph-mon[32599]: pgmap v3478: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:17:49.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:49 smithi167 ceph-mon[37140]: pgmap v3478: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:17:51.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:51 smithi100 ceph-mon[32599]: pgmap v3479: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:17:51.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:51 smithi167 ceph-mon[37140]: pgmap v3479: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:17:51.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:51 smithi138 ceph-mon[36954]: pgmap v3479: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:17:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:53 smithi100 ceph-mon[32599]: pgmap v3480: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:17:53.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:53 smithi167 ceph-mon[37140]: pgmap v3480: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:17:53.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:53 smithi138 ceph-mon[36954]: pgmap v3480: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:17:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:55 smithi100 ceph-mon[32599]: pgmap v3481: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:17:55.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:55 smithi167 ceph-mon[37140]: pgmap v3481: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:17:55.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:55 smithi138 ceph-mon[36954]: pgmap v3481: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:17:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:57 smithi100 ceph-mon[32599]: pgmap v3482: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:17:57.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:57 smithi167 ceph-mon[37140]: pgmap v3482: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:17:57.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:57 smithi138 ceph-mon[36954]: pgmap v3482: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:17:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:17:59 smithi100 ceph-mon[32599]: pgmap v3483: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:17:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:17:59 smithi167 ceph-mon[37140]: pgmap v3483: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:17:59.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:17:59 smithi138 ceph-mon[36954]: pgmap v3483: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:18:01.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:18:01 smithi100 ceph-mon[32599]: pgmap v3484: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:18:01.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:18:01 smithi167 ceph-mon[37140]: pgmap v3484: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:18:01.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:18:01 smithi138 ceph-mon[36954]: pgmap v3484: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:18:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:18:02 smithi100 ceph-mon[32599]: pgmap v3485: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:18:02.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:18:02 smithi167 ceph-mon[37140]: pgmap v3485: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:18:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:18:02 smithi138 ceph-mon[36954]: pgmap v3485: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:18:03.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:18:03.031+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 msgr2=0x7fe5940c0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-07-26T15:18:03.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:18:03.031+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 msgr2=0x7fe5940c0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-07-26T15:18:03.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:18:03.031+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe59000b6b0 tx=0x7fe59005a040 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:18:03.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:18:03.031+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe59000b6b0 tx=0x7fe59005a040 comp rx=0 tx=0).stop 2024-07-26T15:18:03.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:18:03.031+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 msgr2=0x7fe5940c0480 unknown :-1 s=STATE_CLOSED l=1).tick see no progress in more than 10000000 us during connecting to v2:172.21.15.100:6800/3937009534, fault. 2024-07-26T15:18:03.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:18:03.032+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 msgr2=0x7fe5940c0480 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-07-26T15:18:03.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:18:03.032+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T15:18:03.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:18:03.032+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b4ef0 0x7fe5940b5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T15:18:03.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:18:03.032+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b4ef0 0x7fe5940b5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:18:03.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:18:03.032+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b4ef0 0x7fe5940b5300 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe590005210 tx=0x7fe59008b040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T15:18:03.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:18:03.189+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:18:03.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:18:03.189+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:18:03.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:18:03.189+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:18:03.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:18:03.189+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:18:03.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:18:03.190+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:18:03.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:18:03.775+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T15:18:03.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:18:03.775+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T15:18:03.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:18:03.775+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 45 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T15:18:03.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:18:03.775+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 45 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T15:18:05.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:18:05 smithi138 ceph-mon[36954]: pgmap v3486: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:18:05.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:18:05 smithi100 ceph-mon[32599]: pgmap v3486: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:18:05.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:18:05 smithi167 ceph-mon[37140]: pgmap v3486: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:18:07.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:18:07 smithi138 ceph-mon[36954]: pgmap v3487: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:18:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:18:07 smithi100 ceph-mon[32599]: pgmap v3487: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:18:07.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:18:07 smithi167 ceph-mon[37140]: pgmap v3487: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:18:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:18:08 smithi138 ceph-mon[36954]: osdmap e266: 12 total, 12 up, 12 in 2024-07-26T15:18:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:18:08 smithi100 ceph-mon[32599]: osdmap e266: 12 total, 12 up, 12 in 2024-07-26T15:18:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:18:08 smithi167 ceph-mon[37140]: osdmap e266: 12 total, 12 up, 12 in 2024-07-26T15:18:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:18:09 smithi138 ceph-mon[36954]: pgmap v3489: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:18:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:18:09 smithi100 ceph-mon[32599]: pgmap v3489: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:18:09.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:18:09 smithi167 ceph-mon[37140]: pgmap v3489: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:18:11.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:18:11 smithi138 ceph-mon[36954]: pgmap v3490: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:18:11.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:18:11 smithi100 ceph-mon[32599]: pgmap v3490: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:18:11.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:18:11 smithi167 ceph-mon[37140]: pgmap v3490: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:18:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:18:13 smithi138 ceph-mon[36954]: pgmap v3491: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:18:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:18:13 smithi100 ceph-mon[32599]: pgmap v3491: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:18:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:18:13 smithi167 ceph-mon[37140]: pgmap v3491: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:18:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:18:15 smithi138 ceph-mon[36954]: pgmap v3492: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:18:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:18:15 smithi138 ceph-mon[36954]: osdmap e267: 12 total, 12 up, 12 in 2024-07-26T15:18:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:18:15 smithi100 ceph-mon[32599]: pgmap v3492: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:18:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:18:15 smithi100 ceph-mon[32599]: osdmap e267: 12 total, 12 up, 12 in 2024-07-26T15:18:15.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:18:15 smithi167 ceph-mon[37140]: pgmap v3492: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:18:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:18:15 smithi167 ceph-mon[37140]: osdmap e267: 12 total, 12 up, 12 in 2024-07-26T15:18:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:18:17 smithi138 ceph-mon[36954]: pgmap v3494: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:18:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:18:17 smithi100 ceph-mon[32599]: pgmap v3494: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:18:17.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:18:17 smithi167 ceph-mon[37140]: pgmap v3494: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:18:18.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:18:18.205+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:18:18.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:18:18.206+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:18:18.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:18:18.206+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:18:18.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:18:18.206+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:18:18.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:18:18.206+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:18:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:18:19 smithi100 ceph-mon[32599]: pgmap v3495: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:18:19.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:18:19 smithi167 ceph-mon[37140]: pgmap v3495: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:18:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:18:19 smithi138 ceph-mon[36954]: pgmap v3495: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:18:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:18:21 smithi100 ceph-mon[32599]: pgmap v3496: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:18:21.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:18:21 smithi167 ceph-mon[37140]: pgmap v3496: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:18:21.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:18:21 smithi138 ceph-mon[36954]: pgmap v3496: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:18:23.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:18:23 smithi100 ceph-mon[32599]: pgmap v3497: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:18:23.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:18:23 smithi167 ceph-mon[37140]: pgmap v3497: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:18:23.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:18:23 smithi138 ceph-mon[36954]: pgmap v3497: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:18:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:18:25 smithi100 ceph-mon[32599]: pgmap v3498: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:18:25.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:18:25 smithi167 ceph-mon[37140]: pgmap v3498: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:18:25.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:18:25 smithi138 ceph-mon[36954]: pgmap v3498: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:18:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:18:27 smithi100 ceph-mon[32599]: pgmap v3499: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:18:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:18:27 smithi167 ceph-mon[37140]: pgmap v3499: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:18:27.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:18:27 smithi138 ceph-mon[36954]: pgmap v3499: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:18:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:18:29 smithi100 ceph-mon[32599]: pgmap v3500: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:18:29.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:18:29 smithi167 ceph-mon[37140]: pgmap v3500: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:18:29.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:18:29 smithi138 ceph-mon[36954]: pgmap v3500: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:18:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:18:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:18:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:18:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:18:30.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:18:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:18:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:18:31 smithi100 ceph-mon[32599]: pgmap v3501: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:18:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:18:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:18:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:18:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:18:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:18:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:18:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:18:31 smithi167 ceph-mon[37140]: pgmap v3501: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:18:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:18:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:18:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:18:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:18:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:18:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:18:31.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:18:31 smithi138 ceph-mon[36954]: pgmap v3501: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:18:31.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:18:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:18:31.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:18:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:18:31.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:18:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:18:33.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:18:33.215+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:18:33.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:18:33.216+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:18:33.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:18:33.216+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:18:33.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:18:33.216+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:18:33.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:18:33.216+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:18:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:18:33 smithi100 ceph-mon[32599]: pgmap v3502: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:18:33.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:18:33 smithi167 ceph-mon[37140]: pgmap v3502: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:18:33.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:18:33 smithi138 ceph-mon[36954]: pgmap v3502: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:18:34.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:18:34 smithi167 ceph-mon[37140]: pgmap v3503: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:18:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:18:34 smithi138 ceph-mon[36954]: pgmap v3503: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:18:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:18:34 smithi100 ceph-mon[32599]: pgmap v3503: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:18:37.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:18:37 smithi138 ceph-mon[36954]: pgmap v3504: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:18:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:18:37 smithi100 ceph-mon[32599]: pgmap v3504: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:18:37.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:18:37 smithi167 ceph-mon[37140]: pgmap v3504: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:18:39.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:18:39 smithi138 ceph-mon[36954]: pgmap v3505: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:18:39.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:18:39 smithi100 ceph-mon[32599]: pgmap v3505: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:18:39.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:18:39 smithi167 ceph-mon[37140]: pgmap v3505: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:18:41.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:18:41 smithi138 ceph-mon[36954]: pgmap v3506: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:18:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:18:41 smithi100 ceph-mon[32599]: pgmap v3506: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:18:41.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:18:41 smithi167 ceph-mon[37140]: pgmap v3506: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:18:43.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:18:43 smithi138 ceph-mon[36954]: pgmap v3507: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:18:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:18:43 smithi100 ceph-mon[32599]: pgmap v3507: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:18:43.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:18:43 smithi167 ceph-mon[37140]: pgmap v3507: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:18:45.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:18:45 smithi138 ceph-mon[36954]: pgmap v3508: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:18:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:18:45 smithi100 ceph-mon[32599]: pgmap v3508: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:18:45.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:18:45 smithi167 ceph-mon[37140]: pgmap v3508: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:18:47.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:18:47 smithi138 ceph-mon[36954]: pgmap v3509: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:18:47.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:18:47 smithi100 ceph-mon[32599]: pgmap v3509: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:18:47.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:18:47 smithi167 ceph-mon[37140]: pgmap v3509: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:18:48.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:18:48.231+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:18:48.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:18:48.231+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:18:48.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:18:48.231+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:18:48.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:18:48.231+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:18:48.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:18:48.231+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:18:48.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:18:48.781+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T15:18:48.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:18:48.781+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T15:18:48.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:18:48.782+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 46 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T15:18:48.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:18:48.782+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 46 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T15:18:49.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:18:49 smithi138 ceph-mon[36954]: pgmap v3510: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:18:49.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:18:49 smithi100 ceph-mon[32599]: pgmap v3510: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:18:49.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:18:49 smithi167 ceph-mon[37140]: pgmap v3510: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:18:51.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:18:51 smithi138 ceph-mon[36954]: pgmap v3511: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:18:51.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:18:51 smithi100 ceph-mon[32599]: pgmap v3511: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:18:51.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:18:51 smithi167 ceph-mon[37140]: pgmap v3511: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:18:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:18:53 smithi100 ceph-mon[32599]: pgmap v3512: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:18:53.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:18:53 smithi167 ceph-mon[37140]: pgmap v3512: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:18:53.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:18:53 smithi138 ceph-mon[36954]: pgmap v3512: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:18:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:18:55 smithi100 ceph-mon[32599]: pgmap v3513: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:18:55.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:18:55 smithi167 ceph-mon[37140]: pgmap v3513: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:18:55.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:18:55 smithi138 ceph-mon[36954]: pgmap v3513: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:18:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:18:57 smithi100 ceph-mon[32599]: pgmap v3514: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:18:57.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:18:57 smithi167 ceph-mon[37140]: pgmap v3514: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:18:57.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:18:57 smithi138 ceph-mon[36954]: pgmap v3514: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:18:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:18:59 smithi100 ceph-mon[32599]: pgmap v3515: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:18:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:18:59 smithi167 ceph-mon[37140]: pgmap v3515: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:18:59.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:18:59 smithi138 ceph-mon[36954]: pgmap v3515: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:19:01.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:19:01 smithi100 ceph-mon[32599]: pgmap v3516: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:19:01.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:19:01 smithi167 ceph-mon[37140]: pgmap v3516: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:19:01.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:19:01 smithi138 ceph-mon[36954]: pgmap v3516: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:19:03.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:19:03.246+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:19:03.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:19:03.246+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:19:03.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:19:03.246+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:19:03.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:19:03.246+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:19:03.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:19:03.246+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:19:03.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:19:03 smithi100 ceph-mon[32599]: pgmap v3517: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:19:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:19:03 smithi167 ceph-mon[37140]: pgmap v3517: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:19:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:19:03 smithi138 ceph-mon[36954]: pgmap v3517: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:19:05.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:19:05 smithi100 ceph-mon[32599]: pgmap v3518: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:19:05.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:19:05 smithi167 ceph-mon[37140]: pgmap v3518: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:19:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:19:05 smithi138 ceph-mon[36954]: pgmap v3518: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:19:07.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:19:07 smithi100 ceph-mon[32599]: pgmap v3519: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:19:07.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:19:07 smithi167 ceph-mon[37140]: pgmap v3519: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:19:07.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:19:07 smithi138 ceph-mon[36954]: pgmap v3519: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:19:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:19:08 smithi100 ceph-mon[32599]: osdmap e268: 12 total, 12 up, 12 in 2024-07-26T15:19:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:19:08 smithi167 ceph-mon[37140]: osdmap e268: 12 total, 12 up, 12 in 2024-07-26T15:19:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:19:08 smithi138 ceph-mon[36954]: osdmap e268: 12 total, 12 up, 12 in 2024-07-26T15:19:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:19:09 smithi100 ceph-mon[32599]: pgmap v3521: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:19:09.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:19:09 smithi167 ceph-mon[37140]: pgmap v3521: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:19:09.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:19:09 smithi138 ceph-mon[36954]: pgmap v3521: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:19:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:19:10 smithi100 ceph-mon[32599]: pgmap v3522: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:19:10.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:19:10 smithi167 ceph-mon[37140]: pgmap v3522: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:19:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:19:10 smithi138 ceph-mon[36954]: pgmap v3522: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:19:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:19:13 smithi138 ceph-mon[36954]: pgmap v3523: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:19:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:19:13 smithi100 ceph-mon[32599]: pgmap v3523: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:19:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:19:13 smithi167 ceph-mon[37140]: pgmap v3523: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:19:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:19:15 smithi138 ceph-mon[36954]: pgmap v3524: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:19:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:19:15 smithi138 ceph-mon[36954]: osdmap e269: 12 total, 12 up, 12 in 2024-07-26T15:19:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:19:15 smithi100 ceph-mon[32599]: pgmap v3524: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:19:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:19:15 smithi100 ceph-mon[32599]: osdmap e269: 12 total, 12 up, 12 in 2024-07-26T15:19:15.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:19:15 smithi167 ceph-mon[37140]: pgmap v3524: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:19:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:19:15 smithi167 ceph-mon[37140]: osdmap e269: 12 total, 12 up, 12 in 2024-07-26T15:19:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:19:17 smithi138 ceph-mon[36954]: pgmap v3526: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T15:19:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:19:17 smithi100 ceph-mon[32599]: pgmap v3526: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T15:19:17.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:19:17 smithi167 ceph-mon[37140]: pgmap v3526: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T15:19:18.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:19:18.263+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:19:18.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:19:18.264+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:19:18.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:19:18.264+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:19:18.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:19:18.264+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:19:18.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:19:18.264+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:19:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:19:19 smithi138 ceph-mon[36954]: pgmap v3527: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:19:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:19:19 smithi100 ceph-mon[32599]: pgmap v3527: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:19:19.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:19:19 smithi167 ceph-mon[37140]: pgmap v3527: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:19:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:19:21 smithi100 ceph-mon[32599]: pgmap v3528: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:19:21.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:19:21 smithi167 ceph-mon[37140]: pgmap v3528: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:19:21.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:19:21 smithi138 ceph-mon[36954]: pgmap v3528: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:19:23.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:19:23 smithi100 ceph-mon[32599]: pgmap v3529: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:19:23.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:19:23 smithi167 ceph-mon[37140]: pgmap v3529: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:19:23.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:19:23 smithi138 ceph-mon[36954]: pgmap v3529: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:19:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:19:25 smithi100 ceph-mon[32599]: pgmap v3530: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:19:25.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:19:25 smithi167 ceph-mon[37140]: pgmap v3530: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:19:25.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:19:25 smithi138 ceph-mon[36954]: pgmap v3530: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:19:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:19:27 smithi100 ceph-mon[32599]: pgmap v3531: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:19:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:19:27 smithi167 ceph-mon[37140]: pgmap v3531: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:19:27.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:19:27 smithi138 ceph-mon[36954]: pgmap v3531: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:19:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:19:29 smithi100 ceph-mon[32599]: pgmap v3532: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:19:29.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:19:29 smithi167 ceph-mon[37140]: pgmap v3532: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:19:29.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:19:29 smithi138 ceph-mon[36954]: pgmap v3532: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:19:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:19:31 smithi100 ceph-mon[32599]: pgmap v3533: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:19:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:19:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:19:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:19:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:19:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:19:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:19:31.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:19:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:19:31.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:19:31 smithi167 ceph-mon[37140]: pgmap v3533: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:19:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:19:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:19:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:19:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:19:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:19:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:19:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:19:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:19:31.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:19:31 smithi138 ceph-mon[36954]: pgmap v3533: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:19:31.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:19:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:19:31.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:19:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:19:31.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:19:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:19:31.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:19:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:19:33.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:19:33.279+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:19:33.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:19:33.280+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:19:33.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:19:33.280+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:19:33.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:19:33.280+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:19:33.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:19:33.280+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:19:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:19:33 smithi100 ceph-mon[32599]: pgmap v3534: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:19:33.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:19:33 smithi167 ceph-mon[37140]: pgmap v3534: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:19:33.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:19:33 smithi138 ceph-mon[36954]: pgmap v3534: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:19:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:19:35 smithi100 ceph-mon[32599]: pgmap v3535: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:19:35.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:19:35 smithi167 ceph-mon[37140]: pgmap v3535: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:19:35.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:19:35 smithi138 ceph-mon[36954]: pgmap v3535: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:19:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:19:37 smithi100 ceph-mon[32599]: pgmap v3536: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:19:37.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:19:37 smithi167 ceph-mon[37140]: pgmap v3536: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:19:37.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:19:37 smithi138 ceph-mon[36954]: pgmap v3536: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:19:39.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:19:39 smithi100 ceph-mon[32599]: pgmap v3537: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:19:39.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:19:39 smithi167 ceph-mon[37140]: pgmap v3537: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:19:39.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:19:39 smithi138 ceph-mon[36954]: pgmap v3537: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:19:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:19:41 smithi100 ceph-mon[32599]: pgmap v3538: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:19:41.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:19:41 smithi167 ceph-mon[37140]: pgmap v3538: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:19:41.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:19:41 smithi138 ceph-mon[36954]: pgmap v3538: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:19:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:19:43 smithi100 ceph-mon[32599]: pgmap v3539: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:19:43.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:19:43 smithi167 ceph-mon[37140]: pgmap v3539: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:19:43.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:19:43 smithi138 ceph-mon[36954]: pgmap v3539: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:19:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:19:45 smithi100 ceph-mon[32599]: pgmap v3540: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:19:45.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:19:45 smithi167 ceph-mon[37140]: pgmap v3540: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:19:45.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:19:45 smithi138 ceph-mon[36954]: pgmap v3540: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:19:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:19:46 smithi138 ceph-mon[36954]: pgmap v3541: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:19:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:19:46 smithi100 ceph-mon[32599]: pgmap v3541: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:19:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:19:46 smithi167 ceph-mon[37140]: pgmap v3541: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:19:48.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:19:48.295+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:19:48.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:19:48.296+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:19:48.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:19:48.296+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:19:48.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:19:48.296+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:19:48.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:19:48.296+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:19:49.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:19:49 smithi138 ceph-mon[36954]: pgmap v3542: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:19:49.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:19:49 smithi100 ceph-mon[32599]: pgmap v3542: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:19:49.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:19:49 smithi167 ceph-mon[37140]: pgmap v3542: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:19:51.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:19:51 smithi100 ceph-mon[32599]: pgmap v3543: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:19:51.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:19:51 smithi167 ceph-mon[37140]: pgmap v3543: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:19:51.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:19:51 smithi138 ceph-mon[36954]: pgmap v3543: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:19:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:19:53 smithi100 ceph-mon[32599]: pgmap v3544: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:19:53.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:19:53 smithi167 ceph-mon[37140]: pgmap v3544: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:19:53.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:19:53 smithi138 ceph-mon[36954]: pgmap v3544: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:19:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:19:55 smithi100 ceph-mon[32599]: pgmap v3545: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:19:55.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:19:55 smithi167 ceph-mon[37140]: pgmap v3545: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:19:55.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:19:55 smithi138 ceph-mon[36954]: pgmap v3545: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:19:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:19:57 smithi100 ceph-mon[32599]: pgmap v3546: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:19:57.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:19:57 smithi167 ceph-mon[37140]: pgmap v3546: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:19:57.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:19:57 smithi138 ceph-mon[36954]: pgmap v3546: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:19:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:19:59 smithi100 ceph-mon[32599]: pgmap v3547: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:19:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:19:59 smithi167 ceph-mon[37140]: pgmap v3547: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:19:59.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:19:59 smithi138 ceph-mon[36954]: pgmap v3547: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:20:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:00 smithi100 ceph-mon[32599]: overall HEALTH_OK 2024-07-26T15:20:00.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:00 smithi167 ceph-mon[37140]: overall HEALTH_OK 2024-07-26T15:20:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:00 smithi138 ceph-mon[36954]: overall HEALTH_OK 2024-07-26T15:20:01.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:01 smithi100 ceph-mon[32599]: pgmap v3548: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:20:01.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:01 smithi167 ceph-mon[37140]: pgmap v3548: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:20:01.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:01 smithi138 ceph-mon[36954]: pgmap v3548: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:20:03.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:20:03.304+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:20:03.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:20:03.304+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:20:03.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:20:03.305+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:20:03.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:20:03.305+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:20:03.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:20:03.305+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:20:03.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:03 smithi100 ceph-mon[32599]: pgmap v3549: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:20:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:03 smithi167 ceph-mon[37140]: pgmap v3549: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:20:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:03 smithi138 ceph-mon[36954]: pgmap v3549: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:20:05.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:05 smithi100 ceph-mon[32599]: pgmap v3550: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:20:05.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:05 smithi167 ceph-mon[37140]: pgmap v3550: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:20:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:05 smithi138 ceph-mon[36954]: pgmap v3550: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:20:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:07 smithi100 ceph-mon[32599]: pgmap v3551: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:20:07.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:07 smithi167 ceph-mon[37140]: pgmap v3551: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:20:07.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:07 smithi138 ceph-mon[36954]: pgmap v3551: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:20:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:08 smithi167 ceph-mon[37140]: osdmap e270: 12 total, 12 up, 12 in 2024-07-26T15:20:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:08 smithi167 ceph-mon[37140]: pgmap v3553: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:20:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:08 smithi138 ceph-mon[36954]: osdmap e270: 12 total, 12 up, 12 in 2024-07-26T15:20:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:08 smithi138 ceph-mon[36954]: pgmap v3553: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:20:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:08 smithi100 ceph-mon[32599]: osdmap e270: 12 total, 12 up, 12 in 2024-07-26T15:20:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:08 smithi100 ceph-mon[32599]: pgmap v3553: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:20:11.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:11 smithi100 ceph-mon[32599]: pgmap v3554: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:20:11.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:11 smithi167 ceph-mon[37140]: pgmap v3554: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:20:11.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:11 smithi138 ceph-mon[36954]: pgmap v3554: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:20:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:13 smithi100 ceph-mon[32599]: pgmap v3555: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:20:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:13 smithi167 ceph-mon[37140]: pgmap v3555: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:20:13.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:13 smithi138 ceph-mon[36954]: pgmap v3555: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:20:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:15 smithi100 ceph-mon[32599]: pgmap v3556: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:20:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:15 smithi100 ceph-mon[32599]: osdmap e271: 12 total, 12 up, 12 in 2024-07-26T15:20:15.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:15 smithi167 ceph-mon[37140]: pgmap v3556: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:20:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:15 smithi167 ceph-mon[37140]: osdmap e271: 12 total, 12 up, 12 in 2024-07-26T15:20:15.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:15 smithi138 ceph-mon[36954]: pgmap v3556: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:20:15.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:15 smithi138 ceph-mon[36954]: osdmap e271: 12 total, 12 up, 12 in 2024-07-26T15:20:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:17 smithi100 ceph-mon[32599]: pgmap v3558: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:20:17.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:17 smithi167 ceph-mon[37140]: pgmap v3558: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:20:17.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:17 smithi138 ceph-mon[36954]: pgmap v3558: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:20:18.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:20:18.319+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:20:18.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:20:18.320+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:20:18.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:20:18.320+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:20:18.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:20:18.320+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:20:18.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:20:18.320+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:20:19.742 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:19 smithi138 ceph-mon[36954]: pgmap v3559: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:20:19.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:19 smithi100 ceph-mon[32599]: pgmap v3559: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:20:19.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:19 smithi167 ceph-mon[37140]: pgmap v3559: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:20:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:21 smithi100 ceph-mon[32599]: pgmap v3560: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:20:21.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:21 smithi167 ceph-mon[37140]: pgmap v3560: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:20:21.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:21 smithi138 ceph-mon[36954]: pgmap v3560: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:20:23.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:23 smithi100 ceph-mon[32599]: pgmap v3561: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:20:23.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:23 smithi167 ceph-mon[37140]: pgmap v3561: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:20:23.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:23 smithi138 ceph-mon[36954]: pgmap v3561: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:20:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:25 smithi100 ceph-mon[32599]: pgmap v3562: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:20:25.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:25 smithi167 ceph-mon[37140]: pgmap v3562: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:20:25.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:25 smithi138 ceph-mon[36954]: pgmap v3562: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:20:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:27 smithi100 ceph-mon[32599]: pgmap v3563: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:20:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:27 smithi167 ceph-mon[37140]: pgmap v3563: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:20:27.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:27 smithi138 ceph-mon[36954]: pgmap v3563: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:20:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:29 smithi100 ceph-mon[32599]: pgmap v3564: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:20:29.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:29 smithi167 ceph-mon[37140]: pgmap v3564: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:20:29.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:29 smithi138 ceph-mon[36954]: pgmap v3564: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:20:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:31 smithi100 ceph-mon[32599]: pgmap v3565: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:20:31.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:31 smithi167 ceph-mon[37140]: pgmap v3565: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:20:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:31 smithi138 ceph-mon[36954]: pgmap v3565: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:20:32.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:20:32.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:20:32.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:20:33.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:20:33.337+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:20:33.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:20:33.337+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:20:33.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:20:33.337+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:20:33.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:20:33.337+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:20:33.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:20:33.338+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:20:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:33 smithi100 ceph-mon[32599]: pgmap v3566: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:20:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:20:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:20:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:20:33.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:20:33.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:20:33.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:20:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:33 smithi167 ceph-mon[37140]: pgmap v3566: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:20:33.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:20:33.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:20:33.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:20:33.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:20:33.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:20:33.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:20:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:33 smithi138 ceph-mon[36954]: pgmap v3566: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:20:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:20:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:20:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:20:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:20:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:20:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:20:34.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:20:34.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:20:34.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:20:34.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:20:34.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:20:34.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:20:34.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:20:34.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:20:34.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:20:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:35 smithi100 ceph-mon[32599]: pgmap v3567: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:20:35.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:35 smithi167 ceph-mon[37140]: pgmap v3567: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:20:35.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:35 smithi138 ceph-mon[36954]: pgmap v3567: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:20:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:37 smithi100 ceph-mon[32599]: pgmap v3568: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:20:37.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:37 smithi167 ceph-mon[37140]: pgmap v3568: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:20:37.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:37 smithi138 ceph-mon[36954]: pgmap v3568: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:20:39.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:39 smithi100 ceph-mon[32599]: pgmap v3569: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:20:39.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:39 smithi167 ceph-mon[37140]: pgmap v3569: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:20:39.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:39 smithi138 ceph-mon[36954]: pgmap v3569: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:20:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:41 smithi100 ceph-mon[32599]: pgmap v3570: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:20:41.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:41 smithi167 ceph-mon[37140]: pgmap v3570: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:20:41.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:41 smithi138 ceph-mon[36954]: pgmap v3570: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:20:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:43 smithi100 ceph-mon[32599]: pgmap v3571: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:20:43.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:43 smithi167 ceph-mon[37140]: pgmap v3571: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:20:43.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:43 smithi138 ceph-mon[36954]: pgmap v3571: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:20:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:45 smithi100 ceph-mon[32599]: pgmap v3572: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:20:45.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:45 smithi167 ceph-mon[37140]: pgmap v3572: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:20:45.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:45 smithi138 ceph-mon[36954]: pgmap v3572: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:20:47.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:47 smithi100 ceph-mon[32599]: pgmap v3573: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:20:47.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:47 smithi167 ceph-mon[37140]: pgmap v3573: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:20:47.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:47 smithi138 ceph-mon[36954]: pgmap v3573: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:20:48.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:20:48.353+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:20:48.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:20:48.354+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:20:48.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:20:48.354+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:20:48.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:20:48.354+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:20:48.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:20:48.354+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:20:49.743 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:49 smithi138 ceph-mon[36954]: pgmap v3574: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:20:49.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:49 smithi100 ceph-mon[32599]: pgmap v3574: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:20:49.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:49 smithi167 ceph-mon[37140]: pgmap v3574: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:20:51.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:51 smithi100 ceph-mon[32599]: pgmap v3575: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:20:51.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:51 smithi167 ceph-mon[37140]: pgmap v3575: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:20:51.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:51 smithi138 ceph-mon[36954]: pgmap v3575: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:20:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:53 smithi100 ceph-mon[32599]: pgmap v3576: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:20:53.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:53 smithi167 ceph-mon[37140]: pgmap v3576: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:20:53.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:53 smithi138 ceph-mon[36954]: pgmap v3576: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:20:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:55 smithi100 ceph-mon[32599]: pgmap v3577: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:20:55.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:55 smithi167 ceph-mon[37140]: pgmap v3577: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:20:55.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:55 smithi138 ceph-mon[36954]: pgmap v3577: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:20:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:57 smithi100 ceph-mon[32599]: pgmap v3578: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:20:57.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:57 smithi167 ceph-mon[37140]: pgmap v3578: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:20:57.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:57 smithi138 ceph-mon[36954]: pgmap v3578: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:20:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:20:59 smithi100 ceph-mon[32599]: pgmap v3579: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:20:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:20:59 smithi167 ceph-mon[37140]: pgmap v3579: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:20:59.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:20:59 smithi138 ceph-mon[36954]: pgmap v3579: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:21:00.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:00 smithi167 ceph-mon[37140]: pgmap v3580: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:21:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:00 smithi138 ceph-mon[36954]: pgmap v3580: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:21:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:00 smithi100 ceph-mon[32599]: pgmap v3580: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:21:03.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:21:03.359+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:21:03.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:21:03.360+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:21:03.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:21:03.360+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:21:03.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:21:03.360+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:21:03.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:21:03.360+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:21:03.697 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:03 smithi100 ceph-mon[32599]: pgmap v3581: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:21:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:03 smithi167 ceph-mon[37140]: pgmap v3581: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:21:03.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:21:03.801+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T15:21:03.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:21:03.801+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T15:21:03.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:21:03.801+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 47 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T15:21:03.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:21:03.801+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 47 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T15:21:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:03 smithi138 ceph-mon[36954]: pgmap v3581: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:21:05.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:05 smithi100 ceph-mon[32599]: pgmap v3582: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:21:05.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:05 smithi167 ceph-mon[37140]: pgmap v3582: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:21:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:05 smithi138 ceph-mon[36954]: pgmap v3582: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:21:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:07 smithi100 ceph-mon[32599]: pgmap v3583: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:21:07.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:07 smithi167 ceph-mon[37140]: pgmap v3583: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:21:07.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:07 smithi138 ceph-mon[36954]: pgmap v3583: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:21:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:08 smithi100 ceph-mon[32599]: osdmap e272: 12 total, 12 up, 12 in 2024-07-26T15:21:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:08 smithi167 ceph-mon[37140]: osdmap e272: 12 total, 12 up, 12 in 2024-07-26T15:21:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:08 smithi138 ceph-mon[36954]: osdmap e272: 12 total, 12 up, 12 in 2024-07-26T15:21:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:09 smithi100 ceph-mon[32599]: pgmap v3585: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:21:09.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:09 smithi167 ceph-mon[37140]: pgmap v3585: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:21:09.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:09 smithi138 ceph-mon[36954]: pgmap v3585: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:21:11.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:11 smithi100 ceph-mon[32599]: pgmap v3586: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:21:11.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:11 smithi167 ceph-mon[37140]: pgmap v3586: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:21:11.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:11 smithi138 ceph-mon[36954]: pgmap v3586: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:21:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:13 smithi100 ceph-mon[32599]: pgmap v3587: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:21:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:13 smithi167 ceph-mon[37140]: pgmap v3587: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:21:13.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:13 smithi138 ceph-mon[36954]: pgmap v3587: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:21:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:14 smithi138 ceph-mon[36954]: pgmap v3588: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:21:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:14 smithi100 ceph-mon[32599]: pgmap v3588: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:21:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:14 smithi167 ceph-mon[37140]: pgmap v3588: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:21:16.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:16 smithi100 ceph-mon[32599]: osdmap e273: 12 total, 12 up, 12 in 2024-07-26T15:21:16.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:16 smithi167 ceph-mon[37140]: osdmap e273: 12 total, 12 up, 12 in 2024-07-26T15:21:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:16 smithi138 ceph-mon[36954]: osdmap e273: 12 total, 12 up, 12 in 2024-07-26T15:21:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:17 smithi100 ceph-mon[32599]: pgmap v3590: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:21:17.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:17 smithi167 ceph-mon[37140]: pgmap v3590: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:21:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:17 smithi138 ceph-mon[36954]: pgmap v3590: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:21:18.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:21:18.375+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:21:18.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:21:18.376+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:21:18.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:21:18.376+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:21:18.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:21:18.376+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:21:18.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:21:18.376+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:21:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:19 smithi100 ceph-mon[32599]: pgmap v3591: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:21:19.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:19 smithi167 ceph-mon[37140]: pgmap v3591: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:21:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:19 smithi138 ceph-mon[36954]: pgmap v3591: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:21:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:21 smithi100 ceph-mon[32599]: pgmap v3592: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:21:21.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:21 smithi167 ceph-mon[37140]: pgmap v3592: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:21:21.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:21 smithi138 ceph-mon[36954]: pgmap v3592: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:21:23.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:23 smithi100 ceph-mon[32599]: pgmap v3593: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:21:23.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:23 smithi167 ceph-mon[37140]: pgmap v3593: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:21:23.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:23 smithi138 ceph-mon[36954]: pgmap v3593: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:21:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:25 smithi100 ceph-mon[32599]: pgmap v3594: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:21:25.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:25 smithi167 ceph-mon[37140]: pgmap v3594: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:21:25.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:25 smithi138 ceph-mon[36954]: pgmap v3594: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:21:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:27 smithi100 ceph-mon[32599]: pgmap v3595: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T15:21:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:27 smithi167 ceph-mon[37140]: pgmap v3595: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T15:21:27.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:27 smithi138 ceph-mon[36954]: pgmap v3595: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T15:21:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:29 smithi100 ceph-mon[32599]: pgmap v3596: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:21:29.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:29 smithi167 ceph-mon[37140]: pgmap v3596: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:21:29.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:29 smithi138 ceph-mon[36954]: pgmap v3596: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:21:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:31 smithi100 ceph-mon[32599]: pgmap v3597: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:21:31.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:31 smithi167 ceph-mon[37140]: pgmap v3597: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:21:31.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:31 smithi138 ceph-mon[36954]: pgmap v3597: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:21:33.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:21:33.391+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:21:33.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:21:33.392+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:21:33.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:21:33.392+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:21:33.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:21:33.392+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:21:33.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:21:33.392+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:21:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:33 smithi100 ceph-mon[32599]: pgmap v3598: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:21:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:21:33.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:33 smithi167 ceph-mon[37140]: pgmap v3598: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:21:33.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:21:33.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:33 smithi138 ceph-mon[36954]: pgmap v3598: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:21:33.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:21:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:21:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:21:34.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:21:34.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:21:34.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:21:34.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:21:34.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:21:34.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:21:34.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:21:34.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:34 smithi100 ceph-mon[32599]: pgmap v3599: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:21:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:21:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:21:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:21:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:21:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:21:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:21:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:21:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:21:35.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:21:35.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:34 smithi167 ceph-mon[37140]: pgmap v3599: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:21:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:21:35.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:21:35.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:21:35.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:21:35.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:21:35.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:21:35.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:21:35.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:21:35.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:21:35.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:34 smithi138 ceph-mon[36954]: pgmap v3599: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:21:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:37 smithi100 ceph-mon[32599]: pgmap v3600: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:21:37.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:37 smithi167 ceph-mon[37140]: pgmap v3600: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:21:37.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:37 smithi138 ceph-mon[36954]: pgmap v3600: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:21:39.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:39 smithi100 ceph-mon[32599]: pgmap v3601: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:21:39.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:39 smithi167 ceph-mon[37140]: pgmap v3601: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:21:39.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:39 smithi138 ceph-mon[36954]: pgmap v3601: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:21:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:41 smithi100 ceph-mon[32599]: pgmap v3602: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:21:41.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:41 smithi167 ceph-mon[37140]: pgmap v3602: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:21:41.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:41 smithi138 ceph-mon[36954]: pgmap v3602: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:21:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:43 smithi100 ceph-mon[32599]: pgmap v3603: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:21:43.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:43 smithi167 ceph-mon[37140]: pgmap v3603: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:21:43.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:43 smithi138 ceph-mon[36954]: pgmap v3603: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:21:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:45 smithi100 ceph-mon[32599]: pgmap v3604: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:21:45.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:45 smithi167 ceph-mon[37140]: pgmap v3604: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:21:45.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:45 smithi138 ceph-mon[36954]: pgmap v3604: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:21:47.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:47 smithi100 ceph-mon[32599]: pgmap v3605: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:21:47.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:47 smithi167 ceph-mon[37140]: pgmap v3605: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:21:47.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:47 smithi138 ceph-mon[36954]: pgmap v3605: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:21:48.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:21:48.407+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:21:48.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:21:48.408+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:21:48.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:21:48.408+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:21:48.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:21:48.408+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:21:48.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:21:48.408+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:21:49.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:49 smithi167 ceph-mon[37140]: pgmap v3606: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:21:49.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:49 smithi138 ceph-mon[36954]: pgmap v3606: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:21:49.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:49 smithi100 ceph-mon[32599]: pgmap v3606: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:21:51.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:51 smithi167 ceph-mon[37140]: pgmap v3607: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:21:51.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:51 smithi138 ceph-mon[36954]: pgmap v3607: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:21:51.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:51 smithi100 ceph-mon[32599]: pgmap v3607: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:21:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:52 smithi138 ceph-mon[36954]: pgmap v3608: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:21:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:52 smithi100 ceph-mon[32599]: pgmap v3608: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:21:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:52 smithi167 ceph-mon[37140]: pgmap v3608: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:21:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:55 smithi100 ceph-mon[32599]: pgmap v3609: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:21:55.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:55 smithi167 ceph-mon[37140]: pgmap v3609: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:21:55.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:55 smithi138 ceph-mon[36954]: pgmap v3609: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:21:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:57 smithi100 ceph-mon[32599]: pgmap v3610: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:21:57.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:57 smithi167 ceph-mon[37140]: pgmap v3610: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:21:57.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:57 smithi138 ceph-mon[36954]: pgmap v3610: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:21:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:21:59 smithi100 ceph-mon[32599]: pgmap v3611: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:21:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:21:59 smithi167 ceph-mon[37140]: pgmap v3611: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:21:59.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:21:59 smithi138 ceph-mon[36954]: pgmap v3611: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:22:01.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:22:01 smithi100 ceph-mon[32599]: pgmap v3612: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:22:01.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:22:01 smithi167 ceph-mon[37140]: pgmap v3612: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:22:01.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:22:01 smithi138 ceph-mon[36954]: pgmap v3612: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:22:03.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:22:03.424+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:22:03.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:22:03.424+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:22:03.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:22:03.424+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:22:03.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:22:03.424+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:22:03.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:22:03.424+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:22:03.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:22:03 smithi100 ceph-mon[32599]: pgmap v3613: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:22:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:22:03 smithi167 ceph-mon[37140]: pgmap v3613: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:22:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:22:03 smithi138 ceph-mon[36954]: pgmap v3613: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:22:05.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:22:05 smithi167 ceph-mon[37140]: pgmap v3614: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:22:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:22:05 smithi138 ceph-mon[36954]: pgmap v3614: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:22:05.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:22:05 smithi100 ceph-mon[32599]: pgmap v3614: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:22:07.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:22:07 smithi167 ceph-mon[37140]: pgmap v3615: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:22:07.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:22:07 smithi138 ceph-mon[36954]: pgmap v3615: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:22:07.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:22:07 smithi100 ceph-mon[32599]: pgmap v3615: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:22:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:22:08 smithi167 ceph-mon[37140]: osdmap e274: 12 total, 12 up, 12 in 2024-07-26T15:22:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:22:08 smithi167 ceph-mon[37140]: pgmap v3617: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:22:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:22:08 smithi138 ceph-mon[36954]: osdmap e274: 12 total, 12 up, 12 in 2024-07-26T15:22:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:22:08 smithi138 ceph-mon[36954]: pgmap v3617: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:22:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:22:08 smithi100 ceph-mon[32599]: osdmap e274: 12 total, 12 up, 12 in 2024-07-26T15:22:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:22:08 smithi100 ceph-mon[32599]: pgmap v3617: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:22:11.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:22:11 smithi100 ceph-mon[32599]: pgmap v3618: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:22:11.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:22:11 smithi167 ceph-mon[37140]: pgmap v3618: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:22:11.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:22:11 smithi138 ceph-mon[36954]: pgmap v3618: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:22:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:22:13 smithi100 ceph-mon[32599]: pgmap v3619: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:22:13.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:22:13 smithi167 ceph-mon[37140]: pgmap v3619: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:22:13.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:22:13 smithi138 ceph-mon[36954]: pgmap v3619: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:22:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:22:15 smithi167 ceph-mon[37140]: pgmap v3620: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:22:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:22:15 smithi167 ceph-mon[37140]: osdmap e275: 12 total, 12 up, 12 in 2024-07-26T15:22:15.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:22:15 smithi138 ceph-mon[36954]: pgmap v3620: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:22:15.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:22:15 smithi138 ceph-mon[36954]: osdmap e275: 12 total, 12 up, 12 in 2024-07-26T15:22:15.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:22:15 smithi100 ceph-mon[32599]: pgmap v3620: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:22:15.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:22:15 smithi100 ceph-mon[32599]: osdmap e275: 12 total, 12 up, 12 in 2024-07-26T15:22:17.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:22:17 smithi167 ceph-mon[37140]: pgmap v3622: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:22:17.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:22:17 smithi138 ceph-mon[36954]: pgmap v3622: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:22:17.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:22:17 smithi100 ceph-mon[32599]: pgmap v3622: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:22:18.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:22:18.439+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:22:18.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:22:18.439+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:22:18.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:22:18.440+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:22:18.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:22:18.440+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:22:18.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:22:18.440+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:22:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:22:19 smithi167 ceph-mon[37140]: pgmap v3623: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:22:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:22:19 smithi138 ceph-mon[36954]: pgmap v3623: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:22:19.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:22:19 smithi100 ceph-mon[32599]: pgmap v3623: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:22:21.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:22:21 smithi167 ceph-mon[37140]: pgmap v3624: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:22:21.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:22:21 smithi138 ceph-mon[36954]: pgmap v3624: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:22:21.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:22:21 smithi100 ceph-mon[32599]: pgmap v3624: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:22:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:22:22 smithi138 ceph-mon[36954]: pgmap v3625: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:22:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:22:22 smithi100 ceph-mon[32599]: pgmap v3625: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:22:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:22:22 smithi167 ceph-mon[37140]: pgmap v3625: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:22:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:22:25 smithi100 ceph-mon[32599]: pgmap v3626: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:22:25.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:22:25 smithi167 ceph-mon[37140]: pgmap v3626: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:22:25.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:22:25 smithi138 ceph-mon[36954]: pgmap v3626: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:22:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:22:27 smithi167 ceph-mon[37140]: pgmap v3627: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:22:27.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:22:27 smithi138 ceph-mon[36954]: pgmap v3627: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:22:27.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:22:27 smithi100 ceph-mon[32599]: pgmap v3627: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:22:29.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:22:29 smithi167 ceph-mon[37140]: pgmap v3628: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:22:29.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:22:29 smithi138 ceph-mon[36954]: pgmap v3628: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:22:29.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:22:29 smithi100 ceph-mon[32599]: pgmap v3628: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:22:31.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:22:31 smithi167 ceph-mon[37140]: pgmap v3629: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:22:31.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:22:31 smithi138 ceph-mon[36954]: pgmap v3629: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:22:31.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:22:31 smithi100 ceph-mon[32599]: pgmap v3629: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:22:33.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:22:33.447+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:22:33.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:22:33.448+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:22:33.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:22:33.448+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:22:33.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:22:33.448+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:22:33.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:22:33.448+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:22:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:22:33 smithi100 ceph-mon[32599]: pgmap v3630: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:22:33.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:22:33 smithi167 ceph-mon[37140]: pgmap v3630: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:22:33.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:22:33 smithi138 ceph-mon[36954]: pgmap v3630: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:22:34.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:22:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:22:34.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:22:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:22:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:22:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:22:35.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:22:35 smithi167 ceph-mon[37140]: pgmap v3631: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:22:35.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:22:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:22:35.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:22:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:22:35.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:22:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:22:35.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:22:35 smithi138 ceph-mon[36954]: pgmap v3631: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:22:35.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:22:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:22:35.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:22:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:22:35.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:22:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:22:35.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:22:35 smithi100 ceph-mon[32599]: pgmap v3631: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:22:35.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:22:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:22:35.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:22:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:22:35.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:22:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:22:36.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:22:36 smithi167 ceph-mon[37140]: pgmap v3632: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:22:37.096 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:22:36 smithi100 ceph-mon[32599]: pgmap v3632: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:22:37.096 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:22:36 smithi138 ceph-mon[36954]: pgmap v3632: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:22:39.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:22:39 smithi167 ceph-mon[37140]: pgmap v3633: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:22:39.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:22:39 smithi138 ceph-mon[36954]: pgmap v3633: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:22:39.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:22:39 smithi100 ceph-mon[32599]: pgmap v3633: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:22:41.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:22:41 smithi167 ceph-mon[37140]: pgmap v3634: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:22:41.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:22:41 smithi138 ceph-mon[36954]: pgmap v3634: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:22:41.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:22:41 smithi100 ceph-mon[32599]: pgmap v3634: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:22:43.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:22:43 smithi167 ceph-mon[37140]: pgmap v3635: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:22:43.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:22:43 smithi138 ceph-mon[36954]: pgmap v3635: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:22:43.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:22:43 smithi100 ceph-mon[32599]: pgmap v3635: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:22:45.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:22:45 smithi167 ceph-mon[37140]: pgmap v3636: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:22:45.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:22:45 smithi138 ceph-mon[36954]: pgmap v3636: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:22:45.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:22:45 smithi100 ceph-mon[32599]: pgmap v3636: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:22:47.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:22:47 smithi167 ceph-mon[37140]: pgmap v3637: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:22:47.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:22:47 smithi138 ceph-mon[36954]: pgmap v3637: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:22:47.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:22:47 smithi100 ceph-mon[32599]: pgmap v3637: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:22:48.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:22:48.463+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:22:48.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:22:48.464+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:22:48.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:22:48.464+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:22:48.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:22:48.464+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:22:48.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:22:48.464+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:22:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:22:48 smithi100 ceph-mon[32599]: pgmap v3638: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:22:48.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:22:48 smithi167 ceph-mon[37140]: pgmap v3638: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:22:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:22:48 smithi138 ceph-mon[36954]: pgmap v3638: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:22:51.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:22:51 smithi167 ceph-mon[37140]: pgmap v3639: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:22:51.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:22:51 smithi138 ceph-mon[36954]: pgmap v3639: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:22:51.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:22:51 smithi100 ceph-mon[32599]: pgmap v3639: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:22:53.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:22:53 smithi167 ceph-mon[37140]: pgmap v3640: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:22:53.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:22:53 smithi138 ceph-mon[36954]: pgmap v3640: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:22:53.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:22:53 smithi100 ceph-mon[32599]: pgmap v3640: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:22:55.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:22:55 smithi167 ceph-mon[37140]: pgmap v3641: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:22:55.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:22:55 smithi138 ceph-mon[36954]: pgmap v3641: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:22:55.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:22:55 smithi100 ceph-mon[32599]: pgmap v3641: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:22:57.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:22:57 smithi167 ceph-mon[37140]: pgmap v3642: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:22:57.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:22:57 smithi138 ceph-mon[36954]: pgmap v3642: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:22:57.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:22:57 smithi100 ceph-mon[32599]: pgmap v3642: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:22:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:22:58 smithi138 ceph-mon[36954]: pgmap v3643: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:22:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:22:58 smithi100 ceph-mon[32599]: pgmap v3643: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:22:59.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:22:58 smithi167 ceph-mon[37140]: pgmap v3643: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:23:01.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:23:01 smithi167 ceph-mon[37140]: pgmap v3644: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:23:01.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:23:01 smithi138 ceph-mon[36954]: pgmap v3644: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:23:01.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:23:01 smithi100 ceph-mon[32599]: pgmap v3644: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:23:03.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:23:03.479+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:23:03.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:23:03.480+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:23:03.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:23:03.480+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:23:03.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:23:03.480+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:23:03.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:23:03.480+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:23:03.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:23:03 smithi100 ceph-mon[32599]: pgmap v3645: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:23:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:23:03 smithi167 ceph-mon[37140]: pgmap v3645: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:23:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:23:03 smithi138 ceph-mon[36954]: pgmap v3645: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:23:05.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:23:05 smithi167 ceph-mon[37140]: pgmap v3646: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:23:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:23:05 smithi138 ceph-mon[36954]: pgmap v3646: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:23:05.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:23:05 smithi100 ceph-mon[32599]: pgmap v3646: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:23:07.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:23:07 smithi167 ceph-mon[37140]: pgmap v3647: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:23:07.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:23:07 smithi138 ceph-mon[36954]: pgmap v3647: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:23:07.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:23:07 smithi100 ceph-mon[32599]: pgmap v3647: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:23:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:23:08 smithi138 ceph-mon[36954]: osdmap e276: 12 total, 12 up, 12 in 2024-07-26T15:23:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:23:08 smithi138 ceph-mon[36954]: pgmap v3649: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:23:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:23:08 smithi100 ceph-mon[32599]: osdmap e276: 12 total, 12 up, 12 in 2024-07-26T15:23:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:23:08 smithi100 ceph-mon[32599]: pgmap v3649: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:23:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:23:08 smithi167 ceph-mon[37140]: osdmap e276: 12 total, 12 up, 12 in 2024-07-26T15:23:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:23:08 smithi167 ceph-mon[37140]: pgmap v3649: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:23:11.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:23:11 smithi167 ceph-mon[37140]: pgmap v3650: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:23:11.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:23:11 smithi138 ceph-mon[36954]: pgmap v3650: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:23:11.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:23:11 smithi100 ceph-mon[32599]: pgmap v3650: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:23:13.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:23:13 smithi167 ceph-mon[37140]: pgmap v3651: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:23:13.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:23:13 smithi138 ceph-mon[36954]: pgmap v3651: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:23:13.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:23:13 smithi100 ceph-mon[32599]: pgmap v3651: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:23:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:23:15 smithi167 ceph-mon[37140]: pgmap v3652: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:23:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:23:15 smithi167 ceph-mon[37140]: osdmap e277: 12 total, 12 up, 12 in 2024-07-26T15:23:15.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:23:15 smithi138 ceph-mon[36954]: pgmap v3652: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:23:15.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:23:15 smithi138 ceph-mon[36954]: osdmap e277: 12 total, 12 up, 12 in 2024-07-26T15:23:15.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:23:15 smithi100 ceph-mon[32599]: pgmap v3652: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:23:15.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:23:15 smithi100 ceph-mon[32599]: osdmap e277: 12 total, 12 up, 12 in 2024-07-26T15:23:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:23:16 smithi138 ceph-mon[36954]: pgmap v3654: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:23:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:23:16 smithi100 ceph-mon[32599]: pgmap v3654: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:23:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:23:16 smithi167 ceph-mon[37140]: pgmap v3654: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:23:18.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:23:18.495+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:23:18.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:23:18.496+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:23:18.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:23:18.496+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:23:18.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:23:18.496+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:23:18.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:23:18.496+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:23:19.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:23:19 smithi167 ceph-mon[37140]: pgmap v3655: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:23:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:23:19 smithi138 ceph-mon[36954]: pgmap v3655: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:23:19.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:23:19 smithi100 ceph-mon[32599]: pgmap v3655: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:23:21.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:23:21 smithi167 ceph-mon[37140]: pgmap v3656: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:23:21.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:23:21 smithi138 ceph-mon[36954]: pgmap v3656: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:23:21.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:23:21 smithi100 ceph-mon[32599]: pgmap v3656: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:23:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:23:22 smithi138 ceph-mon[36954]: pgmap v3657: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:23:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:23:22 smithi100 ceph-mon[32599]: pgmap v3657: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:23:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:23:22 smithi167 ceph-mon[37140]: pgmap v3657: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:23:25.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:23:25 smithi167 ceph-mon[37140]: pgmap v3658: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:23:25.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:23:25 smithi138 ceph-mon[36954]: pgmap v3658: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:23:25.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:23:25 smithi100 ceph-mon[32599]: pgmap v3658: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:23:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:23:27 smithi167 ceph-mon[37140]: pgmap v3659: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T15:23:27.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:23:27 smithi138 ceph-mon[36954]: pgmap v3659: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T15:23:27.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:23:27 smithi100 ceph-mon[32599]: pgmap v3659: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T15:23:28.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:23:28 smithi167 ceph-mon[37140]: pgmap v3660: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:23:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:23:28 smithi138 ceph-mon[36954]: pgmap v3660: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:23:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:23:28 smithi100 ceph-mon[32599]: pgmap v3660: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:23:31.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:23:31 smithi167 ceph-mon[37140]: pgmap v3661: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:23:31.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:23:31 smithi138 ceph-mon[36954]: pgmap v3661: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:23:31.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:23:31 smithi100 ceph-mon[32599]: pgmap v3661: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:23:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:23:32 smithi138 ceph-mon[36954]: pgmap v3662: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:23:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:23:32 smithi100 ceph-mon[32599]: pgmap v3662: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:23:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:23:32 smithi167 ceph-mon[37140]: pgmap v3662: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:23:33.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:23:33.511+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:23:33.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:23:33.512+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:23:33.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:23:33.512+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:23:33.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:23:33.512+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:23:33.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:23:33.512+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:23:35.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:23:35 smithi167 ceph-mon[37140]: pgmap v3663: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:23:35.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:23:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:23:35.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:23:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:23:35.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:23:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:23:35.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:23:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:23:35.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:23:35 smithi138 ceph-mon[36954]: pgmap v3663: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:23:35.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:23:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:23:35.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:23:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:23:35.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:23:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:23:35.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:23:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:23:35.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:23:35 smithi100 ceph-mon[32599]: pgmap v3663: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:23:35.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:23:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:23:35.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:23:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:23:35.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:23:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:23:35.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:23:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:23:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:23:36 smithi138 ceph-mon[36954]: pgmap v3664: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:23:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:23:36 smithi100 ceph-mon[32599]: pgmap v3664: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:23:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:23:36 smithi167 ceph-mon[37140]: pgmap v3664: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:23:39.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:23:39 smithi167 ceph-mon[37140]: pgmap v3665: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:23:39.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:23:39 smithi138 ceph-mon[36954]: pgmap v3665: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:23:39.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:23:39 smithi100 ceph-mon[32599]: pgmap v3665: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:23:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:23:40 smithi138 ceph-mon[36954]: pgmap v3666: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:23:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:23:40 smithi100 ceph-mon[32599]: pgmap v3666: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:23:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:23:40 smithi167 ceph-mon[37140]: pgmap v3666: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:23:43.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:23:43 smithi167 ceph-mon[37140]: pgmap v3667: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:23:43.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:23:43 smithi138 ceph-mon[36954]: pgmap v3667: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:23:43.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:23:43 smithi100 ceph-mon[32599]: pgmap v3667: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:23:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:23:44 smithi138 ceph-mon[36954]: pgmap v3668: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:23:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:23:44 smithi100 ceph-mon[32599]: pgmap v3668: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:23:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:23:44 smithi167 ceph-mon[37140]: pgmap v3668: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:23:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:23:46 smithi138 ceph-mon[36954]: pgmap v3669: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:23:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:23:46 smithi100 ceph-mon[32599]: pgmap v3669: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:23:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:23:46 smithi167 ceph-mon[37140]: pgmap v3669: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:23:48.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:23:48.527+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:23:48.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:23:48.528+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:23:48.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:23:48.528+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:23:48.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:23:48.528+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:23:48.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:23:48.528+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:23:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:23:48 smithi138 ceph-mon[36954]: pgmap v3670: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:23:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:23:48 smithi100 ceph-mon[32599]: pgmap v3670: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:23:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:23:48 smithi167 ceph-mon[37140]: pgmap v3670: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:23:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:23:50 smithi138 ceph-mon[36954]: pgmap v3671: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:23:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:23:50 smithi100 ceph-mon[32599]: pgmap v3671: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:23:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:23:50 smithi167 ceph-mon[37140]: pgmap v3671: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:23:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:23:52 smithi138 ceph-mon[36954]: pgmap v3672: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:23:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:23:52 smithi100 ceph-mon[32599]: pgmap v3672: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:23:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:23:52 smithi167 ceph-mon[37140]: pgmap v3672: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:23:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:23:54 smithi138 ceph-mon[36954]: pgmap v3673: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:23:54.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:23:54 smithi100 ceph-mon[32599]: pgmap v3673: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:23:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:23:54 smithi167 ceph-mon[37140]: pgmap v3673: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:23:56.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:23:56 smithi167 ceph-mon[37140]: pgmap v3674: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:23:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:23:56 smithi138 ceph-mon[36954]: pgmap v3674: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:23:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:23:56 smithi100 ceph-mon[32599]: pgmap v3674: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:23:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:23:58 smithi138 ceph-mon[36954]: pgmap v3675: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:23:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:23:58 smithi100 ceph-mon[32599]: pgmap v3675: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:23:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:23:58 smithi167 ceph-mon[37140]: pgmap v3675: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:24:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:24:00 smithi138 ceph-mon[36954]: pgmap v3676: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:24:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:24:00 smithi100 ceph-mon[32599]: pgmap v3676: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:24:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:24:00 smithi167 ceph-mon[37140]: pgmap v3676: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:24:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:24:02 smithi138 ceph-mon[36954]: pgmap v3677: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-07-26T15:24:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:24:02 smithi100 ceph-mon[32599]: pgmap v3677: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-07-26T15:24:03.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:24:02 smithi167 ceph-mon[37140]: pgmap v3677: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-07-26T15:24:03.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:24:03.530+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:24:03.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:24:03.530+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:24:03.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:24:03.530+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:24:03.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:24:03.530+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:24:03.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:24:03.530+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:24:03.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:24:03.827+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T15:24:03.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:24:03.827+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T15:24:03.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:24:03.827+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 48 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T15:24:03.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:24:03.827+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 48 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T15:24:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:24:04 smithi138 ceph-mon[36954]: pgmap v3678: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:24:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:24:04 smithi100 ceph-mon[32599]: pgmap v3678: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:24:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:24:04 smithi167 ceph-mon[37140]: pgmap v3678: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:24:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:24:06 smithi138 ceph-mon[36954]: pgmap v3679: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:24:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:24:06 smithi100 ceph-mon[32599]: pgmap v3679: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:24:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:24:06 smithi167 ceph-mon[37140]: pgmap v3679: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:24:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:24:08 smithi138 ceph-mon[36954]: osdmap e278: 12 total, 12 up, 12 in 2024-07-26T15:24:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:24:08 smithi138 ceph-mon[36954]: pgmap v3681: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:24:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:24:08 smithi100 ceph-mon[32599]: osdmap e278: 12 total, 12 up, 12 in 2024-07-26T15:24:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:24:08 smithi100 ceph-mon[32599]: pgmap v3681: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:24:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:24:08 smithi167 ceph-mon[37140]: osdmap e278: 12 total, 12 up, 12 in 2024-07-26T15:24:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:24:08 smithi167 ceph-mon[37140]: pgmap v3681: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:24:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:24:10 smithi138 ceph-mon[36954]: pgmap v3682: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:24:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:24:10 smithi100 ceph-mon[32599]: pgmap v3682: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:24:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:24:10 smithi167 ceph-mon[37140]: pgmap v3682: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:24:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:24:12 smithi138 ceph-mon[36954]: pgmap v3683: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:24:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:24:12 smithi100 ceph-mon[32599]: pgmap v3683: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:24:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:24:12 smithi167 ceph-mon[37140]: pgmap v3683: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:24:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:24:14 smithi138 ceph-mon[36954]: pgmap v3684: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:24:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:24:14 smithi100 ceph-mon[32599]: pgmap v3684: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:24:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:24:14 smithi167 ceph-mon[37140]: pgmap v3684: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:24:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:24:16 smithi138 ceph-mon[36954]: osdmap e279: 12 total, 12 up, 12 in 2024-07-26T15:24:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:24:16 smithi100 ceph-mon[32599]: osdmap e279: 12 total, 12 up, 12 in 2024-07-26T15:24:16.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:24:16 smithi167 ceph-mon[37140]: osdmap e279: 12 total, 12 up, 12 in 2024-07-26T15:24:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:24:17 smithi138 ceph-mon[36954]: pgmap v3686: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T15:24:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:24:17 smithi100 ceph-mon[32599]: pgmap v3686: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T15:24:17.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:24:17 smithi167 ceph-mon[37140]: pgmap v3686: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T15:24:18.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:24:18.546+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:24:18.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:24:18.546+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:24:18.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:24:18.546+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:24:18.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:24:18.546+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:24:18.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:24:18.546+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:24:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:24:18 smithi138 ceph-mon[36954]: pgmap v3687: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:24:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:24:18 smithi100 ceph-mon[32599]: pgmap v3687: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:24:19.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:24:18 smithi167 ceph-mon[37140]: pgmap v3687: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:24:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:24:20 smithi138 ceph-mon[36954]: pgmap v3688: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:24:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:24:20 smithi100 ceph-mon[32599]: pgmap v3688: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:24:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:24:20 smithi167 ceph-mon[37140]: pgmap v3688: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:24:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:24:22 smithi138 ceph-mon[36954]: pgmap v3689: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:24:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:24:22 smithi100 ceph-mon[32599]: pgmap v3689: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:24:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:24:22 smithi167 ceph-mon[37140]: pgmap v3689: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:24:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:24:24 smithi138 ceph-mon[36954]: pgmap v3690: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:24:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:24:24 smithi100 ceph-mon[32599]: pgmap v3690: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:24:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:24:24 smithi167 ceph-mon[37140]: pgmap v3690: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:24:26.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:24:26 smithi167 ceph-mon[37140]: pgmap v3691: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:24:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:24:26 smithi138 ceph-mon[36954]: pgmap v3691: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:24:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:24:26 smithi100 ceph-mon[32599]: pgmap v3691: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:24:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:24:28 smithi138 ceph-mon[36954]: pgmap v3692: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:24:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:24:28 smithi100 ceph-mon[32599]: pgmap v3692: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:24:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:24:28 smithi167 ceph-mon[37140]: pgmap v3692: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:24:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:24:31 smithi138 ceph-mon[36954]: pgmap v3693: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:24:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:24:31 smithi100 ceph-mon[32599]: pgmap v3693: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:24:31.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:24:31 smithi167 ceph-mon[37140]: pgmap v3693: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:24:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:24:32 smithi138 ceph-mon[36954]: pgmap v3694: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:24:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:24:32 smithi100 ceph-mon[32599]: pgmap v3694: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:24:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:24:32 smithi167 ceph-mon[37140]: pgmap v3694: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:24:33.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:24:33.562+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:24:33.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:24:33.562+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:24:33.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:24:33.562+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:24:33.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:24:33.562+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:24:33.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:24:33.562+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:24:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:24:34 smithi138 ceph-mon[36954]: pgmap v3695: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:24:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:24:34 smithi100 ceph-mon[32599]: pgmap v3695: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:24:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:24:34 smithi167 ceph-mon[37140]: pgmap v3695: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:24:35.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:24:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:24:35.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:24:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:24:36.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:24:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:24:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:24:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:24:36.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:24:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:24:36.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:24:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:24:36.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:24:36 smithi138 ceph-mon[36954]: pgmap v3696: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:24:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:24:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:24:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:24:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:24:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:24:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:24:36.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:24:36 smithi100 ceph-mon[32599]: pgmap v3696: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:24:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:24:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:24:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:24:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:24:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:24:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:24:37.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:24:36 smithi167 ceph-mon[37140]: pgmap v3696: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:24:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:24:38 smithi138 ceph-mon[36954]: pgmap v3697: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:24:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:24:38 smithi100 ceph-mon[32599]: pgmap v3697: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:24:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:24:38 smithi167 ceph-mon[37140]: pgmap v3697: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:24:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:24:40 smithi138 ceph-mon[36954]: pgmap v3698: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:24:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:24:40 smithi100 ceph-mon[32599]: pgmap v3698: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:24:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:24:40 smithi167 ceph-mon[37140]: pgmap v3698: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:24:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:24:42 smithi138 ceph-mon[36954]: pgmap v3699: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:24:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:24:42 smithi100 ceph-mon[32599]: pgmap v3699: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:24:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:24:42 smithi167 ceph-mon[37140]: pgmap v3699: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:24:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:24:44 smithi138 ceph-mon[36954]: pgmap v3700: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:24:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:24:44 smithi100 ceph-mon[32599]: pgmap v3700: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:24:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:24:44 smithi167 ceph-mon[37140]: pgmap v3700: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:24:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:24:46 smithi138 ceph-mon[36954]: pgmap v3701: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:24:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:24:46 smithi100 ceph-mon[32599]: pgmap v3701: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:24:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:24:46 smithi167 ceph-mon[37140]: pgmap v3701: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:24:48.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:24:48.577+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:24:48.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:24:48.578+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:24:48.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:24:48.578+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:24:48.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:24:48.578+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:24:48.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:24:48.578+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:24:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:24:48 smithi138 ceph-mon[36954]: pgmap v3702: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:24:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:24:48 smithi100 ceph-mon[32599]: pgmap v3702: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:24:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:24:48 smithi167 ceph-mon[37140]: pgmap v3702: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:24:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:24:50 smithi138 ceph-mon[36954]: pgmap v3703: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:24:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:24:50 smithi100 ceph-mon[32599]: pgmap v3703: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:24:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:24:50 smithi167 ceph-mon[37140]: pgmap v3703: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:24:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:24:52 smithi138 ceph-mon[36954]: pgmap v3704: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:24:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:24:52 smithi100 ceph-mon[32599]: pgmap v3704: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:24:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:24:52 smithi167 ceph-mon[37140]: pgmap v3704: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:24:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:24:54 smithi138 ceph-mon[36954]: pgmap v3705: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:24:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:24:54 smithi100 ceph-mon[32599]: pgmap v3705: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:24:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:24:54 smithi167 ceph-mon[37140]: pgmap v3705: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:24:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:24:56 smithi138 ceph-mon[36954]: pgmap v3706: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:24:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:24:56 smithi100 ceph-mon[32599]: pgmap v3706: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:24:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:24:56 smithi167 ceph-mon[37140]: pgmap v3706: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:24:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:24:58 smithi138 ceph-mon[36954]: pgmap v3707: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:24:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:24:58 smithi100 ceph-mon[32599]: pgmap v3707: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:24:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:24:58 smithi167 ceph-mon[37140]: pgmap v3707: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:25:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:25:00 smithi138 ceph-mon[36954]: pgmap v3708: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:25:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:25:00 smithi100 ceph-mon[32599]: pgmap v3708: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:25:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:25:00 smithi167 ceph-mon[37140]: pgmap v3708: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:25:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:25:02 smithi138 ceph-mon[36954]: pgmap v3709: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:25:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:25:02 smithi100 ceph-mon[32599]: pgmap v3709: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:25:03.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:25:02 smithi167 ceph-mon[37140]: pgmap v3709: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:25:03.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:25:03.593+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:25:03.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:25:03.594+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:25:03.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:25:03.594+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:25:03.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:25:03.594+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:25:03.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:25:03.594+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:25:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:25:04 smithi138 ceph-mon[36954]: pgmap v3710: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:25:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:25:04 smithi100 ceph-mon[32599]: pgmap v3710: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:25:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:25:04 smithi167 ceph-mon[37140]: pgmap v3710: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:25:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:25:06 smithi138 ceph-mon[36954]: pgmap v3711: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:25:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:25:06 smithi100 ceph-mon[32599]: pgmap v3711: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:25:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:25:06 smithi167 ceph-mon[37140]: pgmap v3711: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:25:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:25:08 smithi138 ceph-mon[36954]: osdmap e280: 12 total, 12 up, 12 in 2024-07-26T15:25:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:25:08 smithi138 ceph-mon[36954]: pgmap v3713: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:25:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:25:08 smithi100 ceph-mon[32599]: osdmap e280: 12 total, 12 up, 12 in 2024-07-26T15:25:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:25:08 smithi100 ceph-mon[32599]: pgmap v3713: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:25:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:25:08 smithi167 ceph-mon[37140]: osdmap e280: 12 total, 12 up, 12 in 2024-07-26T15:25:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:25:08 smithi167 ceph-mon[37140]: pgmap v3713: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:25:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:25:10 smithi138 ceph-mon[36954]: pgmap v3714: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:25:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:25:10 smithi100 ceph-mon[32599]: pgmap v3714: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:25:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:25:10 smithi167 ceph-mon[37140]: pgmap v3714: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:25:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:25:12 smithi138 ceph-mon[36954]: pgmap v3715: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:25:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:25:12 smithi100 ceph-mon[32599]: pgmap v3715: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:25:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:25:12 smithi167 ceph-mon[37140]: pgmap v3715: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:25:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:25:14 smithi138 ceph-mon[36954]: pgmap v3716: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:25:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:25:14 smithi100 ceph-mon[32599]: pgmap v3716: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:25:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:25:14 smithi167 ceph-mon[37140]: pgmap v3716: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:25:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:25:16 smithi138 ceph-mon[36954]: osdmap e281: 12 total, 12 up, 12 in 2024-07-26T15:25:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:25:16 smithi100 ceph-mon[32599]: osdmap e281: 12 total, 12 up, 12 in 2024-07-26T15:25:16.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:25:16 smithi167 ceph-mon[37140]: osdmap e281: 12 total, 12 up, 12 in 2024-07-26T15:25:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:25:17 smithi138 ceph-mon[36954]: pgmap v3718: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:25:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:25:17 smithi100 ceph-mon[32599]: pgmap v3718: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:25:17.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:25:17 smithi167 ceph-mon[37140]: pgmap v3718: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:25:18.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:25:18.598+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:25:18.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:25:18.599+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:25:18.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:25:18.599+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:25:18.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:25:18.599+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:25:18.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:25:18.599+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:25:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:25:18 smithi138 ceph-mon[36954]: pgmap v3719: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:25:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:25:18 smithi100 ceph-mon[32599]: pgmap v3719: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:25:19.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:25:18 smithi167 ceph-mon[37140]: pgmap v3719: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:25:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:25:20 smithi138 ceph-mon[36954]: pgmap v3720: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:25:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:25:20 smithi100 ceph-mon[32599]: pgmap v3720: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:25:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:25:20 smithi167 ceph-mon[37140]: pgmap v3720: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:25:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:25:22 smithi138 ceph-mon[36954]: pgmap v3721: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:25:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:25:22 smithi100 ceph-mon[32599]: pgmap v3721: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:25:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:25:22 smithi167 ceph-mon[37140]: pgmap v3721: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:25:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:25:24 smithi138 ceph-mon[36954]: pgmap v3722: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:25:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:25:24 smithi100 ceph-mon[32599]: pgmap v3722: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:25:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:25:24 smithi167 ceph-mon[37140]: pgmap v3722: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:25:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:25:26 smithi138 ceph-mon[36954]: pgmap v3723: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:25:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:25:26 smithi100 ceph-mon[32599]: pgmap v3723: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:25:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:25:26 smithi167 ceph-mon[37140]: pgmap v3723: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:25:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:25:28 smithi138 ceph-mon[36954]: pgmap v3724: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:25:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:25:28 smithi100 ceph-mon[32599]: pgmap v3724: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:25:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:25:28 smithi167 ceph-mon[37140]: pgmap v3724: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:25:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:25:30 smithi138 ceph-mon[36954]: pgmap v3725: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:25:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:25:30 smithi100 ceph-mon[32599]: pgmap v3725: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:25:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:25:30 smithi167 ceph-mon[37140]: pgmap v3725: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:25:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:25:32 smithi138 ceph-mon[36954]: pgmap v3726: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:25:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:25:32 smithi100 ceph-mon[32599]: pgmap v3726: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:25:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:25:32 smithi167 ceph-mon[37140]: pgmap v3726: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:25:33.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:25:33.609+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:25:33.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:25:33.610+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:25:33.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:25:33.610+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:25:33.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:25:33.610+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:25:33.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:25:33.610+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:25:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:25:34 smithi138 ceph-mon[36954]: pgmap v3727: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:25:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:25:34 smithi100 ceph-mon[32599]: pgmap v3727: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:25:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:25:34 smithi167 ceph-mon[37140]: pgmap v3727: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:25:36.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:25:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:25:36.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:25:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:25:36.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:25:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:25:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:25:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:25:37.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:25:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:25:37.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:25:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:25:37.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:25:36 smithi138 ceph-mon[36954]: pgmap v3728: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:25:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:25:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:25:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:25:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:25:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:25:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:25:37.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:25:36 smithi100 ceph-mon[32599]: pgmap v3728: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:25:37.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:25:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:25:37.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:25:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:25:37.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:25:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:25:37.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:25:36 smithi167 ceph-mon[37140]: pgmap v3728: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:25:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:25:38 smithi138 ceph-mon[36954]: pgmap v3729: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:25:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:25:38 smithi100 ceph-mon[32599]: pgmap v3729: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:25:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:25:38 smithi167 ceph-mon[37140]: pgmap v3729: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:25:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:25:40 smithi138 ceph-mon[36954]: pgmap v3730: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:25:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:25:40 smithi100 ceph-mon[32599]: pgmap v3730: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:25:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:25:40 smithi167 ceph-mon[37140]: pgmap v3730: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:25:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:25:42 smithi138 ceph-mon[36954]: pgmap v3731: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:25:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:25:42 smithi100 ceph-mon[32599]: pgmap v3731: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:25:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:25:42 smithi167 ceph-mon[37140]: pgmap v3731: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:25:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:25:44 smithi138 ceph-mon[36954]: pgmap v3732: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:25:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:25:44 smithi100 ceph-mon[32599]: pgmap v3732: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:25:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:25:44 smithi167 ceph-mon[37140]: pgmap v3732: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:25:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:25:46 smithi138 ceph-mon[36954]: pgmap v3733: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:25:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:25:46 smithi100 ceph-mon[32599]: pgmap v3733: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:25:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:25:46 smithi167 ceph-mon[37140]: pgmap v3733: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:25:48.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:25:48.617+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:25:48.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:25:48.617+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:25:48.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:25:48.617+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:25:48.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:25:48.617+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:25:48.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:25:48.617+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:25:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:25:48 smithi138 ceph-mon[36954]: pgmap v3734: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:25:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:25:48 smithi100 ceph-mon[32599]: pgmap v3734: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:25:49.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:25:48 smithi167 ceph-mon[37140]: pgmap v3734: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:25:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:25:50 smithi138 ceph-mon[36954]: pgmap v3735: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:25:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:25:50 smithi100 ceph-mon[32599]: pgmap v3735: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:25:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:25:50 smithi167 ceph-mon[37140]: pgmap v3735: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:25:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:25:52 smithi138 ceph-mon[36954]: pgmap v3736: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:25:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:25:52 smithi100 ceph-mon[32599]: pgmap v3736: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:25:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:25:52 smithi167 ceph-mon[37140]: pgmap v3736: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:25:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:25:54 smithi138 ceph-mon[36954]: pgmap v3737: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:25:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:25:54 smithi100 ceph-mon[32599]: pgmap v3737: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:25:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:25:54 smithi167 ceph-mon[37140]: pgmap v3737: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:25:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:25:56 smithi138 ceph-mon[36954]: pgmap v3738: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:25:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:25:56 smithi100 ceph-mon[32599]: pgmap v3738: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:25:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:25:56 smithi167 ceph-mon[37140]: pgmap v3738: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:25:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:25:58 smithi138 ceph-mon[36954]: pgmap v3739: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:25:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:25:58 smithi100 ceph-mon[32599]: pgmap v3739: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:25:59.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:25:58 smithi167 ceph-mon[37140]: pgmap v3739: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:26:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:26:00 smithi138 ceph-mon[36954]: pgmap v3740: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:26:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:26:00 smithi100 ceph-mon[32599]: pgmap v3740: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:26:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:26:00 smithi167 ceph-mon[37140]: pgmap v3740: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:26:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:26:02 smithi138 ceph-mon[36954]: pgmap v3741: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:26:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:26:02 smithi100 ceph-mon[32599]: pgmap v3741: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:26:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:26:02 smithi167 ceph-mon[37140]: pgmap v3741: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:26:03.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:26:03.630+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:26:03.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:26:03.631+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:26:03.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:26:03.631+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:26:03.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:26:03.631+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:26:03.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:26:03.631+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:26:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:26:04 smithi138 ceph-mon[36954]: pgmap v3742: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:26:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:26:04 smithi100 ceph-mon[32599]: pgmap v3742: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:26:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:26:04 smithi167 ceph-mon[37140]: pgmap v3742: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:26:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:26:06 smithi138 ceph-mon[36954]: pgmap v3743: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:26:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:26:06 smithi100 ceph-mon[32599]: pgmap v3743: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:26:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:26:06 smithi167 ceph-mon[37140]: pgmap v3743: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:26:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:26:08 smithi138 ceph-mon[36954]: osdmap e282: 12 total, 12 up, 12 in 2024-07-26T15:26:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:26:08 smithi138 ceph-mon[36954]: pgmap v3745: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:26:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:26:08 smithi100 ceph-mon[32599]: osdmap e282: 12 total, 12 up, 12 in 2024-07-26T15:26:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:26:08 smithi100 ceph-mon[32599]: pgmap v3745: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:26:09.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:26:08 smithi167 ceph-mon[37140]: osdmap e282: 12 total, 12 up, 12 in 2024-07-26T15:26:09.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:26:08 smithi167 ceph-mon[37140]: pgmap v3745: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:26:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:26:10 smithi138 ceph-mon[36954]: pgmap v3746: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:26:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:26:10 smithi100 ceph-mon[32599]: pgmap v3746: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:26:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:26:10 smithi167 ceph-mon[37140]: pgmap v3746: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:26:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:26:12 smithi138 ceph-mon[36954]: pgmap v3747: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:26:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:26:12 smithi100 ceph-mon[32599]: pgmap v3747: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:26:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:26:12 smithi167 ceph-mon[37140]: pgmap v3747: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:26:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:26:14 smithi138 ceph-mon[36954]: pgmap v3748: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:26:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:26:14 smithi100 ceph-mon[32599]: pgmap v3748: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:26:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:26:14 smithi167 ceph-mon[37140]: pgmap v3748: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:26:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:26:16 smithi138 ceph-mon[36954]: osdmap e283: 12 total, 12 up, 12 in 2024-07-26T15:26:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:26:16 smithi100 ceph-mon[32599]: osdmap e283: 12 total, 12 up, 12 in 2024-07-26T15:26:16.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:26:16 smithi167 ceph-mon[37140]: osdmap e283: 12 total, 12 up, 12 in 2024-07-26T15:26:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:26:17 smithi138 ceph-mon[36954]: pgmap v3750: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:26:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:26:17 smithi100 ceph-mon[32599]: pgmap v3750: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:26:17.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:26:17 smithi167 ceph-mon[37140]: pgmap v3750: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:26:18.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:26:18.642+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:26:18.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:26:18.642+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:26:18.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:26:18.642+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:26:18.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:26:18.642+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:26:18.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:26:18.642+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:26:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:26:18 smithi138 ceph-mon[36954]: pgmap v3751: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:26:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:26:18 smithi100 ceph-mon[32599]: pgmap v3751: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:26:19.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:26:18 smithi167 ceph-mon[37140]: pgmap v3751: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:26:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:26:20 smithi138 ceph-mon[36954]: pgmap v3752: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:26:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:26:20 smithi100 ceph-mon[32599]: pgmap v3752: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:26:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:26:20 smithi167 ceph-mon[37140]: pgmap v3752: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:26:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:26:22 smithi138 ceph-mon[36954]: pgmap v3753: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:26:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:26:22 smithi100 ceph-mon[32599]: pgmap v3753: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:26:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:26:22 smithi167 ceph-mon[37140]: pgmap v3753: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:26:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:26:24 smithi138 ceph-mon[36954]: pgmap v3754: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:26:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:26:24 smithi100 ceph-mon[32599]: pgmap v3754: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:26:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:26:24 smithi167 ceph-mon[37140]: pgmap v3754: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:26:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:26:26 smithi138 ceph-mon[36954]: pgmap v3755: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:26:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:26:26 smithi100 ceph-mon[32599]: pgmap v3755: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:26:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:26:26 smithi167 ceph-mon[37140]: pgmap v3755: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:26:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:26:28 smithi138 ceph-mon[36954]: pgmap v3756: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:26:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:26:28 smithi100 ceph-mon[32599]: pgmap v3756: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:26:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:26:28 smithi167 ceph-mon[37140]: pgmap v3756: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:26:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:26:30 smithi138 ceph-mon[36954]: pgmap v3757: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:26:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:26:30 smithi100 ceph-mon[32599]: pgmap v3757: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:26:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:26:30 smithi167 ceph-mon[37140]: pgmap v3757: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:26:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:26:32 smithi138 ceph-mon[36954]: pgmap v3758: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:26:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:26:32 smithi100 ceph-mon[32599]: pgmap v3758: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:26:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:26:32 smithi167 ceph-mon[37140]: pgmap v3758: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:26:33.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:26:33.658+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:26:33.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:26:33.658+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:26:33.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:26:33.658+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:26:33.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:26:33.658+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:26:33.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:26:33.658+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:26:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:26:34 smithi138 ceph-mon[36954]: pgmap v3759: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:26:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:26:34 smithi100 ceph-mon[32599]: pgmap v3759: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:26:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:26:34 smithi167 ceph-mon[37140]: pgmap v3759: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:26:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:26:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:26:36.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:26:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:26:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:26:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:26:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:26:37 smithi100 ceph-mon[32599]: pgmap v3760: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:26:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:26:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:26:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:26:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:26:37.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:26:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:26:37.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:26:37 smithi167 ceph-mon[37140]: pgmap v3760: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:26:37.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:26:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:26:37.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:26:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:26:37.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:26:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:26:37.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:26:37 smithi138 ceph-mon[36954]: pgmap v3760: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:26:37.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:26:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:26:37.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:26:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:26:37.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:26:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:26:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:26:38 smithi138 ceph-mon[36954]: pgmap v3761: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:26:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:26:38 smithi100 ceph-mon[32599]: pgmap v3761: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:26:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:26:38 smithi167 ceph-mon[37140]: pgmap v3761: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:26:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:26:40 smithi138 ceph-mon[36954]: pgmap v3762: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:26:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:26:40 smithi100 ceph-mon[32599]: pgmap v3762: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:26:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:26:40 smithi167 ceph-mon[37140]: pgmap v3762: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:26:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:26:42 smithi138 ceph-mon[36954]: pgmap v3763: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:26:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:26:42 smithi100 ceph-mon[32599]: pgmap v3763: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:26:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:26:42 smithi167 ceph-mon[37140]: pgmap v3763: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:26:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:26:44 smithi138 ceph-mon[36954]: pgmap v3764: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:26:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:26:44 smithi100 ceph-mon[32599]: pgmap v3764: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:26:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:26:44 smithi167 ceph-mon[37140]: pgmap v3764: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:26:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:26:46 smithi138 ceph-mon[36954]: pgmap v3765: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:26:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:26:46 smithi100 ceph-mon[32599]: pgmap v3765: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:26:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:26:46 smithi167 ceph-mon[37140]: pgmap v3765: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:26:48.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:26:48.674+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:26:48.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:26:48.674+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:26:48.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:26:48.674+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:26:48.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:26:48.674+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:26:48.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:26:48.674+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:26:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:26:48 smithi138 ceph-mon[36954]: pgmap v3766: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:26:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:26:48 smithi100 ceph-mon[32599]: pgmap v3766: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:26:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:26:48 smithi167 ceph-mon[37140]: pgmap v3766: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:26:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:26:50 smithi138 ceph-mon[36954]: pgmap v3767: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:26:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:26:50 smithi100 ceph-mon[32599]: pgmap v3767: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:26:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:26:50 smithi167 ceph-mon[37140]: pgmap v3767: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:26:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:26:52 smithi138 ceph-mon[36954]: pgmap v3768: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:26:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:26:52 smithi100 ceph-mon[32599]: pgmap v3768: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:26:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:26:52 smithi167 ceph-mon[37140]: pgmap v3768: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:26:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:26:54 smithi138 ceph-mon[36954]: pgmap v3769: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:26:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:26:54 smithi100 ceph-mon[32599]: pgmap v3769: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:26:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:26:54 smithi167 ceph-mon[37140]: pgmap v3769: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:26:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:26:56 smithi138 ceph-mon[36954]: pgmap v3770: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:26:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:26:56 smithi100 ceph-mon[32599]: pgmap v3770: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:26:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:26:56 smithi167 ceph-mon[37140]: pgmap v3770: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:26:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:26:58 smithi138 ceph-mon[36954]: pgmap v3771: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:26:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:26:58 smithi100 ceph-mon[32599]: pgmap v3771: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:26:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:26:58 smithi167 ceph-mon[37140]: pgmap v3771: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:27:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:00 smithi138 ceph-mon[36954]: pgmap v3772: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:27:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:00 smithi100 ceph-mon[32599]: pgmap v3772: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:27:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:00 smithi167 ceph-mon[37140]: pgmap v3772: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:27:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:02 smithi138 ceph-mon[36954]: pgmap v3773: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:27:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:02 smithi100 ceph-mon[32599]: pgmap v3773: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:27:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:02 smithi167 ceph-mon[37140]: pgmap v3773: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:27:03.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:27:03.690+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:27:03.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:27:03.690+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:27:03.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:27:03.690+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:27:03.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:27:03.690+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:27:03.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:27:03.690+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:27:03.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:27:03.852+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T15:27:03.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:27:03.852+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T15:27:03.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:27:03.852+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 49 ==== auth_reply(proto 2 0 (0) Success) ==== 783+0+0 (secure 0 0 0) 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T15:27:03.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:27:03.852+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 49 ==== auth_reply(proto 2 0 (0) Success) ==== 612+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T15:27:03.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:27:03.852+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T15:27:03.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:27:03.853+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe5a008c070 con 0x7fe5a81087a0 2024-07-26T15:27:03.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:27:03.853+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 50 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T15:27:03.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:27:03.853+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 50 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c070 con 0x7fe5a81087a0 2024-07-26T15:27:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:04 smithi138 ceph-mon[36954]: pgmap v3774: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:27:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:04 smithi100 ceph-mon[32599]: pgmap v3774: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:27:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:04 smithi167 ceph-mon[37140]: pgmap v3774: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:27:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:06 smithi138 ceph-mon[36954]: pgmap v3775: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:27:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:06 smithi100 ceph-mon[32599]: pgmap v3775: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:27:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:06 smithi167 ceph-mon[37140]: pgmap v3775: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:27:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:08 smithi138 ceph-mon[36954]: osdmap e284: 12 total, 12 up, 12 in 2024-07-26T15:27:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:08 smithi138 ceph-mon[36954]: pgmap v3777: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:27:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:08 smithi100 ceph-mon[32599]: osdmap e284: 12 total, 12 up, 12 in 2024-07-26T15:27:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:08 smithi100 ceph-mon[32599]: pgmap v3777: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:27:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:08 smithi167 ceph-mon[37140]: osdmap e284: 12 total, 12 up, 12 in 2024-07-26T15:27:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:08 smithi167 ceph-mon[37140]: pgmap v3777: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:27:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:10 smithi138 ceph-mon[36954]: pgmap v3778: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:27:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:10 smithi100 ceph-mon[32599]: pgmap v3778: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:27:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:10 smithi167 ceph-mon[37140]: pgmap v3778: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:27:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:12 smithi138 ceph-mon[36954]: pgmap v3779: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:27:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:12 smithi100 ceph-mon[32599]: pgmap v3779: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:27:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:12 smithi167 ceph-mon[37140]: pgmap v3779: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:27:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:14 smithi138 ceph-mon[36954]: pgmap v3780: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:27:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:14 smithi100 ceph-mon[32599]: pgmap v3780: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:27:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:14 smithi167 ceph-mon[37140]: pgmap v3780: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:27:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:16 smithi138 ceph-mon[36954]: osdmap e285: 12 total, 12 up, 12 in 2024-07-26T15:27:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:16 smithi100 ceph-mon[32599]: osdmap e285: 12 total, 12 up, 12 in 2024-07-26T15:27:16.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:16 smithi167 ceph-mon[37140]: osdmap e285: 12 total, 12 up, 12 in 2024-07-26T15:27:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:17 smithi138 ceph-mon[36954]: pgmap v3782: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:27:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:17 smithi100 ceph-mon[32599]: pgmap v3782: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:27:17.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:17 smithi167 ceph-mon[37140]: pgmap v3782: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:27:18.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:27:18.705+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:27:18.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:27:18.706+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:27:18.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:27:18.706+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:27:18.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:27:18.706+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:27:18.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:27:18.706+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:27:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:18 smithi138 ceph-mon[36954]: pgmap v3783: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:27:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:18 smithi100 ceph-mon[32599]: pgmap v3783: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:27:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:18 smithi167 ceph-mon[37140]: pgmap v3783: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:27:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:20 smithi138 ceph-mon[36954]: pgmap v3784: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:27:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:20 smithi100 ceph-mon[32599]: pgmap v3784: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:27:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:20 smithi167 ceph-mon[37140]: pgmap v3784: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:27:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:22 smithi138 ceph-mon[36954]: pgmap v3785: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:27:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:22 smithi100 ceph-mon[32599]: pgmap v3785: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:27:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:22 smithi167 ceph-mon[37140]: pgmap v3785: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:27:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:24 smithi138 ceph-mon[36954]: pgmap v3786: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:27:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:24 smithi100 ceph-mon[32599]: pgmap v3786: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:27:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:24 smithi167 ceph-mon[37140]: pgmap v3786: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:27:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:26 smithi138 ceph-mon[36954]: pgmap v3787: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:27:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:26 smithi100 ceph-mon[32599]: pgmap v3787: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:27:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:26 smithi167 ceph-mon[37140]: pgmap v3787: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:27:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:28 smithi138 ceph-mon[36954]: pgmap v3788: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:27:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:28 smithi100 ceph-mon[32599]: pgmap v3788: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:27:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:28 smithi167 ceph-mon[37140]: pgmap v3788: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:27:30.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:30 smithi138 ceph-mon[36954]: pgmap v3789: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:27:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:30 smithi100 ceph-mon[32599]: pgmap v3789: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:27:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:30 smithi167 ceph-mon[37140]: pgmap v3789: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:27:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:32 smithi138 ceph-mon[36954]: pgmap v3790: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:27:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:32 smithi100 ceph-mon[32599]: pgmap v3790: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:27:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:32 smithi167 ceph-mon[37140]: pgmap v3790: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:27:33.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:27:33.719+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:27:33.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:27:33.720+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:27:33.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:27:33.720+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:27:33.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:27:33.720+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:27:33.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:27:33.720+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:27:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:34 smithi138 ceph-mon[36954]: pgmap v3791: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:27:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:34 smithi100 ceph-mon[32599]: pgmap v3791: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:27:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:34 smithi167 ceph-mon[37140]: pgmap v3791: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:27:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:36 smithi138 ceph-mon[36954]: pgmap v3792: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:27:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:36 smithi100 ceph-mon[32599]: pgmap v3792: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:27:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:36 smithi167 ceph-mon[37140]: pgmap v3792: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:27:37.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:27:37.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:27:37.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:27:37.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:27:37.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:27:37.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:27:37.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:27:37.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:27:37.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:27:37.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:27:37.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:27:37.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:27:37.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:27:37.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:37 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T15:27:37.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:37 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T15:27:37.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:37 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T15:27:37.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:37 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T15:27:37.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:37 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T15:27:37.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:37 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T15:27:37.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:27:37.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:27:37.944 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:27:37.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:27:37.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:27:37.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:27:37.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:27:37.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:27:37.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:27:37.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:27:37.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:27:37.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:27:37.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:27:37.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:27:37.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:27:37.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:27:37.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:37 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T15:27:37.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:37 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T15:27:37.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:37 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T15:27:37.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:37 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T15:27:37.997 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:37 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T15:27:37.997 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:37 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T15:27:37.997 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:27:37.997 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:27:37.997 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:27:38.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:27:38.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:27:38.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:27:38.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:27:38.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:27:38.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:27:38.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:27:38.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:27:38.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:27:38.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:27:38.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:27:38.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:27:38.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:27:38.027 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:37 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T15:27:38.027 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:37 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T15:27:38.027 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:37 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T15:27:38.028 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:37 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T15:27:38.028 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:37 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T15:27:38.028 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:37 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T15:27:38.028 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:27:38.029 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:27:38.029 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:27:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:38 smithi138 ceph-mon[36954]: pgmap v3793: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:27:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:38 smithi100 ceph-mon[32599]: pgmap v3793: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:27:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:38 smithi167 ceph-mon[37140]: pgmap v3793: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:27:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:40 smithi138 ceph-mon[36954]: pgmap v3794: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:27:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:40 smithi100 ceph-mon[32599]: pgmap v3794: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:27:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:40 smithi167 ceph-mon[37140]: pgmap v3794: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:27:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:42 smithi138 ceph-mon[36954]: pgmap v3795: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:27:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:42 smithi100 ceph-mon[32599]: pgmap v3795: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:27:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:42 smithi167 ceph-mon[37140]: pgmap v3795: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:27:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:44 smithi138 ceph-mon[36954]: pgmap v3796: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:27:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:44 smithi100 ceph-mon[32599]: pgmap v3796: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:27:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:44 smithi167 ceph-mon[37140]: pgmap v3796: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:27:46.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:46 smithi138 ceph-mon[36954]: pgmap v3797: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:27:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:46 smithi100 ceph-mon[32599]: pgmap v3797: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:27:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:46 smithi167 ceph-mon[37140]: pgmap v3797: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:27:48.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:27:48.735+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:27:48.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:27:48.736+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:27:48.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:27:48.736+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:27:48.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:27:48.736+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:27:48.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:27:48.736+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:27:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:48 smithi138 ceph-mon[36954]: pgmap v3798: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:27:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:48 smithi100 ceph-mon[32599]: pgmap v3798: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:27:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:48 smithi167 ceph-mon[37140]: pgmap v3798: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:27:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:50 smithi138 ceph-mon[36954]: pgmap v3799: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:27:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:50 smithi100 ceph-mon[32599]: pgmap v3799: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:27:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:50 smithi167 ceph-mon[37140]: pgmap v3799: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:27:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:52 smithi138 ceph-mon[36954]: pgmap v3800: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:27:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:52 smithi100 ceph-mon[32599]: pgmap v3800: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:27:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:52 smithi167 ceph-mon[37140]: pgmap v3800: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:27:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:54 smithi138 ceph-mon[36954]: pgmap v3801: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:27:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:54 smithi100 ceph-mon[32599]: pgmap v3801: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:27:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:54 smithi167 ceph-mon[37140]: pgmap v3801: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:27:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:56 smithi138 ceph-mon[36954]: pgmap v3802: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:27:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:56 smithi100 ceph-mon[32599]: pgmap v3802: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:27:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:56 smithi167 ceph-mon[37140]: pgmap v3802: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:27:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:27:58 smithi138 ceph-mon[36954]: pgmap v3803: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:27:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:27:58 smithi100 ceph-mon[32599]: pgmap v3803: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:27:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:27:58 smithi167 ceph-mon[37140]: pgmap v3803: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:28:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:28:00 smithi138 ceph-mon[36954]: pgmap v3804: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:28:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:28:00 smithi100 ceph-mon[32599]: pgmap v3804: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:28:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:28:00 smithi167 ceph-mon[37140]: pgmap v3804: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:28:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:28:02 smithi138 ceph-mon[36954]: pgmap v3805: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:28:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:28:02 smithi100 ceph-mon[32599]: pgmap v3805: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:28:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:28:02 smithi167 ceph-mon[37140]: pgmap v3805: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:28:03.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:28:03.749+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:28:03.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:28:03.750+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:28:03.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:28:03.750+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:28:03.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:28:03.750+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:28:03.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:28:03.750+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:28:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:28:04 smithi138 ceph-mon[36954]: pgmap v3806: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:28:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:28:04 smithi100 ceph-mon[32599]: pgmap v3806: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:28:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:28:04 smithi167 ceph-mon[37140]: pgmap v3806: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:28:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:28:06 smithi138 ceph-mon[36954]: pgmap v3807: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:28:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:28:06 smithi100 ceph-mon[32599]: pgmap v3807: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:28:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:28:06 smithi167 ceph-mon[37140]: pgmap v3807: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:28:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:28:08 smithi138 ceph-mon[36954]: osdmap e286: 12 total, 12 up, 12 in 2024-07-26T15:28:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:28:08 smithi138 ceph-mon[36954]: pgmap v3809: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:28:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:28:08 smithi100 ceph-mon[32599]: osdmap e286: 12 total, 12 up, 12 in 2024-07-26T15:28:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:28:08 smithi100 ceph-mon[32599]: pgmap v3809: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:28:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:28:08 smithi167 ceph-mon[37140]: osdmap e286: 12 total, 12 up, 12 in 2024-07-26T15:28:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:28:08 smithi167 ceph-mon[37140]: pgmap v3809: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:28:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:28:10 smithi138 ceph-mon[36954]: pgmap v3810: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:28:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:28:10 smithi100 ceph-mon[32599]: pgmap v3810: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:28:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:28:10 smithi167 ceph-mon[37140]: pgmap v3810: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:28:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:28:12 smithi138 ceph-mon[36954]: pgmap v3811: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:28:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:28:12 smithi100 ceph-mon[32599]: pgmap v3811: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:28:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:28:12 smithi167 ceph-mon[37140]: pgmap v3811: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:28:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:28:14 smithi138 ceph-mon[36954]: pgmap v3812: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:28:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:28:14 smithi100 ceph-mon[32599]: pgmap v3812: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:28:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:28:14 smithi167 ceph-mon[37140]: pgmap v3812: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:28:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:28:16 smithi138 ceph-mon[36954]: osdmap e287: 12 total, 12 up, 12 in 2024-07-26T15:28:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:28:16 smithi100 ceph-mon[32599]: osdmap e287: 12 total, 12 up, 12 in 2024-07-26T15:28:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:28:16 smithi167 ceph-mon[37140]: osdmap e287: 12 total, 12 up, 12 in 2024-07-26T15:28:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:28:17 smithi138 ceph-mon[36954]: pgmap v3814: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:28:17.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:28:17 smithi100 ceph-mon[32599]: pgmap v3814: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:28:17.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:28:17 smithi167 ceph-mon[37140]: pgmap v3814: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:28:18.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:28:18.758+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:28:18.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:28:18.758+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:28:18.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:28:18.758+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:28:18.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:28:18.758+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:28:18.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:28:18.758+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:28:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:28:18 smithi138 ceph-mon[36954]: pgmap v3815: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:28:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:28:18 smithi100 ceph-mon[32599]: pgmap v3815: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:28:19.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:28:18 smithi167 ceph-mon[37140]: pgmap v3815: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:28:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:28:20 smithi138 ceph-mon[36954]: pgmap v3816: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:28:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:28:20 smithi100 ceph-mon[32599]: pgmap v3816: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:28:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:28:20 smithi167 ceph-mon[37140]: pgmap v3816: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:28:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:28:22 smithi138 ceph-mon[36954]: pgmap v3817: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:28:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:28:22 smithi100 ceph-mon[32599]: pgmap v3817: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:28:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:28:22 smithi167 ceph-mon[37140]: pgmap v3817: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:28:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:28:24 smithi138 ceph-mon[36954]: pgmap v3818: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:28:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:28:24 smithi100 ceph-mon[32599]: pgmap v3818: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:28:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:28:24 smithi167 ceph-mon[37140]: pgmap v3818: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:28:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:28:26 smithi138 ceph-mon[36954]: pgmap v3819: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:28:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:28:26 smithi100 ceph-mon[32599]: pgmap v3819: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:28:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:28:26 smithi167 ceph-mon[37140]: pgmap v3819: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:28:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:28:28 smithi138 ceph-mon[36954]: pgmap v3820: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:28:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:28:28 smithi100 ceph-mon[32599]: pgmap v3820: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:28:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:28:28 smithi167 ceph-mon[37140]: pgmap v3820: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:28:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:28:31 smithi138 ceph-mon[36954]: pgmap v3821: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:28:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:28:31 smithi100 ceph-mon[32599]: pgmap v3821: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:28:31.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:28:31 smithi167 ceph-mon[37140]: pgmap v3821: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:28:32.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:28:32 smithi138 ceph-mon[36954]: pgmap v3822: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:28:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:28:32 smithi100 ceph-mon[32599]: pgmap v3822: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:28:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:28:32 smithi167 ceph-mon[37140]: pgmap v3822: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:28:33.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:28:33.774+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:28:33.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:28:33.774+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:28:33.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:28:33.774+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:28:33.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:28:33.774+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:28:33.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:28:33.774+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:28:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:28:34 smithi138 ceph-mon[36954]: pgmap v3823: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:28:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:28:34 smithi100 ceph-mon[32599]: pgmap v3823: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:28:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:28:34 smithi167 ceph-mon[37140]: pgmap v3823: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:28:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:28:36 smithi138 ceph-mon[36954]: pgmap v3824: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:28:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:28:36 smithi100 ceph-mon[32599]: pgmap v3824: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:28:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:28:36 smithi167 ceph-mon[37140]: pgmap v3824: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:28:37.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:28:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:28:37.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:28:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:28:38.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:28:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:28:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:28:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:28:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:28:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:28:38.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:28:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:28:38.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:28:38 smithi138 ceph-mon[36954]: pgmap v3825: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:28:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:28:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:28:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:28:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:28:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:28:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:28:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:28:38 smithi100 ceph-mon[32599]: pgmap v3825: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:28:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:28:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:28:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:28:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:28:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:28:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:28:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:28:38 smithi167 ceph-mon[37140]: pgmap v3825: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:28:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:28:40 smithi138 ceph-mon[36954]: pgmap v3826: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:28:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:28:40 smithi100 ceph-mon[32599]: pgmap v3826: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:28:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:28:40 smithi167 ceph-mon[37140]: pgmap v3826: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:28:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:28:42 smithi138 ceph-mon[36954]: pgmap v3827: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:28:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:28:42 smithi100 ceph-mon[32599]: pgmap v3827: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:28:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:28:42 smithi167 ceph-mon[37140]: pgmap v3827: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:28:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:28:44 smithi138 ceph-mon[36954]: pgmap v3828: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:28:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:28:44 smithi100 ceph-mon[32599]: pgmap v3828: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:28:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:28:44 smithi167 ceph-mon[37140]: pgmap v3828: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:28:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:28:46 smithi138 ceph-mon[36954]: pgmap v3829: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:28:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:28:46 smithi100 ceph-mon[32599]: pgmap v3829: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:28:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:28:46 smithi167 ceph-mon[37140]: pgmap v3829: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:28:48.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:28:48.789+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:28:48.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:28:48.790+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:28:48.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:28:48.790+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:28:48.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:28:48.790+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:28:48.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:28:48.790+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:28:49.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:28:49 smithi138 ceph-mon[36954]: pgmap v3830: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:28:49.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:28:49 smithi100 ceph-mon[32599]: pgmap v3830: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:28:50.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:28:49 smithi167 ceph-mon[37140]: pgmap v3830: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:28:51.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:28:51 smithi138 ceph-mon[36954]: pgmap v3831: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:28:51.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:28:51 smithi100 ceph-mon[32599]: pgmap v3831: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:28:52.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:28:51 smithi167 ceph-mon[37140]: pgmap v3831: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:28:53.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:28:53 smithi138 ceph-mon[36954]: pgmap v3832: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:28:53.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:28:53 smithi100 ceph-mon[32599]: pgmap v3832: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:28:54.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:28:53 smithi167 ceph-mon[37140]: pgmap v3832: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:28:55.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:28:55 smithi138 ceph-mon[36954]: pgmap v3833: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:28:55.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:28:55 smithi100 ceph-mon[32599]: pgmap v3833: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:28:56.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:28:55 smithi167 ceph-mon[37140]: pgmap v3833: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:28:57.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:28:57 smithi138 ceph-mon[36954]: pgmap v3834: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:28:57.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:28:57 smithi100 ceph-mon[32599]: pgmap v3834: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:28:58.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:28:57 smithi167 ceph-mon[37140]: pgmap v3834: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:28:59.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:28:59 smithi138 ceph-mon[36954]: pgmap v3835: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:28:59.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:28:59 smithi100 ceph-mon[32599]: pgmap v3835: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:29:00.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:28:59 smithi167 ceph-mon[37140]: pgmap v3835: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:29:01.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:29:01 smithi138 ceph-mon[36954]: pgmap v3836: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:29:01.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:29:01 smithi100 ceph-mon[32599]: pgmap v3836: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:29:02.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:29:01 smithi167 ceph-mon[37140]: pgmap v3836: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:29:03.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:29:03.794+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:29:03.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:29:03.794+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:29:03.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:29:03.794+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:29:03.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:29:03.794+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:29:03.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:29:03.794+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:29:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:29:03 smithi138 ceph-mon[36954]: pgmap v3837: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:29:03.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:29:03 smithi100 ceph-mon[32599]: pgmap v3837: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:29:04.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:29:03 smithi167 ceph-mon[37140]: pgmap v3837: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:29:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:29:05 smithi138 ceph-mon[36954]: pgmap v3838: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:29:05.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:29:05 smithi100 ceph-mon[32599]: pgmap v3838: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:29:06.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:29:05 smithi167 ceph-mon[37140]: pgmap v3838: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:29:07.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:29:07 smithi138 ceph-mon[36954]: pgmap v3839: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:29:07.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:29:07 smithi100 ceph-mon[32599]: pgmap v3839: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:29:08.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:29:07 smithi167 ceph-mon[37140]: pgmap v3839: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:29:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:29:08 smithi138 ceph-mon[36954]: osdmap e288: 12 total, 12 up, 12 in 2024-07-26T15:29:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:29:08 smithi100 ceph-mon[32599]: osdmap e288: 12 total, 12 up, 12 in 2024-07-26T15:29:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:29:08 smithi167 ceph-mon[37140]: osdmap e288: 12 total, 12 up, 12 in 2024-07-26T15:29:09.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:29:09 smithi138 ceph-mon[36954]: pgmap v3841: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:29:09.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:29:09 smithi100 ceph-mon[32599]: pgmap v3841: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:29:10.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:29:09 smithi167 ceph-mon[37140]: pgmap v3841: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:29:11.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:29:11 smithi138 ceph-mon[36954]: pgmap v3842: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:29:11.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:29:11 smithi100 ceph-mon[32599]: pgmap v3842: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:29:12.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:29:11 smithi167 ceph-mon[37140]: pgmap v3842: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:29:13.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:29:13 smithi138 ceph-mon[36954]: pgmap v3843: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:29:13.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:29:13 smithi100 ceph-mon[32599]: pgmap v3843: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:29:14.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:29:13 smithi167 ceph-mon[37140]: pgmap v3843: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:29:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:29:16 smithi138 ceph-mon[36954]: pgmap v3844: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:29:16.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:29:16 smithi138 ceph-mon[36954]: osdmap e289: 12 total, 12 up, 12 in 2024-07-26T15:29:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:29:16 smithi100 ceph-mon[32599]: pgmap v3844: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:29:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:29:16 smithi100 ceph-mon[32599]: osdmap e289: 12 total, 12 up, 12 in 2024-07-26T15:29:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:29:16 smithi167 ceph-mon[37140]: pgmap v3844: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:29:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:29:16 smithi167 ceph-mon[37140]: osdmap e289: 12 total, 12 up, 12 in 2024-07-26T15:29:18.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:29:18 smithi138 ceph-mon[36954]: pgmap v3846: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:29:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:29:18 smithi100 ceph-mon[32599]: pgmap v3846: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:29:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:29:18 smithi167 ceph-mon[37140]: pgmap v3846: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:29:18.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:29:18.810+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:29:18.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:29:18.810+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:29:18.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:29:18.810+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:29:18.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:29:18.810+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:29:18.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:29:18.811+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:29:20.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:29:20 smithi138 ceph-mon[36954]: pgmap v3847: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:29:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:29:20 smithi100 ceph-mon[32599]: pgmap v3847: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:29:20.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:29:20 smithi167 ceph-mon[37140]: pgmap v3847: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:29:22.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:29:22 smithi138 ceph-mon[36954]: pgmap v3848: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:29:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:29:22 smithi100 ceph-mon[32599]: pgmap v3848: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:29:22.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:29:22 smithi167 ceph-mon[37140]: pgmap v3848: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:29:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:29:24 smithi138 ceph-mon[36954]: pgmap v3849: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:29:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:29:24 smithi100 ceph-mon[32599]: pgmap v3849: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:29:24.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:29:24 smithi167 ceph-mon[37140]: pgmap v3849: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:29:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:29:26 smithi138 ceph-mon[36954]: pgmap v3850: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:29:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:29:26 smithi100 ceph-mon[32599]: pgmap v3850: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:29:26.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:29:26 smithi167 ceph-mon[37140]: pgmap v3850: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:29:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:29:28 smithi138 ceph-mon[36954]: pgmap v3851: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:29:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:29:28 smithi100 ceph-mon[32599]: pgmap v3851: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:29:28.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:29:28 smithi167 ceph-mon[37140]: pgmap v3851: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:29:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:29:30 smithi138 ceph-mon[36954]: pgmap v3852: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:29:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:29:30 smithi100 ceph-mon[32599]: pgmap v3852: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:29:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:29:30 smithi167 ceph-mon[37140]: pgmap v3852: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:29:32.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:29:32 smithi138 ceph-mon[36954]: pgmap v3853: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:29:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:29:32 smithi100 ceph-mon[32599]: pgmap v3853: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:29:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:29:32 smithi167 ceph-mon[37140]: pgmap v3853: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:29:33.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:29:33.826+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:29:33.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:29:33.827+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:29:33.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:29:33.827+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:29:33.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:29:33.827+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:29:33.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:29:33.827+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:29:34.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:29:34 smithi138 ceph-mon[36954]: pgmap v3854: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:29:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:29:34 smithi100 ceph-mon[32599]: pgmap v3854: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:29:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:29:34 smithi167 ceph-mon[37140]: pgmap v3854: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:29:36.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:29:36 smithi138 ceph-mon[36954]: pgmap v3855: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:29:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:29:36 smithi100 ceph-mon[32599]: pgmap v3855: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:29:36.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:29:36 smithi167 ceph-mon[37140]: pgmap v3855: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:29:38.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:29:38 smithi138 ceph-mon[36954]: pgmap v3856: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:29:38.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:29:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:29:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:29:38 smithi100 ceph-mon[32599]: pgmap v3856: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:29:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:29:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:29:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:29:38 smithi167 ceph-mon[37140]: pgmap v3856: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:29:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:29:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:29:39.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:29:39 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:29:39.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:29:39 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:29:39.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:29:39 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:29:39.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:29:39 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:29:39.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:29:39 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:29:39.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:29:39 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:29:39.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:29:39 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:29:39.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:29:39 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:29:39.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:29:39 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:29:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:29:40 smithi138 ceph-mon[36954]: pgmap v3857: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:29:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:29:40 smithi100 ceph-mon[32599]: pgmap v3857: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:29:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:29:40 smithi167 ceph-mon[37140]: pgmap v3857: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:29:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:29:42 smithi138 ceph-mon[36954]: pgmap v3858: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:29:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:29:42 smithi100 ceph-mon[32599]: pgmap v3858: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:29:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:29:42 smithi167 ceph-mon[37140]: pgmap v3858: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:29:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:29:44 smithi138 ceph-mon[36954]: pgmap v3859: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:29:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:29:44 smithi100 ceph-mon[32599]: pgmap v3859: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:29:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:29:44 smithi167 ceph-mon[37140]: pgmap v3859: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:29:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:29:46 smithi138 ceph-mon[36954]: pgmap v3860: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:29:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:29:46 smithi100 ceph-mon[32599]: pgmap v3860: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:29:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:29:46 smithi167 ceph-mon[37140]: pgmap v3860: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:29:48.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:29:48.839+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:29:48.841 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:29:48 smithi100 ceph-mon[32599]: pgmap v3861: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:29:48.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:29:48.839+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:29:48.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:29:48.839+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:29:48.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:29:48.839+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:29:48.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:29:48.839+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:29:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:29:48 smithi138 ceph-mon[36954]: pgmap v3861: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:29:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:29:48 smithi167 ceph-mon[37140]: pgmap v3861: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:29:49.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:29:49 smithi138 ceph-mon[36954]: pgmap v3862: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:29:49.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:29:49 smithi100 ceph-mon[32599]: pgmap v3862: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:29:50.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:29:49 smithi167 ceph-mon[37140]: pgmap v3862: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:29:51.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:29:51 smithi138 ceph-mon[36954]: pgmap v3863: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:29:51.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:29:51 smithi100 ceph-mon[32599]: pgmap v3863: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:29:52.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:29:51 smithi167 ceph-mon[37140]: pgmap v3863: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:29:53.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:29:53 smithi138 ceph-mon[36954]: pgmap v3864: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:29:53.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:29:53 smithi100 ceph-mon[32599]: pgmap v3864: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:29:54.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:29:53 smithi167 ceph-mon[37140]: pgmap v3864: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:29:55.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:29:55 smithi138 ceph-mon[36954]: pgmap v3865: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:29:55.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:29:55 smithi100 ceph-mon[32599]: pgmap v3865: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:29:56.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:29:55 smithi167 ceph-mon[37140]: pgmap v3865: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:29:57.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:29:57 smithi138 ceph-mon[36954]: pgmap v3866: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:29:57.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:29:57 smithi100 ceph-mon[32599]: pgmap v3866: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:29:58.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:29:57 smithi167 ceph-mon[37140]: pgmap v3866: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:29:59.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:29:59 smithi138 ceph-mon[36954]: pgmap v3867: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:29:59.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:29:59 smithi100 ceph-mon[32599]: pgmap v3867: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:30:00.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:29:59 smithi167 ceph-mon[37140]: pgmap v3867: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:30:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:00 smithi138 ceph-mon[36954]: overall HEALTH_OK 2024-07-26T15:30:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:00 smithi100 ceph-mon[32599]: overall HEALTH_OK 2024-07-26T15:30:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:00 smithi167 ceph-mon[37140]: overall HEALTH_OK 2024-07-26T15:30:01.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:01 smithi138 ceph-mon[36954]: pgmap v3868: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:30:01.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:01 smithi100 ceph-mon[32599]: pgmap v3868: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:30:02.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:01 smithi167 ceph-mon[37140]: pgmap v3868: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:30:03.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:30:03.855+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:30:03.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:30:03.856+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:30:03.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:30:03.856+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:30:03.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:30:03.856+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:30:03.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:30:03.856+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:30:03.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:30:03.877+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T15:30:03.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:30:03.877+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c003690 con 0x7fe5a81087a0 2024-07-26T15:30:03.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:30:03.877+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 51 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T15:30:03.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:30:03.877+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 51 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c100 con 0x7fe5a81087a0 2024-07-26T15:30:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:03 smithi138 ceph-mon[36954]: pgmap v3869: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:30:03.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:03 smithi100 ceph-mon[32599]: pgmap v3869: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:30:04.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:03 smithi167 ceph-mon[37140]: pgmap v3869: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:30:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:05 smithi138 ceph-mon[36954]: pgmap v3870: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:30:05.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:05 smithi100 ceph-mon[32599]: pgmap v3870: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:30:06.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:05 smithi167 ceph-mon[37140]: pgmap v3870: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:30:07.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:07 smithi138 ceph-mon[36954]: pgmap v3871: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:30:07.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:07 smithi100 ceph-mon[32599]: pgmap v3871: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:30:08.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:07 smithi167 ceph-mon[37140]: pgmap v3871: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:30:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:08 smithi138 ceph-mon[36954]: osdmap e290: 12 total, 12 up, 12 in 2024-07-26T15:30:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:08 smithi100 ceph-mon[32599]: osdmap e290: 12 total, 12 up, 12 in 2024-07-26T15:30:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:08 smithi167 ceph-mon[37140]: osdmap e290: 12 total, 12 up, 12 in 2024-07-26T15:30:09.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:09 smithi138 ceph-mon[36954]: pgmap v3873: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:30:09.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:09 smithi100 ceph-mon[32599]: pgmap v3873: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:30:10.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:09 smithi167 ceph-mon[37140]: pgmap v3873: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:30:11.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:11 smithi100 ceph-mon[32599]: pgmap v3874: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:30:12.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:11 smithi167 ceph-mon[37140]: pgmap v3874: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:30:12.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:11 smithi138 ceph-mon[36954]: pgmap v3874: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:30:13.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:13 smithi100 ceph-mon[32599]: pgmap v3875: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:30:14.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:13 smithi167 ceph-mon[37140]: pgmap v3875: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:30:14.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:13 smithi138 ceph-mon[36954]: pgmap v3875: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:30:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:16 smithi138 ceph-mon[36954]: pgmap v3876: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:30:16.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:16 smithi138 ceph-mon[36954]: osdmap e291: 12 total, 12 up, 12 in 2024-07-26T15:30:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:16 smithi100 ceph-mon[32599]: pgmap v3876: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:30:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:16 smithi100 ceph-mon[32599]: osdmap e291: 12 total, 12 up, 12 in 2024-07-26T15:30:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:16 smithi167 ceph-mon[37140]: pgmap v3876: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:30:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:16 smithi167 ceph-mon[37140]: osdmap e291: 12 total, 12 up, 12 in 2024-07-26T15:30:18.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:18 smithi138 ceph-mon[36954]: pgmap v3878: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:30:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:18 smithi100 ceph-mon[32599]: pgmap v3878: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:30:18.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:18 smithi167 ceph-mon[37140]: pgmap v3878: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:30:18.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:30:18.871+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:30:18.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:30:18.872+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:30:18.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:30:18.872+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:30:18.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:30:18.872+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:30:18.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:30:18.872+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:30:20.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:20 smithi138 ceph-mon[36954]: pgmap v3879: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:30:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:20 smithi100 ceph-mon[32599]: pgmap v3879: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:30:20.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:20 smithi167 ceph-mon[37140]: pgmap v3879: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:30:22.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:22 smithi138 ceph-mon[36954]: pgmap v3880: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:30:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:22 smithi100 ceph-mon[32599]: pgmap v3880: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:30:22.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:22 smithi167 ceph-mon[37140]: pgmap v3880: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:30:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:24 smithi138 ceph-mon[36954]: pgmap v3881: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:30:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:24 smithi100 ceph-mon[32599]: pgmap v3881: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:30:24.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:24 smithi167 ceph-mon[37140]: pgmap v3881: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:30:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:26 smithi138 ceph-mon[36954]: pgmap v3882: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:30:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:26 smithi100 ceph-mon[32599]: pgmap v3882: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:30:26.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:26 smithi167 ceph-mon[37140]: pgmap v3882: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:30:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:28 smithi138 ceph-mon[36954]: pgmap v3883: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:30:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:28 smithi100 ceph-mon[32599]: pgmap v3883: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:30:28.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:28 smithi167 ceph-mon[37140]: pgmap v3883: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:30:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:30 smithi138 ceph-mon[36954]: pgmap v3884: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:30:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:30 smithi100 ceph-mon[32599]: pgmap v3884: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:30:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:30 smithi167 ceph-mon[37140]: pgmap v3884: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:30:32.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:32 smithi138 ceph-mon[36954]: pgmap v3885: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:30:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:32 smithi100 ceph-mon[32599]: pgmap v3885: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:30:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:32 smithi167 ceph-mon[37140]: pgmap v3885: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:30:33.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:30:33.887+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:30:33.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:30:33.888+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:30:33.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:30:33.888+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:30:33.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:30:33.888+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:30:33.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:30:33.888+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:30:34.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:34 smithi138 ceph-mon[36954]: pgmap v3886: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:30:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:34 smithi100 ceph-mon[32599]: pgmap v3886: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:30:34.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:34 smithi167 ceph-mon[37140]: pgmap v3886: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:30:36.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:36 smithi138 ceph-mon[36954]: pgmap v3887: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:30:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:36 smithi100 ceph-mon[32599]: pgmap v3887: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:30:36.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:36 smithi167 ceph-mon[37140]: pgmap v3887: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:30:38.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:38 smithi138 ceph-mon[36954]: pgmap v3888: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:30:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:38 smithi100 ceph-mon[32599]: pgmap v3888: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:30:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:38 smithi167 ceph-mon[37140]: pgmap v3888: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:30:39.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:39 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:30:39.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:39 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:30:39.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:39 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:30:40.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:40 smithi138 ceph-mon[36954]: pgmap v3889: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:30:40.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:30:40.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:30:40.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:30:40.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:30:40.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:30:40.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:30:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:40 smithi100 ceph-mon[32599]: pgmap v3889: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:30:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:30:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:30:40.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:30:40.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:30:40.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:30:40.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:30:40.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:40 smithi167 ceph-mon[37140]: pgmap v3889: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:30:40.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:30:40.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:30:40.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:30:40.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:30:40.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:30:40.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:30:41.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:30:41.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:30:41.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:30:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:30:41.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:30:41.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:30:41.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:30:41.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:30:41.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:30:42.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:42 smithi138 ceph-mon[36954]: pgmap v3890: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:30:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:42 smithi100 ceph-mon[32599]: pgmap v3890: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:30:42.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:42 smithi167 ceph-mon[37140]: pgmap v3890: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:30:44.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:44 smithi138 ceph-mon[36954]: pgmap v3891: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:30:44.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:44 smithi100 ceph-mon[32599]: pgmap v3891: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:30:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:44 smithi167 ceph-mon[37140]: pgmap v3891: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:30:46.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:46 smithi138 ceph-mon[36954]: pgmap v3892: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:30:46.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:46 smithi100 ceph-mon[32599]: pgmap v3892: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:30:46.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:46 smithi167 ceph-mon[37140]: pgmap v3892: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:30:48.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:48 smithi138 ceph-mon[36954]: pgmap v3893: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:30:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:48 smithi100 ceph-mon[32599]: pgmap v3893: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:30:48.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:48 smithi167 ceph-mon[37140]: pgmap v3893: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:30:48.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:30:48.893+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:30:48.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:30:48.894+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:30:48.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:30:48.894+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:30:48.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:30:48.894+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:30:48.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:30:48.894+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:30:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:50 smithi100 ceph-mon[32599]: pgmap v3894: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:30:50.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:50 smithi167 ceph-mon[37140]: pgmap v3894: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:30:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:50 smithi138 ceph-mon[36954]: pgmap v3894: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:30:52.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:52 smithi100 ceph-mon[32599]: pgmap v3895: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:30:52.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:52 smithi167 ceph-mon[37140]: pgmap v3895: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:30:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:52 smithi138 ceph-mon[36954]: pgmap v3895: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:30:54.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:54 smithi100 ceph-mon[32599]: pgmap v3896: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:30:54.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:54 smithi167 ceph-mon[37140]: pgmap v3896: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:30:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:54 smithi138 ceph-mon[36954]: pgmap v3896: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:30:56.743 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:56 smithi138 ceph-mon[36954]: pgmap v3897: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:30:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:56 smithi100 ceph-mon[32599]: pgmap v3897: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:30:56.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:56 smithi167 ceph-mon[37140]: pgmap v3897: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:30:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:30:58 smithi100 ceph-mon[32599]: pgmap v3898: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:30:58.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:30:58 smithi167 ceph-mon[37140]: pgmap v3898: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:30:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:30:58 smithi138 ceph-mon[36954]: pgmap v3898: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:31:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:00 smithi100 ceph-mon[32599]: pgmap v3899: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:31:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:00 smithi167 ceph-mon[37140]: pgmap v3899: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:31:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:00 smithi138 ceph-mon[36954]: pgmap v3899: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:31:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:02 smithi100 ceph-mon[32599]: pgmap v3900: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:31:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:02 smithi167 ceph-mon[37140]: pgmap v3900: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:31:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:02 smithi138 ceph-mon[36954]: pgmap v3900: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:31:03.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:31:03.898+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:31:03.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:31:03.898+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:31:03.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:31:03.898+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:31:03.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:31:03.898+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:31:03.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:31:03.898+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:31:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:04 smithi100 ceph-mon[32599]: pgmap v3901: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:31:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:04 smithi167 ceph-mon[37140]: pgmap v3901: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:31:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:04 smithi138 ceph-mon[36954]: pgmap v3901: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:31:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:06 smithi100 ceph-mon[32599]: pgmap v3902: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:31:06.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:06 smithi167 ceph-mon[37140]: pgmap v3902: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:31:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:06 smithi138 ceph-mon[36954]: pgmap v3902: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:31:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:08 smithi100 ceph-mon[32599]: pgmap v3903: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:31:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:08 smithi100 ceph-mon[32599]: osdmap e292: 12 total, 12 up, 12 in 2024-07-26T15:31:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:08 smithi167 ceph-mon[37140]: pgmap v3903: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:31:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:08 smithi167 ceph-mon[37140]: osdmap e292: 12 total, 12 up, 12 in 2024-07-26T15:31:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:08 smithi138 ceph-mon[36954]: pgmap v3903: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:31:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:08 smithi138 ceph-mon[36954]: osdmap e292: 12 total, 12 up, 12 in 2024-07-26T15:31:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:10 smithi100 ceph-mon[32599]: pgmap v3905: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:31:10.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:10 smithi167 ceph-mon[37140]: pgmap v3905: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:31:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:10 smithi138 ceph-mon[36954]: pgmap v3905: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:31:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:12 smithi100 ceph-mon[32599]: pgmap v3906: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:31:12.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:12 smithi167 ceph-mon[37140]: pgmap v3906: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:31:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:12 smithi138 ceph-mon[36954]: pgmap v3906: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:31:14.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:14 smithi100 ceph-mon[32599]: pgmap v3907: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:31:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:14 smithi167 ceph-mon[37140]: pgmap v3907: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:31:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:14 smithi138 ceph-mon[36954]: pgmap v3907: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:31:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:16 smithi100 ceph-mon[32599]: pgmap v3908: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:31:16.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:16 smithi167 ceph-mon[37140]: pgmap v3908: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:31:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:16 smithi138 ceph-mon[36954]: pgmap v3908: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:31:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:18 smithi167 ceph-mon[37140]: pgmap v3909: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:31:18.917 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:18 smithi100 ceph-mon[32599]: pgmap v3909: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:31:18.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:31:18.915+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:31:18.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:31:18.915+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:31:18.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:31:18.915+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:31:18.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:31:18.915+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:31:18.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:31:18.915+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:31:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:18 smithi138 ceph-mon[36954]: pgmap v3909: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:31:20.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:20 smithi167 ceph-mon[37140]: pgmap v3910: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 183 B/s wr, 6 op/s 2024-07-26T15:31:20.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:20 smithi167 ceph-mon[37140]: osdmap e293: 12 total, 12 up, 12 in 2024-07-26T15:31:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:20 smithi138 ceph-mon[36954]: pgmap v3910: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 183 B/s wr, 6 op/s 2024-07-26T15:31:20.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:20 smithi138 ceph-mon[36954]: osdmap e293: 12 total, 12 up, 12 in 2024-07-26T15:31:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:20 smithi100 ceph-mon[32599]: pgmap v3910: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 183 B/s wr, 6 op/s 2024-07-26T15:31:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:20 smithi100 ceph-mon[32599]: osdmap e293: 12 total, 12 up, 12 in 2024-07-26T15:31:22.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:22 smithi167 ceph-mon[37140]: pgmap v3912: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:31:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:22 smithi138 ceph-mon[36954]: pgmap v3912: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:31:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:22 smithi100 ceph-mon[32599]: pgmap v3912: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:31:24.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:24 smithi167 ceph-mon[37140]: pgmap v3913: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:31:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:24 smithi138 ceph-mon[36954]: pgmap v3913: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:31:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:24 smithi100 ceph-mon[32599]: pgmap v3913: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:31:26.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:26 smithi167 ceph-mon[37140]: pgmap v3914: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:31:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:26 smithi138 ceph-mon[36954]: pgmap v3914: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:31:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:26 smithi100 ceph-mon[32599]: pgmap v3914: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:31:28.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:28 smithi167 ceph-mon[37140]: pgmap v3915: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:31:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:28 smithi138 ceph-mon[36954]: pgmap v3915: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:31:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:28 smithi100 ceph-mon[32599]: pgmap v3915: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:31:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:30 smithi167 ceph-mon[37140]: pgmap v3916: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:31:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:30 smithi138 ceph-mon[36954]: pgmap v3916: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:31:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:30 smithi100 ceph-mon[32599]: pgmap v3916: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:31:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:32 smithi167 ceph-mon[37140]: pgmap v3917: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:31:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:32 smithi138 ceph-mon[36954]: pgmap v3917: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:31:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:32 smithi100 ceph-mon[32599]: pgmap v3917: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:31:33.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:31:33.931+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:31:33.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:31:33.932+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:31:33.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:31:33.932+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:31:33.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:31:33.932+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:31:33.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:31:33.932+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:31:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:34 smithi138 ceph-mon[36954]: pgmap v3918: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:31:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:34 smithi100 ceph-mon[32599]: pgmap v3918: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:31:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:34 smithi167 ceph-mon[37140]: pgmap v3918: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:31:36.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:36 smithi100 ceph-mon[32599]: pgmap v3919: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:31:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:36 smithi138 ceph-mon[36954]: pgmap v3919: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:31:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:36 smithi167 ceph-mon[37140]: pgmap v3919: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:31:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:38 smithi138 ceph-mon[36954]: pgmap v3920: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:31:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:38 smithi100 ceph-mon[32599]: pgmap v3920: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:31:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:38 smithi167 ceph-mon[37140]: pgmap v3920: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:31:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:40 smithi138 ceph-mon[36954]: pgmap v3921: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:31:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:40 smithi100 ceph-mon[32599]: pgmap v3921: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:31:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:40 smithi167 ceph-mon[37140]: pgmap v3921: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:31:41.842 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:31:41.842 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:31:41.842 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:31:41.843 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:31:41.843 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:31:41.843 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:31:41.843 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:31:41.843 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:31:41.843 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:31:41.843 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:31:41.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:31:41.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:31:41.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:31:41.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:31:41.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:31:41.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:31:41.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:31:41.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:31:41.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:31:41.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:31:41.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:31:41.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:31:41.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:31:41.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:31:41.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:31:41.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:31:41.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:31:41.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:31:41.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:31:41.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:31:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:42 smithi138 ceph-mon[36954]: pgmap v3922: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:31:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:42 smithi100 ceph-mon[32599]: pgmap v3922: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:31:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:42 smithi167 ceph-mon[37140]: pgmap v3922: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:31:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:44 smithi138 ceph-mon[36954]: pgmap v3923: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:31:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:44 smithi100 ceph-mon[32599]: pgmap v3923: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:31:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:44 smithi167 ceph-mon[37140]: pgmap v3923: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:31:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:46 smithi138 ceph-mon[36954]: pgmap v3924: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:31:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:46 smithi100 ceph-mon[32599]: pgmap v3924: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:31:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:46 smithi167 ceph-mon[37140]: pgmap v3924: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:31:48.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:31:48.933+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:31:48.935 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:48 smithi100 ceph-mon[32599]: pgmap v3925: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:31:48.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:31:48.934+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:31:48.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:31:48.934+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:31:48.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:31:48.934+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:31:48.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:31:48.934+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:31:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:48 smithi138 ceph-mon[36954]: pgmap v3925: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:31:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:48 smithi167 ceph-mon[37140]: pgmap v3925: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:31:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:50 smithi138 ceph-mon[36954]: pgmap v3926: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:31:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:50 smithi100 ceph-mon[32599]: pgmap v3926: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:31:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:50 smithi167 ceph-mon[37140]: pgmap v3926: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:31:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:52 smithi138 ceph-mon[36954]: pgmap v3927: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:31:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:52 smithi100 ceph-mon[32599]: pgmap v3927: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:31:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:52 smithi167 ceph-mon[37140]: pgmap v3927: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:31:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:54 smithi138 ceph-mon[36954]: pgmap v3928: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:31:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:54 smithi100 ceph-mon[32599]: pgmap v3928: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:31:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:54 smithi167 ceph-mon[37140]: pgmap v3928: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:31:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:56 smithi138 ceph-mon[36954]: pgmap v3929: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:31:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:56 smithi100 ceph-mon[32599]: pgmap v3929: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:31:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:56 smithi167 ceph-mon[37140]: pgmap v3929: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:31:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:31:58 smithi138 ceph-mon[36954]: pgmap v3930: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:31:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:31:58 smithi100 ceph-mon[32599]: pgmap v3930: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:31:59.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:31:58 smithi167 ceph-mon[37140]: pgmap v3930: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:32:00.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:00 smithi167 ceph-mon[37140]: pgmap v3931: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:32:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:00 smithi138 ceph-mon[36954]: pgmap v3931: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:32:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:00 smithi100 ceph-mon[32599]: pgmap v3931: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:32:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:02 smithi138 ceph-mon[36954]: pgmap v3932: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:32:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:02 smithi100 ceph-mon[32599]: pgmap v3932: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:32:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:02 smithi167 ceph-mon[37140]: pgmap v3932: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:32:03.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:32:03.943+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:32:03.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:32:03.943+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:32:03.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:32:03.943+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:32:03.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:32:03.943+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:32:03.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:32:03.943+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:32:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:04 smithi138 ceph-mon[36954]: pgmap v3933: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:32:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:04 smithi100 ceph-mon[32599]: pgmap v3933: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:32:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:04 smithi167 ceph-mon[37140]: pgmap v3933: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:32:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:06 smithi138 ceph-mon[36954]: pgmap v3934: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:32:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:06 smithi100 ceph-mon[32599]: pgmap v3934: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:32:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:06 smithi167 ceph-mon[37140]: pgmap v3934: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:32:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:08 smithi138 ceph-mon[36954]: pgmap v3935: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:32:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:08 smithi138 ceph-mon[36954]: osdmap e294: 12 total, 12 up, 12 in 2024-07-26T15:32:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:08 smithi100 ceph-mon[32599]: pgmap v3935: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:32:08.997 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:08 smithi100 ceph-mon[32599]: osdmap e294: 12 total, 12 up, 12 in 2024-07-26T15:32:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:08 smithi167 ceph-mon[37140]: pgmap v3935: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:32:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:08 smithi167 ceph-mon[37140]: osdmap e294: 12 total, 12 up, 12 in 2024-07-26T15:32:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:10 smithi138 ceph-mon[36954]: pgmap v3937: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:32:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:10 smithi100 ceph-mon[32599]: pgmap v3937: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:32:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:10 smithi167 ceph-mon[37140]: pgmap v3937: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:32:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:12 smithi138 ceph-mon[36954]: pgmap v3938: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:32:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:12 smithi100 ceph-mon[32599]: pgmap v3938: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:32:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:12 smithi167 ceph-mon[37140]: pgmap v3938: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:32:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:14 smithi138 ceph-mon[36954]: pgmap v3939: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:32:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:14 smithi100 ceph-mon[32599]: pgmap v3939: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:32:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:14 smithi167 ceph-mon[37140]: pgmap v3939: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:32:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:16 smithi138 ceph-mon[36954]: pgmap v3940: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:32:16.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:16 smithi138 ceph-mon[36954]: osdmap e295: 12 total, 12 up, 12 in 2024-07-26T15:32:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:16 smithi100 ceph-mon[32599]: pgmap v3940: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:32:16.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:16 smithi100 ceph-mon[32599]: osdmap e295: 12 total, 12 up, 12 in 2024-07-26T15:32:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:16 smithi167 ceph-mon[37140]: pgmap v3940: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:32:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:16 smithi167 ceph-mon[37140]: osdmap e295: 12 total, 12 up, 12 in 2024-07-26T15:32:18.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:18 smithi138 ceph-mon[36954]: pgmap v3942: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 226 B/s wr, 6 op/s 2024-07-26T15:32:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:18 smithi100 ceph-mon[32599]: pgmap v3942: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 226 B/s wr, 6 op/s 2024-07-26T15:32:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:18 smithi167 ceph-mon[37140]: pgmap v3942: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 226 B/s wr, 6 op/s 2024-07-26T15:32:18.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:32:18.958+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:32:18.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:32:18.958+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:32:18.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:32:18.958+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:32:18.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:32:18.958+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:32:18.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:32:18.959+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:32:20.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:20 smithi138 ceph-mon[36954]: pgmap v3943: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:32:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:20 smithi100 ceph-mon[32599]: pgmap v3943: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:32:20.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:20 smithi167 ceph-mon[37140]: pgmap v3943: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:32:22.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:22 smithi138 ceph-mon[36954]: pgmap v3944: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:32:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:22 smithi100 ceph-mon[32599]: pgmap v3944: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:32:22.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:22 smithi167 ceph-mon[37140]: pgmap v3944: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:32:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:24 smithi138 ceph-mon[36954]: pgmap v3945: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:32:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:24 smithi100 ceph-mon[32599]: pgmap v3945: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:32:24.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:24 smithi167 ceph-mon[37140]: pgmap v3945: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:32:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:26 smithi138 ceph-mon[36954]: pgmap v3946: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:32:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:26 smithi100 ceph-mon[32599]: pgmap v3946: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:32:26.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:26 smithi167 ceph-mon[37140]: pgmap v3946: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:32:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:28 smithi138 ceph-mon[36954]: pgmap v3947: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:32:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:28 smithi100 ceph-mon[32599]: pgmap v3947: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:32:28.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:28 smithi167 ceph-mon[37140]: pgmap v3947: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:32:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:30 smithi138 ceph-mon[36954]: pgmap v3948: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:32:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:30 smithi100 ceph-mon[32599]: pgmap v3948: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:32:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:30 smithi167 ceph-mon[37140]: pgmap v3948: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:32:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:32 smithi100 ceph-mon[32599]: pgmap v3949: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:32:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:32 smithi167 ceph-mon[37140]: pgmap v3949: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:32:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:32 smithi138 ceph-mon[36954]: pgmap v3949: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:32:33.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:32:33.968+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:32:33.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:32:33.969+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:32:33.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:32:33.969+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:32:33.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:32:33.969+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:32:33.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:32:33.969+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:32:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:34 smithi100 ceph-mon[32599]: pgmap v3950: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:32:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:34 smithi167 ceph-mon[37140]: pgmap v3950: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:32:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:34 smithi138 ceph-mon[36954]: pgmap v3950: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:32:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:36 smithi100 ceph-mon[32599]: pgmap v3951: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:32:36.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:36 smithi167 ceph-mon[37140]: pgmap v3951: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:32:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:36 smithi138 ceph-mon[36954]: pgmap v3951: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:32:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:38 smithi100 ceph-mon[32599]: pgmap v3952: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:32:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:38 smithi167 ceph-mon[37140]: pgmap v3952: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:32:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:38 smithi138 ceph-mon[36954]: pgmap v3952: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:32:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:40 smithi100 ceph-mon[32599]: pgmap v3953: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:32:40.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:40 smithi167 ceph-mon[37140]: pgmap v3953: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:32:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:40 smithi138 ceph-mon[36954]: pgmap v3953: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:32:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:42 smithi100 ceph-mon[32599]: pgmap v3954: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:32:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:42 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:32:42.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:42 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:32:42.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:42 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:32:42.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:42 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:32:42.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:42 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:32:42.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:42 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:32:42.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:42 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:32:42.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:42 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:32:42.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:42 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:32:42.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:42 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:32:42.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:42 smithi167 ceph-mon[37140]: pgmap v3954: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:32:42.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:42 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:32:42.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:42 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:32:42.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:42 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:32:42.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:42 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:32:42.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:42 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:32:42.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:42 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:32:42.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:42 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:32:42.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:42 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:32:42.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:42 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:32:42.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:42 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:32:42.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:42 smithi138 ceph-mon[36954]: pgmap v3954: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:32:42.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:42 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:32:42.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:42 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:32:42.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:42 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:32:42.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:42 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:32:42.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:42 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:32:42.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:42 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:32:42.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:42 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:32:42.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:42 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:32:42.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:42 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:32:42.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:42 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:32:44.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:44 smithi100 ceph-mon[32599]: pgmap v3955: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:32:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:44 smithi167 ceph-mon[37140]: pgmap v3955: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:32:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:44 smithi138 ceph-mon[36954]: pgmap v3955: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:32:46.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:46 smithi100 ceph-mon[32599]: pgmap v3956: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:32:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:46 smithi167 ceph-mon[37140]: pgmap v3956: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:32:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:46 smithi138 ceph-mon[36954]: pgmap v3956: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:32:48.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:48 smithi100 ceph-mon[32599]: pgmap v3957: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:32:48.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:48 smithi167 ceph-mon[37140]: pgmap v3957: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:32:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:48 smithi138 ceph-mon[36954]: pgmap v3957: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:32:48.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:32:48.983+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:32:48.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:32:48.984+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:32:48.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:32:48.984+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:32:48.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:32:48.984+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:32:48.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:32:48.984+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:32:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:50 smithi100 ceph-mon[32599]: pgmap v3958: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:32:50.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:50 smithi167 ceph-mon[37140]: pgmap v3958: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:32:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:50 smithi138 ceph-mon[36954]: pgmap v3958: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:32:52.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:52 smithi100 ceph-mon[32599]: pgmap v3959: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:32:52.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:52 smithi167 ceph-mon[37140]: pgmap v3959: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:32:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:52 smithi138 ceph-mon[36954]: pgmap v3959: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:32:54.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:54 smithi100 ceph-mon[32599]: pgmap v3960: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:32:54.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:54 smithi167 ceph-mon[37140]: pgmap v3960: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:32:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:54 smithi138 ceph-mon[36954]: pgmap v3960: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:32:56.743 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:56 smithi138 ceph-mon[36954]: pgmap v3961: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:32:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:56 smithi100 ceph-mon[32599]: pgmap v3961: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:32:56.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:56 smithi167 ceph-mon[37140]: pgmap v3961: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:32:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:32:58 smithi100 ceph-mon[32599]: pgmap v3962: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:32:58.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:32:58 smithi167 ceph-mon[37140]: pgmap v3962: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:32:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:32:58 smithi138 ceph-mon[36954]: pgmap v3962: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:33:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:33:00 smithi100 ceph-mon[32599]: pgmap v3963: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:33:00.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:33:00 smithi167 ceph-mon[37140]: pgmap v3963: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:33:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:33:00 smithi138 ceph-mon[36954]: pgmap v3963: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:33:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:33:02 smithi100 ceph-mon[32599]: pgmap v3964: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:33:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:33:02 smithi167 ceph-mon[37140]: pgmap v3964: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:33:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:33:02 smithi138 ceph-mon[36954]: pgmap v3964: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:33:03.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:33:03.032+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b4ef0 msgr2=0x7fe5940b5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-07-26T15:33:03.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:33:03.032+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b4ef0 msgr2=0x7fe5940b5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-07-26T15:33:03.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:33:03.032+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b4ef0 0x7fe5940b5300 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe590005210 tx=0x7fe59008b040 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:33:03.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:33:03.032+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b4ef0 0x7fe5940b5300 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe590005210 tx=0x7fe59008b040 comp rx=0 tx=0).stop 2024-07-26T15:33:03.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:33:03.032+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b4ef0 msgr2=0x7fe5940b5300 unknown :-1 s=STATE_CLOSED l=1).tick see no progress in more than 10000000 us during connecting to v2:172.21.15.100:6800/3937009534, fault. 2024-07-26T15:33:03.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:33:03.032+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b4ef0 msgr2=0x7fe5940b5300 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-07-26T15:33:03.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:33:03.032+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b4ef0 0x7fe5940b5300 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T15:33:03.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:33:03.032+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T15:33:03.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:33:03.032+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:33:03.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:33:03.033+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe590005d00 tx=0x7fe59003b040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T15:33:03.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:33:03.902+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c004740 con 0x7fe5a81087a0 2024-07-26T15:33:03.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:33:03.902+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T15:33:03.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:33:03.903+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 52 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T15:33:03.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:33:03.903+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 52 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c100 con 0x7fe5a81087a0 2024-07-26T15:33:03.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:33:03.984+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:33:03.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:33:03.985+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:33:03.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:33:03.985+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:33:03.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:33:03.985+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:33:03.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:33:03.985+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:33:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:33:04 smithi100 ceph-mon[32599]: pgmap v3965: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-07-26T15:33:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:33:04 smithi167 ceph-mon[37140]: pgmap v3965: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-07-26T15:33:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:33:04 smithi138 ceph-mon[36954]: pgmap v3965: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-07-26T15:33:06.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:33:06 smithi167 ceph-mon[37140]: pgmap v3966: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:33:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:33:06 smithi138 ceph-mon[36954]: pgmap v3966: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:33:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:33:06 smithi100 ceph-mon[32599]: pgmap v3966: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:33:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:33:08 smithi167 ceph-mon[37140]: pgmap v3967: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:33:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:33:08 smithi167 ceph-mon[37140]: osdmap e296: 12 total, 12 up, 12 in 2024-07-26T15:33:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:33:08 smithi138 ceph-mon[36954]: pgmap v3967: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:33:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:33:08 smithi138 ceph-mon[36954]: osdmap e296: 12 total, 12 up, 12 in 2024-07-26T15:33:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:33:08 smithi100 ceph-mon[32599]: pgmap v3967: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:33:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:33:08 smithi100 ceph-mon[32599]: osdmap e296: 12 total, 12 up, 12 in 2024-07-26T15:33:10.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:33:10 smithi167 ceph-mon[37140]: pgmap v3969: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:33:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:33:10 smithi138 ceph-mon[36954]: pgmap v3969: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:33:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:33:10 smithi100 ceph-mon[32599]: pgmap v3969: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:33:12.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:33:12 smithi167 ceph-mon[37140]: pgmap v3970: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:33:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:33:12 smithi138 ceph-mon[36954]: pgmap v3970: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:33:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:33:12 smithi100 ceph-mon[32599]: pgmap v3970: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:33:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:33:14 smithi167 ceph-mon[37140]: pgmap v3971: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:33:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:33:14 smithi138 ceph-mon[36954]: pgmap v3971: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:33:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:33:14 smithi100 ceph-mon[32599]: pgmap v3971: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:33:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:33:16 smithi167 ceph-mon[37140]: pgmap v3972: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:33:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:33:16 smithi138 ceph-mon[36954]: pgmap v3972: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:33:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:33:16 smithi100 ceph-mon[32599]: pgmap v3972: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:33:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:33:18 smithi167 ceph-mon[37140]: pgmap v3973: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:33:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:33:18 smithi138 ceph-mon[36954]: pgmap v3973: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:33:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:33:18 smithi100 ceph-mon[32599]: pgmap v3973: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:33:19.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:33:19.000+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:33:19.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:33:19.001+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:33:19.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:33:19.001+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:33:19.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:33:19.001+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:33:19.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:33:19.001+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:33:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:33:20 smithi138 ceph-mon[36954]: pgmap v3974: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 183 B/s wr, 6 op/s 2024-07-26T15:33:20.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:33:20 smithi138 ceph-mon[36954]: osdmap e297: 12 total, 12 up, 12 in 2024-07-26T15:33:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:33:20 smithi100 ceph-mon[32599]: pgmap v3974: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 183 B/s wr, 6 op/s 2024-07-26T15:33:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:33:20 smithi100 ceph-mon[32599]: osdmap e297: 12 total, 12 up, 12 in 2024-07-26T15:33:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:33:20 smithi167 ceph-mon[37140]: pgmap v3974: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 183 B/s wr, 6 op/s 2024-07-26T15:33:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:33:20 smithi167 ceph-mon[37140]: osdmap e297: 12 total, 12 up, 12 in 2024-07-26T15:33:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:33:22 smithi138 ceph-mon[36954]: pgmap v3976: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:33:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:33:22 smithi100 ceph-mon[32599]: pgmap v3976: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:33:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:33:22 smithi167 ceph-mon[37140]: pgmap v3976: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:33:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:33:24 smithi138 ceph-mon[36954]: pgmap v3977: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:33:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:33:24 smithi100 ceph-mon[32599]: pgmap v3977: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:33:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:33:24 smithi167 ceph-mon[37140]: pgmap v3977: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:33:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:33:26 smithi138 ceph-mon[36954]: pgmap v3978: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:33:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:33:26 smithi100 ceph-mon[32599]: pgmap v3978: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:33:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:33:26 smithi167 ceph-mon[37140]: pgmap v3978: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:33:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:33:28 smithi138 ceph-mon[36954]: pgmap v3979: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:33:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:33:28 smithi100 ceph-mon[32599]: pgmap v3979: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:33:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:33:28 smithi167 ceph-mon[37140]: pgmap v3979: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:33:30.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:33:30 smithi167 ceph-mon[37140]: pgmap v3980: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:33:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:33:30 smithi138 ceph-mon[36954]: pgmap v3980: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:33:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:33:30 smithi100 ceph-mon[32599]: pgmap v3980: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:33:32.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:33:32 smithi138 ceph-mon[36954]: pgmap v3981: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T15:33:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:33:32 smithi100 ceph-mon[32599]: pgmap v3981: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T15:33:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:33:32 smithi167 ceph-mon[37140]: pgmap v3981: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T15:33:34.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:33:34.006+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:33:34.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:33:34.007+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:33:34.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:33:34.007+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:33:34.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:33:34.007+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:33:34.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:33:34.007+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:33:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:33:34 smithi138 ceph-mon[36954]: pgmap v3982: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:33:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:33:34 smithi100 ceph-mon[32599]: pgmap v3982: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:33:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:33:34 smithi167 ceph-mon[37140]: pgmap v3982: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:33:36.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:33:36 smithi138 ceph-mon[36954]: pgmap v3983: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:33:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:33:36 smithi100 ceph-mon[32599]: pgmap v3983: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:33:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:33:36 smithi167 ceph-mon[37140]: pgmap v3983: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:33:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:33:38 smithi138 ceph-mon[36954]: pgmap v3984: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:33:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:33:38 smithi100 ceph-mon[32599]: pgmap v3984: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:33:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:33:38 smithi167 ceph-mon[37140]: pgmap v3984: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:33:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:33:40 smithi138 ceph-mon[36954]: pgmap v3985: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:33:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:33:40 smithi100 ceph-mon[32599]: pgmap v3985: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:33:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:33:40 smithi167 ceph-mon[37140]: pgmap v3985: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:33:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:33:42 smithi138 ceph-mon[36954]: pgmap v3986: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:33:42.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:33:42 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:33:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:33:42 smithi100 ceph-mon[32599]: pgmap v3986: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:33:42.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:33:42 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:33:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:33:42 smithi167 ceph-mon[37140]: pgmap v3986: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:33:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:33:42 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:33:43.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:33:43 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:33:43.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:33:43 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:33:43.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:33:43 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:33:43.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:33:43 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:33:43.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:33:43 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:33:43.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:33:43 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:33:44.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:33:43 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:33:44.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:33:43 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:33:44.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:33:43 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:33:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:33:44 smithi138 ceph-mon[36954]: pgmap v3987: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:33:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:33:44 smithi100 ceph-mon[32599]: pgmap v3987: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:33:45.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:33:44 smithi167 ceph-mon[37140]: pgmap v3987: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:33:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:33:46 smithi138 ceph-mon[36954]: pgmap v3988: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:33:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:33:46 smithi100 ceph-mon[32599]: pgmap v3988: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:33:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:33:46 smithi167 ceph-mon[37140]: pgmap v3988: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:33:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:33:48 smithi138 ceph-mon[36954]: pgmap v3989: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:33:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:33:48 smithi100 ceph-mon[32599]: pgmap v3989: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:33:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:33:48 smithi167 ceph-mon[37140]: pgmap v3989: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:33:49.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:33:49.023+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:33:49.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:33:49.024+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:33:49.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:33:49.024+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:33:49.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:33:49.024+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:33:49.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:33:49.024+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:33:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:33:50 smithi138 ceph-mon[36954]: pgmap v3990: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:33:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:33:50 smithi100 ceph-mon[32599]: pgmap v3990: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:33:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:33:50 smithi167 ceph-mon[37140]: pgmap v3990: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:33:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:33:52 smithi138 ceph-mon[36954]: pgmap v3991: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:33:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:33:52 smithi100 ceph-mon[32599]: pgmap v3991: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:33:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:33:52 smithi167 ceph-mon[37140]: pgmap v3991: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:33:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:33:54 smithi138 ceph-mon[36954]: pgmap v3992: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:33:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:33:54 smithi100 ceph-mon[32599]: pgmap v3992: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:33:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:33:54 smithi167 ceph-mon[37140]: pgmap v3992: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:33:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:33:56 smithi138 ceph-mon[36954]: pgmap v3993: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:33:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:33:56 smithi100 ceph-mon[32599]: pgmap v3993: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:33:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:33:56 smithi167 ceph-mon[37140]: pgmap v3993: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:33:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:33:58 smithi138 ceph-mon[36954]: pgmap v3994: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:33:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:33:58 smithi100 ceph-mon[32599]: pgmap v3994: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:33:59.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:33:58 smithi167 ceph-mon[37140]: pgmap v3994: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:34:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:34:00 smithi138 ceph-mon[36954]: pgmap v3995: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:34:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:34:00 smithi100 ceph-mon[32599]: pgmap v3995: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:34:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:34:00 smithi167 ceph-mon[37140]: pgmap v3995: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:34:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:34:02 smithi138 ceph-mon[36954]: pgmap v3996: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:34:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:34:02 smithi100 ceph-mon[32599]: pgmap v3996: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:34:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:34:02 smithi167 ceph-mon[37140]: pgmap v3996: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:34:04.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:34:04.038+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:34:04.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:34:04.039+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:34:04.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:34:04.039+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:34:04.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:34:04.039+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:34:04.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:34:04.039+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:34:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:34:04 smithi138 ceph-mon[36954]: pgmap v3997: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-07-26T15:34:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:34:04 smithi100 ceph-mon[32599]: pgmap v3997: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-07-26T15:34:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:34:04 smithi167 ceph-mon[37140]: pgmap v3997: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-07-26T15:34:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:34:06 smithi138 ceph-mon[36954]: pgmap v3998: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:34:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:34:06 smithi100 ceph-mon[32599]: pgmap v3998: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:34:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:34:06 smithi167 ceph-mon[37140]: pgmap v3998: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:34:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:34:08 smithi138 ceph-mon[36954]: pgmap v3999: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:34:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:34:08 smithi138 ceph-mon[36954]: osdmap e298: 12 total, 12 up, 12 in 2024-07-26T15:34:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:34:08 smithi100 ceph-mon[32599]: pgmap v3999: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:34:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:34:08 smithi100 ceph-mon[32599]: osdmap e298: 12 total, 12 up, 12 in 2024-07-26T15:34:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:34:08 smithi167 ceph-mon[37140]: pgmap v3999: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:34:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:34:08 smithi167 ceph-mon[37140]: osdmap e298: 12 total, 12 up, 12 in 2024-07-26T15:34:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:34:10 smithi138 ceph-mon[36954]: pgmap v4001: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:34:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:34:10 smithi100 ceph-mon[32599]: pgmap v4001: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:34:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:34:10 smithi167 ceph-mon[37140]: pgmap v4001: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:34:11.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:34:11 smithi138 ceph-mon[36954]: pgmap v4002: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:34:11.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:34:11 smithi100 ceph-mon[32599]: pgmap v4002: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:34:12.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:34:11 smithi167 ceph-mon[37140]: pgmap v4002: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:34:13.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:34:13 smithi100 ceph-mon[32599]: pgmap v4003: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:34:14.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:34:13 smithi167 ceph-mon[37140]: pgmap v4003: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:34:14.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:34:13 smithi138 ceph-mon[36954]: pgmap v4003: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:34:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:34:16 smithi100 ceph-mon[32599]: pgmap v4004: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:34:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:34:16 smithi100 ceph-mon[32599]: osdmap e299: 12 total, 12 up, 12 in 2024-07-26T15:34:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:34:16 smithi167 ceph-mon[37140]: pgmap v4004: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:34:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:34:16 smithi167 ceph-mon[37140]: osdmap e299: 12 total, 12 up, 12 in 2024-07-26T15:34:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:34:16 smithi138 ceph-mon[36954]: pgmap v4004: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:34:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:34:16 smithi138 ceph-mon[36954]: osdmap e299: 12 total, 12 up, 12 in 2024-07-26T15:34:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:34:18 smithi100 ceph-mon[32599]: pgmap v4006: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 226 B/s wr, 6 op/s 2024-07-26T15:34:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:34:18 smithi167 ceph-mon[37140]: pgmap v4006: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 226 B/s wr, 6 op/s 2024-07-26T15:34:18.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:34:18 smithi138 ceph-mon[36954]: pgmap v4006: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 226 B/s wr, 6 op/s 2024-07-26T15:34:19.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:34:19.044+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:34:19.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:34:19.045+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:34:19.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:34:19.045+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:34:19.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:34:19.045+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:34:19.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:34:19.045+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:34:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:34:20 smithi100 ceph-mon[32599]: pgmap v4007: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:34:20.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:34:20 smithi167 ceph-mon[37140]: pgmap v4007: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:34:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:34:20 smithi138 ceph-mon[36954]: pgmap v4007: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:34:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:34:22 smithi100 ceph-mon[32599]: pgmap v4008: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:34:22.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:34:22 smithi167 ceph-mon[37140]: pgmap v4008: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:34:22.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:34:22 smithi138 ceph-mon[36954]: pgmap v4008: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:34:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:34:24 smithi100 ceph-mon[32599]: pgmap v4009: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:34:24.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:34:24 smithi167 ceph-mon[37140]: pgmap v4009: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:34:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:34:24 smithi138 ceph-mon[36954]: pgmap v4009: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:34:26.743 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:34:26 smithi138 ceph-mon[36954]: pgmap v4010: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:34:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:34:26 smithi100 ceph-mon[32599]: pgmap v4010: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:34:26.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:34:26 smithi167 ceph-mon[37140]: pgmap v4010: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:34:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:34:28 smithi100 ceph-mon[32599]: pgmap v4011: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:34:28.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:34:28 smithi167 ceph-mon[37140]: pgmap v4011: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:34:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:34:28 smithi138 ceph-mon[36954]: pgmap v4011: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:34:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:34:30 smithi100 ceph-mon[32599]: pgmap v4012: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:34:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:34:30 smithi167 ceph-mon[37140]: pgmap v4012: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:34:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:34:30 smithi138 ceph-mon[36954]: pgmap v4012: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:34:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:34:32 smithi100 ceph-mon[32599]: pgmap v4013: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:34:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:34:32 smithi167 ceph-mon[37140]: pgmap v4013: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:34:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:34:32 smithi138 ceph-mon[36954]: pgmap v4013: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:34:34.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:34:34.061+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:34:34.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:34:34.061+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:34:34.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:34:34.061+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:34:34.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:34:34.061+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:34:34.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:34:34.061+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:34:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:34:34 smithi100 ceph-mon[32599]: pgmap v4014: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:34:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:34:34 smithi167 ceph-mon[37140]: pgmap v4014: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:34:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:34:34 smithi138 ceph-mon[36954]: pgmap v4014: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:34:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:34:36 smithi100 ceph-mon[32599]: pgmap v4015: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:34:36.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:34:36 smithi167 ceph-mon[37140]: pgmap v4015: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:34:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:34:36 smithi138 ceph-mon[36954]: pgmap v4015: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:34:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:34:38 smithi100 ceph-mon[32599]: pgmap v4016: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:34:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:34:38 smithi167 ceph-mon[37140]: pgmap v4016: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:34:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:34:38 smithi138 ceph-mon[36954]: pgmap v4016: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:34:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:34:40 smithi100 ceph-mon[32599]: pgmap v4017: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:34:40.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:34:40 smithi167 ceph-mon[37140]: pgmap v4017: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:34:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:34:40 smithi138 ceph-mon[36954]: pgmap v4017: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:34:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:34:42 smithi100 ceph-mon[32599]: pgmap v4018: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:34:42.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:34:42 smithi167 ceph-mon[37140]: pgmap v4018: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:34:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:34:42 smithi138 ceph-mon[36954]: pgmap v4018: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:34:43.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:34:43 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:34:43.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:34:43 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:34:43.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:34:43 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:34:43.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:34:43 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:34:43.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:34:43 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:34:43.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:34:43 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:34:43.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:34:43 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:34:43.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:34:43 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:34:43.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:34:43 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:34:43.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:34:43 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:34:43.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:34:43 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:34:43.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:34:43 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:34:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:34:44 smithi167 ceph-mon[37140]: pgmap v4019: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:34:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:34:44 smithi138 ceph-mon[36954]: pgmap v4019: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:34:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:34:44 smithi100 ceph-mon[32599]: pgmap v4019: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:34:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:34:46 smithi167 ceph-mon[37140]: pgmap v4020: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:34:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:34:46 smithi138 ceph-mon[36954]: pgmap v4020: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:34:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:34:46 smithi100 ceph-mon[32599]: pgmap v4020: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:34:48.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:34:48 smithi167 ceph-mon[37140]: pgmap v4021: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:34:48.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:34:48 smithi138 ceph-mon[36954]: pgmap v4021: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:34:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:34:48 smithi100 ceph-mon[32599]: pgmap v4021: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:34:49.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:34:49.076+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:34:49.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:34:49.076+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:34:49.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:34:49.076+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:34:49.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:34:49.076+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:34:49.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:34:49.076+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:34:50.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:34:50 smithi167 ceph-mon[37140]: pgmap v4022: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:34:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:34:50 smithi138 ceph-mon[36954]: pgmap v4022: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:34:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:34:50 smithi100 ceph-mon[32599]: pgmap v4022: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:34:52.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:34:52 smithi167 ceph-mon[37140]: pgmap v4023: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:34:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:34:52 smithi138 ceph-mon[36954]: pgmap v4023: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:34:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:34:52 smithi100 ceph-mon[32599]: pgmap v4023: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:34:54.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:34:54 smithi167 ceph-mon[37140]: pgmap v4024: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:34:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:34:54 smithi138 ceph-mon[36954]: pgmap v4024: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:34:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:34:54 smithi100 ceph-mon[32599]: pgmap v4024: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:34:56.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:34:56 smithi167 ceph-mon[37140]: pgmap v4025: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:34:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:34:56 smithi138 ceph-mon[36954]: pgmap v4025: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:34:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:34:56 smithi100 ceph-mon[32599]: pgmap v4025: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:34:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:34:58 smithi138 ceph-mon[36954]: pgmap v4026: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:34:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:34:58 smithi100 ceph-mon[32599]: pgmap v4026: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:34:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:34:58 smithi167 ceph-mon[37140]: pgmap v4026: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:35:00.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:35:00 smithi167 ceph-mon[37140]: pgmap v4027: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:35:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:35:00 smithi138 ceph-mon[36954]: pgmap v4027: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:35:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:35:00 smithi100 ceph-mon[32599]: pgmap v4027: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:35:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:35:02 smithi138 ceph-mon[36954]: pgmap v4028: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:35:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:35:02 smithi100 ceph-mon[32599]: pgmap v4028: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:35:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:35:02 smithi167 ceph-mon[37140]: pgmap v4028: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:35:04.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:35:04.088+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:35:04.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:35:04.088+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:35:04.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:35:04.088+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:35:04.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:35:04.088+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:35:04.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:35:04.088+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:35:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:35:04 smithi138 ceph-mon[36954]: pgmap v4029: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-07-26T15:35:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:35:04 smithi100 ceph-mon[32599]: pgmap v4029: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-07-26T15:35:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:35:04 smithi167 ceph-mon[37140]: pgmap v4029: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-07-26T15:35:06.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:35:06 smithi100 ceph-mon[32599]: pgmap v4030: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:35:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:35:06 smithi138 ceph-mon[36954]: pgmap v4030: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:35:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:35:06 smithi167 ceph-mon[37140]: pgmap v4030: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:35:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:35:08 smithi138 ceph-mon[36954]: pgmap v4031: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:35:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:35:08 smithi138 ceph-mon[36954]: osdmap e300: 12 total, 12 up, 12 in 2024-07-26T15:35:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:35:08 smithi100 ceph-mon[32599]: pgmap v4031: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:35:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:35:08 smithi100 ceph-mon[32599]: osdmap e300: 12 total, 12 up, 12 in 2024-07-26T15:35:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:35:08 smithi167 ceph-mon[37140]: pgmap v4031: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:35:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:35:08 smithi167 ceph-mon[37140]: osdmap e300: 12 total, 12 up, 12 in 2024-07-26T15:35:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:35:10 smithi138 ceph-mon[36954]: pgmap v4033: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:35:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:35:10 smithi100 ceph-mon[32599]: pgmap v4033: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:35:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:35:10 smithi167 ceph-mon[37140]: pgmap v4033: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:35:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:35:12 smithi138 ceph-mon[36954]: pgmap v4034: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:35:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:35:12 smithi100 ceph-mon[32599]: pgmap v4034: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:35:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:35:12 smithi167 ceph-mon[37140]: pgmap v4034: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:35:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:35:14 smithi138 ceph-mon[36954]: pgmap v4035: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:35:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:35:14 smithi100 ceph-mon[32599]: pgmap v4035: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:35:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:35:14 smithi167 ceph-mon[37140]: pgmap v4035: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:35:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:35:16 smithi138 ceph-mon[36954]: pgmap v4036: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:35:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:35:16 smithi100 ceph-mon[32599]: pgmap v4036: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:35:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:35:16 smithi167 ceph-mon[37140]: pgmap v4036: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:35:18.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:35:18.922+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c004740 con 0x7fe5a81087a0 2024-07-26T15:35:18.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:35:18.922+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T15:35:18.924 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:35:18 smithi100 ceph-mon[32599]: pgmap v4037: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:35:18.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:35:18.922+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 53 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T15:35:18.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:35:18.922+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 53 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe5a008c0e0 con 0x7fe5a81087a0 2024-07-26T15:35:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:35:18 smithi138 ceph-mon[36954]: pgmap v4037: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:35:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:35:18 smithi167 ceph-mon[37140]: pgmap v4037: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:35:19.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:35:19.104+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:35:19.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:35:19.104+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:35:19.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:35:19.104+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:35:19.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:35:19.104+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:35:19.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:35:19.104+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:35:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:35:20 smithi138 ceph-mon[36954]: pgmap v4038: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 183 B/s wr, 6 op/s 2024-07-26T15:35:20.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:35:20 smithi138 ceph-mon[36954]: osdmap e301: 12 total, 12 up, 12 in 2024-07-26T15:35:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:35:20 smithi100 ceph-mon[32599]: pgmap v4038: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 183 B/s wr, 6 op/s 2024-07-26T15:35:20.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:35:20 smithi100 ceph-mon[32599]: osdmap e301: 12 total, 12 up, 12 in 2024-07-26T15:35:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:35:20 smithi167 ceph-mon[37140]: pgmap v4038: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 183 B/s wr, 6 op/s 2024-07-26T15:35:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:35:20 smithi167 ceph-mon[37140]: osdmap e301: 12 total, 12 up, 12 in 2024-07-26T15:35:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:35:22 smithi138 ceph-mon[36954]: pgmap v4040: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:35:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:35:22 smithi100 ceph-mon[32599]: pgmap v4040: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:35:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:35:22 smithi167 ceph-mon[37140]: pgmap v4040: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:35:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:35:24 smithi138 ceph-mon[36954]: pgmap v4041: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:35:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:35:24 smithi100 ceph-mon[32599]: pgmap v4041: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:35:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:35:24 smithi167 ceph-mon[37140]: pgmap v4041: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:35:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:35:26 smithi138 ceph-mon[36954]: pgmap v4042: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:35:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:35:26 smithi100 ceph-mon[32599]: pgmap v4042: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:35:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:35:26 smithi167 ceph-mon[37140]: pgmap v4042: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:35:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:35:28 smithi138 ceph-mon[36954]: pgmap v4043: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:35:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:35:28 smithi100 ceph-mon[32599]: pgmap v4043: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:35:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:35:28 smithi167 ceph-mon[37140]: pgmap v4043: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:35:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:35:30 smithi138 ceph-mon[36954]: pgmap v4044: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:35:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:35:30 smithi100 ceph-mon[32599]: pgmap v4044: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:35:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:35:30 smithi167 ceph-mon[37140]: pgmap v4044: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:35:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:35:32 smithi138 ceph-mon[36954]: pgmap v4045: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T15:35:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:35:32 smithi100 ceph-mon[32599]: pgmap v4045: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T15:35:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:35:32 smithi167 ceph-mon[37140]: pgmap v4045: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T15:35:34.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:35:34.116+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:35:34.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:35:34.117+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:35:34.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:35:34.117+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:35:34.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:35:34.117+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:35:34.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:35:34.117+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:35:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:35:34 smithi138 ceph-mon[36954]: pgmap v4046: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:35:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:35:34 smithi100 ceph-mon[32599]: pgmap v4046: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:35:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:35:34 smithi167 ceph-mon[37140]: pgmap v4046: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:35:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:35:36 smithi138 ceph-mon[36954]: pgmap v4047: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:35:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:35:36 smithi100 ceph-mon[32599]: pgmap v4047: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:35:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:35:36 smithi167 ceph-mon[37140]: pgmap v4047: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:35:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:35:38 smithi138 ceph-mon[36954]: pgmap v4048: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:35:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:35:38 smithi100 ceph-mon[32599]: pgmap v4048: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:35:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:35:38 smithi167 ceph-mon[37140]: pgmap v4048: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:35:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:35:40 smithi138 ceph-mon[36954]: pgmap v4049: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:35:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:35:40 smithi100 ceph-mon[32599]: pgmap v4049: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:35:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:35:40 smithi167 ceph-mon[37140]: pgmap v4049: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:35:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:35:42 smithi138 ceph-mon[36954]: pgmap v4050: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:35:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:35:42 smithi100 ceph-mon[32599]: pgmap v4050: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:35:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:35:42 smithi167 ceph-mon[37140]: pgmap v4050: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:35:43.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:35:43 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:35:43.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:35:43 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:35:44.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:35:43 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:35:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:35:44 smithi138 ceph-mon[36954]: pgmap v4051: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:35:44.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:35:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:35:44.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:35:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:35:44.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:35:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:35:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:35:44 smithi100 ceph-mon[32599]: pgmap v4051: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:35:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:35:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:35:44.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:35:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:35:44.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:35:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:35:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:35:44 smithi167 ceph-mon[37140]: pgmap v4051: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:35:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:35:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:35:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:35:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:35:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:35:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:35:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:35:46 smithi138 ceph-mon[36954]: pgmap v4052: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:35:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:35:46 smithi100 ceph-mon[32599]: pgmap v4052: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:35:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:35:46 smithi167 ceph-mon[37140]: pgmap v4052: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:35:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:35:48 smithi138 ceph-mon[36954]: pgmap v4053: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:35:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:35:48 smithi100 ceph-mon[32599]: pgmap v4053: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:35:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:35:48 smithi167 ceph-mon[37140]: pgmap v4053: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:35:49.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:35:49.119+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:35:49.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:35:49.120+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:35:49.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:35:49.120+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:35:49.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:35:49.120+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:35:49.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:35:49.120+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:35:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:35:50 smithi138 ceph-mon[36954]: pgmap v4054: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:35:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:35:50 smithi100 ceph-mon[32599]: pgmap v4054: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:35:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:35:50 smithi167 ceph-mon[37140]: pgmap v4054: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:35:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:35:52 smithi138 ceph-mon[36954]: pgmap v4055: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:35:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:35:52 smithi100 ceph-mon[32599]: pgmap v4055: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:35:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:35:52 smithi167 ceph-mon[37140]: pgmap v4055: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:35:54.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:35:54 smithi138 ceph-mon[36954]: pgmap v4056: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:35:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:35:54 smithi100 ceph-mon[32599]: pgmap v4056: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:35:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:35:54 smithi167 ceph-mon[37140]: pgmap v4056: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:35:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:35:56 smithi138 ceph-mon[36954]: pgmap v4057: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:35:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:35:56 smithi100 ceph-mon[32599]: pgmap v4057: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:35:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:35:56 smithi167 ceph-mon[37140]: pgmap v4057: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:35:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:35:58 smithi138 ceph-mon[36954]: pgmap v4058: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:35:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:35:58 smithi100 ceph-mon[32599]: pgmap v4058: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:35:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:35:58 smithi167 ceph-mon[37140]: pgmap v4058: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:35:59.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:35:59 smithi138 ceph-mon[36954]: pgmap v4059: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T15:35:59.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:35:59 smithi100 ceph-mon[32599]: pgmap v4059: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T15:36:00.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:35:59 smithi167 ceph-mon[37140]: pgmap v4059: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T15:36:02.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:36:01 smithi167 ceph-mon[37140]: pgmap v4060: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:36:02.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:36:01 smithi138 ceph-mon[36954]: pgmap v4060: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:36:02.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:36:01 smithi100 ceph-mon[32599]: pgmap v4060: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:36:03.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:36:03.928+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T15:36:03.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:36:03.928+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c003690 con 0x7fe5a81087a0 2024-07-26T15:36:03.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:36:03.928+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 54 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T15:36:03.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:36:03.929+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 54 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c100 con 0x7fe5a81087a0 2024-07-26T15:36:04.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:36:03 smithi167 ceph-mon[37140]: pgmap v4061: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:36:04.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:36:04.135+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:36:04.137 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:36:03 smithi100 ceph-mon[32599]: pgmap v4061: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:36:04.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:36:04.135+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:36:04.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:36:04.135+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:36:04.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:36:04.135+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:36:04.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:36:04.135+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:36:04.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:36:03 smithi138 ceph-mon[36954]: pgmap v4061: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:36:06.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:36:05 smithi138 ceph-mon[36954]: pgmap v4062: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:36:06.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:36:05 smithi100 ceph-mon[32599]: pgmap v4062: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:36:06.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:36:05 smithi167 ceph-mon[37140]: pgmap v4062: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:36:08.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:36:07 smithi138 ceph-mon[36954]: pgmap v4063: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:36:08.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:36:07 smithi100 ceph-mon[32599]: pgmap v4063: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:36:08.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:36:07 smithi167 ceph-mon[37140]: pgmap v4063: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:36:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:36:08 smithi138 ceph-mon[36954]: osdmap e302: 12 total, 12 up, 12 in 2024-07-26T15:36:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:36:08 smithi100 ceph-mon[32599]: osdmap e302: 12 total, 12 up, 12 in 2024-07-26T15:36:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:36:08 smithi167 ceph-mon[37140]: osdmap e302: 12 total, 12 up, 12 in 2024-07-26T15:36:10.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:36:09 smithi138 ceph-mon[36954]: pgmap v4065: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:36:10.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:36:09 smithi100 ceph-mon[32599]: pgmap v4065: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:36:10.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:36:09 smithi167 ceph-mon[37140]: pgmap v4065: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:36:12.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:36:11 smithi138 ceph-mon[36954]: pgmap v4066: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:36:12.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:36:11 smithi100 ceph-mon[32599]: pgmap v4066: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:36:12.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:36:11 smithi167 ceph-mon[37140]: pgmap v4066: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:36:14.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:36:13 smithi138 ceph-mon[36954]: pgmap v4067: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:36:14.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:36:13 smithi100 ceph-mon[32599]: pgmap v4067: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:36:14.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:36:13 smithi167 ceph-mon[37140]: pgmap v4067: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:36:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:36:16 smithi100 ceph-mon[32599]: pgmap v4068: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:36:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:36:16 smithi100 ceph-mon[32599]: osdmap e303: 12 total, 12 up, 12 in 2024-07-26T15:36:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:36:16 smithi167 ceph-mon[37140]: pgmap v4068: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:36:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:36:16 smithi167 ceph-mon[37140]: osdmap e303: 12 total, 12 up, 12 in 2024-07-26T15:36:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:36:16 smithi138 ceph-mon[36954]: pgmap v4068: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:36:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:36:16 smithi138 ceph-mon[36954]: osdmap e303: 12 total, 12 up, 12 in 2024-07-26T15:36:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:36:18 smithi100 ceph-mon[32599]: pgmap v4070: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:36:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:36:18 smithi167 ceph-mon[37140]: pgmap v4070: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:36:18.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:36:18 smithi138 ceph-mon[36954]: pgmap v4070: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T15:36:19.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:36:19.151+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:36:19.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:36:19.152+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:36:19.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:36:19.152+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:36:19.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:36:19.152+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:36:19.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:36:19.152+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:36:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:36:20 smithi100 ceph-mon[32599]: pgmap v4071: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:36:20.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:36:20 smithi167 ceph-mon[37140]: pgmap v4071: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:36:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:36:20 smithi138 ceph-mon[36954]: pgmap v4071: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:36:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:36:22 smithi100 ceph-mon[32599]: pgmap v4072: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:36:22.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:36:22 smithi167 ceph-mon[37140]: pgmap v4072: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:36:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:36:22 smithi138 ceph-mon[36954]: pgmap v4072: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:36:24.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:36:24 smithi167 ceph-mon[37140]: pgmap v4073: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:36:24.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:36:24 smithi138 ceph-mon[36954]: pgmap v4073: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:36:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:36:24 smithi100 ceph-mon[32599]: pgmap v4073: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:36:26.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:36:26 smithi167 ceph-mon[37140]: pgmap v4074: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:36:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:36:26 smithi138 ceph-mon[36954]: pgmap v4074: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:36:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:36:26 smithi100 ceph-mon[32599]: pgmap v4074: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:36:28.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:36:28 smithi167 ceph-mon[37140]: pgmap v4075: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:36:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:36:28 smithi138 ceph-mon[36954]: pgmap v4075: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:36:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:36:28 smithi100 ceph-mon[32599]: pgmap v4075: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:36:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:36:30 smithi167 ceph-mon[37140]: pgmap v4076: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:36:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:36:30 smithi138 ceph-mon[36954]: pgmap v4076: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:36:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:36:30 smithi100 ceph-mon[32599]: pgmap v4076: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:36:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:36:32 smithi167 ceph-mon[37140]: pgmap v4077: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:36:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:36:32 smithi138 ceph-mon[36954]: pgmap v4077: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:36:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:36:32 smithi100 ceph-mon[32599]: pgmap v4077: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:36:34.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:36:34.167+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:36:34.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:36:34.168+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:36:34.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:36:34.168+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:36:34.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:36:34.168+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:36:34.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:36:34.168+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:36:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:36:34 smithi167 ceph-mon[37140]: pgmap v4078: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:36:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:36:34 smithi138 ceph-mon[36954]: pgmap v4078: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:36:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:36:34 smithi100 ceph-mon[32599]: pgmap v4078: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:36:36.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:36:36 smithi167 ceph-mon[37140]: pgmap v4079: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:36:36.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:36:36 smithi100 ceph-mon[32599]: pgmap v4079: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:36:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:36:36 smithi138 ceph-mon[36954]: pgmap v4079: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:36:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:36:38 smithi167 ceph-mon[37140]: pgmap v4080: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:36:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:36:38 smithi138 ceph-mon[36954]: pgmap v4080: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:36:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:36:38 smithi100 ceph-mon[32599]: pgmap v4080: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:36:40.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:36:40 smithi167 ceph-mon[37140]: pgmap v4081: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:36:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:36:40 smithi138 ceph-mon[36954]: pgmap v4081: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:36:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:36:40 smithi100 ceph-mon[32599]: pgmap v4081: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:36:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:36:42 smithi138 ceph-mon[36954]: pgmap v4082: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:36:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:36:42 smithi100 ceph-mon[32599]: pgmap v4082: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:36:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:36:42 smithi167 ceph-mon[37140]: pgmap v4082: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:36:44.889 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:36:44 smithi100 ceph-mon[32599]: pgmap v4083: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:36:44.889 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:36:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:36:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:36:44 smithi138 ceph-mon[36954]: pgmap v4083: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:36:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:36:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:36:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:36:44 smithi167 ceph-mon[37140]: pgmap v4083: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:36:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:36:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:36:46.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:36:46 smithi138 ceph-mon[36954]: pgmap v4084: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:36:46.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:36:46 smithi100 ceph-mon[32599]: pgmap v4084: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:36:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:36:46 smithi167 ceph-mon[37140]: pgmap v4084: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:36:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:36:48 smithi100 ceph-mon[32599]: pgmap v4085: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:36:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:36:48 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:36:48.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:36:48 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:36:48.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:36:48 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:36:48.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:36:48 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:36:48.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:36:48 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:36:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:36:48 smithi138 ceph-mon[36954]: pgmap v4085: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:36:48.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:36:48 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:36:48.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:36:48 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:36:48.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:36:48 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:36:48.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:36:48 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:36:48.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:36:48 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:36:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:36:48 smithi167 ceph-mon[37140]: pgmap v4085: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:36:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:36:48 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:36:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:36:48 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:36:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:36:48 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:36:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:36:48 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:36:49.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:36:48 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:36:49.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:36:49.183+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:36:49.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:36:49.184+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:36:49.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:36:49.184+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:36:49.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:36:49.184+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:36:49.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:36:49.184+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:36:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:36:50 smithi138 ceph-mon[36954]: pgmap v4086: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:36:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:36:50 smithi100 ceph-mon[32599]: pgmap v4086: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:36:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:36:50 smithi167 ceph-mon[37140]: pgmap v4086: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:36:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:36:52 smithi138 ceph-mon[36954]: pgmap v4087: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:36:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:36:52 smithi100 ceph-mon[32599]: pgmap v4087: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:36:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:36:52 smithi167 ceph-mon[37140]: pgmap v4087: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:36:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:36:54 smithi138 ceph-mon[36954]: pgmap v4088: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:36:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:36:54 smithi100 ceph-mon[32599]: pgmap v4088: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:36:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:36:54 smithi167 ceph-mon[37140]: pgmap v4088: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:36:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:36:56 smithi138 ceph-mon[36954]: pgmap v4089: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:36:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:36:56 smithi100 ceph-mon[32599]: pgmap v4089: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:36:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:36:56 smithi167 ceph-mon[37140]: pgmap v4089: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:36:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:36:58 smithi100 ceph-mon[32599]: pgmap v4090: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:36:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:36:58 smithi138 ceph-mon[36954]: pgmap v4090: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:36:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:36:58 smithi167 ceph-mon[37140]: pgmap v4090: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:37:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:00 smithi138 ceph-mon[36954]: pgmap v4091: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:37:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:00 smithi100 ceph-mon[32599]: pgmap v4091: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:37:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:00 smithi167 ceph-mon[37140]: pgmap v4091: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:37:02.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:02 smithi167 ceph-mon[37140]: pgmap v4092: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:37:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:02 smithi138 ceph-mon[36954]: pgmap v4092: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:37:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:02 smithi100 ceph-mon[32599]: pgmap v4092: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:37:04.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:37:04.199+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:37:04.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:37:04.200+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:37:04.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:37:04.200+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:37:04.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:37:04.200+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:37:04.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:37:04.200+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:37:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:04 smithi138 ceph-mon[36954]: pgmap v4093: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:37:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:04 smithi100 ceph-mon[32599]: pgmap v4093: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:37:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:04 smithi167 ceph-mon[37140]: pgmap v4093: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:37:06.815 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:06 smithi100 ceph-mon[32599]: pgmap v4094: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:37:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:06 smithi138 ceph-mon[36954]: pgmap v4094: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:37:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:06 smithi167 ceph-mon[37140]: pgmap v4094: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:37:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:08 smithi138 ceph-mon[36954]: pgmap v4095: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:37:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:08 smithi138 ceph-mon[36954]: osdmap e304: 12 total, 12 up, 12 in 2024-07-26T15:37:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:08 smithi100 ceph-mon[32599]: pgmap v4095: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:37:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:08 smithi100 ceph-mon[32599]: osdmap e304: 12 total, 12 up, 12 in 2024-07-26T15:37:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:08 smithi167 ceph-mon[37140]: pgmap v4095: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:37:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:08 smithi167 ceph-mon[37140]: osdmap e304: 12 total, 12 up, 12 in 2024-07-26T15:37:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:10 smithi138 ceph-mon[36954]: pgmap v4097: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:37:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:10 smithi100 ceph-mon[32599]: pgmap v4097: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:37:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:10 smithi167 ceph-mon[37140]: pgmap v4097: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:37:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:12 smithi138 ceph-mon[36954]: pgmap v4098: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:37:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:12 smithi100 ceph-mon[32599]: pgmap v4098: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:37:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:12 smithi167 ceph-mon[37140]: pgmap v4098: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:37:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:14 smithi138 ceph-mon[36954]: pgmap v4099: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:37:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:14 smithi100 ceph-mon[32599]: pgmap v4099: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:37:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:14 smithi167 ceph-mon[37140]: pgmap v4099: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:37:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:16 smithi138 ceph-mon[36954]: pgmap v4100: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:37:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:16 smithi100 ceph-mon[32599]: pgmap v4100: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:37:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:16 smithi167 ceph-mon[37140]: pgmap v4100: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:37:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:18 smithi138 ceph-mon[36954]: pgmap v4101: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:37:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:18 smithi100 ceph-mon[32599]: pgmap v4101: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:37:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:18 smithi167 ceph-mon[37140]: pgmap v4101: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:37:19.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:37:19.216+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:37:19.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:37:19.216+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:37:19.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:37:19.216+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:37:19.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:37:19.216+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:37:19.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:37:19.216+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:37:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:20 smithi138 ceph-mon[36954]: pgmap v4102: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 183 B/s wr, 7 op/s 2024-07-26T15:37:20.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:20 smithi138 ceph-mon[36954]: osdmap e305: 12 total, 12 up, 12 in 2024-07-26T15:37:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:20 smithi100 ceph-mon[32599]: pgmap v4102: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 183 B/s wr, 7 op/s 2024-07-26T15:37:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:20 smithi100 ceph-mon[32599]: osdmap e305: 12 total, 12 up, 12 in 2024-07-26T15:37:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:20 smithi167 ceph-mon[37140]: pgmap v4102: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 183 B/s wr, 7 op/s 2024-07-26T15:37:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:20 smithi167 ceph-mon[37140]: osdmap e305: 12 total, 12 up, 12 in 2024-07-26T15:37:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:22 smithi138 ceph-mon[36954]: pgmap v4104: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:37:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:22 smithi100 ceph-mon[32599]: pgmap v4104: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:37:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:22 smithi167 ceph-mon[37140]: pgmap v4104: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:37:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:24 smithi138 ceph-mon[36954]: pgmap v4105: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:37:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:24 smithi100 ceph-mon[32599]: pgmap v4105: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:37:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:24 smithi167 ceph-mon[37140]: pgmap v4105: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:37:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:26 smithi138 ceph-mon[36954]: pgmap v4106: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:37:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:26 smithi100 ceph-mon[32599]: pgmap v4106: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:37:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:26 smithi167 ceph-mon[37140]: pgmap v4106: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:37:28.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:28 smithi138 ceph-mon[36954]: pgmap v4107: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:37:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:28 smithi100 ceph-mon[32599]: pgmap v4107: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:37:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:28 smithi167 ceph-mon[37140]: pgmap v4107: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:37:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:30 smithi138 ceph-mon[36954]: pgmap v4108: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:37:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:30 smithi100 ceph-mon[32599]: pgmap v4108: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:37:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:30 smithi167 ceph-mon[37140]: pgmap v4108: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:37:32.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:32 smithi138 ceph-mon[36954]: pgmap v4109: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T15:37:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:32 smithi100 ceph-mon[32599]: pgmap v4109: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T15:37:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:32 smithi167 ceph-mon[37140]: pgmap v4109: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T15:37:34.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:37:34.231+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:37:34.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:37:34.232+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:37:34.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:37:34.232+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:37:34.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:37:34.232+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:37:34.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:37:34.232+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:37:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:34 smithi138 ceph-mon[36954]: pgmap v4110: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:37:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:34 smithi100 ceph-mon[32599]: pgmap v4110: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:37:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:34 smithi167 ceph-mon[37140]: pgmap v4110: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:37:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:36 smithi138 ceph-mon[36954]: pgmap v4111: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:37:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:36 smithi100 ceph-mon[32599]: pgmap v4111: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:37:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:36 smithi167 ceph-mon[37140]: pgmap v4111: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:37:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:38 smithi138 ceph-mon[36954]: pgmap v4112: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:37:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:38 smithi100 ceph-mon[32599]: pgmap v4112: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:37:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:38 smithi167 ceph-mon[37140]: pgmap v4112: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:37:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:40 smithi138 ceph-mon[36954]: pgmap v4113: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:37:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:40 smithi100 ceph-mon[32599]: pgmap v4113: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:37:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:40 smithi167 ceph-mon[37140]: pgmap v4113: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:37:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:42 smithi138 ceph-mon[36954]: pgmap v4114: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:37:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:42 smithi100 ceph-mon[32599]: pgmap v4114: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:37:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:42 smithi167 ceph-mon[37140]: pgmap v4114: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:37:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:44 smithi138 ceph-mon[36954]: pgmap v4115: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:37:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:44 smithi100 ceph-mon[32599]: pgmap v4115: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:37:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:44 smithi167 ceph-mon[37140]: pgmap v4115: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:37:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:46 smithi138 ceph-mon[36954]: pgmap v4116: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:37:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:46 smithi100 ceph-mon[32599]: pgmap v4116: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:37:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:46 smithi167 ceph-mon[37140]: pgmap v4116: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:37:48.892 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:48 smithi138 ceph-mon[36954]: pgmap v4117: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:37:48.892 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:48 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:37:48.892 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:48 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:37:48.893 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:48 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:37:48.893 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:48 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:37:48.893 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:48 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:37:48.910 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:48 smithi167 ceph-mon[37140]: pgmap v4117: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:37:48.911 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:48 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:37:48.911 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:48 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:37:48.911 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:48 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:37:48.911 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:48 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:37:48.911 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:48 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:37:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:48 smithi100 ceph-mon[32599]: pgmap v4117: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:37:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:48 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:37:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:48 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:37:48.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:48 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:37:48.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:48 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:37:48.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:48 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:37:49.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:37:49.243+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:37:49.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:37:49.243+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:37:49.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:37:49.243+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:37:49.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:37:49.243+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:37:49.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:37:49.243+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:37:49.896 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:49 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T15:37:49.896 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:49 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T15:37:49.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:49 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T15:37:49.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:49 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T15:37:49.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:49 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T15:37:49.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:49 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T15:37:50.924 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:50 smithi167 ceph-mon[37140]: pgmap v4118: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:37:50.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:50 smithi138 ceph-mon[36954]: pgmap v4118: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:37:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:50 smithi100 ceph-mon[32599]: pgmap v4118: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:37:52.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:52 smithi138 ceph-mon[36954]: pgmap v4119: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:37:52.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:37:52.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:37:52.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:37:52.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:37:52.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:37:52.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:37:52.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:52 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T15:37:52.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:52 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T15:37:52.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:37:52.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:37:52.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:37:52.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:37:52.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:37:52.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:37:52.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:52 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T15:37:52.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:52 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T15:37:52.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:37:52.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:37:52.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:37:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:52 smithi100 ceph-mon[32599]: pgmap v4119: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:37:52.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:37:52.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:37:52.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:37:52.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:37:52.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:37:52.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:37:52.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:52 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T15:37:52.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:52 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T15:37:52.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:37:52.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:37:52.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:37:52.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:37:52.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:37:52.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:37:52.997 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:52 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T15:37:52.997 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:52 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T15:37:52.997 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:37:52.997 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:37:52.997 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:37:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:52 smithi167 ceph-mon[37140]: pgmap v4119: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:37:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:37:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:37:53.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:37:53.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:37:53.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:37:53.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:37:53.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:52 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T15:37:53.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:52 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T15:37:53.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:37:53.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:37:53.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:37:53.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:37:53.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:37:53.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:37:53.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:52 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T15:37:53.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:52 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T15:37:53.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:37:53.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:37:53.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:37:54.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:54 smithi138 ceph-mon[36954]: pgmap v4120: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:37:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:54 smithi100 ceph-mon[32599]: pgmap v4120: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:37:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:54 smithi167 ceph-mon[37140]: pgmap v4120: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:37:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:56 smithi138 ceph-mon[36954]: pgmap v4121: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:37:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:56 smithi100 ceph-mon[32599]: pgmap v4121: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:37:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:56 smithi167 ceph-mon[37140]: pgmap v4121: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:37:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:37:58 smithi138 ceph-mon[36954]: pgmap v4122: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:37:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:37:58 smithi100 ceph-mon[32599]: pgmap v4122: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:37:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:37:58 smithi167 ceph-mon[37140]: pgmap v4122: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:38:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:38:00 smithi138 ceph-mon[36954]: pgmap v4123: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:38:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:38:00 smithi100 ceph-mon[32599]: pgmap v4123: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:38:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:38:00 smithi167 ceph-mon[37140]: pgmap v4123: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:38:02.880 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:38:02 smithi138 ceph-mon[36954]: pgmap v4124: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:38:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:38:02 smithi100 ceph-mon[32599]: pgmap v4124: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:38:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:38:02 smithi167 ceph-mon[37140]: pgmap v4124: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:38:04.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:38:04.259+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:38:04.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:38:04.259+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:38:04.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:38:04.259+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:38:04.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:38:04.259+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:38:04.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:38:04.259+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:38:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:38:04 smithi138 ceph-mon[36954]: pgmap v4125: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:38:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:38:04 smithi100 ceph-mon[32599]: pgmap v4125: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:38:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:38:04 smithi167 ceph-mon[37140]: pgmap v4125: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:38:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:38:06 smithi138 ceph-mon[36954]: pgmap v4126: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:38:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:38:06 smithi100 ceph-mon[32599]: pgmap v4126: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:38:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:38:06 smithi167 ceph-mon[37140]: pgmap v4126: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:38:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:38:08 smithi138 ceph-mon[36954]: pgmap v4127: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:38:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:38:08 smithi138 ceph-mon[36954]: osdmap e306: 12 total, 12 up, 12 in 2024-07-26T15:38:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:38:08 smithi100 ceph-mon[32599]: pgmap v4127: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:38:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:38:08 smithi100 ceph-mon[32599]: osdmap e306: 12 total, 12 up, 12 in 2024-07-26T15:38:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:38:08 smithi167 ceph-mon[37140]: pgmap v4127: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:38:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:38:08 smithi167 ceph-mon[37140]: osdmap e306: 12 total, 12 up, 12 in 2024-07-26T15:38:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:38:10 smithi138 ceph-mon[36954]: pgmap v4129: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:38:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:38:10 smithi100 ceph-mon[32599]: pgmap v4129: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:38:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:38:10 smithi167 ceph-mon[37140]: pgmap v4129: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:38:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:38:12 smithi138 ceph-mon[36954]: pgmap v4130: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:38:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:38:12 smithi100 ceph-mon[32599]: pgmap v4130: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:38:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:38:12 smithi167 ceph-mon[37140]: pgmap v4130: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:38:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:38:14 smithi138 ceph-mon[36954]: pgmap v4131: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:38:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:38:14 smithi100 ceph-mon[32599]: pgmap v4131: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:38:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:38:14 smithi167 ceph-mon[37140]: pgmap v4131: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:38:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:38:16 smithi138 ceph-mon[36954]: pgmap v4132: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:38:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:38:16 smithi100 ceph-mon[32599]: pgmap v4132: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:38:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:38:16 smithi167 ceph-mon[37140]: pgmap v4132: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:38:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:38:18 smithi138 ceph-mon[36954]: pgmap v4133: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:38:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:38:18 smithi100 ceph-mon[32599]: pgmap v4133: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:38:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:38:18 smithi167 ceph-mon[37140]: pgmap v4133: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:38:19.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:38:19.266+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:38:19.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:38:19.267+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:38:19.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:38:19.267+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:38:19.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:38:19.267+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:38:19.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:38:19.267+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:38:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:38:20 smithi138 ceph-mon[36954]: pgmap v4134: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 183 B/s wr, 7 op/s 2024-07-26T15:38:20.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:38:20 smithi138 ceph-mon[36954]: osdmap e307: 12 total, 12 up, 12 in 2024-07-26T15:38:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:38:20 smithi100 ceph-mon[32599]: pgmap v4134: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 183 B/s wr, 7 op/s 2024-07-26T15:38:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:38:20 smithi100 ceph-mon[32599]: osdmap e307: 12 total, 12 up, 12 in 2024-07-26T15:38:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:38:20 smithi167 ceph-mon[37140]: pgmap v4134: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 183 B/s wr, 7 op/s 2024-07-26T15:38:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:38:20 smithi167 ceph-mon[37140]: osdmap e307: 12 total, 12 up, 12 in 2024-07-26T15:38:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:38:22 smithi138 ceph-mon[36954]: pgmap v4136: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:38:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:38:22 smithi100 ceph-mon[32599]: pgmap v4136: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:38:23.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:38:22 smithi167 ceph-mon[37140]: pgmap v4136: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:38:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:38:24 smithi138 ceph-mon[36954]: pgmap v4137: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:38:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:38:24 smithi100 ceph-mon[32599]: pgmap v4137: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:38:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:38:24 smithi167 ceph-mon[37140]: pgmap v4137: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:38:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:38:26 smithi138 ceph-mon[36954]: pgmap v4138: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:38:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:38:26 smithi100 ceph-mon[32599]: pgmap v4138: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:38:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:38:26 smithi167 ceph-mon[37140]: pgmap v4138: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:38:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:38:28 smithi138 ceph-mon[36954]: pgmap v4139: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:38:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:38:28 smithi100 ceph-mon[32599]: pgmap v4139: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:38:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:38:28 smithi167 ceph-mon[37140]: pgmap v4139: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:38:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:38:30 smithi138 ceph-mon[36954]: pgmap v4140: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:38:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:38:30 smithi100 ceph-mon[32599]: pgmap v4140: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:38:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:38:30 smithi167 ceph-mon[37140]: pgmap v4140: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:38:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:38:32 smithi138 ceph-mon[36954]: pgmap v4141: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T15:38:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:38:32 smithi100 ceph-mon[32599]: pgmap v4141: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T15:38:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:38:32 smithi167 ceph-mon[37140]: pgmap v4141: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T15:38:34.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:38:34.283+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:38:34.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:38:34.283+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:38:34.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:38:34.283+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:38:34.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:38:34.283+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:38:34.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:38:34.283+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:38:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:38:34 smithi138 ceph-mon[36954]: pgmap v4142: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:38:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:38:34 smithi100 ceph-mon[32599]: pgmap v4142: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:38:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:38:34 smithi167 ceph-mon[37140]: pgmap v4142: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:38:35.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:38:35 smithi100 ceph-mon[32599]: pgmap v4143: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:38:36.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:38:35 smithi167 ceph-mon[37140]: pgmap v4143: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:38:36.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:38:35 smithi138 ceph-mon[36954]: pgmap v4143: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:38:38.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:38:37 smithi138 ceph-mon[36954]: pgmap v4144: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:38:38.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:38:37 smithi100 ceph-mon[32599]: pgmap v4144: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:38:38.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:38:37 smithi167 ceph-mon[37140]: pgmap v4144: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:38:40.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:38:39 smithi138 ceph-mon[36954]: pgmap v4145: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:38:40.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:38:39 smithi100 ceph-mon[32599]: pgmap v4145: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:38:40.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:38:39 smithi167 ceph-mon[37140]: pgmap v4145: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:38:42.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:38:41 smithi138 ceph-mon[36954]: pgmap v4146: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:38:42.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:38:41 smithi100 ceph-mon[32599]: pgmap v4146: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:38:42.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:38:41 smithi167 ceph-mon[37140]: pgmap v4146: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:38:44.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:38:43 smithi138 ceph-mon[36954]: pgmap v4147: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:38:44.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:38:43 smithi100 ceph-mon[32599]: pgmap v4147: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:38:44.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:38:43 smithi167 ceph-mon[37140]: pgmap v4147: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:38:46.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:38:45 smithi138 ceph-mon[36954]: pgmap v4148: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:38:46.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:38:45 smithi100 ceph-mon[32599]: pgmap v4148: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:38:46.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:38:45 smithi167 ceph-mon[37140]: pgmap v4148: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:38:48.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:38:47 smithi138 ceph-mon[36954]: pgmap v4149: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:38:48.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:38:47 smithi100 ceph-mon[32599]: pgmap v4149: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:38:48.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:38:47 smithi167 ceph-mon[37140]: pgmap v4149: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:38:49.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:38:49.299+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:38:49.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:38:49.299+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:38:49.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:38:49.299+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:38:49.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:38:49.299+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:38:49.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:38:49.299+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:38:50.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:38:49 smithi138 ceph-mon[36954]: pgmap v4150: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:38:50.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:38:49 smithi100 ceph-mon[32599]: pgmap v4150: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:38:50.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:38:49 smithi167 ceph-mon[37140]: pgmap v4150: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:38:52.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:38:51 smithi138 ceph-mon[36954]: pgmap v4151: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:38:52.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:38:51 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:38:52.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:38:51 smithi100 ceph-mon[32599]: pgmap v4151: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:38:52.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:38:51 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:38:52.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:38:51 smithi167 ceph-mon[37140]: pgmap v4151: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:38:52.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:38:51 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:38:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:38:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:38:53.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:38:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:38:53.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:38:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:38:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:38:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:38:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:38:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:38:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:38:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:38:53.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:38:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:38:53.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:38:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:38:53.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:38:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:38:54.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:38:54 smithi100 ceph-mon[32599]: pgmap v4152: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:38:54.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:38:54 smithi167 ceph-mon[37140]: pgmap v4152: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:38:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:38:54 smithi138 ceph-mon[36954]: pgmap v4152: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:38:56.493 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:38:56 smithi138 ceph-mon[36954]: pgmap v4153: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:38:56.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:38:56 smithi100 ceph-mon[32599]: pgmap v4153: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:38:56.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:38:56 smithi167 ceph-mon[37140]: pgmap v4153: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:38:58.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:38:58 smithi100 ceph-mon[32599]: pgmap v4154: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:38:58.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:38:58 smithi167 ceph-mon[37140]: pgmap v4154: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:38:58.684 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:38:58 smithi138 ceph-mon[36954]: pgmap v4154: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:39:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:39:00 smithi100 ceph-mon[32599]: pgmap v4155: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:39:00.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:39:00 smithi167 ceph-mon[37140]: pgmap v4155: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:39:00.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:39:00 smithi138 ceph-mon[36954]: pgmap v4155: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:39:02.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:39:02 smithi100 ceph-mon[32599]: pgmap v4156: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:39:02.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:39:02 smithi167 ceph-mon[37140]: pgmap v4156: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:39:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:39:02 smithi138 ceph-mon[36954]: pgmap v4156: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:39:03.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:39:03.955+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c003690 con 0x7fe5a81087a0 2024-07-26T15:39:03.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:39:03.955+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T15:39:03.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:39:03.955+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 55 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c100 con 0x7fe5a81087a0 2024-07-26T15:39:03.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:39:03.955+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 55 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T15:39:04.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:39:04.315+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:39:04.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:39:04.315+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:39:04.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:39:04.315+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:39:04.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:39:04.315+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:39:04.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:39:04.315+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:39:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:39:04 smithi100 ceph-mon[32599]: pgmap v4157: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:39:04.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:39:04 smithi167 ceph-mon[37140]: pgmap v4157: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:39:04.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:39:04 smithi138 ceph-mon[36954]: pgmap v4157: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:39:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:39:06 smithi138 ceph-mon[36954]: pgmap v4158: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:39:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:39:06 smithi100 ceph-mon[32599]: pgmap v4158: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:39:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:39:06 smithi167 ceph-mon[37140]: pgmap v4158: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:39:08.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:39:08 smithi100 ceph-mon[32599]: pgmap v4159: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:39:08.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:39:08 smithi100 ceph-mon[32599]: osdmap e308: 12 total, 12 up, 12 in 2024-07-26T15:39:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:39:08 smithi138 ceph-mon[36954]: pgmap v4159: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:39:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:39:08 smithi138 ceph-mon[36954]: osdmap e308: 12 total, 12 up, 12 in 2024-07-26T15:39:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:39:08 smithi167 ceph-mon[37140]: pgmap v4159: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:39:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:39:08 smithi167 ceph-mon[37140]: osdmap e308: 12 total, 12 up, 12 in 2024-07-26T15:39:10.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:39:10 smithi138 ceph-mon[36954]: pgmap v4161: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:39:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:39:10 smithi100 ceph-mon[32599]: pgmap v4161: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:39:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:39:10 smithi167 ceph-mon[37140]: pgmap v4161: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:39:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:39:12 smithi138 ceph-mon[36954]: pgmap v4162: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:39:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:39:12 smithi100 ceph-mon[32599]: pgmap v4162: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:39:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:39:12 smithi167 ceph-mon[37140]: pgmap v4162: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:39:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:39:14 smithi138 ceph-mon[36954]: pgmap v4163: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:39:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:39:14 smithi100 ceph-mon[32599]: pgmap v4163: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:39:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:39:14 smithi167 ceph-mon[37140]: pgmap v4163: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:39:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:39:16 smithi138 ceph-mon[36954]: pgmap v4164: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:39:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:39:16 smithi138 ceph-mon[36954]: osdmap e309: 12 total, 12 up, 12 in 2024-07-26T15:39:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:39:16 smithi100 ceph-mon[32599]: pgmap v4164: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:39:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:39:16 smithi100 ceph-mon[32599]: osdmap e309: 12 total, 12 up, 12 in 2024-07-26T15:39:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:39:16 smithi167 ceph-mon[37140]: pgmap v4164: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:39:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:39:16 smithi167 ceph-mon[37140]: osdmap e309: 12 total, 12 up, 12 in 2024-07-26T15:39:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:39:18 smithi138 ceph-mon[36954]: pgmap v4166: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 221 B/s wr, 5 op/s 2024-07-26T15:39:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:39:18 smithi100 ceph-mon[32599]: pgmap v4166: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 221 B/s wr, 5 op/s 2024-07-26T15:39:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:39:18 smithi167 ceph-mon[37140]: pgmap v4166: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 221 B/s wr, 5 op/s 2024-07-26T15:39:19.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:39:19.323+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:39:19.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:39:19.324+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:39:19.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:39:19.324+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:39:19.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:39:19.324+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:39:19.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:39:19.324+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:39:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:39:20 smithi138 ceph-mon[36954]: pgmap v4167: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:39:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:39:20 smithi100 ceph-mon[32599]: pgmap v4167: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:39:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:39:20 smithi167 ceph-mon[37140]: pgmap v4167: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:39:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:39:22 smithi138 ceph-mon[36954]: pgmap v4168: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:39:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:39:22 smithi100 ceph-mon[32599]: pgmap v4168: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:39:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:39:22 smithi167 ceph-mon[37140]: pgmap v4168: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:39:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:39:24 smithi138 ceph-mon[36954]: pgmap v4169: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:39:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:39:24 smithi100 ceph-mon[32599]: pgmap v4169: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:39:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:39:24 smithi167 ceph-mon[37140]: pgmap v4169: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:39:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:39:26 smithi138 ceph-mon[36954]: pgmap v4170: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:39:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:39:26 smithi100 ceph-mon[32599]: pgmap v4170: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:39:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:39:26 smithi167 ceph-mon[37140]: pgmap v4170: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:39:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:39:28 smithi138 ceph-mon[36954]: pgmap v4171: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:39:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:39:28 smithi100 ceph-mon[32599]: pgmap v4171: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:39:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:39:28 smithi167 ceph-mon[37140]: pgmap v4171: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:39:30.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:39:30 smithi138 ceph-mon[36954]: pgmap v4172: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:39:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:39:30 smithi100 ceph-mon[32599]: pgmap v4172: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:39:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:39:30 smithi167 ceph-mon[37140]: pgmap v4172: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:39:32.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:39:32 smithi167 ceph-mon[37140]: pgmap v4173: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:39:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:39:32 smithi138 ceph-mon[36954]: pgmap v4173: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:39:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:39:32 smithi100 ceph-mon[32599]: pgmap v4173: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:39:34.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:39:34.332+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:39:34.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:39:34.332+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:39:34.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:39:34.332+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:39:34.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:39:34.332+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:39:34.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:39:34.332+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:39:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:39:34 smithi100 ceph-mon[32599]: pgmap v4174: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:39:34.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:39:34 smithi138 ceph-mon[36954]: pgmap v4174: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:39:35.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:39:34 smithi167 ceph-mon[37140]: pgmap v4174: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:39:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:39:36 smithi138 ceph-mon[36954]: pgmap v4175: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:39:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:39:36 smithi100 ceph-mon[32599]: pgmap v4175: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:39:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:39:36 smithi167 ceph-mon[37140]: pgmap v4175: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:39:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:39:38 smithi138 ceph-mon[36954]: pgmap v4176: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:39:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:39:38 smithi100 ceph-mon[32599]: pgmap v4176: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:39:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:39:38 smithi167 ceph-mon[37140]: pgmap v4176: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:39:40.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:39:40 smithi138 ceph-mon[36954]: pgmap v4177: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:39:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:39:40 smithi100 ceph-mon[32599]: pgmap v4177: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:39:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:39:40 smithi167 ceph-mon[37140]: pgmap v4177: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:39:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:39:42 smithi138 ceph-mon[36954]: pgmap v4178: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:39:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:39:42 smithi100 ceph-mon[32599]: pgmap v4178: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:39:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:39:42 smithi167 ceph-mon[37140]: pgmap v4178: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:39:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:39:44 smithi138 ceph-mon[36954]: pgmap v4179: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:39:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:39:44 smithi100 ceph-mon[32599]: pgmap v4179: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:39:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:39:44 smithi167 ceph-mon[37140]: pgmap v4179: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:39:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:39:46 smithi138 ceph-mon[36954]: pgmap v4180: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:39:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:39:46 smithi100 ceph-mon[32599]: pgmap v4180: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:39:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:39:46 smithi167 ceph-mon[37140]: pgmap v4180: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:39:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:39:48 smithi138 ceph-mon[36954]: pgmap v4181: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:39:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:39:48 smithi100 ceph-mon[32599]: pgmap v4181: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:39:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:39:48 smithi167 ceph-mon[37140]: pgmap v4181: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:39:49.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:39:49.348+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:39:49.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:39:49.348+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:39:49.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:39:49.348+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:39:49.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:39:49.348+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:39:49.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:39:49.348+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:39:50.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:39:50 smithi138 ceph-mon[36954]: pgmap v4182: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:39:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:39:50 smithi100 ceph-mon[32599]: pgmap v4182: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:39:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:39:50 smithi167 ceph-mon[37140]: pgmap v4182: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:39:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:39:52 smithi138 ceph-mon[36954]: pgmap v4183: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:39:52.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:39:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:39:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:39:52 smithi100 ceph-mon[32599]: pgmap v4183: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:39:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:39:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:39:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:39:52 smithi167 ceph-mon[37140]: pgmap v4183: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:39:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:39:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:39:53.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:39:53 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:39:53.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:39:53 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:39:53.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:39:53 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:39:53.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:39:53 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:39:53.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:39:53 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:39:53.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:39:53 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:39:54.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:39:53 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:39:54.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:39:53 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:39:54.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:39:53 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:39:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:39:54 smithi138 ceph-mon[36954]: pgmap v4184: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:39:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:39:54 smithi100 ceph-mon[32599]: pgmap v4184: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:39:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:39:54 smithi167 ceph-mon[37140]: pgmap v4184: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:39:56.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:39:56 smithi138 ceph-mon[36954]: pgmap v4185: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:39:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:39:56 smithi100 ceph-mon[32599]: pgmap v4185: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:39:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:39:56 smithi167 ceph-mon[37140]: pgmap v4185: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:39:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:39:58 smithi138 ceph-mon[36954]: pgmap v4186: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:39:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:39:58 smithi100 ceph-mon[32599]: pgmap v4186: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:39:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:39:58 smithi167 ceph-mon[37140]: pgmap v4186: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:40:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:00 smithi138 ceph-mon[36954]: pgmap v4187: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:40:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:00 smithi138 ceph-mon[36954]: overall HEALTH_OK 2024-07-26T15:40:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:00 smithi100 ceph-mon[32599]: pgmap v4187: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:40:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:00 smithi100 ceph-mon[32599]: overall HEALTH_OK 2024-07-26T15:40:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:00 smithi167 ceph-mon[37140]: pgmap v4187: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:40:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:00 smithi167 ceph-mon[37140]: overall HEALTH_OK 2024-07-26T15:40:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:02 smithi138 ceph-mon[36954]: pgmap v4188: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:40:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:02 smithi100 ceph-mon[32599]: pgmap v4188: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:40:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:02 smithi167 ceph-mon[37140]: pgmap v4188: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:40:04.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:40:04.364+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:40:04.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:40:04.364+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:40:04.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:40:04.364+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:40:04.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:40:04.364+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:40:04.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:40:04.364+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:40:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:04 smithi100 ceph-mon[32599]: pgmap v4189: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:40:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:04 smithi138 ceph-mon[36954]: pgmap v4189: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:40:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:04 smithi167 ceph-mon[37140]: pgmap v4189: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:40:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:06 smithi138 ceph-mon[36954]: pgmap v4190: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:40:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:06 smithi100 ceph-mon[32599]: pgmap v4190: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:40:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:06 smithi167 ceph-mon[37140]: pgmap v4190: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:40:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:08 smithi138 ceph-mon[36954]: pgmap v4191: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:40:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:08 smithi138 ceph-mon[36954]: osdmap e310: 12 total, 12 up, 12 in 2024-07-26T15:40:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:08 smithi100 ceph-mon[32599]: pgmap v4191: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:40:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:08 smithi100 ceph-mon[32599]: osdmap e310: 12 total, 12 up, 12 in 2024-07-26T15:40:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:08 smithi167 ceph-mon[37140]: pgmap v4191: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:40:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:08 smithi167 ceph-mon[37140]: osdmap e310: 12 total, 12 up, 12 in 2024-07-26T15:40:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:10 smithi138 ceph-mon[36954]: pgmap v4193: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:40:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:10 smithi100 ceph-mon[32599]: pgmap v4193: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:40:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:10 smithi167 ceph-mon[37140]: pgmap v4193: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:40:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:12 smithi138 ceph-mon[36954]: pgmap v4194: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:40:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:12 smithi100 ceph-mon[32599]: pgmap v4194: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:40:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:12 smithi167 ceph-mon[37140]: pgmap v4194: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:40:14.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:14 smithi138 ceph-mon[36954]: pgmap v4195: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:40:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:14 smithi100 ceph-mon[32599]: pgmap v4195: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:40:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:14 smithi167 ceph-mon[37140]: pgmap v4195: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:40:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:16 smithi138 ceph-mon[36954]: pgmap v4196: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:40:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:16 smithi100 ceph-mon[32599]: pgmap v4196: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:40:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:16 smithi167 ceph-mon[37140]: pgmap v4196: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:40:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:18 smithi138 ceph-mon[36954]: pgmap v4197: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:40:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:18 smithi100 ceph-mon[32599]: pgmap v4197: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:40:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:18 smithi167 ceph-mon[37140]: pgmap v4197: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:40:19.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:40:19.376+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:40:19.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:40:19.377+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:40:19.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:40:19.377+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:40:19.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:40:19.377+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:40:19.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:40:19.377+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:40:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:20 smithi138 ceph-mon[36954]: pgmap v4198: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 183 B/s wr, 7 op/s 2024-07-26T15:40:20.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:20 smithi138 ceph-mon[36954]: osdmap e311: 12 total, 12 up, 12 in 2024-07-26T15:40:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:20 smithi100 ceph-mon[32599]: pgmap v4198: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 183 B/s wr, 7 op/s 2024-07-26T15:40:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:20 smithi100 ceph-mon[32599]: osdmap e311: 12 total, 12 up, 12 in 2024-07-26T15:40:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:20 smithi167 ceph-mon[37140]: pgmap v4198: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 183 B/s wr, 7 op/s 2024-07-26T15:40:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:20 smithi167 ceph-mon[37140]: osdmap e311: 12 total, 12 up, 12 in 2024-07-26T15:40:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:22 smithi138 ceph-mon[36954]: pgmap v4200: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:40:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:22 smithi100 ceph-mon[32599]: pgmap v4200: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:40:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:22 smithi167 ceph-mon[37140]: pgmap v4200: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:40:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:24 smithi138 ceph-mon[36954]: pgmap v4201: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:40:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:24 smithi100 ceph-mon[32599]: pgmap v4201: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:40:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:24 smithi167 ceph-mon[37140]: pgmap v4201: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:40:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:26 smithi138 ceph-mon[36954]: pgmap v4202: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:40:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:26 smithi100 ceph-mon[32599]: pgmap v4202: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:40:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:26 smithi167 ceph-mon[37140]: pgmap v4202: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:40:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:28 smithi138 ceph-mon[36954]: pgmap v4203: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:40:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:28 smithi100 ceph-mon[32599]: pgmap v4203: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:40:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:28 smithi167 ceph-mon[37140]: pgmap v4203: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:40:30.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:30 smithi138 ceph-mon[36954]: pgmap v4204: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:40:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:30 smithi100 ceph-mon[32599]: pgmap v4204: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:40:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:30 smithi167 ceph-mon[37140]: pgmap v4204: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:40:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:32 smithi138 ceph-mon[36954]: pgmap v4205: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T15:40:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:32 smithi100 ceph-mon[32599]: pgmap v4205: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T15:40:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:32 smithi167 ceph-mon[37140]: pgmap v4205: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T15:40:34.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:40:34.393+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:40:34.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:40:34.393+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:40:34.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:40:34.393+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:40:34.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:40:34.393+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:40:34.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:40:34.393+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:40:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:34 smithi138 ceph-mon[36954]: pgmap v4206: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:40:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:34 smithi100 ceph-mon[32599]: pgmap v4206: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:40:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:34 smithi167 ceph-mon[37140]: pgmap v4206: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:40:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:36 smithi138 ceph-mon[36954]: pgmap v4207: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:40:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:36 smithi100 ceph-mon[32599]: pgmap v4207: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:40:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:36 smithi167 ceph-mon[37140]: pgmap v4207: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:40:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:38 smithi138 ceph-mon[36954]: pgmap v4208: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:40:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:38 smithi100 ceph-mon[32599]: pgmap v4208: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:40:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:38 smithi167 ceph-mon[37140]: pgmap v4208: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:40:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:40 smithi138 ceph-mon[36954]: pgmap v4209: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:40:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:40 smithi100 ceph-mon[32599]: pgmap v4209: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:40:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:40 smithi167 ceph-mon[37140]: pgmap v4209: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:40:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:42 smithi138 ceph-mon[36954]: pgmap v4210: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:40:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:42 smithi100 ceph-mon[32599]: pgmap v4210: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:40:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:42 smithi167 ceph-mon[37140]: pgmap v4210: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:40:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:44 smithi138 ceph-mon[36954]: pgmap v4211: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:40:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:44 smithi100 ceph-mon[32599]: pgmap v4211: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:40:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:44 smithi167 ceph-mon[37140]: pgmap v4211: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:40:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:46 smithi138 ceph-mon[36954]: pgmap v4212: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:40:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:46 smithi100 ceph-mon[32599]: pgmap v4212: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:40:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:46 smithi167 ceph-mon[37140]: pgmap v4212: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:40:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:48 smithi138 ceph-mon[36954]: pgmap v4213: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:40:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:48 smithi100 ceph-mon[32599]: pgmap v4213: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:40:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:48 smithi167 ceph-mon[37140]: pgmap v4213: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:40:49.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:40:49.399+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:40:49.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:40:49.399+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:40:49.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:40:49.400+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:40:49.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:40:49.400+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:40:49.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:40:49.400+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:40:49.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:49 smithi100 ceph-mon[32599]: pgmap v4214: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:40:50.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:49 smithi167 ceph-mon[37140]: pgmap v4214: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:40:50.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:49 smithi138 ceph-mon[36954]: pgmap v4214: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:40:52.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:51 smithi138 ceph-mon[36954]: pgmap v4215: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:40:52.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:51 smithi100 ceph-mon[32599]: pgmap v4215: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:40:52.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:51 smithi167 ceph-mon[37140]: pgmap v4215: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:40:54.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:53 smithi138 ceph-mon[36954]: pgmap v4216: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:40:54.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:53 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:40:54.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:53 smithi100 ceph-mon[32599]: pgmap v4216: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:40:54.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:53 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:40:54.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:53 smithi167 ceph-mon[37140]: pgmap v4216: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:40:54.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:53 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:40:55.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:40:55.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:40:55.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:40:55.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:40:55.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:40:55.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:40:55.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:40:55.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:40:55.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:40:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:40:55.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:40:55.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:40:55.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:40:55.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:40:55.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:40:55.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:40:55.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:40:55.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:40:55.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:40:55.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:40:55.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:40:55.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:40:55.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:40:55.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:40:55.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:40:55.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:40:55.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:40:56.591 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:56 smithi138 ceph-mon[36954]: pgmap v4217: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:40:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:56 smithi100 ceph-mon[32599]: pgmap v4217: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:40:56.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:56 smithi167 ceph-mon[37140]: pgmap v4217: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:40:58.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:40:58 smithi138 ceph-mon[36954]: pgmap v4218: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:40:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:40:58 smithi100 ceph-mon[32599]: pgmap v4218: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:40:58.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:40:58 smithi167 ceph-mon[37140]: pgmap v4218: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:41:00.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:00 smithi138 ceph-mon[36954]: pgmap v4219: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:41:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:00 smithi100 ceph-mon[32599]: pgmap v4219: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:41:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:00 smithi167 ceph-mon[37140]: pgmap v4219: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:41:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:02 smithi138 ceph-mon[36954]: pgmap v4220: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:41:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:02 smithi100 ceph-mon[32599]: pgmap v4220: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:41:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:02 smithi167 ceph-mon[37140]: pgmap v4220: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:41:04.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:41:04.410+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:41:04.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:41:04.411+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:41:04.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:41:04.411+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:41:04.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:41:04.411+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:41:04.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:41:04.411+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:41:04.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:04 smithi138 ceph-mon[36954]: pgmap v4221: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:41:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:04 smithi100 ceph-mon[32599]: pgmap v4221: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:41:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:04 smithi167 ceph-mon[37140]: pgmap v4221: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:41:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:06 smithi138 ceph-mon[36954]: pgmap v4222: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:41:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:06 smithi100 ceph-mon[32599]: pgmap v4222: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:41:06.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:06 smithi167 ceph-mon[37140]: pgmap v4222: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:41:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:08 smithi138 ceph-mon[36954]: pgmap v4223: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:41:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:08 smithi138 ceph-mon[36954]: osdmap e312: 12 total, 12 up, 12 in 2024-07-26T15:41:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:08 smithi100 ceph-mon[32599]: pgmap v4223: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:41:08.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:08 smithi100 ceph-mon[32599]: osdmap e312: 12 total, 12 up, 12 in 2024-07-26T15:41:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:08 smithi167 ceph-mon[37140]: pgmap v4223: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:41:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:08 smithi167 ceph-mon[37140]: osdmap e312: 12 total, 12 up, 12 in 2024-07-26T15:41:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:10 smithi138 ceph-mon[36954]: pgmap v4225: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:41:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:10 smithi100 ceph-mon[32599]: pgmap v4225: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:41:10.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:10 smithi167 ceph-mon[37140]: pgmap v4225: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:41:12.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:12 smithi138 ceph-mon[36954]: pgmap v4226: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:41:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:12 smithi100 ceph-mon[32599]: pgmap v4226: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:41:12.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:12 smithi167 ceph-mon[37140]: pgmap v4226: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:41:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:14 smithi138 ceph-mon[36954]: pgmap v4227: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:41:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:14 smithi100 ceph-mon[32599]: pgmap v4227: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:41:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:14 smithi167 ceph-mon[37140]: pgmap v4227: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:41:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:16 smithi138 ceph-mon[36954]: pgmap v4228: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:41:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:16 smithi138 ceph-mon[36954]: osdmap e313: 12 total, 12 up, 12 in 2024-07-26T15:41:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:16 smithi100 ceph-mon[32599]: pgmap v4228: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:41:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:16 smithi100 ceph-mon[32599]: osdmap e313: 12 total, 12 up, 12 in 2024-07-26T15:41:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:16 smithi167 ceph-mon[37140]: pgmap v4228: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:41:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:16 smithi167 ceph-mon[37140]: osdmap e313: 12 total, 12 up, 12 in 2024-07-26T15:41:18.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:18 smithi138 ceph-mon[36954]: pgmap v4230: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 216 B/s wr, 6 op/s 2024-07-26T15:41:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:18 smithi100 ceph-mon[32599]: pgmap v4230: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 216 B/s wr, 6 op/s 2024-07-26T15:41:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:18 smithi167 ceph-mon[37140]: pgmap v4230: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 216 B/s wr, 6 op/s 2024-07-26T15:41:19.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:41:19.418+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:41:19.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:41:19.419+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:41:19.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:41:19.419+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:41:19.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:41:19.419+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:41:19.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:41:19.419+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:41:20.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:20 smithi138 ceph-mon[36954]: pgmap v4231: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:41:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:20 smithi100 ceph-mon[32599]: pgmap v4231: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:41:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:20 smithi167 ceph-mon[37140]: pgmap v4231: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:41:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:22 smithi138 ceph-mon[36954]: pgmap v4232: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:41:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:22 smithi100 ceph-mon[32599]: pgmap v4232: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:41:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:22 smithi167 ceph-mon[37140]: pgmap v4232: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:41:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:24 smithi138 ceph-mon[36954]: pgmap v4233: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:41:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:24 smithi100 ceph-mon[32599]: pgmap v4233: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:41:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:24 smithi167 ceph-mon[37140]: pgmap v4233: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:41:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:26 smithi138 ceph-mon[36954]: pgmap v4234: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:41:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:26 smithi100 ceph-mon[32599]: pgmap v4234: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:41:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:26 smithi167 ceph-mon[37140]: pgmap v4234: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:41:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:28 smithi138 ceph-mon[36954]: pgmap v4235: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:41:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:28 smithi100 ceph-mon[32599]: pgmap v4235: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:41:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:28 smithi167 ceph-mon[37140]: pgmap v4235: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:41:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:30 smithi138 ceph-mon[36954]: pgmap v4236: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:41:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:30 smithi100 ceph-mon[32599]: pgmap v4236: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:41:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:30 smithi167 ceph-mon[37140]: pgmap v4236: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:41:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:32 smithi138 ceph-mon[36954]: pgmap v4237: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:41:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:32 smithi100 ceph-mon[32599]: pgmap v4237: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:41:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:32 smithi167 ceph-mon[37140]: pgmap v4237: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:41:34.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:41:34.434+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:41:34.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:41:34.435+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:41:34.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:41:34.435+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:41:34.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:41:34.435+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:41:34.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:41:34.435+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:41:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:34 smithi100 ceph-mon[32599]: pgmap v4238: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:41:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:34 smithi138 ceph-mon[36954]: pgmap v4238: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:41:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:34 smithi167 ceph-mon[37140]: pgmap v4238: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:41:36.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:36 smithi138 ceph-mon[36954]: pgmap v4239: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:41:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:36 smithi100 ceph-mon[32599]: pgmap v4239: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:41:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:36 smithi167 ceph-mon[37140]: pgmap v4239: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:41:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:38 smithi138 ceph-mon[36954]: pgmap v4240: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:41:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:38 smithi100 ceph-mon[32599]: pgmap v4240: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:41:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:38 smithi167 ceph-mon[37140]: pgmap v4240: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:41:40.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:40 smithi138 ceph-mon[36954]: pgmap v4241: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T15:41:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:40 smithi100 ceph-mon[32599]: pgmap v4241: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T15:41:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:40 smithi167 ceph-mon[37140]: pgmap v4241: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T15:41:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:42 smithi138 ceph-mon[36954]: pgmap v4242: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:41:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:42 smithi100 ceph-mon[32599]: pgmap v4242: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:41:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:42 smithi167 ceph-mon[37140]: pgmap v4242: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:41:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:44 smithi138 ceph-mon[36954]: pgmap v4243: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:41:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:44 smithi100 ceph-mon[32599]: pgmap v4243: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:41:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:44 smithi167 ceph-mon[37140]: pgmap v4243: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:41:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:46 smithi138 ceph-mon[36954]: pgmap v4244: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:41:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:46 smithi100 ceph-mon[32599]: pgmap v4244: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:41:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:46 smithi167 ceph-mon[37140]: pgmap v4244: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:41:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:48 smithi138 ceph-mon[36954]: pgmap v4245: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:41:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:48 smithi100 ceph-mon[32599]: pgmap v4245: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:41:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:48 smithi167 ceph-mon[37140]: pgmap v4245: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:41:49.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:41:49.450+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:41:49.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:41:49.451+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:41:49.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:41:49.451+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:41:49.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:41:49.451+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:41:49.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:41:49.451+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:41:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:50 smithi138 ceph-mon[36954]: pgmap v4246: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:41:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:50 smithi100 ceph-mon[32599]: pgmap v4246: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:41:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:50 smithi167 ceph-mon[37140]: pgmap v4246: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:41:52.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:52 smithi138 ceph-mon[36954]: pgmap v4247: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:41:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:52 smithi100 ceph-mon[32599]: pgmap v4247: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:41:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:52 smithi167 ceph-mon[37140]: pgmap v4247: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:41:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:54 smithi138 ceph-mon[36954]: pgmap v4248: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:41:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:54 smithi100 ceph-mon[32599]: pgmap v4248: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:41:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:54 smithi167 ceph-mon[37140]: pgmap v4248: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:41:55.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:41:55.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:41:55.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:41:55.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:41:55.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:41:55.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:41:55.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:41:55.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:41:55.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:41:55.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:41:55.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:41:55.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:41:55.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:41:55.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:41:55.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:41:55.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:41:55.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:41:55.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:41:55.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:41:55.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:41:56.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:41:56.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:41:56.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:41:56.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:41:56.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:41:56.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:41:56.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:41:56.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:41:56.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:41:56.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:41:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:56 smithi138 ceph-mon[36954]: pgmap v4249: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:41:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:56 smithi100 ceph-mon[32599]: pgmap v4249: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:41:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:56 smithi167 ceph-mon[37140]: pgmap v4249: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:41:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:41:58 smithi138 ceph-mon[36954]: pgmap v4250: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:41:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:41:58 smithi100 ceph-mon[32599]: pgmap v4250: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:41:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:41:58 smithi167 ceph-mon[37140]: pgmap v4250: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:42:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:42:00 smithi138 ceph-mon[36954]: pgmap v4251: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:42:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:42:00 smithi100 ceph-mon[32599]: pgmap v4251: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:42:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:42:00 smithi167 ceph-mon[37140]: pgmap v4251: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:42:02.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:42:02 smithi138 ceph-mon[36954]: pgmap v4252: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:42:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:42:02 smithi100 ceph-mon[32599]: pgmap v4252: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:42:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:42:02 smithi167 ceph-mon[37140]: pgmap v4252: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:42:03.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:42:03.980+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T15:42:03.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:42:03.980+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c003690 con 0x7fe5a81087a0 2024-07-26T15:42:03.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:42:03.980+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 56 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c100 con 0x7fe5a81087a0 2024-07-26T15:42:03.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:42:03.980+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 56 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T15:42:04.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:42:04.467+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:42:04.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:42:04.467+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:42:04.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:42:04.467+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:42:04.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:42:04.467+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:42:04.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:42:04.467+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:42:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:42:04 smithi100 ceph-mon[32599]: pgmap v4253: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:42:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:42:04 smithi138 ceph-mon[36954]: pgmap v4253: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:42:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:42:04 smithi167 ceph-mon[37140]: pgmap v4253: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:42:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:42:06 smithi138 ceph-mon[36954]: pgmap v4254: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:42:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:42:06 smithi100 ceph-mon[32599]: pgmap v4254: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:42:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:42:06 smithi167 ceph-mon[37140]: pgmap v4254: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:42:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:42:08 smithi138 ceph-mon[36954]: pgmap v4255: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:42:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:42:08 smithi138 ceph-mon[36954]: osdmap e314: 12 total, 12 up, 12 in 2024-07-26T15:42:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:42:08 smithi100 ceph-mon[32599]: pgmap v4255: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:42:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:42:08 smithi100 ceph-mon[32599]: osdmap e314: 12 total, 12 up, 12 in 2024-07-26T15:42:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:42:08 smithi167 ceph-mon[37140]: pgmap v4255: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:42:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:42:08 smithi167 ceph-mon[37140]: osdmap e314: 12 total, 12 up, 12 in 2024-07-26T15:42:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:42:10 smithi138 ceph-mon[36954]: pgmap v4257: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:42:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:42:10 smithi100 ceph-mon[32599]: pgmap v4257: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:42:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:42:10 smithi167 ceph-mon[37140]: pgmap v4257: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:42:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:42:12 smithi138 ceph-mon[36954]: pgmap v4258: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:42:12.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:42:12 smithi100 ceph-mon[32599]: pgmap v4258: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:42:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:42:12 smithi167 ceph-mon[37140]: pgmap v4258: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:42:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:42:14 smithi138 ceph-mon[36954]: pgmap v4259: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:42:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:42:14 smithi100 ceph-mon[32599]: pgmap v4259: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:42:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:42:14 smithi167 ceph-mon[37140]: pgmap v4259: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:42:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:42:16 smithi138 ceph-mon[36954]: pgmap v4260: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:42:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:42:16 smithi138 ceph-mon[36954]: osdmap e315: 12 total, 12 up, 12 in 2024-07-26T15:42:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:42:16 smithi100 ceph-mon[32599]: pgmap v4260: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:42:16.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:42:16 smithi100 ceph-mon[32599]: osdmap e315: 12 total, 12 up, 12 in 2024-07-26T15:42:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:42:16 smithi167 ceph-mon[37140]: pgmap v4260: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:42:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:42:16 smithi167 ceph-mon[37140]: osdmap e315: 12 total, 12 up, 12 in 2024-07-26T15:42:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:42:18 smithi138 ceph-mon[36954]: pgmap v4262: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 223 B/s wr, 5 op/s 2024-07-26T15:42:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:42:18 smithi100 ceph-mon[32599]: pgmap v4262: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 223 B/s wr, 5 op/s 2024-07-26T15:42:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:42:18 smithi167 ceph-mon[37140]: pgmap v4262: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 223 B/s wr, 5 op/s 2024-07-26T15:42:19.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:42:19.483+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:42:19.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:42:19.483+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:42:19.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:42:19.483+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:42:19.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:42:19.483+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:42:19.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:42:19.484+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:42:20.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:42:20 smithi138 ceph-mon[36954]: pgmap v4263: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:42:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:42:20 smithi100 ceph-mon[32599]: pgmap v4263: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:42:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:42:20 smithi167 ceph-mon[37140]: pgmap v4263: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:42:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:42:22 smithi138 ceph-mon[36954]: pgmap v4264: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:42:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:42:22 smithi100 ceph-mon[32599]: pgmap v4264: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:42:23.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:42:22 smithi167 ceph-mon[37140]: pgmap v4264: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:42:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:42:24 smithi138 ceph-mon[36954]: pgmap v4265: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:42:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:42:24 smithi100 ceph-mon[32599]: pgmap v4265: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:42:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:42:24 smithi167 ceph-mon[37140]: pgmap v4265: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:42:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:42:26 smithi138 ceph-mon[36954]: pgmap v4266: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:42:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:42:26 smithi100 ceph-mon[32599]: pgmap v4266: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:42:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:42:26 smithi167 ceph-mon[37140]: pgmap v4266: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:42:28.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:42:28 smithi138 ceph-mon[36954]: pgmap v4267: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:42:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:42:28 smithi100 ceph-mon[32599]: pgmap v4267: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:42:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:42:28 smithi167 ceph-mon[37140]: pgmap v4267: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:42:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:42:30 smithi138 ceph-mon[36954]: pgmap v4268: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:42:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:42:30 smithi100 ceph-mon[32599]: pgmap v4268: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:42:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:42:30 smithi167 ceph-mon[37140]: pgmap v4268: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:42:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:42:32 smithi138 ceph-mon[36954]: pgmap v4269: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:42:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:42:32 smithi100 ceph-mon[32599]: pgmap v4269: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:42:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:42:32 smithi167 ceph-mon[37140]: pgmap v4269: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:42:34.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:42:34.491+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:42:34.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:42:34.492+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:42:34.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:42:34.492+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:42:34.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:42:34.492+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:42:34.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:42:34.492+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:42:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:42:34 smithi100 ceph-mon[32599]: pgmap v4270: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:42:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:42:34 smithi138 ceph-mon[36954]: pgmap v4270: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:42:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:42:34 smithi167 ceph-mon[37140]: pgmap v4270: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:42:36.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:42:36 smithi138 ceph-mon[36954]: pgmap v4271: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:42:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:42:36 smithi100 ceph-mon[32599]: pgmap v4271: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:42:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:42:36 smithi167 ceph-mon[37140]: pgmap v4271: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:42:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:42:38 smithi138 ceph-mon[36954]: pgmap v4272: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:42:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:42:38 smithi100 ceph-mon[32599]: pgmap v4272: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:42:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:42:38 smithi167 ceph-mon[37140]: pgmap v4272: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:42:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:42:40 smithi138 ceph-mon[36954]: pgmap v4273: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:42:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:42:40 smithi100 ceph-mon[32599]: pgmap v4273: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:42:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:42:40 smithi167 ceph-mon[37140]: pgmap v4273: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:42:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:42:42 smithi138 ceph-mon[36954]: pgmap v4274: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:42:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:42:42 smithi100 ceph-mon[32599]: pgmap v4274: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:42:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:42:42 smithi167 ceph-mon[37140]: pgmap v4274: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:42:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:42:44 smithi138 ceph-mon[36954]: pgmap v4275: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:42:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:42:44 smithi100 ceph-mon[32599]: pgmap v4275: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:42:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:42:44 smithi167 ceph-mon[37140]: pgmap v4275: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:42:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:42:46 smithi138 ceph-mon[36954]: pgmap v4276: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:42:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:42:46 smithi100 ceph-mon[32599]: pgmap v4276: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:42:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:42:46 smithi167 ceph-mon[37140]: pgmap v4276: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:42:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:42:48 smithi138 ceph-mon[36954]: pgmap v4277: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:42:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:42:48 smithi100 ceph-mon[32599]: pgmap v4277: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:42:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:42:48 smithi167 ceph-mon[37140]: pgmap v4277: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:42:49.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:42:49.508+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:42:49.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:42:49.508+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:42:49.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:42:49.508+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:42:49.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:42:49.508+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:42:49.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:42:49.508+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:42:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:42:50 smithi138 ceph-mon[36954]: pgmap v4278: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:42:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:42:50 smithi100 ceph-mon[32599]: pgmap v4278: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:42:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:42:50 smithi167 ceph-mon[37140]: pgmap v4278: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:42:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:42:52 smithi138 ceph-mon[36954]: pgmap v4279: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:42:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:42:52 smithi100 ceph-mon[32599]: pgmap v4279: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:42:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:42:52 smithi167 ceph-mon[37140]: pgmap v4279: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:42:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:42:54 smithi138 ceph-mon[36954]: pgmap v4280: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:42:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:42:54 smithi100 ceph-mon[32599]: pgmap v4280: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:42:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:42:54 smithi167 ceph-mon[37140]: pgmap v4280: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:42:56.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:42:56 smithi138 ceph-mon[36954]: pgmap v4281: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:42:56.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:42:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:42:56.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:42:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:42:56.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:42:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:42:56.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:42:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:42:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:42:56 smithi100 ceph-mon[32599]: pgmap v4281: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:42:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:42:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:42:56.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:42:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:42:56.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:42:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:42:56.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:42:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:42:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:42:56 smithi167 ceph-mon[37140]: pgmap v4281: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:42:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:42:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:42:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:42:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:42:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:42:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:42:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:42:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:42:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:42:58 smithi138 ceph-mon[36954]: pgmap v4282: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:42:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:42:58 smithi100 ceph-mon[32599]: pgmap v4282: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:42:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:42:58 smithi167 ceph-mon[37140]: pgmap v4282: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:43:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:43:00 smithi138 ceph-mon[36954]: pgmap v4283: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:43:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:43:00 smithi100 ceph-mon[32599]: pgmap v4283: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:43:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:43:00 smithi167 ceph-mon[37140]: pgmap v4283: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:43:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:43:02 smithi138 ceph-mon[36954]: pgmap v4284: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:43:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:43:02 smithi100 ceph-mon[32599]: pgmap v4284: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:43:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:43:02 smithi167 ceph-mon[37140]: pgmap v4284: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:43:04.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:43:04.523+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:43:04.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:43:04.523+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:43:04.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:43:04.523+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:43:04.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:43:04.523+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:43:04.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:43:04.523+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:43:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:43:04 smithi138 ceph-mon[36954]: pgmap v4285: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:43:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:43:04 smithi100 ceph-mon[32599]: pgmap v4285: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:43:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:43:04 smithi167 ceph-mon[37140]: pgmap v4285: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:43:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:43:06 smithi138 ceph-mon[36954]: pgmap v4286: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:43:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:43:06 smithi100 ceph-mon[32599]: pgmap v4286: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:43:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:43:06 smithi167 ceph-mon[37140]: pgmap v4286: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:43:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:43:08 smithi138 ceph-mon[36954]: pgmap v4287: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:43:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:43:08 smithi138 ceph-mon[36954]: osdmap e316: 12 total, 12 up, 12 in 2024-07-26T15:43:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:43:08 smithi100 ceph-mon[32599]: pgmap v4287: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:43:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:43:08 smithi100 ceph-mon[32599]: osdmap e316: 12 total, 12 up, 12 in 2024-07-26T15:43:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:43:08 smithi167 ceph-mon[37140]: pgmap v4287: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:43:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:43:08 smithi167 ceph-mon[37140]: osdmap e316: 12 total, 12 up, 12 in 2024-07-26T15:43:10.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:43:10 smithi138 ceph-mon[36954]: pgmap v4289: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:43:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:43:10 smithi100 ceph-mon[32599]: pgmap v4289: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:43:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:43:10 smithi167 ceph-mon[37140]: pgmap v4289: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:43:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:43:12 smithi138 ceph-mon[36954]: pgmap v4290: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:43:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:43:12 smithi100 ceph-mon[32599]: pgmap v4290: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:43:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:43:12 smithi167 ceph-mon[37140]: pgmap v4290: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:43:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:43:14 smithi138 ceph-mon[36954]: pgmap v4291: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:43:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:43:14 smithi100 ceph-mon[32599]: pgmap v4291: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:43:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:43:14 smithi167 ceph-mon[37140]: pgmap v4291: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:43:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:43:16 smithi100 ceph-mon[32599]: pgmap v4292: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:43:17.029 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:43:16 smithi167 ceph-mon[37140]: pgmap v4292: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:43:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:43:16 smithi138 ceph-mon[36954]: pgmap v4292: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:43:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:43:18 smithi100 ceph-mon[32599]: pgmap v4293: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:43:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:43:18 smithi167 ceph-mon[37140]: pgmap v4293: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:43:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:43:18 smithi138 ceph-mon[36954]: pgmap v4293: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:43:19.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:43:19.536+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:43:19.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:43:19.537+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:43:19.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:43:19.537+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:43:19.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:43:19.537+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:43:19.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:43:19.537+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:43:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:43:20 smithi100 ceph-mon[32599]: pgmap v4294: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 183 B/s wr, 6 op/s 2024-07-26T15:43:20.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:43:20 smithi100 ceph-mon[32599]: osdmap e317: 12 total, 12 up, 12 in 2024-07-26T15:43:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:43:20 smithi167 ceph-mon[37140]: pgmap v4294: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 183 B/s wr, 6 op/s 2024-07-26T15:43:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:43:20 smithi167 ceph-mon[37140]: osdmap e317: 12 total, 12 up, 12 in 2024-07-26T15:43:21.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:43:20 smithi138 ceph-mon[36954]: pgmap v4294: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 183 B/s wr, 6 op/s 2024-07-26T15:43:21.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:43:20 smithi138 ceph-mon[36954]: osdmap e317: 12 total, 12 up, 12 in 2024-07-26T15:43:22.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:43:22 smithi100 ceph-mon[32599]: pgmap v4296: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:43:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:43:22 smithi167 ceph-mon[37140]: pgmap v4296: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:43:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:43:22 smithi138 ceph-mon[36954]: pgmap v4296: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:43:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:43:24 smithi100 ceph-mon[32599]: pgmap v4297: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:43:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:43:24 smithi167 ceph-mon[37140]: pgmap v4297: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:43:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:43:24 smithi138 ceph-mon[36954]: pgmap v4297: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:43:26.993 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:43:26 smithi138 ceph-mon[36954]: pgmap v4298: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:43:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:43:26 smithi100 ceph-mon[32599]: pgmap v4298: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:43:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:43:26 smithi167 ceph-mon[37140]: pgmap v4298: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:43:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:43:28 smithi100 ceph-mon[32599]: pgmap v4299: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:43:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:43:28 smithi167 ceph-mon[37140]: pgmap v4299: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:43:29.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:43:28 smithi138 ceph-mon[36954]: pgmap v4299: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:43:29.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:43:29 smithi100 ceph-mon[32599]: pgmap v4300: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:43:30.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:43:29 smithi167 ceph-mon[37140]: pgmap v4300: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:43:30.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:43:29 smithi138 ceph-mon[36954]: pgmap v4300: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:43:32.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:43:31 smithi138 ceph-mon[36954]: pgmap v4301: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:43:32.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:43:31 smithi100 ceph-mon[32599]: pgmap v4301: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:43:32.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:43:31 smithi167 ceph-mon[37140]: pgmap v4301: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:43:33.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:43:33.993+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T15:43:33.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:43:33.993+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c003690 con 0x7fe5a81087a0 2024-07-26T15:43:33.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:43:33.993+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 57 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe5a008c0e0 con 0x7fe5a81087a0 2024-07-26T15:43:33.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:43:33.993+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 57 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T15:43:34.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:43:33 smithi138 ceph-mon[36954]: pgmap v4302: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:43:34.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:43:33 smithi100 ceph-mon[32599]: pgmap v4302: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:43:34.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:43:33 smithi167 ceph-mon[37140]: pgmap v4302: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:43:34.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:43:34.544+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:43:34.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:43:34.544+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:43:34.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:43:34.544+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:43:34.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:43:34.544+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:43:34.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:43:34.544+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:43:36.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:43:35 smithi138 ceph-mon[36954]: pgmap v4303: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:43:36.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:43:35 smithi100 ceph-mon[32599]: pgmap v4303: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:43:36.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:43:35 smithi167 ceph-mon[37140]: pgmap v4303: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:43:38.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:43:37 smithi138 ceph-mon[36954]: pgmap v4304: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:43:38.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:43:37 smithi100 ceph-mon[32599]: pgmap v4304: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:43:38.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:43:37 smithi167 ceph-mon[37140]: pgmap v4304: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:43:40.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:43:39 smithi138 ceph-mon[36954]: pgmap v4305: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:43:40.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:43:39 smithi100 ceph-mon[32599]: pgmap v4305: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:43:40.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:43:39 smithi167 ceph-mon[37140]: pgmap v4305: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:43:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:43:42 smithi138 ceph-mon[36954]: pgmap v4306: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:43:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:43:42 smithi100 ceph-mon[32599]: pgmap v4306: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:43:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:43:42 smithi167 ceph-mon[37140]: pgmap v4306: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:43:44.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:43:44 smithi138 ceph-mon[36954]: pgmap v4307: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:43:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:43:44 smithi100 ceph-mon[32599]: pgmap v4307: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:43:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:43:44 smithi167 ceph-mon[37140]: pgmap v4307: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:43:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:43:46 smithi138 ceph-mon[36954]: pgmap v4308: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:43:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:43:46 smithi100 ceph-mon[32599]: pgmap v4308: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:43:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:43:46 smithi167 ceph-mon[37140]: pgmap v4308: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:43:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:43:48 smithi138 ceph-mon[36954]: pgmap v4309: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:43:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:43:48 smithi100 ceph-mon[32599]: pgmap v4309: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:43:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:43:48 smithi167 ceph-mon[37140]: pgmap v4309: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:43:49.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:43:49.550+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:43:49.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:43:49.551+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:43:49.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:43:49.551+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:43:49.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:43:49.551+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:43:49.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:43:49.551+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:43:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:43:50 smithi138 ceph-mon[36954]: pgmap v4310: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:43:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:43:50 smithi100 ceph-mon[32599]: pgmap v4310: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:43:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:43:50 smithi167 ceph-mon[37140]: pgmap v4310: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:43:52.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:43:52 smithi138 ceph-mon[36954]: pgmap v4311: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:43:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:43:52 smithi100 ceph-mon[32599]: pgmap v4311: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:43:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:43:52 smithi167 ceph-mon[37140]: pgmap v4311: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:43:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:43:54 smithi138 ceph-mon[36954]: pgmap v4312: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:43:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:43:54 smithi100 ceph-mon[32599]: pgmap v4312: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:43:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:43:54 smithi167 ceph-mon[37140]: pgmap v4312: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:43:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:43:56 smithi138 ceph-mon[36954]: pgmap v4313: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:43:56.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:43:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:43:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:43:56 smithi100 ceph-mon[32599]: pgmap v4313: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:43:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:43:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:43:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:43:56 smithi167 ceph-mon[37140]: pgmap v4313: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:43:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:43:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:43:57.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:43:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:43:57.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:43:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:43:57.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:43:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:43:57.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:43:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:43:57.997 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:43:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:43:57.997 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:43:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:43:58.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:43:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:43:58.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:43:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:43:58.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:43:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:43:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:43:58 smithi138 ceph-mon[36954]: pgmap v4314: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:43:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:43:58 smithi100 ceph-mon[32599]: pgmap v4314: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:43:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:43:58 smithi167 ceph-mon[37140]: pgmap v4314: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:44:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:44:00 smithi138 ceph-mon[36954]: pgmap v4315: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:44:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:44:00 smithi100 ceph-mon[32599]: pgmap v4315: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:44:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:44:00 smithi167 ceph-mon[37140]: pgmap v4315: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:44:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:44:02 smithi138 ceph-mon[36954]: pgmap v4316: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:44:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:44:02 smithi100 ceph-mon[32599]: pgmap v4316: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:44:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:44:02 smithi167 ceph-mon[37140]: pgmap v4316: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:44:04.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:44:04.565+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:44:04.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:44:04.565+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:44:04.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:44:04.565+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:44:04.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:44:04.565+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:44:04.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:44:04.566+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:44:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:44:04 smithi138 ceph-mon[36954]: pgmap v4317: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:44:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:44:04 smithi100 ceph-mon[32599]: pgmap v4317: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:44:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:44:04 smithi167 ceph-mon[37140]: pgmap v4317: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:44:06.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:44:06 smithi138 ceph-mon[36954]: pgmap v4318: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:44:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:44:06 smithi100 ceph-mon[32599]: pgmap v4318: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:44:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:44:06 smithi167 ceph-mon[37140]: pgmap v4318: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:44:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:44:08 smithi138 ceph-mon[36954]: pgmap v4319: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:44:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:44:08 smithi138 ceph-mon[36954]: osdmap e318: 12 total, 12 up, 12 in 2024-07-26T15:44:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:44:08 smithi100 ceph-mon[32599]: pgmap v4319: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:44:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:44:08 smithi100 ceph-mon[32599]: osdmap e318: 12 total, 12 up, 12 in 2024-07-26T15:44:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:44:08 smithi167 ceph-mon[37140]: pgmap v4319: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:44:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:44:08 smithi167 ceph-mon[37140]: osdmap e318: 12 total, 12 up, 12 in 2024-07-26T15:44:10.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:44:10 smithi138 ceph-mon[36954]: pgmap v4321: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:44:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:44:10 smithi100 ceph-mon[32599]: pgmap v4321: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:44:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:44:10 smithi167 ceph-mon[37140]: pgmap v4321: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:44:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:44:12 smithi138 ceph-mon[36954]: pgmap v4322: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:44:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:44:12 smithi100 ceph-mon[32599]: pgmap v4322: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:44:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:44:12 smithi167 ceph-mon[37140]: pgmap v4322: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:44:14.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:44:14 smithi138 ceph-mon[36954]: pgmap v4323: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:44:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:44:14 smithi100 ceph-mon[32599]: pgmap v4323: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:44:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:44:14 smithi167 ceph-mon[37140]: pgmap v4323: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:44:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:44:16 smithi138 ceph-mon[36954]: pgmap v4324: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:44:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:44:16 smithi138 ceph-mon[36954]: osdmap e319: 12 total, 12 up, 12 in 2024-07-26T15:44:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:44:16 smithi100 ceph-mon[32599]: pgmap v4324: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:44:16.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:44:16 smithi100 ceph-mon[32599]: osdmap e319: 12 total, 12 up, 12 in 2024-07-26T15:44:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:44:16 smithi167 ceph-mon[37140]: pgmap v4324: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:44:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:44:16 smithi167 ceph-mon[37140]: osdmap e319: 12 total, 12 up, 12 in 2024-07-26T15:44:18.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:44:18 smithi138 ceph-mon[36954]: pgmap v4326: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 222 B/s wr, 5 op/s 2024-07-26T15:44:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:44:18 smithi100 ceph-mon[32599]: pgmap v4326: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 222 B/s wr, 5 op/s 2024-07-26T15:44:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:44:18 smithi167 ceph-mon[37140]: pgmap v4326: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 222 B/s wr, 5 op/s 2024-07-26T15:44:19.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:44:19.581+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:44:19.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:44:19.581+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:44:19.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:44:19.581+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:44:19.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:44:19.581+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:44:19.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:44:19.581+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:44:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:44:20 smithi138 ceph-mon[36954]: pgmap v4327: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:44:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:44:20 smithi100 ceph-mon[32599]: pgmap v4327: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:44:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:44:20 smithi167 ceph-mon[37140]: pgmap v4327: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:44:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:44:22 smithi138 ceph-mon[36954]: pgmap v4328: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:44:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:44:22 smithi100 ceph-mon[32599]: pgmap v4328: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:44:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:44:22 smithi167 ceph-mon[37140]: pgmap v4328: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:44:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:44:24 smithi138 ceph-mon[36954]: pgmap v4329: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:44:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:44:24 smithi100 ceph-mon[32599]: pgmap v4329: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:44:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:44:24 smithi167 ceph-mon[37140]: pgmap v4329: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:44:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:44:26 smithi138 ceph-mon[36954]: pgmap v4330: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:44:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:44:26 smithi100 ceph-mon[32599]: pgmap v4330: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:44:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:44:26 smithi167 ceph-mon[37140]: pgmap v4330: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:44:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:44:28 smithi138 ceph-mon[36954]: pgmap v4331: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:44:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:44:28 smithi100 ceph-mon[32599]: pgmap v4331: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:44:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:44:28 smithi167 ceph-mon[37140]: pgmap v4331: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:44:30.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:44:30 smithi138 ceph-mon[36954]: pgmap v4332: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:44:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:44:30 smithi100 ceph-mon[32599]: pgmap v4332: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:44:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:44:30 smithi167 ceph-mon[37140]: pgmap v4332: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:44:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:44:32 smithi138 ceph-mon[36954]: pgmap v4333: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:44:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:44:32 smithi100 ceph-mon[32599]: pgmap v4333: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:44:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:44:32 smithi167 ceph-mon[37140]: pgmap v4333: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:44:34.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:44:34.597+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:44:34.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:44:34.598+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:44:34.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:44:34.598+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:44:34.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:44:34.598+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:44:34.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:44:34.598+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:44:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:44:34 smithi138 ceph-mon[36954]: pgmap v4334: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:44:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:44:34 smithi100 ceph-mon[32599]: pgmap v4334: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:44:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:44:34 smithi167 ceph-mon[37140]: pgmap v4334: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:44:36.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:44:36 smithi138 ceph-mon[36954]: pgmap v4335: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:44:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:44:36 smithi100 ceph-mon[32599]: pgmap v4335: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:44:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:44:36 smithi167 ceph-mon[37140]: pgmap v4335: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:44:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:44:38 smithi138 ceph-mon[36954]: pgmap v4336: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:44:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:44:38 smithi100 ceph-mon[32599]: pgmap v4336: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:44:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:44:38 smithi167 ceph-mon[37140]: pgmap v4336: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:44:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:44:40 smithi138 ceph-mon[36954]: pgmap v4337: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:44:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:44:40 smithi100 ceph-mon[32599]: pgmap v4337: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:44:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:44:40 smithi167 ceph-mon[37140]: pgmap v4337: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:44:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:44:42 smithi138 ceph-mon[36954]: pgmap v4338: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:44:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:44:42 smithi100 ceph-mon[32599]: pgmap v4338: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:44:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:44:42 smithi167 ceph-mon[37140]: pgmap v4338: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:44:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:44:44 smithi138 ceph-mon[36954]: pgmap v4339: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:44:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:44:44 smithi100 ceph-mon[32599]: pgmap v4339: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:44:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:44:44 smithi167 ceph-mon[37140]: pgmap v4339: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:44:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:44:46 smithi138 ceph-mon[36954]: pgmap v4340: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:44:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:44:46 smithi100 ceph-mon[32599]: pgmap v4340: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:44:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:44:46 smithi167 ceph-mon[37140]: pgmap v4340: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:44:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:44:48 smithi138 ceph-mon[36954]: pgmap v4341: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:44:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:44:48 smithi100 ceph-mon[32599]: pgmap v4341: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:44:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:44:48 smithi167 ceph-mon[37140]: pgmap v4341: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:44:49.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:44:49.614+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:44:49.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:44:49.614+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:44:49.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:44:49.614+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:44:49.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:44:49.614+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:44:49.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:44:49.614+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:44:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:44:50 smithi138 ceph-mon[36954]: pgmap v4342: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:44:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:44:50 smithi100 ceph-mon[32599]: pgmap v4342: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:44:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:44:50 smithi167 ceph-mon[37140]: pgmap v4342: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:44:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:44:52 smithi138 ceph-mon[36954]: pgmap v4343: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:44:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:44:52 smithi100 ceph-mon[32599]: pgmap v4343: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:44:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:44:52 smithi167 ceph-mon[37140]: pgmap v4343: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:44:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:44:54 smithi138 ceph-mon[36954]: pgmap v4344: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:44:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:44:54 smithi100 ceph-mon[32599]: pgmap v4344: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:44:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:44:54 smithi167 ceph-mon[37140]: pgmap v4344: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:44:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:44:56 smithi138 ceph-mon[36954]: pgmap v4345: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:44:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:44:56 smithi100 ceph-mon[32599]: pgmap v4345: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:44:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:44:56 smithi167 ceph-mon[37140]: pgmap v4345: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:44:57.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:44:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:44:57.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:44:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:44:57.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:44:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:44:57.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:44:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:44:57.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:44:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:44:57.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:44:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:44:57.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:44:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:44:57.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:44:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:44:58.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:44:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:44:58.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:44:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:44:58.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:44:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:44:58.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:44:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:44:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:44:58 smithi100 ceph-mon[32599]: pgmap v4346: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:44:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:44:58 smithi167 ceph-mon[37140]: pgmap v4346: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:44:59.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:44:58 smithi138 ceph-mon[36954]: pgmap v4346: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:45:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:45:00 smithi100 ceph-mon[32599]: pgmap v4347: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:45:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:45:00 smithi167 ceph-mon[37140]: pgmap v4347: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:45:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:45:00 smithi138 ceph-mon[36954]: pgmap v4347: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:45:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:45:02 smithi100 ceph-mon[32599]: pgmap v4348: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:45:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:45:02 smithi167 ceph-mon[37140]: pgmap v4348: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:45:03.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:45:02 smithi138 ceph-mon[36954]: pgmap v4348: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:45:04.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:45:04.005+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T15:45:04.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:45:04.005+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c004740 con 0x7fe5a81087a0 2024-07-26T15:45:04.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:45:04.006+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 58 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T15:45:04.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:45:04.006+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 58 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c100 con 0x7fe5a81087a0 2024-07-26T15:45:04.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:45:04.630+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:45:04.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:45:04.630+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:45:04.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:45:04.630+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:45:04.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:45:04.630+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:45:04.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:45:04.630+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:45:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:45:04 smithi100 ceph-mon[32599]: pgmap v4349: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:45:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:45:04 smithi167 ceph-mon[37140]: pgmap v4349: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:45:05.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:45:04 smithi138 ceph-mon[36954]: pgmap v4349: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:45:06.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:45:06 smithi100 ceph-mon[32599]: pgmap v4350: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:45:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:45:06 smithi167 ceph-mon[37140]: pgmap v4350: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:45:07.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:45:06 smithi138 ceph-mon[36954]: pgmap v4350: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:45:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:45:08 smithi100 ceph-mon[32599]: pgmap v4351: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:45:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:45:08 smithi100 ceph-mon[32599]: osdmap e320: 12 total, 12 up, 12 in 2024-07-26T15:45:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:45:08 smithi167 ceph-mon[37140]: pgmap v4351: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:45:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:45:08 smithi167 ceph-mon[37140]: osdmap e320: 12 total, 12 up, 12 in 2024-07-26T15:45:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:45:08 smithi138 ceph-mon[36954]: pgmap v4351: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:45:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:45:08 smithi138 ceph-mon[36954]: osdmap e320: 12 total, 12 up, 12 in 2024-07-26T15:45:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:45:10 smithi100 ceph-mon[32599]: pgmap v4353: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:45:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:45:10 smithi167 ceph-mon[37140]: pgmap v4353: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:45:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:45:10 smithi138 ceph-mon[36954]: pgmap v4353: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:45:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:45:12 smithi100 ceph-mon[32599]: pgmap v4354: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:45:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:45:12 smithi167 ceph-mon[37140]: pgmap v4354: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:45:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:45:12 smithi138 ceph-mon[36954]: pgmap v4354: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:45:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:45:14 smithi100 ceph-mon[32599]: pgmap v4355: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:45:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:45:14 smithi167 ceph-mon[37140]: pgmap v4355: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:45:15.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:45:14 smithi138 ceph-mon[36954]: pgmap v4355: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:45:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:45:16 smithi100 ceph-mon[32599]: pgmap v4356: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:45:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:45:16 smithi167 ceph-mon[37140]: pgmap v4356: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:45:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:45:16 smithi138 ceph-mon[36954]: pgmap v4356: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:45:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:45:18 smithi100 ceph-mon[32599]: pgmap v4357: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:45:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:45:18 smithi167 ceph-mon[37140]: pgmap v4357: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:45:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:45:18 smithi138 ceph-mon[36954]: pgmap v4357: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:45:19.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:45:19.642+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:45:19.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:45:19.643+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:45:19.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:45:19.643+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:45:19.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:45:19.643+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:45:19.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:45:19.643+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:45:19.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:45:19 smithi100 ceph-mon[32599]: pgmap v4358: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 183 B/s wr, 7 op/s 2024-07-26T15:45:20.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:45:19 smithi167 ceph-mon[37140]: pgmap v4358: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 183 B/s wr, 7 op/s 2024-07-26T15:45:20.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:45:19 smithi138 ceph-mon[36954]: pgmap v4358: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 183 B/s wr, 7 op/s 2024-07-26T15:45:21.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:45:21 smithi138 ceph-mon[36954]: osdmap e321: 12 total, 12 up, 12 in 2024-07-26T15:45:21.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:45:21 smithi100 ceph-mon[32599]: osdmap e321: 12 total, 12 up, 12 in 2024-07-26T15:45:22.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:45:21 smithi167 ceph-mon[37140]: osdmap e321: 12 total, 12 up, 12 in 2024-07-26T15:45:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:45:22 smithi138 ceph-mon[36954]: pgmap v4360: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:45:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:45:22 smithi100 ceph-mon[32599]: pgmap v4360: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:45:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:45:22 smithi167 ceph-mon[37140]: pgmap v4360: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:45:24.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:45:24 smithi138 ceph-mon[36954]: pgmap v4361: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:45:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:45:24 smithi100 ceph-mon[32599]: pgmap v4361: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:45:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:45:24 smithi167 ceph-mon[37140]: pgmap v4361: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:45:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:45:26 smithi138 ceph-mon[36954]: pgmap v4362: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:45:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:45:26 smithi100 ceph-mon[32599]: pgmap v4362: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:45:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:45:26 smithi167 ceph-mon[37140]: pgmap v4362: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:45:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:45:28 smithi138 ceph-mon[36954]: pgmap v4363: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:45:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:45:28 smithi100 ceph-mon[32599]: pgmap v4363: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:45:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:45:28 smithi167 ceph-mon[37140]: pgmap v4363: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:45:30.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:45:30 smithi138 ceph-mon[36954]: pgmap v4364: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:45:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:45:30 smithi100 ceph-mon[32599]: pgmap v4364: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:45:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:45:30 smithi167 ceph-mon[37140]: pgmap v4364: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:45:32.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:45:32 smithi138 ceph-mon[36954]: pgmap v4365: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:45:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:45:32 smithi100 ceph-mon[32599]: pgmap v4365: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:45:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:45:32 smithi167 ceph-mon[37140]: pgmap v4365: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:45:34.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:45:34.658+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:45:34.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:45:34.658+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:45:34.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:45:34.658+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:45:34.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:45:34.658+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:45:34.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:45:34.658+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:45:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:45:34 smithi138 ceph-mon[36954]: pgmap v4366: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:45:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:45:34 smithi100 ceph-mon[32599]: pgmap v4366: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:45:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:45:34 smithi167 ceph-mon[37140]: pgmap v4366: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:45:36.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:45:36 smithi138 ceph-mon[36954]: pgmap v4367: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:45:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:45:36 smithi100 ceph-mon[32599]: pgmap v4367: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:45:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:45:36 smithi167 ceph-mon[37140]: pgmap v4367: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:45:38.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:45:38 smithi138 ceph-mon[36954]: pgmap v4368: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:45:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:45:38 smithi100 ceph-mon[32599]: pgmap v4368: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:45:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:45:38 smithi167 ceph-mon[37140]: pgmap v4368: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:45:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:45:40 smithi138 ceph-mon[36954]: pgmap v4369: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:45:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:45:40 smithi100 ceph-mon[32599]: pgmap v4369: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:45:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:45:40 smithi167 ceph-mon[37140]: pgmap v4369: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:45:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:45:42 smithi138 ceph-mon[36954]: pgmap v4370: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:45:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:45:42 smithi100 ceph-mon[32599]: pgmap v4370: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:45:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:45:42 smithi167 ceph-mon[37140]: pgmap v4370: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:45:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:45:44 smithi138 ceph-mon[36954]: pgmap v4371: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:45:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:45:44 smithi100 ceph-mon[32599]: pgmap v4371: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:45:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:45:44 smithi167 ceph-mon[37140]: pgmap v4371: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:45:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:45:46 smithi138 ceph-mon[36954]: pgmap v4372: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:45:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:45:46 smithi100 ceph-mon[32599]: pgmap v4372: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:45:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:45:46 smithi167 ceph-mon[37140]: pgmap v4372: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:45:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:45:48 smithi138 ceph-mon[36954]: pgmap v4373: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:45:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:45:48 smithi100 ceph-mon[32599]: pgmap v4373: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:45:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:45:48 smithi167 ceph-mon[37140]: pgmap v4373: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:45:49.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:45:49.674+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:45:49.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:45:49.675+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:45:49.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:45:49.675+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:45:49.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:45:49.675+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:45:49.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:45:49.675+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:45:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:45:50 smithi100 ceph-mon[32599]: pgmap v4374: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:45:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:45:50 smithi167 ceph-mon[37140]: pgmap v4374: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:45:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:45:50 smithi138 ceph-mon[36954]: pgmap v4374: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:45:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:45:52 smithi100 ceph-mon[32599]: pgmap v4375: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:45:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:45:52 smithi167 ceph-mon[37140]: pgmap v4375: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:45:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:45:52 smithi138 ceph-mon[36954]: pgmap v4375: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:45:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:45:54 smithi100 ceph-mon[32599]: pgmap v4376: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:45:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:45:54 smithi167 ceph-mon[37140]: pgmap v4376: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:45:55.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:45:54 smithi138 ceph-mon[36954]: pgmap v4376: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:45:56.993 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:45:56 smithi138 ceph-mon[36954]: pgmap v4377: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:45:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:45:56 smithi100 ceph-mon[32599]: pgmap v4377: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:45:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:45:56 smithi167 ceph-mon[37140]: pgmap v4377: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:45:57.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:45:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:45:58.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:45:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:45:58.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:45:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:45:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:45:58 smithi100 ceph-mon[32599]: pgmap v4378: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:45:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:45:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:45:58.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:45:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:45:58.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:45:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:45:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:45:58 smithi167 ceph-mon[37140]: pgmap v4378: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:45:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:45:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:45:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:45:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:45:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:45:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:45:59.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:45:58 smithi138 ceph-mon[36954]: pgmap v4378: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:45:59.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:45:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:45:59.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:45:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:45:59.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:45:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:46:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:46:00 smithi100 ceph-mon[32599]: pgmap v4379: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:46:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:46:00 smithi167 ceph-mon[37140]: pgmap v4379: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:46:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:46:00 smithi138 ceph-mon[36954]: pgmap v4379: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:46:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:46:02 smithi100 ceph-mon[32599]: pgmap v4380: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:46:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:46:02 smithi167 ceph-mon[37140]: pgmap v4380: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:46:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:46:02 smithi138 ceph-mon[36954]: pgmap v4380: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:46:04.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:46:04.679+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:46:04.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:46:04.680+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:46:04.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:46:04.680+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:46:04.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:46:04.680+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:46:04.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:46:04.680+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:46:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:46:04 smithi100 ceph-mon[32599]: pgmap v4381: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:46:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:46:04 smithi167 ceph-mon[37140]: pgmap v4381: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:46:05.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:46:04 smithi138 ceph-mon[36954]: pgmap v4381: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:46:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:46:06 smithi100 ceph-mon[32599]: pgmap v4382: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:46:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:46:06 smithi167 ceph-mon[37140]: pgmap v4382: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:46:07.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:46:06 smithi138 ceph-mon[36954]: pgmap v4382: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:46:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:46:08 smithi100 ceph-mon[32599]: pgmap v4383: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:46:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:46:08 smithi100 ceph-mon[32599]: osdmap e322: 12 total, 12 up, 12 in 2024-07-26T15:46:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:46:08 smithi167 ceph-mon[37140]: pgmap v4383: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:46:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:46:08 smithi167 ceph-mon[37140]: osdmap e322: 12 total, 12 up, 12 in 2024-07-26T15:46:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:46:08 smithi138 ceph-mon[36954]: pgmap v4383: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:46:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:46:08 smithi138 ceph-mon[36954]: osdmap e322: 12 total, 12 up, 12 in 2024-07-26T15:46:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:46:10 smithi100 ceph-mon[32599]: pgmap v4385: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:46:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:46:10 smithi167 ceph-mon[37140]: pgmap v4385: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:46:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:46:10 smithi138 ceph-mon[36954]: pgmap v4385: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:46:12.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:46:11 smithi138 ceph-mon[36954]: pgmap v4386: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:46:12.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:46:11 smithi100 ceph-mon[32599]: pgmap v4386: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:46:12.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:46:11 smithi167 ceph-mon[37140]: pgmap v4386: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:46:14.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:46:13 smithi100 ceph-mon[32599]: pgmap v4387: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:46:14.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:46:13 smithi167 ceph-mon[37140]: pgmap v4387: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:46:14.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:46:13 smithi138 ceph-mon[36954]: pgmap v4387: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:46:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:46:16 smithi138 ceph-mon[36954]: pgmap v4388: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:46:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:46:16 smithi138 ceph-mon[36954]: osdmap e323: 12 total, 12 up, 12 in 2024-07-26T15:46:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:46:16 smithi100 ceph-mon[32599]: pgmap v4388: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:46:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:46:16 smithi100 ceph-mon[32599]: osdmap e323: 12 total, 12 up, 12 in 2024-07-26T15:46:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:46:16 smithi167 ceph-mon[37140]: pgmap v4388: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:46:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:46:16 smithi167 ceph-mon[37140]: osdmap e323: 12 total, 12 up, 12 in 2024-07-26T15:46:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:46:18 smithi138 ceph-mon[36954]: pgmap v4390: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 222 B/s wr, 5 op/s 2024-07-26T15:46:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:46:18 smithi100 ceph-mon[32599]: pgmap v4390: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 222 B/s wr, 5 op/s 2024-07-26T15:46:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:46:18 smithi167 ceph-mon[37140]: pgmap v4390: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 222 B/s wr, 5 op/s 2024-07-26T15:46:19.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:46:19.687+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:46:19.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:46:19.687+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:46:19.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:46:19.687+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:46:19.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:46:19.687+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:46:19.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:46:19.688+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:46:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:46:20 smithi138 ceph-mon[36954]: pgmap v4391: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:46:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:46:20 smithi100 ceph-mon[32599]: pgmap v4391: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:46:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:46:20 smithi167 ceph-mon[37140]: pgmap v4391: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:46:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:46:22 smithi138 ceph-mon[36954]: pgmap v4392: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:46:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:46:22 smithi100 ceph-mon[32599]: pgmap v4392: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:46:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:46:22 smithi167 ceph-mon[37140]: pgmap v4392: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:46:24.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:46:24 smithi138 ceph-mon[36954]: pgmap v4393: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:46:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:46:24 smithi100 ceph-mon[32599]: pgmap v4393: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:46:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:46:24 smithi167 ceph-mon[37140]: pgmap v4393: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:46:26.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:46:26 smithi138 ceph-mon[36954]: pgmap v4394: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:46:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:46:26 smithi100 ceph-mon[32599]: pgmap v4394: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:46:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:46:26 smithi167 ceph-mon[37140]: pgmap v4394: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:46:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:46:28 smithi138 ceph-mon[36954]: pgmap v4395: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:46:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:46:28 smithi100 ceph-mon[32599]: pgmap v4395: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:46:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:46:28 smithi167 ceph-mon[37140]: pgmap v4395: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:46:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:46:30 smithi138 ceph-mon[36954]: pgmap v4396: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:46:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:46:30 smithi100 ceph-mon[32599]: pgmap v4396: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:46:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:46:30 smithi167 ceph-mon[37140]: pgmap v4396: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:46:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:46:32 smithi138 ceph-mon[36954]: pgmap v4397: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:46:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:46:32 smithi100 ceph-mon[32599]: pgmap v4397: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:46:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:46:32 smithi167 ceph-mon[37140]: pgmap v4397: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:46:34.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:46:34.696+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:46:34.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:46:34.697+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:46:34.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:46:34.697+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:46:34.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:46:34.697+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:46:34.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:46:34.697+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:46:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:46:34 smithi138 ceph-mon[36954]: pgmap v4398: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:46:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:46:34 smithi100 ceph-mon[32599]: pgmap v4398: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:46:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:46:34 smithi167 ceph-mon[37140]: pgmap v4398: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:46:36.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:46:36 smithi138 ceph-mon[36954]: pgmap v4399: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:46:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:46:36 smithi100 ceph-mon[32599]: pgmap v4399: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:46:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:46:36 smithi167 ceph-mon[37140]: pgmap v4399: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:46:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:46:38 smithi100 ceph-mon[32599]: pgmap v4400: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:46:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:46:38 smithi167 ceph-mon[37140]: pgmap v4400: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:46:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:46:38 smithi138 ceph-mon[36954]: pgmap v4400: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:46:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:46:40 smithi100 ceph-mon[32599]: pgmap v4401: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:46:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:46:40 smithi167 ceph-mon[37140]: pgmap v4401: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:46:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:46:40 smithi138 ceph-mon[36954]: pgmap v4401: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:46:42.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:46:42 smithi100 ceph-mon[32599]: pgmap v4402: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:46:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:46:42 smithi167 ceph-mon[37140]: pgmap v4402: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:46:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:46:42 smithi138 ceph-mon[36954]: pgmap v4402: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:46:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:46:44 smithi100 ceph-mon[32599]: pgmap v4403: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:46:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:46:44 smithi167 ceph-mon[37140]: pgmap v4403: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:46:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:46:44 smithi138 ceph-mon[36954]: pgmap v4403: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:46:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:46:46 smithi100 ceph-mon[32599]: pgmap v4404: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:46:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:46:46 smithi167 ceph-mon[37140]: pgmap v4404: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:46:47.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:46:46 smithi138 ceph-mon[36954]: pgmap v4404: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:46:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:46:48 smithi100 ceph-mon[32599]: pgmap v4405: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:46:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:46:48 smithi167 ceph-mon[37140]: pgmap v4405: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:46:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:46:48 smithi138 ceph-mon[36954]: pgmap v4405: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:46:49.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:46:49.712+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:46:49.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:46:49.712+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:46:49.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:46:49.712+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:46:49.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:46:49.712+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:46:49.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:46:49.712+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:46:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:46:50 smithi100 ceph-mon[32599]: pgmap v4406: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:46:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:46:50 smithi167 ceph-mon[37140]: pgmap v4406: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:46:51.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:46:50 smithi138 ceph-mon[36954]: pgmap v4406: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:46:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:46:52 smithi100 ceph-mon[32599]: pgmap v4407: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:46:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:46:52 smithi167 ceph-mon[37140]: pgmap v4407: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:46:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:46:52 smithi138 ceph-mon[36954]: pgmap v4407: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:46:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:46:54 smithi100 ceph-mon[32599]: pgmap v4408: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:46:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:46:54 smithi167 ceph-mon[37140]: pgmap v4408: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:46:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:46:54 smithi138 ceph-mon[36954]: pgmap v4408: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:46:56.993 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:46:56 smithi138 ceph-mon[36954]: pgmap v4409: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:46:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:46:56 smithi100 ceph-mon[32599]: pgmap v4409: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:46:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:46:56 smithi167 ceph-mon[37140]: pgmap v4409: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:46:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:46:58 smithi100 ceph-mon[32599]: pgmap v4410: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:46:58.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:46:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:46:58.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:46:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:46:58.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:46:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:46:58.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:46:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:46:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:46:58 smithi167 ceph-mon[37140]: pgmap v4410: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:46:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:46:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:46:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:46:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:46:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:46:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:46:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:46:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:46:59.154 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:46:58 smithi138 ceph-mon[36954]: pgmap v4410: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:46:59.154 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:46:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:46:59.154 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:46:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:46:59.154 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:46:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:46:59.155 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:46:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:47:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:00 smithi100 ceph-mon[32599]: pgmap v4411: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T15:47:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:00 smithi167 ceph-mon[37140]: pgmap v4411: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T15:47:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:00 smithi138 ceph-mon[36954]: pgmap v4411: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T15:47:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:02 smithi100 ceph-mon[32599]: pgmap v4412: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:47:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:02 smithi167 ceph-mon[37140]: pgmap v4412: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:47:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:02 smithi138 ceph-mon[36954]: pgmap v4412: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:47:04.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:47:04.723+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:47:04.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:47:04.723+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:47:04.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:47:04.723+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:47:04.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:47:04.723+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:47:04.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:47:04.723+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:47:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:04 smithi100 ceph-mon[32599]: pgmap v4413: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:47:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:04 smithi167 ceph-mon[37140]: pgmap v4413: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:47:05.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:04 smithi138 ceph-mon[36954]: pgmap v4413: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:47:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:06 smithi100 ceph-mon[32599]: pgmap v4414: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:47:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:06 smithi167 ceph-mon[37140]: pgmap v4414: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:47:07.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:06 smithi138 ceph-mon[36954]: pgmap v4414: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:47:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:08 smithi138 ceph-mon[36954]: pgmap v4415: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:47:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:08 smithi138 ceph-mon[36954]: osdmap e324: 12 total, 12 up, 12 in 2024-07-26T15:47:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:08 smithi167 ceph-mon[37140]: pgmap v4415: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:47:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:08 smithi167 ceph-mon[37140]: osdmap e324: 12 total, 12 up, 12 in 2024-07-26T15:47:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:08 smithi100 ceph-mon[32599]: pgmap v4415: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:47:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:08 smithi100 ceph-mon[32599]: osdmap e324: 12 total, 12 up, 12 in 2024-07-26T15:47:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:10 smithi167 ceph-mon[37140]: pgmap v4417: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:47:11.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:10 smithi138 ceph-mon[36954]: pgmap v4417: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:47:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:10 smithi100 ceph-mon[32599]: pgmap v4417: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:47:12.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:11 smithi138 ceph-mon[36954]: pgmap v4418: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:47:12.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:11 smithi100 ceph-mon[32599]: pgmap v4418: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:47:12.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:11 smithi167 ceph-mon[37140]: pgmap v4418: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:47:14.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:13 smithi100 ceph-mon[32599]: pgmap v4419: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:47:14.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:13 smithi167 ceph-mon[37140]: pgmap v4419: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:47:14.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:13 smithi138 ceph-mon[36954]: pgmap v4419: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:47:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:16 smithi138 ceph-mon[36954]: pgmap v4420: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:47:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:16 smithi138 ceph-mon[36954]: osdmap e325: 12 total, 12 up, 12 in 2024-07-26T15:47:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:16 smithi100 ceph-mon[32599]: pgmap v4420: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:47:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:16 smithi100 ceph-mon[32599]: osdmap e325: 12 total, 12 up, 12 in 2024-07-26T15:47:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:16 smithi167 ceph-mon[37140]: pgmap v4420: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:47:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:16 smithi167 ceph-mon[37140]: osdmap e325: 12 total, 12 up, 12 in 2024-07-26T15:47:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:18 smithi138 ceph-mon[36954]: pgmap v4422: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 222 B/s wr, 5 op/s 2024-07-26T15:47:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:18 smithi100 ceph-mon[32599]: pgmap v4422: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 222 B/s wr, 5 op/s 2024-07-26T15:47:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:18 smithi167 ceph-mon[37140]: pgmap v4422: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 222 B/s wr, 5 op/s 2024-07-26T15:47:19.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:47:19.739+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:47:19.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:47:19.740+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:47:19.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:47:19.740+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:47:19.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:47:19.740+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:47:19.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:47:19.740+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:47:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:20 smithi138 ceph-mon[36954]: pgmap v4423: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:47:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:20 smithi100 ceph-mon[32599]: pgmap v4423: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:47:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:20 smithi167 ceph-mon[37140]: pgmap v4423: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:47:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:22 smithi138 ceph-mon[36954]: pgmap v4424: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:47:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:22 smithi100 ceph-mon[32599]: pgmap v4424: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:47:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:22 smithi167 ceph-mon[37140]: pgmap v4424: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:47:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:24 smithi138 ceph-mon[36954]: pgmap v4425: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:47:24.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:24 smithi100 ceph-mon[32599]: pgmap v4425: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:47:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:24 smithi167 ceph-mon[37140]: pgmap v4425: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:47:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:26 smithi138 ceph-mon[36954]: pgmap v4426: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:47:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:26 smithi100 ceph-mon[32599]: pgmap v4426: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:47:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:26 smithi167 ceph-mon[37140]: pgmap v4426: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:47:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:28 smithi138 ceph-mon[36954]: pgmap v4427: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:47:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:28 smithi100 ceph-mon[32599]: pgmap v4427: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:47:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:28 smithi167 ceph-mon[37140]: pgmap v4427: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:47:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:30 smithi100 ceph-mon[32599]: pgmap v4428: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:47:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:30 smithi167 ceph-mon[37140]: pgmap v4428: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:47:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:30 smithi138 ceph-mon[36954]: pgmap v4428: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:47:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:32 smithi100 ceph-mon[32599]: pgmap v4429: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:47:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:32 smithi167 ceph-mon[37140]: pgmap v4429: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:47:33.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:32 smithi138 ceph-mon[36954]: pgmap v4429: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:47:34.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:47:34.755+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:47:34.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:47:34.756+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:47:34.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:47:34.756+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:47:34.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:47:34.756+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:47:34.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:47:34.756+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:47:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:34 smithi100 ceph-mon[32599]: pgmap v4430: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:47:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:34 smithi167 ceph-mon[37140]: pgmap v4430: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:47:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:34 smithi138 ceph-mon[36954]: pgmap v4430: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:47:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:36 smithi100 ceph-mon[32599]: pgmap v4431: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:47:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:36 smithi167 ceph-mon[37140]: pgmap v4431: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:47:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:36 smithi138 ceph-mon[36954]: pgmap v4431: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:47:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:38 smithi100 ceph-mon[32599]: pgmap v4432: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:47:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:38 smithi167 ceph-mon[37140]: pgmap v4432: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:47:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:38 smithi138 ceph-mon[36954]: pgmap v4432: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:47:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:40 smithi100 ceph-mon[32599]: pgmap v4433: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:47:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:40 smithi167 ceph-mon[37140]: pgmap v4433: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:47:41.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:40 smithi138 ceph-mon[36954]: pgmap v4433: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:47:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:42 smithi100 ceph-mon[32599]: pgmap v4434: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:47:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:42 smithi167 ceph-mon[37140]: pgmap v4434: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:47:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:42 smithi138 ceph-mon[36954]: pgmap v4434: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:47:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:44 smithi100 ceph-mon[32599]: pgmap v4435: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:47:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:44 smithi167 ceph-mon[37140]: pgmap v4435: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:47:45.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:44 smithi138 ceph-mon[36954]: pgmap v4435: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:47:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:46 smithi100 ceph-mon[32599]: pgmap v4436: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:47:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:46 smithi167 ceph-mon[37140]: pgmap v4436: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:47:47.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:46 smithi138 ceph-mon[36954]: pgmap v4436: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:47:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:48 smithi100 ceph-mon[32599]: pgmap v4437: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:47:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:48 smithi167 ceph-mon[37140]: pgmap v4437: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:47:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:48 smithi138 ceph-mon[36954]: pgmap v4437: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:47:49.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:47:49.771+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:47:49.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:47:49.772+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:47:49.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:47:49.772+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:47:49.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:47:49.772+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:47:49.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:47:49.772+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:47:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:50 smithi100 ceph-mon[32599]: pgmap v4438: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:47:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:50 smithi167 ceph-mon[37140]: pgmap v4438: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:47:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:50 smithi138 ceph-mon[36954]: pgmap v4438: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:47:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:52 smithi100 ceph-mon[32599]: pgmap v4439: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:47:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:52 smithi167 ceph-mon[37140]: pgmap v4439: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:47:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:52 smithi138 ceph-mon[36954]: pgmap v4439: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:47:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:54 smithi100 ceph-mon[32599]: pgmap v4440: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:47:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:54 smithi167 ceph-mon[37140]: pgmap v4440: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:47:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:54 smithi138 ceph-mon[36954]: pgmap v4440: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:47:56.993 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:56 smithi138 ceph-mon[36954]: pgmap v4441: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:47:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:56 smithi100 ceph-mon[32599]: pgmap v4441: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:47:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:56 smithi167 ceph-mon[37140]: pgmap v4441: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:47:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:58 smithi100 ceph-mon[32599]: pgmap v4442: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:47:58.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:47:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:58 smithi167 ceph-mon[37140]: pgmap v4442: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:47:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:47:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:58 smithi138 ceph-mon[36954]: pgmap v4442: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:47:59.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:47:59.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:47:59.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:47:59.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:47:59.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:47:59.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:47:59.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:47:59.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:47:59.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:47:59.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:47:59.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:47:59.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:47:59.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:47:59.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:47:59.997 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:47:59.997 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:47:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:48:00.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:48:00.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:48:00.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:48:00.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:48:00.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:48:00.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:48:00.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:48:00.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:48:00.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:48:00.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:48:00.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:48:00.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:48:00.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:48:00.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:48:00.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:47:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:48:00.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:48:00.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:48:00.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:48:00.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:48:00.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:48:00.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:48:00.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:48:00.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:48:00.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:48:00.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:48:00.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:48:00.193 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:48:00.193 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:48:00.193 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:48:00.193 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:47:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:48:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:00 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T15:48:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:00 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T15:48:00.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:00 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T15:48:00.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:00 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T15:48:00.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:00 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T15:48:00.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:00 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T15:48:00.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:00 smithi100 ceph-mon[32599]: pgmap v4443: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:48:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:00 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T15:48:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:00 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T15:48:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:00 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T15:48:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:00 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T15:48:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:00 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T15:48:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:00 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T15:48:01.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:00 smithi167 ceph-mon[37140]: pgmap v4443: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:48:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:00 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T15:48:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:00 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T15:48:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:00 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T15:48:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:00 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T15:48:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:00 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T15:48:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:00 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T15:48:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:00 smithi138 ceph-mon[36954]: pgmap v4443: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:48:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:02 smithi100 ceph-mon[32599]: pgmap v4444: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:48:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:02 smithi167 ceph-mon[37140]: pgmap v4444: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:48:03.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:48:03.034+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 msgr2=0x7fe5940c0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-07-26T15:48:03.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:48:03.034+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 msgr2=0x7fe5940c0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-07-26T15:48:03.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:48:03.034+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe590005d00 tx=0x7fe59003b040 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:48:03.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:48:03.034+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe590005d00 tx=0x7fe59003b040 comp rx=0 tx=0).stop 2024-07-26T15:48:03.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:48:03.034+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 msgr2=0x7fe5940c0480 unknown :-1 s=STATE_CLOSED l=1).tick see no progress in more than 10000000 us during connecting to v2:172.21.15.100:6800/3937009534, fault. 2024-07-26T15:48:03.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:48:03.034+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 msgr2=0x7fe5940c0480 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-07-26T15:48:03.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:48:03.034+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T15:48:03.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:48:03.034+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b4ef0 0x7fe5940b5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T15:48:03.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:48:03.034+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b4ef0 0x7fe5940b5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:48:03.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:48:03.035+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b4ef0 0x7fe5940b5300 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fe590004850 tx=0x7fe590069040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T15:48:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:02 smithi138 ceph-mon[36954]: pgmap v4444: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:48:04.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:48:04.030+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T15:48:04.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:48:04.030+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c003690 con 0x7fe5a81087a0 2024-07-26T15:48:04.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:48:04.031+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 59 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c100 con 0x7fe5a81087a0 2024-07-26T15:48:04.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:48:04.031+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 59 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T15:48:04.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:48:04.773+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:48:04.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:48:04.774+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:48:04.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:48:04.774+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:48:04.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:48:04.774+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:48:04.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:48:04.774+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:48:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:04 smithi167 ceph-mon[37140]: pgmap v4445: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:48:05.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:04 smithi138 ceph-mon[36954]: pgmap v4445: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:48:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:04 smithi100 ceph-mon[32599]: pgmap v4445: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:48:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:06 smithi167 ceph-mon[37140]: pgmap v4446: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:48:07.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:06 smithi138 ceph-mon[36954]: pgmap v4446: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:48:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:06 smithi100 ceph-mon[32599]: pgmap v4446: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:48:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:08 smithi167 ceph-mon[37140]: pgmap v4447: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:48:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:08 smithi167 ceph-mon[37140]: osdmap e326: 12 total, 12 up, 12 in 2024-07-26T15:48:09.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:08 smithi100 ceph-mon[32599]: pgmap v4447: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:48:09.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:08 smithi100 ceph-mon[32599]: osdmap e326: 12 total, 12 up, 12 in 2024-07-26T15:48:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:08 smithi138 ceph-mon[36954]: pgmap v4447: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T15:48:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:08 smithi138 ceph-mon[36954]: osdmap e326: 12 total, 12 up, 12 in 2024-07-26T15:48:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:10 smithi167 ceph-mon[37140]: pgmap v4449: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:48:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:10 smithi138 ceph-mon[36954]: pgmap v4449: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:48:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:10 smithi100 ceph-mon[32599]: pgmap v4449: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:48:12.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:11 smithi167 ceph-mon[37140]: pgmap v4450: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:48:12.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:11 smithi138 ceph-mon[36954]: pgmap v4450: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:48:12.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:11 smithi100 ceph-mon[32599]: pgmap v4450: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:48:14.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:14 smithi167 ceph-mon[37140]: pgmap v4451: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:48:14.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:14 smithi138 ceph-mon[36954]: pgmap v4451: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:48:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:14 smithi100 ceph-mon[32599]: pgmap v4451: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:48:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:16 smithi138 ceph-mon[36954]: pgmap v4452: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:48:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:16 smithi138 ceph-mon[36954]: osdmap e327: 12 total, 12 up, 12 in 2024-07-26T15:48:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:16 smithi100 ceph-mon[32599]: pgmap v4452: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:48:16.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:16 smithi100 ceph-mon[32599]: osdmap e327: 12 total, 12 up, 12 in 2024-07-26T15:48:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:16 smithi167 ceph-mon[37140]: pgmap v4452: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:48:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:16 smithi167 ceph-mon[37140]: osdmap e327: 12 total, 12 up, 12 in 2024-07-26T15:48:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:18 smithi138 ceph-mon[36954]: pgmap v4454: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 222 B/s wr, 5 op/s 2024-07-26T15:48:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:18 smithi100 ceph-mon[32599]: pgmap v4454: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 222 B/s wr, 5 op/s 2024-07-26T15:48:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:18 smithi167 ceph-mon[37140]: pgmap v4454: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 222 B/s wr, 5 op/s 2024-07-26T15:48:19.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:48:19.790+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:48:19.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:48:19.791+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:48:19.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:48:19.791+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:48:19.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:48:19.791+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:48:19.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:48:19.791+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:48:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:20 smithi138 ceph-mon[36954]: pgmap v4455: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:48:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:20 smithi100 ceph-mon[32599]: pgmap v4455: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:48:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:20 smithi167 ceph-mon[37140]: pgmap v4455: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:48:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:22 smithi100 ceph-mon[32599]: pgmap v4456: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:48:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:22 smithi167 ceph-mon[37140]: pgmap v4456: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:48:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:22 smithi138 ceph-mon[36954]: pgmap v4456: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:48:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:24 smithi100 ceph-mon[32599]: pgmap v4457: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:48:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:24 smithi167 ceph-mon[37140]: pgmap v4457: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:48:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:24 smithi138 ceph-mon[36954]: pgmap v4457: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:48:26.993 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:26 smithi138 ceph-mon[36954]: pgmap v4458: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:48:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:26 smithi100 ceph-mon[32599]: pgmap v4458: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:48:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:26 smithi167 ceph-mon[37140]: pgmap v4458: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:48:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:28 smithi100 ceph-mon[32599]: pgmap v4459: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:48:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:28 smithi167 ceph-mon[37140]: pgmap v4459: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:48:29.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:28 smithi138 ceph-mon[36954]: pgmap v4459: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T15:48:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:30 smithi100 ceph-mon[32599]: pgmap v4460: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:48:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:30 smithi167 ceph-mon[37140]: pgmap v4460: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:48:31.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:30 smithi138 ceph-mon[36954]: pgmap v4460: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:48:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:32 smithi100 ceph-mon[32599]: pgmap v4461: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:48:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:32 smithi167 ceph-mon[37140]: pgmap v4461: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:48:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:32 smithi138 ceph-mon[36954]: pgmap v4461: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:48:34.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:48:34.806+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:48:34.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:48:34.807+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:48:34.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:48:34.807+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:48:34.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:48:34.807+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:48:34.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:48:34.807+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:48:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:34 smithi100 ceph-mon[32599]: pgmap v4462: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:48:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:34 smithi167 ceph-mon[37140]: pgmap v4462: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:48:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:34 smithi138 ceph-mon[36954]: pgmap v4462: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:48:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:36 smithi100 ceph-mon[32599]: pgmap v4463: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:48:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:36 smithi167 ceph-mon[37140]: pgmap v4463: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:48:37.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:36 smithi138 ceph-mon[36954]: pgmap v4463: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:48:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:38 smithi100 ceph-mon[32599]: pgmap v4464: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:48:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:38 smithi167 ceph-mon[37140]: pgmap v4464: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:48:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:38 smithi138 ceph-mon[36954]: pgmap v4464: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:48:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:40 smithi100 ceph-mon[32599]: pgmap v4465: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:48:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:40 smithi167 ceph-mon[37140]: pgmap v4465: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:48:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:40 smithi138 ceph-mon[36954]: pgmap v4465: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:48:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:42 smithi100 ceph-mon[32599]: pgmap v4466: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:48:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:42 smithi167 ceph-mon[37140]: pgmap v4466: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:48:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:42 smithi138 ceph-mon[36954]: pgmap v4466: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:48:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:44 smithi100 ceph-mon[32599]: pgmap v4467: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:48:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:44 smithi167 ceph-mon[37140]: pgmap v4467: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:48:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:44 smithi138 ceph-mon[36954]: pgmap v4467: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:48:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:46 smithi100 ceph-mon[32599]: pgmap v4468: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:48:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:46 smithi167 ceph-mon[37140]: pgmap v4468: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:48:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:46 smithi138 ceph-mon[36954]: pgmap v4468: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:48:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:48 smithi100 ceph-mon[32599]: pgmap v4469: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:48:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:48 smithi167 ceph-mon[37140]: pgmap v4469: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:48:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:48 smithi138 ceph-mon[36954]: pgmap v4469: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:48:49.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:48:49.822+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:48:49.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:48:49.823+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:48:49.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:48:49.823+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:48:49.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:48:49.823+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:48:49.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:48:49.823+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:48:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:50 smithi167 ceph-mon[37140]: pgmap v4470: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:48:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:50 smithi138 ceph-mon[36954]: pgmap v4470: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:48:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:50 smithi100 ceph-mon[32599]: pgmap v4470: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:48:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:52 smithi167 ceph-mon[37140]: pgmap v4471: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:48:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:52 smithi138 ceph-mon[36954]: pgmap v4471: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:48:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:52 smithi100 ceph-mon[32599]: pgmap v4471: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:48:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:54 smithi167 ceph-mon[37140]: pgmap v4472: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:48:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:54 smithi138 ceph-mon[36954]: pgmap v4472: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:48:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:54 smithi100 ceph-mon[32599]: pgmap v4472: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:48:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:56 smithi167 ceph-mon[37140]: pgmap v4473: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:48:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:56 smithi138 ceph-mon[36954]: pgmap v4473: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:48:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:56 smithi100 ceph-mon[32599]: pgmap v4473: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:48:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:58 smithi167 ceph-mon[37140]: pgmap v4474: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:48:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:58 smithi138 ceph-mon[36954]: pgmap v4474: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:48:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:58 smithi100 ceph-mon[32599]: pgmap v4474: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:49:00.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:49:00.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:49:00.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:49:00.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:48:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:49:00.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:49:00.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:49:00.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:49:00.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:48:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:49:00.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:49:00.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:49:00.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:49:00.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:48:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:49:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:49:00 smithi167 ceph-mon[37140]: pgmap v4475: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:49:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:49:00 smithi138 ceph-mon[36954]: pgmap v4475: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:49:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:49:00 smithi100 ceph-mon[32599]: pgmap v4475: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T15:49:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:49:02 smithi167 ceph-mon[37140]: pgmap v4476: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:49:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:49:02 smithi138 ceph-mon[36954]: pgmap v4476: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:49:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:49:02 smithi100 ceph-mon[32599]: pgmap v4476: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:49:04.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:49:03 smithi138 ceph-mon[36954]: pgmap v4477: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:49:04.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:49:03 smithi100 ceph-mon[32599]: pgmap v4477: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:49:04.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:49:03 smithi167 ceph-mon[37140]: pgmap v4477: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:49:04.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:49:04.829+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:49:04.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:49:04.829+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:49:04.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:49:04.829+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:49:04.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:49:04.829+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:49:04.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:49:04.829+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:49:06.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:49:06 smithi138 ceph-mon[36954]: pgmap v4478: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:49:06.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:49:06 smithi100 ceph-mon[32599]: pgmap v4478: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:49:06.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:49:06 smithi167 ceph-mon[37140]: pgmap v4478: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:49:08.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:49:08 smithi138 ceph-mon[36954]: pgmap v4479: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:49:08.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:49:08 smithi138 ceph-mon[36954]: osdmap e328: 12 total, 12 up, 12 in 2024-07-26T15:49:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:49:08 smithi100 ceph-mon[32599]: pgmap v4479: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:49:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:49:08 smithi100 ceph-mon[32599]: osdmap e328: 12 total, 12 up, 12 in 2024-07-26T15:49:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:49:08 smithi167 ceph-mon[37140]: pgmap v4479: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:49:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:49:08 smithi167 ceph-mon[37140]: osdmap e328: 12 total, 12 up, 12 in 2024-07-26T15:49:10.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:49:10 smithi100 ceph-mon[32599]: pgmap v4481: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:49:10.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:49:10 smithi138 ceph-mon[36954]: pgmap v4481: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:49:10.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:49:10 smithi167 ceph-mon[37140]: pgmap v4481: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:49:12.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:49:12 smithi138 ceph-mon[36954]: pgmap v4482: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:49:12.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:49:12 smithi100 ceph-mon[32599]: pgmap v4482: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:49:12.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:49:12 smithi167 ceph-mon[37140]: pgmap v4482: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:49:14.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:49:14 smithi138 ceph-mon[36954]: pgmap v4483: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:49:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:49:14 smithi100 ceph-mon[32599]: pgmap v4483: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:49:14.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:49:14 smithi167 ceph-mon[37140]: pgmap v4483: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:49:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:49:16 smithi100 ceph-mon[32599]: pgmap v4484: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:49:16.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:49:16 smithi100 ceph-mon[32599]: osdmap e329: 12 total, 12 up, 12 in 2024-07-26T15:49:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:49:16 smithi167 ceph-mon[37140]: pgmap v4484: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:49:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:49:16 smithi167 ceph-mon[37140]: osdmap e329: 12 total, 12 up, 12 in 2024-07-26T15:49:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:49:16 smithi138 ceph-mon[36954]: pgmap v4484: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:49:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:49:16 smithi138 ceph-mon[36954]: osdmap e329: 12 total, 12 up, 12 in 2024-07-26T15:49:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:49:18 smithi100 ceph-mon[32599]: pgmap v4486: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T15:49:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:49:18 smithi167 ceph-mon[37140]: pgmap v4486: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T15:49:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:49:18 smithi138 ceph-mon[36954]: pgmap v4486: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T15:49:19.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:49:19.844+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:49:19.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:49:19.845+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:49:19.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:49:19.845+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:49:19.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:49:19.845+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:49:19.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:49:19.845+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:49:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:49:20 smithi100 ceph-mon[32599]: pgmap v4487: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:49:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:49:20 smithi167 ceph-mon[37140]: pgmap v4487: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:49:21.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:49:20 smithi138 ceph-mon[36954]: pgmap v4487: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:49:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:49:22 smithi100 ceph-mon[32599]: pgmap v4488: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:49:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:49:22 smithi167 ceph-mon[37140]: pgmap v4488: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:49:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:49:22 smithi138 ceph-mon[36954]: pgmap v4488: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:49:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:49:24 smithi100 ceph-mon[32599]: pgmap v4489: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:49:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:49:24 smithi167 ceph-mon[37140]: pgmap v4489: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:49:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:49:24 smithi138 ceph-mon[36954]: pgmap v4489: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:49:26.993 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:49:26 smithi138 ceph-mon[36954]: pgmap v4490: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:49:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:49:26 smithi100 ceph-mon[32599]: pgmap v4490: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:49:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:49:26 smithi167 ceph-mon[37140]: pgmap v4490: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:49:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:49:28 smithi100 ceph-mon[32599]: pgmap v4491: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T15:49:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:49:28 smithi167 ceph-mon[37140]: pgmap v4491: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T15:49:29.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:49:28 smithi138 ceph-mon[36954]: pgmap v4491: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T15:49:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:49:30 smithi100 ceph-mon[32599]: pgmap v4492: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:49:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:49:30 smithi167 ceph-mon[37140]: pgmap v4492: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:49:31.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:49:30 smithi138 ceph-mon[36954]: pgmap v4492: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:49:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:49:32 smithi100 ceph-mon[32599]: pgmap v4493: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:49:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:49:32 smithi167 ceph-mon[37140]: pgmap v4493: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:49:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:49:32 smithi138 ceph-mon[36954]: pgmap v4493: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:49:34.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:49:34.861+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:49:34.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:49:34.862+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:49:34.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:49:34.862+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:49:34.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:49:34.862+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:49:34.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:49:34.862+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:49:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:49:34 smithi100 ceph-mon[32599]: pgmap v4494: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:49:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:49:34 smithi167 ceph-mon[37140]: pgmap v4494: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:49:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:49:34 smithi138 ceph-mon[36954]: pgmap v4494: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:49:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:49:36 smithi100 ceph-mon[32599]: pgmap v4495: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:49:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:49:36 smithi167 ceph-mon[37140]: pgmap v4495: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:49:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:49:36 smithi138 ceph-mon[36954]: pgmap v4495: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:49:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:49:38 smithi100 ceph-mon[32599]: pgmap v4496: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:49:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:49:38 smithi167 ceph-mon[37140]: pgmap v4496: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:49:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:49:38 smithi138 ceph-mon[36954]: pgmap v4496: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:49:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:49:40 smithi100 ceph-mon[32599]: pgmap v4497: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:49:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:49:40 smithi167 ceph-mon[37140]: pgmap v4497: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:49:41.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:49:40 smithi138 ceph-mon[36954]: pgmap v4497: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:49:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:49:42 smithi100 ceph-mon[32599]: pgmap v4498: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:49:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:49:42 smithi167 ceph-mon[37140]: pgmap v4498: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:49:43.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:49:42 smithi138 ceph-mon[36954]: pgmap v4498: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:49:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:49:44 smithi167 ceph-mon[37140]: pgmap v4499: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:49:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:49:44 smithi138 ceph-mon[36954]: pgmap v4499: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:49:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:49:44 smithi100 ceph-mon[32599]: pgmap v4499: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:49:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:49:46 smithi167 ceph-mon[37140]: pgmap v4500: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:49:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:49:46 smithi138 ceph-mon[36954]: pgmap v4500: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:49:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:49:46 smithi100 ceph-mon[32599]: pgmap v4500: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:49:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:49:48 smithi167 ceph-mon[37140]: pgmap v4501: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:49:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:49:48 smithi138 ceph-mon[36954]: pgmap v4501: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:49:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:49:48 smithi100 ceph-mon[32599]: pgmap v4501: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:49:49.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:49:49.877+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:49:49.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:49:49.878+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:49:49.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:49:49.878+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:49:49.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:49:49.878+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:49:49.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:49:49.878+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:49:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:49:50 smithi167 ceph-mon[37140]: pgmap v4502: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:49:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:49:50 smithi138 ceph-mon[36954]: pgmap v4502: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:49:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:49:50 smithi100 ceph-mon[32599]: pgmap v4502: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:49:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:49:52 smithi167 ceph-mon[37140]: pgmap v4503: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:49:53.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:49:52 smithi138 ceph-mon[36954]: pgmap v4503: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:49:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:49:52 smithi100 ceph-mon[32599]: pgmap v4503: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:49:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:49:54 smithi167 ceph-mon[37140]: pgmap v4504: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:49:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:49:54 smithi138 ceph-mon[36954]: pgmap v4504: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:49:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:49:54 smithi100 ceph-mon[32599]: pgmap v4504: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:49:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:49:56 smithi167 ceph-mon[37140]: pgmap v4505: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:49:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:49:56 smithi138 ceph-mon[36954]: pgmap v4505: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:49:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:49:56 smithi100 ceph-mon[32599]: pgmap v4505: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:49:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:49:58 smithi167 ceph-mon[37140]: pgmap v4506: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:49:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:49:58 smithi138 ceph-mon[36954]: pgmap v4506: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:49:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:49:58 smithi100 ceph-mon[32599]: pgmap v4506: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:50:00.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:49:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:50:00.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:49:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:50:00.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:49:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:50:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:50:00 smithi138 ceph-mon[36954]: pgmap v4507: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:50:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:50:00 smithi138 ceph-mon[36954]: overall HEALTH_OK 2024-07-26T15:50:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:50:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:50:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:50:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:50:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:50:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:50:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:50:00 smithi100 ceph-mon[32599]: pgmap v4507: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:50:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:50:00 smithi100 ceph-mon[32599]: overall HEALTH_OK 2024-07-26T15:50:01.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:50:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:50:01.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:50:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:50:01.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:50:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:50:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:50:00 smithi167 ceph-mon[37140]: pgmap v4507: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:50:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:50:00 smithi167 ceph-mon[37140]: overall HEALTH_OK 2024-07-26T15:50:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:50:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:50:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:50:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:50:01.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:50:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:50:02.174 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:50:01 smithi167 ceph-mon[37140]: pgmap v4508: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:50:02.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:50:01 smithi138 ceph-mon[36954]: pgmap v4508: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:50:02.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:50:01 smithi100 ceph-mon[32599]: pgmap v4508: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:50:04.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:50:04 smithi138 ceph-mon[36954]: pgmap v4509: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:50:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:50:04 smithi100 ceph-mon[32599]: pgmap v4509: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:50:04.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:50:04 smithi167 ceph-mon[37140]: pgmap v4509: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:50:04.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:50:04.893+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:50:04.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:50:04.894+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:50:04.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:50:04.894+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:50:04.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:50:04.894+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:50:04.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:50:04.894+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:50:06.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:50:06 smithi138 ceph-mon[36954]: pgmap v4510: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:50:06.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:50:06 smithi100 ceph-mon[32599]: pgmap v4510: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:50:06.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:50:06 smithi167 ceph-mon[37140]: pgmap v4510: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:50:08.064 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:50:08 smithi100 ceph-mon[32599]: pgmap v4511: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:50:08.064 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:50:08 smithi100 ceph-mon[32599]: osdmap e330: 12 total, 12 up, 12 in 2024-07-26T15:50:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:50:08 smithi138 ceph-mon[36954]: pgmap v4511: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:50:08.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:50:08 smithi138 ceph-mon[36954]: osdmap e330: 12 total, 12 up, 12 in 2024-07-26T15:50:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:50:08 smithi167 ceph-mon[37140]: pgmap v4511: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:50:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:50:08 smithi167 ceph-mon[37140]: osdmap e330: 12 total, 12 up, 12 in 2024-07-26T15:50:10.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:50:10 smithi138 ceph-mon[36954]: pgmap v4513: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:50:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:50:10 smithi100 ceph-mon[32599]: pgmap v4513: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:50:10.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:50:10 smithi167 ceph-mon[37140]: pgmap v4513: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:50:12.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:50:12 smithi138 ceph-mon[36954]: pgmap v4514: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:50:12.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:50:12 smithi100 ceph-mon[32599]: pgmap v4514: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:50:12.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:50:12 smithi167 ceph-mon[37140]: pgmap v4514: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:50:14.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:50:14 smithi138 ceph-mon[36954]: pgmap v4515: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:50:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:50:14 smithi100 ceph-mon[32599]: pgmap v4515: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:50:14.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:50:14 smithi167 ceph-mon[37140]: pgmap v4515: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:50:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:50:16 smithi100 ceph-mon[32599]: pgmap v4516: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:50:16.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:50:16 smithi100 ceph-mon[32599]: osdmap e331: 12 total, 12 up, 12 in 2024-07-26T15:50:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:50:16 smithi167 ceph-mon[37140]: pgmap v4516: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:50:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:50:16 smithi167 ceph-mon[37140]: osdmap e331: 12 total, 12 up, 12 in 2024-07-26T15:50:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:50:16 smithi138 ceph-mon[36954]: pgmap v4516: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:50:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:50:16 smithi138 ceph-mon[36954]: osdmap e331: 12 total, 12 up, 12 in 2024-07-26T15:50:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:50:18 smithi100 ceph-mon[32599]: pgmap v4518: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T15:50:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:50:18 smithi167 ceph-mon[37140]: pgmap v4518: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T15:50:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:50:18 smithi138 ceph-mon[36954]: pgmap v4518: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T15:50:19.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:50:19.910+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:50:19.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:50:19.910+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:50:19.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:50:19.910+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:50:19.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:50:19.910+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:50:19.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:50:19.910+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:50:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:50:20 smithi100 ceph-mon[32599]: pgmap v4519: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:50:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:50:20 smithi167 ceph-mon[37140]: pgmap v4519: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:50:21.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:50:20 smithi138 ceph-mon[36954]: pgmap v4519: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:50:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:50:22 smithi100 ceph-mon[32599]: pgmap v4520: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:50:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:50:22 smithi167 ceph-mon[37140]: pgmap v4520: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:50:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:50:22 smithi138 ceph-mon[36954]: pgmap v4520: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:50:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:50:24 smithi100 ceph-mon[32599]: pgmap v4521: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:50:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:50:24 smithi167 ceph-mon[37140]: pgmap v4521: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:50:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:50:24 smithi138 ceph-mon[36954]: pgmap v4521: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:50:26.993 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:50:26 smithi138 ceph-mon[36954]: pgmap v4522: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:50:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:50:26 smithi100 ceph-mon[32599]: pgmap v4522: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:50:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:50:26 smithi167 ceph-mon[37140]: pgmap v4522: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:50:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:50:28 smithi100 ceph-mon[32599]: pgmap v4523: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T15:50:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:50:28 smithi167 ceph-mon[37140]: pgmap v4523: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T15:50:29.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:50:28 smithi138 ceph-mon[36954]: pgmap v4523: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T15:50:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:50:30 smithi100 ceph-mon[32599]: pgmap v4524: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:50:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:50:30 smithi167 ceph-mon[37140]: pgmap v4524: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:50:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:50:30 smithi138 ceph-mon[36954]: pgmap v4524: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:50:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:50:32 smithi100 ceph-mon[32599]: pgmap v4525: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:50:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:50:32 smithi167 ceph-mon[37140]: pgmap v4525: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:50:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:50:32 smithi138 ceph-mon[36954]: pgmap v4525: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:50:34.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:50:34.918+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:50:34.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:50:34.918+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:50:34.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:50:34.918+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:50:34.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:50:34.918+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:50:34.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:50:34.918+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:50:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:50:34 smithi167 ceph-mon[37140]: pgmap v4526: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:50:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:50:34 smithi138 ceph-mon[36954]: pgmap v4526: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:50:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:50:34 smithi100 ceph-mon[32599]: pgmap v4526: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:50:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:50:36 smithi167 ceph-mon[37140]: pgmap v4527: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:50:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:50:36 smithi138 ceph-mon[36954]: pgmap v4527: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:50:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:50:36 smithi100 ceph-mon[32599]: pgmap v4527: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:50:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:50:38 smithi167 ceph-mon[37140]: pgmap v4528: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:50:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:50:38 smithi138 ceph-mon[36954]: pgmap v4528: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:50:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:50:38 smithi100 ceph-mon[32599]: pgmap v4528: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:50:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:50:40 smithi167 ceph-mon[37140]: pgmap v4529: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:50:41.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:50:40 smithi100 ceph-mon[32599]: pgmap v4529: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:50:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:50:40 smithi138 ceph-mon[36954]: pgmap v4529: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:50:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:50:42 smithi167 ceph-mon[37140]: pgmap v4530: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:50:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:50:42 smithi138 ceph-mon[36954]: pgmap v4530: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:50:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:50:42 smithi100 ceph-mon[32599]: pgmap v4530: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:50:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:50:44 smithi167 ceph-mon[37140]: pgmap v4531: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:50:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:50:44 smithi138 ceph-mon[36954]: pgmap v4531: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:50:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:50:44 smithi100 ceph-mon[32599]: pgmap v4531: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:50:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:50:46 smithi167 ceph-mon[37140]: pgmap v4532: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:50:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:50:46 smithi138 ceph-mon[36954]: pgmap v4532: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:50:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:50:46 smithi100 ceph-mon[32599]: pgmap v4532: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:50:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:50:48 smithi167 ceph-mon[37140]: pgmap v4533: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:50:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:50:48 smithi138 ceph-mon[36954]: pgmap v4533: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:50:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:50:48 smithi100 ceph-mon[32599]: pgmap v4533: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:50:49.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:50:49.926+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:50:49.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:50:49.927+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:50:49.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:50:49.927+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:50:49.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:50:49.927+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:50:49.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:50:49.927+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:50:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:50:50 smithi138 ceph-mon[36954]: pgmap v4534: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:50:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:50:50 smithi100 ceph-mon[32599]: pgmap v4534: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:50:51.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:50:50 smithi167 ceph-mon[37140]: pgmap v4534: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:50:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:50:52 smithi138 ceph-mon[36954]: pgmap v4535: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:50:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:50:52 smithi100 ceph-mon[32599]: pgmap v4535: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:50:53.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:50:52 smithi167 ceph-mon[37140]: pgmap v4535: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:50:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:50:54 smithi138 ceph-mon[36954]: pgmap v4536: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:50:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:50:54 smithi100 ceph-mon[32599]: pgmap v4536: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:50:55.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:50:54 smithi167 ceph-mon[37140]: pgmap v4536: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:50:57.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:50:56 smithi138 ceph-mon[36954]: pgmap v4537: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:50:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:50:56 smithi100 ceph-mon[32599]: pgmap v4537: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:50:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:50:56 smithi167 ceph-mon[37140]: pgmap v4537: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:50:59.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:50:58 smithi138 ceph-mon[36954]: pgmap v4538: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:50:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:50:58 smithi100 ceph-mon[32599]: pgmap v4538: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:50:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:50:58 smithi167 ceph-mon[37140]: pgmap v4538: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:51:00.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:50:59 smithi138 ceph-mon[36954]: pgmap v4539: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:51:00.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:50:59 smithi100 ceph-mon[32599]: pgmap v4539: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:51:00.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:50:59 smithi167 ceph-mon[37140]: pgmap v4539: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:51:00.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:51:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:51:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:51:02.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:02 smithi167 ceph-mon[37140]: pgmap v4540: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:51:02.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:51:02.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:51:02.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:51:02.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:51:02.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:51:02.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:51:02.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:51:02.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:51:02.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:51:02.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:02 smithi138 ceph-mon[36954]: pgmap v4540: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:51:02.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:51:02.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:51:02.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:51:02.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:51:02.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:51:02.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:51:02.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:51:02.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:51:02.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:51:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:02 smithi100 ceph-mon[32599]: pgmap v4540: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:51:02.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:51:02.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:51:02.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:51:02.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:51:02.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:51:02.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:51:02.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:51:02.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:51:02.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:51:04.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:51:04.057+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T15:51:04.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:51:04.057+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c004740 con 0x7fe5a81087a0 2024-07-26T15:51:04.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:51:04.057+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 60 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c004740 con 0x7fe5a81087a0 2024-07-26T15:51:04.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:51:04.057+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 60 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T15:51:04.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:04 smithi138 ceph-mon[36954]: pgmap v4541: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:51:04.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:51:04.942+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:51:04.944 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:04 smithi100 ceph-mon[32599]: pgmap v4541: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:51:04.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:51:04.943+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:51:04.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:51:04.943+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:51:04.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:51:04.943+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:51:04.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:51:04.943+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:51:05.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:04 smithi167 ceph-mon[37140]: pgmap v4541: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:51:06.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:06 smithi138 ceph-mon[36954]: pgmap v4542: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:51:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:06 smithi100 ceph-mon[32599]: pgmap v4542: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:51:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:06 smithi167 ceph-mon[37140]: pgmap v4542: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:51:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:08 smithi138 ceph-mon[36954]: pgmap v4543: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:51:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:08 smithi138 ceph-mon[36954]: osdmap e332: 12 total, 12 up, 12 in 2024-07-26T15:51:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:08 smithi100 ceph-mon[32599]: pgmap v4543: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:51:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:08 smithi100 ceph-mon[32599]: osdmap e332: 12 total, 12 up, 12 in 2024-07-26T15:51:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:08 smithi167 ceph-mon[37140]: pgmap v4543: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:51:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:08 smithi167 ceph-mon[37140]: osdmap e332: 12 total, 12 up, 12 in 2024-07-26T15:51:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:10 smithi138 ceph-mon[36954]: pgmap v4545: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:51:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:10 smithi100 ceph-mon[32599]: pgmap v4545: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:51:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:10 smithi167 ceph-mon[37140]: pgmap v4545: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:51:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:12 smithi138 ceph-mon[36954]: pgmap v4546: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:51:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:12 smithi100 ceph-mon[32599]: pgmap v4546: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:51:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:12 smithi167 ceph-mon[37140]: pgmap v4546: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:51:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:14 smithi138 ceph-mon[36954]: pgmap v4547: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:51:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:14 smithi100 ceph-mon[32599]: pgmap v4547: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:51:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:14 smithi167 ceph-mon[37140]: pgmap v4547: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:51:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:16 smithi100 ceph-mon[32599]: pgmap v4548: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:51:16.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:16 smithi100 ceph-mon[32599]: osdmap e333: 12 total, 12 up, 12 in 2024-07-26T15:51:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:16 smithi167 ceph-mon[37140]: pgmap v4548: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:51:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:16 smithi167 ceph-mon[37140]: osdmap e333: 12 total, 12 up, 12 in 2024-07-26T15:51:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:16 smithi138 ceph-mon[36954]: pgmap v4548: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:51:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:16 smithi138 ceph-mon[36954]: osdmap e333: 12 total, 12 up, 12 in 2024-07-26T15:51:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:18 smithi100 ceph-mon[32599]: pgmap v4550: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 216 B/s wr, 6 op/s 2024-07-26T15:51:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:18 smithi167 ceph-mon[37140]: pgmap v4550: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 216 B/s wr, 6 op/s 2024-07-26T15:51:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:18 smithi138 ceph-mon[36954]: pgmap v4550: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 216 B/s wr, 6 op/s 2024-07-26T15:51:19.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:51:19.951+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:51:19.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:51:19.952+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:51:19.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:51:19.952+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:51:19.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:51:19.952+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:51:19.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:51:19.952+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:51:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:20 smithi100 ceph-mon[32599]: pgmap v4551: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:51:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:20 smithi167 ceph-mon[37140]: pgmap v4551: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:51:21.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:20 smithi138 ceph-mon[36954]: pgmap v4551: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:51:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:22 smithi100 ceph-mon[32599]: pgmap v4552: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:51:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:22 smithi167 ceph-mon[37140]: pgmap v4552: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:51:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:22 smithi138 ceph-mon[36954]: pgmap v4552: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:51:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:24 smithi100 ceph-mon[32599]: pgmap v4553: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:51:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:24 smithi167 ceph-mon[37140]: pgmap v4553: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:51:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:24 smithi138 ceph-mon[36954]: pgmap v4553: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:51:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:26 smithi167 ceph-mon[37140]: pgmap v4554: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:51:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:26 smithi138 ceph-mon[36954]: pgmap v4554: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:51:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:26 smithi100 ceph-mon[32599]: pgmap v4554: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:51:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:28 smithi167 ceph-mon[37140]: pgmap v4555: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:51:29.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:28 smithi138 ceph-mon[36954]: pgmap v4555: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:51:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:28 smithi100 ceph-mon[32599]: pgmap v4555: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:51:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:30 smithi167 ceph-mon[37140]: pgmap v4556: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:51:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:30 smithi138 ceph-mon[36954]: pgmap v4556: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:51:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:30 smithi100 ceph-mon[32599]: pgmap v4556: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:51:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:32 smithi167 ceph-mon[37140]: pgmap v4557: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:51:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:32 smithi138 ceph-mon[36954]: pgmap v4557: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:51:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:32 smithi100 ceph-mon[32599]: pgmap v4557: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:51:34.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:51:34.961+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:51:34.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:51:34.961+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:51:34.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:51:34.961+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:51:34.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:51:34.961+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:51:34.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:51:34.961+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:51:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:34 smithi167 ceph-mon[37140]: pgmap v4558: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:51:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:34 smithi138 ceph-mon[36954]: pgmap v4558: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:51:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:34 smithi100 ceph-mon[32599]: pgmap v4558: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:51:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:36 smithi167 ceph-mon[37140]: pgmap v4559: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:51:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:36 smithi138 ceph-mon[36954]: pgmap v4559: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:51:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:36 smithi100 ceph-mon[32599]: pgmap v4559: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:51:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:38 smithi167 ceph-mon[37140]: pgmap v4560: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:51:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:38 smithi138 ceph-mon[36954]: pgmap v4560: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:51:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:38 smithi100 ceph-mon[32599]: pgmap v4560: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:51:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:40 smithi167 ceph-mon[37140]: pgmap v4561: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:51:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:40 smithi138 ceph-mon[36954]: pgmap v4561: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:51:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:40 smithi100 ceph-mon[32599]: pgmap v4561: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:51:43.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:42 smithi100 ceph-mon[32599]: pgmap v4562: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:51:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:42 smithi138 ceph-mon[36954]: pgmap v4562: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:51:43.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:42 smithi167 ceph-mon[37140]: pgmap v4562: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:51:45.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:44 smithi138 ceph-mon[36954]: pgmap v4563: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:51:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:44 smithi100 ceph-mon[32599]: pgmap v4563: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:51:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:44 smithi167 ceph-mon[37140]: pgmap v4563: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:51:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:46 smithi138 ceph-mon[36954]: pgmap v4564: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:51:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:46 smithi100 ceph-mon[32599]: pgmap v4564: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:51:47.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:46 smithi167 ceph-mon[37140]: pgmap v4564: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:51:49.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:51:49.064+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c004740 con 0x7fe5a81087a0 2024-07-26T15:51:49.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:51:49.064+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T15:51:49.066 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:48 smithi100 ceph-mon[32599]: pgmap v4565: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:51:49.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:51:49.064+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 61 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe5a008c0e0 con 0x7fe5a81087a0 2024-07-26T15:51:49.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:51:49.064+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 61 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T15:51:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:48 smithi138 ceph-mon[36954]: pgmap v4565: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:51:49.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:48 smithi167 ceph-mon[37140]: pgmap v4565: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:51:49.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:51:49.962+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:51:49.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:51:49.962+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:51:49.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:51:49.962+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:51:49.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:51:49.962+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:51:49.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:51:49.963+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:51:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:50 smithi138 ceph-mon[36954]: pgmap v4566: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:51:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:50 smithi100 ceph-mon[32599]: pgmap v4566: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:51:51.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:50 smithi167 ceph-mon[37140]: pgmap v4566: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:51:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:52 smithi138 ceph-mon[36954]: pgmap v4567: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:51:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:52 smithi100 ceph-mon[32599]: pgmap v4567: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:51:53.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:52 smithi167 ceph-mon[37140]: pgmap v4567: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:51:55.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:54 smithi138 ceph-mon[36954]: pgmap v4568: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:51:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:54 smithi100 ceph-mon[32599]: pgmap v4568: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:51:55.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:54 smithi167 ceph-mon[37140]: pgmap v4568: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:51:57.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:56 smithi138 ceph-mon[36954]: pgmap v4569: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:51:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:56 smithi100 ceph-mon[32599]: pgmap v4569: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:51:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:56 smithi167 ceph-mon[37140]: pgmap v4569: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:51:58.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:51:57 smithi138 ceph-mon[36954]: pgmap v4570: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:51:58.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:51:57 smithi100 ceph-mon[32599]: pgmap v4570: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:51:58.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:51:57 smithi167 ceph-mon[37140]: pgmap v4570: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:52:00.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:00 smithi138 ceph-mon[36954]: pgmap v4571: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:52:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:00 smithi100 ceph-mon[32599]: pgmap v4571: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:52:00.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:00 smithi167 ceph-mon[37140]: pgmap v4571: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:52:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:02 smithi167 ceph-mon[37140]: pgmap v4572: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:52:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:52:03.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:52:03.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:52:03.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:52:03.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:52:03.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:52:03.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:52:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:02 smithi138 ceph-mon[36954]: pgmap v4572: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:52:03.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:52:03.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:52:03.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:52:03.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:52:03.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:52:03.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:52:03.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:52:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:02 smithi100 ceph-mon[32599]: pgmap v4572: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:52:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:52:03.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:52:03.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:52:03.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:52:03.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:52:03.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:52:03.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:52:04.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:52:04.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:52:04.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:52:04.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:52:04.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:52:04.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:52:04.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:52:04.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:52:04.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:52:04.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:52:04.978+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:52:04.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:52:04.979+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:52:04.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:52:04.979+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:52:04.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:52:04.979+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:52:04.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:52:04.979+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:52:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:04 smithi167 ceph-mon[37140]: pgmap v4573: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:52:05.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:04 smithi138 ceph-mon[36954]: pgmap v4573: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:52:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:04 smithi100 ceph-mon[32599]: pgmap v4573: 97 pgs: 97 active+clean; 234 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:52:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:06 smithi167 ceph-mon[37140]: pgmap v4574: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:52:07.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:06 smithi138 ceph-mon[36954]: pgmap v4574: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:52:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:06 smithi100 ceph-mon[32599]: pgmap v4574: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:52:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:08 smithi167 ceph-mon[37140]: pgmap v4575: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:52:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:08 smithi167 ceph-mon[37140]: osdmap e334: 12 total, 12 up, 12 in 2024-07-26T15:52:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:08 smithi138 ceph-mon[36954]: pgmap v4575: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:52:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:08 smithi138 ceph-mon[36954]: osdmap e334: 12 total, 12 up, 12 in 2024-07-26T15:52:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:08 smithi100 ceph-mon[32599]: pgmap v4575: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:52:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:08 smithi100 ceph-mon[32599]: osdmap e334: 12 total, 12 up, 12 in 2024-07-26T15:52:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:10 smithi167 ceph-mon[37140]: pgmap v4577: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:52:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:10 smithi138 ceph-mon[36954]: pgmap v4577: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:52:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:10 smithi100 ceph-mon[32599]: pgmap v4577: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:52:13.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:12 smithi100 ceph-mon[32599]: pgmap v4578: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:52:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:12 smithi138 ceph-mon[36954]: pgmap v4578: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:52:13.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:12 smithi167 ceph-mon[37140]: pgmap v4578: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:52:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:14 smithi138 ceph-mon[36954]: pgmap v4579: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:52:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:14 smithi100 ceph-mon[32599]: pgmap v4579: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:52:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:14 smithi167 ceph-mon[37140]: pgmap v4579: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:52:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:16 smithi167 ceph-mon[37140]: pgmap v4580: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:52:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:16 smithi167 ceph-mon[37140]: osdmap e335: 12 total, 12 up, 12 in 2024-07-26T15:52:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:16 smithi138 ceph-mon[36954]: pgmap v4580: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:52:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:16 smithi138 ceph-mon[36954]: osdmap e335: 12 total, 12 up, 12 in 2024-07-26T15:52:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:16 smithi100 ceph-mon[32599]: pgmap v4580: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:52:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:16 smithi100 ceph-mon[32599]: osdmap e335: 12 total, 12 up, 12 in 2024-07-26T15:52:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:18 smithi167 ceph-mon[37140]: pgmap v4582: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 220 B/s wr, 6 op/s 2024-07-26T15:52:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:18 smithi138 ceph-mon[36954]: pgmap v4582: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 220 B/s wr, 6 op/s 2024-07-26T15:52:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:18 smithi100 ceph-mon[32599]: pgmap v4582: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 220 B/s wr, 6 op/s 2024-07-26T15:52:19.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:52:19.995+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:52:19.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:52:19.995+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:52:19.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:52:19.995+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:52:19.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:52:19.995+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:52:19.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:52:19.995+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:52:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:20 smithi167 ceph-mon[37140]: pgmap v4583: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:52:21.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:20 smithi138 ceph-mon[36954]: pgmap v4583: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:52:21.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:20 smithi100 ceph-mon[32599]: pgmap v4583: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:52:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:22 smithi167 ceph-mon[37140]: pgmap v4584: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:52:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:22 smithi138 ceph-mon[36954]: pgmap v4584: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:52:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:22 smithi100 ceph-mon[32599]: pgmap v4584: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:52:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:24 smithi167 ceph-mon[37140]: pgmap v4585: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:52:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:24 smithi138 ceph-mon[36954]: pgmap v4585: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:52:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:24 smithi100 ceph-mon[32599]: pgmap v4585: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:52:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:26 smithi167 ceph-mon[37140]: pgmap v4586: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:52:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:26 smithi138 ceph-mon[36954]: pgmap v4586: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:52:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:26 smithi100 ceph-mon[32599]: pgmap v4586: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:52:29.180 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:28 smithi138 ceph-mon[36954]: pgmap v4587: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T15:52:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:28 smithi100 ceph-mon[32599]: pgmap v4587: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T15:52:29.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:28 smithi167 ceph-mon[37140]: pgmap v4587: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T15:52:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:30 smithi138 ceph-mon[36954]: pgmap v4588: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:52:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:30 smithi100 ceph-mon[32599]: pgmap v4588: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:52:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:30 smithi167 ceph-mon[37140]: pgmap v4588: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:52:33.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:32 smithi167 ceph-mon[37140]: pgmap v4589: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:52:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:32 smithi138 ceph-mon[36954]: pgmap v4589: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:52:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:32 smithi100 ceph-mon[32599]: pgmap v4589: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:52:35.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:52:34.999+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:52:35.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:52:35.000+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:52:35.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:52:35.000+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:52:35.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:52:35.000+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:52:35.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:52:35.000+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:52:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:34 smithi138 ceph-mon[36954]: pgmap v4590: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:52:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:34 smithi100 ceph-mon[32599]: pgmap v4590: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:52:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:34 smithi167 ceph-mon[37140]: pgmap v4590: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:52:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:36 smithi138 ceph-mon[36954]: pgmap v4591: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:52:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:36 smithi100 ceph-mon[32599]: pgmap v4591: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:52:37.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:36 smithi167 ceph-mon[37140]: pgmap v4591: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:52:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:38 smithi138 ceph-mon[36954]: pgmap v4592: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:52:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:38 smithi100 ceph-mon[32599]: pgmap v4592: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:52:39.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:38 smithi167 ceph-mon[37140]: pgmap v4592: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:52:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:40 smithi138 ceph-mon[36954]: pgmap v4593: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:52:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:40 smithi100 ceph-mon[32599]: pgmap v4593: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:52:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:40 smithi167 ceph-mon[37140]: pgmap v4593: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:52:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:42 smithi138 ceph-mon[36954]: pgmap v4594: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:52:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:42 smithi100 ceph-mon[32599]: pgmap v4594: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:52:43.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:42 smithi167 ceph-mon[37140]: pgmap v4594: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:52:45.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:44 smithi138 ceph-mon[36954]: pgmap v4595: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:52:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:44 smithi100 ceph-mon[32599]: pgmap v4595: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:52:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:44 smithi167 ceph-mon[37140]: pgmap v4595: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:52:46.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:45 smithi138 ceph-mon[36954]: pgmap v4596: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:52:46.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:45 smithi100 ceph-mon[32599]: pgmap v4596: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:52:46.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:45 smithi167 ceph-mon[37140]: pgmap v4596: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:52:48.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:48 smithi138 ceph-mon[36954]: pgmap v4597: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:52:48.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:48 smithi100 ceph-mon[32599]: pgmap v4597: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:52:48.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:48 smithi167 ceph-mon[37140]: pgmap v4597: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:52:50.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:52:50.005+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:52:50.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:52:50.006+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:52:50.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:52:50.006+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:52:50.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:52:50.006+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:52:50.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:52:50.006+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:52:50.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:50 smithi138 ceph-mon[36954]: pgmap v4598: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:52:50.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:50 smithi100 ceph-mon[32599]: pgmap v4598: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:52:50.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:50 smithi167 ceph-mon[37140]: pgmap v4598: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:52:52.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:52 smithi138 ceph-mon[36954]: pgmap v4599: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:52:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:52 smithi100 ceph-mon[32599]: pgmap v4599: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:52:52.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:52 smithi167 ceph-mon[37140]: pgmap v4599: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:52:54.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:54 smithi138 ceph-mon[36954]: pgmap v4600: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:52:54.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:54 smithi100 ceph-mon[32599]: pgmap v4600: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:52:54.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:54 smithi167 ceph-mon[37140]: pgmap v4600: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:52:56.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:56 smithi138 ceph-mon[36954]: pgmap v4601: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:52:56.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:56 smithi100 ceph-mon[32599]: pgmap v4601: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:52:56.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:56 smithi167 ceph-mon[37140]: pgmap v4601: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:52:58.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:52:58 smithi138 ceph-mon[36954]: pgmap v4602: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:52:58.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:52:58 smithi100 ceph-mon[32599]: pgmap v4602: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:52:58.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:52:58 smithi167 ceph-mon[37140]: pgmap v4602: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:53:00.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:53:00 smithi138 ceph-mon[36954]: pgmap v4603: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:53:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:53:00 smithi100 ceph-mon[32599]: pgmap v4603: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:53:00.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:53:00 smithi167 ceph-mon[37140]: pgmap v4603: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:53:02.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:53:02 smithi138 ceph-mon[36954]: pgmap v4604: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:53:02.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:53:02 smithi100 ceph-mon[32599]: pgmap v4604: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:53:02.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:53:02 smithi167 ceph-mon[37140]: pgmap v4604: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:53:03.424 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:53:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:53:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:53:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:53:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:53:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:53:04.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:53:04 smithi138 ceph-mon[36954]: pgmap v4605: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:53:04.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:53:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:53:04.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:53:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:53:04.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:53:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:53:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:53:04 smithi100 ceph-mon[32599]: pgmap v4605: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:53:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:53:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:53:04.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:53:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:53:04.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:53:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:53:04.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:53:04 smithi167 ceph-mon[37140]: pgmap v4605: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:53:04.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:53:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:53:04.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:53:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:53:04.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:53:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:53:05.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:53:05.015+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:53:05.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:53:05.016+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:53:05.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:53:05.016+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:53:05.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:53:05.016+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:53:05.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:53:05.016+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:53:06.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:53:06 smithi138 ceph-mon[36954]: pgmap v4606: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:53:06.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:53:06 smithi100 ceph-mon[32599]: pgmap v4606: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:53:06.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:53:06 smithi167 ceph-mon[37140]: pgmap v4606: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:53:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:53:08 smithi138 ceph-mon[36954]: pgmap v4607: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:53:08.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:53:08 smithi138 ceph-mon[36954]: osdmap e336: 12 total, 12 up, 12 in 2024-07-26T15:53:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:53:08 smithi100 ceph-mon[32599]: pgmap v4607: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:53:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:53:08 smithi100 ceph-mon[32599]: osdmap e336: 12 total, 12 up, 12 in 2024-07-26T15:53:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:53:08 smithi167 ceph-mon[37140]: pgmap v4607: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:53:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:53:08 smithi167 ceph-mon[37140]: osdmap e336: 12 total, 12 up, 12 in 2024-07-26T15:53:10.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:53:10 smithi138 ceph-mon[36954]: pgmap v4609: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:53:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:53:10 smithi100 ceph-mon[32599]: pgmap v4609: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:53:10.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:53:10 smithi167 ceph-mon[37140]: pgmap v4609: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:53:12.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:53:12 smithi138 ceph-mon[36954]: pgmap v4610: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:53:12.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:53:12 smithi100 ceph-mon[32599]: pgmap v4610: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:53:12.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:53:12 smithi167 ceph-mon[37140]: pgmap v4610: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:53:14.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:53:14 smithi138 ceph-mon[36954]: pgmap v4611: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:53:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:53:14 smithi100 ceph-mon[32599]: pgmap v4611: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:53:14.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:53:14 smithi167 ceph-mon[37140]: pgmap v4611: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:53:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:53:16 smithi167 ceph-mon[37140]: pgmap v4612: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:53:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:53:16 smithi167 ceph-mon[37140]: osdmap e337: 12 total, 12 up, 12 in 2024-07-26T15:53:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:53:16 smithi138 ceph-mon[36954]: pgmap v4612: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:53:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:53:16 smithi138 ceph-mon[36954]: osdmap e337: 12 total, 12 up, 12 in 2024-07-26T15:53:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:53:16 smithi100 ceph-mon[32599]: pgmap v4612: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:53:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:53:16 smithi100 ceph-mon[32599]: osdmap e337: 12 total, 12 up, 12 in 2024-07-26T15:53:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:53:18 smithi167 ceph-mon[37140]: pgmap v4614: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T15:53:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:53:18 smithi138 ceph-mon[36954]: pgmap v4614: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T15:53:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:53:18 smithi100 ceph-mon[32599]: pgmap v4614: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T15:53:20.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:53:20.017+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:53:20.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:53:20.018+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:53:20.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:53:20.018+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:53:20.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:53:20.018+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:53:20.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:53:20.018+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:53:21.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:53:20 smithi138 ceph-mon[36954]: pgmap v4615: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:53:21.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:53:20 smithi100 ceph-mon[32599]: pgmap v4615: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:53:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:53:20 smithi167 ceph-mon[37140]: pgmap v4615: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:53:23.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:53:22 smithi138 ceph-mon[36954]: pgmap v4616: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:53:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:53:22 smithi100 ceph-mon[32599]: pgmap v4616: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:53:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:53:22 smithi167 ceph-mon[37140]: pgmap v4616: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:53:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:53:24 smithi138 ceph-mon[36954]: pgmap v4617: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:53:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:53:24 smithi100 ceph-mon[32599]: pgmap v4617: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:53:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:53:24 smithi167 ceph-mon[37140]: pgmap v4617: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:53:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:53:26 smithi138 ceph-mon[36954]: pgmap v4618: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:53:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:53:26 smithi100 ceph-mon[32599]: pgmap v4618: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:53:27.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:53:26 smithi167 ceph-mon[37140]: pgmap v4618: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:53:29.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:53:28 smithi138 ceph-mon[36954]: pgmap v4619: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T15:53:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:53:28 smithi100 ceph-mon[32599]: pgmap v4619: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T15:53:29.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:53:28 smithi167 ceph-mon[37140]: pgmap v4619: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T15:53:31.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:53:30 smithi138 ceph-mon[36954]: pgmap v4620: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:53:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:53:30 smithi100 ceph-mon[32599]: pgmap v4620: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:53:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:53:30 smithi167 ceph-mon[37140]: pgmap v4620: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:53:33.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:53:32 smithi167 ceph-mon[37140]: pgmap v4621: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:53:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:53:32 smithi138 ceph-mon[36954]: pgmap v4621: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:53:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:53:32 smithi100 ceph-mon[32599]: pgmap v4621: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:53:35.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:53:35.033+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:53:35.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:53:35.034+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:53:35.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:53:35.034+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:53:35.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:53:35.034+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:53:35.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:53:35.034+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:53:35.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:53:34 smithi138 ceph-mon[36954]: pgmap v4622: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:53:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:53:34 smithi100 ceph-mon[32599]: pgmap v4622: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:53:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:53:34 smithi167 ceph-mon[37140]: pgmap v4622: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:53:37.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:53:36 smithi138 ceph-mon[36954]: pgmap v4623: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:53:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:53:36 smithi100 ceph-mon[32599]: pgmap v4623: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:53:37.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:53:36 smithi167 ceph-mon[37140]: pgmap v4623: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:53:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:53:38 smithi138 ceph-mon[36954]: pgmap v4624: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:53:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:53:38 smithi100 ceph-mon[32599]: pgmap v4624: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:53:39.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:53:38 smithi167 ceph-mon[37140]: pgmap v4624: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:53:41.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:53:40 smithi138 ceph-mon[36954]: pgmap v4625: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:53:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:53:40 smithi100 ceph-mon[32599]: pgmap v4625: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:53:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:53:40 smithi167 ceph-mon[37140]: pgmap v4625: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:53:43.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:53:42 smithi138 ceph-mon[36954]: pgmap v4626: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:53:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:53:42 smithi100 ceph-mon[32599]: pgmap v4626: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:53:43.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:53:42 smithi167 ceph-mon[37140]: pgmap v4626: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:53:44.146 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:53:43 smithi100 ceph-mon[32599]: pgmap v4627: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:53:44.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:53:43 smithi138 ceph-mon[36954]: pgmap v4627: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:53:44.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:53:43 smithi167 ceph-mon[37140]: pgmap v4627: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:53:46.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:53:46 smithi138 ceph-mon[36954]: pgmap v4628: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:53:46.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:53:46 smithi100 ceph-mon[32599]: pgmap v4628: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:53:46.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:53:46 smithi167 ceph-mon[37140]: pgmap v4628: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:53:48.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:53:48 smithi138 ceph-mon[36954]: pgmap v4629: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:53:48.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:53:48 smithi100 ceph-mon[32599]: pgmap v4629: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:53:48.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:53:48 smithi167 ceph-mon[37140]: pgmap v4629: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:53:50.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:53:50.037+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:53:50.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:53:50.038+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:53:50.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:53:50.038+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:53:50.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:53:50.038+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:53:50.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:53:50.038+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:53:50.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:53:50 smithi138 ceph-mon[36954]: pgmap v4630: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:53:50.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:53:50 smithi100 ceph-mon[32599]: pgmap v4630: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:53:50.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:53:50 smithi167 ceph-mon[37140]: pgmap v4630: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:53:52.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:53:52 smithi138 ceph-mon[36954]: pgmap v4631: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:53:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:53:52 smithi100 ceph-mon[32599]: pgmap v4631: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:53:52.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:53:52 smithi167 ceph-mon[37140]: pgmap v4631: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T15:53:54.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:53:54 smithi138 ceph-mon[36954]: pgmap v4632: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:53:54.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:53:54 smithi100 ceph-mon[32599]: pgmap v4632: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:53:54.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:53:54 smithi167 ceph-mon[37140]: pgmap v4632: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:53:56.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:53:56 smithi138 ceph-mon[36954]: pgmap v4633: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:53:56.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:53:56 smithi100 ceph-mon[32599]: pgmap v4633: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:53:56.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:53:56 smithi167 ceph-mon[37140]: pgmap v4633: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T15:53:58.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:53:58 smithi138 ceph-mon[36954]: pgmap v4634: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:53:58.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:53:58 smithi100 ceph-mon[32599]: pgmap v4634: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:53:58.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:53:58 smithi167 ceph-mon[37140]: pgmap v4634: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T15:54:00.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:54:00 smithi138 ceph-mon[36954]: pgmap v4635: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:54:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:54:00 smithi100 ceph-mon[32599]: pgmap v4635: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:54:00.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:54:00 smithi167 ceph-mon[37140]: pgmap v4635: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:54:02.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:54:02 smithi138 ceph-mon[36954]: pgmap v4636: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:54:02.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:54:02 smithi100 ceph-mon[32599]: pgmap v4636: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:54:02.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:54:02 smithi167 ceph-mon[37140]: pgmap v4636: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:54:04.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:54:04.083+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T15:54:04.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:54:04.083+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c004740 con 0x7fe5a81087a0 2024-07-26T15:54:04.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:54:04.083+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 62 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c100 con 0x7fe5a81087a0 2024-07-26T15:54:04.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:54:04.084+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 62 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T15:54:04.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:54:04 smithi138 ceph-mon[36954]: pgmap v4637: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:54:04.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:54:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:54:04.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:54:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:54:04.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:54:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:54:04.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:54:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:54:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:54:04 smithi100 ceph-mon[32599]: pgmap v4637: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:54:04.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:54:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:54:04.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:54:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:54:04.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:54:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:54:04.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:54:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:54:04.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:54:04 smithi167 ceph-mon[37140]: pgmap v4637: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:54:04.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:54:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:54:04.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:54:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:54:04.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:54:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:54:04.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:54:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:54:05.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:54:05.049+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:54:05.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:54:05.050+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:54:05.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:54:05.050+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:54:05.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:54:05.050+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:54:05.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:54:05.050+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:54:06.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:54:06 smithi138 ceph-mon[36954]: pgmap v4638: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:54:06.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:54:06 smithi100 ceph-mon[32599]: pgmap v4638: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:54:06.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:54:06 smithi167 ceph-mon[37140]: pgmap v4638: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:54:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:54:08 smithi138 ceph-mon[36954]: pgmap v4639: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:54:08.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:54:08 smithi138 ceph-mon[36954]: osdmap e338: 12 total, 12 up, 12 in 2024-07-26T15:54:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:54:08 smithi100 ceph-mon[32599]: pgmap v4639: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:54:08.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:54:08 smithi100 ceph-mon[32599]: osdmap e338: 12 total, 12 up, 12 in 2024-07-26T15:54:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:54:08 smithi167 ceph-mon[37140]: pgmap v4639: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:54:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:54:08 smithi167 ceph-mon[37140]: osdmap e338: 12 total, 12 up, 12 in 2024-07-26T15:54:10.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:54:10 smithi138 ceph-mon[36954]: pgmap v4641: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:54:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:54:10 smithi100 ceph-mon[32599]: pgmap v4641: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:54:10.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:54:10 smithi167 ceph-mon[37140]: pgmap v4641: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:54:12.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:54:12 smithi138 ceph-mon[36954]: pgmap v4642: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:54:12.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:54:12 smithi100 ceph-mon[32599]: pgmap v4642: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:54:12.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:54:12 smithi167 ceph-mon[37140]: pgmap v4642: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:54:14.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:54:14 smithi138 ceph-mon[36954]: pgmap v4643: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:54:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:54:14 smithi100 ceph-mon[32599]: pgmap v4643: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:54:14.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:54:14 smithi167 ceph-mon[37140]: pgmap v4643: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:54:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:54:16 smithi138 ceph-mon[36954]: pgmap v4644: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:54:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:54:16 smithi138 ceph-mon[36954]: osdmap e339: 12 total, 12 up, 12 in 2024-07-26T15:54:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:54:16 smithi100 ceph-mon[32599]: pgmap v4644: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:54:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:54:16 smithi100 ceph-mon[32599]: osdmap e339: 12 total, 12 up, 12 in 2024-07-26T15:54:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:54:16 smithi167 ceph-mon[37140]: pgmap v4644: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:54:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:54:16 smithi167 ceph-mon[37140]: osdmap e339: 12 total, 12 up, 12 in 2024-07-26T15:54:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:54:18 smithi138 ceph-mon[36954]: pgmap v4646: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T15:54:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:54:18 smithi100 ceph-mon[32599]: pgmap v4646: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T15:54:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:54:18 smithi167 ceph-mon[37140]: pgmap v4646: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T15:54:20.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:54:20.055+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:54:20.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:54:20.056+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:54:20.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:54:20.056+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:54:20.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:54:20.056+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:54:20.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:54:20.056+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:54:21.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:54:20 smithi138 ceph-mon[36954]: pgmap v4647: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:54:21.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:54:20 smithi100 ceph-mon[32599]: pgmap v4647: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:54:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:54:20 smithi167 ceph-mon[37140]: pgmap v4647: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:54:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:54:22 smithi138 ceph-mon[36954]: pgmap v4648: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:54:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:54:22 smithi100 ceph-mon[32599]: pgmap v4648: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:54:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:54:22 smithi167 ceph-mon[37140]: pgmap v4648: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:54:25.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:54:24 smithi138 ceph-mon[36954]: pgmap v4649: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:54:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:54:24 smithi100 ceph-mon[32599]: pgmap v4649: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:54:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:54:24 smithi167 ceph-mon[37140]: pgmap v4649: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:54:27.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:54:26 smithi138 ceph-mon[36954]: pgmap v4650: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:54:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:54:26 smithi100 ceph-mon[32599]: pgmap v4650: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:54:27.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:54:26 smithi167 ceph-mon[37140]: pgmap v4650: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:54:29.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:54:28 smithi138 ceph-mon[36954]: pgmap v4651: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T15:54:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:54:28 smithi100 ceph-mon[32599]: pgmap v4651: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T15:54:29.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:54:28 smithi167 ceph-mon[37140]: pgmap v4651: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T15:54:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:54:30 smithi138 ceph-mon[36954]: pgmap v4652: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:54:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:54:30 smithi100 ceph-mon[32599]: pgmap v4652: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:54:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:54:30 smithi167 ceph-mon[37140]: pgmap v4652: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:54:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:54:32 smithi138 ceph-mon[36954]: pgmap v4653: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:54:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:54:32 smithi100 ceph-mon[32599]: pgmap v4653: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:54:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:54:32 smithi167 ceph-mon[37140]: pgmap v4653: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:54:35.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:54:35.071+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:54:35.074 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:54:34 smithi100 ceph-mon[32599]: pgmap v4654: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:54:35.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:54:35.072+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:54:35.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:54:35.072+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:54:35.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:54:35.072+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:54:35.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:54:35.072+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:54:35.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:54:34 smithi167 ceph-mon[37140]: pgmap v4654: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:54:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:54:34 smithi138 ceph-mon[36954]: pgmap v4654: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:54:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:54:36 smithi138 ceph-mon[36954]: pgmap v4655: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:54:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:54:36 smithi100 ceph-mon[32599]: pgmap v4655: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:54:37.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:54:36 smithi167 ceph-mon[37140]: pgmap v4655: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:54:39.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:54:38 smithi138 ceph-mon[36954]: pgmap v4656: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:54:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:54:38 smithi100 ceph-mon[32599]: pgmap v4656: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:54:39.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:54:38 smithi167 ceph-mon[37140]: pgmap v4656: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:54:40.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:54:39 smithi138 ceph-mon[36954]: pgmap v4657: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:54:40.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:54:39 smithi100 ceph-mon[32599]: pgmap v4657: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:54:40.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:54:39 smithi167 ceph-mon[37140]: pgmap v4657: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:54:42.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:54:42 smithi138 ceph-mon[36954]: pgmap v4658: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:54:42.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:54:42 smithi100 ceph-mon[32599]: pgmap v4658: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:54:42.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:54:42 smithi167 ceph-mon[37140]: pgmap v4658: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:54:44.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:54:44 smithi138 ceph-mon[36954]: pgmap v4659: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:54:44.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:54:44 smithi100 ceph-mon[32599]: pgmap v4659: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:54:44.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:54:44 smithi167 ceph-mon[37140]: pgmap v4659: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:54:46.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:54:46 smithi138 ceph-mon[36954]: pgmap v4660: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:54:46.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:54:46 smithi100 ceph-mon[32599]: pgmap v4660: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:54:46.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:54:46 smithi167 ceph-mon[37140]: pgmap v4660: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:54:48.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:54:48 smithi138 ceph-mon[36954]: pgmap v4661: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:54:48.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:54:48 smithi100 ceph-mon[32599]: pgmap v4661: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:54:48.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:54:48 smithi167 ceph-mon[37140]: pgmap v4661: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:54:50.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:54:50.089+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:54:50.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:54:50.091+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:54:50.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:54:50.091+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:54:50.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:54:50.091+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:54:50.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:54:50.091+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:54:50.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:54:50 smithi138 ceph-mon[36954]: pgmap v4662: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:54:50.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:54:50 smithi100 ceph-mon[32599]: pgmap v4662: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:54:50.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:54:50 smithi167 ceph-mon[37140]: pgmap v4662: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:54:52.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:54:52 smithi138 ceph-mon[36954]: pgmap v4663: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:54:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:54:52 smithi100 ceph-mon[32599]: pgmap v4663: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:54:52.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:54:52 smithi167 ceph-mon[37140]: pgmap v4663: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:54:54.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:54:54 smithi138 ceph-mon[36954]: pgmap v4664: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:54:54.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:54:54 smithi100 ceph-mon[32599]: pgmap v4664: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:54:54.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:54:54 smithi167 ceph-mon[37140]: pgmap v4664: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:54:56.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:54:56 smithi138 ceph-mon[36954]: pgmap v4665: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:54:56.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:54:56 smithi100 ceph-mon[32599]: pgmap v4665: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:54:56.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:54:56 smithi167 ceph-mon[37140]: pgmap v4665: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:54:58.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:54:58 smithi138 ceph-mon[36954]: pgmap v4666: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:54:58.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:54:58 smithi100 ceph-mon[32599]: pgmap v4666: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:54:58.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:54:58 smithi167 ceph-mon[37140]: pgmap v4666: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:55:00.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:55:00 smithi138 ceph-mon[36954]: pgmap v4667: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:55:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:55:00 smithi100 ceph-mon[32599]: pgmap v4667: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:55:00.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:55:00 smithi167 ceph-mon[37140]: pgmap v4667: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:55:02.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:55:02 smithi138 ceph-mon[36954]: pgmap v4668: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:55:02.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:55:02 smithi100 ceph-mon[32599]: pgmap v4668: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:55:02.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:55:02 smithi167 ceph-mon[37140]: pgmap v4668: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:55:04.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:55:04 smithi138 ceph-mon[36954]: pgmap v4669: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:55:04.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:55:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:55:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:55:04 smithi100 ceph-mon[32599]: pgmap v4669: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:55:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:55:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:55:04.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:55:04 smithi167 ceph-mon[37140]: pgmap v4669: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:55:04.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:55:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:55:05.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:55:05.103+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:55:05.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:55:05.103+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:55:05.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:55:05.103+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:55:05.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:55:05.103+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:55:05.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:55:05.103+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:55:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:55:05 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:55:05.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:55:05 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:55:05.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:55:05 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:55:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:55:05 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:55:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:55:05 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:55:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:55:05 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:55:05.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:55:05 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:55:05.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:55:05 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:55:05.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:55:05 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:55:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:55:06 smithi100 ceph-mon[32599]: pgmap v4670: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:55:06.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:55:06 smithi167 ceph-mon[37140]: pgmap v4670: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:55:06.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:55:06 smithi138 ceph-mon[36954]: pgmap v4670: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:55:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:55:08 smithi100 ceph-mon[32599]: pgmap v4671: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:55:08.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:55:08 smithi100 ceph-mon[32599]: osdmap e340: 12 total, 12 up, 12 in 2024-07-26T15:55:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:55:08 smithi167 ceph-mon[37140]: pgmap v4671: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:55:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:55:08 smithi167 ceph-mon[37140]: osdmap e340: 12 total, 12 up, 12 in 2024-07-26T15:55:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:55:08 smithi138 ceph-mon[36954]: pgmap v4671: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:55:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:55:08 smithi138 ceph-mon[36954]: osdmap e340: 12 total, 12 up, 12 in 2024-07-26T15:55:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:55:10 smithi100 ceph-mon[32599]: pgmap v4673: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:55:10.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:55:10 smithi167 ceph-mon[37140]: pgmap v4673: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:55:10.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:55:10 smithi138 ceph-mon[36954]: pgmap v4673: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:55:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:55:12 smithi100 ceph-mon[32599]: pgmap v4674: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:55:12.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:55:12 smithi167 ceph-mon[37140]: pgmap v4674: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:55:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:55:12 smithi138 ceph-mon[36954]: pgmap v4674: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:55:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:55:14 smithi100 ceph-mon[32599]: pgmap v4675: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:55:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:55:14 smithi167 ceph-mon[37140]: pgmap v4675: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:55:14.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:55:14 smithi138 ceph-mon[36954]: pgmap v4675: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:55:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:55:16 smithi138 ceph-mon[36954]: pgmap v4676: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:55:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:55:16 smithi138 ceph-mon[36954]: osdmap e341: 12 total, 12 up, 12 in 2024-07-26T15:55:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:55:16 smithi100 ceph-mon[32599]: pgmap v4676: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:55:17.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:55:16 smithi100 ceph-mon[32599]: osdmap e341: 12 total, 12 up, 12 in 2024-07-26T15:55:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:55:16 smithi167 ceph-mon[37140]: pgmap v4676: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:55:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:55:16 smithi167 ceph-mon[37140]: osdmap e341: 12 total, 12 up, 12 in 2024-07-26T15:55:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:55:18 smithi138 ceph-mon[36954]: pgmap v4678: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 212 B/s wr, 6 op/s 2024-07-26T15:55:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:55:18 smithi100 ceph-mon[32599]: pgmap v4678: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 212 B/s wr, 6 op/s 2024-07-26T15:55:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:55:18 smithi167 ceph-mon[37140]: pgmap v4678: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 212 B/s wr, 6 op/s 2024-07-26T15:55:20.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:55:20.110+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:55:20.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:55:20.111+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:55:20.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:55:20.111+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:55:20.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:55:20.111+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:55:20.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:55:20.111+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:55:21.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:55:20 smithi138 ceph-mon[36954]: pgmap v4679: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:55:21.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:55:20 smithi100 ceph-mon[32599]: pgmap v4679: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:55:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:55:20 smithi167 ceph-mon[37140]: pgmap v4679: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:55:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:55:22 smithi138 ceph-mon[36954]: pgmap v4680: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:55:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:55:22 smithi100 ceph-mon[32599]: pgmap v4680: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:55:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:55:22 smithi167 ceph-mon[37140]: pgmap v4680: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:55:25.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:55:24 smithi138 ceph-mon[36954]: pgmap v4681: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:55:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:55:24 smithi100 ceph-mon[32599]: pgmap v4681: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:55:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:55:24 smithi167 ceph-mon[37140]: pgmap v4681: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:55:27.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:55:26 smithi138 ceph-mon[36954]: pgmap v4682: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:55:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:55:26 smithi100 ceph-mon[32599]: pgmap v4682: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:55:27.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:55:26 smithi167 ceph-mon[37140]: pgmap v4682: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:55:29.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:55:28 smithi138 ceph-mon[36954]: pgmap v4683: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T15:55:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:55:28 smithi100 ceph-mon[32599]: pgmap v4683: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T15:55:29.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:55:28 smithi167 ceph-mon[37140]: pgmap v4683: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T15:55:31.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:55:30 smithi138 ceph-mon[36954]: pgmap v4684: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:55:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:55:30 smithi100 ceph-mon[32599]: pgmap v4684: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:55:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:55:30 smithi167 ceph-mon[37140]: pgmap v4684: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:55:33.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:55:32 smithi138 ceph-mon[36954]: pgmap v4685: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:55:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:55:32 smithi100 ceph-mon[32599]: pgmap v4685: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:55:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:55:32 smithi167 ceph-mon[37140]: pgmap v4685: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:55:34.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:55:33 smithi138 ceph-mon[36954]: pgmap v4686: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:55:34.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:55:33 smithi100 ceph-mon[32599]: pgmap v4686: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:55:34.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:55:33 smithi167 ceph-mon[37140]: pgmap v4686: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:55:35.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:55:35.127+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:55:35.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:55:35.127+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:55:35.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:55:35.127+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:55:35.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:55:35.127+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:55:35.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:55:35.127+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:55:36.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:55:36 smithi138 ceph-mon[36954]: pgmap v4687: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:55:36.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:55:36 smithi100 ceph-mon[32599]: pgmap v4687: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:55:36.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:55:36 smithi167 ceph-mon[37140]: pgmap v4687: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:55:38.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:55:38 smithi138 ceph-mon[36954]: pgmap v4688: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:55:38.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:55:38 smithi100 ceph-mon[32599]: pgmap v4688: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:55:38.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:55:38 smithi167 ceph-mon[37140]: pgmap v4688: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:55:40.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:55:40 smithi138 ceph-mon[36954]: pgmap v4689: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:55:40.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:55:40 smithi100 ceph-mon[32599]: pgmap v4689: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:55:40.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:55:40 smithi167 ceph-mon[37140]: pgmap v4689: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:55:42.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:55:42 smithi138 ceph-mon[36954]: pgmap v4690: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:55:42.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:55:42 smithi100 ceph-mon[32599]: pgmap v4690: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:55:42.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:55:42 smithi167 ceph-mon[37140]: pgmap v4690: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:55:44.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:55:44 smithi138 ceph-mon[36954]: pgmap v4691: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:55:44.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:55:44 smithi100 ceph-mon[32599]: pgmap v4691: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:55:44.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:55:44 smithi167 ceph-mon[37140]: pgmap v4691: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:55:46.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:55:46 smithi138 ceph-mon[36954]: pgmap v4692: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:55:46.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:55:46 smithi100 ceph-mon[32599]: pgmap v4692: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:55:46.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:55:46 smithi167 ceph-mon[37140]: pgmap v4692: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:55:48.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:55:48 smithi138 ceph-mon[36954]: pgmap v4693: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:55:48.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:55:48 smithi100 ceph-mon[32599]: pgmap v4693: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:55:48.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:55:48 smithi167 ceph-mon[37140]: pgmap v4693: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:55:50.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:55:50.135+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:55:50.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:55:50.135+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:55:50.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:55:50.135+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:55:50.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:55:50.135+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:55:50.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:55:50.135+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:55:50.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:55:50 smithi138 ceph-mon[36954]: pgmap v4694: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:55:50.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:55:50 smithi100 ceph-mon[32599]: pgmap v4694: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:55:50.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:55:50 smithi167 ceph-mon[37140]: pgmap v4694: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:55:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:55:52 smithi100 ceph-mon[32599]: pgmap v4695: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:55:52.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:55:52 smithi167 ceph-mon[37140]: pgmap v4695: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:55:52.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:55:52 smithi138 ceph-mon[36954]: pgmap v4695: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:55:54.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:55:54 smithi100 ceph-mon[32599]: pgmap v4696: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:55:54.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:55:54 smithi167 ceph-mon[37140]: pgmap v4696: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:55:54.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:55:54 smithi138 ceph-mon[36954]: pgmap v4696: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:55:56.493 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:55:56 smithi138 ceph-mon[36954]: pgmap v4697: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:55:56.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:55:56 smithi100 ceph-mon[32599]: pgmap v4697: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:55:56.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:55:56 smithi167 ceph-mon[37140]: pgmap v4697: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:55:58.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:55:58 smithi100 ceph-mon[32599]: pgmap v4698: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:55:58.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:55:58 smithi167 ceph-mon[37140]: pgmap v4698: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:55:58.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:55:58 smithi138 ceph-mon[36954]: pgmap v4698: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:56:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:56:00 smithi100 ceph-mon[32599]: pgmap v4699: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:56:00.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:56:00 smithi167 ceph-mon[37140]: pgmap v4699: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:56:00.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:56:00 smithi138 ceph-mon[36954]: pgmap v4699: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:56:02.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:56:02 smithi100 ceph-mon[32599]: pgmap v4700: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:56:02.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:56:02 smithi167 ceph-mon[37140]: pgmap v4700: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:56:02.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:56:02 smithi138 ceph-mon[36954]: pgmap v4700: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:56:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:56:04 smithi100 ceph-mon[32599]: pgmap v4701: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:56:04.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:56:04 smithi167 ceph-mon[37140]: pgmap v4701: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:56:04.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:56:04 smithi138 ceph-mon[36954]: pgmap v4701: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:56:05.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:56:05.151+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:56:05.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:56:05.152+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:56:05.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:56:05.152+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:56:05.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:56:05.152+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:56:05.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:56:05.152+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:56:05.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:56:05 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:56:05.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:56:05 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:56:05.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:56:05 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:56:05.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:56:05 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:56:05.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:56:05 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:56:05.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:56:05 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:56:05.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:56:05 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:56:05.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:56:05 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:56:05.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:56:05 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:56:05.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:56:05 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:56:05.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:56:05 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:56:05.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:56:05 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:56:06.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:56:06 smithi100 ceph-mon[32599]: pgmap v4702: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:56:06.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:56:06 smithi167 ceph-mon[37140]: pgmap v4702: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:56:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:56:06 smithi138 ceph-mon[36954]: pgmap v4702: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:56:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:56:08 smithi100 ceph-mon[32599]: pgmap v4703: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:56:08.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:56:08 smithi100 ceph-mon[32599]: osdmap e342: 12 total, 12 up, 12 in 2024-07-26T15:56:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:56:08 smithi167 ceph-mon[37140]: pgmap v4703: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:56:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:56:08 smithi167 ceph-mon[37140]: osdmap e342: 12 total, 12 up, 12 in 2024-07-26T15:56:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:56:08 smithi138 ceph-mon[36954]: pgmap v4703: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:56:08.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:56:08 smithi138 ceph-mon[36954]: osdmap e342: 12 total, 12 up, 12 in 2024-07-26T15:56:10.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:56:10 smithi100 ceph-mon[32599]: pgmap v4705: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:56:10.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:56:10 smithi167 ceph-mon[37140]: pgmap v4705: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:56:10.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:56:10 smithi138 ceph-mon[36954]: pgmap v4705: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:56:12.498 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:56:12 smithi100 ceph-mon[32599]: pgmap v4706: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:56:12.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:56:12 smithi167 ceph-mon[37140]: pgmap v4706: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:56:12.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:56:12 smithi138 ceph-mon[36954]: pgmap v4706: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:56:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:56:14 smithi100 ceph-mon[32599]: pgmap v4707: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:56:14.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:56:14 smithi167 ceph-mon[37140]: pgmap v4707: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:56:14.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:56:14 smithi138 ceph-mon[36954]: pgmap v4707: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:56:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:56:16 smithi138 ceph-mon[36954]: pgmap v4708: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:56:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:56:16 smithi138 ceph-mon[36954]: osdmap e343: 12 total, 12 up, 12 in 2024-07-26T15:56:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:56:16 smithi100 ceph-mon[32599]: pgmap v4708: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:56:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:56:16 smithi100 ceph-mon[32599]: osdmap e343: 12 total, 12 up, 12 in 2024-07-26T15:56:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:56:16 smithi167 ceph-mon[37140]: pgmap v4708: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:56:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:56:16 smithi167 ceph-mon[37140]: osdmap e343: 12 total, 12 up, 12 in 2024-07-26T15:56:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:56:18 smithi138 ceph-mon[36954]: pgmap v4710: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T15:56:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:56:18 smithi100 ceph-mon[32599]: pgmap v4710: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T15:56:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:56:18 smithi167 ceph-mon[37140]: pgmap v4710: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T15:56:20.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:56:20.167+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:56:20.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:56:20.168+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:56:20.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:56:20.168+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:56:20.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:56:20.168+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:56:20.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:56:20.168+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:56:21.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:56:20 smithi138 ceph-mon[36954]: pgmap v4711: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:56:21.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:56:20 smithi100 ceph-mon[32599]: pgmap v4711: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:56:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:56:20 smithi167 ceph-mon[37140]: pgmap v4711: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:56:23.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:56:22 smithi138 ceph-mon[36954]: pgmap v4712: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:56:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:56:22 smithi100 ceph-mon[32599]: pgmap v4712: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:56:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:56:22 smithi167 ceph-mon[37140]: pgmap v4712: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:56:25.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:56:24 smithi138 ceph-mon[36954]: pgmap v4713: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:56:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:56:24 smithi100 ceph-mon[32599]: pgmap v4713: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:56:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:56:24 smithi167 ceph-mon[37140]: pgmap v4713: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:56:27.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:56:26 smithi138 ceph-mon[36954]: pgmap v4714: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:56:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:56:26 smithi100 ceph-mon[32599]: pgmap v4714: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:56:27.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:56:26 smithi167 ceph-mon[37140]: pgmap v4714: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:56:29.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:56:28 smithi138 ceph-mon[36954]: pgmap v4715: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T15:56:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:56:28 smithi100 ceph-mon[32599]: pgmap v4715: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T15:56:29.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:56:28 smithi167 ceph-mon[37140]: pgmap v4715: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T15:56:31.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:56:30 smithi138 ceph-mon[36954]: pgmap v4716: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:56:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:56:30 smithi100 ceph-mon[32599]: pgmap v4716: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:56:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:56:30 smithi167 ceph-mon[37140]: pgmap v4716: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:56:32.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:56:31 smithi138 ceph-mon[36954]: pgmap v4717: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:56:32.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:56:31 smithi100 ceph-mon[32599]: pgmap v4717: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:56:32.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:56:31 smithi167 ceph-mon[37140]: pgmap v4717: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:56:34.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:56:34 smithi138 ceph-mon[36954]: pgmap v4718: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:56:34.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:56:34 smithi100 ceph-mon[32599]: pgmap v4718: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:56:34.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:56:34 smithi167 ceph-mon[37140]: pgmap v4718: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:56:35.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:56:35.183+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:56:35.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:56:35.183+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:56:35.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:56:35.183+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:56:35.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:56:35.183+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:56:35.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:56:35.183+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:56:36.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:56:36 smithi138 ceph-mon[36954]: pgmap v4719: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:56:36.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:56:36 smithi100 ceph-mon[32599]: pgmap v4719: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:56:36.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:56:36 smithi167 ceph-mon[37140]: pgmap v4719: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:56:38.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:56:38 smithi138 ceph-mon[36954]: pgmap v4720: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:56:38.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:56:38 smithi100 ceph-mon[32599]: pgmap v4720: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:56:38.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:56:38 smithi167 ceph-mon[37140]: pgmap v4720: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:56:40.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:56:40 smithi100 ceph-mon[32599]: pgmap v4721: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:56:40.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:56:40 smithi167 ceph-mon[37140]: pgmap v4721: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:56:40.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:56:40 smithi138 ceph-mon[36954]: pgmap v4721: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:56:42.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:56:42 smithi100 ceph-mon[32599]: pgmap v4722: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:56:42.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:56:42 smithi167 ceph-mon[37140]: pgmap v4722: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:56:42.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:56:42 smithi138 ceph-mon[36954]: pgmap v4722: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:56:44.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:56:44 smithi100 ceph-mon[32599]: pgmap v4723: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:56:44.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:56:44 smithi167 ceph-mon[37140]: pgmap v4723: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:56:44.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:56:44 smithi138 ceph-mon[36954]: pgmap v4723: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:56:46.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:56:46 smithi100 ceph-mon[32599]: pgmap v4724: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:56:46.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:56:46 smithi167 ceph-mon[37140]: pgmap v4724: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:56:46.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:56:46 smithi138 ceph-mon[36954]: pgmap v4724: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:56:48.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:56:48 smithi100 ceph-mon[32599]: pgmap v4725: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:56:48.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:56:48 smithi167 ceph-mon[37140]: pgmap v4725: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:56:48.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:56:48 smithi138 ceph-mon[36954]: pgmap v4725: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:56:50.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:56:50.199+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:56:50.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:56:50.199+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:56:50.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:56:50.199+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:56:50.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:56:50.199+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:56:50.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:56:50.199+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:56:50.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:56:50 smithi100 ceph-mon[32599]: pgmap v4726: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:56:50.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:56:50 smithi167 ceph-mon[37140]: pgmap v4726: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:56:50.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:56:50 smithi138 ceph-mon[36954]: pgmap v4726: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:56:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:56:52 smithi100 ceph-mon[32599]: pgmap v4727: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:56:52.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:56:52 smithi167 ceph-mon[37140]: pgmap v4727: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:56:52.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:56:52 smithi138 ceph-mon[36954]: pgmap v4727: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:56:54.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:56:54 smithi100 ceph-mon[32599]: pgmap v4728: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:56:54.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:56:54 smithi167 ceph-mon[37140]: pgmap v4728: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:56:54.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:56:54 smithi138 ceph-mon[36954]: pgmap v4728: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:56:56.493 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:56:56 smithi138 ceph-mon[36954]: pgmap v4729: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:56:56.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:56:56 smithi100 ceph-mon[32599]: pgmap v4729: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:56:56.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:56:56 smithi167 ceph-mon[37140]: pgmap v4729: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:56:58.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:56:58 smithi100 ceph-mon[32599]: pgmap v4730: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:56:58.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:56:58 smithi167 ceph-mon[37140]: pgmap v4730: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:56:58.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:56:58 smithi138 ceph-mon[36954]: pgmap v4730: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:57:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:57:00 smithi100 ceph-mon[32599]: pgmap v4731: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:57:00.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:57:00 smithi167 ceph-mon[37140]: pgmap v4731: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:57:00.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:57:00 smithi138 ceph-mon[36954]: pgmap v4731: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:57:02.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:57:02 smithi100 ceph-mon[32599]: pgmap v4732: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:57:02.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:57:02 smithi167 ceph-mon[37140]: pgmap v4732: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:57:02.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:57:02 smithi138 ceph-mon[36954]: pgmap v4732: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:57:04.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:57:04.109+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c004740 con 0x7fe5a81087a0 2024-07-26T15:57:04.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:57:04.109+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T15:57:04.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:57:04.109+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 63 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c070 con 0x7fe5a81087a0 2024-07-26T15:57:04.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:57:04.109+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 63 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T15:57:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:57:04 smithi100 ceph-mon[32599]: pgmap v4733: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:57:04.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:57:04 smithi167 ceph-mon[37140]: pgmap v4733: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:57:04.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:57:04 smithi138 ceph-mon[36954]: pgmap v4733: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:57:05.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:57:05.206+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:57:05.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:57:05.206+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:57:05.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:57:05.206+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:57:05.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:57:05.206+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:57:05.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:57:05.206+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:57:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:57:05 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:57:05.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:57:05 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:57:05.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:57:05 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:57:06.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:57:06 smithi100 ceph-mon[32599]: pgmap v4734: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:57:06.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:57:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:57:06.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:57:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:57:06.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:57:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:57:06.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:57:06 smithi167 ceph-mon[37140]: pgmap v4734: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:57:06.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:57:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:57:06.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:57:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:57:06.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:57:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:57:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:57:06 smithi138 ceph-mon[36954]: pgmap v4734: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:57:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:57:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:57:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:57:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:57:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:57:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:57:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:57:08 smithi167 ceph-mon[37140]: pgmap v4735: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:57:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:57:08 smithi167 ceph-mon[37140]: osdmap e344: 12 total, 12 up, 12 in 2024-07-26T15:57:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:57:08 smithi138 ceph-mon[36954]: pgmap v4735: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:57:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:57:08 smithi138 ceph-mon[36954]: osdmap e344: 12 total, 12 up, 12 in 2024-07-26T15:57:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:57:08 smithi100 ceph-mon[32599]: pgmap v4735: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:57:08.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:57:08 smithi100 ceph-mon[32599]: osdmap e344: 12 total, 12 up, 12 in 2024-07-26T15:57:10.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:57:10 smithi167 ceph-mon[37140]: pgmap v4737: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:57:10.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:57:10 smithi138 ceph-mon[36954]: pgmap v4737: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:57:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:57:10 smithi100 ceph-mon[32599]: pgmap v4737: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:57:12.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:57:12 smithi167 ceph-mon[37140]: pgmap v4738: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:57:12.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:57:12 smithi138 ceph-mon[36954]: pgmap v4738: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:57:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:57:12 smithi100 ceph-mon[32599]: pgmap v4738: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:57:14.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:57:14 smithi167 ceph-mon[37140]: pgmap v4739: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:57:14.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:57:14 smithi100 ceph-mon[32599]: pgmap v4739: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:57:14.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:57:14 smithi138 ceph-mon[36954]: pgmap v4739: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:57:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:57:16 smithi138 ceph-mon[36954]: pgmap v4740: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:57:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:57:16 smithi138 ceph-mon[36954]: osdmap e345: 12 total, 12 up, 12 in 2024-07-26T15:57:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:57:16 smithi100 ceph-mon[32599]: pgmap v4740: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:57:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:57:16 smithi100 ceph-mon[32599]: osdmap e345: 12 total, 12 up, 12 in 2024-07-26T15:57:17.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:57:16 smithi167 ceph-mon[37140]: pgmap v4740: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:57:17.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:57:16 smithi167 ceph-mon[37140]: osdmap e345: 12 total, 12 up, 12 in 2024-07-26T15:57:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:57:18 smithi138 ceph-mon[36954]: pgmap v4742: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T15:57:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:57:18 smithi100 ceph-mon[32599]: pgmap v4742: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T15:57:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:57:18 smithi167 ceph-mon[37140]: pgmap v4742: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T15:57:20.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:57:20.222+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:57:20.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:57:20.222+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:57:20.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:57:20.222+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:57:20.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:57:20.222+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:57:20.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:57:20.222+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:57:21.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:57:20 smithi138 ceph-mon[36954]: pgmap v4743: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:57:21.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:57:20 smithi100 ceph-mon[32599]: pgmap v4743: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:57:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:57:20 smithi167 ceph-mon[37140]: pgmap v4743: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:57:23.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:57:22 smithi138 ceph-mon[36954]: pgmap v4744: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:57:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:57:22 smithi100 ceph-mon[32599]: pgmap v4744: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:57:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:57:22 smithi167 ceph-mon[37140]: pgmap v4744: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:57:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:57:24 smithi138 ceph-mon[36954]: pgmap v4745: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:57:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:57:24 smithi100 ceph-mon[32599]: pgmap v4745: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:57:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:57:24 smithi167 ceph-mon[37140]: pgmap v4745: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:57:27.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:57:26 smithi138 ceph-mon[36954]: pgmap v4746: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:57:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:57:26 smithi100 ceph-mon[32599]: pgmap v4746: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:57:27.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:57:26 smithi167 ceph-mon[37140]: pgmap v4746: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:57:28.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:57:27 smithi138 ceph-mon[36954]: pgmap v4747: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T15:57:28.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:57:27 smithi100 ceph-mon[32599]: pgmap v4747: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T15:57:28.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:57:27 smithi167 ceph-mon[37140]: pgmap v4747: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T15:57:30.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:57:30 smithi100 ceph-mon[32599]: pgmap v4748: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:57:30.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:57:30 smithi167 ceph-mon[37140]: pgmap v4748: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:57:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:57:30 smithi138 ceph-mon[36954]: pgmap v4748: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:57:32.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:57:32 smithi100 ceph-mon[32599]: pgmap v4749: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:57:32.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:57:32 smithi167 ceph-mon[37140]: pgmap v4749: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:57:32.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:57:32 smithi138 ceph-mon[36954]: pgmap v4749: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:57:34.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:57:34 smithi100 ceph-mon[32599]: pgmap v4750: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:57:34.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:57:34 smithi167 ceph-mon[37140]: pgmap v4750: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:57:34.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:57:34 smithi138 ceph-mon[36954]: pgmap v4750: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:57:35.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:57:35.238+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:57:35.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:57:35.239+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:57:35.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:57:35.239+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:57:35.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:57:35.239+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:57:35.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:57:35.239+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:57:36.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:57:36 smithi100 ceph-mon[32599]: pgmap v4751: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:57:36.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:57:36 smithi167 ceph-mon[37140]: pgmap v4751: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:57:36.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:57:36 smithi138 ceph-mon[36954]: pgmap v4751: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:57:38.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:57:38 smithi100 ceph-mon[32599]: pgmap v4752: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:57:38.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:57:38 smithi167 ceph-mon[37140]: pgmap v4752: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:57:38.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:57:38 smithi138 ceph-mon[36954]: pgmap v4752: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:57:40.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:57:40 smithi100 ceph-mon[32599]: pgmap v4753: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:57:40.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:57:40 smithi167 ceph-mon[37140]: pgmap v4753: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:57:40.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:57:40 smithi138 ceph-mon[36954]: pgmap v4753: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:57:42.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:57:42 smithi100 ceph-mon[32599]: pgmap v4754: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:57:42.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:57:42 smithi167 ceph-mon[37140]: pgmap v4754: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:57:42.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:57:42 smithi138 ceph-mon[36954]: pgmap v4754: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:57:44.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:57:44 smithi100 ceph-mon[32599]: pgmap v4755: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:57:44.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:57:44 smithi167 ceph-mon[37140]: pgmap v4755: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:57:44.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:57:44 smithi138 ceph-mon[36954]: pgmap v4755: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:57:46.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:57:46 smithi100 ceph-mon[32599]: pgmap v4756: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:57:46.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:57:46 smithi167 ceph-mon[37140]: pgmap v4756: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:57:46.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:57:46 smithi138 ceph-mon[36954]: pgmap v4756: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:57:48.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:57:48 smithi100 ceph-mon[32599]: pgmap v4757: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:57:48.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:57:48 smithi167 ceph-mon[37140]: pgmap v4757: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:57:48.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:57:48 smithi138 ceph-mon[36954]: pgmap v4757: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:57:50.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:57:50.251+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:57:50.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:57:50.252+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:57:50.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:57:50.252+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:57:50.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:57:50.252+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:57:50.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:57:50.252+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:57:50.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:57:50 smithi100 ceph-mon[32599]: pgmap v4758: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:57:50.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:57:50 smithi167 ceph-mon[37140]: pgmap v4758: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:57:50.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:57:50 smithi138 ceph-mon[36954]: pgmap v4758: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:57:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:57:52 smithi100 ceph-mon[32599]: pgmap v4759: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:57:52.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:57:52 smithi167 ceph-mon[37140]: pgmap v4759: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:57:52.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:57:52 smithi138 ceph-mon[36954]: pgmap v4759: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:57:54.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:57:54 smithi167 ceph-mon[37140]: pgmap v4760: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:57:54.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:57:54 smithi138 ceph-mon[36954]: pgmap v4760: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:57:54.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:57:54 smithi100 ceph-mon[32599]: pgmap v4760: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:57:56.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:57:56 smithi167 ceph-mon[37140]: pgmap v4761: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:57:56.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:57:56 smithi138 ceph-mon[36954]: pgmap v4761: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:57:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:57:56 smithi100 ceph-mon[32599]: pgmap v4761: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:57:58.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:57:58 smithi167 ceph-mon[37140]: pgmap v4762: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:57:58.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:57:58 smithi138 ceph-mon[36954]: pgmap v4762: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:57:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:57:58 smithi100 ceph-mon[32599]: pgmap v4762: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:58:00.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:00 smithi167 ceph-mon[37140]: pgmap v4763: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:58:00.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:00 smithi138 ceph-mon[36954]: pgmap v4763: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:58:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:00 smithi100 ceph-mon[32599]: pgmap v4763: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:58:02.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:02 smithi167 ceph-mon[37140]: pgmap v4764: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:58:02.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:02 smithi138 ceph-mon[36954]: pgmap v4764: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:58:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:02 smithi100 ceph-mon[32599]: pgmap v4764: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:58:04.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:04 smithi167 ceph-mon[37140]: pgmap v4765: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:58:04.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:04 smithi138 ceph-mon[36954]: pgmap v4765: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:58:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:04 smithi100 ceph-mon[32599]: pgmap v4765: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:58:05.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:58:05.267+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:58:05.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:58:05.268+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:58:05.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:58:05.268+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:58:05.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:58:05.268+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:58:05.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:58:05.268+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:58:06.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:06 smithi167 ceph-mon[37140]: pgmap v4766: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:58:06.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:58:06.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:58:06.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:58:06.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:58:06.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:58:06.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:58:06.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:58:06.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:58:06.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:58:06.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:58:06.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:58:06.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:58:06.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:58:06.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:58:06.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:58:06.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:58:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:06 smithi138 ceph-mon[36954]: pgmap v4766: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:58:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:58:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:58:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:58:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:58:06.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:58:06.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:58:06.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:58:06.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:58:06.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:58:06.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:58:06.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:58:06.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:58:06.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:58:06.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:58:06.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:58:06.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:58:06.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:06 smithi100 ceph-mon[32599]: pgmap v4766: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:58:06.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:58:06.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:58:06.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:58:06.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:58:06.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:58:06.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:58:06.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:58:06.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:58:06.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:58:06.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:58:06.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:58:06.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:58:06.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T15:58:06.747 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:58:06.747 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:58:06.747 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:58:07.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:07 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T15:58:07.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:07 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T15:58:07.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:07 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T15:58:07.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:07 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T15:58:07.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:07 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T15:58:07.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:07 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T15:58:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:07 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T15:58:07.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:07 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T15:58:07.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:07 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T15:58:07.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:07 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T15:58:07.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:07 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T15:58:07.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:07 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T15:58:07.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:07 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T15:58:07.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:07 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T15:58:07.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:07 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T15:58:07.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:07 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T15:58:07.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:07 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T15:58:07.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:07 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T15:58:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:08 smithi138 ceph-mon[36954]: pgmap v4767: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:58:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:08 smithi138 ceph-mon[36954]: osdmap e346: 12 total, 12 up, 12 in 2024-07-26T15:58:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:08 smithi100 ceph-mon[32599]: pgmap v4767: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:58:08.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:08 smithi100 ceph-mon[32599]: osdmap e346: 12 total, 12 up, 12 in 2024-07-26T15:58:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:08 smithi167 ceph-mon[37140]: pgmap v4767: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:58:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:08 smithi167 ceph-mon[37140]: osdmap e346: 12 total, 12 up, 12 in 2024-07-26T15:58:10.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:10 smithi138 ceph-mon[36954]: pgmap v4769: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:58:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:10 smithi100 ceph-mon[32599]: pgmap v4769: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:58:10.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:10 smithi167 ceph-mon[37140]: pgmap v4769: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:58:12.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:12 smithi138 ceph-mon[36954]: pgmap v4770: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:58:12.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:12 smithi100 ceph-mon[32599]: pgmap v4770: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:58:12.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:12 smithi167 ceph-mon[37140]: pgmap v4770: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:58:14.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:14 smithi100 ceph-mon[32599]: pgmap v4771: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:58:14.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:14 smithi138 ceph-mon[36954]: pgmap v4771: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:58:14.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:14 smithi167 ceph-mon[37140]: pgmap v4771: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:58:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:16 smithi138 ceph-mon[36954]: pgmap v4772: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:58:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:16 smithi138 ceph-mon[36954]: osdmap e347: 12 total, 12 up, 12 in 2024-07-26T15:58:17.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:16 smithi100 ceph-mon[32599]: pgmap v4772: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:58:17.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:16 smithi100 ceph-mon[32599]: osdmap e347: 12 total, 12 up, 12 in 2024-07-26T15:58:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:16 smithi167 ceph-mon[37140]: pgmap v4772: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:58:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:16 smithi167 ceph-mon[37140]: osdmap e347: 12 total, 12 up, 12 in 2024-07-26T15:58:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:18 smithi138 ceph-mon[36954]: pgmap v4774: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T15:58:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:18 smithi100 ceph-mon[32599]: pgmap v4774: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T15:58:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:18 smithi167 ceph-mon[37140]: pgmap v4774: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T15:58:20.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:58:20.283+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:58:20.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:58:20.284+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:58:20.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:58:20.284+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:58:20.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:58:20.284+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:58:20.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:58:20.284+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:58:21.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:20 smithi138 ceph-mon[36954]: pgmap v4775: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:58:21.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:20 smithi100 ceph-mon[32599]: pgmap v4775: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:58:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:20 smithi167 ceph-mon[37140]: pgmap v4775: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:58:23.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:22 smithi138 ceph-mon[36954]: pgmap v4776: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:58:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:22 smithi100 ceph-mon[32599]: pgmap v4776: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:58:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:22 smithi167 ceph-mon[37140]: pgmap v4776: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:58:24.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:23 smithi138 ceph-mon[36954]: pgmap v4777: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:58:24.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:23 smithi100 ceph-mon[32599]: pgmap v4777: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:58:24.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:23 smithi167 ceph-mon[37140]: pgmap v4777: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:58:26.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:26 smithi100 ceph-mon[32599]: pgmap v4778: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:58:26.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:26 smithi167 ceph-mon[37140]: pgmap v4778: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:58:26.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:26 smithi138 ceph-mon[36954]: pgmap v4778: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:58:28.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:28 smithi100 ceph-mon[32599]: pgmap v4779: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T15:58:28.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:28 smithi167 ceph-mon[37140]: pgmap v4779: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T15:58:28.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:28 smithi138 ceph-mon[36954]: pgmap v4779: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T15:58:30.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:30 smithi100 ceph-mon[32599]: pgmap v4780: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:58:30.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:30 smithi167 ceph-mon[37140]: pgmap v4780: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:58:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:30 smithi138 ceph-mon[36954]: pgmap v4780: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:58:32.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:32 smithi100 ceph-mon[32599]: pgmap v4781: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:58:32.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:32 smithi167 ceph-mon[37140]: pgmap v4781: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:58:32.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:32 smithi138 ceph-mon[36954]: pgmap v4781: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:58:34.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:34 smithi100 ceph-mon[32599]: pgmap v4782: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:58:34.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:34 smithi167 ceph-mon[37140]: pgmap v4782: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:58:34.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:34 smithi138 ceph-mon[36954]: pgmap v4782: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:58:35.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:58:35.287+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:58:35.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:58:35.288+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:58:35.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:58:35.288+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:58:35.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:58:35.288+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:58:35.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:58:35.288+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:58:36.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:36 smithi100 ceph-mon[32599]: pgmap v4783: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:58:36.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:36 smithi167 ceph-mon[37140]: pgmap v4783: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:58:36.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:36 smithi138 ceph-mon[36954]: pgmap v4783: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:58:38.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:38 smithi100 ceph-mon[32599]: pgmap v4784: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:58:38.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:38 smithi167 ceph-mon[37140]: pgmap v4784: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:58:38.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:38 smithi138 ceph-mon[36954]: pgmap v4784: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:58:40.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:40 smithi100 ceph-mon[32599]: pgmap v4785: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:58:40.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:40 smithi167 ceph-mon[37140]: pgmap v4785: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:58:40.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:40 smithi138 ceph-mon[36954]: pgmap v4785: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:58:42.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:42 smithi100 ceph-mon[32599]: pgmap v4786: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:58:42.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:42 smithi167 ceph-mon[37140]: pgmap v4786: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:58:42.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:42 smithi138 ceph-mon[36954]: pgmap v4786: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:58:44.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:44 smithi167 ceph-mon[37140]: pgmap v4787: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:58:44.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:44 smithi138 ceph-mon[36954]: pgmap v4787: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:58:44.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:44 smithi100 ceph-mon[32599]: pgmap v4787: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:58:46.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:46 smithi167 ceph-mon[37140]: pgmap v4788: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:58:46.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:46 smithi138 ceph-mon[36954]: pgmap v4788: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:58:46.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:46 smithi100 ceph-mon[32599]: pgmap v4788: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:58:48.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:48 smithi167 ceph-mon[37140]: pgmap v4789: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:58:48.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:48 smithi138 ceph-mon[36954]: pgmap v4789: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:58:48.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:48 smithi100 ceph-mon[32599]: pgmap v4789: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:58:50.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:58:50.299+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:58:50.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:58:50.299+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:58:50.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:58:50.299+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:58:50.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:58:50.299+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:58:50.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:58:50.299+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:58:50.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:50 smithi167 ceph-mon[37140]: pgmap v4790: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:58:50.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:50 smithi138 ceph-mon[36954]: pgmap v4790: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:58:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:50 smithi100 ceph-mon[32599]: pgmap v4790: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:58:52.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:52 smithi167 ceph-mon[37140]: pgmap v4791: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:58:52.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:52 smithi138 ceph-mon[36954]: pgmap v4791: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:58:52.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:52 smithi100 ceph-mon[32599]: pgmap v4791: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:58:54.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:54 smithi167 ceph-mon[37140]: pgmap v4792: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:58:54.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:54 smithi138 ceph-mon[36954]: pgmap v4792: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:58:54.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:54 smithi100 ceph-mon[32599]: pgmap v4792: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:58:56.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:56 smithi167 ceph-mon[37140]: pgmap v4793: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:58:56.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:56 smithi138 ceph-mon[36954]: pgmap v4793: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:58:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:56 smithi100 ceph-mon[32599]: pgmap v4793: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:58:58.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:58:58 smithi167 ceph-mon[37140]: pgmap v4794: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:58:58.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:58:58 smithi138 ceph-mon[36954]: pgmap v4794: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:58:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:58:58 smithi100 ceph-mon[32599]: pgmap v4794: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:59:00.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:59:00 smithi167 ceph-mon[37140]: pgmap v4795: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:59:00.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:59:00 smithi138 ceph-mon[36954]: pgmap v4795: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:59:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:59:00 smithi100 ceph-mon[32599]: pgmap v4795: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:59:02.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:59:02 smithi138 ceph-mon[36954]: pgmap v4796: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:59:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:59:02 smithi100 ceph-mon[32599]: pgmap v4796: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:59:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:59:02 smithi167 ceph-mon[37140]: pgmap v4796: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:59:04.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:59:04 smithi138 ceph-mon[36954]: pgmap v4797: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:59:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:59:04 smithi100 ceph-mon[32599]: pgmap v4797: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:59:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:59:04 smithi167 ceph-mon[37140]: pgmap v4797: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T15:59:05.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:59:05.315+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:59:05.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:59:05.316+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:59:05.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:59:05.316+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:59:05.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:59:05.316+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:59:05.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:59:05.316+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:59:06.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:59:06 smithi167 ceph-mon[37140]: pgmap v4798: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:59:06.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:59:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:59:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:59:06 smithi138 ceph-mon[36954]: pgmap v4798: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:59:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:59:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:59:06.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:59:06 smithi100 ceph-mon[32599]: pgmap v4798: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:59:06.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:59:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T15:59:07.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:59:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:59:07.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:59:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:59:07.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:59:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:59:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:59:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:59:07.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:59:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:59:07.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:59:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:59:07.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:59:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T15:59:07.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:59:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T15:59:07.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:59:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T15:59:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:59:08 smithi138 ceph-mon[36954]: pgmap v4799: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:59:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:59:08 smithi138 ceph-mon[36954]: osdmap e348: 12 total, 12 up, 12 in 2024-07-26T15:59:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:59:08 smithi100 ceph-mon[32599]: pgmap v4799: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:59:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:59:08 smithi100 ceph-mon[32599]: osdmap e348: 12 total, 12 up, 12 in 2024-07-26T15:59:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:59:08 smithi167 ceph-mon[37140]: pgmap v4799: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T15:59:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:59:08 smithi167 ceph-mon[37140]: osdmap e348: 12 total, 12 up, 12 in 2024-07-26T15:59:10.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:59:10 smithi138 ceph-mon[36954]: pgmap v4801: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:59:10.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:59:10 smithi100 ceph-mon[32599]: pgmap v4801: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:59:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:59:10 smithi167 ceph-mon[37140]: pgmap v4801: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:59:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:59:12 smithi138 ceph-mon[36954]: pgmap v4802: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:59:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:59:12 smithi100 ceph-mon[32599]: pgmap v4802: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:59:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:59:12 smithi167 ceph-mon[37140]: pgmap v4802: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T15:59:14.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:59:14 smithi138 ceph-mon[36954]: pgmap v4803: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:59:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:59:14 smithi100 ceph-mon[32599]: pgmap v4803: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:59:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:59:14 smithi167 ceph-mon[37140]: pgmap v4803: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T15:59:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:59:16 smithi138 ceph-mon[36954]: pgmap v4804: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:59:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:59:16 smithi138 ceph-mon[36954]: osdmap e349: 12 total, 12 up, 12 in 2024-07-26T15:59:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:59:16 smithi100 ceph-mon[32599]: pgmap v4804: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:59:17.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:59:16 smithi100 ceph-mon[32599]: osdmap e349: 12 total, 12 up, 12 in 2024-07-26T15:59:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:59:16 smithi167 ceph-mon[37140]: pgmap v4804: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T15:59:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:59:16 smithi167 ceph-mon[37140]: osdmap e349: 12 total, 12 up, 12 in 2024-07-26T15:59:18.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:59:17 smithi138 ceph-mon[36954]: pgmap v4806: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 215 B/s wr, 6 op/s 2024-07-26T15:59:18.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:59:17 smithi100 ceph-mon[32599]: pgmap v4806: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 215 B/s wr, 6 op/s 2024-07-26T15:59:18.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:59:17 smithi167 ceph-mon[37140]: pgmap v4806: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 215 B/s wr, 6 op/s 2024-07-26T15:59:20.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:59:20.331+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:59:20.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:59:20.332+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:59:20.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:59:20.332+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:59:20.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:59:20.332+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:59:20.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:59:20.332+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:59:20.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:59:20 smithi100 ceph-mon[32599]: pgmap v4807: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:59:20.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:59:20 smithi167 ceph-mon[37140]: pgmap v4807: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:59:20.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:59:20 smithi138 ceph-mon[36954]: pgmap v4807: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:59:22.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:59:22 smithi100 ceph-mon[32599]: pgmap v4808: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:59:22.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:59:22 smithi167 ceph-mon[37140]: pgmap v4808: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:59:22.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:59:22 smithi138 ceph-mon[36954]: pgmap v4808: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:59:24.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:59:24 smithi100 ceph-mon[32599]: pgmap v4809: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:59:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:59:24 smithi167 ceph-mon[37140]: pgmap v4809: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:59:24.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:59:24 smithi138 ceph-mon[36954]: pgmap v4809: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T15:59:26.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:59:26 smithi100 ceph-mon[32599]: pgmap v4810: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:59:26.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:59:26 smithi167 ceph-mon[37140]: pgmap v4810: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:59:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:59:26 smithi138 ceph-mon[36954]: pgmap v4810: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T15:59:28.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:59:28 smithi167 ceph-mon[37140]: pgmap v4811: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T15:59:28.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:59:28 smithi138 ceph-mon[36954]: pgmap v4811: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T15:59:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:59:28 smithi100 ceph-mon[32599]: pgmap v4811: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T15:59:30.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:59:30 smithi167 ceph-mon[37140]: pgmap v4812: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:59:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:59:30 smithi138 ceph-mon[36954]: pgmap v4812: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:59:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:59:30 smithi100 ceph-mon[32599]: pgmap v4812: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:59:32.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:59:32 smithi167 ceph-mon[37140]: pgmap v4813: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:59:32.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:59:32 smithi138 ceph-mon[36954]: pgmap v4813: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:59:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:59:32 smithi100 ceph-mon[32599]: pgmap v4813: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:59:34.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:59:34 smithi167 ceph-mon[37140]: pgmap v4814: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:59:34.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:59:34 smithi138 ceph-mon[36954]: pgmap v4814: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:59:34.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:59:34 smithi100 ceph-mon[32599]: pgmap v4814: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:59:35.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:59:35.347+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:59:35.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:59:35.348+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:59:35.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:59:35.348+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:59:35.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:59:35.348+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:59:35.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:59:35.348+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:59:36.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:59:36 smithi167 ceph-mon[37140]: pgmap v4815: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:59:36.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:59:36 smithi138 ceph-mon[36954]: pgmap v4815: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:59:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:59:36 smithi100 ceph-mon[32599]: pgmap v4815: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:59:38.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:59:38 smithi167 ceph-mon[37140]: pgmap v4816: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:59:38.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:59:38 smithi138 ceph-mon[36954]: pgmap v4816: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:59:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:59:38 smithi100 ceph-mon[32599]: pgmap v4816: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:59:40.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:59:40 smithi167 ceph-mon[37140]: pgmap v4817: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:59:40.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:59:40 smithi138 ceph-mon[36954]: pgmap v4817: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:59:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:59:40 smithi100 ceph-mon[32599]: pgmap v4817: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:59:42.527 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:59:42 smithi167 ceph-mon[37140]: pgmap v4818: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:59:42.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:59:42 smithi138 ceph-mon[36954]: pgmap v4818: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:59:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:59:42 smithi100 ceph-mon[32599]: pgmap v4818: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:59:44.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:59:44 smithi100 ceph-mon[32599]: pgmap v4819: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:59:44.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:59:44 smithi138 ceph-mon[36954]: pgmap v4819: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:59:44.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:59:44 smithi167 ceph-mon[37140]: pgmap v4819: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:59:46.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:59:46 smithi138 ceph-mon[36954]: pgmap v4820: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:59:46.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:59:46 smithi100 ceph-mon[32599]: pgmap v4820: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:59:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:59:46 smithi167 ceph-mon[37140]: pgmap v4820: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:59:48.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:59:48 smithi138 ceph-mon[36954]: pgmap v4821: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:59:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:59:48 smithi100 ceph-mon[32599]: pgmap v4821: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:59:48.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:59:48 smithi167 ceph-mon[37140]: pgmap v4821: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:59:50.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:59:50.363+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T15:59:50.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:59:50.364+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T15:59:50.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:59:50.364+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T15:59:50.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:59:50.364+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T15:59:50.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T15:59:50.364+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T15:59:50.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:59:50 smithi138 ceph-mon[36954]: pgmap v4822: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:59:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:59:50 smithi100 ceph-mon[32599]: pgmap v4822: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:59:50.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:59:50 smithi167 ceph-mon[37140]: pgmap v4822: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:59:52.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:59:52 smithi138 ceph-mon[36954]: pgmap v4823: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:59:52.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:59:52 smithi100 ceph-mon[32599]: pgmap v4823: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:59:52.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:59:52 smithi167 ceph-mon[37140]: pgmap v4823: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T15:59:54.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:59:54 smithi138 ceph-mon[36954]: pgmap v4824: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:59:54.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:59:54 smithi100 ceph-mon[32599]: pgmap v4824: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:59:54.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:59:54 smithi167 ceph-mon[37140]: pgmap v4824: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:59:56.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:59:56 smithi138 ceph-mon[36954]: pgmap v4825: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:59:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:59:56 smithi100 ceph-mon[32599]: pgmap v4825: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:59:56.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:59:56 smithi167 ceph-mon[37140]: pgmap v4825: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T15:59:58.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 15:59:58 smithi138 ceph-mon[36954]: pgmap v4826: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:59:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 15:59:58 smithi100 ceph-mon[32599]: pgmap v4826: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T15:59:58.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 15:59:58 smithi167 ceph-mon[37140]: pgmap v4826: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:00:00.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:00:00 smithi138 ceph-mon[36954]: pgmap v4827: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:00:00.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:00:00 smithi138 ceph-mon[36954]: overall HEALTH_OK 2024-07-26T16:00:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:00:00 smithi100 ceph-mon[32599]: pgmap v4827: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:00:00.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:00:00 smithi100 ceph-mon[32599]: overall HEALTH_OK 2024-07-26T16:00:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:00:00 smithi167 ceph-mon[37140]: pgmap v4827: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:00:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:00:00 smithi167 ceph-mon[37140]: overall HEALTH_OK 2024-07-26T16:00:02.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:00:02 smithi138 ceph-mon[36954]: pgmap v4828: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:00:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:00:02 smithi100 ceph-mon[32599]: pgmap v4828: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:00:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:00:02 smithi167 ceph-mon[37140]: pgmap v4828: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:00:04.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:00:04.133+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c004740 con 0x7fe5a81087a0 2024-07-26T16:00:04.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:00:04.133+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T16:00:04.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:00:04.133+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 64 ==== auth_reply(proto 2 0 (0) Success) ==== 783+0+0 (secure 0 0 0) 0x7fe57c004740 con 0x7fe5a81087a0 2024-07-26T16:00:04.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:00:04.133+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T16:00:04.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:00:04.133+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 64 ==== auth_reply(proto 2 0 (0) Success) ==== 612+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T16:00:04.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:00:04.133+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T16:00:04.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:00:04.133+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 65 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T16:00:04.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:00:04.133+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 65 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T16:00:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:00:04 smithi100 ceph-mon[32599]: pgmap v4829: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:00:04.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:00:04 smithi138 ceph-mon[36954]: pgmap v4829: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:00:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:00:04 smithi167 ceph-mon[37140]: pgmap v4829: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:00:05.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:00:05.367+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:00:05.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:00:05.367+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:00:05.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:00:05.367+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:00:05.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:00:05.367+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:00:05.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:00:05.367+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:00:06.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:00:06 smithi167 ceph-mon[37140]: pgmap v4830: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:00:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:00:06 smithi138 ceph-mon[36954]: pgmap v4830: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:00:06.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:00:06 smithi100 ceph-mon[32599]: pgmap v4830: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:00:07.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:00:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:00:07.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:00:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:00:07.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:00:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:00:07.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:00:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:00:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:00:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:00:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:00:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:00:07.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:00:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:00:07.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:00:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:00:07.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:00:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:00:07.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:00:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:00:07.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:00:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:00:07.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:00:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:00:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:00:08 smithi138 ceph-mon[36954]: pgmap v4831: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:00:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:00:08 smithi138 ceph-mon[36954]: osdmap e350: 12 total, 12 up, 12 in 2024-07-26T16:00:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:00:08 smithi100 ceph-mon[32599]: pgmap v4831: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:00:08.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:00:08 smithi100 ceph-mon[32599]: osdmap e350: 12 total, 12 up, 12 in 2024-07-26T16:00:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:00:08 smithi167 ceph-mon[37140]: pgmap v4831: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:00:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:00:08 smithi167 ceph-mon[37140]: osdmap e350: 12 total, 12 up, 12 in 2024-07-26T16:00:10.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:00:10 smithi138 ceph-mon[36954]: pgmap v4833: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:00:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:00:10 smithi100 ceph-mon[32599]: pgmap v4833: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:00:10.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:00:10 smithi167 ceph-mon[37140]: pgmap v4833: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:00:12.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:00:12 smithi138 ceph-mon[36954]: pgmap v4834: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:00:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:00:12 smithi100 ceph-mon[32599]: pgmap v4834: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:00:12.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:00:12 smithi167 ceph-mon[37140]: pgmap v4834: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:00:14.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:00:14 smithi138 ceph-mon[36954]: pgmap v4835: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:00:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:00:14 smithi100 ceph-mon[32599]: pgmap v4835: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:00:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:00:14 smithi167 ceph-mon[37140]: pgmap v4835: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:00:16.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:00:16 smithi138 ceph-mon[36954]: pgmap v4836: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:00:16.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:00:16 smithi138 ceph-mon[36954]: osdmap e351: 12 total, 12 up, 12 in 2024-07-26T16:00:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:00:16 smithi100 ceph-mon[32599]: pgmap v4836: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:00:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:00:16 smithi100 ceph-mon[32599]: osdmap e351: 12 total, 12 up, 12 in 2024-07-26T16:00:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:00:16 smithi167 ceph-mon[37140]: pgmap v4836: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:00:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:00:16 smithi167 ceph-mon[37140]: osdmap e351: 12 total, 12 up, 12 in 2024-07-26T16:00:18.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:00:18 smithi138 ceph-mon[36954]: pgmap v4838: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T16:00:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:00:18 smithi100 ceph-mon[32599]: pgmap v4838: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T16:00:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:00:18 smithi167 ceph-mon[37140]: pgmap v4838: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T16:00:20.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:00:20.375+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:00:20.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:00:20.376+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:00:20.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:00:20.376+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:00:20.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:00:20.376+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:00:20.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:00:20.376+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:00:20.612 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:00:20 smithi167 ceph-mon[37140]: pgmap v4839: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:00:20.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:00:20 smithi138 ceph-mon[36954]: pgmap v4839: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:00:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:00:20 smithi100 ceph-mon[32599]: pgmap v4839: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:00:22.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:00:21 smithi138 ceph-mon[36954]: pgmap v4840: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:00:22.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:00:21 smithi100 ceph-mon[32599]: pgmap v4840: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:00:22.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:00:21 smithi167 ceph-mon[37140]: pgmap v4840: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:00:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:00:24 smithi167 ceph-mon[37140]: pgmap v4841: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:00:24.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:00:24 smithi138 ceph-mon[36954]: pgmap v4841: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:00:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:00:24 smithi100 ceph-mon[32599]: pgmap v4841: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:00:26.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:00:26 smithi167 ceph-mon[37140]: pgmap v4842: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:00:26.694 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:00:26 smithi138 ceph-mon[36954]: pgmap v4842: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:00:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:00:26 smithi100 ceph-mon[32599]: pgmap v4842: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:00:28.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:00:28 smithi167 ceph-mon[37140]: pgmap v4843: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:00:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:00:28 smithi138 ceph-mon[36954]: pgmap v4843: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:00:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:00:28 smithi100 ceph-mon[32599]: pgmap v4843: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:00:30.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:00:30 smithi167 ceph-mon[37140]: pgmap v4844: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:00:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:00:30 smithi138 ceph-mon[36954]: pgmap v4844: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:00:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:00:30 smithi100 ceph-mon[32599]: pgmap v4844: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:00:32.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:00:31 smithi138 ceph-mon[36954]: pgmap v4845: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:00:32.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:00:31 smithi100 ceph-mon[32599]: pgmap v4845: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:00:32.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:00:31 smithi167 ceph-mon[37140]: pgmap v4845: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:00:34.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:00:34 smithi167 ceph-mon[37140]: pgmap v4846: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:00:34.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:00:34 smithi138 ceph-mon[36954]: pgmap v4846: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:00:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:00:34 smithi100 ceph-mon[32599]: pgmap v4846: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:00:35.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:00:35.392+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:00:35.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:00:35.392+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:00:35.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:00:35.392+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:00:35.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:00:35.392+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:00:35.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:00:35.392+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:00:36.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:00:36 smithi167 ceph-mon[37140]: pgmap v4847: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:00:36.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:00:36 smithi138 ceph-mon[36954]: pgmap v4847: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:00:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:00:36 smithi100 ceph-mon[32599]: pgmap v4847: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:00:38.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:00:38 smithi167 ceph-mon[37140]: pgmap v4848: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:00:38.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:00:38 smithi138 ceph-mon[36954]: pgmap v4848: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:00:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:00:38 smithi100 ceph-mon[32599]: pgmap v4848: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:00:40.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:00:40 smithi167 ceph-mon[37140]: pgmap v4849: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:00:40.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:00:40 smithi138 ceph-mon[36954]: pgmap v4849: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:00:40.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:00:40 smithi100 ceph-mon[32599]: pgmap v4849: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:00:42.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:00:42 smithi138 ceph-mon[36954]: pgmap v4850: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:00:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:00:42 smithi100 ceph-mon[32599]: pgmap v4850: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:00:42.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:00:42 smithi167 ceph-mon[37140]: pgmap v4850: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:00:44.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:00:44 smithi100 ceph-mon[32599]: pgmap v4851: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:00:44.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:00:44 smithi138 ceph-mon[36954]: pgmap v4851: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:00:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:00:44 smithi167 ceph-mon[37140]: pgmap v4851: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:00:46.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:00:46 smithi138 ceph-mon[36954]: pgmap v4852: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:00:46.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:00:46 smithi100 ceph-mon[32599]: pgmap v4852: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:00:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:00:46 smithi167 ceph-mon[37140]: pgmap v4852: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:00:48.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:00:48 smithi167 ceph-mon[37140]: pgmap v4853: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:00:48.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:00:48 smithi138 ceph-mon[36954]: pgmap v4853: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:00:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:00:48 smithi100 ceph-mon[32599]: pgmap v4853: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:00:50.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:00:50.406+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:00:50.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:00:50.407+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:00:50.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:00:50.407+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:00:50.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:00:50.407+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:00:50.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:00:50.407+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:00:50.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:00:50 smithi138 ceph-mon[36954]: pgmap v4854: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:00:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:00:50 smithi100 ceph-mon[32599]: pgmap v4854: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:00:50.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:00:50 smithi167 ceph-mon[37140]: pgmap v4854: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:00:52.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:00:52 smithi138 ceph-mon[36954]: pgmap v4855: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:00:52.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:00:52 smithi100 ceph-mon[32599]: pgmap v4855: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:00:52.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:00:52 smithi167 ceph-mon[37140]: pgmap v4855: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:00:54.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:00:54 smithi138 ceph-mon[36954]: pgmap v4856: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:00:54.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:00:54 smithi100 ceph-mon[32599]: pgmap v4856: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:00:54.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:00:54 smithi167 ceph-mon[37140]: pgmap v4856: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:00:56.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:00:56 smithi138 ceph-mon[36954]: pgmap v4857: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:00:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:00:56 smithi100 ceph-mon[32599]: pgmap v4857: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:00:56.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:00:56 smithi167 ceph-mon[37140]: pgmap v4857: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:00:58.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:00:58 smithi138 ceph-mon[36954]: pgmap v4858: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:00:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:00:58 smithi100 ceph-mon[32599]: pgmap v4858: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:00:58.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:00:58 smithi167 ceph-mon[37140]: pgmap v4858: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:01:00.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:00 smithi138 ceph-mon[36954]: pgmap v4859: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:01:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:00 smithi100 ceph-mon[32599]: pgmap v4859: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:01:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:00 smithi167 ceph-mon[37140]: pgmap v4859: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:01:02.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:02 smithi138 ceph-mon[36954]: pgmap v4860: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:01:02.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:02 smithi100 ceph-mon[32599]: pgmap v4860: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:01:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:02 smithi167 ceph-mon[37140]: pgmap v4860: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:01:04.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:04 smithi138 ceph-mon[36954]: pgmap v4861: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:01:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:04 smithi100 ceph-mon[32599]: pgmap v4861: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:01:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:04 smithi167 ceph-mon[37140]: pgmap v4861: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:01:05.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:01:05.422+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:01:05.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:01:05.423+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:01:05.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:01:05.423+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:01:05.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:01:05.423+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:01:05.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:01:05.423+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:01:06.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:06 smithi167 ceph-mon[37140]: pgmap v4862: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:01:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:06 smithi138 ceph-mon[36954]: pgmap v4862: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:01:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:06 smithi100 ceph-mon[32599]: pgmap v4862: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:01:07.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:01:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:01:07.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:01:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:08 smithi138 ceph-mon[36954]: pgmap v4863: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:01:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:08 smithi138 ceph-mon[36954]: osdmap e352: 12 total, 12 up, 12 in 2024-07-26T16:01:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:08 smithi100 ceph-mon[32599]: pgmap v4863: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:01:08.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:08 smithi100 ceph-mon[32599]: osdmap e352: 12 total, 12 up, 12 in 2024-07-26T16:01:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:08 smithi167 ceph-mon[37140]: pgmap v4863: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:01:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:08 smithi167 ceph-mon[37140]: osdmap e352: 12 total, 12 up, 12 in 2024-07-26T16:01:10.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:01:10.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:01:10.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:01:10.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:01:10.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:01:10.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:01:10.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:01:10.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:01:10.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:01:10.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:01:10.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:01:10.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:01:10.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:01:10.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:01:10.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:01:10.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:01:10.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:01:10.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:01:10.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:01:10.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:01:10.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:01:10.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:01:10.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:01:10.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:01:10.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:01:10.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:01:10.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:01:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:10 smithi167 ceph-mon[37140]: pgmap v4865: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:01:11.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:10 smithi138 ceph-mon[36954]: pgmap v4865: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:01:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:10 smithi100 ceph-mon[32599]: pgmap v4865: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:01:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:12 smithi167 ceph-mon[37140]: pgmap v4866: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:01:13.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:12 smithi138 ceph-mon[36954]: pgmap v4866: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:01:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:12 smithi100 ceph-mon[32599]: pgmap v4866: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:01:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:14 smithi167 ceph-mon[37140]: pgmap v4867: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:01:15.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:14 smithi100 ceph-mon[32599]: pgmap v4867: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:01:15.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:14 smithi138 ceph-mon[36954]: pgmap v4867: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:01:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:16 smithi167 ceph-mon[37140]: pgmap v4868: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:01:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:16 smithi167 ceph-mon[37140]: osdmap e353: 12 total, 12 up, 12 in 2024-07-26T16:01:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:16 smithi138 ceph-mon[36954]: pgmap v4868: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:01:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:16 smithi138 ceph-mon[36954]: osdmap e353: 12 total, 12 up, 12 in 2024-07-26T16:01:17.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:16 smithi100 ceph-mon[32599]: pgmap v4868: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:01:17.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:16 smithi100 ceph-mon[32599]: osdmap e353: 12 total, 12 up, 12 in 2024-07-26T16:01:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:18 smithi138 ceph-mon[36954]: pgmap v4870: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:01:19.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:18 smithi100 ceph-mon[32599]: pgmap v4870: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:01:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:18 smithi167 ceph-mon[37140]: pgmap v4870: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:01:20.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:01:20.431+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:01:20.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:01:20.431+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:01:20.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:01:20.431+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:01:20.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:01:20.431+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:01:20.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:01:20.431+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:01:21.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:20 smithi138 ceph-mon[36954]: pgmap v4871: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T16:01:21.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:20 smithi100 ceph-mon[32599]: pgmap v4871: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T16:01:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:20 smithi167 ceph-mon[37140]: pgmap v4871: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T16:01:23.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:22 smithi138 ceph-mon[36954]: pgmap v4872: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T16:01:23.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:22 smithi100 ceph-mon[32599]: pgmap v4872: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T16:01:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:22 smithi167 ceph-mon[37140]: pgmap v4872: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T16:01:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:24 smithi138 ceph-mon[36954]: pgmap v4873: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T16:01:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:24 smithi100 ceph-mon[32599]: pgmap v4873: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T16:01:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:24 smithi167 ceph-mon[37140]: pgmap v4873: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T16:01:27.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:26 smithi138 ceph-mon[36954]: pgmap v4874: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:01:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:26 smithi100 ceph-mon[32599]: pgmap v4874: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:01:27.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:26 smithi167 ceph-mon[37140]: pgmap v4874: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:01:29.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:28 smithi138 ceph-mon[36954]: pgmap v4875: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:01:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:28 smithi100 ceph-mon[32599]: pgmap v4875: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:01:29.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:28 smithi167 ceph-mon[37140]: pgmap v4875: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:01:31.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:30 smithi138 ceph-mon[36954]: pgmap v4876: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:01:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:30 smithi100 ceph-mon[32599]: pgmap v4876: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:01:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:30 smithi167 ceph-mon[37140]: pgmap v4876: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:01:33.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:32 smithi138 ceph-mon[36954]: pgmap v4877: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:01:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:32 smithi100 ceph-mon[32599]: pgmap v4877: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:01:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:32 smithi167 ceph-mon[37140]: pgmap v4877: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:01:35.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:34 smithi138 ceph-mon[36954]: pgmap v4878: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:01:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:34 smithi100 ceph-mon[32599]: pgmap v4878: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:01:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:34 smithi167 ceph-mon[37140]: pgmap v4878: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:01:35.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:01:35.448+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:01:35.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:01:35.448+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:01:35.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:01:35.448+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:01:35.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:01:35.448+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:01:35.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:01:35.448+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:01:37.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:36 smithi167 ceph-mon[37140]: pgmap v4879: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:01:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:36 smithi138 ceph-mon[36954]: pgmap v4879: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:01:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:36 smithi100 ceph-mon[32599]: pgmap v4879: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:01:39.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:38 smithi138 ceph-mon[36954]: pgmap v4880: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:01:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:38 smithi100 ceph-mon[32599]: pgmap v4880: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:01:39.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:38 smithi167 ceph-mon[37140]: pgmap v4880: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:01:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:40 smithi138 ceph-mon[36954]: pgmap v4881: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:01:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:40 smithi100 ceph-mon[32599]: pgmap v4881: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:01:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:40 smithi167 ceph-mon[37140]: pgmap v4881: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:01:43.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:42 smithi138 ceph-mon[36954]: pgmap v4882: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:01:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:42 smithi100 ceph-mon[32599]: pgmap v4882: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:01:43.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:42 smithi167 ceph-mon[37140]: pgmap v4882: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:01:45.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:44 smithi138 ceph-mon[36954]: pgmap v4883: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:01:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:44 smithi100 ceph-mon[32599]: pgmap v4883: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:01:45.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:44 smithi167 ceph-mon[37140]: pgmap v4883: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:01:47.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:46 smithi138 ceph-mon[36954]: pgmap v4884: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:01:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:46 smithi100 ceph-mon[32599]: pgmap v4884: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:01:47.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:46 smithi167 ceph-mon[37140]: pgmap v4884: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:01:49.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:48 smithi138 ceph-mon[36954]: pgmap v4885: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:01:49.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:48 smithi100 ceph-mon[32599]: pgmap v4885: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:01:49.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:48 smithi167 ceph-mon[37140]: pgmap v4885: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:01:50.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:01:50.463+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:01:50.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:01:50.464+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:01:50.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:01:50.464+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:01:50.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:01:50.464+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:01:50.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:01:50.464+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:01:51.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:50 smithi138 ceph-mon[36954]: pgmap v4886: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:01:51.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:50 smithi100 ceph-mon[32599]: pgmap v4886: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:01:51.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:50 smithi167 ceph-mon[37140]: pgmap v4886: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:01:53.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:52 smithi138 ceph-mon[36954]: pgmap v4887: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:01:53.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:52 smithi100 ceph-mon[32599]: pgmap v4887: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:01:53.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:52 smithi167 ceph-mon[37140]: pgmap v4887: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:01:55.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:54 smithi138 ceph-mon[36954]: pgmap v4888: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:01:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:54 smithi100 ceph-mon[32599]: pgmap v4888: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:01:55.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:54 smithi167 ceph-mon[37140]: pgmap v4888: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:01:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:56 smithi138 ceph-mon[36954]: pgmap v4889: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:01:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:56 smithi100 ceph-mon[32599]: pgmap v4889: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:01:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:56 smithi167 ceph-mon[37140]: pgmap v4889: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:01:59.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:01:58 smithi138 ceph-mon[36954]: pgmap v4890: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:01:59.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:01:58 smithi100 ceph-mon[32599]: pgmap v4890: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:01:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:01:58 smithi167 ceph-mon[37140]: pgmap v4890: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:02:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:00 smithi138 ceph-mon[36954]: pgmap v4891: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:02:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:00 smithi100 ceph-mon[32599]: pgmap v4891: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:02:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:00 smithi167 ceph-mon[37140]: pgmap v4891: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:02:03.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:02 smithi138 ceph-mon[36954]: pgmap v4892: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:02:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:02 smithi100 ceph-mon[32599]: pgmap v4892: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:02:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:02 smithi167 ceph-mon[37140]: pgmap v4892: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:02:05.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:04 smithi138 ceph-mon[36954]: pgmap v4893: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:02:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:04 smithi100 ceph-mon[32599]: pgmap v4893: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:02:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:04 smithi167 ceph-mon[37140]: pgmap v4893: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:02:05.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:02:05.479+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:02:05.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:02:05.480+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:02:05.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:02:05.480+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:02:05.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:02:05.480+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:02:05.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:02:05.480+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:02:07.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:06 smithi138 ceph-mon[36954]: pgmap v4894: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:02:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:06 smithi100 ceph-mon[32599]: pgmap v4894: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:02:07.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:06 smithi167 ceph-mon[37140]: pgmap v4894: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:02:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:08 smithi138 ceph-mon[36954]: pgmap v4895: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:02:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:08 smithi138 ceph-mon[36954]: osdmap e354: 12 total, 12 up, 12 in 2024-07-26T16:02:09.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:08 smithi100 ceph-mon[32599]: pgmap v4895: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:02:09.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:08 smithi100 ceph-mon[32599]: osdmap e354: 12 total, 12 up, 12 in 2024-07-26T16:02:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:08 smithi167 ceph-mon[37140]: pgmap v4895: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:02:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:08 smithi167 ceph-mon[37140]: osdmap e354: 12 total, 12 up, 12 in 2024-07-26T16:02:10.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:02:10.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:02:10.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:02:10.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:02:10.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:02:10.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:02:10.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:02:10.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:02:10.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:02:10.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:02:10.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:02:10.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:02:10.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:02:10.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:02:10.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:02:10.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:02:10.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:02:10.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:02:10.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:02:10.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:02:10.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:02:11.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:10 smithi138 ceph-mon[36954]: pgmap v4897: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:02:11.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:10 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:02:11.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:10 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:02:11.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:10 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:02:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:10 smithi100 ceph-mon[32599]: pgmap v4897: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:02:11.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:10 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:02:11.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:10 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:02:11.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:10 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:02:11.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:10 smithi167 ceph-mon[37140]: pgmap v4897: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:02:11.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:10 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:02:11.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:10 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:02:11.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:10 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:02:13.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:12 smithi138 ceph-mon[36954]: pgmap v4898: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:02:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:12 smithi100 ceph-mon[32599]: pgmap v4898: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:02:13.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:12 smithi167 ceph-mon[37140]: pgmap v4898: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:02:15.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:14 smithi138 ceph-mon[36954]: pgmap v4899: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:02:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:14 smithi100 ceph-mon[32599]: pgmap v4899: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:02:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:14 smithi167 ceph-mon[37140]: pgmap v4899: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:02:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:16 smithi138 ceph-mon[36954]: pgmap v4900: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:02:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:16 smithi138 ceph-mon[36954]: osdmap e355: 12 total, 12 up, 12 in 2024-07-26T16:02:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:16 smithi100 ceph-mon[32599]: pgmap v4900: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:02:17.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:16 smithi100 ceph-mon[32599]: osdmap e355: 12 total, 12 up, 12 in 2024-07-26T16:02:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:16 smithi167 ceph-mon[37140]: pgmap v4900: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:02:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:16 smithi167 ceph-mon[37140]: osdmap e355: 12 total, 12 up, 12 in 2024-07-26T16:02:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:18 smithi138 ceph-mon[36954]: pgmap v4902: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 218 B/s wr, 6 op/s 2024-07-26T16:02:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:18 smithi100 ceph-mon[32599]: pgmap v4902: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 218 B/s wr, 6 op/s 2024-07-26T16:02:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:18 smithi167 ceph-mon[37140]: pgmap v4902: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 218 B/s wr, 6 op/s 2024-07-26T16:02:20.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:02:20.488+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:02:20.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:02:20.489+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:02:20.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:02:20.489+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:02:20.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:02:20.489+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:02:20.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:02:20.489+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:02:21.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:20 smithi138 ceph-mon[36954]: pgmap v4903: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:02:21.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:20 smithi100 ceph-mon[32599]: pgmap v4903: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:02:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:20 smithi167 ceph-mon[37140]: pgmap v4903: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:02:22.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:21 smithi100 ceph-mon[32599]: pgmap v4904: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:02:22.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:21 smithi167 ceph-mon[37140]: pgmap v4904: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:02:22.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:21 smithi138 ceph-mon[36954]: pgmap v4904: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:02:24.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:24 smithi138 ceph-mon[36954]: pgmap v4905: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:02:24.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:24 smithi100 ceph-mon[32599]: pgmap v4905: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:02:24.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:24 smithi167 ceph-mon[37140]: pgmap v4905: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:02:26.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:26 smithi138 ceph-mon[36954]: pgmap v4906: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:02:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:26 smithi100 ceph-mon[32599]: pgmap v4906: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:02:26.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:26 smithi167 ceph-mon[37140]: pgmap v4906: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:02:28.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:28 smithi138 ceph-mon[36954]: pgmap v4907: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:02:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:28 smithi100 ceph-mon[32599]: pgmap v4907: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:02:28.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:28 smithi167 ceph-mon[37140]: pgmap v4907: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:02:30.696 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:30 smithi138 ceph-mon[36954]: pgmap v4908: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:02:30.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:30 smithi100 ceph-mon[32599]: pgmap v4908: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:02:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:30 smithi167 ceph-mon[37140]: pgmap v4908: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:02:32.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:32 smithi138 ceph-mon[36954]: pgmap v4909: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T16:02:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:32 smithi100 ceph-mon[32599]: pgmap v4909: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T16:02:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:32 smithi167 ceph-mon[37140]: pgmap v4909: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T16:02:34.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:34 smithi138 ceph-mon[36954]: pgmap v4910: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:02:34.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:34 smithi100 ceph-mon[32599]: pgmap v4910: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:02:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:34 smithi167 ceph-mon[37140]: pgmap v4910: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:02:35.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:02:35.504+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:02:35.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:02:35.504+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:02:35.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:02:35.504+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:02:35.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:02:35.504+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:02:35.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:02:35.504+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:02:36.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:36 smithi138 ceph-mon[36954]: pgmap v4911: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:02:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:36 smithi100 ceph-mon[32599]: pgmap v4911: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:02:36.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:36 smithi167 ceph-mon[37140]: pgmap v4911: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:02:38.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:38 smithi167 ceph-mon[37140]: pgmap v4912: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:02:38.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:38 smithi138 ceph-mon[36954]: pgmap v4912: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:02:38.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:38 smithi100 ceph-mon[32599]: pgmap v4912: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:02:40.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:40 smithi138 ceph-mon[36954]: pgmap v4913: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:02:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:40 smithi100 ceph-mon[32599]: pgmap v4913: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:02:40.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:40 smithi167 ceph-mon[37140]: pgmap v4913: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:02:42.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:42 smithi138 ceph-mon[36954]: pgmap v4914: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:02:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:42 smithi100 ceph-mon[32599]: pgmap v4914: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:02:42.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:42 smithi167 ceph-mon[37140]: pgmap v4914: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:02:44.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:44 smithi138 ceph-mon[36954]: pgmap v4915: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:02:44.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:44 smithi100 ceph-mon[32599]: pgmap v4915: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:02:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:44 smithi167 ceph-mon[37140]: pgmap v4915: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:02:46.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:46 smithi138 ceph-mon[36954]: pgmap v4916: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:02:46.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:46 smithi100 ceph-mon[32599]: pgmap v4916: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:02:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:46 smithi167 ceph-mon[37140]: pgmap v4916: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:02:48.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:48 smithi138 ceph-mon[36954]: pgmap v4917: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:02:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:48 smithi100 ceph-mon[32599]: pgmap v4917: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:02:48.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:48 smithi167 ceph-mon[37140]: pgmap v4917: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:02:50.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:02:50.521+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:02:50.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:02:50.521+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:02:50.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:02:50.521+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:02:50.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:02:50.521+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:02:50.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:02:50.521+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:02:50.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:50 smithi138 ceph-mon[36954]: pgmap v4918: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:02:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:50 smithi100 ceph-mon[32599]: pgmap v4918: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:02:50.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:50 smithi167 ceph-mon[37140]: pgmap v4918: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:02:52.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:52 smithi138 ceph-mon[36954]: pgmap v4919: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:02:52.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:52 smithi100 ceph-mon[32599]: pgmap v4919: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:02:52.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:52 smithi167 ceph-mon[37140]: pgmap v4919: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:02:54.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:54 smithi138 ceph-mon[36954]: pgmap v4920: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:02:54.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:54 smithi100 ceph-mon[32599]: pgmap v4920: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:02:54.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:54 smithi167 ceph-mon[37140]: pgmap v4920: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:02:56.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:56 smithi138 ceph-mon[36954]: pgmap v4921: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:02:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:56 smithi100 ceph-mon[32599]: pgmap v4921: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:02:56.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:56 smithi167 ceph-mon[37140]: pgmap v4921: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:02:58.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:02:58 smithi138 ceph-mon[36954]: pgmap v4922: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:02:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:02:58 smithi100 ceph-mon[32599]: pgmap v4922: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:02:58.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:02:58 smithi167 ceph-mon[37140]: pgmap v4922: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:03:00.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:00 smithi138 ceph-mon[36954]: pgmap v4923: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:03:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:00 smithi100 ceph-mon[32599]: pgmap v4923: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:03:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:00 smithi167 ceph-mon[37140]: pgmap v4923: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:03:02.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:02 smithi138 ceph-mon[36954]: pgmap v4924: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:03:02.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:02 smithi100 ceph-mon[32599]: pgmap v4924: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:03:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:02 smithi167 ceph-mon[37140]: pgmap v4924: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:03:03.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:03:03.035+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b4ef0 msgr2=0x7fe5940b5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-07-26T16:03:03.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:03:03.035+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b4ef0 msgr2=0x7fe5940b5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-07-26T16:03:03.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:03:03.035+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b4ef0 0x7fe5940b5300 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fe590004850 tx=0x7fe590069040 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:03:03.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:03:03.035+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b4ef0 0x7fe5940b5300 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fe590004850 tx=0x7fe590069040 comp rx=0 tx=0).stop 2024-07-26T16:03:03.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:03:03.035+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b4ef0 msgr2=0x7fe5940b5300 unknown :-1 s=STATE_CLOSED l=1).tick see no progress in more than 10000000 us during connecting to v2:172.21.15.100:6800/3937009534, fault. 2024-07-26T16:03:03.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:03:03.035+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b4ef0 msgr2=0x7fe5940b5300 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-07-26T16:03:03.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:03:03.035+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b4ef0 0x7fe5940b5300 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T16:03:03.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:03:03.035+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T16:03:03.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:03:03.035+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:03:03.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:03:03.036+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe590004ee0 tx=0x7fe59005a040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T16:03:04.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:03:04.160+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T16:03:04.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:03:04.160+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T16:03:04.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:03:04.160+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 66 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T16:03:04.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:03:04.160+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 66 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c020 con 0x7fe5a81087a0 2024-07-26T16:03:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:04 smithi100 ceph-mon[32599]: pgmap v4925: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:03:04.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:04 smithi138 ceph-mon[36954]: pgmap v4925: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:03:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:04 smithi167 ceph-mon[37140]: pgmap v4925: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:03:05.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:03:05.524+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:03:05.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:03:05.524+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:03:05.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:03:05.524+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:03:05.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:03:05.524+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:03:05.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:03:05.524+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:03:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:06 smithi138 ceph-mon[36954]: pgmap v4926: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:03:06.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:06 smithi100 ceph-mon[32599]: pgmap v4926: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:03:06.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:06 smithi167 ceph-mon[37140]: pgmap v4926: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:03:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:08 smithi138 ceph-mon[36954]: pgmap v4927: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:03:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:08 smithi138 ceph-mon[36954]: osdmap e356: 12 total, 12 up, 12 in 2024-07-26T16:03:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:08 smithi100 ceph-mon[32599]: pgmap v4927: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:03:08.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:08 smithi100 ceph-mon[32599]: osdmap e356: 12 total, 12 up, 12 in 2024-07-26T16:03:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:08 smithi167 ceph-mon[37140]: pgmap v4927: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:03:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:08 smithi167 ceph-mon[37140]: osdmap e356: 12 total, 12 up, 12 in 2024-07-26T16:03:10.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:10 smithi138 ceph-mon[36954]: pgmap v4929: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:03:10.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:10 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:03:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:10 smithi100 ceph-mon[32599]: pgmap v4929: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:03:10.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:10 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:03:10.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:10 smithi167 ceph-mon[37140]: pgmap v4929: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:03:10.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:10 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:03:12.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:03:12.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:03:12.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:03:12.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:03:12.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:03:12.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:03:12.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:03:12.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:03:12.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:03:12.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:03:12.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:03:12.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:03:12.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:03:12.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:03:12.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:03:12.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:03:12.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:03:12.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:03:12.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:03:12.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:03:12.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:03:12.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:03:12.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:03:12.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:03:12.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:03:12.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:03:12.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:03:13.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:12 smithi138 ceph-mon[36954]: pgmap v4930: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:03:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:12 smithi100 ceph-mon[32599]: pgmap v4930: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:03:13.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:12 smithi167 ceph-mon[37140]: pgmap v4930: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:03:15.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:14 smithi138 ceph-mon[36954]: pgmap v4931: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:03:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:14 smithi100 ceph-mon[32599]: pgmap v4931: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:03:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:14 smithi167 ceph-mon[37140]: pgmap v4931: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:03:16.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:15 smithi100 ceph-mon[32599]: pgmap v4932: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:03:16.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:15 smithi100 ceph-mon[32599]: osdmap e357: 12 total, 12 up, 12 in 2024-07-26T16:03:16.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:15 smithi167 ceph-mon[37140]: pgmap v4932: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:03:16.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:15 smithi167 ceph-mon[37140]: osdmap e357: 12 total, 12 up, 12 in 2024-07-26T16:03:16.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:15 smithi138 ceph-mon[36954]: pgmap v4932: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:03:16.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:15 smithi138 ceph-mon[36954]: osdmap e357: 12 total, 12 up, 12 in 2024-07-26T16:03:18.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:18 smithi138 ceph-mon[36954]: pgmap v4934: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T16:03:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:18 smithi100 ceph-mon[32599]: pgmap v4934: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T16:03:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:18 smithi167 ceph-mon[37140]: pgmap v4934: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T16:03:20.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:03:20.540+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:03:20.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:03:20.540+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:03:20.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:03:20.540+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:03:20.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:03:20.540+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:03:20.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:03:20.540+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:03:20.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:20 smithi138 ceph-mon[36954]: pgmap v4935: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:03:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:20 smithi100 ceph-mon[32599]: pgmap v4935: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:03:20.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:20 smithi167 ceph-mon[37140]: pgmap v4935: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:03:22.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:22 smithi138 ceph-mon[36954]: pgmap v4936: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:03:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:22 smithi100 ceph-mon[32599]: pgmap v4936: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:03:22.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:22 smithi167 ceph-mon[37140]: pgmap v4936: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:03:24.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:24 smithi138 ceph-mon[36954]: pgmap v4937: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:03:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:24 smithi100 ceph-mon[32599]: pgmap v4937: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:03:24.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:24 smithi167 ceph-mon[37140]: pgmap v4937: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:03:26.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:26 smithi138 ceph-mon[36954]: pgmap v4938: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:03:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:26 smithi100 ceph-mon[32599]: pgmap v4938: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:03:26.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:26 smithi167 ceph-mon[37140]: pgmap v4938: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:03:28.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:28 smithi138 ceph-mon[36954]: pgmap v4939: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:03:28.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:28 smithi100 ceph-mon[32599]: pgmap v4939: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:03:28.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:28 smithi167 ceph-mon[37140]: pgmap v4939: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:03:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:30 smithi138 ceph-mon[36954]: pgmap v4940: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:03:30.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:30 smithi100 ceph-mon[32599]: pgmap v4940: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:03:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:30 smithi167 ceph-mon[37140]: pgmap v4940: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:03:32.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:31 smithi100 ceph-mon[32599]: pgmap v4941: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:03:32.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:31 smithi167 ceph-mon[37140]: pgmap v4941: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:03:32.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:31 smithi138 ceph-mon[36954]: pgmap v4941: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:03:34.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:34 smithi138 ceph-mon[36954]: pgmap v4942: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:03:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:34 smithi100 ceph-mon[32599]: pgmap v4942: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:03:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:34 smithi167 ceph-mon[37140]: pgmap v4942: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:03:35.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:03:35.543+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:03:35.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:03:35.544+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:03:35.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:03:35.544+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:03:35.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:03:35.544+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:03:35.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:03:35.544+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:03:36.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:36 smithi138 ceph-mon[36954]: pgmap v4943: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:03:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:36 smithi100 ceph-mon[32599]: pgmap v4943: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:03:36.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:36 smithi167 ceph-mon[37140]: pgmap v4943: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:03:38.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:38 smithi138 ceph-mon[36954]: pgmap v4944: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:03:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:38 smithi100 ceph-mon[32599]: pgmap v4944: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:03:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:38 smithi167 ceph-mon[37140]: pgmap v4944: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:03:40.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:40 smithi138 ceph-mon[36954]: pgmap v4945: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:03:40.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:40 smithi100 ceph-mon[32599]: pgmap v4945: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:03:40.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:40 smithi167 ceph-mon[37140]: pgmap v4945: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:03:42.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:42 smithi138 ceph-mon[36954]: pgmap v4946: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:03:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:42 smithi100 ceph-mon[32599]: pgmap v4946: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:03:42.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:42 smithi167 ceph-mon[37140]: pgmap v4946: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:03:44.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:44 smithi138 ceph-mon[36954]: pgmap v4947: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:03:44.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:44 smithi100 ceph-mon[32599]: pgmap v4947: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:03:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:44 smithi167 ceph-mon[37140]: pgmap v4947: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:03:46.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:46 smithi138 ceph-mon[36954]: pgmap v4948: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:03:46.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:46 smithi100 ceph-mon[32599]: pgmap v4948: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:03:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:46 smithi167 ceph-mon[37140]: pgmap v4948: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:03:48.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:48 smithi138 ceph-mon[36954]: pgmap v4949: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:03:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:48 smithi100 ceph-mon[32599]: pgmap v4949: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:03:48.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:48 smithi167 ceph-mon[37140]: pgmap v4949: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:03:50.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:03:50.558+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:03:50.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:03:50.559+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:03:50.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:03:50.559+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:03:50.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:03:50.559+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:03:50.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:03:50.559+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:03:50.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:50 smithi138 ceph-mon[36954]: pgmap v4950: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:03:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:50 smithi100 ceph-mon[32599]: pgmap v4950: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:03:50.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:50 smithi167 ceph-mon[37140]: pgmap v4950: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:03:52.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:52 smithi138 ceph-mon[36954]: pgmap v4951: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:03:52.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:52 smithi100 ceph-mon[32599]: pgmap v4951: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:03:52.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:52 smithi167 ceph-mon[37140]: pgmap v4951: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:03:54.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:54 smithi138 ceph-mon[36954]: pgmap v4952: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:03:54.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:54 smithi100 ceph-mon[32599]: pgmap v4952: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:03:54.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:54 smithi167 ceph-mon[37140]: pgmap v4952: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:03:56.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:56 smithi138 ceph-mon[36954]: pgmap v4953: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:03:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:56 smithi100 ceph-mon[32599]: pgmap v4953: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:03:56.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:56 smithi167 ceph-mon[37140]: pgmap v4953: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:03:58.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:03:58 smithi138 ceph-mon[36954]: pgmap v4954: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:03:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:03:58 smithi100 ceph-mon[32599]: pgmap v4954: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:03:58.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:03:58 smithi167 ceph-mon[37140]: pgmap v4954: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:04:00.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:04:00 smithi138 ceph-mon[36954]: pgmap v4955: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:04:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:04:00 smithi100 ceph-mon[32599]: pgmap v4955: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:04:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:04:00 smithi167 ceph-mon[37140]: pgmap v4955: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:04:02.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:04:02 smithi138 ceph-mon[36954]: pgmap v4956: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:04:02.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:04:02 smithi100 ceph-mon[32599]: pgmap v4956: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:04:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:04:02 smithi167 ceph-mon[37140]: pgmap v4956: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:04:04.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:04:04 smithi138 ceph-mon[36954]: pgmap v4957: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:04:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:04:04 smithi100 ceph-mon[32599]: pgmap v4957: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:04:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:04:04 smithi167 ceph-mon[37140]: pgmap v4957: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:04:05.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:04:05.575+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:04:05.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:04:05.576+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:04:05.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:04:05.576+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:04:05.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:04:05.576+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:04:05.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:04:05.576+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:04:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:04:06 smithi138 ceph-mon[36954]: pgmap v4958: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:04:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:04:06 smithi100 ceph-mon[32599]: pgmap v4958: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:04:06.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:04:06 smithi167 ceph-mon[37140]: pgmap v4958: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:04:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:04:08 smithi138 ceph-mon[36954]: pgmap v4959: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:04:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:04:08 smithi138 ceph-mon[36954]: osdmap e358: 12 total, 12 up, 12 in 2024-07-26T16:04:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:04:08 smithi100 ceph-mon[32599]: pgmap v4959: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:04:08.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:04:08 smithi100 ceph-mon[32599]: osdmap e358: 12 total, 12 up, 12 in 2024-07-26T16:04:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:04:08 smithi167 ceph-mon[37140]: pgmap v4959: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:04:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:04:08 smithi167 ceph-mon[37140]: osdmap e358: 12 total, 12 up, 12 in 2024-07-26T16:04:10.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:04:10 smithi138 ceph-mon[36954]: pgmap v4961: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:04:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:04:10 smithi100 ceph-mon[32599]: pgmap v4961: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:04:10.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:04:10 smithi167 ceph-mon[37140]: pgmap v4961: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:04:11.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:04:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:04:11.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:04:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:04:11.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:04:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:04:12.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:04:12 smithi138 ceph-mon[36954]: pgmap v4962: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:04:12.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:04:12 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:04:12.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:04:12 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:04:12.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:04:12 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:04:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:04:12 smithi100 ceph-mon[32599]: pgmap v4962: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:04:12.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:04:12 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:04:12.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:04:12 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:04:12.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:04:12 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:04:12.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:04:12 smithi167 ceph-mon[37140]: pgmap v4962: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:04:12.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:04:12 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:04:12.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:04:12 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:04:12.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:04:12 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:04:14.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:04:14 smithi138 ceph-mon[36954]: pgmap v4963: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:04:14.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:04:14 smithi100 ceph-mon[32599]: pgmap v4963: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:04:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:04:14 smithi167 ceph-mon[37140]: pgmap v4963: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:04:16.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:04:16 smithi138 ceph-mon[36954]: pgmap v4964: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:04:16.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:04:16 smithi138 ceph-mon[36954]: osdmap e359: 12 total, 12 up, 12 in 2024-07-26T16:04:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:04:16 smithi100 ceph-mon[32599]: pgmap v4964: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:04:16.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:04:16 smithi100 ceph-mon[32599]: osdmap e359: 12 total, 12 up, 12 in 2024-07-26T16:04:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:04:16 smithi167 ceph-mon[37140]: pgmap v4964: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:04:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:04:16 smithi167 ceph-mon[37140]: osdmap e359: 12 total, 12 up, 12 in 2024-07-26T16:04:18.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:04:18 smithi138 ceph-mon[36954]: pgmap v4966: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:04:18.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:04:18 smithi100 ceph-mon[32599]: pgmap v4966: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:04:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:04:18 smithi167 ceph-mon[37140]: pgmap v4966: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:04:20.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:04:20.586+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:04:20.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:04:20.586+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:04:20.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:04:20.586+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:04:20.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:04:20.586+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:04:20.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:04:20.586+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:04:20.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:04:20 smithi138 ceph-mon[36954]: pgmap v4967: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:04:20.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:04:20 smithi100 ceph-mon[32599]: pgmap v4967: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:04:20.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:04:20 smithi167 ceph-mon[37140]: pgmap v4967: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:04:22.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:04:22 smithi138 ceph-mon[36954]: pgmap v4968: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:04:22.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:04:22 smithi100 ceph-mon[32599]: pgmap v4968: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:04:22.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:04:22 smithi167 ceph-mon[37140]: pgmap v4968: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:04:24.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:04:24 smithi138 ceph-mon[36954]: pgmap v4969: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:04:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:04:24 smithi100 ceph-mon[32599]: pgmap v4969: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:04:24.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:04:24 smithi167 ceph-mon[37140]: pgmap v4969: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:04:26.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:04:26 smithi138 ceph-mon[36954]: pgmap v4970: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:04:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:04:26 smithi100 ceph-mon[32599]: pgmap v4970: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:04:26.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:04:26 smithi167 ceph-mon[37140]: pgmap v4970: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:04:28.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:04:28 smithi138 ceph-mon[36954]: pgmap v4971: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:04:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:04:28 smithi100 ceph-mon[32599]: pgmap v4971: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:04:28.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:04:28 smithi167 ceph-mon[37140]: pgmap v4971: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:04:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:04:30 smithi138 ceph-mon[36954]: pgmap v4972: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:04:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:04:30 smithi100 ceph-mon[32599]: pgmap v4972: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:04:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:04:30 smithi167 ceph-mon[37140]: pgmap v4972: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:04:32.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:04:31 smithi100 ceph-mon[32599]: pgmap v4973: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:04:32.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:04:31 smithi167 ceph-mon[37140]: pgmap v4973: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:04:32.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:04:31 smithi138 ceph-mon[36954]: pgmap v4973: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:04:34.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:04:34 smithi138 ceph-mon[36954]: pgmap v4974: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:04:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:04:34 smithi100 ceph-mon[32599]: pgmap v4974: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:04:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:04:34 smithi167 ceph-mon[37140]: pgmap v4974: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:04:35.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:04:35.599+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:04:35.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:04:35.600+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:04:35.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:04:35.600+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:04:35.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:04:35.600+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:04:35.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:04:35.600+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:04:36.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:04:36 smithi138 ceph-mon[36954]: pgmap v4975: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:04:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:04:36 smithi100 ceph-mon[32599]: pgmap v4975: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:04:36.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:04:36 smithi167 ceph-mon[37140]: pgmap v4975: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:04:38.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:04:38 smithi138 ceph-mon[36954]: pgmap v4976: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:04:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:04:38 smithi100 ceph-mon[32599]: pgmap v4976: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:04:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:04:38 smithi167 ceph-mon[37140]: pgmap v4976: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:04:40.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:04:40 smithi138 ceph-mon[36954]: pgmap v4977: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:04:40.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:04:40 smithi100 ceph-mon[32599]: pgmap v4977: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:04:40.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:04:40 smithi167 ceph-mon[37140]: pgmap v4977: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:04:42.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:04:42 smithi138 ceph-mon[36954]: pgmap v4978: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:04:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:04:42 smithi100 ceph-mon[32599]: pgmap v4978: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:04:42.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:04:42 smithi167 ceph-mon[37140]: pgmap v4978: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:04:44.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:04:44 smithi138 ceph-mon[36954]: pgmap v4979: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:04:44.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:04:44 smithi100 ceph-mon[32599]: pgmap v4979: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:04:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:04:44 smithi167 ceph-mon[37140]: pgmap v4979: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:04:46.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:04:46 smithi138 ceph-mon[36954]: pgmap v4980: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:04:46.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:04:46 smithi100 ceph-mon[32599]: pgmap v4980: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:04:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:04:46 smithi167 ceph-mon[37140]: pgmap v4980: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:04:48.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:04:48 smithi138 ceph-mon[36954]: pgmap v4981: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:04:48.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:04:48 smithi100 ceph-mon[32599]: pgmap v4981: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:04:48.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:04:48 smithi167 ceph-mon[37140]: pgmap v4981: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:04:50.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:04:50.615+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:04:50.618 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:04:50 smithi100 ceph-mon[32599]: pgmap v4982: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:04:50.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:04:50.616+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:04:50.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:04:50.616+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:04:50.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:04:50.616+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:04:50.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:04:50.616+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:04:50.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:04:50 smithi138 ceph-mon[36954]: pgmap v4982: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:04:50.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:04:50 smithi167 ceph-mon[37140]: pgmap v4982: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:04:52.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:04:52 smithi138 ceph-mon[36954]: pgmap v4983: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:04:52.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:04:52 smithi100 ceph-mon[32599]: pgmap v4983: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:04:52.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:04:52 smithi167 ceph-mon[37140]: pgmap v4983: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:04:54.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:04:54 smithi138 ceph-mon[36954]: pgmap v4984: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:04:54.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:04:54 smithi100 ceph-mon[32599]: pgmap v4984: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:04:54.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:04:54 smithi167 ceph-mon[37140]: pgmap v4984: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:04:56.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:04:56 smithi138 ceph-mon[36954]: pgmap v4985: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:04:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:04:56 smithi100 ceph-mon[32599]: pgmap v4985: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:04:56.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:04:56 smithi167 ceph-mon[37140]: pgmap v4985: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:04:58.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:04:58 smithi138 ceph-mon[36954]: pgmap v4986: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:04:58.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:04:58 smithi100 ceph-mon[32599]: pgmap v4986: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:04:58.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:04:58 smithi167 ceph-mon[37140]: pgmap v4986: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:05:00.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:05:00 smithi138 ceph-mon[36954]: pgmap v4987: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:05:00.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:05:00 smithi100 ceph-mon[32599]: pgmap v4987: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:05:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:05:00 smithi167 ceph-mon[37140]: pgmap v4987: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:05:02.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:05:02 smithi138 ceph-mon[36954]: pgmap v4988: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T16:05:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:05:02 smithi100 ceph-mon[32599]: pgmap v4988: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T16:05:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:05:02 smithi167 ceph-mon[37140]: pgmap v4988: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T16:05:04.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:05:04 smithi138 ceph-mon[36954]: pgmap v4989: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-07-26T16:05:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:05:04 smithi100 ceph-mon[32599]: pgmap v4989: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-07-26T16:05:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:05:04 smithi167 ceph-mon[37140]: pgmap v4989: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 11 KiB/s wr, 7 op/s 2024-07-26T16:05:05.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:05:05.630+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:05:05.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:05:05.631+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:05:05.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:05:05.631+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:05:05.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:05:05.631+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:05:05.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:05:05.631+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:05:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:05:06 smithi138 ceph-mon[36954]: pgmap v4990: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-07-26T16:05:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:05:06 smithi100 ceph-mon[32599]: pgmap v4990: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-07-26T16:05:06.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:05:06 smithi167 ceph-mon[37140]: pgmap v4990: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-07-26T16:05:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:05:08 smithi138 ceph-mon[36954]: pgmap v4991: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-07-26T16:05:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:05:08 smithi100 ceph-mon[32599]: pgmap v4991: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-07-26T16:05:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:05:08 smithi167 ceph-mon[37140]: pgmap v4991: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-07-26T16:05:10.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:05:10 smithi138 ceph-mon[36954]: pgmap v4992: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-07-26T16:05:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:05:10 smithi100 ceph-mon[32599]: pgmap v4992: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-07-26T16:05:10.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:05:10 smithi167 ceph-mon[37140]: pgmap v4992: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-07-26T16:05:12.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:05:12 smithi138 ceph-mon[36954]: pgmap v4993: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-07-26T16:05:12.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:05:12 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:05:12.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:05:12 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:05:12.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:05:12 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:05:12.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:05:12 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:05:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:05:12 smithi100 ceph-mon[32599]: pgmap v4993: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-07-26T16:05:12.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:05:12 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:05:12.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:05:12 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:05:12.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:05:12 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:05:12.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:05:12 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:05:12.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:05:12 smithi167 ceph-mon[37140]: pgmap v4993: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-07-26T16:05:12.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:05:12 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:05:12.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:05:12 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:05:12.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:05:12 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:05:12.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:05:12 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:05:14.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:05:14 smithi138 ceph-mon[36954]: pgmap v4994: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-07-26T16:05:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:05:14 smithi100 ceph-mon[32599]: pgmap v4994: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-07-26T16:05:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:05:14 smithi167 ceph-mon[37140]: pgmap v4994: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-07-26T16:05:16.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:05:16 smithi138 ceph-mon[36954]: pgmap v4995: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-07-26T16:05:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:05:16 smithi100 ceph-mon[32599]: pgmap v4995: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-07-26T16:05:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:05:16 smithi167 ceph-mon[37140]: pgmap v4995: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-07-26T16:05:18.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:05:18 smithi138 ceph-mon[36954]: pgmap v4996: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-07-26T16:05:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:05:18 smithi100 ceph-mon[32599]: pgmap v4996: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-07-26T16:05:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:05:18 smithi167 ceph-mon[37140]: pgmap v4996: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-07-26T16:05:20.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:05:20.646+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:05:20.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:05:20.647+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:05:20.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:05:20.647+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:05:20.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:05:20.647+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:05:20.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:05:20.647+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:05:20.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:05:20 smithi138 ceph-mon[36954]: pgmap v4997: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-07-26T16:05:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:05:20 smithi100 ceph-mon[32599]: pgmap v4997: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-07-26T16:05:20.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:05:20 smithi167 ceph-mon[37140]: pgmap v4997: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-07-26T16:05:22.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:05:22 smithi138 ceph-mon[36954]: pgmap v4998: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:05:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:05:22 smithi100 ceph-mon[32599]: pgmap v4998: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:05:22.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:05:22 smithi167 ceph-mon[37140]: pgmap v4998: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:05:24.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:05:24 smithi138 ceph-mon[36954]: pgmap v4999: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:05:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:05:24 smithi100 ceph-mon[32599]: pgmap v4999: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:05:24.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:05:24 smithi167 ceph-mon[37140]: pgmap v4999: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:05:26.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:05:26 smithi138 ceph-mon[36954]: pgmap v5000: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:05:26.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:05:26 smithi100 ceph-mon[32599]: pgmap v5000: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:05:26.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:05:26 smithi167 ceph-mon[37140]: pgmap v5000: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:05:28.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:05:28 smithi138 ceph-mon[36954]: pgmap v5001: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:05:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:05:28 smithi100 ceph-mon[32599]: pgmap v5001: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:05:28.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:05:28 smithi167 ceph-mon[37140]: pgmap v5001: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:05:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:05:30 smithi138 ceph-mon[36954]: pgmap v5002: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:05:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:05:30 smithi100 ceph-mon[32599]: pgmap v5002: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:05:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:05:30 smithi167 ceph-mon[37140]: pgmap v5002: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:05:32.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:05:32 smithi100 ceph-mon[32599]: pgmap v5003: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:05:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:05:32 smithi167 ceph-mon[37140]: pgmap v5003: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:05:32.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:05:32 smithi138 ceph-mon[36954]: pgmap v5003: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:05:34.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:05:34 smithi100 ceph-mon[32599]: pgmap v5004: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:05:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:05:34 smithi167 ceph-mon[37140]: pgmap v5004: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:05:34.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:05:34 smithi138 ceph-mon[36954]: pgmap v5004: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:05:35.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:05:35.662+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:05:35.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:05:35.663+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:05:35.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:05:35.663+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:05:35.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:05:35.663+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:05:35.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:05:35.663+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:05:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:05:36 smithi100 ceph-mon[32599]: pgmap v5005: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:05:36.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:05:36 smithi167 ceph-mon[37140]: pgmap v5005: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:05:36.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:05:36 smithi138 ceph-mon[36954]: pgmap v5005: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:05:38.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:05:38 smithi100 ceph-mon[32599]: pgmap v5006: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:05:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:05:38 smithi167 ceph-mon[37140]: pgmap v5006: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:05:38.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:05:38 smithi138 ceph-mon[36954]: pgmap v5006: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:05:40.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:05:40 smithi100 ceph-mon[32599]: pgmap v5007: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:05:40.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:05:40 smithi167 ceph-mon[37140]: pgmap v5007: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:05:40.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:05:40 smithi138 ceph-mon[36954]: pgmap v5007: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:05:42.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:05:42 smithi100 ceph-mon[32599]: pgmap v5008: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:05:42.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:05:42 smithi167 ceph-mon[37140]: pgmap v5008: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:05:42.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:05:42 smithi138 ceph-mon[36954]: pgmap v5008: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:05:44.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:05:44 smithi100 ceph-mon[32599]: pgmap v5009: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:05:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:05:44 smithi167 ceph-mon[37140]: pgmap v5009: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:05:44.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:05:44 smithi138 ceph-mon[36954]: pgmap v5009: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:05:46.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:05:46 smithi100 ceph-mon[32599]: pgmap v5010: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:05:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:05:46 smithi167 ceph-mon[37140]: pgmap v5010: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:05:46.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:05:46 smithi138 ceph-mon[36954]: pgmap v5010: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:05:48.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:05:48 smithi100 ceph-mon[32599]: pgmap v5011: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:05:48.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:05:48 smithi167 ceph-mon[37140]: pgmap v5011: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:05:48.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:05:48 smithi138 ceph-mon[36954]: pgmap v5011: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:05:50.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:05:50.678+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:05:50.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:05:50.679+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:05:50.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:05:50.679+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:05:50.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:05:50.679+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:05:50.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:05:50.679+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:05:50.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:05:50 smithi100 ceph-mon[32599]: pgmap v5012: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:05:50.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:05:50 smithi167 ceph-mon[37140]: pgmap v5012: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:05:50.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:05:50 smithi138 ceph-mon[36954]: pgmap v5012: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:05:52.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:05:52 smithi100 ceph-mon[32599]: pgmap v5013: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:05:52.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:05:52 smithi167 ceph-mon[37140]: pgmap v5013: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:05:52.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:05:52 smithi138 ceph-mon[36954]: pgmap v5013: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:05:54.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:05:54 smithi100 ceph-mon[32599]: pgmap v5014: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:05:54.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:05:54 smithi167 ceph-mon[37140]: pgmap v5014: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:05:54.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:05:54 smithi138 ceph-mon[36954]: pgmap v5014: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:05:56.743 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:05:56 smithi138 ceph-mon[36954]: pgmap v5015: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:05:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:05:56 smithi100 ceph-mon[32599]: pgmap v5015: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:05:56.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:05:56 smithi167 ceph-mon[37140]: pgmap v5015: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:05:58.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:05:58 smithi100 ceph-mon[32599]: pgmap v5016: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:05:58.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:05:58 smithi167 ceph-mon[37140]: pgmap v5016: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:05:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:05:58 smithi138 ceph-mon[36954]: pgmap v5016: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:06:00.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:06:00 smithi100 ceph-mon[32599]: pgmap v5017: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:06:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:06:00 smithi167 ceph-mon[37140]: pgmap v5017: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:06:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:06:00 smithi138 ceph-mon[36954]: pgmap v5017: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:06:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:06:02 smithi100 ceph-mon[32599]: pgmap v5018: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:06:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:06:02 smithi167 ceph-mon[37140]: pgmap v5018: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:06:02.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:06:02 smithi138 ceph-mon[36954]: pgmap v5018: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:06:04.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:06:04.186+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T16:06:04.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:06:04.186+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T16:06:04.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:06:04.186+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 67 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T16:06:04.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:06:04.186+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 67 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c020 con 0x7fe5a81087a0 2024-07-26T16:06:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:06:04 smithi100 ceph-mon[32599]: pgmap v5019: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:06:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:06:04 smithi167 ceph-mon[37140]: pgmap v5019: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:06:04.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:06:04 smithi138 ceph-mon[36954]: pgmap v5019: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:06:05.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:06:05.684+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:06:05.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:06:05.684+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:06:05.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:06:05.684+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:06:05.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:06:05.684+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:06:05.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:06:05.684+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:06:06.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:06:06 smithi167 ceph-mon[37140]: pgmap v5020: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:06:06.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:06:06 smithi138 ceph-mon[36954]: pgmap v5020: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:06:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:06:06 smithi100 ceph-mon[32599]: pgmap v5020: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:06:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:06:08 smithi167 ceph-mon[37140]: pgmap v5021: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:06:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:06:08 smithi167 ceph-mon[37140]: osdmap e360: 12 total, 12 up, 12 in 2024-07-26T16:06:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:06:08 smithi138 ceph-mon[36954]: pgmap v5021: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:06:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:06:08 smithi138 ceph-mon[36954]: osdmap e360: 12 total, 12 up, 12 in 2024-07-26T16:06:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:06:08 smithi100 ceph-mon[32599]: pgmap v5021: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:06:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:06:08 smithi100 ceph-mon[32599]: osdmap e360: 12 total, 12 up, 12 in 2024-07-26T16:06:10.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:06:10 smithi167 ceph-mon[37140]: pgmap v5023: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:06:10.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:06:10 smithi138 ceph-mon[36954]: pgmap v5023: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:06:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:06:10 smithi100 ceph-mon[32599]: pgmap v5023: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:06:12.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:06:12 smithi167 ceph-mon[37140]: pgmap v5024: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:06:12.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:06:12 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:06:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:06:12 smithi138 ceph-mon[36954]: pgmap v5024: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:06:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:06:12 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:06:12.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:06:12 smithi100 ceph-mon[32599]: pgmap v5024: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:06:12.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:06:12 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:06:13.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:06:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:06:13.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:06:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:06:13.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:06:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:06:13.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:06:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:06:13.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:06:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:06:13.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:06:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:06:13.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:06:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:06:13.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:06:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:06:13.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:06:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:06:14.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:06:14 smithi138 ceph-mon[36954]: pgmap v5025: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:06:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:06:14 smithi100 ceph-mon[32599]: pgmap v5025: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:06:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:06:14 smithi167 ceph-mon[37140]: pgmap v5025: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:06:16.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:06:16 smithi100 ceph-mon[32599]: pgmap v5026: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:06:16.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:06:16 smithi100 ceph-mon[32599]: osdmap e361: 12 total, 12 up, 12 in 2024-07-26T16:06:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:06:16 smithi138 ceph-mon[36954]: pgmap v5026: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:06:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:06:16 smithi138 ceph-mon[36954]: osdmap e361: 12 total, 12 up, 12 in 2024-07-26T16:06:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:06:16 smithi167 ceph-mon[37140]: pgmap v5026: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:06:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:06:16 smithi167 ceph-mon[37140]: osdmap e361: 12 total, 12 up, 12 in 2024-07-26T16:06:18.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:06:18 smithi138 ceph-mon[36954]: pgmap v5028: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 208 B/s wr, 6 op/s 2024-07-26T16:06:18.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:06:18 smithi100 ceph-mon[32599]: pgmap v5028: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 208 B/s wr, 6 op/s 2024-07-26T16:06:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:06:18 smithi167 ceph-mon[37140]: pgmap v5028: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 208 B/s wr, 6 op/s 2024-07-26T16:06:20.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:06:20.700+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:06:20.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:06:20.700+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:06:20.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:06:20.700+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:06:20.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:06:20.700+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:06:20.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:06:20.700+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:06:20.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:06:20 smithi138 ceph-mon[36954]: pgmap v5029: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:06:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:06:20 smithi100 ceph-mon[32599]: pgmap v5029: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:06:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:06:20 smithi167 ceph-mon[37140]: pgmap v5029: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:06:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:06:22 smithi138 ceph-mon[36954]: pgmap v5030: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:06:22.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:06:22 smithi100 ceph-mon[32599]: pgmap v5030: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:06:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:06:22 smithi167 ceph-mon[37140]: pgmap v5030: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:06:24.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:06:24 smithi138 ceph-mon[36954]: pgmap v5031: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:06:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:06:24 smithi100 ceph-mon[32599]: pgmap v5031: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:06:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:06:24 smithi167 ceph-mon[37140]: pgmap v5031: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:06:26.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:06:26 smithi138 ceph-mon[36954]: pgmap v5032: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:06:26.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:06:26 smithi100 ceph-mon[32599]: pgmap v5032: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:06:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:06:26 smithi167 ceph-mon[37140]: pgmap v5032: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:06:28.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:06:28 smithi138 ceph-mon[36954]: pgmap v5033: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:06:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:06:28 smithi100 ceph-mon[32599]: pgmap v5033: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:06:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:06:28 smithi167 ceph-mon[37140]: pgmap v5033: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:06:30.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:06:30 smithi138 ceph-mon[36954]: pgmap v5034: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:06:30.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:06:30 smithi100 ceph-mon[32599]: pgmap v5034: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:06:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:06:30 smithi167 ceph-mon[37140]: pgmap v5034: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:06:32.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:06:32 smithi138 ceph-mon[36954]: pgmap v5035: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:06:32.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:06:32 smithi100 ceph-mon[32599]: pgmap v5035: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:06:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:06:32 smithi167 ceph-mon[37140]: pgmap v5035: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:06:34.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:06:34 smithi138 ceph-mon[36954]: pgmap v5036: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:06:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:06:34 smithi100 ceph-mon[32599]: pgmap v5036: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:06:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:06:34 smithi167 ceph-mon[37140]: pgmap v5036: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:06:35.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:06:35.702+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:06:35.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:06:35.703+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:06:35.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:06:35.703+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:06:35.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:06:35.703+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:06:35.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:06:35.703+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:06:36.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:06:36 smithi138 ceph-mon[36954]: pgmap v5037: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:06:36.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:06:36 smithi100 ceph-mon[32599]: pgmap v5037: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:06:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:06:36 smithi167 ceph-mon[37140]: pgmap v5037: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:06:38.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:06:38 smithi138 ceph-mon[36954]: pgmap v5038: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:06:38.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:06:38 smithi100 ceph-mon[32599]: pgmap v5038: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:06:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:06:38 smithi167 ceph-mon[37140]: pgmap v5038: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:06:40.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:06:40 smithi138 ceph-mon[36954]: pgmap v5039: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:06:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:06:40 smithi100 ceph-mon[32599]: pgmap v5039: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:06:41.027 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:06:40 smithi167 ceph-mon[37140]: pgmap v5039: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:06:42.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:06:42 smithi138 ceph-mon[36954]: pgmap v5040: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:06:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:06:42 smithi100 ceph-mon[32599]: pgmap v5040: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:06:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:06:42 smithi167 ceph-mon[37140]: pgmap v5040: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:06:44.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:06:44 smithi138 ceph-mon[36954]: pgmap v5041: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:06:44.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:06:44 smithi100 ceph-mon[32599]: pgmap v5041: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:06:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:06:44 smithi167 ceph-mon[37140]: pgmap v5041: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:06:46.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:06:46 smithi138 ceph-mon[36954]: pgmap v5042: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:06:46.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:06:46 smithi100 ceph-mon[32599]: pgmap v5042: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:06:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:06:46 smithi167 ceph-mon[37140]: pgmap v5042: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:06:48.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:06:48 smithi138 ceph-mon[36954]: pgmap v5043: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:06:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:06:48 smithi100 ceph-mon[32599]: pgmap v5043: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:06:49.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:06:48 smithi167 ceph-mon[37140]: pgmap v5043: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:06:50.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:06:50.718+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:06:50.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:06:50.719+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:06:50.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:06:50.719+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:06:50.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:06:50.719+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:06:50.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:06:50.719+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:06:50.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:06:50 smithi138 ceph-mon[36954]: pgmap v5044: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:06:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:06:50 smithi100 ceph-mon[32599]: pgmap v5044: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:06:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:06:50 smithi167 ceph-mon[37140]: pgmap v5044: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:06:52.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:06:52 smithi138 ceph-mon[36954]: pgmap v5045: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:06:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:06:52 smithi100 ceph-mon[32599]: pgmap v5045: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:06:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:06:52 smithi167 ceph-mon[37140]: pgmap v5045: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:06:54.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:06:54 smithi138 ceph-mon[36954]: pgmap v5046: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:06:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:06:54 smithi100 ceph-mon[32599]: pgmap v5046: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:06:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:06:54 smithi167 ceph-mon[37140]: pgmap v5046: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:06:56.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:06:56 smithi138 ceph-mon[36954]: pgmap v5047: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:06:56.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:06:56 smithi100 ceph-mon[32599]: pgmap v5047: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:06:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:06:56 smithi167 ceph-mon[37140]: pgmap v5047: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:06:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:06:58 smithi138 ceph-mon[36954]: pgmap v5048: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:06:58.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:06:58 smithi100 ceph-mon[32599]: pgmap v5048: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:06:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:06:58 smithi167 ceph-mon[37140]: pgmap v5048: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:07:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:07:00 smithi138 ceph-mon[36954]: pgmap v5049: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:07:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:07:00 smithi100 ceph-mon[32599]: pgmap v5049: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:07:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:07:00 smithi167 ceph-mon[37140]: pgmap v5049: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:07:02.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:07:02 smithi138 ceph-mon[36954]: pgmap v5050: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:07:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:07:02 smithi100 ceph-mon[32599]: pgmap v5050: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:07:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:07:02 smithi167 ceph-mon[37140]: pgmap v5050: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:07:04.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:07:04 smithi138 ceph-mon[36954]: pgmap v5051: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:07:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:07:04 smithi100 ceph-mon[32599]: pgmap v5051: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:07:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:07:04 smithi167 ceph-mon[37140]: pgmap v5051: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:07:05.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:07:05.734+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:07:05.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:07:05.735+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:07:05.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:07:05.735+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:07:05.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:07:05.735+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:07:05.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:07:05.735+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:07:06.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:07:06 smithi138 ceph-mon[36954]: pgmap v5052: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:07:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:07:06 smithi100 ceph-mon[32599]: pgmap v5052: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:07:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:07:06 smithi167 ceph-mon[37140]: pgmap v5052: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:07:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:07:08 smithi138 ceph-mon[36954]: pgmap v5053: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:07:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:07:08 smithi138 ceph-mon[36954]: osdmap e362: 12 total, 12 up, 12 in 2024-07-26T16:07:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:07:08 smithi100 ceph-mon[32599]: pgmap v5053: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:07:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:07:08 smithi100 ceph-mon[32599]: osdmap e362: 12 total, 12 up, 12 in 2024-07-26T16:07:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:07:08 smithi167 ceph-mon[37140]: pgmap v5053: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:07:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:07:08 smithi167 ceph-mon[37140]: osdmap e362: 12 total, 12 up, 12 in 2024-07-26T16:07:10.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:07:10 smithi138 ceph-mon[36954]: pgmap v5055: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:07:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:07:10 smithi100 ceph-mon[32599]: pgmap v5055: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:07:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:07:10 smithi167 ceph-mon[37140]: pgmap v5055: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:07:12.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:07:12 smithi138 ceph-mon[36954]: pgmap v5056: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:07:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:07:12 smithi100 ceph-mon[32599]: pgmap v5056: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:07:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:07:12 smithi167 ceph-mon[37140]: pgmap v5056: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:07:13.800 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:07:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:07:13.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:07:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:07:14.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:07:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:07:14.832 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:07:14 smithi100 ceph-mon[32599]: pgmap v5057: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:07:14.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:07:14 smithi138 ceph-mon[36954]: pgmap v5057: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:07:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:07:14 smithi167 ceph-mon[37140]: pgmap v5057: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:07:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:07:16 smithi138 ceph-mon[36954]: pgmap v5058: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:07:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:07:16 smithi138 ceph-mon[36954]: osdmap e363: 12 total, 12 up, 12 in 2024-07-26T16:07:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:07:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:07:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:07:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:07:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:07:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:07:16.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:07:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:07:16.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:07:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:07:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:07:16 smithi100 ceph-mon[32599]: pgmap v5058: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:07:16.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:07:16 smithi100 ceph-mon[32599]: osdmap e363: 12 total, 12 up, 12 in 2024-07-26T16:07:16.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:07:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:07:16.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:07:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:07:16.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:07:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:07:16.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:07:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:07:16.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:07:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:07:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:07:16 smithi167 ceph-mon[37140]: pgmap v5058: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:07:17.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:07:16 smithi167 ceph-mon[37140]: osdmap e363: 12 total, 12 up, 12 in 2024-07-26T16:07:17.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:07:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:07:17.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:07:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:07:17.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:07:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:07:17.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:07:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:07:17.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:07:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:07:18.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:07:18 smithi138 ceph-mon[36954]: pgmap v5060: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 210 B/s wr, 6 op/s 2024-07-26T16:07:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:07:18 smithi100 ceph-mon[32599]: pgmap v5060: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 210 B/s wr, 6 op/s 2024-07-26T16:07:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:07:18 smithi167 ceph-mon[37140]: pgmap v5060: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 210 B/s wr, 6 op/s 2024-07-26T16:07:20.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:07:20.751+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:07:20.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:07:20.751+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:07:20.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:07:20.751+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:07:20.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:07:20.751+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:07:20.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:07:20.751+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:07:20.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:07:20 smithi138 ceph-mon[36954]: pgmap v5061: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:07:20.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:07:20 smithi100 ceph-mon[32599]: pgmap v5061: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:07:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:07:20 smithi167 ceph-mon[37140]: pgmap v5061: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:07:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:07:22 smithi138 ceph-mon[36954]: pgmap v5062: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:07:22.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:07:22 smithi100 ceph-mon[32599]: pgmap v5062: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:07:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:07:22 smithi167 ceph-mon[37140]: pgmap v5062: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:07:24.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:07:24 smithi138 ceph-mon[36954]: pgmap v5063: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:07:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:07:24 smithi100 ceph-mon[32599]: pgmap v5063: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:07:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:07:24 smithi167 ceph-mon[37140]: pgmap v5063: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:07:26.931 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:07:26 smithi100 ceph-mon[32599]: pgmap v5064: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:07:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:07:26 smithi138 ceph-mon[36954]: pgmap v5064: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:07:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:07:26 smithi167 ceph-mon[37140]: pgmap v5064: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:07:28.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:07:28 smithi138 ceph-mon[36954]: pgmap v5065: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:07:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:07:28 smithi100 ceph-mon[32599]: pgmap v5065: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:07:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:07:28 smithi167 ceph-mon[37140]: pgmap v5065: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:07:30.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:07:30 smithi138 ceph-mon[36954]: pgmap v5066: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:07:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:07:30 smithi100 ceph-mon[32599]: pgmap v5066: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:07:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:07:30 smithi167 ceph-mon[37140]: pgmap v5066: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:07:32.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:07:32 smithi138 ceph-mon[36954]: pgmap v5067: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:07:32.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:07:32 smithi100 ceph-mon[32599]: pgmap v5067: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:07:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:07:32 smithi167 ceph-mon[37140]: pgmap v5067: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:07:34.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:07:34 smithi138 ceph-mon[36954]: pgmap v5068: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:07:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:07:34 smithi100 ceph-mon[32599]: pgmap v5068: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:07:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:07:34 smithi167 ceph-mon[37140]: pgmap v5068: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:07:35.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:07:35.767+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:07:35.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:07:35.767+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:07:35.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:07:35.767+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:07:35.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:07:35.767+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:07:35.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:07:35.767+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:07:36.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:07:36 smithi138 ceph-mon[36954]: pgmap v5069: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:07:36.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:07:36 smithi100 ceph-mon[32599]: pgmap v5069: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:07:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:07:36 smithi167 ceph-mon[37140]: pgmap v5069: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:07:38.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:07:38 smithi138 ceph-mon[36954]: pgmap v5070: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:07:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:07:38 smithi100 ceph-mon[32599]: pgmap v5070: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:07:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:07:38 smithi167 ceph-mon[37140]: pgmap v5070: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:07:40.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:07:40 smithi100 ceph-mon[32599]: pgmap v5071: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:07:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:07:40 smithi167 ceph-mon[37140]: pgmap v5071: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:07:41.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:07:40 smithi138 ceph-mon[36954]: pgmap v5071: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:07:42.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:07:42 smithi100 ceph-mon[32599]: pgmap v5072: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:07:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:07:42 smithi167 ceph-mon[37140]: pgmap v5072: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:07:43.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:07:42 smithi138 ceph-mon[36954]: pgmap v5072: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:07:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:07:44 smithi100 ceph-mon[32599]: pgmap v5073: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:07:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:07:44 smithi167 ceph-mon[37140]: pgmap v5073: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:07:45.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:07:44 smithi138 ceph-mon[36954]: pgmap v5073: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:07:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:07:46 smithi100 ceph-mon[32599]: pgmap v5074: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:07:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:07:46 smithi167 ceph-mon[37140]: pgmap v5074: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:07:47.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:07:46 smithi138 ceph-mon[36954]: pgmap v5074: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:07:48.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:07:48 smithi100 ceph-mon[32599]: pgmap v5075: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:07:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:07:48 smithi167 ceph-mon[37140]: pgmap v5075: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:07:49.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:07:48 smithi138 ceph-mon[36954]: pgmap v5075: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:07:50.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:07:50.783+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:07:50.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:07:50.784+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:07:50.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:07:50.784+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:07:50.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:07:50.784+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:07:50.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:07:50.784+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:07:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:07:50 smithi100 ceph-mon[32599]: pgmap v5076: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:07:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:07:50 smithi167 ceph-mon[37140]: pgmap v5076: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:07:51.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:07:50 smithi138 ceph-mon[36954]: pgmap v5076: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:07:52.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:07:52 smithi100 ceph-mon[32599]: pgmap v5077: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:07:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:07:52 smithi167 ceph-mon[37140]: pgmap v5077: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:07:53.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:07:52 smithi138 ceph-mon[36954]: pgmap v5077: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:07:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:07:54 smithi100 ceph-mon[32599]: pgmap v5078: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:07:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:07:54 smithi167 ceph-mon[37140]: pgmap v5078: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:07:55.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:07:54 smithi138 ceph-mon[36954]: pgmap v5078: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:07:56.993 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:07:56 smithi138 ceph-mon[36954]: pgmap v5079: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:07:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:07:56 smithi100 ceph-mon[32599]: pgmap v5079: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:07:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:07:56 smithi167 ceph-mon[37140]: pgmap v5079: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:07:58.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:07:58 smithi100 ceph-mon[32599]: pgmap v5080: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:07:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:07:58 smithi167 ceph-mon[37140]: pgmap v5080: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:07:59.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:07:58 smithi138 ceph-mon[36954]: pgmap v5080: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:08:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:00 smithi100 ceph-mon[32599]: pgmap v5081: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:08:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:00 smithi167 ceph-mon[37140]: pgmap v5081: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:08:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:00 smithi138 ceph-mon[36954]: pgmap v5081: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:08:02.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:02 smithi100 ceph-mon[32599]: pgmap v5082: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:08:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:02 smithi167 ceph-mon[37140]: pgmap v5082: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:08:03.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:02 smithi138 ceph-mon[36954]: pgmap v5082: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:08:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:04 smithi100 ceph-mon[32599]: pgmap v5083: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:08:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:04 smithi167 ceph-mon[37140]: pgmap v5083: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:08:05.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:04 smithi138 ceph-mon[36954]: pgmap v5083: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:08:05.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:08:05.799+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:08:05.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:08:05.800+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:08:05.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:08:05.800+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:08:05.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:08:05.800+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:08:05.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:08:05.800+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:08:06.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:06 smithi100 ceph-mon[32599]: pgmap v5084: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:08:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:06 smithi167 ceph-mon[37140]: pgmap v5084: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:08:07.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:06 smithi138 ceph-mon[36954]: pgmap v5084: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:08:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:08 smithi100 ceph-mon[32599]: pgmap v5085: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:08:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:08 smithi100 ceph-mon[32599]: osdmap e364: 12 total, 12 up, 12 in 2024-07-26T16:08:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:08 smithi167 ceph-mon[37140]: pgmap v5085: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:08:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:08 smithi167 ceph-mon[37140]: osdmap e364: 12 total, 12 up, 12 in 2024-07-26T16:08:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:08 smithi138 ceph-mon[36954]: pgmap v5085: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:08:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:08 smithi138 ceph-mon[36954]: osdmap e364: 12 total, 12 up, 12 in 2024-07-26T16:08:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:10 smithi100 ceph-mon[32599]: pgmap v5087: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:08:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:10 smithi167 ceph-mon[37140]: pgmap v5087: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:08:11.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:10 smithi138 ceph-mon[36954]: pgmap v5087: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:08:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:12 smithi167 ceph-mon[37140]: pgmap v5088: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:08:13.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:12 smithi138 ceph-mon[36954]: pgmap v5088: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:08:13.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:12 smithi100 ceph-mon[32599]: pgmap v5088: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:08:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:14 smithi167 ceph-mon[37140]: pgmap v5089: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:08:15.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:14 smithi138 ceph-mon[36954]: pgmap v5089: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:08:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:14 smithi100 ceph-mon[32599]: pgmap v5089: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:08:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:16 smithi167 ceph-mon[37140]: pgmap v5090: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:08:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:16 smithi167 ceph-mon[37140]: osdmap e365: 12 total, 12 up, 12 in 2024-07-26T16:08:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:08:17.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:16 smithi100 ceph-mon[32599]: pgmap v5090: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:08:17.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:16 smithi100 ceph-mon[32599]: osdmap e365: 12 total, 12 up, 12 in 2024-07-26T16:08:17.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:08:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:16 smithi138 ceph-mon[36954]: pgmap v5090: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:08:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:16 smithi138 ceph-mon[36954]: osdmap e365: 12 total, 12 up, 12 in 2024-07-26T16:08:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:08:17.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:08:17.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:08:17.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:08:17.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:08:17.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:17 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T16:08:17.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:17 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T16:08:17.996 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:08:17.996 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:08:17.997 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:08:17.997 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:08:17.997 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:17 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T16:08:17.997 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:17 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T16:08:18.063 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:08:18.063 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:08:18.063 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:08:18.063 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:08:18.064 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:17 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T16:08:18.064 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:17 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T16:08:18.903 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:18 smithi138 ceph-mon[36954]: pgmap v5092: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 212 B/s wr, 6 op/s 2024-07-26T16:08:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:18 smithi167 ceph-mon[37140]: pgmap v5092: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 212 B/s wr, 6 op/s 2024-07-26T16:08:19.207 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:18 smithi100 ceph-mon[32599]: pgmap v5092: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 212 B/s wr, 6 op/s 2024-07-26T16:08:19.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:08:19.204+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T16:08:19.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:08:19.204+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T16:08:19.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:08:19.204+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 68 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T16:08:19.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:08:19.204+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 68 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe5a008c070 con 0x7fe5a81087a0 2024-07-26T16:08:20.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:08:20.807+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:08:20.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:08:20.807+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:08:20.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:08:20.807+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:08:20.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:08:20.807+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:08:20.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:08:20.807+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:08:20.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:20 smithi138 ceph-mon[36954]: pgmap v5093: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:08:20.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:20 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:08:20.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:20 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:08:20.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:20 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:08:20.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:20 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:08:20.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:20 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:08:20.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:20 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:08:20.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:20 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:08:20.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:20 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:08:20.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:20 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:08:20.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:20 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:08:20.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:20 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:08:20.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:20 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:08:20.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:20 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:08:20.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:20 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:08:20.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:20 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:08:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:20 smithi167 ceph-mon[37140]: pgmap v5093: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:08:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:20 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:08:21.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:20 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:08:21.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:20 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:08:21.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:20 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:08:21.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:20 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:08:21.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:20 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:08:21.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:20 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:08:21.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:20 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:08:21.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:20 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:08:21.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:20 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:08:21.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:20 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:08:21.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:20 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:08:21.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:20 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:08:21.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:20 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:08:21.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:20 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:08:21.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:20 smithi100 ceph-mon[32599]: pgmap v5093: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:08:21.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:20 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:08:21.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:20 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:08:21.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:20 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:08:21.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:20 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:08:21.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:20 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:08:21.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:20 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:08:21.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:20 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:08:21.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:20 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:08:21.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:20 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:08:21.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:20 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:08:21.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:20 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:08:21.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:20 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:08:21.247 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:20 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:08:21.247 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:20 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:08:21.247 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:20 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:08:22.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:21 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T16:08:22.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:21 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T16:08:22.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:21 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T16:08:22.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:21 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T16:08:22.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:21 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T16:08:22.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:21 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T16:08:22.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:21 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T16:08:22.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:21 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T16:08:22.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:21 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T16:08:22.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:21 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T16:08:22.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:21 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T16:08:22.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:21 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T16:08:23.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:22 smithi138 ceph-mon[36954]: pgmap v5094: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:08:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:22 smithi100 ceph-mon[32599]: pgmap v5094: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:08:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:22 smithi167 ceph-mon[37140]: pgmap v5094: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:08:25.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:24 smithi138 ceph-mon[36954]: pgmap v5095: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:08:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:24 smithi100 ceph-mon[32599]: pgmap v5095: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:08:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:24 smithi167 ceph-mon[37140]: pgmap v5095: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:08:27.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:26 smithi138 ceph-mon[36954]: pgmap v5096: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:08:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:26 smithi100 ceph-mon[32599]: pgmap v5096: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:08:27.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:26 smithi167 ceph-mon[37140]: pgmap v5096: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:08:29.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:28 smithi138 ceph-mon[36954]: pgmap v5097: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:08:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:28 smithi100 ceph-mon[32599]: pgmap v5097: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:08:29.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:28 smithi167 ceph-mon[37140]: pgmap v5097: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:08:31.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:30 smithi138 ceph-mon[36954]: pgmap v5098: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:08:31.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:30 smithi100 ceph-mon[32599]: pgmap v5098: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:08:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:30 smithi167 ceph-mon[37140]: pgmap v5098: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:08:32.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:32 smithi167 ceph-mon[37140]: pgmap v5099: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:08:32.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:32 smithi138 ceph-mon[36954]: pgmap v5099: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:08:32.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:32 smithi100 ceph-mon[32599]: pgmap v5099: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:08:34.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:34 smithi138 ceph-mon[36954]: pgmap v5100: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:08:34.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:34 smithi100 ceph-mon[32599]: pgmap v5100: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:08:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:34 smithi167 ceph-mon[37140]: pgmap v5100: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:08:35.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:08:35.811+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:08:35.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:08:35.812+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:08:35.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:08:35.812+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:08:35.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:08:35.812+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:08:35.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:08:35.812+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:08:36.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:36 smithi138 ceph-mon[36954]: pgmap v5101: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:08:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:36 smithi100 ceph-mon[32599]: pgmap v5101: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:08:36.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:36 smithi167 ceph-mon[37140]: pgmap v5101: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:08:38.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:38 smithi138 ceph-mon[36954]: pgmap v5102: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:08:38.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:38 smithi100 ceph-mon[32599]: pgmap v5102: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:08:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:38 smithi167 ceph-mon[37140]: pgmap v5102: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:08:40.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:40 smithi138 ceph-mon[36954]: pgmap v5103: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:08:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:40 smithi100 ceph-mon[32599]: pgmap v5103: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:08:40.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:40 smithi167 ceph-mon[37140]: pgmap v5103: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:08:42.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:42 smithi138 ceph-mon[36954]: pgmap v5104: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:08:42.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:42 smithi100 ceph-mon[32599]: pgmap v5104: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:08:42.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:42 smithi167 ceph-mon[37140]: pgmap v5104: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:08:44.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:44 smithi138 ceph-mon[36954]: pgmap v5105: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:08:44.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:44 smithi100 ceph-mon[32599]: pgmap v5105: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:08:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:44 smithi167 ceph-mon[37140]: pgmap v5105: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:08:46.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:46 smithi138 ceph-mon[36954]: pgmap v5106: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:08:46.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:46 smithi100 ceph-mon[32599]: pgmap v5106: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:08:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:46 smithi167 ceph-mon[37140]: pgmap v5106: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:08:48.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:48 smithi138 ceph-mon[36954]: pgmap v5107: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:08:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:48 smithi100 ceph-mon[32599]: pgmap v5107: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:08:48.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:48 smithi167 ceph-mon[37140]: pgmap v5107: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:08:50.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:50 smithi138 ceph-mon[36954]: pgmap v5108: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:08:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:50 smithi100 ceph-mon[32599]: pgmap v5108: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:08:50.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:50 smithi167 ceph-mon[37140]: pgmap v5108: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:08:50.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:08:50.828+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:08:50.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:08:50.828+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:08:50.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:08:50.828+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:08:50.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:08:50.828+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:08:50.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:08:50.828+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:08:52.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:52 smithi138 ceph-mon[36954]: pgmap v5109: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:08:52.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:52 smithi100 ceph-mon[32599]: pgmap v5109: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:08:52.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:52 smithi167 ceph-mon[37140]: pgmap v5109: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:08:54.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:54 smithi138 ceph-mon[36954]: pgmap v5110: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:08:54.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:54 smithi100 ceph-mon[32599]: pgmap v5110: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:08:54.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:54 smithi167 ceph-mon[37140]: pgmap v5110: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:08:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:56 smithi100 ceph-mon[32599]: pgmap v5111: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:08:56.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:56 smithi167 ceph-mon[37140]: pgmap v5111: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:08:56.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:56 smithi138 ceph-mon[36954]: pgmap v5111: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:08:58.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:08:58 smithi100 ceph-mon[32599]: pgmap v5112: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:08:58.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:08:58 smithi167 ceph-mon[37140]: pgmap v5112: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:08:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:08:58 smithi138 ceph-mon[36954]: pgmap v5112: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:09:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:09:00 smithi100 ceph-mon[32599]: pgmap v5113: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:09:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:09:00 smithi167 ceph-mon[37140]: pgmap v5113: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:09:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:09:00 smithi138 ceph-mon[36954]: pgmap v5113: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:09:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:09:02 smithi100 ceph-mon[32599]: pgmap v5114: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T16:09:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:09:02 smithi167 ceph-mon[37140]: pgmap v5114: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T16:09:02.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:09:02 smithi138 ceph-mon[36954]: pgmap v5114: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T16:09:04.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:09:04.211+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T16:09:04.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:09:04.211+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T16:09:04.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:09:04.212+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 69 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T16:09:04.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:09:04.212+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 69 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c070 con 0x7fe5a81087a0 2024-07-26T16:09:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:09:04 smithi100 ceph-mon[32599]: pgmap v5115: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-07-26T16:09:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:09:04 smithi167 ceph-mon[37140]: pgmap v5115: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-07-26T16:09:04.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:09:04 smithi138 ceph-mon[36954]: pgmap v5115: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-07-26T16:09:05.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:09:05.843+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:09:05.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:09:05.844+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:09:05.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:09:05.844+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:09:05.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:09:05.844+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:09:05.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:09:05.844+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:09:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:09:06 smithi100 ceph-mon[32599]: pgmap v5116: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:09:06.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:09:06 smithi167 ceph-mon[37140]: pgmap v5116: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:09:06.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:09:06 smithi138 ceph-mon[36954]: pgmap v5116: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:09:08.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:09:08 smithi100 ceph-mon[32599]: pgmap v5117: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:09:08.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:09:08 smithi100 ceph-mon[32599]: osdmap e366: 12 total, 12 up, 12 in 2024-07-26T16:09:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:09:08 smithi167 ceph-mon[37140]: pgmap v5117: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:09:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:09:08 smithi167 ceph-mon[37140]: osdmap e366: 12 total, 12 up, 12 in 2024-07-26T16:09:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:09:08 smithi138 ceph-mon[36954]: pgmap v5117: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:09:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:09:08 smithi138 ceph-mon[36954]: osdmap e366: 12 total, 12 up, 12 in 2024-07-26T16:09:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:09:10 smithi100 ceph-mon[32599]: pgmap v5119: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:09:10.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:09:10 smithi167 ceph-mon[37140]: pgmap v5119: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:09:10.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:09:10 smithi138 ceph-mon[36954]: pgmap v5119: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:09:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:09:12 smithi100 ceph-mon[32599]: pgmap v5120: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:09:12.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:09:12 smithi167 ceph-mon[37140]: pgmap v5120: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:09:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:09:12 smithi138 ceph-mon[36954]: pgmap v5120: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:09:14.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:09:14 smithi100 ceph-mon[32599]: pgmap v5121: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:09:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:09:14 smithi167 ceph-mon[37140]: pgmap v5121: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:09:14.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:09:14 smithi138 ceph-mon[36954]: pgmap v5121: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:09:16.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:09:16 smithi100 ceph-mon[32599]: pgmap v5122: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:09:16.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:09:16 smithi100 ceph-mon[32599]: osdmap e367: 12 total, 12 up, 12 in 2024-07-26T16:09:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:09:16 smithi167 ceph-mon[37140]: pgmap v5122: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:09:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:09:16 smithi167 ceph-mon[37140]: osdmap e367: 12 total, 12 up, 12 in 2024-07-26T16:09:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:09:16 smithi138 ceph-mon[36954]: pgmap v5122: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:09:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:09:16 smithi138 ceph-mon[36954]: osdmap e367: 12 total, 12 up, 12 in 2024-07-26T16:09:18.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:09:18 smithi100 ceph-mon[32599]: pgmap v5124: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:09:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:09:18 smithi167 ceph-mon[37140]: pgmap v5124: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:09:18.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:09:18 smithi138 ceph-mon[36954]: pgmap v5124: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:09:20.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:09:20 smithi100 ceph-mon[32599]: pgmap v5125: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:09:20.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:09:20 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:09:20.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:09:20 smithi167 ceph-mon[37140]: pgmap v5125: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:09:20.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:09:20 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:09:20.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:09:20.859+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:09:20.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:09:20.859+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:09:20.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:09:20.859+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:09:20.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:09:20.859+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:09:20.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:09:20.859+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:09:20.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:09:20 smithi138 ceph-mon[36954]: pgmap v5125: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:09:20.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:09:20 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:09:21.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:09:21 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:09:21.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:09:21 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:09:21.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:09:21 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:09:21.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:09:21 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:09:21.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:09:21 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:09:21.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:09:21 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:09:21.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:09:21 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:09:21.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:09:21 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:09:21.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:09:21 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:09:23.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:09:22 smithi138 ceph-mon[36954]: pgmap v5126: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:09:23.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:09:22 smithi100 ceph-mon[32599]: pgmap v5126: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:09:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:09:22 smithi167 ceph-mon[37140]: pgmap v5126: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:09:25.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:09:24 smithi138 ceph-mon[36954]: pgmap v5127: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:09:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:09:24 smithi100 ceph-mon[32599]: pgmap v5127: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:09:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:09:24 smithi167 ceph-mon[37140]: pgmap v5127: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:09:27.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:09:26 smithi138 ceph-mon[36954]: pgmap v5128: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:09:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:09:26 smithi100 ceph-mon[32599]: pgmap v5128: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:09:27.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:09:26 smithi167 ceph-mon[37140]: pgmap v5128: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:09:29.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:09:28 smithi138 ceph-mon[36954]: pgmap v5129: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:09:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:09:28 smithi100 ceph-mon[32599]: pgmap v5129: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:09:29.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:09:28 smithi167 ceph-mon[37140]: pgmap v5129: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:09:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:09:30 smithi138 ceph-mon[36954]: pgmap v5130: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:09:31.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:09:30 smithi100 ceph-mon[32599]: pgmap v5130: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:09:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:09:30 smithi167 ceph-mon[37140]: pgmap v5130: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:09:33.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:09:32 smithi138 ceph-mon[36954]: pgmap v5131: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:09:33.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:09:32 smithi100 ceph-mon[32599]: pgmap v5131: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:09:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:09:32 smithi167 ceph-mon[37140]: pgmap v5131: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:09:35.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:09:34 smithi138 ceph-mon[36954]: pgmap v5132: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:09:35.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:09:34 smithi100 ceph-mon[32599]: pgmap v5132: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:09:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:09:34 smithi167 ceph-mon[37140]: pgmap v5132: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:09:35.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:09:35.875+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:09:35.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:09:35.875+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:09:35.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:09:35.875+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:09:35.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:09:35.875+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:09:35.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:09:35.875+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:09:37.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:09:36 smithi138 ceph-mon[36954]: pgmap v5133: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:09:37.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:09:36 smithi100 ceph-mon[32599]: pgmap v5133: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:09:37.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:09:36 smithi167 ceph-mon[37140]: pgmap v5133: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:09:39.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:09:38 smithi138 ceph-mon[36954]: pgmap v5134: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:09:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:09:38 smithi100 ceph-mon[32599]: pgmap v5134: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:09:39.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:09:38 smithi167 ceph-mon[37140]: pgmap v5134: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:09:41.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:09:40 smithi138 ceph-mon[36954]: pgmap v5135: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:09:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:09:40 smithi100 ceph-mon[32599]: pgmap v5135: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:09:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:09:40 smithi167 ceph-mon[37140]: pgmap v5135: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:09:43.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:09:42 smithi138 ceph-mon[36954]: pgmap v5136: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:09:43.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:09:42 smithi100 ceph-mon[32599]: pgmap v5136: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:09:43.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:09:42 smithi167 ceph-mon[37140]: pgmap v5136: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:09:45.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:09:44 smithi138 ceph-mon[36954]: pgmap v5137: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:09:45.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:09:44 smithi100 ceph-mon[32599]: pgmap v5137: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:09:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:09:44 smithi167 ceph-mon[37140]: pgmap v5137: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:09:47.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:09:46 smithi138 ceph-mon[36954]: pgmap v5138: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:09:47.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:09:46 smithi100 ceph-mon[32599]: pgmap v5138: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:09:47.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:09:46 smithi167 ceph-mon[37140]: pgmap v5138: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:09:49.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:09:48 smithi138 ceph-mon[36954]: pgmap v5139: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:09:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:09:48 smithi100 ceph-mon[32599]: pgmap v5139: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:09:49.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:09:48 smithi167 ceph-mon[37140]: pgmap v5139: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:09:50.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:09:50.886+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:09:50.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:09:50.886+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:09:50.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:09:50.886+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:09:50.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:09:50.886+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:09:50.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:09:50.886+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:09:51.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:09:50 smithi138 ceph-mon[36954]: pgmap v5140: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:09:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:09:50 smithi100 ceph-mon[32599]: pgmap v5140: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:09:51.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:09:50 smithi167 ceph-mon[37140]: pgmap v5140: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:09:53.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:09:52 smithi138 ceph-mon[36954]: pgmap v5141: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:09:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:09:52 smithi100 ceph-mon[32599]: pgmap v5141: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:09:53.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:09:52 smithi167 ceph-mon[37140]: pgmap v5141: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:09:55.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:09:54 smithi138 ceph-mon[36954]: pgmap v5142: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:09:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:09:54 smithi100 ceph-mon[32599]: pgmap v5142: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:09:55.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:09:54 smithi167 ceph-mon[37140]: pgmap v5142: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:09:57.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:09:56 smithi138 ceph-mon[36954]: pgmap v5143: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:09:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:09:56 smithi100 ceph-mon[32599]: pgmap v5143: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:09:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:09:56 smithi167 ceph-mon[37140]: pgmap v5143: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:09:59.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:09:58 smithi138 ceph-mon[36954]: pgmap v5144: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:09:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:09:58 smithi100 ceph-mon[32599]: pgmap v5144: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:09:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:09:58 smithi167 ceph-mon[37140]: pgmap v5144: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:10:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:10:00 smithi138 ceph-mon[36954]: pgmap v5145: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:10:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:10:00 smithi138 ceph-mon[36954]: overall HEALTH_OK 2024-07-26T16:10:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:10:00 smithi100 ceph-mon[32599]: pgmap v5145: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:10:01.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:10:00 smithi100 ceph-mon[32599]: overall HEALTH_OK 2024-07-26T16:10:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:10:00 smithi167 ceph-mon[37140]: pgmap v5145: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:10:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:10:00 smithi167 ceph-mon[37140]: overall HEALTH_OK 2024-07-26T16:10:03.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:10:02 smithi138 ceph-mon[36954]: pgmap v5146: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:10:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:10:02 smithi100 ceph-mon[32599]: pgmap v5146: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:10:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:10:02 smithi167 ceph-mon[37140]: pgmap v5146: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:10:05.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:10:04 smithi138 ceph-mon[36954]: pgmap v5147: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:10:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:10:04 smithi100 ceph-mon[32599]: pgmap v5147: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:10:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:10:04 smithi167 ceph-mon[37140]: pgmap v5147: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:10:05.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:10:05.902+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:10:05.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:10:05.902+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:10:05.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:10:05.902+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:10:05.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:10:05.902+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:10:05.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:10:05.902+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:10:07.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:10:06 smithi138 ceph-mon[36954]: pgmap v5148: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:10:07.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:10:06 smithi100 ceph-mon[32599]: pgmap v5148: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:10:07.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:10:06 smithi167 ceph-mon[37140]: pgmap v5148: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:10:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:10:08 smithi138 ceph-mon[36954]: pgmap v5149: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:10:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:10:08 smithi138 ceph-mon[36954]: osdmap e368: 12 total, 12 up, 12 in 2024-07-26T16:10:09.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:10:08 smithi100 ceph-mon[32599]: pgmap v5149: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:10:09.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:10:08 smithi100 ceph-mon[32599]: osdmap e368: 12 total, 12 up, 12 in 2024-07-26T16:10:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:10:08 smithi167 ceph-mon[37140]: pgmap v5149: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:10:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:10:08 smithi167 ceph-mon[37140]: osdmap e368: 12 total, 12 up, 12 in 2024-07-26T16:10:11.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:10:10 smithi138 ceph-mon[36954]: pgmap v5151: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:10:11.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:10:10 smithi100 ceph-mon[32599]: pgmap v5151: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:10:11.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:10:10 smithi167 ceph-mon[37140]: pgmap v5151: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:10:13.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:10:12 smithi138 ceph-mon[36954]: pgmap v5152: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:10:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:10:12 smithi100 ceph-mon[32599]: pgmap v5152: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:10:13.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:10:12 smithi167 ceph-mon[37140]: pgmap v5152: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:10:15.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:10:14 smithi138 ceph-mon[36954]: pgmap v5153: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:10:15.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:10:14 smithi100 ceph-mon[32599]: pgmap v5153: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:10:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:10:14 smithi167 ceph-mon[37140]: pgmap v5153: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:10:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:10:16 smithi138 ceph-mon[36954]: pgmap v5154: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:10:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:10:16 smithi138 ceph-mon[36954]: osdmap e369: 12 total, 12 up, 12 in 2024-07-26T16:10:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:10:16 smithi100 ceph-mon[32599]: pgmap v5154: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:10:17.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:10:16 smithi100 ceph-mon[32599]: osdmap e369: 12 total, 12 up, 12 in 2024-07-26T16:10:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:10:16 smithi167 ceph-mon[37140]: pgmap v5154: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:10:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:10:16 smithi167 ceph-mon[37140]: osdmap e369: 12 total, 12 up, 12 in 2024-07-26T16:10:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:10:18 smithi138 ceph-mon[36954]: pgmap v5156: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 215 B/s wr, 6 op/s 2024-07-26T16:10:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:10:18 smithi100 ceph-mon[32599]: pgmap v5156: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 215 B/s wr, 6 op/s 2024-07-26T16:10:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:10:18 smithi167 ceph-mon[37140]: pgmap v5156: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 215 B/s wr, 6 op/s 2024-07-26T16:10:20.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:10:20.918+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:10:20.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:10:20.919+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:10:20.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:10:20.919+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:10:20.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:10:20.919+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:10:20.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:10:20.919+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:10:21.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:10:20 smithi138 ceph-mon[36954]: pgmap v5157: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:10:21.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:10:20 smithi100 ceph-mon[32599]: pgmap v5157: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:10:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:10:20 smithi167 ceph-mon[37140]: pgmap v5157: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:10:22.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:10:21 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:10:22.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:10:21 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:10:22.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:10:21 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:10:22.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:10:21 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:10:22.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:10:21 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:10:22.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:10:21 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:10:22.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:10:21 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:10:22.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:10:21 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:10:22.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:10:21 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:10:22.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:10:21 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:10:22.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:10:21 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:10:22.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:10:21 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:10:23.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:10:22 smithi138 ceph-mon[36954]: pgmap v5158: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:10:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:10:22 smithi100 ceph-mon[32599]: pgmap v5158: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:10:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:10:22 smithi167 ceph-mon[37140]: pgmap v5158: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:10:25.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:10:24 smithi138 ceph-mon[36954]: pgmap v5159: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:10:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:10:24 smithi100 ceph-mon[32599]: pgmap v5159: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:10:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:10:24 smithi167 ceph-mon[37140]: pgmap v5159: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:10:27.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:10:26 smithi138 ceph-mon[36954]: pgmap v5160: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:10:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:10:26 smithi100 ceph-mon[32599]: pgmap v5160: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:10:27.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:10:26 smithi167 ceph-mon[37140]: pgmap v5160: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:10:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:10:28 smithi100 ceph-mon[32599]: pgmap v5161: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:10:29.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:10:28 smithi167 ceph-mon[37140]: pgmap v5161: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:10:29.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:10:28 smithi138 ceph-mon[36954]: pgmap v5161: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:10:31.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:10:30 smithi100 ceph-mon[32599]: pgmap v5162: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:10:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:10:30 smithi167 ceph-mon[37140]: pgmap v5162: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:10:31.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:10:30 smithi138 ceph-mon[36954]: pgmap v5162: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:10:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:10:32 smithi100 ceph-mon[32599]: pgmap v5163: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:10:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:10:32 smithi167 ceph-mon[37140]: pgmap v5163: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:10:33.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:10:32 smithi138 ceph-mon[36954]: pgmap v5163: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:10:35.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:10:34 smithi100 ceph-mon[32599]: pgmap v5164: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:10:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:10:34 smithi167 ceph-mon[37140]: pgmap v5164: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:10:35.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:10:34 smithi138 ceph-mon[36954]: pgmap v5164: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:10:35.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:10:35.923+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:10:35.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:10:35.923+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:10:35.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:10:35.923+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:10:35.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:10:35.923+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:10:35.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:10:35.923+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:10:37.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:10:36 smithi100 ceph-mon[32599]: pgmap v5165: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:10:37.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:10:36 smithi167 ceph-mon[37140]: pgmap v5165: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:10:37.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:10:36 smithi138 ceph-mon[36954]: pgmap v5165: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:10:39.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:10:38 smithi100 ceph-mon[32599]: pgmap v5166: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:10:39.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:10:38 smithi167 ceph-mon[37140]: pgmap v5166: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:10:39.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:10:38 smithi138 ceph-mon[36954]: pgmap v5166: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:10:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:10:40 smithi100 ceph-mon[32599]: pgmap v5167: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:10:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:10:40 smithi167 ceph-mon[37140]: pgmap v5167: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:10:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:10:40 smithi138 ceph-mon[36954]: pgmap v5167: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:10:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:10:42 smithi100 ceph-mon[32599]: pgmap v5168: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:10:43.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:10:42 smithi167 ceph-mon[37140]: pgmap v5168: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:10:43.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:10:42 smithi138 ceph-mon[36954]: pgmap v5168: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:10:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:10:44 smithi100 ceph-mon[32599]: pgmap v5169: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:10:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:10:44 smithi167 ceph-mon[37140]: pgmap v5169: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:10:45.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:10:44 smithi138 ceph-mon[36954]: pgmap v5169: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:10:47.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:10:46 smithi100 ceph-mon[32599]: pgmap v5170: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:10:47.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:10:46 smithi167 ceph-mon[37140]: pgmap v5170: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:10:47.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:10:46 smithi138 ceph-mon[36954]: pgmap v5170: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:10:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:10:48 smithi100 ceph-mon[32599]: pgmap v5171: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:10:49.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:10:48 smithi167 ceph-mon[37140]: pgmap v5171: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:10:49.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:10:48 smithi138 ceph-mon[36954]: pgmap v5171: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:10:50.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:10:50.934+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:10:50.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:10:50.934+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:10:50.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:10:50.934+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:10:50.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:10:50.934+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:10:50.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:10:50.934+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:10:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:10:50 smithi100 ceph-mon[32599]: pgmap v5172: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:10:51.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:10:50 smithi167 ceph-mon[37140]: pgmap v5172: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:10:51.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:10:50 smithi138 ceph-mon[36954]: pgmap v5172: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:10:53.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:10:52 smithi100 ceph-mon[32599]: pgmap v5173: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:10:53.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:10:52 smithi167 ceph-mon[37140]: pgmap v5173: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:10:53.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:10:52 smithi138 ceph-mon[36954]: pgmap v5173: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:10:54.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:10:54 smithi138 ceph-mon[36954]: pgmap v5174: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:10:54.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:10:54 smithi100 ceph-mon[32599]: pgmap v5174: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:10:54.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:10:54 smithi167 ceph-mon[37140]: pgmap v5174: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:10:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:10:56 smithi100 ceph-mon[32599]: pgmap v5175: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:10:56.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:10:56 smithi167 ceph-mon[37140]: pgmap v5175: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:10:56.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:10:56 smithi138 ceph-mon[36954]: pgmap v5175: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:10:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:10:58 smithi100 ceph-mon[32599]: pgmap v5176: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:10:58.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:10:58 smithi167 ceph-mon[37140]: pgmap v5176: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:10:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:10:58 smithi138 ceph-mon[36954]: pgmap v5176: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:11:00.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:00 smithi100 ceph-mon[32599]: pgmap v5177: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:11:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:00 smithi167 ceph-mon[37140]: pgmap v5177: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:11:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:00 smithi138 ceph-mon[36954]: pgmap v5177: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:11:02.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:02 smithi100 ceph-mon[32599]: pgmap v5178: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:11:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:02 smithi167 ceph-mon[37140]: pgmap v5178: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:11:02.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:02 smithi138 ceph-mon[36954]: pgmap v5178: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:11:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:04 smithi100 ceph-mon[32599]: pgmap v5179: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:11:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:04 smithi167 ceph-mon[37140]: pgmap v5179: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:11:04.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:04 smithi138 ceph-mon[36954]: pgmap v5179: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:11:05.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:11:05.951+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:11:05.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:11:05.951+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:11:05.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:11:05.951+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:11:05.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:11:05.951+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:11:05.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:11:05.951+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:11:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:06 smithi100 ceph-mon[32599]: pgmap v5180: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:11:06.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:06 smithi167 ceph-mon[37140]: pgmap v5180: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:11:06.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:06 smithi138 ceph-mon[36954]: pgmap v5180: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:11:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:08 smithi167 ceph-mon[37140]: pgmap v5181: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:11:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:08 smithi167 ceph-mon[37140]: osdmap e370: 12 total, 12 up, 12 in 2024-07-26T16:11:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:08 smithi138 ceph-mon[36954]: pgmap v5181: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:11:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:08 smithi138 ceph-mon[36954]: osdmap e370: 12 total, 12 up, 12 in 2024-07-26T16:11:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:08 smithi100 ceph-mon[32599]: pgmap v5181: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:11:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:08 smithi100 ceph-mon[32599]: osdmap e370: 12 total, 12 up, 12 in 2024-07-26T16:11:10.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:10 smithi167 ceph-mon[37140]: pgmap v5183: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:11:10.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:10 smithi138 ceph-mon[36954]: pgmap v5183: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:11:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:10 smithi100 ceph-mon[32599]: pgmap v5183: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:11:12.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:12 smithi167 ceph-mon[37140]: pgmap v5184: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:11:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:12 smithi138 ceph-mon[36954]: pgmap v5184: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:11:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:12 smithi100 ceph-mon[32599]: pgmap v5184: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:11:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:14 smithi167 ceph-mon[37140]: pgmap v5185: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:11:14.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:14 smithi138 ceph-mon[36954]: pgmap v5185: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:11:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:14 smithi100 ceph-mon[32599]: pgmap v5185: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:11:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:16 smithi167 ceph-mon[37140]: pgmap v5186: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:11:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:16 smithi167 ceph-mon[37140]: osdmap e371: 12 total, 12 up, 12 in 2024-07-26T16:11:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:16 smithi138 ceph-mon[36954]: pgmap v5186: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:11:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:16 smithi138 ceph-mon[36954]: osdmap e371: 12 total, 12 up, 12 in 2024-07-26T16:11:16.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:16 smithi100 ceph-mon[32599]: pgmap v5186: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:11:16.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:16 smithi100 ceph-mon[32599]: osdmap e371: 12 total, 12 up, 12 in 2024-07-26T16:11:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:18 smithi167 ceph-mon[37140]: pgmap v5188: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:11:18.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:18 smithi138 ceph-mon[36954]: pgmap v5188: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:11:18.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:18 smithi100 ceph-mon[32599]: pgmap v5188: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:11:20.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:20 smithi167 ceph-mon[37140]: pgmap v5189: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:11:20.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:20 smithi100 ceph-mon[32599]: pgmap v5189: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:11:20.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:20 smithi138 ceph-mon[36954]: pgmap v5189: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:11:20.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:11:20.967+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:11:20.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:11:20.967+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:11:20.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:11:20.967+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:11:20.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:11:20.967+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:11:20.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:11:20.967+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:11:21.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:21 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:11:21.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:21 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:11:21.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:21 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:11:22.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:22 smithi167 ceph-mon[37140]: pgmap v5190: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:11:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:22 smithi138 ceph-mon[36954]: pgmap v5190: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:11:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:22 smithi100 ceph-mon[32599]: pgmap v5190: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:11:24.194 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:11:24.194 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:11:24.194 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:11:24.194 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:11:24.194 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:11:24.194 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:11:24.195 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:11:24.195 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:11:24.195 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:11:24.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:11:24.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:11:24.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:11:24.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:11:24.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:11:24.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:11:24.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:11:24.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:11:24.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:11:24.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:11:24.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:11:24.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:11:24.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:11:24.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:11:24.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:11:24.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:11:24.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:11:24.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:11:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:24 smithi100 ceph-mon[32599]: pgmap v5191: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:11:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:24 smithi167 ceph-mon[37140]: pgmap v5191: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:11:25.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:24 smithi138 ceph-mon[36954]: pgmap v5191: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:11:27.243 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:26 smithi138 ceph-mon[36954]: pgmap v5192: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:11:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:26 smithi100 ceph-mon[32599]: pgmap v5192: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:11:27.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:26 smithi167 ceph-mon[37140]: pgmap v5192: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:11:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:28 smithi100 ceph-mon[32599]: pgmap v5193: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:11:29.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:28 smithi167 ceph-mon[37140]: pgmap v5193: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:11:29.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:28 smithi138 ceph-mon[36954]: pgmap v5193: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:11:31.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:30 smithi100 ceph-mon[32599]: pgmap v5194: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:11:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:30 smithi167 ceph-mon[37140]: pgmap v5194: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:11:31.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:30 smithi138 ceph-mon[36954]: pgmap v5194: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:11:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:32 smithi100 ceph-mon[32599]: pgmap v5195: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:11:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:32 smithi167 ceph-mon[37140]: pgmap v5195: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:11:33.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:32 smithi138 ceph-mon[36954]: pgmap v5195: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:11:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:34 smithi100 ceph-mon[32599]: pgmap v5196: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:11:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:34 smithi167 ceph-mon[37140]: pgmap v5196: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:11:35.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:34 smithi138 ceph-mon[36954]: pgmap v5196: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:11:35.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:11:35.979+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:11:35.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:11:35.979+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:11:35.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:11:35.979+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:11:35.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:11:35.979+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:11:35.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:11:35.979+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:11:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:36 smithi100 ceph-mon[32599]: pgmap v5197: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:11:37.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:36 smithi167 ceph-mon[37140]: pgmap v5197: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:11:37.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:36 smithi138 ceph-mon[36954]: pgmap v5197: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:11:39.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:38 smithi100 ceph-mon[32599]: pgmap v5198: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:11:39.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:38 smithi167 ceph-mon[37140]: pgmap v5198: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:11:39.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:38 smithi138 ceph-mon[36954]: pgmap v5198: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:11:41.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:40 smithi100 ceph-mon[32599]: pgmap v5199: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:11:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:40 smithi167 ceph-mon[37140]: pgmap v5199: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:11:41.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:40 smithi138 ceph-mon[36954]: pgmap v5199: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:11:43.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:42 smithi100 ceph-mon[32599]: pgmap v5200: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:11:43.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:42 smithi167 ceph-mon[37140]: pgmap v5200: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:11:43.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:42 smithi138 ceph-mon[36954]: pgmap v5200: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:11:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:44 smithi100 ceph-mon[32599]: pgmap v5201: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:11:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:44 smithi167 ceph-mon[37140]: pgmap v5201: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:11:45.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:44 smithi138 ceph-mon[36954]: pgmap v5201: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:11:47.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:46 smithi100 ceph-mon[32599]: pgmap v5202: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:11:47.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:46 smithi167 ceph-mon[37140]: pgmap v5202: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:11:47.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:46 smithi138 ceph-mon[36954]: pgmap v5202: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:11:49.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:48 smithi100 ceph-mon[32599]: pgmap v5203: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:11:49.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:48 smithi167 ceph-mon[37140]: pgmap v5203: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:11:49.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:48 smithi138 ceph-mon[36954]: pgmap v5203: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:11:50.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:11:50.995+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:11:50.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:11:50.995+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:11:50.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:11:50.995+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:11:50.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:11:50.995+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:11:50.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:11:50.995+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:11:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:50 smithi100 ceph-mon[32599]: pgmap v5204: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:11:51.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:50 smithi167 ceph-mon[37140]: pgmap v5204: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:11:51.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:50 smithi138 ceph-mon[36954]: pgmap v5204: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:11:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:52 smithi100 ceph-mon[32599]: pgmap v5205: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:11:53.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:52 smithi167 ceph-mon[37140]: pgmap v5205: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:11:53.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:52 smithi138 ceph-mon[36954]: pgmap v5205: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:11:55.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:54 smithi100 ceph-mon[32599]: pgmap v5206: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:11:55.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:54 smithi167 ceph-mon[37140]: pgmap v5206: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:11:55.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:54 smithi138 ceph-mon[36954]: pgmap v5206: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:11:56.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:56 smithi138 ceph-mon[36954]: pgmap v5207: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:11:56.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:56 smithi100 ceph-mon[32599]: pgmap v5207: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:11:56.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:56 smithi167 ceph-mon[37140]: pgmap v5207: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:11:58.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:11:58 smithi100 ceph-mon[32599]: pgmap v5208: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:11:58.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:11:58 smithi167 ceph-mon[37140]: pgmap v5208: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:11:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:11:58 smithi138 ceph-mon[36954]: pgmap v5208: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:12:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:00 smithi100 ceph-mon[32599]: pgmap v5209: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:12:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:00 smithi167 ceph-mon[37140]: pgmap v5209: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:12:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:00 smithi138 ceph-mon[36954]: pgmap v5209: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:12:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:02 smithi167 ceph-mon[37140]: pgmap v5210: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:12:02.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:02 smithi138 ceph-mon[36954]: pgmap v5210: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:12:02.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:02 smithi100 ceph-mon[32599]: pgmap v5210: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:12:04.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:12:04.239+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T16:12:04.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:12:04.239+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T16:12:04.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:12:04.239+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 70 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T16:12:04.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:12:04.239+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 70 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c020 con 0x7fe5a81087a0 2024-07-26T16:12:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:04 smithi167 ceph-mon[37140]: pgmap v5211: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:12:04.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:04 smithi138 ceph-mon[36954]: pgmap v5211: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:12:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:04 smithi100 ceph-mon[32599]: pgmap v5211: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:12:06.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:12:06.006+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:12:06.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:12:06.007+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:12:06.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:12:06.007+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:12:06.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:12:06.007+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:12:06.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:12:06.007+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:12:06.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:06 smithi167 ceph-mon[37140]: pgmap v5212: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:12:06.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:06 smithi138 ceph-mon[36954]: pgmap v5212: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:12:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:06 smithi100 ceph-mon[32599]: pgmap v5212: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:12:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:08 smithi167 ceph-mon[37140]: pgmap v5213: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:12:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:08 smithi167 ceph-mon[37140]: osdmap e372: 12 total, 12 up, 12 in 2024-07-26T16:12:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:08 smithi138 ceph-mon[36954]: pgmap v5213: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:12:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:08 smithi138 ceph-mon[36954]: osdmap e372: 12 total, 12 up, 12 in 2024-07-26T16:12:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:08 smithi100 ceph-mon[32599]: pgmap v5213: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:12:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:08 smithi100 ceph-mon[32599]: osdmap e372: 12 total, 12 up, 12 in 2024-07-26T16:12:10.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:10 smithi167 ceph-mon[37140]: pgmap v5215: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:12:10.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:10 smithi138 ceph-mon[36954]: pgmap v5215: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:12:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:10 smithi100 ceph-mon[32599]: pgmap v5215: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:12:12.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:12 smithi167 ceph-mon[37140]: pgmap v5216: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:12:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:12 smithi138 ceph-mon[36954]: pgmap v5216: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:12:12.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:12 smithi100 ceph-mon[32599]: pgmap v5216: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:12:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:14 smithi167 ceph-mon[37140]: pgmap v5217: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:12:14.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:14 smithi138 ceph-mon[36954]: pgmap v5217: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:12:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:14 smithi100 ceph-mon[32599]: pgmap v5217: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:12:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:16 smithi138 ceph-mon[36954]: pgmap v5218: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:12:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:16 smithi138 ceph-mon[36954]: osdmap e373: 12 total, 12 up, 12 in 2024-07-26T16:12:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:16 smithi100 ceph-mon[32599]: pgmap v5218: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:12:16.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:16 smithi100 ceph-mon[32599]: osdmap e373: 12 total, 12 up, 12 in 2024-07-26T16:12:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:16 smithi167 ceph-mon[37140]: pgmap v5218: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:12:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:16 smithi167 ceph-mon[37140]: osdmap e373: 12 total, 12 up, 12 in 2024-07-26T16:12:18.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:18 smithi138 ceph-mon[36954]: pgmap v5220: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:12:18.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:18 smithi100 ceph-mon[32599]: pgmap v5220: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:12:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:18 smithi167 ceph-mon[37140]: pgmap v5220: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:12:20.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:20 smithi138 ceph-mon[36954]: pgmap v5221: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:12:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:20 smithi100 ceph-mon[32599]: pgmap v5221: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:12:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:20 smithi167 ceph-mon[37140]: pgmap v5221: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:12:21.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:12:21.023+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:12:21.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:12:21.024+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:12:21.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:12:21.024+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:12:21.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:12:21.024+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:12:21.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:12:21.024+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:12:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:22 smithi138 ceph-mon[36954]: pgmap v5222: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:12:22.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:22 smithi100 ceph-mon[32599]: pgmap v5222: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:12:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:22 smithi167 ceph-mon[37140]: pgmap v5222: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:12:23.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:12:23.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:12:24.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:12:25.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:24 smithi138 ceph-mon[36954]: pgmap v5223: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:12:25.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:12:25.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:12:25.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:12:25.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:12:25.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:12:25.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:12:25.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:12:25.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:12:25.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:12:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:24 smithi100 ceph-mon[32599]: pgmap v5223: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:12:25.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:12:25.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:12:25.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:12:25.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:12:25.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:12:25.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:12:25.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:12:25.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:12:25.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:12:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:24 smithi167 ceph-mon[37140]: pgmap v5223: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:12:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:12:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:12:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:12:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:12:25.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:12:25.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:12:25.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:12:25.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:12:25.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:12:27.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:26 smithi138 ceph-mon[36954]: pgmap v5224: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:12:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:26 smithi100 ceph-mon[32599]: pgmap v5224: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:12:27.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:26 smithi167 ceph-mon[37140]: pgmap v5224: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:12:29.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:28 smithi138 ceph-mon[36954]: pgmap v5225: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:12:29.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:28 smithi100 ceph-mon[32599]: pgmap v5225: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:12:29.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:28 smithi167 ceph-mon[37140]: pgmap v5225: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:12:31.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:30 smithi138 ceph-mon[36954]: pgmap v5226: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:12:31.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:30 smithi100 ceph-mon[32599]: pgmap v5226: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:12:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:30 smithi167 ceph-mon[37140]: pgmap v5226: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:12:32.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:32 smithi138 ceph-mon[36954]: pgmap v5227: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:12:32.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:32 smithi100 ceph-mon[32599]: pgmap v5227: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:12:32.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:32 smithi167 ceph-mon[37140]: pgmap v5227: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:12:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:34 smithi167 ceph-mon[37140]: pgmap v5228: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:12:34.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:34 smithi138 ceph-mon[36954]: pgmap v5228: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:12:34.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:34 smithi100 ceph-mon[32599]: pgmap v5228: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:12:36.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:12:36.040+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:12:36.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:12:36.040+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:12:36.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:12:36.040+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:12:36.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:12:36.040+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:12:36.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:12:36.040+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:12:36.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:36 smithi167 ceph-mon[37140]: pgmap v5229: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:12:36.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:36 smithi138 ceph-mon[36954]: pgmap v5229: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:12:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:36 smithi100 ceph-mon[32599]: pgmap v5229: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:12:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:38 smithi167 ceph-mon[37140]: pgmap v5230: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:12:38.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:38 smithi138 ceph-mon[36954]: pgmap v5230: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:12:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:38 smithi100 ceph-mon[32599]: pgmap v5230: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:12:40.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:40 smithi167 ceph-mon[37140]: pgmap v5231: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:12:40.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:40 smithi138 ceph-mon[36954]: pgmap v5231: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:12:40.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:40 smithi100 ceph-mon[32599]: pgmap v5231: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:12:42.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:42 smithi167 ceph-mon[37140]: pgmap v5232: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:12:42.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:42 smithi138 ceph-mon[36954]: pgmap v5232: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:12:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:42 smithi100 ceph-mon[32599]: pgmap v5232: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:12:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:44 smithi167 ceph-mon[37140]: pgmap v5233: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:12:44.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:44 smithi138 ceph-mon[36954]: pgmap v5233: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:12:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:44 smithi100 ceph-mon[32599]: pgmap v5233: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:12:46.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:46 smithi138 ceph-mon[36954]: pgmap v5234: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:12:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:46 smithi100 ceph-mon[32599]: pgmap v5234: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:12:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:46 smithi167 ceph-mon[37140]: pgmap v5234: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:12:48.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:48 smithi138 ceph-mon[36954]: pgmap v5235: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:12:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:48 smithi100 ceph-mon[32599]: pgmap v5235: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:12:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:48 smithi167 ceph-mon[37140]: pgmap v5235: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:12:50.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:50 smithi138 ceph-mon[36954]: pgmap v5236: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:12:50.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:50 smithi100 ceph-mon[32599]: pgmap v5236: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:12:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:50 smithi167 ceph-mon[37140]: pgmap v5236: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:12:51.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:12:51.055+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:12:51.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:12:51.056+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:12:51.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:12:51.056+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:12:51.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:12:51.056+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:12:51.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:12:51.056+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:12:52.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:52 smithi138 ceph-mon[36954]: pgmap v5237: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:12:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:52 smithi100 ceph-mon[32599]: pgmap v5237: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:12:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:52 smithi167 ceph-mon[37140]: pgmap v5237: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:12:54.846 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:54 smithi167 ceph-mon[37140]: pgmap v5238: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:12:54.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:54 smithi138 ceph-mon[36954]: pgmap v5238: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:12:54.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:54 smithi100 ceph-mon[32599]: pgmap v5238: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:12:56.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:56 smithi138 ceph-mon[36954]: pgmap v5239: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:12:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:56 smithi100 ceph-mon[32599]: pgmap v5239: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:12:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:56 smithi167 ceph-mon[37140]: pgmap v5239: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:12:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:12:58 smithi138 ceph-mon[36954]: pgmap v5240: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:12:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:12:58 smithi100 ceph-mon[32599]: pgmap v5240: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:12:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:12:58 smithi167 ceph-mon[37140]: pgmap v5240: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:13:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:13:00 smithi138 ceph-mon[36954]: pgmap v5241: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:13:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:13:00 smithi100 ceph-mon[32599]: pgmap v5241: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:13:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:13:00 smithi167 ceph-mon[37140]: pgmap v5241: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:13:02.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:13:02 smithi138 ceph-mon[36954]: pgmap v5242: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:13:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:13:02 smithi100 ceph-mon[32599]: pgmap v5242: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:13:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:13:02 smithi167 ceph-mon[37140]: pgmap v5242: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:13:04.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:13:04 smithi138 ceph-mon[36954]: pgmap v5243: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:13:04.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:13:04 smithi100 ceph-mon[32599]: pgmap v5243: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:13:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:13:04 smithi167 ceph-mon[37140]: pgmap v5243: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:13:06.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:13:06.072+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:13:06.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:13:06.072+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:13:06.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:13:06.072+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:13:06.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:13:06.072+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:13:06.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:13:06.072+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:13:06.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:13:06 smithi138 ceph-mon[36954]: pgmap v5244: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:13:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:13:06 smithi100 ceph-mon[32599]: pgmap v5244: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:13:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:13:06 smithi167 ceph-mon[37140]: pgmap v5244: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:13:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:13:08 smithi138 ceph-mon[36954]: pgmap v5245: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:13:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:13:08 smithi138 ceph-mon[36954]: osdmap e374: 12 total, 12 up, 12 in 2024-07-26T16:13:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:13:08 smithi100 ceph-mon[32599]: pgmap v5245: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:13:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:13:08 smithi100 ceph-mon[32599]: osdmap e374: 12 total, 12 up, 12 in 2024-07-26T16:13:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:13:08 smithi167 ceph-mon[37140]: pgmap v5245: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:13:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:13:08 smithi167 ceph-mon[37140]: osdmap e374: 12 total, 12 up, 12 in 2024-07-26T16:13:10.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:13:10 smithi138 ceph-mon[36954]: pgmap v5247: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:13:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:13:10 smithi100 ceph-mon[32599]: pgmap v5247: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:13:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:13:10 smithi167 ceph-mon[37140]: pgmap v5247: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:13:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:13:12 smithi138 ceph-mon[36954]: pgmap v5248: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:13:12.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:13:12 smithi100 ceph-mon[32599]: pgmap v5248: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:13:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:13:12 smithi167 ceph-mon[37140]: pgmap v5248: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:13:14.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:13:14 smithi138 ceph-mon[36954]: pgmap v5249: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:13:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:13:14 smithi100 ceph-mon[32599]: pgmap v5249: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:13:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:13:14 smithi167 ceph-mon[37140]: pgmap v5249: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:13:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:13:16 smithi138 ceph-mon[36954]: pgmap v5250: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:13:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:13:16 smithi138 ceph-mon[36954]: osdmap e375: 12 total, 12 up, 12 in 2024-07-26T16:13:16.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:13:16 smithi100 ceph-mon[32599]: pgmap v5250: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:13:16.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:13:16 smithi100 ceph-mon[32599]: osdmap e375: 12 total, 12 up, 12 in 2024-07-26T16:13:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:13:16 smithi167 ceph-mon[37140]: pgmap v5250: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:13:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:13:16 smithi167 ceph-mon[37140]: osdmap e375: 12 total, 12 up, 12 in 2024-07-26T16:13:18.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:13:18 smithi138 ceph-mon[36954]: pgmap v5252: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 5 op/s 2024-07-26T16:13:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:13:18 smithi100 ceph-mon[32599]: pgmap v5252: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 5 op/s 2024-07-26T16:13:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:13:18 smithi167 ceph-mon[37140]: pgmap v5252: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 5 op/s 2024-07-26T16:13:20.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:13:20 smithi138 ceph-mon[36954]: pgmap v5253: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:13:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:13:20 smithi100 ceph-mon[32599]: pgmap v5253: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:13:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:13:20 smithi167 ceph-mon[37140]: pgmap v5253: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:13:21.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:13:21.086+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:13:21.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:13:21.087+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:13:21.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:13:21.087+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:13:21.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:13:21.087+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:13:21.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:13:21.087+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:13:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:13:22 smithi138 ceph-mon[36954]: pgmap v5254: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:13:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:13:22 smithi100 ceph-mon[32599]: pgmap v5254: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:13:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:13:22 smithi167 ceph-mon[37140]: pgmap v5254: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:13:24.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:13:24 smithi138 ceph-mon[36954]: pgmap v5255: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:13:24.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:13:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:13:24.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:13:24 smithi100 ceph-mon[32599]: pgmap v5255: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:13:24.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:13:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:13:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:13:24 smithi167 ceph-mon[37140]: pgmap v5255: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:13:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:13:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:13:25.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:13:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:13:25.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:13:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:13:25.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:13:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:13:25.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:13:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:13:25.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:13:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:13:25.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:13:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:13:26.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:13:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:13:26.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:13:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:13:26.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:13:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:13:27.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:13:26 smithi138 ceph-mon[36954]: pgmap v5256: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:13:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:13:26 smithi100 ceph-mon[32599]: pgmap v5256: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:13:27.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:13:26 smithi167 ceph-mon[37140]: pgmap v5256: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:13:29.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:13:28 smithi138 ceph-mon[36954]: pgmap v5257: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:13:29.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:13:28 smithi100 ceph-mon[32599]: pgmap v5257: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:13:29.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:13:28 smithi167 ceph-mon[37140]: pgmap v5257: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:13:31.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:13:30 smithi138 ceph-mon[36954]: pgmap v5258: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:13:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:13:30 smithi100 ceph-mon[32599]: pgmap v5258: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:13:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:13:30 smithi167 ceph-mon[37140]: pgmap v5258: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:13:33.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:13:32 smithi138 ceph-mon[36954]: pgmap v5259: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:13:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:13:32 smithi100 ceph-mon[32599]: pgmap v5259: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:13:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:13:32 smithi167 ceph-mon[37140]: pgmap v5259: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:13:35.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:13:34 smithi138 ceph-mon[36954]: pgmap v5260: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:13:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:13:34 smithi100 ceph-mon[32599]: pgmap v5260: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:13:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:13:34 smithi167 ceph-mon[37140]: pgmap v5260: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:13:36.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:13:36.103+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:13:36.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:13:36.104+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:13:36.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:13:36.104+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:13:36.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:13:36.104+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:13:36.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:13:36.104+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:13:37.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:13:36 smithi138 ceph-mon[36954]: pgmap v5261: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:13:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:13:36 smithi100 ceph-mon[32599]: pgmap v5261: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:13:37.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:13:36 smithi167 ceph-mon[37140]: pgmap v5261: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:13:39.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:13:38 smithi138 ceph-mon[36954]: pgmap v5262: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:13:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:13:38 smithi100 ceph-mon[32599]: pgmap v5262: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:13:39.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:13:38 smithi167 ceph-mon[37140]: pgmap v5262: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:13:41.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:13:40 smithi138 ceph-mon[36954]: pgmap v5263: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:13:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:13:40 smithi100 ceph-mon[32599]: pgmap v5263: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:13:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:13:40 smithi167 ceph-mon[37140]: pgmap v5263: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:13:43.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:13:42 smithi167 ceph-mon[37140]: pgmap v5264: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:13:43.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:13:42 smithi138 ceph-mon[36954]: pgmap v5264: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:13:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:13:42 smithi100 ceph-mon[32599]: pgmap v5264: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:13:45.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:13:44 smithi138 ceph-mon[36954]: pgmap v5265: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:13:45.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:13:44 smithi100 ceph-mon[32599]: pgmap v5265: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:13:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:13:44 smithi167 ceph-mon[37140]: pgmap v5265: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:13:47.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:13:46 smithi138 ceph-mon[36954]: pgmap v5266: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:13:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:13:46 smithi100 ceph-mon[32599]: pgmap v5266: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:13:47.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:13:46 smithi167 ceph-mon[37140]: pgmap v5266: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:13:49.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:13:48 smithi138 ceph-mon[36954]: pgmap v5267: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:13:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:13:48 smithi100 ceph-mon[32599]: pgmap v5267: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:13:49.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:13:48 smithi167 ceph-mon[37140]: pgmap v5267: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:13:51.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:13:51.119+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:13:51.122 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:13:50 smithi100 ceph-mon[32599]: pgmap v5268: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:13:51.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:13:51.120+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:13:51.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:13:51.120+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:13:51.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:13:51.120+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:13:51.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:13:51.120+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:13:51.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:13:50 smithi138 ceph-mon[36954]: pgmap v5268: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:13:51.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:13:50 smithi167 ceph-mon[37140]: pgmap v5268: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:13:53.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:13:52 smithi138 ceph-mon[36954]: pgmap v5269: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T16:13:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:13:52 smithi100 ceph-mon[32599]: pgmap v5269: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T16:13:53.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:13:52 smithi167 ceph-mon[37140]: pgmap v5269: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T16:13:55.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:13:54 smithi138 ceph-mon[36954]: pgmap v5270: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:13:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:13:54 smithi100 ceph-mon[32599]: pgmap v5270: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:13:55.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:13:54 smithi167 ceph-mon[37140]: pgmap v5270: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:13:57.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:13:56 smithi138 ceph-mon[36954]: pgmap v5271: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:13:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:13:56 smithi100 ceph-mon[32599]: pgmap v5271: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:13:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:13:56 smithi167 ceph-mon[37140]: pgmap v5271: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:13:59.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:13:58 smithi138 ceph-mon[36954]: pgmap v5272: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:13:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:13:58 smithi100 ceph-mon[32599]: pgmap v5272: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:13:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:13:58 smithi167 ceph-mon[37140]: pgmap v5272: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:14:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:14:00 smithi138 ceph-mon[36954]: pgmap v5273: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:14:01.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:14:00 smithi100 ceph-mon[32599]: pgmap v5273: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:14:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:14:00 smithi167 ceph-mon[37140]: pgmap v5273: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:14:03.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:14:02 smithi138 ceph-mon[36954]: pgmap v5274: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:14:03.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:14:02 smithi100 ceph-mon[32599]: pgmap v5274: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:14:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:14:02 smithi167 ceph-mon[37140]: pgmap v5274: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:14:05.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:14:04 smithi138 ceph-mon[36954]: pgmap v5275: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:14:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:14:04 smithi100 ceph-mon[32599]: pgmap v5275: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:14:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:14:04 smithi167 ceph-mon[37140]: pgmap v5275: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:14:06.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:14:06.135+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:14:06.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:14:06.136+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:14:06.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:14:06.136+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:14:06.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:14:06.136+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:14:06.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:14:06.136+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:14:07.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:14:06 smithi138 ceph-mon[36954]: pgmap v5276: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:14:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:14:06 smithi100 ceph-mon[32599]: pgmap v5276: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:14:07.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:14:06 smithi167 ceph-mon[37140]: pgmap v5276: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:14:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:14:08 smithi138 ceph-mon[36954]: pgmap v5277: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:14:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:14:08 smithi138 ceph-mon[36954]: osdmap e376: 12 total, 12 up, 12 in 2024-07-26T16:14:09.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:14:08 smithi100 ceph-mon[32599]: pgmap v5277: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:14:09.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:14:08 smithi100 ceph-mon[32599]: osdmap e376: 12 total, 12 up, 12 in 2024-07-26T16:14:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:14:08 smithi167 ceph-mon[37140]: pgmap v5277: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:14:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:14:08 smithi167 ceph-mon[37140]: osdmap e376: 12 total, 12 up, 12 in 2024-07-26T16:14:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:14:10 smithi138 ceph-mon[36954]: pgmap v5279: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:14:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:14:10 smithi100 ceph-mon[32599]: pgmap v5279: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:14:11.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:14:10 smithi167 ceph-mon[37140]: pgmap v5279: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:14:13.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:14:12 smithi138 ceph-mon[36954]: pgmap v5280: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:14:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:14:12 smithi100 ceph-mon[32599]: pgmap v5280: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:14:13.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:14:12 smithi167 ceph-mon[37140]: pgmap v5280: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:14:15.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:14:14 smithi138 ceph-mon[36954]: pgmap v5281: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:14:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:14:14 smithi100 ceph-mon[32599]: pgmap v5281: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:14:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:14:14 smithi167 ceph-mon[37140]: pgmap v5281: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:14:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:14:16 smithi138 ceph-mon[36954]: pgmap v5282: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:14:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:14:16 smithi138 ceph-mon[36954]: osdmap e377: 12 total, 12 up, 12 in 2024-07-26T16:14:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:14:16 smithi100 ceph-mon[32599]: pgmap v5282: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:14:17.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:14:16 smithi100 ceph-mon[32599]: osdmap e377: 12 total, 12 up, 12 in 2024-07-26T16:14:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:14:16 smithi167 ceph-mon[37140]: pgmap v5282: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:14:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:14:16 smithi167 ceph-mon[37140]: osdmap e377: 12 total, 12 up, 12 in 2024-07-26T16:14:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:14:18 smithi138 ceph-mon[36954]: pgmap v5284: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 212 B/s wr, 6 op/s 2024-07-26T16:14:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:14:18 smithi100 ceph-mon[32599]: pgmap v5284: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 212 B/s wr, 6 op/s 2024-07-26T16:14:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:14:18 smithi167 ceph-mon[37140]: pgmap v5284: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 212 B/s wr, 6 op/s 2024-07-26T16:14:21.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:14:21.151+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:14:21.154 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:14:20 smithi100 ceph-mon[32599]: pgmap v5285: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:14:21.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:14:21.152+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:14:21.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:14:21.152+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:14:21.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:14:21.152+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:14:21.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:14:21.152+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:14:21.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:14:20 smithi138 ceph-mon[36954]: pgmap v5285: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:14:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:14:20 smithi167 ceph-mon[37140]: pgmap v5285: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:14:23.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:14:22 smithi138 ceph-mon[36954]: pgmap v5286: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:14:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:14:22 smithi100 ceph-mon[32599]: pgmap v5286: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:14:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:14:22 smithi167 ceph-mon[37140]: pgmap v5286: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:14:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:14:24 smithi138 ceph-mon[36954]: pgmap v5287: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:14:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:14:24 smithi100 ceph-mon[32599]: pgmap v5287: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:14:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:14:24 smithi167 ceph-mon[37140]: pgmap v5287: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:14:26.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:14:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:14:26.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:14:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:14:26.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:14:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:14:26.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:14:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:14:26.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:14:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:14:26.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:14:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:14:26.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:14:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:14:26.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:14:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:14:26.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:14:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:14:26.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:14:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:14:26.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:14:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:14:26.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:14:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:14:27.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:14:26 smithi138 ceph-mon[36954]: pgmap v5288: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:14:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:14:26 smithi100 ceph-mon[32599]: pgmap v5288: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:14:27.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:14:26 smithi167 ceph-mon[37140]: pgmap v5288: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:14:29.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:14:28 smithi138 ceph-mon[36954]: pgmap v5289: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:14:29.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:14:28 smithi100 ceph-mon[32599]: pgmap v5289: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:14:29.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:14:28 smithi167 ceph-mon[37140]: pgmap v5289: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:14:31.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:14:30 smithi138 ceph-mon[36954]: pgmap v5290: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:14:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:14:30 smithi100 ceph-mon[32599]: pgmap v5290: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:14:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:14:30 smithi167 ceph-mon[37140]: pgmap v5290: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:14:33.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:14:32 smithi138 ceph-mon[36954]: pgmap v5291: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:14:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:14:32 smithi100 ceph-mon[32599]: pgmap v5291: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:14:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:14:32 smithi167 ceph-mon[37140]: pgmap v5291: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:14:35.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:14:34 smithi138 ceph-mon[36954]: pgmap v5292: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:14:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:14:34 smithi100 ceph-mon[32599]: pgmap v5292: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:14:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:14:34 smithi167 ceph-mon[37140]: pgmap v5292: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:14:36.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:14:36.169+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:14:36.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:14:36.169+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:14:36.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:14:36.169+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:14:36.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:14:36.169+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:14:36.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:14:36.169+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:14:37.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:14:36 smithi138 ceph-mon[36954]: pgmap v5293: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:14:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:14:36 smithi100 ceph-mon[32599]: pgmap v5293: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:14:37.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:14:36 smithi167 ceph-mon[37140]: pgmap v5293: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:14:39.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:14:38 smithi138 ceph-mon[36954]: pgmap v5294: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:14:39.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:14:38 smithi100 ceph-mon[32599]: pgmap v5294: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:14:39.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:14:38 smithi167 ceph-mon[37140]: pgmap v5294: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:14:41.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:14:40 smithi138 ceph-mon[36954]: pgmap v5295: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:14:41.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:14:40 smithi100 ceph-mon[32599]: pgmap v5295: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:14:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:14:40 smithi167 ceph-mon[37140]: pgmap v5295: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:14:43.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:14:42 smithi138 ceph-mon[36954]: pgmap v5296: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:14:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:14:42 smithi100 ceph-mon[32599]: pgmap v5296: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:14:43.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:14:42 smithi167 ceph-mon[37140]: pgmap v5296: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:14:45.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:14:44 smithi138 ceph-mon[36954]: pgmap v5297: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:14:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:14:44 smithi100 ceph-mon[32599]: pgmap v5297: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:14:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:14:44 smithi167 ceph-mon[37140]: pgmap v5297: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:14:47.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:14:46 smithi138 ceph-mon[36954]: pgmap v5298: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:14:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:14:46 smithi100 ceph-mon[32599]: pgmap v5298: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:14:47.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:14:46 smithi167 ceph-mon[37140]: pgmap v5298: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:14:49.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:14:48 smithi138 ceph-mon[36954]: pgmap v5299: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:14:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:14:48 smithi100 ceph-mon[32599]: pgmap v5299: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:14:49.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:14:48 smithi167 ceph-mon[37140]: pgmap v5299: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:14:51.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:14:51.184+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:14:51.187 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:14:50 smithi100 ceph-mon[32599]: pgmap v5300: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:14:51.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:14:51.184+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:14:51.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:14:51.184+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:14:51.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:14:51.184+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:14:51.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:14:51.184+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:14:51.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:14:50 smithi138 ceph-mon[36954]: pgmap v5300: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:14:51.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:14:50 smithi167 ceph-mon[37140]: pgmap v5300: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:14:53.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:14:52 smithi138 ceph-mon[36954]: pgmap v5301: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:14:53.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:14:52 smithi100 ceph-mon[32599]: pgmap v5301: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:14:53.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:14:52 smithi167 ceph-mon[37140]: pgmap v5301: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:14:55.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:14:54 smithi138 ceph-mon[36954]: pgmap v5302: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:14:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:14:54 smithi100 ceph-mon[32599]: pgmap v5302: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:14:55.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:14:54 smithi167 ceph-mon[37140]: pgmap v5302: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:14:57.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:14:56 smithi138 ceph-mon[36954]: pgmap v5303: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:14:57.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:14:56 smithi100 ceph-mon[32599]: pgmap v5303: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:14:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:14:56 smithi167 ceph-mon[37140]: pgmap v5303: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:14:59.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:14:58 smithi138 ceph-mon[36954]: pgmap v5304: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:14:59.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:14:58 smithi100 ceph-mon[32599]: pgmap v5304: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:14:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:14:58 smithi167 ceph-mon[37140]: pgmap v5304: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:15:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:15:00 smithi138 ceph-mon[36954]: pgmap v5305: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:15:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:15:00 smithi100 ceph-mon[32599]: pgmap v5305: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:15:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:15:00 smithi167 ceph-mon[37140]: pgmap v5305: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:15:03.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:15:02 smithi138 ceph-mon[36954]: pgmap v5306: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:15:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:15:02 smithi100 ceph-mon[32599]: pgmap v5306: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:15:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:15:02 smithi167 ceph-mon[37140]: pgmap v5306: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:15:04.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:15:04.263+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T16:15:04.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:15:04.263+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T16:15:04.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:15:04.263+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 71 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c020 con 0x7fe5a81087a0 2024-07-26T16:15:04.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:15:04.263+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 71 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T16:15:05.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:15:04 smithi138 ceph-mon[36954]: pgmap v5307: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:15:05.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:15:04 smithi100 ceph-mon[32599]: pgmap v5307: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:15:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:15:04 smithi167 ceph-mon[37140]: pgmap v5307: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:15:06.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:15:06.200+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:15:06.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:15:06.201+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:15:06.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:15:06.201+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:15:06.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:15:06.201+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:15:06.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:15:06.201+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:15:07.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:15:06 smithi100 ceph-mon[32599]: pgmap v5308: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:15:07.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:15:06 smithi167 ceph-mon[37140]: pgmap v5308: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:15:07.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:15:06 smithi138 ceph-mon[36954]: pgmap v5308: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:15:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:15:08 smithi100 ceph-mon[32599]: pgmap v5309: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:15:09.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:15:08 smithi100 ceph-mon[32599]: osdmap e378: 12 total, 12 up, 12 in 2024-07-26T16:15:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:15:08 smithi167 ceph-mon[37140]: pgmap v5309: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:15:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:15:08 smithi167 ceph-mon[37140]: osdmap e378: 12 total, 12 up, 12 in 2024-07-26T16:15:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:15:08 smithi138 ceph-mon[36954]: pgmap v5309: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:15:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:15:08 smithi138 ceph-mon[36954]: osdmap e378: 12 total, 12 up, 12 in 2024-07-26T16:15:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:15:10 smithi100 ceph-mon[32599]: pgmap v5311: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:15:11.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:15:10 smithi167 ceph-mon[37140]: pgmap v5311: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:15:11.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:15:10 smithi138 ceph-mon[36954]: pgmap v5311: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:15:13.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:15:12 smithi100 ceph-mon[32599]: pgmap v5312: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:15:13.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:15:12 smithi167 ceph-mon[37140]: pgmap v5312: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:15:13.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:15:12 smithi138 ceph-mon[36954]: pgmap v5312: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:15:15.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:15:14 smithi100 ceph-mon[32599]: pgmap v5313: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:15:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:15:14 smithi167 ceph-mon[37140]: pgmap v5313: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:15:15.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:15:14 smithi138 ceph-mon[36954]: pgmap v5313: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:15:17.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:15:16 smithi100 ceph-mon[32599]: pgmap v5314: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:15:17.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:15:16 smithi100 ceph-mon[32599]: osdmap e379: 12 total, 12 up, 12 in 2024-07-26T16:15:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:15:16 smithi167 ceph-mon[37140]: pgmap v5314: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:15:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:15:16 smithi167 ceph-mon[37140]: osdmap e379: 12 total, 12 up, 12 in 2024-07-26T16:15:17.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:15:16 smithi138 ceph-mon[36954]: pgmap v5314: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:15:17.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:15:16 smithi138 ceph-mon[36954]: osdmap e379: 12 total, 12 up, 12 in 2024-07-26T16:15:19.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:15:18 smithi100 ceph-mon[32599]: pgmap v5316: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 214 B/s wr, 6 op/s 2024-07-26T16:15:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:15:18 smithi167 ceph-mon[37140]: pgmap v5316: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 214 B/s wr, 6 op/s 2024-07-26T16:15:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:15:18 smithi138 ceph-mon[36954]: pgmap v5316: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 214 B/s wr, 6 op/s 2024-07-26T16:15:21.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:15:21.216+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:15:21.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:15:21.217+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:15:21.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:15:21.217+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:15:21.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:15:21.217+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:15:21.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:15:21.217+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:15:21.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:15:20 smithi100 ceph-mon[32599]: pgmap v5317: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:15:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:15:20 smithi167 ceph-mon[37140]: pgmap v5317: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:15:21.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:15:20 smithi138 ceph-mon[36954]: pgmap v5317: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:15:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:15:22 smithi100 ceph-mon[32599]: pgmap v5318: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:15:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:15:22 smithi167 ceph-mon[37140]: pgmap v5318: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:15:23.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:15:22 smithi138 ceph-mon[36954]: pgmap v5318: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:15:25.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:15:24 smithi100 ceph-mon[32599]: pgmap v5319: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:15:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:15:24 smithi167 ceph-mon[37140]: pgmap v5319: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:15:25.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:15:24 smithi138 ceph-mon[36954]: pgmap v5319: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:15:26.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:15:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:15:26.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:15:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:15:26.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:15:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:15:26.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:15:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:15:26.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:15:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:15:26.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:15:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:15:26.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:15:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:15:26.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:15:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:15:26.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:15:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:15:26.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:15:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:15:26.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:15:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:15:26.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:15:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:15:27.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:15:26 smithi100 ceph-mon[32599]: pgmap v5320: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:15:27.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:15:26 smithi167 ceph-mon[37140]: pgmap v5320: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:15:27.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:15:26 smithi138 ceph-mon[36954]: pgmap v5320: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:15:29.243 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:15:28 smithi138 ceph-mon[36954]: pgmap v5321: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:15:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:15:28 smithi100 ceph-mon[32599]: pgmap v5321: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:15:29.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:15:28 smithi167 ceph-mon[37140]: pgmap v5321: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:15:31.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:15:30 smithi100 ceph-mon[32599]: pgmap v5322: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:15:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:15:30 smithi167 ceph-mon[37140]: pgmap v5322: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:15:31.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:15:30 smithi138 ceph-mon[36954]: pgmap v5322: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:15:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:15:32 smithi167 ceph-mon[37140]: pgmap v5323: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:15:33.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:15:32 smithi138 ceph-mon[36954]: pgmap v5323: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:15:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:15:32 smithi100 ceph-mon[32599]: pgmap v5323: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:15:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:15:35 smithi167 ceph-mon[37140]: pgmap v5324: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:15:35.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:15:35 smithi138 ceph-mon[36954]: pgmap v5324: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:15:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:15:34 smithi100 ceph-mon[32599]: pgmap v5324: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:15:36.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:15:36.232+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:15:36.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:15:36.233+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:15:36.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:15:36.233+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:15:36.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:15:36.233+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:15:36.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:15:36.233+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:15:37.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:15:37 smithi167 ceph-mon[37140]: pgmap v5325: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:15:37.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:15:37 smithi138 ceph-mon[36954]: pgmap v5325: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:15:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:15:37 smithi100 ceph-mon[32599]: pgmap v5325: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:15:39.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:15:39 smithi167 ceph-mon[37140]: pgmap v5326: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:15:39.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:15:39 smithi138 ceph-mon[36954]: pgmap v5326: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:15:39.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:15:39 smithi100 ceph-mon[32599]: pgmap v5326: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:15:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:15:41 smithi167 ceph-mon[37140]: pgmap v5327: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:15:41.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:15:41 smithi138 ceph-mon[36954]: pgmap v5327: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:15:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:15:41 smithi100 ceph-mon[32599]: pgmap v5327: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:15:43.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:15:43 smithi167 ceph-mon[37140]: pgmap v5328: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:15:43.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:15:43 smithi138 ceph-mon[36954]: pgmap v5328: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:15:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:15:43 smithi100 ceph-mon[32599]: pgmap v5328: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:15:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:15:45 smithi167 ceph-mon[37140]: pgmap v5329: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:15:45.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:15:45 smithi138 ceph-mon[36954]: pgmap v5329: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:15:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:15:45 smithi100 ceph-mon[32599]: pgmap v5329: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:15:47.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:15:47 smithi167 ceph-mon[37140]: pgmap v5330: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:15:47.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:15:47 smithi138 ceph-mon[36954]: pgmap v5330: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:15:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:15:47 smithi100 ceph-mon[32599]: pgmap v5330: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:15:49.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:15:49 smithi138 ceph-mon[36954]: pgmap v5331: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:15:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:15:49 smithi100 ceph-mon[32599]: pgmap v5331: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:15:49.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:15:49 smithi167 ceph-mon[37140]: pgmap v5331: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:15:51.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:15:51.249+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:15:51.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:15:51.249+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:15:51.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:15:51.249+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:15:51.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:15:51.249+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:15:51.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:15:51.249+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:15:51.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:15:51 smithi100 ceph-mon[32599]: pgmap v5332: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:15:51.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:15:51 smithi138 ceph-mon[36954]: pgmap v5332: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:15:51.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:15:51 smithi167 ceph-mon[37140]: pgmap v5332: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:15:52.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:15:52 smithi138 ceph-mon[36954]: pgmap v5333: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:15:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:15:52 smithi100 ceph-mon[32599]: pgmap v5333: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:15:52.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:15:52 smithi167 ceph-mon[37140]: pgmap v5333: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:15:54.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:15:54 smithi138 ceph-mon[36954]: pgmap v5334: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:15:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:15:54 smithi100 ceph-mon[32599]: pgmap v5334: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:15:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:15:54 smithi167 ceph-mon[37140]: pgmap v5334: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:15:56.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:15:56 smithi138 ceph-mon[36954]: pgmap v5335: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:15:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:15:56 smithi100 ceph-mon[32599]: pgmap v5335: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:15:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:15:56 smithi167 ceph-mon[37140]: pgmap v5335: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:15:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:15:58 smithi138 ceph-mon[36954]: pgmap v5336: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:15:58.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:15:58 smithi100 ceph-mon[32599]: pgmap v5336: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:15:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:15:58 smithi167 ceph-mon[37140]: pgmap v5336: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:16:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:16:00 smithi138 ceph-mon[36954]: pgmap v5337: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:16:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:16:00 smithi100 ceph-mon[32599]: pgmap v5337: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:16:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:16:00 smithi167 ceph-mon[37140]: pgmap v5337: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:16:02.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:16:02 smithi138 ceph-mon[36954]: pgmap v5338: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:16:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:16:02 smithi100 ceph-mon[32599]: pgmap v5338: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:16:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:16:02 smithi167 ceph-mon[37140]: pgmap v5338: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:16:04.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:16:04 smithi138 ceph-mon[36954]: pgmap v5339: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:16:04.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:16:04 smithi100 ceph-mon[32599]: pgmap v5339: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:16:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:16:04 smithi167 ceph-mon[37140]: pgmap v5339: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:16:06.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:16:06.263+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:16:06.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:16:06.264+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:16:06.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:16:06.264+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:16:06.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:16:06.264+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:16:06.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:16:06.264+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:16:06.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:16:06 smithi138 ceph-mon[36954]: pgmap v5340: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:16:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:16:06 smithi100 ceph-mon[32599]: pgmap v5340: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:16:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:16:06 smithi167 ceph-mon[37140]: pgmap v5340: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:16:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:16:08 smithi138 ceph-mon[36954]: pgmap v5341: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:16:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:16:08 smithi138 ceph-mon[36954]: osdmap e380: 12 total, 12 up, 12 in 2024-07-26T16:16:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:16:08 smithi100 ceph-mon[32599]: pgmap v5341: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:16:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:16:08 smithi100 ceph-mon[32599]: osdmap e380: 12 total, 12 up, 12 in 2024-07-26T16:16:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:16:08 smithi167 ceph-mon[37140]: pgmap v5341: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:16:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:16:08 smithi167 ceph-mon[37140]: osdmap e380: 12 total, 12 up, 12 in 2024-07-26T16:16:10.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:16:10 smithi138 ceph-mon[36954]: pgmap v5343: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:16:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:16:10 smithi100 ceph-mon[32599]: pgmap v5343: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:16:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:16:10 smithi167 ceph-mon[37140]: pgmap v5343: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:16:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:16:12 smithi138 ceph-mon[36954]: pgmap v5344: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:16:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:16:12 smithi100 ceph-mon[32599]: pgmap v5344: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:16:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:16:12 smithi167 ceph-mon[37140]: pgmap v5344: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:16:14.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:16:14 smithi138 ceph-mon[36954]: pgmap v5345: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:16:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:16:14 smithi100 ceph-mon[32599]: pgmap v5345: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:16:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:16:14 smithi167 ceph-mon[37140]: pgmap v5345: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:16:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:16:16 smithi138 ceph-mon[36954]: pgmap v5346: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:16:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:16:16 smithi138 ceph-mon[36954]: osdmap e381: 12 total, 12 up, 12 in 2024-07-26T16:16:16.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:16:16 smithi100 ceph-mon[32599]: pgmap v5346: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:16:16.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:16:16 smithi100 ceph-mon[32599]: osdmap e381: 12 total, 12 up, 12 in 2024-07-26T16:16:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:16:16 smithi167 ceph-mon[37140]: pgmap v5346: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:16:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:16:16 smithi167 ceph-mon[37140]: osdmap e381: 12 total, 12 up, 12 in 2024-07-26T16:16:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:16:18 smithi138 ceph-mon[36954]: pgmap v5348: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 5 op/s 2024-07-26T16:16:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:16:18 smithi100 ceph-mon[32599]: pgmap v5348: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 5 op/s 2024-07-26T16:16:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:16:18 smithi167 ceph-mon[37140]: pgmap v5348: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 5 op/s 2024-07-26T16:16:20.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:16:20 smithi138 ceph-mon[36954]: pgmap v5349: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:16:20.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:16:20 smithi100 ceph-mon[32599]: pgmap v5349: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:16:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:16:20 smithi167 ceph-mon[37140]: pgmap v5349: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:16:21.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:16:21.279+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:16:21.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:16:21.280+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:16:21.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:16:21.280+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:16:21.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:16:21.280+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:16:21.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:16:21.280+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:16:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:16:22 smithi138 ceph-mon[36954]: pgmap v5350: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:16:22.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:16:22 smithi100 ceph-mon[32599]: pgmap v5350: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:16:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:16:22 smithi167 ceph-mon[37140]: pgmap v5350: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:16:24.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:16:24 smithi138 ceph-mon[36954]: pgmap v5351: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:16:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:16:24 smithi100 ceph-mon[32599]: pgmap v5351: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:16:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:16:24 smithi167 ceph-mon[37140]: pgmap v5351: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:16:26.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:16:26 smithi138 ceph-mon[36954]: pgmap v5352: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:16:26.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:16:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:16:26.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:16:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:16:26.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:16:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:16:26.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:16:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:16:26.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:16:26 smithi100 ceph-mon[32599]: pgmap v5352: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:16:26.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:16:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:16:26.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:16:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:16:26.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:16:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:16:26.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:16:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:16:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:16:26 smithi167 ceph-mon[37140]: pgmap v5352: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:16:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:16:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:16:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:16:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:16:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:16:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:16:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:16:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:16:28.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:16:28 smithi138 ceph-mon[36954]: pgmap v5353: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:16:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:16:28 smithi100 ceph-mon[32599]: pgmap v5353: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:16:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:16:28 smithi167 ceph-mon[37140]: pgmap v5353: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:16:30.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:16:30 smithi138 ceph-mon[36954]: pgmap v5354: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:16:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:16:30 smithi100 ceph-mon[32599]: pgmap v5354: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:16:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:16:30 smithi167 ceph-mon[37140]: pgmap v5354: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:16:32.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:16:32 smithi138 ceph-mon[36954]: pgmap v5355: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:16:32.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:16:32 smithi100 ceph-mon[32599]: pgmap v5355: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:16:32.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:16:32 smithi167 ceph-mon[37140]: pgmap v5355: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:16:34.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:16:34.275+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T16:16:34.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:16:34.275+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T16:16:34.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:16:34.275+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 72 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T16:16:34.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:16:34.275+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 72 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe5a008c070 con 0x7fe5a81087a0 2024-07-26T16:16:34.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:16:34 smithi138 ceph-mon[36954]: pgmap v5356: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:16:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:16:34 smithi100 ceph-mon[32599]: pgmap v5356: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:16:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:16:34 smithi167 ceph-mon[37140]: pgmap v5356: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:16:36.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:16:36.295+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:16:36.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:16:36.296+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:16:36.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:16:36.296+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:16:36.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:16:36.296+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:16:36.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:16:36.296+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:16:36.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:16:36 smithi138 ceph-mon[36954]: pgmap v5357: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:16:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:16:36 smithi100 ceph-mon[32599]: pgmap v5357: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:16:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:16:36 smithi167 ceph-mon[37140]: pgmap v5357: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:16:38.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:16:38 smithi138 ceph-mon[36954]: pgmap v5358: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:16:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:16:38 smithi100 ceph-mon[32599]: pgmap v5358: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:16:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:16:38 smithi167 ceph-mon[37140]: pgmap v5358: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:16:40.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:16:40 smithi138 ceph-mon[36954]: pgmap v5359: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:16:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:16:40 smithi100 ceph-mon[32599]: pgmap v5359: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:16:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:16:40 smithi167 ceph-mon[37140]: pgmap v5359: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:16:42.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:16:42 smithi138 ceph-mon[36954]: pgmap v5360: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:16:42.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:16:42 smithi100 ceph-mon[32599]: pgmap v5360: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:16:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:16:42 smithi167 ceph-mon[37140]: pgmap v5360: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:16:44.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:16:44 smithi138 ceph-mon[36954]: pgmap v5361: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:16:44.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:16:44 smithi100 ceph-mon[32599]: pgmap v5361: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:16:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:16:44 smithi167 ceph-mon[37140]: pgmap v5361: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:16:46.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:16:46 smithi138 ceph-mon[36954]: pgmap v5362: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:16:46.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:16:46 smithi100 ceph-mon[32599]: pgmap v5362: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:16:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:16:46 smithi167 ceph-mon[37140]: pgmap v5362: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:16:48.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:16:48 smithi138 ceph-mon[36954]: pgmap v5363: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:16:48.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:16:48 smithi100 ceph-mon[32599]: pgmap v5363: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:16:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:16:48 smithi167 ceph-mon[37140]: pgmap v5363: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:16:50.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:16:50 smithi138 ceph-mon[36954]: pgmap v5364: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:16:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:16:50 smithi100 ceph-mon[32599]: pgmap v5364: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:16:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:16:50 smithi167 ceph-mon[37140]: pgmap v5364: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:16:51.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:16:51.305+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:16:51.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:16:51.306+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:16:51.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:16:51.306+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:16:51.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:16:51.306+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:16:51.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:16:51.306+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:16:52.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:16:52 smithi138 ceph-mon[36954]: pgmap v5365: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:16:52.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:16:52 smithi100 ceph-mon[32599]: pgmap v5365: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:16:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:16:52 smithi167 ceph-mon[37140]: pgmap v5365: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:16:54.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:16:54 smithi138 ceph-mon[36954]: pgmap v5366: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:16:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:16:54 smithi100 ceph-mon[32599]: pgmap v5366: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:16:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:16:54 smithi167 ceph-mon[37140]: pgmap v5366: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:16:56.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:16:56 smithi138 ceph-mon[36954]: pgmap v5367: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:16:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:16:56 smithi100 ceph-mon[32599]: pgmap v5367: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:16:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:16:56 smithi167 ceph-mon[37140]: pgmap v5367: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:16:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:16:58 smithi138 ceph-mon[36954]: pgmap v5368: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:16:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:16:58 smithi100 ceph-mon[32599]: pgmap v5368: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:16:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:16:58 smithi167 ceph-mon[37140]: pgmap v5368: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:17:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:17:00 smithi138 ceph-mon[36954]: pgmap v5369: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:17:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:17:00 smithi100 ceph-mon[32599]: pgmap v5369: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:17:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:17:00 smithi167 ceph-mon[37140]: pgmap v5369: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:17:02.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:17:02 smithi138 ceph-mon[36954]: pgmap v5370: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:17:02.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:17:02 smithi100 ceph-mon[32599]: pgmap v5370: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:17:02.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:17:02 smithi167 ceph-mon[37140]: pgmap v5370: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:17:04.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:17:04 smithi138 ceph-mon[36954]: pgmap v5371: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:17:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:17:04 smithi100 ceph-mon[32599]: pgmap v5371: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:17:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:17:04 smithi167 ceph-mon[37140]: pgmap v5371: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:17:06.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:17:06.322+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:17:06.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:17:06.322+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:17:06.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:17:06.322+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:17:06.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:17:06.322+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:17:06.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:17:06.322+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:17:06.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:17:06 smithi138 ceph-mon[36954]: pgmap v5372: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:17:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:17:06 smithi100 ceph-mon[32599]: pgmap v5372: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:17:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:17:06 smithi167 ceph-mon[37140]: pgmap v5372: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:17:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:17:08 smithi138 ceph-mon[36954]: pgmap v5373: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:17:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:17:08 smithi138 ceph-mon[36954]: osdmap e382: 12 total, 12 up, 12 in 2024-07-26T16:17:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:17:08 smithi100 ceph-mon[32599]: pgmap v5373: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:17:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:17:08 smithi100 ceph-mon[32599]: osdmap e382: 12 total, 12 up, 12 in 2024-07-26T16:17:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:17:08 smithi167 ceph-mon[37140]: pgmap v5373: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:17:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:17:08 smithi167 ceph-mon[37140]: osdmap e382: 12 total, 12 up, 12 in 2024-07-26T16:17:10.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:17:10 smithi138 ceph-mon[36954]: pgmap v5375: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:17:10.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:17:10 smithi100 ceph-mon[32599]: pgmap v5375: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:17:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:17:10 smithi167 ceph-mon[37140]: pgmap v5375: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:17:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:17:12 smithi138 ceph-mon[36954]: pgmap v5376: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:17:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:17:12 smithi100 ceph-mon[32599]: pgmap v5376: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:17:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:17:12 smithi167 ceph-mon[37140]: pgmap v5376: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:17:14.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:17:14 smithi138 ceph-mon[36954]: pgmap v5377: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:17:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:17:14 smithi100 ceph-mon[32599]: pgmap v5377: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:17:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:17:14 smithi167 ceph-mon[37140]: pgmap v5377: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:17:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:17:16 smithi138 ceph-mon[36954]: pgmap v5378: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:17:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:17:16 smithi138 ceph-mon[36954]: osdmap e383: 12 total, 12 up, 12 in 2024-07-26T16:17:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:17:16 smithi100 ceph-mon[32599]: pgmap v5378: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:17:16.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:17:16 smithi100 ceph-mon[32599]: osdmap e383: 12 total, 12 up, 12 in 2024-07-26T16:17:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:17:16 smithi167 ceph-mon[37140]: pgmap v5378: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:17:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:17:16 smithi167 ceph-mon[37140]: osdmap e383: 12 total, 12 up, 12 in 2024-07-26T16:17:18.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:17:18 smithi138 ceph-mon[36954]: pgmap v5380: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 5 op/s 2024-07-26T16:17:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:17:18 smithi100 ceph-mon[32599]: pgmap v5380: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 5 op/s 2024-07-26T16:17:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:17:18 smithi167 ceph-mon[37140]: pgmap v5380: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 5 op/s 2024-07-26T16:17:20.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:17:20 smithi138 ceph-mon[36954]: pgmap v5381: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:17:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:17:20 smithi100 ceph-mon[32599]: pgmap v5381: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:17:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:17:20 smithi167 ceph-mon[37140]: pgmap v5381: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:17:21.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:17:21.322+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:17:21.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:17:21.323+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:17:21.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:17:21.323+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:17:21.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:17:21.323+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:17:21.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:17:21.323+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:17:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:17:22 smithi138 ceph-mon[36954]: pgmap v5382: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:17:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:17:22 smithi100 ceph-mon[32599]: pgmap v5382: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:17:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:17:22 smithi167 ceph-mon[37140]: pgmap v5382: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:17:24.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:17:24 smithi138 ceph-mon[36954]: pgmap v5383: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:17:24.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:17:24 smithi100 ceph-mon[32599]: pgmap v5383: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:17:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:17:24 smithi167 ceph-mon[37140]: pgmap v5383: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:17:26.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:17:26 smithi138 ceph-mon[36954]: pgmap v5384: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:17:26.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:17:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:17:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:17:26 smithi100 ceph-mon[32599]: pgmap v5384: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:17:26.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:17:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:17:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:17:26 smithi167 ceph-mon[37140]: pgmap v5384: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:17:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:17:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:17:27.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:17:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:17:27.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:17:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:17:27.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:17:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:17:27.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:17:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:17:27.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:17:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:17:27.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:17:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:17:28.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:17:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:17:28.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:17:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:17:28.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:17:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:17:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:17:28 smithi100 ceph-mon[32599]: pgmap v5385: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:17:29.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:17:28 smithi167 ceph-mon[37140]: pgmap v5385: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:17:29.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:17:28 smithi138 ceph-mon[36954]: pgmap v5385: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-07-26T16:17:31.243 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:17:30 smithi138 ceph-mon[36954]: pgmap v5386: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:17:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:17:30 smithi100 ceph-mon[32599]: pgmap v5386: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:17:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:17:30 smithi167 ceph-mon[37140]: pgmap v5386: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:17:33.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:17:32 smithi100 ceph-mon[32599]: pgmap v5387: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:17:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:17:32 smithi167 ceph-mon[37140]: pgmap v5387: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:17:33.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:17:32 smithi138 ceph-mon[36954]: pgmap v5387: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:17:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:17:34 smithi100 ceph-mon[32599]: pgmap v5388: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:17:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:17:34 smithi167 ceph-mon[37140]: pgmap v5388: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:17:35.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:17:34 smithi138 ceph-mon[36954]: pgmap v5388: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:17:36.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:17:36.339+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:17:36.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:17:36.339+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:17:36.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:17:36.339+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:17:36.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:17:36.339+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:17:36.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:17:36.339+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:17:37.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:17:36 smithi100 ceph-mon[32599]: pgmap v5389: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:17:37.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:17:36 smithi167 ceph-mon[37140]: pgmap v5389: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:17:37.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:17:36 smithi138 ceph-mon[36954]: pgmap v5389: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:17:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:17:38 smithi100 ceph-mon[32599]: pgmap v5390: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:17:39.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:17:38 smithi167 ceph-mon[37140]: pgmap v5390: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:17:39.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:17:38 smithi138 ceph-mon[36954]: pgmap v5390: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:17:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:17:40 smithi167 ceph-mon[37140]: pgmap v5391: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:17:41.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:17:40 smithi138 ceph-mon[36954]: pgmap v5391: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:17:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:17:40 smithi100 ceph-mon[32599]: pgmap v5391: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:17:43.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:17:42 smithi167 ceph-mon[37140]: pgmap v5392: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:17:43.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:17:42 smithi138 ceph-mon[36954]: pgmap v5392: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:17:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:17:42 smithi100 ceph-mon[32599]: pgmap v5392: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:17:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:17:45 smithi167 ceph-mon[37140]: pgmap v5393: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:17:45.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:17:45 smithi138 ceph-mon[36954]: pgmap v5393: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:17:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:17:45 smithi100 ceph-mon[32599]: pgmap v5393: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:17:47.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:17:47 smithi167 ceph-mon[37140]: pgmap v5394: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:17:47.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:17:47 smithi138 ceph-mon[36954]: pgmap v5394: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:17:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:17:47 smithi100 ceph-mon[32599]: pgmap v5394: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:17:49.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:17:49 smithi167 ceph-mon[37140]: pgmap v5395: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:17:49.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:17:49 smithi138 ceph-mon[36954]: pgmap v5395: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:17:49.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:17:49 smithi100 ceph-mon[32599]: pgmap v5395: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:17:51.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:17:51 smithi167 ceph-mon[37140]: pgmap v5396: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:17:51.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:17:51 smithi100 ceph-mon[32599]: pgmap v5396: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:17:51.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:17:51.354+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:17:51.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:17:51.354+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:17:51.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:17:51.354+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:17:51.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:17:51.354+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:17:51.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:17:51.354+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:17:51.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:17:51 smithi138 ceph-mon[36954]: pgmap v5396: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:17:53.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:17:53 smithi167 ceph-mon[37140]: pgmap v5397: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:17:53.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:17:53 smithi138 ceph-mon[36954]: pgmap v5397: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:17:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:17:53 smithi100 ceph-mon[32599]: pgmap v5397: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:17:55.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:17:55 smithi167 ceph-mon[37140]: pgmap v5398: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:17:55.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:17:55 smithi138 ceph-mon[36954]: pgmap v5398: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:17:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:17:55 smithi100 ceph-mon[32599]: pgmap v5398: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:17:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:17:57 smithi167 ceph-mon[37140]: pgmap v5399: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:17:57.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:17:57 smithi138 ceph-mon[36954]: pgmap v5399: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:17:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:17:57 smithi100 ceph-mon[32599]: pgmap v5399: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:17:59.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:17:59 smithi138 ceph-mon[36954]: pgmap v5400: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:17:59.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:17:59 smithi100 ceph-mon[32599]: pgmap v5400: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:17:59.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:17:59 smithi167 ceph-mon[37140]: pgmap v5400: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:18:01.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:01 smithi138 ceph-mon[36954]: pgmap v5401: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:18:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:01 smithi100 ceph-mon[32599]: pgmap v5401: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:18:01.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:01 smithi167 ceph-mon[37140]: pgmap v5401: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:18:03.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:18:03.036+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 msgr2=0x7fe5940c0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-07-26T16:18:03.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:18:03.036+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 msgr2=0x7fe5940c0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-07-26T16:18:03.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:18:03.036+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe590004ee0 tx=0x7fe59005a040 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:18:03.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:18:03.036+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe590004ee0 tx=0x7fe59005a040 comp rx=0 tx=0).stop 2024-07-26T16:18:03.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:18:03.036+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 msgr2=0x7fe5940c0480 unknown :-1 s=STATE_CLOSED l=1).tick see no progress in more than 10000000 us during connecting to v2:172.21.15.100:6800/3937009534, fault. 2024-07-26T16:18:03.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:18:03.036+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 msgr2=0x7fe5940c0480 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-07-26T16:18:03.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:18:03.036+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T16:18:03.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:18:03.036+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b4ef0 0x7fe5940b5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T16:18:03.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:18:03.036+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b4ef0 0x7fe5940b5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:18:03.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:18:03.037+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b4ef0 0x7fe5940b5510 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fe590004ee0 tx=0x7fe59008a040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T16:18:03.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:03 smithi138 ceph-mon[36954]: pgmap v5402: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:18:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:03 smithi100 ceph-mon[32599]: pgmap v5402: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:18:03.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:03 smithi167 ceph-mon[37140]: pgmap v5402: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:18:04.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:18:04.289+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T16:18:04.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:18:04.289+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T16:18:04.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:18:04.289+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 73 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c020 con 0x7fe5a81087a0 2024-07-26T16:18:04.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:18:04.289+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 73 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T16:18:05.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:05 smithi138 ceph-mon[36954]: pgmap v5403: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:18:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:05 smithi100 ceph-mon[32599]: pgmap v5403: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:18:05.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:05 smithi167 ceph-mon[37140]: pgmap v5403: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:18:06.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:18:06.358+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:18:06.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:18:06.358+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:18:06.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:18:06.358+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:18:06.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:18:06.358+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:18:06.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:18:06.358+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:18:07.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:07 smithi138 ceph-mon[36954]: pgmap v5404: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:18:07.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:07 smithi100 ceph-mon[32599]: pgmap v5404: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:18:07.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:07 smithi167 ceph-mon[37140]: pgmap v5404: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:18:08.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:08 smithi138 ceph-mon[36954]: osdmap e384: 12 total, 12 up, 12 in 2024-07-26T16:18:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:08 smithi100 ceph-mon[32599]: osdmap e384: 12 total, 12 up, 12 in 2024-07-26T16:18:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:08 smithi167 ceph-mon[37140]: osdmap e384: 12 total, 12 up, 12 in 2024-07-26T16:18:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:09 smithi138 ceph-mon[36954]: pgmap v5406: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:18:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:09 smithi100 ceph-mon[32599]: pgmap v5406: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:18:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:09 smithi167 ceph-mon[37140]: pgmap v5406: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:18:10.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:10 smithi138 ceph-mon[36954]: pgmap v5407: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:18:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:10 smithi100 ceph-mon[32599]: pgmap v5407: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:18:10.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:10 smithi167 ceph-mon[37140]: pgmap v5407: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:18:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:12 smithi138 ceph-mon[36954]: pgmap v5408: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:18:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:12 smithi100 ceph-mon[32599]: pgmap v5408: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:18:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:12 smithi167 ceph-mon[37140]: pgmap v5408: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:18:14.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:14 smithi138 ceph-mon[36954]: pgmap v5409: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:18:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:14 smithi100 ceph-mon[32599]: pgmap v5409: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:18:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:14 smithi167 ceph-mon[37140]: pgmap v5409: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:18:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:16 smithi138 ceph-mon[36954]: pgmap v5410: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:18:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:16 smithi138 ceph-mon[36954]: osdmap e385: 12 total, 12 up, 12 in 2024-07-26T16:18:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:16 smithi100 ceph-mon[32599]: pgmap v5410: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:18:16.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:16 smithi100 ceph-mon[32599]: osdmap e385: 12 total, 12 up, 12 in 2024-07-26T16:18:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:16 smithi167 ceph-mon[37140]: pgmap v5410: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:18:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:16 smithi167 ceph-mon[37140]: osdmap e385: 12 total, 12 up, 12 in 2024-07-26T16:18:18.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:18 smithi138 ceph-mon[36954]: pgmap v5412: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:18:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:18 smithi100 ceph-mon[32599]: pgmap v5412: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:18:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:18 smithi167 ceph-mon[37140]: pgmap v5412: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:18:20.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:20 smithi138 ceph-mon[36954]: pgmap v5413: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:18:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:20 smithi100 ceph-mon[32599]: pgmap v5413: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:18:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:20 smithi167 ceph-mon[37140]: pgmap v5413: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:18:21.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:18:21.374+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:18:21.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:18:21.374+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:18:21.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:18:21.374+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:18:21.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:18:21.374+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:18:21.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:18:21.374+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:18:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:22 smithi138 ceph-mon[36954]: pgmap v5414: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:18:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:22 smithi100 ceph-mon[32599]: pgmap v5414: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:18:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:22 smithi167 ceph-mon[37140]: pgmap v5414: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:18:24.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:24 smithi138 ceph-mon[36954]: pgmap v5415: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:18:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:24 smithi100 ceph-mon[32599]: pgmap v5415: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:18:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:24 smithi167 ceph-mon[37140]: pgmap v5415: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:18:26.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:26 smithi138 ceph-mon[36954]: pgmap v5416: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:18:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:26 smithi100 ceph-mon[32599]: pgmap v5416: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:18:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:26 smithi167 ceph-mon[37140]: pgmap v5416: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:18:27.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:18:27.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:18:27.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:18:27.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:18:27.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:18:27.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:18:27.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:18:27.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:18:27.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:18:27.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:18:27.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:18:27.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:18:27.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:18:27.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:18:27.944 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:18:27.944 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:18:27.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:18:27.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:18:27.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:18:27.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:18:27.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:18:27.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:18:27.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:18:27.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:18:27.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:18:27.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:18:27.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:18:27.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:18:27.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:18:27.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:18:27.997 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:18:27.997 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:18:28.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:18:28.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:18:28.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:18:28.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:18:28.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:18:28.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:18:28.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:18:28.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:18:28.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:18:28.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:18:28.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:18:28.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:18:28.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:18:28.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:18:28.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:18:28.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:18:28.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:28 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T16:18:28.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:28 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T16:18:28.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:28 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T16:18:28.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:28 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T16:18:28.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:28 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T16:18:28.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:28 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T16:18:28.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:28 smithi138 ceph-mon[36954]: pgmap v5417: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:18:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:28 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T16:18:28.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:28 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T16:18:28.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:28 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T16:18:28.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:28 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T16:18:28.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:28 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T16:18:28.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:28 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T16:18:28.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:28 smithi100 ceph-mon[32599]: pgmap v5417: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:18:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:28 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T16:18:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:28 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T16:18:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:28 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T16:18:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:28 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T16:18:29.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:28 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T16:18:29.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:28 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T16:18:29.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:28 smithi167 ceph-mon[37140]: pgmap v5417: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:18:30.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:30 smithi138 ceph-mon[36954]: pgmap v5418: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:18:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:30 smithi100 ceph-mon[32599]: pgmap v5418: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:18:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:30 smithi167 ceph-mon[37140]: pgmap v5418: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:18:32.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:32 smithi138 ceph-mon[36954]: pgmap v5419: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T16:18:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:32 smithi100 ceph-mon[32599]: pgmap v5419: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T16:18:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:32 smithi167 ceph-mon[37140]: pgmap v5419: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T16:18:34.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:34 smithi138 ceph-mon[36954]: pgmap v5420: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:18:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:34 smithi100 ceph-mon[32599]: pgmap v5420: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:18:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:34 smithi167 ceph-mon[37140]: pgmap v5420: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:18:36.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:18:36.390+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:18:36.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:18:36.390+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:18:36.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:18:36.390+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:18:36.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:18:36.390+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:18:36.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:18:36.390+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:18:36.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:36 smithi138 ceph-mon[36954]: pgmap v5421: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:18:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:36 smithi100 ceph-mon[32599]: pgmap v5421: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:18:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:36 smithi167 ceph-mon[37140]: pgmap v5421: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:18:38.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:38 smithi138 ceph-mon[36954]: pgmap v5422: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:18:38.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:38 smithi100 ceph-mon[32599]: pgmap v5422: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:18:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:38 smithi167 ceph-mon[37140]: pgmap v5422: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:18:40.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:40 smithi138 ceph-mon[36954]: pgmap v5423: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:18:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:40 smithi100 ceph-mon[32599]: pgmap v5423: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:18:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:40 smithi167 ceph-mon[37140]: pgmap v5423: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:18:42.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:42 smithi138 ceph-mon[36954]: pgmap v5424: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:18:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:42 smithi100 ceph-mon[32599]: pgmap v5424: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:18:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:42 smithi167 ceph-mon[37140]: pgmap v5424: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:18:44.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:44 smithi138 ceph-mon[36954]: pgmap v5425: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:18:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:44 smithi100 ceph-mon[32599]: pgmap v5425: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:18:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:44 smithi167 ceph-mon[37140]: pgmap v5425: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:18:46.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:46 smithi138 ceph-mon[36954]: pgmap v5426: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:18:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:46 smithi100 ceph-mon[32599]: pgmap v5426: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:18:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:46 smithi167 ceph-mon[37140]: pgmap v5426: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:18:48.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:48 smithi138 ceph-mon[36954]: pgmap v5427: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:18:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:48 smithi100 ceph-mon[32599]: pgmap v5427: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:18:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:48 smithi167 ceph-mon[37140]: pgmap v5427: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:18:50.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:50 smithi138 ceph-mon[36954]: pgmap v5428: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:18:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:50 smithi100 ceph-mon[32599]: pgmap v5428: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:18:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:50 smithi167 ceph-mon[37140]: pgmap v5428: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:18:51.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:18:51.406+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:18:51.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:18:51.407+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:18:51.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:18:51.407+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:18:51.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:18:51.407+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:18:51.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:18:51.407+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:18:52.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:52 smithi138 ceph-mon[36954]: pgmap v5429: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T16:18:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:52 smithi100 ceph-mon[32599]: pgmap v5429: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T16:18:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:52 smithi167 ceph-mon[37140]: pgmap v5429: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T16:18:54.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:54 smithi100 ceph-mon[32599]: pgmap v5430: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:18:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:54 smithi167 ceph-mon[37140]: pgmap v5430: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:18:55.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:54 smithi138 ceph-mon[36954]: pgmap v5430: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:18:56.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:56 smithi100 ceph-mon[32599]: pgmap v5431: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:18:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:56 smithi167 ceph-mon[37140]: pgmap v5431: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:18:57.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:56 smithi138 ceph-mon[36954]: pgmap v5431: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:18:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:18:58 smithi100 ceph-mon[32599]: pgmap v5432: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:18:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:18:58 smithi167 ceph-mon[37140]: pgmap v5432: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:18:59.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:18:58 smithi138 ceph-mon[36954]: pgmap v5432: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:19:00.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:19:00 smithi100 ceph-mon[32599]: pgmap v5433: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:19:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:19:00 smithi167 ceph-mon[37140]: pgmap v5433: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:19:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:19:00 smithi138 ceph-mon[36954]: pgmap v5433: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:19:02.993 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:19:02 smithi138 ceph-mon[36954]: pgmap v5434: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:19:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:19:02 smithi100 ceph-mon[32599]: pgmap v5434: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:19:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:19:02 smithi167 ceph-mon[37140]: pgmap v5434: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:19:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:19:04 smithi100 ceph-mon[32599]: pgmap v5435: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:19:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:19:04 smithi167 ceph-mon[37140]: pgmap v5435: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:19:05.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:19:04 smithi138 ceph-mon[36954]: pgmap v5435: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:19:06.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:19:06.422+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:19:06.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:19:06.422+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:19:06.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:19:06.422+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:19:06.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:19:06.422+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:19:06.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:19:06.422+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:19:06.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:19:06 smithi100 ceph-mon[32599]: pgmap v5436: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:19:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:19:06 smithi167 ceph-mon[37140]: pgmap v5436: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:19:07.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:19:06 smithi138 ceph-mon[36954]: pgmap v5436: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:19:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:19:08 smithi100 ceph-mon[32599]: pgmap v5437: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:19:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:19:08 smithi100 ceph-mon[32599]: osdmap e386: 12 total, 12 up, 12 in 2024-07-26T16:19:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:19:08 smithi167 ceph-mon[37140]: pgmap v5437: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:19:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:19:08 smithi167 ceph-mon[37140]: osdmap e386: 12 total, 12 up, 12 in 2024-07-26T16:19:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:19:08 smithi138 ceph-mon[36954]: pgmap v5437: 97 pgs: 97 active+clean; 234 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:19:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:19:08 smithi138 ceph-mon[36954]: osdmap e386: 12 total, 12 up, 12 in 2024-07-26T16:19:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:19:10 smithi100 ceph-mon[32599]: pgmap v5439: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:19:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:19:10 smithi167 ceph-mon[37140]: pgmap v5439: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:19:11.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:19:10 smithi138 ceph-mon[36954]: pgmap v5439: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:19:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:19:12 smithi100 ceph-mon[32599]: pgmap v5440: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:19:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:19:12 smithi167 ceph-mon[37140]: pgmap v5440: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:19:13.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:19:12 smithi138 ceph-mon[36954]: pgmap v5440: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:19:14.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:19:14 smithi100 ceph-mon[32599]: pgmap v5441: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:19:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:19:14 smithi167 ceph-mon[37140]: pgmap v5441: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:19:15.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:19:14 smithi138 ceph-mon[36954]: pgmap v5441: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:19:16.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:19:16 smithi100 ceph-mon[32599]: pgmap v5442: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:19:16.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:19:16 smithi100 ceph-mon[32599]: osdmap e387: 12 total, 12 up, 12 in 2024-07-26T16:19:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:19:16 smithi167 ceph-mon[37140]: pgmap v5442: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:19:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:19:16 smithi167 ceph-mon[37140]: osdmap e387: 12 total, 12 up, 12 in 2024-07-26T16:19:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:19:16 smithi138 ceph-mon[36954]: pgmap v5442: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:19:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:19:16 smithi138 ceph-mon[36954]: osdmap e387: 12 total, 12 up, 12 in 2024-07-26T16:19:18.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:19:18 smithi100 ceph-mon[32599]: pgmap v5444: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 5 op/s 2024-07-26T16:19:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:19:18 smithi167 ceph-mon[37140]: pgmap v5444: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 5 op/s 2024-07-26T16:19:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:19:18 smithi138 ceph-mon[36954]: pgmap v5444: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 5 op/s 2024-07-26T16:19:20.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:19:20 smithi100 ceph-mon[32599]: pgmap v5445: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:19:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:19:20 smithi167 ceph-mon[37140]: pgmap v5445: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:19:21.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:19:20 smithi138 ceph-mon[36954]: pgmap v5445: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:19:21.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:19:21.438+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:19:21.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:19:21.438+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:19:21.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:19:21.438+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:19:21.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:19:21.438+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:19:21.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:19:21.438+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:19:22.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:19:22 smithi100 ceph-mon[32599]: pgmap v5446: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:19:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:19:22 smithi167 ceph-mon[37140]: pgmap v5446: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:19:23.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:19:22 smithi138 ceph-mon[36954]: pgmap v5446: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:19:24.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:19:24 smithi100 ceph-mon[32599]: pgmap v5447: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:19:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:19:24 smithi167 ceph-mon[37140]: pgmap v5447: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:19:25.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:19:24 smithi138 ceph-mon[36954]: pgmap v5447: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:19:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:19:26 smithi167 ceph-mon[37140]: pgmap v5448: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:19:27.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:19:26 smithi138 ceph-mon[36954]: pgmap v5448: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:19:27.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:19:26 smithi100 ceph-mon[32599]: pgmap v5448: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:19:28.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:19:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:19:28.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:19:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:19:28.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:19:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:19:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:19:28 smithi167 ceph-mon[37140]: pgmap v5449: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:19:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:19:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:19:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:19:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:19:29.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:19:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:19:29.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:19:28 smithi138 ceph-mon[36954]: pgmap v5449: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:19:29.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:19:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:19:29.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:19:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:19:29.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:19:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:19:29.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:19:28 smithi100 ceph-mon[32599]: pgmap v5449: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:19:29.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:19:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:19:29.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:19:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:19:29.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:19:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:19:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:19:30 smithi167 ceph-mon[37140]: pgmap v5450: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:19:31.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:19:30 smithi138 ceph-mon[36954]: pgmap v5450: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:19:31.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:19:30 smithi100 ceph-mon[32599]: pgmap v5450: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:19:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:19:32 smithi167 ceph-mon[37140]: pgmap v5451: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:19:33.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:19:32 smithi138 ceph-mon[36954]: pgmap v5451: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:19:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:19:32 smithi100 ceph-mon[32599]: pgmap v5451: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:19:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:19:34 smithi167 ceph-mon[37140]: pgmap v5452: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:19:35.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:19:34 smithi138 ceph-mon[36954]: pgmap v5452: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:19:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:19:34 smithi100 ceph-mon[32599]: pgmap v5452: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:19:36.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:19:36.454+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:19:36.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:19:36.454+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:19:36.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:19:36.454+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:19:36.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:19:36.454+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:19:36.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:19:36.454+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:19:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:19:36 smithi167 ceph-mon[37140]: pgmap v5453: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:19:37.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:19:36 smithi138 ceph-mon[36954]: pgmap v5453: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:19:37.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:19:36 smithi100 ceph-mon[32599]: pgmap v5453: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:19:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:19:38 smithi167 ceph-mon[37140]: pgmap v5454: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:19:39.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:19:38 smithi138 ceph-mon[36954]: pgmap v5454: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:19:39.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:19:38 smithi100 ceph-mon[32599]: pgmap v5454: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:19:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:19:40 smithi167 ceph-mon[37140]: pgmap v5455: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:19:41.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:19:40 smithi138 ceph-mon[36954]: pgmap v5455: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:19:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:19:40 smithi100 ceph-mon[32599]: pgmap v5455: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:19:42.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:19:42 smithi100 ceph-mon[32599]: pgmap v5456: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:19:42.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:19:42 smithi167 ceph-mon[37140]: pgmap v5456: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:19:42.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:19:42 smithi138 ceph-mon[36954]: pgmap v5456: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:19:44.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:19:44 smithi138 ceph-mon[36954]: pgmap v5457: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:19:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:19:44 smithi100 ceph-mon[32599]: pgmap v5457: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:19:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:19:44 smithi167 ceph-mon[37140]: pgmap v5457: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:19:46.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:19:46 smithi138 ceph-mon[36954]: pgmap v5458: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:19:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:19:46 smithi100 ceph-mon[32599]: pgmap v5458: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:19:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:19:46 smithi167 ceph-mon[37140]: pgmap v5458: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:19:48.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:19:48 smithi138 ceph-mon[36954]: pgmap v5459: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:19:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:19:48 smithi100 ceph-mon[32599]: pgmap v5459: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:19:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:19:48 smithi167 ceph-mon[37140]: pgmap v5459: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:19:50.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:19:50 smithi138 ceph-mon[36954]: pgmap v5460: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:19:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:19:50 smithi100 ceph-mon[32599]: pgmap v5460: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:19:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:19:50 smithi167 ceph-mon[37140]: pgmap v5460: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:19:51.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:19:51.470+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:19:51.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:19:51.471+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:19:51.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:19:51.471+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:19:51.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:19:51.471+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:19:51.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:19:51.471+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:19:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:19:52 smithi138 ceph-mon[36954]: pgmap v5461: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:19:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:19:52 smithi100 ceph-mon[32599]: pgmap v5461: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:19:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:19:52 smithi167 ceph-mon[37140]: pgmap v5461: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:19:54.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:19:54 smithi138 ceph-mon[36954]: pgmap v5462: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:19:54.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:19:54 smithi100 ceph-mon[32599]: pgmap v5462: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:19:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:19:54 smithi167 ceph-mon[37140]: pgmap v5462: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:19:56.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:19:56 smithi138 ceph-mon[36954]: pgmap v5463: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:19:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:19:56 smithi100 ceph-mon[32599]: pgmap v5463: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:19:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:19:56 smithi167 ceph-mon[37140]: pgmap v5463: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:19:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:19:58 smithi138 ceph-mon[36954]: pgmap v5464: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:19:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:19:58 smithi100 ceph-mon[32599]: pgmap v5464: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:19:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:19:58 smithi167 ceph-mon[37140]: pgmap v5464: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:20:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:20:00 smithi138 ceph-mon[36954]: pgmap v5465: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:20:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:20:00 smithi138 ceph-mon[36954]: overall HEALTH_OK 2024-07-26T16:20:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:20:00 smithi100 ceph-mon[32599]: pgmap v5465: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:20:00.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:20:00 smithi100 ceph-mon[32599]: overall HEALTH_OK 2024-07-26T16:20:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:20:00 smithi167 ceph-mon[37140]: pgmap v5465: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:20:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:20:00 smithi167 ceph-mon[37140]: overall HEALTH_OK 2024-07-26T16:20:02.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:20:02 smithi138 ceph-mon[36954]: pgmap v5466: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:20:02.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:20:02 smithi100 ceph-mon[32599]: pgmap v5466: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:20:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:20:02 smithi167 ceph-mon[37140]: pgmap v5466: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:20:04.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:20:04 smithi138 ceph-mon[36954]: pgmap v5467: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:20:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:20:04 smithi100 ceph-mon[32599]: pgmap v5467: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:20:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:20:04 smithi167 ceph-mon[37140]: pgmap v5467: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:20:06.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:20:06.486+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:20:06.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:20:06.487+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:20:06.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:20:06.487+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:20:06.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:20:06.487+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:20:06.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:20:06.487+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:20:06.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:20:06 smithi100 ceph-mon[32599]: pgmap v5468: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:20:06.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:20:06 smithi138 ceph-mon[36954]: pgmap v5468: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:20:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:20:06 smithi167 ceph-mon[37140]: pgmap v5468: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:20:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:20:08 smithi100 ceph-mon[32599]: pgmap v5469: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:20:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:20:08 smithi100 ceph-mon[32599]: osdmap e388: 12 total, 12 up, 12 in 2024-07-26T16:20:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:20:08 smithi167 ceph-mon[37140]: pgmap v5469: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:20:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:20:08 smithi167 ceph-mon[37140]: osdmap e388: 12 total, 12 up, 12 in 2024-07-26T16:20:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:20:08 smithi138 ceph-mon[36954]: pgmap v5469: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:20:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:20:08 smithi138 ceph-mon[36954]: osdmap e388: 12 total, 12 up, 12 in 2024-07-26T16:20:10.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:20:10 smithi100 ceph-mon[32599]: pgmap v5471: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:20:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:20:10 smithi167 ceph-mon[37140]: pgmap v5471: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:20:11.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:20:10 smithi138 ceph-mon[36954]: pgmap v5471: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:20:12.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:20:12 smithi100 ceph-mon[32599]: pgmap v5472: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:20:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:20:12 smithi167 ceph-mon[37140]: pgmap v5472: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:20:13.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:20:12 smithi138 ceph-mon[36954]: pgmap v5472: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:20:14.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:20:14 smithi100 ceph-mon[32599]: pgmap v5473: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:20:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:20:14 smithi167 ceph-mon[37140]: pgmap v5473: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:20:15.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:20:14 smithi138 ceph-mon[36954]: pgmap v5473: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:20:16.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:20:16 smithi100 ceph-mon[32599]: pgmap v5474: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:20:16.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:20:16 smithi100 ceph-mon[32599]: osdmap e389: 12 total, 12 up, 12 in 2024-07-26T16:20:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:20:16 smithi167 ceph-mon[37140]: pgmap v5474: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:20:17.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:20:16 smithi167 ceph-mon[37140]: osdmap e389: 12 total, 12 up, 12 in 2024-07-26T16:20:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:20:16 smithi138 ceph-mon[36954]: pgmap v5474: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:20:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:20:16 smithi138 ceph-mon[36954]: osdmap e389: 12 total, 12 up, 12 in 2024-07-26T16:20:18.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:20:18 smithi100 ceph-mon[32599]: pgmap v5476: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:20:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:20:18 smithi167 ceph-mon[37140]: pgmap v5476: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:20:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:20:18 smithi138 ceph-mon[36954]: pgmap v5476: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:20:20.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:20:20 smithi100 ceph-mon[32599]: pgmap v5477: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:20:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:20:20 smithi167 ceph-mon[37140]: pgmap v5477: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:20:21.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:20:20 smithi138 ceph-mon[36954]: pgmap v5477: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:20:21.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:20:21.494+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:20:21.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:20:21.495+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:20:21.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:20:21.495+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:20:21.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:20:21.495+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:20:21.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:20:21.495+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:20:22.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:20:22 smithi100 ceph-mon[32599]: pgmap v5478: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:20:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:20:22 smithi167 ceph-mon[37140]: pgmap v5478: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:20:23.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:20:22 smithi138 ceph-mon[36954]: pgmap v5478: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:20:24.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:20:24 smithi100 ceph-mon[32599]: pgmap v5479: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:20:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:20:24 smithi167 ceph-mon[37140]: pgmap v5479: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:20:25.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:20:24 smithi138 ceph-mon[36954]: pgmap v5479: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:20:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:20:26 smithi100 ceph-mon[32599]: pgmap v5480: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:20:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:20:26 smithi167 ceph-mon[37140]: pgmap v5480: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:20:27.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:20:26 smithi138 ceph-mon[36954]: pgmap v5480: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:20:28.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:20:28 smithi100 ceph-mon[32599]: pgmap v5481: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:20:28.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:20:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:20:28.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:20:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:20:28.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:20:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:20:28.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:20:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:20:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:20:28 smithi167 ceph-mon[37140]: pgmap v5481: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:20:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:20:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:20:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:20:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:20:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:20:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:20:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:20:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:20:29.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:20:28 smithi138 ceph-mon[36954]: pgmap v5481: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:20:29.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:20:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:20:29.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:20:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:20:29.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:20:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:20:29.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:20:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:20:30.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:20:30 smithi100 ceph-mon[32599]: pgmap v5482: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:20:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:20:30 smithi167 ceph-mon[37140]: pgmap v5482: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:20:31.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:20:30 smithi138 ceph-mon[36954]: pgmap v5482: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:20:32.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:20:32 smithi100 ceph-mon[32599]: pgmap v5483: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:20:32.526 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:20:32 smithi167 ceph-mon[37140]: pgmap v5483: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:20:32.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:20:32 smithi138 ceph-mon[36954]: pgmap v5483: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:20:34.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:20:34 smithi138 ceph-mon[36954]: pgmap v5484: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:20:34.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:20:34 smithi100 ceph-mon[32599]: pgmap v5484: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:20:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:20:34 smithi167 ceph-mon[37140]: pgmap v5484: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:20:36.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:20:36.511+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:20:36.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:20:36.512+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:20:36.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:20:36.512+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:20:36.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:20:36.512+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:20:36.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:20:36.512+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:20:36.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:20:36 smithi138 ceph-mon[36954]: pgmap v5485: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:20:36.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:20:36 smithi100 ceph-mon[32599]: pgmap v5485: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:20:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:20:36 smithi167 ceph-mon[37140]: pgmap v5485: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:20:38.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:20:38 smithi138 ceph-mon[36954]: pgmap v5486: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:20:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:20:38 smithi100 ceph-mon[32599]: pgmap v5486: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:20:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:20:38 smithi167 ceph-mon[37140]: pgmap v5486: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:20:40.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:20:40 smithi138 ceph-mon[36954]: pgmap v5487: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:20:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:20:40 smithi100 ceph-mon[32599]: pgmap v5487: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:20:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:20:40 smithi167 ceph-mon[37140]: pgmap v5487: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:20:42.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:20:42 smithi138 ceph-mon[36954]: pgmap v5488: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:20:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:20:42 smithi100 ceph-mon[32599]: pgmap v5488: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:20:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:20:42 smithi167 ceph-mon[37140]: pgmap v5488: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:20:44.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:20:44 smithi138 ceph-mon[36954]: pgmap v5489: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:20:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:20:44 smithi100 ceph-mon[32599]: pgmap v5489: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:20:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:20:44 smithi167 ceph-mon[37140]: pgmap v5489: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:20:46.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:20:46 smithi138 ceph-mon[36954]: pgmap v5490: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:20:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:20:46 smithi100 ceph-mon[32599]: pgmap v5490: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:20:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:20:46 smithi167 ceph-mon[37140]: pgmap v5490: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:20:48.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:20:48 smithi138 ceph-mon[36954]: pgmap v5491: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:20:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:20:48 smithi100 ceph-mon[32599]: pgmap v5491: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:20:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:20:48 smithi167 ceph-mon[37140]: pgmap v5491: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:20:50.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:20:50 smithi100 ceph-mon[32599]: pgmap v5492: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:20:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:20:50 smithi167 ceph-mon[37140]: pgmap v5492: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:20:51.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:20:50 smithi138 ceph-mon[36954]: pgmap v5492: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:20:51.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:20:51.528+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:20:51.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:20:51.528+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:20:51.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:20:51.528+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:20:51.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:20:51.528+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:20:51.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:20:51.528+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:20:52.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:20:52 smithi100 ceph-mon[32599]: pgmap v5493: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:20:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:20:52 smithi167 ceph-mon[37140]: pgmap v5493: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:20:53.194 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:20:52 smithi138 ceph-mon[36954]: pgmap v5493: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:20:54.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:20:54 smithi100 ceph-mon[32599]: pgmap v5494: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:20:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:20:54 smithi167 ceph-mon[37140]: pgmap v5494: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:20:55.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:20:54 smithi138 ceph-mon[36954]: pgmap v5494: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:20:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:20:56 smithi100 ceph-mon[32599]: pgmap v5495: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:20:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:20:56 smithi167 ceph-mon[37140]: pgmap v5495: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:20:57.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:20:56 smithi138 ceph-mon[36954]: pgmap v5495: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:20:58.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:20:58 smithi100 ceph-mon[32599]: pgmap v5496: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:20:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:20:58 smithi167 ceph-mon[37140]: pgmap v5496: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:20:59.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:20:58 smithi138 ceph-mon[36954]: pgmap v5496: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:21:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:00 smithi100 ceph-mon[32599]: pgmap v5497: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:21:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:00 smithi167 ceph-mon[37140]: pgmap v5497: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:21:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:00 smithi138 ceph-mon[36954]: pgmap v5497: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:21:02.993 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:02 smithi138 ceph-mon[36954]: pgmap v5498: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:21:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:02 smithi100 ceph-mon[32599]: pgmap v5498: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:21:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:02 smithi167 ceph-mon[37140]: pgmap v5498: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:21:04.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:21:04.314+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T16:21:04.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:21:04.314+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T16:21:04.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:21:04.315+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 74 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c020 con 0x7fe5a81087a0 2024-07-26T16:21:04.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:21:04.315+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 74 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T16:21:04.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:04 smithi100 ceph-mon[32599]: pgmap v5499: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T16:21:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:04 smithi167 ceph-mon[37140]: pgmap v5499: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T16:21:05.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:04 smithi138 ceph-mon[36954]: pgmap v5499: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T16:21:06.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:21:06.544+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:21:06.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:21:06.544+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:21:06.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:21:06.544+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:21:06.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:21:06.544+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:21:06.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:21:06.544+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:21:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:06 smithi100 ceph-mon[32599]: pgmap v5500: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:21:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:06 smithi167 ceph-mon[37140]: pgmap v5500: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:21:07.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:06 smithi138 ceph-mon[36954]: pgmap v5500: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:21:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:08 smithi100 ceph-mon[32599]: pgmap v5501: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:21:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:08 smithi100 ceph-mon[32599]: osdmap e390: 12 total, 12 up, 12 in 2024-07-26T16:21:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:08 smithi167 ceph-mon[37140]: pgmap v5501: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:21:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:08 smithi167 ceph-mon[37140]: osdmap e390: 12 total, 12 up, 12 in 2024-07-26T16:21:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:08 smithi138 ceph-mon[36954]: pgmap v5501: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:21:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:08 smithi138 ceph-mon[36954]: osdmap e390: 12 total, 12 up, 12 in 2024-07-26T16:21:10.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:10 smithi100 ceph-mon[32599]: pgmap v5503: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:21:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:10 smithi167 ceph-mon[37140]: pgmap v5503: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:21:11.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:10 smithi138 ceph-mon[36954]: pgmap v5503: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:21:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:12 smithi100 ceph-mon[32599]: pgmap v5504: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:21:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:12 smithi167 ceph-mon[37140]: pgmap v5504: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:21:13.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:12 smithi138 ceph-mon[36954]: pgmap v5504: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:21:14.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:14 smithi100 ceph-mon[32599]: pgmap v5505: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T16:21:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:14 smithi167 ceph-mon[37140]: pgmap v5505: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T16:21:15.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:14 smithi138 ceph-mon[36954]: pgmap v5505: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T16:21:16.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:16 smithi100 ceph-mon[32599]: pgmap v5506: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:21:16.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:16 smithi100 ceph-mon[32599]: osdmap e391: 12 total, 12 up, 12 in 2024-07-26T16:21:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:16 smithi167 ceph-mon[37140]: pgmap v5506: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:21:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:16 smithi167 ceph-mon[37140]: osdmap e391: 12 total, 12 up, 12 in 2024-07-26T16:21:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:16 smithi138 ceph-mon[36954]: pgmap v5506: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:21:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:16 smithi138 ceph-mon[36954]: osdmap e391: 12 total, 12 up, 12 in 2024-07-26T16:21:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:18 smithi100 ceph-mon[32599]: pgmap v5508: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:21:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:18 smithi167 ceph-mon[37140]: pgmap v5508: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:21:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:18 smithi138 ceph-mon[36954]: pgmap v5508: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:21:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:20 smithi167 ceph-mon[37140]: pgmap v5509: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:21:21.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:20 smithi138 ceph-mon[36954]: pgmap v5509: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:21:21.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:20 smithi100 ceph-mon[32599]: pgmap v5509: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:21:21.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:21:21.559+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:21:21.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:21:21.559+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:21:21.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:21:21.559+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:21:21.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:21:21.559+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:21:21.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:21:21.559+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:21:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:22 smithi167 ceph-mon[37140]: pgmap v5510: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:21:23.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:22 smithi138 ceph-mon[36954]: pgmap v5510: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:21:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:22 smithi100 ceph-mon[32599]: pgmap v5510: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:21:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:24 smithi167 ceph-mon[37140]: pgmap v5511: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:21:25.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:24 smithi138 ceph-mon[36954]: pgmap v5511: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:21:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:24 smithi100 ceph-mon[32599]: pgmap v5511: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:21:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:26 smithi167 ceph-mon[37140]: pgmap v5512: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:21:27.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:26 smithi138 ceph-mon[36954]: pgmap v5512: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:21:27.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:26 smithi100 ceph-mon[32599]: pgmap v5512: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:21:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:28 smithi167 ceph-mon[37140]: pgmap v5513: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:21:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:21:29.194 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:28 smithi138 ceph-mon[36954]: pgmap v5513: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:21:29.195 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:21:29.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:28 smithi100 ceph-mon[32599]: pgmap v5513: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:21:29.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:21:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:30 smithi167 ceph-mon[37140]: pgmap v5514: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:21:31.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:21:31.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:21:31.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:21:31.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:21:31.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:21:31.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:21:31.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:21:31.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:21:31.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:21:31.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:30 smithi138 ceph-mon[36954]: pgmap v5514: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:21:31.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:21:31.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:21:31.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:21:31.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:21:31.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:21:31.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:21:31.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:21:31.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:21:31.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:21:31.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:30 smithi100 ceph-mon[32599]: pgmap v5514: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:21:31.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:21:31.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:21:31.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:21:31.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:21:31.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:21:31.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:21:31.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:21:31.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:21:31.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:21:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:32 smithi167 ceph-mon[37140]: pgmap v5515: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:21:33.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:32 smithi138 ceph-mon[36954]: pgmap v5515: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:21:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:32 smithi100 ceph-mon[32599]: pgmap v5515: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:21:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:34 smithi167 ceph-mon[37140]: pgmap v5516: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:21:35.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:34 smithi138 ceph-mon[36954]: pgmap v5516: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:21:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:34 smithi100 ceph-mon[32599]: pgmap v5516: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:21:36.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:21:36.574+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:21:36.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:21:36.575+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:21:36.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:21:36.575+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:21:36.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:21:36.575+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:21:36.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:21:36.575+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:21:36.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:36 smithi100 ceph-mon[32599]: pgmap v5517: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:21:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:36 smithi167 ceph-mon[37140]: pgmap v5517: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:21:37.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:36 smithi138 ceph-mon[36954]: pgmap v5517: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:21:39.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:38 smithi138 ceph-mon[36954]: pgmap v5518: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:21:39.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:38 smithi100 ceph-mon[32599]: pgmap v5518: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:21:39.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:38 smithi167 ceph-mon[37140]: pgmap v5518: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:21:41.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:40 smithi138 ceph-mon[36954]: pgmap v5519: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:21:41.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:40 smithi100 ceph-mon[32599]: pgmap v5519: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:21:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:40 smithi167 ceph-mon[37140]: pgmap v5519: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:21:43.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:42 smithi138 ceph-mon[36954]: pgmap v5520: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:21:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:42 smithi100 ceph-mon[32599]: pgmap v5520: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:21:43.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:42 smithi167 ceph-mon[37140]: pgmap v5520: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:21:45.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:44 smithi138 ceph-mon[36954]: pgmap v5521: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:21:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:44 smithi100 ceph-mon[32599]: pgmap v5521: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:21:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:44 smithi167 ceph-mon[37140]: pgmap v5521: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:21:47.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:46 smithi167 ceph-mon[37140]: pgmap v5522: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:21:47.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:46 smithi138 ceph-mon[36954]: pgmap v5522: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:21:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:46 smithi100 ceph-mon[32599]: pgmap v5522: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:21:49.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:48 smithi138 ceph-mon[36954]: pgmap v5523: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:21:49.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:48 smithi100 ceph-mon[32599]: pgmap v5523: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:21:49.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:48 smithi167 ceph-mon[37140]: pgmap v5523: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:21:51.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:50 smithi100 ceph-mon[32599]: pgmap v5524: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:21:51.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:50 smithi138 ceph-mon[36954]: pgmap v5524: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:21:51.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:50 smithi167 ceph-mon[37140]: pgmap v5524: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:21:51.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:21:51.591+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:21:51.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:21:51.592+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:21:51.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:21:51.592+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:21:51.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:21:51.592+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:21:51.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:21:51.592+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:21:53.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:52 smithi138 ceph-mon[36954]: pgmap v5525: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:21:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:52 smithi100 ceph-mon[32599]: pgmap v5525: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:21:53.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:52 smithi167 ceph-mon[37140]: pgmap v5525: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:21:55.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:54 smithi138 ceph-mon[36954]: pgmap v5526: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:21:55.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:54 smithi100 ceph-mon[32599]: pgmap v5526: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:21:55.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:54 smithi167 ceph-mon[37140]: pgmap v5526: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:21:57.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:56 smithi138 ceph-mon[36954]: pgmap v5527: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:21:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:56 smithi100 ceph-mon[32599]: pgmap v5527: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:21:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:56 smithi167 ceph-mon[37140]: pgmap v5527: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:21:59.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:21:58 smithi138 ceph-mon[36954]: pgmap v5528: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:21:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:21:58 smithi100 ceph-mon[32599]: pgmap v5528: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:21:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:21:58 smithi167 ceph-mon[37140]: pgmap v5528: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:22:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:00 smithi138 ceph-mon[36954]: pgmap v5529: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:22:01.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:00 smithi100 ceph-mon[32599]: pgmap v5529: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:22:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:00 smithi167 ceph-mon[37140]: pgmap v5529: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:22:03.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:02 smithi138 ceph-mon[36954]: pgmap v5530: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:22:03.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:02 smithi100 ceph-mon[32599]: pgmap v5530: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:22:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:02 smithi167 ceph-mon[37140]: pgmap v5530: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:22:05.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:04 smithi138 ceph-mon[36954]: pgmap v5531: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T16:22:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:04 smithi100 ceph-mon[32599]: pgmap v5531: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T16:22:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:04 smithi167 ceph-mon[37140]: pgmap v5531: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T16:22:06.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:22:06.608+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:22:06.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:22:06.608+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:22:06.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:22:06.608+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:22:06.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:22:06.608+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:22:06.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:22:06.608+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:22:06.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:06 smithi100 ceph-mon[32599]: pgmap v5532: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:22:07.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:06 smithi138 ceph-mon[36954]: pgmap v5532: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:22:07.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:06 smithi167 ceph-mon[37140]: pgmap v5532: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:22:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:08 smithi138 ceph-mon[36954]: pgmap v5533: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:22:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:08 smithi138 ceph-mon[36954]: osdmap e392: 12 total, 12 up, 12 in 2024-07-26T16:22:09.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:08 smithi100 ceph-mon[32599]: pgmap v5533: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:22:09.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:08 smithi100 ceph-mon[32599]: osdmap e392: 12 total, 12 up, 12 in 2024-07-26T16:22:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:08 smithi167 ceph-mon[37140]: pgmap v5533: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:22:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:08 smithi167 ceph-mon[37140]: osdmap e392: 12 total, 12 up, 12 in 2024-07-26T16:22:11.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:10 smithi138 ceph-mon[36954]: pgmap v5535: 97 pgs: 8 active+clean+snaptrim_wait, 89 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:22:11.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:10 smithi100 ceph-mon[32599]: pgmap v5535: 97 pgs: 8 active+clean+snaptrim_wait, 89 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:22:11.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:10 smithi167 ceph-mon[37140]: pgmap v5535: 97 pgs: 8 active+clean+snaptrim_wait, 89 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:22:13.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:12 smithi138 ceph-mon[36954]: pgmap v5536: 97 pgs: 8 active+clean+snaptrim_wait, 89 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:22:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:12 smithi100 ceph-mon[32599]: pgmap v5536: 97 pgs: 8 active+clean+snaptrim_wait, 89 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:22:13.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:12 smithi167 ceph-mon[37140]: pgmap v5536: 97 pgs: 8 active+clean+snaptrim_wait, 89 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:22:15.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:14 smithi138 ceph-mon[36954]: pgmap v5537: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T16:22:15.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:14 smithi100 ceph-mon[32599]: pgmap v5537: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T16:22:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:14 smithi167 ceph-mon[37140]: pgmap v5537: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T16:22:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:16 smithi138 ceph-mon[36954]: pgmap v5538: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:22:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:16 smithi138 ceph-mon[36954]: osdmap e393: 12 total, 12 up, 12 in 2024-07-26T16:22:17.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:16 smithi100 ceph-mon[32599]: pgmap v5538: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:22:17.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:16 smithi100 ceph-mon[32599]: osdmap e393: 12 total, 12 up, 12 in 2024-07-26T16:22:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:16 smithi167 ceph-mon[37140]: pgmap v5538: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:22:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:16 smithi167 ceph-mon[37140]: osdmap e393: 12 total, 12 up, 12 in 2024-07-26T16:22:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:18 smithi138 ceph-mon[36954]: pgmap v5540: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 208 B/s wr, 6 op/s 2024-07-26T16:22:19.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:18 smithi100 ceph-mon[32599]: pgmap v5540: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 208 B/s wr, 6 op/s 2024-07-26T16:22:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:18 smithi167 ceph-mon[37140]: pgmap v5540: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 208 B/s wr, 6 op/s 2024-07-26T16:22:21.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:20 smithi138 ceph-mon[36954]: pgmap v5541: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:22:21.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:20 smithi100 ceph-mon[32599]: pgmap v5541: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:22:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:20 smithi167 ceph-mon[37140]: pgmap v5541: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:22:21.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:22:21.624+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:22:21.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:22:21.624+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:22:21.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:22:21.624+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:22:21.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:22:21.624+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:22:21.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:22:21.624+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:22:23.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:22 smithi138 ceph-mon[36954]: pgmap v5542: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:22:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:22 smithi100 ceph-mon[32599]: pgmap v5542: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:22:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:22 smithi167 ceph-mon[37140]: pgmap v5542: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:22:25.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:24 smithi138 ceph-mon[36954]: pgmap v5543: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:22:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:24 smithi100 ceph-mon[32599]: pgmap v5543: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:22:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:24 smithi167 ceph-mon[37140]: pgmap v5543: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:22:27.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:26 smithi138 ceph-mon[36954]: pgmap v5544: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:22:27.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:26 smithi100 ceph-mon[32599]: pgmap v5544: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:22:27.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:26 smithi167 ceph-mon[37140]: pgmap v5544: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:22:29.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:28 smithi138 ceph-mon[36954]: pgmap v5545: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:22:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:28 smithi100 ceph-mon[32599]: pgmap v5545: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:22:29.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:28 smithi167 ceph-mon[37140]: pgmap v5545: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:22:31.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:30 smithi138 ceph-mon[36954]: pgmap v5546: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:22:31.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:22:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:30 smithi100 ceph-mon[32599]: pgmap v5546: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:22:31.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:22:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:30 smithi167 ceph-mon[37140]: pgmap v5546: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:22:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:22:32.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:22:32.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:22:32.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:22:32.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:22:32.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:22:32.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:22:32.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:22:32.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:22:32.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:22:32.354 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:22:32.354 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:22:32.354 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:22:32.355 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:22:32.355 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:22:32.355 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:22:32.355 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:22:32.355 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:22:32.355 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:22:32.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:22:32.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:22:32.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:22:32.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:22:32.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:22:32.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:22:32.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:22:32.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:22:32.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:22:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:32 smithi167 ceph-mon[37140]: pgmap v5547: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:22:33.277 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:32 smithi138 ceph-mon[36954]: pgmap v5547: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:22:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:32 smithi100 ceph-mon[32599]: pgmap v5547: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:22:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:35 smithi167 ceph-mon[37140]: pgmap v5548: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:22:35.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:35 smithi138 ceph-mon[36954]: pgmap v5548: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:22:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:35 smithi100 ceph-mon[32599]: pgmap v5548: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:22:36.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:22:36.640+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:22:36.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:22:36.640+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:22:36.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:22:36.640+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:22:36.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:22:36.640+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:22:36.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:22:36.640+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:22:37.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:37 smithi167 ceph-mon[37140]: pgmap v5549: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:22:37.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:37 smithi138 ceph-mon[36954]: pgmap v5549: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:22:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:37 smithi100 ceph-mon[32599]: pgmap v5549: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:22:39.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:39 smithi167 ceph-mon[37140]: pgmap v5550: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:22:39.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:39 smithi138 ceph-mon[36954]: pgmap v5550: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:22:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:39 smithi100 ceph-mon[32599]: pgmap v5550: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:22:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:41 smithi167 ceph-mon[37140]: pgmap v5551: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:22:41.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:41 smithi138 ceph-mon[36954]: pgmap v5551: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:22:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:41 smithi100 ceph-mon[32599]: pgmap v5551: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:22:43.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:43 smithi138 ceph-mon[36954]: pgmap v5552: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:22:43.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:43 smithi167 ceph-mon[37140]: pgmap v5552: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:22:43.498 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:43 smithi100 ceph-mon[32599]: pgmap v5552: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:22:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:45 smithi167 ceph-mon[37140]: pgmap v5553: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:22:45.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:45 smithi138 ceph-mon[36954]: pgmap v5553: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:22:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:45 smithi100 ceph-mon[32599]: pgmap v5553: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:22:47.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:47 smithi167 ceph-mon[37140]: pgmap v5554: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:22:47.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:47 smithi138 ceph-mon[36954]: pgmap v5554: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:22:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:47 smithi100 ceph-mon[32599]: pgmap v5554: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:22:49.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:49 smithi138 ceph-mon[36954]: pgmap v5555: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:22:49.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:49 smithi100 ceph-mon[32599]: pgmap v5555: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:22:49.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:49 smithi167 ceph-mon[37140]: pgmap v5555: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:22:51.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:51 smithi100 ceph-mon[32599]: pgmap v5556: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:22:51.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:51 smithi138 ceph-mon[36954]: pgmap v5556: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:22:51.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:51 smithi167 ceph-mon[37140]: pgmap v5556: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:22:51.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:22:51.656+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:22:51.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:22:51.656+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:22:51.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:22:51.656+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:22:51.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:22:51.656+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:22:51.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:22:51.657+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:22:53.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:53 smithi138 ceph-mon[36954]: pgmap v5557: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:22:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:53 smithi100 ceph-mon[32599]: pgmap v5557: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:22:53.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:53 smithi167 ceph-mon[37140]: pgmap v5557: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:22:55.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:55 smithi138 ceph-mon[36954]: pgmap v5558: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:22:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:55 smithi100 ceph-mon[32599]: pgmap v5558: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:22:55.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:55 smithi167 ceph-mon[37140]: pgmap v5558: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:22:57.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:57 smithi138 ceph-mon[36954]: pgmap v5559: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:22:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:57 smithi100 ceph-mon[32599]: pgmap v5559: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:22:57.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:57 smithi167 ceph-mon[37140]: pgmap v5559: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:22:59.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:22:59 smithi138 ceph-mon[36954]: pgmap v5560: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:22:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:22:59 smithi100 ceph-mon[32599]: pgmap v5560: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:22:59.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:22:59 smithi167 ceph-mon[37140]: pgmap v5560: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:23:01.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:23:01 smithi138 ceph-mon[36954]: pgmap v5561: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:23:01.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:23:01 smithi100 ceph-mon[32599]: pgmap v5561: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:23:01.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:23:01 smithi167 ceph-mon[37140]: pgmap v5561: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:23:03.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:23:03 smithi138 ceph-mon[36954]: pgmap v5562: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:23:03.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:23:03 smithi100 ceph-mon[32599]: pgmap v5562: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:23:03.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:23:03 smithi167 ceph-mon[37140]: pgmap v5562: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:23:05.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:23:05 smithi138 ceph-mon[36954]: pgmap v5563: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T16:23:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:23:05 smithi100 ceph-mon[32599]: pgmap v5563: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T16:23:05.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:23:05 smithi167 ceph-mon[37140]: pgmap v5563: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T16:23:06.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:23:06.672+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:23:06.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:23:06.673+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:23:06.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:23:06.673+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:23:06.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:23:06.673+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:23:06.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:23:06.673+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:23:07.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:23:07 smithi138 ceph-mon[36954]: pgmap v5564: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:23:07.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:23:07 smithi100 ceph-mon[32599]: pgmap v5564: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:23:07.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:23:07 smithi167 ceph-mon[37140]: pgmap v5564: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:23:08.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:23:08 smithi138 ceph-mon[36954]: osdmap e394: 12 total, 12 up, 12 in 2024-07-26T16:23:08.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:23:08 smithi100 ceph-mon[32599]: osdmap e394: 12 total, 12 up, 12 in 2024-07-26T16:23:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:23:08 smithi167 ceph-mon[37140]: osdmap e394: 12 total, 12 up, 12 in 2024-07-26T16:23:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:23:09 smithi138 ceph-mon[36954]: pgmap v5566: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:23:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:23:09 smithi100 ceph-mon[32599]: pgmap v5566: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:23:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:23:09 smithi167 ceph-mon[37140]: pgmap v5566: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:23:11.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:23:11 smithi138 ceph-mon[36954]: pgmap v5567: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:23:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:23:11 smithi100 ceph-mon[32599]: pgmap v5567: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:23:11.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:23:11 smithi167 ceph-mon[37140]: pgmap v5567: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:23:13.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:23:13 smithi138 ceph-mon[36954]: pgmap v5568: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:23:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:23:13 smithi100 ceph-mon[32599]: pgmap v5568: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:23:13.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:23:13 smithi167 ceph-mon[37140]: pgmap v5568: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:23:15.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:23:15 smithi138 ceph-mon[36954]: pgmap v5569: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T16:23:15.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:23:15 smithi100 ceph-mon[32599]: pgmap v5569: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T16:23:15.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:23:15 smithi167 ceph-mon[37140]: pgmap v5569: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T16:23:17.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:23:17 smithi138 ceph-mon[36954]: pgmap v5570: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:23:17.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:23:17 smithi138 ceph-mon[36954]: osdmap e395: 12 total, 12 up, 12 in 2024-07-26T16:23:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:23:17 smithi100 ceph-mon[32599]: pgmap v5570: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:23:17.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:23:17 smithi100 ceph-mon[32599]: osdmap e395: 12 total, 12 up, 12 in 2024-07-26T16:23:17.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:23:17 smithi167 ceph-mon[37140]: pgmap v5570: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:23:17.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:23:17 smithi167 ceph-mon[37140]: osdmap e395: 12 total, 12 up, 12 in 2024-07-26T16:23:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:23:19 smithi138 ceph-mon[36954]: pgmap v5572: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:23:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:23:19 smithi100 ceph-mon[32599]: pgmap v5572: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:23:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:23:19 smithi167 ceph-mon[37140]: pgmap v5572: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:23:21.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:23:21 smithi138 ceph-mon[36954]: pgmap v5573: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:23:21.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:23:21 smithi100 ceph-mon[32599]: pgmap v5573: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:23:21.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:23:21 smithi167 ceph-mon[37140]: pgmap v5573: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:23:21.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:23:21.688+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:23:21.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:23:21.689+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:23:21.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:23:21.689+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:23:21.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:23:21.689+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:23:21.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:23:21.689+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:23:23.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:23:23 smithi138 ceph-mon[36954]: pgmap v5574: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:23:23.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:23:23 smithi100 ceph-mon[32599]: pgmap v5574: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:23:23.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:23:23 smithi167 ceph-mon[37140]: pgmap v5574: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:23:25.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:23:25 smithi138 ceph-mon[36954]: pgmap v5575: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:23:25.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:23:25 smithi100 ceph-mon[32599]: pgmap v5575: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:23:25.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:23:25 smithi167 ceph-mon[37140]: pgmap v5575: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:23:26.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:23:26 smithi138 ceph-mon[36954]: pgmap v5576: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:23:26.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:23:26 smithi100 ceph-mon[32599]: pgmap v5576: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:23:26.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:23:26 smithi167 ceph-mon[37140]: pgmap v5576: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:23:28.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:23:28 smithi100 ceph-mon[32599]: pgmap v5577: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:23:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:23:28 smithi167 ceph-mon[37140]: pgmap v5577: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:23:29.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:23:28 smithi138 ceph-mon[36954]: pgmap v5577: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:23:30.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:23:30 smithi100 ceph-mon[32599]: pgmap v5578: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:23:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:23:30 smithi167 ceph-mon[37140]: pgmap v5578: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:23:31.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:23:30 smithi138 ceph-mon[36954]: pgmap v5578: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:23:31.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:23:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:23:32.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:23:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:23:32.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:23:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:23:32.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:23:32 smithi100 ceph-mon[32599]: pgmap v5579: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:23:32.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:23:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:23:32.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:23:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:23:32.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:23:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:23:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:23:32 smithi167 ceph-mon[37140]: pgmap v5579: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:23:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:23:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:23:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:23:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:23:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:23:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:23:33.092 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:23:32 smithi138 ceph-mon[36954]: pgmap v5579: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:23:33.092 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:23:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:23:33.092 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:23:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:23:33.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:23:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:23:34.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:23:34 smithi100 ceph-mon[32599]: pgmap v5580: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T16:23:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:23:34 smithi167 ceph-mon[37140]: pgmap v5580: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T16:23:35.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:23:34 smithi138 ceph-mon[36954]: pgmap v5580: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T16:23:36.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:23:36.699+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:23:36.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:23:36.699+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:23:36.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:23:36.699+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:23:36.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:23:36.699+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:23:36.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:23:36.699+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:23:36.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:23:36 smithi100 ceph-mon[32599]: pgmap v5581: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:23:37.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:23:36 smithi167 ceph-mon[37140]: pgmap v5581: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:23:37.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:23:36 smithi138 ceph-mon[36954]: pgmap v5581: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:23:38.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:23:38 smithi100 ceph-mon[32599]: pgmap v5582: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:23:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:23:38 smithi167 ceph-mon[37140]: pgmap v5582: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:23:39.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:23:38 smithi138 ceph-mon[36954]: pgmap v5582: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:23:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:23:40 smithi167 ceph-mon[37140]: pgmap v5583: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:23:41.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:23:40 smithi138 ceph-mon[36954]: pgmap v5583: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:23:41.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:23:40 smithi100 ceph-mon[32599]: pgmap v5583: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:23:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:23:42 smithi167 ceph-mon[37140]: pgmap v5584: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:23:43.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:23:42 smithi138 ceph-mon[36954]: pgmap v5584: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:23:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:23:42 smithi100 ceph-mon[32599]: pgmap v5584: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:23:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:23:44 smithi167 ceph-mon[37140]: pgmap v5585: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:23:45.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:23:44 smithi138 ceph-mon[36954]: pgmap v5585: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:23:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:23:44 smithi100 ceph-mon[32599]: pgmap v5585: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:23:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:23:46 smithi167 ceph-mon[37140]: pgmap v5586: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:23:47.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:23:46 smithi138 ceph-mon[36954]: pgmap v5586: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:23:47.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:23:46 smithi100 ceph-mon[32599]: pgmap v5586: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:23:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:23:48 smithi167 ceph-mon[37140]: pgmap v5587: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:23:49.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:23:48 smithi138 ceph-mon[36954]: pgmap v5587: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:23:49.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:23:48 smithi100 ceph-mon[32599]: pgmap v5587: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:23:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:23:50 smithi167 ceph-mon[37140]: pgmap v5588: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:23:51.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:23:50 smithi100 ceph-mon[32599]: pgmap v5588: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:23:51.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:23:50 smithi138 ceph-mon[36954]: pgmap v5588: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:23:51.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:23:51.714+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:23:51.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:23:51.715+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:23:51.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:23:51.715+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:23:51.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:23:51.715+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:23:51.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:23:51.715+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:23:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:23:52 smithi167 ceph-mon[37140]: pgmap v5589: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:23:53.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:23:52 smithi138 ceph-mon[36954]: pgmap v5589: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:23:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:23:52 smithi100 ceph-mon[32599]: pgmap v5589: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:23:55.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:23:54 smithi138 ceph-mon[36954]: pgmap v5590: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:23:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:23:54 smithi100 ceph-mon[32599]: pgmap v5590: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:23:55.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:23:54 smithi167 ceph-mon[37140]: pgmap v5590: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:23:57.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:23:56 smithi138 ceph-mon[36954]: pgmap v5591: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:23:57.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:23:56 smithi100 ceph-mon[32599]: pgmap v5591: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:23:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:23:56 smithi167 ceph-mon[37140]: pgmap v5591: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:23:59.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:23:58 smithi138 ceph-mon[36954]: pgmap v5592: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:23:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:23:58 smithi100 ceph-mon[32599]: pgmap v5592: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:23:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:23:58 smithi167 ceph-mon[37140]: pgmap v5592: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:24:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:24:00 smithi138 ceph-mon[36954]: pgmap v5593: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:24:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:24:00 smithi100 ceph-mon[32599]: pgmap v5593: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:24:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:24:00 smithi167 ceph-mon[37140]: pgmap v5593: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:24:03.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:24:02 smithi138 ceph-mon[36954]: pgmap v5594: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:24:03.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:24:02 smithi100 ceph-mon[32599]: pgmap v5594: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:24:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:24:02 smithi167 ceph-mon[37140]: pgmap v5594: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:24:04.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:24:04.340+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T16:24:04.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:24:04.340+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T16:24:04.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:24:04.340+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 75 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c020 con 0x7fe5a81087a0 2024-07-26T16:24:04.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:24:04.340+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 75 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T16:24:05.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:24:04 smithi138 ceph-mon[36954]: pgmap v5595: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T16:24:05.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:24:04 smithi100 ceph-mon[32599]: pgmap v5595: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T16:24:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:24:04 smithi167 ceph-mon[37140]: pgmap v5595: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T16:24:06.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:24:06.731+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:24:06.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:24:06.731+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:24:06.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:24:06.731+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:24:06.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:24:06.731+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:24:06.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:24:06.731+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:24:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:24:06 smithi100 ceph-mon[32599]: pgmap v5596: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:24:07.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:24:06 smithi138 ceph-mon[36954]: pgmap v5596: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:24:07.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:24:06 smithi167 ceph-mon[37140]: pgmap v5596: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:24:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:24:08 smithi138 ceph-mon[36954]: pgmap v5597: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:24:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:24:08 smithi138 ceph-mon[36954]: osdmap e396: 12 total, 12 up, 12 in 2024-07-26T16:24:09.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:24:08 smithi100 ceph-mon[32599]: pgmap v5597: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:24:09.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:24:08 smithi100 ceph-mon[32599]: osdmap e396: 12 total, 12 up, 12 in 2024-07-26T16:24:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:24:08 smithi167 ceph-mon[37140]: pgmap v5597: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:24:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:24:08 smithi167 ceph-mon[37140]: osdmap e396: 12 total, 12 up, 12 in 2024-07-26T16:24:11.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:24:10 smithi138 ceph-mon[36954]: pgmap v5599: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:24:11.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:24:10 smithi100 ceph-mon[32599]: pgmap v5599: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:24:11.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:24:10 smithi167 ceph-mon[37140]: pgmap v5599: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:24:13.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:24:12 smithi138 ceph-mon[36954]: pgmap v5600: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:24:13.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:24:12 smithi100 ceph-mon[32599]: pgmap v5600: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:24:13.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:24:12 smithi167 ceph-mon[37140]: pgmap v5600: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:24:15.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:24:14 smithi138 ceph-mon[36954]: pgmap v5601: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T16:24:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:24:14 smithi100 ceph-mon[32599]: pgmap v5601: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T16:24:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:24:14 smithi167 ceph-mon[37140]: pgmap v5601: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T16:24:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:24:16 smithi138 ceph-mon[36954]: pgmap v5602: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:24:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:24:16 smithi138 ceph-mon[36954]: osdmap e397: 12 total, 12 up, 12 in 2024-07-26T16:24:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:24:16 smithi100 ceph-mon[32599]: pgmap v5602: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:24:17.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:24:16 smithi100 ceph-mon[32599]: osdmap e397: 12 total, 12 up, 12 in 2024-07-26T16:24:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:24:16 smithi167 ceph-mon[37140]: pgmap v5602: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:24:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:24:16 smithi167 ceph-mon[37140]: osdmap e397: 12 total, 12 up, 12 in 2024-07-26T16:24:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:24:18 smithi138 ceph-mon[36954]: pgmap v5604: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T16:24:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:24:18 smithi100 ceph-mon[32599]: pgmap v5604: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T16:24:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:24:18 smithi167 ceph-mon[37140]: pgmap v5604: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T16:24:21.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:24:20 smithi138 ceph-mon[36954]: pgmap v5605: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:24:21.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:24:20 smithi100 ceph-mon[32599]: pgmap v5605: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:24:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:24:20 smithi167 ceph-mon[37140]: pgmap v5605: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:24:21.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:24:21.747+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:24:21.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:24:21.747+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:24:21.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:24:21.747+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:24:21.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:24:21.747+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:24:21.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:24:21.747+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:24:23.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:24:22 smithi138 ceph-mon[36954]: pgmap v5606: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:24:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:24:22 smithi100 ceph-mon[32599]: pgmap v5606: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:24:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:24:22 smithi167 ceph-mon[37140]: pgmap v5606: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:24:25.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:24:24 smithi138 ceph-mon[36954]: pgmap v5607: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:24:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:24:24 smithi100 ceph-mon[32599]: pgmap v5607: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:24:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:24:24 smithi167 ceph-mon[37140]: pgmap v5607: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:24:27.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:24:26 smithi138 ceph-mon[36954]: pgmap v5608: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:24:27.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:24:26 smithi100 ceph-mon[32599]: pgmap v5608: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:24:27.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:24:26 smithi167 ceph-mon[37140]: pgmap v5608: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:24:29.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:24:28 smithi138 ceph-mon[36954]: pgmap v5609: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:24:29.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:24:28 smithi100 ceph-mon[32599]: pgmap v5609: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:24:29.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:24:28 smithi167 ceph-mon[37140]: pgmap v5609: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:24:31.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:24:30 smithi138 ceph-mon[36954]: pgmap v5610: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:24:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:24:30 smithi100 ceph-mon[32599]: pgmap v5610: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:24:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:24:30 smithi167 ceph-mon[37140]: pgmap v5610: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:24:32.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:24:32 smithi138 ceph-mon[36954]: pgmap v5611: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:24:32.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:24:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:24:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:24:32 smithi100 ceph-mon[32599]: pgmap v5611: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:24:32.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:24:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:24:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:24:32 smithi167 ceph-mon[37140]: pgmap v5611: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:24:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:24:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:24:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:24:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:24:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:24:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:24:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:24:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:24:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:24:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:24:33.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:24:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:24:33.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:24:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:24:33.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:24:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:24:33.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:24:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:24:33.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:24:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:24:34.592 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:24:34 smithi138 ceph-mon[36954]: pgmap v5612: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:24:34.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:24:34 smithi100 ceph-mon[32599]: pgmap v5612: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:24:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:24:34 smithi167 ceph-mon[37140]: pgmap v5612: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:24:36.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:24:36.763+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:24:36.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:24:36.763+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:24:36.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:24:36.763+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:24:36.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:24:36.763+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:24:36.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:24:36.763+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:24:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:24:36 smithi167 ceph-mon[37140]: pgmap v5613: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:24:37.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:24:36 smithi138 ceph-mon[36954]: pgmap v5613: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:24:37.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:24:36 smithi100 ceph-mon[32599]: pgmap v5613: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:24:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:24:38 smithi167 ceph-mon[37140]: pgmap v5614: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:24:39.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:24:38 smithi138 ceph-mon[36954]: pgmap v5614: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:24:39.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:24:38 smithi100 ceph-mon[32599]: pgmap v5614: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:24:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:24:40 smithi167 ceph-mon[37140]: pgmap v5615: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:24:41.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:24:40 smithi138 ceph-mon[36954]: pgmap v5615: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:24:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:24:40 smithi100 ceph-mon[32599]: pgmap v5615: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:24:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:24:42 smithi167 ceph-mon[37140]: pgmap v5616: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:24:43.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:24:42 smithi138 ceph-mon[36954]: pgmap v5616: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:24:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:24:42 smithi100 ceph-mon[32599]: pgmap v5616: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:24:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:24:44 smithi167 ceph-mon[37140]: pgmap v5617: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:24:45.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:24:44 smithi138 ceph-mon[36954]: pgmap v5617: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:24:45.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:24:44 smithi100 ceph-mon[32599]: pgmap v5617: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:24:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:24:46 smithi167 ceph-mon[37140]: pgmap v5618: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:24:47.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:24:46 smithi138 ceph-mon[36954]: pgmap v5618: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:24:47.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:24:46 smithi100 ceph-mon[32599]: pgmap v5618: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:24:49.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:24:48 smithi167 ceph-mon[37140]: pgmap v5619: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:24:49.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:24:48 smithi138 ceph-mon[36954]: pgmap v5619: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:24:49.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:24:48 smithi100 ceph-mon[32599]: pgmap v5619: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:24:49.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:24:49.346+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T16:24:49.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:24:49.346+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T16:24:49.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:24:49.346+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 76 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T16:24:49.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:24:49.346+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 76 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe5a008c070 con 0x7fe5a81087a0 2024-07-26T16:24:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:24:50 smithi167 ceph-mon[37140]: pgmap v5620: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:24:51.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:24:50 smithi138 ceph-mon[36954]: pgmap v5620: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:24:51.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:24:50 smithi100 ceph-mon[32599]: pgmap v5620: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:24:51.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:24:51.778+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:24:51.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:24:51.778+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:24:51.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:24:51.778+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:24:51.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:24:51.778+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:24:51.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:24:51.779+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:24:53.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:24:52 smithi138 ceph-mon[36954]: pgmap v5621: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:24:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:24:52 smithi100 ceph-mon[32599]: pgmap v5621: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:24:53.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:24:52 smithi167 ceph-mon[37140]: pgmap v5621: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:24:55.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:24:54 smithi138 ceph-mon[36954]: pgmap v5622: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:24:55.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:24:54 smithi100 ceph-mon[32599]: pgmap v5622: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:24:55.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:24:54 smithi167 ceph-mon[37140]: pgmap v5622: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:24:57.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:24:56 smithi138 ceph-mon[36954]: pgmap v5623: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:24:57.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:24:56 smithi100 ceph-mon[32599]: pgmap v5623: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:24:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:24:56 smithi167 ceph-mon[37140]: pgmap v5623: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:24:59.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:24:58 smithi138 ceph-mon[36954]: pgmap v5624: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:24:59.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:24:58 smithi100 ceph-mon[32599]: pgmap v5624: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:24:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:24:58 smithi167 ceph-mon[37140]: pgmap v5624: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:25:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:25:00 smithi138 ceph-mon[36954]: pgmap v5625: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:25:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:25:00 smithi100 ceph-mon[32599]: pgmap v5625: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:25:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:25:00 smithi167 ceph-mon[37140]: pgmap v5625: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:25:03.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:25:02 smithi138 ceph-mon[36954]: pgmap v5626: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:25:03.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:25:02 smithi100 ceph-mon[32599]: pgmap v5626: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:25:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:25:02 smithi167 ceph-mon[37140]: pgmap v5626: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:25:05.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:25:04 smithi138 ceph-mon[36954]: pgmap v5627: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T16:25:05.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:25:04 smithi100 ceph-mon[32599]: pgmap v5627: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T16:25:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:25:04 smithi167 ceph-mon[37140]: pgmap v5627: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T16:25:06.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:25:06.794+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:25:06.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:25:06.795+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:25:06.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:25:06.795+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:25:06.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:25:06.795+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:25:06.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:25:06.795+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:25:07.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:25:06 smithi138 ceph-mon[36954]: pgmap v5628: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:25:07.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:25:06 smithi100 ceph-mon[32599]: pgmap v5628: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:25:07.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:25:06 smithi167 ceph-mon[37140]: pgmap v5628: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:25:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:25:08 smithi138 ceph-mon[36954]: pgmap v5629: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:25:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:25:08 smithi138 ceph-mon[36954]: osdmap e398: 12 total, 12 up, 12 in 2024-07-26T16:25:09.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:25:08 smithi100 ceph-mon[32599]: pgmap v5629: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:25:09.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:25:08 smithi100 ceph-mon[32599]: osdmap e398: 12 total, 12 up, 12 in 2024-07-26T16:25:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:25:08 smithi167 ceph-mon[37140]: pgmap v5629: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:25:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:25:08 smithi167 ceph-mon[37140]: osdmap e398: 12 total, 12 up, 12 in 2024-07-26T16:25:11.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:25:10 smithi138 ceph-mon[36954]: pgmap v5631: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:25:11.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:25:10 smithi100 ceph-mon[32599]: pgmap v5631: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:25:11.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:25:10 smithi167 ceph-mon[37140]: pgmap v5631: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:25:13.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:25:12 smithi138 ceph-mon[36954]: pgmap v5632: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:25:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:25:12 smithi100 ceph-mon[32599]: pgmap v5632: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:25:13.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:25:12 smithi167 ceph-mon[37140]: pgmap v5632: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:25:15.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:25:14 smithi138 ceph-mon[36954]: pgmap v5633: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T16:25:15.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:25:14 smithi100 ceph-mon[32599]: pgmap v5633: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T16:25:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:25:14 smithi167 ceph-mon[37140]: pgmap v5633: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T16:25:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:25:16 smithi138 ceph-mon[36954]: pgmap v5634: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:25:17.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:25:16 smithi138 ceph-mon[36954]: osdmap e399: 12 total, 12 up, 12 in 2024-07-26T16:25:17.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:25:16 smithi100 ceph-mon[32599]: pgmap v5634: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:25:17.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:25:16 smithi100 ceph-mon[32599]: osdmap e399: 12 total, 12 up, 12 in 2024-07-26T16:25:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:25:16 smithi167 ceph-mon[37140]: pgmap v5634: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:25:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:25:16 smithi167 ceph-mon[37140]: osdmap e399: 12 total, 12 up, 12 in 2024-07-26T16:25:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:25:18 smithi138 ceph-mon[36954]: pgmap v5636: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:25:19.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:25:18 smithi100 ceph-mon[32599]: pgmap v5636: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:25:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:25:18 smithi167 ceph-mon[37140]: pgmap v5636: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:25:21.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:25:20 smithi167 ceph-mon[37140]: pgmap v5637: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:25:21.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:25:20 smithi138 ceph-mon[36954]: pgmap v5637: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:25:21.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:25:20 smithi100 ceph-mon[32599]: pgmap v5637: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:25:21.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:25:21.803+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:25:21.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:25:21.803+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:25:21.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:25:21.803+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:25:21.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:25:21.803+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:25:21.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:25:21.803+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:25:22.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:25:22 smithi138 ceph-mon[36954]: pgmap v5638: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:25:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:25:22 smithi100 ceph-mon[32599]: pgmap v5638: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:25:22.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:25:22 smithi167 ceph-mon[37140]: pgmap v5638: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:25:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:25:24 smithi167 ceph-mon[37140]: pgmap v5639: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:25:25.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:25:24 smithi138 ceph-mon[36954]: pgmap v5639: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:25:25.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:25:24 smithi100 ceph-mon[32599]: pgmap v5639: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:25:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:25:26 smithi167 ceph-mon[37140]: pgmap v5640: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:25:27.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:25:26 smithi138 ceph-mon[36954]: pgmap v5640: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:25:27.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:25:26 smithi100 ceph-mon[32599]: pgmap v5640: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:25:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:25:28 smithi167 ceph-mon[37140]: pgmap v5641: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:25:29.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:25:28 smithi138 ceph-mon[36954]: pgmap v5641: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:25:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:25:28 smithi100 ceph-mon[32599]: pgmap v5641: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:25:31.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:25:30 smithi138 ceph-mon[36954]: pgmap v5642: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:25:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:25:30 smithi100 ceph-mon[32599]: pgmap v5642: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:25:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:25:30 smithi167 ceph-mon[37140]: pgmap v5642: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:25:33.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:25:32 smithi138 ceph-mon[36954]: pgmap v5643: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:25:33.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:25:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:25:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:25:32 smithi100 ceph-mon[32599]: pgmap v5643: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:25:33.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:25:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:25:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:25:32 smithi167 ceph-mon[37140]: pgmap v5643: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:25:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:25:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:25:34.092 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:25:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:25:34.092 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:25:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:25:34.092 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:25:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:25:34.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:25:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:25:34.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:25:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:25:34.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:25:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:25:34.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:25:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:25:34.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:25:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:25:34.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:25:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:25:35.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:25:35 smithi138 ceph-mon[36954]: pgmap v5644: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:25:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:25:35 smithi100 ceph-mon[32599]: pgmap v5644: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:25:35.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:25:35 smithi167 ceph-mon[37140]: pgmap v5644: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:25:36.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:25:36.814+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:25:36.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:25:36.814+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:25:36.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:25:36.814+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:25:36.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:25:36.814+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:25:36.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:25:36.814+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:25:37.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:25:37 smithi138 ceph-mon[36954]: pgmap v5645: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:25:37.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:25:37 smithi100 ceph-mon[32599]: pgmap v5645: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:25:37.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:25:37 smithi167 ceph-mon[37140]: pgmap v5645: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:25:39.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:25:39 smithi138 ceph-mon[36954]: pgmap v5646: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:25:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:25:39 smithi100 ceph-mon[32599]: pgmap v5646: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:25:39.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:25:39 smithi167 ceph-mon[37140]: pgmap v5646: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:25:41.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:25:41 smithi138 ceph-mon[36954]: pgmap v5647: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:25:41.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:25:41 smithi100 ceph-mon[32599]: pgmap v5647: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:25:41.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:25:41 smithi167 ceph-mon[37140]: pgmap v5647: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:25:43.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:25:43 smithi138 ceph-mon[36954]: pgmap v5648: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:25:43.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:25:43 smithi100 ceph-mon[32599]: pgmap v5648: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:25:43.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:25:43 smithi167 ceph-mon[37140]: pgmap v5648: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:25:45.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:25:45 smithi138 ceph-mon[36954]: pgmap v5649: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:25:45.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:25:45 smithi100 ceph-mon[32599]: pgmap v5649: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:25:45.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:25:45 smithi167 ceph-mon[37140]: pgmap v5649: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:25:47.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:25:47 smithi138 ceph-mon[36954]: pgmap v5650: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:25:47.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:25:47 smithi100 ceph-mon[32599]: pgmap v5650: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:25:47.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:25:47 smithi167 ceph-mon[37140]: pgmap v5650: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:25:49.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:25:49 smithi138 ceph-mon[36954]: pgmap v5651: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:25:49.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:25:49 smithi100 ceph-mon[32599]: pgmap v5651: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:25:49.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:25:49 smithi167 ceph-mon[37140]: pgmap v5651: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:25:51.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:25:51 smithi138 ceph-mon[36954]: pgmap v5652: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:25:51.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:25:51 smithi100 ceph-mon[32599]: pgmap v5652: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:25:51.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:25:51 smithi167 ceph-mon[37140]: pgmap v5652: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:25:51.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:25:51.820+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:25:51.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:25:51.821+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:25:51.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:25:51.821+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:25:51.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:25:51.821+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:25:51.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:25:51.821+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:25:53.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:25:53 smithi138 ceph-mon[36954]: pgmap v5653: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:25:53.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:25:53 smithi100 ceph-mon[32599]: pgmap v5653: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:25:53.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:25:53 smithi167 ceph-mon[37140]: pgmap v5653: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:25:55.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:25:55 smithi138 ceph-mon[36954]: pgmap v5654: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:25:55.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:25:55 smithi100 ceph-mon[32599]: pgmap v5654: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:25:55.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:25:55 smithi167 ceph-mon[37140]: pgmap v5654: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:25:57.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:25:57 smithi138 ceph-mon[36954]: pgmap v5655: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:25:57.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:25:57 smithi100 ceph-mon[32599]: pgmap v5655: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:25:57.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:25:57 smithi167 ceph-mon[37140]: pgmap v5655: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:25:59.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:25:59 smithi138 ceph-mon[36954]: pgmap v5656: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:25:59.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:25:59 smithi100 ceph-mon[32599]: pgmap v5656: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:25:59.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:25:59 smithi167 ceph-mon[37140]: pgmap v5656: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:26:01.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:26:01 smithi138 ceph-mon[36954]: pgmap v5657: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:26:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:26:01 smithi100 ceph-mon[32599]: pgmap v5657: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:26:01.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:26:01 smithi167 ceph-mon[37140]: pgmap v5657: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:26:02.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:26:02 smithi138 ceph-mon[36954]: pgmap v5658: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:26:02.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:26:02 smithi100 ceph-mon[32599]: pgmap v5658: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:26:02.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:26:02 smithi167 ceph-mon[37140]: pgmap v5658: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:26:05.144 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:26:04 smithi138 ceph-mon[36954]: pgmap v5659: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T16:26:05.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:26:04 smithi100 ceph-mon[32599]: pgmap v5659: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T16:26:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:26:04 smithi167 ceph-mon[37140]: pgmap v5659: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T16:26:06.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:26:06.835+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:26:06.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:26:06.836+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:26:06.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:26:06.836+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:26:06.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:26:06.836+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:26:06.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:26:06.836+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:26:07.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:26:06 smithi138 ceph-mon[36954]: pgmap v5660: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:26:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:26:06 smithi100 ceph-mon[32599]: pgmap v5660: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:26:07.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:26:06 smithi167 ceph-mon[37140]: pgmap v5660: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:26:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:26:08 smithi138 ceph-mon[36954]: pgmap v5661: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:26:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:26:08 smithi138 ceph-mon[36954]: osdmap e400: 12 total, 12 up, 12 in 2024-07-26T16:26:09.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:26:08 smithi100 ceph-mon[32599]: pgmap v5661: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:26:09.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:26:08 smithi100 ceph-mon[32599]: osdmap e400: 12 total, 12 up, 12 in 2024-07-26T16:26:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:26:08 smithi167 ceph-mon[37140]: pgmap v5661: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:26:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:26:08 smithi167 ceph-mon[37140]: osdmap e400: 12 total, 12 up, 12 in 2024-07-26T16:26:11.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:26:10 smithi138 ceph-mon[36954]: pgmap v5663: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:26:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:26:10 smithi100 ceph-mon[32599]: pgmap v5663: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:26:11.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:26:10 smithi167 ceph-mon[37140]: pgmap v5663: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:26:13.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:26:12 smithi138 ceph-mon[36954]: pgmap v5664: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:26:13.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:26:12 smithi100 ceph-mon[32599]: pgmap v5664: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:26:13.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:26:12 smithi167 ceph-mon[37140]: pgmap v5664: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:26:15.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:26:14 smithi138 ceph-mon[36954]: pgmap v5665: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T16:26:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:26:14 smithi100 ceph-mon[32599]: pgmap v5665: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T16:26:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:26:14 smithi167 ceph-mon[37140]: pgmap v5665: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T16:26:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:26:16 smithi138 ceph-mon[36954]: pgmap v5666: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:26:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:26:16 smithi138 ceph-mon[36954]: osdmap e401: 12 total, 12 up, 12 in 2024-07-26T16:26:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:26:16 smithi100 ceph-mon[32599]: pgmap v5666: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:26:17.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:26:16 smithi100 ceph-mon[32599]: osdmap e401: 12 total, 12 up, 12 in 2024-07-26T16:26:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:26:16 smithi167 ceph-mon[37140]: pgmap v5666: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:26:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:26:16 smithi167 ceph-mon[37140]: osdmap e401: 12 total, 12 up, 12 in 2024-07-26T16:26:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:26:18 smithi138 ceph-mon[36954]: pgmap v5668: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:26:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:26:18 smithi100 ceph-mon[32599]: pgmap v5668: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:26:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:26:18 smithi167 ceph-mon[37140]: pgmap v5668: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:26:21.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:26:20 smithi167 ceph-mon[37140]: pgmap v5669: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:26:21.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:26:20 smithi138 ceph-mon[36954]: pgmap v5669: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:26:21.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:26:20 smithi100 ceph-mon[32599]: pgmap v5669: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:26:21.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:26:21.852+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:26:21.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:26:21.853+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:26:21.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:26:21.853+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:26:21.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:26:21.853+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:26:21.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:26:21.853+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:26:23.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:26:22 smithi138 ceph-mon[36954]: pgmap v5670: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:26:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:26:22 smithi100 ceph-mon[32599]: pgmap v5670: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:26:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:26:22 smithi167 ceph-mon[37140]: pgmap v5670: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:26:25.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:26:24 smithi138 ceph-mon[36954]: pgmap v5671: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:26:25.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:26:24 smithi100 ceph-mon[32599]: pgmap v5671: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:26:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:26:24 smithi167 ceph-mon[37140]: pgmap v5671: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:26:27.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:26:26 smithi138 ceph-mon[36954]: pgmap v5672: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:26:27.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:26:26 smithi100 ceph-mon[32599]: pgmap v5672: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:26:27.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:26:26 smithi167 ceph-mon[37140]: pgmap v5672: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:26:29.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:26:28 smithi138 ceph-mon[36954]: pgmap v5673: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:26:29.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:26:28 smithi100 ceph-mon[32599]: pgmap v5673: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:26:29.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:26:28 smithi167 ceph-mon[37140]: pgmap v5673: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:26:31.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:26:30 smithi138 ceph-mon[36954]: pgmap v5674: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:26:31.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:26:30 smithi100 ceph-mon[32599]: pgmap v5674: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:26:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:26:30 smithi167 ceph-mon[37140]: pgmap v5674: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:26:33.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:26:32 smithi138 ceph-mon[36954]: pgmap v5675: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:26:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:26:32 smithi100 ceph-mon[32599]: pgmap v5675: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:26:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:26:32 smithi167 ceph-mon[37140]: pgmap v5675: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:26:34.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:26:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:26:34.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:26:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:26:34.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:26:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:26:34.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:26:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:26:34.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:26:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:26:34.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:26:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:26:34.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:26:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:26:34.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:26:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:26:34.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:26:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:26:34.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:26:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:26:34.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:26:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:26:34.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:26:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:26:35.092 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:26:34 smithi138 ceph-mon[36954]: pgmap v5676: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:26:35.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:26:34 smithi100 ceph-mon[32599]: pgmap v5676: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:26:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:26:34 smithi167 ceph-mon[37140]: pgmap v5676: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:26:36.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:26:36.869+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:26:36.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:26:36.869+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:26:36.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:26:36.869+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:26:36.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:26:36.869+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:26:36.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:26:36.869+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:26:37.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:26:36 smithi138 ceph-mon[36954]: pgmap v5677: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:26:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:26:36 smithi100 ceph-mon[32599]: pgmap v5677: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:26:37.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:26:36 smithi167 ceph-mon[37140]: pgmap v5677: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:26:39.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:26:38 smithi138 ceph-mon[36954]: pgmap v5678: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:26:39.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:26:38 smithi100 ceph-mon[32599]: pgmap v5678: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:26:39.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:26:38 smithi167 ceph-mon[37140]: pgmap v5678: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:26:41.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:26:40 smithi138 ceph-mon[36954]: pgmap v5679: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:26:41.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:26:40 smithi100 ceph-mon[32599]: pgmap v5679: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:26:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:26:40 smithi167 ceph-mon[37140]: pgmap v5679: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:26:43.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:26:42 smithi138 ceph-mon[36954]: pgmap v5680: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:26:43.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:26:42 smithi100 ceph-mon[32599]: pgmap v5680: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:26:43.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:26:42 smithi167 ceph-mon[37140]: pgmap v5680: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:26:45.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:26:44 smithi138 ceph-mon[36954]: pgmap v5681: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:26:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:26:44 smithi100 ceph-mon[32599]: pgmap v5681: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:26:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:26:44 smithi167 ceph-mon[37140]: pgmap v5681: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:26:47.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:26:46 smithi138 ceph-mon[36954]: pgmap v5682: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:26:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:26:46 smithi100 ceph-mon[32599]: pgmap v5682: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:26:47.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:26:46 smithi167 ceph-mon[37140]: pgmap v5682: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:26:49.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:26:48 smithi138 ceph-mon[36954]: pgmap v5683: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:26:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:26:48 smithi100 ceph-mon[32599]: pgmap v5683: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:26:49.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:26:48 smithi167 ceph-mon[37140]: pgmap v5683: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:26:51.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:26:50 smithi138 ceph-mon[36954]: pgmap v5684: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:26:51.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:26:50 smithi100 ceph-mon[32599]: pgmap v5684: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:26:51.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:26:50 smithi167 ceph-mon[37140]: pgmap v5684: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:26:51.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:26:51.880+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:26:51.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:26:51.881+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:26:51.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:26:51.881+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:26:51.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:26:51.881+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:26:51.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:26:51.881+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:26:53.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:26:52 smithi138 ceph-mon[36954]: pgmap v5685: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:26:53.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:26:52 smithi100 ceph-mon[32599]: pgmap v5685: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:26:53.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:26:52 smithi167 ceph-mon[37140]: pgmap v5685: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:26:55.195 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:26:54 smithi138 ceph-mon[36954]: pgmap v5686: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:26:55.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:26:54 smithi100 ceph-mon[32599]: pgmap v5686: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:26:55.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:26:54 smithi167 ceph-mon[37140]: pgmap v5686: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:26:57.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:26:56 smithi138 ceph-mon[36954]: pgmap v5687: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:26:57.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:26:56 smithi100 ceph-mon[32599]: pgmap v5687: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:26:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:26:56 smithi167 ceph-mon[37140]: pgmap v5687: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:26:59.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:26:58 smithi138 ceph-mon[36954]: pgmap v5688: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:26:59.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:26:58 smithi100 ceph-mon[32599]: pgmap v5688: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:26:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:26:58 smithi167 ceph-mon[37140]: pgmap v5688: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:27:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:27:00 smithi138 ceph-mon[36954]: pgmap v5689: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:27:01.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:27:00 smithi100 ceph-mon[32599]: pgmap v5689: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:27:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:27:00 smithi167 ceph-mon[37140]: pgmap v5689: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:27:03.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:27:02 smithi138 ceph-mon[36954]: pgmap v5690: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:27:03.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:27:02 smithi100 ceph-mon[32599]: pgmap v5690: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:27:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:27:02 smithi167 ceph-mon[37140]: pgmap v5690: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:27:04.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:27:04.365+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T16:27:04.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:27:04.365+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T16:27:04.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:27:04.365+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 77 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T16:27:04.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:27:04.365+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 77 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c020 con 0x7fe5a81087a0 2024-07-26T16:27:05.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:27:04 smithi138 ceph-mon[36954]: pgmap v5691: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T16:27:05.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:27:04 smithi100 ceph-mon[32599]: pgmap v5691: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T16:27:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:27:04 smithi167 ceph-mon[37140]: pgmap v5691: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T16:27:06.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:27:06.895+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:27:06.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:27:06.895+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:27:06.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:27:06.895+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:27:06.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:27:06.895+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:27:06.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:27:06.895+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:27:07.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:27:06 smithi138 ceph-mon[36954]: pgmap v5692: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:27:07.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:27:06 smithi100 ceph-mon[32599]: pgmap v5692: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:27:07.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:27:06 smithi167 ceph-mon[37140]: pgmap v5692: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:27:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:27:08 smithi138 ceph-mon[36954]: pgmap v5693: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:27:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:27:08 smithi138 ceph-mon[36954]: osdmap e402: 12 total, 12 up, 12 in 2024-07-26T16:27:09.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:27:08 smithi100 ceph-mon[32599]: pgmap v5693: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:27:09.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:27:08 smithi100 ceph-mon[32599]: osdmap e402: 12 total, 12 up, 12 in 2024-07-26T16:27:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:27:08 smithi167 ceph-mon[37140]: pgmap v5693: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:27:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:27:08 smithi167 ceph-mon[37140]: osdmap e402: 12 total, 12 up, 12 in 2024-07-26T16:27:11.195 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:27:10 smithi138 ceph-mon[36954]: pgmap v5695: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:27:11.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:27:10 smithi100 ceph-mon[32599]: pgmap v5695: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:27:11.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:27:10 smithi167 ceph-mon[37140]: pgmap v5695: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:27:13.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:27:12 smithi138 ceph-mon[36954]: pgmap v5696: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:27:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:27:12 smithi100 ceph-mon[32599]: pgmap v5696: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:27:13.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:27:12 smithi167 ceph-mon[37140]: pgmap v5696: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:27:15.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:27:14 smithi138 ceph-mon[36954]: pgmap v5697: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T16:27:15.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:27:14 smithi100 ceph-mon[32599]: pgmap v5697: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T16:27:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:27:14 smithi167 ceph-mon[37140]: pgmap v5697: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T16:27:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:27:16 smithi138 ceph-mon[36954]: pgmap v5698: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:27:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:27:16 smithi138 ceph-mon[36954]: osdmap e403: 12 total, 12 up, 12 in 2024-07-26T16:27:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:27:16 smithi100 ceph-mon[32599]: pgmap v5698: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:27:17.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:27:16 smithi100 ceph-mon[32599]: osdmap e403: 12 total, 12 up, 12 in 2024-07-26T16:27:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:27:16 smithi167 ceph-mon[37140]: pgmap v5698: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:27:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:27:16 smithi167 ceph-mon[37140]: osdmap e403: 12 total, 12 up, 12 in 2024-07-26T16:27:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:27:18 smithi138 ceph-mon[36954]: pgmap v5700: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T16:27:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:27:18 smithi100 ceph-mon[32599]: pgmap v5700: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T16:27:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:27:18 smithi167 ceph-mon[37140]: pgmap v5700: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T16:27:21.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:27:20 smithi138 ceph-mon[36954]: pgmap v5701: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:27:21.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:27:20 smithi100 ceph-mon[32599]: pgmap v5701: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:27:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:27:20 smithi167 ceph-mon[37140]: pgmap v5701: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:27:21.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:27:21.904+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:27:21.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:27:21.904+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:27:21.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:27:21.904+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:27:21.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:27:21.904+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:27:21.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:27:21.904+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:27:23.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:27:22 smithi138 ceph-mon[36954]: pgmap v5702: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:27:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:27:22 smithi100 ceph-mon[32599]: pgmap v5702: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:27:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:27:22 smithi167 ceph-mon[37140]: pgmap v5702: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:27:25.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:27:24 smithi138 ceph-mon[36954]: pgmap v5703: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:27:25.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:27:24 smithi100 ceph-mon[32599]: pgmap v5703: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:27:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:27:24 smithi167 ceph-mon[37140]: pgmap v5703: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:27:27.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:27:26 smithi138 ceph-mon[36954]: pgmap v5704: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:27:27.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:27:26 smithi100 ceph-mon[32599]: pgmap v5704: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:27:27.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:27:26 smithi167 ceph-mon[37140]: pgmap v5704: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:27:29.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:27:28 smithi138 ceph-mon[36954]: pgmap v5705: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:27:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:27:28 smithi100 ceph-mon[32599]: pgmap v5705: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:27:29.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:27:28 smithi167 ceph-mon[37140]: pgmap v5705: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:27:31.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:27:30 smithi138 ceph-mon[36954]: pgmap v5706: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:27:31.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:27:30 smithi100 ceph-mon[32599]: pgmap v5706: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:27:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:27:30 smithi167 ceph-mon[37140]: pgmap v5706: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:27:33.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:27:32 smithi138 ceph-mon[36954]: pgmap v5707: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T16:27:33.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:27:32 smithi100 ceph-mon[32599]: pgmap v5707: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T16:27:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:27:32 smithi167 ceph-mon[37140]: pgmap v5707: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T16:27:34.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:27:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:27:34.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:27:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:27:34.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:27:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:27:35.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:27:34 smithi138 ceph-mon[36954]: pgmap v5708: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T16:27:35.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:27:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:27:35.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:27:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:27:35.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:27:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:27:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:27:34 smithi100 ceph-mon[32599]: pgmap v5708: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T16:27:35.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:27:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:27:35.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:27:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:27:35.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:27:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:27:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:27:34 smithi167 ceph-mon[37140]: pgmap v5708: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T16:27:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:27:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:27:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:27:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:27:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:27:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:27:36.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:27:36.920+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:27:36.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:27:36.921+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:27:36.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:27:36.921+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:27:36.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:27:36.921+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:27:36.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:27:36.921+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:27:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:27:36 smithi100 ceph-mon[32599]: pgmap v5709: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:27:37.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:27:36 smithi167 ceph-mon[37140]: pgmap v5709: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:27:37.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:27:36 smithi138 ceph-mon[36954]: pgmap v5709: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:27:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:27:38 smithi100 ceph-mon[32599]: pgmap v5710: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:27:39.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:27:38 smithi167 ceph-mon[37140]: pgmap v5710: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:27:39.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:27:38 smithi138 ceph-mon[36954]: pgmap v5710: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:27:41.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:27:40 smithi100 ceph-mon[32599]: pgmap v5711: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:27:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:27:40 smithi167 ceph-mon[37140]: pgmap v5711: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:27:41.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:27:40 smithi138 ceph-mon[36954]: pgmap v5711: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:27:43.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:27:42 smithi100 ceph-mon[32599]: pgmap v5712: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:27:43.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:27:42 smithi167 ceph-mon[37140]: pgmap v5712: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:27:43.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:27:42 smithi138 ceph-mon[36954]: pgmap v5712: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:27:45.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:27:44 smithi100 ceph-mon[32599]: pgmap v5713: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:27:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:27:44 smithi167 ceph-mon[37140]: pgmap v5713: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:27:45.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:27:44 smithi138 ceph-mon[36954]: pgmap v5713: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:27:47.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:27:46 smithi100 ceph-mon[32599]: pgmap v5714: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:27:47.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:27:46 smithi167 ceph-mon[37140]: pgmap v5714: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:27:47.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:27:46 smithi138 ceph-mon[36954]: pgmap v5714: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:27:49.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:27:48 smithi100 ceph-mon[32599]: pgmap v5715: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:27:49.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:27:48 smithi167 ceph-mon[37140]: pgmap v5715: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:27:49.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:27:48 smithi138 ceph-mon[36954]: pgmap v5715: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:27:51.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:27:50 smithi100 ceph-mon[32599]: pgmap v5716: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:27:51.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:27:50 smithi167 ceph-mon[37140]: pgmap v5716: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:27:51.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:27:50 smithi138 ceph-mon[36954]: pgmap v5716: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:27:51.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:27:51.929+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:27:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:27:51.930+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:27:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:27:51.930+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:27:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:27:51.930+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:27:51.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:27:51.930+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:27:52.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:27:52 smithi138 ceph-mon[36954]: pgmap v5717: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:27:52.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:27:52 smithi100 ceph-mon[32599]: pgmap v5717: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:27:52.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:27:52 smithi167 ceph-mon[37140]: pgmap v5717: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:27:55.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:27:54 smithi138 ceph-mon[36954]: pgmap v5718: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:27:55.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:27:54 smithi100 ceph-mon[32599]: pgmap v5718: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:27:55.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:27:54 smithi167 ceph-mon[37140]: pgmap v5718: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:27:57.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:27:56 smithi138 ceph-mon[36954]: pgmap v5719: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:27:57.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:27:56 smithi100 ceph-mon[32599]: pgmap v5719: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:27:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:27:56 smithi167 ceph-mon[37140]: pgmap v5719: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:27:59.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:27:58 smithi138 ceph-mon[36954]: pgmap v5720: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:27:59.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:27:58 smithi100 ceph-mon[32599]: pgmap v5720: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:27:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:27:58 smithi167 ceph-mon[37140]: pgmap v5720: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:28:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:00 smithi138 ceph-mon[36954]: pgmap v5721: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:28:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:00 smithi100 ceph-mon[32599]: pgmap v5721: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:28:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:00 smithi167 ceph-mon[37140]: pgmap v5721: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:28:03.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:02 smithi138 ceph-mon[36954]: pgmap v5722: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T16:28:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:02 smithi100 ceph-mon[32599]: pgmap v5722: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T16:28:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:02 smithi167 ceph-mon[37140]: pgmap v5722: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T16:28:05.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:04 smithi138 ceph-mon[36954]: pgmap v5723: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-07-26T16:28:05.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:04 smithi100 ceph-mon[32599]: pgmap v5723: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-07-26T16:28:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:04 smithi167 ceph-mon[37140]: pgmap v5723: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-07-26T16:28:06.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:28:06.944+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:28:06.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:28:06.944+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:28:06.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:28:06.944+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:28:06.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:28:06.944+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:28:06.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:28:06.944+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:28:07.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:06 smithi138 ceph-mon[36954]: pgmap v5724: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:28:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:06 smithi100 ceph-mon[32599]: pgmap v5724: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:28:07.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:06 smithi167 ceph-mon[37140]: pgmap v5724: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:28:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:08 smithi138 ceph-mon[36954]: pgmap v5725: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:28:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:08 smithi138 ceph-mon[36954]: osdmap e404: 12 total, 12 up, 12 in 2024-07-26T16:28:09.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:08 smithi100 ceph-mon[32599]: pgmap v5725: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:28:09.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:08 smithi100 ceph-mon[32599]: osdmap e404: 12 total, 12 up, 12 in 2024-07-26T16:28:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:08 smithi167 ceph-mon[37140]: pgmap v5725: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:28:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:08 smithi167 ceph-mon[37140]: osdmap e404: 12 total, 12 up, 12 in 2024-07-26T16:28:11.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:10 smithi138 ceph-mon[36954]: pgmap v5727: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:28:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:10 smithi100 ceph-mon[32599]: pgmap v5727: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:28:11.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:10 smithi167 ceph-mon[37140]: pgmap v5727: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:28:13.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:12 smithi138 ceph-mon[36954]: pgmap v5728: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:28:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:12 smithi100 ceph-mon[32599]: pgmap v5728: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:28:13.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:12 smithi167 ceph-mon[37140]: pgmap v5728: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:28:15.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:14 smithi138 ceph-mon[36954]: pgmap v5729: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T16:28:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:14 smithi100 ceph-mon[32599]: pgmap v5729: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T16:28:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:14 smithi167 ceph-mon[37140]: pgmap v5729: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T16:28:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:16 smithi138 ceph-mon[36954]: pgmap v5730: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:28:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:16 smithi138 ceph-mon[36954]: osdmap e405: 12 total, 12 up, 12 in 2024-07-26T16:28:17.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:16 smithi100 ceph-mon[32599]: pgmap v5730: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:28:17.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:16 smithi100 ceph-mon[32599]: osdmap e405: 12 total, 12 up, 12 in 2024-07-26T16:28:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:16 smithi167 ceph-mon[37140]: pgmap v5730: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:28:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:16 smithi167 ceph-mon[37140]: osdmap e405: 12 total, 12 up, 12 in 2024-07-26T16:28:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:18 smithi138 ceph-mon[36954]: pgmap v5732: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:28:19.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:18 smithi100 ceph-mon[32599]: pgmap v5732: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:28:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:18 smithi167 ceph-mon[37140]: pgmap v5732: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:28:21.194 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:20 smithi138 ceph-mon[36954]: pgmap v5733: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:28:21.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:20 smithi100 ceph-mon[32599]: pgmap v5733: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:28:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:20 smithi167 ceph-mon[37140]: pgmap v5733: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:28:21.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:28:21.951+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:28:21.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:28:21.952+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:28:21.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:28:21.952+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:28:21.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:28:21.952+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:28:21.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:28:21.952+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:28:23.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:22 smithi138 ceph-mon[36954]: pgmap v5734: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:28:23.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:22 smithi100 ceph-mon[32599]: pgmap v5734: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:28:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:22 smithi167 ceph-mon[37140]: pgmap v5734: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:28:25.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:24 smithi138 ceph-mon[36954]: pgmap v5735: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:28:25.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:24 smithi100 ceph-mon[32599]: pgmap v5735: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:28:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:24 smithi167 ceph-mon[37140]: pgmap v5735: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:28:27.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:26 smithi138 ceph-mon[36954]: pgmap v5736: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:28:27.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:26 smithi100 ceph-mon[32599]: pgmap v5736: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:28:27.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:26 smithi167 ceph-mon[37140]: pgmap v5736: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:28:29.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:28 smithi138 ceph-mon[36954]: pgmap v5737: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:28:29.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:28 smithi100 ceph-mon[32599]: pgmap v5737: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:28:29.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:28 smithi167 ceph-mon[37140]: pgmap v5737: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:28:31.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:30 smithi138 ceph-mon[36954]: pgmap v5738: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:28:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:30 smithi100 ceph-mon[32599]: pgmap v5738: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:28:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:30 smithi167 ceph-mon[37140]: pgmap v5738: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:28:32.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:32 smithi138 ceph-mon[36954]: pgmap v5739: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:28:32.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:32 smithi100 ceph-mon[32599]: pgmap v5739: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:28:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:32 smithi167 ceph-mon[37140]: pgmap v5739: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:28:35.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:34 smithi138 ceph-mon[36954]: pgmap v5740: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:28:35.267 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:28:35.267 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:28:35.267 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:28:35.268 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:28:35.268 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:28:35.268 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:28:35.268 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:28:35.268 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:28:35.268 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:28:35.268 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:28:35.269 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:28:35.269 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:28:35.269 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:28:35.269 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:28:35.269 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:28:35.269 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:28:35.270 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:34 smithi100 ceph-mon[32599]: pgmap v5740: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:28:35.270 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:28:35.270 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:28:35.270 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:28:35.271 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:28:35.271 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:28:35.271 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:28:35.271 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:28:35.271 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:28:35.271 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:28:35.271 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:28:35.271 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:28:35.272 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:28:35.272 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:28:35.272 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:28:35.272 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:28:35.272 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:28:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:34 smithi167 ceph-mon[37140]: pgmap v5740: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:28:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:28:35.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:28:35.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:28:35.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:28:35.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:28:35.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:28:35.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:28:35.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:28:35.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:28:35.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:28:35.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:28:35.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:28:35.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:28:35.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:28:35.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:28:35.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:28:36.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:35 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T16:28:36.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:35 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T16:28:36.093 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:35 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T16:28:36.094 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:35 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T16:28:36.094 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:35 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T16:28:36.094 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:35 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T16:28:36.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:35 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T16:28:36.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:35 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T16:28:36.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:35 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T16:28:36.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:35 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T16:28:36.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:35 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T16:28:36.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:35 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T16:28:36.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:35 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T16:28:36.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:35 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T16:28:36.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:35 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T16:28:36.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:35 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T16:28:36.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:35 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T16:28:36.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:35 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T16:28:36.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:28:36.966+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:28:36.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:28:36.967+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:28:36.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:28:36.967+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:28:36.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:28:36.967+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:28:36.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:28:36.967+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:28:37.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:36 smithi138 ceph-mon[36954]: pgmap v5741: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:28:37.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:36 smithi100 ceph-mon[32599]: pgmap v5741: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:28:37.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:36 smithi167 ceph-mon[37140]: pgmap v5741: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:28:39.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:38 smithi138 ceph-mon[36954]: pgmap v5742: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:28:39.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:38 smithi100 ceph-mon[32599]: pgmap v5742: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:28:39.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:38 smithi167 ceph-mon[37140]: pgmap v5742: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:28:41.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:40 smithi138 ceph-mon[36954]: pgmap v5743: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:28:41.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:40 smithi100 ceph-mon[32599]: pgmap v5743: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:28:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:40 smithi167 ceph-mon[37140]: pgmap v5743: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:28:43.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:42 smithi138 ceph-mon[36954]: pgmap v5744: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:28:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:42 smithi100 ceph-mon[32599]: pgmap v5744: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:28:43.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:42 smithi167 ceph-mon[37140]: pgmap v5744: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:28:45.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:44 smithi138 ceph-mon[36954]: pgmap v5745: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:28:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:44 smithi100 ceph-mon[32599]: pgmap v5745: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:28:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:44 smithi167 ceph-mon[37140]: pgmap v5745: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:28:47.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:46 smithi138 ceph-mon[36954]: pgmap v5746: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:28:47.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:46 smithi100 ceph-mon[32599]: pgmap v5746: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:28:47.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:46 smithi167 ceph-mon[37140]: pgmap v5746: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:28:49.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:48 smithi138 ceph-mon[36954]: pgmap v5747: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:28:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:48 smithi100 ceph-mon[32599]: pgmap v5747: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:28:49.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:48 smithi167 ceph-mon[37140]: pgmap v5747: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:28:51.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:50 smithi138 ceph-mon[36954]: pgmap v5748: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:28:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:50 smithi100 ceph-mon[32599]: pgmap v5748: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:28:51.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:50 smithi167 ceph-mon[37140]: pgmap v5748: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:28:51.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:28:51.982+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:28:51.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:28:51.982+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:28:51.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:28:51.982+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:28:51.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:28:51.982+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:28:51.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:28:51.982+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:28:53.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:52 smithi138 ceph-mon[36954]: pgmap v5749: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:28:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:52 smithi100 ceph-mon[32599]: pgmap v5749: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:28:53.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:52 smithi167 ceph-mon[37140]: pgmap v5749: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:28:55.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:54 smithi138 ceph-mon[36954]: pgmap v5750: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:28:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:54 smithi100 ceph-mon[32599]: pgmap v5750: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:28:55.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:54 smithi167 ceph-mon[37140]: pgmap v5750: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:28:57.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:56 smithi138 ceph-mon[36954]: pgmap v5751: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:28:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:56 smithi100 ceph-mon[32599]: pgmap v5751: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:28:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:56 smithi167 ceph-mon[37140]: pgmap v5751: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:28:59.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:28:58 smithi138 ceph-mon[36954]: pgmap v5752: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:28:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:28:58 smithi100 ceph-mon[32599]: pgmap v5752: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:28:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:28:58 smithi167 ceph-mon[37140]: pgmap v5752: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:29:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:29:00 smithi138 ceph-mon[36954]: pgmap v5753: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:29:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:29:00 smithi100 ceph-mon[32599]: pgmap v5753: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:29:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:29:00 smithi167 ceph-mon[37140]: pgmap v5753: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:29:03.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:29:02 smithi138 ceph-mon[36954]: pgmap v5754: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:29:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:29:02 smithi100 ceph-mon[32599]: pgmap v5754: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:29:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:29:02 smithi167 ceph-mon[37140]: pgmap v5754: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:29:05.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:29:04 smithi138 ceph-mon[36954]: pgmap v5755: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T16:29:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:29:04 smithi100 ceph-mon[32599]: pgmap v5755: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T16:29:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:29:04 smithi167 ceph-mon[37140]: pgmap v5755: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T16:29:07.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:29:06.998+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:29:07.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:29:06.998+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:29:07.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:29:06.998+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:29:07.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:29:06.998+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:29:07.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:29:06.998+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:29:07.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:29:06 smithi138 ceph-mon[36954]: pgmap v5756: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:29:07.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:29:06 smithi100 ceph-mon[32599]: pgmap v5756: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:29:07.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:29:06 smithi167 ceph-mon[37140]: pgmap v5756: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:29:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:29:08 smithi138 ceph-mon[36954]: pgmap v5757: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:29:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:29:08 smithi138 ceph-mon[36954]: osdmap e406: 12 total, 12 up, 12 in 2024-07-26T16:29:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:29:08 smithi100 ceph-mon[32599]: pgmap v5757: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:29:09.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:29:08 smithi100 ceph-mon[32599]: osdmap e406: 12 total, 12 up, 12 in 2024-07-26T16:29:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:29:08 smithi167 ceph-mon[37140]: pgmap v5757: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:29:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:29:08 smithi167 ceph-mon[37140]: osdmap e406: 12 total, 12 up, 12 in 2024-07-26T16:29:11.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:29:10 smithi138 ceph-mon[36954]: pgmap v5759: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:29:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:29:10 smithi100 ceph-mon[32599]: pgmap v5759: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:29:11.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:29:10 smithi167 ceph-mon[37140]: pgmap v5759: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:29:13.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:29:12 smithi138 ceph-mon[36954]: pgmap v5760: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:29:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:29:12 smithi100 ceph-mon[32599]: pgmap v5760: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:29:13.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:29:12 smithi167 ceph-mon[37140]: pgmap v5760: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:29:15.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:29:14 smithi138 ceph-mon[36954]: pgmap v5761: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T16:29:15.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:29:14 smithi100 ceph-mon[32599]: pgmap v5761: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T16:29:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:29:14 smithi167 ceph-mon[37140]: pgmap v5761: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T16:29:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:29:16 smithi138 ceph-mon[36954]: pgmap v5762: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:29:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:29:16 smithi138 ceph-mon[36954]: osdmap e407: 12 total, 12 up, 12 in 2024-07-26T16:29:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:29:16 smithi100 ceph-mon[32599]: pgmap v5762: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:29:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:29:16 smithi100 ceph-mon[32599]: osdmap e407: 12 total, 12 up, 12 in 2024-07-26T16:29:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:29:16 smithi167 ceph-mon[37140]: pgmap v5762: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:29:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:29:16 smithi167 ceph-mon[37140]: osdmap e407: 12 total, 12 up, 12 in 2024-07-26T16:29:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:29:18 smithi138 ceph-mon[36954]: pgmap v5764: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T16:29:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:29:18 smithi100 ceph-mon[32599]: pgmap v5764: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T16:29:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:29:18 smithi167 ceph-mon[37140]: pgmap v5764: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 207 B/s wr, 6 op/s 2024-07-26T16:29:21.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:29:20 smithi138 ceph-mon[36954]: pgmap v5765: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:29:21.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:29:20 smithi100 ceph-mon[32599]: pgmap v5765: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:29:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:29:20 smithi167 ceph-mon[37140]: pgmap v5765: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:29:22.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:29:22.000+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:29:22.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:29:22.000+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:29:22.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:29:22.000+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:29:22.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:29:22.000+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:29:22.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:29:22.000+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:29:23.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:29:22 smithi138 ceph-mon[36954]: pgmap v5766: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:29:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:29:22 smithi100 ceph-mon[32599]: pgmap v5766: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:29:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:29:22 smithi167 ceph-mon[37140]: pgmap v5766: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:29:25.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:29:24 smithi138 ceph-mon[36954]: pgmap v5767: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:29:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:29:24 smithi100 ceph-mon[32599]: pgmap v5767: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:29:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:29:24 smithi167 ceph-mon[37140]: pgmap v5767: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:29:27.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:29:26 smithi138 ceph-mon[36954]: pgmap v5768: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:29:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:29:26 smithi100 ceph-mon[32599]: pgmap v5768: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:29:27.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:29:26 smithi167 ceph-mon[37140]: pgmap v5768: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:29:29.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:29:28 smithi138 ceph-mon[36954]: pgmap v5769: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:29:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:29:28 smithi100 ceph-mon[32599]: pgmap v5769: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:29:29.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:29:28 smithi167 ceph-mon[37140]: pgmap v5769: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:29:31.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:29:30 smithi138 ceph-mon[36954]: pgmap v5770: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:29:31.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:29:30 smithi100 ceph-mon[32599]: pgmap v5770: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:29:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:29:30 smithi167 ceph-mon[37140]: pgmap v5770: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:29:33.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:29:32 smithi138 ceph-mon[36954]: pgmap v5771: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:29:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:29:32 smithi100 ceph-mon[32599]: pgmap v5771: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:29:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:29:32 smithi167 ceph-mon[37140]: pgmap v5771: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:29:35.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:29:34 smithi138 ceph-mon[36954]: pgmap v5772: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:29:35.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:29:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:29:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:29:34 smithi100 ceph-mon[32599]: pgmap v5772: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:29:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:29:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:29:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:29:34 smithi167 ceph-mon[37140]: pgmap v5772: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:29:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:29:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:29:36.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:29:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:29:36.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:29:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:29:36.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:29:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:29:36.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:29:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:29:36.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:29:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:29:36.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:29:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:29:36.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:29:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:29:36.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:29:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:29:36.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:29:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:29:37.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:29:37.015+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:29:37.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:29:37.015+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:29:37.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:29:37.015+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:29:37.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:29:37.015+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:29:37.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:29:37.015+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:29:37.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:29:36 smithi100 ceph-mon[32599]: pgmap v5773: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:29:37.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:29:36 smithi167 ceph-mon[37140]: pgmap v5773: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:29:37.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:29:36 smithi138 ceph-mon[36954]: pgmap v5773: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:29:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:29:38 smithi100 ceph-mon[32599]: pgmap v5774: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:29:39.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:29:38 smithi167 ceph-mon[37140]: pgmap v5774: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:29:39.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:29:38 smithi138 ceph-mon[36954]: pgmap v5774: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:29:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:29:40 smithi100 ceph-mon[32599]: pgmap v5775: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:29:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:29:40 smithi167 ceph-mon[37140]: pgmap v5775: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:29:41.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:29:40 smithi138 ceph-mon[36954]: pgmap v5775: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:29:43.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:29:42 smithi100 ceph-mon[32599]: pgmap v5776: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:29:43.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:29:42 smithi167 ceph-mon[37140]: pgmap v5776: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:29:43.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:29:42 smithi138 ceph-mon[36954]: pgmap v5776: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:29:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:29:44 smithi100 ceph-mon[32599]: pgmap v5777: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:29:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:29:44 smithi167 ceph-mon[37140]: pgmap v5777: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:29:45.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:29:44 smithi138 ceph-mon[36954]: pgmap v5777: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:29:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:29:46 smithi100 ceph-mon[32599]: pgmap v5778: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:29:47.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:29:46 smithi167 ceph-mon[37140]: pgmap v5778: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:29:47.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:29:46 smithi138 ceph-mon[36954]: pgmap v5778: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:29:49.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:29:48 smithi100 ceph-mon[32599]: pgmap v5779: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:29:49.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:29:48 smithi167 ceph-mon[37140]: pgmap v5779: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:29:49.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:29:48 smithi138 ceph-mon[36954]: pgmap v5779: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:29:51.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:29:50 smithi100 ceph-mon[32599]: pgmap v5780: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:29:51.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:29:50 smithi167 ceph-mon[37140]: pgmap v5780: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:29:51.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:29:50 smithi138 ceph-mon[36954]: pgmap v5780: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:29:52.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:29:52.030+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:29:52.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:29:52.031+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:29:52.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:29:52.031+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:29:52.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:29:52.031+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:29:52.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:29:52.031+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:29:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:29:52 smithi100 ceph-mon[32599]: pgmap v5781: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:29:53.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:29:52 smithi167 ceph-mon[37140]: pgmap v5781: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:29:53.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:29:52 smithi138 ceph-mon[36954]: pgmap v5781: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:29:55.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:29:54 smithi100 ceph-mon[32599]: pgmap v5782: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:29:55.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:29:54 smithi167 ceph-mon[37140]: pgmap v5782: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:29:55.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:29:54 smithi138 ceph-mon[36954]: pgmap v5782: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:29:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:29:56 smithi100 ceph-mon[32599]: pgmap v5783: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:29:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:29:56 smithi167 ceph-mon[37140]: pgmap v5783: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:29:57.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:29:56 smithi138 ceph-mon[36954]: pgmap v5783: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:29:59.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:29:58 smithi100 ceph-mon[32599]: pgmap v5784: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:29:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:29:58 smithi167 ceph-mon[37140]: pgmap v5784: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:29:59.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:29:58 smithi138 ceph-mon[36954]: pgmap v5784: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:30:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:30:00 smithi100 ceph-mon[32599]: pgmap v5785: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:30:01.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:30:00 smithi100 ceph-mon[32599]: overall HEALTH_OK 2024-07-26T16:30:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:30:00 smithi167 ceph-mon[37140]: pgmap v5785: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:30:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:30:00 smithi167 ceph-mon[37140]: overall HEALTH_OK 2024-07-26T16:30:01.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:30:00 smithi138 ceph-mon[36954]: pgmap v5785: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:30:01.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:30:00 smithi138 ceph-mon[36954]: overall HEALTH_OK 2024-07-26T16:30:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:30:02 smithi100 ceph-mon[32599]: pgmap v5786: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:30:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:30:02 smithi167 ceph-mon[37140]: pgmap v5786: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:30:03.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:30:02 smithi138 ceph-mon[36954]: pgmap v5786: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:30:04.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:30:04.391+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T16:30:04.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:30:04.391+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T16:30:04.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:30:04.391+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 78 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T16:30:04.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:30:04.391+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 78 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c020 con 0x7fe5a81087a0 2024-07-26T16:30:05.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:30:04 smithi100 ceph-mon[32599]: pgmap v5787: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T16:30:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:30:04 smithi167 ceph-mon[37140]: pgmap v5787: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T16:30:05.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:30:04 smithi138 ceph-mon[36954]: pgmap v5787: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T16:30:07.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:30:07.047+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:30:07.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:30:07.048+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:30:07.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:30:07.048+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:30:07.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:30:07.048+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:30:07.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:30:07.048+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:30:07.243 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:30:06 smithi138 ceph-mon[36954]: pgmap v5788: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:30:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:30:06 smithi100 ceph-mon[32599]: pgmap v5788: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:30:07.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:30:06 smithi167 ceph-mon[37140]: pgmap v5788: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:30:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:30:08 smithi167 ceph-mon[37140]: pgmap v5789: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:30:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:30:08 smithi167 ceph-mon[37140]: osdmap e408: 12 total, 12 up, 12 in 2024-07-26T16:30:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:30:08 smithi138 ceph-mon[36954]: pgmap v5789: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:30:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:30:08 smithi138 ceph-mon[36954]: osdmap e408: 12 total, 12 up, 12 in 2024-07-26T16:30:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:30:08 smithi100 ceph-mon[32599]: pgmap v5789: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:30:09.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:30:08 smithi100 ceph-mon[32599]: osdmap e408: 12 total, 12 up, 12 in 2024-07-26T16:30:11.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:30:11 smithi167 ceph-mon[37140]: pgmap v5791: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:30:11.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:30:11 smithi138 ceph-mon[36954]: pgmap v5791: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:30:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:30:11 smithi100 ceph-mon[32599]: pgmap v5791: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:30:13.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:30:13 smithi167 ceph-mon[37140]: pgmap v5792: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:30:13.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:30:13 smithi138 ceph-mon[36954]: pgmap v5792: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:30:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:30:13 smithi100 ceph-mon[32599]: pgmap v5792: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:30:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:30:15 smithi167 ceph-mon[37140]: pgmap v5793: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T16:30:15.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:30:15 smithi138 ceph-mon[36954]: pgmap v5793: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T16:30:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:30:15 smithi100 ceph-mon[32599]: pgmap v5793: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T16:30:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:30:17 smithi167 ceph-mon[37140]: pgmap v5794: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:30:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:30:17 smithi167 ceph-mon[37140]: osdmap e409: 12 total, 12 up, 12 in 2024-07-26T16:30:17.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:30:17 smithi138 ceph-mon[36954]: pgmap v5794: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:30:17.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:30:17 smithi138 ceph-mon[36954]: osdmap e409: 12 total, 12 up, 12 in 2024-07-26T16:30:17.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:30:17 smithi100 ceph-mon[32599]: pgmap v5794: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:30:17.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:30:17 smithi100 ceph-mon[32599]: osdmap e409: 12 total, 12 up, 12 in 2024-07-26T16:30:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:30:19 smithi167 ceph-mon[37140]: pgmap v5796: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 208 B/s wr, 6 op/s 2024-07-26T16:30:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:30:19 smithi138 ceph-mon[36954]: pgmap v5796: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 208 B/s wr, 6 op/s 2024-07-26T16:30:19.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:30:19 smithi100 ceph-mon[32599]: pgmap v5796: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 208 B/s wr, 6 op/s 2024-07-26T16:30:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:30:21 smithi167 ceph-mon[37140]: pgmap v5797: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:30:21.296 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:30:21 smithi100 ceph-mon[32599]: pgmap v5797: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:30:21.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:30:21 smithi138 ceph-mon[36954]: pgmap v5797: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:30:22.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:30:22.064+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:30:22.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:30:22.064+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:30:22.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:30:22.064+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:30:22.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:30:22.064+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:30:22.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:30:22.064+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:30:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:30:23 smithi167 ceph-mon[37140]: pgmap v5798: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:30:23.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:30:23 smithi138 ceph-mon[36954]: pgmap v5798: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:30:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:30:23 smithi100 ceph-mon[32599]: pgmap v5798: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:30:25.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:30:25 smithi138 ceph-mon[36954]: pgmap v5799: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:30:25.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:30:25 smithi100 ceph-mon[32599]: pgmap v5799: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:30:25.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:30:25 smithi167 ceph-mon[37140]: pgmap v5799: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:30:27.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:30:27 smithi138 ceph-mon[36954]: pgmap v5800: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:30:27.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:30:27 smithi100 ceph-mon[32599]: pgmap v5800: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:30:27.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:30:27 smithi167 ceph-mon[37140]: pgmap v5800: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:30:29.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:30:29 smithi138 ceph-mon[36954]: pgmap v5801: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:30:29.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:30:29 smithi100 ceph-mon[32599]: pgmap v5801: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:30:29.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:30:29 smithi167 ceph-mon[37140]: pgmap v5801: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:30:31.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:30:31 smithi138 ceph-mon[36954]: pgmap v5802: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:30:31.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:30:31 smithi100 ceph-mon[32599]: pgmap v5802: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:30:31.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:30:31 smithi167 ceph-mon[37140]: pgmap v5802: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:30:33.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:30:33 smithi138 ceph-mon[36954]: pgmap v5803: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:30:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:30:33 smithi100 ceph-mon[32599]: pgmap v5803: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:30:33.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:30:33 smithi167 ceph-mon[37140]: pgmap v5803: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:30:35.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:30:35 smithi138 ceph-mon[36954]: pgmap v5804: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:30:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:30:35 smithi100 ceph-mon[32599]: pgmap v5804: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:30:35.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:30:35 smithi167 ceph-mon[37140]: pgmap v5804: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:30:36.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:30:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:30:36.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:30:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:30:36.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:30:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:30:36.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:30:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:30:36.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:30:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:30:36.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:30:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:30:36.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:30:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:30:36.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:30:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:30:36.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:30:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:30:36.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:30:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:30:36.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:30:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:30:36.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:30:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:30:37.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:30:37.080+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:30:37.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:30:37.081+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:30:37.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:30:37.081+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:30:37.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:30:37.081+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:30:37.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:30:37.081+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:30:37.342 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:30:37 smithi138 ceph-mon[36954]: pgmap v5805: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:30:37.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:30:37 smithi100 ceph-mon[32599]: pgmap v5805: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:30:37.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:30:37 smithi167 ceph-mon[37140]: pgmap v5805: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:30:39.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:30:39 smithi138 ceph-mon[36954]: pgmap v5806: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:30:39.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:30:39 smithi100 ceph-mon[32599]: pgmap v5806: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:30:39.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:30:39 smithi167 ceph-mon[37140]: pgmap v5806: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:30:41.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:30:41 smithi138 ceph-mon[36954]: pgmap v5807: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:30:41.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:30:41 smithi100 ceph-mon[32599]: pgmap v5807: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:30:41.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:30:41 smithi167 ceph-mon[37140]: pgmap v5807: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:30:43.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:30:43 smithi138 ceph-mon[36954]: pgmap v5808: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:30:43.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:30:43 smithi100 ceph-mon[32599]: pgmap v5808: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:30:43.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:30:43 smithi167 ceph-mon[37140]: pgmap v5808: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:30:45.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:30:45 smithi138 ceph-mon[36954]: pgmap v5809: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:30:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:30:45 smithi100 ceph-mon[32599]: pgmap v5809: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:30:45.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:30:45 smithi167 ceph-mon[37140]: pgmap v5809: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:30:47.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:30:47 smithi138 ceph-mon[36954]: pgmap v5810: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:30:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:30:47 smithi100 ceph-mon[32599]: pgmap v5810: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:30:47.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:30:47 smithi167 ceph-mon[37140]: pgmap v5810: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:30:49.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:30:49 smithi138 ceph-mon[36954]: pgmap v5811: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:30:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:30:49 smithi100 ceph-mon[32599]: pgmap v5811: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:30:49.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:30:49 smithi167 ceph-mon[37140]: pgmap v5811: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:30:51.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:30:51 smithi167 ceph-mon[37140]: pgmap v5812: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:30:51.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:30:51 smithi138 ceph-mon[36954]: pgmap v5812: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:30:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:30:51 smithi100 ceph-mon[32599]: pgmap v5812: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:30:52.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:30:52.096+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:30:52.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:30:52.096+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:30:52.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:30:52.096+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:30:52.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:30:52.096+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:30:52.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:30:52.097+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:30:53.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:30:53 smithi138 ceph-mon[36954]: pgmap v5813: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:30:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:30:53 smithi100 ceph-mon[32599]: pgmap v5813: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:30:53.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:30:53 smithi167 ceph-mon[37140]: pgmap v5813: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:30:55.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:30:55 smithi138 ceph-mon[36954]: pgmap v5814: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:30:55.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:30:55 smithi100 ceph-mon[32599]: pgmap v5814: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:30:55.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:30:55 smithi167 ceph-mon[37140]: pgmap v5814: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:30:57.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:30:57 smithi138 ceph-mon[36954]: pgmap v5815: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:30:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:30:57 smithi100 ceph-mon[32599]: pgmap v5815: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:30:57.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:30:57 smithi167 ceph-mon[37140]: pgmap v5815: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:30:59.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:30:59 smithi138 ceph-mon[36954]: pgmap v5816: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:30:59.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:30:59 smithi100 ceph-mon[32599]: pgmap v5816: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:30:59.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:30:59 smithi167 ceph-mon[37140]: pgmap v5816: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:31:01.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:01 smithi138 ceph-mon[36954]: pgmap v5817: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:31:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:01 smithi100 ceph-mon[32599]: pgmap v5817: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:31:01.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:01 smithi167 ceph-mon[37140]: pgmap v5817: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:31:03.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:03 smithi138 ceph-mon[36954]: pgmap v5818: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:31:03.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:03 smithi100 ceph-mon[32599]: pgmap v5818: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:31:03.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:03 smithi167 ceph-mon[37140]: pgmap v5818: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:31:05.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:05 smithi138 ceph-mon[36954]: pgmap v5819: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T16:31:05.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:05 smithi100 ceph-mon[32599]: pgmap v5819: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T16:31:05.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:05 smithi167 ceph-mon[37140]: pgmap v5819: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T16:31:07.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:31:07.112+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:31:07.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:31:07.112+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:31:07.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:31:07.112+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:31:07.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:31:07.112+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:31:07.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:31:07.112+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:31:07.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:07 smithi138 ceph-mon[36954]: pgmap v5820: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:31:07.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:07 smithi100 ceph-mon[32599]: pgmap v5820: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:31:07.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:07 smithi167 ceph-mon[37140]: pgmap v5820: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:31:08.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:08 smithi138 ceph-mon[36954]: osdmap e410: 12 total, 12 up, 12 in 2024-07-26T16:31:08.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:08 smithi100 ceph-mon[32599]: osdmap e410: 12 total, 12 up, 12 in 2024-07-26T16:31:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:08 smithi167 ceph-mon[37140]: osdmap e410: 12 total, 12 up, 12 in 2024-07-26T16:31:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:09 smithi138 ceph-mon[36954]: pgmap v5822: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:31:09.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:09 smithi100 ceph-mon[32599]: pgmap v5822: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:31:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:09 smithi167 ceph-mon[37140]: pgmap v5822: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:31:11.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:11 smithi138 ceph-mon[36954]: pgmap v5823: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:31:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:11 smithi100 ceph-mon[32599]: pgmap v5823: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:31:11.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:11 smithi167 ceph-mon[37140]: pgmap v5823: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:31:13.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:13 smithi138 ceph-mon[36954]: pgmap v5824: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:31:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:13 smithi100 ceph-mon[32599]: pgmap v5824: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:31:13.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:13 smithi167 ceph-mon[37140]: pgmap v5824: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:31:15.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:15 smithi138 ceph-mon[36954]: pgmap v5825: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:31:15.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:15 smithi100 ceph-mon[32599]: pgmap v5825: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:31:15.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:15 smithi167 ceph-mon[37140]: pgmap v5825: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:31:17.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:17 smithi138 ceph-mon[36954]: pgmap v5826: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:31:17.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:17 smithi138 ceph-mon[36954]: osdmap e411: 12 total, 12 up, 12 in 2024-07-26T16:31:17.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:17 smithi100 ceph-mon[32599]: pgmap v5826: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:31:17.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:17 smithi100 ceph-mon[32599]: osdmap e411: 12 total, 12 up, 12 in 2024-07-26T16:31:17.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:17 smithi167 ceph-mon[37140]: pgmap v5826: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:31:17.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:17 smithi167 ceph-mon[37140]: osdmap e411: 12 total, 12 up, 12 in 2024-07-26T16:31:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:19 smithi138 ceph-mon[36954]: pgmap v5828: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:31:19.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:19 smithi100 ceph-mon[32599]: pgmap v5828: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:31:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:19 smithi167 ceph-mon[37140]: pgmap v5828: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:31:21.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:21 smithi138 ceph-mon[36954]: pgmap v5829: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:31:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:21 smithi100 ceph-mon[32599]: pgmap v5829: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:31:21.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:21 smithi167 ceph-mon[37140]: pgmap v5829: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:31:22.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:31:22.128+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:31:22.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:31:22.129+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:31:22.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:31:22.129+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:31:22.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:31:22.129+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:31:22.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:31:22.129+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:31:23.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:23 smithi138 ceph-mon[36954]: pgmap v5830: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:31:23.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:23 smithi100 ceph-mon[32599]: pgmap v5830: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:31:23.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:23 smithi167 ceph-mon[37140]: pgmap v5830: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:31:25.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:25 smithi138 ceph-mon[36954]: pgmap v5831: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:31:25.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:25 smithi100 ceph-mon[32599]: pgmap v5831: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:31:25.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:25 smithi167 ceph-mon[37140]: pgmap v5831: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:31:27.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:27 smithi138 ceph-mon[36954]: pgmap v5832: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:31:27.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:27 smithi100 ceph-mon[32599]: pgmap v5832: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:31:27.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:27 smithi167 ceph-mon[37140]: pgmap v5832: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:31:29.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:29 smithi138 ceph-mon[36954]: pgmap v5833: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:31:29.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:29 smithi100 ceph-mon[32599]: pgmap v5833: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:31:29.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:29 smithi167 ceph-mon[37140]: pgmap v5833: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:31:31.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:31 smithi138 ceph-mon[36954]: pgmap v5834: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:31:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:31 smithi100 ceph-mon[32599]: pgmap v5834: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:31:31.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:31 smithi167 ceph-mon[37140]: pgmap v5834: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:31:33.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:33 smithi138 ceph-mon[36954]: pgmap v5835: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:31:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:33 smithi100 ceph-mon[32599]: pgmap v5835: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:31:33.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:33 smithi167 ceph-mon[37140]: pgmap v5835: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:31:35.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:35 smithi138 ceph-mon[36954]: pgmap v5836: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:31:35.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:35 smithi100 ceph-mon[32599]: pgmap v5836: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:31:35.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:35 smithi167 ceph-mon[37140]: pgmap v5836: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:31:36.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:31:36.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:31:36.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:31:37.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:31:37.144+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:31:37.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:31:37.144+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:31:37.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:31:37.144+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:31:37.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:31:37.144+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:31:37.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:31:37.144+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:31:37.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:37 smithi138 ceph-mon[36954]: pgmap v5837: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:31:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:37 smithi100 ceph-mon[32599]: pgmap v5837: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:31:37.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:37 smithi167 ceph-mon[37140]: pgmap v5837: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:31:38.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:31:38.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:31:38.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:31:38.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:31:38.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:31:38.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:31:38.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:38 smithi138 ceph-mon[36954]: pgmap v5838: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:31:38.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:31:38.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:31:38.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:31:38.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:31:38.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:31:38.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:31:38.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:31:38.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:31:38.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:31:38.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:38 smithi100 ceph-mon[32599]: pgmap v5838: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:31:38.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:31:38.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:31:38.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:31:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:31:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:31:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:31:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:31:38.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:31:38.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:31:38.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:38 smithi167 ceph-mon[37140]: pgmap v5838: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:31:38.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:31:38.777 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:31:38.778 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:31:41.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:40 smithi138 ceph-mon[36954]: pgmap v5839: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:31:41.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:40 smithi100 ceph-mon[32599]: pgmap v5839: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:31:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:40 smithi167 ceph-mon[37140]: pgmap v5839: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:31:43.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:42 smithi138 ceph-mon[36954]: pgmap v5840: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:31:43.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:42 smithi100 ceph-mon[32599]: pgmap v5840: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:31:43.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:42 smithi167 ceph-mon[37140]: pgmap v5840: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:31:45.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:44 smithi138 ceph-mon[36954]: pgmap v5841: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:31:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:44 smithi100 ceph-mon[32599]: pgmap v5841: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:31:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:44 smithi167 ceph-mon[37140]: pgmap v5841: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:31:47.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:46 smithi138 ceph-mon[36954]: pgmap v5842: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:31:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:46 smithi100 ceph-mon[32599]: pgmap v5842: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:31:47.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:46 smithi167 ceph-mon[37140]: pgmap v5842: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:31:49.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:48 smithi138 ceph-mon[36954]: pgmap v5843: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:31:49.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:48 smithi100 ceph-mon[32599]: pgmap v5843: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:31:49.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:48 smithi167 ceph-mon[37140]: pgmap v5843: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:31:51.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:50 smithi138 ceph-mon[36954]: pgmap v5844: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:31:51.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:50 smithi100 ceph-mon[32599]: pgmap v5844: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:31:51.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:50 smithi167 ceph-mon[37140]: pgmap v5844: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:31:52.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:31:52.145+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:31:52.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:31:52.146+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:31:52.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:31:52.146+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:31:52.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:31:52.146+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:31:52.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:31:52.146+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:31:53.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:52 smithi138 ceph-mon[36954]: pgmap v5845: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:31:53.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:52 smithi100 ceph-mon[32599]: pgmap v5845: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:31:53.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:52 smithi167 ceph-mon[37140]: pgmap v5845: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:31:55.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:54 smithi138 ceph-mon[36954]: pgmap v5846: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:31:55.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:54 smithi100 ceph-mon[32599]: pgmap v5846: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:31:55.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:54 smithi167 ceph-mon[37140]: pgmap v5846: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:31:57.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:56 smithi138 ceph-mon[36954]: pgmap v5847: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:31:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:56 smithi100 ceph-mon[32599]: pgmap v5847: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:31:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:56 smithi167 ceph-mon[37140]: pgmap v5847: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:31:59.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:31:58 smithi138 ceph-mon[36954]: pgmap v5848: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:31:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:31:58 smithi100 ceph-mon[32599]: pgmap v5848: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:31:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:31:58 smithi167 ceph-mon[37140]: pgmap v5848: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:32:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:00 smithi138 ceph-mon[36954]: pgmap v5849: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:32:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:00 smithi100 ceph-mon[32599]: pgmap v5849: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:32:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:00 smithi167 ceph-mon[37140]: pgmap v5849: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:32:03.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:02 smithi138 ceph-mon[36954]: pgmap v5850: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:32:03.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:02 smithi100 ceph-mon[32599]: pgmap v5850: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:32:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:02 smithi167 ceph-mon[37140]: pgmap v5850: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:32:05.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:04 smithi138 ceph-mon[36954]: pgmap v5851: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T16:32:05.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:04 smithi100 ceph-mon[32599]: pgmap v5851: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T16:32:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:04 smithi167 ceph-mon[37140]: pgmap v5851: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T16:32:07.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:32:07.151+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:32:07.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:32:07.152+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:32:07.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:32:07.152+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:32:07.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:32:07.152+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:32:07.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:32:07.152+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:32:07.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:06 smithi100 ceph-mon[32599]: pgmap v5852: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:32:07.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:06 smithi167 ceph-mon[37140]: pgmap v5852: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:32:07.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:06 smithi138 ceph-mon[36954]: pgmap v5852: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:32:09.243 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:08 smithi138 ceph-mon[36954]: pgmap v5853: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:32:09.243 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:08 smithi138 ceph-mon[36954]: osdmap e412: 12 total, 12 up, 12 in 2024-07-26T16:32:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:08 smithi100 ceph-mon[32599]: pgmap v5853: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:32:09.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:08 smithi100 ceph-mon[32599]: osdmap e412: 12 total, 12 up, 12 in 2024-07-26T16:32:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:08 smithi167 ceph-mon[37140]: pgmap v5853: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:32:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:08 smithi167 ceph-mon[37140]: osdmap e412: 12 total, 12 up, 12 in 2024-07-26T16:32:11.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:10 smithi100 ceph-mon[32599]: pgmap v5855: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:32:11.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:10 smithi167 ceph-mon[37140]: pgmap v5855: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:32:11.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:10 smithi138 ceph-mon[36954]: pgmap v5855: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:32:13.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:12 smithi100 ceph-mon[32599]: pgmap v5856: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:32:13.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:12 smithi167 ceph-mon[37140]: pgmap v5856: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:32:13.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:12 smithi138 ceph-mon[36954]: pgmap v5856: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:32:15.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:14 smithi100 ceph-mon[32599]: pgmap v5857: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:32:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:14 smithi167 ceph-mon[37140]: pgmap v5857: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:32:15.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:14 smithi138 ceph-mon[36954]: pgmap v5857: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:32:17.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:16 smithi100 ceph-mon[32599]: pgmap v5858: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:32:17.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:16 smithi100 ceph-mon[32599]: osdmap e413: 12 total, 12 up, 12 in 2024-07-26T16:32:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:16 smithi167 ceph-mon[37140]: pgmap v5858: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:32:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:16 smithi167 ceph-mon[37140]: osdmap e413: 12 total, 12 up, 12 in 2024-07-26T16:32:17.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:16 smithi138 ceph-mon[36954]: pgmap v5858: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:32:17.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:16 smithi138 ceph-mon[36954]: osdmap e413: 12 total, 12 up, 12 in 2024-07-26T16:32:19.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:18 smithi100 ceph-mon[32599]: pgmap v5860: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:32:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:18 smithi167 ceph-mon[37140]: pgmap v5860: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:32:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:18 smithi138 ceph-mon[36954]: pgmap v5860: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:32:21.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:20 smithi100 ceph-mon[32599]: pgmap v5861: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:32:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:20 smithi167 ceph-mon[37140]: pgmap v5861: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:32:21.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:20 smithi138 ceph-mon[36954]: pgmap v5861: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:32:22.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:32:22.165+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:32:22.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:32:22.165+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:32:22.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:32:22.165+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:32:22.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:32:22.165+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:32:22.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:32:22.165+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:32:23.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:22 smithi100 ceph-mon[32599]: pgmap v5862: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:32:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:22 smithi167 ceph-mon[37140]: pgmap v5862: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:32:23.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:22 smithi138 ceph-mon[36954]: pgmap v5862: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:32:25.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:24 smithi100 ceph-mon[32599]: pgmap v5863: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:32:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:24 smithi167 ceph-mon[37140]: pgmap v5863: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:32:25.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:24 smithi138 ceph-mon[36954]: pgmap v5863: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:32:27.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:26 smithi100 ceph-mon[32599]: pgmap v5864: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:32:27.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:26 smithi167 ceph-mon[37140]: pgmap v5864: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:32:27.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:26 smithi138 ceph-mon[36954]: pgmap v5864: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:32:29.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:28 smithi100 ceph-mon[32599]: pgmap v5865: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:32:29.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:28 smithi167 ceph-mon[37140]: pgmap v5865: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:32:29.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:28 smithi138 ceph-mon[36954]: pgmap v5865: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:32:31.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:30 smithi100 ceph-mon[32599]: pgmap v5866: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:32:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:30 smithi167 ceph-mon[37140]: pgmap v5866: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:32:31.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:30 smithi138 ceph-mon[36954]: pgmap v5866: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:32:32.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:32 smithi138 ceph-mon[36954]: pgmap v5867: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:32:32.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:32 smithi100 ceph-mon[32599]: pgmap v5867: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:32:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:32 smithi167 ceph-mon[37140]: pgmap v5867: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:32:35.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:34 smithi138 ceph-mon[36954]: pgmap v5868: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:32:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:34 smithi100 ceph-mon[32599]: pgmap v5868: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:32:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:34 smithi167 ceph-mon[37140]: pgmap v5868: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:32:37.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:32:37.181+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:32:37.184 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:36 smithi100 ceph-mon[32599]: pgmap v5869: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:32:37.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:32:37.182+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:32:37.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:32:37.182+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:32:37.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:32:37.182+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:32:37.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:32:37.182+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:32:37.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:36 smithi138 ceph-mon[36954]: pgmap v5869: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:32:37.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:36 smithi167 ceph-mon[37140]: pgmap v5869: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:32:39.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:38 smithi138 ceph-mon[36954]: pgmap v5870: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:32:39.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:32:39.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:32:39.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:32:39.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:32:39.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:32:39.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:32:39.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:32:39.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:32:39.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:32:39.193 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:32:39.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:38 smithi100 ceph-mon[32599]: pgmap v5870: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:32:39.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:32:39.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:32:39.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:32:39.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:32:39.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:32:39.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:32:39.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:32:39.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:32:39.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:32:39.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:32:39.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:38 smithi167 ceph-mon[37140]: pgmap v5870: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:32:39.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:32:39.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:32:39.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:32:39.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:32:39.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:32:39.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:32:39.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:32:39.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:32:39.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:32:39.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:32:41.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:40 smithi138 ceph-mon[36954]: pgmap v5871: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:32:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:40 smithi100 ceph-mon[32599]: pgmap v5871: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:32:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:40 smithi167 ceph-mon[37140]: pgmap v5871: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:32:42.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:42 smithi138 ceph-mon[36954]: pgmap v5872: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:32:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:42 smithi100 ceph-mon[32599]: pgmap v5872: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:32:42.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:42 smithi167 ceph-mon[37140]: pgmap v5872: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:32:45.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:44 smithi138 ceph-mon[36954]: pgmap v5873: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:32:45.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:44 smithi100 ceph-mon[32599]: pgmap v5873: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:32:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:44 smithi167 ceph-mon[37140]: pgmap v5873: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:32:47.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:46 smithi138 ceph-mon[36954]: pgmap v5874: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:32:47.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:46 smithi100 ceph-mon[32599]: pgmap v5874: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:32:47.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:46 smithi167 ceph-mon[37140]: pgmap v5874: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:32:49.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:48 smithi138 ceph-mon[36954]: pgmap v5875: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:32:49.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:48 smithi100 ceph-mon[32599]: pgmap v5875: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:32:49.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:48 smithi167 ceph-mon[37140]: pgmap v5875: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:32:51.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:50 smithi138 ceph-mon[36954]: pgmap v5876: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:32:51.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:50 smithi100 ceph-mon[32599]: pgmap v5876: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:32:51.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:50 smithi167 ceph-mon[37140]: pgmap v5876: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:32:52.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:32:52.194+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:32:52.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:32:52.194+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:32:52.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:32:52.194+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:32:52.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:32:52.194+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:32:52.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:32:52.194+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:32:53.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:52 smithi138 ceph-mon[36954]: pgmap v5877: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:32:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:52 smithi100 ceph-mon[32599]: pgmap v5877: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:32:53.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:52 smithi167 ceph-mon[37140]: pgmap v5877: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:32:55.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:54 smithi138 ceph-mon[36954]: pgmap v5878: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:32:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:54 smithi100 ceph-mon[32599]: pgmap v5878: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:32:55.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:54 smithi167 ceph-mon[37140]: pgmap v5878: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:32:57.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:56 smithi138 ceph-mon[36954]: pgmap v5879: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:32:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:56 smithi100 ceph-mon[32599]: pgmap v5879: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:32:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:56 smithi167 ceph-mon[37140]: pgmap v5879: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:32:59.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:32:58 smithi138 ceph-mon[36954]: pgmap v5880: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:32:59.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:32:58 smithi100 ceph-mon[32599]: pgmap v5880: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:32:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:32:58 smithi167 ceph-mon[37140]: pgmap v5880: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:33:01.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:00 smithi100 ceph-mon[32599]: pgmap v5881: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:33:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:00 smithi167 ceph-mon[37140]: pgmap v5881: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:33:01.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:00 smithi138 ceph-mon[36954]: pgmap v5881: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:33:03.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:33:03.037+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b4ef0 msgr2=0x7fe5940b5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-07-26T16:33:03.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:33:03.037+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b4ef0 msgr2=0x7fe5940b5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-07-26T16:33:03.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:33:03.037+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b4ef0 0x7fe5940b5510 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fe590004ee0 tx=0x7fe59008a040 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:33:03.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:33:03.037+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b4ef0 0x7fe5940b5510 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fe590004ee0 tx=0x7fe59008a040 comp rx=0 tx=0).stop 2024-07-26T16:33:03.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:33:03.037+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b4ef0 msgr2=0x7fe5940b5510 unknown :-1 s=STATE_CLOSED l=1).tick see no progress in more than 10000000 us during connecting to v2:172.21.15.100:6800/3937009534, fault. 2024-07-26T16:33:03.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:33:03.037+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b4ef0 msgr2=0x7fe5940b5510 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-07-26T16:33:03.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:33:03.037+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b4ef0 0x7fe5940b5510 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T16:33:03.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:33:03.037+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T16:33:03.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:33:03.037+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:33:03.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:33:03.037+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe59000b6b0 tx=0x7fe59002f000 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T16:33:03.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:02 smithi100 ceph-mon[32599]: pgmap v5882: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:33:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:02 smithi167 ceph-mon[37140]: pgmap v5882: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:33:03.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:02 smithi138 ceph-mon[36954]: pgmap v5882: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:33:04.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:33:04.415+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T16:33:04.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:33:04.415+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T16:33:04.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:33:04.415+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 79 ==== auth_reply(proto 2 0 (0) Success) ==== 783+0+0 (secure 0 0 0) 0x7fe5a008c020 con 0x7fe5a81087a0 2024-07-26T16:33:04.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:33:04.415+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c004740 con 0x7fe5a81087a0 2024-07-26T16:33:04.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:33:04.415+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 79 ==== auth_reply(proto 2 0 (0) Success) ==== 612+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T16:33:04.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:33:04.415+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T16:33:04.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:33:04.416+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 80 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c004740 con 0x7fe5a81087a0 2024-07-26T16:33:04.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:33:04.416+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 80 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T16:33:05.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:04 smithi100 ceph-mon[32599]: pgmap v5883: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T16:33:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:04 smithi167 ceph-mon[37140]: pgmap v5883: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T16:33:05.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:04 smithi138 ceph-mon[36954]: pgmap v5883: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T16:33:07.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:33:07.199+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:33:07.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:06 smithi100 ceph-mon[32599]: pgmap v5884: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:33:07.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:33:07.199+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:33:07.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:33:07.199+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:33:07.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:33:07.199+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:33:07.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:33:07.199+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:33:07.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:06 smithi167 ceph-mon[37140]: pgmap v5884: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:33:07.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:06 smithi138 ceph-mon[36954]: pgmap v5884: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:33:09.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:08 smithi100 ceph-mon[32599]: pgmap v5885: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:33:09.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:08 smithi100 ceph-mon[32599]: osdmap e414: 12 total, 12 up, 12 in 2024-07-26T16:33:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:08 smithi167 ceph-mon[37140]: pgmap v5885: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:33:09.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:08 smithi167 ceph-mon[37140]: osdmap e414: 12 total, 12 up, 12 in 2024-07-26T16:33:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:08 smithi138 ceph-mon[36954]: pgmap v5885: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:33:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:08 smithi138 ceph-mon[36954]: osdmap e414: 12 total, 12 up, 12 in 2024-07-26T16:33:11.243 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:10 smithi138 ceph-mon[36954]: pgmap v5887: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:33:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:10 smithi100 ceph-mon[32599]: pgmap v5887: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:33:11.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:10 smithi167 ceph-mon[37140]: pgmap v5887: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:33:13.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:12 smithi100 ceph-mon[32599]: pgmap v5888: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:33:13.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:12 smithi167 ceph-mon[37140]: pgmap v5888: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:33:13.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:12 smithi138 ceph-mon[36954]: pgmap v5888: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:33:15.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:14 smithi100 ceph-mon[32599]: pgmap v5889: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:33:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:14 smithi167 ceph-mon[37140]: pgmap v5889: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:33:15.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:14 smithi138 ceph-mon[36954]: pgmap v5889: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:33:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:16 smithi100 ceph-mon[32599]: pgmap v5890: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:33:17.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:16 smithi100 ceph-mon[32599]: osdmap e415: 12 total, 12 up, 12 in 2024-07-26T16:33:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:16 smithi167 ceph-mon[37140]: pgmap v5890: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:33:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:16 smithi167 ceph-mon[37140]: osdmap e415: 12 total, 12 up, 12 in 2024-07-26T16:33:17.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:16 smithi138 ceph-mon[36954]: pgmap v5890: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:33:17.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:16 smithi138 ceph-mon[36954]: osdmap e415: 12 total, 12 up, 12 in 2024-07-26T16:33:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:18 smithi100 ceph-mon[32599]: pgmap v5892: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:33:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:18 smithi167 ceph-mon[37140]: pgmap v5892: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:33:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:18 smithi138 ceph-mon[36954]: pgmap v5892: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:33:21.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:20 smithi100 ceph-mon[32599]: pgmap v5893: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:33:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:20 smithi167 ceph-mon[37140]: pgmap v5893: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:33:21.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:20 smithi138 ceph-mon[36954]: pgmap v5893: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:33:22.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:33:22.203+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:33:22.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:33:22.204+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:33:22.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:33:22.204+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:33:22.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:33:22.204+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:33:22.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:33:22.204+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:33:23.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:22 smithi100 ceph-mon[32599]: pgmap v5894: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:33:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:22 smithi167 ceph-mon[37140]: pgmap v5894: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:33:23.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:22 smithi138 ceph-mon[36954]: pgmap v5894: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:33:25.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:24 smithi100 ceph-mon[32599]: pgmap v5895: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:33:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:24 smithi167 ceph-mon[37140]: pgmap v5895: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:33:25.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:24 smithi138 ceph-mon[36954]: pgmap v5895: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:33:27.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:26 smithi100 ceph-mon[32599]: pgmap v5896: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:33:27.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:26 smithi167 ceph-mon[37140]: pgmap v5896: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:33:27.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:26 smithi138 ceph-mon[36954]: pgmap v5896: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:33:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:28 smithi100 ceph-mon[32599]: pgmap v5897: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:33:29.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:28 smithi167 ceph-mon[37140]: pgmap v5897: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:33:29.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:28 smithi138 ceph-mon[36954]: pgmap v5897: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:33:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:30 smithi167 ceph-mon[37140]: pgmap v5898: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:33:31.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:30 smithi138 ceph-mon[36954]: pgmap v5898: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:33:31.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:30 smithi100 ceph-mon[32599]: pgmap v5898: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:33:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:32 smithi167 ceph-mon[37140]: pgmap v5899: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:33:33.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:32 smithi138 ceph-mon[36954]: pgmap v5899: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:33:33.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:32 smithi100 ceph-mon[32599]: pgmap v5899: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:33:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:35 smithi167 ceph-mon[37140]: pgmap v5900: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:33:35.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:35 smithi138 ceph-mon[36954]: pgmap v5900: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:33:35.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:35 smithi100 ceph-mon[32599]: pgmap v5900: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:33:37.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:33:37.212+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:33:37.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:33:37.213+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:33:37.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:33:37.213+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:33:37.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:33:37.213+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:33:37.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:33:37.213+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:33:37.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:37 smithi167 ceph-mon[37140]: pgmap v5901: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:33:37.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:37 smithi138 ceph-mon[36954]: pgmap v5901: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:33:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:37 smithi100 ceph-mon[32599]: pgmap v5901: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:33:39.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:39 smithi167 ceph-mon[37140]: pgmap v5902: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:33:39.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:39 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:33:39.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:39 smithi138 ceph-mon[36954]: pgmap v5902: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:33:39.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:39 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:33:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:39 smithi100 ceph-mon[32599]: pgmap v5902: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:33:39.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:39 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:33:40.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:33:40.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:33:40.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:33:40.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:33:40.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:33:40.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:33:40.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:33:40.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:33:40.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:33:40.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:40 smithi167 ceph-mon[37140]: pgmap v5903: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:33:40.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:33:40.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:33:40.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:33:40.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:33:40.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:33:40.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:33:40.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:33:40.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:33:40.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:33:40.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:40 smithi138 ceph-mon[36954]: pgmap v5903: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:33:40.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:33:40.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:33:40.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:33:40.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:33:40.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:33:40.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:33:40.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:33:40.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:33:40.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:33:40.997 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:40 smithi100 ceph-mon[32599]: pgmap v5903: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:33:42.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:42 smithi138 ceph-mon[36954]: pgmap v5904: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:33:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:42 smithi100 ceph-mon[32599]: pgmap v5904: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:33:42.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:42 smithi167 ceph-mon[37140]: pgmap v5904: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:33:45.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:44 smithi138 ceph-mon[36954]: pgmap v5905: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:33:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:44 smithi100 ceph-mon[32599]: pgmap v5905: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:33:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:44 smithi167 ceph-mon[37140]: pgmap v5905: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:33:47.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:46 smithi138 ceph-mon[36954]: pgmap v5906: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:33:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:46 smithi100 ceph-mon[32599]: pgmap v5906: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:33:47.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:46 smithi167 ceph-mon[37140]: pgmap v5906: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:33:49.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:48 smithi138 ceph-mon[36954]: pgmap v5907: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:33:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:48 smithi100 ceph-mon[32599]: pgmap v5907: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:33:49.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:48 smithi167 ceph-mon[37140]: pgmap v5907: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:33:51.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:50 smithi100 ceph-mon[32599]: pgmap v5908: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:33:51.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:50 smithi167 ceph-mon[37140]: pgmap v5908: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:33:51.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:50 smithi138 ceph-mon[36954]: pgmap v5908: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:33:52.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:33:52.228+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:33:52.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:33:52.229+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:33:52.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:33:52.229+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:33:52.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:33:52.229+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:33:52.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:33:52.229+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:33:53.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:52 smithi100 ceph-mon[32599]: pgmap v5909: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:33:53.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:52 smithi167 ceph-mon[37140]: pgmap v5909: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:33:53.393 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:52 smithi138 ceph-mon[36954]: pgmap v5909: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:33:55.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:54 smithi100 ceph-mon[32599]: pgmap v5910: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:33:55.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:54 smithi167 ceph-mon[37140]: pgmap v5910: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:33:55.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:54 smithi138 ceph-mon[36954]: pgmap v5910: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T16:33:57.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:56 smithi100 ceph-mon[32599]: pgmap v5911: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:33:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:56 smithi167 ceph-mon[37140]: pgmap v5911: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:33:57.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:56 smithi138 ceph-mon[36954]: pgmap v5911: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:33:59.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:33:58 smithi100 ceph-mon[32599]: pgmap v5912: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:33:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:33:58 smithi167 ceph-mon[37140]: pgmap v5912: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:33:59.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:33:58 smithi138 ceph-mon[36954]: pgmap v5912: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:34:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:34:00 smithi100 ceph-mon[32599]: pgmap v5913: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:34:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:34:00 smithi167 ceph-mon[37140]: pgmap v5913: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:34:01.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:34:00 smithi138 ceph-mon[36954]: pgmap v5913: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:34:03.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:34:02 smithi100 ceph-mon[32599]: pgmap v5914: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:34:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:34:02 smithi167 ceph-mon[37140]: pgmap v5914: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:34:03.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:34:02 smithi138 ceph-mon[36954]: pgmap v5914: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:34:05.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:34:04 smithi100 ceph-mon[32599]: pgmap v5915: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T16:34:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:34:04 smithi167 ceph-mon[37140]: pgmap v5915: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T16:34:05.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:34:04 smithi138 ceph-mon[36954]: pgmap v5915: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T16:34:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:34:06 smithi100 ceph-mon[32599]: pgmap v5916: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:34:07.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:34:07.244+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:34:07.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:34:07.245+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:34:07.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:34:07.245+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:34:07.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:34:07.245+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:34:07.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:34:07.245+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:34:07.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:34:06 smithi167 ceph-mon[37140]: pgmap v5916: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:34:07.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:34:06 smithi138 ceph-mon[36954]: pgmap v5916: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:34:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:34:08 smithi100 ceph-mon[32599]: pgmap v5917: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:34:09.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:34:08 smithi100 ceph-mon[32599]: osdmap e416: 12 total, 12 up, 12 in 2024-07-26T16:34:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:34:08 smithi167 ceph-mon[37140]: pgmap v5917: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:34:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:34:08 smithi167 ceph-mon[37140]: osdmap e416: 12 total, 12 up, 12 in 2024-07-26T16:34:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:34:08 smithi138 ceph-mon[36954]: pgmap v5917: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:34:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:34:08 smithi138 ceph-mon[36954]: osdmap e416: 12 total, 12 up, 12 in 2024-07-26T16:34:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:34:10 smithi100 ceph-mon[32599]: pgmap v5919: 97 pgs: 3 active+clean+snaptrim_wait, 1 active+clean+snaptrim, 93 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:34:11.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:34:10 smithi167 ceph-mon[37140]: pgmap v5919: 97 pgs: 3 active+clean+snaptrim_wait, 1 active+clean+snaptrim, 93 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:34:11.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:34:10 smithi138 ceph-mon[36954]: pgmap v5919: 97 pgs: 3 active+clean+snaptrim_wait, 1 active+clean+snaptrim, 93 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:34:13.243 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:34:12 smithi138 ceph-mon[36954]: pgmap v5920: 97 pgs: 3 active+clean+snaptrim_wait, 1 active+clean+snaptrim, 93 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:34:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:34:12 smithi100 ceph-mon[32599]: pgmap v5920: 97 pgs: 3 active+clean+snaptrim_wait, 1 active+clean+snaptrim, 93 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:34:13.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:34:12 smithi167 ceph-mon[37140]: pgmap v5920: 97 pgs: 3 active+clean+snaptrim_wait, 1 active+clean+snaptrim, 93 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:34:15.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:34:14 smithi100 ceph-mon[32599]: pgmap v5921: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:34:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:34:14 smithi167 ceph-mon[37140]: pgmap v5921: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:34:15.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:34:14 smithi138 ceph-mon[36954]: pgmap v5921: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:34:17.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:34:16 smithi100 ceph-mon[32599]: pgmap v5922: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:34:17.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:34:16 smithi100 ceph-mon[32599]: osdmap e417: 12 total, 12 up, 12 in 2024-07-26T16:34:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:34:16 smithi167 ceph-mon[37140]: pgmap v5922: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:34:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:34:16 smithi167 ceph-mon[37140]: osdmap e417: 12 total, 12 up, 12 in 2024-07-26T16:34:17.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:34:16 smithi138 ceph-mon[36954]: pgmap v5922: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:34:17.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:34:16 smithi138 ceph-mon[36954]: osdmap e417: 12 total, 12 up, 12 in 2024-07-26T16:34:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:34:18 smithi100 ceph-mon[32599]: pgmap v5924: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:34:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:34:18 smithi167 ceph-mon[37140]: pgmap v5924: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:34:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:34:18 smithi138 ceph-mon[36954]: pgmap v5924: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:34:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:34:20 smithi167 ceph-mon[37140]: pgmap v5925: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:34:21.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:34:20 smithi138 ceph-mon[36954]: pgmap v5925: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:34:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:34:20 smithi100 ceph-mon[32599]: pgmap v5925: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:34:22.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:34:22.260+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:34:22.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:34:22.260+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:34:22.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:34:22.260+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:34:22.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:34:22.260+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:34:22.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:34:22.260+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:34:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:34:22 smithi167 ceph-mon[37140]: pgmap v5926: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:34:23.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:34:22 smithi138 ceph-mon[36954]: pgmap v5926: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:34:23.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:34:22 smithi100 ceph-mon[32599]: pgmap v5926: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:34:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:34:25 smithi167 ceph-mon[37140]: pgmap v5927: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:34:25.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:34:25 smithi138 ceph-mon[36954]: pgmap v5927: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:34:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:34:25 smithi100 ceph-mon[32599]: pgmap v5927: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:34:27.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:34:27 smithi167 ceph-mon[37140]: pgmap v5928: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:34:27.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:34:27 smithi138 ceph-mon[36954]: pgmap v5928: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:34:27.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:34:27 smithi100 ceph-mon[32599]: pgmap v5928: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:34:29.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:34:29 smithi167 ceph-mon[37140]: pgmap v5929: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:34:29.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:34:29 smithi138 ceph-mon[36954]: pgmap v5929: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:34:29.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:34:29 smithi100 ceph-mon[32599]: pgmap v5929: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:34:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:34:31 smithi167 ceph-mon[37140]: pgmap v5930: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:34:31.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:34:31 smithi138 ceph-mon[36954]: pgmap v5930: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:34:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:34:31 smithi100 ceph-mon[32599]: pgmap v5930: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:34:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:34:33 smithi167 ceph-mon[37140]: pgmap v5931: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:34:33.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:34:33 smithi138 ceph-mon[36954]: pgmap v5931: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:34:33.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:34:33 smithi100 ceph-mon[32599]: pgmap v5931: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:34:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:34:35 smithi167 ceph-mon[37140]: pgmap v5932: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:34:35.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:34:35 smithi138 ceph-mon[36954]: pgmap v5932: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:34:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:34:35 smithi100 ceph-mon[32599]: pgmap v5932: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:34:37.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:34:37 smithi167 ceph-mon[37140]: pgmap v5933: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:34:37.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:34:37.277+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:34:37.279 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:34:37 smithi100 ceph-mon[32599]: pgmap v5933: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:34:37.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:34:37.277+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:34:37.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:34:37.277+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:34:37.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:34:37.277+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:34:37.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:34:37.277+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:34:37.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:34:37 smithi138 ceph-mon[36954]: pgmap v5933: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:34:39.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:34:39 smithi138 ceph-mon[36954]: pgmap v5934: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:34:39.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:34:39 smithi100 ceph-mon[32599]: pgmap v5934: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:34:39.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:34:39 smithi167 ceph-mon[37140]: pgmap v5934: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:34:40.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:34:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:34:40.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:34:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:34:40.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:34:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:34:41.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:34:41 smithi138 ceph-mon[36954]: pgmap v5935: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:34:41.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:34:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:34:41.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:34:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:34:41.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:34:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:34:41.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:34:41 smithi100 ceph-mon[32599]: pgmap v5935: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:34:41.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:34:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:34:41.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:34:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:34:41.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:34:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:34:41.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:34:41 smithi167 ceph-mon[37140]: pgmap v5935: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:34:41.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:34:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:34:41.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:34:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:34:41.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:34:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:34:43.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:34:43 smithi138 ceph-mon[36954]: pgmap v5936: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:34:43.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:34:43 smithi100 ceph-mon[32599]: pgmap v5936: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:34:43.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:34:43 smithi167 ceph-mon[37140]: pgmap v5936: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:34:45.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:34:45 smithi138 ceph-mon[36954]: pgmap v5937: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:34:45.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:34:45 smithi100 ceph-mon[32599]: pgmap v5937: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:34:45.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:34:45 smithi167 ceph-mon[37140]: pgmap v5937: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:34:47.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:34:47 smithi138 ceph-mon[36954]: pgmap v5938: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:34:47.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:34:47 smithi100 ceph-mon[32599]: pgmap v5938: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:34:47.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:34:47 smithi167 ceph-mon[37140]: pgmap v5938: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:34:49.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:34:49 smithi138 ceph-mon[36954]: pgmap v5939: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:34:49.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:34:49 smithi100 ceph-mon[32599]: pgmap v5939: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:34:49.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:34:49 smithi167 ceph-mon[37140]: pgmap v5939: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:34:51.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:34:51 smithi100 ceph-mon[32599]: pgmap v5940: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:34:51.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:34:51 smithi167 ceph-mon[37140]: pgmap v5940: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:34:51.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:34:51 smithi138 ceph-mon[36954]: pgmap v5940: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:34:52.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:34:52.293+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:34:52.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:34:52.293+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:34:52.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:34:52.293+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:34:52.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:34:52.293+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:34:52.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:34:52.293+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:34:53.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:34:53 smithi138 ceph-mon[36954]: pgmap v5941: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:34:53.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:34:53 smithi100 ceph-mon[32599]: pgmap v5941: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:34:53.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:34:53 smithi167 ceph-mon[37140]: pgmap v5941: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:34:55.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:34:55 smithi138 ceph-mon[36954]: pgmap v5942: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:34:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:34:55 smithi100 ceph-mon[32599]: pgmap v5942: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:34:55.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:34:55 smithi167 ceph-mon[37140]: pgmap v5942: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:34:57.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:34:57 smithi138 ceph-mon[36954]: pgmap v5943: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:34:57.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:34:57 smithi100 ceph-mon[32599]: pgmap v5943: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:34:57.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:34:57 smithi167 ceph-mon[37140]: pgmap v5943: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:34:59.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:34:59 smithi138 ceph-mon[36954]: pgmap v5944: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:34:59.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:34:59 smithi100 ceph-mon[32599]: pgmap v5944: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:34:59.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:34:59 smithi167 ceph-mon[37140]: pgmap v5944: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:35:01.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:35:01 smithi138 ceph-mon[36954]: pgmap v5945: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:35:01.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:35:01 smithi100 ceph-mon[32599]: pgmap v5945: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:35:01.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:35:01 smithi167 ceph-mon[37140]: pgmap v5945: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:35:03.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:35:03 smithi138 ceph-mon[36954]: pgmap v5946: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:35:03.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:35:03 smithi100 ceph-mon[32599]: pgmap v5946: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:35:03.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:35:03 smithi167 ceph-mon[37140]: pgmap v5946: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:35:05.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:35:05 smithi138 ceph-mon[36954]: pgmap v5947: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:35:05.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:35:05 smithi100 ceph-mon[32599]: pgmap v5947: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:35:05.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:35:05 smithi167 ceph-mon[37140]: pgmap v5947: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:35:07.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:35:07.303+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:35:07.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:35:07.303+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:35:07.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:35:07.304+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:35:07.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:35:07.304+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:35:07.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:35:07.304+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:35:07.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:35:07 smithi138 ceph-mon[36954]: pgmap v5948: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:35:07.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:35:07 smithi100 ceph-mon[32599]: pgmap v5948: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:35:07.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:35:07 smithi167 ceph-mon[37140]: pgmap v5948: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:35:08.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:35:08 smithi138 ceph-mon[36954]: osdmap e418: 12 total, 12 up, 12 in 2024-07-26T16:35:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:35:08 smithi100 ceph-mon[32599]: osdmap e418: 12 total, 12 up, 12 in 2024-07-26T16:35:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:35:08 smithi167 ceph-mon[37140]: osdmap e418: 12 total, 12 up, 12 in 2024-07-26T16:35:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:35:09 smithi138 ceph-mon[36954]: pgmap v5950: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:35:09.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:35:09 smithi100 ceph-mon[32599]: pgmap v5950: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:35:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:35:09 smithi167 ceph-mon[37140]: pgmap v5950: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:35:11.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:35:11 smithi138 ceph-mon[36954]: pgmap v5951: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:35:11.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:35:11 smithi100 ceph-mon[32599]: pgmap v5951: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:35:11.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:35:11 smithi167 ceph-mon[37140]: pgmap v5951: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:35:13.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:35:13 smithi138 ceph-mon[36954]: pgmap v5952: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:35:13.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:35:13 smithi100 ceph-mon[32599]: pgmap v5952: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:35:13.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:35:13 smithi167 ceph-mon[37140]: pgmap v5952: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:35:15.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:35:15 smithi138 ceph-mon[36954]: pgmap v5953: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:35:15.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:35:15 smithi100 ceph-mon[32599]: pgmap v5953: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:35:15.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:35:15 smithi167 ceph-mon[37140]: pgmap v5953: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:35:17.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:35:17 smithi138 ceph-mon[36954]: pgmap v5954: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:35:17.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:35:17 smithi138 ceph-mon[36954]: osdmap e419: 12 total, 12 up, 12 in 2024-07-26T16:35:17.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:35:17 smithi100 ceph-mon[32599]: pgmap v5954: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:35:17.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:35:17 smithi100 ceph-mon[32599]: osdmap e419: 12 total, 12 up, 12 in 2024-07-26T16:35:17.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:35:17 smithi167 ceph-mon[37140]: pgmap v5954: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:35:17.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:35:17 smithi167 ceph-mon[37140]: osdmap e419: 12 total, 12 up, 12 in 2024-07-26T16:35:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:35:19 smithi138 ceph-mon[36954]: pgmap v5956: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:35:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:35:19 smithi100 ceph-mon[32599]: pgmap v5956: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:35:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:35:19 smithi167 ceph-mon[37140]: pgmap v5956: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:35:21.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:35:21 smithi138 ceph-mon[36954]: pgmap v5957: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:35:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:35:21 smithi100 ceph-mon[32599]: pgmap v5957: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:35:21.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:35:21 smithi167 ceph-mon[37140]: pgmap v5957: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:35:22.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:35:22.305+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:35:22.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:35:22.306+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:35:22.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:35:22.306+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:35:22.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:35:22.306+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:35:22.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:35:22.306+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:35:23.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:35:23 smithi138 ceph-mon[36954]: pgmap v5958: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:35:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:35:23 smithi100 ceph-mon[32599]: pgmap v5958: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:35:23.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:35:23 smithi167 ceph-mon[37140]: pgmap v5958: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:35:25.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:35:25 smithi138 ceph-mon[36954]: pgmap v5959: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:35:25.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:35:25 smithi100 ceph-mon[32599]: pgmap v5959: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:35:25.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:35:25 smithi167 ceph-mon[37140]: pgmap v5959: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:35:27.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:35:27 smithi138 ceph-mon[36954]: pgmap v5960: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:35:27.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:35:27 smithi100 ceph-mon[32599]: pgmap v5960: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:35:27.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:35:27 smithi167 ceph-mon[37140]: pgmap v5960: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:35:29.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:35:29 smithi138 ceph-mon[36954]: pgmap v5961: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:35:29.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:35:29 smithi100 ceph-mon[32599]: pgmap v5961: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:35:29.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:35:29 smithi167 ceph-mon[37140]: pgmap v5961: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:35:31.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:35:31 smithi138 ceph-mon[36954]: pgmap v5962: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:35:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:35:31 smithi100 ceph-mon[32599]: pgmap v5962: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:35:31.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:35:31 smithi167 ceph-mon[37140]: pgmap v5962: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:35:33.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:35:33 smithi138 ceph-mon[36954]: pgmap v5963: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:35:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:35:33 smithi100 ceph-mon[32599]: pgmap v5963: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:35:33.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:35:33 smithi167 ceph-mon[37140]: pgmap v5963: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:35:35.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:35:35 smithi138 ceph-mon[36954]: pgmap v5964: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:35:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:35:35 smithi100 ceph-mon[32599]: pgmap v5964: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:35:35.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:35:35 smithi167 ceph-mon[37140]: pgmap v5964: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:35:37.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:35:37.318+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:35:37.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:35:37.319+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:35:37.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:35:37.319+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:35:37.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:35:37.319+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:35:37.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:35:37.319+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:35:37.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:35:37 smithi138 ceph-mon[36954]: pgmap v5965: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-07-26T16:35:37.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:35:37 smithi100 ceph-mon[32599]: pgmap v5965: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-07-26T16:35:37.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:35:37 smithi167 ceph-mon[37140]: pgmap v5965: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-07-26T16:35:39.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:35:39 smithi138 ceph-mon[36954]: pgmap v5966: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T16:35:39.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:35:39 smithi100 ceph-mon[32599]: pgmap v5966: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T16:35:39.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:35:39 smithi167 ceph-mon[37140]: pgmap v5966: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T16:35:41.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:35:41 smithi138 ceph-mon[36954]: pgmap v5967: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-07-26T16:35:41.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:35:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:35:41.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:35:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:35:41.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:35:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:35:41.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:35:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:35:41.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:35:41 smithi100 ceph-mon[32599]: pgmap v5967: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-07-26T16:35:41.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:35:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:35:41.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:35:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:35:41.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:35:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:35:41.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:35:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:35:41.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:35:41 smithi167 ceph-mon[37140]: pgmap v5967: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-07-26T16:35:41.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:35:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:35:41.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:35:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:35:41.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:35:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:35:41.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:35:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:35:43.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:35:43 smithi138 ceph-mon[36954]: pgmap v5968: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T16:35:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:35:43 smithi100 ceph-mon[32599]: pgmap v5968: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T16:35:43.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:35:43 smithi167 ceph-mon[37140]: pgmap v5968: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T16:35:45.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:35:45 smithi138 ceph-mon[36954]: pgmap v5969: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:35:45.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:35:45 smithi100 ceph-mon[32599]: pgmap v5969: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:35:45.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:35:45 smithi167 ceph-mon[37140]: pgmap v5969: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:35:47.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:35:47 smithi138 ceph-mon[36954]: pgmap v5970: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:35:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:35:47 smithi100 ceph-mon[32599]: pgmap v5970: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:35:47.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:35:47 smithi167 ceph-mon[37140]: pgmap v5970: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:35:49.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:35:49 smithi138 ceph-mon[36954]: pgmap v5971: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:35:49.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:35:49 smithi100 ceph-mon[32599]: pgmap v5971: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:35:49.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:35:49 smithi167 ceph-mon[37140]: pgmap v5971: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:35:51.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:35:51 smithi138 ceph-mon[36954]: pgmap v5972: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:35:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:35:51 smithi100 ceph-mon[32599]: pgmap v5972: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:35:51.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:35:51 smithi167 ceph-mon[37140]: pgmap v5972: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:35:52.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:35:52.335+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:35:52.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:35:52.336+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:35:52.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:35:52.336+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:35:52.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:35:52.336+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:35:52.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:35:52.336+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:35:53.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:35:53 smithi138 ceph-mon[36954]: pgmap v5973: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:35:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:35:53 smithi100 ceph-mon[32599]: pgmap v5973: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:35:53.526 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:35:53 smithi167 ceph-mon[37140]: pgmap v5973: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:35:55.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:35:55 smithi138 ceph-mon[36954]: pgmap v5974: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:35:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:35:55 smithi100 ceph-mon[32599]: pgmap v5974: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:35:55.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:35:55 smithi167 ceph-mon[37140]: pgmap v5974: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:35:57.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:35:57 smithi138 ceph-mon[36954]: pgmap v5975: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:35:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:35:57 smithi100 ceph-mon[32599]: pgmap v5975: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:35:57.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:35:57 smithi167 ceph-mon[37140]: pgmap v5975: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:35:59.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:35:59 smithi138 ceph-mon[36954]: pgmap v5976: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:35:59.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:35:59 smithi100 ceph-mon[32599]: pgmap v5976: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:35:59.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:35:59 smithi167 ceph-mon[37140]: pgmap v5976: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:36:01.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:36:01 smithi138 ceph-mon[36954]: pgmap v5977: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:36:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:36:01 smithi100 ceph-mon[32599]: pgmap v5977: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:36:01.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:36:01 smithi167 ceph-mon[37140]: pgmap v5977: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:36:02.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:36:02 smithi100 ceph-mon[32599]: pgmap v5978: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:36:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:36:02 smithi167 ceph-mon[37140]: pgmap v5978: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:36:02.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:36:02 smithi138 ceph-mon[36954]: pgmap v5978: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:36:04.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:36:04.443+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c004740 con 0x7fe5a81087a0 2024-07-26T16:36:04.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:36:04.443+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T16:36:04.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:36:04.443+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 81 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c004740 con 0x7fe5a81087a0 2024-07-26T16:36:04.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:36:04.443+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 81 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T16:36:05.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:36:04 smithi100 ceph-mon[32599]: pgmap v5979: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:36:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:36:04 smithi167 ceph-mon[37140]: pgmap v5979: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:36:05.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:36:04 smithi138 ceph-mon[36954]: pgmap v5979: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:36:07.247 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:36:06 smithi100 ceph-mon[32599]: pgmap v5980: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:36:07.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:36:06 smithi167 ceph-mon[37140]: pgmap v5980: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:36:07.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:36:07.352+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:36:07.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:36:07.352+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:36:07.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:36:07.352+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:36:07.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:36:07.352+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:36:07.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:36:07.352+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:36:07.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:36:06 smithi138 ceph-mon[36954]: pgmap v5980: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:36:09.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:36:08 smithi100 ceph-mon[32599]: pgmap v5981: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:36:09.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:36:08 smithi100 ceph-mon[32599]: osdmap e420: 12 total, 12 up, 12 in 2024-07-26T16:36:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:36:08 smithi167 ceph-mon[37140]: pgmap v5981: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:36:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:36:08 smithi167 ceph-mon[37140]: osdmap e420: 12 total, 12 up, 12 in 2024-07-26T16:36:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:36:08 smithi138 ceph-mon[36954]: pgmap v5981: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:36:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:36:08 smithi138 ceph-mon[36954]: osdmap e420: 12 total, 12 up, 12 in 2024-07-26T16:36:11.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:36:10 smithi100 ceph-mon[32599]: pgmap v5983: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:36:11.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:36:10 smithi167 ceph-mon[37140]: pgmap v5983: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:36:11.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:36:10 smithi138 ceph-mon[36954]: pgmap v5983: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:36:13.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:36:12 smithi167 ceph-mon[37140]: pgmap v5984: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:36:13.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:36:12 smithi138 ceph-mon[36954]: pgmap v5984: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:36:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:36:12 smithi100 ceph-mon[32599]: pgmap v5984: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:36:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:36:14 smithi167 ceph-mon[37140]: pgmap v5985: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:36:15.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:36:14 smithi138 ceph-mon[36954]: pgmap v5985: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:36:15.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:36:14 smithi100 ceph-mon[32599]: pgmap v5985: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:36:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:36:17 smithi167 ceph-mon[37140]: pgmap v5986: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:36:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:36:17 smithi167 ceph-mon[37140]: osdmap e421: 12 total, 12 up, 12 in 2024-07-26T16:36:17.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:36:17 smithi138 ceph-mon[36954]: pgmap v5986: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:36:17.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:36:17 smithi138 ceph-mon[36954]: osdmap e421: 12 total, 12 up, 12 in 2024-07-26T16:36:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:36:17 smithi100 ceph-mon[32599]: pgmap v5986: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:36:17.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:36:17 smithi100 ceph-mon[32599]: osdmap e421: 12 total, 12 up, 12 in 2024-07-26T16:36:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:36:19 smithi167 ceph-mon[37140]: pgmap v5988: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:36:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:36:19 smithi138 ceph-mon[36954]: pgmap v5988: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:36:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:36:19 smithi100 ceph-mon[32599]: pgmap v5988: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T16:36:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:36:21 smithi167 ceph-mon[37140]: pgmap v5989: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:36:21.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:36:21 smithi100 ceph-mon[32599]: pgmap v5989: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:36:21.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:36:21 smithi138 ceph-mon[36954]: pgmap v5989: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:36:22.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:36:22.359+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:36:22.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:36:22.359+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:36:22.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:36:22.359+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:36:22.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:36:22.359+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:36:22.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:36:22.359+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:36:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:36:23 smithi167 ceph-mon[37140]: pgmap v5990: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:36:23.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:36:23 smithi138 ceph-mon[36954]: pgmap v5990: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:36:23.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:36:23 smithi100 ceph-mon[32599]: pgmap v5990: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:36:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:36:25 smithi167 ceph-mon[37140]: pgmap v5991: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:36:25.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:36:25 smithi138 ceph-mon[36954]: pgmap v5991: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:36:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:36:25 smithi100 ceph-mon[32599]: pgmap v5991: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:36:27.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:36:27 smithi167 ceph-mon[37140]: pgmap v5992: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:36:27.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:36:27 smithi138 ceph-mon[36954]: pgmap v5992: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:36:27.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:36:27 smithi100 ceph-mon[32599]: pgmap v5992: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:36:29.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:36:29 smithi138 ceph-mon[36954]: pgmap v5993: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:36:29.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:36:29 smithi100 ceph-mon[32599]: pgmap v5993: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:36:29.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:36:29 smithi167 ceph-mon[37140]: pgmap v5993: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:36:31.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:36:31 smithi138 ceph-mon[36954]: pgmap v5994: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:36:31.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:36:31 smithi100 ceph-mon[32599]: pgmap v5994: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:36:31.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:36:31 smithi167 ceph-mon[37140]: pgmap v5994: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:36:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:36:32 smithi100 ceph-mon[32599]: pgmap v5995: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:36:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:36:32 smithi167 ceph-mon[37140]: pgmap v5995: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:36:32.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:36:32 smithi138 ceph-mon[36954]: pgmap v5995: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:36:35.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:36:34 smithi100 ceph-mon[32599]: pgmap v5996: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:36:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:36:34 smithi167 ceph-mon[37140]: pgmap v5996: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:36:35.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:36:34 smithi138 ceph-mon[36954]: pgmap v5996: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:36:37.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:36:36 smithi100 ceph-mon[32599]: pgmap v5997: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:36:37.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:36:36 smithi167 ceph-mon[37140]: pgmap v5997: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:36:37.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:36:37.375+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:36:37.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:36:37.375+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:36:37.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:36:37.375+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:36:37.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:36:37.375+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:36:37.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:36:37.375+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:36:37.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:36:36 smithi138 ceph-mon[36954]: pgmap v5997: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:36:39.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:36:38 smithi167 ceph-mon[37140]: pgmap v5998: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:36:39.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:36:38 smithi138 ceph-mon[36954]: pgmap v5998: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:36:39.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:36:38 smithi100 ceph-mon[32599]: pgmap v5998: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:36:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:36:40 smithi167 ceph-mon[37140]: pgmap v5999: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:36:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:36:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:36:41.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:36:40 smithi138 ceph-mon[36954]: pgmap v5999: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:36:41.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:36:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:36:41.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:36:40 smithi100 ceph-mon[32599]: pgmap v5999: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:36:41.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:36:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:36:42.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:36:42 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:36:42.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:36:42 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:36:42.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:36:42 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:36:42.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:36:42 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:36:42.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:36:42 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:36:42.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:36:42 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:36:42.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:36:42 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:36:42.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:36:42 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:36:42.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:36:42 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:36:43.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:36:43 smithi100 ceph-mon[32599]: pgmap v6000: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:36:43.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:36:43 smithi167 ceph-mon[37140]: pgmap v6000: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:36:43.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:36:43 smithi138 ceph-mon[36954]: pgmap v6000: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:36:45.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:36:45 smithi100 ceph-mon[32599]: pgmap v6001: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:36:45.526 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:36:45 smithi167 ceph-mon[37140]: pgmap v6001: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:36:45.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:36:45 smithi138 ceph-mon[36954]: pgmap v6001: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:36:47.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:36:47 smithi100 ceph-mon[32599]: pgmap v6002: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:36:47.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:36:47 smithi167 ceph-mon[37140]: pgmap v6002: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:36:47.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:36:47 smithi138 ceph-mon[36954]: pgmap v6002: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:36:49.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:36:49 smithi100 ceph-mon[32599]: pgmap v6003: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:36:49.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:36:49 smithi167 ceph-mon[37140]: pgmap v6003: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:36:49.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:36:49 smithi138 ceph-mon[36954]: pgmap v6003: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:36:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:36:51 smithi100 ceph-mon[32599]: pgmap v6004: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:36:51.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:36:51 smithi167 ceph-mon[37140]: pgmap v6004: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:36:51.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:36:51 smithi138 ceph-mon[36954]: pgmap v6004: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:36:52.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:36:52.390+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:36:52.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:36:52.391+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:36:52.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:36:52.391+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:36:52.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:36:52.391+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:36:52.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:36:52.391+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:36:53.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:36:53 smithi100 ceph-mon[32599]: pgmap v6005: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:36:53.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:36:53 smithi167 ceph-mon[37140]: pgmap v6005: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:36:53.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:36:53 smithi138 ceph-mon[36954]: pgmap v6005: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:36:55.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:36:55 smithi100 ceph-mon[32599]: pgmap v6006: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:36:55.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:36:55 smithi167 ceph-mon[37140]: pgmap v6006: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:36:55.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:36:55 smithi138 ceph-mon[36954]: pgmap v6006: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:36:57.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:36:57 smithi100 ceph-mon[32599]: pgmap v6007: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:36:57.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:36:57 smithi167 ceph-mon[37140]: pgmap v6007: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:36:57.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:36:57 smithi138 ceph-mon[36954]: pgmap v6007: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:36:59.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:36:59 smithi100 ceph-mon[32599]: pgmap v6008: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:36:59.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:36:59 smithi167 ceph-mon[37140]: pgmap v6008: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:36:59.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:36:59 smithi138 ceph-mon[36954]: pgmap v6008: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:37:01.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:37:01 smithi100 ceph-mon[32599]: pgmap v6009: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:37:01.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:37:01 smithi167 ceph-mon[37140]: pgmap v6009: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:37:01.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:37:01 smithi138 ceph-mon[36954]: pgmap v6009: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:37:03.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:37:03 smithi100 ceph-mon[32599]: pgmap v6010: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:37:03.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:37:03 smithi167 ceph-mon[37140]: pgmap v6010: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:37:03.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:37:03 smithi138 ceph-mon[36954]: pgmap v6010: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:37:05.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:37:05 smithi100 ceph-mon[32599]: pgmap v6011: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:37:05.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:37:05 smithi167 ceph-mon[37140]: pgmap v6011: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:37:05.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:37:05 smithi138 ceph-mon[36954]: pgmap v6011: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:37:07.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:37:07.394+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:37:07.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:37:07.395+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:37:07.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:37:07.395+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:37:07.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:37:07.395+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:37:07.397 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:37:07.395+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:37:07.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:37:07 smithi100 ceph-mon[32599]: pgmap v6012: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:37:07.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:37:07 smithi167 ceph-mon[37140]: pgmap v6012: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:37:07.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:37:07 smithi138 ceph-mon[36954]: pgmap v6012: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:37:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:37:08 smithi167 ceph-mon[37140]: osdmap e422: 12 total, 12 up, 12 in 2024-07-26T16:37:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:37:08 smithi167 ceph-mon[37140]: pgmap v6014: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:37:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:37:08 smithi138 ceph-mon[36954]: osdmap e422: 12 total, 12 up, 12 in 2024-07-26T16:37:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:37:08 smithi138 ceph-mon[36954]: pgmap v6014: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:37:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:37:08 smithi100 ceph-mon[32599]: osdmap e422: 12 total, 12 up, 12 in 2024-07-26T16:37:08.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:37:08 smithi100 ceph-mon[32599]: pgmap v6014: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:37:11.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:37:11 smithi167 ceph-mon[37140]: pgmap v6015: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:37:11.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:37:11 smithi138 ceph-mon[36954]: pgmap v6015: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:37:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:37:11 smithi100 ceph-mon[32599]: pgmap v6015: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:37:13.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:37:13 smithi167 ceph-mon[37140]: pgmap v6016: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:37:13.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:37:13 smithi138 ceph-mon[36954]: pgmap v6016: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:37:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:37:13 smithi100 ceph-mon[32599]: pgmap v6016: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:37:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:37:15 smithi167 ceph-mon[37140]: pgmap v6017: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:37:15.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:37:15 smithi138 ceph-mon[36954]: pgmap v6017: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:37:15.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:37:15 smithi100 ceph-mon[32599]: pgmap v6017: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:37:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:37:17 smithi167 ceph-mon[37140]: pgmap v6018: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:37:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:37:17 smithi167 ceph-mon[37140]: osdmap e423: 12 total, 12 up, 12 in 2024-07-26T16:37:17.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:37:17 smithi138 ceph-mon[36954]: pgmap v6018: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:37:17.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:37:17 smithi138 ceph-mon[36954]: osdmap e423: 12 total, 12 up, 12 in 2024-07-26T16:37:17.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:37:17 smithi100 ceph-mon[32599]: pgmap v6018: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:37:17.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:37:17 smithi100 ceph-mon[32599]: osdmap e423: 12 total, 12 up, 12 in 2024-07-26T16:37:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:37:19 smithi167 ceph-mon[37140]: pgmap v6020: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:37:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:37:19 smithi138 ceph-mon[36954]: pgmap v6020: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:37:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:37:19 smithi100 ceph-mon[32599]: pgmap v6020: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:37:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:37:21 smithi167 ceph-mon[37140]: pgmap v6021: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:37:21.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:37:21 smithi100 ceph-mon[32599]: pgmap v6021: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:37:21.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:37:21 smithi138 ceph-mon[36954]: pgmap v6021: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:37:22.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:37:22.411+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:37:22.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:37:22.411+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:37:22.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:37:22.411+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:37:22.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:37:22.411+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:37:22.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:37:22.411+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:37:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:37:23 smithi167 ceph-mon[37140]: pgmap v6022: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:37:23.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:37:23 smithi138 ceph-mon[36954]: pgmap v6022: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:37:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:37:23 smithi100 ceph-mon[32599]: pgmap v6022: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:37:25.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:37:25 smithi138 ceph-mon[36954]: pgmap v6023: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:37:25.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:37:25 smithi100 ceph-mon[32599]: pgmap v6023: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:37:25.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:37:25 smithi167 ceph-mon[37140]: pgmap v6023: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:37:27.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:37:27 smithi138 ceph-mon[36954]: pgmap v6024: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:37:27.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:37:27 smithi100 ceph-mon[32599]: pgmap v6024: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:37:27.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:37:27 smithi167 ceph-mon[37140]: pgmap v6024: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:37:29.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:37:29 smithi138 ceph-mon[36954]: pgmap v6025: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:37:29.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:37:29 smithi100 ceph-mon[32599]: pgmap v6025: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:37:29.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:37:29 smithi167 ceph-mon[37140]: pgmap v6025: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:37:31.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:37:31 smithi138 ceph-mon[36954]: pgmap v6026: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:37:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:37:31 smithi100 ceph-mon[32599]: pgmap v6026: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:37:31.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:37:31 smithi167 ceph-mon[37140]: pgmap v6026: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:37:33.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:37:33 smithi138 ceph-mon[36954]: pgmap v6027: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:37:33.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:37:33 smithi100 ceph-mon[32599]: pgmap v6027: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:37:33.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:37:33 smithi167 ceph-mon[37140]: pgmap v6027: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:37:35.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:37:35 smithi138 ceph-mon[36954]: pgmap v6028: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:37:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:37:35 smithi100 ceph-mon[32599]: pgmap v6028: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:37:35.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:37:35 smithi167 ceph-mon[37140]: pgmap v6028: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:37:37.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:37:37.414+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:37:37.417 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:37:37 smithi100 ceph-mon[32599]: pgmap v6029: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:37:37.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:37:37.415+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:37:37.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:37:37.415+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:37:37.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:37:37.415+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:37:37.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:37:37.415+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:37:37.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:37:37 smithi138 ceph-mon[36954]: pgmap v6029: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:37:37.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:37:37 smithi167 ceph-mon[37140]: pgmap v6029: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:37:39.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:37:39 smithi138 ceph-mon[36954]: pgmap v6030: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:37:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:37:39 smithi100 ceph-mon[32599]: pgmap v6030: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:37:39.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:37:39 smithi167 ceph-mon[37140]: pgmap v6030: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:37:41.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:37:41 smithi138 ceph-mon[36954]: pgmap v6031: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:37:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:37:41 smithi100 ceph-mon[32599]: pgmap v6031: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:37:41.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:37:41 smithi167 ceph-mon[37140]: pgmap v6031: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:37:42.370 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:37:42 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:37:42.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:37:42 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:37:42.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:37:42 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:37:43.342 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:37:43 smithi138 ceph-mon[36954]: pgmap v6032: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:37:43.393 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:37:43 smithi100 ceph-mon[32599]: pgmap v6032: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:37:43.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:37:43 smithi167 ceph-mon[37140]: pgmap v6032: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:37:45.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:37:45 smithi100 ceph-mon[32599]: pgmap v6033: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:37:45.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:37:45 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:37:45.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:37:45 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:37:45.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:37:45 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:37:45.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:37:45 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:37:45.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:37:45 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:37:45.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:37:45 smithi138 ceph-mon[36954]: pgmap v6033: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:37:45.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:37:45 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:37:45.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:37:45 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:37:45.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:37:45 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:37:45.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:37:45 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:37:45.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:37:45 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:37:45.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:37:45 smithi167 ceph-mon[37140]: pgmap v6033: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:37:45.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:37:45 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:37:45.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:37:45 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:37:45.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:37:45 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:37:45.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:37:45 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:37:45.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:37:45 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:37:47.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:37:47 smithi138 ceph-mon[36954]: pgmap v6034: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:37:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:37:47 smithi100 ceph-mon[32599]: pgmap v6034: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:37:47.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:37:47 smithi167 ceph-mon[37140]: pgmap v6034: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:37:49.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:37:49 smithi138 ceph-mon[36954]: pgmap v6035: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:37:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:37:49 smithi100 ceph-mon[32599]: pgmap v6035: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:37:49.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:37:49 smithi167 ceph-mon[37140]: pgmap v6035: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:37:51.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:37:51 smithi167 ceph-mon[37140]: pgmap v6036: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:37:51.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:37:51 smithi138 ceph-mon[36954]: pgmap v6036: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:37:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:37:51 smithi100 ceph-mon[32599]: pgmap v6036: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:37:52.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:37:52.419+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:37:52.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:37:52.420+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:37:52.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:37:52.420+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:37:52.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:37:52.420+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:37:52.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:37:52.420+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:37:53.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:37:53 smithi138 ceph-mon[36954]: pgmap v6037: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:37:53.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:37:53 smithi100 ceph-mon[32599]: pgmap v6037: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:37:53.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:37:53 smithi167 ceph-mon[37140]: pgmap v6037: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:37:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:37:55 smithi100 ceph-mon[32599]: pgmap v6038: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:37:55.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:37:55 smithi138 ceph-mon[36954]: pgmap v6038: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:37:55.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:37:55 smithi167 ceph-mon[37140]: pgmap v6038: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:37:57.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:37:57 smithi138 ceph-mon[36954]: pgmap v6039: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:37:57.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:37:57 smithi100 ceph-mon[32599]: pgmap v6039: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:37:57.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:37:57 smithi167 ceph-mon[37140]: pgmap v6039: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:37:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:37:59 smithi138 ceph-mon[36954]: pgmap v6040: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:37:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:37:59 smithi100 ceph-mon[32599]: pgmap v6040: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:37:59.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:37:59 smithi167 ceph-mon[37140]: pgmap v6040: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:38:01.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:01 smithi138 ceph-mon[36954]: pgmap v6041: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:38:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:01 smithi100 ceph-mon[32599]: pgmap v6041: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:38:01.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:01 smithi167 ceph-mon[37140]: pgmap v6041: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:38:03.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:03 smithi138 ceph-mon[36954]: pgmap v6042: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:38:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:03 smithi100 ceph-mon[32599]: pgmap v6042: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:38:03.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:03 smithi167 ceph-mon[37140]: pgmap v6042: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:38:05.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:05 smithi138 ceph-mon[36954]: pgmap v6043: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:38:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:05 smithi100 ceph-mon[32599]: pgmap v6043: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:38:05.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:05 smithi167 ceph-mon[37140]: pgmap v6043: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:38:07.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:38:07.435+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:38:07.438 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:07 smithi100 ceph-mon[32599]: pgmap v6044: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:38:07.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:38:07.435+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:38:07.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:38:07.435+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:38:07.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:38:07.435+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:38:07.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:38:07.435+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:38:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:07 smithi138 ceph-mon[36954]: pgmap v6044: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:38:07.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:07 smithi167 ceph-mon[37140]: pgmap v6044: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:38:08.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:08 smithi138 ceph-mon[36954]: osdmap e424: 12 total, 12 up, 12 in 2024-07-26T16:38:08.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:08 smithi100 ceph-mon[32599]: osdmap e424: 12 total, 12 up, 12 in 2024-07-26T16:38:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:08 smithi167 ceph-mon[37140]: osdmap e424: 12 total, 12 up, 12 in 2024-07-26T16:38:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:09 smithi138 ceph-mon[36954]: pgmap v6046: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:38:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:09 smithi100 ceph-mon[32599]: pgmap v6046: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:38:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:09 smithi167 ceph-mon[37140]: pgmap v6046: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:38:11.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:11 smithi138 ceph-mon[36954]: pgmap v6047: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:38:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:11 smithi100 ceph-mon[32599]: pgmap v6047: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:38:11.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:11 smithi167 ceph-mon[37140]: pgmap v6047: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:38:13.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:13 smithi138 ceph-mon[36954]: pgmap v6048: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:38:13.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:13 smithi100 ceph-mon[32599]: pgmap v6048: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:38:13.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:13 smithi167 ceph-mon[37140]: pgmap v6048: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:38:15.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:15 smithi138 ceph-mon[36954]: pgmap v6049: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:38:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:15 smithi100 ceph-mon[32599]: pgmap v6049: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:38:15.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:15 smithi167 ceph-mon[37140]: pgmap v6049: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:38:17.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:17 smithi138 ceph-mon[36954]: pgmap v6050: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:38:17.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:17 smithi138 ceph-mon[36954]: osdmap e425: 12 total, 12 up, 12 in 2024-07-26T16:38:17.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:17 smithi100 ceph-mon[32599]: pgmap v6050: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:38:17.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:17 smithi100 ceph-mon[32599]: osdmap e425: 12 total, 12 up, 12 in 2024-07-26T16:38:17.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:17 smithi167 ceph-mon[37140]: pgmap v6050: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:38:17.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:17 smithi167 ceph-mon[37140]: osdmap e425: 12 total, 12 up, 12 in 2024-07-26T16:38:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:19 smithi138 ceph-mon[36954]: pgmap v6052: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:38:19.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:19 smithi100 ceph-mon[32599]: pgmap v6052: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:38:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:19 smithi167 ceph-mon[37140]: pgmap v6052: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:38:21.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:21 smithi138 ceph-mon[36954]: pgmap v6053: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:38:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:21 smithi100 ceph-mon[32599]: pgmap v6053: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:38:21.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:21 smithi167 ceph-mon[37140]: pgmap v6053: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:38:22.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:38:22.451+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:38:22.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:38:22.451+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:38:22.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:38:22.451+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:38:22.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:38:22.451+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:38:22.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:38:22.452+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:38:23.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:23 smithi138 ceph-mon[36954]: pgmap v6054: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:38:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:23 smithi100 ceph-mon[32599]: pgmap v6054: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:38:23.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:23 smithi167 ceph-mon[37140]: pgmap v6054: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:38:25.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:25 smithi138 ceph-mon[36954]: pgmap v6055: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:38:25.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:25 smithi100 ceph-mon[32599]: pgmap v6055: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:38:25.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:25 smithi167 ceph-mon[37140]: pgmap v6055: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:38:27.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:27 smithi138 ceph-mon[36954]: pgmap v6056: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:38:27.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:27 smithi100 ceph-mon[32599]: pgmap v6056: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:38:27.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:27 smithi167 ceph-mon[37140]: pgmap v6056: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:38:29.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:29 smithi138 ceph-mon[36954]: pgmap v6057: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:38:29.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:29 smithi100 ceph-mon[32599]: pgmap v6057: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:38:29.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:29 smithi167 ceph-mon[37140]: pgmap v6057: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:38:31.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:31 smithi138 ceph-mon[36954]: pgmap v6058: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:38:31.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:31 smithi100 ceph-mon[32599]: pgmap v6058: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:38:31.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:31 smithi167 ceph-mon[37140]: pgmap v6058: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:38:33.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:33 smithi138 ceph-mon[36954]: pgmap v6059: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:38:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:33 smithi100 ceph-mon[32599]: pgmap v6059: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:38:33.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:33 smithi167 ceph-mon[37140]: pgmap v6059: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:38:35.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:35 smithi138 ceph-mon[36954]: pgmap v6060: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:38:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:35 smithi100 ceph-mon[32599]: pgmap v6060: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:38:35.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:35 smithi167 ceph-mon[37140]: pgmap v6060: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:38:37.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:37 smithi138 ceph-mon[36954]: pgmap v6061: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:38:37.469 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:37 smithi100 ceph-mon[32599]: pgmap v6061: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:38:37.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:38:37.467+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:38:37.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:38:37.468+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:38:37.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:38:37.468+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:38:37.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:38:37.468+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:38:37.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:38:37.468+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:38:37.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:37 smithi167 ceph-mon[37140]: pgmap v6061: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:38:39.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:39 smithi138 ceph-mon[36954]: pgmap v6062: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:38:39.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:39 smithi100 ceph-mon[32599]: pgmap v6062: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:38:39.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:39 smithi167 ceph-mon[37140]: pgmap v6062: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:38:41.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:41 smithi138 ceph-mon[36954]: pgmap v6063: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:38:41.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:41 smithi100 ceph-mon[32599]: pgmap v6063: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:38:41.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:41 smithi167 ceph-mon[37140]: pgmap v6063: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:38:43.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:43 smithi138 ceph-mon[36954]: pgmap v6064: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:38:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:43 smithi100 ceph-mon[32599]: pgmap v6064: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:38:43.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:43 smithi167 ceph-mon[37140]: pgmap v6064: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:38:45.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:45 smithi138 ceph-mon[36954]: pgmap v6065: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:38:45.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:45 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:38:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:45 smithi100 ceph-mon[32599]: pgmap v6065: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:38:45.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:45 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:38:45.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:45 smithi167 ceph-mon[37140]: pgmap v6065: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:38:45.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:45 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:38:46.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:38:46.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:38:46.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:38:46.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:38:46.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:38:46.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:38:46.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:38:46.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:38:46.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:38:46.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:38:46.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:38:46.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:38:47.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:47 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T16:38:47.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:47 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T16:38:47.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:47 smithi167 ceph-mon[37140]: pgmap v6066: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:38:47.336 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:47 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T16:38:47.336 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:47 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T16:38:47.336 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:47 smithi138 ceph-mon[36954]: pgmap v6066: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:38:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:47 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T16:38:47.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:47 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T16:38:47.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:47 smithi100 ceph-mon[32599]: pgmap v6066: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:38:49.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:49 smithi138 ceph-mon[36954]: pgmap v6067: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:38:49.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:49 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:38:49.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:49 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:38:49.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:49 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:38:49.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:49 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:38:49.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:49 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:38:49.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:49 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:38:49.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:49 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:38:49.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:49 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:38:49.443 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:49 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:38:49.443 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:49 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:38:49.443 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:49 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:38:49.443 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:49 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:38:49.443 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:49 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:38:49.443 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:49 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:38:49.444 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:49 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:38:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:49 smithi100 ceph-mon[32599]: pgmap v6067: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:38:49.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:49 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:38:49.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:49 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:38:49.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:49 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:38:49.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:49 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:38:49.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:49 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:38:49.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:49 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:38:49.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:49 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:38:49.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:49 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:38:49.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:49 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:38:49.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:49 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:38:49.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:49 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:38:49.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:49 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:38:49.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:49 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:38:49.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:49 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:38:49.497 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:49 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:38:49.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:49 smithi167 ceph-mon[37140]: pgmap v6067: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:38:49.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:49 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:38:49.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:49 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:38:49.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:49 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:38:49.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:49 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:38:49.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:49 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:38:49.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:49 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:38:49.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:49 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:38:49.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:49 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:38:49.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:49 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:38:49.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:49 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:38:49.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:49 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:38:49.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:49 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:38:49.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:49 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:38:49.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:49 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:38:49.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:49 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:38:50.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:50 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T16:38:50.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:50 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T16:38:50.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:50 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T16:38:50.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:50 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T16:38:50.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:50 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T16:38:50.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:50 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T16:38:50.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:50 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T16:38:50.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:50 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T16:38:50.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:50 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T16:38:50.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:50 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T16:38:50.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:50 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T16:38:50.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:50 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T16:38:51.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:51 smithi100 ceph-mon[32599]: pgmap v6068: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:38:51.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:51 smithi167 ceph-mon[37140]: pgmap v6068: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:38:51.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:51 smithi138 ceph-mon[36954]: pgmap v6068: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:38:52.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:38:52.482+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:38:52.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:38:52.483+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:38:52.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:38:52.483+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:38:52.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:38:52.483+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:38:52.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:38:52.483+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:38:53.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:53 smithi100 ceph-mon[32599]: pgmap v6069: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:38:53.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:53 smithi167 ceph-mon[37140]: pgmap v6069: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:38:53.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:53 smithi138 ceph-mon[36954]: pgmap v6069: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:38:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:55 smithi100 ceph-mon[32599]: pgmap v6070: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:38:55.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:55 smithi167 ceph-mon[37140]: pgmap v6070: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:38:55.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:55 smithi138 ceph-mon[36954]: pgmap v6070: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:38:57.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:57 smithi100 ceph-mon[32599]: pgmap v6071: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:38:57.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:57 smithi167 ceph-mon[37140]: pgmap v6071: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:38:57.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:57 smithi138 ceph-mon[36954]: pgmap v6071: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:38:59.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:38:59 smithi100 ceph-mon[32599]: pgmap v6072: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:38:59.501 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:38:59 smithi138 ceph-mon[36954]: pgmap v6072: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:38:59.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:38:59 smithi167 ceph-mon[37140]: pgmap v6072: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:39:01.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:39:01 smithi100 ceph-mon[32599]: pgmap v6073: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:39:01.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:39:01 smithi167 ceph-mon[37140]: pgmap v6073: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:39:01.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:39:01 smithi138 ceph-mon[36954]: pgmap v6073: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:39:03.498 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:39:03 smithi100 ceph-mon[32599]: pgmap v6074: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:39:03.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:39:03 smithi167 ceph-mon[37140]: pgmap v6074: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:39:03.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:39:03 smithi138 ceph-mon[36954]: pgmap v6074: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:39:04.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:39:04.468+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T16:39:04.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:39:04.468+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T16:39:04.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:39:04.469+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 82 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T16:39:04.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:39:04.469+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 82 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T16:39:05.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:39:05 smithi100 ceph-mon[32599]: pgmap v6075: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:39:05.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:39:05 smithi167 ceph-mon[37140]: pgmap v6075: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:39:05.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:39:05 smithi138 ceph-mon[36954]: pgmap v6075: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:39:07.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:39:07 smithi100 ceph-mon[32599]: pgmap v6076: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:39:07.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:39:07.498+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:39:07.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:39:07.499+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:39:07.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:39:07.499+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:39:07.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:39:07.499+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:39:07.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:39:07.499+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:39:07.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:39:07 smithi167 ceph-mon[37140]: pgmap v6076: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:39:07.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:39:07 smithi138 ceph-mon[36954]: pgmap v6076: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:39:08.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:39:08 smithi100 ceph-mon[32599]: osdmap e426: 12 total, 12 up, 12 in 2024-07-26T16:39:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:39:08 smithi167 ceph-mon[37140]: osdmap e426: 12 total, 12 up, 12 in 2024-07-26T16:39:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:39:08 smithi138 ceph-mon[36954]: osdmap e426: 12 total, 12 up, 12 in 2024-07-26T16:39:09.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:39:09 smithi100 ceph-mon[32599]: pgmap v6078: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:39:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:39:09 smithi167 ceph-mon[37140]: pgmap v6078: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:39:09.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:39:09 smithi138 ceph-mon[36954]: pgmap v6078: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:39:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:39:11 smithi100 ceph-mon[32599]: pgmap v6079: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:39:11.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:39:11 smithi167 ceph-mon[37140]: pgmap v6079: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:39:11.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:39:11 smithi138 ceph-mon[36954]: pgmap v6079: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:39:13.493 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:39:13 smithi138 ceph-mon[36954]: pgmap v6080: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:39:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:39:13 smithi100 ceph-mon[32599]: pgmap v6080: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:39:13.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:39:13 smithi167 ceph-mon[37140]: pgmap v6080: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:39:15.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:39:15 smithi167 ceph-mon[37140]: pgmap v6081: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:39:15.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:39:15 smithi138 ceph-mon[36954]: pgmap v6081: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:39:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:39:15 smithi100 ceph-mon[32599]: pgmap v6081: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:39:17.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:39:17 smithi167 ceph-mon[37140]: pgmap v6082: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:39:17.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:39:17 smithi167 ceph-mon[37140]: osdmap e427: 12 total, 12 up, 12 in 2024-07-26T16:39:17.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:39:17 smithi138 ceph-mon[36954]: pgmap v6082: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:39:17.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:39:17 smithi138 ceph-mon[36954]: osdmap e427: 12 total, 12 up, 12 in 2024-07-26T16:39:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:39:17 smithi100 ceph-mon[32599]: pgmap v6082: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:39:17.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:39:17 smithi100 ceph-mon[32599]: osdmap e427: 12 total, 12 up, 12 in 2024-07-26T16:39:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:39:19 smithi167 ceph-mon[37140]: pgmap v6084: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:39:19.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:39:19 smithi138 ceph-mon[36954]: pgmap v6084: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:39:19.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:39:19 smithi100 ceph-mon[32599]: pgmap v6084: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:39:21.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:39:21 smithi167 ceph-mon[37140]: pgmap v6085: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:39:21.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:39:21 smithi100 ceph-mon[32599]: pgmap v6085: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:39:21.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:39:21 smithi138 ceph-mon[36954]: pgmap v6085: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:39:22.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:39:22.506+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:39:22.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:39:22.507+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:39:22.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:39:22.507+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:39:22.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:39:22.507+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:39:22.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:39:22.507+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:39:22.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:39:22 smithi138 ceph-mon[36954]: pgmap v6086: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:39:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:39:22 smithi100 ceph-mon[32599]: pgmap v6086: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:39:22.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:39:22 smithi167 ceph-mon[37140]: pgmap v6086: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:39:25.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:39:25 smithi138 ceph-mon[36954]: pgmap v6087: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:39:25.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:39:25 smithi100 ceph-mon[32599]: pgmap v6087: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:39:25.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:39:25 smithi167 ceph-mon[37140]: pgmap v6087: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:39:27.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:39:27 smithi138 ceph-mon[36954]: pgmap v6088: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:39:27.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:39:27 smithi100 ceph-mon[32599]: pgmap v6088: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:39:27.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:39:27 smithi167 ceph-mon[37140]: pgmap v6088: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:39:29.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:39:29 smithi138 ceph-mon[36954]: pgmap v6089: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:39:29.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:39:29 smithi100 ceph-mon[32599]: pgmap v6089: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:39:29.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:39:29 smithi167 ceph-mon[37140]: pgmap v6089: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-07-26T16:39:31.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:39:31 smithi138 ceph-mon[36954]: pgmap v6090: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:39:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:39:31 smithi100 ceph-mon[32599]: pgmap v6090: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:39:31.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:39:31 smithi167 ceph-mon[37140]: pgmap v6090: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:39:33.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:39:33 smithi138 ceph-mon[36954]: pgmap v6091: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:39:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:39:33 smithi100 ceph-mon[32599]: pgmap v6091: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:39:33.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:39:33 smithi167 ceph-mon[37140]: pgmap v6091: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:39:35.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:39:35 smithi138 ceph-mon[36954]: pgmap v6092: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:39:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:39:35 smithi100 ceph-mon[32599]: pgmap v6092: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:39:35.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:39:35 smithi167 ceph-mon[37140]: pgmap v6092: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:39:37.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:39:37 smithi138 ceph-mon[36954]: pgmap v6093: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:39:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:39:37 smithi100 ceph-mon[32599]: pgmap v6093: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:39:37.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:39:37.514+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:39:37.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:39:37.514+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:39:37.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:39:37.514+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:39:37.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:39:37.514+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:39:37.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:39:37.514+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:39:37.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:39:37 smithi167 ceph-mon[37140]: pgmap v6093: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:39:39.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:39:39 smithi138 ceph-mon[36954]: pgmap v6094: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:39:39.497 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:39:39 smithi100 ceph-mon[32599]: pgmap v6094: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:39:39.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:39:39 smithi167 ceph-mon[37140]: pgmap v6094: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:39:41.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:39:41 smithi138 ceph-mon[36954]: pgmap v6095: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:39:41.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:39:41 smithi100 ceph-mon[32599]: pgmap v6095: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:39:41.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:39:41 smithi167 ceph-mon[37140]: pgmap v6095: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:39:43.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:39:43 smithi138 ceph-mon[36954]: pgmap v6096: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:39:43.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:39:43 smithi100 ceph-mon[32599]: pgmap v6096: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:39:43.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:39:43 smithi167 ceph-mon[37140]: pgmap v6096: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:39:45.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:39:45 smithi138 ceph-mon[36954]: pgmap v6097: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:39:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:39:45 smithi100 ceph-mon[32599]: pgmap v6097: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:39:45.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:39:45 smithi167 ceph-mon[37140]: pgmap v6097: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:39:47.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:39:47 smithi138 ceph-mon[36954]: pgmap v6098: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:39:47.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:39:47 smithi100 ceph-mon[32599]: pgmap v6098: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:39:47.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:39:47 smithi167 ceph-mon[37140]: pgmap v6098: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:39:49.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:39:49 smithi138 ceph-mon[36954]: pgmap v6099: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:39:49.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:39:49 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:39:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:39:49 smithi100 ceph-mon[32599]: pgmap v6099: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:39:49.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:39:49 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:39:49.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:39:49 smithi167 ceph-mon[37140]: pgmap v6099: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:39:49.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:39:49 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:39:50.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:39:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:39:50.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:39:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:39:50.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:39:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:39:50.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:39:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:39:50.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:39:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:39:50.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:39:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:39:50.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:39:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:39:50.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:39:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:39:50.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:39:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:39:51.396 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:39:51 smithi100 ceph-mon[32599]: pgmap v6100: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:39:51.424 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:39:51 smithi167 ceph-mon[37140]: pgmap v6100: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:39:51.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:39:51 smithi138 ceph-mon[36954]: pgmap v6100: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:39:52.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:39:52.527+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:39:52.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:39:52.527+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:39:52.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:39:52.527+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:39:52.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:39:52.527+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:39:52.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:39:52.527+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:39:53.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:39:53 smithi138 ceph-mon[36954]: pgmap v6101: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:39:53.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:39:53 smithi100 ceph-mon[32599]: pgmap v6101: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:39:53.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:39:53 smithi167 ceph-mon[37140]: pgmap v6101: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:39:55.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:39:55 smithi138 ceph-mon[36954]: pgmap v6102: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:39:55.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:39:55 smithi100 ceph-mon[32599]: pgmap v6102: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:39:55.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:39:55 smithi167 ceph-mon[37140]: pgmap v6102: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:39:57.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:39:57 smithi138 ceph-mon[36954]: pgmap v6103: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:39:57.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:39:57 smithi100 ceph-mon[32599]: pgmap v6103: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:39:57.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:39:57 smithi167 ceph-mon[37140]: pgmap v6103: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:39:59.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:39:59 smithi138 ceph-mon[36954]: pgmap v6104: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:39:59.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:39:59 smithi100 ceph-mon[32599]: pgmap v6104: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:39:59.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:39:59 smithi167 ceph-mon[37140]: pgmap v6104: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:40:00.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:40:00 smithi138 ceph-mon[36954]: overall HEALTH_OK 2024-07-26T16:40:00.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:40:00 smithi100 ceph-mon[32599]: overall HEALTH_OK 2024-07-26T16:40:00.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:40:00 smithi167 ceph-mon[37140]: overall HEALTH_OK 2024-07-26T16:40:01.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:40:01 smithi138 ceph-mon[36954]: pgmap v6105: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:40:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:40:01 smithi100 ceph-mon[32599]: pgmap v6105: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:40:01.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:40:01 smithi167 ceph-mon[37140]: pgmap v6105: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:40:03.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:40:03 smithi138 ceph-mon[36954]: pgmap v6106: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:40:03.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:40:03 smithi100 ceph-mon[32599]: pgmap v6106: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:40:03.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:40:03 smithi167 ceph-mon[37140]: pgmap v6106: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:40:05.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:40:05 smithi138 ceph-mon[36954]: pgmap v6107: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:40:05.498 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:40:05 smithi100 ceph-mon[32599]: pgmap v6107: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:40:05.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:40:05 smithi167 ceph-mon[37140]: pgmap v6107: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:40:07.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:40:07 smithi138 ceph-mon[36954]: pgmap v6108: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:40:07.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:40:07 smithi100 ceph-mon[32599]: pgmap v6108: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:40:07.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:40:07 smithi167 ceph-mon[37140]: pgmap v6108: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:40:07.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:40:07.543+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:40:07.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:40:07.544+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:40:07.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:40:07.544+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:40:07.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:40:07.544+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:40:07.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:40:07.544+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:40:08.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:40:08 smithi138 ceph-mon[36954]: osdmap e428: 12 total, 12 up, 12 in 2024-07-26T16:40:08.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:40:08 smithi100 ceph-mon[32599]: osdmap e428: 12 total, 12 up, 12 in 2024-07-26T16:40:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:40:08 smithi167 ceph-mon[37140]: osdmap e428: 12 total, 12 up, 12 in 2024-07-26T16:40:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:40:09 smithi138 ceph-mon[36954]: pgmap v6110: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:40:09.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:40:09 smithi100 ceph-mon[32599]: pgmap v6110: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:40:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:40:09 smithi167 ceph-mon[37140]: pgmap v6110: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:40:11.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:40:11 smithi138 ceph-mon[36954]: pgmap v6111: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:40:11.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:40:11 smithi100 ceph-mon[32599]: pgmap v6111: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:40:11.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:40:11 smithi167 ceph-mon[37140]: pgmap v6111: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:40:13.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:40:13 smithi138 ceph-mon[36954]: pgmap v6112: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:40:13.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:40:13 smithi100 ceph-mon[32599]: pgmap v6112: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:40:13.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:40:13 smithi167 ceph-mon[37140]: pgmap v6112: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:40:15.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:40:15 smithi138 ceph-mon[36954]: pgmap v6113: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:40:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:40:15 smithi100 ceph-mon[32599]: pgmap v6113: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:40:15.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:40:15 smithi167 ceph-mon[37140]: pgmap v6113: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:40:17.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:40:17 smithi138 ceph-mon[36954]: pgmap v6114: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:40:17.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:40:17 smithi138 ceph-mon[36954]: osdmap e429: 12 total, 12 up, 12 in 2024-07-26T16:40:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:40:17 smithi100 ceph-mon[32599]: pgmap v6114: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:40:17.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:40:17 smithi100 ceph-mon[32599]: osdmap e429: 12 total, 12 up, 12 in 2024-07-26T16:40:17.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:40:17 smithi167 ceph-mon[37140]: pgmap v6114: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:40:17.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:40:17 smithi167 ceph-mon[37140]: osdmap e429: 12 total, 12 up, 12 in 2024-07-26T16:40:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:40:19 smithi138 ceph-mon[36954]: pgmap v6116: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:40:19.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:40:19 smithi100 ceph-mon[32599]: pgmap v6116: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:40:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:40:19 smithi167 ceph-mon[37140]: pgmap v6116: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:40:21.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:40:21 smithi138 ceph-mon[36954]: pgmap v6117: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:40:21.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:40:21 smithi100 ceph-mon[32599]: pgmap v6117: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:40:21.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:40:21 smithi167 ceph-mon[37140]: pgmap v6117: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:40:22.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:40:22.559+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:40:22.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:40:22.560+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:40:22.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:40:22.560+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:40:22.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:40:22.560+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:40:22.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:40:22.560+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:40:23.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:40:23 smithi138 ceph-mon[36954]: pgmap v6118: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:40:23.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:40:23 smithi100 ceph-mon[32599]: pgmap v6118: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:40:23.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:40:23 smithi167 ceph-mon[37140]: pgmap v6118: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:40:25.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:40:25 smithi138 ceph-mon[36954]: pgmap v6119: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:40:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:40:25 smithi100 ceph-mon[32599]: pgmap v6119: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:40:25.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:40:25 smithi167 ceph-mon[37140]: pgmap v6119: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:40:27.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:40:27 smithi138 ceph-mon[36954]: pgmap v6120: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:40:27.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:40:27 smithi100 ceph-mon[32599]: pgmap v6120: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:40:27.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:40:27 smithi167 ceph-mon[37140]: pgmap v6120: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:40:29.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:40:29 smithi138 ceph-mon[36954]: pgmap v6121: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:40:29.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:40:29 smithi100 ceph-mon[32599]: pgmap v6121: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:40:29.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:40:29 smithi167 ceph-mon[37140]: pgmap v6121: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:40:31.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:40:31 smithi138 ceph-mon[36954]: pgmap v6122: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:40:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:40:31 smithi100 ceph-mon[32599]: pgmap v6122: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:40:31.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:40:31 smithi167 ceph-mon[37140]: pgmap v6122: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:40:32.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:40:32 smithi138 ceph-mon[36954]: pgmap v6123: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:40:32.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:40:32 smithi100 ceph-mon[32599]: pgmap v6123: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:40:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:40:32 smithi167 ceph-mon[37140]: pgmap v6123: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:40:35.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:40:35 smithi138 ceph-mon[36954]: pgmap v6124: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:40:35.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:40:35 smithi100 ceph-mon[32599]: pgmap v6124: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:40:35.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:40:35 smithi167 ceph-mon[37140]: pgmap v6124: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:40:37.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:40:37 smithi138 ceph-mon[36954]: pgmap v6125: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:40:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:40:37 smithi100 ceph-mon[32599]: pgmap v6125: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:40:37.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:40:37 smithi167 ceph-mon[37140]: pgmap v6125: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:40:37.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:40:37.576+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:40:37.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:40:37.576+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:40:37.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:40:37.576+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:40:37.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:40:37.576+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:40:37.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:40:37.576+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:40:39.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:40:39 smithi138 ceph-mon[36954]: pgmap v6126: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:40:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:40:39 smithi100 ceph-mon[32599]: pgmap v6126: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:40:39.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:40:39 smithi167 ceph-mon[37140]: pgmap v6126: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:40:41.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:40:41 smithi138 ceph-mon[36954]: pgmap v6127: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:40:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:40:41 smithi100 ceph-mon[32599]: pgmap v6127: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:40:41.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:40:41 smithi167 ceph-mon[37140]: pgmap v6127: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:40:43.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:40:43 smithi138 ceph-mon[36954]: pgmap v6128: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:40:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:40:43 smithi100 ceph-mon[32599]: pgmap v6128: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:40:43.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:40:43 smithi167 ceph-mon[37140]: pgmap v6128: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:40:45.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:40:45 smithi138 ceph-mon[36954]: pgmap v6129: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:40:45.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:40:45 smithi100 ceph-mon[32599]: pgmap v6129: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:40:45.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:40:45 smithi167 ceph-mon[37140]: pgmap v6129: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:40:47.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:40:47 smithi138 ceph-mon[36954]: pgmap v6130: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:40:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:40:47 smithi100 ceph-mon[32599]: pgmap v6130: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:40:47.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:40:47 smithi167 ceph-mon[37140]: pgmap v6130: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:40:49.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:40:49 smithi138 ceph-mon[36954]: pgmap v6131: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:40:49.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:40:49 smithi100 ceph-mon[32599]: pgmap v6131: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:40:49.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:40:49 smithi167 ceph-mon[37140]: pgmap v6131: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:40:50.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:40:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:40:50.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:40:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:40:50.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:40:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:40:50.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:40:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:40:50.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:40:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:40:50.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:40:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:40:50.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:40:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:40:50.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:40:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:40:50.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:40:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:40:50.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:40:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:40:50.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:40:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:40:50.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:40:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:40:51.396 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:40:51 smithi100 ceph-mon[32599]: pgmap v6132: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:40:51.424 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:40:51 smithi167 ceph-mon[37140]: pgmap v6132: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:40:51.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:40:51 smithi138 ceph-mon[36954]: pgmap v6132: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:40:52.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:40:52.592+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:40:52.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:40:52.592+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:40:52.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:40:52.592+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:40:52.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:40:52.592+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:40:52.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:40:52.592+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:40:53.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:40:53 smithi138 ceph-mon[36954]: pgmap v6133: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:40:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:40:53 smithi100 ceph-mon[32599]: pgmap v6133: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:40:53.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:40:53 smithi167 ceph-mon[37140]: pgmap v6133: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:40:55.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:40:55 smithi138 ceph-mon[36954]: pgmap v6134: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:40:55.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:40:55 smithi100 ceph-mon[32599]: pgmap v6134: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:40:55.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:40:55 smithi167 ceph-mon[37140]: pgmap v6134: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:40:57.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:40:57 smithi138 ceph-mon[36954]: pgmap v6135: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:40:57.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:40:57 smithi100 ceph-mon[32599]: pgmap v6135: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:40:57.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:40:57 smithi167 ceph-mon[37140]: pgmap v6135: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:40:59.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:40:59 smithi138 ceph-mon[36954]: pgmap v6136: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:40:59.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:40:59 smithi100 ceph-mon[32599]: pgmap v6136: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:40:59.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:40:59 smithi167 ceph-mon[37140]: pgmap v6136: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:41:01.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:01 smithi138 ceph-mon[36954]: pgmap v6137: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:41:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:01 smithi100 ceph-mon[32599]: pgmap v6137: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:41:01.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:01 smithi167 ceph-mon[37140]: pgmap v6137: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:41:03.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:03 smithi138 ceph-mon[36954]: pgmap v6138: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:41:03.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:03 smithi100 ceph-mon[32599]: pgmap v6138: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:41:03.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:03 smithi167 ceph-mon[37140]: pgmap v6138: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:41:05.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:05 smithi138 ceph-mon[36954]: pgmap v6139: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:41:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:05 smithi100 ceph-mon[32599]: pgmap v6139: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:41:05.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:05 smithi167 ceph-mon[37140]: pgmap v6139: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:41:07.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:07 smithi138 ceph-mon[36954]: pgmap v6140: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:41:07.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:07 smithi100 ceph-mon[32599]: pgmap v6140: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:41:07.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:07 smithi167 ceph-mon[37140]: pgmap v6140: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:41:07.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:41:07.608+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:41:07.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:41:07.608+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:41:07.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:41:07.608+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:41:07.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:41:07.608+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:41:07.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:41:07.608+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:41:08.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:08 smithi138 ceph-mon[36954]: osdmap e430: 12 total, 12 up, 12 in 2024-07-26T16:41:08.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:08 smithi100 ceph-mon[32599]: osdmap e430: 12 total, 12 up, 12 in 2024-07-26T16:41:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:08 smithi167 ceph-mon[37140]: osdmap e430: 12 total, 12 up, 12 in 2024-07-26T16:41:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:09 smithi138 ceph-mon[36954]: pgmap v6142: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:41:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:09 smithi100 ceph-mon[32599]: pgmap v6142: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:41:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:09 smithi167 ceph-mon[37140]: pgmap v6142: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:41:11.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:11 smithi138 ceph-mon[36954]: pgmap v6143: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:41:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:11 smithi100 ceph-mon[32599]: pgmap v6143: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:41:11.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:11 smithi167 ceph-mon[37140]: pgmap v6143: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:41:13.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:13 smithi138 ceph-mon[36954]: pgmap v6144: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:41:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:13 smithi100 ceph-mon[32599]: pgmap v6144: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:41:13.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:13 smithi167 ceph-mon[37140]: pgmap v6144: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:41:15.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:15 smithi138 ceph-mon[36954]: pgmap v6145: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:41:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:15 smithi100 ceph-mon[32599]: pgmap v6145: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:41:15.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:15 smithi167 ceph-mon[37140]: pgmap v6145: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:41:17.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:17 smithi138 ceph-mon[36954]: pgmap v6146: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:41:17.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:17 smithi138 ceph-mon[36954]: osdmap e431: 12 total, 12 up, 12 in 2024-07-26T16:41:17.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:17 smithi100 ceph-mon[32599]: pgmap v6146: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:41:17.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:17 smithi100 ceph-mon[32599]: osdmap e431: 12 total, 12 up, 12 in 2024-07-26T16:41:17.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:17 smithi167 ceph-mon[37140]: pgmap v6146: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:41:17.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:17 smithi167 ceph-mon[37140]: osdmap e431: 12 total, 12 up, 12 in 2024-07-26T16:41:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:19 smithi138 ceph-mon[36954]: pgmap v6148: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:41:19.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:41:19.487+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T16:41:19.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:41:19.487+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T16:41:19.489 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:19 smithi100 ceph-mon[32599]: pgmap v6148: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:41:19.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:41:19.487+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 83 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T16:41:19.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:41:19.487+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 83 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T16:41:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:19 smithi167 ceph-mon[37140]: pgmap v6148: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:41:21.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:21 smithi138 ceph-mon[36954]: pgmap v6149: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:41:21.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:21 smithi100 ceph-mon[32599]: pgmap v6149: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:41:21.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:21 smithi167 ceph-mon[37140]: pgmap v6149: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:41:22.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:41:22.620+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:41:22.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:41:22.620+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:41:22.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:41:22.620+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:41:22.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:41:22.620+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:41:22.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:41:22.621+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:41:23.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:23 smithi138 ceph-mon[36954]: pgmap v6150: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:41:23.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:23 smithi100 ceph-mon[32599]: pgmap v6150: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:41:23.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:23 smithi167 ceph-mon[37140]: pgmap v6150: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:41:25.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:25 smithi138 ceph-mon[36954]: pgmap v6151: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:41:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:25 smithi100 ceph-mon[32599]: pgmap v6151: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:41:25.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:25 smithi167 ceph-mon[37140]: pgmap v6151: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:41:27.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:27 smithi138 ceph-mon[36954]: pgmap v6152: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:41:27.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:27 smithi100 ceph-mon[32599]: pgmap v6152: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:41:27.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:27 smithi167 ceph-mon[37140]: pgmap v6152: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:41:29.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:29 smithi138 ceph-mon[36954]: pgmap v6153: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:41:29.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:29 smithi100 ceph-mon[32599]: pgmap v6153: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:41:29.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:29 smithi167 ceph-mon[37140]: pgmap v6153: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:41:31.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:31 smithi138 ceph-mon[36954]: pgmap v6154: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:41:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:31 smithi100 ceph-mon[32599]: pgmap v6154: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:41:31.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:31 smithi167 ceph-mon[37140]: pgmap v6154: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:41:33.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:33 smithi138 ceph-mon[36954]: pgmap v6155: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:41:33.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:33 smithi100 ceph-mon[32599]: pgmap v6155: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:41:33.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:33 smithi167 ceph-mon[37140]: pgmap v6155: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:41:35.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:35 smithi138 ceph-mon[36954]: pgmap v6156: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:41:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:35 smithi100 ceph-mon[32599]: pgmap v6156: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:41:35.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:35 smithi167 ceph-mon[37140]: pgmap v6156: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:41:37.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:37 smithi138 ceph-mon[36954]: pgmap v6157: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:41:37.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:37 smithi100 ceph-mon[32599]: pgmap v6157: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:41:37.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:37 smithi167 ceph-mon[37140]: pgmap v6157: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:41:37.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:41:37.631+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:41:37.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:41:37.632+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:41:37.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:41:37.632+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:41:37.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:41:37.632+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:41:37.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:41:37.632+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:41:39.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:39 smithi138 ceph-mon[36954]: pgmap v6158: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:41:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:39 smithi100 ceph-mon[32599]: pgmap v6158: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:41:39.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:39 smithi167 ceph-mon[37140]: pgmap v6158: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:41:41.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:41 smithi138 ceph-mon[36954]: pgmap v6159: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:41:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:41 smithi100 ceph-mon[32599]: pgmap v6159: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:41:41.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:41 smithi167 ceph-mon[37140]: pgmap v6159: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:41:43.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:43 smithi138 ceph-mon[36954]: pgmap v6160: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:41:43.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:43 smithi100 ceph-mon[32599]: pgmap v6160: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:41:43.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:43 smithi167 ceph-mon[37140]: pgmap v6160: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:41:45.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:45 smithi100 ceph-mon[32599]: pgmap v6161: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:41:45.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:45 smithi167 ceph-mon[37140]: pgmap v6161: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:41:45.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:45 smithi138 ceph-mon[36954]: pgmap v6161: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:41:47.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:47 smithi100 ceph-mon[32599]: pgmap v6162: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:41:47.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:47 smithi167 ceph-mon[37140]: pgmap v6162: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:41:47.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:47 smithi138 ceph-mon[36954]: pgmap v6162: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:41:49.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:49 smithi100 ceph-mon[32599]: pgmap v6163: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:41:49.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:49 smithi167 ceph-mon[37140]: pgmap v6163: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:41:49.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:49 smithi138 ceph-mon[36954]: pgmap v6163: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:41:50.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:41:50.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:41:50.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:41:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:51 smithi100 ceph-mon[32599]: pgmap v6164: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:41:51.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:51 smithi167 ceph-mon[37140]: pgmap v6164: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:41:51.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:51 smithi138 ceph-mon[36954]: pgmap v6164: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:41:52.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:41:52.647+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:41:52.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:41:52.648+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:41:52.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:41:52.648+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:41:52.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:41:52.648+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:41:52.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:41:52.648+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:41:52.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:41:52.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:41:52.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:41:52.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:41:52.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:41:52.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:41:52.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:41:52.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:41:52.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:41:52.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:52 smithi138 ceph-mon[36954]: pgmap v6165: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:41:52.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:41:52.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:41:52.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:41:52.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:41:52.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:41:52.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:41:52.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:41:52.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:41:52.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:41:52.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:52 smithi100 ceph-mon[32599]: pgmap v6165: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:41:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:41:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:41:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:41:53.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:41:53.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:41:53.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:41:53.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:41:53.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:41:53.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:41:53.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:52 smithi167 ceph-mon[37140]: pgmap v6165: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:41:55.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:55 smithi138 ceph-mon[36954]: pgmap v6166: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:41:55.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:55 smithi100 ceph-mon[32599]: pgmap v6166: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:41:55.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:55 smithi167 ceph-mon[37140]: pgmap v6166: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:41:57.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:57 smithi138 ceph-mon[36954]: pgmap v6167: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:41:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:57 smithi100 ceph-mon[32599]: pgmap v6167: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:41:57.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:57 smithi167 ceph-mon[37140]: pgmap v6167: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:41:59.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:41:59 smithi138 ceph-mon[36954]: pgmap v6168: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:41:59.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:41:59 smithi100 ceph-mon[32599]: pgmap v6168: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:41:59.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:41:59 smithi167 ceph-mon[37140]: pgmap v6168: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:42:01.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:01 smithi138 ceph-mon[36954]: pgmap v6169: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:42:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:01 smithi100 ceph-mon[32599]: pgmap v6169: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:42:01.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:01 smithi167 ceph-mon[37140]: pgmap v6169: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:42:02.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:02 smithi138 ceph-mon[36954]: pgmap v6170: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:42:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:02 smithi100 ceph-mon[32599]: pgmap v6170: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:42:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:02 smithi167 ceph-mon[37140]: pgmap v6170: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:42:04.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:42:04.493+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T16:42:04.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:42:04.493+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T16:42:04.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:42:04.494+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 84 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T16:42:04.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:42:04.494+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 84 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T16:42:05.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:05 smithi138 ceph-mon[36954]: pgmap v6171: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:42:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:05 smithi100 ceph-mon[32599]: pgmap v6171: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:42:05.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:05 smithi167 ceph-mon[37140]: pgmap v6171: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:42:07.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:07 smithi138 ceph-mon[36954]: pgmap v6172: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:42:07.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:07 smithi100 ceph-mon[32599]: pgmap v6172: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:42:07.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:07 smithi167 ceph-mon[37140]: pgmap v6172: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:42:07.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:42:07.663+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:42:07.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:42:07.664+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:42:07.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:42:07.664+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:42:07.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:42:07.664+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:42:07.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:42:07.664+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:42:08.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:08 smithi138 ceph-mon[36954]: osdmap e432: 12 total, 12 up, 12 in 2024-07-26T16:42:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:08 smithi100 ceph-mon[32599]: osdmap e432: 12 total, 12 up, 12 in 2024-07-26T16:42:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:08 smithi167 ceph-mon[37140]: osdmap e432: 12 total, 12 up, 12 in 2024-07-26T16:42:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:09 smithi138 ceph-mon[36954]: pgmap v6174: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:42:09.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:09 smithi100 ceph-mon[32599]: pgmap v6174: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:42:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:09 smithi167 ceph-mon[37140]: pgmap v6174: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:42:11.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:11 smithi138 ceph-mon[36954]: pgmap v6175: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:42:11.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:11 smithi100 ceph-mon[32599]: pgmap v6175: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:42:11.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:11 smithi167 ceph-mon[37140]: pgmap v6175: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:42:13.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:13 smithi138 ceph-mon[36954]: pgmap v6176: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:42:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:13 smithi100 ceph-mon[32599]: pgmap v6176: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:42:13.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:13 smithi167 ceph-mon[37140]: pgmap v6176: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:42:15.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:15 smithi138 ceph-mon[36954]: pgmap v6177: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:42:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:15 smithi100 ceph-mon[32599]: pgmap v6177: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:42:15.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:15 smithi167 ceph-mon[37140]: pgmap v6177: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:42:17.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:17 smithi138 ceph-mon[36954]: pgmap v6178: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:42:17.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:17 smithi138 ceph-mon[36954]: osdmap e433: 12 total, 12 up, 12 in 2024-07-26T16:42:17.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:17 smithi100 ceph-mon[32599]: pgmap v6178: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:42:17.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:17 smithi100 ceph-mon[32599]: osdmap e433: 12 total, 12 up, 12 in 2024-07-26T16:42:17.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:17 smithi167 ceph-mon[37140]: pgmap v6178: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:42:17.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:17 smithi167 ceph-mon[37140]: osdmap e433: 12 total, 12 up, 12 in 2024-07-26T16:42:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:19 smithi138 ceph-mon[36954]: pgmap v6180: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:42:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:19 smithi100 ceph-mon[32599]: pgmap v6180: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:42:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:19 smithi167 ceph-mon[37140]: pgmap v6180: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:42:21.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:21 smithi138 ceph-mon[36954]: pgmap v6181: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:42:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:21 smithi100 ceph-mon[32599]: pgmap v6181: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:42:21.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:21 smithi167 ceph-mon[37140]: pgmap v6181: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:42:22.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:42:22.680+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:42:22.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:42:22.680+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:42:22.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:42:22.680+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:42:22.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:42:22.680+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:42:22.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:42:22.680+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:42:23.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:23 smithi138 ceph-mon[36954]: pgmap v6182: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:42:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:23 smithi100 ceph-mon[32599]: pgmap v6182: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:42:23.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:23 smithi167 ceph-mon[37140]: pgmap v6182: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:42:25.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:25 smithi138 ceph-mon[36954]: pgmap v6183: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:42:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:25 smithi100 ceph-mon[32599]: pgmap v6183: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:42:25.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:25 smithi167 ceph-mon[37140]: pgmap v6183: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:42:27.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:27 smithi138 ceph-mon[36954]: pgmap v6184: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:42:27.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:27 smithi100 ceph-mon[32599]: pgmap v6184: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:42:27.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:27 smithi167 ceph-mon[37140]: pgmap v6184: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:42:29.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:29 smithi138 ceph-mon[36954]: pgmap v6185: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:42:29.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:29 smithi100 ceph-mon[32599]: pgmap v6185: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:42:29.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:29 smithi167 ceph-mon[37140]: pgmap v6185: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:42:31.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:31 smithi138 ceph-mon[36954]: pgmap v6186: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:42:31.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:31 smithi100 ceph-mon[32599]: pgmap v6186: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:42:31.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:31 smithi167 ceph-mon[37140]: pgmap v6186: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:42:33.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:33 smithi138 ceph-mon[36954]: pgmap v6187: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:42:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:33 smithi100 ceph-mon[32599]: pgmap v6187: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:42:33.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:33 smithi167 ceph-mon[37140]: pgmap v6187: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:42:35.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:35 smithi138 ceph-mon[36954]: pgmap v6188: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:42:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:35 smithi100 ceph-mon[32599]: pgmap v6188: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:42:35.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:35 smithi167 ceph-mon[37140]: pgmap v6188: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:42:37.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:37 smithi138 ceph-mon[36954]: pgmap v6189: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:42:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:37 smithi100 ceph-mon[32599]: pgmap v6189: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:42:37.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:37 smithi167 ceph-mon[37140]: pgmap v6189: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:42:37.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:42:37.695+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:42:37.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:42:37.695+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:42:37.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:42:37.695+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:42:37.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:42:37.695+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:42:37.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:42:37.695+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:42:39.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:39 smithi138 ceph-mon[36954]: pgmap v6190: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:42:39.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:39 smithi100 ceph-mon[32599]: pgmap v6190: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:42:39.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:39 smithi167 ceph-mon[37140]: pgmap v6190: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:42:41.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:41 smithi138 ceph-mon[36954]: pgmap v6191: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:42:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:41 smithi100 ceph-mon[32599]: pgmap v6191: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:42:41.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:41 smithi167 ceph-mon[37140]: pgmap v6191: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:42:43.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:43 smithi138 ceph-mon[36954]: pgmap v6192: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:42:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:43 smithi100 ceph-mon[32599]: pgmap v6192: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:42:43.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:43 smithi167 ceph-mon[37140]: pgmap v6192: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:42:45.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:45 smithi138 ceph-mon[36954]: pgmap v6193: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:42:45.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:45 smithi100 ceph-mon[32599]: pgmap v6193: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:42:45.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:45 smithi167 ceph-mon[37140]: pgmap v6193: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:42:47.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:47 smithi138 ceph-mon[36954]: pgmap v6194: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T16:42:47.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:47 smithi100 ceph-mon[32599]: pgmap v6194: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T16:42:47.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:47 smithi167 ceph-mon[37140]: pgmap v6194: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T16:42:49.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:49 smithi138 ceph-mon[36954]: pgmap v6195: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:42:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:49 smithi100 ceph-mon[32599]: pgmap v6195: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:42:49.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:49 smithi167 ceph-mon[37140]: pgmap v6195: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:42:51.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:51 smithi138 ceph-mon[36954]: pgmap v6196: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:42:51.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:51 smithi100 ceph-mon[32599]: pgmap v6196: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:42:51.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:51 smithi167 ceph-mon[37140]: pgmap v6196: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:42:52.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:42:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:42:52.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:42:52.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:42:52.712+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:42:52.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:42:52.712+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:42:52.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:42:52.712+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:42:52.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:42:52.712+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:42:52.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:42:52.712+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:42:53.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:53 smithi138 ceph-mon[36954]: pgmap v6197: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:42:53.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:53 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:42:53.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:53 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:42:53.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:53 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:42:53.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:53 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:42:53.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:53 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:42:53.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:53 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:42:53.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:53 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:42:53.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:53 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:42:53.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:53 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:42:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:53 smithi100 ceph-mon[32599]: pgmap v6197: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:42:53.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:53 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:42:53.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:53 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:42:53.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:53 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:42:53.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:53 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:42:53.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:53 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:42:53.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:53 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:42:53.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:53 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:42:53.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:53 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:42:53.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:53 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:42:53.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:53 smithi167 ceph-mon[37140]: pgmap v6197: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:42:53.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:53 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:42:53.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:53 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:42:53.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:53 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:42:53.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:53 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:42:53.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:53 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:42:53.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:53 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:42:53.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:53 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:42:53.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:53 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:42:53.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:53 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:42:55.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:55 smithi100 ceph-mon[32599]: pgmap v6198: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:42:55.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:55 smithi167 ceph-mon[37140]: pgmap v6198: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:42:55.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:55 smithi138 ceph-mon[36954]: pgmap v6198: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:42:57.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:57 smithi100 ceph-mon[32599]: pgmap v6199: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:42:57.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:57 smithi167 ceph-mon[37140]: pgmap v6199: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:42:57.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:57 smithi138 ceph-mon[36954]: pgmap v6199: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:42:59.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:42:59 smithi100 ceph-mon[32599]: pgmap v6200: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:42:59.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:42:59 smithi167 ceph-mon[37140]: pgmap v6200: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:42:59.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:42:59 smithi138 ceph-mon[36954]: pgmap v6200: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:43:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:43:01 smithi100 ceph-mon[32599]: pgmap v6201: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:43:01.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:43:01 smithi167 ceph-mon[37140]: pgmap v6201: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:43:01.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:43:01 smithi138 ceph-mon[36954]: pgmap v6201: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:43:03.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:43:03 smithi100 ceph-mon[32599]: pgmap v6202: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:43:03.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:43:03 smithi167 ceph-mon[37140]: pgmap v6202: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:43:03.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:43:03 smithi138 ceph-mon[36954]: pgmap v6202: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:43:05.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:43:05 smithi100 ceph-mon[32599]: pgmap v6203: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:43:05.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:43:05 smithi167 ceph-mon[37140]: pgmap v6203: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:43:05.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:43:05 smithi138 ceph-mon[36954]: pgmap v6203: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:43:07.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:43:07 smithi100 ceph-mon[32599]: pgmap v6204: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:43:07.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:43:07 smithi167 ceph-mon[37140]: pgmap v6204: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:43:07.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:43:07 smithi138 ceph-mon[36954]: pgmap v6204: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:43:07.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:43:07.719+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:43:07.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:43:07.720+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:43:07.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:43:07.720+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:43:07.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:43:07.720+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:43:07.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:43:07.720+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:43:08.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:43:08 smithi100 ceph-mon[32599]: osdmap e434: 12 total, 12 up, 12 in 2024-07-26T16:43:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:43:08 smithi167 ceph-mon[37140]: osdmap e434: 12 total, 12 up, 12 in 2024-07-26T16:43:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:43:08 smithi138 ceph-mon[36954]: osdmap e434: 12 total, 12 up, 12 in 2024-07-26T16:43:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:43:09 smithi100 ceph-mon[32599]: pgmap v6206: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:43:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:43:09 smithi167 ceph-mon[37140]: pgmap v6206: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:43:09.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:43:09 smithi138 ceph-mon[36954]: pgmap v6206: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:43:11.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:43:11 smithi100 ceph-mon[32599]: pgmap v6207: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:43:11.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:43:11 smithi167 ceph-mon[37140]: pgmap v6207: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:43:11.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:43:11 smithi138 ceph-mon[36954]: pgmap v6207: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:43:13.493 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:43:13 smithi138 ceph-mon[36954]: pgmap v6208: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:43:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:43:13 smithi100 ceph-mon[32599]: pgmap v6208: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:43:13.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:43:13 smithi167 ceph-mon[37140]: pgmap v6208: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:43:15.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:43:15 smithi100 ceph-mon[32599]: pgmap v6209: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:43:15.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:43:15 smithi167 ceph-mon[37140]: pgmap v6209: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:43:15.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:43:15 smithi138 ceph-mon[36954]: pgmap v6209: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:43:17.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:43:17 smithi100 ceph-mon[32599]: pgmap v6210: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:43:17.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:43:17 smithi100 ceph-mon[32599]: osdmap e435: 12 total, 12 up, 12 in 2024-07-26T16:43:17.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:43:17 smithi167 ceph-mon[37140]: pgmap v6210: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:43:17.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:43:17 smithi167 ceph-mon[37140]: osdmap e435: 12 total, 12 up, 12 in 2024-07-26T16:43:17.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:43:17 smithi138 ceph-mon[36954]: pgmap v6210: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:43:17.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:43:17 smithi138 ceph-mon[36954]: osdmap e435: 12 total, 12 up, 12 in 2024-07-26T16:43:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:43:19 smithi167 ceph-mon[37140]: pgmap v6212: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:43:19.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:43:19 smithi138 ceph-mon[36954]: pgmap v6212: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:43:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:43:19 smithi100 ceph-mon[32599]: pgmap v6212: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:43:21.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:43:21 smithi167 ceph-mon[37140]: pgmap v6213: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:43:21.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:43:21 smithi100 ceph-mon[32599]: pgmap v6213: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:43:21.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:43:21 smithi138 ceph-mon[36954]: pgmap v6213: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:43:22.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:43:22.735+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:43:22.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:43:22.736+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:43:22.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:43:22.736+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:43:22.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:43:22.736+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:43:22.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:43:22.736+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:43:23.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:43:23 smithi167 ceph-mon[37140]: pgmap v6214: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:43:23.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:43:23 smithi138 ceph-mon[36954]: pgmap v6214: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:43:23.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:43:23 smithi100 ceph-mon[32599]: pgmap v6214: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:43:25.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:43:25 smithi167 ceph-mon[37140]: pgmap v6215: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:43:25.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:43:25 smithi138 ceph-mon[36954]: pgmap v6215: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:43:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:43:25 smithi100 ceph-mon[32599]: pgmap v6215: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:43:27.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:43:27 smithi167 ceph-mon[37140]: pgmap v6216: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:43:27.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:43:27 smithi138 ceph-mon[36954]: pgmap v6216: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:43:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:43:27 smithi100 ceph-mon[32599]: pgmap v6216: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:43:29.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:43:29 smithi167 ceph-mon[37140]: pgmap v6217: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:43:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:43:29 smithi138 ceph-mon[36954]: pgmap v6217: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:43:29.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:43:29 smithi100 ceph-mon[32599]: pgmap v6217: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:43:31.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:43:31 smithi167 ceph-mon[37140]: pgmap v6218: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:43:31.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:43:31 smithi138 ceph-mon[36954]: pgmap v6218: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:43:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:43:31 smithi100 ceph-mon[32599]: pgmap v6218: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:43:33.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:43:33 smithi167 ceph-mon[37140]: pgmap v6219: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:43:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:43:33 smithi138 ceph-mon[36954]: pgmap v6219: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:43:33.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:43:33 smithi100 ceph-mon[32599]: pgmap v6219: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:43:35.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:43:35 smithi138 ceph-mon[36954]: pgmap v6220: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:43:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:43:35 smithi100 ceph-mon[32599]: pgmap v6220: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:43:35.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:43:35 smithi167 ceph-mon[37140]: pgmap v6220: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:43:37.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:43:37 smithi138 ceph-mon[36954]: pgmap v6221: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:43:37.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:43:37 smithi100 ceph-mon[32599]: pgmap v6221: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:43:37.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:43:37.751+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:43:37.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:43:37.752+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:43:37.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:43:37.752+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:43:37.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:43:37.752+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:43:37.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:43:37.752+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:43:37.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:43:37 smithi167 ceph-mon[37140]: pgmap v6221: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:43:39.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:43:39 smithi138 ceph-mon[36954]: pgmap v6222: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:43:39.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:43:39 smithi100 ceph-mon[32599]: pgmap v6222: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:43:39.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:43:39 smithi167 ceph-mon[37140]: pgmap v6222: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:43:41.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:43:41 smithi138 ceph-mon[36954]: pgmap v6223: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:43:41.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:43:41 smithi167 ceph-mon[37140]: pgmap v6223: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:43:41.810 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:43:41 smithi100 ceph-mon[32599]: pgmap v6223: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:43:43.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:43:43 smithi138 ceph-mon[36954]: pgmap v6224: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:43:43.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:43:43 smithi100 ceph-mon[32599]: pgmap v6224: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:43:43.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:43:43 smithi167 ceph-mon[37140]: pgmap v6224: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:43:45.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:43:45 smithi138 ceph-mon[36954]: pgmap v6225: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:43:45.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:43:45 smithi100 ceph-mon[32599]: pgmap v6225: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:43:45.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:43:45 smithi167 ceph-mon[37140]: pgmap v6225: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:43:47.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:43:47 smithi138 ceph-mon[36954]: pgmap v6226: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:43:47.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:43:47 smithi100 ceph-mon[32599]: pgmap v6226: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:43:47.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:43:47 smithi167 ceph-mon[37140]: pgmap v6226: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:43:49.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:43:49 smithi138 ceph-mon[36954]: pgmap v6227: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:43:49.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:43:49 smithi100 ceph-mon[32599]: pgmap v6227: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:43:49.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:43:49 smithi167 ceph-mon[37140]: pgmap v6227: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:43:51.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:43:51 smithi167 ceph-mon[37140]: pgmap v6228: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:43:51.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:43:51 smithi138 ceph-mon[36954]: pgmap v6228: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:43:51.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:43:51 smithi100 ceph-mon[32599]: pgmap v6228: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:43:52.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:43:52.752+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:43:52.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:43:52.753+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:43:52.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:43:52.753+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:43:52.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:43:52.753+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:43:52.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:43:52.753+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:43:52.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:43:52 smithi138 ceph-mon[36954]: pgmap v6229: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:43:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:43:52 smithi100 ceph-mon[32599]: pgmap v6229: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:43:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:43:52 smithi167 ceph-mon[37140]: pgmap v6229: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:43:53.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:43:53 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:43:53.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:43:53 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:43:53.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:43:53 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:43:53.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:43:53 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:43:53.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:43:53 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:43:53.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:43:53 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:43:53.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:43:53 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:43:53.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:43:53 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:43:54.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:43:53 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:43:54.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:43:53 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:43:54.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:43:53 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:43:54.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:43:53 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:43:54.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:43:54 smithi138 ceph-mon[36954]: pgmap v6230: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:43:54.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:43:54 smithi100 ceph-mon[32599]: pgmap v6230: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:43:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:43:54 smithi167 ceph-mon[37140]: pgmap v6230: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:43:57.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:43:57 smithi138 ceph-mon[36954]: pgmap v6231: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:43:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:43:57 smithi100 ceph-mon[32599]: pgmap v6231: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:43:57.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:43:57 smithi167 ceph-mon[37140]: pgmap v6231: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:43:59.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:43:59 smithi138 ceph-mon[36954]: pgmap v6232: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:43:59.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:43:59 smithi100 ceph-mon[32599]: pgmap v6232: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:43:59.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:43:59 smithi167 ceph-mon[37140]: pgmap v6232: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:44:01.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:44:01 smithi138 ceph-mon[36954]: pgmap v6233: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:44:01.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:44:01 smithi100 ceph-mon[32599]: pgmap v6233: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:44:01.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:44:01 smithi167 ceph-mon[37140]: pgmap v6233: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:44:03.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:44:03 smithi138 ceph-mon[36954]: pgmap v6234: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:44:03.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:44:03 smithi100 ceph-mon[32599]: pgmap v6234: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:44:03.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:44:03 smithi167 ceph-mon[37140]: pgmap v6234: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:44:05.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:44:05 smithi138 ceph-mon[36954]: pgmap v6235: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:44:05.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:44:05 smithi100 ceph-mon[32599]: pgmap v6235: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:44:05.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:44:05 smithi167 ceph-mon[37140]: pgmap v6235: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:44:07.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:44:07 smithi138 ceph-mon[36954]: pgmap v6236: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:44:07.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:44:07 smithi100 ceph-mon[32599]: pgmap v6236: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:44:07.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:44:07 smithi167 ceph-mon[37140]: pgmap v6236: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:44:07.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:44:07.769+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:44:07.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:44:07.770+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:44:07.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:44:07.770+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:44:07.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:44:07.770+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:44:07.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:44:07.770+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:44:08.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:44:08 smithi138 ceph-mon[36954]: osdmap e436: 12 total, 12 up, 12 in 2024-07-26T16:44:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:44:08 smithi100 ceph-mon[32599]: osdmap e436: 12 total, 12 up, 12 in 2024-07-26T16:44:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:44:08 smithi167 ceph-mon[37140]: osdmap e436: 12 total, 12 up, 12 in 2024-07-26T16:44:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:44:09 smithi138 ceph-mon[36954]: pgmap v6238: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:44:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:44:09 smithi100 ceph-mon[32599]: pgmap v6238: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:44:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:44:09 smithi167 ceph-mon[37140]: pgmap v6238: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:44:11.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:44:11 smithi138 ceph-mon[36954]: pgmap v6239: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:44:11.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:44:11 smithi100 ceph-mon[32599]: pgmap v6239: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:44:11.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:44:11 smithi167 ceph-mon[37140]: pgmap v6239: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:44:13.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:44:13 smithi138 ceph-mon[36954]: pgmap v6240: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:44:13.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:44:13 smithi100 ceph-mon[32599]: pgmap v6240: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:44:13.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:44:13 smithi167 ceph-mon[37140]: pgmap v6240: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:44:15.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:44:15 smithi138 ceph-mon[36954]: pgmap v6241: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:44:15.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:44:15 smithi100 ceph-mon[32599]: pgmap v6241: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:44:15.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:44:15 smithi167 ceph-mon[37140]: pgmap v6241: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:44:17.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:44:17 smithi138 ceph-mon[36954]: pgmap v6242: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:44:17.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:44:17 smithi138 ceph-mon[36954]: osdmap e437: 12 total, 12 up, 12 in 2024-07-26T16:44:17.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:44:17 smithi100 ceph-mon[32599]: pgmap v6242: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:44:17.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:44:17 smithi100 ceph-mon[32599]: osdmap e437: 12 total, 12 up, 12 in 2024-07-26T16:44:17.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:44:17 smithi167 ceph-mon[37140]: pgmap v6242: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:44:17.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:44:17 smithi167 ceph-mon[37140]: osdmap e437: 12 total, 12 up, 12 in 2024-07-26T16:44:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:44:19 smithi138 ceph-mon[36954]: pgmap v6244: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:44:19.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:44:19 smithi100 ceph-mon[32599]: pgmap v6244: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:44:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:44:19 smithi167 ceph-mon[37140]: pgmap v6244: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:44:21.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:44:21 smithi138 ceph-mon[36954]: pgmap v6245: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:44:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:44:21 smithi100 ceph-mon[32599]: pgmap v6245: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:44:21.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:44:21 smithi167 ceph-mon[37140]: pgmap v6245: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:44:22.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:44:22.786+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:44:22.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:44:22.786+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:44:22.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:44:22.786+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:44:22.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:44:22.786+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:44:22.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:44:22.786+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:44:23.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:44:23 smithi100 ceph-mon[32599]: pgmap v6246: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:44:23.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:44:23 smithi167 ceph-mon[37140]: pgmap v6246: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:44:23.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:44:23 smithi138 ceph-mon[36954]: pgmap v6246: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:44:25.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:44:25 smithi100 ceph-mon[32599]: pgmap v6247: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:44:25.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:44:25 smithi167 ceph-mon[37140]: pgmap v6247: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:44:25.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:44:25 smithi138 ceph-mon[36954]: pgmap v6247: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:44:27.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:44:27 smithi100 ceph-mon[32599]: pgmap v6248: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:44:27.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:44:27 smithi167 ceph-mon[37140]: pgmap v6248: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:44:27.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:44:27 smithi138 ceph-mon[36954]: pgmap v6248: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:44:29.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:44:29 smithi100 ceph-mon[32599]: pgmap v6249: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:44:29.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:44:29 smithi167 ceph-mon[37140]: pgmap v6249: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:44:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:44:29 smithi138 ceph-mon[36954]: pgmap v6249: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:44:31.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:44:31 smithi100 ceph-mon[32599]: pgmap v6250: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:44:31.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:44:31 smithi167 ceph-mon[37140]: pgmap v6250: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:44:31.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:44:31 smithi138 ceph-mon[36954]: pgmap v6250: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:44:32.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:44:32 smithi138 ceph-mon[36954]: pgmap v6251: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:44:32.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:44:32 smithi100 ceph-mon[32599]: pgmap v6251: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:44:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:44:32 smithi167 ceph-mon[37140]: pgmap v6251: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:44:35.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:44:35 smithi138 ceph-mon[36954]: pgmap v6252: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:44:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:44:35 smithi100 ceph-mon[32599]: pgmap v6252: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:44:35.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:44:35 smithi167 ceph-mon[37140]: pgmap v6252: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:44:37.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:44:37 smithi138 ceph-mon[36954]: pgmap v6253: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:44:37.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:44:37 smithi100 ceph-mon[32599]: pgmap v6253: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:44:37.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:44:37 smithi167 ceph-mon[37140]: pgmap v6253: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:44:37.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:44:37.802+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:44:37.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:44:37.802+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:44:37.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:44:37.802+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:44:37.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:44:37.802+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:44:37.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:44:37.802+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:44:39.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:44:39 smithi138 ceph-mon[36954]: pgmap v6254: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:44:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:44:39 smithi100 ceph-mon[32599]: pgmap v6254: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:44:39.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:44:39 smithi167 ceph-mon[37140]: pgmap v6254: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:44:41.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:44:41 smithi138 ceph-mon[36954]: pgmap v6255: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:44:41.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:44:41 smithi100 ceph-mon[32599]: pgmap v6255: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:44:41.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:44:41 smithi167 ceph-mon[37140]: pgmap v6255: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:44:43.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:44:43 smithi138 ceph-mon[36954]: pgmap v6256: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:44:43.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:44:43 smithi100 ceph-mon[32599]: pgmap v6256: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:44:43.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:44:43 smithi167 ceph-mon[37140]: pgmap v6256: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:44:45.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:44:45 smithi138 ceph-mon[36954]: pgmap v6257: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:44:45.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:44:45 smithi100 ceph-mon[32599]: pgmap v6257: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:44:45.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:44:45 smithi167 ceph-mon[37140]: pgmap v6257: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:44:47.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:44:47 smithi138 ceph-mon[36954]: pgmap v6258: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:44:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:44:47 smithi100 ceph-mon[32599]: pgmap v6258: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:44:47.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:44:47 smithi167 ceph-mon[37140]: pgmap v6258: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:44:49.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:44:49 smithi138 ceph-mon[36954]: pgmap v6259: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:44:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:44:49 smithi100 ceph-mon[32599]: pgmap v6259: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:44:49.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:44:49 smithi167 ceph-mon[37140]: pgmap v6259: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:44:51.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:44:51 smithi138 ceph-mon[36954]: pgmap v6260: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:44:51.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:44:51 smithi100 ceph-mon[32599]: pgmap v6260: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:44:51.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:44:51 smithi167 ceph-mon[37140]: pgmap v6260: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:44:52.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:44:52.807+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:44:52.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:44:52.808+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:44:52.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:44:52.808+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:44:52.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:44:52.808+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:44:52.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:44:52.808+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:44:53.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:44:53 smithi138 ceph-mon[36954]: pgmap v6261: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:44:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:44:53 smithi100 ceph-mon[32599]: pgmap v6261: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:44:53.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:44:53 smithi167 ceph-mon[37140]: pgmap v6261: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:44:54.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:44:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:44:54.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:44:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:44:54.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:44:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:44:54.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:44:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:44:54.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:44:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:44:54.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:44:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:44:54.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:44:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:44:54.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:44:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:44:54.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:44:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:44:54.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:44:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:44:54.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:44:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:44:54.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:44:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:44:55.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:44:55 smithi138 ceph-mon[36954]: pgmap v6262: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:44:55.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:44:55 smithi100 ceph-mon[32599]: pgmap v6262: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:44:55.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:44:55 smithi167 ceph-mon[37140]: pgmap v6262: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:44:57.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:44:57 smithi138 ceph-mon[36954]: pgmap v6263: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:44:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:44:57 smithi100 ceph-mon[32599]: pgmap v6263: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:44:57.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:44:57 smithi167 ceph-mon[37140]: pgmap v6263: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:44:59.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:44:59 smithi138 ceph-mon[36954]: pgmap v6264: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:44:59.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:44:59 smithi100 ceph-mon[32599]: pgmap v6264: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:44:59.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:44:59 smithi167 ceph-mon[37140]: pgmap v6264: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:45:01.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:45:01 smithi138 ceph-mon[36954]: pgmap v6265: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:45:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:45:01 smithi100 ceph-mon[32599]: pgmap v6265: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:45:01.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:45:01 smithi167 ceph-mon[37140]: pgmap v6265: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:45:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:45:03 smithi100 ceph-mon[32599]: pgmap v6266: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:45:03.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:45:03 smithi167 ceph-mon[37140]: pgmap v6266: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:45:03.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:45:03 smithi138 ceph-mon[36954]: pgmap v6266: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:45:04.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:45:04.518+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T16:45:04.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:45:04.518+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T16:45:04.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:45:04.518+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 85 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T16:45:04.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:45:04.518+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 85 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T16:45:05.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:45:05 smithi100 ceph-mon[32599]: pgmap v6267: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:45:05.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:45:05 smithi167 ceph-mon[37140]: pgmap v6267: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:45:05.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:45:05 smithi138 ceph-mon[36954]: pgmap v6267: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:45:07.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:45:07 smithi100 ceph-mon[32599]: pgmap v6268: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:45:07.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:45:07 smithi167 ceph-mon[37140]: pgmap v6268: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:45:07.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:45:07 smithi138 ceph-mon[36954]: pgmap v6268: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:45:07.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:45:07.822+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:45:07.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:45:07.823+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:45:07.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:45:07.823+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:45:07.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:45:07.823+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:45:07.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:45:07.823+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:45:08.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:45:08 smithi100 ceph-mon[32599]: osdmap e438: 12 total, 12 up, 12 in 2024-07-26T16:45:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:45:08 smithi167 ceph-mon[37140]: osdmap e438: 12 total, 12 up, 12 in 2024-07-26T16:45:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:45:08 smithi138 ceph-mon[36954]: osdmap e438: 12 total, 12 up, 12 in 2024-07-26T16:45:09.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:45:09 smithi100 ceph-mon[32599]: pgmap v6270: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:45:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:45:09 smithi167 ceph-mon[37140]: pgmap v6270: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:45:09.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:45:09 smithi138 ceph-mon[36954]: pgmap v6270: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:45:11.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:45:11 smithi100 ceph-mon[32599]: pgmap v6271: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:45:11.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:45:11 smithi167 ceph-mon[37140]: pgmap v6271: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:45:11.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:45:11 smithi138 ceph-mon[36954]: pgmap v6271: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:45:13.493 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:45:13 smithi138 ceph-mon[36954]: pgmap v6272: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:45:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:45:13 smithi100 ceph-mon[32599]: pgmap v6272: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:45:13.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:45:13 smithi167 ceph-mon[37140]: pgmap v6272: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:45:15.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:45:15 smithi100 ceph-mon[32599]: pgmap v6273: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:45:15.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:45:15 smithi167 ceph-mon[37140]: pgmap v6273: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:45:15.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:45:15 smithi138 ceph-mon[36954]: pgmap v6273: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:45:17.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:45:17 smithi100 ceph-mon[32599]: pgmap v6274: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:45:17.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:45:17 smithi100 ceph-mon[32599]: osdmap e439: 12 total, 12 up, 12 in 2024-07-26T16:45:17.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:45:17 smithi167 ceph-mon[37140]: pgmap v6274: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:45:17.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:45:17 smithi167 ceph-mon[37140]: osdmap e439: 12 total, 12 up, 12 in 2024-07-26T16:45:17.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:45:17 smithi138 ceph-mon[36954]: pgmap v6274: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:45:17.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:45:17 smithi138 ceph-mon[36954]: osdmap e439: 12 total, 12 up, 12 in 2024-07-26T16:45:19.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:45:19 smithi100 ceph-mon[32599]: pgmap v6276: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:45:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:45:19 smithi167 ceph-mon[37140]: pgmap v6276: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:45:19.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:45:19 smithi138 ceph-mon[36954]: pgmap v6276: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:45:21.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:45:21 smithi100 ceph-mon[32599]: pgmap v6277: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:45:21.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:45:21 smithi167 ceph-mon[37140]: pgmap v6277: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:45:21.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:45:21 smithi138 ceph-mon[36954]: pgmap v6277: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:45:22.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:45:22.836+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:45:22.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:45:22.836+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:45:22.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:45:22.836+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:45:22.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:45:22.836+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:45:22.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:45:22.836+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:45:23.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:45:23 smithi100 ceph-mon[32599]: pgmap v6278: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:45:23.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:45:23 smithi167 ceph-mon[37140]: pgmap v6278: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:45:23.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:45:23 smithi138 ceph-mon[36954]: pgmap v6278: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:45:25.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:45:25 smithi100 ceph-mon[32599]: pgmap v6279: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:45:25.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:45:25 smithi167 ceph-mon[37140]: pgmap v6279: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:45:25.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:45:25 smithi138 ceph-mon[36954]: pgmap v6279: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:45:27.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:45:27 smithi100 ceph-mon[32599]: pgmap v6280: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:45:27.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:45:27 smithi167 ceph-mon[37140]: pgmap v6280: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:45:27.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:45:27 smithi138 ceph-mon[36954]: pgmap v6280: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:45:29.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:45:29 smithi100 ceph-mon[32599]: pgmap v6281: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:45:29.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:45:29 smithi167 ceph-mon[37140]: pgmap v6281: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:45:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:45:29 smithi138 ceph-mon[36954]: pgmap v6281: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:45:31.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:45:31 smithi167 ceph-mon[37140]: pgmap v6282: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-07-26T16:45:31.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:45:31 smithi138 ceph-mon[36954]: pgmap v6282: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-07-26T16:45:31.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:45:31 smithi100 ceph-mon[32599]: pgmap v6282: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-07-26T16:45:33.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:45:33 smithi167 ceph-mon[37140]: pgmap v6283: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T16:45:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:45:33 smithi138 ceph-mon[36954]: pgmap v6283: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T16:45:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:45:33 smithi100 ceph-mon[32599]: pgmap v6283: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T16:45:35.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:45:35 smithi167 ceph-mon[37140]: pgmap v6284: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:45:35.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:45:35 smithi138 ceph-mon[36954]: pgmap v6284: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:45:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:45:35 smithi100 ceph-mon[32599]: pgmap v6284: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:45:37.526 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:45:37 smithi167 ceph-mon[37140]: pgmap v6285: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T16:45:37.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:45:37 smithi138 ceph-mon[36954]: pgmap v6285: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T16:45:37.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:45:37 smithi100 ceph-mon[32599]: pgmap v6285: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T16:45:37.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:45:37.851+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:45:37.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:45:37.852+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:45:37.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:45:37.852+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:45:37.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:45:37.852+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:45:37.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:45:37.852+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:45:39.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:45:39 smithi167 ceph-mon[37140]: pgmap v6286: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:45:39.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:45:39 smithi138 ceph-mon[36954]: pgmap v6286: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:45:39.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:45:39 smithi100 ceph-mon[32599]: pgmap v6286: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:45:41.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:45:41 smithi167 ceph-mon[37140]: pgmap v6287: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:45:41.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:45:41 smithi138 ceph-mon[36954]: pgmap v6287: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:45:41.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:45:41 smithi100 ceph-mon[32599]: pgmap v6287: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:45:43.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:45:43 smithi167 ceph-mon[37140]: pgmap v6288: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:45:43.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:45:43 smithi138 ceph-mon[36954]: pgmap v6288: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:45:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:45:43 smithi100 ceph-mon[32599]: pgmap v6288: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:45:45.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:45:45 smithi167 ceph-mon[37140]: pgmap v6289: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:45:45.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:45:45 smithi138 ceph-mon[36954]: pgmap v6289: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:45:45.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:45:45 smithi100 ceph-mon[32599]: pgmap v6289: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:45:47.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:45:47 smithi167 ceph-mon[37140]: pgmap v6290: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:45:47.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:45:47 smithi138 ceph-mon[36954]: pgmap v6290: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:45:47.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:45:47 smithi100 ceph-mon[32599]: pgmap v6290: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:45:49.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:45:49 smithi138 ceph-mon[36954]: pgmap v6291: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:45:49.972 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:45:49 smithi167 ceph-mon[37140]: pgmap v6291: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:45:49.972 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:45:49 smithi100 ceph-mon[32599]: pgmap v6291: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:45:51.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:45:51 smithi100 ceph-mon[32599]: pgmap v6292: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:45:51.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:45:51 smithi167 ceph-mon[37140]: pgmap v6292: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:45:51.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:45:51 smithi138 ceph-mon[36954]: pgmap v6292: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:45:52.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:45:52.867+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:45:52.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:45:52.868+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:45:52.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:45:52.868+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:45:52.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:45:52.868+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:45:52.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:45:52.868+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:45:53.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:45:53 smithi138 ceph-mon[36954]: pgmap v6293: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:45:53.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:45:53 smithi100 ceph-mon[32599]: pgmap v6293: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:45:53.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:45:53 smithi167 ceph-mon[37140]: pgmap v6293: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:45:54.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:45:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:45:54.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:45:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:45:54.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:45:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:45:55.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:45:55 smithi138 ceph-mon[36954]: pgmap v6294: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:45:55.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:45:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:45:55.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:45:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:45:55.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:45:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:45:55.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:45:55 smithi100 ceph-mon[32599]: pgmap v6294: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:45:55.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:45:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:45:55.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:45:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:45:55.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:45:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:45:55.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:45:55 smithi167 ceph-mon[37140]: pgmap v6294: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:45:55.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:45:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:45:55.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:45:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:45:55.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:45:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:45:57.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:45:57 smithi138 ceph-mon[36954]: pgmap v6295: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:45:57.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:45:57 smithi100 ceph-mon[32599]: pgmap v6295: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:45:57.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:45:57 smithi167 ceph-mon[37140]: pgmap v6295: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:45:59.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:45:59 smithi138 ceph-mon[36954]: pgmap v6296: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:45:59.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:45:59 smithi100 ceph-mon[32599]: pgmap v6296: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:45:59.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:45:59 smithi167 ceph-mon[37140]: pgmap v6296: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:46:01.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:46:01 smithi138 ceph-mon[36954]: pgmap v6297: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:46:01.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:46:01 smithi100 ceph-mon[32599]: pgmap v6297: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:46:01.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:46:01 smithi167 ceph-mon[37140]: pgmap v6297: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:46:03.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:46:03 smithi138 ceph-mon[36954]: pgmap v6298: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:46:03.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:46:03 smithi100 ceph-mon[32599]: pgmap v6298: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:46:03.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:46:03 smithi167 ceph-mon[37140]: pgmap v6298: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:46:05.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:46:05 smithi138 ceph-mon[36954]: pgmap v6299: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:46:05.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:46:05 smithi100 ceph-mon[32599]: pgmap v6299: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:46:05.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:46:05 smithi167 ceph-mon[37140]: pgmap v6299: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:46:07.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:46:07 smithi138 ceph-mon[36954]: pgmap v6300: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:46:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:46:07 smithi100 ceph-mon[32599]: pgmap v6300: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:46:07.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:46:07 smithi167 ceph-mon[37140]: pgmap v6300: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:46:07.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:46:07.876+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:46:07.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:46:07.877+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:46:07.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:46:07.877+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:46:07.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:46:07.877+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:46:07.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:46:07.877+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:46:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:46:08 smithi138 ceph-mon[36954]: osdmap e440: 12 total, 12 up, 12 in 2024-07-26T16:46:08.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:46:08 smithi100 ceph-mon[32599]: osdmap e440: 12 total, 12 up, 12 in 2024-07-26T16:46:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:46:08 smithi167 ceph-mon[37140]: osdmap e440: 12 total, 12 up, 12 in 2024-07-26T16:46:09.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:46:09 smithi138 ceph-mon[36954]: pgmap v6302: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:46:09.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:46:09 smithi100 ceph-mon[32599]: pgmap v6302: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:46:09.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:46:09 smithi167 ceph-mon[37140]: pgmap v6302: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:46:11.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:46:11 smithi138 ceph-mon[36954]: pgmap v6303: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:46:11.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:46:11 smithi100 ceph-mon[32599]: pgmap v6303: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:46:11.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:46:11 smithi167 ceph-mon[37140]: pgmap v6303: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:46:12.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:46:12 smithi138 ceph-mon[36954]: pgmap v6304: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:46:12.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:46:12 smithi100 ceph-mon[32599]: pgmap v6304: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:46:12.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:46:12 smithi167 ceph-mon[37140]: pgmap v6304: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:46:15.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:46:15 smithi138 ceph-mon[36954]: pgmap v6305: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:46:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:46:15 smithi100 ceph-mon[32599]: pgmap v6305: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:46:15.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:46:15 smithi167 ceph-mon[37140]: pgmap v6305: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:46:17.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:46:17 smithi138 ceph-mon[36954]: pgmap v6306: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:46:17.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:46:17 smithi138 ceph-mon[36954]: osdmap e441: 12 total, 12 up, 12 in 2024-07-26T16:46:17.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:46:17 smithi100 ceph-mon[32599]: pgmap v6306: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:46:17.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:46:17 smithi100 ceph-mon[32599]: osdmap e441: 12 total, 12 up, 12 in 2024-07-26T16:46:17.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:46:17 smithi167 ceph-mon[37140]: pgmap v6306: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:46:17.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:46:17 smithi167 ceph-mon[37140]: osdmap e441: 12 total, 12 up, 12 in 2024-07-26T16:46:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:46:19 smithi138 ceph-mon[36954]: pgmap v6308: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:46:19.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:46:19 smithi100 ceph-mon[32599]: pgmap v6308: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:46:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:46:19 smithi167 ceph-mon[37140]: pgmap v6308: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:46:21.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:46:21 smithi100 ceph-mon[32599]: pgmap v6309: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:46:21.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:46:21 smithi167 ceph-mon[37140]: pgmap v6309: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:46:21.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:46:21 smithi138 ceph-mon[36954]: pgmap v6309: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:46:22.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:46:22.893+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:46:22.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:46:22.893+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:46:22.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:46:22.893+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:46:22.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:46:22.893+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:46:22.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:46:22.893+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:46:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:46:23 smithi100 ceph-mon[32599]: pgmap v6310: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:46:23.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:46:23 smithi167 ceph-mon[37140]: pgmap v6310: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:46:23.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:46:23 smithi138 ceph-mon[36954]: pgmap v6310: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:46:25.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:46:25 smithi100 ceph-mon[32599]: pgmap v6311: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:46:25.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:46:25 smithi167 ceph-mon[37140]: pgmap v6311: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:46:25.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:46:25 smithi138 ceph-mon[36954]: pgmap v6311: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:46:27.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:46:27 smithi100 ceph-mon[32599]: pgmap v6312: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:46:27.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:46:27 smithi167 ceph-mon[37140]: pgmap v6312: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:46:27.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:46:27 smithi138 ceph-mon[36954]: pgmap v6312: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:46:29.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:46:29 smithi100 ceph-mon[32599]: pgmap v6313: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:46:29.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:46:29 smithi167 ceph-mon[37140]: pgmap v6313: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:46:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:46:29 smithi138 ceph-mon[36954]: pgmap v6313: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:46:31.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:46:31 smithi100 ceph-mon[32599]: pgmap v6314: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:46:31.526 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:46:31 smithi167 ceph-mon[37140]: pgmap v6314: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:46:31.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:46:31 smithi138 ceph-mon[36954]: pgmap v6314: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:46:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:46:33 smithi100 ceph-mon[32599]: pgmap v6315: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:46:33.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:46:33 smithi167 ceph-mon[37140]: pgmap v6315: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:46:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:46:33 smithi138 ceph-mon[36954]: pgmap v6315: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:46:35.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:46:35 smithi100 ceph-mon[32599]: pgmap v6316: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:46:35.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:46:35 smithi167 ceph-mon[37140]: pgmap v6316: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:46:35.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:46:35 smithi138 ceph-mon[36954]: pgmap v6316: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:46:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:46:37 smithi100 ceph-mon[32599]: pgmap v6317: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:46:37.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:46:37 smithi167 ceph-mon[37140]: pgmap v6317: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:46:37.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:46:37 smithi138 ceph-mon[36954]: pgmap v6317: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:46:37.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:46:37.903+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:46:37.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:46:37.904+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:46:37.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:46:37.904+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:46:37.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:46:37.904+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:46:37.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:46:37.904+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:46:39.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:46:39 smithi100 ceph-mon[32599]: pgmap v6318: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:46:39.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:46:39 smithi167 ceph-mon[37140]: pgmap v6318: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:46:39.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:46:39 smithi138 ceph-mon[36954]: pgmap v6318: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:46:41.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:46:41 smithi100 ceph-mon[32599]: pgmap v6319: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:46:41.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:46:41 smithi167 ceph-mon[37140]: pgmap v6319: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:46:41.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:46:41 smithi138 ceph-mon[36954]: pgmap v6319: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:46:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:46:43 smithi100 ceph-mon[32599]: pgmap v6320: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:46:43.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:46:43 smithi167 ceph-mon[37140]: pgmap v6320: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:46:43.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:46:43 smithi138 ceph-mon[36954]: pgmap v6320: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:46:45.493 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:46:45 smithi138 ceph-mon[36954]: pgmap v6321: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:46:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:46:45 smithi100 ceph-mon[32599]: pgmap v6321: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:46:45.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:46:45 smithi167 ceph-mon[37140]: pgmap v6321: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:46:47.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:46:47 smithi100 ceph-mon[32599]: pgmap v6322: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:46:47.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:46:47 smithi167 ceph-mon[37140]: pgmap v6322: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:46:47.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:46:47 smithi138 ceph-mon[36954]: pgmap v6322: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:46:49.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:46:49 smithi167 ceph-mon[37140]: pgmap v6323: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:46:49.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:46:49 smithi138 ceph-mon[36954]: pgmap v6323: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:46:49.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:46:49 smithi100 ceph-mon[32599]: pgmap v6323: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:46:51.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:46:51 smithi167 ceph-mon[37140]: pgmap v6324: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:46:51.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:46:51 smithi100 ceph-mon[32599]: pgmap v6324: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:46:51.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:46:51 smithi138 ceph-mon[36954]: pgmap v6324: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:46:52.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:46:52.908+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:46:52.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:46:52.908+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:46:52.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:46:52.908+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:46:52.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:46:52.908+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:46:52.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:46:52.908+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:46:53.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:46:53 smithi167 ceph-mon[37140]: pgmap v6325: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:46:53.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:46:53 smithi138 ceph-mon[36954]: pgmap v6325: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:46:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:46:53 smithi100 ceph-mon[32599]: pgmap v6325: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:46:55.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:46:55 smithi167 ceph-mon[37140]: pgmap v6326: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:46:55.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:46:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:46:55.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:46:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:46:55.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:46:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:46:55.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:46:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:46:55.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:46:55 smithi138 ceph-mon[36954]: pgmap v6326: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:46:55.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:46:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:46:55.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:46:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:46:55.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:46:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:46:55.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:46:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:46:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:46:55 smithi100 ceph-mon[32599]: pgmap v6326: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:46:55.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:46:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:46:55.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:46:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:46:55.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:46:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:46:55.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:46:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:46:57.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:46:57 smithi167 ceph-mon[37140]: pgmap v6327: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:46:57.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:46:57 smithi138 ceph-mon[36954]: pgmap v6327: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:46:57.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:46:57 smithi100 ceph-mon[32599]: pgmap v6327: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:46:59.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:46:59 smithi167 ceph-mon[37140]: pgmap v6328: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:46:59.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:46:59 smithi138 ceph-mon[36954]: pgmap v6328: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:46:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:46:59 smithi100 ceph-mon[32599]: pgmap v6328: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:47:01.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:47:01 smithi167 ceph-mon[37140]: pgmap v6329: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:47:01.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:47:01 smithi138 ceph-mon[36954]: pgmap v6329: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:47:01.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:47:01 smithi100 ceph-mon[32599]: pgmap v6329: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:47:03.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:47:03 smithi167 ceph-mon[37140]: pgmap v6330: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:47:03.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:47:03 smithi138 ceph-mon[36954]: pgmap v6330: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:47:03.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:47:03 smithi100 ceph-mon[32599]: pgmap v6330: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:47:05.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:47:05 smithi167 ceph-mon[37140]: pgmap v6331: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T16:47:05.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:47:05 smithi138 ceph-mon[36954]: pgmap v6331: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T16:47:05.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:47:05 smithi100 ceph-mon[32599]: pgmap v6331: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T16:47:07.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:47:07 smithi138 ceph-mon[36954]: pgmap v6332: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:47:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:47:07 smithi100 ceph-mon[32599]: pgmap v6332: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:47:07.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:47:07 smithi167 ceph-mon[37140]: pgmap v6332: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:47:07.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:47:07.913+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:47:07.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:47:07.914+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:47:07.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:47:07.914+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:47:07.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:47:07.914+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:47:07.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:47:07.914+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:47:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:47:08 smithi138 ceph-mon[36954]: osdmap e442: 12 total, 12 up, 12 in 2024-07-26T16:47:08.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:47:08 smithi100 ceph-mon[32599]: osdmap e442: 12 total, 12 up, 12 in 2024-07-26T16:47:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:47:08 smithi167 ceph-mon[37140]: osdmap e442: 12 total, 12 up, 12 in 2024-07-26T16:47:09.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:47:09 smithi138 ceph-mon[36954]: pgmap v6334: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:47:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:47:09 smithi100 ceph-mon[32599]: pgmap v6334: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:47:09.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:47:09 smithi167 ceph-mon[37140]: pgmap v6334: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:47:11.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:47:11 smithi138 ceph-mon[36954]: pgmap v6335: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T16:47:11.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:47:11 smithi100 ceph-mon[32599]: pgmap v6335: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T16:47:11.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:47:11 smithi167 ceph-mon[37140]: pgmap v6335: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T16:47:13.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:47:13 smithi138 ceph-mon[36954]: pgmap v6336: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T16:47:13.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:47:13 smithi100 ceph-mon[32599]: pgmap v6336: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T16:47:13.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:47:13 smithi167 ceph-mon[37140]: pgmap v6336: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T16:47:15.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:47:15 smithi138 ceph-mon[36954]: pgmap v6337: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:47:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:47:15 smithi100 ceph-mon[32599]: pgmap v6337: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:47:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:47:15 smithi167 ceph-mon[37140]: pgmap v6337: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:47:17.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:47:17 smithi138 ceph-mon[36954]: pgmap v6338: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:47:17.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:47:17 smithi138 ceph-mon[36954]: osdmap e443: 12 total, 12 up, 12 in 2024-07-26T16:47:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:47:17 smithi100 ceph-mon[32599]: pgmap v6338: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:47:17.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:47:17 smithi100 ceph-mon[32599]: osdmap e443: 12 total, 12 up, 12 in 2024-07-26T16:47:17.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:47:17 smithi167 ceph-mon[37140]: pgmap v6338: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:47:17.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:47:17 smithi167 ceph-mon[37140]: osdmap e443: 12 total, 12 up, 12 in 2024-07-26T16:47:19.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:47:19 smithi138 ceph-mon[36954]: pgmap v6340: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:47:19.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:47:19 smithi100 ceph-mon[32599]: pgmap v6340: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:47:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:47:19 smithi167 ceph-mon[37140]: pgmap v6340: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:47:21.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:47:21 smithi167 ceph-mon[37140]: pgmap v6341: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:47:21.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:47:21 smithi138 ceph-mon[36954]: pgmap v6341: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:47:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:47:21 smithi100 ceph-mon[32599]: pgmap v6341: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:47:22.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:47:22.919+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:47:22.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:47:22.923+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:47:22.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:47:22.923+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:47:22.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:47:22.923+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:47:22.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:47:22.923+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:47:23.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:47:23 smithi138 ceph-mon[36954]: pgmap v6342: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:47:23.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:47:23 smithi100 ceph-mon[32599]: pgmap v6342: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:47:23.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:47:23 smithi167 ceph-mon[37140]: pgmap v6342: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:47:25.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:47:25 smithi138 ceph-mon[36954]: pgmap v6343: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:47:25.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:47:25 smithi100 ceph-mon[32599]: pgmap v6343: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:47:25.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:47:25 smithi167 ceph-mon[37140]: pgmap v6343: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:47:27.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:47:27 smithi138 ceph-mon[36954]: pgmap v6344: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:47:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:47:27 smithi100 ceph-mon[32599]: pgmap v6344: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:47:27.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:47:27 smithi167 ceph-mon[37140]: pgmap v6344: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:47:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:47:29 smithi138 ceph-mon[36954]: pgmap v6345: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:47:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:47:29 smithi100 ceph-mon[32599]: pgmap v6345: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:47:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:47:29 smithi167 ceph-mon[37140]: pgmap v6345: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:47:31.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:47:31 smithi138 ceph-mon[36954]: pgmap v6346: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:47:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:47:31 smithi100 ceph-mon[32599]: pgmap v6346: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:47:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:47:31 smithi167 ceph-mon[37140]: pgmap v6346: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:47:33.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:47:33 smithi167 ceph-mon[37140]: pgmap v6347: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:47:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:47:33 smithi138 ceph-mon[36954]: pgmap v6347: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:47:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:47:33 smithi100 ceph-mon[32599]: pgmap v6347: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:47:35.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:47:35 smithi138 ceph-mon[36954]: pgmap v6348: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:47:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:47:35 smithi100 ceph-mon[32599]: pgmap v6348: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:47:35.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:47:35 smithi167 ceph-mon[37140]: pgmap v6348: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:47:37.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:47:37 smithi138 ceph-mon[36954]: pgmap v6349: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:47:37.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:47:37 smithi100 ceph-mon[32599]: pgmap v6349: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:47:37.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:47:37 smithi167 ceph-mon[37140]: pgmap v6349: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:47:37.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:47:37.939+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:47:37.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:47:37.939+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:47:37.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:47:37.939+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:47:37.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:47:37.939+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:47:37.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:47:37.939+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:47:39.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:47:39 smithi138 ceph-mon[36954]: pgmap v6350: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:47:39.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:47:39 smithi100 ceph-mon[32599]: pgmap v6350: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:47:39.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:47:39 smithi167 ceph-mon[37140]: pgmap v6350: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:47:40.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:47:40 smithi138 ceph-mon[36954]: pgmap v6351: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:47:40.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:47:40 smithi100 ceph-mon[32599]: pgmap v6351: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:47:40.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:47:40 smithi167 ceph-mon[37140]: pgmap v6351: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:47:43.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:47:43 smithi100 ceph-mon[32599]: pgmap v6352: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:47:43.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:47:43 smithi167 ceph-mon[37140]: pgmap v6352: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:47:43.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:47:43 smithi138 ceph-mon[36954]: pgmap v6352: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:47:45.493 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:47:45 smithi138 ceph-mon[36954]: pgmap v6353: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:47:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:47:45 smithi100 ceph-mon[32599]: pgmap v6353: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:47:45.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:47:45 smithi167 ceph-mon[37140]: pgmap v6353: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:47:47.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:47:47 smithi100 ceph-mon[32599]: pgmap v6354: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:47:47.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:47:47 smithi167 ceph-mon[37140]: pgmap v6354: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:47:47.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:47:47 smithi138 ceph-mon[36954]: pgmap v6354: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:47:49.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:47:49 smithi100 ceph-mon[32599]: pgmap v6355: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:47:49.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:47:49 smithi167 ceph-mon[37140]: pgmap v6355: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:47:49.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:47:49 smithi138 ceph-mon[36954]: pgmap v6355: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:47:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:47:51 smithi100 ceph-mon[32599]: pgmap v6356: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:47:51.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:47:51 smithi167 ceph-mon[37140]: pgmap v6356: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:47:51.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:47:51 smithi138 ceph-mon[36954]: pgmap v6356: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:47:52.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:47:52.954+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:47:52.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:47:52.955+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:47:52.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:47:52.955+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:47:52.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:47:52.955+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:47:52.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:47:52.955+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:47:53.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:47:53 smithi100 ceph-mon[32599]: pgmap v6357: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:47:53.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:47:53 smithi167 ceph-mon[37140]: pgmap v6357: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:47:53.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:47:53 smithi138 ceph-mon[36954]: pgmap v6357: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:47:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:47:55 smithi100 ceph-mon[32599]: pgmap v6358: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:47:55.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:47:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:47:55.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:47:55 smithi167 ceph-mon[37140]: pgmap v6358: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:47:55.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:47:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:47:55.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:47:55 smithi138 ceph-mon[36954]: pgmap v6358: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:47:55.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:47:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:47:56.234 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:47:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:47:56.235 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:47:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:47:56.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:47:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:47:56.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:47:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:47:56.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:47:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:47:56.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:47:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:47:56.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:47:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:47:56.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:47:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:47:57.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:47:57 smithi100 ceph-mon[32599]: pgmap v6359: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:47:57.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:47:57 smithi167 ceph-mon[37140]: pgmap v6359: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:47:57.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:47:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:47:57.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:47:57 smithi138 ceph-mon[36954]: pgmap v6359: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:47:59.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:47:59 smithi100 ceph-mon[32599]: pgmap v6360: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:47:59.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:47:59 smithi167 ceph-mon[37140]: pgmap v6360: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:47:59.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:47:59 smithi138 ceph-mon[36954]: pgmap v6360: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:48:01.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:01 smithi100 ceph-mon[32599]: pgmap v6361: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:48:01.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:01 smithi167 ceph-mon[37140]: pgmap v6361: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:48:01.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:01 smithi138 ceph-mon[36954]: pgmap v6361: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:48:03.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:48:03.038+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 msgr2=0x7fe5940c0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-07-26T16:48:03.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:48:03.038+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 msgr2=0x7fe5940c0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-07-26T16:48:03.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:48:03.038+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe59000b6b0 tx=0x7fe59002f000 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:48:03.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:48:03.038+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe59000b6b0 tx=0x7fe59002f000 comp rx=0 tx=0).stop 2024-07-26T16:48:03.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:48:03.038+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 msgr2=0x7fe5940c0480 unknown :-1 s=STATE_CLOSED l=1).tick see no progress in more than 10000000 us during connecting to v2:172.21.15.100:6800/3937009534, fault. 2024-07-26T16:48:03.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:48:03.038+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 msgr2=0x7fe5940c0480 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-07-26T16:48:03.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:48:03.038+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T16:48:03.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:48:03.038+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 0x7fe5940b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T16:48:03.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:48:03.038+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 0x7fe5940b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:48:03.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:48:03.039+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 0x7fe5940b5b30 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fe590004ee0 tx=0x7fe5900ac040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T16:48:03.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:03 smithi100 ceph-mon[32599]: pgmap v6362: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:48:03.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:03 smithi167 ceph-mon[37140]: pgmap v6362: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:48:03.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:03 smithi138 ceph-mon[36954]: pgmap v6362: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:48:04.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:48:04.544+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T16:48:04.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:48:04.544+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T16:48:04.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:48:04.544+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 86 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T16:48:04.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:48:04.544+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 86 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T16:48:05.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:05 smithi167 ceph-mon[37140]: pgmap v6363: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T16:48:05.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:05 smithi138 ceph-mon[36954]: pgmap v6363: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T16:48:05.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:05 smithi100 ceph-mon[32599]: pgmap v6363: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T16:48:07.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:07 smithi167 ceph-mon[37140]: pgmap v6364: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:48:07.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:07 smithi138 ceph-mon[36954]: pgmap v6364: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:48:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:07 smithi100 ceph-mon[32599]: pgmap v6364: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:48:07.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:48:07.960+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:48:07.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:48:07.962+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:48:07.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:48:07.962+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:48:07.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:48:07.962+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:48:07.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:48:07.962+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:48:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:08 smithi167 ceph-mon[37140]: osdmap e444: 12 total, 12 up, 12 in 2024-07-26T16:48:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:08 smithi138 ceph-mon[36954]: osdmap e444: 12 total, 12 up, 12 in 2024-07-26T16:48:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:08 smithi100 ceph-mon[32599]: osdmap e444: 12 total, 12 up, 12 in 2024-07-26T16:48:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:09 smithi167 ceph-mon[37140]: pgmap v6366: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:48:09.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:09 smithi138 ceph-mon[36954]: pgmap v6366: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:48:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:09 smithi100 ceph-mon[32599]: pgmap v6366: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:48:11.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:11 smithi167 ceph-mon[37140]: pgmap v6367: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T16:48:11.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:11 smithi138 ceph-mon[36954]: pgmap v6367: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T16:48:11.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:11 smithi100 ceph-mon[32599]: pgmap v6367: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T16:48:13.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:13 smithi167 ceph-mon[37140]: pgmap v6368: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T16:48:13.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:13 smithi138 ceph-mon[36954]: pgmap v6368: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T16:48:13.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:13 smithi100 ceph-mon[32599]: pgmap v6368: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T16:48:15.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:15 smithi138 ceph-mon[36954]: pgmap v6369: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:48:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:15 smithi100 ceph-mon[32599]: pgmap v6369: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:48:15.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:15 smithi167 ceph-mon[37140]: pgmap v6369: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:48:17.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:17 smithi138 ceph-mon[36954]: pgmap v6370: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:48:17.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:17 smithi138 ceph-mon[36954]: osdmap e445: 12 total, 12 up, 12 in 2024-07-26T16:48:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:17 smithi100 ceph-mon[32599]: pgmap v6370: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:48:17.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:17 smithi100 ceph-mon[32599]: osdmap e445: 12 total, 12 up, 12 in 2024-07-26T16:48:17.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:17 smithi167 ceph-mon[37140]: pgmap v6370: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:48:17.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:17 smithi167 ceph-mon[37140]: osdmap e445: 12 total, 12 up, 12 in 2024-07-26T16:48:19.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:19 smithi138 ceph-mon[36954]: pgmap v6372: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:48:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:19 smithi100 ceph-mon[32599]: pgmap v6372: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:48:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:19 smithi167 ceph-mon[37140]: pgmap v6372: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:48:21.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:21 smithi100 ceph-mon[32599]: pgmap v6373: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:48:21.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:21 smithi167 ceph-mon[37140]: pgmap v6373: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:48:21.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:21 smithi138 ceph-mon[36954]: pgmap v6373: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:48:22.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:48:22.977+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:48:22.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:48:22.978+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:48:22.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:48:22.978+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:48:22.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:48:22.978+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:48:22.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:48:22.978+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:48:23.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:23 smithi138 ceph-mon[36954]: pgmap v6374: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:48:23.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:23 smithi100 ceph-mon[32599]: pgmap v6374: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:48:23.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:23 smithi167 ceph-mon[37140]: pgmap v6374: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:48:25.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:25 smithi138 ceph-mon[36954]: pgmap v6375: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:48:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:25 smithi100 ceph-mon[32599]: pgmap v6375: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:48:25.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:25 smithi167 ceph-mon[37140]: pgmap v6375: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:48:27.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:27 smithi138 ceph-mon[36954]: pgmap v6376: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:48:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:27 smithi100 ceph-mon[32599]: pgmap v6376: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:48:27.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:27 smithi167 ceph-mon[37140]: pgmap v6376: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:48:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:29 smithi138 ceph-mon[36954]: pgmap v6377: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:48:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:29 smithi100 ceph-mon[32599]: pgmap v6377: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:48:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:29 smithi167 ceph-mon[37140]: pgmap v6377: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:48:31.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:31 smithi138 ceph-mon[36954]: pgmap v6378: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:48:31.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:31 smithi100 ceph-mon[32599]: pgmap v6378: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:48:31.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:31 smithi167 ceph-mon[37140]: pgmap v6378: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:48:32.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:32 smithi138 ceph-mon[36954]: pgmap v6379: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:48:32.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:32 smithi100 ceph-mon[32599]: pgmap v6379: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:48:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:32 smithi167 ceph-mon[37140]: pgmap v6379: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:48:35.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:35 smithi100 ceph-mon[32599]: pgmap v6380: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:48:35.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:35 smithi167 ceph-mon[37140]: pgmap v6380: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:48:35.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:35 smithi138 ceph-mon[36954]: pgmap v6380: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:48:37.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:37 smithi100 ceph-mon[32599]: pgmap v6381: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:48:37.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:37 smithi167 ceph-mon[37140]: pgmap v6381: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:48:37.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:37 smithi138 ceph-mon[36954]: pgmap v6381: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:48:37.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:48:37.991+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:48:37.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:48:37.991+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:48:37.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:48:37.992+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:48:37.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:48:37.992+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:48:37.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:48:37.992+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:48:39.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:39 smithi100 ceph-mon[32599]: pgmap v6382: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:48:39.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:39 smithi167 ceph-mon[37140]: pgmap v6382: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:48:39.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:39 smithi138 ceph-mon[36954]: pgmap v6382: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:48:41.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:41 smithi100 ceph-mon[32599]: pgmap v6383: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:48:41.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:41 smithi167 ceph-mon[37140]: pgmap v6383: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:48:41.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:41 smithi138 ceph-mon[36954]: pgmap v6383: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:48:43.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:43 smithi100 ceph-mon[32599]: pgmap v6384: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:48:43.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:43 smithi167 ceph-mon[37140]: pgmap v6384: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:48:43.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:43 smithi138 ceph-mon[36954]: pgmap v6384: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:48:45.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:45 smithi167 ceph-mon[37140]: pgmap v6385: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:48:45.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:45 smithi138 ceph-mon[36954]: pgmap v6385: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:48:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:45 smithi100 ceph-mon[32599]: pgmap v6385: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:48:47.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:47 smithi167 ceph-mon[37140]: pgmap v6386: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:48:47.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:47 smithi138 ceph-mon[36954]: pgmap v6386: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:48:47.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:47 smithi100 ceph-mon[32599]: pgmap v6386: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:48:49.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:49 smithi167 ceph-mon[37140]: pgmap v6387: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:48:49.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:49 smithi138 ceph-mon[36954]: pgmap v6387: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:48:49.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:49 smithi100 ceph-mon[32599]: pgmap v6387: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:48:51.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:51 smithi167 ceph-mon[37140]: pgmap v6388: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:48:51.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:51 smithi100 ceph-mon[32599]: pgmap v6388: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:48:51.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:51 smithi138 ceph-mon[36954]: pgmap v6388: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:48:53.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:48:53.007+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:48:53.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:48:53.008+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:48:53.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:48:53.008+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:48:53.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:48:53.008+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:48:53.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:48:53.008+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:48:53.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:53 smithi100 ceph-mon[32599]: pgmap v6389: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:48:53.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:53 smithi167 ceph-mon[37140]: pgmap v6389: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:48:53.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:53 smithi138 ceph-mon[36954]: pgmap v6389: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:48:55.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:55 smithi167 ceph-mon[37140]: pgmap v6390: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:48:55.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:55 smithi138 ceph-mon[36954]: pgmap v6390: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:48:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:55 smithi100 ceph-mon[32599]: pgmap v6390: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:48:56.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:48:56.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:48:56.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:48:56.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:48:56.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:48:56.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:48:56.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:48:56.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:48:56.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:48:56.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:48:56.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:48:56.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:48:56.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:48:56.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:48:56.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:48:56.526 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:48:56.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:48:56.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:48:56.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:48:56.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:48:56.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:48:56.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:48:56.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:48:56.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:48:56.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:48:56.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:48:56.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:48:56.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:48:56.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:48:56.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:48:56.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:48:56.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:48:56.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:48:56.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:48:56.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:48:56.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:48:56.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:48:56.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:48:56.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:48:56.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:48:56.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:48:56.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:48:56.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:48:56.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:48:56.747 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:48:56.747 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:48:56.747 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:48:56.747 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:48:57.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:57 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T16:48:57.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:57 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T16:48:57.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:57 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T16:48:57.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:57 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T16:48:57.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:57 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T16:48:57.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:57 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T16:48:57.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:57 smithi167 ceph-mon[37140]: pgmap v6391: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:48:57.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:57 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T16:48:57.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:57 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T16:48:57.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:57 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T16:48:57.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:57 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T16:48:57.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:57 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T16:48:57.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:57 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T16:48:57.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:57 smithi138 ceph-mon[36954]: pgmap v6391: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:48:57.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:57 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T16:48:57.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:57 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T16:48:57.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:57 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T16:48:57.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:57 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T16:48:57.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:57 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T16:48:57.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:57 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T16:48:57.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:57 smithi100 ceph-mon[32599]: pgmap v6391: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:48:59.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:48:59 smithi167 ceph-mon[37140]: pgmap v6392: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:48:59.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:48:59 smithi138 ceph-mon[36954]: pgmap v6392: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:48:59.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:48:59 smithi100 ceph-mon[32599]: pgmap v6392: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:49:01.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:49:01 smithi167 ceph-mon[37140]: pgmap v6393: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:49:01.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:49:01 smithi138 ceph-mon[36954]: pgmap v6393: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:49:01.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:49:01 smithi100 ceph-mon[32599]: pgmap v6393: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:49:03.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:49:03 smithi138 ceph-mon[36954]: pgmap v6394: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:49:03.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:49:03 smithi100 ceph-mon[32599]: pgmap v6394: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:49:03.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:49:03 smithi167 ceph-mon[37140]: pgmap v6394: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:49:05.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:49:05 smithi138 ceph-mon[36954]: pgmap v6395: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T16:49:05.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:49:05 smithi100 ceph-mon[32599]: pgmap v6395: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T16:49:05.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:49:05 smithi167 ceph-mon[37140]: pgmap v6395: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T16:49:07.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:49:07 smithi138 ceph-mon[36954]: pgmap v6396: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:49:07.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:49:07 smithi100 ceph-mon[32599]: pgmap v6396: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:49:07.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:49:07 smithi167 ceph-mon[37140]: pgmap v6396: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:49:08.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:49:08.023+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:49:08.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:49:08.024+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:49:08.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:49:08.024+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:49:08.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:49:08.024+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:49:08.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:49:08.024+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:49:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:49:08 smithi138 ceph-mon[36954]: osdmap e446: 12 total, 12 up, 12 in 2024-07-26T16:49:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:49:08 smithi100 ceph-mon[32599]: osdmap e446: 12 total, 12 up, 12 in 2024-07-26T16:49:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:49:08 smithi167 ceph-mon[37140]: osdmap e446: 12 total, 12 up, 12 in 2024-07-26T16:49:09.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:49:09 smithi138 ceph-mon[36954]: pgmap v6398: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:49:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:49:09 smithi100 ceph-mon[32599]: pgmap v6398: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:49:09.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:49:09 smithi167 ceph-mon[37140]: pgmap v6398: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:49:11.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:49:11 smithi138 ceph-mon[36954]: pgmap v6399: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T16:49:11.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:49:11 smithi100 ceph-mon[32599]: pgmap v6399: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T16:49:11.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:49:11 smithi167 ceph-mon[37140]: pgmap v6399: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T16:49:13.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:49:13 smithi138 ceph-mon[36954]: pgmap v6400: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:49:13.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:49:13 smithi100 ceph-mon[32599]: pgmap v6400: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:49:13.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:49:13 smithi167 ceph-mon[37140]: pgmap v6400: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:49:15.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:49:15 smithi138 ceph-mon[36954]: pgmap v6401: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:49:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:49:15 smithi100 ceph-mon[32599]: pgmap v6401: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:49:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:49:15 smithi167 ceph-mon[37140]: pgmap v6401: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:49:17.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:49:17 smithi138 ceph-mon[36954]: pgmap v6402: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:49:17.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:49:17 smithi138 ceph-mon[36954]: osdmap e447: 12 total, 12 up, 12 in 2024-07-26T16:49:17.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:49:17 smithi100 ceph-mon[32599]: pgmap v6402: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:49:17.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:49:17 smithi100 ceph-mon[32599]: osdmap e447: 12 total, 12 up, 12 in 2024-07-26T16:49:17.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:49:17 smithi167 ceph-mon[37140]: pgmap v6402: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:49:17.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:49:17 smithi167 ceph-mon[37140]: osdmap e447: 12 total, 12 up, 12 in 2024-07-26T16:49:19.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:49:19 smithi138 ceph-mon[36954]: pgmap v6404: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:49:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:49:19 smithi100 ceph-mon[32599]: pgmap v6404: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:49:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:49:19 smithi167 ceph-mon[37140]: pgmap v6404: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:49:21.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:49:21 smithi167 ceph-mon[37140]: pgmap v6405: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:49:21.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:49:21 smithi138 ceph-mon[36954]: pgmap v6405: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:49:21.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:49:21 smithi100 ceph-mon[32599]: pgmap v6405: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:49:23.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:49:23.040+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:49:23.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:49:23.040+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:49:23.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:49:23.040+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:49:23.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:49:23.040+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:49:23.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:49:23.040+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:49:23.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:49:23 smithi138 ceph-mon[36954]: pgmap v6406: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:49:23.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:49:23 smithi100 ceph-mon[32599]: pgmap v6406: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:49:23.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:49:23 smithi167 ceph-mon[37140]: pgmap v6406: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:49:25.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:49:25 smithi138 ceph-mon[36954]: pgmap v6407: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:49:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:49:25 smithi100 ceph-mon[32599]: pgmap v6407: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:49:25.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:49:25 smithi167 ceph-mon[37140]: pgmap v6407: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:49:27.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:49:27 smithi138 ceph-mon[36954]: pgmap v6408: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:49:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:49:27 smithi100 ceph-mon[32599]: pgmap v6408: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:49:27.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:49:27 smithi167 ceph-mon[37140]: pgmap v6408: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:49:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:49:29 smithi138 ceph-mon[36954]: pgmap v6409: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:49:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:49:29 smithi100 ceph-mon[32599]: pgmap v6409: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:49:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:49:29 smithi167 ceph-mon[37140]: pgmap v6409: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:49:31.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:49:31 smithi138 ceph-mon[36954]: pgmap v6410: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:49:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:49:31 smithi100 ceph-mon[32599]: pgmap v6410: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:49:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:49:31 smithi167 ceph-mon[37140]: pgmap v6410: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:49:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:49:33 smithi138 ceph-mon[36954]: pgmap v6411: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:49:33.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:49:33 smithi100 ceph-mon[32599]: pgmap v6411: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:49:33.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:49:33 smithi167 ceph-mon[37140]: pgmap v6411: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:49:34.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:49:34.557+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T16:49:34.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:49:34.557+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T16:49:34.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:49:34.557+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 87 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T16:49:34.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:49:34.557+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 87 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T16:49:35.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:49:35 smithi138 ceph-mon[36954]: pgmap v6412: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:49:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:49:35 smithi100 ceph-mon[32599]: pgmap v6412: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:49:35.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:49:35 smithi167 ceph-mon[37140]: pgmap v6412: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:49:37.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:49:37 smithi138 ceph-mon[36954]: pgmap v6413: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:49:37.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:49:37 smithi100 ceph-mon[32599]: pgmap v6413: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:49:37.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:49:37 smithi167 ceph-mon[37140]: pgmap v6413: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:49:38.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:49:38.056+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:49:38.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:49:38.056+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:49:38.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:49:38.056+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:49:38.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:49:38.056+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:49:38.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:49:38.056+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:49:39.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:49:39 smithi138 ceph-mon[36954]: pgmap v6414: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:49:39.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:49:39 smithi100 ceph-mon[32599]: pgmap v6414: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:49:39.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:49:39 smithi167 ceph-mon[37140]: pgmap v6414: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:49:41.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:49:41 smithi138 ceph-mon[36954]: pgmap v6415: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:49:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:49:41 smithi100 ceph-mon[32599]: pgmap v6415: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:49:41.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:49:41 smithi167 ceph-mon[37140]: pgmap v6415: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:49:43.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:49:43 smithi138 ceph-mon[36954]: pgmap v6416: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:49:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:49:43 smithi100 ceph-mon[32599]: pgmap v6416: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:49:43.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:49:43 smithi167 ceph-mon[37140]: pgmap v6416: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:49:45.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:49:45 smithi138 ceph-mon[36954]: pgmap v6417: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:49:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:49:45 smithi100 ceph-mon[32599]: pgmap v6417: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:49:45.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:49:45 smithi167 ceph-mon[37140]: pgmap v6417: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:49:46.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:49:46 smithi138 ceph-mon[36954]: pgmap v6418: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:49:46.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:49:46 smithi100 ceph-mon[32599]: pgmap v6418: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:49:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:49:46 smithi167 ceph-mon[37140]: pgmap v6418: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:49:49.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:49:49 smithi167 ceph-mon[37140]: pgmap v6419: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:49:49.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:49:49 smithi138 ceph-mon[36954]: pgmap v6419: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:49:49.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:49:49 smithi100 ceph-mon[32599]: pgmap v6419: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:49:51.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:49:51 smithi167 ceph-mon[37140]: pgmap v6420: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:49:51.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:49:51 smithi100 ceph-mon[32599]: pgmap v6420: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:49:51.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:49:51 smithi138 ceph-mon[36954]: pgmap v6420: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:49:53.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:49:53.063+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:49:53.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:49:53.064+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:49:53.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:49:53.064+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:49:53.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:49:53.064+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:49:53.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:49:53.064+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:49:53.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:49:53 smithi100 ceph-mon[32599]: pgmap v6421: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:49:53.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:49:53 smithi167 ceph-mon[37140]: pgmap v6421: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:49:53.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:49:53 smithi138 ceph-mon[36954]: pgmap v6421: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:49:55.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:49:55 smithi167 ceph-mon[37140]: pgmap v6422: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:49:55.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:49:55 smithi138 ceph-mon[36954]: pgmap v6422: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:49:55.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:49:55 smithi100 ceph-mon[32599]: pgmap v6422: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:49:57.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:49:57 smithi167 ceph-mon[37140]: pgmap v6423: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:49:57.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:49:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:49:57.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:49:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:49:57.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:49:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:49:57.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:49:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:49:57.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:49:57 smithi138 ceph-mon[36954]: pgmap v6423: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:49:57.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:49:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:49:57.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:49:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:49:57.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:49:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:49:57.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:49:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:49:57.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:49:57 smithi100 ceph-mon[32599]: pgmap v6423: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:49:57.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:49:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:49:57.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:49:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:49:57.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:49:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:49:57.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:49:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:49:59.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:49:59 smithi167 ceph-mon[37140]: pgmap v6424: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:49:59.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:49:59 smithi138 ceph-mon[36954]: pgmap v6424: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:49:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:49:59 smithi100 ceph-mon[32599]: pgmap v6424: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:50:00.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:50:00 smithi138 ceph-mon[36954]: overall HEALTH_OK 2024-07-26T16:50:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:50:00 smithi100 ceph-mon[32599]: overall HEALTH_OK 2024-07-26T16:50:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:50:00 smithi167 ceph-mon[37140]: overall HEALTH_OK 2024-07-26T16:50:01.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:50:01 smithi138 ceph-mon[36954]: pgmap v6425: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:50:01.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:50:01 smithi100 ceph-mon[32599]: pgmap v6425: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:50:01.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:50:01 smithi167 ceph-mon[37140]: pgmap v6425: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:50:03.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:50:03 smithi138 ceph-mon[36954]: pgmap v6426: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:50:03.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:50:03 smithi100 ceph-mon[32599]: pgmap v6426: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:50:03.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:50:03 smithi167 ceph-mon[37140]: pgmap v6426: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:50:05.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:50:05 smithi138 ceph-mon[36954]: pgmap v6427: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T16:50:05.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:50:05 smithi100 ceph-mon[32599]: pgmap v6427: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T16:50:05.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:50:05 smithi167 ceph-mon[37140]: pgmap v6427: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T16:50:07.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:50:07 smithi138 ceph-mon[36954]: pgmap v6428: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:50:07.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:50:07 smithi100 ceph-mon[32599]: pgmap v6428: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:50:07.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:50:07 smithi167 ceph-mon[37140]: pgmap v6428: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:50:08.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:50:08.079+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:50:08.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:50:08.080+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:50:08.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:50:08.080+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:50:08.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:50:08.080+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:50:08.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:50:08.080+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:50:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:50:08 smithi100 ceph-mon[32599]: osdmap e448: 12 total, 12 up, 12 in 2024-07-26T16:50:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:50:08 smithi138 ceph-mon[36954]: osdmap e448: 12 total, 12 up, 12 in 2024-07-26T16:50:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:50:08 smithi167 ceph-mon[37140]: osdmap e448: 12 total, 12 up, 12 in 2024-07-26T16:50:09.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:50:09 smithi138 ceph-mon[36954]: pgmap v6430: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:50:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:50:09 smithi100 ceph-mon[32599]: pgmap v6430: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:50:09.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:50:09 smithi167 ceph-mon[37140]: pgmap v6430: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:50:11.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:50:11 smithi138 ceph-mon[36954]: pgmap v6431: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T16:50:11.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:50:11 smithi100 ceph-mon[32599]: pgmap v6431: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T16:50:11.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:50:11 smithi167 ceph-mon[37140]: pgmap v6431: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T16:50:13.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:50:13 smithi138 ceph-mon[36954]: pgmap v6432: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:50:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:50:13 smithi100 ceph-mon[32599]: pgmap v6432: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:50:13.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:50:13 smithi167 ceph-mon[37140]: pgmap v6432: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:50:15.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:50:15 smithi138 ceph-mon[36954]: pgmap v6433: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:50:15.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:50:15 smithi100 ceph-mon[32599]: pgmap v6433: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:50:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:50:15 smithi167 ceph-mon[37140]: pgmap v6433: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:50:17.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:50:17 smithi138 ceph-mon[36954]: pgmap v6434: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:50:17.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:50:17 smithi138 ceph-mon[36954]: osdmap e449: 12 total, 12 up, 12 in 2024-07-26T16:50:17.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:50:17 smithi100 ceph-mon[32599]: pgmap v6434: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:50:17.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:50:17 smithi100 ceph-mon[32599]: osdmap e449: 12 total, 12 up, 12 in 2024-07-26T16:50:17.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:50:17 smithi167 ceph-mon[37140]: pgmap v6434: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:50:17.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:50:17 smithi167 ceph-mon[37140]: osdmap e449: 12 total, 12 up, 12 in 2024-07-26T16:50:19.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:50:19 smithi138 ceph-mon[36954]: pgmap v6436: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:50:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:50:19 smithi100 ceph-mon[32599]: pgmap v6436: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:50:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:50:19 smithi167 ceph-mon[37140]: pgmap v6436: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:50:21.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:50:21 smithi138 ceph-mon[36954]: pgmap v6437: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:50:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:50:21 smithi100 ceph-mon[32599]: pgmap v6437: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:50:21.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:50:21 smithi167 ceph-mon[37140]: pgmap v6437: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:50:22.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:50:22 smithi167 ceph-mon[37140]: pgmap v6438: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:50:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:50:22 smithi100 ceph-mon[32599]: pgmap v6438: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:50:23.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:50:23.087+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:50:23.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:50:23.088+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:50:23.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:50:23.088+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:50:23.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:50:23.088+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:50:23.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:50:23.088+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:50:23.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:50:22 smithi138 ceph-mon[36954]: pgmap v6438: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:50:25.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:50:25 smithi167 ceph-mon[37140]: pgmap v6439: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:50:25.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:50:25 smithi138 ceph-mon[36954]: pgmap v6439: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:50:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:50:25 smithi100 ceph-mon[32599]: pgmap v6439: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:50:27.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:50:27 smithi167 ceph-mon[37140]: pgmap v6440: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:50:27.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:50:27 smithi138 ceph-mon[36954]: pgmap v6440: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:50:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:50:27 smithi100 ceph-mon[32599]: pgmap v6440: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:50:29.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:50:29 smithi167 ceph-mon[37140]: pgmap v6441: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:50:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:50:29 smithi138 ceph-mon[36954]: pgmap v6441: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:50:29.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:50:29 smithi100 ceph-mon[32599]: pgmap v6441: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:50:31.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:50:31 smithi138 ceph-mon[36954]: pgmap v6442: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:50:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:50:31 smithi100 ceph-mon[32599]: pgmap v6442: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:50:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:50:31 smithi167 ceph-mon[37140]: pgmap v6442: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:50:33.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:50:33 smithi167 ceph-mon[37140]: pgmap v6443: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:50:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:50:33 smithi138 ceph-mon[36954]: pgmap v6443: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:50:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:50:33 smithi100 ceph-mon[32599]: pgmap v6443: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:50:35.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:50:35 smithi138 ceph-mon[36954]: pgmap v6444: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:50:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:50:35 smithi100 ceph-mon[32599]: pgmap v6444: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:50:35.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:50:35 smithi167 ceph-mon[37140]: pgmap v6444: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:50:37.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:50:37 smithi138 ceph-mon[36954]: pgmap v6445: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T16:50:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:50:37 smithi100 ceph-mon[32599]: pgmap v6445: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T16:50:37.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:50:37 smithi167 ceph-mon[37140]: pgmap v6445: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T16:50:38.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:50:38.104+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:50:38.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:50:38.104+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:50:38.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:50:38.104+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:50:38.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:50:38.104+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:50:38.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:50:38.104+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:50:39.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:50:39 smithi138 ceph-mon[36954]: pgmap v6446: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:50:39.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:50:39 smithi100 ceph-mon[32599]: pgmap v6446: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:50:39.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:50:39 smithi167 ceph-mon[37140]: pgmap v6446: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:50:41.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:50:41 smithi138 ceph-mon[36954]: pgmap v6447: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:50:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:50:41 smithi100 ceph-mon[32599]: pgmap v6447: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:50:41.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:50:41 smithi167 ceph-mon[37140]: pgmap v6447: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:50:43.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:50:43 smithi138 ceph-mon[36954]: pgmap v6448: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:50:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:50:43 smithi100 ceph-mon[32599]: pgmap v6448: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:50:43.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:50:43 smithi167 ceph-mon[37140]: pgmap v6448: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:50:45.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:50:45 smithi138 ceph-mon[36954]: pgmap v6449: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:50:45.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:50:45 smithi100 ceph-mon[32599]: pgmap v6449: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:50:45.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:50:45 smithi167 ceph-mon[37140]: pgmap v6449: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:50:47.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:50:47 smithi138 ceph-mon[36954]: pgmap v6450: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:50:47.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:50:47 smithi100 ceph-mon[32599]: pgmap v6450: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:50:47.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:50:47 smithi167 ceph-mon[37140]: pgmap v6450: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:50:49.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:50:49 smithi138 ceph-mon[36954]: pgmap v6451: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:50:49.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:50:49 smithi100 ceph-mon[32599]: pgmap v6451: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:50:49.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:50:49 smithi167 ceph-mon[37140]: pgmap v6451: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:50:51.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:50:51 smithi167 ceph-mon[37140]: pgmap v6452: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:50:51.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:50:51 smithi138 ceph-mon[36954]: pgmap v6452: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:50:51.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:50:51 smithi100 ceph-mon[32599]: pgmap v6452: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:50:53.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:50:53.118+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:50:53.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:50:53.119+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:50:53.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:50:53.119+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:50:53.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:50:53.119+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:50:53.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:50:53.119+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:50:53.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:50:53 smithi100 ceph-mon[32599]: pgmap v6453: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:50:53.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:50:53 smithi138 ceph-mon[36954]: pgmap v6453: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:50:53.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:50:53 smithi167 ceph-mon[37140]: pgmap v6453: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:50:55.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:50:55 smithi138 ceph-mon[36954]: pgmap v6454: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:50:55.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:50:55 smithi100 ceph-mon[32599]: pgmap v6454: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:50:55.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:50:55 smithi167 ceph-mon[37140]: pgmap v6454: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:50:57.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:50:57 smithi138 ceph-mon[36954]: pgmap v6455: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:50:57.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:50:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:50:57.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:50:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:50:57.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:50:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:50:57.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:50:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:50:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:50:57 smithi100 ceph-mon[32599]: pgmap v6455: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:50:57.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:50:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:50:57.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:50:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:50:57.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:50:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:50:57.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:50:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:50:57.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:50:57 smithi167 ceph-mon[37140]: pgmap v6455: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:50:57.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:50:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:50:57.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:50:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:50:57.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:50:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:50:57.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:50:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:50:59.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:50:59 smithi138 ceph-mon[36954]: pgmap v6456: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:50:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:50:59 smithi100 ceph-mon[32599]: pgmap v6456: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:50:59.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:50:59 smithi167 ceph-mon[37140]: pgmap v6456: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:51:01.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:01 smithi138 ceph-mon[36954]: pgmap v6457: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:51:01.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:01 smithi100 ceph-mon[32599]: pgmap v6457: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:51:01.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:01 smithi167 ceph-mon[37140]: pgmap v6457: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:51:03.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:03 smithi138 ceph-mon[36954]: pgmap v6458: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:51:03.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:03 smithi100 ceph-mon[32599]: pgmap v6458: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:51:03.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:03 smithi167 ceph-mon[37140]: pgmap v6458: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:51:04.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:51:04.570+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T16:51:04.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:51:04.570+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T16:51:04.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:51:04.570+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 88 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T16:51:04.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:51:04.570+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 88 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T16:51:05.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:05 smithi138 ceph-mon[36954]: pgmap v6459: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T16:51:05.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:05 smithi100 ceph-mon[32599]: pgmap v6459: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T16:51:05.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:05 smithi167 ceph-mon[37140]: pgmap v6459: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T16:51:07.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:07 smithi138 ceph-mon[36954]: pgmap v6460: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:51:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:07 smithi100 ceph-mon[32599]: pgmap v6460: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:51:07.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:07 smithi167 ceph-mon[37140]: pgmap v6460: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:51:08.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:51:08.135+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:51:08.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:51:08.136+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:51:08.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:51:08.136+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:51:08.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:51:08.136+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:51:08.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:51:08.136+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:51:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:08 smithi100 ceph-mon[32599]: osdmap e450: 12 total, 12 up, 12 in 2024-07-26T16:51:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:08 smithi138 ceph-mon[36954]: osdmap e450: 12 total, 12 up, 12 in 2024-07-26T16:51:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:08 smithi167 ceph-mon[37140]: osdmap e450: 12 total, 12 up, 12 in 2024-07-26T16:51:09.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:09 smithi138 ceph-mon[36954]: pgmap v6462: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:51:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:09 smithi100 ceph-mon[32599]: pgmap v6462: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:51:09.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:09 smithi167 ceph-mon[37140]: pgmap v6462: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:51:11.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:11 smithi138 ceph-mon[36954]: pgmap v6463: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T16:51:11.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:11 smithi100 ceph-mon[32599]: pgmap v6463: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T16:51:11.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:11 smithi167 ceph-mon[37140]: pgmap v6463: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T16:51:13.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:13 smithi138 ceph-mon[36954]: pgmap v6464: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:51:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:13 smithi100 ceph-mon[32599]: pgmap v6464: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:51:13.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:13 smithi167 ceph-mon[37140]: pgmap v6464: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:51:15.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:15 smithi138 ceph-mon[36954]: pgmap v6465: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:51:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:15 smithi100 ceph-mon[32599]: pgmap v6465: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:51:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:15 smithi167 ceph-mon[37140]: pgmap v6465: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:51:17.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:17 smithi138 ceph-mon[36954]: pgmap v6466: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:51:17.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:17 smithi138 ceph-mon[36954]: osdmap e451: 12 total, 12 up, 12 in 2024-07-26T16:51:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:17 smithi100 ceph-mon[32599]: pgmap v6466: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:51:17.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:17 smithi100 ceph-mon[32599]: osdmap e451: 12 total, 12 up, 12 in 2024-07-26T16:51:17.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:17 smithi167 ceph-mon[37140]: pgmap v6466: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:51:17.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:17 smithi167 ceph-mon[37140]: osdmap e451: 12 total, 12 up, 12 in 2024-07-26T16:51:19.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:19 smithi138 ceph-mon[36954]: pgmap v6468: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:51:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:19 smithi100 ceph-mon[32599]: pgmap v6468: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:51:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:19 smithi167 ceph-mon[37140]: pgmap v6468: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:51:21.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:21 smithi138 ceph-mon[36954]: pgmap v6469: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:51:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:21 smithi100 ceph-mon[32599]: pgmap v6469: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:51:21.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:21 smithi167 ceph-mon[37140]: pgmap v6469: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:51:22.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:22 smithi138 ceph-mon[36954]: pgmap v6470: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:51:22.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:22 smithi100 ceph-mon[32599]: pgmap v6470: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:51:22.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:22 smithi167 ceph-mon[37140]: pgmap v6470: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:51:23.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:51:23.151+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:51:23.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:51:23.152+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:51:23.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:51:23.152+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:51:23.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:51:23.152+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:51:23.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:51:23.152+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:51:25.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:25 smithi138 ceph-mon[36954]: pgmap v6471: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:51:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:25 smithi100 ceph-mon[32599]: pgmap v6471: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:51:25.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:25 smithi167 ceph-mon[37140]: pgmap v6471: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:51:27.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:27 smithi138 ceph-mon[36954]: pgmap v6472: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:51:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:27 smithi100 ceph-mon[32599]: pgmap v6472: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:51:27.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:27 smithi167 ceph-mon[37140]: pgmap v6472: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:51:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:29 smithi138 ceph-mon[36954]: pgmap v6473: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:51:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:29 smithi100 ceph-mon[32599]: pgmap v6473: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:51:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:29 smithi167 ceph-mon[37140]: pgmap v6473: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:51:31.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:31 smithi138 ceph-mon[36954]: pgmap v6474: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:51:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:31 smithi100 ceph-mon[32599]: pgmap v6474: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:51:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:31 smithi167 ceph-mon[37140]: pgmap v6474: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:51:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:33 smithi138 ceph-mon[36954]: pgmap v6475: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:51:33.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:33 smithi100 ceph-mon[32599]: pgmap v6475: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:51:33.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:33 smithi167 ceph-mon[37140]: pgmap v6475: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:51:35.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:35 smithi138 ceph-mon[36954]: pgmap v6476: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:51:35.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:35 smithi100 ceph-mon[32599]: pgmap v6476: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:51:35.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:35 smithi167 ceph-mon[37140]: pgmap v6476: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:51:37.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:37 smithi138 ceph-mon[36954]: pgmap v6477: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:51:37.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:37 smithi100 ceph-mon[32599]: pgmap v6477: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:51:37.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:37 smithi167 ceph-mon[37140]: pgmap v6477: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:51:38.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:51:38.158+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:51:38.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:51:38.159+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:51:38.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:51:38.159+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:51:38.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:51:38.159+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:51:38.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:51:38.159+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:51:39.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:39 smithi138 ceph-mon[36954]: pgmap v6478: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:51:39.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:39 smithi100 ceph-mon[32599]: pgmap v6478: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:51:39.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:39 smithi167 ceph-mon[37140]: pgmap v6478: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:51:41.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:41 smithi138 ceph-mon[36954]: pgmap v6479: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:51:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:41 smithi100 ceph-mon[32599]: pgmap v6479: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:51:41.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:41 smithi167 ceph-mon[37140]: pgmap v6479: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:51:43.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:43 smithi138 ceph-mon[36954]: pgmap v6480: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:51:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:43 smithi100 ceph-mon[32599]: pgmap v6480: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:51:43.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:43 smithi167 ceph-mon[37140]: pgmap v6480: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:51:45.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:45 smithi138 ceph-mon[36954]: pgmap v6481: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:51:45.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:45 smithi100 ceph-mon[32599]: pgmap v6481: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:51:45.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:45 smithi167 ceph-mon[37140]: pgmap v6481: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:51:47.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:47 smithi138 ceph-mon[36954]: pgmap v6482: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:51:47.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:47 smithi100 ceph-mon[32599]: pgmap v6482: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:51:47.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:47 smithi167 ceph-mon[37140]: pgmap v6482: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:51:49.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:49 smithi138 ceph-mon[36954]: pgmap v6483: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:51:49.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:49 smithi100 ceph-mon[32599]: pgmap v6483: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:51:49.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:49 smithi167 ceph-mon[37140]: pgmap v6483: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:51:51.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:51 smithi138 ceph-mon[36954]: pgmap v6484: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:51:51.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:51 smithi100 ceph-mon[32599]: pgmap v6484: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:51:51.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:51 smithi167 ceph-mon[37140]: pgmap v6484: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:51:53.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:51:53.163+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:51:53.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:51:53.164+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:51:53.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:51:53.164+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:51:53.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:51:53.164+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:51:53.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:51:53.164+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:51:53.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:53 smithi100 ceph-mon[32599]: pgmap v6485: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:51:53.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:53 smithi138 ceph-mon[36954]: pgmap v6485: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:51:53.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:53 smithi167 ceph-mon[37140]: pgmap v6485: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:51:55.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:55 smithi138 ceph-mon[36954]: pgmap v6486: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:51:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:55 smithi100 ceph-mon[32599]: pgmap v6486: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:51:55.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:55 smithi167 ceph-mon[37140]: pgmap v6486: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:51:57.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:57 smithi138 ceph-mon[36954]: pgmap v6487: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:51:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:57 smithi100 ceph-mon[32599]: pgmap v6487: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:51:57.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:57 smithi167 ceph-mon[37140]: pgmap v6487: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:51:58.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:51:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:51:58.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:51:59.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:59 smithi138 ceph-mon[36954]: pgmap v6488: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:51:59.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:51:59.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:51:59.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:51:59.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:51:59.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:51:59.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:51:59.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:51:59.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:51:59.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:51:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:51:59.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:59 smithi100 ceph-mon[32599]: pgmap v6488: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:51:59.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:51:59.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:51:59.747 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:51:59.747 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:51:59.747 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:51:59.747 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:51:59.747 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:51:59.747 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:51:59.747 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:51:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:51:59.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:59 smithi167 ceph-mon[37140]: pgmap v6488: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:51:59.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:51:59.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:51:59.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:51:59.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:51:59.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:51:59.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:51:59.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:51:59.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:51:59.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:51:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:52:01.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:52:01 smithi138 ceph-mon[36954]: pgmap v6489: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:52:01.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:52:01 smithi100 ceph-mon[32599]: pgmap v6489: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:52:01.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:52:01 smithi167 ceph-mon[37140]: pgmap v6489: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:52:03.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:52:03 smithi138 ceph-mon[36954]: pgmap v6490: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:52:03.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:52:03 smithi100 ceph-mon[32599]: pgmap v6490: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:52:03.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:52:03 smithi167 ceph-mon[37140]: pgmap v6490: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:52:05.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:52:05 smithi138 ceph-mon[36954]: pgmap v6491: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T16:52:05.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:52:05 smithi100 ceph-mon[32599]: pgmap v6491: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T16:52:05.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:52:05 smithi167 ceph-mon[37140]: pgmap v6491: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T16:52:07.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:52:07 smithi138 ceph-mon[36954]: pgmap v6492: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:52:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:52:07 smithi100 ceph-mon[32599]: pgmap v6492: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:52:07.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:52:07 smithi167 ceph-mon[37140]: pgmap v6492: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:52:08.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:52:08.179+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:52:08.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:52:08.179+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:52:08.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:52:08.179+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:52:08.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:52:08.179+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:52:08.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:52:08.179+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:52:08.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:52:08 smithi100 ceph-mon[32599]: osdmap e452: 12 total, 12 up, 12 in 2024-07-26T16:52:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:52:08 smithi138 ceph-mon[36954]: osdmap e452: 12 total, 12 up, 12 in 2024-07-26T16:52:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:52:08 smithi167 ceph-mon[37140]: osdmap e452: 12 total, 12 up, 12 in 2024-07-26T16:52:09.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:52:09 smithi138 ceph-mon[36954]: pgmap v6494: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:52:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:52:09 smithi100 ceph-mon[32599]: pgmap v6494: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:52:09.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:52:09 smithi167 ceph-mon[37140]: pgmap v6494: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:52:11.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:52:11 smithi138 ceph-mon[36954]: pgmap v6495: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T16:52:11.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:52:11 smithi100 ceph-mon[32599]: pgmap v6495: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T16:52:11.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:52:11 smithi167 ceph-mon[37140]: pgmap v6495: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T16:52:13.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:52:13 smithi138 ceph-mon[36954]: pgmap v6496: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:52:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:52:13 smithi100 ceph-mon[32599]: pgmap v6496: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:52:13.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:52:13 smithi167 ceph-mon[37140]: pgmap v6496: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:52:15.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:52:15 smithi138 ceph-mon[36954]: pgmap v6497: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:52:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:52:15 smithi100 ceph-mon[32599]: pgmap v6497: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:52:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:52:15 smithi167 ceph-mon[37140]: pgmap v6497: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:52:17.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:52:17 smithi138 ceph-mon[36954]: pgmap v6498: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:52:17.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:52:17 smithi138 ceph-mon[36954]: osdmap e453: 12 total, 12 up, 12 in 2024-07-26T16:52:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:52:17 smithi100 ceph-mon[32599]: pgmap v6498: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:52:17.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:52:17 smithi100 ceph-mon[32599]: osdmap e453: 12 total, 12 up, 12 in 2024-07-26T16:52:17.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:52:17 smithi167 ceph-mon[37140]: pgmap v6498: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:52:17.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:52:17 smithi167 ceph-mon[37140]: osdmap e453: 12 total, 12 up, 12 in 2024-07-26T16:52:18.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:52:18 smithi138 ceph-mon[36954]: pgmap v6500: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:52:18.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:52:18 smithi100 ceph-mon[32599]: pgmap v6500: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:52:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:52:18 smithi167 ceph-mon[37140]: pgmap v6500: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:52:21.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:52:21 smithi138 ceph-mon[36954]: pgmap v6501: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:52:21.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:52:21 smithi100 ceph-mon[32599]: pgmap v6501: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:52:21.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:52:21 smithi167 ceph-mon[37140]: pgmap v6501: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:52:23.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:52:23.185+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:52:23.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:52:23.188+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:52:23.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:52:23.188+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:52:23.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:52:23.188+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:52:23.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:52:23.188+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:52:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:52:23 smithi100 ceph-mon[32599]: pgmap v6502: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:52:23.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:52:23 smithi138 ceph-mon[36954]: pgmap v6502: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:52:23.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:52:23 smithi167 ceph-mon[37140]: pgmap v6502: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:52:25.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:52:25 smithi138 ceph-mon[36954]: pgmap v6503: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:52:25.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:52:25 smithi100 ceph-mon[32599]: pgmap v6503: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:52:25.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:52:25 smithi167 ceph-mon[37140]: pgmap v6503: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:52:27.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:52:27 smithi138 ceph-mon[36954]: pgmap v6504: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:52:27.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:52:27 smithi100 ceph-mon[32599]: pgmap v6504: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:52:27.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:52:27 smithi167 ceph-mon[37140]: pgmap v6504: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:52:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:52:29 smithi138 ceph-mon[36954]: pgmap v6505: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:52:29.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:52:29 smithi100 ceph-mon[32599]: pgmap v6505: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:52:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:52:29 smithi167 ceph-mon[37140]: pgmap v6505: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:52:31.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:52:31 smithi138 ceph-mon[36954]: pgmap v6506: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:52:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:52:31 smithi100 ceph-mon[32599]: pgmap v6506: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:52:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:52:31 smithi167 ceph-mon[37140]: pgmap v6506: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:52:32.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:52:32 smithi100 ceph-mon[32599]: pgmap v6507: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:52:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:52:32 smithi167 ceph-mon[37140]: pgmap v6507: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:52:33.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:52:32 smithi138 ceph-mon[36954]: pgmap v6507: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:52:35.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:52:35 smithi138 ceph-mon[36954]: pgmap v6508: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:52:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:52:35 smithi100 ceph-mon[32599]: pgmap v6508: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:52:35.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:52:35 smithi167 ceph-mon[37140]: pgmap v6508: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:52:37.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:52:37 smithi138 ceph-mon[36954]: pgmap v6509: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:52:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:52:37 smithi100 ceph-mon[32599]: pgmap v6509: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:52:37.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:52:37 smithi167 ceph-mon[37140]: pgmap v6509: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:52:38.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:52:38.199+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:52:38.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:52:38.199+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:52:38.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:52:38.200+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:52:38.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:52:38.200+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:52:38.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:52:38.200+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:52:39.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:52:39 smithi138 ceph-mon[36954]: pgmap v6510: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:52:39.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:52:39 smithi100 ceph-mon[32599]: pgmap v6510: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:52:39.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:52:39 smithi167 ceph-mon[37140]: pgmap v6510: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:52:41.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:52:41 smithi138 ceph-mon[36954]: pgmap v6511: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:52:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:52:41 smithi100 ceph-mon[32599]: pgmap v6511: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:52:41.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:52:41 smithi167 ceph-mon[37140]: pgmap v6511: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:52:43.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:52:43 smithi138 ceph-mon[36954]: pgmap v6512: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:52:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:52:43 smithi100 ceph-mon[32599]: pgmap v6512: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:52:43.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:52:43 smithi167 ceph-mon[37140]: pgmap v6512: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:52:45.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:52:45 smithi138 ceph-mon[36954]: pgmap v6513: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:52:45.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:52:45 smithi100 ceph-mon[32599]: pgmap v6513: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:52:45.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:52:45 smithi167 ceph-mon[37140]: pgmap v6513: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:52:47.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:52:47 smithi138 ceph-mon[36954]: pgmap v6514: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:52:47.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:52:47 smithi100 ceph-mon[32599]: pgmap v6514: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:52:47.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:52:47 smithi167 ceph-mon[37140]: pgmap v6514: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:52:49.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:52:49 smithi138 ceph-mon[36954]: pgmap v6515: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:52:49.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:52:49 smithi100 ceph-mon[32599]: pgmap v6515: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:52:49.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:52:49 smithi167 ceph-mon[37140]: pgmap v6515: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:52:51.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:52:51 smithi138 ceph-mon[36954]: pgmap v6516: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:52:51.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:52:51 smithi100 ceph-mon[32599]: pgmap v6516: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:52:51.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:52:51 smithi167 ceph-mon[37140]: pgmap v6516: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:52:53.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:52:53.215+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:52:53.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:52:53.216+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:52:53.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:52:53.216+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:52:53.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:52:53.216+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:52:53.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:52:53.216+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:52:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:52:53 smithi100 ceph-mon[32599]: pgmap v6517: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:52:53.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:52:53 smithi138 ceph-mon[36954]: pgmap v6517: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:52:53.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:52:53 smithi167 ceph-mon[37140]: pgmap v6517: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:52:55.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:52:55 smithi138 ceph-mon[36954]: pgmap v6518: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:52:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:52:55 smithi100 ceph-mon[32599]: pgmap v6518: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:52:55.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:52:55 smithi167 ceph-mon[37140]: pgmap v6518: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:52:57.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:52:57 smithi138 ceph-mon[36954]: pgmap v6519: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:52:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:52:57 smithi100 ceph-mon[32599]: pgmap v6519: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:52:57.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:52:57 smithi167 ceph-mon[37140]: pgmap v6519: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:52:59.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:52:59 smithi138 ceph-mon[36954]: pgmap v6520: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:52:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:52:59 smithi100 ceph-mon[32599]: pgmap v6520: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:52:59.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:52:59 smithi167 ceph-mon[37140]: pgmap v6520: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:53:00.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:53:00.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:53:00.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:53:00.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:53:00.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:53:00.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:53:00.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:53:00.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:53:00.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:53:00.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:53:00.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:53:00.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:53:00.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:53:00.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:53:00.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:53:00.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:53:00.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:53:00.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:53:00.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:53:00.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:53:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:53:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:53:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:53:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:53:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:53:00.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:53:00.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:53:00.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:53:00.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:53:00.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:53:01.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:01 smithi138 ceph-mon[36954]: pgmap v6521: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:53:01.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:01 smithi100 ceph-mon[32599]: pgmap v6521: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:53:01.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:01 smithi167 ceph-mon[37140]: pgmap v6521: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:53:03.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:03 smithi138 ceph-mon[36954]: pgmap v6522: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:53:03.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:03 smithi100 ceph-mon[32599]: pgmap v6522: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:53:03.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:03 smithi167 ceph-mon[37140]: pgmap v6522: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:53:05.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:05 smithi138 ceph-mon[36954]: pgmap v6523: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:53:05.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:05 smithi100 ceph-mon[32599]: pgmap v6523: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:53:05.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:05 smithi167 ceph-mon[37140]: pgmap v6523: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:53:07.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:07 smithi138 ceph-mon[36954]: pgmap v6524: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:53:07.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:07 smithi100 ceph-mon[32599]: pgmap v6524: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:53:07.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:07 smithi167 ceph-mon[37140]: pgmap v6524: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:53:08.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:53:08.231+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:53:08.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:53:08.231+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:53:08.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:53:08.232+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:53:08.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:53:08.232+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:53:08.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:53:08.232+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:53:08.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:08 smithi100 ceph-mon[32599]: osdmap e454: 12 total, 12 up, 12 in 2024-07-26T16:53:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:08 smithi138 ceph-mon[36954]: osdmap e454: 12 total, 12 up, 12 in 2024-07-26T16:53:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:08 smithi167 ceph-mon[37140]: osdmap e454: 12 total, 12 up, 12 in 2024-07-26T16:53:09.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:09 smithi138 ceph-mon[36954]: pgmap v6526: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:53:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:09 smithi100 ceph-mon[32599]: pgmap v6526: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:53:09.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:09 smithi167 ceph-mon[37140]: pgmap v6526: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:53:11.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:11 smithi138 ceph-mon[36954]: pgmap v6527: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:53:11.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:11 smithi100 ceph-mon[32599]: pgmap v6527: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:53:11.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:11 smithi167 ceph-mon[37140]: pgmap v6527: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:53:13.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:13 smithi138 ceph-mon[36954]: pgmap v6528: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:53:13.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:13 smithi100 ceph-mon[32599]: pgmap v6528: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:53:13.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:13 smithi167 ceph-mon[37140]: pgmap v6528: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:53:15.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:15 smithi138 ceph-mon[36954]: pgmap v6529: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:53:15.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:15 smithi100 ceph-mon[32599]: pgmap v6529: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:53:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:15 smithi167 ceph-mon[37140]: pgmap v6529: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:53:17.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:17 smithi138 ceph-mon[36954]: pgmap v6530: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:53:17.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:17 smithi138 ceph-mon[36954]: osdmap e455: 12 total, 12 up, 12 in 2024-07-26T16:53:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:17 smithi100 ceph-mon[32599]: pgmap v6530: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:53:17.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:17 smithi100 ceph-mon[32599]: osdmap e455: 12 total, 12 up, 12 in 2024-07-26T16:53:17.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:17 smithi167 ceph-mon[37140]: pgmap v6530: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:53:17.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:17 smithi167 ceph-mon[37140]: osdmap e455: 12 total, 12 up, 12 in 2024-07-26T16:53:19.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:19 smithi138 ceph-mon[36954]: pgmap v6532: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:53:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:19 smithi100 ceph-mon[32599]: pgmap v6532: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:53:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:19 smithi167 ceph-mon[37140]: pgmap v6532: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:53:21.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:21 smithi138 ceph-mon[36954]: pgmap v6533: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:53:21.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:21 smithi100 ceph-mon[32599]: pgmap v6533: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:53:21.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:21 smithi167 ceph-mon[37140]: pgmap v6533: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:53:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:22 smithi100 ceph-mon[32599]: pgmap v6534: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:53:22.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:22 smithi167 ceph-mon[37140]: pgmap v6534: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:53:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:22 smithi138 ceph-mon[36954]: pgmap v6534: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:53:23.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:53:23.239+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:53:23.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:53:23.240+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:53:23.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:53:23.240+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:53:23.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:53:23.240+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:53:23.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:53:23.240+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:53:25.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:25 smithi138 ceph-mon[36954]: pgmap v6535: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:53:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:25 smithi100 ceph-mon[32599]: pgmap v6535: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:53:25.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:25 smithi167 ceph-mon[37140]: pgmap v6535: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:53:27.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:27 smithi138 ceph-mon[36954]: pgmap v6536: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:53:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:27 smithi100 ceph-mon[32599]: pgmap v6536: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:53:27.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:27 smithi167 ceph-mon[37140]: pgmap v6536: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:53:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:29 smithi138 ceph-mon[36954]: pgmap v6537: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:53:29.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:29 smithi100 ceph-mon[32599]: pgmap v6537: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:53:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:29 smithi167 ceph-mon[37140]: pgmap v6537: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:53:31.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:31 smithi138 ceph-mon[36954]: pgmap v6538: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:53:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:31 smithi100 ceph-mon[32599]: pgmap v6538: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:53:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:31 smithi167 ceph-mon[37140]: pgmap v6538: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:53:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:33 smithi138 ceph-mon[36954]: pgmap v6539: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:53:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:33 smithi100 ceph-mon[32599]: pgmap v6539: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:53:33.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:33 smithi167 ceph-mon[37140]: pgmap v6539: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:53:35.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:35 smithi138 ceph-mon[36954]: pgmap v6540: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:53:35.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:35 smithi100 ceph-mon[32599]: pgmap v6540: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:53:35.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:35 smithi167 ceph-mon[37140]: pgmap v6540: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:53:37.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:37 smithi138 ceph-mon[36954]: pgmap v6541: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:53:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:37 smithi100 ceph-mon[32599]: pgmap v6541: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:53:37.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:37 smithi167 ceph-mon[37140]: pgmap v6541: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:53:38.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:53:38.252+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:53:38.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:53:38.253+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:53:38.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:53:38.253+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:53:38.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:53:38.253+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:53:38.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:53:38.253+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:53:39.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:39 smithi138 ceph-mon[36954]: pgmap v6542: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:53:39.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:39 smithi100 ceph-mon[32599]: pgmap v6542: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:53:39.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:39 smithi167 ceph-mon[37140]: pgmap v6542: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:53:41.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:41 smithi138 ceph-mon[36954]: pgmap v6543: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:53:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:41 smithi100 ceph-mon[32599]: pgmap v6543: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:53:41.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:41 smithi167 ceph-mon[37140]: pgmap v6543: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:53:43.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:43 smithi138 ceph-mon[36954]: pgmap v6544: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:53:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:43 smithi100 ceph-mon[32599]: pgmap v6544: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:53:43.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:43 smithi167 ceph-mon[37140]: pgmap v6544: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:53:45.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:45 smithi138 ceph-mon[36954]: pgmap v6545: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:53:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:45 smithi100 ceph-mon[32599]: pgmap v6545: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:53:45.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:45 smithi167 ceph-mon[37140]: pgmap v6545: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:53:47.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:47 smithi138 ceph-mon[36954]: pgmap v6546: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:53:47.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:47 smithi100 ceph-mon[32599]: pgmap v6546: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:53:47.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:47 smithi167 ceph-mon[37140]: pgmap v6546: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:53:49.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:49 smithi138 ceph-mon[36954]: pgmap v6547: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:53:49.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:49 smithi100 ceph-mon[32599]: pgmap v6547: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:53:49.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:49 smithi167 ceph-mon[37140]: pgmap v6547: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:53:51.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:51 smithi138 ceph-mon[36954]: pgmap v6548: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:53:51.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:51 smithi100 ceph-mon[32599]: pgmap v6548: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:53:51.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:51 smithi167 ceph-mon[37140]: pgmap v6548: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:53:53.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:53:53.269+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:53:53.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:53:53.270+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:53:53.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:53:53.270+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:53:53.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:53:53.270+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:53:53.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:53:53.270+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:53:53.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:53 smithi138 ceph-mon[36954]: pgmap v6549: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:53:53.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:53 smithi100 ceph-mon[32599]: pgmap v6549: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:53:53.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:53 smithi167 ceph-mon[37140]: pgmap v6549: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:53:55.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:55 smithi138 ceph-mon[36954]: pgmap v6550: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:53:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:55 smithi100 ceph-mon[32599]: pgmap v6550: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:53:55.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:55 smithi167 ceph-mon[37140]: pgmap v6550: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:53:57.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:57 smithi138 ceph-mon[36954]: pgmap v6551: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:53:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:57 smithi100 ceph-mon[32599]: pgmap v6551: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:53:57.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:57 smithi167 ceph-mon[37140]: pgmap v6551: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:53:59.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:53:59 smithi138 ceph-mon[36954]: pgmap v6552: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:53:59.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:53:59 smithi100 ceph-mon[32599]: pgmap v6552: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:53:59.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:53:59 smithi167 ceph-mon[37140]: pgmap v6552: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:54:00.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:54:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:54:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:54:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:54:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:54:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:54:01.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:54:01 smithi138 ceph-mon[36954]: pgmap v6553: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:54:01.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:54:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:54:01.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:54:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:54:01.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:54:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:54:01.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:54:01 smithi100 ceph-mon[32599]: pgmap v6553: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:54:01.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:54:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:54:01.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:54:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:54:01.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:54:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:54:01.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:54:01 smithi167 ceph-mon[37140]: pgmap v6553: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:54:01.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:54:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:54:01.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:54:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:54:01.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:54:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:54:03.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:54:03 smithi138 ceph-mon[36954]: pgmap v6554: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:54:03.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:54:03 smithi100 ceph-mon[32599]: pgmap v6554: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:54:03.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:54:03 smithi167 ceph-mon[37140]: pgmap v6554: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:54:04.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:54:04.596+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T16:54:04.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:54:04.596+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T16:54:04.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:54:04.596+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 89 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T16:54:04.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:54:04.596+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 89 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T16:54:05.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:54:05 smithi138 ceph-mon[36954]: pgmap v6555: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:54:05.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:54:05 smithi100 ceph-mon[32599]: pgmap v6555: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:54:05.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:54:05 smithi167 ceph-mon[37140]: pgmap v6555: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:54:07.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:54:07 smithi138 ceph-mon[36954]: pgmap v6556: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:54:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:54:07 smithi100 ceph-mon[32599]: pgmap v6556: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:54:07.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:54:07 smithi167 ceph-mon[37140]: pgmap v6556: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:54:08.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:54:08.285+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:54:08.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:54:08.286+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:54:08.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:54:08.286+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:54:08.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:54:08.286+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:54:08.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:54:08.286+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:54:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:54:08 smithi138 ceph-mon[36954]: osdmap e456: 12 total, 12 up, 12 in 2024-07-26T16:54:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:54:08 smithi138 ceph-mon[36954]: pgmap v6558: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:54:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:54:08 smithi100 ceph-mon[32599]: osdmap e456: 12 total, 12 up, 12 in 2024-07-26T16:54:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:54:08 smithi100 ceph-mon[32599]: pgmap v6558: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:54:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:54:08 smithi167 ceph-mon[37140]: osdmap e456: 12 total, 12 up, 12 in 2024-07-26T16:54:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:54:08 smithi167 ceph-mon[37140]: pgmap v6558: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:54:11.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:54:11 smithi138 ceph-mon[36954]: pgmap v6559: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:54:11.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:54:11 smithi100 ceph-mon[32599]: pgmap v6559: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:54:11.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:54:11 smithi167 ceph-mon[37140]: pgmap v6559: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:54:13.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:54:13 smithi138 ceph-mon[36954]: pgmap v6560: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:54:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:54:13 smithi100 ceph-mon[32599]: pgmap v6560: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:54:13.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:54:13 smithi167 ceph-mon[37140]: pgmap v6560: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:54:15.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:54:15 smithi138 ceph-mon[36954]: pgmap v6561: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:54:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:54:15 smithi100 ceph-mon[32599]: pgmap v6561: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:54:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:54:15 smithi167 ceph-mon[37140]: pgmap v6561: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:54:17.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:54:17 smithi138 ceph-mon[36954]: pgmap v6562: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:54:17.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:54:17 smithi138 ceph-mon[36954]: osdmap e457: 12 total, 12 up, 12 in 2024-07-26T16:54:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:54:17 smithi100 ceph-mon[32599]: pgmap v6562: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:54:17.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:54:17 smithi100 ceph-mon[32599]: osdmap e457: 12 total, 12 up, 12 in 2024-07-26T16:54:17.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:54:17 smithi167 ceph-mon[37140]: pgmap v6562: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:54:17.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:54:17 smithi167 ceph-mon[37140]: osdmap e457: 12 total, 12 up, 12 in 2024-07-26T16:54:19.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:54:19 smithi138 ceph-mon[36954]: pgmap v6564: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:54:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:54:19 smithi100 ceph-mon[32599]: pgmap v6564: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:54:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:54:19 smithi167 ceph-mon[37140]: pgmap v6564: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:54:21.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:54:21 smithi138 ceph-mon[36954]: pgmap v6565: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T16:54:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:54:21 smithi100 ceph-mon[32599]: pgmap v6565: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T16:54:21.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:54:21 smithi167 ceph-mon[37140]: pgmap v6565: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T16:54:23.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:54:23.289+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:54:23.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:54:23.289+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:54:23.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:54:23.289+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:54:23.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:54:23.289+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:54:23.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:54:23.289+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:54:23.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:54:23 smithi138 ceph-mon[36954]: pgmap v6566: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T16:54:23.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:54:23 smithi100 ceph-mon[32599]: pgmap v6566: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T16:54:23.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:54:23 smithi167 ceph-mon[37140]: pgmap v6566: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T16:54:25.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:54:25 smithi138 ceph-mon[36954]: pgmap v6567: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:54:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:54:25 smithi100 ceph-mon[32599]: pgmap v6567: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:54:25.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:54:25 smithi167 ceph-mon[37140]: pgmap v6567: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:54:27.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:54:27 smithi138 ceph-mon[36954]: pgmap v6568: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:54:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:54:27 smithi100 ceph-mon[32599]: pgmap v6568: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:54:27.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:54:27 smithi167 ceph-mon[37140]: pgmap v6568: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:54:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:54:29 smithi138 ceph-mon[36954]: pgmap v6569: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:54:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:54:29 smithi100 ceph-mon[32599]: pgmap v6569: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:54:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:54:29 smithi167 ceph-mon[37140]: pgmap v6569: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:54:31.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:54:31 smithi138 ceph-mon[36954]: pgmap v6570: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:54:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:54:31 smithi100 ceph-mon[32599]: pgmap v6570: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:54:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:54:31 smithi167 ceph-mon[37140]: pgmap v6570: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:54:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:54:33 smithi138 ceph-mon[36954]: pgmap v6571: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:54:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:54:33 smithi100 ceph-mon[32599]: pgmap v6571: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:54:33.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:54:33 smithi167 ceph-mon[37140]: pgmap v6571: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:54:35.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:54:35 smithi138 ceph-mon[36954]: pgmap v6572: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:54:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:54:35 smithi100 ceph-mon[32599]: pgmap v6572: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:54:35.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:54:35 smithi167 ceph-mon[37140]: pgmap v6572: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:54:37.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:54:37 smithi138 ceph-mon[36954]: pgmap v6573: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:54:37.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:54:37 smithi100 ceph-mon[32599]: pgmap v6573: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:54:37.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:54:37 smithi167 ceph-mon[37140]: pgmap v6573: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:54:38.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:54:38.303+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:54:38.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:54:38.304+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:54:38.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:54:38.304+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:54:38.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:54:38.304+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:54:38.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:54:38.304+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:54:39.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:54:39 smithi138 ceph-mon[36954]: pgmap v6574: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:54:39.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:54:39 smithi100 ceph-mon[32599]: pgmap v6574: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:54:39.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:54:39 smithi167 ceph-mon[37140]: pgmap v6574: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:54:41.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:54:41 smithi138 ceph-mon[36954]: pgmap v6575: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:54:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:54:41 smithi100 ceph-mon[32599]: pgmap v6575: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:54:41.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:54:41 smithi167 ceph-mon[37140]: pgmap v6575: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:54:43.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:54:43 smithi138 ceph-mon[36954]: pgmap v6576: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:54:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:54:43 smithi100 ceph-mon[32599]: pgmap v6576: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:54:43.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:54:43 smithi167 ceph-mon[37140]: pgmap v6576: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:54:45.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:54:45 smithi138 ceph-mon[36954]: pgmap v6577: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:54:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:54:45 smithi100 ceph-mon[32599]: pgmap v6577: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:54:45.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:54:45 smithi167 ceph-mon[37140]: pgmap v6577: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:54:46.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:54:46 smithi100 ceph-mon[32599]: pgmap v6578: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:54:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:54:46 smithi167 ceph-mon[37140]: pgmap v6578: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:54:46.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:54:46 smithi138 ceph-mon[36954]: pgmap v6578: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:54:49.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:54:49 smithi138 ceph-mon[36954]: pgmap v6579: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:54:49.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:54:49 smithi100 ceph-mon[32599]: pgmap v6579: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:54:49.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:54:49 smithi167 ceph-mon[37140]: pgmap v6579: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:54:51.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:54:51 smithi138 ceph-mon[36954]: pgmap v6580: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:54:51.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:54:51 smithi100 ceph-mon[32599]: pgmap v6580: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:54:51.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:54:51 smithi167 ceph-mon[37140]: pgmap v6580: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:54:53.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:54:53.317+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:54:53.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:54:53.317+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:54:53.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:54:53.317+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:54:53.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:54:53.317+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:54:53.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:54:53.317+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:54:53.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:54:53 smithi138 ceph-mon[36954]: pgmap v6581: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:54:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:54:53 smithi100 ceph-mon[32599]: pgmap v6581: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:54:53.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:54:53 smithi167 ceph-mon[37140]: pgmap v6581: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:54:55.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:54:55 smithi138 ceph-mon[36954]: pgmap v6582: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:54:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:54:55 smithi100 ceph-mon[32599]: pgmap v6582: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:54:55.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:54:55 smithi167 ceph-mon[37140]: pgmap v6582: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:54:57.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:54:57 smithi138 ceph-mon[36954]: pgmap v6583: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:54:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:54:57 smithi100 ceph-mon[32599]: pgmap v6583: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:54:57.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:54:57 smithi167 ceph-mon[37140]: pgmap v6583: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:54:59.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:54:59 smithi138 ceph-mon[36954]: pgmap v6584: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:54:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:54:59 smithi100 ceph-mon[32599]: pgmap v6584: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:54:59.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:54:59 smithi167 ceph-mon[37140]: pgmap v6584: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:55:01.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:55:01 smithi138 ceph-mon[36954]: pgmap v6585: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:55:01.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:55:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:55:01.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:55:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:55:01.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:55:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:55:01.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:55:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:55:01.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:55:01 smithi100 ceph-mon[32599]: pgmap v6585: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:55:01.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:55:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:55:01.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:55:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:55:01.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:55:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:55:01.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:55:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:55:01.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:55:01 smithi167 ceph-mon[37140]: pgmap v6585: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:55:01.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:55:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:55:01.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:55:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:55:01.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:55:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:55:01.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:55:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:55:03.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:55:03 smithi138 ceph-mon[36954]: pgmap v6586: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:55:03.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:55:03 smithi100 ceph-mon[32599]: pgmap v6586: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:55:03.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:55:03 smithi167 ceph-mon[37140]: pgmap v6586: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:55:05.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:55:05 smithi138 ceph-mon[36954]: pgmap v6587: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:55:05.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:55:05 smithi100 ceph-mon[32599]: pgmap v6587: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:55:05.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:55:05 smithi167 ceph-mon[37140]: pgmap v6587: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:55:07.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:55:07 smithi138 ceph-mon[36954]: pgmap v6588: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:55:07.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:55:07 smithi100 ceph-mon[32599]: pgmap v6588: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:55:07.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:55:07 smithi167 ceph-mon[37140]: pgmap v6588: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:55:08.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:55:08.323+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:55:08.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:55:08.324+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:55:08.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:55:08.324+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:55:08.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:55:08.324+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:55:08.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:55:08.324+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:55:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:55:08 smithi138 ceph-mon[36954]: osdmap e458: 12 total, 12 up, 12 in 2024-07-26T16:55:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:55:08 smithi100 ceph-mon[32599]: osdmap e458: 12 total, 12 up, 12 in 2024-07-26T16:55:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:55:08 smithi167 ceph-mon[37140]: osdmap e458: 12 total, 12 up, 12 in 2024-07-26T16:55:09.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:55:09 smithi138 ceph-mon[36954]: pgmap v6590: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:55:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:55:09 smithi100 ceph-mon[32599]: pgmap v6590: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:55:09.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:55:09 smithi167 ceph-mon[37140]: pgmap v6590: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:55:11.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:55:11 smithi138 ceph-mon[36954]: pgmap v6591: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:55:11.747 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:55:11 smithi100 ceph-mon[32599]: pgmap v6591: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:55:11.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:55:11 smithi167 ceph-mon[37140]: pgmap v6591: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:55:13.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:55:13 smithi138 ceph-mon[36954]: pgmap v6592: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:55:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:55:13 smithi100 ceph-mon[32599]: pgmap v6592: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:55:13.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:55:13 smithi167 ceph-mon[37140]: pgmap v6592: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:55:15.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:55:15 smithi138 ceph-mon[36954]: pgmap v6593: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:55:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:55:15 smithi100 ceph-mon[32599]: pgmap v6593: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:55:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:55:15 smithi167 ceph-mon[37140]: pgmap v6593: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:55:17.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:55:17 smithi138 ceph-mon[36954]: pgmap v6594: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:55:17.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:55:17 smithi138 ceph-mon[36954]: osdmap e459: 12 total, 12 up, 12 in 2024-07-26T16:55:17.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:55:17 smithi100 ceph-mon[32599]: pgmap v6594: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:55:17.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:55:17 smithi100 ceph-mon[32599]: osdmap e459: 12 total, 12 up, 12 in 2024-07-26T16:55:17.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:55:17 smithi167 ceph-mon[37140]: pgmap v6594: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:55:17.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:55:17 smithi167 ceph-mon[37140]: osdmap e459: 12 total, 12 up, 12 in 2024-07-26T16:55:18.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:55:18 smithi100 ceph-mon[32599]: pgmap v6596: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:55:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:55:18 smithi167 ceph-mon[37140]: pgmap v6596: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:55:18.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:55:18 smithi138 ceph-mon[36954]: pgmap v6596: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:55:21.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:55:21 smithi138 ceph-mon[36954]: pgmap v6597: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:55:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:55:21 smithi100 ceph-mon[32599]: pgmap v6597: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:55:21.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:55:21 smithi167 ceph-mon[37140]: pgmap v6597: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:55:23.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:55:23.340+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:55:23.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:55:23.340+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:55:23.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:55:23.340+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:55:23.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:55:23.340+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:55:23.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:55:23.340+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:55:23.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:55:23 smithi138 ceph-mon[36954]: pgmap v6598: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:55:23.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:55:23 smithi100 ceph-mon[32599]: pgmap v6598: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:55:23.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:55:23 smithi167 ceph-mon[37140]: pgmap v6598: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:55:25.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:55:25 smithi138 ceph-mon[36954]: pgmap v6599: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:55:25.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:55:25 smithi100 ceph-mon[32599]: pgmap v6599: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:55:25.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:55:25 smithi167 ceph-mon[37140]: pgmap v6599: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:55:27.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:55:27 smithi138 ceph-mon[36954]: pgmap v6600: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:55:27.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:55:27 smithi100 ceph-mon[32599]: pgmap v6600: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:55:27.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:55:27 smithi167 ceph-mon[37140]: pgmap v6600: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:55:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:55:29 smithi138 ceph-mon[36954]: pgmap v6601: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:55:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:55:29 smithi100 ceph-mon[32599]: pgmap v6601: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:55:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:55:29 smithi167 ceph-mon[37140]: pgmap v6601: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:55:31.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:55:31 smithi138 ceph-mon[36954]: pgmap v6602: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:55:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:55:31 smithi100 ceph-mon[32599]: pgmap v6602: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:55:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:55:31 smithi167 ceph-mon[37140]: pgmap v6602: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:55:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:55:33 smithi138 ceph-mon[36954]: pgmap v6603: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:55:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:55:33 smithi100 ceph-mon[32599]: pgmap v6603: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:55:33.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:55:33 smithi167 ceph-mon[37140]: pgmap v6603: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T16:55:35.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:55:35 smithi138 ceph-mon[36954]: pgmap v6604: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:55:35.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:55:35 smithi100 ceph-mon[32599]: pgmap v6604: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:55:35.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:55:35 smithi167 ceph-mon[37140]: pgmap v6604: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T16:55:37.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:55:37 smithi138 ceph-mon[36954]: pgmap v6605: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:55:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:55:37 smithi100 ceph-mon[32599]: pgmap v6605: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:55:37.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:55:37 smithi167 ceph-mon[37140]: pgmap v6605: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:55:38.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:55:38.343+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:55:38.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:55:38.344+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:55:38.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:55:38.344+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:55:38.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:55:38.344+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:55:38.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:55:38.344+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:55:39.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:55:39 smithi138 ceph-mon[36954]: pgmap v6606: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:55:39.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:55:39 smithi100 ceph-mon[32599]: pgmap v6606: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:55:39.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:55:39 smithi167 ceph-mon[37140]: pgmap v6606: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:55:41.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:55:41 smithi138 ceph-mon[36954]: pgmap v6607: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:55:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:55:41 smithi100 ceph-mon[32599]: pgmap v6607: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:55:41.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:55:41 smithi167 ceph-mon[37140]: pgmap v6607: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:55:43.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:55:43 smithi138 ceph-mon[36954]: pgmap v6608: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:55:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:55:43 smithi100 ceph-mon[32599]: pgmap v6608: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:55:43.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:55:43 smithi167 ceph-mon[37140]: pgmap v6608: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:55:45.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:55:45 smithi138 ceph-mon[36954]: pgmap v6609: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:55:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:55:45 smithi100 ceph-mon[32599]: pgmap v6609: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:55:45.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:55:45 smithi167 ceph-mon[37140]: pgmap v6609: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:55:47.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:55:47 smithi138 ceph-mon[36954]: pgmap v6610: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:55:47.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:55:47 smithi100 ceph-mon[32599]: pgmap v6610: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:55:47.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:55:47 smithi167 ceph-mon[37140]: pgmap v6610: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:55:49.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:55:49 smithi138 ceph-mon[36954]: pgmap v6611: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:55:49.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:55:49 smithi100 ceph-mon[32599]: pgmap v6611: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:55:49.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:55:49 smithi167 ceph-mon[37140]: pgmap v6611: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:55:51.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:55:51 smithi138 ceph-mon[36954]: pgmap v6612: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:55:51.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:55:51 smithi100 ceph-mon[32599]: pgmap v6612: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:55:51.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:55:51 smithi167 ceph-mon[37140]: pgmap v6612: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:55:53.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:55:53.359+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:55:53.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:55:53.359+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:55:53.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:55:53.359+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:55:53.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:55:53.359+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:55:53.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:55:53.360+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:55:53.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:55:53 smithi138 ceph-mon[36954]: pgmap v6613: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:55:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:55:53 smithi100 ceph-mon[32599]: pgmap v6613: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:55:53.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:55:53 smithi167 ceph-mon[37140]: pgmap v6613: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:55:54.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:55:54 smithi100 ceph-mon[32599]: pgmap v6614: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:55:54.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:55:54 smithi167 ceph-mon[37140]: pgmap v6614: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:55:54.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:55:54 smithi138 ceph-mon[36954]: pgmap v6614: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:55:57.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:55:57 smithi138 ceph-mon[36954]: pgmap v6615: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:55:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:55:57 smithi100 ceph-mon[32599]: pgmap v6615: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:55:57.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:55:57 smithi167 ceph-mon[37140]: pgmap v6615: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:55:59.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:55:59 smithi138 ceph-mon[36954]: pgmap v6616: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:55:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:55:59 smithi100 ceph-mon[32599]: pgmap v6616: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:55:59.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:55:59 smithi167 ceph-mon[37140]: pgmap v6616: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:56:01.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:56:01 smithi138 ceph-mon[36954]: pgmap v6617: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:56:01.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:56:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:56:01.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:56:01 smithi100 ceph-mon[32599]: pgmap v6617: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:56:01.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:56:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:56:01.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:56:01 smithi167 ceph-mon[37140]: pgmap v6617: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:56:01.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:56:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:56:02.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:56:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:56:02.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:56:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:56:02.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:56:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:56:02.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:56:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:56:02.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:56:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:56:02.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:56:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:56:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:56:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:56:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:56:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:56:02.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:56:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:56:03.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:56:03 smithi138 ceph-mon[36954]: pgmap v6618: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:56:03.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:56:03 smithi100 ceph-mon[32599]: pgmap v6618: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:56:03.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:56:03 smithi167 ceph-mon[37140]: pgmap v6618: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:56:05.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:56:05 smithi138 ceph-mon[36954]: pgmap v6619: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:56:05.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:56:05 smithi100 ceph-mon[32599]: pgmap v6619: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:56:05.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:56:05 smithi167 ceph-mon[37140]: pgmap v6619: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:56:07.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:56:07 smithi138 ceph-mon[36954]: pgmap v6620: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:56:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:56:07 smithi100 ceph-mon[32599]: pgmap v6620: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:56:07.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:56:07 smithi167 ceph-mon[37140]: pgmap v6620: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:56:08.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:56:08.361+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:56:08.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:56:08.362+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:56:08.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:56:08.362+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:56:08.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:56:08.362+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:56:08.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:56:08.362+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:56:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:56:08 smithi138 ceph-mon[36954]: osdmap e460: 12 total, 12 up, 12 in 2024-07-26T16:56:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:56:08 smithi100 ceph-mon[32599]: osdmap e460: 12 total, 12 up, 12 in 2024-07-26T16:56:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:56:08 smithi167 ceph-mon[37140]: osdmap e460: 12 total, 12 up, 12 in 2024-07-26T16:56:09.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:56:09 smithi138 ceph-mon[36954]: pgmap v6622: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:56:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:56:09 smithi100 ceph-mon[32599]: pgmap v6622: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:56:09.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:56:09 smithi167 ceph-mon[37140]: pgmap v6622: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:56:11.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:56:11 smithi138 ceph-mon[36954]: pgmap v6623: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:56:11.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:56:11 smithi100 ceph-mon[32599]: pgmap v6623: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:56:11.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:56:11 smithi167 ceph-mon[37140]: pgmap v6623: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:56:13.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:56:13 smithi138 ceph-mon[36954]: pgmap v6624: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:56:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:56:13 smithi100 ceph-mon[32599]: pgmap v6624: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:56:13.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:56:13 smithi167 ceph-mon[37140]: pgmap v6624: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:56:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:56:14 smithi100 ceph-mon[32599]: pgmap v6625: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:56:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:56:14 smithi167 ceph-mon[37140]: pgmap v6625: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:56:14.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:56:14 smithi138 ceph-mon[36954]: pgmap v6625: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:56:17.644 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:56:17 smithi138 ceph-mon[36954]: pgmap v6626: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:56:17.645 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:56:17 smithi138 ceph-mon[36954]: osdmap e461: 12 total, 12 up, 12 in 2024-07-26T16:56:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:56:17 smithi100 ceph-mon[32599]: pgmap v6626: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:56:17.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:56:17 smithi100 ceph-mon[32599]: osdmap e461: 12 total, 12 up, 12 in 2024-07-26T16:56:17.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:56:17 smithi167 ceph-mon[37140]: pgmap v6626: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:56:17.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:56:17 smithi167 ceph-mon[37140]: osdmap e461: 12 total, 12 up, 12 in 2024-07-26T16:56:19.694 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:56:19 smithi138 ceph-mon[36954]: pgmap v6628: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:56:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:56:19 smithi100 ceph-mon[32599]: pgmap v6628: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:56:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:56:19 smithi167 ceph-mon[37140]: pgmap v6628: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:56:21.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:56:21 smithi138 ceph-mon[36954]: pgmap v6629: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:56:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:56:21 smithi100 ceph-mon[32599]: pgmap v6629: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:56:21.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:56:21 smithi167 ceph-mon[37140]: pgmap v6629: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:56:23.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:56:23.364+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:56:23.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:56:23.364+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:56:23.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:56:23.364+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:56:23.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:56:23.364+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:56:23.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:56:23.364+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:56:23.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:56:23 smithi138 ceph-mon[36954]: pgmap v6630: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:56:23.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:56:23 smithi100 ceph-mon[32599]: pgmap v6630: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:56:23.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:56:23 smithi167 ceph-mon[37140]: pgmap v6630: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:56:25.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:56:25 smithi138 ceph-mon[36954]: pgmap v6631: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:56:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:56:25 smithi100 ceph-mon[32599]: pgmap v6631: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:56:25.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:56:25 smithi167 ceph-mon[37140]: pgmap v6631: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:56:27.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:56:27 smithi138 ceph-mon[36954]: pgmap v6632: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:56:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:56:27 smithi100 ceph-mon[32599]: pgmap v6632: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:56:27.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:56:27 smithi167 ceph-mon[37140]: pgmap v6632: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:56:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:56:29 smithi138 ceph-mon[36954]: pgmap v6633: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:56:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:56:29 smithi100 ceph-mon[32599]: pgmap v6633: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:56:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:56:29 smithi167 ceph-mon[37140]: pgmap v6633: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:56:31.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:56:31 smithi138 ceph-mon[36954]: pgmap v6634: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:56:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:56:31 smithi100 ceph-mon[32599]: pgmap v6634: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:56:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:56:31 smithi167 ceph-mon[37140]: pgmap v6634: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:56:33.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:56:32 smithi138 ceph-mon[36954]: pgmap v6635: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:56:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:56:32 smithi100 ceph-mon[32599]: pgmap v6635: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:56:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:56:32 smithi167 ceph-mon[37140]: pgmap v6635: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:56:35.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:56:35 smithi138 ceph-mon[36954]: pgmap v6636: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:56:35.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:56:35 smithi100 ceph-mon[32599]: pgmap v6636: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:56:35.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:56:35 smithi167 ceph-mon[37140]: pgmap v6636: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:56:37.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:56:37 smithi138 ceph-mon[36954]: pgmap v6637: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:56:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:56:37 smithi100 ceph-mon[32599]: pgmap v6637: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:56:37.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:56:37 smithi167 ceph-mon[37140]: pgmap v6637: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:56:38.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:56:38.368+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:56:38.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:56:38.368+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:56:38.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:56:38.368+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:56:38.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:56:38.368+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:56:38.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:56:38.368+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:56:39.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:56:39 smithi138 ceph-mon[36954]: pgmap v6638: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:56:39.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:56:39 smithi100 ceph-mon[32599]: pgmap v6638: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:56:39.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:56:39 smithi167 ceph-mon[37140]: pgmap v6638: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:56:41.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:56:41 smithi138 ceph-mon[36954]: pgmap v6639: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:56:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:56:41 smithi100 ceph-mon[32599]: pgmap v6639: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:56:41.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:56:41 smithi167 ceph-mon[37140]: pgmap v6639: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:56:43.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:56:43 smithi138 ceph-mon[36954]: pgmap v6640: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:56:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:56:43 smithi100 ceph-mon[32599]: pgmap v6640: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:56:43.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:56:43 smithi167 ceph-mon[37140]: pgmap v6640: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:56:45.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:56:45 smithi138 ceph-mon[36954]: pgmap v6641: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:56:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:56:45 smithi100 ceph-mon[32599]: pgmap v6641: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:56:45.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:56:45 smithi167 ceph-mon[37140]: pgmap v6641: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:56:47.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:56:47 smithi138 ceph-mon[36954]: pgmap v6642: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:56:47.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:56:47 smithi100 ceph-mon[32599]: pgmap v6642: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:56:47.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:56:47 smithi167 ceph-mon[37140]: pgmap v6642: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:56:49.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:56:49 smithi138 ceph-mon[36954]: pgmap v6643: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:56:49.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:56:49 smithi100 ceph-mon[32599]: pgmap v6643: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:56:49.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:56:49 smithi167 ceph-mon[37140]: pgmap v6643: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:56:51.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:56:51 smithi138 ceph-mon[36954]: pgmap v6644: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:56:51.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:56:51 smithi100 ceph-mon[32599]: pgmap v6644: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:56:51.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:56:51 smithi167 ceph-mon[37140]: pgmap v6644: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:56:53.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:56:53.375+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:56:53.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:56:53.376+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:56:53.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:56:53.376+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:56:53.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:56:53.376+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:56:53.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:56:53.376+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:56:53.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:56:53 smithi138 ceph-mon[36954]: pgmap v6645: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:56:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:56:53 smithi100 ceph-mon[32599]: pgmap v6645: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:56:53.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:56:53 smithi167 ceph-mon[37140]: pgmap v6645: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:56:55.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:56:55 smithi138 ceph-mon[36954]: pgmap v6646: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:56:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:56:55 smithi100 ceph-mon[32599]: pgmap v6646: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:56:55.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:56:55 smithi167 ceph-mon[37140]: pgmap v6646: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:56:57.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:56:57 smithi138 ceph-mon[36954]: pgmap v6647: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:56:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:56:57 smithi100 ceph-mon[32599]: pgmap v6647: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:56:57.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:56:57 smithi167 ceph-mon[37140]: pgmap v6647: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:56:59.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:56:59 smithi138 ceph-mon[36954]: pgmap v6648: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:56:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:56:59 smithi100 ceph-mon[32599]: pgmap v6648: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:56:59.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:56:59 smithi167 ceph-mon[37140]: pgmap v6648: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:57:00.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:57:00 smithi138 ceph-mon[36954]: pgmap v6649: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:57:00.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:57:00 smithi100 ceph-mon[32599]: pgmap v6649: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:57:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:57:00 smithi167 ceph-mon[37140]: pgmap v6649: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:57:02.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:57:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:57:02.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:57:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:57:02.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:57:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:57:03.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:57:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:57:03.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:57:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:57:03.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:57:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:57:03.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:57:02 smithi138 ceph-mon[36954]: pgmap v6650: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:57:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:57:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:57:03.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:57:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:57:03.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:57:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:57:03.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:57:02 smithi100 ceph-mon[32599]: pgmap v6650: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:57:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:57:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:57:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:57:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:57:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:57:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:57:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:57:02 smithi167 ceph-mon[37140]: pgmap v6650: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:57:04.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:57:04.622+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T16:57:04.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:57:04.622+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T16:57:04.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:57:04.622+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 90 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T16:57:04.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:57:04.622+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 90 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T16:57:05.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:57:05 smithi138 ceph-mon[36954]: pgmap v6651: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:57:05.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:57:05 smithi100 ceph-mon[32599]: pgmap v6651: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:57:05.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:57:05 smithi167 ceph-mon[37140]: pgmap v6651: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:57:07.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:57:07 smithi138 ceph-mon[36954]: pgmap v6652: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:57:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:57:07 smithi100 ceph-mon[32599]: pgmap v6652: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:57:07.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:57:07 smithi167 ceph-mon[37140]: pgmap v6652: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:57:08.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:57:08.392+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:57:08.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:57:08.392+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:57:08.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:57:08.392+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:57:08.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:57:08.392+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:57:08.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:57:08.392+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:57:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:57:08 smithi138 ceph-mon[36954]: osdmap e462: 12 total, 12 up, 12 in 2024-07-26T16:57:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:57:08 smithi100 ceph-mon[32599]: osdmap e462: 12 total, 12 up, 12 in 2024-07-26T16:57:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:57:08 smithi167 ceph-mon[37140]: osdmap e462: 12 total, 12 up, 12 in 2024-07-26T16:57:09.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:57:09 smithi138 ceph-mon[36954]: pgmap v6654: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:57:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:57:09 smithi100 ceph-mon[32599]: pgmap v6654: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:57:09.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:57:09 smithi167 ceph-mon[37140]: pgmap v6654: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:57:11.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:57:11 smithi138 ceph-mon[36954]: pgmap v6655: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:57:11.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:57:11 smithi100 ceph-mon[32599]: pgmap v6655: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:57:11.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:57:11 smithi167 ceph-mon[37140]: pgmap v6655: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:57:13.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:57:13 smithi138 ceph-mon[36954]: pgmap v6656: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:57:13.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:57:13 smithi100 ceph-mon[32599]: pgmap v6656: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:57:13.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:57:13 smithi167 ceph-mon[37140]: pgmap v6656: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:57:14.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:57:14 smithi138 ceph-mon[36954]: pgmap v6657: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:57:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:57:14 smithi100 ceph-mon[32599]: pgmap v6657: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:57:14.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:57:14 smithi167 ceph-mon[37140]: pgmap v6657: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:57:17.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:57:17 smithi138 ceph-mon[36954]: pgmap v6658: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:57:17.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:57:17 smithi138 ceph-mon[36954]: osdmap e463: 12 total, 12 up, 12 in 2024-07-26T16:57:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:57:17 smithi100 ceph-mon[32599]: pgmap v6658: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:57:17.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:57:17 smithi100 ceph-mon[32599]: osdmap e463: 12 total, 12 up, 12 in 2024-07-26T16:57:17.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:57:17 smithi167 ceph-mon[37140]: pgmap v6658: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:57:17.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:57:17 smithi167 ceph-mon[37140]: osdmap e463: 12 total, 12 up, 12 in 2024-07-26T16:57:19.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:57:19 smithi138 ceph-mon[36954]: pgmap v6660: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:57:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:57:19 smithi100 ceph-mon[32599]: pgmap v6660: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:57:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:57:19 smithi167 ceph-mon[37140]: pgmap v6660: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:57:21.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:57:21 smithi138 ceph-mon[36954]: pgmap v6661: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:57:21.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:57:21 smithi100 ceph-mon[32599]: pgmap v6661: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:57:21.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:57:21 smithi167 ceph-mon[37140]: pgmap v6661: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:57:23.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:57:23.393+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:57:23.395 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:57:23.393+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:57:23.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:57:23.393+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:57:23.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:57:23.393+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:57:23.396 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:57:23.393+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:57:23.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:57:23 smithi138 ceph-mon[36954]: pgmap v6662: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:57:23.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:57:23 smithi100 ceph-mon[32599]: pgmap v6662: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:57:23.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:57:23 smithi167 ceph-mon[37140]: pgmap v6662: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:57:25.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:57:25 smithi138 ceph-mon[36954]: pgmap v6663: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:57:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:57:25 smithi100 ceph-mon[32599]: pgmap v6663: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:57:25.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:57:25 smithi167 ceph-mon[37140]: pgmap v6663: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:57:27.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:57:27 smithi138 ceph-mon[36954]: pgmap v6664: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:57:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:57:27 smithi100 ceph-mon[32599]: pgmap v6664: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:57:27.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:57:27 smithi167 ceph-mon[37140]: pgmap v6664: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:57:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:57:29 smithi138 ceph-mon[36954]: pgmap v6665: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:57:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:57:29 smithi100 ceph-mon[32599]: pgmap v6665: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:57:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:57:29 smithi167 ceph-mon[37140]: pgmap v6665: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:57:31.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:57:31 smithi138 ceph-mon[36954]: pgmap v6666: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:57:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:57:31 smithi100 ceph-mon[32599]: pgmap v6666: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:57:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:57:31 smithi167 ceph-mon[37140]: pgmap v6666: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:57:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:57:33 smithi138 ceph-mon[36954]: pgmap v6667: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:57:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:57:33 smithi100 ceph-mon[32599]: pgmap v6667: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:57:33.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:57:33 smithi167 ceph-mon[37140]: pgmap v6667: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:57:35.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:57:35 smithi138 ceph-mon[36954]: pgmap v6668: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:57:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:57:35 smithi100 ceph-mon[32599]: pgmap v6668: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:57:35.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:57:35 smithi167 ceph-mon[37140]: pgmap v6668: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:57:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:57:36 smithi100 ceph-mon[32599]: pgmap v6669: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:57:36.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:57:36 smithi167 ceph-mon[37140]: pgmap v6669: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:57:36.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:57:36 smithi138 ceph-mon[36954]: pgmap v6669: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:57:38.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:57:38.396+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:57:38.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:57:38.397+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:57:38.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:57:38.397+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:57:38.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:57:38.397+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:57:38.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:57:38.397+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:57:39.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:57:39 smithi138 ceph-mon[36954]: pgmap v6670: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:57:39.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:57:39 smithi100 ceph-mon[32599]: pgmap v6670: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:57:39.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:57:39 smithi167 ceph-mon[37140]: pgmap v6670: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:57:41.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:57:41 smithi138 ceph-mon[36954]: pgmap v6671: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:57:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:57:41 smithi100 ceph-mon[32599]: pgmap v6671: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:57:41.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:57:41 smithi167 ceph-mon[37140]: pgmap v6671: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:57:43.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:57:43 smithi138 ceph-mon[36954]: pgmap v6672: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:57:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:57:43 smithi100 ceph-mon[32599]: pgmap v6672: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:57:43.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:57:43 smithi167 ceph-mon[37140]: pgmap v6672: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:57:45.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:57:45 smithi138 ceph-mon[36954]: pgmap v6673: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:57:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:57:45 smithi100 ceph-mon[32599]: pgmap v6673: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:57:45.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:57:45 smithi167 ceph-mon[37140]: pgmap v6673: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:57:47.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:57:47 smithi138 ceph-mon[36954]: pgmap v6674: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:57:47.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:57:47 smithi100 ceph-mon[32599]: pgmap v6674: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:57:47.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:57:47 smithi167 ceph-mon[37140]: pgmap v6674: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:57:49.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:57:49.628+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T16:57:49.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:57:49.628+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T16:57:49.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:57:49.628+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 91 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T16:57:49.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:57:49.628+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 91 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T16:57:49.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:57:49 smithi138 ceph-mon[36954]: pgmap v6675: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:57:49.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:57:49 smithi100 ceph-mon[32599]: pgmap v6675: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:57:49.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:57:49 smithi167 ceph-mon[37140]: pgmap v6675: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:57:51.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:57:51 smithi138 ceph-mon[36954]: pgmap v6676: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:57:51.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:57:51 smithi100 ceph-mon[32599]: pgmap v6676: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:57:51.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:57:51 smithi167 ceph-mon[37140]: pgmap v6676: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:57:53.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:57:53.408+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:57:53.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:57:53.409+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:57:53.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:57:53.409+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:57:53.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:57:53.409+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:57:53.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:57:53.409+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:57:53.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:57:53 smithi138 ceph-mon[36954]: pgmap v6677: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:57:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:57:53 smithi100 ceph-mon[32599]: pgmap v6677: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:57:53.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:57:53 smithi167 ceph-mon[37140]: pgmap v6677: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:57:55.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:57:55 smithi138 ceph-mon[36954]: pgmap v6678: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:57:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:57:55 smithi100 ceph-mon[32599]: pgmap v6678: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:57:55.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:57:55 smithi167 ceph-mon[37140]: pgmap v6678: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:57:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:57:56 smithi100 ceph-mon[32599]: pgmap v6679: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:57:56.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:57:56 smithi167 ceph-mon[37140]: pgmap v6679: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:57:56.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:57:56 smithi138 ceph-mon[36954]: pgmap v6679: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:57:59.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:57:59 smithi138 ceph-mon[36954]: pgmap v6680: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:57:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:57:59 smithi100 ceph-mon[32599]: pgmap v6680: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:57:59.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:57:59 smithi167 ceph-mon[37140]: pgmap v6680: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:58:01.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:58:01 smithi138 ceph-mon[36954]: pgmap v6681: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:58:01.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:58:01 smithi100 ceph-mon[32599]: pgmap v6681: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:58:01.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:58:01 smithi167 ceph-mon[37140]: pgmap v6681: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:58:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:58:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:58:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:58:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:58:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:58:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:58:03.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:58:03 smithi138 ceph-mon[36954]: pgmap v6682: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:58:03.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:58:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:58:03.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:58:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:58:03.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:58:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:58:03.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:58:03 smithi100 ceph-mon[32599]: pgmap v6682: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:58:03.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:58:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:58:03.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:58:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:58:03.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:58:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:58:03.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:58:03 smithi167 ceph-mon[37140]: pgmap v6682: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:58:03.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:58:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:58:03.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:58:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:58:03.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:58:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:58:05.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:58:05 smithi138 ceph-mon[36954]: pgmap v6683: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:58:05.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:58:05 smithi100 ceph-mon[32599]: pgmap v6683: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:58:05.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:58:05 smithi167 ceph-mon[37140]: pgmap v6683: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:58:07.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:58:07 smithi138 ceph-mon[36954]: pgmap v6684: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:58:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:58:07 smithi100 ceph-mon[32599]: pgmap v6684: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:58:07.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:58:07 smithi167 ceph-mon[37140]: pgmap v6684: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:58:08.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:58:08.425+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:58:08.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:58:08.426+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:58:08.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:58:08.426+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:58:08.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:58:08.426+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:58:08.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:58:08.426+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:58:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:58:08 smithi100 ceph-mon[32599]: osdmap e464: 12 total, 12 up, 12 in 2024-07-26T16:58:08.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:58:08 smithi100 ceph-mon[32599]: pgmap v6686: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:58:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:58:08 smithi167 ceph-mon[37140]: osdmap e464: 12 total, 12 up, 12 in 2024-07-26T16:58:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:58:08 smithi167 ceph-mon[37140]: pgmap v6686: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:58:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:58:08 smithi138 ceph-mon[36954]: osdmap e464: 12 total, 12 up, 12 in 2024-07-26T16:58:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:58:08 smithi138 ceph-mon[36954]: pgmap v6686: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:58:11.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:58:11 smithi138 ceph-mon[36954]: pgmap v6687: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:58:11.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:58:11 smithi100 ceph-mon[32599]: pgmap v6687: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:58:11.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:58:11 smithi167 ceph-mon[37140]: pgmap v6687: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:58:13.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:58:13 smithi138 ceph-mon[36954]: pgmap v6688: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:58:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:58:13 smithi100 ceph-mon[32599]: pgmap v6688: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:58:13.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:58:13 smithi167 ceph-mon[37140]: pgmap v6688: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:58:15.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:58:15 smithi138 ceph-mon[36954]: pgmap v6689: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:58:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:58:15 smithi100 ceph-mon[32599]: pgmap v6689: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:58:15.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:58:15 smithi167 ceph-mon[37140]: pgmap v6689: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:58:17.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:58:17 smithi138 ceph-mon[36954]: pgmap v6690: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:58:17.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:58:17 smithi138 ceph-mon[36954]: osdmap e465: 12 total, 12 up, 12 in 2024-07-26T16:58:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:58:17 smithi100 ceph-mon[32599]: pgmap v6690: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:58:17.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:58:17 smithi100 ceph-mon[32599]: osdmap e465: 12 total, 12 up, 12 in 2024-07-26T16:58:17.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:58:17 smithi167 ceph-mon[37140]: pgmap v6690: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:58:17.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:58:17 smithi167 ceph-mon[37140]: osdmap e465: 12 total, 12 up, 12 in 2024-07-26T16:58:19.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:58:19 smithi138 ceph-mon[36954]: pgmap v6692: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:58:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:58:19 smithi100 ceph-mon[32599]: pgmap v6692: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:58:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:58:19 smithi167 ceph-mon[37140]: pgmap v6692: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:58:21.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:58:21 smithi138 ceph-mon[36954]: pgmap v6693: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:58:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:58:21 smithi100 ceph-mon[32599]: pgmap v6693: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:58:21.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:58:21 smithi167 ceph-mon[37140]: pgmap v6693: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:58:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:58:22 smithi100 ceph-mon[32599]: pgmap v6694: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:58:22.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:58:22 smithi167 ceph-mon[37140]: pgmap v6694: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:58:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:58:22 smithi138 ceph-mon[36954]: pgmap v6694: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:58:23.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:58:23.431+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:58:23.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:58:23.432+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:58:23.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:58:23.432+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:58:23.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:58:23.432+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:58:23.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:58:23.432+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:58:25.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:58:25 smithi138 ceph-mon[36954]: pgmap v6695: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:58:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:58:25 smithi100 ceph-mon[32599]: pgmap v6695: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:58:25.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:58:25 smithi167 ceph-mon[37140]: pgmap v6695: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:58:27.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:58:27 smithi138 ceph-mon[36954]: pgmap v6696: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:58:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:58:27 smithi100 ceph-mon[32599]: pgmap v6696: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:58:27.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:58:27 smithi167 ceph-mon[37140]: pgmap v6696: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:58:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:58:29 smithi138 ceph-mon[36954]: pgmap v6697: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:58:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:58:29 smithi100 ceph-mon[32599]: pgmap v6697: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:58:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:58:29 smithi167 ceph-mon[37140]: pgmap v6697: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:58:31.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:58:31 smithi138 ceph-mon[36954]: pgmap v6698: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:58:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:58:31 smithi100 ceph-mon[32599]: pgmap v6698: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:58:31.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:58:31 smithi167 ceph-mon[37140]: pgmap v6698: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:58:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:58:33 smithi138 ceph-mon[36954]: pgmap v6699: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:58:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:58:33 smithi100 ceph-mon[32599]: pgmap v6699: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:58:33.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:58:33 smithi167 ceph-mon[37140]: pgmap v6699: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:58:35.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:58:35 smithi100 ceph-mon[32599]: pgmap v6700: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:58:35.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:58:35 smithi167 ceph-mon[37140]: pgmap v6700: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:58:35.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:58:35 smithi138 ceph-mon[36954]: pgmap v6700: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:58:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:58:36 smithi100 ceph-mon[32599]: pgmap v6701: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:58:36.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:58:36 smithi167 ceph-mon[37140]: pgmap v6701: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:58:36.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:58:36 smithi138 ceph-mon[36954]: pgmap v6701: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:58:38.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:58:38.445+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:58:38.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:58:38.445+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:58:38.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:58:38.445+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:58:38.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:58:38.445+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:58:38.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:58:38.445+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:58:39.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:58:39 smithi138 ceph-mon[36954]: pgmap v6702: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:58:39.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:58:39 smithi100 ceph-mon[32599]: pgmap v6702: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:58:39.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:58:39 smithi167 ceph-mon[37140]: pgmap v6702: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:58:41.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:58:41 smithi138 ceph-mon[36954]: pgmap v6703: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:58:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:58:41 smithi100 ceph-mon[32599]: pgmap v6703: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:58:41.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:58:41 smithi167 ceph-mon[37140]: pgmap v6703: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:58:43.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:58:42 smithi138 ceph-mon[36954]: pgmap v6704: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:58:43.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:58:42 smithi100 ceph-mon[32599]: pgmap v6704: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:58:43.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:58:42 smithi167 ceph-mon[37140]: pgmap v6704: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:58:45.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:58:45 smithi138 ceph-mon[36954]: pgmap v6705: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:58:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:58:45 smithi100 ceph-mon[32599]: pgmap v6705: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:58:45.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:58:45 smithi167 ceph-mon[37140]: pgmap v6705: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:58:47.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:58:47 smithi138 ceph-mon[36954]: pgmap v6706: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:58:47.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:58:47 smithi100 ceph-mon[32599]: pgmap v6706: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:58:47.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:58:47 smithi167 ceph-mon[37140]: pgmap v6706: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:58:49.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:58:49 smithi138 ceph-mon[36954]: pgmap v6707: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:58:49.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:58:49 smithi100 ceph-mon[32599]: pgmap v6707: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:58:49.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:58:49 smithi167 ceph-mon[37140]: pgmap v6707: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:58:51.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:58:51 smithi100 ceph-mon[32599]: pgmap v6708: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:58:51.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:58:51 smithi167 ceph-mon[37140]: pgmap v6708: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:58:51.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:58:51 smithi138 ceph-mon[36954]: pgmap v6708: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:58:53.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:58:53.461+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:58:53.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:58:53.461+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:58:53.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:58:53.461+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:58:53.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:58:53.461+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:58:53.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:58:53.461+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:58:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:58:53 smithi100 ceph-mon[32599]: pgmap v6709: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:58:53.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:58:53 smithi167 ceph-mon[37140]: pgmap v6709: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:58:53.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:58:53 smithi138 ceph-mon[36954]: pgmap v6709: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:58:54.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:58:54 smithi100 ceph-mon[32599]: pgmap v6710: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:58:54.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:58:54 smithi167 ceph-mon[37140]: pgmap v6710: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:58:54.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:58:54 smithi138 ceph-mon[36954]: pgmap v6710: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:58:57.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:58:57 smithi138 ceph-mon[36954]: pgmap v6711: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:58:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:58:57 smithi100 ceph-mon[32599]: pgmap v6711: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:58:57.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:58:57 smithi167 ceph-mon[37140]: pgmap v6711: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:58:59.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:58:59 smithi138 ceph-mon[36954]: pgmap v6712: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:58:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:58:59 smithi100 ceph-mon[32599]: pgmap v6712: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:58:59.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:58:59 smithi167 ceph-mon[37140]: pgmap v6712: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:59:01.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:01 smithi100 ceph-mon[32599]: pgmap v6713: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:59:01.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:01 smithi167 ceph-mon[37140]: pgmap v6713: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:59:01.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:01 smithi138 ceph-mon[36954]: pgmap v6713: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:59:03.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:03 smithi100 ceph-mon[32599]: pgmap v6714: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:59:03.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:59:03.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:59:03.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:59:03.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:59:03.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:59:03.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:59:03.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:59:03.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:59:03.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:59:03.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:59:03.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:59:03.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:59:03.747 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:59:03.747 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:59:03.747 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:59:03.747 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:59:03.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:03 smithi167 ceph-mon[37140]: pgmap v6714: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:59:03.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:59:03.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:59:03.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:59:03.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:59:03.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:59:03.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:59:03.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:59:03.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:59:03.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:59:03.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:59:03.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:59:03.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:59:03.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:59:03.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:59:03.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:59:03.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:59:03.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:03 smithi138 ceph-mon[36954]: pgmap v6714: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T16:59:03.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T16:59:03.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:59:03.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:59:03.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:59:03.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:59:03.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:59:03.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:59:03.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:59:03.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:59:03.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:59:03.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:59:03.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:59:03.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T16:59:03.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T16:59:03.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T16:59:03.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T16:59:04.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:04 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T16:59:04.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:04 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T16:59:04.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:04 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T16:59:04.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:04 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T16:59:04.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:04 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T16:59:04.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:04 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T16:59:04.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:04 smithi100 ceph-mon[32599]: pgmap v6715: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:59:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:04 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T16:59:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:04 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T16:59:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:04 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T16:59:04.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:04 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T16:59:04.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:04 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T16:59:04.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:04 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T16:59:04.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:04 smithi167 ceph-mon[37140]: pgmap v6715: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:59:04.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:04 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T16:59:04.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:04 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T16:59:04.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:04 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T16:59:04.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:04 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T16:59:04.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:04 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T16:59:04.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:04 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T16:59:04.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:04 smithi138 ceph-mon[36954]: pgmap v6715: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T16:59:07.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:07 smithi100 ceph-mon[32599]: pgmap v6716: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:59:07.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:07 smithi167 ceph-mon[37140]: pgmap v6716: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:59:07.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:07 smithi138 ceph-mon[36954]: pgmap v6716: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T16:59:08.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:59:08.471+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:59:08.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:59:08.471+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:59:08.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:59:08.471+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:59:08.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:59:08.471+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:59:08.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:59:08.471+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:59:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:08 smithi100 ceph-mon[32599]: osdmap e466: 12 total, 12 up, 12 in 2024-07-26T16:59:08.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:08 smithi100 ceph-mon[32599]: pgmap v6718: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:59:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:08 smithi167 ceph-mon[37140]: osdmap e466: 12 total, 12 up, 12 in 2024-07-26T16:59:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:08 smithi167 ceph-mon[37140]: pgmap v6718: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:59:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:08 smithi138 ceph-mon[36954]: osdmap e466: 12 total, 12 up, 12 in 2024-07-26T16:59:08.999 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:08 smithi138 ceph-mon[36954]: pgmap v6718: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:59:11.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:11 smithi100 ceph-mon[32599]: pgmap v6719: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:59:11.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:11 smithi167 ceph-mon[37140]: pgmap v6719: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:59:11.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:11 smithi138 ceph-mon[36954]: pgmap v6719: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T16:59:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:13 smithi100 ceph-mon[32599]: pgmap v6720: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:59:13.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:13 smithi167 ceph-mon[37140]: pgmap v6720: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:59:13.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:13 smithi138 ceph-mon[36954]: pgmap v6720: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T16:59:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:15 smithi100 ceph-mon[32599]: pgmap v6721: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:59:15.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:15 smithi167 ceph-mon[37140]: pgmap v6721: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:59:15.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:15 smithi138 ceph-mon[36954]: pgmap v6721: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:59:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:16 smithi167 ceph-mon[37140]: pgmap v6722: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:59:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:16 smithi138 ceph-mon[36954]: pgmap v6722: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:59:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:16 smithi100 ceph-mon[32599]: pgmap v6722: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:59:18.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:17 smithi138 ceph-mon[36954]: osdmap e467: 12 total, 12 up, 12 in 2024-07-26T16:59:18.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:17 smithi100 ceph-mon[32599]: osdmap e467: 12 total, 12 up, 12 in 2024-07-26T16:59:18.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:17 smithi167 ceph-mon[37140]: osdmap e467: 12 total, 12 up, 12 in 2024-07-26T16:59:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:18 smithi138 ceph-mon[36954]: pgmap v6724: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:59:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:18 smithi100 ceph-mon[32599]: pgmap v6724: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:59:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:18 smithi167 ceph-mon[37140]: pgmap v6724: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T16:59:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:21 smithi100 ceph-mon[32599]: pgmap v6725: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:59:21.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:21 smithi167 ceph-mon[37140]: pgmap v6725: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:59:21.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:21 smithi138 ceph-mon[36954]: pgmap v6725: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:59:23.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:59:23.487+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:59:23.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:59:23.488+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:59:23.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:59:23.488+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:59:23.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:59:23.488+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:59:23.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:59:23.488+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:59:23.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:23 smithi100 ceph-mon[32599]: pgmap v6726: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:59:23.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:23 smithi167 ceph-mon[37140]: pgmap v6726: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:59:23.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:23 smithi138 ceph-mon[36954]: pgmap v6726: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T16:59:24.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:24 smithi167 ceph-mon[37140]: pgmap v6727: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:59:24.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:24 smithi138 ceph-mon[36954]: pgmap v6727: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:59:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:24 smithi100 ceph-mon[32599]: pgmap v6727: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:59:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:27 smithi100 ceph-mon[32599]: pgmap v6728: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:59:27.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:27 smithi167 ceph-mon[37140]: pgmap v6728: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:59:27.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:27 smithi138 ceph-mon[36954]: pgmap v6728: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T16:59:29.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:29 smithi100 ceph-mon[32599]: pgmap v6729: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:59:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:29 smithi167 ceph-mon[37140]: pgmap v6729: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:59:29.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:29 smithi138 ceph-mon[36954]: pgmap v6729: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T16:59:30.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:30 smithi100 ceph-mon[32599]: pgmap v6730: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:59:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:30 smithi167 ceph-mon[37140]: pgmap v6730: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:59:30.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:30 smithi138 ceph-mon[36954]: pgmap v6730: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:59:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:33 smithi100 ceph-mon[32599]: pgmap v6731: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:59:33.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:33 smithi167 ceph-mon[37140]: pgmap v6731: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:59:33.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:33 smithi138 ceph-mon[36954]: pgmap v6731: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:59:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:34 smithi167 ceph-mon[37140]: pgmap v6732: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:59:34.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:34 smithi138 ceph-mon[36954]: pgmap v6732: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:59:34.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:34 smithi100 ceph-mon[32599]: pgmap v6732: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:59:37.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:37 smithi100 ceph-mon[32599]: pgmap v6733: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:59:37.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:37 smithi167 ceph-mon[37140]: pgmap v6733: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:59:37.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:37 smithi138 ceph-mon[36954]: pgmap v6733: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:59:38.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:59:38.503+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:59:38.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:59:38.503+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:59:38.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:59:38.503+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:59:38.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:59:38.503+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:59:38.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:59:38.503+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:59:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:38 smithi167 ceph-mon[37140]: pgmap v6734: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:59:38.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:38 smithi138 ceph-mon[36954]: pgmap v6734: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:59:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:38 smithi100 ceph-mon[32599]: pgmap v6734: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:59:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:41 smithi100 ceph-mon[32599]: pgmap v6735: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:59:41.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:41 smithi167 ceph-mon[37140]: pgmap v6735: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:59:41.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:41 smithi138 ceph-mon[36954]: pgmap v6735: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:59:43.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:42 smithi138 ceph-mon[36954]: pgmap v6736: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:59:43.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:42 smithi100 ceph-mon[32599]: pgmap v6736: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:59:43.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:42 smithi167 ceph-mon[37140]: pgmap v6736: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:59:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:45 smithi100 ceph-mon[32599]: pgmap v6737: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:59:45.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:45 smithi167 ceph-mon[37140]: pgmap v6737: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:59:45.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:45 smithi138 ceph-mon[36954]: pgmap v6737: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:59:46.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:46 smithi100 ceph-mon[32599]: pgmap v6738: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:59:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:46 smithi167 ceph-mon[37140]: pgmap v6738: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:59:46.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:46 smithi138 ceph-mon[36954]: pgmap v6738: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:59:48.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:48 smithi167 ceph-mon[37140]: pgmap v6739: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:59:48.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:48 smithi138 ceph-mon[36954]: pgmap v6739: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:59:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:48 smithi100 ceph-mon[32599]: pgmap v6739: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:59:50.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:50 smithi167 ceph-mon[37140]: pgmap v6740: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:59:50.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:50 smithi138 ceph-mon[36954]: pgmap v6740: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:59:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:50 smithi100 ceph-mon[32599]: pgmap v6740: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:59:52.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:52 smithi167 ceph-mon[37140]: pgmap v6741: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:59:52.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:52 smithi100 ceph-mon[32599]: pgmap v6741: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:59:52.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:52 smithi138 ceph-mon[36954]: pgmap v6741: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:59:53.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:59:53.512+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T16:59:53.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:59:53.512+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T16:59:53.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:59:53.512+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T16:59:53.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:59:53.512+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T16:59:53.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T16:59:53.512+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T16:59:54.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:54 smithi167 ceph-mon[37140]: pgmap v6742: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:59:54.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:54 smithi138 ceph-mon[36954]: pgmap v6742: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:59:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:54 smithi100 ceph-mon[32599]: pgmap v6742: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T16:59:56.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:56 smithi167 ceph-mon[37140]: pgmap v6743: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:59:56.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:56 smithi138 ceph-mon[36954]: pgmap v6743: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:59:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:56 smithi100 ceph-mon[32599]: pgmap v6743: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T16:59:58.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 16:59:58 smithi167 ceph-mon[37140]: pgmap v6744: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:59:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 16:59:58 smithi138 ceph-mon[36954]: pgmap v6744: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T16:59:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 16:59:58 smithi100 ceph-mon[32599]: pgmap v6744: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:00:00.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:00:00 smithi138 ceph-mon[36954]: overall HEALTH_OK 2024-07-26T17:00:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:00:00 smithi100 ceph-mon[32599]: overall HEALTH_OK 2024-07-26T17:00:00.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:00:00 smithi167 ceph-mon[37140]: overall HEALTH_OK 2024-07-26T17:00:01.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:00:01 smithi138 ceph-mon[36954]: pgmap v6745: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:00:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:00:01 smithi100 ceph-mon[32599]: pgmap v6745: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:00:01.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:00:01 smithi167 ceph-mon[37140]: pgmap v6745: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:00:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:00:02 smithi167 ceph-mon[37140]: pgmap v6746: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:00:02.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:00:02 smithi138 ceph-mon[36954]: pgmap v6746: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:00:02.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:00:02 smithi100 ceph-mon[32599]: pgmap v6746: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:00:03.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:00:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:00:03.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:00:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:00:04.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:00:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:00:04.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:00:04.647+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T17:00:04.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:00:04.647+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T17:00:04.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:00:04.647+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 92 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T17:00:04.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:00:04.647+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 92 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T17:00:04.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:00:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:00:04.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:00:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:00:04.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:00:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:00:04.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:00:04 smithi138 ceph-mon[36954]: pgmap v6747: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:00:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:00:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:00:04.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:00:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:00:04.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:00:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:00:04.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:00:04 smithi100 ceph-mon[32599]: pgmap v6747: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:00:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:00:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:00:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:00:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:00:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:00:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:00:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:00:04 smithi167 ceph-mon[37140]: pgmap v6747: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:00:06.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:00:06 smithi167 ceph-mon[37140]: pgmap v6748: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:00:06.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:00:06 smithi138 ceph-mon[36954]: pgmap v6748: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:00:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:00:06 smithi100 ceph-mon[32599]: pgmap v6748: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:00:08.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:00:08.527+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:00:08.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:00:08.527+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:00:08.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:00:08.527+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:00:08.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:00:08.527+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:00:08.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:00:08.528+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:00:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:00:08 smithi167 ceph-mon[37140]: osdmap e468: 12 total, 12 up, 12 in 2024-07-26T17:00:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:00:08 smithi167 ceph-mon[37140]: pgmap v6750: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:00:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:00:08 smithi138 ceph-mon[36954]: osdmap e468: 12 total, 12 up, 12 in 2024-07-26T17:00:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:00:08 smithi138 ceph-mon[36954]: pgmap v6750: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:00:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:00:08 smithi100 ceph-mon[32599]: osdmap e468: 12 total, 12 up, 12 in 2024-07-26T17:00:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:00:08 smithi100 ceph-mon[32599]: pgmap v6750: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:00:10.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:00:10 smithi167 ceph-mon[37140]: pgmap v6751: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:00:10.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:00:10 smithi138 ceph-mon[36954]: pgmap v6751: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:00:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:00:10 smithi100 ceph-mon[32599]: pgmap v6751: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:00:12.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:00:12 smithi167 ceph-mon[37140]: pgmap v6752: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:00:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:00:12 smithi138 ceph-mon[36954]: pgmap v6752: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:00:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:00:12 smithi100 ceph-mon[32599]: pgmap v6752: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:00:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:00:14 smithi167 ceph-mon[37140]: pgmap v6753: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:00:14.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:00:14 smithi138 ceph-mon[36954]: pgmap v6753: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:00:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:00:14 smithi100 ceph-mon[32599]: pgmap v6753: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:00:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:00:16 smithi167 ceph-mon[37140]: pgmap v6754: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:00:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:00:16 smithi138 ceph-mon[36954]: pgmap v6754: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:00:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:00:16 smithi100 ceph-mon[32599]: pgmap v6754: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:00:18.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:00:17 smithi138 ceph-mon[36954]: osdmap e469: 12 total, 12 up, 12 in 2024-07-26T17:00:18.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:00:17 smithi100 ceph-mon[32599]: osdmap e469: 12 total, 12 up, 12 in 2024-07-26T17:00:18.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:00:17 smithi167 ceph-mon[37140]: osdmap e469: 12 total, 12 up, 12 in 2024-07-26T17:00:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:00:18 smithi138 ceph-mon[36954]: pgmap v6756: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:00:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:00:18 smithi100 ceph-mon[32599]: pgmap v6756: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:00:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:00:18 smithi167 ceph-mon[37140]: pgmap v6756: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:00:20.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:00:20 smithi167 ceph-mon[37140]: pgmap v6757: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:00:20.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:00:20 smithi138 ceph-mon[36954]: pgmap v6757: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:00:20.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:00:20 smithi100 ceph-mon[32599]: pgmap v6757: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:00:22.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:00:22 smithi167 ceph-mon[37140]: pgmap v6758: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:00:22.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:00:22 smithi100 ceph-mon[32599]: pgmap v6758: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:00:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:00:22 smithi138 ceph-mon[36954]: pgmap v6758: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:00:23.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:00:23.535+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:00:23.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:00:23.535+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:00:23.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:00:23.535+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:00:23.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:00:23.535+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:00:23.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:00:23.535+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:00:24.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:00:24 smithi167 ceph-mon[37140]: pgmap v6759: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:00:24.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:00:24 smithi138 ceph-mon[36954]: pgmap v6759: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:00:24.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:00:24 smithi100 ceph-mon[32599]: pgmap v6759: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:00:26.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:00:26 smithi167 ceph-mon[37140]: pgmap v6760: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:00:26.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:00:26 smithi138 ceph-mon[36954]: pgmap v6760: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:00:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:00:26 smithi100 ceph-mon[32599]: pgmap v6760: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:00:28.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:00:28 smithi167 ceph-mon[37140]: pgmap v6761: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T17:00:28.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:00:28 smithi138 ceph-mon[36954]: pgmap v6761: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T17:00:28.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:00:28 smithi100 ceph-mon[32599]: pgmap v6761: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T17:00:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:00:30 smithi167 ceph-mon[37140]: pgmap v6762: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:00:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:00:30 smithi138 ceph-mon[36954]: pgmap v6762: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:00:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:00:30 smithi100 ceph-mon[32599]: pgmap v6762: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:00:33.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:00:32 smithi138 ceph-mon[36954]: pgmap v6763: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:00:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:00:32 smithi100 ceph-mon[32599]: pgmap v6763: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:00:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:00:32 smithi167 ceph-mon[37140]: pgmap v6763: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:00:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:00:34 smithi167 ceph-mon[37140]: pgmap v6764: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:00:34.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:00:34 smithi138 ceph-mon[36954]: pgmap v6764: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:00:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:00:34 smithi100 ceph-mon[32599]: pgmap v6764: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:00:36.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:00:36 smithi167 ceph-mon[37140]: pgmap v6765: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:00:36.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:00:36 smithi138 ceph-mon[36954]: pgmap v6765: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:00:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:00:36 smithi100 ceph-mon[32599]: pgmap v6765: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:00:38.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:00:38.551+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:00:38.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:00:38.552+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:00:38.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:00:38.552+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:00:38.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:00:38.552+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:00:38.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:00:38.552+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:00:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:00:38 smithi167 ceph-mon[37140]: pgmap v6766: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:00:38.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:00:38 smithi138 ceph-mon[36954]: pgmap v6766: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:00:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:00:38 smithi100 ceph-mon[32599]: pgmap v6766: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:00:40.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:00:40 smithi167 ceph-mon[37140]: pgmap v6767: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:00:40.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:00:40 smithi138 ceph-mon[36954]: pgmap v6767: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:00:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:00:40 smithi100 ceph-mon[32599]: pgmap v6767: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:00:42.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:00:42 smithi167 ceph-mon[37140]: pgmap v6768: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:00:42.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:00:42 smithi138 ceph-mon[36954]: pgmap v6768: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:00:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:00:42 smithi100 ceph-mon[32599]: pgmap v6768: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:00:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:00:44 smithi167 ceph-mon[37140]: pgmap v6769: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:00:44.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:00:44 smithi138 ceph-mon[36954]: pgmap v6769: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:00:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:00:44 smithi100 ceph-mon[32599]: pgmap v6769: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:00:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:00:46 smithi167 ceph-mon[37140]: pgmap v6770: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:00:46.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:00:46 smithi138 ceph-mon[36954]: pgmap v6770: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:00:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:00:46 smithi100 ceph-mon[32599]: pgmap v6770: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:00:48.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:00:48 smithi167 ceph-mon[37140]: pgmap v6771: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:00:48.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:00:48 smithi138 ceph-mon[36954]: pgmap v6771: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:00:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:00:48 smithi100 ceph-mon[32599]: pgmap v6771: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:00:50.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:00:50 smithi167 ceph-mon[37140]: pgmap v6772: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:00:50.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:00:50 smithi138 ceph-mon[36954]: pgmap v6772: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:00:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:00:50 smithi100 ceph-mon[32599]: pgmap v6772: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:00:52.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:00:52 smithi167 ceph-mon[37140]: pgmap v6773: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:00:52.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:00:52 smithi100 ceph-mon[32599]: pgmap v6773: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:00:52.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:00:52 smithi138 ceph-mon[36954]: pgmap v6773: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:00:53.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:00:53.559+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:00:53.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:00:53.560+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:00:53.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:00:53.560+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:00:53.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:00:53.560+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:00:53.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:00:53.560+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:00:54.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:00:54 smithi167 ceph-mon[37140]: pgmap v6774: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:00:54.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:00:54 smithi138 ceph-mon[36954]: pgmap v6774: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:00:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:00:54 smithi100 ceph-mon[32599]: pgmap v6774: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:00:56.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:00:56 smithi167 ceph-mon[37140]: pgmap v6775: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:00:56.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:00:56 smithi138 ceph-mon[36954]: pgmap v6775: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:00:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:00:56 smithi100 ceph-mon[32599]: pgmap v6775: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:00:58.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:00:58 smithi167 ceph-mon[37140]: pgmap v6776: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:00:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:00:58 smithi138 ceph-mon[36954]: pgmap v6776: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:00:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:00:58 smithi100 ceph-mon[32599]: pgmap v6776: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:01:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:01:00 smithi167 ceph-mon[37140]: pgmap v6777: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:01:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:01:00 smithi138 ceph-mon[36954]: pgmap v6777: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:01:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:01:00 smithi100 ceph-mon[32599]: pgmap v6777: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:01:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:01:02 smithi167 ceph-mon[37140]: pgmap v6778: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:01:02.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:01:02 smithi138 ceph-mon[36954]: pgmap v6778: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:01:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:01:02 smithi100 ceph-mon[32599]: pgmap v6778: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:01:04.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:01:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:01:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:01:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:01:04.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:01:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:01:05.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:01:05 smithi138 ceph-mon[36954]: pgmap v6779: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:01:05.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:01:05 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:01:05.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:01:05 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:01:05.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:01:05 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:01:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:01:05 smithi100 ceph-mon[32599]: pgmap v6779: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:01:05.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:01:05 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:01:05.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:01:05 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:01:05.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:01:05 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:01:05.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:01:05 smithi167 ceph-mon[37140]: pgmap v6779: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:01:05.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:01:05 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:01:05.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:01:05 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:01:05.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:01:05 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:01:06.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:01:06 smithi167 ceph-mon[37140]: pgmap v6780: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:01:06.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:01:06 smithi138 ceph-mon[36954]: pgmap v6780: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:01:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:01:06 smithi100 ceph-mon[32599]: pgmap v6780: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:01:08.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:01:08.575+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:01:08.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:01:08.576+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:01:08.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:01:08.576+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:01:08.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:01:08.576+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:01:08.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:01:08.576+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:01:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:01:08 smithi138 ceph-mon[36954]: osdmap e470: 12 total, 12 up, 12 in 2024-07-26T17:01:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:01:08 smithi138 ceph-mon[36954]: pgmap v6782: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:01:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:01:08 smithi100 ceph-mon[32599]: osdmap e470: 12 total, 12 up, 12 in 2024-07-26T17:01:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:01:08 smithi100 ceph-mon[32599]: pgmap v6782: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:01:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:01:08 smithi167 ceph-mon[37140]: osdmap e470: 12 total, 12 up, 12 in 2024-07-26T17:01:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:01:08 smithi167 ceph-mon[37140]: pgmap v6782: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:01:10.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:01:10 smithi167 ceph-mon[37140]: pgmap v6783: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:01:10.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:01:10 smithi138 ceph-mon[36954]: pgmap v6783: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:01:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:01:10 smithi100 ceph-mon[32599]: pgmap v6783: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:01:12.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:01:12 smithi167 ceph-mon[37140]: pgmap v6784: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:01:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:01:12 smithi138 ceph-mon[36954]: pgmap v6784: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:01:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:01:12 smithi100 ceph-mon[32599]: pgmap v6784: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:01:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:01:14 smithi167 ceph-mon[37140]: pgmap v6785: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:01:14.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:01:14 smithi138 ceph-mon[36954]: pgmap v6785: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:01:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:01:14 smithi100 ceph-mon[32599]: pgmap v6785: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:01:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:01:16 smithi167 ceph-mon[37140]: pgmap v6786: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:01:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:01:16 smithi138 ceph-mon[36954]: pgmap v6786: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:01:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:01:16 smithi100 ceph-mon[32599]: pgmap v6786: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:01:18.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:01:17 smithi138 ceph-mon[36954]: osdmap e471: 12 total, 12 up, 12 in 2024-07-26T17:01:18.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:01:17 smithi100 ceph-mon[32599]: osdmap e471: 12 total, 12 up, 12 in 2024-07-26T17:01:18.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:01:17 smithi167 ceph-mon[37140]: osdmap e471: 12 total, 12 up, 12 in 2024-07-26T17:01:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:01:18 smithi138 ceph-mon[36954]: pgmap v6788: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:01:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:01:18 smithi100 ceph-mon[32599]: pgmap v6788: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:01:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:01:18 smithi167 ceph-mon[37140]: pgmap v6788: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:01:20.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:01:20 smithi167 ceph-mon[37140]: pgmap v6789: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:01:20.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:01:20 smithi138 ceph-mon[36954]: pgmap v6789: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:01:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:01:20 smithi100 ceph-mon[32599]: pgmap v6789: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:01:22.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:01:22 smithi167 ceph-mon[37140]: pgmap v6790: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:01:22.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:01:22 smithi100 ceph-mon[32599]: pgmap v6790: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:01:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:01:22 smithi138 ceph-mon[36954]: pgmap v6790: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:01:23.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:01:23.591+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:01:23.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:01:23.592+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:01:23.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:01:23.592+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:01:23.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:01:23.592+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:01:23.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:01:23.592+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:01:24.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:01:24 smithi167 ceph-mon[37140]: pgmap v6791: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:01:24.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:01:24 smithi138 ceph-mon[36954]: pgmap v6791: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:01:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:01:24 smithi100 ceph-mon[32599]: pgmap v6791: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:01:26.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:01:26 smithi167 ceph-mon[37140]: pgmap v6792: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:01:26.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:01:26 smithi138 ceph-mon[36954]: pgmap v6792: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:01:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:01:26 smithi100 ceph-mon[32599]: pgmap v6792: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:01:28.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:01:28 smithi167 ceph-mon[37140]: pgmap v6793: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T17:01:28.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:01:28 smithi138 ceph-mon[36954]: pgmap v6793: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T17:01:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:01:28 smithi100 ceph-mon[32599]: pgmap v6793: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T17:01:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:01:30 smithi167 ceph-mon[37140]: pgmap v6794: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:01:30.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:01:30 smithi138 ceph-mon[36954]: pgmap v6794: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:01:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:01:30 smithi100 ceph-mon[32599]: pgmap v6794: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:01:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:01:32 smithi167 ceph-mon[37140]: pgmap v6795: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:01:32.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:01:32 smithi138 ceph-mon[36954]: pgmap v6795: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:01:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:01:32 smithi100 ceph-mon[32599]: pgmap v6795: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:01:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:01:34 smithi167 ceph-mon[37140]: pgmap v6796: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:01:34.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:01:34 smithi138 ceph-mon[36954]: pgmap v6796: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:01:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:01:34 smithi100 ceph-mon[32599]: pgmap v6796: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:01:36.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:01:36 smithi167 ceph-mon[37140]: pgmap v6797: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:01:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:01:36 smithi138 ceph-mon[36954]: pgmap v6797: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:01:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:01:36 smithi100 ceph-mon[32599]: pgmap v6797: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:01:38.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:01:38.595+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:01:38.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:01:38.595+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:01:38.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:01:38.595+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:01:38.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:01:38.595+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:01:38.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:01:38.595+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:01:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:01:38 smithi167 ceph-mon[37140]: pgmap v6798: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:01:38.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:01:38 smithi138 ceph-mon[36954]: pgmap v6798: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:01:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:01:38 smithi100 ceph-mon[32599]: pgmap v6798: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:01:40.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:01:40 smithi167 ceph-mon[37140]: pgmap v6799: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:01:40.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:01:40 smithi138 ceph-mon[36954]: pgmap v6799: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:01:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:01:40 smithi100 ceph-mon[32599]: pgmap v6799: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:01:42.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:01:42 smithi138 ceph-mon[36954]: pgmap v6800: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:01:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:01:42 smithi100 ceph-mon[32599]: pgmap v6800: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:01:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:01:42 smithi167 ceph-mon[37140]: pgmap v6800: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:01:44.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:01:44 smithi138 ceph-mon[36954]: pgmap v6801: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:01:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:01:44 smithi100 ceph-mon[32599]: pgmap v6801: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:01:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:01:44 smithi167 ceph-mon[37140]: pgmap v6801: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:01:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:01:46 smithi167 ceph-mon[37140]: pgmap v6802: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:01:46.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:01:46 smithi138 ceph-mon[36954]: pgmap v6802: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:01:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:01:46 smithi100 ceph-mon[32599]: pgmap v6802: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:01:48.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:01:48 smithi167 ceph-mon[37140]: pgmap v6803: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:01:48.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:01:48 smithi138 ceph-mon[36954]: pgmap v6803: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:01:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:01:48 smithi100 ceph-mon[32599]: pgmap v6803: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:01:50.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:01:50 smithi138 ceph-mon[36954]: pgmap v6804: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:01:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:01:50 smithi100 ceph-mon[32599]: pgmap v6804: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:01:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:01:50 smithi167 ceph-mon[37140]: pgmap v6804: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:01:52.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:01:52 smithi100 ceph-mon[32599]: pgmap v6805: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:01:52.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:01:52 smithi167 ceph-mon[37140]: pgmap v6805: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:01:52.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:01:52 smithi138 ceph-mon[36954]: pgmap v6805: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:01:53.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:01:53.611+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:01:53.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:01:53.611+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:01:53.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:01:53.611+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:01:53.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:01:53.611+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:01:53.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:01:53.611+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:01:54.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:01:54 smithi138 ceph-mon[36954]: pgmap v6806: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:01:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:01:54 smithi100 ceph-mon[32599]: pgmap v6806: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:01:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:01:54 smithi167 ceph-mon[37140]: pgmap v6806: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:01:56.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:01:56 smithi138 ceph-mon[36954]: pgmap v6807: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:01:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:01:56 smithi100 ceph-mon[32599]: pgmap v6807: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:01:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:01:56 smithi167 ceph-mon[37140]: pgmap v6807: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:01:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:01:58 smithi138 ceph-mon[36954]: pgmap v6808: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:01:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:01:58 smithi100 ceph-mon[32599]: pgmap v6808: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:01:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:01:58 smithi167 ceph-mon[37140]: pgmap v6808: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:02:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:00 smithi138 ceph-mon[36954]: pgmap v6809: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:02:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:00 smithi100 ceph-mon[32599]: pgmap v6809: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:02:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:00 smithi167 ceph-mon[37140]: pgmap v6809: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:02:02.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:02 smithi138 ceph-mon[36954]: pgmap v6810: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:02:02.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:02 smithi100 ceph-mon[32599]: pgmap v6810: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:02:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:02 smithi167 ceph-mon[37140]: pgmap v6810: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:02:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:04 smithi138 ceph-mon[36954]: pgmap v6811: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:02:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:04 smithi100 ceph-mon[32599]: pgmap v6811: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:02:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:04 smithi167 ceph-mon[37140]: pgmap v6811: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:02:05.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:05 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:02:05.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:05 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:02:06.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:05 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:02:07.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:02:07.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:02:07.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:02:07.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:02:07.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:02:07.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:02:07.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:07 smithi138 ceph-mon[36954]: pgmap v6812: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:02:07.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:02:07.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:02:07.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:02:07.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:02:07.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:02:07.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:02:07.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:02:07.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:02:07.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:02:07.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:07 smithi100 ceph-mon[32599]: pgmap v6812: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:02:07.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:02:07.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:02:07.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:02:07.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:02:07.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:02:07.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:02:07.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:02:07.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:02:07.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:02:07.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:07 smithi167 ceph-mon[37140]: pgmap v6812: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:02:07.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:02:07.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:02:07.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:02:08.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:02:08.627+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:02:08.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:02:08.628+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:02:08.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:02:08.628+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:02:08.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:02:08.628+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:02:08.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:02:08.628+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:02:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:09 smithi138 ceph-mon[36954]: osdmap e472: 12 total, 12 up, 12 in 2024-07-26T17:02:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:09 smithi138 ceph-mon[36954]: pgmap v6814: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:02:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:09 smithi100 ceph-mon[32599]: osdmap e472: 12 total, 12 up, 12 in 2024-07-26T17:02:09.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:09 smithi100 ceph-mon[32599]: pgmap v6814: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:02:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:09 smithi167 ceph-mon[37140]: osdmap e472: 12 total, 12 up, 12 in 2024-07-26T17:02:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:09 smithi167 ceph-mon[37140]: pgmap v6814: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:02:10.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:10 smithi138 ceph-mon[36954]: pgmap v6815: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:02:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:10 smithi100 ceph-mon[32599]: pgmap v6815: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:02:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:10 smithi167 ceph-mon[37140]: pgmap v6815: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:02:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:12 smithi138 ceph-mon[36954]: pgmap v6816: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:02:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:12 smithi100 ceph-mon[32599]: pgmap v6816: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:02:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:12 smithi167 ceph-mon[37140]: pgmap v6816: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:02:14.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:14 smithi138 ceph-mon[36954]: pgmap v6817: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:02:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:14 smithi100 ceph-mon[32599]: pgmap v6817: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:02:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:14 smithi167 ceph-mon[37140]: pgmap v6817: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:02:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:16 smithi138 ceph-mon[36954]: pgmap v6818: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:02:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:16 smithi100 ceph-mon[32599]: pgmap v6818: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:02:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:16 smithi167 ceph-mon[37140]: pgmap v6818: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:02:18.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:17 smithi138 ceph-mon[36954]: osdmap e473: 12 total, 12 up, 12 in 2024-07-26T17:02:18.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:17 smithi100 ceph-mon[32599]: osdmap e473: 12 total, 12 up, 12 in 2024-07-26T17:02:18.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:17 smithi167 ceph-mon[37140]: osdmap e473: 12 total, 12 up, 12 in 2024-07-26T17:02:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:18 smithi138 ceph-mon[36954]: pgmap v6820: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:02:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:18 smithi100 ceph-mon[32599]: pgmap v6820: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:02:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:18 smithi167 ceph-mon[37140]: pgmap v6820: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:02:20.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:20 smithi138 ceph-mon[36954]: pgmap v6821: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:02:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:20 smithi100 ceph-mon[32599]: pgmap v6821: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:02:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:20 smithi167 ceph-mon[37140]: pgmap v6821: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:02:22.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:22 smithi100 ceph-mon[32599]: pgmap v6822: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:02:22.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:22 smithi167 ceph-mon[37140]: pgmap v6822: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:02:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:22 smithi138 ceph-mon[36954]: pgmap v6822: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:02:23.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:02:23.643+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:02:23.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:02:23.643+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:02:23.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:02:23.643+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:02:23.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:02:23.643+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:02:23.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:02:23.643+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:02:24.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:24 smithi138 ceph-mon[36954]: pgmap v6823: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:02:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:24 smithi100 ceph-mon[32599]: pgmap v6823: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:02:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:24 smithi167 ceph-mon[37140]: pgmap v6823: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:02:26.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:26 smithi138 ceph-mon[36954]: pgmap v6824: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:02:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:26 smithi100 ceph-mon[32599]: pgmap v6824: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:02:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:26 smithi167 ceph-mon[37140]: pgmap v6824: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:02:28.891 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:28 smithi138 ceph-mon[36954]: pgmap v6825: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T17:02:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:28 smithi100 ceph-mon[32599]: pgmap v6825: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T17:02:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:28 smithi167 ceph-mon[37140]: pgmap v6825: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T17:02:30.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:30 smithi138 ceph-mon[36954]: pgmap v6826: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:02:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:30 smithi100 ceph-mon[32599]: pgmap v6826: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:02:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:30 smithi167 ceph-mon[37140]: pgmap v6826: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:02:32.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:32 smithi138 ceph-mon[36954]: pgmap v6827: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:02:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:32 smithi100 ceph-mon[32599]: pgmap v6827: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:02:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:32 smithi167 ceph-mon[37140]: pgmap v6827: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:02:34.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:34 smithi138 ceph-mon[36954]: pgmap v6828: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:02:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:34 smithi100 ceph-mon[32599]: pgmap v6828: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:02:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:34 smithi167 ceph-mon[37140]: pgmap v6828: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:02:36.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:36 smithi138 ceph-mon[36954]: pgmap v6829: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:02:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:36 smithi100 ceph-mon[32599]: pgmap v6829: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:02:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:36 smithi167 ceph-mon[37140]: pgmap v6829: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:02:38.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:02:38.659+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:02:38.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:02:38.659+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:02:38.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:02:38.659+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:02:38.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:02:38.659+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:02:38.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:02:38.660+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:02:38.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:38 smithi138 ceph-mon[36954]: pgmap v6830: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:02:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:38 smithi100 ceph-mon[32599]: pgmap v6830: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:02:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:38 smithi167 ceph-mon[37140]: pgmap v6830: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:02:40.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:40 smithi138 ceph-mon[36954]: pgmap v6831: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:02:40.999 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:40 smithi100 ceph-mon[32599]: pgmap v6831: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:02:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:40 smithi167 ceph-mon[37140]: pgmap v6831: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:02:42.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:42 smithi138 ceph-mon[36954]: pgmap v6832: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:02:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:42 smithi100 ceph-mon[32599]: pgmap v6832: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:02:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:42 smithi167 ceph-mon[37140]: pgmap v6832: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:02:44.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:44 smithi138 ceph-mon[36954]: pgmap v6833: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:02:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:44 smithi100 ceph-mon[32599]: pgmap v6833: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:02:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:44 smithi167 ceph-mon[37140]: pgmap v6833: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:02:46.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:46 smithi138 ceph-mon[36954]: pgmap v6834: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:02:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:46 smithi100 ceph-mon[32599]: pgmap v6834: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:02:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:46 smithi167 ceph-mon[37140]: pgmap v6834: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:02:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:48 smithi138 ceph-mon[36954]: pgmap v6835: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:02:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:48 smithi100 ceph-mon[32599]: pgmap v6835: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:02:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:48 smithi167 ceph-mon[37140]: pgmap v6835: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:02:50.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:50 smithi138 ceph-mon[36954]: pgmap v6836: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:02:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:50 smithi100 ceph-mon[32599]: pgmap v6836: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:02:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:50 smithi167 ceph-mon[37140]: pgmap v6836: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:02:52.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:52 smithi100 ceph-mon[32599]: pgmap v6837: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:02:52.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:52 smithi167 ceph-mon[37140]: pgmap v6837: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:02:52.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:52 smithi138 ceph-mon[36954]: pgmap v6837: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:02:53.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:02:53.672+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:02:53.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:02:53.672+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:02:53.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:02:53.672+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:02:53.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:02:53.672+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:02:53.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:02:53.672+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:02:54.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:54 smithi138 ceph-mon[36954]: pgmap v6838: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:02:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:54 smithi100 ceph-mon[32599]: pgmap v6838: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:02:55.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:54 smithi167 ceph-mon[37140]: pgmap v6838: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:02:56.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:56 smithi138 ceph-mon[36954]: pgmap v6839: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:02:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:56 smithi100 ceph-mon[32599]: pgmap v6839: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:02:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:56 smithi167 ceph-mon[37140]: pgmap v6839: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:02:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:02:58 smithi138 ceph-mon[36954]: pgmap v6840: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:02:58.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:02:58 smithi100 ceph-mon[32599]: pgmap v6840: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:02:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:02:58 smithi167 ceph-mon[37140]: pgmap v6840: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:03:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:00 smithi138 ceph-mon[36954]: pgmap v6841: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:03:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:00 smithi100 ceph-mon[32599]: pgmap v6841: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:03:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:00 smithi167 ceph-mon[37140]: pgmap v6841: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:03:02.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:02 smithi138 ceph-mon[36954]: pgmap v6842: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:03:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:02 smithi100 ceph-mon[32599]: pgmap v6842: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:03:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:02 smithi167 ceph-mon[37140]: pgmap v6842: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:03:03.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:03:03.039+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 msgr2=0x7fe5940b5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-07-26T17:03:03.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:03:03.039+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 msgr2=0x7fe5940b5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-07-26T17:03:03.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:03:03.039+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 0x7fe5940b5b30 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fe590004ee0 tx=0x7fe5900ac040 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:03:03.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:03:03.039+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 0x7fe5940b5b30 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fe590004ee0 tx=0x7fe5900ac040 comp rx=0 tx=0).stop 2024-07-26T17:03:03.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:03:03.039+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 msgr2=0x7fe5940b5b30 unknown :-1 s=STATE_CLOSED l=1).tick see no progress in more than 10000000 us during connecting to v2:172.21.15.100:6800/3937009534, fault. 2024-07-26T17:03:03.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:03:03.039+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 msgr2=0x7fe5940b5b30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-07-26T17:03:03.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:03:03.039+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 0x7fe5940b5b30 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T17:03:03.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:03:03.039+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T17:03:03.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:03:03.039+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:03:03.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:03:03.040+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe59000be60 tx=0x7fe59008a040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T17:03:04.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:03:04.673+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T17:03:04.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:03:04.673+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T17:03:04.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:03:04.673+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 93 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T17:03:04.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:03:04.674+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 93 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T17:03:04.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:04 smithi138 ceph-mon[36954]: pgmap v6843: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:03:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:04 smithi100 ceph-mon[32599]: pgmap v6843: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:03:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:04 smithi167 ceph-mon[37140]: pgmap v6843: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:03:06.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:06 smithi138 ceph-mon[36954]: pgmap v6844: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:03:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:06 smithi100 ceph-mon[32599]: pgmap v6844: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:03:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:06 smithi167 ceph-mon[37140]: pgmap v6844: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:03:07.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:03:07.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:03:07.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:03:07.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:03:07.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:03:07.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:03:07.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:03:07.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:03:07.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:03:07.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:03:07.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:03:07.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:03:07.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:03:07.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:03:07.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:03:07.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:03:07.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:03:07.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:03:07.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:03:07.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:03:08.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:03:08.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:03:08.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:03:08.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:03:08.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:03:08.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:03:08.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:03:08.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:03:08.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:03:08.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:03:08.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:03:08.677+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:03:08.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:03:08.678+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:03:08.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:03:08.678+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:03:08.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:03:08.678+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:03:08.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:03:08.678+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:03:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:08 smithi100 ceph-mon[32599]: osdmap e474: 12 total, 12 up, 12 in 2024-07-26T17:03:09.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:08 smithi100 ceph-mon[32599]: pgmap v6846: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:03:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:08 smithi167 ceph-mon[37140]: osdmap e474: 12 total, 12 up, 12 in 2024-07-26T17:03:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:08 smithi167 ceph-mon[37140]: pgmap v6846: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:03:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:08 smithi138 ceph-mon[36954]: osdmap e474: 12 total, 12 up, 12 in 2024-07-26T17:03:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:08 smithi138 ceph-mon[36954]: pgmap v6846: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:03:10.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:10 smithi138 ceph-mon[36954]: pgmap v6847: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:03:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:10 smithi100 ceph-mon[32599]: pgmap v6847: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:03:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:10 smithi167 ceph-mon[37140]: pgmap v6847: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:03:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:12 smithi138 ceph-mon[36954]: pgmap v6848: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:03:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:12 smithi100 ceph-mon[32599]: pgmap v6848: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:03:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:12 smithi167 ceph-mon[37140]: pgmap v6848: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:03:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:14 smithi138 ceph-mon[36954]: pgmap v6849: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:03:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:14 smithi100 ceph-mon[32599]: pgmap v6849: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:03:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:14 smithi167 ceph-mon[37140]: pgmap v6849: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:03:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:16 smithi138 ceph-mon[36954]: pgmap v6850: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:03:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:16 smithi100 ceph-mon[32599]: pgmap v6850: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:03:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:16 smithi167 ceph-mon[37140]: pgmap v6850: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:03:18.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:17 smithi138 ceph-mon[36954]: osdmap e475: 12 total, 12 up, 12 in 2024-07-26T17:03:18.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:17 smithi100 ceph-mon[32599]: osdmap e475: 12 total, 12 up, 12 in 2024-07-26T17:03:18.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:17 smithi167 ceph-mon[37140]: osdmap e475: 12 total, 12 up, 12 in 2024-07-26T17:03:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:18 smithi138 ceph-mon[36954]: pgmap v6852: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:03:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:18 smithi100 ceph-mon[32599]: pgmap v6852: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:03:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:18 smithi167 ceph-mon[37140]: pgmap v6852: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:03:20.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:20 smithi138 ceph-mon[36954]: pgmap v6853: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:03:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:20 smithi100 ceph-mon[32599]: pgmap v6853: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:03:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:20 smithi167 ceph-mon[37140]: pgmap v6853: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:03:22.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:22 smithi100 ceph-mon[32599]: pgmap v6854: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:03:22.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:22 smithi167 ceph-mon[37140]: pgmap v6854: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:03:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:22 smithi138 ceph-mon[36954]: pgmap v6854: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:03:23.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:03:23.693+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:03:23.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:03:23.694+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:03:23.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:03:23.694+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:03:23.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:03:23.694+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:03:23.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:03:23.694+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:03:24.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:24 smithi138 ceph-mon[36954]: pgmap v6855: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:03:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:24 smithi100 ceph-mon[32599]: pgmap v6855: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:03:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:24 smithi167 ceph-mon[37140]: pgmap v6855: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:03:26.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:26 smithi138 ceph-mon[36954]: pgmap v6856: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:03:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:26 smithi100 ceph-mon[32599]: pgmap v6856: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:03:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:26 smithi167 ceph-mon[37140]: pgmap v6856: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:03:28.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:28 smithi138 ceph-mon[36954]: pgmap v6857: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T17:03:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:28 smithi100 ceph-mon[32599]: pgmap v6857: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T17:03:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:28 smithi167 ceph-mon[37140]: pgmap v6857: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T17:03:30.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:30 smithi138 ceph-mon[36954]: pgmap v6858: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:03:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:30 smithi100 ceph-mon[32599]: pgmap v6858: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:03:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:30 smithi167 ceph-mon[37140]: pgmap v6858: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:03:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:32 smithi138 ceph-mon[36954]: pgmap v6859: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:03:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:32 smithi100 ceph-mon[32599]: pgmap v6859: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:03:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:32 smithi167 ceph-mon[37140]: pgmap v6859: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:03:34.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:34 smithi138 ceph-mon[36954]: pgmap v6860: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:03:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:34 smithi100 ceph-mon[32599]: pgmap v6860: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:03:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:34 smithi167 ceph-mon[37140]: pgmap v6860: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:03:36.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:36 smithi138 ceph-mon[36954]: pgmap v6861: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:03:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:36 smithi100 ceph-mon[32599]: pgmap v6861: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:03:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:36 smithi167 ceph-mon[37140]: pgmap v6861: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:03:38.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:03:38.694+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:03:38.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:03:38.695+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:03:38.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:03:38.695+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:03:38.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:03:38.695+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:03:38.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:03:38.695+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:03:38.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:38 smithi138 ceph-mon[36954]: pgmap v6862: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:03:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:38 smithi100 ceph-mon[32599]: pgmap v6862: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:03:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:38 smithi167 ceph-mon[37140]: pgmap v6862: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:03:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:40 smithi138 ceph-mon[36954]: pgmap v6863: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:03:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:40 smithi100 ceph-mon[32599]: pgmap v6863: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:03:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:40 smithi167 ceph-mon[37140]: pgmap v6863: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:03:42.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:42 smithi138 ceph-mon[36954]: pgmap v6864: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:03:42.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:42 smithi100 ceph-mon[32599]: pgmap v6864: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:03:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:42 smithi167 ceph-mon[37140]: pgmap v6864: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:03:44.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:44 smithi138 ceph-mon[36954]: pgmap v6865: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:03:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:44 smithi100 ceph-mon[32599]: pgmap v6865: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:03:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:44 smithi167 ceph-mon[37140]: pgmap v6865: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:03:46.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:46 smithi138 ceph-mon[36954]: pgmap v6866: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:03:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:46 smithi100 ceph-mon[32599]: pgmap v6866: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:03:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:46 smithi167 ceph-mon[37140]: pgmap v6866: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:03:48.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:48 smithi138 ceph-mon[36954]: pgmap v6867: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:03:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:48 smithi100 ceph-mon[32599]: pgmap v6867: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:03:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:48 smithi167 ceph-mon[37140]: pgmap v6867: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:03:50.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:50 smithi138 ceph-mon[36954]: pgmap v6868: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:03:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:50 smithi100 ceph-mon[32599]: pgmap v6868: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:03:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:50 smithi167 ceph-mon[37140]: pgmap v6868: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:03:52.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:52 smithi100 ceph-mon[32599]: pgmap v6869: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:03:52.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:52 smithi167 ceph-mon[37140]: pgmap v6869: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:03:52.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:52 smithi138 ceph-mon[36954]: pgmap v6869: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:03:53.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:03:53.712+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:03:53.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:03:53.712+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:03:53.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:03:53.712+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:03:53.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:03:53.712+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:03:53.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:03:53.712+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:03:54.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:54 smithi138 ceph-mon[36954]: pgmap v6870: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:03:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:54 smithi100 ceph-mon[32599]: pgmap v6870: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:03:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:54 smithi167 ceph-mon[37140]: pgmap v6870: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:03:56.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:56 smithi138 ceph-mon[36954]: pgmap v6871: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:03:56.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:56 smithi100 ceph-mon[32599]: pgmap v6871: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:03:57.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:56 smithi167 ceph-mon[37140]: pgmap v6871: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:03:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:03:58 smithi138 ceph-mon[36954]: pgmap v6872: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:03:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:03:58 smithi100 ceph-mon[32599]: pgmap v6872: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:03:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:03:58 smithi167 ceph-mon[37140]: pgmap v6872: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:04:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:00 smithi138 ceph-mon[36954]: pgmap v6873: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:04:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:00 smithi100 ceph-mon[32599]: pgmap v6873: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:04:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:00 smithi167 ceph-mon[37140]: pgmap v6873: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:04:02.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:02 smithi138 ceph-mon[36954]: pgmap v6874: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:04:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:02 smithi100 ceph-mon[32599]: pgmap v6874: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:04:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:02 smithi167 ceph-mon[37140]: pgmap v6874: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:04:04.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:04 smithi138 ceph-mon[36954]: pgmap v6875: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:04:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:04 smithi100 ceph-mon[32599]: pgmap v6875: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:04:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:04 smithi167 ceph-mon[37140]: pgmap v6875: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:04:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:06 smithi138 ceph-mon[36954]: pgmap v6876: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:04:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:06 smithi100 ceph-mon[32599]: pgmap v6876: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:04:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:06 smithi167 ceph-mon[37140]: pgmap v6876: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:04:07.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:04:07.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:04:08.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:04:08.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:04:08.728+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:04:08.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:04:08.728+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:04:08.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:04:08.728+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:04:08.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:04:08.728+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:04:08.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:04:08.728+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:04:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:08 smithi138 ceph-mon[36954]: osdmap e476: 12 total, 12 up, 12 in 2024-07-26T17:04:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:04:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:04:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:04:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:04:08.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:04:08.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:04:08.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:04:08.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:04:08.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:04:08.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:08 smithi138 ceph-mon[36954]: pgmap v6878: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:04:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:08 smithi100 ceph-mon[32599]: osdmap e476: 12 total, 12 up, 12 in 2024-07-26T17:04:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:04:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:04:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:04:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:04:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:04:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:04:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:04:08.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:04:08.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:04:08.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:08 smithi100 ceph-mon[32599]: pgmap v6878: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:04:09.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:08 smithi167 ceph-mon[37140]: osdmap e476: 12 total, 12 up, 12 in 2024-07-26T17:04:09.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:04:09.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:04:09.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:04:09.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:04:09.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:04:09.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:04:09.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:04:09.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:04:09.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:04:09.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:08 smithi167 ceph-mon[37140]: pgmap v6878: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:04:10.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:10 smithi138 ceph-mon[36954]: pgmap v6879: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:04:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:10 smithi100 ceph-mon[32599]: pgmap v6879: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:04:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:10 smithi167 ceph-mon[37140]: pgmap v6879: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:04:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:12 smithi138 ceph-mon[36954]: pgmap v6880: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:04:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:12 smithi100 ceph-mon[32599]: pgmap v6880: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:04:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:12 smithi167 ceph-mon[37140]: pgmap v6880: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:04:14.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:14 smithi138 ceph-mon[36954]: pgmap v6881: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:04:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:14 smithi100 ceph-mon[32599]: pgmap v6881: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:04:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:14 smithi167 ceph-mon[37140]: pgmap v6881: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:04:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:16 smithi138 ceph-mon[36954]: pgmap v6882: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:04:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:16 smithi100 ceph-mon[32599]: pgmap v6882: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:04:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:16 smithi167 ceph-mon[37140]: pgmap v6882: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:04:18.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:17 smithi138 ceph-mon[36954]: osdmap e477: 12 total, 12 up, 12 in 2024-07-26T17:04:18.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:17 smithi100 ceph-mon[32599]: osdmap e477: 12 total, 12 up, 12 in 2024-07-26T17:04:18.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:17 smithi167 ceph-mon[37140]: osdmap e477: 12 total, 12 up, 12 in 2024-07-26T17:04:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:18 smithi138 ceph-mon[36954]: pgmap v6884: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:04:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:18 smithi100 ceph-mon[32599]: pgmap v6884: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:04:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:18 smithi167 ceph-mon[37140]: pgmap v6884: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:04:20.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:20 smithi138 ceph-mon[36954]: pgmap v6885: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T17:04:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:20 smithi100 ceph-mon[32599]: pgmap v6885: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T17:04:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:20 smithi167 ceph-mon[37140]: pgmap v6885: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T17:04:22.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:22 smithi167 ceph-mon[37140]: pgmap v6886: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T17:04:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:22 smithi138 ceph-mon[36954]: pgmap v6886: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T17:04:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:22 smithi100 ceph-mon[32599]: pgmap v6886: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T17:04:23.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:04:23.744+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:04:23.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:04:23.744+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:04:23.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:04:23.744+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:04:23.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:04:23.744+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:04:23.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:04:23.744+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:04:24.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:24 smithi138 ceph-mon[36954]: pgmap v6887: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:04:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:24 smithi100 ceph-mon[32599]: pgmap v6887: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:04:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:24 smithi167 ceph-mon[37140]: pgmap v6887: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:04:26.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:26 smithi138 ceph-mon[36954]: pgmap v6888: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:04:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:26 smithi100 ceph-mon[32599]: pgmap v6888: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:04:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:26 smithi167 ceph-mon[37140]: pgmap v6888: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:04:28.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:28 smithi138 ceph-mon[36954]: pgmap v6889: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T17:04:28.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:28 smithi100 ceph-mon[32599]: pgmap v6889: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T17:04:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:28 smithi167 ceph-mon[37140]: pgmap v6889: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T17:04:30.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:30 smithi138 ceph-mon[36954]: pgmap v6890: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:04:30.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:30 smithi100 ceph-mon[32599]: pgmap v6890: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:04:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:30 smithi167 ceph-mon[37140]: pgmap v6890: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:04:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:32 smithi100 ceph-mon[32599]: pgmap v6891: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:04:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:32 smithi167 ceph-mon[37140]: pgmap v6891: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:04:33.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:32 smithi138 ceph-mon[36954]: pgmap v6891: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:04:34.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:34 smithi138 ceph-mon[36954]: pgmap v6892: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:04:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:34 smithi100 ceph-mon[32599]: pgmap v6892: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:04:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:34 smithi167 ceph-mon[37140]: pgmap v6892: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:04:36.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:36 smithi138 ceph-mon[36954]: pgmap v6893: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:04:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:36 smithi100 ceph-mon[32599]: pgmap v6893: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:04:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:36 smithi167 ceph-mon[37140]: pgmap v6893: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:04:38.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:04:38.751+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:04:38.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:04:38.752+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:04:38.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:04:38.752+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:04:38.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:04:38.752+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:04:38.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:04:38.752+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:04:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:38 smithi138 ceph-mon[36954]: pgmap v6894: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:04:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:38 smithi100 ceph-mon[32599]: pgmap v6894: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:04:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:38 smithi167 ceph-mon[37140]: pgmap v6894: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:04:41.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:41 smithi138 ceph-mon[36954]: pgmap v6895: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:04:41.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:41 smithi100 ceph-mon[32599]: pgmap v6895: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:04:42.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:41 smithi167 ceph-mon[37140]: pgmap v6895: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:04:43.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:43 smithi138 ceph-mon[36954]: pgmap v6896: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:04:43.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:43 smithi100 ceph-mon[32599]: pgmap v6896: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:04:44.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:43 smithi167 ceph-mon[37140]: pgmap v6896: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:04:45.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:45 smithi138 ceph-mon[36954]: pgmap v6897: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:04:45.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:45 smithi100 ceph-mon[32599]: pgmap v6897: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:04:46.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:45 smithi167 ceph-mon[37140]: pgmap v6897: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:04:47.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:47 smithi138 ceph-mon[36954]: pgmap v6898: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:04:47.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:47 smithi100 ceph-mon[32599]: pgmap v6898: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:04:48.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:47 smithi167 ceph-mon[37140]: pgmap v6898: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:04:49.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:49 smithi138 ceph-mon[36954]: pgmap v6899: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:04:49.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:49 smithi100 ceph-mon[32599]: pgmap v6899: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:04:50.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:49 smithi167 ceph-mon[37140]: pgmap v6899: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:04:51.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:51 smithi138 ceph-mon[36954]: pgmap v6900: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:04:51.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:51 smithi100 ceph-mon[32599]: pgmap v6900: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:04:52.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:51 smithi167 ceph-mon[37140]: pgmap v6900: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:04:53.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:04:53.758+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:04:53.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:04:53.759+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:04:53.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:04:53.759+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:04:53.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:04:53.759+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:04:53.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:04:53.759+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:04:53.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:53 smithi167 ceph-mon[37140]: pgmap v6901: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:04:53.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:53 smithi138 ceph-mon[36954]: pgmap v6901: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:04:53.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:53 smithi100 ceph-mon[32599]: pgmap v6901: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:04:55.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:55 smithi138 ceph-mon[36954]: pgmap v6902: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:04:55.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:55 smithi100 ceph-mon[32599]: pgmap v6902: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:04:56.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:55 smithi167 ceph-mon[37140]: pgmap v6902: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:04:57.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:57 smithi138 ceph-mon[36954]: pgmap v6903: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:04:57.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:57 smithi100 ceph-mon[32599]: pgmap v6903: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:04:58.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:57 smithi167 ceph-mon[37140]: pgmap v6903: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:04:59.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:04:59 smithi138 ceph-mon[36954]: pgmap v6904: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:04:59.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:04:59 smithi100 ceph-mon[32599]: pgmap v6904: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:05:00.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:04:59 smithi167 ceph-mon[37140]: pgmap v6904: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:05:01.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:05:01 smithi138 ceph-mon[36954]: pgmap v6905: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:05:01.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:05:01 smithi100 ceph-mon[32599]: pgmap v6905: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:05:02.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:05:01 smithi167 ceph-mon[37140]: pgmap v6905: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:05:03.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:05:03 smithi138 ceph-mon[36954]: pgmap v6906: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:05:03.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:05:03 smithi100 ceph-mon[32599]: pgmap v6906: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:05:04.028 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:05:03 smithi167 ceph-mon[37140]: pgmap v6906: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:05:05.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:05:05 smithi138 ceph-mon[36954]: pgmap v6907: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:05:05.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:05:05 smithi100 ceph-mon[32599]: pgmap v6907: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:05:06.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:05:05 smithi167 ceph-mon[37140]: pgmap v6907: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:05:07.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:05:07 smithi138 ceph-mon[36954]: pgmap v6908: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:05:07.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:05:07 smithi100 ceph-mon[32599]: pgmap v6908: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:05:08.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:05:07 smithi167 ceph-mon[37140]: pgmap v6908: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:05:08.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:05:08.775+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:05:08.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:05:08.775+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:05:08.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:05:08.775+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:05:08.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:05:08.775+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:05:08.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:05:08.775+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:05:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:05:08 smithi138 ceph-mon[36954]: osdmap e478: 12 total, 12 up, 12 in 2024-07-26T17:05:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:05:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:05:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:05:08 smithi100 ceph-mon[32599]: osdmap e478: 12 total, 12 up, 12 in 2024-07-26T17:05:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:05:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:05:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:05:08 smithi167 ceph-mon[37140]: osdmap e478: 12 total, 12 up, 12 in 2024-07-26T17:05:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:05:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:05:09.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:05:09 smithi138 ceph-mon[36954]: pgmap v6910: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:05:09.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:05:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:05:09.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:05:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:05:09.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:05:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:05:09.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:05:09 smithi100 ceph-mon[32599]: pgmap v6910: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:05:09.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:05:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:05:09.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:05:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:05:09.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:05:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:05:10.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:05:09 smithi167 ceph-mon[37140]: pgmap v6910: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:05:10.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:05:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:05:10.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:05:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:05:10.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:05:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:05:11.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:05:11 smithi138 ceph-mon[36954]: pgmap v6911: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:05:11.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:05:11 smithi100 ceph-mon[32599]: pgmap v6911: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:05:12.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:05:11 smithi167 ceph-mon[37140]: pgmap v6911: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:05:13.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:05:13 smithi138 ceph-mon[36954]: pgmap v6912: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:05:13.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:05:13 smithi100 ceph-mon[32599]: pgmap v6912: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:05:14.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:05:13 smithi167 ceph-mon[37140]: pgmap v6912: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:05:15.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:05:15 smithi138 ceph-mon[36954]: pgmap v6913: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:05:15.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:05:15 smithi100 ceph-mon[32599]: pgmap v6913: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:05:16.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:05:15 smithi167 ceph-mon[37140]: pgmap v6913: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:05:18.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:05:17 smithi100 ceph-mon[32599]: pgmap v6914: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:05:18.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:05:17 smithi100 ceph-mon[32599]: osdmap e479: 12 total, 12 up, 12 in 2024-07-26T17:05:18.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:05:17 smithi167 ceph-mon[37140]: pgmap v6914: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:05:18.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:05:17 smithi167 ceph-mon[37140]: osdmap e479: 12 total, 12 up, 12 in 2024-07-26T17:05:18.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:05:17 smithi138 ceph-mon[36954]: pgmap v6914: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:05:18.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:05:17 smithi138 ceph-mon[36954]: osdmap e479: 12 total, 12 up, 12 in 2024-07-26T17:05:20.243 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:05:19 smithi138 ceph-mon[36954]: pgmap v6916: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:05:20.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:05:19 smithi100 ceph-mon[32599]: pgmap v6916: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:05:20.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:05:19 smithi167 ceph-mon[37140]: pgmap v6916: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:05:22.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:05:21 smithi100 ceph-mon[32599]: pgmap v6917: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:05:22.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:05:21 smithi167 ceph-mon[37140]: pgmap v6917: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:05:22.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:05:21 smithi138 ceph-mon[36954]: pgmap v6917: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:05:23.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:05:23.777+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:05:23.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:05:23.778+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:05:23.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:05:23.778+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:05:23.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:05:23.778+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:05:23.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:05:23.778+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:05:24.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:05:23 smithi100 ceph-mon[32599]: pgmap v6918: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:05:24.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:05:23 smithi167 ceph-mon[37140]: pgmap v6918: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:05:24.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:05:23 smithi138 ceph-mon[36954]: pgmap v6918: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:05:26.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:05:25 smithi100 ceph-mon[32599]: pgmap v6919: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:05:26.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:05:25 smithi167 ceph-mon[37140]: pgmap v6919: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:05:26.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:05:25 smithi138 ceph-mon[36954]: pgmap v6919: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:05:28.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:05:27 smithi100 ceph-mon[32599]: pgmap v6920: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:05:28.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:05:27 smithi167 ceph-mon[37140]: pgmap v6920: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:05:28.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:05:27 smithi138 ceph-mon[36954]: pgmap v6920: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:05:30.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:05:29 smithi100 ceph-mon[32599]: pgmap v6921: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T17:05:30.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:05:29 smithi167 ceph-mon[37140]: pgmap v6921: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T17:05:30.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:05:29 smithi138 ceph-mon[36954]: pgmap v6921: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T17:05:32.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:05:31 smithi100 ceph-mon[32599]: pgmap v6922: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:05:32.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:05:31 smithi167 ceph-mon[37140]: pgmap v6922: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:05:32.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:05:31 smithi138 ceph-mon[36954]: pgmap v6922: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:05:34.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:05:33 smithi100 ceph-mon[32599]: pgmap v6923: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:05:34.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:05:33 smithi167 ceph-mon[37140]: pgmap v6923: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:05:34.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:05:33 smithi138 ceph-mon[36954]: pgmap v6923: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:05:36.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:05:35 smithi100 ceph-mon[32599]: pgmap v6924: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:05:36.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:05:35 smithi167 ceph-mon[37140]: pgmap v6924: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:05:36.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:05:35 smithi138 ceph-mon[36954]: pgmap v6924: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:05:38.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:05:37 smithi100 ceph-mon[32599]: pgmap v6925: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:05:38.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:05:37 smithi167 ceph-mon[37140]: pgmap v6925: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:05:38.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:05:37 smithi138 ceph-mon[36954]: pgmap v6925: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:05:38.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:05:38.783+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:05:38.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:05:38.784+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:05:38.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:05:38.784+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:05:38.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:05:38.784+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:05:38.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:05:38.784+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:05:40.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:05:39 smithi100 ceph-mon[32599]: pgmap v6926: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:05:40.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:05:39 smithi167 ceph-mon[37140]: pgmap v6926: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:05:40.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:05:39 smithi138 ceph-mon[36954]: pgmap v6926: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:05:42.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:05:41 smithi167 ceph-mon[37140]: pgmap v6927: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:05:42.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:05:41 smithi138 ceph-mon[36954]: pgmap v6927: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:05:42.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:05:41 smithi100 ceph-mon[32599]: pgmap v6927: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:05:44.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:05:43 smithi167 ceph-mon[37140]: pgmap v6928: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:05:44.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:05:43 smithi138 ceph-mon[36954]: pgmap v6928: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:05:44.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:05:43 smithi100 ceph-mon[32599]: pgmap v6928: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:05:46.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:05:46 smithi167 ceph-mon[37140]: pgmap v6929: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:05:46.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:05:46 smithi138 ceph-mon[36954]: pgmap v6929: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:05:46.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:05:46 smithi100 ceph-mon[32599]: pgmap v6929: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:05:48.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:05:48 smithi167 ceph-mon[37140]: pgmap v6930: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:05:48.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:05:48 smithi138 ceph-mon[36954]: pgmap v6930: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:05:48.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:05:48 smithi100 ceph-mon[32599]: pgmap v6930: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:05:50.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:05:50 smithi167 ceph-mon[37140]: pgmap v6931: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:05:50.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:05:50 smithi138 ceph-mon[36954]: pgmap v6931: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:05:50.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:05:50 smithi100 ceph-mon[32599]: pgmap v6931: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:05:52.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:05:52 smithi167 ceph-mon[37140]: pgmap v6932: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:05:52.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:05:52 smithi138 ceph-mon[36954]: pgmap v6932: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:05:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:05:52 smithi100 ceph-mon[32599]: pgmap v6932: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:05:53.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:05:53.799+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:05:53.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:05:53.800+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:05:53.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:05:53.800+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:05:53.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:05:53.800+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:05:53.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:05:53.800+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:05:54.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:05:54 smithi100 ceph-mon[32599]: pgmap v6933: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:05:54.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:05:54 smithi167 ceph-mon[37140]: pgmap v6933: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:05:54.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:05:54 smithi138 ceph-mon[36954]: pgmap v6933: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:05:56.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:05:56 smithi167 ceph-mon[37140]: pgmap v6934: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:05:56.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:05:56 smithi138 ceph-mon[36954]: pgmap v6934: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:05:56.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:05:56 smithi100 ceph-mon[32599]: pgmap v6934: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:05:58.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:05:58 smithi138 ceph-mon[36954]: pgmap v6935: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:05:58.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:05:58 smithi100 ceph-mon[32599]: pgmap v6935: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:05:58.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:05:58 smithi167 ceph-mon[37140]: pgmap v6935: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:06:00.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:06:00 smithi138 ceph-mon[36954]: pgmap v6936: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:06:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:06:00 smithi100 ceph-mon[32599]: pgmap v6936: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:06:00.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:06:00 smithi167 ceph-mon[37140]: pgmap v6936: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:06:02.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:06:02 smithi138 ceph-mon[36954]: pgmap v6937: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:06:02.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:06:02 smithi100 ceph-mon[32599]: pgmap v6937: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:06:02.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:06:02 smithi167 ceph-mon[37140]: pgmap v6937: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:06:04.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:06:04 smithi138 ceph-mon[36954]: pgmap v6938: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:06:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:06:04 smithi100 ceph-mon[32599]: pgmap v6938: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:06:04.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:06:04 smithi167 ceph-mon[37140]: pgmap v6938: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:06:04.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:06:04.699+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T17:06:04.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:06:04.699+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T17:06:04.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:06:04.699+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 94 ==== auth_reply(proto 2 0 (0) Success) ==== 612+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T17:06:04.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:06:04.699+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T17:06:04.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:06:04.699+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 94 ==== auth_reply(proto 2 0 (0) Success) ==== 783+0+0 (secure 0 0 0) 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T17:06:04.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:06:04.699+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe5a008c020 con 0x7fe5a81087a0 2024-07-26T17:06:04.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:06:04.700+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 95 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T17:06:04.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:06:04.700+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 95 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c020 con 0x7fe5a81087a0 2024-07-26T17:06:06.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:06:06 smithi138 ceph-mon[36954]: pgmap v6939: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:06:06.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:06:06 smithi100 ceph-mon[32599]: pgmap v6939: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:06:06.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:06:06 smithi167 ceph-mon[37140]: pgmap v6939: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:06:08.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:06:08 smithi138 ceph-mon[36954]: pgmap v6940: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:06:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:06:08 smithi100 ceph-mon[32599]: pgmap v6940: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:06:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:06:08 smithi167 ceph-mon[37140]: pgmap v6940: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:06:08.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:06:08.816+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:06:08.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:06:08.816+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:06:08.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:06:08.816+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:06:08.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:06:08.816+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:06:08.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:06:08.816+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:06:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:06:09 smithi100 ceph-mon[32599]: osdmap e480: 12 total, 12 up, 12 in 2024-07-26T17:06:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:06:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:06:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:06:09 smithi138 ceph-mon[36954]: osdmap e480: 12 total, 12 up, 12 in 2024-07-26T17:06:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:06:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:06:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:06:09 smithi167 ceph-mon[37140]: osdmap e480: 12 total, 12 up, 12 in 2024-07-26T17:06:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:06:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:06:10.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:06:10 smithi138 ceph-mon[36954]: pgmap v6942: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:06:10.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:06:10 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:06:10.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:06:10 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:06:10.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:06:10 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:06:10.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:06:10 smithi100 ceph-mon[32599]: pgmap v6942: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:06:10.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:06:10 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:06:10.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:06:10 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:06:10.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:06:10 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:06:10.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:06:10 smithi167 ceph-mon[37140]: pgmap v6942: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:06:10.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:06:10 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:06:10.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:06:10 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:06:10.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:06:10 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:06:12.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:06:12 smithi138 ceph-mon[36954]: pgmap v6943: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:06:12.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:06:12 smithi100 ceph-mon[32599]: pgmap v6943: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:06:12.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:06:12 smithi167 ceph-mon[37140]: pgmap v6943: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:06:14.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:06:14 smithi138 ceph-mon[36954]: pgmap v6944: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:06:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:06:14 smithi100 ceph-mon[32599]: pgmap v6944: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:06:14.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:06:14 smithi167 ceph-mon[37140]: pgmap v6944: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:06:16.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:06:16 smithi138 ceph-mon[36954]: pgmap v6945: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:06:16.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:06:16 smithi100 ceph-mon[32599]: pgmap v6945: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:06:16.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:06:16 smithi167 ceph-mon[37140]: pgmap v6945: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:06:18.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:06:17 smithi100 ceph-mon[32599]: pgmap v6946: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:06:18.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:06:17 smithi100 ceph-mon[32599]: osdmap e481: 12 total, 12 up, 12 in 2024-07-26T17:06:18.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:06:17 smithi167 ceph-mon[37140]: pgmap v6946: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:06:18.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:06:17 smithi167 ceph-mon[37140]: osdmap e481: 12 total, 12 up, 12 in 2024-07-26T17:06:18.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:06:17 smithi138 ceph-mon[36954]: pgmap v6946: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:06:18.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:06:17 smithi138 ceph-mon[36954]: osdmap e481: 12 total, 12 up, 12 in 2024-07-26T17:06:20.243 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:06:19 smithi138 ceph-mon[36954]: pgmap v6948: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:06:20.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:06:19 smithi100 ceph-mon[32599]: pgmap v6948: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:06:20.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:06:19 smithi167 ceph-mon[37140]: pgmap v6948: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:06:22.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:06:21 smithi100 ceph-mon[32599]: pgmap v6949: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:06:22.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:06:21 smithi167 ceph-mon[37140]: pgmap v6949: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:06:22.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:06:21 smithi138 ceph-mon[36954]: pgmap v6949: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:06:23.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:06:23.819+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:06:23.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:06:23.820+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:06:23.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:06:23.820+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:06:23.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:06:23.820+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:06:23.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:06:23.820+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:06:24.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:06:23 smithi100 ceph-mon[32599]: pgmap v6950: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:06:24.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:06:23 smithi167 ceph-mon[37140]: pgmap v6950: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:06:24.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:06:23 smithi138 ceph-mon[36954]: pgmap v6950: 97 pgs: 97 active+clean; 234 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:06:26.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:06:25 smithi100 ceph-mon[32599]: pgmap v6951: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:06:26.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:06:25 smithi167 ceph-mon[37140]: pgmap v6951: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:06:26.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:06:25 smithi138 ceph-mon[36954]: pgmap v6951: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:06:28.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:06:27 smithi100 ceph-mon[32599]: pgmap v6952: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:06:28.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:06:27 smithi167 ceph-mon[37140]: pgmap v6952: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:06:28.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:06:27 smithi138 ceph-mon[36954]: pgmap v6952: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:06:30.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:06:29 smithi100 ceph-mon[32599]: pgmap v6953: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T17:06:30.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:06:29 smithi167 ceph-mon[37140]: pgmap v6953: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T17:06:30.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:06:29 smithi138 ceph-mon[36954]: pgmap v6953: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T17:06:32.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:06:31 smithi100 ceph-mon[32599]: pgmap v6954: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:06:32.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:06:31 smithi167 ceph-mon[37140]: pgmap v6954: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:06:32.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:06:31 smithi138 ceph-mon[36954]: pgmap v6954: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:06:34.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:06:33 smithi100 ceph-mon[32599]: pgmap v6955: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:06:34.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:06:33 smithi167 ceph-mon[37140]: pgmap v6955: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:06:34.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:06:33 smithi138 ceph-mon[36954]: pgmap v6955: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:06:36.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:06:35 smithi100 ceph-mon[32599]: pgmap v6956: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:06:36.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:06:35 smithi167 ceph-mon[37140]: pgmap v6956: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:06:36.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:06:35 smithi138 ceph-mon[36954]: pgmap v6956: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:06:38.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:06:37 smithi167 ceph-mon[37140]: pgmap v6957: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:06:38.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:06:37 smithi138 ceph-mon[36954]: pgmap v6957: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:06:38.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:06:37 smithi100 ceph-mon[32599]: pgmap v6957: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:06:38.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:06:38.832+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:06:38.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:06:38.833+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:06:38.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:06:38.833+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:06:38.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:06:38.833+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:06:38.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:06:38.833+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:06:40.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:06:40 smithi167 ceph-mon[37140]: pgmap v6958: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T17:06:40.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:06:40 smithi138 ceph-mon[36954]: pgmap v6958: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T17:06:40.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:06:40 smithi100 ceph-mon[32599]: pgmap v6958: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T17:06:42.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:06:42 smithi167 ceph-mon[37140]: pgmap v6959: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:06:42.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:06:42 smithi138 ceph-mon[36954]: pgmap v6959: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:06:42.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:06:42 smithi100 ceph-mon[32599]: pgmap v6959: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:06:44.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:06:44 smithi167 ceph-mon[37140]: pgmap v6960: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T17:06:44.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:06:44 smithi138 ceph-mon[36954]: pgmap v6960: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T17:06:44.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:06:44 smithi100 ceph-mon[32599]: pgmap v6960: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T17:06:46.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:06:46 smithi167 ceph-mon[37140]: pgmap v6961: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:06:46.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:06:46 smithi138 ceph-mon[36954]: pgmap v6961: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:06:46.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:06:46 smithi100 ceph-mon[32599]: pgmap v6961: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:06:48.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:06:48 smithi167 ceph-mon[37140]: pgmap v6962: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:06:48.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:06:48 smithi138 ceph-mon[36954]: pgmap v6962: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:06:48.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:06:48 smithi100 ceph-mon[32599]: pgmap v6962: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:06:50.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:06:50 smithi167 ceph-mon[37140]: pgmap v6963: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:06:50.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:06:50 smithi138 ceph-mon[36954]: pgmap v6963: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:06:50.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:06:50 smithi100 ceph-mon[32599]: pgmap v6963: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:06:52.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:06:52 smithi167 ceph-mon[37140]: pgmap v6964: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:06:52.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:06:52 smithi138 ceph-mon[36954]: pgmap v6964: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:06:52.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:06:52 smithi100 ceph-mon[32599]: pgmap v6964: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:06:53.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:06:53.847+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:06:53.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:06:53.848+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:06:53.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:06:53.848+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:06:53.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:06:53.848+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:06:53.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:06:53.848+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:06:54.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:06:54 smithi100 ceph-mon[32599]: pgmap v6965: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:06:54.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:06:54 smithi138 ceph-mon[36954]: pgmap v6965: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:06:54.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:06:54 smithi167 ceph-mon[37140]: pgmap v6965: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:06:56.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:06:56 smithi138 ceph-mon[36954]: pgmap v6966: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:06:56.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:06:56 smithi100 ceph-mon[32599]: pgmap v6966: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:06:56.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:06:56 smithi167 ceph-mon[37140]: pgmap v6966: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:06:58.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:06:58 smithi138 ceph-mon[36954]: pgmap v6967: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:06:58.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:06:58 smithi100 ceph-mon[32599]: pgmap v6967: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:06:58.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:06:58 smithi167 ceph-mon[37140]: pgmap v6967: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:07:00.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:07:00 smithi138 ceph-mon[36954]: pgmap v6968: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:07:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:07:00 smithi100 ceph-mon[32599]: pgmap v6968: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:07:00.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:07:00 smithi167 ceph-mon[37140]: pgmap v6968: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:07:02.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:07:02 smithi138 ceph-mon[36954]: pgmap v6969: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:07:02.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:07:02 smithi100 ceph-mon[32599]: pgmap v6969: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:07:02.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:07:02 smithi167 ceph-mon[37140]: pgmap v6969: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:07:04.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:07:04 smithi138 ceph-mon[36954]: pgmap v6970: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:07:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:07:04 smithi100 ceph-mon[32599]: pgmap v6970: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:07:04.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:07:04 smithi167 ceph-mon[37140]: pgmap v6970: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:07:06.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:07:06 smithi138 ceph-mon[36954]: pgmap v6971: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:07:06.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:07:06 smithi100 ceph-mon[32599]: pgmap v6971: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:07:06.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:07:06 smithi167 ceph-mon[37140]: pgmap v6971: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:07:08.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:07:08 smithi138 ceph-mon[36954]: pgmap v6972: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:07:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:07:08 smithi100 ceph-mon[32599]: pgmap v6972: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:07:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:07:08 smithi167 ceph-mon[37140]: pgmap v6972: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:07:08.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:07:08.856+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:07:08.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:07:08.856+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:07:08.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:07:08.856+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:07:08.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:07:08.856+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:07:08.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:07:08.856+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:07:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:07:09 smithi100 ceph-mon[32599]: osdmap e482: 12 total, 12 up, 12 in 2024-07-26T17:07:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:07:09 smithi138 ceph-mon[36954]: osdmap e482: 12 total, 12 up, 12 in 2024-07-26T17:07:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:07:09 smithi167 ceph-mon[37140]: osdmap e482: 12 total, 12 up, 12 in 2024-07-26T17:07:10.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:07:10 smithi138 ceph-mon[36954]: pgmap v6974: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:07:10.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:07:10 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:07:10.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:07:10 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:07:10.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:07:10 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:07:10.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:07:10 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:07:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:07:10 smithi100 ceph-mon[32599]: pgmap v6974: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:07:10.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:07:10 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:07:10.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:07:10 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:07:10.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:07:10 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:07:10.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:07:10 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:07:10.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:07:10 smithi167 ceph-mon[37140]: pgmap v6974: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:07:10.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:07:10 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:07:10.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:07:10 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:07:10.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:07:10 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:07:10.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:07:10 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:07:12.443 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:07:12 smithi138 ceph-mon[36954]: pgmap v6975: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:07:12.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:07:12 smithi100 ceph-mon[32599]: pgmap v6975: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:07:12.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:07:12 smithi167 ceph-mon[37140]: pgmap v6975: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:07:14.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:07:14 smithi138 ceph-mon[36954]: pgmap v6976: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:07:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:07:14 smithi100 ceph-mon[32599]: pgmap v6976: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:07:14.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:07:14 smithi167 ceph-mon[37140]: pgmap v6976: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:07:16.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:07:16 smithi138 ceph-mon[36954]: pgmap v6977: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:07:16.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:07:16 smithi100 ceph-mon[32599]: pgmap v6977: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:07:16.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:07:16 smithi167 ceph-mon[37140]: pgmap v6977: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:07:18.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:07:17 smithi100 ceph-mon[32599]: pgmap v6978: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:07:18.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:07:17 smithi100 ceph-mon[32599]: osdmap e483: 12 total, 12 up, 12 in 2024-07-26T17:07:18.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:07:17 smithi167 ceph-mon[37140]: pgmap v6978: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:07:18.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:07:17 smithi167 ceph-mon[37140]: osdmap e483: 12 total, 12 up, 12 in 2024-07-26T17:07:18.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:07:17 smithi138 ceph-mon[36954]: pgmap v6978: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:07:18.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:07:17 smithi138 ceph-mon[36954]: osdmap e483: 12 total, 12 up, 12 in 2024-07-26T17:07:20.243 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:07:19 smithi138 ceph-mon[36954]: pgmap v6980: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:07:20.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:07:19 smithi100 ceph-mon[32599]: pgmap v6980: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:07:20.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:07:19 smithi167 ceph-mon[37140]: pgmap v6980: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:07:22.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:07:21 smithi100 ceph-mon[32599]: pgmap v6981: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:07:22.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:07:21 smithi167 ceph-mon[37140]: pgmap v6981: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:07:22.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:07:21 smithi138 ceph-mon[36954]: pgmap v6981: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:07:23.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:07:23.870+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:07:23.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:07:23.870+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:07:23.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:07:23.870+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:07:23.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:07:23.870+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:07:23.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:07:23.870+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:07:24.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:07:23 smithi100 ceph-mon[32599]: pgmap v6982: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:07:24.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:07:23 smithi167 ceph-mon[37140]: pgmap v6982: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:07:24.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:07:23 smithi138 ceph-mon[36954]: pgmap v6982: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:07:26.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:07:25 smithi167 ceph-mon[37140]: pgmap v6983: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:07:26.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:07:25 smithi138 ceph-mon[36954]: pgmap v6983: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:07:26.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:07:25 smithi100 ceph-mon[32599]: pgmap v6983: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:07:28.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:07:28 smithi167 ceph-mon[37140]: pgmap v6984: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:07:28.443 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:07:28 smithi138 ceph-mon[36954]: pgmap v6984: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:07:28.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:07:28 smithi100 ceph-mon[32599]: pgmap v6984: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:07:30.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:07:30 smithi167 ceph-mon[37140]: pgmap v6985: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T17:07:30.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:07:30 smithi138 ceph-mon[36954]: pgmap v6985: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T17:07:30.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:07:30 smithi100 ceph-mon[32599]: pgmap v6985: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T17:07:32.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:07:32 smithi167 ceph-mon[37140]: pgmap v6986: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:07:32.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:07:32 smithi138 ceph-mon[36954]: pgmap v6986: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:07:32.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:07:32 smithi100 ceph-mon[32599]: pgmap v6986: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:07:34.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:07:34 smithi167 ceph-mon[37140]: pgmap v6987: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:07:34.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:07:34 smithi138 ceph-mon[36954]: pgmap v6987: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:07:34.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:07:34 smithi100 ceph-mon[32599]: pgmap v6987: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:07:36.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:07:36 smithi167 ceph-mon[37140]: pgmap v6988: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:07:36.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:07:36 smithi138 ceph-mon[36954]: pgmap v6988: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:07:36.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:07:36 smithi100 ceph-mon[32599]: pgmap v6988: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:07:38.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:07:38 smithi167 ceph-mon[37140]: pgmap v6989: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:07:38.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:07:38 smithi138 ceph-mon[36954]: pgmap v6989: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:07:38.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:07:38 smithi100 ceph-mon[32599]: pgmap v6989: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:07:38.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:07:38.885+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:07:38.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:07:38.886+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:07:38.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:07:38.886+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:07:38.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:07:38.886+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:07:38.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:07:38.886+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:07:40.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:07:40 smithi167 ceph-mon[37140]: pgmap v6990: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:07:40.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:07:40 smithi138 ceph-mon[36954]: pgmap v6990: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:07:40.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:07:40 smithi100 ceph-mon[32599]: pgmap v6990: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:07:42.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:07:42 smithi138 ceph-mon[36954]: pgmap v6991: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:07:42.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:07:42 smithi100 ceph-mon[32599]: pgmap v6991: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:07:42.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:07:42 smithi167 ceph-mon[37140]: pgmap v6991: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:07:44.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:07:44 smithi138 ceph-mon[36954]: pgmap v6992: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:07:44.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:07:44 smithi100 ceph-mon[32599]: pgmap v6992: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:07:44.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:07:44 smithi167 ceph-mon[37140]: pgmap v6992: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:07:46.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:07:46 smithi138 ceph-mon[36954]: pgmap v6993: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:07:46.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:07:46 smithi100 ceph-mon[32599]: pgmap v6993: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:07:46.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:07:46 smithi167 ceph-mon[37140]: pgmap v6993: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:07:48.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:07:48 smithi138 ceph-mon[36954]: pgmap v6994: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:07:48.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:07:48 smithi100 ceph-mon[32599]: pgmap v6994: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:07:48.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:07:48 smithi167 ceph-mon[37140]: pgmap v6994: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:07:50.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:07:50 smithi138 ceph-mon[36954]: pgmap v6995: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:07:50.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:07:50 smithi100 ceph-mon[32599]: pgmap v6995: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:07:50.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:07:50 smithi167 ceph-mon[37140]: pgmap v6995: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:07:52.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:07:52 smithi138 ceph-mon[36954]: pgmap v6996: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:07:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:07:52 smithi100 ceph-mon[32599]: pgmap v6996: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:07:52.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:07:52 smithi167 ceph-mon[37140]: pgmap v6996: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:07:53.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:07:53.901+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:07:53.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:07:53.901+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:07:53.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:07:53.901+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:07:53.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:07:53.901+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:07:53.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:07:53.902+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:07:54.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:07:54 smithi100 ceph-mon[32599]: pgmap v6997: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:07:54.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:07:54 smithi167 ceph-mon[37140]: pgmap v6997: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:07:54.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:07:54 smithi138 ceph-mon[36954]: pgmap v6997: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:07:56.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:07:56 smithi138 ceph-mon[36954]: pgmap v6998: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:07:56.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:07:56 smithi100 ceph-mon[32599]: pgmap v6998: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:07:56.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:07:56 smithi167 ceph-mon[37140]: pgmap v6998: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:07:58.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:07:58 smithi138 ceph-mon[36954]: pgmap v6999: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:07:58.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:07:58 smithi100 ceph-mon[32599]: pgmap v6999: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:07:58.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:07:58 smithi167 ceph-mon[37140]: pgmap v6999: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:08:00.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:08:00 smithi138 ceph-mon[36954]: pgmap v7000: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:08:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:08:00 smithi100 ceph-mon[32599]: pgmap v7000: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:08:00.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:08:00 smithi167 ceph-mon[37140]: pgmap v7000: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:08:02.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:08:02 smithi138 ceph-mon[36954]: pgmap v7001: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:08:02.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:08:02 smithi100 ceph-mon[32599]: pgmap v7001: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:08:02.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:08:02 smithi167 ceph-mon[37140]: pgmap v7001: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:08:04.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:08:04 smithi138 ceph-mon[36954]: pgmap v7002: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:08:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:08:04 smithi100 ceph-mon[32599]: pgmap v7002: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:08:04.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:08:04 smithi167 ceph-mon[37140]: pgmap v7002: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:08:06.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:08:06 smithi138 ceph-mon[36954]: pgmap v7003: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:08:06.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:08:06 smithi100 ceph-mon[32599]: pgmap v7003: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:08:06.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:08:06 smithi167 ceph-mon[37140]: pgmap v7003: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:08:08.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:08:08 smithi138 ceph-mon[36954]: pgmap v7004: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:08:08.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:08:08 smithi138 ceph-mon[36954]: osdmap e484: 12 total, 12 up, 12 in 2024-07-26T17:08:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:08:08 smithi100 ceph-mon[32599]: pgmap v7004: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:08:08.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:08:08 smithi100 ceph-mon[32599]: osdmap e484: 12 total, 12 up, 12 in 2024-07-26T17:08:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:08:08 smithi167 ceph-mon[37140]: pgmap v7004: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:08:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:08:08 smithi167 ceph-mon[37140]: osdmap e484: 12 total, 12 up, 12 in 2024-07-26T17:08:08.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:08:08.917+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:08:08.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:08:08.918+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:08:08.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:08:08.918+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:08:08.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:08:08.918+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:08:08.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:08:08.918+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:08:10.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:08:10 smithi138 ceph-mon[36954]: pgmap v7006: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:08:10.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:08:10 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:08:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:08:10 smithi100 ceph-mon[32599]: pgmap v7006: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:08:10.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:08:10 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:08:10.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:08:10 smithi167 ceph-mon[37140]: pgmap v7006: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:08:10.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:08:10 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:08:12.092 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:08:12 smithi100 ceph-mon[32599]: pgmap v7007: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:08:12.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:08:12 smithi138 ceph-mon[36954]: pgmap v7007: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:08:12.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:08:12 smithi167 ceph-mon[37140]: pgmap v7007: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:08:14.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:08:14 smithi100 ceph-mon[32599]: pgmap v7008: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:08:14.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:08:14 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:08:14.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:08:14 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:08:14.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:08:14 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:08:14.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:08:14 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:08:14.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:08:14 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:08:14.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:08:14 smithi138 ceph-mon[36954]: pgmap v7008: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:08:14.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:08:14 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:08:14.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:08:14 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:08:14.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:08:14 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:08:14.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:08:14 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:08:14.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:08:14 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:08:14.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:08:14 smithi167 ceph-mon[37140]: pgmap v7008: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:08:14.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:08:14 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:08:14.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:08:14 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:08:14.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:08:14 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:08:14.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:08:14 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:08:14.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:08:14 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:08:16.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:08:16 smithi138 ceph-mon[36954]: pgmap v7009: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:08:16.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:08:16 smithi100 ceph-mon[32599]: pgmap v7009: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:08:16.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:08:16 smithi167 ceph-mon[37140]: pgmap v7009: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:08:18.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:08:18 smithi167 ceph-mon[37140]: pgmap v7010: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:08:18.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:08:18 smithi167 ceph-mon[37140]: osdmap e485: 12 total, 12 up, 12 in 2024-07-26T17:08:18.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:08:18 smithi138 ceph-mon[36954]: pgmap v7010: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:08:18.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:08:18 smithi138 ceph-mon[36954]: osdmap e485: 12 total, 12 up, 12 in 2024-07-26T17:08:18.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:08:18 smithi100 ceph-mon[32599]: pgmap v7010: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:08:18.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:08:18 smithi100 ceph-mon[32599]: osdmap e485: 12 total, 12 up, 12 in 2024-07-26T17:08:20.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:08:20 smithi167 ceph-mon[37140]: pgmap v7012: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:08:20.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:08:20 smithi138 ceph-mon[36954]: pgmap v7012: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:08:20.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:08:20 smithi100 ceph-mon[32599]: pgmap v7012: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:08:22.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:08:22 smithi167 ceph-mon[37140]: pgmap v7013: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:08:22.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:08:22 smithi138 ceph-mon[36954]: pgmap v7013: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:08:22.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:08:22 smithi100 ceph-mon[32599]: pgmap v7013: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:08:23.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:08:23.933+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:08:23.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:08:23.934+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:08:23.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:08:23.934+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:08:23.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:08:23.934+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:08:23.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:08:23.934+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:08:24.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:08:24 smithi100 ceph-mon[32599]: pgmap v7014: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:08:24.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:08:24 smithi167 ceph-mon[37140]: pgmap v7014: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:08:24.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:08:24 smithi138 ceph-mon[36954]: pgmap v7014: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:08:26.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:08:26 smithi167 ceph-mon[37140]: pgmap v7015: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:08:26.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:08:26 smithi138 ceph-mon[36954]: pgmap v7015: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:08:26.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:08:26 smithi100 ceph-mon[32599]: pgmap v7015: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:08:28.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:08:28 smithi167 ceph-mon[37140]: pgmap v7016: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:08:28.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:08:28 smithi138 ceph-mon[36954]: pgmap v7016: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:08:28.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:08:28 smithi100 ceph-mon[32599]: pgmap v7016: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:08:30.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:08:30 smithi167 ceph-mon[37140]: pgmap v7017: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T17:08:30.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:08:30 smithi138 ceph-mon[36954]: pgmap v7017: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T17:08:30.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:08:30 smithi100 ceph-mon[32599]: pgmap v7017: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T17:08:32.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:08:32 smithi138 ceph-mon[36954]: pgmap v7018: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:08:32.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:08:32 smithi100 ceph-mon[32599]: pgmap v7018: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:08:32.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:08:32 smithi167 ceph-mon[37140]: pgmap v7018: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:08:34.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:08:34 smithi138 ceph-mon[36954]: pgmap v7019: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:08:34.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:08:34 smithi100 ceph-mon[32599]: pgmap v7019: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:08:34.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:08:34 smithi167 ceph-mon[37140]: pgmap v7019: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:08:36.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:08:36 smithi138 ceph-mon[36954]: pgmap v7020: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:08:36.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:08:36 smithi100 ceph-mon[32599]: pgmap v7020: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:08:36.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:08:36 smithi167 ceph-mon[37140]: pgmap v7020: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:08:38.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:08:38 smithi138 ceph-mon[36954]: pgmap v7021: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:08:38.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:08:38 smithi100 ceph-mon[32599]: pgmap v7021: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:08:38.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:08:38 smithi167 ceph-mon[37140]: pgmap v7021: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:08:38.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:08:38.938+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:08:38.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:08:38.938+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:08:38.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:08:38.938+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:08:38.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:08:38.938+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:08:38.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:08:38.938+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:08:40.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:08:40 smithi138 ceph-mon[36954]: pgmap v7022: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:08:40.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:08:40 smithi100 ceph-mon[32599]: pgmap v7022: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:08:40.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:08:40 smithi167 ceph-mon[37140]: pgmap v7022: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:08:42.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:08:42 smithi138 ceph-mon[36954]: pgmap v7023: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:08:42.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:08:42 smithi100 ceph-mon[32599]: pgmap v7023: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:08:42.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:08:42 smithi167 ceph-mon[37140]: pgmap v7023: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:08:44.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:08:44 smithi138 ceph-mon[36954]: pgmap v7024: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:08:44.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:08:44 smithi100 ceph-mon[32599]: pgmap v7024: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:08:44.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:08:44 smithi167 ceph-mon[37140]: pgmap v7024: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:08:46.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:08:46 smithi138 ceph-mon[36954]: pgmap v7025: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:08:46.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:08:46 smithi100 ceph-mon[32599]: pgmap v7025: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:08:46.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:08:46 smithi167 ceph-mon[37140]: pgmap v7025: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:08:48.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:08:48 smithi138 ceph-mon[36954]: pgmap v7026: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:08:48.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:08:48 smithi100 ceph-mon[32599]: pgmap v7026: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:08:48.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:08:48 smithi167 ceph-mon[37140]: pgmap v7026: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:08:50.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:08:50 smithi138 ceph-mon[36954]: pgmap v7027: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:08:50.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:08:50 smithi100 ceph-mon[32599]: pgmap v7027: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:08:50.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:08:50 smithi167 ceph-mon[37140]: pgmap v7027: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:08:52.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:08:52 smithi138 ceph-mon[36954]: pgmap v7028: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:08:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:08:52 smithi100 ceph-mon[32599]: pgmap v7028: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:08:52.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:08:52 smithi167 ceph-mon[37140]: pgmap v7028: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:08:53.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:08:53.954+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:08:53.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:08:53.954+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:08:53.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:08:53.954+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:08:53.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:08:53.954+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:08:53.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:08:53.954+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:08:54.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:08:54 smithi100 ceph-mon[32599]: pgmap v7029: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:08:54.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:08:54 smithi167 ceph-mon[37140]: pgmap v7029: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:08:54.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:08:54 smithi138 ceph-mon[36954]: pgmap v7029: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:08:56.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:08:56 smithi138 ceph-mon[36954]: pgmap v7030: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:08:56.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:08:56 smithi100 ceph-mon[32599]: pgmap v7030: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:08:56.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:08:56 smithi167 ceph-mon[37140]: pgmap v7030: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:08:58.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:08:58 smithi138 ceph-mon[36954]: pgmap v7031: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:08:58.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:08:58 smithi100 ceph-mon[32599]: pgmap v7031: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:08:58.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:08:58 smithi167 ceph-mon[37140]: pgmap v7031: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:09:00.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:00 smithi138 ceph-mon[36954]: pgmap v7032: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:09:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:00 smithi100 ceph-mon[32599]: pgmap v7032: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:09:00.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:00 smithi167 ceph-mon[37140]: pgmap v7032: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:09:02.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:02 smithi138 ceph-mon[36954]: pgmap v7033: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:09:02.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:02 smithi100 ceph-mon[32599]: pgmap v7033: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:09:02.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:02 smithi167 ceph-mon[37140]: pgmap v7033: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:09:04.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:04 smithi138 ceph-mon[36954]: pgmap v7034: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:09:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:04 smithi100 ceph-mon[32599]: pgmap v7034: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:09:04.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:04 smithi167 ceph-mon[37140]: pgmap v7034: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:09:04.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:09:04.725+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T17:09:04.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:09:04.725+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c003690 con 0x7fe5a81087a0 2024-07-26T17:09:04.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:09:04.725+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 96 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c070 con 0x7fe5a81087a0 2024-07-26T17:09:04.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:09:04.725+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 96 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T17:09:06.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:06 smithi138 ceph-mon[36954]: pgmap v7035: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:09:06.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:06 smithi100 ceph-mon[32599]: pgmap v7035: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:09:06.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:06 smithi167 ceph-mon[37140]: pgmap v7035: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:09:08.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:08 smithi138 ceph-mon[36954]: pgmap v7036: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:09:08.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:08 smithi138 ceph-mon[36954]: osdmap e486: 12 total, 12 up, 12 in 2024-07-26T17:09:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:08 smithi100 ceph-mon[32599]: pgmap v7036: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:09:08.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:08 smithi100 ceph-mon[32599]: osdmap e486: 12 total, 12 up, 12 in 2024-07-26T17:09:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:08 smithi167 ceph-mon[37140]: pgmap v7036: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:09:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:08 smithi167 ceph-mon[37140]: osdmap e486: 12 total, 12 up, 12 in 2024-07-26T17:09:08.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:09:08.967+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:09:08.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:09:08.968+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:09:08.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:09:08.968+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:09:08.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:09:08.968+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:09:08.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:09:08.968+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:09:10.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:10 smithi138 ceph-mon[36954]: pgmap v7038: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:09:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:10 smithi100 ceph-mon[32599]: pgmap v7038: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:09:10.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:10 smithi167 ceph-mon[37140]: pgmap v7038: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:09:12.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:12 smithi138 ceph-mon[36954]: pgmap v7039: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:09:12.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:12 smithi100 ceph-mon[32599]: pgmap v7039: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:09:12.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:12 smithi167 ceph-mon[37140]: pgmap v7039: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:09:14.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:14 smithi138 ceph-mon[36954]: pgmap v7040: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:09:14.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:14 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:09:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:14 smithi100 ceph-mon[32599]: pgmap v7040: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:09:14.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:14 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:09:14.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:14 smithi167 ceph-mon[37140]: pgmap v7040: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:09:14.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:14 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:09:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:15 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:09:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:15 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:09:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:15 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:09:15.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:15 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:09:15.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:15 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T17:09:15.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:15 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T17:09:15.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:15 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:09:15.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:15 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:09:15.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:15 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:09:15.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:15 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:09:15.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:15 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T17:09:15.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:15 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T17:09:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:15 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:09:15.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:15 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:09:15.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:15 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:09:15.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:15 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:09:15.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:15 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T17:09:15.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:15 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T17:09:16.267 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:16 smithi167 ceph-mon[37140]: pgmap v7041: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:09:16.396 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:16 smithi138 ceph-mon[36954]: pgmap v7041: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:09:16.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:16 smithi100 ceph-mon[32599]: pgmap v7041: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:09:18.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:18 smithi167 ceph-mon[37140]: pgmap v7042: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:09:18.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:18 smithi167 ceph-mon[37140]: osdmap e487: 12 total, 12 up, 12 in 2024-07-26T17:09:18.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:18 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:09:18.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:18 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:09:18.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:18 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:09:18.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:18 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:09:18.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:18 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:09:18.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:18 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:09:18.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:18 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:09:18.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:18 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:09:18.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:18 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:09:18.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:18 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:09:18.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:18 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:09:18.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:18 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:09:18.276 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:18 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:09:18.276 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:18 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:09:18.276 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:18 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:09:18.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:18 smithi138 ceph-mon[36954]: pgmap v7042: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:09:18.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:18 smithi138 ceph-mon[36954]: osdmap e487: 12 total, 12 up, 12 in 2024-07-26T17:09:18.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:18 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:09:18.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:18 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:09:18.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:18 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:09:18.443 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:18 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:09:18.443 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:18 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:09:18.443 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:18 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:09:18.443 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:18 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:09:18.444 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:18 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:09:18.444 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:18 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:09:18.444 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:18 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:09:18.444 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:18 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:09:18.445 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:18 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:09:18.445 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:18 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:09:18.445 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:18 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:09:18.445 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:18 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:09:18.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:18 smithi100 ceph-mon[32599]: pgmap v7042: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:09:18.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:18 smithi100 ceph-mon[32599]: osdmap e487: 12 total, 12 up, 12 in 2024-07-26T17:09:18.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:18 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:09:18.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:18 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:09:18.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:18 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:09:18.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:18 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:09:18.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:18 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:09:18.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:18 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:09:18.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:18 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:09:18.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:18 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:09:18.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:18 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:09:18.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:18 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:09:18.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:18 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:09:18.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:18 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:09:18.497 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:18 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:09:18.497 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:18 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:09:18.497 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:18 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:09:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:19 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T17:09:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:19 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T17:09:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:19 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T17:09:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:19 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T17:09:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:19 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T17:09:19.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:19 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T17:09:19.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:19 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T17:09:19.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:19 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T17:09:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:19 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T17:09:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:19 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T17:09:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:19 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T17:09:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:19 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T17:09:20.342 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:20 smithi138 ceph-mon[36954]: pgmap v7044: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:09:20.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:20 smithi100 ceph-mon[32599]: pgmap v7044: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:09:20.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:20 smithi167 ceph-mon[37140]: pgmap v7044: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:09:22.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:22 smithi138 ceph-mon[36954]: pgmap v7045: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:09:22.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:22 smithi100 ceph-mon[32599]: pgmap v7045: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:09:22.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:22 smithi167 ceph-mon[37140]: pgmap v7045: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:09:23.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:09:23.983+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:09:23.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:09:23.983+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:09:23.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:09:23.983+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:09:23.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:09:23.983+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:09:23.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:09:23.983+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:09:24.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:24 smithi100 ceph-mon[32599]: pgmap v7046: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:09:24.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:24 smithi167 ceph-mon[37140]: pgmap v7046: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:09:24.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:24 smithi138 ceph-mon[36954]: pgmap v7046: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:09:26.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:26 smithi138 ceph-mon[36954]: pgmap v7047: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:09:26.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:26 smithi100 ceph-mon[32599]: pgmap v7047: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:09:26.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:26 smithi167 ceph-mon[37140]: pgmap v7047: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:09:28.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:28 smithi138 ceph-mon[36954]: pgmap v7048: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:09:28.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:28 smithi100 ceph-mon[32599]: pgmap v7048: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:09:28.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:28 smithi167 ceph-mon[37140]: pgmap v7048: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:09:30.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:30 smithi138 ceph-mon[36954]: pgmap v7049: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T17:09:30.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:30 smithi100 ceph-mon[32599]: pgmap v7049: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T17:09:30.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:30 smithi167 ceph-mon[37140]: pgmap v7049: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T17:09:32.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:32 smithi138 ceph-mon[36954]: pgmap v7050: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:09:32.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:32 smithi100 ceph-mon[32599]: pgmap v7050: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:09:32.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:32 smithi167 ceph-mon[37140]: pgmap v7050: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:09:34.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:34 smithi138 ceph-mon[36954]: pgmap v7051: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:09:34.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:34 smithi100 ceph-mon[32599]: pgmap v7051: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:09:34.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:34 smithi167 ceph-mon[37140]: pgmap v7051: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:09:36.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:36 smithi138 ceph-mon[36954]: pgmap v7052: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:09:36.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:36 smithi100 ceph-mon[32599]: pgmap v7052: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:09:36.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:36 smithi167 ceph-mon[37140]: pgmap v7052: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:09:38.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:38 smithi138 ceph-mon[36954]: pgmap v7053: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:09:38.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:38 smithi100 ceph-mon[32599]: pgmap v7053: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:09:38.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:38 smithi167 ceph-mon[37140]: pgmap v7053: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:09:38.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:09:38.991+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:09:38.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:09:38.992+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:09:38.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:09:38.992+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:09:38.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:09:38.992+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:09:38.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:09:38.992+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:09:40.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:40 smithi138 ceph-mon[36954]: pgmap v7054: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:09:40.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:40 smithi100 ceph-mon[32599]: pgmap v7054: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:09:40.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:40 smithi167 ceph-mon[37140]: pgmap v7054: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:09:42.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:42 smithi138 ceph-mon[36954]: pgmap v7055: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:09:42.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:42 smithi100 ceph-mon[32599]: pgmap v7055: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:09:42.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:42 smithi167 ceph-mon[37140]: pgmap v7055: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:09:44.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:44 smithi138 ceph-mon[36954]: pgmap v7056: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:09:44.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:44 smithi100 ceph-mon[32599]: pgmap v7056: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:09:44.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:44 smithi167 ceph-mon[37140]: pgmap v7056: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:09:46.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:46 smithi138 ceph-mon[36954]: pgmap v7057: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:09:46.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:46 smithi100 ceph-mon[32599]: pgmap v7057: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:09:46.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:46 smithi167 ceph-mon[37140]: pgmap v7057: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:09:48.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:48 smithi138 ceph-mon[36954]: pgmap v7058: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:09:48.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:48 smithi100 ceph-mon[32599]: pgmap v7058: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:09:48.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:48 smithi167 ceph-mon[37140]: pgmap v7058: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:09:50.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:50 smithi138 ceph-mon[36954]: pgmap v7059: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:09:50.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:50 smithi100 ceph-mon[32599]: pgmap v7059: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:09:50.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:50 smithi167 ceph-mon[37140]: pgmap v7059: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:09:52.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:52 smithi138 ceph-mon[36954]: pgmap v7060: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:09:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:52 smithi100 ceph-mon[32599]: pgmap v7060: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:09:52.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:52 smithi167 ceph-mon[37140]: pgmap v7060: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:09:54.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:09:54.001+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:09:54.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:09:54.001+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:09:54.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:09:54.001+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:09:54.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:09:54.001+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:09:54.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:09:54.001+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:09:54.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:54 smithi100 ceph-mon[32599]: pgmap v7061: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:09:54.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:54 smithi138 ceph-mon[36954]: pgmap v7061: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:09:54.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:54 smithi167 ceph-mon[37140]: pgmap v7061: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:09:56.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:56 smithi138 ceph-mon[36954]: pgmap v7062: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:09:56.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:56 smithi100 ceph-mon[32599]: pgmap v7062: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:09:56.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:56 smithi167 ceph-mon[37140]: pgmap v7062: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:09:58.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:09:58 smithi138 ceph-mon[36954]: pgmap v7063: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:09:58.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:09:58 smithi100 ceph-mon[32599]: pgmap v7063: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:09:58.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:09:58 smithi167 ceph-mon[37140]: pgmap v7063: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:10:00.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:10:00 smithi138 ceph-mon[36954]: pgmap v7064: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:10:00.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:10:00 smithi138 ceph-mon[36954]: overall HEALTH_OK 2024-07-26T17:10:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:10:00 smithi100 ceph-mon[32599]: pgmap v7064: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:10:00.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:10:00 smithi100 ceph-mon[32599]: overall HEALTH_OK 2024-07-26T17:10:00.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:10:00 smithi167 ceph-mon[37140]: pgmap v7064: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:10:00.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:10:00 smithi167 ceph-mon[37140]: overall HEALTH_OK 2024-07-26T17:10:02.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:10:02 smithi138 ceph-mon[36954]: pgmap v7065: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:10:02.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:10:02 smithi100 ceph-mon[32599]: pgmap v7065: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:10:02.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:10:02 smithi167 ceph-mon[37140]: pgmap v7065: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:10:04.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:10:04 smithi138 ceph-mon[36954]: pgmap v7066: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:10:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:10:04 smithi100 ceph-mon[32599]: pgmap v7066: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:10:04.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:10:04 smithi167 ceph-mon[37140]: pgmap v7066: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:10:06.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:10:06 smithi138 ceph-mon[36954]: pgmap v7067: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:10:06.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:10:06 smithi100 ceph-mon[32599]: pgmap v7067: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:10:06.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:10:06 smithi167 ceph-mon[37140]: pgmap v7067: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:10:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:10:08 smithi138 ceph-mon[36954]: pgmap v7068: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:10:08.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:10:08 smithi138 ceph-mon[36954]: osdmap e488: 12 total, 12 up, 12 in 2024-07-26T17:10:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:10:08 smithi100 ceph-mon[32599]: pgmap v7068: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:10:08.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:10:08 smithi100 ceph-mon[32599]: osdmap e488: 12 total, 12 up, 12 in 2024-07-26T17:10:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:10:08 smithi167 ceph-mon[37140]: pgmap v7068: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:10:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:10:08 smithi167 ceph-mon[37140]: osdmap e488: 12 total, 12 up, 12 in 2024-07-26T17:10:09.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:10:09.008+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:10:09.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:10:09.009+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:10:09.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:10:09.009+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:10:09.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:10:09.009+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:10:09.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:10:09.009+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:10:10.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:10:10 smithi138 ceph-mon[36954]: pgmap v7070: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:10:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:10:10 smithi100 ceph-mon[32599]: pgmap v7070: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:10:10.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:10:10 smithi167 ceph-mon[37140]: pgmap v7070: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:10:12.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:10:12 smithi138 ceph-mon[36954]: pgmap v7071: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:10:12.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:10:12 smithi100 ceph-mon[32599]: pgmap v7071: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:10:12.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:10:12 smithi167 ceph-mon[37140]: pgmap v7071: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:10:14.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:10:14 smithi138 ceph-mon[36954]: pgmap v7072: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:10:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:10:14 smithi100 ceph-mon[32599]: pgmap v7072: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:10:14.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:10:14 smithi167 ceph-mon[37140]: pgmap v7072: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:10:16.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:10:16 smithi138 ceph-mon[36954]: pgmap v7073: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:10:16.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:10:16 smithi100 ceph-mon[32599]: pgmap v7073: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:10:16.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:10:16 smithi167 ceph-mon[37140]: pgmap v7073: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:10:18.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:10:18 smithi138 ceph-mon[36954]: pgmap v7074: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:10:18.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:10:18 smithi138 ceph-mon[36954]: osdmap e489: 12 total, 12 up, 12 in 2024-07-26T17:10:18.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:10:18 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:10:18.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:10:18 smithi100 ceph-mon[32599]: pgmap v7074: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:10:18.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:10:18 smithi100 ceph-mon[32599]: osdmap e489: 12 total, 12 up, 12 in 2024-07-26T17:10:18.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:10:18 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:10:18.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:10:18 smithi167 ceph-mon[37140]: pgmap v7074: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:10:18.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:10:18 smithi167 ceph-mon[37140]: osdmap e489: 12 total, 12 up, 12 in 2024-07-26T17:10:18.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:10:18 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:10:19.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:10:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:10:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:10:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:10:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:10:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:10:19.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:10:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:10:19.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:10:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:10:19.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:10:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:10:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:10:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:10:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:10:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:10:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:10:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:10:20.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:10:20 smithi100 ceph-mon[32599]: pgmap v7076: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:10:20.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:10:20 smithi167 ceph-mon[37140]: pgmap v7076: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:10:20.619 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:10:20 smithi138 ceph-mon[36954]: pgmap v7076: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:10:22.497 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:10:22 smithi100 ceph-mon[32599]: pgmap v7077: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:10:22.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:10:22 smithi167 ceph-mon[37140]: pgmap v7077: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:10:22.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:10:22 smithi138 ceph-mon[36954]: pgmap v7077: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:10:24.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:10:24.024+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:10:24.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:10:24.025+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:10:24.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:10:24.025+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:10:24.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:10:24.025+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:10:24.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:10:24.025+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:10:24.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:10:24 smithi100 ceph-mon[32599]: pgmap v7078: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:10:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:10:24 smithi167 ceph-mon[37140]: pgmap v7078: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:10:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:10:24 smithi138 ceph-mon[36954]: pgmap v7078: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:10:26.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:10:26 smithi100 ceph-mon[32599]: pgmap v7079: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:10:26.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:10:26 smithi167 ceph-mon[37140]: pgmap v7079: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:10:26.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:10:26 smithi138 ceph-mon[36954]: pgmap v7079: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:10:28.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:10:28 smithi100 ceph-mon[32599]: pgmap v7080: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:10:28.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:10:28 smithi167 ceph-mon[37140]: pgmap v7080: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:10:28.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:10:28 smithi138 ceph-mon[36954]: pgmap v7080: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:10:30.498 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:10:30 smithi100 ceph-mon[32599]: pgmap v7081: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T17:10:30.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:10:30 smithi167 ceph-mon[37140]: pgmap v7081: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T17:10:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:10:30 smithi138 ceph-mon[36954]: pgmap v7081: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T17:10:32.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:10:32 smithi100 ceph-mon[32599]: pgmap v7082: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:10:32.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:10:32 smithi167 ceph-mon[37140]: pgmap v7082: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:10:32.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:10:32 smithi138 ceph-mon[36954]: pgmap v7082: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:10:34.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:10:34 smithi100 ceph-mon[32599]: pgmap v7083: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:10:34.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:10:34 smithi167 ceph-mon[37140]: pgmap v7083: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:10:34.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:10:34 smithi138 ceph-mon[36954]: pgmap v7083: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:10:36.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:10:36 smithi100 ceph-mon[32599]: pgmap v7084: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:10:36.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:10:36 smithi167 ceph-mon[37140]: pgmap v7084: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:10:36.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:10:36 smithi138 ceph-mon[36954]: pgmap v7084: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:10:38.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:10:38 smithi167 ceph-mon[37140]: pgmap v7085: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:10:38.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:10:38 smithi138 ceph-mon[36954]: pgmap v7085: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:10:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:10:38 smithi100 ceph-mon[32599]: pgmap v7085: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:10:39.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:10:39.039+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:10:39.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:10:39.039+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:10:39.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:10:39.039+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:10:39.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:10:39.039+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:10:39.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:10:39.039+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:10:40.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:10:40 smithi167 ceph-mon[37140]: pgmap v7086: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:10:40.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:10:40 smithi138 ceph-mon[36954]: pgmap v7086: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:10:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:10:40 smithi100 ceph-mon[32599]: pgmap v7086: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:10:42.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:10:42 smithi167 ceph-mon[37140]: pgmap v7087: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:10:42.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:10:42 smithi138 ceph-mon[36954]: pgmap v7087: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:10:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:10:42 smithi100 ceph-mon[32599]: pgmap v7087: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:10:44.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:10:44 smithi167 ceph-mon[37140]: pgmap v7088: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:10:44.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:10:44 smithi138 ceph-mon[36954]: pgmap v7088: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:10:44.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:10:44 smithi100 ceph-mon[32599]: pgmap v7088: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:10:46.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:10:46 smithi167 ceph-mon[37140]: pgmap v7089: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:10:46.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:10:46 smithi138 ceph-mon[36954]: pgmap v7089: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:10:46.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:10:46 smithi100 ceph-mon[32599]: pgmap v7089: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:10:48.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:10:48 smithi167 ceph-mon[37140]: pgmap v7090: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:10:48.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:10:48 smithi138 ceph-mon[36954]: pgmap v7090: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:10:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:10:48 smithi100 ceph-mon[32599]: pgmap v7090: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:10:50.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:10:50 smithi167 ceph-mon[37140]: pgmap v7091: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:10:50.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:10:50 smithi138 ceph-mon[36954]: pgmap v7091: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:10:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:10:50 smithi100 ceph-mon[32599]: pgmap v7091: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:10:52.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:10:52 smithi167 ceph-mon[37140]: pgmap v7092: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:10:52.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:10:52 smithi138 ceph-mon[36954]: pgmap v7092: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:10:52.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:10:52 smithi100 ceph-mon[32599]: pgmap v7092: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:10:54.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:10:54.053+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:10:54.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:10:54.054+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:10:54.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:10:54.054+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:10:54.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:10:54.054+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:10:54.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:10:54.054+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:10:54.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:10:54 smithi100 ceph-mon[32599]: pgmap v7093: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:10:54.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:10:54 smithi167 ceph-mon[37140]: pgmap v7093: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:10:54.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:10:54 smithi138 ceph-mon[36954]: pgmap v7093: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:10:56.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:10:56 smithi138 ceph-mon[36954]: pgmap v7094: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:10:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:10:56 smithi100 ceph-mon[32599]: pgmap v7094: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:10:56.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:10:56 smithi167 ceph-mon[37140]: pgmap v7094: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:10:58.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:10:58 smithi138 ceph-mon[36954]: pgmap v7095: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:10:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:10:58 smithi100 ceph-mon[32599]: pgmap v7095: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:10:58.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:10:58 smithi167 ceph-mon[37140]: pgmap v7095: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:11:00.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:11:00 smithi138 ceph-mon[36954]: pgmap v7096: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:11:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:11:00 smithi100 ceph-mon[32599]: pgmap v7096: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:11:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:11:00 smithi167 ceph-mon[37140]: pgmap v7096: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:11:02.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:11:02 smithi138 ceph-mon[36954]: pgmap v7097: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:11:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:11:02 smithi100 ceph-mon[32599]: pgmap v7097: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:11:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:11:02 smithi167 ceph-mon[37140]: pgmap v7097: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:11:04.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:11:04 smithi138 ceph-mon[36954]: pgmap v7098: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:11:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:11:04 smithi100 ceph-mon[32599]: pgmap v7098: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:11:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:11:04 smithi167 ceph-mon[37140]: pgmap v7098: 97 pgs: 97 active+clean; 234 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:11:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:11:06 smithi138 ceph-mon[36954]: pgmap v7099: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:11:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:11:06 smithi100 ceph-mon[32599]: pgmap v7099: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:11:06.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:11:06 smithi167 ceph-mon[37140]: pgmap v7099: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:11:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:11:08 smithi138 ceph-mon[36954]: pgmap v7100: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:11:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:11:08 smithi138 ceph-mon[36954]: osdmap e490: 12 total, 12 up, 12 in 2024-07-26T17:11:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:11:08 smithi100 ceph-mon[32599]: pgmap v7100: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:11:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:11:08 smithi100 ceph-mon[32599]: osdmap e490: 12 total, 12 up, 12 in 2024-07-26T17:11:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:11:08 smithi167 ceph-mon[37140]: pgmap v7100: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:11:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:11:08 smithi167 ceph-mon[37140]: osdmap e490: 12 total, 12 up, 12 in 2024-07-26T17:11:09.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:11:09.055+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:11:09.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:11:09.056+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:11:09.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:11:09.056+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:11:09.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:11:09.056+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:11:09.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:11:09.056+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:11:10.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:11:10 smithi138 ceph-mon[36954]: pgmap v7102: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:11:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:11:10 smithi100 ceph-mon[32599]: pgmap v7102: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:11:10.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:11:10 smithi167 ceph-mon[37140]: pgmap v7102: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:11:12.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:11:12 smithi138 ceph-mon[36954]: pgmap v7103: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:11:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:11:12 smithi100 ceph-mon[32599]: pgmap v7103: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:11:12.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:11:12 smithi167 ceph-mon[37140]: pgmap v7103: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:11:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:11:14 smithi138 ceph-mon[36954]: pgmap v7104: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:11:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:11:14 smithi100 ceph-mon[32599]: pgmap v7104: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:11:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:11:14 smithi167 ceph-mon[37140]: pgmap v7104: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:11:16.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:11:16 smithi138 ceph-mon[36954]: pgmap v7105: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:11:16.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:11:16 smithi100 ceph-mon[32599]: pgmap v7105: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:11:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:11:16 smithi167 ceph-mon[37140]: pgmap v7105: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:11:18.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:11:18 smithi138 ceph-mon[36954]: pgmap v7106: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:11:18.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:11:18 smithi138 ceph-mon[36954]: osdmap e491: 12 total, 12 up, 12 in 2024-07-26T17:11:18.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:11:18 smithi100 ceph-mon[32599]: pgmap v7106: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:11:18.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:11:18 smithi100 ceph-mon[32599]: osdmap e491: 12 total, 12 up, 12 in 2024-07-26T17:11:18.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:11:18 smithi167 ceph-mon[37140]: pgmap v7106: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:11:18.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:11:18 smithi167 ceph-mon[37140]: osdmap e491: 12 total, 12 up, 12 in 2024-07-26T17:11:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:11:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:11:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:11:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:11:19.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:11:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:11:19.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:11:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:11:19.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:11:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:11:19.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:11:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:11:19.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:11:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:11:19.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:11:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:11:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:11:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:11:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:11:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:11:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:11:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:11:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:11:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:11:20.342 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:11:20 smithi138 ceph-mon[36954]: pgmap v7108: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:11:20.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:11:20 smithi100 ceph-mon[32599]: pgmap v7108: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:11:20.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:11:20 smithi167 ceph-mon[37140]: pgmap v7108: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:11:22.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:11:22 smithi138 ceph-mon[36954]: pgmap v7109: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:11:22.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:11:22 smithi100 ceph-mon[32599]: pgmap v7109: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:11:22.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:11:22 smithi167 ceph-mon[37140]: pgmap v7109: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:11:24.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:11:24.072+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:11:24.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:11:24.072+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:11:24.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:11:24.072+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:11:24.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:11:24.072+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:11:24.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:11:24.072+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:11:24.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:11:24 smithi167 ceph-mon[37140]: pgmap v7110: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:11:24.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:11:24 smithi138 ceph-mon[36954]: pgmap v7110: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:11:24.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:11:24 smithi100 ceph-mon[32599]: pgmap v7110: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:11:26.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:11:26 smithi138 ceph-mon[36954]: pgmap v7111: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:11:26.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:11:26 smithi100 ceph-mon[32599]: pgmap v7111: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:11:26.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:11:26 smithi167 ceph-mon[37140]: pgmap v7111: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:11:28.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:11:28 smithi138 ceph-mon[36954]: pgmap v7112: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:11:28.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:11:28 smithi100 ceph-mon[32599]: pgmap v7112: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:11:28.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:11:28 smithi167 ceph-mon[37140]: pgmap v7112: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:11:30.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:11:30 smithi138 ceph-mon[36954]: pgmap v7113: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T17:11:30.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:11:30 smithi100 ceph-mon[32599]: pgmap v7113: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T17:11:30.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:11:30 smithi167 ceph-mon[37140]: pgmap v7113: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T17:11:32.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:11:32 smithi138 ceph-mon[36954]: pgmap v7114: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:11:32.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:11:32 smithi100 ceph-mon[32599]: pgmap v7114: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:11:32.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:11:32 smithi167 ceph-mon[37140]: pgmap v7114: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:11:34.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:11:34 smithi138 ceph-mon[36954]: pgmap v7115: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:11:34.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:11:34 smithi100 ceph-mon[32599]: pgmap v7115: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:11:34.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:11:34 smithi167 ceph-mon[37140]: pgmap v7115: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:11:36.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:11:36 smithi138 ceph-mon[36954]: pgmap v7116: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:11:36.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:11:36 smithi100 ceph-mon[32599]: pgmap v7116: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:11:36.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:11:36 smithi167 ceph-mon[37140]: pgmap v7116: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:11:38.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:11:38 smithi138 ceph-mon[36954]: pgmap v7117: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:11:38.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:11:38 smithi100 ceph-mon[32599]: pgmap v7117: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:11:38.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:11:38 smithi167 ceph-mon[37140]: pgmap v7117: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:11:39.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:11:39.087+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:11:39.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:11:39.088+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:11:39.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:11:39.088+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:11:39.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:11:39.088+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:11:39.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:11:39.088+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:11:40.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:11:40 smithi138 ceph-mon[36954]: pgmap v7118: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:11:40.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:11:40 smithi100 ceph-mon[32599]: pgmap v7118: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:11:40.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:11:40 smithi167 ceph-mon[37140]: pgmap v7118: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:11:42.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:11:42 smithi138 ceph-mon[36954]: pgmap v7119: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:11:42.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:11:42 smithi100 ceph-mon[32599]: pgmap v7119: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:11:42.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:11:42 smithi167 ceph-mon[37140]: pgmap v7119: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:11:44.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:11:44 smithi138 ceph-mon[36954]: pgmap v7120: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:11:44.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:11:44 smithi100 ceph-mon[32599]: pgmap v7120: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:11:44.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:11:44 smithi167 ceph-mon[37140]: pgmap v7120: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:11:46.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:11:46 smithi138 ceph-mon[36954]: pgmap v7121: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:11:46.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:11:46 smithi100 ceph-mon[32599]: pgmap v7121: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:11:46.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:11:46 smithi167 ceph-mon[37140]: pgmap v7121: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:11:48.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:11:48 smithi138 ceph-mon[36954]: pgmap v7122: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:11:48.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:11:48 smithi100 ceph-mon[32599]: pgmap v7122: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:11:48.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:11:48 smithi167 ceph-mon[37140]: pgmap v7122: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:11:50.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:11:50 smithi138 ceph-mon[36954]: pgmap v7123: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:11:50.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:11:50 smithi100 ceph-mon[32599]: pgmap v7123: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:11:50.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:11:50 smithi167 ceph-mon[37140]: pgmap v7123: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:11:52.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:11:52 smithi138 ceph-mon[36954]: pgmap v7124: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:11:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:11:52 smithi100 ceph-mon[32599]: pgmap v7124: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:11:52.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:11:52 smithi167 ceph-mon[37140]: pgmap v7124: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:11:54.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:11:54.096+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:11:54.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:11:54.097+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:11:54.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:11:54.097+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:11:54.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:11:54.097+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:11:54.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:11:54.097+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:11:54.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:11:54 smithi138 ceph-mon[36954]: pgmap v7125: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:11:54.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:11:54 smithi100 ceph-mon[32599]: pgmap v7125: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:11:54.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:11:54 smithi167 ceph-mon[37140]: pgmap v7125: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:11:56.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:11:56 smithi138 ceph-mon[36954]: pgmap v7126: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:11:56.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:11:56 smithi100 ceph-mon[32599]: pgmap v7126: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:11:56.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:11:56 smithi167 ceph-mon[37140]: pgmap v7126: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:11:58.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:11:58 smithi138 ceph-mon[36954]: pgmap v7127: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:11:58.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:11:58 smithi100 ceph-mon[32599]: pgmap v7127: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:11:58.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:11:58 smithi167 ceph-mon[37140]: pgmap v7127: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:12:00.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:00 smithi138 ceph-mon[36954]: pgmap v7128: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:12:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:00 smithi100 ceph-mon[32599]: pgmap v7128: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:12:00.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:00 smithi167 ceph-mon[37140]: pgmap v7128: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:12:02.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:02 smithi138 ceph-mon[36954]: pgmap v7129: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:12:02.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:02 smithi100 ceph-mon[32599]: pgmap v7129: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:12:02.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:02 smithi167 ceph-mon[37140]: pgmap v7129: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:12:04.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:04 smithi138 ceph-mon[36954]: pgmap v7130: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:12:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:04 smithi100 ceph-mon[32599]: pgmap v7130: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:12:04.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:04 smithi167 ceph-mon[37140]: pgmap v7130: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:12:04.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:12:04.751+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c004740 con 0x7fe5a81087a0 2024-07-26T17:12:04.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:12:04.751+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T17:12:04.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:12:04.751+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 97 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c070 con 0x7fe5a81087a0 2024-07-26T17:12:04.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:12:04.751+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 97 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T17:12:06.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:06 smithi138 ceph-mon[36954]: pgmap v7131: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:12:06.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:06 smithi100 ceph-mon[32599]: pgmap v7131: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:12:06.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:06 smithi167 ceph-mon[37140]: pgmap v7131: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:12:08.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:08 smithi138 ceph-mon[36954]: pgmap v7132: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:12:08.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:08 smithi138 ceph-mon[36954]: osdmap e492: 12 total, 12 up, 12 in 2024-07-26T17:12:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:08 smithi100 ceph-mon[32599]: pgmap v7132: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:12:08.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:08 smithi100 ceph-mon[32599]: osdmap e492: 12 total, 12 up, 12 in 2024-07-26T17:12:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:08 smithi167 ceph-mon[37140]: pgmap v7132: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:12:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:08 smithi167 ceph-mon[37140]: osdmap e492: 12 total, 12 up, 12 in 2024-07-26T17:12:09.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:12:09.104+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:12:09.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:12:09.105+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:12:09.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:12:09.105+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:12:09.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:12:09.105+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:12:09.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:12:09.105+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:12:10.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:10 smithi138 ceph-mon[36954]: pgmap v7134: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:12:10.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:10 smithi100 ceph-mon[32599]: pgmap v7134: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:12:10.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:10 smithi167 ceph-mon[37140]: pgmap v7134: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:12:12.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:12 smithi138 ceph-mon[36954]: pgmap v7135: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:12:12.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:12 smithi100 ceph-mon[32599]: pgmap v7135: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:12:12.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:12 smithi167 ceph-mon[37140]: pgmap v7135: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:12:14.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:14 smithi138 ceph-mon[36954]: pgmap v7136: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:12:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:14 smithi100 ceph-mon[32599]: pgmap v7136: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:12:14.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:14 smithi167 ceph-mon[37140]: pgmap v7136: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:12:16.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:16 smithi138 ceph-mon[36954]: pgmap v7137: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:12:16.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:16 smithi100 ceph-mon[32599]: pgmap v7137: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:12:16.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:16 smithi167 ceph-mon[37140]: pgmap v7137: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:12:18.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:18 smithi138 ceph-mon[36954]: pgmap v7138: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:12:18.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:18 smithi138 ceph-mon[36954]: osdmap e493: 12 total, 12 up, 12 in 2024-07-26T17:12:18.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:18 smithi100 ceph-mon[32599]: pgmap v7138: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:12:18.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:18 smithi100 ceph-mon[32599]: osdmap e493: 12 total, 12 up, 12 in 2024-07-26T17:12:18.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:18 smithi167 ceph-mon[37140]: pgmap v7138: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:12:18.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:18 smithi167 ceph-mon[37140]: osdmap e493: 12 total, 12 up, 12 in 2024-07-26T17:12:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:12:19.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:12:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:12:20.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:20 smithi138 ceph-mon[36954]: pgmap v7140: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:12:20.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:20 smithi100 ceph-mon[32599]: pgmap v7140: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:12:20.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:20 smithi167 ceph-mon[37140]: pgmap v7140: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:12:21.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:21 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:12:21.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:21 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:12:21.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:21 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:12:21.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:21 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:12:21.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:21 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:12:21.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:21 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:12:21.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:21 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:12:21.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:21 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:12:21.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:21 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:12:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:21 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:12:21.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:21 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:12:21.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:21 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:12:21.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:21 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:12:21.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:21 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:12:21.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:21 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:12:21.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:21 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:12:21.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:21 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:12:21.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:21 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:12:21.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:21 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:12:21.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:21 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:12:21.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:21 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:12:21.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:21 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:12:21.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:21 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:12:21.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:21 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:12:21.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:21 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:12:21.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:21 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:12:21.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:21 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:12:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:22 smithi100 ceph-mon[32599]: pgmap v7141: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:12:22.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:22 smithi167 ceph-mon[37140]: pgmap v7141: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:12:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:22 smithi138 ceph-mon[36954]: pgmap v7141: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:12:24.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:12:24.107+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:12:24.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:12:24.107+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:12:24.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:12:24.107+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:12:24.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:12:24.107+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:12:24.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:12:24.107+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:12:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:24 smithi100 ceph-mon[32599]: pgmap v7142: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:12:24.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:24 smithi167 ceph-mon[37140]: pgmap v7142: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:12:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:24 smithi138 ceph-mon[36954]: pgmap v7142: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:12:26.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:26 smithi100 ceph-mon[32599]: pgmap v7143: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:12:26.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:26 smithi167 ceph-mon[37140]: pgmap v7143: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:12:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:26 smithi138 ceph-mon[36954]: pgmap v7143: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:12:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:28 smithi100 ceph-mon[32599]: pgmap v7144: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:12:28.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:28 smithi167 ceph-mon[37140]: pgmap v7144: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:12:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:28 smithi138 ceph-mon[36954]: pgmap v7144: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:12:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:30 smithi100 ceph-mon[32599]: pgmap v7145: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T17:12:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:30 smithi167 ceph-mon[37140]: pgmap v7145: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T17:12:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:30 smithi138 ceph-mon[36954]: pgmap v7145: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T17:12:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:32 smithi100 ceph-mon[32599]: pgmap v7146: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:12:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:32 smithi167 ceph-mon[37140]: pgmap v7146: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:12:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:32 smithi138 ceph-mon[36954]: pgmap v7146: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:12:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:34 smithi100 ceph-mon[32599]: pgmap v7147: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:12:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:34 smithi167 ceph-mon[37140]: pgmap v7147: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:12:34.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:34 smithi138 ceph-mon[36954]: pgmap v7147: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:12:36.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:36 smithi100 ceph-mon[32599]: pgmap v7148: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:12:36.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:36 smithi167 ceph-mon[37140]: pgmap v7148: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:12:36.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:36 smithi138 ceph-mon[36954]: pgmap v7148: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:12:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:38 smithi100 ceph-mon[32599]: pgmap v7149: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:12:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:38 smithi167 ceph-mon[37140]: pgmap v7149: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:12:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:38 smithi138 ceph-mon[36954]: pgmap v7149: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:12:39.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:12:39.123+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:12:39.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:12:39.124+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:12:39.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:12:39.124+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:12:39.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:12:39.124+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:12:39.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:12:39.124+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:12:40.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:40 smithi167 ceph-mon[37140]: pgmap v7150: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:12:40.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:40 smithi138 ceph-mon[36954]: pgmap v7150: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:12:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:40 smithi100 ceph-mon[32599]: pgmap v7150: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:12:42.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:42 smithi167 ceph-mon[37140]: pgmap v7151: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:12:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:42 smithi138 ceph-mon[36954]: pgmap v7151: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:12:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:42 smithi100 ceph-mon[32599]: pgmap v7151: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:12:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:44 smithi167 ceph-mon[37140]: pgmap v7152: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:12:44.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:44 smithi138 ceph-mon[36954]: pgmap v7152: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:12:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:44 smithi100 ceph-mon[32599]: pgmap v7152: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:12:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:46 smithi167 ceph-mon[37140]: pgmap v7153: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:12:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:46 smithi138 ceph-mon[36954]: pgmap v7153: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:12:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:46 smithi100 ceph-mon[32599]: pgmap v7153: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:12:48.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:48 smithi167 ceph-mon[37140]: pgmap v7154: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:12:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:48 smithi138 ceph-mon[36954]: pgmap v7154: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:12:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:48 smithi100 ceph-mon[32599]: pgmap v7154: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:12:50.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:50 smithi167 ceph-mon[37140]: pgmap v7155: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:12:50.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:50 smithi138 ceph-mon[36954]: pgmap v7155: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:12:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:50 smithi100 ceph-mon[32599]: pgmap v7155: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:12:52.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:52 smithi167 ceph-mon[37140]: pgmap v7156: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:12:52.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:52 smithi138 ceph-mon[36954]: pgmap v7156: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:12:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:52 smithi100 ceph-mon[32599]: pgmap v7156: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:12:54.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:12:54.138+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:12:54.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:12:54.138+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:12:54.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:12:54.138+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:12:54.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:12:54.138+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:12:54.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:12:54.138+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:12:54.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:54 smithi167 ceph-mon[37140]: pgmap v7157: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:12:54.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:54 smithi138 ceph-mon[36954]: pgmap v7157: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:12:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:54 smithi100 ceph-mon[32599]: pgmap v7157: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:12:56.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:56 smithi138 ceph-mon[36954]: pgmap v7158: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:12:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:56 smithi100 ceph-mon[32599]: pgmap v7158: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:12:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:56 smithi167 ceph-mon[37140]: pgmap v7158: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:12:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:12:58 smithi138 ceph-mon[36954]: pgmap v7159: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:12:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:12:58 smithi100 ceph-mon[32599]: pgmap v7159: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:12:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:12:58 smithi167 ceph-mon[37140]: pgmap v7159: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:13:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:00 smithi138 ceph-mon[36954]: pgmap v7160: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:13:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:00 smithi100 ceph-mon[32599]: pgmap v7160: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:13:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:00 smithi167 ceph-mon[37140]: pgmap v7160: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:13:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:02 smithi138 ceph-mon[36954]: pgmap v7161: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:13:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:02 smithi100 ceph-mon[32599]: pgmap v7161: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:13:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:02 smithi167 ceph-mon[37140]: pgmap v7161: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:13:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:04 smithi138 ceph-mon[36954]: pgmap v7162: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:13:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:04 smithi100 ceph-mon[32599]: pgmap v7162: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:13:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:04 smithi167 ceph-mon[37140]: pgmap v7162: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:13:06.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:06 smithi138 ceph-mon[36954]: pgmap v7163: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:13:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:06 smithi100 ceph-mon[32599]: pgmap v7163: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:13:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:06 smithi167 ceph-mon[37140]: pgmap v7163: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:13:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:08 smithi138 ceph-mon[36954]: pgmap v7164: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:13:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:08 smithi138 ceph-mon[36954]: osdmap e494: 12 total, 12 up, 12 in 2024-07-26T17:13:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:08 smithi100 ceph-mon[32599]: pgmap v7164: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:13:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:08 smithi100 ceph-mon[32599]: osdmap e494: 12 total, 12 up, 12 in 2024-07-26T17:13:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:08 smithi167 ceph-mon[37140]: pgmap v7164: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:13:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:08 smithi167 ceph-mon[37140]: osdmap e494: 12 total, 12 up, 12 in 2024-07-26T17:13:09.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:13:09.154+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:13:09.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:13:09.155+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:13:09.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:13:09.155+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:13:09.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:13:09.155+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:13:09.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:13:09.155+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:13:10.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:10 smithi138 ceph-mon[36954]: pgmap v7166: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:13:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:10 smithi100 ceph-mon[32599]: pgmap v7166: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:13:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:10 smithi167 ceph-mon[37140]: pgmap v7166: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:13:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:12 smithi138 ceph-mon[36954]: pgmap v7167: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:13:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:12 smithi100 ceph-mon[32599]: pgmap v7167: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:13:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:12 smithi167 ceph-mon[37140]: pgmap v7167: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:13:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:14 smithi138 ceph-mon[36954]: pgmap v7168: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:13:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:14 smithi100 ceph-mon[32599]: pgmap v7168: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:13:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:14 smithi167 ceph-mon[37140]: pgmap v7168: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:13:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:16 smithi138 ceph-mon[36954]: pgmap v7169: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:13:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:16 smithi100 ceph-mon[32599]: pgmap v7169: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:13:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:16 smithi167 ceph-mon[37140]: pgmap v7169: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:13:18.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:18 smithi138 ceph-mon[36954]: pgmap v7170: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:13:18.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:18 smithi138 ceph-mon[36954]: osdmap e495: 12 total, 12 up, 12 in 2024-07-26T17:13:18.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:18 smithi100 ceph-mon[32599]: pgmap v7170: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:13:18.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:18 smithi100 ceph-mon[32599]: osdmap e495: 12 total, 12 up, 12 in 2024-07-26T17:13:18.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:18 smithi167 ceph-mon[37140]: pgmap v7170: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:13:18.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:18 smithi167 ceph-mon[37140]: osdmap e495: 12 total, 12 up, 12 in 2024-07-26T17:13:20.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:20 smithi138 ceph-mon[36954]: pgmap v7172: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:13:20.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:20 smithi100 ceph-mon[32599]: pgmap v7172: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:13:20.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:20 smithi167 ceph-mon[37140]: pgmap v7172: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:13:21.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:21 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:13:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:21 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:13:21.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:21 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:13:22.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:22 smithi167 ceph-mon[37140]: pgmap v7173: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:13:22.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:13:22.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:13:22.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:13:22.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:13:22.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:13:22.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:13:22.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:13:22.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:13:22.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:13:22.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:22 smithi138 ceph-mon[36954]: pgmap v7173: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:13:22.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:13:22.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:13:22.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:13:22.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:13:22.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:13:22.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:13:22.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:13:22.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:13:22.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:13:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:22 smithi100 ceph-mon[32599]: pgmap v7173: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:13:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:13:22.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:13:22.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:13:22.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:13:22.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:13:22.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:13:22.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:13:22.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:13:22.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:13:24.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:13:24.170+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:13:24.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:13:24.171+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:13:24.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:13:24.171+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:13:24.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:13:24.171+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:13:24.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:13:24.171+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:13:24.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:24 smithi100 ceph-mon[32599]: pgmap v7174: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:13:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:24 smithi167 ceph-mon[37140]: pgmap v7174: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:13:24.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:24 smithi138 ceph-mon[36954]: pgmap v7174: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:13:26.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:26 smithi167 ceph-mon[37140]: pgmap v7175: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:13:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:26 smithi138 ceph-mon[36954]: pgmap v7175: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:13:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:26 smithi100 ceph-mon[32599]: pgmap v7175: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:13:28.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:28 smithi167 ceph-mon[37140]: pgmap v7176: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:13:28.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:28 smithi138 ceph-mon[36954]: pgmap v7176: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:13:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:28 smithi100 ceph-mon[32599]: pgmap v7176: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:13:30.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:30 smithi167 ceph-mon[37140]: pgmap v7177: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T17:13:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:30 smithi138 ceph-mon[36954]: pgmap v7177: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T17:13:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:30 smithi100 ceph-mon[32599]: pgmap v7177: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-07-26T17:13:32.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:32 smithi167 ceph-mon[37140]: pgmap v7178: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:13:32.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:32 smithi138 ceph-mon[36954]: pgmap v7178: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:13:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:32 smithi100 ceph-mon[32599]: pgmap v7178: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:13:34.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:34 smithi138 ceph-mon[36954]: pgmap v7179: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:13:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:34 smithi100 ceph-mon[32599]: pgmap v7179: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:13:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:34 smithi167 ceph-mon[37140]: pgmap v7179: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:13:36.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:36 smithi138 ceph-mon[36954]: pgmap v7180: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:13:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:36 smithi100 ceph-mon[32599]: pgmap v7180: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:13:36.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:36 smithi167 ceph-mon[37140]: pgmap v7180: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:13:38.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:38 smithi138 ceph-mon[36954]: pgmap v7181: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:13:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:38 smithi100 ceph-mon[32599]: pgmap v7181: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:13:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:38 smithi167 ceph-mon[37140]: pgmap v7181: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:13:39.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:13:39.175+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:13:39.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:13:39.176+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:13:39.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:13:39.176+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:13:39.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:13:39.176+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:13:39.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:13:39.176+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:13:40.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:40 smithi138 ceph-mon[36954]: pgmap v7182: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:13:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:40 smithi100 ceph-mon[32599]: pgmap v7182: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:13:40.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:40 smithi167 ceph-mon[37140]: pgmap v7182: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:13:42.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:42 smithi138 ceph-mon[36954]: pgmap v7183: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:13:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:42 smithi100 ceph-mon[32599]: pgmap v7183: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:13:42.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:42 smithi167 ceph-mon[37140]: pgmap v7183: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:13:44.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:44 smithi138 ceph-mon[36954]: pgmap v7184: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:13:44.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:44 smithi100 ceph-mon[32599]: pgmap v7184: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:13:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:44 smithi167 ceph-mon[37140]: pgmap v7184: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:13:46.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:46 smithi138 ceph-mon[36954]: pgmap v7185: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:13:46.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:46 smithi100 ceph-mon[32599]: pgmap v7185: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:13:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:46 smithi167 ceph-mon[37140]: pgmap v7185: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:13:48.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:48 smithi138 ceph-mon[36954]: pgmap v7186: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:13:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:48 smithi100 ceph-mon[32599]: pgmap v7186: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:13:48.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:48 smithi167 ceph-mon[37140]: pgmap v7186: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:13:50.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:50 smithi138 ceph-mon[36954]: pgmap v7187: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:13:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:50 smithi100 ceph-mon[32599]: pgmap v7187: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:13:50.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:50 smithi167 ceph-mon[37140]: pgmap v7187: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:13:52.685 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:52 smithi138 ceph-mon[36954]: pgmap v7188: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:13:52.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:52 smithi100 ceph-mon[32599]: pgmap v7188: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:13:52.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:52 smithi167 ceph-mon[37140]: pgmap v7188: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:13:54.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:13:54.191+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:13:54.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:13:54.192+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:13:54.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:13:54.192+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:13:54.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:13:54.192+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:13:54.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:13:54.192+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:13:54.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:54 smithi100 ceph-mon[32599]: pgmap v7189: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:13:54.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:54 smithi167 ceph-mon[37140]: pgmap v7189: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:13:54.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:54 smithi138 ceph-mon[36954]: pgmap v7189: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:13:56.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:56 smithi138 ceph-mon[36954]: pgmap v7190: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:13:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:56 smithi100 ceph-mon[32599]: pgmap v7190: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:13:56.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:56 smithi167 ceph-mon[37140]: pgmap v7190: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:13:58.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:13:58 smithi138 ceph-mon[36954]: pgmap v7191: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:13:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:13:58 smithi100 ceph-mon[32599]: pgmap v7191: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:13:58.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:13:58 smithi167 ceph-mon[37140]: pgmap v7191: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:14:00.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:14:00 smithi138 ceph-mon[36954]: pgmap v7192: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T17:14:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:14:00 smithi100 ceph-mon[32599]: pgmap v7192: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T17:14:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:14:00 smithi167 ceph-mon[37140]: pgmap v7192: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T17:14:02.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:14:02 smithi138 ceph-mon[36954]: pgmap v7193: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:14:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:14:02 smithi100 ceph-mon[32599]: pgmap v7193: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:14:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:14:02 smithi167 ceph-mon[37140]: pgmap v7193: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:14:04.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:14:04 smithi138 ceph-mon[36954]: pgmap v7194: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-07-26T17:14:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:14:04 smithi100 ceph-mon[32599]: pgmap v7194: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-07-26T17:14:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:14:04 smithi167 ceph-mon[37140]: pgmap v7194: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-07-26T17:14:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:14:06 smithi138 ceph-mon[36954]: pgmap v7195: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:14:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:14:06 smithi100 ceph-mon[32599]: pgmap v7195: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:14:06.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:14:06 smithi167 ceph-mon[37140]: pgmap v7195: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:14:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:14:08 smithi138 ceph-mon[36954]: pgmap v7196: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:14:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:14:08 smithi138 ceph-mon[36954]: osdmap e496: 12 total, 12 up, 12 in 2024-07-26T17:14:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:14:08 smithi100 ceph-mon[32599]: pgmap v7196: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:14:08.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:14:08 smithi100 ceph-mon[32599]: osdmap e496: 12 total, 12 up, 12 in 2024-07-26T17:14:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:14:08 smithi167 ceph-mon[37140]: pgmap v7196: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:14:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:14:08 smithi167 ceph-mon[37140]: osdmap e496: 12 total, 12 up, 12 in 2024-07-26T17:14:09.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:14:09.207+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:14:09.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:14:09.208+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:14:09.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:14:09.208+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:14:09.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:14:09.208+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:14:09.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:14:09.208+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:14:10.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:14:10 smithi138 ceph-mon[36954]: pgmap v7198: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:14:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:14:10 smithi100 ceph-mon[32599]: pgmap v7198: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:14:10.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:14:10 smithi167 ceph-mon[37140]: pgmap v7198: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:14:12.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:14:12 smithi138 ceph-mon[36954]: pgmap v7199: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:14:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:14:12 smithi100 ceph-mon[32599]: pgmap v7199: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:14:12.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:14:12 smithi167 ceph-mon[37140]: pgmap v7199: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:14:14.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:14:14 smithi138 ceph-mon[36954]: pgmap v7200: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:14:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:14:14 smithi100 ceph-mon[32599]: pgmap v7200: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:14:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:14:14 smithi167 ceph-mon[37140]: pgmap v7200: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:14:16.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:14:16 smithi138 ceph-mon[36954]: pgmap v7201: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:14:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:14:16 smithi100 ceph-mon[32599]: pgmap v7201: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:14:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:14:16 smithi167 ceph-mon[37140]: pgmap v7201: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:14:18.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:14:18 smithi138 ceph-mon[36954]: pgmap v7202: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:14:18.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:14:18 smithi138 ceph-mon[36954]: osdmap e497: 12 total, 12 up, 12 in 2024-07-26T17:14:18.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:14:18 smithi100 ceph-mon[32599]: pgmap v7202: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:14:18.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:14:18 smithi100 ceph-mon[32599]: osdmap e497: 12 total, 12 up, 12 in 2024-07-26T17:14:18.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:14:18 smithi167 ceph-mon[37140]: pgmap v7202: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:14:18.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:14:18 smithi167 ceph-mon[37140]: osdmap e497: 12 total, 12 up, 12 in 2024-07-26T17:14:19.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:14:19.770+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c004740 con 0x7fe5a81087a0 2024-07-26T17:14:19.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:14:19.770+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T17:14:19.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:14:19.770+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 98 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T17:14:19.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:14:19.770+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 98 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe5a008c070 con 0x7fe5a81087a0 2024-07-26T17:14:20.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:14:20 smithi138 ceph-mon[36954]: pgmap v7204: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:14:20.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:14:20 smithi100 ceph-mon[32599]: pgmap v7204: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:14:20.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:14:20 smithi167 ceph-mon[37140]: pgmap v7204: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:14:22.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:14:22 smithi138 ceph-mon[36954]: pgmap v7205: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:14:22.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:14:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:14:22.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:14:22 smithi100 ceph-mon[32599]: pgmap v7205: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:14:22.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:14:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:14:22.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:14:22 smithi167 ceph-mon[37140]: pgmap v7205: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:14:22.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:14:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:14:23.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:14:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:14:23.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:14:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:14:23.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:14:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:14:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:14:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:14:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:14:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:14:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:14:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:14:23.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:14:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:14:23.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:14:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:14:23.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:14:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:14:24.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:14:24.220+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:14:24.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:14:24.221+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:14:24.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:14:24.221+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:14:24.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:14:24.221+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:14:24.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:14:24.221+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:14:24.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:14:24 smithi100 ceph-mon[32599]: pgmap v7206: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:14:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:14:24 smithi167 ceph-mon[37140]: pgmap v7206: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:14:24.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:14:24 smithi138 ceph-mon[36954]: pgmap v7206: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:14:26.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:14:26 smithi167 ceph-mon[37140]: pgmap v7207: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:14:26.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:14:26 smithi138 ceph-mon[36954]: pgmap v7207: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:14:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:14:26 smithi100 ceph-mon[32599]: pgmap v7207: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:14:28.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:14:28 smithi167 ceph-mon[37140]: pgmap v7208: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:14:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:14:28 smithi138 ceph-mon[36954]: pgmap v7208: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:14:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:14:28 smithi100 ceph-mon[32599]: pgmap v7208: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:14:30.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:14:30 smithi167 ceph-mon[37140]: pgmap v7209: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:14:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:14:30 smithi138 ceph-mon[36954]: pgmap v7209: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:14:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:14:30 smithi100 ceph-mon[32599]: pgmap v7209: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:14:32.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:14:32 smithi167 ceph-mon[37140]: pgmap v7210: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:14:32.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:14:32 smithi138 ceph-mon[36954]: pgmap v7210: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:14:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:14:32 smithi100 ceph-mon[32599]: pgmap v7210: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:14:34.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:14:34 smithi167 ceph-mon[37140]: pgmap v7211: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:14:34.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:14:34 smithi138 ceph-mon[36954]: pgmap v7211: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:14:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:14:34 smithi100 ceph-mon[32599]: pgmap v7211: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:14:36.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:14:36 smithi167 ceph-mon[37140]: pgmap v7212: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:14:36.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:14:36 smithi138 ceph-mon[36954]: pgmap v7212: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:14:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:14:36 smithi100 ceph-mon[32599]: pgmap v7212: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:14:38.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:14:38 smithi167 ceph-mon[37140]: pgmap v7213: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:14:38.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:14:38 smithi138 ceph-mon[36954]: pgmap v7213: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:14:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:14:38 smithi100 ceph-mon[32599]: pgmap v7213: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:14:39.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:14:39.236+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:14:39.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:14:39.237+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:14:39.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:14:39.237+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:14:39.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:14:39.237+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:14:39.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:14:39.237+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:14:40.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:14:40 smithi167 ceph-mon[37140]: pgmap v7214: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:14:40.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:14:40 smithi138 ceph-mon[36954]: pgmap v7214: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:14:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:14:40 smithi100 ceph-mon[32599]: pgmap v7214: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:14:42.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:14:42 smithi138 ceph-mon[36954]: pgmap v7215: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:14:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:14:42 smithi100 ceph-mon[32599]: pgmap v7215: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:14:42.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:14:42 smithi167 ceph-mon[37140]: pgmap v7215: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:14:44.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:14:44 smithi138 ceph-mon[36954]: pgmap v7216: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:14:44.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:14:44 smithi100 ceph-mon[32599]: pgmap v7216: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:14:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:14:44 smithi167 ceph-mon[37140]: pgmap v7216: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:14:46.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:14:46 smithi138 ceph-mon[36954]: pgmap v7217: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:14:46.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:14:46 smithi100 ceph-mon[32599]: pgmap v7217: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:14:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:14:46 smithi167 ceph-mon[37140]: pgmap v7217: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:14:48.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:14:48 smithi138 ceph-mon[36954]: pgmap v7218: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:14:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:14:48 smithi100 ceph-mon[32599]: pgmap v7218: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:14:48.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:14:48 smithi167 ceph-mon[37140]: pgmap v7218: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:14:50.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:14:50 smithi138 ceph-mon[36954]: pgmap v7219: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:14:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:14:50 smithi100 ceph-mon[32599]: pgmap v7219: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:14:50.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:14:50 smithi167 ceph-mon[37140]: pgmap v7219: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:14:52.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:14:52 smithi138 ceph-mon[36954]: pgmap v7220: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:14:52.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:14:52 smithi100 ceph-mon[32599]: pgmap v7220: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:14:52.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:14:52 smithi167 ceph-mon[37140]: pgmap v7220: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:14:54.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:14:54.253+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:14:54.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:14:54.253+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:14:54.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:14:54.253+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:14:54.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:14:54.253+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:14:54.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:14:54.253+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:14:54.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:14:54 smithi100 ceph-mon[32599]: pgmap v7221: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:14:54.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:14:54 smithi167 ceph-mon[37140]: pgmap v7221: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:14:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:14:54 smithi138 ceph-mon[36954]: pgmap v7221: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:14:56.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:14:56 smithi138 ceph-mon[36954]: pgmap v7222: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:14:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:14:56 smithi100 ceph-mon[32599]: pgmap v7222: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:14:56.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:14:56 smithi167 ceph-mon[37140]: pgmap v7222: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:14:58.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:14:58 smithi138 ceph-mon[36954]: pgmap v7223: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:14:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:14:58 smithi100 ceph-mon[32599]: pgmap v7223: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:14:58.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:14:58 smithi167 ceph-mon[37140]: pgmap v7223: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:15:00.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:15:00 smithi138 ceph-mon[36954]: pgmap v7224: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:15:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:15:00 smithi100 ceph-mon[32599]: pgmap v7224: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:15:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:15:00 smithi167 ceph-mon[37140]: pgmap v7224: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:15:02.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:15:02 smithi138 ceph-mon[36954]: pgmap v7225: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:15:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:15:02 smithi100 ceph-mon[32599]: pgmap v7225: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:15:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:15:02 smithi167 ceph-mon[37140]: pgmap v7225: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:15:04.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:15:04 smithi138 ceph-mon[36954]: pgmap v7226: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:15:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:15:04 smithi100 ceph-mon[32599]: pgmap v7226: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:15:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:15:04 smithi167 ceph-mon[37140]: pgmap v7226: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:15:04.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:15:04.777+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c004740 con 0x7fe5a81087a0 2024-07-26T17:15:04.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:15:04.777+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T17:15:04.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:15:04.777+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 99 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c070 con 0x7fe5a81087a0 2024-07-26T17:15:04.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:15:04.777+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 99 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T17:15:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:15:06 smithi138 ceph-mon[36954]: pgmap v7227: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:15:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:15:06 smithi100 ceph-mon[32599]: pgmap v7227: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:15:06.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:15:06 smithi167 ceph-mon[37140]: pgmap v7227: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:15:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:15:08 smithi138 ceph-mon[36954]: pgmap v7228: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:15:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:15:08 smithi138 ceph-mon[36954]: osdmap e498: 12 total, 12 up, 12 in 2024-07-26T17:15:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:15:08 smithi100 ceph-mon[32599]: pgmap v7228: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:15:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:15:08 smithi100 ceph-mon[32599]: osdmap e498: 12 total, 12 up, 12 in 2024-07-26T17:15:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:15:08 smithi167 ceph-mon[37140]: pgmap v7228: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:15:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:15:08 smithi167 ceph-mon[37140]: osdmap e498: 12 total, 12 up, 12 in 2024-07-26T17:15:09.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:15:09.267+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:15:09.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:15:09.267+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:15:09.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:15:09.267+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:15:09.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:15:09.267+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:15:09.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:15:09.267+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:15:10.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:15:10 smithi138 ceph-mon[36954]: pgmap v7230: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:15:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:15:10 smithi100 ceph-mon[32599]: pgmap v7230: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:15:10.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:15:10 smithi167 ceph-mon[37140]: pgmap v7230: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:15:12.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:15:12 smithi138 ceph-mon[36954]: pgmap v7231: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:15:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:15:12 smithi100 ceph-mon[32599]: pgmap v7231: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:15:12.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:15:12 smithi167 ceph-mon[37140]: pgmap v7231: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:15:14.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:15:14 smithi138 ceph-mon[36954]: pgmap v7232: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:15:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:15:14 smithi100 ceph-mon[32599]: pgmap v7232: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:15:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:15:14 smithi167 ceph-mon[37140]: pgmap v7232: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:15:16.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:15:16 smithi138 ceph-mon[36954]: pgmap v7233: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:15:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:15:16 smithi100 ceph-mon[32599]: pgmap v7233: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:15:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:15:16 smithi167 ceph-mon[37140]: pgmap v7233: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:15:18.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:15:18 smithi138 ceph-mon[36954]: pgmap v7234: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:15:18.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:15:18 smithi138 ceph-mon[36954]: osdmap e499: 12 total, 12 up, 12 in 2024-07-26T17:15:18.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:15:18 smithi100 ceph-mon[32599]: pgmap v7234: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:15:18.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:15:18 smithi100 ceph-mon[32599]: osdmap e499: 12 total, 12 up, 12 in 2024-07-26T17:15:18.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:15:18 smithi167 ceph-mon[37140]: pgmap v7234: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:15:18.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:15:18 smithi167 ceph-mon[37140]: osdmap e499: 12 total, 12 up, 12 in 2024-07-26T17:15:20.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:15:20 smithi138 ceph-mon[36954]: pgmap v7236: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:15:20.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:15:20 smithi100 ceph-mon[32599]: pgmap v7236: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:15:20.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:15:20 smithi167 ceph-mon[37140]: pgmap v7236: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:15:22.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:15:22 smithi138 ceph-mon[36954]: pgmap v7237: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:15:22.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:15:22 smithi100 ceph-mon[32599]: pgmap v7237: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:15:22.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:15:22 smithi167 ceph-mon[37140]: pgmap v7237: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:15:23.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:15:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:15:23.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:15:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:15:23.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:15:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:15:23.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:15:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:15:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:15:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:15:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:15:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:15:23.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:15:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:15:23.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:15:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:15:23.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:15:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:15:23.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:15:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:15:23.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:15:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:15:23.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:15:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:15:24.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:15:24.283+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:15:24.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:15:24.284+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:15:24.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:15:24.284+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:15:24.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:15:24.284+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:15:24.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:15:24.284+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:15:24.395 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:15:24 smithi100 ceph-mon[32599]: pgmap v7238: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:15:24.424 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:15:24 smithi167 ceph-mon[37140]: pgmap v7238: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:15:24.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:15:24 smithi138 ceph-mon[36954]: pgmap v7238: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:15:26.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:15:26 smithi138 ceph-mon[36954]: pgmap v7239: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:15:26.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:15:26 smithi100 ceph-mon[32599]: pgmap v7239: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:15:26.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:15:26 smithi167 ceph-mon[37140]: pgmap v7239: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:15:28.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:15:28 smithi138 ceph-mon[36954]: pgmap v7240: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:15:28.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:15:28 smithi100 ceph-mon[32599]: pgmap v7240: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:15:28.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:15:28 smithi167 ceph-mon[37140]: pgmap v7240: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:15:30.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:15:30 smithi138 ceph-mon[36954]: pgmap v7241: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:15:30.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:15:30 smithi100 ceph-mon[32599]: pgmap v7241: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:15:30.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:15:30 smithi167 ceph-mon[37140]: pgmap v7241: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:15:32.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:15:32 smithi138 ceph-mon[36954]: pgmap v7242: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:15:32.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:15:32 smithi100 ceph-mon[32599]: pgmap v7242: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:15:32.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:15:32 smithi167 ceph-mon[37140]: pgmap v7242: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:15:34.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:15:34 smithi138 ceph-mon[36954]: pgmap v7243: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:15:34.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:15:34 smithi100 ceph-mon[32599]: pgmap v7243: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:15:34.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:15:34 smithi167 ceph-mon[37140]: pgmap v7243: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:15:36.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:15:36 smithi138 ceph-mon[36954]: pgmap v7244: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:15:36.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:15:36 smithi100 ceph-mon[32599]: pgmap v7244: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:15:36.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:15:36 smithi167 ceph-mon[37140]: pgmap v7244: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:15:38.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:15:38 smithi138 ceph-mon[36954]: pgmap v7245: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:15:38.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:15:38 smithi100 ceph-mon[32599]: pgmap v7245: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:15:38.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:15:38 smithi167 ceph-mon[37140]: pgmap v7245: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:15:39.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:15:39.300+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:15:39.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:15:39.300+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:15:39.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:15:39.300+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:15:39.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:15:39.300+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:15:39.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:15:39.300+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:15:40.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:15:40 smithi138 ceph-mon[36954]: pgmap v7246: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:15:40.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:15:40 smithi100 ceph-mon[32599]: pgmap v7246: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:15:40.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:15:40 smithi167 ceph-mon[37140]: pgmap v7246: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:15:42.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:15:42 smithi138 ceph-mon[36954]: pgmap v7247: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:15:42.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:15:42 smithi100 ceph-mon[32599]: pgmap v7247: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:15:42.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:15:42 smithi167 ceph-mon[37140]: pgmap v7247: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:15:44.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:15:44 smithi138 ceph-mon[36954]: pgmap v7248: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:15:44.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:15:44 smithi100 ceph-mon[32599]: pgmap v7248: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:15:44.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:15:44 smithi167 ceph-mon[37140]: pgmap v7248: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:15:46.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:15:46 smithi138 ceph-mon[36954]: pgmap v7249: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:15:46.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:15:46 smithi100 ceph-mon[32599]: pgmap v7249: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:15:46.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:15:46 smithi167 ceph-mon[37140]: pgmap v7249: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:15:48.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:15:48 smithi138 ceph-mon[36954]: pgmap v7250: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:15:48.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:15:48 smithi100 ceph-mon[32599]: pgmap v7250: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:15:48.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:15:48 smithi167 ceph-mon[37140]: pgmap v7250: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:15:50.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:15:50 smithi138 ceph-mon[36954]: pgmap v7251: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:15:50.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:15:50 smithi100 ceph-mon[32599]: pgmap v7251: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:15:50.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:15:50 smithi167 ceph-mon[37140]: pgmap v7251: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:15:52.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:15:52 smithi138 ceph-mon[36954]: pgmap v7252: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:15:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:15:52 smithi100 ceph-mon[32599]: pgmap v7252: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:15:52.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:15:52 smithi167 ceph-mon[37140]: pgmap v7252: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:15:54.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:15:54.316+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:15:54.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:15:54.316+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:15:54.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:15:54.316+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:15:54.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:15:54.316+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:15:54.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:15:54.316+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:15:54.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:15:54 smithi100 ceph-mon[32599]: pgmap v7253: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:15:54.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:15:54 smithi167 ceph-mon[37140]: pgmap v7253: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:15:54.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:15:54 smithi138 ceph-mon[36954]: pgmap v7253: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:15:56.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:15:56 smithi100 ceph-mon[32599]: pgmap v7254: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:15:56.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:15:56 smithi167 ceph-mon[37140]: pgmap v7254: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:15:56.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:15:56 smithi138 ceph-mon[36954]: pgmap v7254: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:15:58.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:15:58 smithi100 ceph-mon[32599]: pgmap v7255: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:15:58.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:15:58 smithi167 ceph-mon[37140]: pgmap v7255: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:15:58.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:15:58 smithi138 ceph-mon[36954]: pgmap v7255: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:16:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:16:00 smithi100 ceph-mon[32599]: pgmap v7256: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:16:00.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:16:00 smithi167 ceph-mon[37140]: pgmap v7256: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:16:00.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:16:00 smithi138 ceph-mon[36954]: pgmap v7256: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:16:02.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:16:02 smithi100 ceph-mon[32599]: pgmap v7257: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:16:02.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:16:02 smithi167 ceph-mon[37140]: pgmap v7257: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:16:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:16:02 smithi138 ceph-mon[36954]: pgmap v7257: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:16:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:16:04 smithi100 ceph-mon[32599]: pgmap v7258: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:16:04.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:16:04 smithi167 ceph-mon[37140]: pgmap v7258: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:16:04.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:16:04 smithi138 ceph-mon[36954]: pgmap v7258: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:16:06.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:16:06 smithi100 ceph-mon[32599]: pgmap v7259: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:16:06.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:16:06 smithi167 ceph-mon[37140]: pgmap v7259: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:16:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:16:06 smithi138 ceph-mon[36954]: pgmap v7259: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:16:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:16:08 smithi100 ceph-mon[32599]: pgmap v7260: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:16:08.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:16:08 smithi100 ceph-mon[32599]: osdmap e500: 12 total, 12 up, 12 in 2024-07-26T17:16:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:16:08 smithi167 ceph-mon[37140]: pgmap v7260: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:16:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:16:08 smithi167 ceph-mon[37140]: osdmap e500: 12 total, 12 up, 12 in 2024-07-26T17:16:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:16:08 smithi138 ceph-mon[36954]: pgmap v7260: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:16:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:16:08 smithi138 ceph-mon[36954]: osdmap e500: 12 total, 12 up, 12 in 2024-07-26T17:16:09.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:16:09.332+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:16:09.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:16:09.332+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:16:09.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:16:09.332+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:16:09.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:16:09.332+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:16:09.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:16:09.332+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:16:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:16:10 smithi100 ceph-mon[32599]: pgmap v7262: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:16:10.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:16:10 smithi167 ceph-mon[37140]: pgmap v7262: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:16:10.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:16:10 smithi138 ceph-mon[36954]: pgmap v7262: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:16:12.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:16:12 smithi100 ceph-mon[32599]: pgmap v7263: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:16:12.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:16:12 smithi167 ceph-mon[37140]: pgmap v7263: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:16:12.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:16:12 smithi138 ceph-mon[36954]: pgmap v7263: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:16:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:16:14 smithi100 ceph-mon[32599]: pgmap v7264: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:16:14.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:16:14 smithi167 ceph-mon[37140]: pgmap v7264: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:16:14.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:16:14 smithi138 ceph-mon[36954]: pgmap v7264: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:16:16.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:16:16 smithi100 ceph-mon[32599]: pgmap v7265: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:16:16.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:16:16 smithi167 ceph-mon[37140]: pgmap v7265: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:16:16.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:16:16 smithi138 ceph-mon[36954]: pgmap v7265: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:16:18.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:16:18 smithi138 ceph-mon[36954]: pgmap v7266: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:16:18.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:16:18 smithi138 ceph-mon[36954]: osdmap e501: 12 total, 12 up, 12 in 2024-07-26T17:16:18.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:16:18 smithi100 ceph-mon[32599]: pgmap v7266: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:16:18.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:16:18 smithi100 ceph-mon[32599]: osdmap e501: 12 total, 12 up, 12 in 2024-07-26T17:16:18.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:16:18 smithi167 ceph-mon[37140]: pgmap v7266: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:16:18.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:16:18 smithi167 ceph-mon[37140]: osdmap e501: 12 total, 12 up, 12 in 2024-07-26T17:16:20.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:16:20 smithi138 ceph-mon[36954]: pgmap v7268: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:16:20.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:16:20 smithi100 ceph-mon[32599]: pgmap v7268: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:16:20.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:16:20 smithi167 ceph-mon[37140]: pgmap v7268: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:16:22.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:16:22 smithi138 ceph-mon[36954]: pgmap v7269: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:16:22.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:16:22 smithi100 ceph-mon[32599]: pgmap v7269: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:16:22.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:16:22 smithi167 ceph-mon[37140]: pgmap v7269: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:16:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:16:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:16:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:16:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:16:23.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:16:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:16:24.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:16:24.335+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:16:24.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:16:24.335+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:16:24.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:16:24.335+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:16:24.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:16:24.335+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:16:24.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:16:24.335+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:16:24.424 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:16:24 smithi167 ceph-mon[37140]: pgmap v7270: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:16:24.425 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:16:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:16:24.425 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:16:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:16:24.425 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:16:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:16:24.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:16:24 smithi138 ceph-mon[36954]: pgmap v7270: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:16:24.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:16:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:16:24.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:16:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:16:24.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:16:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:16:24.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:16:24 smithi100 ceph-mon[32599]: pgmap v7270: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:16:24.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:16:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:16:24.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:16:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:16:24.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:16:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:16:26.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:16:26 smithi138 ceph-mon[36954]: pgmap v7271: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:16:26.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:16:26 smithi100 ceph-mon[32599]: pgmap v7271: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:16:26.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:16:26 smithi167 ceph-mon[37140]: pgmap v7271: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:16:28.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:16:28 smithi138 ceph-mon[36954]: pgmap v7272: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:16:28.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:16:28 smithi100 ceph-mon[32599]: pgmap v7272: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:16:28.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:16:28 smithi167 ceph-mon[37140]: pgmap v7272: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:16:30.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:16:30 smithi138 ceph-mon[36954]: pgmap v7273: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:16:30.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:16:30 smithi100 ceph-mon[32599]: pgmap v7273: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:16:30.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:16:30 smithi167 ceph-mon[37140]: pgmap v7273: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:16:32.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:16:32 smithi138 ceph-mon[36954]: pgmap v7274: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:16:32.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:16:32 smithi100 ceph-mon[32599]: pgmap v7274: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:16:32.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:16:32 smithi167 ceph-mon[37140]: pgmap v7274: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:16:34.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:16:34 smithi138 ceph-mon[36954]: pgmap v7275: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:16:34.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:16:34 smithi100 ceph-mon[32599]: pgmap v7275: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:16:34.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:16:34 smithi167 ceph-mon[37140]: pgmap v7275: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:16:36.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:16:36 smithi138 ceph-mon[36954]: pgmap v7276: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:16:36.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:16:36 smithi100 ceph-mon[32599]: pgmap v7276: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:16:36.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:16:36 smithi167 ceph-mon[37140]: pgmap v7276: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:16:38.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:16:38 smithi138 ceph-mon[36954]: pgmap v7277: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:16:38.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:16:38 smithi100 ceph-mon[32599]: pgmap v7277: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:16:38.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:16:38 smithi167 ceph-mon[37140]: pgmap v7277: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:16:39.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:16:39.352+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:16:39.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:16:39.352+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:16:39.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:16:39.352+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:16:39.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:16:39.352+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:16:39.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:16:39.353+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:16:40.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:16:40 smithi138 ceph-mon[36954]: pgmap v7278: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T17:16:40.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:16:40 smithi100 ceph-mon[32599]: pgmap v7278: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T17:16:40.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:16:40 smithi167 ceph-mon[37140]: pgmap v7278: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T17:16:42.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:16:42 smithi138 ceph-mon[36954]: pgmap v7279: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:16:42.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:16:42 smithi100 ceph-mon[32599]: pgmap v7279: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:16:42.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:16:42 smithi167 ceph-mon[37140]: pgmap v7279: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:16:44.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:16:44 smithi138 ceph-mon[36954]: pgmap v7280: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:16:44.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:16:44 smithi100 ceph-mon[32599]: pgmap v7280: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:16:44.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:16:44 smithi167 ceph-mon[37140]: pgmap v7280: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:16:46.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:16:46 smithi100 ceph-mon[32599]: pgmap v7281: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:16:46.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:16:46 smithi167 ceph-mon[37140]: pgmap v7281: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:16:46.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:16:46 smithi138 ceph-mon[36954]: pgmap v7281: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:16:48.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:16:48 smithi100 ceph-mon[32599]: pgmap v7282: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:16:48.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:16:48 smithi167 ceph-mon[37140]: pgmap v7282: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:16:48.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:16:48 smithi138 ceph-mon[36954]: pgmap v7282: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:16:50.493 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:16:50 smithi138 ceph-mon[36954]: pgmap v7283: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:16:50.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:16:50 smithi100 ceph-mon[32599]: pgmap v7283: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:16:50.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:16:50 smithi167 ceph-mon[37140]: pgmap v7283: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:16:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:16:52 smithi100 ceph-mon[32599]: pgmap v7284: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:16:52.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:16:52 smithi167 ceph-mon[37140]: pgmap v7284: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:16:52.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:16:52 smithi138 ceph-mon[36954]: pgmap v7284: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:16:54.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:16:54.367+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:16:54.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:16:54.368+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:16:54.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:16:54.368+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:16:54.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:16:54.368+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:16:54.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:16:54.368+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:16:54.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:16:54 smithi100 ceph-mon[32599]: pgmap v7285: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:16:54.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:16:54 smithi167 ceph-mon[37140]: pgmap v7285: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:16:54.694 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:16:54 smithi138 ceph-mon[36954]: pgmap v7285: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:16:56.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:16:56 smithi100 ceph-mon[32599]: pgmap v7286: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:16:56.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:16:56 smithi167 ceph-mon[37140]: pgmap v7286: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:16:56.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:16:56 smithi138 ceph-mon[36954]: pgmap v7286: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:16:58.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:16:58 smithi100 ceph-mon[32599]: pgmap v7287: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:16:58.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:16:58 smithi167 ceph-mon[37140]: pgmap v7287: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:16:58.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:16:58 smithi138 ceph-mon[36954]: pgmap v7287: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:17:00.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:17:00 smithi100 ceph-mon[32599]: pgmap v7288: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:17:00.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:17:00 smithi167 ceph-mon[37140]: pgmap v7288: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:17:00.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:17:00 smithi138 ceph-mon[36954]: pgmap v7288: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:17:02.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:17:02 smithi100 ceph-mon[32599]: pgmap v7289: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:17:02.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:17:02 smithi167 ceph-mon[37140]: pgmap v7289: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:17:02.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:17:02 smithi138 ceph-mon[36954]: pgmap v7289: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:17:04.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:17:04 smithi100 ceph-mon[32599]: pgmap v7290: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:17:04.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:17:04 smithi167 ceph-mon[37140]: pgmap v7290: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:17:04.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:17:04 smithi138 ceph-mon[36954]: pgmap v7290: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:17:06.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:17:06 smithi100 ceph-mon[32599]: pgmap v7291: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:17:06.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:17:06 smithi167 ceph-mon[37140]: pgmap v7291: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:17:06.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:17:06 smithi138 ceph-mon[36954]: pgmap v7291: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:17:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:17:08 smithi167 ceph-mon[37140]: pgmap v7292: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:17:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:17:08 smithi167 ceph-mon[37140]: osdmap e502: 12 total, 12 up, 12 in 2024-07-26T17:17:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:17:08 smithi138 ceph-mon[36954]: pgmap v7292: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:17:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:17:08 smithi138 ceph-mon[36954]: osdmap e502: 12 total, 12 up, 12 in 2024-07-26T17:17:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:17:08 smithi100 ceph-mon[32599]: pgmap v7292: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:17:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:17:08 smithi100 ceph-mon[32599]: osdmap e502: 12 total, 12 up, 12 in 2024-07-26T17:17:09.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:17:09.383+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:17:09.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:17:09.384+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:17:09.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:17:09.384+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:17:09.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:17:09.384+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:17:09.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:17:09.384+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:17:10.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:17:10 smithi167 ceph-mon[37140]: pgmap v7294: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:17:10.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:17:10 smithi138 ceph-mon[36954]: pgmap v7294: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:17:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:17:10 smithi100 ceph-mon[32599]: pgmap v7294: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:17:12.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:17:12 smithi167 ceph-mon[37140]: pgmap v7295: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:17:12.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:17:12 smithi138 ceph-mon[36954]: pgmap v7295: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:17:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:17:12 smithi100 ceph-mon[32599]: pgmap v7295: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:17:14.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:17:14 smithi167 ceph-mon[37140]: pgmap v7296: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:17:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:17:14 smithi138 ceph-mon[36954]: pgmap v7296: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:17:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:17:14 smithi100 ceph-mon[32599]: pgmap v7296: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:17:16.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:17:16 smithi167 ceph-mon[37140]: pgmap v7297: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:17:16.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:17:16 smithi138 ceph-mon[36954]: pgmap v7297: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:17:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:17:16 smithi100 ceph-mon[32599]: pgmap v7297: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:17:18.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:17:18 smithi138 ceph-mon[36954]: pgmap v7298: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:17:18.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:17:18 smithi138 ceph-mon[36954]: osdmap e503: 12 total, 12 up, 12 in 2024-07-26T17:17:18.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:17:18 smithi100 ceph-mon[32599]: pgmap v7298: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:17:18.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:17:18 smithi100 ceph-mon[32599]: osdmap e503: 12 total, 12 up, 12 in 2024-07-26T17:17:18.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:17:18 smithi167 ceph-mon[37140]: pgmap v7298: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:17:18.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:17:18 smithi167 ceph-mon[37140]: osdmap e503: 12 total, 12 up, 12 in 2024-07-26T17:17:20.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:17:20 smithi138 ceph-mon[36954]: pgmap v7300: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:17:20.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:17:20 smithi100 ceph-mon[32599]: pgmap v7300: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:17:20.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:17:20 smithi167 ceph-mon[37140]: pgmap v7300: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:17:22.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:17:22 smithi138 ceph-mon[36954]: pgmap v7301: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:17:22.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:17:22 smithi100 ceph-mon[32599]: pgmap v7301: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:17:22.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:17:22 smithi167 ceph-mon[37140]: pgmap v7301: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:17:24.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:17:24.389+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:17:24.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:17:24.390+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:17:24.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:17:24.390+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:17:24.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:17:24.390+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:17:24.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:17:24.390+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:17:24.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:17:24 smithi138 ceph-mon[36954]: pgmap v7302: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:17:24.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:17:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:17:24.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:17:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:17:24.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:17:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:17:24.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:17:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:17:24.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:17:24 smithi100 ceph-mon[32599]: pgmap v7302: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:17:24.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:17:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:17:24.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:17:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:17:24.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:17:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:17:24.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:17:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:17:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:17:24 smithi167 ceph-mon[37140]: pgmap v7302: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:17:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:17:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:17:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:17:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:17:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:17:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:17:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:17:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:17:26.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:17:26 smithi138 ceph-mon[36954]: pgmap v7303: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:17:26.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:17:26 smithi100 ceph-mon[32599]: pgmap v7303: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:17:26.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:17:26 smithi167 ceph-mon[37140]: pgmap v7303: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:17:28.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:17:28 smithi138 ceph-mon[36954]: pgmap v7304: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:17:28.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:17:28 smithi100 ceph-mon[32599]: pgmap v7304: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:17:28.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:17:28 smithi167 ceph-mon[37140]: pgmap v7304: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:17:30.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:17:30 smithi138 ceph-mon[36954]: pgmap v7305: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:17:30.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:17:30 smithi100 ceph-mon[32599]: pgmap v7305: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:17:30.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:17:30 smithi167 ceph-mon[37140]: pgmap v7305: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:17:32.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:17:32 smithi138 ceph-mon[36954]: pgmap v7306: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:17:32.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:17:32 smithi100 ceph-mon[32599]: pgmap v7306: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:17:32.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:17:32 smithi167 ceph-mon[37140]: pgmap v7306: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:17:34.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:17:34 smithi138 ceph-mon[36954]: pgmap v7307: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:17:34.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:17:34 smithi100 ceph-mon[32599]: pgmap v7307: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:17:34.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:17:34 smithi167 ceph-mon[37140]: pgmap v7307: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:17:36.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:17:36 smithi138 ceph-mon[36954]: pgmap v7308: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:17:36.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:17:36 smithi100 ceph-mon[32599]: pgmap v7308: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:17:36.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:17:36 smithi167 ceph-mon[37140]: pgmap v7308: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:17:38.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:17:38 smithi100 ceph-mon[32599]: pgmap v7309: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:17:38.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:17:38 smithi167 ceph-mon[37140]: pgmap v7309: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:17:38.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:17:38 smithi138 ceph-mon[36954]: pgmap v7309: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:17:39.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:17:39.405+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:17:39.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:17:39.405+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:17:39.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:17:39.405+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:17:39.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:17:39.405+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:17:39.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:17:39.405+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:17:40.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:17:40 smithi100 ceph-mon[32599]: pgmap v7310: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T17:17:40.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:17:40 smithi167 ceph-mon[37140]: pgmap v7310: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T17:17:40.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:17:40 smithi138 ceph-mon[36954]: pgmap v7310: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T17:17:42.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:17:42 smithi100 ceph-mon[32599]: pgmap v7311: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:17:42.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:17:42 smithi167 ceph-mon[37140]: pgmap v7311: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:17:42.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:17:42 smithi138 ceph-mon[36954]: pgmap v7311: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:17:44.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:17:44 smithi100 ceph-mon[32599]: pgmap v7312: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T17:17:44.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:17:44 smithi167 ceph-mon[37140]: pgmap v7312: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T17:17:44.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:17:44 smithi138 ceph-mon[36954]: pgmap v7312: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T17:17:46.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:17:46 smithi100 ceph-mon[32599]: pgmap v7313: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:17:46.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:17:46 smithi167 ceph-mon[37140]: pgmap v7313: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:17:46.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:17:46 smithi138 ceph-mon[36954]: pgmap v7313: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:17:48.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:17:48 smithi100 ceph-mon[32599]: pgmap v7314: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:17:48.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:17:48 smithi167 ceph-mon[37140]: pgmap v7314: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:17:48.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:17:48 smithi138 ceph-mon[36954]: pgmap v7314: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:17:50.493 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:17:50 smithi138 ceph-mon[36954]: pgmap v7315: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:17:50.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:17:50 smithi100 ceph-mon[32599]: pgmap v7315: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:17:50.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:17:50 smithi167 ceph-mon[37140]: pgmap v7315: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:17:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:17:52 smithi100 ceph-mon[32599]: pgmap v7316: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:17:52.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:17:52 smithi167 ceph-mon[37140]: pgmap v7316: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:17:52.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:17:52 smithi138 ceph-mon[36954]: pgmap v7316: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:17:54.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:17:54.422+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:17:54.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:17:54.422+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:17:54.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:17:54.422+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:17:54.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:17:54.422+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:17:54.424 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:17:54.422+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:17:54.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:17:54 smithi100 ceph-mon[32599]: pgmap v7317: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:17:54.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:17:54 smithi167 ceph-mon[37140]: pgmap v7317: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:17:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:17:54 smithi138 ceph-mon[36954]: pgmap v7317: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:17:56.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:17:56 smithi100 ceph-mon[32599]: pgmap v7318: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:17:56.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:17:56 smithi167 ceph-mon[37140]: pgmap v7318: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:17:56.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:17:56 smithi138 ceph-mon[36954]: pgmap v7318: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:17:58.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:17:58 smithi100 ceph-mon[32599]: pgmap v7319: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:17:58.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:17:58 smithi167 ceph-mon[37140]: pgmap v7319: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:17:58.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:17:58 smithi138 ceph-mon[36954]: pgmap v7319: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:18:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:18:00 smithi100 ceph-mon[32599]: pgmap v7320: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:18:00.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:18:00 smithi167 ceph-mon[37140]: pgmap v7320: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:18:00.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:18:00 smithi138 ceph-mon[36954]: pgmap v7320: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:18:02.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:18:02 smithi100 ceph-mon[32599]: pgmap v7321: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:18:02.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:18:02 smithi167 ceph-mon[37140]: pgmap v7321: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:18:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:18:02 smithi138 ceph-mon[36954]: pgmap v7321: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:18:03.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:18:03.041+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 msgr2=0x7fe5940c0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).tick idle (900000505) for more than 900000000 us, fault. 2024-07-26T17:18:03.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:18:03.041+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe59000be60 tx=0x7fe59008a040 comp rx=0 tx=0).stop 2024-07-26T17:18:03.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:18:03.041+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 msgr2=0x7fe5940c0480 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-07-26T17:18:03.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:18:03.041+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T17:18:03.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:18:03.041+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 0x7fe5940b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T17:18:03.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:18:03.041+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 0x7fe5940b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:18:03.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:18:03.042+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 0x7fe5940b5b30 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe590005d00 tx=0x7fe590097040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T17:18:04.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:18:04 smithi100 ceph-mon[32599]: pgmap v7322: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:18:04.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:18:04 smithi167 ceph-mon[37140]: pgmap v7322: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:18:04.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:18:04 smithi138 ceph-mon[36954]: pgmap v7322: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:18:04.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:18:04.802+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T17:18:04.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:18:04.802+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c004740 con 0x7fe5a81087a0 2024-07-26T17:18:04.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:18:04.803+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 100 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c070 con 0x7fe5a81087a0 2024-07-26T17:18:04.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:18:04.803+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 100 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T17:18:06.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:18:06 smithi100 ceph-mon[32599]: pgmap v7323: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:18:06.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:18:06 smithi167 ceph-mon[37140]: pgmap v7323: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:18:06.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:18:06 smithi138 ceph-mon[36954]: pgmap v7323: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:18:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:18:08 smithi167 ceph-mon[37140]: pgmap v7324: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:18:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:18:08 smithi167 ceph-mon[37140]: osdmap e504: 12 total, 12 up, 12 in 2024-07-26T17:18:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:18:08 smithi138 ceph-mon[36954]: pgmap v7324: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:18:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:18:08 smithi138 ceph-mon[36954]: osdmap e504: 12 total, 12 up, 12 in 2024-07-26T17:18:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:18:08 smithi100 ceph-mon[32599]: pgmap v7324: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:18:08.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:18:08 smithi100 ceph-mon[32599]: osdmap e504: 12 total, 12 up, 12 in 2024-07-26T17:18:09.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:18:09.429+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:18:09.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:18:09.429+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:18:09.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:18:09.429+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:18:09.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:18:09.429+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:18:09.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:18:09.429+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:18:10.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:18:10 smithi167 ceph-mon[37140]: pgmap v7326: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:18:10.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:18:10 smithi138 ceph-mon[36954]: pgmap v7326: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:18:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:18:10 smithi100 ceph-mon[32599]: pgmap v7326: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:18:12.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:18:12 smithi167 ceph-mon[37140]: pgmap v7327: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:18:12.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:18:12 smithi138 ceph-mon[36954]: pgmap v7327: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:18:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:18:12 smithi100 ceph-mon[32599]: pgmap v7327: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:18:14.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:18:14 smithi167 ceph-mon[37140]: pgmap v7328: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:18:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:18:14 smithi138 ceph-mon[36954]: pgmap v7328: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:18:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:18:14 smithi100 ceph-mon[32599]: pgmap v7328: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:18:16.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:18:16 smithi167 ceph-mon[37140]: pgmap v7329: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:18:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:18:16 smithi138 ceph-mon[36954]: pgmap v7329: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:18:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:18:16 smithi100 ceph-mon[32599]: pgmap v7329: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:18:18.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:18:18 smithi138 ceph-mon[36954]: pgmap v7330: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:18:18.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:18:18 smithi138 ceph-mon[36954]: osdmap e505: 12 total, 12 up, 12 in 2024-07-26T17:18:18.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:18:18 smithi100 ceph-mon[32599]: pgmap v7330: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:18:18.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:18:18 smithi100 ceph-mon[32599]: osdmap e505: 12 total, 12 up, 12 in 2024-07-26T17:18:18.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:18:18 smithi167 ceph-mon[37140]: pgmap v7330: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:18:18.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:18:18 smithi167 ceph-mon[37140]: osdmap e505: 12 total, 12 up, 12 in 2024-07-26T17:18:20.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:18:20 smithi138 ceph-mon[36954]: pgmap v7332: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:18:20.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:18:20 smithi100 ceph-mon[32599]: pgmap v7332: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:18:20.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:18:20 smithi167 ceph-mon[37140]: pgmap v7332: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:18:22.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:18:22 smithi138 ceph-mon[36954]: pgmap v7333: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:18:22.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:18:22 smithi100 ceph-mon[32599]: pgmap v7333: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:18:22.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:18:22 smithi167 ceph-mon[37140]: pgmap v7333: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:18:24.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:18:24 smithi138 ceph-mon[36954]: pgmap v7334: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:18:24.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:18:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:18:24.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:18:24.444+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:18:24.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:18:24 smithi100 ceph-mon[32599]: pgmap v7334: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:18:24.446 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:18:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:18:24.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:18:24.444+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:18:24.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:18:24.444+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:18:24.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:18:24.444+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:18:24.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:18:24.444+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:18:24.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:18:24 smithi167 ceph-mon[37140]: pgmap v7334: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:18:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:18:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:18:25.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:18:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:18:25.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:18:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:18:25.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:18:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:18:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:18:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:18:25.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:18:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:18:25.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:18:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:18:25.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:18:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:18:25.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:18:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:18:25.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:18:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:18:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:18:26 smithi100 ceph-mon[32599]: pgmap v7335: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:18:26.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:18:26 smithi167 ceph-mon[37140]: pgmap v7335: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:18:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:18:26 smithi138 ceph-mon[36954]: pgmap v7335: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:18:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:18:28 smithi100 ceph-mon[32599]: pgmap v7336: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:18:28.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:18:28 smithi167 ceph-mon[37140]: pgmap v7336: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:18:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:18:28 smithi138 ceph-mon[36954]: pgmap v7336: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:18:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:18:30 smithi100 ceph-mon[32599]: pgmap v7337: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:18:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:18:30 smithi167 ceph-mon[37140]: pgmap v7337: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:18:30.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:18:30 smithi138 ceph-mon[36954]: pgmap v7337: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:18:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:18:32 smithi100 ceph-mon[32599]: pgmap v7338: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:18:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:18:32 smithi167 ceph-mon[37140]: pgmap v7338: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:18:32.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:18:32 smithi138 ceph-mon[36954]: pgmap v7338: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:18:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:18:34 smithi100 ceph-mon[32599]: pgmap v7339: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:18:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:18:34 smithi167 ceph-mon[37140]: pgmap v7339: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:18:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:18:34 smithi138 ceph-mon[36954]: pgmap v7339: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:18:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:18:36 smithi100 ceph-mon[32599]: pgmap v7340: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:18:36.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:18:36 smithi167 ceph-mon[37140]: pgmap v7340: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:18:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:18:36 smithi138 ceph-mon[36954]: pgmap v7340: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:18:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:18:38 smithi100 ceph-mon[32599]: pgmap v7341: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:18:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:18:38 smithi167 ceph-mon[37140]: pgmap v7341: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:18:38.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:18:38 smithi138 ceph-mon[36954]: pgmap v7341: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:18:39.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:18:39.449+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:18:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:18:39.449+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:18:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:18:39.449+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:18:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:18:39.449+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:18:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:18:39.449+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:18:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:18:40 smithi100 ceph-mon[32599]: pgmap v7342: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:18:40.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:18:40 smithi167 ceph-mon[37140]: pgmap v7342: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:18:40.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:18:40 smithi138 ceph-mon[36954]: pgmap v7342: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:18:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:18:42 smithi100 ceph-mon[32599]: pgmap v7343: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:18:42.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:18:42 smithi167 ceph-mon[37140]: pgmap v7343: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:18:42.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:18:42 smithi138 ceph-mon[36954]: pgmap v7343: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:18:44.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:18:44 smithi100 ceph-mon[32599]: pgmap v7344: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:18:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:18:44 smithi167 ceph-mon[37140]: pgmap v7344: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:18:44.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:18:44 smithi138 ceph-mon[36954]: pgmap v7344: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:18:46.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:18:46 smithi100 ceph-mon[32599]: pgmap v7345: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:18:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:18:46 smithi167 ceph-mon[37140]: pgmap v7345: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:18:46.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:18:46 smithi138 ceph-mon[36954]: pgmap v7345: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:18:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:18:48 smithi100 ceph-mon[32599]: pgmap v7346: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:18:48.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:18:48 smithi167 ceph-mon[37140]: pgmap v7346: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:18:48.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:18:48 smithi138 ceph-mon[36954]: pgmap v7346: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:18:50.743 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:18:50 smithi138 ceph-mon[36954]: pgmap v7347: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:18:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:18:50 smithi100 ceph-mon[32599]: pgmap v7347: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:18:50.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:18:50 smithi167 ceph-mon[37140]: pgmap v7347: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:18:52.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:18:52 smithi100 ceph-mon[32599]: pgmap v7348: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:18:52.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:18:52 smithi167 ceph-mon[37140]: pgmap v7348: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:18:52.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:18:52 smithi138 ceph-mon[36954]: pgmap v7348: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:18:54.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:18:54.465+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:18:54.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:18:54.465+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:18:54.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:18:54.465+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:18:54.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:18:54.465+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:18:54.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:18:54.465+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:18:54.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:18:54 smithi100 ceph-mon[32599]: pgmap v7349: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:18:54.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:18:54 smithi167 ceph-mon[37140]: pgmap v7349: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:18:54.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:18:54 smithi138 ceph-mon[36954]: pgmap v7349: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:18:56.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:18:56 smithi167 ceph-mon[37140]: pgmap v7350: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:18:56.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:18:56 smithi138 ceph-mon[36954]: pgmap v7350: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:18:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:18:56 smithi100 ceph-mon[32599]: pgmap v7350: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:18:58.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:18:58 smithi167 ceph-mon[37140]: pgmap v7351: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:18:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:18:58 smithi138 ceph-mon[36954]: pgmap v7351: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:18:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:18:58 smithi100 ceph-mon[32599]: pgmap v7351: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:19:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:00 smithi167 ceph-mon[37140]: pgmap v7352: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:19:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:00 smithi138 ceph-mon[36954]: pgmap v7352: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:19:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:00 smithi100 ceph-mon[32599]: pgmap v7352: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:19:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:02 smithi167 ceph-mon[37140]: pgmap v7353: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:19:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:02 smithi138 ceph-mon[36954]: pgmap v7353: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:19:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:02 smithi100 ceph-mon[32599]: pgmap v7353: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:19:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:04 smithi167 ceph-mon[37140]: pgmap v7354: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:19:04.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:04 smithi138 ceph-mon[36954]: pgmap v7354: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:19:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:04 smithi100 ceph-mon[32599]: pgmap v7354: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:19:06.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:06 smithi167 ceph-mon[37140]: pgmap v7355: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:19:06.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:06 smithi138 ceph-mon[36954]: pgmap v7355: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:19:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:06 smithi100 ceph-mon[32599]: pgmap v7355: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:19:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:08 smithi167 ceph-mon[37140]: pgmap v7356: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:19:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:08 smithi167 ceph-mon[37140]: osdmap e506: 12 total, 12 up, 12 in 2024-07-26T17:19:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:08 smithi138 ceph-mon[36954]: pgmap v7356: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:19:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:08 smithi138 ceph-mon[36954]: osdmap e506: 12 total, 12 up, 12 in 2024-07-26T17:19:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:08 smithi100 ceph-mon[32599]: pgmap v7356: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:19:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:08 smithi100 ceph-mon[32599]: osdmap e506: 12 total, 12 up, 12 in 2024-07-26T17:19:09.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:19:09.474+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:19:09.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:19:09.475+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:19:09.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:19:09.475+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:19:09.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:19:09.475+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:19:09.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:19:09.475+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:19:10.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:10 smithi138 ceph-mon[36954]: pgmap v7358: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:19:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:10 smithi100 ceph-mon[32599]: pgmap v7358: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:19:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:10 smithi167 ceph-mon[37140]: pgmap v7358: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:19:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:12 smithi138 ceph-mon[36954]: pgmap v7359: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:19:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:12 smithi100 ceph-mon[32599]: pgmap v7359: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:19:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:12 smithi167 ceph-mon[37140]: pgmap v7359: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:19:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:14 smithi138 ceph-mon[36954]: pgmap v7360: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:19:14.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:14 smithi100 ceph-mon[32599]: pgmap v7360: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:19:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:14 smithi167 ceph-mon[37140]: pgmap v7360: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:19:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:16 smithi138 ceph-mon[36954]: pgmap v7361: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:19:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:16 smithi100 ceph-mon[32599]: pgmap v7361: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:19:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:16 smithi167 ceph-mon[37140]: pgmap v7361: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:19:18.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:18 smithi138 ceph-mon[36954]: pgmap v7362: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:19:18.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:18 smithi138 ceph-mon[36954]: osdmap e507: 12 total, 12 up, 12 in 2024-07-26T17:19:18.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:18 smithi100 ceph-mon[32599]: pgmap v7362: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:19:18.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:18 smithi100 ceph-mon[32599]: osdmap e507: 12 total, 12 up, 12 in 2024-07-26T17:19:18.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:18 smithi167 ceph-mon[37140]: pgmap v7362: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:19:18.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:18 smithi167 ceph-mon[37140]: osdmap e507: 12 total, 12 up, 12 in 2024-07-26T17:19:20.497 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:20 smithi100 ceph-mon[32599]: pgmap v7364: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:19:20.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:20 smithi167 ceph-mon[37140]: pgmap v7364: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:19:20.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:20 smithi138 ceph-mon[36954]: pgmap v7364: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:19:22.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:22 smithi100 ceph-mon[32599]: pgmap v7365: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:19:22.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:22 smithi167 ceph-mon[37140]: pgmap v7365: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:19:22.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:22 smithi138 ceph-mon[36954]: pgmap v7365: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:19:24.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:19:24.486+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:19:24.489 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:24 smithi100 ceph-mon[32599]: pgmap v7366: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:19:24.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:19:24.487+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:19:24.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:19:24.487+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:19:24.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:19:24.487+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:19:24.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:19:24.487+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:19:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:24 smithi167 ceph-mon[37140]: pgmap v7366: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:19:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:24 smithi138 ceph-mon[36954]: pgmap v7366: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:19:25.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:19:25.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:19:25.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:19:25.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:19:25.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:19:25.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:19:25.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:19:25.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:19:25.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:19:25.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:19:25.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:19:25.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:19:25.497 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:19:25.497 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:19:25.497 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:19:25.497 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:19:25.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:19:25.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:19:25.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:19:25.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:19:25.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:19:25.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:19:25.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:19:25.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:19:25.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:19:25.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:19:25.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:19:25.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:19:25.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:19:25.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:19:25.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:19:25.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:19:25.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:19:25.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:19:25.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:19:25.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:19:25.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:19:25.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:19:25.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:19:25.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:19:25.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:19:25.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:19:25.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:19:25.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:19:25.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:19:25.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:19:25.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:19:25.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:19:26.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:26 smithi100 ceph-mon[32599]: pgmap v7367: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:19:26.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:26 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T17:19:26.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:26 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T17:19:26.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:26 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T17:19:26.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:26 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T17:19:26.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:26 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T17:19:26.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:26 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T17:19:26.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:26 smithi167 ceph-mon[37140]: pgmap v7367: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:19:26.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:26 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T17:19:26.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:26 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T17:19:26.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:26 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T17:19:26.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:26 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T17:19:26.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:26 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T17:19:26.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:26 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T17:19:26.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:26 smithi138 ceph-mon[36954]: pgmap v7367: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:19:26.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:26 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T17:19:26.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:26 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T17:19:26.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:26 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T17:19:26.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:26 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T17:19:26.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:26 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T17:19:26.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:26 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T17:19:28.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:28 smithi100 ceph-mon[32599]: pgmap v7368: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:19:28.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:28 smithi167 ceph-mon[37140]: pgmap v7368: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:19:28.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:28 smithi138 ceph-mon[36954]: pgmap v7368: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:19:30.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:30 smithi100 ceph-mon[32599]: pgmap v7369: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:19:30.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:30 smithi167 ceph-mon[37140]: pgmap v7369: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:19:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:30 smithi138 ceph-mon[36954]: pgmap v7369: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:19:32.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:32 smithi100 ceph-mon[32599]: pgmap v7370: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:19:32.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:32 smithi167 ceph-mon[37140]: pgmap v7370: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:19:32.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:32 smithi138 ceph-mon[36954]: pgmap v7370: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:19:34.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:34 smithi100 ceph-mon[32599]: pgmap v7371: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:19:34.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:34 smithi167 ceph-mon[37140]: pgmap v7371: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:19:34.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:34 smithi138 ceph-mon[36954]: pgmap v7371: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:19:36.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:36 smithi100 ceph-mon[32599]: pgmap v7372: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:19:36.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:36 smithi167 ceph-mon[37140]: pgmap v7372: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:19:36.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:36 smithi138 ceph-mon[36954]: pgmap v7372: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:19:38.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:38 smithi100 ceph-mon[32599]: pgmap v7373: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:19:38.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:38 smithi167 ceph-mon[37140]: pgmap v7373: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:19:38.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:38 smithi138 ceph-mon[36954]: pgmap v7373: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:19:39.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:19:39.502+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:19:39.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:19:39.503+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:19:39.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:19:39.503+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:19:39.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:19:39.503+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:19:39.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:19:39.503+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:19:40.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:40 smithi100 ceph-mon[32599]: pgmap v7374: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:19:40.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:40 smithi167 ceph-mon[37140]: pgmap v7374: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:19:40.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:40 smithi138 ceph-mon[36954]: pgmap v7374: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:19:42.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:42 smithi100 ceph-mon[32599]: pgmap v7375: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:19:42.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:42 smithi167 ceph-mon[37140]: pgmap v7375: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:19:42.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:42 smithi138 ceph-mon[36954]: pgmap v7375: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:19:44.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:44 smithi100 ceph-mon[32599]: pgmap v7376: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:19:44.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:44 smithi167 ceph-mon[37140]: pgmap v7376: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:19:44.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:44 smithi138 ceph-mon[36954]: pgmap v7376: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:19:46.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:46 smithi100 ceph-mon[32599]: pgmap v7377: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:19:46.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:46 smithi167 ceph-mon[37140]: pgmap v7377: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:19:46.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:46 smithi138 ceph-mon[36954]: pgmap v7377: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:19:48.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:48 smithi100 ceph-mon[32599]: pgmap v7378: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:19:48.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:48 smithi167 ceph-mon[37140]: pgmap v7378: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:19:48.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:48 smithi138 ceph-mon[36954]: pgmap v7378: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:19:50.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:50 smithi167 ceph-mon[37140]: pgmap v7379: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:19:50.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:50 smithi138 ceph-mon[36954]: pgmap v7379: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:19:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:50 smithi100 ceph-mon[32599]: pgmap v7379: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:19:52.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:52 smithi167 ceph-mon[37140]: pgmap v7380: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:19:52.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:52 smithi138 ceph-mon[36954]: pgmap v7380: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:19:52.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:52 smithi100 ceph-mon[32599]: pgmap v7380: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:19:54.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:19:54.519+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:19:54.521 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:54 smithi100 ceph-mon[32599]: pgmap v7381: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:19:54.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:19:54.519+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:19:54.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:19:54.519+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:19:54.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:19:54.519+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:19:54.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:19:54.519+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:19:54.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:54 smithi167 ceph-mon[37140]: pgmap v7381: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:19:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:54 smithi138 ceph-mon[36954]: pgmap v7381: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:19:56.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:56 smithi167 ceph-mon[37140]: pgmap v7382: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:19:56.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:56 smithi138 ceph-mon[36954]: pgmap v7382: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:19:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:56 smithi100 ceph-mon[32599]: pgmap v7382: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:19:58.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:19:58 smithi167 ceph-mon[37140]: pgmap v7383: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:19:58.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:19:58 smithi138 ceph-mon[36954]: pgmap v7383: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:19:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:19:58 smithi100 ceph-mon[32599]: pgmap v7383: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:20:00.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:20:00 smithi167 ceph-mon[37140]: pgmap v7384: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:20:00.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:20:00 smithi167 ceph-mon[37140]: overall HEALTH_OK 2024-07-26T17:20:00.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:20:00 smithi138 ceph-mon[36954]: pgmap v7384: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:20:00.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:20:00 smithi138 ceph-mon[36954]: overall HEALTH_OK 2024-07-26T17:20:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:20:00 smithi100 ceph-mon[32599]: pgmap v7384: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:20:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:20:00 smithi100 ceph-mon[32599]: overall HEALTH_OK 2024-07-26T17:20:02.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:20:02 smithi167 ceph-mon[37140]: pgmap v7385: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:20:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:20:02 smithi138 ceph-mon[36954]: pgmap v7385: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:20:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:20:02 smithi100 ceph-mon[32599]: pgmap v7385: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:20:04.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:20:04 smithi167 ceph-mon[37140]: pgmap v7386: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:20:04.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:20:04 smithi138 ceph-mon[36954]: pgmap v7386: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:20:04.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:20:04 smithi100 ceph-mon[32599]: pgmap v7386: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:20:06.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:20:06 smithi167 ceph-mon[37140]: pgmap v7387: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:20:06.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:20:06 smithi138 ceph-mon[36954]: pgmap v7387: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:20:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:20:06 smithi100 ceph-mon[32599]: pgmap v7387: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:20:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:20:08 smithi138 ceph-mon[36954]: pgmap v7388: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:20:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:20:08 smithi138 ceph-mon[36954]: osdmap e508: 12 total, 12 up, 12 in 2024-07-26T17:20:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:20:08 smithi100 ceph-mon[32599]: pgmap v7388: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:20:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:20:08 smithi100 ceph-mon[32599]: osdmap e508: 12 total, 12 up, 12 in 2024-07-26T17:20:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:20:08 smithi167 ceph-mon[37140]: pgmap v7388: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:20:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:20:08 smithi167 ceph-mon[37140]: osdmap e508: 12 total, 12 up, 12 in 2024-07-26T17:20:09.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:20:09.535+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:20:09.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:20:09.535+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:20:09.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:20:09.535+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:20:09.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:20:09.535+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:20:09.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:20:09.535+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:20:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:20:10 smithi138 ceph-mon[36954]: pgmap v7390: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:20:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:20:10 smithi100 ceph-mon[32599]: pgmap v7390: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:20:10.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:20:10 smithi167 ceph-mon[37140]: pgmap v7390: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:20:12.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:20:12 smithi138 ceph-mon[36954]: pgmap v7391: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:20:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:20:12 smithi100 ceph-mon[32599]: pgmap v7391: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:20:12.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:20:12 smithi167 ceph-mon[37140]: pgmap v7391: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:20:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:20:14 smithi138 ceph-mon[36954]: pgmap v7392: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:20:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:20:14 smithi100 ceph-mon[32599]: pgmap v7392: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:20:14.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:20:14 smithi167 ceph-mon[37140]: pgmap v7392: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:20:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:20:16 smithi138 ceph-mon[36954]: pgmap v7393: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:20:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:20:16 smithi100 ceph-mon[32599]: pgmap v7393: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:20:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:20:16 smithi167 ceph-mon[37140]: pgmap v7393: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:20:18.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:20:18 smithi100 ceph-mon[32599]: pgmap v7394: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:20:18.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:20:18 smithi100 ceph-mon[32599]: osdmap e509: 12 total, 12 up, 12 in 2024-07-26T17:20:18.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:20:18 smithi167 ceph-mon[37140]: pgmap v7394: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:20:18.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:20:18 smithi167 ceph-mon[37140]: osdmap e509: 12 total, 12 up, 12 in 2024-07-26T17:20:18.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:20:18 smithi138 ceph-mon[36954]: pgmap v7394: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:20:18.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:20:18 smithi138 ceph-mon[36954]: osdmap e509: 12 total, 12 up, 12 in 2024-07-26T17:20:20.493 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:20:20 smithi138 ceph-mon[36954]: pgmap v7396: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:20:20.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:20:20 smithi100 ceph-mon[32599]: pgmap v7396: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:20:20.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:20:20 smithi167 ceph-mon[37140]: pgmap v7396: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:20:22.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:20:22 smithi100 ceph-mon[32599]: pgmap v7397: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:20:22.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:20:22 smithi167 ceph-mon[37140]: pgmap v7397: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:20:22.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:20:22 smithi138 ceph-mon[36954]: pgmap v7397: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:20:24.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:20:24 smithi100 ceph-mon[32599]: pgmap v7398: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:20:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:20:24 smithi167 ceph-mon[37140]: pgmap v7398: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:20:24.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:20:24.548+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:20:24.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:20:24.548+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:20:24.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:20:24.548+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:20:24.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:20:24.548+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:20:24.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:20:24.548+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:20:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:20:24 smithi138 ceph-mon[36954]: pgmap v7398: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:20:25.391 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:20:25 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:20:25.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:20:25 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:20:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:20:25 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:20:26.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:20:26 smithi100 ceph-mon[32599]: pgmap v7399: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:20:26.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:20:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:20:26.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:20:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:20:26.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:20:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:20:26.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:20:26 smithi167 ceph-mon[37140]: pgmap v7399: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:20:26.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:20:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:20:26.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:20:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:20:26.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:20:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:20:26.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:20:26 smithi138 ceph-mon[36954]: pgmap v7399: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:20:26.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:20:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:20:26.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:20:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:20:26.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:20:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:20:28.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:20:28 smithi100 ceph-mon[32599]: pgmap v7400: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:20:28.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:20:28 smithi167 ceph-mon[37140]: pgmap v7400: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:20:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:20:28 smithi138 ceph-mon[36954]: pgmap v7400: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:20:30.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:20:30 smithi100 ceph-mon[32599]: pgmap v7401: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:20:30.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:20:30 smithi167 ceph-mon[37140]: pgmap v7401: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:20:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:20:30 smithi138 ceph-mon[36954]: pgmap v7401: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:20:32.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:20:32 smithi100 ceph-mon[32599]: pgmap v7402: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:20:32.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:20:32 smithi167 ceph-mon[37140]: pgmap v7402: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:20:32.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:20:32 smithi138 ceph-mon[36954]: pgmap v7402: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:20:34.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:20:34 smithi100 ceph-mon[32599]: pgmap v7403: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:20:34.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:20:34 smithi167 ceph-mon[37140]: pgmap v7403: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:20:34.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:20:34 smithi138 ceph-mon[36954]: pgmap v7403: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:20:36.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:20:36 smithi100 ceph-mon[32599]: pgmap v7404: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:20:36.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:20:36 smithi167 ceph-mon[37140]: pgmap v7404: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:20:36.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:20:36 smithi138 ceph-mon[36954]: pgmap v7404: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:20:38.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:20:38 smithi100 ceph-mon[32599]: pgmap v7405: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:20:38.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:20:38 smithi167 ceph-mon[37140]: pgmap v7405: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:20:38.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:20:38 smithi138 ceph-mon[36954]: pgmap v7405: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:20:39.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:20:39.564+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:20:39.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:20:39.565+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:20:39.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:20:39.565+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:20:39.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:20:39.565+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:20:39.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:20:39.565+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:20:40.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:20:40 smithi167 ceph-mon[37140]: pgmap v7406: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:20:40.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:20:40 smithi138 ceph-mon[36954]: pgmap v7406: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:20:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:20:40 smithi100 ceph-mon[32599]: pgmap v7406: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:20:42.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:20:42 smithi167 ceph-mon[37140]: pgmap v7407: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:20:42.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:20:42 smithi138 ceph-mon[36954]: pgmap v7407: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:20:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:20:42 smithi100 ceph-mon[32599]: pgmap v7407: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:20:44.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:20:44 smithi167 ceph-mon[37140]: pgmap v7408: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:20:44.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:20:44 smithi138 ceph-mon[36954]: pgmap v7408: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:20:44.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:20:44 smithi100 ceph-mon[32599]: pgmap v7408: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:20:46.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:20:46 smithi167 ceph-mon[37140]: pgmap v7409: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:20:46.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:20:46 smithi138 ceph-mon[36954]: pgmap v7409: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:20:46.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:20:46 smithi100 ceph-mon[32599]: pgmap v7409: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:20:48.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:20:48 smithi167 ceph-mon[37140]: pgmap v7410: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:20:48.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:20:48 smithi138 ceph-mon[36954]: pgmap v7410: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:20:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:20:48 smithi100 ceph-mon[32599]: pgmap v7410: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:20:50.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:20:50 smithi167 ceph-mon[37140]: pgmap v7411: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:20:50.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:20:50 smithi138 ceph-mon[36954]: pgmap v7411: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:20:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:20:50 smithi100 ceph-mon[32599]: pgmap v7411: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:20:52.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:20:52 smithi167 ceph-mon[37140]: pgmap v7412: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:20:52.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:20:52 smithi138 ceph-mon[36954]: pgmap v7412: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:20:52.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:20:52 smithi100 ceph-mon[32599]: pgmap v7412: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:20:54.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:20:54 smithi100 ceph-mon[32599]: pgmap v7413: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:20:54.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:20:54.580+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:20:54.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:20:54.580+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:20:54.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:20:54.580+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:20:54.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:20:54.580+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:20:54.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:20:54.580+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:20:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:20:54 smithi138 ceph-mon[36954]: pgmap v7413: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:20:54.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:20:54 smithi167 ceph-mon[37140]: pgmap v7413: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:20:56.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:20:56 smithi138 ceph-mon[36954]: pgmap v7414: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:20:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:20:56 smithi100 ceph-mon[32599]: pgmap v7414: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:20:56.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:20:56 smithi167 ceph-mon[37140]: pgmap v7414: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:20:58.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:20:58 smithi138 ceph-mon[36954]: pgmap v7415: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:20:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:20:58 smithi100 ceph-mon[32599]: pgmap v7415: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:20:58.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:20:58 smithi167 ceph-mon[37140]: pgmap v7415: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:21:00.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:21:00 smithi138 ceph-mon[36954]: pgmap v7416: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:21:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:21:00 smithi100 ceph-mon[32599]: pgmap v7416: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:21:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:21:00 smithi167 ceph-mon[37140]: pgmap v7416: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:21:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:21:02 smithi138 ceph-mon[36954]: pgmap v7417: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:21:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:21:02 smithi100 ceph-mon[32599]: pgmap v7417: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:21:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:21:02 smithi167 ceph-mon[37140]: pgmap v7417: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:21:04.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:21:04 smithi138 ceph-mon[36954]: pgmap v7418: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:21:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:21:04 smithi100 ceph-mon[32599]: pgmap v7418: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:21:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:21:04 smithi167 ceph-mon[37140]: pgmap v7418: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:21:04.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:21:04.828+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T17:21:04.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:21:04.828+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c004740 con 0x7fe5a81087a0 2024-07-26T17:21:04.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:21:04.828+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 101 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c070 con 0x7fe5a81087a0 2024-07-26T17:21:04.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:21:04.828+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 101 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T17:21:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:21:06 smithi138 ceph-mon[36954]: pgmap v7419: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:21:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:21:06 smithi100 ceph-mon[32599]: pgmap v7419: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:21:06.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:21:06 smithi167 ceph-mon[37140]: pgmap v7419: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:21:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:21:08 smithi138 ceph-mon[36954]: pgmap v7420: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:21:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:21:08 smithi138 ceph-mon[36954]: osdmap e510: 12 total, 12 up, 12 in 2024-07-26T17:21:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:21:08 smithi100 ceph-mon[32599]: pgmap v7420: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:21:08.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:21:08 smithi100 ceph-mon[32599]: osdmap e510: 12 total, 12 up, 12 in 2024-07-26T17:21:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:21:08 smithi167 ceph-mon[37140]: pgmap v7420: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:21:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:21:08 smithi167 ceph-mon[37140]: osdmap e510: 12 total, 12 up, 12 in 2024-07-26T17:21:09.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:21:09.596+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:21:09.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:21:09.597+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:21:09.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:21:09.597+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:21:09.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:21:09.597+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:21:09.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:21:09.597+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:21:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:21:10 smithi138 ceph-mon[36954]: pgmap v7422: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:21:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:21:10 smithi100 ceph-mon[32599]: pgmap v7422: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:21:10.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:21:10 smithi167 ceph-mon[37140]: pgmap v7422: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:21:12.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:21:12 smithi138 ceph-mon[36954]: pgmap v7423: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:21:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:21:12 smithi100 ceph-mon[32599]: pgmap v7423: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:21:12.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:21:12 smithi167 ceph-mon[37140]: pgmap v7423: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:21:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:21:14 smithi138 ceph-mon[36954]: pgmap v7424: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:21:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:21:14 smithi100 ceph-mon[32599]: pgmap v7424: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:21:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:21:14 smithi167 ceph-mon[37140]: pgmap v7424: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:21:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:21:16 smithi138 ceph-mon[36954]: pgmap v7425: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:21:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:21:16 smithi100 ceph-mon[32599]: pgmap v7425: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:21:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:21:16 smithi167 ceph-mon[37140]: pgmap v7425: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:21:18.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:21:18 smithi100 ceph-mon[32599]: pgmap v7426: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:21:18.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:21:18 smithi100 ceph-mon[32599]: osdmap e511: 12 total, 12 up, 12 in 2024-07-26T17:21:18.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:21:18 smithi167 ceph-mon[37140]: pgmap v7426: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:21:18.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:21:18 smithi167 ceph-mon[37140]: osdmap e511: 12 total, 12 up, 12 in 2024-07-26T17:21:18.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:21:18 smithi138 ceph-mon[36954]: pgmap v7426: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:21:18.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:21:18 smithi138 ceph-mon[36954]: osdmap e511: 12 total, 12 up, 12 in 2024-07-26T17:21:20.493 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:21:20 smithi138 ceph-mon[36954]: pgmap v7428: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:21:20.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:21:20 smithi100 ceph-mon[32599]: pgmap v7428: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:21:20.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:21:20 smithi167 ceph-mon[37140]: pgmap v7428: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:21:22.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:21:22 smithi100 ceph-mon[32599]: pgmap v7429: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:21:22.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:21:22 smithi167 ceph-mon[37140]: pgmap v7429: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:21:22.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:21:22 smithi138 ceph-mon[36954]: pgmap v7429: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:21:24.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:21:24 smithi100 ceph-mon[32599]: pgmap v7430: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:21:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:21:24 smithi167 ceph-mon[37140]: pgmap v7430: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:21:24.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:21:24.612+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:21:24.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:21:24.613+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:21:24.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:21:24.613+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:21:24.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:21:24.613+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:21:24.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:21:24.613+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:21:24.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:21:24 smithi138 ceph-mon[36954]: pgmap v7430: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:21:26.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:21:26 smithi100 ceph-mon[32599]: pgmap v7431: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:21:26.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:21:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:21:26.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:21:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:21:26.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:21:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:21:26.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:21:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:21:26.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:21:26 smithi167 ceph-mon[37140]: pgmap v7431: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:21:26.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:21:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:21:26.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:21:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:21:26.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:21:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:21:26.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:21:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:21:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:21:26 smithi138 ceph-mon[36954]: pgmap v7431: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:21:26.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:21:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:21:26.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:21:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:21:26.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:21:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:21:26.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:21:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:21:28.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:21:28 smithi100 ceph-mon[32599]: pgmap v7432: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:21:28.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:21:28 smithi167 ceph-mon[37140]: pgmap v7432: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:21:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:21:28 smithi138 ceph-mon[36954]: pgmap v7432: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:21:30.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:21:30 smithi100 ceph-mon[32599]: pgmap v7433: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:21:30.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:21:30 smithi167 ceph-mon[37140]: pgmap v7433: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:21:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:21:30 smithi138 ceph-mon[36954]: pgmap v7433: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:21:32.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:21:32 smithi167 ceph-mon[37140]: pgmap v7434: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:21:32.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:21:32 smithi138 ceph-mon[36954]: pgmap v7434: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:21:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:21:32 smithi100 ceph-mon[32599]: pgmap v7434: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:21:34.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:21:34 smithi167 ceph-mon[37140]: pgmap v7435: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:21:34.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:21:34 smithi138 ceph-mon[36954]: pgmap v7435: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:21:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:21:34 smithi100 ceph-mon[32599]: pgmap v7435: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:21:36.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:21:36 smithi167 ceph-mon[37140]: pgmap v7436: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:21:36.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:21:36 smithi138 ceph-mon[36954]: pgmap v7436: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:21:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:21:36 smithi100 ceph-mon[32599]: pgmap v7436: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:21:38.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:21:38 smithi167 ceph-mon[37140]: pgmap v7437: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:21:38.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:21:38 smithi138 ceph-mon[36954]: pgmap v7437: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:21:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:21:38 smithi100 ceph-mon[32599]: pgmap v7437: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:21:39.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:21:39.623+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:21:39.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:21:39.624+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:21:39.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:21:39.624+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:21:39.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:21:39.624+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:21:39.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:21:39.624+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:21:40.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:21:40 smithi167 ceph-mon[37140]: pgmap v7438: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:21:40.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:21:40 smithi138 ceph-mon[36954]: pgmap v7438: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:21:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:21:40 smithi100 ceph-mon[32599]: pgmap v7438: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:21:42.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:21:42 smithi167 ceph-mon[37140]: pgmap v7439: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:21:42.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:21:42 smithi138 ceph-mon[36954]: pgmap v7439: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:21:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:21:42 smithi100 ceph-mon[32599]: pgmap v7439: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:21:44.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:21:44 smithi167 ceph-mon[37140]: pgmap v7440: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:21:44.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:21:44 smithi138 ceph-mon[36954]: pgmap v7440: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:21:44.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:21:44 smithi100 ceph-mon[32599]: pgmap v7440: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:21:46.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:21:46 smithi167 ceph-mon[37140]: pgmap v7441: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:21:46.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:21:46 smithi138 ceph-mon[36954]: pgmap v7441: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:21:46.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:21:46 smithi100 ceph-mon[32599]: pgmap v7441: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:21:48.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:21:48 smithi167 ceph-mon[37140]: pgmap v7442: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:21:48.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:21:48 smithi138 ceph-mon[36954]: pgmap v7442: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:21:48.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:21:48 smithi100 ceph-mon[32599]: pgmap v7442: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:21:50.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:21:50 smithi138 ceph-mon[36954]: pgmap v7443: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T17:21:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:21:50 smithi100 ceph-mon[32599]: pgmap v7443: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T17:21:50.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:21:50 smithi167 ceph-mon[37140]: pgmap v7443: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T17:21:52.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:21:52 smithi138 ceph-mon[36954]: pgmap v7444: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:21:52.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:21:52 smithi100 ceph-mon[32599]: pgmap v7444: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:21:52.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:21:52 smithi167 ceph-mon[37140]: pgmap v7444: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:21:54.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:21:54 smithi100 ceph-mon[32599]: pgmap v7445: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T17:21:54.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:21:54 smithi167 ceph-mon[37140]: pgmap v7445: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T17:21:54.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:21:54.639+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:21:54.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:21:54.640+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:21:54.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:21:54.640+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:21:54.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:21:54.640+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:21:54.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:21:54.640+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:21:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:21:54 smithi138 ceph-mon[36954]: pgmap v7445: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T17:21:56.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:21:56 smithi138 ceph-mon[36954]: pgmap v7446: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:21:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:21:56 smithi100 ceph-mon[32599]: pgmap v7446: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:21:56.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:21:56 smithi167 ceph-mon[37140]: pgmap v7446: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:21:58.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:21:58 smithi138 ceph-mon[36954]: pgmap v7447: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:21:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:21:58 smithi100 ceph-mon[32599]: pgmap v7447: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:21:58.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:21:58 smithi167 ceph-mon[37140]: pgmap v7447: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:22:00.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:00 smithi138 ceph-mon[36954]: pgmap v7448: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T17:22:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:00 smithi100 ceph-mon[32599]: pgmap v7448: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T17:22:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:00 smithi167 ceph-mon[37140]: pgmap v7448: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T17:22:02.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:02 smithi138 ceph-mon[36954]: pgmap v7449: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:22:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:02 smithi100 ceph-mon[32599]: pgmap v7449: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:22:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:02 smithi167 ceph-mon[37140]: pgmap v7449: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:22:04.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:04 smithi138 ceph-mon[36954]: pgmap v7450: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:22:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:04 smithi100 ceph-mon[32599]: pgmap v7450: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:22:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:04 smithi167 ceph-mon[37140]: pgmap v7450: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:22:06.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:06 smithi138 ceph-mon[36954]: pgmap v7451: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:22:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:06 smithi100 ceph-mon[32599]: pgmap v7451: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:22:06.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:06 smithi167 ceph-mon[37140]: pgmap v7451: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:22:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:08 smithi138 ceph-mon[36954]: pgmap v7452: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:22:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:08 smithi138 ceph-mon[36954]: osdmap e512: 12 total, 12 up, 12 in 2024-07-26T17:22:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:08 smithi100 ceph-mon[32599]: pgmap v7452: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:22:08.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:08 smithi100 ceph-mon[32599]: osdmap e512: 12 total, 12 up, 12 in 2024-07-26T17:22:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:08 smithi167 ceph-mon[37140]: pgmap v7452: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T17:22:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:08 smithi167 ceph-mon[37140]: osdmap e512: 12 total, 12 up, 12 in 2024-07-26T17:22:09.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:22:09.655+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:22:09.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:22:09.655+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:22:09.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:22:09.655+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:22:09.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:22:09.655+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:22:09.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:22:09.655+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:22:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:10 smithi138 ceph-mon[36954]: pgmap v7454: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:22:10.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:10 smithi100 ceph-mon[32599]: pgmap v7454: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:22:10.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:10 smithi167 ceph-mon[37140]: pgmap v7454: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:22:12.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:12 smithi138 ceph-mon[36954]: pgmap v7455: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:22:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:12 smithi100 ceph-mon[32599]: pgmap v7455: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:22:12.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:12 smithi167 ceph-mon[37140]: pgmap v7455: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:22:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:14 smithi138 ceph-mon[36954]: pgmap v7456: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:22:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:14 smithi100 ceph-mon[32599]: pgmap v7456: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:22:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:14 smithi167 ceph-mon[37140]: pgmap v7456: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:22:16.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:16 smithi138 ceph-mon[36954]: pgmap v7457: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:22:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:16 smithi100 ceph-mon[32599]: pgmap v7457: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:22:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:16 smithi167 ceph-mon[37140]: pgmap v7457: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:22:18.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:18 smithi100 ceph-mon[32599]: pgmap v7458: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:22:18.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:18 smithi100 ceph-mon[32599]: osdmap e513: 12 total, 12 up, 12 in 2024-07-26T17:22:18.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:18 smithi167 ceph-mon[37140]: pgmap v7458: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:22:18.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:18 smithi167 ceph-mon[37140]: osdmap e513: 12 total, 12 up, 12 in 2024-07-26T17:22:18.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:18 smithi138 ceph-mon[36954]: pgmap v7458: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:22:18.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:18 smithi138 ceph-mon[36954]: osdmap e513: 12 total, 12 up, 12 in 2024-07-26T17:22:20.493 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:20 smithi138 ceph-mon[36954]: pgmap v7460: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:22:20.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:20 smithi100 ceph-mon[32599]: pgmap v7460: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:22:20.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:20 smithi167 ceph-mon[37140]: pgmap v7460: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:22:22.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:22 smithi167 ceph-mon[37140]: pgmap v7461: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:22:22.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:22 smithi138 ceph-mon[36954]: pgmap v7461: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:22:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:22 smithi100 ceph-mon[32599]: pgmap v7461: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:22:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:24 smithi167 ceph-mon[37140]: pgmap v7462: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:22:24.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:24 smithi100 ceph-mon[32599]: pgmap v7462: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:22:24.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:22:24.664+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:22:24.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:22:24.665+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:22:24.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:22:24.665+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:22:24.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:22:24.665+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:22:24.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:22:24.665+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:22:24.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:24 smithi138 ceph-mon[36954]: pgmap v7462: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:22:26.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:26 smithi167 ceph-mon[37140]: pgmap v7463: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:22:26.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:22:26.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:26 smithi138 ceph-mon[36954]: pgmap v7463: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:22:26.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:22:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:26 smithi100 ceph-mon[32599]: pgmap v7463: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:22:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:22:28.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:28 smithi138 ceph-mon[36954]: pgmap v7464: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:22:28.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:22:28.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:22:28.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:22:28.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:22:28.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:22:28.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:22:28.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:22:28.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:22:28.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:22:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:28 smithi100 ceph-mon[32599]: pgmap v7464: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:22:28.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:22:28.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:22:28.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:22:28.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:22:28.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:22:28.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:22:28.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:22:28.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:22:28.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:22:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:28 smithi167 ceph-mon[37140]: pgmap v7464: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:22:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:22:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:22:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:22:29.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:22:29.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:22:29.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:22:29.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:22:29.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:22:29.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:22:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:30 smithi138 ceph-mon[36954]: pgmap v7465: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:22:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:30 smithi100 ceph-mon[32599]: pgmap v7465: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:22:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:30 smithi167 ceph-mon[37140]: pgmap v7465: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:22:32.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:32 smithi138 ceph-mon[36954]: pgmap v7466: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:22:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:32 smithi100 ceph-mon[32599]: pgmap v7466: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:22:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:32 smithi167 ceph-mon[37140]: pgmap v7466: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:22:34.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:22:34.840+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T17:22:34.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:22:34.840+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c004740 con 0x7fe5a81087a0 2024-07-26T17:22:34.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:22:34.840+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 102 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T17:22:34.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:22:34.841+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 102 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe5a008c050 con 0x7fe5a81087a0 2024-07-26T17:22:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:34 smithi138 ceph-mon[36954]: pgmap v7467: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:22:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:34 smithi100 ceph-mon[32599]: pgmap v7467: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:22:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:34 smithi167 ceph-mon[37140]: pgmap v7467: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:22:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:36 smithi138 ceph-mon[36954]: pgmap v7468: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:22:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:36 smithi100 ceph-mon[32599]: pgmap v7468: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:22:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:36 smithi167 ceph-mon[37140]: pgmap v7468: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:22:38.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:38 smithi138 ceph-mon[36954]: pgmap v7469: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:22:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:38 smithi100 ceph-mon[32599]: pgmap v7469: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:22:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:38 smithi167 ceph-mon[37140]: pgmap v7469: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:22:39.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:22:39.680+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:22:39.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:22:39.681+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:22:39.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:22:39.681+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:22:39.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:22:39.681+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:22:39.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:22:39.681+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:22:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:40 smithi138 ceph-mon[36954]: pgmap v7470: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:22:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:40 smithi100 ceph-mon[32599]: pgmap v7470: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:22:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:40 smithi167 ceph-mon[37140]: pgmap v7470: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:22:42.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:42 smithi138 ceph-mon[36954]: pgmap v7471: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:22:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:42 smithi100 ceph-mon[32599]: pgmap v7471: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:22:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:42 smithi167 ceph-mon[37140]: pgmap v7471: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:22:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:44 smithi138 ceph-mon[36954]: pgmap v7472: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:22:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:44 smithi100 ceph-mon[32599]: pgmap v7472: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:22:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:44 smithi167 ceph-mon[37140]: pgmap v7472: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:22:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:46 smithi138 ceph-mon[36954]: pgmap v7473: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:22:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:46 smithi100 ceph-mon[32599]: pgmap v7473: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:22:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:46 smithi167 ceph-mon[37140]: pgmap v7473: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:22:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:48 smithi138 ceph-mon[36954]: pgmap v7474: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:22:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:48 smithi100 ceph-mon[32599]: pgmap v7474: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:22:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:48 smithi167 ceph-mon[37140]: pgmap v7474: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:22:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:50 smithi138 ceph-mon[36954]: pgmap v7475: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:22:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:50 smithi100 ceph-mon[32599]: pgmap v7475: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:22:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:50 smithi167 ceph-mon[37140]: pgmap v7475: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:22:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:52 smithi138 ceph-mon[36954]: pgmap v7476: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:22:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:52 smithi100 ceph-mon[32599]: pgmap v7476: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:22:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:52 smithi167 ceph-mon[37140]: pgmap v7476: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:22:54.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:22:54.696+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:22:54.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:22:54.697+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:22:54.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:22:54.697+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:22:54.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:22:54.697+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:22:54.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:22:54.697+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:22:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:54 smithi138 ceph-mon[36954]: pgmap v7477: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:22:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:54 smithi100 ceph-mon[32599]: pgmap v7477: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:22:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:54 smithi167 ceph-mon[37140]: pgmap v7477: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:22:56.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:56 smithi138 ceph-mon[36954]: pgmap v7478: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:22:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:56 smithi100 ceph-mon[32599]: pgmap v7478: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:22:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:56 smithi167 ceph-mon[37140]: pgmap v7478: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:22:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:22:58 smithi138 ceph-mon[36954]: pgmap v7479: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:22:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:22:58 smithi100 ceph-mon[32599]: pgmap v7479: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:22:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:22:58 smithi167 ceph-mon[37140]: pgmap v7479: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:23:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:00 smithi138 ceph-mon[36954]: pgmap v7480: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:23:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:00 smithi100 ceph-mon[32599]: pgmap v7480: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:23:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:00 smithi167 ceph-mon[37140]: pgmap v7480: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T17:23:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:02 smithi138 ceph-mon[36954]: pgmap v7481: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:23:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:02 smithi100 ceph-mon[32599]: pgmap v7481: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:23:03.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:02 smithi167 ceph-mon[37140]: pgmap v7481: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:23:04.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:04 smithi138 ceph-mon[36954]: pgmap v7482: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:23:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:04 smithi100 ceph-mon[32599]: pgmap v7482: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:23:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:04 smithi167 ceph-mon[37140]: pgmap v7482: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:23:05.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:05 smithi138 ceph-mon[36954]: pgmap v7483: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:23:05.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:05 smithi100 ceph-mon[32599]: pgmap v7483: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:23:06.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:05 smithi167 ceph-mon[37140]: pgmap v7483: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:23:08.194 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:07 smithi138 ceph-mon[36954]: pgmap v7484: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:23:08.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:07 smithi100 ceph-mon[32599]: pgmap v7484: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:23:08.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:07 smithi167 ceph-mon[37140]: pgmap v7484: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:23:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:08 smithi138 ceph-mon[36954]: osdmap e514: 12 total, 12 up, 12 in 2024-07-26T17:23:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:08 smithi100 ceph-mon[32599]: osdmap e514: 12 total, 12 up, 12 in 2024-07-26T17:23:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:08 smithi167 ceph-mon[37140]: osdmap e514: 12 total, 12 up, 12 in 2024-07-26T17:23:09.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:23:09.711+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:23:09.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:23:09.712+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:23:09.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:23:09.712+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:23:09.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:23:09.712+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:23:09.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:23:09.712+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:23:09.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:09 smithi100 ceph-mon[32599]: pgmap v7486: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:23:10.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:09 smithi167 ceph-mon[37140]: pgmap v7486: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:23:10.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:09 smithi138 ceph-mon[36954]: pgmap v7486: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:23:12.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:11 smithi100 ceph-mon[32599]: pgmap v7487: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:23:12.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:11 smithi167 ceph-mon[37140]: pgmap v7487: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:23:12.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:11 smithi138 ceph-mon[36954]: pgmap v7487: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:23:14.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:13 smithi100 ceph-mon[32599]: pgmap v7488: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:23:14.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:13 smithi167 ceph-mon[37140]: pgmap v7488: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:23:14.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:13 smithi138 ceph-mon[36954]: pgmap v7488: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:23:16.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:15 smithi100 ceph-mon[32599]: pgmap v7489: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:23:16.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:15 smithi167 ceph-mon[37140]: pgmap v7489: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:23:16.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:15 smithi138 ceph-mon[36954]: pgmap v7489: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:23:18.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:18 smithi167 ceph-mon[37140]: pgmap v7490: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:23:18.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:18 smithi167 ceph-mon[37140]: osdmap e515: 12 total, 12 up, 12 in 2024-07-26T17:23:18.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:18 smithi138 ceph-mon[36954]: pgmap v7490: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:23:18.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:18 smithi138 ceph-mon[36954]: osdmap e515: 12 total, 12 up, 12 in 2024-07-26T17:23:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:18 smithi100 ceph-mon[32599]: pgmap v7490: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:23:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:18 smithi100 ceph-mon[32599]: osdmap e515: 12 total, 12 up, 12 in 2024-07-26T17:23:20.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:20 smithi167 ceph-mon[37140]: pgmap v7492: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:23:20.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:20 smithi138 ceph-mon[36954]: pgmap v7492: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:23:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:20 smithi100 ceph-mon[32599]: pgmap v7492: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:23:22.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:22 smithi167 ceph-mon[37140]: pgmap v7493: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:23:22.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:22 smithi138 ceph-mon[36954]: pgmap v7493: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:23:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:22 smithi100 ceph-mon[32599]: pgmap v7493: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:23:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:24 smithi167 ceph-mon[37140]: pgmap v7494: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:23:24.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:24 smithi100 ceph-mon[32599]: pgmap v7494: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:23:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:24 smithi138 ceph-mon[36954]: pgmap v7494: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:23:24.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:23:24.725+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:23:24.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:23:24.725+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:23:24.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:23:24.725+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:23:24.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:23:24.725+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:23:24.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:23:24.725+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:23:26.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:26 smithi167 ceph-mon[37140]: pgmap v7495: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:23:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:26 smithi138 ceph-mon[36954]: pgmap v7495: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:23:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:26 smithi100 ceph-mon[32599]: pgmap v7495: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:23:28.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:28 smithi167 ceph-mon[37140]: pgmap v7496: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:23:28.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:23:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:28 smithi138 ceph-mon[36954]: pgmap v7496: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:23:28.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:23:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:28 smithi100 ceph-mon[32599]: pgmap v7496: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:23:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:23:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:23:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:23:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:23:29.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:23:29.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:23:29.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:23:29.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:23:29.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:23:29.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:23:29.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:23:29.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:23:29.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:23:29.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:23:29.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:23:29.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:23:29.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:23:29.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:23:29.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:23:29.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:23:29.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:23:29.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:23:29.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:23:29.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:23:29.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:23:29.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:23:29.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:23:29.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:23:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:30 smithi138 ceph-mon[36954]: pgmap v7497: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:23:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:30 smithi100 ceph-mon[32599]: pgmap v7497: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:23:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:30 smithi167 ceph-mon[37140]: pgmap v7497: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:23:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:32 smithi138 ceph-mon[36954]: pgmap v7498: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:23:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:32 smithi100 ceph-mon[32599]: pgmap v7498: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:23:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:32 smithi167 ceph-mon[37140]: pgmap v7498: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:23:34.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:34 smithi138 ceph-mon[36954]: pgmap v7499: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:23:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:34 smithi100 ceph-mon[32599]: pgmap v7499: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:23:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:34 smithi167 ceph-mon[37140]: pgmap v7499: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:23:36.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:36 smithi138 ceph-mon[36954]: pgmap v7500: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:23:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:36 smithi100 ceph-mon[32599]: pgmap v7500: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:23:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:36 smithi167 ceph-mon[37140]: pgmap v7500: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:23:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:38 smithi138 ceph-mon[36954]: pgmap v7501: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:23:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:38 smithi100 ceph-mon[32599]: pgmap v7501: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:23:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:38 smithi167 ceph-mon[37140]: pgmap v7501: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:23:39.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:23:39.731+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:23:39.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:23:39.731+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:23:39.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:23:39.731+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:23:39.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:23:39.731+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:23:39.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:23:39.732+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:23:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:40 smithi138 ceph-mon[36954]: pgmap v7502: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:23:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:40 smithi100 ceph-mon[32599]: pgmap v7502: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:23:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:40 smithi167 ceph-mon[37140]: pgmap v7502: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:23:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:42 smithi138 ceph-mon[36954]: pgmap v7503: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:23:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:42 smithi100 ceph-mon[32599]: pgmap v7503: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:23:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:42 smithi167 ceph-mon[37140]: pgmap v7503: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:23:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:44 smithi138 ceph-mon[36954]: pgmap v7504: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:23:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:44 smithi100 ceph-mon[32599]: pgmap v7504: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:23:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:44 smithi167 ceph-mon[37140]: pgmap v7504: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:23:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:46 smithi138 ceph-mon[36954]: pgmap v7505: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:23:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:46 smithi100 ceph-mon[32599]: pgmap v7505: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:23:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:46 smithi167 ceph-mon[37140]: pgmap v7505: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:23:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:48 smithi138 ceph-mon[36954]: pgmap v7506: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:23:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:48 smithi100 ceph-mon[32599]: pgmap v7506: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:23:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:48 smithi167 ceph-mon[37140]: pgmap v7506: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:23:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:50 smithi138 ceph-mon[36954]: pgmap v7507: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:23:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:50 smithi100 ceph-mon[32599]: pgmap v7507: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:23:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:50 smithi167 ceph-mon[37140]: pgmap v7507: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:23:52.937 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:52 smithi138 ceph-mon[36954]: pgmap v7508: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:23:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:52 smithi100 ceph-mon[32599]: pgmap v7508: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:23:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:52 smithi167 ceph-mon[37140]: pgmap v7508: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:23:54.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:23:54.747+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:23:54.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:23:54.748+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:23:54.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:23:54.748+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:23:54.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:23:54.748+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:23:54.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:23:54.748+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:23:54.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:54 smithi167 ceph-mon[37140]: pgmap v7509: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:23:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:54 smithi138 ceph-mon[36954]: pgmap v7509: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:23:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:54 smithi100 ceph-mon[32599]: pgmap v7509: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:23:56.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:56 smithi138 ceph-mon[36954]: pgmap v7510: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:23:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:56 smithi100 ceph-mon[32599]: pgmap v7510: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:23:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:56 smithi167 ceph-mon[37140]: pgmap v7510: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:23:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:23:58 smithi138 ceph-mon[36954]: pgmap v7511: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:23:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:23:58 smithi100 ceph-mon[32599]: pgmap v7511: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:23:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:23:58 smithi167 ceph-mon[37140]: pgmap v7511: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:24:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:24:00 smithi138 ceph-mon[36954]: pgmap v7512: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:24:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:24:00 smithi100 ceph-mon[32599]: pgmap v7512: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:24:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:24:00 smithi167 ceph-mon[37140]: pgmap v7512: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:24:02.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:24:02 smithi138 ceph-mon[36954]: pgmap v7513: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:24:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:24:02 smithi100 ceph-mon[32599]: pgmap v7513: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:24:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:24:02 smithi167 ceph-mon[37140]: pgmap v7513: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:24:04.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:24:04.851+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T17:24:04.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:24:04.851+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c004740 con 0x7fe5a81087a0 2024-07-26T17:24:04.854 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:24:04 smithi100 ceph-mon[32599]: pgmap v7514: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:24:04.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:24:04.851+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 103 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c070 con 0x7fe5a81087a0 2024-07-26T17:24:04.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:24:04.851+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 103 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T17:24:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:24:04 smithi138 ceph-mon[36954]: pgmap v7514: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:24:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:24:04 smithi167 ceph-mon[37140]: pgmap v7514: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:24:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:24:06 smithi138 ceph-mon[36954]: pgmap v7515: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:24:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:24:06 smithi100 ceph-mon[32599]: pgmap v7515: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:24:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:24:06 smithi167 ceph-mon[37140]: pgmap v7515: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:24:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:24:08 smithi138 ceph-mon[36954]: pgmap v7516: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:24:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:24:08 smithi138 ceph-mon[36954]: osdmap e516: 12 total, 12 up, 12 in 2024-07-26T17:24:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:24:08 smithi100 ceph-mon[32599]: pgmap v7516: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:24:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:24:08 smithi100 ceph-mon[32599]: osdmap e516: 12 total, 12 up, 12 in 2024-07-26T17:24:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:24:08 smithi167 ceph-mon[37140]: pgmap v7516: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:24:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:24:08 smithi167 ceph-mon[37140]: osdmap e516: 12 total, 12 up, 12 in 2024-07-26T17:24:09.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:24:09.763+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:24:09.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:24:09.764+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:24:09.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:24:09.764+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:24:09.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:24:09.764+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:24:09.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:24:09.764+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:24:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:24:10 smithi138 ceph-mon[36954]: pgmap v7518: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:24:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:24:10 smithi100 ceph-mon[32599]: pgmap v7518: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:24:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:24:10 smithi167 ceph-mon[37140]: pgmap v7518: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:24:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:24:12 smithi138 ceph-mon[36954]: pgmap v7519: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:24:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:24:12 smithi100 ceph-mon[32599]: pgmap v7519: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:24:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:24:12 smithi167 ceph-mon[37140]: pgmap v7519: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:24:14.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:24:14 smithi138 ceph-mon[36954]: pgmap v7520: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:24:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:24:14 smithi100 ceph-mon[32599]: pgmap v7520: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:24:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:24:14 smithi167 ceph-mon[37140]: pgmap v7520: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:24:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:24:16 smithi138 ceph-mon[36954]: pgmap v7521: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:24:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:24:16 smithi100 ceph-mon[32599]: pgmap v7521: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:24:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:24:16 smithi167 ceph-mon[37140]: pgmap v7521: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:24:18.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:24:18 smithi138 ceph-mon[36954]: pgmap v7522: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:24:18.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:24:18 smithi138 ceph-mon[36954]: osdmap e517: 12 total, 12 up, 12 in 2024-07-26T17:24:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:24:18 smithi100 ceph-mon[32599]: pgmap v7522: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:24:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:24:18 smithi100 ceph-mon[32599]: osdmap e517: 12 total, 12 up, 12 in 2024-07-26T17:24:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:24:18 smithi167 ceph-mon[37140]: pgmap v7522: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:24:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:24:18 smithi167 ceph-mon[37140]: osdmap e517: 12 total, 12 up, 12 in 2024-07-26T17:24:20.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:24:20 smithi138 ceph-mon[36954]: pgmap v7524: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:24:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:24:20 smithi100 ceph-mon[32599]: pgmap v7524: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:24:20.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:24:20 smithi167 ceph-mon[37140]: pgmap v7524: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:24:22.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:24:22 smithi138 ceph-mon[36954]: pgmap v7525: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:24:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:24:22 smithi100 ceph-mon[32599]: pgmap v7525: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:24:22.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:24:22 smithi167 ceph-mon[37140]: pgmap v7525: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:24:24.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:24:24 smithi100 ceph-mon[32599]: pgmap v7526: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:24:24.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:24:24 smithi167 ceph-mon[37140]: pgmap v7526: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:24:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:24:24 smithi138 ceph-mon[36954]: pgmap v7526: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:24:24.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:24:24.780+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:24:24.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:24:24.780+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:24:24.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:24:24.780+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:24:24.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:24:24.780+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:24:24.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:24:24.780+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:24:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:24:26 smithi138 ceph-mon[36954]: pgmap v7527: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:24:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:24:26 smithi100 ceph-mon[32599]: pgmap v7527: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:24:26.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:24:26 smithi167 ceph-mon[37140]: pgmap v7527: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:24:28.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:24:28 smithi138 ceph-mon[36954]: pgmap v7528: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:24:28.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:24:28 smithi100 ceph-mon[32599]: pgmap v7528: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:24:28.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:24:28 smithi167 ceph-mon[37140]: pgmap v7528: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:24:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:24:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:24:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:24:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:24:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:24:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:24:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:24:30 smithi138 ceph-mon[36954]: pgmap v7529: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:24:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:24:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:24:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:24:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:24:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:24:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:24:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:24:30 smithi100 ceph-mon[32599]: pgmap v7529: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:24:30.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:24:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:24:30.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:24:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:24:30.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:24:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:24:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:24:30 smithi167 ceph-mon[37140]: pgmap v7529: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:24:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:24:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:24:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:24:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:24:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:24:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:24:32.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:24:32 smithi138 ceph-mon[36954]: pgmap v7530: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:24:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:24:32 smithi100 ceph-mon[32599]: pgmap v7530: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:24:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:24:32 smithi167 ceph-mon[37140]: pgmap v7530: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:24:34.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:24:34 smithi138 ceph-mon[36954]: pgmap v7531: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T17:24:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:24:34 smithi100 ceph-mon[32599]: pgmap v7531: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T17:24:34.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:24:34 smithi167 ceph-mon[37140]: pgmap v7531: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T17:24:36.695 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:24:36 smithi138 ceph-mon[36954]: pgmap v7532: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:24:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:24:36 smithi100 ceph-mon[32599]: pgmap v7532: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:24:36.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:24:36 smithi167 ceph-mon[37140]: pgmap v7532: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:24:38.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:24:38 smithi138 ceph-mon[36954]: pgmap v7533: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T17:24:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:24:38 smithi100 ceph-mon[32599]: pgmap v7533: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T17:24:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:24:38 smithi167 ceph-mon[37140]: pgmap v7533: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T17:24:39.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:24:39.788+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:24:39.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:24:39.789+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:24:39.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:24:39.789+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:24:39.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:24:39.789+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:24:39.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:24:39.789+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:24:40.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:24:40 smithi138 ceph-mon[36954]: pgmap v7534: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:24:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:24:40 smithi100 ceph-mon[32599]: pgmap v7534: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:24:40.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:24:40 smithi167 ceph-mon[37140]: pgmap v7534: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:24:42.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:24:42 smithi138 ceph-mon[36954]: pgmap v7535: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:24:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:24:42 smithi100 ceph-mon[32599]: pgmap v7535: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:24:42.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:24:42 smithi167 ceph-mon[37140]: pgmap v7535: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:24:44.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:24:44 smithi138 ceph-mon[36954]: pgmap v7536: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T17:24:44.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:24:44 smithi100 ceph-mon[32599]: pgmap v7536: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T17:24:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:24:44 smithi167 ceph-mon[37140]: pgmap v7536: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T17:24:46.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:24:46 smithi138 ceph-mon[36954]: pgmap v7537: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:24:46.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:24:46 smithi100 ceph-mon[32599]: pgmap v7537: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:24:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:24:46 smithi167 ceph-mon[37140]: pgmap v7537: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:24:48.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:24:48 smithi138 ceph-mon[36954]: pgmap v7538: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:24:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:24:48 smithi100 ceph-mon[32599]: pgmap v7538: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:24:48.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:24:48 smithi167 ceph-mon[37140]: pgmap v7538: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:24:50.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:24:50 smithi138 ceph-mon[36954]: pgmap v7539: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:24:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:24:50 smithi100 ceph-mon[32599]: pgmap v7539: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:24:50.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:24:50 smithi167 ceph-mon[37140]: pgmap v7539: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:24:52.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:24:52 smithi138 ceph-mon[36954]: pgmap v7540: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:24:52.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:24:52 smithi100 ceph-mon[32599]: pgmap v7540: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:24:52.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:24:52 smithi167 ceph-mon[37140]: pgmap v7540: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:24:54.695 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:24:54 smithi138 ceph-mon[36954]: pgmap v7541: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:24:54.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:24:54 smithi100 ceph-mon[32599]: pgmap v7541: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:24:54.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:24:54 smithi167 ceph-mon[37140]: pgmap v7541: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:24:54.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:24:54.791+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:24:54.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:24:54.791+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:24:54.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:24:54.791+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:24:54.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:24:54.791+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:24:54.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:24:54.791+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:24:56.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:24:56 smithi138 ceph-mon[36954]: pgmap v7542: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:24:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:24:56 smithi100 ceph-mon[32599]: pgmap v7542: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:24:56.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:24:56 smithi167 ceph-mon[37140]: pgmap v7542: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:24:58.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:24:58 smithi138 ceph-mon[36954]: pgmap v7543: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:24:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:24:58 smithi100 ceph-mon[32599]: pgmap v7543: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:24:58.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:24:58 smithi167 ceph-mon[37140]: pgmap v7543: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:25:00.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:25:00 smithi138 ceph-mon[36954]: pgmap v7544: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:25:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:25:00 smithi100 ceph-mon[32599]: pgmap v7544: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:25:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:25:00 smithi167 ceph-mon[37140]: pgmap v7544: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:25:02.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:25:02 smithi138 ceph-mon[36954]: pgmap v7545: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:25:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:25:02 smithi100 ceph-mon[32599]: pgmap v7545: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:25:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:25:02 smithi167 ceph-mon[37140]: pgmap v7545: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:25:04.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:25:04 smithi138 ceph-mon[36954]: pgmap v7546: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:25:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:25:04 smithi100 ceph-mon[32599]: pgmap v7546: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:25:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:25:04 smithi167 ceph-mon[37140]: pgmap v7546: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:25:06.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:25:06 smithi138 ceph-mon[36954]: pgmap v7547: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:25:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:25:06 smithi100 ceph-mon[32599]: pgmap v7547: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:25:06.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:25:06 smithi167 ceph-mon[37140]: pgmap v7547: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:25:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:25:08 smithi138 ceph-mon[36954]: pgmap v7548: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:25:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:25:08 smithi138 ceph-mon[36954]: osdmap e518: 12 total, 12 up, 12 in 2024-07-26T17:25:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:25:08 smithi100 ceph-mon[32599]: pgmap v7548: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:25:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:25:08 smithi100 ceph-mon[32599]: osdmap e518: 12 total, 12 up, 12 in 2024-07-26T17:25:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:25:08 smithi167 ceph-mon[37140]: pgmap v7548: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:25:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:25:08 smithi167 ceph-mon[37140]: osdmap e518: 12 total, 12 up, 12 in 2024-07-26T17:25:09.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:25:09.807+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:25:09.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:25:09.807+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:25:09.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:25:09.807+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:25:09.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:25:09.807+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:25:09.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:25:09.807+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:25:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:25:10 smithi138 ceph-mon[36954]: pgmap v7550: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:25:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:25:10 smithi100 ceph-mon[32599]: pgmap v7550: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:25:10.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:25:10 smithi167 ceph-mon[37140]: pgmap v7550: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:25:12.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:25:12 smithi138 ceph-mon[36954]: pgmap v7551: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:25:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:25:12 smithi100 ceph-mon[32599]: pgmap v7551: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:25:12.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:25:12 smithi167 ceph-mon[37140]: pgmap v7551: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:25:14.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:25:14 smithi138 ceph-mon[36954]: pgmap v7552: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:25:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:25:14 smithi100 ceph-mon[32599]: pgmap v7552: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:25:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:25:14 smithi167 ceph-mon[37140]: pgmap v7552: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:25:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:25:16 smithi138 ceph-mon[36954]: pgmap v7553: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:25:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:25:16 smithi100 ceph-mon[32599]: pgmap v7553: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:25:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:25:16 smithi167 ceph-mon[37140]: pgmap v7553: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:25:18.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:25:18 smithi138 ceph-mon[36954]: pgmap v7554: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:25:18.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:25:18 smithi138 ceph-mon[36954]: osdmap e519: 12 total, 12 up, 12 in 2024-07-26T17:25:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:25:18 smithi100 ceph-mon[32599]: pgmap v7554: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:25:18.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:25:18 smithi100 ceph-mon[32599]: osdmap e519: 12 total, 12 up, 12 in 2024-07-26T17:25:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:25:18 smithi167 ceph-mon[37140]: pgmap v7554: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:25:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:25:18 smithi167 ceph-mon[37140]: osdmap e519: 12 total, 12 up, 12 in 2024-07-26T17:25:20.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:25:20 smithi138 ceph-mon[36954]: pgmap v7556: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:25:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:25:20 smithi100 ceph-mon[32599]: pgmap v7556: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:25:20.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:25:20 smithi167 ceph-mon[37140]: pgmap v7556: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:25:22.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:25:22 smithi138 ceph-mon[36954]: pgmap v7557: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:25:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:25:22 smithi100 ceph-mon[32599]: pgmap v7557: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:25:22.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:25:22 smithi167 ceph-mon[37140]: pgmap v7557: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:25:24.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:25:24 smithi167 ceph-mon[37140]: pgmap v7558: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:25:24.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:25:24 smithi138 ceph-mon[36954]: pgmap v7558: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:25:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:25:24 smithi100 ceph-mon[32599]: pgmap v7558: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:25:24.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:25:24.822+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:25:24.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:25:24.823+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:25:24.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:25:24.823+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:25:24.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:25:24.823+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:25:24.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:25:24.823+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:25:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:25:26 smithi138 ceph-mon[36954]: pgmap v7559: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:25:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:25:26 smithi100 ceph-mon[32599]: pgmap v7559: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:25:26.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:25:26 smithi167 ceph-mon[37140]: pgmap v7559: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:25:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:25:28 smithi138 ceph-mon[36954]: pgmap v7560: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:25:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:25:28 smithi100 ceph-mon[32599]: pgmap v7560: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:25:28.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:25:28 smithi167 ceph-mon[37140]: pgmap v7560: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:25:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:25:30 smithi138 ceph-mon[36954]: pgmap v7561: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:25:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:25:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:25:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:25:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:25:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:25:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:25:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:25:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:25:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:25:30 smithi100 ceph-mon[32599]: pgmap v7561: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:25:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:25:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:25:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:25:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:25:30.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:25:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:25:30.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:25:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:25:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:25:30 smithi167 ceph-mon[37140]: pgmap v7561: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:25:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:25:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:25:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:25:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:25:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:25:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:25:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:25:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:25:32.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:25:32 smithi138 ceph-mon[36954]: pgmap v7562: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:25:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:25:32 smithi100 ceph-mon[32599]: pgmap v7562: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:25:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:25:32 smithi167 ceph-mon[37140]: pgmap v7562: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:25:34.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:25:34 smithi138 ceph-mon[36954]: pgmap v7563: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:25:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:25:34 smithi100 ceph-mon[32599]: pgmap v7563: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:25:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:25:34 smithi167 ceph-mon[37140]: pgmap v7563: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:25:36.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:25:36 smithi138 ceph-mon[36954]: pgmap v7564: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:25:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:25:36 smithi100 ceph-mon[32599]: pgmap v7564: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:25:36.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:25:36 smithi167 ceph-mon[37140]: pgmap v7564: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:25:38.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:25:38 smithi138 ceph-mon[36954]: pgmap v7565: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:25:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:25:38 smithi100 ceph-mon[32599]: pgmap v7565: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:25:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:25:38 smithi167 ceph-mon[37140]: pgmap v7565: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:25:39.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:25:39.839+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:25:39.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:25:39.839+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:25:39.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:25:39.839+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:25:39.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:25:39.839+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:25:39.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:25:39.839+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:25:40.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:25:40 smithi138 ceph-mon[36954]: pgmap v7566: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:25:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:25:40 smithi100 ceph-mon[32599]: pgmap v7566: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:25:40.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:25:40 smithi167 ceph-mon[37140]: pgmap v7566: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:25:42.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:25:42 smithi138 ceph-mon[36954]: pgmap v7567: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:25:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:25:42 smithi100 ceph-mon[32599]: pgmap v7567: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:25:42.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:25:42 smithi167 ceph-mon[37140]: pgmap v7567: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:25:44.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:25:44 smithi138 ceph-mon[36954]: pgmap v7568: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:25:44.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:25:44 smithi100 ceph-mon[32599]: pgmap v7568: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:25:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:25:44 smithi167 ceph-mon[37140]: pgmap v7568: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:25:46.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:25:46 smithi138 ceph-mon[36954]: pgmap v7569: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:25:46.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:25:46 smithi100 ceph-mon[32599]: pgmap v7569: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:25:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:25:46 smithi167 ceph-mon[37140]: pgmap v7569: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:25:48.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:25:48 smithi138 ceph-mon[36954]: pgmap v7570: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:25:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:25:48 smithi100 ceph-mon[32599]: pgmap v7570: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:25:48.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:25:48 smithi167 ceph-mon[37140]: pgmap v7570: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:25:50.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:25:50 smithi138 ceph-mon[36954]: pgmap v7571: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:25:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:25:50 smithi100 ceph-mon[32599]: pgmap v7571: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:25:50.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:25:50 smithi167 ceph-mon[37140]: pgmap v7571: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:25:52.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:25:52 smithi138 ceph-mon[36954]: pgmap v7572: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:25:52.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:25:52 smithi100 ceph-mon[32599]: pgmap v7572: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:25:52.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:25:52 smithi167 ceph-mon[37140]: pgmap v7572: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:25:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:25:54 smithi138 ceph-mon[36954]: pgmap v7573: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:25:54.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:25:54 smithi100 ceph-mon[32599]: pgmap v7573: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:25:54.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:25:54 smithi167 ceph-mon[37140]: pgmap v7573: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:25:54.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:25:54.855+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:25:54.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:25:54.855+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:25:54.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:25:54.855+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:25:54.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:25:54.855+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:25:54.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:25:54.855+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:25:56.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:25:56 smithi138 ceph-mon[36954]: pgmap v7574: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:25:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:25:56 smithi100 ceph-mon[32599]: pgmap v7574: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:25:56.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:25:56 smithi167 ceph-mon[37140]: pgmap v7574: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:25:58.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:25:58 smithi138 ceph-mon[36954]: pgmap v7575: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:25:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:25:58 smithi100 ceph-mon[32599]: pgmap v7575: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:25:58.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:25:58 smithi167 ceph-mon[37140]: pgmap v7575: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:26:00.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:26:00 smithi138 ceph-mon[36954]: pgmap v7576: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:26:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:26:00 smithi100 ceph-mon[32599]: pgmap v7576: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:26:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:26:00 smithi167 ceph-mon[37140]: pgmap v7576: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:26:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:26:02 smithi138 ceph-mon[36954]: pgmap v7577: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:26:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:26:02 smithi100 ceph-mon[32599]: pgmap v7577: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:26:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:26:02 smithi167 ceph-mon[37140]: pgmap v7577: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:26:04.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:26:04 smithi138 ceph-mon[36954]: pgmap v7578: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:26:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:26:04 smithi100 ceph-mon[32599]: pgmap v7578: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:26:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:26:04 smithi167 ceph-mon[37140]: pgmap v7578: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:26:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:26:06 smithi138 ceph-mon[36954]: pgmap v7579: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:26:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:26:06 smithi100 ceph-mon[32599]: pgmap v7579: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:26:06.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:26:06 smithi167 ceph-mon[37140]: pgmap v7579: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:26:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:26:08 smithi138 ceph-mon[36954]: pgmap v7580: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:26:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:26:08 smithi138 ceph-mon[36954]: osdmap e520: 12 total, 12 up, 12 in 2024-07-26T17:26:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:26:08 smithi100 ceph-mon[32599]: pgmap v7580: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:26:08.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:26:08 smithi100 ceph-mon[32599]: osdmap e520: 12 total, 12 up, 12 in 2024-07-26T17:26:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:26:08 smithi167 ceph-mon[37140]: pgmap v7580: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:26:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:26:08 smithi167 ceph-mon[37140]: osdmap e520: 12 total, 12 up, 12 in 2024-07-26T17:26:09.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:26:09.867+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:26:09.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:26:09.868+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:26:09.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:26:09.868+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:26:09.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:26:09.868+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:26:09.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:26:09.868+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:26:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:26:10 smithi138 ceph-mon[36954]: pgmap v7582: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:26:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:26:10 smithi100 ceph-mon[32599]: pgmap v7582: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:26:10.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:26:10 smithi167 ceph-mon[37140]: pgmap v7582: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:26:12.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:26:12 smithi138 ceph-mon[36954]: pgmap v7583: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:26:12.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:26:12 smithi100 ceph-mon[32599]: pgmap v7583: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:26:12.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:26:12 smithi167 ceph-mon[37140]: pgmap v7583: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:26:14.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:26:14 smithi138 ceph-mon[36954]: pgmap v7584: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:26:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:26:14 smithi100 ceph-mon[32599]: pgmap v7584: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:26:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:26:14 smithi167 ceph-mon[37140]: pgmap v7584: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:26:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:26:16 smithi138 ceph-mon[36954]: pgmap v7585: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:26:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:26:16 smithi100 ceph-mon[32599]: pgmap v7585: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:26:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:26:16 smithi167 ceph-mon[37140]: pgmap v7585: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:26:18.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:26:18 smithi138 ceph-mon[36954]: pgmap v7586: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:26:18.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:26:18 smithi138 ceph-mon[36954]: osdmap e521: 12 total, 12 up, 12 in 2024-07-26T17:26:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:26:18 smithi100 ceph-mon[32599]: pgmap v7586: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:26:18.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:26:18 smithi100 ceph-mon[32599]: osdmap e521: 12 total, 12 up, 12 in 2024-07-26T17:26:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:26:18 smithi167 ceph-mon[37140]: pgmap v7586: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:26:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:26:18 smithi167 ceph-mon[37140]: osdmap e521: 12 total, 12 up, 12 in 2024-07-26T17:26:20.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:26:20 smithi138 ceph-mon[36954]: pgmap v7588: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:26:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:26:20 smithi100 ceph-mon[32599]: pgmap v7588: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:26:20.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:26:20 smithi167 ceph-mon[37140]: pgmap v7588: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:26:22.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:26:22 smithi138 ceph-mon[36954]: pgmap v7589: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:26:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:26:22 smithi100 ceph-mon[32599]: pgmap v7589: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:26:22.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:26:22 smithi167 ceph-mon[37140]: pgmap v7589: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:26:24.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:26:24 smithi167 ceph-mon[37140]: pgmap v7590: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:26:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:26:24 smithi138 ceph-mon[36954]: pgmap v7590: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:26:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:26:24 smithi100 ceph-mon[32599]: pgmap v7590: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:26:24.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:26:24.882+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:26:24.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:26:24.883+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:26:24.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:26:24.883+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:26:24.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:26:24.883+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:26:24.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:26:24.883+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:26:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:26:26 smithi138 ceph-mon[36954]: pgmap v7591: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:26:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:26:26 smithi100 ceph-mon[32599]: pgmap v7591: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:26:26.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:26:26 smithi167 ceph-mon[37140]: pgmap v7591: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:26:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:26:28 smithi138 ceph-mon[36954]: pgmap v7592: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:26:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:26:28 smithi100 ceph-mon[32599]: pgmap v7592: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:26:28.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:26:28 smithi167 ceph-mon[37140]: pgmap v7592: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:26:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:26:30 smithi138 ceph-mon[36954]: pgmap v7593: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:26:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:26:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:26:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:26:30 smithi100 ceph-mon[32599]: pgmap v7593: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:26:30.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:26:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:26:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:26:30 smithi167 ceph-mon[37140]: pgmap v7593: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:26:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:26:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:26:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:26:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:26:31.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:26:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:26:31.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:26:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:26:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:26:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:26:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:26:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:26:31.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:26:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:26:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:26:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:26:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:26:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:26:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:26:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:26:32.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:26:32 smithi138 ceph-mon[36954]: pgmap v7594: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:26:32.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:26:32 smithi100 ceph-mon[32599]: pgmap v7594: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:26:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:26:32 smithi167 ceph-mon[37140]: pgmap v7594: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:26:34.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:26:34 smithi138 ceph-mon[36954]: pgmap v7595: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:26:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:26:34 smithi100 ceph-mon[32599]: pgmap v7595: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:26:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:26:34 smithi167 ceph-mon[37140]: pgmap v7595: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:26:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:26:36 smithi138 ceph-mon[36954]: pgmap v7596: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:26:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:26:36 smithi100 ceph-mon[32599]: pgmap v7596: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:26:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:26:36 smithi167 ceph-mon[37140]: pgmap v7596: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:26:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:26:38 smithi138 ceph-mon[36954]: pgmap v7597: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:26:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:26:38 smithi100 ceph-mon[32599]: pgmap v7597: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:26:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:26:38 smithi167 ceph-mon[37140]: pgmap v7597: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:26:39.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:26:39.898+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:26:39.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:26:39.898+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:26:39.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:26:39.898+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:26:39.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:26:39.898+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:26:39.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:26:39.898+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:26:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:26:40 smithi138 ceph-mon[36954]: pgmap v7598: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:26:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:26:40 smithi100 ceph-mon[32599]: pgmap v7598: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:26:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:26:40 smithi167 ceph-mon[37140]: pgmap v7598: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:26:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:26:42 smithi138 ceph-mon[36954]: pgmap v7599: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:26:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:26:42 smithi100 ceph-mon[32599]: pgmap v7599: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:26:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:26:42 smithi167 ceph-mon[37140]: pgmap v7599: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:26:44.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:26:44 smithi138 ceph-mon[36954]: pgmap v7600: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:26:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:26:44 smithi100 ceph-mon[32599]: pgmap v7600: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:26:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:26:44 smithi167 ceph-mon[37140]: pgmap v7600: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:26:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:26:46 smithi138 ceph-mon[36954]: pgmap v7601: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:26:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:26:46 smithi100 ceph-mon[32599]: pgmap v7601: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:26:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:26:46 smithi167 ceph-mon[37140]: pgmap v7601: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:26:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:26:48 smithi138 ceph-mon[36954]: pgmap v7602: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:26:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:26:48 smithi100 ceph-mon[32599]: pgmap v7602: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:26:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:26:48 smithi167 ceph-mon[37140]: pgmap v7602: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:26:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:26:50 smithi138 ceph-mon[36954]: pgmap v7603: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:26:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:26:50 smithi100 ceph-mon[32599]: pgmap v7603: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:26:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:26:50 smithi167 ceph-mon[37140]: pgmap v7603: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:26:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:26:52 smithi138 ceph-mon[36954]: pgmap v7604: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:26:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:26:52 smithi100 ceph-mon[32599]: pgmap v7604: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:26:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:26:52 smithi167 ceph-mon[37140]: pgmap v7604: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:26:54.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:26:54.907+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:26:54.909 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:26:54 smithi100 ceph-mon[32599]: pgmap v7605: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:26:54.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:26:54.908+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:26:54.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:26:54.908+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:26:54.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:26:54.908+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:26:54.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:26:54.908+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:26:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:26:54 smithi138 ceph-mon[36954]: pgmap v7605: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:26:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:26:54 smithi167 ceph-mon[37140]: pgmap v7605: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:26:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:26:56 smithi138 ceph-mon[36954]: pgmap v7606: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:26:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:26:56 smithi100 ceph-mon[32599]: pgmap v7606: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:26:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:26:56 smithi167 ceph-mon[37140]: pgmap v7606: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:26:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:26:58 smithi138 ceph-mon[36954]: pgmap v7607: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:26:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:26:58 smithi100 ceph-mon[32599]: pgmap v7607: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:26:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:26:58 smithi167 ceph-mon[37140]: pgmap v7607: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:27:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:27:00 smithi138 ceph-mon[36954]: pgmap v7608: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:27:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:27:00 smithi100 ceph-mon[32599]: pgmap v7608: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:27:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:27:00 smithi167 ceph-mon[37140]: pgmap v7608: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:27:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:27:02 smithi138 ceph-mon[36954]: pgmap v7609: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:27:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:27:02 smithi100 ceph-mon[32599]: pgmap v7609: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:27:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:27:02 smithi167 ceph-mon[37140]: pgmap v7609: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:27:04.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:27:04.877+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c004740 con 0x7fe5a81087a0 2024-07-26T17:27:04.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:27:04.877+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T17:27:04.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:27:04.877+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 104 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c070 con 0x7fe5a81087a0 2024-07-26T17:27:04.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:27:04.877+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 104 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T17:27:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:27:04 smithi138 ceph-mon[36954]: pgmap v7610: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:27:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:27:04 smithi100 ceph-mon[32599]: pgmap v7610: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:27:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:27:04 smithi167 ceph-mon[37140]: pgmap v7610: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:27:06.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:27:06 smithi138 ceph-mon[36954]: pgmap v7611: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:27:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:27:06 smithi100 ceph-mon[32599]: pgmap v7611: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:27:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:27:06 smithi167 ceph-mon[37140]: pgmap v7611: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:27:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:27:08 smithi138 ceph-mon[36954]: pgmap v7612: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:27:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:27:08 smithi138 ceph-mon[36954]: osdmap e522: 12 total, 12 up, 12 in 2024-07-26T17:27:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:27:08 smithi100 ceph-mon[32599]: pgmap v7612: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:27:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:27:08 smithi100 ceph-mon[32599]: osdmap e522: 12 total, 12 up, 12 in 2024-07-26T17:27:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:27:08 smithi167 ceph-mon[37140]: pgmap v7612: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:27:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:27:08 smithi167 ceph-mon[37140]: osdmap e522: 12 total, 12 up, 12 in 2024-07-26T17:27:09.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:27:09.919+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:27:09.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:27:09.919+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:27:09.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:27:09.919+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:27:09.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:27:09.919+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:27:09.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:27:09.919+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:27:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:27:10 smithi138 ceph-mon[36954]: pgmap v7614: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:27:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:27:10 smithi100 ceph-mon[32599]: pgmap v7614: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:27:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:27:10 smithi167 ceph-mon[37140]: pgmap v7614: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:27:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:27:12 smithi138 ceph-mon[36954]: pgmap v7615: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:27:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:27:12 smithi100 ceph-mon[32599]: pgmap v7615: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:27:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:27:12 smithi167 ceph-mon[37140]: pgmap v7615: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:27:14.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:27:14 smithi138 ceph-mon[36954]: pgmap v7616: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:27:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:27:14 smithi100 ceph-mon[32599]: pgmap v7616: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:27:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:27:14 smithi167 ceph-mon[37140]: pgmap v7616: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:27:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:27:16 smithi100 ceph-mon[32599]: pgmap v7617: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:27:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:27:16 smithi167 ceph-mon[37140]: pgmap v7617: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:27:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:27:16 smithi138 ceph-mon[36954]: pgmap v7617: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:27:18.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:27:18 smithi138 ceph-mon[36954]: pgmap v7618: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:27:18.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:27:18 smithi138 ceph-mon[36954]: osdmap e523: 12 total, 12 up, 12 in 2024-07-26T17:27:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:27:18 smithi100 ceph-mon[32599]: pgmap v7618: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:27:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:27:18 smithi100 ceph-mon[32599]: osdmap e523: 12 total, 12 up, 12 in 2024-07-26T17:27:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:27:18 smithi167 ceph-mon[37140]: pgmap v7618: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:27:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:27:18 smithi167 ceph-mon[37140]: osdmap e523: 12 total, 12 up, 12 in 2024-07-26T17:27:20.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:27:20 smithi138 ceph-mon[36954]: pgmap v7620: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:27:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:27:20 smithi100 ceph-mon[32599]: pgmap v7620: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:27:20.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:27:20 smithi167 ceph-mon[37140]: pgmap v7620: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:27:22.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:27:22 smithi138 ceph-mon[36954]: pgmap v7621: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:27:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:27:22 smithi100 ceph-mon[32599]: pgmap v7621: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:27:22.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:27:22 smithi167 ceph-mon[37140]: pgmap v7621: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:27:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:27:24 smithi138 ceph-mon[36954]: pgmap v7622: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:27:24.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:27:24 smithi100 ceph-mon[32599]: pgmap v7622: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:27:24.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:27:24 smithi167 ceph-mon[37140]: pgmap v7622: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:27:24.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:27:24.923+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:27:24.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:27:24.924+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:27:24.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:27:24.924+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:27:24.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:27:24.924+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:27:24.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:27:24.924+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:27:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:27:26 smithi138 ceph-mon[36954]: pgmap v7623: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:27:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:27:26 smithi100 ceph-mon[32599]: pgmap v7623: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:27:26.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:27:26 smithi167 ceph-mon[37140]: pgmap v7623: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:27:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:27:28 smithi138 ceph-mon[36954]: pgmap v7624: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:27:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:27:28 smithi100 ceph-mon[32599]: pgmap v7624: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:27:28.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:27:28 smithi167 ceph-mon[37140]: pgmap v7624: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:27:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:27:30 smithi138 ceph-mon[36954]: pgmap v7625: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:27:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:27:30 smithi100 ceph-mon[32599]: pgmap v7625: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:27:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:27:30 smithi167 ceph-mon[37140]: pgmap v7625: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:27:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:27:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:27:31.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:27:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:27:31.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:27:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:27:31.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:27:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:27:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:27:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:27:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:27:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:27:31.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:27:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:27:31.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:27:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:27:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:27:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:27:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:27:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:27:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:27:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:27:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:27:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:27:32.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:27:32 smithi138 ceph-mon[36954]: pgmap v7626: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:27:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:27:32 smithi100 ceph-mon[32599]: pgmap v7626: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:27:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:27:32 smithi167 ceph-mon[37140]: pgmap v7626: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:27:34.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:27:34 smithi138 ceph-mon[36954]: pgmap v7627: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:27:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:27:34 smithi100 ceph-mon[32599]: pgmap v7627: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:27:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:27:34 smithi167 ceph-mon[37140]: pgmap v7627: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:27:36.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:27:36 smithi138 ceph-mon[36954]: pgmap v7628: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:27:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:27:36 smithi100 ceph-mon[32599]: pgmap v7628: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:27:36.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:27:36 smithi167 ceph-mon[37140]: pgmap v7628: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:27:38.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:27:38 smithi138 ceph-mon[36954]: pgmap v7629: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:27:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:27:38 smithi100 ceph-mon[32599]: pgmap v7629: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:27:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:27:38 smithi167 ceph-mon[37140]: pgmap v7629: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:27:39.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:27:39.927+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:27:39.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:27:39.928+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:27:39.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:27:39.928+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:27:39.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:27:39.928+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:27:39.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:27:39.928+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:27:40.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:27:40 smithi138 ceph-mon[36954]: pgmap v7630: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:27:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:27:40 smithi100 ceph-mon[32599]: pgmap v7630: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:27:40.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:27:40 smithi167 ceph-mon[37140]: pgmap v7630: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:27:42.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:27:42 smithi138 ceph-mon[36954]: pgmap v7631: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:27:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:27:42 smithi100 ceph-mon[32599]: pgmap v7631: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:27:42.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:27:42 smithi167 ceph-mon[37140]: pgmap v7631: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:27:44.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:27:44 smithi138 ceph-mon[36954]: pgmap v7632: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:27:44.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:27:44 smithi100 ceph-mon[32599]: pgmap v7632: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:27:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:27:44 smithi167 ceph-mon[37140]: pgmap v7632: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:27:46.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:27:46 smithi138 ceph-mon[36954]: pgmap v7633: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:27:46.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:27:46 smithi100 ceph-mon[32599]: pgmap v7633: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:27:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:27:46 smithi167 ceph-mon[37140]: pgmap v7633: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:27:48.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:27:48 smithi138 ceph-mon[36954]: pgmap v7634: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:27:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:27:48 smithi100 ceph-mon[32599]: pgmap v7634: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:27:48.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:27:48 smithi167 ceph-mon[37140]: pgmap v7634: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:27:50.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:27:50 smithi138 ceph-mon[36954]: pgmap v7635: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:27:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:27:50 smithi100 ceph-mon[32599]: pgmap v7635: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:27:50.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:27:50 smithi167 ceph-mon[37140]: pgmap v7635: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:27:52.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:27:52 smithi138 ceph-mon[36954]: pgmap v7636: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:27:52.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:27:52 smithi100 ceph-mon[32599]: pgmap v7636: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:27:52.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:27:52 smithi167 ceph-mon[37140]: pgmap v7636: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:27:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:27:54 smithi138 ceph-mon[36954]: pgmap v7637: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:27:54.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:27:54 smithi100 ceph-mon[32599]: pgmap v7637: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:27:54.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:27:54 smithi167 ceph-mon[37140]: pgmap v7637: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:27:54.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:27:54.942+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:27:54.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:27:54.943+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:27:54.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:27:54.943+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:27:54.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:27:54.943+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:27:54.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:27:54.943+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:27:56.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:27:56 smithi138 ceph-mon[36954]: pgmap v7638: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:27:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:27:56 smithi100 ceph-mon[32599]: pgmap v7638: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:27:56.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:27:56 smithi167 ceph-mon[37140]: pgmap v7638: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:27:58.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:27:58 smithi138 ceph-mon[36954]: pgmap v7639: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:27:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:27:58 smithi100 ceph-mon[32599]: pgmap v7639: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:27:58.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:27:58 smithi167 ceph-mon[37140]: pgmap v7639: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:28:00.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:28:00 smithi138 ceph-mon[36954]: pgmap v7640: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:28:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:28:00 smithi100 ceph-mon[32599]: pgmap v7640: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:28:00.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:28:00 smithi167 ceph-mon[37140]: pgmap v7640: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:28:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:28:02 smithi138 ceph-mon[36954]: pgmap v7641: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:28:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:28:02 smithi100 ceph-mon[32599]: pgmap v7641: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:28:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:28:02 smithi167 ceph-mon[37140]: pgmap v7641: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:28:04.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:28:04 smithi138 ceph-mon[36954]: pgmap v7642: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:28:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:28:04 smithi100 ceph-mon[32599]: pgmap v7642: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:28:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:28:04 smithi167 ceph-mon[37140]: pgmap v7642: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:28:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:28:06 smithi138 ceph-mon[36954]: pgmap v7643: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:28:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:28:06 smithi100 ceph-mon[32599]: pgmap v7643: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:28:06.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:28:06 smithi167 ceph-mon[37140]: pgmap v7643: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:28:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:28:08 smithi138 ceph-mon[36954]: pgmap v7644: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:28:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:28:08 smithi138 ceph-mon[36954]: osdmap e524: 12 total, 12 up, 12 in 2024-07-26T17:28:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:28:08 smithi100 ceph-mon[32599]: pgmap v7644: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:28:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:28:08 smithi100 ceph-mon[32599]: osdmap e524: 12 total, 12 up, 12 in 2024-07-26T17:28:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:28:08 smithi167 ceph-mon[37140]: pgmap v7644: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:28:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:28:08 smithi167 ceph-mon[37140]: osdmap e524: 12 total, 12 up, 12 in 2024-07-26T17:28:09.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:28:09.959+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:28:09.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:28:09.960+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:28:09.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:28:09.960+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:28:09.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:28:09.960+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:28:09.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:28:09.960+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:28:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:28:10 smithi138 ceph-mon[36954]: pgmap v7646: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:28:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:28:10 smithi100 ceph-mon[32599]: pgmap v7646: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:28:10.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:28:10 smithi167 ceph-mon[37140]: pgmap v7646: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:28:12.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:28:12 smithi138 ceph-mon[36954]: pgmap v7647: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:28:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:28:12 smithi100 ceph-mon[32599]: pgmap v7647: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:28:12.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:28:12 smithi167 ceph-mon[37140]: pgmap v7647: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:28:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:28:14 smithi138 ceph-mon[36954]: pgmap v7648: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:28:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:28:14 smithi100 ceph-mon[32599]: pgmap v7648: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:28:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:28:14 smithi167 ceph-mon[37140]: pgmap v7648: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:28:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:28:16 smithi138 ceph-mon[36954]: pgmap v7649: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:28:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:28:16 smithi100 ceph-mon[32599]: pgmap v7649: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:28:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:28:16 smithi167 ceph-mon[37140]: pgmap v7649: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:28:18.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:28:18 smithi138 ceph-mon[36954]: pgmap v7650: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:28:18.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:28:18 smithi138 ceph-mon[36954]: osdmap e525: 12 total, 12 up, 12 in 2024-07-26T17:28:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:28:18 smithi100 ceph-mon[32599]: pgmap v7650: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:28:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:28:18 smithi100 ceph-mon[32599]: osdmap e525: 12 total, 12 up, 12 in 2024-07-26T17:28:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:28:18 smithi167 ceph-mon[37140]: pgmap v7650: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:28:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:28:18 smithi167 ceph-mon[37140]: osdmap e525: 12 total, 12 up, 12 in 2024-07-26T17:28:20.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:28:20 smithi138 ceph-mon[36954]: pgmap v7652: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:28:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:28:20 smithi100 ceph-mon[32599]: pgmap v7652: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:28:20.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:28:20 smithi167 ceph-mon[37140]: pgmap v7652: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:28:22.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:28:22 smithi138 ceph-mon[36954]: pgmap v7653: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:28:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:28:22 smithi100 ceph-mon[32599]: pgmap v7653: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:28:22.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:28:22 smithi167 ceph-mon[37140]: pgmap v7653: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:28:24.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:28:24 smithi138 ceph-mon[36954]: pgmap v7654: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:28:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:28:24 smithi100 ceph-mon[32599]: pgmap v7654: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:28:24.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:28:24 smithi167 ceph-mon[37140]: pgmap v7654: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:28:24.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:28:24.975+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:28:24.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:28:24.975+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:28:24.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:28:24.975+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:28:24.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:28:24.975+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:28:24.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:28:24.975+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:28:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:28:26 smithi138 ceph-mon[36954]: pgmap v7655: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:28:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:28:26 smithi100 ceph-mon[32599]: pgmap v7655: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:28:26.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:28:26 smithi167 ceph-mon[37140]: pgmap v7655: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:28:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:28:28 smithi138 ceph-mon[36954]: pgmap v7656: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:28:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:28:28 smithi100 ceph-mon[32599]: pgmap v7656: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:28:28.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:28:28 smithi167 ceph-mon[37140]: pgmap v7656: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:28:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:28:30 smithi138 ceph-mon[36954]: pgmap v7657: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:28:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:28:30 smithi100 ceph-mon[32599]: pgmap v7657: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:28:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:28:30 smithi167 ceph-mon[37140]: pgmap v7657: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:28:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:28:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:28:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:28:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:28:31.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:28:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:28:32.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:28:32 smithi138 ceph-mon[36954]: pgmap v7658: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:28:32.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:28:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:28:32.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:28:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:28:32.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:28:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:28:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:28:32 smithi100 ceph-mon[32599]: pgmap v7658: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:28:32.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:28:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:28:32.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:28:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:28:32.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:28:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:28:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:28:32 smithi167 ceph-mon[37140]: pgmap v7658: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:28:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:28:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:28:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:28:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:28:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:28:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:28:34.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:28:34 smithi138 ceph-mon[36954]: pgmap v7659: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:28:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:28:34 smithi100 ceph-mon[32599]: pgmap v7659: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:28:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:28:34 smithi167 ceph-mon[37140]: pgmap v7659: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:28:36.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:28:36 smithi138 ceph-mon[36954]: pgmap v7660: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:28:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:28:36 smithi100 ceph-mon[32599]: pgmap v7660: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:28:36.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:28:36 smithi167 ceph-mon[37140]: pgmap v7660: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:28:38.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:28:38 smithi138 ceph-mon[36954]: pgmap v7661: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:28:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:28:38 smithi100 ceph-mon[32599]: pgmap v7661: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:28:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:28:38 smithi167 ceph-mon[37140]: pgmap v7661: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:28:39.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:28:39.991+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:28:39.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:28:39.991+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:28:39.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:28:39.991+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:28:39.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:28:39.991+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:28:39.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:28:39.991+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:28:40.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:28:40 smithi138 ceph-mon[36954]: pgmap v7662: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:28:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:28:40 smithi100 ceph-mon[32599]: pgmap v7662: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:28:40.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:28:40 smithi167 ceph-mon[37140]: pgmap v7662: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:28:42.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:28:42 smithi138 ceph-mon[36954]: pgmap v7663: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:28:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:28:42 smithi100 ceph-mon[32599]: pgmap v7663: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:28:42.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:28:42 smithi167 ceph-mon[37140]: pgmap v7663: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:28:44.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:28:44 smithi138 ceph-mon[36954]: pgmap v7664: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:28:44.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:28:44 smithi100 ceph-mon[32599]: pgmap v7664: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:28:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:28:44 smithi167 ceph-mon[37140]: pgmap v7664: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:28:46.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:28:46 smithi138 ceph-mon[36954]: pgmap v7665: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:28:46.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:28:46 smithi100 ceph-mon[32599]: pgmap v7665: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:28:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:28:46 smithi167 ceph-mon[37140]: pgmap v7665: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:28:48.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:28:48 smithi138 ceph-mon[36954]: pgmap v7666: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:28:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:28:48 smithi100 ceph-mon[32599]: pgmap v7666: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:28:48.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:28:48 smithi167 ceph-mon[37140]: pgmap v7666: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:28:50.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:28:50 smithi138 ceph-mon[36954]: pgmap v7667: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:28:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:28:50 smithi100 ceph-mon[32599]: pgmap v7667: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:28:50.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:28:50 smithi167 ceph-mon[37140]: pgmap v7667: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:28:52.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:28:52 smithi138 ceph-mon[36954]: pgmap v7668: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:28:52.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:28:52 smithi100 ceph-mon[32599]: pgmap v7668: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:28:52.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:28:52 smithi167 ceph-mon[37140]: pgmap v7668: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:28:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:28:54 smithi138 ceph-mon[36954]: pgmap v7669: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:28:54.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:28:54 smithi100 ceph-mon[32599]: pgmap v7669: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:28:54.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:28:54 smithi167 ceph-mon[37140]: pgmap v7669: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:28:55.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:28:55.000+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:28:55.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:28:55.001+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:28:55.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:28:55.001+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:28:55.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:28:55.001+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:28:55.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:28:55.001+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:28:56.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:28:56 smithi138 ceph-mon[36954]: pgmap v7670: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:28:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:28:56 smithi100 ceph-mon[32599]: pgmap v7670: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:28:56.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:28:56 smithi167 ceph-mon[37140]: pgmap v7670: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:28:58.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:28:58 smithi138 ceph-mon[36954]: pgmap v7671: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:28:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:28:58 smithi100 ceph-mon[32599]: pgmap v7671: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:28:58.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:28:58 smithi167 ceph-mon[37140]: pgmap v7671: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:29:00.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:00 smithi138 ceph-mon[36954]: pgmap v7672: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:29:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:00 smithi100 ceph-mon[32599]: pgmap v7672: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:29:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:00 smithi167 ceph-mon[37140]: pgmap v7672: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:29:02.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:02 smithi138 ceph-mon[36954]: pgmap v7673: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:29:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:02 smithi100 ceph-mon[32599]: pgmap v7673: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:29:02.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:02 smithi167 ceph-mon[37140]: pgmap v7673: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:29:04.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:04 smithi138 ceph-mon[36954]: pgmap v7674: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:29:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:04 smithi100 ceph-mon[32599]: pgmap v7674: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:29:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:04 smithi167 ceph-mon[37140]: pgmap v7674: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:29:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:06 smithi138 ceph-mon[36954]: pgmap v7675: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:29:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:06 smithi100 ceph-mon[32599]: pgmap v7675: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:29:06.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:06 smithi167 ceph-mon[37140]: pgmap v7675: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:29:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:08 smithi100 ceph-mon[32599]: pgmap v7676: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:29:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:08 smithi100 ceph-mon[32599]: osdmap e526: 12 total, 12 up, 12 in 2024-07-26T17:29:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:08 smithi167 ceph-mon[37140]: pgmap v7676: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:29:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:08 smithi167 ceph-mon[37140]: osdmap e526: 12 total, 12 up, 12 in 2024-07-26T17:29:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:08 smithi138 ceph-mon[36954]: pgmap v7676: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:29:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:08 smithi138 ceph-mon[36954]: osdmap e526: 12 total, 12 up, 12 in 2024-07-26T17:29:10.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:29:10.017+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:29:10.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:29:10.017+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:29:10.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:29:10.017+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:29:10.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:29:10.017+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:29:10.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:29:10.017+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:29:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:10 smithi100 ceph-mon[32599]: pgmap v7678: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:29:10.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:10 smithi167 ceph-mon[37140]: pgmap v7678: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:29:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:10 smithi138 ceph-mon[36954]: pgmap v7678: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:29:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:12 smithi100 ceph-mon[32599]: pgmap v7679: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:29:12.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:12 smithi167 ceph-mon[37140]: pgmap v7679: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:29:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:12 smithi138 ceph-mon[36954]: pgmap v7679: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:29:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:14 smithi100 ceph-mon[32599]: pgmap v7680: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:29:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:14 smithi167 ceph-mon[37140]: pgmap v7680: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:29:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:14 smithi138 ceph-mon[36954]: pgmap v7680: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:29:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:16 smithi100 ceph-mon[32599]: pgmap v7681: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:29:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:16 smithi167 ceph-mon[37140]: pgmap v7681: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:29:16.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:16 smithi138 ceph-mon[36954]: pgmap v7681: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:29:18.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:18 smithi138 ceph-mon[36954]: pgmap v7682: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:29:18.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:18 smithi138 ceph-mon[36954]: osdmap e527: 12 total, 12 up, 12 in 2024-07-26T17:29:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:18 smithi100 ceph-mon[32599]: pgmap v7682: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:29:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:18 smithi100 ceph-mon[32599]: osdmap e527: 12 total, 12 up, 12 in 2024-07-26T17:29:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:18 smithi167 ceph-mon[37140]: pgmap v7682: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:29:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:18 smithi167 ceph-mon[37140]: osdmap e527: 12 total, 12 up, 12 in 2024-07-26T17:29:20.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:20 smithi138 ceph-mon[36954]: pgmap v7684: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:29:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:20 smithi100 ceph-mon[32599]: pgmap v7684: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:29:20.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:20 smithi167 ceph-mon[37140]: pgmap v7684: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:29:22.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:22 smithi138 ceph-mon[36954]: pgmap v7685: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:29:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:22 smithi100 ceph-mon[32599]: pgmap v7685: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:29:22.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:22 smithi167 ceph-mon[37140]: pgmap v7685: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:29:24.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:24 smithi138 ceph-mon[36954]: pgmap v7686: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:29:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:24 smithi100 ceph-mon[32599]: pgmap v7686: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:29:24.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:24 smithi167 ceph-mon[37140]: pgmap v7686: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:29:25.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:29:25.025+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:29:25.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:29:25.025+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:29:25.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:29:25.025+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:29:25.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:29:25.025+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:29:25.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:29:25.025+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:29:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:26 smithi138 ceph-mon[36954]: pgmap v7687: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:29:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:26 smithi100 ceph-mon[32599]: pgmap v7687: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:29:26.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:26 smithi167 ceph-mon[37140]: pgmap v7687: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:29:28.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:28 smithi138 ceph-mon[36954]: pgmap v7688: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:29:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:28 smithi100 ceph-mon[32599]: pgmap v7688: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:29:28.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:28 smithi167 ceph-mon[37140]: pgmap v7688: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:29:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:30 smithi138 ceph-mon[36954]: pgmap v7689: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:29:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:30 smithi100 ceph-mon[32599]: pgmap v7689: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:29:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:30 smithi167 ceph-mon[37140]: pgmap v7689: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:29:32.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:32 smithi138 ceph-mon[36954]: pgmap v7690: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:29:32.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:29:32.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:29:32.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:29:32.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:29:32.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:29:32.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:29:32.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:29:32.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:29:32.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:29:32.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:29:32.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:29:32.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:29:32.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:29:32.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:29:32.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:29:32.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:29:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:32 smithi100 ceph-mon[32599]: pgmap v7690: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:29:32.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:29:32.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:29:32.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:29:32.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:29:32.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:29:32.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:29:32.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:29:32.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:29:32.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:29:32.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:29:32.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:29:32.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:29:32.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:29:32.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:29:32.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:29:32.747 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:29:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:32 smithi167 ceph-mon[37140]: pgmap v7690: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:29:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:29:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:29:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:29:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:29:32.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:29:32.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:29:32.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:29:32.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:29:32.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:29:32.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:29:32.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:29:32.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:29:32.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:29:32.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:29:32.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:29:32.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:29:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:33 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T17:29:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:33 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T17:29:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:33 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T17:29:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:33 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T17:29:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:33 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T17:29:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:33 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T17:29:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:33 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T17:29:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:33 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T17:29:33.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:33 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T17:29:33.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:33 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T17:29:33.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:33 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T17:29:33.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:33 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T17:29:33.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:33 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T17:29:33.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:33 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T17:29:33.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:33 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T17:29:33.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:33 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T17:29:33.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:33 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T17:29:33.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:33 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T17:29:34.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:34 smithi138 ceph-mon[36954]: pgmap v7691: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:29:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:34 smithi100 ceph-mon[32599]: pgmap v7691: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:29:34.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:34 smithi167 ceph-mon[37140]: pgmap v7691: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:29:36.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:36 smithi138 ceph-mon[36954]: pgmap v7692: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:29:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:36 smithi100 ceph-mon[32599]: pgmap v7692: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:29:36.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:36 smithi167 ceph-mon[37140]: pgmap v7692: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:29:38.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:38 smithi138 ceph-mon[36954]: pgmap v7693: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:29:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:38 smithi100 ceph-mon[32599]: pgmap v7693: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:29:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:38 smithi167 ceph-mon[37140]: pgmap v7693: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:29:40.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:29:40.039+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:29:40.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:29:40.040+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:29:40.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:29:40.040+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:29:40.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:29:40.040+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:29:40.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:29:40.040+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:29:40.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:40 smithi138 ceph-mon[36954]: pgmap v7694: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:29:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:40 smithi100 ceph-mon[32599]: pgmap v7694: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:29:40.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:40 smithi167 ceph-mon[37140]: pgmap v7694: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:29:42.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:42 smithi138 ceph-mon[36954]: pgmap v7695: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:29:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:42 smithi100 ceph-mon[32599]: pgmap v7695: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:29:42.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:42 smithi167 ceph-mon[37140]: pgmap v7695: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:29:44.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:44 smithi138 ceph-mon[36954]: pgmap v7696: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:29:44.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:44 smithi100 ceph-mon[32599]: pgmap v7696: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:29:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:44 smithi167 ceph-mon[37140]: pgmap v7696: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:29:46.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:46 smithi138 ceph-mon[36954]: pgmap v7697: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:29:46.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:46 smithi100 ceph-mon[32599]: pgmap v7697: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:29:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:46 smithi167 ceph-mon[37140]: pgmap v7697: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:29:48.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:48 smithi138 ceph-mon[36954]: pgmap v7698: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:29:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:48 smithi100 ceph-mon[32599]: pgmap v7698: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:29:48.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:48 smithi167 ceph-mon[37140]: pgmap v7698: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:29:50.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:50 smithi138 ceph-mon[36954]: pgmap v7699: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:29:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:50 smithi100 ceph-mon[32599]: pgmap v7699: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:29:50.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:50 smithi167 ceph-mon[37140]: pgmap v7699: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:29:52.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:52 smithi138 ceph-mon[36954]: pgmap v7700: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:29:52.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:52 smithi100 ceph-mon[32599]: pgmap v7700: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:29:52.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:52 smithi167 ceph-mon[37140]: pgmap v7700: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:29:54.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:54 smithi138 ceph-mon[36954]: pgmap v7701: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:29:54.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:54 smithi100 ceph-mon[32599]: pgmap v7701: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:29:54.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:54 smithi167 ceph-mon[37140]: pgmap v7701: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:29:55.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:29:55.055+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:29:55.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:29:55.056+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:29:55.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:29:55.056+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:29:55.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:29:55.056+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:29:55.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:29:55.056+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:29:56.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:56 smithi138 ceph-mon[36954]: pgmap v7702: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:29:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:56 smithi100 ceph-mon[32599]: pgmap v7702: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:29:56.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:56 smithi167 ceph-mon[37140]: pgmap v7702: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:29:58.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:29:58 smithi138 ceph-mon[36954]: pgmap v7703: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:29:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:29:58 smithi100 ceph-mon[32599]: pgmap v7703: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:29:58.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:29:58 smithi167 ceph-mon[37140]: pgmap v7703: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:30:00.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:30:00 smithi138 ceph-mon[36954]: pgmap v7704: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:30:00.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:30:00 smithi138 ceph-mon[36954]: overall HEALTH_OK 2024-07-26T17:30:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:30:00 smithi100 ceph-mon[32599]: pgmap v7704: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:30:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:30:00 smithi100 ceph-mon[32599]: overall HEALTH_OK 2024-07-26T17:30:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:30:00 smithi167 ceph-mon[37140]: pgmap v7704: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:30:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:30:00 smithi167 ceph-mon[37140]: overall HEALTH_OK 2024-07-26T17:30:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:30:02 smithi138 ceph-mon[36954]: pgmap v7705: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:30:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:30:02 smithi100 ceph-mon[32599]: pgmap v7705: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:30:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:30:02 smithi167 ceph-mon[37140]: pgmap v7705: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:30:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:30:04 smithi100 ceph-mon[32599]: pgmap v7706: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:30:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:30:04 smithi167 ceph-mon[37140]: pgmap v7706: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:30:04.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:30:04.902+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c004740 con 0x7fe5a81087a0 2024-07-26T17:30:04.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:30:04.902+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T17:30:04.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:30:04.902+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 105 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c050 con 0x7fe5a81087a0 2024-07-26T17:30:04.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:30:04.903+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 105 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T17:30:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:30:04 smithi138 ceph-mon[36954]: pgmap v7706: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:30:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:30:06 smithi100 ceph-mon[32599]: pgmap v7707: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:30:06.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:30:06 smithi167 ceph-mon[37140]: pgmap v7707: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:30:06.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:30:06 smithi138 ceph-mon[36954]: pgmap v7707: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:30:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:30:08 smithi100 ceph-mon[32599]: pgmap v7708: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:30:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:30:08 smithi100 ceph-mon[32599]: osdmap e528: 12 total, 12 up, 12 in 2024-07-26T17:30:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:30:08 smithi167 ceph-mon[37140]: pgmap v7708: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:30:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:30:08 smithi167 ceph-mon[37140]: osdmap e528: 12 total, 12 up, 12 in 2024-07-26T17:30:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:30:08 smithi138 ceph-mon[36954]: pgmap v7708: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:30:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:30:08 smithi138 ceph-mon[36954]: osdmap e528: 12 total, 12 up, 12 in 2024-07-26T17:30:10.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:30:10.071+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:30:10.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:30:10.072+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:30:10.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:30:10.072+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:30:10.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:30:10.072+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:30:10.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:30:10.072+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:30:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:30:10 smithi100 ceph-mon[32599]: pgmap v7710: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:30:10.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:30:10 smithi167 ceph-mon[37140]: pgmap v7710: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:30:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:30:10 smithi138 ceph-mon[36954]: pgmap v7710: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:30:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:30:12 smithi100 ceph-mon[32599]: pgmap v7711: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:30:12.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:30:12 smithi167 ceph-mon[37140]: pgmap v7711: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:30:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:30:12 smithi138 ceph-mon[36954]: pgmap v7711: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:30:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:30:14 smithi100 ceph-mon[32599]: pgmap v7712: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:30:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:30:14 smithi167 ceph-mon[37140]: pgmap v7712: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:30:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:30:14 smithi138 ceph-mon[36954]: pgmap v7712: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:30:16.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:30:16 smithi100 ceph-mon[32599]: pgmap v7713: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:30:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:30:16 smithi167 ceph-mon[37140]: pgmap v7713: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:30:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:30:16 smithi138 ceph-mon[36954]: pgmap v7713: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:30:18.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:30:18 smithi138 ceph-mon[36954]: pgmap v7714: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:30:18.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:30:18 smithi138 ceph-mon[36954]: osdmap e529: 12 total, 12 up, 12 in 2024-07-26T17:30:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:30:18 smithi100 ceph-mon[32599]: pgmap v7714: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:30:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:30:18 smithi100 ceph-mon[32599]: osdmap e529: 12 total, 12 up, 12 in 2024-07-26T17:30:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:30:18 smithi167 ceph-mon[37140]: pgmap v7714: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:30:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:30:18 smithi167 ceph-mon[37140]: osdmap e529: 12 total, 12 up, 12 in 2024-07-26T17:30:20.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:30:20 smithi138 ceph-mon[36954]: pgmap v7716: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:30:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:30:20 smithi100 ceph-mon[32599]: pgmap v7716: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:30:20.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:30:20 smithi167 ceph-mon[37140]: pgmap v7716: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:30:22.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:30:22 smithi138 ceph-mon[36954]: pgmap v7717: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:30:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:30:22 smithi100 ceph-mon[32599]: pgmap v7717: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:30:22.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:30:22 smithi167 ceph-mon[37140]: pgmap v7717: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:30:24.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:30:24 smithi138 ceph-mon[36954]: pgmap v7718: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:30:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:30:24 smithi100 ceph-mon[32599]: pgmap v7718: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:30:24.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:30:24 smithi167 ceph-mon[37140]: pgmap v7718: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:30:25.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:30:25.079+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:30:25.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:30:25.080+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:30:25.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:30:25.080+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:30:25.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:30:25.080+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:30:25.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:30:25.080+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:30:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:30:26 smithi138 ceph-mon[36954]: pgmap v7719: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:30:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:30:26 smithi100 ceph-mon[32599]: pgmap v7719: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:30:26.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:30:26 smithi167 ceph-mon[37140]: pgmap v7719: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:30:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:30:28 smithi138 ceph-mon[36954]: pgmap v7720: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:30:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:30:28 smithi100 ceph-mon[32599]: pgmap v7720: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:30:28.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:30:28 smithi167 ceph-mon[37140]: pgmap v7720: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:30:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:30:30 smithi138 ceph-mon[36954]: pgmap v7721: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:30:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:30:30 smithi100 ceph-mon[32599]: pgmap v7721: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:30:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:30:30 smithi167 ceph-mon[37140]: pgmap v7721: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:30:32.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:30:32 smithi138 ceph-mon[36954]: pgmap v7722: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:30:32.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:30:32 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:30:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:30:32 smithi100 ceph-mon[32599]: pgmap v7722: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:30:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:30:32 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:30:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:30:32 smithi167 ceph-mon[37140]: pgmap v7722: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:30:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:30:32 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:30:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:30:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:30:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:30:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:30:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:30:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:30:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:30:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:30:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:30:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:30:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:30:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:30:33.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:30:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:30:33.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:30:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:30:33.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:30:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:30:34.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:30:34 smithi138 ceph-mon[36954]: pgmap v7723: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:30:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:30:34 smithi100 ceph-mon[32599]: pgmap v7723: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:30:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:30:34 smithi167 ceph-mon[37140]: pgmap v7723: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:30:36.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:30:36 smithi138 ceph-mon[36954]: pgmap v7724: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:30:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:30:36 smithi100 ceph-mon[32599]: pgmap v7724: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:30:36.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:30:36 smithi167 ceph-mon[37140]: pgmap v7724: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:30:38.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:30:38 smithi138 ceph-mon[36954]: pgmap v7725: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:30:38.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:30:38 smithi100 ceph-mon[32599]: pgmap v7725: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:30:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:30:38 smithi167 ceph-mon[37140]: pgmap v7725: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:30:40.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:30:40.095+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:30:40.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:30:40.096+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:30:40.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:30:40.096+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:30:40.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:30:40.096+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:30:40.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:30:40.096+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:30:40.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:30:40 smithi138 ceph-mon[36954]: pgmap v7726: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:30:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:30:40 smithi100 ceph-mon[32599]: pgmap v7726: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:30:40.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:30:40 smithi167 ceph-mon[37140]: pgmap v7726: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:30:42.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:30:42 smithi138 ceph-mon[36954]: pgmap v7727: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:30:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:30:42 smithi100 ceph-mon[32599]: pgmap v7727: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:30:42.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:30:42 smithi167 ceph-mon[37140]: pgmap v7727: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:30:44.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:30:44 smithi138 ceph-mon[36954]: pgmap v7728: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:30:44.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:30:44 smithi100 ceph-mon[32599]: pgmap v7728: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:30:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:30:44 smithi167 ceph-mon[37140]: pgmap v7728: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:30:46.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:30:46 smithi138 ceph-mon[36954]: pgmap v7729: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:30:46.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:30:46 smithi100 ceph-mon[32599]: pgmap v7729: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:30:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:30:46 smithi167 ceph-mon[37140]: pgmap v7729: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:30:48.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:30:48 smithi138 ceph-mon[36954]: pgmap v7730: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:30:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:30:48 smithi100 ceph-mon[32599]: pgmap v7730: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:30:48.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:30:48 smithi167 ceph-mon[37140]: pgmap v7730: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:30:49.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:30:49.909+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T17:30:49.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:30:49.909+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c004740 con 0x7fe5a81087a0 2024-07-26T17:30:49.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:30:49.909+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 106 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T17:30:49.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:30:49.909+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 106 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe5a008c070 con 0x7fe5a81087a0 2024-07-26T17:30:50.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:30:50 smithi138 ceph-mon[36954]: pgmap v7731: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:30:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:30:50 smithi100 ceph-mon[32599]: pgmap v7731: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:30:50.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:30:50 smithi167 ceph-mon[37140]: pgmap v7731: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:30:52.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:30:52 smithi138 ceph-mon[36954]: pgmap v7732: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:30:52.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:30:52 smithi100 ceph-mon[32599]: pgmap v7732: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:30:52.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:30:52 smithi167 ceph-mon[37140]: pgmap v7732: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:30:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:30:54 smithi138 ceph-mon[36954]: pgmap v7733: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:30:54.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:30:54 smithi100 ceph-mon[32599]: pgmap v7733: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:30:54.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:30:54 smithi167 ceph-mon[37140]: pgmap v7733: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:30:55.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:30:55.109+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:30:55.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:30:55.110+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:30:55.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:30:55.110+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:30:55.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:30:55.110+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:30:55.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:30:55.110+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:30:56.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:30:56 smithi138 ceph-mon[36954]: pgmap v7734: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:30:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:30:56 smithi100 ceph-mon[32599]: pgmap v7734: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:30:56.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:30:56 smithi167 ceph-mon[37140]: pgmap v7734: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:30:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:30:58 smithi100 ceph-mon[32599]: pgmap v7735: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:30:58.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:30:58 smithi167 ceph-mon[37140]: pgmap v7735: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:30:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:30:58 smithi138 ceph-mon[36954]: pgmap v7735: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:31:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:31:00 smithi100 ceph-mon[32599]: pgmap v7736: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:31:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:31:00 smithi167 ceph-mon[37140]: pgmap v7736: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:31:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:31:00 smithi138 ceph-mon[36954]: pgmap v7736: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:31:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:31:02 smithi100 ceph-mon[32599]: pgmap v7737: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:31:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:31:02 smithi167 ceph-mon[37140]: pgmap v7737: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:31:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:31:02 smithi138 ceph-mon[36954]: pgmap v7737: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:31:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:31:04 smithi100 ceph-mon[32599]: pgmap v7738: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:31:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:31:04 smithi167 ceph-mon[37140]: pgmap v7738: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:31:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:31:04 smithi138 ceph-mon[36954]: pgmap v7738: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:31:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:31:06 smithi100 ceph-mon[32599]: pgmap v7739: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:31:06.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:31:06 smithi167 ceph-mon[37140]: pgmap v7739: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:31:06.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:31:06 smithi138 ceph-mon[36954]: pgmap v7739: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:31:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:31:08 smithi100 ceph-mon[32599]: pgmap v7740: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:31:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:31:08 smithi100 ceph-mon[32599]: osdmap e530: 12 total, 12 up, 12 in 2024-07-26T17:31:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:31:08 smithi167 ceph-mon[37140]: pgmap v7740: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:31:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:31:08 smithi167 ceph-mon[37140]: osdmap e530: 12 total, 12 up, 12 in 2024-07-26T17:31:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:31:08 smithi138 ceph-mon[36954]: pgmap v7740: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:31:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:31:08 smithi138 ceph-mon[36954]: osdmap e530: 12 total, 12 up, 12 in 2024-07-26T17:31:10.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:31:10.119+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:31:10.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:31:10.120+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:31:10.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:31:10.120+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:31:10.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:31:10.120+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:31:10.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:31:10.120+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:31:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:31:10 smithi100 ceph-mon[32599]: pgmap v7742: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:31:10.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:31:10 smithi167 ceph-mon[37140]: pgmap v7742: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:31:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:31:10 smithi138 ceph-mon[36954]: pgmap v7742: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:31:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:31:12 smithi100 ceph-mon[32599]: pgmap v7743: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:31:12.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:31:12 smithi167 ceph-mon[37140]: pgmap v7743: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:31:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:31:12 smithi138 ceph-mon[36954]: pgmap v7743: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:31:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:31:14 smithi100 ceph-mon[32599]: pgmap v7744: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:31:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:31:14 smithi167 ceph-mon[37140]: pgmap v7744: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:31:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:31:14 smithi138 ceph-mon[36954]: pgmap v7744: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:31:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:31:16 smithi100 ceph-mon[32599]: pgmap v7745: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:31:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:31:16 smithi167 ceph-mon[37140]: pgmap v7745: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:31:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:31:16 smithi138 ceph-mon[36954]: pgmap v7745: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:31:18.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:31:18 smithi138 ceph-mon[36954]: pgmap v7746: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:31:18.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:31:18 smithi138 ceph-mon[36954]: osdmap e531: 12 total, 12 up, 12 in 2024-07-26T17:31:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:31:18 smithi100 ceph-mon[32599]: pgmap v7746: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:31:18.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:31:18 smithi100 ceph-mon[32599]: osdmap e531: 12 total, 12 up, 12 in 2024-07-26T17:31:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:31:18 smithi167 ceph-mon[37140]: pgmap v7746: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:31:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:31:18 smithi167 ceph-mon[37140]: osdmap e531: 12 total, 12 up, 12 in 2024-07-26T17:31:20.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:31:20 smithi138 ceph-mon[36954]: pgmap v7748: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T17:31:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:31:20 smithi100 ceph-mon[32599]: pgmap v7748: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T17:31:20.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:31:20 smithi167 ceph-mon[37140]: pgmap v7748: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T17:31:22.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:31:22 smithi138 ceph-mon[36954]: pgmap v7749: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T17:31:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:31:22 smithi100 ceph-mon[32599]: pgmap v7749: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T17:31:22.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:31:22 smithi167 ceph-mon[37140]: pgmap v7749: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T17:31:24.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:31:24 smithi138 ceph-mon[36954]: pgmap v7750: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T17:31:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:31:24 smithi100 ceph-mon[32599]: pgmap v7750: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T17:31:24.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:31:24 smithi167 ceph-mon[37140]: pgmap v7750: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T17:31:25.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:31:25.135+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:31:25.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:31:25.135+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:31:25.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:31:25.135+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:31:25.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:31:25.135+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:31:25.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:31:25.135+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:31:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:31:26 smithi138 ceph-mon[36954]: pgmap v7751: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:31:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:31:26 smithi100 ceph-mon[32599]: pgmap v7751: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:31:26.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:31:26 smithi167 ceph-mon[37140]: pgmap v7751: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:31:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:31:28 smithi138 ceph-mon[36954]: pgmap v7752: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:31:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:31:28 smithi100 ceph-mon[32599]: pgmap v7752: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:31:28.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:31:28 smithi167 ceph-mon[37140]: pgmap v7752: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:31:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:31:30 smithi138 ceph-mon[36954]: pgmap v7753: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:31:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:31:30 smithi100 ceph-mon[32599]: pgmap v7753: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:31:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:31:30 smithi167 ceph-mon[37140]: pgmap v7753: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:31:32.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:31:32 smithi138 ceph-mon[36954]: pgmap v7754: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:31:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:31:32 smithi100 ceph-mon[32599]: pgmap v7754: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:31:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:31:32 smithi167 ceph-mon[37140]: pgmap v7754: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:31:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:31:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:31:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:31:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:31:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:31:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:31:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:31:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:31:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:31:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:31:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:31:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:31:33.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:31:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:31:33.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:31:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:31:33.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:31:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:31:33.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:31:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:31:33.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:31:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:31:33.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:31:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:31:34.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:31:34 smithi138 ceph-mon[36954]: pgmap v7755: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:31:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:31:34 smithi100 ceph-mon[32599]: pgmap v7755: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:31:34.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:31:34 smithi167 ceph-mon[37140]: pgmap v7755: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:31:36.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:31:36 smithi138 ceph-mon[36954]: pgmap v7756: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:31:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:31:36 smithi100 ceph-mon[32599]: pgmap v7756: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:31:36.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:31:36 smithi167 ceph-mon[37140]: pgmap v7756: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:31:38.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:31:38 smithi138 ceph-mon[36954]: pgmap v7757: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:31:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:31:38 smithi100 ceph-mon[32599]: pgmap v7757: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:31:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:31:38 smithi167 ceph-mon[37140]: pgmap v7757: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:31:40.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:31:40.138+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:31:40.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:31:40.139+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:31:40.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:31:40.139+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:31:40.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:31:40.139+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:31:40.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:31:40.139+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:31:40.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:31:40 smithi138 ceph-mon[36954]: pgmap v7758: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:31:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:31:40 smithi100 ceph-mon[32599]: pgmap v7758: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:31:40.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:31:40 smithi167 ceph-mon[37140]: pgmap v7758: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:31:42.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:31:42 smithi138 ceph-mon[36954]: pgmap v7759: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:31:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:31:42 smithi100 ceph-mon[32599]: pgmap v7759: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:31:42.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:31:42 smithi167 ceph-mon[37140]: pgmap v7759: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:31:44.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:31:44 smithi138 ceph-mon[36954]: pgmap v7760: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:31:44.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:31:44 smithi100 ceph-mon[32599]: pgmap v7760: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:31:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:31:44 smithi167 ceph-mon[37140]: pgmap v7760: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:31:46.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:31:46 smithi100 ceph-mon[32599]: pgmap v7761: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:31:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:31:46 smithi167 ceph-mon[37140]: pgmap v7761: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:31:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:31:46 smithi138 ceph-mon[36954]: pgmap v7761: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:31:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:31:48 smithi100 ceph-mon[32599]: pgmap v7762: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:31:48.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:31:48 smithi167 ceph-mon[37140]: pgmap v7762: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:31:48.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:31:48 smithi138 ceph-mon[36954]: pgmap v7762: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:31:50.743 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:31:50 smithi138 ceph-mon[36954]: pgmap v7763: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:31:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:31:50 smithi100 ceph-mon[32599]: pgmap v7763: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:31:50.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:31:50 smithi167 ceph-mon[37140]: pgmap v7763: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:31:52.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:31:52 smithi100 ceph-mon[32599]: pgmap v7764: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:31:52.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:31:52 smithi167 ceph-mon[37140]: pgmap v7764: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:31:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:31:52 smithi138 ceph-mon[36954]: pgmap v7764: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:31:54.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:31:54 smithi100 ceph-mon[32599]: pgmap v7765: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:31:54.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:31:54 smithi167 ceph-mon[37140]: pgmap v7765: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:31:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:31:54 smithi138 ceph-mon[36954]: pgmap v7765: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:31:55.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:31:55.148+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:31:55.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:31:55.149+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:31:55.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:31:55.149+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:31:55.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:31:55.149+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:31:55.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:31:55.149+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:31:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:31:56 smithi100 ceph-mon[32599]: pgmap v7766: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:31:56.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:31:56 smithi167 ceph-mon[37140]: pgmap v7766: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:31:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:31:56 smithi138 ceph-mon[36954]: pgmap v7766: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:31:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:31:58 smithi100 ceph-mon[32599]: pgmap v7767: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:31:58.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:31:58 smithi167 ceph-mon[37140]: pgmap v7767: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:31:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:31:58 smithi138 ceph-mon[36954]: pgmap v7767: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:32:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:00 smithi100 ceph-mon[32599]: pgmap v7768: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:32:00.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:00 smithi167 ceph-mon[37140]: pgmap v7768: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:32:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:00 smithi138 ceph-mon[36954]: pgmap v7768: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:32:02.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:02 smithi100 ceph-mon[32599]: pgmap v7769: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:32:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:02 smithi167 ceph-mon[37140]: pgmap v7769: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:32:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:02 smithi138 ceph-mon[36954]: pgmap v7769: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:32:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:04 smithi100 ceph-mon[32599]: pgmap v7770: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:32:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:04 smithi167 ceph-mon[37140]: pgmap v7770: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:32:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:04 smithi138 ceph-mon[36954]: pgmap v7770: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:32:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:06 smithi100 ceph-mon[32599]: pgmap v7771: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:32:06.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:06 smithi167 ceph-mon[37140]: pgmap v7771: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:32:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:06 smithi138 ceph-mon[36954]: pgmap v7771: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:32:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:08 smithi100 ceph-mon[32599]: pgmap v7772: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:32:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:08 smithi100 ceph-mon[32599]: osdmap e532: 12 total, 12 up, 12 in 2024-07-26T17:32:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:08 smithi167 ceph-mon[37140]: pgmap v7772: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:32:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:08 smithi167 ceph-mon[37140]: osdmap e532: 12 total, 12 up, 12 in 2024-07-26T17:32:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:08 smithi138 ceph-mon[36954]: pgmap v7772: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:32:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:08 smithi138 ceph-mon[36954]: osdmap e532: 12 total, 12 up, 12 in 2024-07-26T17:32:10.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:32:10.164+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:32:10.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:32:10.165+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:32:10.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:32:10.165+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:32:10.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:32:10.165+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:32:10.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:32:10.165+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:32:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:10 smithi100 ceph-mon[32599]: pgmap v7774: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:32:10.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:10 smithi167 ceph-mon[37140]: pgmap v7774: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:32:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:10 smithi138 ceph-mon[36954]: pgmap v7774: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:32:12.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:12 smithi167 ceph-mon[37140]: pgmap v7775: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:32:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:12 smithi138 ceph-mon[36954]: pgmap v7775: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:32:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:12 smithi100 ceph-mon[32599]: pgmap v7775: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:32:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:14 smithi167 ceph-mon[37140]: pgmap v7776: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:32:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:14 smithi138 ceph-mon[36954]: pgmap v7776: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:32:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:14 smithi100 ceph-mon[32599]: pgmap v7776: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:32:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:16 smithi167 ceph-mon[37140]: pgmap v7777: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:32:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:16 smithi138 ceph-mon[36954]: pgmap v7777: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:32:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:16 smithi100 ceph-mon[32599]: pgmap v7777: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:32:18.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:18 smithi138 ceph-mon[36954]: pgmap v7778: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:32:18.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:18 smithi138 ceph-mon[36954]: osdmap e533: 12 total, 12 up, 12 in 2024-07-26T17:32:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:18 smithi100 ceph-mon[32599]: pgmap v7778: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:32:18.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:18 smithi100 ceph-mon[32599]: osdmap e533: 12 total, 12 up, 12 in 2024-07-26T17:32:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:18 smithi167 ceph-mon[37140]: pgmap v7778: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:32:18.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:18 smithi167 ceph-mon[37140]: osdmap e533: 12 total, 12 up, 12 in 2024-07-26T17:32:20.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:20 smithi138 ceph-mon[36954]: pgmap v7780: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:32:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:20 smithi100 ceph-mon[32599]: pgmap v7780: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:32:20.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:20 smithi167 ceph-mon[37140]: pgmap v7780: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:32:22.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:22 smithi138 ceph-mon[36954]: pgmap v7781: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:32:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:22 smithi100 ceph-mon[32599]: pgmap v7781: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:32:22.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:22 smithi167 ceph-mon[37140]: pgmap v7781: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:32:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:24 smithi138 ceph-mon[36954]: pgmap v7782: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:32:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:24 smithi100 ceph-mon[32599]: pgmap v7782: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:32:24.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:24 smithi167 ceph-mon[37140]: pgmap v7782: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:32:25.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:32:25.178+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:32:25.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:32:25.178+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:32:25.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:32:25.178+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:32:25.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:32:25.178+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:32:25.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:32:25.178+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:32:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:26 smithi138 ceph-mon[36954]: pgmap v7783: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:32:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:26 smithi100 ceph-mon[32599]: pgmap v7783: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:32:26.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:26 smithi167 ceph-mon[37140]: pgmap v7783: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:32:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:28 smithi138 ceph-mon[36954]: pgmap v7784: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:32:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:28 smithi100 ceph-mon[32599]: pgmap v7784: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:32:28.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:28 smithi167 ceph-mon[37140]: pgmap v7784: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:32:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:30 smithi138 ceph-mon[36954]: pgmap v7785: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:32:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:30 smithi100 ceph-mon[32599]: pgmap v7785: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:32:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:30 smithi167 ceph-mon[37140]: pgmap v7785: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:32:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:32 smithi100 ceph-mon[32599]: pgmap v7786: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:32:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:32 smithi167 ceph-mon[37140]: pgmap v7786: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:32:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:32 smithi138 ceph-mon[36954]: pgmap v7786: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:32:34.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:34 smithi138 ceph-mon[36954]: pgmap v7787: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:32:34.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:32:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:34 smithi100 ceph-mon[32599]: pgmap v7787: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:32:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:32:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:34 smithi167 ceph-mon[37140]: pgmap v7787: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:32:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:32:36.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:32:36.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:32:36.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:32:36.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:32:36.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:32:36.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:32:36.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:35 smithi138 ceph-mon[36954]: pgmap v7788: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:32:36.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:32:36.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:32:36.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:32:36.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:32:36.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:32:36.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:32:36.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:32:36.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:32:36.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:32:36.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:35 smithi100 ceph-mon[32599]: pgmap v7788: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:32:36.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:32:36.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:32:36.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:32:36.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:32:36.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:32:36.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:32:36.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:32:36.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:32:36.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:32:36.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:35 smithi167 ceph-mon[37140]: pgmap v7788: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:32:36.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:32:36.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:32:36.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:32:38.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:38 smithi138 ceph-mon[36954]: pgmap v7789: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:32:38.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:38 smithi100 ceph-mon[32599]: pgmap v7789: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:32:38.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:38 smithi167 ceph-mon[37140]: pgmap v7789: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:32:40.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:32:40.194+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:32:40.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:32:40.194+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:32:40.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:32:40.194+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:32:40.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:32:40.194+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:32:40.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:32:40.194+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:32:40.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:40 smithi138 ceph-mon[36954]: pgmap v7790: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:32:40.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:40 smithi100 ceph-mon[32599]: pgmap v7790: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:32:40.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:40 smithi167 ceph-mon[37140]: pgmap v7790: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:32:42.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:42 smithi138 ceph-mon[36954]: pgmap v7791: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:32:42.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:42 smithi100 ceph-mon[32599]: pgmap v7791: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:32:42.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:42 smithi167 ceph-mon[37140]: pgmap v7791: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:32:44.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:44 smithi138 ceph-mon[36954]: pgmap v7792: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:32:44.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:44 smithi100 ceph-mon[32599]: pgmap v7792: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:32:44.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:44 smithi167 ceph-mon[37140]: pgmap v7792: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:32:46.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:46 smithi138 ceph-mon[36954]: pgmap v7793: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:32:46.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:46 smithi100 ceph-mon[32599]: pgmap v7793: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:32:46.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:46 smithi167 ceph-mon[37140]: pgmap v7793: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:32:48.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:48 smithi138 ceph-mon[36954]: pgmap v7794: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:32:48.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:48 smithi100 ceph-mon[32599]: pgmap v7794: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:32:48.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:48 smithi167 ceph-mon[37140]: pgmap v7794: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:32:50.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:50 smithi138 ceph-mon[36954]: pgmap v7795: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:32:50.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:50 smithi100 ceph-mon[32599]: pgmap v7795: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:32:50.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:50 smithi167 ceph-mon[37140]: pgmap v7795: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:32:52.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:52 smithi138 ceph-mon[36954]: pgmap v7796: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:32:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:52 smithi100 ceph-mon[32599]: pgmap v7796: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:32:52.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:52 smithi167 ceph-mon[37140]: pgmap v7796: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:32:54.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:54 smithi138 ceph-mon[36954]: pgmap v7797: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:32:54.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:54 smithi100 ceph-mon[32599]: pgmap v7797: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:32:54.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:54 smithi167 ceph-mon[37140]: pgmap v7797: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:32:55.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:32:55.210+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:32:55.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:32:55.210+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:32:55.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:32:55.210+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:32:55.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:32:55.210+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:32:55.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:32:55.210+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:32:56.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:56 smithi138 ceph-mon[36954]: pgmap v7798: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:32:56.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:56 smithi100 ceph-mon[32599]: pgmap v7798: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:32:56.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:56 smithi167 ceph-mon[37140]: pgmap v7798: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:32:58.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:32:58 smithi138 ceph-mon[36954]: pgmap v7799: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:32:58.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:32:58 smithi100 ceph-mon[32599]: pgmap v7799: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:32:58.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:32:58 smithi167 ceph-mon[37140]: pgmap v7799: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:33:00.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:00 smithi138 ceph-mon[36954]: pgmap v7800: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:33:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:00 smithi100 ceph-mon[32599]: pgmap v7800: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:33:00.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:00 smithi167 ceph-mon[37140]: pgmap v7800: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:33:02.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:02 smithi138 ceph-mon[36954]: pgmap v7801: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:33:02.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:02 smithi100 ceph-mon[32599]: pgmap v7801: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:33:02.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:02 smithi167 ceph-mon[37140]: pgmap v7801: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:33:03.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:33:03.042+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 msgr2=0x7fe5940b5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-07-26T17:33:03.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:33:03.042+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 msgr2=0x7fe5940b5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-07-26T17:33:03.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:33:03.042+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 0x7fe5940b5b30 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe590005d00 tx=0x7fe590097040 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:33:03.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:33:03.042+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 0x7fe5940b5b30 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe590005d00 tx=0x7fe590097040 comp rx=0 tx=0).stop 2024-07-26T17:33:03.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:33:03.042+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 msgr2=0x7fe5940b5b30 unknown :-1 s=STATE_CLOSED l=1).tick see no progress in more than 10000000 us during connecting to v2:172.21.15.100:6800/3937009534, fault. 2024-07-26T17:33:03.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:33:03.042+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 msgr2=0x7fe5940b5b30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-07-26T17:33:03.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:33:03.042+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 0x7fe5940b5b30 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T17:33:03.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:33:03.042+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T17:33:03.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:33:03.042+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:33:03.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:33:03.042+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fe590005d00 tx=0x7fe590074040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T17:33:04.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:04 smithi138 ceph-mon[36954]: pgmap v7802: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:33:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:04 smithi100 ceph-mon[32599]: pgmap v7802: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:33:04.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:04 smithi167 ceph-mon[37140]: pgmap v7802: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:33:04.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:33:04.928+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c004740 con 0x7fe5a81087a0 2024-07-26T17:33:04.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:33:04.928+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T17:33:04.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:33:04.928+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 107 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T17:33:04.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:33:04.928+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 107 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c070 con 0x7fe5a81087a0 2024-07-26T17:33:06.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:06 smithi138 ceph-mon[36954]: pgmap v7803: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:33:06.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:06 smithi100 ceph-mon[32599]: pgmap v7803: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:33:06.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:06 smithi167 ceph-mon[37140]: pgmap v7803: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:33:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:08 smithi138 ceph-mon[36954]: pgmap v7804: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:33:08.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:08 smithi138 ceph-mon[36954]: osdmap e534: 12 total, 12 up, 12 in 2024-07-26T17:33:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:08 smithi100 ceph-mon[32599]: pgmap v7804: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:33:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:08 smithi100 ceph-mon[32599]: osdmap e534: 12 total, 12 up, 12 in 2024-07-26T17:33:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:08 smithi167 ceph-mon[37140]: pgmap v7804: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:33:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:08 smithi167 ceph-mon[37140]: osdmap e534: 12 total, 12 up, 12 in 2024-07-26T17:33:10.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:33:10.218+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:33:10.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:33:10.218+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:33:10.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:33:10.218+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:33:10.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:33:10.218+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:33:10.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:33:10.219+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:33:10.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:10 smithi138 ceph-mon[36954]: pgmap v7806: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:33:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:10 smithi100 ceph-mon[32599]: pgmap v7806: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:33:10.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:10 smithi167 ceph-mon[37140]: pgmap v7806: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:33:12.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:12 smithi138 ceph-mon[36954]: pgmap v7807: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:33:12.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:12 smithi100 ceph-mon[32599]: pgmap v7807: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:33:12.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:12 smithi167 ceph-mon[37140]: pgmap v7807: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:33:14.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:14 smithi138 ceph-mon[36954]: pgmap v7808: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:33:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:14 smithi100 ceph-mon[32599]: pgmap v7808: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:33:14.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:14 smithi167 ceph-mon[37140]: pgmap v7808: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:33:16.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:16 smithi138 ceph-mon[36954]: pgmap v7809: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:33:16.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:16 smithi100 ceph-mon[32599]: pgmap v7809: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:33:16.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:16 smithi167 ceph-mon[37140]: pgmap v7809: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:33:18.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:18 smithi138 ceph-mon[36954]: pgmap v7810: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:33:18.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:18 smithi138 ceph-mon[36954]: osdmap e535: 12 total, 12 up, 12 in 2024-07-26T17:33:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:18 smithi100 ceph-mon[32599]: pgmap v7810: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:33:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:18 smithi100 ceph-mon[32599]: osdmap e535: 12 total, 12 up, 12 in 2024-07-26T17:33:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:18 smithi167 ceph-mon[37140]: pgmap v7810: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:33:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:18 smithi167 ceph-mon[37140]: osdmap e535: 12 total, 12 up, 12 in 2024-07-26T17:33:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:20 smithi100 ceph-mon[32599]: pgmap v7812: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:33:20.763 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:20 smithi138 ceph-mon[36954]: pgmap v7812: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:33:20.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:20 smithi167 ceph-mon[37140]: pgmap v7812: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:33:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:22 smithi100 ceph-mon[32599]: pgmap v7813: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:33:22.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:22 smithi167 ceph-mon[37140]: pgmap v7813: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:33:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:22 smithi138 ceph-mon[36954]: pgmap v7813: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:33:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:24 smithi100 ceph-mon[32599]: pgmap v7814: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:33:24.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:24 smithi167 ceph-mon[37140]: pgmap v7814: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:33:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:24 smithi138 ceph-mon[36954]: pgmap v7814: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:33:25.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:33:25.221+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:33:25.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:33:25.222+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:33:25.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:33:25.222+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:33:25.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:33:25.222+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:33:25.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:33:25.222+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:33:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:26 smithi100 ceph-mon[32599]: pgmap v7815: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:33:26.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:26 smithi167 ceph-mon[37140]: pgmap v7815: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:33:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:26 smithi138 ceph-mon[36954]: pgmap v7815: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:33:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:28 smithi100 ceph-mon[32599]: pgmap v7816: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:33:28.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:28 smithi167 ceph-mon[37140]: pgmap v7816: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:33:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:28 smithi138 ceph-mon[36954]: pgmap v7816: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:33:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:30 smithi100 ceph-mon[32599]: pgmap v7817: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:33:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:30 smithi167 ceph-mon[37140]: pgmap v7817: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:33:30.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:30 smithi138 ceph-mon[36954]: pgmap v7817: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:33:32.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:32 smithi100 ceph-mon[32599]: pgmap v7818: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:33:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:32 smithi167 ceph-mon[37140]: pgmap v7818: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:33:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:32 smithi138 ceph-mon[36954]: pgmap v7818: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:33:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:34 smithi100 ceph-mon[32599]: pgmap v7819: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:33:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:34 smithi167 ceph-mon[37140]: pgmap v7819: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:33:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:34 smithi138 ceph-mon[36954]: pgmap v7819: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:33:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:33:35.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:33:35.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:33:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:36 smithi167 ceph-mon[37140]: pgmap v7820: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:33:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:33:37.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:33:37.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:33:37.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:33:37.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:33:37.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:33:37.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:33:37.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:33:37.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:33:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:36 smithi138 ceph-mon[36954]: pgmap v7820: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:33:37.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:33:37.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:33:37.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:33:37.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:33:37.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:33:37.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:33:37.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:33:37.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:33:37.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:33:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:36 smithi100 ceph-mon[32599]: pgmap v7820: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:33:37.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:33:37.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:33:37.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:33:37.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:33:37.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:33:37.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:33:37.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:33:37.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:33:37.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:33:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:38 smithi167 ceph-mon[37140]: pgmap v7821: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:33:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:38 smithi138 ceph-mon[36954]: pgmap v7821: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:33:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:38 smithi100 ceph-mon[32599]: pgmap v7821: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:33:40.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:33:40.237+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:33:40.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:33:40.237+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:33:40.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:33:40.237+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:33:40.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:33:40.237+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:33:40.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:33:40.237+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:33:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:40 smithi167 ceph-mon[37140]: pgmap v7822: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:33:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:40 smithi138 ceph-mon[36954]: pgmap v7822: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:33:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:40 smithi100 ceph-mon[32599]: pgmap v7822: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:33:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:42 smithi138 ceph-mon[36954]: pgmap v7823: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:33:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:42 smithi100 ceph-mon[32599]: pgmap v7823: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:33:43.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:42 smithi167 ceph-mon[37140]: pgmap v7823: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:33:45.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:44 smithi138 ceph-mon[36954]: pgmap v7824: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:33:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:44 smithi100 ceph-mon[32599]: pgmap v7824: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:33:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:44 smithi167 ceph-mon[37140]: pgmap v7824: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:33:47.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:46 smithi138 ceph-mon[36954]: pgmap v7825: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:33:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:46 smithi100 ceph-mon[32599]: pgmap v7825: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:33:47.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:46 smithi167 ceph-mon[37140]: pgmap v7825: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:33:48.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:47 smithi138 ceph-mon[36954]: pgmap v7826: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:33:48.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:47 smithi100 ceph-mon[32599]: pgmap v7826: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:33:48.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:47 smithi167 ceph-mon[37140]: pgmap v7826: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:33:50.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:50 smithi138 ceph-mon[36954]: pgmap v7827: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:33:50.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:50 smithi100 ceph-mon[32599]: pgmap v7827: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:33:50.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:50 smithi167 ceph-mon[37140]: pgmap v7827: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:33:52.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:52 smithi138 ceph-mon[36954]: pgmap v7828: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:33:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:52 smithi100 ceph-mon[32599]: pgmap v7828: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:33:52.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:52 smithi167 ceph-mon[37140]: pgmap v7828: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:33:54.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:54 smithi138 ceph-mon[36954]: pgmap v7829: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:33:54.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:54 smithi100 ceph-mon[32599]: pgmap v7829: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:33:54.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:54 smithi167 ceph-mon[37140]: pgmap v7829: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:33:55.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:33:55.247+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:33:55.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:33:55.247+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:33:55.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:33:55.247+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:33:55.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:33:55.247+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:33:55.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:33:55.248+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:33:56.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:56 smithi138 ceph-mon[36954]: pgmap v7830: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:33:56.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:56 smithi100 ceph-mon[32599]: pgmap v7830: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:33:56.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:56 smithi167 ceph-mon[37140]: pgmap v7830: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:33:58.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:33:58 smithi138 ceph-mon[36954]: pgmap v7831: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:33:58.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:33:58 smithi100 ceph-mon[32599]: pgmap v7831: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:33:58.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:33:58 smithi167 ceph-mon[37140]: pgmap v7831: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:34:00.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:00 smithi138 ceph-mon[36954]: pgmap v7832: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:34:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:00 smithi100 ceph-mon[32599]: pgmap v7832: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:34:00.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:00 smithi167 ceph-mon[37140]: pgmap v7832: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:34:02.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:02 smithi138 ceph-mon[36954]: pgmap v7833: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:34:02.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:02 smithi100 ceph-mon[32599]: pgmap v7833: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:34:02.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:02 smithi167 ceph-mon[37140]: pgmap v7833: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:34:04.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:04 smithi138 ceph-mon[36954]: pgmap v7834: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:34:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:04 smithi100 ceph-mon[32599]: pgmap v7834: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:34:04.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:04 smithi167 ceph-mon[37140]: pgmap v7834: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:34:06.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:06 smithi138 ceph-mon[36954]: pgmap v7835: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:34:06.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:06 smithi100 ceph-mon[32599]: pgmap v7835: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:34:06.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:06 smithi167 ceph-mon[37140]: pgmap v7835: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:34:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:08 smithi138 ceph-mon[36954]: pgmap v7836: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:34:08.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:08 smithi138 ceph-mon[36954]: osdmap e536: 12 total, 12 up, 12 in 2024-07-26T17:34:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:08 smithi100 ceph-mon[32599]: pgmap v7836: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:34:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:08 smithi100 ceph-mon[32599]: osdmap e536: 12 total, 12 up, 12 in 2024-07-26T17:34:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:08 smithi167 ceph-mon[37140]: pgmap v7836: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:34:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:08 smithi167 ceph-mon[37140]: osdmap e536: 12 total, 12 up, 12 in 2024-07-26T17:34:10.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:34:10.263+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:34:10.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:34:10.264+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:34:10.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:34:10.264+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:34:10.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:34:10.264+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:34:10.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:34:10.264+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:34:10.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:10 smithi138 ceph-mon[36954]: pgmap v7838: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:34:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:10 smithi100 ceph-mon[32599]: pgmap v7838: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:34:10.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:10 smithi167 ceph-mon[37140]: pgmap v7838: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:34:12.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:12 smithi138 ceph-mon[36954]: pgmap v7839: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:34:12.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:12 smithi100 ceph-mon[32599]: pgmap v7839: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:34:12.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:12 smithi167 ceph-mon[37140]: pgmap v7839: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:34:14.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:14 smithi138 ceph-mon[36954]: pgmap v7840: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:34:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:14 smithi100 ceph-mon[32599]: pgmap v7840: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:34:14.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:14 smithi167 ceph-mon[37140]: pgmap v7840: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:34:16.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:16 smithi138 ceph-mon[36954]: pgmap v7841: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:34:16.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:16 smithi100 ceph-mon[32599]: pgmap v7841: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:34:16.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:16 smithi167 ceph-mon[37140]: pgmap v7841: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:34:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:18 smithi100 ceph-mon[32599]: pgmap v7842: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:34:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:18 smithi100 ceph-mon[32599]: osdmap e537: 12 total, 12 up, 12 in 2024-07-26T17:34:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:18 smithi167 ceph-mon[37140]: pgmap v7842: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:34:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:18 smithi167 ceph-mon[37140]: osdmap e537: 12 total, 12 up, 12 in 2024-07-26T17:34:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:18 smithi138 ceph-mon[36954]: pgmap v7842: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:34:18.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:18 smithi138 ceph-mon[36954]: osdmap e537: 12 total, 12 up, 12 in 2024-07-26T17:34:20.743 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:20 smithi138 ceph-mon[36954]: pgmap v7844: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:34:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:20 smithi100 ceph-mon[32599]: pgmap v7844: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:34:20.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:20 smithi167 ceph-mon[37140]: pgmap v7844: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:34:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:22 smithi100 ceph-mon[32599]: pgmap v7845: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:34:22.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:22 smithi167 ceph-mon[37140]: pgmap v7845: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:34:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:22 smithi138 ceph-mon[36954]: pgmap v7845: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:34:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:24 smithi100 ceph-mon[32599]: pgmap v7846: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:34:24.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:24 smithi167 ceph-mon[37140]: pgmap v7846: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:34:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:24 smithi138 ceph-mon[36954]: pgmap v7846: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:34:25.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:34:25.267+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:34:25.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:34:25.268+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:34:25.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:34:25.268+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:34:25.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:34:25.268+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:34:25.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:34:25.268+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:34:26.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:26 smithi100 ceph-mon[32599]: pgmap v7847: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:34:26.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:26 smithi167 ceph-mon[37140]: pgmap v7847: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:34:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:26 smithi138 ceph-mon[36954]: pgmap v7847: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:34:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:28 smithi100 ceph-mon[32599]: pgmap v7848: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:34:28.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:28 smithi167 ceph-mon[37140]: pgmap v7848: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:34:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:28 smithi138 ceph-mon[36954]: pgmap v7848: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:34:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:30 smithi100 ceph-mon[32599]: pgmap v7849: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:34:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:30 smithi167 ceph-mon[37140]: pgmap v7849: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:34:30.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:30 smithi138 ceph-mon[36954]: pgmap v7849: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:34:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:32 smithi100 ceph-mon[32599]: pgmap v7850: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:34:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:32 smithi167 ceph-mon[37140]: pgmap v7850: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:34:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:32 smithi138 ceph-mon[36954]: pgmap v7850: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:34:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:34 smithi100 ceph-mon[32599]: pgmap v7851: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:34:34.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:34 smithi167 ceph-mon[37140]: pgmap v7851: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:34:34.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:34 smithi138 ceph-mon[36954]: pgmap v7851: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:34:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:36 smithi100 ceph-mon[32599]: pgmap v7852: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:34:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:34:36.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:36 smithi167 ceph-mon[37140]: pgmap v7852: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:34:36.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:34:36.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:36 smithi138 ceph-mon[36954]: pgmap v7852: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:34:36.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:34:38.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:34:38.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:34:38.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:34:38.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:34:38.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:34:38.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:34:38.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:34:38.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:34:38.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:34:38.443 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:38 smithi138 ceph-mon[36954]: pgmap v7853: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:34:38.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:34:38.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:34:38.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:34:38.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:34:38.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:34:38.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:34:38.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:34:38.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:34:38.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:34:38.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:38 smithi100 ceph-mon[32599]: pgmap v7853: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:34:38.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:34:38.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:34:38.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:34:38.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:34:38.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:34:38.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:34:38.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:34:38.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:34:38.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:34:38.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:38 smithi167 ceph-mon[37140]: pgmap v7853: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:34:40.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:34:40.284+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:34:40.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:34:40.284+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:34:40.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:34:40.284+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:34:40.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:34:40.284+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:34:40.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:34:40.284+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:34:40.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:40 smithi138 ceph-mon[36954]: pgmap v7854: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:34:40.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:40 smithi100 ceph-mon[32599]: pgmap v7854: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:34:40.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:40 smithi167 ceph-mon[37140]: pgmap v7854: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:34:42.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:42 smithi138 ceph-mon[36954]: pgmap v7855: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:34:42.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:42 smithi100 ceph-mon[32599]: pgmap v7855: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:34:42.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:42 smithi167 ceph-mon[37140]: pgmap v7855: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:34:44.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:44 smithi138 ceph-mon[36954]: pgmap v7856: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:34:44.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:44 smithi100 ceph-mon[32599]: pgmap v7856: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:34:44.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:44 smithi167 ceph-mon[37140]: pgmap v7856: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:34:46.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:46 smithi138 ceph-mon[36954]: pgmap v7857: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:34:46.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:46 smithi100 ceph-mon[32599]: pgmap v7857: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:34:46.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:46 smithi167 ceph-mon[37140]: pgmap v7857: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:34:48.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:48 smithi138 ceph-mon[36954]: pgmap v7858: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:34:48.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:48 smithi100 ceph-mon[32599]: pgmap v7858: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:34:48.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:48 smithi167 ceph-mon[37140]: pgmap v7858: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:34:50.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:50 smithi138 ceph-mon[36954]: pgmap v7859: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:34:50.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:50 smithi100 ceph-mon[32599]: pgmap v7859: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:34:50.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:50 smithi167 ceph-mon[37140]: pgmap v7859: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:34:52.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:52 smithi138 ceph-mon[36954]: pgmap v7860: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:34:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:52 smithi100 ceph-mon[32599]: pgmap v7860: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:34:52.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:52 smithi167 ceph-mon[37140]: pgmap v7860: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:34:54.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:54 smithi138 ceph-mon[36954]: pgmap v7861: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:34:54.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:54 smithi100 ceph-mon[32599]: pgmap v7861: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:34:54.526 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:54 smithi167 ceph-mon[37140]: pgmap v7861: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:34:55.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:34:55.300+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:34:55.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:34:55.301+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:34:55.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:34:55.301+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:34:55.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:34:55.301+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:34:55.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:34:55.301+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:34:56.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:56 smithi138 ceph-mon[36954]: pgmap v7862: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:34:56.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:56 smithi100 ceph-mon[32599]: pgmap v7862: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:34:56.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:56 smithi167 ceph-mon[37140]: pgmap v7862: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:34:58.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:34:58 smithi138 ceph-mon[36954]: pgmap v7863: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:34:58.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:34:58 smithi100 ceph-mon[32599]: pgmap v7863: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:34:58.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:34:58 smithi167 ceph-mon[37140]: pgmap v7863: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:35:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:35:00 smithi100 ceph-mon[32599]: pgmap v7864: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:35:00.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:35:00 smithi167 ceph-mon[37140]: pgmap v7864: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:35:00.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:35:00 smithi138 ceph-mon[36954]: pgmap v7864: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:35:02.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:35:02 smithi100 ceph-mon[32599]: pgmap v7865: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:35:02.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:35:02 smithi167 ceph-mon[37140]: pgmap v7865: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:35:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:35:02 smithi138 ceph-mon[36954]: pgmap v7865: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:35:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:35:04 smithi100 ceph-mon[32599]: pgmap v7866: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:35:04.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:35:04 smithi167 ceph-mon[37140]: pgmap v7866: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:35:04.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:35:04 smithi138 ceph-mon[36954]: pgmap v7866: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:35:06.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:35:06 smithi100 ceph-mon[32599]: pgmap v7867: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:35:06.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:35:06 smithi167 ceph-mon[37140]: pgmap v7867: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:35:06.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:35:06 smithi138 ceph-mon[36954]: pgmap v7867: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:35:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:35:08 smithi100 ceph-mon[32599]: pgmap v7868: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:35:08.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:35:08 smithi100 ceph-mon[32599]: osdmap e538: 12 total, 12 up, 12 in 2024-07-26T17:35:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:35:08 smithi167 ceph-mon[37140]: pgmap v7868: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:35:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:35:08 smithi167 ceph-mon[37140]: osdmap e538: 12 total, 12 up, 12 in 2024-07-26T17:35:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:35:08 smithi138 ceph-mon[36954]: pgmap v7868: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:35:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:35:08 smithi138 ceph-mon[36954]: osdmap e538: 12 total, 12 up, 12 in 2024-07-26T17:35:10.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:35:10.307+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:35:10.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:35:10.308+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:35:10.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:35:10.308+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:35:10.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:35:10.308+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:35:10.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:35:10.308+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:35:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:35:10 smithi100 ceph-mon[32599]: pgmap v7870: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:35:10.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:35:10 smithi167 ceph-mon[37140]: pgmap v7870: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:35:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:35:10 smithi138 ceph-mon[36954]: pgmap v7870: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:35:12.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:35:12 smithi100 ceph-mon[32599]: pgmap v7871: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:35:12.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:35:12 smithi167 ceph-mon[37140]: pgmap v7871: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:35:12.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:35:12 smithi138 ceph-mon[36954]: pgmap v7871: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:35:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:35:14 smithi100 ceph-mon[32599]: pgmap v7872: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:35:14.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:35:14 smithi167 ceph-mon[37140]: pgmap v7872: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:35:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:35:14 smithi138 ceph-mon[36954]: pgmap v7872: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:35:16.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:35:16 smithi100 ceph-mon[32599]: pgmap v7873: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:35:16.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:35:16 smithi167 ceph-mon[37140]: pgmap v7873: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:35:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:35:16 smithi138 ceph-mon[36954]: pgmap v7873: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:35:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:35:18 smithi100 ceph-mon[32599]: pgmap v7874: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:35:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:35:18 smithi100 ceph-mon[32599]: osdmap e539: 12 total, 12 up, 12 in 2024-07-26T17:35:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:35:18 smithi167 ceph-mon[37140]: pgmap v7874: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:35:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:35:18 smithi167 ceph-mon[37140]: osdmap e539: 12 total, 12 up, 12 in 2024-07-26T17:35:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:35:18 smithi138 ceph-mon[36954]: pgmap v7874: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:35:18.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:35:18 smithi138 ceph-mon[36954]: osdmap e539: 12 total, 12 up, 12 in 2024-07-26T17:35:20.743 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:35:20 smithi138 ceph-mon[36954]: pgmap v7876: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:35:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:35:20 smithi100 ceph-mon[32599]: pgmap v7876: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:35:20.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:35:20 smithi167 ceph-mon[37140]: pgmap v7876: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:35:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:35:22 smithi100 ceph-mon[32599]: pgmap v7877: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:35:22.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:35:22 smithi167 ceph-mon[37140]: pgmap v7877: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:35:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:35:22 smithi138 ceph-mon[36954]: pgmap v7877: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:35:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:35:24 smithi100 ceph-mon[32599]: pgmap v7878: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:35:24.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:35:24 smithi167 ceph-mon[37140]: pgmap v7878: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:35:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:35:24 smithi138 ceph-mon[36954]: pgmap v7878: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:35:25.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:35:25.319+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:35:25.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:35:25.320+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:35:25.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:35:25.320+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:35:25.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:35:25.320+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:35:25.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:35:25.320+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:35:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:35:26 smithi100 ceph-mon[32599]: pgmap v7879: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:35:26.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:35:26 smithi167 ceph-mon[37140]: pgmap v7879: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:35:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:35:26 smithi138 ceph-mon[36954]: pgmap v7879: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:35:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:35:28 smithi100 ceph-mon[32599]: pgmap v7880: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:35:28.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:35:28 smithi167 ceph-mon[37140]: pgmap v7880: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:35:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:35:28 smithi138 ceph-mon[36954]: pgmap v7880: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:35:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:35:30 smithi100 ceph-mon[32599]: pgmap v7881: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:35:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:35:30 smithi167 ceph-mon[37140]: pgmap v7881: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:35:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:35:30 smithi138 ceph-mon[36954]: pgmap v7881: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:35:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:35:32 smithi100 ceph-mon[32599]: pgmap v7882: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:35:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:35:32 smithi167 ceph-mon[37140]: pgmap v7882: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:35:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:35:32 smithi138 ceph-mon[36954]: pgmap v7882: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:35:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:35:34 smithi100 ceph-mon[32599]: pgmap v7883: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:35:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:35:34 smithi167 ceph-mon[37140]: pgmap v7883: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:35:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:35:34 smithi138 ceph-mon[36954]: pgmap v7883: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:35:36.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:35:36 smithi167 ceph-mon[37140]: pgmap v7884: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:35:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:35:36 smithi138 ceph-mon[36954]: pgmap v7884: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:35:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:35:36 smithi100 ceph-mon[32599]: pgmap v7884: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:35:37.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:35:37 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:35:37.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:35:37 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:35:37.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:35:37 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:35:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:35:38 smithi167 ceph-mon[37140]: pgmap v7885: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:35:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:35:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:35:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:35:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:35:38.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:35:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:35:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:35:38 smithi138 ceph-mon[36954]: pgmap v7885: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:35:38.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:35:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:35:38.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:35:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:35:38.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:35:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:35:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:35:38 smithi100 ceph-mon[32599]: pgmap v7885: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:35:38.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:35:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:35:38.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:35:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:35:38.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:35:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:35:40.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:35:40.335+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:35:40.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:35:40.336+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:35:40.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:35:40.336+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:35:40.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:35:40.336+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:35:40.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:35:40.336+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:35:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:35:40 smithi100 ceph-mon[32599]: pgmap v7886: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:35:40.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:35:40 smithi167 ceph-mon[37140]: pgmap v7886: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:35:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:35:40 smithi138 ceph-mon[36954]: pgmap v7886: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:35:42.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:35:42 smithi167 ceph-mon[37140]: pgmap v7887: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:35:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:35:42 smithi138 ceph-mon[36954]: pgmap v7887: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:35:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:35:42 smithi100 ceph-mon[32599]: pgmap v7887: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:35:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:35:44 smithi167 ceph-mon[37140]: pgmap v7888: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:35:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:35:44 smithi138 ceph-mon[36954]: pgmap v7888: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:35:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:35:44 smithi100 ceph-mon[32599]: pgmap v7888: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:35:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:35:46 smithi167 ceph-mon[37140]: pgmap v7889: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:35:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:35:46 smithi138 ceph-mon[36954]: pgmap v7889: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:35:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:35:46 smithi100 ceph-mon[32599]: pgmap v7889: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:35:48.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:35:48 smithi138 ceph-mon[36954]: pgmap v7890: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:35:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:35:48 smithi100 ceph-mon[32599]: pgmap v7890: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:35:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:35:48 smithi167 ceph-mon[37140]: pgmap v7890: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:35:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:35:50 smithi138 ceph-mon[36954]: pgmap v7891: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:35:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:35:50 smithi100 ceph-mon[32599]: pgmap v7891: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:35:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:35:50 smithi167 ceph-mon[37140]: pgmap v7891: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:35:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:35:52 smithi138 ceph-mon[36954]: pgmap v7892: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:35:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:35:52 smithi100 ceph-mon[32599]: pgmap v7892: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:35:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:35:52 smithi167 ceph-mon[37140]: pgmap v7892: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:35:54.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:35:54 smithi100 ceph-mon[32599]: pgmap v7893: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:35:54.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:35:54 smithi167 ceph-mon[37140]: pgmap v7893: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:35:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:35:54 smithi138 ceph-mon[36954]: pgmap v7893: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:35:55.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:35:55.351+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:35:55.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:35:55.352+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:35:55.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:35:55.352+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:35:55.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:35:55.352+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:35:55.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:35:55.352+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:35:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:35:56 smithi138 ceph-mon[36954]: pgmap v7894: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:35:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:35:56 smithi100 ceph-mon[32599]: pgmap v7894: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:35:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:35:56 smithi167 ceph-mon[37140]: pgmap v7894: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:35:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:35:58 smithi138 ceph-mon[36954]: pgmap v7895: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:35:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:35:58 smithi100 ceph-mon[32599]: pgmap v7895: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:35:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:35:58 smithi167 ceph-mon[37140]: pgmap v7895: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:36:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:36:00 smithi138 ceph-mon[36954]: pgmap v7896: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:36:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:36:00 smithi100 ceph-mon[32599]: pgmap v7896: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:36:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:36:00 smithi167 ceph-mon[37140]: pgmap v7896: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:36:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:36:02 smithi138 ceph-mon[36954]: pgmap v7897: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:36:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:36:02 smithi100 ceph-mon[32599]: pgmap v7897: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:36:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:36:02 smithi167 ceph-mon[37140]: pgmap v7897: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:36:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:36:04 smithi138 ceph-mon[36954]: pgmap v7898: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:36:04.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:36:04.954+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c004740 con 0x7fe5a81087a0 2024-07-26T17:36:04.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:36:04.954+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T17:36:04.957 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:36:04 smithi100 ceph-mon[32599]: pgmap v7898: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:36:04.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:36:04.955+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 108 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c070 con 0x7fe5a81087a0 2024-07-26T17:36:04.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:36:04.955+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 108 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T17:36:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:36:04 smithi167 ceph-mon[37140]: pgmap v7898: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:36:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:36:06 smithi138 ceph-mon[36954]: pgmap v7899: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:36:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:36:06 smithi100 ceph-mon[32599]: pgmap v7899: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:36:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:36:06 smithi167 ceph-mon[37140]: pgmap v7899: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:36:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:36:08 smithi138 ceph-mon[36954]: pgmap v7900: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:36:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:36:08 smithi138 ceph-mon[36954]: osdmap e540: 12 total, 12 up, 12 in 2024-07-26T17:36:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:36:08 smithi100 ceph-mon[32599]: pgmap v7900: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:36:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:36:08 smithi100 ceph-mon[32599]: osdmap e540: 12 total, 12 up, 12 in 2024-07-26T17:36:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:36:08 smithi167 ceph-mon[37140]: pgmap v7900: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:36:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:36:08 smithi167 ceph-mon[37140]: osdmap e540: 12 total, 12 up, 12 in 2024-07-26T17:36:10.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:36:10.367+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:36:10.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:36:10.368+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:36:10.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:36:10.368+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:36:10.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:36:10.368+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:36:10.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:36:10.368+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:36:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:36:10 smithi100 ceph-mon[32599]: pgmap v7902: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:36:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:36:10 smithi138 ceph-mon[36954]: pgmap v7902: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:36:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:36:10 smithi167 ceph-mon[37140]: pgmap v7902: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:36:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:36:12 smithi138 ceph-mon[36954]: pgmap v7903: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:36:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:36:12 smithi100 ceph-mon[32599]: pgmap v7903: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:36:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:36:12 smithi167 ceph-mon[37140]: pgmap v7903: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:36:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:36:14 smithi138 ceph-mon[36954]: pgmap v7904: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:36:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:36:14 smithi100 ceph-mon[32599]: pgmap v7904: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:36:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:36:14 smithi167 ceph-mon[37140]: pgmap v7904: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:36:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:36:16 smithi138 ceph-mon[36954]: pgmap v7905: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:36:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:36:16 smithi100 ceph-mon[32599]: pgmap v7905: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:36:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:36:16 smithi167 ceph-mon[37140]: pgmap v7905: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:36:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:36:18 smithi100 ceph-mon[32599]: pgmap v7906: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:36:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:36:18 smithi100 ceph-mon[32599]: osdmap e541: 12 total, 12 up, 12 in 2024-07-26T17:36:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:36:18 smithi167 ceph-mon[37140]: pgmap v7906: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:36:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:36:18 smithi167 ceph-mon[37140]: osdmap e541: 12 total, 12 up, 12 in 2024-07-26T17:36:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:36:18 smithi138 ceph-mon[36954]: pgmap v7906: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:36:18.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:36:18 smithi138 ceph-mon[36954]: osdmap e541: 12 total, 12 up, 12 in 2024-07-26T17:36:20.743 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:36:20 smithi138 ceph-mon[36954]: pgmap v7908: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:36:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:36:20 smithi100 ceph-mon[32599]: pgmap v7908: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:36:20.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:36:20 smithi167 ceph-mon[37140]: pgmap v7908: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:36:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:36:22 smithi100 ceph-mon[32599]: pgmap v7909: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:36:22.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:36:22 smithi167 ceph-mon[37140]: pgmap v7909: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:36:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:36:22 smithi138 ceph-mon[36954]: pgmap v7909: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:36:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:36:24 smithi100 ceph-mon[32599]: pgmap v7910: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:36:24.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:36:24 smithi167 ceph-mon[37140]: pgmap v7910: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:36:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:36:24 smithi138 ceph-mon[36954]: pgmap v7910: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:36:25.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:36:25.383+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:36:25.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:36:25.384+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:36:25.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:36:25.384+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:36:25.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:36:25.384+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:36:25.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:36:25.384+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:36:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:36:26 smithi100 ceph-mon[32599]: pgmap v7911: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:36:26.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:36:26 smithi167 ceph-mon[37140]: pgmap v7911: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:36:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:36:26 smithi138 ceph-mon[36954]: pgmap v7911: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:36:28.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:36:28 smithi167 ceph-mon[37140]: pgmap v7912: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:36:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:36:28 smithi138 ceph-mon[36954]: pgmap v7912: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:36:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:36:28 smithi100 ceph-mon[32599]: pgmap v7912: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:36:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:36:30 smithi167 ceph-mon[37140]: pgmap v7913: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:36:30.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:36:30 smithi138 ceph-mon[36954]: pgmap v7913: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:36:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:36:30 smithi100 ceph-mon[32599]: pgmap v7913: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:36:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:36:32 smithi167 ceph-mon[37140]: pgmap v7914: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:36:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:36:32 smithi138 ceph-mon[36954]: pgmap v7914: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:36:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:36:32 smithi100 ceph-mon[32599]: pgmap v7914: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:36:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:36:34 smithi167 ceph-mon[37140]: pgmap v7915: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T17:36:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:36:34 smithi138 ceph-mon[36954]: pgmap v7915: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T17:36:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:36:34 smithi100 ceph-mon[32599]: pgmap v7915: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T17:36:36.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:36:36 smithi167 ceph-mon[37140]: pgmap v7916: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:36:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:36:36 smithi138 ceph-mon[36954]: pgmap v7916: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:36:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:36:36 smithi100 ceph-mon[32599]: pgmap v7916: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:36:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:36:38 smithi167 ceph-mon[37140]: pgmap v7917: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:36:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:36:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:36:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:36:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:36:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:36:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:36:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:36:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:36:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:36:38 smithi138 ceph-mon[36954]: pgmap v7917: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:36:38.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:36:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:36:38.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:36:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:36:38.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:36:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:36:38.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:36:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:36:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:36:38 smithi100 ceph-mon[32599]: pgmap v7917: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:36:38.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:36:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:36:38.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:36:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:36:38.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:36:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:36:38.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:36:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:36:40.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:36:40.389+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:36:40.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:36:40.390+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:36:40.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:36:40.390+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:36:40.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:36:40.390+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:36:40.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:36:40.390+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:36:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:36:40 smithi100 ceph-mon[32599]: pgmap v7918: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:36:40.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:36:40 smithi167 ceph-mon[37140]: pgmap v7918: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:36:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:36:40 smithi138 ceph-mon[36954]: pgmap v7918: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:36:42.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:36:42 smithi167 ceph-mon[37140]: pgmap v7919: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:36:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:36:42 smithi138 ceph-mon[36954]: pgmap v7919: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:36:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:36:42 smithi100 ceph-mon[32599]: pgmap v7919: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:36:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:36:44 smithi138 ceph-mon[36954]: pgmap v7920: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:36:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:36:44 smithi100 ceph-mon[32599]: pgmap v7920: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:36:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:36:44 smithi167 ceph-mon[37140]: pgmap v7920: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:36:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:36:46 smithi138 ceph-mon[36954]: pgmap v7921: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:36:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:36:46 smithi100 ceph-mon[32599]: pgmap v7921: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:36:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:36:46 smithi167 ceph-mon[37140]: pgmap v7921: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:36:48.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:36:48 smithi138 ceph-mon[36954]: pgmap v7922: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:36:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:36:48 smithi100 ceph-mon[32599]: pgmap v7922: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:36:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:36:48 smithi167 ceph-mon[37140]: pgmap v7922: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:36:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:36:50 smithi138 ceph-mon[36954]: pgmap v7923: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:36:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:36:50 smithi100 ceph-mon[32599]: pgmap v7923: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:36:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:36:50 smithi167 ceph-mon[37140]: pgmap v7923: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:36:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:36:52 smithi138 ceph-mon[36954]: pgmap v7924: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:36:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:36:52 smithi100 ceph-mon[32599]: pgmap v7924: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:36:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:36:52 smithi167 ceph-mon[37140]: pgmap v7924: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:36:54.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:36:54 smithi100 ceph-mon[32599]: pgmap v7925: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:36:54.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:36:54 smithi167 ceph-mon[37140]: pgmap v7925: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:36:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:36:54 smithi138 ceph-mon[36954]: pgmap v7925: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:36:55.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:36:55.405+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:36:55.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:36:55.406+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:36:55.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:36:55.406+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:36:55.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:36:55.406+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:36:55.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:36:55.406+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:36:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:36:56 smithi138 ceph-mon[36954]: pgmap v7926: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:36:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:36:56 smithi100 ceph-mon[32599]: pgmap v7926: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:36:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:36:56 smithi167 ceph-mon[37140]: pgmap v7926: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:36:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:36:58 smithi138 ceph-mon[36954]: pgmap v7927: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:36:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:36:58 smithi100 ceph-mon[32599]: pgmap v7927: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:36:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:36:58 smithi167 ceph-mon[37140]: pgmap v7927: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:37:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:37:00 smithi138 ceph-mon[36954]: pgmap v7928: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:37:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:37:00 smithi100 ceph-mon[32599]: pgmap v7928: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:37:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:37:00 smithi167 ceph-mon[37140]: pgmap v7928: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:37:02.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:37:02 smithi138 ceph-mon[36954]: pgmap v7929: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:37:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:37:02 smithi100 ceph-mon[32599]: pgmap v7929: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:37:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:37:02 smithi167 ceph-mon[37140]: pgmap v7929: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:37:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:37:04 smithi138 ceph-mon[36954]: pgmap v7930: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:37:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:37:04 smithi100 ceph-mon[32599]: pgmap v7930: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:37:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:37:04 smithi167 ceph-mon[37140]: pgmap v7930: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:37:06.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:37:06 smithi138 ceph-mon[36954]: pgmap v7931: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:37:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:37:06 smithi100 ceph-mon[32599]: pgmap v7931: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:37:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:37:06 smithi167 ceph-mon[37140]: pgmap v7931: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:37:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:37:08 smithi138 ceph-mon[36954]: pgmap v7932: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:37:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:37:08 smithi138 ceph-mon[36954]: osdmap e542: 12 total, 12 up, 12 in 2024-07-26T17:37:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:37:08 smithi100 ceph-mon[32599]: pgmap v7932: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:37:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:37:08 smithi100 ceph-mon[32599]: osdmap e542: 12 total, 12 up, 12 in 2024-07-26T17:37:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:37:08 smithi167 ceph-mon[37140]: pgmap v7932: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:37:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:37:08 smithi167 ceph-mon[37140]: osdmap e542: 12 total, 12 up, 12 in 2024-07-26T17:37:10.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:37:10.415+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:37:10.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:37:10.416+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:37:10.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:37:10.416+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:37:10.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:37:10.416+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:37:10.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:37:10.416+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:37:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:37:10 smithi100 ceph-mon[32599]: pgmap v7934: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:37:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:37:10 smithi138 ceph-mon[36954]: pgmap v7934: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:37:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:37:10 smithi167 ceph-mon[37140]: pgmap v7934: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:37:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:37:12 smithi138 ceph-mon[36954]: pgmap v7935: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:37:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:37:12 smithi100 ceph-mon[32599]: pgmap v7935: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:37:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:37:12 smithi167 ceph-mon[37140]: pgmap v7935: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:37:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:37:14 smithi138 ceph-mon[36954]: pgmap v7936: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:37:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:37:14 smithi100 ceph-mon[32599]: pgmap v7936: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:37:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:37:14 smithi167 ceph-mon[37140]: pgmap v7936: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:37:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:37:16 smithi138 ceph-mon[36954]: pgmap v7937: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:37:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:37:16 smithi100 ceph-mon[32599]: pgmap v7937: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:37:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:37:16 smithi167 ceph-mon[37140]: pgmap v7937: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:37:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:37:18 smithi167 ceph-mon[37140]: pgmap v7938: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:37:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:37:18 smithi167 ceph-mon[37140]: osdmap e543: 12 total, 12 up, 12 in 2024-07-26T17:37:18.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:37:18 smithi138 ceph-mon[36954]: pgmap v7938: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:37:18.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:37:18 smithi138 ceph-mon[36954]: osdmap e543: 12 total, 12 up, 12 in 2024-07-26T17:37:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:37:18 smithi100 ceph-mon[32599]: pgmap v7938: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:37:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:37:18 smithi100 ceph-mon[32599]: osdmap e543: 12 total, 12 up, 12 in 2024-07-26T17:37:20.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:37:20 smithi167 ceph-mon[37140]: pgmap v7940: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:37:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:37:20 smithi138 ceph-mon[36954]: pgmap v7940: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:37:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:37:20 smithi100 ceph-mon[32599]: pgmap v7940: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:37:22.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:37:22 smithi167 ceph-mon[37140]: pgmap v7941: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:37:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:37:22 smithi138 ceph-mon[36954]: pgmap v7941: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:37:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:37:22 smithi100 ceph-mon[32599]: pgmap v7941: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:37:24.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:37:24 smithi167 ceph-mon[37140]: pgmap v7942: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:37:24.797 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:37:24 smithi100 ceph-mon[32599]: pgmap v7942: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:37:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:37:24 smithi138 ceph-mon[36954]: pgmap v7942: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:37:25.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:37:25.417+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:37:25.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:37:25.418+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:37:25.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:37:25.418+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:37:25.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:37:25.418+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:37:25.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:37:25.418+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:37:26.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:37:26 smithi167 ceph-mon[37140]: pgmap v7943: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:37:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:37:26 smithi138 ceph-mon[36954]: pgmap v7943: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:37:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:37:26 smithi100 ceph-mon[32599]: pgmap v7943: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:37:28.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:37:28 smithi167 ceph-mon[37140]: pgmap v7944: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:37:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:37:28 smithi138 ceph-mon[36954]: pgmap v7944: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:37:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:37:28 smithi100 ceph-mon[32599]: pgmap v7944: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:37:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:37:30 smithi167 ceph-mon[37140]: pgmap v7945: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:37:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:37:30 smithi138 ceph-mon[36954]: pgmap v7945: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:37:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:37:30 smithi100 ceph-mon[32599]: pgmap v7945: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:37:32.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:37:32 smithi138 ceph-mon[36954]: pgmap v7946: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:37:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:37:32 smithi100 ceph-mon[32599]: pgmap v7946: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:37:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:37:32 smithi167 ceph-mon[37140]: pgmap v7946: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:37:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:37:34 smithi138 ceph-mon[36954]: pgmap v7947: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:37:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:37:34 smithi100 ceph-mon[32599]: pgmap v7947: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:37:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:37:34 smithi167 ceph-mon[37140]: pgmap v7947: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:37:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:37:36 smithi138 ceph-mon[36954]: pgmap v7948: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:37:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:37:36 smithi100 ceph-mon[32599]: pgmap v7948: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:37:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:37:36 smithi167 ceph-mon[37140]: pgmap v7948: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:37:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:37:38 smithi138 ceph-mon[36954]: pgmap v7949: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:37:38.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:37:38 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:37:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:37:38 smithi100 ceph-mon[32599]: pgmap v7949: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:37:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:37:38 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:37:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:37:38 smithi167 ceph-mon[37140]: pgmap v7949: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:37:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:37:38 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:37:39.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:37:39 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:37:39.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:37:39 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:37:39.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:37:39 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:37:39.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:37:39 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:37:39.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:37:39 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:37:39.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:37:39 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:37:40.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:37:39 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:37:40.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:37:39 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:37:40.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:37:39 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:37:40.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:37:40.420+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:37:40.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:37:40.421+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:37:40.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:37:40.421+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:37:40.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:37:40.421+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:37:40.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:37:40.421+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:37:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:37:40 smithi100 ceph-mon[32599]: pgmap v7950: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:37:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:37:40 smithi167 ceph-mon[37140]: pgmap v7950: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:37:41.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:37:40 smithi138 ceph-mon[36954]: pgmap v7950: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:37:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:37:42 smithi100 ceph-mon[32599]: pgmap v7951: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:37:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:37:42 smithi167 ceph-mon[37140]: pgmap v7951: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:37:43.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:37:42 smithi138 ceph-mon[36954]: pgmap v7951: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:37:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:37:44 smithi100 ceph-mon[32599]: pgmap v7952: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:37:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:37:44 smithi167 ceph-mon[37140]: pgmap v7952: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:37:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:37:44 smithi138 ceph-mon[36954]: pgmap v7952: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:37:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:37:46 smithi100 ceph-mon[32599]: pgmap v7953: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:37:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:37:46 smithi167 ceph-mon[37140]: pgmap v7953: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:37:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:37:46 smithi138 ceph-mon[36954]: pgmap v7953: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:37:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:37:48 smithi100 ceph-mon[32599]: pgmap v7954: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:37:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:37:48 smithi167 ceph-mon[37140]: pgmap v7954: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:37:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:37:48 smithi138 ceph-mon[36954]: pgmap v7954: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:37:50.993 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:37:50 smithi138 ceph-mon[36954]: pgmap v7955: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:37:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:37:50 smithi100 ceph-mon[32599]: pgmap v7955: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:37:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:37:50 smithi167 ceph-mon[37140]: pgmap v7955: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:37:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:37:52 smithi100 ceph-mon[32599]: pgmap v7956: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:37:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:37:52 smithi167 ceph-mon[37140]: pgmap v7956: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:37:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:37:52 smithi138 ceph-mon[36954]: pgmap v7956: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:37:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:37:54 smithi167 ceph-mon[37140]: pgmap v7957: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:37:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:37:54 smithi138 ceph-mon[36954]: pgmap v7957: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:37:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:37:54 smithi100 ceph-mon[32599]: pgmap v7957: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:37:55.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:37:55.436+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:37:55.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:37:55.437+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:37:55.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:37:55.437+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:37:55.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:37:55.437+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:37:55.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:37:55.437+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:37:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:37:56 smithi167 ceph-mon[37140]: pgmap v7958: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:37:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:37:56 smithi138 ceph-mon[36954]: pgmap v7958: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:37:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:37:56 smithi100 ceph-mon[32599]: pgmap v7958: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:37:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:37:58 smithi167 ceph-mon[37140]: pgmap v7959: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:37:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:37:58 smithi138 ceph-mon[36954]: pgmap v7959: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:37:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:37:58 smithi100 ceph-mon[32599]: pgmap v7959: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:38:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:38:00 smithi167 ceph-mon[37140]: pgmap v7960: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:38:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:38:00 smithi138 ceph-mon[36954]: pgmap v7960: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:38:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:38:00 smithi100 ceph-mon[32599]: pgmap v7960: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:38:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:38:02 smithi167 ceph-mon[37140]: pgmap v7961: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:38:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:38:02 smithi138 ceph-mon[36954]: pgmap v7961: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:38:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:38:02 smithi100 ceph-mon[32599]: pgmap v7961: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:38:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:38:04 smithi167 ceph-mon[37140]: pgmap v7962: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:38:05.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:38:04 smithi138 ceph-mon[36954]: pgmap v7962: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:38:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:38:04 smithi100 ceph-mon[32599]: pgmap v7962: 97 pgs: 97 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:38:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:38:06 smithi167 ceph-mon[37140]: pgmap v7963: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:38:07.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:38:06 smithi138 ceph-mon[36954]: pgmap v7963: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:38:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:38:06 smithi100 ceph-mon[32599]: pgmap v7963: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:38:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:38:08 smithi167 ceph-mon[37140]: pgmap v7964: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:38:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:38:08 smithi167 ceph-mon[37140]: osdmap e544: 12 total, 12 up, 12 in 2024-07-26T17:38:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:38:08 smithi138 ceph-mon[36954]: pgmap v7964: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:38:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:38:08 smithi138 ceph-mon[36954]: osdmap e544: 12 total, 12 up, 12 in 2024-07-26T17:38:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:38:08 smithi100 ceph-mon[32599]: pgmap v7964: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:38:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:38:08 smithi100 ceph-mon[32599]: osdmap e544: 12 total, 12 up, 12 in 2024-07-26T17:38:10.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:38:10.452+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:38:10.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:38:10.452+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:38:10.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:38:10.452+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:38:10.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:38:10.452+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:38:10.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:38:10.452+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:38:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:38:10 smithi138 ceph-mon[36954]: pgmap v7966: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:38:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:38:10 smithi100 ceph-mon[32599]: pgmap v7966: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:38:11.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:38:10 smithi167 ceph-mon[37140]: pgmap v7966: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:38:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:38:12 smithi138 ceph-mon[36954]: pgmap v7967: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:38:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:38:12 smithi100 ceph-mon[32599]: pgmap v7967: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:38:13.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:38:12 smithi167 ceph-mon[37140]: pgmap v7967: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:38:15.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:38:14 smithi138 ceph-mon[36954]: pgmap v7968: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:38:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:38:14 smithi100 ceph-mon[32599]: pgmap v7968: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:38:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:38:14 smithi167 ceph-mon[37140]: pgmap v7968: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:38:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:38:16 smithi138 ceph-mon[36954]: pgmap v7969: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:38:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:38:16 smithi100 ceph-mon[32599]: pgmap v7969: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:38:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:38:16 smithi167 ceph-mon[37140]: pgmap v7969: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:38:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:38:18 smithi167 ceph-mon[37140]: pgmap v7970: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:38:18.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:38:18 smithi167 ceph-mon[37140]: osdmap e545: 12 total, 12 up, 12 in 2024-07-26T17:38:18.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:38:18 smithi138 ceph-mon[36954]: pgmap v7970: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:38:18.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:38:18 smithi138 ceph-mon[36954]: osdmap e545: 12 total, 12 up, 12 in 2024-07-26T17:38:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:38:18 smithi100 ceph-mon[32599]: pgmap v7970: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:38:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:38:18 smithi100 ceph-mon[32599]: osdmap e545: 12 total, 12 up, 12 in 2024-07-26T17:38:20.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:38:20 smithi167 ceph-mon[37140]: pgmap v7972: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:38:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:38:20 smithi138 ceph-mon[36954]: pgmap v7972: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:38:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:38:20 smithi100 ceph-mon[32599]: pgmap v7972: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:38:22.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:38:22 smithi167 ceph-mon[37140]: pgmap v7973: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:38:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:38:22 smithi138 ceph-mon[36954]: pgmap v7973: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:38:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:38:22 smithi100 ceph-mon[32599]: pgmap v7973: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:38:24.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:38:24 smithi100 ceph-mon[32599]: pgmap v7974: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:38:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:38:24 smithi138 ceph-mon[36954]: pgmap v7974: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:38:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:38:24 smithi167 ceph-mon[37140]: pgmap v7974: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:38:25.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:38:25.455+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:38:25.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:38:25.456+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:38:25.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:38:25.456+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:38:25.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:38:25.456+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:38:25.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:38:25.456+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:38:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:38:26 smithi138 ceph-mon[36954]: pgmap v7975: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:38:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:38:26 smithi100 ceph-mon[32599]: pgmap v7975: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:38:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:38:26 smithi167 ceph-mon[37140]: pgmap v7975: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:38:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:38:28 smithi138 ceph-mon[36954]: pgmap v7976: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:38:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:38:28 smithi100 ceph-mon[32599]: pgmap v7976: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:38:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:38:28 smithi167 ceph-mon[37140]: pgmap v7976: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:38:30.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:38:30 smithi138 ceph-mon[36954]: pgmap v7977: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:38:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:38:30 smithi100 ceph-mon[32599]: pgmap v7977: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:38:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:38:30 smithi167 ceph-mon[37140]: pgmap v7977: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:38:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:38:32 smithi138 ceph-mon[36954]: pgmap v7978: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:38:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:38:32 smithi100 ceph-mon[32599]: pgmap v7978: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:38:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:38:32 smithi167 ceph-mon[37140]: pgmap v7978: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:38:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:38:34 smithi138 ceph-mon[36954]: pgmap v7979: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:38:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:38:34 smithi100 ceph-mon[32599]: pgmap v7979: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:38:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:38:34 smithi167 ceph-mon[37140]: pgmap v7979: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:38:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:38:36 smithi138 ceph-mon[36954]: pgmap v7980: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:38:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:38:36 smithi100 ceph-mon[32599]: pgmap v7980: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:38:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:38:36 smithi167 ceph-mon[37140]: pgmap v7980: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:38:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:38:38 smithi138 ceph-mon[36954]: pgmap v7981: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:38:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:38:38 smithi100 ceph-mon[32599]: pgmap v7981: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:38:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:38:38 smithi167 ceph-mon[37140]: pgmap v7981: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:38:39.824 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:38:39 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:38:39.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:38:39 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:38:40.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:38:39 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:38:40.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:38:40.472+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:38:40.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:38:40.472+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:38:40.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:38:40.472+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:38:40.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:38:40.472+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:38:40.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:38:40.472+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:38:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:38:40 smithi100 ceph-mon[32599]: pgmap v7982: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:38:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:38:40 smithi138 ceph-mon[36954]: pgmap v7982: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:38:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:38:40 smithi167 ceph-mon[37140]: pgmap v7982: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:38:42.607 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:38:42 smithi100 ceph-mon[32599]: pgmap v7983: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:38:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:38:42 smithi138 ceph-mon[36954]: pgmap v7983: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:38:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:38:42 smithi167 ceph-mon[37140]: pgmap v7983: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:38:43.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:38:43 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:38:43.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:38:43 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:38:43.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:38:43 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:38:43.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:38:43 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:38:43.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:38:43 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:38:43.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:38:43 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:38:43.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:38:43 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:38:43.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:38:43 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:38:43.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:38:43 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:38:43.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:38:43 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:38:44.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:38:43 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:38:44.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:38:43 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:38:44.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:38:43 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:38:44.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:38:43 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:38:44.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:38:43 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:38:44.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:38:44 smithi138 ceph-mon[36954]: pgmap v7984: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:38:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:38:44 smithi100 ceph-mon[32599]: pgmap v7984: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:38:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:38:44 smithi167 ceph-mon[37140]: pgmap v7984: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:38:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:38:46 smithi138 ceph-mon[36954]: pgmap v7985: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:38:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:38:46 smithi100 ceph-mon[32599]: pgmap v7985: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:38:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:38:46 smithi167 ceph-mon[37140]: pgmap v7985: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:38:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:38:48 smithi138 ceph-mon[36954]: pgmap v7986: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:38:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:38:48 smithi100 ceph-mon[32599]: pgmap v7986: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:38:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:38:48 smithi167 ceph-mon[37140]: pgmap v7986: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:38:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:38:50 smithi138 ceph-mon[36954]: pgmap v7987: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:38:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:38:50 smithi100 ceph-mon[32599]: pgmap v7987: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:38:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:38:50 smithi167 ceph-mon[37140]: pgmap v7987: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:38:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:38:52 smithi138 ceph-mon[36954]: pgmap v7988: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:38:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:38:52 smithi100 ceph-mon[32599]: pgmap v7988: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:38:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:38:52 smithi167 ceph-mon[37140]: pgmap v7988: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:38:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:38:54 smithi138 ceph-mon[36954]: pgmap v7989: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:38:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:38:54 smithi100 ceph-mon[32599]: pgmap v7989: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:38:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:38:54 smithi167 ceph-mon[37140]: pgmap v7989: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:38:55.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:38:55.487+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:38:55.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:38:55.488+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:38:55.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:38:55.488+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:38:55.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:38:55.488+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:38:55.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:38:55.488+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:38:56.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:38:56 smithi138 ceph-mon[36954]: pgmap v7990: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:38:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:38:56 smithi100 ceph-mon[32599]: pgmap v7990: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:38:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:38:56 smithi167 ceph-mon[37140]: pgmap v7990: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:38:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:38:58 smithi138 ceph-mon[36954]: pgmap v7991: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:38:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:38:58 smithi100 ceph-mon[32599]: pgmap v7991: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:38:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:38:58 smithi167 ceph-mon[37140]: pgmap v7991: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:39:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:00 smithi138 ceph-mon[36954]: pgmap v7992: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:39:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:00 smithi100 ceph-mon[32599]: pgmap v7992: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:39:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:00 smithi167 ceph-mon[37140]: pgmap v7992: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:39:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:02 smithi138 ceph-mon[36954]: pgmap v7993: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:39:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:02 smithi100 ceph-mon[32599]: pgmap v7993: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:39:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:02 smithi167 ceph-mon[37140]: pgmap v7993: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:39:04.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:39:04.980+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T17:39:04.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:39:04.980+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c004740 con 0x7fe5a81087a0 2024-07-26T17:39:04.983 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:04 smithi100 ceph-mon[32599]: pgmap v7994: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:39:04.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:39:04.980+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 109 ==== auth_reply(proto 2 0 (0) Success) ==== 783+0+0 (secure 0 0 0) 0x7fe5a008c070 con 0x7fe5a81087a0 2024-07-26T17:39:04.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:39:04.981+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T17:39:04.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:39:04.981+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 109 ==== auth_reply(proto 2 0 (0) Success) ==== 612+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T17:39:04.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:39:04.981+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T17:39:04.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:39:04.981+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 110 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T17:39:04.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:39:04.981+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 110 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T17:39:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:04 smithi167 ceph-mon[37140]: pgmap v7994: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:39:05.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:04 smithi138 ceph-mon[36954]: pgmap v7994: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:39:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:06 smithi100 ceph-mon[32599]: pgmap v7995: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:39:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:06 smithi167 ceph-mon[37140]: pgmap v7995: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:39:07.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:06 smithi138 ceph-mon[36954]: pgmap v7995: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:39:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:08 smithi100 ceph-mon[32599]: pgmap v7996: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:39:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:08 smithi100 ceph-mon[32599]: osdmap e546: 12 total, 12 up, 12 in 2024-07-26T17:39:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:08 smithi167 ceph-mon[37140]: pgmap v7996: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:39:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:08 smithi167 ceph-mon[37140]: osdmap e546: 12 total, 12 up, 12 in 2024-07-26T17:39:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:08 smithi138 ceph-mon[36954]: pgmap v7996: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:39:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:08 smithi138 ceph-mon[36954]: osdmap e546: 12 total, 12 up, 12 in 2024-07-26T17:39:10.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:39:10.503+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:39:10.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:39:10.504+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:39:10.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:39:10.504+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:39:10.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:39:10.504+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:39:10.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:39:10.504+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:39:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:10 smithi100 ceph-mon[32599]: pgmap v7998: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:39:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:10 smithi167 ceph-mon[37140]: pgmap v7998: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:39:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:10 smithi138 ceph-mon[36954]: pgmap v7998: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:39:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:12 smithi100 ceph-mon[32599]: pgmap v7999: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:39:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:12 smithi167 ceph-mon[37140]: pgmap v7999: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:39:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:12 smithi138 ceph-mon[36954]: pgmap v7999: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:39:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:14 smithi100 ceph-mon[32599]: pgmap v8000: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 25 op/s 2024-07-26T17:39:15.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:14 smithi167 ceph-mon[37140]: pgmap v8000: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 25 op/s 2024-07-26T17:39:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:14 smithi138 ceph-mon[36954]: pgmap v8000: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 25 op/s 2024-07-26T17:39:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:16 smithi100 ceph-mon[32599]: pgmap v8001: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-07-26T17:39:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:16 smithi167 ceph-mon[37140]: pgmap v8001: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-07-26T17:39:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:16 smithi138 ceph-mon[36954]: pgmap v8001: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-07-26T17:39:18.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:18 smithi138 ceph-mon[36954]: pgmap v8002: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-07-26T17:39:18.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:18 smithi138 ceph-mon[36954]: osdmap e547: 12 total, 12 up, 12 in 2024-07-26T17:39:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:18 smithi100 ceph-mon[32599]: pgmap v8002: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-07-26T17:39:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:18 smithi100 ceph-mon[32599]: osdmap e547: 12 total, 12 up, 12 in 2024-07-26T17:39:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:18 smithi167 ceph-mon[37140]: pgmap v8002: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-07-26T17:39:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:18 smithi167 ceph-mon[37140]: osdmap e547: 12 total, 12 up, 12 in 2024-07-26T17:39:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:20 smithi138 ceph-mon[36954]: pgmap v8004: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-07-26T17:39:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:20 smithi100 ceph-mon[32599]: pgmap v8004: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-07-26T17:39:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:20 smithi167 ceph-mon[37140]: pgmap v8004: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-07-26T17:39:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:22 smithi138 ceph-mon[36954]: pgmap v8005: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-07-26T17:39:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:22 smithi100 ceph-mon[32599]: pgmap v8005: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-07-26T17:39:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:22 smithi167 ceph-mon[37140]: pgmap v8005: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-07-26T17:39:24.796 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:24 smithi100 ceph-mon[32599]: pgmap v8006: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-07-26T17:39:24.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:24 smithi167 ceph-mon[37140]: pgmap v8006: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-07-26T17:39:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:24 smithi138 ceph-mon[36954]: pgmap v8006: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2024-07-26T17:39:25.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:39:25.519+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:39:25.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:39:25.519+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:39:25.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:39:25.519+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:39:25.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:39:25.519+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:39:25.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:39:25.519+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:39:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:26 smithi138 ceph-mon[36954]: pgmap v8007: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T17:39:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:26 smithi100 ceph-mon[32599]: pgmap v8007: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T17:39:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:26 smithi167 ceph-mon[37140]: pgmap v8007: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T17:39:28.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:28 smithi138 ceph-mon[36954]: pgmap v8008: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T17:39:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:28 smithi100 ceph-mon[32599]: pgmap v8008: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T17:39:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:28 smithi167 ceph-mon[37140]: pgmap v8008: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-07-26T17:39:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:30 smithi138 ceph-mon[36954]: pgmap v8009: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 8 op/s 2024-07-26T17:39:30.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:30 smithi100 ceph-mon[32599]: pgmap v8009: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 8 op/s 2024-07-26T17:39:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:30 smithi167 ceph-mon[37140]: pgmap v8009: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 8 op/s 2024-07-26T17:39:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:32 smithi138 ceph-mon[36954]: pgmap v8010: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-07-26T17:39:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:32 smithi100 ceph-mon[32599]: pgmap v8010: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-07-26T17:39:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:32 smithi167 ceph-mon[37140]: pgmap v8010: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-07-26T17:39:34.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:34 smithi138 ceph-mon[36954]: pgmap v8011: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:39:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:34 smithi100 ceph-mon[32599]: pgmap v8011: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:39:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:34 smithi167 ceph-mon[37140]: pgmap v8011: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:39:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:36 smithi138 ceph-mon[36954]: pgmap v8012: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-07-26T17:39:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:36 smithi100 ceph-mon[32599]: pgmap v8012: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-07-26T17:39:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:36 smithi167 ceph-mon[37140]: pgmap v8012: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-07-26T17:39:38.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:38 smithi138 ceph-mon[36954]: pgmap v8013: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:39:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:38 smithi100 ceph-mon[32599]: pgmap v8013: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:39:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:38 smithi167 ceph-mon[37140]: pgmap v8013: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:39:40.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:39:40.535+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:39:40.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:39:40.536+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:39:40.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:39:40.536+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:39:40.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:39:40.536+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:39:40.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:39:40.536+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:39:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:40 smithi138 ceph-mon[36954]: pgmap v8014: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-07-26T17:39:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:40 smithi100 ceph-mon[32599]: pgmap v8014: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-07-26T17:39:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:40 smithi167 ceph-mon[37140]: pgmap v8014: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-07-26T17:39:42.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:42 smithi138 ceph-mon[36954]: pgmap v8015: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-07-26T17:39:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:42 smithi100 ceph-mon[32599]: pgmap v8015: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-07-26T17:39:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:42 smithi167 ceph-mon[37140]: pgmap v8015: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-07-26T17:39:43.891 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:43 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:39:43.891 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:43 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:39:43.892 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:43 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:39:43.892 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:43 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:39:43.892 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:43 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:39:43.910 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:43 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:39:43.911 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:43 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:39:43.911 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:43 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:39:43.911 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:43 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:39:43.911 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:43 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:39:43.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:43 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:39:44.059 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:43 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:39:44.059 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:43 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:39:44.059 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:43 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:39:44.060 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:43 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:39:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:44 smithi167 ceph-mon[37140]: pgmap v8016: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:39:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:44 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T17:39:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:44 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T17:39:44.901 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:44 smithi138 ceph-mon[36954]: pgmap v8016: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:39:44.901 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:44 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T17:39:44.901 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:44 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T17:39:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:44 smithi100 ceph-mon[32599]: pgmap v8016: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:39:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:44 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T17:39:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:44 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T17:39:46.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:46 smithi138 ceph-mon[36954]: pgmap v8017: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-07-26T17:39:46.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:39:46.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:39:46.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:39:46.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:39:46.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:39:46.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:39:46.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:39:46.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:39:46.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:39:46.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:39:46.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:39:46.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:39:46.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:39:46.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:39:46.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:46 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:39:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:46 smithi167 ceph-mon[37140]: pgmap v8017: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-07-26T17:39:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:39:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:39:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:39:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:39:46.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:39:46.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:39:46.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:39:46.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:39:46.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:39:46.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:39:46.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:39:46.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:39:46.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:39:46.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:39:46.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:46 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:39:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:46 smithi100 ceph-mon[32599]: pgmap v8017: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-07-26T17:39:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:39:46.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:39:46.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:39:46.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:39:46.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:39:46.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:39:46.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:39:46.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:39:46.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:39:46.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:39:46.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:39:46.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:39:46.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:39:46.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:39:46.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:46 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:39:47.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:47 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T17:39:47.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:47 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T17:39:47.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:47 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T17:39:47.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:47 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T17:39:47.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:47 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T17:39:47.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:47 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T17:39:47.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:47 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T17:39:47.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:47 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T17:39:48.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:47 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T17:39:48.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:47 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T17:39:48.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:47 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T17:39:48.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:47 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T17:39:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:48 smithi138 ceph-mon[36954]: pgmap v8018: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:39:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:48 smithi100 ceph-mon[32599]: pgmap v8018: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:39:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:48 smithi167 ceph-mon[37140]: pgmap v8018: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:39:50.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:50 smithi138 ceph-mon[36954]: pgmap v8019: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-07-26T17:39:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:50 smithi100 ceph-mon[32599]: pgmap v8019: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-07-26T17:39:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:50 smithi167 ceph-mon[37140]: pgmap v8019: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-07-26T17:39:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:52 smithi138 ceph-mon[36954]: pgmap v8020: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-07-26T17:39:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:52 smithi100 ceph-mon[32599]: pgmap v8020: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-07-26T17:39:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:52 smithi167 ceph-mon[37140]: pgmap v8020: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-07-26T17:39:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:54 smithi138 ceph-mon[36954]: pgmap v8021: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:39:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:54 smithi100 ceph-mon[32599]: pgmap v8021: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:39:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:54 smithi167 ceph-mon[37140]: pgmap v8021: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:39:55.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:39:55.552+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:39:55.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:39:55.552+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:39:55.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:39:55.552+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:39:55.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:39:55.552+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:39:55.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:39:55.552+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:39:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:56 smithi138 ceph-mon[36954]: pgmap v8022: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 7 op/s 2024-07-26T17:39:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:56 smithi100 ceph-mon[32599]: pgmap v8022: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 7 op/s 2024-07-26T17:39:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:56 smithi167 ceph-mon[37140]: pgmap v8022: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 7 op/s 2024-07-26T17:39:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:39:58 smithi138 ceph-mon[36954]: pgmap v8023: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-07-26T17:39:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:39:58 smithi100 ceph-mon[32599]: pgmap v8023: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-07-26T17:39:59.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:39:58 smithi167 ceph-mon[37140]: pgmap v8023: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-07-26T17:40:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:40:00 smithi138 ceph-mon[36954]: pgmap v8024: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 7 op/s 2024-07-26T17:40:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:40:00 smithi138 ceph-mon[36954]: overall HEALTH_OK 2024-07-26T17:40:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:40:00 smithi100 ceph-mon[32599]: pgmap v8024: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 7 op/s 2024-07-26T17:40:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:40:00 smithi100 ceph-mon[32599]: overall HEALTH_OK 2024-07-26T17:40:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:40:00 smithi167 ceph-mon[37140]: pgmap v8024: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 7 op/s 2024-07-26T17:40:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:40:00 smithi167 ceph-mon[37140]: overall HEALTH_OK 2024-07-26T17:40:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:40:02 smithi138 ceph-mon[36954]: pgmap v8025: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 6 op/s 2024-07-26T17:40:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:40:02 smithi100 ceph-mon[32599]: pgmap v8025: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 6 op/s 2024-07-26T17:40:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:40:02 smithi167 ceph-mon[37140]: pgmap v8025: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 6 op/s 2024-07-26T17:40:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:40:04 smithi138 ceph-mon[36954]: pgmap v8026: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:40:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:40:04 smithi100 ceph-mon[32599]: pgmap v8026: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:40:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:40:04 smithi167 ceph-mon[37140]: pgmap v8026: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:40:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:40:06 smithi138 ceph-mon[36954]: pgmap v8027: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T17:40:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:40:06 smithi100 ceph-mon[32599]: pgmap v8027: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T17:40:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:40:06 smithi167 ceph-mon[37140]: pgmap v8027: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T17:40:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:40:08 smithi138 ceph-mon[36954]: pgmap v8028: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:40:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:40:08 smithi138 ceph-mon[36954]: osdmap e548: 12 total, 12 up, 12 in 2024-07-26T17:40:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:40:08 smithi100 ceph-mon[32599]: pgmap v8028: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:40:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:40:08 smithi100 ceph-mon[32599]: osdmap e548: 12 total, 12 up, 12 in 2024-07-26T17:40:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:40:08 smithi167 ceph-mon[37140]: pgmap v8028: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:40:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:40:08 smithi167 ceph-mon[37140]: osdmap e548: 12 total, 12 up, 12 in 2024-07-26T17:40:10.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:40:10.556+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:40:10.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:40:10.557+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:40:10.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:40:10.557+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:40:10.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:40:10.557+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:40:10.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:40:10.557+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:40:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:40:10 smithi138 ceph-mon[36954]: pgmap v8030: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:40:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:40:10 smithi100 ceph-mon[32599]: pgmap v8030: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:40:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:40:10 smithi167 ceph-mon[37140]: pgmap v8030: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:40:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:40:12 smithi138 ceph-mon[36954]: pgmap v8031: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:40:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:40:12 smithi100 ceph-mon[32599]: pgmap v8031: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:40:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:40:12 smithi167 ceph-mon[37140]: pgmap v8031: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:40:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:40:14 smithi138 ceph-mon[36954]: pgmap v8032: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:40:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:40:14 smithi100 ceph-mon[32599]: pgmap v8032: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:40:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:40:14 smithi167 ceph-mon[37140]: pgmap v8032: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:40:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:40:16 smithi138 ceph-mon[36954]: pgmap v8033: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:40:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:40:16 smithi100 ceph-mon[32599]: pgmap v8033: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:40:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:40:16 smithi167 ceph-mon[37140]: pgmap v8033: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:40:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:40:18 smithi138 ceph-mon[36954]: pgmap v8034: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:40:18.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:40:18 smithi138 ceph-mon[36954]: osdmap e549: 12 total, 12 up, 12 in 2024-07-26T17:40:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:40:18 smithi100 ceph-mon[32599]: pgmap v8034: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:40:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:40:18 smithi100 ceph-mon[32599]: osdmap e549: 12 total, 12 up, 12 in 2024-07-26T17:40:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:40:18 smithi167 ceph-mon[37140]: pgmap v8034: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:40:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:40:18 smithi167 ceph-mon[37140]: osdmap e549: 12 total, 12 up, 12 in 2024-07-26T17:40:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:40:20 smithi138 ceph-mon[36954]: pgmap v8036: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:40:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:40:20 smithi100 ceph-mon[32599]: pgmap v8036: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:40:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:40:20 smithi167 ceph-mon[37140]: pgmap v8036: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:40:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:40:22 smithi138 ceph-mon[36954]: pgmap v8037: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:40:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:40:22 smithi100 ceph-mon[32599]: pgmap v8037: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:40:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:40:22 smithi167 ceph-mon[37140]: pgmap v8037: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:40:24.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:40:24 smithi167 ceph-mon[37140]: pgmap v8038: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:40:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:40:24 smithi138 ceph-mon[36954]: pgmap v8038: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:40:25.010 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:40:24 smithi100 ceph-mon[32599]: pgmap v8038: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:40:25.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:40:25.560+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:40:25.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:40:25.561+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:40:25.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:40:25.561+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:40:25.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:40:25.561+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:40:25.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:40:25.561+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:40:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:40:26 smithi138 ceph-mon[36954]: pgmap v8039: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:40:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:40:26 smithi100 ceph-mon[32599]: pgmap v8039: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:40:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:40:26 smithi167 ceph-mon[37140]: pgmap v8039: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:40:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:40:28 smithi138 ceph-mon[36954]: pgmap v8040: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:40:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:40:28 smithi100 ceph-mon[32599]: pgmap v8040: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:40:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:40:28 smithi167 ceph-mon[37140]: pgmap v8040: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:40:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:40:30 smithi138 ceph-mon[36954]: pgmap v8041: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:40:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:40:30 smithi100 ceph-mon[32599]: pgmap v8041: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:40:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:40:30 smithi167 ceph-mon[37140]: pgmap v8041: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:40:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:40:32 smithi138 ceph-mon[36954]: pgmap v8042: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:40:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:40:32 smithi100 ceph-mon[32599]: pgmap v8042: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:40:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:40:32 smithi167 ceph-mon[37140]: pgmap v8042: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:40:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:40:34 smithi138 ceph-mon[36954]: pgmap v8043: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:40:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:40:34 smithi100 ceph-mon[32599]: pgmap v8043: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:40:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:40:34 smithi167 ceph-mon[37140]: pgmap v8043: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:40:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:40:36 smithi138 ceph-mon[36954]: pgmap v8044: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:40:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:40:36 smithi100 ceph-mon[32599]: pgmap v8044: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:40:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:40:36 smithi167 ceph-mon[37140]: pgmap v8044: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:40:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:40:38 smithi138 ceph-mon[36954]: pgmap v8045: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:40:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:40:38 smithi100 ceph-mon[32599]: pgmap v8045: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:40:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:40:38 smithi167 ceph-mon[37140]: pgmap v8045: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:40:40.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:40:40.571+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:40:40.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:40:40.572+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:40:40.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:40:40.572+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:40:40.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:40:40.572+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:40:40.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:40:40.572+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:40:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:40:40 smithi138 ceph-mon[36954]: pgmap v8046: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:40:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:40:40 smithi100 ceph-mon[32599]: pgmap v8046: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:40:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:40:40 smithi167 ceph-mon[37140]: pgmap v8046: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:40:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:40:42 smithi138 ceph-mon[36954]: pgmap v8047: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:40:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:40:42 smithi100 ceph-mon[32599]: pgmap v8047: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:40:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:40:42 smithi167 ceph-mon[37140]: pgmap v8047: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:40:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:40:44 smithi138 ceph-mon[36954]: pgmap v8048: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:40:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:40:44 smithi100 ceph-mon[32599]: pgmap v8048: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:40:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:40:44 smithi167 ceph-mon[37140]: pgmap v8048: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:40:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:40:46 smithi138 ceph-mon[36954]: pgmap v8049: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:40:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:40:46 smithi100 ceph-mon[32599]: pgmap v8049: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:40:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:40:46 smithi167 ceph-mon[37140]: pgmap v8049: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:40:47.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:40:47 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:40:47.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:40:47 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:40:47.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:40:47 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:40:47.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:40:47 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:40:47.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:40:47 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:40:47.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:40:47 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:40:47.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:40:47 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:40:47.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:40:47 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:40:48.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:40:47 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:40:48.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:40:47 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:40:48.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:40:47 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:40:48.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:40:47 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:40:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:40:48 smithi138 ceph-mon[36954]: pgmap v8050: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:40:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:40:48 smithi100 ceph-mon[32599]: pgmap v8050: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:40:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:40:48 smithi167 ceph-mon[37140]: pgmap v8050: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:40:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:40:50 smithi138 ceph-mon[36954]: pgmap v8051: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:40:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:40:50 smithi100 ceph-mon[32599]: pgmap v8051: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:40:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:40:50 smithi167 ceph-mon[37140]: pgmap v8051: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:40:52.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:40:52 smithi138 ceph-mon[36954]: pgmap v8052: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:40:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:40:52 smithi100 ceph-mon[32599]: pgmap v8052: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:40:53.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:40:52 smithi167 ceph-mon[37140]: pgmap v8052: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:40:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:40:54 smithi138 ceph-mon[36954]: pgmap v8053: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:40:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:40:54 smithi100 ceph-mon[32599]: pgmap v8053: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:40:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:40:54 smithi167 ceph-mon[37140]: pgmap v8053: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:40:55.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:40:55.574+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:40:55.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:40:55.574+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:40:55.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:40:55.574+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:40:55.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:40:55.574+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:40:55.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:40:55.574+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:40:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:40:56 smithi138 ceph-mon[36954]: pgmap v8054: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:40:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:40:56 smithi100 ceph-mon[32599]: pgmap v8054: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:40:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:40:56 smithi167 ceph-mon[37140]: pgmap v8054: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:40:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:40:58 smithi138 ceph-mon[36954]: pgmap v8055: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:40:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:40:58 smithi100 ceph-mon[32599]: pgmap v8055: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:40:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:40:58 smithi167 ceph-mon[37140]: pgmap v8055: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:41:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:41:00 smithi138 ceph-mon[36954]: pgmap v8056: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:41:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:41:00 smithi100 ceph-mon[32599]: pgmap v8056: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:41:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:41:00 smithi167 ceph-mon[37140]: pgmap v8056: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:41:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:41:02 smithi138 ceph-mon[36954]: pgmap v8057: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:41:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:41:02 smithi100 ceph-mon[32599]: pgmap v8057: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:41:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:41:02 smithi167 ceph-mon[37140]: pgmap v8057: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:41:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:41:04 smithi138 ceph-mon[36954]: pgmap v8058: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:41:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:41:04 smithi100 ceph-mon[32599]: pgmap v8058: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:41:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:41:04 smithi167 ceph-mon[37140]: pgmap v8058: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:41:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:41:06 smithi138 ceph-mon[36954]: pgmap v8059: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:41:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:41:06 smithi100 ceph-mon[32599]: pgmap v8059: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:41:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:41:06 smithi167 ceph-mon[37140]: pgmap v8059: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:41:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:41:08 smithi138 ceph-mon[36954]: pgmap v8060: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:41:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:41:08 smithi138 ceph-mon[36954]: osdmap e550: 12 total, 12 up, 12 in 2024-07-26T17:41:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:41:08 smithi100 ceph-mon[32599]: pgmap v8060: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:41:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:41:08 smithi100 ceph-mon[32599]: osdmap e550: 12 total, 12 up, 12 in 2024-07-26T17:41:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:41:08 smithi167 ceph-mon[37140]: pgmap v8060: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:41:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:41:08 smithi167 ceph-mon[37140]: osdmap e550: 12 total, 12 up, 12 in 2024-07-26T17:41:10.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:41:10.590+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:41:10.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:41:10.590+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:41:10.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:41:10.590+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:41:10.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:41:10.590+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:41:10.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:41:10.590+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:41:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:41:10 smithi138 ceph-mon[36954]: pgmap v8062: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:41:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:41:10 smithi100 ceph-mon[32599]: pgmap v8062: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:41:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:41:10 smithi167 ceph-mon[37140]: pgmap v8062: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:41:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:41:12 smithi138 ceph-mon[36954]: pgmap v8063: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:41:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:41:12 smithi100 ceph-mon[32599]: pgmap v8063: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:41:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:41:12 smithi167 ceph-mon[37140]: pgmap v8063: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:41:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:41:14 smithi138 ceph-mon[36954]: pgmap v8064: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:41:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:41:14 smithi100 ceph-mon[32599]: pgmap v8064: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:41:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:41:14 smithi167 ceph-mon[37140]: pgmap v8064: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:41:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:41:16 smithi138 ceph-mon[36954]: pgmap v8065: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:41:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:41:16 smithi100 ceph-mon[32599]: pgmap v8065: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:41:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:41:16 smithi167 ceph-mon[37140]: pgmap v8065: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:41:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:41:18 smithi138 ceph-mon[36954]: pgmap v8066: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:41:18.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:41:18 smithi138 ceph-mon[36954]: osdmap e551: 12 total, 12 up, 12 in 2024-07-26T17:41:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:41:18 smithi100 ceph-mon[32599]: pgmap v8066: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:41:18.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:41:18 smithi100 ceph-mon[32599]: osdmap e551: 12 total, 12 up, 12 in 2024-07-26T17:41:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:41:18 smithi167 ceph-mon[37140]: pgmap v8066: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:41:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:41:18 smithi167 ceph-mon[37140]: osdmap e551: 12 total, 12 up, 12 in 2024-07-26T17:41:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:41:20 smithi138 ceph-mon[36954]: pgmap v8068: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:41:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:41:20 smithi100 ceph-mon[32599]: pgmap v8068: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:41:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:41:20 smithi167 ceph-mon[37140]: pgmap v8068: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:41:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:41:22 smithi138 ceph-mon[36954]: pgmap v8069: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:41:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:41:22 smithi100 ceph-mon[32599]: pgmap v8069: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:41:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:41:22 smithi167 ceph-mon[37140]: pgmap v8069: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:41:24.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:41:24 smithi138 ceph-mon[36954]: pgmap v8070: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:41:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:41:24 smithi100 ceph-mon[32599]: pgmap v8070: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:41:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:41:24 smithi167 ceph-mon[37140]: pgmap v8070: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:41:25.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:41:25.597+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:41:25.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:41:25.597+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:41:25.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:41:25.597+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:41:25.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:41:25.597+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:41:25.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:41:25.597+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:41:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:41:26 smithi138 ceph-mon[36954]: pgmap v8071: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:41:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:41:26 smithi100 ceph-mon[32599]: pgmap v8071: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:41:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:41:26 smithi167 ceph-mon[37140]: pgmap v8071: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:41:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:41:28 smithi138 ceph-mon[36954]: pgmap v8072: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:41:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:41:28 smithi100 ceph-mon[32599]: pgmap v8072: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:41:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:41:28 smithi167 ceph-mon[37140]: pgmap v8072: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:41:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:41:30 smithi138 ceph-mon[36954]: pgmap v8073: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:41:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:41:30 smithi100 ceph-mon[32599]: pgmap v8073: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:41:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:41:30 smithi167 ceph-mon[37140]: pgmap v8073: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:41:32.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:41:32 smithi138 ceph-mon[36954]: pgmap v8074: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:41:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:41:32 smithi100 ceph-mon[32599]: pgmap v8074: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:41:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:41:32 smithi167 ceph-mon[37140]: pgmap v8074: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:41:34.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:41:34 smithi138 ceph-mon[36954]: pgmap v8075: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:41:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:41:34 smithi100 ceph-mon[32599]: pgmap v8075: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:41:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:41:34 smithi167 ceph-mon[37140]: pgmap v8075: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:41:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:41:36 smithi138 ceph-mon[36954]: pgmap v8076: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:41:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:41:36 smithi100 ceph-mon[32599]: pgmap v8076: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:41:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:41:36 smithi167 ceph-mon[37140]: pgmap v8076: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:41:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:41:38 smithi138 ceph-mon[36954]: pgmap v8077: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:41:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:41:38 smithi100 ceph-mon[32599]: pgmap v8077: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:41:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:41:38 smithi167 ceph-mon[37140]: pgmap v8077: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:41:40.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:41:40.613+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:41:40.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:41:40.613+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:41:40.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:41:40.613+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:41:40.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:41:40.613+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:41:40.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:41:40.613+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:41:40.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:41:40 smithi138 ceph-mon[36954]: pgmap v8078: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:41:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:41:40 smithi100 ceph-mon[32599]: pgmap v8078: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:41:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:41:40 smithi167 ceph-mon[37140]: pgmap v8078: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:41:42.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:41:42 smithi138 ceph-mon[36954]: pgmap v8079: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:41:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:41:42 smithi100 ceph-mon[32599]: pgmap v8079: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:41:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:41:42 smithi167 ceph-mon[37140]: pgmap v8079: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:41:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:41:44 smithi138 ceph-mon[36954]: pgmap v8080: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:41:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:41:44 smithi100 ceph-mon[32599]: pgmap v8080: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:41:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:41:44 smithi167 ceph-mon[37140]: pgmap v8080: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:41:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:41:46 smithi138 ceph-mon[36954]: pgmap v8081: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:41:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:41:46 smithi100 ceph-mon[32599]: pgmap v8081: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:41:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:41:46 smithi167 ceph-mon[37140]: pgmap v8081: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:41:47.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:41:47 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:41:47.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:41:47 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:41:48.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:41:47 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:41:48.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:41:48 smithi138 ceph-mon[36954]: pgmap v8082: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:41:48.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:41:48 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:41:48.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:41:48 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:41:48.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:41:48 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:41:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:41:48 smithi100 ceph-mon[32599]: pgmap v8082: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:41:48.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:41:48 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:41:48.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:41:48 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:41:48.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:41:48 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:41:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:41:48 smithi167 ceph-mon[37140]: pgmap v8082: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:41:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:41:48 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:41:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:41:48 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:41:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:41:48 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:41:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:41:50 smithi138 ceph-mon[36954]: pgmap v8083: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:41:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:41:50 smithi100 ceph-mon[32599]: pgmap v8083: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:41:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:41:50 smithi167 ceph-mon[37140]: pgmap v8083: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:41:52.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:41:52 smithi138 ceph-mon[36954]: pgmap v8084: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:41:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:41:52 smithi100 ceph-mon[32599]: pgmap v8084: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:41:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:41:52 smithi167 ceph-mon[37140]: pgmap v8084: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:41:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:41:54 smithi138 ceph-mon[36954]: pgmap v8085: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:41:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:41:54 smithi100 ceph-mon[32599]: pgmap v8085: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:41:55.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:41:54 smithi167 ceph-mon[37140]: pgmap v8085: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:41:55.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:41:55.623+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:41:55.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:41:55.623+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:41:55.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:41:55.623+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:41:55.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:41:55.623+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:41:55.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:41:55.623+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:41:56.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:41:56 smithi138 ceph-mon[36954]: pgmap v8086: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:41:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:41:56 smithi100 ceph-mon[32599]: pgmap v8086: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:41:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:41:56 smithi167 ceph-mon[37140]: pgmap v8086: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:41:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:41:58 smithi138 ceph-mon[36954]: pgmap v8087: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:41:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:41:58 smithi100 ceph-mon[32599]: pgmap v8087: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:41:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:41:58 smithi167 ceph-mon[37140]: pgmap v8087: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:42:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:00 smithi138 ceph-mon[36954]: pgmap v8088: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:42:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:00 smithi100 ceph-mon[32599]: pgmap v8088: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:42:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:00 smithi167 ceph-mon[37140]: pgmap v8088: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:42:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:02 smithi138 ceph-mon[36954]: pgmap v8089: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:42:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:02 smithi100 ceph-mon[32599]: pgmap v8089: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:42:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:02 smithi167 ceph-mon[37140]: pgmap v8089: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:42:04.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:04 smithi138 ceph-mon[36954]: pgmap v8090: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:42:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:04 smithi100 ceph-mon[32599]: pgmap v8090: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:42:05.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:42:05.005+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T17:42:05.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:42:05.005+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T17:42:05.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:42:05.005+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 111 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c0065c0 con 0x7fe5a81087a0 2024-07-26T17:42:05.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:42:05.005+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 111 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T17:42:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:04 smithi167 ceph-mon[37140]: pgmap v8090: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:42:06.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:06 smithi138 ceph-mon[36954]: pgmap v8091: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:42:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:06 smithi100 ceph-mon[32599]: pgmap v8091: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:42:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:06 smithi167 ceph-mon[37140]: pgmap v8091: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:42:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:08 smithi138 ceph-mon[36954]: pgmap v8092: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:42:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:08 smithi138 ceph-mon[36954]: osdmap e552: 12 total, 12 up, 12 in 2024-07-26T17:42:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:08 smithi100 ceph-mon[32599]: pgmap v8092: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:42:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:08 smithi100 ceph-mon[32599]: osdmap e552: 12 total, 12 up, 12 in 2024-07-26T17:42:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:08 smithi167 ceph-mon[37140]: pgmap v8092: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:42:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:08 smithi167 ceph-mon[37140]: osdmap e552: 12 total, 12 up, 12 in 2024-07-26T17:42:10.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:42:10.631+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:42:10.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:42:10.632+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:42:10.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:42:10.632+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:42:10.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:42:10.632+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:42:10.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:42:10.632+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:42:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:10 smithi138 ceph-mon[36954]: pgmap v8094: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:42:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:10 smithi100 ceph-mon[32599]: pgmap v8094: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:42:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:10 smithi167 ceph-mon[37140]: pgmap v8094: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:42:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:12 smithi138 ceph-mon[36954]: pgmap v8095: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:42:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:12 smithi100 ceph-mon[32599]: pgmap v8095: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:42:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:12 smithi167 ceph-mon[37140]: pgmap v8095: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:42:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:14 smithi138 ceph-mon[36954]: pgmap v8096: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:42:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:14 smithi100 ceph-mon[32599]: pgmap v8096: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:42:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:14 smithi167 ceph-mon[37140]: pgmap v8096: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:42:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:16 smithi138 ceph-mon[36954]: pgmap v8097: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:42:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:16 smithi100 ceph-mon[32599]: pgmap v8097: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:42:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:16 smithi167 ceph-mon[37140]: pgmap v8097: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:42:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:18 smithi138 ceph-mon[36954]: pgmap v8098: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:42:18.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:18 smithi138 ceph-mon[36954]: osdmap e553: 12 total, 12 up, 12 in 2024-07-26T17:42:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:18 smithi100 ceph-mon[32599]: pgmap v8098: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:42:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:18 smithi100 ceph-mon[32599]: osdmap e553: 12 total, 12 up, 12 in 2024-07-26T17:42:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:18 smithi167 ceph-mon[37140]: pgmap v8098: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:42:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:18 smithi167 ceph-mon[37140]: osdmap e553: 12 total, 12 up, 12 in 2024-07-26T17:42:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:20 smithi138 ceph-mon[36954]: pgmap v8100: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:42:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:20 smithi100 ceph-mon[32599]: pgmap v8100: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:42:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:20 smithi167 ceph-mon[37140]: pgmap v8100: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:42:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:22 smithi138 ceph-mon[36954]: pgmap v8101: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:42:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:22 smithi100 ceph-mon[32599]: pgmap v8101: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:42:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:22 smithi167 ceph-mon[37140]: pgmap v8101: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:42:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:24 smithi138 ceph-mon[36954]: pgmap v8102: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:42:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:24 smithi100 ceph-mon[32599]: pgmap v8102: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:42:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:24 smithi167 ceph-mon[37140]: pgmap v8102: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:42:25.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:42:25.641+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:42:25.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:42:25.642+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:42:25.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:42:25.642+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:42:25.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:42:25.642+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:42:25.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:42:25.642+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:42:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:26 smithi138 ceph-mon[36954]: pgmap v8103: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:42:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:26 smithi100 ceph-mon[32599]: pgmap v8103: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:42:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:26 smithi167 ceph-mon[37140]: pgmap v8103: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:42:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:28 smithi138 ceph-mon[36954]: pgmap v8104: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:42:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:28 smithi100 ceph-mon[32599]: pgmap v8104: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:42:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:28 smithi167 ceph-mon[37140]: pgmap v8104: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:42:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:30 smithi138 ceph-mon[36954]: pgmap v8105: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:42:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:30 smithi100 ceph-mon[32599]: pgmap v8105: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:42:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:30 smithi167 ceph-mon[37140]: pgmap v8105: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:42:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:32 smithi138 ceph-mon[36954]: pgmap v8106: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:42:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:32 smithi100 ceph-mon[32599]: pgmap v8106: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:42:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:32 smithi167 ceph-mon[37140]: pgmap v8106: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:42:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:34 smithi138 ceph-mon[36954]: pgmap v8107: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:42:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:34 smithi100 ceph-mon[32599]: pgmap v8107: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:42:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:34 smithi167 ceph-mon[37140]: pgmap v8107: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:42:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:36 smithi138 ceph-mon[36954]: pgmap v8108: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:42:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:36 smithi100 ceph-mon[32599]: pgmap v8108: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:42:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:36 smithi167 ceph-mon[37140]: pgmap v8108: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:42:38.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:38 smithi138 ceph-mon[36954]: pgmap v8109: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:42:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:38 smithi100 ceph-mon[32599]: pgmap v8109: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:42:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:38 smithi167 ceph-mon[37140]: pgmap v8109: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:42:40.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:42:40.657+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:42:40.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:42:40.658+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:42:40.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:42:40.658+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:42:40.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:42:40.658+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:42:40.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:42:40.658+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:42:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:40 smithi138 ceph-mon[36954]: pgmap v8110: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:42:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:40 smithi100 ceph-mon[32599]: pgmap v8110: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:42:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:40 smithi167 ceph-mon[37140]: pgmap v8110: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:42:42.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:42 smithi138 ceph-mon[36954]: pgmap v8111: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:42:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:42 smithi100 ceph-mon[32599]: pgmap v8111: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:42:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:42 smithi167 ceph-mon[37140]: pgmap v8111: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:42:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:44 smithi138 ceph-mon[36954]: pgmap v8112: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:42:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:44 smithi100 ceph-mon[32599]: pgmap v8112: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:42:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:44 smithi167 ceph-mon[37140]: pgmap v8112: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:42:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:46 smithi138 ceph-mon[36954]: pgmap v8113: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:42:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:46 smithi100 ceph-mon[32599]: pgmap v8113: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:42:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:46 smithi167 ceph-mon[37140]: pgmap v8113: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:42:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:48 smithi100 ceph-mon[32599]: pgmap v8114: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:42:48.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:48 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:42:48.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:48 smithi167 ceph-mon[37140]: pgmap v8114: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:42:48.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:48 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:42:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:48 smithi138 ceph-mon[36954]: pgmap v8114: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:42:48.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:48 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:42:50.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:42:50.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:42:50.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:50 smithi100 ceph-mon[32599]: pgmap v8115: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:42:50.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:42:50.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:42:50.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:42:50.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:42:50.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:42:50.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:42:50.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:42:50.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:42:50.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:42:50.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:50 smithi167 ceph-mon[37140]: pgmap v8115: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:42:50.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:42:50.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:42:50.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:42:50.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:42:50.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:42:50.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:42:50.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:42:50.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:42:50.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:42:50.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:50 smithi138 ceph-mon[36954]: pgmap v8115: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:42:50.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:42:50.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:42:50.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:42:50.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:42:50.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:42:50.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:42:50.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:42:52.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:52 smithi138 ceph-mon[36954]: pgmap v8116: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:42:52.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:52 smithi100 ceph-mon[32599]: pgmap v8116: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:42:52.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:52 smithi167 ceph-mon[37140]: pgmap v8116: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:42:54.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:54 smithi138 ceph-mon[36954]: pgmap v8117: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:42:54.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:54 smithi100 ceph-mon[32599]: pgmap v8117: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:42:54.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:54 smithi167 ceph-mon[37140]: pgmap v8117: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:42:55.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:42:55.674+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:42:55.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:42:55.674+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:42:55.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:42:55.674+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:42:55.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:42:55.674+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:42:55.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:42:55.674+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:42:56.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:56 smithi138 ceph-mon[36954]: pgmap v8118: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:42:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:56 smithi100 ceph-mon[32599]: pgmap v8118: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:42:56.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:56 smithi167 ceph-mon[37140]: pgmap v8118: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:42:58.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:42:58 smithi138 ceph-mon[36954]: pgmap v8119: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:42:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:42:58 smithi100 ceph-mon[32599]: pgmap v8119: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:42:58.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:42:58 smithi167 ceph-mon[37140]: pgmap v8119: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:43:00.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:00 smithi138 ceph-mon[36954]: pgmap v8120: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:43:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:00 smithi100 ceph-mon[32599]: pgmap v8120: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:43:00.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:00 smithi167 ceph-mon[37140]: pgmap v8120: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:43:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:02 smithi138 ceph-mon[36954]: pgmap v8121: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:43:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:02 smithi100 ceph-mon[32599]: pgmap v8121: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:43:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:02 smithi167 ceph-mon[37140]: pgmap v8121: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:43:04.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:04 smithi138 ceph-mon[36954]: pgmap v8122: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:43:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:04 smithi100 ceph-mon[32599]: pgmap v8122: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:43:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:04 smithi167 ceph-mon[37140]: pgmap v8122: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:43:06.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:06 smithi138 ceph-mon[36954]: pgmap v8123: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:43:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:06 smithi100 ceph-mon[32599]: pgmap v8123: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:43:06.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:06 smithi167 ceph-mon[37140]: pgmap v8123: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:43:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:08 smithi138 ceph-mon[36954]: pgmap v8124: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:43:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:08 smithi138 ceph-mon[36954]: osdmap e554: 12 total, 12 up, 12 in 2024-07-26T17:43:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:08 smithi100 ceph-mon[32599]: pgmap v8124: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:43:08.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:08 smithi100 ceph-mon[32599]: osdmap e554: 12 total, 12 up, 12 in 2024-07-26T17:43:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:08 smithi167 ceph-mon[37140]: pgmap v8124: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:43:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:08 smithi167 ceph-mon[37140]: osdmap e554: 12 total, 12 up, 12 in 2024-07-26T17:43:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:10 smithi138 ceph-mon[36954]: pgmap v8126: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:43:10.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:43:10.690+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:43:10.692 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:10 smithi100 ceph-mon[32599]: pgmap v8126: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:43:10.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:43:10.691+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:43:10.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:43:10.691+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:43:10.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:43:10.691+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:43:10.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:43:10.691+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:43:10.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:10 smithi167 ceph-mon[37140]: pgmap v8126: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:43:12.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:12 smithi138 ceph-mon[36954]: pgmap v8127: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:43:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:12 smithi100 ceph-mon[32599]: pgmap v8127: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:43:12.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:12 smithi167 ceph-mon[37140]: pgmap v8127: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:43:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:14 smithi138 ceph-mon[36954]: pgmap v8128: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:43:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:14 smithi100 ceph-mon[32599]: pgmap v8128: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:43:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:14 smithi167 ceph-mon[37140]: pgmap v8128: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:43:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:16 smithi138 ceph-mon[36954]: pgmap v8129: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:43:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:16 smithi100 ceph-mon[32599]: pgmap v8129: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:43:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:16 smithi167 ceph-mon[37140]: pgmap v8129: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:43:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:18 smithi138 ceph-mon[36954]: pgmap v8130: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:43:18.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:18 smithi138 ceph-mon[36954]: osdmap e555: 12 total, 12 up, 12 in 2024-07-26T17:43:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:18 smithi100 ceph-mon[32599]: pgmap v8130: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:43:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:18 smithi100 ceph-mon[32599]: osdmap e555: 12 total, 12 up, 12 in 2024-07-26T17:43:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:18 smithi167 ceph-mon[37140]: pgmap v8130: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:43:19.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:18 smithi167 ceph-mon[37140]: osdmap e555: 12 total, 12 up, 12 in 2024-07-26T17:43:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:20 smithi138 ceph-mon[36954]: pgmap v8132: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:43:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:20 smithi100 ceph-mon[32599]: pgmap v8132: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:43:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:20 smithi167 ceph-mon[37140]: pgmap v8132: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:43:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:22 smithi138 ceph-mon[36954]: pgmap v8133: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:43:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:22 smithi100 ceph-mon[32599]: pgmap v8133: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:43:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:22 smithi167 ceph-mon[37140]: pgmap v8133: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:43:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:24 smithi138 ceph-mon[36954]: pgmap v8134: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:43:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:24 smithi100 ceph-mon[32599]: pgmap v8134: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:43:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:24 smithi167 ceph-mon[37140]: pgmap v8134: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:43:25.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:43:25.706+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:43:25.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:43:25.707+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:43:25.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:43:25.707+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:43:25.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:43:25.707+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:43:25.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:43:25.707+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:43:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:26 smithi138 ceph-mon[36954]: pgmap v8135: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:43:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:26 smithi100 ceph-mon[32599]: pgmap v8135: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:43:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:26 smithi167 ceph-mon[37140]: pgmap v8135: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:43:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:28 smithi138 ceph-mon[36954]: pgmap v8136: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:43:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:28 smithi100 ceph-mon[32599]: pgmap v8136: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:43:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:28 smithi167 ceph-mon[37140]: pgmap v8136: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:43:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:30 smithi138 ceph-mon[36954]: pgmap v8137: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:43:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:30 smithi100 ceph-mon[32599]: pgmap v8137: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:43:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:30 smithi167 ceph-mon[37140]: pgmap v8137: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:43:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:32 smithi138 ceph-mon[36954]: pgmap v8138: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:43:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:32 smithi100 ceph-mon[32599]: pgmap v8138: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:43:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:32 smithi167 ceph-mon[37140]: pgmap v8138: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:43:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:34 smithi138 ceph-mon[36954]: pgmap v8139: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:43:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:34 smithi100 ceph-mon[32599]: pgmap v8139: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:43:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:34 smithi167 ceph-mon[37140]: pgmap v8139: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:43:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:36 smithi138 ceph-mon[36954]: pgmap v8140: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:43:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:36 smithi100 ceph-mon[32599]: pgmap v8140: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:43:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:36 smithi167 ceph-mon[37140]: pgmap v8140: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:43:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:38 smithi138 ceph-mon[36954]: pgmap v8141: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:43:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:38 smithi100 ceph-mon[32599]: pgmap v8141: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:43:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:38 smithi167 ceph-mon[37140]: pgmap v8141: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:43:40.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:43:40.723+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:43:40.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:43:40.723+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:43:40.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:43:40.723+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:43:40.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:43:40.723+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:43:40.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:43:40.723+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:43:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:40 smithi138 ceph-mon[36954]: pgmap v8142: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:43:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:40 smithi100 ceph-mon[32599]: pgmap v8142: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:43:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:40 smithi167 ceph-mon[37140]: pgmap v8142: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:43:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:42 smithi138 ceph-mon[36954]: pgmap v8143: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:43:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:42 smithi100 ceph-mon[32599]: pgmap v8143: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:43:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:42 smithi167 ceph-mon[37140]: pgmap v8143: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:43:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:44 smithi138 ceph-mon[36954]: pgmap v8144: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:43:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:44 smithi100 ceph-mon[32599]: pgmap v8144: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:43:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:44 smithi167 ceph-mon[37140]: pgmap v8144: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:43:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:46 smithi138 ceph-mon[36954]: pgmap v8145: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:43:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:46 smithi100 ceph-mon[32599]: pgmap v8145: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:43:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:46 smithi167 ceph-mon[37140]: pgmap v8145: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:43:48.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:48 smithi138 ceph-mon[36954]: pgmap v8146: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:43:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:48 smithi100 ceph-mon[32599]: pgmap v8146: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:43:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:48 smithi167 ceph-mon[37140]: pgmap v8146: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:43:50.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:50 smithi138 ceph-mon[36954]: pgmap v8147: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:43:50.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:43:50.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:43:50.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:43:50.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:43:50.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:43:50.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:43:50.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:43:50.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:43:50.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:43:50.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:43:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:50 smithi100 ceph-mon[32599]: pgmap v8147: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:43:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:43:50.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:43:50.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:43:50.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:43:50.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:43:50.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:43:50.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:43:50.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:43:50.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:43:50.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:43:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:50 smithi167 ceph-mon[37140]: pgmap v8147: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:43:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:43:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:43:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:43:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:43:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:43:51.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:43:51.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:43:51.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:43:51.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:43:51.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:43:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:52 smithi138 ceph-mon[36954]: pgmap v8148: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:43:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:52 smithi100 ceph-mon[32599]: pgmap v8148: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:43:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:52 smithi167 ceph-mon[37140]: pgmap v8148: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:43:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:54 smithi138 ceph-mon[36954]: pgmap v8149: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:43:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:54 smithi100 ceph-mon[32599]: pgmap v8149: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:43:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:54 smithi167 ceph-mon[37140]: pgmap v8149: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:43:55.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:43:55.738+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:43:55.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:43:55.738+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:43:55.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:43:55.738+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:43:55.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:43:55.738+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:43:55.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:43:55.739+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:43:56.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:56 smithi138 ceph-mon[36954]: pgmap v8150: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:43:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:56 smithi100 ceph-mon[32599]: pgmap v8150: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:43:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:56 smithi167 ceph-mon[37140]: pgmap v8150: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:43:58.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:43:58 smithi138 ceph-mon[36954]: pgmap v8151: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:43:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:43:58 smithi100 ceph-mon[32599]: pgmap v8151: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:43:59.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:43:58 smithi167 ceph-mon[37140]: pgmap v8151: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:44:00.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:44:00 smithi138 ceph-mon[36954]: pgmap v8152: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:44:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:44:00 smithi100 ceph-mon[32599]: pgmap v8152: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:44:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:44:00 smithi167 ceph-mon[37140]: pgmap v8152: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:44:02.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:44:02 smithi138 ceph-mon[36954]: pgmap v8153: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:44:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:44:02 smithi100 ceph-mon[32599]: pgmap v8153: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:44:03.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:44:02 smithi167 ceph-mon[37140]: pgmap v8153: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:44:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:44:04 smithi138 ceph-mon[36954]: pgmap v8154: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:44:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:44:04 smithi100 ceph-mon[32599]: pgmap v8154: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:44:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:44:04 smithi167 ceph-mon[37140]: pgmap v8154: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:44:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:44:06 smithi138 ceph-mon[36954]: pgmap v8155: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:44:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:44:06 smithi100 ceph-mon[32599]: pgmap v8155: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:44:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:44:06 smithi167 ceph-mon[37140]: pgmap v8155: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:44:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:44:08 smithi100 ceph-mon[32599]: pgmap v8156: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:44:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:44:08 smithi100 ceph-mon[32599]: osdmap e556: 12 total, 12 up, 12 in 2024-07-26T17:44:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:44:08 smithi167 ceph-mon[37140]: pgmap v8156: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:44:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:44:08 smithi167 ceph-mon[37140]: osdmap e556: 12 total, 12 up, 12 in 2024-07-26T17:44:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:44:08 smithi138 ceph-mon[36954]: pgmap v8156: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:44:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:44:08 smithi138 ceph-mon[36954]: osdmap e556: 12 total, 12 up, 12 in 2024-07-26T17:44:10.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:44:10.751+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:44:10.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:44:10.752+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:44:10.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:44:10.752+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:44:10.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:44:10.752+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:44:10.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:44:10.752+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:44:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:44:10 smithi100 ceph-mon[32599]: pgmap v8158: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:44:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:44:10 smithi167 ceph-mon[37140]: pgmap v8158: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:44:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:44:10 smithi138 ceph-mon[36954]: pgmap v8158: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:44:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:44:12 smithi100 ceph-mon[32599]: pgmap v8159: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:44:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:44:12 smithi167 ceph-mon[37140]: pgmap v8159: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:44:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:44:12 smithi138 ceph-mon[36954]: pgmap v8159: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:44:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:44:14 smithi100 ceph-mon[32599]: pgmap v8160: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:44:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:44:14 smithi167 ceph-mon[37140]: pgmap v8160: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:44:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:44:14 smithi138 ceph-mon[36954]: pgmap v8160: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:44:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:44:16 smithi100 ceph-mon[32599]: pgmap v8161: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:44:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:44:16 smithi167 ceph-mon[37140]: pgmap v8161: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:44:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:44:16 smithi138 ceph-mon[36954]: pgmap v8161: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:44:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:44:18 smithi138 ceph-mon[36954]: pgmap v8162: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:44:18.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:44:18 smithi138 ceph-mon[36954]: osdmap e557: 12 total, 12 up, 12 in 2024-07-26T17:44:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:44:18 smithi100 ceph-mon[32599]: pgmap v8162: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:44:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:44:18 smithi100 ceph-mon[32599]: osdmap e557: 12 total, 12 up, 12 in 2024-07-26T17:44:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:44:18 smithi167 ceph-mon[37140]: pgmap v8162: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:44:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:44:18 smithi167 ceph-mon[37140]: osdmap e557: 12 total, 12 up, 12 in 2024-07-26T17:44:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:44:20 smithi138 ceph-mon[36954]: pgmap v8164: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:44:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:44:20 smithi100 ceph-mon[32599]: pgmap v8164: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:44:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:44:20 smithi167 ceph-mon[37140]: pgmap v8164: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:44:22.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:44:22 smithi138 ceph-mon[36954]: pgmap v8165: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:44:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:44:22 smithi100 ceph-mon[32599]: pgmap v8165: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:44:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:44:22 smithi167 ceph-mon[37140]: pgmap v8165: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:44:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:44:24 smithi138 ceph-mon[36954]: pgmap v8166: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:44:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:44:24 smithi100 ceph-mon[32599]: pgmap v8166: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:44:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:44:24 smithi167 ceph-mon[37140]: pgmap v8166: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:44:25.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:44:25.768+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:44:25.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:44:25.768+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:44:25.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:44:25.768+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:44:25.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:44:25.768+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:44:25.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:44:25.768+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:44:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:44:26 smithi138 ceph-mon[36954]: pgmap v8167: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:44:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:44:26 smithi100 ceph-mon[32599]: pgmap v8167: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:44:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:44:26 smithi167 ceph-mon[37140]: pgmap v8167: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:44:28.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:44:28 smithi138 ceph-mon[36954]: pgmap v8168: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:44:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:44:28 smithi100 ceph-mon[32599]: pgmap v8168: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:44:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:44:28 smithi167 ceph-mon[37140]: pgmap v8168: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:44:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:44:30 smithi138 ceph-mon[36954]: pgmap v8169: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:44:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:44:30 smithi100 ceph-mon[32599]: pgmap v8169: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:44:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:44:30 smithi167 ceph-mon[37140]: pgmap v8169: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:44:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:44:32 smithi138 ceph-mon[36954]: pgmap v8170: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:44:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:44:32 smithi100 ceph-mon[32599]: pgmap v8170: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:44:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:44:32 smithi167 ceph-mon[37140]: pgmap v8170: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:44:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:44:34 smithi138 ceph-mon[36954]: pgmap v8171: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:44:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:44:34 smithi100 ceph-mon[32599]: pgmap v8171: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:44:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:44:34 smithi167 ceph-mon[37140]: pgmap v8171: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:44:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:44:36 smithi138 ceph-mon[36954]: pgmap v8172: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:44:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:44:36 smithi100 ceph-mon[32599]: pgmap v8172: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:44:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:44:36 smithi167 ceph-mon[37140]: pgmap v8172: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:44:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:44:38 smithi138 ceph-mon[36954]: pgmap v8173: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:44:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:44:38 smithi100 ceph-mon[32599]: pgmap v8173: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:44:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:44:38 smithi167 ceph-mon[37140]: pgmap v8173: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:44:40.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:44:40.770+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:44:40.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:44:40.771+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:44:40.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:44:40.771+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:44:40.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:44:40.771+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:44:40.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:44:40.771+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:44:40.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:44:40 smithi138 ceph-mon[36954]: pgmap v8174: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:44:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:44:40 smithi100 ceph-mon[32599]: pgmap v8174: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:44:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:44:40 smithi167 ceph-mon[37140]: pgmap v8174: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:44:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:44:42 smithi138 ceph-mon[36954]: pgmap v8175: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:44:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:44:42 smithi100 ceph-mon[32599]: pgmap v8175: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:44:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:44:42 smithi167 ceph-mon[37140]: pgmap v8175: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:44:44.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:44:44 smithi138 ceph-mon[36954]: pgmap v8176: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:44:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:44:44 smithi100 ceph-mon[32599]: pgmap v8176: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:44:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:44:44 smithi167 ceph-mon[37140]: pgmap v8176: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:44:46.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:44:46 smithi138 ceph-mon[36954]: pgmap v8177: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:44:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:44:46 smithi100 ceph-mon[32599]: pgmap v8177: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:44:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:44:46 smithi167 ceph-mon[37140]: pgmap v8177: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:44:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:44:48 smithi138 ceph-mon[36954]: pgmap v8178: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:44:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:44:48 smithi100 ceph-mon[32599]: pgmap v8178: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:44:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:44:48 smithi167 ceph-mon[37140]: pgmap v8178: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:44:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:44:50 smithi138 ceph-mon[36954]: pgmap v8179: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:44:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:44:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:44:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:44:50 smithi100 ceph-mon[32599]: pgmap v8179: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:44:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:44:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:44:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:44:50 smithi167 ceph-mon[37140]: pgmap v8179: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:44:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:44:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:44:51.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:44:51 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:44:51.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:44:51 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:44:51.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:44:51 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:44:51.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:44:51 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:44:51.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:44:51 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:44:51.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:44:51 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:44:52.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:44:51 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:44:52.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:44:51 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:44:52.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:44:51 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:44:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:44:52 smithi138 ceph-mon[36954]: pgmap v8180: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:44:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:44:52 smithi100 ceph-mon[32599]: pgmap v8180: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:44:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:44:52 smithi167 ceph-mon[37140]: pgmap v8180: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:44:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:44:54 smithi138 ceph-mon[36954]: pgmap v8181: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:44:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:44:54 smithi100 ceph-mon[32599]: pgmap v8181: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:44:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:44:54 smithi167 ceph-mon[37140]: pgmap v8181: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:44:55.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:44:55.787+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:44:55.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:44:55.787+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:44:55.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:44:55.787+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:44:55.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:44:55.787+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:44:55.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:44:55.787+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:44:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:44:56 smithi138 ceph-mon[36954]: pgmap v8182: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:44:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:44:56 smithi100 ceph-mon[32599]: pgmap v8182: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:44:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:44:56 smithi167 ceph-mon[37140]: pgmap v8182: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:44:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:44:58 smithi138 ceph-mon[36954]: pgmap v8183: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:44:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:44:58 smithi100 ceph-mon[32599]: pgmap v8183: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:44:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:44:58 smithi167 ceph-mon[37140]: pgmap v8183: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:45:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:45:00 smithi138 ceph-mon[36954]: pgmap v8184: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:45:00.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:45:00 smithi100 ceph-mon[32599]: pgmap v8184: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:45:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:45:00 smithi167 ceph-mon[37140]: pgmap v8184: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:45:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:45:02 smithi138 ceph-mon[36954]: pgmap v8185: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:45:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:45:02 smithi100 ceph-mon[32599]: pgmap v8185: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:45:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:45:02 smithi167 ceph-mon[37140]: pgmap v8185: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:45:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:45:04 smithi100 ceph-mon[32599]: pgmap v8186: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:45:05.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:45:04 smithi167 ceph-mon[37140]: pgmap v8186: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:45:05.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:45:05.030+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c004740 con 0x7fe5a81087a0 2024-07-26T17:45:05.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:45:05.030+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T17:45:05.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:45:05.031+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 112 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c004740 con 0x7fe5a81087a0 2024-07-26T17:45:05.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:45:05.031+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 112 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T17:45:05.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:45:04 smithi138 ceph-mon[36954]: pgmap v8186: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:45:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:45:06 smithi100 ceph-mon[32599]: pgmap v8187: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:45:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:45:06 smithi167 ceph-mon[37140]: pgmap v8187: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:45:07.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:45:06 smithi138 ceph-mon[36954]: pgmap v8187: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:45:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:45:08 smithi100 ceph-mon[32599]: pgmap v8188: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:45:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:45:08 smithi100 ceph-mon[32599]: osdmap e558: 12 total, 12 up, 12 in 2024-07-26T17:45:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:45:08 smithi167 ceph-mon[37140]: pgmap v8188: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:45:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:45:08 smithi167 ceph-mon[37140]: osdmap e558: 12 total, 12 up, 12 in 2024-07-26T17:45:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:45:08 smithi138 ceph-mon[36954]: pgmap v8188: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:45:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:45:08 smithi138 ceph-mon[36954]: osdmap e558: 12 total, 12 up, 12 in 2024-07-26T17:45:10.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:45:10.802+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:45:10.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:45:10.802+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:45:10.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:45:10.802+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:45:10.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:45:10.802+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:45:10.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:45:10.802+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:45:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:45:10 smithi100 ceph-mon[32599]: pgmap v8190: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:45:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:45:10 smithi167 ceph-mon[37140]: pgmap v8190: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:45:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:45:10 smithi138 ceph-mon[36954]: pgmap v8190: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:45:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:45:12 smithi100 ceph-mon[32599]: pgmap v8191: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:45:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:45:12 smithi167 ceph-mon[37140]: pgmap v8191: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:45:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:45:12 smithi138 ceph-mon[36954]: pgmap v8191: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:45:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:45:14 smithi100 ceph-mon[32599]: pgmap v8192: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:45:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:45:14 smithi167 ceph-mon[37140]: pgmap v8192: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:45:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:45:14 smithi138 ceph-mon[36954]: pgmap v8192: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:45:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:45:16 smithi100 ceph-mon[32599]: pgmap v8193: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:45:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:45:16 smithi167 ceph-mon[37140]: pgmap v8193: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:45:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:45:16 smithi138 ceph-mon[36954]: pgmap v8193: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:45:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:45:18 smithi138 ceph-mon[36954]: pgmap v8194: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:45:18.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:45:18 smithi138 ceph-mon[36954]: osdmap e559: 12 total, 12 up, 12 in 2024-07-26T17:45:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:45:18 smithi100 ceph-mon[32599]: pgmap v8194: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:45:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:45:18 smithi100 ceph-mon[32599]: osdmap e559: 12 total, 12 up, 12 in 2024-07-26T17:45:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:45:18 smithi167 ceph-mon[37140]: pgmap v8194: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:45:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:45:18 smithi167 ceph-mon[37140]: osdmap e559: 12 total, 12 up, 12 in 2024-07-26T17:45:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:45:20 smithi138 ceph-mon[36954]: pgmap v8196: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:45:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:45:20 smithi100 ceph-mon[32599]: pgmap v8196: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:45:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:45:20 smithi167 ceph-mon[37140]: pgmap v8196: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:45:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:45:22 smithi138 ceph-mon[36954]: pgmap v8197: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:45:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:45:22 smithi100 ceph-mon[32599]: pgmap v8197: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:45:23.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:45:22 smithi167 ceph-mon[37140]: pgmap v8197: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:45:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:45:24 smithi138 ceph-mon[36954]: pgmap v8198: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:45:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:45:24 smithi100 ceph-mon[32599]: pgmap v8198: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:45:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:45:24 smithi167 ceph-mon[37140]: pgmap v8198: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:45:25.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:45:25.818+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:45:25.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:45:25.819+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:45:25.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:45:25.819+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:45:25.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:45:25.819+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:45:25.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:45:25.819+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:45:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:45:26 smithi138 ceph-mon[36954]: pgmap v8199: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:45:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:45:26 smithi100 ceph-mon[32599]: pgmap v8199: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:45:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:45:26 smithi167 ceph-mon[37140]: pgmap v8199: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:45:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:45:28 smithi138 ceph-mon[36954]: pgmap v8200: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:45:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:45:28 smithi100 ceph-mon[32599]: pgmap v8200: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:45:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:45:28 smithi167 ceph-mon[37140]: pgmap v8200: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:45:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:45:30 smithi138 ceph-mon[36954]: pgmap v8201: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:45:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:45:30 smithi100 ceph-mon[32599]: pgmap v8201: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:45:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:45:30 smithi167 ceph-mon[37140]: pgmap v8201: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:45:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:45:32 smithi138 ceph-mon[36954]: pgmap v8202: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:45:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:45:32 smithi100 ceph-mon[32599]: pgmap v8202: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:45:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:45:32 smithi167 ceph-mon[37140]: pgmap v8202: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:45:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:45:34 smithi138 ceph-mon[36954]: pgmap v8203: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:45:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:45:34 smithi100 ceph-mon[32599]: pgmap v8203: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:45:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:45:34 smithi167 ceph-mon[37140]: pgmap v8203: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:45:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:45:36 smithi138 ceph-mon[36954]: pgmap v8204: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:45:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:45:36 smithi100 ceph-mon[32599]: pgmap v8204: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:45:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:45:36 smithi167 ceph-mon[37140]: pgmap v8204: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:45:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:45:38 smithi138 ceph-mon[36954]: pgmap v8205: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:45:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:45:38 smithi100 ceph-mon[32599]: pgmap v8205: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:45:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:45:38 smithi167 ceph-mon[37140]: pgmap v8205: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:45:40.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:45:40.834+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:45:40.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:45:40.835+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:45:40.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:45:40.835+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:45:40.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:45:40.835+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:45:40.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:45:40.835+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:45:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:45:40 smithi138 ceph-mon[36954]: pgmap v8206: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:45:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:45:40 smithi100 ceph-mon[32599]: pgmap v8206: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:45:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:45:40 smithi167 ceph-mon[37140]: pgmap v8206: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:45:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:45:42 smithi138 ceph-mon[36954]: pgmap v8207: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:45:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:45:42 smithi100 ceph-mon[32599]: pgmap v8207: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:45:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:45:42 smithi167 ceph-mon[37140]: pgmap v8207: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:45:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:45:44 smithi138 ceph-mon[36954]: pgmap v8208: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:45:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:45:44 smithi100 ceph-mon[32599]: pgmap v8208: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:45:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:45:44 smithi167 ceph-mon[37140]: pgmap v8208: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:45:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:45:46 smithi138 ceph-mon[36954]: pgmap v8209: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:45:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:45:46 smithi100 ceph-mon[32599]: pgmap v8209: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:45:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:45:46 smithi167 ceph-mon[37140]: pgmap v8209: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:45:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:45:48 smithi138 ceph-mon[36954]: pgmap v8210: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:45:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:45:48 smithi100 ceph-mon[32599]: pgmap v8210: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:45:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:45:48 smithi167 ceph-mon[37140]: pgmap v8210: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:45:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:45:50 smithi138 ceph-mon[36954]: pgmap v8211: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:45:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:45:50 smithi100 ceph-mon[32599]: pgmap v8211: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:45:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:45:50 smithi167 ceph-mon[37140]: pgmap v8211: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:45:51.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:45:51 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:45:51.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:45:51 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:45:51.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:45:51 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:45:51.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:45:51 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:45:52.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:45:51 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:45:52.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:45:51 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:45:52.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:45:51 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:45:52.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:45:51 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:45:52.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:45:51 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:45:52.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:45:51 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:45:52.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:45:51 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:45:52.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:45:51 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:45:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:45:52 smithi100 ceph-mon[32599]: pgmap v8212: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:45:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:45:52 smithi167 ceph-mon[37140]: pgmap v8212: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:45:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:45:52 smithi138 ceph-mon[36954]: pgmap v8212: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:45:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:45:54 smithi100 ceph-mon[32599]: pgmap v8213: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:45:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:45:54 smithi167 ceph-mon[37140]: pgmap v8213: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:45:55.194 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:45:54 smithi138 ceph-mon[36954]: pgmap v8213: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:45:55.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:45:55.847+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:45:55.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:45:55.847+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:45:55.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:45:55.847+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:45:55.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:45:55.847+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:45:55.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:45:55.847+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:45:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:45:56 smithi100 ceph-mon[32599]: pgmap v8214: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:45:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:45:56 smithi167 ceph-mon[37140]: pgmap v8214: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:45:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:45:56 smithi138 ceph-mon[36954]: pgmap v8214: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:45:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:45:58 smithi100 ceph-mon[32599]: pgmap v8215: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:45:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:45:58 smithi167 ceph-mon[37140]: pgmap v8215: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:45:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:45:58 smithi138 ceph-mon[36954]: pgmap v8215: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:46:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:46:00 smithi100 ceph-mon[32599]: pgmap v8216: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:46:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:46:00 smithi167 ceph-mon[37140]: pgmap v8216: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:46:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:46:00 smithi138 ceph-mon[36954]: pgmap v8216: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:46:02.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:46:02 smithi100 ceph-mon[32599]: pgmap v8217: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:46:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:46:02 smithi167 ceph-mon[37140]: pgmap v8217: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:46:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:46:02 smithi138 ceph-mon[36954]: pgmap v8217: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:46:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:46:04 smithi100 ceph-mon[32599]: pgmap v8218: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:46:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:46:04 smithi167 ceph-mon[37140]: pgmap v8218: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:46:05.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:46:04 smithi138 ceph-mon[36954]: pgmap v8218: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:46:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:46:06 smithi100 ceph-mon[32599]: pgmap v8219: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:46:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:46:06 smithi167 ceph-mon[37140]: pgmap v8219: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:46:07.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:46:06 smithi138 ceph-mon[36954]: pgmap v8219: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:46:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:46:08 smithi100 ceph-mon[32599]: pgmap v8220: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:46:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:46:08 smithi100 ceph-mon[32599]: osdmap e560: 12 total, 12 up, 12 in 2024-07-26T17:46:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:46:08 smithi167 ceph-mon[37140]: pgmap v8220: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:46:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:46:08 smithi167 ceph-mon[37140]: osdmap e560: 12 total, 12 up, 12 in 2024-07-26T17:46:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:46:08 smithi138 ceph-mon[36954]: pgmap v8220: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:46:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:46:08 smithi138 ceph-mon[36954]: osdmap e560: 12 total, 12 up, 12 in 2024-07-26T17:46:10.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:46:10.863+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:46:10.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:46:10.863+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:46:10.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:46:10.863+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:46:10.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:46:10.864+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:46:10.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:46:10.864+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:46:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:46:10 smithi100 ceph-mon[32599]: pgmap v8222: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:46:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:46:10 smithi167 ceph-mon[37140]: pgmap v8222: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:46:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:46:10 smithi138 ceph-mon[36954]: pgmap v8222: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:46:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:46:12 smithi100 ceph-mon[32599]: pgmap v8223: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:46:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:46:12 smithi167 ceph-mon[37140]: pgmap v8223: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:46:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:46:12 smithi138 ceph-mon[36954]: pgmap v8223: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:46:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:46:14 smithi100 ceph-mon[32599]: pgmap v8224: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:46:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:46:14 smithi167 ceph-mon[37140]: pgmap v8224: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:46:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:46:14 smithi138 ceph-mon[36954]: pgmap v8224: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:46:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:46:16 smithi100 ceph-mon[32599]: pgmap v8225: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:46:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:46:16 smithi167 ceph-mon[37140]: pgmap v8225: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:46:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:46:16 smithi138 ceph-mon[36954]: pgmap v8225: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:46:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:46:18 smithi138 ceph-mon[36954]: pgmap v8226: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:46:18.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:46:18 smithi138 ceph-mon[36954]: osdmap e561: 12 total, 12 up, 12 in 2024-07-26T17:46:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:46:18 smithi100 ceph-mon[32599]: pgmap v8226: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:46:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:46:18 smithi100 ceph-mon[32599]: osdmap e561: 12 total, 12 up, 12 in 2024-07-26T17:46:19.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:46:18 smithi167 ceph-mon[37140]: pgmap v8226: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:46:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:46:18 smithi167 ceph-mon[37140]: osdmap e561: 12 total, 12 up, 12 in 2024-07-26T17:46:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:46:20 smithi138 ceph-mon[36954]: pgmap v8228: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:46:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:46:20 smithi100 ceph-mon[32599]: pgmap v8228: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:46:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:46:20 smithi167 ceph-mon[37140]: pgmap v8228: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:46:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:46:22 smithi138 ceph-mon[36954]: pgmap v8229: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:46:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:46:22 smithi100 ceph-mon[32599]: pgmap v8229: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:46:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:46:22 smithi167 ceph-mon[37140]: pgmap v8229: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:46:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:46:24 smithi138 ceph-mon[36954]: pgmap v8230: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:46:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:46:24 smithi100 ceph-mon[32599]: pgmap v8230: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:46:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:46:24 smithi167 ceph-mon[37140]: pgmap v8230: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:46:25.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:46:25.879+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:46:25.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:46:25.880+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:46:25.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:46:25.880+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:46:25.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:46:25.880+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:46:25.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:46:25.880+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:46:26.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:46:26 smithi138 ceph-mon[36954]: pgmap v8231: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:46:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:46:26 smithi100 ceph-mon[32599]: pgmap v8231: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:46:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:46:26 smithi167 ceph-mon[37140]: pgmap v8231: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:46:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:46:28 smithi138 ceph-mon[36954]: pgmap v8232: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:46:28.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:46:28 smithi100 ceph-mon[32599]: pgmap v8232: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:46:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:46:28 smithi167 ceph-mon[37140]: pgmap v8232: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:46:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:46:30 smithi138 ceph-mon[36954]: pgmap v8233: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:46:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:46:30 smithi100 ceph-mon[32599]: pgmap v8233: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:46:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:46:30 smithi167 ceph-mon[37140]: pgmap v8233: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:46:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:46:32 smithi138 ceph-mon[36954]: pgmap v8234: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:46:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:46:32 smithi100 ceph-mon[32599]: pgmap v8234: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:46:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:46:32 smithi167 ceph-mon[37140]: pgmap v8234: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:46:34.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:46:34 smithi138 ceph-mon[36954]: pgmap v8235: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:46:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:46:34 smithi100 ceph-mon[32599]: pgmap v8235: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:46:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:46:34 smithi167 ceph-mon[37140]: pgmap v8235: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:46:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:46:36 smithi138 ceph-mon[36954]: pgmap v8236: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:46:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:46:36 smithi100 ceph-mon[32599]: pgmap v8236: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:46:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:46:36 smithi167 ceph-mon[37140]: pgmap v8236: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:46:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:46:38 smithi138 ceph-mon[36954]: pgmap v8237: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:46:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:46:38 smithi100 ceph-mon[32599]: pgmap v8237: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:46:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:46:38 smithi167 ceph-mon[37140]: pgmap v8237: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:46:40.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:46:40.895+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:46:40.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:46:40.896+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:46:40.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:46:40.896+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:46:40.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:46:40.896+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:46:40.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:46:40.896+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:46:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:46:40 smithi138 ceph-mon[36954]: pgmap v8238: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:46:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:46:40 smithi100 ceph-mon[32599]: pgmap v8238: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:46:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:46:40 smithi167 ceph-mon[37140]: pgmap v8238: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:46:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:46:42 smithi100 ceph-mon[32599]: pgmap v8239: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:46:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:46:42 smithi167 ceph-mon[37140]: pgmap v8239: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:46:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:46:42 smithi138 ceph-mon[36954]: pgmap v8239: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:46:44.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:46:44 smithi100 ceph-mon[32599]: pgmap v8240: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:46:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:46:44 smithi167 ceph-mon[37140]: pgmap v8240: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:46:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:46:44 smithi138 ceph-mon[36954]: pgmap v8240: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:46:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:46:46 smithi100 ceph-mon[32599]: pgmap v8241: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:46:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:46:46 smithi167 ceph-mon[37140]: pgmap v8241: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:46:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:46:46 smithi138 ceph-mon[36954]: pgmap v8241: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:46:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:46:48 smithi100 ceph-mon[32599]: pgmap v8242: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:46:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:46:48 smithi167 ceph-mon[37140]: pgmap v8242: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:46:49.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:46:48 smithi138 ceph-mon[36954]: pgmap v8242: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:46:50.993 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:46:50 smithi138 ceph-mon[36954]: pgmap v8243: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:46:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:46:50 smithi100 ceph-mon[32599]: pgmap v8243: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:46:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:46:50 smithi167 ceph-mon[37140]: pgmap v8243: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:46:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:46:52 smithi100 ceph-mon[32599]: pgmap v8244: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:46:52.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:46:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:46:52.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:46:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:46:52.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:46:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:46:52.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:46:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:46:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:46:52 smithi167 ceph-mon[37140]: pgmap v8244: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:46:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:46:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:46:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:46:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:46:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:46:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:46:53.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:46:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:46:53.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:46:52 smithi138 ceph-mon[36954]: pgmap v8244: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:46:53.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:46:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:46:53.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:46:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:46:53.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:46:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:46:53.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:46:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:46:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:46:54 smithi100 ceph-mon[32599]: pgmap v8245: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:46:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:46:54 smithi167 ceph-mon[37140]: pgmap v8245: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:46:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:46:54 smithi138 ceph-mon[36954]: pgmap v8245: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:46:55.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:46:55.911+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:46:55.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:46:55.912+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:46:55.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:46:55.912+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:46:55.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:46:55.912+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:46:55.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:46:55.912+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:46:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:46:56 smithi100 ceph-mon[32599]: pgmap v8246: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:46:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:46:56 smithi167 ceph-mon[37140]: pgmap v8246: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:46:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:46:56 smithi138 ceph-mon[36954]: pgmap v8246: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:46:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:46:58 smithi100 ceph-mon[32599]: pgmap v8247: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:46:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:46:58 smithi167 ceph-mon[37140]: pgmap v8247: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:46:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:46:58 smithi138 ceph-mon[36954]: pgmap v8247: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:47:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:47:00 smithi100 ceph-mon[32599]: pgmap v8248: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:47:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:47:00 smithi167 ceph-mon[37140]: pgmap v8248: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:47:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:47:00 smithi138 ceph-mon[36954]: pgmap v8248: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:47:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:47:02 smithi100 ceph-mon[32599]: pgmap v8249: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:47:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:47:02 smithi167 ceph-mon[37140]: pgmap v8249: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:47:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:47:02 smithi138 ceph-mon[36954]: pgmap v8249: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:47:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:47:04 smithi100 ceph-mon[32599]: pgmap v8250: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:47:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:47:04 smithi167 ceph-mon[37140]: pgmap v8250: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:47:05.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:47:04 smithi138 ceph-mon[36954]: pgmap v8250: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:47:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:47:06 smithi100 ceph-mon[32599]: pgmap v8251: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:47:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:47:06 smithi167 ceph-mon[37140]: pgmap v8251: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:47:07.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:47:06 smithi138 ceph-mon[36954]: pgmap v8251: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:47:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:47:08 smithi100 ceph-mon[32599]: pgmap v8252: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:47:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:47:08 smithi100 ceph-mon[32599]: osdmap e562: 12 total, 12 up, 12 in 2024-07-26T17:47:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:47:08 smithi167 ceph-mon[37140]: pgmap v8252: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:47:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:47:08 smithi167 ceph-mon[37140]: osdmap e562: 12 total, 12 up, 12 in 2024-07-26T17:47:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:47:08 smithi138 ceph-mon[36954]: pgmap v8252: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:47:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:47:08 smithi138 ceph-mon[36954]: osdmap e562: 12 total, 12 up, 12 in 2024-07-26T17:47:10.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:47:10.915+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:47:10.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:47:10.916+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:47:10.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:47:10.916+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:47:10.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:47:10.916+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:47:10.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:47:10.916+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:47:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:47:10 smithi167 ceph-mon[37140]: pgmap v8254: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:47:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:47:10 smithi138 ceph-mon[36954]: pgmap v8254: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:47:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:47:10 smithi100 ceph-mon[32599]: pgmap v8254: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:47:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:47:12 smithi167 ceph-mon[37140]: pgmap v8255: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:47:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:47:12 smithi138 ceph-mon[36954]: pgmap v8255: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:47:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:47:12 smithi100 ceph-mon[32599]: pgmap v8255: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:47:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:47:14 smithi167 ceph-mon[37140]: pgmap v8256: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:47:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:47:14 smithi138 ceph-mon[36954]: pgmap v8256: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:47:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:47:14 smithi100 ceph-mon[32599]: pgmap v8256: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:47:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:47:16 smithi167 ceph-mon[37140]: pgmap v8257: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:47:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:47:16 smithi138 ceph-mon[36954]: pgmap v8257: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:47:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:47:16 smithi100 ceph-mon[32599]: pgmap v8257: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:47:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:47:18 smithi138 ceph-mon[36954]: pgmap v8258: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:47:18.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:47:18 smithi138 ceph-mon[36954]: osdmap e563: 12 total, 12 up, 12 in 2024-07-26T17:47:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:47:18 smithi100 ceph-mon[32599]: pgmap v8258: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:47:18.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:47:18 smithi100 ceph-mon[32599]: osdmap e563: 12 total, 12 up, 12 in 2024-07-26T17:47:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:47:18 smithi167 ceph-mon[37140]: pgmap v8258: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:47:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:47:18 smithi167 ceph-mon[37140]: osdmap e563: 12 total, 12 up, 12 in 2024-07-26T17:47:20.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:47:20.049+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c004740 con 0x7fe5a81087a0 2024-07-26T17:47:20.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:47:20.049+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T17:47:20.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:47:20.050+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 113 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe57c004740 con 0x7fe5a81087a0 2024-07-26T17:47:20.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:47:20.050+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 113 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T17:47:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:47:20 smithi138 ceph-mon[36954]: pgmap v8260: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:47:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:47:20 smithi100 ceph-mon[32599]: pgmap v8260: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:47:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:47:20 smithi167 ceph-mon[37140]: pgmap v8260: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:47:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:47:22 smithi138 ceph-mon[36954]: pgmap v8261: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:47:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:47:22 smithi100 ceph-mon[32599]: pgmap v8261: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:47:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:47:22 smithi167 ceph-mon[37140]: pgmap v8261: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:47:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:47:24 smithi138 ceph-mon[36954]: pgmap v8262: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:47:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:47:24 smithi100 ceph-mon[32599]: pgmap v8262: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:47:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:47:24 smithi167 ceph-mon[37140]: pgmap v8262: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:47:25.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:47:25.927+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:47:25.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:47:25.929+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:47:25.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:47:25.929+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:47:25.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:47:25.929+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:47:25.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:47:25.929+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:47:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:47:26 smithi138 ceph-mon[36954]: pgmap v8263: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:47:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:47:26 smithi100 ceph-mon[32599]: pgmap v8263: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:47:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:47:26 smithi167 ceph-mon[37140]: pgmap v8263: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:47:28.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:47:28 smithi138 ceph-mon[36954]: pgmap v8264: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:47:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:47:28 smithi100 ceph-mon[32599]: pgmap v8264: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:47:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:47:28 smithi167 ceph-mon[37140]: pgmap v8264: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:47:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:47:30 smithi138 ceph-mon[36954]: pgmap v8265: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:47:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:47:30 smithi100 ceph-mon[32599]: pgmap v8265: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:47:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:47:30 smithi167 ceph-mon[37140]: pgmap v8265: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:47:32.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:47:32 smithi138 ceph-mon[36954]: pgmap v8266: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:47:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:47:32 smithi100 ceph-mon[32599]: pgmap v8266: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:47:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:47:32 smithi167 ceph-mon[37140]: pgmap v8266: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:47:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:47:34 smithi100 ceph-mon[32599]: pgmap v8267: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:47:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:47:34 smithi167 ceph-mon[37140]: pgmap v8267: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:47:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:47:34 smithi138 ceph-mon[36954]: pgmap v8267: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:47:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:47:36 smithi100 ceph-mon[32599]: pgmap v8268: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:47:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:47:36 smithi167 ceph-mon[37140]: pgmap v8268: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:47:37.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:47:36 smithi138 ceph-mon[36954]: pgmap v8268: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:47:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:47:38 smithi100 ceph-mon[32599]: pgmap v8269: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:47:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:47:38 smithi167 ceph-mon[37140]: pgmap v8269: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:47:39.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:47:38 smithi138 ceph-mon[36954]: pgmap v8269: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:47:40.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:47:40.945+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:47:40.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:47:40.945+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:47:40.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:47:40.945+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:47:40.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:47:40.945+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:47:40.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:47:40.945+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:47:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:47:40 smithi100 ceph-mon[32599]: pgmap v8270: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:47:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:47:40 smithi167 ceph-mon[37140]: pgmap v8270: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:47:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:47:40 smithi138 ceph-mon[36954]: pgmap v8270: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:47:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:47:42 smithi100 ceph-mon[32599]: pgmap v8271: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:47:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:47:42 smithi167 ceph-mon[37140]: pgmap v8271: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:47:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:47:42 smithi138 ceph-mon[36954]: pgmap v8271: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:47:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:47:44 smithi100 ceph-mon[32599]: pgmap v8272: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T17:47:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:47:44 smithi167 ceph-mon[37140]: pgmap v8272: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T17:47:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:47:44 smithi138 ceph-mon[36954]: pgmap v8272: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T17:47:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:47:46 smithi100 ceph-mon[32599]: pgmap v8273: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:47:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:47:46 smithi167 ceph-mon[37140]: pgmap v8273: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:47:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:47:46 smithi138 ceph-mon[36954]: pgmap v8273: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:47:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:47:48 smithi100 ceph-mon[32599]: pgmap v8274: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T17:47:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:47:48 smithi167 ceph-mon[37140]: pgmap v8274: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T17:47:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:47:48 smithi138 ceph-mon[36954]: pgmap v8274: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T17:47:50.993 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:47:50 smithi138 ceph-mon[36954]: pgmap v8275: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:47:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:47:50 smithi100 ceph-mon[32599]: pgmap v8275: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:47:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:47:50 smithi167 ceph-mon[37140]: pgmap v8275: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:47:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:47:52 smithi100 ceph-mon[32599]: pgmap v8276: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:47:52.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:47:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:47:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:47:52 smithi167 ceph-mon[37140]: pgmap v8276: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:47:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:47:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:47:53.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:47:52 smithi138 ceph-mon[36954]: pgmap v8276: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:47:53.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:47:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:47:53.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:47:53 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:47:53.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:47:53 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:47:53.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:47:53 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:47:54.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:47:53 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:47:54.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:47:53 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:47:54.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:47:53 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:47:54.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:47:53 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:47:54.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:47:53 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:47:54.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:47:53 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:47:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:47:54 smithi100 ceph-mon[32599]: pgmap v8277: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T17:47:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:47:54 smithi167 ceph-mon[37140]: pgmap v8277: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T17:47:55.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:47:54 smithi138 ceph-mon[36954]: pgmap v8277: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T17:47:55.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:47:55.961+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:47:55.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:47:55.961+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:47:55.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:47:55.961+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:47:55.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:47:55.961+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:47:55.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:47:55.961+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:47:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:47:56 smithi100 ceph-mon[32599]: pgmap v8278: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:47:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:47:56 smithi167 ceph-mon[37140]: pgmap v8278: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:47:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:47:56 smithi138 ceph-mon[36954]: pgmap v8278: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:47:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:47:58 smithi100 ceph-mon[32599]: pgmap v8279: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:47:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:47:58 smithi167 ceph-mon[37140]: pgmap v8279: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:47:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:47:58 smithi138 ceph-mon[36954]: pgmap v8279: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:48:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:48:00 smithi100 ceph-mon[32599]: pgmap v8280: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:48:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:48:00 smithi167 ceph-mon[37140]: pgmap v8280: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:48:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:48:00 smithi138 ceph-mon[36954]: pgmap v8280: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:48:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:48:02 smithi167 ceph-mon[37140]: pgmap v8281: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:48:03.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:48:03.043+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 msgr2=0x7fe5940c0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-07-26T17:48:03.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:48:03.043+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 msgr2=0x7fe5940c0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-07-26T17:48:03.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:48:03.043+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fe590005d00 tx=0x7fe590074040 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:48:03.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:48:03.043+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fe590005d00 tx=0x7fe590074040 comp rx=0 tx=0).stop 2024-07-26T17:48:03.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:48:03.043+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 msgr2=0x7fe5940c0480 unknown :-1 s=STATE_CLOSED l=1).tick see no progress in more than 10000000 us during connecting to v2:172.21.15.100:6800/3937009534, fault. 2024-07-26T17:48:03.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:48:03.043+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 msgr2=0x7fe5940c0480 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-07-26T17:48:03.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:48:03.043+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T17:48:03.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:48:03.043+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 0x7fe5940b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T17:48:03.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:48:02 smithi100 ceph-mon[32599]: pgmap v8281: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:48:03.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:48:03.043+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 0x7fe5940b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:48:03.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:48:03.044+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 0x7fe5940b5b30 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fe590004ee0 tx=0x7fe59005a040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T17:48:03.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:48:02 smithi138 ceph-mon[36954]: pgmap v8281: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:48:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:48:04 smithi167 ceph-mon[37140]: pgmap v8282: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:48:05.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:48:05.056+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c003690 con 0x7fe5a81087a0 2024-07-26T17:48:05.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:48:05.056+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T17:48:05.058 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:48:04 smithi100 ceph-mon[32599]: pgmap v8282: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:48:05.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:48:05.056+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 114 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c003690 con 0x7fe5a81087a0 2024-07-26T17:48:05.059 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:48:05.056+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 114 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T17:48:05.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:48:04 smithi138 ceph-mon[36954]: pgmap v8282: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:48:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:48:06 smithi167 ceph-mon[37140]: pgmap v8283: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:48:07.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:48:06 smithi138 ceph-mon[36954]: pgmap v8283: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:48:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:48:06 smithi100 ceph-mon[32599]: pgmap v8283: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:48:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:48:08 smithi167 ceph-mon[37140]: pgmap v8284: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:48:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:48:08 smithi167 ceph-mon[37140]: osdmap e564: 12 total, 12 up, 12 in 2024-07-26T17:48:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:48:08 smithi138 ceph-mon[36954]: pgmap v8284: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:48:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:48:08 smithi138 ceph-mon[36954]: osdmap e564: 12 total, 12 up, 12 in 2024-07-26T17:48:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:48:08 smithi100 ceph-mon[32599]: pgmap v8284: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:48:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:48:08 smithi100 ceph-mon[32599]: osdmap e564: 12 total, 12 up, 12 in 2024-07-26T17:48:10.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:48:10.969+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:48:10.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:48:10.970+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:48:10.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:48:10.970+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:48:10.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:48:10.970+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:48:10.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:48:10.970+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:48:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:48:10 smithi167 ceph-mon[37140]: pgmap v8286: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:48:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:48:10 smithi138 ceph-mon[36954]: pgmap v8286: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:48:11.339 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:48:10 smithi100 ceph-mon[32599]: pgmap v8286: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:48:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:48:12 smithi167 ceph-mon[37140]: pgmap v8287: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:48:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:48:12 smithi138 ceph-mon[36954]: pgmap v8287: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:48:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:48:12 smithi100 ceph-mon[32599]: pgmap v8287: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:48:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:48:14 smithi167 ceph-mon[37140]: pgmap v8288: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:48:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:48:14 smithi138 ceph-mon[36954]: pgmap v8288: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:48:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:48:14 smithi100 ceph-mon[32599]: pgmap v8288: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:48:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:48:16 smithi138 ceph-mon[36954]: pgmap v8289: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:48:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:48:16 smithi100 ceph-mon[32599]: pgmap v8289: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:48:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:48:16 smithi167 ceph-mon[37140]: pgmap v8289: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:48:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:48:18 smithi138 ceph-mon[36954]: pgmap v8290: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:48:18.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:48:18 smithi138 ceph-mon[36954]: osdmap e565: 12 total, 12 up, 12 in 2024-07-26T17:48:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:48:18 smithi100 ceph-mon[32599]: pgmap v8290: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:48:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:48:18 smithi100 ceph-mon[32599]: osdmap e565: 12 total, 12 up, 12 in 2024-07-26T17:48:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:48:18 smithi167 ceph-mon[37140]: pgmap v8290: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:48:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:48:18 smithi167 ceph-mon[37140]: osdmap e565: 12 total, 12 up, 12 in 2024-07-26T17:48:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:48:20 smithi138 ceph-mon[36954]: pgmap v8292: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:48:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:48:20 smithi100 ceph-mon[32599]: pgmap v8292: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:48:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:48:20 smithi167 ceph-mon[37140]: pgmap v8292: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:48:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:48:22 smithi138 ceph-mon[36954]: pgmap v8293: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:48:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:48:22 smithi100 ceph-mon[32599]: pgmap v8293: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:48:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:48:22 smithi167 ceph-mon[37140]: pgmap v8293: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:48:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:48:24 smithi100 ceph-mon[32599]: pgmap v8294: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:48:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:48:24 smithi167 ceph-mon[37140]: pgmap v8294: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:48:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:48:24 smithi138 ceph-mon[36954]: pgmap v8294: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:48:25.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:48:25.986+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:48:25.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:48:25.986+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:48:25.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:48:25.986+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:48:25.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:48:25.986+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:48:25.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:48:25.986+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:48:26.997 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:48:26 smithi100 ceph-mon[32599]: pgmap v8295: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:48:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:48:26 smithi167 ceph-mon[37140]: pgmap v8295: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:48:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:48:26 smithi138 ceph-mon[36954]: pgmap v8295: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:48:28.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:48:28 smithi100 ceph-mon[32599]: pgmap v8296: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:48:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:48:28 smithi167 ceph-mon[37140]: pgmap v8296: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:48:29.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:48:28 smithi138 ceph-mon[36954]: pgmap v8296: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:48:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:48:30 smithi100 ceph-mon[32599]: pgmap v8297: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:48:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:48:30 smithi167 ceph-mon[37140]: pgmap v8297: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:48:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:48:30 smithi138 ceph-mon[36954]: pgmap v8297: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:48:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:48:32 smithi100 ceph-mon[32599]: pgmap v8298: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:48:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:48:32 smithi167 ceph-mon[37140]: pgmap v8298: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:48:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:48:32 smithi138 ceph-mon[36954]: pgmap v8298: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:48:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:48:34 smithi100 ceph-mon[32599]: pgmap v8299: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T17:48:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:48:34 smithi167 ceph-mon[37140]: pgmap v8299: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T17:48:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:48:34 smithi138 ceph-mon[36954]: pgmap v8299: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T17:48:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:48:36 smithi100 ceph-mon[32599]: pgmap v8300: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:48:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:48:36 smithi167 ceph-mon[37140]: pgmap v8300: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:48:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:48:36 smithi138 ceph-mon[36954]: pgmap v8300: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:48:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:48:38 smithi100 ceph-mon[32599]: pgmap v8301: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T17:48:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:48:38 smithi167 ceph-mon[37140]: pgmap v8301: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T17:48:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:48:38 smithi138 ceph-mon[36954]: pgmap v8301: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T17:48:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:48:40 smithi100 ceph-mon[32599]: pgmap v8302: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:48:41.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:48:41.001+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:48:41.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:48:41.002+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:48:41.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:48:41.002+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:48:41.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:48:41.002+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:48:41.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:48:41.002+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:48:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:48:40 smithi167 ceph-mon[37140]: pgmap v8302: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:48:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:48:40 smithi138 ceph-mon[36954]: pgmap v8302: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:48:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:48:42 smithi100 ceph-mon[32599]: pgmap v8303: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:48:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:48:42 smithi167 ceph-mon[37140]: pgmap v8303: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:48:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:48:42 smithi138 ceph-mon[36954]: pgmap v8303: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:48:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:48:44 smithi100 ceph-mon[32599]: pgmap v8304: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:48:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:48:44 smithi167 ceph-mon[37140]: pgmap v8304: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:48:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:48:44 smithi138 ceph-mon[36954]: pgmap v8304: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:48:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:48:46 smithi100 ceph-mon[32599]: pgmap v8305: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:48:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:48:46 smithi167 ceph-mon[37140]: pgmap v8305: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:48:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:48:46 smithi138 ceph-mon[36954]: pgmap v8305: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:48:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:48:48 smithi100 ceph-mon[32599]: pgmap v8306: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:48:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:48:48 smithi167 ceph-mon[37140]: pgmap v8306: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:48:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:48:48 smithi138 ceph-mon[36954]: pgmap v8306: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:48:50.993 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:48:50 smithi138 ceph-mon[36954]: pgmap v8307: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:48:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:48:50 smithi100 ceph-mon[32599]: pgmap v8307: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:48:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:48:50 smithi167 ceph-mon[37140]: pgmap v8307: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:48:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:48:52 smithi100 ceph-mon[32599]: pgmap v8308: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:48:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:48:52 smithi167 ceph-mon[37140]: pgmap v8308: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:48:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:48:52 smithi138 ceph-mon[36954]: pgmap v8308: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:48:53.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:48:53 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:48:53.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:48:53 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:48:53.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:48:53 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:48:53.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:48:53 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:48:54.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:48:53 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:48:54.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:48:53 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:48:54.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:48:53 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:48:54.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:48:53 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:48:54.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:48:53 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:48:54.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:48:53 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:48:54.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:48:53 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:48:54.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:48:53 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:48:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:48:54 smithi100 ceph-mon[32599]: pgmap v8309: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:48:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:48:54 smithi167 ceph-mon[37140]: pgmap v8309: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:48:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:48:54 smithi138 ceph-mon[36954]: pgmap v8309: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:48:56.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:48:56.015+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:48:56.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:48:56.016+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:48:56.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:48:56.016+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:48:56.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:48:56.016+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:48:56.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:48:56.016+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:48:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:48:56 smithi100 ceph-mon[32599]: pgmap v8310: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:48:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:48:56 smithi167 ceph-mon[37140]: pgmap v8310: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:48:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:48:56 smithi138 ceph-mon[36954]: pgmap v8310: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:48:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:48:58 smithi167 ceph-mon[37140]: pgmap v8311: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:48:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:48:58 smithi138 ceph-mon[36954]: pgmap v8311: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:48:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:48:58 smithi100 ceph-mon[32599]: pgmap v8311: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:49:01.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:00 smithi167 ceph-mon[37140]: pgmap v8312: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:49:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:00 smithi138 ceph-mon[36954]: pgmap v8312: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:49:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:00 smithi100 ceph-mon[32599]: pgmap v8312: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:49:03.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:02 smithi167 ceph-mon[37140]: pgmap v8313: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:49:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:02 smithi138 ceph-mon[36954]: pgmap v8313: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:49:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:02 smithi100 ceph-mon[32599]: pgmap v8313: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:49:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:04 smithi167 ceph-mon[37140]: pgmap v8314: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:49:05.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:04 smithi138 ceph-mon[36954]: pgmap v8314: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:49:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:04 smithi100 ceph-mon[32599]: pgmap v8314: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:49:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:06 smithi167 ceph-mon[37140]: pgmap v8315: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:49:07.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:06 smithi138 ceph-mon[36954]: pgmap v8315: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:49:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:06 smithi100 ceph-mon[32599]: pgmap v8315: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:49:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:08 smithi167 ceph-mon[37140]: pgmap v8316: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:49:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:08 smithi167 ceph-mon[37140]: osdmap e566: 12 total, 12 up, 12 in 2024-07-26T17:49:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:08 smithi138 ceph-mon[36954]: pgmap v8316: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:49:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:08 smithi138 ceph-mon[36954]: osdmap e566: 12 total, 12 up, 12 in 2024-07-26T17:49:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:08 smithi100 ceph-mon[32599]: pgmap v8316: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:49:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:08 smithi100 ceph-mon[32599]: osdmap e566: 12 total, 12 up, 12 in 2024-07-26T17:49:11.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:49:11.031+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:49:11.034 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:10 smithi100 ceph-mon[32599]: pgmap v8318: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:49:11.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:49:11.032+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:49:11.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:49:11.032+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:49:11.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:49:11.032+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:49:11.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:49:11.032+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:49:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:10 smithi138 ceph-mon[36954]: pgmap v8318: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:49:11.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:10 smithi167 ceph-mon[37140]: pgmap v8318: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:49:13.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:12 smithi138 ceph-mon[36954]: pgmap v8319: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:49:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:12 smithi100 ceph-mon[32599]: pgmap v8319: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:49:13.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:12 smithi167 ceph-mon[37140]: pgmap v8319: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:49:15.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:14 smithi138 ceph-mon[36954]: pgmap v8320: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:49:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:14 smithi100 ceph-mon[32599]: pgmap v8320: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:49:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:14 smithi167 ceph-mon[37140]: pgmap v8320: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:49:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:16 smithi138 ceph-mon[36954]: pgmap v8321: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:49:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:16 smithi100 ceph-mon[32599]: pgmap v8321: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:49:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:16 smithi167 ceph-mon[37140]: pgmap v8321: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:49:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:18 smithi100 ceph-mon[32599]: pgmap v8322: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:49:18.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:18 smithi100 ceph-mon[32599]: osdmap e567: 12 total, 12 up, 12 in 2024-07-26T17:49:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:18 smithi167 ceph-mon[37140]: pgmap v8322: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:49:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:18 smithi167 ceph-mon[37140]: osdmap e567: 12 total, 12 up, 12 in 2024-07-26T17:49:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:18 smithi138 ceph-mon[36954]: pgmap v8322: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:49:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:18 smithi138 ceph-mon[36954]: osdmap e567: 12 total, 12 up, 12 in 2024-07-26T17:49:20.993 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:20 smithi138 ceph-mon[36954]: pgmap v8324: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:49:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:20 smithi100 ceph-mon[32599]: pgmap v8324: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:49:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:20 smithi167 ceph-mon[37140]: pgmap v8324: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:49:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:22 smithi100 ceph-mon[32599]: pgmap v8325: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:49:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:22 smithi167 ceph-mon[37140]: pgmap v8325: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:49:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:22 smithi138 ceph-mon[36954]: pgmap v8325: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:49:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:24 smithi100 ceph-mon[32599]: pgmap v8326: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:49:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:24 smithi167 ceph-mon[37140]: pgmap v8326: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:49:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:24 smithi138 ceph-mon[36954]: pgmap v8326: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:49:26.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:49:26.048+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:49:26.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:49:26.048+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:49:26.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:49:26.048+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:49:26.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:49:26.048+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:49:26.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:49:26.048+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:49:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:26 smithi100 ceph-mon[32599]: pgmap v8327: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:49:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:26 smithi167 ceph-mon[37140]: pgmap v8327: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:49:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:26 smithi138 ceph-mon[36954]: pgmap v8327: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:49:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:28 smithi100 ceph-mon[32599]: pgmap v8328: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:49:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:28 smithi167 ceph-mon[37140]: pgmap v8328: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:49:29.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:28 smithi138 ceph-mon[36954]: pgmap v8328: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:49:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:30 smithi100 ceph-mon[32599]: pgmap v8329: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:49:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:30 smithi167 ceph-mon[37140]: pgmap v8329: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:49:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:30 smithi138 ceph-mon[36954]: pgmap v8329: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:49:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:32 smithi100 ceph-mon[32599]: pgmap v8330: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:49:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:32 smithi167 ceph-mon[37140]: pgmap v8330: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:49:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:32 smithi138 ceph-mon[36954]: pgmap v8330: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:49:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:34 smithi100 ceph-mon[32599]: pgmap v8331: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:49:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:34 smithi167 ceph-mon[37140]: pgmap v8331: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:49:35.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:34 smithi138 ceph-mon[36954]: pgmap v8331: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:49:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:36 smithi100 ceph-mon[32599]: pgmap v8332: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:49:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:36 smithi167 ceph-mon[37140]: pgmap v8332: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:49:37.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:36 smithi138 ceph-mon[36954]: pgmap v8332: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:49:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:38 smithi100 ceph-mon[32599]: pgmap v8333: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:49:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:38 smithi167 ceph-mon[37140]: pgmap v8333: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:49:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:38 smithi138 ceph-mon[36954]: pgmap v8333: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:49:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:40 smithi100 ceph-mon[32599]: pgmap v8334: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:49:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:40 smithi167 ceph-mon[37140]: pgmap v8334: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:49:41.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:49:41.064+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:49:41.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:49:41.064+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:49:41.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:49:41.064+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:49:41.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:49:41.064+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:49:41.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:49:41.064+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:49:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:40 smithi138 ceph-mon[36954]: pgmap v8334: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:49:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:42 smithi100 ceph-mon[32599]: pgmap v8335: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:49:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:42 smithi167 ceph-mon[37140]: pgmap v8335: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:49:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:42 smithi138 ceph-mon[36954]: pgmap v8335: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:49:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:44 smithi167 ceph-mon[37140]: pgmap v8336: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:49:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:44 smithi138 ceph-mon[36954]: pgmap v8336: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:49:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:44 smithi100 ceph-mon[32599]: pgmap v8336: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:49:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:46 smithi167 ceph-mon[37140]: pgmap v8337: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:49:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:46 smithi138 ceph-mon[36954]: pgmap v8337: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:49:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:46 smithi100 ceph-mon[32599]: pgmap v8337: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:49:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:48 smithi167 ceph-mon[37140]: pgmap v8338: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:49:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:48 smithi138 ceph-mon[36954]: pgmap v8338: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:49:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:48 smithi100 ceph-mon[32599]: pgmap v8338: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:49:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:50 smithi167 ceph-mon[37140]: pgmap v8339: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:49:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:50 smithi138 ceph-mon[36954]: pgmap v8339: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:49:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:50 smithi100 ceph-mon[32599]: pgmap v8339: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:49:53.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:52 smithi167 ceph-mon[37140]: pgmap v8340: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:49:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:52 smithi138 ceph-mon[36954]: pgmap v8340: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:49:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:52 smithi100 ceph-mon[32599]: pgmap v8340: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:49:54.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:53 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:49:54.145 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:53 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:49:54.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:53 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:49:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:54 smithi167 ceph-mon[37140]: pgmap v8341: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:49:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:49:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:49:55.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:49:55.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:49:55.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:49:55.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:49:55.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:49:55.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:49:55.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:49:55.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:49:55.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:49:55.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:49:55.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:54 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T17:49:55.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:54 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T17:49:55.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:54 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T17:49:55.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:54 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T17:49:55.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:54 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T17:49:55.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:54 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T17:49:55.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:49:55.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:49:55.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:49:55.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:54 smithi138 ceph-mon[36954]: pgmap v8341: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:49:55.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:49:55.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:49:55.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:49:55.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:49:55.193 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:49:55.193 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:49:55.193 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:49:55.193 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:49:55.193 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:49:55.194 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:49:55.194 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:49:55.194 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:49:55.194 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:54 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T17:49:55.194 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:54 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T17:49:55.194 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:54 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T17:49:55.194 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:54 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T17:49:55.195 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:54 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T17:49:55.195 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:54 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T17:49:55.195 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:49:55.195 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:49:55.195 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:49:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:54 smithi100 ceph-mon[32599]: pgmap v8341: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:49:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:49:55.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:49:55.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:49:55.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:49:55.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:49:55.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:49:55.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:49:55.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:49:55.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:49:55.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:49:55.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:49:55.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T17:49:55.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:54 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T17:49:55.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:54 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T17:49:55.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:54 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T17:49:55.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:54 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T17:49:55.247 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:54 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T17:49:55.247 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:54 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T17:49:55.247 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:49:55.247 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:49:55.247 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:49:56.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:49:56.080+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:49:56.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:49:56.080+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:49:56.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:49:56.080+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:49:56.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:49:56.080+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:49:56.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:49:56.081+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:49:57.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:56 smithi167 ceph-mon[37140]: pgmap v8342: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:49:57.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:56 smithi138 ceph-mon[36954]: pgmap v8342: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:49:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:56 smithi100 ceph-mon[32599]: pgmap v8342: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:49:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:49:58 smithi167 ceph-mon[37140]: pgmap v8343: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:49:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:49:58 smithi138 ceph-mon[36954]: pgmap v8343: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:49:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:49:58 smithi100 ceph-mon[32599]: pgmap v8343: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:50:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:50:00 smithi138 ceph-mon[36954]: pgmap v8344: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:50:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:50:00 smithi138 ceph-mon[36954]: overall HEALTH_OK 2024-07-26T17:50:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:50:00 smithi100 ceph-mon[32599]: pgmap v8344: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:50:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:50:00 smithi100 ceph-mon[32599]: overall HEALTH_OK 2024-07-26T17:50:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:50:00 smithi167 ceph-mon[37140]: pgmap v8344: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:50:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:50:00 smithi167 ceph-mon[37140]: overall HEALTH_OK 2024-07-26T17:50:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:50:02 smithi138 ceph-mon[36954]: pgmap v8345: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:50:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:50:02 smithi100 ceph-mon[32599]: pgmap v8345: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:50:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:50:02 smithi167 ceph-mon[37140]: pgmap v8345: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:50:05.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:50:04 smithi138 ceph-mon[36954]: pgmap v8346: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:50:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:50:04 smithi100 ceph-mon[32599]: pgmap v8346: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:50:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:50:04 smithi167 ceph-mon[37140]: pgmap v8346: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:50:07.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:50:06 smithi138 ceph-mon[36954]: pgmap v8347: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:50:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:50:06 smithi100 ceph-mon[32599]: pgmap v8347: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:50:07.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:50:06 smithi167 ceph-mon[37140]: pgmap v8347: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:50:09.194 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:50:08 smithi138 ceph-mon[36954]: pgmap v8348: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:50:09.194 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:50:08 smithi138 ceph-mon[36954]: osdmap e568: 12 total, 12 up, 12 in 2024-07-26T17:50:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:50:08 smithi100 ceph-mon[32599]: pgmap v8348: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:50:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:50:08 smithi100 ceph-mon[32599]: osdmap e568: 12 total, 12 up, 12 in 2024-07-26T17:50:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:50:08 smithi167 ceph-mon[37140]: pgmap v8348: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:50:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:50:08 smithi167 ceph-mon[37140]: osdmap e568: 12 total, 12 up, 12 in 2024-07-26T17:50:11.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:50:11.088+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:50:11.090 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:50:10 smithi100 ceph-mon[32599]: pgmap v8350: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:50:11.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:50:11.089+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:50:11.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:50:11.089+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:50:11.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:50:11.089+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:50:11.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:50:11.089+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:50:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:50:10 smithi138 ceph-mon[36954]: pgmap v8350: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:50:11.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:50:10 smithi167 ceph-mon[37140]: pgmap v8350: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:50:13.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:50:12 smithi138 ceph-mon[36954]: pgmap v8351: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:50:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:50:12 smithi100 ceph-mon[32599]: pgmap v8351: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:50:13.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:50:12 smithi167 ceph-mon[37140]: pgmap v8351: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:50:15.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:50:14 smithi138 ceph-mon[36954]: pgmap v8352: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:50:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:50:14 smithi100 ceph-mon[32599]: pgmap v8352: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:50:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:50:14 smithi167 ceph-mon[37140]: pgmap v8352: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:50:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:50:16 smithi138 ceph-mon[36954]: pgmap v8353: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:50:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:50:16 smithi100 ceph-mon[32599]: pgmap v8353: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:50:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:50:16 smithi167 ceph-mon[37140]: pgmap v8353: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:50:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:50:18 smithi100 ceph-mon[32599]: pgmap v8354: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:50:18.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:50:18 smithi100 ceph-mon[32599]: osdmap e569: 12 total, 12 up, 12 in 2024-07-26T17:50:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:50:18 smithi167 ceph-mon[37140]: pgmap v8354: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:50:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:50:18 smithi167 ceph-mon[37140]: osdmap e569: 12 total, 12 up, 12 in 2024-07-26T17:50:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:50:18 smithi138 ceph-mon[36954]: pgmap v8354: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:50:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:50:18 smithi138 ceph-mon[36954]: osdmap e569: 12 total, 12 up, 12 in 2024-07-26T17:50:20.993 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:50:20 smithi138 ceph-mon[36954]: pgmap v8356: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:50:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:50:20 smithi100 ceph-mon[32599]: pgmap v8356: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:50:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:50:20 smithi167 ceph-mon[37140]: pgmap v8356: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:50:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:50:22 smithi100 ceph-mon[32599]: pgmap v8357: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:50:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:50:22 smithi167 ceph-mon[37140]: pgmap v8357: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:50:23.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:50:22 smithi138 ceph-mon[36954]: pgmap v8357: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:50:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:50:24 smithi100 ceph-mon[32599]: pgmap v8358: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:50:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:50:24 smithi167 ceph-mon[37140]: pgmap v8358: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:50:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:50:24 smithi138 ceph-mon[36954]: pgmap v8358: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:50:26.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:50:26.104+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:50:26.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:50:26.105+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:50:26.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:50:26.105+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:50:26.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:50:26.105+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:50:26.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:50:26.105+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:50:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:50:26 smithi100 ceph-mon[32599]: pgmap v8359: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:50:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:50:26 smithi167 ceph-mon[37140]: pgmap v8359: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:50:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:50:26 smithi138 ceph-mon[36954]: pgmap v8359: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:50:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:50:28 smithi100 ceph-mon[32599]: pgmap v8360: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:50:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:50:28 smithi167 ceph-mon[37140]: pgmap v8360: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:50:29.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:50:28 smithi138 ceph-mon[36954]: pgmap v8360: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:50:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:50:30 smithi100 ceph-mon[32599]: pgmap v8361: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:50:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:50:30 smithi167 ceph-mon[37140]: pgmap v8361: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:50:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:50:30 smithi138 ceph-mon[36954]: pgmap v8361: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:50:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:50:32 smithi100 ceph-mon[32599]: pgmap v8362: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:50:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:50:32 smithi167 ceph-mon[37140]: pgmap v8362: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:50:33.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:50:32 smithi138 ceph-mon[36954]: pgmap v8362: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:50:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:50:34 smithi167 ceph-mon[37140]: pgmap v8363: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:50:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:50:34 smithi138 ceph-mon[36954]: pgmap v8363: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:50:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:50:34 smithi100 ceph-mon[32599]: pgmap v8363: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:50:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:50:36 smithi167 ceph-mon[37140]: pgmap v8364: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:50:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:50:36 smithi138 ceph-mon[36954]: pgmap v8364: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:50:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:50:36 smithi100 ceph-mon[32599]: pgmap v8364: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:50:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:50:38 smithi167 ceph-mon[37140]: pgmap v8365: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:50:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:50:38 smithi138 ceph-mon[36954]: pgmap v8365: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:50:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:50:38 smithi100 ceph-mon[32599]: pgmap v8365: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:50:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:50:40 smithi167 ceph-mon[37140]: pgmap v8366: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:50:41.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:50:41.111+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:50:41.113 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:50:40 smithi100 ceph-mon[32599]: pgmap v8366: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:50:41.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:50:41.112+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:50:41.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:50:41.112+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:50:41.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:50:41.112+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:50:41.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:50:41.112+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:50:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:50:40 smithi138 ceph-mon[36954]: pgmap v8366: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:50:43.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:50:42 smithi167 ceph-mon[37140]: pgmap v8367: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:50:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:50:42 smithi138 ceph-mon[36954]: pgmap v8367: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:50:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:50:42 smithi100 ceph-mon[32599]: pgmap v8367: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:50:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:50:44 smithi167 ceph-mon[37140]: pgmap v8368: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:50:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:50:44 smithi138 ceph-mon[36954]: pgmap v8368: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:50:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:50:44 smithi100 ceph-mon[32599]: pgmap v8368: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:50:47.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:50:46 smithi167 ceph-mon[37140]: pgmap v8369: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:50:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:50:46 smithi138 ceph-mon[36954]: pgmap v8369: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:50:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:50:46 smithi100 ceph-mon[32599]: pgmap v8369: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:50:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:50:48 smithi167 ceph-mon[37140]: pgmap v8370: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:50:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:50:48 smithi138 ceph-mon[36954]: pgmap v8370: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:50:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:50:48 smithi100 ceph-mon[32599]: pgmap v8370: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:50:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:50:50 smithi138 ceph-mon[36954]: pgmap v8371: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:50:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:50:50 smithi100 ceph-mon[32599]: pgmap v8371: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:50:51.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:50:50 smithi167 ceph-mon[37140]: pgmap v8371: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:50:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:50:52 smithi138 ceph-mon[36954]: pgmap v8372: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:50:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:50:52 smithi100 ceph-mon[32599]: pgmap v8372: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:50:53.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:50:52 smithi167 ceph-mon[37140]: pgmap v8372: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:50:55.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:50:54 smithi100 ceph-mon[32599]: pgmap v8373: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:50:55.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:50:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:50:55.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:50:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:50:55.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:50:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:50:55.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:50:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:50:55.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:50:54 smithi167 ceph-mon[37140]: pgmap v8373: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:50:55.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:50:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:50:55.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:50:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:50:55.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:50:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:50:55.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:50:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:50:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:50:54 smithi138 ceph-mon[36954]: pgmap v8373: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:50:55.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:50:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:50:55.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:50:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:50:55.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:50:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:50:55.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:50:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:50:56.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:50:56.116+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:50:56.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:50:56.116+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:50:56.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:50:56.116+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:50:56.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:50:56.116+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:50:56.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:50:56.116+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:50:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:50:56 smithi138 ceph-mon[36954]: pgmap v8374: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:50:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:50:56 smithi100 ceph-mon[32599]: pgmap v8374: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:50:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:50:56 smithi167 ceph-mon[37140]: pgmap v8374: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:50:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:50:58 smithi138 ceph-mon[36954]: pgmap v8375: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:50:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:50:58 smithi100 ceph-mon[32599]: pgmap v8375: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:50:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:50:58 smithi167 ceph-mon[37140]: pgmap v8375: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:51:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:51:00 smithi138 ceph-mon[36954]: pgmap v8376: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:51:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:51:00 smithi100 ceph-mon[32599]: pgmap v8376: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:51:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:51:00 smithi167 ceph-mon[37140]: pgmap v8376: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:51:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:51:02 smithi138 ceph-mon[36954]: pgmap v8377: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:51:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:51:02 smithi100 ceph-mon[32599]: pgmap v8377: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:51:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:51:02 smithi167 ceph-mon[37140]: pgmap v8377: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:51:05.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:51:05.082+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c003690 con 0x7fe5a81087a0 2024-07-26T17:51:05.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:51:05.082+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T17:51:05.084 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:51:04 smithi100 ceph-mon[32599]: pgmap v8378: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:51:05.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:51:05.082+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 115 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c003690 con 0x7fe5a81087a0 2024-07-26T17:51:05.084 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:51:05.082+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 115 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T17:51:05.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:51:04 smithi138 ceph-mon[36954]: pgmap v8378: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:51:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:51:04 smithi167 ceph-mon[37140]: pgmap v8378: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:51:07.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:51:06 smithi138 ceph-mon[36954]: pgmap v8379: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:51:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:51:06 smithi100 ceph-mon[32599]: pgmap v8379: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:51:07.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:51:06 smithi167 ceph-mon[37140]: pgmap v8379: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:51:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:51:08 smithi138 ceph-mon[36954]: pgmap v8380: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:51:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:51:08 smithi138 ceph-mon[36954]: osdmap e570: 12 total, 12 up, 12 in 2024-07-26T17:51:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:51:08 smithi100 ceph-mon[32599]: pgmap v8380: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:51:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:51:08 smithi100 ceph-mon[32599]: osdmap e570: 12 total, 12 up, 12 in 2024-07-26T17:51:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:51:08 smithi167 ceph-mon[37140]: pgmap v8380: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:51:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:51:08 smithi167 ceph-mon[37140]: osdmap e570: 12 total, 12 up, 12 in 2024-07-26T17:51:11.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:51:11.131+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:51:11.133 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:51:10 smithi100 ceph-mon[32599]: pgmap v8382: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:51:11.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:51:11.132+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:51:11.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:51:11.132+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:51:11.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:51:11.132+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:51:11.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:51:11.132+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:51:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:51:10 smithi138 ceph-mon[36954]: pgmap v8382: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:51:11.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:51:10 smithi167 ceph-mon[37140]: pgmap v8382: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:51:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:51:12 smithi138 ceph-mon[36954]: pgmap v8383: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:51:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:51:12 smithi100 ceph-mon[32599]: pgmap v8383: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:51:13.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:51:12 smithi167 ceph-mon[37140]: pgmap v8383: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:51:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:51:14 smithi138 ceph-mon[36954]: pgmap v8384: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:51:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:51:14 smithi100 ceph-mon[32599]: pgmap v8384: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:51:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:51:14 smithi167 ceph-mon[37140]: pgmap v8384: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:51:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:51:16 smithi138 ceph-mon[36954]: pgmap v8385: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:51:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:51:16 smithi100 ceph-mon[32599]: pgmap v8385: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:51:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:51:16 smithi167 ceph-mon[37140]: pgmap v8385: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:51:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:51:18 smithi100 ceph-mon[32599]: pgmap v8386: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:51:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:51:18 smithi100 ceph-mon[32599]: osdmap e571: 12 total, 12 up, 12 in 2024-07-26T17:51:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:51:18 smithi167 ceph-mon[37140]: pgmap v8386: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:51:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:51:18 smithi167 ceph-mon[37140]: osdmap e571: 12 total, 12 up, 12 in 2024-07-26T17:51:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:51:18 smithi138 ceph-mon[36954]: pgmap v8386: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:51:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:51:18 smithi138 ceph-mon[36954]: osdmap e571: 12 total, 12 up, 12 in 2024-07-26T17:51:20.993 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:51:20 smithi138 ceph-mon[36954]: pgmap v8388: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:51:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:51:20 smithi100 ceph-mon[32599]: pgmap v8388: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:51:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:51:20 smithi167 ceph-mon[37140]: pgmap v8388: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:51:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:51:22 smithi100 ceph-mon[32599]: pgmap v8389: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:51:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:51:22 smithi167 ceph-mon[37140]: pgmap v8389: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:51:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:51:22 smithi138 ceph-mon[36954]: pgmap v8389: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:51:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:51:24 smithi167 ceph-mon[37140]: pgmap v8390: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:51:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:51:24 smithi138 ceph-mon[36954]: pgmap v8390: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:51:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:51:24 smithi100 ceph-mon[32599]: pgmap v8390: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:51:26.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:51:26.147+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:51:26.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:51:26.148+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:51:26.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:51:26.148+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:51:26.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:51:26.148+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:51:26.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:51:26.148+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:51:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:51:26 smithi167 ceph-mon[37140]: pgmap v8391: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:51:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:51:26 smithi138 ceph-mon[36954]: pgmap v8391: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:51:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:51:26 smithi100 ceph-mon[32599]: pgmap v8391: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:51:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:51:28 smithi167 ceph-mon[37140]: pgmap v8392: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:51:29.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:51:28 smithi138 ceph-mon[36954]: pgmap v8392: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:51:29.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:51:28 smithi100 ceph-mon[32599]: pgmap v8392: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:51:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:51:30 smithi167 ceph-mon[37140]: pgmap v8393: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:51:31.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:51:30 smithi138 ceph-mon[36954]: pgmap v8393: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:51:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:51:30 smithi100 ceph-mon[32599]: pgmap v8393: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:51:33.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:51:32 smithi167 ceph-mon[37140]: pgmap v8394: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:51:33.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:51:32 smithi138 ceph-mon[36954]: pgmap v8394: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:51:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:51:32 smithi100 ceph-mon[32599]: pgmap v8394: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:51:35.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:51:34 smithi167 ceph-mon[37140]: pgmap v8395: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:51:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:51:34 smithi138 ceph-mon[36954]: pgmap v8395: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:51:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:51:34 smithi100 ceph-mon[32599]: pgmap v8395: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:51:37.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:51:36 smithi167 ceph-mon[37140]: pgmap v8396: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:51:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:51:36 smithi138 ceph-mon[36954]: pgmap v8396: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:51:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:51:36 smithi100 ceph-mon[32599]: pgmap v8396: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:51:39.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:51:38 smithi167 ceph-mon[37140]: pgmap v8397: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:51:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:51:38 smithi138 ceph-mon[36954]: pgmap v8397: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:51:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:51:38 smithi100 ceph-mon[32599]: pgmap v8397: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:51:41.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:51:40 smithi167 ceph-mon[37140]: pgmap v8398: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:51:41.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:51:41.163+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:51:41.166 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:51:40 smithi100 ceph-mon[32599]: pgmap v8398: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:51:41.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:51:41.164+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:51:41.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:51:41.164+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:51:41.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:51:41.164+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:51:41.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:51:41.164+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:51:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:51:40 smithi138 ceph-mon[36954]: pgmap v8398: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:51:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:51:42 smithi138 ceph-mon[36954]: pgmap v8399: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:51:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:51:42 smithi100 ceph-mon[32599]: pgmap v8399: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:51:43.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:51:42 smithi167 ceph-mon[37140]: pgmap v8399: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:51:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:51:44 smithi138 ceph-mon[36954]: pgmap v8400: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:51:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:51:44 smithi100 ceph-mon[32599]: pgmap v8400: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:51:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:51:44 smithi167 ceph-mon[37140]: pgmap v8400: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:51:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:51:46 smithi138 ceph-mon[36954]: pgmap v8401: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:51:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:51:46 smithi100 ceph-mon[32599]: pgmap v8401: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:51:47.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:51:46 smithi167 ceph-mon[37140]: pgmap v8401: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:51:49.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:51:48 smithi138 ceph-mon[36954]: pgmap v8402: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:51:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:51:48 smithi100 ceph-mon[32599]: pgmap v8402: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:51:49.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:51:48 smithi167 ceph-mon[37140]: pgmap v8402: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:51:51.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:51:50 smithi138 ceph-mon[36954]: pgmap v8403: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:51:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:51:50 smithi100 ceph-mon[32599]: pgmap v8403: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:51:51.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:51:50 smithi167 ceph-mon[37140]: pgmap v8403: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:51:53.194 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:51:52 smithi138 ceph-mon[36954]: pgmap v8404: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:51:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:51:52 smithi100 ceph-mon[32599]: pgmap v8404: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:51:53.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:51:52 smithi167 ceph-mon[37140]: pgmap v8404: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:51:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:51:54 smithi138 ceph-mon[36954]: pgmap v8405: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:51:55.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:51:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:51:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:51:54 smithi100 ceph-mon[32599]: pgmap v8405: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:51:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:51:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:51:55.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:51:54 smithi167 ceph-mon[37140]: pgmap v8405: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:51:55.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:51:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:51:56.145 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:51:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:51:56.146 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:51:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:51:56.146 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:51:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:51:56.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:51:56.173+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:51:56.175 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:51:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:51:56.175 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:51:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:51:56.175 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:51:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:51:56.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:51:56.173+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:51:56.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:51:56.173+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:51:56.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:51:56.173+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:51:56.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:51:56.173+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:51:56.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:51:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:51:56.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:51:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:51:56.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:51:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:51:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:51:56 smithi100 ceph-mon[32599]: pgmap v8406: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:51:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:51:56 smithi167 ceph-mon[37140]: pgmap v8406: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:51:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:51:56 smithi138 ceph-mon[36954]: pgmap v8406: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:51:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:51:58 smithi100 ceph-mon[32599]: pgmap v8407: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:51:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:51:58 smithi167 ceph-mon[37140]: pgmap v8407: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:51:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:51:58 smithi138 ceph-mon[36954]: pgmap v8407: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:52:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:00 smithi100 ceph-mon[32599]: pgmap v8408: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:52:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:00 smithi167 ceph-mon[37140]: pgmap v8408: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:52:01.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:00 smithi138 ceph-mon[36954]: pgmap v8408: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:52:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:02 smithi100 ceph-mon[32599]: pgmap v8409: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:52:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:02 smithi167 ceph-mon[37140]: pgmap v8409: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:52:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:02 smithi138 ceph-mon[36954]: pgmap v8409: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:52:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:04 smithi100 ceph-mon[32599]: pgmap v8410: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:52:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:04 smithi167 ceph-mon[37140]: pgmap v8410: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:52:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:04 smithi138 ceph-mon[36954]: pgmap v8410: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:52:06.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:06 smithi167 ceph-mon[37140]: pgmap v8411: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:52:06.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:06 smithi138 ceph-mon[36954]: pgmap v8411: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:52:06.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:06 smithi100 ceph-mon[32599]: pgmap v8411: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:52:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:08 smithi138 ceph-mon[36954]: osdmap e572: 12 total, 12 up, 12 in 2024-07-26T17:52:08.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:08 smithi138 ceph-mon[36954]: pgmap v8413: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:52:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:08 smithi100 ceph-mon[32599]: osdmap e572: 12 total, 12 up, 12 in 2024-07-26T17:52:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:08 smithi100 ceph-mon[32599]: pgmap v8413: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:52:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:08 smithi167 ceph-mon[37140]: osdmap e572: 12 total, 12 up, 12 in 2024-07-26T17:52:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:08 smithi167 ceph-mon[37140]: pgmap v8413: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:52:10.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:10 smithi167 ceph-mon[37140]: pgmap v8414: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:52:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:10 smithi138 ceph-mon[36954]: pgmap v8414: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:52:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:10 smithi100 ceph-mon[32599]: pgmap v8414: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:52:11.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:52:11.186+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:52:11.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:52:11.187+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:52:11.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:52:11.187+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:52:11.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:52:11.187+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:52:11.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:52:11.187+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:52:12.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:12 smithi167 ceph-mon[37140]: pgmap v8415: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:52:12.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:12 smithi138 ceph-mon[36954]: pgmap v8415: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:52:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:12 smithi100 ceph-mon[32599]: pgmap v8415: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:52:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:14 smithi167 ceph-mon[37140]: pgmap v8416: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:52:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:14 smithi138 ceph-mon[36954]: pgmap v8416: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:52:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:14 smithi100 ceph-mon[32599]: pgmap v8416: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:52:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:16 smithi167 ceph-mon[37140]: pgmap v8417: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:52:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:16 smithi138 ceph-mon[36954]: pgmap v8417: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:52:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:16 smithi100 ceph-mon[32599]: pgmap v8417: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:52:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:18 smithi167 ceph-mon[37140]: pgmap v8418: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 198 B/s wr, 6 op/s 2024-07-26T17:52:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:18 smithi167 ceph-mon[37140]: osdmap e573: 12 total, 12 up, 12 in 2024-07-26T17:52:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:18 smithi138 ceph-mon[36954]: pgmap v8418: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 198 B/s wr, 6 op/s 2024-07-26T17:52:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:18 smithi138 ceph-mon[36954]: osdmap e573: 12 total, 12 up, 12 in 2024-07-26T17:52:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:18 smithi100 ceph-mon[32599]: pgmap v8418: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 198 B/s wr, 6 op/s 2024-07-26T17:52:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:18 smithi100 ceph-mon[32599]: osdmap e573: 12 total, 12 up, 12 in 2024-07-26T17:52:21.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:20 smithi167 ceph-mon[37140]: pgmap v8420: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:52:21.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:20 smithi138 ceph-mon[36954]: pgmap v8420: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:52:21.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:20 smithi100 ceph-mon[32599]: pgmap v8420: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:52:23.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:22 smithi167 ceph-mon[37140]: pgmap v8421: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:52:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:22 smithi138 ceph-mon[36954]: pgmap v8421: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:52:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:22 smithi100 ceph-mon[32599]: pgmap v8421: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:52:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:24 smithi167 ceph-mon[37140]: pgmap v8422: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:52:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:24 smithi138 ceph-mon[36954]: pgmap v8422: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:52:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:24 smithi100 ceph-mon[32599]: pgmap v8422: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:52:26.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:52:26.199+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:52:26.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:52:26.200+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:52:26.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:52:26.200+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:52:26.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:52:26.200+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:52:26.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:52:26.200+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:52:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:26 smithi167 ceph-mon[37140]: pgmap v8423: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:52:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:26 smithi138 ceph-mon[36954]: pgmap v8423: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:52:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:26 smithi100 ceph-mon[32599]: pgmap v8423: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:52:29.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:28 smithi167 ceph-mon[37140]: pgmap v8424: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:52:29.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:28 smithi138 ceph-mon[36954]: pgmap v8424: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:52:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:28 smithi100 ceph-mon[32599]: pgmap v8424: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:52:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:30 smithi167 ceph-mon[37140]: pgmap v8425: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:52:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:30 smithi138 ceph-mon[36954]: pgmap v8425: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:52:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:30 smithi100 ceph-mon[32599]: pgmap v8425: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:52:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:32 smithi138 ceph-mon[36954]: pgmap v8426: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:52:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:32 smithi100 ceph-mon[32599]: pgmap v8426: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:52:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:32 smithi167 ceph-mon[37140]: pgmap v8426: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:52:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:34 smithi138 ceph-mon[36954]: pgmap v8427: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:52:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:34 smithi100 ceph-mon[32599]: pgmap v8427: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:52:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:34 smithi167 ceph-mon[37140]: pgmap v8427: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:52:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:36 smithi138 ceph-mon[36954]: pgmap v8428: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:52:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:36 smithi100 ceph-mon[32599]: pgmap v8428: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:52:37.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:36 smithi167 ceph-mon[37140]: pgmap v8428: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:52:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:38 smithi138 ceph-mon[36954]: pgmap v8429: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:52:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:38 smithi100 ceph-mon[32599]: pgmap v8429: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:52:39.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:38 smithi167 ceph-mon[37140]: pgmap v8429: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:52:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:40 smithi138 ceph-mon[36954]: pgmap v8430: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:52:41.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:52:41.210+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:52:41.211 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:40 smithi100 ceph-mon[32599]: pgmap v8430: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:52:41.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:52:41.210+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:52:41.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:52:41.210+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:52:41.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:52:41.210+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:52:41.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:52:41.210+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:52:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:40 smithi167 ceph-mon[37140]: pgmap v8430: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:52:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:42 smithi138 ceph-mon[36954]: pgmap v8431: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:52:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:42 smithi100 ceph-mon[32599]: pgmap v8431: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:52:43.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:42 smithi167 ceph-mon[37140]: pgmap v8431: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:52:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:44 smithi138 ceph-mon[36954]: pgmap v8432: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:52:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:44 smithi100 ceph-mon[32599]: pgmap v8432: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:52:45.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:44 smithi167 ceph-mon[37140]: pgmap v8432: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:52:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:46 smithi138 ceph-mon[36954]: pgmap v8433: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:52:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:46 smithi100 ceph-mon[32599]: pgmap v8433: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:52:47.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:46 smithi167 ceph-mon[37140]: pgmap v8433: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:52:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:48 smithi138 ceph-mon[36954]: pgmap v8434: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:52:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:48 smithi100 ceph-mon[32599]: pgmap v8434: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:52:49.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:48 smithi167 ceph-mon[37140]: pgmap v8434: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:52:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:50 smithi138 ceph-mon[36954]: pgmap v8435: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:52:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:50 smithi100 ceph-mon[32599]: pgmap v8435: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:52:51.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:50 smithi167 ceph-mon[37140]: pgmap v8435: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:52:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:52 smithi138 ceph-mon[36954]: pgmap v8436: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:52:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:52 smithi100 ceph-mon[32599]: pgmap v8436: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:52:53.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:52 smithi167 ceph-mon[37140]: pgmap v8436: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:52:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:54 smithi138 ceph-mon[36954]: pgmap v8437: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:52:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:54 smithi100 ceph-mon[32599]: pgmap v8437: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:52:55.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:54 smithi167 ceph-mon[37140]: pgmap v8437: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:52:55.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:52:55.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:52:56.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:52:56.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:52:56.225+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:52:56.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:52:56.226+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:52:56.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:52:56.226+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:52:56.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:52:56.226+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:52:56.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:52:56.226+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:52:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:56 smithi138 ceph-mon[36954]: pgmap v8438: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:52:57.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:52:57.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:52:57.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:52:57.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:52:57.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:52:57.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:52:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:56 smithi100 ceph-mon[32599]: pgmap v8438: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:52:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:52:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:52:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:52:57.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:52:57.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:52:57.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:52:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:56 smithi167 ceph-mon[37140]: pgmap v8438: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:52:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:52:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:52:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:52:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:52:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:52:57.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:52:58.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:52:58.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:52:58.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:52:58.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:52:58.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:52:58.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:52:58.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:52:58.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:52:58.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:52:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:52:58 smithi138 ceph-mon[36954]: pgmap v8439: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:52:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:52:58 smithi100 ceph-mon[32599]: pgmap v8439: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:52:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:52:58 smithi167 ceph-mon[37140]: pgmap v8439: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:53:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:00 smithi138 ceph-mon[36954]: pgmap v8440: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:53:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:00 smithi100 ceph-mon[32599]: pgmap v8440: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:53:01.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:00 smithi167 ceph-mon[37140]: pgmap v8440: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:53:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:02 smithi138 ceph-mon[36954]: pgmap v8441: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:53:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:02 smithi100 ceph-mon[32599]: pgmap v8441: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:53:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:02 smithi167 ceph-mon[37140]: pgmap v8441: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:53:05.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:04 smithi138 ceph-mon[36954]: pgmap v8442: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:53:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:04 smithi100 ceph-mon[32599]: pgmap v8442: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:53:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:04 smithi167 ceph-mon[37140]: pgmap v8442: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:53:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:06 smithi100 ceph-mon[32599]: pgmap v8443: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:53:07.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:06 smithi167 ceph-mon[37140]: pgmap v8443: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:53:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:06 smithi138 ceph-mon[36954]: pgmap v8443: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:53:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:08 smithi100 ceph-mon[32599]: pgmap v8444: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:53:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:08 smithi100 ceph-mon[32599]: osdmap e574: 12 total, 12 up, 12 in 2024-07-26T17:53:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:08 smithi167 ceph-mon[37140]: pgmap v8444: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:53:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:08 smithi167 ceph-mon[37140]: osdmap e574: 12 total, 12 up, 12 in 2024-07-26T17:53:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:08 smithi138 ceph-mon[36954]: pgmap v8444: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:53:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:08 smithi138 ceph-mon[36954]: osdmap e574: 12 total, 12 up, 12 in 2024-07-26T17:53:11.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:10 smithi100 ceph-mon[32599]: pgmap v8446: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:53:11.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:53:11.241+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:53:11.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:53:11.241+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:53:11.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:53:11.241+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:53:11.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:53:11.241+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:53:11.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:53:11.241+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:53:11.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:10 smithi167 ceph-mon[37140]: pgmap v8446: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:53:11.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:10 smithi138 ceph-mon[36954]: pgmap v8446: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:53:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:12 smithi100 ceph-mon[32599]: pgmap v8447: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:53:13.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:12 smithi167 ceph-mon[37140]: pgmap v8447: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:53:13.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:12 smithi138 ceph-mon[36954]: pgmap v8447: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:53:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:14 smithi100 ceph-mon[32599]: pgmap v8448: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:53:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:14 smithi167 ceph-mon[37140]: pgmap v8448: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:53:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:14 smithi138 ceph-mon[36954]: pgmap v8448: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:53:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:16 smithi100 ceph-mon[32599]: pgmap v8449: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:53:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:16 smithi167 ceph-mon[37140]: pgmap v8449: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:53:17.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:16 smithi138 ceph-mon[36954]: pgmap v8449: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:53:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:18 smithi167 ceph-mon[37140]: pgmap v8450: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:53:19.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:18 smithi167 ceph-mon[37140]: osdmap e575: 12 total, 12 up, 12 in 2024-07-26T17:53:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:18 smithi138 ceph-mon[36954]: pgmap v8450: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:53:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:18 smithi138 ceph-mon[36954]: osdmap e575: 12 total, 12 up, 12 in 2024-07-26T17:53:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:18 smithi100 ceph-mon[32599]: pgmap v8450: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:53:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:18 smithi100 ceph-mon[32599]: osdmap e575: 12 total, 12 up, 12 in 2024-07-26T17:53:21.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:20 smithi138 ceph-mon[36954]: pgmap v8452: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:53:21.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:20 smithi100 ceph-mon[32599]: pgmap v8452: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:53:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:20 smithi167 ceph-mon[37140]: pgmap v8452: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:53:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:22 smithi138 ceph-mon[36954]: pgmap v8453: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:53:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:22 smithi100 ceph-mon[32599]: pgmap v8453: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:53:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:22 smithi167 ceph-mon[37140]: pgmap v8453: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:53:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:24 smithi138 ceph-mon[36954]: pgmap v8454: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:53:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:24 smithi100 ceph-mon[32599]: pgmap v8454: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:53:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:24 smithi167 ceph-mon[37140]: pgmap v8454: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:53:26.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:53:26.250+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:53:26.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:53:26.251+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:53:26.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:53:26.251+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:53:26.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:53:26.251+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:53:26.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:53:26.251+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:53:27.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:26 smithi167 ceph-mon[37140]: pgmap v8455: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:53:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:26 smithi138 ceph-mon[36954]: pgmap v8455: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:53:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:26 smithi100 ceph-mon[32599]: pgmap v8455: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:53:29.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:28 smithi138 ceph-mon[36954]: pgmap v8456: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:53:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:28 smithi100 ceph-mon[32599]: pgmap v8456: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:53:29.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:28 smithi167 ceph-mon[37140]: pgmap v8456: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:53:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:30 smithi138 ceph-mon[36954]: pgmap v8457: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:53:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:30 smithi100 ceph-mon[32599]: pgmap v8457: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:53:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:30 smithi167 ceph-mon[37140]: pgmap v8457: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:53:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:32 smithi138 ceph-mon[36954]: pgmap v8458: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:53:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:32 smithi100 ceph-mon[32599]: pgmap v8458: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:53:33.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:32 smithi167 ceph-mon[37140]: pgmap v8458: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:53:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:34 smithi138 ceph-mon[36954]: pgmap v8459: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:53:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:34 smithi100 ceph-mon[32599]: pgmap v8459: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:53:35.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:34 smithi167 ceph-mon[37140]: pgmap v8459: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:53:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:36 smithi138 ceph-mon[36954]: pgmap v8460: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:53:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:36 smithi100 ceph-mon[32599]: pgmap v8460: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:53:37.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:36 smithi167 ceph-mon[37140]: pgmap v8460: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:53:39.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:38 smithi138 ceph-mon[36954]: pgmap v8461: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:53:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:38 smithi100 ceph-mon[32599]: pgmap v8461: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:53:39.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:38 smithi167 ceph-mon[37140]: pgmap v8461: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:53:41.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:40 smithi138 ceph-mon[36954]: pgmap v8462: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:53:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:40 smithi100 ceph-mon[32599]: pgmap v8462: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:53:41.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:53:41.266+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:53:41.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:53:41.267+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:53:41.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:53:41.267+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:53:41.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:53:41.267+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:53:41.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:53:41.267+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:53:41.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:40 smithi167 ceph-mon[37140]: pgmap v8462: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:53:43.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:42 smithi138 ceph-mon[36954]: pgmap v8463: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:53:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:42 smithi100 ceph-mon[32599]: pgmap v8463: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:53:43.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:42 smithi167 ceph-mon[37140]: pgmap v8463: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:53:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:44 smithi138 ceph-mon[36954]: pgmap v8464: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:53:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:44 smithi100 ceph-mon[32599]: pgmap v8464: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:53:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:44 smithi167 ceph-mon[37140]: pgmap v8464: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:53:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:46 smithi138 ceph-mon[36954]: pgmap v8465: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:53:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:46 smithi100 ceph-mon[32599]: pgmap v8465: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:53:47.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:46 smithi167 ceph-mon[37140]: pgmap v8465: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:53:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:48 smithi138 ceph-mon[36954]: pgmap v8466: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:53:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:48 smithi100 ceph-mon[32599]: pgmap v8466: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:53:49.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:48 smithi167 ceph-mon[37140]: pgmap v8466: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:53:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:50 smithi138 ceph-mon[36954]: pgmap v8467: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:53:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:50 smithi100 ceph-mon[32599]: pgmap v8467: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:53:51.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:50 smithi167 ceph-mon[37140]: pgmap v8467: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:53:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:52 smithi138 ceph-mon[36954]: pgmap v8468: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:53:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:52 smithi100 ceph-mon[32599]: pgmap v8468: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:53:53.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:52 smithi167 ceph-mon[37140]: pgmap v8468: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:53:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:54 smithi138 ceph-mon[36954]: pgmap v8469: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:53:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:54 smithi100 ceph-mon[32599]: pgmap v8469: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:53:55.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:54 smithi167 ceph-mon[37140]: pgmap v8469: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:53:56.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:53:56.283+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:53:56.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:53:56.283+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:53:56.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:53:56.283+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:53:56.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:53:56.283+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:53:56.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:53:56.283+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:53:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:56 smithi138 ceph-mon[36954]: pgmap v8470: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:53:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:56 smithi100 ceph-mon[32599]: pgmap v8470: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:53:57.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:56 smithi167 ceph-mon[37140]: pgmap v8470: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:53:58.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:53:58.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:53:58.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:53:58.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:53:58.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:53:58.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:53:58.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:53:58.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:53:58.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:53:58.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:53:58.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:53:58.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:53:58.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:53:58.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:53:58.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:53:58.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:53:58.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:53:58.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:53:58.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:53:58.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:53:58.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:53:58.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:53:58.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:53:58.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:53:58.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:53:58.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:53:58.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:53:58.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:53:58.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:53:58.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:53:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:53:58 smithi138 ceph-mon[36954]: pgmap v8471: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:53:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:53:58 smithi100 ceph-mon[32599]: pgmap v8471: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:53:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:53:58 smithi167 ceph-mon[37140]: pgmap v8471: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:54:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:54:00 smithi138 ceph-mon[36954]: pgmap v8472: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:54:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:54:00 smithi100 ceph-mon[32599]: pgmap v8472: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:54:01.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:54:00 smithi167 ceph-mon[37140]: pgmap v8472: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:54:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:54:02 smithi138 ceph-mon[36954]: pgmap v8473: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:54:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:54:02 smithi100 ceph-mon[32599]: pgmap v8473: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:54:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:54:02 smithi167 ceph-mon[37140]: pgmap v8473: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:54:05.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:54:05.108+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c0066b0 con 0x7fe5a81087a0 2024-07-26T17:54:05.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:54:05.108+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T17:54:05.109 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:54:04 smithi100 ceph-mon[32599]: pgmap v8474: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:54:05.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:54:05.108+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 116 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c0066b0 con 0x7fe5a81087a0 2024-07-26T17:54:05.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:54:05.108+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 116 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T17:54:05.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:54:04 smithi138 ceph-mon[36954]: pgmap v8474: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:54:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:54:04 smithi167 ceph-mon[37140]: pgmap v8474: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:54:07.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:54:06 smithi138 ceph-mon[36954]: pgmap v8475: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:54:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:54:06 smithi100 ceph-mon[32599]: pgmap v8475: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:54:07.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:54:06 smithi167 ceph-mon[37140]: pgmap v8475: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:54:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:54:08 smithi138 ceph-mon[36954]: pgmap v8476: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:54:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:54:08 smithi138 ceph-mon[36954]: osdmap e576: 12 total, 12 up, 12 in 2024-07-26T17:54:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:54:08 smithi100 ceph-mon[32599]: pgmap v8476: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:54:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:54:08 smithi100 ceph-mon[32599]: osdmap e576: 12 total, 12 up, 12 in 2024-07-26T17:54:09.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:54:08 smithi167 ceph-mon[37140]: pgmap v8476: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:54:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:54:08 smithi167 ceph-mon[37140]: osdmap e576: 12 total, 12 up, 12 in 2024-07-26T17:54:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:54:10 smithi138 ceph-mon[36954]: pgmap v8478: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:54:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:54:10 smithi100 ceph-mon[32599]: pgmap v8478: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:54:11.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:54:10 smithi167 ceph-mon[37140]: pgmap v8478: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:54:11.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:54:11.286+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:54:11.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:54:11.287+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:54:11.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:54:11.287+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:54:11.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:54:11.287+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:54:11.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:54:11.287+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:54:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:54:12 smithi138 ceph-mon[36954]: pgmap v8479: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:54:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:54:12 smithi100 ceph-mon[32599]: pgmap v8479: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:54:13.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:54:12 smithi167 ceph-mon[37140]: pgmap v8479: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:54:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:54:14 smithi138 ceph-mon[36954]: pgmap v8480: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:54:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:54:14 smithi100 ceph-mon[32599]: pgmap v8480: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:54:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:54:14 smithi167 ceph-mon[37140]: pgmap v8480: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:54:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:54:16 smithi138 ceph-mon[36954]: pgmap v8481: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:54:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:54:16 smithi100 ceph-mon[32599]: pgmap v8481: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:54:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:54:16 smithi167 ceph-mon[37140]: pgmap v8481: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:54:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:54:18 smithi138 ceph-mon[36954]: pgmap v8482: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:54:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:54:18 smithi138 ceph-mon[36954]: osdmap e577: 12 total, 12 up, 12 in 2024-07-26T17:54:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:54:18 smithi100 ceph-mon[32599]: pgmap v8482: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:54:19.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:54:18 smithi100 ceph-mon[32599]: osdmap e577: 12 total, 12 up, 12 in 2024-07-26T17:54:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:54:18 smithi167 ceph-mon[37140]: pgmap v8482: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:54:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:54:18 smithi167 ceph-mon[37140]: osdmap e577: 12 total, 12 up, 12 in 2024-07-26T17:54:21.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:54:20 smithi138 ceph-mon[36954]: pgmap v8484: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:54:21.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:54:20 smithi100 ceph-mon[32599]: pgmap v8484: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:54:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:54:20 smithi167 ceph-mon[37140]: pgmap v8484: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:54:23.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:54:22 smithi138 ceph-mon[36954]: pgmap v8485: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:54:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:54:22 smithi100 ceph-mon[32599]: pgmap v8485: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:54:23.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:54:22 smithi167 ceph-mon[37140]: pgmap v8485: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:54:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:54:24 smithi138 ceph-mon[36954]: pgmap v8486: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:54:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:54:24 smithi100 ceph-mon[32599]: pgmap v8486: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:54:25.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:54:24 smithi167 ceph-mon[37140]: pgmap v8486: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:54:26.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:54:26.302+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:54:26.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:54:26.303+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:54:26.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:54:26.303+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:54:26.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:54:26.303+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:54:26.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:54:26.303+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:54:27.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:54:26 smithi167 ceph-mon[37140]: pgmap v8487: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:54:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:54:26 smithi138 ceph-mon[36954]: pgmap v8487: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:54:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:54:26 smithi100 ceph-mon[32599]: pgmap v8487: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:54:29.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:54:28 smithi138 ceph-mon[36954]: pgmap v8488: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:54:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:54:28 smithi100 ceph-mon[32599]: pgmap v8488: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:54:29.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:54:28 smithi167 ceph-mon[37140]: pgmap v8488: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:54:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:54:30 smithi138 ceph-mon[36954]: pgmap v8489: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:54:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:54:30 smithi100 ceph-mon[32599]: pgmap v8489: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:54:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:54:30 smithi167 ceph-mon[37140]: pgmap v8489: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:54:33.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:54:32 smithi138 ceph-mon[36954]: pgmap v8490: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:54:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:54:32 smithi100 ceph-mon[32599]: pgmap v8490: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:54:33.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:54:32 smithi167 ceph-mon[37140]: pgmap v8490: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:54:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:54:34 smithi138 ceph-mon[36954]: pgmap v8491: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:54:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:54:34 smithi100 ceph-mon[32599]: pgmap v8491: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:54:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:54:34 smithi167 ceph-mon[37140]: pgmap v8491: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:54:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:54:36 smithi138 ceph-mon[36954]: pgmap v8492: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:54:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:54:36 smithi100 ceph-mon[32599]: pgmap v8492: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:54:37.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:54:36 smithi167 ceph-mon[37140]: pgmap v8492: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:54:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:54:38 smithi138 ceph-mon[36954]: pgmap v8493: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:54:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:54:38 smithi100 ceph-mon[32599]: pgmap v8493: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:54:39.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:54:38 smithi167 ceph-mon[37140]: pgmap v8493: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:54:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:54:40 smithi138 ceph-mon[36954]: pgmap v8494: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:54:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:54:40 smithi100 ceph-mon[32599]: pgmap v8494: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:54:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:54:40 smithi167 ceph-mon[37140]: pgmap v8494: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:54:41.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:54:41.319+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:54:41.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:54:41.319+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:54:41.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:54:41.319+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:54:41.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:54:41.319+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:54:41.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:54:41.319+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:54:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:54:42 smithi138 ceph-mon[36954]: pgmap v8495: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:54:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:54:42 smithi100 ceph-mon[32599]: pgmap v8495: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:54:43.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:54:42 smithi167 ceph-mon[37140]: pgmap v8495: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:54:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:54:44 smithi138 ceph-mon[36954]: pgmap v8496: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:54:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:54:44 smithi100 ceph-mon[32599]: pgmap v8496: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:54:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:54:44 smithi167 ceph-mon[37140]: pgmap v8496: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:54:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:54:46 smithi138 ceph-mon[36954]: pgmap v8497: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:54:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:54:46 smithi100 ceph-mon[32599]: pgmap v8497: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:54:47.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:54:46 smithi167 ceph-mon[37140]: pgmap v8497: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:54:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:54:48 smithi138 ceph-mon[36954]: pgmap v8498: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:54:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:54:48 smithi100 ceph-mon[32599]: pgmap v8498: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:54:49.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:54:48 smithi167 ceph-mon[37140]: pgmap v8498: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:54:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:54:50 smithi138 ceph-mon[36954]: pgmap v8499: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:54:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:54:50 smithi100 ceph-mon[32599]: pgmap v8499: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:54:51.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:54:50 smithi167 ceph-mon[37140]: pgmap v8499: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:54:53.194 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:54:52 smithi138 ceph-mon[36954]: pgmap v8500: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:54:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:54:52 smithi100 ceph-mon[32599]: pgmap v8500: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:54:53.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:54:52 smithi167 ceph-mon[37140]: pgmap v8500: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:54:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:54:54 smithi138 ceph-mon[36954]: pgmap v8501: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:54:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:54:54 smithi100 ceph-mon[32599]: pgmap v8501: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:54:55.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:54:54 smithi167 ceph-mon[37140]: pgmap v8501: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:54:56.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:54:56.327+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:54:56.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:54:56.327+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:54:56.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:54:56.327+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:54:56.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:54:56.327+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:54:56.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:54:56.328+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:54:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:54:56 smithi138 ceph-mon[36954]: pgmap v8502: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:54:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:54:56 smithi100 ceph-mon[32599]: pgmap v8502: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:54:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:54:56 smithi167 ceph-mon[37140]: pgmap v8502: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:54:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:54:58 smithi138 ceph-mon[36954]: pgmap v8503: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:54:59.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:54:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:54:59.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:54:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:54:59.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:54:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:54:59.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:54:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:54:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:54:58 smithi100 ceph-mon[32599]: pgmap v8503: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:54:59.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:54:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:54:59.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:54:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:54:59.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:54:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:54:59.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:54:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:54:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:54:58 smithi167 ceph-mon[37140]: pgmap v8503: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:54:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:54:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:54:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:54:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:54:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:54:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:54:59.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:54:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:55:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:55:00 smithi138 ceph-mon[36954]: pgmap v8504: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:55:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:55:00 smithi100 ceph-mon[32599]: pgmap v8504: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:55:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:55:00 smithi167 ceph-mon[37140]: pgmap v8504: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:55:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:55:02 smithi138 ceph-mon[36954]: pgmap v8505: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:55:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:55:02 smithi100 ceph-mon[32599]: pgmap v8505: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:55:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:55:02 smithi167 ceph-mon[37140]: pgmap v8505: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:55:05.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:55:04 smithi138 ceph-mon[36954]: pgmap v8506: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:55:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:55:04 smithi100 ceph-mon[32599]: pgmap v8506: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:55:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:55:04 smithi167 ceph-mon[37140]: pgmap v8506: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:55:07.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:55:06 smithi138 ceph-mon[36954]: pgmap v8507: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:55:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:55:06 smithi100 ceph-mon[32599]: pgmap v8507: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:55:07.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:55:06 smithi167 ceph-mon[37140]: pgmap v8507: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:55:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:55:08 smithi138 ceph-mon[36954]: pgmap v8508: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:55:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:55:08 smithi138 ceph-mon[36954]: osdmap e578: 12 total, 12 up, 12 in 2024-07-26T17:55:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:55:08 smithi100 ceph-mon[32599]: pgmap v8508: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:55:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:55:08 smithi100 ceph-mon[32599]: osdmap e578: 12 total, 12 up, 12 in 2024-07-26T17:55:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:55:08 smithi167 ceph-mon[37140]: pgmap v8508: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:55:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:55:08 smithi167 ceph-mon[37140]: osdmap e578: 12 total, 12 up, 12 in 2024-07-26T17:55:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:55:10 smithi138 ceph-mon[36954]: pgmap v8510: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:55:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:55:10 smithi100 ceph-mon[32599]: pgmap v8510: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:55:11.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:55:10 smithi167 ceph-mon[37140]: pgmap v8510: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:55:11.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:55:11.343+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:55:11.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:55:11.344+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:55:11.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:55:11.344+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:55:11.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:55:11.344+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:55:11.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:55:11.344+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:55:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:55:12 smithi138 ceph-mon[36954]: pgmap v8511: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:55:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:55:12 smithi100 ceph-mon[32599]: pgmap v8511: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:55:13.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:55:12 smithi167 ceph-mon[37140]: pgmap v8511: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:55:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:55:14 smithi138 ceph-mon[36954]: pgmap v8512: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:55:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:55:14 smithi100 ceph-mon[32599]: pgmap v8512: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:55:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:55:14 smithi167 ceph-mon[37140]: pgmap v8512: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:55:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:55:16 smithi138 ceph-mon[36954]: pgmap v8513: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:55:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:55:16 smithi100 ceph-mon[32599]: pgmap v8513: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:55:17.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:55:16 smithi167 ceph-mon[37140]: pgmap v8513: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:55:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:55:18 smithi138 ceph-mon[36954]: pgmap v8514: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:55:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:55:18 smithi138 ceph-mon[36954]: osdmap e579: 12 total, 12 up, 12 in 2024-07-26T17:55:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:55:18 smithi100 ceph-mon[32599]: pgmap v8514: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:55:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:55:18 smithi100 ceph-mon[32599]: osdmap e579: 12 total, 12 up, 12 in 2024-07-26T17:55:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:55:18 smithi167 ceph-mon[37140]: pgmap v8514: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:55:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:55:18 smithi167 ceph-mon[37140]: osdmap e579: 12 total, 12 up, 12 in 2024-07-26T17:55:21.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:55:20 smithi138 ceph-mon[36954]: pgmap v8516: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:55:21.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:55:20 smithi100 ceph-mon[32599]: pgmap v8516: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:55:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:55:20 smithi167 ceph-mon[37140]: pgmap v8516: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:55:23.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:55:22 smithi138 ceph-mon[36954]: pgmap v8517: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:55:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:55:22 smithi100 ceph-mon[32599]: pgmap v8517: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:55:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:55:22 smithi167 ceph-mon[37140]: pgmap v8517: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:55:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:55:24 smithi138 ceph-mon[36954]: pgmap v8518: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:55:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:55:24 smithi100 ceph-mon[32599]: pgmap v8518: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:55:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:55:24 smithi167 ceph-mon[37140]: pgmap v8518: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:55:26.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:55:26.359+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:55:26.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:55:26.359+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:55:26.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:55:26.359+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:55:26.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:55:26.359+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:55:26.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:55:26.359+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:55:27.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:55:26 smithi167 ceph-mon[37140]: pgmap v8519: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:55:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:55:26 smithi138 ceph-mon[36954]: pgmap v8519: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:55:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:55:26 smithi100 ceph-mon[32599]: pgmap v8519: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:55:29.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:55:28 smithi138 ceph-mon[36954]: pgmap v8520: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:55:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:55:28 smithi100 ceph-mon[32599]: pgmap v8520: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:55:29.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:55:28 smithi167 ceph-mon[37140]: pgmap v8520: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:55:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:55:30 smithi138 ceph-mon[36954]: pgmap v8521: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:55:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:55:30 smithi100 ceph-mon[32599]: pgmap v8521: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:55:31.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:55:30 smithi167 ceph-mon[37140]: pgmap v8521: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:55:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:55:32 smithi138 ceph-mon[36954]: pgmap v8522: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:55:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:55:32 smithi100 ceph-mon[32599]: pgmap v8522: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:55:33.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:55:32 smithi167 ceph-mon[37140]: pgmap v8522: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:55:35.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:55:35.120+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T17:55:35.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:55:35.120+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c0066b0 con 0x7fe5a81087a0 2024-07-26T17:55:35.122 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:55:34 smithi100 ceph-mon[32599]: pgmap v8523: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T17:55:35.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:55:35.120+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 117 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T17:55:35.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:55:35.120+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 117 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe57c0066b0 con 0x7fe5a81087a0 2024-07-26T17:55:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:55:34 smithi138 ceph-mon[36954]: pgmap v8523: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T17:55:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:55:34 smithi167 ceph-mon[37140]: pgmap v8523: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T17:55:37.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:55:36 smithi138 ceph-mon[36954]: pgmap v8524: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:55:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:55:36 smithi100 ceph-mon[32599]: pgmap v8524: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:55:37.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:55:36 smithi167 ceph-mon[37140]: pgmap v8524: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:55:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:55:38 smithi138 ceph-mon[36954]: pgmap v8525: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:55:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:55:38 smithi100 ceph-mon[32599]: pgmap v8525: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:55:39.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:55:38 smithi167 ceph-mon[37140]: pgmap v8525: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:55:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:55:40 smithi138 ceph-mon[36954]: pgmap v8526: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:55:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:55:40 smithi100 ceph-mon[32599]: pgmap v8526: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:55:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:55:40 smithi167 ceph-mon[37140]: pgmap v8526: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:55:41.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:55:41.375+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:55:41.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:55:41.376+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:55:41.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:55:41.376+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:55:41.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:55:41.376+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:55:41.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:55:41.376+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:55:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:55:42 smithi138 ceph-mon[36954]: pgmap v8527: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:55:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:55:42 smithi100 ceph-mon[32599]: pgmap v8527: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:55:43.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:55:42 smithi167 ceph-mon[37140]: pgmap v8527: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:55:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:55:44 smithi138 ceph-mon[36954]: pgmap v8528: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:55:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:55:44 smithi100 ceph-mon[32599]: pgmap v8528: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:55:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:55:44 smithi167 ceph-mon[37140]: pgmap v8528: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:55:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:55:46 smithi138 ceph-mon[36954]: pgmap v8529: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:55:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:55:46 smithi100 ceph-mon[32599]: pgmap v8529: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:55:47.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:55:46 smithi167 ceph-mon[37140]: pgmap v8529: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:55:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:55:48 smithi138 ceph-mon[36954]: pgmap v8530: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:55:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:55:48 smithi100 ceph-mon[32599]: pgmap v8530: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:55:49.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:55:48 smithi167 ceph-mon[37140]: pgmap v8530: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:55:51.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:55:50 smithi138 ceph-mon[36954]: pgmap v8531: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:55:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:55:50 smithi100 ceph-mon[32599]: pgmap v8531: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:55:51.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:55:50 smithi167 ceph-mon[37140]: pgmap v8531: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:55:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:55:52 smithi138 ceph-mon[36954]: pgmap v8532: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:55:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:55:52 smithi100 ceph-mon[32599]: pgmap v8532: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:55:53.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:55:52 smithi167 ceph-mon[37140]: pgmap v8532: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:55:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:55:54 smithi138 ceph-mon[36954]: pgmap v8533: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:55:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:55:54 smithi100 ceph-mon[32599]: pgmap v8533: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:55:55.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:55:54 smithi167 ceph-mon[37140]: pgmap v8533: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:55:56.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:55:56.392+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:55:56.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:55:56.392+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:55:56.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:55:56.392+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:55:56.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:55:56.392+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:55:56.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:55:56.392+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:55:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:55:56 smithi138 ceph-mon[36954]: pgmap v8534: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:55:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:55:56 smithi100 ceph-mon[32599]: pgmap v8534: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:55:57.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:55:56 smithi167 ceph-mon[37140]: pgmap v8534: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:55:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:55:58 smithi138 ceph-mon[36954]: pgmap v8535: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:55:59.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:55:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:55:59.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:55:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:55:59.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:55:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:55:59.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:55:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:55:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:55:58 smithi100 ceph-mon[32599]: pgmap v8535: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:55:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:55:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:55:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:55:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:55:59.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:55:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:55:59.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:55:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:55:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:55:58 smithi167 ceph-mon[37140]: pgmap v8535: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:55:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:55:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:55:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:55:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:55:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:55:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:55:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:55:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:56:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:56:00 smithi138 ceph-mon[36954]: pgmap v8536: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:56:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:56:00 smithi100 ceph-mon[32599]: pgmap v8536: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:56:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:56:00 smithi167 ceph-mon[37140]: pgmap v8536: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:56:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:56:02 smithi138 ceph-mon[36954]: pgmap v8537: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:56:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:56:02 smithi100 ceph-mon[32599]: pgmap v8537: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:56:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:56:02 smithi167 ceph-mon[37140]: pgmap v8537: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:56:05.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:56:04 smithi138 ceph-mon[36954]: pgmap v8538: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:56:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:56:04 smithi100 ceph-mon[32599]: pgmap v8538: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:56:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:56:04 smithi167 ceph-mon[37140]: pgmap v8538: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:56:07.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:56:06 smithi138 ceph-mon[36954]: pgmap v8539: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:56:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:56:06 smithi100 ceph-mon[32599]: pgmap v8539: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:56:07.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:56:06 smithi167 ceph-mon[37140]: pgmap v8539: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:56:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:56:08 smithi138 ceph-mon[36954]: pgmap v8540: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:56:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:56:08 smithi138 ceph-mon[36954]: osdmap e580: 12 total, 12 up, 12 in 2024-07-26T17:56:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:56:08 smithi100 ceph-mon[32599]: pgmap v8540: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:56:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:56:08 smithi100 ceph-mon[32599]: osdmap e580: 12 total, 12 up, 12 in 2024-07-26T17:56:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:56:08 smithi167 ceph-mon[37140]: pgmap v8540: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:56:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:56:08 smithi167 ceph-mon[37140]: osdmap e580: 12 total, 12 up, 12 in 2024-07-26T17:56:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:56:10 smithi138 ceph-mon[36954]: pgmap v8542: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:56:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:56:10 smithi100 ceph-mon[32599]: pgmap v8542: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:56:11.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:56:10 smithi167 ceph-mon[37140]: pgmap v8542: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:56:11.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:56:11.408+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:56:11.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:56:11.408+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:56:11.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:56:11.408+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:56:11.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:56:11.408+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:56:11.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:56:11.408+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:56:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:56:12 smithi138 ceph-mon[36954]: pgmap v8543: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:56:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:56:12 smithi100 ceph-mon[32599]: pgmap v8543: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:56:13.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:56:12 smithi167 ceph-mon[37140]: pgmap v8543: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T17:56:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:56:14 smithi138 ceph-mon[36954]: pgmap v8544: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:56:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:56:14 smithi100 ceph-mon[32599]: pgmap v8544: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:56:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:56:14 smithi167 ceph-mon[37140]: pgmap v8544: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:56:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:56:16 smithi138 ceph-mon[36954]: pgmap v8545: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:56:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:56:16 smithi100 ceph-mon[32599]: pgmap v8545: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:56:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:56:16 smithi167 ceph-mon[37140]: pgmap v8545: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:56:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:56:18 smithi138 ceph-mon[36954]: pgmap v8546: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:56:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:56:18 smithi138 ceph-mon[36954]: osdmap e581: 12 total, 12 up, 12 in 2024-07-26T17:56:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:56:18 smithi100 ceph-mon[32599]: pgmap v8546: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:56:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:56:18 smithi100 ceph-mon[32599]: osdmap e581: 12 total, 12 up, 12 in 2024-07-26T17:56:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:56:18 smithi167 ceph-mon[37140]: pgmap v8546: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:56:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:56:18 smithi167 ceph-mon[37140]: osdmap e581: 12 total, 12 up, 12 in 2024-07-26T17:56:21.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:56:20 smithi138 ceph-mon[36954]: pgmap v8548: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:56:21.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:56:20 smithi100 ceph-mon[32599]: pgmap v8548: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:56:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:56:20 smithi167 ceph-mon[37140]: pgmap v8548: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:56:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:56:22 smithi138 ceph-mon[36954]: pgmap v8549: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:56:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:56:22 smithi100 ceph-mon[32599]: pgmap v8549: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:56:23.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:56:22 smithi167 ceph-mon[37140]: pgmap v8549: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:56:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:56:24 smithi138 ceph-mon[36954]: pgmap v8550: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:56:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:56:24 smithi100 ceph-mon[32599]: pgmap v8550: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:56:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:56:24 smithi167 ceph-mon[37140]: pgmap v8550: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:56:26.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:56:26.424+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:56:26.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:56:26.424+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:56:26.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:56:26.424+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:56:26.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:56:26.424+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:56:26.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:56:26.424+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:56:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:56:26 smithi138 ceph-mon[36954]: pgmap v8551: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:56:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:56:26 smithi100 ceph-mon[32599]: pgmap v8551: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:56:27.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:56:26 smithi167 ceph-mon[37140]: pgmap v8551: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:56:29.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:56:28 smithi138 ceph-mon[36954]: pgmap v8552: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:56:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:56:28 smithi100 ceph-mon[32599]: pgmap v8552: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:56:29.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:56:28 smithi167 ceph-mon[37140]: pgmap v8552: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:56:31.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:56:30 smithi138 ceph-mon[36954]: pgmap v8553: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:56:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:56:30 smithi100 ceph-mon[32599]: pgmap v8553: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:56:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:56:30 smithi167 ceph-mon[37140]: pgmap v8553: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:56:33.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:56:32 smithi138 ceph-mon[36954]: pgmap v8554: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:56:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:56:32 smithi100 ceph-mon[32599]: pgmap v8554: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:56:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:56:32 smithi167 ceph-mon[37140]: pgmap v8554: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:56:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:56:34 smithi138 ceph-mon[36954]: pgmap v8555: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:56:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:56:34 smithi100 ceph-mon[32599]: pgmap v8555: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:56:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:56:34 smithi167 ceph-mon[37140]: pgmap v8555: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:56:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:56:36 smithi138 ceph-mon[36954]: pgmap v8556: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:56:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:56:36 smithi100 ceph-mon[32599]: pgmap v8556: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:56:37.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:56:36 smithi167 ceph-mon[37140]: pgmap v8556: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:56:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:56:38 smithi138 ceph-mon[36954]: pgmap v8557: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:56:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:56:38 smithi100 ceph-mon[32599]: pgmap v8557: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:56:39.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:56:38 smithi167 ceph-mon[37140]: pgmap v8557: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:56:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:56:40 smithi138 ceph-mon[36954]: pgmap v8558: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:56:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:56:40 smithi100 ceph-mon[32599]: pgmap v8558: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:56:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:56:40 smithi167 ceph-mon[37140]: pgmap v8558: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:56:41.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:56:41.439+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:56:41.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:56:41.439+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:56:41.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:56:41.439+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:56:41.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:56:41.439+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:56:41.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:56:41.439+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:56:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:56:42 smithi138 ceph-mon[36954]: pgmap v8559: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:56:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:56:42 smithi100 ceph-mon[32599]: pgmap v8559: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:56:43.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:56:42 smithi167 ceph-mon[37140]: pgmap v8559: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:56:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:56:44 smithi138 ceph-mon[36954]: pgmap v8560: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:56:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:56:44 smithi100 ceph-mon[32599]: pgmap v8560: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:56:45.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:56:44 smithi167 ceph-mon[37140]: pgmap v8560: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:56:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:56:46 smithi138 ceph-mon[36954]: pgmap v8561: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:56:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:56:46 smithi100 ceph-mon[32599]: pgmap v8561: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:56:47.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:56:46 smithi167 ceph-mon[37140]: pgmap v8561: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:56:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:56:48 smithi138 ceph-mon[36954]: pgmap v8562: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:56:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:56:48 smithi100 ceph-mon[32599]: pgmap v8562: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:56:49.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:56:48 smithi167 ceph-mon[37140]: pgmap v8562: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:56:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:56:50 smithi138 ceph-mon[36954]: pgmap v8563: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:56:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:56:50 smithi100 ceph-mon[32599]: pgmap v8563: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:56:51.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:56:50 smithi167 ceph-mon[37140]: pgmap v8563: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:56:53.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:56:52 smithi138 ceph-mon[36954]: pgmap v8564: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:56:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:56:52 smithi100 ceph-mon[32599]: pgmap v8564: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:56:53.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:56:52 smithi167 ceph-mon[37140]: pgmap v8564: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:56:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:56:54 smithi138 ceph-mon[36954]: pgmap v8565: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:56:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:56:54 smithi100 ceph-mon[32599]: pgmap v8565: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:56:55.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:56:54 smithi167 ceph-mon[37140]: pgmap v8565: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:56:56.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:56:56.451+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:56:56.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:56:56.452+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:56:56.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:56:56.452+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:56:56.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:56:56.452+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:56:56.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:56:56.452+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:56:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:56:56 smithi138 ceph-mon[36954]: pgmap v8566: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:56:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:56:56 smithi100 ceph-mon[32599]: pgmap v8566: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:56:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:56:56 smithi167 ceph-mon[37140]: pgmap v8566: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:56:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:56:58 smithi138 ceph-mon[36954]: pgmap v8567: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:56:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:56:58 smithi100 ceph-mon[32599]: pgmap v8567: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:56:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:56:58 smithi167 ceph-mon[37140]: pgmap v8567: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:57:00.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:56:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:57:00.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:56:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:57:00.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:56:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:57:00.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:56:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:57:00.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:56:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:57:00.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:56:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:57:00.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:56:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:57:00.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:56:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:57:00.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:56:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:57:00.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:56:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:57:00.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:56:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:57:00.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:56:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:57:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:57:00 smithi138 ceph-mon[36954]: pgmap v8568: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:57:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:57:00 smithi100 ceph-mon[32599]: pgmap v8568: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:57:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:57:00 smithi167 ceph-mon[37140]: pgmap v8568: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:57:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:57:02 smithi138 ceph-mon[36954]: pgmap v8569: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:57:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:57:02 smithi100 ceph-mon[32599]: pgmap v8569: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:57:03.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:57:02 smithi167 ceph-mon[37140]: pgmap v8569: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:57:05.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:57:05.133+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T17:57:05.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:57:05.133+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c0066b0 con 0x7fe5a81087a0 2024-07-26T17:57:05.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:57:05.133+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 118 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c0066b0 con 0x7fe5a81087a0 2024-07-26T17:57:05.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:57:05.133+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 118 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T17:57:05.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:57:04 smithi138 ceph-mon[36954]: pgmap v8570: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:57:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:57:04 smithi100 ceph-mon[32599]: pgmap v8570: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:57:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:57:04 smithi167 ceph-mon[37140]: pgmap v8570: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T17:57:07.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:57:06 smithi138 ceph-mon[36954]: pgmap v8571: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:57:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:57:06 smithi100 ceph-mon[32599]: pgmap v8571: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:57:07.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:57:06 smithi167 ceph-mon[37140]: pgmap v8571: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:57:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:57:08 smithi138 ceph-mon[36954]: pgmap v8572: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:57:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:57:08 smithi138 ceph-mon[36954]: osdmap e582: 12 total, 12 up, 12 in 2024-07-26T17:57:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:57:08 smithi100 ceph-mon[32599]: pgmap v8572: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:57:09.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:57:08 smithi100 ceph-mon[32599]: osdmap e582: 12 total, 12 up, 12 in 2024-07-26T17:57:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:57:08 smithi167 ceph-mon[37140]: pgmap v8572: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:57:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:57:08 smithi167 ceph-mon[37140]: osdmap e582: 12 total, 12 up, 12 in 2024-07-26T17:57:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:57:10 smithi138 ceph-mon[36954]: pgmap v8574: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:57:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:57:10 smithi100 ceph-mon[32599]: pgmap v8574: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:57:11.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:57:10 smithi167 ceph-mon[37140]: pgmap v8574: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:57:11.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:57:11.453+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:57:11.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:57:11.453+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:57:11.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:57:11.453+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:57:11.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:57:11.453+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:57:11.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:57:11.454+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:57:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:57:12 smithi138 ceph-mon[36954]: pgmap v8575: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:57:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:57:12 smithi100 ceph-mon[32599]: pgmap v8575: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:57:13.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:57:12 smithi167 ceph-mon[37140]: pgmap v8575: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:57:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:57:14 smithi138 ceph-mon[36954]: pgmap v8576: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:57:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:57:14 smithi100 ceph-mon[32599]: pgmap v8576: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:57:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:57:14 smithi167 ceph-mon[37140]: pgmap v8576: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:57:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:57:16 smithi138 ceph-mon[36954]: pgmap v8577: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:57:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:57:16 smithi100 ceph-mon[32599]: pgmap v8577: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:57:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:57:16 smithi167 ceph-mon[37140]: pgmap v8577: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:57:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:57:18 smithi138 ceph-mon[36954]: pgmap v8578: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:57:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:57:18 smithi138 ceph-mon[36954]: osdmap e583: 12 total, 12 up, 12 in 2024-07-26T17:57:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:57:18 smithi100 ceph-mon[32599]: pgmap v8578: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:57:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:57:18 smithi100 ceph-mon[32599]: osdmap e583: 12 total, 12 up, 12 in 2024-07-26T17:57:19.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:57:18 smithi167 ceph-mon[37140]: pgmap v8578: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:57:19.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:57:18 smithi167 ceph-mon[37140]: osdmap e583: 12 total, 12 up, 12 in 2024-07-26T17:57:21.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:57:20 smithi138 ceph-mon[36954]: pgmap v8580: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:57:21.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:57:20 smithi100 ceph-mon[32599]: pgmap v8580: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:57:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:57:20 smithi167 ceph-mon[37140]: pgmap v8580: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:57:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:57:22 smithi138 ceph-mon[36954]: pgmap v8581: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:57:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:57:22 smithi100 ceph-mon[32599]: pgmap v8581: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:57:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:57:22 smithi167 ceph-mon[37140]: pgmap v8581: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:57:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:57:24 smithi138 ceph-mon[36954]: pgmap v8582: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:57:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:57:24 smithi100 ceph-mon[32599]: pgmap v8582: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:57:25.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:57:24 smithi167 ceph-mon[37140]: pgmap v8582: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:57:26.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:57:26.469+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:57:26.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:57:26.470+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:57:26.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:57:26.470+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:57:26.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:57:26.470+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:57:26.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:57:26.470+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:57:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:57:26 smithi138 ceph-mon[36954]: pgmap v8583: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:57:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:57:26 smithi100 ceph-mon[32599]: pgmap v8583: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:57:27.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:57:26 smithi167 ceph-mon[37140]: pgmap v8583: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:57:29.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:57:28 smithi138 ceph-mon[36954]: pgmap v8584: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:57:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:57:28 smithi100 ceph-mon[32599]: pgmap v8584: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:57:29.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:57:28 smithi167 ceph-mon[37140]: pgmap v8584: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:57:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:57:30 smithi138 ceph-mon[36954]: pgmap v8585: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:57:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:57:30 smithi100 ceph-mon[32599]: pgmap v8585: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:57:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:57:30 smithi167 ceph-mon[37140]: pgmap v8585: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:57:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:57:32 smithi138 ceph-mon[36954]: pgmap v8586: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:57:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:57:32 smithi100 ceph-mon[32599]: pgmap v8586: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:57:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:57:32 smithi167 ceph-mon[37140]: pgmap v8586: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:57:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:57:34 smithi138 ceph-mon[36954]: pgmap v8587: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:57:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:57:34 smithi100 ceph-mon[32599]: pgmap v8587: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:57:35.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:57:34 smithi167 ceph-mon[37140]: pgmap v8587: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:57:37.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:57:36 smithi138 ceph-mon[36954]: pgmap v8588: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:57:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:57:36 smithi100 ceph-mon[32599]: pgmap v8588: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:57:37.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:57:36 smithi167 ceph-mon[37140]: pgmap v8588: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:57:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:57:38 smithi138 ceph-mon[36954]: pgmap v8589: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:57:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:57:38 smithi100 ceph-mon[32599]: pgmap v8589: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:57:39.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:57:38 smithi167 ceph-mon[37140]: pgmap v8589: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:57:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:57:40 smithi138 ceph-mon[36954]: pgmap v8590: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:57:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:57:40 smithi100 ceph-mon[32599]: pgmap v8590: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:57:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:57:40 smithi167 ceph-mon[37140]: pgmap v8590: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:57:41.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:57:41.485+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:57:41.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:57:41.486+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:57:41.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:57:41.486+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:57:41.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:57:41.486+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:57:41.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:57:41.486+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:57:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:57:42 smithi138 ceph-mon[36954]: pgmap v8591: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:57:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:57:42 smithi100 ceph-mon[32599]: pgmap v8591: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:57:43.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:57:42 smithi167 ceph-mon[37140]: pgmap v8591: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:57:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:57:44 smithi138 ceph-mon[36954]: pgmap v8592: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:57:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:57:44 smithi100 ceph-mon[32599]: pgmap v8592: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:57:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:57:44 smithi167 ceph-mon[37140]: pgmap v8592: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:57:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:57:46 smithi138 ceph-mon[36954]: pgmap v8593: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:57:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:57:46 smithi100 ceph-mon[32599]: pgmap v8593: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:57:47.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:57:46 smithi167 ceph-mon[37140]: pgmap v8593: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:57:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:57:48 smithi138 ceph-mon[36954]: pgmap v8594: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:57:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:57:48 smithi100 ceph-mon[32599]: pgmap v8594: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:57:49.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:57:48 smithi167 ceph-mon[37140]: pgmap v8594: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:57:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:57:50 smithi138 ceph-mon[36954]: pgmap v8595: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:57:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:57:50 smithi100 ceph-mon[32599]: pgmap v8595: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:57:51.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:57:50 smithi167 ceph-mon[37140]: pgmap v8595: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:57:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:57:52 smithi138 ceph-mon[36954]: pgmap v8596: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:57:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:57:52 smithi100 ceph-mon[32599]: pgmap v8596: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:57:53.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:57:52 smithi167 ceph-mon[37140]: pgmap v8596: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:57:55.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:57:54 smithi138 ceph-mon[36954]: pgmap v8597: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:57:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:57:54 smithi100 ceph-mon[32599]: pgmap v8597: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:57:55.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:57:54 smithi167 ceph-mon[37140]: pgmap v8597: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:57:56.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:57:56.502+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:57:56.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:57:56.502+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:57:56.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:57:56.502+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:57:56.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:57:56.502+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:57:56.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:57:56.502+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:57:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:57:56 smithi138 ceph-mon[36954]: pgmap v8598: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:57:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:57:56 smithi100 ceph-mon[32599]: pgmap v8598: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:57:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:57:56 smithi167 ceph-mon[37140]: pgmap v8598: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:57:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:57:58 smithi138 ceph-mon[36954]: pgmap v8599: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:57:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:57:58 smithi100 ceph-mon[32599]: pgmap v8599: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:57:59.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:57:58 smithi167 ceph-mon[37140]: pgmap v8599: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:58:00.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:57:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:58:00.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:57:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:58:00.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:57:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:58:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:58:00 smithi138 ceph-mon[36954]: pgmap v8600: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:58:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:58:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:58:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:58:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:58:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:58:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:58:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:58:00 smithi100 ceph-mon[32599]: pgmap v8600: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:58:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:58:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:58:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:58:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:58:01.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:58:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:58:01.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:58:00 smithi167 ceph-mon[37140]: pgmap v8600: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:58:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:58:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:58:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:58:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:58:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:58:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:58:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:58:02 smithi138 ceph-mon[36954]: pgmap v8601: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:58:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:58:02 smithi100 ceph-mon[32599]: pgmap v8601: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:58:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:58:02 smithi167 ceph-mon[37140]: pgmap v8601: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:58:05.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:58:04 smithi138 ceph-mon[36954]: pgmap v8602: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-07-26T17:58:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:58:04 smithi100 ceph-mon[32599]: pgmap v8602: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-07-26T17:58:05.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:58:04 smithi167 ceph-mon[37140]: pgmap v8602: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-07-26T17:58:07.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:58:06 smithi138 ceph-mon[36954]: pgmap v8603: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:58:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:58:06 smithi100 ceph-mon[32599]: pgmap v8603: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:58:07.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:58:06 smithi167 ceph-mon[37140]: pgmap v8603: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:58:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:58:08 smithi100 ceph-mon[32599]: pgmap v8604: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:58:09.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:58:08 smithi100 ceph-mon[32599]: osdmap e584: 12 total, 12 up, 12 in 2024-07-26T17:58:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:58:08 smithi167 ceph-mon[37140]: pgmap v8604: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:58:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:58:08 smithi167 ceph-mon[37140]: osdmap e584: 12 total, 12 up, 12 in 2024-07-26T17:58:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:58:08 smithi138 ceph-mon[36954]: pgmap v8604: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:58:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:58:08 smithi138 ceph-mon[36954]: osdmap e584: 12 total, 12 up, 12 in 2024-07-26T17:58:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:58:10 smithi100 ceph-mon[32599]: pgmap v8606: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:58:11.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:58:10 smithi167 ceph-mon[37140]: pgmap v8606: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:58:11.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:58:10 smithi138 ceph-mon[36954]: pgmap v8606: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:58:11.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:58:11.518+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:58:11.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:58:11.519+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:58:11.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:58:11.519+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:58:11.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:58:11.519+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:58:11.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:58:11.519+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:58:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:58:12 smithi100 ceph-mon[32599]: pgmap v8607: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:58:13.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:58:12 smithi167 ceph-mon[37140]: pgmap v8607: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:58:13.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:58:12 smithi138 ceph-mon[36954]: pgmap v8607: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:58:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:58:14 smithi100 ceph-mon[32599]: pgmap v8608: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:58:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:58:14 smithi167 ceph-mon[37140]: pgmap v8608: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:58:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:58:14 smithi138 ceph-mon[36954]: pgmap v8608: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:58:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:58:16 smithi100 ceph-mon[32599]: pgmap v8609: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:58:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:58:16 smithi167 ceph-mon[37140]: pgmap v8609: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:58:17.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:58:16 smithi138 ceph-mon[36954]: pgmap v8609: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:58:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:58:18 smithi138 ceph-mon[36954]: pgmap v8610: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:58:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:58:18 smithi138 ceph-mon[36954]: osdmap e585: 12 total, 12 up, 12 in 2024-07-26T17:58:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:58:18 smithi100 ceph-mon[32599]: pgmap v8610: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:58:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:58:18 smithi100 ceph-mon[32599]: osdmap e585: 12 total, 12 up, 12 in 2024-07-26T17:58:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:58:18 smithi167 ceph-mon[37140]: pgmap v8610: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:58:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:58:18 smithi167 ceph-mon[37140]: osdmap e585: 12 total, 12 up, 12 in 2024-07-26T17:58:21.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:58:20 smithi138 ceph-mon[36954]: pgmap v8612: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:58:21.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:58:20 smithi100 ceph-mon[32599]: pgmap v8612: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:58:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:58:20 smithi167 ceph-mon[37140]: pgmap v8612: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:58:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:58:22 smithi138 ceph-mon[36954]: pgmap v8613: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:58:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:58:22 smithi100 ceph-mon[32599]: pgmap v8613: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:58:23.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:58:22 smithi167 ceph-mon[37140]: pgmap v8613: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:58:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:58:24 smithi138 ceph-mon[36954]: pgmap v8614: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:58:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:58:24 smithi100 ceph-mon[32599]: pgmap v8614: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:58:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:58:24 smithi167 ceph-mon[37140]: pgmap v8614: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:58:26.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:58:26.527+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:58:26.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:58:26.527+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:58:26.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:58:26.527+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:58:26.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:58:26.527+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:58:26.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:58:26.527+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:58:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:58:26 smithi138 ceph-mon[36954]: pgmap v8615: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:58:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:58:26 smithi100 ceph-mon[32599]: pgmap v8615: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:58:27.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:58:26 smithi167 ceph-mon[37140]: pgmap v8615: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:58:29.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:58:28 smithi138 ceph-mon[36954]: pgmap v8616: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:58:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:58:28 smithi100 ceph-mon[32599]: pgmap v8616: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:58:29.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:58:28 smithi167 ceph-mon[37140]: pgmap v8616: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:58:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:58:30 smithi138 ceph-mon[36954]: pgmap v8617: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:58:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:58:30 smithi100 ceph-mon[32599]: pgmap v8617: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:58:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:58:30 smithi167 ceph-mon[37140]: pgmap v8617: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:58:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:58:32 smithi138 ceph-mon[36954]: pgmap v8618: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:58:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:58:32 smithi100 ceph-mon[32599]: pgmap v8618: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:58:33.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:58:32 smithi167 ceph-mon[37140]: pgmap v8618: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:58:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:58:34 smithi138 ceph-mon[36954]: pgmap v8619: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:58:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:58:34 smithi100 ceph-mon[32599]: pgmap v8619: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:58:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:58:34 smithi167 ceph-mon[37140]: pgmap v8619: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:58:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:58:36 smithi138 ceph-mon[36954]: pgmap v8620: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:58:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:58:36 smithi100 ceph-mon[32599]: pgmap v8620: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:58:37.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:58:36 smithi167 ceph-mon[37140]: pgmap v8620: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:58:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:58:38 smithi138 ceph-mon[36954]: pgmap v8621: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:58:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:58:38 smithi100 ceph-mon[32599]: pgmap v8621: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:58:39.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:58:38 smithi167 ceph-mon[37140]: pgmap v8621: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:58:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:58:40 smithi138 ceph-mon[36954]: pgmap v8622: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:58:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:58:40 smithi100 ceph-mon[32599]: pgmap v8622: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:58:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:58:40 smithi167 ceph-mon[37140]: pgmap v8622: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:58:41.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:58:41.536+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:58:41.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:58:41.537+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:58:41.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:58:41.537+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:58:41.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:58:41.537+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:58:41.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:58:41.537+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:58:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:58:42 smithi138 ceph-mon[36954]: pgmap v8623: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:58:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:58:42 smithi100 ceph-mon[32599]: pgmap v8623: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:58:43.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:58:42 smithi167 ceph-mon[37140]: pgmap v8623: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:58:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:58:44 smithi138 ceph-mon[36954]: pgmap v8624: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:58:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:58:44 smithi100 ceph-mon[32599]: pgmap v8624: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:58:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:58:44 smithi167 ceph-mon[37140]: pgmap v8624: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:58:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:58:46 smithi138 ceph-mon[36954]: pgmap v8625: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:58:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:58:46 smithi100 ceph-mon[32599]: pgmap v8625: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:58:47.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:58:46 smithi167 ceph-mon[37140]: pgmap v8625: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:58:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:58:48 smithi138 ceph-mon[36954]: pgmap v8626: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:58:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:58:48 smithi100 ceph-mon[32599]: pgmap v8626: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:58:49.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:58:48 smithi167 ceph-mon[37140]: pgmap v8626: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:58:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:58:50 smithi138 ceph-mon[36954]: pgmap v8627: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:58:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:58:50 smithi100 ceph-mon[32599]: pgmap v8627: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:58:51.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:58:50 smithi167 ceph-mon[37140]: pgmap v8627: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:58:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:58:52 smithi138 ceph-mon[36954]: pgmap v8628: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:58:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:58:52 smithi100 ceph-mon[32599]: pgmap v8628: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:58:53.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:58:52 smithi167 ceph-mon[37140]: pgmap v8628: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:58:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:58:54 smithi138 ceph-mon[36954]: pgmap v8629: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:58:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:58:54 smithi100 ceph-mon[32599]: pgmap v8629: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:58:55.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:58:54 smithi167 ceph-mon[37140]: pgmap v8629: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:58:56.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:58:56.552+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:58:57.397 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:58:56 smithi167 ceph-mon[37140]: pgmap v8630: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:58:57.398 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:58:56 smithi138 ceph-mon[36954]: pgmap v8630: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:58:57.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:58:56.553+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:58:57.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:58:56.553+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:58:57.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:58:56.553+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:58:57.398 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:58:56.553+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:58:57.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:58:56 smithi100 ceph-mon[32599]: pgmap v8630: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:58:59.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:58:58 smithi138 ceph-mon[36954]: pgmap v8631: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-07-26T17:58:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:58:58 smithi100 ceph-mon[32599]: pgmap v8631: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-07-26T17:58:59.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:58:58 smithi167 ceph-mon[37140]: pgmap v8631: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-07-26T17:59:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:59:00 smithi138 ceph-mon[36954]: pgmap v8632: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:59:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:59:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:59:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:59:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:59:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:59:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:59:01.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:59:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:59:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:59:00 smithi100 ceph-mon[32599]: pgmap v8632: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:59:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:59:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:59:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:59:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:59:01.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:59:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:59:01.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:59:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:59:01.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:59:00 smithi167 ceph-mon[37140]: pgmap v8632: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T17:59:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:59:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T17:59:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:59:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T17:59:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:59:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T17:59:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:59:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T17:59:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:59:02 smithi100 ceph-mon[32599]: pgmap v8633: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:59:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:59:02 smithi167 ceph-mon[37140]: pgmap v8633: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:59:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:59:02 smithi138 ceph-mon[36954]: pgmap v8633: 97 pgs: 97 active+clean; 235 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T17:59:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:59:04 smithi100 ceph-mon[32599]: pgmap v8634: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-07-26T17:59:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:59:04 smithi167 ceph-mon[37140]: pgmap v8634: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-07-26T17:59:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:59:04 smithi138 ceph-mon[36954]: pgmap v8634: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-07-26T17:59:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:59:06 smithi100 ceph-mon[32599]: pgmap v8635: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:59:07.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:59:06 smithi167 ceph-mon[37140]: pgmap v8635: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:59:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:59:06 smithi138 ceph-mon[36954]: pgmap v8635: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:59:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:59:08 smithi100 ceph-mon[32599]: pgmap v8636: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:59:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:59:08 smithi100 ceph-mon[32599]: osdmap e586: 12 total, 12 up, 12 in 2024-07-26T17:59:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:59:08 smithi167 ceph-mon[37140]: pgmap v8636: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:59:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:59:08 smithi167 ceph-mon[37140]: osdmap e586: 12 total, 12 up, 12 in 2024-07-26T17:59:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:59:08 smithi138 ceph-mon[36954]: pgmap v8636: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T17:59:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:59:08 smithi138 ceph-mon[36954]: osdmap e586: 12 total, 12 up, 12 in 2024-07-26T17:59:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:59:10 smithi100 ceph-mon[32599]: pgmap v8638: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:59:11.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:59:10 smithi167 ceph-mon[37140]: pgmap v8638: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:59:11.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:59:10 smithi138 ceph-mon[36954]: pgmap v8638: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:59:11.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:59:11.562+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:59:11.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:59:11.562+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:59:11.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:59:11.562+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:59:11.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:59:11.562+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:59:11.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:59:11.562+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:59:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:59:12 smithi100 ceph-mon[32599]: pgmap v8639: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:59:13.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:59:12 smithi167 ceph-mon[37140]: pgmap v8639: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:59:13.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:59:12 smithi138 ceph-mon[36954]: pgmap v8639: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T17:59:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:59:14 smithi100 ceph-mon[32599]: pgmap v8640: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:59:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:59:14 smithi167 ceph-mon[37140]: pgmap v8640: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:59:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:59:14 smithi138 ceph-mon[36954]: pgmap v8640: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T17:59:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:59:16 smithi100 ceph-mon[32599]: pgmap v8641: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:59:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:59:16 smithi167 ceph-mon[37140]: pgmap v8641: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:59:17.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:59:16 smithi138 ceph-mon[36954]: pgmap v8641: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:59:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:59:18 smithi138 ceph-mon[36954]: pgmap v8642: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:59:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:59:18 smithi138 ceph-mon[36954]: osdmap e587: 12 total, 12 up, 12 in 2024-07-26T17:59:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:59:18 smithi100 ceph-mon[32599]: pgmap v8642: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:59:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:59:18 smithi100 ceph-mon[32599]: osdmap e587: 12 total, 12 up, 12 in 2024-07-26T17:59:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:59:18 smithi167 ceph-mon[37140]: pgmap v8642: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T17:59:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:59:18 smithi167 ceph-mon[37140]: osdmap e587: 12 total, 12 up, 12 in 2024-07-26T17:59:21.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:59:20 smithi138 ceph-mon[36954]: pgmap v8644: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:59:21.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:59:20 smithi100 ceph-mon[32599]: pgmap v8644: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:59:21.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:59:20 smithi167 ceph-mon[37140]: pgmap v8644: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:59:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:59:22 smithi138 ceph-mon[36954]: pgmap v8645: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:59:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:59:22 smithi100 ceph-mon[32599]: pgmap v8645: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:59:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:59:22 smithi167 ceph-mon[37140]: pgmap v8645: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:59:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:59:24 smithi138 ceph-mon[36954]: pgmap v8646: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:59:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:59:24 smithi100 ceph-mon[32599]: pgmap v8646: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:59:25.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:59:24 smithi167 ceph-mon[37140]: pgmap v8646: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T17:59:26.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:59:26.578+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:59:26.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:59:26.578+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:59:26.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:59:26.578+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:59:26.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:59:26.578+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:59:26.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:59:26.578+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:59:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:59:26 smithi138 ceph-mon[36954]: pgmap v8647: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:59:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:59:26 smithi100 ceph-mon[32599]: pgmap v8647: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:59:27.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:59:26 smithi167 ceph-mon[37140]: pgmap v8647: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:59:29.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:59:28 smithi138 ceph-mon[36954]: pgmap v8648: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:59:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:59:28 smithi100 ceph-mon[32599]: pgmap v8648: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:59:29.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:59:28 smithi167 ceph-mon[37140]: pgmap v8648: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T17:59:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:59:30 smithi138 ceph-mon[36954]: pgmap v8649: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:59:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:59:30 smithi100 ceph-mon[32599]: pgmap v8649: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:59:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:59:30 smithi167 ceph-mon[37140]: pgmap v8649: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T17:59:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:59:32 smithi138 ceph-mon[36954]: pgmap v8650: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:59:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:59:32 smithi100 ceph-mon[32599]: pgmap v8650: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:59:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:59:32 smithi167 ceph-mon[37140]: pgmap v8650: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:59:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:59:34 smithi138 ceph-mon[36954]: pgmap v8651: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:59:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:59:34 smithi100 ceph-mon[32599]: pgmap v8651: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:59:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:59:34 smithi167 ceph-mon[37140]: pgmap v8651: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:59:37.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:59:36 smithi138 ceph-mon[36954]: pgmap v8652: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:59:37.247 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:59:36 smithi100 ceph-mon[32599]: pgmap v8652: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:59:37.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:59:36 smithi167 ceph-mon[37140]: pgmap v8652: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:59:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:59:38 smithi138 ceph-mon[36954]: pgmap v8653: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:59:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:59:38 smithi100 ceph-mon[32599]: pgmap v8653: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:59:39.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:59:38 smithi167 ceph-mon[37140]: pgmap v8653: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:59:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:59:40 smithi138 ceph-mon[36954]: pgmap v8654: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:59:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:59:40 smithi100 ceph-mon[32599]: pgmap v8654: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:59:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:59:40 smithi167 ceph-mon[37140]: pgmap v8654: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:59:41.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:59:41.594+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:59:41.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:59:41.594+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:59:41.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:59:41.594+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:59:41.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:59:41.594+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:59:41.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:59:41.594+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:59:43.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:59:42 smithi138 ceph-mon[36954]: pgmap v8655: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:59:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:59:42 smithi100 ceph-mon[32599]: pgmap v8655: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:59:43.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:59:42 smithi167 ceph-mon[37140]: pgmap v8655: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:59:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:59:44 smithi138 ceph-mon[36954]: pgmap v8656: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:59:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:59:44 smithi100 ceph-mon[32599]: pgmap v8656: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:59:45.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:59:44 smithi167 ceph-mon[37140]: pgmap v8656: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:59:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:59:46 smithi138 ceph-mon[36954]: pgmap v8657: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:59:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:59:46 smithi100 ceph-mon[32599]: pgmap v8657: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:59:47.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:59:46 smithi167 ceph-mon[37140]: pgmap v8657: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:59:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:59:48 smithi138 ceph-mon[36954]: pgmap v8658: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:59:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:59:48 smithi100 ceph-mon[32599]: pgmap v8658: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:59:49.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:59:48 smithi167 ceph-mon[37140]: pgmap v8658: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:59:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:59:50 smithi138 ceph-mon[36954]: pgmap v8659: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:59:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:59:50 smithi100 ceph-mon[32599]: pgmap v8659: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:59:51.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:59:50 smithi167 ceph-mon[37140]: pgmap v8659: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:59:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:59:52 smithi138 ceph-mon[36954]: pgmap v8660: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:59:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:59:52 smithi100 ceph-mon[32599]: pgmap v8660: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:59:53.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:59:52 smithi167 ceph-mon[37140]: pgmap v8660: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T17:59:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:59:54 smithi138 ceph-mon[36954]: pgmap v8661: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:59:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:59:54 smithi100 ceph-mon[32599]: pgmap v8661: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:59:55.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:59:54 smithi167 ceph-mon[37140]: pgmap v8661: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T17:59:56.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:59:56.610+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T17:59:56.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:59:56.611+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T17:59:56.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:59:56.611+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T17:59:56.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:59:56.611+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T17:59:56.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T17:59:56.611+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T17:59:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:59:56 smithi138 ceph-mon[36954]: pgmap v8662: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:59:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:59:56 smithi100 ceph-mon[32599]: pgmap v8662: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:59:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:59:56 smithi167 ceph-mon[37140]: pgmap v8662: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T17:59:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 17:59:58 smithi138 ceph-mon[36954]: pgmap v8663: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:59:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 17:59:58 smithi100 ceph-mon[32599]: pgmap v8663: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T17:59:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 17:59:58 smithi167 ceph-mon[37140]: pgmap v8663: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T18:00:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:00 smithi100 ceph-mon[32599]: pgmap v8664: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:00:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:00 smithi100 ceph-mon[32599]: overall HEALTH_OK 2024-07-26T18:00:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:00 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:00:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:00 smithi167 ceph-mon[37140]: pgmap v8664: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:00:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:00 smithi167 ceph-mon[37140]: overall HEALTH_OK 2024-07-26T18:00:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:00 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:00:01.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:00 smithi138 ceph-mon[36954]: pgmap v8664: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:00:01.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:00 smithi138 ceph-mon[36954]: overall HEALTH_OK 2024-07-26T18:00:01.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:00 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:00:02.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:00:02.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:00:02.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:00:02.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:00:02.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:00:02.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:00:02.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:00:02.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:00:02.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:00:02.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:00:02.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:00:02.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:00:02.247 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:00:02.247 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:00:02.247 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:00:02.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:00:02.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:00:02.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:00:02.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:00:02.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:00:02.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:00:02.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:00:02.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:00:02.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:00:02.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:00:02.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:00:02.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:00:02.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:00:02.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:00:02.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:00:02.443 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:00:02.444 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:00:02.444 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:00:02.444 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:00:02.444 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:00:02.444 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:00:02.444 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:00:02.444 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:00:02.444 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:00:02.445 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:00:02.445 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:00:02.445 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:00:02.445 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:00:02.445 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:00:02.445 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:00:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:02 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T18:00:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:02 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T18:00:03.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:02 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T18:00:03.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:02 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T18:00:03.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:02 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T18:00:03.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:02 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T18:00:03.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:02 smithi100 ceph-mon[32599]: pgmap v8665: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:00:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:02 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T18:00:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:02 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T18:00:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:02 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T18:00:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:02 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T18:00:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:02 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T18:00:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:02 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T18:00:03.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:02 smithi167 ceph-mon[37140]: pgmap v8665: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:00:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:02 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T18:00:03.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:02 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T18:00:03.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:02 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T18:00:03.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:02 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T18:00:03.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:02 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T18:00:03.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:02 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T18:00:03.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:02 smithi138 ceph-mon[36954]: pgmap v8665: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:00:05.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:00:05.158+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T18:00:05.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:00:05.158+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c006690 con 0x7fe5a81087a0 2024-07-26T18:00:05.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:00:05.159+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 119 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T18:00:05.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:00:05.159+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 119 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c006690 con 0x7fe5a81087a0 2024-07-26T18:00:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:04 smithi100 ceph-mon[32599]: pgmap v8666: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:00:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:04 smithi167 ceph-mon[37140]: pgmap v8666: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:00:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:04 smithi138 ceph-mon[36954]: pgmap v8666: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:00:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:06 smithi100 ceph-mon[32599]: pgmap v8667: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:00:07.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:06 smithi167 ceph-mon[37140]: pgmap v8667: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:00:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:06 smithi138 ceph-mon[36954]: pgmap v8667: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:00:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:08 smithi100 ceph-mon[32599]: pgmap v8668: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:00:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:08 smithi100 ceph-mon[32599]: osdmap e588: 12 total, 12 up, 12 in 2024-07-26T18:00:09.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:08 smithi167 ceph-mon[37140]: pgmap v8668: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:00:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:08 smithi167 ceph-mon[37140]: osdmap e588: 12 total, 12 up, 12 in 2024-07-26T18:00:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:08 smithi138 ceph-mon[36954]: pgmap v8668: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:00:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:08 smithi138 ceph-mon[36954]: osdmap e588: 12 total, 12 up, 12 in 2024-07-26T18:00:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:10 smithi100 ceph-mon[32599]: pgmap v8670: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:00:11.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:10 smithi167 ceph-mon[37140]: pgmap v8670: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:00:11.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:10 smithi138 ceph-mon[36954]: pgmap v8670: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:00:11.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:00:11.620+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:00:11.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:00:11.621+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:00:11.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:00:11.621+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:00:11.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:00:11.621+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:00:11.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:00:11.621+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:00:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:12 smithi100 ceph-mon[32599]: pgmap v8671: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:00:13.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:12 smithi167 ceph-mon[37140]: pgmap v8671: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:00:13.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:12 smithi138 ceph-mon[36954]: pgmap v8671: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:00:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:14 smithi100 ceph-mon[32599]: pgmap v8672: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:00:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:14 smithi167 ceph-mon[37140]: pgmap v8672: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:00:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:14 smithi138 ceph-mon[36954]: pgmap v8672: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:00:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:16 smithi100 ceph-mon[32599]: pgmap v8673: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:00:17.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:16 smithi167 ceph-mon[37140]: pgmap v8673: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:00:17.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:16 smithi138 ceph-mon[36954]: pgmap v8673: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:00:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:18 smithi138 ceph-mon[36954]: pgmap v8674: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:00:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:18 smithi138 ceph-mon[36954]: osdmap e589: 12 total, 12 up, 12 in 2024-07-26T18:00:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:18 smithi100 ceph-mon[32599]: pgmap v8674: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:00:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:18 smithi100 ceph-mon[32599]: osdmap e589: 12 total, 12 up, 12 in 2024-07-26T18:00:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:18 smithi167 ceph-mon[37140]: pgmap v8674: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:00:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:18 smithi167 ceph-mon[37140]: osdmap e589: 12 total, 12 up, 12 in 2024-07-26T18:00:21.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:20 smithi138 ceph-mon[36954]: pgmap v8676: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:00:21.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:20 smithi100 ceph-mon[32599]: pgmap v8676: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:00:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:20 smithi167 ceph-mon[37140]: pgmap v8676: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:00:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:22 smithi138 ceph-mon[36954]: pgmap v8677: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:00:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:22 smithi100 ceph-mon[32599]: pgmap v8677: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:00:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:22 smithi167 ceph-mon[37140]: pgmap v8677: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:00:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:24 smithi138 ceph-mon[36954]: pgmap v8678: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:00:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:24 smithi100 ceph-mon[32599]: pgmap v8678: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:00:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:24 smithi167 ceph-mon[37140]: pgmap v8678: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:00:26.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:00:26.636+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:00:26.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:00:26.637+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:00:26.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:00:26.637+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:00:26.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:00:26.637+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:00:26.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:00:26.637+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:00:27.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:26 smithi138 ceph-mon[36954]: pgmap v8679: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:00:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:26 smithi100 ceph-mon[32599]: pgmap v8679: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:00:27.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:26 smithi167 ceph-mon[37140]: pgmap v8679: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:00:29.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:28 smithi138 ceph-mon[36954]: pgmap v8680: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:00:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:28 smithi100 ceph-mon[32599]: pgmap v8680: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:00:29.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:28 smithi167 ceph-mon[37140]: pgmap v8680: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:00:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:30 smithi138 ceph-mon[36954]: pgmap v8681: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:00:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:30 smithi100 ceph-mon[32599]: pgmap v8681: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:00:31.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:30 smithi167 ceph-mon[37140]: pgmap v8681: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:00:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:32 smithi138 ceph-mon[36954]: pgmap v8682: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:00:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:32 smithi100 ceph-mon[32599]: pgmap v8682: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:00:33.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:32 smithi167 ceph-mon[37140]: pgmap v8682: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:00:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:34 smithi138 ceph-mon[36954]: pgmap v8683: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:00:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:34 smithi100 ceph-mon[32599]: pgmap v8683: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:00:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:34 smithi167 ceph-mon[37140]: pgmap v8683: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:00:37.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:36 smithi138 ceph-mon[36954]: pgmap v8684: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:00:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:36 smithi100 ceph-mon[32599]: pgmap v8684: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:00:37.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:36 smithi167 ceph-mon[37140]: pgmap v8684: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:00:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:38 smithi138 ceph-mon[36954]: pgmap v8685: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T18:00:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:38 smithi100 ceph-mon[32599]: pgmap v8685: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T18:00:39.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:38 smithi167 ceph-mon[37140]: pgmap v8685: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T18:00:41.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:40 smithi138 ceph-mon[36954]: pgmap v8686: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:00:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:40 smithi100 ceph-mon[32599]: pgmap v8686: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:00:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:40 smithi167 ceph-mon[37140]: pgmap v8686: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:00:41.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:00:41.653+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:00:41.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:00:41.653+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:00:41.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:00:41.653+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:00:41.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:00:41.653+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:00:41.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:00:41.653+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:00:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:42 smithi138 ceph-mon[36954]: pgmap v8687: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:00:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:42 smithi100 ceph-mon[32599]: pgmap v8687: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:00:43.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:42 smithi167 ceph-mon[37140]: pgmap v8687: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:00:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:44 smithi138 ceph-mon[36954]: pgmap v8688: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:00:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:44 smithi100 ceph-mon[32599]: pgmap v8688: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:00:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:44 smithi167 ceph-mon[37140]: pgmap v8688: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:00:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:46 smithi138 ceph-mon[36954]: pgmap v8689: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:00:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:46 smithi100 ceph-mon[32599]: pgmap v8689: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:00:47.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:46 smithi167 ceph-mon[37140]: pgmap v8689: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:00:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:48 smithi138 ceph-mon[36954]: pgmap v8690: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T18:00:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:48 smithi100 ceph-mon[32599]: pgmap v8690: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T18:00:49.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:48 smithi167 ceph-mon[37140]: pgmap v8690: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T18:00:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:50 smithi138 ceph-mon[36954]: pgmap v8691: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:00:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:50 smithi100 ceph-mon[32599]: pgmap v8691: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:00:51.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:50 smithi167 ceph-mon[37140]: pgmap v8691: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:00:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:52 smithi100 ceph-mon[32599]: pgmap v8692: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:00:53.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:52 smithi167 ceph-mon[37140]: pgmap v8692: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:00:53.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:52 smithi138 ceph-mon[36954]: pgmap v8692: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:00:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:54 smithi100 ceph-mon[32599]: pgmap v8693: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:00:55.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:54 smithi167 ceph-mon[37140]: pgmap v8693: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:00:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:54 smithi138 ceph-mon[36954]: pgmap v8693: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:00:56.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:00:56.669+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:00:56.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:00:56.669+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:00:56.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:00:56.669+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:00:56.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:00:56.669+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:00:56.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:00:56.669+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:00:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:56 smithi100 ceph-mon[32599]: pgmap v8694: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:00:57.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:56 smithi167 ceph-mon[37140]: pgmap v8694: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:00:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:56 smithi138 ceph-mon[36954]: pgmap v8694: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:00:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:00:58 smithi100 ceph-mon[32599]: pgmap v8695: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T18:00:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:00:58 smithi167 ceph-mon[37140]: pgmap v8695: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T18:00:59.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:00:58 smithi138 ceph-mon[36954]: pgmap v8695: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T18:01:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:01:00 smithi100 ceph-mon[32599]: pgmap v8696: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:01:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:01:00 smithi167 ceph-mon[37140]: pgmap v8696: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:01:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:01:00 smithi138 ceph-mon[36954]: pgmap v8696: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:01:02.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:01:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:01:02.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:01:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:01:02.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:01:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:01:02.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:01:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:01:02.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:01:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:01:02.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:01:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:01:02.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:01:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:01:02.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:01:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:01:02.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:01:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:01:02.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:01:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:01:02.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:01:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:01:02.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:01:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:01:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:01:02 smithi100 ceph-mon[32599]: pgmap v8697: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:01:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:01:02 smithi167 ceph-mon[37140]: pgmap v8697: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:01:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:01:02 smithi138 ceph-mon[36954]: pgmap v8697: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:01:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:01:04 smithi100 ceph-mon[32599]: pgmap v8698: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:01:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:01:04 smithi167 ceph-mon[37140]: pgmap v8698: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:01:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:01:04 smithi138 ceph-mon[36954]: pgmap v8698: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:01:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:01:06 smithi100 ceph-mon[32599]: pgmap v8699: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:01:07.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:01:06 smithi167 ceph-mon[37140]: pgmap v8699: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:01:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:01:06 smithi138 ceph-mon[36954]: pgmap v8699: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:01:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:01:08 smithi100 ceph-mon[32599]: pgmap v8700: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:01:09.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:01:08 smithi100 ceph-mon[32599]: osdmap e590: 12 total, 12 up, 12 in 2024-07-26T18:01:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:01:08 smithi167 ceph-mon[37140]: pgmap v8700: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:01:09.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:01:08 smithi167 ceph-mon[37140]: osdmap e590: 12 total, 12 up, 12 in 2024-07-26T18:01:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:01:08 smithi138 ceph-mon[36954]: pgmap v8700: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:01:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:01:08 smithi138 ceph-mon[36954]: osdmap e590: 12 total, 12 up, 12 in 2024-07-26T18:01:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:01:10 smithi100 ceph-mon[32599]: pgmap v8702: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:01:11.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:01:10 smithi167 ceph-mon[37140]: pgmap v8702: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:01:11.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:01:10 smithi138 ceph-mon[36954]: pgmap v8702: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:01:11.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:01:11.670+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:01:11.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:01:11.670+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:01:11.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:01:11.670+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:01:11.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:01:11.670+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:01:11.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:01:11.670+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:01:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:01:12 smithi100 ceph-mon[32599]: pgmap v8703: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:01:13.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:01:12 smithi167 ceph-mon[37140]: pgmap v8703: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:01:13.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:01:12 smithi138 ceph-mon[36954]: pgmap v8703: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:01:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:01:14 smithi100 ceph-mon[32599]: pgmap v8704: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:01:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:01:14 smithi167 ceph-mon[37140]: pgmap v8704: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:01:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:01:14 smithi138 ceph-mon[36954]: pgmap v8704: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:01:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:01:16 smithi100 ceph-mon[32599]: pgmap v8705: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:01:17.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:01:16 smithi167 ceph-mon[37140]: pgmap v8705: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:01:17.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:01:16 smithi138 ceph-mon[36954]: pgmap v8705: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:01:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:01:18 smithi138 ceph-mon[36954]: pgmap v8706: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:01:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:01:18 smithi138 ceph-mon[36954]: osdmap e591: 12 total, 12 up, 12 in 2024-07-26T18:01:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:01:18 smithi100 ceph-mon[32599]: pgmap v8706: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:01:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:01:18 smithi100 ceph-mon[32599]: osdmap e591: 12 total, 12 up, 12 in 2024-07-26T18:01:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:01:18 smithi167 ceph-mon[37140]: pgmap v8706: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:01:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:01:18 smithi167 ceph-mon[37140]: osdmap e591: 12 total, 12 up, 12 in 2024-07-26T18:01:21.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:01:20 smithi138 ceph-mon[36954]: pgmap v8708: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:01:21.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:01:20 smithi100 ceph-mon[32599]: pgmap v8708: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:01:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:01:20 smithi167 ceph-mon[37140]: pgmap v8708: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:01:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:01:22 smithi138 ceph-mon[36954]: pgmap v8709: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:01:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:01:22 smithi100 ceph-mon[32599]: pgmap v8709: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:01:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:01:22 smithi167 ceph-mon[37140]: pgmap v8709: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:01:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:01:24 smithi138 ceph-mon[36954]: pgmap v8710: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:01:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:01:24 smithi100 ceph-mon[32599]: pgmap v8710: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:01:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:01:24 smithi167 ceph-mon[37140]: pgmap v8710: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:01:26.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:01:26.686+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:01:26.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:01:26.687+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:01:26.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:01:26.687+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:01:26.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:01:26.687+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:01:26.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:01:26.687+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:01:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:01:26 smithi100 ceph-mon[32599]: pgmap v8711: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:01:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:01:26 smithi138 ceph-mon[36954]: pgmap v8711: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:01:27.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:01:26 smithi167 ceph-mon[37140]: pgmap v8711: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:01:29.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:01:28 smithi138 ceph-mon[36954]: pgmap v8712: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:01:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:01:28 smithi100 ceph-mon[32599]: pgmap v8712: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:01:29.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:01:28 smithi167 ceph-mon[37140]: pgmap v8712: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:01:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:01:30 smithi138 ceph-mon[36954]: pgmap v8713: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:01:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:01:30 smithi100 ceph-mon[32599]: pgmap v8713: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:01:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:01:30 smithi167 ceph-mon[37140]: pgmap v8713: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:01:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:01:32 smithi138 ceph-mon[36954]: pgmap v8714: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:01:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:01:32 smithi100 ceph-mon[32599]: pgmap v8714: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:01:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:01:32 smithi167 ceph-mon[37140]: pgmap v8714: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:01:35.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:01:34 smithi138 ceph-mon[36954]: pgmap v8715: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:01:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:01:34 smithi100 ceph-mon[32599]: pgmap v8715: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:01:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:01:34 smithi167 ceph-mon[37140]: pgmap v8715: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:01:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:01:36 smithi138 ceph-mon[36954]: pgmap v8716: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:01:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:01:36 smithi100 ceph-mon[32599]: pgmap v8716: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:01:37.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:01:36 smithi167 ceph-mon[37140]: pgmap v8716: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:01:39.195 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:01:38 smithi138 ceph-mon[36954]: pgmap v8717: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T18:01:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:01:38 smithi100 ceph-mon[32599]: pgmap v8717: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T18:01:39.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:01:38 smithi167 ceph-mon[37140]: pgmap v8717: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T18:01:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:01:40 smithi100 ceph-mon[32599]: pgmap v8718: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:01:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:01:40 smithi167 ceph-mon[37140]: pgmap v8718: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:01:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:01:40 smithi138 ceph-mon[36954]: pgmap v8718: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:01:41.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:01:41.702+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:01:41.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:01:41.703+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:01:41.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:01:41.703+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:01:41.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:01:41.703+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:01:41.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:01:41.703+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:01:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:01:42 smithi100 ceph-mon[32599]: pgmap v8719: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:01:43.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:01:42 smithi167 ceph-mon[37140]: pgmap v8719: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:01:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:01:42 smithi138 ceph-mon[36954]: pgmap v8719: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:01:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:01:44 smithi100 ceph-mon[32599]: pgmap v8720: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:01:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:01:44 smithi167 ceph-mon[37140]: pgmap v8720: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:01:45.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:01:44 smithi138 ceph-mon[36954]: pgmap v8720: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:01:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:01:46 smithi100 ceph-mon[32599]: pgmap v8721: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:01:47.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:01:46 smithi167 ceph-mon[37140]: pgmap v8721: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:01:47.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:01:46 smithi138 ceph-mon[36954]: pgmap v8721: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:01:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:01:48 smithi100 ceph-mon[32599]: pgmap v8722: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T18:01:49.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:01:48 smithi167 ceph-mon[37140]: pgmap v8722: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T18:01:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:01:48 smithi138 ceph-mon[36954]: pgmap v8722: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T18:01:51.243 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:01:50 smithi138 ceph-mon[36954]: pgmap v8723: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:01:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:01:50 smithi100 ceph-mon[32599]: pgmap v8723: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:01:51.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:01:50 smithi167 ceph-mon[37140]: pgmap v8723: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:01:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:01:52 smithi100 ceph-mon[32599]: pgmap v8724: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:01:53.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:01:52 smithi167 ceph-mon[37140]: pgmap v8724: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:01:53.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:01:52 smithi138 ceph-mon[36954]: pgmap v8724: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:01:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:01:54 smithi100 ceph-mon[32599]: pgmap v8725: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:01:55.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:01:54 smithi167 ceph-mon[37140]: pgmap v8725: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:01:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:01:54 smithi138 ceph-mon[36954]: pgmap v8725: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:01:56.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:01:56.718+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:01:56.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:01:56.719+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:01:56.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:01:56.719+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:01:56.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:01:56.719+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:01:56.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:01:56.719+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:01:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:01:56 smithi100 ceph-mon[32599]: pgmap v8726: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:01:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:01:56 smithi167 ceph-mon[37140]: pgmap v8726: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:01:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:01:56 smithi138 ceph-mon[36954]: pgmap v8726: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:01:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:01:58 smithi100 ceph-mon[32599]: pgmap v8727: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T18:01:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:01:58 smithi167 ceph-mon[37140]: pgmap v8727: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T18:01:59.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:01:58 smithi138 ceph-mon[36954]: pgmap v8727: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T18:02:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:02:00 smithi100 ceph-mon[32599]: pgmap v8728: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:02:01.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:02:00 smithi167 ceph-mon[37140]: pgmap v8728: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:02:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:02:00 smithi138 ceph-mon[36954]: pgmap v8728: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:02:02.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:02:01 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:02:02.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:02:01 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:02:02.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:02:01 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:02:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:02:02 smithi100 ceph-mon[32599]: pgmap v8729: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:02:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:02:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:02:03.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:02:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:02:03.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:02:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:02:03.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:02:02 smithi167 ceph-mon[37140]: pgmap v8729: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:02:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:02:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:02:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:02:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:02:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:02:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:02:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:02:02 smithi138 ceph-mon[36954]: pgmap v8729: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:02:03.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:02:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:02:03.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:02:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:02:03.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:02:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:02:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:02:04 smithi100 ceph-mon[32599]: pgmap v8730: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:02:05.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:02:04 smithi167 ceph-mon[37140]: pgmap v8730: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:02:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:02:04 smithi138 ceph-mon[36954]: pgmap v8730: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:02:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:02:06 smithi100 ceph-mon[32599]: pgmap v8731: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:02:07.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:02:06 smithi167 ceph-mon[37140]: pgmap v8731: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:02:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:02:06 smithi138 ceph-mon[36954]: pgmap v8731: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:02:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:02:08 smithi167 ceph-mon[37140]: pgmap v8732: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:02:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:02:08 smithi167 ceph-mon[37140]: osdmap e592: 12 total, 12 up, 12 in 2024-07-26T18:02:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:02:08 smithi138 ceph-mon[36954]: pgmap v8732: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:02:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:02:08 smithi138 ceph-mon[36954]: osdmap e592: 12 total, 12 up, 12 in 2024-07-26T18:02:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:02:08 smithi100 ceph-mon[32599]: pgmap v8732: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:02:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:02:08 smithi100 ceph-mon[32599]: osdmap e592: 12 total, 12 up, 12 in 2024-07-26T18:02:11.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:02:10 smithi167 ceph-mon[37140]: pgmap v8734: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:02:11.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:02:10 smithi138 ceph-mon[36954]: pgmap v8734: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:02:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:02:10 smithi100 ceph-mon[32599]: pgmap v8734: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:02:11.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:02:11.734+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:02:11.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:02:11.735+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:02:11.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:02:11.735+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:02:11.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:02:11.735+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:02:11.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:02:11.735+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:02:13.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:02:13 smithi167 ceph-mon[37140]: pgmap v8735: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:02:13.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:02:13 smithi138 ceph-mon[36954]: pgmap v8735: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:02:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:02:13 smithi100 ceph-mon[32599]: pgmap v8735: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:02:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:02:15 smithi167 ceph-mon[37140]: pgmap v8736: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:02:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:02:15 smithi138 ceph-mon[36954]: pgmap v8736: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:02:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:02:15 smithi100 ceph-mon[32599]: pgmap v8736: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:02:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:02:17 smithi167 ceph-mon[37140]: pgmap v8737: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:02:17.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:02:17 smithi138 ceph-mon[36954]: pgmap v8737: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:02:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:02:17 smithi100 ceph-mon[32599]: pgmap v8737: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:02:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:02:18 smithi138 ceph-mon[36954]: pgmap v8738: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:02:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:02:18 smithi138 ceph-mon[36954]: osdmap e593: 12 total, 12 up, 12 in 2024-07-26T18:02:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:02:18 smithi100 ceph-mon[32599]: pgmap v8738: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:02:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:02:18 smithi100 ceph-mon[32599]: osdmap e593: 12 total, 12 up, 12 in 2024-07-26T18:02:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:02:18 smithi167 ceph-mon[37140]: pgmap v8738: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:02:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:02:18 smithi167 ceph-mon[37140]: osdmap e593: 12 total, 12 up, 12 in 2024-07-26T18:02:21.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:02:20 smithi138 ceph-mon[36954]: pgmap v8740: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:02:21.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:02:20 smithi100 ceph-mon[32599]: pgmap v8740: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:02:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:02:20 smithi167 ceph-mon[37140]: pgmap v8740: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:02:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:02:22 smithi138 ceph-mon[36954]: pgmap v8741: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:02:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:02:22 smithi100 ceph-mon[32599]: pgmap v8741: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:02:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:02:22 smithi167 ceph-mon[37140]: pgmap v8741: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:02:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:02:24 smithi138 ceph-mon[36954]: pgmap v8742: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:02:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:02:24 smithi100 ceph-mon[32599]: pgmap v8742: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:02:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:02:24 smithi167 ceph-mon[37140]: pgmap v8742: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:02:26.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:02:26.750+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:02:26.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:02:26.750+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:02:26.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:02:26.750+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:02:26.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:02:26.750+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:02:26.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:02:26.750+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:02:27.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:02:26 smithi100 ceph-mon[32599]: pgmap v8743: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:02:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:02:26 smithi138 ceph-mon[36954]: pgmap v8743: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:02:27.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:02:26 smithi167 ceph-mon[37140]: pgmap v8743: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:02:29.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:02:28 smithi138 ceph-mon[36954]: pgmap v8744: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:02:29.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:02:28 smithi100 ceph-mon[32599]: pgmap v8744: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:02:29.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:02:28 smithi167 ceph-mon[37140]: pgmap v8744: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:02:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:02:30 smithi100 ceph-mon[32599]: pgmap v8745: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:02:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:02:30 smithi167 ceph-mon[37140]: pgmap v8745: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:02:31.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:02:30 smithi138 ceph-mon[36954]: pgmap v8745: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:02:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:02:32 smithi100 ceph-mon[32599]: pgmap v8746: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:02:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:02:32 smithi167 ceph-mon[37140]: pgmap v8746: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:02:33.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:02:32 smithi138 ceph-mon[36954]: pgmap v8746: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:02:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:02:34 smithi100 ceph-mon[32599]: pgmap v8747: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:02:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:02:34 smithi167 ceph-mon[37140]: pgmap v8747: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:02:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:02:34 smithi138 ceph-mon[36954]: pgmap v8747: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:02:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:02:36 smithi100 ceph-mon[32599]: pgmap v8748: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:02:37.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:02:36 smithi167 ceph-mon[37140]: pgmap v8748: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:02:37.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:02:36 smithi138 ceph-mon[36954]: pgmap v8748: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:02:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:02:38 smithi100 ceph-mon[32599]: pgmap v8749: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:02:39.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:02:38 smithi167 ceph-mon[37140]: pgmap v8749: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:02:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:02:38 smithi138 ceph-mon[36954]: pgmap v8749: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:02:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:02:40 smithi100 ceph-mon[32599]: pgmap v8750: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:02:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:02:40 smithi167 ceph-mon[37140]: pgmap v8750: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:02:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:02:40 smithi138 ceph-mon[36954]: pgmap v8750: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:02:41.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:02:41.762+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:02:41.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:02:41.764+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:02:41.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:02:41.764+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:02:41.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:02:41.764+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:02:41.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:02:41.764+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:02:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:02:42 smithi100 ceph-mon[32599]: pgmap v8751: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:02:43.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:02:42 smithi167 ceph-mon[37140]: pgmap v8751: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:02:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:02:42 smithi138 ceph-mon[36954]: pgmap v8751: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:02:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:02:44 smithi100 ceph-mon[32599]: pgmap v8752: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:02:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:02:44 smithi167 ceph-mon[37140]: pgmap v8752: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:02:45.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:02:44 smithi138 ceph-mon[36954]: pgmap v8752: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:02:47.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:02:46 smithi100 ceph-mon[32599]: pgmap v8753: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:02:47.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:02:46 smithi167 ceph-mon[37140]: pgmap v8753: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:02:47.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:02:46 smithi138 ceph-mon[36954]: pgmap v8753: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:02:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:02:48 smithi100 ceph-mon[32599]: pgmap v8754: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:02:49.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:02:48 smithi167 ceph-mon[37140]: pgmap v8754: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:02:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:02:48 smithi138 ceph-mon[36954]: pgmap v8754: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:02:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:02:50 smithi100 ceph-mon[32599]: pgmap v8755: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:02:51.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:02:50 smithi167 ceph-mon[37140]: pgmap v8755: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:02:51.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:02:50 smithi138 ceph-mon[36954]: pgmap v8755: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:02:53.243 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:02:52 smithi138 ceph-mon[36954]: pgmap v8756: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:02:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:02:52 smithi100 ceph-mon[32599]: pgmap v8756: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:02:53.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:02:52 smithi167 ceph-mon[37140]: pgmap v8756: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:02:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:02:54 smithi100 ceph-mon[32599]: pgmap v8757: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:02:55.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:02:54 smithi167 ceph-mon[37140]: pgmap v8757: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:02:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:02:54 smithi138 ceph-mon[36954]: pgmap v8757: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:02:56.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:02:56.780+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:02:56.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:02:56.780+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:02:56.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:02:56.780+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:02:56.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:02:56.780+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:02:56.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:02:56.780+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:02:57.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:02:56 smithi100 ceph-mon[32599]: pgmap v8758: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:02:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:02:56 smithi167 ceph-mon[37140]: pgmap v8758: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:02:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:02:56 smithi138 ceph-mon[36954]: pgmap v8758: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:02:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:02:58 smithi100 ceph-mon[32599]: pgmap v8759: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:02:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:02:58 smithi167 ceph-mon[37140]: pgmap v8759: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:02:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:02:58 smithi138 ceph-mon[36954]: pgmap v8759: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:03:01.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:00 smithi167 ceph-mon[37140]: pgmap v8760: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:03:01.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:00 smithi138 ceph-mon[36954]: pgmap v8760: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:03:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:00 smithi100 ceph-mon[32599]: pgmap v8760: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:03:03.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:03:03.044+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 msgr2=0x7fe5940b5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-07-26T18:03:03.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:03:03.044+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 msgr2=0x7fe5940b5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-07-26T18:03:03.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:03:03.044+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 0x7fe5940b5b30 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fe590004ee0 tx=0x7fe59005a040 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:03:03.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:03:03.044+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 0x7fe5940b5b30 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fe590004ee0 tx=0x7fe59005a040 comp rx=0 tx=0).stop 2024-07-26T18:03:03.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:03:03.044+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 msgr2=0x7fe5940b5b30 unknown :-1 s=STATE_CLOSED l=1).tick see no progress in more than 10000000 us during connecting to v2:172.21.15.100:6800/3937009534, fault. 2024-07-26T18:03:03.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:03:03.044+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 msgr2=0x7fe5940b5b30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-07-26T18:03:03.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:03:03.044+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 0x7fe5940b5b30 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T18:03:03.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:03:03.044+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T18:03:03.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:03:03.044+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:03:03.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:03:03.044+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fe590004d70 tx=0x7fe5900ab000 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T18:03:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:02 smithi100 ceph-mon[32599]: pgmap v8761: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:03:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:03:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:02 smithi167 ceph-mon[37140]: pgmap v8761: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:03:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:03:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:02 smithi138 ceph-mon[36954]: pgmap v8761: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:03:03.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:03:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:04 smithi100 ceph-mon[32599]: pgmap v8762: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:03:04.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:03:04.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:03:04.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:03:04.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:03:04.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:03:04.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:03:04.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:03:04.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:03:04.996 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:03:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:04 smithi167 ceph-mon[37140]: pgmap v8762: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:03:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:03:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:03:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:03:05.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:03:05.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:03:05.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:03:05.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:03:05.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:03:05.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:03:05.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:03:05.183+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c006690 con 0x7fe5a81087a0 2024-07-26T18:03:05.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:03:05.183+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T18:03:05.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:03:05.183+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 120 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T18:03:05.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:03:05.183+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 120 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c006690 con 0x7fe5a81087a0 2024-07-26T18:03:05.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:04 smithi138 ceph-mon[36954]: pgmap v8762: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:03:05.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:03:05.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:03:05.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:03:05.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:03:05.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:03:05.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:03:05.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:03:05.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:03:05.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:03:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:06 smithi100 ceph-mon[32599]: pgmap v8763: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:03:07.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:06 smithi167 ceph-mon[37140]: pgmap v8763: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:03:07.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:06 smithi138 ceph-mon[36954]: pgmap v8763: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:03:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:08 smithi100 ceph-mon[32599]: pgmap v8764: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:03:08.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:08 smithi100 ceph-mon[32599]: osdmap e594: 12 total, 12 up, 12 in 2024-07-26T18:03:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:08 smithi167 ceph-mon[37140]: pgmap v8764: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:03:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:08 smithi167 ceph-mon[37140]: osdmap e594: 12 total, 12 up, 12 in 2024-07-26T18:03:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:08 smithi138 ceph-mon[36954]: pgmap v8764: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:03:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:08 smithi138 ceph-mon[36954]: osdmap e594: 12 total, 12 up, 12 in 2024-07-26T18:03:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:10 smithi100 ceph-mon[32599]: pgmap v8766: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:03:11.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:10 smithi167 ceph-mon[37140]: pgmap v8766: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:03:11.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:10 smithi138 ceph-mon[36954]: pgmap v8766: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:03:11.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:03:11.789+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:03:11.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:03:11.789+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:03:11.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:03:11.789+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:03:11.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:03:11.789+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:03:11.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:03:11.789+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:03:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:12 smithi100 ceph-mon[32599]: pgmap v8767: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:03:13.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:12 smithi167 ceph-mon[37140]: pgmap v8767: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:03:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:12 smithi138 ceph-mon[36954]: pgmap v8767: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:03:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:14 smithi100 ceph-mon[32599]: pgmap v8768: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:03:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:14 smithi167 ceph-mon[37140]: pgmap v8768: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:03:15.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:14 smithi138 ceph-mon[36954]: pgmap v8768: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:03:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:16 smithi100 ceph-mon[32599]: pgmap v8769: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:03:17.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:16 smithi167 ceph-mon[37140]: pgmap v8769: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:03:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:16 smithi138 ceph-mon[36954]: pgmap v8769: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:03:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:18 smithi138 ceph-mon[36954]: pgmap v8770: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:03:19.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:18 smithi138 ceph-mon[36954]: osdmap e595: 12 total, 12 up, 12 in 2024-07-26T18:03:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:18 smithi100 ceph-mon[32599]: pgmap v8770: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:03:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:18 smithi100 ceph-mon[32599]: osdmap e595: 12 total, 12 up, 12 in 2024-07-26T18:03:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:18 smithi167 ceph-mon[37140]: pgmap v8770: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:03:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:18 smithi167 ceph-mon[37140]: osdmap e595: 12 total, 12 up, 12 in 2024-07-26T18:03:21.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:20 smithi138 ceph-mon[36954]: pgmap v8772: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:03:21.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:20 smithi100 ceph-mon[32599]: pgmap v8772: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:03:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:20 smithi167 ceph-mon[37140]: pgmap v8772: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:03:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:22 smithi100 ceph-mon[32599]: pgmap v8773: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:03:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:22 smithi167 ceph-mon[37140]: pgmap v8773: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:03:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:22 smithi138 ceph-mon[36954]: pgmap v8773: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:03:25.243 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:24 smithi138 ceph-mon[36954]: pgmap v8774: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:03:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:24 smithi100 ceph-mon[32599]: pgmap v8774: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:03:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:24 smithi167 ceph-mon[37140]: pgmap v8774: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:03:26.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:03:26.791+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:03:26.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:03:26.792+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:03:26.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:03:26.792+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:03:26.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:03:26.792+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:03:26.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:03:26.792+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:03:27.047 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:26 smithi100 ceph-mon[32599]: pgmap v8775: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:03:27.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:26 smithi167 ceph-mon[37140]: pgmap v8775: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:03:27.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:26 smithi138 ceph-mon[36954]: pgmap v8775: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:03:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:28 smithi100 ceph-mon[32599]: pgmap v8776: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:03:29.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:28 smithi167 ceph-mon[37140]: pgmap v8776: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:03:29.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:28 smithi138 ceph-mon[36954]: pgmap v8776: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:03:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:30 smithi100 ceph-mon[32599]: pgmap v8777: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:03:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:30 smithi167 ceph-mon[37140]: pgmap v8777: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:03:31.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:30 smithi138 ceph-mon[36954]: pgmap v8777: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:03:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:32 smithi100 ceph-mon[32599]: pgmap v8778: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:03:33.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:32 smithi167 ceph-mon[37140]: pgmap v8778: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:03:33.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:32 smithi138 ceph-mon[36954]: pgmap v8778: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:03:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:34 smithi100 ceph-mon[32599]: pgmap v8779: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:03:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:34 smithi167 ceph-mon[37140]: pgmap v8779: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:03:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:34 smithi138 ceph-mon[36954]: pgmap v8779: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:03:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:36 smithi100 ceph-mon[32599]: pgmap v8780: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:03:37.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:36 smithi167 ceph-mon[37140]: pgmap v8780: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:03:37.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:36 smithi138 ceph-mon[36954]: pgmap v8780: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:03:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:38 smithi100 ceph-mon[32599]: pgmap v8781: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:03:39.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:38 smithi167 ceph-mon[37140]: pgmap v8781: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:03:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:38 smithi138 ceph-mon[36954]: pgmap v8781: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:03:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:40 smithi100 ceph-mon[32599]: pgmap v8782: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:03:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:40 smithi167 ceph-mon[37140]: pgmap v8782: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:03:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:40 smithi138 ceph-mon[36954]: pgmap v8782: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:03:41.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:03:41.808+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:03:41.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:03:41.808+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:03:41.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:03:41.808+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:03:41.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:03:41.808+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:03:41.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:03:41.808+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:03:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:42 smithi100 ceph-mon[32599]: pgmap v8783: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:03:43.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:42 smithi167 ceph-mon[37140]: pgmap v8783: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:03:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:42 smithi138 ceph-mon[36954]: pgmap v8783: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:03:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:44 smithi100 ceph-mon[32599]: pgmap v8784: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:03:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:44 smithi167 ceph-mon[37140]: pgmap v8784: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:03:45.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:44 smithi138 ceph-mon[36954]: pgmap v8784: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:03:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:46 smithi100 ceph-mon[32599]: pgmap v8785: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:03:47.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:46 smithi167 ceph-mon[37140]: pgmap v8785: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:03:47.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:46 smithi138 ceph-mon[36954]: pgmap v8785: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:03:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:48 smithi100 ceph-mon[32599]: pgmap v8786: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:03:49.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:48 smithi167 ceph-mon[37140]: pgmap v8786: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:03:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:48 smithi138 ceph-mon[36954]: pgmap v8786: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:03:50.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:03:50.190+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T18:03:50.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:03:50.190+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c006690 con 0x7fe5a81087a0 2024-07-26T18:03:50.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:03:50.190+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 121 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T18:03:50.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:03:50.191+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 121 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe57c006690 con 0x7fe5a81087a0 2024-07-26T18:03:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:50 smithi100 ceph-mon[32599]: pgmap v8787: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:03:51.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:50 smithi167 ceph-mon[37140]: pgmap v8787: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:03:51.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:50 smithi138 ceph-mon[36954]: pgmap v8787: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:03:53.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:52 smithi167 ceph-mon[37140]: pgmap v8788: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:03:53.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:52 smithi138 ceph-mon[36954]: pgmap v8788: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:03:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:52 smithi100 ceph-mon[32599]: pgmap v8788: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:03:55.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:54 smithi167 ceph-mon[37140]: pgmap v8789: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:03:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:54 smithi138 ceph-mon[36954]: pgmap v8789: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:03:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:54 smithi100 ceph-mon[32599]: pgmap v8789: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:03:56.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:03:56.809+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:03:56.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:03:56.809+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:03:56.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:03:56.809+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:03:56.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:03:56.809+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:03:56.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:03:56.809+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:03:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:57 smithi100 ceph-mon[32599]: pgmap v8790: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:03:57.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:57 smithi167 ceph-mon[37140]: pgmap v8790: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:03:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:57 smithi138 ceph-mon[36954]: pgmap v8790: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:03:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:03:59 smithi167 ceph-mon[37140]: pgmap v8791: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:03:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:03:59 smithi138 ceph-mon[36954]: pgmap v8791: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:03:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:03:59 smithi100 ceph-mon[32599]: pgmap v8791: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:04:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:01 smithi167 ceph-mon[37140]: pgmap v8792: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:04:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:01 smithi138 ceph-mon[36954]: pgmap v8792: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:04:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:01 smithi100 ceph-mon[32599]: pgmap v8792: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:04:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:03 smithi167 ceph-mon[37140]: pgmap v8793: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:04:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:03 smithi138 ceph-mon[36954]: pgmap v8793: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:04:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:03 smithi100 ceph-mon[32599]: pgmap v8793: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:04:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:05 smithi167 ceph-mon[37140]: pgmap v8794: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:04:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:05 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:04:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:05 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:04:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:05 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:04:05.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:05 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:04:05.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:05 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:04:05.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:05 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:04:05.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:05 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:04:05.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:05 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:04:05.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:05 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:04:05.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:05 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:04:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:05 smithi138 ceph-mon[36954]: pgmap v8794: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:04:05.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:05 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:04:05.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:05 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:04:05.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:05 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:04:05.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:05 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:04:05.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:05 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:04:05.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:05 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:04:05.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:05 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:04:05.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:05 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:04:05.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:05 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:04:05.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:05 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:04:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:05 smithi100 ceph-mon[32599]: pgmap v8794: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:04:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:05 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:04:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:05 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:04:05.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:05 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:04:05.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:05 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:04:05.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:05 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:04:05.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:05 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:04:05.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:05 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:04:05.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:05 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:04:05.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:05 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:04:05.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:05 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:04:07.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:07 smithi167 ceph-mon[37140]: pgmap v8795: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:04:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:07 smithi138 ceph-mon[36954]: pgmap v8795: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:04:07.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:07 smithi100 ceph-mon[32599]: pgmap v8795: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:04:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:09 smithi138 ceph-mon[36954]: pgmap v8796: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:04:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:09 smithi138 ceph-mon[36954]: osdmap e596: 12 total, 12 up, 12 in 2024-07-26T18:04:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:09 smithi100 ceph-mon[32599]: pgmap v8796: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:04:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:09 smithi100 ceph-mon[32599]: osdmap e596: 12 total, 12 up, 12 in 2024-07-26T18:04:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:09 smithi167 ceph-mon[37140]: pgmap v8796: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:04:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:09 smithi167 ceph-mon[37140]: osdmap e596: 12 total, 12 up, 12 in 2024-07-26T18:04:11.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:11 smithi138 ceph-mon[36954]: pgmap v8798: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:04:11.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:11 smithi100 ceph-mon[32599]: pgmap v8798: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:04:11.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:11 smithi167 ceph-mon[37140]: pgmap v8798: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:04:11.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:04:11.815+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:04:11.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:04:11.816+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:04:11.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:04:11.816+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:04:11.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:04:11.816+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:04:11.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:04:11.816+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:04:13.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:13 smithi138 ceph-mon[36954]: pgmap v8799: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:04:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:13 smithi100 ceph-mon[32599]: pgmap v8799: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:04:13.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:13 smithi167 ceph-mon[37140]: pgmap v8799: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:04:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:15 smithi138 ceph-mon[36954]: pgmap v8800: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:04:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:15 smithi100 ceph-mon[32599]: pgmap v8800: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:04:15.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:15 smithi167 ceph-mon[37140]: pgmap v8800: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:04:17.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:17 smithi138 ceph-mon[36954]: pgmap v8801: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:04:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:17 smithi100 ceph-mon[32599]: pgmap v8801: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:04:17.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:17 smithi167 ceph-mon[37140]: pgmap v8801: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:04:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:18 smithi100 ceph-mon[32599]: pgmap v8802: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:04:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:18 smithi100 ceph-mon[32599]: osdmap e597: 12 total, 12 up, 12 in 2024-07-26T18:04:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:18 smithi167 ceph-mon[37140]: pgmap v8802: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:04:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:18 smithi167 ceph-mon[37140]: osdmap e597: 12 total, 12 up, 12 in 2024-07-26T18:04:19.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:18 smithi138 ceph-mon[36954]: pgmap v8802: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:04:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:18 smithi138 ceph-mon[36954]: osdmap e597: 12 total, 12 up, 12 in 2024-07-26T18:04:21.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:20 smithi100 ceph-mon[32599]: pgmap v8804: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:04:21.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:20 smithi167 ceph-mon[37140]: pgmap v8804: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:04:21.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:20 smithi138 ceph-mon[36954]: pgmap v8804: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:04:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:22 smithi100 ceph-mon[32599]: pgmap v8805: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:04:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:22 smithi167 ceph-mon[37140]: pgmap v8805: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:04:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:22 smithi138 ceph-mon[36954]: pgmap v8805: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:04:25.243 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:24 smithi138 ceph-mon[36954]: pgmap v8806: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:04:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:24 smithi100 ceph-mon[32599]: pgmap v8806: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:04:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:24 smithi167 ceph-mon[37140]: pgmap v8806: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:04:26.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:04:26.831+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:04:26.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:04:26.832+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:04:26.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:04:26.832+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:04:26.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:04:26.832+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:04:26.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:04:26.832+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:04:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:26 smithi100 ceph-mon[32599]: pgmap v8807: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:04:27.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:26 smithi167 ceph-mon[37140]: pgmap v8807: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:04:27.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:26 smithi138 ceph-mon[36954]: pgmap v8807: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:04:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:28 smithi100 ceph-mon[32599]: pgmap v8808: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:04:29.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:28 smithi167 ceph-mon[37140]: pgmap v8808: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:04:29.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:28 smithi138 ceph-mon[36954]: pgmap v8808: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:04:31.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:30 smithi100 ceph-mon[32599]: pgmap v8809: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:04:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:30 smithi167 ceph-mon[37140]: pgmap v8809: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:04:31.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:30 smithi138 ceph-mon[36954]: pgmap v8809: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:04:33.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:32 smithi100 ceph-mon[32599]: pgmap v8810: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:04:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:32 smithi167 ceph-mon[37140]: pgmap v8810: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:04:33.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:32 smithi138 ceph-mon[36954]: pgmap v8810: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:04:35.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:34 smithi100 ceph-mon[32599]: pgmap v8811: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:04:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:34 smithi167 ceph-mon[37140]: pgmap v8811: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:04:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:34 smithi138 ceph-mon[36954]: pgmap v8811: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:04:37.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:36 smithi100 ceph-mon[32599]: pgmap v8812: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:04:37.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:36 smithi167 ceph-mon[37140]: pgmap v8812: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:04:37.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:36 smithi138 ceph-mon[36954]: pgmap v8812: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:04:39.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:38 smithi167 ceph-mon[37140]: pgmap v8813: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:04:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:38 smithi138 ceph-mon[36954]: pgmap v8813: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:04:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:38 smithi100 ceph-mon[32599]: pgmap v8813: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:04:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:41 smithi167 ceph-mon[37140]: pgmap v8814: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:04:41.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:41 smithi138 ceph-mon[36954]: pgmap v8814: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:04:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:40 smithi100 ceph-mon[32599]: pgmap v8814: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:04:41.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:04:41.847+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:04:41.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:04:41.848+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:04:41.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:04:41.848+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:04:41.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:04:41.848+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:04:41.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:04:41.848+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:04:43.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:43 smithi167 ceph-mon[37140]: pgmap v8815: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:04:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:43 smithi138 ceph-mon[36954]: pgmap v8815: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:04:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:43 smithi100 ceph-mon[32599]: pgmap v8815: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:04:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:45 smithi167 ceph-mon[37140]: pgmap v8816: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:04:45.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:45 smithi138 ceph-mon[36954]: pgmap v8816: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:04:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:45 smithi100 ceph-mon[32599]: pgmap v8816: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:04:47.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:47 smithi167 ceph-mon[37140]: pgmap v8817: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:04:47.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:47 smithi138 ceph-mon[36954]: pgmap v8817: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:04:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:47 smithi100 ceph-mon[32599]: pgmap v8817: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:04:49.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:49 smithi167 ceph-mon[37140]: pgmap v8818: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:04:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:49 smithi138 ceph-mon[36954]: pgmap v8818: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:04:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:49 smithi100 ceph-mon[32599]: pgmap v8818: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:04:51.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:51 smithi167 ceph-mon[37140]: pgmap v8819: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:04:51.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:51 smithi138 ceph-mon[36954]: pgmap v8819: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:04:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:51 smithi100 ceph-mon[32599]: pgmap v8819: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:04:53.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:53 smithi167 ceph-mon[37140]: pgmap v8820: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:04:53.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:53 smithi138 ceph-mon[36954]: pgmap v8820: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:04:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:53 smithi100 ceph-mon[32599]: pgmap v8820: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:04:55.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:55 smithi167 ceph-mon[37140]: pgmap v8821: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:04:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:55 smithi138 ceph-mon[36954]: pgmap v8821: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:04:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:55 smithi100 ceph-mon[32599]: pgmap v8821: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:04:56.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:04:56.864+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:04:56.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:04:56.864+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:04:56.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:04:56.864+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:04:56.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:04:56.864+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:04:56.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:04:56.864+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:04:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:57 smithi100 ceph-mon[32599]: pgmap v8822: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:04:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:57 smithi138 ceph-mon[36954]: pgmap v8822: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:04:57.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:57 smithi167 ceph-mon[37140]: pgmap v8822: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:04:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:04:59 smithi138 ceph-mon[36954]: pgmap v8823: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:04:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:04:59 smithi100 ceph-mon[32599]: pgmap v8823: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:04:59.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:04:59 smithi167 ceph-mon[37140]: pgmap v8823: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:05:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:01 smithi138 ceph-mon[36954]: pgmap v8824: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:05:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:01 smithi100 ceph-mon[32599]: pgmap v8824: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:05:01.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:01 smithi167 ceph-mon[37140]: pgmap v8824: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:05:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:03 smithi138 ceph-mon[36954]: pgmap v8825: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:05:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:03 smithi100 ceph-mon[32599]: pgmap v8825: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:05:03.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:03 smithi167 ceph-mon[37140]: pgmap v8825: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:05:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:05 smithi138 ceph-mon[36954]: pgmap v8826: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:05:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:05 smithi100 ceph-mon[32599]: pgmap v8826: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:05:05.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:05 smithi167 ceph-mon[37140]: pgmap v8826: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:05:06.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:05:06.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:05:06.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:05:06.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:05:06.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:05:06.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:05:06.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:05:06.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:05:06.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:05:06.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:05:06.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:05:06.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:05:06.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:05:06.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:05:06.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:05:06.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:05:06.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:05:06.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:05:06.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:05:06.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:05:06.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:05:06.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:05:06.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:05:06.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:05:06.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:05:06.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:05:06.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:05:06.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:05:06.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:05:06.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:05:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:07 smithi138 ceph-mon[36954]: pgmap v8827: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:05:07.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:07 smithi100 ceph-mon[32599]: pgmap v8827: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:05:07.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:07 smithi167 ceph-mon[37140]: pgmap v8827: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:05:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:09 smithi138 ceph-mon[36954]: pgmap v8828: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:05:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:09 smithi138 ceph-mon[36954]: osdmap e598: 12 total, 12 up, 12 in 2024-07-26T18:05:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:09 smithi100 ceph-mon[32599]: pgmap v8828: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:05:09.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:09 smithi100 ceph-mon[32599]: osdmap e598: 12 total, 12 up, 12 in 2024-07-26T18:05:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:09 smithi167 ceph-mon[37140]: pgmap v8828: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:05:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:09 smithi167 ceph-mon[37140]: osdmap e598: 12 total, 12 up, 12 in 2024-07-26T18:05:11.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:11 smithi138 ceph-mon[36954]: pgmap v8830: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:05:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:11 smithi100 ceph-mon[32599]: pgmap v8830: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:05:11.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:11 smithi167 ceph-mon[37140]: pgmap v8830: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:05:11.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:05:11.879+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:05:11.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:05:11.880+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:05:11.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:05:11.880+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:05:11.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:05:11.880+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:05:11.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:05:11.880+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:05:13.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:13 smithi138 ceph-mon[36954]: pgmap v8831: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:05:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:13 smithi100 ceph-mon[32599]: pgmap v8831: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:05:13.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:13 smithi167 ceph-mon[37140]: pgmap v8831: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:05:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:15 smithi138 ceph-mon[36954]: pgmap v8832: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:05:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:15 smithi100 ceph-mon[32599]: pgmap v8832: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:05:15.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:15 smithi167 ceph-mon[37140]: pgmap v8832: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:05:17.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:17 smithi138 ceph-mon[36954]: pgmap v8833: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:05:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:17 smithi100 ceph-mon[32599]: pgmap v8833: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:05:17.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:17 smithi167 ceph-mon[37140]: pgmap v8833: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:05:19.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:18 smithi100 ceph-mon[32599]: pgmap v8834: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:05:19.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:18 smithi100 ceph-mon[32599]: osdmap e599: 12 total, 12 up, 12 in 2024-07-26T18:05:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:18 smithi167 ceph-mon[37140]: pgmap v8834: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:05:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:18 smithi167 ceph-mon[37140]: osdmap e599: 12 total, 12 up, 12 in 2024-07-26T18:05:19.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:18 smithi138 ceph-mon[36954]: pgmap v8834: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:05:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:18 smithi138 ceph-mon[36954]: osdmap e599: 12 total, 12 up, 12 in 2024-07-26T18:05:21.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:20 smithi100 ceph-mon[32599]: pgmap v8836: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:05:21.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:20 smithi167 ceph-mon[37140]: pgmap v8836: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:05:21.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:20 smithi138 ceph-mon[36954]: pgmap v8836: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:05:23.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:22 smithi100 ceph-mon[32599]: pgmap v8837: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:05:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:22 smithi167 ceph-mon[37140]: pgmap v8837: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:05:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:22 smithi138 ceph-mon[36954]: pgmap v8837: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:05:25.243 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:24 smithi138 ceph-mon[36954]: pgmap v8838: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:05:25.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:24 smithi100 ceph-mon[32599]: pgmap v8838: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:05:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:24 smithi167 ceph-mon[37140]: pgmap v8838: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:05:26.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:05:26.888+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:05:26.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:05:26.889+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:05:26.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:05:26.889+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:05:26.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:05:26.889+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:05:26.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:05:26.889+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:05:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:26 smithi100 ceph-mon[32599]: pgmap v8839: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:05:27.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:26 smithi167 ceph-mon[37140]: pgmap v8839: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:05:27.443 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:26 smithi138 ceph-mon[36954]: pgmap v8839: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:05:29.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:28 smithi167 ceph-mon[37140]: pgmap v8840: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:05:29.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:28 smithi138 ceph-mon[36954]: pgmap v8840: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:05:29.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:28 smithi100 ceph-mon[32599]: pgmap v8840: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:05:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:31 smithi167 ceph-mon[37140]: pgmap v8841: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:05:31.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:31 smithi138 ceph-mon[36954]: pgmap v8841: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:05:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:30 smithi100 ceph-mon[32599]: pgmap v8841: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:05:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:33 smithi167 ceph-mon[37140]: pgmap v8842: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:05:33.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:33 smithi138 ceph-mon[36954]: pgmap v8842: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:05:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:33 smithi100 ceph-mon[32599]: pgmap v8842: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:05:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:35 smithi167 ceph-mon[37140]: pgmap v8843: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T18:05:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:35 smithi138 ceph-mon[36954]: pgmap v8843: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T18:05:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:35 smithi100 ceph-mon[32599]: pgmap v8843: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T18:05:37.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:37 smithi167 ceph-mon[37140]: pgmap v8844: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:05:37.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:37 smithi138 ceph-mon[36954]: pgmap v8844: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:05:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:37 smithi100 ceph-mon[32599]: pgmap v8844: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:05:39.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:39 smithi167 ceph-mon[37140]: pgmap v8845: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:05:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:39 smithi138 ceph-mon[36954]: pgmap v8845: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:05:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:39 smithi100 ceph-mon[32599]: pgmap v8845: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:05:41.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:41 smithi167 ceph-mon[37140]: pgmap v8846: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:05:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:41 smithi138 ceph-mon[36954]: pgmap v8846: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:05:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:41 smithi100 ceph-mon[32599]: pgmap v8846: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:05:41.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:05:41.896+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:05:41.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:05:41.896+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:05:41.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:05:41.896+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:05:41.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:05:41.896+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:05:41.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:05:41.897+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:05:43.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:43 smithi167 ceph-mon[37140]: pgmap v8847: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:05:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:43 smithi138 ceph-mon[36954]: pgmap v8847: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:05:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:43 smithi100 ceph-mon[32599]: pgmap v8847: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:05:45.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:45 smithi138 ceph-mon[36954]: pgmap v8848: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T18:05:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:45 smithi100 ceph-mon[32599]: pgmap v8848: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T18:05:45.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:45 smithi167 ceph-mon[37140]: pgmap v8848: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T18:05:47.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:47 smithi138 ceph-mon[36954]: pgmap v8849: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:05:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:47 smithi100 ceph-mon[32599]: pgmap v8849: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:05:47.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:47 smithi167 ceph-mon[37140]: pgmap v8849: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:05:49.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:49 smithi138 ceph-mon[36954]: pgmap v8850: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:05:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:49 smithi100 ceph-mon[32599]: pgmap v8850: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:05:49.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:49 smithi167 ceph-mon[37140]: pgmap v8850: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:05:51.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:51 smithi138 ceph-mon[36954]: pgmap v8851: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:05:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:51 smithi100 ceph-mon[32599]: pgmap v8851: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:05:51.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:51 smithi167 ceph-mon[37140]: pgmap v8851: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:05:53.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:53 smithi138 ceph-mon[36954]: pgmap v8852: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:05:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:53 smithi100 ceph-mon[32599]: pgmap v8852: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:05:53.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:53 smithi167 ceph-mon[37140]: pgmap v8852: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:05:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:55 smithi138 ceph-mon[36954]: pgmap v8853: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:05:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:55 smithi100 ceph-mon[32599]: pgmap v8853: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:05:55.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:55 smithi167 ceph-mon[37140]: pgmap v8853: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:05:56.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:05:56.912+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:05:56.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:05:56.913+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:05:56.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:05:56.913+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:05:56.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:05:56.913+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:05:56.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:05:56.913+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:05:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:57 smithi100 ceph-mon[32599]: pgmap v8854: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:05:57.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:57 smithi167 ceph-mon[37140]: pgmap v8854: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:05:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:57 smithi138 ceph-mon[36954]: pgmap v8854: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:05:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:05:59 smithi138 ceph-mon[36954]: pgmap v8855: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:05:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:05:59 smithi100 ceph-mon[32599]: pgmap v8855: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:05:59.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:05:59 smithi167 ceph-mon[37140]: pgmap v8855: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:06:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:06:01 smithi138 ceph-mon[36954]: pgmap v8856: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:06:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:06:01 smithi100 ceph-mon[32599]: pgmap v8856: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:06:01.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:06:01 smithi167 ceph-mon[37140]: pgmap v8856: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:06:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:06:03 smithi138 ceph-mon[36954]: pgmap v8857: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:06:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:06:03 smithi100 ceph-mon[32599]: pgmap v8857: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:06:03.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:06:03 smithi167 ceph-mon[37140]: pgmap v8857: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:06:05.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:06:05.211+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T18:06:05.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:06:05.211+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c006690 con 0x7fe5a81087a0 2024-07-26T18:06:05.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:06:05.211+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 122 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c006690 con 0x7fe5a81087a0 2024-07-26T18:06:05.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:06:05.211+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 122 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T18:06:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:06:05 smithi138 ceph-mon[36954]: pgmap v8858: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T18:06:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:06:05 smithi100 ceph-mon[32599]: pgmap v8858: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T18:06:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:06:05 smithi167 ceph-mon[37140]: pgmap v8858: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-07-26T18:06:06.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:06:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:06:06.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:06:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:06:06.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:06:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:06:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:06:07 smithi138 ceph-mon[36954]: pgmap v8859: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:06:07.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:06:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:06:07.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:06:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:06:07.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:06:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:06:07.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:06:07 smithi100 ceph-mon[32599]: pgmap v8859: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:06:07.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:06:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:06:07.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:06:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:06:07.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:06:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:06:07.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:06:07 smithi167 ceph-mon[37140]: pgmap v8859: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:06:07.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:06:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:06:07.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:06:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:06:07.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:06:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:06:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:06:09 smithi138 ceph-mon[36954]: pgmap v8860: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:06:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:06:09 smithi138 ceph-mon[36954]: osdmap e600: 12 total, 12 up, 12 in 2024-07-26T18:06:09.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:06:09 smithi100 ceph-mon[32599]: pgmap v8860: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:06:09.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:06:09 smithi100 ceph-mon[32599]: osdmap e600: 12 total, 12 up, 12 in 2024-07-26T18:06:09.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:06:09 smithi167 ceph-mon[37140]: pgmap v8860: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:06:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:06:09 smithi167 ceph-mon[37140]: osdmap e600: 12 total, 12 up, 12 in 2024-07-26T18:06:11.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:06:11 smithi138 ceph-mon[36954]: pgmap v8862: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:06:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:06:11 smithi100 ceph-mon[32599]: pgmap v8862: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:06:11.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:06:11 smithi167 ceph-mon[37140]: pgmap v8862: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:06:11.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:06:11.929+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:06:11.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:06:11.929+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:06:11.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:06:11.929+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:06:11.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:06:11.929+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:06:11.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:06:11.929+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:06:13.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:06:13 smithi138 ceph-mon[36954]: pgmap v8863: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:06:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:06:13 smithi100 ceph-mon[32599]: pgmap v8863: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:06:13.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:06:13 smithi167 ceph-mon[37140]: pgmap v8863: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:06:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:06:15 smithi138 ceph-mon[36954]: pgmap v8864: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T18:06:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:06:15 smithi100 ceph-mon[32599]: pgmap v8864: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T18:06:15.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:06:15 smithi167 ceph-mon[37140]: pgmap v8864: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-07-26T18:06:17.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:06:17 smithi138 ceph-mon[36954]: pgmap v8865: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:06:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:06:17 smithi100 ceph-mon[32599]: pgmap v8865: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:06:17.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:06:17 smithi167 ceph-mon[37140]: pgmap v8865: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:06:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:06:18 smithi167 ceph-mon[37140]: pgmap v8866: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:06:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:06:18 smithi167 ceph-mon[37140]: osdmap e601: 12 total, 12 up, 12 in 2024-07-26T18:06:19.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:06:18 smithi138 ceph-mon[36954]: pgmap v8866: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:06:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:06:18 smithi138 ceph-mon[36954]: osdmap e601: 12 total, 12 up, 12 in 2024-07-26T18:06:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:06:18 smithi100 ceph-mon[32599]: pgmap v8866: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:06:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:06:18 smithi100 ceph-mon[32599]: osdmap e601: 12 total, 12 up, 12 in 2024-07-26T18:06:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:06:20 smithi167 ceph-mon[37140]: pgmap v8868: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:06:21.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:06:20 smithi138 ceph-mon[36954]: pgmap v8868: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:06:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:06:20 smithi100 ceph-mon[32599]: pgmap v8868: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:06:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:06:23 smithi167 ceph-mon[37140]: pgmap v8869: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:06:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:06:23 smithi138 ceph-mon[36954]: pgmap v8869: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:06:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:06:23 smithi100 ceph-mon[32599]: pgmap v8869: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:06:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:06:25 smithi167 ceph-mon[37140]: pgmap v8870: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:06:25.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:06:25 smithi138 ceph-mon[36954]: pgmap v8870: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:06:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:06:25 smithi100 ceph-mon[32599]: pgmap v8870: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:06:26.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:06:26.942+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:06:26.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:06:26.943+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:06:26.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:06:26.943+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:06:26.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:06:26.943+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:06:26.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:06:26.943+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:06:27.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:06:27 smithi100 ceph-mon[32599]: pgmap v8871: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:06:27.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:06:27 smithi167 ceph-mon[37140]: pgmap v8871: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:06:27.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:06:27 smithi138 ceph-mon[36954]: pgmap v8871: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:06:29.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:06:29 smithi167 ceph-mon[37140]: pgmap v8872: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:06:29.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:06:29 smithi138 ceph-mon[36954]: pgmap v8872: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:06:29.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:06:29 smithi100 ceph-mon[32599]: pgmap v8872: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:06:31.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:06:31 smithi167 ceph-mon[37140]: pgmap v8873: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:06:31.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:06:31 smithi138 ceph-mon[36954]: pgmap v8873: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:06:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:06:31 smithi100 ceph-mon[32599]: pgmap v8873: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:06:33.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:06:33 smithi167 ceph-mon[37140]: pgmap v8874: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:06:33.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:06:33 smithi138 ceph-mon[36954]: pgmap v8874: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:06:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:06:33 smithi100 ceph-mon[32599]: pgmap v8874: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:06:35.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:06:35 smithi167 ceph-mon[37140]: pgmap v8875: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:06:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:06:35 smithi138 ceph-mon[36954]: pgmap v8875: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:06:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:06:35 smithi100 ceph-mon[32599]: pgmap v8875: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:06:37.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:06:37 smithi138 ceph-mon[36954]: pgmap v8876: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:06:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:06:37 smithi100 ceph-mon[32599]: pgmap v8876: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:06:37.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:06:37 smithi167 ceph-mon[37140]: pgmap v8876: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:06:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:06:39 smithi138 ceph-mon[36954]: pgmap v8877: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:06:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:06:39 smithi100 ceph-mon[32599]: pgmap v8877: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:06:39.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:06:39 smithi167 ceph-mon[37140]: pgmap v8877: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:06:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:06:41 smithi138 ceph-mon[36954]: pgmap v8878: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:06:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:06:41 smithi100 ceph-mon[32599]: pgmap v8878: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:06:41.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:06:41 smithi167 ceph-mon[37140]: pgmap v8878: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:06:41.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:06:41.958+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:06:41.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:06:41.959+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:06:41.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:06:41.959+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:06:41.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:06:41.959+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:06:41.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:06:41.959+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:06:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:06:43 smithi138 ceph-mon[36954]: pgmap v8879: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:06:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:06:43 smithi100 ceph-mon[32599]: pgmap v8879: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:06:43.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:06:43 smithi167 ceph-mon[37140]: pgmap v8879: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:06:45.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:06:45 smithi138 ceph-mon[36954]: pgmap v8880: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:06:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:06:45 smithi100 ceph-mon[32599]: pgmap v8880: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:06:45.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:06:45 smithi167 ceph-mon[37140]: pgmap v8880: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:06:47.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:06:47 smithi138 ceph-mon[36954]: pgmap v8881: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:06:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:06:47 smithi100 ceph-mon[32599]: pgmap v8881: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:06:47.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:06:47 smithi167 ceph-mon[37140]: pgmap v8881: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:06:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:06:49 smithi138 ceph-mon[36954]: pgmap v8882: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:06:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:06:49 smithi100 ceph-mon[32599]: pgmap v8882: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:06:49.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:06:49 smithi167 ceph-mon[37140]: pgmap v8882: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:06:51.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:06:51 smithi138 ceph-mon[36954]: pgmap v8883: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:06:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:06:51 smithi100 ceph-mon[32599]: pgmap v8883: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:06:51.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:06:51 smithi167 ceph-mon[37140]: pgmap v8883: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:06:53.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:06:53 smithi138 ceph-mon[36954]: pgmap v8884: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:06:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:06:53 smithi100 ceph-mon[32599]: pgmap v8884: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:06:53.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:06:53 smithi167 ceph-mon[37140]: pgmap v8884: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:06:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:06:55 smithi138 ceph-mon[36954]: pgmap v8885: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:06:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:06:55 smithi100 ceph-mon[32599]: pgmap v8885: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:06:55.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:06:55 smithi167 ceph-mon[37140]: pgmap v8885: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:06:56.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:06:56.967+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:06:56.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:06:56.968+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:06:56.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:06:56.968+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:06:56.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:06:56.968+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:06:56.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:06:56.968+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:06:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:06:57 smithi100 ceph-mon[32599]: pgmap v8886: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:06:57.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:06:57 smithi167 ceph-mon[37140]: pgmap v8886: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:06:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:06:57 smithi138 ceph-mon[36954]: pgmap v8886: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:06:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:06:59 smithi138 ceph-mon[36954]: pgmap v8887: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:06:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:06:59 smithi100 ceph-mon[32599]: pgmap v8887: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:06:59.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:06:59 smithi167 ceph-mon[37140]: pgmap v8887: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:07:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:07:01 smithi138 ceph-mon[36954]: pgmap v8888: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:07:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:07:01 smithi100 ceph-mon[32599]: pgmap v8888: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:07:01.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:07:01 smithi167 ceph-mon[37140]: pgmap v8888: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:07:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:07:03 smithi138 ceph-mon[36954]: pgmap v8889: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:07:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:07:03 smithi100 ceph-mon[32599]: pgmap v8889: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:07:03.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:07:03 smithi167 ceph-mon[37140]: pgmap v8889: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:07:05.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:07:05 smithi138 ceph-mon[36954]: pgmap v8890: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:07:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:07:05 smithi100 ceph-mon[32599]: pgmap v8890: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:07:05.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:07:05 smithi167 ceph-mon[37140]: pgmap v8890: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:07:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:07:07 smithi138 ceph-mon[36954]: pgmap v8891: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:07:07.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:07:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:07:07.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:07:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:07:07.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:07:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:07:07.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:07:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:07:07.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:07:07 smithi100 ceph-mon[32599]: pgmap v8891: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:07:07.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:07:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:07:07.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:07:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:07:07.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:07:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:07:07.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:07:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:07:07.526 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:07:07 smithi167 ceph-mon[37140]: pgmap v8891: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:07:07.526 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:07:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:07:07.526 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:07:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:07:07.527 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:07:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:07:07.527 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:07:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:07:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:07:09 smithi138 ceph-mon[36954]: pgmap v8892: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:07:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:07:09 smithi138 ceph-mon[36954]: osdmap e602: 12 total, 12 up, 12 in 2024-07-26T18:07:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:07:09 smithi100 ceph-mon[32599]: pgmap v8892: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:07:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:07:09 smithi100 ceph-mon[32599]: osdmap e602: 12 total, 12 up, 12 in 2024-07-26T18:07:09.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:07:09 smithi167 ceph-mon[37140]: pgmap v8892: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:07:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:07:09 smithi167 ceph-mon[37140]: osdmap e602: 12 total, 12 up, 12 in 2024-07-26T18:07:11.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:07:11 smithi138 ceph-mon[36954]: pgmap v8894: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:07:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:07:11 smithi100 ceph-mon[32599]: pgmap v8894: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:07:11.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:07:11 smithi167 ceph-mon[37140]: pgmap v8894: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:07:11.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:07:11.983+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:07:11.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:07:11.984+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:07:11.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:07:11.984+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:07:11.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:07:11.984+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:07:11.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:07:11.984+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:07:13.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:07:13 smithi138 ceph-mon[36954]: pgmap v8895: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:07:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:07:13 smithi100 ceph-mon[32599]: pgmap v8895: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:07:13.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:07:13 smithi167 ceph-mon[37140]: pgmap v8895: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:07:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:07:15 smithi138 ceph-mon[36954]: pgmap v8896: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:07:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:07:15 smithi100 ceph-mon[32599]: pgmap v8896: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:07:15.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:07:15 smithi167 ceph-mon[37140]: pgmap v8896: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:07:17.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:07:17 smithi138 ceph-mon[36954]: pgmap v8897: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:07:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:07:17 smithi100 ceph-mon[32599]: pgmap v8897: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:07:17.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:07:17 smithi167 ceph-mon[37140]: pgmap v8897: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:07:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:07:19 smithi167 ceph-mon[37140]: pgmap v8898: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:07:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:07:19 smithi167 ceph-mon[37140]: osdmap e603: 12 total, 12 up, 12 in 2024-07-26T18:07:19.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:07:19 smithi138 ceph-mon[36954]: pgmap v8898: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:07:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:07:19 smithi138 ceph-mon[36954]: osdmap e603: 12 total, 12 up, 12 in 2024-07-26T18:07:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:07:19 smithi100 ceph-mon[32599]: pgmap v8898: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:07:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:07:19 smithi100 ceph-mon[32599]: osdmap e603: 12 total, 12 up, 12 in 2024-07-26T18:07:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:07:21 smithi167 ceph-mon[37140]: pgmap v8900: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:07:21.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:07:21 smithi138 ceph-mon[36954]: pgmap v8900: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:07:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:07:21 smithi100 ceph-mon[32599]: pgmap v8900: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:07:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:07:23 smithi167 ceph-mon[37140]: pgmap v8901: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:07:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:07:23 smithi138 ceph-mon[36954]: pgmap v8901: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:07:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:07:23 smithi100 ceph-mon[32599]: pgmap v8901: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:07:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:07:25 smithi167 ceph-mon[37140]: pgmap v8902: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:07:25.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:07:25 smithi138 ceph-mon[36954]: pgmap v8902: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:07:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:07:25 smithi100 ceph-mon[32599]: pgmap v8902: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:07:27.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:07:27.000+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:07:27.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:07:27.000+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:07:27.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:07:27.000+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:07:27.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:07:27.000+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:07:27.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:07:27.000+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:07:27.296 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:07:27 smithi100 ceph-mon[32599]: pgmap v8903: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:07:27.443 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:07:27 smithi138 ceph-mon[36954]: pgmap v8903: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:07:27.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:07:27 smithi167 ceph-mon[37140]: pgmap v8903: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:07:29.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:07:29 smithi138 ceph-mon[36954]: pgmap v8904: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:07:29.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:07:29 smithi100 ceph-mon[32599]: pgmap v8904: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:07:29.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:07:29 smithi167 ceph-mon[37140]: pgmap v8904: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:07:31.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:07:31 smithi138 ceph-mon[36954]: pgmap v8905: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:07:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:07:31 smithi100 ceph-mon[32599]: pgmap v8905: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:07:31.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:07:31 smithi167 ceph-mon[37140]: pgmap v8905: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:07:33.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:07:33 smithi138 ceph-mon[36954]: pgmap v8906: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:07:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:07:33 smithi100 ceph-mon[32599]: pgmap v8906: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:07:33.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:07:33 smithi167 ceph-mon[37140]: pgmap v8906: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:07:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:07:35 smithi138 ceph-mon[36954]: pgmap v8907: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:07:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:07:35 smithi100 ceph-mon[32599]: pgmap v8907: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:07:35.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:07:35 smithi167 ceph-mon[37140]: pgmap v8907: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:07:37.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:07:37 smithi138 ceph-mon[36954]: pgmap v8908: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:07:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:07:37 smithi100 ceph-mon[32599]: pgmap v8908: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:07:37.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:07:37 smithi167 ceph-mon[37140]: pgmap v8908: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:07:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:07:39 smithi138 ceph-mon[36954]: pgmap v8909: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:07:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:07:39 smithi100 ceph-mon[32599]: pgmap v8909: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:07:39.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:07:39 smithi167 ceph-mon[37140]: pgmap v8909: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:07:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:07:41 smithi138 ceph-mon[36954]: pgmap v8910: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:07:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:07:41 smithi100 ceph-mon[32599]: pgmap v8910: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:07:41.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:07:41 smithi167 ceph-mon[37140]: pgmap v8910: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:07:42.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:07:42.011+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:07:42.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:07:42.012+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:07:42.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:07:42.012+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:07:42.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:07:42.012+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:07:42.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:07:42.012+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:07:43.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:07:43 smithi138 ceph-mon[36954]: pgmap v8911: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:07:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:07:43 smithi100 ceph-mon[32599]: pgmap v8911: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:07:43.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:07:43 smithi167 ceph-mon[37140]: pgmap v8911: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:07:45.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:07:45 smithi138 ceph-mon[36954]: pgmap v8912: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T18:07:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:07:45 smithi100 ceph-mon[32599]: pgmap v8912: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T18:07:45.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:07:45 smithi167 ceph-mon[37140]: pgmap v8912: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T18:07:47.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:07:47 smithi138 ceph-mon[36954]: pgmap v8913: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:07:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:07:47 smithi100 ceph-mon[32599]: pgmap v8913: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:07:47.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:07:47 smithi167 ceph-mon[37140]: pgmap v8913: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:07:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:07:49 smithi138 ceph-mon[36954]: pgmap v8914: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:07:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:07:49 smithi100 ceph-mon[32599]: pgmap v8914: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:07:49.526 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:07:49 smithi167 ceph-mon[37140]: pgmap v8914: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:07:51.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:07:51 smithi138 ceph-mon[36954]: pgmap v8915: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:07:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:07:51 smithi100 ceph-mon[32599]: pgmap v8915: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:07:51.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:07:51 smithi167 ceph-mon[37140]: pgmap v8915: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:07:53.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:07:53 smithi138 ceph-mon[36954]: pgmap v8916: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:07:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:07:53 smithi100 ceph-mon[32599]: pgmap v8916: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:07:53.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:07:53 smithi167 ceph-mon[37140]: pgmap v8916: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:07:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:07:55 smithi138 ceph-mon[36954]: pgmap v8917: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T18:07:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:07:55 smithi100 ceph-mon[32599]: pgmap v8917: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T18:07:55.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:07:55 smithi167 ceph-mon[37140]: pgmap v8917: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T18:07:57.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:07:57.019+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:07:57.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:07:57.019+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:07:57.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:07:57.019+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:07:57.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:07:57.019+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:07:57.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:07:57.019+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:07:57.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:07:57 smithi100 ceph-mon[32599]: pgmap v8918: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:07:57.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:07:57 smithi167 ceph-mon[37140]: pgmap v8918: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:07:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:07:57 smithi138 ceph-mon[36954]: pgmap v8918: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:07:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:07:59 smithi138 ceph-mon[36954]: pgmap v8919: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:07:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:07:59 smithi100 ceph-mon[32599]: pgmap v8919: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:07:59.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:07:59 smithi167 ceph-mon[37140]: pgmap v8919: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:08:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:08:01 smithi138 ceph-mon[36954]: pgmap v8920: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:08:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:08:01 smithi100 ceph-mon[32599]: pgmap v8920: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:08:01.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:08:01 smithi167 ceph-mon[37140]: pgmap v8920: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:08:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:08:03 smithi138 ceph-mon[36954]: pgmap v8921: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:08:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:08:03 smithi100 ceph-mon[32599]: pgmap v8921: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:08:03.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:08:03 smithi167 ceph-mon[37140]: pgmap v8921: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:08:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:08:05 smithi138 ceph-mon[36954]: pgmap v8922: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:08:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:08:05 smithi100 ceph-mon[32599]: pgmap v8922: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:08:05.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:08:05 smithi167 ceph-mon[37140]: pgmap v8922: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:08:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:08:07 smithi138 ceph-mon[36954]: pgmap v8923: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:08:07.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:08:07 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:08:07.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:08:07 smithi100 ceph-mon[32599]: pgmap v8923: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:08:07.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:08:07 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:08:07.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:08:07 smithi167 ceph-mon[37140]: pgmap v8923: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:08:07.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:08:07 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:08:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:08:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:08:08.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:08:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:08:08.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:08:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:08:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:08:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:08:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:08:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:08:08.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:08:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:08:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:08:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:08:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:08:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:08:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:08:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:08:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:08:09 smithi138 ceph-mon[36954]: pgmap v8924: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:08:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:08:09 smithi138 ceph-mon[36954]: osdmap e604: 12 total, 12 up, 12 in 2024-07-26T18:08:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:08:09 smithi100 ceph-mon[32599]: pgmap v8924: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:08:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:08:09 smithi100 ceph-mon[32599]: osdmap e604: 12 total, 12 up, 12 in 2024-07-26T18:08:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:08:09 smithi167 ceph-mon[37140]: pgmap v8924: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:08:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:08:09 smithi167 ceph-mon[37140]: osdmap e604: 12 total, 12 up, 12 in 2024-07-26T18:08:11.443 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:08:11 smithi138 ceph-mon[36954]: pgmap v8926: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:08:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:08:11 smithi100 ceph-mon[32599]: pgmap v8926: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:08:11.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:08:11 smithi167 ceph-mon[37140]: pgmap v8926: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:08:12.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:08:12.029+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:08:12.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:08:12.029+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:08:12.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:08:12.029+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:08:12.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:08:12.029+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:08:12.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:08:12.029+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:08:13.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:08:13 smithi138 ceph-mon[36954]: pgmap v8927: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:08:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:08:13 smithi100 ceph-mon[32599]: pgmap v8927: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:08:13.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:08:13 smithi167 ceph-mon[37140]: pgmap v8927: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:08:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:08:15 smithi138 ceph-mon[36954]: pgmap v8928: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:08:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:08:15 smithi100 ceph-mon[32599]: pgmap v8928: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:08:15.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:08:15 smithi167 ceph-mon[37140]: pgmap v8928: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:08:17.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:08:17 smithi138 ceph-mon[36954]: pgmap v8929: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:08:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:08:17 smithi100 ceph-mon[32599]: pgmap v8929: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:08:17.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:08:17 smithi167 ceph-mon[37140]: pgmap v8929: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:08:19.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:08:19 smithi138 ceph-mon[36954]: pgmap v8930: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:08:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:08:19 smithi138 ceph-mon[36954]: osdmap e605: 12 total, 12 up, 12 in 2024-07-26T18:08:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:08:19 smithi100 ceph-mon[32599]: pgmap v8930: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:08:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:08:19 smithi100 ceph-mon[32599]: osdmap e605: 12 total, 12 up, 12 in 2024-07-26T18:08:19.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:08:19 smithi167 ceph-mon[37140]: pgmap v8930: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:08:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:08:19 smithi167 ceph-mon[37140]: osdmap e605: 12 total, 12 up, 12 in 2024-07-26T18:08:21.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:08:21 smithi138 ceph-mon[36954]: pgmap v8932: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:08:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:08:21 smithi100 ceph-mon[32599]: pgmap v8932: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:08:21.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:08:21 smithi167 ceph-mon[37140]: pgmap v8932: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:08:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:08:23 smithi138 ceph-mon[36954]: pgmap v8933: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:08:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:08:23 smithi100 ceph-mon[32599]: pgmap v8933: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:08:23.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:08:23 smithi167 ceph-mon[37140]: pgmap v8933: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:08:25.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:08:25 smithi138 ceph-mon[36954]: pgmap v8934: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:08:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:08:25 smithi100 ceph-mon[32599]: pgmap v8934: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:08:25.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:08:25 smithi167 ceph-mon[37140]: pgmap v8934: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:08:27.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:08:27.040+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:08:27.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:08:27.040+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:08:27.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:08:27.040+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:08:27.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:08:27.040+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:08:27.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:08:27.041+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:08:27.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:08:27 smithi100 ceph-mon[32599]: pgmap v8935: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:08:27.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:08:27 smithi167 ceph-mon[37140]: pgmap v8935: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:08:27.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:08:27 smithi138 ceph-mon[36954]: pgmap v8935: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:08:29.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:08:29 smithi138 ceph-mon[36954]: pgmap v8936: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:08:29.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:08:29 smithi100 ceph-mon[32599]: pgmap v8936: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:08:29.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:08:29 smithi167 ceph-mon[37140]: pgmap v8936: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:08:31.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:08:31 smithi138 ceph-mon[36954]: pgmap v8937: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:08:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:08:31 smithi100 ceph-mon[32599]: pgmap v8937: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:08:31.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:08:31 smithi167 ceph-mon[37140]: pgmap v8937: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:08:33.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:08:33 smithi138 ceph-mon[36954]: pgmap v8938: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:08:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:08:33 smithi100 ceph-mon[32599]: pgmap v8938: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:08:33.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:08:33 smithi167 ceph-mon[37140]: pgmap v8938: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:08:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:08:35 smithi138 ceph-mon[36954]: pgmap v8939: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:08:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:08:35 smithi100 ceph-mon[32599]: pgmap v8939: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:08:35.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:08:35 smithi167 ceph-mon[37140]: pgmap v8939: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:08:37.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:08:37 smithi138 ceph-mon[36954]: pgmap v8940: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:08:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:08:37 smithi100 ceph-mon[32599]: pgmap v8940: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:08:37.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:08:37 smithi167 ceph-mon[37140]: pgmap v8940: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:08:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:08:39 smithi138 ceph-mon[36954]: pgmap v8941: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:08:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:08:39 smithi100 ceph-mon[32599]: pgmap v8941: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:08:39.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:08:39 smithi167 ceph-mon[37140]: pgmap v8941: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:08:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:08:41 smithi138 ceph-mon[36954]: pgmap v8942: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:08:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:08:41 smithi100 ceph-mon[32599]: pgmap v8942: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:08:41.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:08:41 smithi167 ceph-mon[37140]: pgmap v8942: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:08:42.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:08:42.051+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:08:42.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:08:42.052+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:08:42.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:08:42.052+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:08:42.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:08:42.052+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:08:42.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:08:42.052+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:08:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:08:43 smithi138 ceph-mon[36954]: pgmap v8943: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:08:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:08:43 smithi100 ceph-mon[32599]: pgmap v8943: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:08:43.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:08:43 smithi167 ceph-mon[37140]: pgmap v8943: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:08:45.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:08:45 smithi138 ceph-mon[36954]: pgmap v8944: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:08:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:08:45 smithi100 ceph-mon[32599]: pgmap v8944: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:08:45.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:08:45 smithi167 ceph-mon[37140]: pgmap v8944: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:08:47.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:08:47 smithi138 ceph-mon[36954]: pgmap v8945: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:08:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:08:47 smithi100 ceph-mon[32599]: pgmap v8945: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:08:47.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:08:47 smithi167 ceph-mon[37140]: pgmap v8945: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:08:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:08:49 smithi138 ceph-mon[36954]: pgmap v8946: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:08:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:08:49 smithi100 ceph-mon[32599]: pgmap v8946: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:08:49.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:08:49 smithi167 ceph-mon[37140]: pgmap v8946: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:08:51.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:08:51 smithi138 ceph-mon[36954]: pgmap v8947: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:08:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:08:51 smithi100 ceph-mon[32599]: pgmap v8947: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:08:51.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:08:51 smithi167 ceph-mon[37140]: pgmap v8947: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:08:53.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:08:53 smithi138 ceph-mon[36954]: pgmap v8948: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:08:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:08:53 smithi100 ceph-mon[32599]: pgmap v8948: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:08:53.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:08:53 smithi167 ceph-mon[37140]: pgmap v8948: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:08:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:08:55 smithi138 ceph-mon[36954]: pgmap v8949: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:08:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:08:55 smithi100 ceph-mon[32599]: pgmap v8949: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:08:55.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:08:55 smithi167 ceph-mon[37140]: pgmap v8949: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:08:57.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:08:57.067+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:08:57.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:08:57.068+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:08:57.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:08:57.068+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:08:57.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:08:57.068+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:08:57.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:08:57.068+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:08:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:08:57 smithi138 ceph-mon[36954]: pgmap v8950: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:08:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:08:57 smithi100 ceph-mon[32599]: pgmap v8950: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:08:57.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:08:57 smithi167 ceph-mon[37140]: pgmap v8950: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:08:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:08:59 smithi138 ceph-mon[36954]: pgmap v8951: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:08:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:08:59 smithi100 ceph-mon[32599]: pgmap v8951: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:08:59.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:08:59 smithi167 ceph-mon[37140]: pgmap v8951: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:09:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:09:01 smithi138 ceph-mon[36954]: pgmap v8952: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:09:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:09:01 smithi100 ceph-mon[32599]: pgmap v8952: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:09:01.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:09:01 smithi167 ceph-mon[37140]: pgmap v8952: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:09:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:09:03 smithi138 ceph-mon[36954]: pgmap v8953: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:09:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:09:03 smithi100 ceph-mon[32599]: pgmap v8953: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:09:03.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:09:03 smithi167 ceph-mon[37140]: pgmap v8953: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:09:05.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:09:05.235+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T18:09:05.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:09:05.235+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c006690 con 0x7fe5a81087a0 2024-07-26T18:09:05.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:09:05.236+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 123 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T18:09:05.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:09:05.236+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 123 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c006690 con 0x7fe5a81087a0 2024-07-26T18:09:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:09:05 smithi138 ceph-mon[36954]: pgmap v8954: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:09:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:09:05 smithi100 ceph-mon[32599]: pgmap v8954: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:09:05.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:09:05 smithi167 ceph-mon[37140]: pgmap v8954: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:09:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:09:07 smithi138 ceph-mon[36954]: pgmap v8955: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:09:07.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:09:07 smithi100 ceph-mon[32599]: pgmap v8955: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:09:07.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:09:07 smithi167 ceph-mon[37140]: pgmap v8955: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:09:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:09:08 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:09:08.475 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:09:08 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:09:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:09:08 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:09:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:09:09 smithi138 ceph-mon[36954]: pgmap v8956: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:09:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:09:09 smithi138 ceph-mon[36954]: osdmap e606: 12 total, 12 up, 12 in 2024-07-26T18:09:09.491 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:09:09 smithi100 ceph-mon[32599]: pgmap v8956: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:09:09.492 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:09:09 smithi100 ceph-mon[32599]: osdmap e606: 12 total, 12 up, 12 in 2024-07-26T18:09:09.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:09:09 smithi167 ceph-mon[37140]: pgmap v8956: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:09:09.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:09:09 smithi167 ceph-mon[37140]: osdmap e606: 12 total, 12 up, 12 in 2024-07-26T18:09:11.224 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:09:11 smithi100 ceph-mon[32599]: pgmap v8958: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:09:11.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:09:11 smithi138 ceph-mon[36954]: pgmap v8958: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:09:11.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:09:11 smithi167 ceph-mon[37140]: pgmap v8958: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:09:12.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:09:12.079+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:09:12.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:09:12.080+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:09:12.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:09:12.080+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:09:12.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:09:12.080+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:09:12.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:09:12.080+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:09:12.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:09:12 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:09:12.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:09:12 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:09:12.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:09:12 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:09:12.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:09:12 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:09:12.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:09:12 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:09:12.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:09:12 smithi167 ceph-mon[37140]: pgmap v8959: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:09:12.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:09:12 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:09:12.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:09:12 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:09:12.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:09:12 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:09:12.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:09:12 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:09:12.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:09:12 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:09:12.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:09:12 smithi138 ceph-mon[36954]: pgmap v8959: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:09:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:09:12 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:09:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:09:12 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:09:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:09:12 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:09:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:09:12 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:09:12.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:09:12 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:09:12.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:09:12 smithi100 ceph-mon[32599]: pgmap v8959: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:09:15.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:09:14 smithi138 ceph-mon[36954]: pgmap v8960: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:09:15.247 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:09:14 smithi100 ceph-mon[32599]: pgmap v8960: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:09:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:09:14 smithi167 ceph-mon[37140]: pgmap v8960: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:09:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:09:16 smithi138 ceph-mon[36954]: pgmap v8961: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:09:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:09:16 smithi100 ceph-mon[32599]: pgmap v8961: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:09:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:09:16 smithi167 ceph-mon[37140]: pgmap v8961: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:09:19.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:09:19 smithi138 ceph-mon[36954]: pgmap v8962: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:09:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:09:19 smithi138 ceph-mon[36954]: osdmap e607: 12 total, 12 up, 12 in 2024-07-26T18:09:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:09:19 smithi100 ceph-mon[32599]: pgmap v8962: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:09:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:09:19 smithi100 ceph-mon[32599]: osdmap e607: 12 total, 12 up, 12 in 2024-07-26T18:09:19.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:09:19 smithi167 ceph-mon[37140]: pgmap v8962: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:09:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:09:19 smithi167 ceph-mon[37140]: osdmap e607: 12 total, 12 up, 12 in 2024-07-26T18:09:21.344 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:09:21 smithi100 ceph-mon[32599]: pgmap v8964: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:09:21.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:09:21 smithi138 ceph-mon[36954]: pgmap v8964: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:09:21.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:09:21 smithi167 ceph-mon[37140]: pgmap v8964: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:09:23.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:09:23 smithi138 ceph-mon[36954]: pgmap v8965: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:09:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:09:23 smithi100 ceph-mon[32599]: pgmap v8965: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:09:23.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:09:23 smithi167 ceph-mon[37140]: pgmap v8965: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:09:25.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:09:25 smithi138 ceph-mon[36954]: pgmap v8966: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:09:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:09:25 smithi100 ceph-mon[32599]: pgmap v8966: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:09:25.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:09:25 smithi167 ceph-mon[37140]: pgmap v8966: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:09:27.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:09:27.095+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:09:27.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:09:27.095+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:09:27.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:09:27.095+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:09:27.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:09:27.095+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:09:27.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:09:27.095+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:09:27.310 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:09:27 smithi100 ceph-mon[32599]: pgmap v8967: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:09:27.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:09:27 smithi167 ceph-mon[37140]: pgmap v8967: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:09:27.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:09:27 smithi138 ceph-mon[36954]: pgmap v8967: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:09:29.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:09:29 smithi138 ceph-mon[36954]: pgmap v8968: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:09:29.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:09:29 smithi100 ceph-mon[32599]: pgmap v8968: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:09:29.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:09:29 smithi167 ceph-mon[37140]: pgmap v8968: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:09:31.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:09:31 smithi138 ceph-mon[36954]: pgmap v8969: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:09:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:09:31 smithi100 ceph-mon[32599]: pgmap v8969: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:09:31.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:09:31 smithi167 ceph-mon[37140]: pgmap v8969: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:09:33.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:09:33 smithi138 ceph-mon[36954]: pgmap v8970: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:09:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:09:33 smithi100 ceph-mon[32599]: pgmap v8970: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:09:33.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:09:33 smithi167 ceph-mon[37140]: pgmap v8970: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:09:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:09:35 smithi138 ceph-mon[36954]: pgmap v8971: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:09:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:09:35 smithi100 ceph-mon[32599]: pgmap v8971: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:09:35.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:09:35 smithi167 ceph-mon[37140]: pgmap v8971: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:09:37.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:09:37 smithi138 ceph-mon[36954]: pgmap v8972: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:09:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:09:37 smithi100 ceph-mon[32599]: pgmap v8972: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:09:37.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:09:37 smithi167 ceph-mon[37140]: pgmap v8972: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:09:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:09:39 smithi138 ceph-mon[36954]: pgmap v8973: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:09:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:09:39 smithi100 ceph-mon[32599]: pgmap v8973: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:09:39.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:09:39 smithi167 ceph-mon[37140]: pgmap v8973: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:09:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:09:41 smithi138 ceph-mon[36954]: pgmap v8974: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:09:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:09:41 smithi100 ceph-mon[32599]: pgmap v8974: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:09:41.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:09:41 smithi167 ceph-mon[37140]: pgmap v8974: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:09:42.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:09:42.111+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:09:42.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:09:42.112+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:09:42.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:09:42.112+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:09:42.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:09:42.112+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:09:42.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:09:42.112+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:09:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:09:43 smithi138 ceph-mon[36954]: pgmap v8975: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:09:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:09:43 smithi100 ceph-mon[32599]: pgmap v8975: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:09:43.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:09:43 smithi167 ceph-mon[37140]: pgmap v8975: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:09:45.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:09:45 smithi138 ceph-mon[36954]: pgmap v8976: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:09:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:09:45 smithi100 ceph-mon[32599]: pgmap v8976: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:09:45.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:09:45 smithi167 ceph-mon[37140]: pgmap v8976: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:09:47.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:09:47 smithi138 ceph-mon[36954]: pgmap v8977: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:09:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:09:47 smithi100 ceph-mon[32599]: pgmap v8977: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:09:47.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:09:47 smithi167 ceph-mon[37140]: pgmap v8977: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:09:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:09:49 smithi138 ceph-mon[36954]: pgmap v8978: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:09:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:09:49 smithi100 ceph-mon[32599]: pgmap v8978: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:09:49.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:09:49 smithi167 ceph-mon[37140]: pgmap v8978: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:09:51.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:09:51 smithi138 ceph-mon[36954]: pgmap v8979: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:09:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:09:51 smithi100 ceph-mon[32599]: pgmap v8979: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:09:51.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:09:51 smithi167 ceph-mon[37140]: pgmap v8979: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:09:53.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:09:53 smithi138 ceph-mon[36954]: pgmap v8980: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:09:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:09:53 smithi100 ceph-mon[32599]: pgmap v8980: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:09:53.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:09:53 smithi167 ceph-mon[37140]: pgmap v8980: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:09:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:09:55 smithi138 ceph-mon[36954]: pgmap v8981: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:09:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:09:55 smithi100 ceph-mon[32599]: pgmap v8981: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:09:55.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:09:55 smithi167 ceph-mon[37140]: pgmap v8981: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:09:57.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:09:57.127+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:09:57.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:09:57.128+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:09:57.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:09:57.128+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:09:57.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:09:57.128+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:09:57.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:09:57.128+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:09:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:09:57 smithi138 ceph-mon[36954]: pgmap v8982: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:09:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:09:57 smithi100 ceph-mon[32599]: pgmap v8982: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:09:57.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:09:57 smithi167 ceph-mon[37140]: pgmap v8982: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:09:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:09:59 smithi138 ceph-mon[36954]: pgmap v8983: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:09:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:09:59 smithi100 ceph-mon[32599]: pgmap v8983: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:09:59.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:09:59 smithi167 ceph-mon[37140]: pgmap v8983: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:10:00.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:00 smithi138 ceph-mon[36954]: overall HEALTH_OK 2024-07-26T18:10:00.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:00 smithi100 ceph-mon[32599]: overall HEALTH_OK 2024-07-26T18:10:00.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:00 smithi167 ceph-mon[37140]: overall HEALTH_OK 2024-07-26T18:10:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:01 smithi138 ceph-mon[36954]: pgmap v8984: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:10:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:01 smithi100 ceph-mon[32599]: pgmap v8984: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:10:01.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:01 smithi167 ceph-mon[37140]: pgmap v8984: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:10:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:03 smithi138 ceph-mon[36954]: pgmap v8985: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:10:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:03 smithi100 ceph-mon[32599]: pgmap v8985: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:10:03.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:03 smithi167 ceph-mon[37140]: pgmap v8985: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:10:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:05 smithi138 ceph-mon[36954]: pgmap v8986: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:10:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:05 smithi100 ceph-mon[32599]: pgmap v8986: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:10:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:05 smithi167 ceph-mon[37140]: pgmap v8986: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:10:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:07 smithi138 ceph-mon[36954]: pgmap v8987: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:10:07.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:07 smithi100 ceph-mon[32599]: pgmap v8987: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:10:07.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:07 smithi167 ceph-mon[37140]: pgmap v8987: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:10:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:08 smithi138 ceph-mon[36954]: pgmap v8988: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:10:08.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:08 smithi138 ceph-mon[36954]: osdmap e608: 12 total, 12 up, 12 in 2024-07-26T18:10:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:08 smithi100 ceph-mon[32599]: pgmap v8988: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:10:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:08 smithi100 ceph-mon[32599]: osdmap e608: 12 total, 12 up, 12 in 2024-07-26T18:10:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:08 smithi167 ceph-mon[37140]: pgmap v8988: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:10:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:08 smithi167 ceph-mon[37140]: osdmap e608: 12 total, 12 up, 12 in 2024-07-26T18:10:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:10 smithi138 ceph-mon[36954]: pgmap v8990: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:10:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:10 smithi100 ceph-mon[32599]: pgmap v8990: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:10:11.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:10 smithi167 ceph-mon[37140]: pgmap v8990: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:10:12.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:10:12.143+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:10:12.145 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:10:12.145 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:10:12.146 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:10:12.146 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:10:12.146 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:11 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:10:12.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:10:12.144+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:10:12.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:10:12.144+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:10:12.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:10:12.144+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:10:12.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:10:12.144+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:10:12.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:10:12.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:10:12.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:10:12.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:10:12.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:11 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:10:12.248 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:10:12.248 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:10:12.248 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:10:12.248 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:10:12.249 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:11 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:10:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:12 smithi138 ceph-mon[36954]: pgmap v8991: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:10:13.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:12 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T18:10:13.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:12 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T18:10:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:12 smithi100 ceph-mon[32599]: pgmap v8991: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:10:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:12 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T18:10:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:12 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T18:10:13.267 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:12 smithi167 ceph-mon[37140]: pgmap v8991: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:10:13.267 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:12 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T18:10:13.267 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:12 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T18:10:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:14 smithi167 ceph-mon[37140]: pgmap v8992: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:10:15.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:14 smithi138 ceph-mon[36954]: pgmap v8992: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:10:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:14 smithi100 ceph-mon[32599]: pgmap v8992: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:10:16.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:10:16.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:10:16.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:10:16.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:10:16.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:10:16.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:10:16.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:16 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T18:10:16.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:16 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T18:10:16.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:10:16.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:10:16.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:10:16.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:10:16.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:10:16.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:10:16.497 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:16 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T18:10:16.497 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:16 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T18:10:16.497 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:10:16.497 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:10:16.497 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:10:16.497 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:16 smithi100 ceph-mon[32599]: pgmap v8993: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:10:16.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:10:16.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:10:16.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:10:16.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:10:16.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:10:16.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:10:16.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:16 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T18:10:16.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:16 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T18:10:16.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:10:16.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:10:16.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:10:16.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:10:16.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:10:16.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:10:16.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:16 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T18:10:16.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:16 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T18:10:16.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:10:16.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:10:16.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:10:16.526 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:16 smithi167 ceph-mon[37140]: pgmap v8993: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:10:16.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:10:16.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:10:16.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:10:16.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:10:16.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:10:16.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:10:16.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:16 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T18:10:16.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:16 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T18:10:16.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:10:16.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:10:16.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:10:16.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:10:16.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:10:16.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:10:16.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:16 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T18:10:16.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:16 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T18:10:16.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:10:16.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:10:16.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:10:16.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:16 smithi138 ceph-mon[36954]: pgmap v8993: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:10:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:19 smithi138 ceph-mon[36954]: pgmap v8994: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:10:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:19 smithi138 ceph-mon[36954]: osdmap e609: 12 total, 12 up, 12 in 2024-07-26T18:10:19.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:19 smithi100 ceph-mon[32599]: pgmap v8994: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:10:19.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:19 smithi100 ceph-mon[32599]: osdmap e609: 12 total, 12 up, 12 in 2024-07-26T18:10:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:19 smithi167 ceph-mon[37140]: pgmap v8994: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:10:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:19 smithi167 ceph-mon[37140]: osdmap e609: 12 total, 12 up, 12 in 2024-07-26T18:10:21.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:21 smithi138 ceph-mon[36954]: pgmap v8996: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:10:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:21 smithi100 ceph-mon[32599]: pgmap v8996: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:10:21.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:21 smithi167 ceph-mon[37140]: pgmap v8996: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:10:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:23 smithi138 ceph-mon[36954]: pgmap v8997: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:10:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:23 smithi100 ceph-mon[32599]: pgmap v8997: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:10:23.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:23 smithi167 ceph-mon[37140]: pgmap v8997: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:10:25.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:25 smithi138 ceph-mon[36954]: pgmap v8998: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:10:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:25 smithi100 ceph-mon[32599]: pgmap v8998: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:10:25.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:25 smithi167 ceph-mon[37140]: pgmap v8998: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:10:27.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:10:27.151+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:10:27.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:10:27.152+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:10:27.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:10:27.152+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:10:27.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:10:27.152+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:10:27.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:10:27.152+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:10:27.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:27 smithi138 ceph-mon[36954]: pgmap v8999: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:10:27.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:27 smithi100 ceph-mon[32599]: pgmap v8999: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:10:27.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:27 smithi167 ceph-mon[37140]: pgmap v8999: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:10:29.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:29 smithi138 ceph-mon[36954]: pgmap v9000: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:10:29.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:29 smithi100 ceph-mon[32599]: pgmap v9000: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:10:29.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:29 smithi167 ceph-mon[37140]: pgmap v9000: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:10:31.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:31 smithi138 ceph-mon[36954]: pgmap v9001: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:10:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:31 smithi100 ceph-mon[32599]: pgmap v9001: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:10:31.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:31 smithi167 ceph-mon[37140]: pgmap v9001: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:10:33.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:33 smithi138 ceph-mon[36954]: pgmap v9002: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:10:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:33 smithi100 ceph-mon[32599]: pgmap v9002: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:10:33.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:33 smithi167 ceph-mon[37140]: pgmap v9002: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:10:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:35 smithi138 ceph-mon[36954]: pgmap v9003: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:10:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:35 smithi100 ceph-mon[32599]: pgmap v9003: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:10:35.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:35 smithi167 ceph-mon[37140]: pgmap v9003: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:10:37.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:37 smithi138 ceph-mon[36954]: pgmap v9004: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:10:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:37 smithi100 ceph-mon[32599]: pgmap v9004: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:10:37.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:37 smithi167 ceph-mon[37140]: pgmap v9004: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:10:39.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:39 smithi138 ceph-mon[36954]: pgmap v9005: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:10:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:39 smithi100 ceph-mon[32599]: pgmap v9005: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:10:39.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:39 smithi167 ceph-mon[37140]: pgmap v9005: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:10:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:41 smithi138 ceph-mon[36954]: pgmap v9006: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:10:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:41 smithi100 ceph-mon[32599]: pgmap v9006: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:10:41.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:41 smithi167 ceph-mon[37140]: pgmap v9006: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:10:42.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:10:42.166+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:10:42.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:10:42.167+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:10:42.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:10:42.167+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:10:42.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:10:42.167+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:10:42.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:10:42.167+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:10:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:43 smithi138 ceph-mon[36954]: pgmap v9007: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:10:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:43 smithi100 ceph-mon[32599]: pgmap v9007: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:10:43.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:43 smithi167 ceph-mon[37140]: pgmap v9007: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:10:45.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:45 smithi138 ceph-mon[36954]: pgmap v9008: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:10:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:45 smithi100 ceph-mon[32599]: pgmap v9008: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:10:45.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:45 smithi167 ceph-mon[37140]: pgmap v9008: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:10:47.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:47 smithi138 ceph-mon[36954]: pgmap v9009: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:10:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:47 smithi100 ceph-mon[32599]: pgmap v9009: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:10:47.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:47 smithi167 ceph-mon[37140]: pgmap v9009: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:10:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:49 smithi138 ceph-mon[36954]: pgmap v9010: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:10:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:49 smithi100 ceph-mon[32599]: pgmap v9010: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:10:49.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:49 smithi167 ceph-mon[37140]: pgmap v9010: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:10:51.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:51 smithi138 ceph-mon[36954]: pgmap v9011: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:10:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:51 smithi100 ceph-mon[32599]: pgmap v9011: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:10:51.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:51 smithi167 ceph-mon[37140]: pgmap v9011: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:10:53.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:53 smithi138 ceph-mon[36954]: pgmap v9012: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:10:53.498 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:53 smithi100 ceph-mon[32599]: pgmap v9012: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:10:53.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:53 smithi167 ceph-mon[37140]: pgmap v9012: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:10:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:55 smithi138 ceph-mon[36954]: pgmap v9013: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:10:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:55 smithi100 ceph-mon[32599]: pgmap v9013: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:10:55.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:55 smithi167 ceph-mon[37140]: pgmap v9013: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:10:57.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:10:57.183+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:10:57.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:10:57.184+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:10:57.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:10:57.184+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:10:57.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:10:57.184+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:10:57.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:10:57.184+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:10:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:57 smithi138 ceph-mon[36954]: pgmap v9014: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:10:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:57 smithi100 ceph-mon[32599]: pgmap v9014: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:10:57.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:57 smithi167 ceph-mon[37140]: pgmap v9014: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:10:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:10:59 smithi138 ceph-mon[36954]: pgmap v9015: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:10:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:10:59 smithi100 ceph-mon[32599]: pgmap v9015: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:10:59.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:10:59 smithi167 ceph-mon[37140]: pgmap v9015: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:11:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:11:01 smithi138 ceph-mon[36954]: pgmap v9016: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:11:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:11:01 smithi100 ceph-mon[32599]: pgmap v9016: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:11:01.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:11:01 smithi167 ceph-mon[37140]: pgmap v9016: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:11:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:11:03 smithi138 ceph-mon[36954]: pgmap v9017: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:11:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:11:03 smithi100 ceph-mon[32599]: pgmap v9017: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:11:03.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:11:03 smithi167 ceph-mon[37140]: pgmap v9017: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:11:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:11:05 smithi138 ceph-mon[36954]: pgmap v9018: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:11:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:11:05 smithi100 ceph-mon[32599]: pgmap v9018: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:11:05.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:11:05 smithi167 ceph-mon[37140]: pgmap v9018: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:11:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:11:07 smithi138 ceph-mon[36954]: pgmap v9019: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:11:07.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:11:07 smithi100 ceph-mon[32599]: pgmap v9019: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:11:07.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:11:07 smithi167 ceph-mon[37140]: pgmap v9019: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:11:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:11:08 smithi100 ceph-mon[32599]: pgmap v9020: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:11:08.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:11:08 smithi100 ceph-mon[32599]: osdmap e610: 12 total, 12 up, 12 in 2024-07-26T18:11:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:11:08 smithi167 ceph-mon[37140]: pgmap v9020: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:11:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:11:08 smithi167 ceph-mon[37140]: osdmap e610: 12 total, 12 up, 12 in 2024-07-26T18:11:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:11:08 smithi138 ceph-mon[36954]: pgmap v9020: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:11:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:11:08 smithi138 ceph-mon[36954]: osdmap e610: 12 total, 12 up, 12 in 2024-07-26T18:11:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:11:10 smithi138 ceph-mon[36954]: pgmap v9022: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:11:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:11:10 smithi100 ceph-mon[32599]: pgmap v9022: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:11:11.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:11:10 smithi167 ceph-mon[37140]: pgmap v9022: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:11:12.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:11:12.200+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:11:12.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:11:12.200+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:11:12.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:11:12.200+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:11:12.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:11:12.200+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:11:12.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:11:12.200+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:11:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:11:12 smithi138 ceph-mon[36954]: pgmap v9023: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:11:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:11:12 smithi100 ceph-mon[32599]: pgmap v9023: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:11:13.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:11:12 smithi167 ceph-mon[37140]: pgmap v9023: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:11:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:11:14 smithi138 ceph-mon[36954]: pgmap v9024: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:11:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:11:14 smithi100 ceph-mon[32599]: pgmap v9024: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:11:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:11:14 smithi167 ceph-mon[37140]: pgmap v9024: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:11:16.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:11:15 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:11:16.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:11:15 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:11:16.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:11:15 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:11:16.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:11:15 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:11:16.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:11:15 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:11:16.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:11:15 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:11:16.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:11:15 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:11:16.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:11:15 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:11:16.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:11:15 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:11:16.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:11:15 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:11:16.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:11:15 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:11:16.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:11:15 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:11:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:11:16 smithi138 ceph-mon[36954]: pgmap v9025: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:11:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:11:16 smithi100 ceph-mon[32599]: pgmap v9025: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:11:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:11:16 smithi167 ceph-mon[37140]: pgmap v9025: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:11:19.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:11:19 smithi138 ceph-mon[36954]: pgmap v9026: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:11:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:11:19 smithi138 ceph-mon[36954]: osdmap e611: 12 total, 12 up, 12 in 2024-07-26T18:11:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:11:19 smithi100 ceph-mon[32599]: pgmap v9026: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:11:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:11:19 smithi100 ceph-mon[32599]: osdmap e611: 12 total, 12 up, 12 in 2024-07-26T18:11:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:11:19 smithi167 ceph-mon[37140]: pgmap v9026: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:11:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:11:19 smithi167 ceph-mon[37140]: osdmap e611: 12 total, 12 up, 12 in 2024-07-26T18:11:21.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:11:21 smithi138 ceph-mon[36954]: pgmap v9028: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:11:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:11:21 smithi100 ceph-mon[32599]: pgmap v9028: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:11:21.526 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:11:21 smithi167 ceph-mon[37140]: pgmap v9028: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:11:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:11:23 smithi138 ceph-mon[36954]: pgmap v9029: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:11:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:11:23 smithi100 ceph-mon[32599]: pgmap v9029: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:11:23.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:11:23 smithi167 ceph-mon[37140]: pgmap v9029: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:11:25.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:11:25 smithi138 ceph-mon[36954]: pgmap v9030: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:11:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:11:25 smithi100 ceph-mon[32599]: pgmap v9030: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:11:25.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:11:25 smithi167 ceph-mon[37140]: pgmap v9030: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:11:27.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:11:27.206+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:11:27.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:11:27.207+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:11:27.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:11:27.207+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:11:27.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:11:27.207+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:11:27.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:11:27.207+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:11:27.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:11:27 smithi138 ceph-mon[36954]: pgmap v9031: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:11:27.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:11:27 smithi100 ceph-mon[32599]: pgmap v9031: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:11:27.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:11:27 smithi167 ceph-mon[37140]: pgmap v9031: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:11:29.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:11:29 smithi138 ceph-mon[36954]: pgmap v9032: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:11:29.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:11:29 smithi100 ceph-mon[32599]: pgmap v9032: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:11:29.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:11:29 smithi167 ceph-mon[37140]: pgmap v9032: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:11:31.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:11:31 smithi138 ceph-mon[36954]: pgmap v9033: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:11:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:11:31 smithi100 ceph-mon[32599]: pgmap v9033: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:11:31.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:11:31 smithi167 ceph-mon[37140]: pgmap v9033: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:11:33.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:11:33 smithi138 ceph-mon[36954]: pgmap v9034: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:11:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:11:33 smithi100 ceph-mon[32599]: pgmap v9034: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:11:33.526 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:11:33 smithi167 ceph-mon[37140]: pgmap v9034: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:11:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:11:35 smithi138 ceph-mon[36954]: pgmap v9035: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:11:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:11:35 smithi100 ceph-mon[32599]: pgmap v9035: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:11:35.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:11:35 smithi167 ceph-mon[37140]: pgmap v9035: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:11:37.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:11:37 smithi138 ceph-mon[36954]: pgmap v9036: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:11:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:11:37 smithi100 ceph-mon[32599]: pgmap v9036: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:11:37.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:11:37 smithi167 ceph-mon[37140]: pgmap v9036: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:11:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:11:39 smithi138 ceph-mon[36954]: pgmap v9037: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:11:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:11:39 smithi100 ceph-mon[32599]: pgmap v9037: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:11:39.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:11:39 smithi167 ceph-mon[37140]: pgmap v9037: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:11:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:11:41 smithi138 ceph-mon[36954]: pgmap v9038: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:11:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:11:41 smithi100 ceph-mon[32599]: pgmap v9038: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:11:41.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:11:41 smithi167 ceph-mon[37140]: pgmap v9038: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:11:42.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:11:42.215+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:11:42.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:11:42.216+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:11:42.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:11:42.216+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:11:42.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:11:42.216+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:11:42.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:11:42.216+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:11:43.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:11:43 smithi138 ceph-mon[36954]: pgmap v9039: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:11:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:11:43 smithi100 ceph-mon[32599]: pgmap v9039: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:11:43.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:11:43 smithi167 ceph-mon[37140]: pgmap v9039: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:11:45.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:11:45 smithi138 ceph-mon[36954]: pgmap v9040: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T18:11:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:11:45 smithi100 ceph-mon[32599]: pgmap v9040: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T18:11:45.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:11:45 smithi167 ceph-mon[37140]: pgmap v9040: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T18:11:47.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:11:47 smithi138 ceph-mon[36954]: pgmap v9041: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:11:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:11:47 smithi100 ceph-mon[32599]: pgmap v9041: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:11:47.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:11:47 smithi167 ceph-mon[37140]: pgmap v9041: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:11:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:11:49 smithi138 ceph-mon[36954]: pgmap v9042: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:11:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:11:49 smithi100 ceph-mon[32599]: pgmap v9042: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:11:49.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:11:49 smithi167 ceph-mon[37140]: pgmap v9042: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:11:51.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:11:51 smithi138 ceph-mon[36954]: pgmap v9043: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:11:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:11:51 smithi100 ceph-mon[32599]: pgmap v9043: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:11:51.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:11:51 smithi167 ceph-mon[37140]: pgmap v9043: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:11:53.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:11:53 smithi138 ceph-mon[36954]: pgmap v9044: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:11:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:11:53 smithi100 ceph-mon[32599]: pgmap v9044: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:11:53.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:11:53 smithi167 ceph-mon[37140]: pgmap v9044: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:11:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:11:55 smithi138 ceph-mon[36954]: pgmap v9045: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:11:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:11:55 smithi100 ceph-mon[32599]: pgmap v9045: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:11:55.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:11:55 smithi167 ceph-mon[37140]: pgmap v9045: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:11:57.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:11:57.230+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:11:57.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:11:57.231+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:11:57.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:11:57.231+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:11:57.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:11:57.231+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:11:57.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:11:57.231+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:11:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:11:57 smithi138 ceph-mon[36954]: pgmap v9046: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:11:57.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:11:57 smithi100 ceph-mon[32599]: pgmap v9046: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:11:57.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:11:57 smithi167 ceph-mon[37140]: pgmap v9046: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:11:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:11:59 smithi138 ceph-mon[36954]: pgmap v9047: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:11:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:11:59 smithi100 ceph-mon[32599]: pgmap v9047: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:11:59.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:11:59 smithi167 ceph-mon[37140]: pgmap v9047: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:12:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:12:01 smithi138 ceph-mon[36954]: pgmap v9048: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:12:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:12:01 smithi100 ceph-mon[32599]: pgmap v9048: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:12:01.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:12:01 smithi167 ceph-mon[37140]: pgmap v9048: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:12:02.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:12:02 smithi100 ceph-mon[32599]: pgmap v9049: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:12:02.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:12:02 smithi167 ceph-mon[37140]: pgmap v9049: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:12:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:12:02 smithi138 ceph-mon[36954]: pgmap v9049: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:12:05.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:12:04 smithi138 ceph-mon[36954]: pgmap v9050: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:12:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:12:04 smithi100 ceph-mon[32599]: pgmap v9050: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:12:05.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:12:05.261+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c006690 con 0x7fe5a81087a0 2024-07-26T18:12:05.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:12:05.261+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T18:12:05.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:12:05.261+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 124 ==== auth_reply(proto 2 0 (0) Success) ==== 783+0+0 (secure 0 0 0) 0x7fe57c006690 con 0x7fe5a81087a0 2024-07-26T18:12:05.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:12:05.261+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe5a008c070 con 0x7fe5a81087a0 2024-07-26T18:12:05.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:12:05.261+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 124 ==== auth_reply(proto 2 0 (0) Success) ==== 612+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T18:12:05.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:12:05.261+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T18:12:05.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:12:05.261+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 125 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c070 con 0x7fe5a81087a0 2024-07-26T18:12:05.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:12:05.261+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 125 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T18:12:05.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:12:04 smithi167 ceph-mon[37140]: pgmap v9050: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:12:07.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:12:06 smithi138 ceph-mon[36954]: pgmap v9051: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:12:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:12:06 smithi100 ceph-mon[32599]: pgmap v9051: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:12:07.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:12:06 smithi167 ceph-mon[37140]: pgmap v9051: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:12:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:12:08 smithi138 ceph-mon[36954]: pgmap v9052: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:12:09.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:12:08 smithi138 ceph-mon[36954]: osdmap e612: 12 total, 12 up, 12 in 2024-07-26T18:12:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:12:08 smithi100 ceph-mon[32599]: pgmap v9052: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:12:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:12:08 smithi100 ceph-mon[32599]: osdmap e612: 12 total, 12 up, 12 in 2024-07-26T18:12:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:12:08 smithi167 ceph-mon[37140]: pgmap v9052: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:12:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:12:08 smithi167 ceph-mon[37140]: osdmap e612: 12 total, 12 up, 12 in 2024-07-26T18:12:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:12:10 smithi138 ceph-mon[36954]: pgmap v9054: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:12:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:12:10 smithi100 ceph-mon[32599]: pgmap v9054: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:12:11.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:12:10 smithi167 ceph-mon[37140]: pgmap v9054: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:12:12.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:12:12.236+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:12:12.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:12:12.237+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:12:12.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:12:12.237+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:12:12.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:12:12.237+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:12:12.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:12:12.237+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:12:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:12:12 smithi138 ceph-mon[36954]: pgmap v9055: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:12:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:12:12 smithi100 ceph-mon[32599]: pgmap v9055: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:12:13.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:12:12 smithi167 ceph-mon[37140]: pgmap v9055: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:12:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:12:14 smithi138 ceph-mon[36954]: pgmap v9056: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:12:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:12:14 smithi100 ceph-mon[32599]: pgmap v9056: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:12:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:12:14 smithi167 ceph-mon[37140]: pgmap v9056: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:12:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:12:16 smithi138 ceph-mon[36954]: pgmap v9057: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:12:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:12:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:12:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:12:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:12:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:12:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:12:17.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:12:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:12:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:12:16 smithi100 ceph-mon[32599]: pgmap v9057: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:12:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:12:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:12:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:12:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:12:17.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:12:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:12:17.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:12:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:12:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:12:16 smithi167 ceph-mon[37140]: pgmap v9057: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:12:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:12:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:12:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:12:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:12:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:12:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:12:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:12:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:12:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:12:19 smithi138 ceph-mon[36954]: pgmap v9058: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:12:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:12:19 smithi138 ceph-mon[36954]: osdmap e613: 12 total, 12 up, 12 in 2024-07-26T18:12:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:12:19 smithi100 ceph-mon[32599]: pgmap v9058: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:12:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:12:19 smithi100 ceph-mon[32599]: osdmap e613: 12 total, 12 up, 12 in 2024-07-26T18:12:19.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:12:19 smithi167 ceph-mon[37140]: pgmap v9058: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:12:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:12:19 smithi167 ceph-mon[37140]: osdmap e613: 12 total, 12 up, 12 in 2024-07-26T18:12:21.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:12:21 smithi138 ceph-mon[36954]: pgmap v9060: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:12:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:12:21 smithi100 ceph-mon[32599]: pgmap v9060: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:12:21.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:12:21 smithi167 ceph-mon[37140]: pgmap v9060: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:12:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:12:23 smithi138 ceph-mon[36954]: pgmap v9061: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:12:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:12:23 smithi100 ceph-mon[32599]: pgmap v9061: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:12:23.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:12:23 smithi167 ceph-mon[37140]: pgmap v9061: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:12:25.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:12:25 smithi138 ceph-mon[36954]: pgmap v9062: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:12:25.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:12:25 smithi100 ceph-mon[32599]: pgmap v9062: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:12:25.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:12:25 smithi167 ceph-mon[37140]: pgmap v9062: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:12:27.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:12:27.247+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:12:27.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:12:27.248+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:12:27.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:12:27.248+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:12:27.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:12:27.248+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:12:27.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:12:27.248+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:12:27.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:12:27 smithi138 ceph-mon[36954]: pgmap v9063: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:12:27.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:12:27 smithi100 ceph-mon[32599]: pgmap v9063: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:12:27.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:12:27 smithi167 ceph-mon[37140]: pgmap v9063: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:12:29.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:12:29 smithi138 ceph-mon[36954]: pgmap v9064: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:12:29.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:12:29 smithi100 ceph-mon[32599]: pgmap v9064: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:12:29.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:12:29 smithi167 ceph-mon[37140]: pgmap v9064: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:12:31.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:12:31 smithi138 ceph-mon[36954]: pgmap v9065: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:12:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:12:31 smithi100 ceph-mon[32599]: pgmap v9065: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:12:31.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:12:31 smithi167 ceph-mon[37140]: pgmap v9065: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:12:33.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:12:33 smithi138 ceph-mon[36954]: pgmap v9066: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:12:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:12:33 smithi100 ceph-mon[32599]: pgmap v9066: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:12:33.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:12:33 smithi167 ceph-mon[37140]: pgmap v9066: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:12:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:12:35 smithi138 ceph-mon[36954]: pgmap v9067: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:12:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:12:35 smithi100 ceph-mon[32599]: pgmap v9067: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:12:35.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:12:35 smithi167 ceph-mon[37140]: pgmap v9067: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:12:37.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:12:37 smithi138 ceph-mon[36954]: pgmap v9068: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:12:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:12:37 smithi100 ceph-mon[32599]: pgmap v9068: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:12:37.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:12:37 smithi167 ceph-mon[37140]: pgmap v9068: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:12:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:12:39 smithi138 ceph-mon[36954]: pgmap v9069: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:12:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:12:39 smithi100 ceph-mon[32599]: pgmap v9069: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:12:39.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:12:39 smithi167 ceph-mon[37140]: pgmap v9069: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:12:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:12:41 smithi138 ceph-mon[36954]: pgmap v9070: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:12:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:12:41 smithi100 ceph-mon[32599]: pgmap v9070: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:12:41.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:12:41 smithi167 ceph-mon[37140]: pgmap v9070: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:12:42.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:12:42.263+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:12:42.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:12:42.264+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:12:42.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:12:42.264+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:12:42.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:12:42.264+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:12:42.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:12:42.264+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:12:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:12:43 smithi138 ceph-mon[36954]: pgmap v9071: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:12:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:12:43 smithi100 ceph-mon[32599]: pgmap v9071: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:12:43.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:12:43 smithi167 ceph-mon[37140]: pgmap v9071: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:12:45.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:12:45 smithi138 ceph-mon[36954]: pgmap v9072: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:12:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:12:45 smithi100 ceph-mon[32599]: pgmap v9072: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:12:45.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:12:45 smithi167 ceph-mon[37140]: pgmap v9072: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:12:47.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:12:47 smithi138 ceph-mon[36954]: pgmap v9073: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:12:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:12:47 smithi100 ceph-mon[32599]: pgmap v9073: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:12:47.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:12:47 smithi167 ceph-mon[37140]: pgmap v9073: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:12:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:12:49 smithi138 ceph-mon[36954]: pgmap v9074: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:12:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:12:49 smithi100 ceph-mon[32599]: pgmap v9074: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:12:49.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:12:49 smithi167 ceph-mon[37140]: pgmap v9074: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:12:51.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:12:51 smithi138 ceph-mon[36954]: pgmap v9075: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:12:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:12:51 smithi100 ceph-mon[32599]: pgmap v9075: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:12:51.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:12:51 smithi167 ceph-mon[37140]: pgmap v9075: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:12:53.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:12:53 smithi138 ceph-mon[36954]: pgmap v9076: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:12:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:12:53 smithi100 ceph-mon[32599]: pgmap v9076: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:12:53.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:12:53 smithi167 ceph-mon[37140]: pgmap v9076: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:12:54.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:12:54 smithi100 ceph-mon[32599]: pgmap v9077: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:12:54.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:12:54 smithi167 ceph-mon[37140]: pgmap v9077: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:12:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:12:54 smithi138 ceph-mon[36954]: pgmap v9077: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:12:57.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:12:56 smithi138 ceph-mon[36954]: pgmap v9078: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:12:57.197 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:12:56 smithi100 ceph-mon[32599]: pgmap v9078: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:12:57.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:12:56 smithi167 ceph-mon[37140]: pgmap v9078: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:12:57.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:12:57.268+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:12:57.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:12:57.269+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:12:57.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:12:57.269+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:12:57.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:12:57.269+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:12:57.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:12:57.269+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:12:59.194 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:12:58 smithi138 ceph-mon[36954]: pgmap v9079: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:12:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:12:58 smithi100 ceph-mon[32599]: pgmap v9079: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:12:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:12:58 smithi167 ceph-mon[37140]: pgmap v9079: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:13:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:00 smithi138 ceph-mon[36954]: pgmap v9080: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:13:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:00 smithi100 ceph-mon[32599]: pgmap v9080: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:13:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:00 smithi167 ceph-mon[37140]: pgmap v9080: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:13:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:02 smithi100 ceph-mon[32599]: pgmap v9081: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:13:03.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:02 smithi167 ceph-mon[37140]: pgmap v9081: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:13:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:02 smithi138 ceph-mon[36954]: pgmap v9081: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:13:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:04 smithi100 ceph-mon[32599]: pgmap v9082: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:13:05.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:04 smithi167 ceph-mon[37140]: pgmap v9082: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:13:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:04 smithi138 ceph-mon[36954]: pgmap v9082: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:13:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:06 smithi100 ceph-mon[32599]: pgmap v9083: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:13:07.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:06 smithi167 ceph-mon[37140]: pgmap v9083: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:13:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:06 smithi138 ceph-mon[36954]: pgmap v9083: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:13:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:08 smithi100 ceph-mon[32599]: pgmap v9084: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:13:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:08 smithi100 ceph-mon[32599]: osdmap e614: 12 total, 12 up, 12 in 2024-07-26T18:13:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:08 smithi167 ceph-mon[37140]: pgmap v9084: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:13:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:08 smithi167 ceph-mon[37140]: osdmap e614: 12 total, 12 up, 12 in 2024-07-26T18:13:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:08 smithi138 ceph-mon[36954]: pgmap v9084: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:13:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:08 smithi138 ceph-mon[36954]: osdmap e614: 12 total, 12 up, 12 in 2024-07-26T18:13:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:10 smithi100 ceph-mon[32599]: pgmap v9086: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:13:11.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:10 smithi167 ceph-mon[37140]: pgmap v9086: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:13:11.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:10 smithi138 ceph-mon[36954]: pgmap v9086: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:13:12.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:13:12.286+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:13:12.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:13:12.286+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:13:12.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:13:12.286+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:13:12.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:13:12.286+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:13:12.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:13:12.286+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:13:13.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:12 smithi100 ceph-mon[32599]: pgmap v9087: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:13:13.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:12 smithi167 ceph-mon[37140]: pgmap v9087: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:13:13.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:12 smithi138 ceph-mon[36954]: pgmap v9087: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:13:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:14 smithi100 ceph-mon[32599]: pgmap v9088: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:13:15.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:14 smithi167 ceph-mon[37140]: pgmap v9088: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:13:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:14 smithi138 ceph-mon[36954]: pgmap v9088: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:13:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:16 smithi100 ceph-mon[32599]: pgmap v9089: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:13:17.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:13:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:16 smithi167 ceph-mon[37140]: pgmap v9089: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:13:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:13:17.307 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:16 smithi138 ceph-mon[36954]: pgmap v9089: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:13:17.307 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:13:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:19 smithi167 ceph-mon[37140]: pgmap v9090: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:13:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:13:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:13:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:13:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:13:19.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:13:19.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:13:19.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:19 smithi167 ceph-mon[37140]: osdmap e615: 12 total, 12 up, 12 in 2024-07-26T18:13:19.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:13:19.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:13:19.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:13:19.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:19 smithi138 ceph-mon[36954]: pgmap v9090: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:13:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:13:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:13:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:13:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:13:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:13:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:13:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:19 smithi138 ceph-mon[36954]: osdmap e615: 12 total, 12 up, 12 in 2024-07-26T18:13:19.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:13:19.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:13:19.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:13:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:19 smithi100 ceph-mon[32599]: pgmap v9090: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:13:19.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:13:19.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:13:19.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:13:19.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:13:19.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:13:19.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:13:19.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:19 smithi100 ceph-mon[32599]: osdmap e615: 12 total, 12 up, 12 in 2024-07-26T18:13:19.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:13:19.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:13:19.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:13:21.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:21 smithi167 ceph-mon[37140]: pgmap v9092: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:13:21.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:21 smithi138 ceph-mon[36954]: pgmap v9092: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:13:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:21 smithi100 ceph-mon[32599]: pgmap v9092: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:13:23.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:23 smithi167 ceph-mon[37140]: pgmap v9093: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:13:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:23 smithi138 ceph-mon[36954]: pgmap v9093: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:13:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:23 smithi100 ceph-mon[32599]: pgmap v9093: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:13:25.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:25 smithi167 ceph-mon[37140]: pgmap v9094: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:13:25.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:25 smithi138 ceph-mon[36954]: pgmap v9094: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:13:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:25 smithi100 ceph-mon[32599]: pgmap v9094: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:13:27.296 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:27 smithi100 ceph-mon[32599]: pgmap v9095: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:13:27.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:13:27.301+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:13:27.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:13:27.301+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:13:27.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:13:27.301+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:13:27.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:13:27.301+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:13:27.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:13:27.302+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:13:27.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:27 smithi138 ceph-mon[36954]: pgmap v9095: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:13:27.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:27 smithi167 ceph-mon[37140]: pgmap v9095: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:13:29.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:29 smithi138 ceph-mon[36954]: pgmap v9096: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:13:29.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:29 smithi100 ceph-mon[32599]: pgmap v9096: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:13:29.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:29 smithi167 ceph-mon[37140]: pgmap v9096: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:13:31.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:31 smithi138 ceph-mon[36954]: pgmap v9097: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:13:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:31 smithi100 ceph-mon[32599]: pgmap v9097: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:13:31.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:31 smithi167 ceph-mon[37140]: pgmap v9097: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:13:33.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:33 smithi138 ceph-mon[36954]: pgmap v9098: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:13:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:33 smithi100 ceph-mon[32599]: pgmap v9098: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:13:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:33 smithi167 ceph-mon[37140]: pgmap v9098: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:13:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:35 smithi138 ceph-mon[36954]: pgmap v9099: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:13:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:35 smithi100 ceph-mon[32599]: pgmap v9099: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:13:35.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:35 smithi167 ceph-mon[37140]: pgmap v9099: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:13:37.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:37 smithi138 ceph-mon[36954]: pgmap v9100: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:13:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:37 smithi100 ceph-mon[32599]: pgmap v9100: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:13:37.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:37 smithi167 ceph-mon[37140]: pgmap v9100: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:13:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:39 smithi138 ceph-mon[36954]: pgmap v9101: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:13:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:39 smithi100 ceph-mon[32599]: pgmap v9101: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:13:39.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:39 smithi167 ceph-mon[37140]: pgmap v9101: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:13:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:41 smithi138 ceph-mon[36954]: pgmap v9102: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:13:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:41 smithi100 ceph-mon[32599]: pgmap v9102: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:13:41.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:41 smithi167 ceph-mon[37140]: pgmap v9102: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:13:42.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:13:42.317+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:13:42.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:13:42.318+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:13:42.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:13:42.318+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:13:42.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:13:42.318+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:13:42.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:13:42.318+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:13:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:43 smithi138 ceph-mon[36954]: pgmap v9103: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:13:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:43 smithi100 ceph-mon[32599]: pgmap v9103: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:13:43.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:43 smithi167 ceph-mon[37140]: pgmap v9103: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:13:45.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:45 smithi138 ceph-mon[36954]: pgmap v9104: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:13:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:45 smithi100 ceph-mon[32599]: pgmap v9104: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:13:45.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:45 smithi167 ceph-mon[37140]: pgmap v9104: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:13:47.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:47 smithi138 ceph-mon[36954]: pgmap v9105: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:13:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:47 smithi100 ceph-mon[32599]: pgmap v9105: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:13:47.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:47 smithi167 ceph-mon[37140]: pgmap v9105: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:13:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:49 smithi138 ceph-mon[36954]: pgmap v9106: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:13:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:49 smithi100 ceph-mon[32599]: pgmap v9106: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:13:49.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:49 smithi167 ceph-mon[37140]: pgmap v9106: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:13:51.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:51 smithi138 ceph-mon[36954]: pgmap v9107: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:13:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:51 smithi100 ceph-mon[32599]: pgmap v9107: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:13:51.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:51 smithi167 ceph-mon[37140]: pgmap v9107: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:13:53.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:53 smithi138 ceph-mon[36954]: pgmap v9108: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:13:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:53 smithi100 ceph-mon[32599]: pgmap v9108: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:13:53.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:53 smithi167 ceph-mon[37140]: pgmap v9108: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:13:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:55 smithi138 ceph-mon[36954]: pgmap v9109: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T18:13:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:55 smithi100 ceph-mon[32599]: pgmap v9109: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T18:13:55.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:55 smithi167 ceph-mon[37140]: pgmap v9109: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T18:13:57.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:13:57.329+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:13:57.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:13:57.330+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:13:57.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:13:57.330+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:13:57.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:13:57.330+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:13:57.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:13:57.330+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:13:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:57 smithi138 ceph-mon[36954]: pgmap v9110: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:13:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:57 smithi100 ceph-mon[32599]: pgmap v9110: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:13:57.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:57 smithi167 ceph-mon[37140]: pgmap v9110: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:13:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:13:59 smithi138 ceph-mon[36954]: pgmap v9111: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:13:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:13:59 smithi100 ceph-mon[32599]: pgmap v9111: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:13:59.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:13:59 smithi167 ceph-mon[37140]: pgmap v9111: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:14:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:01 smithi138 ceph-mon[36954]: pgmap v9112: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:14:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:01 smithi100 ceph-mon[32599]: pgmap v9112: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:14:01.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:01 smithi167 ceph-mon[37140]: pgmap v9112: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:14:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:03 smithi138 ceph-mon[36954]: pgmap v9113: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:14:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:03 smithi100 ceph-mon[32599]: pgmap v9113: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:14:03.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:03 smithi167 ceph-mon[37140]: pgmap v9113: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:14:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:05 smithi138 ceph-mon[36954]: pgmap v9114: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:14:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:05 smithi100 ceph-mon[32599]: pgmap v9114: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:14:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:05 smithi167 ceph-mon[37140]: pgmap v9114: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:14:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:07 smithi138 ceph-mon[36954]: pgmap v9115: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:14:07.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:07 smithi100 ceph-mon[32599]: pgmap v9115: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:14:07.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:07 smithi167 ceph-mon[37140]: pgmap v9115: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:14:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:08 smithi138 ceph-mon[36954]: osdmap e616: 12 total, 12 up, 12 in 2024-07-26T18:14:08.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:08 smithi138 ceph-mon[36954]: pgmap v9117: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:14:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:08 smithi100 ceph-mon[32599]: osdmap e616: 12 total, 12 up, 12 in 2024-07-26T18:14:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:08 smithi100 ceph-mon[32599]: pgmap v9117: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:14:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:08 smithi167 ceph-mon[37140]: osdmap e616: 12 total, 12 up, 12 in 2024-07-26T18:14:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:08 smithi167 ceph-mon[37140]: pgmap v9117: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:14:11.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:10 smithi100 ceph-mon[32599]: pgmap v9118: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:14:11.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:10 smithi167 ceph-mon[37140]: pgmap v9118: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:14:11.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:10 smithi138 ceph-mon[36954]: pgmap v9118: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:14:12.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:14:12.337+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:14:12.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:14:12.338+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:14:12.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:14:12.338+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:14:12.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:14:12.338+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:14:12.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:14:12.338+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:14:13.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:12 smithi100 ceph-mon[32599]: pgmap v9119: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:14:13.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:12 smithi167 ceph-mon[37140]: pgmap v9119: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:14:13.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:12 smithi138 ceph-mon[36954]: pgmap v9119: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:14:15.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:14 smithi100 ceph-mon[32599]: pgmap v9120: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:14:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:14 smithi167 ceph-mon[37140]: pgmap v9120: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:14:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:14 smithi138 ceph-mon[36954]: pgmap v9120: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:14:17.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:16 smithi100 ceph-mon[32599]: pgmap v9121: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:14:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:16 smithi167 ceph-mon[37140]: pgmap v9121: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:14:17.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:16 smithi138 ceph-mon[36954]: pgmap v9121: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:14:19.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:19 smithi138 ceph-mon[36954]: pgmap v9122: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 190 B/s wr, 6 op/s 2024-07-26T18:14:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:19 smithi138 ceph-mon[36954]: osdmap e617: 12 total, 12 up, 12 in 2024-07-26T18:14:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:14:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:14:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:14:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:14:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:14:19.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:14:19.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:19 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:14:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:19 smithi100 ceph-mon[32599]: pgmap v9122: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 190 B/s wr, 6 op/s 2024-07-26T18:14:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:19 smithi100 ceph-mon[32599]: osdmap e617: 12 total, 12 up, 12 in 2024-07-26T18:14:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:14:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:14:19.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:14:19.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:14:19.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:14:19.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:14:19.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:19 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:14:19.526 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:19 smithi167 ceph-mon[37140]: pgmap v9122: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 190 B/s wr, 6 op/s 2024-07-26T18:14:19.526 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:19 smithi167 ceph-mon[37140]: osdmap e617: 12 total, 12 up, 12 in 2024-07-26T18:14:19.527 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:14:19.527 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:14:19.527 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:14:19.527 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:14:19.528 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:14:19.528 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:14:19.528 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:19 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:14:20.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:20 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:14:20.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:20 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:14:20.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:20 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:14:20.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:20 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:14:20.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:20 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:14:20.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:20 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:14:20.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:20 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:14:20.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:20 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:14:20.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:20 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:14:21.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:21 smithi138 ceph-mon[36954]: pgmap v9124: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:14:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:21 smithi100 ceph-mon[32599]: pgmap v9124: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:14:21.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:21 smithi167 ceph-mon[37140]: pgmap v9124: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:14:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:23 smithi138 ceph-mon[36954]: pgmap v9125: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:14:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:23 smithi100 ceph-mon[32599]: pgmap v9125: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:14:23.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:23 smithi167 ceph-mon[37140]: pgmap v9125: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:14:25.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:25 smithi138 ceph-mon[36954]: pgmap v9126: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:14:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:25 smithi100 ceph-mon[32599]: pgmap v9126: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:14:25.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:25 smithi167 ceph-mon[37140]: pgmap v9126: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:14:27.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:14:27.354+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:14:27.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:14:27.354+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:14:27.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:14:27.354+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:14:27.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:14:27.354+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:14:27.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:14:27.354+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:14:27.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:27 smithi138 ceph-mon[36954]: pgmap v9127: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:14:27.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:27 smithi100 ceph-mon[32599]: pgmap v9127: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:14:27.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:27 smithi167 ceph-mon[37140]: pgmap v9127: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:14:29.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:29 smithi138 ceph-mon[36954]: pgmap v9128: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:14:29.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:29 smithi100 ceph-mon[32599]: pgmap v9128: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:14:29.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:29 smithi167 ceph-mon[37140]: pgmap v9128: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:14:31.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:31 smithi138 ceph-mon[36954]: pgmap v9129: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:14:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:31 smithi100 ceph-mon[32599]: pgmap v9129: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:14:31.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:31 smithi167 ceph-mon[37140]: pgmap v9129: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:14:33.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:33 smithi138 ceph-mon[36954]: pgmap v9130: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:14:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:33 smithi100 ceph-mon[32599]: pgmap v9130: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:14:33.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:33 smithi167 ceph-mon[37140]: pgmap v9130: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:14:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:35 smithi138 ceph-mon[36954]: pgmap v9131: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:14:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:35 smithi100 ceph-mon[32599]: pgmap v9131: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:14:35.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:35 smithi167 ceph-mon[37140]: pgmap v9131: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:14:37.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:37 smithi138 ceph-mon[36954]: pgmap v9132: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:14:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:37 smithi100 ceph-mon[32599]: pgmap v9132: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:14:37.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:37 smithi167 ceph-mon[37140]: pgmap v9132: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:14:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:39 smithi138 ceph-mon[36954]: pgmap v9133: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:14:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:39 smithi100 ceph-mon[32599]: pgmap v9133: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:14:39.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:39 smithi167 ceph-mon[37140]: pgmap v9133: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:14:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:41 smithi138 ceph-mon[36954]: pgmap v9134: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:14:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:41 smithi100 ceph-mon[32599]: pgmap v9134: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:14:41.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:41 smithi167 ceph-mon[37140]: pgmap v9134: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:14:42.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:14:42.370+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:14:42.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:14:42.370+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:14:42.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:14:42.370+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:14:42.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:14:42.370+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:14:42.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:14:42.370+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:14:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:43 smithi138 ceph-mon[36954]: pgmap v9135: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:14:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:43 smithi100 ceph-mon[32599]: pgmap v9135: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:14:43.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:43 smithi167 ceph-mon[37140]: pgmap v9135: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:14:45.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:45 smithi138 ceph-mon[36954]: pgmap v9136: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:14:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:45 smithi100 ceph-mon[32599]: pgmap v9136: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:14:45.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:45 smithi167 ceph-mon[37140]: pgmap v9136: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:14:47.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:47 smithi138 ceph-mon[36954]: pgmap v9137: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:14:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:47 smithi100 ceph-mon[32599]: pgmap v9137: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:14:47.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:47 smithi167 ceph-mon[37140]: pgmap v9137: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:14:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:49 smithi138 ceph-mon[36954]: pgmap v9138: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:14:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:49 smithi100 ceph-mon[32599]: pgmap v9138: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:14:49.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:49 smithi167 ceph-mon[37140]: pgmap v9138: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:14:50.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:50 smithi138 ceph-mon[36954]: pgmap v9139: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:14:50.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:50 smithi100 ceph-mon[32599]: pgmap v9139: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:14:50.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:50 smithi167 ceph-mon[37140]: pgmap v9139: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:14:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:52 smithi100 ceph-mon[32599]: pgmap v9140: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:14:53.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:52 smithi167 ceph-mon[37140]: pgmap v9140: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:14:53.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:52 smithi138 ceph-mon[36954]: pgmap v9140: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:14:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:54 smithi100 ceph-mon[32599]: pgmap v9141: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:14:55.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:54 smithi167 ceph-mon[37140]: pgmap v9141: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:14:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:54 smithi138 ceph-mon[36954]: pgmap v9141: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:14:57.243 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:56 smithi138 ceph-mon[36954]: pgmap v9142: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:14:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:56 smithi100 ceph-mon[32599]: pgmap v9142: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:14:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:56 smithi167 ceph-mon[37140]: pgmap v9142: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:14:57.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:14:57.386+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:14:57.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:14:57.387+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:14:57.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:14:57.387+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:14:57.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:14:57.387+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:14:57.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:14:57.387+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:14:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:14:58 smithi100 ceph-mon[32599]: pgmap v9143: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:14:59.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:14:58 smithi167 ceph-mon[37140]: pgmap v9143: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:14:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:14:58 smithi138 ceph-mon[36954]: pgmap v9143: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:15:01.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:15:00 smithi100 ceph-mon[32599]: pgmap v9144: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:15:01.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:15:00 smithi167 ceph-mon[37140]: pgmap v9144: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:15:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:15:00 smithi138 ceph-mon[36954]: pgmap v9144: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:15:03.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:15:02 smithi100 ceph-mon[32599]: pgmap v9145: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:15:03.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:15:02 smithi167 ceph-mon[37140]: pgmap v9145: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:15:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:15:02 smithi138 ceph-mon[36954]: pgmap v9145: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:15:05.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:15:04 smithi100 ceph-mon[32599]: pgmap v9146: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:15:05.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:15:04 smithi167 ceph-mon[37140]: pgmap v9146: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:15:05.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:15:05.286+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T18:15:05.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:15:05.286+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c003690 con 0x7fe5a81087a0 2024-07-26T18:15:05.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:15:05.287+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 126 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T18:15:05.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:15:05.287+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 126 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c020 con 0x7fe5a81087a0 2024-07-26T18:15:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:15:04 smithi138 ceph-mon[36954]: pgmap v9146: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:15:07.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:15:06 smithi100 ceph-mon[32599]: pgmap v9147: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:15:07.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:15:06 smithi167 ceph-mon[37140]: pgmap v9147: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:15:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:15:06 smithi138 ceph-mon[36954]: pgmap v9147: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:15:09.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:15:09 smithi167 ceph-mon[37140]: pgmap v9148: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:15:09.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:15:09 smithi167 ceph-mon[37140]: osdmap e618: 12 total, 12 up, 12 in 2024-07-26T18:15:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:15:09 smithi138 ceph-mon[36954]: pgmap v9148: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:15:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:15:09 smithi138 ceph-mon[36954]: osdmap e618: 12 total, 12 up, 12 in 2024-07-26T18:15:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:15:09 smithi100 ceph-mon[32599]: pgmap v9148: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:15:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:15:09 smithi100 ceph-mon[32599]: osdmap e618: 12 total, 12 up, 12 in 2024-07-26T18:15:11.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:15:11 smithi167 ceph-mon[37140]: pgmap v9150: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:15:11.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:15:11 smithi138 ceph-mon[36954]: pgmap v9150: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:15:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:15:11 smithi100 ceph-mon[32599]: pgmap v9150: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:15:12.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:15:12.393+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:15:12.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:15:12.393+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:15:12.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:15:12.393+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:15:12.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:15:12.393+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:15:12.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:15:12.393+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:15:13.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:15:13 smithi167 ceph-mon[37140]: pgmap v9151: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:15:13.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:15:13 smithi138 ceph-mon[36954]: pgmap v9151: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:15:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:15:13 smithi100 ceph-mon[32599]: pgmap v9151: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:15:15.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:15:15 smithi167 ceph-mon[37140]: pgmap v9152: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:15:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:15:15 smithi138 ceph-mon[36954]: pgmap v9152: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:15:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:15:15 smithi100 ceph-mon[32599]: pgmap v9152: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:15:17.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:15:17 smithi167 ceph-mon[37140]: pgmap v9153: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:15:17.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:15:17 smithi138 ceph-mon[36954]: pgmap v9153: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:15:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:15:17 smithi100 ceph-mon[32599]: pgmap v9153: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:15:19.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:15:19 smithi138 ceph-mon[36954]: pgmap v9154: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:15:19.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:15:19 smithi138 ceph-mon[36954]: osdmap e619: 12 total, 12 up, 12 in 2024-07-26T18:15:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:15:19 smithi100 ceph-mon[32599]: pgmap v9154: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:15:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:15:19 smithi100 ceph-mon[32599]: osdmap e619: 12 total, 12 up, 12 in 2024-07-26T18:15:19.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:15:19 smithi167 ceph-mon[37140]: pgmap v9154: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:15:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:15:19 smithi167 ceph-mon[37140]: osdmap e619: 12 total, 12 up, 12 in 2024-07-26T18:15:20.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:15:20 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:15:20.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:15:20 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:15:20.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:15:20 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:15:20.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:15:20 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:15:20.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:15:20 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:15:20.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:15:20 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:15:20.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:15:20 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:15:20.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:15:20 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:15:20.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:15:20 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:15:20.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:15:20 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:15:20.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:15:20 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:15:20.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:15:20 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:15:21.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:15:21 smithi138 ceph-mon[36954]: pgmap v9156: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:15:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:15:21 smithi100 ceph-mon[32599]: pgmap v9156: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:15:21.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:15:21 smithi167 ceph-mon[37140]: pgmap v9156: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:15:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:15:23 smithi138 ceph-mon[36954]: pgmap v9157: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:15:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:15:23 smithi100 ceph-mon[32599]: pgmap v9157: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:15:23.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:15:23 smithi167 ceph-mon[37140]: pgmap v9157: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:15:25.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:15:25 smithi138 ceph-mon[36954]: pgmap v9158: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:15:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:15:25 smithi100 ceph-mon[32599]: pgmap v9158: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:15:25.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:15:25 smithi167 ceph-mon[37140]: pgmap v9158: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:15:27.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:15:27.399+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:15:27.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:15:27.400+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:15:27.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:15:27.400+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:15:27.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:15:27.400+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:15:27.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:15:27.400+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:15:27.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:15:27 smithi138 ceph-mon[36954]: pgmap v9159: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:15:27.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:15:27 smithi100 ceph-mon[32599]: pgmap v9159: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:15:27.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:15:27 smithi167 ceph-mon[37140]: pgmap v9159: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:15:29.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:15:29 smithi138 ceph-mon[36954]: pgmap v9160: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:15:29.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:15:29 smithi100 ceph-mon[32599]: pgmap v9160: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:15:29.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:15:29 smithi167 ceph-mon[37140]: pgmap v9160: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:15:31.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:15:31 smithi138 ceph-mon[36954]: pgmap v9161: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:15:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:15:31 smithi100 ceph-mon[32599]: pgmap v9161: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:15:31.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:15:31 smithi167 ceph-mon[37140]: pgmap v9161: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:15:33.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:15:33 smithi138 ceph-mon[36954]: pgmap v9162: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:15:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:15:33 smithi100 ceph-mon[32599]: pgmap v9162: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:15:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:15:33 smithi167 ceph-mon[37140]: pgmap v9162: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:15:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:15:35 smithi138 ceph-mon[36954]: pgmap v9163: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:15:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:15:35 smithi100 ceph-mon[32599]: pgmap v9163: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:15:35.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:15:35 smithi167 ceph-mon[37140]: pgmap v9163: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:15:37.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:15:37 smithi138 ceph-mon[36954]: pgmap v9164: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:15:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:15:37 smithi100 ceph-mon[32599]: pgmap v9164: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:15:37.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:15:37 smithi167 ceph-mon[37140]: pgmap v9164: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:15:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:15:39 smithi138 ceph-mon[36954]: pgmap v9165: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:15:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:15:39 smithi100 ceph-mon[32599]: pgmap v9165: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:15:39.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:15:39 smithi167 ceph-mon[37140]: pgmap v9165: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:15:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:15:41 smithi138 ceph-mon[36954]: pgmap v9166: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:15:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:15:41 smithi100 ceph-mon[32599]: pgmap v9166: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:15:41.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:15:41 smithi167 ceph-mon[37140]: pgmap v9166: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:15:42.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:15:42.416+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:15:42.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:15:42.416+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:15:42.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:15:42.416+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:15:42.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:15:42.416+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:15:42.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:15:42.416+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:15:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:15:43 smithi138 ceph-mon[36954]: pgmap v9167: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:15:43.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:15:43 smithi100 ceph-mon[32599]: pgmap v9167: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:15:43.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:15:43 smithi167 ceph-mon[37140]: pgmap v9167: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:15:45.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:15:45 smithi138 ceph-mon[36954]: pgmap v9168: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:15:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:15:45 smithi100 ceph-mon[32599]: pgmap v9168: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:15:45.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:15:45 smithi167 ceph-mon[37140]: pgmap v9168: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:15:46.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:15:46 smithi100 ceph-mon[32599]: pgmap v9169: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:15:46.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:15:46 smithi167 ceph-mon[37140]: pgmap v9169: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:15:46.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:15:46 smithi138 ceph-mon[36954]: pgmap v9169: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:15:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:15:48 smithi100 ceph-mon[32599]: pgmap v9170: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:15:49.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:15:48 smithi167 ceph-mon[37140]: pgmap v9170: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:15:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:15:48 smithi138 ceph-mon[36954]: pgmap v9170: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:15:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:15:50 smithi100 ceph-mon[32599]: pgmap v9171: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:15:51.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:15:50 smithi167 ceph-mon[37140]: pgmap v9171: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:15:51.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:15:50 smithi138 ceph-mon[36954]: pgmap v9171: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:15:53.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:15:52 smithi167 ceph-mon[37140]: pgmap v9172: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:15:53.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:15:52 smithi138 ceph-mon[36954]: pgmap v9172: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:15:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:15:52 smithi100 ceph-mon[32599]: pgmap v9172: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:15:55.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:15:54 smithi167 ceph-mon[37140]: pgmap v9173: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:15:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:15:54 smithi138 ceph-mon[36954]: pgmap v9173: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:15:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:15:54 smithi100 ceph-mon[32599]: pgmap v9173: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:15:57.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:15:57 smithi167 ceph-mon[37140]: pgmap v9174: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:15:57.296 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:15:57 smithi100 ceph-mon[32599]: pgmap v9174: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:15:57.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:15:57.432+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:15:57.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:15:57.432+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:15:57.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:15:57.432+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:15:57.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:15:57.432+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:15:57.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:15:57.432+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:15:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:15:57 smithi138 ceph-mon[36954]: pgmap v9174: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:15:59.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:15:59 smithi167 ceph-mon[37140]: pgmap v9175: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:15:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:15:59 smithi138 ceph-mon[36954]: pgmap v9175: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:15:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:15:59 smithi100 ceph-mon[32599]: pgmap v9175: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:16:01.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:16:01 smithi167 ceph-mon[37140]: pgmap v9176: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:16:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:16:01 smithi138 ceph-mon[36954]: pgmap v9176: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:16:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:16:01 smithi100 ceph-mon[32599]: pgmap v9176: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:16:03.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:16:03 smithi167 ceph-mon[37140]: pgmap v9177: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:16:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:16:03 smithi138 ceph-mon[36954]: pgmap v9177: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:16:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:16:03 smithi100 ceph-mon[32599]: pgmap v9177: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:16:05.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:16:05 smithi167 ceph-mon[37140]: pgmap v9178: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:16:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:16:05 smithi138 ceph-mon[36954]: pgmap v9178: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:16:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:16:05 smithi100 ceph-mon[32599]: pgmap v9178: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:16:07.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:16:07 smithi167 ceph-mon[37140]: pgmap v9179: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:16:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:16:07 smithi138 ceph-mon[36954]: pgmap v9179: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:16:07.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:16:07 smithi100 ceph-mon[32599]: pgmap v9179: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:16:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:16:09 smithi138 ceph-mon[36954]: pgmap v9180: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:16:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:16:09 smithi138 ceph-mon[36954]: osdmap e620: 12 total, 12 up, 12 in 2024-07-26T18:16:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:16:09 smithi100 ceph-mon[32599]: pgmap v9180: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:16:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:16:09 smithi100 ceph-mon[32599]: osdmap e620: 12 total, 12 up, 12 in 2024-07-26T18:16:09.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:16:09 smithi167 ceph-mon[37140]: pgmap v9180: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:16:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:16:09 smithi167 ceph-mon[37140]: osdmap e620: 12 total, 12 up, 12 in 2024-07-26T18:16:11.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:16:11 smithi138 ceph-mon[36954]: pgmap v9182: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:16:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:16:11 smithi100 ceph-mon[32599]: pgmap v9182: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:16:11.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:16:11 smithi167 ceph-mon[37140]: pgmap v9182: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:16:12.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:16:12.444+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:16:12.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:16:12.445+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:16:12.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:16:12.445+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:16:12.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:16:12.445+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:16:12.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:16:12.445+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:16:13.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:16:13 smithi138 ceph-mon[36954]: pgmap v9183: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:16:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:16:13 smithi100 ceph-mon[32599]: pgmap v9183: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:16:13.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:16:13 smithi167 ceph-mon[37140]: pgmap v9183: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:16:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:16:15 smithi138 ceph-mon[36954]: pgmap v9184: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:16:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:16:15 smithi100 ceph-mon[32599]: pgmap v9184: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:16:15.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:16:15 smithi167 ceph-mon[37140]: pgmap v9184: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:16:17.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:16:17 smithi138 ceph-mon[36954]: pgmap v9185: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:16:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:16:17 smithi100 ceph-mon[32599]: pgmap v9185: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:16:17.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:16:17 smithi167 ceph-mon[37140]: pgmap v9185: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:16:19.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:16:19 smithi138 ceph-mon[36954]: pgmap v9186: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:16:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:16:19 smithi138 ceph-mon[36954]: osdmap e621: 12 total, 12 up, 12 in 2024-07-26T18:16:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:16:19 smithi100 ceph-mon[32599]: pgmap v9186: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:16:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:16:19 smithi100 ceph-mon[32599]: osdmap e621: 12 total, 12 up, 12 in 2024-07-26T18:16:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:16:19 smithi167 ceph-mon[37140]: pgmap v9186: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:16:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:16:19 smithi167 ceph-mon[37140]: osdmap e621: 12 total, 12 up, 12 in 2024-07-26T18:16:20.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:16:20 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:16:20.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:16:20 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:16:20.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:16:20 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:16:21.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:16:21 smithi138 ceph-mon[36954]: pgmap v9188: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:16:21.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:16:21 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:16:21.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:16:21 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:16:21.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:16:21 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:16:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:16:21 smithi100 ceph-mon[32599]: pgmap v9188: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:16:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:16:21 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:16:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:16:21 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:16:21.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:16:21 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:16:21.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:16:21 smithi167 ceph-mon[37140]: pgmap v9188: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:16:21.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:16:21 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:16:21.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:16:21 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:16:21.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:16:21 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:16:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:16:23 smithi138 ceph-mon[36954]: pgmap v9189: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:16:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:16:23 smithi100 ceph-mon[32599]: pgmap v9189: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:16:23.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:16:23 smithi167 ceph-mon[37140]: pgmap v9189: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:16:25.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:16:25 smithi138 ceph-mon[36954]: pgmap v9190: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:16:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:16:25 smithi100 ceph-mon[32599]: pgmap v9190: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:16:25.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:16:25 smithi167 ceph-mon[37140]: pgmap v9190: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:16:27.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:16:27 smithi138 ceph-mon[36954]: pgmap v9191: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:16:27.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:16:27.460+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:16:27.463 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:16:27 smithi100 ceph-mon[32599]: pgmap v9191: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:16:27.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:16:27.460+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:16:27.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:16:27.460+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:16:27.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:16:27.460+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:16:27.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:16:27.460+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:16:27.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:16:27 smithi167 ceph-mon[37140]: pgmap v9191: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:16:29.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:16:29 smithi138 ceph-mon[36954]: pgmap v9192: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:16:29.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:16:29 smithi100 ceph-mon[32599]: pgmap v9192: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:16:29.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:16:29 smithi167 ceph-mon[37140]: pgmap v9192: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:16:31.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:16:31 smithi138 ceph-mon[36954]: pgmap v9193: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:16:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:16:31 smithi100 ceph-mon[32599]: pgmap v9193: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:16:31.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:16:31 smithi167 ceph-mon[37140]: pgmap v9193: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:16:33.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:16:33 smithi138 ceph-mon[36954]: pgmap v9194: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:16:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:16:33 smithi100 ceph-mon[32599]: pgmap v9194: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:16:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:16:33 smithi167 ceph-mon[37140]: pgmap v9194: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:16:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:16:35 smithi138 ceph-mon[36954]: pgmap v9195: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:16:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:16:35 smithi100 ceph-mon[32599]: pgmap v9195: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:16:35.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:16:35 smithi167 ceph-mon[37140]: pgmap v9195: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:16:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:16:37 smithi100 ceph-mon[32599]: pgmap v9196: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:16:37.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:16:37 smithi167 ceph-mon[37140]: pgmap v9196: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:16:37.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:16:37 smithi138 ceph-mon[36954]: pgmap v9196: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:16:38.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:16:38 smithi100 ceph-mon[32599]: pgmap v9197: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:16:38.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:16:38 smithi167 ceph-mon[37140]: pgmap v9197: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:16:38.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:16:38 smithi138 ceph-mon[36954]: pgmap v9197: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:16:41.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:16:41 smithi167 ceph-mon[37140]: pgmap v9198: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:16:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:16:41 smithi138 ceph-mon[36954]: pgmap v9198: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:16:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:16:41 smithi100 ceph-mon[32599]: pgmap v9198: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:16:42.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:16:42.469+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:16:42.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:16:42.470+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:16:42.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:16:42.470+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:16:42.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:16:42.470+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:16:42.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:16:42.470+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:16:43.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:16:43 smithi167 ceph-mon[37140]: pgmap v9199: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:16:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:16:43 smithi138 ceph-mon[36954]: pgmap v9199: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:16:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:16:43 smithi100 ceph-mon[32599]: pgmap v9199: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:16:45.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:16:45 smithi167 ceph-mon[37140]: pgmap v9200: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:16:45.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:16:45 smithi138 ceph-mon[36954]: pgmap v9200: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:16:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:16:45 smithi100 ceph-mon[32599]: pgmap v9200: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:16:47.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:16:47 smithi167 ceph-mon[37140]: pgmap v9201: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:16:47.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:16:47 smithi138 ceph-mon[36954]: pgmap v9201: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:16:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:16:47 smithi100 ceph-mon[32599]: pgmap v9201: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:16:49.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:16:49 smithi167 ceph-mon[37140]: pgmap v9202: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:16:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:16:49 smithi138 ceph-mon[36954]: pgmap v9202: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:16:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:16:49 smithi100 ceph-mon[32599]: pgmap v9202: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:16:51.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:16:51 smithi138 ceph-mon[36954]: pgmap v9203: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:16:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:16:51 smithi100 ceph-mon[32599]: pgmap v9203: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:16:51.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:16:51 smithi167 ceph-mon[37140]: pgmap v9203: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:16:53.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:16:53 smithi138 ceph-mon[36954]: pgmap v9204: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:16:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:16:53 smithi100 ceph-mon[32599]: pgmap v9204: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:16:53.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:16:53 smithi167 ceph-mon[37140]: pgmap v9204: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:16:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:16:55 smithi138 ceph-mon[36954]: pgmap v9205: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:16:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:16:55 smithi100 ceph-mon[32599]: pgmap v9205: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:16:55.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:16:55 smithi167 ceph-mon[37140]: pgmap v9205: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:16:57.296 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:16:57 smithi100 ceph-mon[32599]: pgmap v9206: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:16:57.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:16:57 smithi167 ceph-mon[37140]: pgmap v9206: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:16:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:16:57 smithi138 ceph-mon[36954]: pgmap v9206: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:16:57.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:16:57.485+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:16:57.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:16:57.486+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:16:57.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:16:57.486+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:16:57.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:16:57.486+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:16:57.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:16:57.486+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:16:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:16:59 smithi138 ceph-mon[36954]: pgmap v9207: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:16:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:16:59 smithi100 ceph-mon[32599]: pgmap v9207: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:16:59.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:16:59 smithi167 ceph-mon[37140]: pgmap v9207: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:17:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:17:01 smithi138 ceph-mon[36954]: pgmap v9208: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:17:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:17:01 smithi100 ceph-mon[32599]: pgmap v9208: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:17:01.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:17:01 smithi167 ceph-mon[37140]: pgmap v9208: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:17:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:17:03 smithi138 ceph-mon[36954]: pgmap v9209: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:17:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:17:03 smithi100 ceph-mon[32599]: pgmap v9209: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:17:03.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:17:03 smithi167 ceph-mon[37140]: pgmap v9209: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:17:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:17:05 smithi138 ceph-mon[36954]: pgmap v9210: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:17:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:17:05 smithi100 ceph-mon[32599]: pgmap v9210: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:17:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:17:05 smithi167 ceph-mon[37140]: pgmap v9210: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:17:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:17:07 smithi138 ceph-mon[36954]: pgmap v9211: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:17:07.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:17:07 smithi100 ceph-mon[32599]: pgmap v9211: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:17:07.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:17:07 smithi167 ceph-mon[37140]: pgmap v9211: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:17:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:17:09 smithi138 ceph-mon[36954]: pgmap v9212: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:17:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:17:09 smithi138 ceph-mon[36954]: osdmap e622: 12 total, 12 up, 12 in 2024-07-26T18:17:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:17:09 smithi100 ceph-mon[32599]: pgmap v9212: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:17:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:17:09 smithi100 ceph-mon[32599]: osdmap e622: 12 total, 12 up, 12 in 2024-07-26T18:17:09.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:17:09 smithi167 ceph-mon[37140]: pgmap v9212: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:17:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:17:09 smithi167 ceph-mon[37140]: osdmap e622: 12 total, 12 up, 12 in 2024-07-26T18:17:11.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:17:11 smithi138 ceph-mon[36954]: pgmap v9214: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:17:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:17:11 smithi100 ceph-mon[32599]: pgmap v9214: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:17:11.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:17:11 smithi167 ceph-mon[37140]: pgmap v9214: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:17:12.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:17:12.501+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:17:12.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:17:12.502+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:17:12.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:17:12.502+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:17:12.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:17:12.502+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:17:12.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:17:12.502+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:17:13.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:17:13 smithi138 ceph-mon[36954]: pgmap v9215: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:17:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:17:13 smithi100 ceph-mon[32599]: pgmap v9215: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:17:13.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:17:13 smithi167 ceph-mon[37140]: pgmap v9215: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:17:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:17:15 smithi138 ceph-mon[36954]: pgmap v9216: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:17:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:17:15 smithi100 ceph-mon[32599]: pgmap v9216: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:17:15.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:17:15 smithi167 ceph-mon[37140]: pgmap v9216: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:17:17.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:17:17 smithi138 ceph-mon[36954]: pgmap v9217: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:17:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:17:17 smithi100 ceph-mon[32599]: pgmap v9217: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:17:17.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:17:17 smithi167 ceph-mon[37140]: pgmap v9217: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:17:19.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:17:19 smithi138 ceph-mon[36954]: pgmap v9218: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:17:19.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:17:19 smithi138 ceph-mon[36954]: osdmap e623: 12 total, 12 up, 12 in 2024-07-26T18:17:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:17:19 smithi100 ceph-mon[32599]: pgmap v9218: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:17:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:17:19 smithi100 ceph-mon[32599]: osdmap e623: 12 total, 12 up, 12 in 2024-07-26T18:17:19.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:17:19 smithi167 ceph-mon[37140]: pgmap v9218: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:17:19.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:17:19 smithi167 ceph-mon[37140]: osdmap e623: 12 total, 12 up, 12 in 2024-07-26T18:17:21.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:17:21 smithi138 ceph-mon[36954]: pgmap v9220: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:17:21.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:17:21 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:17:21.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:17:21 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:17:21.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:17:21 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:17:21.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:17:21 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:17:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:17:21 smithi100 ceph-mon[32599]: pgmap v9220: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:17:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:17:21 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:17:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:17:21 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:17:21.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:17:21 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:17:21.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:17:21 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:17:21.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:17:21 smithi167 ceph-mon[37140]: pgmap v9220: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:17:21.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:17:21 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:17:21.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:17:21 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:17:21.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:17:21 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:17:21.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:17:21 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:17:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:17:23 smithi138 ceph-mon[36954]: pgmap v9221: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:17:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:17:23 smithi100 ceph-mon[32599]: pgmap v9221: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:17:23.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:17:23 smithi167 ceph-mon[37140]: pgmap v9221: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:17:25.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:17:25 smithi138 ceph-mon[36954]: pgmap v9222: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:17:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:17:25 smithi100 ceph-mon[32599]: pgmap v9222: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:17:25.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:17:25 smithi167 ceph-mon[37140]: pgmap v9222: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:17:27.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:17:27 smithi138 ceph-mon[36954]: pgmap v9223: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:17:27.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:17:27 smithi100 ceph-mon[32599]: pgmap v9223: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:17:27.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:17:27.518+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:17:27.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:17:27.518+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:17:27.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:17:27.518+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:17:27.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:17:27.518+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:17:27.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:17:27.518+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:17:27.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:17:27 smithi167 ceph-mon[37140]: pgmap v9223: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:17:29.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:17:29 smithi100 ceph-mon[32599]: pgmap v9224: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:17:29.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:17:29 smithi167 ceph-mon[37140]: pgmap v9224: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:17:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:17:29 smithi138 ceph-mon[36954]: pgmap v9224: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:17:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:17:31 smithi100 ceph-mon[32599]: pgmap v9225: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:17:31.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:17:31 smithi167 ceph-mon[37140]: pgmap v9225: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:17:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:17:31 smithi138 ceph-mon[36954]: pgmap v9225: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:17:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:17:33 smithi100 ceph-mon[32599]: pgmap v9226: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:17:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:17:33 smithi167 ceph-mon[37140]: pgmap v9226: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:17:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:17:33 smithi138 ceph-mon[36954]: pgmap v9226: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:17:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:17:35 smithi100 ceph-mon[32599]: pgmap v9227: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:17:35.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:17:35 smithi167 ceph-mon[37140]: pgmap v9227: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:17:35.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:17:35 smithi138 ceph-mon[36954]: pgmap v9227: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:17:36.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:17:36 smithi100 ceph-mon[32599]: pgmap v9228: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:17:36.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:17:36 smithi167 ceph-mon[37140]: pgmap v9228: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:17:36.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:17:36 smithi138 ceph-mon[36954]: pgmap v9228: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:17:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:17:39 smithi138 ceph-mon[36954]: pgmap v9229: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:17:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:17:39 smithi100 ceph-mon[32599]: pgmap v9229: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:17:39.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:17:39 smithi167 ceph-mon[37140]: pgmap v9229: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:17:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:17:41 smithi138 ceph-mon[36954]: pgmap v9230: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:17:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:17:41 smithi100 ceph-mon[32599]: pgmap v9230: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:17:41.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:17:41 smithi167 ceph-mon[37140]: pgmap v9230: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:17:42.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:17:42.526+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:17:42.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:17:42.527+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:17:42.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:17:42.527+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:17:42.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:17:42.527+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:17:42.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:17:42.527+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:17:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:17:43 smithi138 ceph-mon[36954]: pgmap v9231: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:17:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:17:43 smithi100 ceph-mon[32599]: pgmap v9231: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:17:43.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:17:43 smithi167 ceph-mon[37140]: pgmap v9231: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:17:45.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:17:45 smithi138 ceph-mon[36954]: pgmap v9232: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:17:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:17:45 smithi100 ceph-mon[32599]: pgmap v9232: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:17:45.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:17:45 smithi167 ceph-mon[37140]: pgmap v9232: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:17:47.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:17:47 smithi138 ceph-mon[36954]: pgmap v9233: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:17:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:17:47 smithi100 ceph-mon[32599]: pgmap v9233: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:17:47.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:17:47 smithi167 ceph-mon[37140]: pgmap v9233: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:17:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:17:49 smithi138 ceph-mon[36954]: pgmap v9234: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:17:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:17:49 smithi100 ceph-mon[32599]: pgmap v9234: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:17:49.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:17:49 smithi167 ceph-mon[37140]: pgmap v9234: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:17:51.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:17:51 smithi138 ceph-mon[36954]: pgmap v9235: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:17:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:17:51 smithi100 ceph-mon[32599]: pgmap v9235: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:17:51.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:17:51 smithi167 ceph-mon[37140]: pgmap v9235: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:17:53.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:17:53 smithi138 ceph-mon[36954]: pgmap v9236: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:17:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:17:53 smithi100 ceph-mon[32599]: pgmap v9236: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:17:53.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:17:53 smithi167 ceph-mon[37140]: pgmap v9236: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:17:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:17:55 smithi138 ceph-mon[36954]: pgmap v9237: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:17:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:17:55 smithi100 ceph-mon[32599]: pgmap v9237: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:17:55.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:17:55 smithi167 ceph-mon[37140]: pgmap v9237: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:17:57.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:17:57 smithi167 ceph-mon[37140]: pgmap v9238: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:17:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:17:57 smithi138 ceph-mon[36954]: pgmap v9238: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:17:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:17:57 smithi100 ceph-mon[32599]: pgmap v9238: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:17:57.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:17:57.542+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:17:57.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:17:57.543+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:17:57.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:17:57.543+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:17:57.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:17:57.543+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:17:57.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:17:57.543+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:17:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:17:59 smithi138 ceph-mon[36954]: pgmap v9239: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:17:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:17:59 smithi100 ceph-mon[32599]: pgmap v9239: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:17:59.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:17:59 smithi167 ceph-mon[37140]: pgmap v9239: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:18:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:18:01 smithi138 ceph-mon[36954]: pgmap v9240: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:18:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:18:01 smithi100 ceph-mon[32599]: pgmap v9240: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:18:01.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:18:01 smithi167 ceph-mon[37140]: pgmap v9240: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:18:03.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:18:03.046+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 msgr2=0x7fe5940c0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-07-26T18:18:03.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:18:03.046+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 msgr2=0x7fe5940c0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-07-26T18:18:03.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:18:03.046+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fe590004d70 tx=0x7fe5900ab000 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:18:03.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:18:03.046+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fe590004d70 tx=0x7fe5900ab000 comp rx=0 tx=0).stop 2024-07-26T18:18:03.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:18:03.046+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 msgr2=0x7fe5940c0480 unknown :-1 s=STATE_CLOSED l=1).tick see no progress in more than 10000000 us during connecting to v2:172.21.15.100:6800/3937009534, fault. 2024-07-26T18:18:03.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:18:03.046+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 msgr2=0x7fe5940c0480 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-07-26T18:18:03.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:18:03.046+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T18:18:03.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:18:03.046+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 0x7fe5940b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T18:18:03.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:18:03.046+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 0x7fe5940b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:18:03.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:18:03.046+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 0x7fe5940b5b30 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fe590004ee0 tx=0x7fe59008a040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T18:18:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:18:03 smithi138 ceph-mon[36954]: pgmap v9241: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:18:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:18:03 smithi100 ceph-mon[32599]: pgmap v9241: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:18:03.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:18:03 smithi167 ceph-mon[37140]: pgmap v9241: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:18:05.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:18:05.312+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c004740 con 0x7fe5a81087a0 2024-07-26T18:18:05.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:18:05.312+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T18:18:05.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:18:05.312+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 127 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T18:18:05.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:18:05.312+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 127 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c100 con 0x7fe5a81087a0 2024-07-26T18:18:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:18:05 smithi138 ceph-mon[36954]: pgmap v9242: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:18:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:18:05 smithi100 ceph-mon[32599]: pgmap v9242: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:18:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:18:05 smithi167 ceph-mon[37140]: pgmap v9242: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:18:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:18:07 smithi138 ceph-mon[36954]: pgmap v9243: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:18:07.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:18:07 smithi100 ceph-mon[32599]: pgmap v9243: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:18:07.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:18:07 smithi167 ceph-mon[37140]: pgmap v9243: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:18:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:18:09 smithi138 ceph-mon[36954]: pgmap v9244: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:18:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:18:09 smithi138 ceph-mon[36954]: osdmap e624: 12 total, 12 up, 12 in 2024-07-26T18:18:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:18:09 smithi100 ceph-mon[32599]: pgmap v9244: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:18:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:18:09 smithi100 ceph-mon[32599]: osdmap e624: 12 total, 12 up, 12 in 2024-07-26T18:18:09.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:18:09 smithi167 ceph-mon[37140]: pgmap v9244: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:18:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:18:09 smithi167 ceph-mon[37140]: osdmap e624: 12 total, 12 up, 12 in 2024-07-26T18:18:11.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:18:11 smithi138 ceph-mon[36954]: pgmap v9246: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:18:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:18:11 smithi100 ceph-mon[32599]: pgmap v9246: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:18:11.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:18:11 smithi167 ceph-mon[37140]: pgmap v9246: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:18:12.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:18:12.554+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:18:12.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:18:12.555+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:18:12.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:18:12.555+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:18:12.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:18:12.555+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:18:12.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:18:12.555+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:18:13.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:18:13 smithi138 ceph-mon[36954]: pgmap v9247: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:18:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:18:13 smithi100 ceph-mon[32599]: pgmap v9247: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:18:13.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:18:13 smithi167 ceph-mon[37140]: pgmap v9247: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:18:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:18:15 smithi138 ceph-mon[36954]: pgmap v9248: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:18:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:18:15 smithi100 ceph-mon[32599]: pgmap v9248: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:18:15.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:18:15 smithi167 ceph-mon[37140]: pgmap v9248: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:18:17.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:18:17 smithi138 ceph-mon[36954]: pgmap v9249: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:18:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:18:17 smithi100 ceph-mon[32599]: pgmap v9249: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:18:17.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:18:17 smithi167 ceph-mon[37140]: pgmap v9249: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:18:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:18:19 smithi100 ceph-mon[32599]: pgmap v9250: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:18:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:18:19 smithi100 ceph-mon[32599]: osdmap e625: 12 total, 12 up, 12 in 2024-07-26T18:18:19.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:18:19 smithi167 ceph-mon[37140]: pgmap v9250: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:18:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:18:19 smithi167 ceph-mon[37140]: osdmap e625: 12 total, 12 up, 12 in 2024-07-26T18:18:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:18:19 smithi138 ceph-mon[36954]: pgmap v9250: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:18:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:18:19 smithi138 ceph-mon[36954]: osdmap e625: 12 total, 12 up, 12 in 2024-07-26T18:18:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:18:21 smithi100 ceph-mon[32599]: pgmap v9252: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:18:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:18:21 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:18:21.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:18:21 smithi167 ceph-mon[37140]: pgmap v9252: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:18:21.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:18:21 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:18:21.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:18:21 smithi138 ceph-mon[36954]: pgmap v9252: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:18:21.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:18:21 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:18:22.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:18:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:18:22.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:18:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:18:22.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:18:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:18:22.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:18:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:18:22.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:18:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:18:22.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:18:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:18:22.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:18:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:18:22.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:18:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:18:22.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:18:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:18:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:18:23 smithi100 ceph-mon[32599]: pgmap v9253: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:18:23.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:18:23 smithi167 ceph-mon[37140]: pgmap v9253: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:18:23.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:18:23 smithi138 ceph-mon[36954]: pgmap v9253: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:18:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:18:25 smithi100 ceph-mon[32599]: pgmap v9254: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:18:25.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:18:25 smithi167 ceph-mon[37140]: pgmap v9254: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:18:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:18:25 smithi138 ceph-mon[36954]: pgmap v9254: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:18:27.492 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:18:27 smithi138 ceph-mon[36954]: pgmap v9255: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:18:27.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:18:27 smithi100 ceph-mon[32599]: pgmap v9255: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:18:27.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:18:27 smithi167 ceph-mon[37140]: pgmap v9255: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:18:27.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:18:27.570+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:18:27.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:18:27.571+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:18:27.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:18:27.571+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:18:27.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:18:27.571+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:18:27.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:18:27.571+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:18:29.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:18:29 smithi100 ceph-mon[32599]: pgmap v9256: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:18:29.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:18:29 smithi167 ceph-mon[37140]: pgmap v9256: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:18:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:18:29 smithi138 ceph-mon[36954]: pgmap v9256: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:18:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:18:31 smithi100 ceph-mon[32599]: pgmap v9257: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:18:31.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:18:31 smithi167 ceph-mon[37140]: pgmap v9257: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:18:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:18:31 smithi138 ceph-mon[36954]: pgmap v9257: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:18:32.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:18:32 smithi100 ceph-mon[32599]: pgmap v9258: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:18:32.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:18:32 smithi167 ceph-mon[37140]: pgmap v9258: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:18:32.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:18:32 smithi138 ceph-mon[36954]: pgmap v9258: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:18:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:18:35 smithi138 ceph-mon[36954]: pgmap v9259: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:18:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:18:35 smithi100 ceph-mon[32599]: pgmap v9259: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:18:35.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:18:35 smithi167 ceph-mon[37140]: pgmap v9259: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:18:37.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:18:37 smithi138 ceph-mon[36954]: pgmap v9260: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:18:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:18:37 smithi100 ceph-mon[32599]: pgmap v9260: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:18:37.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:18:37 smithi167 ceph-mon[37140]: pgmap v9260: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:18:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:18:39 smithi138 ceph-mon[36954]: pgmap v9261: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:18:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:18:39 smithi100 ceph-mon[32599]: pgmap v9261: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:18:39.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:18:39 smithi167 ceph-mon[37140]: pgmap v9261: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:18:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:18:41 smithi138 ceph-mon[36954]: pgmap v9262: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:18:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:18:41 smithi100 ceph-mon[32599]: pgmap v9262: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:18:41.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:18:41 smithi167 ceph-mon[37140]: pgmap v9262: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:18:42.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:18:42.586+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:18:42.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:18:42.587+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:18:42.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:18:42.587+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:18:42.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:18:42.587+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:18:42.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:18:42.587+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:18:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:18:43 smithi138 ceph-mon[36954]: pgmap v9263: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:18:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:18:43 smithi100 ceph-mon[32599]: pgmap v9263: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:18:43.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:18:43 smithi167 ceph-mon[37140]: pgmap v9263: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:18:45.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:18:45 smithi138 ceph-mon[36954]: pgmap v9264: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:18:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:18:45 smithi100 ceph-mon[32599]: pgmap v9264: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:18:45.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:18:45 smithi167 ceph-mon[37140]: pgmap v9264: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:18:47.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:18:47 smithi138 ceph-mon[36954]: pgmap v9265: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:18:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:18:47 smithi100 ceph-mon[32599]: pgmap v9265: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:18:47.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:18:47 smithi167 ceph-mon[37140]: pgmap v9265: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:18:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:18:49 smithi138 ceph-mon[36954]: pgmap v9266: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:18:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:18:49 smithi100 ceph-mon[32599]: pgmap v9266: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:18:49.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:18:49 smithi167 ceph-mon[37140]: pgmap v9266: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:18:51.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:18:51 smithi138 ceph-mon[36954]: pgmap v9267: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:18:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:18:51 smithi100 ceph-mon[32599]: pgmap v9267: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:18:51.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:18:51 smithi167 ceph-mon[37140]: pgmap v9267: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:18:53.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:18:53 smithi138 ceph-mon[36954]: pgmap v9268: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:18:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:18:53 smithi100 ceph-mon[32599]: pgmap v9268: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:18:53.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:18:53 smithi167 ceph-mon[37140]: pgmap v9268: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:18:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:18:55 smithi138 ceph-mon[36954]: pgmap v9269: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:18:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:18:55 smithi100 ceph-mon[32599]: pgmap v9269: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:18:55.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:18:55 smithi167 ceph-mon[37140]: pgmap v9269: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:18:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:18:57 smithi138 ceph-mon[36954]: pgmap v9270: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:18:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:18:57 smithi100 ceph-mon[32599]: pgmap v9270: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:18:57.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:18:57 smithi167 ceph-mon[37140]: pgmap v9270: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:18:57.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:18:57.602+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:18:57.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:18:57.602+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:18:57.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:18:57.602+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:18:57.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:18:57.602+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:18:57.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:18:57.602+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:18:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:18:59 smithi138 ceph-mon[36954]: pgmap v9271: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:18:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:18:59 smithi100 ceph-mon[32599]: pgmap v9271: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:18:59.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:18:59 smithi167 ceph-mon[37140]: pgmap v9271: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:19:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:19:01 smithi138 ceph-mon[36954]: pgmap v9272: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:19:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:19:01 smithi100 ceph-mon[32599]: pgmap v9272: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:19:01.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:19:01 smithi167 ceph-mon[37140]: pgmap v9272: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:19:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:19:03 smithi138 ceph-mon[36954]: pgmap v9273: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:19:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:19:03 smithi100 ceph-mon[32599]: pgmap v9273: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:19:03.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:19:03 smithi167 ceph-mon[37140]: pgmap v9273: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:19:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:19:05 smithi138 ceph-mon[36954]: pgmap v9274: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:19:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:19:05 smithi100 ceph-mon[32599]: pgmap v9274: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:19:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:19:05 smithi167 ceph-mon[37140]: pgmap v9274: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:19:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:19:07 smithi138 ceph-mon[36954]: pgmap v9275: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:19:07.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:19:07 smithi100 ceph-mon[32599]: pgmap v9275: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:19:07.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:19:07 smithi167 ceph-mon[37140]: pgmap v9275: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:19:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:19:09 smithi138 ceph-mon[36954]: pgmap v9276: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:19:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:19:09 smithi138 ceph-mon[36954]: osdmap e626: 12 total, 12 up, 12 in 2024-07-26T18:19:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:19:09 smithi100 ceph-mon[32599]: pgmap v9276: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:19:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:19:09 smithi100 ceph-mon[32599]: osdmap e626: 12 total, 12 up, 12 in 2024-07-26T18:19:09.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:19:09 smithi167 ceph-mon[37140]: pgmap v9276: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:19:09.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:19:09 smithi167 ceph-mon[37140]: osdmap e626: 12 total, 12 up, 12 in 2024-07-26T18:19:11.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:19:11 smithi138 ceph-mon[36954]: pgmap v9278: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:19:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:19:11 smithi100 ceph-mon[32599]: pgmap v9278: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:19:11.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:19:11 smithi167 ceph-mon[37140]: pgmap v9278: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:19:12.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:19:12.605+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:19:12.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:19:12.606+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:19:12.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:19:12.606+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:19:12.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:19:12.606+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:19:12.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:19:12.606+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:19:13.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:19:13 smithi138 ceph-mon[36954]: pgmap v9279: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:19:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:19:13 smithi100 ceph-mon[32599]: pgmap v9279: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:19:13.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:19:13 smithi167 ceph-mon[37140]: pgmap v9279: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:19:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:19:15 smithi138 ceph-mon[36954]: pgmap v9280: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:19:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:19:15 smithi100 ceph-mon[32599]: pgmap v9280: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:19:15.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:19:15 smithi167 ceph-mon[37140]: pgmap v9280: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:19:17.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:19:17 smithi138 ceph-mon[36954]: pgmap v9281: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:19:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:19:17 smithi100 ceph-mon[32599]: pgmap v9281: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:19:17.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:19:17 smithi167 ceph-mon[37140]: pgmap v9281: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:19:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:19:19 smithi100 ceph-mon[32599]: pgmap v9282: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:19:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:19:19 smithi100 ceph-mon[32599]: osdmap e627: 12 total, 12 up, 12 in 2024-07-26T18:19:19.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:19:19 smithi167 ceph-mon[37140]: pgmap v9282: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:19:19.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:19:19 smithi167 ceph-mon[37140]: osdmap e627: 12 total, 12 up, 12 in 2024-07-26T18:19:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:19:19 smithi138 ceph-mon[36954]: pgmap v9282: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:19:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:19:19 smithi138 ceph-mon[36954]: osdmap e627: 12 total, 12 up, 12 in 2024-07-26T18:19:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:19:21 smithi100 ceph-mon[32599]: pgmap v9284: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:19:21.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:19:21 smithi167 ceph-mon[37140]: pgmap v9284: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:19:21.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:19:21 smithi138 ceph-mon[36954]: pgmap v9284: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:19:22.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:19:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:19:22.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:19:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:19:22.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:19:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:19:22.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:19:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:19:22.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:19:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:19:22.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:19:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:19:22.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:19:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:19:22.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:19:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:19:22.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:19:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:19:22.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:19:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:19:22.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:19:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:19:22.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:19:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:19:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:19:23 smithi100 ceph-mon[32599]: pgmap v9285: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:19:23.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:19:23 smithi167 ceph-mon[37140]: pgmap v9285: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:19:23.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:19:23 smithi138 ceph-mon[36954]: pgmap v9285: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:19:24.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:19:24 smithi167 ceph-mon[37140]: pgmap v9286: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:19:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:19:24 smithi138 ceph-mon[36954]: pgmap v9286: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:19:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:19:24 smithi100 ceph-mon[32599]: pgmap v9286: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:19:27.392 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:19:27 smithi138 ceph-mon[36954]: pgmap v9287: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:19:27.447 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:19:27 smithi100 ceph-mon[32599]: pgmap v9287: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:19:27.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:19:27 smithi167 ceph-mon[37140]: pgmap v9287: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:19:27.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:19:27.615+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:19:27.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:19:27.616+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:19:27.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:19:27.616+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:19:27.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:19:27.616+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:19:27.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:19:27.616+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:19:29.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:19:29 smithi138 ceph-mon[36954]: pgmap v9288: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:19:29.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:19:29 smithi100 ceph-mon[32599]: pgmap v9288: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:19:29.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:19:29 smithi167 ceph-mon[37140]: pgmap v9288: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:19:31.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:19:31 smithi138 ceph-mon[36954]: pgmap v9289: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:19:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:19:31 smithi100 ceph-mon[32599]: pgmap v9289: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:19:31.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:19:31 smithi167 ceph-mon[37140]: pgmap v9289: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:19:33.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:19:33 smithi138 ceph-mon[36954]: pgmap v9290: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:19:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:19:33 smithi100 ceph-mon[32599]: pgmap v9290: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:19:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:19:33 smithi167 ceph-mon[37140]: pgmap v9290: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:19:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:19:35 smithi138 ceph-mon[36954]: pgmap v9291: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:19:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:19:35 smithi100 ceph-mon[32599]: pgmap v9291: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:19:35.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:19:35 smithi167 ceph-mon[37140]: pgmap v9291: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:19:37.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:19:37 smithi138 ceph-mon[36954]: pgmap v9292: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:19:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:19:37 smithi100 ceph-mon[32599]: pgmap v9292: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:19:37.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:19:37 smithi167 ceph-mon[37140]: pgmap v9292: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:19:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:19:39 smithi138 ceph-mon[36954]: pgmap v9293: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:19:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:19:39 smithi100 ceph-mon[32599]: pgmap v9293: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:19:39.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:19:39 smithi167 ceph-mon[37140]: pgmap v9293: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:19:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:19:41 smithi138 ceph-mon[36954]: pgmap v9294: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:19:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:19:41 smithi100 ceph-mon[32599]: pgmap v9294: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:19:41.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:19:41 smithi167 ceph-mon[37140]: pgmap v9294: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:19:42.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:19:42.631+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:19:42.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:19:42.632+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:19:42.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:19:42.632+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:19:42.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:19:42.632+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:19:42.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:19:42.632+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:19:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:19:43 smithi138 ceph-mon[36954]: pgmap v9295: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:19:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:19:43 smithi100 ceph-mon[32599]: pgmap v9295: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:19:43.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:19:43 smithi167 ceph-mon[37140]: pgmap v9295: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:19:45.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:19:45 smithi138 ceph-mon[36954]: pgmap v9296: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:19:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:19:45 smithi100 ceph-mon[32599]: pgmap v9296: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:19:45.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:19:45 smithi167 ceph-mon[37140]: pgmap v9296: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:19:47.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:19:47 smithi138 ceph-mon[36954]: pgmap v9297: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:19:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:19:47 smithi100 ceph-mon[32599]: pgmap v9297: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:19:47.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:19:47 smithi167 ceph-mon[37140]: pgmap v9297: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:19:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:19:49 smithi138 ceph-mon[36954]: pgmap v9298: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:19:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:19:49 smithi100 ceph-mon[32599]: pgmap v9298: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:19:49.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:19:49 smithi167 ceph-mon[37140]: pgmap v9298: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:19:51.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:19:51 smithi138 ceph-mon[36954]: pgmap v9299: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:19:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:19:51 smithi100 ceph-mon[32599]: pgmap v9299: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:19:51.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:19:51 smithi167 ceph-mon[37140]: pgmap v9299: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:19:53.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:19:53 smithi138 ceph-mon[36954]: pgmap v9300: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:19:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:19:53 smithi100 ceph-mon[32599]: pgmap v9300: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:19:53.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:19:53 smithi167 ceph-mon[37140]: pgmap v9300: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:19:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:19:55 smithi138 ceph-mon[36954]: pgmap v9301: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:19:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:19:55 smithi100 ceph-mon[32599]: pgmap v9301: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:19:55.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:19:55 smithi167 ceph-mon[37140]: pgmap v9301: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:19:57.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:19:57 smithi138 ceph-mon[36954]: pgmap v9302: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:19:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:19:57 smithi100 ceph-mon[32599]: pgmap v9302: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:19:57.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:19:57 smithi167 ceph-mon[37140]: pgmap v9302: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:19:57.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:19:57.648+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:19:57.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:19:57.648+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:19:57.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:19:57.648+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:19:57.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:19:57.648+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:19:57.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:19:57.648+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:19:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:19:59 smithi138 ceph-mon[36954]: pgmap v9303: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:19:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:19:59 smithi100 ceph-mon[32599]: pgmap v9303: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:19:59.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:19:59 smithi167 ceph-mon[37140]: pgmap v9303: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:20:00.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:00 smithi138 ceph-mon[36954]: overall HEALTH_OK 2024-07-26T18:20:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:00 smithi100 ceph-mon[32599]: overall HEALTH_OK 2024-07-26T18:20:00.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:00 smithi167 ceph-mon[37140]: overall HEALTH_OK 2024-07-26T18:20:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:01 smithi138 ceph-mon[36954]: pgmap v9304: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:20:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:01 smithi100 ceph-mon[32599]: pgmap v9304: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:20:01.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:01 smithi167 ceph-mon[37140]: pgmap v9304: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:20:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:03 smithi138 ceph-mon[36954]: pgmap v9305: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:20:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:03 smithi100 ceph-mon[32599]: pgmap v9305: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:20:03.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:03 smithi167 ceph-mon[37140]: pgmap v9305: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:20:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:05 smithi138 ceph-mon[36954]: pgmap v9306: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:20:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:05 smithi100 ceph-mon[32599]: pgmap v9306: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:20:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:05 smithi167 ceph-mon[37140]: pgmap v9306: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:20:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:07 smithi138 ceph-mon[36954]: pgmap v9307: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:20:07.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:07 smithi100 ceph-mon[32599]: pgmap v9307: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:20:07.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:07 smithi167 ceph-mon[37140]: pgmap v9307: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:20:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:09 smithi138 ceph-mon[36954]: pgmap v9308: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:20:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:09 smithi138 ceph-mon[36954]: osdmap e628: 12 total, 12 up, 12 in 2024-07-26T18:20:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:09 smithi100 ceph-mon[32599]: pgmap v9308: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:20:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:09 smithi100 ceph-mon[32599]: osdmap e628: 12 total, 12 up, 12 in 2024-07-26T18:20:09.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:09 smithi167 ceph-mon[37140]: pgmap v9308: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:20:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:09 smithi167 ceph-mon[37140]: osdmap e628: 12 total, 12 up, 12 in 2024-07-26T18:20:11.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:11 smithi138 ceph-mon[36954]: pgmap v9310: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:20:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:11 smithi100 ceph-mon[32599]: pgmap v9310: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:20:11.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:11 smithi167 ceph-mon[37140]: pgmap v9310: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:20:12.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:20:12.650+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:20:12.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:20:12.650+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:20:12.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:20:12.650+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:20:12.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:20:12.650+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:20:12.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:20:12.650+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:20:13.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:13 smithi138 ceph-mon[36954]: pgmap v9311: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:20:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:13 smithi100 ceph-mon[32599]: pgmap v9311: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:20:13.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:13 smithi167 ceph-mon[37140]: pgmap v9311: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:20:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:15 smithi138 ceph-mon[36954]: pgmap v9312: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:20:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:15 smithi100 ceph-mon[32599]: pgmap v9312: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:20:15.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:15 smithi167 ceph-mon[37140]: pgmap v9312: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:20:17.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:17 smithi138 ceph-mon[36954]: pgmap v9313: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:20:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:17 smithi100 ceph-mon[32599]: pgmap v9313: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:20:17.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:17 smithi167 ceph-mon[37140]: pgmap v9313: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:20:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:19 smithi100 ceph-mon[32599]: pgmap v9314: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:20:19.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:19 smithi100 ceph-mon[32599]: osdmap e629: 12 total, 12 up, 12 in 2024-07-26T18:20:19.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:19 smithi167 ceph-mon[37140]: pgmap v9314: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:20:19.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:19 smithi167 ceph-mon[37140]: osdmap e629: 12 total, 12 up, 12 in 2024-07-26T18:20:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:19 smithi138 ceph-mon[36954]: pgmap v9314: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:20:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:19 smithi138 ceph-mon[36954]: osdmap e629: 12 total, 12 up, 12 in 2024-07-26T18:20:20.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:20:20.331+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c003690 con 0x7fe5a81087a0 2024-07-26T18:20:20.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:20:20.331+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T18:20:20.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:20:20.332+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 128 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T18:20:20.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:20:20.332+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 128 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe5a008c0e0 con 0x7fe5a81087a0 2024-07-26T18:20:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:21 smithi100 ceph-mon[32599]: pgmap v9316: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:20:21.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:21 smithi167 ceph-mon[37140]: pgmap v9316: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:20:21.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:21 smithi138 ceph-mon[36954]: pgmap v9316: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:20:22.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:22 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:20:22.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:22 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:20:22.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:22 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:20:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:23 smithi100 ceph-mon[32599]: pgmap v9317: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:20:23.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:20:23.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:20:23.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:20:23.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:20:23.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:20:23.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:20:23.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:20:23.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:20:23.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:20:23.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:20:23.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:20:23.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:20:23.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:23 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T18:20:23.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:23 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T18:20:23.496 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:23 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T18:20:23.497 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:23 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T18:20:23.497 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:23 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T18:20:23.497 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:23 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T18:20:23.497 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:20:23.497 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:20:23.497 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:20:23.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:23 smithi167 ceph-mon[37140]: pgmap v9317: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:20:23.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:20:23.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:20:23.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:20:23.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:20:23.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:20:23.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:20:23.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:20:23.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:20:23.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:20:23.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:20:23.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:20:23.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:20:23.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:23 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T18:20:23.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:23 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T18:20:23.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:23 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T18:20:23.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:23 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T18:20:23.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:23 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T18:20:23.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:23 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T18:20:23.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:20:23.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:20:23.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:20:23.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:23 smithi138 ceph-mon[36954]: pgmap v9317: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:20:23.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:20:23.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:20:23.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:20:23.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:20:23.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:20:23.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:20:23.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:20:23.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:20:23.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:20:23.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:20:23.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:20:23.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:20:23.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:23 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T18:20:23.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:23 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T18:20:23.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:23 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T18:20:23.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:23 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T18:20:23.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:23 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T18:20:23.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:23 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T18:20:23.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:20:23.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:20:23.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:20:24.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:24 smithi100 ceph-mon[32599]: pgmap v9318: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:20:24.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:24 smithi167 ceph-mon[37140]: pgmap v9318: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:20:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:24 smithi138 ceph-mon[36954]: pgmap v9318: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:20:27.393 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:27 smithi138 ceph-mon[36954]: pgmap v9319: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:20:27.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:27 smithi100 ceph-mon[32599]: pgmap v9319: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:20:27.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:27 smithi167 ceph-mon[37140]: pgmap v9319: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:20:27.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:20:27.665+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:20:27.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:20:27.666+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:20:27.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:20:27.666+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:20:27.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:20:27.666+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:20:27.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:20:27.666+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:20:29.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:29 smithi138 ceph-mon[36954]: pgmap v9320: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:20:29.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:29 smithi100 ceph-mon[32599]: pgmap v9320: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:20:29.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:29 smithi167 ceph-mon[37140]: pgmap v9320: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:20:31.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:31 smithi138 ceph-mon[36954]: pgmap v9321: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:20:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:31 smithi100 ceph-mon[32599]: pgmap v9321: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:20:31.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:31 smithi167 ceph-mon[37140]: pgmap v9321: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:20:33.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:33 smithi138 ceph-mon[36954]: pgmap v9322: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:20:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:33 smithi100 ceph-mon[32599]: pgmap v9322: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:20:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:33 smithi167 ceph-mon[37140]: pgmap v9322: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:20:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:35 smithi100 ceph-mon[32599]: pgmap v9323: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:20:35.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:35 smithi167 ceph-mon[37140]: pgmap v9323: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:20:35.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:35 smithi138 ceph-mon[36954]: pgmap v9323: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:20:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:37 smithi100 ceph-mon[32599]: pgmap v9324: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:20:37.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:37 smithi167 ceph-mon[37140]: pgmap v9324: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:20:37.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:37 smithi138 ceph-mon[36954]: pgmap v9324: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:20:38.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:38 smithi138 ceph-mon[36954]: pgmap v9325: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:20:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:38 smithi100 ceph-mon[32599]: pgmap v9325: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:20:38.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:38 smithi167 ceph-mon[37140]: pgmap v9325: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:20:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:41 smithi138 ceph-mon[36954]: pgmap v9326: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:20:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:41 smithi100 ceph-mon[32599]: pgmap v9326: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:20:41.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:41 smithi167 ceph-mon[37140]: pgmap v9326: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:20:42.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:20:42.672+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:20:42.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:20:42.673+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:20:42.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:20:42.673+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:20:42.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:20:42.673+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:20:42.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:20:42.673+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:20:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:43 smithi138 ceph-mon[36954]: pgmap v9327: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:20:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:43 smithi100 ceph-mon[32599]: pgmap v9327: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:20:43.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:43 smithi167 ceph-mon[37140]: pgmap v9327: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:20:45.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:45 smithi138 ceph-mon[36954]: pgmap v9328: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:20:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:45 smithi100 ceph-mon[32599]: pgmap v9328: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:20:45.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:45 smithi167 ceph-mon[37140]: pgmap v9328: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:20:47.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:47 smithi138 ceph-mon[36954]: pgmap v9329: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:20:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:47 smithi100 ceph-mon[32599]: pgmap v9329: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:20:47.526 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:47 smithi167 ceph-mon[37140]: pgmap v9329: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:20:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:49 smithi138 ceph-mon[36954]: pgmap v9330: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:20:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:49 smithi100 ceph-mon[32599]: pgmap v9330: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:20:49.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:49 smithi167 ceph-mon[37140]: pgmap v9330: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:20:51.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:51 smithi138 ceph-mon[36954]: pgmap v9331: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:20:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:51 smithi100 ceph-mon[32599]: pgmap v9331: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:20:51.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:51 smithi167 ceph-mon[37140]: pgmap v9331: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:20:53.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:53 smithi138 ceph-mon[36954]: pgmap v9332: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:20:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:53 smithi100 ceph-mon[32599]: pgmap v9332: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:20:53.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:53 smithi167 ceph-mon[37140]: pgmap v9332: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:20:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:54 smithi138 ceph-mon[36954]: pgmap v9333: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:20:54.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:54 smithi100 ceph-mon[32599]: pgmap v9333: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:20:54.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:54 smithi167 ceph-mon[37140]: pgmap v9333: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:20:57.393 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:57 smithi138 ceph-mon[36954]: pgmap v9334: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:20:57.447 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:57 smithi100 ceph-mon[32599]: pgmap v9334: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:20:57.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:57 smithi167 ceph-mon[37140]: pgmap v9334: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:20:57.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:20:57.687+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:20:57.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:20:57.688+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:20:57.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:20:57.688+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:20:57.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:20:57.688+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:20:57.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:20:57.688+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:20:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:20:59 smithi138 ceph-mon[36954]: pgmap v9335: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:20:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:20:59 smithi100 ceph-mon[32599]: pgmap v9335: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:20:59.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:20:59 smithi167 ceph-mon[37140]: pgmap v9335: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:21:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:21:01 smithi138 ceph-mon[36954]: pgmap v9336: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:21:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:21:01 smithi100 ceph-mon[32599]: pgmap v9336: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:21:01.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:21:01 smithi167 ceph-mon[37140]: pgmap v9336: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:21:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:21:03 smithi138 ceph-mon[36954]: pgmap v9337: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:21:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:21:03 smithi100 ceph-mon[32599]: pgmap v9337: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:21:03.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:21:03 smithi167 ceph-mon[37140]: pgmap v9337: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:21:05.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:21:05.338+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T18:21:05.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:21:05.338+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c004740 con 0x7fe5a81087a0 2024-07-26T18:21:05.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:21:05.338+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 129 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T18:21:05.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:21:05.338+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 129 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c100 con 0x7fe5a81087a0 2024-07-26T18:21:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:21:05 smithi138 ceph-mon[36954]: pgmap v9338: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:21:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:21:05 smithi100 ceph-mon[32599]: pgmap v9338: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:21:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:21:05 smithi167 ceph-mon[37140]: pgmap v9338: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:21:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:21:07 smithi138 ceph-mon[36954]: pgmap v9339: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:21:07.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:21:07 smithi100 ceph-mon[32599]: pgmap v9339: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:21:07.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:21:07 smithi167 ceph-mon[37140]: pgmap v9339: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:21:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:21:09 smithi138 ceph-mon[36954]: pgmap v9340: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:21:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:21:09 smithi138 ceph-mon[36954]: osdmap e630: 12 total, 12 up, 12 in 2024-07-26T18:21:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:21:09 smithi100 ceph-mon[32599]: pgmap v9340: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:21:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:21:09 smithi100 ceph-mon[32599]: osdmap e630: 12 total, 12 up, 12 in 2024-07-26T18:21:09.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:21:09 smithi167 ceph-mon[37140]: pgmap v9340: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:21:09.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:21:09 smithi167 ceph-mon[37140]: osdmap e630: 12 total, 12 up, 12 in 2024-07-26T18:21:11.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:21:11 smithi138 ceph-mon[36954]: pgmap v9342: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T18:21:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:21:11 smithi100 ceph-mon[32599]: pgmap v9342: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T18:21:11.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:21:11 smithi167 ceph-mon[37140]: pgmap v9342: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T18:21:12.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:21:12.704+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:21:12.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:21:12.704+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:21:12.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:21:12.704+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:21:12.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:21:12.704+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:21:12.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:21:12.704+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:21:13.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:21:13 smithi138 ceph-mon[36954]: pgmap v9343: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T18:21:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:21:13 smithi100 ceph-mon[32599]: pgmap v9343: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T18:21:13.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:21:13 smithi167 ceph-mon[37140]: pgmap v9343: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T18:21:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:21:15 smithi138 ceph-mon[36954]: pgmap v9344: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:21:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:21:15 smithi100 ceph-mon[32599]: pgmap v9344: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:21:15.526 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:21:15 smithi167 ceph-mon[37140]: pgmap v9344: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:21:17.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:21:17 smithi138 ceph-mon[36954]: pgmap v9345: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:21:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:21:17 smithi100 ceph-mon[32599]: pgmap v9345: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:21:17.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:21:17 smithi167 ceph-mon[37140]: pgmap v9345: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:21:19.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:21:19 smithi138 ceph-mon[36954]: pgmap v9346: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:21:19.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:21:19 smithi138 ceph-mon[36954]: osdmap e631: 12 total, 12 up, 12 in 2024-07-26T18:21:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:21:19 smithi100 ceph-mon[32599]: pgmap v9346: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:21:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:21:19 smithi100 ceph-mon[32599]: osdmap e631: 12 total, 12 up, 12 in 2024-07-26T18:21:19.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:21:19 smithi167 ceph-mon[37140]: pgmap v9346: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:21:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:21:19 smithi167 ceph-mon[37140]: osdmap e631: 12 total, 12 up, 12 in 2024-07-26T18:21:21.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:21:21 smithi138 ceph-mon[36954]: pgmap v9348: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:21:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:21:21 smithi100 ceph-mon[32599]: pgmap v9348: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:21:21.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:21:21 smithi167 ceph-mon[37140]: pgmap v9348: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:21:23.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:21:23 smithi138 ceph-mon[36954]: pgmap v9349: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:21:23.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:21:23 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:21:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:21:23 smithi100 ceph-mon[32599]: pgmap v9349: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:21:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:21:23 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:21:23.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:21:23 smithi167 ceph-mon[37140]: pgmap v9349: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:21:23.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:21:23 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:21:24.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:21:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:21:24.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:21:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:21:24.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:21:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:21:24.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:21:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:21:24.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:21:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:21:24.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:21:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:21:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:21:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:21:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:21:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:21:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:21:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:21:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:21:25 smithi100 ceph-mon[32599]: pgmap v9350: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:21:25.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:21:25 smithi167 ceph-mon[37140]: pgmap v9350: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:21:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:21:25 smithi138 ceph-mon[36954]: pgmap v9350: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:21:27.492 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:21:27 smithi138 ceph-mon[36954]: pgmap v9351: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:21:27.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:21:27 smithi100 ceph-mon[32599]: pgmap v9351: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:21:27.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:21:27 smithi167 ceph-mon[37140]: pgmap v9351: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:21:27.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:21:27.720+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:21:27.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:21:27.721+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:21:27.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:21:27.721+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:21:27.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:21:27.721+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:21:27.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:21:27.721+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:21:29.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:21:29 smithi100 ceph-mon[32599]: pgmap v9352: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:21:29.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:21:29 smithi167 ceph-mon[37140]: pgmap v9352: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:21:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:21:29 smithi138 ceph-mon[36954]: pgmap v9352: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:21:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:21:31 smithi100 ceph-mon[32599]: pgmap v9353: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:21:31.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:21:31 smithi167 ceph-mon[37140]: pgmap v9353: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:21:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:21:31 smithi138 ceph-mon[36954]: pgmap v9353: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:21:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:21:33 smithi100 ceph-mon[32599]: pgmap v9354: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:21:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:21:33 smithi167 ceph-mon[37140]: pgmap v9354: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:21:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:21:33 smithi138 ceph-mon[36954]: pgmap v9354: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:21:34.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:21:34 smithi138 ceph-mon[36954]: pgmap v9355: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:21:34.747 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:21:34 smithi100 ceph-mon[32599]: pgmap v9355: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:21:34.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:21:34 smithi167 ceph-mon[37140]: pgmap v9355: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:21:37.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:21:37 smithi138 ceph-mon[36954]: pgmap v9356: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:21:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:21:37 smithi100 ceph-mon[32599]: pgmap v9356: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:21:37.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:21:37 smithi167 ceph-mon[37140]: pgmap v9356: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:21:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:21:39 smithi138 ceph-mon[36954]: pgmap v9357: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:21:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:21:39 smithi100 ceph-mon[32599]: pgmap v9357: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:21:39.631 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:21:39 smithi167 ceph-mon[37140]: pgmap v9357: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:21:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:21:41 smithi138 ceph-mon[36954]: pgmap v9358: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:21:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:21:41 smithi100 ceph-mon[32599]: pgmap v9358: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:21:41.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:21:41 smithi167 ceph-mon[37140]: pgmap v9358: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:21:42.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:21:42.735+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:21:42.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:21:42.736+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:21:42.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:21:42.736+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:21:42.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:21:42.736+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:21:42.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:21:42.736+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:21:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:21:43 smithi138 ceph-mon[36954]: pgmap v9359: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:21:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:21:43 smithi100 ceph-mon[32599]: pgmap v9359: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:21:43.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:21:43 smithi167 ceph-mon[37140]: pgmap v9359: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:21:45.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:21:45 smithi138 ceph-mon[36954]: pgmap v9360: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:21:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:21:45 smithi100 ceph-mon[32599]: pgmap v9360: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:21:45.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:21:45 smithi167 ceph-mon[37140]: pgmap v9360: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:21:47.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:21:47 smithi138 ceph-mon[36954]: pgmap v9361: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:21:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:21:47 smithi100 ceph-mon[32599]: pgmap v9361: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:21:47.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:21:47 smithi167 ceph-mon[37140]: pgmap v9361: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:21:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:21:49 smithi138 ceph-mon[36954]: pgmap v9362: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:21:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:21:49 smithi100 ceph-mon[32599]: pgmap v9362: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:21:49.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:21:49 smithi167 ceph-mon[37140]: pgmap v9362: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:21:51.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:21:51 smithi138 ceph-mon[36954]: pgmap v9363: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:21:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:21:51 smithi100 ceph-mon[32599]: pgmap v9363: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:21:51.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:21:51 smithi167 ceph-mon[37140]: pgmap v9363: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:21:53.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:21:53 smithi138 ceph-mon[36954]: pgmap v9364: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:21:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:21:53 smithi100 ceph-mon[32599]: pgmap v9364: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:21:53.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:21:53 smithi167 ceph-mon[37140]: pgmap v9364: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:21:55.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:21:55 smithi138 ceph-mon[36954]: pgmap v9365: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:21:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:21:55 smithi100 ceph-mon[32599]: pgmap v9365: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:21:55.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:21:55 smithi167 ceph-mon[37140]: pgmap v9365: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:21:57.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:21:57 smithi138 ceph-mon[36954]: pgmap v9366: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:21:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:21:57 smithi100 ceph-mon[32599]: pgmap v9366: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:21:57.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:21:57 smithi167 ceph-mon[37140]: pgmap v9366: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:21:57.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:21:57.751+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:21:57.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:21:57.752+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:21:57.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:21:57.752+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:21:57.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:21:57.752+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:21:57.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:21:57.752+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:21:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:21:59 smithi138 ceph-mon[36954]: pgmap v9367: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:21:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:21:59 smithi100 ceph-mon[32599]: pgmap v9367: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:21:59.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:21:59 smithi167 ceph-mon[37140]: pgmap v9367: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:22:01.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:22:01 smithi138 ceph-mon[36954]: pgmap v9368: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:22:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:22:01 smithi100 ceph-mon[32599]: pgmap v9368: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:22:01.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:22:01 smithi167 ceph-mon[37140]: pgmap v9368: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:22:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:22:03 smithi138 ceph-mon[36954]: pgmap v9369: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:22:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:22:03 smithi100 ceph-mon[32599]: pgmap v9369: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:22:03.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:22:03 smithi167 ceph-mon[37140]: pgmap v9369: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:22:04.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:22:04 smithi167 ceph-mon[37140]: pgmap v9370: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:22:04.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:22:04 smithi138 ceph-mon[36954]: pgmap v9370: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:22:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:22:04 smithi100 ceph-mon[32599]: pgmap v9370: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:22:07.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:22:07 smithi138 ceph-mon[36954]: pgmap v9371: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:22:07.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:22:07 smithi100 ceph-mon[32599]: pgmap v9371: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:22:07.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:22:07 smithi167 ceph-mon[37140]: pgmap v9371: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:22:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:22:09 smithi138 ceph-mon[36954]: pgmap v9372: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:22:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:22:09 smithi138 ceph-mon[36954]: osdmap e632: 12 total, 12 up, 12 in 2024-07-26T18:22:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:22:09 smithi100 ceph-mon[32599]: pgmap v9372: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:22:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:22:09 smithi100 ceph-mon[32599]: osdmap e632: 12 total, 12 up, 12 in 2024-07-26T18:22:09.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:22:09 smithi167 ceph-mon[37140]: pgmap v9372: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:22:09.536 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:22:09 smithi167 ceph-mon[37140]: osdmap e632: 12 total, 12 up, 12 in 2024-07-26T18:22:11.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:22:11 smithi138 ceph-mon[36954]: pgmap v9374: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T18:22:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:22:11 smithi100 ceph-mon[32599]: pgmap v9374: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T18:22:11.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:22:11 smithi167 ceph-mon[37140]: pgmap v9374: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T18:22:12.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:22:12.767+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:22:12.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:22:12.768+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:22:12.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:22:12.768+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:22:12.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:22:12.768+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:22:12.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:22:12.768+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:22:13.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:22:13 smithi138 ceph-mon[36954]: pgmap v9375: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T18:22:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:22:13 smithi100 ceph-mon[32599]: pgmap v9375: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T18:22:13.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:22:13 smithi167 ceph-mon[37140]: pgmap v9375: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T18:22:15.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:22:15 smithi138 ceph-mon[36954]: pgmap v9376: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:22:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:22:15 smithi100 ceph-mon[32599]: pgmap v9376: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:22:15.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:22:15 smithi167 ceph-mon[37140]: pgmap v9376: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:22:17.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:22:17 smithi138 ceph-mon[36954]: pgmap v9377: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:22:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:22:17 smithi100 ceph-mon[32599]: pgmap v9377: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:22:17.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:22:17 smithi167 ceph-mon[37140]: pgmap v9377: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:22:19.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:22:19 smithi138 ceph-mon[36954]: pgmap v9378: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:22:19.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:22:19 smithi138 ceph-mon[36954]: osdmap e633: 12 total, 12 up, 12 in 2024-07-26T18:22:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:22:19 smithi100 ceph-mon[32599]: pgmap v9378: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:22:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:22:19 smithi100 ceph-mon[32599]: osdmap e633: 12 total, 12 up, 12 in 2024-07-26T18:22:19.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:22:19 smithi167 ceph-mon[37140]: pgmap v9378: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:22:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:22:19 smithi167 ceph-mon[37140]: osdmap e633: 12 total, 12 up, 12 in 2024-07-26T18:22:21.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:22:21 smithi138 ceph-mon[36954]: pgmap v9380: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:22:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:22:21 smithi100 ceph-mon[32599]: pgmap v9380: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:22:21.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:22:21 smithi167 ceph-mon[37140]: pgmap v9380: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:22:23.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:22:23 smithi138 ceph-mon[36954]: pgmap v9381: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:22:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:22:23 smithi100 ceph-mon[32599]: pgmap v9381: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:22:23.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:22:23 smithi167 ceph-mon[37140]: pgmap v9381: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:22:24.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:22:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:22:24.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:22:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:22:24.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:22:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:22:24.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:22:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:22:24.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:22:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:22:24.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:22:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:22:24.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:22:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:22:24.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:22:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:22:24.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:22:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:22:24.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:22:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:22:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:22:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:22:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:22:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:22:25.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:22:25 smithi138 ceph-mon[36954]: pgmap v9382: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:22:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:22:25 smithi100 ceph-mon[32599]: pgmap v9382: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:22:25.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:22:25 smithi167 ceph-mon[37140]: pgmap v9382: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:22:27.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:22:27 smithi138 ceph-mon[36954]: pgmap v9383: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:22:27.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:22:27 smithi100 ceph-mon[32599]: pgmap v9383: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:22:27.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:22:27 smithi167 ceph-mon[37140]: pgmap v9383: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:22:27.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:22:27.784+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:22:27.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:22:27.784+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:22:27.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:22:27.784+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:22:27.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:22:27.784+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:22:27.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:22:27.784+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:22:29.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:22:29 smithi138 ceph-mon[36954]: pgmap v9384: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:22:29.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:22:29 smithi100 ceph-mon[32599]: pgmap v9384: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:22:29.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:22:29 smithi167 ceph-mon[37140]: pgmap v9384: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:22:31.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:22:31 smithi138 ceph-mon[36954]: pgmap v9385: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:22:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:22:31 smithi100 ceph-mon[32599]: pgmap v9385: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:22:31.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:22:31 smithi167 ceph-mon[37140]: pgmap v9385: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:22:33.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:22:33 smithi138 ceph-mon[36954]: pgmap v9386: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:22:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:22:33 smithi100 ceph-mon[32599]: pgmap v9386: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:22:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:22:33 smithi167 ceph-mon[37140]: pgmap v9386: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:22:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:22:35 smithi138 ceph-mon[36954]: pgmap v9387: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:22:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:22:35 smithi100 ceph-mon[32599]: pgmap v9387: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:22:35.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:22:35 smithi167 ceph-mon[37140]: pgmap v9387: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:22:37.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:22:37 smithi138 ceph-mon[36954]: pgmap v9388: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:22:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:22:37 smithi100 ceph-mon[32599]: pgmap v9388: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:22:37.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:22:37 smithi167 ceph-mon[37140]: pgmap v9388: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:22:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:22:39 smithi138 ceph-mon[36954]: pgmap v9389: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:22:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:22:39 smithi100 ceph-mon[32599]: pgmap v9389: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:22:39.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:22:39 smithi167 ceph-mon[37140]: pgmap v9389: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:22:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:22:41 smithi138 ceph-mon[36954]: pgmap v9390: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:22:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:22:41 smithi100 ceph-mon[32599]: pgmap v9390: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:22:41.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:22:41 smithi167 ceph-mon[37140]: pgmap v9390: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:22:42.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:22:42.793+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:22:42.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:22:42.793+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:22:42.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:22:42.793+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:22:42.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:22:42.793+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:22:42.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:22:42.793+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:22:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:22:43 smithi138 ceph-mon[36954]: pgmap v9391: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:22:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:22:43 smithi100 ceph-mon[32599]: pgmap v9391: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:22:43.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:22:43 smithi167 ceph-mon[37140]: pgmap v9391: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:22:45.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:22:45 smithi138 ceph-mon[36954]: pgmap v9392: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:22:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:22:45 smithi100 ceph-mon[32599]: pgmap v9392: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:22:45.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:22:45 smithi167 ceph-mon[37140]: pgmap v9392: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:22:47.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:22:47 smithi138 ceph-mon[36954]: pgmap v9393: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:22:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:22:47 smithi100 ceph-mon[32599]: pgmap v9393: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:22:47.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:22:47 smithi167 ceph-mon[37140]: pgmap v9393: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:22:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:22:49 smithi138 ceph-mon[36954]: pgmap v9394: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:22:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:22:49 smithi100 ceph-mon[32599]: pgmap v9394: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:22:49.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:22:49 smithi167 ceph-mon[37140]: pgmap v9394: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:22:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:22:51 smithi100 ceph-mon[32599]: pgmap v9395: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:22:51.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:22:51 smithi167 ceph-mon[37140]: pgmap v9395: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:22:51.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:22:51 smithi138 ceph-mon[36954]: pgmap v9395: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:22:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:22:53 smithi100 ceph-mon[32599]: pgmap v9396: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:22:53.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:22:53 smithi167 ceph-mon[37140]: pgmap v9396: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:22:53.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:22:53 smithi138 ceph-mon[36954]: pgmap v9396: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:22:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:22:55 smithi100 ceph-mon[32599]: pgmap v9397: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:22:55.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:22:55 smithi167 ceph-mon[37140]: pgmap v9397: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:22:55.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:22:55 smithi138 ceph-mon[36954]: pgmap v9397: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:22:57.493 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:22:57 smithi138 ceph-mon[36954]: pgmap v9398: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:22:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:22:57 smithi100 ceph-mon[32599]: pgmap v9398: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:22:57.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:22:57 smithi167 ceph-mon[37140]: pgmap v9398: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:22:57.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:22:57.799+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:22:57.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:22:57.800+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:22:57.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:22:57.800+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:22:57.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:22:57.800+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:22:57.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:22:57.800+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:22:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:22:59 smithi100 ceph-mon[32599]: pgmap v9399: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:22:59.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:22:59 smithi167 ceph-mon[37140]: pgmap v9399: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:22:59.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:22:59 smithi138 ceph-mon[36954]: pgmap v9399: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:23:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:01 smithi100 ceph-mon[32599]: pgmap v9400: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:23:01.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:01 smithi167 ceph-mon[37140]: pgmap v9400: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:23:01.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:01 smithi138 ceph-mon[36954]: pgmap v9400: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:23:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:03 smithi100 ceph-mon[32599]: pgmap v9401: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:23:03.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:03 smithi167 ceph-mon[37140]: pgmap v9401: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:23:03.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:03 smithi138 ceph-mon[36954]: pgmap v9401: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:23:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:05 smithi100 ceph-mon[32599]: pgmap v9402: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:23:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:05 smithi167 ceph-mon[37140]: pgmap v9402: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:23:05.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:05 smithi138 ceph-mon[36954]: pgmap v9402: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:23:07.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:07 smithi100 ceph-mon[32599]: pgmap v9403: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:23:07.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:07 smithi167 ceph-mon[37140]: pgmap v9403: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:23:07.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:07 smithi138 ceph-mon[36954]: pgmap v9403: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:23:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:08 smithi100 ceph-mon[32599]: osdmap e634: 12 total, 12 up, 12 in 2024-07-26T18:23:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:08 smithi167 ceph-mon[37140]: osdmap e634: 12 total, 12 up, 12 in 2024-07-26T18:23:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:08 smithi138 ceph-mon[36954]: osdmap e634: 12 total, 12 up, 12 in 2024-07-26T18:23:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:09 smithi100 ceph-mon[32599]: pgmap v9405: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:23:09.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:09 smithi167 ceph-mon[37140]: pgmap v9405: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:23:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:09 smithi138 ceph-mon[36954]: pgmap v9405: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:23:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:11 smithi100 ceph-mon[32599]: pgmap v9406: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T18:23:11.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:11 smithi167 ceph-mon[37140]: pgmap v9406: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T18:23:11.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:11 smithi138 ceph-mon[36954]: pgmap v9406: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T18:23:12.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:23:12.807+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:23:12.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:23:12.808+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:23:12.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:23:12.808+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:23:12.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:23:12.808+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:23:12.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:23:12.808+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:23:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:13 smithi100 ceph-mon[32599]: pgmap v9407: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T18:23:13.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:13 smithi167 ceph-mon[37140]: pgmap v9407: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T18:23:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:13 smithi138 ceph-mon[36954]: pgmap v9407: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T18:23:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:15 smithi100 ceph-mon[32599]: pgmap v9408: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:23:15.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:15 smithi167 ceph-mon[37140]: pgmap v9408: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:23:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:15 smithi138 ceph-mon[36954]: pgmap v9408: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:23:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:17 smithi100 ceph-mon[32599]: pgmap v9409: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:23:17.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:17 smithi167 ceph-mon[37140]: pgmap v9409: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:23:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:17 smithi138 ceph-mon[36954]: pgmap v9409: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:23:19.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:19 smithi167 ceph-mon[37140]: pgmap v9410: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 188 B/s wr, 6 op/s 2024-07-26T18:23:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:19 smithi167 ceph-mon[37140]: osdmap e635: 12 total, 12 up, 12 in 2024-07-26T18:23:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:19 smithi138 ceph-mon[36954]: pgmap v9410: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 188 B/s wr, 6 op/s 2024-07-26T18:23:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:19 smithi138 ceph-mon[36954]: osdmap e635: 12 total, 12 up, 12 in 2024-07-26T18:23:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:19 smithi100 ceph-mon[32599]: pgmap v9410: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 188 B/s wr, 6 op/s 2024-07-26T18:23:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:19 smithi100 ceph-mon[32599]: osdmap e635: 12 total, 12 up, 12 in 2024-07-26T18:23:21.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:21 smithi167 ceph-mon[37140]: pgmap v9412: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:23:21.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:21 smithi138 ceph-mon[36954]: pgmap v9412: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:23:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:21 smithi100 ceph-mon[32599]: pgmap v9412: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:23:23.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:23 smithi167 ceph-mon[37140]: pgmap v9413: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:23:23.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:23 smithi138 ceph-mon[36954]: pgmap v9413: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:23:23.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:23 smithi100 ceph-mon[32599]: pgmap v9413: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:23:24.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:24 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:23:24.608 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:24 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:23:24.659 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:24 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:23:25.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:25 smithi167 ceph-mon[37140]: pgmap v9414: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:23:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:25 smithi138 ceph-mon[36954]: pgmap v9414: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:23:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:25 smithi100 ceph-mon[32599]: pgmap v9414: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:23:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:23:26.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:23:26.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:23:26.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:23:26.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:23:26.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:23:26.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:23:26.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:23:26.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:23:26.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:26 smithi138 ceph-mon[36954]: pgmap v9415: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:23:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:23:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:23:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:23:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:23:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:23:26.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:23:26.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:23:26.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:23:26.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:23:26.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:26 smithi100 ceph-mon[32599]: pgmap v9415: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:23:26.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:23:26.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:23:26.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:23:26.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:23:26.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:23:26.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:23:26.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:23:26.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:23:26.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:23:26.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:26 smithi167 ceph-mon[37140]: pgmap v9415: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:23:27.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:23:27.815+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:23:27.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:23:27.815+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:23:27.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:23:27.815+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:23:27.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:23:27.815+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:23:27.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:23:27.815+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:23:29.415 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:29 smithi138 ceph-mon[36954]: pgmap v9416: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:23:29.449 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:29 smithi100 ceph-mon[32599]: pgmap v9416: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:23:29.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:29 smithi167 ceph-mon[37140]: pgmap v9416: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:23:31.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:31 smithi138 ceph-mon[36954]: pgmap v9417: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:23:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:31 smithi100 ceph-mon[32599]: pgmap v9417: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:23:31.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:31 smithi167 ceph-mon[37140]: pgmap v9417: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:23:33.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:33 smithi138 ceph-mon[36954]: pgmap v9418: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:23:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:33 smithi100 ceph-mon[32599]: pgmap v9418: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:23:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:33 smithi167 ceph-mon[37140]: pgmap v9418: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:23:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:35 smithi138 ceph-mon[36954]: pgmap v9419: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:23:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:35 smithi100 ceph-mon[32599]: pgmap v9419: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:23:35.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:35 smithi167 ceph-mon[37140]: pgmap v9419: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:23:37.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:37 smithi138 ceph-mon[36954]: pgmap v9420: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:23:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:37 smithi100 ceph-mon[32599]: pgmap v9420: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:23:37.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:37 smithi167 ceph-mon[37140]: pgmap v9420: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:23:39.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:39 smithi100 ceph-mon[32599]: pgmap v9421: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:23:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:39 smithi138 ceph-mon[36954]: pgmap v9421: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:23:39.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:39 smithi167 ceph-mon[37140]: pgmap v9421: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:23:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:41 smithi138 ceph-mon[36954]: pgmap v9422: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:23:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:41 smithi100 ceph-mon[32599]: pgmap v9422: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:23:41.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:41 smithi167 ceph-mon[37140]: pgmap v9422: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:23:42.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:23:42.831+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:23:42.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:23:42.832+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:23:42.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:23:42.832+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:23:42.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:23:42.832+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:23:42.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:23:42.832+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:23:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:43 smithi138 ceph-mon[36954]: pgmap v9423: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:23:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:43 smithi100 ceph-mon[32599]: pgmap v9423: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:23:43.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:43 smithi167 ceph-mon[37140]: pgmap v9423: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:23:45.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:45 smithi138 ceph-mon[36954]: pgmap v9424: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:23:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:45 smithi100 ceph-mon[32599]: pgmap v9424: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:23:45.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:45 smithi167 ceph-mon[37140]: pgmap v9424: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:23:47.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:47 smithi138 ceph-mon[36954]: pgmap v9425: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:23:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:47 smithi100 ceph-mon[32599]: pgmap v9425: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:23:47.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:47 smithi167 ceph-mon[37140]: pgmap v9425: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:23:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:49 smithi138 ceph-mon[36954]: pgmap v9426: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:23:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:49 smithi100 ceph-mon[32599]: pgmap v9426: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:23:49.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:49 smithi167 ceph-mon[37140]: pgmap v9426: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:23:51.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:51 smithi138 ceph-mon[36954]: pgmap v9427: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:23:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:51 smithi100 ceph-mon[32599]: pgmap v9427: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:23:51.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:51 smithi167 ceph-mon[37140]: pgmap v9427: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:23:53.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:53 smithi138 ceph-mon[36954]: pgmap v9428: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:23:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:53 smithi100 ceph-mon[32599]: pgmap v9428: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:23:53.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:53 smithi167 ceph-mon[37140]: pgmap v9428: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:23:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:55 smithi100 ceph-mon[32599]: pgmap v9429: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:23:55.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:55 smithi167 ceph-mon[37140]: pgmap v9429: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:23:55.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:55 smithi138 ceph-mon[36954]: pgmap v9429: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:23:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:57 smithi100 ceph-mon[32599]: pgmap v9430: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:23:57.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:57 smithi167 ceph-mon[37140]: pgmap v9430: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:23:57.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:57 smithi138 ceph-mon[36954]: pgmap v9430: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:23:57.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:23:57.834+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:23:57.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:23:57.834+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:23:57.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:23:57.834+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:23:57.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:23:57.834+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:23:57.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:23:57.835+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:23:59.493 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:23:59 smithi138 ceph-mon[36954]: pgmap v9431: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:23:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:23:59 smithi100 ceph-mon[32599]: pgmap v9431: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:23:59.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:23:59 smithi167 ceph-mon[37140]: pgmap v9431: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:24:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:01 smithi100 ceph-mon[32599]: pgmap v9432: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:24:01.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:01 smithi167 ceph-mon[37140]: pgmap v9432: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:24:01.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:01 smithi138 ceph-mon[36954]: pgmap v9432: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:24:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:03 smithi100 ceph-mon[32599]: pgmap v9433: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:24:03.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:03 smithi167 ceph-mon[37140]: pgmap v9433: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:24:03.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:03 smithi138 ceph-mon[36954]: pgmap v9433: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:24:05.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:24:05.364+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c003690 con 0x7fe5a81087a0 2024-07-26T18:24:05.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:24:05.364+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T18:24:05.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:24:05.364+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 130 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c003690 con 0x7fe5a81087a0 2024-07-26T18:24:05.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:24:05.364+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 130 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T18:24:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:05 smithi100 ceph-mon[32599]: pgmap v9434: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:24:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:05 smithi167 ceph-mon[37140]: pgmap v9434: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:24:05.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:05 smithi138 ceph-mon[36954]: pgmap v9434: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:24:07.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:07 smithi100 ceph-mon[32599]: pgmap v9435: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:24:07.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:07 smithi167 ceph-mon[37140]: pgmap v9435: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:24:07.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:07 smithi138 ceph-mon[36954]: pgmap v9435: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:24:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:08 smithi100 ceph-mon[32599]: osdmap e636: 12 total, 12 up, 12 in 2024-07-26T18:24:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:08 smithi167 ceph-mon[37140]: osdmap e636: 12 total, 12 up, 12 in 2024-07-26T18:24:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:08 smithi138 ceph-mon[36954]: osdmap e636: 12 total, 12 up, 12 in 2024-07-26T18:24:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:09 smithi100 ceph-mon[32599]: pgmap v9437: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:24:09.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:09 smithi167 ceph-mon[37140]: pgmap v9437: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:24:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:09 smithi138 ceph-mon[36954]: pgmap v9437: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:24:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:11 smithi100 ceph-mon[32599]: pgmap v9438: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T18:24:11.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:11 smithi167 ceph-mon[37140]: pgmap v9438: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T18:24:11.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:11 smithi138 ceph-mon[36954]: pgmap v9438: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T18:24:12.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:24:12.850+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:24:12.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:24:12.851+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:24:12.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:24:12.851+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:24:12.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:24:12.851+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:24:12.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:24:12.851+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:24:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:13 smithi100 ceph-mon[32599]: pgmap v9439: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T18:24:13.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:13 smithi167 ceph-mon[37140]: pgmap v9439: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T18:24:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:13 smithi138 ceph-mon[36954]: pgmap v9439: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T18:24:15.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:15 smithi100 ceph-mon[32599]: pgmap v9440: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:24:15.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:15 smithi167 ceph-mon[37140]: pgmap v9440: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:24:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:15 smithi138 ceph-mon[36954]: pgmap v9440: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:24:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:17 smithi100 ceph-mon[32599]: pgmap v9441: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:24:17.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:17 smithi167 ceph-mon[37140]: pgmap v9441: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:24:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:17 smithi138 ceph-mon[36954]: pgmap v9441: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:24:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:19 smithi100 ceph-mon[32599]: pgmap v9442: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 188 B/s wr, 6 op/s 2024-07-26T18:24:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:19 smithi100 ceph-mon[32599]: osdmap e637: 12 total, 12 up, 12 in 2024-07-26T18:24:19.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:19 smithi167 ceph-mon[37140]: pgmap v9442: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 188 B/s wr, 6 op/s 2024-07-26T18:24:19.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:19 smithi167 ceph-mon[37140]: osdmap e637: 12 total, 12 up, 12 in 2024-07-26T18:24:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:19 smithi138 ceph-mon[36954]: pgmap v9442: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 188 B/s wr, 6 op/s 2024-07-26T18:24:19.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:19 smithi138 ceph-mon[36954]: osdmap e637: 12 total, 12 up, 12 in 2024-07-26T18:24:21.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:21 smithi167 ceph-mon[37140]: pgmap v9444: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:24:21.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:21 smithi138 ceph-mon[36954]: pgmap v9444: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:24:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:21 smithi100 ceph-mon[32599]: pgmap v9444: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:24:23.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:23 smithi167 ceph-mon[37140]: pgmap v9445: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:24:23.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:23 smithi138 ceph-mon[36954]: pgmap v9445: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:24:23.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:23 smithi100 ceph-mon[32599]: pgmap v9445: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:24:25.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:25 smithi167 ceph-mon[37140]: pgmap v9446: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:24:25.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:25 smithi138 ceph-mon[36954]: pgmap v9446: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:24:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:25 smithi100 ceph-mon[32599]: pgmap v9446: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:24:26.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:24:26.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:24:26.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:24:26.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:24:26.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:24:26.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:24:26.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:26 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:24:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:24:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:24:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:24:26.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:24:26.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:24:26.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:24:26.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:26 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:24:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:24:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:24:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:24:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:24:26.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:24:26.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:24:26.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:26 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:24:27.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:27 smithi167 ceph-mon[37140]: pgmap v9447: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:24:27.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:24:27.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:24:27.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:24:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:27 smithi138 ceph-mon[36954]: pgmap v9447: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:24:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:24:27.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:24:27.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:24:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:27 smithi100 ceph-mon[32599]: pgmap v9447: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:24:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:24:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:24:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:24:27.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:24:27.866+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:24:27.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:24:27.866+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:24:27.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:24:27.866+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:24:27.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:24:27.866+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:24:27.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:24:27.867+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:24:29.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:29 smithi167 ceph-mon[37140]: pgmap v9448: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:24:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:29 smithi138 ceph-mon[36954]: pgmap v9448: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:24:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:29 smithi100 ceph-mon[32599]: pgmap v9448: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:24:30.531 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:30 smithi167 ceph-mon[37140]: pgmap v9449: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:24:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:30 smithi138 ceph-mon[36954]: pgmap v9449: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:24:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:30 smithi100 ceph-mon[32599]: pgmap v9449: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:24:33.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:33 smithi138 ceph-mon[36954]: pgmap v9450: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:24:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:33 smithi100 ceph-mon[32599]: pgmap v9450: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:24:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:33 smithi167 ceph-mon[37140]: pgmap v9450: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:24:34.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:34 smithi138 ceph-mon[36954]: pgmap v9451: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:24:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:34 smithi100 ceph-mon[32599]: pgmap v9451: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:24:34.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:34 smithi167 ceph-mon[37140]: pgmap v9451: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:24:37.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:37 smithi138 ceph-mon[36954]: pgmap v9452: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:24:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:37 smithi100 ceph-mon[32599]: pgmap v9452: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:24:37.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:37 smithi167 ceph-mon[37140]: pgmap v9452: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:24:39.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:39 smithi138 ceph-mon[36954]: pgmap v9453: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:24:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:39 smithi100 ceph-mon[32599]: pgmap v9453: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:24:39.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:39 smithi167 ceph-mon[37140]: pgmap v9453: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:24:41.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:41 smithi138 ceph-mon[36954]: pgmap v9454: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:24:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:41 smithi100 ceph-mon[32599]: pgmap v9454: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:24:41.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:41 smithi167 ceph-mon[37140]: pgmap v9454: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:24:42.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:24:42.882+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:24:42.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:24:42.882+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:24:42.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:24:42.882+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:24:42.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:24:42.882+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:24:42.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:24:42.882+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:24:43.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:43 smithi138 ceph-mon[36954]: pgmap v9455: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:24:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:43 smithi100 ceph-mon[32599]: pgmap v9455: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:24:43.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:43 smithi167 ceph-mon[37140]: pgmap v9455: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:24:45.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:45 smithi138 ceph-mon[36954]: pgmap v9456: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:24:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:45 smithi100 ceph-mon[32599]: pgmap v9456: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:24:45.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:45 smithi167 ceph-mon[37140]: pgmap v9456: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:24:47.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:47 smithi138 ceph-mon[36954]: pgmap v9457: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:24:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:47 smithi100 ceph-mon[32599]: pgmap v9457: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:24:47.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:47 smithi167 ceph-mon[37140]: pgmap v9457: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:24:49.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:49 smithi138 ceph-mon[36954]: pgmap v9458: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:24:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:49 smithi100 ceph-mon[32599]: pgmap v9458: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:24:49.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:49 smithi167 ceph-mon[37140]: pgmap v9458: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:24:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:51 smithi100 ceph-mon[32599]: pgmap v9459: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:24:51.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:51 smithi167 ceph-mon[37140]: pgmap v9459: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:24:51.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:51 smithi138 ceph-mon[36954]: pgmap v9459: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:24:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:53 smithi100 ceph-mon[32599]: pgmap v9460: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:24:53.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:53 smithi167 ceph-mon[37140]: pgmap v9460: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:24:53.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:53 smithi138 ceph-mon[36954]: pgmap v9460: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:24:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:55 smithi100 ceph-mon[32599]: pgmap v9461: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:24:55.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:55 smithi167 ceph-mon[37140]: pgmap v9461: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:24:55.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:55 smithi138 ceph-mon[36954]: pgmap v9461: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:24:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:57 smithi100 ceph-mon[32599]: pgmap v9462: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:24:57.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:57 smithi167 ceph-mon[37140]: pgmap v9462: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:24:57.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:57 smithi138 ceph-mon[36954]: pgmap v9462: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:24:57.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:24:57.898+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:24:57.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:24:57.899+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:24:57.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:24:57.899+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:24:57.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:24:57.899+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:24:57.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:24:57.899+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:24:59.492 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:24:59 smithi138 ceph-mon[36954]: pgmap v9463: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:24:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:24:59 smithi100 ceph-mon[32599]: pgmap v9463: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:24:59.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:24:59 smithi167 ceph-mon[37140]: pgmap v9463: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:25:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:25:01 smithi100 ceph-mon[32599]: pgmap v9464: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:25:01.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:25:01 smithi167 ceph-mon[37140]: pgmap v9464: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:25:01.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:25:01 smithi138 ceph-mon[36954]: pgmap v9464: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:25:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:25:03 smithi100 ceph-mon[32599]: pgmap v9465: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:25:03.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:25:03 smithi167 ceph-mon[37140]: pgmap v9465: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:25:03.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:25:03 smithi138 ceph-mon[36954]: pgmap v9465: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:25:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:25:05 smithi100 ceph-mon[32599]: pgmap v9466: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:25:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:25:05 smithi167 ceph-mon[37140]: pgmap v9466: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:25:05.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:25:05 smithi138 ceph-mon[36954]: pgmap v9466: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:25:07.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:25:07 smithi100 ceph-mon[32599]: pgmap v9467: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:25:07.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:25:07 smithi167 ceph-mon[37140]: pgmap v9467: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:25:07.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:25:07 smithi138 ceph-mon[36954]: pgmap v9467: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:25:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:25:08 smithi100 ceph-mon[32599]: osdmap e638: 12 total, 12 up, 12 in 2024-07-26T18:25:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:25:08 smithi167 ceph-mon[37140]: osdmap e638: 12 total, 12 up, 12 in 2024-07-26T18:25:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:25:08 smithi138 ceph-mon[36954]: osdmap e638: 12 total, 12 up, 12 in 2024-07-26T18:25:09.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:25:09 smithi100 ceph-mon[32599]: pgmap v9469: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:25:09.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:25:09 smithi167 ceph-mon[37140]: pgmap v9469: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:25:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:25:09 smithi138 ceph-mon[36954]: pgmap v9469: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:25:11.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:25:11 smithi100 ceph-mon[32599]: pgmap v9470: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:25:11.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:25:11 smithi167 ceph-mon[37140]: pgmap v9470: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:25:11.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:25:11 smithi138 ceph-mon[36954]: pgmap v9470: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:25:12.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:25:12.913+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:25:12.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:25:12.914+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:25:12.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:25:12.914+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:25:12.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:25:12.914+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:25:12.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:25:12.914+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:25:13.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:25:13 smithi167 ceph-mon[37140]: pgmap v9471: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:25:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:25:13 smithi138 ceph-mon[36954]: pgmap v9471: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:25:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:25:13 smithi100 ceph-mon[32599]: pgmap v9471: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:25:15.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:25:15 smithi167 ceph-mon[37140]: pgmap v9472: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:25:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:25:15 smithi138 ceph-mon[36954]: pgmap v9472: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:25:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:25:15 smithi100 ceph-mon[32599]: pgmap v9472: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:25:17.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:25:17 smithi167 ceph-mon[37140]: pgmap v9473: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:25:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:25:17 smithi138 ceph-mon[36954]: pgmap v9473: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:25:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:25:17 smithi100 ceph-mon[32599]: pgmap v9473: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:25:19.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:25:19 smithi167 ceph-mon[37140]: pgmap v9474: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 187 B/s wr, 6 op/s 2024-07-26T18:25:19.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:25:19 smithi167 ceph-mon[37140]: osdmap e639: 12 total, 12 up, 12 in 2024-07-26T18:25:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:25:19 smithi138 ceph-mon[36954]: pgmap v9474: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 187 B/s wr, 6 op/s 2024-07-26T18:25:19.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:25:19 smithi138 ceph-mon[36954]: osdmap e639: 12 total, 12 up, 12 in 2024-07-26T18:25:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:25:19 smithi100 ceph-mon[32599]: pgmap v9474: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 187 B/s wr, 6 op/s 2024-07-26T18:25:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:25:19 smithi100 ceph-mon[32599]: osdmap e639: 12 total, 12 up, 12 in 2024-07-26T18:25:21.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:25:21 smithi167 ceph-mon[37140]: pgmap v9476: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:25:21.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:25:21 smithi138 ceph-mon[36954]: pgmap v9476: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:25:21.747 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:25:21 smithi100 ceph-mon[32599]: pgmap v9476: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:25:23.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:25:23 smithi167 ceph-mon[37140]: pgmap v9477: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:25:23.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:25:23 smithi138 ceph-mon[36954]: pgmap v9477: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:25:23.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:25:23 smithi100 ceph-mon[32599]: pgmap v9477: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:25:25.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:25:25 smithi167 ceph-mon[37140]: pgmap v9478: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:25:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:25:25 smithi138 ceph-mon[36954]: pgmap v9478: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:25:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:25:25 smithi100 ceph-mon[32599]: pgmap v9478: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:25:27.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:25:27 smithi167 ceph-mon[37140]: pgmap v9479: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:25:27.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:25:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:25:27.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:25:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:25:27.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:25:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:25:27.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:25:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:25:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:25:27 smithi138 ceph-mon[36954]: pgmap v9479: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:25:27.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:25:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:25:27.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:25:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:25:27.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:25:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:25:27.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:25:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:25:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:25:27 smithi100 ceph-mon[32599]: pgmap v9479: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:25:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:25:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:25:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:25:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:25:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:25:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:25:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:25:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:25:27.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:25:27.929+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:25:27.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:25:27.930+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:25:27.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:25:27.930+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:25:27.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:25:27.930+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:25:27.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:25:27.930+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:25:29.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:25:29 smithi167 ceph-mon[37140]: pgmap v9480: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:25:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:25:29 smithi138 ceph-mon[36954]: pgmap v9480: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:25:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:25:29 smithi100 ceph-mon[32599]: pgmap v9480: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:25:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:25:31 smithi138 ceph-mon[36954]: pgmap v9481: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:25:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:25:31 smithi100 ceph-mon[32599]: pgmap v9481: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:25:31.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:25:31 smithi167 ceph-mon[37140]: pgmap v9481: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:25:32.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:25:32 smithi138 ceph-mon[36954]: pgmap v9482: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:25:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:25:32 smithi100 ceph-mon[32599]: pgmap v9482: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:25:32.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:25:32 smithi167 ceph-mon[37140]: pgmap v9482: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:25:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:25:35 smithi138 ceph-mon[36954]: pgmap v9483: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:25:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:25:35 smithi100 ceph-mon[32599]: pgmap v9483: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:25:35.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:25:35 smithi167 ceph-mon[37140]: pgmap v9483: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:25:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:25:37 smithi100 ceph-mon[32599]: pgmap v9484: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:25:37.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:25:37 smithi167 ceph-mon[37140]: pgmap v9484: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:25:37.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:25:37 smithi138 ceph-mon[36954]: pgmap v9484: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:25:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:25:39 smithi100 ceph-mon[32599]: pgmap v9485: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:25:39.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:25:39 smithi167 ceph-mon[37140]: pgmap v9485: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:25:39.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:25:39 smithi138 ceph-mon[36954]: pgmap v9485: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:25:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:25:41 smithi100 ceph-mon[32599]: pgmap v9486: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:25:41.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:25:41 smithi167 ceph-mon[37140]: pgmap v9486: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:25:41.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:25:41 smithi138 ceph-mon[36954]: pgmap v9486: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:25:42.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:25:42.945+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:25:42.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:25:42.946+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:25:42.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:25:42.946+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:25:42.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:25:42.946+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:25:42.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:25:42.946+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:25:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:25:43 smithi100 ceph-mon[32599]: pgmap v9487: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:25:43.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:25:43 smithi167 ceph-mon[37140]: pgmap v9487: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:25:43.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:25:43 smithi138 ceph-mon[36954]: pgmap v9487: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:25:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:25:45 smithi100 ceph-mon[32599]: pgmap v9488: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:25:45.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:25:45 smithi167 ceph-mon[37140]: pgmap v9488: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:25:45.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:25:45 smithi138 ceph-mon[36954]: pgmap v9488: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:25:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:25:47 smithi100 ceph-mon[32599]: pgmap v9489: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:25:47.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:25:47 smithi167 ceph-mon[37140]: pgmap v9489: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:25:47.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:25:47 smithi138 ceph-mon[36954]: pgmap v9489: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:25:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:25:49 smithi100 ceph-mon[32599]: pgmap v9490: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:25:49.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:25:49 smithi167 ceph-mon[37140]: pgmap v9490: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:25:49.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:25:49 smithi138 ceph-mon[36954]: pgmap v9490: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:25:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:25:51 smithi100 ceph-mon[32599]: pgmap v9491: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:25:51.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:25:51 smithi167 ceph-mon[37140]: pgmap v9491: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:25:51.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:25:51 smithi138 ceph-mon[36954]: pgmap v9491: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:25:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:25:53 smithi100 ceph-mon[32599]: pgmap v9492: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:25:53.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:25:53 smithi167 ceph-mon[37140]: pgmap v9492: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:25:53.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:25:53 smithi138 ceph-mon[36954]: pgmap v9492: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:25:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:25:55 smithi100 ceph-mon[32599]: pgmap v9493: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:25:55.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:25:55 smithi167 ceph-mon[37140]: pgmap v9493: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:25:55.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:25:55 smithi138 ceph-mon[36954]: pgmap v9493: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:25:57.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:25:57 smithi100 ceph-mon[32599]: pgmap v9494: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:25:57.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:25:57 smithi167 ceph-mon[37140]: pgmap v9494: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:25:57.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:25:57 smithi138 ceph-mon[36954]: pgmap v9494: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:25:57.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:25:57.950+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:25:57.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:25:57.951+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:25:57.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:25:57.951+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:25:57.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:25:57.951+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:25:57.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:25:57.951+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:25:59.493 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:25:59 smithi138 ceph-mon[36954]: pgmap v9495: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:25:59.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:25:59 smithi100 ceph-mon[32599]: pgmap v9495: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:25:59.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:25:59 smithi167 ceph-mon[37140]: pgmap v9495: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:26:01.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:26:01 smithi100 ceph-mon[32599]: pgmap v9496: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:26:01.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:26:01 smithi167 ceph-mon[37140]: pgmap v9496: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:26:01.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:26:01 smithi138 ceph-mon[36954]: pgmap v9496: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:26:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:26:03 smithi100 ceph-mon[32599]: pgmap v9497: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:26:03.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:26:03 smithi167 ceph-mon[37140]: pgmap v9497: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:26:03.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:26:03 smithi138 ceph-mon[36954]: pgmap v9497: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:26:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:26:05 smithi100 ceph-mon[32599]: pgmap v9498: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:26:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:26:05 smithi167 ceph-mon[37140]: pgmap v9498: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:26:05.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:26:05 smithi138 ceph-mon[36954]: pgmap v9498: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:26:07.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:26:07 smithi167 ceph-mon[37140]: pgmap v9499: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:26:07.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:26:07 smithi138 ceph-mon[36954]: pgmap v9499: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:26:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:26:07 smithi100 ceph-mon[32599]: pgmap v9499: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:26:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:26:08 smithi167 ceph-mon[37140]: osdmap e640: 12 total, 12 up, 12 in 2024-07-26T18:26:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:26:08 smithi138 ceph-mon[36954]: osdmap e640: 12 total, 12 up, 12 in 2024-07-26T18:26:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:26:08 smithi100 ceph-mon[32599]: osdmap e640: 12 total, 12 up, 12 in 2024-07-26T18:26:09.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:26:09 smithi167 ceph-mon[37140]: pgmap v9501: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:26:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:26:09 smithi138 ceph-mon[36954]: pgmap v9501: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:26:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:26:09 smithi100 ceph-mon[32599]: pgmap v9501: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:26:11.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:26:11 smithi167 ceph-mon[37140]: pgmap v9502: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:26:11.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:26:11 smithi138 ceph-mon[36954]: pgmap v9502: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:26:11.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:26:11 smithi100 ceph-mon[32599]: pgmap v9502: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:26:12.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:26:12.967+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:26:12.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:26:12.968+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:26:12.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:26:12.968+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:26:12.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:26:12.968+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:26:12.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:26:12.968+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:26:13.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:26:13 smithi167 ceph-mon[37140]: pgmap v9503: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:26:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:26:13 smithi138 ceph-mon[36954]: pgmap v9503: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:26:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:26:13 smithi100 ceph-mon[32599]: pgmap v9503: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:26:15.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:26:15 smithi167 ceph-mon[37140]: pgmap v9504: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:26:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:26:15 smithi138 ceph-mon[36954]: pgmap v9504: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:26:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:26:15 smithi100 ceph-mon[32599]: pgmap v9504: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:26:17.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:26:17 smithi167 ceph-mon[37140]: pgmap v9505: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:26:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:26:17 smithi138 ceph-mon[36954]: pgmap v9505: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:26:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:26:17 smithi100 ceph-mon[32599]: pgmap v9505: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:26:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:26:19 smithi138 ceph-mon[36954]: pgmap v9506: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 187 B/s wr, 6 op/s 2024-07-26T18:26:19.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:26:19 smithi138 ceph-mon[36954]: osdmap e641: 12 total, 12 up, 12 in 2024-07-26T18:26:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:26:19 smithi100 ceph-mon[32599]: pgmap v9506: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 187 B/s wr, 6 op/s 2024-07-26T18:26:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:26:19 smithi100 ceph-mon[32599]: osdmap e641: 12 total, 12 up, 12 in 2024-07-26T18:26:19.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:26:19 smithi167 ceph-mon[37140]: pgmap v9506: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 187 B/s wr, 6 op/s 2024-07-26T18:26:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:26:19 smithi167 ceph-mon[37140]: osdmap e641: 12 total, 12 up, 12 in 2024-07-26T18:26:21.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:26:21 smithi138 ceph-mon[36954]: pgmap v9508: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:26:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:26:21 smithi100 ceph-mon[32599]: pgmap v9508: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:26:21.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:26:21 smithi167 ceph-mon[37140]: pgmap v9508: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:26:23.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:26:23 smithi138 ceph-mon[36954]: pgmap v9509: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:26:23.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:26:23 smithi100 ceph-mon[32599]: pgmap v9509: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:26:23.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:26:23 smithi167 ceph-mon[37140]: pgmap v9509: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:26:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:26:25 smithi138 ceph-mon[36954]: pgmap v9510: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:26:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:26:25 smithi100 ceph-mon[32599]: pgmap v9510: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:26:25.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:26:25 smithi167 ceph-mon[37140]: pgmap v9510: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:26:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:26:27 smithi138 ceph-mon[36954]: pgmap v9511: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:26:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:26:27 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:26:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:26:27 smithi100 ceph-mon[32599]: pgmap v9511: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:26:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:26:27 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:26:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:26:27 smithi167 ceph-mon[37140]: pgmap v9511: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:26:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:26:27 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:26:27.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:26:27.983+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:26:27.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:26:27.984+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:26:27.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:26:27.984+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:26:27.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:26:27.984+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:26:27.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:26:27.984+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:26:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:26:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:26:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:26:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:26:28.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:26:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:26:28.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:26:28 smithi138 ceph-mon[36954]: pgmap v9512: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:26:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:26:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:26:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:26:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:26:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:26:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:26:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:26:28 smithi100 ceph-mon[32599]: pgmap v9512: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:26:28.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:26:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:26:28.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:26:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:26:28.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:26:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:26:28.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:26:28 smithi167 ceph-mon[37140]: pgmap v9512: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:26:31.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:26:31 smithi167 ceph-mon[37140]: pgmap v9513: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:26:31.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:26:31 smithi100 ceph-mon[32599]: pgmap v9513: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:26:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:26:31 smithi138 ceph-mon[36954]: pgmap v9513: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:26:33.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:26:33 smithi100 ceph-mon[32599]: pgmap v9514: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:26:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:26:33 smithi167 ceph-mon[37140]: pgmap v9514: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:26:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:26:33 smithi138 ceph-mon[36954]: pgmap v9514: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:26:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:26:35 smithi100 ceph-mon[32599]: pgmap v9515: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:26:35.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:26:35 smithi167 ceph-mon[37140]: pgmap v9515: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:26:35.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:26:35 smithi138 ceph-mon[36954]: pgmap v9515: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:26:37.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:26:37 smithi100 ceph-mon[32599]: pgmap v9516: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:26:37.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:26:37 smithi167 ceph-mon[37140]: pgmap v9516: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:26:37.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:26:37 smithi138 ceph-mon[36954]: pgmap v9516: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:26:39.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:26:39 smithi100 ceph-mon[32599]: pgmap v9517: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:26:39.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:26:39 smithi167 ceph-mon[37140]: pgmap v9517: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:26:39.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:26:39 smithi138 ceph-mon[36954]: pgmap v9517: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:26:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:26:41 smithi100 ceph-mon[32599]: pgmap v9518: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:26:41.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:26:41 smithi167 ceph-mon[37140]: pgmap v9518: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:26:41.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:26:41 smithi138 ceph-mon[36954]: pgmap v9518: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:26:43.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:26:42.999+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:26:43.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:26:43.000+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:26:43.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:26:43.000+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:26:43.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:26:43.000+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:26:43.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:26:43.000+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:26:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:26:43 smithi100 ceph-mon[32599]: pgmap v9519: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:26:43.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:26:43 smithi167 ceph-mon[37140]: pgmap v9519: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:26:43.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:26:43 smithi138 ceph-mon[36954]: pgmap v9519: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:26:45.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:26:45 smithi100 ceph-mon[32599]: pgmap v9520: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:26:45.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:26:45 smithi167 ceph-mon[37140]: pgmap v9520: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:26:45.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:26:45 smithi138 ceph-mon[36954]: pgmap v9520: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:26:47.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:26:47 smithi100 ceph-mon[32599]: pgmap v9521: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:26:47.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:26:47 smithi167 ceph-mon[37140]: pgmap v9521: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:26:47.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:26:47 smithi138 ceph-mon[36954]: pgmap v9521: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:26:49.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:26:49 smithi100 ceph-mon[32599]: pgmap v9522: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:26:49.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:26:49 smithi167 ceph-mon[37140]: pgmap v9522: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:26:49.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:26:49 smithi138 ceph-mon[36954]: pgmap v9522: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:26:51.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:26:51 smithi100 ceph-mon[32599]: pgmap v9523: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:26:51.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:26:51 smithi167 ceph-mon[37140]: pgmap v9523: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:26:51.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:26:51 smithi138 ceph-mon[36954]: pgmap v9523: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:26:53.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:26:53 smithi100 ceph-mon[32599]: pgmap v9524: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:26:53.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:26:53 smithi167 ceph-mon[37140]: pgmap v9524: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:26:53.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:26:53 smithi138 ceph-mon[36954]: pgmap v9524: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:26:55.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:26:55 smithi167 ceph-mon[37140]: pgmap v9525: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:26:55.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:26:55 smithi138 ceph-mon[36954]: pgmap v9525: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:26:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:26:55 smithi100 ceph-mon[32599]: pgmap v9525: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:26:57.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:26:57 smithi167 ceph-mon[37140]: pgmap v9526: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:26:57.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:26:57 smithi138 ceph-mon[36954]: pgmap v9526: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:26:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:26:57 smithi100 ceph-mon[32599]: pgmap v9526: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:26:58.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:26:58.015+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:26:58.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:26:58.016+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:26:58.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:26:58.016+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:26:58.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:26:58.016+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:26:58.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:26:58.016+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:26:59.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:26:59 smithi167 ceph-mon[37140]: pgmap v9527: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:26:59.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:26:59 smithi138 ceph-mon[36954]: pgmap v9527: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:26:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:26:59 smithi100 ceph-mon[32599]: pgmap v9527: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:27:01.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:27:01 smithi167 ceph-mon[37140]: pgmap v9528: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:27:01.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:27:01 smithi138 ceph-mon[36954]: pgmap v9528: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:27:01.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:27:01 smithi100 ceph-mon[32599]: pgmap v9528: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:27:03.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:27:03 smithi167 ceph-mon[37140]: pgmap v9529: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:27:03.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:27:03 smithi138 ceph-mon[36954]: pgmap v9529: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:27:03.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:27:03 smithi100 ceph-mon[32599]: pgmap v9529: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:27:05.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:27:05.390+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c003690 con 0x7fe5a81087a0 2024-07-26T18:27:05.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:27:05.390+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T18:27:05.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:27:05.390+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 131 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c100 con 0x7fe5a81087a0 2024-07-26T18:27:05.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:27:05.390+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 131 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T18:27:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:27:05 smithi167 ceph-mon[37140]: pgmap v9530: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:27:05.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:27:05 smithi138 ceph-mon[36954]: pgmap v9530: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:27:05.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:27:05 smithi100 ceph-mon[32599]: pgmap v9530: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:27:07.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:27:07 smithi167 ceph-mon[37140]: pgmap v9531: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:27:07.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:27:07 smithi138 ceph-mon[36954]: pgmap v9531: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:27:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:27:07 smithi100 ceph-mon[32599]: pgmap v9531: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:27:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:27:08 smithi138 ceph-mon[36954]: osdmap e642: 12 total, 12 up, 12 in 2024-07-26T18:27:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:27:08 smithi100 ceph-mon[32599]: osdmap e642: 12 total, 12 up, 12 in 2024-07-26T18:27:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:27:08 smithi167 ceph-mon[37140]: osdmap e642: 12 total, 12 up, 12 in 2024-07-26T18:27:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:27:09 smithi138 ceph-mon[36954]: pgmap v9533: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:27:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:27:09 smithi100 ceph-mon[32599]: pgmap v9533: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:27:09.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:27:09 smithi167 ceph-mon[37140]: pgmap v9533: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:27:11.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:27:11 smithi138 ceph-mon[36954]: pgmap v9534: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:27:11.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:27:11 smithi100 ceph-mon[32599]: pgmap v9534: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:27:11.845 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:27:11 smithi167 ceph-mon[37140]: pgmap v9534: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:27:13.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:27:13.032+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:27:13.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:27:13.032+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:27:13.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:27:13.032+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:27:13.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:27:13.032+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:27:13.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:27:13.032+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:27:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:27:13 smithi138 ceph-mon[36954]: pgmap v9535: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:27:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:27:13 smithi100 ceph-mon[32599]: pgmap v9535: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:27:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:27:13 smithi167 ceph-mon[37140]: pgmap v9535: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:27:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:27:14 smithi138 ceph-mon[36954]: osdmap e643: 12 total, 12 up, 12 in 2024-07-26T18:27:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:27:14 smithi138 ceph-mon[36954]: pgmap v9537: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T18:27:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:27:14 smithi100 ceph-mon[32599]: osdmap e643: 12 total, 12 up, 12 in 2024-07-26T18:27:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:27:14 smithi100 ceph-mon[32599]: pgmap v9537: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T18:27:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:27:14 smithi167 ceph-mon[37140]: osdmap e643: 12 total, 12 up, 12 in 2024-07-26T18:27:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:27:14 smithi167 ceph-mon[37140]: pgmap v9537: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T18:27:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:27:17 smithi100 ceph-mon[32599]: pgmap v9538: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T18:27:17.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:27:17 smithi167 ceph-mon[37140]: pgmap v9538: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T18:27:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:27:17 smithi138 ceph-mon[36954]: pgmap v9538: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T18:27:19.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:27:19 smithi100 ceph-mon[32599]: pgmap v9539: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:27:19.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:27:19 smithi167 ceph-mon[37140]: pgmap v9539: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:27:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:27:19 smithi138 ceph-mon[36954]: pgmap v9539: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:27:21.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:27:21 smithi100 ceph-mon[32599]: pgmap v9540: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:27:21.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:27:21 smithi167 ceph-mon[37140]: pgmap v9540: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:27:21.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:27:21 smithi138 ceph-mon[36954]: pgmap v9540: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:27:23.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:27:23 smithi100 ceph-mon[32599]: pgmap v9541: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:27:23.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:27:23 smithi167 ceph-mon[37140]: pgmap v9541: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:27:23.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:27:23 smithi138 ceph-mon[36954]: pgmap v9541: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:27:25.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:27:25 smithi100 ceph-mon[32599]: pgmap v9542: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 94 B/s wr, 7 op/s 2024-07-26T18:27:25.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:27:25 smithi167 ceph-mon[37140]: pgmap v9542: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 94 B/s wr, 7 op/s 2024-07-26T18:27:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:27:25 smithi138 ceph-mon[36954]: pgmap v9542: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 94 B/s wr, 7 op/s 2024-07-26T18:27:27.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:27:27 smithi100 ceph-mon[32599]: pgmap v9543: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:27:27.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:27:27 smithi167 ceph-mon[37140]: pgmap v9543: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:27:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:27:27 smithi138 ceph-mon[36954]: pgmap v9543: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:27:28.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:27:28.047+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:27:28.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:27:28.047+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:27:28.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:27:28.047+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:27:28.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:27:28.047+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:27:28.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:27:28.047+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:27:28.395 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:27:28 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:27:28.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:27:28 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:27:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:27:28 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:27:29.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:27:29 smithi167 ceph-mon[37140]: pgmap v9544: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:27:29.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:27:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:27:29.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:27:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:27:29.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:27:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:27:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:27:29 smithi138 ceph-mon[36954]: pgmap v9544: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:27:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:27:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:27:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:27:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:27:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:27:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:27:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:27:29 smithi100 ceph-mon[32599]: pgmap v9544: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:27:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:27:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:27:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:27:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:27:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:27:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:27:31.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:27:31 smithi167 ceph-mon[37140]: pgmap v9545: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:27:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:27:31 smithi138 ceph-mon[36954]: pgmap v9545: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:27:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:27:31 smithi100 ceph-mon[32599]: pgmap v9545: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:27:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:27:33 smithi167 ceph-mon[37140]: pgmap v9546: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:27:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:27:33 smithi138 ceph-mon[36954]: pgmap v9546: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:27:33.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:27:33 smithi100 ceph-mon[32599]: pgmap v9546: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:27:35.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:27:35 smithi167 ceph-mon[37140]: pgmap v9547: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:27:35.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:27:35 smithi138 ceph-mon[36954]: pgmap v9547: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:27:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:27:35 smithi100 ceph-mon[32599]: pgmap v9547: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:27:37.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:27:37 smithi167 ceph-mon[37140]: pgmap v9548: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:27:37.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:27:37 smithi138 ceph-mon[36954]: pgmap v9548: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:27:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:27:37 smithi100 ceph-mon[32599]: pgmap v9548: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:27:39.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:27:39 smithi167 ceph-mon[37140]: pgmap v9549: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:27:39.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:27:39 smithi138 ceph-mon[36954]: pgmap v9549: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:27:39.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:27:39 smithi100 ceph-mon[32599]: pgmap v9549: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:27:41.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:27:41 smithi167 ceph-mon[37140]: pgmap v9550: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:27:41.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:27:41 smithi138 ceph-mon[36954]: pgmap v9550: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:27:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:27:41 smithi100 ceph-mon[32599]: pgmap v9550: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:27:43.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:27:43.063+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:27:43.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:27:43.063+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:27:43.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:27:43.063+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:27:43.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:27:43.063+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:27:46.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:27:43.063+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:27:46.372 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:27:43 smithi138 ceph-mon[36954]: pgmap v9551: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:27:46.373 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:27:45 smithi138 ceph-mon[36954]: pgmap v9552: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:27:46.373 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:27:43 smithi167 ceph-mon[37140]: pgmap v9551: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:27:46.373 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:27:45 smithi167 ceph-mon[37140]: pgmap v9552: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:27:46.373 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:27:43 smithi100 ceph-mon[32599]: pgmap v9551: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:27:46.373 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:27:45 smithi100 ceph-mon[32599]: pgmap v9552: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:27:47.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:27:47 smithi138 ceph-mon[36954]: pgmap v9553: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:27:47.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:27:47 smithi100 ceph-mon[32599]: pgmap v9553: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:27:47.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:27:47 smithi167 ceph-mon[37140]: pgmap v9553: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:27:49.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:27:49 smithi138 ceph-mon[36954]: pgmap v9554: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:27:49.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:27:49 smithi100 ceph-mon[32599]: pgmap v9554: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:27:49.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:27:49 smithi167 ceph-mon[37140]: pgmap v9554: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:27:51.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:27:51 smithi138 ceph-mon[36954]: pgmap v9555: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:27:51.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:27:51 smithi100 ceph-mon[32599]: pgmap v9555: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:27:51.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:27:51 smithi167 ceph-mon[37140]: pgmap v9555: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:27:53.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:27:53 smithi138 ceph-mon[36954]: pgmap v9556: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:27:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:27:53 smithi100 ceph-mon[32599]: pgmap v9556: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:27:53.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:27:53 smithi167 ceph-mon[37140]: pgmap v9556: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:27:55.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:27:55 smithi138 ceph-mon[36954]: pgmap v9557: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:27:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:27:55 smithi100 ceph-mon[32599]: pgmap v9557: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:27:55.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:27:55 smithi167 ceph-mon[37140]: pgmap v9557: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:27:57.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:27:57 smithi138 ceph-mon[36954]: pgmap v9558: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:27:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:27:57 smithi100 ceph-mon[32599]: pgmap v9558: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:27:57.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:27:57 smithi167 ceph-mon[37140]: pgmap v9558: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:27:58.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:27:58.078+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:27:58.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:27:58.079+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:27:58.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:27:58.079+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:27:58.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:27:58.079+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:27:58.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:27:58.079+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:27:59.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:27:59 smithi138 ceph-mon[36954]: pgmap v9559: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:27:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:27:59 smithi100 ceph-mon[32599]: pgmap v9559: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:27:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:27:59 smithi167 ceph-mon[37140]: pgmap v9559: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:28:01.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:28:01 smithi167 ceph-mon[37140]: pgmap v9560: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:28:01.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:28:01 smithi138 ceph-mon[36954]: pgmap v9560: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:28:01.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:28:01 smithi100 ceph-mon[32599]: pgmap v9560: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:28:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:28:02 smithi138 ceph-mon[36954]: pgmap v9561: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:28:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:28:02 smithi100 ceph-mon[32599]: pgmap v9561: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:28:02.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:28:02 smithi167 ceph-mon[37140]: pgmap v9561: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:28:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:28:05 smithi100 ceph-mon[32599]: pgmap v9562: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:28:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:28:05 smithi167 ceph-mon[37140]: pgmap v9562: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:28:05.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:28:05 smithi138 ceph-mon[36954]: pgmap v9562: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:28:07.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:28:07 smithi100 ceph-mon[32599]: pgmap v9563: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:28:07.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:28:07 smithi167 ceph-mon[37140]: pgmap v9563: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:28:07.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:28:07 smithi138 ceph-mon[36954]: pgmap v9563: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:28:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:28:08 smithi167 ceph-mon[37140]: osdmap e644: 12 total, 12 up, 12 in 2024-07-26T18:28:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:28:08 smithi138 ceph-mon[36954]: osdmap e644: 12 total, 12 up, 12 in 2024-07-26T18:28:08.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:28:08 smithi100 ceph-mon[32599]: osdmap e644: 12 total, 12 up, 12 in 2024-07-26T18:28:09.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:28:09 smithi167 ceph-mon[37140]: pgmap v9565: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:28:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:28:09 smithi138 ceph-mon[36954]: pgmap v9565: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:28:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:28:09 smithi100 ceph-mon[32599]: pgmap v9565: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:28:11.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:28:11 smithi167 ceph-mon[37140]: pgmap v9566: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:28:11.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:28:11 smithi138 ceph-mon[36954]: pgmap v9566: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:28:11.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:28:11 smithi100 ceph-mon[32599]: pgmap v9566: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:28:13.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:28:13.094+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:28:13.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:28:13.095+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:28:13.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:28:13.095+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:28:13.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:28:13.095+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:28:13.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:28:13.095+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:28:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:28:13 smithi100 ceph-mon[32599]: pgmap v9567: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:28:13.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:28:13 smithi167 ceph-mon[37140]: pgmap v9567: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:28:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:28:13 smithi138 ceph-mon[36954]: pgmap v9567: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:28:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:28:14 smithi138 ceph-mon[36954]: osdmap e645: 12 total, 12 up, 12 in 2024-07-26T18:28:14.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:28:14 smithi138 ceph-mon[36954]: pgmap v9569: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T18:28:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:28:14 smithi100 ceph-mon[32599]: osdmap e645: 12 total, 12 up, 12 in 2024-07-26T18:28:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:28:14 smithi100 ceph-mon[32599]: pgmap v9569: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T18:28:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:28:14 smithi167 ceph-mon[37140]: osdmap e645: 12 total, 12 up, 12 in 2024-07-26T18:28:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:28:14 smithi167 ceph-mon[37140]: pgmap v9569: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T18:28:17.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:28:17 smithi100 ceph-mon[32599]: pgmap v9570: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T18:28:17.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:28:17 smithi167 ceph-mon[37140]: pgmap v9570: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T18:28:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:28:17 smithi138 ceph-mon[36954]: pgmap v9570: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T18:28:19.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:28:19 smithi167 ceph-mon[37140]: pgmap v9571: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:28:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:28:19 smithi138 ceph-mon[36954]: pgmap v9571: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:28:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:28:19 smithi100 ceph-mon[32599]: pgmap v9571: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:28:21.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:28:21 smithi167 ceph-mon[37140]: pgmap v9572: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:28:21.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:28:21 smithi138 ceph-mon[36954]: pgmap v9572: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:28:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:28:21 smithi100 ceph-mon[32599]: pgmap v9572: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:28:23.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:28:23 smithi167 ceph-mon[37140]: pgmap v9573: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:28:23.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:28:23 smithi138 ceph-mon[36954]: pgmap v9573: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:28:23.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:28:23 smithi100 ceph-mon[32599]: pgmap v9573: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:28:25.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:28:25 smithi167 ceph-mon[37140]: pgmap v9574: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 94 B/s wr, 7 op/s 2024-07-26T18:28:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:28:25 smithi138 ceph-mon[36954]: pgmap v9574: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 94 B/s wr, 7 op/s 2024-07-26T18:28:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:28:25 smithi100 ceph-mon[32599]: pgmap v9574: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 94 B/s wr, 7 op/s 2024-07-26T18:28:27.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:28:27 smithi167 ceph-mon[37140]: pgmap v9575: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:28:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:28:27 smithi138 ceph-mon[36954]: pgmap v9575: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:28:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:28:27 smithi100 ceph-mon[32599]: pgmap v9575: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:28:28.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:28:28.111+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:28:28.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:28:28.111+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:28:28.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:28:28.111+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:28:28.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:28:28.111+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:28:28.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:28:28.111+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:28:29.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:28:29 smithi167 ceph-mon[37140]: pgmap v9576: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:28:29.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:28:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:28:29.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:28:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:28:29.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:28:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:28:29.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:28:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:28:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:28:29 smithi138 ceph-mon[36954]: pgmap v9576: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:28:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:28:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:28:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:28:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:28:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:28:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:28:29.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:28:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:28:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:28:29 smithi100 ceph-mon[32599]: pgmap v9576: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:28:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:28:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:28:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:28:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:28:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:28:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:28:29.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:28:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:28:31.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:28:31 smithi167 ceph-mon[37140]: pgmap v9577: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:28:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:28:31 smithi138 ceph-mon[36954]: pgmap v9577: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:28:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:28:31 smithi100 ceph-mon[32599]: pgmap v9577: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:28:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:28:33 smithi167 ceph-mon[37140]: pgmap v9578: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:28:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:28:33 smithi138 ceph-mon[36954]: pgmap v9578: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:28:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:28:33 smithi100 ceph-mon[32599]: pgmap v9578: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:28:34.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:28:34 smithi138 ceph-mon[36954]: pgmap v9579: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:28:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:28:34 smithi100 ceph-mon[32599]: pgmap v9579: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:28:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:28:34 smithi167 ceph-mon[37140]: pgmap v9579: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:28:35.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:28:35.403+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c004740 con 0x7fe5a81087a0 2024-07-26T18:28:35.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:28:35.403+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T18:28:35.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:28:35.403+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 132 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T18:28:35.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:28:35.403+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 132 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe5a008c0e0 con 0x7fe5a81087a0 2024-07-26T18:28:37.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:28:37 smithi167 ceph-mon[37140]: pgmap v9580: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:28:37.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:28:37 smithi138 ceph-mon[36954]: pgmap v9580: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:28:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:28:37 smithi100 ceph-mon[32599]: pgmap v9580: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:28:39.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:28:39 smithi167 ceph-mon[37140]: pgmap v9581: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:28:39.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:28:39 smithi138 ceph-mon[36954]: pgmap v9581: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:28:39.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:28:39 smithi100 ceph-mon[32599]: pgmap v9581: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:28:41.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:28:41 smithi167 ceph-mon[37140]: pgmap v9582: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:28:41.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:28:41 smithi138 ceph-mon[36954]: pgmap v9582: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:28:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:28:41 smithi100 ceph-mon[32599]: pgmap v9582: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:28:43.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:28:43.127+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:28:43.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:28:43.127+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:28:43.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:28:43.127+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:28:43.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:28:43.127+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:28:43.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:28:43.127+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:28:43.966 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:28:43 smithi138 ceph-mon[36954]: pgmap v9583: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:28:43.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:28:43 smithi167 ceph-mon[37140]: pgmap v9583: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:28:43.967 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:28:43 smithi100 ceph-mon[32599]: pgmap v9583: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:28:45.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:28:45 smithi167 ceph-mon[37140]: pgmap v9584: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:28:45.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:28:45 smithi138 ceph-mon[36954]: pgmap v9584: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:28:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:28:45 smithi100 ceph-mon[32599]: pgmap v9584: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:28:47.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:28:47 smithi167 ceph-mon[37140]: pgmap v9585: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:28:47.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:28:47 smithi138 ceph-mon[36954]: pgmap v9585: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:28:47.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:28:47 smithi100 ceph-mon[32599]: pgmap v9585: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:28:49.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:28:49 smithi167 ceph-mon[37140]: pgmap v9586: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T18:28:49.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:28:49 smithi138 ceph-mon[36954]: pgmap v9586: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T18:28:49.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:28:49 smithi100 ceph-mon[32599]: pgmap v9586: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T18:28:51.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:28:51 smithi138 ceph-mon[36954]: pgmap v9587: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:28:51.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:28:51 smithi100 ceph-mon[32599]: pgmap v9587: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:28:51.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:28:51 smithi167 ceph-mon[37140]: pgmap v9587: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:28:53.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:28:53 smithi138 ceph-mon[36954]: pgmap v9588: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T18:28:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:28:53 smithi100 ceph-mon[32599]: pgmap v9588: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T18:28:53.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:28:53 smithi167 ceph-mon[37140]: pgmap v9588: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T18:28:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:28:54 smithi138 ceph-mon[36954]: pgmap v9589: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:28:54.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:28:54 smithi100 ceph-mon[32599]: pgmap v9589: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:28:54.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:28:54 smithi167 ceph-mon[37140]: pgmap v9589: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:28:57.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:28:57 smithi167 ceph-mon[37140]: pgmap v9590: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:28:57.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:28:57 smithi138 ceph-mon[36954]: pgmap v9590: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:28:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:28:57 smithi100 ceph-mon[32599]: pgmap v9590: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:28:58.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:28:58.143+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:28:58.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:28:58.143+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:28:58.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:28:58.143+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:28:58.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:28:58.143+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:28:58.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:28:58.143+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:28:59.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:28:59 smithi167 ceph-mon[37140]: pgmap v9591: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:28:59.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:28:59 smithi138 ceph-mon[36954]: pgmap v9591: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:28:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:28:59 smithi100 ceph-mon[32599]: pgmap v9591: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:29:01.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:29:01 smithi167 ceph-mon[37140]: pgmap v9592: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:29:01.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:29:01 smithi138 ceph-mon[36954]: pgmap v9592: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:29:01.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:29:01 smithi100 ceph-mon[32599]: pgmap v9592: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:29:03.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:29:03 smithi167 ceph-mon[37140]: pgmap v9593: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:29:03.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:29:03 smithi138 ceph-mon[36954]: pgmap v9593: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:29:03.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:29:03 smithi100 ceph-mon[32599]: pgmap v9593: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:29:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:29:05 smithi167 ceph-mon[37140]: pgmap v9594: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:29:05.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:29:05 smithi138 ceph-mon[36954]: pgmap v9594: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:29:05.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:29:05 smithi100 ceph-mon[32599]: pgmap v9594: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:29:07.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:29:07 smithi167 ceph-mon[37140]: pgmap v9595: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:29:07.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:29:07 smithi138 ceph-mon[36954]: pgmap v9595: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:29:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:29:07 smithi100 ceph-mon[32599]: pgmap v9595: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:29:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:29:08 smithi138 ceph-mon[36954]: osdmap e646: 12 total, 12 up, 12 in 2024-07-26T18:29:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:29:08 smithi100 ceph-mon[32599]: osdmap e646: 12 total, 12 up, 12 in 2024-07-26T18:29:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:29:08 smithi167 ceph-mon[37140]: osdmap e646: 12 total, 12 up, 12 in 2024-07-26T18:29:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:29:09 smithi138 ceph-mon[36954]: pgmap v9597: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:29:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:29:09 smithi100 ceph-mon[32599]: pgmap v9597: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:29:09.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:29:09 smithi167 ceph-mon[37140]: pgmap v9597: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:29:11.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:29:11 smithi138 ceph-mon[36954]: pgmap v9598: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:29:11.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:29:11 smithi100 ceph-mon[32599]: pgmap v9598: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:29:11.796 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:29:11 smithi167 ceph-mon[37140]: pgmap v9598: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:29:13.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:29:13.159+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:29:13.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:29:13.160+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:29:13.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:29:13.160+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:29:13.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:29:13.160+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:29:13.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:29:13.160+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:29:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:29:13 smithi100 ceph-mon[32599]: pgmap v9599: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:29:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:29:13 smithi138 ceph-mon[36954]: pgmap v9599: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:29:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:29:13 smithi167 ceph-mon[37140]: pgmap v9599: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:29:15.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:29:15 smithi138 ceph-mon[36954]: pgmap v9600: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:29:15.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:29:15 smithi100 ceph-mon[32599]: pgmap v9600: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:29:15.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:29:15 smithi167 ceph-mon[37140]: pgmap v9600: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:29:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:29:17 smithi138 ceph-mon[36954]: pgmap v9601: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:29:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:29:17 smithi100 ceph-mon[32599]: pgmap v9601: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:29:17.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:29:17 smithi167 ceph-mon[37140]: pgmap v9601: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:29:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:29:19 smithi138 ceph-mon[36954]: pgmap v9602: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 187 B/s wr, 6 op/s 2024-07-26T18:29:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:29:19 smithi138 ceph-mon[36954]: osdmap e647: 12 total, 12 up, 12 in 2024-07-26T18:29:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:29:19 smithi100 ceph-mon[32599]: pgmap v9602: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 187 B/s wr, 6 op/s 2024-07-26T18:29:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:29:19 smithi100 ceph-mon[32599]: osdmap e647: 12 total, 12 up, 12 in 2024-07-26T18:29:19.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:29:19 smithi167 ceph-mon[37140]: pgmap v9602: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 187 B/s wr, 6 op/s 2024-07-26T18:29:19.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:29:19 smithi167 ceph-mon[37140]: osdmap e647: 12 total, 12 up, 12 in 2024-07-26T18:29:21.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:29:21 smithi138 ceph-mon[36954]: pgmap v9604: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:29:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:29:21 smithi100 ceph-mon[32599]: pgmap v9604: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:29:21.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:29:21 smithi167 ceph-mon[37140]: pgmap v9604: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:29:23.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:29:23 smithi138 ceph-mon[36954]: pgmap v9605: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:29:23.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:29:23 smithi100 ceph-mon[32599]: pgmap v9605: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:29:23.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:29:23 smithi167 ceph-mon[37140]: pgmap v9605: 97 pgs: 97 active+clean; 235 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:29:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:29:25 smithi138 ceph-mon[36954]: pgmap v9606: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:29:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:29:25 smithi100 ceph-mon[32599]: pgmap v9606: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:29:25.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:29:25 smithi167 ceph-mon[37140]: pgmap v9606: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:29:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:29:27 smithi138 ceph-mon[36954]: pgmap v9607: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:29:27.747 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:29:27 smithi100 ceph-mon[32599]: pgmap v9607: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:29:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:29:27 smithi167 ceph-mon[37140]: pgmap v9607: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:29:28.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:29:28.172+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:29:28.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:29:28.173+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:29:28.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:29:28.173+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:29:28.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:29:28.173+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:29:28.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:29:28.173+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:29:28.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:29:28 smithi100 ceph-mon[32599]: pgmap v9608: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:29:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:29:28 smithi138 ceph-mon[36954]: pgmap v9608: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:29:28.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:29:28 smithi167 ceph-mon[37140]: pgmap v9608: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:29:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:29:29 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:29:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:29:29 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:29:29.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:29:29 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:29:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:29:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:29:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:29:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:29:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:29:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:29:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:29:30 smithi138 ceph-mon[36954]: pgmap v9609: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:29:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:29:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:29:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:29:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:29:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:29:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:29:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:29:30 smithi100 ceph-mon[32599]: pgmap v9609: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:29:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:29:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:29:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:29:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:29:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:29:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:29:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:29:30 smithi167 ceph-mon[37140]: pgmap v9609: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:29:33.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:29:33 smithi167 ceph-mon[37140]: pgmap v9610: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:29:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:29:33 smithi138 ceph-mon[36954]: pgmap v9610: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:29:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:29:33 smithi100 ceph-mon[32599]: pgmap v9610: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:29:35.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:29:35 smithi138 ceph-mon[36954]: pgmap v9611: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:29:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:29:35 smithi100 ceph-mon[32599]: pgmap v9611: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:29:35.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:29:35 smithi167 ceph-mon[37140]: pgmap v9611: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:29:37.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:29:37 smithi138 ceph-mon[36954]: pgmap v9612: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:29:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:29:37 smithi100 ceph-mon[32599]: pgmap v9612: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:29:37.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:29:37 smithi167 ceph-mon[37140]: pgmap v9612: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:29:39.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:29:39 smithi138 ceph-mon[36954]: pgmap v9613: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:29:39.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:29:39 smithi100 ceph-mon[32599]: pgmap v9613: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:29:39.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:29:39 smithi167 ceph-mon[37140]: pgmap v9613: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:29:41.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:29:41 smithi138 ceph-mon[36954]: pgmap v9614: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:29:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:29:41 smithi100 ceph-mon[32599]: pgmap v9614: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:29:41.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:29:41 smithi167 ceph-mon[37140]: pgmap v9614: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:29:43.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:29:43.175+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:29:43.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:29:43.175+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:29:43.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:29:43.175+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:29:43.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:29:43.175+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:29:43.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:29:43.175+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:29:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:29:43 smithi100 ceph-mon[32599]: pgmap v9615: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:29:43.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:29:43 smithi138 ceph-mon[36954]: pgmap v9615: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:29:43.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:29:43 smithi167 ceph-mon[37140]: pgmap v9615: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:29:45.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:29:45 smithi138 ceph-mon[36954]: pgmap v9616: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:29:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:29:45 smithi100 ceph-mon[32599]: pgmap v9616: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:29:45.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:29:45 smithi167 ceph-mon[37140]: pgmap v9616: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:29:47.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:29:47 smithi138 ceph-mon[36954]: pgmap v9617: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:29:48.840 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:29:47 smithi167 ceph-mon[37140]: pgmap v9617: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:29:48.840 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:29:47 smithi100 ceph-mon[32599]: pgmap v9617: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:29:49.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:29:49 smithi138 ceph-mon[36954]: pgmap v9618: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:29:49.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:29:49 smithi100 ceph-mon[32599]: pgmap v9618: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:29:49.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:29:49 smithi167 ceph-mon[37140]: pgmap v9618: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:29:51.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:29:51 smithi138 ceph-mon[36954]: pgmap v9619: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:29:51.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:29:51 smithi100 ceph-mon[32599]: pgmap v9619: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:29:51.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:29:51 smithi167 ceph-mon[37140]: pgmap v9619: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:29:53.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:29:53 smithi138 ceph-mon[36954]: pgmap v9620: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:29:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:29:53 smithi100 ceph-mon[32599]: pgmap v9620: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:29:53.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:29:53 smithi167 ceph-mon[37140]: pgmap v9620: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:29:55.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:29:55 smithi138 ceph-mon[36954]: pgmap v9621: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:29:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:29:55 smithi100 ceph-mon[32599]: pgmap v9621: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:29:55.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:29:55 smithi167 ceph-mon[37140]: pgmap v9621: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:29:57.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:29:57 smithi138 ceph-mon[36954]: pgmap v9622: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:29:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:29:57 smithi100 ceph-mon[32599]: pgmap v9622: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:29:57.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:29:57 smithi167 ceph-mon[37140]: pgmap v9622: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:29:58.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:29:58.185+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:29:58.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:29:58.189+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:29:58.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:29:58.189+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:29:58.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:29:58.189+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:29:58.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:29:58.189+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:29:59.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:29:59 smithi138 ceph-mon[36954]: pgmap v9623: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:29:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:29:59 smithi100 ceph-mon[32599]: pgmap v9623: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:29:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:29:59 smithi167 ceph-mon[37140]: pgmap v9623: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:30:00.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:00 smithi138 ceph-mon[36954]: overall HEALTH_OK 2024-07-26T18:30:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:00 smithi100 ceph-mon[32599]: overall HEALTH_OK 2024-07-26T18:30:00.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:00 smithi167 ceph-mon[37140]: overall HEALTH_OK 2024-07-26T18:30:01.699 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:01 smithi138 ceph-mon[36954]: pgmap v9624: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:30:01.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:01 smithi100 ceph-mon[32599]: pgmap v9624: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:30:01.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:01 smithi167 ceph-mon[37140]: pgmap v9624: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:30:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:02 smithi138 ceph-mon[36954]: pgmap v9625: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:30:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:02 smithi100 ceph-mon[32599]: pgmap v9625: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:30:02.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:02 smithi167 ceph-mon[37140]: pgmap v9625: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:30:05.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:30:05.416+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T18:30:05.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:30:05.416+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c003690 con 0x7fe5a81087a0 2024-07-26T18:30:05.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:30:05.416+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 133 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T18:30:05.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:30:05.416+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 133 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c070 con 0x7fe5a81087a0 2024-07-26T18:30:05.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:05 smithi138 ceph-mon[36954]: pgmap v9626: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:30:05.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:05 smithi100 ceph-mon[32599]: pgmap v9626: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:30:05.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:05 smithi167 ceph-mon[37140]: pgmap v9626: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:30:07.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:07 smithi138 ceph-mon[36954]: pgmap v9627: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:30:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:07 smithi100 ceph-mon[32599]: pgmap v9627: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:30:07.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:07 smithi167 ceph-mon[37140]: pgmap v9627: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:30:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:08 smithi138 ceph-mon[36954]: osdmap e648: 12 total, 12 up, 12 in 2024-07-26T18:30:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:08 smithi100 ceph-mon[32599]: osdmap e648: 12 total, 12 up, 12 in 2024-07-26T18:30:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:08 smithi167 ceph-mon[37140]: osdmap e648: 12 total, 12 up, 12 in 2024-07-26T18:30:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:09 smithi138 ceph-mon[36954]: pgmap v9629: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:30:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:09 smithi100 ceph-mon[32599]: pgmap v9629: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:30:09.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:09 smithi167 ceph-mon[37140]: pgmap v9629: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:30:11.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:11 smithi138 ceph-mon[36954]: pgmap v9630: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:30:11.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:11 smithi100 ceph-mon[32599]: pgmap v9630: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:30:11.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:11 smithi167 ceph-mon[37140]: pgmap v9630: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:30:13.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:30:13.204+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:30:13.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:30:13.205+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:30:13.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:30:13.205+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:30:13.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:30:13.205+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:30:13.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:30:13.205+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:30:13.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:13 smithi100 ceph-mon[32599]: pgmap v9631: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:30:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:13 smithi138 ceph-mon[36954]: pgmap v9631: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:30:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:13 smithi167 ceph-mon[37140]: pgmap v9631: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:30:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:14 smithi138 ceph-mon[36954]: osdmap e649: 12 total, 12 up, 12 in 2024-07-26T18:30:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:14 smithi138 ceph-mon[36954]: pgmap v9633: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T18:30:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:14 smithi100 ceph-mon[32599]: osdmap e649: 12 total, 12 up, 12 in 2024-07-26T18:30:14.767 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:14 smithi100 ceph-mon[32599]: pgmap v9633: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T18:30:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:14 smithi167 ceph-mon[37140]: osdmap e649: 12 total, 12 up, 12 in 2024-07-26T18:30:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:14 smithi167 ceph-mon[37140]: pgmap v9633: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T18:30:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:17 smithi138 ceph-mon[36954]: pgmap v9634: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T18:30:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:17 smithi100 ceph-mon[32599]: pgmap v9634: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T18:30:17.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:17 smithi167 ceph-mon[37140]: pgmap v9634: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T18:30:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:19 smithi138 ceph-mon[36954]: pgmap v9635: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:30:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:19 smithi100 ceph-mon[32599]: pgmap v9635: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:30:19.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:19 smithi167 ceph-mon[37140]: pgmap v9635: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:30:21.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:21 smithi138 ceph-mon[36954]: pgmap v9636: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:30:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:21 smithi100 ceph-mon[32599]: pgmap v9636: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:30:21.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:21 smithi167 ceph-mon[37140]: pgmap v9636: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:30:23.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:23 smithi138 ceph-mon[36954]: pgmap v9637: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:30:23.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:23 smithi100 ceph-mon[32599]: pgmap v9637: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:30:23.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:23 smithi167 ceph-mon[37140]: pgmap v9637: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:30:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:24 smithi138 ceph-mon[36954]: pgmap v9638: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 94 B/s wr, 6 op/s 2024-07-26T18:30:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:24 smithi100 ceph-mon[32599]: pgmap v9638: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 94 B/s wr, 6 op/s 2024-07-26T18:30:24.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:24 smithi167 ceph-mon[37140]: pgmap v9638: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 94 B/s wr, 6 op/s 2024-07-26T18:30:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:27 smithi138 ceph-mon[36954]: pgmap v9639: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:30:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:27 smithi100 ceph-mon[32599]: pgmap v9639: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:30:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:27 smithi167 ceph-mon[37140]: pgmap v9639: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:30:28.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:30:28.218+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:30:28.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:30:28.218+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:30:28.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:30:28.218+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:30:28.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:30:28.218+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:30:28.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:30:28.218+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:30:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:29 smithi138 ceph-mon[36954]: pgmap v9640: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:30:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:29 smithi100 ceph-mon[32599]: pgmap v9640: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:30:29.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:29 smithi167 ceph-mon[37140]: pgmap v9640: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:30:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:30:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:30:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:30:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:30:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:30:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:30:30.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:30:30.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:30:30.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:30:30.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:30:30.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:30:30.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:30:30.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:30:30.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:30:30.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:30:30.693 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:30:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:30:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:30:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:30:30.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:30:30.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:30:30.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:30:30.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:30:30.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:30:30.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:30:30.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:30:30.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:30:30.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:30:30.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:30:30.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:30:30.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:30:30.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:30:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:30:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:30:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:30:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:30:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:30:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:30:30.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:30:30.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:30:30.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:30:30.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:30:30.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:30:30.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:30:30.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:30:30.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:30:30.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:30:30.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:30:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:31 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T18:30:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:31 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T18:30:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:31 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T18:30:31.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:31 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T18:30:31.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:31 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T18:30:31.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:31 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T18:30:31.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:31 smithi138 ceph-mon[36954]: pgmap v9641: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:30:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:31 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T18:30:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:31 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T18:30:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:31 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T18:30:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:31 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T18:30:31.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:31 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T18:30:31.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:31 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T18:30:31.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:31 smithi100 ceph-mon[32599]: pgmap v9641: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:30:31.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:31 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T18:30:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:31 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T18:30:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:31 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T18:30:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:31 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T18:30:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:31 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T18:30:31.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:31 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T18:30:31.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:31 smithi167 ceph-mon[37140]: pgmap v9641: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:30:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:33 smithi138 ceph-mon[36954]: pgmap v9642: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:30:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:33 smithi100 ceph-mon[32599]: pgmap v9642: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:30:33.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:33 smithi167 ceph-mon[37140]: pgmap v9642: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:30:35.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:35 smithi167 ceph-mon[37140]: pgmap v9643: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:30:35.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:35 smithi138 ceph-mon[36954]: pgmap v9643: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:30:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:35 smithi100 ceph-mon[32599]: pgmap v9643: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:30:37.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:37 smithi138 ceph-mon[36954]: pgmap v9644: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:30:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:37 smithi100 ceph-mon[32599]: pgmap v9644: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:30:37.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:37 smithi167 ceph-mon[37140]: pgmap v9644: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:30:39.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:39 smithi138 ceph-mon[36954]: pgmap v9645: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:30:39.747 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:39 smithi100 ceph-mon[32599]: pgmap v9645: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:30:39.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:39 smithi167 ceph-mon[37140]: pgmap v9645: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:30:41.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:41 smithi138 ceph-mon[36954]: pgmap v9646: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:30:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:41 smithi100 ceph-mon[32599]: pgmap v9646: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:30:41.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:41 smithi167 ceph-mon[37140]: pgmap v9646: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:30:43.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:30:43.234+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:30:43.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:30:43.235+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:30:43.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:30:43.235+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:30:43.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:30:43.235+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:30:43.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:30:43.235+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:30:43.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:43 smithi100 ceph-mon[32599]: pgmap v9647: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:30:43.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:43 smithi138 ceph-mon[36954]: pgmap v9647: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:30:43.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:43 smithi167 ceph-mon[37140]: pgmap v9647: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:30:45.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:45 smithi138 ceph-mon[36954]: pgmap v9648: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:30:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:45 smithi100 ceph-mon[32599]: pgmap v9648: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:30:45.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:45 smithi167 ceph-mon[37140]: pgmap v9648: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:30:47.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:47 smithi138 ceph-mon[36954]: pgmap v9649: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:30:47.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:47 smithi100 ceph-mon[32599]: pgmap v9649: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:30:47.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:47 smithi167 ceph-mon[37140]: pgmap v9649: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:30:49.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:49 smithi138 ceph-mon[36954]: pgmap v9650: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:30:49.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:49 smithi100 ceph-mon[32599]: pgmap v9650: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:30:49.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:49 smithi167 ceph-mon[37140]: pgmap v9650: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:30:51.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:51 smithi138 ceph-mon[36954]: pgmap v9651: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:30:51.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:51 smithi100 ceph-mon[32599]: pgmap v9651: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:30:51.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:51 smithi167 ceph-mon[37140]: pgmap v9651: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:30:52.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:52 smithi138 ceph-mon[36954]: pgmap v9652: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:30:52.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:52 smithi100 ceph-mon[32599]: pgmap v9652: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:30:52.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:52 smithi167 ceph-mon[37140]: pgmap v9652: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:30:55.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:55 smithi138 ceph-mon[36954]: pgmap v9653: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:30:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:55 smithi100 ceph-mon[32599]: pgmap v9653: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:30:55.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:55 smithi167 ceph-mon[37140]: pgmap v9653: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:30:57.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:57 smithi138 ceph-mon[36954]: pgmap v9654: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:30:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:57 smithi100 ceph-mon[32599]: pgmap v9654: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:30:57.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:57 smithi167 ceph-mon[37140]: pgmap v9654: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:30:58.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:30:58.251+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:30:58.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:30:58.251+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:30:58.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:30:58.251+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:30:58.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:30:58.251+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:30:58.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:30:58.251+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:30:59.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:30:59 smithi138 ceph-mon[36954]: pgmap v9655: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:30:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:30:59 smithi100 ceph-mon[32599]: pgmap v9655: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:30:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:30:59 smithi167 ceph-mon[37140]: pgmap v9655: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:31:01.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:31:01 smithi138 ceph-mon[36954]: pgmap v9656: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:31:01.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:31:01 smithi100 ceph-mon[32599]: pgmap v9656: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:31:01.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:31:01 smithi167 ceph-mon[37140]: pgmap v9656: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:31:03.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:31:03 smithi138 ceph-mon[36954]: pgmap v9657: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:31:03.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:31:03 smithi100 ceph-mon[32599]: pgmap v9657: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:31:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:31:03 smithi167 ceph-mon[37140]: pgmap v9657: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:31:05.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:31:05 smithi167 ceph-mon[37140]: pgmap v9658: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:31:05.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:31:05 smithi138 ceph-mon[36954]: pgmap v9658: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:31:05.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:31:05 smithi100 ceph-mon[32599]: pgmap v9658: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:31:07.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:31:07 smithi138 ceph-mon[36954]: pgmap v9659: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:31:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:31:07 smithi100 ceph-mon[32599]: pgmap v9659: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:31:07.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:31:07 smithi167 ceph-mon[37140]: pgmap v9659: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:31:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:31:08 smithi138 ceph-mon[36954]: osdmap e650: 12 total, 12 up, 12 in 2024-07-26T18:31:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:31:08 smithi100 ceph-mon[32599]: osdmap e650: 12 total, 12 up, 12 in 2024-07-26T18:31:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:31:08 smithi167 ceph-mon[37140]: osdmap e650: 12 total, 12 up, 12 in 2024-07-26T18:31:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:31:09 smithi138 ceph-mon[36954]: pgmap v9661: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:31:09.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:31:09 smithi100 ceph-mon[32599]: pgmap v9661: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:31:09.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:31:09 smithi167 ceph-mon[37140]: pgmap v9661: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:31:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:31:10 smithi138 ceph-mon[36954]: pgmap v9662: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:31:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:31:10 smithi100 ceph-mon[32599]: pgmap v9662: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:31:10.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:31:10 smithi167 ceph-mon[37140]: pgmap v9662: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:31:13.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:31:13.256+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:31:13.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:31:13.257+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:31:13.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:31:13.257+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:31:13.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:31:13.257+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:31:13.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:31:13.257+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:31:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:31:13 smithi138 ceph-mon[36954]: pgmap v9663: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:31:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:31:13 smithi100 ceph-mon[32599]: pgmap v9663: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:31:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:31:13 smithi167 ceph-mon[37140]: pgmap v9663: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:31:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:31:14 smithi138 ceph-mon[36954]: osdmap e651: 12 total, 12 up, 12 in 2024-07-26T18:31:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:31:14 smithi138 ceph-mon[36954]: pgmap v9665: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T18:31:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:31:14 smithi100 ceph-mon[32599]: osdmap e651: 12 total, 12 up, 12 in 2024-07-26T18:31:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:31:14 smithi100 ceph-mon[32599]: pgmap v9665: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T18:31:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:31:14 smithi167 ceph-mon[37140]: osdmap e651: 12 total, 12 up, 12 in 2024-07-26T18:31:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:31:14 smithi167 ceph-mon[37140]: pgmap v9665: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T18:31:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:31:17 smithi138 ceph-mon[36954]: pgmap v9666: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T18:31:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:31:17 smithi100 ceph-mon[32599]: pgmap v9666: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T18:31:17.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:31:17 smithi167 ceph-mon[37140]: pgmap v9666: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T18:31:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:31:19 smithi138 ceph-mon[36954]: pgmap v9667: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:31:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:31:19 smithi100 ceph-mon[32599]: pgmap v9667: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:31:19.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:31:19 smithi167 ceph-mon[37140]: pgmap v9667: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:31:21.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:31:21 smithi138 ceph-mon[36954]: pgmap v9668: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:31:21.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:31:21 smithi100 ceph-mon[32599]: pgmap v9668: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:31:21.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:31:21 smithi167 ceph-mon[37140]: pgmap v9668: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:31:23.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:31:23 smithi138 ceph-mon[36954]: pgmap v9669: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:31:23.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:31:23 smithi100 ceph-mon[32599]: pgmap v9669: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:31:23.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:31:23 smithi167 ceph-mon[37140]: pgmap v9669: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:31:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:31:25 smithi138 ceph-mon[36954]: pgmap v9670: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 94 B/s wr, 7 op/s 2024-07-26T18:31:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:31:25 smithi100 ceph-mon[32599]: pgmap v9670: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 94 B/s wr, 7 op/s 2024-07-26T18:31:25.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:31:25 smithi167 ceph-mon[37140]: pgmap v9670: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 94 B/s wr, 7 op/s 2024-07-26T18:31:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:31:27 smithi138 ceph-mon[36954]: pgmap v9671: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:31:27.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:31:27 smithi100 ceph-mon[32599]: pgmap v9671: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:31:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:31:27 smithi167 ceph-mon[37140]: pgmap v9671: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:31:28.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:31:28.273+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:31:28.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:31:28.273+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:31:28.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:31:28.273+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:31:28.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:31:28.273+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:31:28.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:31:28.273+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:31:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:31:29 smithi138 ceph-mon[36954]: pgmap v9672: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:31:29.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:31:29 smithi100 ceph-mon[32599]: pgmap v9672: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:31:29.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:31:29 smithi167 ceph-mon[37140]: pgmap v9672: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:31:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:31:30 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:31:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:31:30 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:31:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:31:30 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:31:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:31:31 smithi138 ceph-mon[36954]: pgmap v9673: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:31:31.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:31:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:31:31.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:31:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:31:31.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:31:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:31:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:31:31 smithi100 ceph-mon[32599]: pgmap v9673: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:31:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:31:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:31:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:31:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:31:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:31:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:31:31.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:31:31 smithi167 ceph-mon[37140]: pgmap v9673: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:31:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:31:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:31:31.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:31:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:31:31.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:31:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:31:33.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:31:33 smithi138 ceph-mon[36954]: pgmap v9674: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:31:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:31:33 smithi100 ceph-mon[32599]: pgmap v9674: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:31:33.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:31:33 smithi167 ceph-mon[37140]: pgmap v9674: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:31:34.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:31:34 smithi138 ceph-mon[36954]: pgmap v9675: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:31:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:31:34 smithi100 ceph-mon[32599]: pgmap v9675: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:31:34.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:31:34 smithi167 ceph-mon[37140]: pgmap v9675: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:31:37.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:31:37 smithi138 ceph-mon[36954]: pgmap v9676: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:31:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:31:37 smithi100 ceph-mon[32599]: pgmap v9676: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:31:37.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:31:37 smithi167 ceph-mon[37140]: pgmap v9676: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:31:39.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:31:39 smithi138 ceph-mon[36954]: pgmap v9677: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:31:39.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:31:39 smithi100 ceph-mon[32599]: pgmap v9677: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:31:39.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:31:39 smithi167 ceph-mon[37140]: pgmap v9677: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:31:41.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:31:41 smithi138 ceph-mon[36954]: pgmap v9678: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:31:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:31:41 smithi100 ceph-mon[32599]: pgmap v9678: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:31:41.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:31:41 smithi167 ceph-mon[37140]: pgmap v9678: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:31:43.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:31:43.289+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:31:43.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:31:43.289+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:31:43.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:31:43.289+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:31:43.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:31:43.289+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:31:43.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:31:43.289+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:31:43.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:31:43 smithi138 ceph-mon[36954]: pgmap v9679: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:31:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:31:43 smithi100 ceph-mon[32599]: pgmap v9679: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:31:43.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:31:43 smithi167 ceph-mon[37140]: pgmap v9679: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:31:45.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:31:45 smithi138 ceph-mon[36954]: pgmap v9680: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:31:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:31:45 smithi100 ceph-mon[32599]: pgmap v9680: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:31:45.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:31:45 smithi167 ceph-mon[37140]: pgmap v9680: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:31:47.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:31:47 smithi138 ceph-mon[36954]: pgmap v9681: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:31:47.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:31:47 smithi100 ceph-mon[32599]: pgmap v9681: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:31:47.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:31:47 smithi167 ceph-mon[37140]: pgmap v9681: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:31:49.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:31:49 smithi138 ceph-mon[36954]: pgmap v9682: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:31:49.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:31:49 smithi100 ceph-mon[32599]: pgmap v9682: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:31:49.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:31:49 smithi167 ceph-mon[37140]: pgmap v9682: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:31:51.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:31:51 smithi138 ceph-mon[36954]: pgmap v9683: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:31:51.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:31:51 smithi100 ceph-mon[32599]: pgmap v9683: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:31:51.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:31:51 smithi167 ceph-mon[37140]: pgmap v9683: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:31:53.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:31:53 smithi138 ceph-mon[36954]: pgmap v9684: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:31:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:31:53 smithi100 ceph-mon[32599]: pgmap v9684: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:31:53.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:31:53 smithi167 ceph-mon[37140]: pgmap v9684: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:31:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:31:54 smithi138 ceph-mon[36954]: pgmap v9685: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:31:54.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:31:54 smithi100 ceph-mon[32599]: pgmap v9685: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:31:54.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:31:54 smithi167 ceph-mon[37140]: pgmap v9685: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:31:57.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:31:57 smithi138 ceph-mon[36954]: pgmap v9686: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:31:57.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:31:57 smithi100 ceph-mon[32599]: pgmap v9686: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:31:57.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:31:57 smithi167 ceph-mon[37140]: pgmap v9686: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:31:58.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:31:58.305+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:31:58.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:31:58.305+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:31:58.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:31:58.305+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:31:58.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:31:58.305+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:31:58.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:31:58.305+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:31:59.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:31:59 smithi138 ceph-mon[36954]: pgmap v9687: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:31:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:31:59 smithi100 ceph-mon[32599]: pgmap v9687: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:31:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:31:59 smithi167 ceph-mon[37140]: pgmap v9687: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:32:01.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:32:01 smithi138 ceph-mon[36954]: pgmap v9688: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:32:01.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:32:01 smithi100 ceph-mon[32599]: pgmap v9688: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:32:01.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:32:01 smithi167 ceph-mon[37140]: pgmap v9688: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:32:03.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:32:03 smithi138 ceph-mon[36954]: pgmap v9689: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:32:03.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:32:03 smithi100 ceph-mon[32599]: pgmap v9689: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:32:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:32:03 smithi167 ceph-mon[37140]: pgmap v9689: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:32:05.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:32:05 smithi138 ceph-mon[36954]: pgmap v9690: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:32:05.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:32:05 smithi100 ceph-mon[32599]: pgmap v9690: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:32:05.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:32:05 smithi167 ceph-mon[37140]: pgmap v9690: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:32:07.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:32:07 smithi138 ceph-mon[36954]: pgmap v9691: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:32:07.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:32:07 smithi100 ceph-mon[32599]: pgmap v9691: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:32:07.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:32:07 smithi167 ceph-mon[37140]: pgmap v9691: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:32:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:32:08 smithi138 ceph-mon[36954]: osdmap e652: 12 total, 12 up, 12 in 2024-07-26T18:32:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:32:08 smithi138 ceph-mon[36954]: pgmap v9693: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:32:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:32:08 smithi100 ceph-mon[32599]: osdmap e652: 12 total, 12 up, 12 in 2024-07-26T18:32:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:32:08 smithi100 ceph-mon[32599]: pgmap v9693: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:32:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:32:08 smithi167 ceph-mon[37140]: osdmap e652: 12 total, 12 up, 12 in 2024-07-26T18:32:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:32:08 smithi167 ceph-mon[37140]: pgmap v9693: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:32:11.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:32:11 smithi138 ceph-mon[36954]: pgmap v9694: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:32:11.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:32:11 smithi100 ceph-mon[32599]: pgmap v9694: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:32:11.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:32:11 smithi167 ceph-mon[37140]: pgmap v9694: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:32:13.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:32:13.321+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:32:13.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:32:13.321+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:32:13.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:32:13.321+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:32:13.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:32:13.321+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:32:13.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:32:13.321+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:32:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:32:13 smithi138 ceph-mon[36954]: pgmap v9695: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:32:13.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:32:13 smithi100 ceph-mon[32599]: pgmap v9695: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:32:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:32:13 smithi167 ceph-mon[37140]: pgmap v9695: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:32:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:32:14 smithi138 ceph-mon[36954]: osdmap e653: 12 total, 12 up, 12 in 2024-07-26T18:32:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:32:14 smithi138 ceph-mon[36954]: pgmap v9697: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T18:32:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:32:14 smithi100 ceph-mon[32599]: osdmap e653: 12 total, 12 up, 12 in 2024-07-26T18:32:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:32:14 smithi100 ceph-mon[32599]: pgmap v9697: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T18:32:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:32:14 smithi167 ceph-mon[37140]: osdmap e653: 12 total, 12 up, 12 in 2024-07-26T18:32:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:32:14 smithi167 ceph-mon[37140]: pgmap v9697: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T18:32:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:32:17 smithi138 ceph-mon[36954]: pgmap v9698: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T18:32:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:32:17 smithi100 ceph-mon[32599]: pgmap v9698: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T18:32:17.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:32:17 smithi167 ceph-mon[37140]: pgmap v9698: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T18:32:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:32:19 smithi138 ceph-mon[36954]: pgmap v9699: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:32:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:32:19 smithi100 ceph-mon[32599]: pgmap v9699: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:32:19.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:32:19 smithi167 ceph-mon[37140]: pgmap v9699: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:32:21.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:32:21 smithi138 ceph-mon[36954]: pgmap v9700: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:32:21.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:32:21 smithi100 ceph-mon[32599]: pgmap v9700: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:32:21.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:32:21 smithi167 ceph-mon[37140]: pgmap v9700: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:32:23.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:32:23 smithi138 ceph-mon[36954]: pgmap v9701: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:32:23.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:32:23 smithi100 ceph-mon[32599]: pgmap v9701: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:32:23.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:32:23 smithi167 ceph-mon[37140]: pgmap v9701: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:32:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:32:25 smithi138 ceph-mon[36954]: pgmap v9702: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 94 B/s wr, 7 op/s 2024-07-26T18:32:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:32:25 smithi100 ceph-mon[32599]: pgmap v9702: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 94 B/s wr, 7 op/s 2024-07-26T18:32:25.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:32:25 smithi167 ceph-mon[37140]: pgmap v9702: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 94 B/s wr, 7 op/s 2024-07-26T18:32:27.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:32:27 smithi138 ceph-mon[36954]: pgmap v9703: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:32:27.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:32:27 smithi100 ceph-mon[32599]: pgmap v9703: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:32:27.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:32:27 smithi167 ceph-mon[37140]: pgmap v9703: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:32:28.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:32:28.335+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:32:28.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:32:28.335+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:32:28.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:32:28.335+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:32:28.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:32:28.336+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:32:28.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:32:28.336+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:32:28.645 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:32:28 smithi100 ceph-mon[32599]: pgmap v9704: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:32:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:32:28 smithi138 ceph-mon[36954]: pgmap v9704: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:32:28.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:32:28 smithi167 ceph-mon[37140]: pgmap v9704: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:32:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:32:31 smithi138 ceph-mon[36954]: pgmap v9705: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:32:31.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:32:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:32:31.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:32:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:32:31.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:32:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:32:31.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:32:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:32:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:32:31 smithi100 ceph-mon[32599]: pgmap v9705: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:32:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:32:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:32:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:32:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:32:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:32:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:32:31.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:32:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:32:31.776 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:32:31 smithi167 ceph-mon[37140]: pgmap v9705: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:32:31.776 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:32:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:32:31.776 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:32:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:32:31.777 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:32:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:32:31.777 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:32:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:32:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:32:33 smithi138 ceph-mon[36954]: pgmap v9706: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:32:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:32:33 smithi100 ceph-mon[32599]: pgmap v9706: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:32:33.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:32:33 smithi167 ceph-mon[37140]: pgmap v9706: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:32:34.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:32:34 smithi138 ceph-mon[36954]: pgmap v9707: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:32:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:32:34 smithi100 ceph-mon[32599]: pgmap v9707: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:32:34.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:32:34 smithi167 ceph-mon[37140]: pgmap v9707: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:32:37.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:32:37 smithi138 ceph-mon[36954]: pgmap v9708: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:32:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:32:37 smithi100 ceph-mon[32599]: pgmap v9708: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:32:37.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:32:37 smithi167 ceph-mon[37140]: pgmap v9708: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:32:39.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:32:39 smithi138 ceph-mon[36954]: pgmap v9709: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:32:39.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:32:39 smithi100 ceph-mon[32599]: pgmap v9709: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:32:39.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:32:39 smithi167 ceph-mon[37140]: pgmap v9709: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:32:41.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:32:41 smithi138 ceph-mon[36954]: pgmap v9710: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:32:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:32:41 smithi100 ceph-mon[32599]: pgmap v9710: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:32:41.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:32:41 smithi167 ceph-mon[37140]: pgmap v9710: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:32:43.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:32:43.351+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:32:43.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:32:43.352+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:32:43.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:32:43.352+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:32:43.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:32:43.352+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:32:43.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:32:43.352+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:32:43.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:32:43 smithi138 ceph-mon[36954]: pgmap v9711: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:32:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:32:43 smithi100 ceph-mon[32599]: pgmap v9711: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:32:43.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:32:43 smithi167 ceph-mon[37140]: pgmap v9711: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:32:45.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:32:45 smithi138 ceph-mon[36954]: pgmap v9712: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:32:45.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:32:45 smithi100 ceph-mon[32599]: pgmap v9712: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:32:45.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:32:45 smithi167 ceph-mon[37140]: pgmap v9712: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:32:46.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:32:46 smithi138 ceph-mon[36954]: pgmap v9713: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:32:46.747 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:32:46 smithi100 ceph-mon[32599]: pgmap v9713: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:32:46.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:32:46 smithi167 ceph-mon[37140]: pgmap v9713: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:32:49.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:32:49 smithi138 ceph-mon[36954]: pgmap v9714: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:32:49.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:32:49 smithi100 ceph-mon[32599]: pgmap v9714: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:32:49.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:32:49 smithi167 ceph-mon[37140]: pgmap v9714: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:32:51.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:32:51 smithi138 ceph-mon[36954]: pgmap v9715: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:32:51.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:32:51 smithi100 ceph-mon[32599]: pgmap v9715: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:32:51.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:32:51 smithi167 ceph-mon[37140]: pgmap v9715: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:32:53.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:32:53 smithi138 ceph-mon[36954]: pgmap v9716: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:32:53.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:32:53 smithi100 ceph-mon[32599]: pgmap v9716: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:32:53.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:32:53 smithi167 ceph-mon[37140]: pgmap v9716: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:32:55.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:32:55 smithi138 ceph-mon[36954]: pgmap v9717: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:32:55.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:32:55 smithi100 ceph-mon[32599]: pgmap v9717: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:32:55.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:32:55 smithi167 ceph-mon[37140]: pgmap v9717: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:32:56.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:32:56 smithi138 ceph-mon[36954]: pgmap v9718: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:32:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:32:56 smithi100 ceph-mon[32599]: pgmap v9718: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:32:56.776 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:32:56 smithi167 ceph-mon[37140]: pgmap v9718: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:32:58.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:32:58.367+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:32:58.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:32:58.367+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:32:58.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:32:58.367+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:32:58.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:32:58.367+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:32:58.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:32:58.367+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:32:59.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:32:59 smithi138 ceph-mon[36954]: pgmap v9719: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:32:59.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:32:59 smithi100 ceph-mon[32599]: pgmap v9719: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:32:59.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:32:59 smithi167 ceph-mon[37140]: pgmap v9719: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:33:01.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:01 smithi138 ceph-mon[36954]: pgmap v9720: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:33:01.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:01 smithi100 ceph-mon[32599]: pgmap v9720: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:33:01.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:01 smithi167 ceph-mon[37140]: pgmap v9720: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:33:03.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:33:03.046+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 msgr2=0x7fe5940b5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-07-26T18:33:03.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:33:03.046+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 msgr2=0x7fe5940b5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-07-26T18:33:03.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:33:03.046+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 0x7fe5940b5b30 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fe590004ee0 tx=0x7fe59008a040 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:33:03.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:33:03.046+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 0x7fe5940b5b30 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fe590004ee0 tx=0x7fe59008a040 comp rx=0 tx=0).stop 2024-07-26T18:33:03.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:33:03.046+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 msgr2=0x7fe5940b5b30 unknown :-1 s=STATE_CLOSED l=1).tick see no progress in more than 10000000 us during connecting to v2:172.21.15.100:6800/3937009534, fault. 2024-07-26T18:33:03.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:33:03.046+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 msgr2=0x7fe5940b5b30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-07-26T18:33:03.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:33:03.046+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 0x7fe5940b5b30 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T18:33:03.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:33:03.046+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T18:33:03.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:33:03.046+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:33:03.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:33:03.047+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fe590004b30 tx=0x7fe5900bb040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T18:33:03.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:03 smithi138 ceph-mon[36954]: pgmap v9721: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:33:03.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:03 smithi100 ceph-mon[32599]: pgmap v9721: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:33:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:03 smithi167 ceph-mon[37140]: pgmap v9721: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:33:05.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:33:05.442+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T18:33:05.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:33:05.442+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c004740 con 0x7fe5a81087a0 2024-07-26T18:33:05.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:33:05.443+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 134 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T18:33:05.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:33:05.443+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 134 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c070 con 0x7fe5a81087a0 2024-07-26T18:33:05.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:05 smithi138 ceph-mon[36954]: pgmap v9722: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:33:05.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:05 smithi100 ceph-mon[32599]: pgmap v9722: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:33:05.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:05 smithi167 ceph-mon[37140]: pgmap v9722: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:33:06.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:06 smithi138 ceph-mon[36954]: pgmap v9723: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:33:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:06 smithi100 ceph-mon[32599]: pgmap v9723: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:33:06.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:06 smithi167 ceph-mon[37140]: pgmap v9723: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:33:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:08 smithi138 ceph-mon[36954]: osdmap e654: 12 total, 12 up, 12 in 2024-07-26T18:33:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:08 smithi138 ceph-mon[36954]: pgmap v9725: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:33:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:08 smithi100 ceph-mon[32599]: osdmap e654: 12 total, 12 up, 12 in 2024-07-26T18:33:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:08 smithi100 ceph-mon[32599]: pgmap v9725: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:33:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:08 smithi167 ceph-mon[37140]: osdmap e654: 12 total, 12 up, 12 in 2024-07-26T18:33:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:08 smithi167 ceph-mon[37140]: pgmap v9725: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:33:11.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:11 smithi138 ceph-mon[36954]: pgmap v9726: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:33:11.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:11 smithi100 ceph-mon[32599]: pgmap v9726: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:33:11.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:11 smithi167 ceph-mon[37140]: pgmap v9726: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:33:13.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:33:13.375+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:33:13.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:33:13.376+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:33:13.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:33:13.376+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:33:13.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:33:13.376+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:33:13.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:33:13.376+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:33:13.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:13 smithi138 ceph-mon[36954]: pgmap v9727: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:33:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:13 smithi100 ceph-mon[32599]: pgmap v9727: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:33:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:13 smithi167 ceph-mon[37140]: pgmap v9727: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:33:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:14 smithi138 ceph-mon[36954]: osdmap e655: 12 total, 12 up, 12 in 2024-07-26T18:33:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:14 smithi138 ceph-mon[36954]: pgmap v9729: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T18:33:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:14 smithi100 ceph-mon[32599]: osdmap e655: 12 total, 12 up, 12 in 2024-07-26T18:33:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:14 smithi100 ceph-mon[32599]: pgmap v9729: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T18:33:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:14 smithi167 ceph-mon[37140]: osdmap e655: 12 total, 12 up, 12 in 2024-07-26T18:33:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:14 smithi167 ceph-mon[37140]: pgmap v9729: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T18:33:17.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:17 smithi138 ceph-mon[36954]: pgmap v9730: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T18:33:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:17 smithi100 ceph-mon[32599]: pgmap v9730: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T18:33:17.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:17 smithi167 ceph-mon[37140]: pgmap v9730: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T18:33:19.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:19 smithi138 ceph-mon[36954]: pgmap v9731: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:33:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:19 smithi100 ceph-mon[32599]: pgmap v9731: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:33:19.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:19 smithi167 ceph-mon[37140]: pgmap v9731: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:33:20.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:20 smithi138 ceph-mon[36954]: pgmap v9732: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:33:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:20 smithi100 ceph-mon[32599]: pgmap v9732: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:33:20.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:20 smithi167 ceph-mon[37140]: pgmap v9732: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:33:23.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:23 smithi138 ceph-mon[36954]: pgmap v9733: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:33:23.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:23 smithi100 ceph-mon[32599]: pgmap v9733: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:33:23.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:23 smithi167 ceph-mon[37140]: pgmap v9733: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:33:25.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:25 smithi138 ceph-mon[36954]: pgmap v9734: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 94 B/s wr, 7 op/s 2024-07-26T18:33:25.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:25 smithi100 ceph-mon[32599]: pgmap v9734: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 94 B/s wr, 7 op/s 2024-07-26T18:33:25.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:25 smithi167 ceph-mon[37140]: pgmap v9734: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 94 B/s wr, 7 op/s 2024-07-26T18:33:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:26 smithi138 ceph-mon[36954]: pgmap v9735: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:33:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:26 smithi100 ceph-mon[32599]: pgmap v9735: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:33:26.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:26 smithi167 ceph-mon[37140]: pgmap v9735: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:33:28.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:33:28.391+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:33:28.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:33:28.392+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:33:28.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:33:28.392+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:33:28.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:33:28.392+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:33:28.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:33:28.392+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:33:29.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:29 smithi138 ceph-mon[36954]: pgmap v9736: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:33:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:29 smithi100 ceph-mon[32599]: pgmap v9736: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:33:29.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:29 smithi167 ceph-mon[37140]: pgmap v9736: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:33:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:30 smithi138 ceph-mon[36954]: pgmap v9737: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:33:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:30 smithi100 ceph-mon[32599]: pgmap v9737: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:33:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:30 smithi167 ceph-mon[37140]: pgmap v9737: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:33:31.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:31 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:33:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:31 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:33:31.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:31 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:33:32.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:32 smithi138 ceph-mon[36954]: pgmap v9738: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:33:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:32 smithi100 ceph-mon[32599]: pgmap v9738: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:33:32.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:32 smithi167 ceph-mon[37140]: pgmap v9738: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:33:33.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:33:33.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:33:33.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:33:33.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:33:33.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:33:33.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:33:33.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:33:33.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:33:33.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:33:33.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:33:33.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:33:33.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:33:33.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:33:33.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:33:33.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:33:33.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:33:33.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:33:33.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:33:34.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:33:34.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:33:34.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:33:34.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:33:34.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:33:34.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:33:34.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:33:34.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:33:34.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:33:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:34 smithi138 ceph-mon[36954]: pgmap v9739: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:33:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:34 smithi100 ceph-mon[32599]: pgmap v9739: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:33:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:34 smithi167 ceph-mon[37140]: pgmap v9739: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:33:37.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:37 smithi138 ceph-mon[36954]: pgmap v9740: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:33:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:37 smithi100 ceph-mon[32599]: pgmap v9740: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:33:37.776 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:37 smithi167 ceph-mon[37140]: pgmap v9740: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:33:38.674 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:38 smithi167 ceph-mon[37140]: pgmap v9741: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:33:38.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:38 smithi138 ceph-mon[36954]: pgmap v9741: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:33:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:38 smithi100 ceph-mon[32599]: pgmap v9741: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:33:41.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:41 smithi138 ceph-mon[36954]: pgmap v9742: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:33:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:41 smithi100 ceph-mon[32599]: pgmap v9742: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:33:41.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:41 smithi167 ceph-mon[37140]: pgmap v9742: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:33:42.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:42 smithi138 ceph-mon[36954]: pgmap v9743: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:33:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:42 smithi100 ceph-mon[32599]: pgmap v9743: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:33:42.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:42 smithi167 ceph-mon[37140]: pgmap v9743: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:33:43.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:33:43.401+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:33:43.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:33:43.401+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:33:43.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:33:43.401+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:33:43.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:33:43.401+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:33:43.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:33:43.401+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:33:44.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:44 smithi138 ceph-mon[36954]: pgmap v9744: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:33:44.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:44 smithi100 ceph-mon[32599]: pgmap v9744: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:33:44.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:44 smithi167 ceph-mon[37140]: pgmap v9744: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:33:46.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:46 smithi138 ceph-mon[36954]: pgmap v9745: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:33:46.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:46 smithi100 ceph-mon[32599]: pgmap v9745: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:33:46.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:46 smithi167 ceph-mon[37140]: pgmap v9745: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:33:48.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:48 smithi138 ceph-mon[36954]: pgmap v9746: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:33:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:48 smithi100 ceph-mon[32599]: pgmap v9746: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:33:48.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:48 smithi167 ceph-mon[37140]: pgmap v9746: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:33:50.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:50 smithi138 ceph-mon[36954]: pgmap v9747: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:33:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:50 smithi100 ceph-mon[32599]: pgmap v9747: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:33:50.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:50 smithi167 ceph-mon[37140]: pgmap v9747: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:33:52.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:52 smithi138 ceph-mon[36954]: pgmap v9748: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:33:52.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:52 smithi100 ceph-mon[32599]: pgmap v9748: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:33:52.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:52 smithi167 ceph-mon[37140]: pgmap v9748: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:33:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:54 smithi138 ceph-mon[36954]: pgmap v9749: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:33:54.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:54 smithi100 ceph-mon[32599]: pgmap v9749: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:33:54.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:54 smithi167 ceph-mon[37140]: pgmap v9749: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:33:56.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:56 smithi138 ceph-mon[36954]: pgmap v9750: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:33:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:56 smithi100 ceph-mon[32599]: pgmap v9750: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:33:56.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:56 smithi167 ceph-mon[37140]: pgmap v9750: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:33:58.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:33:58.403+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:33:58.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:33:58.404+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:33:58.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:33:58.404+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:33:58.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:33:58.404+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:33:58.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:33:58.404+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:33:58.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:33:58 smithi138 ceph-mon[36954]: pgmap v9751: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:33:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:33:58 smithi100 ceph-mon[32599]: pgmap v9751: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:33:58.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:33:58 smithi167 ceph-mon[37140]: pgmap v9751: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:34:00.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:00 smithi138 ceph-mon[36954]: pgmap v9752: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:34:00.747 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:00 smithi100 ceph-mon[32599]: pgmap v9752: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:34:00.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:00 smithi167 ceph-mon[37140]: pgmap v9752: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:34:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:02 smithi138 ceph-mon[36954]: pgmap v9753: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:34:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:02 smithi100 ceph-mon[32599]: pgmap v9753: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:34:02.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:02 smithi167 ceph-mon[37140]: pgmap v9753: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:34:04.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:04 smithi138 ceph-mon[36954]: pgmap v9754: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:34:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:04 smithi100 ceph-mon[32599]: pgmap v9754: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:34:04.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:04 smithi167 ceph-mon[37140]: pgmap v9754: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:34:06.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:06 smithi138 ceph-mon[36954]: pgmap v9755: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:34:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:06 smithi100 ceph-mon[32599]: pgmap v9755: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:34:06.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:06 smithi167 ceph-mon[37140]: pgmap v9755: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:34:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:08 smithi138 ceph-mon[36954]: osdmap e656: 12 total, 12 up, 12 in 2024-07-26T18:34:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:08 smithi138 ceph-mon[36954]: pgmap v9757: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:34:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:08 smithi100 ceph-mon[32599]: osdmap e656: 12 total, 12 up, 12 in 2024-07-26T18:34:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:08 smithi100 ceph-mon[32599]: pgmap v9757: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:34:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:08 smithi167 ceph-mon[37140]: osdmap e656: 12 total, 12 up, 12 in 2024-07-26T18:34:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:08 smithi167 ceph-mon[37140]: pgmap v9757: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:34:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:10 smithi138 ceph-mon[36954]: pgmap v9758: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:34:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:10 smithi100 ceph-mon[32599]: pgmap v9758: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:34:10.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:10 smithi167 ceph-mon[37140]: pgmap v9758: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:34:12.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:12 smithi138 ceph-mon[36954]: pgmap v9759: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:34:12.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:12 smithi100 ceph-mon[32599]: pgmap v9759: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:34:12.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:12 smithi167 ceph-mon[37140]: pgmap v9759: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:34:13.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:34:13.404+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:34:13.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:34:13.405+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:34:13.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:34:13.405+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:34:13.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:34:13.405+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:34:13.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:34:13.405+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:34:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:14 smithi138 ceph-mon[36954]: pgmap v9760: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:34:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:14 smithi100 ceph-mon[32599]: pgmap v9760: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:34:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:14 smithi167 ceph-mon[37140]: pgmap v9760: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:34:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:16 smithi138 ceph-mon[36954]: pgmap v9761: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:34:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:16 smithi100 ceph-mon[32599]: pgmap v9761: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:34:16.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:16 smithi167 ceph-mon[37140]: pgmap v9761: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:34:18.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:18 smithi138 ceph-mon[36954]: pgmap v9762: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 187 B/s wr, 6 op/s 2024-07-26T18:34:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:18 smithi100 ceph-mon[32599]: pgmap v9762: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 187 B/s wr, 6 op/s 2024-07-26T18:34:18.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:18 smithi167 ceph-mon[37140]: pgmap v9762: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 187 B/s wr, 6 op/s 2024-07-26T18:34:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:19 smithi100 ceph-mon[32599]: osdmap e657: 12 total, 12 up, 12 in 2024-07-26T18:34:19.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:19 smithi167 ceph-mon[37140]: osdmap e657: 12 total, 12 up, 12 in 2024-07-26T18:34:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:19 smithi138 ceph-mon[36954]: osdmap e657: 12 total, 12 up, 12 in 2024-07-26T18:34:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:20 smithi100 ceph-mon[32599]: pgmap v9764: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:34:20.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:20 smithi167 ceph-mon[37140]: pgmap v9764: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:34:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:20 smithi138 ceph-mon[36954]: pgmap v9764: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:34:22.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:22 smithi138 ceph-mon[36954]: pgmap v9765: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:34:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:22 smithi100 ceph-mon[32599]: pgmap v9765: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:34:22.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:22 smithi167 ceph-mon[37140]: pgmap v9765: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:34:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:24 smithi138 ceph-mon[36954]: pgmap v9766: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:34:24.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:24 smithi100 ceph-mon[32599]: pgmap v9766: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:34:24.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:24 smithi167 ceph-mon[37140]: pgmap v9766: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:34:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:26 smithi138 ceph-mon[36954]: pgmap v9767: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:34:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:26 smithi100 ceph-mon[32599]: pgmap v9767: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:34:26.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:26 smithi167 ceph-mon[37140]: pgmap v9767: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:34:28.422 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:34:28.421+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:34:28.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:34:28.422+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:34:28.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:34:28.422+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:34:28.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:34:28.422+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:34:28.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:34:28.422+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:34:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:28 smithi138 ceph-mon[36954]: pgmap v9768: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:34:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:28 smithi100 ceph-mon[32599]: pgmap v9768: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:34:28.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:28 smithi167 ceph-mon[37140]: pgmap v9768: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:34:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:30 smithi138 ceph-mon[36954]: pgmap v9769: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:34:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:30 smithi100 ceph-mon[32599]: pgmap v9769: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:34:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:30 smithi167 ceph-mon[37140]: pgmap v9769: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:34:32.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:32 smithi138 ceph-mon[36954]: pgmap v9770: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:34:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:32 smithi100 ceph-mon[32599]: pgmap v9770: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:34:32.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:32 smithi167 ceph-mon[37140]: pgmap v9770: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:34:33.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:33 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:34:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:33 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:34:33.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:33 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:34:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:34:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:34:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:34:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:34:34.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:34:34.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:34:34.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:34:34.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:34:34.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:34:34.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:34 smithi100 ceph-mon[32599]: pgmap v9771: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:34:34.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:34:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:34:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:34:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:34:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:34:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:34:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:34:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:34:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:34:34.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:34 smithi167 ceph-mon[37140]: pgmap v9771: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:34:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:34:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:34:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:34:34.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:34:34.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:34:34.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:34:34.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:34:34.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:34:34.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:34:34.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:34 smithi138 ceph-mon[36954]: pgmap v9771: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:34:36.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:36 smithi138 ceph-mon[36954]: pgmap v9772: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:34:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:36 smithi100 ceph-mon[32599]: pgmap v9772: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:34:36.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:36 smithi167 ceph-mon[37140]: pgmap v9772: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:34:38.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:38 smithi138 ceph-mon[36954]: pgmap v9773: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:34:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:38 smithi100 ceph-mon[32599]: pgmap v9773: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:34:38.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:38 smithi167 ceph-mon[37140]: pgmap v9773: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:34:40.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:40 smithi138 ceph-mon[36954]: pgmap v9774: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:34:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:40 smithi100 ceph-mon[32599]: pgmap v9774: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:34:40.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:40 smithi167 ceph-mon[37140]: pgmap v9774: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:34:42.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:42 smithi138 ceph-mon[36954]: pgmap v9775: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:34:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:42 smithi100 ceph-mon[32599]: pgmap v9775: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:34:42.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:42 smithi167 ceph-mon[37140]: pgmap v9775: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:34:43.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:34:43.437+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:34:43.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:34:43.437+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:34:43.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:34:43.437+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:34:43.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:34:43.437+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:34:43.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:34:43.438+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:34:44.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:44 smithi138 ceph-mon[36954]: pgmap v9776: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:34:44.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:44 smithi100 ceph-mon[32599]: pgmap v9776: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:34:44.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:44 smithi167 ceph-mon[37140]: pgmap v9776: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:34:46.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:46 smithi138 ceph-mon[36954]: pgmap v9777: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:34:46.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:46 smithi100 ceph-mon[32599]: pgmap v9777: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:34:46.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:46 smithi167 ceph-mon[37140]: pgmap v9777: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:34:48.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:48 smithi138 ceph-mon[36954]: pgmap v9778: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:34:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:48 smithi100 ceph-mon[32599]: pgmap v9778: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:34:48.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:48 smithi167 ceph-mon[37140]: pgmap v9778: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:34:50.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:50 smithi138 ceph-mon[36954]: pgmap v9779: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:34:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:50 smithi100 ceph-mon[32599]: pgmap v9779: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:34:50.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:50 smithi167 ceph-mon[37140]: pgmap v9779: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:34:52.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:52 smithi138 ceph-mon[36954]: pgmap v9780: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:34:52.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:52 smithi100 ceph-mon[32599]: pgmap v9780: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:34:52.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:52 smithi167 ceph-mon[37140]: pgmap v9780: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:34:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:54 smithi138 ceph-mon[36954]: pgmap v9781: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:34:54.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:54 smithi100 ceph-mon[32599]: pgmap v9781: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:34:54.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:54 smithi167 ceph-mon[37140]: pgmap v9781: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:34:56.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:56 smithi138 ceph-mon[36954]: pgmap v9782: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:34:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:56 smithi100 ceph-mon[32599]: pgmap v9782: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:34:56.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:56 smithi167 ceph-mon[37140]: pgmap v9782: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:34:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:34:58.442+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:34:58.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:34:58.442+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:34:58.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:34:58.442+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:34:58.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:34:58.442+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:34:58.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:34:58.442+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:34:58.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:34:58 smithi138 ceph-mon[36954]: pgmap v9783: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:34:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:34:58 smithi100 ceph-mon[32599]: pgmap v9783: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:34:58.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:34:58 smithi167 ceph-mon[37140]: pgmap v9783: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:35:00.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:00 smithi138 ceph-mon[36954]: pgmap v9784: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:35:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:00 smithi100 ceph-mon[32599]: pgmap v9784: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:35:00.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:00 smithi167 ceph-mon[37140]: pgmap v9784: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:35:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:02 smithi138 ceph-mon[36954]: pgmap v9785: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:35:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:02 smithi100 ceph-mon[32599]: pgmap v9785: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:35:02.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:02 smithi167 ceph-mon[37140]: pgmap v9785: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:35:04.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:04 smithi138 ceph-mon[36954]: pgmap v9786: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:35:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:04 smithi100 ceph-mon[32599]: pgmap v9786: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:35:04.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:04 smithi167 ceph-mon[37140]: pgmap v9786: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:35:06.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:06 smithi138 ceph-mon[36954]: pgmap v9787: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:35:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:06 smithi100 ceph-mon[32599]: pgmap v9787: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:35:06.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:06 smithi167 ceph-mon[37140]: pgmap v9787: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:35:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:08 smithi138 ceph-mon[36954]: osdmap e658: 12 total, 12 up, 12 in 2024-07-26T18:35:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:08 smithi138 ceph-mon[36954]: pgmap v9789: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:35:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:08 smithi100 ceph-mon[32599]: osdmap e658: 12 total, 12 up, 12 in 2024-07-26T18:35:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:08 smithi100 ceph-mon[32599]: pgmap v9789: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:35:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:08 smithi167 ceph-mon[37140]: osdmap e658: 12 total, 12 up, 12 in 2024-07-26T18:35:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:08 smithi167 ceph-mon[37140]: pgmap v9789: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:35:10.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:10 smithi138 ceph-mon[36954]: pgmap v9790: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:35:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:10 smithi100 ceph-mon[32599]: pgmap v9790: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:35:10.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:10 smithi167 ceph-mon[37140]: pgmap v9790: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:35:12.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:12 smithi138 ceph-mon[36954]: pgmap v9791: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:35:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:12 smithi100 ceph-mon[32599]: pgmap v9791: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:35:12.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:12 smithi167 ceph-mon[37140]: pgmap v9791: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:35:13.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:35:13.458+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:35:13.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:35:13.458+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:35:13.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:35:13.458+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:35:13.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:35:13.458+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:35:13.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:35:13.458+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:35:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:14 smithi138 ceph-mon[36954]: pgmap v9792: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:35:14.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:14 smithi100 ceph-mon[32599]: pgmap v9792: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:35:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:14 smithi167 ceph-mon[37140]: pgmap v9792: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:35:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:16 smithi138 ceph-mon[36954]: pgmap v9793: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:35:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:16 smithi100 ceph-mon[32599]: pgmap v9793: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:35:16.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:16 smithi167 ceph-mon[37140]: pgmap v9793: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:35:18.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:18 smithi138 ceph-mon[36954]: pgmap v9794: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 187 B/s wr, 6 op/s 2024-07-26T18:35:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:18 smithi100 ceph-mon[32599]: pgmap v9794: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 187 B/s wr, 6 op/s 2024-07-26T18:35:18.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:18 smithi167 ceph-mon[37140]: pgmap v9794: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 187 B/s wr, 6 op/s 2024-07-26T18:35:19.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:19 smithi100 ceph-mon[32599]: osdmap e659: 12 total, 12 up, 12 in 2024-07-26T18:35:19.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:19 smithi167 ceph-mon[37140]: osdmap e659: 12 total, 12 up, 12 in 2024-07-26T18:35:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:19 smithi138 ceph-mon[36954]: osdmap e659: 12 total, 12 up, 12 in 2024-07-26T18:35:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:20 smithi100 ceph-mon[32599]: pgmap v9796: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:35:20.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:20 smithi167 ceph-mon[37140]: pgmap v9796: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:35:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:20 smithi138 ceph-mon[36954]: pgmap v9796: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:35:22.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:22 smithi138 ceph-mon[36954]: pgmap v9797: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:35:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:22 smithi100 ceph-mon[32599]: pgmap v9797: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:35:22.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:22 smithi167 ceph-mon[37140]: pgmap v9797: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:35:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:24 smithi138 ceph-mon[36954]: pgmap v9798: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:35:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:24 smithi100 ceph-mon[32599]: pgmap v9798: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:35:24.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:24 smithi167 ceph-mon[37140]: pgmap v9798: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:35:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:26 smithi138 ceph-mon[36954]: pgmap v9799: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:35:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:26 smithi100 ceph-mon[32599]: pgmap v9799: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:35:26.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:26 smithi167 ceph-mon[37140]: pgmap v9799: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:35:28.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:35:28.474+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:35:28.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:35:28.476+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:35:28.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:35:28.476+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:35:28.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:35:28.476+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:35:28.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:35:28.476+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:35:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:28 smithi138 ceph-mon[36954]: pgmap v9800: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:35:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:28 smithi100 ceph-mon[32599]: pgmap v9800: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:35:28.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:28 smithi167 ceph-mon[37140]: pgmap v9800: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:35:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:30 smithi138 ceph-mon[36954]: pgmap v9801: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:35:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:30 smithi100 ceph-mon[32599]: pgmap v9801: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:35:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:30 smithi167 ceph-mon[37140]: pgmap v9801: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:35:32.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:32 smithi138 ceph-mon[36954]: pgmap v9802: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:35:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:32 smithi100 ceph-mon[32599]: pgmap v9802: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:35:32.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:32 smithi167 ceph-mon[37140]: pgmap v9802: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:35:34.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:34 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:35:34.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:34 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:35:34.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:34 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:35:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:35 smithi138 ceph-mon[36954]: pgmap v9803: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:35:35.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:35:35.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:35:35.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:35:35.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:35:35.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:35:35.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:35:35.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:35:35.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:35:35.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:35:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:35 smithi100 ceph-mon[32599]: pgmap v9803: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:35:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:35:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:35:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:35:35.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:35:35.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:35:35.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:35:35.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:35:35.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:35:35.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:35:35.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:35 smithi167 ceph-mon[37140]: pgmap v9803: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:35:35.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:35:35.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:35:35.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:35:35.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:35:35.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:35:35.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:35:35.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:35:35.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:35:35.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:35:36.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:36 smithi138 ceph-mon[36954]: pgmap v9804: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:35:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:36 smithi100 ceph-mon[32599]: pgmap v9804: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:35:36.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:36 smithi167 ceph-mon[37140]: pgmap v9804: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:35:38.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:38 smithi138 ceph-mon[36954]: pgmap v9805: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:35:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:38 smithi100 ceph-mon[32599]: pgmap v9805: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:35:38.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:38 smithi167 ceph-mon[37140]: pgmap v9805: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:35:40.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:40 smithi138 ceph-mon[36954]: pgmap v9806: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:35:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:40 smithi100 ceph-mon[32599]: pgmap v9806: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:35:40.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:40 smithi167 ceph-mon[37140]: pgmap v9806: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:35:42.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:42 smithi138 ceph-mon[36954]: pgmap v9807: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:35:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:42 smithi100 ceph-mon[32599]: pgmap v9807: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:35:42.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:42 smithi167 ceph-mon[37140]: pgmap v9807: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:35:43.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:35:43.489+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:35:43.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:35:43.489+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:35:43.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:35:43.489+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:35:43.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:35:43.489+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:35:43.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:35:43.489+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:35:44.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:44 smithi138 ceph-mon[36954]: pgmap v9808: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:35:44.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:44 smithi100 ceph-mon[32599]: pgmap v9808: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:35:44.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:44 smithi167 ceph-mon[37140]: pgmap v9808: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:35:46.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:46 smithi138 ceph-mon[36954]: pgmap v9809: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:35:46.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:46 smithi100 ceph-mon[32599]: pgmap v9809: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:35:46.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:46 smithi167 ceph-mon[37140]: pgmap v9809: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:35:48.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:48 smithi138 ceph-mon[36954]: pgmap v9810: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:35:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:48 smithi100 ceph-mon[32599]: pgmap v9810: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:35:48.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:48 smithi167 ceph-mon[37140]: pgmap v9810: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:35:50.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:50 smithi138 ceph-mon[36954]: pgmap v9811: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:35:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:50 smithi100 ceph-mon[32599]: pgmap v9811: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:35:50.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:50 smithi167 ceph-mon[37140]: pgmap v9811: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:35:52.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:52 smithi138 ceph-mon[36954]: pgmap v9812: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:35:52.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:52 smithi100 ceph-mon[32599]: pgmap v9812: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:35:52.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:52 smithi167 ceph-mon[37140]: pgmap v9812: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:35:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:54 smithi138 ceph-mon[36954]: pgmap v9813: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:35:54.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:54 smithi100 ceph-mon[32599]: pgmap v9813: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:35:54.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:54 smithi167 ceph-mon[37140]: pgmap v9813: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:35:56.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:56 smithi138 ceph-mon[36954]: pgmap v9814: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:35:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:56 smithi100 ceph-mon[32599]: pgmap v9814: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:35:56.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:56 smithi167 ceph-mon[37140]: pgmap v9814: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:35:58.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:35:58.505+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:35:58.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:35:58.506+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:35:58.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:35:58.506+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:35:58.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:35:58.506+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:35:58.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:35:58.506+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:35:58.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:35:58 smithi138 ceph-mon[36954]: pgmap v9815: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:35:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:35:58 smithi100 ceph-mon[32599]: pgmap v9815: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:35:58.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:35:58 smithi167 ceph-mon[37140]: pgmap v9815: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:36:00.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:36:00 smithi138 ceph-mon[36954]: pgmap v9816: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:36:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:36:00 smithi100 ceph-mon[32599]: pgmap v9816: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:36:00.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:36:00 smithi167 ceph-mon[37140]: pgmap v9816: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:36:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:36:02 smithi138 ceph-mon[36954]: pgmap v9817: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:36:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:36:02 smithi100 ceph-mon[32599]: pgmap v9817: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:36:02.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:36:02 smithi167 ceph-mon[37140]: pgmap v9817: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:36:04.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:36:04 smithi138 ceph-mon[36954]: pgmap v9818: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:36:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:36:04 smithi100 ceph-mon[32599]: pgmap v9818: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:36:04.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:36:04 smithi167 ceph-mon[37140]: pgmap v9818: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:36:05.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:36:05.468+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T18:36:05.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:36:05.468+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c004740 con 0x7fe5a81087a0 2024-07-26T18:36:05.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:36:05.468+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 135 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T18:36:05.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:36:05.468+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 135 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c070 con 0x7fe5a81087a0 2024-07-26T18:36:06.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:36:06 smithi138 ceph-mon[36954]: pgmap v9819: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:36:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:36:06 smithi100 ceph-mon[32599]: pgmap v9819: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:36:06.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:36:06 smithi167 ceph-mon[37140]: pgmap v9819: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:36:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:36:08 smithi138 ceph-mon[36954]: osdmap e660: 12 total, 12 up, 12 in 2024-07-26T18:36:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:36:08 smithi138 ceph-mon[36954]: pgmap v9821: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:36:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:36:08 smithi100 ceph-mon[32599]: osdmap e660: 12 total, 12 up, 12 in 2024-07-26T18:36:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:36:08 smithi100 ceph-mon[32599]: pgmap v9821: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:36:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:36:08 smithi167 ceph-mon[37140]: osdmap e660: 12 total, 12 up, 12 in 2024-07-26T18:36:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:36:08 smithi167 ceph-mon[37140]: pgmap v9821: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:36:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:36:10 smithi138 ceph-mon[36954]: pgmap v9822: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:36:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:36:10 smithi100 ceph-mon[32599]: pgmap v9822: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:36:10.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:36:10 smithi167 ceph-mon[37140]: pgmap v9822: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:36:12.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:36:12 smithi138 ceph-mon[36954]: pgmap v9823: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:36:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:36:12 smithi100 ceph-mon[32599]: pgmap v9823: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:36:12.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:36:12 smithi167 ceph-mon[37140]: pgmap v9823: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:36:13.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:36:13.508+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:36:13.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:36:13.509+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:36:13.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:36:13.509+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:36:13.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:36:13.509+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:36:13.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:36:13.509+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:36:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:36:14 smithi100 ceph-mon[32599]: osdmap e661: 12 total, 12 up, 12 in 2024-07-26T18:36:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:36:14 smithi100 ceph-mon[32599]: pgmap v9825: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T18:36:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:36:14 smithi167 ceph-mon[37140]: osdmap e661: 12 total, 12 up, 12 in 2024-07-26T18:36:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:36:14 smithi167 ceph-mon[37140]: pgmap v9825: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T18:36:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:36:14 smithi138 ceph-mon[36954]: osdmap e661: 12 total, 12 up, 12 in 2024-07-26T18:36:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:36:14 smithi138 ceph-mon[36954]: pgmap v9825: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T18:36:16.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:36:16 smithi138 ceph-mon[36954]: pgmap v9826: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T18:36:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:36:16 smithi100 ceph-mon[32599]: pgmap v9826: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T18:36:16.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:36:16 smithi167 ceph-mon[37140]: pgmap v9826: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T18:36:18.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:36:18 smithi138 ceph-mon[36954]: pgmap v9827: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:36:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:36:18 smithi100 ceph-mon[32599]: pgmap v9827: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:36:18.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:36:18 smithi167 ceph-mon[37140]: pgmap v9827: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:36:20.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:36:20 smithi138 ceph-mon[36954]: pgmap v9828: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:36:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:36:20 smithi100 ceph-mon[32599]: pgmap v9828: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:36:20.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:36:20 smithi167 ceph-mon[37140]: pgmap v9828: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:36:22.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:36:22 smithi138 ceph-mon[36954]: pgmap v9829: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:36:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:36:22 smithi100 ceph-mon[32599]: pgmap v9829: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:36:22.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:36:22 smithi167 ceph-mon[37140]: pgmap v9829: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:36:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:36:24 smithi138 ceph-mon[36954]: pgmap v9830: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 94 B/s wr, 7 op/s 2024-07-26T18:36:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:36:24 smithi100 ceph-mon[32599]: pgmap v9830: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 94 B/s wr, 7 op/s 2024-07-26T18:36:24.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:36:24 smithi167 ceph-mon[37140]: pgmap v9830: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 94 B/s wr, 7 op/s 2024-07-26T18:36:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:36:26 smithi138 ceph-mon[36954]: pgmap v9831: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:36:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:36:26 smithi100 ceph-mon[32599]: pgmap v9831: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:36:26.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:36:26 smithi167 ceph-mon[37140]: pgmap v9831: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:36:28.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:36:28.523+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:36:28.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:36:28.524+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:36:28.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:36:28.524+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:36:28.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:36:28.524+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:36:28.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:36:28.524+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:36:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:36:28 smithi138 ceph-mon[36954]: pgmap v9832: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:36:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:36:28 smithi100 ceph-mon[32599]: pgmap v9832: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:36:28.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:36:28 smithi167 ceph-mon[37140]: pgmap v9832: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:36:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:36:30 smithi138 ceph-mon[36954]: pgmap v9833: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:36:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:36:30 smithi100 ceph-mon[32599]: pgmap v9833: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:36:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:36:30 smithi167 ceph-mon[37140]: pgmap v9833: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:36:32.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:36:32 smithi138 ceph-mon[36954]: pgmap v9834: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:36:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:36:32 smithi100 ceph-mon[32599]: pgmap v9834: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:36:32.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:36:32 smithi167 ceph-mon[37140]: pgmap v9834: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:36:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:36:34 smithi100 ceph-mon[32599]: pgmap v9835: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:36:34.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:36:34 smithi167 ceph-mon[37140]: pgmap v9835: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:36:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:36:34 smithi138 ceph-mon[36954]: pgmap v9835: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:36:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:36:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:36:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:36:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:36:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:36:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:36:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:36:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:36:35.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:36:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:36:35.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:36:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:36:35.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:36:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:36:35.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:36:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:36:35.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:36:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:36:35.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:36:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:36:35.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:36:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:36:35.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:36:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:36:36.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:36:36 smithi167 ceph-mon[37140]: pgmap v9836: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:36:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:36:36 smithi138 ceph-mon[36954]: pgmap v9836: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:36:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:36:36 smithi100 ceph-mon[32599]: pgmap v9836: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:36:38.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:36:38 smithi138 ceph-mon[36954]: pgmap v9837: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:36:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:36:38 smithi100 ceph-mon[32599]: pgmap v9837: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:36:38.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:36:38 smithi167 ceph-mon[37140]: pgmap v9837: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:36:40.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:36:40 smithi138 ceph-mon[36954]: pgmap v9838: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:36:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:36:40 smithi100 ceph-mon[32599]: pgmap v9838: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:36:40.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:36:40 smithi167 ceph-mon[37140]: pgmap v9838: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:36:42.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:36:42 smithi138 ceph-mon[36954]: pgmap v9839: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:36:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:36:42 smithi100 ceph-mon[32599]: pgmap v9839: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:36:42.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:36:42 smithi167 ceph-mon[37140]: pgmap v9839: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:36:43.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:36:43.539+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:36:43.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:36:43.540+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:36:43.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:36:43.540+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:36:43.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:36:43.540+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:36:43.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:36:43.540+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:36:44.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:36:44 smithi138 ceph-mon[36954]: pgmap v9840: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:36:44.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:36:44 smithi100 ceph-mon[32599]: pgmap v9840: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:36:44.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:36:44 smithi167 ceph-mon[37140]: pgmap v9840: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:36:46.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:36:46 smithi138 ceph-mon[36954]: pgmap v9841: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:36:46.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:36:46 smithi100 ceph-mon[32599]: pgmap v9841: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:36:46.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:36:46 smithi167 ceph-mon[37140]: pgmap v9841: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:36:48.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:36:48 smithi138 ceph-mon[36954]: pgmap v9842: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:36:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:36:48 smithi100 ceph-mon[32599]: pgmap v9842: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:36:48.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:36:48 smithi167 ceph-mon[37140]: pgmap v9842: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:36:50.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:36:50.473+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c003690 con 0x7fe5a81087a0 2024-07-26T18:36:50.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:36:50.473+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T18:36:50.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:36:50.474+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 136 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T18:36:50.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:36:50.474+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 136 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe5a008c070 con 0x7fe5a81087a0 2024-07-26T18:36:50.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:36:50 smithi138 ceph-mon[36954]: pgmap v9843: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:36:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:36:50 smithi100 ceph-mon[32599]: pgmap v9843: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:36:50.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:36:50 smithi167 ceph-mon[37140]: pgmap v9843: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:36:52.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:36:52 smithi138 ceph-mon[36954]: pgmap v9844: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:36:52.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:36:52 smithi100 ceph-mon[32599]: pgmap v9844: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:36:52.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:36:52 smithi167 ceph-mon[37140]: pgmap v9844: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:36:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:36:54 smithi138 ceph-mon[36954]: pgmap v9845: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:36:54.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:36:54 smithi100 ceph-mon[32599]: pgmap v9845: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:36:54.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:36:54 smithi167 ceph-mon[37140]: pgmap v9845: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:36:56.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:36:56 smithi138 ceph-mon[36954]: pgmap v9846: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:36:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:36:56 smithi100 ceph-mon[32599]: pgmap v9846: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:36:56.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:36:56 smithi167 ceph-mon[37140]: pgmap v9846: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:36:58.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:36:58.555+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:36:58.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:36:58.555+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:36:58.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:36:58.555+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:36:58.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:36:58.555+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:36:58.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:36:58.555+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:36:58.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:36:58 smithi138 ceph-mon[36954]: pgmap v9847: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:36:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:36:58 smithi100 ceph-mon[32599]: pgmap v9847: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:36:58.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:36:58 smithi167 ceph-mon[37140]: pgmap v9847: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:37:00.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:37:00 smithi138 ceph-mon[36954]: pgmap v9848: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:37:00.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:37:00 smithi100 ceph-mon[32599]: pgmap v9848: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:37:00.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:37:00 smithi167 ceph-mon[37140]: pgmap v9848: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:37:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:37:02 smithi138 ceph-mon[36954]: pgmap v9849: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:37:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:37:02 smithi100 ceph-mon[32599]: pgmap v9849: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:37:02.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:37:02 smithi167 ceph-mon[37140]: pgmap v9849: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:37:04.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:37:04 smithi138 ceph-mon[36954]: pgmap v9850: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:37:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:37:04 smithi100 ceph-mon[32599]: pgmap v9850: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:37:04.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:37:04 smithi167 ceph-mon[37140]: pgmap v9850: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:37:06.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:37:06 smithi138 ceph-mon[36954]: pgmap v9851: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:37:06.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:37:06 smithi100 ceph-mon[32599]: pgmap v9851: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:37:06.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:37:06 smithi167 ceph-mon[37140]: pgmap v9851: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:37:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:37:08 smithi100 ceph-mon[32599]: osdmap e662: 12 total, 12 up, 12 in 2024-07-26T18:37:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:37:08 smithi100 ceph-mon[32599]: pgmap v9853: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:37:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:37:08 smithi167 ceph-mon[37140]: osdmap e662: 12 total, 12 up, 12 in 2024-07-26T18:37:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:37:08 smithi167 ceph-mon[37140]: pgmap v9853: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:37:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:37:08 smithi138 ceph-mon[36954]: osdmap e662: 12 total, 12 up, 12 in 2024-07-26T18:37:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:37:08 smithi138 ceph-mon[36954]: pgmap v9853: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:37:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:37:10 smithi138 ceph-mon[36954]: pgmap v9854: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:37:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:37:10 smithi100 ceph-mon[32599]: pgmap v9854: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:37:10.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:37:10 smithi167 ceph-mon[37140]: pgmap v9854: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:37:12.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:37:12 smithi138 ceph-mon[36954]: pgmap v9855: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:37:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:37:12 smithi100 ceph-mon[32599]: pgmap v9855: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:37:12.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:37:12 smithi167 ceph-mon[37140]: pgmap v9855: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:37:13.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:37:13.571+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:37:13.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:37:13.572+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:37:13.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:37:13.572+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:37:13.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:37:13.572+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:37:13.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:37:13.572+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:37:14.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:37:14 smithi138 ceph-mon[36954]: pgmap v9856: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:37:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:37:14 smithi100 ceph-mon[32599]: pgmap v9856: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:37:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:37:14 smithi167 ceph-mon[37140]: pgmap v9856: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:37:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:37:16 smithi138 ceph-mon[36954]: pgmap v9857: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:37:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:37:16 smithi100 ceph-mon[32599]: pgmap v9857: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:37:16.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:37:16 smithi167 ceph-mon[37140]: pgmap v9857: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:37:18.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:37:18 smithi138 ceph-mon[36954]: pgmap v9858: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 187 B/s wr, 6 op/s 2024-07-26T18:37:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:37:18 smithi100 ceph-mon[32599]: pgmap v9858: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 187 B/s wr, 6 op/s 2024-07-26T18:37:18.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:37:18 smithi167 ceph-mon[37140]: pgmap v9858: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 187 B/s wr, 6 op/s 2024-07-26T18:37:19.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:37:19 smithi167 ceph-mon[37140]: osdmap e663: 12 total, 12 up, 12 in 2024-07-26T18:37:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:37:19 smithi138 ceph-mon[36954]: osdmap e663: 12 total, 12 up, 12 in 2024-07-26T18:37:19.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:37:19 smithi100 ceph-mon[32599]: osdmap e663: 12 total, 12 up, 12 in 2024-07-26T18:37:20.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:37:20 smithi167 ceph-mon[37140]: pgmap v9860: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:37:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:37:20 smithi138 ceph-mon[36954]: pgmap v9860: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:37:20.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:37:20 smithi100 ceph-mon[32599]: pgmap v9860: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:37:22.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:37:22 smithi138 ceph-mon[36954]: pgmap v9861: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:37:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:37:22 smithi100 ceph-mon[32599]: pgmap v9861: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:37:22.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:37:22 smithi167 ceph-mon[37140]: pgmap v9861: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:37:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:37:24 smithi138 ceph-mon[36954]: pgmap v9862: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:37:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:37:24 smithi100 ceph-mon[32599]: pgmap v9862: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:37:24.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:37:24 smithi167 ceph-mon[37140]: pgmap v9862: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:37:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:37:26 smithi100 ceph-mon[32599]: pgmap v9863: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:37:26.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:37:26 smithi167 ceph-mon[37140]: pgmap v9863: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:37:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:37:26 smithi138 ceph-mon[36954]: pgmap v9863: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:37:28.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:37:28.578+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:37:28.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:37:28.578+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:37:28.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:37:28.578+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:37:28.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:37:28.578+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:37:28.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:37:28.578+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:37:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:37:28 smithi100 ceph-mon[32599]: pgmap v9864: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:37:28.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:37:28 smithi167 ceph-mon[37140]: pgmap v9864: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:37:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:37:28 smithi138 ceph-mon[36954]: pgmap v9864: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:37:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:37:30 smithi138 ceph-mon[36954]: pgmap v9865: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:37:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:37:30 smithi100 ceph-mon[32599]: pgmap v9865: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:37:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:37:30 smithi167 ceph-mon[37140]: pgmap v9865: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:37:32.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:37:32 smithi138 ceph-mon[36954]: pgmap v9866: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:37:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:37:32 smithi100 ceph-mon[32599]: pgmap v9866: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:37:32.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:37:32 smithi167 ceph-mon[37140]: pgmap v9866: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:37:34.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:37:34 smithi138 ceph-mon[36954]: pgmap v9867: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:37:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:37:34 smithi100 ceph-mon[32599]: pgmap v9867: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:37:34.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:37:34 smithi167 ceph-mon[37140]: pgmap v9867: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:37:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:37:35 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:37:35.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:37:35 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:37:35.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:37:35 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:37:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:37:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:37:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:37:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:37:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:37:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:37:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:37:36 smithi100 ceph-mon[32599]: pgmap v9868: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:37:36.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:37:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:37:36.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:37:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:37:36.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:37:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:37:36.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:37:36 smithi167 ceph-mon[37140]: pgmap v9868: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:37:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:37:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:37:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:37:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:37:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:37:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:37:36.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:37:36 smithi138 ceph-mon[36954]: pgmap v9868: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:37:38.743 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:37:38 smithi138 ceph-mon[36954]: pgmap v9869: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:37:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:37:38 smithi100 ceph-mon[32599]: pgmap v9869: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:37:38.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:37:38 smithi167 ceph-mon[37140]: pgmap v9869: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:37:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:37:40 smithi100 ceph-mon[32599]: pgmap v9870: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:37:40.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:37:40 smithi167 ceph-mon[37140]: pgmap v9870: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:37:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:37:40 smithi138 ceph-mon[36954]: pgmap v9870: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:37:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:37:42 smithi100 ceph-mon[32599]: pgmap v9871: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:37:42.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:37:42 smithi167 ceph-mon[37140]: pgmap v9871: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:37:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:37:42 smithi138 ceph-mon[36954]: pgmap v9871: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:37:43.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:37:43.592+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:37:43.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:37:43.593+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:37:43.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:37:43.593+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:37:43.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:37:43.593+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:37:43.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:37:43.593+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:37:44.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:37:44 smithi100 ceph-mon[32599]: pgmap v9872: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:37:44.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:37:44 smithi167 ceph-mon[37140]: pgmap v9872: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:37:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:37:44 smithi138 ceph-mon[36954]: pgmap v9872: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:37:46.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:37:46 smithi100 ceph-mon[32599]: pgmap v9873: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:37:46.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:37:46 smithi167 ceph-mon[37140]: pgmap v9873: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:37:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:37:46 smithi138 ceph-mon[36954]: pgmap v9873: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:37:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:37:48 smithi100 ceph-mon[32599]: pgmap v9874: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:37:48.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:37:48 smithi167 ceph-mon[37140]: pgmap v9874: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:37:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:37:48 smithi138 ceph-mon[36954]: pgmap v9874: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:37:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:37:50 smithi100 ceph-mon[32599]: pgmap v9875: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:37:50.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:37:50 smithi167 ceph-mon[37140]: pgmap v9875: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:37:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:37:50 smithi138 ceph-mon[36954]: pgmap v9875: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:37:52.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:37:52 smithi100 ceph-mon[32599]: pgmap v9876: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:37:52.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:37:52 smithi167 ceph-mon[37140]: pgmap v9876: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:37:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:37:52 smithi138 ceph-mon[36954]: pgmap v9876: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:37:54.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:37:54 smithi100 ceph-mon[32599]: pgmap v9877: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:37:54.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:37:54 smithi167 ceph-mon[37140]: pgmap v9877: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:37:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:37:54 smithi138 ceph-mon[36954]: pgmap v9877: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:37:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:37:56 smithi100 ceph-mon[32599]: pgmap v9878: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:37:56.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:37:56 smithi167 ceph-mon[37140]: pgmap v9878: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:37:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:37:56 smithi138 ceph-mon[36954]: pgmap v9878: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:37:58.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:37:58.608+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:37:58.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:37:58.609+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:37:58.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:37:58.609+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:37:58.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:37:58.609+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:37:58.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:37:58.609+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:37:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:37:58 smithi100 ceph-mon[32599]: pgmap v9879: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:37:58.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:37:58 smithi167 ceph-mon[37140]: pgmap v9879: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:37:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:37:58 smithi138 ceph-mon[36954]: pgmap v9879: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:38:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:38:00 smithi100 ceph-mon[32599]: pgmap v9880: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:38:00.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:38:00 smithi167 ceph-mon[37140]: pgmap v9880: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:38:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:38:00 smithi138 ceph-mon[36954]: pgmap v9880: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:38:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:38:02 smithi100 ceph-mon[32599]: pgmap v9881: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:38:02.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:38:02 smithi167 ceph-mon[37140]: pgmap v9881: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:38:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:38:02 smithi138 ceph-mon[36954]: pgmap v9881: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:38:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:38:04 smithi100 ceph-mon[32599]: pgmap v9882: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:38:04.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:38:04 smithi167 ceph-mon[37140]: pgmap v9882: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:38:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:38:04 smithi138 ceph-mon[36954]: pgmap v9882: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:38:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:38:06 smithi100 ceph-mon[32599]: pgmap v9883: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:38:06.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:38:06 smithi167 ceph-mon[37140]: pgmap v9883: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:38:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:38:06 smithi138 ceph-mon[36954]: pgmap v9883: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:38:08.743 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:38:08 smithi138 ceph-mon[36954]: osdmap e664: 12 total, 12 up, 12 in 2024-07-26T18:38:08.743 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:38:08 smithi138 ceph-mon[36954]: pgmap v9885: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:38:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:38:08 smithi100 ceph-mon[32599]: osdmap e664: 12 total, 12 up, 12 in 2024-07-26T18:38:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:38:08 smithi100 ceph-mon[32599]: pgmap v9885: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:38:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:38:08 smithi167 ceph-mon[37140]: osdmap e664: 12 total, 12 up, 12 in 2024-07-26T18:38:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:38:08 smithi167 ceph-mon[37140]: pgmap v9885: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:38:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:38:10 smithi100 ceph-mon[32599]: pgmap v9886: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:38:10.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:38:10 smithi167 ceph-mon[37140]: pgmap v9886: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:38:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:38:10 smithi138 ceph-mon[36954]: pgmap v9886: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:38:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:38:12 smithi100 ceph-mon[32599]: pgmap v9887: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:38:12.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:38:12 smithi167 ceph-mon[37140]: pgmap v9887: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:38:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:38:12 smithi138 ceph-mon[36954]: pgmap v9887: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:38:13.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:38:13.624+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:38:13.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:38:13.625+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:38:13.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:38:13.625+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:38:13.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:38:13.625+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:38:13.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:38:13.625+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:38:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:38:14 smithi167 ceph-mon[37140]: osdmap e665: 12 total, 12 up, 12 in 2024-07-26T18:38:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:38:14 smithi167 ceph-mon[37140]: pgmap v9889: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T18:38:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:38:14 smithi138 ceph-mon[36954]: osdmap e665: 12 total, 12 up, 12 in 2024-07-26T18:38:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:38:14 smithi138 ceph-mon[36954]: pgmap v9889: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T18:38:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:38:14 smithi100 ceph-mon[32599]: osdmap e665: 12 total, 12 up, 12 in 2024-07-26T18:38:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:38:14 smithi100 ceph-mon[32599]: pgmap v9889: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T18:38:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:38:16 smithi100 ceph-mon[32599]: pgmap v9890: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 228 B/s wr, 6 op/s 2024-07-26T18:38:16.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:38:16 smithi167 ceph-mon[37140]: pgmap v9890: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 228 B/s wr, 6 op/s 2024-07-26T18:38:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:38:16 smithi138 ceph-mon[36954]: pgmap v9890: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 228 B/s wr, 6 op/s 2024-07-26T18:38:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:38:18 smithi100 ceph-mon[32599]: pgmap v9891: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:38:18.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:38:18 smithi167 ceph-mon[37140]: pgmap v9891: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:38:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:38:18 smithi138 ceph-mon[36954]: pgmap v9891: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:38:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:38:20 smithi100 ceph-mon[32599]: pgmap v9892: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:38:20.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:38:20 smithi167 ceph-mon[37140]: pgmap v9892: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:38:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:38:20 smithi138 ceph-mon[36954]: pgmap v9892: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:38:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:38:22 smithi100 ceph-mon[32599]: pgmap v9893: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:38:22.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:38:22 smithi167 ceph-mon[37140]: pgmap v9893: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:38:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:38:22 smithi138 ceph-mon[36954]: pgmap v9893: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:38:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:38:24 smithi100 ceph-mon[32599]: pgmap v9894: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 94 B/s wr, 7 op/s 2024-07-26T18:38:24.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:38:24 smithi167 ceph-mon[37140]: pgmap v9894: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 94 B/s wr, 7 op/s 2024-07-26T18:38:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:38:24 smithi138 ceph-mon[36954]: pgmap v9894: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 94 B/s wr, 7 op/s 2024-07-26T18:38:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:38:26 smithi100 ceph-mon[32599]: pgmap v9895: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:38:26.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:38:26 smithi167 ceph-mon[37140]: pgmap v9895: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:38:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:38:26 smithi138 ceph-mon[36954]: pgmap v9895: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:38:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:38:28.632+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:38:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:38:28.632+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:38:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:38:28.632+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:38:28.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:38:28.632+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:38:28.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:38:28.632+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:38:29.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:38:29 smithi100 ceph-mon[32599]: pgmap v9896: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:38:29.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:38:29 smithi167 ceph-mon[37140]: pgmap v9896: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:38:29.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:38:29 smithi138 ceph-mon[36954]: pgmap v9896: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:38:31.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:38:31 smithi100 ceph-mon[32599]: pgmap v9897: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:38:31.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:38:31 smithi167 ceph-mon[37140]: pgmap v9897: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:38:31.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:38:31 smithi138 ceph-mon[36954]: pgmap v9897: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:38:33.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:38:33 smithi100 ceph-mon[32599]: pgmap v9898: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:38:33.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:38:33 smithi167 ceph-mon[37140]: pgmap v9898: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:38:33.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:38:33 smithi138 ceph-mon[36954]: pgmap v9898: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:38:35.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:38:35 smithi100 ceph-mon[32599]: pgmap v9899: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:38:35.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:38:35 smithi167 ceph-mon[37140]: pgmap v9899: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:38:35.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:38:35 smithi138 ceph-mon[36954]: pgmap v9899: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:38:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:38:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:38:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:38:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:38:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:38:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:38:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:38:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:38:36.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:38:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:38:36.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:38:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:38:36.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:38:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:38:36.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:38:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:38:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:38:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:38:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:38:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:38:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:38:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:38:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:38:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:38:37.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:38:37 smithi100 ceph-mon[32599]: pgmap v9900: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:38:37.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:38:37 smithi167 ceph-mon[37140]: pgmap v9900: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:38:37.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:38:37 smithi138 ceph-mon[36954]: pgmap v9900: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:38:39.742 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:38:39 smithi138 ceph-mon[36954]: pgmap v9901: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:38:39.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:38:39 smithi100 ceph-mon[32599]: pgmap v9901: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:38:39.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:38:39 smithi167 ceph-mon[37140]: pgmap v9901: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:38:41.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:38:41 smithi100 ceph-mon[32599]: pgmap v9902: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:38:41.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:38:41 smithi167 ceph-mon[37140]: pgmap v9902: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:38:41.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:38:41 smithi138 ceph-mon[36954]: pgmap v9902: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:38:43.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:38:43.648+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:38:43.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:38:43.648+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:38:43.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:38:43.648+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:38:43.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:38:43.648+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:38:43.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:38:43.648+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:38:43.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:38:43 smithi100 ceph-mon[32599]: pgmap v9903: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:38:43.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:38:43 smithi167 ceph-mon[37140]: pgmap v9903: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:38:43.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:38:43 smithi138 ceph-mon[36954]: pgmap v9903: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:38:45.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:38:45 smithi138 ceph-mon[36954]: pgmap v9904: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:38:45.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:38:45 smithi100 ceph-mon[32599]: pgmap v9904: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:38:46.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:38:45 smithi167 ceph-mon[37140]: pgmap v9904: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:38:47.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:38:47 smithi138 ceph-mon[36954]: pgmap v9905: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:38:47.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:38:47 smithi100 ceph-mon[32599]: pgmap v9905: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:38:48.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:38:47 smithi167 ceph-mon[37140]: pgmap v9905: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:38:49.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:38:49 smithi138 ceph-mon[36954]: pgmap v9906: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:38:49.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:38:49 smithi100 ceph-mon[32599]: pgmap v9906: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:38:50.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:38:49 smithi167 ceph-mon[37140]: pgmap v9906: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:38:51.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:38:51 smithi138 ceph-mon[36954]: pgmap v9907: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:38:51.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:38:51 smithi100 ceph-mon[32599]: pgmap v9907: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:38:52.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:38:51 smithi167 ceph-mon[37140]: pgmap v9907: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:38:53.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:38:53 smithi138 ceph-mon[36954]: pgmap v9908: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:38:53.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:38:53 smithi100 ceph-mon[32599]: pgmap v9908: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:38:54.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:38:53 smithi167 ceph-mon[37140]: pgmap v9908: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:38:55.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:38:55 smithi138 ceph-mon[36954]: pgmap v9909: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:38:55.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:38:55 smithi100 ceph-mon[32599]: pgmap v9909: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:38:56.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:38:55 smithi167 ceph-mon[37140]: pgmap v9909: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:38:57.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:38:57 smithi138 ceph-mon[36954]: pgmap v9910: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:38:57.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:38:57 smithi100 ceph-mon[32599]: pgmap v9910: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:38:58.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:38:57 smithi167 ceph-mon[37140]: pgmap v9910: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:38:58.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:38:58.664+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:38:58.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:38:58.665+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:38:58.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:38:58.665+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:38:58.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:38:58.665+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:38:58.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:38:58.665+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:38:59.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:38:59 smithi138 ceph-mon[36954]: pgmap v9911: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:38:59.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:38:59 smithi100 ceph-mon[32599]: pgmap v9911: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:39:00.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:38:59 smithi167 ceph-mon[37140]: pgmap v9911: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:39:01.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:39:01 smithi138 ceph-mon[36954]: pgmap v9912: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:39:01.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:39:01 smithi100 ceph-mon[32599]: pgmap v9912: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:39:02.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:39:01 smithi167 ceph-mon[37140]: pgmap v9912: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:39:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:39:03 smithi138 ceph-mon[36954]: pgmap v9913: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:39:03.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:39:03 smithi100 ceph-mon[32599]: pgmap v9913: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:39:04.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:39:03 smithi167 ceph-mon[37140]: pgmap v9913: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:39:05.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:39:05.492+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c004740 con 0x7fe5a81087a0 2024-07-26T18:39:05.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:39:05.492+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T18:39:05.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:39:05.492+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 137 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c070 con 0x7fe5a81087a0 2024-07-26T18:39:05.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:39:05.492+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 137 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T18:39:05.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:39:05 smithi100 ceph-mon[32599]: pgmap v9914: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:39:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:39:05 smithi138 ceph-mon[36954]: pgmap v9914: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:39:06.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:39:05 smithi167 ceph-mon[37140]: pgmap v9914: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:39:07.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:39:07 smithi138 ceph-mon[36954]: pgmap v9915: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:39:07.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:39:07 smithi100 ceph-mon[32599]: pgmap v9915: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:39:08.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:39:07 smithi167 ceph-mon[37140]: pgmap v9915: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:39:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:39:08 smithi138 ceph-mon[36954]: osdmap e666: 12 total, 12 up, 12 in 2024-07-26T18:39:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:39:08 smithi100 ceph-mon[32599]: osdmap e666: 12 total, 12 up, 12 in 2024-07-26T18:39:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:39:08 smithi167 ceph-mon[37140]: osdmap e666: 12 total, 12 up, 12 in 2024-07-26T18:39:09.841 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:39:09 smithi138 ceph-mon[36954]: pgmap v9917: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:39:09.924 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:39:09 smithi167 ceph-mon[37140]: pgmap v9917: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:39:09.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:39:09 smithi100 ceph-mon[32599]: pgmap v9917: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:39:11.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:39:11 smithi138 ceph-mon[36954]: pgmap v9918: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:39:11.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:39:11 smithi100 ceph-mon[32599]: pgmap v9918: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:39:12.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:39:11 smithi167 ceph-mon[37140]: pgmap v9918: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:39:13.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:39:13.680+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:39:13.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:39:13.681+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:39:13.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:39:13.681+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:39:13.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:39:13.681+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:39:13.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:39:13.681+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:39:13.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:39:13 smithi138 ceph-mon[36954]: pgmap v9919: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:39:13.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:39:13 smithi100 ceph-mon[32599]: pgmap v9919: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:39:14.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:39:13 smithi167 ceph-mon[37140]: pgmap v9919: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:39:15.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:39:15 smithi138 ceph-mon[36954]: pgmap v9920: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:39:15.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:39:15 smithi100 ceph-mon[32599]: pgmap v9920: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:39:16.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:39:15 smithi167 ceph-mon[37140]: pgmap v9920: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:39:17.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:39:17 smithi138 ceph-mon[36954]: pgmap v9921: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:39:17.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:39:17 smithi100 ceph-mon[32599]: pgmap v9921: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:39:18.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:39:17 smithi167 ceph-mon[37140]: pgmap v9921: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:39:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:39:19 smithi138 ceph-mon[36954]: pgmap v9922: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 188 B/s wr, 5 op/s 2024-07-26T18:39:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:39:19 smithi138 ceph-mon[36954]: osdmap e667: 12 total, 12 up, 12 in 2024-07-26T18:39:19.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:39:19 smithi100 ceph-mon[32599]: pgmap v9922: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 188 B/s wr, 5 op/s 2024-07-26T18:39:19.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:39:19 smithi100 ceph-mon[32599]: osdmap e667: 12 total, 12 up, 12 in 2024-07-26T18:39:20.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:39:19 smithi167 ceph-mon[37140]: pgmap v9922: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 188 B/s wr, 5 op/s 2024-07-26T18:39:20.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:39:19 smithi167 ceph-mon[37140]: osdmap e667: 12 total, 12 up, 12 in 2024-07-26T18:39:21.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:39:21 smithi138 ceph-mon[36954]: pgmap v9924: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:39:21.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:39:21 smithi100 ceph-mon[32599]: pgmap v9924: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:39:22.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:39:21 smithi167 ceph-mon[37140]: pgmap v9924: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:39:23.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:39:23 smithi138 ceph-mon[36954]: pgmap v9925: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:39:23.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:39:23 smithi100 ceph-mon[32599]: pgmap v9925: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:39:24.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:39:23 smithi167 ceph-mon[37140]: pgmap v9925: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:39:25.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:39:25 smithi138 ceph-mon[36954]: pgmap v9926: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:39:25.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:39:25 smithi100 ceph-mon[32599]: pgmap v9926: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:39:26.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:39:25 smithi167 ceph-mon[37140]: pgmap v9926: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:39:27.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:39:27 smithi138 ceph-mon[36954]: pgmap v9927: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:39:27.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:39:27 smithi100 ceph-mon[32599]: pgmap v9927: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:39:28.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:39:27 smithi167 ceph-mon[37140]: pgmap v9927: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:39:28.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:39:28.690+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:39:28.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:39:28.690+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:39:28.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:39:28.690+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:39:28.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:39:28.690+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:39:28.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:39:28.690+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:39:29.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:39:29 smithi138 ceph-mon[36954]: pgmap v9928: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:39:29.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:39:29 smithi100 ceph-mon[32599]: pgmap v9928: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:39:30.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:39:29 smithi167 ceph-mon[37140]: pgmap v9928: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:39:31.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:39:31 smithi138 ceph-mon[36954]: pgmap v9929: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:39:31.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:39:31 smithi100 ceph-mon[32599]: pgmap v9929: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:39:32.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:39:31 smithi167 ceph-mon[37140]: pgmap v9929: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:39:33.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:39:33 smithi138 ceph-mon[36954]: pgmap v9930: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:39:33.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:39:33 smithi100 ceph-mon[32599]: pgmap v9930: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:39:34.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:39:33 smithi167 ceph-mon[37140]: pgmap v9930: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:39:35.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:39:35 smithi138 ceph-mon[36954]: pgmap v9931: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:39:35.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:39:35 smithi100 ceph-mon[32599]: pgmap v9931: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:39:36.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:39:35 smithi167 ceph-mon[37140]: pgmap v9931: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:39:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:39:36 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:39:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:39:36 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:39:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:39:36 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:39:37.646 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:39:37 smithi100 ceph-mon[32599]: pgmap v9932: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:39:37.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:39:37 smithi138 ceph-mon[36954]: pgmap v9932: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:39:38.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:39:37 smithi167 ceph-mon[37140]: pgmap v9932: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:39:39.681 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:39:39 smithi100 ceph-mon[32599]: pgmap v9933: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:39:39.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:39:39 smithi167 ceph-mon[37140]: pgmap v9933: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:39:39.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:39:39 smithi138 ceph-mon[36954]: pgmap v9933: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:39:41.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:39:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:39:41.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:39:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:39:41.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:39:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:39:41.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:39:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:39:41.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:39:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:39:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:39:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:39:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:39:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:39:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:39:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:39:41.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:39:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:39:41.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:39:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:39:41.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:39:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:39:41.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:39:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:39:41.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:39:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:39:41.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:39:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:39:41.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:39:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:39:42.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:39:42 smithi138 ceph-mon[36954]: pgmap v9934: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:39:42.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:39:42 smithi100 ceph-mon[32599]: pgmap v9934: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:39:42.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:39:42 smithi167 ceph-mon[37140]: pgmap v9934: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:39:43.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:39:43.704+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:39:43.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:39:43.704+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:39:43.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:39:43.704+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:39:43.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:39:43.704+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:39:43.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:39:43.704+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:39:44.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:39:44 smithi138 ceph-mon[36954]: pgmap v9935: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:39:44.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:39:44 smithi100 ceph-mon[32599]: pgmap v9935: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:39:44.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:39:44 smithi167 ceph-mon[37140]: pgmap v9935: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:39:46.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:39:46 smithi138 ceph-mon[36954]: pgmap v9936: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:39:46.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:39:46 smithi100 ceph-mon[32599]: pgmap v9936: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:39:46.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:39:46 smithi167 ceph-mon[37140]: pgmap v9936: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:39:48.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:39:48 smithi138 ceph-mon[36954]: pgmap v9937: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:39:48.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:39:48 smithi100 ceph-mon[32599]: pgmap v9937: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:39:48.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:39:48 smithi167 ceph-mon[37140]: pgmap v9937: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:39:50.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:39:50 smithi138 ceph-mon[36954]: pgmap v9938: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:39:50.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:39:50 smithi100 ceph-mon[32599]: pgmap v9938: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:39:50.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:39:50 smithi167 ceph-mon[37140]: pgmap v9938: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:39:52.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:39:52 smithi138 ceph-mon[36954]: pgmap v9939: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:39:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:39:52 smithi100 ceph-mon[32599]: pgmap v9939: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:39:52.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:39:52 smithi167 ceph-mon[37140]: pgmap v9939: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:39:54.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:39:54 smithi138 ceph-mon[36954]: pgmap v9940: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:39:54.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:39:54 smithi100 ceph-mon[32599]: pgmap v9940: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:39:54.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:39:54 smithi167 ceph-mon[37140]: pgmap v9940: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:39:56.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:39:56 smithi138 ceph-mon[36954]: pgmap v9941: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:39:56.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:39:56 smithi100 ceph-mon[32599]: pgmap v9941: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:39:56.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:39:56 smithi167 ceph-mon[37140]: pgmap v9941: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:39:58.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:39:58 smithi138 ceph-mon[36954]: pgmap v9942: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:39:58.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:39:58 smithi100 ceph-mon[32599]: pgmap v9942: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:39:58.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:39:58 smithi167 ceph-mon[37140]: pgmap v9942: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:39:58.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:39:58.719+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:39:58.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:39:58.719+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:39:58.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:39:58.719+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:39:58.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:39:58.719+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:39:58.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:39:58.719+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:40:00.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:00 smithi138 ceph-mon[36954]: pgmap v9943: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:40:00.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:00 smithi138 ceph-mon[36954]: overall HEALTH_OK 2024-07-26T18:40:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:00 smithi100 ceph-mon[32599]: pgmap v9943: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:40:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:00 smithi100 ceph-mon[32599]: overall HEALTH_OK 2024-07-26T18:40:00.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:00 smithi167 ceph-mon[37140]: pgmap v9943: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:40:00.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:00 smithi167 ceph-mon[37140]: overall HEALTH_OK 2024-07-26T18:40:02.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:02 smithi138 ceph-mon[36954]: pgmap v9944: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:40:02.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:02 smithi100 ceph-mon[32599]: pgmap v9944: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:40:02.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:02 smithi167 ceph-mon[37140]: pgmap v9944: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:40:04.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:04 smithi138 ceph-mon[36954]: pgmap v9945: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:40:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:04 smithi100 ceph-mon[32599]: pgmap v9945: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:40:04.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:04 smithi167 ceph-mon[37140]: pgmap v9945: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:40:06.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:06 smithi138 ceph-mon[36954]: pgmap v9946: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:40:06.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:06 smithi100 ceph-mon[32599]: pgmap v9946: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:40:06.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:06 smithi167 ceph-mon[37140]: pgmap v9946: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:40:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:08 smithi138 ceph-mon[36954]: pgmap v9947: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:40:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:08 smithi100 ceph-mon[32599]: pgmap v9947: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:40:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:08 smithi167 ceph-mon[37140]: pgmap v9947: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:40:09.423 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:09 smithi167 ceph-mon[37140]: osdmap e668: 12 total, 12 up, 12 in 2024-07-26T18:40:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:09 smithi138 ceph-mon[36954]: osdmap e668: 12 total, 12 up, 12 in 2024-07-26T18:40:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:09 smithi100 ceph-mon[32599]: osdmap e668: 12 total, 12 up, 12 in 2024-07-26T18:40:10.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:10 smithi138 ceph-mon[36954]: pgmap v9949: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:40:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:10 smithi100 ceph-mon[32599]: pgmap v9949: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:40:10.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:10 smithi167 ceph-mon[37140]: pgmap v9949: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:40:12.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:12 smithi138 ceph-mon[36954]: pgmap v9950: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:40:12.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:12 smithi100 ceph-mon[32599]: pgmap v9950: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:40:12.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:12 smithi167 ceph-mon[37140]: pgmap v9950: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:40:13.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:40:13.723+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:40:13.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:40:13.723+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:40:13.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:40:13.723+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:40:13.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:40:13.723+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:40:13.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:40:13.724+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:40:14.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:14 smithi138 ceph-mon[36954]: pgmap v9951: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:40:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:14 smithi100 ceph-mon[32599]: pgmap v9951: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:40:14.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:14 smithi167 ceph-mon[37140]: pgmap v9951: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:40:16.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:16 smithi138 ceph-mon[36954]: pgmap v9952: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:40:16.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:16 smithi100 ceph-mon[32599]: pgmap v9952: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:40:16.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:16 smithi167 ceph-mon[37140]: pgmap v9952: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:40:18.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:18 smithi138 ceph-mon[36954]: pgmap v9953: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:40:18.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:18 smithi100 ceph-mon[32599]: pgmap v9953: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:40:18.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:18 smithi167 ceph-mon[37140]: pgmap v9953: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:40:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:19 smithi138 ceph-mon[36954]: pgmap v9954: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 198 B/s wr, 6 op/s 2024-07-26T18:40:19.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:19 smithi138 ceph-mon[36954]: osdmap e669: 12 total, 12 up, 12 in 2024-07-26T18:40:19.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:19 smithi100 ceph-mon[32599]: pgmap v9954: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 198 B/s wr, 6 op/s 2024-07-26T18:40:19.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:19 smithi100 ceph-mon[32599]: osdmap e669: 12 total, 12 up, 12 in 2024-07-26T18:40:20.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:19 smithi167 ceph-mon[37140]: pgmap v9954: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 198 B/s wr, 6 op/s 2024-07-26T18:40:20.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:19 smithi167 ceph-mon[37140]: osdmap e669: 12 total, 12 up, 12 in 2024-07-26T18:40:21.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:21 smithi138 ceph-mon[36954]: pgmap v9956: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:40:21.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:21 smithi100 ceph-mon[32599]: pgmap v9956: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:40:22.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:21 smithi167 ceph-mon[37140]: pgmap v9956: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:40:23.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:23 smithi138 ceph-mon[36954]: pgmap v9957: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:40:23.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:23 smithi100 ceph-mon[32599]: pgmap v9957: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:40:24.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:23 smithi167 ceph-mon[37140]: pgmap v9957: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:40:25.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:25 smithi138 ceph-mon[36954]: pgmap v9958: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:40:25.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:25 smithi100 ceph-mon[32599]: pgmap v9958: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:40:26.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:25 smithi167 ceph-mon[37140]: pgmap v9958: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:40:27.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:27 smithi138 ceph-mon[36954]: pgmap v9959: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:40:27.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:27 smithi100 ceph-mon[32599]: pgmap v9959: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:40:28.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:27 smithi167 ceph-mon[37140]: pgmap v9959: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:40:28.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:40:28.739+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:40:28.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:40:28.740+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:40:28.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:40:28.740+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:40:28.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:40:28.740+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:40:28.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:40:28.740+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:40:29.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:29 smithi138 ceph-mon[36954]: pgmap v9960: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:40:29.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:29 smithi100 ceph-mon[32599]: pgmap v9960: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:40:30.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:29 smithi167 ceph-mon[37140]: pgmap v9960: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:40:31.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:31 smithi138 ceph-mon[36954]: pgmap v9961: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:40:31.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:31 smithi100 ceph-mon[32599]: pgmap v9961: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:40:32.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:31 smithi167 ceph-mon[37140]: pgmap v9961: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:40:33.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:33 smithi138 ceph-mon[36954]: pgmap v9962: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:40:33.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:33 smithi100 ceph-mon[32599]: pgmap v9962: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:40:34.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:33 smithi167 ceph-mon[37140]: pgmap v9962: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:40:35.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:35 smithi138 ceph-mon[36954]: pgmap v9963: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:40:35.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:35 smithi100 ceph-mon[32599]: pgmap v9963: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:40:36.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:35 smithi167 ceph-mon[37140]: pgmap v9963: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:40:37.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:37 smithi138 ceph-mon[36954]: pgmap v9964: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:40:37.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:37 smithi100 ceph-mon[32599]: pgmap v9964: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:40:38.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:37 smithi167 ceph-mon[37140]: pgmap v9964: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:40:39.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:39 smithi138 ceph-mon[36954]: pgmap v9965: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:40:39.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:39 smithi100 ceph-mon[32599]: pgmap v9965: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:40:40.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:39 smithi167 ceph-mon[37140]: pgmap v9965: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:40:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:40 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:40:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:40 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:40:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:40 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:40:41.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:41 smithi138 ceph-mon[36954]: pgmap v9966: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:40:41.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:40:41.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:40:41.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:40:41.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:41 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:40:41.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:41 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T18:40:41.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:41 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T18:40:41.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:41 smithi100 ceph-mon[32599]: pgmap v9966: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:40:41.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:40:41.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:40:41.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:40:41.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:41 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:40:41.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:41 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T18:40:41.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:41 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T18:40:42.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:41 smithi167 ceph-mon[37140]: pgmap v9966: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:40:42.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:40:42.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:40:42.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:40:42.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:41 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:40:42.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:41 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T18:40:42.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:41 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T18:40:43.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:43 smithi138 ceph-mon[36954]: pgmap v9967: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:40:43.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:40:43.755+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:40:43.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:40:43.756+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:40:43.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:40:43.756+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:40:43.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:40:43.756+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:40:43.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:40:43.756+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:40:43.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:43 smithi167 ceph-mon[37140]: pgmap v9967: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:40:43.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:43 smithi100 ceph-mon[32599]: pgmap v9967: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:40:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:40:45.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:40:45.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:40:45.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:40:45.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:40:45.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:40:45.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:44 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T18:40:45.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:44 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T18:40:45.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:40:45.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:40:45.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:40:45.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:40:45.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:40:45.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:40:45.193 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:44 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T18:40:45.193 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:44 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T18:40:45.193 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:40:45.193 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:40:45.193 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:40:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:40:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:40:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:40:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:40:45.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:40:45.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:40:45.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:44 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T18:40:45.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:44 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T18:40:45.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:40:45.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:40:45.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:40:45.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:40:45.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:40:45.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:40:45.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:44 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T18:40:45.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:44 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T18:40:45.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:40:45.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:40:45.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:40:45.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:40:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:40:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:40:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:40:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:40:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:40:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:44 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T18:40:45.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:44 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T18:40:45.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:40:45.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:40:45.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:40:45.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:40:45.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:40:45.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:40:45.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:44 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T18:40:45.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:44 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T18:40:45.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:40:45.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:40:45.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:40:46.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:46 smithi167 ceph-mon[37140]: pgmap v9968: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:40:46.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:46 smithi138 ceph-mon[36954]: pgmap v9968: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:40:46.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:46 smithi100 ceph-mon[32599]: pgmap v9968: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:40:48.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:48 smithi167 ceph-mon[37140]: pgmap v9969: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:40:48.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:48 smithi138 ceph-mon[36954]: pgmap v9969: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:40:48.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:48 smithi100 ceph-mon[32599]: pgmap v9969: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:40:50.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:50 smithi167 ceph-mon[37140]: pgmap v9970: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:40:50.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:50 smithi138 ceph-mon[36954]: pgmap v9970: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:40:50.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:50 smithi100 ceph-mon[32599]: pgmap v9970: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:40:52.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:52 smithi167 ceph-mon[37140]: pgmap v9971: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:40:52.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:52 smithi138 ceph-mon[36954]: pgmap v9971: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:40:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:52 smithi100 ceph-mon[32599]: pgmap v9971: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:40:54.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:54 smithi138 ceph-mon[36954]: pgmap v9972: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:40:54.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:54 smithi167 ceph-mon[37140]: pgmap v9972: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:40:54.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:54 smithi100 ceph-mon[32599]: pgmap v9972: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:40:56.443 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:56 smithi138 ceph-mon[36954]: pgmap v9973: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:40:56.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:56 smithi100 ceph-mon[32599]: pgmap v9973: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:40:56.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:56 smithi167 ceph-mon[37140]: pgmap v9973: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:40:58.297 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:40:58 smithi100 ceph-mon[32599]: pgmap v9974: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:40:58.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:40:58 smithi138 ceph-mon[36954]: pgmap v9974: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:40:58.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:40:58 smithi167 ceph-mon[37140]: pgmap v9974: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:40:58.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:40:58.772+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:40:58.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:40:58.772+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:40:58.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:40:58.772+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:40:58.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:40:58.772+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:40:58.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:40:58.772+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:41:00.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:41:00 smithi138 ceph-mon[36954]: pgmap v9975: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:41:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:41:00 smithi100 ceph-mon[32599]: pgmap v9975: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:41:00.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:41:00 smithi167 ceph-mon[37140]: pgmap v9975: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:41:02.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:41:02 smithi138 ceph-mon[36954]: pgmap v9976: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:41:02.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:41:02 smithi100 ceph-mon[32599]: pgmap v9976: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:41:02.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:41:02 smithi167 ceph-mon[37140]: pgmap v9976: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:41:04.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:41:04 smithi138 ceph-mon[36954]: pgmap v9977: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:41:04.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:41:04 smithi100 ceph-mon[32599]: pgmap v9977: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:41:04.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:41:04 smithi167 ceph-mon[37140]: pgmap v9977: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:41:06.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:41:06 smithi138 ceph-mon[36954]: pgmap v9978: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:41:06.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:41:06 smithi100 ceph-mon[32599]: pgmap v9978: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:41:06.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:41:06 smithi167 ceph-mon[37140]: pgmap v9978: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:41:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:41:08 smithi138 ceph-mon[36954]: pgmap v9979: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:41:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:41:08 smithi100 ceph-mon[32599]: pgmap v9979: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:41:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:41:08 smithi167 ceph-mon[37140]: pgmap v9979: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:41:09.341 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:41:09 smithi138 ceph-mon[36954]: osdmap e670: 12 total, 12 up, 12 in 2024-07-26T18:41:09.424 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:41:09 smithi167 ceph-mon[37140]: osdmap e670: 12 total, 12 up, 12 in 2024-07-26T18:41:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:41:09 smithi100 ceph-mon[32599]: osdmap e670: 12 total, 12 up, 12 in 2024-07-26T18:41:10.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:41:10 smithi138 ceph-mon[36954]: pgmap v9981: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:41:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:41:10 smithi100 ceph-mon[32599]: pgmap v9981: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:41:10.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:41:10 smithi167 ceph-mon[37140]: pgmap v9981: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:41:12.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:41:12 smithi138 ceph-mon[36954]: pgmap v9982: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:41:12.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:41:12 smithi100 ceph-mon[32599]: pgmap v9982: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:41:12.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:41:12 smithi167 ceph-mon[37140]: pgmap v9982: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:41:13.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:41:13.782+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:41:13.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:41:13.783+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:41:13.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:41:13.783+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:41:13.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:41:13.783+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:41:13.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:41:13.783+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:41:14.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:41:14 smithi138 ceph-mon[36954]: pgmap v9983: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:41:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:41:14 smithi100 ceph-mon[32599]: pgmap v9983: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:41:14.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:41:14 smithi167 ceph-mon[37140]: pgmap v9983: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:41:16.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:41:16 smithi138 ceph-mon[36954]: pgmap v9984: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:41:16.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:41:16 smithi100 ceph-mon[32599]: pgmap v9984: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:41:16.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:41:16 smithi167 ceph-mon[37140]: pgmap v9984: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:41:18.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:41:18 smithi138 ceph-mon[36954]: pgmap v9985: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:41:18.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:41:18 smithi100 ceph-mon[32599]: pgmap v9985: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:41:18.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:41:18 smithi167 ceph-mon[37140]: pgmap v9985: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:41:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:41:19 smithi138 ceph-mon[36954]: pgmap v9986: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 196 B/s wr, 6 op/s 2024-07-26T18:41:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:41:19 smithi138 ceph-mon[36954]: osdmap e671: 12 total, 12 up, 12 in 2024-07-26T18:41:19.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:41:19 smithi100 ceph-mon[32599]: pgmap v9986: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 196 B/s wr, 6 op/s 2024-07-26T18:41:19.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:41:19 smithi100 ceph-mon[32599]: osdmap e671: 12 total, 12 up, 12 in 2024-07-26T18:41:20.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:41:19 smithi167 ceph-mon[37140]: pgmap v9986: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 196 B/s wr, 6 op/s 2024-07-26T18:41:20.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:41:19 smithi167 ceph-mon[37140]: osdmap e671: 12 total, 12 up, 12 in 2024-07-26T18:41:21.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:41:21 smithi138 ceph-mon[36954]: pgmap v9988: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:41:21.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:41:21 smithi100 ceph-mon[32599]: pgmap v9988: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:41:22.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:41:21 smithi167 ceph-mon[37140]: pgmap v9988: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:41:23.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:41:23 smithi138 ceph-mon[36954]: pgmap v9989: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:41:23.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:41:23 smithi100 ceph-mon[32599]: pgmap v9989: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:41:24.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:41:23 smithi167 ceph-mon[37140]: pgmap v9989: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:41:25.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:41:25 smithi138 ceph-mon[36954]: pgmap v9990: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:41:25.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:41:25 smithi100 ceph-mon[32599]: pgmap v9990: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:41:26.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:41:25 smithi167 ceph-mon[37140]: pgmap v9990: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:41:27.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:41:27 smithi138 ceph-mon[36954]: pgmap v9991: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:41:27.999 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:41:27 smithi100 ceph-mon[32599]: pgmap v9991: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:41:28.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:41:27 smithi167 ceph-mon[37140]: pgmap v9991: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:41:28.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:41:28.787+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:41:28.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:41:28.788+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:41:28.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:41:28.788+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:41:28.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:41:28.788+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:41:28.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:41:28.788+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:41:29.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:41:29 smithi138 ceph-mon[36954]: pgmap v9992: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:41:29.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:41:29 smithi100 ceph-mon[32599]: pgmap v9992: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:41:30.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:41:29 smithi167 ceph-mon[37140]: pgmap v9992: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:41:31.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:41:31 smithi138 ceph-mon[36954]: pgmap v9993: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:41:31.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:41:31 smithi100 ceph-mon[32599]: pgmap v9993: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:41:32.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:41:31 smithi167 ceph-mon[37140]: pgmap v9993: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:41:33.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:41:33 smithi138 ceph-mon[36954]: pgmap v9994: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:41:33.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:41:33 smithi100 ceph-mon[32599]: pgmap v9994: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:41:34.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:41:33 smithi167 ceph-mon[37140]: pgmap v9994: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:41:35.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:41:35 smithi138 ceph-mon[36954]: pgmap v9995: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:41:35.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:41:35 smithi100 ceph-mon[32599]: pgmap v9995: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:41:36.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:41:35 smithi167 ceph-mon[37140]: pgmap v9995: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:41:37.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:41:37 smithi138 ceph-mon[36954]: pgmap v9996: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:41:37.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:41:37 smithi100 ceph-mon[32599]: pgmap v9996: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:41:38.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:41:37 smithi167 ceph-mon[37140]: pgmap v9996: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:41:39.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:41:39 smithi138 ceph-mon[36954]: pgmap v9997: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:41:39.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:41:39 smithi100 ceph-mon[32599]: pgmap v9997: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:41:40.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:41:39 smithi167 ceph-mon[37140]: pgmap v9997: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:41:41.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:41:41 smithi138 ceph-mon[36954]: pgmap v9998: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:41:41.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:41:41 smithi100 ceph-mon[32599]: pgmap v9998: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:41:42.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:41:41 smithi167 ceph-mon[37140]: pgmap v9998: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:41:43.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:41:43.795+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:41:43.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:41:43.796+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:41:43.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:41:43.796+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:41:43.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:41:43.796+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:41:43.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:41:43.796+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:41:43.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:41:43 smithi138 ceph-mon[36954]: pgmap v9999: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:41:43.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:41:43 smithi100 ceph-mon[32599]: pgmap v9999: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:41:44.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:41:43 smithi167 ceph-mon[37140]: pgmap v9999: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:41:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:41:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:41:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:41:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:41:44.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:41:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:41:44.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:41:44 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:41:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:41:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:41:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:41:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:41:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:41:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:41:44.998 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:41:44 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:41:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:41:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:41:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:41:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:41:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:41:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:41:45.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:41:44 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:41:45.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:41:45 smithi138 ceph-mon[36954]: pgmap v10000: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:41:45.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:41:45 smithi100 ceph-mon[32599]: pgmap v10000: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:41:46.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:41:45 smithi167 ceph-mon[37140]: pgmap v10000: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:41:47.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:41:47 smithi138 ceph-mon[36954]: pgmap v10001: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:41:47.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:41:47 smithi100 ceph-mon[32599]: pgmap v10001: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:41:48.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:41:47 smithi167 ceph-mon[37140]: pgmap v10001: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:41:49.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:41:49 smithi138 ceph-mon[36954]: pgmap v10002: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:41:49.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:41:49 smithi100 ceph-mon[32599]: pgmap v10002: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:41:50.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:41:49 smithi167 ceph-mon[37140]: pgmap v10002: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:41:51.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:41:51 smithi138 ceph-mon[36954]: pgmap v10003: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:41:51.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:41:51 smithi100 ceph-mon[32599]: pgmap v10003: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:41:52.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:41:51 smithi167 ceph-mon[37140]: pgmap v10003: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:41:53.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:41:53 smithi138 ceph-mon[36954]: pgmap v10004: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:41:53.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:41:53 smithi100 ceph-mon[32599]: pgmap v10004: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:41:54.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:41:53 smithi167 ceph-mon[37140]: pgmap v10004: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:41:55.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:41:55 smithi138 ceph-mon[36954]: pgmap v10005: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:41:55.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:41:55 smithi100 ceph-mon[32599]: pgmap v10005: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:41:56.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:41:55 smithi167 ceph-mon[37140]: pgmap v10005: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:41:57.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:41:57 smithi138 ceph-mon[36954]: pgmap v10006: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:41:57.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:41:57 smithi100 ceph-mon[32599]: pgmap v10006: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:41:58.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:41:57 smithi167 ceph-mon[37140]: pgmap v10006: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:41:58.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:41:58.811+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:41:58.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:41:58.812+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:41:58.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:41:58.812+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:41:58.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:41:58.812+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:41:58.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:41:58.812+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:41:59.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:41:59 smithi138 ceph-mon[36954]: pgmap v10007: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:41:59.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:41:59 smithi100 ceph-mon[32599]: pgmap v10007: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:42:00.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:41:59 smithi167 ceph-mon[37140]: pgmap v10007: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:42:01.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:42:01 smithi138 ceph-mon[36954]: pgmap v10008: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:42:01.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:42:01 smithi100 ceph-mon[32599]: pgmap v10008: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:42:02.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:42:01 smithi167 ceph-mon[37140]: pgmap v10008: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:42:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:42:03 smithi138 ceph-mon[36954]: pgmap v10009: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:42:03.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:42:03 smithi100 ceph-mon[32599]: pgmap v10009: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:42:04.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:42:03 smithi167 ceph-mon[37140]: pgmap v10009: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:42:05.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:42:05.517+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T18:42:05.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:42:05.517+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c003690 con 0x7fe5a81087a0 2024-07-26T18:42:05.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:42:05.518+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 138 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe5a008c070 con 0x7fe5a81087a0 2024-07-26T18:42:05.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:42:05.518+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 138 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T18:42:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:42:05 smithi138 ceph-mon[36954]: pgmap v10010: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:42:05.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:42:05 smithi100 ceph-mon[32599]: pgmap v10010: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:42:06.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:42:05 smithi167 ceph-mon[37140]: pgmap v10010: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:42:07.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:42:07 smithi138 ceph-mon[36954]: pgmap v10011: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:42:07.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:42:07 smithi100 ceph-mon[32599]: pgmap v10011: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:42:08.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:42:07 smithi167 ceph-mon[37140]: pgmap v10011: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:42:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:42:08 smithi138 ceph-mon[36954]: osdmap e672: 12 total, 12 up, 12 in 2024-07-26T18:42:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:42:08 smithi100 ceph-mon[32599]: osdmap e672: 12 total, 12 up, 12 in 2024-07-26T18:42:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:42:08 smithi167 ceph-mon[37140]: osdmap e672: 12 total, 12 up, 12 in 2024-07-26T18:42:09.924 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:42:09 smithi167 ceph-mon[37140]: pgmap v10013: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:42:09.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:42:09 smithi138 ceph-mon[36954]: pgmap v10013: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:42:09.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:42:09 smithi100 ceph-mon[32599]: pgmap v10013: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:42:11.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:42:11 smithi138 ceph-mon[36954]: pgmap v10014: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:42:11.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:42:11 smithi100 ceph-mon[32599]: pgmap v10014: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:42:12.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:42:11 smithi167 ceph-mon[37140]: pgmap v10014: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:42:13.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:42:13.826+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:42:13.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:42:13.826+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:42:13.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:42:13.826+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:42:13.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:42:13.826+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:42:13.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:42:13.826+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:42:13.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:42:13 smithi138 ceph-mon[36954]: pgmap v10015: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:42:13.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:42:13 smithi100 ceph-mon[32599]: pgmap v10015: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:42:14.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:42:13 smithi167 ceph-mon[37140]: pgmap v10015: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:42:15.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:42:15 smithi138 ceph-mon[36954]: pgmap v10016: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:42:15.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:42:15 smithi100 ceph-mon[32599]: pgmap v10016: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:42:16.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:42:15 smithi167 ceph-mon[37140]: pgmap v10016: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:42:17.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:42:17 smithi138 ceph-mon[36954]: pgmap v10017: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:42:17.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:42:17 smithi100 ceph-mon[32599]: pgmap v10017: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:42:18.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:42:17 smithi167 ceph-mon[37140]: pgmap v10017: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:42:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:42:19 smithi138 ceph-mon[36954]: pgmap v10018: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 189 B/s wr, 5 op/s 2024-07-26T18:42:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:42:19 smithi138 ceph-mon[36954]: osdmap e673: 12 total, 12 up, 12 in 2024-07-26T18:42:19.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:42:19 smithi100 ceph-mon[32599]: pgmap v10018: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 189 B/s wr, 5 op/s 2024-07-26T18:42:19.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:42:19 smithi100 ceph-mon[32599]: osdmap e673: 12 total, 12 up, 12 in 2024-07-26T18:42:20.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:42:19 smithi167 ceph-mon[37140]: pgmap v10018: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 189 B/s wr, 5 op/s 2024-07-26T18:42:20.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:42:19 smithi167 ceph-mon[37140]: osdmap e673: 12 total, 12 up, 12 in 2024-07-26T18:42:21.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:42:21 smithi138 ceph-mon[36954]: pgmap v10020: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:42:21.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:42:21 smithi100 ceph-mon[32599]: pgmap v10020: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:42:22.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:42:21 smithi167 ceph-mon[37140]: pgmap v10020: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:42:23.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:42:23 smithi138 ceph-mon[36954]: pgmap v10021: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:42:23.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:42:23 smithi100 ceph-mon[32599]: pgmap v10021: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:42:24.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:42:23 smithi167 ceph-mon[37140]: pgmap v10021: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:42:25.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:42:25 smithi138 ceph-mon[36954]: pgmap v10022: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:42:25.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:42:25 smithi100 ceph-mon[32599]: pgmap v10022: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:42:26.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:42:25 smithi167 ceph-mon[37140]: pgmap v10022: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:42:27.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:42:27 smithi138 ceph-mon[36954]: pgmap v10023: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:42:27.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:42:27 smithi100 ceph-mon[32599]: pgmap v10023: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:42:28.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:42:27 smithi167 ceph-mon[37140]: pgmap v10023: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:42:28.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:42:28.841+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:42:28.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:42:28.842+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:42:28.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:42:28.842+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:42:28.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:42:28.842+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:42:28.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:42:28.842+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:42:29.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:42:29 smithi138 ceph-mon[36954]: pgmap v10024: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:42:29.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:42:29 smithi100 ceph-mon[32599]: pgmap v10024: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:42:30.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:42:29 smithi167 ceph-mon[37140]: pgmap v10024: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:42:31.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:42:31 smithi138 ceph-mon[36954]: pgmap v10025: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:42:31.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:42:31 smithi100 ceph-mon[32599]: pgmap v10025: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:42:32.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:42:31 smithi167 ceph-mon[37140]: pgmap v10025: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:42:33.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:42:33 smithi138 ceph-mon[36954]: pgmap v10026: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:42:33.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:42:33 smithi100 ceph-mon[32599]: pgmap v10026: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:42:34.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:42:33 smithi167 ceph-mon[37140]: pgmap v10026: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:42:35.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:42:35 smithi138 ceph-mon[36954]: pgmap v10027: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:42:35.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:42:35 smithi100 ceph-mon[32599]: pgmap v10027: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:42:36.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:42:35 smithi167 ceph-mon[37140]: pgmap v10027: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:42:37.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:42:37 smithi138 ceph-mon[36954]: pgmap v10028: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:42:37.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:42:37 smithi100 ceph-mon[32599]: pgmap v10028: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:42:38.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:42:37 smithi167 ceph-mon[37140]: pgmap v10028: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:42:39.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:42:39 smithi138 ceph-mon[36954]: pgmap v10029: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:42:39.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:42:39 smithi100 ceph-mon[32599]: pgmap v10029: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:42:40.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:42:39 smithi167 ceph-mon[37140]: pgmap v10029: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:42:41.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:42:41 smithi138 ceph-mon[36954]: pgmap v10030: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:42:41.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:42:41 smithi100 ceph-mon[32599]: pgmap v10030: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:42:42.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:42:41 smithi167 ceph-mon[37140]: pgmap v10030: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:42:43.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:42:43.852+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:42:43.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:42:43.852+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:42:43.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:42:43.852+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:42:43.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:42:43.852+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:42:43.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:42:43.852+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:42:43.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:42:43 smithi138 ceph-mon[36954]: pgmap v10031: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:42:43.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:42:43 smithi100 ceph-mon[32599]: pgmap v10031: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:42:44.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:42:43 smithi167 ceph-mon[37140]: pgmap v10031: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:42:45.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:42:45 smithi138 ceph-mon[36954]: pgmap v10032: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:42:45.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:42:45 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:42:45.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:42:45 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:42:45.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:42:45 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:42:45.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:42:45 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:42:45.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:42:45 smithi100 ceph-mon[32599]: pgmap v10032: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:42:45.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:42:45 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:42:45.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:42:45 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:42:45.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:42:45 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:42:45.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:42:45 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:42:46.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:42:45 smithi167 ceph-mon[37140]: pgmap v10032: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:42:46.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:42:45 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:42:46.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:42:45 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:42:46.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:42:45 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:42:46.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:42:45 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:42:47.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:42:47 smithi138 ceph-mon[36954]: pgmap v10033: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:42:47.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:42:47 smithi100 ceph-mon[32599]: pgmap v10033: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:42:48.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:42:47 smithi167 ceph-mon[37140]: pgmap v10033: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:42:49.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:42:49 smithi138 ceph-mon[36954]: pgmap v10034: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:42:49.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:42:49 smithi100 ceph-mon[32599]: pgmap v10034: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:42:50.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:42:49 smithi167 ceph-mon[37140]: pgmap v10034: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:42:51.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:42:51 smithi138 ceph-mon[36954]: pgmap v10035: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:42:51.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:42:51 smithi100 ceph-mon[32599]: pgmap v10035: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:42:52.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:42:51 smithi167 ceph-mon[37140]: pgmap v10035: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:42:53.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:42:53 smithi138 ceph-mon[36954]: pgmap v10036: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:42:53.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:42:53 smithi100 ceph-mon[32599]: pgmap v10036: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:42:54.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:42:53 smithi167 ceph-mon[37140]: pgmap v10036: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:42:55.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:42:55 smithi138 ceph-mon[36954]: pgmap v10037: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:42:55.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:42:55 smithi100 ceph-mon[32599]: pgmap v10037: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:42:56.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:42:55 smithi167 ceph-mon[37140]: pgmap v10037: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:42:57.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:42:57 smithi138 ceph-mon[36954]: pgmap v10038: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:42:57.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:42:57 smithi100 ceph-mon[32599]: pgmap v10038: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:42:58.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:42:57 smithi167 ceph-mon[37140]: pgmap v10038: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:42:58.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:42:58.868+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:42:58.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:42:58.869+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:42:58.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:42:58.869+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:42:58.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:42:58.869+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:42:58.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:42:58.869+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:42:59.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:42:59 smithi138 ceph-mon[36954]: pgmap v10039: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:42:59.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:42:59 smithi100 ceph-mon[32599]: pgmap v10039: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:43:00.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:42:59 smithi167 ceph-mon[37140]: pgmap v10039: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:43:01.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:01 smithi138 ceph-mon[36954]: pgmap v10040: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:43:01.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:01 smithi100 ceph-mon[32599]: pgmap v10040: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:43:02.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:01 smithi167 ceph-mon[37140]: pgmap v10040: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:43:03.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:03 smithi138 ceph-mon[36954]: pgmap v10041: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:43:03.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:03 smithi100 ceph-mon[32599]: pgmap v10041: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:43:04.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:03 smithi167 ceph-mon[37140]: pgmap v10041: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:43:05.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:05 smithi138 ceph-mon[36954]: pgmap v10042: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:43:05.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:05 smithi100 ceph-mon[32599]: pgmap v10042: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:43:06.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:05 smithi167 ceph-mon[37140]: pgmap v10042: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:43:07.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:07 smithi138 ceph-mon[36954]: pgmap v10043: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:43:07.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:07 smithi100 ceph-mon[32599]: pgmap v10043: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:43:08.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:07 smithi167 ceph-mon[37140]: pgmap v10043: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:43:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:08 smithi138 ceph-mon[36954]: osdmap e674: 12 total, 12 up, 12 in 2024-07-26T18:43:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:08 smithi100 ceph-mon[32599]: osdmap e674: 12 total, 12 up, 12 in 2024-07-26T18:43:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:08 smithi167 ceph-mon[37140]: osdmap e674: 12 total, 12 up, 12 in 2024-07-26T18:43:09.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:09 smithi138 ceph-mon[36954]: pgmap v10045: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:43:09.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:09 smithi100 ceph-mon[32599]: pgmap v10045: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:43:10.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:09 smithi167 ceph-mon[37140]: pgmap v10045: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:43:11.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:11 smithi138 ceph-mon[36954]: pgmap v10046: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:43:11.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:11 smithi100 ceph-mon[32599]: pgmap v10046: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:43:12.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:11 smithi167 ceph-mon[37140]: pgmap v10046: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:43:13.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:43:13.884+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:43:13.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:43:13.885+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:43:13.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:43:13.885+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:43:13.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:43:13.885+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:43:13.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:43:13.885+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:43:13.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:13 smithi138 ceph-mon[36954]: pgmap v10047: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:43:13.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:13 smithi100 ceph-mon[32599]: pgmap v10047: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:43:14.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:13 smithi167 ceph-mon[37140]: pgmap v10047: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:43:15.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:15 smithi100 ceph-mon[32599]: pgmap v10048: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:43:16.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:15 smithi167 ceph-mon[37140]: pgmap v10048: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:43:16.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:15 smithi138 ceph-mon[36954]: pgmap v10048: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:43:17.999 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:17 smithi100 ceph-mon[32599]: pgmap v10049: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:43:18.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:17 smithi167 ceph-mon[37140]: pgmap v10049: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:43:18.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:17 smithi138 ceph-mon[36954]: pgmap v10049: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:43:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:19 smithi138 ceph-mon[36954]: pgmap v10050: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 189 B/s wr, 5 op/s 2024-07-26T18:43:20.075 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:19 smithi138 ceph-mon[36954]: osdmap e675: 12 total, 12 up, 12 in 2024-07-26T18:43:20.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:19 smithi167 ceph-mon[37140]: pgmap v10050: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 189 B/s wr, 5 op/s 2024-07-26T18:43:20.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:19 smithi167 ceph-mon[37140]: osdmap e675: 12 total, 12 up, 12 in 2024-07-26T18:43:20.077 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:19 smithi100 ceph-mon[32599]: pgmap v10050: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 189 B/s wr, 5 op/s 2024-07-26T18:43:20.077 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:19 smithi100 ceph-mon[32599]: osdmap e675: 12 total, 12 up, 12 in 2024-07-26T18:43:21.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:21 smithi138 ceph-mon[36954]: pgmap v10052: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:43:21.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:21 smithi100 ceph-mon[32599]: pgmap v10052: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:43:22.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:21 smithi167 ceph-mon[37140]: pgmap v10052: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:43:23.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:23 smithi138 ceph-mon[36954]: pgmap v10053: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:43:23.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:23 smithi100 ceph-mon[32599]: pgmap v10053: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:43:24.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:23 smithi167 ceph-mon[37140]: pgmap v10053: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:43:25.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:25 smithi138 ceph-mon[36954]: pgmap v10054: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:43:25.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:25 smithi100 ceph-mon[32599]: pgmap v10054: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:43:26.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:25 smithi167 ceph-mon[37140]: pgmap v10054: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:43:27.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:27 smithi138 ceph-mon[36954]: pgmap v10055: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:43:27.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:27 smithi100 ceph-mon[32599]: pgmap v10055: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:43:28.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:27 smithi167 ceph-mon[37140]: pgmap v10055: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:43:28.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:43:28.894+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:43:28.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:43:28.895+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:43:28.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:43:28.895+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:43:28.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:43:28.895+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:43:28.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:43:28.895+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:43:29.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:29 smithi138 ceph-mon[36954]: pgmap v10056: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:43:29.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:29 smithi100 ceph-mon[32599]: pgmap v10056: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:43:30.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:29 smithi167 ceph-mon[37140]: pgmap v10056: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:43:31.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:31 smithi138 ceph-mon[36954]: pgmap v10057: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:43:31.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:31 smithi100 ceph-mon[32599]: pgmap v10057: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:43:32.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:31 smithi167 ceph-mon[37140]: pgmap v10057: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:43:33.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:33 smithi138 ceph-mon[36954]: pgmap v10058: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:43:33.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:33 smithi100 ceph-mon[32599]: pgmap v10058: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:43:34.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:33 smithi167 ceph-mon[37140]: pgmap v10058: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:43:35.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:35 smithi138 ceph-mon[36954]: pgmap v10059: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:43:35.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:35 smithi100 ceph-mon[32599]: pgmap v10059: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:43:36.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:35 smithi167 ceph-mon[37140]: pgmap v10059: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:43:37.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:37 smithi138 ceph-mon[36954]: pgmap v10060: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:43:37.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:37 smithi100 ceph-mon[32599]: pgmap v10060: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:43:38.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:37 smithi167 ceph-mon[37140]: pgmap v10060: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:43:39.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:39 smithi138 ceph-mon[36954]: pgmap v10061: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:43:39.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:39 smithi100 ceph-mon[32599]: pgmap v10061: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:43:40.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:39 smithi167 ceph-mon[37140]: pgmap v10061: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:43:41.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:41 smithi138 ceph-mon[36954]: pgmap v10062: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:43:41.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:41 smithi100 ceph-mon[32599]: pgmap v10062: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:43:42.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:41 smithi167 ceph-mon[37140]: pgmap v10062: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:43:43.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:43:43.903+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:43:43.905 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:43 smithi100 ceph-mon[32599]: pgmap v10063: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:43:43.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:43:43.904+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:43:43.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:43:43.904+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:43:43.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:43:43.904+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:43:43.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:43:43.904+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:43:43.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:43 smithi138 ceph-mon[36954]: pgmap v10063: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:43:44.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:43 smithi167 ceph-mon[37140]: pgmap v10063: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:43:45.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:45 smithi138 ceph-mon[36954]: pgmap v10064: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:43:45.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:45 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:43:45.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:45 smithi100 ceph-mon[32599]: pgmap v10064: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:43:45.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:45 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:43:46.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:45 smithi167 ceph-mon[37140]: pgmap v10064: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:43:46.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:45 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:43:47.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:47 smithi100 ceph-mon[32599]: pgmap v10065: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:43:47.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:47 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:43:47.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:47 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:43:47.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:47 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:43:47.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:47 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:43:47.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:47 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:43:47.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:47 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:43:47.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:47 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:43:47.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:47 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:43:47.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:47 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:43:48.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:47 smithi167 ceph-mon[37140]: pgmap v10065: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:43:48.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:47 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:43:48.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:47 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:43:48.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:47 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:43:48.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:47 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:43:48.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:47 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:43:48.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:47 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:43:48.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:47 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:43:48.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:47 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:43:48.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:47 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:43:48.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:47 smithi138 ceph-mon[36954]: pgmap v10065: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:43:48.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:47 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:43:48.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:47 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:43:48.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:47 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:43:48.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:47 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:43:48.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:47 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:43:48.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:47 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:43:48.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:47 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:43:48.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:47 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:43:48.193 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:47 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:43:49.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:49 smithi100 ceph-mon[32599]: pgmap v10066: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:43:50.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:49 smithi167 ceph-mon[37140]: pgmap v10066: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:43:50.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:49 smithi138 ceph-mon[36954]: pgmap v10066: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:43:51.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:51 smithi100 ceph-mon[32599]: pgmap v10067: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:43:52.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:51 smithi167 ceph-mon[37140]: pgmap v10067: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:43:52.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:51 smithi138 ceph-mon[36954]: pgmap v10067: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:43:53.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:53 smithi100 ceph-mon[32599]: pgmap v10068: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:43:54.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:53 smithi167 ceph-mon[37140]: pgmap v10068: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:43:54.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:53 smithi138 ceph-mon[36954]: pgmap v10068: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:43:55.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:55 smithi100 ceph-mon[32599]: pgmap v10069: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:43:56.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:55 smithi167 ceph-mon[37140]: pgmap v10069: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:43:56.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:55 smithi138 ceph-mon[36954]: pgmap v10069: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:43:57.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:57 smithi100 ceph-mon[32599]: pgmap v10070: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:43:58.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:57 smithi167 ceph-mon[37140]: pgmap v10070: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:43:58.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:57 smithi138 ceph-mon[36954]: pgmap v10070: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:43:58.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:43:58.919+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:43:58.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:43:58.920+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:43:58.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:43:58.920+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:43:58.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:43:58.920+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:43:58.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:43:58.920+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:43:59.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:43:59 smithi100 ceph-mon[32599]: pgmap v10071: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:44:00.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:43:59 smithi167 ceph-mon[37140]: pgmap v10071: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:44:00.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:43:59 smithi138 ceph-mon[36954]: pgmap v10071: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T18:44:01.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:01 smithi100 ceph-mon[32599]: pgmap v10072: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:44:02.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:01 smithi167 ceph-mon[37140]: pgmap v10072: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:44:02.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:01 smithi138 ceph-mon[36954]: pgmap v10072: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:44:03.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:03 smithi100 ceph-mon[32599]: pgmap v10073: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:44:04.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:03 smithi167 ceph-mon[37140]: pgmap v10073: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:44:04.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:03 smithi138 ceph-mon[36954]: pgmap v10073: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:44:05.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:05 smithi100 ceph-mon[32599]: pgmap v10074: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:44:06.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:05 smithi167 ceph-mon[37140]: pgmap v10074: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:44:06.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:05 smithi138 ceph-mon[36954]: pgmap v10074: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:44:07.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:07 smithi100 ceph-mon[32599]: pgmap v10075: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:44:08.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:07 smithi167 ceph-mon[37140]: pgmap v10075: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:44:08.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:07 smithi138 ceph-mon[36954]: pgmap v10075: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:44:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:08 smithi100 ceph-mon[32599]: osdmap e676: 12 total, 12 up, 12 in 2024-07-26T18:44:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:08 smithi167 ceph-mon[37140]: osdmap e676: 12 total, 12 up, 12 in 2024-07-26T18:44:09.091 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:08 smithi138 ceph-mon[36954]: osdmap e676: 12 total, 12 up, 12 in 2024-07-26T18:44:09.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:09 smithi100 ceph-mon[32599]: pgmap v10077: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:44:10.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:09 smithi167 ceph-mon[37140]: pgmap v10077: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:44:10.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:09 smithi138 ceph-mon[36954]: pgmap v10077: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:44:11.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:11 smithi100 ceph-mon[32599]: pgmap v10078: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T18:44:12.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:11 smithi167 ceph-mon[37140]: pgmap v10078: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T18:44:12.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:11 smithi138 ceph-mon[36954]: pgmap v10078: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T18:44:13.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:44:13.936+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:44:13.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:44:13.936+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:44:13.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:44:13.936+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:44:13.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:44:13.936+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:44:13.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:44:13.936+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:44:14.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:13 smithi167 ceph-mon[37140]: pgmap v10079: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:44:14.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:13 smithi138 ceph-mon[36954]: pgmap v10079: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:44:14.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:13 smithi100 ceph-mon[32599]: pgmap v10079: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:44:16.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:15 smithi167 ceph-mon[37140]: pgmap v10080: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:44:16.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:15 smithi138 ceph-mon[36954]: pgmap v10080: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:44:16.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:15 smithi100 ceph-mon[32599]: pgmap v10080: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:44:18.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:17 smithi167 ceph-mon[37140]: pgmap v10081: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:44:18.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:17 smithi138 ceph-mon[36954]: pgmap v10081: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:44:18.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:17 smithi100 ceph-mon[32599]: pgmap v10081: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:44:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:19 smithi138 ceph-mon[36954]: pgmap v10082: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 189 B/s wr, 5 op/s 2024-07-26T18:44:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:19 smithi138 ceph-mon[36954]: osdmap e677: 12 total, 12 up, 12 in 2024-07-26T18:44:19.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:19 smithi100 ceph-mon[32599]: pgmap v10082: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 189 B/s wr, 5 op/s 2024-07-26T18:44:19.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:19 smithi100 ceph-mon[32599]: osdmap e677: 12 total, 12 up, 12 in 2024-07-26T18:44:20.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:19 smithi167 ceph-mon[37140]: pgmap v10082: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 189 B/s wr, 5 op/s 2024-07-26T18:44:20.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:19 smithi167 ceph-mon[37140]: osdmap e677: 12 total, 12 up, 12 in 2024-07-26T18:44:21.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:21 smithi138 ceph-mon[36954]: pgmap v10084: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:44:21.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:21 smithi100 ceph-mon[32599]: pgmap v10084: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:44:22.028 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:21 smithi167 ceph-mon[37140]: pgmap v10084: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:44:23.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:23 smithi138 ceph-mon[36954]: pgmap v10085: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:44:23.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:23 smithi100 ceph-mon[32599]: pgmap v10085: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:44:24.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:23 smithi167 ceph-mon[37140]: pgmap v10085: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:44:25.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:25 smithi138 ceph-mon[36954]: pgmap v10086: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:44:25.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:25 smithi100 ceph-mon[32599]: pgmap v10086: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:44:26.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:25 smithi167 ceph-mon[37140]: pgmap v10086: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:44:27.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:27 smithi138 ceph-mon[36954]: pgmap v10087: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:44:27.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:27 smithi100 ceph-mon[32599]: pgmap v10087: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:44:28.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:27 smithi167 ceph-mon[37140]: pgmap v10087: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:44:28.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:44:28.939+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:44:28.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:44:28.940+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:44:28.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:44:28.940+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:44:28.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:44:28.940+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:44:28.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:44:28.940+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:44:29.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:29 smithi138 ceph-mon[36954]: pgmap v10088: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:44:29.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:29 smithi100 ceph-mon[32599]: pgmap v10088: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:44:30.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:29 smithi167 ceph-mon[37140]: pgmap v10088: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:44:31.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:31 smithi138 ceph-mon[36954]: pgmap v10089: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:44:31.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:31 smithi100 ceph-mon[32599]: pgmap v10089: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:44:32.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:31 smithi167 ceph-mon[37140]: pgmap v10089: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:44:33.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:33 smithi138 ceph-mon[36954]: pgmap v10090: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:44:33.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:33 smithi100 ceph-mon[32599]: pgmap v10090: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:44:34.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:33 smithi167 ceph-mon[37140]: pgmap v10090: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:44:35.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:35 smithi138 ceph-mon[36954]: pgmap v10091: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:44:35.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:35 smithi100 ceph-mon[32599]: pgmap v10091: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:44:36.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:35 smithi167 ceph-mon[37140]: pgmap v10091: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:44:37.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:37 smithi138 ceph-mon[36954]: pgmap v10092: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:44:37.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:37 smithi100 ceph-mon[32599]: pgmap v10092: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:44:38.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:37 smithi167 ceph-mon[37140]: pgmap v10092: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:44:39.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:39 smithi138 ceph-mon[36954]: pgmap v10093: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:44:39.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:39 smithi100 ceph-mon[32599]: pgmap v10093: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:44:40.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:39 smithi167 ceph-mon[37140]: pgmap v10093: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:44:41.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:41 smithi138 ceph-mon[36954]: pgmap v10094: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:44:41.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:41 smithi100 ceph-mon[32599]: pgmap v10094: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:44:42.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:41 smithi167 ceph-mon[37140]: pgmap v10094: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:44:43.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:43 smithi138 ceph-mon[36954]: pgmap v10095: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:44:43.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:44:43.956+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:44:43.957 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:43 smithi100 ceph-mon[32599]: pgmap v10095: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:44:43.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:44:43.958+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:44:43.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:44:43.958+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:44:43.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:44:43.958+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:44:43.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:44:43.958+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:44:44.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:43 smithi167 ceph-mon[37140]: pgmap v10095: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:44:45.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:45 smithi138 ceph-mon[36954]: pgmap v10096: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:44:45.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:45 smithi100 ceph-mon[32599]: pgmap v10096: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:44:46.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:45 smithi167 ceph-mon[37140]: pgmap v10096: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:44:47.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:47 smithi138 ceph-mon[36954]: pgmap v10097: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:44:47.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:47 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:44:47.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:47 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:44:47.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:47 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:44:47.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:47 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:44:47.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:47 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:44:47.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:47 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:44:47.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:47 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:44:47.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:47 smithi100 ceph-mon[32599]: pgmap v10097: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:44:47.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:47 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:44:47.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:47 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:44:47.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:47 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:44:47.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:47 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:44:47.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:47 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:44:47.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:47 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:44:47.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:47 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:44:48.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:47 smithi167 ceph-mon[37140]: pgmap v10097: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:44:48.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:47 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:44:48.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:47 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:44:48.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:47 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:44:48.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:47 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:44:48.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:47 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:44:48.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:47 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:44:48.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:47 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:44:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:48 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:44:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:48 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:44:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:48 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:44:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:48 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:44:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:48 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:44:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:48 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:44:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:48 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:44:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:48 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:44:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:48 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:44:50.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:50 smithi167 ceph-mon[37140]: pgmap v10098: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:44:50.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:50 smithi138 ceph-mon[36954]: pgmap v10098: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:44:50.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:50 smithi100 ceph-mon[32599]: pgmap v10098: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:44:52.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:52 smithi167 ceph-mon[37140]: pgmap v10099: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:44:52.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:52 smithi138 ceph-mon[36954]: pgmap v10099: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:44:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:52 smithi100 ceph-mon[32599]: pgmap v10099: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:44:54.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:54 smithi167 ceph-mon[37140]: pgmap v10100: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:44:54.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:54 smithi138 ceph-mon[36954]: pgmap v10100: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:44:54.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:54 smithi100 ceph-mon[32599]: pgmap v10100: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:44:56.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:56 smithi167 ceph-mon[37140]: pgmap v10101: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:44:56.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:56 smithi138 ceph-mon[36954]: pgmap v10101: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:44:56.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:56 smithi100 ceph-mon[32599]: pgmap v10101: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:44:58.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:44:58 smithi138 ceph-mon[36954]: pgmap v10102: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:44:58.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:44:58 smithi100 ceph-mon[32599]: pgmap v10102: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:44:58.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:44:58 smithi167 ceph-mon[37140]: pgmap v10102: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:44:58.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:44:58.960+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:44:58.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:44:58.960+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:44:58.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:44:58.960+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:44:58.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:44:58.960+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:44:58.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:44:58.960+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:45:00.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:45:00 smithi138 ceph-mon[36954]: pgmap v10103: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:45:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:45:00 smithi100 ceph-mon[32599]: pgmap v10103: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:45:00.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:45:00 smithi167 ceph-mon[37140]: pgmap v10103: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:45:02.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:45:02 smithi138 ceph-mon[36954]: pgmap v10104: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:45:02.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:45:02 smithi100 ceph-mon[32599]: pgmap v10104: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:45:02.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:45:02 smithi167 ceph-mon[37140]: pgmap v10104: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:45:04.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:45:04 smithi138 ceph-mon[36954]: pgmap v10105: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:45:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:45:04 smithi100 ceph-mon[32599]: pgmap v10105: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:45:04.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:45:04 smithi167 ceph-mon[37140]: pgmap v10105: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:45:05.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:45:05.543+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c004740 con 0x7fe5a81087a0 2024-07-26T18:45:05.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:45:05.543+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580006660 con 0x55c0ec887390 2024-07-26T18:45:05.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:45:05.544+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 139 ==== auth_reply(proto 2 0 (0) Success) ==== 612+0+0 (secure 0 0 0) 0x7fe580006660 con 0x55c0ec887390 2024-07-26T18:45:05.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:45:05.544+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T18:45:05.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:45:05.544+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 139 ==== auth_reply(proto 2 0 (0) Success) ==== 783+0+0 (secure 0 0 0) 0x7fe5a008c020 con 0x7fe5a81087a0 2024-07-26T18:45:05.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:45:05.544+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c006690 con 0x7fe5a81087a0 2024-07-26T18:45:05.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:45:05.544+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 140 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T18:45:05.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:45:05.544+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 140 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c006690 con 0x7fe5a81087a0 2024-07-26T18:45:06.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:45:06 smithi138 ceph-mon[36954]: pgmap v10106: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:45:06.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:45:06 smithi100 ceph-mon[32599]: pgmap v10106: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:45:06.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:45:06 smithi167 ceph-mon[37140]: pgmap v10106: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:45:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:45:08 smithi138 ceph-mon[36954]: pgmap v10107: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:45:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:45:08 smithi100 ceph-mon[32599]: pgmap v10107: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:45:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:45:08 smithi167 ceph-mon[37140]: pgmap v10107: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:45:09.341 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:45:09 smithi138 ceph-mon[36954]: osdmap e678: 12 total, 12 up, 12 in 2024-07-26T18:45:09.423 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:45:09 smithi167 ceph-mon[37140]: osdmap e678: 12 total, 12 up, 12 in 2024-07-26T18:45:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:45:09 smithi100 ceph-mon[32599]: osdmap e678: 12 total, 12 up, 12 in 2024-07-26T18:45:10.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:45:10 smithi138 ceph-mon[36954]: pgmap v10109: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:45:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:45:10 smithi100 ceph-mon[32599]: pgmap v10109: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:45:10.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:45:10 smithi167 ceph-mon[37140]: pgmap v10109: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:45:12.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:45:12 smithi138 ceph-mon[36954]: pgmap v10110: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T18:45:12.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:45:12 smithi100 ceph-mon[32599]: pgmap v10110: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T18:45:12.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:45:12 smithi167 ceph-mon[37140]: pgmap v10110: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T18:45:13.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:45:13.975+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:45:13.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:45:13.976+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:45:13.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:45:13.976+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:45:13.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:45:13.976+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:45:13.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:45:13.976+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:45:14.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:45:14 smithi100 ceph-mon[32599]: pgmap v10111: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:45:14.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:45:14 smithi138 ceph-mon[36954]: pgmap v10111: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:45:14.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:45:14 smithi167 ceph-mon[37140]: pgmap v10111: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:45:16.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:45:16 smithi138 ceph-mon[36954]: pgmap v10112: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:45:16.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:45:16 smithi100 ceph-mon[32599]: pgmap v10112: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:45:16.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:45:16 smithi167 ceph-mon[37140]: pgmap v10112: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:45:18.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:45:18 smithi138 ceph-mon[36954]: pgmap v10113: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:45:18.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:45:18 smithi100 ceph-mon[32599]: pgmap v10113: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:45:18.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:45:18 smithi167 ceph-mon[37140]: pgmap v10113: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:45:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:45:19 smithi138 ceph-mon[36954]: pgmap v10114: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 195 B/s wr, 6 op/s 2024-07-26T18:45:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:45:19 smithi138 ceph-mon[36954]: osdmap e679: 12 total, 12 up, 12 in 2024-07-26T18:45:19.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:45:19 smithi100 ceph-mon[32599]: pgmap v10114: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 195 B/s wr, 6 op/s 2024-07-26T18:45:19.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:45:19 smithi100 ceph-mon[32599]: osdmap e679: 12 total, 12 up, 12 in 2024-07-26T18:45:20.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:45:19 smithi167 ceph-mon[37140]: pgmap v10114: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 195 B/s wr, 6 op/s 2024-07-26T18:45:20.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:45:19 smithi167 ceph-mon[37140]: osdmap e679: 12 total, 12 up, 12 in 2024-07-26T18:45:21.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:45:21 smithi138 ceph-mon[36954]: pgmap v10116: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:45:21.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:45:21 smithi100 ceph-mon[32599]: pgmap v10116: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:45:22.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:45:21 smithi167 ceph-mon[37140]: pgmap v10116: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:45:23.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:45:23 smithi138 ceph-mon[36954]: pgmap v10117: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:45:23.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:45:23 smithi100 ceph-mon[32599]: pgmap v10117: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:45:24.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:45:23 smithi167 ceph-mon[37140]: pgmap v10117: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:45:25.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:45:25 smithi138 ceph-mon[36954]: pgmap v10118: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:45:25.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:45:25 smithi100 ceph-mon[32599]: pgmap v10118: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:45:26.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:45:25 smithi167 ceph-mon[37140]: pgmap v10118: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:45:27.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:45:27 smithi138 ceph-mon[36954]: pgmap v10119: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:45:27.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:45:27 smithi100 ceph-mon[32599]: pgmap v10119: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:45:28.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:45:27 smithi167 ceph-mon[37140]: pgmap v10119: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:45:28.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:45:28.991+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:45:28.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:45:28.992+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:45:28.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:45:28.992+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:45:28.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:45:28.992+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:45:28.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:45:28.992+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:45:29.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:45:29 smithi138 ceph-mon[36954]: pgmap v10120: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:45:29.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:45:29 smithi100 ceph-mon[32599]: pgmap v10120: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:45:30.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:45:29 smithi167 ceph-mon[37140]: pgmap v10120: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:45:31.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:45:31 smithi138 ceph-mon[36954]: pgmap v10121: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:45:31.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:45:31 smithi100 ceph-mon[32599]: pgmap v10121: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:45:32.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:45:31 smithi167 ceph-mon[37140]: pgmap v10121: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:45:33.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:45:33 smithi138 ceph-mon[36954]: pgmap v10122: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:45:33.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:45:33 smithi100 ceph-mon[32599]: pgmap v10122: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:45:34.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:45:33 smithi167 ceph-mon[37140]: pgmap v10122: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:45:35.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:45:35 smithi138 ceph-mon[36954]: pgmap v10123: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:45:35.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:45:35 smithi100 ceph-mon[32599]: pgmap v10123: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:45:36.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:45:35 smithi167 ceph-mon[37140]: pgmap v10123: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:45:37.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:45:37 smithi138 ceph-mon[36954]: pgmap v10124: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:45:37.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:45:37 smithi100 ceph-mon[32599]: pgmap v10124: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:45:38.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:45:37 smithi167 ceph-mon[37140]: pgmap v10124: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:45:39.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:45:39 smithi138 ceph-mon[36954]: pgmap v10125: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:45:39.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:45:39 smithi100 ceph-mon[32599]: pgmap v10125: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:45:40.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:45:39 smithi167 ceph-mon[37140]: pgmap v10125: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:45:41.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:45:41 smithi138 ceph-mon[36954]: pgmap v10126: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:45:41.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:45:41 smithi100 ceph-mon[32599]: pgmap v10126: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:45:42.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:45:41 smithi167 ceph-mon[37140]: pgmap v10126: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:45:43.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:45:43 smithi138 ceph-mon[36954]: pgmap v10127: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:45:43.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:45:43 smithi100 ceph-mon[32599]: pgmap v10127: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:45:44.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:45:44.007+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:45:44.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:45:44.008+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:45:44.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:45:44.008+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:45:44.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:45:44.008+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:45:44.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:45:44.008+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:45:44.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:45:43 smithi167 ceph-mon[37140]: pgmap v10127: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:45:45.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:45:45 smithi138 ceph-mon[36954]: pgmap v10128: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:45:45.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:45:45 smithi100 ceph-mon[32599]: pgmap v10128: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:45:46.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:45:45 smithi167 ceph-mon[37140]: pgmap v10128: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:45:47.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:45:47 smithi138 ceph-mon[36954]: pgmap v10129: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:45:47.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:45:47 smithi100 ceph-mon[32599]: pgmap v10129: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:45:48.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:45:47 smithi167 ceph-mon[37140]: pgmap v10129: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:45:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:45:48 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:45:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:45:48 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:45:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:45:48 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:45:48.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:45:48 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:45:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:45:48 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:45:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:45:48 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:45:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:45:48 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:45:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:45:48 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:45:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:45:48 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:45:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:45:48 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:45:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:45:48 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:45:49.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:45:48 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:45:49.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:45:49 smithi138 ceph-mon[36954]: pgmap v10130: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:45:49.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:45:49 smithi100 ceph-mon[32599]: pgmap v10130: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:45:50.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:45:49 smithi167 ceph-mon[37140]: pgmap v10130: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:45:51.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:45:51 smithi138 ceph-mon[36954]: pgmap v10131: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:45:51.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:45:51 smithi100 ceph-mon[32599]: pgmap v10131: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:45:52.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:45:51 smithi167 ceph-mon[37140]: pgmap v10131: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:45:53.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:45:53 smithi138 ceph-mon[36954]: pgmap v10132: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:45:53.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:45:53 smithi100 ceph-mon[32599]: pgmap v10132: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:45:54.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:45:53 smithi167 ceph-mon[37140]: pgmap v10132: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:45:55.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:45:55 smithi100 ceph-mon[32599]: pgmap v10133: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:45:56.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:45:55 smithi167 ceph-mon[37140]: pgmap v10133: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:45:56.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:45:55 smithi138 ceph-mon[36954]: pgmap v10133: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:45:57.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:45:57 smithi100 ceph-mon[32599]: pgmap v10134: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:45:58.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:45:57 smithi167 ceph-mon[37140]: pgmap v10134: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:45:58.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:45:57 smithi138 ceph-mon[36954]: pgmap v10134: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:45:59.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:45:59.023+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:45:59.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:45:59.024+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:45:59.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:45:59.024+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:45:59.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:45:59.024+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:45:59.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:45:59.024+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:45:59.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:45:59 smithi100 ceph-mon[32599]: pgmap v10135: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:46:00.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:45:59 smithi167 ceph-mon[37140]: pgmap v10135: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:46:00.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:45:59 smithi138 ceph-mon[36954]: pgmap v10135: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:46:01.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:46:01 smithi100 ceph-mon[32599]: pgmap v10136: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:46:02.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:46:01 smithi167 ceph-mon[37140]: pgmap v10136: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:46:02.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:46:01 smithi138 ceph-mon[36954]: pgmap v10136: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:46:03.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:46:03 smithi100 ceph-mon[32599]: pgmap v10137: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:46:04.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:46:03 smithi167 ceph-mon[37140]: pgmap v10137: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:46:04.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:46:03 smithi138 ceph-mon[36954]: pgmap v10137: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:46:05.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:46:05 smithi100 ceph-mon[32599]: pgmap v10138: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:46:06.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:46:05 smithi167 ceph-mon[37140]: pgmap v10138: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:46:06.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:46:05 smithi138 ceph-mon[36954]: pgmap v10138: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-07-26T18:46:07.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:46:07 smithi100 ceph-mon[32599]: pgmap v10139: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:46:08.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:46:07 smithi167 ceph-mon[37140]: pgmap v10139: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:46:08.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:46:07 smithi138 ceph-mon[36954]: pgmap v10139: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:46:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:46:08 smithi100 ceph-mon[32599]: osdmap e680: 12 total, 12 up, 12 in 2024-07-26T18:46:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:46:08 smithi167 ceph-mon[37140]: osdmap e680: 12 total, 12 up, 12 in 2024-07-26T18:46:09.091 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:46:08 smithi138 ceph-mon[36954]: osdmap e680: 12 total, 12 up, 12 in 2024-07-26T18:46:09.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:46:09 smithi100 ceph-mon[32599]: pgmap v10141: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:46:10.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:46:09 smithi167 ceph-mon[37140]: pgmap v10141: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:46:10.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:46:09 smithi138 ceph-mon[36954]: pgmap v10141: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:46:11.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:46:11 smithi100 ceph-mon[32599]: pgmap v10142: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T18:46:12.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:46:11 smithi167 ceph-mon[37140]: pgmap v10142: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T18:46:12.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:46:11 smithi138 ceph-mon[36954]: pgmap v10142: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T18:46:13.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:46:13 smithi100 ceph-mon[32599]: pgmap v10143: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:46:14.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:46:13 smithi167 ceph-mon[37140]: pgmap v10143: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:46:14.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:46:14.037+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:46:14.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:46:14.038+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:46:14.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:46:14.038+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:46:14.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:46:14.038+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:46:14.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:46:14.038+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:46:14.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:46:13 smithi138 ceph-mon[36954]: pgmap v10143: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:46:15.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:46:15 smithi100 ceph-mon[32599]: pgmap v10144: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:46:16.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:46:15 smithi167 ceph-mon[37140]: pgmap v10144: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:46:16.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:46:15 smithi138 ceph-mon[36954]: pgmap v10144: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:46:17.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:46:17 smithi100 ceph-mon[32599]: pgmap v10145: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:46:18.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:46:17 smithi167 ceph-mon[37140]: pgmap v10145: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:46:18.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:46:17 smithi138 ceph-mon[36954]: pgmap v10145: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:46:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:46:19 smithi138 ceph-mon[36954]: pgmap v10146: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 188 B/s wr, 5 op/s 2024-07-26T18:46:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:46:19 smithi138 ceph-mon[36954]: osdmap e681: 12 total, 12 up, 12 in 2024-07-26T18:46:19.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:46:19 smithi100 ceph-mon[32599]: pgmap v10146: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 188 B/s wr, 5 op/s 2024-07-26T18:46:19.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:46:19 smithi100 ceph-mon[32599]: osdmap e681: 12 total, 12 up, 12 in 2024-07-26T18:46:20.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:46:19 smithi167 ceph-mon[37140]: pgmap v10146: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 188 B/s wr, 5 op/s 2024-07-26T18:46:20.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:46:19 smithi167 ceph-mon[37140]: osdmap e681: 12 total, 12 up, 12 in 2024-07-26T18:46:21.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:46:21 smithi138 ceph-mon[36954]: pgmap v10148: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:46:21.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:46:21 smithi100 ceph-mon[32599]: pgmap v10148: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:46:22.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:46:21 smithi167 ceph-mon[37140]: pgmap v10148: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:46:23.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:46:23 smithi138 ceph-mon[36954]: pgmap v10149: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:46:23.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:46:23 smithi100 ceph-mon[32599]: pgmap v10149: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:46:24.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:46:23 smithi167 ceph-mon[37140]: pgmap v10149: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:46:25.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:46:25 smithi138 ceph-mon[36954]: pgmap v10150: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:46:25.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:46:25 smithi100 ceph-mon[32599]: pgmap v10150: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:46:26.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:46:25 smithi167 ceph-mon[37140]: pgmap v10150: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:46:27.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:46:27 smithi138 ceph-mon[36954]: pgmap v10151: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:46:27.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:46:27 smithi100 ceph-mon[32599]: pgmap v10151: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:46:28.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:46:27 smithi167 ceph-mon[37140]: pgmap v10151: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:46:29.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:46:29.053+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:46:29.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:46:29.054+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:46:29.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:46:29.054+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:46:29.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:46:29.054+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:46:29.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:46:29.054+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:46:29.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:46:29 smithi138 ceph-mon[36954]: pgmap v10152: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:46:29.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:46:29 smithi100 ceph-mon[32599]: pgmap v10152: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:46:30.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:46:29 smithi167 ceph-mon[37140]: pgmap v10152: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:46:31.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:46:31 smithi138 ceph-mon[36954]: pgmap v10153: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:46:31.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:46:31 smithi100 ceph-mon[32599]: pgmap v10153: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:46:32.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:46:31 smithi167 ceph-mon[37140]: pgmap v10153: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:46:33.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:46:33 smithi138 ceph-mon[36954]: pgmap v10154: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:46:33.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:46:33 smithi100 ceph-mon[32599]: pgmap v10154: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:46:34.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:46:33 smithi167 ceph-mon[37140]: pgmap v10154: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:46:35.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:46:35 smithi138 ceph-mon[36954]: pgmap v10155: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:46:35.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:46:35 smithi100 ceph-mon[32599]: pgmap v10155: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:46:36.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:46:35 smithi167 ceph-mon[37140]: pgmap v10155: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:46:37.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:46:37 smithi138 ceph-mon[36954]: pgmap v10156: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:46:37.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:46:37 smithi100 ceph-mon[32599]: pgmap v10156: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:46:38.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:46:37 smithi167 ceph-mon[37140]: pgmap v10156: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:46:39.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:46:39 smithi138 ceph-mon[36954]: pgmap v10157: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:46:39.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:46:39 smithi100 ceph-mon[32599]: pgmap v10157: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:46:40.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:46:39 smithi167 ceph-mon[37140]: pgmap v10157: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:46:41.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:46:41 smithi138 ceph-mon[36954]: pgmap v10158: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:46:41.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:46:41 smithi100 ceph-mon[32599]: pgmap v10158: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:46:42.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:46:41 smithi167 ceph-mon[37140]: pgmap v10158: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:46:43.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:46:43 smithi138 ceph-mon[36954]: pgmap v10159: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:46:43.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:46:43 smithi100 ceph-mon[32599]: pgmap v10159: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:46:44.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:46:43 smithi167 ceph-mon[37140]: pgmap v10159: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:46:44.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:46:44.067+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:46:44.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:46:44.068+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:46:44.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:46:44.068+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:46:44.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:46:44.068+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:46:44.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:46:44.068+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:46:45.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:46:45 smithi138 ceph-mon[36954]: pgmap v10160: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:46:45.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:46:45 smithi100 ceph-mon[32599]: pgmap v10160: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:46:46.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:46:45 smithi167 ceph-mon[37140]: pgmap v10160: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:46:47.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:46:47 smithi138 ceph-mon[36954]: pgmap v10161: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:46:47.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:46:47 smithi100 ceph-mon[32599]: pgmap v10161: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:46:48.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:46:47 smithi167 ceph-mon[37140]: pgmap v10161: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:46:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:46:48 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:46:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:46:48 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:46:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:46:48 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:46:49.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:46:49 smithi100 ceph-mon[32599]: pgmap v10162: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:46:49.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:46:49 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:46:49.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:46:49 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:46:49.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:46:49 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:46:50.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:46:49 smithi167 ceph-mon[37140]: pgmap v10162: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:46:50.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:46:49 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:46:50.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:46:49 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:46:50.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:46:49 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:46:50.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:46:49 smithi138 ceph-mon[36954]: pgmap v10162: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:46:50.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:46:49 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:46:50.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:46:49 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:46:50.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:46:49 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:46:51.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:46:51 smithi100 ceph-mon[32599]: pgmap v10163: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:46:52.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:46:51 smithi167 ceph-mon[37140]: pgmap v10163: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:46:52.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:46:51 smithi138 ceph-mon[36954]: pgmap v10163: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:46:53.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:46:53 smithi100 ceph-mon[32599]: pgmap v10164: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:46:54.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:46:53 smithi167 ceph-mon[37140]: pgmap v10164: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:46:54.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:46:53 smithi138 ceph-mon[36954]: pgmap v10164: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:46:55.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:46:55 smithi100 ceph-mon[32599]: pgmap v10165: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:46:56.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:46:55 smithi167 ceph-mon[37140]: pgmap v10165: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:46:56.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:46:55 smithi138 ceph-mon[36954]: pgmap v10165: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:46:57.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:46:57 smithi100 ceph-mon[32599]: pgmap v10166: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:46:58.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:46:57 smithi167 ceph-mon[37140]: pgmap v10166: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:46:58.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:46:57 smithi138 ceph-mon[36954]: pgmap v10166: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:46:59.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:46:59.072+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:46:59.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:46:59.072+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:46:59.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:46:59.072+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:46:59.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:46:59.072+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:46:59.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:46:59.072+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:46:59.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:46:59 smithi100 ceph-mon[32599]: pgmap v10167: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:47:00.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:46:59 smithi167 ceph-mon[37140]: pgmap v10167: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:47:00.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:46:59 smithi138 ceph-mon[36954]: pgmap v10167: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:47:01.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:47:01 smithi100 ceph-mon[32599]: pgmap v10168: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:47:02.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:47:01 smithi167 ceph-mon[37140]: pgmap v10168: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:47:02.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:47:01 smithi138 ceph-mon[36954]: pgmap v10168: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:47:03.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:47:03 smithi100 ceph-mon[32599]: pgmap v10169: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:47:04.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:47:03 smithi167 ceph-mon[37140]: pgmap v10169: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:47:04.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:47:03 smithi138 ceph-mon[36954]: pgmap v10169: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:47:05.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:47:05 smithi100 ceph-mon[32599]: pgmap v10170: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:47:06.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:47:05 smithi167 ceph-mon[37140]: pgmap v10170: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:47:06.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:47:05 smithi138 ceph-mon[36954]: pgmap v10170: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:47:07.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:47:07 smithi100 ceph-mon[32599]: pgmap v10171: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:47:08.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:47:07 smithi167 ceph-mon[37140]: pgmap v10171: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:47:08.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:47:07 smithi138 ceph-mon[36954]: pgmap v10171: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:47:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:47:08 smithi167 ceph-mon[37140]: osdmap e682: 12 total, 12 up, 12 in 2024-07-26T18:47:09.091 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:47:08 smithi138 ceph-mon[36954]: osdmap e682: 12 total, 12 up, 12 in 2024-07-26T18:47:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:47:08 smithi100 ceph-mon[32599]: osdmap e682: 12 total, 12 up, 12 in 2024-07-26T18:47:10.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:47:09 smithi167 ceph-mon[37140]: pgmap v10173: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:47:10.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:47:09 smithi138 ceph-mon[36954]: pgmap v10173: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:47:10.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:47:09 smithi100 ceph-mon[32599]: pgmap v10173: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:47:12.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:47:11 smithi167 ceph-mon[37140]: pgmap v10174: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:47:12.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:47:11 smithi138 ceph-mon[36954]: pgmap v10174: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:47:12.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:47:11 smithi100 ceph-mon[32599]: pgmap v10174: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:47:14.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:47:13 smithi167 ceph-mon[37140]: pgmap v10175: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:47:14.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:47:14.087+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:47:14.089 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:47:13 smithi100 ceph-mon[32599]: pgmap v10175: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:47:14.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:47:14.088+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:47:14.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:47:14.088+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:47:14.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:47:14.088+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:47:14.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:47:14.088+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:47:14.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:47:13 smithi138 ceph-mon[36954]: pgmap v10175: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:47:16.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:47:15 smithi167 ceph-mon[37140]: pgmap v10176: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:47:16.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:47:15 smithi138 ceph-mon[36954]: pgmap v10176: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:47:16.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:47:15 smithi100 ceph-mon[32599]: pgmap v10176: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:47:18.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:47:17 smithi167 ceph-mon[37140]: pgmap v10177: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:47:18.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:47:17 smithi138 ceph-mon[36954]: pgmap v10177: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:47:18.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:47:17 smithi100 ceph-mon[32599]: pgmap v10177: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:47:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:47:19 smithi138 ceph-mon[36954]: pgmap v10178: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 188 B/s wr, 5 op/s 2024-07-26T18:47:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:47:19 smithi138 ceph-mon[36954]: osdmap e683: 12 total, 12 up, 12 in 2024-07-26T18:47:19.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:47:19 smithi100 ceph-mon[32599]: pgmap v10178: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 188 B/s wr, 5 op/s 2024-07-26T18:47:19.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:47:19 smithi100 ceph-mon[32599]: osdmap e683: 12 total, 12 up, 12 in 2024-07-26T18:47:20.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:47:19 smithi167 ceph-mon[37140]: pgmap v10178: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 188 B/s wr, 5 op/s 2024-07-26T18:47:20.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:47:19 smithi167 ceph-mon[37140]: osdmap e683: 12 total, 12 up, 12 in 2024-07-26T18:47:21.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:47:21 smithi138 ceph-mon[36954]: pgmap v10180: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:47:21.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:47:21 smithi100 ceph-mon[32599]: pgmap v10180: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:47:22.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:47:21 smithi167 ceph-mon[37140]: pgmap v10180: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:47:23.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:47:23 smithi138 ceph-mon[36954]: pgmap v10181: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:47:23.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:47:23 smithi100 ceph-mon[32599]: pgmap v10181: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:47:24.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:47:23 smithi167 ceph-mon[37140]: pgmap v10181: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:47:25.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:47:25 smithi138 ceph-mon[36954]: pgmap v10182: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:47:25.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:47:25 smithi100 ceph-mon[32599]: pgmap v10182: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:47:26.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:47:25 smithi167 ceph-mon[37140]: pgmap v10182: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:47:27.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:47:27 smithi138 ceph-mon[36954]: pgmap v10183: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:47:27.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:47:27 smithi100 ceph-mon[32599]: pgmap v10183: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:47:28.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:47:27 smithi167 ceph-mon[37140]: pgmap v10183: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:47:29.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:47:29.095+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:47:29.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:47:29.096+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:47:29.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:47:29.096+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:47:29.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:47:29.096+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:47:29.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:47:29.096+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:47:29.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:47:29 smithi138 ceph-mon[36954]: pgmap v10184: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:47:29.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:47:29 smithi100 ceph-mon[32599]: pgmap v10184: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:47:30.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:47:29 smithi167 ceph-mon[37140]: pgmap v10184: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:47:31.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:47:31 smithi138 ceph-mon[36954]: pgmap v10185: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:47:31.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:47:31 smithi100 ceph-mon[32599]: pgmap v10185: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:47:32.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:47:31 smithi167 ceph-mon[37140]: pgmap v10185: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:47:33.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:47:33 smithi138 ceph-mon[36954]: pgmap v10186: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:47:33.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:47:33 smithi100 ceph-mon[32599]: pgmap v10186: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:47:34.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:47:33 smithi167 ceph-mon[37140]: pgmap v10186: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:47:35.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:47:35 smithi100 ceph-mon[32599]: pgmap v10187: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:47:36.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:47:35 smithi167 ceph-mon[37140]: pgmap v10187: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:47:36.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:47:35 smithi138 ceph-mon[36954]: pgmap v10187: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:47:37.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:47:37 smithi100 ceph-mon[32599]: pgmap v10188: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:47:38.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:47:37 smithi167 ceph-mon[37140]: pgmap v10188: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:47:38.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:47:37 smithi138 ceph-mon[36954]: pgmap v10188: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:47:39.992 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:47:39 smithi138 ceph-mon[36954]: pgmap v10189: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:47:39.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:47:39 smithi100 ceph-mon[32599]: pgmap v10189: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:47:40.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:47:39 smithi167 ceph-mon[37140]: pgmap v10189: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:47:41.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:47:41 smithi100 ceph-mon[32599]: pgmap v10190: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:47:42.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:47:41 smithi167 ceph-mon[37140]: pgmap v10190: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:47:42.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:47:41 smithi138 ceph-mon[36954]: pgmap v10190: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:47:43.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:47:43 smithi100 ceph-mon[32599]: pgmap v10191: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:47:44.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:47:43 smithi167 ceph-mon[37140]: pgmap v10191: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:47:44.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:47:44.111+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:47:44.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:47:44.112+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:47:44.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:47:44.112+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:47:44.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:47:44.112+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:47:44.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:47:44.112+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:47:44.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:47:43 smithi138 ceph-mon[36954]: pgmap v10191: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:47:45.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:47:45 smithi100 ceph-mon[32599]: pgmap v10192: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:47:46.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:47:45 smithi167 ceph-mon[37140]: pgmap v10192: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:47:46.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:47:45 smithi138 ceph-mon[36954]: pgmap v10192: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:47:47.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:47:47 smithi100 ceph-mon[32599]: pgmap v10193: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:47:48.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:47:47 smithi167 ceph-mon[37140]: pgmap v10193: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:47:48.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:47:47 smithi138 ceph-mon[36954]: pgmap v10193: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:47:49.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:47:49 smithi100 ceph-mon[32599]: pgmap v10194: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:47:49.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:47:49 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:47:49.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:47:49 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:47:49.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:47:49 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:47:49.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:47:49 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:47:50.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:47:49 smithi167 ceph-mon[37140]: pgmap v10194: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:47:50.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:47:49 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:47:50.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:47:49 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:47:50.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:47:49 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:47:50.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:47:49 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:47:50.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:47:49 smithi138 ceph-mon[36954]: pgmap v10194: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:47:50.231 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:47:49 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:47:50.231 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:47:49 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:47:50.231 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:47:49 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:47:50.231 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:47:49 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:47:51.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:47:51 smithi100 ceph-mon[32599]: pgmap v10195: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:47:52.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:47:51 smithi167 ceph-mon[37140]: pgmap v10195: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:47:52.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:47:51 smithi138 ceph-mon[36954]: pgmap v10195: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:47:53.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:47:53 smithi100 ceph-mon[32599]: pgmap v10196: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:47:54.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:47:53 smithi167 ceph-mon[37140]: pgmap v10196: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:47:54.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:47:53 smithi138 ceph-mon[36954]: pgmap v10196: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:47:55.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:47:55 smithi100 ceph-mon[32599]: pgmap v10197: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:47:56.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:47:55 smithi167 ceph-mon[37140]: pgmap v10197: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:47:56.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:47:55 smithi138 ceph-mon[36954]: pgmap v10197: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:47:57.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:47:57 smithi100 ceph-mon[32599]: pgmap v10198: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:47:58.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:47:57 smithi167 ceph-mon[37140]: pgmap v10198: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:47:58.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:47:57 smithi138 ceph-mon[36954]: pgmap v10198: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:47:59.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:47:59.127+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:47:59.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:47:59.128+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:47:59.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:47:59.128+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:47:59.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:47:59.128+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:47:59.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:47:59.128+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:47:59.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:47:59 smithi100 ceph-mon[32599]: pgmap v10199: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:48:00.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:47:59 smithi167 ceph-mon[37140]: pgmap v10199: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:48:00.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:47:59 smithi138 ceph-mon[36954]: pgmap v10199: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:48:01.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:48:01 smithi100 ceph-mon[32599]: pgmap v10200: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:48:02.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:48:01 smithi167 ceph-mon[37140]: pgmap v10200: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:48:02.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:48:01 smithi138 ceph-mon[36954]: pgmap v10200: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:48:03.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:48:03.049+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 msgr2=0x7fe5940c0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-07-26T18:48:03.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:48:03.049+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 msgr2=0x7fe5940c0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-07-26T18:48:03.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:48:03.049+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fe590004b30 tx=0x7fe5900bb040 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:48:03.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:48:03.049+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fe590004b30 tx=0x7fe5900bb040 comp rx=0 tx=0).stop 2024-07-26T18:48:03.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:48:03.049+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 msgr2=0x7fe5940c0480 unknown :-1 s=STATE_CLOSED l=1).tick see no progress in more than 10000000 us during connecting to v2:172.21.15.100:6800/3937009534, fault. 2024-07-26T18:48:03.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:48:03.049+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 msgr2=0x7fe5940c0480 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-07-26T18:48:03.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:48:03.049+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T18:48:03.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:48:03.049+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 0x7fe5940b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T18:48:03.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:48:03.049+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 0x7fe5940b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:48:03.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:48:03.049+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 0x7fe5940b5b30 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fe590005d00 tx=0x7fe59003b040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T18:48:03.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:48:03 smithi100 ceph-mon[32599]: pgmap v10201: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:48:04.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:48:03 smithi167 ceph-mon[37140]: pgmap v10201: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:48:04.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:48:03 smithi138 ceph-mon[36954]: pgmap v10201: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:48:05.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:48:05.569+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T18:48:05.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:48:05.569+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c006690 con 0x7fe5a81087a0 2024-07-26T18:48:05.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:48:05.569+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 141 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c006690 con 0x7fe5a81087a0 2024-07-26T18:48:05.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:48:05.569+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 141 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T18:48:05.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:48:05 smithi100 ceph-mon[32599]: pgmap v10202: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:48:06.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:48:05 smithi167 ceph-mon[37140]: pgmap v10202: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:48:06.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:48:05 smithi138 ceph-mon[36954]: pgmap v10202: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:48:08.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:48:07 smithi167 ceph-mon[37140]: pgmap v10203: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:48:08.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:48:07 smithi138 ceph-mon[36954]: pgmap v10203: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:48:08.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:48:07 smithi100 ceph-mon[32599]: pgmap v10203: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:48:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:48:08 smithi167 ceph-mon[37140]: osdmap e684: 12 total, 12 up, 12 in 2024-07-26T18:48:09.091 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:48:08 smithi138 ceph-mon[36954]: osdmap e684: 12 total, 12 up, 12 in 2024-07-26T18:48:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:48:08 smithi100 ceph-mon[32599]: osdmap e684: 12 total, 12 up, 12 in 2024-07-26T18:48:10.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:48:09 smithi167 ceph-mon[37140]: pgmap v10205: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:48:10.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:48:09 smithi138 ceph-mon[36954]: pgmap v10205: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:48:10.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:48:09 smithi100 ceph-mon[32599]: pgmap v10205: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:48:12.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:48:11 smithi167 ceph-mon[37140]: pgmap v10206: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:48:12.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:48:11 smithi138 ceph-mon[36954]: pgmap v10206: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:48:12.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:48:11 smithi100 ceph-mon[32599]: pgmap v10206: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:48:14.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:48:13 smithi167 ceph-mon[37140]: pgmap v10207: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:48:14.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:48:14.139+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:48:14.143 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:48:13 smithi100 ceph-mon[32599]: pgmap v10207: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:48:14.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:48:14.139+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:48:14.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:48:14.139+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:48:14.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:48:14.139+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:48:14.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:48:14.140+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:48:14.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:48:13 smithi138 ceph-mon[36954]: pgmap v10207: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:48:16.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:48:15 smithi138 ceph-mon[36954]: pgmap v10208: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:48:16.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:48:15 smithi100 ceph-mon[32599]: pgmap v10208: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:48:16.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:48:15 smithi167 ceph-mon[37140]: pgmap v10208: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:48:18.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:48:17 smithi138 ceph-mon[36954]: pgmap v10209: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:48:18.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:48:17 smithi100 ceph-mon[32599]: pgmap v10209: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:48:18.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:48:17 smithi167 ceph-mon[37140]: pgmap v10209: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:48:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:48:19 smithi138 ceph-mon[36954]: pgmap v10210: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 188 B/s wr, 5 op/s 2024-07-26T18:48:19.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:48:19 smithi138 ceph-mon[36954]: osdmap e685: 12 total, 12 up, 12 in 2024-07-26T18:48:19.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:48:19 smithi100 ceph-mon[32599]: pgmap v10210: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 188 B/s wr, 5 op/s 2024-07-26T18:48:19.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:48:19 smithi100 ceph-mon[32599]: osdmap e685: 12 total, 12 up, 12 in 2024-07-26T18:48:20.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:48:19 smithi167 ceph-mon[37140]: pgmap v10210: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 188 B/s wr, 5 op/s 2024-07-26T18:48:20.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:48:19 smithi167 ceph-mon[37140]: osdmap e685: 12 total, 12 up, 12 in 2024-07-26T18:48:21.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:48:21 smithi138 ceph-mon[36954]: pgmap v10212: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:48:21.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:48:21 smithi100 ceph-mon[32599]: pgmap v10212: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:48:22.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:48:21 smithi167 ceph-mon[37140]: pgmap v10212: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:48:23.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:48:23 smithi138 ceph-mon[36954]: pgmap v10213: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:48:23.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:48:23 smithi100 ceph-mon[32599]: pgmap v10213: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:48:24.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:48:23 smithi167 ceph-mon[37140]: pgmap v10213: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:48:25.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:48:25 smithi100 ceph-mon[32599]: pgmap v10214: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:48:26.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:48:25 smithi167 ceph-mon[37140]: pgmap v10214: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:48:26.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:48:25 smithi138 ceph-mon[36954]: pgmap v10214: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:48:27.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:48:27 smithi100 ceph-mon[32599]: pgmap v10215: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:48:28.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:48:27 smithi167 ceph-mon[37140]: pgmap v10215: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:48:28.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:48:27 smithi138 ceph-mon[36954]: pgmap v10215: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:48:29.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:48:29.147+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:48:29.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:48:29.147+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:48:29.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:48:29.147+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:48:29.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:48:29.147+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:48:29.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:48:29.148+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:48:29.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:48:29 smithi100 ceph-mon[32599]: pgmap v10216: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:48:30.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:48:29 smithi167 ceph-mon[37140]: pgmap v10216: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:48:30.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:48:29 smithi138 ceph-mon[36954]: pgmap v10216: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:48:31.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:48:31 smithi100 ceph-mon[32599]: pgmap v10217: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:48:32.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:48:31 smithi167 ceph-mon[37140]: pgmap v10217: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:48:32.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:48:31 smithi138 ceph-mon[36954]: pgmap v10217: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:48:33.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:48:33 smithi100 ceph-mon[32599]: pgmap v10218: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-07-26T18:48:34.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:48:33 smithi167 ceph-mon[37140]: pgmap v10218: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-07-26T18:48:34.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:48:33 smithi138 ceph-mon[36954]: pgmap v10218: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-07-26T18:48:35.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:48:35 smithi100 ceph-mon[32599]: pgmap v10219: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:48:36.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:48:35 smithi167 ceph-mon[37140]: pgmap v10219: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:48:36.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:48:35 smithi138 ceph-mon[36954]: pgmap v10219: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:48:37.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:48:37 smithi100 ceph-mon[32599]: pgmap v10220: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:48:38.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:48:37 smithi167 ceph-mon[37140]: pgmap v10220: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:48:38.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:48:37 smithi138 ceph-mon[36954]: pgmap v10220: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:48:39.992 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:48:39 smithi138 ceph-mon[36954]: pgmap v10221: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:48:39.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:48:39 smithi100 ceph-mon[32599]: pgmap v10221: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:48:40.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:48:39 smithi167 ceph-mon[37140]: pgmap v10221: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:48:41.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:48:41 smithi100 ceph-mon[32599]: pgmap v10222: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:48:42.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:48:41 smithi167 ceph-mon[37140]: pgmap v10222: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:48:42.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:48:41 smithi138 ceph-mon[36954]: pgmap v10222: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:48:43.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:48:43 smithi100 ceph-mon[32599]: pgmap v10223: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T18:48:44.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:48:43 smithi167 ceph-mon[37140]: pgmap v10223: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T18:48:44.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:48:44.163+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:48:44.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:48:44.164+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:48:44.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:48:44.164+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:48:44.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:48:44.164+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:48:44.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:48:44.164+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:48:44.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:48:43 smithi138 ceph-mon[36954]: pgmap v10223: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T18:48:45.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:48:45 smithi100 ceph-mon[32599]: pgmap v10224: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:48:46.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:48:45 smithi167 ceph-mon[37140]: pgmap v10224: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:48:46.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:48:45 smithi138 ceph-mon[36954]: pgmap v10224: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:48:47.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:48:47 smithi100 ceph-mon[32599]: pgmap v10225: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:48:48.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:48:47 smithi167 ceph-mon[37140]: pgmap v10225: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:48:48.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:48:47 smithi138 ceph-mon[36954]: pgmap v10225: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:48:49.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:48:49 smithi100 ceph-mon[32599]: pgmap v10226: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:48:50.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:48:49 smithi167 ceph-mon[37140]: pgmap v10226: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:48:50.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:48:49 smithi138 ceph-mon[36954]: pgmap v10226: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:48:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:48:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:48:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:48:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:48:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:48:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:48:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:48:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:48:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:48:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:48:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:48:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:48:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:48:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:48:51.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:48:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:48:51.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:48:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:48:51.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:48:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:48:51.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:48:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:48:51.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:48:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:48:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:48:52 smithi100 ceph-mon[32599]: pgmap v10227: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:48:52.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:48:52 smithi167 ceph-mon[37140]: pgmap v10227: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:48:52.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:48:52 smithi138 ceph-mon[36954]: pgmap v10227: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:48:54.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:48:54 smithi100 ceph-mon[32599]: pgmap v10228: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T18:48:54.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:48:54 smithi167 ceph-mon[37140]: pgmap v10228: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T18:48:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:48:54 smithi138 ceph-mon[36954]: pgmap v10228: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T18:48:56.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:48:56 smithi100 ceph-mon[32599]: pgmap v10229: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:48:56.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:48:56 smithi167 ceph-mon[37140]: pgmap v10229: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:48:56.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:48:56 smithi138 ceph-mon[36954]: pgmap v10229: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:48:58.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:48:58 smithi100 ceph-mon[32599]: pgmap v10230: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:48:58.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:48:58 smithi167 ceph-mon[37140]: pgmap v10230: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:48:58.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:48:58 smithi138 ceph-mon[36954]: pgmap v10230: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:48:59.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:48:59.179+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:48:59.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:48:59.180+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:48:59.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:48:59.180+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:48:59.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:48:59.180+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:48:59.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:48:59.180+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:49:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:49:00 smithi100 ceph-mon[32599]: pgmap v10231: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:49:00.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:49:00 smithi167 ceph-mon[37140]: pgmap v10231: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:49:00.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:49:00 smithi138 ceph-mon[36954]: pgmap v10231: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:49:02.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:49:02 smithi100 ceph-mon[32599]: pgmap v10232: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:49:02.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:49:02 smithi167 ceph-mon[37140]: pgmap v10232: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:49:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:49:02 smithi138 ceph-mon[36954]: pgmap v10232: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:49:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:49:04 smithi100 ceph-mon[32599]: pgmap v10233: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:49:04.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:49:04 smithi167 ceph-mon[37140]: pgmap v10233: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:49:04.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:49:04 smithi138 ceph-mon[36954]: pgmap v10233: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:49:06.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:49:06 smithi100 ceph-mon[32599]: pgmap v10234: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:49:06.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:49:06 smithi167 ceph-mon[37140]: pgmap v10234: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:49:06.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:49:06 smithi138 ceph-mon[36954]: pgmap v10234: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:49:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:49:08 smithi100 ceph-mon[32599]: pgmap v10235: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:49:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:49:08 smithi100 ceph-mon[32599]: osdmap e686: 12 total, 12 up, 12 in 2024-07-26T18:49:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:49:08 smithi167 ceph-mon[37140]: pgmap v10235: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:49:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:49:08 smithi167 ceph-mon[37140]: osdmap e686: 12 total, 12 up, 12 in 2024-07-26T18:49:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:49:08 smithi138 ceph-mon[36954]: pgmap v10235: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:49:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:49:08 smithi138 ceph-mon[36954]: osdmap e686: 12 total, 12 up, 12 in 2024-07-26T18:49:10.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:49:10 smithi167 ceph-mon[37140]: pgmap v10237: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:49:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:49:10 smithi138 ceph-mon[36954]: pgmap v10237: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:49:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:49:10 smithi100 ceph-mon[32599]: pgmap v10237: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:49:12.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:49:12 smithi167 ceph-mon[37140]: pgmap v10238: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:49:12.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:49:12 smithi138 ceph-mon[36954]: pgmap v10238: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:49:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:49:12 smithi100 ceph-mon[32599]: pgmap v10238: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:49:14.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:49:14.196+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:49:14.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:49:14.196+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:49:14.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:49:14.196+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:49:14.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:49:14.196+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:49:14.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:49:14.196+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:49:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:49:14 smithi100 ceph-mon[32599]: pgmap v10239: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:49:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:49:14 smithi100 ceph-mon[32599]: osdmap e687: 12 total, 12 up, 12 in 2024-07-26T18:49:14.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:49:14 smithi167 ceph-mon[37140]: pgmap v10239: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:49:14.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:49:14 smithi167 ceph-mon[37140]: osdmap e687: 12 total, 12 up, 12 in 2024-07-26T18:49:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:49:14 smithi138 ceph-mon[36954]: pgmap v10239: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:49:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:49:14 smithi138 ceph-mon[36954]: osdmap e687: 12 total, 12 up, 12 in 2024-07-26T18:49:16.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:49:16 smithi167 ceph-mon[37140]: pgmap v10241: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T18:49:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:49:16 smithi138 ceph-mon[36954]: pgmap v10241: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T18:49:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:49:16 smithi100 ceph-mon[32599]: pgmap v10241: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T18:49:18.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:49:18 smithi167 ceph-mon[37140]: pgmap v10242: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 218 B/s wr, 6 op/s 2024-07-26T18:49:18.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:49:18 smithi138 ceph-mon[36954]: pgmap v10242: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 218 B/s wr, 6 op/s 2024-07-26T18:49:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:49:18 smithi100 ceph-mon[32599]: pgmap v10242: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 218 B/s wr, 6 op/s 2024-07-26T18:49:20.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:49:20 smithi167 ceph-mon[37140]: pgmap v10243: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:49:20.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:49:20 smithi138 ceph-mon[36954]: pgmap v10243: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:49:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:49:20 smithi100 ceph-mon[32599]: pgmap v10243: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:49:22.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:49:22 smithi167 ceph-mon[37140]: pgmap v10244: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:49:22.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:49:22 smithi138 ceph-mon[36954]: pgmap v10244: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:49:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:49:22 smithi100 ceph-mon[32599]: pgmap v10244: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:49:24.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:49:24 smithi167 ceph-mon[37140]: pgmap v10245: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:49:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:49:24 smithi138 ceph-mon[36954]: pgmap v10245: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:49:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:49:24 smithi100 ceph-mon[32599]: pgmap v10245: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:49:26.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:49:26 smithi167 ceph-mon[37140]: pgmap v10246: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 93 B/s wr, 7 op/s 2024-07-26T18:49:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:49:26 smithi138 ceph-mon[36954]: pgmap v10246: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 93 B/s wr, 7 op/s 2024-07-26T18:49:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:49:26 smithi100 ceph-mon[32599]: pgmap v10246: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 93 B/s wr, 7 op/s 2024-07-26T18:49:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:49:28 smithi138 ceph-mon[36954]: pgmap v10247: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:49:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:49:28 smithi100 ceph-mon[32599]: pgmap v10247: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:49:28.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:49:28 smithi167 ceph-mon[37140]: pgmap v10247: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:49:29.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:49:29.199+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:49:29.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:49:29.199+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:49:29.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:49:29.199+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:49:29.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:49:29.199+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:49:29.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:49:29.199+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:49:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:49:30 smithi138 ceph-mon[36954]: pgmap v10248: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:49:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:49:30 smithi100 ceph-mon[32599]: pgmap v10248: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:49:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:49:30 smithi167 ceph-mon[37140]: pgmap v10248: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:49:32.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:49:32 smithi138 ceph-mon[36954]: pgmap v10249: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:49:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:49:32 smithi100 ceph-mon[32599]: pgmap v10249: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:49:32.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:49:32 smithi167 ceph-mon[37140]: pgmap v10249: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:49:34.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:49:34 smithi138 ceph-mon[36954]: pgmap v10250: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T18:49:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:49:34 smithi100 ceph-mon[32599]: pgmap v10250: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T18:49:34.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:49:34 smithi167 ceph-mon[37140]: pgmap v10250: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T18:49:36.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:49:36 smithi138 ceph-mon[36954]: pgmap v10251: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:49:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:49:36 smithi100 ceph-mon[32599]: pgmap v10251: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:49:36.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:49:36 smithi167 ceph-mon[37140]: pgmap v10251: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:49:38.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:49:38 smithi138 ceph-mon[36954]: pgmap v10252: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:49:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:49:38 smithi100 ceph-mon[32599]: pgmap v10252: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:49:38.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:49:38 smithi167 ceph-mon[37140]: pgmap v10252: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:49:40.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:49:40 smithi167 ceph-mon[37140]: pgmap v10253: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:49:40.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:49:40 smithi138 ceph-mon[36954]: pgmap v10253: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:49:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:49:40 smithi100 ceph-mon[32599]: pgmap v10253: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:49:42.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:49:42 smithi138 ceph-mon[36954]: pgmap v10254: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:49:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:49:42 smithi100 ceph-mon[32599]: pgmap v10254: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:49:42.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:49:42 smithi167 ceph-mon[37140]: pgmap v10254: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:49:44.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:49:44.215+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:49:44.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:49:44.216+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:49:44.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:49:44.216+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:49:44.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:49:44.216+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:49:44.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:49:44.216+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:49:44.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:49:44 smithi100 ceph-mon[32599]: pgmap v10255: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T18:49:44.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:49:44 smithi138 ceph-mon[36954]: pgmap v10255: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T18:49:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:49:44 smithi167 ceph-mon[37140]: pgmap v10255: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T18:49:46.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:49:46 smithi138 ceph-mon[36954]: pgmap v10256: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:49:46.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:49:46 smithi100 ceph-mon[32599]: pgmap v10256: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:49:46.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:49:46 smithi167 ceph-mon[37140]: pgmap v10256: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:49:48.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:49:48 smithi138 ceph-mon[36954]: pgmap v10257: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:49:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:49:48 smithi100 ceph-mon[32599]: pgmap v10257: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:49:48.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:49:48 smithi167 ceph-mon[37140]: pgmap v10257: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:49:50.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:49:50 smithi138 ceph-mon[36954]: pgmap v10258: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:49:50.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:49:50 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:49:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:49:50 smithi100 ceph-mon[32599]: pgmap v10258: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:49:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:49:50 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:49:50.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:49:50 smithi167 ceph-mon[37140]: pgmap v10258: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:49:50.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:49:50 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:49:51.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:49:51 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:49:51.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:49:51 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:49:51.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:49:51 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:49:51.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:49:51 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:49:51.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:49:51 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:49:51.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:49:51 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:49:51.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:49:51 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:49:51.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:49:51 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:49:51.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:49:51 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:49:52.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:49:52 smithi138 ceph-mon[36954]: pgmap v10259: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:49:52.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:49:52 smithi100 ceph-mon[32599]: pgmap v10259: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:49:52.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:49:52 smithi167 ceph-mon[37140]: pgmap v10259: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:49:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:49:54 smithi138 ceph-mon[36954]: pgmap v10260: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T18:49:54.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:49:54 smithi100 ceph-mon[32599]: pgmap v10260: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T18:49:54.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:49:54 smithi167 ceph-mon[37140]: pgmap v10260: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T18:49:56.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:49:56 smithi138 ceph-mon[36954]: pgmap v10261: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:49:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:49:56 smithi100 ceph-mon[32599]: pgmap v10261: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:49:56.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:49:56 smithi167 ceph-mon[37140]: pgmap v10261: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:49:58.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:49:58 smithi138 ceph-mon[36954]: pgmap v10262: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:49:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:49:58 smithi100 ceph-mon[32599]: pgmap v10262: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:49:58.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:49:58 smithi167 ceph-mon[37140]: pgmap v10262: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:49:59.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:49:59.223+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:49:59.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:49:59.225+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:49:59.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:49:59.225+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:49:59.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:49:59.225+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:49:59.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:49:59.225+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:50:00.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:00 smithi138 ceph-mon[36954]: pgmap v10263: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:50:00.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:00 smithi138 ceph-mon[36954]: overall HEALTH_OK 2024-07-26T18:50:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:00 smithi100 ceph-mon[32599]: pgmap v10263: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:50:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:00 smithi100 ceph-mon[32599]: overall HEALTH_OK 2024-07-26T18:50:00.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:00 smithi167 ceph-mon[37140]: pgmap v10263: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:50:00.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:00 smithi167 ceph-mon[37140]: overall HEALTH_OK 2024-07-26T18:50:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:02 smithi138 ceph-mon[36954]: pgmap v10264: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:50:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:02 smithi100 ceph-mon[32599]: pgmap v10264: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:50:02.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:02 smithi167 ceph-mon[37140]: pgmap v10264: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:50:04.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:04 smithi138 ceph-mon[36954]: pgmap v10265: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:50:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:04 smithi100 ceph-mon[32599]: pgmap v10265: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:50:04.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:04 smithi167 ceph-mon[37140]: pgmap v10265: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:50:06.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:06 smithi138 ceph-mon[36954]: pgmap v10266: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:50:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:06 smithi100 ceph-mon[32599]: pgmap v10266: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:50:06.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:06 smithi167 ceph-mon[37140]: pgmap v10266: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:50:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:08 smithi138 ceph-mon[36954]: pgmap v10267: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:50:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:08 smithi138 ceph-mon[36954]: osdmap e688: 12 total, 12 up, 12 in 2024-07-26T18:50:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:08 smithi100 ceph-mon[32599]: pgmap v10267: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:50:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:08 smithi100 ceph-mon[32599]: osdmap e688: 12 total, 12 up, 12 in 2024-07-26T18:50:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:08 smithi167 ceph-mon[37140]: pgmap v10267: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:50:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:08 smithi167 ceph-mon[37140]: osdmap e688: 12 total, 12 up, 12 in 2024-07-26T18:50:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:10 smithi138 ceph-mon[36954]: pgmap v10269: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:50:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:10 smithi100 ceph-mon[32599]: pgmap v10269: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:50:10.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:10 smithi167 ceph-mon[37140]: pgmap v10269: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:50:12.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:12 smithi138 ceph-mon[36954]: pgmap v10270: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:50:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:12 smithi100 ceph-mon[32599]: pgmap v10270: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:50:12.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:12 smithi167 ceph-mon[37140]: pgmap v10270: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:50:14.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:50:14.240+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:50:14.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:50:14.241+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:50:14.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:50:14.241+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:50:14.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:50:14.241+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:50:14.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:50:14.241+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:50:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:14 smithi100 ceph-mon[32599]: pgmap v10271: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:50:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:14 smithi138 ceph-mon[36954]: pgmap v10271: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:50:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:14 smithi167 ceph-mon[37140]: pgmap v10271: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:50:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:16 smithi138 ceph-mon[36954]: pgmap v10272: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:50:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:16 smithi100 ceph-mon[32599]: pgmap v10272: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:50:16.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:16 smithi167 ceph-mon[37140]: pgmap v10272: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:50:18.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:18 smithi138 ceph-mon[36954]: pgmap v10273: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:50:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:18 smithi100 ceph-mon[32599]: pgmap v10273: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:50:18.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:18 smithi167 ceph-mon[37140]: pgmap v10273: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:50:19.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:19 smithi100 ceph-mon[32599]: pgmap v10274: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 181 B/s wr, 6 op/s 2024-07-26T18:50:19.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:19 smithi100 ceph-mon[32599]: osdmap e689: 12 total, 12 up, 12 in 2024-07-26T18:50:20.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:19 smithi167 ceph-mon[37140]: pgmap v10274: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 181 B/s wr, 6 op/s 2024-07-26T18:50:20.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:19 smithi167 ceph-mon[37140]: osdmap e689: 12 total, 12 up, 12 in 2024-07-26T18:50:20.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:19 smithi138 ceph-mon[36954]: pgmap v10274: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 181 B/s wr, 6 op/s 2024-07-26T18:50:20.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:19 smithi138 ceph-mon[36954]: osdmap e689: 12 total, 12 up, 12 in 2024-07-26T18:50:21.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:21 smithi100 ceph-mon[32599]: pgmap v10276: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:50:22.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:21 smithi167 ceph-mon[37140]: pgmap v10276: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:50:22.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:21 smithi138 ceph-mon[36954]: pgmap v10276: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:50:23.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:23 smithi100 ceph-mon[32599]: pgmap v10277: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:50:24.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:23 smithi167 ceph-mon[37140]: pgmap v10277: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:50:24.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:23 smithi138 ceph-mon[36954]: pgmap v10277: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:50:25.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:25 smithi100 ceph-mon[32599]: pgmap v10278: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:50:26.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:25 smithi167 ceph-mon[37140]: pgmap v10278: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:50:26.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:25 smithi138 ceph-mon[36954]: pgmap v10278: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:50:27.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:27 smithi100 ceph-mon[32599]: pgmap v10279: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:50:28.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:27 smithi167 ceph-mon[37140]: pgmap v10279: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:50:28.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:27 smithi138 ceph-mon[36954]: pgmap v10279: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:50:29.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:50:29.256+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:50:29.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:50:29.257+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:50:29.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:50:29.257+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:50:29.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:50:29.257+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:50:29.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:50:29.257+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:50:29.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:29 smithi100 ceph-mon[32599]: pgmap v10280: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:50:30.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:29 smithi167 ceph-mon[37140]: pgmap v10280: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:50:30.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:29 smithi138 ceph-mon[36954]: pgmap v10280: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:50:31.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:31 smithi100 ceph-mon[32599]: pgmap v10281: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:50:32.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:31 smithi167 ceph-mon[37140]: pgmap v10281: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:50:32.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:31 smithi138 ceph-mon[36954]: pgmap v10281: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-07-26T18:50:33.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:33 smithi100 ceph-mon[32599]: pgmap v10282: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T18:50:34.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:33 smithi167 ceph-mon[37140]: pgmap v10282: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T18:50:34.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:33 smithi138 ceph-mon[36954]: pgmap v10282: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-07-26T18:50:35.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:35 smithi100 ceph-mon[32599]: pgmap v10283: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:50:36.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:35 smithi167 ceph-mon[37140]: pgmap v10283: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:50:36.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:35 smithi138 ceph-mon[36954]: pgmap v10283: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:50:37.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:37 smithi100 ceph-mon[32599]: pgmap v10284: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:50:38.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:37 smithi167 ceph-mon[37140]: pgmap v10284: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:50:38.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:37 smithi138 ceph-mon[36954]: pgmap v10284: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:50:39.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:39 smithi100 ceph-mon[32599]: pgmap v10285: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T18:50:40.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:39 smithi167 ceph-mon[37140]: pgmap v10285: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T18:50:40.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:39 smithi138 ceph-mon[36954]: pgmap v10285: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T18:50:42.011 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:41 smithi138 ceph-mon[36954]: pgmap v10286: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:50:42.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:41 smithi167 ceph-mon[37140]: pgmap v10286: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:50:42.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:41 smithi100 ceph-mon[32599]: pgmap v10286: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:50:44.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:43 smithi167 ceph-mon[37140]: pgmap v10287: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-07-26T18:50:44.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:43 smithi138 ceph-mon[36954]: pgmap v10287: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-07-26T18:50:44.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:43 smithi100 ceph-mon[32599]: pgmap v10287: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-07-26T18:50:44.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:50:44.273+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:50:44.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:50:44.273+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:50:44.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:50:44.273+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:50:44.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:50:44.273+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:50:44.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:50:44.273+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:50:46.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:45 smithi167 ceph-mon[37140]: pgmap v10288: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:50:46.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:45 smithi138 ceph-mon[36954]: pgmap v10288: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:50:46.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:45 smithi100 ceph-mon[32599]: pgmap v10288: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T18:50:48.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:47 smithi167 ceph-mon[37140]: pgmap v10289: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:50:48.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:47 smithi138 ceph-mon[36954]: pgmap v10289: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:50:48.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:47 smithi100 ceph-mon[32599]: pgmap v10289: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:50:50.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:49 smithi167 ceph-mon[37140]: pgmap v10290: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:50:50.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:49 smithi138 ceph-mon[36954]: pgmap v10290: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:50:50.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:49 smithi100 ceph-mon[32599]: pgmap v10290: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:50:52.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:51 smithi167 ceph-mon[37140]: pgmap v10291: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:50:52.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:51 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:50:52.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:51 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:50:52.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:51 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:50:52.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:51 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:50:52.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:51 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:50:52.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:51 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:50:52.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:51 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:50:52.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:51 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:50:52.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:51 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:50:52.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:51 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:50:52.024 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:51 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:50:52.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:51 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:50:52.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:51 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:50:52.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:51 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T18:50:52.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:51 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T18:50:52.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:51 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T18:50:52.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:51 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T18:50:52.025 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:51 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T18:50:52.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:51 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T18:50:52.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:51 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:50:52.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:51 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:50:52.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:51 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:50:52.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:51 smithi138 ceph-mon[36954]: pgmap v10291: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:50:52.193 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:51 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:50:52.193 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:51 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:50:52.194 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:51 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:50:52.194 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:51 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:50:52.194 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:51 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:50:52.194 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:51 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:50:52.194 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:51 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:50:52.194 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:51 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:50:52.195 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:51 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:50:52.195 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:51 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:50:52.195 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:51 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:50:52.195 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:51 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:50:52.195 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:51 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:50:52.195 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:51 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T18:50:52.195 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:51 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T18:50:52.195 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:51 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T18:50:52.196 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:51 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T18:50:52.196 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:51 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T18:50:52.196 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:51 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T18:50:52.196 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:51 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:50:52.196 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:51 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:50:52.196 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:51 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:50:52.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:51 smithi100 ceph-mon[32599]: pgmap v10291: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:50:52.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:51 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:50:52.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:51 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:50:52.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:51 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:50:52.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:51 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:50:52.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:51 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:50:52.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:51 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:50:52.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:51 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:50:52.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:51 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:50:52.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:51 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:50:52.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:51 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:50:52.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:51 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:50:52.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:51 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:50:52.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:51 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T18:50:52.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:51 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T18:50:52.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:51 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T18:50:52.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:51 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T18:50:52.247 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:51 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T18:50:52.247 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:51 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T18:50:52.247 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:51 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T18:50:52.247 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:51 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:50:52.247 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:51 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:50:52.247 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:51 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:50:54.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:53 smithi167 ceph-mon[37140]: pgmap v10292: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:50:54.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:53 smithi138 ceph-mon[36954]: pgmap v10292: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:50:54.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:53 smithi100 ceph-mon[32599]: pgmap v10292: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:50:56.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:55 smithi167 ceph-mon[37140]: pgmap v10293: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:50:56.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:55 smithi138 ceph-mon[36954]: pgmap v10293: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:50:56.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:55 smithi100 ceph-mon[32599]: pgmap v10293: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:50:58.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:57 smithi138 ceph-mon[36954]: pgmap v10294: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:50:58.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:57 smithi100 ceph-mon[32599]: pgmap v10294: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:50:58.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:57 smithi167 ceph-mon[37140]: pgmap v10294: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:50:59.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:50:59.280+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:50:59.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:50:59.280+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:50:59.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:50:59.280+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:50:59.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:50:59.280+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:50:59.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:50:59.280+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:51:00.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:50:59 smithi138 ceph-mon[36954]: pgmap v10295: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:51:00.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:50:59 smithi100 ceph-mon[32599]: pgmap v10295: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:51:00.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:50:59 smithi167 ceph-mon[37140]: pgmap v10295: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:51:02.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:51:01 smithi138 ceph-mon[36954]: pgmap v10296: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:51:02.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:51:01 smithi100 ceph-mon[32599]: pgmap v10296: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:51:02.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:51:01 smithi167 ceph-mon[37140]: pgmap v10296: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:51:04.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:51:03 smithi138 ceph-mon[36954]: pgmap v10297: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:51:04.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:51:03 smithi100 ceph-mon[32599]: pgmap v10297: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:51:04.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:51:03 smithi167 ceph-mon[37140]: pgmap v10297: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:51:05.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:51:05.596+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c006690 con 0x7fe5a81087a0 2024-07-26T18:51:05.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:51:05.596+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T18:51:05.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:51:05.596+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 142 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c006690 con 0x7fe5a81087a0 2024-07-26T18:51:05.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:51:05.596+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 142 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T18:51:05.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:51:05 smithi100 ceph-mon[32599]: pgmap v10298: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:51:06.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:51:05 smithi138 ceph-mon[36954]: pgmap v10298: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:51:06.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:51:05 smithi167 ceph-mon[37140]: pgmap v10298: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:51:08.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:51:07 smithi138 ceph-mon[36954]: pgmap v10299: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:51:08.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:51:07 smithi100 ceph-mon[32599]: pgmap v10299: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:51:08.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:51:07 smithi167 ceph-mon[37140]: pgmap v10299: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:51:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:51:08 smithi138 ceph-mon[36954]: osdmap e690: 12 total, 12 up, 12 in 2024-07-26T18:51:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:51:08 smithi100 ceph-mon[32599]: osdmap e690: 12 total, 12 up, 12 in 2024-07-26T18:51:09.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:51:08 smithi167 ceph-mon[37140]: osdmap e690: 12 total, 12 up, 12 in 2024-07-26T18:51:10.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:51:09 smithi138 ceph-mon[36954]: pgmap v10301: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:51:10.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:51:09 smithi100 ceph-mon[32599]: pgmap v10301: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:51:10.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:51:09 smithi167 ceph-mon[37140]: pgmap v10301: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:51:12.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:51:11 smithi167 ceph-mon[37140]: pgmap v10302: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:51:12.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:51:11 smithi138 ceph-mon[36954]: pgmap v10302: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:51:12.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:51:11 smithi100 ceph-mon[32599]: pgmap v10302: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:51:14.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:51:13 smithi138 ceph-mon[36954]: pgmap v10303: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:51:14.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:51:13 smithi100 ceph-mon[32599]: pgmap v10303: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:51:14.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:51:13 smithi167 ceph-mon[37140]: pgmap v10303: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:51:14.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:51:14.296+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:51:14.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:51:14.296+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:51:14.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:51:14.296+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:51:14.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:51:14.296+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:51:14.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:51:14.296+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:51:16.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:51:15 smithi138 ceph-mon[36954]: pgmap v10304: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:51:16.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:51:15 smithi100 ceph-mon[32599]: pgmap v10304: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:51:16.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:51:15 smithi167 ceph-mon[37140]: pgmap v10304: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:51:18.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:51:17 smithi138 ceph-mon[36954]: pgmap v10305: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:51:18.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:51:17 smithi100 ceph-mon[32599]: pgmap v10305: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:51:18.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:51:17 smithi167 ceph-mon[37140]: pgmap v10305: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:51:19.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:51:19 smithi100 ceph-mon[32599]: pgmap v10306: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 188 B/s wr, 7 op/s 2024-07-26T18:51:19.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:51:19 smithi100 ceph-mon[32599]: osdmap e691: 12 total, 12 up, 12 in 2024-07-26T18:51:20.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:51:19 smithi167 ceph-mon[37140]: pgmap v10306: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 188 B/s wr, 7 op/s 2024-07-26T18:51:20.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:51:19 smithi167 ceph-mon[37140]: osdmap e691: 12 total, 12 up, 12 in 2024-07-26T18:51:20.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:51:19 smithi138 ceph-mon[36954]: pgmap v10306: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 188 B/s wr, 7 op/s 2024-07-26T18:51:20.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:51:19 smithi138 ceph-mon[36954]: osdmap e691: 12 total, 12 up, 12 in 2024-07-26T18:51:21.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:51:21 smithi100 ceph-mon[32599]: pgmap v10308: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:51:22.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:51:21 smithi167 ceph-mon[37140]: pgmap v10308: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:51:22.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:51:21 smithi138 ceph-mon[36954]: pgmap v10308: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:51:23.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:51:23 smithi100 ceph-mon[32599]: pgmap v10309: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:51:24.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:51:23 smithi167 ceph-mon[37140]: pgmap v10309: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:51:24.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:51:23 smithi138 ceph-mon[36954]: pgmap v10309: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:51:25.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:51:25 smithi100 ceph-mon[32599]: pgmap v10310: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:51:26.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:51:25 smithi167 ceph-mon[37140]: pgmap v10310: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:51:26.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:51:25 smithi138 ceph-mon[36954]: pgmap v10310: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:51:28.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:51:27 smithi167 ceph-mon[37140]: pgmap v10311: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:51:28.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:51:27 smithi138 ceph-mon[36954]: pgmap v10311: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:51:28.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:51:27 smithi100 ceph-mon[32599]: pgmap v10311: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:51:29.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:51:29.299+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:51:29.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:51:29.300+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:51:29.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:51:29.300+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:51:29.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:51:29.300+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:51:29.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:51:29.300+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:51:30.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:51:29 smithi167 ceph-mon[37140]: pgmap v10312: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:51:30.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:51:29 smithi138 ceph-mon[36954]: pgmap v10312: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:51:30.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:51:29 smithi100 ceph-mon[32599]: pgmap v10312: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:51:32.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:51:31 smithi167 ceph-mon[37140]: pgmap v10313: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:51:32.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:51:31 smithi138 ceph-mon[36954]: pgmap v10313: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:51:32.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:51:31 smithi100 ceph-mon[32599]: pgmap v10313: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:51:34.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:51:33 smithi167 ceph-mon[37140]: pgmap v10314: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:51:34.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:51:33 smithi138 ceph-mon[36954]: pgmap v10314: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:51:34.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:51:33 smithi100 ceph-mon[32599]: pgmap v10314: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:51:36.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:51:35 smithi167 ceph-mon[37140]: pgmap v10315: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:51:36.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:51:35 smithi138 ceph-mon[36954]: pgmap v10315: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:51:36.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:51:35 smithi100 ceph-mon[32599]: pgmap v10315: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:51:38.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:51:37 smithi167 ceph-mon[37140]: pgmap v10316: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:51:38.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:51:37 smithi138 ceph-mon[36954]: pgmap v10316: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:51:38.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:51:37 smithi100 ceph-mon[32599]: pgmap v10316: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:51:40.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:51:39 smithi167 ceph-mon[37140]: pgmap v10317: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:51:40.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:51:39 smithi138 ceph-mon[36954]: pgmap v10317: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:51:40.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:51:39 smithi100 ceph-mon[32599]: pgmap v10317: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:51:42.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:51:41 smithi167 ceph-mon[37140]: pgmap v10318: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:51:42.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:51:41 smithi138 ceph-mon[36954]: pgmap v10318: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:51:42.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:51:41 smithi100 ceph-mon[32599]: pgmap v10318: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:51:44.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:51:43 smithi167 ceph-mon[37140]: pgmap v10319: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:51:44.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:51:43 smithi138 ceph-mon[36954]: pgmap v10319: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:51:44.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:51:43 smithi100 ceph-mon[32599]: pgmap v10319: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:51:44.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:51:44.314+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:51:44.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:51:44.315+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:51:44.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:51:44.315+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:51:44.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:51:44.315+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:51:44.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:51:44.315+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:51:46.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:51:45 smithi138 ceph-mon[36954]: pgmap v10320: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:51:46.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:51:45 smithi100 ceph-mon[32599]: pgmap v10320: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:51:46.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:51:45 smithi167 ceph-mon[37140]: pgmap v10320: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:51:48.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:51:47 smithi138 ceph-mon[36954]: pgmap v10321: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:51:48.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:51:47 smithi100 ceph-mon[32599]: pgmap v10321: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:51:48.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:51:47 smithi167 ceph-mon[37140]: pgmap v10321: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:51:50.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:51:49 smithi138 ceph-mon[36954]: pgmap v10322: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:51:50.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:51:49 smithi100 ceph-mon[32599]: pgmap v10322: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:51:50.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:51:49 smithi167 ceph-mon[37140]: pgmap v10322: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:51:52.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:51:51 smithi138 ceph-mon[36954]: pgmap v10323: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:51:52.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:51:51 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:51:52.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:51:51 smithi100 ceph-mon[32599]: pgmap v10323: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:51:52.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:51:51 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:51:52.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:51:51 smithi167 ceph-mon[37140]: pgmap v10323: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:51:52.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:51:51 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:51:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:51:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:51:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:51:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:51:53.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:51:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:51:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:51:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:51:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:51:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:51:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:51:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:51:53.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:51:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:51:53.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:51:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:51:53.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:51:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:51:54.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:51:53 smithi138 ceph-mon[36954]: pgmap v10324: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:51:54.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:51:53 smithi100 ceph-mon[32599]: pgmap v10324: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:51:54.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:51:53 smithi167 ceph-mon[37140]: pgmap v10324: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:51:56.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:51:55 smithi138 ceph-mon[36954]: pgmap v10325: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:51:56.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:51:55 smithi100 ceph-mon[32599]: pgmap v10325: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:51:56.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:51:55 smithi167 ceph-mon[37140]: pgmap v10325: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:51:58.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:51:57 smithi138 ceph-mon[36954]: pgmap v10326: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:51:58.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:51:57 smithi100 ceph-mon[32599]: pgmap v10326: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:51:58.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:51:57 smithi167 ceph-mon[37140]: pgmap v10326: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:51:59.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:51:59.330+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:51:59.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:51:59.331+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:51:59.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:51:59.331+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:51:59.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:51:59.331+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:51:59.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:51:59.331+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:52:00.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:51:59 smithi138 ceph-mon[36954]: pgmap v10327: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T18:52:00.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:51:59 smithi100 ceph-mon[32599]: pgmap v10327: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T18:52:00.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:51:59 smithi167 ceph-mon[37140]: pgmap v10327: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T18:52:02.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:52:01 smithi138 ceph-mon[36954]: pgmap v10328: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:52:02.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:52:01 smithi100 ceph-mon[32599]: pgmap v10328: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:52:02.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:52:01 smithi167 ceph-mon[37140]: pgmap v10328: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T18:52:04.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:52:03 smithi138 ceph-mon[36954]: pgmap v10329: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:52:04.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:52:03 smithi100 ceph-mon[32599]: pgmap v10329: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:52:04.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:52:03 smithi167 ceph-mon[37140]: pgmap v10329: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:52:06.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:52:05 smithi138 ceph-mon[36954]: pgmap v10330: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:52:06.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:52:05 smithi100 ceph-mon[32599]: pgmap v10330: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:52:06.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:52:05 smithi167 ceph-mon[37140]: pgmap v10330: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:52:08.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:52:07 smithi138 ceph-mon[36954]: pgmap v10331: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:52:08.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:52:07 smithi100 ceph-mon[32599]: pgmap v10331: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:52:08.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:52:07 smithi167 ceph-mon[37140]: pgmap v10331: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:52:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:52:08 smithi138 ceph-mon[36954]: osdmap e692: 12 total, 12 up, 12 in 2024-07-26T18:52:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:52:08 smithi100 ceph-mon[32599]: osdmap e692: 12 total, 12 up, 12 in 2024-07-26T18:52:09.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:52:08 smithi167 ceph-mon[37140]: osdmap e692: 12 total, 12 up, 12 in 2024-07-26T18:52:10.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:52:09 smithi138 ceph-mon[36954]: pgmap v10333: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:52:10.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:52:09 smithi100 ceph-mon[32599]: pgmap v10333: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:52:10.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:52:09 smithi167 ceph-mon[37140]: pgmap v10333: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:52:12.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:52:11 smithi138 ceph-mon[36954]: pgmap v10334: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:52:12.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:52:11 smithi100 ceph-mon[32599]: pgmap v10334: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:52:12.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:52:11 smithi167 ceph-mon[37140]: pgmap v10334: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:52:14.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:52:13 smithi138 ceph-mon[36954]: pgmap v10335: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:52:14.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:52:13 smithi100 ceph-mon[32599]: pgmap v10335: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:52:14.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:52:13 smithi167 ceph-mon[37140]: pgmap v10335: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:52:14.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:52:14.346+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:52:14.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:52:14.346+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:52:14.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:52:14.346+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:52:14.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:52:14.346+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:52:14.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:52:14.346+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:52:16.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:52:15 smithi138 ceph-mon[36954]: pgmap v10336: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:52:16.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:52:15 smithi100 ceph-mon[32599]: pgmap v10336: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:52:16.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:52:15 smithi167 ceph-mon[37140]: pgmap v10336: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:52:18.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:52:17 smithi138 ceph-mon[36954]: pgmap v10337: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:52:18.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:52:17 smithi100 ceph-mon[32599]: pgmap v10337: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:52:18.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:52:17 smithi167 ceph-mon[37140]: pgmap v10337: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:52:19.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:52:19 smithi100 ceph-mon[32599]: pgmap v10338: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 190 B/s wr, 7 op/s 2024-07-26T18:52:19.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:52:19 smithi100 ceph-mon[32599]: osdmap e693: 12 total, 12 up, 12 in 2024-07-26T18:52:20.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:52:19 smithi167 ceph-mon[37140]: pgmap v10338: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 190 B/s wr, 7 op/s 2024-07-26T18:52:20.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:52:19 smithi167 ceph-mon[37140]: osdmap e693: 12 total, 12 up, 12 in 2024-07-26T18:52:20.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:52:19 smithi138 ceph-mon[36954]: pgmap v10338: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 190 B/s wr, 7 op/s 2024-07-26T18:52:20.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:52:19 smithi138 ceph-mon[36954]: osdmap e693: 12 total, 12 up, 12 in 2024-07-26T18:52:22.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:52:21 smithi167 ceph-mon[37140]: pgmap v10340: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:52:22.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:52:21 smithi138 ceph-mon[36954]: pgmap v10340: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:52:22.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:52:21 smithi100 ceph-mon[32599]: pgmap v10340: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:52:24.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:52:23 smithi167 ceph-mon[37140]: pgmap v10341: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:52:24.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:52:23 smithi138 ceph-mon[36954]: pgmap v10341: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:52:24.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:52:23 smithi100 ceph-mon[32599]: pgmap v10341: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:52:26.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:52:25 smithi167 ceph-mon[37140]: pgmap v10342: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:52:26.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:52:25 smithi138 ceph-mon[36954]: pgmap v10342: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:52:26.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:52:25 smithi100 ceph-mon[32599]: pgmap v10342: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:52:28.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:52:27 smithi167 ceph-mon[37140]: pgmap v10343: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:52:28.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:52:27 smithi138 ceph-mon[36954]: pgmap v10343: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:52:28.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:52:27 smithi100 ceph-mon[32599]: pgmap v10343: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:52:29.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:52:29.361+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:52:29.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:52:29.362+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:52:29.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:52:29.362+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:52:29.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:52:29.362+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:52:29.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:52:29.362+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:52:30.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:52:29 smithi167 ceph-mon[37140]: pgmap v10344: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:52:30.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:52:29 smithi138 ceph-mon[36954]: pgmap v10344: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:52:30.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:52:29 smithi100 ceph-mon[32599]: pgmap v10344: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:52:32.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:52:31 smithi167 ceph-mon[37140]: pgmap v10345: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:52:32.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:52:31 smithi138 ceph-mon[36954]: pgmap v10345: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:52:32.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:52:31 smithi100 ceph-mon[32599]: pgmap v10345: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:52:34.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:52:33 smithi167 ceph-mon[37140]: pgmap v10346: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:52:34.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:52:33 smithi138 ceph-mon[36954]: pgmap v10346: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:52:34.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:52:33 smithi100 ceph-mon[32599]: pgmap v10346: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:52:36.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:52:35 smithi167 ceph-mon[37140]: pgmap v10347: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:52:36.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:52:35 smithi138 ceph-mon[36954]: pgmap v10347: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:52:36.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:52:35 smithi100 ceph-mon[32599]: pgmap v10347: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:52:38.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:52:37 smithi138 ceph-mon[36954]: pgmap v10348: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:52:38.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:52:37 smithi100 ceph-mon[32599]: pgmap v10348: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:52:38.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:52:37 smithi167 ceph-mon[37140]: pgmap v10348: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:52:40.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:52:39 smithi138 ceph-mon[36954]: pgmap v10349: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:52:40.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:52:39 smithi100 ceph-mon[32599]: pgmap v10349: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:52:40.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:52:39 smithi167 ceph-mon[37140]: pgmap v10349: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:52:42.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:52:41 smithi138 ceph-mon[36954]: pgmap v10350: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:52:42.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:52:41 smithi100 ceph-mon[32599]: pgmap v10350: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:52:42.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:52:41 smithi167 ceph-mon[37140]: pgmap v10350: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:52:44.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:52:43 smithi167 ceph-mon[37140]: pgmap v10351: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:52:44.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:52:43 smithi138 ceph-mon[36954]: pgmap v10351: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:52:44.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:52:43 smithi100 ceph-mon[32599]: pgmap v10351: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:52:44.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:52:44.370+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:52:44.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:52:44.370+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:52:44.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:52:44.370+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:52:44.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:52:44.370+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:52:44.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:52:44.370+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:52:46.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:52:45 smithi138 ceph-mon[36954]: pgmap v10352: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:52:46.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:52:45 smithi100 ceph-mon[32599]: pgmap v10352: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:52:46.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:52:45 smithi167 ceph-mon[37140]: pgmap v10352: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:52:48.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:52:47 smithi138 ceph-mon[36954]: pgmap v10353: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:52:48.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:52:47 smithi100 ceph-mon[32599]: pgmap v10353: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:52:48.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:52:47 smithi167 ceph-mon[37140]: pgmap v10353: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:52:50.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:52:49 smithi138 ceph-mon[36954]: pgmap v10354: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:52:50.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:52:49 smithi100 ceph-mon[32599]: pgmap v10354: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:52:50.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:52:49 smithi167 ceph-mon[37140]: pgmap v10354: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:52:52.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:52:51 smithi138 ceph-mon[36954]: pgmap v10355: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:52:52.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:52:51 smithi100 ceph-mon[32599]: pgmap v10355: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:52:52.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:52:51 smithi167 ceph-mon[37140]: pgmap v10355: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:52:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:52:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:52:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:52:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:52:53.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:52:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:52:53.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:52:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:52:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:52:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:52:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:52:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:52:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:52:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:52:53.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:52:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:52:53.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:52:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:52:53.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:52:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:52:53.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:52:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:52:53.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:52:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:52:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:52:54 smithi138 ceph-mon[36954]: pgmap v10356: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:52:54.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:52:54 smithi100 ceph-mon[32599]: pgmap v10356: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:52:54.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:52:54 smithi167 ceph-mon[37140]: pgmap v10356: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:52:56.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:52:56 smithi138 ceph-mon[36954]: pgmap v10357: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:52:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:52:56 smithi100 ceph-mon[32599]: pgmap v10357: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:52:56.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:52:56 smithi167 ceph-mon[37140]: pgmap v10357: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:52:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:52:58 smithi100 ceph-mon[32599]: pgmap v10358: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:52:58.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:52:58 smithi167 ceph-mon[37140]: pgmap v10358: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:52:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:52:58 smithi138 ceph-mon[36954]: pgmap v10358: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:52:59.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:52:59.385+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:52:59.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:52:59.386+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:52:59.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:52:59.386+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:52:59.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:52:59.386+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:52:59.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:52:59.386+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:53:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:00 smithi100 ceph-mon[32599]: pgmap v10359: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:53:00.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:00 smithi167 ceph-mon[37140]: pgmap v10359: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:53:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:00 smithi138 ceph-mon[36954]: pgmap v10359: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:53:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:02 smithi100 ceph-mon[32599]: pgmap v10360: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:53:02.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:02 smithi167 ceph-mon[37140]: pgmap v10360: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:53:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:02 smithi138 ceph-mon[36954]: pgmap v10360: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:53:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:04 smithi100 ceph-mon[32599]: pgmap v10361: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:53:04.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:04 smithi167 ceph-mon[37140]: pgmap v10361: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:53:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:04 smithi138 ceph-mon[36954]: pgmap v10361: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:53:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:06 smithi100 ceph-mon[32599]: pgmap v10362: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:53:06.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:06 smithi167 ceph-mon[37140]: pgmap v10362: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:53:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:06 smithi138 ceph-mon[36954]: pgmap v10362: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:53:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:08 smithi100 ceph-mon[32599]: pgmap v10363: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:53:08.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:08 smithi100 ceph-mon[32599]: osdmap e694: 12 total, 12 up, 12 in 2024-07-26T18:53:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:08 smithi167 ceph-mon[37140]: pgmap v10363: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:53:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:08 smithi167 ceph-mon[37140]: osdmap e694: 12 total, 12 up, 12 in 2024-07-26T18:53:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:08 smithi138 ceph-mon[36954]: pgmap v10363: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:53:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:08 smithi138 ceph-mon[36954]: osdmap e694: 12 total, 12 up, 12 in 2024-07-26T18:53:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:10 smithi100 ceph-mon[32599]: pgmap v10365: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:53:10.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:10 smithi167 ceph-mon[37140]: pgmap v10365: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:53:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:10 smithi138 ceph-mon[36954]: pgmap v10365: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:53:12.742 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:12 smithi138 ceph-mon[36954]: pgmap v10366: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:53:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:12 smithi100 ceph-mon[32599]: pgmap v10366: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:53:12.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:12 smithi167 ceph-mon[37140]: pgmap v10366: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:53:14.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:53:14.401+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:53:14.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:53:14.402+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:53:14.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:53:14.402+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:53:14.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:53:14.402+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:53:14.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:53:14.402+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:53:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:14 smithi100 ceph-mon[32599]: pgmap v10367: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:53:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:14 smithi167 ceph-mon[37140]: pgmap v10367: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:53:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:14 smithi138 ceph-mon[36954]: pgmap v10367: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:53:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:16 smithi100 ceph-mon[32599]: pgmap v10368: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:53:16.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:16 smithi167 ceph-mon[37140]: pgmap v10368: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:53:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:16 smithi138 ceph-mon[36954]: pgmap v10368: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:53:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:18 smithi100 ceph-mon[32599]: pgmap v10369: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:53:18.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:18 smithi167 ceph-mon[37140]: pgmap v10369: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:53:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:18 smithi138 ceph-mon[36954]: pgmap v10369: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:53:20.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:19 smithi167 ceph-mon[37140]: pgmap v10370: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 182 B/s wr, 7 op/s 2024-07-26T18:53:20.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:19 smithi167 ceph-mon[37140]: osdmap e695: 12 total, 12 up, 12 in 2024-07-26T18:53:20.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:19 smithi138 ceph-mon[36954]: pgmap v10370: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 182 B/s wr, 7 op/s 2024-07-26T18:53:20.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:19 smithi138 ceph-mon[36954]: osdmap e695: 12 total, 12 up, 12 in 2024-07-26T18:53:20.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:19 smithi100 ceph-mon[32599]: pgmap v10370: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 182 B/s wr, 7 op/s 2024-07-26T18:53:20.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:19 smithi100 ceph-mon[32599]: osdmap e695: 12 total, 12 up, 12 in 2024-07-26T18:53:20.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:53:20.614+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c006690 con 0x7fe5a81087a0 2024-07-26T18:53:20.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:53:20.614+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T18:53:20.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:53:20.615+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 143 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe57c006690 con 0x7fe5a81087a0 2024-07-26T18:53:20.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:53:20.615+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 143 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T18:53:22.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:21 smithi167 ceph-mon[37140]: pgmap v10372: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:53:22.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:21 smithi138 ceph-mon[36954]: pgmap v10372: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:53:22.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:21 smithi100 ceph-mon[32599]: pgmap v10372: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:53:24.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:23 smithi138 ceph-mon[36954]: pgmap v10373: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:53:24.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:23 smithi100 ceph-mon[32599]: pgmap v10373: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:53:24.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:23 smithi167 ceph-mon[37140]: pgmap v10373: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:53:26.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:25 smithi138 ceph-mon[36954]: pgmap v10374: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:53:26.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:25 smithi100 ceph-mon[32599]: pgmap v10374: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:53:26.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:25 smithi167 ceph-mon[37140]: pgmap v10374: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:53:28.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:27 smithi138 ceph-mon[36954]: pgmap v10375: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:53:28.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:27 smithi100 ceph-mon[32599]: pgmap v10375: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:53:28.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:27 smithi167 ceph-mon[37140]: pgmap v10375: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:53:29.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:53:29.417+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:53:29.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:53:29.418+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:53:29.419 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:53:29.418+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:53:29.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:53:29.418+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:53:29.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:53:29.418+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:53:30.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:29 smithi138 ceph-mon[36954]: pgmap v10376: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:53:30.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:29 smithi100 ceph-mon[32599]: pgmap v10376: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:53:30.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:29 smithi167 ceph-mon[37140]: pgmap v10376: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:53:32.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:31 smithi138 ceph-mon[36954]: pgmap v10377: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:53:32.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:31 smithi100 ceph-mon[32599]: pgmap v10377: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:53:32.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:31 smithi167 ceph-mon[37140]: pgmap v10377: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:53:34.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:33 smithi138 ceph-mon[36954]: pgmap v10378: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:53:34.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:33 smithi100 ceph-mon[32599]: pgmap v10378: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:53:34.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:33 smithi167 ceph-mon[37140]: pgmap v10378: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:53:36.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:35 smithi138 ceph-mon[36954]: pgmap v10379: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:53:36.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:35 smithi100 ceph-mon[32599]: pgmap v10379: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:53:36.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:35 smithi167 ceph-mon[37140]: pgmap v10379: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:53:38.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:37 smithi138 ceph-mon[36954]: pgmap v10380: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:53:38.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:37 smithi100 ceph-mon[32599]: pgmap v10380: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:53:38.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:37 smithi167 ceph-mon[37140]: pgmap v10380: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:53:40.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:39 smithi138 ceph-mon[36954]: pgmap v10381: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:53:40.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:39 smithi100 ceph-mon[32599]: pgmap v10381: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:53:40.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:39 smithi167 ceph-mon[37140]: pgmap v10381: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:53:42.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:41 smithi138 ceph-mon[36954]: pgmap v10382: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:53:42.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:41 smithi100 ceph-mon[32599]: pgmap v10382: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:53:42.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:41 smithi167 ceph-mon[37140]: pgmap v10382: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:53:44.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:43 smithi167 ceph-mon[37140]: pgmap v10383: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:53:44.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:43 smithi138 ceph-mon[36954]: pgmap v10383: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:53:44.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:43 smithi100 ceph-mon[32599]: pgmap v10383: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:53:44.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:53:44.434+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:53:44.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:53:44.434+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:53:44.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:53:44.434+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:53:44.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:53:44.434+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:53:44.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:53:44.434+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:53:46.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:45 smithi138 ceph-mon[36954]: pgmap v10384: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:53:46.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:45 smithi100 ceph-mon[32599]: pgmap v10384: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:53:46.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:45 smithi167 ceph-mon[37140]: pgmap v10384: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:53:48.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:47 smithi138 ceph-mon[36954]: pgmap v10385: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:53:48.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:47 smithi100 ceph-mon[32599]: pgmap v10385: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:53:48.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:47 smithi167 ceph-mon[37140]: pgmap v10385: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:53:50.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:49 smithi138 ceph-mon[36954]: pgmap v10386: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:53:50.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:49 smithi100 ceph-mon[32599]: pgmap v10386: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:53:50.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:49 smithi167 ceph-mon[37140]: pgmap v10386: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:53:52.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:51 smithi138 ceph-mon[36954]: pgmap v10387: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:53:52.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:51 smithi100 ceph-mon[32599]: pgmap v10387: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:53:52.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:51 smithi167 ceph-mon[37140]: pgmap v10387: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:53:53.142 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:52 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:53:53.162 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:52 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:53:53.169 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:52 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:53:54.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:53 smithi138 ceph-mon[36954]: pgmap v10388: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:53:54.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:53 smithi100 ceph-mon[32599]: pgmap v10388: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:53:54.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:53 smithi167 ceph-mon[37140]: pgmap v10388: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:53:55.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:53:55.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:53:55.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:53:55.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:53:55.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:53:55.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:53:55.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:53:55.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:53:55.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:53:55.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:53:55.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:53:55.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:53:55.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:53:55.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:53:55.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:53:55.443 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:53:55.443 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:53:55.443 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:53:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:53:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:53:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:53:55.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:53:55.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:53:55.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:53:55.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:53:55.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:53:55.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:53:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:56 smithi100 ceph-mon[32599]: pgmap v10389: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:53:56.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:56 smithi167 ceph-mon[37140]: pgmap v10389: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:53:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:56 smithi138 ceph-mon[36954]: pgmap v10389: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:53:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:53:58 smithi100 ceph-mon[32599]: pgmap v10390: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:53:58.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:53:58 smithi167 ceph-mon[37140]: pgmap v10390: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:53:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:53:58 smithi138 ceph-mon[36954]: pgmap v10390: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:53:59.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:53:59.448+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:53:59.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:53:59.448+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:53:59.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:53:59.448+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:53:59.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:53:59.448+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:53:59.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:53:59.448+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:54:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:00 smithi100 ceph-mon[32599]: pgmap v10391: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:54:00.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:00 smithi167 ceph-mon[37140]: pgmap v10391: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:54:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:00 smithi138 ceph-mon[36954]: pgmap v10391: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:54:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:02 smithi100 ceph-mon[32599]: pgmap v10392: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:54:02.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:02 smithi167 ceph-mon[37140]: pgmap v10392: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:54:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:02 smithi138 ceph-mon[36954]: pgmap v10392: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:54:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:04 smithi100 ceph-mon[32599]: pgmap v10393: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:54:04.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:04 smithi167 ceph-mon[37140]: pgmap v10393: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:54:04.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:04 smithi138 ceph-mon[36954]: pgmap v10393: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:54:05.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:54:05.622+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T18:54:05.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:54:05.622+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c006660 con 0x7fe5a81087a0 2024-07-26T18:54:05.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:54:05.622+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 144 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T18:54:05.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:54:05.622+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 144 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c006660 con 0x7fe5a81087a0 2024-07-26T18:54:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:06 smithi100 ceph-mon[32599]: pgmap v10394: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:54:06.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:06 smithi167 ceph-mon[37140]: pgmap v10394: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:54:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:06 smithi138 ceph-mon[36954]: pgmap v10394: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:54:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:08 smithi100 ceph-mon[32599]: pgmap v10395: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:54:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:08 smithi100 ceph-mon[32599]: osdmap e696: 12 total, 12 up, 12 in 2024-07-26T18:54:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:08 smithi167 ceph-mon[37140]: pgmap v10395: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:54:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:08 smithi167 ceph-mon[37140]: osdmap e696: 12 total, 12 up, 12 in 2024-07-26T18:54:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:08 smithi138 ceph-mon[36954]: pgmap v10395: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:54:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:08 smithi138 ceph-mon[36954]: osdmap e696: 12 total, 12 up, 12 in 2024-07-26T18:54:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:10 smithi100 ceph-mon[32599]: pgmap v10397: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:54:10.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:10 smithi167 ceph-mon[37140]: pgmap v10397: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:54:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:10 smithi138 ceph-mon[36954]: pgmap v10397: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:54:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:12 smithi100 ceph-mon[32599]: pgmap v10398: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:54:12.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:12 smithi167 ceph-mon[37140]: pgmap v10398: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:54:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:12 smithi138 ceph-mon[36954]: pgmap v10398: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:54:14.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:54:14.463+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:54:14.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:54:14.464+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:54:14.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:54:14.464+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:54:14.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:54:14.464+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:54:14.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:54:14.464+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:54:14.742 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:14 smithi138 ceph-mon[36954]: pgmap v10399: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:54:14.743 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:14 smithi138 ceph-mon[36954]: osdmap e697: 12 total, 12 up, 12 in 2024-07-26T18:54:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:14 smithi100 ceph-mon[32599]: pgmap v10399: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:54:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:14 smithi100 ceph-mon[32599]: osdmap e697: 12 total, 12 up, 12 in 2024-07-26T18:54:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:14 smithi167 ceph-mon[37140]: pgmap v10399: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:54:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:14 smithi167 ceph-mon[37140]: osdmap e697: 12 total, 12 up, 12 in 2024-07-26T18:54:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:16 smithi100 ceph-mon[32599]: pgmap v10401: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T18:54:16.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:16 smithi167 ceph-mon[37140]: pgmap v10401: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T18:54:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:16 smithi138 ceph-mon[36954]: pgmap v10401: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T18:54:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:18 smithi100 ceph-mon[32599]: pgmap v10402: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 221 B/s wr, 6 op/s 2024-07-26T18:54:18.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:18 smithi167 ceph-mon[37140]: pgmap v10402: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 221 B/s wr, 6 op/s 2024-07-26T18:54:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:18 smithi138 ceph-mon[36954]: pgmap v10402: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 221 B/s wr, 6 op/s 2024-07-26T18:54:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:20 smithi100 ceph-mon[32599]: pgmap v10403: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:54:20.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:20 smithi167 ceph-mon[37140]: pgmap v10403: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:54:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:20 smithi138 ceph-mon[36954]: pgmap v10403: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:54:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:22 smithi100 ceph-mon[32599]: pgmap v10404: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:54:22.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:22 smithi167 ceph-mon[37140]: pgmap v10404: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:54:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:22 smithi138 ceph-mon[36954]: pgmap v10404: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:54:24.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:24 smithi167 ceph-mon[37140]: pgmap v10405: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:54:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:24 smithi138 ceph-mon[36954]: pgmap v10405: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:54:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:24 smithi100 ceph-mon[32599]: pgmap v10405: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:54:26.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:26 smithi167 ceph-mon[37140]: pgmap v10406: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T18:54:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:26 smithi138 ceph-mon[36954]: pgmap v10406: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T18:54:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:26 smithi100 ceph-mon[32599]: pgmap v10406: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T18:54:28.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:28 smithi167 ceph-mon[37140]: pgmap v10407: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:54:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:28 smithi138 ceph-mon[36954]: pgmap v10407: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:54:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:28 smithi100 ceph-mon[32599]: pgmap v10407: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:54:29.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:54:29.476+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:54:29.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:54:29.476+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:54:29.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:54:29.476+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:54:29.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:54:29.476+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:54:29.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:54:29.476+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:54:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:30 smithi167 ceph-mon[37140]: pgmap v10408: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:54:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:30 smithi138 ceph-mon[36954]: pgmap v10408: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:54:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:30 smithi100 ceph-mon[32599]: pgmap v10408: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:54:32.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:32 smithi167 ceph-mon[37140]: pgmap v10409: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:54:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:32 smithi138 ceph-mon[36954]: pgmap v10409: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:54:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:32 smithi100 ceph-mon[32599]: pgmap v10409: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:54:34.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:34 smithi167 ceph-mon[37140]: pgmap v10410: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:54:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:34 smithi138 ceph-mon[36954]: pgmap v10410: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:54:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:34 smithi100 ceph-mon[32599]: pgmap v10410: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:54:36.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:36 smithi167 ceph-mon[37140]: pgmap v10411: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:54:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:36 smithi138 ceph-mon[36954]: pgmap v10411: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:54:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:36 smithi100 ceph-mon[32599]: pgmap v10411: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:54:38.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:38 smithi167 ceph-mon[37140]: pgmap v10412: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:54:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:38 smithi138 ceph-mon[36954]: pgmap v10412: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:54:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:38 smithi100 ceph-mon[32599]: pgmap v10412: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:54:40.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:40 smithi167 ceph-mon[37140]: pgmap v10413: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:54:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:40 smithi138 ceph-mon[36954]: pgmap v10413: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:54:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:40 smithi100 ceph-mon[32599]: pgmap v10413: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:54:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:42 smithi138 ceph-mon[36954]: pgmap v10414: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:54:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:42 smithi100 ceph-mon[32599]: pgmap v10414: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:54:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:42 smithi167 ceph-mon[37140]: pgmap v10414: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:54:44.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:54:44.492+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:54:44.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:54:44.493+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:54:44.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:54:44.493+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:54:44.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:54:44.493+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:54:44.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:54:44.493+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:54:44.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:44 smithi100 ceph-mon[32599]: pgmap v10415: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:54:44.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:44 smithi167 ceph-mon[37140]: pgmap v10415: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:54:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:44 smithi138 ceph-mon[36954]: pgmap v10415: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:54:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:46 smithi138 ceph-mon[36954]: pgmap v10416: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:54:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:46 smithi100 ceph-mon[32599]: pgmap v10416: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:54:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:46 smithi167 ceph-mon[37140]: pgmap v10416: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:54:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:48 smithi138 ceph-mon[36954]: pgmap v10417: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:54:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:48 smithi100 ceph-mon[32599]: pgmap v10417: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:54:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:48 smithi167 ceph-mon[37140]: pgmap v10417: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:54:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:50 smithi138 ceph-mon[36954]: pgmap v10418: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:54:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:50 smithi100 ceph-mon[32599]: pgmap v10418: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:54:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:50 smithi167 ceph-mon[37140]: pgmap v10418: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:54:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:52 smithi138 ceph-mon[36954]: pgmap v10419: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:54:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:52 smithi100 ceph-mon[32599]: pgmap v10419: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:54:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:52 smithi167 ceph-mon[37140]: pgmap v10419: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:54:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:54 smithi138 ceph-mon[36954]: pgmap v10420: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:54:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:54 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:54:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:54 smithi100 ceph-mon[32599]: pgmap v10420: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:54:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:54 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:54:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:54 smithi167 ceph-mon[37140]: pgmap v10420: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:54:55.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:54 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:54:56.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:55 smithi138 ceph-mon[36954]: pgmap v10421: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:54:56.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:54:56.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:54:56.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:54:56.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:54:56.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:54:56.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:54:56.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:54:56.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:54:56.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:54:56.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:55 smithi100 ceph-mon[32599]: pgmap v10421: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:54:56.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:54:56.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:54:56.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:54:56.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:54:56.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:54:56.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:54:56.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:54:56.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:54:56.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:54:56.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:55 smithi167 ceph-mon[37140]: pgmap v10421: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:54:56.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:54:56.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:54:56.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:54:56.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:54:56.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:54:56.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:54:56.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:54:56.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:54:56.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:54:58.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:57 smithi138 ceph-mon[36954]: pgmap v10422: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:54:58.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:57 smithi100 ceph-mon[32599]: pgmap v10422: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:54:58.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:57 smithi167 ceph-mon[37140]: pgmap v10422: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:54:59.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:54:59.495+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:54:59.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:54:59.496+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:54:59.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:54:59.496+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:54:59.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:54:59.496+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:54:59.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:54:59.496+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:55:00.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:54:59 smithi138 ceph-mon[36954]: pgmap v10423: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:55:00.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:54:59 smithi100 ceph-mon[32599]: pgmap v10423: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:55:00.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:54:59 smithi167 ceph-mon[37140]: pgmap v10423: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:55:02.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:55:01 smithi138 ceph-mon[36954]: pgmap v10424: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:55:02.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:55:01 smithi100 ceph-mon[32599]: pgmap v10424: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:55:02.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:55:01 smithi167 ceph-mon[37140]: pgmap v10424: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:55:04.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:55:03 smithi138 ceph-mon[36954]: pgmap v10425: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:55:04.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:55:03 smithi100 ceph-mon[32599]: pgmap v10425: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:55:04.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:55:03 smithi167 ceph-mon[37140]: pgmap v10425: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:55:06.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:55:05 smithi138 ceph-mon[36954]: pgmap v10426: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:55:06.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:55:05 smithi100 ceph-mon[32599]: pgmap v10426: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:55:06.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:55:05 smithi167 ceph-mon[37140]: pgmap v10426: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:55:08.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:55:07 smithi138 ceph-mon[36954]: pgmap v10427: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:55:08.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:55:07 smithi100 ceph-mon[32599]: pgmap v10427: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:55:08.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:55:07 smithi167 ceph-mon[37140]: pgmap v10427: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:55:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:55:08 smithi138 ceph-mon[36954]: osdmap e698: 12 total, 12 up, 12 in 2024-07-26T18:55:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:55:08 smithi100 ceph-mon[32599]: osdmap e698: 12 total, 12 up, 12 in 2024-07-26T18:55:09.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:55:08 smithi167 ceph-mon[37140]: osdmap e698: 12 total, 12 up, 12 in 2024-07-26T18:55:10.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:55:09 smithi138 ceph-mon[36954]: pgmap v10429: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:55:10.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:55:09 smithi100 ceph-mon[32599]: pgmap v10429: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:55:10.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:55:09 smithi167 ceph-mon[37140]: pgmap v10429: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:55:12.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:55:11 smithi138 ceph-mon[36954]: pgmap v10430: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:55:12.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:55:11 smithi100 ceph-mon[32599]: pgmap v10430: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:55:12.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:55:11 smithi167 ceph-mon[37140]: pgmap v10430: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:55:14.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:55:13 smithi138 ceph-mon[36954]: pgmap v10431: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:55:14.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:55:13 smithi100 ceph-mon[32599]: pgmap v10431: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:55:14.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:55:13 smithi167 ceph-mon[37140]: pgmap v10431: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:55:14.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:55:14.510+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:55:14.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:55:14.511+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:55:14.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:55:14.511+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:55:14.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:55:14.511+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:55:14.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:55:14.511+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:55:16.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:55:15 smithi138 ceph-mon[36954]: pgmap v10432: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:55:16.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:55:15 smithi100 ceph-mon[32599]: pgmap v10432: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:55:16.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:55:15 smithi167 ceph-mon[37140]: pgmap v10432: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:55:18.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:55:17 smithi138 ceph-mon[36954]: pgmap v10433: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:55:18.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:55:17 smithi100 ceph-mon[32599]: pgmap v10433: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:55:18.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:55:17 smithi167 ceph-mon[37140]: pgmap v10433: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:55:20.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:55:19 smithi138 ceph-mon[36954]: pgmap v10434: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 188 B/s wr, 7 op/s 2024-07-26T18:55:20.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:55:19 smithi138 ceph-mon[36954]: osdmap e699: 12 total, 12 up, 12 in 2024-07-26T18:55:20.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:55:19 smithi100 ceph-mon[32599]: pgmap v10434: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 188 B/s wr, 7 op/s 2024-07-26T18:55:20.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:55:19 smithi100 ceph-mon[32599]: osdmap e699: 12 total, 12 up, 12 in 2024-07-26T18:55:20.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:55:19 smithi167 ceph-mon[37140]: pgmap v10434: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 188 B/s wr, 7 op/s 2024-07-26T18:55:20.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:55:19 smithi167 ceph-mon[37140]: osdmap e699: 12 total, 12 up, 12 in 2024-07-26T18:55:22.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:55:21 smithi138 ceph-mon[36954]: pgmap v10436: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T18:55:22.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:55:21 smithi100 ceph-mon[32599]: pgmap v10436: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T18:55:22.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:55:21 smithi167 ceph-mon[37140]: pgmap v10436: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T18:55:24.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:55:23 smithi138 ceph-mon[36954]: pgmap v10437: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T18:55:24.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:55:23 smithi100 ceph-mon[32599]: pgmap v10437: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T18:55:24.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:55:23 smithi167 ceph-mon[37140]: pgmap v10437: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-07-26T18:55:26.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:55:25 smithi138 ceph-mon[36954]: pgmap v10438: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:55:26.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:55:25 smithi100 ceph-mon[32599]: pgmap v10438: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:55:26.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:55:25 smithi167 ceph-mon[37140]: pgmap v10438: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:55:28.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:55:27 smithi138 ceph-mon[36954]: pgmap v10439: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:55:28.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:55:27 smithi100 ceph-mon[32599]: pgmap v10439: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:55:28.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:55:27 smithi167 ceph-mon[37140]: pgmap v10439: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:55:29.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:55:29.525+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:55:29.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:55:29.525+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:55:29.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:55:29.525+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:55:29.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:55:29.525+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:55:29.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:55:29.525+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:55:30.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:55:29 smithi138 ceph-mon[36954]: pgmap v10440: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:55:30.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:55:29 smithi100 ceph-mon[32599]: pgmap v10440: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:55:30.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:55:29 smithi167 ceph-mon[37140]: pgmap v10440: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:55:32.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:55:31 smithi138 ceph-mon[36954]: pgmap v10441: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:55:32.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:55:31 smithi100 ceph-mon[32599]: pgmap v10441: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:55:32.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:55:31 smithi167 ceph-mon[37140]: pgmap v10441: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:55:34.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:55:33 smithi138 ceph-mon[36954]: pgmap v10442: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:55:34.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:55:33 smithi100 ceph-mon[32599]: pgmap v10442: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:55:34.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:55:33 smithi167 ceph-mon[37140]: pgmap v10442: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:55:36.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:55:35 smithi138 ceph-mon[36954]: pgmap v10443: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:55:36.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:55:35 smithi100 ceph-mon[32599]: pgmap v10443: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:55:36.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:55:35 smithi167 ceph-mon[37140]: pgmap v10443: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:55:38.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:55:37 smithi138 ceph-mon[36954]: pgmap v10444: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:55:38.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:55:37 smithi100 ceph-mon[32599]: pgmap v10444: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:55:38.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:55:37 smithi167 ceph-mon[37140]: pgmap v10444: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:55:40.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:55:39 smithi138 ceph-mon[36954]: pgmap v10445: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:55:40.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:55:39 smithi100 ceph-mon[32599]: pgmap v10445: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:55:40.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:55:39 smithi167 ceph-mon[37140]: pgmap v10445: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:55:42.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:55:41 smithi138 ceph-mon[36954]: pgmap v10446: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:55:42.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:55:41 smithi100 ceph-mon[32599]: pgmap v10446: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:55:42.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:55:41 smithi167 ceph-mon[37140]: pgmap v10446: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:55:44.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:55:43 smithi138 ceph-mon[36954]: pgmap v10447: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:55:44.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:55:43 smithi100 ceph-mon[32599]: pgmap v10447: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:55:44.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:55:43 smithi167 ceph-mon[37140]: pgmap v10447: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:55:44.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:55:44.541+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:55:44.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:55:44.541+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:55:44.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:55:44.541+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:55:44.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:55:44.541+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:55:44.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:55:44.541+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:55:46.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:55:45 smithi138 ceph-mon[36954]: pgmap v10448: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:55:46.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:55:45 smithi100 ceph-mon[32599]: pgmap v10448: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:55:46.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:55:45 smithi167 ceph-mon[37140]: pgmap v10448: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:55:48.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:55:47 smithi138 ceph-mon[36954]: pgmap v10449: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:55:48.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:55:47 smithi100 ceph-mon[32599]: pgmap v10449: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:55:48.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:55:47 smithi167 ceph-mon[37140]: pgmap v10449: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:55:50.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:55:49 smithi138 ceph-mon[36954]: pgmap v10450: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:55:50.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:55:49 smithi100 ceph-mon[32599]: pgmap v10450: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:55:50.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:55:49 smithi167 ceph-mon[37140]: pgmap v10450: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:55:52.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:55:51 smithi138 ceph-mon[36954]: pgmap v10451: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:55:52.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:55:51 smithi100 ceph-mon[32599]: pgmap v10451: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:55:52.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:55:51 smithi167 ceph-mon[37140]: pgmap v10451: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:55:54.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:55:53 smithi138 ceph-mon[36954]: pgmap v10452: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:55:54.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:55:53 smithi100 ceph-mon[32599]: pgmap v10452: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:55:54.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:55:53 smithi167 ceph-mon[37140]: pgmap v10452: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:55:56.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:55:55 smithi138 ceph-mon[36954]: pgmap v10453: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:55:56.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:55:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:55:56.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:55:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:55:56.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:55:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:55:56.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:55:55 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:55:56.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:55:55 smithi100 ceph-mon[32599]: pgmap v10453: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:55:56.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:55:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:55:56.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:55:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:55:56.247 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:55:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:55:56.247 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:55:55 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:55:56.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:55:55 smithi167 ceph-mon[37140]: pgmap v10453: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:55:56.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:55:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:55:56.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:55:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:55:56.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:55:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:55:56.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:55:55 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:55:58.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:55:57 smithi138 ceph-mon[36954]: pgmap v10454: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:55:58.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:55:57 smithi100 ceph-mon[32599]: pgmap v10454: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:55:58.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:55:57 smithi167 ceph-mon[37140]: pgmap v10454: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:55:59.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:55:59.557+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:55:59.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:55:59.558+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:55:59.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:55:59.558+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:55:59.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:55:59.558+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:55:59.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:55:59.558+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:56:00.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:55:59 smithi138 ceph-mon[36954]: pgmap v10455: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:56:00.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:55:59 smithi100 ceph-mon[32599]: pgmap v10455: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:56:00.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:55:59 smithi167 ceph-mon[37140]: pgmap v10455: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:56:02.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:56:01 smithi138 ceph-mon[36954]: pgmap v10456: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:56:02.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:56:01 smithi100 ceph-mon[32599]: pgmap v10456: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:56:02.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:56:01 smithi167 ceph-mon[37140]: pgmap v10456: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:56:04.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:56:03 smithi138 ceph-mon[36954]: pgmap v10457: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:56:04.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:56:03 smithi100 ceph-mon[32599]: pgmap v10457: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:56:04.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:56:03 smithi167 ceph-mon[37140]: pgmap v10457: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:56:06.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:56:05 smithi138 ceph-mon[36954]: pgmap v10458: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:56:06.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:56:05 smithi100 ceph-mon[32599]: pgmap v10458: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:56:06.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:56:05 smithi167 ceph-mon[37140]: pgmap v10458: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:56:08.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:56:07 smithi138 ceph-mon[36954]: pgmap v10459: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:56:08.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:56:07 smithi100 ceph-mon[32599]: pgmap v10459: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:56:08.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:56:07 smithi167 ceph-mon[37140]: pgmap v10459: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:56:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:56:08 smithi138 ceph-mon[36954]: osdmap e700: 12 total, 12 up, 12 in 2024-07-26T18:56:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:56:08 smithi100 ceph-mon[32599]: osdmap e700: 12 total, 12 up, 12 in 2024-07-26T18:56:09.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:56:08 smithi167 ceph-mon[37140]: osdmap e700: 12 total, 12 up, 12 in 2024-07-26T18:56:10.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:56:09 smithi138 ceph-mon[36954]: pgmap v10461: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:56:10.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:56:09 smithi100 ceph-mon[32599]: pgmap v10461: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:56:10.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:56:09 smithi167 ceph-mon[37140]: pgmap v10461: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:56:12.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:56:11 smithi138 ceph-mon[36954]: pgmap v10462: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:56:12.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:56:11 smithi100 ceph-mon[32599]: pgmap v10462: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:56:12.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:56:11 smithi167 ceph-mon[37140]: pgmap v10462: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:56:14.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:56:13 smithi138 ceph-mon[36954]: pgmap v10463: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:56:14.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:56:13 smithi100 ceph-mon[32599]: pgmap v10463: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:56:14.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:56:13 smithi167 ceph-mon[37140]: pgmap v10463: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:56:14.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:56:14.570+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:56:14.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:56:14.571+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:56:14.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:56:14.571+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:56:14.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:56:14.571+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:56:14.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:56:14.571+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:56:15.994 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:56:15 smithi138 ceph-mon[36954]: pgmap v10464: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:56:16.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:56:15 smithi100 ceph-mon[32599]: pgmap v10464: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:56:16.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:56:15 smithi167 ceph-mon[37140]: pgmap v10464: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:56:18.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:56:17 smithi138 ceph-mon[36954]: pgmap v10465: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:56:18.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:56:17 smithi100 ceph-mon[32599]: pgmap v10465: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:56:18.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:56:17 smithi167 ceph-mon[37140]: pgmap v10465: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:56:20.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:56:19 smithi138 ceph-mon[36954]: pgmap v10466: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 188 B/s wr, 7 op/s 2024-07-26T18:56:20.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:56:19 smithi138 ceph-mon[36954]: osdmap e701: 12 total, 12 up, 12 in 2024-07-26T18:56:20.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:56:19 smithi100 ceph-mon[32599]: pgmap v10466: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 188 B/s wr, 7 op/s 2024-07-26T18:56:20.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:56:19 smithi100 ceph-mon[32599]: osdmap e701: 12 total, 12 up, 12 in 2024-07-26T18:56:20.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:56:19 smithi167 ceph-mon[37140]: pgmap v10466: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 188 B/s wr, 7 op/s 2024-07-26T18:56:20.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:56:19 smithi167 ceph-mon[37140]: osdmap e701: 12 total, 12 up, 12 in 2024-07-26T18:56:22.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:56:21 smithi138 ceph-mon[36954]: pgmap v10468: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:56:22.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:56:21 smithi100 ceph-mon[32599]: pgmap v10468: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:56:22.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:56:21 smithi167 ceph-mon[37140]: pgmap v10468: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:56:24.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:56:23 smithi138 ceph-mon[36954]: pgmap v10469: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:56:24.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:56:23 smithi100 ceph-mon[32599]: pgmap v10469: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:56:24.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:56:23 smithi167 ceph-mon[37140]: pgmap v10469: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:56:26.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:56:25 smithi138 ceph-mon[36954]: pgmap v10470: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:56:26.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:56:25 smithi100 ceph-mon[32599]: pgmap v10470: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:56:26.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:56:25 smithi167 ceph-mon[37140]: pgmap v10470: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:56:28.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:56:27 smithi138 ceph-mon[36954]: pgmap v10471: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:56:28.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:56:27 smithi100 ceph-mon[32599]: pgmap v10471: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:56:28.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:56:27 smithi167 ceph-mon[37140]: pgmap v10471: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:56:29.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:56:29.587+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:56:29.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:56:29.587+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:56:29.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:56:29.587+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:56:29.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:56:29.587+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:56:29.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:56:29.587+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:56:29.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:56:29 smithi100 ceph-mon[32599]: pgmap v10472: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:56:30.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:56:29 smithi138 ceph-mon[36954]: pgmap v10472: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:56:30.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:56:29 smithi167 ceph-mon[37140]: pgmap v10472: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:56:32.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:56:31 smithi138 ceph-mon[36954]: pgmap v10473: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:56:32.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:56:31 smithi100 ceph-mon[32599]: pgmap v10473: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:56:32.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:56:31 smithi167 ceph-mon[37140]: pgmap v10473: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:56:34.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:56:33 smithi138 ceph-mon[36954]: pgmap v10474: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:56:34.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:56:33 smithi100 ceph-mon[32599]: pgmap v10474: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:56:34.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:56:33 smithi167 ceph-mon[37140]: pgmap v10474: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:56:36.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:56:35 smithi138 ceph-mon[36954]: pgmap v10475: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:56:36.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:56:35 smithi100 ceph-mon[32599]: pgmap v10475: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:56:36.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:56:35 smithi167 ceph-mon[37140]: pgmap v10475: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:56:38.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:56:37 smithi138 ceph-mon[36954]: pgmap v10476: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:56:38.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:56:37 smithi100 ceph-mon[32599]: pgmap v10476: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:56:38.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:56:37 smithi167 ceph-mon[37140]: pgmap v10476: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:56:40.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:56:39 smithi138 ceph-mon[36954]: pgmap v10477: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:56:40.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:56:39 smithi100 ceph-mon[32599]: pgmap v10477: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:56:40.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:56:39 smithi167 ceph-mon[37140]: pgmap v10477: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:56:42.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:56:41 smithi138 ceph-mon[36954]: pgmap v10478: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:56:42.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:56:41 smithi100 ceph-mon[32599]: pgmap v10478: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:56:42.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:56:41 smithi167 ceph-mon[37140]: pgmap v10478: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:56:44.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:56:43 smithi138 ceph-mon[36954]: pgmap v10479: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:56:44.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:56:43 smithi100 ceph-mon[32599]: pgmap v10479: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:56:44.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:56:43 smithi167 ceph-mon[37140]: pgmap v10479: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:56:44.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:56:44.602+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:56:44.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:56:44.603+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:56:44.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:56:44.603+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:56:44.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:56:44.603+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:56:44.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:56:44.603+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:56:46.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:56:45 smithi138 ceph-mon[36954]: pgmap v10480: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:56:46.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:56:45 smithi100 ceph-mon[32599]: pgmap v10480: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:56:46.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:56:45 smithi167 ceph-mon[37140]: pgmap v10480: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:56:48.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:56:47 smithi138 ceph-mon[36954]: pgmap v10481: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:56:48.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:56:47 smithi100 ceph-mon[32599]: pgmap v10481: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:56:48.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:56:47 smithi167 ceph-mon[37140]: pgmap v10481: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:56:50.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:56:49 smithi138 ceph-mon[36954]: pgmap v10482: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:56:50.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:56:49 smithi100 ceph-mon[32599]: pgmap v10482: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:56:50.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:56:49 smithi167 ceph-mon[37140]: pgmap v10482: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:56:52.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:56:51 smithi138 ceph-mon[36954]: pgmap v10483: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:56:52.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:56:51 smithi100 ceph-mon[32599]: pgmap v10483: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:56:52.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:56:51 smithi167 ceph-mon[37140]: pgmap v10483: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:56:54.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:56:53 smithi138 ceph-mon[36954]: pgmap v10484: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:56:54.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:56:53 smithi100 ceph-mon[32599]: pgmap v10484: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:56:54.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:56:53 smithi167 ceph-mon[37140]: pgmap v10484: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:56:56.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:56:55 smithi138 ceph-mon[36954]: pgmap v10485: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:56:56.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:56:55 smithi100 ceph-mon[32599]: pgmap v10485: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:56:56.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:56:55 smithi167 ceph-mon[37140]: pgmap v10485: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:56:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:56:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:56:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:56:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:56:57.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:56:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:56:57.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:56:56 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:56:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:56:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:56:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:56:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:56:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:56:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:56:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:56:56 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:56:57.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:56:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:56:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:56:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:56:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:56:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:56:57.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:56:56 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:56:58.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:56:58 smithi138 ceph-mon[36954]: pgmap v10486: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:56:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:56:58 smithi100 ceph-mon[32599]: pgmap v10486: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:56:58.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:56:58 smithi167 ceph-mon[37140]: pgmap v10486: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:56:59.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:56:59.615+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:56:59.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:56:59.616+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:56:59.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:56:59.616+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:56:59.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:56:59.616+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:56:59.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:56:59.616+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:57:00.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:57:00 smithi138 ceph-mon[36954]: pgmap v10487: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:57:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:57:00 smithi100 ceph-mon[32599]: pgmap v10487: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:57:00.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:57:00 smithi167 ceph-mon[37140]: pgmap v10487: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:57:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:57:02 smithi138 ceph-mon[36954]: pgmap v10488: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:57:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:57:02 smithi100 ceph-mon[32599]: pgmap v10488: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:57:02.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:57:02 smithi167 ceph-mon[37140]: pgmap v10488: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:57:04.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:57:04 smithi138 ceph-mon[36954]: pgmap v10489: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:57:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:57:04 smithi100 ceph-mon[32599]: pgmap v10489: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:57:04.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:57:04 smithi167 ceph-mon[37140]: pgmap v10489: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:57:05.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:57:05.648+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T18:57:05.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:57:05.648+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c006660 con 0x7fe5a81087a0 2024-07-26T18:57:05.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:57:05.648+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 145 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c006660 con 0x7fe5a81087a0 2024-07-26T18:57:05.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:57:05.648+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 145 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T18:57:06.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:57:06 smithi138 ceph-mon[36954]: pgmap v10490: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:57:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:57:06 smithi100 ceph-mon[32599]: pgmap v10490: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:57:06.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:57:06 smithi167 ceph-mon[37140]: pgmap v10490: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:57:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:57:08 smithi138 ceph-mon[36954]: pgmap v10491: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:57:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:57:08 smithi138 ceph-mon[36954]: osdmap e702: 12 total, 12 up, 12 in 2024-07-26T18:57:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:57:08 smithi100 ceph-mon[32599]: pgmap v10491: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:57:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:57:08 smithi100 ceph-mon[32599]: osdmap e702: 12 total, 12 up, 12 in 2024-07-26T18:57:08.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:57:08 smithi167 ceph-mon[37140]: pgmap v10491: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:57:08.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:57:08 smithi167 ceph-mon[37140]: osdmap e702: 12 total, 12 up, 12 in 2024-07-26T18:57:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:57:10 smithi138 ceph-mon[36954]: pgmap v10493: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:57:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:57:10 smithi100 ceph-mon[32599]: pgmap v10493: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:57:10.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:57:10 smithi167 ceph-mon[37140]: pgmap v10493: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:57:12.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:57:12 smithi138 ceph-mon[36954]: pgmap v10494: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:57:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:57:12 smithi100 ceph-mon[32599]: pgmap v10494: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:57:12.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:57:12 smithi167 ceph-mon[37140]: pgmap v10494: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:57:14.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:57:14.631+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:57:14.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:57:14.632+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:57:14.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:57:14.632+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:57:14.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:57:14.632+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:57:14.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:57:14.632+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:57:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:57:14 smithi138 ceph-mon[36954]: pgmap v10495: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:57:14.708 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:57:14 smithi138 ceph-mon[36954]: osdmap e703: 12 total, 12 up, 12 in 2024-07-26T18:57:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:57:14 smithi100 ceph-mon[32599]: pgmap v10495: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:57:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:57:14 smithi100 ceph-mon[32599]: osdmap e703: 12 total, 12 up, 12 in 2024-07-26T18:57:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:57:14 smithi167 ceph-mon[37140]: pgmap v10495: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:57:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:57:14 smithi167 ceph-mon[37140]: osdmap e703: 12 total, 12 up, 12 in 2024-07-26T18:57:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:57:16 smithi138 ceph-mon[36954]: pgmap v10497: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T18:57:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:57:16 smithi100 ceph-mon[32599]: pgmap v10497: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T18:57:16.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:57:16 smithi167 ceph-mon[37140]: pgmap v10497: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T18:57:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:57:18 smithi100 ceph-mon[32599]: pgmap v10498: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 218 B/s wr, 6 op/s 2024-07-26T18:57:18.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:57:18 smithi167 ceph-mon[37140]: pgmap v10498: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 218 B/s wr, 6 op/s 2024-07-26T18:57:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:57:18 smithi138 ceph-mon[36954]: pgmap v10498: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 218 B/s wr, 6 op/s 2024-07-26T18:57:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:57:20 smithi100 ceph-mon[32599]: pgmap v10499: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:57:20.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:57:20 smithi167 ceph-mon[37140]: pgmap v10499: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:57:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:57:20 smithi138 ceph-mon[36954]: pgmap v10499: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:57:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:57:22 smithi100 ceph-mon[32599]: pgmap v10500: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:57:22.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:57:22 smithi167 ceph-mon[37140]: pgmap v10500: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:57:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:57:22 smithi138 ceph-mon[36954]: pgmap v10500: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:57:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:57:24 smithi100 ceph-mon[32599]: pgmap v10501: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:57:24.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:57:24 smithi167 ceph-mon[37140]: pgmap v10501: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:57:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:57:24 smithi138 ceph-mon[36954]: pgmap v10501: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:57:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:57:26 smithi100 ceph-mon[32599]: pgmap v10502: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T18:57:26.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:57:26 smithi167 ceph-mon[37140]: pgmap v10502: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T18:57:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:57:26 smithi138 ceph-mon[36954]: pgmap v10502: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T18:57:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:57:28 smithi100 ceph-mon[32599]: pgmap v10503: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:57:28.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:57:28 smithi167 ceph-mon[37140]: pgmap v10503: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:57:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:57:28 smithi138 ceph-mon[36954]: pgmap v10503: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:57:29.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:57:29.635+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:57:29.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:57:29.636+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:57:29.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:57:29.636+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:57:29.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:57:29.636+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:57:29.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:57:29.636+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:57:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:57:30 smithi100 ceph-mon[32599]: pgmap v10504: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:57:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:57:30 smithi167 ceph-mon[37140]: pgmap v10504: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:57:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:57:30 smithi138 ceph-mon[36954]: pgmap v10504: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:57:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:57:32 smithi100 ceph-mon[32599]: pgmap v10505: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:57:32.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:57:32 smithi167 ceph-mon[37140]: pgmap v10505: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:57:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:57:32 smithi138 ceph-mon[36954]: pgmap v10505: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:57:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:57:34 smithi100 ceph-mon[32599]: pgmap v10506: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:57:34.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:57:34 smithi167 ceph-mon[37140]: pgmap v10506: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:57:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:57:34 smithi138 ceph-mon[36954]: pgmap v10506: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:57:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:57:36 smithi100 ceph-mon[32599]: pgmap v10507: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:57:36.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:57:36 smithi167 ceph-mon[37140]: pgmap v10507: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:57:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:57:36 smithi138 ceph-mon[36954]: pgmap v10507: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:57:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:57:38 smithi100 ceph-mon[32599]: pgmap v10508: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:57:38.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:57:38 smithi167 ceph-mon[37140]: pgmap v10508: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:57:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:57:38 smithi138 ceph-mon[36954]: pgmap v10508: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:57:40.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:57:40 smithi100 ceph-mon[32599]: pgmap v10509: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:57:40.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:57:40 smithi167 ceph-mon[37140]: pgmap v10509: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:57:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:57:40 smithi138 ceph-mon[36954]: pgmap v10509: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:57:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:57:42 smithi100 ceph-mon[32599]: pgmap v10510: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:57:42.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:57:42 smithi167 ceph-mon[37140]: pgmap v10510: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:57:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:57:42 smithi138 ceph-mon[36954]: pgmap v10510: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:57:44.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:57:44.639+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:57:44.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:57:44.639+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:57:44.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:57:44.639+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:57:44.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:57:44.639+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:57:44.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:57:44.639+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:57:44.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:57:44 smithi100 ceph-mon[32599]: pgmap v10511: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:57:44.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:57:44 smithi167 ceph-mon[37140]: pgmap v10511: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:57:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:57:44 smithi138 ceph-mon[36954]: pgmap v10511: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:57:46.742 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:57:46 smithi138 ceph-mon[36954]: pgmap v10512: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:57:46.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:57:46 smithi100 ceph-mon[32599]: pgmap v10512: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:57:46.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:57:46 smithi167 ceph-mon[37140]: pgmap v10512: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:57:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:57:48 smithi100 ceph-mon[32599]: pgmap v10513: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:57:48.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:57:48 smithi167 ceph-mon[37140]: pgmap v10513: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:57:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:57:48 smithi138 ceph-mon[36954]: pgmap v10513: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:57:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:57:50 smithi100 ceph-mon[32599]: pgmap v10514: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:57:50.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:57:50 smithi167 ceph-mon[37140]: pgmap v10514: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:57:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:57:50 smithi138 ceph-mon[36954]: pgmap v10514: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:57:52.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:57:52 smithi167 ceph-mon[37140]: pgmap v10515: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:57:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:57:52 smithi138 ceph-mon[36954]: pgmap v10515: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:57:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:57:52 smithi100 ceph-mon[32599]: pgmap v10515: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:57:54.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:57:54 smithi167 ceph-mon[37140]: pgmap v10516: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:57:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:57:54 smithi138 ceph-mon[36954]: pgmap v10516: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:57:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:57:54 smithi100 ceph-mon[32599]: pgmap v10516: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:57:56.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:57:56 smithi167 ceph-mon[37140]: pgmap v10517: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:57:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:57:56 smithi138 ceph-mon[36954]: pgmap v10517: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:57:56.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:57:56 smithi100 ceph-mon[32599]: pgmap v10517: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:57:57.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:57:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:57:57.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:57:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:57:57.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:57:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:57:57.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:57:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:57:57.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:57:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:57:57.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:57:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:57:57.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:57:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:57:57.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:57:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:57:57.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:57:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:57:57.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:57:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:57:57.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:57:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:57:57.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:57:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:57:58.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:57:58 smithi167 ceph-mon[37140]: pgmap v10518: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:57:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:57:58 smithi138 ceph-mon[36954]: pgmap v10518: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:57:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:57:58 smithi100 ceph-mon[32599]: pgmap v10518: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:57:59.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:57:59.644+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:57:59.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:57:59.645+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:57:59.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:57:59.645+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:57:59.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:57:59.645+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:57:59.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:57:59.645+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:58:00.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:58:00 smithi167 ceph-mon[37140]: pgmap v10519: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:58:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:58:00 smithi138 ceph-mon[36954]: pgmap v10519: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:58:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:58:00 smithi100 ceph-mon[32599]: pgmap v10519: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:58:02.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:58:02 smithi167 ceph-mon[37140]: pgmap v10520: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:58:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:58:02 smithi138 ceph-mon[36954]: pgmap v10520: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:58:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:58:02 smithi100 ceph-mon[32599]: pgmap v10520: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:58:04.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:58:04 smithi167 ceph-mon[37140]: pgmap v10521: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:58:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:58:04 smithi138 ceph-mon[36954]: pgmap v10521: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:58:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:58:04 smithi100 ceph-mon[32599]: pgmap v10521: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:58:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:58:06 smithi138 ceph-mon[36954]: pgmap v10522: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:58:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:58:06 smithi100 ceph-mon[32599]: pgmap v10522: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:58:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:58:06 smithi167 ceph-mon[37140]: pgmap v10522: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:58:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:58:08 smithi138 ceph-mon[36954]: pgmap v10523: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:58:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:58:08 smithi138 ceph-mon[36954]: osdmap e704: 12 total, 12 up, 12 in 2024-07-26T18:58:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:58:08 smithi100 ceph-mon[32599]: pgmap v10523: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:58:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:58:08 smithi100 ceph-mon[32599]: osdmap e704: 12 total, 12 up, 12 in 2024-07-26T18:58:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:58:08 smithi167 ceph-mon[37140]: pgmap v10523: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:58:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:58:08 smithi167 ceph-mon[37140]: osdmap e704: 12 total, 12 up, 12 in 2024-07-26T18:58:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:58:10 smithi138 ceph-mon[36954]: pgmap v10525: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:58:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:58:10 smithi100 ceph-mon[32599]: pgmap v10525: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:58:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:58:10 smithi167 ceph-mon[37140]: pgmap v10525: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:58:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:58:12 smithi138 ceph-mon[36954]: pgmap v10526: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:58:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:58:12 smithi100 ceph-mon[32599]: pgmap v10526: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:58:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:58:12 smithi167 ceph-mon[37140]: pgmap v10526: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:58:14.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:58:14.659+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:58:14.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:58:14.659+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:58:14.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:58:14.659+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:58:14.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:58:14.659+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:58:14.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:58:14.659+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:58:14.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:58:14 smithi167 ceph-mon[37140]: pgmap v10527: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:58:14.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:58:14 smithi167 ceph-mon[37140]: osdmap e705: 12 total, 12 up, 12 in 2024-07-26T18:58:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:58:14 smithi138 ceph-mon[36954]: pgmap v10527: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:58:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:58:14 smithi138 ceph-mon[36954]: osdmap e705: 12 total, 12 up, 12 in 2024-07-26T18:58:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:58:14 smithi100 ceph-mon[32599]: pgmap v10527: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:58:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:58:14 smithi100 ceph-mon[32599]: osdmap e705: 12 total, 12 up, 12 in 2024-07-26T18:58:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:58:16 smithi138 ceph-mon[36954]: pgmap v10529: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T18:58:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:58:16 smithi100 ceph-mon[32599]: pgmap v10529: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T18:58:17.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:58:16 smithi167 ceph-mon[37140]: pgmap v10529: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T18:58:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:58:18 smithi138 ceph-mon[36954]: pgmap v10530: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 221 B/s wr, 6 op/s 2024-07-26T18:58:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:58:18 smithi100 ceph-mon[32599]: pgmap v10530: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 221 B/s wr, 6 op/s 2024-07-26T18:58:19.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:58:18 smithi167 ceph-mon[37140]: pgmap v10530: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 221 B/s wr, 6 op/s 2024-07-26T18:58:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:58:20 smithi138 ceph-mon[36954]: pgmap v10531: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:58:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:58:20 smithi100 ceph-mon[32599]: pgmap v10531: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:58:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:58:20 smithi167 ceph-mon[37140]: pgmap v10531: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:58:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:58:22 smithi138 ceph-mon[36954]: pgmap v10532: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:58:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:58:22 smithi100 ceph-mon[32599]: pgmap v10532: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:58:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:58:22 smithi167 ceph-mon[37140]: pgmap v10532: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:58:24.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:58:24 smithi138 ceph-mon[36954]: pgmap v10533: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:58:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:58:24 smithi100 ceph-mon[32599]: pgmap v10533: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:58:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:58:24 smithi167 ceph-mon[37140]: pgmap v10533: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:58:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:58:26 smithi138 ceph-mon[36954]: pgmap v10534: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T18:58:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:58:26 smithi100 ceph-mon[32599]: pgmap v10534: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T18:58:27.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:58:26 smithi167 ceph-mon[37140]: pgmap v10534: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T18:58:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:58:28 smithi138 ceph-mon[36954]: pgmap v10535: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:58:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:58:28 smithi100 ceph-mon[32599]: pgmap v10535: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:58:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:58:28 smithi167 ceph-mon[37140]: pgmap v10535: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:58:29.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:58:29.675+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:58:29.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:58:29.675+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:58:29.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:58:29.675+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:58:29.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:58:29.675+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:58:29.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:58:29.675+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:58:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:58:30 smithi138 ceph-mon[36954]: pgmap v10536: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T18:58:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:58:30 smithi100 ceph-mon[32599]: pgmap v10536: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T18:58:31.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:58:30 smithi167 ceph-mon[37140]: pgmap v10536: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T18:58:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:58:32 smithi138 ceph-mon[36954]: pgmap v10537: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:58:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:58:32 smithi100 ceph-mon[32599]: pgmap v10537: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:58:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:58:32 smithi167 ceph-mon[37140]: pgmap v10537: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:58:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:58:34 smithi138 ceph-mon[36954]: pgmap v10538: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:58:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:58:34 smithi100 ceph-mon[32599]: pgmap v10538: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:58:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:58:34 smithi167 ceph-mon[37140]: pgmap v10538: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:58:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:58:36 smithi138 ceph-mon[36954]: pgmap v10539: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:58:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:58:36 smithi100 ceph-mon[32599]: pgmap v10539: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:58:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:58:36 smithi167 ceph-mon[37140]: pgmap v10539: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:58:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:58:38 smithi138 ceph-mon[36954]: pgmap v10540: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:58:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:58:38 smithi100 ceph-mon[32599]: pgmap v10540: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:58:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:58:38 smithi167 ceph-mon[37140]: pgmap v10540: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:58:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:58:40 smithi138 ceph-mon[36954]: pgmap v10541: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T18:58:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:58:40 smithi100 ceph-mon[32599]: pgmap v10541: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T18:58:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:58:40 smithi167 ceph-mon[37140]: pgmap v10541: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T18:58:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:58:42 smithi138 ceph-mon[36954]: pgmap v10542: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:58:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:58:42 smithi100 ceph-mon[32599]: pgmap v10542: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:58:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:58:42 smithi167 ceph-mon[37140]: pgmap v10542: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:58:44.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:58:44.691+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:58:44.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:58:44.692+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:58:44.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:58:44.692+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:58:44.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:58:44.692+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:58:44.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:58:44.692+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:58:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:58:44 smithi138 ceph-mon[36954]: pgmap v10543: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:58:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:58:44 smithi100 ceph-mon[32599]: pgmap v10543: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:58:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:58:44 smithi167 ceph-mon[37140]: pgmap v10543: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:58:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:58:46 smithi138 ceph-mon[36954]: pgmap v10544: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:58:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:58:46 smithi100 ceph-mon[32599]: pgmap v10544: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:58:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:58:46 smithi167 ceph-mon[37140]: pgmap v10544: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:58:47.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:58:47 smithi138 ceph-mon[36954]: pgmap v10545: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:58:47.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:58:47 smithi100 ceph-mon[32599]: pgmap v10545: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:58:48.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:58:47 smithi167 ceph-mon[37140]: pgmap v10545: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:58:50.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:58:49 smithi138 ceph-mon[36954]: pgmap v10546: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:58:50.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:58:49 smithi100 ceph-mon[32599]: pgmap v10546: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:58:50.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:58:49 smithi167 ceph-mon[37140]: pgmap v10546: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:58:52.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:58:51 smithi138 ceph-mon[36954]: pgmap v10547: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:58:52.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:58:51 smithi100 ceph-mon[32599]: pgmap v10547: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:58:52.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:58:51 smithi167 ceph-mon[37140]: pgmap v10547: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:58:54.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:58:53 smithi138 ceph-mon[36954]: pgmap v10548: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:58:54.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:58:53 smithi100 ceph-mon[32599]: pgmap v10548: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:58:54.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:58:53 smithi167 ceph-mon[37140]: pgmap v10548: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:58:56.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:58:55 smithi138 ceph-mon[36954]: pgmap v10549: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:58:56.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:58:55 smithi100 ceph-mon[32599]: pgmap v10549: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:58:56.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:58:55 smithi167 ceph-mon[37140]: pgmap v10549: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:58:58.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:58:57 smithi138 ceph-mon[36954]: pgmap v10550: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:58:58.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:58:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:58:58.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:58:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:58:58.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:58:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:58:58.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:58:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:58:58.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:58:57 smithi100 ceph-mon[32599]: pgmap v10550: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:58:58.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:58:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:58:58.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:58:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:58:58.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:58:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:58:58.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:58:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:58:58.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:58:57 smithi167 ceph-mon[37140]: pgmap v10550: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:58:58.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:58:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:58:58.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:58:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:58:58.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:58:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:58:58.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:58:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:58:59.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:58:59.707+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:58:59.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:58:59.708+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:58:59.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:58:59.708+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:58:59.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:58:59.708+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:58:59.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:58:59.708+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:58:59.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:58:59 smithi100 ceph-mon[32599]: pgmap v10551: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:59:00.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:58:59 smithi138 ceph-mon[36954]: pgmap v10551: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:59:00.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:58:59 smithi167 ceph-mon[37140]: pgmap v10551: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:59:02.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:59:01 smithi138 ceph-mon[36954]: pgmap v10552: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:59:02.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:59:01 smithi100 ceph-mon[32599]: pgmap v10552: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:59:02.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:59:01 smithi167 ceph-mon[37140]: pgmap v10552: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:59:04.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:59:03 smithi138 ceph-mon[36954]: pgmap v10553: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:59:04.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:59:03 smithi100 ceph-mon[32599]: pgmap v10553: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:59:04.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:59:03 smithi167 ceph-mon[37140]: pgmap v10553: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T18:59:06.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:59:05 smithi138 ceph-mon[36954]: pgmap v10554: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:59:06.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:59:05 smithi100 ceph-mon[32599]: pgmap v10554: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:59:06.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:59:05 smithi167 ceph-mon[37140]: pgmap v10554: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T18:59:08.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:59:07 smithi138 ceph-mon[36954]: pgmap v10555: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:59:08.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:59:07 smithi100 ceph-mon[32599]: pgmap v10555: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:59:08.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:59:07 smithi167 ceph-mon[37140]: pgmap v10555: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T18:59:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:59:08 smithi138 ceph-mon[36954]: osdmap e706: 12 total, 12 up, 12 in 2024-07-26T18:59:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:59:08 smithi100 ceph-mon[32599]: osdmap e706: 12 total, 12 up, 12 in 2024-07-26T18:59:09.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:59:08 smithi167 ceph-mon[37140]: osdmap e706: 12 total, 12 up, 12 in 2024-07-26T18:59:10.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:59:09 smithi138 ceph-mon[36954]: pgmap v10557: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:59:10.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:59:09 smithi100 ceph-mon[32599]: pgmap v10557: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:59:10.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:59:09 smithi167 ceph-mon[37140]: pgmap v10557: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:59:12.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:59:11 smithi138 ceph-mon[36954]: pgmap v10558: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:59:12.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:59:11 smithi100 ceph-mon[32599]: pgmap v10558: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:59:12.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:59:11 smithi167 ceph-mon[37140]: pgmap v10558: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T18:59:14.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:59:13 smithi138 ceph-mon[36954]: pgmap v10559: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:59:14.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:59:13 smithi100 ceph-mon[32599]: pgmap v10559: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:59:14.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:59:13 smithi167 ceph-mon[37140]: pgmap v10559: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T18:59:14.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:59:14.723+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:59:14.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:59:14.724+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:59:14.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:59:14.724+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:59:14.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:59:14.724+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:59:14.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:59:14.724+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:59:16.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:59:15 smithi138 ceph-mon[36954]: pgmap v10560: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:59:16.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:59:15 smithi100 ceph-mon[32599]: pgmap v10560: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:59:16.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:59:15 smithi167 ceph-mon[37140]: pgmap v10560: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:59:18.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:59:17 smithi138 ceph-mon[36954]: pgmap v10561: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:59:18.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:59:17 smithi100 ceph-mon[32599]: pgmap v10561: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:59:18.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:59:17 smithi167 ceph-mon[37140]: pgmap v10561: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T18:59:20.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:59:19 smithi138 ceph-mon[36954]: pgmap v10562: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 187 B/s wr, 7 op/s 2024-07-26T18:59:20.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:59:19 smithi138 ceph-mon[36954]: osdmap e707: 12 total, 12 up, 12 in 2024-07-26T18:59:20.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:59:19 smithi100 ceph-mon[32599]: pgmap v10562: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 187 B/s wr, 7 op/s 2024-07-26T18:59:20.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:59:19 smithi100 ceph-mon[32599]: osdmap e707: 12 total, 12 up, 12 in 2024-07-26T18:59:20.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:59:19 smithi167 ceph-mon[37140]: pgmap v10562: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 187 B/s wr, 7 op/s 2024-07-26T18:59:20.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:59:19 smithi167 ceph-mon[37140]: osdmap e707: 12 total, 12 up, 12 in 2024-07-26T18:59:22.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:59:21 smithi138 ceph-mon[36954]: pgmap v10564: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:59:22.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:59:21 smithi100 ceph-mon[32599]: pgmap v10564: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:59:22.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:59:21 smithi167 ceph-mon[37140]: pgmap v10564: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:59:24.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:59:23 smithi138 ceph-mon[36954]: pgmap v10565: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:59:24.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:59:23 smithi100 ceph-mon[32599]: pgmap v10565: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:59:24.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:59:23 smithi167 ceph-mon[37140]: pgmap v10565: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T18:59:26.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:59:25 smithi138 ceph-mon[36954]: pgmap v10566: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:59:26.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:59:25 smithi100 ceph-mon[32599]: pgmap v10566: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:59:26.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:59:25 smithi167 ceph-mon[37140]: pgmap v10566: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:59:28.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:59:27 smithi138 ceph-mon[36954]: pgmap v10567: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:59:28.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:59:27 smithi100 ceph-mon[32599]: pgmap v10567: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:59:28.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:59:27 smithi167 ceph-mon[37140]: pgmap v10567: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:59:29.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:59:29.738+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:59:29.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:59:29.739+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:59:29.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:59:29.739+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:59:29.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:59:29.739+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:59:29.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:59:29.739+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:59:29.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:59:29 smithi100 ceph-mon[32599]: pgmap v10568: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:59:30.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:59:29 smithi138 ceph-mon[36954]: pgmap v10568: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:59:30.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:59:29 smithi167 ceph-mon[37140]: pgmap v10568: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T18:59:32.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:59:31 smithi138 ceph-mon[36954]: pgmap v10569: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:59:32.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:59:31 smithi100 ceph-mon[32599]: pgmap v10569: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:59:32.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:59:31 smithi167 ceph-mon[37140]: pgmap v10569: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:59:34.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:59:33 smithi138 ceph-mon[36954]: pgmap v10570: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:59:34.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:59:33 smithi100 ceph-mon[32599]: pgmap v10570: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:59:34.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:59:33 smithi167 ceph-mon[37140]: pgmap v10570: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:59:36.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:59:35 smithi138 ceph-mon[36954]: pgmap v10571: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:59:36.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:59:35 smithi100 ceph-mon[32599]: pgmap v10571: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:59:36.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:59:35 smithi167 ceph-mon[37140]: pgmap v10571: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:59:38.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:59:37 smithi138 ceph-mon[36954]: pgmap v10572: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:59:38.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:59:37 smithi100 ceph-mon[32599]: pgmap v10572: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:59:38.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:59:37 smithi167 ceph-mon[37140]: pgmap v10572: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:59:40.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:59:39 smithi138 ceph-mon[36954]: pgmap v10573: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:59:40.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:59:39 smithi100 ceph-mon[32599]: pgmap v10573: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:59:40.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:59:39 smithi167 ceph-mon[37140]: pgmap v10573: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:59:42.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:59:41 smithi138 ceph-mon[36954]: pgmap v10574: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:59:42.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:59:41 smithi100 ceph-mon[32599]: pgmap v10574: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:59:42.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:59:41 smithi167 ceph-mon[37140]: pgmap v10574: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:59:44.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:59:43 smithi138 ceph-mon[36954]: pgmap v10575: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:59:44.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:59:43 smithi100 ceph-mon[32599]: pgmap v10575: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:59:44.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:59:43 smithi167 ceph-mon[37140]: pgmap v10575: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:59:44.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:59:44.744+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:59:44.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:59:44.744+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:59:44.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:59:44.744+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:59:44.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:59:44.744+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:59:44.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:59:44.744+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T18:59:46.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:59:45 smithi138 ceph-mon[36954]: pgmap v10576: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:59:46.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:59:45 smithi100 ceph-mon[32599]: pgmap v10576: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:59:46.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:59:45 smithi167 ceph-mon[37140]: pgmap v10576: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:59:48.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:59:47 smithi138 ceph-mon[36954]: pgmap v10577: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:59:48.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:59:47 smithi100 ceph-mon[32599]: pgmap v10577: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:59:48.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:59:47 smithi167 ceph-mon[37140]: pgmap v10577: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:59:50.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:59:49 smithi138 ceph-mon[36954]: pgmap v10578: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:59:50.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:59:49 smithi100 ceph-mon[32599]: pgmap v10578: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:59:50.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:59:49 smithi167 ceph-mon[37140]: pgmap v10578: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T18:59:52.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:59:51 smithi138 ceph-mon[36954]: pgmap v10579: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:59:52.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:59:51 smithi100 ceph-mon[32599]: pgmap v10579: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:59:52.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:59:51 smithi167 ceph-mon[37140]: pgmap v10579: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:59:54.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:59:53 smithi138 ceph-mon[36954]: pgmap v10580: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:59:54.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:59:53 smithi100 ceph-mon[32599]: pgmap v10580: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:59:54.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:59:53 smithi167 ceph-mon[37140]: pgmap v10580: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T18:59:56.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:59:55 smithi138 ceph-mon[36954]: pgmap v10581: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:59:56.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:59:55 smithi100 ceph-mon[32599]: pgmap v10581: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:59:56.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:59:55 smithi167 ceph-mon[37140]: pgmap v10581: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T18:59:58.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:59:57 smithi138 ceph-mon[36954]: pgmap v10582: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:59:58.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:59:57 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:59:58.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:59:57 smithi100 ceph-mon[32599]: pgmap v10582: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:59:58.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:59:57 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:59:58.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:59:57 smithi167 ceph-mon[37140]: pgmap v10582: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T18:59:58.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:59:57 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T18:59:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:59:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:59:59.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:59:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:59:59.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 18:59:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:59:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:59:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:59:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:59:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:59:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 18:59:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:59:59.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:59:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T18:59:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:59:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T18:59:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 18:59:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T18:59:59.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:59:59.761+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T18:59:59.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:59:59.762+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T18:59:59.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:59:59.762+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T18:59:59.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:59:59.762+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T18:59:59.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T18:59:59.762+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:00:00.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:00 smithi138 ceph-mon[36954]: pgmap v10583: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:00:00.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:00 smithi138 ceph-mon[36954]: overall HEALTH_OK 2024-07-26T19:00:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:00 smithi100 ceph-mon[32599]: pgmap v10583: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:00:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:00 smithi100 ceph-mon[32599]: overall HEALTH_OK 2024-07-26T19:00:00.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:00 smithi167 ceph-mon[37140]: pgmap v10583: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:00:00.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:00 smithi167 ceph-mon[37140]: overall HEALTH_OK 2024-07-26T19:00:02.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:02 smithi138 ceph-mon[36954]: pgmap v10584: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:00:02.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:02 smithi100 ceph-mon[32599]: pgmap v10584: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:00:02.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:02 smithi167 ceph-mon[37140]: pgmap v10584: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:00:04.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:04 smithi138 ceph-mon[36954]: pgmap v10585: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:00:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:04 smithi100 ceph-mon[32599]: pgmap v10585: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:00:04.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:04 smithi167 ceph-mon[37140]: pgmap v10585: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:00:05.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:00:05.675+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c006660 con 0x7fe5a81087a0 2024-07-26T19:00:05.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:00:05.675+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T19:00:05.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:00:05.675+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 146 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T19:00:05.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:00:05.675+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 146 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c006660 con 0x7fe5a81087a0 2024-07-26T19:00:06.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:06 smithi138 ceph-mon[36954]: pgmap v10586: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:00:06.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:06 smithi100 ceph-mon[32599]: pgmap v10586: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:00:06.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:06 smithi167 ceph-mon[37140]: pgmap v10586: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:00:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:08 smithi138 ceph-mon[36954]: pgmap v10587: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T19:00:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:08 smithi100 ceph-mon[32599]: pgmap v10587: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T19:00:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:08 smithi167 ceph-mon[37140]: pgmap v10587: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T19:00:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:09 smithi138 ceph-mon[36954]: osdmap e708: 12 total, 12 up, 12 in 2024-07-26T19:00:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:09 smithi100 ceph-mon[32599]: osdmap e708: 12 total, 12 up, 12 in 2024-07-26T19:00:09.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:09 smithi167 ceph-mon[37140]: osdmap e708: 12 total, 12 up, 12 in 2024-07-26T19:00:10.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:10 smithi138 ceph-mon[36954]: pgmap v10589: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:00:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:10 smithi100 ceph-mon[32599]: pgmap v10589: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:00:10.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:10 smithi167 ceph-mon[37140]: pgmap v10589: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:00:12.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:12 smithi138 ceph-mon[36954]: pgmap v10590: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:00:12.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:12 smithi100 ceph-mon[32599]: pgmap v10590: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:00:12.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:12 smithi167 ceph-mon[37140]: pgmap v10590: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:00:14.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:14 smithi138 ceph-mon[36954]: pgmap v10591: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:00:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:14 smithi100 ceph-mon[32599]: pgmap v10591: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:00:14.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:14 smithi167 ceph-mon[37140]: pgmap v10591: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:00:14.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:00:14.770+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:00:14.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:00:14.770+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:00:14.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:00:14.770+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:00:14.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:00:14.770+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:00:14.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:00:14.770+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:00:16.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:16 smithi138 ceph-mon[36954]: pgmap v10592: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T19:00:16.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:16 smithi100 ceph-mon[32599]: pgmap v10592: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T19:00:16.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:16 smithi167 ceph-mon[37140]: pgmap v10592: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T19:00:18.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:18 smithi138 ceph-mon[36954]: pgmap v10593: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T19:00:18.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:18 smithi100 ceph-mon[32599]: pgmap v10593: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T19:00:18.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:18 smithi167 ceph-mon[37140]: pgmap v10593: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T19:00:20.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:19 smithi138 ceph-mon[36954]: pgmap v10594: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 190 B/s wr, 7 op/s 2024-07-26T19:00:20.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:19 smithi138 ceph-mon[36954]: osdmap e709: 12 total, 12 up, 12 in 2024-07-26T19:00:20.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:19 smithi100 ceph-mon[32599]: pgmap v10594: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 190 B/s wr, 7 op/s 2024-07-26T19:00:20.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:19 smithi100 ceph-mon[32599]: osdmap e709: 12 total, 12 up, 12 in 2024-07-26T19:00:20.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:19 smithi167 ceph-mon[37140]: pgmap v10594: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 190 B/s wr, 7 op/s 2024-07-26T19:00:20.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:19 smithi167 ceph-mon[37140]: osdmap e709: 12 total, 12 up, 12 in 2024-07-26T19:00:22.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:21 smithi138 ceph-mon[36954]: pgmap v10596: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:00:22.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:21 smithi100 ceph-mon[32599]: pgmap v10596: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:00:22.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:21 smithi167 ceph-mon[37140]: pgmap v10596: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:00:24.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:23 smithi138 ceph-mon[36954]: pgmap v10597: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:00:24.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:23 smithi100 ceph-mon[32599]: pgmap v10597: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:00:24.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:23 smithi167 ceph-mon[37140]: pgmap v10597: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:00:26.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:25 smithi138 ceph-mon[36954]: pgmap v10598: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:00:26.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:25 smithi100 ceph-mon[32599]: pgmap v10598: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:00:26.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:25 smithi167 ceph-mon[37140]: pgmap v10598: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:00:28.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:27 smithi138 ceph-mon[36954]: pgmap v10599: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:00:28.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:27 smithi100 ceph-mon[32599]: pgmap v10599: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:00:28.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:27 smithi167 ceph-mon[37140]: pgmap v10599: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:00:29.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:00:29.786+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:00:29.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:00:29.786+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:00:29.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:00:29.786+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:00:29.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:00:29.786+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:00:29.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:00:29.786+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:00:30.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:29 smithi100 ceph-mon[32599]: pgmap v10600: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:00:30.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:29 smithi138 ceph-mon[36954]: pgmap v10600: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:00:30.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:29 smithi167 ceph-mon[37140]: pgmap v10600: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:00:32.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:31 smithi138 ceph-mon[36954]: pgmap v10601: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:00:32.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:31 smithi100 ceph-mon[32599]: pgmap v10601: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:00:32.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:31 smithi167 ceph-mon[37140]: pgmap v10601: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:00:34.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:33 smithi138 ceph-mon[36954]: pgmap v10602: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:00:34.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:33 smithi100 ceph-mon[32599]: pgmap v10602: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:00:34.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:33 smithi167 ceph-mon[37140]: pgmap v10602: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:00:36.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:35 smithi138 ceph-mon[36954]: pgmap v10603: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:00:36.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:35 smithi100 ceph-mon[32599]: pgmap v10603: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:00:36.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:35 smithi167 ceph-mon[37140]: pgmap v10603: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:00:38.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:37 smithi138 ceph-mon[36954]: pgmap v10604: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:00:38.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:37 smithi100 ceph-mon[32599]: pgmap v10604: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:00:38.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:37 smithi167 ceph-mon[37140]: pgmap v10604: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:00:40.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:39 smithi138 ceph-mon[36954]: pgmap v10605: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:00:40.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:39 smithi100 ceph-mon[32599]: pgmap v10605: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:00:40.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:39 smithi167 ceph-mon[37140]: pgmap v10605: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:00:42.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:41 smithi138 ceph-mon[36954]: pgmap v10606: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:00:42.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:41 smithi100 ceph-mon[32599]: pgmap v10606: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:00:42.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:41 smithi167 ceph-mon[37140]: pgmap v10606: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:00:44.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:43 smithi138 ceph-mon[36954]: pgmap v10607: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T19:00:44.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:43 smithi100 ceph-mon[32599]: pgmap v10607: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T19:00:44.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:43 smithi167 ceph-mon[37140]: pgmap v10607: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T19:00:44.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:00:44.802+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:00:44.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:00:44.803+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:00:44.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:00:44.803+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:00:44.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:00:44.803+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:00:44.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:00:44.803+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:00:46.192 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:45 smithi138 ceph-mon[36954]: pgmap v10608: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T19:00:46.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:45 smithi100 ceph-mon[32599]: pgmap v10608: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T19:00:46.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:45 smithi167 ceph-mon[37140]: pgmap v10608: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T19:00:48.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:47 smithi138 ceph-mon[36954]: pgmap v10609: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T19:00:48.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:47 smithi100 ceph-mon[32599]: pgmap v10609: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T19:00:48.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:47 smithi167 ceph-mon[37140]: pgmap v10609: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T19:00:50.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:49 smithi138 ceph-mon[36954]: pgmap v10610: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T19:00:50.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:49 smithi100 ceph-mon[32599]: pgmap v10610: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T19:00:50.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:49 smithi167 ceph-mon[37140]: pgmap v10610: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T19:00:52.183 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:51 smithi138 ceph-mon[36954]: pgmap v10611: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T19:00:52.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:51 smithi100 ceph-mon[32599]: pgmap v10611: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T19:00:52.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:51 smithi167 ceph-mon[37140]: pgmap v10611: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T19:00:54.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:53 smithi138 ceph-mon[36954]: pgmap v10612: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T19:00:54.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:53 smithi100 ceph-mon[32599]: pgmap v10612: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T19:00:54.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:53 smithi167 ceph-mon[37140]: pgmap v10612: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T19:00:56.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:55 smithi138 ceph-mon[36954]: pgmap v10613: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:00:56.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:55 smithi100 ceph-mon[32599]: pgmap v10613: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:00:56.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:55 smithi167 ceph-mon[37140]: pgmap v10613: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:00:58.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:57 smithi100 ceph-mon[32599]: pgmap v10614: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:00:58.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:57 smithi167 ceph-mon[37140]: pgmap v10614: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:00:58.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:57 smithi138 ceph-mon[36954]: pgmap v10614: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:00:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:00:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:00:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:00:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:00:59.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:00:59.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:00:59.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:00:59.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:00:59.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:00:59.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:00:59.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:00:59.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:00:59.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:00:59.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:00:59.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:00:59.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:00:59.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:00:59.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:00:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:00:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:00:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:00:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:00:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:00:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:00:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:00:59.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:00:59.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:00:59.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:00:59.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:00:59.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:00:59.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:00:59.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:00:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:00:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:00:59.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:00:59.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:00:59.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:00:59.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:00:59.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:00:59.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:00:59.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:00:59.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:00:59.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:00:59.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:00:59.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:00:59.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:00:59.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:00:59.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:00:59.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:00:59.819+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:00:59.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:00:59.819+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:00:59.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:00:59.819+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:00:59.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:00:59.819+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:00:59.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:00:59.819+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:01:00.145 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:59 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T19:01:00.146 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:59 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T19:01:00.146 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:59 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T19:01:00.146 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:59 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T19:01:00.146 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:59 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T19:01:00.146 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:59 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T19:01:00.146 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:00:59 smithi100 ceph-mon[32599]: pgmap v10615: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:01:00.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:59 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T19:01:00.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:59 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T19:01:00.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:59 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T19:01:00.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:59 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T19:01:00.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:59 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T19:01:00.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:59 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T19:01:00.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:00:59 smithi167 ceph-mon[37140]: pgmap v10615: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:01:00.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:59 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T19:01:00.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:59 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T19:01:00.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:59 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T19:01:00.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:59 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T19:01:00.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:59 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T19:01:00.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:59 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T19:01:00.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:00:59 smithi138 ceph-mon[36954]: pgmap v10615: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:01:02.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:01:01 smithi167 ceph-mon[37140]: pgmap v10616: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:01:02.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:01:01 smithi100 ceph-mon[32599]: pgmap v10616: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:01:02.252 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:01:01 smithi138 ceph-mon[36954]: pgmap v10616: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:01:04.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:01:03 smithi100 ceph-mon[32599]: pgmap v10617: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:01:04.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:01:03 smithi167 ceph-mon[37140]: pgmap v10617: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:01:04.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:01:03 smithi138 ceph-mon[36954]: pgmap v10617: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:01:06.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:01:05 smithi100 ceph-mon[32599]: pgmap v10618: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:01:06.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:01:05 smithi167 ceph-mon[37140]: pgmap v10618: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:01:06.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:01:05 smithi138 ceph-mon[36954]: pgmap v10618: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:01:08.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:01:07 smithi100 ceph-mon[32599]: pgmap v10619: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T19:01:08.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:01:07 smithi167 ceph-mon[37140]: pgmap v10619: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T19:01:08.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:01:07 smithi138 ceph-mon[36954]: pgmap v10619: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T19:01:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:01:08 smithi100 ceph-mon[32599]: osdmap e710: 12 total, 12 up, 12 in 2024-07-26T19:01:09.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:01:08 smithi167 ceph-mon[37140]: osdmap e710: 12 total, 12 up, 12 in 2024-07-26T19:01:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:01:08 smithi138 ceph-mon[36954]: osdmap e710: 12 total, 12 up, 12 in 2024-07-26T19:01:10.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:01:09 smithi100 ceph-mon[32599]: pgmap v10621: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:01:10.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:01:09 smithi167 ceph-mon[37140]: pgmap v10621: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:01:10.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:01:09 smithi138 ceph-mon[36954]: pgmap v10621: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:01:12.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:01:11 smithi100 ceph-mon[32599]: pgmap v10622: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:01:12.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:01:11 smithi167 ceph-mon[37140]: pgmap v10622: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:01:12.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:01:11 smithi138 ceph-mon[36954]: pgmap v10622: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:01:14.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:01:13 smithi100 ceph-mon[32599]: pgmap v10623: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:01:14.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:01:13 smithi167 ceph-mon[37140]: pgmap v10623: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:01:14.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:01:13 smithi138 ceph-mon[36954]: pgmap v10623: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:01:14.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:01:14.834+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:01:14.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:01:14.835+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:01:14.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:01:14.835+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:01:14.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:01:14.835+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:01:14.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:01:14.835+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:01:16.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:01:15 smithi100 ceph-mon[32599]: pgmap v10624: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T19:01:16.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:01:15 smithi167 ceph-mon[37140]: pgmap v10624: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T19:01:16.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:01:15 smithi138 ceph-mon[36954]: pgmap v10624: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T19:01:18.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:01:17 smithi100 ceph-mon[32599]: pgmap v10625: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T19:01:18.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:01:17 smithi167 ceph-mon[37140]: pgmap v10625: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T19:01:18.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:01:17 smithi138 ceph-mon[36954]: pgmap v10625: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T19:01:20.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:01:19 smithi138 ceph-mon[36954]: pgmap v10626: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 187 B/s wr, 7 op/s 2024-07-26T19:01:20.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:01:19 smithi138 ceph-mon[36954]: osdmap e711: 12 total, 12 up, 12 in 2024-07-26T19:01:20.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:01:19 smithi100 ceph-mon[32599]: pgmap v10626: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 187 B/s wr, 7 op/s 2024-07-26T19:01:20.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:01:19 smithi100 ceph-mon[32599]: osdmap e711: 12 total, 12 up, 12 in 2024-07-26T19:01:20.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:01:19 smithi167 ceph-mon[37140]: pgmap v10626: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 187 B/s wr, 7 op/s 2024-07-26T19:01:20.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:01:19 smithi167 ceph-mon[37140]: osdmap e711: 12 total, 12 up, 12 in 2024-07-26T19:01:22.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:01:21 smithi138 ceph-mon[36954]: pgmap v10628: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:01:22.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:01:21 smithi100 ceph-mon[32599]: pgmap v10628: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:01:22.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:01:21 smithi167 ceph-mon[37140]: pgmap v10628: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:01:24.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:01:23 smithi138 ceph-mon[36954]: pgmap v10629: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:01:24.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:01:23 smithi100 ceph-mon[32599]: pgmap v10629: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:01:24.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:01:23 smithi167 ceph-mon[37140]: pgmap v10629: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:01:26.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:01:25 smithi138 ceph-mon[36954]: pgmap v10630: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:01:26.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:01:25 smithi100 ceph-mon[32599]: pgmap v10630: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:01:26.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:01:25 smithi167 ceph-mon[37140]: pgmap v10630: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:01:28.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:01:27 smithi138 ceph-mon[36954]: pgmap v10631: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:01:28.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:01:27 smithi100 ceph-mon[32599]: pgmap v10631: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:01:28.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:01:27 smithi167 ceph-mon[37140]: pgmap v10631: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:01:29.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:01:29.851+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:01:29.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:01:29.851+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:01:29.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:01:29.851+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:01:29.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:01:29.851+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:01:29.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:01:29.851+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:01:30.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:01:29 smithi138 ceph-mon[36954]: pgmap v10632: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:01:30.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:01:29 smithi100 ceph-mon[32599]: pgmap v10632: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:01:30.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:01:29 smithi167 ceph-mon[37140]: pgmap v10632: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:01:32.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:01:31 smithi138 ceph-mon[36954]: pgmap v10633: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:01:32.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:01:31 smithi100 ceph-mon[32599]: pgmap v10633: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:01:32.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:01:31 smithi167 ceph-mon[37140]: pgmap v10633: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:01:34.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:01:33 smithi138 ceph-mon[36954]: pgmap v10634: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:01:34.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:01:33 smithi100 ceph-mon[32599]: pgmap v10634: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:01:34.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:01:33 smithi167 ceph-mon[37140]: pgmap v10634: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:01:35.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:01:35.687+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T19:01:35.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:01:35.687+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c006660 con 0x7fe5a81087a0 2024-07-26T19:01:35.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:01:35.687+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 147 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T19:01:35.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:01:35.688+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 147 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe57c006660 con 0x7fe5a81087a0 2024-07-26T19:01:35.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:01:35 smithi100 ceph-mon[32599]: pgmap v10635: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:01:36.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:01:35 smithi138 ceph-mon[36954]: pgmap v10635: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:01:36.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:01:35 smithi167 ceph-mon[37140]: pgmap v10635: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:01:38.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:01:37 smithi138 ceph-mon[36954]: pgmap v10636: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:01:38.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:01:37 smithi100 ceph-mon[32599]: pgmap v10636: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:01:38.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:01:37 smithi167 ceph-mon[37140]: pgmap v10636: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:01:40.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:01:39 smithi138 ceph-mon[36954]: pgmap v10637: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:01:40.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:01:39 smithi100 ceph-mon[32599]: pgmap v10637: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:01:40.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:01:39 smithi167 ceph-mon[37140]: pgmap v10637: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:01:42.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:01:41 smithi138 ceph-mon[36954]: pgmap v10638: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:01:42.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:01:41 smithi100 ceph-mon[32599]: pgmap v10638: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:01:42.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:01:41 smithi167 ceph-mon[37140]: pgmap v10638: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:01:44.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:01:43 smithi100 ceph-mon[32599]: pgmap v10639: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:01:44.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:01:43 smithi167 ceph-mon[37140]: pgmap v10639: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:01:44.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:01:43 smithi138 ceph-mon[36954]: pgmap v10639: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:01:44.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:01:44.859+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:01:44.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:01:44.859+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:01:44.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:01:44.859+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:01:44.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:01:44.859+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:01:44.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:01:44.859+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:01:46.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:01:45 smithi100 ceph-mon[32599]: pgmap v10640: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:01:46.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:01:45 smithi167 ceph-mon[37140]: pgmap v10640: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:01:46.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:01:45 smithi138 ceph-mon[36954]: pgmap v10640: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:01:48.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:01:47 smithi100 ceph-mon[32599]: pgmap v10641: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:01:48.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:01:47 smithi167 ceph-mon[37140]: pgmap v10641: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:01:48.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:01:47 smithi138 ceph-mon[36954]: pgmap v10641: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:01:50.242 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:01:49 smithi138 ceph-mon[36954]: pgmap v10642: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:01:50.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:01:49 smithi100 ceph-mon[32599]: pgmap v10642: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:01:50.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:01:49 smithi167 ceph-mon[37140]: pgmap v10642: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:01:52.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:01:51 smithi100 ceph-mon[32599]: pgmap v10643: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:01:52.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:01:51 smithi167 ceph-mon[37140]: pgmap v10643: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:01:52.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:01:51 smithi138 ceph-mon[36954]: pgmap v10643: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:01:54.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:01:53 smithi100 ceph-mon[32599]: pgmap v10644: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:01:54.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:01:53 smithi167 ceph-mon[37140]: pgmap v10644: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:01:54.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:01:53 smithi138 ceph-mon[36954]: pgmap v10644: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:01:56.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:01:55 smithi100 ceph-mon[32599]: pgmap v10645: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:01:56.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:01:55 smithi167 ceph-mon[37140]: pgmap v10645: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:01:56.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:01:55 smithi138 ceph-mon[36954]: pgmap v10645: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:01:58.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:01:57 smithi100 ceph-mon[32599]: pgmap v10646: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:01:58.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:01:57 smithi167 ceph-mon[37140]: pgmap v10646: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:01:58.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:01:57 smithi138 ceph-mon[36954]: pgmap v10646: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:01:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:01:58 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:01:59.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:01:58 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:01:59.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:01:58 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:01:59.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:01:59.875+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:01:59.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:01:59.875+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:01:59.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:01:59.875+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:01:59.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:01:59.875+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:01:59.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:01:59.875+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:02:00.145 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:01:59 smithi100 ceph-mon[32599]: pgmap v10647: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:02:00.145 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:01:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:02:00.145 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:01:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:02:00.146 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:01:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:02:00.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:01:59 smithi167 ceph-mon[37140]: pgmap v10647: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:02:00.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:01:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:02:00.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:01:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:02:00.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:01:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:02:00.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:01:59 smithi138 ceph-mon[36954]: pgmap v10647: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:02:00.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:01:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:02:00.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:01:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:02:00.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:01:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:02:02.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:02:01 smithi100 ceph-mon[32599]: pgmap v10648: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:02:02.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:02:01 smithi167 ceph-mon[37140]: pgmap v10648: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:02:02.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:02:01 smithi138 ceph-mon[36954]: pgmap v10648: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:02:04.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:02:03 smithi100 ceph-mon[32599]: pgmap v10649: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:02:04.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:02:03 smithi167 ceph-mon[37140]: pgmap v10649: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:02:04.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:02:03 smithi138 ceph-mon[36954]: pgmap v10649: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:02:06.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:02:05 smithi100 ceph-mon[32599]: pgmap v10650: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:02:06.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:02:05 smithi167 ceph-mon[37140]: pgmap v10650: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:02:06.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:02:05 smithi138 ceph-mon[36954]: pgmap v10650: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:02:08.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:02:07 smithi100 ceph-mon[32599]: pgmap v10651: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T19:02:08.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:02:07 smithi167 ceph-mon[37140]: pgmap v10651: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T19:02:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:02:07 smithi138 ceph-mon[36954]: pgmap v10651: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T19:02:09.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:02:08 smithi100 ceph-mon[32599]: osdmap e712: 12 total, 12 up, 12 in 2024-07-26T19:02:09.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:02:08 smithi167 ceph-mon[37140]: osdmap e712: 12 total, 12 up, 12 in 2024-07-26T19:02:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:02:08 smithi138 ceph-mon[36954]: osdmap e712: 12 total, 12 up, 12 in 2024-07-26T19:02:10.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:02:09 smithi167 ceph-mon[37140]: pgmap v10653: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:02:10.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:02:09 smithi138 ceph-mon[36954]: pgmap v10653: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:02:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:02:09 smithi100 ceph-mon[32599]: pgmap v10653: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:02:12.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:02:11 smithi167 ceph-mon[37140]: pgmap v10654: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:02:12.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:02:11 smithi138 ceph-mon[36954]: pgmap v10654: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:02:12.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:02:11 smithi100 ceph-mon[32599]: pgmap v10654: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:02:14.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:02:14 smithi167 ceph-mon[37140]: pgmap v10655: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:02:14.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:02:14 smithi138 ceph-mon[36954]: pgmap v10655: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:02:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:02:14 smithi100 ceph-mon[32599]: pgmap v10655: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:02:14.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:02:14.891+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:02:14.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:02:14.891+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:02:14.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:02:14.891+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:02:14.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:02:14.891+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:02:14.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:02:14.891+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:02:16.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:02:16 smithi167 ceph-mon[37140]: pgmap v10656: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T19:02:16.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:02:16 smithi138 ceph-mon[36954]: pgmap v10656: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T19:02:16.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:02:16 smithi100 ceph-mon[32599]: pgmap v10656: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T19:02:18.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:02:18 smithi167 ceph-mon[37140]: pgmap v10657: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T19:02:18.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:02:18 smithi138 ceph-mon[36954]: pgmap v10657: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T19:02:18.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:02:18 smithi100 ceph-mon[32599]: pgmap v10657: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T19:02:20.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:02:19 smithi138 ceph-mon[36954]: pgmap v10658: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 188 B/s wr, 7 op/s 2024-07-26T19:02:20.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:02:19 smithi138 ceph-mon[36954]: osdmap e713: 12 total, 12 up, 12 in 2024-07-26T19:02:20.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:02:19 smithi100 ceph-mon[32599]: pgmap v10658: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 188 B/s wr, 7 op/s 2024-07-26T19:02:20.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:02:19 smithi100 ceph-mon[32599]: osdmap e713: 12 total, 12 up, 12 in 2024-07-26T19:02:20.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:02:19 smithi167 ceph-mon[37140]: pgmap v10658: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 188 B/s wr, 7 op/s 2024-07-26T19:02:20.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:02:19 smithi167 ceph-mon[37140]: osdmap e713: 12 total, 12 up, 12 in 2024-07-26T19:02:22.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:02:21 smithi138 ceph-mon[36954]: pgmap v10660: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:02:22.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:02:21 smithi100 ceph-mon[32599]: pgmap v10660: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:02:22.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:02:21 smithi167 ceph-mon[37140]: pgmap v10660: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:02:24.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:02:23 smithi138 ceph-mon[36954]: pgmap v10661: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:02:24.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:02:23 smithi100 ceph-mon[32599]: pgmap v10661: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:02:24.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:02:23 smithi167 ceph-mon[37140]: pgmap v10661: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:02:26.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:02:25 smithi138 ceph-mon[36954]: pgmap v10662: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:02:26.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:02:25 smithi100 ceph-mon[32599]: pgmap v10662: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:02:26.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:02:25 smithi167 ceph-mon[37140]: pgmap v10662: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:02:28.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:02:27 smithi138 ceph-mon[36954]: pgmap v10663: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:02:28.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:02:27 smithi100 ceph-mon[32599]: pgmap v10663: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:02:28.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:02:27 smithi167 ceph-mon[37140]: pgmap v10663: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:02:29.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:02:29.904+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:02:29.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:02:29.904+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:02:29.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:02:29.904+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:02:29.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:02:29.904+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:02:29.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:02:29.904+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:02:30.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:02:29 smithi138 ceph-mon[36954]: pgmap v10664: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:02:30.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:02:29 smithi100 ceph-mon[32599]: pgmap v10664: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:02:30.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:02:29 smithi167 ceph-mon[37140]: pgmap v10664: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:02:32.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:02:31 smithi138 ceph-mon[36954]: pgmap v10665: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:02:32.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:02:31 smithi100 ceph-mon[32599]: pgmap v10665: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:02:32.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:02:31 smithi167 ceph-mon[37140]: pgmap v10665: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:02:34.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:02:33 smithi100 ceph-mon[32599]: pgmap v10666: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:02:34.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:02:33 smithi167 ceph-mon[37140]: pgmap v10666: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:02:34.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:02:33 smithi138 ceph-mon[36954]: pgmap v10666: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:02:36.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:02:35 smithi100 ceph-mon[32599]: pgmap v10667: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:02:36.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:02:35 smithi167 ceph-mon[37140]: pgmap v10667: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:02:36.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:02:35 smithi138 ceph-mon[36954]: pgmap v10667: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:02:38.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:02:37 smithi100 ceph-mon[32599]: pgmap v10668: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:02:38.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:02:37 smithi167 ceph-mon[37140]: pgmap v10668: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:02:38.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:02:37 smithi138 ceph-mon[36954]: pgmap v10668: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:02:40.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:02:39 smithi100 ceph-mon[32599]: pgmap v10669: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:02:40.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:02:39 smithi167 ceph-mon[37140]: pgmap v10669: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:02:40.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:02:39 smithi138 ceph-mon[36954]: pgmap v10669: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:02:42.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:02:41 smithi100 ceph-mon[32599]: pgmap v10670: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:02:42.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:02:41 smithi167 ceph-mon[37140]: pgmap v10670: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:02:42.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:02:41 smithi138 ceph-mon[36954]: pgmap v10670: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:02:44.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:02:43 smithi100 ceph-mon[32599]: pgmap v10671: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:02:44.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:02:43 smithi167 ceph-mon[37140]: pgmap v10671: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:02:44.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:02:43 smithi138 ceph-mon[36954]: pgmap v10671: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:02:44.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:02:44.920+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:02:44.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:02:44.920+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:02:44.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:02:44.920+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:02:44.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:02:44.920+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:02:44.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:02:44.920+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:02:46.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:02:45 smithi100 ceph-mon[32599]: pgmap v10672: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:02:46.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:02:45 smithi167 ceph-mon[37140]: pgmap v10672: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:02:46.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:02:45 smithi138 ceph-mon[36954]: pgmap v10672: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:02:48.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:02:47 smithi100 ceph-mon[32599]: pgmap v10673: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:02:48.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:02:47 smithi167 ceph-mon[37140]: pgmap v10673: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:02:48.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:02:47 smithi138 ceph-mon[36954]: pgmap v10673: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:02:50.242 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:02:49 smithi138 ceph-mon[36954]: pgmap v10674: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:02:50.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:02:49 smithi100 ceph-mon[32599]: pgmap v10674: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:02:50.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:02:49 smithi167 ceph-mon[37140]: pgmap v10674: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:02:52.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:02:51 smithi100 ceph-mon[32599]: pgmap v10675: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:02:52.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:02:51 smithi167 ceph-mon[37140]: pgmap v10675: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:02:52.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:02:51 smithi138 ceph-mon[36954]: pgmap v10675: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:02:54.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:02:53 smithi100 ceph-mon[32599]: pgmap v10676: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:02:54.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:02:53 smithi167 ceph-mon[37140]: pgmap v10676: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:02:54.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:02:53 smithi138 ceph-mon[36954]: pgmap v10676: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:02:56.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:02:55 smithi100 ceph-mon[32599]: pgmap v10677: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:02:56.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:02:55 smithi167 ceph-mon[37140]: pgmap v10677: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:02:56.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:02:55 smithi138 ceph-mon[36954]: pgmap v10677: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:02:58.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:02:57 smithi100 ceph-mon[32599]: pgmap v10678: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:02:58.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:02:57 smithi167 ceph-mon[37140]: pgmap v10678: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:02:58.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:02:57 smithi138 ceph-mon[36954]: pgmap v10678: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:02:59.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:02:59.935+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:02:59.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:02:59.935+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:02:59.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:02:59.935+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:02:59.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:02:59.935+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:02:59.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:02:59.935+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:03:00.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:02:59 smithi100 ceph-mon[32599]: pgmap v10679: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:03:00.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:02:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:03:00.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:02:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:03:00.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:02:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:03:00.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:02:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:03:00.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:02:59 smithi167 ceph-mon[37140]: pgmap v10679: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:03:00.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:02:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:03:00.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:02:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:03:00.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:02:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:03:00.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:02:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:03:00.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:02:59 smithi138 ceph-mon[36954]: pgmap v10679: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:03:00.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:02:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:03:00.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:02:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:03:00.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:02:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:03:00.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:02:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:03:02.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:03:01 smithi100 ceph-mon[32599]: pgmap v10680: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:03:02.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:03:01 smithi167 ceph-mon[37140]: pgmap v10680: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:03:02.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:03:01 smithi138 ceph-mon[36954]: pgmap v10680: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:03:03.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:03:03.049+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 msgr2=0x7fe5940b5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-07-26T19:03:03.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:03:03.049+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 msgr2=0x7fe5940b5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-07-26T19:03:03.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:03:03.049+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 0x7fe5940b5b30 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fe590005d00 tx=0x7fe59003b040 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:03:03.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:03:03.049+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 0x7fe5940b5b30 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fe590005d00 tx=0x7fe59003b040 comp rx=0 tx=0).stop 2024-07-26T19:03:03.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:03:03.049+0000 7fe59ffff640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 msgr2=0x7fe5940b5b30 unknown :-1 s=STATE_CLOSED l=1).tick see no progress in more than 10000000 us during connecting to v2:172.21.15.100:6800/3937009534, fault. 2024-07-26T19:03:03.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:03:03.049+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 msgr2=0x7fe5940b5b30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-07-26T19:03:03.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:03:03.049+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940b5720 0x7fe5940b5b30 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-26T19:03:03.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:03:03.049+0000 7fe59f7fe640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-26T19:03:03.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:03:03.049+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:03:03.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:03:03.050+0000 7fe59ffff640 1 --2- 172.21.15.100:0/2403309286 >> [v2:172.21.15.100:6800/3937009534,v1:172.21.15.100:6801/3937009534] conn(0x7fe5940be060 0x7fe5940c0480 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fe59000be60 tx=0x7fe590074040 comp rx=0 tx=0).ready entity=mgr.14214 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-26T19:03:04.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:03:03 smithi167 ceph-mon[37140]: pgmap v10681: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:03:04.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:03:03 smithi138 ceph-mon[36954]: pgmap v10681: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:03:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:03:03 smithi100 ceph-mon[32599]: pgmap v10681: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:03:05.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:03:05.700+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T19:03:05.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:03:05.700+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c006660 con 0x7fe5a81087a0 2024-07-26T19:03:05.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:03:05.700+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 148 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c006660 con 0x7fe5a81087a0 2024-07-26T19:03:05.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:03:05.700+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 148 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T19:03:06.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:03:06 smithi167 ceph-mon[37140]: pgmap v10682: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:03:06.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:03:06 smithi138 ceph-mon[36954]: pgmap v10682: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:03:06.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:03:05 smithi100 ceph-mon[32599]: pgmap v10682: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:03:08.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:03:08 smithi167 ceph-mon[37140]: pgmap v10683: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T19:03:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:03:08 smithi138 ceph-mon[36954]: pgmap v10683: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T19:03:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:03:08 smithi100 ceph-mon[32599]: pgmap v10683: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T19:03:09.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:03:09 smithi167 ceph-mon[37140]: osdmap e714: 12 total, 12 up, 12 in 2024-07-26T19:03:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:03:09 smithi138 ceph-mon[36954]: osdmap e714: 12 total, 12 up, 12 in 2024-07-26T19:03:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:03:09 smithi100 ceph-mon[32599]: osdmap e714: 12 total, 12 up, 12 in 2024-07-26T19:03:10.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:03:10 smithi167 ceph-mon[37140]: pgmap v10685: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:03:10.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:03:10 smithi138 ceph-mon[36954]: pgmap v10685: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:03:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:03:10 smithi100 ceph-mon[32599]: pgmap v10685: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:03:12.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:03:12 smithi138 ceph-mon[36954]: pgmap v10686: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:03:12.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:03:12 smithi100 ceph-mon[32599]: pgmap v10686: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:03:12.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:03:12 smithi167 ceph-mon[37140]: pgmap v10686: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:03:14.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:03:14 smithi167 ceph-mon[37140]: pgmap v10687: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:03:14.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:03:14 smithi138 ceph-mon[36954]: pgmap v10687: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:03:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:03:14 smithi100 ceph-mon[32599]: pgmap v10687: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:03:14.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:03:14.943+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:03:14.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:03:14.944+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:03:14.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:03:14.944+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:03:14.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:03:14.944+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:03:14.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:03:14.944+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:03:16.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:03:16 smithi138 ceph-mon[36954]: pgmap v10688: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T19:03:16.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:03:16 smithi100 ceph-mon[32599]: pgmap v10688: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T19:03:16.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:03:16 smithi167 ceph-mon[37140]: pgmap v10688: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T19:03:18.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:03:18 smithi138 ceph-mon[36954]: pgmap v10689: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T19:03:18.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:03:18 smithi100 ceph-mon[32599]: pgmap v10689: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T19:03:18.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:03:18 smithi167 ceph-mon[37140]: pgmap v10689: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T19:03:20.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:03:19 smithi138 ceph-mon[36954]: pgmap v10690: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 187 B/s wr, 7 op/s 2024-07-26T19:03:20.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:03:19 smithi138 ceph-mon[36954]: osdmap e715: 12 total, 12 up, 12 in 2024-07-26T19:03:20.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:03:19 smithi100 ceph-mon[32599]: pgmap v10690: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 187 B/s wr, 7 op/s 2024-07-26T19:03:20.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:03:19 smithi100 ceph-mon[32599]: osdmap e715: 12 total, 12 up, 12 in 2024-07-26T19:03:20.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:03:19 smithi167 ceph-mon[37140]: pgmap v10690: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 187 B/s wr, 7 op/s 2024-07-26T19:03:20.281 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:03:19 smithi167 ceph-mon[37140]: osdmap e715: 12 total, 12 up, 12 in 2024-07-26T19:03:22.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:03:21 smithi138 ceph-mon[36954]: pgmap v10692: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:03:22.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:03:21 smithi100 ceph-mon[32599]: pgmap v10692: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:03:22.280 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:03:21 smithi167 ceph-mon[37140]: pgmap v10692: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:03:24.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:03:23 smithi138 ceph-mon[36954]: pgmap v10693: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:03:24.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:03:23 smithi100 ceph-mon[32599]: pgmap v10693: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:03:24.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:03:23 smithi167 ceph-mon[37140]: pgmap v10693: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:03:26.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:03:25 smithi100 ceph-mon[32599]: pgmap v10694: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:03:26.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:03:25 smithi167 ceph-mon[37140]: pgmap v10694: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:03:26.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:03:25 smithi138 ceph-mon[36954]: pgmap v10694: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:03:28.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:03:27 smithi100 ceph-mon[32599]: pgmap v10695: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:03:28.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:03:27 smithi167 ceph-mon[37140]: pgmap v10695: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:03:28.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:03:27 smithi138 ceph-mon[36954]: pgmap v10695: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:03:29.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:03:29.959+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:03:29.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:03:29.960+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:03:29.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:03:29.960+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:03:29.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:03:29.960+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:03:29.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:03:29.960+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:03:30.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:03:29 smithi100 ceph-mon[32599]: pgmap v10696: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:03:30.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:03:29 smithi167 ceph-mon[37140]: pgmap v10696: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:03:30.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:03:29 smithi138 ceph-mon[36954]: pgmap v10696: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:03:32.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:03:31 smithi100 ceph-mon[32599]: pgmap v10697: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:03:32.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:03:31 smithi167 ceph-mon[37140]: pgmap v10697: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:03:32.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:03:31 smithi138 ceph-mon[36954]: pgmap v10697: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:03:34.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:03:33 smithi100 ceph-mon[32599]: pgmap v10698: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:03:34.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:03:33 smithi167 ceph-mon[37140]: pgmap v10698: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:03:34.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:03:33 smithi138 ceph-mon[36954]: pgmap v10698: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:03:36.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:03:35 smithi100 ceph-mon[32599]: pgmap v10699: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:03:36.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:03:35 smithi167 ceph-mon[37140]: pgmap v10699: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:03:36.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:03:35 smithi138 ceph-mon[36954]: pgmap v10699: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:03:38.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:03:37 smithi100 ceph-mon[32599]: pgmap v10700: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:03:38.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:03:37 smithi167 ceph-mon[37140]: pgmap v10700: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:03:38.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:03:37 smithi138 ceph-mon[36954]: pgmap v10700: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:03:40.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:03:39 smithi100 ceph-mon[32599]: pgmap v10701: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:03:40.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:03:39 smithi167 ceph-mon[37140]: pgmap v10701: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:03:40.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:03:39 smithi138 ceph-mon[36954]: pgmap v10701: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:03:42.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:03:41 smithi100 ceph-mon[32599]: pgmap v10702: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:03:42.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:03:41 smithi167 ceph-mon[37140]: pgmap v10702: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:03:42.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:03:41 smithi138 ceph-mon[36954]: pgmap v10702: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:03:44.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:03:43 smithi100 ceph-mon[32599]: pgmap v10703: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:03:44.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:03:43 smithi167 ceph-mon[37140]: pgmap v10703: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:03:44.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:03:43 smithi138 ceph-mon[36954]: pgmap v10703: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:03:44.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:03:44.971+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:03:44.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:03:44.971+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:03:44.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:03:44.971+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:03:44.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:03:44.971+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:03:44.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:03:44.971+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:03:46.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:03:45 smithi100 ceph-mon[32599]: pgmap v10704: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:03:46.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:03:45 smithi167 ceph-mon[37140]: pgmap v10704: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:03:46.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:03:45 smithi138 ceph-mon[36954]: pgmap v10704: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:03:48.246 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:03:47 smithi100 ceph-mon[32599]: pgmap v10705: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:03:48.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:03:47 smithi167 ceph-mon[37140]: pgmap v10705: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:03:48.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:03:47 smithi138 ceph-mon[36954]: pgmap v10705: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:03:50.242 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:03:49 smithi138 ceph-mon[36954]: pgmap v10706: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:03:50.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:03:49 smithi100 ceph-mon[32599]: pgmap v10706: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:03:50.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:03:49 smithi167 ceph-mon[37140]: pgmap v10706: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:03:52.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:03:51 smithi100 ceph-mon[32599]: pgmap v10707: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:03:52.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:03:51 smithi167 ceph-mon[37140]: pgmap v10707: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:03:52.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:03:51 smithi138 ceph-mon[36954]: pgmap v10707: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:03:54.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:03:53 smithi100 ceph-mon[32599]: pgmap v10708: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:03:54.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:03:53 smithi167 ceph-mon[37140]: pgmap v10708: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:03:54.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:03:53 smithi138 ceph-mon[36954]: pgmap v10708: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:03:56.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:03:55 smithi100 ceph-mon[32599]: pgmap v10709: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:03:56.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:03:55 smithi167 ceph-mon[37140]: pgmap v10709: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:03:56.443 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:03:55 smithi138 ceph-mon[36954]: pgmap v10709: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:03:58.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:03:57 smithi167 ceph-mon[37140]: pgmap v10710: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:03:58.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:03:57 smithi138 ceph-mon[36954]: pgmap v10710: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:03:58.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:03:57 smithi100 ceph-mon[32599]: pgmap v10710: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:03:59.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:03:59.985+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:03:59.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:03:59.986+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:03:59.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:03:59.986+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:03:59.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:03:59.986+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:03:59.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:03:59.986+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:04:00.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:03:59 smithi100 ceph-mon[32599]: pgmap v10711: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T19:04:00.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:03:59 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:04:00.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:03:59 smithi167 ceph-mon[37140]: pgmap v10711: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T19:04:00.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:03:59 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:04:00.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:03:59 smithi138 ceph-mon[36954]: pgmap v10711: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T19:04:00.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:03:59 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:04:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:02 smithi138 ceph-mon[36954]: pgmap v10712: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T19:04:02.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:04:02.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:04:02.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:04:02.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:04:02.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:04:02.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:04:02.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:04:02.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:04:02.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:04:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:02 smithi100 ceph-mon[32599]: pgmap v10712: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T19:04:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:04:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:04:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:04:02.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:04:02.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:04:02.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:04:02.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:04:02.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:04:02.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:04:02.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:02 smithi167 ceph-mon[37140]: pgmap v10712: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T19:04:02.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:04:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:04:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:04:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:04:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:04:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:04:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:04:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:04:02.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:04:04.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:04 smithi138 ceph-mon[36954]: pgmap v10713: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-07-26T19:04:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:04 smithi100 ceph-mon[32599]: pgmap v10713: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-07-26T19:04:04.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:04 smithi167 ceph-mon[37140]: pgmap v10713: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 7 op/s 2024-07-26T19:04:06.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:06 smithi138 ceph-mon[36954]: pgmap v10714: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:04:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:06 smithi100 ceph-mon[32599]: pgmap v10714: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:04:06.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:06 smithi167 ceph-mon[37140]: pgmap v10714: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:04:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:08 smithi138 ceph-mon[36954]: pgmap v10715: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T19:04:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:08 smithi138 ceph-mon[36954]: osdmap e716: 12 total, 12 up, 12 in 2024-07-26T19:04:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:08 smithi100 ceph-mon[32599]: pgmap v10715: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T19:04:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:08 smithi100 ceph-mon[32599]: osdmap e716: 12 total, 12 up, 12 in 2024-07-26T19:04:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:08 smithi167 ceph-mon[37140]: pgmap v10715: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T19:04:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:08 smithi167 ceph-mon[37140]: osdmap e716: 12 total, 12 up, 12 in 2024-07-26T19:04:10.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:10 smithi138 ceph-mon[36954]: pgmap v10717: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T19:04:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:10 smithi100 ceph-mon[32599]: pgmap v10717: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T19:04:10.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:10 smithi167 ceph-mon[37140]: pgmap v10717: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T19:04:12.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:12 smithi138 ceph-mon[36954]: pgmap v10718: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T19:04:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:12 smithi100 ceph-mon[32599]: pgmap v10718: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T19:04:12.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:12 smithi167 ceph-mon[37140]: pgmap v10718: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T19:04:14.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:14 smithi167 ceph-mon[37140]: pgmap v10719: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:04:14.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:14 smithi167 ceph-mon[37140]: osdmap e717: 12 total, 12 up, 12 in 2024-07-26T19:04:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:14 smithi138 ceph-mon[36954]: pgmap v10719: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:04:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:14 smithi138 ceph-mon[36954]: osdmap e717: 12 total, 12 up, 12 in 2024-07-26T19:04:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:14 smithi100 ceph-mon[32599]: pgmap v10719: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:04:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:14 smithi100 ceph-mon[32599]: osdmap e717: 12 total, 12 up, 12 in 2024-07-26T19:04:15.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:04:15.001+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:04:15.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:04:15.001+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:04:15.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:04:15.001+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:04:15.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:04:15.001+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:04:15.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:04:15.001+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:04:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:16 smithi138 ceph-mon[36954]: pgmap v10721: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T19:04:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:16 smithi100 ceph-mon[32599]: pgmap v10721: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T19:04:16.767 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:16 smithi167 ceph-mon[37140]: pgmap v10721: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T19:04:18.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:18 smithi138 ceph-mon[36954]: pgmap v10722: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 213 B/s wr, 6 op/s 2024-07-26T19:04:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:18 smithi100 ceph-mon[32599]: pgmap v10722: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 213 B/s wr, 6 op/s 2024-07-26T19:04:18.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:18 smithi167 ceph-mon[37140]: pgmap v10722: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 213 B/s wr, 6 op/s 2024-07-26T19:04:20.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:20 smithi138 ceph-mon[36954]: pgmap v10723: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:04:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:20 smithi100 ceph-mon[32599]: pgmap v10723: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:04:20.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:20 smithi167 ceph-mon[37140]: pgmap v10723: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:04:22.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:22 smithi138 ceph-mon[36954]: pgmap v10724: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:04:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:22 smithi100 ceph-mon[32599]: pgmap v10724: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:04:22.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:22 smithi167 ceph-mon[37140]: pgmap v10724: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:04:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:24 smithi138 ceph-mon[36954]: pgmap v10725: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:04:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:24 smithi100 ceph-mon[32599]: pgmap v10725: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:04:24.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:24 smithi167 ceph-mon[37140]: pgmap v10725: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:04:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:26 smithi138 ceph-mon[36954]: pgmap v10726: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:04:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:26 smithi100 ceph-mon[32599]: pgmap v10726: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:04:26.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:26 smithi167 ceph-mon[37140]: pgmap v10726: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:04:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:28 smithi138 ceph-mon[36954]: pgmap v10727: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:04:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:28 smithi100 ceph-mon[32599]: pgmap v10727: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:04:28.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:28 smithi167 ceph-mon[37140]: pgmap v10727: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:04:30.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:04:30.003+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:04:30.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:04:30.004+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:04:30.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:04:30.004+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:04:30.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:04:30.004+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:04:30.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:04:30.004+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:04:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:30 smithi138 ceph-mon[36954]: pgmap v10728: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:04:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:30 smithi100 ceph-mon[32599]: pgmap v10728: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:04:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:30 smithi167 ceph-mon[37140]: pgmap v10728: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:04:32.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:32 smithi138 ceph-mon[36954]: pgmap v10729: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:04:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:32 smithi100 ceph-mon[32599]: pgmap v10729: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:04:32.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:32 smithi167 ceph-mon[37140]: pgmap v10729: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:04:34.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:34 smithi138 ceph-mon[36954]: pgmap v10730: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:04:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:34 smithi100 ceph-mon[32599]: pgmap v10730: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:04:34.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:34 smithi167 ceph-mon[37140]: pgmap v10730: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:04:36.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:36 smithi138 ceph-mon[36954]: pgmap v10731: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:04:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:36 smithi100 ceph-mon[32599]: pgmap v10731: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:04:36.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:36 smithi167 ceph-mon[37140]: pgmap v10731: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:04:38.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:38 smithi138 ceph-mon[36954]: pgmap v10732: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:04:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:38 smithi100 ceph-mon[32599]: pgmap v10732: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:04:38.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:38 smithi167 ceph-mon[37140]: pgmap v10732: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:04:40.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:40 smithi138 ceph-mon[36954]: pgmap v10733: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:04:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:40 smithi100 ceph-mon[32599]: pgmap v10733: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:04:40.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:40 smithi167 ceph-mon[37140]: pgmap v10733: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:04:42.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:42 smithi138 ceph-mon[36954]: pgmap v10734: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:04:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:42 smithi100 ceph-mon[32599]: pgmap v10734: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:04:42.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:42 smithi167 ceph-mon[37140]: pgmap v10734: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:04:44.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:44 smithi138 ceph-mon[36954]: pgmap v10735: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:04:44.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:44 smithi100 ceph-mon[32599]: pgmap v10735: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:04:44.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:44 smithi167 ceph-mon[37140]: pgmap v10735: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:04:45.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:04:45.020+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:04:45.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:04:45.020+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:04:45.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:04:45.020+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:04:45.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:04:45.020+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:04:45.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:04:45.020+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:04:46.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:46 smithi138 ceph-mon[36954]: pgmap v10736: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:04:46.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:46 smithi100 ceph-mon[32599]: pgmap v10736: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:04:46.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:46 smithi167 ceph-mon[37140]: pgmap v10736: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:04:48.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:48 smithi138 ceph-mon[36954]: pgmap v10737: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:04:48.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:48 smithi100 ceph-mon[32599]: pgmap v10737: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:04:48.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:48 smithi167 ceph-mon[37140]: pgmap v10737: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:04:50.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:50 smithi138 ceph-mon[36954]: pgmap v10738: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:04:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:50 smithi100 ceph-mon[32599]: pgmap v10738: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:04:50.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:50 smithi167 ceph-mon[37140]: pgmap v10738: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:04:52.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:52 smithi138 ceph-mon[36954]: pgmap v10739: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:04:52.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:52 smithi100 ceph-mon[32599]: pgmap v10739: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:04:52.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:52 smithi167 ceph-mon[37140]: pgmap v10739: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:04:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:54 smithi138 ceph-mon[36954]: pgmap v10740: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:04:54.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:54 smithi100 ceph-mon[32599]: pgmap v10740: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:04:54.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:54 smithi167 ceph-mon[37140]: pgmap v10740: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:04:56.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:56 smithi138 ceph-mon[36954]: pgmap v10741: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:04:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:56 smithi100 ceph-mon[32599]: pgmap v10741: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:04:56.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:56 smithi167 ceph-mon[37140]: pgmap v10741: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:04:58.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:04:58 smithi138 ceph-mon[36954]: pgmap v10742: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:04:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:04:58 smithi100 ceph-mon[32599]: pgmap v10742: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:04:58.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:04:58 smithi167 ceph-mon[37140]: pgmap v10742: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:05:00.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:05:00.036+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:05:00.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:05:00.036+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:05:00.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:05:00.036+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:05:00.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:05:00.036+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:05:00.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:05:00.036+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:05:00.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:00 smithi138 ceph-mon[36954]: pgmap v10743: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:05:00.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:00 smithi100 ceph-mon[32599]: pgmap v10743: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:05:00.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:00 smithi167 ceph-mon[37140]: pgmap v10743: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:05:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:02 smithi138 ceph-mon[36954]: pgmap v10744: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:05:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:05:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:05:02.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:05:02.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:05:02.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:05:02.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:05:02.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:02 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:05:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:02 smithi100 ceph-mon[32599]: pgmap v10744: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:05:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:05:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:05:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:05:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:05:02.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:05:02.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:05:02.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:02 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:05:02.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:02 smithi167 ceph-mon[37140]: pgmap v10744: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:05:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:05:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:05:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:05:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:05:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:05:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:05:02.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:02 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:05:03.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:05:03.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:05:03.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:05:03.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:05:03.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:05:03.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:05:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:05:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:05:03.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:05:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:04 smithi138 ceph-mon[36954]: pgmap v10745: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:05:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:04 smithi100 ceph-mon[32599]: pgmap v10745: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:05:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:04 smithi167 ceph-mon[37140]: pgmap v10745: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:05:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:06 smithi138 ceph-mon[36954]: pgmap v10746: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:05:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:06 smithi100 ceph-mon[32599]: pgmap v10746: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:05:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:06 smithi167 ceph-mon[37140]: pgmap v10746: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:05:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:08 smithi138 ceph-mon[36954]: pgmap v10747: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T19:05:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:08 smithi138 ceph-mon[36954]: osdmap e718: 12 total, 12 up, 12 in 2024-07-26T19:05:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:08 smithi100 ceph-mon[32599]: pgmap v10747: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T19:05:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:08 smithi100 ceph-mon[32599]: osdmap e718: 12 total, 12 up, 12 in 2024-07-26T19:05:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:08 smithi167 ceph-mon[37140]: pgmap v10747: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T19:05:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:08 smithi167 ceph-mon[37140]: osdmap e718: 12 total, 12 up, 12 in 2024-07-26T19:05:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:10 smithi138 ceph-mon[36954]: pgmap v10749: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T19:05:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:10 smithi100 ceph-mon[32599]: pgmap v10749: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T19:05:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:10 smithi167 ceph-mon[37140]: pgmap v10749: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T19:05:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:12 smithi138 ceph-mon[36954]: pgmap v10750: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T19:05:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:12 smithi100 ceph-mon[32599]: pgmap v10750: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T19:05:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:12 smithi167 ceph-mon[37140]: pgmap v10750: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T19:05:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:14 smithi138 ceph-mon[36954]: pgmap v10751: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:05:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:14 smithi138 ceph-mon[36954]: osdmap e719: 12 total, 12 up, 12 in 2024-07-26T19:05:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:14 smithi100 ceph-mon[32599]: pgmap v10751: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:05:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:14 smithi100 ceph-mon[32599]: osdmap e719: 12 total, 12 up, 12 in 2024-07-26T19:05:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:14 smithi167 ceph-mon[37140]: pgmap v10751: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:05:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:14 smithi167 ceph-mon[37140]: osdmap e719: 12 total, 12 up, 12 in 2024-07-26T19:05:15.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:05:15.052+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:05:15.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:05:15.053+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:05:15.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:05:15.053+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:05:15.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:05:15.053+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:05:15.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:05:15.053+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:05:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:16 smithi138 ceph-mon[36954]: pgmap v10753: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T19:05:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:16 smithi100 ceph-mon[32599]: pgmap v10753: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T19:05:17.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:16 smithi167 ceph-mon[37140]: pgmap v10753: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T19:05:18.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:18 smithi138 ceph-mon[36954]: pgmap v10754: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 219 B/s wr, 6 op/s 2024-07-26T19:05:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:18 smithi100 ceph-mon[32599]: pgmap v10754: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 219 B/s wr, 6 op/s 2024-07-26T19:05:19.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:18 smithi167 ceph-mon[37140]: pgmap v10754: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 219 B/s wr, 6 op/s 2024-07-26T19:05:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:20 smithi138 ceph-mon[36954]: pgmap v10755: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:05:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:20 smithi100 ceph-mon[32599]: pgmap v10755: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:05:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:20 smithi167 ceph-mon[37140]: pgmap v10755: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:05:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:22 smithi138 ceph-mon[36954]: pgmap v10756: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:05:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:22 smithi100 ceph-mon[32599]: pgmap v10756: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:05:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:22 smithi167 ceph-mon[37140]: pgmap v10756: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:05:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:24 smithi138 ceph-mon[36954]: pgmap v10757: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:05:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:24 smithi100 ceph-mon[32599]: pgmap v10757: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:05:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:24 smithi167 ceph-mon[37140]: pgmap v10757: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:05:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:26 smithi138 ceph-mon[36954]: pgmap v10758: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:05:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:26 smithi100 ceph-mon[32599]: pgmap v10758: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:05:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:26 smithi167 ceph-mon[37140]: pgmap v10758: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:05:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:28 smithi138 ceph-mon[36954]: pgmap v10759: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:05:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:28 smithi100 ceph-mon[32599]: pgmap v10759: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:05:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:28 smithi167 ceph-mon[37140]: pgmap v10759: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:05:30.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:05:30.068+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:05:30.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:05:30.069+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:05:30.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:05:30.069+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:05:30.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:05:30.069+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:05:30.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:05:30.069+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:05:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:30 smithi138 ceph-mon[36954]: pgmap v10760: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:05:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:30 smithi100 ceph-mon[32599]: pgmap v10760: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:05:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:30 smithi167 ceph-mon[37140]: pgmap v10760: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:05:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:32 smithi138 ceph-mon[36954]: pgmap v10761: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:05:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:32 smithi100 ceph-mon[32599]: pgmap v10761: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:05:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:32 smithi167 ceph-mon[37140]: pgmap v10761: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:05:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:34 smithi138 ceph-mon[36954]: pgmap v10762: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:05:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:34 smithi100 ceph-mon[32599]: pgmap v10762: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:05:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:34 smithi167 ceph-mon[37140]: pgmap v10762: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:05:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:36 smithi138 ceph-mon[36954]: pgmap v10763: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:05:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:36 smithi100 ceph-mon[32599]: pgmap v10763: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:05:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:36 smithi167 ceph-mon[37140]: pgmap v10763: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:05:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:38 smithi138 ceph-mon[36954]: pgmap v10764: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:05:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:38 smithi100 ceph-mon[32599]: pgmap v10764: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:05:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:38 smithi167 ceph-mon[37140]: pgmap v10764: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:05:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:40 smithi138 ceph-mon[36954]: pgmap v10765: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:05:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:40 smithi100 ceph-mon[32599]: pgmap v10765: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:05:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:40 smithi167 ceph-mon[37140]: pgmap v10765: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:05:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:42 smithi138 ceph-mon[36954]: pgmap v10766: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:05:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:42 smithi100 ceph-mon[32599]: pgmap v10766: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:05:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:42 smithi167 ceph-mon[37140]: pgmap v10766: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:05:44.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:44 smithi138 ceph-mon[36954]: pgmap v10767: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:05:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:44 smithi100 ceph-mon[32599]: pgmap v10767: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:05:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:44 smithi167 ceph-mon[37140]: pgmap v10767: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:05:45.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:05:45.084+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:05:45.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:05:45.085+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:05:45.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:05:45.085+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:05:45.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:05:45.085+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:05:45.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:05:45.085+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:05:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:46 smithi138 ceph-mon[36954]: pgmap v10768: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:05:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:46 smithi100 ceph-mon[32599]: pgmap v10768: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:05:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:46 smithi167 ceph-mon[37140]: pgmap v10768: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:05:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:48 smithi138 ceph-mon[36954]: pgmap v10769: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:05:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:48 smithi100 ceph-mon[32599]: pgmap v10769: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:05:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:48 smithi167 ceph-mon[37140]: pgmap v10769: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:05:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:50 smithi138 ceph-mon[36954]: pgmap v10770: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:05:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:50 smithi100 ceph-mon[32599]: pgmap v10770: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:05:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:50 smithi167 ceph-mon[37140]: pgmap v10770: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:05:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:52 smithi138 ceph-mon[36954]: pgmap v10771: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:05:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:52 smithi100 ceph-mon[32599]: pgmap v10771: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:05:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:52 smithi167 ceph-mon[37140]: pgmap v10771: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:05:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:54 smithi138 ceph-mon[36954]: pgmap v10772: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:05:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:54 smithi100 ceph-mon[32599]: pgmap v10772: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:05:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:54 smithi167 ceph-mon[37140]: pgmap v10772: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:05:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:56 smithi138 ceph-mon[36954]: pgmap v10773: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:05:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:56 smithi100 ceph-mon[32599]: pgmap v10773: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:05:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:56 smithi167 ceph-mon[37140]: pgmap v10773: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:05:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:58 smithi138 ceph-mon[36954]: pgmap v10774: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:05:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:58 smithi100 ceph-mon[32599]: pgmap v10774: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:05:59.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:58 smithi167 ceph-mon[37140]: pgmap v10774: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:05:59.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:05:59 smithi138 ceph-mon[36954]: pgmap v10775: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T19:05:59.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:05:59 smithi100 ceph-mon[32599]: pgmap v10775: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T19:06:00.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:05:59 smithi167 ceph-mon[37140]: pgmap v10775: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-07-26T19:06:00.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:06:00.101+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:06:00.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:06:00.101+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:06:00.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:06:00.101+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:06:00.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:06:00.101+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:06:00.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:06:00.101+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:06:02.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:02 smithi167 ceph-mon[37140]: pgmap v10776: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:06:02.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:02 smithi138 ceph-mon[36954]: pgmap v10776: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:06:02.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:02 smithi100 ceph-mon[32599]: pgmap v10776: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:06:03.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:03 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:06:03.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:03 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:06:03.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:03 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:06:04.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:04 smithi138 ceph-mon[36954]: pgmap v10777: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:06:04.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:06:04.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:06:04.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:06:04.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:06:04.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:06:04.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:06:04.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:06:04.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:06:04.692 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:06:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:04 smithi100 ceph-mon[32599]: pgmap v10777: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:06:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:06:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:06:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:06:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:06:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:06:04.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:06:04.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:06:04.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:06:04.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:06:04.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:04 smithi167 ceph-mon[37140]: pgmap v10777: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:06:04.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:06:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:06:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:06:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:06:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:06:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:06:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:06:04.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:06:04.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:06:05.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:06:05.727+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c006660 con 0x7fe5a81087a0 2024-07-26T19:06:05.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:06:05.727+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T19:06:05.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:06:05.727+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 149 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c006660 con 0x7fe5a81087a0 2024-07-26T19:06:05.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:06:05.727+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 149 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T19:06:06.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:06 smithi138 ceph-mon[36954]: pgmap v10778: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:06:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:06 smithi100 ceph-mon[32599]: pgmap v10778: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:06:06.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:06 smithi167 ceph-mon[37140]: pgmap v10778: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:06:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:08 smithi138 ceph-mon[36954]: pgmap v10779: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T19:06:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:08 smithi138 ceph-mon[36954]: osdmap e720: 12 total, 12 up, 12 in 2024-07-26T19:06:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:08 smithi100 ceph-mon[32599]: pgmap v10779: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T19:06:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:08 smithi100 ceph-mon[32599]: osdmap e720: 12 total, 12 up, 12 in 2024-07-26T19:06:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:08 smithi167 ceph-mon[37140]: pgmap v10779: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T19:06:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:08 smithi167 ceph-mon[37140]: osdmap e720: 12 total, 12 up, 12 in 2024-07-26T19:06:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:10 smithi138 ceph-mon[36954]: pgmap v10781: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T19:06:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:10 smithi100 ceph-mon[32599]: pgmap v10781: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T19:06:10.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:10 smithi167 ceph-mon[37140]: pgmap v10781: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T19:06:12.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:12 smithi138 ceph-mon[36954]: pgmap v10782: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T19:06:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:12 smithi100 ceph-mon[32599]: pgmap v10782: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T19:06:12.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:12 smithi167 ceph-mon[37140]: pgmap v10782: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-07-26T19:06:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:14 smithi138 ceph-mon[36954]: pgmap v10783: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:06:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:14 smithi138 ceph-mon[36954]: osdmap e721: 12 total, 12 up, 12 in 2024-07-26T19:06:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:14 smithi100 ceph-mon[32599]: pgmap v10783: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:06:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:14 smithi100 ceph-mon[32599]: osdmap e721: 12 total, 12 up, 12 in 2024-07-26T19:06:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:14 smithi167 ceph-mon[37140]: pgmap v10783: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:06:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:14 smithi167 ceph-mon[37140]: osdmap e721: 12 total, 12 up, 12 in 2024-07-26T19:06:15.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:06:15.117+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:06:15.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:06:15.117+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:06:15.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:06:15.117+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:06:15.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:06:15.117+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:06:15.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:06:15.117+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:06:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:16 smithi138 ceph-mon[36954]: pgmap v10785: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T19:06:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:16 smithi100 ceph-mon[32599]: pgmap v10785: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T19:06:16.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:16 smithi167 ceph-mon[37140]: pgmap v10785: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T19:06:18.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:18 smithi138 ceph-mon[36954]: pgmap v10786: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 214 B/s wr, 6 op/s 2024-07-26T19:06:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:18 smithi100 ceph-mon[32599]: pgmap v10786: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 214 B/s wr, 6 op/s 2024-07-26T19:06:18.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:18 smithi167 ceph-mon[37140]: pgmap v10786: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 214 B/s wr, 6 op/s 2024-07-26T19:06:20.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:20 smithi138 ceph-mon[36954]: pgmap v10787: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:06:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:20 smithi100 ceph-mon[32599]: pgmap v10787: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:06:20.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:20 smithi167 ceph-mon[37140]: pgmap v10787: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:06:22.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:22 smithi138 ceph-mon[36954]: pgmap v10788: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:06:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:22 smithi100 ceph-mon[32599]: pgmap v10788: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:06:22.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:22 smithi167 ceph-mon[37140]: pgmap v10788: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:06:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:24 smithi138 ceph-mon[36954]: pgmap v10789: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:06:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:24 smithi100 ceph-mon[32599]: pgmap v10789: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:06:24.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:24 smithi167 ceph-mon[37140]: pgmap v10789: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:06:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:26 smithi138 ceph-mon[36954]: pgmap v10790: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:06:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:26 smithi100 ceph-mon[32599]: pgmap v10790: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:06:26.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:26 smithi167 ceph-mon[37140]: pgmap v10790: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:06:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:28 smithi100 ceph-mon[32599]: pgmap v10791: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:06:28.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:28 smithi167 ceph-mon[37140]: pgmap v10791: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:06:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:28 smithi138 ceph-mon[36954]: pgmap v10791: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:06:30.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:06:30.124+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:06:30.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:06:30.124+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:06:30.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:06:30.124+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:06:30.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:06:30.124+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:06:30.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:06:30.124+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:06:30.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:30 smithi100 ceph-mon[32599]: pgmap v10792: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:06:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:30 smithi167 ceph-mon[37140]: pgmap v10792: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:06:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:30 smithi138 ceph-mon[36954]: pgmap v10792: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:06:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:32 smithi100 ceph-mon[32599]: pgmap v10793: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:06:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:32 smithi167 ceph-mon[37140]: pgmap v10793: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:06:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:32 smithi138 ceph-mon[36954]: pgmap v10793: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:06:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:34 smithi100 ceph-mon[32599]: pgmap v10794: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:06:34.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:34 smithi167 ceph-mon[37140]: pgmap v10794: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:06:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:34 smithi138 ceph-mon[36954]: pgmap v10794: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:06:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:36 smithi100 ceph-mon[32599]: pgmap v10795: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:06:36.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:36 smithi167 ceph-mon[37140]: pgmap v10795: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:06:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:36 smithi138 ceph-mon[36954]: pgmap v10795: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:06:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:38 smithi100 ceph-mon[32599]: pgmap v10796: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:06:38.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:38 smithi167 ceph-mon[37140]: pgmap v10796: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:06:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:38 smithi138 ceph-mon[36954]: pgmap v10796: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:06:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:40 smithi100 ceph-mon[32599]: pgmap v10797: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:06:40.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:40 smithi167 ceph-mon[37140]: pgmap v10797: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:06:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:40 smithi138 ceph-mon[36954]: pgmap v10797: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:06:42.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:42 smithi100 ceph-mon[32599]: pgmap v10798: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:06:42.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:42 smithi167 ceph-mon[37140]: pgmap v10798: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:06:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:42 smithi138 ceph-mon[36954]: pgmap v10798: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:06:44.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:44 smithi100 ceph-mon[32599]: pgmap v10799: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:06:44.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:44 smithi167 ceph-mon[37140]: pgmap v10799: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:06:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:44 smithi138 ceph-mon[36954]: pgmap v10799: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:06:45.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:06:45.140+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:06:45.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:06:45.141+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:06:45.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:06:45.141+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:06:45.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:06:45.141+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:06:45.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:06:45.141+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:06:46.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:46 smithi100 ceph-mon[32599]: pgmap v10800: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:06:46.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:46 smithi167 ceph-mon[37140]: pgmap v10800: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:06:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:46 smithi138 ceph-mon[36954]: pgmap v10800: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:06:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:48 smithi100 ceph-mon[32599]: pgmap v10801: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:06:48.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:48 smithi167 ceph-mon[37140]: pgmap v10801: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:06:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:48 smithi138 ceph-mon[36954]: pgmap v10801: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:06:50.742 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:50 smithi138 ceph-mon[36954]: pgmap v10802: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:06:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:50 smithi100 ceph-mon[32599]: pgmap v10802: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:06:50.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:50 smithi167 ceph-mon[37140]: pgmap v10802: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:06:52.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:52 smithi100 ceph-mon[32599]: pgmap v10803: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:06:52.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:52 smithi167 ceph-mon[37140]: pgmap v10803: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:06:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:52 smithi138 ceph-mon[36954]: pgmap v10803: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:06:54.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:54 smithi100 ceph-mon[32599]: pgmap v10804: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:06:54.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:54 smithi167 ceph-mon[37140]: pgmap v10804: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:06:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:54 smithi138 ceph-mon[36954]: pgmap v10804: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:06:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:56 smithi100 ceph-mon[32599]: pgmap v10805: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:06:56.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:56 smithi167 ceph-mon[37140]: pgmap v10805: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:06:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:56 smithi138 ceph-mon[36954]: pgmap v10805: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:06:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:06:58 smithi100 ceph-mon[32599]: pgmap v10806: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:06:58.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:06:58 smithi167 ceph-mon[37140]: pgmap v10806: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:06:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:06:58 smithi138 ceph-mon[36954]: pgmap v10806: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:07:00.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:07:00.156+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:07:00.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:07:00.157+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:07:00.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:07:00.157+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:07:00.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:07:00.157+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:07:00.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:07:00.157+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:07:00.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:07:00 smithi100 ceph-mon[32599]: pgmap v10807: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:07:00.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:07:00 smithi167 ceph-mon[37140]: pgmap v10807: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:07:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:07:00 smithi138 ceph-mon[36954]: pgmap v10807: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-07-26T19:07:02.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:07:02 smithi167 ceph-mon[37140]: pgmap v10808: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:07:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:07:02 smithi138 ceph-mon[36954]: pgmap v10808: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:07:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:07:02 smithi100 ceph-mon[32599]: pgmap v10808: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:07:04.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:07:04 smithi167 ceph-mon[37140]: pgmap v10809: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:07:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:07:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:07:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:07:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:07:04.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:07:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:07:04.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:07:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:07:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:07:04 smithi138 ceph-mon[36954]: pgmap v10809: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:07:04.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:07:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:07:04.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:07:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:07:04.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:07:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:07:04.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:07:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:07:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:07:04 smithi100 ceph-mon[32599]: pgmap v10809: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:07:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:07:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:07:04.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:07:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:07:04.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:07:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:07:04.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:07:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:07:06.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:07:06 smithi167 ceph-mon[37140]: pgmap v10810: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:07:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:07:06 smithi138 ceph-mon[36954]: pgmap v10810: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:07:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:07:06 smithi100 ceph-mon[32599]: pgmap v10810: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:07:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:07:08 smithi167 ceph-mon[37140]: pgmap v10811: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T19:07:08.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:07:08 smithi167 ceph-mon[37140]: osdmap e722: 12 total, 12 up, 12 in 2024-07-26T19:07:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:07:08 smithi138 ceph-mon[36954]: pgmap v10811: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T19:07:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:07:08 smithi138 ceph-mon[36954]: osdmap e722: 12 total, 12 up, 12 in 2024-07-26T19:07:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:07:08 smithi100 ceph-mon[32599]: pgmap v10811: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-07-26T19:07:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:07:08 smithi100 ceph-mon[32599]: osdmap e722: 12 total, 12 up, 12 in 2024-07-26T19:07:10.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:07:10 smithi167 ceph-mon[37140]: pgmap v10813: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:07:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:07:10 smithi138 ceph-mon[36954]: pgmap v10813: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:07:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:07:10 smithi100 ceph-mon[32599]: pgmap v10813: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:07:12.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:07:12 smithi167 ceph-mon[37140]: pgmap v10814: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:07:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:07:12 smithi138 ceph-mon[36954]: pgmap v10814: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:07:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:07:12 smithi100 ceph-mon[32599]: pgmap v10814: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:07:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:07:14 smithi167 ceph-mon[37140]: pgmap v10815: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:07:14.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:07:14 smithi167 ceph-mon[37140]: osdmap e723: 12 total, 12 up, 12 in 2024-07-26T19:07:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:07:14 smithi138 ceph-mon[36954]: pgmap v10815: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:07:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:07:14 smithi138 ceph-mon[36954]: osdmap e723: 12 total, 12 up, 12 in 2024-07-26T19:07:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:07:14 smithi100 ceph-mon[32599]: pgmap v10815: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:07:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:07:14 smithi100 ceph-mon[32599]: osdmap e723: 12 total, 12 up, 12 in 2024-07-26T19:07:15.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:07:15.160+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:07:15.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:07:15.161+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:07:15.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:07:15.161+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:07:15.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:07:15.161+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:07:15.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:07:15.161+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:07:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:07:16 smithi138 ceph-mon[36954]: pgmap v10817: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T19:07:16.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:07:16 smithi100 ceph-mon[32599]: pgmap v10817: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T19:07:17.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:07:16 smithi167 ceph-mon[37140]: pgmap v10817: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-07-26T19:07:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:07:18 smithi138 ceph-mon[36954]: pgmap v10818: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 216 B/s wr, 6 op/s 2024-07-26T19:07:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:07:18 smithi100 ceph-mon[32599]: pgmap v10818: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 216 B/s wr, 6 op/s 2024-07-26T19:07:19.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:07:18 smithi167 ceph-mon[37140]: pgmap v10818: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 216 B/s wr, 6 op/s 2024-07-26T19:07:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:07:20 smithi138 ceph-mon[36954]: pgmap v10819: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:07:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:07:20 smithi100 ceph-mon[32599]: pgmap v10819: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:07:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:07:20 smithi167 ceph-mon[37140]: pgmap v10819: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:07:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:07:22 smithi138 ceph-mon[36954]: pgmap v10820: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:07:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:07:22 smithi100 ceph-mon[32599]: pgmap v10820: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:07:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:07:22 smithi167 ceph-mon[37140]: pgmap v10820: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:07:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:07:24 smithi138 ceph-mon[36954]: pgmap v10821: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:07:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:07:24 smithi100 ceph-mon[32599]: pgmap v10821: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:07:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:07:24 smithi167 ceph-mon[37140]: pgmap v10821: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:07:26.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:07:26 smithi138 ceph-mon[36954]: pgmap v10822: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:07:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:07:26 smithi100 ceph-mon[32599]: pgmap v10822: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:07:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:07:26 smithi167 ceph-mon[37140]: pgmap v10822: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:07:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:07:28 smithi138 ceph-mon[36954]: pgmap v10823: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:07:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:07:28 smithi100 ceph-mon[32599]: pgmap v10823: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:07:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:07:28 smithi167 ceph-mon[37140]: pgmap v10823: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:07:30.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:07:30.167+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:07:30.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:07:30.168+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:07:30.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:07:30.168+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:07:30.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:07:30.168+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:07:30.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:07:30.168+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:07:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:07:30 smithi138 ceph-mon[36954]: pgmap v10824: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:07:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:07:30 smithi100 ceph-mon[32599]: pgmap v10824: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:07:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:07:30 smithi167 ceph-mon[37140]: pgmap v10824: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:07:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:07:32 smithi138 ceph-mon[36954]: pgmap v10825: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:07:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:07:32 smithi100 ceph-mon[32599]: pgmap v10825: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:07:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:07:32 smithi167 ceph-mon[37140]: pgmap v10825: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:07:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:07:34 smithi138 ceph-mon[36954]: pgmap v10826: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:07:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:07:34 smithi100 ceph-mon[32599]: pgmap v10826: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:07:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:07:34 smithi167 ceph-mon[37140]: pgmap v10826: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:07:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:07:36 smithi138 ceph-mon[36954]: pgmap v10827: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:07:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:07:36 smithi100 ceph-mon[32599]: pgmap v10827: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:07:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:07:36 smithi167 ceph-mon[37140]: pgmap v10827: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:07:38.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:07:38 smithi138 ceph-mon[36954]: pgmap v10828: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:07:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:07:38 smithi100 ceph-mon[32599]: pgmap v10828: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:07:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:07:38 smithi167 ceph-mon[37140]: pgmap v10828: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:07:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:07:40 smithi138 ceph-mon[36954]: pgmap v10829: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:07:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:07:40 smithi100 ceph-mon[32599]: pgmap v10829: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:07:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:07:40 smithi167 ceph-mon[37140]: pgmap v10829: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:07:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:07:42 smithi138 ceph-mon[36954]: pgmap v10830: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:07:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:07:42 smithi100 ceph-mon[32599]: pgmap v10830: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:07:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:07:42 smithi167 ceph-mon[37140]: pgmap v10830: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:07:44.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:07:44 smithi167 ceph-mon[37140]: pgmap v10831: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:07:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:07:44 smithi138 ceph-mon[36954]: pgmap v10831: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:07:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:07:44 smithi100 ceph-mon[32599]: pgmap v10831: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:07:45.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:07:45.184+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:07:45.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:07:45.185+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:07:45.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:07:45.185+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:07:45.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:07:45.185+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:07:45.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:07:45.185+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:07:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:07:46 smithi138 ceph-mon[36954]: pgmap v10832: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:07:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:07:46 smithi100 ceph-mon[32599]: pgmap v10832: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:07:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:07:46 smithi167 ceph-mon[37140]: pgmap v10832: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:07:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:07:48 smithi138 ceph-mon[36954]: pgmap v10833: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:07:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:07:48 smithi100 ceph-mon[32599]: pgmap v10833: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:07:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:07:48 smithi167 ceph-mon[37140]: pgmap v10833: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:07:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:07:50 smithi138 ceph-mon[36954]: pgmap v10834: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:07:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:07:50 smithi100 ceph-mon[32599]: pgmap v10834: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:07:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:07:50 smithi167 ceph-mon[37140]: pgmap v10834: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:07:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:07:52 smithi138 ceph-mon[36954]: pgmap v10835: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:07:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:07:52 smithi100 ceph-mon[32599]: pgmap v10835: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:07:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:07:52 smithi167 ceph-mon[37140]: pgmap v10835: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:07:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:07:54 smithi138 ceph-mon[36954]: pgmap v10836: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:07:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:07:54 smithi100 ceph-mon[32599]: pgmap v10836: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:07:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:07:54 smithi167 ceph-mon[37140]: pgmap v10836: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:07:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:07:56 smithi138 ceph-mon[36954]: pgmap v10837: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:07:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:07:56 smithi100 ceph-mon[32599]: pgmap v10837: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:07:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:07:56 smithi167 ceph-mon[37140]: pgmap v10837: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:07:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:07:58 smithi138 ceph-mon[36954]: pgmap v10838: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:07:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:07:58 smithi100 ceph-mon[32599]: pgmap v10838: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:07:59.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:07:58 smithi167 ceph-mon[37140]: pgmap v10838: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:08:00.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:08:00.200+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:08:00.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:08:00.200+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:08:00.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:08:00.200+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:08:00.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:08:00.200+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:08:00.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:08:00.200+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:08:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:08:00 smithi138 ceph-mon[36954]: pgmap v10839: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:08:00.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:08:00 smithi100 ceph-mon[32599]: pgmap v10839: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:08:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:08:00 smithi167 ceph-mon[37140]: pgmap v10839: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:08:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:08:02 smithi138 ceph-mon[36954]: pgmap v10840: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:08:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:08:02 smithi100 ceph-mon[32599]: pgmap v10840: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:08:03.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:08:02 smithi167 ceph-mon[37140]: pgmap v10840: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:08:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:08:04 smithi138 ceph-mon[36954]: pgmap v10841: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:08:04.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:08:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:08:04.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:08:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:08:04.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:08:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:08:04.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:08:04 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:08:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:08:04 smithi100 ceph-mon[32599]: pgmap v10841: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:08:04.999 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:08:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:08:04.999 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:08:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:08:04.999 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:08:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:08:05.000 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:08:04 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:08:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:08:04 smithi167 ceph-mon[37140]: pgmap v10841: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:08:05.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:08:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:08:05.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:08:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:08:05.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:08:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:08:05.027 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:08:04 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:08:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:08:06 smithi138 ceph-mon[36954]: pgmap v10842: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:08:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:08:06 smithi100 ceph-mon[32599]: pgmap v10842: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:08:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:08:06 smithi167 ceph-mon[37140]: pgmap v10842: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:08:08.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:08:08 smithi138 ceph-mon[36954]: pgmap v10843: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:08:08.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:08:08 smithi138 ceph-mon[36954]: osdmap e724: 12 total, 12 up, 12 in 2024-07-26T19:08:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:08:08 smithi100 ceph-mon[32599]: pgmap v10843: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:08:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:08:08 smithi100 ceph-mon[32599]: osdmap e724: 12 total, 12 up, 12 in 2024-07-26T19:08:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:08:08 smithi167 ceph-mon[37140]: pgmap v10843: 97 pgs: 97 active+clean; 235 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:08:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:08:08 smithi167 ceph-mon[37140]: osdmap e724: 12 total, 12 up, 12 in 2024-07-26T19:08:10.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:08:10 smithi138 ceph-mon[36954]: pgmap v10845: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:08:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:08:10 smithi100 ceph-mon[32599]: pgmap v10845: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:08:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:08:10 smithi167 ceph-mon[37140]: pgmap v10845: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:08:12.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:08:12 smithi138 ceph-mon[36954]: pgmap v10846: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:08:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:08:12 smithi100 ceph-mon[32599]: pgmap v10846: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:08:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:08:12 smithi167 ceph-mon[37140]: pgmap v10846: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:08:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:08:14 smithi138 ceph-mon[36954]: pgmap v10847: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:08:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:08:14 smithi138 ceph-mon[36954]: osdmap e725: 12 total, 12 up, 12 in 2024-07-26T19:08:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:08:14 smithi100 ceph-mon[32599]: pgmap v10847: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:08:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:08:14 smithi100 ceph-mon[32599]: osdmap e725: 12 total, 12 up, 12 in 2024-07-26T19:08:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:08:14 smithi167 ceph-mon[37140]: pgmap v10847: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:08:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:08:14 smithi167 ceph-mon[37140]: osdmap e725: 12 total, 12 up, 12 in 2024-07-26T19:08:15.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:08:15.207+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:08:15.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:08:15.208+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:08:15.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:08:15.208+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:08:15.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:08:15.208+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:08:15.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:08:15.208+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:08:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:08:16 smithi138 ceph-mon[36954]: pgmap v10849: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T19:08:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:08:16 smithi100 ceph-mon[32599]: pgmap v10849: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T19:08:17.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:08:16 smithi167 ceph-mon[37140]: pgmap v10849: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T19:08:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:08:18 smithi138 ceph-mon[36954]: pgmap v10850: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 218 B/s wr, 6 op/s 2024-07-26T19:08:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:08:18 smithi100 ceph-mon[32599]: pgmap v10850: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 218 B/s wr, 6 op/s 2024-07-26T19:08:19.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:08:18 smithi167 ceph-mon[37140]: pgmap v10850: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 218 B/s wr, 6 op/s 2024-07-26T19:08:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:08:20 smithi138 ceph-mon[36954]: pgmap v10851: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:08:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:08:20 smithi100 ceph-mon[32599]: pgmap v10851: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:08:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:08:20 smithi167 ceph-mon[37140]: pgmap v10851: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:08:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:08:22 smithi138 ceph-mon[36954]: pgmap v10852: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:08:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:08:22 smithi100 ceph-mon[32599]: pgmap v10852: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:08:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:08:22 smithi167 ceph-mon[37140]: pgmap v10852: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:08:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:08:24 smithi138 ceph-mon[36954]: pgmap v10853: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:08:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:08:24 smithi100 ceph-mon[32599]: pgmap v10853: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:08:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:08:24 smithi167 ceph-mon[37140]: pgmap v10853: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:08:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:08:26 smithi138 ceph-mon[36954]: pgmap v10854: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:08:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:08:26 smithi100 ceph-mon[32599]: pgmap v10854: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:08:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:08:26 smithi167 ceph-mon[37140]: pgmap v10854: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:08:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:08:28 smithi138 ceph-mon[36954]: pgmap v10855: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:08:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:08:28 smithi100 ceph-mon[32599]: pgmap v10855: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:08:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:08:28 smithi167 ceph-mon[37140]: pgmap v10855: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:08:30.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:08:30.223+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:08:30.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:08:30.224+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:08:30.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:08:30.224+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:08:30.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:08:30.224+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:08:30.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:08:30.224+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:08:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:08:30 smithi138 ceph-mon[36954]: pgmap v10856: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:08:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:08:30 smithi100 ceph-mon[32599]: pgmap v10856: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:08:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:08:30 smithi167 ceph-mon[37140]: pgmap v10856: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:08:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:08:32 smithi138 ceph-mon[36954]: pgmap v10857: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:08:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:08:32 smithi100 ceph-mon[32599]: pgmap v10857: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:08:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:08:32 smithi167 ceph-mon[37140]: pgmap v10857: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:08:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:08:34 smithi138 ceph-mon[36954]: pgmap v10858: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:08:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:08:34 smithi100 ceph-mon[32599]: pgmap v10858: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:08:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:08:34 smithi167 ceph-mon[37140]: pgmap v10858: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:08:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:08:36 smithi138 ceph-mon[36954]: pgmap v10859: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:08:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:08:36 smithi100 ceph-mon[32599]: pgmap v10859: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:08:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:08:36 smithi167 ceph-mon[37140]: pgmap v10859: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:08:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:08:38 smithi138 ceph-mon[36954]: pgmap v10860: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:08:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:08:38 smithi100 ceph-mon[32599]: pgmap v10860: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:08:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:08:38 smithi167 ceph-mon[37140]: pgmap v10860: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:08:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:08:40 smithi138 ceph-mon[36954]: pgmap v10861: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:08:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:08:40 smithi100 ceph-mon[32599]: pgmap v10861: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:08:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:08:40 smithi167 ceph-mon[37140]: pgmap v10861: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:08:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:08:42 smithi138 ceph-mon[36954]: pgmap v10862: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:08:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:08:42 smithi100 ceph-mon[32599]: pgmap v10862: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:08:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:08:42 smithi167 ceph-mon[37140]: pgmap v10862: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:08:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:08:44 smithi138 ceph-mon[36954]: pgmap v10863: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:08:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:08:44 smithi100 ceph-mon[32599]: pgmap v10863: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:08:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:08:44 smithi167 ceph-mon[37140]: pgmap v10863: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:08:45.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:08:45.239+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:08:45.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:08:45.239+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:08:45.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:08:45.239+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:08:45.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:08:45.239+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:08:45.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:08:45.239+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:08:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:08:46 smithi138 ceph-mon[36954]: pgmap v10864: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:08:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:08:46 smithi100 ceph-mon[32599]: pgmap v10864: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:08:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:08:46 smithi167 ceph-mon[37140]: pgmap v10864: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:08:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:08:48 smithi138 ceph-mon[36954]: pgmap v10865: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:08:48.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:08:48 smithi100 ceph-mon[32599]: pgmap v10865: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:08:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:08:48 smithi167 ceph-mon[37140]: pgmap v10865: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:08:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:08:50 smithi138 ceph-mon[36954]: pgmap v10866: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:08:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:08:50 smithi100 ceph-mon[32599]: pgmap v10866: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:08:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:08:50 smithi167 ceph-mon[37140]: pgmap v10866: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:08:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:08:52 smithi138 ceph-mon[36954]: pgmap v10867: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:08:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:08:52 smithi100 ceph-mon[32599]: pgmap v10867: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:08:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:08:52 smithi167 ceph-mon[37140]: pgmap v10867: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:08:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:08:54 smithi138 ceph-mon[36954]: pgmap v10868: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:08:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:08:54 smithi100 ceph-mon[32599]: pgmap v10868: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:08:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:08:54 smithi167 ceph-mon[37140]: pgmap v10868: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:08:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:08:56 smithi100 ceph-mon[32599]: pgmap v10869: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:08:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:08:56 smithi167 ceph-mon[37140]: pgmap v10869: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:08:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:08:56 smithi138 ceph-mon[36954]: pgmap v10869: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:08:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:08:58 smithi100 ceph-mon[32599]: pgmap v10870: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:08:59.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:08:58 smithi167 ceph-mon[37140]: pgmap v10870: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:08:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:08:58 smithi138 ceph-mon[36954]: pgmap v10870: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:09:00.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:09:00.256+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:09:00.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:09:00.256+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:09:00.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:09:00.256+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:09:00.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:09:00.256+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:09:00.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:09:00.256+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:09:00.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:09:00 smithi100 ceph-mon[32599]: pgmap v10871: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:09:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:09:00 smithi167 ceph-mon[37140]: pgmap v10871: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:09:01.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:09:00 smithi138 ceph-mon[36954]: pgmap v10871: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:09:02.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:09:01 smithi167 ceph-mon[37140]: pgmap v10872: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:09:02.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:09:01 smithi138 ceph-mon[36954]: pgmap v10872: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:09:02.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:09:01 smithi100 ceph-mon[32599]: pgmap v10872: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:09:04.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:09:04 smithi138 ceph-mon[36954]: pgmap v10873: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:09:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:09:04 smithi100 ceph-mon[32599]: pgmap v10873: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:09:04.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:09:04 smithi167 ceph-mon[37140]: pgmap v10873: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:09:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:09:05 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:09:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:09:05 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:09:05.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:09:05 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:09:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:09:05 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:09:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:09:05 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:09:05.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:09:05 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:09:05.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:09:05 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:09:05.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:09:05 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:09:05.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:09:05 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:09:05.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:09:05.753+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T19:09:05.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:09:05.753+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c006660 con 0x7fe5a81087a0 2024-07-26T19:09:05.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:09:05.753+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 150 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T19:09:05.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:09:05.753+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 150 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c006660 con 0x7fe5a81087a0 2024-07-26T19:09:06.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:09:06 smithi138 ceph-mon[36954]: pgmap v10874: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:09:06.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:09:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:09:06.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:09:06 smithi100 ceph-mon[32599]: pgmap v10874: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:09:06.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:09:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:09:06.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:09:06 smithi167 ceph-mon[37140]: pgmap v10874: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:09:06.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:09:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:09:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:09:08 smithi138 ceph-mon[36954]: pgmap v10875: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:09:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:09:08 smithi100 ceph-mon[32599]: pgmap v10875: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:09:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:09:08 smithi167 ceph-mon[37140]: pgmap v10875: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:09:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:09:09 smithi138 ceph-mon[36954]: osdmap e726: 12 total, 12 up, 12 in 2024-07-26T19:09:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:09:09 smithi100 ceph-mon[32599]: osdmap e726: 12 total, 12 up, 12 in 2024-07-26T19:09:09.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:09:09 smithi167 ceph-mon[37140]: osdmap e726: 12 total, 12 up, 12 in 2024-07-26T19:09:10.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:09:10 smithi138 ceph-mon[36954]: pgmap v10877: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:09:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:09:10 smithi100 ceph-mon[32599]: pgmap v10877: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:09:10.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:09:10 smithi167 ceph-mon[37140]: pgmap v10877: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:09:12.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:09:12 smithi138 ceph-mon[36954]: pgmap v10878: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:09:12.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:09:12 smithi100 ceph-mon[32599]: pgmap v10878: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:09:12.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:09:12 smithi167 ceph-mon[37140]: pgmap v10878: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:09:14.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:09:14 smithi138 ceph-mon[36954]: pgmap v10879: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:09:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:09:14 smithi100 ceph-mon[32599]: pgmap v10879: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:09:14.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:09:14 smithi167 ceph-mon[37140]: pgmap v10879: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:09:15.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:09:15.272+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:09:15.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:09:15.272+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:09:15.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:09:15.272+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:09:15.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:09:15.272+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:09:15.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:09:15.272+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:09:16.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:09:16 smithi138 ceph-mon[36954]: pgmap v10880: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T19:09:16.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:09:16 smithi100 ceph-mon[32599]: pgmap v10880: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T19:09:16.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:09:16 smithi167 ceph-mon[37140]: pgmap v10880: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T19:09:18.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:09:18 smithi138 ceph-mon[36954]: pgmap v10881: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T19:09:18.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:09:18 smithi100 ceph-mon[32599]: pgmap v10881: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T19:09:18.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:09:18 smithi167 ceph-mon[37140]: pgmap v10881: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-07-26T19:09:20.442 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:09:20 smithi138 ceph-mon[36954]: pgmap v10882: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 187 B/s wr, 7 op/s 2024-07-26T19:09:20.443 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:09:20 smithi138 ceph-mon[36954]: osdmap e727: 12 total, 12 up, 12 in 2024-07-26T19:09:20.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:09:20 smithi100 ceph-mon[32599]: pgmap v10882: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 187 B/s wr, 7 op/s 2024-07-26T19:09:20.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:09:20 smithi100 ceph-mon[32599]: osdmap e727: 12 total, 12 up, 12 in 2024-07-26T19:09:20.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:09:20 smithi167 ceph-mon[37140]: pgmap v10882: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 187 B/s wr, 7 op/s 2024-07-26T19:09:20.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:09:20 smithi167 ceph-mon[37140]: osdmap e727: 12 total, 12 up, 12 in 2024-07-26T19:09:22.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:09:22 smithi138 ceph-mon[36954]: pgmap v10884: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:09:22.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:09:22 smithi100 ceph-mon[32599]: pgmap v10884: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:09:22.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:09:22 smithi167 ceph-mon[37140]: pgmap v10884: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:09:24.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:09:24 smithi138 ceph-mon[36954]: pgmap v10885: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:09:24.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:09:24 smithi100 ceph-mon[32599]: pgmap v10885: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:09:24.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:09:24 smithi167 ceph-mon[37140]: pgmap v10885: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:09:26.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:09:26 smithi138 ceph-mon[36954]: pgmap v10886: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:09:26.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:09:26 smithi100 ceph-mon[32599]: pgmap v10886: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:09:26.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:09:26 smithi167 ceph-mon[37140]: pgmap v10886: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:09:28.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:09:28 smithi138 ceph-mon[36954]: pgmap v10887: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:09:28.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:09:28 smithi100 ceph-mon[32599]: pgmap v10887: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:09:28.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:09:28 smithi167 ceph-mon[37140]: pgmap v10887: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:09:30.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:09:30.277+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:09:30.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:09:30.277+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:09:30.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:09:30.277+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:09:30.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:09:30.277+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:09:30.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:09:30.277+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:09:30.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:09:30 smithi138 ceph-mon[36954]: pgmap v10888: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:09:30.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:09:30 smithi100 ceph-mon[32599]: pgmap v10888: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:09:30.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:09:30 smithi167 ceph-mon[37140]: pgmap v10888: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-07-26T19:09:32.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:09:32 smithi138 ceph-mon[36954]: pgmap v10889: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:09:32.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:09:32 smithi100 ceph-mon[32599]: pgmap v10889: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:09:32.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:09:32 smithi167 ceph-mon[37140]: pgmap v10889: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:09:34.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:09:34 smithi138 ceph-mon[36954]: pgmap v10890: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:09:34.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:09:34 smithi100 ceph-mon[32599]: pgmap v10890: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:09:34.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:09:34 smithi167 ceph-mon[37140]: pgmap v10890: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:09:36.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:09:36 smithi138 ceph-mon[36954]: pgmap v10891: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:09:36.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:09:36 smithi100 ceph-mon[32599]: pgmap v10891: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:09:36.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:09:36 smithi167 ceph-mon[37140]: pgmap v10891: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:09:38.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:09:38 smithi138 ceph-mon[36954]: pgmap v10892: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:09:38.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:09:38 smithi100 ceph-mon[32599]: pgmap v10892: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:09:38.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:09:38 smithi167 ceph-mon[37140]: pgmap v10892: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:09:40.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:09:40 smithi138 ceph-mon[36954]: pgmap v10893: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:09:40.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:09:40 smithi100 ceph-mon[32599]: pgmap v10893: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:09:40.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:09:40 smithi167 ceph-mon[37140]: pgmap v10893: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:09:42.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:09:42 smithi138 ceph-mon[36954]: pgmap v10894: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:09:42.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:09:42 smithi100 ceph-mon[32599]: pgmap v10894: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:09:42.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:09:42 smithi167 ceph-mon[37140]: pgmap v10894: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:09:44.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:09:44 smithi167 ceph-mon[37140]: pgmap v10895: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:09:44.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:09:44 smithi138 ceph-mon[36954]: pgmap v10895: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:09:44.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:09:44 smithi100 ceph-mon[32599]: pgmap v10895: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:09:45.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:09:45.287+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:09:45.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:09:45.288+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:09:45.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:09:45.288+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:09:45.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:09:45.288+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:09:45.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:09:45.288+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:09:46.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:09:46 smithi138 ceph-mon[36954]: pgmap v10896: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:09:46.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:09:46 smithi100 ceph-mon[32599]: pgmap v10896: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:09:46.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:09:46 smithi167 ceph-mon[37140]: pgmap v10896: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:09:48.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:09:48 smithi138 ceph-mon[36954]: pgmap v10897: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:09:48.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:09:48 smithi100 ceph-mon[32599]: pgmap v10897: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:09:48.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:09:48 smithi167 ceph-mon[37140]: pgmap v10897: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:09:50.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:09:50 smithi138 ceph-mon[36954]: pgmap v10898: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:09:50.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:09:50 smithi100 ceph-mon[32599]: pgmap v10898: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:09:50.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:09:50 smithi167 ceph-mon[37140]: pgmap v10898: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:09:50.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:09:50.759+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe57c006660 con 0x7fe5a81087a0 2024-07-26T19:09:50.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:09:50.759+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T19:09:50.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:09:50.760+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 151 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T19:09:50.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:09:50.760+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 151 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe57c006660 con 0x7fe5a81087a0 2024-07-26T19:09:52.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:09:52 smithi138 ceph-mon[36954]: pgmap v10899: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:09:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:09:52 smithi100 ceph-mon[32599]: pgmap v10899: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:09:52.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:09:52 smithi167 ceph-mon[37140]: pgmap v10899: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:09:54.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:09:54 smithi138 ceph-mon[36954]: pgmap v10900: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:09:54.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:09:54 smithi100 ceph-mon[32599]: pgmap v10900: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:09:54.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:09:54 smithi167 ceph-mon[37140]: pgmap v10900: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:09:56.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:09:56 smithi138 ceph-mon[36954]: pgmap v10901: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:09:56.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:09:56 smithi100 ceph-mon[32599]: pgmap v10901: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:09:56.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:09:56 smithi167 ceph-mon[37140]: pgmap v10901: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:09:58.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:09:58 smithi138 ceph-mon[36954]: pgmap v10902: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:09:58.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:09:58 smithi100 ceph-mon[32599]: pgmap v10902: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:09:58.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:09:58 smithi167 ceph-mon[37140]: pgmap v10902: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:10:00.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:10:00.303+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:10:00.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:10:00.304+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:10:00.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:10:00.304+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:10:00.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:10:00.304+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:10:00.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:10:00.304+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:10:00.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:00 smithi138 ceph-mon[36954]: pgmap v10903: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:10:00.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:00 smithi138 ceph-mon[36954]: overall HEALTH_OK 2024-07-26T19:10:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:00 smithi100 ceph-mon[32599]: pgmap v10903: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:10:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:00 smithi100 ceph-mon[32599]: overall HEALTH_OK 2024-07-26T19:10:00.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:00 smithi167 ceph-mon[37140]: pgmap v10903: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:10:00.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:00 smithi167 ceph-mon[37140]: overall HEALTH_OK 2024-07-26T19:10:02.443 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:02 smithi138 ceph-mon[36954]: pgmap v10904: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:10:02.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:02 smithi100 ceph-mon[32599]: pgmap v10904: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:10:02.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:02 smithi167 ceph-mon[37140]: pgmap v10904: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:10:04.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:04 smithi138 ceph-mon[36954]: pgmap v10905: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:10:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:04 smithi100 ceph-mon[32599]: pgmap v10905: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:10:04.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:04 smithi167 ceph-mon[37140]: pgmap v10905: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:10:06.346 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:06 smithi100 ceph-mon[32599]: pgmap v10906: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:10:06.346 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:06 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:10:06.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:06 smithi138 ceph-mon[36954]: pgmap v10906: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:10:06.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:06 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:10:06.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:06 smithi167 ceph-mon[37140]: pgmap v10906: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:10:06.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:06 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:10:08.350 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:08 smithi100 ceph-mon[32599]: pgmap v10907: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:10:08.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:08 smithi138 ceph-mon[36954]: pgmap v10907: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:10:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:08 smithi167 ceph-mon[37140]: pgmap v10907: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:10:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:09 smithi138 ceph-mon[36954]: osdmap e728: 12 total, 12 up, 12 in 2024-07-26T19:10:09.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:10:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:10:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:10:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:10:09.441 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:10:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:09 smithi100 ceph-mon[32599]: osdmap e728: 12 total, 12 up, 12 in 2024-07-26T19:10:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:10:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:10:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:10:09.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:10:09.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:10:09.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:09 smithi167 ceph-mon[37140]: osdmap e728: 12 total, 12 up, 12 in 2024-07-26T19:10:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:10:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:10:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:10:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:10:09.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:10:10.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:10 smithi138 ceph-mon[36954]: pgmap v10909: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:10:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:10 smithi100 ceph-mon[32599]: pgmap v10909: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:10:10.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:10 smithi167 ceph-mon[37140]: pgmap v10909: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:10:12.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:12 smithi138 ceph-mon[36954]: pgmap v10910: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:10:12.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:12 smithi100 ceph-mon[32599]: pgmap v10910: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:10:12.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:12 smithi167 ceph-mon[37140]: pgmap v10910: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:10:14.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:14 smithi138 ceph-mon[36954]: pgmap v10911: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:10:14.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:14 smithi138 ceph-mon[36954]: osdmap e729: 12 total, 12 up, 12 in 2024-07-26T19:10:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:14 smithi100 ceph-mon[32599]: pgmap v10911: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:10:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:14 smithi100 ceph-mon[32599]: osdmap e729: 12 total, 12 up, 12 in 2024-07-26T19:10:14.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:14 smithi167 ceph-mon[37140]: pgmap v10911: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:10:14.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:14 smithi167 ceph-mon[37140]: osdmap e729: 12 total, 12 up, 12 in 2024-07-26T19:10:15.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:10:15.311+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:10:15.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:10:15.312+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:10:15.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:10:15.312+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:10:15.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:10:15.312+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:10:15.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:10:15.312+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:10:16.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:16 smithi138 ceph-mon[36954]: pgmap v10913: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T19:10:16.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:16 smithi100 ceph-mon[32599]: pgmap v10913: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T19:10:16.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:16 smithi167 ceph-mon[37140]: pgmap v10913: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T19:10:18.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:18 smithi138 ceph-mon[36954]: pgmap v10914: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T19:10:18.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:18 smithi100 ceph-mon[32599]: pgmap v10914: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T19:10:18.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:18 smithi167 ceph-mon[37140]: pgmap v10914: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 229 B/s wr, 6 op/s 2024-07-26T19:10:20.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:20 smithi138 ceph-mon[36954]: pgmap v10915: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:10:20.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:20 smithi100 ceph-mon[32599]: pgmap v10915: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:10:20.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:20 smithi167 ceph-mon[37140]: pgmap v10915: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:10:22.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:22 smithi138 ceph-mon[36954]: pgmap v10916: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:10:22.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:22 smithi100 ceph-mon[32599]: pgmap v10916: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:10:22.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:22 smithi167 ceph-mon[37140]: pgmap v10916: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:10:24.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:24 smithi138 ceph-mon[36954]: pgmap v10917: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:10:24.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:24 smithi100 ceph-mon[32599]: pgmap v10917: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:10:24.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:24 smithi167 ceph-mon[37140]: pgmap v10917: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:10:26.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:26 smithi138 ceph-mon[36954]: pgmap v10918: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:10:26.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:26 smithi100 ceph-mon[32599]: pgmap v10918: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:10:26.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:26 smithi167 ceph-mon[37140]: pgmap v10918: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:10:28.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:28 smithi138 ceph-mon[36954]: pgmap v10919: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:10:28.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:28 smithi100 ceph-mon[32599]: pgmap v10919: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:10:28.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:28 smithi167 ceph-mon[37140]: pgmap v10919: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:10:30.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:10:30.327+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:10:30.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:10:30.328+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:10:30.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:10:30.328+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:10:30.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:10:30.328+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:10:30.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:10:30.328+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:10:30.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:30 smithi138 ceph-mon[36954]: pgmap v10920: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T19:10:30.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:30 smithi100 ceph-mon[32599]: pgmap v10920: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T19:10:30.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:30 smithi167 ceph-mon[37140]: pgmap v10920: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T19:10:32.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:32 smithi138 ceph-mon[36954]: pgmap v10921: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T19:10:32.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:32 smithi100 ceph-mon[32599]: pgmap v10921: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T19:10:32.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:32 smithi167 ceph-mon[37140]: pgmap v10921: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T19:10:34.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:34 smithi138 ceph-mon[36954]: pgmap v10922: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T19:10:34.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:34 smithi100 ceph-mon[32599]: pgmap v10922: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T19:10:34.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:34 smithi167 ceph-mon[37140]: pgmap v10922: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T19:10:36.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:36 smithi138 ceph-mon[36954]: pgmap v10923: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T19:10:36.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:36 smithi100 ceph-mon[32599]: pgmap v10923: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T19:10:36.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:36 smithi167 ceph-mon[37140]: pgmap v10923: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T19:10:38.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:38 smithi138 ceph-mon[36954]: pgmap v10924: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T19:10:38.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:38 smithi100 ceph-mon[32599]: pgmap v10924: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T19:10:38.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:38 smithi167 ceph-mon[37140]: pgmap v10924: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T19:10:40.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:40 smithi138 ceph-mon[36954]: pgmap v10925: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T19:10:40.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:40 smithi100 ceph-mon[32599]: pgmap v10925: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T19:10:40.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:40 smithi167 ceph-mon[37140]: pgmap v10925: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T19:10:42.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:42 smithi138 ceph-mon[36954]: pgmap v10926: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:10:42.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:42 smithi100 ceph-mon[32599]: pgmap v10926: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:10:42.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:42 smithi167 ceph-mon[37140]: pgmap v10926: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:10:44.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:44 smithi138 ceph-mon[36954]: pgmap v10927: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:10:44.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:44 smithi100 ceph-mon[32599]: pgmap v10927: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:10:44.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:44 smithi167 ceph-mon[37140]: pgmap v10927: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:10:45.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:10:45.343+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:10:45.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:10:45.344+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:10:45.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:10:45.344+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:10:45.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:10:45.344+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:10:45.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:10:45.344+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:10:46.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:46 smithi138 ceph-mon[36954]: pgmap v10928: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:10:46.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:46 smithi100 ceph-mon[32599]: pgmap v10928: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:10:46.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:46 smithi167 ceph-mon[37140]: pgmap v10928: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:10:48.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:48 smithi138 ceph-mon[36954]: pgmap v10929: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:10:48.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:48 smithi100 ceph-mon[32599]: pgmap v10929: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:10:48.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:48 smithi167 ceph-mon[37140]: pgmap v10929: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:10:50.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:50 smithi138 ceph-mon[36954]: pgmap v10930: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:10:50.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:50 smithi100 ceph-mon[32599]: pgmap v10930: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:10:50.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:50 smithi167 ceph-mon[37140]: pgmap v10930: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:10:52.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:52 smithi138 ceph-mon[36954]: pgmap v10931: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:10:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:52 smithi100 ceph-mon[32599]: pgmap v10931: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:10:52.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:52 smithi167 ceph-mon[37140]: pgmap v10931: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:10:54.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:54 smithi100 ceph-mon[32599]: pgmap v10932: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:10:54.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:54 smithi167 ceph-mon[37140]: pgmap v10932: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:10:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:54 smithi138 ceph-mon[36954]: pgmap v10932: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:10:56.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:56 smithi100 ceph-mon[32599]: pgmap v10933: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:10:56.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:56 smithi167 ceph-mon[37140]: pgmap v10933: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:10:56.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:56 smithi138 ceph-mon[36954]: pgmap v10933: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:10:58.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:10:58 smithi100 ceph-mon[32599]: pgmap v10934: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:10:58.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:10:58 smithi167 ceph-mon[37140]: pgmap v10934: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:10:58.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:10:58 smithi138 ceph-mon[36954]: pgmap v10934: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:11:00.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:11:00.359+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:11:00.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:11:00.360+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:11:00.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:11:00.360+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:11:00.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:11:00.360+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:11:00.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:11:00.360+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:11:00.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:00 smithi100 ceph-mon[32599]: pgmap v10935: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:11:00.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:00 smithi167 ceph-mon[37140]: pgmap v10935: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:11:00.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:00 smithi138 ceph-mon[36954]: pgmap v10935: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:11:02.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:02 smithi100 ceph-mon[32599]: pgmap v10936: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:11:02.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:02 smithi167 ceph-mon[37140]: pgmap v10936: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:11:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:02 smithi138 ceph-mon[36954]: pgmap v10936: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:11:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:04 smithi100 ceph-mon[32599]: pgmap v10937: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:11:04.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:04 smithi167 ceph-mon[37140]: pgmap v10937: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:11:04.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:04 smithi138 ceph-mon[36954]: pgmap v10937: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:11:06.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:06 smithi100 ceph-mon[32599]: pgmap v10938: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:11:06.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:06 smithi167 ceph-mon[37140]: pgmap v10938: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:11:06.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:06 smithi138 ceph-mon[36954]: pgmap v10938: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:11:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:08 smithi100 ceph-mon[32599]: pgmap v10939: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:11:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:08 smithi167 ceph-mon[37140]: pgmap v10939: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:11:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:08 smithi138 ceph-mon[36954]: pgmap v10939: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:11:09.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:09 smithi100 ceph-mon[32599]: osdmap e730: 12 total, 12 up, 12 in 2024-07-26T19:11:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:09 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:11:09.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:09 smithi167 ceph-mon[37140]: osdmap e730: 12 total, 12 up, 12 in 2024-07-26T19:11:09.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:09 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:11:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:09 smithi138 ceph-mon[36954]: osdmap e730: 12 total, 12 up, 12 in 2024-07-26T19:11:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:09 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:11:10.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:10 smithi167 ceph-mon[37140]: pgmap v10941: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:11:10.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:10 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:11:10.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:10 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:11:10.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:10 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:11:10.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:10 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:11:10.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:10 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T19:11:10.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:10 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T19:11:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:10 smithi100 ceph-mon[32599]: pgmap v10941: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:11:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:10 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:11:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:10 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:11:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:10 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:11:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:10 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:11:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:10 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T19:11:10.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:10 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T19:11:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:10 smithi138 ceph-mon[36954]: pgmap v10941: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:11:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:10 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:11:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:10 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:11:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:10 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:11:10.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:10 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:11:10.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:10 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi100 to 175.9M 2024-07-26T19:11:10.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:10 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi100 to 184537292: error parsing value: Value '184537292' is below minimum 939524096 2024-07-26T19:11:12.431 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:12 smithi167 ceph-mon[37140]: pgmap v10942: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:11:12.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:12 smithi100 ceph-mon[32599]: pgmap v10942: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:11:12.544 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:12 smithi138 ceph-mon[36954]: pgmap v10942: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:11:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:11:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:11:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:11:13.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:11:13.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:11:13.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:11:13.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:13 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T19:11:13.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:13 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T19:11:13.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:11:13.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:11:13.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:11:13.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:11:13.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:11:13.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:11:13.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:13 smithi100 ceph-mon[32599]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T19:11:13.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:13 smithi100 ceph-mon[32599]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T19:11:13.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:11:13.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:11:13.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:11:13.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:11:13.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:11:13.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:11:13.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:11:13.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:11:13.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:11:13.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:13 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T19:11:13.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:13 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T19:11:13.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:11:13.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:11:13.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:11:13.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:11:13.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:11:13.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:11:13.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:13 smithi167 ceph-mon[37140]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T19:11:13.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:13 smithi167 ceph-mon[37140]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T19:11:13.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:11:13.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:11:13.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:11:13.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:11:13.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:11:13.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:11:13.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:11:13.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:11:13.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:11:13.941 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:13 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-07-26T19:11:13.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:13 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi167 to 184788172: error parsing value: Value '184788172' is below minimum 939524096 2024-07-26T19:11:13.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:11:13.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:11:13.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:11:13.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:11:13.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:11:13.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-26T19:11:13.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:13 smithi138 ceph-mon[36954]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-07-26T19:11:13.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:13 smithi138 ceph-mon[36954]: Unable to set osd_memory_target on smithi138 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-07-26T19:11:13.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:11:13.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:11:13.943 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:11:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:14 smithi138 ceph-mon[36954]: pgmap v10943: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:11:14.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:14 smithi138 ceph-mon[36954]: osdmap e731: 12 total, 12 up, 12 in 2024-07-26T19:11:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:14 smithi100 ceph-mon[32599]: pgmap v10943: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:11:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:14 smithi100 ceph-mon[32599]: osdmap e731: 12 total, 12 up, 12 in 2024-07-26T19:11:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:14 smithi167 ceph-mon[37140]: pgmap v10943: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:11:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:14 smithi167 ceph-mon[37140]: osdmap e731: 12 total, 12 up, 12 in 2024-07-26T19:11:15.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:11:15.367+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:11:15.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:11:15.368+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:11:15.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:11:15.368+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:11:15.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:11:15.368+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:11:15.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:11:15.368+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:11:16.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:16 smithi138 ceph-mon[36954]: pgmap v10945: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T19:11:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:16 smithi100 ceph-mon[32599]: pgmap v10945: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T19:11:17.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:16 smithi167 ceph-mon[37140]: pgmap v10945: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T19:11:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:18 smithi138 ceph-mon[36954]: pgmap v10946: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 231 B/s wr, 6 op/s 2024-07-26T19:11:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:18 smithi100 ceph-mon[32599]: pgmap v10946: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 231 B/s wr, 6 op/s 2024-07-26T19:11:19.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:18 smithi167 ceph-mon[37140]: pgmap v10946: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 231 B/s wr, 6 op/s 2024-07-26T19:11:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:20 smithi138 ceph-mon[36954]: pgmap v10947: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:11:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:20 smithi100 ceph-mon[32599]: pgmap v10947: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:11:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:20 smithi167 ceph-mon[37140]: pgmap v10947: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:11:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:22 smithi138 ceph-mon[36954]: pgmap v10948: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:11:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:22 smithi100 ceph-mon[32599]: pgmap v10948: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:11:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:22 smithi167 ceph-mon[37140]: pgmap v10948: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:11:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:24 smithi138 ceph-mon[36954]: pgmap v10949: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:11:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:24 smithi100 ceph-mon[32599]: pgmap v10949: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:11:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:24 smithi167 ceph-mon[37140]: pgmap v10949: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:11:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:26 smithi138 ceph-mon[36954]: pgmap v10950: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:11:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:26 smithi100 ceph-mon[32599]: pgmap v10950: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:11:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:26 smithi167 ceph-mon[37140]: pgmap v10950: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:11:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:28 smithi138 ceph-mon[36954]: pgmap v10951: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:11:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:28 smithi100 ceph-mon[32599]: pgmap v10951: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:11:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:28 smithi167 ceph-mon[37140]: pgmap v10951: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:11:30.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:11:30.383+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:11:30.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:11:30.384+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:11:30.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:11:30.384+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:11:30.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:11:30.384+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:11:30.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:11:30.384+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:11:30.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:30 smithi138 ceph-mon[36954]: pgmap v10952: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:11:30.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:30 smithi100 ceph-mon[32599]: pgmap v10952: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:11:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:30 smithi167 ceph-mon[37140]: pgmap v10952: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:11:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:32 smithi138 ceph-mon[36954]: pgmap v10953: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:11:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:32 smithi100 ceph-mon[32599]: pgmap v10953: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:11:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:32 smithi167 ceph-mon[37140]: pgmap v10953: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:11:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:34 smithi138 ceph-mon[36954]: pgmap v10954: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:11:34.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:34 smithi100 ceph-mon[32599]: pgmap v10954: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:11:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:34 smithi167 ceph-mon[37140]: pgmap v10954: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:11:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:36 smithi138 ceph-mon[36954]: pgmap v10955: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:11:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:36 smithi100 ceph-mon[32599]: pgmap v10955: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:11:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:36 smithi167 ceph-mon[37140]: pgmap v10955: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:11:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:38 smithi138 ceph-mon[36954]: pgmap v10956: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:11:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:38 smithi100 ceph-mon[32599]: pgmap v10956: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:11:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:38 smithi167 ceph-mon[37140]: pgmap v10956: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:11:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:40 smithi138 ceph-mon[36954]: pgmap v10957: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:11:40.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:40 smithi100 ceph-mon[32599]: pgmap v10957: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:11:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:40 smithi167 ceph-mon[37140]: pgmap v10957: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:11:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:42 smithi138 ceph-mon[36954]: pgmap v10958: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:11:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:42 smithi100 ceph-mon[32599]: pgmap v10958: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:11:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:42 smithi167 ceph-mon[37140]: pgmap v10958: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:11:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:44 smithi138 ceph-mon[36954]: pgmap v10959: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:11:44.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:44 smithi100 ceph-mon[32599]: pgmap v10959: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:11:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:44 smithi167 ceph-mon[37140]: pgmap v10959: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:11:45.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:11:45.399+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:11:45.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:11:45.400+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:11:45.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:11:45.400+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:11:45.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:11:45.400+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:11:45.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:11:45.400+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:11:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:46 smithi138 ceph-mon[36954]: pgmap v10960: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:11:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:46 smithi100 ceph-mon[32599]: pgmap v10960: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:11:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:46 smithi167 ceph-mon[37140]: pgmap v10960: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:11:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:48 smithi138 ceph-mon[36954]: pgmap v10961: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:11:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:48 smithi100 ceph-mon[32599]: pgmap v10961: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:11:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:48 smithi167 ceph-mon[37140]: pgmap v10961: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:11:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:50 smithi138 ceph-mon[36954]: pgmap v10962: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T19:11:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:50 smithi100 ceph-mon[32599]: pgmap v10962: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T19:11:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:50 smithi167 ceph-mon[37140]: pgmap v10962: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T19:11:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:52 smithi138 ceph-mon[36954]: pgmap v10963: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T19:11:52.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:52 smithi100 ceph-mon[32599]: pgmap v10963: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T19:11:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:52 smithi167 ceph-mon[37140]: pgmap v10963: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-07-26T19:11:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:54 smithi138 ceph-mon[36954]: pgmap v10964: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T19:11:54.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:54 smithi100 ceph-mon[32599]: pgmap v10964: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T19:11:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:54 smithi167 ceph-mon[37140]: pgmap v10964: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T19:11:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:56 smithi138 ceph-mon[36954]: pgmap v10965: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T19:11:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:56 smithi100 ceph-mon[32599]: pgmap v10965: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T19:11:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:56 smithi167 ceph-mon[37140]: pgmap v10965: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T19:11:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:11:58 smithi138 ceph-mon[36954]: pgmap v10966: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T19:11:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:11:58 smithi100 ceph-mon[32599]: pgmap v10966: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T19:11:59.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:11:58 smithi167 ceph-mon[37140]: pgmap v10966: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-07-26T19:12:00.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:12:00.415+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:12:00.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:12:00.415+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:12:00.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:12:00.415+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:12:00.417 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:12:00.415+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:12:00.418 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:12:00.415+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:12:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:12:00 smithi138 ceph-mon[36954]: pgmap v10967: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T19:12:00.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:12:00 smithi100 ceph-mon[32599]: pgmap v10967: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T19:12:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:12:00 smithi167 ceph-mon[37140]: pgmap v10967: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-07-26T19:12:02.995 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:12:02 smithi100 ceph-mon[32599]: pgmap v10968: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:12:03.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:12:02 smithi167 ceph-mon[37140]: pgmap v10968: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:12:03.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:12:02 smithi138 ceph-mon[36954]: pgmap v10968: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:12:04.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:12:04 smithi100 ceph-mon[32599]: pgmap v10969: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:12:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:12:04 smithi167 ceph-mon[37140]: pgmap v10969: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:12:05.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:12:04 smithi138 ceph-mon[36954]: pgmap v10969: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:12:05.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:12:05.779+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T19:12:05.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:12:05.779+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c006660 con 0x7fe5a81087a0 2024-07-26T19:12:05.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:12:05.779+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 152 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T19:12:05.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:12:05.779+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 152 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c006660 con 0x7fe5a81087a0 2024-07-26T19:12:06.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:12:06 smithi100 ceph-mon[32599]: pgmap v10970: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:12:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:12:06 smithi167 ceph-mon[37140]: pgmap v10970: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:12:07.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:12:06 smithi138 ceph-mon[36954]: pgmap v10970: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:12:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:12:08 smithi100 ceph-mon[32599]: pgmap v10971: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:12:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:12:08 smithi100 ceph-mon[32599]: osdmap e732: 12 total, 12 up, 12 in 2024-07-26T19:12:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:12:08 smithi167 ceph-mon[37140]: pgmap v10971: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:12:09.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:12:08 smithi167 ceph-mon[37140]: osdmap e732: 12 total, 12 up, 12 in 2024-07-26T19:12:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:12:08 smithi138 ceph-mon[36954]: pgmap v10971: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:12:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:12:08 smithi138 ceph-mon[36954]: osdmap e732: 12 total, 12 up, 12 in 2024-07-26T19:12:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:12:10 smithi100 ceph-mon[32599]: pgmap v10973: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:12:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:12:10 smithi167 ceph-mon[37140]: pgmap v10973: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:12:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:12:10 smithi138 ceph-mon[36954]: pgmap v10973: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:12:12.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:12:12 smithi100 ceph-mon[32599]: pgmap v10974: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:12:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:12:12 smithi167 ceph-mon[37140]: pgmap v10974: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:12:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:12:12 smithi138 ceph-mon[36954]: pgmap v10974: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:12:13.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:12:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:12:13.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:12:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:12:13.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:12:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:12:13.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:12:13 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:12:14.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:12:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:12:14.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:12:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:12:14.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:12:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:12:14.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:12:13 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:12:14.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:12:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:12:14.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:12:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:12:14.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:12:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:12:14.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:12:13 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:12:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:12:14 smithi100 ceph-mon[32599]: pgmap v10975: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:12:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:12:14 smithi100 ceph-mon[32599]: osdmap e733: 12 total, 12 up, 12 in 2024-07-26T19:12:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:12:14 smithi167 ceph-mon[37140]: pgmap v10975: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:12:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:12:14 smithi167 ceph-mon[37140]: osdmap e733: 12 total, 12 up, 12 in 2024-07-26T19:12:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:12:14 smithi138 ceph-mon[36954]: pgmap v10975: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:12:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:12:14 smithi138 ceph-mon[36954]: osdmap e733: 12 total, 12 up, 12 in 2024-07-26T19:12:15.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:12:15.425+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:12:15.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:12:15.426+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:12:15.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:12:15.426+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:12:15.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:12:15.426+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:12:15.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:12:15.426+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:12:16.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:12:16 smithi100 ceph-mon[32599]: pgmap v10977: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T19:12:17.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:12:16 smithi167 ceph-mon[37140]: pgmap v10977: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T19:12:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:12:16 smithi138 ceph-mon[36954]: pgmap v10977: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T19:12:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:12:18 smithi100 ceph-mon[32599]: pgmap v10978: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 218 B/s wr, 6 op/s 2024-07-26T19:12:19.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:12:18 smithi167 ceph-mon[37140]: pgmap v10978: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 218 B/s wr, 6 op/s 2024-07-26T19:12:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:12:18 smithi138 ceph-mon[36954]: pgmap v10978: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 218 B/s wr, 6 op/s 2024-07-26T19:12:20.992 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:12:20 smithi138 ceph-mon[36954]: pgmap v10979: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:12:20.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:12:20 smithi100 ceph-mon[32599]: pgmap v10979: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:12:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:12:20 smithi167 ceph-mon[37140]: pgmap v10979: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:12:21.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:12:21 smithi100 ceph-mon[32599]: pgmap v10980: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:12:22.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:12:21 smithi167 ceph-mon[37140]: pgmap v10980: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:12:22.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:12:21 smithi138 ceph-mon[36954]: pgmap v10980: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:12:24.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:12:24 smithi138 ceph-mon[36954]: pgmap v10981: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:12:24.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:12:24 smithi100 ceph-mon[32599]: pgmap v10981: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:12:24.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:12:24 smithi167 ceph-mon[37140]: pgmap v10981: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:12:26.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:12:26 smithi138 ceph-mon[36954]: pgmap v10982: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:12:26.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:12:26 smithi100 ceph-mon[32599]: pgmap v10982: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:12:26.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:12:26 smithi167 ceph-mon[37140]: pgmap v10982: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:12:28.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:12:28 smithi138 ceph-mon[36954]: pgmap v10983: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:12:28.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:12:28 smithi100 ceph-mon[32599]: pgmap v10983: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:12:28.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:12:28 smithi167 ceph-mon[37140]: pgmap v10983: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:12:30.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:12:30.428+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:12:30.430 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:12:30 smithi100 ceph-mon[32599]: pgmap v10984: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:12:30.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:12:30.429+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:12:30.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:12:30.429+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:12:30.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:12:30.429+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:12:30.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:12:30.429+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:12:30.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:12:30 smithi138 ceph-mon[36954]: pgmap v10984: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:12:30.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:12:30 smithi167 ceph-mon[37140]: pgmap v10984: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:12:32.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:12:32 smithi138 ceph-mon[36954]: pgmap v10985: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:12:32.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:12:32 smithi100 ceph-mon[32599]: pgmap v10985: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:12:32.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:12:32 smithi167 ceph-mon[37140]: pgmap v10985: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:12:34.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:12:34 smithi138 ceph-mon[36954]: pgmap v10986: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:12:34.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:12:34 smithi100 ceph-mon[32599]: pgmap v10986: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:12:34.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:12:34 smithi167 ceph-mon[37140]: pgmap v10986: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:12:36.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:12:36 smithi138 ceph-mon[36954]: pgmap v10987: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:12:36.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:12:36 smithi100 ceph-mon[32599]: pgmap v10987: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:12:36.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:12:36 smithi167 ceph-mon[37140]: pgmap v10987: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:12:38.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:12:38 smithi138 ceph-mon[36954]: pgmap v10988: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:12:38.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:12:38 smithi100 ceph-mon[32599]: pgmap v10988: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:12:38.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:12:38 smithi167 ceph-mon[37140]: pgmap v10988: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:12:40.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:12:40 smithi138 ceph-mon[36954]: pgmap v10989: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:12:40.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:12:40 smithi100 ceph-mon[32599]: pgmap v10989: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:12:40.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:12:40 smithi167 ceph-mon[37140]: pgmap v10989: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:12:42.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:12:42 smithi138 ceph-mon[36954]: pgmap v10990: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:12:42.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:12:42 smithi100 ceph-mon[32599]: pgmap v10990: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:12:42.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:12:42 smithi167 ceph-mon[37140]: pgmap v10990: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:12:44.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:12:44 smithi138 ceph-mon[36954]: pgmap v10991: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:12:44.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:12:44 smithi100 ceph-mon[32599]: pgmap v10991: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:12:44.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:12:44 smithi167 ceph-mon[37140]: pgmap v10991: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:12:45.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:12:45.444+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:12:45.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:12:45.445+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:12:45.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:12:45.445+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:12:45.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:12:45.445+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:12:45.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:12:45.445+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:12:46.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:12:46 smithi138 ceph-mon[36954]: pgmap v10992: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:12:46.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:12:46 smithi100 ceph-mon[32599]: pgmap v10992: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:12:46.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:12:46 smithi167 ceph-mon[37140]: pgmap v10992: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:12:48.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:12:48 smithi138 ceph-mon[36954]: pgmap v10993: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:12:48.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:12:48 smithi100 ceph-mon[32599]: pgmap v10993: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:12:48.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:12:48 smithi167 ceph-mon[37140]: pgmap v10993: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:12:50.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:12:50 smithi138 ceph-mon[36954]: pgmap v10994: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:12:50.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:12:50 smithi100 ceph-mon[32599]: pgmap v10994: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:12:50.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:12:50 smithi167 ceph-mon[37140]: pgmap v10994: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:12:52.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:12:52 smithi138 ceph-mon[36954]: pgmap v10995: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:12:52.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:12:52 smithi100 ceph-mon[32599]: pgmap v10995: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:12:52.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:12:52 smithi167 ceph-mon[37140]: pgmap v10995: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:12:54.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:12:54 smithi138 ceph-mon[36954]: pgmap v10996: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:12:54.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:12:54 smithi100 ceph-mon[32599]: pgmap v10996: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:12:54.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:12:54 smithi167 ceph-mon[37140]: pgmap v10996: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:12:56.440 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:12:56 smithi138 ceph-mon[36954]: pgmap v10997: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:12:56.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:12:56 smithi100 ceph-mon[32599]: pgmap v10997: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:12:56.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:12:56 smithi167 ceph-mon[37140]: pgmap v10997: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:12:58.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:12:58 smithi100 ceph-mon[32599]: pgmap v10998: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:12:58.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:12:58 smithi167 ceph-mon[37140]: pgmap v10998: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:12:58.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:12:58 smithi138 ceph-mon[36954]: pgmap v10998: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:13:00.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:13:00.455+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:13:00.457 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:13:00 smithi100 ceph-mon[32599]: pgmap v10999: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:13:00.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:13:00.456+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:13:00.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:13:00.456+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:13:00.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:13:00.456+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:13:00.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:13:00.456+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:13:00.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:13:00 smithi167 ceph-mon[37140]: pgmap v10999: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:13:00.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:13:00 smithi138 ceph-mon[36954]: pgmap v10999: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:13:02.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:13:02 smithi100 ceph-mon[32599]: pgmap v11000: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:13:02.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:13:02 smithi167 ceph-mon[37140]: pgmap v11000: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:13:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:13:02 smithi138 ceph-mon[36954]: pgmap v11000: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:13:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:13:04 smithi100 ceph-mon[32599]: pgmap v11001: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:13:04.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:13:04 smithi167 ceph-mon[37140]: pgmap v11001: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:13:04.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:13:04 smithi138 ceph-mon[36954]: pgmap v11001: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:13:06.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:13:06 smithi100 ceph-mon[32599]: pgmap v11002: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:13:06.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:13:06 smithi167 ceph-mon[37140]: pgmap v11002: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:13:06.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:13:06 smithi138 ceph-mon[36954]: pgmap v11002: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:13:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:13:08 smithi100 ceph-mon[32599]: pgmap v11003: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:13:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:13:08 smithi167 ceph-mon[37140]: pgmap v11003: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:13:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:13:08 smithi138 ceph-mon[36954]: pgmap v11003: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:13:09.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:13:09 smithi100 ceph-mon[32599]: osdmap e734: 12 total, 12 up, 12 in 2024-07-26T19:13:09.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:13:09 smithi167 ceph-mon[37140]: osdmap e734: 12 total, 12 up, 12 in 2024-07-26T19:13:09.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:13:09 smithi138 ceph-mon[36954]: osdmap e734: 12 total, 12 up, 12 in 2024-07-26T19:13:10.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:13:10 smithi100 ceph-mon[32599]: pgmap v11005: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:13:10.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:13:10 smithi167 ceph-mon[37140]: pgmap v11005: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:13:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:13:10 smithi138 ceph-mon[36954]: pgmap v11005: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:13:12.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:13:12 smithi100 ceph-mon[32599]: pgmap v11006: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:13:12.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:13:12 smithi167 ceph-mon[37140]: pgmap v11006: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:13:12.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:13:12 smithi138 ceph-mon[36954]: pgmap v11006: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:13:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:13:14 smithi100 ceph-mon[32599]: pgmap v11007: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:13:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:13:14 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:13:14.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:13:14 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:13:14.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:13:14 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:13:14.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:13:14 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:13:14.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:13:14 smithi100 ceph-mon[32599]: osdmap e735: 12 total, 12 up, 12 in 2024-07-26T19:13:14.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:13:14 smithi167 ceph-mon[37140]: pgmap v11007: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:13:14.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:13:14 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:13:14.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:13:14 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:13:14.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:13:14 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:13:14.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:13:14 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:13:14.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:13:14 smithi167 ceph-mon[37140]: osdmap e735: 12 total, 12 up, 12 in 2024-07-26T19:13:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:13:14 smithi138 ceph-mon[36954]: pgmap v11007: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:13:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:13:14 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:13:14.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:13:14 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:13:14.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:13:14 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:13:14.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:13:14 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:13:14.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:13:14 smithi138 ceph-mon[36954]: osdmap e735: 12 total, 12 up, 12 in 2024-07-26T19:13:15.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:13:15.466+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:13:15.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:13:15.467+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:13:15.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:13:15.467+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:13:15.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:13:15.467+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:13:15.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:13:15.467+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:13:16.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:13:16 smithi100 ceph-mon[32599]: pgmap v11009: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T19:13:16.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:13:16 smithi167 ceph-mon[37140]: pgmap v11009: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T19:13:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:13:16 smithi138 ceph-mon[36954]: pgmap v11009: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T19:13:18.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:13:18 smithi100 ceph-mon[32599]: pgmap v11010: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T19:13:18.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:13:18 smithi167 ceph-mon[37140]: pgmap v11010: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T19:13:18.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:13:18 smithi138 ceph-mon[36954]: pgmap v11010: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 230 B/s wr, 6 op/s 2024-07-26T19:13:20.492 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:13:20 smithi138 ceph-mon[36954]: pgmap v11011: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:13:20.493 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:13:20 smithi100 ceph-mon[32599]: pgmap v11011: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:13:20.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:13:20 smithi167 ceph-mon[37140]: pgmap v11011: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:13:22.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:13:22 smithi100 ceph-mon[32599]: pgmap v11012: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:13:22.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:13:22 smithi167 ceph-mon[37140]: pgmap v11012: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:13:22.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:13:22 smithi138 ceph-mon[36954]: pgmap v11012: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:13:24.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:13:24 smithi167 ceph-mon[37140]: pgmap v11013: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:13:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:13:24 smithi138 ceph-mon[36954]: pgmap v11013: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:13:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:13:24 smithi100 ceph-mon[32599]: pgmap v11013: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:13:26.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:13:26 smithi167 ceph-mon[37140]: pgmap v11014: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:13:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:13:26 smithi138 ceph-mon[36954]: pgmap v11014: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:13:26.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:13:26 smithi100 ceph-mon[32599]: pgmap v11014: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:13:28.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:13:28 smithi167 ceph-mon[37140]: pgmap v11015: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:13:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:13:28 smithi138 ceph-mon[36954]: pgmap v11015: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:13:28.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:13:28 smithi100 ceph-mon[32599]: pgmap v11015: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:13:30.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:13:30.483+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:13:30.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:13:30.484+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:13:30.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:13:30.484+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:13:30.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:13:30.484+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:13:30.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:13:30.484+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:13:30.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:13:30 smithi167 ceph-mon[37140]: pgmap v11016: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:13:30.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:13:30 smithi100 ceph-mon[32599]: pgmap v11016: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:13:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:13:30 smithi138 ceph-mon[36954]: pgmap v11016: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:13:32.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:13:32 smithi167 ceph-mon[37140]: pgmap v11017: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:13:32.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:13:32 smithi138 ceph-mon[36954]: pgmap v11017: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:13:32.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:13:32 smithi100 ceph-mon[32599]: pgmap v11017: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:13:34.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:13:34 smithi167 ceph-mon[37140]: pgmap v11018: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:13:34.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:13:34 smithi138 ceph-mon[36954]: pgmap v11018: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:13:34.746 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:13:34 smithi100 ceph-mon[32599]: pgmap v11018: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:13:36.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:13:36 smithi167 ceph-mon[37140]: pgmap v11019: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:13:36.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:13:36 smithi138 ceph-mon[36954]: pgmap v11019: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:13:36.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:13:36 smithi100 ceph-mon[32599]: pgmap v11019: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:13:38.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:13:38 smithi167 ceph-mon[37140]: pgmap v11020: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:13:38.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:13:38 smithi138 ceph-mon[36954]: pgmap v11020: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:13:38.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:13:38 smithi100 ceph-mon[32599]: pgmap v11020: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:13:40.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:13:40 smithi167 ceph-mon[37140]: pgmap v11021: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:13:40.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:13:40 smithi138 ceph-mon[36954]: pgmap v11021: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:13:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:13:40 smithi100 ceph-mon[32599]: pgmap v11021: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:13:42.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:13:42 smithi138 ceph-mon[36954]: pgmap v11022: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:13:42.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:13:42 smithi100 ceph-mon[32599]: pgmap v11022: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:13:42.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:13:42 smithi167 ceph-mon[37140]: pgmap v11022: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:13:44.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:13:44 smithi167 ceph-mon[37140]: pgmap v11023: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:13:44.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:13:44 smithi138 ceph-mon[36954]: pgmap v11023: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:13:44.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:13:44 smithi100 ceph-mon[32599]: pgmap v11023: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:13:45.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:13:45.488+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:13:45.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:13:45.488+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:13:45.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:13:45.488+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:13:45.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:13:45.488+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:13:45.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:13:45.488+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:13:46.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:13:46 smithi138 ceph-mon[36954]: pgmap v11024: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:13:46.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:13:46 smithi100 ceph-mon[32599]: pgmap v11024: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:13:46.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:13:46 smithi167 ceph-mon[37140]: pgmap v11024: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:13:48.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:13:48 smithi138 ceph-mon[36954]: pgmap v11025: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:13:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:13:48 smithi100 ceph-mon[32599]: pgmap v11025: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:13:48.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:13:48 smithi167 ceph-mon[37140]: pgmap v11025: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:13:50.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:13:50 smithi138 ceph-mon[36954]: pgmap v11026: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:13:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:13:50 smithi100 ceph-mon[32599]: pgmap v11026: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:13:50.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:13:50 smithi167 ceph-mon[37140]: pgmap v11026: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:13:52.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:13:52 smithi138 ceph-mon[36954]: pgmap v11027: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:13:52.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:13:52 smithi100 ceph-mon[32599]: pgmap v11027: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:13:52.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:13:52 smithi167 ceph-mon[37140]: pgmap v11027: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:13:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:13:54 smithi138 ceph-mon[36954]: pgmap v11028: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:13:54.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:13:54 smithi100 ceph-mon[32599]: pgmap v11028: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:13:54.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:13:54 smithi167 ceph-mon[37140]: pgmap v11028: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:13:56.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:13:56 smithi138 ceph-mon[36954]: pgmap v11029: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:13:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:13:56 smithi100 ceph-mon[32599]: pgmap v11029: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:13:56.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:13:56 smithi167 ceph-mon[37140]: pgmap v11029: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:13:58.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:13:58 smithi138 ceph-mon[36954]: pgmap v11030: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:13:58.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:13:58 smithi100 ceph-mon[32599]: pgmap v11030: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:13:58.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:13:58 smithi167 ceph-mon[37140]: pgmap v11030: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:14:00.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:14:00.497+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:14:00.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:14:00.498+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:14:00.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:14:00.498+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:14:00.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:14:00.498+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:14:00.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:14:00.498+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:14:00.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:00 smithi138 ceph-mon[36954]: pgmap v11031: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:14:00.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:00 smithi100 ceph-mon[32599]: pgmap v11031: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:14:00.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:00 smithi167 ceph-mon[37140]: pgmap v11031: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:14:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:02 smithi138 ceph-mon[36954]: pgmap v11032: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:14:02.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:02 smithi100 ceph-mon[32599]: pgmap v11032: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:14:02.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:02 smithi167 ceph-mon[37140]: pgmap v11032: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:14:04.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:04 smithi138 ceph-mon[36954]: pgmap v11033: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:14:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:04 smithi100 ceph-mon[32599]: pgmap v11033: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:14:04.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:04 smithi167 ceph-mon[37140]: pgmap v11033: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:14:06.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:06 smithi138 ceph-mon[36954]: pgmap v11034: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:14:06.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:06 smithi100 ceph-mon[32599]: pgmap v11034: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:14:06.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:06 smithi167 ceph-mon[37140]: pgmap v11034: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:14:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:08 smithi138 ceph-mon[36954]: pgmap v11035: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:14:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:08 smithi138 ceph-mon[36954]: osdmap e736: 12 total, 12 up, 12 in 2024-07-26T19:14:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:08 smithi100 ceph-mon[32599]: pgmap v11035: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:14:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:08 smithi100 ceph-mon[32599]: osdmap e736: 12 total, 12 up, 12 in 2024-07-26T19:14:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:08 smithi167 ceph-mon[37140]: pgmap v11035: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:14:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:08 smithi167 ceph-mon[37140]: osdmap e736: 12 total, 12 up, 12 in 2024-07-26T19:14:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:10 smithi138 ceph-mon[36954]: pgmap v11037: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:14:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:10 smithi100 ceph-mon[32599]: pgmap v11037: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:14:10.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:10 smithi167 ceph-mon[37140]: pgmap v11037: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:14:12.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:12 smithi138 ceph-mon[36954]: pgmap v11038: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:14:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:12 smithi100 ceph-mon[32599]: pgmap v11038: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:14:12.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:12 smithi167 ceph-mon[37140]: pgmap v11038: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:14:14.599 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:14 smithi138 ceph-mon[36954]: pgmap v11039: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:14:14.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:14 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:14:14.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:14 smithi138 ceph-mon[36954]: osdmap e737: 12 total, 12 up, 12 in 2024-07-26T19:14:14.600 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:14 smithi100 ceph-mon[32599]: pgmap v11039: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:14:14.600 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:14 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:14:14.601 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:14 smithi100 ceph-mon[32599]: osdmap e737: 12 total, 12 up, 12 in 2024-07-26T19:14:14.662 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:14 smithi167 ceph-mon[37140]: pgmap v11039: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:14:14.662 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:14 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:14:14.662 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:14 smithi167 ceph-mon[37140]: osdmap e737: 12 total, 12 up, 12 in 2024-07-26T19:14:15.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:14:15.503+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:14:15.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:14:15.504+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:14:15.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:14:15.504+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:14:15.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:14:15.504+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:14:15.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:14:15.504+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:14:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:16 smithi138 ceph-mon[36954]: pgmap v11041: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T19:14:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:14:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:14:16.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:14:16.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:14:16.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:14:16.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:14:16.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:14:16.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:14:16.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:14:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:16 smithi100 ceph-mon[32599]: pgmap v11041: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T19:14:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:14:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:14:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:14:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:14:16.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:14:16.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:14:16.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:14:16.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:14:16.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:14:16.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:16 smithi167 ceph-mon[37140]: pgmap v11041: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T19:14:16.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:14:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:14:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:14:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:14:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:14:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:14:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:14:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:14:16.774 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:14:18.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:18 smithi138 ceph-mon[36954]: pgmap v11042: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 209 B/s wr, 6 op/s 2024-07-26T19:14:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:18 smithi100 ceph-mon[32599]: pgmap v11042: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 209 B/s wr, 6 op/s 2024-07-26T19:14:18.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:18 smithi167 ceph-mon[37140]: pgmap v11042: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 209 B/s wr, 6 op/s 2024-07-26T19:14:20.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:20 smithi138 ceph-mon[36954]: pgmap v11043: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:14:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:20 smithi100 ceph-mon[32599]: pgmap v11043: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:14:20.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:20 smithi167 ceph-mon[37140]: pgmap v11043: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:14:22.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:22 smithi138 ceph-mon[36954]: pgmap v11044: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:14:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:22 smithi100 ceph-mon[32599]: pgmap v11044: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:14:22.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:22 smithi167 ceph-mon[37140]: pgmap v11044: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:14:24.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:24 smithi138 ceph-mon[36954]: pgmap v11045: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:14:24.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:24 smithi100 ceph-mon[32599]: pgmap v11045: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:14:24.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:24 smithi167 ceph-mon[37140]: pgmap v11045: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:14:26.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:26 smithi138 ceph-mon[36954]: pgmap v11046: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:14:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:26 smithi100 ceph-mon[32599]: pgmap v11046: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:14:26.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:26 smithi167 ceph-mon[37140]: pgmap v11046: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:14:28.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:28 smithi138 ceph-mon[36954]: pgmap v11047: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:14:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:28 smithi100 ceph-mon[32599]: pgmap v11047: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:14:28.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:28 smithi167 ceph-mon[37140]: pgmap v11047: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:14:30.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:14:30.519+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:14:30.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:14:30.520+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:14:30.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:14:30.520+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:14:30.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:14:30.520+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:14:30.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:14:30.520+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:14:30.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:30 smithi138 ceph-mon[36954]: pgmap v11048: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:14:30.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:30 smithi100 ceph-mon[32599]: pgmap v11048: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:14:30.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:30 smithi167 ceph-mon[37140]: pgmap v11048: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:14:32.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:32 smithi138 ceph-mon[36954]: pgmap v11049: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:14:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:32 smithi100 ceph-mon[32599]: pgmap v11049: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:14:32.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:32 smithi167 ceph-mon[37140]: pgmap v11049: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:14:34.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:34 smithi138 ceph-mon[36954]: pgmap v11050: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:14:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:34 smithi100 ceph-mon[32599]: pgmap v11050: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:14:34.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:34 smithi167 ceph-mon[37140]: pgmap v11050: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:14:36.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:36 smithi138 ceph-mon[36954]: pgmap v11051: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:14:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:36 smithi100 ceph-mon[32599]: pgmap v11051: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:14:36.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:36 smithi167 ceph-mon[37140]: pgmap v11051: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:14:38.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:38 smithi138 ceph-mon[36954]: pgmap v11052: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:14:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:38 smithi100 ceph-mon[32599]: pgmap v11052: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:14:38.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:38 smithi167 ceph-mon[37140]: pgmap v11052: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:14:40.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:40 smithi138 ceph-mon[36954]: pgmap v11053: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:14:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:40 smithi100 ceph-mon[32599]: pgmap v11053: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:14:40.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:40 smithi167 ceph-mon[37140]: pgmap v11053: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:14:42.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:42 smithi138 ceph-mon[36954]: pgmap v11054: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:14:42.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:42 smithi100 ceph-mon[32599]: pgmap v11054: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:14:42.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:42 smithi167 ceph-mon[37140]: pgmap v11054: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:14:44.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:44 smithi138 ceph-mon[36954]: pgmap v11055: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:14:44.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:44 smithi100 ceph-mon[32599]: pgmap v11055: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:14:44.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:44 smithi167 ceph-mon[37140]: pgmap v11055: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:14:45.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:14:45.535+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:14:45.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:14:45.536+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:14:45.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:14:45.536+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:14:45.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:14:45.536+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:14:45.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:14:45.536+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:14:46.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:46 smithi138 ceph-mon[36954]: pgmap v11056: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:14:46.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:46 smithi100 ceph-mon[32599]: pgmap v11056: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:14:46.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:46 smithi167 ceph-mon[37140]: pgmap v11056: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:14:48.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:48 smithi138 ceph-mon[36954]: pgmap v11057: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:14:48.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:48 smithi100 ceph-mon[32599]: pgmap v11057: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:14:48.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:48 smithi167 ceph-mon[37140]: pgmap v11057: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:14:50.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:50 smithi138 ceph-mon[36954]: pgmap v11058: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:14:50.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:50 smithi100 ceph-mon[32599]: pgmap v11058: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:14:50.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:50 smithi167 ceph-mon[37140]: pgmap v11058: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:14:52.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:52 smithi138 ceph-mon[36954]: pgmap v11059: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:14:52.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:52 smithi100 ceph-mon[32599]: pgmap v11059: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:14:52.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:52 smithi167 ceph-mon[37140]: pgmap v11059: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:14:54.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:54 smithi138 ceph-mon[36954]: pgmap v11060: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:14:54.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:54 smithi100 ceph-mon[32599]: pgmap v11060: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:14:54.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:54 smithi167 ceph-mon[37140]: pgmap v11060: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:14:56.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:56 smithi138 ceph-mon[36954]: pgmap v11061: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:14:56.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:56 smithi100 ceph-mon[32599]: pgmap v11061: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:14:56.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:56 smithi167 ceph-mon[37140]: pgmap v11061: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:14:58.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:14:58 smithi138 ceph-mon[36954]: pgmap v11062: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:14:58.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:14:58 smithi100 ceph-mon[32599]: pgmap v11062: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:14:58.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:14:58 smithi167 ceph-mon[37140]: pgmap v11062: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:15:00.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:15:00.551+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:15:00.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:15:00.551+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:15:00.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:15:00.551+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:15:00.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:15:00.551+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:15:00.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:15:00.551+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:15:00.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:00 smithi138 ceph-mon[36954]: pgmap v11063: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:15:00.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:00 smithi100 ceph-mon[32599]: pgmap v11063: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:15:00.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:00 smithi167 ceph-mon[37140]: pgmap v11063: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:15:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:02 smithi138 ceph-mon[36954]: pgmap v11064: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:15:02.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:02 smithi100 ceph-mon[32599]: pgmap v11064: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:15:02.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:02 smithi167 ceph-mon[37140]: pgmap v11064: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:15:04.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:04 smithi138 ceph-mon[36954]: pgmap v11065: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:15:04.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:04 smithi100 ceph-mon[32599]: pgmap v11065: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:15:04.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:04 smithi167 ceph-mon[37140]: pgmap v11065: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:15:05.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:15:05.805+0000 7fe58cff9640 1 -- 172.21.15.100:0/972906438 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe580006490 con 0x55c0ec887390 2024-07-26T19:15:05.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:15:05.805+0000 7fe59e7fc640 1 -- 172.21.15.100:0/2403309286 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe57c006660 con 0x7fe5a81087a0 2024-07-26T19:15:05.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:15:05.805+0000 7fe58dffb640 1 -- 172.21.15.100:0/972906438 <== mon.0 v2:172.21.15.100:3300/0 153 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7fe580006490 con 0x55c0ec887390 2024-07-26T19:15:05.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:15:05.805+0000 7fe59f7fe640 1 -- 172.21.15.100:0/2403309286 <== mon.0 v2:172.21.15.100:3300/0 153 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe57c006660 con 0x7fe5a81087a0 2024-07-26T19:15:06.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:06 smithi138 ceph-mon[36954]: pgmap v11066: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:15:06.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:06 smithi100 ceph-mon[32599]: pgmap v11066: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:15:06.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:06 smithi167 ceph-mon[37140]: pgmap v11066: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:15:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:08 smithi138 ceph-mon[36954]: pgmap v11067: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:15:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:08 smithi138 ceph-mon[36954]: osdmap e738: 12 total, 12 up, 12 in 2024-07-26T19:15:08.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:08 smithi100 ceph-mon[32599]: pgmap v11067: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:15:08.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:08 smithi100 ceph-mon[32599]: osdmap e738: 12 total, 12 up, 12 in 2024-07-26T19:15:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:08 smithi167 ceph-mon[37140]: pgmap v11067: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:15:08.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:08 smithi167 ceph-mon[37140]: osdmap e738: 12 total, 12 up, 12 in 2024-07-26T19:15:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:10 smithi138 ceph-mon[36954]: pgmap v11069: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:15:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:10 smithi100 ceph-mon[32599]: pgmap v11069: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:15:10.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:10 smithi167 ceph-mon[37140]: pgmap v11069: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:15:12.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:12 smithi138 ceph-mon[36954]: pgmap v11070: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:15:12.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:12 smithi100 ceph-mon[32599]: pgmap v11070: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:15:12.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:12 smithi167 ceph-mon[37140]: pgmap v11070: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:15:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:14 smithi138 ceph-mon[36954]: pgmap v11071: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:15:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:14 smithi138 ceph-mon[36954]: osdmap e739: 12 total, 12 up, 12 in 2024-07-26T19:15:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:14 smithi100 ceph-mon[32599]: pgmap v11071: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:15:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:14 smithi100 ceph-mon[32599]: osdmap e739: 12 total, 12 up, 12 in 2024-07-26T19:15:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:14 smithi167 ceph-mon[37140]: pgmap v11071: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:15:14.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:14 smithi167 ceph-mon[37140]: osdmap e739: 12 total, 12 up, 12 in 2024-07-26T19:15:15.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:15:15.560+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:15:15.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:15:15.561+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:15:15.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:15:15.561+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:15:15.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:15:15.561+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:15:15.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:15:15.561+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:15:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:16 smithi138 ceph-mon[36954]: pgmap v11073: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T19:15:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:15:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:15:16.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:15:16.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:15:16.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:15:16.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:15:16.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:16 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:15:16.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:16 smithi100 ceph-mon[32599]: pgmap v11073: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T19:15:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:15:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:15:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:15:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:15:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:15:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:15:16.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:16 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:15:16.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:16 smithi167 ceph-mon[37140]: pgmap v11073: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T19:15:16.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:15:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:15:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:15:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:15:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:15:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:15:16.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:16 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:15:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:15:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:15:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:15:17.743 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:15:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:15:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:15:17.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:15:17.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:15:17.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:15:18.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:18 smithi138 ceph-mon[36954]: pgmap v11074: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 210 B/s wr, 6 op/s 2024-07-26T19:15:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:18 smithi100 ceph-mon[32599]: pgmap v11074: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 210 B/s wr, 6 op/s 2024-07-26T19:15:19.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:18 smithi167 ceph-mon[37140]: pgmap v11074: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 210 B/s wr, 6 op/s 2024-07-26T19:15:20.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:20 smithi138 ceph-mon[36954]: pgmap v11075: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:15:20.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:20 smithi100 ceph-mon[32599]: pgmap v11075: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:15:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:20 smithi167 ceph-mon[37140]: pgmap v11075: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:15:22.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:22 smithi138 ceph-mon[36954]: pgmap v11076: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:15:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:22 smithi100 ceph-mon[32599]: pgmap v11076: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:15:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:22 smithi167 ceph-mon[37140]: pgmap v11076: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:15:24.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:24 smithi138 ceph-mon[36954]: pgmap v11077: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:15:24.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:24 smithi100 ceph-mon[32599]: pgmap v11077: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:15:25.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:24 smithi167 ceph-mon[37140]: pgmap v11077: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:15:26.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:26 smithi138 ceph-mon[36954]: pgmap v11078: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:15:26.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:26 smithi100 ceph-mon[32599]: pgmap v11078: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:15:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:26 smithi167 ceph-mon[37140]: pgmap v11078: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:15:28.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:28 smithi138 ceph-mon[36954]: pgmap v11079: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:15:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:28 smithi100 ceph-mon[32599]: pgmap v11079: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:15:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:28 smithi167 ceph-mon[37140]: pgmap v11079: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:15:30.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:15:30.577+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:15:30.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:15:30.577+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:15:30.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:15:30.577+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:15:30.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:15:30.577+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:15:30.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:15:30.577+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:15:30.942 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:30 smithi138 ceph-mon[36954]: pgmap v11080: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:15:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:30 smithi100 ceph-mon[32599]: pgmap v11080: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:15:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:30 smithi167 ceph-mon[37140]: pgmap v11080: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:15:32.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:32 smithi138 ceph-mon[36954]: pgmap v11081: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:15:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:32 smithi100 ceph-mon[32599]: pgmap v11081: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:15:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:32 smithi167 ceph-mon[37140]: pgmap v11081: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:15:34.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:34 smithi138 ceph-mon[36954]: pgmap v11082: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:15:34.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:34 smithi100 ceph-mon[32599]: pgmap v11082: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:15:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:34 smithi167 ceph-mon[37140]: pgmap v11082: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:15:36.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:36 smithi138 ceph-mon[36954]: pgmap v11083: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:15:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:36 smithi100 ceph-mon[32599]: pgmap v11083: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:15:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:36 smithi167 ceph-mon[37140]: pgmap v11083: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:15:38.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:38 smithi138 ceph-mon[36954]: pgmap v11084: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:15:38.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:38 smithi100 ceph-mon[32599]: pgmap v11084: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:15:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:38 smithi167 ceph-mon[37140]: pgmap v11084: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:15:40.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:40 smithi138 ceph-mon[36954]: pgmap v11085: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:15:40.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:40 smithi100 ceph-mon[32599]: pgmap v11085: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:15:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:40 smithi167 ceph-mon[37140]: pgmap v11085: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:15:42.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:42 smithi138 ceph-mon[36954]: pgmap v11086: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:15:42.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:42 smithi100 ceph-mon[32599]: pgmap v11086: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:15:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:42 smithi167 ceph-mon[37140]: pgmap v11086: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:15:44.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:44 smithi138 ceph-mon[36954]: pgmap v11087: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:15:44.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:44 smithi100 ceph-mon[32599]: pgmap v11087: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:15:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:44 smithi167 ceph-mon[37140]: pgmap v11087: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:15:45.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:15:45.593+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:15:45.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:15:45.594+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:15:45.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:15:45.594+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:15:45.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:15:45.594+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:15:45.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:15:45.594+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:15:46.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:46 smithi138 ceph-mon[36954]: pgmap v11088: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:15:46.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:46 smithi100 ceph-mon[32599]: pgmap v11088: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:15:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:46 smithi167 ceph-mon[37140]: pgmap v11088: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:15:48.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:48 smithi138 ceph-mon[36954]: pgmap v11089: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:15:48.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:48 smithi100 ceph-mon[32599]: pgmap v11089: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:15:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:48 smithi167 ceph-mon[37140]: pgmap v11089: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:15:50.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:50 smithi138 ceph-mon[36954]: pgmap v11090: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:15:50.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:50 smithi100 ceph-mon[32599]: pgmap v11090: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:15:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:50 smithi167 ceph-mon[37140]: pgmap v11090: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:15:52.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:52 smithi138 ceph-mon[36954]: pgmap v11091: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:15:52.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:52 smithi100 ceph-mon[32599]: pgmap v11091: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:15:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:52 smithi167 ceph-mon[37140]: pgmap v11091: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:15:54.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:54 smithi138 ceph-mon[36954]: pgmap v11092: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:15:54.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:54 smithi100 ceph-mon[32599]: pgmap v11092: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:15:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:54 smithi167 ceph-mon[37140]: pgmap v11092: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:15:56.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:56 smithi138 ceph-mon[36954]: pgmap v11093: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:15:56.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:56 smithi100 ceph-mon[32599]: pgmap v11093: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:15:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:56 smithi167 ceph-mon[37140]: pgmap v11093: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:15:58.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:15:58 smithi138 ceph-mon[36954]: pgmap v11094: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:15:58.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:15:58 smithi100 ceph-mon[32599]: pgmap v11094: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:15:59.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:15:58 smithi167 ceph-mon[37140]: pgmap v11094: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:16:00.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:16:00.610+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:16:00.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:16:00.612+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:16:00.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:16:00.612+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:16:00.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:16:00.612+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:16:00.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:16:00.612+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:16:00.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:16:00 smithi138 ceph-mon[36954]: pgmap v11095: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:16:00.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:16:00 smithi100 ceph-mon[32599]: pgmap v11095: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:16:01.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:16:00 smithi167 ceph-mon[37140]: pgmap v11095: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:16:02.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:16:02 smithi138 ceph-mon[36954]: pgmap v11096: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:16:02.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:16:02 smithi100 ceph-mon[32599]: pgmap v11096: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:16:03.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:16:02 smithi167 ceph-mon[37140]: pgmap v11096: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:16:04.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:16:04 smithi138 ceph-mon[36954]: pgmap v11097: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:16:04.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:16:04 smithi100 ceph-mon[32599]: pgmap v11097: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:16:05.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:16:04 smithi167 ceph-mon[37140]: pgmap v11097: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:16:06.940 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:16:06 smithi138 ceph-mon[36954]: pgmap v11098: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:16:06.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:16:06 smithi100 ceph-mon[32599]: pgmap v11098: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:16:07.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:16:06 smithi167 ceph-mon[37140]: pgmap v11098: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:16:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:16:08 smithi100 ceph-mon[32599]: pgmap v11099: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:16:08.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:16:08 smithi100 ceph-mon[32599]: osdmap e740: 12 total, 12 up, 12 in 2024-07-26T19:16:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:16:08 smithi167 ceph-mon[37140]: pgmap v11099: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:16:09.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:16:08 smithi167 ceph-mon[37140]: osdmap e740: 12 total, 12 up, 12 in 2024-07-26T19:16:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:16:08 smithi138 ceph-mon[36954]: pgmap v11099: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:16:09.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:16:08 smithi138 ceph-mon[36954]: osdmap e740: 12 total, 12 up, 12 in 2024-07-26T19:16:10.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:16:10 smithi100 ceph-mon[32599]: pgmap v11101: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:16:11.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:16:10 smithi167 ceph-mon[37140]: pgmap v11101: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:16:11.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:16:10 smithi138 ceph-mon[36954]: pgmap v11101: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:16:12.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:16:12 smithi100 ceph-mon[32599]: pgmap v11102: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:16:13.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:16:12 smithi167 ceph-mon[37140]: pgmap v11102: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:16:13.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:16:12 smithi138 ceph-mon[36954]: pgmap v11102: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:16:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:16:14 smithi100 ceph-mon[32599]: pgmap v11103: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:16:14.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:16:14 smithi100 ceph-mon[32599]: osdmap e741: 12 total, 12 up, 12 in 2024-07-26T19:16:15.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:16:14 smithi167 ceph-mon[37140]: pgmap v11103: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:16:15.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:16:14 smithi167 ceph-mon[37140]: osdmap e741: 12 total, 12 up, 12 in 2024-07-26T19:16:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:16:14 smithi138 ceph-mon[36954]: pgmap v11103: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:16:15.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:16:14 smithi138 ceph-mon[36954]: osdmap e741: 12 total, 12 up, 12 in 2024-07-26T19:16:15.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:16:15.620+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:16:15.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:16:15.621+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:16:15.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:16:15.621+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:16:15.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:16:15.621+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:16:15.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:16:15.621+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:16:16.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:16:16 smithi100 ceph-mon[32599]: pgmap v11105: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T19:16:17.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:16:16 smithi167 ceph-mon[37140]: pgmap v11105: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T19:16:17.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:16:16 smithi138 ceph-mon[36954]: pgmap v11105: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T19:16:17.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:16:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:16:17.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:16:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:16:17.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:16:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:16:17.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:16:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:16:18.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:16:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:16:18.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:16:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:16:18.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:16:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:16:18.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:16:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:16:18.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:16:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:16:18.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:16:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:16:18.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:16:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:16:18.191 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:16:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:16:18.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:16:18 smithi100 ceph-mon[32599]: pgmap v11106: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 216 B/s wr, 6 op/s 2024-07-26T19:16:19.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:16:18 smithi167 ceph-mon[37140]: pgmap v11106: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 216 B/s wr, 6 op/s 2024-07-26T19:16:19.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:16:18 smithi138 ceph-mon[36954]: pgmap v11106: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 216 B/s wr, 6 op/s 2024-07-26T19:16:20.992 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:16:20 smithi138 ceph-mon[36954]: pgmap v11107: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:16:20.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:16:20 smithi100 ceph-mon[32599]: pgmap v11107: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:16:21.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:16:20 smithi167 ceph-mon[37140]: pgmap v11107: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:16:22.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:16:22 smithi100 ceph-mon[32599]: pgmap v11108: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:16:23.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:16:22 smithi167 ceph-mon[37140]: pgmap v11108: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:16:23.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:16:22 smithi138 ceph-mon[36954]: pgmap v11108: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:16:24.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:16:24 smithi100 ceph-mon[32599]: pgmap v11109: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:16:25.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:16:24 smithi167 ceph-mon[37140]: pgmap v11109: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:16:25.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:16:24 smithi138 ceph-mon[36954]: pgmap v11109: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:16:26.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:16:26 smithi100 ceph-mon[32599]: pgmap v11110: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:16:27.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:16:26 smithi167 ceph-mon[37140]: pgmap v11110: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:16:27.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:16:26 smithi138 ceph-mon[36954]: pgmap v11110: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:16:28.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:16:28 smithi100 ceph-mon[32599]: pgmap v11111: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:16:29.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:16:28 smithi167 ceph-mon[37140]: pgmap v11111: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:16:29.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:16:28 smithi138 ceph-mon[36954]: pgmap v11111: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:16:30.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:16:30.636+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:16:30.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:16:30.636+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:16:30.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:16:30.636+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:16:30.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:16:30.636+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:16:30.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:16:30.636+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:16:30.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:16:30 smithi100 ceph-mon[32599]: pgmap v11112: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:16:31.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:16:30 smithi167 ceph-mon[37140]: pgmap v11112: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:16:31.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:16:30 smithi138 ceph-mon[36954]: pgmap v11112: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:16:32.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:16:32 smithi100 ceph-mon[32599]: pgmap v11113: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:16:33.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:16:32 smithi167 ceph-mon[37140]: pgmap v11113: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:16:33.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:16:32 smithi138 ceph-mon[36954]: pgmap v11113: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:16:34.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:16:34 smithi100 ceph-mon[32599]: pgmap v11114: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:16:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:16:34 smithi167 ceph-mon[37140]: pgmap v11114: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:16:35.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:16:34 smithi138 ceph-mon[36954]: pgmap v11114: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:16:36.994 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:16:36 smithi100 ceph-mon[32599]: pgmap v11115: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:16:37.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:16:36 smithi167 ceph-mon[37140]: pgmap v11115: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:16:37.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:16:36 smithi138 ceph-mon[36954]: pgmap v11115: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:16:38.993 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:16:38 smithi100 ceph-mon[32599]: pgmap v11116: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:16:39.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:16:38 smithi167 ceph-mon[37140]: pgmap v11116: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:16:39.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:16:38 smithi138 ceph-mon[36954]: pgmap v11116: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:16:41.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:16:40 smithi167 ceph-mon[37140]: pgmap v11117: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:16:41.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:16:40 smithi138 ceph-mon[36954]: pgmap v11117: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:16:41.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:16:40 smithi100 ceph-mon[32599]: pgmap v11117: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:16:43.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:16:42 smithi167 ceph-mon[37140]: pgmap v11118: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:16:43.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:16:42 smithi138 ceph-mon[36954]: pgmap v11118: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:16:43.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:16:42 smithi100 ceph-mon[32599]: pgmap v11118: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:16:45.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:16:44 smithi167 ceph-mon[37140]: pgmap v11119: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:16:45.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:16:44 smithi138 ceph-mon[36954]: pgmap v11119: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:16:45.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:16:44 smithi100 ceph-mon[32599]: pgmap v11119: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:16:45.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:16:45.652+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:16:45.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:16:45.653+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:16:45.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:16:45.653+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:16:45.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:16:45.653+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:16:45.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:16:45.653+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:16:47.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:16:46 smithi167 ceph-mon[37140]: pgmap v11120: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:16:47.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:16:46 smithi138 ceph-mon[36954]: pgmap v11120: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:16:47.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:16:46 smithi100 ceph-mon[32599]: pgmap v11120: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:16:49.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:16:48 smithi167 ceph-mon[37140]: pgmap v11121: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:16:49.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:16:48 smithi138 ceph-mon[36954]: pgmap v11121: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:16:49.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:16:48 smithi100 ceph-mon[32599]: pgmap v11121: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:16:51.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:16:50 smithi167 ceph-mon[37140]: pgmap v11122: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:16:51.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:16:50 smithi138 ceph-mon[36954]: pgmap v11122: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:16:51.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:16:50 smithi100 ceph-mon[32599]: pgmap v11122: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:16:53.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:16:52 smithi167 ceph-mon[37140]: pgmap v11123: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:16:53.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:16:52 smithi138 ceph-mon[36954]: pgmap v11123: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:16:53.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:16:52 smithi100 ceph-mon[32599]: pgmap v11123: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:16:55.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:16:54 smithi167 ceph-mon[37140]: pgmap v11124: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:16:55.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:16:54 smithi138 ceph-mon[36954]: pgmap v11124: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:16:55.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:16:54 smithi100 ceph-mon[32599]: pgmap v11124: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:16:57.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:16:56 smithi167 ceph-mon[37140]: pgmap v11125: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:16:57.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:16:56 smithi138 ceph-mon[36954]: pgmap v11125: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:16:57.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:16:56 smithi100 ceph-mon[32599]: pgmap v11125: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:16:59.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:16:58 smithi138 ceph-mon[36954]: pgmap v11126: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:16:59.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:16:58 smithi100 ceph-mon[32599]: pgmap v11126: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:16:59.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:16:58 smithi167 ceph-mon[37140]: pgmap v11126: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:17:00.145 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:16:59 smithi100 ceph-mon[32599]: pgmap v11127: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:17:00.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:16:59 smithi138 ceph-mon[36954]: pgmap v11127: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:17:00.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:16:59 smithi167 ceph-mon[37140]: pgmap v11127: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:17:00.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:17:00.662+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:17:00.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:17:00.663+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:17:00.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:17:00.663+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:17:00.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:17:00.663+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:17:00.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:17:00.663+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:17:02.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:17:02 smithi100 ceph-mon[32599]: pgmap v11128: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:17:02.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:17:02 smithi167 ceph-mon[37140]: pgmap v11128: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:17:02.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:17:02 smithi138 ceph-mon[36954]: pgmap v11128: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:17:04.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:17:04 smithi100 ceph-mon[32599]: pgmap v11129: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:17:04.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:17:04 smithi167 ceph-mon[37140]: pgmap v11129: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:17:04.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:17:04 smithi138 ceph-mon[36954]: pgmap v11129: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 22 KiB/s wr, 8 op/s 2024-07-26T19:17:06.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:17:06 smithi100 ceph-mon[32599]: pgmap v11130: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:17:06.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:17:06 smithi167 ceph-mon[37140]: pgmap v11130: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:17:06.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:17:06 smithi138 ceph-mon[36954]: pgmap v11130: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:17:08.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:17:08 smithi100 ceph-mon[32599]: pgmap v11131: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:17:08.495 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:17:08 smithi100 ceph-mon[32599]: osdmap e742: 12 total, 12 up, 12 in 2024-07-26T19:17:08.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:17:08 smithi167 ceph-mon[37140]: pgmap v11131: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:17:08.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:17:08 smithi167 ceph-mon[37140]: osdmap e742: 12 total, 12 up, 12 in 2024-07-26T19:17:08.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:17:08 smithi138 ceph-mon[36954]: pgmap v11131: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-07-26T19:17:08.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:17:08 smithi138 ceph-mon[36954]: osdmap e742: 12 total, 12 up, 12 in 2024-07-26T19:17:10.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:17:10 smithi167 ceph-mon[37140]: pgmap v11133: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:17:10.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:17:10 smithi138 ceph-mon[36954]: pgmap v11133: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:17:10.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:17:10 smithi100 ceph-mon[32599]: pgmap v11133: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:17:12.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:17:12 smithi167 ceph-mon[37140]: pgmap v11134: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:17:12.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:17:12 smithi138 ceph-mon[36954]: pgmap v11134: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:17:12.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:17:12 smithi100 ceph-mon[32599]: pgmap v11134: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-07-26T19:17:14.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:17:14 smithi167 ceph-mon[37140]: pgmap v11135: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:17:14.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:17:14 smithi167 ceph-mon[37140]: osdmap e743: 12 total, 12 up, 12 in 2024-07-26T19:17:14.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:17:14 smithi138 ceph-mon[36954]: pgmap v11135: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:17:14.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:17:14 smithi138 ceph-mon[36954]: osdmap e743: 12 total, 12 up, 12 in 2024-07-26T19:17:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:17:14 smithi100 ceph-mon[32599]: pgmap v11135: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-07-26T19:17:14.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:17:14 smithi100 ceph-mon[32599]: osdmap e743: 12 total, 12 up, 12 in 2024-07-26T19:17:15.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:17:15.678+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:17:15.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:17:15.679+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:17:15.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:17:15.679+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:17:15.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:17:15.679+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:17:15.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:17:15.679+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:17:16.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:17:16 smithi167 ceph-mon[37140]: pgmap v11137: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T19:17:16.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:17:16 smithi138 ceph-mon[36954]: pgmap v11137: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T19:17:16.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:17:16 smithi100 ceph-mon[32599]: pgmap v11137: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-07-26T19:17:17.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:17:17 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:17:17.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:17:17 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:17:17.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:17:17 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-26T19:17:18.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:17:18 smithi167 ceph-mon[37140]: pgmap v11138: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T19:17:18.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:17:18 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:17:18.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:17:18 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:17:18.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:17:18 smithi167 ceph-mon[37140]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:17:18.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:17:18 smithi138 ceph-mon[36954]: pgmap v11138: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T19:17:18.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:17:18 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:17:18.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:17:18 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:17:18.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:17:18 smithi138 ceph-mon[36954]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:17:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:17:18 smithi100 ceph-mon[32599]: pgmap v11138: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-07-26T19:17:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:17:18 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-26T19:17:18.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:17:18 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-26T19:17:18.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:17:18 smithi100 ceph-mon[32599]: from='mgr.14214 172.21.15.100:0/604029468' entity='mgr.x' 2024-07-26T19:17:20.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:17:20 smithi167 ceph-mon[37140]: pgmap v11139: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:17:20.690 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:17:20 smithi138 ceph-mon[36954]: pgmap v11139: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:17:20.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:17:20 smithi100 ceph-mon[32599]: pgmap v11139: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:17:22.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:17:22 smithi167 ceph-mon[37140]: pgmap v11140: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:17:22.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:17:22 smithi138 ceph-mon[36954]: pgmap v11140: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:17:22.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:17:22 smithi100 ceph-mon[32599]: pgmap v11140: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:17:24.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:17:24 smithi167 ceph-mon[37140]: pgmap v11141: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:17:24.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:17:24 smithi138 ceph-mon[36954]: pgmap v11141: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:17:24.745 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:17:24 smithi100 ceph-mon[32599]: pgmap v11141: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-07-26T19:17:26.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:17:26 smithi138 ceph-mon[36954]: pgmap v11142: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:17:26.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:17:26 smithi100 ceph-mon[32599]: pgmap v11142: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:17:26.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:17:26 smithi167 ceph-mon[37140]: pgmap v11142: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-07-26T19:17:28.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:17:28 smithi138 ceph-mon[36954]: pgmap v11143: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:17:28.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:17:28 smithi100 ceph-mon[32599]: pgmap v11143: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:17:28.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:17:28 smithi167 ceph-mon[37140]: pgmap v11143: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:17:30.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:17:30 smithi100 ceph-mon[32599]: pgmap v11144: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:17:30.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:17:30 smithi138 ceph-mon[36954]: pgmap v11144: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:17:30.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:17:30.694+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-26T19:17:30.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:17:30.695+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-07-26T19:17:30.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:17:30.695+0000 7fe59ffff640 1 -- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 msgr2=0x55c0ecc6b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-07-26T19:17:30.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:17:30.695+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-07-26T19:17:30.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-07-26T19:17:30.695+0000 7fe59ffff640 1 --2- 172.21.15.100:0/972906438 >> [v2:172.21.15.167:6838/398150514,v1:172.21.15.167:6839/398150514] conn(0x55c0ec8669f0 0x55c0ecc6b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-07-26T19:17:30.775 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:17:30 smithi167 ceph-mon[37140]: pgmap v11144: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:17:32.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:17:32 smithi138 ceph-mon[36954]: pgmap v11145: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:17:32.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:17:32 smithi100 ceph-mon[32599]: pgmap v11145: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:17:32.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:17:32 smithi167 ceph-mon[37140]: pgmap v11145: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-07-26T19:17:34.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:17:34 smithi138 ceph-mon[36954]: pgmap v11146: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:17:34.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:17:34 smithi100 ceph-mon[32599]: pgmap v11146: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:17:34.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:17:34 smithi167 ceph-mon[37140]: pgmap v11146: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:17:36.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:17:36 smithi138 ceph-mon[36954]: pgmap v11147: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:17:36.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:17:36 smithi100 ceph-mon[32599]: pgmap v11147: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:17:36.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:17:36 smithi167 ceph-mon[37140]: pgmap v11147: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:17:38.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:17:38 smithi138 ceph-mon[36954]: pgmap v11148: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:17:38.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:17:38 smithi100 ceph-mon[32599]: pgmap v11148: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:17:38.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:17:38 smithi167 ceph-mon[37140]: pgmap v11148: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-07-26T19:17:40.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Jul 26 19:17:40 smithi138 ceph-mon[36954]: pgmap v11149: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:17:40.744 INFO:journalctl@ceph.mon.a.smithi100.stdout:Jul 26 19:17:40 smithi100 ceph-mon[32599]: pgmap v11149: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:17:40.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Jul 26 19:17:40 smithi167 ceph-mon[37140]: pgmap v11149: 97 pgs: 97 active+clean; 235 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-07-26T19:17:41.426 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2024-07-26T19:17:41.426 DEBUG:teuthology.task.console_log:Killing console logger for smithi100 2024-07-26T19:17:41.427 DEBUG:teuthology.task.console_log:Killing console logger for smithi138 2024-07-26T19:17:41.428 DEBUG:teuthology.task.console_log:Killing console logger for smithi167 2024-07-26T19:17:41.428 DEBUG:teuthology.exit:Finished running handlers